On December 13, 2000, he was traded to the Chicago White Sox as the player to be named later in an early trade for Chad Bradford. In July 2001, while playing for White Sox Class AA affiliate in Birmingham, Olivo was suspended for six games and barred from that year's All-Star Futures Game for using a corked bat. Olivo maintained the bat, which had an A's logo on it, had been given to him by a former teammate while playing in the Athletics system. He made his major league debut on September 15, 2002 with the White Sox and hit a home run in his first Major League at-bat, off Andy Pettitte of the New York Yankees. He played 6 games that year, with 4 hits in 19 at-bats. He was the backup catcher for the White Sox for all of 2003 and the first half of 2004. In 166 games with the White Sox, he hit.245 with 14 homers and 58 RBI.
After struggling with the Mariners he was traded to the San Diego Padres in on July 31, 2005 for minor leaguer Daniel Mateo and Miguel Ojeda. He played in 37 games for the Padres and hit.304. Olivo is remembered by Padres broadcaster Mark Grant for an interview in which he started talking about newly acquired pitcher Chan Ho Park, when instead he was asked about the Padres stadium, Petco Park
He signed as a free agent with the Florida Marlins on January 3, 2006. He spent the next two seasons with the Marlins, hitting.263 in 127 games in 2006 and.237 in 122 games in 2007. In 2007, he had the lowest fielding percentage of all NL catchers On September 29, 2007 the second to last game of the season in a game against the New York Mets, Olivo charged across the diamond and attempted to throw a punch at Jose Reyes starting a bench-clearing brawl between the teams. He was ejected from the game for starting the brawl and Reyes stayed in the game. Olivo was not offered a new contract by the Marlins and became a free agent on December 12, 2007.
On December 27, 2007, Olivo signed a one-year contract with the Kansas City Royals. On August 3, 2008, during a game against the White Sox, he was hit by a pitch thrown by D. J. Carrasco. Olivo charged after Carrasco and started another bench-clearing brawl. On November 6, 2009, the Royals declined his option, making him a free agent. In 198 games over two seasons, he hit.251 with 35 homers and 106 RBI.
On January 4, 2010, Olivo signed a one-year deal with the Colorado Rockies. Olivo made headlines in early 2010 for having to leave the Rockies' April 29 game against the Arizona Diamondbacksat the beginning of the eighth inning to pass a kidney stone. Remarkably, he was able to re-enter soon after and finished the game. On October 1, Olivo was hit in the back of the head by the bat of Albert Pujols. He suffered a mild concussion, but returned to the team later in the game. In 112 games with the Rockies, he hit.269 with 14 homers and 58 RBI.
Second stint with the Mariners
On November 4, 2010, Olivo was traded to the Toronto Blue Jays as part of a conditional deal. The Blue Jays declined to exercise his contract option for 2011 but offered him arbitration; he declined, becoming a free agent. Olivo signed a 2-year, $7 million contract with the Seattle Mariners on December 9, 2010. Olivo hit his second grand slam of his career on July 21, 2011, against the Blue Jays in Rogers Centre. In 2011, Olivo had the worst walk-to-strikeouts ratio in the major leagues, with 0.14 walks for every strikeout. On October 24, 2012, the Mariners announced that they would not pick up Olivo's $3 million club option for 2013. Olivo was paid a $750,000 buyout.
On February 1, 2013, the Cincinnati Reds signed Olivo to a minor-league deal with an invitation to Spring training. On March 27, 2013, he asked for, and was granted, his release from the Reds.
Second stint with the Marlins
On March 27, 2013, Olivo signed a one-year deal with the Miami Marlins, just one day after requesting his release from the Reds. On June 14, 2013, in frustration over a lack of playing time, Olivo 'walked out' on the Marlins at the start of their game that evening, demanding that the team release him. Olivo watched the team play the first inning of the game from the clubhouse, then changed into street clothes and left the ballpark. In response, the Marlins declined the request and instead placed him on the 'restricted list', during which time he is not paid and cannot seek employment with another team.
On January 21, 2014, he signed a minor league contract with the Los Angeles Dodgers and was assigned to the Triple-A Albuquerque Isotopes. He was purchased by the Dodgers and added to the 25 man active roster on April 30, 2014. In 8 games with the Dodgers, he hit only.217 before he was optioned back to Triple-A on May 14. On May 20, 2014, Olivo was involved in an altercation with Isotopes teammate Alex Guerrero. During the incident, Olivo bit off a piece of Guerrero's ear, an injury which required cosmetic surgery to repair. Two days later the Dodgers gave him his unconditional release. He had been hitting.368 in Triple-A at the time of his release, with 4 home runs and 20 RBI in 20 games.
