Born in Liverpool, Merseyside, Orr started his senior career with Newcastle United, and was loaned out to Burnley due to lack of opportunities in the first team side.
Shortly after this loan period in July 2004, he signed for Bristol City. He scored his first goal for his new team against Yeovil Town. Although he was signed as a central midfielder he was converted to right-back in 2005–06 and has played most of his games in that position. On 29 August 2006, Orr was sent off for attempting to head-butt his own teammate, Louis Carey, during a game against Northampton Town. On 1 September 2006, Orr was sentenced to 28 days in jail relating to an incident outside the popular Bristol nightclub 'Romeo Browns' the previous October. Also jailed in relation to the same incident were teammates Steve Brooker for 28 days, and David Partridge for two months. Orr and Brooker were released on 14 September 2006, serving only half of their original sentence. He scored against Manchester City in the League Cup second round. He was named in the PFA's Championship team of the season 2007–08. According to media sources, he broke his cheekbone in the play-off final against Hull City. On 27 November 2008, Bristol City decided to place Orr on the transfer list, due to rejecting several contract offers. On 1 January 2009, Bristol City announced that Orr had signed a two-and-a-half-year contract at the club.
On 26 July 2010, Orr signed for Queens Park Rangers. He scored his first goal for the club in a 2–0 win over Scunthorpe United on 21 August 2010. He became a regular in Rangers Championship winning season with a number of impressive performances at right back helping Rangers to a high number of clean sheets. Once promoted to the Premier League his first team opportunities at Loftus Road became increasingly limited following the signing of former England right back Luke Young.
His time at QPR was short-lived however, due to a lack of first team games. He made a move in the January 2012 transfer window to Blackburn Rovers on a three-and-a-half-year deal for an undisclosed fee a move he said he badly needed. Orr made his debut for Blackburn when he replaced Jason Lowe in the 36th minute of the game in the 7–1 loss against Arsenal at the Emirates Stadium on Saturday 4 February 2012. He made his full home debut against his former employers QPR at Ewood Park, exactly a week later on 11 February 2012, in a game Rovers went on to win 3–2 and he also completed the full 90 minutes in his right-back slot. On Monday 2 April 2012, Orr started and played the full 90 minutes in the right-back berth against Manchester United at Ewood Park which ended in a 2–0 defeat on the night of his sixth appearance for Rovers in the Barclays Premier League. On Saturday 21 April 2012, he started and played the full 90 minutes in the right-back spot with Jason Lowe playing further forward in a 2–0 victory over Norwich City at Ewood Park, thanks to strikes from Mauro Formica and Junior Hoilett. On Monday 7 May 2012, Orr started the 1–0 defeat against Lancashire rivals Wigan Athletic at Ewood Park, but was replaced on 80 minutes for Scottish international striker David Goodwillie, this loss meant that Rovers were relegated from the top flight.
On 25 January 2014, it was announced that Orr would be placed out on loan to MLS side Toronto FC. The loan is set to run to December 2014 and would see Orr reunited with his former teammate at Blackburn, Ryan Nelsen. Nelsen was the head coach of Toronto FC having been sacked, along with his assistants, on 31 August 2014. Orr made his debut with Toronto in their season opener at Seattle Sounders FC on 15 March, the game ended in a 2–1 away victory with both goals coming from newly acquired Jermain Defoe He scored his first goal for Toronto on 23 May against Sporting Kansas City.
