Kavanagh's first clubs were St Joseph's Boys and Ballybrack Boys, where his talent was soon spotted by scouts. He signed for League of Ireland club Bohemians in 2004 but his opportunities at the club were restricted to appearances in the Gypsies' Under 21 side. Following his brief stint at Bohemians, Kavanagh signed for Wayside Celtic and he became an instant hit, helping them win the FAI Intermediate Cup in his first season. A year later, they reached the top of the Leinster Senior League table and in the last round of games where there was a play-off to decide the champions. His side clinched the Leinster Senior League title, winning 5–0 and he scored twice and was credited with assists for the other three goals. During his time at Wayside Celtic, he played a Republic of Ireland under-20 amateur international against Spain and helped the Irish win 3–0.
Following his performances at Wayside Celtic offers flooded in and the first offer came from UCD, not far from home and he signed in June 2006 on amateur terms. He scored five minutes into his League of Ireland debut as a substitute against Waterford United on 26 June 2006. His season was interrupted as he had already booked holidays before signing for UCD; however, he got a run in the side towards the end of the season and scored three goals in UCD's last three games to help the Students to sixth place in the league. The highlight was the winner at Dalymount Park against Bohemians on 27 October 2006. A week later he was sprung from the bench and scored the winner against Sligo as UCD finished mid-table.
In November 2016, Kavanagh went on trial with English Championship side Birmingham City. He made the move permanent on 31 January 2017, but was released at the end of the season without making a single senior appearance.
Kavanagh returned to Ireland in July 2007 and signed for Bray Wanderers. He instantly became a regular starter for Bray and made 71 appearances for the Seagulls in 2 years. His performances for Bray attracted attention from high-profile clubs including Derry City, Shamrock Rovers and Bohemians. While at Bray he earned a Republic of Ireland U23 cap in November 2007. He was named Player of the Month in the League of Ireland in April 2008.
Shamrock Rovers
Kavanagh signed for Shamrock Rovers on 27 July2009. He made his Rovers debut against Cork City on 11 August 2009, and made his home debut at Tallaght Stadium on 15 August 2009 against UCD in the FAI Cup, scoring his first goal for the club. In 2010, Kavanagh won a League of Ireland Premier Division winners medal helping Shamrock Rovers to their first league title in 16 years. Also during the 2010 season Kavanagh appeared twice in Rovers' UEFA Europa League ties against Italian giants Juventus.
Following his departure from Shamrock Rovers at the end of 2011 season, Kavanagh signed for newly promoted Shelbourne on 20 January 2012.
Derry City
Kavanagh spent the 2013 season at Derry City and helped the Candystripes secure a place in Europe with a 4th-place finish.
Bohemians
On 24 November 2013 Kavanagh confirmed via his Twitter account that he had signed with Bohemians for the 2014 season. On 25 October 2017 Kavanagh Confirmed via his Twitter account that he has re-sign with Bohemians for the 2018 season for his 5th year with the Dublin side.
