Wheeler started playing in village and mid week football in the Evesham area for teams such as Cleeve Prior Amateurs and Evesham Early Closers, he also played in the Evesham Townreserve team before returning to mid week football. While at Evesham Early Closers he had a trial with Coventry City. He joined Cheltenham Town in the summer of 1937 and played for them in the Southern League before joining First Division Birmingham as a professional in 1938. He was signed as understudy and potential successor to England goalkeeper Harry Hibbs, but played only five first-team matches – as the youngest goalkeeper in the First Division – before the Second World War interrupted his career. During the war he served in the 6th Armoured Division in North Africa and Italy, but when he returned to his club, future England internationalGil Merrick had established himself as first-choice goalkeeper. Wheeler played six matches in Birmingham's Second Division championship-winning 1947–48 season when Merrick was injured, but before the next season started he was transferred to Huddersfield Town. Wheeler had to spend some time in the reserves before becoming a first-team regular at Huddersfield. They were relegated in 1952, but promoted back to the First Division in the following season. Wheeler and the five defenders in the side shared the distinction of playing every game of the club's 42-match promotion campaign, a league record. In 1956 he joined Southern League club Kettering Town, where the former England international Tommy Lawton was player-manager. Though the club had struggled the previous season, Wheeler helped them win the Southern League title in 1956–57, playing 47 matches in all competitions. When Lawton took over as manager of Notts County at the end of that season, he asked Wheeler to come with him as coach and trainer.
Notts County
Between joining the club in 1957 and retirement enforced by arthritis in the hip in 1983, during which time he filled numerous roles including trainer, coach, caretaker manager and scout, Wheeler never missed a first-team match – 1,152 consecutive games. In recognition of such service, the club awarded him life membership and a permanent seat in the Meadow Lane directors' box, and had announced plans to rename their clubhouse "Wheeler's" in his honour.