Gulliver was born in Bishop Auckland, County Durham. He was a trainee at Middlesbrough where he was a key member of their reserve side but never made a first team appearance. He made his league debut on 30 November 2002 while on loan at Blackpool, in the starting eleven for the Division Two game against Notts County which finished 1–1. He also spent a month at Carlisle United, and finished the season at A.F.C. Bournemouth, where he helped the Cherries gain promotion from Division Three via the playoffs. The following season he had two more loan spells in Division Three, with Bury and Scunthorpe United, before he was released by Boro at the end of the season. He signed for Rushden & Diamonds where he was a first-team regular in central defence in his two seasons with the club. Indeed, he captained the side in his second season, when Rushden were ultimately relegated from the Football League. In July 2006 he apparently signed for Hereford United on a free transfer and appeared in the squad photo. A few days later he returned to Rushden, pending clarification of his status: if a player aged under 24 rejects the offer of a new deal, the club he leaves is entitled to compensation. He returned to Hereford after an agreement was reached between the two clubs. He was mostly named on the bench for Hereford's opening matches of the season, but after a change in formation to three centre-backs he was given a run in the team. He was named Hereford's jointPlayer of the Month for December, but for the last three months of the season he was usually confined to the bench. He rejoined Rushden at the end of the 2007–08 season. In October 2008 he was one of eight players transfer-listed following a number of poor team performances. After a move to his local club, Corby Town of the Southern League Premier Division, broke down, Gulliver joined league rivals Oxford City on a short term deal. He signed for Corby at the end of 2008, and his arrival in a team which had conceded 14 goals in their last five league games prompted a run of 530 minutes without conceding a goal. Appointed captain, Gulliver went on to help the club to promotion to the Conference North in his first season.
