Stuart embarked upon a military career in 1768, when he enlisted as an ensign in the 37th Regiment of Foot. He purchased a lieutenancy in the 7th Regiment of Foot in 1770 and a captaincy in the 37th Foot in 1775. Late that year, he became a major commanding a battalion of the regiment, and saw service in the American War of Independence. In 1777, he was commissioned as a lieutenant-colonel of the 26th Regiment of Foot, which he commanded until 1779. On a visit home to England, he married Anne Louisa Bertie, daughter of Lord Vere Bertie, on 19 April 1778. He returned briefly to America, before coming back to London as a liaison to the ministry. A harsh critic of the Army's conduct, he was, however, highly favored by Sir Henry Clinton, with whom he corresponded regularly. His two sons were born after his return from America:
Captain John James Stuart, died aboard his command, the frigate. John James's son Charles Stuart was British Army general and MP.
He was promoted to colonel in 1782, but his criticisms and the disfavor of George III towards his father prevented further military commands. He had been elected MP for Bossiney in 1776, succeeding his elder brother Lord Mount Stuart, who had been created Baron Cardiff. Stuart continued an MP for the remainder of his life, except the years 1794–1796, but showed little interest in politics. In 1792, on the death of his father, he inherited the estate of Highcliffe House in Hampshire. With the opening of hostilities against France by the First Coalition, he returned to active service. On 23 May 1794, he took command of the army in Corsica, and supervised the taking of Calvi. Col. John Moore was at the time his adjutant general. Stuart was promoted to lieutenant-general for this action, and on 24 October 1794, was made colonel of the 68th Regiment of Foot. However, his pride and violent temper led him to quarrel with Lord Hood, commanding the Mediterranean Fleet, and with the civilian viceroy of Corsica, Sir Gilbert Elliot, Bt. His partiality for Pasquale Paoli against Elliot, and other conflicts, led Stuart to resign in February 1795. On 25 March 1795, he left the colonelcy of the 68th for that of the 26th Regiment of Foot, which he held for the remainder of his life.
He took command of a force sent to Portugal in January 1797 to defend Lisbon, and was notably successful in instilling discipline and spirit into the force, which was partly foreign in composition.
In 1798, he was sent to attack Menorca with 3,000 men, an appointment heartily approved by Lord St Vincent, who praised Stuart as an excellent general and inspiring leader of troops. Though unequipped with siege artillery, he successfully dissimulated and bluffed the Spaniards into surrendering the island without loss of life, an exploit for which he was made a Knight of the Bath. From 15 November 1798 until 1800, he served as the British governor of the island. In March 1799, he responded to an appeal by Admiral Nelson, and brought the 30th and 89th Regiments under Col. Blayney to Palermo, from whence they were dispatched to secure Messina against French invasion. An able general and administrator, Stuart's quarrelsome disposition and tendency toward insubordination blighted an otherwise promising military career.