Foster was born in Walden, Vermont, the son of Ephraim and Emily Foster. Of his early life little is known except that he was a teacher in his hometown school district.
He was commissioned captain of Company G, 4th Vermont Infantry, on September 21, 1861, promoted major on July 18, 1862, lieutenant colonelNovember 5, 1862 and colonel on February 3, 1864, replacing Charles B. Stoughton, who had resigned. According to Vermont's military historian, George Benedict, Foster was "of stalwart proportions, and handsome face and figure, he was one of the finest looking officers in the brigade. He was a favorite with his men, distinguished himself as emphatically a fighting colonel." He was present in every action of the Vermont Brigade until he was severely wounded in the thigh on the first day of the Battle of the Wilderness, on May 5, 1864. While home recuperating from his wounds, he married Sarah Salome Hubbell, of Burlington, on July 5, 1864, in Wolcott, Vermont. At the Battle at Lee's Mills, April 16, 1862, he led a company of skirmishers early in the engagement. On December 13, 1862, at the Battle of Fredericksburg, Lieutenant Colonel Foster led the 4th Vermont Infantry on the skirmish line in front of Howe's division of the VI Corps. At the battle of Winchester, Colonel Foster had command of the Vermont Brigade. He was nominated to receive the brevet rank of brigadier general by President Abraham Lincoln on January 1, 1865, "for gallant and meritorious service before Richmond and in the Shenandoah Valley, Virginia," to date from August 1, 1864. The Senate approved the appointment on February 14, 1865. He mustered out of Federal service with his regiment on July 13, 1865.
Postwar career
President Ulysses S. Grant nominated Foster to be U.S. Marshal for the district of Vermont, on January 10, 1870, succeeding Hugh H. Henry, who had died in office. He was appointed January 24, and served in that position until his death. His tenure was highlighted "by his bold arrest of the Fenian commander, General John O'Neill, in the midst of his army, during the Fenian invasion of Canada, in 1870." He made the arrest immediately after O'Neill's defeat at the Battle of Trout River. He died in Burlington, Vermont, and is buried in Burlington's Lakeview Cemetery.
