Richardson was born February 26, 1810, in Burlington, New York, son of Caleb and Clarissa Richardson. He attended the local public primary school, and taught school himself until the age of 21. At that time, he took charge of a small school in Cooperstown, New York, where he was assisted by Elizabeth "Betsy" W. Spafford for three years. Ill health forced him to leave the teaching profession. He married Spafford on October 23, 1834, in Cooperstown; they would have one child, Elizabeth. He wandered in the unsettled western territoriesfor a while, recovering his health; then returned to Cooperstown and worked some years as a bookkeeper. On August 30, 1841, Betsy died there. In 1842, the widowed Richardson came to Walworth County, Wisconsin, settling on farmland in Hudson which was annexed by the village of Geneva in 1844. He soon found work at a store owned by the Spafford family, and on February 16, 1843, he married Alma O. Spafford, sister of his first wife, Elizabeth, in Geneva. He was an Episcopalian, and in 1850 was one of the officers elected on the organization of an Episcopalian church in Geneva.
Public office
Soon after coming to Walworth County, Richardson was elected a justice of the peace, a position he would hold for thirteen years. In 1845-46, he was town clerk of Geneva. Richardson was a Democrat, and was elected from the Walworth County Assembly district that included the Towns of Bloomfield, Geneva, and Hudson. He was succeeded in the next session by Samuel D. Hastings, a Freesoiler. He was again elected town clerk in 1850, and became village president of the village of Geneva in 1856, although he left office early. In 1869, he was elected clerk of the village. In 1870, he served as chairman of the county board of supervisors. He was village president from 1870–71, and once more in 1877.
Business activities
Richardon began his private Bank of Geneva in 1848, and continued as its sole proprietor until his death. In 1871, he was part of a group who obtained a charter for the State Line and Union Railway Company, which was soon absorbed into the Chicago and Northwestern system. When he died on January 2, 1892, it was discovered that his bank, which had survived the Panic of 1857, the 1871 loss to burglary of its entire cash holdings, and the Panic of 1873, was "somewhat involved" and partially insolvent. Given his history of safe and sound financial practices, the confusion was laid to the infirmities of old age in his final years.