Smith was born in the Mormon settlement of Zodiac, Texas. His grandfather was Lyman Wight, one of the original apostles of the Church of Christ established by Joseph Smith, Jr. After Joseph Smith's death, the church divided into several groups, one of which Wight led to Texas. In 1858, Heman's family left Texas, and they eventually settled in Shelby County, Iowa. In 1862, Smith and his family were baptized into the RLDS Church.

Smith was a Seventy and an RLDS missionary in Iowa, Indiana, Illinois, Michigan, Louisiana, Arkansas, Florida, Kentucky, Tennessee, Mississippi, Texas, Indian Territory, and eastern Canada. In 1885, Smith became one of the church's seven presidents of the Seventy, and in 1887 he was selected as a member of the Council of the Twelve; he was officially ordained an apostle by Joseph Smith III in 1888. In 1893, Smith moved to Lamoni, Iowa. During 1897 and 1898, Smith oversaw the RLDS Church mission in England and Wales.

In 1897, Smith became the official historian of the RLDS Church, and in this capacity co-wrote with Joseph Smith III the first four volumes of the RLDS Church's History of the Reorganized Church of Jesus Christ of Latter Day Saints.

In 1886, Smith married Vida Elizabeth Smith, who was the daughter of Alexander Hale Smith and a granddaughter of church founder Joseph Smith, Jr.

• "Heman C. Smith", in J. M. Howell (ed.) (1915). History of Decateur County, Iowa and Its People (Chicago: S. J. Clarke Publishing) 2:45.