Stumm was born in the Free City of Frankfurt, but was brought to his Australia by his parents at the age of 2 and raised in Toowoomba, Queensland, where he was educated at public schools. He moved to Gympie at the age of 15 and lived there for the rest of his life, with the exception of a few years at Maryborough. He worked as a Hansard reporter and worked as a journalist for The Gympie Times before purchasing the newspaper in partnership with A. G. Ramsey in 1880. Stumm subsequently took on the newspaper's editorship.^[1][2][3] He was also a member of the Ambulance Brigade Committee, Fire Brigade Board and Gympie Turf Club Committee and the School of Arts and Technical College Committee.^[1] Stumm used his newspaper to campaign against the sitting member for Gympie, Andrew Fisher (who later became Labor's second Prime Minister of Australia), accusing Fisher of being a dangerous revolutionary and an anti-Catholic.^[4][5]

Stumm ran against Fisher for his Gympie seat as in the Legislative Assembly of Queensland at the 1896 colonial election, contesting as a Ministerialist, and won. However, he retired from parliament in 1899 after only one term, citing a need to attend to his business interests and his frustration with "the growing tendency to make the Legislative Assembly a mere House of Talk".^[6][7][8]

Stumm returned to his business interests after his retirement from state politics. He was involved in the formation of the Wide Bay Dairy Co-Operative Ltd in 1906 and served on its board of directors, later serving several years as chairman.^[9][1][10] He also had significant interests in mining investments. He was an unsuccessful candidate for Wide Bay at the 1910 federal election.^[11][12]

In 1913, he was elected to the Australian House of Representatives as the Commonwealth Liberal Party member for the new seat of Lilley and was re-elected in 1914.^[1] Stumm encountered substantial prejudice during World War I as a consequence of his German birth, and he spoke out publicly in 1917 about the "humiliating treatment" he had received.^[13][14] He subsequently retired from parliament at the 1917 election, having had to be talked out of resigning and causing a by-election earlier that year.^[15][16]

After his retirement he again returned to The Gympie Times, which he had continued to own throughout, though he had ceded the editorship to his brother, A. L. Stumm.^[1] Stumm died in 1921 in the Gympie Hospital after suffering a brain hemorrhage in The Gympie Times' office the previous day. He was buried in Gympie Cemetery.^[1][17]

He married Margaret Pride in 1878, and they had five sons and four daughters. His daughter Annie married future general and senator William Glasgow in 1904.^[18][19][5]