Klein was born in Hungary in 1886, and received his education in the common schools of that country. In 1903 he moved to the United States, coming directly to Milwaukee. He became a machinist and eventually found a long-term job with the Chicago & Milwaukee Railway. He became active in the trade union movement and the Milwaukee Socialist party.
He served as a Milwaukee County deputy sheriff from 1914 to 1916. He was elected to the Assembly's Second Milwaukee County district (the 2nd Ward of the City of Milwaukee) in 1918 to succeed Republican William A. Campbell, receiving 658 votes to 627 for William Mielahn and 581 for Democrat Carl Heim. He was assigned to the standing committee on labor.