King was born on St. Simons Island, off the coast of Georgia on April 20, 1842, to Georgia Congressman Thomas Butler King. He was the nephew of Pennsylvania Congressman Henry King.

King attended the Russell School, New Haven, Connecticut, Bartlett's College Hill School, Poughkeepsie, New York, the Military Institute of Georgia, and the University of Virginia at Charlottesville. Enlisted in the Confederate States Army and served in the Army of Virginia throughout the Civil War, attaining the rank of colonel of Artillery. He moved to Louisiana and engaged in planting. He studied law. He was admitted to the bar in 1872 and commenced practice in Vidalia, Louisiana. He was appointed brigadier general of State troops.

King was elected inspector of levees and president of the board of school directors of his district and also a trustee of the University of the South.

King was elected as a Democrat to the Forty-sixth and to the three succeeding Congresses (March 4, 1879 – March 3, 1887). He served as chairman of the Committee on Levees and Improvements of the Mississippi River (Forty-eighth and Forty-ninth Congresses). He was an unsuccessful candidate for renomination in 1886. He engaged in mining operations, with residence in Washington, D.C. Assistant Register of the United States Treasury from May 19, 1914, until his death in Washington, D.C., on May 8, 1915. He is interred in Arlington National Cemetery.

• United States Congress. "J. Floyd King (id: K000206)". Biographical Directory of the United States Congress.