Born in Huntsville, Texas, his father was a former Walker County sheriff, but died when Elkins was just a boy. He was reared by his mother and attended public schools. He attended Sam Houston Normal School, now known as Sam Houston University. He later attended the University of Texas, receiving his law degree in 1901.^[1][2]

After 1901, Elkins returned home to Huntsville. In 1903, he began to practice law and was elected Walker County Judge. Although serving as County Judge for only a few years in the earlier part of his career (1904-1905), the title "Judge" remained synonymous with Elkins throughout his years practicing law and banking. In 1917, he joined with his friend, William Ashton Vinson, to open up a law practice together in Houston - Vinson and Elkins. In 1924, Elkins ventured into the banking industry, opening the Guaranty Trust Company, with just $110,000 in capital and no deposits, at a cramped location in the Rusk Building. The two concerns were synergistic, as many Vinson and Elkins clients started accounts with the Elkins banks, and many of his banking clients sought counsel with Vinson and Elkins. Corporate clients with Vinson and Elkins included Great Southern Life Insurance, Moody-Seagraves, Pure Oil, and United Gas. Gus Wortham, a Vinson and Elkins client, gained financing from Guaranty Trust Bank, and later Wortham's company grew into the giant American General Insurance.^[2]

The bank eventually grew into Houston's largest — First City National Bank. Politically well connected, Elkins was one of the most powerful business leaders in Houston during the city's most formative years.^[3] Elkins was a fundraiser for the University of Houston, and served as a regent for the university as well.

Elkins was a regular attendee of the so-called Suite 8F Group (named after the room in the Lamar Hotel where gathering were held). This behind-the-scenes socialization amongst leading Texas politicians and businessmen included the likes of Jesse Jones, Gus Wortham, James Abercrombie, George R. Brown, Herman Brown, Lyndon Johnson, William L. Clayton, William P. Hobby, Oscar Holcombe, Hugh Roy Cullen, and John Connally.