Born in Washington, D.C., while his father, Henry Clay, was serving in the United States Congress, James Brown Clay was named for the husband of his maternal aunt, James Brown. His brothers were Henry Clay, Jr. and John Morrison Clay. Clay attended a boys' school associated with Kenyon College, Gambier, Ohio (founded by family friend Bishop Philander Chase). Later, Clay attended Transylvania University in Lexington, Kentucky.

He worked at a countinghouse in Boston from 1832 to 1834 before studying law and being admitted to the bar. He practiced law with his father in Lexington, Kentucky.

Clay served as chargé d'affaires to Portugal from August 1, 1849, to July 19, 1850. He farmed in Missouri in 1851 and 1852 before returning to Lexington. Clay had been a lifelong member of the Whig Party. But when the Whig Party disintegrated following Henry Clay's death, and due to the controversy surrounding the Kansas–Nebraska Act, Clay joined the Democratic Party. He was elected to the Thirty-fifth Congress (March 4, 1857 – March 3, 1859). Clay did not run for renomion in 1858 and declined an appointment by President James Buchanan to a mission to Germany. His father having died, Clay returned to Lexington and farmed using enslaved labor, and his household also included three male boarders.^[1] In the 1860 census he owned a dozen slaves in Fayette County (surrounding Lexington).^[2]

Clay was a member of the Peace Conference of 1861 held in Washington, D.C., an attempt to prevent the impending American Civil War. During the Civil War Clay supported the Confederacy and was commissioned to raise a regiment. Ill-health from tuberculosis prevented Clay from doing so.

In 1843, Clay married Susan Maria Jacob, the daughter of Louisville's first millionaire and sister of its later mayor, Charles Donald Jacob. The couple eventually had ten children.

Clay died in Montreal, Quebec, Canada, where he had gone for his health. He is interred at his family plot in Lexington Cemetery.

• Henry Clay
• Ashland (Henry Clay home)
• Clay family
• Henry Clay, Jr.
• Susan Clay Sawitzky
• Thomas Clay McDowell

• This article incorporates facts obtained from the public domain Biographical Directory of the United States Congress.
• On Clay's service as American chargé d'affaires to Portugal, see Sara B. Bearss, "Henry Clay and the American Claims against Portugal, 1850," Journal of the Early Republic 7 (Summer 1987): 167–80.

• James Brown Clay's official Congressional biography
• The Filson Historical Society
• Clay Family Papers at the Library of Congress
• Ashland, the Henry Clay Estate
• Allen, William B. (1872). A History of Kentucky: Embracing Gleanings, Reminiscences, Antiquities, Natural Curiosities, Statistics, and Biographical Sketches of Pioneers, Soldiers, Jurists, Lawyers, Statesmen, Divines, Mechanics, Farmers, Merchants, and Other Leading Men, of All Occupations and Pursuits. Bradley & Gilbert. pp. 270–271. Retrieved 2008-11-10.