Brady was born in Portsmouth, Virginia to Irish immigrant parents Bartholomew and Elizabeth Brady, who had four other children. His parents died in the 1855 Yellow Fever epidemic, which claimed the lives of approximately 10% of the Portsmouth population.

Brady enlisted on March 9, 1861 as a private in Company A, 37th New York Volunteers, "Irish Rifles", and with the Irish Brigade. He was commissioned as Lieutenant then Adjutant, promoted to Captain, Major, Lieutenant and Colonel of the regiment and last named officer honorably mustered out of service on 1865-05-26, claiming to be the "Youngest colonel in the Army of the Potomac."

He fought in all the great battles in which the Irish Brigade was engaged, and commanded "The Color Company" in the Battle of Fredericksburg in 1862. He suffered four notable injuries including being wounded in the head while leading his company in the assault of the Irish Brigade upon Marye’s heights; in the leg in the second day of the battle of Fair Oaks, the morning that General Howard lost his arm; wounded in the mouth at Malvern Hill; and again wounded in the arm at Cold Harbor 1864-06-03, in which a shot also passed through his abdomen. Personally complimented by General Hancock at the battle of Gettysburg on the afternoon that General Zook was mortally wounded. (Brady was with him).

James Brady returned to Virginia after the war, taking home the colors (flag) of the Irish Brigade, as was the tradition. He later donated the flag to Notre Dame and[?] wrote a book on the flag called Blue for the Union, Green for Ireland.

Brady was elected Clerk of the Court in Portsmouth and served from 1865 to 1877. President Hayes appointed Brady Collector of Internal Revenue for the second district of Virginia from 1877 to 1885, and from 1889 to 1900.

He served as delegate to the Republican National Conventions, 1880, 1888, and 1896.

He had a successful run for the Senate in 1875 that was overturned due to a ballot stuffing scandal. But he wrote, "...There is nothing that discourages me... ", and was later elected to the Forty-ninth Congress (March 4, 1885 – March 3, 1887). He was not a candidate for renomination in 1886.

He died on November 30, 1900, in Petersburg, Virginia and was interred in St. Joseph's Cemetery, Petersburg Virginia.

Brady was elected to the U.S. House of Representatives with 40.48% of the vote, defeating Democrat George E. Rives and Independent Republican Joseph P. Evans.

• United States Congress. "James Dennis Brady (id: B000752)". Biographical Directory of the United States Congress. Retrieved on 2008-11-05

  This article incorporates public domain material from the Biographical Directory of the United States Congress