The youngest son of the former Elizabeth Hill and her husband John Carter was born at the Hill family's historic Shirley Plantation in Charles City County, which his parents received as a wedding present. His eldest brother, John Hill Carter, was alive in 1728 when their middle (and ultimately wealthiest) brother Charles Hill Carter was born and their enormously wealthy grandfather Robert Carter I died, and may have still been alive in 1777, but has left little historical footprint. Both Charles and Edward were underage when their father died in 1742, so they became wards of their uncle Landon Carter, who raised them with his children at Sabine Hall.^[4] Complicating matters, his brother Charles Hill Carter would name one of his twin sons Edward, and that son also served in the Virginia House of Delegates, but represented Prince William County, Virginia, where his estate was known as "Cloverdale". After Landon Carter's death, Charles and Edward became wards of Richard Byrd III, the husband of their aunt Elizabeth and whose plantation Berkeley plantation was much nearer Shirley plantation, which young Charles Hill Carter inherited upon reaching legal age.

When Edward Carter reached legal age and came into his own inheritance, he married his distant cousin Sarah Champe (1733–1814), daughter of Burgess, John Champe and his wife Jane of King George County, Virginia. Sarah Carter survived her husband despite bearing thirteen children, of whom eleven reached adulthood and had children.^[5] Sarah Champe definitely preferred living in Fredericksburg, Virginia, the nearest town to where she had been raised, rather than still-rural Albemarle County, even if it was nearer her husband's vast inherited acreage. Thus, for years they wintered in Fredericksburg, and spent summers in Albemarle County. Their surviving sons included Charles Carter (1751–1826), Edward Carter Jr. (1756– ), John Champe Carter (1758–1826), Robert Carter (1778–1809), William Champe Carter (d. 1834), Whitaker Carter (d. 1821) and Hill Carter. Their daughters included Sarah Carter (who married her cousin George Carter, son of Edward's former playmate and burgess Robert Wormeley Carter of Sabine Hall; and after his death married Dr. John Brown Cutting and had a son), Elizabeth Carter (1762–1818; who married William Stanard of Roxbury Plantation in Spotsylvania County and who cared for her parents in their final years), Jane Carter (1765–1846; who married first British Major Samuel Kellet Bradford, then Major Jean Artur Verminet of Alexandria, Virginia), Mary Champe Carter (1778–1846; who married future judge Francis Taliaferro Brooke who owned plantations near Fredericksburg) and Anne (Nancy) Williams Carter (who married future Georgia Governor George M. Troupe). Edward Carter would ultimately disinherit his son John Champe Carter (for reasons which caused varied later speculation, ranging from mental distress caused by his Revolutionary War service or a gambling problem which caused his father-in-law to establish a trust for his daughter and grandchildren), and enlisted his neighbor Thomas Jefferson for legal help in breaking entail so as to split his inherited and accumulated acreage among his other sons. Charles Carter owned the plantations "Eastern View" in Culpeper County and "Deerwood" in Pittsylvania County, and married Betty Lewis (1765–1830), daughter of their Fredericksburg neighbors Col. Fielding Lewis and Betsy Washington Lewis, so they too spent a great deal of time in Fredericksburg. Edward Carter Jr. married three times. Robert Carter received the Redlands plantation in Albemarle County and married neighbor Mary Eliza Coles (1776–1856). William Carter inherited "Viewmont" in Albemarle County from his parents and named his other plantation "Farley" to honor his wife's family. Their youngest son Hill Carter received "Mine Hill" plantation in Amherst County, which he represented in the Virginia House of Delegates.^[6]

John Carter had patented (claimed) 9350 acres of land immediately south of Monticello plantation in Albemarle County in 1730. John Carter also acquired 10,000 acres in Amherst County. Both parcels would become Edward's inheritance, and he acquired additional adjacent land in Albemarle County, on which he built a house known as "Blenheim". Carter farmed it using indentured but mostly using enslaved labor. He also established a forge, as discussed below. Carter was by far the largest taxpayer and slaveholder in Albemarle County in 1782, with 237 slaves, 198 cattle, 62 horses and mules, and 4 carriage wheels.^[7] According to the first Virginia tax census following the American Revolutionary War, in 1787, Edward Carter Esq. owned 112 enslaved people above age 16 in Albemarle County, as well as 115 younger slaves, 60 horses, and 116 cattle.^[8]

Carter first represented Albemarle County in the House of Burgesses, during the 1766–1768 term, when he defeated veteran Henry Fry.^[9][10]

In 1768, Edward Carter joined with local notables Thomas Walker, William Cabell and Alexander Trent to finance John Wilkerson, who founded the Albemarle Iron Works (Wilkinson holding 1/3 of the stock and each of the rest owning 1/6 of the corporation)^[11] In 1776 Carter was among the signers of a petition against subversion of religious freedom (alongside George Gilmer, Philip Mazzei, Jacob Moon and others).^[12] Major General William Phillips, the surrendered British commander, was housed at Blenheim before being exchanged for captured American General Benjamin Lincoln in 1780.^[13] And Carter was among the many Amherst County property owners who in 1782 filed for property impressed or taken for public service in that county during the conflict.^[14]

In his final years Edward Carter moved to near Fredericksburg, where he and his wife lived at a townhouse which would later be destroyed by fire (and start a conflaguration which destroyed nearly half of that city) in 1807. His daughter Elizabeth and her husband lived nearby at his Roxbury plantation in Spotsylvania County. Edward Carter died in 1793, having written a will in 1792, which was duly admitted to probate in Spotsylvania County, despite freeing some enslaved. The Library of Virginia has also acquired some of the Carter family papers.^[15]

Carter's Blenheim manor house disappeared before 1840, and Congressman Andrew Stevenson acquired the underlying land and erected a now-historic house. However, another part of the original Carter property is now cultivated as Blenheim Vineyards, which includes a tasting room.^[16]