Peck was born in Lyme, Connecticut, one of thirteen children of Elijah Peck and Hepzibah Pierson. He was raised on the family farm, and his formal education was limited to attending a country grammar school, but he taught himself by reading the Bible many times over. In 1771 at about age 23, Peck returned from a sea voyage to learn that his parents, three brothers and a sister had died. He became extremely depressed and wrote in his journal that he longed for:

Days and times past when may father and Mother and all my bretherin and Sisters were about me in helth and prosperity but alas! trubel and Sorow hath Sorounded me and I am a poor Disconsolate Cretur. There is no place that Seemes to be home to me.

He kept that journal in a secret drawer in his desk throughout the remainder of his life, and from that time he became quite evangelical about his faith.

He served four years as an enlisted man in the American Revolutionary Army. In 1790 he settled in what was to become the town of Burlington, Otsego County, New York. When Burlington was formed from a part of the Town of Otsego in 1792, Jedediah Peck became Burlington's first Town Supervisor and remained in that job for eight years. He is said to have been elected to the position three times, the latest in 1820, when he would have been 73.

He also worked as a surveyor and millwright, studied law, was appointed as a judge, wrote political tracts, and conducted religious services on request. He is said to have been seen as an awkward figure, with his "drawling, nasal, yankee twang" and his saddle-bags "filled with political papers and scraps" that he distributed to all who would listen.

Peck was a strong anti-Federalist, and in 1798, Judge William Cooper, an ardent Federalist, had him arrested by a United States Marshal under the Alien and Sedition Acts for circulating petitions against those very acts. Peck was taken in irons to be tried in New York City. The spectacle of the martyred war hero being transported in chains only served to help the Republican cause. Peck was soon released without trial.

He was a member of the New York State Legislature for eleven years, in the Assembly (1798–1804) and in the Senate (1804 to 1808). In 1801 he submitted an amendment, forerunner to the Twelfth Amendment, to adopt designations for the votes for president and vice president.

His granddaughter, Anna Peck Sill, was the founder and principal of Rockford Female Seminary.

While in the State Assembly (in 1801, 1803 and 1804), Jedediah Peck sponsored bills to establish common schools in the state, but each resolution was rejected. In 1811, after Peck's retirement from active politics, Governor Daniel D. Tompkins appointed Peck chairman of a five-man commission to study the problem of public school education. In five months the commission reported the fundamental principals of New York's educational system. In 1812, a bill become law and the basic foundation of the state's public school system was established. The law requires:

• that there be a division of towns into school districts;
• that there be trustees in each district to superintend those schools;
• that each town raise taxes as much as it received from the state school fund;
• that the funds be divided among the various towns on a population basis, and then subdivided among the school districts.

In addition to his work in establishing the common school system of New York, he introduced a bill for the abolition of imprisonment for debt which later became a law.

Although nearly seventy years of age at the time, he served in the War of 1812, and took part in the Battle of Queenston Heights. He died at age 74 and is buried at the [1] in the Town of Burlington, New York. A New York State Historical Marker at the site reads:

In memory of Hon. Jedediah Peck, a Revolutionary Patriot, who died Aug 15, 1821, in the 74th year of his age. The annals of the State bear record of his public usefulness and the recollection of his virtues bear testimony of his private work.

• History of Otsego County, New York 1740-1878: Ch. XXX, Town of Burlington, Everts and Fariss, 1878.
• Sullivan, James; Williams, Melvin E.; Conklin, Edwin P.; Fitzpatrick, Benedict, eds. (1927), "Chapter III. Politics in New York State. Federal Period to 1800.", History of New York State, 1523–1927 (PDF), vol. 4, New York City, Chicago: Lewis Historical Publishing Co., pp. 1516–18, hdl:2027/mdp.39015067924855, Wikidata Q114149633
• Lyman H. Butterfield Judge William Cooper (1754-1809): A Sketch of his Character and Accomplishment, October, 1949.
• Lyman H. Butterfield Cooper's Inheritance: The Otsego Country and its Founders, October, 1954.
• Alan Taylor, William Cooper's Town: Power and Persuasion on the Frontier of the Early American Republic, New York: Alfred A. Knopf, 1995.
• Mark Simonson, Farmer clashed with local opinions, The Daily Star, Oneonta, New York, June 5, 2004.
• Births from Lyme town records