Born into a family of musicians, Freeman grew up in Needham, Massachusetts and attended Milton Academy. His paternal grandfather played trumpet and cornet in Sousa's Band. His father was a double bass player in the Boston Symphony Orchestra, ultimately principal bass. In his youth he studied the oboe with Fernand Gillet in addition to studying the piano with Gregory Tucker.^[2] He went on concurrently to earn a Bachelor of Arts degree in music with highest honors from Harvard and a diploma in piano performance from the Longy School of Music in 1957. He also studied privately with Artur Balsam and Rudolf Serkin during the summers of 1955 and 1956. In 1957–58 he held one of Harvard's Sheldon Travelling Fellowships. He went on to pursue graduate studies at Princeton where he was awarded both a MFA and PhD in musicology. A Fulbright Scholarship enabled him to pursue further studies in Vienna in 1960–1962. He was also awarded a Martha Baird Rockefeller Foundation Award in 1962 and later an honorary doctorate from Hamilton College. In 1984 he was awarded Rochester, New York's Civic Medal, in connection with his work on downtown development.

In 1963 Freeman joined the music faculty at Princeton, leaving in 1968 to join the music faculty of the Massachusetts Institute of Technology. In 1972 he was named director of the Eastman School of Music (University of Rochester), a position he held for 24 years. From the fall of 1996 through the spring of 1999 he served as president of the New England Conservatory, then as dean of the College of Fine Arts at The University of Texas at Austin till 2006. In 2015 he retired from the Susan Menefee Ragan Regents Professorship of Fine Arts at UT Austin, where he taught courses on the history and future of music.

A Steinway artist, Freeman performed in concerts and recitals throughout North America and Europe. He also made several recordings, mainly with colleagues from Eastman and the University of Texas. As a musicologist, his publications focused on 18th-century music history and on the history and future of musical education.^[3] His book, "The Crisis of Classical Music in America; Lessons from a Life in the Education of Musicians," was published in August 2014. He was awarded an honorary degree in April 2015 by the Eastman School of Music, which named the atrium of its Sibley Music Library in his honor. He was an emeritus professor of musicology at the University of Texas at Austin, as well as the senior educational liaison for Music in the Air (MITA), a revolutionary computer-mediated means of learning music, developed by UCLA's Robert Winter and Peter Bogdanoff and published by ArtsInteractive Inc., designed to develop broader audiences for music of all kinds while extending human attention spans.