Scharf grew up in Crown Heights and attended Winthrop Junior High School. A mathematics major in college, he graduated with honors from the City College of New York in 1962.^[1]

Scharf was a friend of guitarist Jay Berliner, who influenced his career. During the early 1960s, he was the lead guitarist for folk-singer Leon Bibb. He also worked with arranger Walter Raim and folk-singer Judy Collins as well as bassist Bill Lee (father of Spike Lee).

For several years, he partnered with Martin Gersten, chief engineer of WNCN, in a recording studio at 18 Jones Street in Greenwich Village. They shared this space with folk music broadcaster Skip Weshner.

Scharf was a prolific studio musician in New York City during the 1960s, playing guitar with Chad Mitchell, Janis Ian, Al Kooper, and Carly Simon. He also had a producing partnership with Bob Dorough for many years; together, they produced albums by Spanky and Our Gang. Scharf was the composer of Spanky and Our Gang's hit "Like to Get to Know You."

In 1980, he moved to Hamilton Township, Monroe County, Pennsylvania, where he continued his recording business.^[2]

As sideman edit

With Charles Earland

• Charles III (Prestige, 1973)

With J. J. Johnson and Kai Winding

• Betwixt & Between (A&M/CTI, 1969)

With Al Kooper

• You Never Know Who Your Friends Are (Columbia, 1969)
• Easy Does It (Columbia, 1970)
• Naked Songs (Columbia, 1973)

With Hubert Laws

• The Rite of Spring (CTI, 1971)

With Pearls Before Swine

• Beautiful Lies You Could Live In (Reprise, 1971)

With Phil Woods

• Greek Cooking (Impulse!, 1967)

As producer edit

With Spanky and Our Gang

• Like to Get to Know You (Mercury, 1968)
• "Anything You Choose b/w Without Rhyme or Reason" (Mercury, 1969) wrote 6 songs for this album, including the politically-significant 'Give a Damn', which was adopted as a theme song by the New York Urban Coalition, and by New York Mayor John Lindsay during his 1969 re-election campaign.