Rollins wrote his two compositions ("Pent-Up House" and "Valse Hot") while a sideman in the Max Roach/Clifford Brown Quintet. It was more common in the 1950s for a sideman recording his own work to record with either the rhythm section or leader; thus it was unusual when Rollins recorded with the same musicians that he played with in the Quintet. Rollins had joined the Quintet five months beforehand, replacing Harold Land, who left New York to care for his sick wife in California.

Rollins was working as a janitor in Chicago, spending most of his time practicing and rethinking his life (a shorter sabbatical compared to ones he would take). The Quintet was in Chicago in November 1955 and was playing at the Bee Hive Club in Hyde Park. After sitting in with the Brown/Roach Quintet at the Bee Hive, Rollins was added as tenor saxophonist.

After returning to New York, Rollins recorded Plus 4 and used the Quintet as part of an arrangement between his record label, Prestige, and EmArcy, the jazz subsidiary of Mercury, which had the Brown/Roach Quintet under contract. To use Rollins on the Brown-Roach Quintet's EmArcy albums, Prestige owner Bob Weinstock insisted that the group record one album for Prestige under Rollins's name for every EmArcy album it recorded as the Quintet. But this arrangement produced only two albums, one for each label, before the tragic deaths of Quintet members Clifford Brown and Richie Powell in June 1956 ended the deal. Rollins had an idea for the album, as well as several original compositions, so the album had a sound distinct from the Quintet's.

Rollins heard Rosemary Clooney sing "Count Your Blessings" in the film White Christmas and decided to record a version due to his fondness for Irving Berlin standards. "Kiss and Run" is a duet between Brown and Rollins. "I Feel a Song Coming On" is a fast cover of a Dorothy Fields standard. "Valse Hot" is a jazz waltz which remains one of Rollins's most well-known pieces, just like his other composition, "Pent-Up House".

The AllMusic review by Scott Yanow calls the album a "particularly strong hard bop set."^[3] Author and musician Peter Niklas Wilson called it "a hastily produced – though ultimately rewarding – session."^[6]

All tracks by Sonny Rollins except where noted.

Side one

1. "Valse Hot" – 8:36
2. "Kiss and Run" (Sam Coslow) – 7:08

Side two

1. "I Feel a Song Coming On" (Dorothy Fields, Jimmy McHugh, George Oppenheimer) – 5:13
2. "Count Your Blessings (Instead of Sheep)" (Irving Berlin) – 2:30
3. "Pent-Up House" – 8:50

• Sonny Rollins – tenor saxophone
• Clifford Brown – trumpet - except Count Your Blessings
• Max Roach – drums
• Richie Powell – piano
• George Morrow – bass