The song is about the stresses of touring and how being away from loved ones can make musicians unwilling to trust others or carry on with their career, contrasting it with the humour of ironic, pretentious or rude things people say to touring musicians. Tennant has said that many of the lines in the song, such as 'And someone said, "It's fabulous you're still around today—you've both made such a little go a very long way!"' actually happened.^{[citation needed]} Speaking to NME in 1993, Tennant commented that the song was "basically about the strange things that happened to us when we were on the last tour."^[4]

"Euroboy", a dance track written by Neil Tennant and Chris Lowe^[5] in the Eurodance mould of such bands as Cappella and Livin' Joy, was released as one of the B-sides on "Yesterday, When I Was Mad". It later appeared on the US release of the album Disco 2, the B-side collection Alternative and the 2001 2-disc re-release of Very.

The track includes Chris Lowe in one of his rare lead-vocals performances, singing through a vocoder. The Boys claimed ^{[citation needed]} to have been unaware at the time of release that Euroboy was also the name of a softcore gay pornographic magazine.

It was occasionally performed live on the Asian leg of the band's 1994 Discovery tour.

Dave Jennings of Melody Maker picked "Yesterday, When I Was Mad" as one of the magazine's "singles of the week", calling it a "magnificent piece of bitchery" with "glorious lyrics". He remarked, "Pet Shop Boys aren't exactly the first band you'd expect to write a brilliant rock 'n' roll on-the-road song; but that's just what they've done here, skewering the sycophants and patronising slimeballs hanging around their tour with malicious delight."^[6] Alan Jones from Music Week gave the song three out of five, calling it "a bright tongue-in-cheek romp, but its galloping disco style makes few concessions to melody."^[3]

David Quantick from NME commented, "My theory is that they are now entering a period of being completely barking mad. This single bears it out, with its Noël Coward cover and its Broadway hell version of 'Can You Forgive Her?'."^[7] Brad Beatnik from the Record Mirror Dance Update noted, "This duo seem to be getting more excited about dance mixes with each single they put out. This one, another idiosyncratic and charming pop song, has about eight mixes."^[8] Another Record Mirror editor, James Hamilton, declared it as a "hi-NRG galloper".^[2] Jonathan Bernstein from Spin viewed it as "a wry litany of faint praise with which the pair have been damned".^[9]

A music video was produced to promote the single. It was directed by British director of music videos and advertising Howard Greenhalgh, and as with his previous videos for the Very campaign, makes prominent use of computer graphics. Taking the song's theme of "madness" literally (insanity rather than anger), it features a straitjacket-clad Tennant trapped in a surreal psychiatric hospital, all the while being taunted by a tuxedo-wearing version of himself, who represents the critic in the song's lyrics. Saturated colours were added in to give the video a nightmarish, unsettling quality. ^{[10][better source needed]}