The Norse were fond of zoomorphising smaller islands - for example, smaller islands lying off a larger one are often termed "Calf", e.g. Calf of Flotta, Calf of Man or even Calf of Cava (the latter a tautology). Some are even "hens", like the Hen of Gairsay. However "horses" are fairly rare. Coincidentally, the old name of Mainland, Orkney meant "horse island".^[3]

Like most of the other islands of Orkney, the bedrock is Middle Old Red Randstone of Rousay type of the Devonian period, but much eroded and tilted.^[4]

The islet is separated from Copinsay by Horse Sound, and to the southwest is Corn Holm. Mainland Orkney is due west, and Auskerry and Stronsay are much further to the north. The Horse is the easternmost of the southern Orkney islands.

The Blaster Hole is a sea jet, of the type known in the Northern Isles as a gloup. When a storm blows in from the east, the aptly named Blaster Hole can produce a spout of nearly 60 metres (200 ft) highly, solely by wave power.^[4]

The tiny islet was used for grazing, and appears never to have been inhabited (although a number of the smaller islands were often used by culdee anchorites). Pigs and sheep were grazed here, but not horses as the name might suggest. As Haswell-Smith says:

"As with many small islands, the demand for grazing was so great that the islanders [of Copinsay] would carry sheep to the top of the Horse, but it was said that it could fatten one sheep, and feed two, but three would starve. In the Spring, pigs were also hoisted up to fatten on the huge 'crop' of sea-birds' eggs. The pigs' coarse hair was used to make the strong rope needed for fowling. 'Lee-running' was the local name for the organised collection of sea-birds' eggs, which was still practised on Copinsay in the 1940s."^[4]