There has probably been a stronghold on this site since the twelfth century. The Charteris family held the land from the 13th century onwards. The present tower was built by the Charteris family in the 16th century. That family feuded with the Kilpatricks of Kirkmichael leading to the murder of Roger Kilpatrick in 1526. Sir Robert Charteris of Amisfield fought a duel with Sir James Douglas of Drumlanrig in 1530.^[4]

The property passed to John Dalziel of Newton in 1636. The Dalziels supported the Stuarts in the Wars of the Three Kingdoms, and captain Alexander Dalziel was executed as a royalist in 1650.

Although the basic plan of Amisfield is a simple square with four stories and an attic, its richness in corbelling and turrets gives it a more romantic guise. Three corners have double-storeyed turrets while the fourth is decked. It has a steeply-pitched roof. These upper features are built in warm, red ashlar in contrast to the rubble walls below. As the tower was not built primarily for defence all of its shotholes are in the upper levels.

From first floor to base of the tower there is a projecting stair-tower, round for two stories, corbelling out to the square turret above . The entrance is defended by a machicolation.

Within the castle the rooms have fireplaces, and a painted border with lion faces in one room. The first floor hall has a garderobe and three windows. There is a vaulted basement, lit only by three gun-loops.

An oak door from the tower, fashioned by a local craftsman, is on display in Edinburgh at the National Museum of Scotland. It depicts Samson tearing open the jaws of a lion, and with a shield bearing the Arms of Charteris and Herries and dated 1600.

Hubert Fenwick described Amisfield as “simply marvellous”, saying that it “displays almost every Jacobean baronial conceit”.

There is a 17th-century mansion that was doubled in size ca. 1803 immediately adjacent to the tower. The property is owned by the Johnstone family.

Archaeological excavations in 2010 and 2011 have discovered the original tower house likely dates earlier than 1600.

The corbelling is so-called billet-and-cable design, the stonework imitating logs and rope. The dormer windows adapted the old French form of bretèche. Dog-toothed motifs surround the armorial panels and some of the windows.

• The Castles of Scotland, Martin Coventry, Goblinshead, 2001
• Scotland's Castles, Hubert Fenwick, Robert Hale Ltd, 1976.