The manor of Little Durnford was mentioned in an inquisition at Amesbury on 6 October 1470, which found that John Wodhull, the great nephew of the owner at the time, was the heir to the manor and Tytherley.^[1]

The current house was built in the late 17th century and remodelled for Edward Younge, a friend of Lord Pembroke, around 1720–1740. The house was designated a Grade I listing in 1953.^[2]

Little Durnford Manor has been the seat of John Pelham, 9th Earl of Chichester, since he purchased it in 1966; the family's previous seat, Stanmer House in East Sussex, was sold by trustees for the earl when he was three years old.^[3]

A drive approaches the main house from the southeast, which is separated from a landscaped park by a strip of trees.^[4] The house has two storeys faced with flint and stone chequerwork, with five bays under a slate roof, and central glazed doors set within a modest Tuscan portico.^[5] It has a dining room described by Pevsner as "a splendid mid-C18 room with a proud chimmneypiece and wall panels of tapestry framed in plaster".^[6] The doors feature pulvinated frieze and pediments, and the walls inside the house feature an entitled plaster cornice with frieze and baroque plasterwork, dating to the late 1740s. The main staircase is at the end of the hall and features turned balusters. The service wing was heightened in the early 20th century.^[5]

A walled cottage garden of approximately half a hectare lies to the north of the house and is divided into several sub-plots, each with their own identity. The grounds also contain a lake, a swimming pool, and a cluster of cedar trees.^[4]