Ulstrup Castle traces its history back to the end of the 14th century when it was owned by Jens Brandsen. Later owners include Queen Margaret I but the original house later disappeared and Ulstrup continued to exist as a village. It was acquired by privy councilor Christen Skeel in 1579 and dissolved to make way for his new manor house which was built in 1591. The building was expanded between 1615 and 1617 by his son, Jørgen Skeel, and an agricultural building (avlsgården) was built in 1668.^[1]

The estate remained in the possession of the Skeel/Scheel family until 1809. Later in the century, Ulstrup reappeared as a railway town located on the Langå-Viborg railway line which opened in 1863.

In the 1920s, Ulstrup was dissolved and most of the land sold in parcels. From 1951 the remaining grounds housed an amusement park and zoo.^[1]

The north wing of the complex incorporates Christen Skeel's original house from 1591. His son's expansion added a south and a west wing. The complex was later closed with the construction of a lower gate wing to the east. The gate wing is flanked by two octagonal corner pavilions and its central section, above the gate, is topped by a small tower with a lantern. The west wing was demolished in 1755.^[1]

Since 1980, Ulstrup Castle has been reestablished as a manor house after much of the land was reacquired and the buildings thoroughly restored.^[1]