Said to have come from an 'old and respected family in Brittany and Normandy',^[1] it is uncertain where Robert de Romille came from. The oldest references state that the family was located in both Brittany and Normandy. There is a town in eastern Brittany named Romillé that gave rise to a de Romillé family, but Robert may instead have been a younger son of Rainfred Rumille from Remilly-sur-Lozon,^[2] 14 km from Saint-Lô in the County of Mortain.

In the Domesday Book the multiple estate of Bolton Abbey was listed as going from the lordship of Edwin, Earl of Mercia to the Clamores of Yorkshire.

The Bolton Abbey estate then included 77 carucates of ploughland (9240 acres/3850ha): Halton East, Embsay, Draughton, Skibeden, Skipton, Low Snaygill, Thorlby, Addingham, Beamsley, Holme, Gargrave, Stainton, Otterburn, Scosthrop, Malham, Anley, Coniston Cold, Hellifield, Hanlith.

But after 1086 that lordship was transferred to Romille. Finding the Saxon manse at Bolton Abbey beyond repair Romille selected a rocky outcrop in more strategic location in 1090 and built the eminently defensible Skipton Castle^[3][4]

After 1102 King Henry I confiscated the nearby Craven lands of the rebellious lords Roger the Poitevin, Erneis of Burun and Gilbert Tison. He put those in upper Wharfedale and upper Airedale into the governance of Romille. Clearly intent that Craven become a compact structure the King added in estates from his own demesne.^[5]

Robert de Romille's heir was his daughter Cecilia, who married William le Meschines, the lord of Copeland. Cecilia became famous as foundress of the Embsay Priory that became Bolton Abbey. Cecilia and William had two sons: Ranulph and Matthew but both died without issue, so their daughter Alice, wife of William fitz Duncan, was the eventual heir.