Hylton Castle was intended to reflect the family’s status. Whether there were earlier buildings here is not known but the impressive gatehouse tower, the only part of the castle still standing, was almost certainly intended to be the dominant element of the new building. Other buildings were situated beyond the tower to the east, possibly arranged around a courtyard.
The Hyltons maintained their rank and wealth until the Civil War. Despite a reduction in their means, the family remained prosperous enough to refurbish the interior and add wings to the north and south side of the gatehouse in the first half of the 18th century.
The last Hylton died in 1746 and shortly afterwards the estate was sold. A century later it was bought by a local man, William Briggs, who demolished the 18th century wings, added larger windows and rebuilt the interior. The appearance of the 18th century house is, however, known from a number of antiquarian illustrations. Further deterioration in the 20th century led to only the exterior walls being saved.