The original church on the site was founded in 1055 and dedicated to Saint Olaf II of Norway. The abbey church was refounded in 1088 for Abbot Stephen and a group of monks from Whitby by the Anglo-Breton magnate Alan Rufus, who laid the foundation stone of the Norman church in January or February that year.
The abbey occupied an extensive, precinct site immediately outside the city walls, between Bootham and the River Ouse.
The abbey church is aligned northeast-southwest, due to restrictions of the site. The original Norman church had an apsidal liturgical east end, and its side aisles ended in apses, though they were square on the exterior.
The abbot's house, built of brick in 1483, survived as the "King's Manor" because it became the seat of the Council of the North in 1539; the abbots of St Mary's and the abbey featured in the medieval and early modern ballads of Robin Hood, with the abbot usually as Robin Hood's nemesis.
All that remains today are the north and west walls, plus a few other remnants: the half-timbered Pilgrims' Hospitium, the West Gate and the 14th-century timber-framed Abbot's House (now called the King's Manor).