Construction in 2003 along nearby Prestwich St unearthed many skeletons. These were the unmarked graves of slaves and others executed by the Dutch in the 17th and 18th centuries on what was then known as Gallows Hill. The bones were exhumed and this memorial building, with an attractive facade of Robben Island slate, was created. It includes an ossuary and excellent interpretive displays, including a replica of the remarkable 360-degree panorama of Table Bay painted by Robert Gordon in 1778).

There's a branch of the coffee shop Truth in the memorial building.