A birch-lined park along the Savinja River's northern embankment has an open-air lapidary of Roman remains unearthed in the Celje area, overlooked by the 16th-century Old Counts' Mansion, a lovely Renaissance building housing a branch of the Celje Regional Museum. The collection includes a large lapidarium in the basement with some 200 finds, and a huge collection of folk items and furnishings from the Savinja region on the ground floor.

On the 1st floor a dozen rooms are done up in styles from different periods (eg baroque, neoclassical, Biedemeier, Secessionist), but the main attraction here is the astonishing Celje Ceiling (Celjski Strop) in the central main hall. This is an enormous trompe l'oeil painting of columns, towers, angels frolicking skyward, noblemen and ladies looking down at you looking up. Completed in about 1600 by an Italian artist, the mural was meant to lift the ceiling up to the sky, and it does just that. Other panels represent the four seasons and show scenes from Roman and Greek mythology.