By the southeastern corner of the palace stands the attractive 17th-century stone Siddhi Lakshmi Temple. There was minor earthquake damage here but the temple is still standing. The steps up to the temple are flanked by male and female attendants, each leading a child and a rather eager-looking dog. On successive levels the stairs are flanked by horses, garlanded rhinos, human-faced lions and camels. The temple itself is built in the classic shikhara style, commonly seen in the north of India.

Behind the temple is a neglected corner of the square that contains a pair of lost-looking curly-haired stone lions who, depending on what theory you subscribe to, are either guarding the palace or the site of a lost temple that crumbled to dust in the 1934 earthquake. The small red-brick Vatsala Temple collapsed in the 2015 earthquake.