About 1.5km past Tmogvi Castle, up on the left of the road, are the remains of Vanis Qvabebi, a cave monastery that predated Vardzia by four centuries, with a maze of tunnels inside the rock.