Below Kars Castle, the imposing, basalt Kümbet Camii was built as a church between 932 and 937 when Kars was (briefly) capital of the Bagratuni kingdom of Armenia. The church was converted to a mosque in 1064, when the Seljuks conquered Kars, then used as a church again in the 19th century by the Russians, who added the porches. Naive reliefs of the 12 apostles still adorn the upper drum tower beneath the conical dome.

As a young man, before developing his own very distinctive spiritual path, George Gurdjieff sang in the church choir here and was said to have been deeply influenced by his conversations with the then priest, Damian Borsh.