The Cueva de Punta del Este, a national monument 59km southeast of Nueva Gerona, has been called the 'Sistine Chapel' of Caribbean indigenous art. Long before the Spanish conquest (experts estimate around AD 800), indigenous people painted some 235 pictographs on the walls and ceiling of several caves. The largest has 28 concentric circles of red and black, and the paintings have been interpreted as a solar calendar. Discovered in 1910, they're considered the most important of their kind in the Caribbean.
There's a small visitor center and meteorological station. The long, shadeless white beach nearby is another draw (for you and the mosquitoes – bring repellent).