Local buses circle the island and run along the inland road to El Cove; they're the cheapest way to get around (COP$2000 per ride). They can drop you off close to all the major attractions.
A bus marked 'San Luis' travels along the east-coast road to the southern tip of the island; take this bus to San Luis and Hoyo Soplador. The bus marked 'El Cove' runs along the inner road to El Cove, passing through La Loma. It'll drop you in front of the Baptist church, within easy walking distance of La Piscinita. You can catch both buses at the end of Carrera 5 in San Andrés Town.
The best way to travel independently around the island is by scooter (from COP$70,000 per day, but as much as COP$120,000 in high season) or golf buggy (known locally as a mula; from COP$100,000 per day). Many of the dozens of rental businesses are on Av Newball in San Andrés Town and are concentrated around the tip of the island. Most will also deliver to your hotel. Shop around, as prices and conditions vary.
A taxi for a sightseeing trip around the island will cost about COP$70,000.
Cycling around San Andrés is a great way to get a feel for the island. Roads are paved, there are very few hills and there's little traffic to contend with. Rentals start at around COP$10,000/20,000 per half/full day.