The hulking fortress that gave the city its name lurks on the far side of La Savane and dates from 1640, although most of what stands today is the result of subsequent additions. After years of being closed to the public, Fort St-Louis is finally open to tours, and it's easily the top sight in town. Buy tickets and join tours at the Tourist Information Kiosk on La Savane.
Bear in mind that this is a working French naval base, and so tours only take you to some parts of the fort. While there is little to see in terms of furnishings or display, just walking through a massive colonial fortress in the Caribbean is a hugely atmospheric experience, particularly on the rooftop, where a massive French flag flies over the bay. Tours last just over an hour, and are available in English at least a couple of times a day. Do be aware that when cruise ships are in town, tours can quickly get booked up to their maximum of 20 people; you can go early and enlist for a tour later in the day. The fortress is not disabled accessible.