A few miles south of Ellis Grove, Fort Kaskaskia sits on a bluff beside the river. The French built it around 1759 to defend against British attacks. All that remains today are lonely earthworks around the perimeter, a cemetery from the late 1800s and a view-tastic overlook. It's a great spot for a picnic, with tables and grills.
If you're into French colonial architecture, take the footpath down to ogle the Pierre Menard Home, built in 1802 for the gent who eventually became Illinois' first lieutenant governor. Trivia tip: the town of Kaskaskia was Illinois' first capital, though its tenure barely lasted a year.