Modest in size, this 1829 Anglican church sits primly by the waterfront. It's usually always open if you want to look inside.