It's best not to drink tap water anywhere in Côte d'Ivoire. Bottled water is available in shops throughout the country; Awa and Celeste are trusted brands.
Whether you're travelling by air or by land, you'll need a yellow fever certificate (aka 'the yellow card') to enter Côte d'Ivoire. If you don't have one, you'll be ushered behind a curtain for an on-the-spot jab when you arrive.
Make sure you have a comprehensive health insurance before you travel, which will cover evacuation and/or repatriation in case of emergency. Also ensure that you have sufficient funds available to cover initial costs of emergency healthcare.