Hi, You should absolutely go to Tainan, it's far far more interesting that Taipei if you are interested in historical architecture (in Taipei almost everything was rebuilt in the end of 19th and beginning of the 20th century by Japanese and next by Chinese again after 1945. Tainan, the first city in Taiwan and the oldest capital of Taiwan until mid 19th century, has not been rebuilt,. You'll find there remnants of Dutch architecture from 17th century, southern Ming architecture, Qing architecture, British colonial architecture.etc. the oldest Confucius Temple in Taiwan, and a very lovely district of old streets and Ming loyalist temple (I have forgotten it's name) across the street to Confucius temple, on the right from the main entrance (with a passage between the houses as narrow as 3 feet, close to the very famous 2nd hand book store) - and many shops with art creatioins behind.
The Red Tower (Chih-kan-lou, Fort Provintia) and Anping (Fort Zeelandia) are a must, but then close to the second one you have the First Old Street (Laojie) with its famous market and wonderful restaurants. Dont forget to eat the Coffin sandwich in one of them.
Opposite to Fort Provintia you will find the Guangong Temple (Ssu-dian wumiao, temple of Martial Arts) and Tianhou gong (Mazu's Temple "Later Heaven"), old residence of the Ming royalists.
Near Anping, don't forget to visit the British colonial merchant houses, the Tree House and the Salt Production Area, and many more. Tainan is evidently the most interesting city in Taiwan, and it is withing only 2 hrs ride by the THSR (High speed rail) - staying there for 2 days with the wandering through the streets in the night is an unforgettable experience.
Tainan is overlooked by most tourist routes, because it is far from Taipei, the main airport, except by those coming from the south (Filippines, Macau, Hong Kong), so unlike in Taipei where you can only hera cenglish, Japanese (sometimes also Russian) as foreign languages, here you can hear Spanish, Portuguese and Dutch as well - and you will really feel the atmosphere of a historic city there. And the Tainan night markets have definitely different atmosphere than those of the north, go e.g. to Hua yuan ye shi (Flower Garden Night Market), it's not a permanent market, with shops, like to\hose in taipei, it's open only on two or three days of the week, so it is more like a Jahrmarkt in many European cities..
I am going to Taiwan again in the middle of May, this time with my wife, and we are planning Tainan as one of the most important points of our trip,