Taiwan is a good place to immerse yourself in a different kind of Chinese culture, or at least the Taiwanese version of it. But I agree with the above posters that Taiwan is unique and Chinese cultural heritage is just one aspect to what this awesome little country is about.
It's a fine place to learn Chinese though, and the number of still-living old Mainlanders ensures that if you are craving food from any given part of Mainland China, you can find a restaurant full of old folks that serves it.
Taoyuan city though? Well it's close to Taipei so you'll be able to escape on weekends (though Taoyuan County offers lots of great day trips too) to someplace with more comfort and style...but otherwise Taoyuan is small, boring and well...small and boring. With very bad public transit as has been noted above (there are buses, it's just...there aren't enough of them and there's no subway).
The upside is that you'll be forced to experience a lot more of local life. When you can't do what Taipei's expats seem to do and spend their days in Starbucks, their evenings at Sai Baba and their nights at Carnegie's, thereby completely avoiding all the local charms of Taipei...you really have no choice but to live like a local. That can be immensely rewarding (though frustrating at times...just wait until the first neighborhood Auntie tells you that you have a nicer ass than you did a month ago, and no I am not joking, that happened to me last week). But...you might find yourself bored in Taoyuan.
The good news is that if, after arriving, you realize you prefer Taipei, it's just a half hour away so you can come in on weekends. A lot of people commute daily between the two cities. Then, when your contract is up you can jump to a gig here or whatever city suits your fancy. Taoyuan is no more or less interesting than cities like Miaoli, Jiayi or Taizhong (Miaoli slightly beats the others due to the prevalence of Hakka culture and therefore delicious Hakka food)....and plenty of long-term expats love living in those cities.
