Cheap hotel near the airport

Well, there is a transit hotel inside the airport itself. It's called the Evergreen Transit Hotel and you'll need to be inside Terminal II to use it. I don't think it's very cheap though, but it's certainly convenient. My understanding is that most hotels near the airport aren't any cheaper and may in fact cost more. You may have to go into Taoyuan (the city) itself to find something that meets your criteria of being cheap.

How about heading into Taipei and pulling an all-nighter?

This is the only one I know that is reasonably priced:
http://www.gueylin.com.tw/intro-e.html