ATMs abound in Taipei and most are connected to either the Cirrus or Plus network and accept foreign cards from aligned banks. In addition to banks, look for ATMs at 7-Elevens, and other convenience stores. ATMs give you the choice of handling the transaction in English or Chinese.
You can exchange money or travellers cheques (US$ denominations are best) at the airport, or most commercial banks in the city, especially any Bank of Taiwan. There is no black market for currency exchange, nor are there currency-exchange kiosks.