ATMs are found around the Plaza de Armas, at the airport, Huanchaq train station and the bus terminal. All accept Visa, most accept MasterCard.
There are several big bank branches on Av El Sol; go inside for cash advances above daily ATM limits. Casas de cambio (foreign-exchange bureaus) give better exchange rates than banks, and are scattered around the main plazas and especially along Av El Sol. Moneychangers can be found outside banks, but rip-offs are common.
Credit cards are accepted at more and more hotels, restaurants and tour operators, though it's always smart to ask in advance.