Marshrutkas and minibuses are the most effective way to get around, though they're cramped for long journeys and you'll often have to wait for some time until they're full for them to depart. Shared taxis are a good alternative to marshrutkas, being faster and more comfortable (and you can even buy the remaining seats in a vehicle if you're in a hurry to get going).
Buses are a slow but cheap way to get around. The Ministry of Motor Transport lists routes, timetables and fares – all, rather remarkably, in English: http://www.awtomenzil.gov.tm.
Trains are slow but comfortable and a great way to see the countryside and meet people.
Train fares are likely to be charged in US dollars, although some travellers have reported paying in manat. You can expect to pay US$15 to US$20 for a journey of around eight hours.