Taxis are only really necessary in Kigali. It’s also possible to find the odd taxi in most other major towns.
Most towns are compact enough to walk around, but otherwise a moto-taxi is a good bet. It’s just a motorcycle, but the driver can usually sling a pack across the petrol tank. They’re generally fast but safe, and there’s usually a helmet for the passenger.
Before the civil war, there were ferries on Lake Kivu that connected the Rwandan ports of Cyangugu, Kibuye and Gisenyi, but services have not been resumed. Speedboat charters are currently the only option between these ports, but they are prohibitively expensive.
Rwanda has efficient and reliable public transport. Privately run buses cover the entire country, and with scheduled departure times you won’t find yourself waiting for hours while the driver scouts for more passengers. Tickets are bought in advance from a ticket office, which is usually the point of departure.
You will also find plenty of well-maintained, modern minibuses serving all the main routes. Head to the bus stand in any town between dawn and about 3pm and it is quite easy to find one heading to Kigali and nearby towns. Destinations are displayed in the front window and the fares are fixed (you can ask other passengers to be sure). However, anyone who gets stuck somewhere late in the afternoon is going to have to pay top price for the privilege of getting out. Minibuses leave when full. Neither buses nor minibuses are supposed to charge extra for baggage.