As Burundi is so small, it is feasible to visit these sights during the day and return to Bujumbura before nightfall. However, most towns have some form of basic accommodation so if you want to prolong your enjoyment of the stunning Burundian countryside then it's perfectly possible, and safe, to overnight close to any of these attractions.
Source du Nil
It is not quite as obvious and impressive as Jinja in Uganda, but this insignificant-looking little spring at Kasumo, 115km southeast of Bujumbura, may well be the southernmost Source du Nil. In a nice touch, a stone pyramid marks the site, but unless you have your own transport it is almost impossible to reach.
Gitega is the second-largest town in Burundi, which makes it more a village than a town. It's a worthy day trip from Bujumbura and there are a selection of reasonable places to stay. Minibuses (BFr5000) and shared taxis (BFr7000) from Bujumbura’s Gare du Nord bus station make the run to Gitega throughout the day (two hours).