Kabale is an interesting place to stay for a day to position yourself to enter Rwanda (or VV). Also an easy jump off place to visit Lake Bunyonyi. We stayed at the Whitehorse in Kabale for a night--inconveniently located. A decent backpacker spot would be the Edirisa in the center of Kabale. They can provide good info for transport to Rwanda. They might hook you up with "Milton" who drives to Kigali daily to deliver newspapers. Fast, reasonably priced and he'll take you right to your hotel in Kigali.
We stayed for a couple of nights at "Nature's Prime" on the lake--a splurge but very comfortable.
