I was there in 2012 and went via public transport from Ft. Portal to Kabale in one day. I didn't leave until 9am, so if you leave earlier you might be able to get to Kisoro. I took the small minivans (matatus?) the entire way.
It went: Ft. Portal to Kasese, Kasese to Mbarara, Mbarara to Kabale. From Kabale I took a shared taxi to Kisoro a few days later.
I always insist on the front seat, for the view and (usually) a working seat belt. If you start very early you might be able to get to Kisoro in one day....and of course, money talks....If you want to spend a bit more I'm sure you can get the vehicles to leave sooner, etc. (that's your biggest risk....vehicles waiting to fill up) and perhaps hire your own car from Kabale to Kisoro. Regardless, it's a long day...but great adventure and scenery.