Bring them! Uganda is an incredible place for kids.
I started bringing both of my children to Uganda when they were less than a year old. Our older boy lived here from 2 1/2 to 5 years old, and our 2-year-old is currently here in Uganda on his 4th 6-week trip. We have had them in villages, cities, remote areas throughout the country for extended periods, even though we are usually based in Kampala.
Both our boys are up-to-date on all the regular vaccinations that any kid gets in the US, but beyond that we haven't given them anything special. When we are in Kampala we don't have them on malaria prophylaxis, but when we go "upcountry" we give them each larium. Some people don't deal well with larium, so if you choose to give them that, you should definitely start it when you are still at home to see how they react. Doxy is a pretty benign medication for malaria. Ask at a travel clinic what they recommend. Don't bother with your regular doctor, as he/she probably doesn't know anything about malaria or other tropical diseases.
As far as someone to care for your kids, rather than investing in bringing someone over, consider having someone at the place you will be working recommend someone local who can watch them. You will be supporting a local, putting money in the local economy, and your kids will have the chance to interact with - and maybe learn some language from - a Ugandan. You can easily get someone full-time for about $100 for the month, and that will feel generous.
At that age, it might be difficult to have your kids "drop into" a school. The schools here are very different from schools in the US. Again, you might be able to ask around at the place where you will be working.
TB is mostly an issue if you will be in, or have your kids in, small, confined spaces for an extended period of time with someone who is infected. Your lungs pretty easily "flush out" TB when you get into fresh air if you have just been around it for a short time. It is unlikely to be an issue in a school. That said, I'm not a doctor, although I have two fathers-in-law who have both worked as doctors here in Uganda and that is what they have told me.
Your kids are at an age where they will really get a lot out of an experience like this. You have to make your own decision, but I can say from my experience that I have always felt extremely safe having my kids here with some simple precautions.
If you do it, I hope you have a great experience!