There are two main border crossing points into Uganda: Malaba and Busia. Both are generally pain-free as Ugandan (or Kenyan) visas are available on arrival.
Both towns have a couple of banks where you can exchange cash, but unless you’re a fan of bureaucracy it's easier to use one of the numerous moneychangers prowling around. Just make sure you know the exchange rate beforehand (but don't expect them to match it) and count your money carefully.
The Kenyan border is open 24 hours, but we’ve heard the Ugandan one runs to a more ‘flexible’ timetable, so try to arrive in daylight hours. It's also prudent to keep an eye on your bus. Nothing irritates the drivers more than someone who hasn’t the sense to take note of where they park.
If you get stuck in either town for the night, you’ll find a couple of ropy places to put your head down. Thankfully, onward matatus are fairly easy to come by on both sides of the border.
If you're travelling into Tanzania, the main border crossing at Sirare is open 24 hours and is generally hassle-free. Don't believe the Kenyan moneychangers who'll tell you that there is no bank on the Tanzanian side – there is (open 8.30am to 4.30pm Monday to Friday and 8.30am to 12.30pm Saturday), in the same building as immigration. There is no cross-border public transport, but matatus are easy to pick up on either side if you don't mind waiting.