Hi there!
Where are you flying in from? Have you looked into the cost of connecting flights? While it may save you money to fly in to HCMC and take the bus to Siem Reap (~$20), it will eat up a lot of your time. Just be forewarned. You're ideally going to want two or three days to explore Angkor. Three is optimal, as the heat can tire you out and thus you can still have a few days to explore the vast ruins at a bearable speed. That leaves you with two days to take the direct bus, which, if I recall correctly, is a 12-hour ride with a short, half-hour stop in Phnom Penh (I only did the SR to HCMC leg, having flown into SR from Luang Prabang). That would be cutting it close if you had to be back to make a flight home from HCMC. It would be a shame, in my opinion, to spend so much time from a 5-day trip on a bus. Maybe bite the bullet and fly into SR so you can really get the most of your time or try to find a cheap connecting flight when you get to HCMC?
PS. Angkor Wat is UNBELIEVABLE. Enjoy!