You can buy a combined EPIRB and GPS. So, great, you know exactly where you are drowning. Why? What am I missing? If you had a VHF on the thing, perhaps, but I think I've missed the big picture somehow

Because if you had to abandon ship hopefully you are bright enough to have a handheld radio or even a mobile near your EPIRB so you can help your rescuers to stop you from drowning.
But you have a GPS and VHF in the grab bag, and the liferaft, with half a brain. Why do you need a personal GPS, I bloody well want a personal EPIRB? Unless its an Irridium, a mobile is useless, 40 miles offshore. It seems like an expensive gimmick to me.

An EPIRB with a GPS cuts down the search time because the SAR will know exactly where you are. It goes beep every 90 seconds and so your actual position is updated every 90 secs. One without a GPS will place you within a grid of about 5 square NM. In rough seas, you'll be very difficult to spot.