Rio Claro is an excellent destination and worth the hassle to get there.
We went from Medellin and got dropped off at the entrance of the reserva. From there, you still have to walk a kilometer to the actual hotel. Depending on how much time you have, there are a number of places to visit along that way, in particular, Manizales (great non-travelled Colombian city in the mountains), Santa Elena (outside of Medellin), Guatape, and some small villages (Salamina and Jardin, between Manizales and Medellin).
The easiest way to get to Rio Claro is probably like you mention (Pereira to Medellin, Medellin towards Bogota and just get off at the reserva), but you could get there via Manizales and Honda as well if you feel more adventurous.
Good luck! Have fun! Say Hi to Juan, the Director of the Reserve. He's a true visionary!