Excellent advice given above !
I have been to all the places you mention, and like them all a lot, for different reasons. I dont know your interests, but personally if I had to choose only one, it would be Hermanus (as you are travelling in Whale season). Hermanus (and the surrounding areas) is a fantastic place for Whale watching, from the shore and from a boat. Its also a pleasant town with a relaxed holiday athmosphere, good restaurants etc. If you go to Hermanus, I would also drive one way along the coast (and the other way inland, as I also like Sir Lowrys Pass). If you like wine, there are some lovely places in the nearby Hemel and Aarde Valley. I also like the small village of De Kelders, with spectacular cliffs, great Whale watching from the shore and nice walks.
Paternoster is a very beautiful small place. Its very pretty, with small cottages, flowers, a wide and very beautiful beach, colorful fishing boats, a couple of good restaurants and beautiful sunsets. Its a lovely place, but there isnt much "to do or see" (which for me is probably part of the charme).
In the Cederberg, besides hiking and beautiful nature in general, you can also see ancient cave paintings from the San bushmen. Also in spring, beautiful "carpets" of flowers. But you are probably a week or 2 late for the "main flowering". Or what do you mean Ghalied ?