My travel agent, Flight Centre HK suggested we get a 3 day city pass since we are going to be in London for 5 days. It says that it covers most of the big tourist destinations. Is it worth doing the city pass with travel?
Many of the "big tourist destinations" are free. And "with travel" is likely to cost far more than you need. Unless you are staying in an outer suburb of London, "with travel" will be a waste of money.
Whether the cost is worth it depends on what you are planning on seeing. You can look up the individual places on Google to see the admission charges (if any), and do the maths. Be realistic about how many places you will be able to see; rushing about trying to get your money's worth because it's all included is a bit silly.

Unfortunately it is all about maths and options and your post doesn't really provide enough info.
Questions include: How much is the pass? How many places (included in the pass) do you intend to visit? How many of you is 'we'? Because if 'we' is two then the 2-4-1 deal which you can get by buying a travelcard from a mainline railway station (not a travelcard from a tube station) might be a better option.
Basically, take the cost of what it saves you compared to the cost of what it costs you and that'll tell you. And compare it to 2-4-1 tickets with travelcard.
This is a good outline: http://www.londontoolkit.com/whattodo/london_travelcard_2for1.html
At a guess, IMO it won't be worthwhile but that depends on your interests. You could fill a good solid three days in London just doing free stuff.

Most of the major attractions are free
Most of the paying attractions are covered by www.daysoutguide.com which can be used if you buy a Travelcard at a National Rail ticket office rather than at a Tube station or using Oystercard
Travel option is a massive waste of money as it covers all of London whilst most people stay in the central zones - read the following, if there is more than one of you intending to do touristy things then it's the biggest bargain you are likely to see as it means you get half price entry AT NO ADDITIONAL COST TO YOURSELF as you'd be needing some means of getting around London.
http://www.londontoolkit.com/whattodo/london_travelcard_2for1.html
If your TA thinks a London Pass is the best option then they either know very little about London or get a large amount of commission (or both)
I'm in london for a couple of days maximum, arrinving in the afternoon via Victtoria bus station and travelling probabaly as far out as Ealing common. Maybe a bit further at a push. What pass would I be recommended to get?

Just buy an Oyster card, regular, not tourist, and load, say £10 initially, and add more as you need - office staff can advise how much you're likely to spend, if you explain where you're intending to travel. Of course, if there are 2 of you, then buying a paper Travelcard (daily or weekly, so you'd need 2 x daily ones) means you can use 2 for 1 entry discounts for some attractions. See website in #3 to select what best for you.

With regard to post #5 it is vitally important that the travelcard is purchased from a mainline railway ticket office and not a tube station ticket office.