Thanks for the responses. We are back from an amazing trip, I would like to share my experience to help someone with the same questions I had.
We ended up skipping the stroller and taking only our Beco carrier for the baby.
The baby carrier worked great in Venice since there were a lot of bridges. The carrier worked great in public transit (we used the metro in Milan and Rome, it would have been a hassle to look for an elevator each time or lug the stroller up and down stairs).
We did notice a lot of people with strollers, Milan, Florence and Rome were all very stroller friendly. Also, I had reports on the internet that strollers were not allowed in museums like the Vatican Museum. This info seems inaccurate, We saw strollers allowed in the museums we were in (Academia and Uffizi in Florence, Vatican).
Overall, I think taking a stroller would have not have been really that big a hassle. But, we are used to carrying our baby in the Beco and walking (a lot), so this option worked well for us.
Other things I would like to mention regarding the trip:
. Diapers & Formula: We stayed close to the big sights in most places, the pharmacies and grocery stores we visited did not have a wide variety of diapers. Only could find basic huggies or pampers, if you have a specific need (organic, hypo allergic etc), I recommend bringing /shipping for the length of the trip. Same applies to formula, if your baby only tolerates a certain brand, do not count on finding it easily over there.
. Ready to use baby food: Again only could find very basic stuff like bottled carrots, no fancy stuff like in the US. Also, there was baby food which had prosciutto, veal and such (did not help us since we are vegetarians).