Yeah, there's really nothing unusual about this, but I can see why it would make you uncomfortable--I might have had doubts my first time in Manaus, too. Amazon Backpackers is a well-established agency, and while the staff shifts, they're unlikely to work with anyone who would be risky--and actually, anyone picking you up then is likely to be the owner or owner's direct assistant. They only want to take you directly to the office so they can lock in your business. Competition for tourists is insane. I have watched people walking down the sidewalk with the intent of going with one tour company and being picked up by a rival company two doors away. I've seen guides in the airport spot a lost-looking tourist and say "well, instead of calling your company, why don't you just sign up with mine?"
In other words--Amazon Backpackers, not sketchy. Just trying to hold onto your business before you get booked with another agency, whether sketchy or legitimate, and there's reason for them to worry.
Also note that (at least last time I was in Manaus) the Amazon Backpackers office is located inside a hotel. (Not a great hotel but it's perfectly safe.) There'll be people around.
And finally--look at it this way, you've been offered a free ride from the airport into central Manaus late at night. That's a VERY good thing. I don't feel that a taxi driver would be any more safe. (And hell, the taxi driver might try to sell you a tour, too.)