Yes, I agree the Brazilian phrase is ambiguous, but I'm sure it would be understood in conversation. In writing, something else may be preferable.
I hope you don't resent the following corrections. Sem querer ofender.
Por o should be pelo, but in fact the more usual preposition following chateado/a is com. Portuguese prepositions are as erratic as English. Since a gente is feminine, and the state of being annoyed is not inherent, the verb is estar and the adjective ends with a.
In Portugal, A gente está chateada com... would be taken as meaning 'people are annoyed with'. 'Estamos chateados com' means we are annoyed with. The nós is optional. To say 'We’re all annoyed with... it would be 'estamos todos chateados com...'
In the context of being annoyed, ‘pessoas’ would be unlikely, but not incorrect. O povo means the people, and as in English, can have political implications. The people are starving. In Brazil, todo mundo, means everyone, and pessoal is used colloquially for ‘all you’. Olhe pessoal! = Listen everyone! Oi pessoal = Hey you lot (a group of friends).