Any loud noise can be distressing if it goes on for a long time, but no, I do not find babies annoying. They have as much right to be who they are as anyone else, and only very limited mean of communication available to them. It's not their fault if those methods are loud.
I remember when my daughter had what they call "Colic" as an infant - which is basically an inexplicable and extended gas attack, causing the child severe abdominal discomfort, resulting in a crying jag that may go on for several days. It wasn't her endless crying at the time that bothered me, but the emotional distress of knowing she was in discomfort, and being unable to do anything about it, but walk and rock with her for hours on end, until she fell asleep.
_________________
"I don't mean to sound bitter, cynical or cruel - but I am, so that's how it comes out." - Bill Hicks