It depends on how well you spoke at 3. Most kids start speaking earlier than that, and are using complete sentences, with reasonable grammar, by 4. So it depends on which your mom meant -- first words, or complex phrases?
In the end, it's probably irrelevant. My kid went back and forth between the two, because when he was 3 or 4, and lagging on speech, that was consistent with an AS diagnosis. Now it precludes an AS diagnosis, since they've changed the definition. But any aspie who meets him immediately identifies him as an aspie.
As someone pointed out about Einstein, he was (using the late speech standard) HFA, so obviously HFAs can be extremely intelligent and function perfectly well in society.
It's not something I'd worry about one way or the other.
---
From Gillman's diagnostic criteria for AS, 1991:
"C. Speech and language problems, as manifested by at least three of the
following five:
1. Delayed development of language."
From DSM IV, 1994-2000:
"D. There is no clinically significant general delay in language
(eg: single words used by age 2 years, communicative phrases used by
age 3 years)."
So the definition depends not only on who you ask, but when you ask it.