I am not a doctor, but it sounds like you are just very intelligent and shy. You may have some AS traits. I am the parent of a little boy almost 5 that has been diagnosed as PDD-NOS/possible AS. It was evident at 1 yr of age that he had differences. He still has very little interest in children his age---the fact that you had friends when you were young seems and that you had no real apparent issues indicates to me that no, you don't have AS.
My son's father sounds a lot like you. His memory is off the charts. He knows almost every answer to Jeopardy. He was very shy and highly intelligent, but did not marry until he was 40. I think I was his first serious girlfriend. I am very social and outgoing, so I seemed to draw him out of his shell somewhat. Plus, I loved the fact that he was so intelligent, and did not seem to be a player or womanizer like so many of the men I had dated. Anyway, since we have been married and my son was diagnosed, I have been noticing a lot about him that seems to indicate that he has many AS traits. He is a computer engineer and has a degree in math. He has a few friends, but is not very outgoing. He seems to have difficulty expressing his emotions. But overall, I don't think he has enough symptoms to get a diagnosis. Even if he did, it would not matter because I love him just the way he is.
I think you just maybe need to get some practice on your social skills, and when the right woman comes along, it will happen!! Good luck to you!