Another issue for me -
When I'm with women irl I feel more of a pressure that I'm supposed to act like them, at least to a certain extent, because of my gender. Most of the women I know are NT and I just can't act like them. I know we're all individuals and all that stuff. Even if they aren't judging me, I'm acutely aware of how different I am to them in my mannerisms, my behaviour, my speech, my everything. Even if they don't mind, I still notice and I feel ... alien or different like I don't fit in, or don't want to fit in.
When I'm with men (or NB people), I don't feel that way. They wouldn't expect me to act like a man, and I wouldn't feel awkward for not being manly. They might find me enigmatic because I'm not like men but I'm also not like (most) women, but for the most part men don't make me feel uncomfortable like I need to fit in with them.
_________________
And in the end, the love you take is equal to the love you make.