I usually prefer mens t-shirts because I'm 5'9" tall and womens tops are too short most of the time. I find if I buy a small mens shirt then it'll be fitted enough not to look like a tent and they actually look better on me than the women's tops anyway.
The thing that annoys me the most is the inconsistancies in the sizing even amongst the same retailer. I remember trying on some jeans and I ended up needing to try about four different clothes sizes across the styles.