I think it depends on whether or not you can have any comfortable social contacts with them at work. If they bore you, hate you, or otherwise don't connect with you at work, trying to socialize with them outside work is unlikely to work out. Lots of times too people feel their connection is the work, so if you or they go to a new department or company they fade away. I call my current coworkers "the gang of shallow" because there's nothing worthwhile in any of them. So I avoid socializing with them even when they do invite me. Other jobs, though, I found a few friends among some folks who have some intelligence. It just depends.
_________________
I would rather have my liver pecked out by a giant crow than spend a day at the mall. But I'd pay money to see a giant crow eat a mall.
Your Aspie score: 155 of 200 * Your neurotypical (non-autistic) score: 49 of 200 * You are very likely an Aspie