Developmental background, in my opinion. There's a lot of overlap between autism and other developmental disorders, and even some personality disorders, if we're talking about borderline autism. But the early developmental markers/traits are more reliable. NVLD is very similar to autism later in life, for example, but one of its primary signs in early development is advanced verbal skills, rather than language delays/abnormalities as in HFA. Some specific cognitive weaknesses and skills also set HFA apart from other developmental disorders (some of these are still debated, but I think they'll become more important as diagnostic markers eventually). Last, I don't believe that special interests (when they're clinically significant) are associated with any other disorder. Although they're far from universal in HFA, I think they're the easiest "tell" for autism.
It's tempting to look for general traits that create an easy distinction between autism and other issues in an individual, but if we're committed to the current inclusiveness of the spectrum, the differences will mostly be found in very subtle trends. Because of that, I don't think there's any way to know whether an individual has autism specifically unless they've undergone neuropsychological testing. Otherwise, it's difficult to rule out a diagnosis that's more specific.