Psychometrics, the Autism spectrum, and the Internet

Post Reply New Topic

I just want everyone to be careful about the tests online that are often linked here. Most of them are not empirically reliable or tested for validity.

I'm a psych student specializing in research so I thought I would spread the knowledge.... unfortunately I experience a "word vomit" in my head where so much information comes forward when I know a lot about something that its hard to put into words.

Basically, it is possible that a test someone made and put on the internet will not demonstrate exactly how Autistic one is, or how much of a specific trait one displays. For example, a test may give 10 of its questions about obsessive behaviors. However, of those 10 questions, 2 of them do not show different results between Aspies/Auties vs NTs. Then, some of those other questions are only applicable to the most extreme on the spectrum, and so on.

Questions that are based on a Likhert Scale (ie strongly agree, slightly agree, etc) are quite controversial in the field of psychometrics. This is because "strongly" may mean the same thing to one person as "slightly" means to another. So, it is better to ask "If you experience meltdowns, how many do you have per month?" than "Do you often have meltdowns?" and so on.

I'll add more to this later as I put it into words. Perhaps if I can make it detailed and clear enough it can be stickied so all can read it.

I would never be able to answer a question about how many times I had a meltdown(which first would need to be defined)because I have a very poor concept of time and seldom remember the details of the past(unless it is a specific area of interest and then I an remember it for years and even "see' the book I read it in or recall what house I lived in at the time.

I thought most tests were designed as follows.....

They ask the questions of people who are known to have AS DX and those known not to have AS and if you answered more like the people answered who have AS then you were more likely to be AS.Personally,I cant wait for a less subjective test then an "expert" asking me for my memories(subjective as well)even my parents answers are subjective....some body needs to come up with something a little more scientific...like a blood test,but that is the flaw in most "psychiatric" vs neurological Dxes and even though AS is Neurological they use the psych methods for testing it.

As a frame of reference.....I received a 163 on the RDOS test and was DXed with AS by someone who is supposed to be an "expert".The DX seems feasible to me but I will never be 100% certain until they come up with something more scientific.

I spend a sizeable amount of my time in the same position, but you explained yourself very well. Thanks for watching out for us.

Ha, I'm still waiting for the question on the tests: 'Do you talk?'

Talk about being biased towards certain people. ;)

(This is why I like people to apply themselves to the DSM-IV-TR criteria for Asperger's and what have you; that's what the professionals use as a diagnostic reference -- and it's impossible to tell how severe you are to another without an objective individual's opinion; a psychiatrist who sees diagnosed people for example.)

I think questions such as "do you experience meltdowns?" could have similar responses. Some may see any overload as triggering a small meltdown, or a big one. Some may not think of them as meltdowns every time. It would almost be impossible to say that the question would give much better of a response when the term "meltdown" rarely means the same thing for most people, as there is no real description of exactly what it is for everybody.

Then, more questions could possibly come about... what types of meltdowns are you looking for? The type when you shut off and don't want to talk to anyone? ...or the type that leaves me feeling like a spazz after it happens? The ones that happen for no real apparent reason? ...or the ones that have specific triggers?

Maybe I'm analyzing too much, but I think with any given question, no matter what it is, there will always be difficulty trying to get any solid results.

One problem with this is that people with AS are used to "looking for" stims, meltdowns, and that sort of thing. An NT may very well have meltdowns, but they probably wouldn't describe them using that terminology. Likewise, they may stim (as has been discussed lately), but not identify it as stimming. Also, the more you focus on the "odd" things you do, the larger the problem seems in magnitude, so someone who obsesses about their AS is likely to give their problems a higher severity rating than someone who doesn't really. Once you start thinking you have AS, you tend to pathologize things, and see AS symptoms everywhere.

Anyway, you can't really determine which answers are most linked to AS versus NT without doing, as Rocko said, an actual study where you got a norm, standard deviation, etc. Even if the average NT score is 30, and the average AS score is 60, you don't actually know if that difference is significant or not until you've analyzed the results statistically. If there is a lot of variation of scores within the groups, there is a good chance that the results aren't significant- which would lead to the conclusion that the test does not discriminate between AS and NT. But you *can't* do that just by looking at mean scores.

Part of the problem is that it is difficult, especially for more high functioning people, to get a diagnosis in the first place. So you are put in a position where you have to justify to the medical professional why you think you have AS and essentially argue your case. This forces you to 'look for' things and does not necessarily mean that your symptoms aren't that severe or that you are an NT really.

Symptoms are still valid and should be recognised as such. The shortcomings of the diagnostic process and the lack of understanding among medical professionals is to blame for forcing people with AS to come accross as though they are 'looking' for symptoms and problems.

It doesn't make their ASD any less valid.

I couldn't have said that any better myself! You hit the nail on the head. Thank you so much for posting this.

Right, and I'm very high functioning and my Dx process has been a real pain in the ass. My point wasn't that symptoms are invalid, but that online tests are not a good way of validating symptoms.

I agree.I also think that even the RDOS test and the MMPI,have questions that are so ambigious to me that I have difficulty answering them.One of the things we were taught in philosophy is...."define your terms".Until the "experts" can agree on the terms,how are we supposed to know what they mean by "stimms" or "melt-downs"?We have had many people here who have had threads on defining these terms and we can't seem to agree and I dont see a consistancy in the "experts" from the books I have read about AS.Another issue with AS DX is that there is so many over lapping traits with other DXes.I might score low on a trait that is most similiar to OCD but high in the area of social anxiety.According to the Statistical manual,you dont have to have All 8 criteria...just 1 out of 4 in one section and 2 out of 4 in another.According to the psych I went to I did have them in all 8,yet I am an aspie who has worked since the age of 16 and graduated from college in 5 years.I also used chemicals(alcohol) to function socially and had several breakdowns that resulted in suicide attempts,CD treatment,psychotic breaks and some time in psychwards....SO,am I high functioning or......other?

I would love to see better defined criteria for DX and they hopefully will resolve this in the upcoming DX manuel.I hope they include more of what has been learned from the "experts"who have gotten to meet "female aspies",I believe we present in different ways,the same as we do in other areas of life...socilization and hormones do have an effect that I hope can be better understood......(but I'm not holding my breath on this one,as they keep doing research based on results using only male subjects.)

Ironically I see RockoTDF list he has undiagnosed aspergers.

0_equals_true and I wrote javascript version of a schizotypal personality disorder test I found online that has been empirically tested. I did add a between option of "maybe". Personally I think a between option is critical. I also added an unsure option. Since presumably most psychologist are not IT trained they want paper test with two choices that are easy to grade.

http://www.wrongplanet.net/modules.php? ... ic&t=39878

I consider that some on line tests can be used as a 'platform' from which one can seek a profesional diagnosis. Although I had previously been told that I should seek a DX for AS from a friend that worked in this autistic field, it was not until I had researched AS and completed a couple of tests that I felt that I should seek a consultation, and was thus dianosed with AS.

I also would acknowledge all the work and time that RDOS has put into his test in an effort to help those considering of they have AS.I think they are pretty good tests and I would like to take the opportunity to thank him for all his work.I wish some of the "professionals" would be so observant and "think out side the box" in asking the question..."what is AS".I couldnt find a "hats off to you emoticom",but hats off.

I don't have a lot of faith in "professionals" myself. I think a good portion of the people that come to this message board who are diagnosed AS know a hell of a lot more about AS than so called "professionals". I'll go further to say that a lot of those here who are NOT diagnosed but who have spent many hours studying AS know more than a lot of "professionals."

Too bad said professionals are the ones who define the disorder -- this ain't an appeal to authority, it's how it is if you wish to be labeled by their definitions.

I agree with those before who said the online tests are good starting point -- I took that wired one when I was first researching autism, my score was high and illuminating, and it lead me to where I am now ("diagnosed").

If its empirically verified, then thats ok....however by modifying the test based on your own opinions you more or less throw the validity out the window.

Right now I'm in the midst of getting a formal diagnosis (and a mild one at that) but my mentor in the department agrees with my self diagnosis, and it was my uncles idea (he's and I/O psychologist) that I may have it. I took Baron-Cohen's tests, and although they are non diagnostic they are probably a better assessment than an online quiz.