The trouble with "Research shows"

Post Reply New Topic

In some circles of psychology, there has been a discussion for some time now that fraud in the published results of experimental academic psychology is increasing. The discussions I have been party to have not focused of if, but why.

I think it is probable that the "person in the street" tends to have a fairly strong trust in scientific research by postgraduates. I wince every time I hear the casual claim that "research shows.." What research often doesn't show is how unsound and misused psychological method can be.

In an age where individualism is paramount, the stakes are high: publish or die. And this, I think, is driving the perceived increase in fraud. There are lots of ways to fake results. LOTS.

The Economist, in this article from October 2013 explains a few of the more common ones.

http://www.economist.com/news/briefing/ ... ot-trouble

I hope this will encourage people to think about this. We need to have more critical thinking when looking at each new study that comes along and claims to have some brilliant new insight into diagnosing, controlling or curing autism. It is reported in very gullible ways by the media and frankly, it is depressing me that it is taken as seriously as it is.

Mark Twain said it best: "There are 3 types of lies: lies, damn lies, and statistics."

In a way, I'm glad I didn't get farther than my Master's degree and didn't spend my career in the Ivory towers. What I read in the above-posted article pretty much confirmed my suspicions for plenty of years. Researchers are slicing and dicing their research in order to drag more money out of an entity for a project whose impact is probably quite infinitesimal.

Besides, what passes for research at the academic level nowadays would have had me thrown out of college as an undergraduate some 35 years ago.

Yes. Though if only it was only confined to misuse of statistics. This form of falsification isn't so hard to discover for competent statisticians when they request the raw data and re-analyse it themselves. Missing raw data also speaks very loudly in those circumstances.

Suppression is a huge issue, and more serious issue now:

From Wikipedia, "Scientific Misconduct":

Suppression/non-publication of data

A related issue concerns the deliberate suppression, failure to publish, or selective release of the findings of scientific studies. Such cases may not be strictly definable as scientific misconduct as the deliberate falsification of results is not present. However, in such cases the intent may nevertheless be to deliberately deceive. Studies may be suppressed or remain unpublished because the findings are perceived to undermine the commercial, political or other interests of the sponsoring agent or because they fail to support the ideological goals of the researcher. Examples include the failure to publish studies if they demonstrate the harm of a new drug, or truthfully publishing the benefits of a treatment while omitting harmful side-effects.

This is distinguishable from other concepts such as bad science, junk science or pseudoscience where the criticism centres on the methodology or underlying assumptions. It may be possible in some cases to use statistical methods to show that the datasets offered in relation to a given field are incomplete. However this may simply reflect the existence of real-world restrictions on researchers without justifying more sinister conclusions.

Some cases go beyond the failure to publish complete reports of all findings with researchers knowingly making false claims based on falsified data. This falls clearly under the definition of scientific misconduct, even if the result was achieved by suppressing data" (end of quote).

The American Government set this trend by turning a blind eye and allowing Big Pharma to only report favourable tests. (They still do it and there is no effort to haul them in). Unfortunately, some unscrupulous researchers in other fields took this as encouragement, carte blanche to do the same. However even in the 1980s, this was happening in psychology, though not to the extent that it endangered people's lives, as the Big Pharma cases did.

Also there are certain areas of study that have to justify their existence,
feminist subjects like 'women's studies' and other social justice degrees like 'African Studies'
if the women's studies professors admitted that women were being treated well they would be out of a job.

The catholic girl who did porn (under the name belle knox) to pay her student loan debts studied 'women's studies' and of course couldnt get a job because no one is looking to hire people who have that degree.

The professors have a financial incentive to push a victim agenda and publish papers that condemn men.

Pritty sad OP. I interpret it as a psychologist treating people like sencless experiments (including themselves). This doesn't work.

Thanks for sharing this. I guess I shouldn’t be surprised that the same chicanery and incompetence that exists in the business world also exists in the research world. Sigh.

I suppose, this is one of the reasons why I like to see discussions on research posted in WP. So, we can see the community’s response to the latest research.

Psychology, itself, is a bit flawed in that it is a bit subjective. Every Psychiatrist / Psychologist who reads the DSM does so with a mind from their experiences----which, I think, would be near impossible NOT to, but..... For instance, one Psychologist who had experienced murder in their family would know that someone ELSE who had experienced murder in their family could "get over it" (come to grips with it); whereas a Psychologist who had never had that experience might think that every person's "issues" who HAD that experience stemmed from that experience.

In regard to these "studies"----I, basically, take them with a grain of salt. There MIGHT be something to gain from them..... Maybe, for instance, if there were 5-10 studies on the EXACT SAME thing, one MIGHT be able to gain SOME knowledge in the COMBINED studies; but, so many of them are such a bunch of whooey!! If people who conducted these studies would just talk to people who actually have experience in whatever phenomenon; but, sadly, they think they can figure things out, by reading BOOKS!! Gimme a break, already!!

Statisticians can prove almost anything, a new study finds

"researcher degrees of freedom" - I love that - thanks for brightening up my morning with a laugh

http://newmr.org/blog/most-published-re ... ioannidis/

This professor - who researches research did a meta analysis, working backwards from the studies to determine how many positive findings could plausibly be expected in relation to MRI research on brain imaging.

The answer: even being generous, there were twice as many positive findings reported as could realistically be expected from the amount of data reported on.

corporations fund a lot of research and they have an interest in how the research turns out.

Some people believe in university funded studies....

Autism Speaks has provided some funding to university researchers, including an MRI study which fits well with its agenda, (we have defective "underconnected" brains, Just et al) though that's hardly surprising... the research by others which found exactly the opposite result didn't get the same over the top publicity. What a crock!

http://crackingtheenigma.blogspot.co.nz ... ected.html

The current hype is questioned in the link.

The blog has a lot of other interesting comment from the research fellow who writes it.

That's quite interesting thanks

Oh, yeah----like, "Big Pharma". I wouldn't be a bit surprised if cures have been found for things like certain types of cancer, or diabetes, but the results have been squelched so that the company can continue to peddle its drugs----to the tune of BILLIONS of dollars a year!!

yeah chronic diseases like asthma make loadsa money for drug companies too