“If enough data is collected, anything may be proven by statistical methods”
It’s an amazing day for scientific research! Hold on, don’t leave me yet. I know I typically try to post things here that are inspiring or at the very least interesting and directly applicable to your everyday life. And the behind-the-scenes politics and procedures of conducting scientific research generally wouldn’t be thought to fall into the “inspiring” category. But trust me, this matters to you!
The All Trials Campaign has organized experts from around the world who are demanding that all unpublished data from clinical medication trials be published and all misreported data be formally corrected. Just this month, the British Medical Journal (BMJ) and PLOS Medicine have taken up the call of the “Restoring Invisible and Abandoned Trials” initiative (RIAT), endorsing the proposal that sponsors and researchers begin publishing the results of their previously confidential clinical trial documents within one year. If they fail to take these actions, RIAT would call for independent scientists to publish those previously confidential trial documents.
To understand why this is such a potentially momentous move, it’s helpful to first understand a bit about how the world of scientific publishing works. At the heart of the problem is the fact that, in general, only studies that find “significant” results get published, and here the word “significant” means that the study found that the particular drug/intervention/treatment being tested was effective. On the surface this practice seems to make sense. I mean, would you sit down to read a newspaper that had a bunch of titles like “Nothing at all happened in Portland last night” and “Nobody did anything of significance in Congress yesterday” (ok, well, maybe that one doesn’t seem like a stretch!). Those of us who read journal articles to get our news about the latest developments in our field want to spend what little time we have reading articles about treatments that actually seem to work. We’re generally less interested in studies that fail to find that a particular treatment works. The result is that studies showing that a treatment doesn’t work, or worse, caused harm, are often unpublished.
However, the problem with state of affairs is that it gives health care providers and the public very skewed information. For every study we hear about that shows a particular drug or treatment supposedly works, we never know about the potentially countless other studies that showed that it didn’t work, or even that it caused harm. And the picture gets even more worrisome when you take into account how most research is funded in the first place.
The vast majority of scientific researchers are only able to do their work through grant funding (though we have a different model here at Portland Psychotherapy for funding our research which you can read more about here). One way this happens is that a researcher, who is very interested in a particular treatment, spends months writing and rewriting a grant application to ask some institution, such as the National Institutes of Health (NIH), essentially asking them for money to study their idea. But even more frequently, it isn’t the NIH or some other arguably unbiased institution that is funding research. As funding from places like the NIH have dropped drastically in recent years, “industry funded research” (e.g. research paid for by a company that is highly invested in its outcome) has soared, with industry-funded research in universities increasing 250% from 1985 to 2005. Increasingly, researchers are paid by a particular company, often a big pharmaceutical company, who has a vested interest in showing that their product (e.g., their drug) is effective.
Now let’s return to the problem of only publishing “significant” findings. If only those studies that show a “significant” result (e.g. that the drug “Y” was more effective than placebo) are going to be published, the company has every incentive in the world to just keep funding study after study until they finally get one that shows the result they want, not because it is a real result, but because of the natural variation and error that is part of research. And these companies have the deep pockets to do that. So theoretically, they could fund 100 clinical trials and even if they only found a “significant” result in 1 out of 100 studies they ran, that one “significant” finding gets published in a journal, health care providers read about it, the press picks up on it, there are ads in magazines touting the positive findings, and now it’s the new wonder drug. However, the 99 other studies showed that drug “Y” was ineffective were never published.
From a consumer standpoint, would you purchase something if the advertisers told you that 99 times out of 100 it was shown to be completely ineffective? No, we’re more likely to buy (or in the case of health care providers, prescribe them to our patients) products when they are backed by claims like “Clinical studies prove that drug “Y” significantly reduced symptoms of X”. What the RIAT initiative will do is give us a more complete picture so that we can know about the studies that showed that a drug or other product was harmful or ineffective, versus only hearing about the studies that happen to work out.
Unfortunately, the RIAT initiative doesn’t have the ability to force drug companies or researchers to publish their negative findings. However, it does shine light on this incredibly important issue and, if the public demands it, will put new pressure on researchers and the industries to commit to making ALL their data available. This will allow researchers do what they are meant to do, be scientists, rather than being PR machines for companies with very deep pockets.
If I’ve convinced you here that this issue really does impact you and you’d like to read more about this problem of only publishing “significant” findings, you can read this great, in-depth article on the topic published in Scientific America.
You can also sign the petition to support the All Trial Registry here.
Author: Jenna LeJeune, Ph.D
Jenna LeJeune, Ph.D. is co-founder and President of Portland Psychotherapy Clinic, Research, and Training Center in Portland, Oregon. As a clinical psychologist, Jenna specializes in working with clients struggling with relationship difficulties, including problems with intimacy and sexuality, trauma-related relationship challenges, and struggles people have in their relationship with their own bodies. She is the co-author of the forthcoming book, “Values in Therapy: A Clinician’s Guide to Helping Clients Explore Values, Increase Psychological Flexibility, and Live a More Meaningful Life.” Jenna is also a peer-reviewed ACT trainer and provides ACT trainings to professionals around the world.
