Subscribe X
Back to Top

Learn

July 5, 2013 by Jeffrey Lacasse, PhD

Saving Science: It’s Time to Solve Publication Bias

Erick Turner has published an important piece in CNS Drugs entitled “Publication Bias, with a Focus on Psychiatry: Causes and Solutions.” It should be required reading for any medical student, non-medical helping professional, or practicing prescriber. Rather than being an unbiased reflection of the underlying data, the psychiatric literature is instead sort of a funhouse mirror – the published results are consistently, overwhelmingly, positive – no matter what the actual data are. The implications are enormous. Note that the methodology in such trials is often deliberately constructed to bias such trials in favor of the sponsor’s drug – even so, publication strategies are used to minimize any bad news resulting from such trials, and to maximize the impression that the drug is effective.

Professionals that read these studies in the peer-reviewed literature (or systematic reviews based on these studies) will likely believe the results, and adopt biased impressions of psychiatric medications – they will believe them to be more effective, and less risky, than the actual data show them to be. When it comes time to inform their patients about treatment options, they will do their best to accurately inform their patients … but physicians exposed to biased literature are likely to then misinform their patients. The misinformation will almost inevitably be in the direction of lower thresholds for treatment and higher efficacy/lower risk for psychiatric medications.Turner is correct in writing that these problems are in science across the board, but the potential for real-world harm is much higher in medicine in general and psychiatry in particular. There also seem to be different levels of awareness of these problems across scientific fields. I have had many conversations with social scientists who express awareness of the problems with publication bias, and are concerned – they often see the issues as a problem that must be solved over time. However, anecdotally I also hear of these same social scientists going to their prescriber and expressing concern about the evidence base in medicine and/or psychiatry – having read articles like Turner’s, and having some understanding of the challenges of making clinical decisions based on experimental evidence. Apparently, in many cases these conversations do not go well.

Having been trained in evidence-based medicine and in the primacy of randomized controlled trial data, many prescribers are apparently made very uncomfortable by the questioning of the paradigm. And one can easily imagine a clinician reading Turner’s article and then feeling completely overwhelmed…if Turner’s analysis is valid (I believe it is), the ramifications are so enormous that they may be difficult for the average clinician to believe. Human beings are justificationary creatures and physicians are no different. One strategy for addressing the problem would be to integrate this material into medical schools as soon as possible so that emerging physicians are well-trained on these issues. Of course, this is easier said than done, and could create significant angst among medical students as they try to reconcile a curriculum focused on clinical trial evidence with the unfortunate reality of selective and biased reporting.

So, what to do, moving forward? Turner focuses on problem-solving at the level of regulatory authorities, authors, and journals. His suggestions are all thoughtful, and most are realistic. Here I would like to close by repeating some more radical proposals that I believe would help solve the problem of publication bias (and associated problems) in psychiatric research. If academic medicine is actually interested in solving the deep scientific problems in the psychiatric literature, it could:

  1. Admit that medical journals “have devolved into information laundering operations for the pharmaceutical industry” (Richard Horton, Lancet editor) and adopt a new mission for medical journals in which they only publish scientific criticism of trials (Richard Smith, BMJ editor).
  2. Ban ghostwriting of trials and enforce the same authorship norms on academic psychiatrists as researchers in psychology or economics.
  3. Encourage dissent and transparency in journal publications. When publishing a clinical trial in an academic journal, provide patient-level data to a researcher with a dissenting point of view and allow them to publish a brief summary following the article. Deliberately subject the underlying data to a rigorous, public critique. Put any statistical tricks or changes in endpoints (which Turner explains well) on display and allow the reader to make their own decisions regarding the validity of the trial results.
  4. Support efforts for data-sharing of unpublished patient-level data. Realize that sociologists and philosophers of science (and increasingly, the general public) will always be skeptical of any claims made where the underlying data are not available for re-analysis. Change scientific norms so that data unavailable for external re-analysis is simply not taken seriously.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.


Related Blogs

  • Dr. David Healy

    Dr. David Healy

    Dr. Healy is a professor of psychiatry at Cardiff University in Wales and an author on the history of pharmaceuticals and government regulation.
    READ BLOG
  • Mad In America: Robert Whitaker

    Mad In America: Robert Whitaker

    Journalist and author Bob Whitaker distills the latest in pharmaceutical and mental health research.
    READ BLOG
  • Selling Sickness

    Selling Sickness

    Creating a new partnership movement to challenge the selling of sickness.
    READ BLOG
  • Kathy Brous

    Kathy Brous

    A serial of Kathy's recovery journey as an adult with attachment disorder.
    READ BLOG
  • Nev Jones

    Nev Jones

    Exploring the intersections of psychiatry, philosophy, neuroscience, cultural theory, critical community psychology and the mad/user/survivor movement.
    READ BLOG
  • 1boringoldman

    1boringoldman

    Retired psychiatrist and raconteur offers insightful analysis of the day's events from the woods of Georgia.
    READ BLOG