Healthy Skepticism Library item: 14688

Warning: This library includes all items relevant to health product marketing that we are aware of regardless of quality. Often we do not agree with all or part of the contents.

Publication type: news

Absence of evidence
The Economist 2008 Nov 27
http://www.economist.com/science/displaystory.cfm?story_id=12675892

Abstract:

Do drug firms suppress unfavourable information about new products?

Full text:

RICHARD FEYNMAN, a Nobel-prize-winning physicist, declared in a speech in 1974 that science requires “a kind of utter honesty”. He insisted that researchers must publicise all the outcomes of their work, and “not just the information that leads to judgment in one particular direction or another”. To judge by the mounting evidence of publication bias involving studies on new drugs, his words have not yet reached the pharmaceuticals industry.

A study published this week in PLoS Medicine, an online journal, confirms what many have suspected and what previous studies have hinted at: drug companies try to spin the results of clinical trials. If this were done merely in marketing materials, it might be tolerable. What Lisa Bero of the University of California, San Francisco, and her colleagues found, however, was troubling evidence of suppression and manipulation of data in studies published in (or often withheld from) peer-reviewed medical journals.

To see if firms massaged data from trials of new drugs, Dr Bero and her team compared the information that companies share with America’s Food and Drug Administration (FDA) about those drugs with what they eventually publish in medical journals. By law, submissions to the FDA must be comprehensive, whereas decisions on whether and how to publish those same data later in journals are governed by self-interest, scientific convention and peer review.

In theory, there should be no difference between what the regulators are told about a new drug’s strengths and weaknesses and what doctors and other researchers read in journals. But Dr Bero’s study found a yawning gap between the two. By looking at all the new-drug applications approved by the FDA in 2001 and 2002, the researchers got hold of a comprehensive set of data. They checked whether, five years later, details on all of those trials had made it into print intact.

The results are distressing. Only three-quarters of the original trials were ever published, and it turned out that those with positive outcomes were nearly five times as likely to be published as those that were negative. Earlier investigations have shown that the explanation is not editorial bias: well-designed studies in which drugs fail have as good a chance of being published in leading journals as those in which drugs succeed.

In the years studied by Dr Bero, 155 primary outcomes appeared in both regulatory filings and in the medical literature. However, the FDA knew about 41 others that never made it into a journal. On top of that, 17 outcomes appeared in publications without having first been discussed in regulatory filings. Fifteen of these 17 made the drugs in question look better. And even when published studies had been filed with the FDA, on several occasions the published conclusions differed from those reported to the FDA in ways that favoured the new drugs.

Industry experts insist there are benign reasons for some of these discrepancies. For example, in the hope of making a name for themselves, overworked researchers may choose to publish successful studies rather than push to get a failed or problematic investigation into print. Sometimes the trials as submitted to the FDA are flawed, so researchers try to fix them before getting them published (though that would not explain why so many of the “fixes” seem to make the drug in question look better).

Dr Bero is unimpressed by such arguments. She points out that many patients enter drug trials believing the results will be published, so that researchers can develop new therapies. Others complain that misleading presentations in journals may lead ill-informed doctors to favour new drugs over cheaper, older ones even if they work no better or have undesirable (but unadvertised) side effects.

Such grouses have resulted in a few changes. The main federal sponsor of biomedical research in America, the National Institutes of Health (NIH), runs a website, clinicaltrials.gov, on which all drugs trials must be registered in addition to being reported to the FDA. And since September the NIH and FDA have published brief summaries of trial results. But critics would like to see more action. An-Wen Chan of the Mayo Clinic, a large medical-research centre in Minnesota, wants the public to have access to the full details of the protocols used to study all new drugs. Dr Bero argues that it is unethical not to publish the full results of medical trials. Feynman would surely have agreed.