Healthy Skepticism Library item: 14563

Warning: This library includes all items relevant to health product marketing that we are aware of regardless of quality. Often we do not agree with all or part of the contents.

Publication type: news

Goldacre B.
Listen carefully, I shall say this only once
The Guardian 2008 Oct 25
http://www.guardian.co.uk/commentisfree/2008/oct/25/medical-research-science-health

Full text:

Welcome to nerds’ corner, and yet another small print criticism of a trivial act of borderline dubiousness which will lead to distorted evidence, irrational decisions, and bad outcomes in what I like to call “the real world”.

So the ClinPsyc blog (clinpsyc.blogspot.com) has spotted that the drug company Lilly has published identical data on duloxetine – a newish antidepressant drug – twice over, in two entirely separate scientific papers.

The first article is from the January 2008 edition of the Journal of Clinical Psychiatry, a study which concludes that the “switch to duloxetine was associated with significant improvements in both emotional and painful physical symptoms of depression”. The second concluded the same thing.

ClinPsyc went through both papers and checked all the numbers in the data tables, finding that they were essentially identical. A few different subscales were reported in each paper, and the emphasis in the second is more on pain than depression, but other than that, this is identical data.

There are several reasons why this is interesting. Firstly, duplicate publication distorts a reader’s impression of how much evidence is out there. If you think there are two trials showing that something works, then obviously that’s much more impressive than if there’s just one. “Of course I prescribe it,” you can hear the doctors say. “I’ve seen two trials showing that it works.”

I got on to the lead author of the paper, who explained that the second paper expanded more on the “pain” aspects of the results. That is slightly fair enough. He also claimed that the second paper referenced the first.

This is true in the strictest sense of the phrase: it did indeed make reference to its existence as a previous experiment, but it gave no indication that this was the same experiment, and for the reader, without going forensic on the numbers, there was no way to know that the data here was all from that previous study, and largely, simply, reproduced. It looked like two studies. It just did.

Duplicate publication can also distort the results of “meta-analyses”, big studies where the results of lots of trials are brought together into one big spreadsheet.

Because then, if you can’t spot what’s duplicated, some evidence is actually counted twice in the numerical results. This is why it is more acceptable to publish duplicates if you at least acknowledge that you have done so. By way of example, I am being clear that I will now rehash a paragraph I wrote several years ago on the work of Dr Martin Tramer.

Tramer was looking at the efficacy of a nausea drug called ondansetron, and noticed that lots of the data seemed to be replicated: the results for many individual patients had been written up several times, in slightly different forms, in apparently different studies, in different journals.

Crucially, data which showed the drug in a better light were more likely to be duplicated than the data which showed it to be less impressive, and overall this led to a 23% overestimate of the drug’s efficacy.

But the other thing to notice about this duloxetine experiment is that its design made the Durham fish oil “trial” look like a work of genius. There was no control group, and it simply looks at whether pain improves after swapping to duloxetine from a previous antidepressant (either instantly, or with a gradual crossover of prescriptions, which might induce a vague sense that one thing is somehow being compared with another).

You don’t need to be a professor of clinical trial methodology to recognise that some people’s pain will improve anyway, under those conditions, regardless of what’s in the pill (and regardless of whether prescriptions are tapered into each other), through a combination of the placebo effect, and the fact that sometimes, in fact quite often, things like pain do just get better with time.

And this might have been a worthwhile study to do if you had good grounds to believe that duloxetine really did improve physical pain in depression – as Lilly has claimed for a while – and you just needed to work out the best dosing regime. But a meta-analysis published earlier this year looked at all the evidence for that claim. Its title is Duloxetine Does Not Relieve Painful Physical Symptoms in Depression: A Meta-Analysis.

Nobody knows how common duplicate publication is in academia. Two days after ClinPsyc published its story The MacGuffin (chekhovsgun.blogspot.com) found an identical story around a different drug. Just from mentioning this story I’ve picked up another from my friend Will in the next room. These are afterthoughts by academics, water cooler comments, but once posted on the internet they become searchable, and notable, and slightly embarrassing.