Healthy Skepticism
Join us to help reduce harm from misleading health information.
Increase font size   Decrease font size   Print-friendly view   Print
Register Log in

Healthy Skepticism Library item: 19591

Warning: This library includes all items relevant to health product marketing that we are aware of regardless of quality. Often we do not agree with all or part of the contents.


Publication type: Journal Article

Vollenweider D, Boyd CM, Puhan MA
High prevalence of potential biases threatens the interpretation of trials in patients with chronic disease
BMC Medicine 2011 Jun 13; 9:73


The complexity of chronic diseases challenges investigators conducting randomized trials. The cause for this is the often more difficult control for confounding, the selection of outcomes from many potentially important outcomes, the risk for missing data with long follow-up or the detection of heterogeneity of treatment effects. Our aim was to assess such aspects of trial design and analysis for four prevalent chronic diseases.

We included 161 randomized trials on drug and non-drug treatments for chronic obstructive pulmonary disease, type II diabetes mellitus, stroke and heart failure that were included in current Cochrane reviews. We assessed whether these trials defined a single or several primary outcomes, statistically compared baseline characteristics to assess comparability of treatment groups, reported on between-group comparisons and we assessed how they handled missing data and whether appropriate methods for subgroups effects were used.

We found that only 21% of all chronic disease trials had a single primary outcome while 33% reported one or more primary outcomes. Two out of the 51 trials that tested for statistical significance of baseline characteristics adjusted the comparison for a characteristic that was statistically significantly different. 10% of trials reported a within-group comparison only. 17% (n=28) of trials reported how missing data were handled: 50% (n=14) carried forward last values, 27% (n=8) did a complete case analysis, 13% (n=4) used a fixed value imputation and 10% (n=3) used more advanced methods. 27% of trials performed a subgroup analysis but only 23% of them (n=10) reported an interaction test. Drug trials, trials published after wide adoption of the CONSORT statement (2001 or later) and in journals with higher impact factors were more likely to report on some of these aspects of trial design and analysis.

Our survey showed that an alarmingly large proportion of chronic disease trials do not define a primary outcome, do not use appropriate methods for subgroup analyses or use naive methods to handle missing data, if at all. As a consequence, biases are likely to be introduced in many trials on widely prescribed treatments in patients with chronic disease.


  Healthy Skepticism on RSS   Healthy Skepticism on Facebook   Healthy Skepticism on Twitter

Click to Register

(read more)

Click to Log in
for free access to more features of this website.

Forgot your username or password?

You are invited to
apply for membership
of Healthy Skepticism,
if you support our aims.

Pay a subscription

Support our work with a donation

Buy Healthy Skepticism T Shirts

If there is something you don't like, please tell us. If you like our work, please tell others.

Email a Friend

Cases of wilful misrepresentation are a rarity in medical advertising. For every advertisement in which nonexistent doctors are called on to testify or deliberately irrelevant references are bunched up in [fine print], you will find a hundred or more whose greatest offenses are unquestioning enthusiasm and the skill to communicate it.

The best defence the physician can muster against this kind of advertising is a healthy skepticism and a willingness, not always apparent in the past, to do his homework. He must cultivate a flair for spotting the logical loophole, the invalid clinical trial, the unreliable or meaningless testimonial, the unneeded improvement and the unlikely claim. Above all, he must develop greater resistance to the lure of the fashionable and the new.
- Pierre R. Garai (advertising executive) 1963