Healthy Skepticism International News

Newsletter May/Jun 1998

June 1998

Vol 16 Issue 5/6 News A Website for MaLAM Reports Applying clinical trial outcomes to the individual patient by David Sackett Communicating about prevention by Peter Mansfield

A Web site for MaLAM

MaLAM now has a Website. The correct URL is: www.camtech.net.au/malam

Our site is located on the Camtech and Flinders University servers. The Australasian Cochrane Centre (ACC) based at Flinders University, South Australia, has given MaLAM access to the University server free of charge. As a way of saying thank you this edition features the transcript of a talk about evidence based medicine. The talk was given by Professor David Sackett at the UK Drug and Therapeutics Bulletin’s 1996 “From trial outcomes to clinical practice” seminar. The seminar report deserves promotion. Copies are available for £12.00 plus postage from Which? Ltd, 2 Marylebone Rd, London NW1 4DF, UK, fax +44 171 850 7665. We thank the DTB and Prof Sackett for giving permission for the talk to be reprinted.

Professor Sackett’s talk is relevant to MaLAM because of his insights on how to communicate about the evidence from trials in ways that will assist front line health workers to provide appropriate therapy. He also describes the realities of how difficult it is for we health professionals to keep our knowledge up-to-date. The reasons help explain why we are so vulnerable to misleading promotion.

Thanks are due to the following for contributions to the Web site:

Adelaide: Robyn Clothier, Peter Mansfield, Libby Roughead, Lynn Telfer, Agnès Vitry

Amsterdam: Mark Raijmakers

Paris: Ellen ‘t Hoen

Perth: Brett Montgomery

Toronto: Joel Lexchin

Vancover: James McCormack

and also to David Badger of the ACC and Bellinda
Allomes of Camtech for technical assistance.

MaLAM has signed a US $ 15,000 contract with the World Health Organization’s Drug Action Program for the production of teaching materials for critical appraisal of pharmaceutical promotion. These will be published as a special section of our Website and as a slide tape show for use in areas where access to the Internet is difficult or impossible.

“What exactly is the healthy dose of scepticism?”

Applying clinical trial outcomes to the individual patient

Professor David Sackett

How does the prescriber integrate external evidence with internal clinical expertise and patients’ values (evidence-based medicine) in order to determine whether and how trial results apply?
Is it possible to generate clinically useful measures of the effects of treatment such as the number of patients who need to be treated to prevent one bad outcome?
How should one hone the skills necessary for tracking down and critically appraising the best external evidence?
Can one conduct randomised trials in individual patients?
Here we consider these questions and the application of trial data to the individual patient.

Integrating external evidence with internal clinical expertise

This is what evidence-based medicine is all about. This simply consists of making decisions about the care of the individual patient using the current best evidence. In other words, an attempt to integrate both individual clinical expertise and the best available external clinical evidence from systematic research.

Internal expertise is the proficiency in judgement that individual clinicians acquire through clinical experience in clinical practice. This is reflected in more efficient and effective diagnosis and through the more thoughtful identification and compassionate consideration of individual patient’s predicaments, rights and preferences in making clinical decisions about their care.

External clinical evidence is clinically relevant research. Increasingly, it incorporates patient-centred research into the accuracy and provision of diagnostic tests, including particularly the clinical exam, the power of prognostic markers and the efficacy and safety of therapeutic, rehabilitative and preventive regimens. The external clinical evidence has a short doubling time and, although sometimes it confirms what we are now doing, it frequently invalidates previously accepted diagnostic tests and treatments and replaces them with new ones that are more powerful, more accurate, more efficacious and safer.

Good doctors use both individual clinical expertise and the best available external evidence in making decisions, because neither one is going to be sufficient by itself. Without internal expertise, practice risks becoming evidence tyrannised, because often, excellent external evidence may be inapplicable or inappropriate for individual patients. Without external expertise, there is risk of practice going rapidly out of date to the detriment of patients and patient care. So we need to try to integrate the sort of evidence gained from good randomised controlled trials into discussions and interactions with our patients in order to try to help them make decisions.

Clinically useful measures of the effects of treatment

Can we generate clinically useful measures that would permit us to integrate external evidence with individual internal clinical expertise? An ideal measure of clinical efficacy might have several properties. It might compare the consequences of doing nothing with the potential effects of doing something. It would summarise the harm that accompanies the treatment as well as its benefits. It would identify patients who are both at high risk for an event and responsive to therapy, i.e. the people for whom the payoff is going to be the greatest. It would permit a comparison of the consequences of applying one therapy for a disorder with the consequences of applying other therapies for other disorders in terms of how does one contrast the social benefits of drug A for treatment X versus behavioural treatment Y for condition Z.

One example is the occurrence of death, stroke or other major complications including the unwanted effects of therapy for individuals with moderate hypertension. Among individuals with prior target organ damage receiving placebo tablets, death, stroke or other major complications would occur in about one-fifth within the next few years. This can be reduced to less than one tenth with active drug therapy, even with old-fashioned drugs. Patients without prior target organ damage to the eye, heart, brain, kidney, major vessels are at about equal risk as those with target organ damage who receive interventions but their risk can also be lowered, [i.e. lowered further], with antihypertensive drugs.

The question is how can this type of information be expressed in a clinically useful way. The standard approach has been to generate a proportional or relative risk reduction by looking at the absolute risk reduction and then expressing it as a proportion of what would occur without the intervention, 64% in the above example. The trouble with this is that it gives information about the therapy but not about patients . Indeed if this were a trivial condition and the risk of getting into trouble on placebo was 0.0001 and the risk on active therapy was 0.00004 there would still be a relative risk reduction of 60%. The next approach would be to take a public health approach and simply look at the numerator of the relative risk reduction, the absolute risk reduction, that preserves both the treatment effect and the risk among those individuals who do not get treated [the initial level of risk]. This method would generate different numbers, 0.14 and 0.06, but the trouble with these is that they are hard to remember. But dividing the absolute risk into one gives a whole number and indeed shows the number of individuals that need to be treated in order to prevent one event. So for people with moderate to severe hypertension and target organ damage only seven would need to be treated for a few years to prevent one of them having a stroke, dying or having another major complication, whereas 17 without target organ damage would need to be treated to accomplish that same sort of result.

How might skills be honed to find the best external evidence?

It has been estimated that each general physician should examine at least 19 articles a day for 365 days a year to keep abreast with his or her field. When doctors are asked how often they need information they say about twice a week, sourcing it in text books and journals. When actually shadowed coming out of examining rooms they need [information] up to 60 times a week, about twice for every three patients seen. The information often comes from their clinical colleagues and when asked why they do not use text books doctors point out that they are out of date. When asked why they do not use journals they admit that they are so disorganised that even if they knew the citation they would not be able to find the article. It is probably OK not to use text books because they tend to not recommend therapy for up to 10 years after it has been shown to be efficacious, and continue to recommend therapy for about 10 years after its been shown to be useless or harmful. A survey of reading times at health sciences centres in the UK has found that 75% of house officers in their first post graduate year do not read around their patients at all from one week to the next. Older hospital doctors read for about 30 minutes a week. General practitioners have even more trouble reading because they have to travel back and forth to the library. In general they spend about twice as much time travelling as they do reading.

It is not surprising, then, that when doctors are asked simple questions about the treatment of hypertension their scores deteriorate as a function of years since graduation from medical school. In fact, whether or not high blood pressure gets treated has less to do with target organ damage to the patient’s eyes, heart, brain, kidney or major vessels than it has to do with how many years it is since the treating clinician graduated from medical school.

One approach to the problem of lack of reading time is to translate information needs into answerable questions, making it easy to track down the best information, then critically appraising a research report rapidly for two things: “is it true?” and “is it useful?” A study did this in general medicine at a district general hospital and instead of the usual figure of 20% of the health care being evidence-based, slightly more than half was based on randomised trials or overviews of trials. In addition, medical students who learn this strategy for learning (as opposed to simply memorising and jumping hurdles) have been shown not to deteriorate.

Randomised trials in individual patients

Often there are no trials to guide us. Or the patient might not have met the eligibility for a trial and there is a concern about extrapolation. Or in positive trials the experimental therapy might not work in everyone and, indeed, in negative trials the experimental therapy often appears to work in some folks. The problem is that if an open trial of therapy is started in a patient it can frequently be misleading. The illness may have run its course, and would have improved anyway whatever the treatment given in the interim. Extreme symptoms, signs and laboratory test results tend to regress towards the mean.

The unblinded assessment of treatment responses means that our unconscious bias may influence our evaluation of results and patients tend to be very grateful or at least polite. However, it is possible to conduct an “N-of-1 randomised controlled trial”, i.e. a trial done in a single patient. For example, in one study theophylline and placebo were compared in an asthmatic patient with terrible symptoms. He was given treatment periods of ten days. These were randomised into pairs of treatment periods consisting of active treatment or placebo in a blind fashion. The effect of treatment was regularly monitored and the code was not broken until the patient wished it. In this particular patient’s case, the theophylline was stopped and another trial was carried out for his puffers, which showed him and us that in fact his puffers were quite worthwhile. We have done about 100 such trials looking at the sorts of treatment decisions that we would make and quite frequently they lead to the medication being stopped.

There is an attractiveness to trials of one patient. Every patient receives active treatment. N-of–1 trials seek the best treatment for a specific patient not for the average patient and permit direct patient involvement in selecting treatments. Thus there is no need to wait for a randomised controlled trial.

Conclusion

When possible, treatment should be based on the results of clinical trials. Often, however, trial data are not readily available or not known to the prescriber. In addition, the trial may be designed or the results presented in ways that are difficult to relate to an individual patient. In these situations trial-based prescribing is not feasible. In some circumstances, and after assessing the published data, it may be reasonable (and useful) to do a randomised controlled trial in an individual patient, an ‘n-of-l’ trial. The advantages of this approach are that it is relatively simple and quick to do, is relevant to the particular patient, and avoids the need to wait for the results of a conventional RCT.

Communicating about prevention
Dr Peter Mansfield

“What is simple is false and what is not is useless.”

– Paul Valery

I agree with Prof Sackett that the “number needed to treat” (NNT) is a much better way to communicate about prevention therapies than the relative risk reduction (RRR). In Prof Sackett’s example of hypertension without target organ damage above, the RRR was 64% whereas the NNT was 17. Drug advertisements often use the RRR because it gives a much more positive impression. A number of studies have found a significant difference between prescribing intentions depending of whether the RRR or the NNT was presented. However, I wonder if the NNT may also mislead. Also the NNT concept seems to start from the output of current controlled trial reports rather than starting from what health professionals need to know. As Prof Sackett said we ned to be able to “compare the consequences of doing nothing with the potential effects of doing something.”

I will modify Prof Sackett’s hypertension example a little to make the calculations easier. My example will use an imaginary randomised controlled trial (RCT) of Therapy X for hypertension. At 5 years 10 of 100 subjects in the control group and 5 of 100 subjects in the therapy group had a myocardial infarct. To keep things simple I will assume no other benefits and no adverse effects.

The question is: How should we communicate about Therapy X? Drug company promotion would use the RRR method:

“Therapy X reduces the risk by 50%.”

That is simple, true and misleading.

The NNT is less misleading because it gets closer to reality by incorporating allowance for the initial risk (IR). The IR is the risk of the outcome that you hope to prevent.

If the IR for heart attacks is 1 in 2 in a very high risk patient then with a RRR of 50% for Therapy X the NNT is 4. Therapy X is likely to be judged worth the money and the adverse effects. By contrast, if the IR is only 1 in 2,000, in a low risk patient, then the NNT is 4,000. This will not impress doctors and patients as much even though the RRR is still 50%. (With some therapies the RRR may not be stable. If the RRR is lower with low risk patients then the NNT will be even higher.)

The IR is sometimes assumed to be the rate at which the outcome occurred in the placebo group for a controlled trial. However, placebos may have dramatic effects. Also, an individual patient’s initial level of risk may be much higher or lower than that of the people who entered the trial. If the IR is lower then the NNT is higher. This indicates that the therapy is less worthwhile. For example, treating hypertension provides less benefit for slim young women who do not smoke or have diabetes because they have a much lower risk of suffering any of the complications of hypertension than other people. Such women have a low IR and thus a high NNT.

Using the NNT method to communicate about my imaginary RCT I could tell patients with the same IR as those in the trial:
“If I treat 20 people like you for 5 years with Therapy X then I will prevent one heart attack.”

The problem is that patients may interpret this more literally than Prof Sachett intended. They may take this statement to mean that one patient will receive a large benefit and 19 patients will receive no benefit. The reality is more complex. Is there an acceptable, reasonably simple way to communicate information about efficacy that would enable greater understanding of the complexity?

It is true that 18 of 20 patients (90%) will not benefit during 5 years on Therapy X. They could not because they would not have had a heart attack during that time anyway. However, they may benefit later if they would otherwise have had a heart attack later.

I will now focus on the 2 in 20 hypertensives who would have a heart attack within 5 years if they were not treated, and therefore have a potential to benefit from treatment. I will call them Mr A and Mr B.

Communicating with the NNT could be misunderstood to mean that only one of those 2 gets all the benefit (a heart attack “prevented”) and the other one gets no benefit and thus has a major complication. However it is unlikely that the effect of therapy is “all or nothing” prevention. Therapy X could merely delay heart attacks. Therapy X could produce the results found in my imaginary RCT merely by delaying heart attacks for 5 subject just beyond the 5 year study period rather than completely preventing them. However, Therapy X may sometimes delay attacks long enough to enable death to occur from another cause. Additionally, perhaps the benefit from Therapy X is shared. Let us assume that the effect of Therapy X is to delay major heart attacks by 50% of the time the therapy is taken, ie 6 months for every year it is taken. Let us also assume that without therapy Mr A was destined to have a major complication one year into the 5 year treatment period and Mr B was destined to have a major complication 4 years into the 5 year treatment period. With treatment Mr A now has his major complication at 1 year 6 months and Mr B has his at the 6 year mark. Both have shared the benefit from therapy, but neither has benefited as much as could be misunderstood from the NNT method of communicating about Therapy X.

I would like to suggest another way to communicate about prevention therapy - the “Initial Risk plus Outcome Delay” (IROD). It is less simple but, with the Paul Valery quotation in mind, I hope it is less misleading without being too complex to be useful. It is based on my need as a GP to be able to inform my patients about what may happen without therapy vs with therapy so that we can make a choice. Using IROD I could tell my hypertensive patients about Therapy X as follows:

“Without therapy you have about a 1 in 10 risk of a heart attack during the next 5 years.

If without therapy, you would have a heart attack later than 5 years from now, then Therapy X may delay it. However we do not know what the effects are in the long term because that has not been studied yet.”

It is interesting to compare the IROD method with the RRR and NNT methods:

RRR: “Therapy X reduces the risk by 50%.”

NNT: “If I treat 20 people like you for 5 years I will prevent one heart attack.”

Many industry staff would like us to use the RRR method because communicating that way will increase “compliance” even amongst people with minimal chances of receiving any benefit because of a low initial risk. They might conclude that therapy was not worth the cost if they were better informed. That would increase the resources available for more cost-effective therapy.

Before we can use IROD, we health professionals need to be better informed ourselves. We need adequate IR calculators and we need clinical trials of prevention therapies to be analysed differently so as to tell us about outcome delays.

I thank Anne Holbrook for her comments on an early draft. I would very much appreciate readers’ comments on the IROD concept and on communication about therapy in general.

View/Hide References

HS Int News index

Comments

Our members can see and make comments on this page.