Pragmatic Randomised TrialsDavid Torgerson

Director, York Trials Unit

djt6@york.ac.uk

www.rcts.org

Placebo trials

• These are usually seen as the ‘gold-standard’ method of evaluating drug treatments. Because the control patients receive identical looking treatment the effect of being in a trial and the psychological effects (placebo effect) of being in a trial are neutralised.

• This allows us to estimate the true value of a treatment within a patient population.

Placebos are not pragmatic

• A problem with placebo trials is that they are not pragmatic.

• We do not use placebos in routine clincial practice. If there is a ‘true’ placebo effect it is worth paying for and having. Using a placebo removes this potentially worthwhile effect.

Explanatory trials

• Many clinical trials take place in artificial conditions that do not represent NORMAL clinical practice.

• Often trials are EXPLANATORY or MECHANISTIC in that their main aim is to identify biological, physiological mechanisms for how a treatment works.

Explanatory trial outcomes

• Explanatory trials often measure outcomes that are not relevant to the patient (e.g. changes in various blood markers).

• Surrogate outcomes may be wrong: sodium floride increases bone density but also INCREASES fracture.

• They are also undertaken by ‘expert’ clinicians who carefully select patients.

Explanatory View

• How does it work?

• By what biological mechanism can we explain the effects?

• Can be seen as a more robust approach, than qualitative methods, of answering the how or why questions.

Pragmatic Attitude

• Does it work?

• For whom does it work?

• How much does it cost?

• Of secondary importance – how or why does it or does not work?

Removing malaria - Rome

• Romans noted an association between outbreaks of malaria and smell from swamps near to Rome. Decided to drain swamp to remove smell and remove malaria. The treatment worked outbreaks of malaria were reduced as was the smell. The ‘theory’ of how it works was wrong but the treatment was correct.

Selection Criteria – Explanatory Trials

• An Explanatory study will often select patients with very tight clinical characteristics – for instance the same gender, small age range, defined clinical characteristics.

• This makes it possible to reduce response variation and allow inferences of effect from small sample sizes.

Comparators

• Explanatory trials often use the ‘wrong’ comparator (e.g., placebo).

• For most conditions there are existing treatments, what we want to know is whether the new treatment is better than existing care NOT whether it is better than no treatment or placebo.

Generalisability

• Because explanatory studies are undertaken in tightly defined clinical circumstances, and usually use a placebo, they are not very generalisable to routine clinical practice.

• An alternative approach is to use the PRAGMATIC design.

Pragmatic Trials

• In the 1960s Schwarz and Llellouch coined the phrase ‘pragmatic trial’.

• In a pragmatic trial the design mimics as closely as possible ROUTINE clinical practice, with the exception that patients are randomly allocated to treatment.

Advantages of pragmatic trials

• An advantage of the pragmatic approach is that because placebos are not used and EFFECTIVENESS is estimated.

• Because conditions mimic routine clinical practice this makes the results more applicable to the ‘average’ patient.

Antibiotics for sore throats

• The Little trial was a 3 armed trial of: immediate antibiotics; no antibiotics or delayed antibiotics. It was pragmatic BECAUSE:» Set in primary care where most sore throats are dealt

with;» Used ‘bog standard’ GPs;» Did NOT use placebos;» Outcome was clinical severity from patient NOT

microbial swabs.

Prevalence of Sore Throat

01020

3040

5060

708090

100

Antibiotics None Delayed

Antibiotics = 37%; None = 35%; Delayed = 30%

Proportion of patients better by day 3 (P=0.28)

Antibiotics

• Little et al showed that immediate use of antibiotics for a sore throat had NO significant effect on the resolution of symptoms

• A placebo trial would demonstrate this BUT this trial also showed that those who did not get antibiotics were less likely to visit their GP again

Re-attendance within 12 months

0

5

10

15

20

25

30

35

40

Antibiotics None

Odds Ratio = 1.39 (1.03 to 1.89)

Antibiotics = 38%; None = 27%

Pragmatic information

• GPs prescribing antibiotics for uncomplicated sore throats has very little, if any, effect

• Patients prescribed antibiotics are more likely to re-attend when they next have a sore throat compared with giving no antibiotics

Cranberry Juice for urinary tract infection

• Avorn et al randomised elderly women to receive cranberrry juice or a placebo.

• Outcomes were microbiological (I.e., bacterial counts in urine samples).

• Result – Cranberry juice significantly reduced bacteria in the urine.

• SO WHAT – what we need to know is whether it reduces symptoms.

Cranberry again.

• Kontiokari et al randomised a group of young women (mainly students) to an ‘open’ trial of cranberry, lactobacillus (yakult type of drink) or open control.

• Outcome was time to recurrence of urinary tract symptoms (e.,g., pain on passing urine, flank pain).

• Infection confirmed with swabs.

Cranberry results

Cranberry conclusion

• First trial was suggestive in that it showed an effect on a SURROGATE outcome measure of urinary tract infection.

• The second trial was more definite – cranberry supplementation reduces symptoms of urinary tract infection in young women.

Pragmatic with explanatory features

• Many, if not most, pragmatic trials will also include features of explanatory studies. We might measure blood pressure, cholesterol or bone mass, to elucidate mechanisms to explain why or why something did not work. A RCT of vitamin D injection measured a PTH levels to check the treatment was being absorbed. This information can be collected cheaply from pragmatic trials to enhance their scientific value.

Summary

• Explanatory trials follow the tradition of very tight control of sample – difficult to generalise from results.

• Pragmatic study much more generalisable interested in EFFECTIVENESS not too interested in how things work.

• Economic evaluations are best conducted using pragmatic studies.

References

• Avorn et al. Reduction of bacteriuria and pyuria after ingestion of cranberry juice. JAMA 1994;271:751-4.

• Godwin et al. Pragmatic controlled clinical trials in primary care: the struggle between external and internal validity. BMC Medical Research Methodology 2003;3:28.

• Helms PJ. ‘Real world’ pragmatic clinical trials: what are they and what do they tell us? Pediatr Allergy Immunol 2002;13:4-9.

• Kontiokari et al. Randomised trial of cranberry-lingonberry juice and Lactobacillus GG drink for the prevention of urinary tract infections in women. BMJ 2001;322:1571.

References

• Little et al. Reattendance and complications in a randomised trial of prescribing strategies for sore throat: the medicalising effect of prescribing antibiotics. BMJ 1997;315:350-2 (and BMJ;314:722-7).

• MacPherson H. Pragmatic clinical trials. Comp Ther Med 2004;12:136-40.

• MacRae KD. Pragmatic versus explanatory trials. Int J Technol Assess Health Care 1989;5:333-9.

• Medical Research Council. A framework for development and evaluation of RCTs for complex interventions to improve health. April 2000. http://www.mrc.ac.uk/index/publications/publications-ethics_and_best_practice/publications-clinical_trials_guidelines.htm

References

• Pocock SJ (1983). The Justification for Randomized Controlled Trials. In Clinical Trials - a practical approach. John Wiley & Sons, Chichester. p.182

• Riggs et al. Effect of fluoride treatment on the fracture error rate in post-menopausal women with osteoporosis. N Engl J Med 1990;322:802-9.

• Roland M, Torgerson DT. Understanding controlled trials: What are pragmatic trials? BMJ 1998;316:285.

• Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutic trials. J Chron Dis 1967;20:637-648.

• Thompson SG, Barber JA. How should cost data in pragmatic randomised trials be analysed? BMJ 2000;320:1197-1200.