Applying Trials and Systematic Reviews to Individual Patients

Paul GlasziouCentre for Evidence Based MedicineUniversity of Oxford

Overview

Why is applicability a problem? What are the issues? How can we improve?

• The “5-step” processCheck on the transferability Application to an individual

www.sph.uq.edu.au/CGP/training/CochraneMethodsGroup.html

The problem: The “Leaks” between research & practice

Aware Accept Target Doable Recall Agree Done

ValidResearch

Thomson R BMJ 1998;316:509-13

Variation in Guidelines: % of AF patients “needing” warfarin

The Trialpatients

The Trialreport

The actualpatients

The problem of applying trial results

Should Mr RM buy an electric toothbrush?

72 year old pensioner with Parkinson’s Disease• Has gingivitis and frequent caries

Trials in young healthy folk showing improvements in gingivitis scores but not caries.• Would the electric brush “work” for him?• What should he do?

What did you think?

Osteoarthritis N-of-1s

Comparison of• 1,000mg paracetamol tds• 400mg ibuprofen tds

Two weeks x 6• Outcome diary of pain and

stiffness of target joint

NSAID Paracetamol

Paracetamol NSAID

NSAID Paracetamol

Pair 1

Pair 2

Pair 3

N-of-1: overall & examples

0

2

4

6

8

AVERAGE PAIN

DRUG

PA

IN S

CO

RE

(ME

AN

+9

5%C

I)

Panadol Actiprofen0

2

4

6

8

AVERAGE PAIN

DRUG

PA

IN S

CO

RE

(ME

AN

+9

5%C

I)

Panadol Actiprofen

NSAID non-responder NSAID responder

Interventions: Levels of Evidence

N-of-1 Trial Systematic review of randomised trials A single randomised trial Controlled, non-randomised

• Parallel control• Historical control• Case-control

Case-series

Guyatt, JAMA, 2000

*Controlled Clinical Trials; 10: 151S-160S.

When n-of-1 not possible:The benefit-harm model (Lubsen, Tijssen*)

When does benefit outweigh harm? Assumptions

• Benefit (rate difference) proportional to event rate• Harm constant over event rate

Net benefit = benefit - harm

02468

0 10 20

Benefit

Harm

Transferability and applicability of results

A. TRANSFERABILITY (across groups)1. What are the benefits and harms?2. Is there predictable variation in the

effects?3. How does effect vary with predicted risk?

B. APPLICATION (to individual)4. What are the predicted absolute risk

reductions for individuals?5. Do the benefits outweigh the harms?

Questions so far?

1. What are the benefits and harms?

List all important outcomes • beneficial and harmful

Get best estimate (from meta-analysis)

Summarise in a “clinical balance sheet”

Antibiotics for Acute Otitis Media

For Pain(at 2-7 days)RRR = 28%

C Cates: www.nntonline.net

Clinical Balance Sheet

Outcome % inPlacebo

RRR ARR/100

comments

Pain <1 day 38% 0 0

Pain 2-7days 14% 28% 5 Greater if fever, vomiting

Mastoiditis 0 ? - 1 case in 2,250 (AB grp)

“Glue ear” 3M

26% - -

Adverse effect

11% 55% 5 Vomiting, rash, diarrhoea

All or some responders?

I. Everyone gets small benefit? II. A few get a larger benefit?

OK

Earache

Diarrhoea

2. Are there predictable variations in the effects?

Does effect vary by (PICO)• Patient features, e.g., comorbidity or disease

features, e.g., stage• Intervention features e.g.,

dose/intensity/timing?• Comparator, e.g., placebo, add-on, or active• Outcome measures, e.g., reliability, duration

But beware of artefactual causes• Differences in followup, compliance, measures ,

…

Effect Modifiers for AOM

P – impact greater if fever, vomiting?• Data from 1 trial (Little)

I – no difference between antibiotics O – outcome for pain varies with

time• No impact in 24 hours;

Subgroup Analysis

Does statin work in those with a stroke history (Hx)?

(Circulation. 2001;103:387-392.)

3. How does effect vary with predicted risk?

Is Relative Risk constant across low to high risk groups?• Relative Risk is most often constant• Need to check using:

PlotsHeterogeneity statistics

For biological effect &

transferabilityFor clinical

decision making

Impact of changing risk

BaselineRisk

RelativeRiskReduction

AbsoluteRiskReduction

Numberneeded toTreat

20% 75% 15% 7

8% 75% 6% 16

4% 75% 3% 33

1% 75% .75% 133

Trial patients

Typical patients

Rate versus rate plots

L’Abbe plot of trials of Warfarin in Atrial Fibrillation

Control group rate

Tre

atm

ent

gro

up

rat

e Line of equality

Constant relative reduction

Constant absolute risk reduction

Which risk measure is most constant?

Measure % varying with control group risk

Odds Ratio 13%

Relative Risk 14%

Risk Difference 31%

Analysis of the effect of control rate in 115 meta-analysis Schmid et al Stats in Med 1998: 1923-42.

Possible approaches to applying reviews and trials

1. Inclusion/exclusion criteria• For reviews: overlap or combination?

2. Subgroup analysis• Appropriate methods needed

3. Cross-design synthesis• Combining RCT and “database” evidence

4. 5-Steps of Transferability/Applicability• Benefits versus harms; predicted risk

Glasziou, Irwig BMJ 1995 O’Connell, Glasziou, Hill. NHMRC How to use the

evidence

Clinical Balance sheet From trial to Individual

TrialBalance Sheet

IndividualBalance Sheet

Stability and modifiersOf effects across groups(steps 2 and 3)

Individual featuresAnd risk & values(step 4 & 5)