03.11.2011

1

Holger Schünemann, MD, PhD Chair and Professor, Department of Clinical Epidemiology & BiostatisticsProfessor of MedicineMichael Gent Chair in Healthcare ResearchMcMaster University, Hamilton, Canada

Evidence-based medicine

Tartu UniversityOctober 14, 2011

Disclosure

• Co-chair GRADE Working Group

• Work with various guideline groups using GRADE

– American College of Physicians (ACP) Clinical Practice

Guidelines Committee

– American College of Chest Physicians (ACCP)

• World Health Organization: Advisory Committee for

Health Research, Expert Advisory Panel on Clinical

Practice Guidelines and Clinical Research Methods and

Ethics & Chair of various guideline panels; funding for

guideline development

• No direct/personal for profit payments

03.11.2011

2

Today’s presentation

Brief history and

context of EBM

Evidence based

decision making

Evidence to action =

recommendations

Today’s presentation

Brief history and

context of EBM

Evidence based

decision making

Evidence to action =

recommendations

03.11.2011

3

History

- 1967 – Founded by David Sackett

- 6 chairs since

- Instrumental in specialty of Clinical

Epidemiology, INCLEN

- Birthplace of “Evidence-Based Medicine”

People

45 full time and joint faculty

~ 120 associate & part time faculty; 19 emeritus

~ 180 staff

~ 200 PhD and Master students

Faculty of Health Sciences amongst top 50 in the world

The Department of Clinical

Epidemiology & Biostatistics at McMaster

03.11.2011

4

Questions and answers

Let’s go back in time. You are in high school. The teacher asks a question and you do not know the answer.

What are you feeling?

– fear

– resentment

– shame

– anger

How do you react?

– avoid eye contact

– answer a different question

– guess

– answer: I do not know!

Questions and answers

Let’s try again…You are a resident (physician in

training). You are asking the attending physician

(consultant) a question…

– What is the attending feeling?

– How will s/he react?

03.11.2011

5

Message

As much as EBM is about asking

questions, to practice it we need to look

at our attitudes when responding!

03.11.2011

6

No

t M

en

tio

ne

d

Ro

uti

ne

Ex

pe

rim

en

tal

Ra

re/N

eve

r

Sp

ec

ific

Textbook/ReviewRecommendations

0.5 1.0 2.0

Favors Treatment Favors Control

P < 0.01

P < 0.001

P < 0.00001

21

5

1 10

1 2

2 8

7

8

1 12

1 8 4

1 7 3

5 2 2 1

15 8 1

6 1

2121

55

11 1010

11 22

22 88

77

88

11 1212

11 88 44

11 77 33

55 22 22 11

1515 88 11

66 11M

M

M

M

M

M

Year RCTs

1960 1

2

1965 3

1970 4

7

10

11

15

17

22

1980 23

27

1985 30

33

43

54

65

1990 67

70

YearYear RCTsRCTs

19601960 11

22

19651965 33

19701970 44

77

1010

1111

1515

1717

2222

19801980 2323

2727

19851985 3030

3333

4343

5454

6565

19901990 6767

7070

Pts

23

65

149

316

1763

2544

2651

3311

3929

5452

5767

6125

6346

6571

21059

22051

47185

47531

48154

PtsPts

2323

6565

149149

316316

17631763

25442544

26512651

33113311

39293929

54525452

57675767

61256125

63466346

65716571

2105921059

2205122051

4718547185

4753147531

4815448154

Cumulative Odds Ratio (Log Scale)

Why EBM?Thrombolysis in Myocardial infarction

Antman et al., JAMA, 1992; 268: 240-248

Why EBM

Lau et al., NEJM, 1992

Line of identity/null

effect

Summary point

estimate

Point estimateConfidence

interval

After > 5000 pts

03.11.2011

7

Why EBM?Lidocaine in Myocardial infarction

Antman et al., JAMA, 1992; 268: 240-248

Precise and bias free?Beta-blockers in Myocardial infarction

Antman et al., JAMA, 1992

03.11.2011

8

Where did “EBM” come from?

• Alvan Feinstein

• David Sackett– Critical appraisal

– Bringing critical appraisal to the bedside

• Gordon Guyatt (McMaster residency program)– Users’ Guide to the Medical Literature

• Brian Haynes

– ACP Journal Club

– Search strategies (Pubmed)

Evidence Based Health Care

The conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence-based medicine requires integration of individual clinical expertise and patient preferences with the best available external clinical evidence from systematic research.

03.11.2011

9

Today’s presentation

Brief history and

context of EBM

Evidence based

decision making

Evidence to action =

recommendations

Evidence based healthcare

decisions

Research evidence

Population/societalvaluesand preferences

(Clinical) state and circumstances

Expertise

Haynes et al. 2002

03.11.2011

10

Your patient…as an

internist

• 68 year old man with hypertension and non-

valvular atrial fibrillation > 3 months

Atrial Fibrillation - Stroke

03.11.2011

11

Your patient…as an

internist

• 68 year old man with hypertension and non-

valvular atrial fibrillation > 3 months

– diabetes

– large left atrium (→ cardioversion unlikely to be successful)

– no history of strokes or transient ischemic attacks (TIAs)

• Afraid of having a stroke

The clinically sensible question

Population: Does in patients with atrial fibrillation

Intervention: oral anticoagulation (comparison) compared with no therapy

Outcomes: reduce the risk for embolic stroke, increase the risk for bleeding, increase burden…?

PICO

03.11.2011

12

Risk factors for stroke with NVAF

CHADS2 score for assessment of stroke risk in patients with non-rheumatic AF

Risk factor Points

Recent Congestive heart failure

exacerbation 1

History of Hypertension 1

Age 75 years or older 1

Diabetes 1

Prior history of Stroke or TIA 2

Risk factors for stroke with NVAF

CHADS2 score for assessment of stroke risk in patients with non-rheumatic AF

Risk factor Points

Recent Congestive heart failure

exacerbation 1

History of Hypertension 1

Age 75 years or older 1

Diabetes 1

Prior history of Stroke or TIA 2

CHADS = 2

03.11.2011

13

Evidence concerning NVAF and

stroke*• Risk of stroke if untreated (CHADS =2):

45/1000 per year

• Relative Risk Reduction for stroke

– Warfarin: 0.64 (95%CI 0.51-0.77)

• RRI for major bleeding

– Warfarin: 2.58 (95%CI 1.12-5.97)

* Pooled estimates of treatment effect in this evidence profile are from a meta-analysis conducted for these guidelines, including data from 6 RCTs of adjusted-dose vitamin K antagonist therapy versus no antithrombotic therapy (AFASAK I, BAATAF, CAFA, EAFT, SPAF I, SPINAFPhysician accuracy in estimating risk: no better than chance…

Primum non nocere

“Primum non net nocere”

03.11.2011

14

Balancing desirable and undesirable

consequences

↑ burden ↑ resources

↑ dietary restriction

↑ bleeding

↑ QoL ↓ stroke

↓ Morbidity

↑ survival

For Against

For Against

Balancing desirable and undesirable

consequences

03.11.2011

15

For Against

Balancing desirable and undesirable

consequences

Summary from the practitioner’s

perspective for this patient

• must anticoagulate 100 people with NVAF

for 1 year to prevent 3 strokes per year

(30 fewer per 1000 or NNT of 33)

• for 100 anticoagulated patients in the

community, this will cause 1 additional

people to have a major bleed per year

(8 more per 1000 or NNT of 125)

03.11.2011

16

Summary from this patient’s

perspective

• If you take anticoagulants

– your risk of stroke in the coming year will

decrease from 4.5% to 1.5% per year

but

– your risk of having a major bleed will increase

from 0.5% to 1% per year

How are you going to make a

decision?

1. Parental model

– Offer and administer medication (VKA)

2. Shared decision

– Obtain values, encourage patient to try

medication

3. Informed decision

– Inform patient about stroke reduction, bleeding,

hassle

– Patient chooses

03.11.2011

17

Warfarin (vitamin K antagonist)

RRR for stroke =

66%

(95% CI 51 to 77%)

CHADS 0 points (no HTN, no diabetes)

8 per 1000

5 fewer strokes per 1000

(from 4 fewer to 6 fewer)

CHADS 1 point (HTN, no diabetes)

22 per 1000 15 fewer strokes per 1000

(from 11 fewer to 17 fewer)

CHADS 2 points

45 per 1000 30 fewer strokes per 1000

(from 23 fewer to 35 fewer)

CHADS 3 to 6 points

96 per 1000 63 fewer strokes per 1000

(from 49 fewer to 74 fewer)

RRI for EC bleed =

158%

(95% CI 12 to 497%)

5 per 1000 8 8 more bleeds per 1000

(from 1 more to 25 more)

Physician and patient

mean bleeding thresholds for warfarin*

• Baseline risk of 12 strokes and 3 major bleeds in 100 patients over 2 years

• Given warfarin would decrease the risk of stroke to 4 in 100 patients, the authors determined the maximum number of excess bleeds that participants were willing to accept

Physician mean

threshold

Patient mean

threshold P value

Maximum

increase in

bleeding risk

acceptable

10.3 17.4 <0.001

* Devereaux et al, BMJ 2001; 323:1218-22

03.11.2011

18

0

5

10

15

20

25

30

35

40

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

NU

MB

ER O

F P

HYS

ICIA

NS/

PATI

ENTS

MAXIMUM NUMBER OF ACCEPTABLE EXCESS BLEEDS

Physicians N=63

Patients N=61

Physician and patient mean bleeding

thresholds for warfarin*

Detection of patient’s actions

Doctors’ judgements of their patients’

adherence:

• sensitivity 10%

• specificity 88%

• Gilbert et al, CMAJ 1980

• Physician accuracy in estimating

compliance: no better than chance…

03.11.2011

19

Eliciting and incorporating

preferences

• Qualitative

– discussion with patient

• Quantitative

– likelihood of help vs. harm

– decision analytic modeling

• standard gamble

• time trade-off

• visual analog scale

– probability trade off technique

Today’s presentation

Brief history and

context of EBM

Evidence based

decision making

Evidence to action =

recommendations

03.11.2011

20

From Idea to Practice

Glasziou and Haynes, ACP JC

Many “Leaks” from research &

practice

Aware Accept Target Doable Recall Agree Done

Valid

Research

If 80% achieved at each stage then

0.8 x 0.8 x 0.8 x 0.8 x 0.8 x 0.8 x 0.8 = 0.21 of the time would patients

receive intervention

03.11.2011

21

What are your options to obtain the

evidence and make evidence based

decisions about anticoagulation in

stroke?Consult a textbook

Remember what you read in the journals

Look for trials on Pubmed/Medline

Look for a systematic review

Look for a pre-appraised summary

Look for a guideline

What is a guideline?

• "Guidelines are recommendations intended to assist providers and recipients of health care and other stakeholders to make informed decisions.Recommendations may relate to clinical interventions, public health activities, or government policies."

WHO 2003, 2007

03.11.2011

22

Clinical Practice Guidelines

…are a result of the desire:

• of healthcare workers to offer and of patients to receive the best possible care

• to make care more efficient and consistent by bridging the gap between what clinicians do and what the evidence shows

S. Weingarten. Hospital Medicine 2005

“Practice guidelines … have been demonstrated to improve patient outcomes and lower cost”

S. Weingarten. Hospital Medicine 2005

…be based on sound scientific evidence and

implemented in an effective manner

03.11.2011

23

Which approach?

Evidence Recommendation

• B Class I

• A 1

• IV C

Organization

� AHA

� ACCP

� SIGN

Recommendation for use of oral anticoagulation in patients with atrial fibrillation and rheumatic mitral valve disease

What to do?

03.11.2011

24

Hierarchy of evidencebased on quality

STUDY DESIGN

� Randomized Controlled

Trials

� Cohort Studies and Case

Control Studies

� Case Reports and Case

Series, Non-systematic

observations

� Expert Opinion

BIAS

BMJ 2003

BMJ, 2003

03.11.2011

25

BMJ 2003Relative risk reduction:

….> 99.9 % (1/100,000)

U.S. Parachute Association

reported 821 injuries and 18

deaths out of 2.2 million jumps

in 2007

Simple hierarchies are

(too) simplistic

STUDY DESIGN

� Randomized Controlled

Trials

� Cohort Studies and Case

Control Studies

� Case Reports and Case

Series, Non-systematic

observations

BIAS

Expert Opinion

Exp

ert O

pin

ion

Schünemann & Bone, 2003

03.11.2011

26

GRADE Working Group

Grades of Recommendation

Assessment, Development and

Evaluation

CMAJ 2003, BMJ 2004, BMC 2004, BMC 2005, AJRCCM 2006, Chest 2006, BMJ 2008

Aim: to develop a common,

transparent and sensible system for

grading the quality of evidence and

the strength of recommendations

- Since 2000

- Guideline

developers,

methodologists &

clinicians from

around the world

(> 260)

GRADE Uptake� World Health Organization� Allergic Rhinitis in Asthma Guidelines (ARIA) � American Thoracic Society � American College of Physicians� European Respiratory Society� European Society of Thoracic Surgeons� British Medical Journal� Infectious Disease Society of America � American College of Chest Physicians � UpToDate® � National Institutes of Health and Clinical Excellence (NICE)� Scottish Intercollegiate Guideline Network (SIGN)� Cochrane Collaboration � Infectious Disease Society of America� Clinical Evidence � Agency for Health Care Research and Quality (AHRQ)� Partner of GIN� Over 60 major organizations

03.11.2011

27

GRADE: recommendations & quality

of (a body of)evidenceClear separation, but judgments required:

1) Recommendation: 2 grades – conditional (aka weak) or strong (for or against an action)?– Balance of benefits and downsides, values and

preferences, resource use and quality of evidence

2) 4 categories of quality of evidence: ⊕⊕⊕⊕ (High), ⊕⊕⊕�(Moderate), ⊕⊕��(Low), ⊕���(Very low)?

– methodological quality of evidence– likelihood of bias related to recommendation– by outcome and across outcomes

*www.GradeWorking-Group.org

Likelihood

of and

confidence

in an

outcome

03.11.2011

28

Determinants of quality

• RCTs ⊕⊕⊕⊕

• observational studies ⊕⊕��

• 5 factors that can lower quality1. limitations in detailed design and execution (risk of bias criteria)

2. Inconsistency (or heterogeneity)3. Indirectness (PICO and applicability)4. Imprecision (number of events and confidence intervals)

5. Publication bias

• 3 factors can increase quality1. large magnitude of effect2. plausible residual bias or confounding3. dose-response gradient

↑ burden ↑ resources

↑ dietary restriction

↑ bleeding

↑ QoL ↓ stroke

↓ Morbidity

↑ survival

Conditional

Strong For Against

Balancing desirable and undesirable

consequences

03.11.2011

29

Conditional

Strong For Against

Balancing desirable and undesirable

consequences

Conditional

Strong For Against

Balancing desirable and undesirable

consequences

03.11.2011

30

Balancing desirable and undesirable

consequences

Conditional

Strong For Against

Balancing desirable and undesirable

consequences

Conditional

Strong For Against

03.11.2011

31

Criteria for when

we do not need

RCTs

How to elicit and

integrate Values

and Preferences

Guideline

development

Information

presentation

Summary

• Evidence based practice is required for

delivering appropriate care

• Basic skills help understanding and

communicating with patients

• Leaky pipeline from evidence to practice

• Guidelines one way to support decision

makers

03.11.2011

32