Evidence Based Medicine&

Critical Appraisal

Dr Richard de Ferrars

October 2009

Realistic Goals

What is realistic in 1 session?

– What does EBM mean?– Why (and how) do we

read medical papers?– How to spot papers that

matter

What Does it Mean?

“There are lies,

damned lies

and statistics”

Mark Twain

Can you see through the statistics and spot the lies?

Why Bother?

There must be a better way than “being told what to do” by protocols & guidelines

– Learning how to think for yourself and develop skills in “critical appraisal”

– Learning how to link this in with the care of your patients Learning not to be a Lemming

Where Did EBM Come From?

• Archie CochraneScottish epidemiologist who, in 1972 wrote a book:

“Effectiveness and Efficiency: Random Reflections on Health Services”

Gradual increasing acceptance of the concepts behind evidence-based practice

Term "evidence-based medicine" first appeared in the medical literature in 1992 in a paper by Guyatt

What About Critical Appraisal?

Critical (adjective)

skillful judgment as to truth & merit

Appraisal (noun)

the act of estimating or judging the nature or value of something or someone.

• More than just knowing how “believable” the contents of some research & papers are.

• Also an element of judging the value & relevance, in practical terms, for your own work.

Evidence Based Medicine

1. I read a review in the BMJ last week that showed that showed thiazides gave better outcomes that ACEi in the elderly

2. My GPR (who has just passed his MRCGP) says thiazides are good anti-hypertensives

3. The new NICE guidelines say first choice BP drug for over 55’s is thiazide or CCB

4. Most people that I see in my surgery who have high BP seem to be on medication that includes a thiazide

5. That drug rep last week showed my some impressive graphs for his new CCB-thiazide combination drug

Rate these 5 fonts of wisdom in order of “good & sound”

Exercise – What is good quality?

Evidence Based Medicine

Meta-analysis & systematic review – aggregation of several similar studies

Double-blind randomised placebo controlled studies

Larger, generic products, paid for by neutral body

Double-blind randomised placebo controlled studiesSmaller, branded products, paid for by the manufacturer

Observational studies

Case reports

Anecdotal experience

Big is Beautiful!

Good Studies & Bad studies – Study Hierarchy

Evidence Based Medicine

Big is Beautiful!

Who Says?

A Based directly on category 1+ evidence.

B Based directly on category 2++ evidence

or extrapolated from category 1

C Based directly on category 2 +/-

evidenceor extrapolated from category 2++

D Based on category ¾ or

extrapolated from category 2+

1++ Meta-analysis of randomised controlled trials.

1- Randomised controlled studies

 

2++ Systematic reviews of case-control & cohort studies

2+/- Case-control studies & cohort studies

 

3 Non-analytical studies(comparitive studies, case

reports)

 

4 Expert opinions, clinical experience of respected authorities

Strength of Recommendation:

Who Says?

The hierarchy of studies is reasonably obvious

But the source of information is probably equally important

Where do you go to get information, advice & recommendations?

What sources & resources do you use?

Who Says?

• Compromise between quality of information and accessibility (ranked in quality order)

National bodies – NICE, NPC, CKS (Prodigy), SIGN, Cochrane

Publications - Journals (BMJ), Daily Mail?

Local Guidelines – PCT, hospital, department

Your “team”

Your patients????

Accessibility

Quality and

believability

Problems with EBM

Great in theory

What about the real world?

What are the main problems & issues with putting EBM into practice?

Problems with EBM

• RCT’s now might be unethical Open heart surgery, thyroxine

• Under-researched groupsWomen, racial minorities, multiple problems

• “Gold standard" DB-RCT’s are expensiveDrug companies fund studies looking at their

drugs Public authorities fund preventive medicine studies

• Published studies may not represent all studiesUnpublished studies – publication biasNon-publication agreement if not declared at

outset

Problems with EBM

• The quality of studies performed varies

• “Trials world” and “real world” are different:Reality - does not have exclusion criteria

Reality - patients have multiple pathologies

Reality - patient’s problem won’t match EBM

• EBM changes slowly, medical advances are rapidExample – trials that compared Clopidogrel to angioplasty in preventing reinfarction used doses of Clopidogrel that would now seem too low

I Hate Statistics…

How to Read a Paper

Trisha Greenhalgh

Must buy & must read– Medical statistics– EBM– Critical appraisal

Medical Statistics

Cannot escape it

You must have a basic level of understanding

Tested in the AKT

Talking statistics with patients may be required in CSA

Medical Statistics - Taking Risks

Taking Risks

The annual stroke risk in high risk AF:

Taking aspirin 12%

Taking warfarin 6%

1. Is it fair to say that Warfarin halves stroke risk?

2. How many people need to have Warfarin rather than Aspirin for a year to stop one stroke?

Taking Risks

• Absolute risk (AR)– statement of how often the “event” happened

• Relative risk (RR)– risk of the event in one group “relative” to another group

• Absolute risk reduction (ARR)– statement of how much less the risk is in one group cf. the other

• Relative risk reduction (RRR)– How much your relative risk has fallen

Making Sense of Risks

• Annual stroke risk in high risk AF patients:Warfarin 6% Aspirin 12%

• AR of a stroke on warfarin is 6% per year (0.06) • AR of a stroke on aspirin is 12% (0.12)

• RR of a stroke on warfarin compared with aspirin is 50% (calculated by: 6 / 12)

• ARR from taking warfarin rather than aspirin, for one year, is 6% (AR reduced form 12% to 6% )

Lies, Damned Lies & Statistics

You could say “using warfarin instead of aspirin for a year, your risk is reduced by 50%” (relative risk)

Or you could say “using warfarin instead of aspirin for a year, your risk is 6% less” (absolute risk)

• Both are true but one is a distortion. • Drug companies normally sell you relative risks!

The word “risk” means absolutely nothing unless prefaced by “relative” or “absolute”

Big “relative risk” changes mean little if “absolute risk” is small.

Make a small risk smaller and you still have a small risk

Turning ARR’s into NNT’s

• Number of patients who would need to be treated for one to benefit.

• How many ARR’s make 100%?

Taking warfarin rather than aspirin, for one year:– ARR is 6%

– NNT is 16 (as 16 x 6% = 100%)

• Longer-term medications & interventions usually

quoted as NNT over 5-10 year period

Numbers Needed to Treat

• Number of patients who would need to be treated for one to benefit.

– NNT of under 10 is pretty good– NNT 10-30 is useful– NNT over 50 are stretching resources a bit

Useful EBM Resources

• National Library for Health (www.nlh.nhs.uk/)an online library for NHS staff, patients and the public

• National Prescribing Centre (www.npc.co.uk) supporting high-quality & cost-effective prescribing within the NHS

• Cochrane Library (www.thecochranelibrary.com)contains high quality, independent evidence to inform healthcare decision making

• Clinical Knowledge Summaries (put CKS into Google)up to date source of clinical knowledge on common conditions for healthcare professionals and patients

• GP notebook (www.gpnotebook.co.uk)an online encyclopaedia of medicine

Professional Reading

• Browsing– Flicking the BMJ, BJGP, Pulse, GP– Has value, BUT restrict to 2-3 journals per week

• Targeted Reading– Using the EBM resources to answer a particular

question

• Critical Appraisal– How valuable is an interesting paper when (or if!) you

do come across one?

Critical Appraisal

• Knowledge of EBM & statistics:– Knowing how “believable” the contents of

research & papers are.

• Developing critical appraisal skills:– Judging the value & relevance, in practical

terms, for your own work.

Critical Appraisal - READERRelevance

1-5 How relevant to your practice?

Education1-5 How likely to change your behaviour/

practice?

Applicability1-5 How easy to apply & implement changes?

Discrimination1-10 How good is the quality of the paper?

Evaluation4-25 Total the score

Reaction4-15 Forget it16-20 Think about it21+ Do it

Using READER

• Practice using the READER model to critically appraise journal articles

The End