Clinical Epidemiology – the basics

Clinical Epidemiology – the basics

• What do the terms relative risk and absolute risk mean? What are the advantages and disadvantages of each?

• A new screening test is described as having a sensitivity of 78%, a specificity of 89%,a positive likelihood ratio of 13 and a negative likelihood ratio of 0.05. Explain these terms.

A 2 minute exercise

• What do you know about this topic? Could you explain what you know to others?

• What bits would you like clarifying? (Hint – do you understand RR, OR, CIs etc)

EVIDENCE BASED MEDICINE

EBM is an approach to practicing medicine in which the clinician is aware of the evidence in support of his / her clinical practice, and the strength of that evidence.

EVIDENCE BASED HEALTHCARE

Evidence based health care promotes the collection, interpretation, and integration of valid, important and applicable patient-reported, clinician-observed and research derived evidence. The best available evidence, moderated by patient circumstances and preferences, is applied to improve the quality of clinical judgements and facilitate effective healthcare.

EFFECTIVE SAFE

COST PATIENTFACTORS

QUALITYOF CARE

SYSTEMATICAPPLICATIONOF THERAPY

100%0%

LEVELS OF EVIDENCE

• LARGE WELL DESIGNED RCT

• META ANALYSIS OF SMALLER RCTs

• CASE CONTROL AND COHORT STUDIES

• (CASE REPORTS AND CASE SERIES)

• CONSENSUS FROM EXPERT PANELS

• I THINK

Why don’t we always use an RCT then?

• Ethics

• Cost

• Feasibility

• Practicality

Why read journals?

• Need to make the best possible decisions for patients

• Need to make the best possible decisions for healthcare

• Need to feel confident about being “on top of the job”

• Need to feel knowledgeable themselves and credible with peers

WHY THE MOVE TO EBM?

• RANDOMISED CONTROLLED TRIALS PRE-1960 WERE ODDITIES

• REVIEWS AND META-ANALYSES AVAILABLE AS ACCESSIBLE DIGESTS OF EVIDENCE

• ACCESS TO EVIDENCE VIA I.T.

• METHODOLOGICAL ADVANCEMENTS E.G. NUMBERS NEEDED TO TREAT

EBM IS ABOUT ...

• CLINICAL EXPERIENCE, DIAGNOSTIC SKILLS AND CLINICAL INSTINCT ARE A NECESSARY PART OF A COMPENTENT PHYSICIAN.

• HOWEVER, CLINICAL PRACTICE BASED SOLELY UPON CLINICAL EXPERIENCE “BECOMES TOMORROW’S BAD JOKE”.

• “RATIONAL” TREATMENT BASED SOLELY UPON BASIC PATHOLOGICAL PRINCIPLES MAY IN FACT BE INCORRECT, LEADING TO INACCURATE TREATMENT.

• UNDERSTANDING CERTAIN RULES OF EVIDENCE IS NECESSARY TO CORRECTLY INTERPRET LITERATURE ON CAUSATION, PROGNOSIS, DIAGNOSTIC TESTS AND TREATMENT STRATEGY.

20,000 biomedical journals in print. So why isn’t all practice based on scientific evidence?

• Not RELEVANT– Upstream to clinical decisions being made, e.g. animal or in vitro

studies

– Study populations and / or settings do not reflect question type, practice population and settings.

• Not RELIABLE– Poor study design

– Bias and confounding

– Measurement validity

– Insufficient power

BIAS

• Selection bias

• Observer bias

• Participant bias

• Withdrawal or drop out bias

• Recall bias

• Measurement bias

• Publication bias

CONFOUNDING

COFFEE DRINKING LUNG CANCER

SMOKING

STUDY

CONFOUNDING VARIABLE

Power

The ability of the study to detect an effect if in truth there is an effect.

An RCT may be underpowered if:-

• The duration is too short (too few events)

• It includes too few people (too few events)

• The wrong outcome was used (too few events)

• Expecting a higher level of statistical proof than is realistic for the condition and the intervention being tested

EBM SKILLS - STATISTICS

• CHANCE - p = 1 in 20 (0.05).

• > 1 in 20 (0.051) = not significant

• < 1 in 20 (0.049) = statistically significant

• CONFIDENCE INTERVALS

• what is the range of values between which we could be 95% certain that this result would lie if this intervention was applied to the general population

TYPES OF STUDY - HYPOTHESIS FORMING

• CASE REPORTS / CASE SERIES

• CROSS SECTIONAL / PREVALENCE STUDIES measure personal factors & disease states – a snapshot

• CORRELATIONAL / ECOLOGICAL / GEOGRAPHIC STUDIES. prevalence &/or incidence measurement in one population c/w another pop.

TYPES OF STUDY - HYPOTHESIS TESTING

CASE CONTROL STUDIES

Controls

Population

CasesYes

No

Yes

No

Exposure to Risk Factor

TIME

STUDY

CASE CONTROL EXAMPLE -SMOKING & LUNG CANCER

DISEASE

Cases Controls

EXPOSURE Yes a b

EXPOSURE No c d

Odds Ratio = ad/bc

(1 = no association, > 1 = possible association, < 1 = protective effect)

DISEASE

Cases Controls

(lung cancer)

EXPOSURE Yes 56 230

(smoking) No 7 246

The odds ratio would therefore be 56 x 246 = 13776 = 8.6.

7 x 230 1610

TYPES OF STUDY - HYPOTHESIS TESTING

COHORT STUDIES

Population

Sample

Yes

No

Yes

No

Time

Exposed

Not exposed

COHORT STUDIES

OUTCOME

Yes No

Exposed a b

Not exposed c d

Relative risk "How many times are exposed persons more likely to

develop the disease, relative to non-exposed persons?" i.e. the incidence in the exposed divided by the incidence in the non-exposed.

This is expressed as a divided by c .

a+b c+d

COHORT STUDY EXAMPLE

Deep vein thromboses (DVT) in oral contraceptive users. (Hypothetical results).

OUTCOME (DVT)

Yes No

Exposed ( on oral contraceptive ) 41 9998

Not exposed (not on o.c.) 7 10009

These results would give a relative risk of 6 - significantly large enough numbers to indicate the possibility of a real association between exposure

and outcome. However, NB biases.

RANDOMISED CONTROLLED TRIALS

Population Sample Time

Improved

Not improved

Not improved

Improved

Experimental intervention

Comparisonintervention

RANDOMISED CONTROLLED TRIALS

OUTCOME

Yes No

Comparison intervention a b

Experimental intervention c d

Absolute risk reduction: “What is the size of this effect in the population”

Control event rate - experimental event rate

a/a+b - c/c+d

Relative risk reduction: “ How many fewer patients will get the outcome measured if they get active treatment versus comparison intervention”

a /a+b - c/c+d

a/a+b

ARR and RRRA quick test

• In a study lasting 12 months, the death rate on placebo was 10% and the death rate on Marvelicoxib was 5%.

• What is the ARR?

• What is the RRR?

ARR and RRR in more detail4S STUDY

• STABLE ANGINA OR MYOCARDIAL INFARCTION MORE THAN 6 MONTHS PREVIOUSLY

• SERUM CHOLESTEROL > 6.2mmol/l

• EXCLUDED PATIENTS WITH ARYHTHMIAS AND HEART FAILURE

• ALL PATIENTS GIVEN 8 WEEKS OF DIETARY THERAPY

• IF CHOLESTEROL STILL RAISED (>5.5) RANDOMISED TO RECEIVE SIMVASTATIN (20mg > 40mg) OR PLACEBO

• OUTCOME DEATH OR MYOCARDIAL INFARCTION (LENGTH OF TREATMENT 5.4 YEARS ) WERE THE OUTCOMES

RCT EXAMPLE - 4S STUDY

OUTCOME (death)

Yes No

Comparison intervention (placebo) 256 1967 2223

Experimental intervention (simvastatin) 182 2039 2221

The ARR is (256/2223) - (182/2221) = 0.115 - 0.082 = 0.033.

The RRR is 0.033/0.115 = 0.29 or expressed as a percentage 29%.

1/ARR = NUMBER NEEDED TO TREAT.

1/0.033 = 30.

i.e. if we treat 30 patients with IHD with simvastatin as per 4S study, in 5.4 years we will have prevented 1 death.

Another way of calculating NNTs

OUTCOME (death)

Yes No

Comparison intervention (placebo) 256 1967 2223

Experimental intervention (simvastatin) 182 2039 2221

Prevalence of event in control group = 256/2223x100=11.5%

RRR = 29%

Now that’s magic!

NNT EXAMPLES

Intervention Outcome NNT

Streptokinase + aspririn v.placebo (ISIS 2)

prevent 1 deathat 5 weeks

20

tPA v. streptokinase(GUSTO trial)

save 1 life withtPA usage

100

Simvastatin v. placebo in IHD(4S study)

prevent 1event in 5y

15

Treating hypertension in the over-60s

prevent 1 eventin 5y

18

Aspirin v. placebo in healthyadults

prevent MI ordeath in 1 year

500

Why are RCTs the “gold standard”Breast cancer mortality in studies of screening with mammography; women

aged 50 and over (55 in Malmo study, 45 in UK)

Relative risk

0.1 0.2 0.5 1.0 2.0

Reduced RR Increased RR

Randomised Trials

Geographical study

Case control studies

HIP

Two County

Malmo

Edinburgh

Stockholm

UK

BCDDP

Nijmegen

Utrecht

Florence

Egger M et al. Meta-analysis Spurious precision? Meta-analysis of observational studies BMJ 1998;316:140-144

Odds ratios or relative risks?Macfarlane J et al. BMJ 2002; 13: 105-9

Patients who took antibiotics

Patients who did not take

antibiotics

TOTAL

Patients who were given a leaflet

49 55 104

Patients not given a leaflet

63 38 101

TOTAL 112 93 205

Patients who took antibiotics

Patients who did not take

antibiotics

TOTAL

Patients who were given a leaflet

49 55 104

Patients not given a leaflet

63 38 101

TOTAL 112 93 205

Relative risk: (49/104) / (63/101) = 0.76.i.e the relative risk of patients taking an antibiotic if they were given a leaflet is reduced by 24%. Also calledrisk ratio.

Patients who took antibiotics

Patients who did not take

antibiotics

TOTAL

Patients who were given a leaflet

49 55 104

Patients not given a leaflet

63 38 101

TOTAL 112 93 205

Odds ratio: (49/55) / (63/38) = 0.54.There was a 46% reduction in the ratio of those taking antibiotics who had a leaflet compared with the ratio of those taking antibiotics who did not have a leaflet.

Patients who took antibiotics

Patients who did not take

antibiotics

TOTAL

Patients who were given a leaflet

49 55 104

Patients not given a leaflet

63 38 101

TOTAL 112 93 205

Absolute risk reduction: (49/104) – (63/101) = 0.15.Also known as the risk difference. i.e. the difference in the riskof taking antibiotics depending on whether a leaflet was usedor not.

Patients who took antibiotics

Patients who did not take

antibiotics

TOTAL

Patients who were given a leaflet

49 55 104

Patients not given a leaflet

63 38 101

TOTAL 112 93 205

NNT: 1 / 0.15 = 7. i.e. 7 people need to be given a leafletIn order for 1 additional person not to take antibiotics

Jüni P, Rutjes AWS, Dieppe PA. Are selective COX 2 inhibitors superior

to traditional non steroidal anti-inflammatory drugs? BMJ 2002; 324: 1287-1288

Screening and Diagnostic Tests

SCREENING - WILSON & JUNGEN (WHO, 1968)

• IS THE DISORDER COMMON / IMPORTANT

• ARE THERE TREATMENTS FOR THE DISORDER

• IS THERE A KNOWN NATURAL HISTORY & “WINDOW OF OPPORTUNITY” WHERE SCREENING CAN DETECT DISEASE EARLY WITH IMPROVED CHANCE OF CURE

• IS THE TEST ACCEPTABLE TO PATIENTS

• SENSITIVE AND SPECIFIC

• GENERALISABLE

• CHEAP / COST EFFECTIVE

• APPLY TO GROUP AT HIGH RISK

Tests ain’t what they used to beJoseph Heller Catch 22 1962

“Gus and Wes had succeeded in elevating medicine to an

exact science. All men reporting on sick call with

temperatures above 102 were rushed to hospital. All those

except Yossarian reporting on sick call with temperatures

below 102 had their gums and toes painted with gentian

violet solution and were given a laxative to throwaway in

the bushes. All those reporting on sick call with

temperatures of exactly 102 were asked to return in an

hour to have their temperatures taken again.”

0 10 20 30

NoDisease

A BDisease

Percentof population

No Disease

DiseaseNo Disease

VALUEArbitrary Units

Set cut off at A A lot of people who do not have the disease arelabeled as having it (false positives)

Set cut off at B A lot of people who do have the disease arelabeled as not having it (false negatives)

Disease

C

DISEASE

Present Absent

TEST

Positive

Negative

50

0

0

50

a b

c d

Measure the usefulness of the TEST by..

Positive

Negative

45

5

5

45

a b

c dTEST

Present Absent

DISEASE

Sensitivity Specificity

Sensitivity = a = 45 = 90% high sensy = a + c 50 few false negatives

Specificity = d = 45 = 90% high specy = b + d 50 few false positives

Test with a high specificity useful to rule in a diagnosis

e.g. before cancer chemotherapy

Test with high sensitivity useful to rule out a diagnosis

e.g. antenatal for syphilis

Sensitivity and specificity are properties of the test and are taken into account when deciding whether to test.

But……(and this is the hard bit so concentrate NOW)

When the test result is available the usefulness of the result depends on:-

1. How good (or bad) the test was at detecting true positives and true negatives

2. The pre-test probability of the person being tested actually having the disease for which they are being tested.

What is the pre-test probability of someone with dyspepsia being H pylori positive?

What is the pre-test probability of someone with dyspepsia being H pylori negative?

TEST

Positive

Negative

45 5

5 45

a bc d

Present Absent

DISEASE

Positive predictivevalue a = 45 = 90%a+b 50

Prevalence = 50%

Negative predictivevalue d = 45 = 90%c+d 50

Sens = 45/50i.e. 90%

Spec = 45/50i.e. 90%

The Impact of Prevalence on Predictive Value (Bayes Theorem)

DISEASE

Present Absent

9

1

9

81

a b

c d

TEST

POSITIVE

NEGATIVE

PPV = 9 = 50% 18

NPV = 81 = 99% 82

Sensitivity = 9 = 90% Specificity = 81 = 90% 10 90

Watch what happens when the prevalence drops to 10%…….

Likelihood ratios express how many more times (or less times) a test result is to be found in diseased people compared with non-diseased people.

TEST

Positive

Negative

Present Absent

a

c

b

d

DISEASE

LR +ve = a LR -ve = ca + c a + c

b db + d b + d

This change can be described arithmetically by likelihood ratios.

Likelihood ratios - EXAMPLE

TEST

POSITIVE

NEGATIVE

DISEASE

PRESENT ABSENT

9 9

1 81

a b

c d

LR +ve = 0.9 = 10 LR -ve = 0.1 = 0.120.09 0.81

New non-invasive tests for H. Pylori Gastritis. Comparison with tissue-based gold standard.

Douglas O, et al. Digestive Diseases and Sciences 1996; 41:740-8

Urea Breath Test

Sens. Spec.

LR +ve LR -ve

90 96 22 0.10

Serum Anti-bodies 74 89 7 0.30

Here comes another (different) magic nomogram!

+ve -vePrevalence PTP PTP

UBT 20% 85% 2%40% 95% 5%

Sab 20% 60% 5%40% 80% 12%

PTP - Post-test probabilityUBT - urea breath testSab - serum antibody test

+ve -vePrevalence PTP PTP

UBT 20% 85% 2%40% 95% 5%

Sab 20% 60% 5%40% 80% 12%

PTP - Post-test probabilityUBT - urea breath testSab - serum antibody test

H Pylori infection in a population with a 25% prevalenceMeReC Bulletin 2001; 12 (1): 1-4

646945475.586Near-patient serological tests

32597759091Laboratory serological tests

112998895.597.5Breath test (14C)

11199899696.5Breath test (13C)

False Negative results (%)

False positive results (%)

Negative predictive value (%)

Positive predictive value (%)

Specificity (%)

Sensitivity (%)

SUMMARY

EVIDENCE BASED MEDICINE

FORMULATE QUESTION

EFFICIENTLY TRACK DOWN BESTAVAILABLEEVIDENCE

CRITICALLY REVIEW THEVALIDITY AND USEFULNESSOF THE EVIDENCE

IMPLEMENT CHANGESIN CLINICAL PRACTICE

EVALUATE PERFORMANCE

“The evidence isn’t there” (whinge, moan) OR “I don’t have the time” (whine, complain)

• Clinical Evidence

• Cochrane

• DTB, MeReC Bulletin

• PRODIGY

• Evidence Based Medicine

LIMITATIONS

• STILL LOTS OF ROOM FOR DEBATE ABOUT THE EVIDENCE BASE

• EBM = WHAT IS BEST FOR AN INDIVIDUAL PATIENT (patient utility)

• EVIDENCE BASED PURCHASING = BEST USE OF HEALTH CARE RESOURCES FOR THE LOCAL POPULATION (cost utility). i.e. knowledge of local needs, priorities and constraints

• WHAT IF THESE CONFLICT? (Anybody want to mention beta interferon and MS?!)

EBM VISION FROM 1996

DOCTOR Consultation PATIENT

Critical Appraisal Skills

Systematic reviewsIndividual studies and reports

GUIDELINES

ELECTRONIC, PATIENT-SPECIFIC REMINDERS

Individual database searches

Disease

informationspecific

CONSULTATION SKILLSCONSULTATION SKILLS