How to Appraise Diagnosis

8/12/2019 How to Appraise Diagnosis

1/39

Appraising Diagnostic Study

Muhammad Ali

Medical School

University of Sumatera Utara


2/39


3/39


4/39

What are tests used for?

Log of reasons by several docs:

Diagnosismost common

but alsoMonitoringhas it changed?

Prognosisrisk/stage within Dx

Treatment planning


5/39

Basic principles (1)

Ideal diagnostic testsright answers:

(+) results in everyone with the disease and

( - ) results in everyone else

Usual clinical practice:

The test be studied in the same way it would be used in

the clinical setting

Observational study, and consists of:

Predictor variable (test result)

Outcome variable (presence / absence of the disease)


6/39

Basic principles (2)

Sensitivity, specificity

Prevalence, prior probability, predictive values

Likelihood ratios

Dichotomous scale, cutoff points (continuous scale)

Positive (true and false), negative (true and false)

ROC (receiver operator characteristic) curve


7/39

EBM Issues:

Is this evidence about the accuracy of adiagnostic test valid?

Does this valid evidence demonstrate an

importantability to accurately distinguishpatients who do and dont have a specificdisorder?

Can I applythis valid,importantdiagnostic

test to a specific patient?


8/39

Critical appraisal

Use worksheet

Use supporting software

CAT Maker

Save in CAT Banks
http://localhost/var/www/apps/conversion/tmp/scratch_5/CATmaker/CATMAKER.EXEhttp://localhost/var/www/apps/conversion/tmp/scratch_5/CATmaker/CATMAKER.EXE


9/39

Is this evidence about a diagnostic test

valid?

Was there an independent, blind comparison with areference (gold) standardof diagnosis?

Was the diagnostic test evaluated in an appropriatespectrum of patients (like those in whom we would useit in practice)?

Was the reference standard applied regardless of thediagnostic test result?

Was the test (or cluster of tests) validated in a second,independent group of patients?


10/39

Sensitivity

Specificity

Likelihood ratios

Does this valid evidence demonstrate an important

ability to accurately distinguish patients who doand dont have a specific disorder?


11/39

SnNoutDiagnostic test with a very high sensitivity ,

a negative result effectively rules out the

diagnosis

SpPin

Diagnostic test with a very high specificity ,a positive result effectively rules in the

diagnosis


12/39

Questions to answer in applying a validdiagnostic test to an individual patient

Is the diagnostic test available, affordable, accurate,

and precise in our setting?

Can we generate a clinically sensible estimate of our

patients pre-test probability?

From personal experience, prevalence statistics,

practice databases, or primary studies

Are the study patients similar to our own? Is it unlikely that the disease possibilities or

probabilities have changed since this evidence was

gathered?


13/39

Does early diagnosis really lead to improvedsurvival, or quality of life, or both?

Are the early diagnosed patients willing partnersin the treatment strategy?

Is the time and energy it will take us to confirmthe diagnosis and provide (lifelong) care wellspent?

Do the frequency and severity of the targetdisorder warrant this degree of effort andexpenditure?

Guides for deciding whether a screeningor earlydiagnostic maneuver does more good than harm:


14/39

Diagnostic tests

Is not about finding absolute truth, butabout limiting uncertainty

establishes both the necessity and thelogical base for introducing

probabilities, pragmatic test-treatment

thresholds ..Start thinking about

what youre going to do with the results of thediagnostic test, and

whether doing the test will help your patients


15/39

TRADITIONAL 2x2 TABLE

DISEASE

Yes No Total

3 7Yesa b

10a+b

c d

No 1 89 90 c+d

4 96 100

TEST

Totala+c b+d a+b+c+d


16/39

SENSITIVITY

DISEASE

Yes No Total

3 7Yesa b

10a+b

c d

No 1 89 90 c+d

4 96 100

TEST


SENSITIVITY

The proportion of people with the diagnosis (n=4)who are correctly identified (n=3)

Sensitivity = a/(a+c) = 3/4 = 75%

FALSENEGATIVES


17/39

SPECIFICITY

DISEASE

Yes No Total

3 7Yesa b

10a+b

c d

No 1 89 90 c+d

4 96 100

TEST


SPECIFICITY

The proportion of people without the diagnosis (n=96)who are correctly identified (n=89)

Specificity = d/(b+d) = 89/96 = 93%

FALSEPOSITIVES


18/39

PREDICTIVE VALUES (POS or NEG)

DISEASE

Yes No Total

3 7Yesa b

10a+b

c dNo 1 89 90 c+d

4 96 100

TEST


Positife or Negative predictive valuesare the proportion of disease / nondisease people to all who havepositive/negative tests.

PPV = 3/10 = 30%

NPV = 89/90=89%


19/39

PRE-TEST ODDS

DISEASE

Yes No Total

3 7Yes

a b

10a+b

c d

No 1 89 90 c+d

4 96 100

TEST


In the sample as a whole, the odds of having the diseaseare 4 to 96 or 4% (the PRE-TEST ODDS)


20/39

POST-TEST ODDS

DISEASE

Yes No Total

3 7Yes

a b

10a+b

c d

No 1 89 90 c+d

4 96 100

TEST


In those who score positive on the test, the odds of havingthe disease are 3 to 7 or 43% (the POST-TEST ODDS)

In the sample as a whole, the odds of having the diseaseare 4 to 96 or 4% (the PRE-TEST ODDS)


21/39

POST-TEST ODDS

DISEASE

Yes No Total

3 7Yes

a b

10a+b

c d

No 1 89 90 c+d

4 96 100

TEST


In those who score positive on the test, the odds of having the disease are3 to 7 or 43% (the POST-TEST ODDS)

In the sample as a whole, the odds of having the disease are 4 to 96 or4% (the PRE-TEST ODDS)

In those who score negative on the test, the odds of having the diseaseare 1 to 89 or approximately 1%


22/39

DIAGNOSTIC ODDS RATIO

DISEASE

Yes No Total

3 7Yesa b

10a+b

c d

No 1 89 90 c+d

4 96 100

TEST


The Diagnostic Odds Ratio is the ratio ofodds of having the diagnosis given apositive test to those of having thediagnosis given a negative test

2.38011.0

429.0

891

73

DOR

Potentially useful as an overallsummary measure, but only inconjunction with othermeasures (LR, sensitivity,specificity)


23/39

BAYES THEOREM

POST-TEST ODDS =LIKELIHOOD RATIO x PRE-TEST ODDS


24/39

Natural frequencies provide a more graphic, easy to

understand way to portray probabilities for both

physicians and patients

Interpreting

Sensitivity &

Specificity

Using natural

frequencies to

calculate Bayes

theorem


25/39

In small groups what is the likelihood that this

patient has the disease?

A disease with a prevalence of 3% mustbe diagnosed

There is a test for the disease

It has a sensitivity of 50% and a

specificity of 90%


26/39

For a prevalence of 3% - in every 100 patients

97

3 will have the disease

Will not have the disease


27/39

97How many will nothave the disease?

For a test with a sensitivity of 50%

For every 100

patients with the

disease 1-2 will test

positive


28/39

87

For a test with a specificity of 90%

False positive rate

= 1-Specificity(10%)

For those who dont have the

disease 10 patients will go on

to test positive


29/39

87

Given a positive test what are my chances of disease doc?

11 people have a positive test Of

which 1 have will have the disease

13% chance of having the

disease given a positive test


30/39

87

Given a negative test what are my chances of not having the

disease doc?

88 people

will have a negative test of

which 1 will have the

disease

False reassurance rate of

1.7%


31/39

Describe the result to your patient

For every 100 people who have a

positive test 13 will have the

disease


32/39

What happens if I have an negative test

Negative test pretty much

rules out the disease but a

small number 1-2 people

in every 100 will still have

the disease


33/39

A disease with a prevalence of 30%must be diagnosed

There is a test for this disease

It has a sensitivity of 50% and aspecificity of 90%

Try it again

what is the likelihood that this patient has the disease?


34/39

Prevalence of 30%

Sensitivity of 50%

Specificity of 90%

30

70

15

7

100

22 positive

tests in totalof which 15

have the

disease

About 70%

Disease +ve

Disease -ve


35/39

Likelihood Ratio

Sensitivity/1-specificity

50% / 10%

=5

LR a

b


36/39

Likelihood Ratio

LR a

b

30

70

100

5 150

70Prevalence of 30%

Sensitivity of 50%

Specificity of 90%


37/39

Now consider the FOB screening tests

You find out that your father has undertaken the test

and has a positive resultHe ask you whether he has

cancer?

Prevalence of disease is 0.3% must be

diagnosed.

Sensitivity of 50%

False positive rate 3%.


38/39

APPLYING A DIAGNOSTIC TEST IN

DIFFERENT SETTINGS

The Positive Predictive Value of a test will vary(according to the prevalence of the condition in thechosen setting)

Sensitivity and Specificity are usually consideredproperties of the testrather than the setting, andare therefore usually considered to remainconstant

However, sensitivity and specificity are likely to beinfluenced by complexity of differential diagnosesand a multitude of other factors


39/39

THANK YOU

Documents

How to Appraise Diagnosis