20100707 - Attributes of Screening Tests (Revised)

8/8/2019 20100707 - Attributes of Screening Tests (Revised)

http://slidepdf.com/reader/full/20100707-attributes-of-screening-tests-revised 1/39





C ASE

The year is 1990; first AIDS case identified in 1981has two tests for detection ELISA, Western Blot

Main drawback is 3 mo. window period for antibodies

Your involvement requested in clinical trial of newqualitative point-of-care Rapid Test (target release 1992)

What criteria will you use to:y Assess effectiveness of test

y Decide whether to adopt for our regional health authority



A TTRIBUTES O F S CREENING T ESTS



O BJECTIVES O F S CREENING P RO GRAMS

1. Identification and mitigation of risk factors thatincrease likelihood of developing the disease.

2. Detection of disease at a stage when treatment

can be more effective than it would be after thepatient develops signs and symptoms



F RAMEW O RK : S CREENING TEST A TTRIBUTES

Á RABID CASE"

Diseasey Burden of disease i.e. prevalence (not too high or low),

morbidity, mortalityy Asymptomatic phase of sufficient duration (known natural

hx)Testy A vailable widelyy Improves health outcomes (evidence)y Detects high proportion of disease in preclinical state

(validity, predictive value, reliability)y Safe to administer (risk)y Cost (incl. opportunity cost)y Rapid

Interventiony A vailable widelyy E ffectiveness increases with early detection http://wordsmith.org/anagram



S CREENING A TTRIBUTES (A N O THER V IEW )

Benefits

Costs (incl. Risk and O pportunity Cost)

Value Proposition = Benefits - Costs



F RAMEW O RK : S CREENING TEST A TTRIBUTES

Á RABID CASE"

Diseasey Burden of disease i.e. prevalence (not too high or low),

morbidity, mortalityy Asymptomatic phase of sufficient duration (known natural

hx)

Testy A vailable widelyy I mproves health outcomes (evidence)y D etects high proportion of disease in preclinical

state (validity, predictive value, reliability)y Safe to administer (risk)y Cost (incl. opportunity cost)y Rapid

Interventiony A vailable widelyy E ffectiveness increases with early detection http://wordsmith.org/anagram



F O CUS O N 2 A TTRIBUTES

Detects high proportion of disease in preclinical statey Discuss Screening Test Characteristics

Improves health outcomes (evidence)y Discuss Screening Test Evaluation



REVIEW ² A TTRIBUTES OF S CREENING TESTS

Screening for primary or secondary prevention

Eight attributes of good screening tests

When in doubt, think of value proposition



S CREENING T EST C HARACTERISTICS



K EY TEST CHARACTERISTICS

The attribute ´ Detects high proportion of disease inpreclinical stateµ requires 3 key test characteristics

y Validityy Predictive Valuey Reliability



V ALIDITY

Ability of test to distinguish between those withand without the disease

Two components:y Sensitivityy Specificity



V ALIDITY - E XAMPLE

HIV Rapid Test clinical trial resultsCalculate and interpret sensitivity and specificity

TestResult

HIV Status(Gold Standard)D+ D- Total

T+ 370 2 372

T- 4 624 628

Total 374 626 1000




Sensitivity (Sn) = 370 / 374Specificity (Sp) = 624 / 626

TestResult

HIV Status(Gold Standard)

D+ D- Total

T+ 370 2 372

T- 4 624 628

Total 374 626 1000




Sensitivity1. Ability of test to identify correctly those who have the disease2. Proportion of diseased people correctly identified as ¶positive· by the test3. P(T+|D+)4. TP / TP + F N5. ´SNoutµ = Sensitive tests rule out disease

Specificity1. Ability of test to identify correctly those who do not have the disease2. Proportion of non-diseased people correctly identified as ¶negative· by the test3. P(T-|D-)4. TN / TN + F P5. ´SPinµ = Specific tests rule in disease

TestResult


D+ D- Total

T+ TP F P TP + F P

T- F N TN F N + TNTotal TP + F N F P+ TN



V ALIDITY

If you had to choose: higher Se or Sp??Depends on objective

y Sensitivity desired at expense of specificity when thedisease is serious and curable in preclinical phase

y High specificity desired over sensitivity when the costs orrisks of further testing are significant e.g. biopsy



M ULTI -S TAGE TESTING

HIV Rapid Test is for pre-screeningUltimately two groups formed by screening test

y People with + test

y People with - testWhat to do next?

y Second-stage sequential testy Is parallel-stage testing better or worse?



S EQUENTIAL TESTING

372 individuals brought back for second testNew test with different Sn, SpNet Sensitivity ?Net Specificity ?

TestResult


D+ D- Total

T+ 370 2 372

T- 4 624 628

Total 374 626 1000







P ARALLEL TESTING

Net Gain in Sn; Net Loss in SpIntuitive:

y Two opportunities (tests) to ´pick up onµ the diseasei.e. easier to rule out disease (Sn)

y Harder to rule in disease (Sp) since two tests must agree

TestResult


D+ D- Total

T+ TP F P TP + F P

T- F N TN F N + TN

Total TP + F N F P+ TN



P REDICTIVE V ALUE

Ability to predict disease presence and absencebased on test results

Two components:y Positive Predictive Valuey Negative Predictive Value



P REDICTIVE V ALUE - E XAMPLE

HIV Rapid Test clinical trial resultsCalculate and interpret PPV and NPV

TestResult


D+ D- Total

T+ 370 2 372

T- 4 624 628

Total 374 626 1000




Positive Predictive Value (PPV) = 370 / 372Negative Predictive Value (NPV) = 624 / 628

TestResult


D+ D- Total

T+ 370 2 372

T- 4 624 628

Total 374 626 1000




PPV1. Ability to predict if a disease is present given that the test is positive2. Proportion of people testing positive for the disease that were actually

diseased3. P(D+|T+)4. TP / TP + F P

Specificity1. Ability to predict if a disease is not present given that the test is negative2. Proportion of people testing negative for disease that were actually non-

diseased3. P(D-|T-)4. TN / TN + F N

TestResult


D+ D- Total

T+ TP F P TP + F P

T- F N TN F N + TNTotal TP + F N F P+ TN



P REDICTIVE V ALUE AND P REVALENCE

Prevalence = 374 / 1000 = 37.4%What happens to PPV and NPV when prevalenceincreases?

TestResult


D+ D- Total

T+ 370 2 372

T- 4 624 628

Total 374 626 1000



P REDICTIVE V ALUE AND P REVALENCE

Increasing prevalence to 50% = 500 / 1000 with sameSe, Se, PPV increases from 495 / 497 (diff. of 0.1%)Intuitive

y More diseased cases in population means greater chancethat a positive test means true disease

Screening most productive for high-risk populations

TestResult


D+ D- Total

T+ 495 2 497

T- 5 498 503

Total 500 500 1000



P REDICTIVE V ALUE AND S PECI F ICITY

Increasing specificity increases PPV

Intuitivey If more cases ruled in (´SPinµ) then ability of a positive test

to correctly predict disease is enhancedSp has much greater impact on PPV than Se



RELIABILITY

Ability of a test to replicate results when repeated

F our components of variability:y Patient variationy Test variationy Intra-observer variationy Inter-observer variation



REVIEW ² S CREENING TEST CHARACTERISTICS

Validityy Sensitivityy Specificity

Multi-Stage Testingy

Sequentialy Parallel

Predictive Valuey Positive Predictive Valuey Negative Predictive Valuey Effects of prevalence & specificity

Reliabilityy Patient variationy Test variationy Intra-observer variationy Inter-observer variation



S CREENING T EST E VALUATION



B IAS

Systematic error in the design, conduct or analysis of a study resulting in a mistaken estimate of therelationship between exposure and disease

Distinguish from random error associated withstatistical sampling (imagine ínfinitely large studyµ)



P RO BLEM W ITH B IAS

Makes it difficult to assess the attribute: ´ Improveshealth outcomes (evidence)µ

y Is screening test doing goody Is screening test doing no harm



Selection bias

Recall bias

Measurement biasInformation bias

Length bias

Sampling bias

Referral bias

O utcome bias

Lead time biasSurveillance bias

Surrogate bias

Reporting bias

T YPES OF B IAS

How to criticallyappraise

multiple sources of bias?



Critical Appraisal

Internal Validity

Error

Systematic

Selection bias

Measurementbias

RandomEffect

modification

Confounding

Association

Causation

External Validity

F O RESHAD O WF RAMEW O RK : A PPR O ACH T O CRITICAL A PPRAISAL

Fu ndamentals of Epidemiology, grad u ate co u rse, University of Calgary, 2009.



O ccurs when a test diagnoses a diseaseearlier but there is no effect of Rx on outcomeof disease. Survival falsely appears prolonged

S O URCES OF B IAS IN S CREENING

O ccurs when a test favours diagnoses of diseases with naturally long pre-clinicalphases. Survival falsely appears prolonged.

Pre-clinical Clinical

Pre-clinical Clinical

Dz #1

Dz #2

UsualDx

UsualDeath

DelayedDeath

EarlyDx

Lead time bias:

Length time bias:



REVIEW ² S CREENING TEST E VALUATI O N

Definition

Types of bias

Sources of bias in screeningy Lead time biasy Length time bias



RE F ERENCES

Gordis, L. Épidemiologyµ, 4 th Edition, SaundersElsevier, 2009.

Rothman, K. Épidemiology, an Introductionµ, O xfordUniversity Press, 2002.

´Principles of Epidemiology in Public Health Practiceµ,3 rd Edition, Self-study course SS1000, Centers forDisease Control and Prevention

Herman CR, Gill HK, Eng J, F ajardo LL. Screening forpreclinical disease: test and disease characteristics. AmJ Roentgenol. 2002;179:825-831.

Documents

20100707 - Attributes of Screening Tests (Revised)