Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
Criterion based Validity
Dr. Alaa El Dine HassanMB BCh, MSc (PH), MD
Assist. Professor - Community and Family
Medicine
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
The Policeman’s The Policeman’s capability of detecting capability of detecting thieves is in challengethieves is in challenge
3 thieves3 thieves
6 Good People6 Good People
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
ThievesThievesGood Good
PeoplePeople
33 66
We Have: – 3 thieves – 6 good People
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
4 In Prison4 In Prison
5 Free5 Free
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
ThievesThievesGood Good
PeoplePeople
In In PrisonPrison 44
FreeFree 55
33 66
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
4 In Prison4 In Prison
5 Free5 Free
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
ThievesThievesGood Good
PeoplePeople
In In PrisonPrison 44
FreeFree 55
33 66
22
44
How Many good People Free?
How Many Thieves in Prison?
22
11
False Classification?
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
4 In Prison4 In Prison
5 Free5 Free
The policeman Correctly Identified:– 2 out of 3 thieves (66.66%)– 4 out of 6 Good People (66.66%)
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
• A new instrument is released in the market which is claimed to identify people with lung cancers.
• The instrument is supposed to give:– “Positive reading” with diseased individuals
and – “Negative reading” with Non Diseased individuals
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
• A young motivated physician wishes to evaluate the performance of the instrument.
• He applies the instrument to 100 persons of whom – 30 are known to be diseased (Diagnosed lung
cancer Cases) – 70 are known to be not diseased
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
DiseasedDiseasedNot Not
DiseasedDiseased
3030 7070
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
• The instrument gave Positive reading with 35 subjects and Negative readings with 65 Subjects.
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
DiseasedDiseasedNot Not
DiseasedDiseased
Test Test PositivePositive 3535
Test Test NegativeNegative 6565
3030 7070
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
• Out of the 35 subjects who were labeled Positive, 25 were actually Diseased.
• Out of the 65 subjects who were labeled Negative , 60 were truly disease free.
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
DiseasedDiseasedNot Not
DiseasedDiseased
Test Test PositivePositive 3535
Test Test NegativeNegative 6565
3030 7070
2525
6060
True Positives (TP)
True Negatives (TN)
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
• By Simple subtraction , we can deduce that 5 diseased subjects were falsely labeled Negative
• And 10 Non Diseased Subjects were falsely labeled Positive
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
DiseasedDiseasedNot Not
DiseasedDiseased
Test Test PositivePositive 3535
Test Test NegativeNegative 6565
3030 7070
2525
6060
True Positives (TP)
True Negatives (TN)
1010
55
False Positives (FP)
False Negatives (FN)
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
• So the test was able to correctly identify 25 out of 30 Diseased individuals (83.33%)
• 60 out of 70 Non Diseased individuals (85.7%)
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
Sensitivity
• The sensitivity is how good the test is at picking out patients with disease.
• = the proportion of patients with disease who test positive.
• = the proportion of cases labeled Positive by the test, relative to all cases that actually have the disease:
= TP / All Diseased= TP / (TP + FN).
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
DiseasedDiseasedNot Not
DiseasedDiseased
Test Test PositivePositive
TP+FPTP+FP
Test Test NegativeNegative
TN+FNTN+FN
TP+FNTP+FN(All Diseased)(All Diseased)
TN+FPTN+FP(All Disease Free)(All Disease Free)
TPTP
TNTN
FPFP
FNFN
Sensitivity = TP / (TP + FN)
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
Specificity
• Specificity is the ability of the test to pick out patients who do NOT have the disease.
• is the proportion of patients without disease who test negative.
• = TN / All Non Diseased• = TN / (TN + FP).
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
DiseasedDiseasedNot Not
DiseasedDiseased
Test Test PositivePositive
TP+FPTP+FP
Test Test NegativeNegative
TN+FNTN+FN
TP+FNTP+FN(All Diseased))(All Diseased))
TN+FPTN+FP(All Disease Free)(All Disease Free)
TPTP
TNTN
FPFP
FNFN
Specificity = TN / (TN + FP)
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
In real Life practice• Tests have false positive and false negative results,
so……
• If a patient who has undergone an ELISA test for HIV, tested positive…….
• How confident will you be that he is really infected?
• What is the probability of being REALLY infected with HIV?
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
Predictive value of a positive test
• Is the probability that a person is Diseased when a positive test result is observed.
• Is the proportion of patients with positive tests who have disease:
= = TP / All Test PositivesTP / All Test Positives= TP / (TP + FP).
• This is the same thing as posttest probability of disease given a positive test.
• It measures how well the test rules in disease.
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
DiseasedDiseasedNot Not
DiseasedDiseased
Test Test PositivePositive
TP+FPTP+FP(All Test Positive)(All Test Positive)
Test Test NegativeNegative
TN+FNTN+FN(All Test Negative)(All Test Negative)
TP+FNTP+FN TN+FPTN+FP
TPTP
TNTN
FPFP
FNFN
PPV = TP / (TP + FP)
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
Lung Cancer Example Revisited
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
DiseasedDiseasedNot Not
DiseasedDiseased
Test Test PositivePositive 3535
Test Test NegativeNegative 6565
3030 7070
2525
6060
True Positives (TP)
True Negatives (TN)
1010
55
False Positives (FP)
False Negatives (FN)
Predictive value of Positive Test
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
Predictive value of Positive test
= TP / All test Positives
= 25 / 35
= 71.4%
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
So if a patient of unknown status undergoes the
test and gets a Positive Reading, There is a
71.4% Possibility of being truly Diseased.
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
• Predictive value of a negative test is the proportion of patients with negative tests who do not have disease:
= TN / All Test Negatives= TN / All Test Negatives= TN / (TN + FN).
• It measures how well the test rules out disease.
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
DiseasedDiseasedNot Not
DiseasedDiseased
Test Test PositivePositive
TP+FPTP+FP(All Test Positive)(All Test Positive)
Test Test NegativeNegative
TN+FNTN+FN(All Test Negative)(All Test Negative)
TP+FNTP+FN(All Diseased)(All Diseased)
TN+FPTN+FP(All Disease Free)(All Disease Free)
TPTP
TNTN
FPFP
FNFN
NPV = TN / (TN + FN)
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
Lung Cancer Example Revisited
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
DiseasedDiseasedNot Not
DiseasedDiseased
Test Test PositivePositive 3535
Test Test NegativeNegative 6565
3030 7070
2525
6060
True Positives (TP)
True Negatives (TN)
1010
55
False Positives (FP)
False Negatives (FN)
Predictive value of Negative Test
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
Predictive value of Negative test
= TN / TN + FN
= 60 / 65
= 92.3%
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
So if a patient of unknown status undergoes the
test and gets a Negative Reading, There is a
92.3% Possibility of being truly Disease free.
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
Positive Likelihood Ratios
• likelihood ratios are not yet commonly reported in peer-reviewed literature or in marketing information provided by test manufacturers.
• They can be a valuable tool for comparing the accuracy of several tests to the gold standard, and
• They are NOT dependent upon the prevalence of disease.
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
The positive LR represents :
the odds of a positive test in an infected population compared to
the odds of a positive test a non infected population.
TP/ All Diseased TP/ All Diseased ————--------—————--------—
FP/ All Disease freeFP/ All Disease free
Or it can also be expressed as :Or it can also be expressed as : sensitivity sensitivity——————————
1-specificity1-specificity
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
DiseasedDiseasedNot Not
DiseasedDiseased
Test Test PositivePositive
TP+FPTP+FP(All Test Positive)(All Test Positive)
Test Test NegativeNegative
TN+FNTN+FN(All Test Negative)(All Test Negative)
TP+FNTP+FN(All Diseased)(All Diseased)
TN+FPTN+FP(All Disease Free)(All Disease Free)
TPTP
TNTN
FPFP
FNFN
Positive Likelihood Ratio
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
• Useful tests have larger positive LR • Less useful tests will have smaller positive LR.
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
Example:
A test with a positive LR of 5.0 :– For every
• 1 non-infected subjects that test positive • 5 of the infected subjects will test positive.
– For every • 1 false positive• 5 true positives
– True positives are 5 times False Positives
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
The Negative LR represents :
the odds of a the odds of a Negative testNegative test in an in an infected populationinfected population compared to compared to
the odds of a the odds of a Negative testNegative test in a in a non infectednon infected population population.
FN/ All DiseasedFN/ All Diseased------------—————------------—————TN/ All Disease freeTN/ All Disease free
Or it can also be expressed as :Or it can also be expressed as : 1- Sensitivity 1- Sensitivity
——----------------——----------------——SpecificitySpecificity
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
DiseasedDiseasedNot Not
DiseasedDiseased
Test Test PositivePositive
TP+FPTP+FP(All Test Positive)(All Test Positive)
Test Test NegativeNegative
TN+FNTN+FN(All Test Negative)(All Test Negative)
TP+FNTP+FN(All Diseased)(All Diseased)
TN+FPTN+FP(All Disease Free)(All Disease Free)
TPTP
TNTN
FPFP
FNFN
Negative Likelihood Ratio
Dr. Alaa Hassan - MB BCh, MSc (PH), MDDr. Alaa Hassan - MB BCh, MSc (PH), MD
• Useful tests have negative LR close to 0• Less useful tests have higher negative LR.