View
225
Download
0
Category
Tags:
Preview:
Citation preview
Statistical Issues for DevelopingAlzheimer Screening Tests
J. Wesson Ashford, M.D., Ph.D.Stanford / VA Alzheimer’s Center
VAMC, Palo Alto, California&
Helena C. Kraemer, Ph.D.Department of Psychiatry and Behavioral Sciences
Stanford University, Palo Alto, California
June 19, 2005Washington, D.C.
Slides at: www.medafile.com (Dr. Ashford’s lectures)
Screening for Alzheimer’s Disease
What does it mean?
• Annual assessments (or bi- or semi-)• Positive screen recommends more
tests• Contrast with current lack of system
– 50% not diagnosed until moderate AD
• Screening will progressively improve• Change over time can be detected
AD Can Be Readily Diagnosed
• A diagnosis of Alzheimer’s disease can be made with a high degree of certainty
• Using NINCDS-ADRDA criteria, accuracy in autopsy-verified cases is approximately 90%
• Diagnosis is a 2-step process:– Detection through screening (test vs. family
concern)– Confirmation through patient history and
physical, caregiver interview, brain imaging, and appropriate laboratory studies
McKhann G et al. Neurology. 1984;34:939-944. Kazee AM et al. Alzheimer Dis Assoc Disord. 1993;7:152-164.Ashford JW et al, Psychiaric Annals, 1996;26:262-268.
AD is Under-diagnosed• Early Alzheimer’s disease is subtle, the diagnosis
continues to be missed – it is easy for family members to avoid the problem and
compensate for the patient – physicians tend to miss the initial signs and symptoms
• Less than half of AD patients are diagnosed– Estimates are that 25% to 50% of cases remain
undiagnosed– Diagnoses are missed at all levels of severity: mild,
moderate, severe
• No definitive laboratory test for diagnosing AD exists– Efforts to develop biomarkers, early recognition by
brain scan have not provided a screening methodology
Evans DA. Milbank Quarterly. 1990; 68:267-289
Reasons to Diagnose Alzheimer’s Disease Early
Social• Undiagnosed AD patients face avoidable
problems – social, financial
• Early education of caregivers– how to handle patient (choices, getting started)
• Advance planning while patient is competent– will, proxy, power of attorney, advance directives
• Reduce family stress and misunderstanding– caregiver burden, blame, denial
• Promote safety– driving, compliance, cooking, etc.
• Patient’s and Family’s right to know– especially about genetic risks
• Promote advocacy– for research and treatment development
Reasons to Diagnose Alzheimer’s Disease Early
Medical• Early diagnosis and appropriate intervention may lessen
disease burden and early treatment may improve overall course substantially– Neurophysiological pathways in patients with AD are
still viable and are a target for treatment
• Specific treatments now available (anti-cholinesterases, memantine)– Improve cognition– Improve function (ADLs)– Delay conversion to AD from Mild Cognitive
Impairment– Slow underlying disease process, the sooner the better– Decreased development of behavior problems– Delay nursing home placement, possibly over 20
months– Delay nursing home placement longer if started earlier
UNDERLYING CONTINUUM OF ALZHEIMER SEVERITY- progressive disruption of neuroplasticity -
(Ashford & Jarvik, 1985; Ashford et al., 1995; Ashford, 2004)
?? unidimensional, but how to measure ??
• CROSS-SECTIONAL MEASURES– DEMENTIA SEVERITY (cognitive, ADL)
• COGNITIVE SCALE SCORE• Z-SCORE• PRINCIPAL COMPONENT ANALYSIS
– BRAIN ATROPHY, DYSFUNCTION– AUTOPSY MEASURES: plaques, tangles– TIME TO DEATH
• LONGITUDINAL MEASUREMENT– TIME INTO THE DISEASE PROCESS
• Considerable heterogeneity in disease clinical presentation and brain distribution
Estimate MMSE as a function of time
0
5
10
15
20
25
30
-10 -8 -6 -4 -2 0 2 4 6 8 10
Estimated years into illness
MM
SE
scor
e
AAMI / MCI/ early AD -- + -- DEMENTIA
ALZHEIMER’S DISEASE CONTINUUM
Ashford et al., 1995; Ashford & Schmitt, 2001
derived from CERAD dataset
Alzheimer’s Disease CategoriesNORMAL
MILD COGNITIVE IMPAIRMENT (MCI) CRITERIA1. Memory complaint, preferably corroborated by an
informant2. Objective memory impairment3. Normal general cognitive function4. Intact activities of daily living5. Not demented
American Academy of Neurology
Petersen et al., 2001 – Neurology 56:1133
Earlier descriptions by: Jonker, Hooyer, 1990 Flicker, Ferris, Reisberg, 1991 Zaudig, 1992
DEMENTIA OF ALZHEIMER TYPE (DSM-IV - APA)Kraemer et al., 2004
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Age
Per
cen
tag
e
AD
MCI
Non-Affected
Yesavavage et al., 2002Markov Chain model
ALZHEIMER’S DISEASE CATEGORIZATION
Current Approaches to Early Assessment
• Genetic vulnerability testing (trait not diagnosis)• Vulnerability factors
– education, occupation, head injury (not diagnosis)• Early concern
– Alzheimer Association’s 10 warning signs, ADL dysfunction• Dementia severity assessment tools (lack early
power)• Positive diagnostic tests (too invasive for screening)
– CSF – tau levels elevated, amyloid levels low– Brain scan – PET –
• NFTs: DDNP • Amyloid: Thioflavin-S, Congo-red derivatives
Need new screening tools (6th vital sign in elderly)
Available Short TestsUsed for Screening
• MMSE 10 -- 15 min• Too long
• 7-Minute Screen 7 – 10 min• Too complex
• Clock Drawing Test 2 – 4 min• Not sensitive
• Mini-cog 3 – 5 min• To be considered
• Memory Impairment Screen 4 min• To be considered
• Need to know the cost of administering the test• Need to have prospective measurement of
sensitivity and specificity for the target population
• Need to develop progressively better tests
Control: What happens without screening?
Total Population Risk=P
P
Have ADNo effective intervention
Do not have AD
P’
Helena Kraemer, 2003
Testing: What happens with testing?
Total Population
No ADAD
Unnecessary intervention OK No effective intervention Effective intervention
$ Testing $Testing $ Testing $ Testing$ Intervention $ Intervention
Iatragenic Damage? Clinical Wash Clinical Wash Clinical Gain
Major(?) Loss Minor (?) Loss Minor(?) Loss Major(?) Gain Some gain
PP’
SeSe’
SpSp’
Helena Kraemer, 2003
$W = Cost–Worthiness Calculation
• I = incidence (new occurrences each year, by age)• $T = cost of test, time to take (Subject, Tester)• Se = sensitivity of test = True positive / I• Sp = specificity of test = True negative / (1-I)• Cost:
– $B = benefit of a true positive diagnosis• Estimate: (100 years – age ) x $1000• Save $50,000 NH cost / 1year (after treatment cost deduction)
– $C = cost of a false positive diagnosis• $500 for further evaluation (time, stress of suspecting dementia)
– True negative (real peace of mind) (no price)– False negative = false peace of mind (no price)
$W = ($B x I x Se) – ($C x (1-I) x (1-Sp)) - $T
Kraemer, Evaluating Medical Tests, Sage, 1992
JW Ashford, MD PhD, 2003
U.S. mortality rate by age1999 CDC / 2000 census
0.0001
0.0010
0.0100
0.1000
1.0000
0 10 20 30 40 50 60 70 80 90 100Age
Yea
rly
Haz
ard
Males, 2t = 8.2yrsFemales, 2t = 7.5 yrsAlzheimer incidence
Cost-Worthy Test EvaluationBenefit = $50,000 - 0; False Pos = $500
-100-50
050
100150200250300350400450500550600
50 55 60 65 70 75 80 85 90 95
AGE
Cos
t Jus
tifie
d fo
r D
emen
tia S
cree
n .8, .8
.9, .9
.95, .95
1,1
Se, SpSe, Sp
Cost-Worthy Test EvaluationBenefit = $50,000 - 0; False Pos = $500
-20-10
0102030405060708090
100
55 60 65 70 75 80
AGE
Cos
t Jus
tifie
d fo
r D
emen
tia S
cree
n .8, .8
.9, .9
.95, .95
1,1
Sp, Se
Dementia rate, for Td = 5 yrs
0.0001
0.001
0.01
0.1
1
10
100
1000
50 60 70 80 90 100
Age
Nu
mb
er
of
peo
ple
/yr
mean rateAPOE 4/4APOE 3/4APOE 3/3presenilin
See: Raber et al., 2004
Cost-Worthy Test EvaluationSe=0.9; Sp=0.9
Benefit = $50,000 - 0; False Pos = $500
-1000
100200300400500600700800900
1000
50 55 60 65 70 75 80 85 90 95
AGE
Cos
t Jus
tifie
d fo
r D
emen
tia S
cree
n mean
ApoE 4/4
ApoE 3/4
ApoE 3/3
Cost-Worthy Test EvaluationSe=0.9; Sp=0.9
Benefit = $50,000 - 0; False Pos = $500
0102030405060708090
100
50 55 60 65 70 75 80
AGE
Cos
t Jus
tifie
d fo
r D
emen
tia S
cree
n mean
ApoE 4/4
ApoE 3/4
ApoE 3/3
Animals named in 1 min (mms>19) - CERAD data set
0
2
4
6
8
10
12
0 10 20 30 40
number of animals named
pe
rce
nt
of
tota
l
Normal Controls, CS = 1, n = 386
Alzheimer patients, CS = 0, n = 380
Animal naming ROCs (AUC 15: 0.74; 30: 0.83; 45: 0.86; 60: 0.87)
5
6
7
8
7
8
9
10
11
.
10
11
12
13
1415
16 17
0
10
20
30
40
50
60
70
80
90
100
0 10 20 30 40 50 60 70 80 90 100
False Positive Rate (%)
Sens
itivity
(%)
animals in 15 secs
animals in 30 secs
animals in 45 secs
animals in 60 secs
(CERAD dataset)
BRIEF ALZHEIMER SCREEN (Normal vs Mild AD, MMS>19)
9
20
1413
1211
10
9
6
7
8
2627
25
0
10
20
30
40
50
60
70
80
90
100
0 10 20 30 40 50 60 70 80 90 100
False Positive Rate (%) (1-Specificity)
Tru
e P
osi
tiv
e R
ate
(%
) (
Se
nsi
tiv
ity)
animals 1 m AUC = 0.868
animals 30 s AUC = 0.828
MMSE AUC = 0.965
Date+3 Rec AUC = 0.875
BAS AUC = 0.983
JW Ashford, MD PhD, 2003
Mendiondo et al., 2003
Issues in Screening• ROC analysis provides independent values of test performance
– how the screening test values affect the normal and patient populations– plots of their relationship with respect to each other (specificity vs sensitivity)– data must be derived from the represented population!!!
• The value of the test must be calculated with respect to the risk of the disease– In the specific population to which it is being applied– Risk is affected by age, genotype, many other factors– Accounting for a priori probability is Bayesian analysis
• The cost-benefit must be assessed: – Apply the test cost and the costs of false positive and false negative results– Apply the benefits of correct positive and negative results
• Alzheimer’s disease is not a dichotomous diagnosis but a continuum– Diagnosis would be better described with a probabilistic statement– Item Response Theory would better calculate probability (Modern Test Theory)
Item Response Theory and Factor Analysis allow combination of test components
Kraemer, 1992; Ashford & Schmitt, 2001
AD all (easiest to hardest at p=.5)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
-4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 10
DISABILITY ("time-index" year units)
PR
OB
AB
ILIT
Y C
OR
RE
CT
PENCILAPPL-REPWATCLOCATIONPENY-REPTABL-REPCLOS-ISRIT-HANDCITYFOLD-HLFSENTENCECOUNTYNO-IFSFLOORSEASONYEARPUT-LAPMONTHADDRESSDRAW-PNTDAYSPEL_ALLDATEAPPL-MEMPENY-MEMTABL-MEM
Mini-Mental State Exam items
MMSEitems
Based on Ashford et al., 1989; 1995; applied to CERAD data set
AD all (easiest to hardest at p=.5)
0.00
0.02
0.04
0.06
0.08
0.10
0.12
0.14
0.16
-4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 10
DISABILITY ("time-index" year units)
ITE
M IN
FO
RM
AT
ION
PENCAPWATCLOCAPETAREDORIGHCITYFOLDSENTCOUNPHRALEVESEASYEARALAPMONTADDRDRAWDAYASPEL_1DATEAPPLPENNTABL
See: Hambleton et al., 1990; Ashford & Schmitt, 2001)
AD all
0.00
0.10
0.20
0.30
0.40
0.50
0.60
-4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 10
DISABILITY SCALE
TES
T IN
FOR
MA
TIO
N
MMSE Item-Response Analysis
Summary Requirements for Screening Test Evaluation
• Sensitivity, specificity for the population to which test is to be applied– values change with level of disease being screened (must be
prospective)• Portion of population at risk
– Risks according to age, gender, genotype, family hx, education, etc.• Cost of test -$1, $10, $100, $1000
– Need brief screen to replace MMSE (ASAP) - $10 Medicare allowance• Costs of false positive, false negative tests• Benefits of true positive, true negative
– Longevity is increasing over time, needs to be factored in.• Cascaded tests – preliminary screen, confirmatory exam• Longitudinal tests may provide much more reliable information• Different tests for clinic (cascaded), research (outcomes)
– Clinic: brief screen, brain scan; Research: CSF - is disease stopped?• Benefit to society relative to other societal needs.
Dementia Screening Test Requirements for the Future
• For Alzheimer’s disease (other tests for non-AD)– Validated against more stringent tests – brain scans, CSF measures– Specificities and sensitivities valid for comparison with other tests– Item Response Theory analysis of discriminatory power on disability
continuum• Multiple platforms:
– Doctor’s offices– Best if computerized for rapid, objective assessment– World-Wide Web – based testing, – CD-distribution– KIOSK administration – drug stores, shopping malls
• Very brief (about 1-minute)• Multiple test forms
– so it can be repeated often, e.g., every 3 months• Cost-effective yearly after age 50
– repeatable every 3 months over 65 years of age or with concerns• Sensitive to change over time• Nominal cost
Recommended