A Computerized/Web-Based Test to Assess the Continuum of Normal Memory/Cognition to

Dementia

J. Wesson Ashford, M.D., Ph.D.Stanford/VA Aging Clinical Research Center

Palo Alto, California

American Association of Geriatric PsychiatryMarch 6, 2009

Need for Mass Screening

• Alzheimer’s disease, dementia, and memory problems are difficult to detect when they are mild

• There are important accommodations and interventions that should be made when there are cognitive impairments– (like needing glasses or having driving

restrictions if you have vision problems)

Issues for Memory Screening• Current testing for memory problems is

based on having a tester sit in front of a subject for a prolonged period of time and administer unpleasant tests

• Testing must be – Inexpensive (minimal need for administrator)– Fun (so people will return for frequent testing)– More precise, reliable, and valid

• To improve sensitivity• To improve specificity

Audience Screening

• Presentation of complex pictures (that are easily remembered normally) are useful for detecting memory difficulties

• Testing memory using a pictures approach needs standardization for population use

• Picture memory is less affected by education

• Picture memory can be tested by computer

• Audiences can be shown slide presentations

IRB FORM (exempt)Screening for Memory Problems

Information Sheet• Description: You are invited to participate in a research study of memory and aging. You will take

a memory test that involves looking at a number of pictures and indicating which are duplicated. You may also be asked to remember a list of words, or to take other brief memory tests. If the results of these tests indicate that you may have some memory concerns, we may offer you the opportunity to participate in more detailed memory studies.

• For Questions About This Study: If you have any questions about this study, please contact: Dr. Wes Ashford, Aging Clinical Research Center, 3801 Miranda Avenue (151Y), VA Palo Alto Health Care System, Palo Alto, CA 94304, (650) 852-3287.

• Purpose: This is a research program to screen for memory problems. Information we collect about you will be added to information about other people and analyzed to help researchers and clinicians better understand how memory changes with aging. The results of this research study may be presented at scientific or medical meetings or published in scientific journals. However, personal information or your identity will not be disclosed. Your participation in this research study will take approximately 30 minutes to one hour.

• Participation is Voluntary: If you have read this form and have decided to participate in this project, please understand your participation is voluntary and you have the right to withdraw your consent or discontinue participation at any time without penalty or loss of benefits to which you are otherwise entitled. You have the right to refuse to answer particular questions. Your individual privacy will be maintained in all published and written data resulting from the study.

• Independent Contact: If you are not satisfied with the manner in which this study is being conducted, or if you have any concerns, complaints, or general questions about the research or your rights as a research study subject, please contact the Stanford Institutional Review Board (IRB) to speak to an informed individual who is independent of the research team at (650)-723-5244 or toll free at 1-866-680-2906. Or write the Stanford IRB, Administrative Panels Office, Stanford University, Stanford, CA 94305-5401.

Demographic Information (front of sheet)(circle, check, or fill in answers)

Month of Birth: Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov DecYear of Birth: 19 _____ ____Gender: Male __________ Female__________ Education Level (number of years): ________1 Elementary (0-6) 2 Jr. High School (7-8) 3 High

School (12)4 Some College (13-15) 5 AA Degree (14) 6 Bachelors Degree (16) 7 Masters Degree (18) 8 PhD, MD, JD (20+) 9 Other or unknownVeteran Status: Vet_____ NonVet______ Unknown_____Occupation: _______1 Never employed 2 Student 3 Homemaker4 Unskilled/semiskilled 5 Skilled trade/craft 6 Clerical/office Worker7 Manager Business/Gov 8 Professional/Technical 9 Other or UnknownEmployment Status: Full Time____ Part Time____ Unemployed____ Retired____Primary Race: __________1 White 2 Black or African American3 American Indian or Native Alaskan 4 Native Hawaiian or other Pacific Islander5 North Asian (i.e., China, Japan) 6 Middle East/South Asian7 More than one race 8 Decline to state9 UnknownHispanic Ethnic Background? No____ Yes____ Unknown____ Have you had problems with your memory recently? Yes_______ No__________

Answer Sheet for Memory Screening (back of sheet)Carefully look at each picture. If you see a picture that you have seen before, mark the circle next to the number of the repeat picture. For the main test, you will see 50 pictures. Each picture is numbered. The pictures will stay on the screen for 5 seconds. 25 pictures are new, 25 pictures are repeated.

MEMTRAX Memory Test MEMTRAX Memory Test (Power-Point Presentation)(Power-Point Presentation)

Selection of Subjects for Analysis

• Eliminated subjects - about 50about 50– 20 had marks that were un-interpretable20 had marks that were un-interpretable– 31 were below chance (12/25 or less) on the 31 were below chance (12/25 or less) on the

true negative or true positive scoretrue negative or true positive score• True-: 3 males, 10 females True-: 3 males, 10 females • True+: 8 males, 11 femalesTrue+: 8 males, 11 females

• Included subjects - 1018

Group Size

AGE (years)

Genderm/f

Ed (years) True-% True+%

#reports 1018 971 947 941 1018 1018

Mean 39/site 74.6 295/652 16 90.2% 92.3%

St Dev 34 13.8 2.5 10.3% 13.7%

Range 9 - 142 20-98 6-21 50-100 50-100

Study PopulationIncludes only subjects who performed

non-randomly on the test (1018)- # of reporting individuals 7/11/2007 – 8/14/2009 at 26 sites -

(not all subjects answered every question)805 reported being “white”

31 reported they were less than 40y/o

Test Performancefor 1018 subjects

• 82 (8%) had perfect scores,82 (8%) had perfect scores,

• 230 (23%) made 1 error (98% correct), 230 (23%) made 1 error (98% correct),

• 700 (69%) made 5 or fewer errors (700 (69%) made 5 or fewer errors (>>90% correct),90% correct),

• 132 (13%) made 6 – 10 errors (80 – 88% correct),132 (13%) made 6 – 10 errors (80 – 88% correct),

• 186 (18%) made > 10 errors (<80% correct).186 (18%) made > 10 errors (<80% correct).

----------------------------------------------------------------------------------------------------------------------------------------

• 70 (7%) scored < 80% correct for True Negatives70 (7%) scored < 80% correct for True Negatives– 19 (6%) males, 51 (8%) females 19 (6%) males, 51 (8%) females

• (false positive responses = saying a picture is repeated when not),(false positive responses = saying a picture is repeated when not),

• 79 (8%) scored < 80% correct for True Positives79 (8%) scored < 80% correct for True Positives– 25 (7%) males, 54 (8%) females 25 (7%) males, 54 (8%) females

• (false negative responses = failure to recognize/recall repeat picture).(false negative responses = failure to recognize/recall repeat picture).

Key to Plots

• Plots are shown for the 858 individuals with Plots are shown for the 858 individuals with age, gender, education data, age, gender, education data,

• First plot shows responses by item (1 - 50)First plot shows responses by item (1 - 50)– Red is first presentation (no mark required), Red is first presentation (no mark required), – Green is repeat (mark required),Green is repeat (mark required),

• Second plot shows performance by genderSecond plot shows performance by gender– Males in blue, Males in blue, – Females in pinkFemales in pink

Percent Correct by Item for Males (40-95 y/o, m=77.7, n=275)

0

2

4

6

8

10

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49

Item Number

Pe

rce

nt

Co

rre

ct

Percent Correct by Item for Females (40-97 y/o, m=74.9, n=597)

0

2

4

6

8

10

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49

Item Number

Pe

rce

nt

Co

rre

ct

New Pictures - Males, Females

0

2

4

6

8

10

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49

Item Number

Pe

rce

nt

Co

rre

ct

Repeat Pictures - Males, Females

0

2

4

6

8

10

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49

Item Number

Pe

rce

nt

Co

rre

ct

Plots of Performanceby Age for each Gender

• Percent correct, linear-linear plot• Percent incorrect, linear-linear plot• Percent incorrect, log-linear plot

• captures a greater proportion of the variance than the linear – linear plots – consistent with an exponential increase of failures with age – as predicted by the Gompertz Law

• Note scores less than 20 indicate less than 80% correct for that aspect of performance.

True Negative Performance

y = -0.0352x + 25.564

R2 = 0.039

y = -0.0597x + 27.24

R2 = 0.141

1213141516171819202122232425

40.0 50.0 60.0 70.0 80.0 90.0 100.0

Age (years)

Nu

mb

er

Co

rre

ct

Male true-

Female true-

Linear (Male true-)

Linear (Female true-)

True Positive Performance

y = -0.0438x + 27.029

R2 = 0.0617

y = -0.0418x + 26.746

R2 = 0.0605

1213141516171819202122232425

40.0 50.0 60.0 70.0 80.0 90.0 100.0

Age (years)

Nu

mb

er

Co

rre

ct

Male true+

Female true+

Linear (Male true+)

Linear (Female true+)

False Positive (incorrect guesses)

y = 0.0352x - 0.5641

R2 = 0.039

y = 0.0597x - 2.2399

R2 = 0.141

0

2

4

6

8

10

12

40.0 50.0 60.0 70.0 80.0 90.0 100.0

Age (years)

Nu

mb

er

Wro

ng

Male False+

Female False+

Linear (Male False+)

Linear (Female False+)

False Negatives (memory failures)

y = 0.0438x - 2.0289

R2 = 0.0617

y = 0.0418x - 1.7456

R2 = 0.0605

0

2

4

6

8

10

12

40.0 50.0 60.0 70.0 80.0 90.0 100.0

Age (years)

Nu

mb

er

Wro

ng

Male False-

Female False-

Linear (Male False-)

Linear (Female False-)

False Positives (incorrect guesses) - exponential trend line

y = 0.0129x - 0.0019

R2 = 0.0559

y = 0.0218x - 0.6245

R2 = 0.1896

0

0.5

1

1.5

2

2.5

3

40.0 50.0 60.0 70.0 80.0 90.0 100.0

Age (years)

LN

(N

um

ber

Wro

ng

)

Male False+

Female False+

Log Male False +

Log Female False +

False Negatives (memory failures) - exponential trend line

y = 0.0165x - 0.6334

R2 = 0.0699

y = 0.0142x - 0.4357

R2 = 0.0663

0

0.5

1

1.5

2

2.5

3

40.0 50.0 60.0 70.0 80.0 90.0 100.0

Age (years)

LN

(N

um

be

r W

ron

g)

Male False-

Female False-

Log Male False -

Log Female False -

Plots of Performanceby Education

• Note no significant effect of education

False Positives (incorrect guesses)

y = -0.0935x + 3.7674

R2 = 0.0153

y = -0.021x + 2.5605

R2 = 0.0007

0

2

4

6

8

10

12

6 8 10 12 14 16 18 20

Education (years)

Nu

mb

er

Wro

ng

Male False+

Female False+

Linear (Male False+)

Linear (Female False+)

False Negatives (memory failures)

y = -0.0042x + 1.4457

R2 = 3E-05

y = -0.0398x + 1.255

R2 = 0.02

0

2

4

6

8

10

12

6 8 10 12 14 16 18 20

Education (years)

Nu

mb

er

Wro

ng

Male False-

Female False-

Linear (Male False-)

Linear (Female False-)

Summary of Findings

• There is minimal difference in performance of individual items between males and females, in spite of significant “male-role” and “female-role” items.

• There is a significant decline of function with age on both aspects of the test, with the age-effect best explained by an exponential increase of errors with age (“Failure Theory”).

• Females had a greater association of false-positive errors with age than males, while the false-negative error association with age was similar by gender.

• Education was not significant in performance

Implications of Findings

• MemTrax is a brief, convenient, fun, audience-pleasing test of the type of complex memory affected by Alzheimer pathology

– an “audience-voting” system is needed (can also do web or computer-base)

• Recognition failure (False-) indicates failure of learning circuits

– typical of Alzheimer’s disease memory difficulty

• 80 – 88% correct is suggestive of early, mild memory problems

• less than 80% suggests impairment of function consistent with dementia

• False-recognition (False+) responses are indicative that the subject is not paying attention and is failing to inhibit the recognition response

– suggests other types of psychopathology: e.g., fronto-temporal dementia

• MemTrax can test many levels of memory impairment

– but need to establish accuracy, validly, and reliability

• Alzheimer’s disease is not a dichotomous diagnosis but a continuum of impairment best assessed probabilistically using Item Response Theory (Modern Test Theory), requiring gradation of an individual’s function

– such testing is an excellent approach for the longitudinal automated assessment

Acknowledgements

• Pictures courtesy of Henry Bowles, of Bowles-Langley Technology

• Audience performance measurements coordinated by Emily Gere and associates

• Presentations coordinated by Deryl Wicks

• Data entry by Drs. Raj Sharma, Samina Gul

• Presentations under the auspices of the Stanford/VA Aging Clinical Research Center and approved by Stanford and VA IRBs

Tests Available On-Line

• www.medafile.com• www.memtrax.net/BLT • www.cogolog.com• www.cognitivelabs.com• www.ibaglobal.net/BLT

• For further information, contact:– Wes Ashford: washford@medafile.com