Friday Harbor 2005

Cultural and Demographic Influences on

Neuropsychological Test Scores

Jennifer J. Manly, Ph.D.

Columbia University Medical Center

Friday Harbor 2005

National Institutes on Aging

R01 AG16206

Alzheimer’s Association

New York City Council Speaker’s Fund Award for Biomedical Research

Friday Harbor 2005

G.H. Sergievsky CenterTaub Institute for Research on

Alzheimer’s Disease and the Aging Brain

Yaakov Stern, Ph.D.Nicole Schupf, Ph.D.

Richard Mayeux, M.D.Danurys SanchezCherita McDowellJudes Fleurimont

Iris PaltinRaquel Cabo

Akanksha Thakur

Mount Sinai School of Medicine

Desiree Byrd, Ph.D.

Methods in the Social Sciences Program Department of Sociology

Christopher Weiss

Columbia University

The problem of low specificity of neuropsychological measures

• Several studies show reduced specificity of cognitive measures among African Americans and Hispanics

• Reduced specificity may result in increased misdiagnosis of cognitive impairment

• Ethnic differences occur across cognitive domains

SAN DIEGO AANP

neuropsychological

test % impaired using

existing norms Category 31 Block Design 37 Trails A Time 26 Digit Symbol 21 Digit Vigilance Time 18 Story Loss 32 Figure Loss 2 Story Learn 39 Figure Learn 58 Information 35 Boston Naming 64 Sens-Perc Errors 25

IMPAIRMENT RATES for NORMAL AFRICAN AMERICANS (n = 137)

An Early Deconstruction of Race

• Army Alpha & Beta tests

From Klineberg, 1935

Cultural effects on test performance:WAIS-III Full Scale IQ

0%5%

10%15%20%25%30%35%40%45%50%

Perc

ent M

iscl

assi

fied

White

African-American

Hispanic

Age Corrected Age, Education & Gender Corrected

From Heaton & Taylor

Friday Harbor 2005

INWOOD

WASHINGTON HEIGHTS

HAMILTON HEIGHTS• N = 2,801 in 1999

• Age 65 and older• Spanish or English speaking• Education range 0 - 20

0

10

20

30

40

50

Per

cent of co

hort

Caucasian AfricanAmerican

Hispanic

PI: Richard Mayeux

Age-Specific Prevalence of Alzheimer’s Disease (1990-1992)

BlacksCaribbeanHispanic Whites

65-74

75-84

85+

0

5

10

15

20

25

30

35

40

CDR1.0

Friday Harbor 2005

The Important Questions• How can we improve the accuracy of

neuropsychological detection of MCI and AD among ethnic minorities?

– Are there any measures that can be used across groups with high sensitivity and specificity?

– Are separate norms or tests for ethnic minorities the best solution?

– What are the cultural, linguistic, racial, educational, and socioeconomic influences on cognitive test performance?

• Is AD more frequent among ethnic minorities?– What design and populations would best address this question?– If so, what are the underlying causes of this discrepancy?– Can these underlying causes be treated or changed?

Friday Harbor 2005

Are there any measures that can be used across groups with high

sensitivity and specificity?

• Use culturally neutral or “culture-free” tests

• Follows belief that there are universal constructs of cognition that can be assessed accurately across cultures

Failure of Universalism

“…that a test or test battery is “culture free” is a description born in the never-never land of psychometric wishful thinking that at once brands the claim as spurious.”

Nell, 2000

Universalism

• Neuropsychologists are trained that their measures assess “pure” cognitive abilities

• Universalist belief allows us to avoid issues of construct validity

• Source of naitivist beliefs (vs. environmentalist claims)

• BUT... tests are bound to the culture in which they are developed and normed

WHO Battery• Designed for cross-national studies

• Tests were selected and created to be “culture-free”

• Large cultural differences appeared on several measures, independent of years of education Reaction time Manual dexterity Psychomotor speed (Digit Symbol & Color Trails) Digit Span

• All WAIS-III performance subtests

– Picture Arrangement

• Benton Visual Retention Test

• Simple Reaction Time tests

• Cancellation tests

Nonverbal is not culture free

Racial/ethnic differences are found on:

Friday Harbor 2005

Are separate norms for ethnic minorities the best solution?

• Collection of separate group norms is the most popular approach in the literature, especially among geriatric neuropsychologists

Friday Harbor 2005

No significant ethnic differences in frequency of MCI

0

2

4

6

8

10

12

14

MCI with memoryimpairment

MCI with no memoryimpairment

White

Black

HispanicPer

cent

Manly et al, Arch Neurol (in press)

Friday Harbor 2005

Disadvantages of separate norms

• Used as a proxy for assumed educational, socioeconomic, or behavioral differences

• Diversity within ethnic groups:– educational– geographical– economic– exposure to Mainstream culture

• Separate norms leave ethnic differences unexplained and open for misinterpretation

Friday Harbor 2005

US CENSUS CLASSIFICATIONS

500 different tribesAfrican American

HaitiJamaicaWest Indian

EuropeMiddle EastNorth Africa

CambodiaChinaIndiaJapanKoreaMalaysiaPakistanPhilippine IslandsThailandVietnam

HawaiiGuamSamoa

CubanMexicanPuerto Rican

DominicanSouth or Central AmericanOther Spanish origin

Native Hawaiian or

Pacific Islander

White

American Indian or Alaska Native

Black or African American

Asian

Hispanic or Latino

Friday Harbor 2005

What are the cultural, linguistic, racial, educational, and

socioeconomic influences on cognitive test performance?

• Deconstruct race and ethnicity

DECONSTRUCT RACE/ETHNICITY

• Measure factors that vary within and between ethnic groups

• Determine relationship to test performance

• Adjust for significant factors before interpreting scores, regardless of race

CULTURAL EXPERIENCE

• Acculturation = level at which values, language, and cognitive styles are shared with ethnic community versus those of the dominant culture

OPERATIONALIZING CULTURE

ACCULTURATION

TRADITIONAL ETHNIC

UNACCULTURATED

MAINSTREAM

ACCULTURATED

Friday Harbor 2005

OPERATIONALIZING CULTURE

High EthnicLow Ethnic

Low Majority

High Majority

“marginal”

acculturated

unacculturated

bicultural

African American Acculturation Scale

Preference For African American Things1. Most of the music I listen to is by Black artists.6. Most of my friends are Black.

Traditional Childhood17. I went to a mostly Black elementary school.19. I grew up in a mostly Black neighborhood.

Religious Beliefs And Practices10. I am currently a member of a Black church.12. The church is the heart of the Black community.

Selected items from “The African American Acculturation Scale II: Cross-validation and short form,” by H. Landrine and E.A. Klonoff, 1995, Journal of Black Psychology, 21, p. 136.

Hispanic Acculturation Scale

1. Por lo general, que idioma(s) lee y habla usted?

6. Por lo general, en que idioma(s) son los programas de television que useted ve?

17. Sus amigos y amigas mas cercanos son:

Selected items from Marin, Gerardo; Sabogal, Fabio; Marin, Barbara V; Otero-Sabogal, Regina; et al. Development of a short acculturation scale for Hispanics. Hispanic Journal of Behavioral Sciences. 9 ,1987.

Solo Latinos

Mas Latinos que Americanos

Casi mitad y mitad

Mas Americanos que Latinos

Solo Americanos

Solo

Español

Español mejor que Inglés

Ambos por igual

Inglés mejor que Español

Solo Inglés

Solo

Español

Español mejor que Inglés

Ambos por igual

Inglés mejor que Español

Solo Inglés

Puente & Ardila2000

• Acculturation may be most critical factor in assessment of Hispanics

• What we measure = whether patient understands and answers questions in a manner that examiner feels is “normal”

• This is determined by acculturation

• We do not directly determine if there is brain damage - we infer it through performance (acculturation)

San Diego African American Neuropsychological Norms

Project (AANP)

Exclusion criteria:current or past drug dependenceserious mental illnessHI with LOC > 5 minutesneurological disorderurine analysis performed day of testing

MEASURES

• Neuropsychological battery

• Acculturation– self report - African American Acculturation

Scale– linguistic - Black English use

BLACK ENGLISH USEan objective measure of

acculturation

Language sample recorded during Story Memory Test

Transcribed X 2 Good interrater reliability Coded for certain Black English variants

RESULTSAMONG NORMAL AFRICAN AMERICANS

Unacculturated African Americans obtained lower scores on measures of verbal skills, even after accounting for age, education, and gender

Linguistically unacculturated African Americans obtained lower scores on measures of nonverbal abstraction and verbal skills

San Diego HIV Neurobehavioral Research

Center (HNRC)

Exclusion criteria: non-HIV related neurological or medical

disorder past IV drug use substance dependence HI with LOC > 30 minutes serious mental illness

RESULTSAMONG HIV+ INDIVIDUALS

Self-reported acculturation accounted for existing ethnic group differences abstraction, visuospatial ability, figure learning,

verbal skill

NOT STORY LEARNING gist vs. detail

ELDERLY PARTICIPANTS

No stroke, Alzheimer’s or Parkinson’s disease, substance abuse (past or current), or major psychiatric illness

Nondemented based on neurologist diagnosis

MEASURES Neuropsychological battery

verbal & nonverbal learning and memory, orientation, abstraction, naming, fluency, comprehension, and visuospatial ability

Functional Capacity Blessed Functional Activities Scale Schwab & England ADL Scale

African American Acculturation Scale 33-item self report

Friday Harbor 2005

TEST Sex Age Years education

SRT total recall .001 .119 .166

SRT delayed recall .001 .070 .091

BVRT recognition memory .004 .059 .156

MMSE Orientation .004 .012 .020

WAIS-R Similarities Raw .005 .062 .319

DRS Identities & Oddities .005 .045 .077

Boston Naming .002 .025 .148

Letter fluency .002 .046 .216

Category fluency .000 .070 .184

BDAE repetition .004 .010 .057

BDAE Comprehension .003 .012 .096

Rosen Drawing .001 .036 .071

BVRT matching .001 .058 .139

Demographics and cognitive test performance (R2)

Elderly African Americansn = 503

Test R2 for acculturation

Memory

Total recall .03* Delayed recall .01 Benton recognition .03*

Orientation

Mini-Mental Status Exam .00 Abstract Reasoning

Similarities .06* Identities/oddities total .00

* p < .01

Test R2 for acculturation

Language

Naming total .05* CFL mean .03* Category fluency mean .03* Repetition .03* Comprehension .02* Visuospatial Ability

Rosen .04* Benton matching .04*

* p < .01

Elderly African Americansn = 503

Deconstructing Race• Cultural experience

– Acculturation level

• Educational experience– Quality of education

EDUCATIONElementary in South 77%

Of high school educated, HS in South 62%

Of college educated, college in South 80%

Rural elementary 46%

Rural high school 15%

Mostly Black (>90%) elementary 84%

Mostly Black high school 64%

Friday Harbor 2005

Friday Harbor 2005

Per-pupil expendituresc. 1935

State Black White Ratio Alabama 17.50 53.18 .33 Florida 17.71 39.80 .45 Maryland 80.63 102.84 .78 Mississippi 13.36 58.61 .23 N. Carolina 32.92 51.43 .64 S. Carolina 18.62 67.74 .28 Virginia 33.05 63.81 .52 New York 110.97 Pennsylvania 75.74

Friday Harbor 2005

Length of School Term

60

80

100

120

140

160

180

200

NC White

NC Black

SC White

SC Black

NY All

Friday Harbor 2005

Student Teacher Ratio

25

35

45

55

65

75NC White

NC Black

SC White

SC Black

NY All

Friday Harbor 2005

Person-specific teacher-student ratio

• Alabama, Georgia, North Carolina, and South Carolina

• Calculated using date of birth and number of years of school

• Average state-level student-teacher ratio for years the respondent was in school

• Ranges from a low of 27 to a high of 76

Friday Harbor 2005

Quality of education & reading level

• In 1960’s, economists began to relate school characteristics to achievement of students– Coleman report

• Achievement was measured with standardized tests of reading, math, and spelling– Reading level is relatively stable across adulthood &

early dementia

– We could use reading level to estimate quality of school among elders

English Reading Level

Wide Range Achievement Test - reading subtest

I V Z J Q see red milk was

between cliff stalk grunt

clarify residence urge rancid

conspiracy deny quarantine deteriorate

regime beatify internecine regicidal

puerile factitious lucubration

epithalamion inefficacious synecdoche

Friday Harbor 2005

WRAT-3 reading

Spanish Reading Level Woodcock Johnson Batería - Revised

O B S R U de la ….

medicoquigúrico

Friday Harbor 2005

Woodcock Johnson Batería - Revised

Spanish Reading Level Word Accentuation Test

ACULLA

ABOGACIA

ANOMALO

CELIBE

ALELI

RABI

APATRIDA

HUSAR

ALEGORIA

MANCHU

DIAMETRO

MOARE

CONCAVO

AMBAR

PUGIL

POLIGAMO

ACME

SILICE

GRISU

ALBEDRIO

CANON

PIFANO

TACTIL

VOLATIL

DESCORTES

DISCOLO

BULGARO

BALADI

ACOLITO

CUPULA

Friday Harbor 2005

Word Accentuation

Test

Friday Harbor 2005

Quality of school and current reading level

• Controlling for race, age, and unobserved state level factors (HLM)

• Higher student-teacher ratios associated with lower scores on the WRAT-3

( = -0.54, p <.001).

Self-reported WRAT-3 years of education

estimate 0-3 4-6 7-8 9-12 13+ 0-3 3 8 5 11 4-6 5 6 11 1 7-8 2 2 2 16 3 9-12 2 9 30 40 13+ 1 3 40 40

READING LEVEL AND YEARS OF EDUCATION

ETHNICITY, GRADE, AND READING LEVEL

African-American

non-Hispanic

White

N 125 74reading level = actual grade 29% 47%reading level > actual grade 24% 35%reading level < actual grade 47% 18%

Friday Harbor 2005

Reading ability accounts for ethnic group differences

Test

African American N = 192

White

N = 192

ANOVA

ANCOVA with

WRAT-3

Mean Mean F F Memory

SRT Total recall 39.8 43.5 12.9** 1.7 SRT Delayed recall 5.8 6.7 8.9* 0.7 Benton figure recognition 7.4 8.1 20.4** 2.5 Orientation

10 Items from MMSE 9.7 9.8 3.5 Abstract Reasoning

WAIS-R Similarities 12.0 16.0 34.8** 3.7 Identities/oddities total 14.6 15.1 10.8* 1.8

** p < .01; *** p < .001Manly et al, 2002, J Int Neuropsychol Soc 8: 341-348

Friday Harbor 2005

Reading ability accounts for ethnic group differences

Test African American N = 192

White

N = 192

ANOVA

ANCOVA with

WRAT-3

Mean Mean F F Language

15-item Boston Naming 14.0 14.1 0.5 Letter fluency 9.9 12.2 31.8** 10.0* Category fluency 14.6 16.8 31.4** 0.8 BDAE Repetition 7.8 7.8 0.0 BDAE Comprehension 5.5 5.8 13.4** 3.2 Visuospatial skill

Rosen Drawing 2.6 3.0 30.0** 10.9* Benton Figure Matching 8.9 9.4 13.1** 3.4

* p < .05; ** p < .01Manly et al, 2002, J Int Neuropsychol Soc 8: 341-348

Friday Harbor 2005

TEST Age Years education Reading level

SRT total recall .119 .166 .189

SRT delayed recall .070 .091 .105

BVRT recognition memory .059 .156 .211

MMSE Orientation .012 .020 .033

WAIS-R Similarities Raw .062 .319 .386

DRS Identities & Oddities .045 .077 .088

Boston Naming .025 .148 .316

Letter fluency .046 .216 .401

Category fluency .070 .184 .184

BDAE repetition .010 .057 .095

BDAE Comprehension .012 .096 .137

Rosen Drawing .036 .071 .116

BVRT matching .058 .139 .156

Demographics and cognitive test performance (R2)

Friday Harbor 2005

Quality of school and current cognitive ability

• Controlling for race and sex

• Student-teacher ratio was a significant predictors of performance on measures of memory ( = -1.08, p = .001), executive function ( = -.54, p < .001), and language ( = -.10, p = .005).

Statistical limitations• The following must be eliminated for

accurate “adjustment” for SES– Residual confounding due to categorization– Measurement error– Aggregation– Incommensurability

• In a large study, race effects will ALWAYS persist after adjustment, even if there is in fact no true difference between groups

Deconstructing Race• Cultural experience

– Acculturation level

• Educational experience– Quality of education – Literacy level

ILLITERACY AMONG ETHNIC MINORITY ELDERS

• Ethnic minority and immigrant elders had fewer opportunities to receive formal education

• Illiteracy rate is 1.4% among age 65+, five times higher than age 25-44

• 21-23% of those with low literacy levels are 65+• In 1981, illiteracy among 65+ was 61% in

Dominican Republic

Limitations of prior studies

• Many compared schooled populations to unschooled populations

• The IQ problem: inability to eliminate possibility that differences (in behavioral data or imaging data) exist due to differences in overall “intellectual” ability between groups

An example• Ardila, 1994

• Compared illiterates to literates and brain-damaged literates

• Illiterates performed like brain damaged persons on several tests– Rey-O figure frequently drawn right to left– Digit span = 3 1– Naming parts rather than whole line drawings– Low scores on repetition measures

Friday Harbor 2005

EFFECTS OF LITERACY

• Literacy skills affect cognitive development

• Test bias

• Literacy increases cognitive reserve

PARTICIPANTS

• WHICAP - Washington Heights - Inwood Columbia Aging Project– 6.5% of WHICAP cohort is illiterate

• all report receiving no formal education

• all age 65 and above (range from 66-97)

• fluent in English or Spanish

LITERACYTest (max. score) LITERATE ILLITERATE F

LEARNING AND MEMORY SRT total recall (72) 31.19 28.30 2.01 SRT delayed recall (12) 4.07 3.70 .51 SRT recognition memory (12) 9.33 9.25 .02 BVRT recognition memory (10) 4.92 3.57 8.04** ORIENTATION MMSE orientation (10) 9.27 8.51 6.17* ABSTRACT REASONING WAIS-R Similarities (19) 6.96 5.34 8.69** DRS Identities & Oddities (16) 12.38 12.79 .45 LANGUAGE Boston Naming (15) 12.75 11.09 9.09** Category fluency 11.30 12.19 1.31 BDAE repetition (8) 7.52 7.11 2.60 BDAE Comprehension (6) 4.48 3.48 9.37** VISUOSPATIAL ABILITY Rosen Drawing (5) 1.58 .89 8.26** BVRT matching (10) 6.58 4.96 9.52**

* p<.05; ** p<.01

Education & AD

• Low education has been established as a significant risk factor for AD and other dementias

• Prevalence (Brazil, China, Finland, France, Italy, Israel, Netherlands, Sweden, USA)

• Incidence (Evans et al. 1993, Letenneur et al. 1994, Stern et al., 1994; White

et al., 1994)

Education and cognitive decline among

nondemented elders• More rapid cognitive and functional decline

among individuals with lower educational attainment Albert et al., 1995, Butler et al., 1996; Chodosh et al., 2002; Christensen et al., 1997; Farmer et al., 1995)

• Same education-related factors that delay the onset of dementia also allow individuals to cope more effectively with normal age-related changes

Cognitive reserve: The link between low education and

higher risk of dementia/cognitive decline?

• Native ability – Years of education is a proxy for brain or cognitive

characteristics that are already present at birth. – The same cognitive strengths or robust brain structures

that allow for academic or occupational success are also are less susceptible to sudden or gradual insult.

• Lifetime experience– Educational or workplace experience affects the brain

through either functional or structural pathways. – Education could be a proxy for the presence of

protective environmental factors

Education as a proxy for innate ability among

ethnic minorities• Educational opportunities were limited due to

institutionalized racism and poverty among minority or immigrant elders.

• Indices of reserve (whether assessed either in childhood or as an adult) may also underestimate the strength of the relationship between native ability and protection from cognitive decline among ethnic minorities

Education as a proxy for exposure to “educational

process”

• Great deal of discordance between years of education and quality of education among ethnic minorities

The Literacy AdvantageWe propose that:• Literacy or reading level, as an estimate of the

quality of education, will serve as a more meaningful proxy for reserve among ethnic minorities – a more sensitive proxy for reserve than years of

education because it more accurately reflects the quality of the educational experience provided to ethnic minority elders.

– a more accurate reflection of native ability because it does not assume that all individuals get the same amount of learning from a certain grade level

PARTICIPANTS

Population-based study of aging and dementia in the Northern Manhattan neighborhoods of Harlem and Washington Heights

Age 65 and older English is first and primary language

PARTICIPANTS

• Cognitively intact at all time points based on physician’s diagnosis

• No stroke, Parkinson’s disease, substance abuse (past or current), or major psychiatric illness

• Four completed evaluations required

DEPENDENT MEASURES Selective Reminding Test

Total words recalled over 6 learning trials Number of words recalled after a 15-minute delay

PREDICTORS Age (67 – 79 vs. >80 years) Years of education (split at median of 12 years) WRAT-3 Reading Level (split at median of 47) Race/Ethnicity (white vs. ethnic minority)

  Low literacy group High literacy group

Number of subjects 67 69

  M SD range M SD range

Age 79.8 5.8 67-94 80.3 5.2 68-94

Years of educationa 10.6 3.4 0 - 16 13.0 3.2 1 - 18

WRAT-3 Reading scoreb 40.3 7.8 0 - 47 51.2 2.2 48-56

Sex, % female 76.1% 63.8%

Ethnicity, % minorityc 76.1% 39.1%

Friday Harbor 2005

Literacy and SRT Total Recall

353739414345474951

0 1 2 3

Time

To

tal R

ecal

l Sco

re

High literacy

Low literacy

Manly et al, 2003, J Clin Exp Neuropsychol, 25: 680-690

Friday Harbor 2005

Literacy and SRT Delayed Recall

5

5.5

6

6.5

7

7.5

8

8.5

0 1 2 3

Time

Del

ayed

Rec

all

Sco

re

High literacy

Low literacy

Manly et al, 2003, J Clin Exp Neuropsychol, 25: 680-690

Friday Harbor 2005

Literacy and Incident AD

0

1

2

3

4

5

Hi educHi lit

Lo educHi lit

Hi educLo lit

Lo educLo lit

1.3 (0.4 - 4.2)

2.0 (0.6 – 6.1)

4.7 (1.8 – 12.2)

p= .001

N – 1,192

Age-Specific Prevalence of Alzheimer’s Disease (1990-1992)

BlacksCaribbeanHispanic Whites

65-74

75-84

85+

0

5

10

15

20

25

30

35

40

CDR1.0

Friday Harbor 2005

Race, education, literacy, & incident AD

1.0

1.5

2.0

2.5

3.0

Model 1 Model 2 Model 3

Race

Education

Reading level

Rel

ativ

e R

isk

** p < .01; *** p < .001

***

***

***

***

**

Deconstructing Race• Cultural experience

– Acculturation level

• Educational experience– Quality of education– Literacy

• Racial socialization– Stereotype threat

STEREOTYPE THREAT

• Steele et al. (1995)• Concern that

performance will confirm a negative stereotype

• May affect some test-takers more than others within the same ethnic group

0%

2%

4%

6%

8%

10%

12%

Black White

“AbilityDiagnostic”

“Mental Challenge”

Cultural experience

Quality of education

Racial socialization

• Salience of task in everyday life

• Familiarity with items and procedures

• Emphasis on particular skills during development

• Comfort during testing session

• Attitude toward testing/motivation

Friday Harbor 2005

CONCLUSIONS• Cognitive tests have reduced specificity

among ethnic minorities• No cognitive measure is “culture free”• Cultural and educational experiences are

significantly related to test performance• Years of education represent

incommensurate values across ethnic groups

• Literacy is a superior predictor of cognitive decline to years of education

Friday Harbor 2005

Future of Researchon Cognitive Aging and Dementia

• Race-specific norms only partial solution• “Race/ethnicity” should be deconstructed into more

meaningful and predictive variables• Explore role of cultural and linguistic experience on

cognitive function with aging

Friday Harbor 2005

Future of Researchon Cognitive Aging and Dementia

• Quality of education must be considered in cognitive studies of diverse elders

• Quality of education is important in cross-sectional and longitudinal studies of cognition among age cohorts

• Determine the best indicator of quality of education and literacy across life span

• Is benefit of literacy skills limited to school age experiences?

Friday Harbor 2005

Jen’s Friday Harbor initial wish list

• How does IRT address construct validity?

• Can reading level/quality of education/test-wiseness/IQ be distinguished in a non-birth cohort?

• How to equate reading level measures in Spanish and English for comparison?