David Hessl, PhD - HealthMeasures · 2019-06-24 · David Hessl, PhD. Professor. Director,...

David Hessl, PhDProfessorDirector, Translational Psychophysiology and Assessment Laboratory (T-PAL)UC Davis School of Medicine

The NIH Toolbox Cognitive Battery for Special Populations: The Case of Intellectual Disability

Fragile X Syndrome Clinical Trials Review

Why Cognition, and Why Now?• Behavioral outcome measures in ID are dependent on parental impressions, which are

inherently subjective, often biased, and greatly influenced by environmental factors across development

• Behavioral ratings are susceptible to high placebo response rates• Behavioral ratings are more difficult to establish links to core underlying brain mechanisms• Above issues may partially explain “failure” of FXS trials• Cognitive deficits are the defining feature of intellectual disabilities (ID), including fragile X

and Down syndromes• Cognitive measures are objective, can be measured reliably, and can be matched to the

cognitive phenotype and underlying brain mechanisms of the disorder• They may be more sensitive to changes associated with targeted treatment

=

Dimensional Change Card Sort (DCCS; Cognitive Flexibility)

• “Match Color” or “Match Shape”; changes after set established• Cognitive flexibility (executive function)• Mediated by frontal brain regions

Flanker Inhibitory Control and Attention (FICA)

• Click arrow that MIDDLE fish is pointing• “Congruent” (all same direction) and “Incongruent” (flankers opposite

direction) trials• Attention and inhibitory control (executive function)• Based on Attention Network Test (ANT)• Mediated by frontal brain regions

Pattern Comparison Processing Speed (PCPS)

• “Same” or “Different”; # of items correct in set time• Processing Speed (based on Salthouse’s Task)

List Sorting Working Memory (LSWM)

• Listen and see objects, report back order by size• Working memory• Fronto-parietal and dorso-lateral prefrontal cortex

Oral Reading (ORT)

Picture Vocabulary (PRT)

Picture Sequence Memory (PSM)

NICHD R01 “A Cognitive Test Battery for Intellectual Disability” Study Aims

1. To validate the NIH-TCB, including measures of executive function, episodic memory, working memory, processing speed, attention, and language, for individuals with ID, and modify it where necessary to meet the needs of this population.

2. To examine the sensitivity of the NIH-TCB to detect clinically meaningful differences in persons with ID, including detection of expected syndrome-specific cognitive phenotypes.

3. To examine the validity of the NIH-TCB composite scores in ID groups compared with the established composites in general population controls, and to compare their psychometric properties to analogous gold standard measures of general intelligence.

4. To examine the sensitivity of the NIH-TCB to detect changes in cognition across a two-year period of development.

NIH Toolbox in ID Pilot Studies (N=45): Ecological Validity

Chronological AgeMental Age Equivalent FSIQ (Deviation)

Adaptive Behavior

Composite

DCCS .31 .72*** .66*** .33

Flanker .27 .61*** .70*** .36*

Picture Vocabulary .48** .67*** .70*** .52**

Oral Reading .39* .62*** .71*** .42**

Picture Sequence Memory .34* .55** .57*** .16

Pattern Comparison .17 .45* .46** .20

List Sorting .57** .49* .52* -.03

NIH-TCB Cognitive Composite - - .89*** .42*

Hessl, David, et al. "The NIH Toolbox Cognitive Battery for intellectual disabilities: three preliminary studies and future directions." Journal of Neurodevelopmental Disorders 8.1 (2016): 35.

Pilot Studies: Toolbox Syndrome Profiles

Hessl, David, et al. "The NIH Toolbox Cognitive Battery for intellectual disabilities: three preliminary studies and future directions." Journal of Neurodevelopmental Disorders 8.1 (2016): 35.

Key Contributions• Mental age start, education

override• Model for accommodations• Reinforcement Inventory

and Token Board• Reinforcement Guidelines• Administration Form• Subtest Adaptations• Scoring• New Tests (Speeded

Matching)

Participant Orientation Video (Child)

Hessl, David, et al. "The NIH Toolbox Cognitive Battery for intellectual disabilities: three preliminary studies and future directions." Journal of Neurodevelopmental Disorders 8.1 (2016): 35.

Hessl, David, et al. "A solution to limitations of cognitive testing in children with intellectual disabilities: the case of fragile X syndrome." Journal of Neurodevelopmental Disorders 1.1 (2008): 33.

“True Deviation” Scoring Method• Calculate the exact deviation of each person’s performance on a cognitive

test from population norms• zij = (rij - µj)/ σj• rij = subtest raw score• µj = mean subtest score of corresponding age band from standardization

sample• σj = standard deviation of subtest score from standardization sample• Consider a 10-year-old participant obtained a subtest raw score of 10 on a

given subtest. In the standardization sample, for children 10 years old, the mean raw score is 24.31 and the SD is 3.37. Therefore, the child’s deviation z score is (10 – 24.31) / 3.37 = -4.25, or 4.25 SD below the mean for same age-peers

Total DS + ID FXS + ID O-IDN 241 91 75 75RaceAmer Indian/Alaska Native 3 1 1 1Asian 6 2 2 2Black 24 4 8 12

Native Hawaiian/Pacific Islander 3 1 1 1

White 170 70 60 40More Than One 26 10 2 14Not Reported 9 3 1 5

Ethnicity (% Hispanic or Latino) 19.9% 18.7% 8.0% 33.3Gender

(% male) 59.3% 45.1% 73.3% 62.7%

Primary Caregiver Education (% with at least a 4yr degree) 61.4% 58.2% 61.3% 57.3%

Full Validation Study Demographics

3 4 5 6 7 8+

Flanker 32 (55.2%) 48 (75.0%) 49 (90.7%) 25 (96.2%) 20 (100.0%) 19 (100.0%)

Flanker DE** 30 (90.9%) 19 (100.0%) 10 (90.9%) 1 (100.0%) - -

DCCS 14 (25.0%) 33 (51.6%) 40 (76.9%) 25 (96.2%) 19 (95.0%) 19 (100.0%)

DCCS DE** 43 (97.7%) 44 (95.7%) 19 (95.0%) 2 (100.0%) 4 (100.0%) 1 (100.0%)

LSWM 16 (34.0%) 40 (64.5%) 40 (75.5%) 25 (96.2%) 20 (100.0%) 19 (100.0%)

PCPS 29 (51.8%) 41 (64.1%) 49 (90.7%) 26 (100.0%) 20 (100.0%) 19 (100.0%)

PSM 45 (81.8%) 60 (95.2%) 52 (96.3%) 25 (100.0%) 19 (100.0%) 19 (100.0%)

PVT 55 (96.5%) 62 (96.9%) 54 (100.0%) 26 (100.0%) 20 (100.0%) 19 (100.0%)

ORT 55 (98.2%) 60 (95.2%) 53 (98.2%) 26 (100.0%) 20 (100.0%) 18 (100.0%)

Toolbox Feasibility by Mental Age

Shields et al (In prep)

Toolbox Flooring Concerns and Solutions?

Shields et al (In prep)

Chronological ageMental

age ABC-C total FSIQ VABS-3 ABCFlanker .25*** .58*** -.01 .51*** .17*Flanker DE .25 .37** -.24 .18 .10DCCS .30*** .63*** .07 .56*** .13DCCS DE .13 .67*** -.25** .50*** .19*List Sorting .20** .71*** .17* .65*** .15

Pattern Comparison .10 .59*** -.01 .56*** .16*Picture SequenceMemory .05 .54*** .03 .55*** .26***

Picture Vocabulary .35*** .76*** -.08 .73*** .39***Oral Reading .18** .68*** -.05 .69*** .38***Fluid Composite - - - .68*** .14

Crystallized Composite - - - .77*** .43***

Cognitive Composite - - - .79*** .24**

Ecological Validity: Full Study (n~240)

Shields et al (In prep)

Shields et al (In prep)

Test-Retest Reliability and Practice Effects Full Validation Study

n r (p) Cohen’s d (p)

Flanker 146 .73 (<.001) 0.105 (.014)

DCCS 98 .71 (<.001) 0.115 (.11)

List Sort 122 .76 (<.001) 0.14 (.093)

PCPS 138 .81 (<.001) 0.318 (<.001)

PSM A-A 58 .51 (<.001) -0.042 (.328)

PSM A-B 62 .57 (<.001) 0.305 (.024)

PVT 188 .84 (<.001) 0.028 (.407)

ORT 183 .96 (<.001) 0.006 (.384)Shields et al (In prep)

Convergent and Discriminant Validity

Convergent Validity Discriminant Validity

n r (p) n r (p)

Flanker with KCPT 144 -.52 (<.001) Flanker with PPVT 191 .57 (<.001)

DCCS with NEPSY Prorated 107 .47 (<.001) DCCS with PPVT 151 .52 (<.001)

List Sort with SB5 VWM 167 .64 (<.001) List Sort with PPVT 167 .57 (<.001)

PCPS with WPPSI Bug Search 167 .67 (<.001) PCPS with PPVT 182 .52 (<.001)

PSM with Leiter FM 182 .43 (<.001) PSM with PPVT 170 .49 (<.001)

PVT with PPVT 233 .83 (<.001) PVT with Leiter FM 209 .46 (<.001)

ORT with WJ LWID 228 .92 (<.001) ORT with Leiter FM 206 .57 (<.001)

Shields et al (In prep)

Full Validation Study (n~240): Toolbox Syndrome Profiles

Shields et al (In prep)

Adaptations and New Measure DevelopmentList Sort Task Understanding

• 2 versions: Age 3-6 and Age 7+• A common reason for questionable validity of List Sort in the younger version seems to

be issues understanding the task in the transition from practice to test.• In the Age 3-6 version, practice does not involve naming multiple items at a time, or

sorting the animals into a certain order.• We piloted additional practice items to bridge this gap before beginning test items.• This has now been implemented as an Experimental Version.• We are also collecting pilot data on partial credit responses (similar to existing WM

tests such as SB5 WM)

Which animal is smallest?Which animal is next biggest?Which animal is biggest?

Name the animals in size order.

Adaptations and New Measure DevelopmentDevelopment of a New Processing Speed Task

Speeded Matching• We have developed a new processing speed

task in collaboration with Toolbox developers, which is now available as an Experimental Version

• We had high feasibility with the WPPSI Bug Search processing speed task, and wanted to create a similar matching task

• In Speeded Matching, the task is to simply select the animal that matches the top animal out of four response options

Collaborators and SupportUC DavisDavid Hessl (PI)Stephanie SansoneRebecca ShieldsAndie DraytonLen AbbedutoForrest McKenzieMadeleine SchloetterAndrea SchneiderEthan Hessl

Rush UniversityElizabeth Berry-Kravis (PI)Claire MichalakMerve SenalanCrystal HerveyErin Carmody & Meraj BaigMichael NelsonAnne HoffmannKyle Bersted

Northwestern UniversityRichard Gershon (PI)Aaron KaatCindy NowinskiMaria KharitonovaRebekah AbelJerry SlotkinShalini PatelIrina Kontsevaia

University of DenverKaren Riley (PI)Jeanine ColemanTalia ThompsonSuzanne DelapLacey PomerantzDarian CrowleyMikayla BrownShanelle Rodriguez

UC RiversideKeith Widaman

ConsultantsBruce Pennington (DU)Philip Zelazo (UMinn)Jamie Edgin (UAriz)Nicole Tartaglia (U Colo)Francis Hickey (U Colo)