5-talk symposium on DRDP

Symposium title: Supporting Children’s Kindergarten and Early Learning Goals with a Standards-Aligned Assessment -Effective Practices in Statewide Aggregate Reporting

1. Policy implications

2. The correspondence between the foundations/standards and DRDP

3. This talk– use of DRDP data by states4. the Criterion Zone Boundary process

5. DRDP reports

The Use of DRDP Data for Statewide

Aggregate Reporting and Data-Based

Decision Making

Josh Sussman and Perman Gochyyev

BEAR Center

UC Berkeley

Objective

Demonstrate how the DRDP – designed as a formative assessment --also produces valid and reliable information that supports the needs of states for

1. Information about child development and kindergarten readiness (using Criterion Zone Boundaries)

2. Disaggregated data

3. Actionable evidence to support data based policymaking

Some typical obstacles to the use of data in

educational settings

Availability &

Quality

• Technical infrastructure & data fidelity (Ackerman, 2018)

Some typical obstacles to the use of data in

educational settings

Availability &

Quality

Analysis &

Interpretation

• Technical infrastructure & data fidelity (Ackerman, 2018)

• Measurement issues -- reliability, validity, and relevance of data to educational practice (Russo et al., 2019)

• Technical skills

Some typical obstacles to the use of data in

educational settings

Availability &

Quality

Analysis &

Interpretation

Data Use

• Technical infrastructure & data fidelity (Ackerman, 2018)

• Measurement issues -- reliability, validity, and relevance of data to educational practice (Russo et al., 2019)

• Technical skills

• Clarity on what the data can be used for (Shen & Cooley, 2008)

• Data leadership (Roegman et al., 2018)

Remainder of the presentation

I. Features of DRDP measurement that support data use

II. DRDP data for understanding, reporting, and accountability

III. Additional possibilities for DRDP data to support state-level DBDM

Approaches to

Learning

Social and

Emotional

Development

Language and

LiteracyCognition

Perceptual, Motor,

and Physical

Development

Approaches to

Learning

Social and

Emotional

Development

Language and

LiteracyCognition

Perceptual, Motor,

and Physical

Development

Approaches to

Learning

Social and

Emotional

Development

Language and

LiteracyCognition

Perceptual, Motor,

and Physical

Development

Designed as a formative

assessment (e.g., Akers,

2014; Black & Wiliam,

1998)

Criterion Zone Boundaries:

(1) Information about

students’ readiness for kindergarten

(2) Information to support

state-level policymaking

Approaches to

Learning

Social and

Emotional

Development

Language and

LiteracyCognition

Perceptual, Motor,

and Physical

Development

Designed as a formative

assessment (e.g., Akers,

2014; Black & Wiliam,

1998)

Reporting

• What percent of eligible kindergarteners are ready for kindergarten?• What about disaggregated groups

Data analysis

• 1.5 million cases of simulated DRDP data “based on a true story”

wletc’ = wletc + r + s, where

r = N(0.4, 1)

s = 0.4

• Panel data, unbalanced, cross-nested, complex (e.g., students join and leave)

Statewide information and reporting

What percent of eligible

kindergarteners are ready for

kindergarten?

• Use of criterion zone boundaries to define K-

readiness

III. More advanced analytics has the

possibility to support state-level DBDM

1. At what age does inequality begin and in which domains is it greatest?

2. Which types of preschool programs best support student development?

3. How much longer, on average, does it take for students classified as ELLs to become K-ready than those classified as proficient?

Average development of K-eligible children

Do

ma

in S

core

When does equity begin to be a concern?

When does equity begin to be a concern?

When does equity begin to be a concern?

800

300

Lan

gu

ag

e a

nd

Lit

era

cy D

ev

elo

pm

en

t

Infancy

End of K

570

i1

i2

i3

i4

When does equity begin to be a concern?

800

300

Lan

gu

ag

e a

nd

Lit

era

cy D

ev

elo

pm

en

t

Infancy

End of K

570

i1

i2

i3

i4

Which types of preschool programs best support

child development?

Advanced analytics can help contextualize

gaps in an actionable metric

• Survival models: how much longer do ELLs, a group who are known to struggle, take to become K-ready?

• The answers have the promise to inform policy decisions about the timing and intensity of interventions

Survival models in education

• It is about time• From whether to when• How many months does it take for children to become kindergarten-ready?

• Why?• The observations are censored

• Some kids may not be categorized as K-ready within the period of data collection• If a child reached K-ready level after the data collection, researchers will not know that

• Singer & Willett (1991):• We believe that researchers avoid asking questions about time-to-event ("When?") because of

methodological difficulties introduced when members of the sample do not experience the target events during the data collection period.

• One approach: dichotomize at a particular time and model the binary outcome• Does not eliminate censoring• Does not answer: how long does it take for an average child to reach• How to incorporate time-varying covariates

Survival models: K-readiness

• Time to K-readiness (instead of K-readiness itself)• We can also identify factors predicting students’ K-readiness

• ELL vs. non-ELL: differences in proportions of K-ready children• if ELL status is varies across time, survival models will accommodate that • Investigate gaps between ELLs and non-ELLs across ages as a function of probability of being K-

ready?

• Hazard probability: conditional probability that a child will become K-ready in that month, given that he or she stayed “not K-ready” up to that point.

• A chronological summary of the probability of becoming K-ready• States (“K-ready”) can be absorbing or recurring• Accommodate ”delayed entry:” kids becoming parts of the “snapshot” observation at different

ages

• “Of all the survival methods available, we believe that discrete-time survival analysis offers the most promise for exploring educational transitions.” (Willett and Singer, 1991)