The Results are In! Child Outcomes for OSEP EI and ECSE Programs Donna Spiker Early Childhood Outcomes Center at SRI International October 13, 2011 (CCSSO-SCASS

The Results are In!

Child Outcomes for OSEP EI and ECSE Programs

Donna SpikerEarly Childhood Outcomes Center

at SRI International

October 13, 2011(CCSSO-SCASS Meeting)

San Francisco, CA

What we will cover

• Quick review of the reporting requirement and state approaches

• Share the national data• Describe how the national data were computed• Discuss the quality of the national data• Discuss next steps for states• Framework and self-assessment (briefly. …)

2Early Childhood Outcomes Center


OSEP Reporting Requirements: Child Outcomes

• Positive social emotional skills (including positive social relationships)

• Acquisition and use of knowledge and skills (including early language/ communication [and early literacy])

• Use of appropriate behaviors to meet their needs


OSEP Reporting Categories

Percentage of children who: a. Did not improve functioningb. Improved functioning, but not sufficient to move nearer

to functioning comparable to same-aged peers c. Improved functioning to a level nearer to same-aged

peers but did not reach itd. Improved functioning to reach a level comparable to

same-aged peerse. Maintained functioning at a level comparable to same-

aged peers

3 outcomes x 5 “measures” = 15 numbers

Illustration of 5 Possible Paths

0

10

20

30

40

50

60

70

1 6 11 16 21 26 31 36 41 46 51 56

Age in Months

Sco

re

Maintained functioning comparable to age peers

Achieved functioning comparable to age peers

Moved nearer functioning comparable to age peers

Made progress; no change in trajectory

Did not make progress

The Summary Statements

1. Of those children who entered the program below

age expectations in each outcome, the percent who

substantially increased their rate of growth by the

time they turned 3 years of age or exited the

program.

2. The percent of children who were functioning within

age expectations in each outcome by the time they

turned 3 years of age or exited the program.


State Approaches to Outcomes Data

Approach Part C (56

states/jurisdictions)

Preschool(59

states/jurisdictions)

COS 7 pt. scale 41/56 (73%) 37/59 (63%)

One tool statewide 7/56 (13%) 9/59 (15%)

Publishers’ online analysis

3/56 (5%) 6/59 (10%)

Other 5/56 (9%) 7/59 (12%)

Early Childhood Outcomes Center 8

Note: Based on 29 States with highest quality data

Social relationships Knowledge and skills Action to meet needs

a 0.016548176502116 0.0148974177446063 0.0145225115393539

b 0.179791044501509 0.199542744006337 0.174681665026758

c 0.185932320590048 0.247993957138402 0.214074779812331

d 0.294138593838747 0.369311305766879 0.370708975497374

e 0.323587775084598 0.168236271568794 0.226016769460911

3%

8%

13%

18%

23%

28%

33%

38%

Estimated Data for Part C, 2009-10




SS1 0.70973347275746 0.742180533377054 0.755546347980436

SS2 0.61772636892334 0.537547577335669 0.596725744958282

5%

15%

25%

35%

45%

55%

65%

75%

Estimated Summary Statement Data for Part C, 2009-10



a 0.0173423475321513 0.0182367034048671 0.0164717096861928

b 0.114520004403404 0.133727833654954 0.108153036923834

c 0.281561785450637 0.32470410889358 0.208844242808853

d 0.343437919521324 0.343977608174791 0.356458315050576

e 0.243152978902516 0.179335862613809 0.310081127789364

3%

8%

13%

18%

23%

28%

33%

38%

Estimated National Data for Early Childhood Special Education, 2009-2010





SS1 0.825777563121165 0.814823334298583 0.819365394291995

SS2 0.586590898423841 0.523313470788598 0.666539442839937

10%

30%

50%

70%

90%

Estimated National Summary Statements for Early Childhood Special Education, 2009-2010

Criteria for States with Quality Data

1. Low percentage of missing data

2. No odd patterns in “a” or “e” categories

3. Did not use questionable data collection methods


Problem with Missing Data

• We don’t know how well the data the state has represent the entire state.

• If the data are representative, the percentages for the a to e Progress Categories and the Summary Statements won’t change as data are added on more children.


Can we trust these data?


Pattern checking for validity

• Checking across years–How do the 2009-10 compare to the

data for 2008-09?• Checking across methods

–How do the data for all states compare to states with highest quality data?


All states 19 best All states 29 bestFFY 08-09 FFY 09-10

0

10

20

30

40

50

60

70

80

64.570.2

64

71

SS1: % who IncreasedGrowth Rates

Part C, Outcome A: Social Relationships


0

10

20

30

40

50

60

70

80

6761.3

64.761.8

SS2: % who Exited at Age Expectations

Part C, Outcome B: Knowledge and Skills


0

10

20

30

40

50

60

70

8070.4

76.6

68.1

74.2

SS1: % who Increased Growth Rates


0

10

20

30

40

50

60

70

80

5953.9 55.8 53.8


Part C, Outcome C: Meets Needs


0

10

20

30

40

50

60

70

80

69.7

76

68.1

75.6



0

10

20

30

40

50

60

70

80

64.660.5 61.5 59.7


Part B Preschool: Social Relationships


0

10

20

30

40

50

60

70

80

90

77.282.7

79.7 82.6



0

10

20

30

40

50

60

70

80

90

61.758.7

62.158.7


Part B Preschool: Knowledge and Skills


0

10

20

30

40

50

60

70

80

90

76.282.7

78.381.5



0

10

20

30

40

50

60

70

80

90

55.851.2

55.552.3


Part B Preschool: Meets Needs


0

10

20

30

40

50

60

70

80

90

75.381.6

78.181.9



0

10

20

30

40

50

60

70

80

90

67.8 67.2 66.7 66.7


What do these data tell us?

• Nationally, a high proportion of children who receive Part C and ECSE services are showing greater than expected progress.

• Nationally, many (over half) are exiting the program functioning like same age peers in at least one of the outcomes.


Using data at state level……

• Never intended to compare states• OSEP wants states to begin by comparing

own data to past data• States set targets to compare yearly

performance to those targets• States implement program improvement

activities• OSEP looking at data in monitoring visits


Linkages to other state EC efforts……

• Some states are beginning to develop linkages between these OSEP data and other assessment in their states, for example:– Colorado’s Results Matter– Results Matter in Nebraska– Kentucky Early Childhood Data System (KEDS)


Colorado’s Results Matter


Supporting States in Building a Child Outcomes Measurement System

(just briefly………)

Framework and Self-Assessment

• FRAMEWORK– Set of components and quality elements– Provides the structure for the self-assessment

• SELF-ASSESSMENT – Scale that provides criteria for levels of

implementation within each quality element– Rating assigned based on level of

implementation within each element



Additional Information & Questions

Additional information:

www.the-eco-center.org


Questions

http://www.the-eco-center.org/

http://www.the-eco-center.org/

Documents

The Results are In! Child Outcomes for OSEP EI and ECSE Programs Donna Spiker Early Childhood Outcomes Center at SRI International October 13, 2011 (CCSSO-SCASS