Embed Size (px)
Citation preview
Measuring Child Progress:Measuring Child Progress:Two State’s JourneysTwo State’s Journeys
Barbara Jackson, NEBarbara Jackson, NE
Beppie Shapiro, HIBeppie Shapiro, HI
Measuring Child and Family Outcomes
Albuquerque, NM
April 25, 2006
Format of PresentationFormat of Presentation
• Meet each stateMeet each state
• Overview of the 4 key questions to Overview of the 4 key questions to be discussedbe discussed
• For each key question:For each key question:
– Description of each state’s plansDescription of each state’s plans
– Questions and comments from Questions and comments from participantsparticipants
What Counts:
Measuring Benefits of Early Intervention in Hawai`i
Beppie Shapiro, University of Hawai`[email protected]
About Hawai`i
About 17,000 births per year
Over 7% of each birth cohort served in Part C
No majority ethnicity
4.1 % of population are immigrants, primarily Asians and Pacific Islanders
Geographic isolation
Five main islands – different access to services
Special education in Hawai`i
Part C under Department of Health
Part B under Department of Education
Outcomes process differs from C to 619
SEA = 1 LEA
Part C in Hawai`i, 2006
Broad eligibility
Three DOH agencies provide services statewide
Recent development of statewide IFSP to be used by all programs
No Part C database – 2 agencies have their own
Standardized IFSP form, process
2005: All Part C programs trained to use new statewide IFSP form– Present Levels of Development– Family-driven Outcomes* and Objectives– Services to support Objectives– Transition
What Counts?....What Counts?.... Measuring the Outcomes of Measuring the Outcomes of
Early Intervention in NebraskaEarly Intervention in Nebraska
Barbara Jackson, Munroe Meyer Institute
About NebraskaAbout Nebraska
Birth Mandate State
Co-Leads for Part C: Health and Human Services & Education
Outcome data process will be the same across the birth through 5 age group
Nebraska (continued)Nebraska (continued)
460 School Districts
Number of children birth to 3: 1303
Number of children ages 3 and 4: 2811
Number of children 3 through 5: 4707
Key Question…………Key Question…………
1.1. How will the state determine children’s How will the state determine children’s status on each of the outcomes?status on each of the outcomes?
Assessment in Hawai`i
Part B Section 619, Hawai`i Assessment: Early Brigance
Part C: Since 2004: 3 assessment tools approved for CDE – Hawaii Development Charts (HELP), “Michigan” EIDP, or ASQ
Hawai`i’s Part C Assessment : Ages and Stages
For babies eligible due to environmental risk
Forms for specific age levelsScores indicate only “Typical”, “Monitor”, “Refer” Referred for CDE and services by program serving DDCare coordination may remain with original program
1. What Counts Design Team convenes monthly to develop/review progress
2. How can we assign a “score” to each child on each EI Goal?
What Counts: Measuring the Benefits of Early Intervention in Hawai`i:
Hawai`i’s Part C Outcomes (Goals) Measurement Process
Data Collection Intervals & Rationale– At every IFSP: initial, review, annual
–Starting when child is at least 4 months old
Hawai`i’s Part C Outcomes (Goals) Measurement Process
Assigning scores: decisions & rationale
WHO: IFSP team
Following “Present levels of Development” description
HOW: all team members reach consensus on rating using modified ECO Child Summary Reporting Form
Hawai`i’s 619 Outcomes Measurement Process
Family involved in assessment process Family provides input on written form with open-ended questionsA professional assigns rating on each goal Other providers asked for input to ratingRatings assigned at entry and annually
Nebraska’s ProcessNebraska’s Process
Child Outcome Task Force convened to guide process
Nebraska calls for Child Assessment Nebraska calls for Child Assessment that….that….
Is based on ongoing observation of children engaged in real activities, with people they know, in natural settings
Engage families and primary caregivers as active participants
Is individualized to address each child’s unique ways of learning
Reflects that development and learning are rooted in culture and supported by the family
Integrates information across settings
Nebraska’s Assessment ProcessNebraska’s Assessment Process
Which assessments?
AEPS Creative Curriculum High Scope COR for Infants/
Toddlers & Preschoolers
Why Selected?Why Selected?
Assessment approach parallels other Early Childhood Program assessment processes
Use information from multiple sources (e.g., family, providers) and multiple observations
Curriculum-based Assessment – can be used for multiple purposes
Can build the capacity of our system to support children and their families
Nebraska’s Nebraska’s Process for parent input?Process for parent input?
Parents provide input during the assessment process
How is the assessment information How is the assessment information “transformed”?“transformed”?
Working with publishers to determine feasibility of computer-based formulas
Scores will be reviewed by team to assure validity of score
Nebraska’s Assessment ProcessNebraska’s Assessment Process
Data Collection Schedule Districts will be mandated to report entry
and exit data Entry data will collected within the 45-60
days after IFSP/IEP meeting Exit data collected within 2 months of
leaving the program Districts will be encouraged follow
publishers guidelines for frequency of assessments
Key Question…………Key Question…………
2. What reporting categories has the 2. What reporting categories has the state chosen to use?state chosen to use?
Reporting Categories in Hawai`i
What Counts: Measuring Benefits of Early Intervention in Hawai`i
Reporting Categories in Hawai`i
Part C Decision & RationaleUse ECO 5 CategoriesA. maintained typical functioning
B. made progress to achieve typical functioning
C. moved nearer to typical functioning
D. progress but not enough to move nearer to typical
E. did not make progress
Meaningful, Program Improvement
Reporting Categories in Hawai`i
619– OSEP Categories
Reporting Categories in NEReporting Categories in NE
Four Categories OSEP’s 3 categories
Plus % of children who attain typical development
Key Question…………Key Question…………
3. How and in what form will data get 3. How and in what form will data get from local programs to the state?from local programs to the state?
Getting Part C Data to the StateIn Hawai`i
Simpler than most states?
More difficult than many states?
Each “Agency” will collect and summarize data from its programs
Data to explain results
Length of enrollmentAge at enrollmentICD9s/conditions
Getting Data to the State: Part C
Electronic data entry at local programTransmitted to AgencyAgency calculates number of children in each categoryAgency sends these numbers to State State creates OSEP and cross-Agency reports
Unresolved data questions
Local programs send “explanatory data” to Agency– How much of this data goes to State?
How?
How to identify potential duplicates
Getting Data to the State: 619
Teacher enters data into stand-alone EXCEL spreadsheet
Spreadsheets collected at school, sent to State
NE: Getting Data to the StateNE: Getting Data to the State
Using the publisher’s internet system- State will be the licensed manager
Link with the State Data System
Key Question…………Key Question…………
4. What will you do to maximize the 4. What will you do to maximize the reliability of the data?reliability of the data?
Training, Support to Maximize Data Reliability in Hawai`i
Training:Training:
Support:Support:
Web:Web:
7 hrs, all EI programs, by 7 hrs, all EI programs, by communitycommunity
1-2 months post training: 1-2 months post training:
On site? Conference call? On site? Conference call?
FAQs, email listservFAQs, email listserv
Maximizing Data Reliability in Hawai`i: Evidence for Rating
IFSP teams document evidence used to select rating category
Pilot showed need for training on what makes good evidence
Supervisors provide support and QA
Criteria for Evidence Criteria for Evidence (in progress)(in progress)
Different contextsDifferent contexts
Highest level of Highest level of achievementachievement
Specific to Goal AreaSpecific to Goal Area
Examining Reliability in Hawai`i
Another measurement of “how well child is doing”
– Compare ratings
Compare ratings for children with different conditions/diagnoses
Challenges for Hawai`i: Quality Assurance - Part CHow reliable are assessments?
Does every IFSP team have someone knowledgeable about typical child development?
How reliable are summary form ratings?
Reliability of Data in NE: TrainingReliability of Data in NE: Training
Intensive training on assessments throughout state with consistent trainers
Reliability of Data in NE: Reliability of Data in NE: ImplementationImplementation
Review of data at the state level
Ongoing questions/answers communication document based on local questions
Team approach to assessment process
Pilot process to determine reliability issues
