View
242
Download
0
Category
Tags:
Preview:
Citation preview
1
Idaho’s Idaho’s Early Childhood Early Childhood
Outcomes System Outcomes System (Idaho ECOS)(Idaho ECOS)
Measuring Early Childhood Outcomes
Idaho Infant Toddler ProgramIdaho Early Childhood Special Education Program
2
AgendaAgenda• Measuring Child Outcomes:
– Why, What, When, Where, – Who & How
• Review Child Outcome Summary Form, Tools & Resources
• Practice Exercises•
3
Federal Accountability Federal Accountability and and
Early Childhood OutcomesEarly Childhood Outcomes
U.S. CongressOffice of Budget and
Management
U.S. Department of Education - Office of Special Education Programs
State Education Agencies and State Part C
Agencies
Local Education Agencies and
Early Intervention Programs
4
Training ObjectivesTraining ObjectivesParticipants will be able to :
• Demonstrate the ability to accurately identify, record, and report a child’s outcome data using the Idaho Child Outcome Summary Form (Idaho COSF)
5
Idaho’s goalIdaho’s goal
Document our program’s
impact for children with IFSPs or IEPs
6
Measuring outcomes make Measuring outcomes make sense for Idahosense for Idaho
– Support DEC/NAEYC best practice guidelines
– Examine and refine current assessment practices
– Demonstrate value of• Early Intervention (Part C), • Early Childhood Special Ed (Part B) programs
– Meet Federal Reporting Requirements
7
What are the What are the OSEP’s Early OSEP’s Early
Childhood Childhood Outcomes?Outcomes?
8
OSEP’s outcomes focus on OSEP’s outcomes focus on measuring the results of measuring the results of
our our service systemservice system
9
OSEP’s 3 Child OutcomesOSEP’s 3 Child Outcomes1. Children have positive social-
emotional skills (including positive social relationships)
2. Children acquire and use knowledge and skills (including early language /communication)
3. Children use appropriate behaviors to meet their needs
10
OSEP’s 3 Child OutcomesOSEP’s 3 Child Outcomes1. Children have positive social-
emotional skills (including positive social relationships)
• Such as: responding to others, expressing emotions, turn-taking, using appropriate social greetings,
play skills or having friendships with same-age peers, etc.
11
OSEP’s 3 Child OutcomesOSEP’s 3 Child Outcomes2. Children acquire and use
knowledge and skills (including early language
/communication)
• Such as: expressing thoughts and ideas, listening to and enjoying stories and books, or learning new ways to do things, etc.
12
OSEP’s 3 Child OutcomesOSEP’s 3 Child Outcomes3. Children use appropriate
behaviors to meet their needs
• Such as getting from place to place, using tools like forks or crayon, and feeding or dressing, etc.
13
The Child Outcome Areas…The Child Outcome Areas…
• Represent critical functional outcomes necessary in every day activities and routines
• Not domains based – not trying to separate child development into discrete areas (communication, gross motor, etc.)
• Emphasize how the child is able to integrate across domains to carry out complex meaningful behaviors
14
Outcomes are FunctionalOutcomes are Functional
• Meaningful to the child in the context of everyday living
• Integrated series of behaviors or skills that allow the child to achieve the outcomes.
• Not – a single behavior– the sum of a series of discrete
behaviors
15
Functional OutcomesFunctional Outcomes
• What does a child typically do?
• Actual performance across settings and situations
• How child uses his/her skills to accomplish tasks
• Not the child’s capacity to function under ideal circumstances
16
So…how do we So…how do we demonstrate that our demonstrate that our
programs produce programs produce good outcomes for good outcomes for
children?…children?…
17
Demonstrate that…Demonstrate that…• Children have positive outcomes
given who they are, their delays, disabilities, functioning when they entered, etc.
• Outcomes are better than they would have been without the program
18
Complicating IssuesComplicating Issues• Many children progress with no
intervention (maturing with age)
• Children with disabilities’ show diverse progress and under the best of programs, will experience different outcomes
19
SolutionSolution
Document the number of children for whom the program has changed their developmental trajectories
20
Thinking about how children are doing Thinking about how children are doing with regard to each outcome.with regard to each outcome.
Age-expected skills & behavior
Movement away from age-expected
Movement toward age-expected
21
OSEP Indicators – categories OSEP Indicators – categories of progressof progress
• a. % of children who maintain functioning at a level comparable to same-age peers
• b. % of children who reach functioning at a level comparable to same-age peers
• c. % of children who improved functioning but did not achieve functioning comparable to same-age peers
• d. % of children who did not improve functioning.
22
OSEP Indicators & OSEP Indicators & Measurement CategoriesMeasurement Categories
Group a: maintained or reached typical
Entry Exit
5 year old level
2 year old level
23
Indicators & Measurement Indicators & Measurement CategoriesCategories
Entry Exit
5 year old level
2 year old level
Group b: made progress but…
24
Indicators & Measurement Indicators & Measurement CategoriesCategories
Entry
5 year old level
2 year old level
Group c: did not make progress
Exit
25
Enry
Indicators & Measurement Indicators & Measurement CategoriesCategories
Exit
5 year old level
2 year old level
Group b: made progress but..
Group c: did not make progress
Group a: maintained or reached typical
26
How do we document change in developmental How do we document change in developmental trajectories while in services ?trajectories while in services ?
Obtain and compare a child’s assessment data
at program entry and exit.
27
Quality
Assessment
Quality Service
Better Outcomes
28
“Much of developmental psychology (early childhood testing) as it now exists is the science of the strange behavior of children with strange adults in strange settings for the briefest possible periods of time.”
(Bronfenbrenner, 1979)
29
What is assessment?What is assessment?
“Early childhood assessment is flexible, collaborative decision-making process in which teams of parents and professionals repeatedly revise their judgments and reach consensus about the changing developmental, educational, medical, and mental health services needs of young children and their families.”
Bagnato and Neisworth, 1991Quoted in DEC Recommended Practices, 2005
30
DEC Recommended DEC Recommended Practices for AssessmentPractices for Assessment
• Involves multiple sources (e.g., families, professional team members, service providers, caregivers)
• Involves multiple measures (e.g., observations, criterion-curriculum-based instruments, interviews, curriculum-compatible norm-referenced scales, informed clinical opinion, work samples)
31
What information is What information is gathered?gathered?
Multiple sources of data are used to rate
a child’s functioningInformed Professional Judgment & Observation
Parent Input
Norm-referenced, criterion-referenced or curriculum-based assessments
32
Which anchor assessment Which anchor assessment tools will we use in Idaho?tools will we use in Idaho?
The following tools have been adopted for completing the outcome rating scale.
• BDI – II (Battelle)• HELP (Hawaii)• Creative Curriculum• AEPS• Carolina• OUNCE• Work Sampling• High Scope (COR)• Brigance • Bayley III (ITP program only)
33
Informed professional judgment– Teachers, paraprofessionals, related
service providers– Anecdotal records, documented
observations and data, progress reports, work samples, portfolios
Parental input– Best practice– Information from caregivers provide
critical information to determine how child is doing across a variety of settings
Information to gather…Information to gather…
34
So…our challengeSo…our challenge
How do we take what we know about assessment and apply it to measuring the 3 outcomes…
…when there is no one assessment tool that assesses all 3 outcomes directly.
35
And through our And through our assessment lens…assessment lens…
• Each child is a collection of numerous behaviors, skills, traits, capabilities, interests, strengths, and needs
• What an individual “tunes into” depends on his/her orientation
• Individual professional training has provided an “organizing framework” for how one sees the child
36
How do assessment tools fit How do assessment tools fit with with
the three Child Outcome the three Child Outcome Areas?Areas?
• CROSSWALKS CAN HELP!
• A national center, the ECO Center is crosswalking the most common assessments to the 3 child outcomes
• Crosswalks give a visual indication of how items on an assessment tool covers the 3 outcomes
• Crosswalks show which areas/subareas map to which outcome
37
Sample Crosswalk Sample Crosswalk
The High/Scope Preschool Child Observation Record (2003): Crosswalk to Child Outcomes
Outcome 1:
Positive social relationships
Outcome 2:
Knowledge and skills
Outcome 3: Action to meet needs
I. Initiative C. Initiating play [social context of play] II. Social Relations E. Relating to adults F. Relating to other children G. Resolving interpersonal conflict H. Understanding and expressing feelings V. Language & Literacy Q. Listening to and understanding speech
III. Creative Representation I. Making and building models J. Drawing and painting pictures K. Pretending V. Language & Literacy R. Using vocabulary S. Using complex patterns of speech T. Showing awareness of sounds in words U. Demonstrating knowledge about books V. Using letter names and sounds W. Reading X. Writing VI. Mathematics & Science Y. Sorting objects Z. Identifying patterns AA. Comparing properties BB. Counting CC. Identifying position and direction DD. Identifying sequence, change, and
causality EE. Identifying materials and properties FF. Identifying natural and living things
I. Initiative A. Making choices and plans B. Solving problems with materials D. Taking care of personal needs
Note: Areas that are not precursor to or components of any of the three outcomes, and therefore not included in the crosswalk, were: IV. Movement & Music:
L. Moving in various ways M. Moving with objects N. Feeling and expressing steady beat O. Moving to music P. Singing
38
What will the process look What will the process look like in Idaho?....like in Idaho?....
• Parents will be well informed and contribute information to the process
• An anchor assessment will be completed to assess the child in the three outcome areas
• Information will be collected, compiled and documented using anchor assessments, parent information, and informed professional observation and judgment.
• A rating decision regarding a child’s level of functioning will be made by the team utilizing all the information gathered
39
The Specifics:The Specifics:
Who, What, Who, What, andand
When When
40
Target Population for Entry Target Population for Entry (Part C) (Part C)
• Entry baseline data is required for: – all children entering services on or after
July 1, 2006 will have COSF completed within 45 days of IFSP development.
– except premature infants less than 6 months adjusted age
• Includes children receiving only a single related-service (i.e. Speech only or OT only)
41
Target Population for Exit Target Population for Exit (Part C)(Part C)
• Exit data is required for all children in services for at least 6-months as of January 1, 2007.
• Unanticipated exits– use best information available to complete form following exit.
• Assessment (with anchor tool) recommended for all exiting children
• If necessary, those receiving only one related-service can use ASQ and ASQ-SE in lieu of full developmental evaluation as one data source in determining exit outcome rating
42
Time Lines at Exit (Part C)Time Lines at Exit (Part C)• COSF completed near exit or transition
meeting with Part B• Anchor assessment completed between
2.6 IFSP meeting and child’s 3rd birthday• Outcome data is due to Data-Tot system
(and if appropriate, Part B system):– no later than 30 days after child’s exit from
Part C, or– no later than 30 days after child’s 3rd birthday
(whichever comes first).
43
Additional information about Additional information about child indicators (Part C)child indicators (Part C)
•Child must be in program at least 6 months for EXIT data to be counted•Data must be collected near entry and near exit•Collection of Entry data begins:
•for ITP July 1, 2006•for SDE September 1, 2006
44
Roles and Responsibilities Roles and Responsibilities (Part C)(Part C)
• Child information compiled from multiple sources – Service Coordinator
• Scoring/completion of COS Form – Primary Therapist or Multi-disciplinary Team
• Outcome Ratings transferred to Data-Tot Entry form – Service Coordinator
• Data entered into Data-Tot – Data Entry operator
45
Difference of Opinion? Difference of Opinion? (Part C)(Part C)
• If therapists or other team members can not agree on a rating…– Review data at MDT, seek consensus– If no resolution, provide Child Team
supervisor with all data– Child Team Supervisor will assign final
rating– Note different perspectives on form
46
Summary Steps in COS Summary Steps in COS Process (Part C)Process (Part C)
• Gather information from multiple sources (complete anchor assessment if necessary)
• Synthesize data and complete COS Form
• Update Data-Tot enrollment form and submit for entry
• Share data with Part B if appropriate
47
48
Data Collection Process for Data Collection Process for Part BPart B
Target populations and Entry data• All children enrolled in early childhood
special education on September 1, 2006 or later must have entry data (baseline) collected within 45 calendar days of initial consent for placement.
• Children transitioning from Part C -Exit from Part C may be used for Part B entry if an approved anchor tool was administered
49
Exit data will be collected Exit data will be collected Part BPart B
• EC Outcome Exit data will be collected within 30 days of the end of the school year (including ESY) for all children who are 5 years old on or before Sept 1st of the current school year (cut-off date for kindergarten entry) no matter if they are staying in a preschool setting or going into kindergarten.
• If moves to another preschool, transfer outcome entry data and information with eligibility report and student file.
50
Exit data will be collected Exit data will be collected Part B (cont’d)Part B (cont’d)
• If determined no longer eligible before 5, exit outcome data is collected on all three outcomes within 30 days of no longer meeting eligibility criteria.
• Prior to a move out of state, collect data 30 days prior to leaving the program
• If child leaves before 5 without notice and prior to administering the anchor assessment, complete the COSF with the information you have.
51
Roles (Part B)Roles (Part B)• Preschool teachers have the lead role
to gather the necessary information for the COSF, preferably during team meetings.
• Related service providers also provide assessment data, observation, etc.
52
Determining the score Determining the score 1-7 (Part B)1-7 (Part B)
• This should be a team decision – by consensus to insure validity and reliability.
• Scores may be averaged• If there is disagreement, an
administrator should make the final determination after reviewing the COSF.
53
Process (Part B)Process (Part B)• Gather all assessment information• Complete the COSF – may be
incorporated at time of IEP meeting. • Consent is not required for this
process.
Enter score 1-7 & (Y or N for exit) in web-based data system
54
Exit Data Collection (Part B)Exit Data Collection (Part B)
All children that have been in the program for 6 months or more must have exit data collected.
55
Data reporting to the state Data reporting to the state (Part B)(Part B)
• A real-time data collection system is being constructed so data can be entered at the convenience of the preschool teacher to avoid lost data.
• Data should be entered as soon as possible.
56
The Child Outcome Summary Form
(COSF)
57
Child Outcome Summary FormChild Outcome Summary Form Key Features Key Features
• Is NOT an Assessment• Is NOT about eligibility determination• Does NOT rate or summarize
– Info about services provided to child– Family’s satisfaction with services– Information for planning services for child
• Blends diverse data from multiple sources into a consistent, reliable unit of measurement that can be aggregated and tracked over time
58
Child Outcome Summary FormChild Outcome Summary Form Key Features Key Features
• Uses diverse information for global view of a child
• Compares a child’s functioning to same-aged peers
• Documents progress and movement toward typical development
• Considers functional abilities across diverse settings (not under ideal circumstances)
• Provides information about child’s overall sense of functioning in the three outcome areas
59
The Child Outcome The Child Outcome Summary Form (COSF)Summary Form (COSF)
– Cover sheet• Summary of evidence used to complete
ratings • People involved in assigning rating
– 7-point rating , with an overall rating for each required outcome area• Highest score (7) = outcome achieved at
age-expected level• Lowest score (1) = farthest distance from
age-expectations
– Progress Question (to be completed only at exit)
60
61
Thinking about how children are doing Thinking about how children are doing with regard to each outcome.with regard to each outcome.
7Age-expected
skills & behavior
Movement away from age-expected
Movement toward age-expected
62
Outcome Ratings: Outcome Ratings: A 7-Point ScaleA 7-Point Scale
1 - Not Yet 2 - Between emerging and not yet3 - Emerging 4 - Between somewhat and emerging5 - Somewhat 6 - Between completely and somewhat7 -Completely
63
64
Considerations when Considerations when completing Child Outcome completing Child Outcome
Summary FormSummary Form
– Team’s input is required – Consider role of assistive
technology/accommodations (i.e. sign language, wheel chair)
– Consider child’s culture and language– Document special considerations that
impacted child’s development– Reflect the child’s actual functioning – not
what the child is capable of under the ideal or highly unusual circumstances
65
Considerations when Considerations when completing Child Outcome completing Child Outcome
Summary FormSummary Form• OSEP reporting requires two data
points for each outcome– Near entry (sets baseline)– Near exit (shows progress)
• At Exit only, answer question “Has child shown new skills or made progress in outcome area?”
• Child CAN make progress without “changing numbers” on the scale
66
Identifying who made Identifying who made progressprogress
• Progress = moving up a point on the scale in a subsequent rating, e.g., 3 to 4
• Progress= staying at the same rating but having a “yes” for the progress question
67
Parental InvolvementParental Involvement• Parental consent for evaluations is required, as
always.• Parents give input to the COSF through
assessment process and IFSP/IEP development.• Parents need not be present when COSF is
completed.• Completion of the COSF is NOT an assessment
and does not require parental consent.• Discuss/share Parent’s Handout to inform
parents about process and purpose.
70
What Did We Cover?What Did We Cover?• Background and History• OSEP’s Three Functional Outcomes• Assessment Practices• Specific Requirements for Part C & Part B• Tools (Anchor Assessment List,
Crosswalks, COSF, Instructions, Decision Tree, Parent Talking points)
• Practice• Local Planning
72
Good outcome data is a tool to produce good outcomes for children and
families.
Recommended