Using Data to Guide Decision Making at the Pre-school Level

Tim Lewis, Ph.D. & Susan BrawleyUniversity of Missouri

Center on School-wide Systemsof Positive Behavior Support University of Missouri - Columbia

Pre-school / EC Settings

• Program-wide• Classroom focus• Reliance on “informal” decision making• Need for better data to a) conduct systems

analyses (problem solve), b) identify at-risk students sooner, c) guide intervention and environmental support, and d) progress monitor

Data-Based Decision Making

• Needs Assessment • System/Intervention Evaluation• Identifying at-risk students for more intensive

supports

Data Decision Logic

Data-Based Decision Making1. Determine what questions you want to

answer2. Determine what data will help to answer

questions3. Determine the simplest way to get data4. Put system in place to collect data 5. Analyze data to answer questions

1. Determine what questions you want to answer

Examples• Can we predict problems/success?– When/where/who?

• Possible “function” of problem behavior?• What environmental changes/supports are

needed?

2. Determine what data will help to answer questions

• Existing data set(s)• Current data collection• Additional / new data

• Confidence in accuracy?• Complete picture?

3. Determine the simplest way to get data

• Agreement on definitions• Standard forms / process• Frequency of collection• Target “Multi-purpose” data/use

Train ALL staff on use & provide on-going TA

4. Put system in place to collect data

• Build on existing systems• Add components over time• Central entry point

5. Analyze data to answer questions

• Trends• Instruction & supports in place/not in-place• Pre/post “big outcomes”• Comparisons (norm / local)– Relative growth– Absolute growth

Needs Assessment

Pre-school SASPre-school SET

Pre-school SAS

PRE-SET (Horner, Benedict, & Todd, 2005)

• Adaptation of an assessment tool called the School-wide Evaluation Tool (SET) used in K-12 educational settings to measure critical features of school-wide PBS.

• The Pre-SET assesses classroom and program-wide variables across 9 categories:

A. Expectations DefinedB. Behavioral Expectations TaughtC. Appropriate Behavior AcknowledgedD. Organized and Predictable EnvironmentE. Additional SupportsF. Family InvolvementG. Monitoring & Decision-MakingH. ManagementI. Program & District-Wide Support

Pre-SET Administration

• Information necessary for completion of the Pre-SET is gathered from multiple sources including review of permanent products, observations, and staff and child interviews.

• A Pre-SET should be conducted for each classroom within an early childhood program if the implementation status or practices (e.g., classroom rules) are different for each classroom.

• The Pre-SET may be conducted at the program level if all classrooms within the early childhood program are at the same implementation status and use the same practices (e.g., have same classroom rules).

System/Intervention Evaluation

Pre-school ODRs?

EC Behavioral Report

Identifying at-risk students

Tier II / III

At-risk Students

• Behavioral incident reports– Create data decision rules – capture 10-15%

• Teacher Referral– Clear process with supporting data

• Screening

Early Screening Project(Walker, Severson, & Feil)

Proactive multiple-gated screening:

Stage One: Teacher ranking of externalizing & internalizing behaviors

Stage Two: Teacher ratings of the 5 highest ranked children

Stage Three: Direct observations & parent questionnaires of children exceeding Stage Two criteria

Teacher RankingTeacher Ranking

Teacher RatingsTeacher Ratings

ObservationsObservations

Missouri SW-PBS

Missouri Regional Professional Development Centers

MO SW-PBS Early Childhood PD

• Annual Statewide Early Childhood Summit• Topics: – Problem-Solving with Class-wide Data– Family Involvement– Maintaining Developmentally Appropriate Practices with

Program-wide PBS

• 15 Programs that Responded to Informal Survey on Data Collection Systems Indicated:– 4 use Excel– 4 use SWIS– 7 use nothing

EC Summit Feedback• Barriers to Implementing PW-PBS

– Time– Buy in – Lack of data system

• To Improve Data Collection and Use Need– Time to record– Handy and practical forms– Training about data– Consistent definition of terms– Staff buy in– Communication and collaboration between EC and Kindergarten– Time for purposeful data collection

Pilot of EC Data Collection Tools• Sample fields

– Name– Date– Age– Challenging Behavior (vs. Problem Behavior)– Routine (vs. Location)– Teacher Response

• Graphs generated– Per day per month– Behavior– Time– Location (routine)– Student

Rationale for Data Focus• Increasingly schools are using three-tiered models for

prevention and intervention in K-12 (Ardoin, Witt, Connell, & Koenig, 2005).

• Early childhood programs are beginning to use three tiered models but challenges exist when trying to implement the key features (Hawken, Vincent and Schumann, 2008).

• One of the critical parts of successfully implemented tiered models for social behavior and academic skills is the collection of data for decision making (Hawken, Vincent and Schumann, 2008).

Rationale for Data Focus• Practitioners need data to identify the type and level of

support that children need (Barnett, et al., 2006; Hojnoski, Caskie, Gischlar, Key, Barry, & Hughes, 2009).

• Early childhood educators tend to rely on intuition and informal observations of children rather than the collection and systematic use of data to guide decision making (Sandall, Schwartz LaCroix, 2004).

• Early childhood programs do not, and should not, use office discipline referrals, so there is a less formalized method of tracking and reporting challenging behavior (Stormont, Lewis, Beckner, 2005).

Rationale for Data Focus

•There is a need to understand more about the current practices of data in in order to build capacity for more early childhood programs to collect and use data (Fulignia, Howesa, Lara-Cinisomob & Karolyb, 2009; Kincaid, Childs, Blasé, & Wallace, 2007; Schwartz & Olswang,1996).

• Potential barriers to data collection and use in decision making are time, lack of knowledge and lack of experience with data (Hojnoski, et al., 2009).

Rationale for Data Focus

Potential factors that increase the likelihood of data collection and use in decision making include:

•Additional professional development for all staff, terminology changes suitable to EC programs and streamlined data collection systems (Muscott, Pomerleau, and Szczesiul, 2009).

•Simplified methodology of data collection, reduction in number of variables on which to collect data and reduction of time required for data collection (Gunter, Callicot, Denny, & Gerber, 2003).

Research Study: Status of Data Collection and Use in Early Childhood

• Study of Head Start, early childhood special education and early childhood general education programs in Missouri

• Administrators and teaching staff• On-line survey

Research Questions

• To what extent are preschool programs currently collecting and using data for decision-making?

• What barriers exist in data collection and use in decision-making?

• What factors may increase the likelihood that programs collect and use data?

• What relationships can be found between administrator and teacher views on data practices?

Preliminary Study Findings

–Data collection–Analyzing and Using Data–Data systems–Barriers–Positive Factors–Beliefs

Data Collection: Importance of Practice

Data Collection: Frequency of Practice

Data Analysis: Importance of Practice

Data Analysis: Frequency of Practice

Barriers: Level of Agreement

Positive Factors: Level of Agreement

Final Thoughts

• Don’t collect data for collection sake – make sure informs the process

• Don’t “drown” in data – keep focused on the question

• Data without context are simply numbers