Chris Borgmeier, Dave McKay, Anne Todd, Celeste Dickey, Rob Horner

October 2008cborgmei@pdx.edu; david.McKay@lblesd.k12.or.us

awt@uoregon.edu; robh@uoregon.edu; cdickey@uoregon.edu

Clarify evaluation questions for effective decision making around discipline and school climate

Define critical features for organizing the data

Define data sources needed ◦ Pbseval for discipline, SIS for achievement &

attendance Define the cycles and timelines for data

collection and reporting

Guide Decision Making◦ Plan◦ Progress monitor◦ Inform Change/Need

Visibility/Support

FUNDING

Celebrate accomplishments

How many schools are adopting PBIS?◦ What percentage of schools in state, region/district?

SET, TIC, BoQ How many schools are implementing SWPBIS at

criteria? SET, TIC, BoQ

Are schools implementing SWPBIS perceived as safe? School Safety Survey

What are discipline patterns? SWIS

What are attendance patterns? SIS

What are achievement patterns? SIS, State Testing scores DIBELS, AIMS Web

Written Documents◦ Visual/Graphic Display◦ Easy to Read

Oral Reports/presentations◦ Status reports

Website◦ Post surveys and other resources◦ Schedule reminders to school to complete survey

Make fair comparisons◦ Account for 20% change in attendance◦ Calculate per 100 students◦ Percent of total◦ Account for days of week if used for report

District/ Regional Leadership Team Meetings◦ To guide decision making

Quarterly, annually Monitor Progress Annual Action Planning

School Board Meetings State Leadership Team Meetings State/regional conferences Newsletters

Establish a rhythm

Measure Year One Year 2 Year Three

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4

EBS Survey X

Team Checklist

(TIC)

X X X X X

SET or BoQ X X X

ClassroomSelf-

Assessment

X X X X

CISS X X X X

I-SSET or I-BoQ

X X

An Evaluation Model for School-wide PBSDesigned for a school or cohort of schools

SWIS + PBS Surveys + PBS Eval

Online survey application for school teams and staff (free of charge)◦ www.pbssurveys.org ◦ Complete Local Coordinator Form and submit to

PBS Applications Manager 1235 UO Eugene, Oregon 97403

PBS Surveys 5 different measures for schools to use that measure the:◦ percent of implementation of SW-PBS◦ risk and safety factors◦ Rates of office discipline referrals

◦ Each of the 5 measures are used at different times and for different purposes

Team Implementation ChecklistEBS Self-Assessment SurveySchool Safety SurveyBenchmarks of QualitySET

PBSEval™ is ◦ a web-based progress monitoring service that

permits specific district and state-level organizations such as the Licensee to monitor and review selected area-wide administrative data and generate automated reports (collectively “Data”) in a manner that does not disclose personally identifiable information

Accessing PBS Eval ◦ License agreement

State $1000/year District $500/year

◦ Participate in 90 minute training

284 schools 674 schools as of 10/24/08

There are 1239 public schools in Oregon

SET’s x Year

2004-05 73 2005-06 148 2006-07 224 2007-08 249

Elementary Schools (K-6)

Elementary Schools

TIC’s x Year from OR MS

2004-05 27 2005-06 30 2006-07 76 2007-08 107

SET’s x Year

2004-05 16 2005-06 35 2006-07 71 2007-08 71

Middle School

SET’s x Year

2004-05 7 2005-06 18 2006-07 21 2007-08 44

High School

Oregon Schools 2003-2008ODR/100/Day - Major Referrals Only

0

0.2

0.4

0.6

0.8

1

1.2

K-6 MS HS K*-K12

School Levels

Mea

n O

DR

/100

/Day

2003-04

2004-05

2005-06

2006-07

2007-08

81

105

103147 170

31

37

3147

43

8

15

1314

19

25 31

25

27

36

Nat’l Elem Mean .39 ODR/100/ Day

Nat’l MS Mean .97 ODR/100/ Day

Outcome Data ODR Suspension/Expulsion LRE Attendance Academic Achievement School climate surveys Staff retention

Implementation Data Self Assessment SET

Outcome Data Academic

Achievement State Test Scores CBM/ DIBELS/

EZ CBM/ AimsWeb LRE Attendance

Implementation Data

PET-R Healthy Systems

Checklist In Program

Assessments School/ Classroom

Observations

Across Schools/District-wide Data◦ SWIS won’t do it◦ Capabilities of new SAMI?◦ PBS Surveys won’t do it

Can access all data but have to compile it your self◦ DIBELS Survey◦ PBS Eval

◦ Other programs -- eSIS? Etc?◦ Need some Excel skills

Northwest School District Elementary SchoolsDistrict-wide Means

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

ODR/100 stud/Day DaysOSS/100stud/Day

2004-05

2005-06

2006-07n=8n=5n=5

Nat'l Elem Mean ODR/100

Stud/Day = .39

Northwest School District Middle Schools

District-wide Means

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

ODR/100 stud/Day DaysOSS/100stud/Day

2004-05

2005-06

2006-07n=2n=2n=2

Nat'l MS Mean = .97

ODR/100 stud/Day

Northwest School District Elementary Schools

2006-07

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

ODR/100 stud/Day DaysOSS/100stud/Day

Elem 1

Elem 2

Elem 3

Elem 4

Elem 5

Elem 6

Elem 7

Elem 8

Nat'l Elem Mean ODR/100

Stud/Day = .39

Northwest School DistrictMiddle Schools

2006-07

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

ODR/100 stud/Day DaysOSS/100stud/Day

MS 1

MS 2

Nat'l MS Mean = .97

ODR/100 stud/Day

Carol Sadler

16 Schools, 12,000 students◦ 10 elementary, 1 charter, 3 middle, 2 high

Special Programs participation◦ 1,200 Special Education (10%)◦ 1,800 English Language Learners (15%)◦ 1,500 Talented and Gifted (12%)

Socio-economic status◦ Title 1 in 5 elementary schools

Free/Reduced ranges from 7% to 58%

20

40

60

80

100

ES1

ES2

ES3

ES4

ES5

ES6

ES7

ES8

ES9

MS1

MS2

MS3

HS1

HS2

2003-04

2004-05

ODR per 100 Students Per Day T-TSD High Schools

00.2

0.40.60.8

1

1.21.4

99-00 00-01 01-02 02-03 03-04 04-05

High Schools

Ref

erra

ls

U of OSWIS Avg 04-05: 1.23 per 100 student/per day

.52

ODR per 100 Students Per Day T-TSD Middle Schools

0

0.2

0.4

0.6

0.8

1

99-00 00-01 01-02 02-03 03-04 04-05

Middle Schools

Ref

erra

ls

U of OSWIS Avg 04-05: .97 per 100 student/per day .45

ODR per 100 Students Per Day T-TSD Elementary Schools

0

0.1

0.2

0.3

0.4

99-00 00-01 01-02 02-03 03-04 04-05

Elementary Schools

Ref

erra

ls

U of O SWIS Avg 04-05: .39 per 100 student/per day

.16

For 5th grade students who have attended T-T since kindergarten or first grade (i.e., intact groups)

94% met or exceeded OSA Reading/Literature benchmark

in 2004-05

~80% of Students

~15%

~5%

(Walker, et al. 1996)

80% received NO referrals;

90% received 0-1 referrals

8% received 2-5 referrals

1% received 6 or more referrals

Dave McKay and Scott Perry