School-wide PBIS: Using Data for Effective Coaching Rob Horner University of Oregon

School-wide PBIS:Using Data for Effective Coaching

Rob HornerUniversity of Oregon

www.pbis.org www.uoecs.org

http://www.pbis.org/

Goals Address questions you face as a PBIS Coach

Building Decision-Systems Basic Foundations Access to core data sources Rubric for using data to make decisions On-going review and improvement using data

Data needed for effective SWPBIS coaching Using Fidelity data to assist teams in SWPBIS implementation

Team problem solving processes Establishing the Foundations Using data for action planning Using data for implementation assessment and adaptation

Using Data for Decision-makingWhat decisions are needed?

Who needs what information at what time in what form to make effective decisions?

ActionPlanning

Improving Decision-Making

Problem

Problem

Solution

From

To

Problem

Solving Informati

on

Solution

School-Team DecisionsAre we implementing SWPBIS?

Are we implementing at all three tiers? What would be the next smallest change that would

make the biggest effect?

If we are implementing, are the procedures benefiting students?

Are there problems? Define problem with precision:

What, Where, When, Who, Why What proven actions (interventions, practices, packages)

would address the problem yet fit with current strengths? Are selected actions being implemented and effective?

How do they need to be adapted? Are selection actions producing desired outcomes?

Fidelity of ImplementationMeasuring fidelity is more

than accountability. Measuring fidelity and regular action

planning are key procedures for “getting it right”

Help teams use their strengths to find the path that best helps them achieve the SWPBIS core features.

Be careful about adoption too many strategies/ practices.

Do a small number of things well.

Level of Support

Research Measures(2-4 hours)

Annual Self-Assessment Measures

(45-60 min)

Progress Monitoring Measures

(15 min)Universal (Tier I) School-wide

Evaluation Tool (SET)

Self-Assessment Survey (SAS)

Benchmarks of Quality (BoQ)

Team Implementation Checklist (TIC)

Secondary and Tertiary (Tier II, Tier III)

Individual Student School-wide Evaluation Tool

(ISSET)

Benchmarks of Advanced Tiers (BAT)

Measure of Advanced Tiers Tool (MATT)

Overall Implementation

Implementation Phases Inventory (IPI)

Phases of Implementation (POI)

Phases of Implementation Checklist (PIC)

SWPBIS Measures of Fidelity( See PBIS Evaluation Blueprint; Measures available at www.pbisassessment.org)

How Knowledgeable/Comfortable are you with ( 1= low; 5 = high):

School-wide Evaluation Tool SETSelf-Assessment Survey SASTeam Implementation Checklist TICBenchmark of Quality BoQ

Individual Student SET ISSETBenchmark of Advanced Tiers BATMeasure of Advanced Tiers Tool MATT

Coaching for Implementation Are we implementing SWPBIS?

Team Implementation Checklist (15 min) Benchmark of Quality (45-60 min) School-wide Evaluation Tool (2-4 hours)

How Often Should Data be Collected? Initial implementation (every 3rd or 4th meeting) On-going (annually)

------------------------------------------------------------------------

Total Score Are we improving?

Sub-scale score What is working where do we focus next?

Item score Action planning

Elementary

Middle

Team Checklist: Subscale ScoresPercentage of Total Points

Comm

itmen

t

Asse

ssm

ent

Team

Define

Exp

Teac

h Ex

p

Rewar

d

Conse

quen

ce

Class

room

Info

rmat

ion

Dist S

uppo

rt0

10

20

30

40

50

60

70

80

90

100

Time 1 Time 2 Time 3 Time 4


Comm

itmen

t

Asse

ssm

ent

Team

Define

Exp

Teac

h Ex

p

Rewar

d

Conse

quen

ce

Class

room

Info

rmat

ion

Dist S

uppo

rt0

10

20

30

40

50

60

70

80

90

100

Time 1


Comm

itmen

t

Asse

ssm

ent

Team

Define

Exp

Teac

h Ex

p

Rewar

d

Conse

quen

ce

Class

room

Info

rmat

ion

Dist S

uppo

rt0

10

20

30

40

50

60

70

80

90

100

Time 1 Time 2


Comm

itmen

t

Asse

ssm

ent

Team

Define

Exp

Teac

h Ex

p

Rewar

d

Conse

quen

ce

Class

room

Info

rmat

ion

Dist S

uppo

rt0

10

20

30

40

50

60

70

80

90

100

Time 1 Time 2 Time 3


Comm

itmen

t

Asse

ssm

ent

Team

Define

Exp

Teac

h Ex

p

Rewar

d

Conse

quen

ce

Class

room

Info

rmat

ion

Dist S

uppo

rt0

10

20

30

40

50

60

70

80

90

100

Time 1 Time 2 Time 3 Time 4

Your Turn: What should team focus on ?

Comm

itmen

t

Asse

ssm

ent

Team

Define

Exp

Teac

h Ex

p

Rewar

d

Conse

quen

ce

Class

room

Info

rmat

ion

Dist S

uppo

rt0

102030405060708090

100

Team Implementation Checklist


Comm

itmen

t

Asse

ssm

ent

Team

Define

Exp

Teac

h Ex

p

Rewar

d

Conse

quen

ce

Class

room

Info

rmat

ion

Dist S

uppo

rt0

102030405060708090

100

Team Implementation Checklist


Comm

itmen

t

Asse

ssm

ent

Team

Define

Exp

Teac

h Ex

p

Rewar

d

Conse

quen

ce

Class

room

Info

rmat

ion

Dist S

uppo

rt0

10

20

30

40

50

60

70

80

90

100Time 1 versus Time 2

Demonstration School ExemplarNCES ID: Zenith, Winnemac

Demonstration DistrictNCES ID :

School Year Number of Responses

Date Collected Action Who/When

2009-10 1 09/30/2009

Feature Score (0, 1, 2)

Establish Commitment

1. Administrator's Support & Active Involvement. 12. Faculty/Staff Support. 1

Establish & Maintain Team

3. Team Established (Representative). 14. Team has regular meeting schedule, effective operating procedures.

2

5. Audit is completed for efficient integration of team with other teams/initiatives addressing behavior support.

0

Conduct Self-Assessment

6. Team completes the Team Implementation Checklist (TIC).

2

7. Team summarizes existing school discipline data. 18. Team uses self-assessment information to build implementation Action Plan (areas of immediate focus).

0

Demonstration School ExemplarNCES ID: Zenith, Winnemac

Demonstration DistrictNCES ID :

School Year Number of Responses

Date Collected Action Who/ When

2011-12 1 09/15/2011

Feature Score (0, 1, 2)

Establish Commitment

1. Administrator's Support & Active Involvement. 2

2. Faculty/Staff Support. 2

Establish & Maintain Team

3. Team Established (Representative). 1

4. Team has regular meeting schedule, effective operating procedures.

2

5. Audit is completed for efficient integration of team with other teams/initiatives addressing behavior support.

0

Conduct Self-Assessment

6. Team completes the Team Implementation Checklist (TIC).

2

7. Team summarizes existing school discipline data. 2

8. Team uses self-assessment information to build implementation Action Plan (areas of immediate focus).

2

Using Student Impact DataUniversal ScreeningProgress Monitoring

--------------------------------------------------

Team Meetings and Use of Data for Problem Solving

Universal Screening and Progress Monitoring.

Data Sources Decisions

Universal Screening

Behavior : Oct, Feb

Reading: Sept, Nov, Mar

SWPBIS Fidelity Measures

Teacher reportSSBD/ SSRS/ SSISOffice Discipline Referrals

Oral Reading Fluency (etc)

Are Tier I supports in place?

What students need more intensive support?

Initiate early intervention

Progress Monitoring

Fidelity: Match to level of intensity of support

Impact: Match to level of intensity of support

CICO/ ISIS fidelity measures

Office Discipline ReferralsCheck-in Check-out pointsISIS intensive support

Oral Reading FluencyPhonemic SegmentationComprehension

Are Tier II and Tier III supports implemented as planned?

Are supports effective?

Are more intensive/ individualized supports needed?

~80% of Students

~15%

~5%

0-1 office discipline referral

6+ office discipline referrals

2-5 office discipline referrals

Using office discipline referrals as a metric for universal screening of student social behavior

Newton, J. S., Todd, A. W., Algozzine, K., Horner, R. H., & Algozzine, B. (2009). The Team Initiated Problem Solving (TIPS) Training Manual. Educational and Community Supports, University of Oregon, unpublished training manual. 32

Using the Referrals by Student report as a Universal Screening

Tool

33

Aug Sept Oct Nov Dec Jan Feb Mar Apr May Jun0

2

4

6

8

10

12

0-12-56+

Cum

ula

tive M

ean

OD

Rs Cumulative Mean ODRs Per Month

for 325+ Elementary Schools 08-09

Jennifer Frank, Kent McIntosh, Seth May

Helping Teams Use Data at All Three TiersTier I Office discipline

referrals

~80% of Students

~15%

~5%

ESTABLISHING a CONTINUUM of Data Options within SWPBS

SECONDARY PREVENTION• Fidelity:

• BAT, MATT* Student Outcomes:

• CICO-SWIS• ODRs

TERTIARY PREVENTION• Fidelity: ISSET, BAT, MATT

• ISIS-SWIS

• Student Outcomes: • ISIS-SWIS

PRIMARY PREVENTION• Fidelity: TIC, BoQ, SET

• Student Outcomes: • Office Discipline Referrals

SECONDARY PREVENTION

•

TERTIARY PREVENTION•

PRIMARY PREVENTION• • • • • •

Coaching Tier III Data UseNeeds

◦ System to assist coordination of individualized Team

◦ System for managing (a) assessments, (b) plan development/modification, and (c) team meeting minutes

◦ System for building data collection tailored to individualized plan

Collect Fidelity Data: Are we doing plan? Collect Impact Data: Is plan benefiting student?

◦ System for on-going collection, summarizing and reporting of data for team decision-making.

Helping Teams use Data for Decision-Making

Build team foundation Roles, Agenda, Schedule

Use data to define problems with precision

Use data to build “Action Plans” that fit the local context and will work.

Use data to assess if action plans are being implemented with effect

Problem-Solving Meeting Foundations

Structure of meetings lays foundation for efficiency & effectiveness

Meeting Foundations ElementsFour features of effective

meetings Predictability Participation Accountability Communication

Define roles & responsibilities Facilitator, Minute Taker, Data Analyst

Use electronic meeting minutes format

43

Newton, J. S., Todd, A. W., Algozzine, K., Horner, R. H., & Algozzine, B. (2009). The Team Initiated Problem Solving (TIPS) Training Manual. Educational and Community Supports, University of Oregon, unpublished training manual.

A. Predictability1. Defined roles, responsibilities and expectations

for the meeting2. Start & end on time, if meeting needs to be

extended, get agreement from all members3. Agenda is used to guide meeting topics4. Data are reviewed in first 5 minutes of the

meeting5. Next meeting is scheduled

B. Participation5. 75% of team members present & engaged in

topic(s) 6. Decision makers are present when needed

What makes a successful meeting?

What makes a successful meeting?

C. Accountability7. Facilitator, Minute Taker & Data Analyst come prepared for

meeting & complete during the meeting responsibilities8. System is used for monitoring progress of implemented

solutions (review previous meeting minutes, goal setting)9. System is used for documenting decisions10. Efforts are making a difference in the lives of

children/students.

D. Communication11.All regular team members (absent or present) get access

to the meeting minutes within 24 hours of the meeting12.Team member support to practice team meeting

norms/agreements

Define roles for effective meetingsCore roles

◦Facilitator◦Minute taker◦Data analyst◦Active team member◦Administrator

Backup for each role

Can one person serve multiple roles?

Are there other roles needed?

Typically NOT the administrator

46


Who is Responsible?Action Person Responsible

Reserve Room Facilitator

Recruit items for Agenda Facilitator

Review data prior to the meeting

Data Analyst

Reserve projector and computer for meeting

Minute Taker

Keep discussion focused Facilitator

Record Topics and Decisions on agenda/minutes

Minute taker

Ensure that problems are defined with precision

Facilitator

Ensure that solutions have action plans

Facilitator

Provide “drill down” data during discussion

Data Analyst

End on time Facilitator

Prepare minutes and send to all members

Minute taker

Activity # 1 (7 min)Select

◦Facilitator◦Data Analyst◦Minute Taker

Back up for each

50

Role Primary Backup

Facilitator

Data Analyst

Minute Taker

Next role review date:


Organizing for an effective problem solving conversation

Problem

SolutionOut of

Time

Use Data

A key to collective problem solving is to provide a visual

context that allows everyone to follow and contribute

51


Newton, J. S., Todd, A. W., Horner, R. H., Algozzine, B., & Algozzine, K. (2012) (Version 1.2)



Where in the Form would you place:

1.Planning for next PTA meeting?

2.There have been five fights on the playground in the past 3 weeks.

3.Update on CICO implementation

4.Increasing gang recruitment as an agenda topic for today.

5.Next meeting report on lunch-room status.

Summary: Team-based Problem Solving

Team FoundationsRolesMinute form

Defining ProblemsBuilding SolutionsAction Planning/ Follow up / Adaptation

Documents

School-wide PBIS: Using Data for Effective Coaching Rob Horner University of Oregon