National Leadership Forum Chicago, Illinois October 2010 presented by Anne Todd, UO Developed by...

National Leadership ForumChicago, Illinois

October 2010presented by Anne Todd, UO

Developed by Steve Newton, Anne Todd, Rob Horner, UO Bob Algozzine, & Kate Algozzine, University of NC at Charlotte

Provide an overview of the TIPS system◦ Research results from two studies

Preview effective meeting practices through use of the TIPS Meeting Foundations◦ Preview Foundations Checklist◦ Roles & Responsibilities◦ Preview Meeting Minute Form

Be able to identify a problem that includes (not include) precision elements critical for problem solving

Every school has teams Teams are being expected to do problem

solvingSelect curriculaGet training and implement new ideas/programsProvide efficient leadership

“Communities of Practice” Teams need to report data to staff, families,

administration, district, state Teams NEED data to do good problem solving. Most teams are not skilled at running problem

solving meetings and using data for decision-making.

Includes:◦ Tools to define a system for effective meetings

Roles, responsibilities, materials, accountability and procedures

◦ Steps of effective problem solving A strategy for assessing, monitoring and evaluating

the implementation and results of solutions Can be used with other data sets

TIPS TrainingOne full day team trainingTwo coached meetings

Team MeetingUse of electronic meeting minute systemFormal roles (facilitator, recorder, data analyst)Specific expectations (before meeting, during meeting,

after meeting)Access and use of dataProjected meeting minutes

Research tool to measure effectiveness of TIPS TrainingDORA (decision, observation, recording and analysis)Measures “Meeting Foundations” & “Thoroughness of

Problem Solving”

SYSTEMS

PRACTICES

INFORMATION

SupportingStaff & Student Behavior and Decision Making

Building Capacity and Sustainability

OUTCOMES

For Social Competence,Academic Achievement, and Safety

Hold effective meetings that use data to problem solve and plan AND that result in

positive student outcomes

Team-based, documentation

, regular communicatio

ncycles Meeting

FoundationsMeeting Minute

FormatProblem solving

routine

SWIS DIBELS

Aims Web

2008-09 Single Subject Study ◦ 4 teams in Oregon◦ Multiple baseline design◦ SW PBIS meetings & progress monitoring literacy meetings

2009-2010 Randomized Control Trial Study with 34 teams◦ 22 teams in NC◦ 12 teams in Oregon

Need to conduct Team Training ◦ Team includes all members and a coach◦ Define Roles and Responsibilities is critical

Plan for absences (have back up people) Coaching is critical Training critical skills to facilitator, minute taker,

and data analyst◦ Keep people on track, ◦ document relevant information for progress monitoring and

evaluation◦ Launch the meeting with a data summary

Documenting decisions, actions, timelines, evaluation plan is critical for sustainability

Role◦ To create data summaries that will facilitate the

team in determining if there are problems jump starting a problem solving discussion, and evaluating the impact of solutions and fidelity of

implementation General Responsibilities

◦ Prepares a brief written summary for distribution at meetings using each of the data sources needed for problem solving and decision making

◦ Help to generate reports during the meeting as questions of the data arise

How?◦ Establish the role of a data analyst (and backup

person)◦ Teach data analyst to develop data summary

Oakes, DIBELS, SWIS…. Etc◦ Start meeting with defining the problem with

precision◦ Refine precision of problem statement through

inferences and hypothesis Have data accessible for custom report generation

during the meeting

Structure of meetings lays foundation for efficiency & effectiveness

04/19/23 13

Define purpose of the team◦ Decisions to be made, cycle of decision making,

and data source(s) to use Define roles & responsibilities Define team agreements about meeting

processes1. Inform facilitator of absence/tardy before meeting2. Be prepared for meeting by completing previously

assigned tasks3. Avoid side talk: Remind each other to stay focused4. Start and end on time5. Be an active participant6. Use electronic meeting minutes

04/19/23 14

Core roles◦ Facilitator◦ Minute taker◦ Data analyst◦ Active team member◦ Administrator

Backup for each role

04/19/23 15

Can one person serve multiple roles?

Are there other roles needed?

Typically NOT the administrator

04/19/23 16

04/19/23 17

Documentation of Logistics of meeting (date, time, location, roles) Agenda items for today’s meeting ( and next meeting) Discussion items, decisions made, tasks and timelines assigned Problem statements, solutions/decisions/tasks, people assigned to

implement with timelines assigned, and an evaluation plan to determine the effect on student behavior

Reviewing Meeting minutes An effective strategy for getting a snapshot of what happened at the

previous meeting and what needs to be reviewed during the upcoming meeting What was the issue/problem?, What were we going to do?, Who was going

to do it and by When?, and How are we measuring progress toward the goal?

Visual tracking of focus topics during and after meetings Prevents side conversations Prevents repetition Encourages completion of tasks

04/19/23 18

04/19/23 19

Problem

SolutionOut of

Time

Use Data

A key to collective problem solving is to provide a visual

context that allows everyone to follow and contribute

Langley Elementary PBIS Team Meeting Minutes and Problem-Solving Action Plan Form

Today’s Meeting: Date, time, location: Facilitator: Minute Taker: Data Analyst:

Next Meeting: Date, time, location: Facilitator: Minute Taker: Data Analyst:

Team Members (bold are present today)

Today’s Agenda Items Next Meeting Agenda Items01. 02. 03.

1. 2.

Information for Team, or Issue for Team to Address

Discussion/Decision/Task (if applicable) Who? By When?

Administrative/General Information and Issues

Implementation and EvaluationPrecise Problem Statement, based on review of

data(What, When, Where, Who, Why)

Solution Actions (e.g., Prevent, Teach, Prompt, Reward, Correction, Extinction,

Safety)Who? By When?

Goal, Timeline, Decision Rule, & Updates

Problem-Solving Action Plan

Our RatingYes So-So No

1. Was today’s meeting a good use of our time?2. In general, did we do a good job of tracking whether we’re completing the tasks we agreed on at previous meetings?

3. In general, have we done a good job of actually completing the tasks we agreed on at previous meetings?4. In general, are the completed tasks having the desired effects on student behavior?

Evaluation of Team Meeting (Mark your ratings with an “X”)

Where in the Form would you place:

1.Planning for next PTA meeting?

2.Too many students in the “intensive support” for literacy

3.Schedule for hallway monitoring for next month

4.There have been five fights on playground in last month.

5.Next meeting report on lunch-room status.

04/19/23 20

04/19/23 22

Any tasks assigned get copied to the meeting minutes of the next meeting as a follow up item

Meeting Agenda Item: Meeting Foundations Tasks: What, by whom, by when

04/19/23 23

Meeting Foundations

CollectCollect and Useand Use

DataData

Develop Hypothesis

Discuss andSelect

SolutionsDevelop andImplementAction Plan

Evaluate andRevise

Action Plan

Problem Solving Meeting Foundations

Team Initiated Problem Solving (TIPS) Model

Identify Problems

24

Defined◦ SWISTM is a web-based information system

for gathering, entering, summarizing, reporting and using office discipline referral information

Purpose◦ A progress monitoring tool for improving the

ability of school personnel to develop safe and effective learning environments

Universal Screening Tool◦ Proportion of students with

0-1 Office Discipline Referrals (ODRs) 2-5 ODRs 6+ ODRs

Progress Monitoring Tool Compare data across time

◦ Prevent previous problem patterns Define Problems with precision that lead to

solvable problems

0

10

20

Num

ber

of R

efe

rrals

per

Stu

dent

Students

~80% of Students

~15%

~5%

0-1 office discipline referral

6+ office discipline referrals

2-5 office discipline referrals

Using office discipline referrals as a metric for universal screening of student social behavior

Build a picture for the pattern of office referrals in your school.

Compare the picture with a national average

Compare the picture with previous years

Compare the picture with social standards of faculty, families, students.

Goal

1.Identify problems empirically2.Identify problems early3.Identify problems in a manner that leads to problem solving not just whining

Tot

al O

ffic

e D

isci

plin

e R

efer

rals

Total Office Discipline Referrals as of January 10

Change Report OptionsChange Report Options1.41.82.72.52.753.4900.000

Grade Range Number of Schools

Mean Enrollment per school

Median ODRs per 100 per school day

K-6 2565 452 .22

6-9 713 648 .50

9-12 266 897 .68

K-(8-12) 474 423 .42

Elementary School 465 students (465/ 100 = 4.6 X .22= 1.01)

Our rate of problem behavior has been above

the national average for

schools our size across 9 of 10

months this year. There has been a decreasing trend

since Dec.

Elementary School 1000 Students (1000/100 =10 X .22= 2.2) The rate of

problem behavior has

been at or below the national average for

schools our size across 6 of 10

months. The past 4 months have been below the

national average

Middle School 765 students (765/100 = 7.6 X .50= 3.8)

The rate of problem

behavior has been at or below

the national average for

schools our size across 9 of 10

months. The past 8 months have been below the

national average with a

decreasing trend

Describe the narrative for this school

Describe the narrative for this school

Describe the narrative for this school

Describe the narrative for this school

Our average Major ODRs per school day per month are higher than national average for a school of our enrollment size

Our average ODRs per school day per month are higher this year than for corresponding months of previous year

Our average ODRs per school day per month are showing an increasing trend

Faculty, parents, and students say our ODR levels are too high

43

1. Define problem by identifying What problem behaviors are involved in ODRs

2. Clarify problem by identifyinga) When ODRs are occurring (time of day)b) Where ODRs are occurring (location)c) Who is engaging in problem behaviors that

result in ODRs d) Why are problem behaviors continuing to

occur

44

1a. Too many ODRs 1b. Total of 22 aggression ODRs on playground last month; twice as many as last year & showing increasing trend this year; occurring during first recess; 15 different students involved; aggression appears to provide peer attention, and resolve unclear playground rules (who gets equipment),

2a. Verbal threats and gender harassment in the cafeteria are increasing; 80% of events are from 4 students during second lunch; We are unclear what is maintaining these behaviors.

2b. Behavior in cafeteria is uncivil and unsafe.

3a. Hallway noise is too loud (disruptive) during 7th grade passing periods before and after lunch.

3b. Hallway noise is unbearable.

4a. The number of ODRs per day has increased by 20% each month since school started.

4b. The number of ODRs per day has increased by 20% each month since school started. Most incidences are with 4-6 grade, in the afternoon. Students are engaging in inappropriate language and harassment.

45

CollectCollect and Useand Use

DataData

Develop Hypothesis

Discuss andSelect

SolutionsDevelop andImplementAction Plan

Evaluate andRevise

Action Plan

Problem Solving Meeting Foundations

Team Initiated Problem Solving (TIPS) Model

Identify Problems

46

Carly is having reading difficulties

50% of 2nd graders are not meeting math benchmarks

Carly is reading 20 cwpm (goal is 60), skips or guesses at words she doesn’t know, mostly during language arts

2nd graders, who entered school after Oct 31, do not know whole numbers 75-100 and are not accurately adding two digit numbers because of lack of skills

Our school did not meet AYP last year

The 5th graders are below the state proficiency score as compared to 5th graders in Oregon

The past two years this cohort of students (3rd to 5th grade) has gradually decreased in overall proficiency, their comprehension strand scores are low, we shifted to blended grade classes during their 4th grade year

Role◦ To create data summaries that will facilitate the

team in determining if there are problems jump starting a problem solving discussion, and evaluating the impact of solutions and fidelity of

implementation General Responsibilities

◦ Prepares a brief written summary for distribution at meetings using each of the data sources needed for problem solving and decision making

◦ Help to generate reports during the meeting as questions of the data arise

How?◦ Establish the role of a data analyst (and backup

person)◦ Teach data analyst to develop data summary

Oakes, DIBELS, SWIS…. Etc◦ Start meeting with defining the problem with

precision◦ Refine precision of problem statement through

inferences and hypothesis Have data accessible for custom report generation

during the meeting

Middle School – Grades 6, 7, & 8 565 students

51

0

5

10

15

20

Sep Oct Nov Dec Jan

School Months

Avg

. O

DR

s P

er S

cho

ol

Day

School Avg. National Avg. = 4.8

52

Identified problem◦ for last 4 mos., Major ODRs per day higher than

national avg.◦ increasing trend across all 5 mos.

53

Trevor Test Middle School 11/01/2007 through 01/31/2008 (last 3 mos.)

Referrals by Problem Behavior

0

10

20

30

40

50

60

70

80

90

100

110

120

130

140

Min

or

Tard

yB

om

bA

rson

Weapons

Oth

er

Unknow

nD

rugs

M-P

rpty

Mis

use

M-O

ther

M-D

ress

M-T

ech

Tech

Inapp a

ffectio

nO

ut bounds

M-U

nknow

nG

ang d

ispla

yS

kip

Tru

an

Lyin

gM

-Dis

ruptio

nD

ress

Tobacco

Alc

ohol

Com

bust

M-I

napp la

nF

org

e/T

heft

Vandal

M-C

onta

ct

M-D

isre

spt

Pro

p d

am

Agg/F

ight

M-T

ard

yS

kip

Hara

ss

Dis

respt

Inapp la

nD

isru

ptio

n

Nu

mb

er

of

Refe

rrals

Referrals by Time

0102030405060708090

100110120130

7:00

AM

7:30

AM

8:00

AM

8:30

AM

9:00

AM

9:30

AM

10:0

0 A

M

10:3

0 A

M

11:0

0 A

M

11:3

0 A

M

12:0

0 P

M

12:3

0 P

M

1:00

PM

1:30

PM

2:00

PM

2:30

PM

3:00

PM

3:30

PM

4:00

PM

4:30

PM

5:00

PM

Num

ber

of R

efer

rals

Referrals by Location

0

20

40

60

80

100

120

140

160

180

200

Ply

gd

Par

k lo

t

Unk

now

n

Off

ice

Lock

er r

m

Off

-Cam

pus

Sta

dium

Mus

ic r

m

Libr

ary

Bat

hrm

Gym

Bus

Zn

Bus

Spe

cial

evt

Oth

er

Cla

ss

Com

mon Hal

l

Caf

é

Nu

mb

er o

f R

efer

rals

Referrals by Student

0

10

20

30

40

50

60

70

80

1 13 16 18 2 20 24 28 30 33 38 4 9 17 21 37 43 23 31 39 40 41 5 8 11 29 12 22 25 35 42 6 14 34 15 26 36 7 3 19 32 27 10

Student No.

Num

ber

of R

efer

rals

54

1. Most Disruptions occur in Cafeteria2. Most Disruptions occur in Cafeteria

between 11:30 AM and 12:00 PM3. Most instances Inappropriate Language

occur in Cafeteria between 11:30 AM and 12:00 AM

Now…use a Custom Graph to confirm (or disconfirm) your inferences, starting with Disruptions, by grade level

55

Many instances of disruption (what)… occurring in cafeteria (where)… between 11:30 AM and 12:00 PM (when)… with large majority involving 6th graders

(who)… particularly Student #10 (who)

56

Write your precise Problem Statement as one element of your “Problem-Solving Action Plan”

The P-S Action Plan is simply a record of◦ team decisions and ◦ actions needed to implement the decisions

Here’s Problem Statement section of P-S Action Plan for Trevor Test Middle School

57

58

59

Many instances of disruption occurring in cafeteria between 11:30 AM and 12:00 PM; large majority involving 6th graders, particularly Student #10…

because (a) cafeteria overcrowded at that time, (b) 6th graders have received insufficient instruction in cafeteria expectations, and (c) disruption results in attention from adults and peers

Here’s hypothesis statement incorporated into P-S Action Plan

60

61

Is best explanation for what the data and your experience tell you

Provides a possible “why” for other Ws you discovered

AND guides you toward possible solutions

62

Prevent – Remove or alter “trigger” for problem behavior

Define & Teach – Define behavioral expectations; provide demonstration/instruction in expected behavior (alternative to problem behavior

Reward/reinforce – The expected/alternative behavior when it occurs; prompt for it, as necessary

Withhold reward/reinforcement – For the problem behavior, if possible (“Extinction”)

Use non-rewarding/non-reinforcing corrective consequences – When problem behavior occurs

Consider Safety issues

63

Prevent “Trigger”

Define & Teach

Reward/Reinforce

Withhold Reward

Corrective consequence

Other

Safety

64

Trevor Test Middle School

Hypothesis - cafeteria overcrowded; 6th graders with insufficient instruction in cafeteria expectations; attention from adults and peers rewarding disruption

Prevent “Trigger” Change lunch schedule so fewer students are eating between 11:30 AM & 12:00 PM?

Define & Teach Focus on 6th graders; define cafeteria expectations; develop and post expectation signage in cafeteria; demonstrate/teach expectations in class periods occurring just prior to lunch

Reward/Reinforce Set up “Friday 5” (extra 5 mins. of lunch time on Friday, if no ODRs occur in cafeteria during lunch time)

Withhold Reward Ensure staff don’t argue back and forth with student if instance of disruption occurs (may be an inadvertent reward); remind students that paying attention to a disruptive student can mess up Friday 5

Corrective consequence Ensure active supervision during lunch (add one supervisor between 11:30 AM and 12:00 PM?); ensure quick corrective consequence, per our handbook

Other Determine whether Behavior Support Program has been initiated for Student #10; if it has, make sure it includes focus on disruption in cafeteria

Safety

65

Trevor Test Middle School

Hypothesis - cafeteria overcrowded; 6th graders with insufficient instruction in cafeteria expectations; attention from adults and peers rewarding disruption

Choose the least number of solution(s)that will have the biggest impact on decreasing the problem.

Implementing the solution requires action Here are solution actions, added to the P-S

Action Plan…

66

Fidelity◦ Did we do what we said we would do?

Make it simple Student Outcomes

◦ Did our intervention produce the outcomes we were expecting Use the right data to answer the questions you are

asking

Choose the solutions that will create an environment that makes the problem irrelevant, inefficient, and ineffective.Choose least amount of work that will have the

biggest impact on decreasing the problem. Implementing the solution requires action and

time lines Problems need goals so that we can measure

progress and know when to move on.

Use weekly 1-5 survey of playground monitors to assess implementation of plan

68

Are we doing the plan?

1 ….. 2 …..3 ….. 4 ….. 5No Yes

CollectCollect and Useand Use

DataData

Develop Hypothesis

Discuss andSelect

SolutionsDevelop andImplementAction Plan

Evaluate andRevise

Action Plan

Problem Solving Meeting Foundations

Team Initiated Problem Solving (TIPS) Model

Identify Problems

69