Download ppt - Team-Initiated Problem Solving (TIPS) Overview Presented by: Dale Cusumano, Ph.D., NCSP Research Associate…

Team-Initiated Problem Solving (TIPS)

OverviewPresented by:

Dale Cusumano, Ph.D., NCSPResearch Associate

University of North Carolina at Charlotte

Presented to: Seventh Annual Response to Intervention Best Practices Institute

October 15, 2013

90,000 public schools in the United States

Each school has 1+ teams to address

challenges and build solutions

Each team meets at least monthly

On average there are 5 people on each

team

810,000 hours of meetings

4,050,000 hours of personal

time annually

Problem

Solution

Organizing for an Effective Problem Solving Conversation

Use Data

Bob Algozzine, Kate Algozzine, and Dale CusumanoUNC at Charlotte

Rob Horner, Anne Todd, and Steve NewtonUniversity of Oregon

Data+

Outcomes

To what extent do teams follow problem solving steps and include

critical components?• Include a behavioral definition of

target behavior• Have a direct measure of the target

behavior prior to intervention• Include a step-by-step intervention

plan• Graph intervention results• Compare pre-intervention and post-

intervention performance• Develop a hypothesized reason for

the problem• Gather evidence that the

intervention was implemented as designed

Behavioral definition of target behavior, baseline

data, step-by-step plan, or graphic or numeric data

comparison from pre to post intervention were not

included (Flugum & Reschly, 1994)

Lowest implementation for hypotheses for problem,

treatment fidelity, and pre and post data comparison(Telzow, McNamara, & Hollinger,

2000)

Team members rated implementation as higher than observers with observers rating identifying antecedents and consequences for

behavior, identifying data to monitor progress, scheduling a follow up meeting as unmet

(Telzow, McNamara, & Hollinger, 2000)

People aren’t tired from solving problems – they are tired from solving the same problem over and over.

6Newton, J. S., Todd, A. W., Algozzine, K., Horner, R. H., & Algozzine, B. (2009). The Team Initiated Problem Solving (TIPS) Training Manual. Educational and Community Supports, University of Oregon, unpublished training manual.

Efficient

Effective

Logical

Thorough

Team-Initiated Problem Solving II(TIPS II)

Decisions are more likely to be effective and

efficient when they are based on data

Quality of decision-making depends most on the first step (defining the

problem to be solved)

Data help us ask the right question…they do not

provide the answers: Use data to identify problems, refine the problems, and define the questions that

lead to solutions

Data help place the “problem” in the context rather than in students

Main Ideas

Build “decision systems” not “data systems”• Problem solving is driven by

decision systems, which are fueled by data

Use data in “decision layers”

(a) Is there a problem? (b) Localize the problem

(location, problem behavior, students, time of day), and

(c) Get specific

Do not drown in data

Be efficient

It is OK to be doing well!

More Main Ideas

What do we need?

• A clear model with steps for problem solving

• Access to the right information at the right time in the right format

• A formal process that a group of people can use to build and implement solutions.

10Newton, J. S., Todd, A. W., Algozzine, K., Horner, R. H., & Algozzine, B. (2009). The Team Initiated Problem Solving (TIPS) Training Manual. Educational and Community Supports,

University of Oregon, unpublished training manual.

Eight Keys to Effective Meetings

1. Organization (team roles, meeting process, agenda)2. Data (right information at right time in right format)3. Separate (a) Review of On-going Problem Solving (b)

Administrative Logistics and (c) New Problem Solving4. Define Problems with Precision5. Build Comprehensive Solutions that “fit”6. Add “Action Plans” for all solutions7. Review Fidelity and Impact regularly8. Adapt Solutions in response to data.

Implement Solution with High Integrity

Identify Goal for Change

Identify Problemwith

Precision

Monitor Impactof Solution and

Compare against Goal

Make SummativeEvaluationDecision

MeetingFoundations

Team-Initiated Problem Solving (TIPS II) Model

IdentifySolution and

CreateImplementation

Plan with Contextual Fit

Collect and Use

Data

TIPS ModelTIPS Training

• One full day team training• Two coached meetings

Team Meeting• Use of electronic meeting minute system• Formal roles (facilitator, recorder, data analyst)• Specific expectations (before meeting, during meeting, after meeting)• Access and use of data• Projected meeting minutes

Research tool to measure effectiveness of TIPS Training• DORA (decision, observation, recording and analysis)• Measures “Meeting Foundations” & “Thoroughness of Problem Solving”

Newton, J. S., Todd, A. W., Algozzine, K., Horner, R. H., & Algozzine, B. (2009). The Team Initiated Problem Solving (TIPS) Training Manual. Educational and Community Supports, University of Oregon, unpublished manual.

9

TIPS I Study: Todd et al., 2011

OctNov Dec Ja

nFeb Mar Apr

May0%

20%

40%

60%

80%

100%

OctNov Dec Ja

nFeb Mar Apr

May0%

20%

40%

60%

80%

100%

OctNov Dec Ja

nFeb Mar Apr

May0%

20%

40%

60%

80%

100%

OctNov Dec Ja

nFeb Mar Apr

May0%

10%20%30%40%50%60%70%80%90%

100%

School A

School B

School C

School D

Baseline Coaching TIPS%

DOR

A Fo

unda

tions

Sco

re

Solid = SW PBIS meetings using SWISOpen = progress monitoring meeting using DIBELS

Journal of Applied School Psychology

Pre Post/Exp0

10

20

30

40

50

60

70

80Exp Control

N = 17 N = 17TIPS Control TIPS Control

DORA

Tho

roug

hnes

s of D

ecisi

on M

akin

g Sc

ore

(Sim

ple)

Newton et al., (2012):Effects of TIPS Training on Team Decision-making

Pre TIPS Training Post-TIPS Training

Problem-Solving Meeting

Foundations

Structure of meetings lays foundation for efficiency & effectiveness


Identify Goal for Change


Precision




MeetingFoundations




Plan with Contextual Fit

Collect and Use

Data

PREDICTABILITYPARTICIPATION

ACCOUNTABILITYCOMMUNICATION


ACCOUNTABILITYCOMMUNICATIONRoles & Responsibilities

The process a

team uses is

important

Roles

FacilitatorRecorder

Data analystActive Member

Organization

Agenda, old business, new business, action plan for

decisionsWhat happens BEFORE a

meetingWhat happens DURING a

meetingWhat happens AFTER a

meeting

Foundations of an Effective and Efficient Meeting

Who is Responsible?Action Person Responsible

Reserve Room

Recruit items for Agenda

Review data prior to the meeting

Reserve projector and computer for meetingKeep discussion focused

Record Topics and Decisions on agenda/minutesEnsure that problems are defined with precisionEnsure that solutions have action plans

Provide “drill down” data during discussionEnd on time

Prepare minutes and send to all members



Start and end times

Agenda is used

TIPS Meeting Minutes and Problem-Solving Action Plan FormToday’s Meeting: Date, time, location: Facilitator: Minute Taker: Data Analyst:

Next Meeting: Date, time, location: Facilitator: Minute Taker: Data Analyst: Team Members (bold are present today________________________________________________________________

Information for Team, or Issue for Team to Address

Discussion/Decision/Task (if applicable) Who? By When?

Administrative/General Information and Issues

Implementation and Evaluation

Precise Problem Statement, based on review of data

(What, When, Where, Who, Why)

Solution Actions (e.g., Prevent, Teach, Prompt, Reward, Correction, Extinction,

Safety)Who? By When?

Goal, Timeline, Decision Rule, & Updates

Problem-Solving Action Plan

Agenda for NEXT Meeting

1. 2.3.

Implementation and EvaluationPrecise Problem Statement, based on

review of data(What, When, Where, Who, Why)

Solution Actions (Prevent, Teach, Prompt, Reward,

Correction, Extinction, Adaptations, Safety)

Who? By When? Goal with Timeline

Fidelity of Imp measure

Effectiveness of Solution/Plan

Not started Partially Imp Imp Fidelity Done

Goal Met Better Same Worse

Agenda for Today:1. 3. 5.2. 4. 6.Previously Defined Problems/Solutions (Update)

TIPS Meeting Minutes and Problem-Solving Action Plan FormToday’s Meeting: Date, time, location: Facilitator: Minute Taker: Data Analyst:

Next Meeting: Date, time, location: Facilitator: Minute Taker: Data Analyst: Team Members (bold are present today________________________________________________________________

Information for Team, or Issue for Team to Address

Discussion/Decision/Task (if applicable) Who? By When?

Administrative/General Information and Issues

Implementation and Evaluation

Precise Problem Statement, based on review of data

(What, When, Where, Who, Why)

Solution Actions (e.g., Prevent, Teach, Prompt, Reward, Correction, Extinction,

Safety)Who? By When?

Goal, Timeline, Decision Rule, & Updates

Problem-Solving Action Plan

Agenda for NEXT Meeting

1. 2. ‘3.

Implementation and EvaluationPrecise Problem Statement, based on

review of data(What, When, Where, Who, Why)

Solution Actions (Prevent, Teach, Prompt, Reward,

Correction, Extinction, Adaptations, Safety)

Who?

By When? Goal with Timeline

Fidelity of Imp measure

Effectiveness of Solution/Plan

Not started Partially Imp Imp Fidelity Done

Goal Met Better Same Worse

Agenda for Today:1. 3. 5.2. 4. 6.

Previously Defined Problems/Solutions (Update)

Where in the Form would you place:

1. Planning for next PTA meeting?

2. There have been five fights on the playground in the past 3 weeks.

3. Update on CICO implementation

4. Increasing gang recruitment as an agenda topic for today.

5. Next meeting report on lunch-room status.



Start and end times

Agenda is used

Meeting schedule

PARTICIPATION

Engaged in Topic

Decision Makers

COMMUNICATIONNorms for the meeting

Access to Meeting Minutes

ACCOUNTABILITY

Prepared for meetings

Are solutions working?

Documentation of decisions

Differences in lives of students


ACCOUNTABILITYCOMMUNICATION


IdentifyGoal for Change


Precision




MeetingFoundations


Identify Solution and


Plan withContextual Fit

Collect and Use

Data

Problem = A difference between what is expected and what is observed that a team feels is worthy of addressing.

Start with Primary Problem Statements

Look at the Big Picture. Then use data to refine the problem to a Precise Problem

Statement.

Office discipline referrals for 3rd graders are above national medians for schools our size.

Morphing Data into Useful Information

Develop Primary Problem Statements• Look first at your patterns (tell the story)

• Level, Trend• Peaks• Match data to current perceptions

• Compare your data• With national median or percentiles• With last year• With what your faculty/students/ families want

Start with Primary Problem Statements

Look at the Big Picture. Then use data to refine the problem to a Precise Problem

Statement.

Move to Precise Problem Statements

Office discipline referrals for 3rd graders are above national medians for schools our size.

Referrals for defiance among third grade students from 11:30-12:30 in the cafeteria are increasing over time. It is believed that this is happening because students want

to avoid silent reading that happens after lunch.

What When

Who

Why

Where

What

WhoWhy

Precision Components for Behavior Problem Statements

Precision Components for Academic Problem Statements

Examples: Primary to Precise

Gang-like behavior is increasing.

Our fourth graders cannot comprehend when reading!

• Bullying (verbal and physical aggression) on the playground is increasing during “first recess,” is being done mostly by four 4th grade boys, and seems to be maintained by social praise from the bystander peer group.

• Forty-seven percent of 4th grade students did not meet reading comprehension targets on AIMSweb Maze benchmark assessments when 80% of students at a grade level should meet this target. It appears that weak vocabulary skills are lowering students’ comprehension skills.

Examples: Primary to Precise

Carly is having reading difficulties.

• Carly is reading 20 words correctly per minute (goal is 60), skips or guesses at words she doesn’t know, mostly during language arts.

• Carly can not decode and struggles to read words containing R controlled vowels, digraphs, & long vowels.

• Jack’s math CBM scores fell at the 10th percentile as compared to national norms. Jack is not fluent in his knowledge of basic math facts and often does not attend to addition or subtraction signs on written math problems.

Jack cannot add or subtract.

Precise or Primary Statement?

Minor disrespect and disruption are increasing over time, and are most likely during the last 15 minutes of our block periods when students are engaged in independent seat work. This pattern is most common in 7th and 8th grades, involves many students, and appears to be maintained peer attention.

Precise or Primary Statement?

James is instructional in reading content that falls two grades below his current placement. His reading fluency scores fall below the 25th %ile and his rate of improvement is predicting that he will not meet end of the year goals for his grade. He has difficulty decoding unfamiliar words and does not track accurately from left to right when reading.

The boys in third grade are performing well below the 3rd grade girls in reading.

Three 5th grade students are having great difficulty expressing their thoughts in writing. Their fine motor skills are well developed but writing test scores are well below target. Since they can verbally share well-constructed and grammatically correct stories, it is thought that their spelling skills are hindering their writing skills with average weekly spelling test scores of C’s and D’s obtained.

Next, We Need Goals…

We need measurable and realistic SMART goals.

• SMART Goals• Specific • Measurable • Achievable• Relevant• Timely

Precision Problem StatementMinor disrespect and disruption are increasing over time, and are most likely during the last 15 minutes of our block periods when students are engaged in independent seat work. This pattern is most common in 7th and 8th grades, involves many students, and appears to be maintained by peer attention.

Precision GoalThe rate of disrespect and disruption among 7th and 8th graders (as measured by SWIS) will decrease to 2 per week by the 15th of next month (~.40 per day per month) and decrease to one or fewer per week (<.20 per day per month)by the end of the following month to be maintained across the remainder of the school year.

Examples

Trend

Our average Major and Minor ODRs per school day per month are higher than national median for a school of our enrollment size for all months except June. We have peaks in frequency of problems in Nov, Dec, Jan, and March with an increasing trend from September to March.

What When

Who

Why

Where

What

WhoWhy

Precision Components for Behavior Problem Statements

Precision Components for Academic Problem Statements

SWIS Big 4 for October 1, 2011 through December 31, 2011

Defiance


ClassroomPlaygroun

d

11:30-12:15

Avoid Work


Possible Problem Precision Statement

Average referrals per day per month for All Referrals & Minors are above the 75th percentile for 5 of 6 months with an increasing trend. Defiance (what) in classrooms and on the playground (where) for a group of 15 fifth graders (who) between 11:30 and12:15 (when) is the most frequently reported behavior problem and occurs as a means of avoiding work (why).

How does this work for academics?

80% meeting

expectation

15% need more

5% need the most

Is there a problem? Academic Reference Points

50th Percentil

e on national norms

Low Risk indicator

DIBELS Universal Screening

Primary Problem StatementOur DIBELS Distribution summary shows that 49% of our kindergarten students at Adams Elementary fall in the strategic and intensive range. We have over 50% of our students requiring strategic and intensive supports for ISF, LNF.

Primary Goal At least 80% of our Kinders will be in Benchmark range at Winter Universal Screening Time

1 2 3 4 50%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of Students Meeting Expectation in Reading Comprehension (AIMSweb Maze)

FallWinterSpringGoal

Grade Level

Only 62% of 4th graders and 65% of 5th graders are meeting expectations in reading comprehension at the winter

benchmark; although a slight improvement from fall to winter is noted, it is below the goal of having 80% of students meet

expectation.

What?

For today, we will focus on 4th

graders

Four 4th graders have the lowest scores on measures of reading comprehension. Other

students are below expectation but their scores are within + 1 correct responses (CR) of the target (14 CR). Scores for these four students fall in the

At Risk range, which is below the 10th percentile on national norms.

Who?

Low Risk > 14 CR

All close

to goal of 14 CR

Why?

Why?

ICEL

Skill Developm

ent

Functional

Drivers

Three 4th grade students are performing well below expectation comprehending written text that is presented at grade level (scores on Maze measures fall below the 10th percentile on national norms). These students also have weak reading

fluency skills that fall well below expectation, which is thought to be the reason for their low reading comprehension skills. A

fourth student (Sally) also is performing well below expectation in reading comprehension, but her reading fluency skills are in expected ranges. Weak vocabulary skills may be lowering her

comprehension skills.

Why?

Let us look at reading fluency as a possible reason for these

four students’ low reading comprehension.

Harris Word list for Sally = 2nd grade




Precision




MeetingFoundations

Team-Initiated Problem Solving (TIPS-II) Model

Identify Solution and



Collect and Use

Data

Using Precision Problem Statements to Build Solutions, Action & Evaluation

PlansSolutions

Prevention: How can we avoid the problem context? Teaching: How can we define, teach, and monitor what we want? Recognition: How can we build in systematic reward for desired behavior?Extinction: How can we prevent problem behavior from being rewarded?Consequences: What are efficient, consistent consequences for problem behavior?

Action PlanWho will do each task & when will it be completed?

Evaluation

How will we collect and what data will we use to evaluate: • Implementat

ion fidelity?• Impact on

student outcomes?

Prevent “Trigger”

Define & Teach

Reward/Reinforce

Withhold Reward

Corrective consequence

Other

Safety62

Example for Precise Behavior Problem Statement6th and 7th graders are engaging in inappropriate language, harassment, disrespect and aggression in two classrooms at 9:45 and 12:45 to get peer and adult attention and to escape the work. There are 175 total instances of problem behavior in 6th and 7th grade classrooms, for 2010-11 school year.

Re-review 6th and 7th graders classroom expectations/Respecting others, daily

Focus on Respect, Re-teach stop-walk routine

Set up “Daily Double”: Class period without problem behavior occurrence receives extra 2 mins, at end of period to talk.Provide specific feedback for using stop-walk routineEnsure staff use routine for responding to a report when student comes to talk.Use School Defined Process

Prevent “Trigger”

Define & Teach

Reward/Reinforce

Withhold Reward

Corrective consequence

Other

Safety63

Example for Precise Academic Problem StatementFour 4th grade students are performing below expectation in reading comprehension. For three of these students, their performance is being impacted by low reading fluency skills. For one student, weak vocabulary skills are keeping her from comprehending written text.

Provide instructional level materials for independent reading activities.

Participation in a Repeated Readings intervention group.Self monitoring of reading fluency skills with gains tied to rewards.Self monitoring of known and unknown vocabulary words and accuracy rates.

Participation in a Drill Sandwich vocabulary practice.

Every solution/task needs an action plan• Who will do the task?• When will the task be completed?

Action Planning




Precision




MeetingFoundations





Collect and Use

Data

Evaluation Planning

Evaluation Plan for monitoring fidelity of implementation AND impact on student behavior

• Evaluate fidelity of implementation compared to the goal

• Define how, when, criteria• Evaluate effect of solutions on student behavior

(impact) as compared to the goal• Define data to be used, how often and criteria

• Data analyst with data summaries and data access

Fidelity of ImplementationMeasure the degree in which the intervention was implemented as defined/expected

• Use percent/absolute value/ rate/scale as metric• Strive for 80% fidelity of implementation as measured

weekly (bi-weekly) on scale of 1-5Make easy for staff to record data

• Fidelity Check Board: X on number line• Fist of five• Fidelity check basket• Direct observation

Are we implementing the plan?

1 2 3 4 5 No Yes

Other Data for Monitoring Fidelity of Implementation

Fidelity Check RoutineWe do what we say we will do and we do it

with 80% fidelityEstablish a fidelity check routine that relates to Implementation

A 1-5 scale is used for all questions, with up to three questions per weekAt staff meeting, use fist of five while asking questionsIn staff room, create number line poster with questions

Did you provide instructional level reading to students?

1 2 3 4 5No Yes Did you acknowledge 5 students, not in

your classroom, daily? 1 2 3 4

5No

Yes

Evaluation PlanningEvery problem needs to be monitored and evaluated

• Fidelity of Implementation• Effectiveness of Implementation

Impact of SolutionFor Behavior Example

Did we make a difference?

Impact of SolutionFor Academic Example

Evaluate the Problem: Did we make a difference?

1 2 3 4 50%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percentage of Students Meeting Expectation in Reading Comprehension (AIMSweb Maze)

FallWinterSpringGoal

Grade Level

✔

Evaluate the Problem: Did we make a difference?

✔ ✔

✔


PrecisionMake Summative

EvaluationDecision




Precision




MeetingFoundations





Collect and Use

Data

TIPS Team

Training

Readiness

Team Membershi

p

Team Data Access

Team Commitme

nt

Coaching Commitme

nt

Team Membership1. Representation needed for meeting their purpose 2. Inclusion and presence of administrator with authority to

make decisions

Team Data Access3. Data available for problem solving & decision-making

before and during the meeting4. Consistent process & procedures for documenting &

entering data exists5. Team member is fluent in generating basic and custom

reports from data set(s) being used

TIPS Team Training Readiness10 readiness guidelines

TIPS Team Training Readiness10 readiness guidelinesTeam Commitment

6. Implementation of TIPS Meeting Foundations7. Team & coach attendance at TIPS Team Training

• ** one full day or two half day team trainings8. Application of the TIPS model through the school year &

annual TIPS boosters

Coaching Commitment9. Team has access to a coach who knows the TIPS system

& who is available before, during, & after meetings to support fidelity of implementation

10. Commitment to attend team training and provide coaching before, during and after the meetings

Where are you in the implementation process? Adapted from Fixsen & Blase, 2005

• We think we know what we need so we are planning to move forward (evidence-based)

Exploration & Adoption

• Let’s make sure we’re ready to implement (capacity infrastructure)

Installation

• Let’s give it a try & evaluate (demonstration)Initial Implementation

• That worked, let’s do it for real (investment)Full Implementation

• Let’s make it our way of doing business (institutionalized use)Sustainability & Continuous Regeneration

Putting “it” in place.

Should we do it?

Now make it better

Initiative is way of work

TIPS II: Contacts and ReferencesUniversity of

OregonRob Horner

[email protected]

Anne [email protected]

Steve [email protected]

UNC at CharlotteBob [email protected]

Kate [email protected]

Dale [email protected]