Team Initiated Problem Solving

Team-Initiated Problem Solving (TIPS)

Team Initiated Problem SolvingRob Horner, Steve Newton, & Anne Todd,University of Oregon

Bob Algozzine & Kate Algozzine,University of North Carolina at Charlotte

Coaches Conference Feb 3, 2010Oregon State University

[email protected]@uoregon.edu

Anne ToddThis session provides a model and some strategies for teams to use as they use SWIS data for problem-solving and decision making. The TIPS model (team initiated problem solving) is a model that has been built as a result of a federal IES grant charged with figuring out how to teach and coach teams when using SWIS data. The project is a collaboration between UO (Horner, Newton, Todd) and UNC at Charlotte (Algozzines)

Something to remember: The word data is plural. Say data are1Todays GoalsCoaches are able to:Prompt & support facilitator, minute taker and data analyst to prepare for meetingsMeeting Foundations ChecklistPrompt the use of the TIPS model during meetingsData-based Decision-making rulesHelp teams stay focused during meetingsElectronic Meeting Minute format

ClarificationCoaches are NOT expected to be TrainersTrainers deliver TIPS team training & help Coaches anticipate errors while guiding them through the possible solutions & adaptationsSome participants will be coaches, some will be trainers, and some will be both. Today we want coaches to understand how the Meeting Foundations Checklist and Meeting Minute form are used to support team functioning and sustainability.Be able to use the TIPS problem solving model to simulated SWIS summary dataPlus the three goals above2ContextEvery school has teamsTeams are being expected to do problem solvingSelect curriculaGet training and implement new ideas/programsProvide efficient leadershipCommunities of PracticeTeams need to report data to administration, district, stateTeams NEED data to do good problem solving.Most teams are not skilled at running problem solving meetings and using data for decision-making.

Assumption: Coaching is CriticalTeams will need more than a manual or brief training to become skilled at use of data for efficient problem solving

Coaching will be a key element to successful use of good problem solving procedures.What do we need?A clear model with steps for problem solvingAccess to the right information at the right time in the right formatA formal process that a group of people can use to build and implement solutions.Collect and UseData

Review Status and Identify ProblemsDevelop andRefineHypothesesDiscuss andSelectSolutionsDevelop andImplementAction Plan

Evaluate andReviseAction Plan

Problem Solving Meeting FoundationsTeam Initiated Problem Solving (TIPS) ModelThe outer circle ( Problem Solving Meeting Foundations) refer to the process and procedures and team uses to function. Roles are determined and defined, meetings are scheduled for the year, electronic equipment and internet access are available, and an agenda is established.The inner circles and arrows define a problem-solving model designed to improve the decision-making and problem solving of PBIS Team. This model is called TIPS, which stands for Team-Initiated Problem Solving. The model uses data for during problem solving and decision making, during meetings. We will also apply a metric for determining if rates of problem behavior at a school or below, at, or above the national average to determine if there is a problem or not. TIPS teaches teams to use their SWIS data to define precision problem statements. Once the problem statement is precise, a variety of solutions are discussed based on prevention, teaching, reward, correction and extinction and teams will determine which solution(s) they want to implement. The TIPS model then moves the team to action planning ,evaluation and measurement determination.

We are finding it more difficult to use these skills in the REAL context, the purpose of this session is to build the skills and build the fluency of using those skills.

6TIPS ModelTIPS TrainingOne full day team trainingTwo coached meetings

Team MeetingUse of electronic meeting minute systemFormal roles (facilitator, recorder, data analyst)Specific expectations (before meeting, during meeting, after meeting)Access and use of dataProjected meeting minutes

Research tool to measure effectiveness of TIPS TrainingDORA (decision, observation, recording and analysis)Measures Meeting Foundations & Thoroughness of Problem SolvingTIPS Training is a package across time starting with one full day of team training (the binder has all the materials, the thumb drive has the files) Followed by two coached meetings. Coaching includes:Prompting & supporting the facilitator, minute taker and data analyst to prepare for meetingsPrompting the use of the TIPS model during meetingsHelping the team stay focused during meetings

Points to make: tough to build fluency if meeting once a month. The frequency is too lean.7

This is the observation tool used to measure Meeting Foundations and Thoroughness of Problem-Solving for the research studies.

DORA does not stand for Dora the Explorer!8Evidence of EffectivenessEvaluation Study (2007-08)Newton et al.,Single-case Study (2008-09)Todd et al.,Group Design Study (2009-10)TIPS Study: Todd et al., 2009School ASchool BSchool CSchool DMeeting Foundations ScoreBaseline Coaching TIPS% DORA Foundations Score10TIPS Study: Todd, et al, 2009. School ASchool DSchool CBaseline Coaching TIPSThoroughness of decision-making% DORA Thoroughness ScoreSYSTEMSPRACTICES DATASupportingStaff & Student Behavior and Decision MakingBuilding Capacity and SustainabilityOUTCOMESFor Social Competence,Academic Achievement, and SafetySWISElectronicMeeting Minutes Form*Meeting time*Support*Report to FacultyBuilding Capacity and Sustainability using the TIPS model to:1. Conduct effective, efficient meetings when using SWIS data for problem solving and decision making.2. Implement and evaluate solutions that result in positive effects on student achievement, social behavior and safety.a. The Information system is the use of the School Wide Information System. The data are current, accurate, believableb. The Practices include the use of the meeting minute form and the problem solving processc. The System is the implementation of Meeting Foundations, the TIPS model, and the documentation of decisions, action plans, and evaluation plans12Improving Decision-Making via Problem SolvingProblemProblemSolvingSolutionInformation/ DataAction Planning & Evaluation13Steps in the problem solving model.Problem-Solving Meeting FoundationsStructure of meetings lays foundation for efficiency & effectiveness

Using Meeting MinutesDocumentation of Logistics of meeting (date, time, location, roles)Agenda items for todays meeting ( and next meeting)Discussion items, decisions made, tasks and timelines assignedProblem statements, solutions/decisions/tasks, people assigned to implement with timelines assigned, and an evaluation plan to determine the effect on student behavior

Reviewing Meeting minutes An effective strategy for getting a snapshot of what happened at the previous meeting and what needs to be reviewed during the upcoming meetingWhat was the issue/problem?, What were we going to do?, Who was going to do it and by When?, and How are we measuring progress toward the goal?

Visual tracking of focus topics during and after meetingsPrevents side conversationsPrevents repetition Encourages completion of tasks Organizing for an effective problem solving conversationProblemSolutionOut of TimeUse Data A key to collective problem solving is to provide a visual context that allows everyone to follow and contributeBuilding a system that is NOT person dependentWe want to walk into a meeting having no previous history, review the previous meeting minutes be able to fit into any role neededFacilitatorMinute takerData analystActive team member

ExamplePBIS Team Meeting Minutes and Problem-Solving Action Plan FormTodays Meeting: Date, time, location: Facilitator: Minute Taker:Data Analyst: Next Meeting:Date, time, location: Facilitator: Minute Taker:Data Analyst: Team Members (bold are present today)Todays Agenda Items Next Meeting Agenda Items01. 02. 03. Information for Team, or Issue for Team to AddressDiscussion/Decision/Task (if applicable)Who?By When? Administrative/General Information and IssuesImplementation and EvaluationPrecise Problem Statement, based on review of data(What, When, Where, Who, Why)Solution Actions (e.g., Prevent, Teach, Prompt, Reward, Correction, Extinction, Safety)Who?By When?Goal, Timeline, Decision Rule, & UpdatesProblem-Solving Action PlanOur RatingYesSo-SoNo1. Was todays meeting a good use of our time?2. In general, did we do a good job of tracking whether were completing the tasks we agreed on at previous meetings?3. In general, have we done a good job of actually completing the tasks we agreed on at previous meetings?4. In general, are the completed tasks having the desired effects on student behavior? Evaluation of Team Meeting (Mark your ratings with an X)This slide is animated to teach the different parts of the meeting minute form each click adds the next sectionMost schools have the title at the top and write/type as the meeting progresses

Make a point that we dont need to document everything that happened (i.e., NM rolled her eyes KJ entered the room, SW continued to repeat the same issue, we took at 5 minute bathroom break)18

A completed exampleIF a person knows how to use the meeting minute form, the person should be able to pick these minutes up from Jan 7, 2010 and be able to organize previous items to update and facilitate creation of the Feb 3, 2010 agenda19Important Structural ComponentsRegular meetings & regular attendanceThe right peopleThe right rolesFacilitatorMinute TakerData AnalystActive Team MembersThe right information for problem solving & decision makingAccomplishments Products of successful meetingMeeting Minutes (record of decisions & tasks concerning administrative/general issues)Problem-Solving Action Plan (record of decisions & tasks concerning problems identified by team)

202021

22

Before the MeetingRoom reservedNew items solicited for agendaAgenda produced Team member roles determinedData reviewed by Data Analyst before the meeting; Analyst ready to lead team through discussion of (a) possible new problems and (b) effects of in-process solutions on old problemsComputer reserved; access to SWIS online database assuredLCD projector reserved & set up to project data (or team has some other strategy for ensuring team members can review data at meeting)Team members have individual TIPS Notebooks to bring to meeting

(Well review the (a) before-meeting, (b) during-meeting, and (c) after-meetings responsibilities of individual team members later in this workshop)

23At Close of and After MeetingMeeting Minutes and Problem-Solving Action Plan completedCopy of Meeting Minutes & Problem-Solving Action Plan distributed to each member within 24 hrs.

2425

ActivityComplete the Foundations ChecklistUse the PBIS team you know best

How would you use the Foundations Checklist to help a school team that was preparing to adopt TIPS procedures?Collect and UseData

Develop Hypothesis

Discuss andSelectSolutions

Develop andImplementAction Plan

Evaluate andReviseAction Plan

Problem Solving Meeting FoundationsTeam Initiated Problem Solving (TIPS) ModelIdentify Problems

27

Ask participants about knowledge and experience of SWIS27SWISTM(School-Wide Information System)DefinedSWISTM is a web-based information system for gathering, entering, summarizing, reporting and using office discipline referral informationPurposeA progress monitoring tool for improving the ability of school personnel to develop safe and effective learning environments28Three Key Elements of SWISTMData Collection SystemCoherent system for assigning referralsProb. behavior definitions, referral form, rules for referral Allocation of FTE to enter data, build reports.Computer ApplicationWeb-based, continuously available, secureDecision-making Use of data School-wideIndividual Student29Features of SWISTMOnly reports discipline dataMajor office referralsMinor discipline offencesHighly efficient (30 sec per referral)Local control Formatted for decision-making (pictures)Information is available continuouslyConfidential, secureCan be combined with district data base30Teams need access to the data AND a person or two who know how to access and navigate through the data. Ideally, this should happen during a meeting when team members are asking questions about the data. Since SWIS is web-based, it is always available through an internet browser. We need to help people learn how to use the data in an interactive format during a meeting, when the data are most needed. moving the group through the simulations helps to build fluency of the skills needed.

How SWISTM works

Data EntrySchool Address and ContactEnrollment/Ethnicity/Days per monthStaff InformationStudent InformationReferralsReportingAverage Referrals per Day per monthReferrals by Problem BehaviorReferrals by LocationReferrals by TimeReferrals by StudentOther ReportsTools32

Total Office Discipline ReferralsTotal Office Discipline Referrals as of January 10Lets talk about accuracy of the data again. When you begin to use the data and draw comparisons, the data need to be comparable. Look at the data above. First, as a data analyst, you look and see, wow.. Things are getting better, the graph is going down. Then you do what you are supposed to do first, and look at the label on the Y axis. This label says total office discipline referrals. It is great to compare the total ODRs, but now look at the X-axis. There are a different number of days in each month and the number of schools day in each much has a wide range (Dec may have 10 school days, January may have 19 school days). These months, the way they are arrayed here, are not comparable and this data should not be used! If you arent using SWIS, do the math to get average referrals per day per month by using the total referrals and the total days each month. If you are using SWIS, do not fear.. (next slide)34

SWIS does that calculation for you. look at the Y-axis label now. Average referrals per day per month allow us to compare months. Now look at the trend.. we are going to have a wild spring term if we dont do anything differently!). This is the same set of data on the previous slide and look at what the pattern of data does for the problem solving process.Accurate data and data that are formatted for purposes of making decisions is critical.I like to make this a bit dramatic by going back and forth between this slide and previous, telling them they are the team and they are reviewing this data 35What behaviors are problematic?

What behaviors are problematic?

What behaviors are problematic?

Where are the problems occurring?

Where are the problems occurring?When are the problems occurring?

When are the problems occurring?

Who is contributing to the problem?

Organizing SWIS Data for Decision-makingUniversal Screening ToolProportion of students with 0-1 Office Discipline Referrals (ODRs)2-5 ODRs6+ ODRsProgress Monitoring Tool Compare data across timePrevent previous problem patterns Define Problems with precision that lead to solvable problemsOK. Building precision problem statements is a skill that is needed for using the data. organizing and interpreting the data requires another set of skills.Slides 25-47 provide a sequence of slides to illustrate different precision statements based on different pictures of the same type of data.44~80% of Students~15% ~5% 0-1 office discipline referral6+ office discipline referrals

2-5 office discipline referrals

Using office discipline referrals as a metric for universal screening of student social behaviorUsing the Referrals by Student report as a Universal Screening Tool

Cumulative Mean ODRsCumulative Mean ODRs Per Month for 325+ Elementary Schools 08-09Jennifer Frank, Kent McIntosh, Seth MayUsing ODRs to Identify ProblemsBuild a picture for the pattern of office referrals in your school.

Compare the picture with a national average

Compare the picture with previous years

Compare the picture with social standards of faculty, families, students.Goal

Identify problems empiricallyIdentify problems earlyIdentify problems in a manner that leads to problem solving not just whining Using ODRs to Identify ProblemsBuild a picture for the pattern of office referrals in your school.

Compare the picture with a national average

Compare the picture with previous years

Compare the picture with social standards of faculty, families, students.SWIS summary 2008-2009 (Majors Only)3,410 schools; 1,737,432 students; 1,500,770 ODRsGrade RangeNumber of SchoolsAvg. Enrollment per schoolNational Avg. for Major ODRs per 100 students, per school dayK-62,162450.34 = about 1 Major ODR every 3 school days, or about 34 every 100 days6-9602

657.85 = a little less than 1 Major ODR per school day, or about 85 every 100 days9-122158871.27 = more than 1 Major ODR per school day, or about 127 every 100 daysK-(8-12)4314081.06 = about 1 Major ODR per school day, or about 106 every 100 days50How to use these numbers:Your enrollment (e.g. 400 students or 225 students)Divide by 100 (e.g. 400/100 = 4; 225/100 = 2.25Multiply by the National Average to get ODR per day(4 X .34 = 1.36 2.25 X .34 = .76)SWIS summary 2008-2009 (Majors Only)3,410 schools; 1,737,432 students; 1,500,770 ODRsGrade RangeNumber of SchoolsAvg. Enrollment per schoolNational Avg. for Major ODRs per 100 students, per school dayK-62,162450.34 = about 1 Major ODR every 3 school days, or about 34 every 100 days6-9602

657.85 = a little less than 1 Major ODR per school day, or about 85 every 100 days9-122158871.27 = more than 1 Major ODR per school day, or about 127 every 100 daysK-(8-12)4314081.06 = about 1 Major ODR per school day, or about 106 every 100 days51

Elementary School with 150 StudentsCompare with National Average150 / 100 = 1.50 1.50 X .34 = .51

Elementary School with 450 StudentsCompare with National Average450 / 100 = 4.50 4.50 X .34 = 1.53

Application Activity: Absolute ValueIs there a Problem?

Middle School of 625 students?Compare with national average:625/100 = 6.25 6.25 X .85 = 5.31 Office Discipline Referrals per School Day54

High School of 1800 studentsHigh School: Compare with National Average1800 / 100 = 18 18 X 1.27= 22.86Build the routine when reviewing these slides:How many students?How many hundreds?What is the per 100 rate for your school?How are you doing?Is there a trend, are there peaks, patterns?; what can we anticipate?

55

High School of 450 studentsHigh School: Compare with National Average450 / 100 = 4.5 4.5 X 1.27= 5.17Build the routine when reviewing these slides:How many students?How many hundreds?What is the per 100 rate for your school?How are you doing?Is there a trend, are there peaks, patterns?; what can we anticipate?

56

Middle School of 700 studentsSlides 31-33 work as a set. This slide is a precursor to the next slide to show how to start with this years data and then use it to compare to last year (slide 32)

Build the routine when reviewing these slides:How many students?How many hundreds?What is the per 100 rate for your school?How are you doing?Is there a trend, are there peaks, patterns?; what can we anticipate?

57

Build the routine when reviewing these slides:How many students?How many hundreds?What is the per 100 rate for your school?How are you doing?Is there a trend, are there peaks, patterns?; what can we anticipate?What happened last year that we dont want to repeat?58

Final slide of this set

Build the routine when reviewing these slides:How many students?How many hundreds?What is the per 100 rate for your school?How are you doing?Is there a trend, are there peaks, patterns?; what can we anticipate?What are we going to do next fall to maintain this success?

59Identification of Problem(for example...)Our average Major ODRs per school day per month are higher than national average for a school of our enrollment sizeOur average ODRs per school day per month are higher this year than for corresponding months of previous yearOur average ODRs per school day per month are showing an increasing trendFaculty, parents, and students say our ODR levels are too high60More Precision Is Required to Solve the Identified ProblemDefine problem by identifying What problem behaviors are involved in ODRsClarify problem by identifyingWhen ODRs are occurring (time of day)Where ODRs are occurring (location)Who is engaging in problem behaviors that result in ODRs Why are problem behaviors continuing to occur

61Problem StatementsUltimately, you want to write a problem statement that precisely specifies the problem you identifiedThe more Ws (what, when, where, who why) you incorporate into the problem statement, the more precise the problem statement will beThe more precise the problem statement, the easier it will be to generate a solution that fits the problem62Which Statement Is More Precise?1a. Too many ODRs1b. Total of 22 aggression ODRs on playground last month; twice as many as last year & showing increasing trend this year; occurring during first recess; 15 different students involved; aggression appears to provide peer attention, and resolve unclear playground rules (who gets equipment),2a. Behavior in cafeteria is uncivil and unsafe.2b. Verbal threats and gender harassment in the cafeteria are increasing; 80% of events are from 4 students during second lunch; We are unclear what is maintaining these behaviors.3a. Hallway noise is unbearable.3b. 4a. The number of ODRs per day has increased by 20% each month since school started.4b. 63Which Statement Is More Precise?1a. Too many ODRs1b. Too many instances of disrespect2a. Too many ODRs between 1:00pm and 1:30pm2b. Too many ODRs in the afternoon3a. Too many ODRs occurring outside the classrooms3b. Too many ODRs on the playground4a. 25% of students have at least 2 ODRs 4b. Many students are experiencing ODRs5a. Too many ODRs on the playground5b. Total of 12 aggression ODRs on playground last month; twice as many as last year & showing increasing trend this year; occurring during first recess; 8 different students involved; aggression appears to provide peer attention.64Use Schoolwide Information System (SWIS) Data to Achieve PrecisionQuestionSWIS Table/GraphWhat problem behaviors are occurring?Referrals by problem behaviorWhen are problem behaviors occurring?Referrals by timeWhere are problem behaviors occurring?Referrals by locationWho is engaging in problem behaviors?Referrals by studentWhy do problem behaviors keep happeningReferrals by motivation65Solutions Generic StrategiesPrevent Remove or alter trigger for problem behaviorDefine & Teach Define behavioral expectations; provide demonstration/instruction in expected behavior (alternative to problem behaviorReward/reinforce The expected/alternative behavior when it occurs; prompt for it, as necessaryWithhold reward/reinforcement For the problem behavior, if possible (Extinction)Use non-rewarding/non-reinforcing corrective consequences When problem behavior occurs

Although not a solution strategy, Safety may need to be considered (i.e., procedures that may be required to decrease likelihood of injuries or property damage)

6666Prevent TriggerDefine & TeachReward/ReinforceWithhold RewardCorrective consequenceOtherSafety67Trevor Test Middle SchoolHypothesis:Implementing SolutionsWho is going to do it?When will they do it?

Minute Taker writes this information down, facilitator follows up at next meeting on status of implementationEvaluating SolutionsDefine the goal for solving the problemWhat will it look like when you say it is not a problemDefine how you will know that the solutions were implemented as planned (with fidelity)?How often will you conduct a status review?Define how you will know that the solutions had a positive effect on student achievement, social competence, and/or safety?How often will you monitor student progress?Achieving a Precise Problem Statementfor Fictional Trevor Test SchoolMiddle School Grades 6, 7, & 8565 students

70Trevor Test Middle Schooln= 565 grades 6-8Is there a problem? Compare to national average, compare to last year, examine trend, examine peaks?

71565/100 = 5.65; 6.65 X .85 = 4.8Trevor Test Middle SchoolIdentified ProblemIdentified problemfor last 4 mos., Major ODRs per day higher than national avg.increasing trend across all 5 mos.

72Trevor Test Middle School 11/01/2007 through 01/31/2008 (last 3 mos.)

7374

What information do we need?Who is involved in problem behavior in the cafeteria?ODRs in the CafeteriaMain problemThe sixth graders are disruptive & use inappropriate language in the cafeteria between 11:30 AM and 12:00 PM to get peer attention.

Trevor Test

The sixth graders are disruptive & use inappropriate language in the cafeteria between 11:30 AM and 12:00 PM to get peer attention.77Prevent TriggerChange lunch schedule so fewer students are eating between 11:30 AM & 12:00 PM?Define & TeachFocus on 6th graders; define cafeteria expectations; develop and post expectation signage in cafeteria; demonstrate/teach expectations in class periods occurring just prior to lunchReward/ReinforceSet up Friday 5 (extra 5 mins. of lunch time on Friday, if no ODRs occur in cafeteria during lunch time)Withhold RewardEnsure staff dont argue back and forth with student if instance of disruption occurs (may be an inadvertent reward); remind students that paying attention to a disruptive student can mess up Friday 5Corrective consequenceEnsure active supervision during lunch (add one supervisor between 11:30 AM and 12:00 PM?); ensure quick corrective consequence, per our handbookOtherDetermine whether Behavior Support Program has been initiated for Student #10; if it has, make sure it includes focus on disruption in cafeteriaSafety78Trevor Test Middle SchoolHypothesis - cafeteria overcrowded; 6th graders with insufficient instruction in cafeteria expectations; attention from adults and peers rewarding disruption

Trevor TestSolution ActionsChoose the solutions that will create an environment that makes the problem irrelevant, inefficient, and ineffective.Choose least amount of work that will have the biggest impact on decreasing the problem.Implementing the solution requires action and time linesProblems need goals so that we can measure progress and know when to move on.

Use weekly 1-5 survey of cafeteria monitors to assess implementation of plan

79Are we doing the plan?

1 .. 2 ..3 .. 4 .. 5No Yes Trevor TestSolution ActionsChoose the solutions that will create an environment that makes the problem irrelevant, inefficient, and ineffective.Choose least amount of work that will have the biggest impact on decreasing the problem.Implementing the solution requires action and time linesProblems need goals so that we can measure progress and know when to move on.

Use weekly 1-5 survey of cafeteria monitors to assess implementation of plan

80Problem Solving Action PlanPrecise Problem StatementSolution ActionsWho?When?Goal, Timeline, Rule & UpdatesMany 6th grade students are engaging in disruption, inappropriate language and harassment in cafeteria and hallway during lunch, and the behavior is maintained by peer attentionPrevention: Maintain current lunch schedule, but shift classes to balance numbers

Teach: Teach behavioral expectations in cafeteria

Principal to adjust schedule and send to staff

Teachers will take class to cafeteria; Cafeteria staff will teach the expectations

Changes begin on Monday

Rotating schedule on November 15

Goal: Reduce cafeteria ODRs by 50% per month (Currently 24 per month average)

Measure: 1. SWIS ODRs 2. Brief fidelity survey

Timeline: Review monthlyRecognition: Establish Friday Five: Extra 5 min of lunch on Friday for five good days

Extinction: Encourage all students to work for Friday Five make reward for problem behavior less likelySchool Counselor and Principal will create chart & staff extra recessPrincipal to give announcement on intercom on MondayCorrective Consequence- Active supervision and continued early consequence (minor/major ODRs)Hall and Cafeteria SupervisorsOngoingData Collection Maintain ODR record & supervisor weekly reportSWIS data entry person & Principal shares report with supervisorsWeeklyThe minute taker/recorder needs to build fluency in getting relevant information documented. This is the plan for Trevor Test Middle School.8182

Application of TIPS model with DIBELS & CICO data

JMApplication of model when monitoring individual student progress

Phoenix Elementary265 Students K-5 A second simulationJudge the time you have left to determine how to go through each of these. The big idea here is to have data for one school in one spot for discussion to define a precision problem statement, solutions, action plan and evaluation plan. An action plan says who will do what by when. An evaluation plan is a plan for specifying how the team will know that their efforts have (1) been implemented as planned (fidelity) and (2) that student outcomes have been effected (decreased rates of problem behavior). A question that needs to be answered is what is the goal? or what will it look like for us to say, we dont have a problem?87Phoenix: Previous Meeting minutes

Phoenix Elementary265/100 = 2.65 2.65 x .34 = .901Do we have a problem? 03-0404-05Trends, patterns. Peaks?What can we anticipate?

89Phoenix Elementary Problem Behaviors

Build the precision problem statementAggression, disrespect, inappropriate language90Phoenix ElementaryLocations

Year OneYear TwoOn the playground91Phoenix Elementary - Time

When are the problems happening? Get school schedule and look at recesses, who is out there, who is supervising, etc?92Phoenix ElementaryReferrals Per Student (2 + Referrals)

Lots of studentsThis leads me to wanting to build a custom report by grade level. Matching that data with the by time data gives more specific information93Problem StatementDo we have a problem?Build a precise problem statementGive best guess on hypothesisOther information sources lead to sharing equipment, taking turns on swings, different games rules for soccer during recess and during soccer games.Have people write the problem statement down. It sounds silly, but it is something people dont want to take the time to do, but we need them to practice. We want to build fluency for writing out precision problem statements so that it is easier to do during a meeting context.94Solution Developmentproblem statement & hypothesis: PreventionTeachingRewardExtinctionCorrective ConsequenceData CollectionHave participants come up with ideasThis table is a prompt for getting the discussion going. teams do not need to identify a solution for each category above, but the categories provide prompts for the team: What can we do to prevent the problem?What can we do to teach the students to do what we expect?What can we do to reinforce appropriate student behavior?What can we do to minimize reinforcement for the problem behavior?What can we do to correct inappropriate behavior?What data will we collect to measure effectiveness of solutions?Choose a solution or a package of solutions to implement. Do not default to doing everything listed. Always choose the least amount of tasks/actions that you think will produce the biggest effect.95Phoenix: Previous Meeting minutes

Langley Elementary School478 StudentsK-597Generate a discussion after 5 minute review of the data.What is the first thing we do? Calculate ODR per 100 studentsReview the other graphs, which of the big 5 are missing? How would you create that report? (custom graph by motivation). What else would you want to know about this school?

A completed exampleIF a person knows how to use the meeting minute form, the person should be able to pick these minutes up from Jan 7, 2010 and be able to organize previous items to update and facilitate creation of the Feb 3, 2010 agenda98

99Pattern of problem behavior is getting better. what did we do in Feb and March to see this outcome? How will we celebrate this success? How will we maintain this success?However, there are two possible problems (as you look at location and at problem behavior. Students typically get tardies from the classroom not the playground) start with two problem statements:Classroom, tardies Playground, 10:30, 12:00, 12:30.To get more precise with these two statements, use SWIS custom reports to generate a by grade report and a by motivation for each of the two locations

100Talk about options for this graph. Get names of students. We have many students getting many referrals.. Classroom, playground, both?Precision Statement/HypothesisWhatWhereWhenWhoWhyWhat other info needed?

Possible Solutions?What are Langleys precision statements. Since you dont have access to other data, make best guesses for the sake of the discussion or at a minimum talk about what other reports would be needed.Then move into discussing possible solutions. The more you can reinforce function-based solutions, the better. IF the possible motivation is escape, what solutions might work best?; what about if the possible motivation is peer attention?, how would that change the discussion?101Solution Developmentproblem statement & hypothesis: PreventionTeachingRewardExtinctionCorrective ConsequenceData Collection102PBIS Team Meeting Minutes and Problem-Solving Action Plan FormTodays Meeting: Date, time, location: Facilitator: Minute Taker:Data Analyst: Next Meeting:Date, time, location: Facilitator: Minute Taker:Data Analyst: Team Members (bold are present today)Todays Agenda Items Next Meeting Agenda Items01. 02. 03. Information for Team, or Issue for Team to AddressDiscussion/Decision/Task (if applicable)Who?By When? Administrative/General Information and IssuesImplementation and EvaluationPrecise Problem Statement, based on review of data(What, When, Where, Who, Why)Solution Actions (e.g., Prevent, Teach, Prompt, Reward, Correction, Extinction, Safety)Who?By When?Goal, Timeline, Decision Rule, & UpdatesProblem-Solving Action PlanOur RatingYesSo-SoNo1. Was todays meeting a good use of our time?2. In general, did we do a good job of tracking whether were completing the tasks we agreed on at previous meetings?3. In general, have we done a good job of actually completing the tasks we agreed on at previous meetings?4. In general, are the completed tasks having the desired effects on student behavior? Evaluation of Team Meeting (Mark your ratings with an X)This slide is animated to teach the different parts of the meeting minute form each click adds the next sectionMost schools have the title at the top and write/type as the meeting progresses

Make a point that we dont need to document everything that happened (i.e., NM rolled her eyes KJ entered the room, SW continued to repeat the same issue, we took at 5 minute bathroom break)103

Sandhill High school354 studentsAnother simulation to work through as a small or larger group. Go through the same steps as in previous simulations.105

Sandhill High School: 354 studentsWhat is our calculated rate per 100 students?How are we doing? As compared to national average? As compared to last year?What are our patterns, trends?What should we anticipate, try to prevent?106Sandhill - Problem Behavior

2009-2010skipping107Sandhill - Location

class108Sandhill Referrals by Student

Lots of students109

Sandhill - Time7:00 was used as a default if time was not written on the referral formFirst period and many unknown. The staff need to be more precise with time of incident . 7:00am has been used as a default for unknown. THIS SWIS Facilitator did not hold the team to a compatible ODR form and didnt put a place for time of incident. If the time was not noted on the ODR, the data entry person was taught to enter 7:00am as unknown110Sandhill hypothesisStudents are skipping class to avoid doing the work8:00---- oversleeping?Other times: avoid class? Gain more social time with peers? We dont know exactly when due to the 7:00 time used as a default

Precision Statement/HypothesisWhatWhereWhenWhoWhyWhat other info needed?

Possible Solutions?Verbalize and write down the precision problem statement. What more do you need to know and how will you get that information?112Solution Developmentproblem statement & hypothesis: PreventionTeachingRewardExtinctionCorrective ConsequenceData CollectionSolutions for tardies in class?The best strategy I have heard from high school teachers is to model being on time, define what being on time looks and sounds like, and before each class period starts, have a potential test question on the board for students to copy as they construct a study guide.113Sandhill: Previous Meeting minutes

Team Training & Follow UpSwift at SWIS TrainingTeam Meetings:First time: Simulated DataSecond time: Use First Month of DataThird time: Problem Solving StatementsWorking with the team overtime is essential. Plan how this will work with your teams. Will you teach the data analyst to create the big 5 report, or will you do it in your SAMI account and get it to them before the meeting? what will your coaching look like? How will you build fluency of team members to build precision problem statements, discuss possible solutions, and build an action and evaluation plan?115Next StepsAs a field:Add TIPS training to Trainer repertoire Integrate messages, language, and processes for using data for problem solving and progress monitoring across the stateDetermine impact of TIPS on student outcomes (next grant proposal)

Coaches: Prompt teams to not only define precision problem statements but to also define a goal for what it will look like when we dont have a problemPrompt team members to be effective and efficient in their rolesdata analysts create and summarize data to jump start the meetingminute takers record relevant information(not novels) about problems discussed, solutions determined and action plan to implement solutionsfacilitators ask questions to facilitate problem solving and decision makingAsk for supportTell the Network what you need in order to be successful in your role(s)Getting a SWIS accountwww.swis.orgWork with a SWIS Facilitator to complete a License AgreementTen readiness requirements includingPositive School culture is a priorityTeam identified to use the data at least monthlyConsistent, coherent procedures for dealing with problem behavior (process and documentation)Data entry time and person scheduledCost of SWIS$250 per year (additional $50 for Check in Check out)Chart86.587121013151198

# per day per 100 studentsOffice Discipline Referrals per Day per Month per 100 Students

SET TrevorTrevor Test Middle SchoolPre PBSYear 1Year 2Expect Defined5010075Expect Taught126040Acknowledge506750Corrections387575Monitoring508875Management448867District Support255050Overall mean387562

SET Trevor

Features% implementedTrevor Test Middle SchoolPre-PBS SET Scores

SET Pond

Pre PBSYear 1Features% implementedTrevor Test Middle SchoolSET Scores

SSS Risk

Pre PBSYear 1Year 2% implementedTrevor Test Middle SchoolSET Scores

SSS ProtectivePre- PBSYear 1Year 2Expect Defined5050100Expect Taught505083Acknow671783Corrections2583100Monitoring138862Leadership288892District Support75100100mean446889

SSS Protective

Pre- PBSFeatures% implementedPond Middle SchoolSET Scores

TIC

Pre- PBSYear 1Year 2Features% implementedPond Middle SchoolSET Scores

ODRsPre-PBSYear 1Weapons0.50.5Vandalism1.71High Mobility22Graffitti1.60.7Gang Activity0.60.5Truancy1.51Suspensions/ Expulsions1.71Court0.70.5Withdrawn0.80.6Child Abuse Home0.30.3Trespassing School0.50.5Poverty1.71.7Crimes10.7Drug/ Alcohol1.91.6Fights Conflict1.61.7Bullying Harrassment1.91.5Deterioration0.70.8

ODRs

FactorsRatingSchool Safety Survey Risk Factors

Pre-PBSYear 1FactorsRatingSchool Safety Survey Risk Factors

Pre-PBSYear 1year 2Extracurricular Opportunities221.5Professional Dev.1.71.91.4Crisis Response2.42.62.5Discipline Plans22.32.4Support Services1.61.82Parent Involvement1.51.41.8Crisis Preparation1.81.92Supervision All1.51.82Sucide Prevention1.822Academic Preparation221.9Positive Climate22.32Diversity Acceptance1.722Conflict Response221.7Community Resources1.61.81.5High Expectations2.32.42Student-Teacher221.7

Pre-PBSFeaturesRatingSchool Safety Survey Protective Factors

Pre-PBSYear 1FeaturesRatingSchool Safety Survey Protective Factors

Pre-PBSYear 1year 2FeaturesRatingSchool Saftey Survey Protective Factors

Pre-PBSYear 1Year 2Commitment10010050Team348350Self Assessment50100100SW Expectations428358Information System5010050Function-based Support05050overall477962

Pre-PBS% implementedTeam Implementation Checklist

Pre-PBSYear 1% implementedTeam Implementation Checklist

Pre-PBSYear 1Year 2% implementedTeam Implementation Checklist

Sept6.5Sept5Oct8Oct7Nov7Nov8Dec12Dec8Jan10Jan9Feb13Feb11March15March13April11April5May9May6June8June5

# per day per 100 studentsOffice Discipline Referrals per Day per Month

# per day per 100 studentsOffice Discipline Referrals per Day per Month

Chart12.5679.510.25

School Avg.School MonthsAvg. ODRs Per School Day

TTNatAvgMiddleTrevor Test Middle School 565 StudentsSchool MonthsODRsSep2.50Oct6.00Nov7.00Dec9.50Jan10.25National Avg. = 4.804.814.8Note: Updated to 2008-2009 dataMean Major ODRs per 100 students per school day = .854.80Information useful for building sample Custom ReportSep-JanMonths in graph5Avg. no. school days per mo.20Total school days for 5 mos.100Avg. ODRs per school day7.05Expected Total ODRs705Nov-JanMonths in graph3Avg. no. school days per mo.20Total school days for 3 mos.60Avg. ODRs per school day8.92Expected Total ODRs535Big 4 Graphs for the Nov-Jan time period shouldTotal 535 MajorsPlus additional MinorsOriginal TotalsRevised TotalsRoundingMajors190534.85535Minors1850.6751Combined208586To achieve believeable result for Nov-Jan time periodMultiply Majors and Minors by 2.815The Rounded Totals should appear in the Big 4 Reports

TTNatAvgMiddle

School Avg.National Avg. = 4.8School MonthsAvg. ODRs Per School Day

TTProbBeh2.815Original Master List, UnsortedOriginal Master List, Sorted1Revised Master List, Multiplied by 2.815Problem BehaviorTypeFrequencyProblem BehaviorTypeFrequencyPercentNewFreqProblem BehaviorTypeFrequencyX 2.815 RoundedTardyMajor0TardyMajor000ArsonMajor00Forge/TheftMajor0BombMajor000BombMajor00Dress CodeMajor0ArsonMajor000DrugsMajor00TobaccoMajor0WeaponsMajor000Gang displayMajor00AlcoholMajor0OtherMajor000Iapp affectionMajor00CombustiblesMajor0UnknownMajor000M-DressMinor00BombMajor0DrugsMajor000M-OtherMinor00ArsonMajor0TechMajor000M-Prpty MisuseMinor00WeaponsMajor0Iapp affectionMajor000M-TechMinor00OtherMajor0Out boundsMajor000M-UnknownMinor00UnknownMajor0Gang displayMajor000M-WarningMinor00DrugsMajor0M-WarningMinor000Other behavMajor00TechMajor0M-Prpty MisuesMinor000Out boundsMajor00Iapp affectionMajor0M-OtherMinor000TardyMajor00Out boundsMajor0M-DressMinor000TechMajor00Gang displayMajor0M-TechMinor000Unknown behavMajor00Inapp lanMajor1M-UnknownMinor000WeaponsMajor00Agg/FightMajor1LyingMajor10.00480769230.5961538462LyingMajor13LyingMajor1M-DisruptionMinor10.00480769230.5961538462M-DisruptionMinor13Prop damMajor1TobaccoMajor20.00961538461.1923076923AlcoholMajor26VandalMajor1AlcoholMajor20.00961538461.1923076923CombustMajor26HarassMajor3CombustiblesMajor20.00961538461.1923076923DressMajor26DisruptionMajor5Dress CodeMajor20.00961538461.1923076923M-Inapp lanMinor26SkipMajor5M-Inapp lanMinor20.00961538461.1923076923TobaccoMajor26DisresptMajor15Forge/TheftMajor30.01442307691.7884615385M-ContactMinor38M-WarningMinor0VandalMajor30.01442307691.7884615385M-DisresptMinor38M-Prpty MisuesMinor0M-ContactMinor30.01442307691.7884615385TheftMajor38M-OtherMinor0M-DisresptMinor30.01442307691.7884615385VandalMajor38M-DressMinor0Prop damMajor70.03365384624.1730769231Prop damMajor720M-TechMinor0Agg/FightMajor90.04326923085.3653846154Agg/FightMajor925M-TardyMinor0M-TardyMinor90.04326923085.3653846154M-TardyMinor925M-UnknownMinor0SkipMajor180.086538461510.7307692308SkipMajor1851M-DisruptionMinor1HarassMajor270.129807692316.0961538462HarassMajor2776M-Inapp lanMinor2DisresptMajor320.153846153819.0769230769DisresptMajor3290M-ContactMinor3Inapp lanMajor350.168269230820.8653846154Inapp lanMajor3599M-DisresptMinor3DisruptionMajor470.225961538528.0192307692DisruptionMajor4713220812420858636All Referrals & Minors08/01/2006-10/31/2006

TTProbBeh

Number of ReferralsReferrals by Problem Behavior

TTTime2.815OriginalRevisedTimeFrequencyTimeFrequencyX 2.815 Rounded7:00 AM07:00 AM007:15 AM07:15 AM007:30 AM07:30 AM007:45 AM07:45 AM008:00 AM78:00 AM7208:15 AM48:15 AM4118:30 AM38:30 AM388:45 AM28:45 AM269:00 AM59:00 AM5149:15 AM89:15 AM8239:30 AM09:30 AM009:45 AM89:45 AM82310:00 AM010:00 AM0010:15 AM410:15 AM41110:30 AM510:30 AM51410:45 AM610:45 AM61711:00 AM111:00 AM1311:15 AM311:15 AM3811:30 AM2011:30 AM205611:45 AM4211:45 AM4211812:00 PM2212:00 PM226212:15 PM1312:15 PM133712:30 PM412:30 PM41112:45 PM312:45 PM381:00 PM01:00 PM001:15 PM31:15 PM381:30 PM41:30 PM4111:45 PM51:45 PM5142:00 PM22:00 PM262:15 PM42:15 PM4112:30 PM22:30 PM262:45 PM112:45 PM11313:00 PM83:00 PM8233:15 PM43:15 PM4123:30 PM53:30 PM5143:45 PM03:45 PM004:00 PM04:00 PM004:15 PM04:15 PM004:30 PM04:30 PM004:45 PM04:45 PM005:00 PM05:00 PM00208208586

TTTime

Number of ReferralsReferrals by Time

TTLocation2.815OriginalRevisedLocationFrequencyLocationFrequencyX 2.815Park lot0Locker rm00Unknown0Off-Campus00Office0Office00Locker rm0Park lot00Off-Campus0Plygd00Stadium0Stadium00Plygd0Unknown00Library4Library412Bathrm5Bathrm514Bus Zn5Bus Zn514Gym5Gym514Special evt7Bus720Bus7Special evt720Other8Other loc823Class26Class2673Common30Common3084Hall41Hall41115Caf70Caf70197208208586

TTLocation

Number of ReferralsReferrals by Location

TTStudent2.815OriginalRevisedStudentFrequencyStudentFrequencyX 2.815211133121351310281114131315382016514221772061183862191365110257066111411681125146911313122146172121572058216136721726131813103191542173201323321263732251447323385932413184255146042672024527226234528133952941148530135453138635321645256331364634617407355145073672056737265773813351039383815403849164138522242514422543264320843208586CountSumCountSumSum

TTStudent

Student No.Number of ReferralsReferrals by Student

TTClarifyAllTrevor Test Middle School All Referrals & Minors 11/01/2006 - 01/31/2007

TTClarifyAll000000000000000003366666888820252551769099132


Sheet100002011861423023011141738561186237118081114611631231214000000


DataForCR0000000121414142020237384115197


2.8152.81528.152.8158.44514.07519.7058.4452.81570.37511.2614.0752.81516.8919.7052.8155.632.81542.2252.8155.6314.0758.4452.81514.07519.70561.932.81511.262.8158.44545.042.81516.8914.07519.7055.632.8158.4458.4458.44514.0755.63


389994192413841645482031272348524308938126318150513371028342933180713781301085431530173344213003371582391556524604041503147912336144131618155029410511104742108212422435885626362215725160434109114376115492136417774045262223313321951532377727251610

Data for Creating Trevor Test Elementary School Custom ReportDemographics: Grades 6, 7, & 8Number of students: 565Custom Report Time Period: 11/01/2006 - 01/31/2007Problem BehaviorTimeLocationStudentRevised Master List, Multiplied by 2.815RevisedRevisedRevisedProblem BehaviorTypeTT FreqTimeTT FreqCF FreqDifferenceLocationTT FreqCF FreqDifferenceStudentX 2.815TardyMajor07:00 AM0Park lot02023BombMajor07:15 AM0Unknown0233ArsonMajor07:30 AM0Office0353WeaponsMajor07:45 AM0Locker rm00113OtherMajor08:00 AM20Off-Campus00133UnknownMajor08:15 AM11Stadium00203DrugsMajor08:30 AM8Plygd073223TechMajor08:45 AM6Library1120613Iapp affectionMajor09:00 AM14Bathrm1447623Out boundsMajor09:15 AM23Bus Zn1414653Gang displayMajor09:30 AM0Gym1421663M-WarningMinor09:45 AM23Special evt2011683M-Prpty MisuesMinor010:00 AM0Bus2014693M-OtherMinor010:15 AM11Other238126M-DressMinor010:30 AM14Class73175216M-TechMinor010:45 AM17Common8441586M-UnknownMinor011:00 AM3Hall11586676LyingMajor311:15 AM8Caf1973218M-DisruptionMinor311:30 AM56586567108TobaccoMajor611:45 AM118178AlcoholMajor612:00 PM62238CombustiblesMajor612:15 PM37378Dress CodeMajor612:30 PM11478M-Inapp lanMinor612:45 PM8598Forge/TheftMajor81:00 PM01811VandalMajor81:15 PM86011M-ContactMinor81:30 PM112414M-DisresptMinor81:45 PM143414Prop damMajor202:00 PM63914Agg/FightMajor252:15 PM114814M-TardyMinor252:30 PM65414SkipMajor512:45 PM316314HarassMajor763:00 PM232517DisresptMajor903:15 PM116417Inapp lanMajor993:30 PM144020DisruptionMajor1323:45 PM050205864:00 PM056204:15 PM057204:30 PM035284:45 PM038425:00 PM049455865262427043586CountSum

Chart100000000000000000003366666888820252551769099132


TTProbBehAll Referrals & Minors08/01/2006-10/31/2006Problem BehaviorFrequencyMinor0Tardy0Bomb0Arson0Weapons0Other0Unknown0Drugs0M-Prpty Misuse0M-Other0M-Dress0M-Tech0Tech0Inapp affection0Out bounds0M-Unknown0Gang display0Skip0Truan0Lying3M-Disruption3Dress6Tobacco6Alcohol6Combust6M-Inapp lan6Forge/Theft8Vandal8M-Contact8M-Disrespt8Prop dam20Agg/Fight25M-Tardy25Skip51Harass76Disrespt90Inapp lan99Disruption132586

TTProbBeh


TTTimeAll Referrals & Minors08/01/2006-10/31/2006TimeFrequency7:00 AM07:15 AM07:30 AM07:45 AM08:00 AM208:15 AM118:30 AM88:45 AM69:00 AM149:15 AM239:30 AM09:45 AM2310:00 AM010:15 AM1110:30 AM1410:45 AM1711:00 AM311:15 AM811:30 AM6811:45 AM11512:00 PM5312:15 PM3712:30 PM1112:45 PM81:00 PM01:15 PM81:30 PM111:45 PM142:00 PM62:15 PM112:30 PM62:45 PM313:00 PM233:15 PM123:30 PM143:45 PM04:00 PM04:15 PM04:30 PM04:45 PM05:00 PM0586

TTTime


TTLocationAll Referrals & Minors08/01/2006-10/31/2006LocationFrequencyPlygd0Park lot0Unknown0Office0Locker rm0Off-Campus0Stadium0Music rm0Library12Bathrm14Gym14Bus Zn14Bus20Special evt20Other23Class73Common84Hall115Caf197586

TTLocation


TTStudentAll Referrals & Minors08/01/2006-10/31/2006StudentFrequency131331631832320324328330333338343931762163764362383183984084185888111129111214221425143514421461414173417152026203620720328194232452762107043586CountSum

TTStudent


Sheet1

Sheet2

Sheet3

Chart200002011861423023011141738681155337118081114611631231214000000



TTProbBeh



TTTime



TTLocation



TTStudent


Sheet1

Sheet2

Sheet3

Chart300000000121414142020237384115197



TTProbBeh



TTTime



TTLocation



TTStudent


Sheet1

Sheet2

Sheet3

Chart42.8152.8152.8152.8152.8152.8152.8152.8152.8152.8152.8152.8152.8155.635.635.635.638.4458.4458.4458.4458.4458.4458.44511.2611.2614.07514.07514.07514.07514.07514.07516.8916.8919.70519.70519.70519.70528.1542.22545.0461.9370.375



TTProbBeh



TTTime



TTLocation



TTStudent


Sheet1

Sheet2

Sheet3

Documents

Team Initiated Problem Solving