110
Team Initiated Problem Solving Rob Horner, Steve Newton, & Anne Todd, University of Oregon Bob Algozzine & Kate Algozzine, University of North Carolina at Charlotte Coaches Conference Feb 3, 2010 Oregon State University www.swis.org [email protected] [email protected] Anne Todd

Team Initiated Problem Solving Rob Horner, Steve Newton, & Anne Todd, University of Oregon Bob Algozzine & Kate Algozzine, University of North Carolina

Embed Size (px)

Citation preview

  • Slide 1

Team Initiated Problem Solving Rob Horner, Steve Newton, & Anne Todd, University of Oregon Bob Algozzine & Kate Algozzine, University of North Carolina at Charlotte Coaches Conference Feb 3, 2010 Oregon State University www.swis.org [email protected] [email protected] Anne Todd Slide 2 Todays Goals Coaches are able to: Prompt & support facilitator, minute taker and data analyst to prepare for meetings Meeting Foundations Checklist Prompt the use of the TIPS model during meetings Data-based Decision-making rules Help teams stay focused during meetings Electronic Meeting Minute format Clarification Coaches are NOT expected to be Trainers Trainers deliver TIPS team training & help Coaches anticipate errors while guiding them through the possible solutions & adaptations Slide 3 Context Every school has teams Teams are being expected to do problem solving Select curricula Get training and implement new ideas/programs Provide efficient leadership Communities of Practice Teams need to report data to administration, district, state Teams NEED data to do good problem solving. Most teams are not skilled at running problem solving meetings and using data for decision-making. Slide 4 Assumption: Coaching is Critical Teams will need more than a manual or brief training to become skilled at use of data for efficient problem solving Coaching will be a key element to successful use of good problem solving procedures. Slide 5 What do we need? A clear model with steps for problem solving Access to the right information at the right time in the right format A formal process that a group of people can use to build and implement solutions. Slide 6 Collect and Use and UseData Review Status and Identify Problems Develop and Refine Hypotheses Discuss and Select Solutions Develop and Implement Action Plan Evaluate and Revise Action Plan Problem Solving Meeting Foundations Team Initiated Problem Solving (TIPS) Model Slide 7 TIPS Model TIPS Training One full day team training Two coached meetings Team Meeting Use of electronic meeting minute system Formal roles (facilitator, recorder, data analyst) Specific expectations (before meeting, during meeting, after meeting) Access and use of data Projected meeting minutes Research tool to measure effectiveness of TIPS Training DORA (decision, observation, recording and analysis) Measures Meeting Foundations & Thoroughness of Problem Solving Slide 8 Slide 9 Evidence of Effectiveness Evaluation Study (2007-08) Newton et al., Single-case Study (2008-09) Todd et al., Group Design Study (2009-10) Slide 10 TIPS Study: Todd et al., 2009 School A School B School C School D Meeting Foundations Score Baseline Coaching TIPS % DORA Foundations Score Slide 11 TIPS Study: Todd, et al, 2009. School A School D School C Baseline Coaching TIPS Thoroughness of decision-making % DORA Thoroughness Score Slide 12 SYSTEMS PRACTICES DATA Supporting Staff & Student Behavior and Decision Making Building Capacity and Sustainability OUTCOMES For Social Competence, Academic Achievement, and Safety SWIS Electronic Meeting Minutes Form *Meeting time *Support *Report to Faculty Slide 13 Improving Decision-Making via Problem Solving Problem Solving Solution Information/ Data Action Planning & Evaluation Slide 14 Structure of meetings lays foundation for efficiency & effectiveness Slide 15 Using Meeting Minutes Documentation of Logistics of meeting (date, time, location, roles) Agenda items for todays meeting ( and next meeting) Discussion items, decisions made, tasks and timelines assigned Problem statements, solutions/decisions/tasks, people assigned to implement with timelines assigned, and an evaluation plan to determine the effect on student behavior Reviewing Meeting minutes An effective strategy for getting a snapshot of what happened at the previous meeting and what needs to be reviewed during the upcoming meeting What was the issue/problem?, What were we going to do?, Who was going to do it and by When?, and How are we measuring progress toward the goal? Visual tracking of focus topics during and after meetings Prevents side conversations Prevents repetition Encourages completion of tasks Slide 16 Organizing for an effective problem solving conversation Problem Solution Out of Time Use Data A key to collective problem solving is to provide a visual context that allows everyone to follow and contribute Slide 17 Building a system that is NOT person dependent We want to walk into a meeting having no previous history, review the previous meeting minutes be able to fit into any role needed Facilitator Minute taker Data analyst Active team member Example Slide 18 PBIS Team Meeting Minutes and Problem-Solving Action Plan Form Todays Meeting: Date, time, location: Facilitator: Minute Taker:Data Analyst: Next Meeting:Date, time, location: Facilitator: Minute Taker:Data Analyst: Team Members (bold are present today) Todays Agenda Items Next Meeting Agenda Items 01. 02. 03. 1. 2. Information for Team, or Issue for Team to Address Discussion/Decision/Task (if applicable)Who?By When? Administrative/General Information and Issues Implementation and Evaluation Precise Problem Statement, based on review of data (What, When, Where, Who, Why) Solution Actions (e.g., Prevent, Teach, Prompt, Reward, Correction, Extinction, Safety) Who?By When? Goal, Timeline, Decision Rule, & Updates Problem-Solving Action Plan Our Rating YesSo-SoNo 1. Was todays meeting a good use of our time? 2. In general, did we do a good job of tracking whether were completing the tasks we agreed on at previous meetings? 3. In general, have we done a good job of actually completing the tasks we agreed on at previous meetings? 4. In general, are the completed tasks having the desired effects on student behavior? Evaluation of Team Meeting (Mark your ratings with an X) Slide 19 Slide 20 Important Structural Components Regular meetings & regular attendance The right people The right roles Facilitator Minute Taker Data Analyst Active Team Members The right information for problem solving & decision making Accomplishments Products of successful meeting Meeting Minutes (record of decisions & tasks concerning administrative/general issues) Problem-Solving Action Plan (record of decisions & tasks concerning problems identified by team) 20 Slide 21 21 Slide 22 22 Slide 23 Before the Meeting Room reserved New items solicited for agenda Agenda produced Team member roles determined Data reviewed by Data Analyst before the meeting; Analyst ready to lead team through discussion of (a) possible new problems and (b) effects of in-process solutions on old problems Computer reserved; access to SWIS online database assured LCD projector reserved & set up to project data (or team has some other strategy for ensuring team members can review data at meeting) Team members have individual TIPS Notebooks to bring to meeting (Well review the (a) before-meeting, (b) during-meeting, and (c) after- meetings responsibilities of individual team members later in this workshop) 23 Slide 24 At Close of and After Meeting Meeting Minutes and Problem-Solving Action Plan completed Copy of Meeting Minutes & Problem-Solving Action Plan distributed to each member within 24 hrs. 24 Slide 25 25 Slide 26 Activity Complete the Foundations Checklist Use the PBIS team you know best How would you use the Foundations Checklist to help a school team that was preparing to adopt TIPS procedures? Slide 27 Collect and Use and UseData Develop Hypothesis Discuss and Select Solutions Develop and Implement Action Plan Evaluate and Revise Action Plan Problem Solving Meeting Foundations Team Initiated Problem Solving (TIPS) Model Identify Problems 27 Slide 28 SWIS TM (School-Wide Information System) Defined SWIS TM is a web-based information system for gathering, entering, summarizing, reporting and using office discipline referral information Purpose A progress monitoring tool for improving the ability of school personnel to develop safe and effective learning environments Slide 29 Three Key Elements of SWIS TM Data Collection System Coherent system for assigning referrals Prob. behavior definitions, referral form, rules for referral Allocation of FTE to enter data, build reports. Computer Application Web-based, continuously available, secure Decision-making Use of data School-wide Individual Student Slide 30 Features of SWIS TM Only reports discipline data Major office referrals Minor discipline offences Highly efficient (30 sec per referral) Local control Formatted for decision-making (pictures) Information is available continuously Confidential, secure Can be combined with district data base Slide 31 Slide 32 How SWIS TM works Data Entry School Address and Contact Enrollment/Ethnicity/Days per month Staff Information Student Information Referrals Reporting Average Referrals per Day per month Referrals by Problem Behavior Referrals by Location Referrals by Time Referrals by Student Other Reports Tools Slide 33 Slide 34 Total Office Discipline Referrals Total Office Discipline Referrals as of January 10 Slide 35 Slide 36 What behaviors are problematic? Slide 37 Slide 38 Slide 39 Where are the problems occurring? Slide 40 Slide 41 When are the problems occurring? Slide 42 Slide 43 Who is contributing to the problem? Slide 44 Organizing SWIS Data for Decision-making Universal Screening Tool Proportion of students with 0-1 Office Discipline Referrals (ODRs) 2-5 ODRs 6+ ODRs Progress Monitoring Tool Compare data across time Prevent previous problem patterns Define Problems with precision that lead to solvable problems Slide 45 ~80% of Students ~15% ~5% 0-1 office discipline referral 6+ office discipline referrals 2-5 office discipline referrals Using office discipline referrals as a metric for universal screening of student social behavior Slide 46 Using the Referrals by Student report as a Universal Screening Tool Slide 47 Cumulative Mean ODRs Cumulative Mean ODRs Per Month for 325+ Elementary Schools 08-09 Jennifer Frank, Kent McIntosh, Seth May Slide 48 Using ODRs to Identify Problems Build a picture for the pattern of office referrals in your school. Compare the picture with a national average Compare the picture with previous years Compare the picture with social standards of faculty, families, students. Goal 1.Identify problems empirically 2.Identify problems early 3.Identify problems in a manner that leads to problem solving not just whining Slide 49 Using ODRs to Identify Problems Build a picture for the pattern of office referrals in your school. Compare the picture with a national average Compare the picture with previous years Compare the picture with social standards of faculty, families, students. Slide 50 SWIS summary 2008-2009 (Majors Only) 3,410 schools; 1,737,432 students; 1,500,770 ODRs Grade Range Number of Schools Avg. Enrollment per school National Avg. for Major ODRs per 100 students, per school day K-62,162450.34 = about 1 Major ODR every 3 school days, or about 34 every 100 days 6-9602657.85 = a little less than 1 Major ODR per school day, or about 85 every 100 days 9-122158871.27 = more than 1 Major ODR per school day, or about 127 every 100 days K- (8-12) 4314081.06 = about 1 Major ODR per school day, or about 106 every 100 days 50 How to use these numbers: 1.Your enrollment (e.g. 400 students or 225 students) 2.Divide by 100 (e.g. 400/100 = 4; 225/100 = 2.25 3.Multiply by the National Average to get ODR per day 1.(4 X.34 = 1.36 2.25 X.34 =.76) Slide 51 SWIS summary 2008-2009 (Majors Only) 3,410 schools; 1,737,432 students; 1,500,770 ODRs Grade Range Number of Schools Avg. Enrollment per school National Avg. for Major ODRs per 100 students, per school day K-62,162450.34 = about 1 Major ODR every 3 school days, or about 34 every 100 days 6-9602657.85 = a little less than 1 Major ODR per school day, or about 85 every 100 days 9-122158871.27 = more than 1 Major ODR per school day, or about 127 every 100 days K- (8-12) 4314081.06 = about 1 Major ODR per school day, or about 106 every 100 days 51 Slide 52 Elementary School with 150 Students Compare with National Average 150 / 100 = 1.50 1.50 X.34 =.51 Slide 53 Elementary School with 450 Students Compare with National Average 450 / 100 = 4.50 4.50 X.34 = 1.53 Slide 54 Application Activity: Absolute Value Is there a Problem? Middle School of 625 students? Compare with national average: 625/100 = 6.25 6.25 X.85 = 5.31 Office Discipline Referrals per School Day Slide 55 High School of 1800 students High School: Compare with National Average 1800 / 100 = 18 18 X 1.27= 22.86 Slide 56 High School of 450 students High School: Compare with National Average 450 / 100 = 4.5 4.5 X 1.27= 5.17 Slide 57 Middle School of 700 students Slide 58 Slide 59 Slide 60 Identification of Problem (for example...) Our average Major ODRs per school day per month are higher than national average for a school of our enrollment size Our average ODRs per school day per month are higher this year than for corresponding months of previous year Our average ODRs per school day per month are showing an increasing trend Faculty, parents, and students say our ODR levels are too high 60 Slide 61 More Precision Is Required to Solve the Identified Problem 1.Define problem by identifying What problem behaviors are involved in ODRs 2.Clarify problem by identifying a)When ODRs are occurring (time of day) b)Where ODRs are occurring (location) c)Who is engaging in problem behaviors that result in ODRs d)Why are problem behaviors continuing to occur 61 Slide 62 Problem Statements Ultimately, you want to write a problem statement that precisely specifies the problem you identified The more Ws (what, when, where, who why) you incorporate into the problem statement, the more precise the problem statement will be The more precise the problem statement, the easier it will be to generate a solution that fits the problem 62 Slide 63 Which Statement Is More Precise? 1a. Too many ODRs1b. Total of 22 aggression ODRs on playground last month; twice as many as last year & showing increasing trend this year; occurring during first recess; 15 different students involved; aggression appears to provide peer attention, and resolve unclear playground rules (who gets equipment), 2a. Behavior in cafeteria is uncivil and unsafe. 2b. Verbal threats and gender harassment in the cafeteria are increasing; 80% of events are from 4 students during second lunch; We are unclear what is maintaining these behaviors. 3a. Hallway noise is unbearable.3b. 4a. The number of ODRs per day has increased by 20% each month since school started. 4b. 63 Slide 64 Which Statement Is More Precise? 1a. Too many ODRs1b. Too many instances of disrespect 2a. Too many ODRs between 1:00pm and 1:30pm 2b. Too many ODRs in the afternoon 3a. Too many ODRs occurring outside the classrooms 3b. Too many ODRs on the playground 4a. 25% of students have at least 2 ODRs 4b. Many students are experiencing ODRs 5a. Too many ODRs on the playground 5b. Total of 12 aggression ODRs on playground last month; twice as many as last year & showing increasing trend this year; occurring during first recess; 8 different students involved; aggression appears to provide peer attention. 64 Slide 65 Use Schoolwide Information System (SWIS) Data to Achieve Precision QuestionSWIS Table/Graph What problem behaviors are occurring? Referrals by problem behavior When are problem behaviors occurring? Referrals by time Where are problem behaviors occurring? Referrals by location Who is engaging in problem behaviors? Referrals by student Why do problem behaviors keep happening Referrals by motivation 65 Slide 66 Solutions Generic Strategies Prevent Remove or alter trigger for problem behavior Define & Teach Define behavioral expectations; provide demonstration/instruction in expected behavior (alternative to problem behavior Reward/reinforce The expected/alternative behavior when it occurs; prompt for it, as necessary Withhold reward/reinforcement For the problem behavior, if possible (Extinction) Use non-rewarding/non-reinforcing corrective consequences When problem behavior occurs Although not a solution strategy, Safety may need to be considered (i.e., procedures that may be required to decrease likelihood of injuries or property damage) 66 Slide 67 Prevent Trigger Define & Teach Reward/Reinforce Withhold Reward Corrective consequence Other Safety 67 Trevor Test Middle School Hypothesis: Slide 68 Implementing Solutions Who is going to do it? When will they do it? Minute Taker writes this information down, facilitator follows up at next meeting on status of implementation Slide 69 Evaluating Solutions Define the goal for solving the problem What will it look like when you say it is not a problem Define how you will know that the solutions were implemented as planned (with fidelity)? How often will you conduct a status review? Define how you will know that the solutions had a positive effect on student achievement, social competence, and/or safety? How often will you monitor student progress? Slide 70 Achieving a Precise Problem Statement for Fictional Trevor Test School Middle School Grades 6, 7, & 8 565 students 70 Slide 71 Trevor Test Middle School n= 565 grades 6-8 Is there a problem? Compare to national average, compare to last year, examine trend, examine peaks? 71 565/100 = 5.65; 6.65 X.85 = 4.8 Slide 72 Trevor Test Middle School Identified Problem Identified problem for last 4 mos., Major ODRs per day higher than national avg. increasing trend across all 5 mos. 72 Slide 73 Trevor Test Middle School 11/01/2007 through 01/31/2008 (last 3 mos.) 73 Slide 74 74 Slide 75 What information do we need? Who is involved in problem behavior in the cafeteria? ODRs in the Cafeteria Slide 76 Main problem The sixth graders are disruptive & use inappropriate language in the cafeteria between 11:30 AM and 12:00 PM to get peer attention. Slide 77 Trevor Test The sixth graders are disruptive & use inappropriate language in the cafeteria between 11:30 AM and 12:00 PM to get peer attention. 77 Slide 78 Prevent TriggerChange lunch schedule so fewer students are eating between 11:30 AM & 12:00 PM? Define & TeachFocus on 6 th graders; define cafeteria expectations; develop and post expectation signage in cafeteria; demonstrate/teach expectations in class periods occurring just prior to lunch Reward/ReinforceSet up Friday 5 (extra 5 mins. of lunch time on Friday, if no ODRs occur in cafeteria during lunch time) Withhold RewardEnsure staff dont argue back and forth with student if instance of disruption occurs (may be an inadvertent reward); remind students that paying attention to a disruptive student can mess up Friday 5 Corrective consequenceEnsure active supervision during lunch (add one supervisor between 11:30 AM and 12:00 PM?); ensure quick corrective consequence, per our handbook OtherDetermine whether Behavior Support Program has been initiated for Student #10; if it has, make sure it includes focus on disruption in cafeteria Safety 78 Trevor Test Middle School Hypothesis - cafeteria overcrowded; 6th graders with insufficient instruction in cafeteria expectations; attention from adults and peers rewarding disruption Slide 79 Trevor Test Solution Actions Choose the solutions that will create an environment that makes the problem irrelevant, inefficient, and ineffective. Choose least amount of work that will have the biggest impact on decreasing the problem. Implementing the solution requires action and time lines Problems need goals so that we can measure progress and know when to move on. Use weekly 1-5 survey of cafeteria monitors to assess implementation of plan 79 Are we doing the plan? 1 .. 2 ..3 .. 4 .. 5 No Yes Slide 80 Trevor Test Solution Actions Choose the solutions that will create an environment that makes the problem irrelevant, inefficient, and ineffective. Choose least amount of work that will have the biggest impact on decreasing the problem. Implementing the solution requires action and time lines Problems need goals so that we can measure progress and know when to move on. Use weekly 1-5 survey of cafeteria monitors to assess implementation of plan 80 Slide 81 Problem Solving Action Plan Precise Problem Statement Solution ActionsWho?When?Goal, Timeline, Rule & Updates Many 6 th grade students are engaging in disruption, inappropriate language and harassment in cafeteria and hallway during lunch, and the behavior is maintained by peer attention Prevention: Maintain current lunch schedule, but shift classes to balance numbers Teach: Teach behavioral expectations in cafeteria Principal to adjust schedule and send to staff Teachers will take class to cafeteria; Cafeteria staff will teach the expectations Changes begin on Monday Rotating schedule on November 15 Goal: Reduce cafeteria ODRs by 50% per month (Currently 24 per month average) Measure: 1. SWIS ODRs 2. Brief fidelity survey Timeline: Review monthly Recognition: Establish Friday Five: Extra 5 min of lunch on Friday for five good days Extinction: Encourage all students to work for Friday Five make reward for problem behavior less likely School Counselor and Principal will create chart & staff extra recess Principal to give announcement on intercom on Monday Corrective Consequence- Active supervision and continued early consequence (minor/major ODRs) Hall and Cafeteria SupervisorsOngoing Data Collection Maintain ODR record & supervisor weekly report SWIS data entry person & Principal shares report with supervisors Weekly Slide 82 82 Slide 83 Slide 84 Slide 85 JM Slide 86 Application of model when monitoring individual student progress Slide 87 265 Students K-5 Slide 88 Phoenix: Previous Meeting minutes Slide 89 Phoenix Elementary 265/100 = 2.65 2.65 x.34 =.901 Slide 90 Phoenix Elementary Problem Behaviors Slide 91 Phoenix Elementary Locations Year One Year Two Slide 92 Phoenix Elementary - Time Slide 93 Phoenix Elementary Referrals Per Student (2 + Referrals) Slide 94 Problem Statement Do we have a problem? Build a precise problem statement Give best guess on hypothesis Other information sources lead to sharing equipment, taking turns on swings, different games rules for soccer during recess and during soccer games. Slide 95 Solution Development problem statement & hypothesis: Prevention Teaching Reward Extinction Corrective Consequence Data Collection Slide 96 Phoenix: Previous Meeting minutes Slide 97 478 Students K-5 Slide 98 Slide 99 Slide 100 Slide 101 Precision Statement/Hypothesis What Where When Who Why What other info needed? Possible Solutions? Slide 102 Solution Development problem statement & hypothesis: Prevention Teaching Reward Extinction Corrective Consequence Data Collection Slide 103 PBIS Team Meeting Minutes and Problem-Solving Action Plan Form Todays Meeting: Date, time, location: Facilitator: Minute Taker:Data Analyst: Next Meeting:Date, time, location: Facilitator: Minute Taker:Data Analyst: Team Members (bold are present today) Todays Agenda Items Next Meeting Agenda Items 01. 02. 03. 1. 2. Information for Team, or Issue for Team to Address Discussion/Decision/Task (if applicable)Who?By When? Administrative/General Information and Issues Implementation and Evaluation Precise Problem Statement, based on review of data (What, When, Where, Who, Why) Solution Actions (e.g., Prevent, Teach, Prompt, Reward, Correction, Extinction, Safety) Who?By When? Goal, Timeline, Decision Rule, & Updates Problem-Solving Action Plan Our Rating YesSo-SoNo 1. Was todays meeting a good use of our time? 2. In general, did we do a good job of tracking whether were completing the tasks we agreed on at previous meetings? 3. In general, have we done a good job of actually completing the tasks we agreed on at previous meetings? 4. In general, are the completed tasks having the desired effects on student behavior? Evaluation of Team Meeting (Mark your ratings with an X) Slide 104 Slide 105 Sandhill High school 354 students Slide 106 Sandhill High School: 354 students Slide 107 Sandhill - Problem Behavior 2009-2010 Slide 108 Sandhill - Location Slide 109 Sandhill Referrals by Student Slide 110 Sandhill - Time 7:00 was used as a default if time was not written on the referral form Slide 111 Sandhill hypothesis Students are skipping class to avoid doing the work 8:00---- oversleeping? Other times: avoid class? Gain more social time with peers? We dont know exactly when due to the 7:00 time used as a default Slide 112 Precision Statement/Hypothesis What Where When Who Why What other info needed? Possible Solutions? Slide 113 Solution Development problem statement & hypothesis: Prevention Teaching Reward Extinction Corrective Consequence Data Collection Slide 114 Sandhill: Previous Meeting minutes Slide 115 Team Training & Follow Up Swift at SWIS Training Team Meetings: First time: Simulated Data Second time: Use First Month of Data Third time: Problem Solving Statements Slide 116 Next Steps As a field: Add TIPS training to Trainer repertoire Integrate messages, language, and processes for using data for problem solving and progress monitoring across the state Determine impact of TIPS on student outcomes (next grant proposal) Coaches: Prompt teams to not only define precision problem statements but to also define a goal for what it will look like when we dont have a problem Prompt team members to be effective and efficient in their roles data analysts create and summarize data to jump start the meeting minute takers record relevant information(not novels) about problems discussed, solutions determined and action plan to implement solutions facilitators ask questions to facilitate problem solving and decision making Ask for support Tell the Network what you need in order to be successful in your role(s) Slide 117 Getting a SWIS account www.swis.org Work with a SWIS Facilitator to complete a License Agreement Ten readiness requirements including Positive School culture is a priority Team identified to use the data at least monthly Consistent, coherent procedures for dealing with problem behavior (process and documentation) Data entry time and person scheduled Cost of SWIS $250 per year (additional $50 for Check in Check out)