37
School-Based Leadership Team Training Survey & Skill Assessments Administration Guidelines for Day 3 Directions : The evaluation instruments you received for the School-Based Leadership Team (SBLT) trainings are organized by school and by instrument in mailing envelopes. The name of the school and instrument is included on each envelope. Each of the instruments is coded with a district and school ID number so it is important that you administer the instruments for a given school only to members of the SBLT from that school. To facilitate administration of these measures to the appropriate SBLT members, it is recommended that you ask that participants from the same school sit together. What follows are guidelines for administering evaluation tools during Day 3 of the SBLT trainings. Administration Guidelines For each survey or skill assessment administered that requests a Project ID, participants will be asked to bubble in the (1) last 4 digits of their Social Security Number and (2) the last 2 digits of the year they were born. Remind the participants that the Project ID is intended to keep their responses confidential, but allow us to link responses across different instruments to the same individual. We recommend reminding participants that we do not need to know who you are nor are we allowed to identify you as per our Institutional Review Board approval for the Project. Day 3 1) Tier I Problem ID Worksheet (i.e., Building Level Data Review Worksheet) a. Assesses educators’ skills with identifying problems at a Tier 1 level b. Response format = free response c. Suggested administration procedures i. Administer at slide entitled “Tier I Problem ID Worksheet” (the slide contains a reminder about the Project ID) which is after the slides that ask participants to answer multiple questions based on the data represented in three triangles. ii. Suggested context to provide 1. Say something like “For this last worksheet, we want to give you some additional practice as well as get a sense of how effectively we taught examining data on groups of students, just like we did at the conclusion of the last training. For this last worksheet, please complete individually and we will come by to collect them when you are finished.” 2) Hypothesis Evaluation Exercise a. Assesses the degree to which educators’ can identify good, questionable, and poor hypotheses as well as identify potential data sources to evaluate the validity of the hypotheses

School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

School-Based Leadership Team Training Survey & Skill Assessments Administration Guidelines for Day 3

Directions: The evaluation instruments you received for the School-Based Leadership Team (SBLT) trainings are organized by school and by instrument in mailing envelopes. The name of the school and instrument is included on each envelope. Each of the instruments is coded with a district and school ID number so it is important that you administer the instruments for a given school only to members of the SBLT from that school. To facilitate administration of these measures to the appropriate SBLT members, it is recommended that you ask that participants from the same school sit together. What follows are guidelines for administering evaluation tools during Day 3 of the SBLT trainings. Administration Guidelines For each survey or skill assessment administered that requests a Project ID, participants will be asked to bubble in the (1) last 4 digits of their Social Security Number and (2) the last 2 digits of the year they were born. Remind the participants that the Project ID is intended to keep their responses confidential, but allow us to link responses across different instruments to the same individual. We recommend reminding participants that we do not need to know who you are nor are we allowed to identify you as per our Institutional Review Board approval for the Project. Day 3

1) Tier I Problem ID Worksheet (i.e., Building Level Data Review Worksheet) a. Assesses educators’ skills with identifying problems at a Tier 1 level b. Response format = free response c. Suggested administration procedures

i. Administer at slide entitled “Tier I Problem ID Worksheet” (the slide contains a reminder about the Project ID) which is after the slides that ask participants to answer multiple questions based on the data represented in three triangles.

ii. Suggested context to provide 1. Say something like “For this last worksheet, we want to

give you some additional practice as well as get a sense of how effectively we taught examining data on groups of students, just like we did at the conclusion of the last training. For this last worksheet, please complete individually and we will come by to collect them when you are finished.”

2) Hypothesis Evaluation Exercise

a. Assesses the degree to which educators’ can identify good, questionable, and poor hypotheses as well as identify potential data sources to evaluate the validity of the hypotheses

Page 2: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

b. Response format = Combination of multiple choice and free response c. Suggested administration procedures

i. Administer at the slide entitled, “Hypothesis Evaluation Worksheet” (the slide contains a reminder about the Project ID) which is after several slides that ask participants to evaluate hypotheses and determine what data sources might be used to validate them.

ii. Suggested context to provide 1. Say something like “For this last worksheet, we again want

to give you some additional practice as well as get a sense of how effectively we taught hypothesis generation and identifying assessment sources. For this last worksheet, please complete individually and we will come by to collect them when you are finished.”

3) Problem Analysis Exercise

a. Assesses the degree to which teams can generate hypotheses, determine which types of data are appropriate to collect (i.e,. RIOT), and determine what specific data should be collected to confirm/reject the hypotheses

b. Response format = Free response c. Suggested administration procedures

i. Administer after the slide entitled “Reviewing Process of Problem Analysis.” Clark will add slides to model the generation and evaluation of hypotheses, and introduce the administration of the skill assessment.

ii. Suggested context to provide 1. Say something like “For this last worksheet, we once again

want to give you some additional practice as well as get a sense of how effectively we taught hypothesis generation and evaluation. For this last worksheet, please complete as a team (i.e., one per team) and we will come by to collect them when you are finished.”

4) Training Evaluation Survey

a. Assesses participants’ thoughts on the training b. Response format = scantron c. Remind participants that we want to make the trainings as helpful as

possible so we want your feedback on the structure and the process of the training we provided today.

Page 3: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

Problem Solving/Response to Intervention Training Evaluation Survey, Day 3

1

Training Evaluation Survey Please provide us feedback on the training session/workshop in which you have participated. Your feedback will be most useful in helping us develop or refine future training in problem-solving for school-based personnel. 1. Title of Training Workshop/Session: ______________________________________________ 2. Date: _________________________________________ 3. Your Job Title:

Teacher-General Education School Administrator

Teacher-Special Education School Support Personnel

District Administrator Directions: Based on your experience as a participant in this training workshop, please indicate your level of agreement or disagreement with each of the statements below using the response scale given. Please shade in the option that best represents your response.

= Strongly Disagree (SD) = Disagree (D) = Neutral (N) = Agree (A) = Strongly Agree (SA)

SD D N A SA

4. The content presented in the training workshop addressed the needs of my school/district.

5. The content presented was appropriate given my knowledge and skill level.

6. The amount of content covered was appropriate given the time allotted for the training.

7. The pacing of the training was appropriate.

8. The presenter(s) clearly articulated the major concepts and ideas that were the focus of the training.

9. The presenter(s) provided sufficient detail for me to understand important concepts about:

Page 4: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

Problem Solving/Response to Intervention Training Evaluation Survey, Day 3

2

SD D N A SA

a. Problem identification

b. Problem analysis

c. The role and purpose of assessment in problem analysis

d. The role of hypotheses and prediction statements in problem analysis

10. The presenter(s) adequately demonstrated how to use strategies that were the focus of the training.

11. The practice materials and exercises facilitated my understanding of the content presented.

12. The timeline for follow-up after this training is appropriate.

13. I am satisfied with the quality of materials provided me for this training.

14. As a result of this training I believe that I have gained a good understanding of how to:

a. Use data for problem identification

b. Develop (generate) hypotheses

c. Generate prediction statements

d. Conduct assessment to confirm/reject hypotheses

15. Identify those aspects of the training that you found most beneficial.

16. Identify those aspects of the training that you found least beneficial.

Page 5: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

Problem Solving/Response to Intervention Training Evaluation Survey, Day 3

3

17. Are there specific area(s) in the problem-solving process in which you believe you will need additional training in order to help you to more effectively meet the needs of ALL students as described in this project?

18. Overall, how satisfied were you with the quality of the training received in this workshop?

Very Dissatisfied Dissatisfied Somewhat Satisfied Satisfied Very Satisfied

19. Additional comments

Thank you!

Page 6: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback
Page 7: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

Pre/

Post

Bel

iefs

09/

06/0

7Ex

ampl

e El

emen

tary

1

1.52

2.53

3.54

4.55

7a7b

8a8b

11a

11b

1213

1415

1617

2022

23

Item

PRE

POST

Mean Score

Page 8: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback
Page 9: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

Perc

eptio

ns o

f Pra

ctic

es 0

9/06

/07

Exam

ple

Elem

enta

ry

1

1.52

2.53

3.54

4.55

2a2b

3a3b

4a4b

5a5b

8a8b

14a

14b

16a

16b

18a

18b

Item

Mean Score

School

District

Page 10: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback
Page 11: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

Exam

ple

Elem

enta

ry S

choo

l: D

irect

Ski

lls A

sses

smen

t

2.83

1.29

1.43

1.57

1.43

1.71

0

0.51

1.52

2.53

Cor

eef

fect

iven

ess

Impr

ovem

ents

toco

reR

espo

nse

tosu

pple

men

tal

inst

ruct

ion

Tier

3 re

ferr

alLi

tera

cy F

ailu

reTi

er e

ffect

iven

ess

Que

stio

ns

Score

P

ossi

ble

poin

ts

Page 12: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback
Page 13: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

Problem Solving/Response to Intervention Beliefs Survey, Day 3

1

Beliefs Survey Graph Items 7a: Core instruction should be effective enough to result in 80% of the students achieving benchmarks in reading 7b: Core instruction should be effective enough to result in 80% of the students achieving benchmarks in math 8a: The primary function of supplemental instruction is to ensure that students meet grade-level benchmarks in reading 8b: The primary function of supplemental instruction is to ensure that students meet grade-level benchmarks in math 11a: Students with high-incidence disabilities (e.g. SLD, EBD) who are receiving special education services are capable of achieving grade-level benchmarks (i.e., general education standards) in reading 11b: Students with high-incidence disabilities (e.g. SLD, EBD) who are receiving special education services are capable of achieving grade-level benchmarks (i.e., general education standards) in math 12: General education classroom teachers should implement more differentiated and flexible instructional practices to address the needs of a more diverse student body. 13. General education classroom teachers would be able to implement more differentiated and flexible interventions if they had additional staff support. 14. The use of additional interventions in the general education classroom would result in success for more students. 15. Prevention activities and early intervention strategies in schools would result in fewer referrals to problem-solving teams and placements in special education. 16. The “severity” of a student’s academic problem is determined not by how far behind the student is in terms of his/her academic performance but by how quickly the student responds to intervention. 17. The “severity” of a student’s behavioral problem is determined not by how inappropriate a student is in terms of his/her behavioral performance but by how quickly the student responds to intervention. 20. Using student-based data to determine intervention effectiveness is more accurate than using only “teacher judgment.” 22. Additional time and resources should be allocated first to students who are not reaching benchmarks (i.e., general education standards) before significant time and resources are directed to students who are at or above benchmarks. 23. Graphing student data makes it easier for one to make decisions about student performance and needed interventions.

Page 14: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback
Page 15: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

1

Practices Graph Items 2a. Data (e.g., Curriculum-Based Measurement, DIBELS, FCAT, Office Discipline Referrals) are used to determine the percent of students receiving core instruction (general education classroom only) who achieve benchmarks (district grade-level standards) in academics 2b. Data (e.g., Curriculum-Based Measurement, DIBELS, FCAT, Office Discipline Referrals) are used to determine the percent of students receiving core instruction (general education classroom only) who achieve benchmarks (district grade-level standards) in behavior 3a. Data are used to make decisions about necessary changes to the core curriculum or discipline procedures to increase the percent of students achieving benchmarks (district grade-level standards) in academics 3b. Data are used to make decisions about necessary changes to the core curriculum or discipline procedures to increase the percent of students achieving benchmarks (district grade-level standards) in behavior 4a. Data are used (e.g., Curriculum-Based Measurement, DIBELS, Office Discipline Referrals) to identify at-risk students in need of supplemental and/or intensive interventions for academics 4b. Data are used (e.g., Curriculum-Based Measurement, DIBELS, Office Discipline Referrals) to identify at-risk students in need of supplemental and/or intensive interventions for behavior 5a. The students identified as at-risk routinely receive additional (i.e., supplemental) intervention(s) for academics 5b. The students identified as at-risk routinely receive additional (i.e., supplemental) intervention(s) for behavior 8a. A standard protocol intervention (i.e., the same type of intervention used for similar problems) is used initially for all students who require supplemental instruction for academics 8b. A standard protocol intervention (i.e., the same type of intervention used for similar problems) is used initially for all students who require supplemental instruction for behavior 14a. The teacher of a student referred for problem-solving routinely receives staff support to implement the intervention plan developed by the Problem Solving Team for academics 14b. The teacher of a student referred for problem-solving routinely receives staff support to implement the intervention plan developed by the Problem Solving Team for behavior 16a. Data are graphed routinely to simplify interpretation of student performance for academics 16b. Data are graphed routinely to simplify interpretation of student performance for behavior 18a. A student’s response-to-intervention data (e.g., rate of improvement) are used routinely to determine whether a student is simply behind and can learn new skills or whether the student’s performance is due to a disability for academics 18b. A student’s response-to-intervention data (e.g., rate of improvement) are used routinely to determine whether a student is simply behind and can learn new skills or whether the student’s performance is due to a disability for behavior

Page 16: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback
Page 17: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

Problem Solving/Response to Intervention School Level Data Review, Day 3

1

Case Study Questions

1. Is the Core Instruction effective? Justify your decision.

2. Which group(s) of students is likely to improve the most with positive changes in core instruction? Justify your decision.

3. Which group(s) of students responded best to supplemental instruction? Justify your decision.

4. Who is most likely to be referred for Tier 3 interventions in this school setting? Justify your decision.

5. Which group of students is most at-risk for literacy failure in this building? Justify your decision.

6. What, in general, can you say about the effectiveness of the different instruction tiers in this building? Justify your decision.

Page 18: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback
Page 19: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

1

Tier One ProblemIdentification

Tier One ProblemIdentification

1. Rank from highest to lowest the groups and years for which core instructionis most effective. Be sure to include all 6 possibilities in your response.

2. Which group(s) of students should receive highest priority for monitoringwhile modifications to core instruction are being made? Justify your decision.

3. Which group(s) of students is most likely to be referred for additionalintervention—regardless of any label they might have? Justify your decision.

4. Based on the data from the previous two school years, for which of the threegroups of students depicted above, if any, will core instruction potentially beeffective at the end of this school year (i.e., 2007-08)? Justify your decision.

5. Assume that modifications were made between the 05/06 and 06/07 schoolyears for all groups of students at all levels of risk. Which group(s) ofstudents at what level(s) of risk made the greatest improvement across thetwo years? Justify your decision.

Page 20: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

2

Tier One ProblemIdentification

Tier One ProblemIdentification

1. Rank from highest to lowest the groups and years for which core instructionis most effective. Be sure to include all 6 possibilities in your response.

2. Which group(s) of students should receive highest priority for monitoringwhile modifications to core instruction are being made? Justify your decision.

3. Which group(s) of students is most likely to be referred for additionalintervention—regardless of any label they might have? Justify your decision.

4. Based on the data from the previous two school years, for which of the threegroups of students depicted above, if any, will core instruction potentially beeffective at the end of this school year (i.e., 2007-08)? Justify your decision.

5. Assume that modifications were made between the 05/06 and 06/07 schoolyears for all groups of students at all levels of risk. Which group(s) ofstudents at what level(s) of risk made the greatest improvement across thetwo years? Justify your decision.

Page 21: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

Problem Solving/Response to Intervention School Level Data Review, Day 3

1

School Level Data Review Worksheet

Case Study You are asked by your school principal to review school-level data and answer a number of questions for her. The data that are provided in the graphic below are DIBELS Oral Reading Fluency data and represent the % of students in the identified categories who scored within the low-, moderate-, and high-risk ranges. The first pair of charts represents all students in grades 1-5. The second pair of charts represents students on free-reduced lunch (i.e., Economically Disadvantaged) in grades 1-5. The final pair of charts represents students with disabilities (SWDs) in grades 1-5. Data from the end of the year DIBELS window (i.e., Spring) for the 2005-06 and 2006-07 school years are provided. After reviewing the data from the graphic below, please answer the questions that follow.

Your PS/RtI Project ID: Your PS/RtI Project ID was designed to assure confidentiality while also providing a method to match an individual’s responses across instruments. In the space provided (first row), please write in the last four digits of your Social Security Number and the last two digits of the year you were born. Then, shade in the corresponding circles.

Page 22: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback
Page 23: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

Problem Solving/Response to Intervention School Level Data Review, Day 3

3

Case Study Questions

1. Rank from highest to lowest the groups and years for which core instruction is most effective. Be

sure to include all 6 possibilities in your response.

2. Which group(s) of students should receive highest priority for monitoring while modifications to core instruction are being made? Justify your decision.

3. Which group(s) of students is most likely to be referred for additional intervention—regardless of any label they might have? Justify your decision.

4. Based on the data from the previous two school years, for which of the three groups of students depicted above, if any, will core instruction potentially be effective at the end of this school year (i.e., 2007-08)? Justify your decision.

5. Assume that modifications were made between the 05/06 and 06/07 school years for all groups of students at all levels of risk. Which group(s) of students at what level(s) of risk made the greatest improvement across the two years? Justify your decision.

Page 24: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback
Page 25: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

Problem Solving/Response to Intervention Hypothesis Skill Assessment, Day 3

1

Hypothesis Evaluation Exercise

Directions: Read each of the hypotheses below. Please use the following response scale:

A = Acceptable AM = Acceptable with modifications (Suggest a modification if you rate the

hypothesis as an “AM”.) U = Unacceptable

Remember that hypotheses should be observable, measurable, and lead to interventions. After you evaluate each hypothesis, mark the ONE method of data collection that you think would be most appropriate. ONLY check a method of data collection for an “A” or “AM” hypothesis. Tier 1 Hypotheses

1. ….because the children in this school come from poor homes. A AM U

Modification: Method of Data Collection (Mark only one):

Review Interview Observe Test

Your PS/RtI Project ID: Your PS/RtI Project ID was designed to assure confidentiality while also providing a method to match an individual’s responses across instruments. In the space provided (first row), please write in the last four digits of your Social Security Number and the last two digits of the year you were born. Then, shade in the corresponding circles.

Page 26: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

Problem Solving/Response to Intervention Hypothesis Skill Assessment, Day 3

2

2. …because the school does not post school rules using expectations for specific behaviors. A AM U

Modification: Method of Data Collection (Mark only one):

Review Interview Observe Test

3. …because the families do not speak English. A AM U

Modification: Method of Data Collection (Mark only one):

Review Interview Observe Test

Tier 3 Hypotheses

4. …because there are neurotransmitter deficits hindering his success in the classroom. A AM U

Modification: Method of Data Collection (Mark only one):

Review Interview Observe Test

Page 27: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

Problem Solving/Response to Intervention Hypothesis Skill Assessment, Day 3

3

5. …because peers reinforce his behaviors by laughing at him when he gets into trouble. A AM U

Modification: Method of Data Collection (Mark only one):

Review Interview Observe Test

6. …because he is highly impulsive. A AM U

Modification: Method of Data Collection (Mark only one):

Review Interview Observe Test

7. …because s/he loses materials necessary to engage in and complete assigned tasks. A AM U

Modification: Method of Data Collection (Mark only one):

Review Interview Observe Test

Page 28: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

Problem Solving/Response to Intervention Hypothesis Skill Assessment, Day 3

4

8. …because the teacher is tired and provides less positive reinforcement for appropriate behavior than at other times during the day.

A AM U Modification: Method of Data Collection (Mark only one):

Review Interview Observe Test

9. …because the parents do not provide reinforcement in the home setting for working independently.

A AM U Modification: Method of Data Collection (Mark only one):

Review Interview Observe Test

Thank you!

Page 29: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

Hypothesis Skill Assessment Scoring Rubric

Directions: The purpose of this skill assessment is to determine if participants can determine (1) the acceptability of hypotheses (i.e., alterable and measurable) and (2) the appropriate method of data collection to confirm/reject hypotheses. For each item, the participants are asked to determine if the identified hypothesis is unacceptable, acceptable with modifications, or acceptable. If acceptable with modifications is selected, the participants are asked to rewrite the hypothesis so it is acceptable. For any hypothesis rated as acceptable or acceptable with modifications, the participants are then asked to identify an appropriate source of data collection (i.e., RIOT procedures).

1. ….because the children in this school come from poor homes.

a. Hypothesis i. 0 points = Rates the hypothesis as acceptable or acceptable with

modifications. If rates as acceptable with modifications, rewritten hypothesis must not assess an alterable and measurable variable (e.g., because the children’s parents are farm workers)

ii. 1 point = Rates the hypothesis as unacceptable or acceptable with modifications. If rates as acceptable with modifications, rewritten hypothesis must assesses an alterable and measurable variable (e.g., because the children do not have access to books at home)

b. Data Collection i. 0 points = Selects test (unless modification is made that results in

test being an appropriate data collection method) ii. 1 point = Selects review, interview, or observe, or leaves the item

blank if unacceptable rating given to hypothesis

2. …because the school does not post school rules using expectations for specific behaviors.

a. Hypothesis i. 0 points = Rates the hypothesis as unacceptable or acceptable with

modifications. If rates as acceptable with modifications, rewritten hypothesis must not assess an alterable and measurable variable (e.g., because the school rules are no good)

ii. 1 point = Rates the hypothesis as acceptable or acceptable with modifications. If rates as acceptable with modifications, rewritten hypothesis must assess an alterable and measurable variable (e.g., because the school rules are not posted throughout the school using expectations for specific behaviors)

b. Data Collection i. 0 points = Selects test (unless modification is made that results in

test being an appropriate data collection method) or leaves the item blank

ii. 1 point = Selects review, interview, or observe

Page 30: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

3. …because the families do not speak English.

a. Hypothesis i. 0 points = Rates the hypothesis as acceptable or acceptable with

modifications. If rates as acceptable with modifications, provides a rewritten hypothesis that does not assess an alterable and measurable variable (e.g., because the families are migrant workers)

ii. 1 point = Rates the hypothesis as unacceptable or acceptable with modifications. If selects acceptable with modifications, provides a rewritten hypothesis that assesses an alterable and measurable variable (e.g., because the families do not have the English language skills to help with school-work that is in English)

b. Data Collection i. 0 points = Selects test (unless modification is made that results in

test being an appropriate data collection method) ii. 1 point = Selects review, interview, or observe, or leaves the item

blank if rates hypothesis as unacceptable

4. …because there are neurotransmitter deficits hindering his success in the classroom.

a. Hypothesis i. 0 points = Rates the hypothesis as acceptable or acceptable with

modifications. If rates as acceptable with modifications, provides a rewritten hypothesis that does not assess an alterable and measurable variable (e.g., because he has cognitive processing deficits that require differentiated instruction)

ii. 1 point = Rates the hypothesis as unacceptable or acceptable with modifications. If selects acceptable with modifications, provides a rewritten hypothesis that assesses an alterable and measurable variable (e.g., because he has organizational skill deficits that inhibit him from completing assignments accurately)

b. Data Collection i. 0 points = Selects review, interview, observe, or test (unless

modification results in review, interview, observe, or test being an appropriate data collection method)

ii. 1 point = Leaves the item blank if hypothesis given unacceptable rating, or selects review, interview, observe, or test if modification provided for which the selected RIOT data collection methods is appropriate.

5. …because peers reinforce his behaviors by laughing at him when he gets into

trouble. a. Hypothesis

i. 0 points = Rates the hypothesis as unacceptable or acceptable with modifications. If rates as acceptable with modifications, the

Page 31: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

rewritten hypothesis provided must not assess an alterable or measurable variable (e.g., because he likes peer attention)

ii. 1 point = Rates the hypothesis as acceptable or acceptable with modifications. If rates as acceptable with modifications, a rewritten hypothesis that assesses an alterable and measurable variable must be provided (e.g., because peers reinforce his behaviors by laughing at him after the teacher redirects him to complete a task)

b. Data Collection i. 0 points = Selects test (unless modification is made that results in

test being an appropriate data collection method) or leaves the item blank

ii. 1 point = Selects review, interview, or observe

6. …because he is highly impulsive. a. Hypothesis

i. 0 points = Rates the hypothesis as acceptable or acceptable with modifications. If selects acceptable with modifications, the rewritten hypothesis must not assess an alterable and measurable variable (e.g., because he cannot control himself)

ii. 1 point = Rates the hypothesis as unacceptable or acceptable with modifications. If selects acceptable with modifications, the rewritten hypothesis must assess an alterable and measurable variable (e.g., because he does not possess the private speech to select an alternative to the problem behavior)

b. Data Collection i. 0 points = Selects test (unless modification is made that results in

test being an appropriate data collection method) ii. 1 point = Selects review, interview, or observe, or leaves blank if

unacceptable rating given to hypothesis

7. …because s/he loses materials necessary to engage in and complete assigned tasks.

a. Hypothesis i. 0 points = Rates the hypothesis as unacceptable or acceptable with

modifications. If rates as acceptable with modifications, rewritten hypothesis must not assess an alterable and measurable variable (e.g., because she doesn’t have the cognitive ability to complete assigned tasks)

ii. 1 point = Rates the hypothesis as acceptable or acceptable with modifications. If rates as acceptable with modifications, rewritten hypothesis must assess an alterable and measurable variable (e.g., because s/he does not posses the skills to organize and manage the materials necessary to engage in and complete assigned tasks)

b. Data Collection

Page 32: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

i. 0 points = Selects test (unless modification is made that results in test being an appropriate data collection method) or leaves the item blank

ii. 1 point = Selects review, interview, or observe

8. …because the teacher is tired and provides less positive reinforcement for appropriate behavior than at other times during the day.

a. Hypothesis i. 0 points = Rates the hypothesis as acceptable or unacceptable.

Acceptable with modifications may also be scored as 0 if the rewritten hypothesis does not assess an alterable and measurable variable (e.g., the teacher is old school and likes to yell at the kids)

ii. 1 point = Rates the hypothesis as acceptable with modifications and provides a rewritten hypothesis that assesses an alterable and measurable variable (e.g., the teacher provides less positive reinforcement for appropriate behavior in the afternoon than in the morning)

b. Data Collection i. 0 points = Selects review or test (unless modification is made that

results in review or test being an appropriate data collection method), or leaves the item blank

ii. 1 point = Selects interview or observe

9. …because the parents do not provide reinforcement in the home setting for working independently.

a. Hypothesis i. 0 points = Rates the hypothesis as unacceptable or acceptable with

modifications. If rates as acceptable with modifications, rewritten hypothesis must not assess an alterable and measurable variable (e.g., because the parents coddle him/her)

ii. 1 point = Rates the hypothesis as acceptable or acceptable with modifications. If rates as acceptable with modifications, rewritten hypothesis must assess an alterable and measurable variable (e.g., the parents provide no opportunities for independent work in the home setting)

b. Data Collection i. 0 points = Selects review or test (unless modification is made that

results in review or test being an appropriate data collection method), or leaves the item blank

ii. 1 point = Selects interview or observe

Page 33: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

Problem Solving/Response to Intervention Problem Analysis Skill Assessment, Day 3

1

Problem Analysis Exercise

Directions: Please read each of the scenarios below. For each one, please complete the following:

1. Write one hypothesis for each of the Domains listed. 2. Identify the method that you would use to collect data to confirm or reject the hypothesis

(RIOT — R=Records, I=Interview, O=Observation, T=Test). 3. Identify the specific type of data that you would collect using your method (e.g., observe on-task

behavior, interview the teacher to find out specific information, test fluency skills, review work samples to find error patterns).

Tier 1 Example The school-based leadership team is concerned by the number of students who are referred to the office each month for discipline problems. Too many students are missing valuable instruction time and the team is concerned that the academic performance of these students will be adversely affected by the loss of academic engaged time. A review of the Office Discipline Referral (ODR) data indicate the following: 42% of all students referred to the office are referred for noncompliance. The team has identified one replacement behavior: improve compliance (reduce noncompliance referrals by 50%).

Your PS/RtI Project ID: Your PS/RtI Project ID was designed to assure confidentiality while also providing a method to match an individual’s responses across instruments. In the space provided (first row), please write in the last four digits of your Social Security Number and the last two digits of the year you were born. Then, shade in the corresponding circles.

Page 34: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

Problem Solving/Response to Intervention Problem Analysis Skill Assessment, Day 3

2

Identify one hypothesis in each of the ICEL (Instruction, Child, Environment, Learner) domains, identify the assessment method and identify specific assessment data that you will collect using that method. Hypothesis: “Students in this school are unable to demonstrate high rates of compliance because…” INSTRUCTION: Method of Assessment: R I O T Specific Data to be Collected: CURRICULUM: Method of Assessment: R I O T Specific Data to be Collected: ENVIRONMENT: Method of Assessment: R I O T Specific Data to be Collected: LEARNER: Method of Assessment: R I O T Specific Data to be Collected:

Page 35: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

Problem Solving/Response to Intervention Problem Analysis Skill Assessment, Day 3

3

Tier 3 Example Cameron rarely turns in his math assignments on time. His computation is generally accurate on the 3-minute CBM probes Mr. Jeffers administers once each week. Cameron doesn’t seem to have problems with assignments in other content areas. Lately, Cameron has been talking out and getting out of his seat more than Mr. Jeffers is willing to tolerate. The replacement behavior identified by Mr. Jeffers was: Cameron will turn in 95% of his math assignments on time. Identify one hypothesis in each of the ICEL domains, identify the assessment method and identify specific assessment data that you will collect using that method. Hypothesis: “Cameron is unable to complete math assignments 95% of the time because…” INSTRUCTION: Method of Assessment: R I O T Specific Data to be Collected: CURRICULUM: Method of Assessment: R I O T Specific Data to be Collected: ENVIRONMENT: Method of Assessment: R I O T Specific Data to be Collected:

Page 36: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

Problem Solving/Response to Intervention Problem Analysis Skill Assessment, Day 3

4

LEARNER: Method of Assessment: R I O T Specific Data to be Collected:

Thank you!

Page 37: School-Based Leadership Team Training Survey & Skill ...Problem Solving/Response to Intervention Training Evaluation Survey, Day 3 1 Training Evaluation Survey Please provide us feedback

Problem Analysis Skill Assessment Directions: Use the following guidelines when scoring hypotheses for the Tier I and Tier III Examples. Hypothesis

a. 0 points = Hypothesis provided does not assess an alterable and measurable variable in the identified domain

b. 1 point = Hypothesis provided assesses an alterable and measurable variables in the identified domain

Data Collection Method

a. 0 points = The method(s) of data collection selected under the RIOT rubric would not allow educators to confirm or reject the hypothesis

b. 1 point = The method(s) of data collection selected under the RIOT rubric would allow educators to confirm or reject the hypothesis

Specific Types(s) of Data

a. 0 points = The specific type(s) of data selected do not match the RIOT data collection method or would not allow educators to confirm or reject the hypothesis

b. 1 point = The specific type(s) of data selected match the RIOT data collection method and would allow educators to confirm or reject the hypothesis