National Center on Response to Intervention OSPI & National Center on Response to Intervention November 1, 2011 Implementer Series Module 3: Multi-level

National Center on Response to Intervention

OSPI & National Center on Response to Intervention

November 1, 2011

Implementer SeriesModule 3: Multi-level Prevention

System


Session Agenda Welcome and Introductions Review What Is Multi-level Instruction? Selecting Evidence-Based Practices Wrap-Up: Review, Questions, and Resources

2


Upon Completion Participants Will Be Able To: Use screening and progress monitoring data

to make decisions at all levels of the multi-level prevention system, including movement between levels.

Develop a multi-level prevention system. Select evidence-based interventions and

practices.

3


REVIEW: SCREENING AND PROGRESS MONITORING

4

National Center on Response to Intervention 5

Response to intervention (RTI) integrates assessment and intervention within a schoolwide, multi level prevention system to ‑maximize student achievement and reduce behavior problems.

Defining RTI

(NCRTI)


With RTI, schools identify students at risk for poor learning outcomes, monitor student progress, provide evidence-based interventions and adjust the intensity and nature of those interventions based on a student’s responsiveness; and

RTI may be used as part of the determination process for identifying students with specific learning disabilities or other disabilities.

Defining RTI

(NCRTI)


RTI as a Preventive Framework RTI is a multi-level instructional framework

aimed at improving outcomes for ALL students. RTI is preventive and provides immediate

support to students who are at risk for poor learning outcomes.

RTI may be a component of a comprehensive evaluation for students with learning disabilities.

7


Essential Components of RTI Screening Progress Monitoring Schoolwide, Multi-level Prevention System

• Primary Level• Secondary Level• Tertiary Level

Data-Based Decision Making for• Instruction • Evaluating Effectiveness• Movement within the multi-level system• Disability identification (in accordance with state law)

8


Essential Components of RTI

9


Screening PURPOSE: Identify students who are at risk for poor

learning outcomes FOCUS: ALL students TOOLS: Brief assessments that are valid, reliable, and

demonstrate diagnostic accuracy for predicting learning or behavioral problems

TIMEFRAME: Administered more than one time per year (e.g., fall, winter, spring)

10


Progress Monitoring PURPOSE: Monitor students’ response to primary,

secondary, or tertiary instruction in order to estimate rates of improvement, identify students who are not demonstrating adequate progress, and compare the efficacy of different forms of instruction

FOCUS: Students identified through screening as at risk for poor learning outcomes

TOOLS: Brief assessments that are valid, reliable, and evidence based

TIMEFRAME: Students are assessed at regular intervals (e.g., weekly, biweekly, or monthly)

11


Data-Based Decision Making Analyze data at all levels of RTI implementation (e.g.,

state, district, school, grade level) and all levels of prevention (e.g., primary, secondary, or tertiary).

Establish routines and procedures for making decisions. Set explicit decision rules for assessing student progress

(e.g., state and district benchmarks, level and/or rate). Use data to compare and contrast the adequacy of the

core curriculum and the effectiveness of different instructional and behavioral strategies.

12


Review Activity What is the difference between a mastery measure and a

general outcome measure? T or F: All progress monitoring tools are created equal. Where can I find evidence of the reliability and the validity of

progress monitoring tools? Name three uses of progress monitoring data. What is a trend line? What are three ways to establish progress monitoring goals? Describe two ways to analyze progress monitoring data.

13


WHAT IS A MULTI-LEVEL PREVENTION SYSTEM?

14


Essential Components of RTI

15


Levels, Tiers, and Interventions

FRAMEWORK:3 levels of intensity• Primary• Secondary (Targeted/Strategic) • Tertiary (Intensive)

Secondary Level of

Prevention(~15% of students)

Tertiary Level of

Prevention(~ 5 % of students)

Primary Level of Prevention

(~80% of students)

16



MODEL:Minimum of 3 tiers representing each level of intensity

Tier II (secondary)

Tier I (primary)

17

Tier III (tertiary)



Interventions are provided at each level and within each tier.

Secondary Level of Prevention

Tertiary Level of Prevention

Primary Level of Prevention

18


NCRTI recommends different evidence standards across intervention levels.

Research-based curricula• Recommended for

primary prevention across subjects.

• Components have been researched and found to be generally effective.

• Curriculum has not been rigorously evaluated.

Evidence-based intervention • Recommended for

secondary and tertiary prevention

• Evaluated using rigorous research design

• Evidence of positive effects for students who received the intervention

19


PRIMARY PREVENTION LEVEL

20


FOCUS: ALL students INSTRUCTION: District curriculum and instructional

practices that are research based; aligned with state or district standards; and incorporate differentiated instruction

SETTING: General education environment ASSESSMENTS: Screening, continuous progress

monitoring, and outcome measures or summative assessments

21


Primary Prevention Focus ALL students Includes students with disabilities, learning

differences, or language barriers Increase access through

• Differentiated instruction.• Practices that are linguistically and culturally responsive.• Accommodations.• Modifications.

22


Primary Level Instruction Research-based curriculum materials for students

(including sub groups) Implementation fidelity Articulation of teaching and learning within and

across grades Differentiation of instruction based on data Ongoing professional development

See NCRTI Integrity Rubric

23


What Is Core Curriculum in RTI? Grade level standards Usually mandatory for all students of a school or a

school system Often instituted at the elementary and secondary

levels by local school boards, departments of education, or other administrative agencies charged with overseeing education

May or may not include instructional materials

24


What Are Differentiated Learning Activities? Offers students in the same class different teaching

and learning strategies based on• Student assessment data and knowledge of student

readiness

• Learning preferences,

• Language and culture

25


What Are Differentiated Learning Activities? Involves

• Mixed instructional groupings,

• Team teaching,

• Peer tutoring,

• Learning centers, and

• Accommodations to ensure that all students have access to the instructional program

Is NOT the same as providing more intensive interventions to students with low achievement or learning disabilities

26


Primary Prevention Setting General education environment Various grouping strategies

• Whole class • Cooperative learning groups

27


Primary Prevention Assessment Universal screening to determine students’ current

level of performance Continuous progress monitoring to confirm risk

status and monitor progress of at-risk students Outcome measures or summative assessments for

accountability

28


Progress Monitoring and Screening Data Within Primary Prevention Screening data

• Identify students who need additional assessment or instruction.

• Evaluate the effectiveness of primary prevention for all students.

Progress monitoring data• Confirm and disconfirm risk.

29


Screening: Identify Students Who Need Additional Assessment and Instruction

10

60

50

30

40

20

70

Fall

Above average

Average

Below average

Student

Benchmark Scores for Grade 2 Screening Measure

Sco

re

30


Screening: Evaluate Effectiveness of Primary Prevention

20

120

100

60

80

40

140

Fall SpringWinter

Target score

General population.

Title ISpecial educationS

core

31


Progress Monitoring: Confirming Risk Status

0

5

10

15

20

25

1 2 3 4 5 6 7 8 9 10 11 12 13 14

Weeks of Instruction

Dig

its

Co

rrec

t in

3 M

inu

tes

X

X

32


0

5

10

15

20

25

1 2 3 4 5 6 7 8 9 10 11 12 13 14


Pro

ble

ms

Co

rrec

t in

3 M

inu

tes

XX

Progress Monitoring: Confirming Risk Status

33


Team Activity: Primary Prevention Complete Primary prevention (column 2) in

Appendix C “Developing an RTI Model”:• Tier(s)• Focus• Instruction • Setting• Assessment• Data-based decision making• Other

34


SECONDARY PREVENTION LEVEL

35


FOCUS: Students identified through screening as at risk for poor learning outcomes

INSTRUCTION: Targeted, supplemental instruction delivered to small groups

SETTING: General education classroom or other general education location within the school

ASSESSMENTS: Progress monitoring, diagnostic

36


Secondary Prevention Focus Students identified through screening as at risk for

poor learning outcomes Typically 15% of entire population

37


Secondary Level Instruction Evidence based Aligns with and supports core instruction Implementation fidelity based on developer

guidelines. Delivered by well-trained staff in optimal group sizes Decisions are based on valid and reliable data and

criteria are implemented accurately. Supplements core instruction


38


Secondary Prevention Setting General education classroom or similar setting Adult-led instruction Small group rather than whole class

39


Secondary Prevention Assessment Decisions about responsiveness to intervention

• Are based on reliable and valid progress monitoring data.

• Reflect judgment based on the slope of improvement or final status at the end of the intervention period.

Decision-making rules are applied accurately and consistently

40


Secondary Prevention Assessment Progress monitoring

• Monitor student response to secondary instruction.• Evaluate the efficacy of the secondary system.• Conduct at least monthly.

Diagnostic assessment• Match students needs to interventions.

41


Secondary Prevention Goal Setting End-of-year benchmarking National norms for weekly rate of improvement

(slope)

42


Progress Monitoring Data Within Secondary Prevention Progress monitoring data

• Determine response to secondary interventions using The four-point rule. Trend-line analysis.

• Compare efficacy of secondary interventions .

43


Progress Monitoring: Determining Response Using the Four-Point Rule

44

Goal line


Progress Monitoring: Determining Response Using Trend Line Analysis

X

Goal line

Trend line

45


0

10

20

30

40

50

Week 1 Week 2 Week 3 Week 4

Intervention A

Intervention B

Intervention C

Growth by Intervention Type

Progress Monitoring: Compare Efficacy of Secondary Interventions

46

Wor

ds R

ead

Cor

rect

ly


Progress Monitoring: Evaluate Efficacy of Secondary System Data should indicate the following:

• 70%-80% of students in secondary interventions demonstrating adequate progress

• Implementation fidelity for interventions and data-based decision rules

47


Team Activity: Secondary Prevention Complete secondary prevention (column 3) in

Appendix C “Developing an RTI Model”:• Tier(s)• Focus• Instruction • Setting• Assessment• Data-based decision making• Other

48


TERTIARY PREVENTION LEVEL

49


Tertiary Prevention Level FOCUS: Students who have not responded to

primary or secondary level prevention INSTRUCTION: Intensive, supplemental instruction

delivered to small groups or individually SETTING: General education classroom or other

general education location within the school ASSESSMENTS: Progress monitoring, diagnostic

50


Tertiary Prevention Focus Students who have not responded to primary or

secondary level prevention Typically 3%-5% of the entire population

51


Tertiary Level Instruction Evidence based or based on validated progress

monitoring methods for individualizing instruction More intense than secondary Implementation fidelity Delivered by well-trained staff in optimal group sizes Decisions are based on valid and reliable data, and

criteria are implemented accurately. Address general education curriculum in appropriate

manner for students.


52


Tertiary Prevention Setting General education setting or other appropriate

setting Decisions are made on a case-by-case basis. Optimal group size is chosen for ages and needs of

students.

53


Tertiary Prevention Assessment Decisions about responsiveness to intervention

• Are based on reliable and valid progress monitoring data.

• Reflect judgment based on the slope of improvement or final status at the end of the intervention period.

Decision-making rules are applied accurately.

54


Tertiary Prevention Assessment Progress monitoring

• Frequent progress monitoring (at least bi-weekly) is recommended.

• Regularly monitor based on established learning trajectories indicated by the goal line.

Diagnostic• Match instruction to needs.• Inform individualized instructional planning.

55


Tertiary Prevention: Goal SettingThree goal-setting options: End-of-year benchmarking National norms for weekly rate of improvement

(slope) Intra-individual

56


Progress Monitoring Data Within Tertiary Prevention Progress monitoring data

• Determine response to secondary interventions using The four-point rule. Trend line analysis. Trend line analysis and slope.

• Compare efficacy of tertiary interventions.

57


Progress Monitoring: Determining Response Using Four-Point Rule

58

Goal line


Progress Monitoring: Determining Response Using Trend Line Analysis

59

02468

101214161820

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20


Wor

ds

Rea

d C

orr

ectl

y

X XGoal line

Trend line


Progress Monitoring: Determining Response Using Trend Line Analysis and Slope

60

XStudent’s new goal and slope:

(28 – 6) ÷ 11 = 2.0 slope

02468

1012141618202224262830

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20


Wo

rds

Rea

d C

orr

ectl

y

X

Goal line

Trend line


Progress Monitoring: Evaluate Efficacy of Tertiary System Data should indicate the following:

• Majority of students in tertiary prevention are demonstrating adequate progress (e.g., eventually move from tertiary to secondary)

• Implementation fidelity for interventions and data decision rules

61


Team Activity: Tertiary Prevention Complete tertiary prevention (column 4) Appendix C

“Developing an RTI Model”: • Tier(s)• Focus• Instruction • Setting• Assessment• Data-based decision making• Other

62


Changing the Intensity and Nature of Instruction Intervention Duration Frequency Interventionist Group size

63


IDEA AND MULTI-LEVEL PREVENTION SYSTEM

64


What About Special Education?

Two groups to consider: Students with disabilities who are currently receiving

special education Students being referred for special education eligibility

consideration

65


What About Students with Disabilities?

66

~15%

~5% Tier III (tertiary)

Specialized individualizedsystems for students with

intensive needs

Tier II (secondary) Supplemental group

systems for students with at-risk response to

primary level

Tier I (primary)Schoolwide

instruction forall Students, including

differentiated instruction

~80%


Disability IdentificationTo ensure that underachievement in a child suspected of having a specific learning disability is not due to a lack of appropriate instruction in reading or math, the group must consider, as part of the evaluation, what is described in 34 CFR 300.304 through 300.306:Data demonstrate that prior to, or as a part of, the referral process, the child was provided appropriate instruction in regular education settings, delivered by qualified personnel.Data-based documentation of repeated assessments of achievement at reasonable intervals reflect formal assessment of student progress during instruction, which was provided to the child’s parents.

67

(www.idea.ed.gov)


When Do We Refer to Special Education?

~15%

~5% Tier III (tertiary)

Tier II (secondary)

Tier I (primary)

~80% of students

Example 1: After nonresponsiveness to two secondary interventions


When Do We Refer to Special Education?

~15%

~5%

Tier III (tertiary)

Tier II (secondary)

Tier I (primary)

~80% of Students

Example 2: After nonresponsiveness to one secondary and one tertiary intervention


Recommendations Collaborate with special education to develop an

inclusive multi-level instruction model Create written guidance on the following:

• How do special education services “fit” in the model?• What are the roles and responsibilities of special

education staff?• When should students be referred for eligibility

consideration?

70


Prereferral Model Versus RTI

71


SELECTING EVIDENCE-BASED PRACTICES

72


Selecting Evidence Based Practices: Definitions

73

Evidence-based intervention • Data demonstrates (or empirically validates) efficacy

through scientific, rigorous research designs. Research-based curricula

• May incorporate design features that have been researched generally.

• The curriculum or program as a whole has not been studied using a rigorous research design. See Training

Manual Page 9


Selecting Evidence-Based Practices1. Identify needs and priorities.2. Select practices to address needs.

• Resources for identifying evidence-based practices3. Evaluate evidence claims.

• Standards of evidence across intervention levels/tiers within RTI

4. Implement practices.5. Evaluate effectiveness. See Training

Manual Page 9

74


1) Identifying Needs and Priorities

1. Gather a team.2. Conduct a “needs assessment.”

• Gather information from multiple sources.• Compile and summarize data.

3. Determine priorities. See Training

Manual Page 9 and Instructional

Intervention Tools Chart

User’s Guide

75


1) Identifying Needs and Priorities Evaluate existing data to determine baseline

performance on indicators of interest.• Academic achievement• Discipline referrals• Attendance/truancy data

76


1) Identifying Needs: Interventions For what skills do we need a secondary intervention

instructional program? Is there a specific academic outcome or measure we are interested in providing supplemental instruction for?

For what grades do we need an instructional program? Will this program be used with all students who are not

progressing in the core curriculum or only with specific sub groups of students? Tier I? Tier II? Tier III?

Which sub groups? English language learners (ELLs)? Special education?

77

See Instructional Intervention Tools Chart

User’s Guide


1) Identifying Priorities: Interventions Is it a program that can be purchased for a reasonable cost? Is it a program with a reasonable implementation time? Is it a program that does not require specialized expertise or

lengthy training to administer? Is it a program that offers ready access to training and technical

support for staff? Is it a program that has documented evidence of efficacy

through the most rigorous research? Is it a program whose effectiveness has been studied and

demonstrated in our district or state?

78

See Instructional Intervention Tools

Chart User’s Guide





4. Implement practices.5. Evaluate effectiveness.

79


2) Selecting Evidence-Based Practices Screening and progress monitoring Data-based decision making Instruction/interventions

80


2) Selecting Evidence-Based Practices: Screening and Progress Monitoring NCRTI: Screening Tools Chart NCRTI: Progress Monitoring Tools Chart IES practice guide: RTI for mathematics and reading IRIS Center webinars and RTI/assessment training

modules

81


2) Selecting Evidence-Based Practices: Data-Based Decision Making

IES Practice Guide: Using Student Achievement Data

U.S. Department of Education: Doing What Works and the American Recovery and Reinvestment Act

IRIS Center Case Study Unit RTI: Data-based Decision Making

82


2) Selecting Evidence-Based Practices: Instruction/Intervention What Works Clearinghouse

Best Evidence Encyclopedia

NCRTI Instructional Intervention Tool Chart

Florida Center for Reading Research

IES Practice Guide: RTI for Mathematics and Reading

IRIS Center Learning Strategies Modules

83






84


3) Evaluating Evidence Where can I find evidence? What type of evidence exists? What is the quality of the evidence? What were the desired outcomes? What are the effects of the intervention? Is the sample population similar?

85

See Training Manual Page

10


3a) Evaluating Evidence: Where? Curriculum websites (use with caution) Peer-reviewed journals (various)

• ERIC (www.eric.ed.gov/)• Google Scholar (www.scholar.google.com/)• Education Abstracts (Education Full Text) • Psychological Abstracts

See Training Manual Page 10

86


3a) Evaluating Evidence: Where? What Works Clearinghouse Best Evidence Encyclopedia NCRTI Instructional Intervention Tools Chart Florida Center for Reading Research

87

See Training Manual

Appendix B


3b) Evaluating Evidence: Type? What type of evidence is available?

• Research study• Summary of existing research• Technical report• Peer/publisher claims• Other methods?


11

88


3b) Evaluating Evidence: Type?

89

Randomized control trial* Quasi-experiment Single-case design Quantitative research synthesis

* Most rigorous form of research See Training Manual Page

11


3c) Evaluating Evidence: Quality? How was the program implemented? Is it realistic under

normal circumstances?• Who implemented the intervention?• Was the intervention described?• Was there a manual or a script?• How often did the intervention occur?• Was fidelity evaluated in the study? If so, how?• What phase of research? (exploratory, pilot, efficacy, scale-

up)See Training

Manual Page 11

90


3d) Evaluating Evidence: Desired Outcomes? Were the outcomes assessed relevant to your outcomes?

• What outcome measures were used to evaluate the intervention?

• Do the outcome measures seem reasonable?

• Are they relevant to your concerns?

• Are they reliable and valid?

• Is it feasible to implement with your

population?See Training Manual Page

12

91


3e) Evaluating Evidence: Effects? Are the effects large enough to be meaningful?• Significance of differences • Interpretation of effect sizes

92

Practitioners can use this information to compare programs and identify those most likely to meet their specific needs.


12


3f) Evaluating Evidence: Population? For which population does the evidence show an

effect?• Is the sample described? • Can you tell who was studied?• Is the participant similar to or representative of your

student population?• Are there different effects for different population groups?

See Training Manual Page 12

93


Demonstration What Works Clearinghouse Best Evidence Encyclopedia NCRTI Instructional Intervention Tools Chart

94


Click here to select a topic area

What Works Clearinghouse


You can use search terms to search for interventions here

You can also click on the Find what works link for a more extensive search feature


96


What Works ClearinghouseWhat outcome are you focused on?

What grade level are you looking for?

Is there a specific focus or population?



98

Click here to create a report

What level of effectiveness are you looking for?

What is the extent of evidence you are looking for?

How will the intervention be delivered?

Are you looking for a curriculum, a supplemental intervention, or a practice?


•Use the filter to select your interest

•Look at the results provided for the student outcome, the improvement index, the effectiveness rating, and the extent of evidence.

•Click the link to learn more about the intervention and research


Evaluating Evidence

100

Click here to learn more about the interventions


Evaluating Evidence

101


Evaluating EvidenceAccelerated Reader™ (Reviewed for Beginning Reading)

102


Evaluating Evidence

Accelerated Reader™ (Reviewed for Beginning Reading)


Evaluating Evidence

104

Accelerated Reader™ (Reviewed for Beginning Reading)


Evaluating Evidence

105


Program Information

106


Team Activity: What Works Clearinghouse

1. What effective supplemental (Tier II/III), small group, and reading comprehension programs are available for eighth-grade students?

2. How many programs show a positive improvement index for mathematics achievement?

3. Identify a program used in your district/school• What is the improvement index, the evidence rating, and the

extent of evidence?

• What additional information is available about the evidence in the program report?

107



Click to select the topic:• subject matter• grade level• focus area (e.g., ELLs, struggling readers)


Finding an Evidence-Based ProgramFind programs that show high or moderate levels of effectiveness

Find programs that show limited evidence of effectiveness

Find programs with insufficient evidence or no qualifying studies

See the key findings, the summary and, the methods



110


Reviewcriteria

Studies are rated based on the effect size, the type of study, and the size of study


Selecting an

Evidence-Based

Program


Evaluating Evidence

113


Determine the Type of Program

114

The type of program is defined as


Learn About the Program

115

The program name, a brief description, and contact information for more information are provided.


Team Activity: Best Evidence Encyclopedia1. What programs show strong or moderate levels of

effectiveness for eighth-grade reading?2. Were any small group tutorials (SGT in Column 3)

shown to have a strong level of effectiveness for struggling readers?

3. Identify a program used in your district/school:• What level of evidence is reported for the program?

116


www.rti4success.org

NCRTI Instructional Intervention Tools Chart


NCRTI Instructional Interventions Tools Chart

118

The tools chart lists instructional programs that can be used as secondary interventions within an RTI context.

The technical review committee (TRC) does not rate instructional programs; instead the TRC rates studies of program efficacy.


Instructional Intervention Tools Chart

NCRTI definition of instruction: Additional or alternative intervention programs to

the core curriculum conducted in small groups or individually with evidence of efficacy for improving academic outcomes for students whose performance is unsatisfactory in the core program.

119


Instructional Intervention Tools Chart Purpose: help consumers identify secondary

programs that• Have been evaluated through rigorous design.• Have shown positive, meaningful treatment effects.

120


Search by Content: Reading, Math, or Writing

Search by Grade: Elementary or Secondary


Comparing ToolsNarrow search selection to programs that fit your needs


Comparing Tools

Compare them side by side


Instructional Intervention Tools Chart

Click on the name of the program to see the implementation requirements.


Implementation Requirements


Technical Rigor of the Study

126

Study Quality


Participants

127


Design

128


Fidelity of Implementation

129


Measures

130



131

Effect Size

Adjusted posttest: corrects for differences in groups on pretest



132

Effect Size

Unadjusted posttest: does not account for differences in groups on pretest


Effect Size: Sample 1

133

Developer was unable to provide necessary data for NCRTI to calculate effect sizes.


Effect Size: Sample 2

134

Mean for proximal and distal measures


Effect Sizes

135


Disaggregated Data

The column reports effect size data that have been disaggregated for sub groups, if available.

• Students with disabilities• ELLs• Students from diverse racial-ethnic groups

136


Team Activity: Instructional Intervention Tools Chart1. What secondary instruction programs are available

for elementary mathematics students?2. Which study has the highest average adjusted effect

size? 3. What are the average unadjusted effect sizes for

SSRD Writing Strategies?4. Identify a program used in your district/school

• What level of evidence is reported for the program?

137


Team Activity Appendix D: Selecting Evidence-Based Practices As a table group, choose one of the following

examples Think of intervention materials you have or may

considering adopting to support the student Using the three websites, investigate the evidence

for that program (or programs) We’ll reconvene in 20 minutes to report decisions

138


Appendix D, Example 1: Bell Top Elementary School

Ms. Jones, a third-grade math teacher, has noticed that Jack, Bobby, and Jane seem to be struggling with word problems despite the strong research-based core curriculum. In looking at their scores on the screening measure, Ms. Jones noticed that they are all below the cut score. After monitoring their progress over a number of weeks she continued to see no progress. Can you identify an intervention that might help Jack, Bobby, and Jane improve? Why did you select that intervention?

139


Appendix D, Example 2: Lake Ridge Middle School

Mr. Morris has identified that Jessica does not have a strong grasp on the basic skills of reading and is therefore struggling in his class. After looking at her screening data and monitoring her progress, he has decided to implement an intervention to help her with basic reading skills. Is there an intervention available that Mr. Morris might use to help Jessica? What questions might he ask in selecting an appropriate intervention?

140






141


4) Implement Practices Provide initial recommended training and

professional development. Plan for initial implementation (e.g., scheduling,

materials). Provide ongoing coaching and professional

development. Monitor and evaluate fidelity of implementation.

142


What is Fidelity of Implementation?

143


4) Implement Practices: Fidelity

The best way to monitor fidelity is to measure it. Consider adherence to schedule, protocols, and

intervention plans Self-Report Data Observation Logs/Lesson Plans

Training Manual Page 11

144


Evaluating Implementation: Monitoring Fidelity

Self-Report DataQuestionnaires, surveys, interviewsMay provide an indicator of teacher knowledge, context of implementation Often unreliable when used alone


Evaluating Implementation: Monitoring Fidelity

Evaluating fidelity through observation Develop checklists of critical implementation components Record and listen to sessions at random Spot checksPeer observationsPeer coaching


Evaluating Implementation: Monitoring Fidelity Logs/lesson plans

Student work

Allows for evaluation of what was done

Content covered

Student progress

Provides less information about:

Delivery

Dosage

Adherence to scripts (if applicable)


6) Evaluating Implementation Using Fidelity Data

Distinguish curriculum/intervention vs. quality of implementation when problems occur

Identify implementation strengths (people, potential coaches, parts of intervention)

Target areas in need of improvementCoachingProfessional DevelopmentRetraining


Selecting Evidence-based Practices1. Identify needs and priorities.2. Select practices to address needs.




149


5) Evaluate Effectiveness Evaluate general intervention effectiveness. Evaluate effectiveness for individual students.

• Ongoing, graphed progress monitoring data• Reviewed at least every 4-6 weeks• Evidence-based decision-making criteria• General outcome measure versus mastery measure

150


0

10

20

30

40

50

Week 1 Week 2 Week 3 Week 4

Intervention A

Intervention B

Intervention C

Growth by Intervention Type

Compare Efficacy of Interventions

151

Wor

ds R

ead

Cor

rect

ly


Intervention Effectiveness

152

Intervention A

Goal line

Trend line

Trend line Intervention B

Indicates Change


CLOSING

153


Things to Remember Good data IN … Good data OUT

• Know where your data came from and the validity of that data

Focus on the big picture or ALL students• Are most students making progress?

ALL instructional and curriculum decisions should be based on DATA.

Keep it SIMPLE and EFFICIENT!

154


Implementing the RTI Framework Select and implement research and evidence-based

practices and procedures. Implement essential components with integrity. Ensure cultural, linguistic, and socioeconomic factors

students bring to the classroom are reflected in tool selection and implementation.

155


Review Activity Why does the NCRTI use levels instead of tiers? Where can districts and schools find evidence of an

intervention’s effectiveness? How can schools monitor the fidelity of intervention

implementation?

156


Next Steps Develop a district/school multi-level prevention

framework model (e.g., guidance document). Develop an RTI district implementation plan.

157


National Center on Response to Interventionwww.rti4success.org

RTI Action Networkwww.rtinetwork.org

IDEA Partnershipwww.ideapartnership.org

Need More Information?



www.rti4success.org

Questions?

159


This document was produced under U.S. Department of Education, Office of Special Education Programs Grant No. H326E07000.4 Grace Zamora Durán and Tina Diamond served as the OSEP project officers. The views expressed herein do not necessarily represent the positions or policies of the Department of Education. No official endorsement by the U.S. Department of Education of any product, commodity, service or enterprise mentioned in this publication is intended or should be inferred. This product is public domain. Authorization to reproduce it in whole or in part is granted. While permission to reprint this publication is not necessary, the citation should be: www.rti4success.org.



DISCLAIMER

The opinions and positions expressed herein are not intended to ensure compliance with any particular law or regulation pertaining to the provision of educational services for eligible students. This presentation and/or materials should be viewed and applied by users according to their specific needs. This presentation and/or materials represent the views of the presenter(s) regarding what constitutes preferred practice based on research available at the time of this publication. The presentation and/or materials should be used as guidance. Any references specific to any particular education product are illustrative, and do not imply endorsement of these products by OSPI, or to the exclusion of other products that are not referenced in the presentation materials. OSPI, Special Education, is not responsible for the content of those educational product(s) referenced in this presentation.

Douglas H. Gill, Ed.D.,Director, Special Education

WA OSPI

Documents

National Center on Response to Intervention OSPI & National Center on Response to Intervention November 1, 2011 Implementer Series Module 3: Multi-level