Teaming/Data/Interventions

RtI Infrastructure:Data

RtI Partnership Coaches meetingJanuary 6, 2011

Terry Schuster, RtI Partnership Lead Coach

Key Features:•4 Purposes of Assessment•Knowing if it’s Scientifically Based•Building a Common Data System•Importance of Tools, Training, and Support

Integrated Assessment Systems

Assessment

Instruction

Assessment Instruction

Assessment FOR Instruction..

Not this

Schools Use Specific Tools for Specific Assessment Purposes

Type Feature Example

Universal Screening Reliable, Valid, Low Cost, Accurate, Production Type Responses, Sensitive to Between Persons Differences

CBM Family Members; Common Assessments; ODRs

Diagnostic Lots of Items, Production-Type Responses

Placement Tests; Curriculum Assessments; Can’t Do/Won’t Do; Curriculum-Based Evaluation thought process

Progress Monitoring Reliable, Valid, Low Cost, Accurate, Production Type Responses, REPEATABLE, Sensitive to Within Persons

Differences

CBM Family Members; Common Formative Assessments

Program Evaluation Linked to Important Outcomes

ISAT; CBM; Common Assessments; GPA; ODRs

Linking Screening and Progress Monitoring Assessment Across Tiers

Academic Basic Skills

Tier 1:

Universal Screening / Benchmarking

Tier 2:

Strategic Monitoring

Tier 3:

Frequent Progress Monitoring

Material Generalized Curriculum CBM probes

Generalized Curriculum CBM probes

Generalized Curriculum CBM probes

Frequency 3 x per year 2 x per month 1-2 x per week

Common Assessments

• Definition• Rationale• Formative / Summative

What Would Be The Consequences of Allocating Services Based on This Triangle?

• Inefficient• Spreads supplemental & intensive resources too thin• Makes specialized services less special

Or THIS Triangle?

Images and Analyses Courtesy of Ben Ditkowsky, Ph.D.ben@measuredeffects.com

Triangles are About Program Evaluation

Schools Use CBM in Universal Screening Instead of Referral Driven Practices

• Local – school or district• Think box plots or %-ile charts

e.g., < 25thTier 2 Candidates

e.g., <10thIndividual Problem Solving and/or Tier 3 Candidates

Progress Monitoring• Survey Level

Assessment– Too often skipped

• Goal Setting– Should NOT be set in

Instructional Level material

– Criterion for success should at least begin to close the gap with peers

R-CBM: Indian Prairie Average Range of Performance

9

27

58 58

85

104

82

106

121

101

119

132

113

133

150

61

94

121

112

138

154

138

155

171

148

168

182

167

182

199

0

20

40

60

80

100

120

140

160

180

200

220

Fall

Wint

er

Spring Fall

Wint

er

Spring Fall

Wint

er

Spring Fall

Wint

er

Spring Fall

Wint

er

Spring

First (ROI = 1.9) Second (ROI = 1.3) Third (ROI = 1.0) Fourth (ROI = .9) Fifth (ROI = 1.0)

# o

f WR

CSurvey Level Assessment: Test backward using successively lower grade level probes until the student's score falls within the green…Choose at least one grade level above for goal writing and progress monitoring… Use a value greater than the specified average ROI to begin to close the gap. Goal = SLA Score in Goal Material + (Goal Material ROI) (# of weeks)

Set goal & monitorAt 4th or 5th gr.

DiagnosticsIt’s not just about one measure . . .

• Classroom performance– Observations– Independent work

• District assessments/ ISAT• Curriculum assessments

– Pre-tests; Placement tests– Chapter & Unit tests– Curriculum skill/indicator checklists

• Skill Deficit versus Performance Deficit• When Tier 3 & still need more info - CBE

It’s not always about one student . . .

Diagnostic Assessment

• Needs to connect with instruction; tells teachers what needs to be taught

• Skills, not scores

• The process is more important than the “test”– Adequate number of items per skill– Error analysis

What happens when these data systems aren’t in place?

• Lack of direction for knowing what needs to be improved

• Don’t know which tier and students need additional support/intervention

• Don’t know if change efforts have been successful

Key Questions

• What assessments do you currently use for each purpose in your district/building? • Do they each meet the requirements for that purpose? • Do you have too many assessments for a particular purpose? Too few?• Tools? Training? Support?

Assessment Worksheet

Type Assessments Currentlyin Place

Meets Criteria for Purpose(comments)

Action Plan

Universal Screening

Frequency?

Rigby/Fountas & Pinnell Leveling 3x/yrISEL 2x/yrGates 3rd -8th 1x/yrR-CBM 1st-5th 3x/yr

EL (all 4) K-1st 3x/yrMaze 6th-8th 3x/yr

NoNoNoYes

NoNo

Move to Diagnostic tool – not all students 3x/yrMove to Diagnostic tool – not all students 3x/yrMove to Diagnostic tool – not all students Keep as Universal Screener – firm up US use in target schools - separate US from Program EvalEstablish a multi-gate systemEither use as true universal screener or drop – consider using R-CBM on bottom 25%

Diagnostic/Intervention Planning

WIATWJ-IIIDRAIRI’sWADE (Wilson placement test)Horizons placement test

Add Rigby/Fountas & Pinnell Leveling , ISEL, and Gates Add regular curriculum assessments – need to decide this at a district, or at least a school levelAdd assessments for determining Skill vs Performance deficitsAdd some CBE-type skills

Progress Monitoring

Frequency?

CBM (all types) Tier 2 2x/mo Tier 3 1x/wk

Yes

Continue to provide coaching support to schools/staff with low compliance on monitoring;Decrease Maze monitoring;Decrease # of students being monitored on more than one CBM measure

Program Evaluation

ISAT 1x/yrCBM – ISAT correlate triangles 3x/yr

YesYes

ContinueContinue, but separate from US decisions

Example

R-CBM

Ongoing for new staff; annual AIRS; sd/coaching during grade level data mtgs

3x per year

R-CBM

yes yes

Ongoing for new staff; annual AIRS; sd/coaching during data review mtgs

2x /mo for sts in Tier 21x/wk for sts in Tier 3

‘15 hour’ grade level data mtgs

4-6 wk data review mtgs – all sts being monitored

School level percentiles

District ISAT cut scores into triangles

R-CBM

yes

2x per year

SIP team &‘15 hour’ grade level data mtgs

sd/coaching during SIP & grade level mtgs

Key Features

• 4 Purposes of Assessment

• Knowing if it’s Scientifically Based

• Building a Common Data System

• Importance of Tools, Training, and Support

Assessment Worksheet

Type Assessments Currentlyin Place

Meets Criteria for Purpose

(comments)

Action Plan

Universal Screening

Diagnostic/Intervention Planning

Progress Monitoring

Program Evaluation