Upload
valentina-leon
View
214
Download
0
Embed Size (px)
Citation preview
MN RtI CenterMN RtI Center1
Progress Monitoring in Reading:How to Use the Data
A module for pre-service and in-service professional development
MN RTI CenterAuthors: Lisa Habedank Stewart, PhD & Adam Christ, graduate
student Minnesota State University Moorhead
www.scred.k12.mn.us click on RTI Center
MN RtI CenterMN RtI Center2
MN RTI Center Training Modules
This module was developed with funding from the MN legislature It is part of a series of modules available from the MN RTI Center
for use in preservice and inservice training:
Module Title Authors
1. RTI Overview Kim Gibbons & Lisa Stewart
2. Measurement and RTI Overview Lisa Stewart
3. Curriculum Based Measurement and RTI Lisa Stewart
4. Universal Screening (Benchmarking): (Two parts)
What, Why and How
Using Screening Data
Lisa Stewart
5. Progress Monitoring: (Two parts)
What, Why and How
Using Progress Monitoring Data
Lisa Stewart & Adam Christ
6. Evidence-Based Practices Ann Casey
7. Problem Solving in RTI Kerry Bollman
8. Differentiated Instruction Peggy Ballard
9. Tiered Service Delivery and Instruction Wendy Robinson
10. Leadership and RTI Jane Thompson & Ann Casey
11. Family involvement and RTI Amy Reschly
12. Five Areas of Reading Kerry Bollman
13. Schoolwide Organization Kim Gibbons
MN RtI CenterMN RtI Center3
Overview
This module is Part 2 of 2 Part 1: Why, What, How to Progress Monitor
Part 2: Using Progress Monitoring Data
MN RtI CenterMN RtI Center4
Why Progress Monitor? When teachers USE progress monitoring
Students learn more! Teachers design better instructional
programs Teacher decision making
improves Students become more aware of
their performance Safer & Fleishman, 2005
MN RtI Center5
Is this student making progress? Adam Gr 4
0
10
20
30
40
50
60
70
80
90
100
110
120
130
140
week
wo
rds
corr
ect
pe
r m
in (
gra
de
leve
l pa
ssa
ge
)
wrc
errors
Repeated Reading 1:1, 20 min day Repeated Reading 1:1, 10 min 2xday
MN RtI Center6
Graphing/Displaying the Data
“A picture is worth a thousand words”
MN RtI Center7
Making a Graph
Label your axes
Have an “aimline” that shows what the end goal is
Show changes in instruction with “lines” and labels
MN RtI Center8
Use Graphs!
MN RtI CenterMN RtI Center9
Aimline
Shows general trajectory needed for student to reach his/her goal Typically set so student gets back “on target” or “on grade level”
within a set amount of time (e.g., by the end of the year) if possible
Simply draw a straight line from the student’s first data point on the graph to the date and score representing his target or goal
Use SMART goals: specific, measurable, attainable, realistic, with a clear timeframe
MN RtI CenterMN RtI Center10
Aimline and Setting Goals (Cont’d)
For setting CBM goals
Can use local norms or benchmark targets set by your district or based on national datasets and research (e.g., DIBELS targets, AIMSweb targets)
Can use information on the amount of progress students who were successful have made in the past in this intervention or curriculum (e.g., what was the slope of progress in the research?)
MN RtI Center11
Example Gr 1-5 “Targets” for Aimline
Based on the St. Croix River Education District 08-09 Targets
linked to success on Minnesota Comprehensive Assessment – II
Grade Measure Target
1 Nonsense Word Fluency January = 52 letter sounds correct/min
1 CBM Grade Level Oral Reading Fluency (ORF)
Spring = 52 words correct/min
2 CBM ORF Spring = 90 words correct/min
3 CBM ORF Spring = 109 words correct/min
4 CBM ORF Spring = 127 words correct/min
5 CBM ORF Spring = 141 words correct/min
6 CBM ORF Spring = 166 words correct/min
MN RtI Center
Early CBM national “norms” and growth rates in oral reading (words correct per min.)Grade Percentile Fall ORF Winter ORF Spring ORF Weekly
Progress
1 50 27 54 1.80
2 75
50
25
82
53
23
106
78
46
124
94
65
1.66
3 75
50
25
107
79
65
123
93
70
142
114
87
1.18
4 75
50
25
125
99
72
133
112
89
143
118
92
1.01
5 75
50
25
126
105
77
143
118
93
151
128
100
0.58
6+ 50 125-150 125-150 125-150 0.66
Hasbruck & Tindal 1992, Teaching Exc. Children, Deno et al, 2001 School Psych Review
MN RtI Center13
Draw an Aimline for Adam (Gr 4)Adam Gr 4
0
10
20
30
40
50
60
70
80
90
100
110
120
130
140
1/13
/200
9
1/20
/200
9
1/27
/200
9
2/3/
2009
2/10
/200
9
2/17
/200
9
2/24
/200
9
3/3/
2009
3/10
/200
9
3/17
/200
9
3/24
/200
9
3/31
/200
9
4/7/
2009
4/14
/200
9
4/21
/200
9
4/28
/200
9
5/5/
2009
5/12
/200
9
5/19
/200
9
5/26
/200
9
week
wor
ds
corr
ect
per
min
(g
rad
e le
vel p
ass
age
)
wrc
errors
Repeated Reading 1:1, 20 min day Repeated Reading 1:1, 10 min 2xday
MN RtI Center14
Adam’s Aimline Example Using SCRED TargetsAdam Gr 4
0
10
20
30
40
50
60
70
80
90
100
110
120
130
140
week
wor
ds c
orre
ct p
er m
in (
grad
e le
vel p
assa
ge)
Repeated Reading 1:1, 20 min day Repeated Reading 1:1, 10 min 2xday
MN RtI CenterMN RtI Center15
Looking at the Graphs
Is there “go upness”????
Is there ENOUGH “go upness”????
MN RtI Center16
Basic Visual Analysis: “Go Upness”?Moira, Grade 3
0
10
20
30
40
50
60
70
80
90
100
110
120
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Weeks
wor
ds c
orre
ct p
er m
in.
(gra
de le
vel p
assa
ges)
MN RtI Center17
Using an AimlineMoira, Grade 3
0
10
20
30
40
50
60
70
80
90
100
110
120
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Weeks
wor
ds c
orre
ct p
er m
in.
(gra
de le
vel p
assa
ges) AIMLINE
MN RtI CenterMN RtI Center18
Data Decision Guidelines If the student has some data points above and
some below the aimline (doing the “aimline hug”), keep doing what you are doing!
If the student has 4 consecutive data points above the aimline, consider moving the student to less intervention (e.g., decreasing minutes, or moving from Tier 2 to Tier 1 or Tier 3 to Tier 2) Also use other pieces of information
Continue to progress monitor
MN RtI CenterMN RtI Center19
Data Decision Guidelines (Cont’d) If the student has 4 consecutive data points below the
aimline, ASK THE FOLLOWING QUESTIONS (and continue to progress monitor):
What does the “other” evidence available suggest about the student’s progress? Error rates? Behavior during the intervention?
What is the general “trend” of the data? Is the student likely to get where we want if this continues? Use visual analysis and other evidence Use “trendlines” and “aimlines”
MN RtI CenterMN RtI Center20
Trendline Shows the general “trend” or trajectory of the student’s
data so far Web-based programs typically use an Ordinary Least Squares
regression line AIMSweb, DIBELS data system, Excel
Need approx. 7 to 9 data points Trendlines on few data points or on highly variable data are NOT
reliable!!! Christ, T. (2006). Short term estimates of growth using CBM
ORF: Estimating Standard Error of Slope to construct confidence intervals. School Psychology Review, 35(1) 128-133.
MN RtI Center21
Moira, Grade 3
y = -0.8x + 64.6
R2 = 0.0623
0
10
20
30
40
50
60
70
80
90
100
110
120
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Weeks
wor
ds c
orre
ct p
er
min
. (g
rad
e le
vel p
assa
ges) AIMLINE
TRENDLINE
Trendline and Aimline
MN RtI CenterMN RtI Center22
How Much Progress is “Enough”? What is “adequate” progress?
Criterion referenced Will student meet goal? In reasonable amount of time? Growth is at or above “target” growth rate
Norm referenced Growth is at or above growth of grade level peers
Individually referenced Growth is better than before
“Intervention”/research referenced Growth is similar to what was seen in research on this
intervention (with similar population)
MN RtI CenterMN RtI Center23
Remember to Use your Brain! (And Eyes and Ears)
If overall trend of progress is good
but s/he happens to have 4 data points just barely below the aimline, you may decide to continue your intervention for a week and see what happens.
Use convergence of data (teacher report, mastery monitoring, behavioral indicators)
These are guidelines, THINKING is REQUIRED…
MN RtI CenterMN RtI Center24
Practice Exercises:
1. Is there go upness?
2. Is there enough go upness?
3. What else would you like to know?
4. What would you do?• Exit to less intense service • Keep going and collect more data• Problem solve and change something
MN RtI Center25
Finn Gr 2 CBM-ORFFinnegan Grade 2
0
10
20
30
40
50
60
70
80
90
100
110
120
9/9/
2009
9/23
/200
9
10/7
/200
9
10/2
1/20
09
11/4
/200
9
11/1
8/20
09
12/2
/200
9
12/1
6/20
09
12/3
0/20
09
1/13
/201
0
1/27
/201
0
2/10
/201
0
2/24
/201
0
3/10
/201
0
3/24
/201
0
4/7/
2010
4/21
/201
0
5/5/
2010
5/19
/201
0
week
wor
ds c
orre
ct p
er m
in (
grad
e le
vel p
assa
ge)
Aimline
Reading Links 1:5 for 15 min.
MN RtI Center26
Finn Gr 2 CBM-ORF (Cont’d)Finnegan Grade 2
0
10
20
30
40
50
60
70
80
90
100
110
120
9/9/
2009
9/23
/200
9
10/7
/200
9
10/2
1/20
09
11/4
/200
9
11/1
8/20
09
12/2
/200
9
12/1
6/20
09
12/3
0/20
09
1/13
/201
0
1/27
/201
0
2/10
/201
0
2/24
/201
0
3/10
/201
0
3/24
/201
0
4/7/
2010
4/21
/201
0
5/5/
2010
5/19
/201
0
week
wor
ds c
orre
ct p
er m
in (
grad
e le
vel p
assa
ge)
Aimline
Added distributed practice and preteachingReading Links 1:5 for 15 min.
MN RtI Center27
Justan Gr 1 NWFIs Justan Making Progress?
MN RtI Center28
And Now?
MN RtI Center29
And Now????
MN RtI Center30
On Track…
MN RtI Center31
What Decision Would You Make?
MN RtI Center32
And Now?
MN RtI Center33
Is There “Enough” Go Upness?
MN RtI Center34
Can Also Make Decisions About Exiting to Less Intensive Service!
MN RtI Center35
What Can You Do About “Bounce” in the Data?
MN RtI CenterMN RtI Center36
Dealing With Bounce
Is there a “measurement” problem?
Fidelity of administration and scoring
Materials aren’t well designed or are too difficult
Who, where, and when measurement takes place can matter (esp. for some kids)
Motivation issues (can’t do vs. won’t do)
MN RtI CenterMN RtI Center37
Dealing with Bounce (Cont’d)
Other ways to minimize bounce or make decisions despite bounce Do more probes at one time and take median or
average score Do more frequent measurement (e.g., weekly or 2x
week) Look at trend over time with many data points
Look at ALL data together (errors, mastery data, etc.) Use the least dangerous assumption…
MN RtI CenterMN RtI Center38
What if There isn’t Adequate Progress?
If you keep doing what you’ve been doing then you will keep getting
what you’ve got.
MN RtI CenterMN RtI Center39
Back to Problem Solving
MN RtI CenterMN RtI Center40
What if There isn’t Adequate Progress?
Is the intervention being done with fidelity?
Has fidelity checks been done?
Is the student in the right level of materials?
Has the student been in school? Are they getting enough minutes of intervention per week?
MN RtI CenterMN RtI CenterDRAFT May 27, 2009 41
What if There isn’t Adequate Progress? (Cont’d)
Should the intervention be “tweaked”? Changed? Is there an intervention better “matched” to this student’s needs?
Changes could include trying a different intervention or just “tweaking” the current intervention such as adding a 5th repeat to a repeated reading or a sticker incentive for accurate reading.
Grade level or problem solving team members work together to discuss the data, the student, and what intervention changes would have the best chance of success.
MN RtI CenterMN RtI Center42
What Could We Change? Focus or skill
Teaching strategies: More explicit, more modeling, more practice, more previewing, better matched with core
Materials: Easier, better matched (cultural, interests, etc.)
Arrangements: Size group, location, who is teaching?
Time: Amount of time, days per week, time of day
Motivation: Interests, goals, rewards, home/school
MN RtI CenterMN RtI Center43
Adam, Gr 4 Benchmark data
Winter: 85 wrc (target= 114) Fall: 89 wrc (target= 93) Error rate moderate (4, 4, & 6)
Very inconsistent academically; good attendance but attention, accuracy and work completion issues; basic decoding skills ok; can correct errors; can read better (with expression, meaning) in high interest material?
Grade Level Team put Adam in Tier 2 intervention- working with MRC 1:1 on repeated reading intervention 20 min per day
MN RtI Center44
Is the Intervention Working?Adam Gr 4
0102030405060708090
100110120130140
w eek
wor
ds c
orre
ct p
er m
in (g
rade
leve
l pas
sage
)
w rc
errors
MN RtI CenterMN RtI Center45
If We Do Change, What Should We Change?
What else would you want to know about Adam and his intervention, curriculum and class?
What are at least 5 different ideas for changes that could be made?
Is this likely to be a tweak or a major shift?
How would you know if you made a good decision?
MN RtI Center46
And Now?Adam Gr 4
0102030405060708090
100110120130140
w eek
wor
ds c
orre
ct p
er m
in (
grad
e le
vel p
assa
ge)
w rc
errors
Repeated Reading 1:1, 20 min day Repeated Reading 1:1, 10 min 2xday
MN RtI CenterMN RtI Center47
Sharing the Data
Just having progress monitoring data is not enough. You need to USE it.
Scheduled graph review dates
Grade level meetings
Problem solving meetings
MN RtI CenterMN RtI Center48
Remember: Garbage IN…. Garbage OUT….
Make sure your data have integrity or they won’t be good fer nuthin… Training Integrity checks/refreshers Well chosen measures and
materials
MN RtI CenterMN RtI Center49
Avoid Common Mistakes
Don’t use the same passage/probe every week!
Have an organized system in place Progress monitoring schedule for students Preprinted passages/probes in a binder An easy way to graph and look at the data Scheduled time to share/look at the data
MN RtI CenterMN RtI Center50
Remember… When teachers USE progress monitoring
Students learn more! Teachers design better instructional
programs Teacher decision making
improves Students become more aware of
their performance Safer & Fleishman, 2005
MN RtI CenterMN RtI Center51
Web Resources Research Institute on Progress Monitoring
http://progressmonitoring.net/ Includes…
A Study Group Content Module with 15 sections on CBM including activities http://progressmonitoring.org/pdf/cbmMOD1.pdf
Progress Monitoring Leadership Team Content Module with 6 sections (e.g. measureable goals, decision making) including activities http://progressmonitoring.org/pdf/cbmMODldrshp.pdf
Handouts, videos, and power point presentations Technical reports of CBM measures
MN RtI CenterMN RtI CenterDRAFT May 27, 2009 52
Web Resources, Cont’d www.studentprogress.org
Growth rates, use in RTI model, etc
http://www.rti4success.org/ click on Progress monitoring on right side
www.interventioncentral.org look for information on CBM, graphing, etc.
www.aimsweb.com, www.edcheckup.com, dibels.uoregon.edu Look for information about progress monitoring as well as
access to materials and graphing for progress monitoring
MN RtI CenterMN RtI Center53
Print Resources available with this module
Safer & Fleishman. (2005). How student progress monitoring improves instruction, Educational Leadership, 62(5), 81-83.
Fuchs. Progress monitoring within a multi-level prevention system. Retrieved June 5, 2009, from RTI Action Network Web site: http://www.rtinetwork.org/Essential/Assessment/Progress/ar/MultilevelPrevention
Fuchs & Fuchs What is scientifically-based research on progress monitoring? From the National Center on Student Progress Monitoring (studentprogress.org). Retrieved June 14, 2009,
Jenkins, Hudson, & Hee Lee. Using CBM-Reading assessments to monitor progress. Retrieved June 5, 2009, from RTI Action Network Web site: http://www.rtinetwork.org/Essential/Assessment/Progress/ar/Using CBM/1
MN RtI CenterMN RtI Center54
Other Recommended Articles & Texts
Riley-Tillman & Burns. (2009). Evaluating Educational Interventions. Guilford Press.
Stecker, Lembke, & Fogen (2008). Using progress-monitoring data to improve instructional decision making. Preventing School Failure, 52(2), 48-58. Case study included
MN RtI CenterMN RtI CenterDRAFT May 27, 2009 55
Activity for Teachers or Practicum Students Obtain progress monitoring probes and graphs
Passages and graphing materials self-created or downloaded www.interventioncentral.org dibels.uoregon.edu
Sign up for an account with AIMSweb (instructor accounts and student accounts available) www.aimsweb.org
Practice administration and scoring Progress Monitor a “real” kid (ideally 2-4 kids of varying risk
levels monitored for at least 7-10 weeks) Graph, analyze, and use data
MN RtI CenterMN RtI Center
Articles
Safer & Fleishman. (2005). How student progress monitoring improves instruction, Educational Leadership, 62(5), 81-83.
Fuchs & Fuchs What is scientifically-based research on progress monitoring? From the National Center on Student Progress Monitoring (studentprogress.org). Retrieved June 14, 2009, from AIMSweb, Web site: ???
DRAFT May 27, 2009 56
MN RtI CenterMN RtI CenterDRAFT May 27, 2009 57
Quiz
1.) What shows the general trajectory of the student’s data so far? A.) Axis B.) Trendline C.) Aimline D.) Target
2.) What shows the general trajectory needed to reach the end goal? A.) Axis B.) Trendline C.) Aimline D.) Target
MN RtI CenterMN RtI CenterDRAFT May 27, 2009 58
Quiz (Cont’d)
3.) When should you keep doing what you are doing? A.) If the student has 4 consecutive data points
above the aimline B.) If the student has 4 consecutive data points
below the aimline C.) If the student is doing the “aimline hug” D.) None of the above
MN RtI CenterMN RtI CenterDRAFT May 27, 2009 59
Quiz (Cont’d)
4.) Describe “go upness.”
5.) If the student has 4 consecutive data points below the aimline, what would you do?
MN RtI CenterMN RtI Center
Note: The MN RTI Center does not endorse any particular product. Examples used are for instructional purposes only.
Special Thanks: Thank you to Dr. Ann Casey, director of the MN RTI Center, for
her leadership Thank you to Aimee Hochstein, Kristen Bouwman, and Nathan
Rowe, Minnesota State University Moorhead graduate students, for editing, writing quizzes, and enhancing the quality of these training materials