Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
DATA-BASED DECISION MAKING
Progress Monitoring and Measuring Fidelity At Tier 2
1
2
Progress Monitoring
Essentials
Monitoring Effect
Monitoring Fidelity
Comprehensive Assessment System 3
Screener Diagnostic Progress Monitoring
(Formative Assessment)
Problem Solving Model
Progress Monitoring Essentials 4
What makes an assessment able to be used for progress monitoring? Efficient (easy to give, easy to understand) Global indicator of skills Reliable and valid Recent (old data is useless data)
Monitoring Student Progress 5
Tier 2 Student Progress Monitoring System: Assess specific skills (not high-inference skills; observable
behaviors) Sensitive to small increments of growth over time Administered efficiently and repeatedly (quickly and
easily) Easily summarized in teacher/parent-friendly formats for
communication purposes (graphs) Structured so that comparisons can be made across
students Have direct relevance to intervention development for
specific skills
Monitoring is Two-fold 6
1. Monitoring outcomes (is it working?) 2. Monitoring fidelity/implementation (are we doing
what we said we would?)
Data Collection Progress Monitoring & Frequency
Person Collecting/ Interpreting Data
Once/day Several times/day
Teachers with Behavioral Expertise
Paraprofessionals School Counselors
School Psychologists
Once/day Several times/week
1-2 times/week
Teachers Paraprofessionals School counselors
School psychologist
1-2 times/month Monthly
3 times/year
Teachers All staff
-Behavior Tracking Forms (ODRs) -Suspension, Expulsion reports
*SET, BoQ, PIC, TIC
Daily Progress Reports Checklists, ODRs
Permanent Products *BAT, Checklists
Direct Observation Behavior Rating Scale
*BAT, Checklists, Interviews
7 *indicates tool for fidelity/implementation
8
Progress Monitoring
Essentials
Monitoring Effect
Monitoring Fidelity
Monitoring Effect of Interventions: Two questions
9
1. Is it working for most students (overall effectiveness)?
2. Is it working for one particular student (individual effectiveness)?
Has to be effective for most before you can really determine if
it’s effective for one
10
0
20
40
60
80
100
120
student 1Linear (student 1)
Improve fidelity and redo Intervention
Not sure if Intervention would
have worked
Intervention didn’t work
Measuring Fidelity
Fidelity was good
Fidelity was unclear, not good,
or don’t know
Checking for good fidelity avoids misattributing poor response to diet when poor fidelity/implementation is the culprit
Goal Not Met
12
0
20
40
60
80
100
120
student 1Linear (student 1)
13
0
20
40
60
80
100
120
student 1student 2student 3student 4
14
0
20
40
60
80
100
120
student 1student 2student 3student 4
15
Evaluate overall effectiveness of Tier 2 For instance, here is CICO
Examples 16
ODRs Daily points on a behavior tracking card Daily self-report or checklist Percentage of work completion Others?
18
19
Progress Monitoring
Essentials
Monitoring Effect
Monitoring Fidelity
Monitoring Effect 20
1. Check fidelity 2. Check if it’s working for most students (is it
effective at all?) 3. Check if it’s working for a particular student
Fidelity? Most students? One student?
Measuring Fidelity: Did it occur?
21
Direct Assessment Components of an intervention are clearly outlined in
operational terms and directly observed. Often calculate a total percentage of number of
components implemented
Indirect Assessment Assess degree of implementation without directly
observing instruction/intervention. Typically based on self-reports, interviews, and review of permanent products.
Checklists/Observations 22
Checklists can break down the components, mark if implemented and calculate a % of parts implemented
Examples of Indirect Assessment
24
Look at… self reports (rating scales, checklists) on implementation lessons plans to ensure instruction was planned for and
provided attendance to ensure student was present to receive
instruction student products to see that students participated in the
instruction
25
10/12 35 45 Fire drill ended early, 10/13 45 45
Social Skills
School-Generated Checklists… 27
Daily Behavior Plan Assessment—CICO Teacher
28
Fidelity 29
Fidelity is not an evaluation tool (there are other processes for that)
Coaching tool No surprises (share what’s on it) It’s about following the “prescription”, not fooling or
evaluating people
Measuring Fidelity: Did it match?
Can be more than just ‘Did it occur?’. Can also ask if the intervention targeted the student’s deficits.
If the student has a deficit in academics, would an intervention targeting social skills be appropriate? If the function is peer avoidance, would a study hall
(for adult tutoring) be appropriate?
Is Check In/Check Out appropriate for a student who doesn’t want adult attention?
Measuring Fidelity: Did it match?
Measuring Fidelity: Did it match?
STAY ON TARGET! • Ensure the intervention is
on target by doing good problem analysis and using research-based instruction.