Upload
jonathan-brent-mitchell
View
213
Download
0
Tags:
Embed Size (px)
Citation preview
Who is it?On the next slide two photographs will appear.
Do you think you will recognize the images in the photos?
YES or NO
Coleman Report“[S]chools bring little influence to bear upon a child’s achievement that is independent of his background and general social context.”
(Coleman et al., 1966, p. 325)
Measuring Achievement Is Not Enough
It is part of the “picture” . . .but let’s look a picture of
achievement . . .
The Achievement “Lens”Provides a measure of what students
know and are able to do relative to the Ohio standards, benchmarks and grade level indicators.
Achievement tests are an important measure of the academic learning students have accumulated since birth.
Creates a set of criteria for a statewide accountability system.
But Issues Arise When We ConnectSchool Quality only to achievement What part of academic achievement
is due to SES and what part is due to the school?
Achievement scores alone have problematic diagnostic value
A judgment of school quality is flawed if it is based solely on the achievement levels of its students. (see next slide)
Proficiency Bar — 75% passage rate
Ach
ieve
men
t L
eve
ls
Grade Levels 2 3 4 5 6 7 8
High SES school
Low SES
school
The Progress “Lens”Uses existing achievement measures to do the
analysis.Is a reliable measure of student academic growth
from one test to the next, i.e., spring to spring.Has substantial diagnostic value as well as
accountability value.Provides data on
average progress of students in each subject and grade level.
average progress of students at different prior achievement levels.
comparisons of progress across curricular areas.
How is it Done?
Reading 489 220 535 218
Math 551 230 595 238
Science 525 215 565 220
SocialStudies 575 240 605 252
2001 2002 2003 2004
SAT9 OPT4 SAT9 OPT6 3rd 4th 5th 6th
1. We collect all of the individual student data that is available for students.
2. All of these data points are used to develop individual student academic profiles.
Copyright © 2003. Battelle for Kids
Profile for “Student One”
Student Profile 1
Student Profile 2
Student Profile 3
Student Profile 4
Student Profile 5
Student Profile 6
Student Profile 7
Student Profile 8
Student Profile 9
Student Profile 10
Student Profile 11
Student Profile 12
Student Profile 13
Student Profile 14
Student Profile 15
Student Profile 16
Student Profile 17
Student Profile 18
Student Profile 19
Student Profile 20
3. Each student’s profile is added to a pool that contains the profiles of all students, present and past, who have taken the
same year-end assessments.
Student Data
Includes the profiles of all students from this year and from past years who have taken the
same year-end assessments
Copyright © 2003. Battelle for Kids
The Pool
Student Projection1
Student Projection 2
Student Projection 3
Student Projection 4
Student Projection 5
Student Projection 6
Student Projection 7
Student Projection 8
Student Projection 9
Student Projection 10
Student Projection 11
Student Projection 12
Student Projection 13
Student Projection 14
Student Projection 15
Student Projection 16
Student Projection 17
Student Projection 18
Student Projection 19
Student Projection 20
The actual results of other students who have profiles very similar to Student One are used to create a statistical projection of where Student
One is likely to be at the end of a given academic year.
4. The profiles of other students with similar performance
histories are then used to create statistically reliable projected scores for each student .
The Pool
Your School
Student Projection1
Student Projection 2
Student Projection 3
Student Projection 4
Student Projection 5
Student Projection 6
Student Projection 7
Student Projection 8
Student Projection 9
Student Projection 10
Student Projection 11
Student Projection 12
Student Projection 13
Student Projection 14
Student Projection 15
Student Projection 16
Student Projection 17
Student Projection 18
Student Projection 19
Student Projection 20
Mean Projected Score
Student Score 1
Student Score 2
Student Score 3
Student Score 4
Student Score 5
Student Score 6
Student Score 7
Student Score 8
Student Score 9
Student Score 10
Student Score 11
Student Score 12
Student Score 13
Student Score 14
Student Score 15
Student Score 16
Student Score 17
Student Score 18
Student Score 19
Student Score 20
Mean Student Score
5. Mean projections and mean student scores are calculated.
Copyright © 2003. Battelle for Kids
Mean Student Score - Mean Predicted Score with additional statistical reliability factored in
= School Effect
6. The mean student score is compared to the building’s mean predicted score to determine the School Effect.
Copyright © 2003. Battelle for Kids
16
Ohio’s Value-Added Measure
Measures the contribution of a school or district to the progress of its students on test scores
Scores are measured in “Normal Curve Equivalent” gains
“0” gain represents the typical or “expected” gain
“Value-Added” is based on the 2006 – 07 distribution of scores
Scores use a measure of precision (1 Standard Error) to help describe the Value-Added classification
17
There are three classification “bands”using “gain score” and 1 Standard Error
• + Above expected growth (Green)
• Met expected growth (“one year of growth in one year of time”) (Yellow)
• - Below expected growth (Red)
Ohio’s Value-Added Measure
-10
0
10
Gain = 4 SE = 3
Gain = 4 SE = 5
Gain = 3 SE = 1
Gain = -4 SE = 5
Gain = -3 SE = 2
Gain = -5 SE = 1
Meet ( )
Exceed (+)
Meet ( )
Not Meet (-)
Not Meet (-)
Gain Score, Standard Error and Classifications Relative to Value-Added Standard
Exceed (+)
Connection ofValue-Added to
State Report Card System
How Value-Added ImpactsThe Ohio School Report Card Scores
Ohio’s Report Card System. . . Ohio's Accountability Scheme
The yearly “Report Card” is Ohio’s way to insure accountability among its schools and communicate this accountability to the public.
The Ohio Department of Education (aka, ODE) publishes an annual guide for understanding the Report Card.
Ohio School Performance DesignationsSchools and school districts “earn” designations
from the State of Ohio based on their performance. These designations are: Excellent with Distinction Excellent Effective Continuous Improvement Academic Watch Academic Emergency
The School Year Report Card contains the evidence on which these designations are based and includes four components . . . One of these components is Value Added
State Report Card(component 4 – Value Added . . .)
Value-added results are calculated for schools with grades 4-8 in reading and mathematics. Additionally, a composite is displayed on the front of the report card. The composite impacts the district’s or school’s rating. Results will be displayed using the following symbols:( + ) A plus symbol indicates that a district or school
has achieved more than one year of expected growth in student progress.
( √ ) A check symbol indicates that a district or school has achieved one year of expected growth in student progress.
( - ) A minus symbol indicates that a district or school has achieved less than one year of expected growth in student progress.
State Report Card(component 4 – Value Added - continued . . .)
Consequences:A district or school that achieves, on the
composite result, an “Above Expected Growth” for two consecutive years may increase its rating by one category. For instance, a school otherwise rated as Effective may be raised to Excellent if it achieves a “+” designation for two consecutive years.
Districts and schools that show “Below Expected Growth” for three consecutive years will have their designation reduced.
Measurement of Learning TypesAchievement Status Achievement Gain
Often use Standardized Test Scores
Reflect School Quality
Both are important to parents, teachers, and community
Is the ultimate metric for student post-secondary opportunities
Compares year to year performances of unlike cohorts on School Report Card
Is significantly related to student family background
Simply put, achievement status conveys the degree to which a student or cohort demonstrates performance mastery
Much of the foundational research using this form reported discouraging school influence on student achievement
Measures the progress students make between two points in time
Is largely unrelated to student background factors
Though the calculations require involved data structures, capacity, and statistical technology, the concept is straightforward – student progressThe more recent research using this metric found the school’s / teacher’s influence to be paramount for facilitating student academic progress
A usable gauge for measuring school program effectiveness for anyway that students are grouped
Research NuggetsTeachers matterThe difference between having a high and a low
quality teacher 3 years in a row is roughly 50 percentile points.
Differences within schools are typically greater than differences across schools.
Only the most effective teachers - the top 20 per cent – are providing instruction that produces adequate gain in high-achieving students, while students in the lower achievement levels profit from all but the least effective teachers.
Having a high quality teacher throughout elementary school can substantially offset or even eliminate the disadvantage of low socio-economic background.
https://ohiova.sas.com/evaas/signin.faces
And Finally . . . .
Use the above link to explore school districts and buildings throughout Ohio to view how they performed on the Value-Added standard.