Embed Size (px)
DESCRIPTION
Overview of descriptive and inferential options for evaluating alignment of internal and external assessments to help improve student achievement. Presented at Data Director User Conference in November 2009.
Citation preview
Aligning Benchmarks with High Stakes AssessmentsDevin Vodicka, Ed.D.
Outline
Context Data-Driven Decision-Making Understanding Data Descriptive Analysis Inferential Analysis Impact Reflection
The Context: Carlsbad Unified School District Currently in Year 3 Data Director Implementation 14 Schools & 10,500 ADA
9 K-5 3 Middle Schools 1 High School 1 Alternative School
District API: 2007: 829 2008: 843 2009: 858
Eight Title I Schools 0 schools in PI
What is Data-Driven Decision-Making?
http://www.portical.org/ http://www.clrn.org/elar/
Understanding Data
Variables Numeric: Continuous scale Attribute: Discontinuous scale
Descriptive Analysis Use visual displays such as charts and graphs Numeric
• Central Tendency, Standard Deviation Attribute:
• Frequency Tables Inferential Analysis
Determines if relationships between variables are “statistically significant”
• Examples: Chi-Square, ANOVA, Regression Analysis
Descriptive Analysis
Create Test Series Use Pre-Built Reports
Pivot Table Create Custom Reports
3rd Grade ELA Fall Benchmark Alignment
20092008
6th Grade ELA 1st Semester Benchmark Alignment
2008 2009
Custom Reports
Inferential Analysis 2008-09
Download Excel data from Custom Reports
Import into Stats Program NCSS
Three Calculations:Attribute to Attribute (Chi Square)Attribute to Numeric (ANOVA)Numeric to Numeric (Regression)
Fall ELA Alignment
Attribute Alignment Chi-Square Statistics Section Probability Level 0.000000
Reject H0
Fall ELA Alignment
Analysis of Variance TableProb Level0.000000** Term significant at alpha = 0.05
250.00
300.00
350.00
400.00
450.00
1 2 3 4 5
Means of X2008_CST_ELA_Scaled_Scores
X_4th___5th_Grade_ELA_Fall__Proficie
X2008_C
ST
_E
LA
_S
cale
d_S
core
s
200.00
300.00
400.00
500.00
600.00
1 2 3 4 5
Box Plot
X_4th___5th_Grade_ELA_Fall__Proficie
X2008_C
ST
_E
LA
_S
cale
d_S
core
s
Fall ELA Alignment
100.0
225.0
350.0
475.0
600.0
0.0 25.0 50.0 75.0 100.0
X2008_CST_ELA_Scaled_Scores vs X_4th___5th_Grade_ELA_Fall__Total_Pe
X_4th___5th_Grade_ELA_Fall__Total_Pe
X2008_C
ST
_E
LA
_S
cale
d_S
core
s
Fall Summary Statement
The equation of the straight line relating X2008_CST_ELA_Scaled_Scores and X_4th___5th_Grade_ELA_Fall__Total_Pe is estimated as: X2008_CST_ELA_Scaled_Scores = (185.8394) + (2.9938) X_4th___5th_Grade_ELA_Fall__Total_Pe using the 743 observations in this dataset. The y-intercept, the estimated value of X2008_CST_ELA_Scaled_Scores when X_4th___5th_Grade_ELA_Fall__Total_Pe is zero, is 185.8394 with a standard error of 5.4943.
The slope, the estimated change in X2008_CST_ELA_Scaled_Scores per unit change in X_4th___5th_Grade_ELA_Fall__Total_Pe, is 2.9938 with a standard error of 0.0829. The value of R-Squared, the proportion of the variation in X2008_CST_ELA_Scaled_Scores that can be accounted for by variation in X_4th___5th_Grade_ELA_Fall__Total_Pe, is 0.6376.
The correlation between X2008_CST_ELA_Scaled_Scores and X_4th___5th_Grade_ELA_Fall__Total_Pe is 0.7985. A significance test that the slope is zero resulted in a t-value of 36.1041. The significance level of this t-test is 0.0000. Since 0.0000 < 0.0500, the hypothesis that the slope is zero is rejected.
The estimated slope is 2.9938. The lower limit of the 95% confidence interval for the slope is 2.8313 and the upper limit is 3.1563. The estimated intercept is 185.8394. The lower limit of the 95% confidence interval for the intercept is 175.0709 and the upper limit is 196.6080.
What does this mean?
There is a statistically-significant relationship between 5th Grade Fall ELA Performance Levels and CST ELA Performance Levels.
The correlation between the 5th Grade Fall ELA Percentage Correct and the CST ELA Scale Scores is about 80%.
64% of the variation in CST Scale Scores can be predicted by the Fall ELA Percentage Correct.
CST Scale Score = 185 + (2.99 x %)
2nd-5th Grade Summary
Calculating Cut-Points
Create algebra formula for %(Scale – Intercept) / ( Slope) = %
Feed Scale Scores into formula
5th Grade Revised Cut Scores
Grade Term Performance Level Percentage Bands
5 Fall FBB 0 - 28%
BB 29% - 38%
Basic 39% - 55%
Prof 56% - 70%
Adv 71% +
Mid FBB 0 – 32%
BB 33% - 44%
Basic 45% - 62%
Prof 63% - 79%
Adv 80% +
Spring FBB 0 – 32%
BB 33% - 42%
Basic 43% - 58%
Prof 59% - 74%
Adv 75% +
4th Grade Revised Cut Scores
Grade Term Performance Level Percentage Bands
4 Fall FBB 0 – 24%
BB 25% - 36%
Basic 37% - 52%
Prof 53% - 70%
Adv 71% +
Mid FBB 0 – 33%
BB 34% - 43%
Basic 44% - 61%
Prof 62% - 76%
Adv 77% +
Spring FBB 0 – 33%
BB 34% - 44%
Basic 45% - 61%
Prof 62% - 75%
Adv 76% +
3rd Grade Revised Cut Scores
Grade Term Performance Level Percentage Bands
3 Fall FBB 0 – 25%
BB 26% - 40%
Basic 41% - 59%
Prof 60% - 78%
Adv 79% +
Mid FBB 0 – 41%
BB 42% - 54%
Basic 55% - 70%
Prof 71% - 88%
Adv 89% +
Spring FBB 0 – 36%
BB 37% - 50%
Basic 51% - 67%
Prof 68% - 84%
Adv 85% +
2nd Grade Running Records
200.00
300.00
400.00
500.00
600.00
0 1 2 3 4 5 6 7 8 9 10 11
Box Plot
Fall_HM_Level
X2008_C
ST
_E
LA
_S
cale
d_S
core
s
250.00
312.50
375.00
437.50
500.00
0 1 2 3 4 5 6 7 8 9 10 11
Means of X2008_CST_ELA_Scaled_Scores
Fall_HM_Level
X2008_C
ST
_E
LA
_S
cale
d_S
core
s
200.00
300.00
400.00
500.00
600.00
0 1 2 3 4 5 6 7 8 9 10 11
Box Plot
Spring_HM_Level
X2008_C
ST
_E
LA
_S
cale
d_S
core
s
200.00
262.50
325.00
387.50
450.00
0 1 2 3 4 5 6 7 8 9 10 11
Means of X2008_CST_ELA_Scaled_Scores
Spring_HM_Level
X2008_C
ST
_E
LA
_S
cale
d_S
core
s
Fall
Spring
2nd Grade Revised Cut Scores
Grade Term Performance Level Percentage Bands
2 Fall FBB 0
BB 1-2
Basic 3
Prof 4-5
Adv 6 +
Mid FBB 0 – 45%
BB 46% - 56%
Basic 57% - 70%
Prof 71% - 85%
Adv 86% +
Spring FBB 0-1
BB 2-3
Basic 4-5
Prof 6-8
Adv 9-11
Middle School ELA: Writing Prompts
Grade Semester Significant Relationship? R-Squared
6
1st Yes 23%
2nd Yes 30%
7
1st
Yes to CST Overall 22%
Yes to CST Writing Cluster 10%
2nd
Yes to CST Overall 29%
Yes to CST Writing Cluster 12%
8
1st Yes 25%
2nd Yes 41%
Impact
Presentations & FeedbackTeacher LeadersPrincipals
Revised Cut ScoresPerformance Level Descriptors
• http://www.cde.ca.gov/ta/tg/sr/documents/pldreport.pdf
Increased Confidence in Conclusions Improvement in Organizational Integrity?
Next Steps
Math Alignment K-8 New Adoptions implemented in 2009-10
High School Alignment English Math Social Studies Science
Identify and Promote “Best Practices” Grades
Reflection
In your environment, how aligned are your local assessments with the high-stakes tests?
How do you know? How could you find out? What would be the impact in your
district of going through an alignment analysis?
Conclusion
Check out www.acsa.org and then search for “Vodicka” to find article“Building Trust Through Data” (with
Lisa Gonzales)
