Embed Size (px)
DESCRIPTION
Wyoming Department of Education. Assessment & Accountability. Statewide System of Support Meeting October 15, 2014 Deb Lindsey, Division Administrator. Assessment Topics. Assessment program requirements Assessment transition, spring 2013 to spring 2015, PAWS PAWS and ACT standard-setting - PowerPoint PPT Presentation
Citation preview
Assessment & Accountability
Statewide System of SupportMeeting
October 15, 2014Deb Lindsey, Division Administrator
Wyoming Department of Education
2
• Assessment program requirements• Assessment transition, spring 2013 to spring
2015, PAWS• PAWS and ACT standard-setting• Growth: student growth percentiles, median
growth percentiles, adequate growth percentiles, what??!!
Assessment Topics
3
Current Assessment Program
• ESEA requires annual testing in grades 3-8, plus once in high school, of ELA/reading and math
• ESEA also requires science testing once in each grade span, elementary, middle and high
• Wyoming statute requires a writing test in grades 3, 5 and 7 during a window that’s different from the reading and math tests
• Wyoming statute also requires the ACT Explore, Plan and ACT Plus Writing in grades 9, 10, and 11; the ACT at grade 11 serves as the ESEA-required accountability assessment
4
Current Assessment Program
• ESEA requires that all assessments measure performance on the state’s adopted content standards
• ESEA and IDEA permit adoption of alternate content standards for students with significant cognitive disabilities (1%); these are to be directly aligned to the “primary” content standards
• Students with significant cognitive disabilities must participate in alternate assessments aligned to the alternate standards (extensions of the “regular” standards)
5
PAWS Design Changes
PAWS blueprint necessitated new proficiency level descriptors, standard-setting and cut scores before 2014 results were released
CCSS
PAWS
2014
PAWS
CCSS
CCSS
PAWS
2013 2015
6
Standard-Setting Context
MS NM LA CA NV WV AK AZ SC TX OK AL HI MI AR SD ID OR GA IL ND TN MOMT NC WI KY ME NE NY OH UT WY DE IN IA KS RI FL CO PA WAMN NJ VT CT VA NH MDMA0
10
20
30
40
50
60
70
80
90
100
21 22 2327 27 27 28 28 28 28 29 30 30 30 32 32 33 33 34 34 34 34 35 35 35 35 37 37 37 37 37 37 37 38 38 38 38 38 39 40 40 40 41 42 42 43 43 44 45
48
38
24
8
50
38
4447
51
44
17
34
58
4338
53
43
57
26
59
3539
14
19
47
9
12
32
42
51
40 4135
46
38
50
31
21
2832 33
13
17
28
35
27
44
33
5
State by State Proficiency Data, 4th Grade NAEP Reading: Wyoming
The Gap between NAEP and State Proficiency Levels, 2013 data
Gap between percentage of students State and NAEP ProficiencyPercentage of students proficient on the National Assessment of Educational Progress (NAEP), the "Nation's Report Card"
States
Perc
ent P
rofic
ient
-2 -7
The sum of the two lines is equal to the percentof students proficient on the state assessment
7
Standard-Setting Context
AL LA MS NMWV OK CA TN AZ NV GA KY AR FL MI SC NY AK DE HI MO OR IL IA NE NC RI UT CT ID IN MD SD TX VA WY ME MT OH WI KS ND CO PA WA NH MN VT NJ MA0
10
20
30
40
50
60
70
80
90
100
19 21 21 22 23 25 27 27 28 29 29 30 31 31 31 31 32 33 33 33 33 34 36 36 36 36 36 36 37 37 38 38 38 38 38 38 40 40 40 40 41 41 42 42 4246 47 47 49
54
6052
20 19
43
4
21
41
54
10
15
27
20
4
39
29
3338
9
29
44
23
39
30
22
49
37
43
29
3639
23
3021 23
32
5
43
36
9
34
11
22
10
17
23
1
State by State Proficiency Data, 8th Grade NAEP Math: Wyoming
The Gap between NAEP and State Proficiency Levels, 2013 data
Gap between percentages of students meeting State and NAEP ProficiencyPercent of students proficient on the National Assessment of Educational Progress (NAEP), the "Nation's Report Card"
States
Perc
ent P
rofic
ient
-2
-6-10
The sum of the two lines is equal to the percentof students proficient on the state assessment
8
• Conducted for PAWS, SAWS (gr 5 & 7), and ACT • Plans approved by WDE and its Technical Advisory
Committee (TAC)• Participants included Wyoming teachers; for ACT,
also included administrators and higher ed representatives
• PAWS-SAWS judgments were standards-based; ACT judgments were empirically-based
• Consistent with experiences of other states, different expectations result in lower proficiency rates
2014 Standard-Setting
9
2014 Results
• Baseline data from new test based on different expectations in 2012 WY Content and Performance Standards in ELA and Math
• Results are not directly comparable to prior years because of the content shift and the “break” in scale -- but can use concordance tables to roughly compare 2013 to 2014
• Focus is on continuous improvement; student performance will improve over time as districts continue to implement new standards
• Primary purpose of assessment in WY: support improved teaching and learning, school and program improvement, and measuring performance indicators under W.S. 21-2-204
10
2015 Assessment
• No changes to state statute in 2014 Legislative Session • Continue PAWS, grades 3 – 8, with current blueprints
& summer 2014 cut scores • Continue SAWS, grades 3, 5, & 7; use summer 2014
cuts for 5 & 7 (grade 3 cuts to be set in summer 2015)• Adopt new ALT beginning in 2015• Continue with ACT Explore, ACT Plan, and ACT Plus
Writing at grades 9 – 11; use new Wyoming ACT scale for cut scores needed for WAEA and federal reporting
11
WAEA Indicators
• Elementary and Middle– Achievement– Growth– Equity
• High School– Achievement– Equity– College Readiness
Wyoming School Rating System
12
What is growth?• PAWS shows how each student is achieving
relative to state standards– Is John proficient in 6th grade mathematics? What percent of his peers
are proficient?
• Growth measures change in an individual student’s performance over time, using scale scores from one year to the next– How much did John improve in mathematics from 5th grade to 6th
grade relative to his academic peers?
13
Why measure growth?• Measures progress for students at all
performance levels– A student can achieve at a low level but still improve relative to his academic
peers– Another could achieve well but not improve much from year to year– Considered to be more fair in school accountability systems
• Provides evidence of improvement even among those with low achievement
• Gives high achieving students and schools something to strive for beyond proficiency
14
Growth Across Years
Grade 4 Performance Levels: PAWS Reading
Grade 5 Performance Levels: PAWS Reading
Below Basic
Below Basic
Basic
Basic
Proficient
Proficient
Advanced
Advanced
15
Growth Across Years
Grade 4 Performance Levels: PAWS Reading
Grade 5 Performance Levels: PAWS Reading
Below Basic
Below Basic
Basic
Basic
Proficient
Proficient
Advanced
Advanced
16
Growth Models Have Shown…• Growth not correlated with proficient status
• Growth gaps can narrow among student groups
• Many low-achieving students not growing fast enough to catch up
• Many proficient students not keeping up
• High-achieving schools can show low growth and low-achieving schools can show high growth
17
Growth TermsStudent Growth Percentiles (SGP) – a normative
measure that compares students with other like-performing students across the state. A SGP produces a relative percentile score (such as 70th percentile) that tells the student that s/he scored equal to or better than 70% of students who had scores like him in the previous year in the state. Median Growth Percentiles (MGPs) are used to summarize SGPs across classes, grades, and schools.
18
Growth TermsAdequate Student Growth Percentiles (AGP) – a
criterion-referenced measure relative to proficiency that measures how far away from proficiency a student is and answers: “how much growth does a student have to make to reach proficiency in three years or by the end of 8th grade?” A student can make 70th percentile growth and still not meet AGP goals. AGPs are NOT reported in 2014 because of the change in cut scores but will return with the 2015 SPRs. When SGP => AGP, then the student is on track.
19
A student’s PAWS scores can be plotted from one year to the next
500
450
400
350
300
250
Grade
Mat
hem
atics
PAW
S sc
aled
sco
re
3 4 5 6
425
20
500
450
400
350
300
250
Grade
Mat
hem
atics
PAW
S sc
aled
sco
re
3 4 5 6
425455
A student’s PAWS scores can be plotted from one year to the next
21
The fourth grade scores of students with the same third grade score can differ and form a distribution
500
450
400
350
300
250
3 4 5 6Grade
Mat
hem
atics
PAW
S sc
aled
sco
re
425
455
22
Comparing the example student’s score to students with similar score histories yields a percentile
500
450
400
350
300
250
Grade
Mat
hem
atics
PAW
S sc
aled
sco
re
82nd
3 4 5 6
50th
23
The fifth grade growth percentile is calculated relative to students with similar score histories at both grades three and four
500
450
400
350
300
250
Grade
Mat
hem
atics
PAW
S sc
aled
sco
re
46th
3 4 5 6
Other students whose scores diverged from the example student are no longer considered to have a similar score history
24
Six students across WY
Grade 3 mathematics PAWS scaled score
Grade 4 mathematics PAWS scaled score
Grade 4 mathematics student growth percentile
A 400 318 16
B 400 400 28
C 400 400 28
D 400 434 49
E 400 482 64
F 400 530 89
A student growth percentile compares the student’s current PAWS score with the scores of students throughout the state with similar score histories
SGP Features
25
SGP Features
Student growth percentiles range from 1 to 99
A student growth percentile compares the student’s current PAWS score with students throughout the state
Each year, a student’s growth percentile is calculated in reference to other students with the same test taking sequence and score history
SGP: 1-99
26
Example 1
Student 3rd grade 4th grade 5th grade SGP associated with 5th grade score
Suzie 270 300 365 70
What does Suzie’s 5th grade scale score tell you?
What can you tell from Suzie’s growth percentile of 70?
If you have the scale score ranges with each performance level, then you’ll know which proficiency category she’s in..
At fifth grade, Suzie outperformed 70 percent of students with similar score histories.
Can you calculate Suzie’s growth percentile just by knowing her previous years’ scores?
No, because we do not have the distribution of scores from students with similar score histories.
28
Student 3rd grade 4th grade 5th grade SGP associated with 5th grade score
Suzie 270 300 365 70
Victor 310 340 365 30
Emily 410 435 460 60
Dante 400 - 460 -
Jamar - 470 500 50
Mya 260 290 335 65
Zachary 420 450 440 8 Why does Jamar but not Dante, have a student growth percentile?
Should Zachary’s teacher be concerned about his performance, given his scale score and growth percentiles?
Jamar has two consecutive years’ worth of data; Dante does not.
Zachary is achieving at a high level but his progress relative to other students in the state who also have this score history, is low. This is concerning because all students, including high achieving ones, can and do achieve high growth.
Example 3
29
Rules of Thumb• Typical student growth percentiles are between
about 40 and 60
• WY School Performance Reports >=60 Exceeding, >=45 to <60 Meeting, <45Below
• Students or groups outside this range has higher or lower than typical growth
• Differences of fewer than 10 SGP points are likely not educationally meaningful
30
• Growth is distinct from achievement– A student can achieve at a low level but grow quickly, and vice versa
• Each student is compared only to his statewide academic peers, not to all students statewide– Others with a similar test score history– All students can potentially grow at the 1st or 99th percentile
• Growth is subject-, grade-, and year-specific– Different academic peer groups for each subject, grade, and year– Therefore, the same change in scaled scores can yield different student growth
percentiles
• The percentile is calculated on the change in achievement, not the absolute level – Differs from more familiar norm-referenced measures
Key Concepts
31
Median Student Growth PercentileLast name SGP
Lennon 6
McCartney 12
Starr 21
Harrison 32
Jagger 34
Richards 47
Crosby 55
Stills 61
Nash 63
Young 74
Joplin 81
Hendrix 88
Jones 95
Imagine that the list of students to the left are all the students in your 6th grade class. Note that they are sorted from lowest to highest SGP.
The point where 50% of students have a higher SGP and 50% have a lower SGP is the median.Median SGP for the 6th grade
class
The Median Growth Percentile (MGP) is used in the Wyoming school rating system under the Wyoming Accountabilityin Education Act (WAEA), Enrolled Act 65.
