Embed Size (px)
Citation preview
NYC ACHIEVEMENT GAINS COMPARED TO OTHER LARGE CITIES SINCE 2003Changes in NAEP scores 2003 -2011
Leonie Haimson & Elli Marcus
Class Size Matters
January 2012
www.classsizematters.org
NAEP Scores: Why are they important?• The National Assessment of Educational Progress (NAEP) is largest continuing
assessment of the knowledge and abilities of American students.
• NAEP assessments are given by the federal govt. every two years to statistical samples of students, change little over time & are low-stakes, and so can be used as a reliable metric to compare achievement trends among states and urban districts.
• The Trial Urban District Assessment (TUDA) has been given in 10 large cities incl. NYC since 2003 in four categories: reading and math in 4th and 8th grades.
• What follows is an analysis of the changes in NYC NAEP scores since 2003, when Bloomberg’s educational policies were first implemented, compared to changes in scores in the 9 other cities, plus large cities in general (w/ at least 250,000 inhabitants).
How did we compare trends among the large urban districts?• Since overall scores can change depending on changes in student
population, we compared changes in scores since 2003 for six major NYC subgroups (white, black, Hispanic, Asian, free lunch and non-free lunch students) compared to their peers in other large cities.
• Only major subgroups whose results we did not compare were students with disabilities and English language learners, since rates of identification and exclusion from testing differ widely among the ten cities.
• Our comparisons give insights into where NYC stands nationally, and allows us to assess the reality of DOE’s claims of great improvement.
• These comparisons give insight into where NYC stands nationally and provides a robust examination of the DOE’s claims o
When 2011 NAEP scores were released this fall, NYC DOE claimed great progress *
• Claim: “NYC students have improved significantly on three of the four math and reading tests between 2003 and 2011.”
• Reality: This is true in nearly every city tested since 2003.
• Claim: “….since 2003, the gap between black and white students in New York City has narrowed on all four exams, and on all four since 2009.”
• Reality: There has been no statistically significant narrowing of the achievement gap between any of the racial/ethnic groups in NYC in any subject tested since 2003.
*Source: NYC DOE Press release , December 7, 2011
DOE’s other unfounded claims of progress• Claim: “’On all four tests, low-income students in NYC now outperform their
peers across the nation, and that’s a reason to be proud,’ said Chief Academic Officer Shael Polakow-Suransky.”
• Reality: In 2003, NYC low-income students already outperformed their peers nationwide in all four categories tested, and since then have made fewer gains than peers in several other cities.
• Claim: “By the ‘gold standard’ for measuring academic progress, our students have made impressive gains since 2003—especially compared to their peers across New York State and the nation,” said Chancellor Walcott.”
• Reality: When measured across subgroups, NYC students have made less academic progress since 2003, compared to their peers, in every other city except one.
*Source: NYC DOE Press release , December 7, 2011
NYC comes in 2nd to last among all 10 cities + “large city” category when NAEP score gains are averaged across 6 subgroups*
Cleveland NYC Charlotte large city Chicago SD Houston DC LA Boston Atlanta0
2
4
6
8
10
12
14
16
18
1
4.3
7.9 8.8 8.910.3 10.4 10.9
12.4 12.915.3
*Subgroups include white, Hispanic, Black, Asian, free-lunch & non-free lunch
Test score gains since 2003, averaged across all four categories: reading & math in 4th & 8th grades
Scores by subgroup: In NYC, Black students scores rose less than their peers in most other cities
• In 4th grade reading, NYC black students dropped from tied for 3rd to 4th place among all cities since 2003.
• In 8th grade reading, NYC blacks were tied for 2nd and dropped to 3rd.
• In 4th grade math, NYC blacks dropped from 3rd to 4th place.
• in 8th grade math, NYC blacks went from 3rd to tied for 4th place.
NYC scores by subgroup: Black Students 4th and 8th grade reading and math gains in average scale scores since 2003
Clevela
nd
Houst
on
Charlo
tte
Chicag
oNYC LA SD
Bosto
n
large
city DC
Atlant
a-4
6 68 8 9 9 9 10
2023
change in 4th grade reading scores 2003-2011
Clevela
nd
Charlo
tte SD LANYC
Atlant
a
Houst
on DC
Chicag
o
large
city
Bosto
n
13
6 7 7 8 810 10 10
14
Change in 4th grade math scores 2003-2011
DC
Clevela
nd
Bosto
n
Chicag
oSD
Houst
onNYC
large
city
Charlo
tte LA
Atlant
a
-5 -4
1 2 2 3 3 46
912
change in 8th grade reading scores 2003-2011
Clevela
nd SDNYC DC
Charlo
tte LA
Houst
on
Chicag
o
large
city
Bosto
n
Atlant
a
04
9 9 1012 12
15 15
21 21
change in 8th grade math scores 2003-2011
Subgroup: White students fell sharply behind their peers in other large cities since 2003, especially in 8th grade reading & math
• In 4th grade reading, NYC white student scores dropped from 5th to 7th place.
• In 4th grade math, NYC white students dropped from 5 th place to 8th place.
• In 8th grade reading, NYC white students dropped from tied for 2nd to 7th place, and came in last in score gains.
• In 8th grade math, NYC white student scores dropped from 4 th to 8th place and came in last in score gains.
NYC scores by subgroup: White Students
Clevela
nd DC
Atlant
aNYC
Chicag
o
large
city
Charlo
tte
Houst
on LA SD
Bosto
n
1 1 14 5 6 7 8 8 9
16
change in 4th grade reading scores 2003-2011
NYC
Charlo
tte
large
city
Chicag
oSD LA
Bosto
n
Clevela
nd
Houst
on
1
5 5 6 6 7 810
13
Change in 8th gr reading scores
2003-2011 white students
Clevela
nd LANYC
Houst
on
Charlo
tte
large
city DC
Atlant
a
Chicag
oSD
Bosto
n-1
24 5
7 810 11 11
15
21
change in 4th grade math scores 2003-2011
NYC
Clevela
nd
Charlo
tte
large
city
Atlant
a LA
Houst
on
Bosto
nSD
Chicag
o
38 10 10 11
14 16 16 18 20
change in 8th gr math scores
2003-2011 white students
Subgroup: Hispanic Students fell sharply behind peers since 2003
• In 4th grade reading, NYC Hispanic students dropped from 1st place among large cities to tied for 4th.
• In 4th grade math, NYC Hispanic students dropped from third place to sixth place among other large cities.
• In 8th grade reading, NYC Hispanic students dropped from 2nd to 5th place, with a net negative change in scores.
• In 8th grade math, NYC Hispanic students came in last place in score gains, falling from third place to 7th place.
NYC scores by subgroup: Hispanic Students
Clevela
ndNYC
Chicag
oSD
Houst
on
large
city LA
Charlo
tte
Bosto
nDC
Atlant
a-5
4 5 6 6 6 710
1317
28
Change in 4th grade reading scores 2003-2011
-8-1
0
6 7 7 8
12 13
change in 8th grade reading scores 2003-11
NYC DC
Clevela
nd
Charlo
tte
large
city
Chicag
oSD LA
Houst
on
Bosto
n
1
79 10 11 12
15 1517
19
change in 8th grade math scores 2003-2011
Clevela
nd
Chicag
o
Charlo
tteNYC LA
large
city
Houst
on SD DC
Bosto
n-2
6 7 7 9 9 10 1318 19
Change in 4th grade math scores
2003-2011
Subgroup: Asian Students were the only NYC group to make substantial gains compared to peers in other cities.
• 4th grade reading, NYC Asian student scores dropped from first place to second place, and placed fourth in overall score improvement among large cities.
• In 4th grade math, Asian student scores dropped from second place to third place among large cities.
• In 8th grade reading, NYC Asian student scores moved up from third place to second place among large cities.
• In 8th grade math, NYC Asian student scored moved up from third place to second place
Subgroup: Asian Students
large city SD NYC Boston LA Charlotte
1 2 3 3
7
15
Change in 4th grade read-ing scores 2003-2011
Asian students
Chicago Boston SD NYC large city LA
-4
67
910
12
change in 8th grade reading scores 2003-2011 Asian students
large city NYC Charlotte LA SD Boston
3 46
10 10
16
change in 4th grade math scores 2003-2011 Asian
students
Chicago Charlotte SD large city NYC Boston LA
10 11
15 15
18 19 20
change in 8th grade math scores 2003-2011 Asian students
Changes in demographics: Asian student pop rising faster in NYC than elsewhere; otherwise progress on NAEPS would have been even smaller
Atlanta Boston Chicago DC Houston LA large city NYC02468
101214161820
0 0
31
3
65
8
1
8
5
23
68
19
Asians as % of total students tested4th grade reading
2002
2011
Atlant
a
Charlo
tte
Chicag
o
Clevela
nd DC
Houst
on LA
large
city
NYC SD0
4
8
12
16
20
0
8
4 31 2
6 7
12
18
1
8
5 6
2 3
68
19
15
Asians as % of total students tested4th grade math
2003
2011
NYC scores by subgroup: Free Lunch students had only middling gains• In 4th grade reading, NYC free lunch student scores remained in
1st place but placed behind five other large cities in gains since 2003.
• In 4th grade math, NYC free lunch student scores dropped from second place to third place, and placed fifth in score gains among large cities.
• In 8th grade reading, NYC free lunch student scores remained in 1st place but placed behind three other large cities in score gains.
• In 8th grade math, NYC free lunch student scores dropped from 1st place to 3rd place.
Subgroup: free lunch
Clevela
nd SD
Houst
on DC LANYC
Chicag
o
Bosto
n
large
city
Charlo
tte
Atlant
a-2
6 67 7 7
8 8 8
1112
change in 4th grade reading scores 2003-2011
DC
Clevela
nd
Bosto
n
Chicag
oNYC SD
Houst
on
large
city
Charlo
tte LA
-4
0
23
5 56
78
11
change in 8th grade reading scores 2003-2011
Clevela
nd
Charlo
tte LANYC
Atlant
a
Chicag
o
Houst
on
large
city DC SD
Bosto
n
1
6 79 9 9 10 10 11 12
16
change in 4th grade math scores 2003-2011
Clevela
ndNYC
Charlo
tte DC SD
large
city
Chicag
o LA
Houst
on
Bosto
n
Atlant
a
3
9 11 11 13 14 15 17 17 19 21
change in 8th grade math scores 2003-2011
NYC non-free lunch students made the smallest gains of any city in every category; and dropped sharply at 8th grade
• In 4th grade reading, NYC non-free lunch students fell from 1st place to 2nd place.
• In 4th grade math, NYC non-free lunch students fell from 2nd place to 3rd place.
• In 8th grade reading, NYC non-free lunch student scores dropped 11 points – the only city where scores dropped – and fell from 1st place to 8th place.
• In 8th grade math, NYC non-free lunch students dropped seven points – the only city where scores dropped -- and fell sharply from 1st to 8th place
• In 8th grade reading and math, basic and proficient levels of non-free lunch also dropped sharply.
Subgroup: non-free lunch
NYC
Chicag
o
Charlo
tte
large
city
Atlant
a
Bosto
n LA SD
Houst
on DC
1 36
914 15 16 16 18
28
change in 4th reading scores 2003-2011
NYC
Charlo
tte
Chicag
o
large
city
Houst
on
Bosto
nSD DC
Atlant
a LA
-11
5 6 8 10 10 11 11
1924
change in 8th grade reading scores 2003-2011
NYC Charlottelarge city Houston LA Chicago Boston SD DC
58
1014
16 1618 19
29
change in 4th grade math scores2003-2011
NYC
Chicag
o
Charlo
tte
large
city
Houst
on
Bosto
nSD DC
Atlant
a LA-7
1014 14 15
20 2124
27
38
change in 8th grade math scores2003-2011
NYC is ONLY city where proficiency levels in 8th grade reading and math have dropped for non-free lunch students
at or a
bove basic
at or a
bove pro
ficie
nt0
10
20
30
40
50
60
70
80
90 82
49
75
41
8th grade math for non-free lunch students
20032011
at or above basic
at or above proficient
0
10
20
30
40
50
60
70
80
90
100
87
48
79
34
8th grade reading for non-free lunch students
20032011
All other cities made gains in 8th grade proficiency in reading & math for non-free lunch students, while in NYC they dropped
NYC Chicago Charlotte large city Boston Houston SD DC Atlanta LA
-20
-10
0
10
20
30
40
-7
7 711
14 15 16
2429
37
-8
1015 14
22
16
24 23 2530
change in % non-free lunch students at or above basic & proficient in 8th grade math 2003-2011
diff basic
diff proficient
NYC Chicago Charlotte Boston large city SD Houston DC Atlanta LA
-20-15-10-505
10152025
-8
25 6 7
10 10 12
19 21
-14
138
139
1511
17 1922
change in % non-free lunch students at or above basic & pro-ficient in 8th grade reading 2003-2011
diff basic
diff proficient
Summary of findings:• When analyzing subgroup performance, NYC’s relative progress since 2003
compared to other large cities has been mediocre to poor.
• NYC came in 2nd to last in NAEP gains among 10 cities and “large city” category tested since 2003 when averaged across six subgroups.
• All NYC subgroups fell in ranking, compared to peers in other large cities, with White, Hispanic and non-free lunch students dropping most sharply.
• White students made the smallest gains compared to their peers in other cities in both 8th grade reading and math; Hispanics in 8th grade math.
• Asian students were only NYC subgroup to advance in ranking in any subject or grade;
• NYC was only city in which non-free lunch students scored lower in 2011 than in 2003, in both 8th grade reading and math, and their proficiency levels also dropped sharply.
What about mayoral control? Two districts under mayoral control made least progress & on average, cities with elected school boards have done better
Cleveland NYC Charlotte large city Chicago SD Houston DC LA Boston Atlanta0
2
4
6
8
10
12
14
16
18
1
4.3
7.98.8 8.9
10.3 10.4 10.912.4 12.9
15.3
Cities with mayoral control since 2003 or earlier in red; DC has had mayoral control since 2007.
What else do these results suggest?
• The administration’s aggressive free-market strategies of high-stakes accountability, school report cards, “fair student funding”, principal empowerment, and the closing of more than one hundred schools & the opening of more than 400 new schools & charters, while allowing class sizes to increase sharply, have not worked to increase achievement compared to cities elsewhere.
• In fact, the relative positions of white, Hispanic and non-free lunch students in NYC have all dropped substantially, with the declines especially sharp at the 8th grade level.
