New York: New York: The State The State

Of AccountabilityOf Accountability

Presentation toStatewide Network of Middle Level Liaisons

November 2010

College and Career Ready

Students

Highly EffectiveSchool Leaders

Highly Effective Teachers

The Race is Already UnderwayIncreasing College and Career Increasing College and Career Readiness of All StudentsReadiness of All Students•Common Core State Standards

adopted•Proficiency bar raised on state

assessments to match college-readiness

•State assessments more rigorous and performance-based

•Committed to building sequenced, content-rich statewide curriculum with embedded formative assessments

•PARCC – Governing State

Regents Strategic Plan9 Goals9 Goals

Regents Strategic Goals RTTT Assurance Areas

Curriculum and Professional Development B. Standards and Assessments

Assessment B. Standards and Assessments

Teacher Preparation and Effectiveness D. Great Teachers and Leaders

Replace Failing Schools E. Turning Around Low Performing Schools

Raise Graduation Rates for At-Risk Students Sum of B, C, D, and E

Transform the Field of School Leader Preparation D. Great Teachers and Leaders

Transform NYSED A. State Success Factors

Build a P-20 Data System C. Data Systems

Early Childhood Invitational Priorities: Early Learning Outcomes

College Readiness Current 2013 Target

Points Gain

ELA Regents Exam score 75 or above 56% 69% 13Math Regents Exam score 80 or above 42% 59% 17

Graduation Rate 72% 80% 8

College Persistence 72% 76% 4

All Students

All Students Prepared to Succeed in College and All Students Prepared to Succeed in College and CareersCareersMeasurable, substantial progress toward student Measurable, substantial progress toward student successsuccess

Close the gaps in achievement for Black students, Hispanic students, English language learners, students with disabilities,

and economically disadvantaged students

Higher Education Degree Holders Earn More And Contribute More To Economic Growth

5Source: Current Population Survey, 2008

6

7 Of The Top 10 Fastest-growing Occupations

Require A Postsecondary Degree Education or training level for fastest growing occupations, 2008 to 2018

Rank Title Education or training level

1 Biomedical engineers Bachelor's degree

2 Network systems analysts Bachelor's degree

3 Home health aides Short-term on-the-job training

4 Personal and home care aides

Short-term on-the-job training

5 Financial examiners Bachelor's degree

6 Medical scientists Doctoral degree

7 Physician assistants Master’s degree

8 Skin care specialists Postsecondary vocational award

9 Biochemists and biophysicists

Doctoral degree

10 Athletic trainers Bachelor's degreeSource: Employment Projections Program, U.S. Department of Labor, U.S. Bureau of Labor Statistics

7

Entry-level and manufacturing jobs require higher skill levels than ever before

• The material that many entry-level blue-collar workers must read – technical manuals and installation instructions – is complex and critical to job performance. (Daggett 2003)

• During the current recession, U.S. manufacturers are eliminating lower-skilled jobs and moving towards automation. They need people who can operate sophisticated computerized machinery and follow complex blueprints. (NYT front-page article 7/1/10)

U.S. College Graduation Rates Have Stagnated Relative To The Rest Of The Developed World

College and university graduation rates in 1995 and 2006 (first-time graduation)

0

10

20

30

40

50

60

70

Icel

and1

Aust

ralia

1

New

Zea

land

1

Finl

and1

Pola

nd1

Den

mar

k1

Net

herl

ands

1

Nor

way

1

Swed

en1

Italy

Irel

and

Uni

ted

King

dom

1

Japa

n

OEC

D a

vera

ge

Isra

el

Uni

ted

Stat

es

EU19

ave

rage

Cana

da1,

2

Slov

ak R

epub

lic1

Port

ugal

1

Spai

n

Hun

gary

Switz

erla

nd1

Czec

h Re

publ

ic1

Aust

ria1

Ger

man

y1

Slov

enia

Gre

ece1

Turk

ey

2006 1995

1. Net graduation rate is calculated by summing the graduation rates by single year of age in 2006.2. Year of reference 2005.Countries are ranked in descending order of the graduation rates for tertiary-type A education in 2006.Source: OECD. Table A3.2 See Annex 3 for notes (www.oecd.org/edu/eag2008 )

%

15th

2nd

Decline in relative position of U.S. from 1995

to 2006

8

Nearly A Quarter Of Students In All NYS Two- And Four-year Institutions Of Higher Education Take Remedial Courses

New York State First-Time Students Taking Remedial Coursework

By Type of Institution, 1998-2007

24%13%

44%

0%

20%

40%

60%

80%

100%

4-Year & 2-Year 2-Year 4-Year

1998

1999

2000

2001

2002

2003

2004

2005

2006

2007

Year

2-Year Institutions4-Year InstitutionsAll Institutions

Source: NYSED Administrative Data for all Public, Independent and Proprietary 2- and 4-year institutions of higher education 9

Institutions Of Higher Education Consider A Regents Exam Score of 75 to 85 The Bare Minimum For College

Readiness

Admissions directors of two- and four-year public and private colleges in the Western NY, Central NY, Hudson Valley and New York City metropolitan regions say that:– 75 to 85 on Regents exams is considered by selective

schools (as part of their holistic review of applicants) the lower threshold for admissions

– SUNY campuses use 85 as a mark of solid competence; below 75 is a mark of “inadequately prepared”

– 75 on Regents exams is a threshold for placement in remediation for CUNY

– 75 on Regents is considered roughly equivalent to a 500 on the SAT and serves as a threshold for remediation

Admissions directors of two- and four-year public and private colleges in the Western NY, Central NY, Hudson Valley and New York City metropolitan regions say that:– 75 to 85 on Regents exams is considered by selective

schools (as part of their holistic review of applicants) the lower threshold for admissions

– SUNY campuses use 85 as a mark of solid competence; below 75 is a mark of “inadequately prepared”

– 75 on Regents exams is a threshold for placement in remediation for CUNY

– 75 on Regents is considered roughly equivalent to a 500 on the SAT and serves as a threshold for remediation

10

Students who score below an 80 on their Math Regents have a much greater likelihood of being placed in a

remedial college course

ArithmeticElementary

AlgebraIntermediate

Algebra**College Algebra

Pre-Calculus

Calculus

Less than 55 68.3% 29.7% 0.0% 1.4% 0.7% 0.0% 0.0%

55 to 64.9 61.4% 33.7% 0.6% 3.2% 0.8% 0.3% 0.0%

65 to 69.9 38.9% 44.7% 1.8% 8.0% 4.8% 1.8% 0.7%

70 to 79.9 14.7% 24.6% 5.9% 23.5% 21.3% 8.1% 1.8%

80 to 89.9 0.8% 2.8% 4.3% 17.3% 30.6% 32.3% 12.0%

Above 90 0.0% 0.2% 0.5% 3.4% 12.7% 39.2% 44.2%

**Intermediate Algebra is considered a remedial course in some schools in the CUNY system and a credit-bearing course in others.

Totals sum to 100 percent along rows, but not down columns.

Source: CUNY Office of Institutional Research and Assessment, Math A Regents; all CUNY 2- and 4-year institutions

11

12

Students who score above an 80 on their Regents exam have a good chance of earning at least a C in college-level math

12Source: CUNY Office of Institutional Research and Assessment; all CUNY 2- and 4-year institutions

Students At The Former Level 3 Proficiency Standard On Their 8th Grade Math Exam Had Less Than a 1 in 3 Chance of

Earning an 80 On Their Math Regents

Source: NYSED Administrative Data, Math A Regents, 2006-2010 Cohort 13

Current cut score of 650

gives students 30% chance of scoring 80 on

Regents

Current cut score of 650

gives students 30% chance of scoring 80 on

Regents

Students In High-Needs Districts At The Former Level 3 Proficiency Standard On Their 8th Grade ELA Exam Had About A 50-50 Chance Of Earning A 75 On Their ELA

Regents

Source: NYSED Administrative Data, English Regents, 2006-2010 Cohort 14

Current cut score of 650 gives

students in high needs districts 57% chance of scoring 75 on

Regents

Current cut score of 650 gives

students in high needs districts 57% chance of scoring 75 on

Regents

Since 2006, New York’s 8th Grade Students Had Improved Substantially On The State Math Test, But Their Performance On The National Assessment of Educational Progress (NAEP)

Had Remained Nearly Flat

15

0

0.1

0.2

0.3

0.4

0.5

0.6

2005 2006 2007 2008 2009

Ch

an

ge i

n S

Ds

NYS_Public NY NAEP_Public

Source: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2005-2009 Mathematics Assessments. NYSED Administrative Data

Possible Reasons For This Divergence Include:

• Increased learning by students on content tested by state assessments

• Instruction focused on narrow strands of content that appear on the state tests

• Differences between what the state assessments and the NAEP measure

• Technical issues in equating scores from year to year

• Increased learning by students on content tested by state assessments

• Instruction focused on narrow strands of content that appear on the state tests

• Differences between what the state assessments and the NAEP measure

• Technical issues in equating scores from year to year

16

17

GradeMath ELA

8 673 658

7 670 664

6 674 662

5 674 666

4 676 668

3 684 662

Grade 3-8 Proficiency Cut scores have been raised from the 650 that was in place in the

past.

Improving The State Assessments Year By Year

• 2010– Raised cut scores for Level 2 and Level 3 Proficiency– Included 25 to 30 percent more tested performance

indicators on the math assessment– Added audit items

• 2011-2013– Increase number of items on the Math and ELA tests– Test more new performance indicators– Make test items more difficult to predict– Improve quality of ELA Regents exam

• 2014-2015– Use the Common Core Assessments

• 2010– Raised cut scores for Level 2 and Level 3 Proficiency– Included 25 to 30 percent more tested performance

indicators on the math assessment– Added audit items

• 2011-2013– Increase number of items on the Math and ELA tests– Test more new performance indicators– Make test items more difficult to predict– Improve quality of ELA Regents exam

• 2014-2015– Use the Common Core Assessments

18

19

Changes in Grade 3-8 ELA and math test administration

Time Adjusted Grade 3-8 ELA and math cut scores

SWDs and AYP: Sunset of USED’s 34 point flexibility; Declassified Students in Subgroup for 2 Additional Years

New Graduation Rate Goal and Targets

Accountability Changes: 2009-10

20

Schools Identified for Improvement, Corrective Action or Restructuring in 2009-2010

Improvement Status NYC Rest of State Total

Improvement (year 1) - Basic 34 42 76

Improvement (year 1) - Focused 2 7 9

Improvement (year 1) - Comprehensive 13 4 17

Improvement (year 2) - Basic 23 29 52

Improvement (year 2) - Focused 5 3 8

Improvement (year 2) - Comprehensive 12 5 17

Corrective Action (year 1) - Focused 16 17 33

Corrective Action (year 1) - Comprehensive 6 4 10

Corrective Action (year 2) - Focused 17 14 31

Corrective Action (year 2) - Comprehensive 3 0  3

Restructuring (year 1) - Focused 20 14 34

Restructuring (year 1) - Comprehensive 8 14 22

Restructuring (year 2) - Focused 26 13 39

Restructuring (year 2) - Comprehensive 7 5 12

Restructuring (Advanced) - Focused 64 13 77

Restructuring (Advanced) - Comprehensive 41 18 59

 TOTAL 297 202 499

• How do we address the potential large increase in schools and districts not making AYP?– Increase student performance?– Reset Annual Measurable Objectives?– Have approved a growth model that allows a significant

percentage of schools and districts to make AYP?

Issues for 2010-11 Accountability

21

NYSED Growth Model: Draft Working Principles

22

Growth shall be measured in terms of growth to standards and percentile growth across a minimum of two points in time ( i.e., comparing a student to all other students in the State who have comparable assessment histories).

For purposes of school and district accountability, no factors other than a student’s assessment history should be the basis of making a determination regarding the student’s performance in relationship to growth.

The model shall only utilize such data as is collected through State data collection processes to make a growth determination.

The growth model shall be designed to make its first growth determinations using 2011-12 school year results for English language arts and mathematics in grades 4-8.

The growth model shall be designed to meet the requirements of Chapter 57 of the Laws of 2007 that require that such growth model be one that is determined by external experts in educational testing and measurement to be valid, reliable and in accordance with established standards for educational and psychological testing.

The growth model shall be designed so that it can be integrated into NY’s ESEA accountability system.

The growth model shall be designed to inform determinations regarding teacher and principal effectiveness based on student growth beginning with ELA and math results in grades 4-8 in 2011-12.

NYSED Growth Model: Draft Working Principles

231

The model shall be sufficiently flexible so as to remain valid and reliable as NY’s assessment program is modified over time.

The model shall use an “open architecture.” The methodology used to calculate growth should be transparent.

The model shall be able to measure growth for all students, including those who perform at the low basic and high advanced levels.

The model should address key design issues, such as students who take alternative assessments and students who have been taught by more than one teacher during the school year.

To the maximum extent, consistent with good psychometric practice, a student’s entire state assessment history should be used to determine a students’ growth performance.

The growth model shall be extended to include measures of growth for students at the high school level in, at a minimum, ELA and math beginning with 2012-13 school year results.

24

Growth Percentile: An Example

Starting proficiency

Nex

t ye

ar’s

pro

fici

ency

L 3

L 2L 3

• What should be the graduation goal and progress target for high school graduation rate beginning with 2011-12 school year results to reflect:– New federal graduation rate definition– AYP graduation rate accountability based on

disaggregated groups– Use of five year extended graduation rate?

Issues for 2010-11 Accountability

25

• How should student growth be measured for purposes of teacher and principal evaluation:– Grade 4-8 ELA and math teachers and their principals beginning with the

2011-12 school year results;– All other teachers and principals beginning with 2012-13 school year results:

• Teachers with one state assessment• Teachers of grade three and below• Teachers of other subjects and grades

• Regents Task Force on Teacher and Principal Effectiveness:– Charged to advise NYSED on implementing regulations for §3012-c – Membership: 1/3 Teachers, 1/3 Principals, 1/3 stakeholders– Preliminary recommendations to Board scheduled for January 2011,

with recommendations for 1st phase in Spring 2011

Issues for 2010-11 Accountability

26

– Purposes: Improve Accountability and Teaching and Learning by measuring students’ college and career readiness. Students meeting the standards will be eligible for placement in college courses without remediation.

– Assessment results should:• Be comparable across states• Meet internationally rigorous benchmarks• Assess student growth over time

– Results should:• Inform decisions about promotion and graduation• Teacher and leader evaluations• School and district accountability

– Design should include:• End of year component• Through course periodic components• Computer Based Testing and Innovative Items Types• Common Performance Levels and Growth Measures• Released Items and Item Analysis• Model Instructional Supports• Comparable testing conditions and accommodations

– Design should consider development of K-2 Assessments

Assessment of Readiness for College and Career: Guiding Principles

27

And… on the horizon

• Re-authorization of the Elementary, Secondary Education Act

• Continuing Fiscal Challenges

28

Key NYSED DeliverablesKey NYSED Deliverables2011-12

Training on implementation of Common Core Standards for ELA and Math Implementation and training of school-based Inquiry Teams Training on implementation of new performance evaluations for teachers and principals in

ELA and Math

TodayToday – Measuring the impact of delivery to teachersTomorrowTomorrow – Measuring the impact on student achievement

2012-13 Training on access and use of Early Warning Data System Training on access and use of Instructional Improvement Reporting System Training on implementation of ELA and Math curriculum models aligned with Common Core Training on use of ELA and Math formative assessments aligned with Common Core Implement new performance evaluations for all remaining teachers and principals

2013-14 Training for implementation of curriculum models for science, social studies and arts Training on use of formative assessments in science, social studies and arts Implement performance evaluations for all teachers and principals utilizing the State- developed value-added growth model

2014-15 Training for implementation of PARCC summative assessments in ELA and Math

30

Additional Questions

Contact: Ira SchwartzAssistant Commissioner for Accountability

ischwart@mail.nysed.gov

718 722-2796