Preparing for Cycle III School and District Accountability

Ratings and AYP Determinations

Information Sessions August 26 & 27, 2004

Juliane Dow, Associate Commissioner

Accountability, Improvement Planning and Targeted Assistance

Massachusetts Accountability System Basics

School Performance Ratings are• Biannual (2002, 2004, 2006…), as required by state law• Descriptive terms • Ratings for Performance and Improvement• Based on aggregate MCAS student results

AYP determinations are • Issued every year, as required by federal law• For students in aggregate and for subgroups• Based on four factors:

participation, performance, improvement, and attendance(elementary and middle schools) or graduation rate (high schools)

Why Proficiency?

Proficiency in core academic subjects is the gateway to:

• Opportunities for higher education

• Meaningful choices for employment in our 21st-century high tech economy

• Full participation in community and civic life

Calculating the Composite Performance Index (“CPI”)

• Points awarded based on number of students performing at each level

• Different performance measures for students with significant cognitive disabilities participating in MCAS- Alt

• Just arithmetic! Multiply, add, then divide.

Table 1: MCAS Proficiency Index Table 2: MCAS-Alt Index

For students taking standard MCAS tests(and MCAS-Alt for students who do not have

significant cognitive disabilities)

For students with significant cognitivedisabilities taking MCAS-Alt

(up to 1% of all assessed students in a district)

MCAS SCALED SCORE(or MCAS-Alt equivalent)

POINTSAWARDED MCAS-ALT SCORE

POINTSAWARDED

200 – 208Failing/Warning – Low(Awareness)

0 Portfolio not submitted 0

210 – 218Failing/Warning – High(Emerging/Progressing)

25 Incomplete 25

220 – 228(Needs Improvement – Low)

50 Awareness 50

230 – 238(Needs Improvement – High)

75 Emerging 75

240 – 280(Proficient/Advanced)

100 Progressing 100

Massachusetts NCLB Performance Targets

for ELA and Mathematics 2002 - 2014

60.8

68.7

53.0

76.5

84.3

92.2

75.6

70.7

80.5

85.4

90.2

95.1

100.0

40

50

60

70

80

90

100

2001 & 02 2003 & 04 2005 & 06 2007 & 08 2009 & 10 2011 & 12 2013 & 14

Co

mp

osi

te P

erf

orm

an

ce I

nd

ex

Math ELA

(39.7% P&A)

(19.5% P&A)

Cycle III

100

90

80

70

60

40

20142002

2004

200620

08

2010

2012

Com

posi

te P

erfo

rman

ce I

ndex

MA Performance and Improvement Ratings

Critically Low

Very Low

Low

Moderate

High

Very High

On Target

Declined

No Change

Improved, Below Target

Above Target

X

XXX

X

Baseline

B

What is AYP?• AYP stands for

Adequate Yearly Progress

• Means progress towards 100% of students achieving proficiency by 2014

• Measures progress against specific expectations each year

Cycle III AYP Criteria

70% + CD Attainment

B: Performance

C: Improvement

A: Participation

D: Attendance Graduation Rate

95%

CPI

100 - Baseline CPI Time (# of Cycles)

or92% or ^ 1%

ELA: 75.6Math: 60.8

Two Ways to Make AYP:

(A+B+D) Participation + Performance

(A+C+D) Participation + Improvementor

+ Attendance or Graduation Rate

+ Attendance or Graduation Rate

New In 2004• Assessment results for LEP students in US

schools for first year not included in calculation of CPI

• Minimum sample size for subgroups is greater of

a) 40 orb) 5% of students assessed or 200

• Additional indicator (attendance or graduation rate) considered in all AYP determinations

18 AYP Determinations Per School!

Each school and district receives AYP determinations

• in ELA/reading and in mathematics

• for students in the aggregate and each NCLB subgroup

1999 2000 2001 2002 2003 2004 Status in '05Action

Required

No No No No No No R - Y1 C & SS & Plan

No No No No No Yes CA-Y2 C & SS & CA

No No No No Yes No IMP - Y4 C & SS

No No No No Yes Yes None

No No Yes Yes No No IMP - Y1 C

Yes Yes No No Yes No IMP - Y3 C only

Yes Yes No No No No CA-Y1 C & SS & CA

Yes Yes No No No Yes IMP - Y3 C & SS

No No No No Yes Yes None

Yes Yes Yes Yes No Yes None

R RestructuringCA Corrective Action

IMP Identified for Improvement

SS Supplemental Services

School Accountability Status: ELA / MATH

Preliminary AYP Data ReviewAugust 25 - August 31• Review preliminary 2004 End-of-Cycle III AYP data files

for schools and for district as a whole (data files posted to MADOE security portal

• Attend explanatory workshops (optional)• Identify and report any AYP data or determination

discrepancies evident in preliminary data run.

September 1• Notice to districts and schools on preliminary list of those

identified for improvement, corrective action or restructuring

September 10• Preliminary list of districts and schools identified for

improvement, corrective action or restructuring released to public.

AYP Reporting Schedule

September 29 - October 6• Districts preview End-of-Cycle School and

District Accountability System (SDAS) reports (Cycle III performance and improvement ratings and 2004 AYP determinations)

• Identify and report any data or determination discrepancies for resolution prior to public release of Cycle III accountability reports.

October 8• Public release of Cycle III Accountability reports.

Consequences When A School Does Not Make AYP

Schools that fail to make AYP for two consecutive years in either subject for any group must be “identified for improvement”

• All schools identified for improvement must develop a plan for improving student performance

• Title 1 schools identified for improvement must also– offer school choice (year 1 in improvement status)– offer supplemental services (year 2 if fail to make AYP

in year 1)

Notice to Parents: School Choice and Supplemental Services

Title 1 Schools identified for Improvement, Corrective Action or Restructuring must offer school choice if alternative assignment options exist.

If school choice is required, parents must be notified, in writing, as soon as possible after districts receive notice of the designation.

Corrective Action and Restructuring

• Schools identified for improvement that fail to make AYP for 2 additional years must be identified for corrective action.

• Schools that did not make AYP in 2000, 2001, 2002, and 2003 were identified for corrective action last fall.

• Schools in corrective action that fail to make AYP in 2004 are subject to restructuring.

Appeal Standards and Processes• Objective is to render fair, reliable decisions• Appeals may be filed to address perceived erroneous or

unfair determinations where:– there are student enrollment data discrepancies– data is missing or misplaced– circumstances render the reported data invalid or unreliable– application of the standard rules would render a result

inconsistent with statutory intent, public policy, or sound professional judgement

– school or district provides evidence that it has taken specific action to improve the performance of an identified subgroup and MCAS results demonstrate significantly improved results for that subgroup in the following year.

School Accountability Steps

1. Performance Ratings & AYP Determinations

2. Panel Review

3. Determination of Under-performance

4. Fact-Finding

5. Performance Improvement Mapping

6. Improvement Plan Reviewed by Board

7. DOE Monitors Improvement Initiatives

8. Year 2 Follow-up Review

9. Determination of Chronic Under-performance

Link Between NCLB and Our State’s On-Site Performance Review Process

Schools identified for corrective action or restructuring based on aggregate results undergo state panel review.

Panel Review Questions:1. Is there a sound plan for improving student

performance?2. Are the conditions in place for successful

implementation of such a plan?

Where Can I Find Accountability Information When I Need It?

Visit the Department of Education website, www.doe.mass.edu. – Click on Assessment and Accountability to find

explanatory materials about the MA School and District Accountability System

– Go to “School and District Profiles” to find performance data for the state, a district or school.

Send questions to: ATA@doe.mass.edu