1 Mississippi Statewide Accountability System Adequate Yearly Progress Model Improving Mississippi Schools Conference June 11-13, 2003 Mississippi Department

1

Mississippi Statewide Accountability System

Adequate Yearly Progress Model

Improving Mississippi Schools ConferenceJune 11-13, 2003

Mississippi Department of EducationInnovation and School Improvement

2

Mississippi Statewide Accountability System

Steve Hebbler, Director Office of Research and Statistics

[email protected]

Kris Kaase, DirectorOffice of Student Assessment

[email protected]

3

Statewide Accountability SystemSchool Districts

Accreditation Status

AYP Model

Annual Accountabilit

y Designation

4

Statewide Accountability SystemSchools

Achievement Model

Growth Model

AYP Model

Annual Accountabilit

y Designation

School Performanc

e Classificati

on

5

Which Students Must Be Assessed?

All eligible students are required to be assessed under:

Mississippi Public School Accountability Standards 2001

MS Code 37-16-3 1972 Annotated

P.L. 107-110 Section 1111(b)(3)(C)(ix)

6

What does assess all students mean?

Students in grades 2-8 in reading, language, and mathematics

Non-graded students ages 7-15 (age on September 1) Students enrolled in Algebra I and English II Students required to take the Biology I test, U.S. History

test, or FLE for graduation Students enrolled in vocational programs in which

non-disabled students are tested Exemption for LEP students in their first or second year

in a U.S. school without sufficient English language proficiency in effect for Spring 2003 only

7

What about the 95% rule?

95% is the minimum participation rate in assessments required under AYP.

The rule acknowledges that even with scheduled and make-up test administrations there will be some unusual circumstances in which students are not assessed despite every effort.

8

Which Students Determine the Participation Rate?

Denominator Students enrolled in the grade level or course in

the 8th month. Non-graded students ages 7-15 (age on

September 1).

Numerator Students that took a MCT, Instructional Level

Test, or Subject Area Test whose score was not excluded.

Students that took an alternate assessment.

9


The appeals process plus a new MSIS ID verification/correction process developed in spring 2003 provide ways for schools and districts to detect data problems resulting from incorrectly coded MSIS ID numbers and to provide corrected information to MDE to insure accurate participation rates.

10


Special Rules for Algebra I Required because achievement has to be counted

at the grades 10-12 levelDenominator Students enrolled in the 10th grade levelNumerator Students that took the Algebra I test in 10th grade

that school year Students that took the Algebra I test at any time

prior to their enrollment in 10th grade at any school in the state (first test score will be used for school accountability)

11


Special Rules for Algebra I – 2002-03 Only Required because testing of all students enrolled in

Algebra I for the first time has not been required previously

Denominator Students in the 10th grade enrolled in Algebra I for

the first timeNumerator Students that took the Algebra I test at 10th grade

12


<-----------------BASIC AYP------------------> | 2001/2002 TEST DATA | MCT 2002 TESTED | 2000/2001 TEST DATA | OTHER INDICATORS | | <----------------------> | <-----------------> | <----------------------> | <--------------> | SUB | N-COUNT PROFIC CONF MET | NUM NUM PCT MET | PROFIC REQD MET | GROWTH GRAD MET | MET GRP | MIN=40 INDEX INTER OBJ | ELIG TEST TEST 95% | N-COUNT INDEX DIFF DIF | INDEX RATE ONE | AYP --- | ------- ------ ----- --- | ---- ---- ----- --- | ------- ------ ----- --- | ------ ----- --- | ---- ALL 257 3.60 3.31 YES 285 273 95.8 YES 250 -1.69 0.17 YES 0.070 YES YES IEP 15 -6.62 13.68 YES 16 15 93.8 N/A 9 -12.53 1.25 YES N/A <MIN LEP 0 N/A 0 0 N/A 0 N/A N/A <MIN ED 175 -2.25 2.22 NO 185 180 97.3 YES 169 -3.93 0.39 YES 0.020 YES *YES ASI 0 N/A 0 0 N/A 0 N/A N/A <MIN BLK 130 3.50 3.20 YES 130 125 96.2 YES 128 3.10 0.31 YES 0.080 YES YES HIS 7 2.47 13.87 YES 8 7 87.5 N/A 1 -3.75 0.38 YES N/A <MIN NAM 0 N/A 0 0 N/A 0 N/A N/A <MIN WHT 120 3.70 3.42 YES 131 126 96.2 YES 117 4.64 0.46 NO 0.060 YES YES

13

Students Included for Accountability (Achievement, Growth, & AYP)

Spring Testing Data (MCT and Traditional Schedule SATP) If Student’s End of Month 8 School = Same School on 6 of the 7

Earlier End of Month Records (Month 1 through Month 7) the student is included. (Represents 75% of instructional time)

If Student’s End of Month 7 School = Same School on all 6 of the Earlier End of Month Records (Months 1 through 6) the student is included. (Represents 75% of instructional time)

Fall Testing Data (SATP Semester/Block Schedule) If Student’s End of Month 3 School = Same School on End of

Month 1 and End of Month 2 Records the student is included. (Represents 67% of instructional time)

Spring Testing Data (SATP Semester/Block Schedule) If Student’s End of Month 8 School = Same School on End of

Month 5, 6, and 7 Records the student is included. (Represents 75% of instructional time)

14

Students Included in Accountability/Proficiency Determination


<--------------------------SAFE HARBOR-------------------------->

15

How Many Students?

Minimum Number of Students (4 definitions for 4 purposes)

Minimum Number for Reporting = 10

Minimum Number for Achievement Model = No Minimum

Minimum Number for Growth Model = 10

Minimum Number for AYP Model = 40 in the AYP group summed across the grades represented in the school (determined separately for reading/language and mathematics)

16

AYP Groups

There are nine groups for which Adequate Yearly Progress is measured. All Students Students with Disabilities Economically Disadvantaged Students Limited English Proficient Students Asian Students Black Students Hispanic Students Native American Students White Students

17

AYP Groups


18

AYP Groups

All students will be included in at least one group, and some students could be included in as many as five groups!

Schools will want to make sure that they are aware of Students in Overlapping Categories (SOCs).

The minimum number of students for AYP model must be met for the group to be included (40, separately for reading/language and mathematics).

19

AYP Goals (Example)

AYP READING Starting Points and Intermediate Goals For Creating a READING Index

Goals: Percentage of Proficient Students Based on Previous School Year Test DataAssessment Variable

2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014

MCT Grade 3 Reading 61 61 71 71 71 81 81 81 90 90 90 100






English II Grade 10 16 16 37 37 37 58 58 58 79 79 79 100

The 2002-03 AYP goal for all nine student groups in all schools, districts, and the state is to have 61% of students or more scoring at or above Proficient on the grade 3 Reading MCT. This goal is also called the annual measurable objective.

20

AYP Goals (Example)

Setting the starting point for AYP Goals as required by NCLB

83

74

27

16

39

54

61.4

61.4

61.2

31.3

30.8

29.6

6,677

6,751

6,834

109

93

54

Percent

Proficient

Number of

Students

Cumulative

Number 20th percentile of state total in

the grade and

subject

21

AYP Goals (Example)

The percent of students scoring at or above Proficient for each grade is compared to the goal. The comparisons for each grade are then combined for all grades across the school (or district) using a weighted index.



2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014







English II Grade 10 16 16 37 37 37 58 58 58 79 79 79 100

22

AYP Goals (Example)

For the first two years of implementation, the AYP goal is the same. Beginning in 2004-05 the goal increases to a new level for three years. The goal increases again in 2007-08, 2010-11, and 2013-14.



2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014







English II Grade 10 16 16 37 37 37 58 58 58 79 79 79 100

23

AYP Goals (Example)

AYP Starting Points and Intermediate Goals For Creating a MATHEMATICS Index


2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014

MCT Grade 3 Math 72 72 79 79 79 86 86 86 93 93 93 100

MCT Grade 4 Math 49 49 62 62 62 75 75 75 87 87 87 100

MCT Grade 5 Math 35 35 51 51 51 68 68 68 84 84 84 100

MCT Grade 6 Math 39 39 54 54 54 70 70 70 85 85 85 100

MCT Grade 7 Math 19 19 39 39 39 60 60 60 80 80 80 100

MCT Grade 8 Math 23 23 42 42 42 62 62 62 81 81 81 100

Algebra I Grade 8 59 59 69 69 69 79 79 79 89 89 89 100

Algebra I Grade 9 13 13 35 35 35 56 56 56 78 78 78 100

Algebra I Grade 10 5 5 28 28 28 52 52 52 76 76 76 100

Algebra I Grade 11 0 0 25 25 25 50 50 50 75 75 75 100

Algebra I Grade 12 0 0 25 25 25 50 50 50 75 75 75 100

24

How Do You Calculate Proficiency?

Example – Mathematics K-5 School Hispanic/Latino Students - Number of Students

Tested and Percent Proficient in Mathematics

Grade

Number ofStudentsTested

PercentProficient or

Above3 35 71%

4 20 49%

5 30 43%

25


Step 1. Compare the actual percent of students proficient or above to the AYP goal.

Grade


Actual: PercentProficient or

Above

Goal: PercentProficient orAbove 2003

Difference:Actual Minus

Goal

3 35 71% 72% -1%

4 20 49% 49% 0%

5 30 43% 35% +8%

26


Step 2. Calculate the weight of each grade level group (so that groups are counted equitably toward the calculation of the proficiency index).

Grade


Weight:Number Divided

by Total

3 35 0.41

4 20 0.24

5 30 0.35

Total 85 1.00

27


Step 3. Calculate the proficiency index by multiplying the difference between the actual percent of students proficient or above and the AYP goal times the weight, and then sum.

Grade



Goal


by Total

Differencetimes Weight

3 35 -1% 0.41 -0.41

4 20 0% 0.24 0.00

5 30 +8% 0.35 +2.80

Proficiency Index +2.39

In this example, the group met the proficiency goal, also called the annual measurable objective.

28


Example - Reading/Language K-5 School Hispanic/Latino Students - Number of Students

Tested and Percent Proficient in Reading and Language Reading Language

GradeNumber ofStudentsTested


Above



Above

3 33 58% 33 48%

4 18 61% 17 47%

5 28 64% 28 46%

29


Step 1. Compare the actual percent of students proficient or above to the AYP goal for Reading.

Grade



Above



Goal

3 33 58% 61% -3%

4 18 61% 66% -5%

5 28 64% 58% +6%

30


Step 1. Compare the actual percent of students proficient or above to the AYP goal for Language.

Grade



Above



Goal

3 33 48% 51% -3%

4 17 47% 49% -2%

5 28 46% 43% +3%

31


Step 2. Calculate the weight of each grade level group (so that groups are counted equitably toward the calculation of the proficiency index).

Reading Language

GradeNumber ofStudentsTested


by Total



by Total

3 33 0.42 33 0.42

4 18 0.23 17 0.22

5 28 0.35 28 0.36

Total 79 1.00 78 1.00

32


Step 3. Calculate the proficiency index by multiplying the difference between the actual percent of students proficient or above and the AYP goal times the weight, calculate the weighted average of Reading and Language, and then sum.

Reading

Grade



Goal


by Total


3 33 -3% 0.42 -1.26

4 18 -5% 0.23 -1.15

5 28 +6% 0.35 +2.10

33



Language

Grade



Goal


by Total


3 33 -3% 0.42 -1.26

4 17 -2% 0.22 -0.44

5 28 +3% 0.36 +1.08



Reading Language

Grade

Numberof

Students

Differencetimes

Weight

Numberof

Students

Differencetimes

Weight

WeightedAverage of

Reading andLanguage

3 33 -1.26 33 -1.26 -1.26

4 18 -1.15 17 -0.44 -0.81

5 28 +2.10 28 +1.08 +1.59

Proficiency Index -0.48

In this example, the group did not meet the proficiency goal, also called the annual measurable objective.

35



Example of Calculating a Weighted Average for Reading and Language

Grade 4 Example

(18 X -1.15) + (17 X -0.44) (18+17)

= -0.81

36

AYP Model

The following decisions must be made separately in reading/language arts and mathematics for each of the AYP groups represented in the school and school district.

1.Does the AYP group meet the minimum number of students?

A. If No, then group is not included in AYP.

B. If Yes, go to step 2.

2. Were at least 95% of the students in the AYP group tested?

A. If No, then group did not meet AYP standard.

B. If Yes, then go to step 3.

37

AYP Model (continued)

3. Is the percent of students proficient in the AYP group at or above the annual measurable objective?

A. If No, go to step 4.

B. If Yes, is the group “All Students?”

1) If No, the group met AYP.

2) If Yes, go to step 5.

38

Step 3. Was the Annual Measurable Objective Met for the Group?


<--------------------------SAFE HARBOR-------------------------->

39


4. Did the proficiency index increase by 10% compared to the proficiency index value for the previous year?

A. If No, the group did not meet AYP.

B. If Yes, go to step 5.

40


Step 4. Did the Proficiency Index Increase Compared to the Previous Year?

<--------------------------SAFE HARBOR-------------------------->

41


5. Did the group improve on the other academic indicator(s)?

For schools with grades 3-8, the other academic indicator is the growth index (met level).

For high schools, the other academic indicator is the graduation rate (required by NCLB; beginning in 2004-05).

A. If No, the group did not meet AYP.

B. If Yes, the group met AYP.

42


Step 5. Other Academic Indicators

| OTHER INDICATORS || <--------------> || GROWTH GRAD MET | MET| INDEX RATE ONE | AYP| ------ ----- --- | ---- 0.070 YES YES

N/A <MINN/A <MIN

0.020 YES *YESN/A <MIN

0.080 YES YESN/A <MINN/A <MIN

0.060 YES YES

43


6. Did all the AYP groups that met the criterion for inclusion in the model meet the AYP standard?

A. If No, then the school or school district did not meet AYP.

B. If Yes, then the school or school district met AYP.

44


Step 6. Did the School/District Meet AYP?

| OTHER INDICATORS || <--------------> || GROWTH GRAD MET | MET| INDEX RATE ONE | AYP| ------ ----- --- | ---- 0.070 YES YES

N/A <MINN/A <MIN

0.020 YES *YESN/A <MIN

0.080 YES YESN/A <MINN/A <MIN

0.060 YES YES

45

Graduation Rate Indicator

Graduation Rate Indicator = 4-Year Graduation Rate

The graduation rate goal has not been set.

Graduation Rate Indicator will be lagged one year (e.g., 2002-03 results will use 2001-02 graduation rate)

Graduation Rate will be used for the All Students group only, until individual students can be tracked and graduation rates for each of the other groups can be calculated (tentatively 2004-05).

For the disaggregated groups in high schools, the growth index will be used as the other academic indicator until graduation rates are available OR a one year graduation rate will be used. MDE will select one of these to implement (the one that is most beneficial).

46

AYP Model for Schools with No Data

For (approximately 38) public schools with no accountability assessment data, the AYP decision will be based on feeder patterns.

The AYP decision will be derived using assessment data from the lowest accountability grade level in the school receiving the students.

For a K-1 or K-2 school, for example, AYP will be determined using the grade 3 assessment data at the receiving school.

47

Tentative Dates for Statewide Accountability Model

July 15 Districts receive assessment results from test vendors.

July 15-31 MDE receives student level data files from test vendors.

Aug. 1-8 MDE prepares data files with student test data matchedto MSIS records. (Accuracy of student MSIS ID numberswill be critical.)

Aug. 1-8 MDE runs the accountability model and providespreliminary reports to districts.

48

Tentative Dates for Statewide Accountability Model (continued)

Aug. 11-20 Districts and schools review results and respondconcerning potential identification for improvement.School informs parents.

Aug. 21-30 MDE makes final determination of AYP for districts andschools.

Sept. 1-5 Recommendation by Mississippi Commission on SchoolAccreditation.

Sept. 8-12 Approval by Mississippi State Board of Education (earlymeeting date).

49

Assessment of Students with Disabilities

Fall 2002 Districts informed that NCLB may require

assessing all students with disabilities using an assessment based on students’ grade classification rather than instructional level

Districts told to continue assessment of students with disabilities under existing guidelines until MDE received further clarification from the USDE (regional administrators meetings)

50


December 2002 Preliminary review of state accountability plan

with USDE; USDE says MDE cannot count instructional level or alternate assessments as proficient.

January 2003 Formal review of state accountability plan with

USDE Plan allows for continued use of instructional

level and alternate assessments

51

January 2003 (continued) Students taking instructional level or

alternate assessments will be counted as “Not Proficient” for purposes of determining Adequate Yearly Progress (AYP) (see January 31 memo from Offices of Special Education and Student Assessment)


52

March 2003 MDE requests permission to count

students taking instructional level assessments according to their proficiency level for two years to allow school districts time to adjust to requirements of NCLB. After two years, students assessed on instructional level would be counted as “not proficient.”


53

March 2003 USDE issues proposed rule to allow the

use of an alternate achievement standard for severe/profound cognitively disabled students (students three standard deviations below in IQ and adaptive behavior).


54


Do not confuse rules for students with disabilities under AYP model with achievement and growth models.

Achievement model - students assessed with instructional level test or alternate assessment are included.

Growth model - only students taking a standard test (with or without accommodations) are included.

Documents

1 Mississippi Statewide Accountability System Adequate Yearly Progress Model Improving Mississippi Schools Conference June 11-13, 2003 Mississippi Department