Upload
jack-ball
View
217
Download
2
Tags:
Embed Size (px)
Citation preview
Review and Discussionof AMAO 2 Criteria & Targets
NC Department of Public InstructionWith
WestEd & Wisconsin Center for Education Research
Statewide Web ConferenceAugust 30, 2010
NCDPIAMAO 2 Meeting
2
Purpose
Review and comment on recommendations for changes to NC State Board of Education policy GCS-A-012, Annual Measurable Achievement Objectives for NCLB Title III
NCDPI Recommendations in August Policy Revisions to SBE in September
Session Agenda Introduction and Purpose (Ground Rules) Historical Perspective and 1-year Recap AMAO 2 Proficiency Criteria
(Comprehensive Objective Composite,COC) Review and Comparison (2009 & 2010)
AMAO 2 Targets for Consideration: 2009-10 and Beyond
Next Steps & Meeting Wrap Up
NCDPIAMAO 2 Meeting
3
NCDPIAMAO 2 Meeting
4
Introductions
New NCDPI Staff Scott Beaudry, Testing Policy & Operations Special Guests Robert Linquanti, WestEd Gary Cook, Wisconsin Center for Education
Research Shirley Carraway, Appalachia Regional
Comprehensive Center
NCDPIAMAO 2 Meeting
5
Historical Perspective In 2007-2008:In 2007-2008: 1. Determined revisions to AMAOs 1 & 2 needed
Criteria were too loose or too stringent Targets set without federal guidance Targets not based on empirical data
2. Determined new standards and assessments needed ACCESS for ELLs would replace IPT
3. Determined that AMAO 1 criteria and targets would be revised after two years of ACCESS for ELLs data was gathered.
NCDPIAMAO 2 Meeting
6
Historical Perspective: AMAO 2
In 2007-2008, cont’d.:In 2007-2008, cont’d.: Decided to keep the criterion for
proficiency the same for 2007-08
2007-08 target was set at 17% to account for differences in using Form A and Form B of the IPT
Targets for 2008-09 and beyond removed as they needed to be based on empirical results from new ELP assessment
NCDPIAMAO 2 Meeting
7
Last Year Recap: AMAO 2
In 2008-09:In 2008-09: Per USED Notice of Final Interpretations (2008):
Only one data point needed to calculate AMAO 2 for each ELL
All ELLs (K-12) must be included in calculation
2008-09 ACCESS results used to define COC and new, one-year target for 2008-09 (14.7%)
Decision made to set future targets after examining another year of ACCESS results
8
Last Year Recap: AMAO 2
In 2008-09, cont’d.:In 2008-09, cont’d.: Stakeholders endorsed state-recommended
COC derived from analyses of student performance on 2008-09 ACCESS and state’s reading and math assessments Overall 4.8, R & W each 4.0 minimum
NCDPIAMAO 2 Meeting
NCDPIAMAO 2 Meeting
9
AMAO 2 Proficiency Criteria (COC) Review & Comparison (2009 & 2010)
Purpose: Replicate analysis performed in 2008-09 to validate COC criteria chosen Apply same decision consistency method
to 2009-10 ACCESS and EOG/EOC reading and math assessments
NCDPIAMAO 2 Meeting
10
Decision Consistency Method
These analyses identify language proficiency level that optimally classifies students as true-positives or true-negatives on both NC EOG/EOC Reading & Math Assessments and ACCESS
11
Decision MatrixCriterion B
Below Above
BelowTrue
NegativeFalse
Positive
AboveFalse
NegativeTrue
Positive
Criterion A
samplein number Total
Negatives True Positives TrueCorrect
NCDPIAMAO 2 Meeting
12
Decision Matrix
ELP AssessmentBelow Above
Below 43 14
Above 18 25
Content Assessment
Correct = 68%
NCDPIAMAO 2 Meeting
13
Decision Matrix
ELP AssessmentBelow Above
Below 51 5
Above 10 34
Content Assessment
Correct = 85%
NCDPIAMAO 2 Meeting
14
Reading to ACCESS: 2009
NC EOG Reading to ACCESS Decision Consistency across Clusters
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
<2.0 2.0-2.4 2.5-2.9 3.0-3.4 3.5-3.9 4.0-4.4 4.5-4.9 5.0-5.4 5.5-6.0
WIDA Composite Proficiency Band
Per
cen
t C
lass
ifie
d C
orr
ectl
y
Cluster 3-5 Cluster 6-8 Cluster 9-12
NCDPIAMAO 2 Meeting
15
Mathematics to ACCESS: 2009
NC EOG Mathematics to ACCESS Decision Consistency across Clusters
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
<2.0 2.0-2.4 2.5-2.9 3.0-3.4 3.5-3.9 4.0-4.4 4.5-4.9 5.0-5.4 5.5-6.0
WIDA Composite Proficiency Band
Pe
rce
nt
Cla
ss
ifie
d C
orr
ec
tly
Cluster 3-5 Cluster 6-8 Cluster 9-12
NCDPIAMAO 2 Meeting
Comparing DC Analysis, 2009 to 2010: Reading to ACCESS, Grades 3-5
16NCDPIAMAO 2 Meeting
17
Comparing DC Analysis, 2009 to 2010: Reading to ACCESS, Grades 6-8
NCDPIAMAO 2 Meeting
18
Comparing DC Analysis, 2009 to 2010: Reading to ACCESS, Grades 9-12
NCDPIAMAO 2 Meeting
19
Comparing DC Analysis, 2009 to 2010: Math to ACCESS, Grades 3-5
NCDPIAMAO 2 Meeting
20
Comparing DC Analysis, 2009 to 2010: Math to ACCESS, Grades 6-8
NCDPIAMAO 2 Meeting
21
Comparing DC Analysis, 2009 to 2010: Math to ACCESS, Grades 9-12
NCDPIAMAO 2 Meeting
NCDPIAMAO 2 Meeting
22
AMAO 2 Criterion ConfirmedComprehensive Objective Composite (COC)Current English language proficiency definition
on the ACCESS test holds: Composite score of at least 4.8 and at least 4.0 on
Reading subtest and 4.0 on Writing subtest.
Note: Students who attain the COC as defined above exit LEP identification. Those who do not remain identified LEP.
NCDPIAMAO 2 Meeting
23
REMINDER:2009-10 AMAO 2 Cohort Definition
Cohort definition required by federal law: ALL LEP students (K-12) must be included in AMAO 2
calculation
Numerator = # of LEP students attaining COC Denominator = #of LEP students required to test
Historical AMAO 2 Target Data
Year Target Met Not Met Missing data Total
2003-04 20.0% 86 6 0 922004-05 25.0% 81 1 1 832005-06 30.0% 4 76 3 832006-07 35.0%2007-08 17.0% 38 47 0 852008-09 14.7% 25 63 0 882009-10* 11.8% 68 20 0 88
*All 2009-10 results are unofficial
25
NC LEA and State AMAO 2 Performance Using Current AMAO 2 Criterion:
Shows percentage of LEPs meeting COC performance level for LEAs at that ranking and Statewide
AMAO 2 Criterion Overall 4.8 (R&W GE 4.0) % LEP meeting AMAO 2 Criterion by LEA Percentile Rank STATE
P10 P15 P20 P25 P50 P75 P80 P90 % LEP
2010 Analysis 9.5% 10.7% 11.5% 11.8% 13.8% 16.8% 17.4% 19.7% 14.5%
NCDPIAMAO 2 Meeting
26
2010-2019 Annual Targets for LEAs & State using 2010 as Base Year • Proposed
target for 2009-10 is 11.8% of LEP students in an LEA attaining English language proficiency.
• Proposed end point in 2018-19 is 16.8% of LEP students in an LEA attaining English language proficiency.
Recommended Targets for Consideration
AMAO 2AMAO 2
25 %ile
75 %ile
NCDPIAMAO 2 Meeting
Target Recommendation: 2010-2019
Set the 2009-10 target at 11.8% (25th %ile)
Set the 2018-19 target at 16.8% (75th %ile)
Structure targets to increase by equal increments each year (0.55 percentage points)
NCDPIAMAO 2 Meeting
2009-10 AMAO Analysis
AMAO 1 Criteria and targets are the same improve at least one proficiency level in
at least one of the subtests of reading, writing, speaking, or listening
Target = 70%
2009-10 Analysis (continued)
Sanctions for AMAOs Not Met In 2008-09, Title III status based on
whether or not the LEA failed to make progress toward meeting the same AMAO
Starting in 2009-10, Title III status based on failure to meet the AMAOs
GCS-A-012 Revisions
Show Draft Policy Policy to SBE in September as Action
on First Read
Next Steps (for 2009-10 data) Updated GCS-A-012 sent to USED for Title III
Workbook and Title III Plan submission Preliminary AMAO report sent to districts for
review in September AMAO report presented to SBE in November
31NCDPIAMAO 2 Meeting
Next Steps (2010 -11 and beyond)
Analysis of potential changes to AMAO 1 progress definitions and targets to occur during Fall 2010
AMAO 1 analyses and draft recommendations vetted with stakeholders during 2010-11 school year
AMAO 1 policy approval in 2010-11 Updated policy sent to USED for
submission with Title III Workbook and Title III Plan
Federal Title III Audit in Spring 2011