Embed Size (px)
Citation preview
Presenter:Presenter:Dr. Charity Fleming SmithDr. Charity Fleming SmithAssistant CommissionerAssistant Commissioner
Office Public School Academic AccountabilityOffice Public School Academic AccountabilityArkansas Department of EducationArkansas Department of Education
Building a Professional Building a Professional Learning Community:Learning Community:
Three Accountability Issues Three Accountability Issues
Agenda: Three Accountability Agenda: Three Accountability IssuesIssues
1.1. Statewide Student Participation in the Central Statewide Student Participation in the Central High High School 50th Anniversary CelebrationSchool 50th Anniversary Celebration– September 25, 2007September 25, 2007
2.2. Test Security IssuesTest Security Issues
3.3. Act 35 Gains Model Act 35 Gains Model
1. Statewide Student Participation in the 1. Statewide Student Participation in the Central High School Central High School 50th Anniversary50th Anniversary
CommemorationCommemoration September 25, 2007September 25, 2007
““It is our mission to It is our mission to encourage young encourage young people to take people to take executive executive responsibility for their responsibility for their education, to step education, to step forward boldly and forward boldly and seize any available seize any available opportunity to expend opportunity to expend awareness and awareness and understanding.understanding.
Little Rock Nine FoundationLittle Rock Nine Foundation
Theme: Seizing Educational Theme: Seizing Educational Opportunities:Opportunities: Character and CourageCharacter and Courage Art and Literacy Scholarship ContestArt and Literacy Scholarship Contest
This contest is open to all 12th grade students. Students will:This contest is open to all 12th grade students. Students will:
Meet distinguished public officials and visit historical, national Meet distinguished public officials and visit historical, national monuments and museums in the state’s capitol, including the monuments and museums in the state’s capitol, including the Clinton Library and the Emancipation Proclamation ExhibitClinton Library and the Emancipation Proclamation Exhibit
Attend an awards luncheon for all state finalists to receive Attend an awards luncheon for all state finalists to receive recognition for their efforts, and the announcement of winnersrecognition for their efforts, and the announcement of winners
Participating by satellite in the events at Central High from historic Participating by satellite in the events at Central High from historic Philander Smith CollegePhilander Smith College
Six Thousand dollars in scholarships will be awarded to student Six Thousand dollars in scholarships will be awarded to student winners, and All schools are encouraged to participatewinners, and All schools are encouraged to participate
Note: This is made possible by financial support from the Arkansas General Note: This is made possible by financial support from the Arkansas General Assembly (Cost of transportation, and meals will be covered).Assembly (Cost of transportation, and meals will be covered).
What is What is Correct English Desktop EditorCorrect English Desktop Editor Writing Enhancement Software?Writing Enhancement Software?
Software that… Software that… IdentifiesIdentifies core grammar mistakescore grammar mistakes, ,
Provides Provides task-specifictask-specific revision tipsrevision tips, and, and Supports proper Supports proper Word ChoiceWord Choice && EnrichmentEnrichment
……to make writing more to make writing more clearclear, , polishedpolished, and , and effectiveeffective..
Software that targets Software that targets all types of writers, purposes all types of writers, purposes and applicationsand applications, and, and
Improves writing as a Improves writing as a form of communicationform of communication..
What is What is Correct English Desktop EditorCorrect English Desktop Editor Writing Enhancement Software?Writing Enhancement Software?
Software that… Software that… IdentifiesIdentifies core grammar core grammar
mistakesmistakes, , Provides Provides task-specifictask-specific revision revision
tipstips, and, and Supports proper Supports proper Word ChoiceWord Choice
&& EnrichmentEnrichment ……to make writing more to make writing more clearclear, , polishedpolished, and , and effectiveeffective..
Software that targets Software that targets all types all types of writers, purposes and of writers, purposes and applicationsapplications, and, and Improves writing as a Improves writing as a
form of form of communicationcommunication..Student writers will have a 30-day opportunity to enhance writing skills,
and edit essays for the scholarship contest Sponsored by Vantage Learning
2. Test Security2. Test Security
The Arkansas Department of Education (ADE), The Arkansas Department of Education (ADE), through the Arkansas State Board of Education through the Arkansas State Board of Education (SBE), is authorized pursuant to specific federal and (SBE), is authorized pursuant to specific federal and state statutes and regulations to investigate alleged state statutes and regulations to investigate alleged violations by a public education agency of violations by a public education agency of requirements associated with its provision of requirements associated with its provision of educational programs. Specifically:educational programs. Specifically:
Arkansas Code Ann. Sections 6-11-105 (Supp.1989) Arkansas Code Ann. Sections 6-11-105 (Supp.1989) and 6-15-401et seq., as amended.and 6-15-401et seq., as amended.
Potential Disciplinary ActionPotential Disciplinary Action
The Attorney’s office shall make a The Attorney’s office shall make a preliminary report based on the team’s preliminary report based on the team’s recommendations and submit the report to recommendations and submit the report to the ADE’s Commissioner.the ADE’s Commissioner.
After a review by the Commissioner, a After a review by the Commissioner, a course of action shall be taken.course of action shall be taken.
The Commissioner may recommend possible The Commissioner may recommend possible action taken against an individual’s Teaching action taken against an individual’s Teaching License.License.
3. Act 35 Annual 3. Act 35 Annual
Improvement Gain ModelImprovement Gain ModelIntention of Act 35Intention of Act 35
The intention of Act 35 is to promote The intention of Act 35 is to promote student learning at all levels so that student learning at all levels so that “all students have an opportunity to “all students have an opportunity to demonstrate increased learning” and demonstrate increased learning” and “meet the expected academic “meet the expected academic standards.”standards.”
Intention of Act 35 Cont.Intention of Act 35 Cont.
The Act 35 Annual Rating System was designed for the The Act 35 Annual Rating System was designed for the purpose of improving student achievement through purpose of improving student achievement through school accountability and recognition. Act 35, §6-15-school accountability and recognition. Act 35, §6-15-1901 (c), requires that each school be classified into 1901 (c), requires that each school be classified into “two (2) category levels” as follows:“two (2) category levels” as follows:
– ““Category One” for the “school’s improvement Category One” for the “school’s improvement gains” tracked longitudinally using value-added gains” tracked longitudinally using value-added calculation known as the annual improvement calculation known as the annual improvement category level and category level and
– ““Category Two” based on “performance from the Category Two” based on “performance from the prior year” referred to as the annual performance prior year” referred to as the annual performance category (or “status”). category (or “status”).
Annual Improvement CategoryAnnual Improvement CategoryThe annual improvement category for rating schools will report The annual improvement category for rating schools will report each school’s improvement gains by tracking students’ each school’s improvement gains by tracking students’ longitudinal achievement gains on the state’s criterion-longitudinal achievement gains on the state’s criterion-referenced tests. referenced tests.
The base year for the growth model is 06-07. The first reports The base year for the growth model is 06-07. The first reports will be available in December of 2008.will be available in December of 2008.
A school’s annual improvement gain is based upon the changes A school’s annual improvement gain is based upon the changes in student achievement from one year to the next. in student achievement from one year to the next.
No value will be added if instruction does not move a student’s No value will be added if instruction does not move a student’s achievement from a given performance category to a higher achievement from a given performance category to a higher performance category. performance category.
If a student’s achievement moves to a lower performance If a student’s achievement moves to a lower performance category, then value is lost.category, then value is lost.
Initial Design ConsiderationsInitial Design Considerations
1.1. As required by Act 35, growth will be based on the As required by Act 35, growth will be based on the tests included in the Arkansas Comprehension tests included in the Arkansas Comprehension Testing, Assessment and Accountability Program Testing, Assessment and Accountability Program (ACTAAP).(ACTAAP).
2.2. In subsequent years, if there are changes in any of In subsequent years, if there are changes in any of the tests, the new test versions will be linked to the tests, the new test versions will be linked to current performance standards. current performance standards.
3.3. This annual improvement gain model is applicable This annual improvement gain model is applicable to any type of student achievement test and is not to any type of student achievement test and is not dependent upon a particular scale score.dependent upon a particular scale score.
Initial Design ConsiderationsInitial Design Considerations4.4. Any test used for school accountability shall be Any test used for school accountability shall be
aligned to the Arkansas Content Standards.aligned to the Arkansas Content Standards.
5. 5. An annual improvement gain model will be based on An annual improvement gain model will be based on an aggregation of student changes in literacy and an aggregation of student changes in literacy and mathematics for grades 3-8 and for literacy in grade mathematics for grades 3-8 and for literacy in grade 11. 11.
6. 6. The annual improvement gain model shall be The annual improvement gain model shall be designed with the expectation that (a) students who designed with the expectation that (a) students who are proficient or higher will either maintain or improve are proficient or higher will either maintain or improve their performance classification, and (b) students who their performance classification, and (b) students who are basic or below basic will reach proficiency.are basic or below basic will reach proficiency.
7. 7. The annual improvement gain model shall be The annual improvement gain model shall be transparent, replicable, and easily understood by transparent, replicable, and easily understood by Arkansas stakeholdersArkansas stakeholders..
Calculations for Annual Calculations for Annual Student GrowthStudent Growth
Student growth is based upon changes in student Student growth is based upon changes in student performance levels across two adjacent years. performance levels across two adjacent years.
To assess annual changes more precisely each To assess annual changes more precisely each student performance level (Below Basic, Basic, student performance level (Below Basic, Basic, Proficient, and Advanced) is split into two sub-Proficient, and Advanced) is split into two sub-categories. The following table lists the points categories. The following table lists the points assigned to the sub-categoriesassigned to the sub-categories..
Table 1: Student Performance Table 1: Student Performance Sub-categoriesSub-categories
Student Performance Sub-categories
Points Assigned
Below Basic 1 1
Below Basic 2 1.5
Basic 1 2
Basic 2 2.5
Proficient 1 3
Proficient 2 3.5
Advanced 1 4
Advanced 2 4.5
75
74 59
43
41
26
27
41
57 59 69
65 60 56
42 31
35 40 43
58
0
20
40
60
80
BelowProficiency
At or AboveProficiency
BelowProficiency
At or AboveProficiency
2002 2003 2004 2005 2006
L i t e r a c y M a t h e m a t i c s
Arkansas Benchmark (Grade 6)Arkansas Benchmark (Grade 6)
Combined PopulationLongitudinal Gains
68 58 49 43 35 32 41
51 57
65 79
78
69 67 56
20
22 31
33 44
0
20
40
60
80
BelowProficiency
At or AboveProficiency
BelowProficiency
At or AboveProficiency
2002 2003 2004 2005 2006
L i t e r a c y M a t h e m a t i c s
Arkansas Benchmark (Grade 8)Arkansas Benchmark (Grade 8)
Combined PopulationLongitudinal Gains
91
70 68
52 48
10 30 32 47
52
63 56 47
40 36
37
44 53
60 64
0
20
40
60
80
100
BelowProficiency
At or AboveProficiency
BelowProficiency
At or AboveProficiency
2002 2003 2004 2005 2006
Mid - Year (January) Spring (April)
Arkansas Algebra IArkansas Algebra I
Combined PopulationLongitudinal Gains
81
78 74
56 49
19 22 26
43 51 69 60
52
45 40
32
39
48
55 60
0102030405060708090
BelowProficiency
At or AboveProficiency
BelowProficiency
At or AboveProficiency
2002 2003 2004 2005 2006
Mid - Year (January) Spring (April)
Arkansas GeometryArkansas Geometry
Combined PopulationLongitudinal Gains
Scale Scores That Define Scale Scores That Define the Sub-categoriesthe Sub-categories
The scale scores that define the sub-The scale scores that define the sub-categories are listed in the Technical categories are listed in the Technical Memorandum from Drs. Huynh Huynh, Memorandum from Drs. Huynh Huynh, Robert Kennedy, and Eugene Kennedy, Robert Kennedy, and Eugene Kennedy, dated April 19, 2007. dated April 19, 2007.
The memorandum was reviewed and The memorandum was reviewed and approved by the Technical Advisory approved by the Technical Advisory Committee on Accountability.Committee on Accountability.
Value-added PointsValue-added Points
Table 2 shows the value-added points for a student Table 2 shows the value-added points for a student based upon whether or not he/she improves in based upon whether or not he/she improves in achievement, stays the same in achievement, or achievement, stays the same in achievement, or regresses in achievement. regresses in achievement.
For example, if a student in third grade earns a For example, if a student in third grade earns a achievement rating of Proficient 2 and the following achievement rating of Proficient 2 and the following year the same student in fourth grade receives an year the same student in fourth grade receives an achievement rating of advanced 1, he/she would be achievement rating of advanced 1, he/she would be given a value-added score of +0.5. Similar value-given a value-added score of +0.5. Similar value-added scores would be calculated for each child in added scores would be calculated for each child in each school. each school.
Table 2: Value-added Points Table 2: Value-added Points for Changes in Student for Changes in Student
AchievementAchievement
Previous Year
Current Year
1 1.5 2 2.5 3 3.5 4 4.5
1 0 .5 1 1.5 2 2.5 3 3.5
1.5 -.5 0 .5 1 1.5 2 2.5 3
2 -1 -.5 0 .5 1 1.5 2 2.5
2.5 -1.5 -1 -.5 0 .5 1 1.5 2
3 -2 -1.5 -1 -.5 0 .5 1 1.5
3.5 -2.5 -2 -1.5 -1 -.5 0 .5 1
4 -3 -2.5 -2 -1.5 -1 -.5 0 .5
4.5 -3.5 -3 -2.5 -2 -1.5 -1 -.5 0
Computation of Annual Computation of Annual Improvement Improvement
Gain IndexGain Index
For each school, the Annual Improvement Gain For each school, the Annual Improvement Gain Index is the average of all value-added points Index is the average of all value-added points across subject areas and grades. across subject areas and grades.
Potentially, the range of the Annual Potentially, the range of the Annual Improvement Gain Index is from -3.5 to +3.5. Improvement Gain Index is from -3.5 to +3.5.
The value of zero indicates no growth, positive The value of zero indicates no growth, positive values indicate improvement, and negative values indicate improvement, and negative values indicate a decline in achievement.values indicate a decline in achievement.
Multiply the number of students (5) by the points assigned to the category. For example multiply (5) times (4) to get the points produced by the students in the
Advanced category 1. Add up the points for each category. Divide the total points for the school (285) by the number of students (100) to get a rating score
(2.85), which is meeting standards.Number of Students
Categories Points Assigned to Categories
Total
5 Below Basic 1 1 5
5 Below Basic 2 1.5 7.5
10 Basic 1 2 20
20 Basic 2 2.5 50
30 Proficient 1 3 90
20 Proficient 2 3.5 70
5 Advanced 1 4 20
5 Advanced 2 4.5 22.5
Total Points for the school for all categories 285
How School Ratings Are Calculated
Cut Score Performance Category Rating
3.23 - 4.0 Schools of excellence 5
2.79 - 3.22 Schools exceeding the standards
4
2.21 – 2.78 Schools meeting standards 3
1.719 – 2.20 Schools approaching the standards (alert)
2
1.0 - 1.718 Schools in need of immediate improvement
1
Example of Standards Example of Standards SettingSetting
Procedures for Setting Cut Procedures for Setting Cut Scores for Annual Improvement Scores for Annual Improvement
Gain LevelsGain LevelsAccording to Act 35, schools will be classified into one of five According to Act 35, schools will be classified into one of five Annual Improvement Gain Levels. Therefore, four (4) cut Annual Improvement Gain Levels. Therefore, four (4) cut scores will need to be set. scores will need to be set.
The general steps previously used in setting cut scores for The general steps previously used in setting cut scores for Category Two, Annual Performance Category Levels (Status Category Two, Annual Performance Category Levels (Status Index), will be implemented. Index), will be implemented.
The steps are: (1) preliminary work by the Technical The steps are: (1) preliminary work by the Technical Advisory Committee on Accountability, (2) advice by Advisory Committee on Accountability, (2) advice by Arkansas stakeholders at a meeting, and (3) adoption by Arkansas stakeholders at a meeting, and (3) adoption by the Arkansas State Board of Education.the Arkansas State Board of Education.
Preliminary Work by the Preliminary Work by the Technical Advisory Committee Technical Advisory Committee
on Accountabilityon Accountability
The distribution of the school Annual The distribution of the school Annual Improvement Gain Index will be compiled Improvement Gain Index will be compiled based on the matched Spring 2006 and based on the matched Spring 2006 and Spring 2007 data.Spring 2007 data.
The above distribution will be divided into The above distribution will be divided into ten equal parts. A representative sample ten equal parts. A representative sample of schools in each part will be selected and of schools in each part will be selected and relevant data about those schools will be relevant data about those schools will be provided.provided.
Preliminary Work by the Preliminary Work by the Technical Advisory Committee Technical Advisory Committee
on Accountabilityon AccountabilityThose data shall include, but not be limited to, the Those data shall include, but not be limited to, the following variables:following variables:
– School sizeSchool size– School grade level configurationSchool grade level configuration– Demographic data, such as proportion of each Demographic data, such as proportion of each
ethnic group, mobility, free and reduced lunch, ethnic group, mobility, free and reduced lunch, ELL, Special Education students, etc.ELL, Special Education students, etc.
– Proportion of students who meet the AYP Proportion of students who meet the AYP growth trajectory as defined in the Arkansas growth trajectory as defined in the Arkansas AYP Growth ModelAYP Growth Model
– Ratings on status models in the 2005, 2006, Ratings on status models in the 2005, 2006, 20072007
– Other characteristics as defined by field reviewOther characteristics as defined by field review
Advice by Arkansas Advice by Arkansas StakeholdersStakeholders
A meeting of Arkansas stakeholders will be A meeting of Arkansas stakeholders will be convened to recommend Annual convened to recommend Annual Improvement Gain cut scores to Arkansas Improvement Gain cut scores to Arkansas Department of Education. Department of Education.
The same process used in the status The same process used in the status standard setting meeting will be standard setting meeting will be implemented. Additionally for continuity, implemented. Additionally for continuity, some of the same panelists will be invited some of the same panelists will be invited to participate. to participate.
The new panel will be supplemented with The new panel will be supplemented with new persons.new persons.
Presentation Made at State Presentation Made at State Board of Education MeetingBoard of Education Meeting
The recommended cut scores will be The recommended cut scores will be presented to the State Board along with presented to the State Board along with relevant data for their deliberation and final relevant data for their deliberation and final action.action.
TimelineTimelineNovember 2008 to Rate Schools on Growth 07-08 Testing
(Using data from 06-07 and 07-08)
July 2007 Present Initial Plan to State Board for information
September 20-21, 2007(TAC Meeting)
Run computation of baseline year school year 2006-2007. Review growth analysis across the two years
October 2007 School Awareness Training
November 15-16, 2007 (TAC Meeting)
Standard Setting for school gain index
December 2007 Initial report on school gain index to State Board
January 2008 State Board action on criteria for annual improvement gain category levels
February 2008 School awareness with stakeholders (second round)
TimelineTimeline
March 13-14, 2008 (TAC Meeting)
First draft of Technical Report-Annual Improvement Gains Category
April 2008 Students are assessed
July 2008 NORMES produces initial matched data set
August 2008 Initial matched data sent to ADE for review
September 1-15, 2008
Schools review matched data set and report any necessary revisions to the ADE and NORMES (online)
October-November 2008
NORMES and ADE validate preliminary gain scores
December 2008 Preliminary release of Annual Improvement Gains Report on ADE website
Annual School Performance Report Annual School Performance Report
TimelineTimeline September 30, September 30, 20092009
Release of achievement data from NORMESRelease of achievement data from NORMES
October 15, 2009 October 15, 2009 Deadline for requesting the Annual Deadline for requesting the Annual Performance Report (Rating Schools)Performance Report (Rating Schools)
November 2009 November 2009 Annual Performance Reports are available for Annual Performance Reports are available for those who request it those who request it
March 15, 2010 March 15, 2010 Annual Improvement Gains Report in the Annual Improvement Gains Report in the Annual School Performance Report (Report Annual School Performance Report (Report Card) Card)
March 15, 2011 March 15, 2011 Official release of both reports: Annual Official release of both reports: Annual Improvement Gains Report and the Annual Improvement Gains Report and the Annual School Performance Report reflecting the School Performance Report reflecting the Spring 2010 achievement data in the Annual Spring 2010 achievement data in the Annual School Performance Report (Report Card) School Performance Report (Report Card)
