Wyoming Alternative School Accountability SystemPresentation to the Select Committee on Statewide Education Accountability September 21, 2016 . Legislative Charge (1) The department

Wyoming Alternative School Accountability System

Erika Hall

Center for Assessment

Presentation to the Select Committee on Statewide Education Accountability

September 21, 2016

Legislative Charge (1)

The department of education shall commence development of an alternative school level accountability model to be piloted in school year 2016-2017. The model shall adhere, as closely as possible, to the recommendations contained in the "Wyoming Alternative School Accountability Framework: Recommendations from the Alternative Accountability Advisory Committee" report dated October 15, 2015, submitted by the technical advisory group

Alternative Schools , Select Committee 9/16 2

Legislative Charge (2)

Not later than July 1, 2016, the department of education shall submit a report to the technical advisory group, the select committee on statewide education accountability and the joint education interim committee containing proposed business rules, data collection requirements and a proposal for the student success plan required under paragraph (b)(iii) of this section.

Alternative Schools , Select Committee 9/16 3

Timeline

Task Date

Technical advisory group meetings April 7, 2016May 2-3, 2016June 16-17, 2016

Recommendations for business rules, student success plan, data collection, and other modifications

July 1, 2016

Pilot selected components of model; Information only reporting (i.e. ‘hold harmless’)

2016-2017

Evaluate and refine Spring/ Summer 2017

Full pilot of model; information only reporting 2017-2018

Evaluate and refine Spring/ Summer 2018

Establish performance expectations (PJP), Determine support system

October 15, 2018

Alternative Schools , Select Committee 9/16

4

System Overview


5

Goals


6

Goals of the Accountability System Mechanism by which Goals will be Attained

Incentivize and support attainment

of broad skills and appropriate

credentials to promote success in a

variety of post-secondary pursuits

such as college and careers.

Encourage the development and provision of a school climate

characterized by flexibility, an engaged community and

personalized support

Establish a set of indicators that help schools evaluate and

improve the effectiveness of their programs.

Encourage the development of programs, initiatives, and

collaborations that serve to increase student opportunities for

success.

Establish a valid measure of school

performance that accounts for

contextual factors unique to

alternative schools.

Provide for a reasonable degree of flexibility with respect to

the manner in which indicators are selected, operationalized

and calculated for a given alternative school (i.e., enough to

allow for valid inferences, but not too much to restrict the

ability to make any comparisons among alternative schools).

Increase credibility and support for

WY’s alternative schools

Transparency around the procedures used to evaluate student

and school performance for purposes of accountability and

their rationale.

Intended Uses

• Inform local decisions related to the quality of new programs or initiatives

• Allow for comparisons between and among alternative schools within and across years (with respect to some indicators of school performance) with the goal of identifying areas of weakness or variability.

• Report changes/improvements in alternative school performance over years.


7

System Components

Academic Performance: the extent to which students meet identified performance standards and demonstrate appropriate annual academic growth

Readiness: the extent to which students earn course credit and attain outcomes that position the student for success in college or career

Engagement: the extent to which students attend school regularly and participate in a range of activities that promote holistic development of life skills associated with post-secondary success

School Climate: the extent to which parents, teachers, and students report that the school achieves and improves with respect to creating a safe, positive environment that promotes collaboration and is conducive to learning and growth


8

System Overview

Overall Performance

Academic Performance

Student Achievement

Growth

Readiness

Credit Earning Attainment

Hathaway Graduation Other Indicators

Climate

Climate Survey

Engagement

Student Success Plan (includes attendance)


9

Technical Advisory Group


10

The Technical Advisory Group


11

Name Organization

Michael Maloney Sweetwater #1

Shawna Trujillo Natrona #1

Darlene Hartman-Hallam Crook #1

Beth Auge Teton #1

Teresa Chaulk Lincoln #1

Mike Helenbolt Laramie #1

Kathleen Milligan-Hitt Fremont #1

Kathy Scheurman WEA

Deb Lindsey WDE

Mike Flicek Contractor - WDE

Dianne Frazer WDE

Chris Domaleski Center for Assessment

Erika Hall Center for Assessment

Julie Magee WDE

The committee includes:

- School principals- District

superintendents- WDE leadership- Technical

consultants

Technical Advisory Group Meetings

• Three in-person meetings

– April 7, 2016

– May 2-3, 2016

– June 16-17, 2016

• Focal areas:

– Development of school climate measure and administration guidance

– Development of student success plan template and guidance

– Development of recommendation to incorporate attendance

– Identification of approach and data elements for credit earning and attainment index

– Review/ refine plans for achievement and growth


12

Summary of Completed Work & Next Steps


13

School Climate

• Decided to include only student climate measure in the model

• Developed a custom survey and administration guidance to be piloted in fall 2016 and spring of 2017

– participation is mandatory for all alternative schools

– it will also be administered to a sample of non-alternative schools

• The survey includes 20 items that address the following dimensions:

– Support/Trust

– High Expectations

– Respect

• Administration guidance outlines:

– Purpose of the survey

– Role of the survey coordinator

– Procedures for administration

– Administration script

– Templates for class and school participation reports


14

School Climate

• The TAG will review results from the 2016 pilot to inform requirements for participation and overall school performance.

• Ideally both school status and growth will be included within the context of the accountability model.

• Data from the 2017 pilot will be used to establish baseline thresholds for expected growth.

• Final thresholds defining different levels of growth and status will be set by the PJP in Fall of 2018.


15

Engagement

• This category represents an attempt to gather evidence to support the claim that students are fully participating in experiences that support the acquisition of skills that positively influence holistic development and success.

• The TAG acknowledged that it takes more than just academic skills to help students succeed and that one on one mentoring and planning is critical to help students in alternative schools achieve their goals.


16

Student Success Plan

• TAG recommended that engagement be addressed by the student success plan (SSP)

• The SSP is a student-specific plan or set of objectives to accumulate evidence of positive engagement within and across school years.

• Areas addressed in the SSP include:

– Personal Data

– Educational History

– Learning Style

– Goals and Interests

– Course/Credit History and Planning, and Hathaway Progress (includes attendance)

– School and Community/Civic Activities

– Work Experience

– Career Preparation Activities and Artifacts


17

School Success Plan - Pilot

• Volunteer pilot of SSP in 2016-17

• Goal: obtain feedback related to the utility and feasibility of the SSP process and identify supports/resources necessary to support implementation.

• Draft guidelines for implementation and a model SSP template have been developed for use during in the pilot.

• Guidelines outline the purpose and intent of the SSP, roles and responsibilities underlying the SSP process, and caveats underlying the use of locally developed materials, templates and procedures.

– It is expected that SSP mentors will meet with their assigned students at least two times per school year to maintain the SSP and associated evidence.

–Procedures and materials must promote review and collection of evidence in the same areas addressed within the SSP.


18

Student Success Plan - Pilot

• After the 2016 pilot the TAG will:

– consider feedback and comments provided by stakeholders, with the goal of making necessary improvements to the process and materials

– determine how a school’s implementation of the SSP should be documented and endorsed for inclusion in the alternative school accountability system

– pilot this process in 2017-2018

• Current plan is to have principals submit a document to WDE acknowledging that:

– the school participated in the SSP process

– materials resulting from the process are available upon request


19

• TAG recommendations:

– Produce for grades 9, 10, and 11

– Operationalize a the percentage of credits earned to credits attempted.

– Obtained from student transcripts

• Proposed rules for calculating credit earning at the school level are outlined in the Accountability Pilot Implementation Handbook

• Credit earning results will be evaluated following pilot to inform business rules (e.g., enrollment period) and performance expectations.

Readiness - Credit Earning

20Alternative Schools , Select Committee

9/16

Readiness: Attainment

• The alternative school accountability framework calls for an attainment index to measure the extent to which students earn valued outcomes when they exit the school.

• TAG recommends the following outcomes be included:

– Graduation (includes extended time graduation)

– Hathaway scholarship eligibility

– High School Equivalency Certification (HSEC)

– Post-secondary credit while in high-school (e.g. dual enrollment, AP/IB)

– Career industry certification

• WDE and the TAG will evaluate data obtained following 2016 and 2017 pilot to recommend points associated with different outcomes and procedures for aggregation.

• PJP will establish the final thresholds and aggregation rules in fall of 2018.


21

Student Achievement and Growth

• It is expected that the same measures used in the general model will apply to the alternative model

• However, the achievement score for alternative schools in 2016-2017 will include results from the Grade 9-10 ACT Aspire in addition to Grade 11 ACT performance

• The TAG recommended an achievement index for the pilot year that has 5 performance levels, with values of 20, 40, 60, 80 and 100.

– Students receive an score for each test taken within a given grade

– Students who score Proficient or better on the ACT in grade 11 will receive all 100 score points for that content area.

– Scaled score ranges will be established using 2016 statewide frequency distributions.

• The alternative model also incorporates the Student Growth Percentile (SGP) as used in the general model


22

Next Steps

• Review data and feedback resulting from pilot administration

• Make revisions as needed to support the 2017-2018 pilot

• Revise data collection requirements

• Revise business rules, implementation guidance for 2017-2018 pilot

• Establish data and recommendations that inform the work of the PJP in fall of 2018.


23

Documents

Wyoming Alternative School Accountability SystemPresentation to the Select Committee on Statewide Education Accountability September 21, 2016 . Legislative Charge (1) The department