Accountability for Alternative Education Lessons Learned from States and Districts
November 14, 2016
@CCRSCenter @AYPF_Tweets#AltEdAccountability
GoToWebinar Technical Assistance: 1-800-263-6317
To submit live questions, please use the “Questions” box
A recording of the webinar and other resources will be available at www.ccrscenter.org and www.aypf.org.
Webinar Technical Support
#AltEdAccountability
CCRS Center
Who? What? How?
State Department of Education Agencies
Build SEA capacity to implement college and career ready policies
Provide technical assistance, including universal, targeted and intensive support
www.ccrscenter.org
#AltEdAccountability
§ Jennifer L. DePaoli, Senior Education Advisor, Civic Enterprises; co-author of Building a Grad Nation: Progress and Challenge in Raising High School Graduation Rates
§ Jessica Knevals, Principal Consultant, Accountability and Data Analysis Office, Colorado Department of Education
§ Kirsten Plumeau, Program Director, Contracted Alternative Schools, Portland Public Schools
§ Carla Gay, Director of Early Warning Systems, Portland Public Schools § Carinne Deeds, Policy Associate, American Youth Policy Forum;
co-author of What Can States Learn About College and Career Readiness Accountability Measures from Alternative Education?
Today’s Presenters
#AltEdAccountability
Low-Graduation-RateHighSchools
TokeepinlinewithESSA,wemovedfromlookingsolelyatthelargehighschools(300ormorestudents)producingsignificantnumbersofnon-graduatestofurtherexaminingthehighschoolsenrolling100ormorestudentsthatreportedanACGRof67percentorless.
Low-Graduation-RateSchools§ Whenexamininglow-graduation-ratehighschoolsbytype:
§ 41%areregulardistrictschools§ 28%arealternativeschools§ 26%arecharterschools§ 7%arevirtualschools
§ Tobreakitdownfurther:§ Whenremovingalternativecharterandalternativevirtualfromthealternativeschoolcategory(10and2percentoftheseschools,respectively),23percentofalllow-graduation-ratehighschoolswerealternativeschools(district-operated).
§ Thisallowsustofocusmoreintentlyontheschoolsthatmakeuplargepercentagesineachschooltypecategory.
Challenges&LimitationsofCurrentData
• Identificationofprogramsvs.schools• Misidentificationofalternativeprograms/schoolsinfederaldata– Apublicelementary/secondaryschoolthat(1)addressesneedsofstudentsthattypicallycannotbemetinaregularschool,(2)providesnontraditionaleducation,(3)servesasanadjuncttoaregularschool,or(4)fallsoutsidethecategoriesofregular,specialeducation,orvocationaleducation(NCES).
• Issueswithusingafour-yearcohortgradrate
Extended-YearGraduationRates
§ Five-yeargraduationrateswereavailablefor31states,across73graduatingcohortsoverfouryears.§ Onaverage,five-yearratesledtoathreepercentincreaseinoverallgraduationrates.
§ Six-yeargraduationrateswereavailablefor23graduatingcohortsin13states.§ Six-yeargradratesshowedanaveragegainofonepercent.
§ Whenfactoringin5- and6-yeargraduationrates,thenationalgraduationratewouldbecloserto86-87%.
Background on Alternative Education Campuses (AECs) in Colorado
JessicaKnevals,MPAAccountabilityandPolicySpecialist,AccountabilityandDataAnalysisColoradoDepartmentofEducation
§ InColorado,schoolsthatserveprimarilyhigh-riskstudentsarecalled“AlternativeEducationCampuses”orAECsforshort.
§ Asof2014,Coloradohad84AECswhichservejustover16,000students
§ AECsareoutlinedinC.R.S.22-7-604.5asschools:§ (I)“Havingaspecializedmissionandservingaspecialneedsorat-riskpopulation”,
§ (V)“Havingnontraditionalmethodsofinstructiondelivery”,§ (VI)(A)“Servingstudentswhohaveseverelimitations…”,and§ (VI)(B)“Servingastudentpopulationinwhichmorethan90% ofthestudentshaveanindividualizededucationprogram…ormeetthedefinitionofahigh-riskstudent”.
Alternative Education Campuses
§ juveniledelinquent§ droppedoutofschool§ expelledfromschool§ historyofpersonaldrugor
alcoholuse§ historyofpersonalstreet
ganginvolvement§ historyofchildabuseor
neglect§ hasaparentorguardianin
prison§ hasanIEP
§ familyhistoryofdomesticviolence
§ repeatedschoolsuspensions§ parentorpregnantwoman§ migrantchild*§ homelesschild§ historyofaserious
psychiatricorbehavioraldisorder*
§ isovertraditionalschoolageforhisorhergradelevelandlacksadequatecredithoursforhisorhergradelevel**
“High-Risk Student” is a student who has one or more of the following conditions
History of AECs in Colorado
2002
• C.R.S.22-7-604.5
• EstablisheddefinitionofAECs
2008
• COCoalitionofAltEdCampusescommissionedtoestablishbasicframeworkforalt.ed.
2009
• SB09-163,COEducationAccountabilityAct
• DeterminedAECsnolongerexemptfromaccountability
2010
• SchoolPerformanceFramework(SPF)forAECsincludesAcademicAchievement,AcademicGrowth,StudentEngagement,andPostsecondaryandWorkforceReadiness
2011• AECsallowedtoincludeoptionalmeasuresinSchoolPerformanceFramework
2015• HB15-1350• CreatedAECaccountabilityworkgrouptorefineandupdatethecurrentAECaccountabilitysystem
2016• HB16-1429(basedonworkgrouprecs)
•Modifiesminimum%ofhigh-riskstudentsandcertain“high-riskindicators”
School and District Performance Frameworks & AEC School Performance Framework
Achieve-ment15%
Growth35%
StudentEngage-ment20%
PWR30%
AlternativeEducationCampuses
Achievement40%
Growth&GrowthGaps
60%
ElementaryandMiddleSchools
Achievement30%
Growth&GrowthGaps
40%
PWR30%
HighSchoolsandDistricts
AEC Accountability: Flexibility with Optional Measures
22
AlternativeEducationCampusesreceiveaSchoolPerformanceFrameworkannually, s imilar totradit ional schools. Themainexception is AECsaremeasuredonStudentEngagementmeasure, rather thanGrowthGaps.PerformanceIndicator
Weight State-RequiredMeasuresandMetrics
Optional MeasuresandMetricsE/MS HS
AcademicAchievement
20% 15% CMAS/PARCC%ofstudentsproficientinReading,Math,Writing,Science
NWEAMAP, Scantron, Acuity, Galileo,WideRangeAchievementTest(WRAT),TestforAdultBasicEducation(TABE),STAR,and/orAccuplacer
AcademicGrowth
50% 35% CMAS/PARCCmediangrowthpercentilesinReading,Math,Writing,andACCESS(Englishlanguageproficiency)
NWEAMAP, Scantron, Acuity, Galileo,WideRangeAchievementTest(WRAT),TestforAdultBasicEducation(TABE),ACCESS,STAR,and/orAccuplacer
StudentEngagement
30% 20% 1. Attendancerate2. Truancyrate
1. StudentRe-engagement,2. Returningstudents,3. Socio-EmotionalorPsychological
AdjustmentPostsecondary&WorkforceReadiness
N/A 30% 1. Completionrate(bestof4,5,6,or7yearrate)
2. Dropoutrate3. ColoradoACTscore
(average)
1. Credit/coursecompletion,2. WorkforceReadiness,3. Post-CompletionSuccess,4. Successfultransition(fornon-degree
grantingschoolsonly),5. Graduationrate
Use of Additional Measures on 2014 AEC SPF
PerformanceIndicator
SchoolHasState-RequiredMeasuresandMetricsPlusSubmittedOptionalMeasuresandMetrics
SchoolOnlyHasState-RequiredMeasuresandMetrics
SchoolDoesNotHaveState-RequiredMeasuresandMetricsbutSubmittedOptionalMeasuresandMetrics
TotalPercentageofAECSchools thatSubmittedOptionalMeasures
AcademicAchievement
36.5% 41.3% 11.1% 47.6%
AcademicGrowth 52.4% 28.6% 17.5% 69.8%
StudentEngagement 47.6% 50.1% 0.0% 47.6%
Postsecondary&WorkforceReadiness
44.4% 55.6% 0.0% 44.0%
§ Schoolsreceivearating oneachoftheperformanceindicators:§Exceeds (4pts),Meets (3),Approaching (2),DoesNotMeet(1)
§ Theratingsrolluptoanoverallevaluationoftheschool’sperformance,whichdeterminestheschoolplantyperating:§Performance,Improvement,PriorityImprovement,Turnaround
§ UnderSB09-163,the“ColoradoEducationAccountabilityAct”,ifapublicschoolisrequiredtoimplementapriorityimprovementplanorturnaroundplanfor5consecutiveschoolyears,thestateboardmustrecommendthatthepublicschool'sschooldistrictortheinstitutetakeoneofseveralactionsspecifiedinstatutewithregardtothepublicschool.
AEC School Performance Framework: Indicator Ratings & Overall Rating
§ AECsinColoradoaremeasuredsimilarlytotraditionalschools,buttheweightingsareloweredtotakeintoaccountthehigh-riskpopulationserved.
§ Withoutallowingadditionalmeasuresandrevisedcut-points intheAECSPF,86%ofAECswouldbeonpriorityimprovement orturnaround plans,whereas,nowonly24%were.
§ AECsinColoradoaregraduallyimprovingovertime.In2011,39%ofAECswereonpriorityimprovement orturnaroundplans,andin2014,only24%were.
§ AECsonlyconstitute5%oftotalschoolsinColorado,ofthe190schoolsonpriorityimprovement orturnaround plans,
§ However,21ofthose190areAECs,whichrepresents11%ofallschoolsonpriorityimprovement orturnaround plans.
Adjusted AEC SPF Cut-Points:Impact for Accountability
TheDepartmentofEducationshallconvenestakeholdermeetingswiththepurposetoproviderecommendationstothe
Commissioner,theeducationcommitteesoftheHouseofRepresentativesandtheSenate,andtheStateBoardofEducationregardingperformanceindicatorsforthenextiterationoftheAlternativeEducationCampusSchool
PerformanceFramework(AECSPF)forreleaseinthefallof2016.
Purpose and Charge for the AEC AWG
StateDept.ofEd.
Districts
CommunityMembersParentsStudents
AECSchools
AEC Work Group Participants
ThecommissionerselectedatleastoneworkgroupmemberfromeachofthesubcategoriesoutlinedinHB15-1350tocomprehensively
representtheAECcommunityinColorado.
LargeDistrictSmallDistrict
CharterSchoolInstitute
AccountabilityOfficeDropoutPrevention&
StudentRe-EngagementOffice
DropoutRe-engagementSchoolConcurrentEnrollmentSchoolOnlineSchoolCharterSchoolIEPSchoolPart-timeSchoolOAUCSchool
Charge of the Work Group
QualitativeandQuantitativeMeasures
Measure-specificcutpoints
Currentweightingsystem
Investigateacomparisongrouptocomparehigh-riskstudentsacross
schools
95%high-riskthresholdasAlt.Ed.Campusdesignation&studentgroups
included
Changes Needed for AEC Work Group Charge
95%thresholdforAlt.Ed.Campusdesignation
&studentgroupsincluded
Developmentofmeasure-specificcut
pointsWeightingSystem
Methods/costsassociatedwithusingcross-schoolstudentcomparisongroups
Documentationandverification methodsfor
certifyingthat95%thresholdhasbeenmet
QualitativeandQuantitativeMeasures
Statute
Rule
Policy
Recommendations of the AEC Accountability Work Group
•OpportunityMeasuresindicatoruniquetoaschool’sdesignandmission
•PilotaschoolqualityreviewprocessQualitativeandQuantitativeMeasures
•ProposedaprocessfordeterminingAECappropriatecut-pointsforAECSPFmeasures
• CreatedaguidanceforhowallmeasuresaredevelopedforAECsMeasure-specificcutpoints
•Weighachievementandgrowthresultsbythenumberofstudentsincludedineachmeasure(asopposedtoweightingeachmeasureequally).
Currentweightingsystem
• IdentifyacomparisongroupbyusingeasilyavailabledataforidentifyinghighriskconditionsbasedonAECstudent’scharacteristicspriortoenrollingintheAEC.
Investigateacomparisongrouptocomparehigh-riskstudentsacross
schools
• Lowerthehigh-riskthresholdfordesignationofanalternativeeducationcampusfrom95%to90%high-risk
• Expand5criteriaofstudentgroupsincludedinhigh-riskthreshold
95%high-riskthresholdasAlt.Ed.Campusdesignation&studentgroups
included
EARLY WARNING SYSTEMS AND APPROPRIATE ACCOUNTABILITY METRICSNOVEMBER 14, 2016
Carla Gay, Director Early Warning Systems
Kirsten Plumeau, Director Contracted Alternative Schools
OVERVIEW OF CONTRACTING IN PORTLAND PUBLIC SCHOOLS
¡ Determining Contractors: Five year bid process, all contracts are reviewed and renewed annually
¡ Annual Contracts: Calls for alternative accountability measures
¡ Alternative Schools: Programs (non-profits or other private alternative schools) with data that feeds the district data
¡ Paying Contractors: Oregon law allows district to use state school dollars to pay for contracted students at either the full amount or at 80% of per pupil net operating expense-based on daily attendance
¡ Attending an Alternative: Students must meet one of the three indicators – Attendance, Behavior, Course Performance
THE ACCOUNTABILITY PROCESS
¡Metrics established by PPS staff and alternative school leaders over a two-year period
¡Use the Annual CBO Program Accountability Goals form to establish goals with each school
¡Data is compiled at the end of the year to create the Alternative Accountability Report Card
KEY METRICS DESCRIPTION INCLUSION CRITERIA TARGET
ACADEMIC PROGRESS:Skill Growth in Reading and Math on either MAP orCASAS
Percent of Students who meet or exceed growth targets
Students who have:-45 Days/75 Hours of Enrollment-Pre and Post Scores-Reading: Pre-Score Below 10th
Grade Reading
MAP:3 point gains
CASAS:5 point gains
SUCCESSFUL COMPLETION:Postsecondary Readiness
Percent of students who meet/exceed target college readiness scores on COMPASS or ACT
Students who take the COMPASSat PCC or ACT through PPS.
COMPASS: ACT:88 Reading 18 English56 Algebra 22 Math
SCHOOL CONNECTION: Annual Retention Rate
Percent of studentsenrolled at the end of the school year who remained enrolled or completed
Students enrolled at end of year who did not transfer outside of the district
-complete with HSD or GED or-remain enrolled
SCHOOL CLIMATE:School Climate
Under Development Under Development Under Development
EARLY WARNING SYSTEM (EWS)
¡ EWS Indicators¡ Attendance¡ Behavior ¡ Course Performance
¡ Goals¡ To promote the systemic use of data¡ To use data to identify, intervene and monitor students
¡ To intervene early
DROPPING OUT IS A PROCESS NOT AN EVENT
A COMPREHENSIVE STRATEGY
Prevention Intervention Intensive Intervention Reengagement
Teams use data to determine and apply
appropriate interventions based on
level of risk
Establish a proactive system for identifying
indicators of risk factorsEliminate the
dropout pipeline
Monitor and evaluate impact of interventions so that fewer students
require intensive intervention and reengagement
The Goal = Align the Data Tracking and Intervention Efforts to Support All Students
Attendance Behavior Course Performance
An Early Warning System Supports
Multi-tiered Systems of Support (MTSS)
ALIGNED IMPLEMENTATION
Tier 1 Universal Supports= Prevention
Tier II Targeted Supports = Intervention
Tier III Intensive Supports = Intensive Intervention and Reengagement
DATA TRACKING SYSTEMS
DESIGN APPROPRIATE METRICS: THE ABCS OF DIFFERENTIATED EWS METRICS
TIERS ATTENDANCESchool Connection
BEHAVIORSchool Climate
COURSE PERFORMANCE
Academic Progress Successful CompletionI Maintain 90% or better attendance Zero behavioral incidents; sense of
belonging and goalsStandard 1 year growth in 1 year
4 year, “on-time” graduation
II Improved attendance with 90% or better
1-2 behavior incidents or referrals; target is 0 exclusionary disc
Accelerated growth in 1year
4 & 5 year graduation, with HS diploma
III Improved attendance with target of 80% or better
Fewer behavior incidents or referrals than prior year; target is 0 exclusionary disc
Accelerated growth over 2 years
5 year HS diploma or equivalent
IV Improved attendance from prior school enrollment; target of 90% or greater in alt setting
After returning to school, improved self-management and goal setting; individualized metrics
Accelerated growth over 2 + years
5 -8 years HS diploma or equivalent
What Can States Learn About College and Career Readiness Accountability Measures from Alternative Education?
Carinne DeedsPolicy Associate, AYPF
November 14, 2016
• Co-authored with Zachary Malter, Policy Research Assistant, AYPF
• Overview of “alternative” accountability measures used by states and districts to assess college and career readiness
• Focused on settings that serve at-risk or high-risk students
About the Brief
Participants in Alternative EducationAlternative schools are designed to serve at-risk students who are:
• Chronically absent• Pregnant/parenting• Have disciplinary problems• Re-engaging with school• Primary caregivers• Returning from incarceration/adjudication• Wards of the state• In need of extra assistance
Types of Alternative EducationSetting • Alternative Schools/Campuses
• Alternative Programs within Traditional School
Instructional Format • Traditional Learning• Online/Blended Learning• Personalized Learning
Authorizer • State Mandated, Authorized and Defined
• Locally Mandated, Authorized, and Defined
Participation • Mandatory• Voluntary
§ Assess college and career readiness using a variety of measures throughout students’ academic trajectories.
§ Identify measures that reflect the overall growth of the student and not just academic proficiency.
§ Leverage ESSA to support at-risk students.
Considerations for States
Note: While critically important for alternative settings, these measures can be useful for all students in all settings.
Jennifer DePaoliCivic [email protected]
Jessica KnevalsColorado Department of Education [email protected]
Kirsten PlumeauPortland Public [email protected]
Carla GayPortland Public Schools [email protected]
Carinne DeedsAmerican Youth Policy [email protected]
Contact Today’s Presenters
#AltEdAccountability