School and District Performance Frameworks – 2018 Sub-indicator Targets

Information Item

Accountability and Data Analysis UnitOctober 12, 2017

10/3/2017 2

• Review State Board of Education’s role in target setting for state accountability

• Review existing target setting methodology and discuss plans for new PSAT 9 results

• Review existing baseline achievement targets and compare to most recent 2017 results

• Prepare Board to vote on the 2018 school and district performance framework targets at the November 2017 meeting

10/3/2017 3

• 22-11-201(1) (a) The state board shall annually review the performance of the statewide public education system, including but not limited to reviewing the levels of attainment on the performance indicators demonstrated by each public school, each school district, and the institute. Following said review, the state board shall set, reaffirm, or revise, as appropriate, ambitious but attainable statewide targets for the measures used to determine the levels of attainment of the performance indicatorsfor the coming academic year with the goal of raising the level of academic performance in the public schools throughout the state. In setting the targets, the state board shall, to the extent possible, ensure that the targets meet federal law requirements.

10/3/2017 4

These:

Not these:

10/3/2017 6

Elementary Middle High

Academic Achievement

English Language Arts /EBRW* X

Mathematics X

Science

Academic Growth

English Language Arts

Adequate Growth MetricMathematics

English Language Proficiency

Postsecondary and Workforce

Readiness

Graduation Rate

Dropout Rate

CO SAT EBRW*

CO SAT Mathematics

Matriculation Rate“Other

Indicator”Change in chronic absenteeism**

X X

* Evidence-Based Reading and Writing (EBRW)** If directed by the state legislature to include in state accountability.

10/3/2017 7

• HB 17-1181 required the department to select a 9th

grade exam that was aligned to Colorado’s 10th and 11th grade exams, thereby creating a cohesive suite of assessments for high school.

• The grade 9 PSAT was selected because it is aligned in content, scales, reporting categories, and scores to Colorado’s existing high school assessments, the PSAT 10 and SAT.

• The SAT suite of assessments will provide the state, districts, and schools with a longitudinal, evidence based system that measures growth in relation to essential college and career readiness success outcomes from grade 9 through grade 11.

10/3/2017 8

• Ninth graders attending public schools in Colorado will begin taking the PSAT 9 in the spring of 2018. • 2017 PSAT 10 results were included on the 2017 school and

district performance frameworks

• For the 2018 performance frameworks, CDE is currently intending to include PSAT 9 results: • along with PSAT 10 results in the Achievement indicator

• by content area (Evidence Based Reading and Writing and Mathematics)

• with PSAT 10 results from 2016 and 2017 for multi-year framework calculations

• in growth calculations from CMAS grade 8 to PSAT 9 and PSAT 9 to PSAT 10

10/3/2017 9

• Student results from the Spring 2018 PSAT 9 administration will be available in late June. Until then the specific values for the performance framework targets cannot be calculated.

• Instead, CDE will suggest that the state board approve the calculation methodology for establishing performance framework targets.

• As soon as the 2018 PSAT 9 data become available, this methodology will be used to set the specifictargets for the 2018 school and district performance frameworks.

10/3/2017 10

• To align with the other performance framework targets, CDE recommends the following norm-based percentile rank methodology for setting PSAT 9/10 targets for 2018.

• PSAT 10 targets were temporarily baselined on the 2016-2017 data but can now be revised for 2018 with the inclusion of PSAT 9 results.

Sub-indicator RatingPercentile

Target

Percentile

RangeRationale for Target

Exceeds Expectations 85th 85-100 Approximately 1 standard deviation above average

Meets Expectations 50th 50-84 Average result for the state

Approaching Expectations 15th 15-49 Approximately 1 standard deviation below average

Does Not Meet Expectations - 1-14 -

10/3/2017 11

• This norm-based methodology ensures the following distribution of school-level ratings for each sub-indicator by content-area for the baseline year.

• The same targets will also be applied to the district performance framework results.

10/3/2017 12

• Adequate growth was defined in SB09-163 based on the previous state assessments (CSAP and TCAP).• Adequate growth is the idea of how much growth is needed for

a student to make a particular proficiency level goal, within a given timeframe.

• We can measure if a student is “on-track” to meet those goals based on their growth percentile.

• With the new CMAS assessments, it seems to make sense to revisit the definitions of adequate growth.

• CDE plans to work with the Longitudinal Growth Technical Advisory Panel (TAP) over the coming months to develop options for measuring how schools are helping students grow toward college and career readiness standards, based on the new assessments.

• The goal is to have a relevant and meaningful adequate growth measure on the 2018 performance frameworks.

10/3/2017 14

• A single year of data is chosen as the baseline for creating percentile rank and target values (ex. Using the 2016 state assessment data results to set the 15th

percentile as the scale score 722.3).

• Future years of data can then be compared against this consistent baseline expectation to determine whether schools and districts are making progress in meeting state expectations.

• Baselining is required for new assessments and should also be considered whenever significant changes occur in the underlying assessments or tested populations.

• Atypical scores are often seen in the first year or two of a new assessment due to implementation differences, so initial baselines should be reviewed once additional data become available.

10/3/2017 15

Elementary Middle High

Academic Achievement

English Language Arts /EBRW* 2016 2016 2018

Mathematics 2016 2016 2018

Science 2016 2016 2016

Academic Growth

English Language Arts 2016 2016 2017

Mathematics 2016 2016 2017

English Language Proficiency 2016 2016 2016

Postsecondary and Workforce

Readiness

Graduation Rate - - 2016

Dropout Rate - - 2016

CO SAT EBRW* - - 2017

CO SAT Mathematics - - 2017

Matriculation Rate - - 2016

* Evidence-Based Reading and Writing

10/3/2017 16

10/3/2017 17

10/3/2017 18

10/3/2017 19

10/3/2017 20

• For CMAS Math the 2017 15-50-85th percentiles would be between 0.3 and 1.1 scale score points higher than the current 2016 baselines.

• For CMAS English Language Arts the 2017 15-50-85th

percentiles would be between 1.0 and 2.1 scale score points higher than the current 2016 baselines.

• Given the small differences between 2016 and 2017, CDE staff would recommend continuing to use the existing 2016 baseline targets for the 2018 performance frameworks.

• This ensures consistent interpretation by the field and the opportunity for schools and districts to demonstrate improvement over time.

10/3/2017 21

10/3/2017 22

10/3/2017 23

10/3/2017 24

• At the elementary and high school levels, CMAS Science 15-50-85th percentiles for 2017 would be between 0.0 and -3.6 points below the current 2016 baseline.

• Larger declines exist at the middle school level, with 2017 CMAS Science 15-50-85th percentiles decreasing between -2.1 and -9.7 points.

• Given the small differences between 2016 and 2017 at the elementary and high school levels and the decline in middle school scores, CDE staff would recommend continuing to use the existing 2016 baseline targets for the 2018 performance frameworks.

10/3/2017 25

• State board will vote on targets and target-setting methodology at the November meeting.

• CDE will start meeting with stakeholders to discuss adequate growth options.

• Once available, PSAT 9 and PSAT 10 data from 2018 spring testing will be used to establish the actual targets for high school achievement.