Performance Assessment for Accountability: Measuring

what Matters or Distracting from the Classroom

6/30/2017

Presenters

• Caroline Wylie

Research Director, Educational Testing Service, Princeton, NJ

• Joyce Zurkowski

Executive Director of Assessment, Colorado Department of Education

• John Weiss

Director, Bureau of Assessment and Accountability, Pennsylvania Department of Education

2

Session Outline

• A brief history of performance assessments

• Overview of state work with respect to performance assessment

• Colorado

• Pennsylvania

• Discussion questions to further explore each state approach

• Audience discussion

3

What’s In a Name?

4

A performance task is a structured situation in which stimulus materials and a request for information or action are presented to an individual, who generates a response that can be rated for quality using explicit standards. The standards may apply to the final product or to the process of creating it. A performance assessment is a collection of performance tasks.

(Stecher, 2010)

“”

Middle Ages to 19th Century

Modern Era 1900 to 1980

Post Modern Era 1980 to 2001

2001 -2010210 B.C.E. to Middle Ages

2010 present

Chinese assessments

Performance assessments for entrance to civil service and for military officers

1916 Norms used in Boston public schools to compare students

Renewed interest in performance assessments (by 1990, 24 states interested in, developing or using performance assessments)

In Europe

Certifying guild members: moving from apprentice to journeyman to master required demonstration of skill – a “masterpiece”

2001 No Child Left Behind

Many states moved away from performance assessments as tested grade requirements increased, need for individual student scores, and concerns about cost and technical quality

Brief History

1. Pre-modern from 210 B.C.E. to 1900 C.E.

2. 1900 to 1980

3. 1980 to 2001

4. 2001 to present

Four Eras:

University Oral Examinations

Oral examinations popular in an era when paper is scarce and expensive. Technology drives format.

In Europe

Introduction of greater standardization (rubrics) , and written examinations

1955 Optical scanner

1914 First multiple choice item used

1970s First CAT 2010 Common Core

State Standards released and assessment consortia formed

Re-renewed interest in performance assessments

2015 New Hampshire PACE model

Pros and Cons of Performance Assessment

6

Advantages• Authentic (or more authentic) than some other forms of assessment• Engages students in demonstrating their learning by presenting them with “real-world”

problems• Impacts classroom teaching and learning by sending a signal to teachers about the

kind of content and performance that is valued (greater emphasis in mathematical practices such as problem-solving or extended writing and revising in ELA)

• Use of rubrics etc can help students understand complex learning goals • Aligned with current theories of learning

Challenges• Negative impact on teaching such as “rubric-driven instruction”• Technical quality concerns, particularly with open-ended portfolios (although group

level reliability can be very high even as student level reliability is adequate)• Time and cost to develop tasks with clear rubrics and to train raters• Rater reliability issues• Teacher professional development costs• Time consuming

Overview of State Work

7

The Centennial State

8

What are our goals for performance-based assessment?

• Increased educator, parent and student relevance

• Clearly demonstrate the connection between instruction and assessment

• Timely results

• Focus on what matters – higher order thinking skills and concepts

• Improved instruction and therefore, increased student learning

9

What have we learned through other change experiences?

• Forced high stakes uses and innovation are a problematic combination.

• The association of accountability does not always provide a positive incentive for new learning.

10

Key Influences on Our Performance Assessment Work

• Content Collaboratives

• Colorado Assessment Literacy Program

• Graduation Guidelines

• State Law

11

Content Collaboratives• Assessment Resource Bank

• Reviews of assessments in each content area (degree of alignment to the Colorado Academic Standards and appropriateness for use in Colorado classrooms) (2012-2013)

• Performance Assessment Development Process

• Performance Assessment/Task Development

• Rubric Development

• Performance Assessment Trials

• Inter-Rater Agreement

• Stanford Center for Assessment, Learning and Equity

• District Sample Curriculum

• Curriculum Overview Samples (2012)

• Refining Workshops (Spring 2013)

• Instructional Unit Samples (Fall 2013 - Winter 2015)

• High Impact Instructional Strategies (Summer 2016 - Present)

12

Colorado Assessment Literacy Program

http://www.cde.state.co.us/assessment/coassessmentlitprog

Colorado Assessment Literacy Online Professional Learning Modules

Creating Meaningful Classroom Assessments - Workshop

Assessment Design Toolkit

CDE Assessment Resource Bank

Formative Assessment Insights Course

• Adams State University

• Aurora Public Schools

• Centennial BOCES

• Eagle School District

• Estes Park School District

• Fort Morgan School District

• Fountain-Ft. Carson School District

• Global Village Academy

• Harrison School District

• Las Animas School District

• Moffat County School District

• Monte Vista School District

• Montrose School District

• Northwest BOCES

• Norwood Schools

• Pikes Peak BOCES

• Platte Valley School District

• Salida Schools

• San Luis Valley BOCES

• Santa Fe Trails BOCES

• Steamboat Springs School District

Colorado Assessment Literacy Program:Participating Districts

Graduation Guidelines

In September 2015, the Colorado State Board of Education approved a Graduation Guidelines Menu of College and Career Ready Demonstration Options that requires local school boards to establish high school graduation requirements that meet or exceed the Colorado Graduation Guidelines for the graduating class of 2021.

- Students must demonstrate college or career readiness in English and math based on at least one assessment measure, which will include an option for Collaboratively-developed, Standards-based Performance Assessments.

15

Graduation Guidelines

• Collaboratively-Developed, Standards-Based Performance Assessments:

• Serve as an option for students to demonstrate Postsecondary and Workforce Readiness in Math and English

• Create a process by which supports to the field would be provided in the development of these performance assessments as part of their local assessment systems

• Honor and build upon existing work• Co-create tools, processes, and resources • Design, implement, score, and validate

performance assessments.

State Law

AS SOON AS PRACTICABLE AFTER THE EFFECTIVE DATE OF THIS SUBSECTION (2.5), THE DEPARTMENT OF EDUCATION SHALL APPLY TO THE FEDERAL DEPARTMENT OF EDUCATION FOR INNOVATIVE ASSESSMENT AND ACCOUNTABILITY DEMONSTRATION AUTHORITY AS AUTHORIZED IN SECTION 1201 OF TITLE I OF PART B OF THE FEDERAL "EVERY STUDENT SUCCEEDS ACT", PUB. L. 114-95, ENACTED BY THE 114TH CONGRESS.

H.B. 16-1234

17

Innovative Assessment and Accountability Demonstration Authority

• Innovative assessment systems may make use of:

• “competency-based assessments, instructionally embedded assessments, interim assessments, performance-based assessments that combine into an annual summative determination for a student,” and

• “assessments that validate when students are ready to demonstrate mastery or proficiency and allow for differentiated support based on individual learning needs”

--ESSA, 2015, Section 1204(a)

18

Next Phase

• The department will be working with interested districts to build their local assessment systems to include more performance-based assessments options that may be used to satisfy local high school graduation requirements.

• The department intends to leverage the work related to assessment literacy and graduation guidelines to inform our innovative assessment and accountability proposal.

19

Targeted Outcomes

• Create a validated process for developing, identifying, administering, and scoring performance assessments for high school graduation purposes;

• Advance educators’ instructional and assessment practices to enhance learning experiences to include, more substantively, the opportunity for students to apply their academic content knowledge through demonstrated, effective use of 21st

century/employability skills, and effective, guided practice of giving and receiving feedback; and

• Develop well-articulated, coherent local systems of assessment.

20

21

The Keystone

State

Performance Assessment

• According to ETS, a performance assessment is, “A test in which the test taker actually demonstrates the skills the test is intended to measure by doing real world tasks that require those skills, rather than by answering questions asking how to do them.”

Performance Assessment

Performance assessment evaluates thinking skills such as

• Analysis

• Synthesis

• Evaluation

• Interpretation of facts and ideas

Pennsylvania Driver’s Manual

• The Knowledge Test consists of 18 multiple choice questions.

• You must answer 15 questions correctly to pass the test.

Performance

• Children under 18 must complete 65 hours of behind the wheel practice driving before eligible to take the driver’s test

– 10 of those hours must be nighttime driving

– 5 of those hours must be in inclement weather

• Individual is now prepared to take the “driving” test

Performance Assessment

• Answering the questions on a written driver’s test may demonstrate what a person knows, or correctly “guessed,” but the person must also demonstrate what he/she can do.

– Analyze

– Synthesize

– Evaluate

– Interpret

Standards

What a student should know and be able to do

House Bill 202/Act 6

• Signed into law by Governor Tom Wolf, June 21, 2017

• Allows for CTE students to graduate by demonstrating a high likelihood of success on an approved industry-based competency assessment

– NOCTI, NIMS

Moving away from test emphasis

• School Performance Profile

• Test scores make up 80% of the score

• Future Ready PA Index

• Test scores make up less than 50%.

PDE believes

• Standardized assessments are not the exclusive measure of college and/or career readiness

• There should be multiple options to demonstrate college and/or career readiness

• Some indicators of college and/or career readiness should be measured at the local level

• Students need not be equally strong in all academic areas to demonstrate college and/or career readiness

• Graduation requirements should incentivize holistic instructional opportunities and should not result in a narrowing of the curriculum

Discussion Questions

• In an ideal balanced assessment system where would you locate the use of performance assessments?

• What are your views on balancing the trade-offs for performance assessments?

• What are the key policy (or other) conditions necessary at the state level to support performance assessment as a component of summative/accountability assessment?

• How can state departments of education support performance assessment at the school and district level?

38

39

40