Copyright © 2013 by Educational Testing Service. All rights reserved. Vertical Articulation in the Context of States’ Transition to the Common Core State

Copyright © 2013 by Educational Testing Service. All rights reserved.

Vertical Articulation in the Context of States’ Transition to the

Common Core State Standards

How did we get here, Where are we now, and How is that working out?

National Conference on Student AssessmentJune 25-27, 2014


How did we get here?

Vertical Articulation in the Context of States’ Transition to the Common Core State Standards

Patricia A. BaronEducational Testing Service

National Conference on Student AssessmentJune 25-27, 2014

Copyright © 2013 by Educational Testing Service. All rights reserved.Copyright © 2013 by Educational Testing Service. All rights reserved.

Where are we?• The current standards and testing movement is in

transition from the multi-state standards to the Common Core State Standards (45 states and DC)

http://www.academicbenchmarks.com/ccss-state-status


How did we get here?Historical Highlights

• U.S. Federal Policy and Initiatives • Research – ongoing and results since 1995


US Education Policy

1965 1969 1990 2001 2006 2009Title I ESEA Launched development of the field of educational evaluation and school accountability

NAEPSubject-area National Survey, Grades 4,8, and 12; subject area frameworks developed by NAGB

NAEP State Assessments

No Child Left Behind Act Annual testing for all subjects in grades 3-8; Annual progress objectives; State Standards in reading and mathematics

ED Invites states to include achievement and growth models; vertical scaling not required but promise of greater comparability

RTT Created to spur innovation & reforms in state & local districts; points for complying with CCSS


Setting the Stage

Three Related Areas of Research and Practice

1. Defining proficiency 2. Linking tests. Linking scales. Vertical

scales and growth. 3. Standard Setting: Content and

Performance Standards


Defining proficiency

1968NAEP: The Nation’s Report Card.Introduced as a data gathering tool and not an accountability tool;

1988Scale anchors included;NAGB changed from Can Do to …Should Do ;Standard setting process under fire

2001

NCLB –Increased the amount of testing and increased the consequences to the statesStates defined proficiency according to state content standards, grade-level assessments and proficiency standards

2003

Currently the variability in the stringency of state standards defining proficiency is so great that the concept of proficient achievement lacks meaning (Linn)

The Common Core is designed to promote deep learning through rigorous standards aligned with college and career readiness

Defining proficiency


Vertical Scales & Growth Models

• Vertical scales introduced with ED invitation (2006) to include growth models in state performance reporting.

• Methods existed for linking different tests, different scales– Linking statewide tests to NAEP, accuracy of

combining test results across states (Linn and Kiplinger, Ercikan)

– Linking NAEP to international tests, e.g., TIMMS (Johnson)



• Issues for measuring growth & change scores– Scales measuring latent variables such as

proficiency or achievement are not equal interval scales (e.g., Patz, 2007).

____+____+___+__+__+__+– Valid vertical scales require a set of content

standards that provide continuity across the grade span

• Learning progressions (e.g., Wilson, 2009)



• Issues for measuring growth & change scores– Linking tests across grades without an

anchor (common items) problematic– Standard setting conducted by grade not

sufficient


Standard Setting: Content and Performance Standards

Content Standards: Content standards define the knowledge, concepts, and skills that students should acquire at each grade level.

Performance Standards: These standards specify how much understanding of content students need at each level of performance (e.g., basic, proficient, advanced), relative to the content standards.


Standard Setting: Content and Performance Standards

• Best practice in standard setting– appropriately-informed panelists – alignment between test and content standards

• Cross-grade expectations and learning progressions:– Cohesive content standards, e.g., Grades 3 to 4

to 5, …Grade 11.

• Options for cross-grade alignment of performance standards– Vertical scaling and vertical moderation


Good sources for further reading

• The Future of Test-Based Educational Accountability, Ed. Ryan & Shepard, (2008)

• Vertical Scaling in Standards-Based Educational Assessment and Accountability Systems, published by CCSSO, Rich Patz (2007)

• Vertical Scaling– in Test Equating, Scaling and Linking: Methods

and Practices, Kolen and Brennan (2004)


Research Highlights• 2003

– Vertical equating for state assessments: Issues and solutions in AYP and school accountability (Lissitz and Huynh)

• 2005– Vertically Moderated Standards: Special Issue of Applied

Measurement in Education

• 2009– Impact of vertical scaling decisions on growth

interpretations (Briggs and Weeks) • 2010

– Post-standard-setting panel considerations for decision-makers (Geisinger and McCormick)

• 2012– Growth, Standards and Accountabiilty (Betebenner) in

Cizek, Ed., Setting performance standards


Today

Current Research• Priya Kannan, Research Scientist, Educational

Testing ServiceCurrent Practice• Deb Lindsey, Director of Assessment, Wyoming

Department of EducationDiscussion• Marianne Perie, Co-Director at Center for

Educational Testing and Evaluation, University of Kansas

Documents

Copyright © 2013 by Educational Testing Service. All rights reserved. Vertical Articulation in the Context of States’ Transition to the Common Core State