The Smarter Balanced

Assessment System: An Overview

Joe Willhoft, Ph.D. Executive Director

NAESP Webinar September 18, 2012

Issues Addressed in Today’s Webinar

• How will these assessments be different from our current state assessments?

• Will all states use the same cut scores?

• Who makes decisions for Smarter Balanced?

• How will being in Smarter Balanced help my school?

• How will teachers and school leaders be involved?

• What is the timetable/roll-out for these new assessments?

Common Core State Standards

• Define the knowledge and skills students need for college and career

• Developed voluntarily and cooperatively by states; more than 40 states have adopted

• Provide clear, consistent standards in English language arts/Literacy and mathematics

Source: www.corestandards.org

A Next Generation of Assessments

• Rigorous assessment of progress toward “college and career readiness”

• Common cut scores across all Consortium states

• Provide both achievement and growth information

• Valid, reliable, and fair for all students, except those with “significant cognitive disabilities”

• Administered online

• Use multiple measures

• Operational in 2014-15 school year

US Dept. of Ed has funded two consortia of states with development

grants for new assessments aligned to Common Core State Standards

(Federal Register / Vol. 75, No. 68 / Friday, April 9, 2010 pp. 18171-85)

Smarter Balanced:

A State-led Consortium

A National Consortium of States

• 25 states representing 40% of K-12 students

• 21 governing, 4 advisory states

• Washington state is fiscal agent

• WestEd provides project management services

Smarter Balanced Theory of Action:

Seven Key Principles

1. An integrated system

2. Evidence-based approach

3. Teacher involvement

4. State-led with transparent governance

5. Focus: improving teaching and learning

6. Actionable information – multiple measures

7. Established professional standards

The Smarter Balanced

Assessment System

A Balanced Assessment System

Common

Core State

Standards

specify

K-12

expectations

for college

and career

readiness

All students

leave

high school

college

and career

ready

Teachers and

schools have

information and

tools they need to

improve teaching

and learning

Summative: College and career

readiness assessments for

accountability

Interim: Flexible and open

assessments, used for actionable

feedback

Formative resources: Supporting

classroom-based assessments to

improve instruction

Summative Assessments for

Accountability

•Built on solid technology

•Coverage of full breadth/depth of Common Core

•Precise assessment of all students

Computer Adaptive Testing (CAT) Portion

•Deeper learning with thematic and scenario-based tasks

•Real-world problems aligned to Common Core

•PT scores combined with CAT for overall score

Performance Task (PT) Portion

•K-12, higher ed, business, and policymakers part of decision-making

•Broad review of “Achievement Level Descriptors”

•Field testing includes PISA, TIMSS, NAEP items

•Longitudinal data systems used for triangulation

Setting Performance Standards

(Cut Scores)

Page 10

✔

Interim Assessments

to Signal Improvement

•Non-Secure

•Timing and frequency are locally determined

• Interim test-builder creates aligned assessments

Flexible

•Teachers can match assessments with scope and sequence

•Teachers can review student responses

•Teachers can score student responses

Supports Proficiency Based Instruction

• Includes full range of item types

•Uses the same scale as the Summative Assessment

• Includes performance assessments Authentic Measures

Page 11

✔

Summative and Interim Use of

Computer Adaptive Technology

•Turnaround time is significantly reduced

•Can assess broad range with fewer items

Faster results, fewer items

Deeper Analysis

•Accurate measurement across range of students

• Improved measures of student growth over time Increased precision

• Item difficulty based on student responses Tailored to student

ability

•Large item pool means not all students receive the same questions

Greater security

•GMAT, GRE, COMPASS (ACT), Measures of Academic Progress (MAP)

Mature technology

Page 12

✔

✔

•Reports for classrooms and schools draw from the

full range of items seen by many students

Formative Tools for Classroom-

Based Assessment Practices

•Tools/materials for Classroom-based Assessments

•Fully aligned to Common Core State Standards

•Available for in-service and pre-service development

Improving

Instruction

•Access to the best resources available

•Collaborate with other states on special projects

•Professional social networking across the Consortium

•Tools to evaluate publishers’ tests

Pooling

Resources

Page 13

✔

A Balanced Assessment System

School Year Last 12 weeks of the year*

DIGITAL CLEARINGHOUSE OF FORMATIVE TOOLS, PROCESSES AND EXEMPLARS

Released items and tasks; Model curriculum units; Educator training; Professional development tools and

resources; Scorer training modules; Teacher collaboration tools; Evaluation of publishers’ assessments.

English Language Arts/Literacy and Mathematics, Grades 3-8 and High School

Computer Adaptive

Assessment and

Performance Tasks

Computer Adaptive

Assessment and

Performance Tasks

Scope, sequence, number and timing of interim assessments locally determined

*Time windows may be adjusted based on results from the research agenda and final implementation decisions.

PERFORMANCE

TASKS

• ELA/Literacy

• Mathematics

Re-take option

COMPUTER

ADAPTIVE TESTS

• ELA/Literacy

• Mathematics

Optional Interim

Assessment

Optional Interim

Assessment

Engaging Educators

K-12 Educator Involvement

•Support for implementation of the Common Core State Standards (2011-12)

•Write and review items/tasks for the for the pilot test (2012-13) and field test (2013-14)

•Development of educator leadership teams in each state (2012-14)

•Evaluate formative assessment practices and curriculum tools for inclusion in digital library (2013-14)

•Score portions of the interim and summative assessments (2014-15 and beyond)

Higher Education Collaboration

•Involved 175 public and 13 private systems/institutions of higher education in application

•Two higher education reps on the Executive Committee

•Higher education lead in each state and higher education faculty participating in work groups

•Goal: The high school assessment qualifies students for entry-level, credit-bearing coursework in college or university

Progress and Timeline

Assessment Claims for ELA / Literacy

“Students can demonstrate progress toward college and career readiness in English Language arts and literacy.”

“Students can demonstrate college and career readiness in English language arts and literacy.”

“Students can read closely and analytically to comprehend a range of increasingly complex literary and informational texts.”

“Students can produce effective and well-grounded writing for a range of purposes and audiences.”

“Students can employ effective speaking and listening skills for a range of purposes and audiences.”

“Students can engage in research and inquiry to investigate topics, and to analyze, integrate, and present information.”

Overall Claim (Gr. 3-8)

Overall Claim (High School)

Reading

Writing

Speaking and Listening

Research/Inquiry

Assessment Claims for Mathematics

“Students can demonstrate progress toward college and career readiness in mathematics.”

“Students can demonstrate college and career readiness in mathematics.”

“Students can explain and apply mathematical concepts and interpret and carry out mathematical procedures with precision and fluency.”

“Students can solve a range of complex well-posed problems in pure and applied mathematics, making productive use of knowledge and problem solving strategies.”

“Students can clearly and precisely construct viable arguments to support their own reasoning and to critique the reasoning of others.”

“Students can analyze complex, real-world scenarios and can construct and use mathematical models to interpret and solve problems.”

Overall Claim (Gr. 3-8)

Overall Claim (High School)

Concepts and Procedures

Problem Solving

Communicating Reasoning

Modeling and Data Analysis

21

Technology Guidelines

for New Purchases

Minimum

for New

Hardware

Processor

Speed RAM

Available

Memory/Storage Resolution

Display

Size

1.0 GHz 1 GB 1 GB 1024x768 10” Class

Desktops, laptops, netbooks (Windows, Mac, Chrome, Linux), thin client, and tablets (iPad, Windows, and Android) will be compatible devices provided they are configured to meet the established hardware, operating system, and networking specifications -- and are able to be “locked down”.

Operating

Systems

Windows 7

Mac 10.7

Linux (Ubuntu 11.10; Fedora 16)

Chrome

Apple iOS

Android 4.0

(v1.0 Apr. 2012)

Field Test

37,000

Items/Tasks

Major Milestones

All-Call for Pilot

Testing

Cognitive Labs &

Field Trials

IT Readiness (Round 1)

Interim &

Formative

Available for Use

Operational Summative

Assessment

Set Performance

Standards (Cut

Scores)

Verify

Performance

Standards

Pilot of 10,000

Items/Tasks

Item/Task

Writing

Begins

IT Readiness (Round 2)

All-Call for Field

Testing

State

Educator

Teams Begin

IT Readiness (Round 3)

Released Items & Tasks

Item Development Process

• Early 2012: Assessment claims for ELA/literacy and mathematics approved

• April 2012: Item/task specifications and review guidelines complete

• June 2012: Training modules available for item writers/reviewers

• Summer 2012: Educators from Governing States begin writing items and tasks; cognitive labs / small scale trials begin

• October 8, 2012: Sample items available

• February / March 2013: Pilot Test of first 10,000 items and performance tasks

Purpose of the Sample Items

• Illustrate rigor and complexity of ELA/literacy and mathematics items on Smarter Balanced assessments

• Signal to educators the shifts in instruction that will be required to help students meet the demands of the Common Core and new assessments

• Showcase variety of item types:

• Selected response

• Constructed response

• Technology enhanced

• Performance tasks

Exploring the Sample Items

• Accessed online through a simulated test platform • Viewable by:

– Grade band (3-5, 6-8, and high school) – Content claim – Select types (technology enhanced and performance tasks) – Themes to illustrate learning across grades and difficulty progressions

• Metadata for each item/task includes: – Grade level – Smarter Balanced content claim & assessment target – Alignment to Common Core State Standards – Brief descriptions – Selected scoring rubrics

• Selected response and technology enhanced items are machine scorable

• Online feedback and phone support available

Accessibility and Accommodations

• Sample items do not include accessibility and accommodations features

• Full range of accessibility tools and accommodations options under development guided by: – Magda Chia, Ph.D., Director of Support for Under-

Represented Students – Accessibility and Accommodations Work Group – Students with Disabilities Advisory Committee

• Chair: Martha Thurlow (NCEO) – English Language Learners Advisory Committee

• Co-Chairs: Jamal Abedi (UC Davis) & Kenji Hakuta (Stanford)

Visit us at: SmarterBalanced.org