Preparing for 2015 Smarter Balanced Summative … · Preparing for 2015 SBAC 2 . PreTest ... K–12 expectations for college and career ... by grade level, as the summative blueprints

Preparing for 2015 Smarter Balanced Summative Assessments

Rick Phelan

February 24, 2015

Sonoma County Office of Education www.scoe.org/sbac

Appreciations to Rocklin Unified School District and the California Department of Education for slide deck elements.

Goals Participants will:

o  Understand roles & responsibilities for administering the 2015 Smarter Balanced Summative Assessments

o  Obtain tools to support scheduling and logistical planning o  Understand student access options o  Gain materials to prepare test administrators/proctors o  Learn about communication resources for students & parents o  See an overview of the Test Operations Management System

2 Preparing for 2015 SBAC

Pre-‐‑Test

Preparing for 2015 SBAC 3

SBAC Test Design


Smarter Balanced Assessment System

Common Core State Standards

specify K–12

expectations for college and career readiness

All students leave

high school ready for college

and career

Teachers and schools have

information and tools they need to improve teaching

and learning Interim assessments

flexible, open, used for actionable feedback

Summative assessments benchmarked to college

and career readiness

Formative assessment tools and practices

for teachers to improve instruction

Preparing for 2015 SBAC

Smarter Balanced Computer Tests •  Test Administrators Directions for Administration

(DFA) are located at: http://caaspp.org/administration/instructions/index.html

•  Grades 3-8 and 11 in ELA and Math The only students exempt from participating in the test are:

•  Students participating in the Alternate Assessment •  English Language learners who have been attending schools in

the United States for less than 12 months at the time of the test

•  Computer-adaptive test (CAT) in each content area •  Performance Task (PT) in each content area

•  Classroom activity to familiarize students with question content •  Students respond to the PT through online platform


Smarter Balanced Item Response Types •  Multiple choice

–  one correct response –  multiple correct responses –  two-part

•  Matching tables –  yes/no –  true/false

•  Fill-in tables •  Select or order text or graphics •  Drag and drop •  Graphing •  Equation or numeric response •  Short text •  Long essay

SBAC Practice Tests

http://www.scoe.org/pub/htdocs/sbac-practice.html

8

•  Allow students and administrators to become familiar with and experience •  Grade-level assessments •  A variety of grade-specific items and difficulty levels •  Performance Tasks •  Format and structure of the operational

assessments •  Allow students to become familiar with accessing

the test (log-in process), testing software and navigational tools.


Interim Assessments

9

•  The Smarter Balanced Interim Assessments are comprised of interim comprehensive assessments (ICAs) and interim assessment blocks (IABs)

•  ICAs - Examples of the use include:

•  Beginning of the year a student from another state is given the previous year’s ICA.

•  Mid-year a teacher gives an ICA to gauge how students might perform on the summative assessment.

•  Scoring Protocol for Interim Assessments will be available at the end of February.


Interim Assessments (cont.)

10

•  The Smarter Balanced Interim Assessments are comprised of interim comprehensive assessments (ICAs) and interim assessment blocks (IABs)

•  IABs - Assess fewer sets of skills, and: •  Use the same targets, by grade level, as the

summative blueprints. •  Consists of short, focused sets of items. •  Provide information about a student’s strengths and

needs in relation to the assessment targets. •  Offer varied blocks by grade level and subject area.


SBAC Roles & Responsibilities


Roles and Responsibilities

12

District Test Coordinator (DC) •  Designate Site Coordinators (SCs), add them to the Test

Operations Management System (TOMS) •  Promote understanding of SBAC with stakeholder groups •  Coordinate with SC’s so that tests are administered in the

appropriate grade(s) and content area(s) •  Set appropriate supports and accommodations in TOMS •  Ensure that the SC’s and TA’s are appropriately trained

regarding test administration and security policies •  Report test security incidents to CDE •  Work closely with Computer Lab Tech/Site Tech Coordinators •  Relay issues/questions to the CalTAC call center


Roles and Responsibilities School Test or Site Coordinators (SC)

•  Collect security affidavits from Test Administrators and retain at test site •  Train TA’s and verify their accounts on TOMS

•  Promote understanding of SBAC with stakeholder groups

•  Verify supports and accommodations in TOMS minimum of three days before your testing window

•  Print student log-in strips/cards (confirmation code and SSID)

•  Confirmation code is the student’s first name from SIS

•  If you have a new students, notify your district CALPADS engineer

•  Ensure that testing at the school sites is conducted in accordance to test security policies

•  Create/approve test schedules and procedures for the school.

•  Monitor testing and address problems, as needed

•  Report testing security incidents to the DC


Roles and Responsibilities Test Administrator (TA)

•  Completely read and sign security affidavit •  Promote understanding of SBAC with stakeholder groups •  Complete test administration training •  Log in and verify access to the testing system using the CAASPP

account sent via email •  View student information prior to testing, to ensure it is correct •  Ensure that appropriate designated supports and

accommodations have been set prior to test •  Set up and proctor testing •  Report all potential test security to the SC and District CAASPP

Coordinator


Technical Support Staff •  Read and sign security affidavit •  Complete test administration training •  Ensure all student computers can launch and run

secure browser application •  Ensure bandwidth test is completed at each school

prior to testing windows •  Ensure all student computers have functional

headphones •  Assist the TA with set up and test proctoring •  Increase comfort levels and reduce anxiety of

students & staff

15

Roles and Responsibilities


SBAC Technology


Technology Setup

17

•  Insure devices are operational •  Secure browsers properly installed a minimum

2 weeks before testing •  Necessary hardware available and working

(headphones, external keyboards, etc.) •  Content filters, firewalls, and proxy servers

configured •  Stable (wired or wireless) internet connection


Secure Browser

18

The Secure Browser is: •  Specifically designed for CA to provide

secure access •  Prevents students from accessing other

hardware and software functions NOTE: Test Administrator computers will use a standard Web browser Preparing for 2015 SBAC

Secure Browser

19

•  Students must use the browser to log on to take a summative test

•  Prior to testing, TA’s should verify that:

•  External user applications are closed (browser will not work if they’re on)

•  Testing does not occur on computers with dual monitors



Technology Tools & Resources http://www.scoe.org/pub/htdocs/sbac-technical.html

SBAC Scheduling Issues


Smarter Balanced Computer Based Tests

Time Estimates


Smarter Balanced Test Schedule

• Overall testing will take approximately 7 to 8.5 hours • Recommended: Non-PT first, PT second (Performance

Task) • Classroom activity should be completed within 3 days

prior to starting PT • ELA PT’s have 2 parts and can be completed in 2 days • Math PT’s have 1 part and can be completed in 1 day • Recommended: PT should not be administered on the

same day as the non-PT


Smarter Balanced Test Schedule (continued)

• SC’s must ensure that TA’s know which students take which tests

• Schedule activities for students not taking exam or for those who finish early (eg. may read a hardcopy book

• A students CAT test remains active until the student completes and submits the test or 45 calendar days after the student has begun the CAT

• A student’s PT remains active for 10 calendar days after the student has begun.


2015 SBAC Scheduling Estimator


hFp://www.scoe.org/files/sbac-‐‑estimator-‐‑2015.xls

SBAC System Operations


Three Systems Connect to Deliver

Tests to Students

27

CALPADS •  Student enrollment information •  Student demographics

TOMS •  Manage test users •  Manage test windows •  Manage student supports •  Student rosters

Smarter Balanced •  Verify student test settings •  Select student tests •  Test delivery portal


CALPADS

28

•  Sole source for student demographics and program participation data for TOMS

•  Districts cannot update student demographics or program

participation data in TOMS; must be updated in CALPADS first

•  Contact your CALPADS administrator to correct student grade level or demographics

•  Data from CALPADS is rolled to TOMS daily and the turnaround time is about 48 hours.

•  If a new student enrolls during the District Testing window contact your CALPADS administrator to get them set up in TOMS Preparing for 2015 SBAC

Test Operations Management System (TOMS)

29

•  TOMS is a password-protected Web site that provides administrators with tools to manage users and students participating in Smarter Balanced assessments

•  Repository for student information, including initial test settings and

accommodations •  Different levels of user access (DC, SC, TA) •  District and school staff must set designated supports and

accommodations for students in TOMS prior to testing •  In TOMS, DCs and SCs can

•  Add and manage users •  View student roster and update student test settings


30

•  District and site coordinators authorize users for the system. •  Authorized users receive an email with login instructions and must

activate their login before the temporary password expires. •  Usernames and passwords used to access last year’s TIDES system

are no longer active. •  A manual on how to use TOMS is available:

http://goo.gl/GCSgBv

Accessing TOMS Step 1. hFp://californiatac.org/ Step 2. Log In Step 3. Navigate the System


Operating an SBAC Test Session STEP 1: Preparation •  Prepare computers

o  Student computers (No testing on computers with dual monitors ) •  A secure browser must be installed on all student computers/devices (updated

annually) o  Test Administrator computers

•  Use a standard Web browser. •  Prior to testing, TAs should verify on student computers that external user

applications are closed to allow the lockdown browser to work •  Secure browser available for download through the portal:

http://ca.browsers.airast.org/

•  Prepare student login information o  SSID, and confirmation code (CALPADS student first name) for login.

•  This may be provided on a card or piece of paper. NOTE: Student personal information is secure and must be collected and securely destroyed at the end of a test session.

•  Verify student test settings •  Ensure that the correct student test settings have been uploaded to TOMS (see the

later section on accommodations and designate supports)


Smarter Balanced- Additional Features Test Settings and Accommodations •  Before you approve students to test, you can

review student test settings. •  Only DC and SC may change certain settings •  If changes are made, select [Set] to confirm

the current test settings and return to the list of students awaiting approval. You will still have to approve the student for testing.

•  Select [Set & Approve] to establish the updated settings and approve the student for testing.

•  Select [Cancel] if you want to return to the Approvals and Student Test Settings screen.

•  Note: Translations (stacked) (for math items) and turning off universal tools only occurs in the test delivery system.


Operating an SBAC Test Session

STEP 2: TAs Log into the system and generate a test session o  Go to http://californiatac.org/ and select “Test Administrator Interface.” o  Enter your user name and password. o  Select the test(s) you will be administering during that session and click on Start

Session. o  Write the test Session ID on the board or large paper where students can see it.

Keep a record of this test session ID in the event that the session suddenly shuts down.

o  Do NOT navigate away from the TA interface page. The session will stop and all students will be logged out.

o  Read the testing script from the Test Administrator Manual o  Provide students with their SSID, and confirmation code (CALPADS student first

name) for login. o  Make sure students know that the Test Session ID must be entered exactly as it is

written, without extra spaces or characters.



o  Step 3: Students log into the system

•  To log in to the Smarter Balanced test, each student will need:

•  Confirmation Code: Student’s legal first name as spelled in CALPADS

•  State-SSID: State abbreviation (CA) followed by a hyphen and the student’s Statewide Student Identifier

•  Session ID

•  Students will select their test •  Students must wait for TA approval for entry

into the test session •  Students must then verify their test



Step 4: Approving Students for Testing

•  TAs can either select [Approve] for each individual student (recommended) or select [Approve All Students]. •  Reminder: If any student’s test settings are incorrect, do

NOT approve that student. •  To refresh the list of students awaiting approval at any time,

select the [Refresh] button at the top of the pop-up window. Do not use the browser refresh button.


Operating an SBAC Test Session Step 5: Students Begin their Test •  Students will be taken through a sound check

verification (two if text to speech is activated) •  Followed by a Test Instructions and Help screen


Operating an SBAC Test Session STEP 6. Monitoring Student Progress •  The table displays:

•  Students who have logged in and been approved for testing. •  Test completion Status •  Print requests •  Paused tests

•  Additional tests can be added to a session; you will receive a confirmation message box


Ending an SBAC Test Session

38

STEP 7: Ending the Test Students complete the test:

TA ends the test session: To stop the ENTIRE session (and pause tests for all students in the session):

–  Select the [Stop Session] buFon in the upper-‐‑left corner of the screen. An “Important!” box will appear, requesting verification to end the session and log students out.

–  Select [OK] to continue or [Cancel] to keep the test session open. Users should exit or log out of the Test Administrator Interface only after stopping the test session.


39

Pausing and Inactivity Rules •  Through the TA Interface, TAs may:

•  Stop an entire session; or •  Pause individual student tests.

•  Users should exit or log out of the Test Administrator Interface only after stopping the test session.

•  If there is a technical issue (i.e. power outage or network failure), students will be logged out and the test will automatically be paused.

•  Regardless of when or how users log out or navigate away from the Test

Administrator Interface, student data will NOT be lost.

•  Note: If a test is paused for any reason or length of time, the student must log in again to resume testing. Highlighted text and item notes will not be preserved on the computer adaptive tests.

•  There is no pause option for the PT (performance task). Note for ELA (2

parts) a student cannot go pack to part 1 after completing part 2.

Do NOT stop tests without forewarning students


SBAC Pausing and Inactivity Rules

•  As a security measure, students are automatically logged out of the test after 20 minutes of inactivity. When this occurs, students are:

•  Presented with the test page containing the test item he or she was last working on (if page contains at least 1 unanswered item) OR

•  Presented with the next test page (if all items on the previous test page were answered); •  NOT permitted to review or change any test items on previous test pages.

•  Activity means: •  Selecting an answer •  Using a navigation option in the test (i.e. selecting [Next] or [Back], using the Past/Marked

Questions drop-down list) •  NOTE: Moving the mouse or selecting an empty space on the screen is NOT considered

activity.

•  Before the system logs out, a warning message will be displayed. •  If a student does NOT select [Ok] within 30 seconds, the student will

be logged out. •  Selecting [Ok] will restart the 20 minute inactivity timer.


General Rules & Conditions for

SBAC Student Responses •  Students must answer all test items on a page before moving

on to the next page •  Students may review already answered questions before

submitting test •  Students may not return to a test segment once it has been

completed and submitted •  Students must answer all test items before the test can be

submitted •  Students may not return to a test once it has been completed

and submitted •  There is a difference between a “segment” and a “session” •  Expiration and pause rules vary between PT and non-PT


Administration of the Classroom Activity and

Performance Task

Classroom Activity Administration

Guidelines •  Classroom Activity Purpose: To provide students

with important context, concepts, and key terms prior to taking the PT.

•  For each content area, all students will have a classroom activity before they complete the PT.

•  May be administered in a classroom or any other appropriate space.

•  May take place on a separate day from the PT. •  Recommended: No more than a 3-day lapse between

the classroom activity and the PT administration. •  TA may need a chalkboard or dry-erase board.



Guidelines (cont.) •  Computers, projectors, and other technology

are allowed but not required for the classroom activity.

•  Involves the participation of all students in an instructional task.

•  PT will be completed by individual students in the computer-based test delivery system.

•  Students may take notes during the classroom activity, but the notes must be collected before proceeding to the PT.



Guidelines (cont.) •  Designed to fit into a 30-minute window.

•  Will vary due to complexity of topic and individual student needs.

•  The SC should download the assigned classroom activity and provide to TAs at least 1–2 days prior.

•  Download from http://californiatac.org/ •  After receiving the activity from the SC, the TA or

classroom teacher should administer the classroom activity.

•  Provide the appropriate accommodations for students who normally use them during instruction.


Classroom Activity and Performance

Task Administration Ø  Step 1: Identify the classroom activities for your school.

Ø  Step 2: Download the classroom activity.

Ø  Step 3: Review the teacher directions for the classroom activity carefully.

Ø  Step 4: Complete the classroom activity. Ø  Step 5: Administer the PT to students online. The students must

work independently


Universal Supports, Designated Supports, &

Documented Accommodations

Accessibility and Accommodations

Three categories of support… •  Universal Tools:

o  Available to all; based on student preference and selection (e.g., breaks, strikethrough, highlighter).

•  Designated Supports: o  Need to be indicated by an educator or group of educators (e.g., color contrast,

read aloud, glossary, magnification).

•  Accommodation: o  Based on individualized education programs (IEPs) or Section 504 plans (e.g.,

American Sign Language, scribe).


Accessibility and Accommodations Two systems of delivery Ø  Embedded supports: digitally delivered through the

technology platform(i.e., digital notepad, calculator) MUST be activated in TOMS prior to testing.

Ø  Non-embedded supports: provided by the District(i.e.,

adapted keyboard, multiplication table) but are still noted in TOMS


Accessibility and Accommodations


Accessibility and Accommodations •  Questions:

•  How do I know what supports are available? •  How do I match the IEP and 504 supports to the supports listed?

•  Answers and Resources: •  Matrix One: Universal tools, designated supports, and accommodations for the

CAASPP system http://www.cde.ca.gov/ta/tg/ai/documents/caasppmatrix1.pdf •  Shows tables of all available items by exam and level of support

•  Smarter Balanced Usability, Accessibility, and Accommodation Guidelines •  Provides details about each of the supports and contexts in which they are

appropriate •  Describes who is eligible for them and guides the decision making process •  Detailed Q & A section in an appendix

•  ISAAP Tool & SEIS •  Helps staff select and organize appropriate support categories •  Summarizes students’ supports that can be uploaded to TOMS


Accessibility and Accommodations •  Not all tools, supports, and accommodations are available for all

exams or to all grade levels. For example: o  ELA – a dictionary is not allowed on short paragraph responses but are allowed on the PT

long essay. o  ELA - read-aloud of passages or text-to-speech is not allowed on passages for students

in grades 3-5. o  Math – multiplication tables are not allowed in grade 3.

•  When an IEP or 504 specifies that a “disallowed” tool is required for a student to participate in testing, special permission must be granted to use these “Individualized Aids.” o  The District CAASPP Coordinator or the CAASPP Test Site Coordinator must submit the

CAASPP Individualized Aid Request Form. Approval to use these is granted by the California Department of Education.

o  When an Individualized Aid changes the construct being measured, the scores for that student will not count towards the participation rates. This is similar to the “modifications” policy under the prior STAR program – the student gets a score report, but the score is not used for accountability.


Accessibility and Accommodations Planning Ahead

•  Determination of which accommodations an individual student will have available for the assessment is necessary because these accommodations must be made available before the assessment, either by entering information into the TOMS for embedded accommodations, or by ensuring that the materials or setting are available for the assessment for non-embedded accommodations.

•  Accommodations could increase cognitive load or create other challenges for students who do not need them or who have not had experience using them. It is wise for students to practice the tests using the selected tools.

•  Parents/guardians should be made aware of the conditions under which their child participates in the assessment.

•  Once the appropriate supports are identified, site and district coordinators will work together to ensure that they are uploaded to TOMS.

•  TA’s are not authorized to change student test settings in TOMS •  Prior to testing review the students test settings and verify that they are

correct.


Accessibility and Accommodations Online Resources •  Smarter Balanced Usability, Accessibility, and Accommodations Guidelines:

http://www.smarterbalanced.org/wordpress/wp-content/uploads/2014/08/SmarterBalanced_Guidelines.pdf

•  Smarter Balanced Usability, Accessibility, and Accommodations Implementation

Guide: http://www.smarterbalanced.org/wordpress/wp-content/uploads/2014/03/Usability-Accessibility-and-Accommodations-Implementation-Guide.pdf

•  Guidelines: Frequently Asked Questions:

http://www.smarterbalanced.org/wordpress/wp-content/uploads/2013/12/SmarterBalanced_Guidelines_FAQ.pdf

•  Accessibility and Accommodations for CAASPP 2014–15: An Overview archived

Webcast http://caaspp.org/rsc/videos/archived-webcast_102214.html •  Matrix One: http://www.cde.ca.gov/ta/tg/ai/documents/caasppmatrix1.pdf •  Using the CAASPP Individual Student Assessment Accessibility Profile (ISAAP)

Tool video tutorial: http://caaspp.org/rsc/videos/archived-webcast_ISAAP-overview-and-demo.html

•  Embedded Universal Tools and Online Features training module: http://www.brainshark.com/ets/vu?pi=zI9zwnui5zJ0cZz0&intk=37891690


Test Security


56

Test Security There are 3 different types of test security incidents •  Impropriety (Correctable and containable at local level)

o  Unusual circumstance, low risk of affecting student performance, test security, or test validity

o  Must be reported to District CAASPP Coordinator or Test Site Coordinator o  Example: Students talking during testing

•  Irregularity (Correctable and containable at local level) o  Unusual circumstance, may affect student performance, test security, or test validity o  Must be reported to District CAASPP Coordinator within 24 hours o  Examples: Student cheating or providing answers to another; student accessing or using

electronic equipment (e.g., cellphone)

•  Breach (External implications for the Consortium) o  Event that threatens test validity (e.g., release of secure materials) o  A breach incident must be reported to District CAASPP Coordinator within 24 hours o  Examples:

•  Administrator or coordinator modifying student responses or record at any time

•  Administrator allowing students to take home test items, passage, prompts, or scratch paper


Resources

•  Sonoma County Office of Education SBAC Support Pages http://www.scoe.org/sbac

•  Manuals http://goo.gl/9MQhfl o  Administration Manuals o  Universal Tools, Designated Supports, and Accommodations o  TOMS Users Guide o  Administration of Paper and Pencil Tests

•  Smarter Balanced Test login site http://californiatac.org/


Documents

Preparing for 2015 Smarter Balanced Summative … · Preparing for 2015 SBAC 2 . PreTest ... K–12 expectations for college and career ... by grade level, as the summative blueprints