California Assessment of Student Performance and Progress (CAASPP) CAASPP Test Site Coordinator...
If you can't read please download the document
California Assessment of Student Performance and Progress (CAASPP) CAASPP Test Site Coordinator Training SBAC Online Test Administration February 25, 26
California Assessment of Student Performance and Progress
(CAASPP) CAASPP Test Site Coordinator Training SBAC Online Test
Administration February 25, 26 2015 March 4, 5 2015 Presented by
Stephanie Hearne, Assessment Supervisor
Slide 2
Overview- Student Participation Roles and Responsibilities
Technology Test Administration Using the TOMS System Student
Accessibility Supports Creating and Administering an Online Test
Session Test Security- Improprieties, Irregularities, Breaches
Resources and Site Support Questions Agenda 2
Slide 3
By the end of this training, attendees will understand: The
administration of the SBAC summative assessments using appropriate
procedures The navigation and uses for Test Operations Management
System (TOMS) Objective 3
Slide 4
Overview 4
Slide 5
Overview Student Participation Q: Who will take the online SBAC
summative assessments? A: All students in grades 3-8 and 11 are
required to take the summative assessments. The only students
exempt from participating in the SBAC summative assessments are:
Students participating in the California Alternate Assessments
(CAA) English learners (ELs) who have been attending school in the
United States for less than 12 months (for the ELA test only)
Students will be required to take the math assessment 5
Slide 6
Overview Student Participation: Early Assessment Program The
Early Assessment Program (EAP) will be using the grade 11 SBAC
summative assessments in ELA and mathematics in place of the grade
11 CSTs in ELA and/or mathematics, additional 15 questions and the
ELA essay Grade 11 students taking the SBAC summative assessments
may choose to release their results to the CSU and participating
CCCs at the end of the computer adaptive test Students who do not
authorize the release of their test results at the time of testing
may need to provide a copy of the score report upon request of the
campus Pgs. 32-33 TAM 6
Slide 7
Content areas; English language arts/literacy (ELA) Mathematics
Two components per content area: Computer adaptive test (CAT)
Performance task (PT) Overview About the Summative Assessments Test
Item Types: Page 59 in the TAM 7
Slide 8
Q: What is the CAT component of the online summative
assessment? A: The computer adaptive portion (CAT) of the online
test includes a wide variety of item types. As students progress
throughout a CAT, the computer program adjusts the difficulty of
questions throughout the assessment based on their responses. For
example, a student who answers a question correctly will receive a
more challenging item, while an incorrect answer generates an
easier question. Overview About the Summative Assessments (Cont.)
8
Slide 9
Q: What is the PT component of the online summative assessment?
A: The PT is designed to provide students with an opportunity to
demonstrate their ability to apply their knowledge and high-order
thinking skills to explore and analyze a complex, real world
scenario. Prior to the PT, teachers or instructional staff conduct
a Classroom Activity for all students to ensure that a lack of
understanding of the context of the task does not interfere with a
students ability to address the content of the task. 9
Slide 10
Overview Systems and Processes for Online Administration Test
Administrator Interface & Student Testing Site Administer
online Smarter Balanced assessments Test Administrator Interface
& Student Testing Site Administer online Smarter Balanced
assessments TOMS Verify student demographic data Configure online
test settings (designated supports and accommodations) TOMS Verify
student demographic data Configure online test settings (designated
supports and accommodations) CALPADS (California Longitudinal Pupil
Achievement Data System) Manage student demographic data CALPADS
(California Longitudinal Pupil Achievement Data System) Manage
student demographic data 123 10
Slide 11
11 Roles and Responsibilities LEA CAASPP Coordinator Test Site
Coordinator Test Administrator Technology Coordinator CALPADS
Administrator
Slide 12
Roles and Responsibilities LEA CAASPP Coordinator (DC) Add SCs
and/or TAs in the TOMS Ensure that the SCs are appropriately
trained regarding the state and SBAC assessment administration as
well as security policies and procedures Approve testing schedules
and security plans for the schools Report all test security
irregularities and/or breaches that occur before, during, and after
test administration to the CDE Oversee all CAASPP administration
activities in schoo ls 12
Slide 13
Identify TAs, ensure that they have submitted signed CAASPP
Test Security Affidavits Add TAs into TOMS Ensure TAs are properly
trained so they can administer the assessments Enter and/or verify
students test settings in TOMS Ensure that testing in the school is
conducted in accordance to test security policies Create/approve
test schedules and security procedures for the school Work with
technology staff to ensure that necessary secure browsers are
installed and any other technical issues are resolved Monitor
testing progress and ensure that all eligible students participate
Report test security incidents to the DC Relay issues/questions to
the DC Oversee CAASPP administration activities in the school Roles
and Responsibilities School Coordinator (SC) 13
Slide 14
Read and sign the CAASPP Test Security Affidavit Carefully and
completely read and sign security affidavit Complete training and
review all SBAC policy and administration documents prior to
administering any SBAC assessments View student information prior
to testing to ensure it is correct Ensure that appropriate
designated supports and accommodations have been set prior to
testing Administer the SBAC assessments Report all potential test
security incidents to the SC Relay issues/questions to the SC
Encourage students to do their best Roles and Responsibilities Test
Administrator (TA) 14
Slide 15
Roles and Responsibilities Technology Coordinator Assist
District in setting up and overseeing technology infrastructure
Support each school site Work closely with the LEA CAASPP
Coordinator to ensure network and testing devices are properly
prepared by installing the most current secure browser Help
troubleshoot during time of testing 15
Slide 16
Roles and Responsibilities CALPADS Administrator Ensures
student demographic, enrollment, and program participation data is
correct in the CALPADS Work closely with the District CAASPP
Coordinator 16
Slide 17
Technology Infrastructure 17
Slide 18
Prior to the test administration, District CAASPP Coordinator
with Technology Coordinator should review the technology
infrastructure Testing devices are working Ensure that secure
browsers are properly installed a minimum of two weeks before
testing Ensure the necessary hardware is available and working
(i.e., headphones, external keyboards, printers) Content filters,
firewalls, and proxy servers configured Stable (wired or wireless)
internet connection Technology Infrastructure 18
Slide 19
Students must use the secure browser to log in to the Student
Interface and access the summative test Prevents students from
accessing other hardware and software functions Prior to testing,
TAs should verify that: External user applications are closed,
secure browser will not work if the computer detects that a
forbidden application is running Volume must be tested before
launching browser Test Administrators (TA) will use a standard
browser No testing on computers with dual monitors Students must
not take the test on computers connected to more than one monitor
Secure Browser Pgs. 74-75 TAM 19
Slide 20
Hardware Requirements Headphones are required for each test
station English language arts/literacy (ELA) tests Text-to-speech
accommodation Job Access with Speech (JAWS) Microphones are not
required 20
Slide 21
21 Test Administration
Slide 22
There are multiple ways to prepare students and staff for the
SBAC summative assessments: Practice Test Training Test Purpose of
the Practice Tests Allows students and administrators to become
familiar with the format and structure of the operational
assessments Scoring rubrics and answers are available on the TA
Resource Web page at http://caaspp.org/ta-
resources/index.htmlhttp://caaspp.org/ta- resources/index.html All
universal tools, designated supports, and accommodations are
available Preparing Students and Staff Practice and Training Test
22
Slide 23
Preparing Students and Staff Practice and Training Test (Cont.)
23 There are two ways for accessing the Practice and Training Test:
Through caaspp.org on a Web browser Through the secure browser
Simulates operational online testing environment TAs may create
test sessions for the Practice or Training Tests on the TA Practice
and Training Site
Slide 24
Test Administration Scheduling Testing Time Content AreaGrades
Computer Adaptive Test (CAT) items hrs : mins Performance Task (PT)
hrs : mins Total hrs : mins Classroom Activity (administered prior
to the PT)* hrs : mins Total hrs : mins English Language
Arts/Literacy 351:302:003:30:304:00 681:302:003:30:304:00 112:00
4:00:304:30 Mathematics 351:301:002:30:303:00 682:001:003:00:303:30
112:001:303:30:304:00 Both 353:00 6:001:007:00
683:303:006:301:007:30 114:003:307:301:008:30 Pgs. 27-31 TAM
24
Slide 25
Test Administration Scheduling Testing Time (Cont.) The
estimates do not account for any time needed to start computers,
and log in students Recommended: CAT first, PTs second The
classroom activity should be completed within 3 days prior to
administering a PT ELA PTs have 2 parts and can be completed over 2
days Mathematic PTs have 1 part and can completed in 1 day PTs
should not be administered on the same day as the classroom
activity or the CAT CAT Classroom Activity Performance Task 25
Slide 26
Test Administrator Training Ensure that TAs: Sign Security
Affidavits Review the TA Reference Guide Review Online Test
Administration Manual Section 2.1 and 10.0 SBAC Usability,
Accessibility, and Accommodations Guidelines Review Training
Modules What is a CAT What are Universal Tools TAs should know
testing procedures: they have school site schedules they are
familiar with security procedures, and conduct associated with the
administration of standardize assessments 26
Slide 27
Test Administration Establishing Appropriate Testing Conditions
Testing students in classroom-sized groups is preferable
Facilitates monitoring and control for TAs Remove or cover
displayed information that might be used by students to help answer
questions Rubrics, vocabulary charts, student work, graphs, charts,
etc. Establish procedures to maintain a quiet testing environment
Remember some students will finish more quickly than others Cell
phones or other electronic devices are not allowed to be used
during testing Post an Unauthorized Electronic Devices sign so that
it is clearly visible to students and place a testing Do Not
Disturb sign on doors to testing rooms 27
Slide 28
Test Administration General Testing Information Students must
answer all test items on a page before moving on to the next page
Students may not return to a test segment once it has been
completed and submitted Students must answer all test items before
the test can be submitted TAs must follow the SAY script exactly
each time a test session is administered Refer to the SAY boxes of
the Online Test Administration Manual or Online Directions for
Administration 28
Slide 29
Using the Test Operations Management System (TOMS) Task in Toms
Add and manage TAs Gives them access to TA interface Review student
data Demographic Data from CALPADS Test enrollment Assign online
test setting to students Designated Supports and Accommodations
TOMS can be accessed from the CAASPP Portal at http://caaspp.org
http://caaspp.org 29
Slide 30
TOMS and CALPADS CALPADS is the sole source for student
demographic and program participation data for TOMS Student
accommodations and supports will not be uploaded in CALPADS LEAs
cannot update student demographic or program participation data in
TOMS; must be updated in CALPADS Processing time from CALPADS to
TOMS is 48 hours SCs must set designated supports and
accommodations for the students in TOMS 30
Slide 31
Reviewing Student Data Each student profile in TOMS includes:
Demographic data Read-only; must be modified in CALPADS Test
enrollment All students in grades 3-8 and 11 are automatically
enrolled in the online SBAC summative assessments Online test
settings Universal tools, designated supports, and accommodation
31
Slide 32
Reviewing Student Data (Cont.) Students grade levels must be
correct in TOMS before testing Grade levels are only correctable in
CALPADS All students who are going to test must appear in TOMS If a
students designated supports and accommodations information is
incorrect, it must be updated in TOMS before he or she tests
32
Slide 33
Student Profile in TOMS: Demographics All demographic data is
read- only 33
Slide 34
Student Profile in TOMS: Test Mode All students in grades 38
and 11 are automatically enrolled in the online Smarter Balanced
summative tests 34
Slide 35
Student Profile in TOMS: Test Settings 35
Slide 36
Assigning Accessibility Supports in TOMS Student online test
settings must be configured at least 24 hours prior to testing
Allow up to 24 hours from when you configure test settings in TOMS
to when the changes appear in the TA and Student Testing Interface
LEA and School Coordinators are authorized to set accessibility
supports for students in TOMS. TAs do not have the ability 1.
One-by-one through student profile 2. Special Ed will initially
upload students settings from SEIS 3. Coordinators will verify that
information is correct 36
Slide 37
What are Accessibility Supports ? Universal Tools: Available to
all; based on student preference and selection (i.e.,
strikethrough, highlighter) Designated Supports: Need to be
indicated by an educator or guardian (i.e., color contrast, read
aloud, glossary, magnification) Accommodations: Based on
individualized education programs (IEPs) or Section 504 plans
(i.e., American Sign Language, scribe) 37
Slide 38
What are Accessibility Supports? (Cont.) The online assessments
include embedded and non-embedded accessibility supports Embedded:
part of the computer administration (i.e., digital notepad,
calculator) Non-embedded: provide outside the computer
administration (i.e., scratch paper, multiplication table) 38
Slide 39
What are Accessibility Supports? (Cont.) EmbeddedNon-Embedded
Designated SupportAccommodationDesignated SupportAccommodation
Color Contrast American Sign Language (ELA listening and math
items) Bilingual Dictionary (ELA performance task full writes)
Abacus MaskingBrailleColor Contrast Alternate Response Options
Text-to-Speech (math and ELA items) Closed Captioning (ELA
listening items) Color OverlaysCalculator Translated Test
Directions (math items) Streamlined
InterfaceMagnificationMultiplication Table Translations
(glossaries) (math items) Text-to-speech (ELA reading passages)
Noise BuffersPrint on Demand** Translations (stacked) (for math
items)* Read Aloud (math and ELA items) Read Aloud (ELA reading
passages) Turn off any universal tools* Scribe (ELA non-writing and
math items) Scribe Permissive ModeSeparate SettingSpeech-to-Text
Print SizeTranslated Test DirectionsIndividualized Aids Translation
(glossaries) (math items) Special Lighting or Acoustics Translated
Test Directions (math items) *Must be activated in the test
delivery system. **Print on demand for stimuli only can be
activated in TOMS. If a student(s) need print on demand for items
and/or items and stimuli, contact CalTAC at least 24 hours prior to
testing. 39
Slide 40
Administering a Test Session The TA logs on to the TA Interface
from http://caaspp.org on a standard Web browser http://caaspp.org
The TA logs on using his/her username and password 40 Pg. 41
TAM
Slide 41
Administering a Test Session (Cont.) The TA selects the test
category or the grade and content area 41
Slide 42
Administering a Test Session (Cont.) The TA selects the tests
to administer Additional test can be added to a session; you will
receive a confirmation message box The TA starts a test session
42
Slide 43
Administering a Test Session (Cont.) The TA informs students of
the test Session ID Write or display clearly in a place all
students can see Session IDs are automatically generated upon
selecting the [Start Session] button Additional test can be added
to a session. You will receive a confirmation message box 43
Slide 44
Distributing Student Logon Information TAs need to provide
students with the Session ID, the students Statewide Student
Identifier (SSID), and confirmation code (student first name in
CALPADS) for logon May be provided on a card or piece of paper TAs
may write the Session ID on a board where students can see it Note:
if the session ID is not written down or known and the TA tries to
log on to the session after getting involuntarily logged off of the
system, the TA will be logged off for eight hours before he or she
is able to start a new session 44
Slide 45
Administering a Test Session (Cont.) Students log on through
the secure browser To log on, students need: First Name
(Confirmation Code): Students legal first name as spelled in
CALPADS State-SSID CA followed by a hyphen - and the students SSID
Session ID: TA-generated session ID 45
Slide 46
Administering a Test Session (Cont.) Students verify their
identity After logging on, students will see the Is This You screen
Students should select [Yes] to continue 46
Slide 47
Administering a Test Session (Cont.) Students select the test
All grade-level tests that the student is eligible to take are
displayed Only the tests that the TA selected for the test session
and those that have not been completed are selectable by students
47
Slide 48
Administering a Test Session (Cont.) Upon selecting a test,
students will wait for TA approval As students select tests, the TA
will see approvals populate in the Approvals queue of the TA
Interface 48
Slide 49
Administering a Test Session (Cont.) The TA selects the
[Approvals (#)] button to access the Approvals and Student Test
Settings screen 49
Slide 50
Administering a Test Session (Cont.) The TA verifies the
student has selected the appropriate test and reviews the students
test settings by selecting [See/Edit Details] button next to each
student Only need to check for students in session that are using
special accommodations, etc. 50
Slide 51
Administering a Test Session (Cont.) The TA selects [Set] to
confirm the current test settings and return to the list of
students awaiting approval The TA will still have to approve the
student for testing OR The TA selects [Set & Approve] to
establish the existing settings and approve the student for testing
Note: If a students test settings are incorrect, TAs should deny
the student, contact the School CAASPP Coordinator to correct the
test settings in TOMS, and test the student on another day 51
Slide 52
Administering a Test Session (Cont.) Upon reviewing test
settings for all students, the TA approves students to test on the
Approvals and Student Test Setting screen Select [Approve] to
approve a specific student for testing Select [Approve All
Students] to approve all students 52
Slide 53
Administering a Test Session (Cont.) After the TA has approved
a student for testing, the student will verify the test information
and settings The student will select [Yes, Start My test] to begin
testing or [No] if their settings are incorrect 53
Slide 54
Administering a Test Session (Cont.) If a student has a
text-to-speech accommodation or is taking a test with ELA listening
items, he or she will verify the audio checks through headphones
Text-to-speech audio check Test with listening items audio check
54
Slide 55
Administering a Test Session (Cont.) Finally, students will see
the Test Instructions and Help page before they begin testing
Students will select [Begin Test Now] to begin testing 55
Slide 56
Monitoring Student Progress TAs can monitor student testing
through the Students in Your Test Session table on the TA Interface
Displays students who have logged in and been approved for testing
56
Slide 57
Pausing Test TAs may pause an individual students test in the
Students in Your Test Session table This will not affect other
students tests Pgs. 24-25 TAM 57
Slide 58
General Pause Rules If there is a technical issue (i.e., power
outage, network failure), students will be logged out and the test
will be automatically paused. Student work will not be lost it will
be saved. Students log back on with Session ID that was written on
board to pick up where they left off If a test is paused, students
will have to log back in to resume testing Note: Highlighted text
and global notes will not be preserved 58
Slide 59
Pause Rules: Computer Adaptive Test (CAT) If a CAT is paused
for more than 20 minutes, the student is: Presented with the test
page containing the test item he or she was working on (if the page
contains at least 1 unanswered item) OR Presented with next test
page (if all items on the previous test page was answered) Not
permitted to review or change any test items on previous test pages
If a Cat is paused for less than 20 minutes, the student is:
Presented with the test item or passage he or she was working on
when the test was paused Permitted to answer previously shown items
within a segment 59
Slide 60
Pause Rules: Performance Task There are no pause rules for PTs
Even if a PT is paused for more than 20 minutes, the student can
return to the current section and continue ELA PTs are divided into
two parts After a student completes the first part, he or she
cannot return to it A message is given to students letting them
know they cannot return 60
Slide 61
Timeout Due to Inactivity As a security measure, students are
automatically logged out of the test after 30 minutes of inactivity
Activity means Selecting an answer Using a navigation option in the
test (i.e. selecting [Next] or [Back], using the questions
drop-down list) NOTE: Moving the mouse or selecting an empty space
on the screen is NOT considered activity 61
Slide 62
Stopping an Entire Test Session TAs can stop a test session (
and therefore pause tests for all students in a session Select the
[Stop Session] on the upper-left corner of the TA Interface An
Important! box will appear, requesting verification to end the
session and log students out Select [OK] to continue or [Cancel] to
keep the test session open 62
Slide 63
Reaching the End of the Test After students answer the last
item of the test, the [End] test button will appear in the
upper-left corner of the screen 63
Slide 64
Reaching the End of the Test Upon selecting the [End Test]
button, an Attention message will appear Students will select [Yes]
to proceed with reviewing answers and submitting the test 64
Slide 65
Students may review answers before submitting a test Reaching
the End of the Test 65
Slide 66
Reaching the End of the Test When a student is ready to submit
the test, he or she will receive a confirmation screen 66
Slide 67
Test Expiration Rules A students CAT test remains active until
the student completes and submits the test or 45 calendar days
after the students have begun the CAT A PT test remains active for
10 calendar days after a student has begun it Important: Student
personal information is secure and must be collected and securely
stored at the end of a test session, then securely destroyed at the
end of the test administration. 67
Slide 68
Classroom Activity Administration Guidelines Purpose: to
provide students with important context, concepts, and key terms
prior to taking the PT All students will have a classroom activity
before they complete a PT A class activity may be administered in a
classroom or any other appropriate space Class activity will take
place on a separate day from the PT No more than a 3-day lapse
between the class activity and the PT administration Pgs. 28-29 TAM
68
Slide 69
Classroom Activity Administration Guidelines (Cont.) Computers,
projectors, and other technology are allowed but not required for
the class activity Involves the participation of all students in an
instructional tasks PT will be completed by individual students on
the testing device Students may take notes during the class
activity, but notes must be collected prior to taking the PT
69
Slide 70
Classroom Activity Administration Guidelines (Cont.) Class
activity is designed to fit into a 30-minute window It will vary
due to complexity of topic and individual student needs The SC
should download the assigned classroom activity and provide to the
TAs at least-1-2 days prior to test After receiving the activity
from the SC, the TA or classroom teacher should administer the
classroom activity 70
Slide 71
Test Security There are three different types of test security
incidents: 1. Impropriety 2. Irregularity 3. Breach Test
Impropriety Unusual circumstance that has a low impact on the
testing individual or group of students Low risk of affecting
student performance, test security, or test validity Correctable
and containable at the district level Must be reported to the
District Coordinator using the Test Administration Testing Incident
or Impropriety Report Form May be reported in the Appeals module
should an Appeal be required Example: Students talking Pg. 64 TAM
71
Slide 72
Test Security (Cont.) Test Irregularity Unusual circumstance
that impacts the testing individual or group of students May affect
student performance, test security, or test validity Correctable
and containable at the district level Must be reported to the
District Coordinator using the Test Administration Testing Incident
or Impropriety Report Form, who must them report it to the CDE
within 24 hours Submitted in the Appeals module for resolution Read
pgs. 20-21 for appeals information Examples: student cheating or
providing answers to another, student accessing or using electronic
equipment (i.e., cellphones) 72
Slide 73
Test Security (Cont.) Test Breach Event that threatens test
validity (i.e., release of secure materials) A breach incident must
be reported to the CDE within 24 hours Examples: Administrator or
coordinator modifying student responses or record at any time
Administrator allowing students to take home test items, passage,
prompts, or scratch paper Adult or student posting items or test
materials on social media 73
Slide 74
Resources CAASPP TA Training Modules
http://caaspp.org/training/caaspp/
http://caaspp.org/training/caaspp/ TA Directions for Administration
http://caaspp.org/rsc/pdfs/CAASPP.ta-odfa.2015.pdf
http://caaspp.org/rsc/pdfs/CAASPP.ta-odfa.2015.pdf Accessibility
Supports Resources SBAC Usability, Accessibility and Accommodations
Guidelines http://www.smarterbalanced.org/wordpress/wp-
content/uploads/2014/08/SmarterBalanced_Guidelines.pdf
http://www.smarterbalanced.org/wordpress/wp-
content/uploads/2014/08/SmarterBalanced_Guidelines.pdf Guidelines
Frequently Asked Questions
http://www.smarterbalanced.org/wordpress/wp-
content/uploads/2013/12/SmarterBalanced_Guidelines_FAQ.pdf
http://www.smarterbalanced.org/wordpress/wp-
content/uploads/2013/12/SmarterBalanced_Guidelines_FAQ.pdf TOMS
online Student Test Settings User Guide
http://caaspp.org/rsc/pdfs/CAASPP.student-test-settings.2015.pdf
http://caaspp.org/rsc/pdfs/CAASPP.student-test-settings.2015.pdf
74
Slide 75
Resources (Cont.) Accessibility Supports Resources (Cont.)
Matrix One
http://www.cde.ca.gov/ta/tg/ai/documents/caasppmatrix1.pdf
http://www.cde.ca.gov/ta/tg/ai/documents/caasppmatrix1.pdf Embedded
Universal Tools and Online Features Training Module
http://www.brainshark.com/ets/vu?pi=zI9zwnui5zJ0cZz0&int
k=37891690
http://www.brainshark.com/ets/vu?pi=zI9zwnui5zJ0cZz0&int
k=37891690 75
Slide 76
Site Support Assessment: 307-4515
[email protected]@wccusd.net Questions, Policy and
Procedures, Passwords, Test Security Issues Tests not loading TOMS
and CAASPP website issues Student login to secure browser
Technology/ IT Helpdesk: [email protected]@wccusd.net
Broken keyboards Student logins (to computer or tablet not secure
browser) Forbidden applications running in background Do not call
or email IT helpdesk for questions regarding TOMS or CAASPP Website
issues CALPADS: [email protected]@wccusd.net Enrollment
Special Education: [email protected] or Program Specialist for
your [email protected] Accessibility Supports IEP questions
76