2013 CRT &ALT2013 CRT &ALTTest Administrator TrainingTest Administrator Training

Presenter:Date:

OBJECTIVE:OBJECTIVE:

TAs will be able to maintain the integrity of the MontCAS to ensure accurate, reliable, and valid results.

AGENDAAGENDAOnline test materials, resources, and

trainingsTest SecurityTest Administration Accommodations

WHO CAN BE A TEST ADMINISTRATOR WHO CAN BE A TEST ADMINISTRATOR (TA)?(TA)?

Licensed professionalsMay not adminster but can assist

under direct supervision of a TA.◦Substitute teachers◦Student teachers◦Paraprofessionals

All TAs must be trained

TEST ADMINISTRATOR TEST ADMINISTRATOR RESPONSIBILITIESRESPONSIBILITIESAttend training Read test administration manualEnsure test securityReport test irregularities or security

breachesPrepare testing environmentCorrectly administer the assessmentUnderstand and provide accommodations

correctly

ONLINE TEST MATERIALS, ONLINE TEST MATERIALS, RESOURCES, AND TRAININGSRESOURCES, AND TRAININGS

TEST DOCUMENTS, RESOURCES, AND TEST DOCUMENTS, RESOURCES, AND TRAININGS ONLINETRAININGS ONLINE

7

TEST SECURITYTEST SECURITY

TEST SECURITY STATEMENTTEST SECURITY STATEMENTAll test booklets, test questions, and

responses to those items in the Montana Comprehensive Assessment System Criterion-Referenced Test (MontCAS CRT) are secure materials and may not be copied or duplicated in any way or retained in the school after testing is completed. Systems and schools are required to return all test booklets to Measured Progress after testing is completed.

TESTING IRREGULARITYTESTING IRREGULARITY

10

Definition: Any event (before, during, or after testing) that could potentially impact the security of the test or the accuracy of the test data.

Includes any actions or precautions that vary from directions specified by testing contractor manuals or OPI.

Can constitute a security breach which could result in invalidating student scores and impacting AYP determinations.

TEST SECURITY/IRREGULARITY TEST SECURITY/IRREGULARITY ISSUESISSUES

◦Securing Test Materials◦Late or Missing Materials◦Procedural/Administration Issues

Security Breach Coaching Testing environment

11

SECURE TEST MATERIALSSECURE TEST MATERIALSTest booklets, answer booklets and other

materials defined by OPI or testing contractor as secure.

These must be kept in a secure location and not copied.

System and school test coordinators, principals, and test administrators are responsible for secure test materials.

Not maintaining the security of materials is a security breach.

12

TEST MATERIAL SECURITY- ROLES TEST MATERIAL SECURITY- ROLES AND RESPONSIBILITIESAND RESPONSIBILITIES

13

• The System Test Coordinator is responsible for collecting, inventorying, and returning all test booklets to Measured Progress.

• School test coordinators are responsible for the security of testing materials and their return to the System Test Coordinator.

• The Test Administrator is responsible for security. of test materials and their prompt return to the school testing coordinator

SPEEDY AND ACCURATE RETURN SPEEDY AND ACCURATE RETURN OF MATERIALSOF MATERIALS

14

Test BookletsAnswer Booklets

LATE OR MISSING MATERIALSLATE OR MISSING MATERIALS

Any missing test booklets or delays in returning them to Measured Progress are considered a security breach and will be treated as a serious testing irregularity and could impact AYP results.

SECURING TEST MATERIALSSECURING TEST MATERIALSTest materials must be kept secure when not

being used for testing[Insert school/district procedures for securing

test materials]◦ Central locked location◦ Material check in and check out procedure◦ Procedure for locking materials in classroom between

sessions◦ Procedure for when a student who needs extra time

beyond the scheduled session moves with test materials to alternate location.

PRO

CED

URA

L--S

ECU

RITY

BRE

ACH

PRO

CED

URA

L--S

ECU

RITY

BRE

ACH

17

A security breach is a violation of a testing procedure that gives an unfair advantage to a student or a group of students. A security breach could jeopardize the security and integrity of the tests and/or resulting data.

DEFINITION

MAINTAINING A STANDARDIZED TEST MAINTAINING A STANDARDIZED TEST ENVIRONMENT- DO ACTIVITIESENVIRONMENT- DO ACTIVITIES

Do cover or remove bulletin boards and instructional aids

Do ensure that no student has any form of assistance or material that other students in the state do not have.

Do clear home screen (not memory) of calculators both before and after calculator use sessions.

Spring 201318PR

OCE

DU

RAL-

-SEC

URI

TY B

REAC

HPR

OCE

DU

RAL-

-SEC

URI

TY B

REAC

H

Test Security and Cell PhonesTest Security and Cell PhonesDo not allow students to have cell phones in

testing areas. (This constitutes a test irregularity)◦ Not to be used as calculators◦ Not to be used when a student has completed the test and

is waiting◦ Not to be in testing area EVER

Recommendations◦ Cell phones should be left in a locker or at home on testing

days. (include this in any communications with parents)◦ Let students know consequences◦ Do a cell phone reminder before the testing session starts,

as students enter the room.

Maintaining a Standardized Maintaining a Standardized Test Environment- Do not activitiesTest Environment- Do not activities

20

Do not ◦ Answer questions related to the test.

◦ Allow students to have cell phones, headphones,* or other electronic communication devices in testing areas.

◦ Allow students to use dictionaries, thesauri, or reference sources.*

◦ Allow students to use calculators when not specified.

◦ Fill in any unanswered questions or provide actual answers.

◦ Give students access to secure test questions prior to testing or discuss test questions at any time.

◦ Do not… deviate from the procedures specified in the MontCAS test coordinator and administrator manuals. (NOTE: there may be some cases in which the OPI requires procedures different from those in the manuals published by the test contractors. DO follow the OPI procedures.)

◦ Leave students unattended with testing materials or to take them unaccompanied to another location.

◦ Allow students to go back to previous test sections to review, complete, or revise answers.

◦ Allow students to re-test once they have started or completed a test section.

◦ Provide accommodations other than those specified and allowed in MontCAS accommodations guidance.

PRO

CED

URA

L--S

ECU

RITY

BRE

ACH

PRO

CED

URA

L--S

ECU

RITY

BRE

ACH

COACHINGCOACHINGCoaching is a type of a procedural security

breach. Examples:

◦ Providing answers to students◦ Changing student responses◦ Influencing student responses to test questions

by offering hints, clues, cues, facial expressions, nods, voice inflections, or any other manner of assistance that could impact a student’s answers.

21PRO

CED

URA

L--S

ECU

RITY

BRE

ACH

PRO

CED

URA

L--S

ECU

RITY

BRE

ACH

WHAT DO I DO IF….WHAT DO I DO IF….A student asks me what a word is…

https://tellagami.com/gami/AFC338/I see a student bubbling in the wrong place?

https://tellagami.com/gami/41NDX1/

I want to provide global instructions; can I do that and how do I do it properly?https://tellagami.com/gami/FM58IC/

A student tells me there is a problem with one of the items on the CRT?https://tellagami.com/gami/FM58IC/

TEST SECURITY DOCUMENTATIONTEST SECURITY DOCUMENTATION

23

Information sheet for teachers◦ Information sheet for teachers describing their test

security responsibilities and what training they should expect.

◦ Each educator handling materials or administering the CRT or CRT-Alternate must receive a copy of this form so they will know what training to expect and what is expected of them.

SECURITY REMINDERSSECURITY REMINDERS

24

Delays in accurate AIM data entry or return of used Answer Booklets could impact timely reporting of results and/or impact individual district or school AYP results.

Any missing Test Booklets or delays in returning them to Measured Progress are considered a security breach and will be treated as a serious testing irregularity and could impact AYP results.

25

PROCEDURAL--SECURITY BREACHPROCEDURAL--SECURITY BREACH

A security breach is a violation of a testing procedure that gives an unfair advantage to a student or a group of students and could jeopardize the security and integrity of the tests and/or resulting data.

TEST IRREGULARITY TEST IRREGULARITY REPORTING REPORTING PROCESSPROCESS

2013 Test Security Manual, pages 8 and 22

Who should be informed?◦ System Test Coordinator◦ Principal◦ Superintendent◦ OPI—within three days by phone and five

days online

Spring 2013 26

TEST TEST ADMINISTRATIONADMINISTRATION

PREPARING TESTING PREPARING TESTING ENVIRONMENTENVIRONMENTPost “Testing in Progress – Do Not Disturb” signs on all

entrances to the testing location. (available on OPI site)Remove or cover all instructional materials

◦ E.G. anchor charts, word walls, anything that students use to activate memory.

◦ Hint: can use newspapers or roll paper to cover materials instead of taking them down and putting them back up again.

Collect necessary materials

TO COVER OR NOT TO COVER?TO COVER OR NOT TO COVER?

MATERIALSMATERIALS Test booklets Answer booklets (with barcode label attached prior to testing)

◦ 3rd graders will write all answers directly in the test booklet.

#2 Pencils Highlighters (optional) Scratch Paper

◦ Collect and destroy after each day of testing Calculators

◦ Only for calculator use sessions

◦ One per student

◦ Clear screens and collect after each session Math reference sheets (during math session only)

BEFORE THE TEST SESSION: WRITE BEFORE THE TEST SESSION: WRITE ON BOARDON BOARD Teacher’s name Keep working until you reach the stop sign for the session, located

on the bottom-right corner of the page. Do not go beyond the stop sign for the session. When you finish the test session, you may check your work in that

test session only. For constructed-response questions, make sure your work fits in

the space provided. Mark the answer to each multiple-choice question by filling in the

correct bubble in your Answer Booklet. (Grade 3 students will mark their answers directly in their Test Booklets.)

STUDENT BARCODE LABEL SAMPLESTUDENT BARCODE LABEL SAMPLE

32

Example of Front Page

If you did not receive a label for a student or it

was damaged, boxes A, B, D & E must be bubbled.

Required for all grades 4-8

Printed on the cover of each test booklet is a form number (1-4.)

The test booklet form that a student tests with must be bubbled here in Box F. Note: grade 3 has an integrated test booklet and the form number is already printed on it

Example of Page 2

For Special Education & 504

students.

Use pg. 33-36 from the TAM to code in accommodations.

BUBBLINGBUBBLINGAs part of the TA script you will go over

proper bubbling including what to do when changing answers.

TEST ADMINISTRATION SCRIPTSTEST ADMINISTRATION SCRIPTSGo over test administration scripts in the

TAMSuggested activity: do a quick review and

have TAs mark the pertinent parts of the script. (highlighter or tabs)

Each TA will need to have a TAM for this.

WHEN STUDENTS FINISH EARLY:WHEN STUDENTS FINISH EARLY:Announce ahead of time

◦Students who finish early may: Check their work in that session of the test only Sit quietly Use scratch paper to draw Read silently

WHAT IF THERE IS A MAJOR WHAT IF THERE IS A MAJOR DISRUPTION?DISRUPTION?E.g.: Fire Alarm, power outageIf possible collect and secure tests[insert your school procedures]

DISRUPTIVE STUDENTS?DISRUPTIVE STUDENTS?

Plan ahead◦Have a location where you can bring a student

who is disruptive.◦Have a plan for staff to escort and then

administer the test in that location.[Discuss your school’s plan]

MAKE UP TESTSMAKE UP TESTS

40

Make up Tests◦If one session is missed does the student

need to make up the whole assessment in order? No the student should continue on with the

sessions as scheduled and make up only the session/sessions that were missed.

◦[Discuss your school’s make up test plan]

ACCOMMODATIONSACCOMMODATIONS

41

Accommodations are changes in the standard Accommodations are changes in the standard practices and procedures used to teach and practices and procedures used to teach and assess students.assess students.

42

•Accommodations decisions are made for individual students not groups of students.

•For use on a standardized test, accommodations should be in place and used in formal classroom assessment routinely three months prior to the standardized assessment.

ACCOMMODATION ELIGIBILITY --ACCOMMODATION ELIGIBILITY --GENERAL POPULATIONGENERAL POPULATION

General Student Population

Parameters for use of Standard Accommodations

Parameters for Use of Nonstandard Accommodations

General student population

• Can be used, must be based on individual student need and not an instructional strategy determined for a group of students.

• Must be a practice routinely used by this student in classroom instruction and assessment for at least 2-3 months prior to testing;

• Accommodation(s) used must be coded in the Student Answer Booklet on page 2.

• **Accommodations intended for students with IEPs /504 or LEP plans only. Approval is required for general education students using the online approval program.

• Not allowed under any circumstances.

Spring 201343

ACCOMMODATION ELIGIBILITY --ACCOMMODATION ELIGIBILITY --STUDENTS WITH IEPS OR 504 PLANSSTUDENTS WITH IEPS OR 504 PLANS

Students with IEPs or 504 Plans

Parameters for use of Standard Accommodations

Parameters for use of Non-Standard Accommodations

Students with IEPs or 504 Plans

• Can be used, based on individual student need.

• Need for accommodation is documented in the student’s IEP/504 Plan.

• Allowable standard accommodations for the CRT that are described in a student’s IEP/504 Plan should match those used during the administration of the CRT.

• Accommodation(s) used must be coded in the Student Answer Booklet on Page 2.

• Accommodations coded ** can be used.

• Can be used if need for accommodation is documented in the student’s IEP/504 Plan;

• Accommodation(s) used must be coded in the Student Answer Booklet on Page 2;

• Student’s results for content area will not be calculated in the averages for AYP determination.

• Student will be considered a non-participant in the calculation of the AYP participation rate.

• Non-standard accommodations for the CRT that are described in a student’s IEP/504 Plan should match those used during the administration of the CRT.

Spring 2013

ACCOMMODATION ELIGIBILITY--LEP ACCOMMODATION ELIGIBILITY--LEP STUDENTSSTUDENTS

LEP Students Parameters for use of Standard Accommodations

Parameters for use of Non- Standard

AccommodationsLEP Students • Can be used, based on

individual student need;• Must be a practice

routinely used by this student in classroom instruction and assessment for at least 2-3 months prior to testing;

• Accommodation(s) used must be coded in the Student Answer Booklet on page 2.

• Can be used if need for accommodation is documented in the student’s IEP/504 Plan or after consultation with the OPI state assessment director and the OPI bilingual specialist.

• Accommodation(s) used must be coded in the Student Answer Booklet on Page 2;

• Student’s results for content area will not be calculated in the averages for AYP Determination.

• Student will be considered a non-participant in the calculation of the AYP participation rate.

Spring 2013

IMPACT OF ACCOMMODATIONS ON IMPACT OF ACCOMMODATIONS ON TEST SCORESTEST SCORES

Standard Accommodations change the way in which a student participates in a test, but do not alter what the test is designed to measure.

Nonstandard Accommodations

change the nature of what is being measured by a test.

Impact on Validity of Test Score: none; score can be aggregated with scores of students taking the test without accommodations.

Impact on Validity of Test Score: The score becomes invalid; the student becomes a non-participant when calculating participation rates for AYP, and the score is not included in calculating a school’s proficiency rate for AYP.

46

WHAT PRACTICES DOES THIS WHAT PRACTICES DOES THIS INCLUDE?INCLUDE?

47

Presentation accommodations– Allow students to access information in ways that do not require visual reading of standard print.

Response accommodations – Allow students to complete activities or respond to questions in different ways.

Setting accommodations – Change the location in which an activity or test occurs or the conditions of the setting.

Timing and Scheduling accommodations –Change the way in which the time is organized.

ACCO

MM

ODA

TIO

N R

EMIN

DER

SAC

COM

MO

DATI

ON

REM

IND

ERS

48

Oral Presentation- StandardMath and Science:

◦ Tests may be read aloud to a student by the Test Administrator as a standard accommodation.

◦ Special math and science text features Reading Digits Reading Symbols Reading charts and graphs

Reading:◦ Test questions and answer choices may be read aloud to a

student by the Test Administrator as a standard accommodation. The reading passages MAY NOT be read aloud to a student as a standard accommodation.

PRESENTATION

Oral presentation-non standardReading

◦Reading aloud of the reading passages to a student or the student using text-reader software for reading passages is considered a non-standard accommodation.

Spring 201349AC

COM

MO

DATI

ON

REM

IND

ERS

ACCO

MM

ODA

TIO

N R

EMIN

DER

SPRESENTATION

RESPONSERESPONSE

50

Response Accommodations ◦ Accommodations requiring TA to transfer or scribe a

written response Word for word including punctuation No. 2 pencil Directly into answer booklet

◦ Test administrators may not: Clarify Elaborate Make any changes not initiated by the student

ACCOMMODATION REMINDERS

ACCOMMODATION REMINDERS

SETTINGSETTING

Small Group AdministrationNo more than 15 studentsOral presentation

◦ Ideally 3◦No more than 5

ACCOMMODATION REMINDERS

ACCOMMODATION REMINDERS

ACCO

MM

ODA

TIO

NS

& S

ECU

RITY

ACCO

MM

ODA

TIO

NS

& S

ECU

RITY

52

There is no standard accommodation (including oral presentation) that provides an opportunity for a student to request or receive help on a specific word, phrase, line, pronunciation, definition, item, question, answer choice or any part of the assessment.

Standard accommodations do not override standard administration of the CRT or the need for independent work by the students.

REMINDERS

2011-2012 School/District Data2011-2012 School/District Datawww.gems.opi.mt.gov

AT OR ABOVE PROFICIENCY AT OR ABOVE PROFICIENCY TRENDS—CRT DATATRENDS—CRT DATA

YOUR SCHOOL/DISTRICT TESTING YOUR SCHOOL/DISTRICT TESTING SCHEDULESCHEDULE[include your school/district testing

schedule]

Questions?Questions?

OPI Assessment ContactsOPI Assessment Contacts

57

Ashley Makowski, Administrative Assistant406-444-3511 or amakowski@mt.gov

Yvonne Field, Assessment Specialist406-444-0748 or yfield@mt.gov

Judy Snow, State Assessment Director406-444-3656 or jsnow@mt.gov

Measured Progress ContactsMeasured Progress Contacts

The CRT and the CRT-Alternate—Measured Progress, Inc.Dan Verdick, Montana CRT Program Manager

◦ dverdick@measuredprogress.org or 800-431-8901 x2220Nancy Hebb, Montana CRT Program Assistant

◦ hebb.nancyl@measuredprogress.org or 888-792-2741Tim Greenlaw, Montana CRT-Alternate Program Manager

◦ greenlaw.timothy@measuredprogress.org or 800-431-8901x2309