1

Fall 2014 Test Coordinator

Training

End of Course Exams (EoCs)

Purpose of an EoC Exam• Statewide curriculum alignment.

• Determining and measuring teacher and school leader effectiveness—Educator Effectiveness System (EES) (6.60.8 NMAC).– The evaluation system must include measures of student

achievement growth (50 percent):• SBA/PARCC—35%• Department approved assessments—15%

• Providing an Alternative Demonstration of Competency (ADC) in the academic content standards required for graduation (6.19.7.7 NMAC).

Developing an EoC Exam• Step 1: Identify the construct, domains, and

indicators to be measured – standards based assessment.

– The knowledge, skills and abilities proficient students should know and be able to do at the end of a course or grade.

Developing an EoC Exam• Step 2: Develop a Table of Specifications.– Table of Specifications becomes the concrete

assessment plan for the EoC exam.–What type of assessment is most appropriate for each

domain?

–What is the cognitive difficulty or cognitive demand for each domain?

– How many items are necessary to comprehensively assess the knowledge, skills and abilities for each domain of the assessment?

Developing an EoC Exam• Step 3: Develop the Assessment– Groups of teachers from across the state.– Training on item writing.– Collectively create items.– Produce test form.– Create and verify answer key.– Produce rubrics and administration guide for all

constructed response and performance items.

Developing an EoC Exam• Step 4: Content/Bias Review– Groups of teachers from across the state.• New and Previous

– Training– Review blueprints– Review test items for content and bias– Verify logic of test form– Verify answer key

Developing an EoC Exam• Step 5: Give the test and submit data.

– Tests will be available online on Blackboard beginning spring 2015

– (Yay!)– Pilot fall 2014 (select districts/schools)– No cost to districts/schools– More information forthcoming

Data Submission

STARS

STARS REPORTS

EES ADC

SOAP

Item Analysis Standard Setting

Revision

Submission Full Approval

Why is Submitting EoC Data Important?

• Student Level Data is Used for– Standard Setting (passing scores for new

tests)– Item Review– Student Records for passing an exam– Alternative Demonstration of Competency

for Graduation– Educator Effectiveness System – Provides larger sample size

Data Submission

Two Required Submissions:

1) SOAP Item Data– Only for Select EoCs– Used for setting passing scores– Minimum 50 RANDOM student scores per test

2) STARS Overall Score– For ALL EoCs– For ALL Students

EoC Reports in STARS

STARS Reports

• EOC Submission Validation Report

–Student level validation errors for tests.–Report used by districts to validate 40, 80, 120

and End of Year EOC submissions.–Becomes a formal process for Fall 2014

STARS EoC Report Elements

• District Name• District Code• Location Name• Location ID• Student ID• Student First Name• Student Last Name• Subject• Subtest Identifiers• Grade From• Grade To• Student Grade Level

• Max Score• Raw Score• Max Tests• Subject Count• Date Test Active• Date Test Inactive• Test Date• Exists Error• Date Error• Score Error• Grade Error• Max Test

Data Quality is Essential!

• Data needs to be accurate• Follow instructions given by PED– Data Submission Guide– STARS Manual– Not sure—Ask!

• Errors create delay in results• STARS EoC data submissions tied to

40/80/120/EOY day reporting

Developing an EoC Exam• Step 6: Data Analysis

– P-values– IRT analysis– Possible elimination of items– Flag items for further review

Developing an EoC Exam• Step 7: Standard Setting

– Groups of teachers• New and Previous

– Set passing scores• Bookmark Method • Body of Work Method

Developing an EoC Exam• Step 8: Content Review #2

– Groups of teachers• New and Previous

– Review flagged items– Consider district/school/DTC/STC feedback

Developing an EoC Exam• Step 9: Re-version and re-administer

– Any change new version– Minor changes, passing score remains– Major changes, new passing scores must be set

End of EoCs

•New Version—version control• Adoption of new standards—Common Core

•The performance does not meet scientific standards• Item level analysis• Reliability• pValues

EoC Procedures

• Current: paper tests• Upcoming: online hosting on Blackboard

• December: Blackboard pilot with select • districts• Spring: Widespread Blackboard • administration

• Paper administration will be permitted.

• Data will be automatically be submitted to PED!

Summer 2014 EoCs

Summer 2014 EoCs

EoC Administration WindowsWindow Students Dates

Fall Students in 1-semester courses and HS retakes

2 consecutive weeks during last 3 weeks of fall semester

Winter HS retakes 1 week during first 3 weeks of spring semester

Spring All students 2 consecutive weeks during last 3 weeks of spring semester

Summer HS retakes and students in summer school

6/22 – 6/26/15 or last week of summer school course

EoC Test Structure

• Most Tests: Multiple Choice • (request from AAAC)

• Elective Courses: Performance Components

• HS: 90 minutes• MS and ES: 60 minutes

EoCs: High Stakes Tests• Treat as Final Exams

• Security Procedures are Essential!

• Directions for Administration (DFA): administration procedures, teacher access, use of calculators, use of scratch paper, make-ups, accommodations, materials chain of custody, scoring requirements, administration script

EoCs: High Stakes Tests

• Testing Irregularities: submit to PED

• DFA administration procedures will change with implementation of CBT.

• Security will be easier to maintain.

EoCs: Student Preparation

• Teachers:

• Access Blueprints (on PED website)• PBA: Access administration guides (on PED

website)• DFA (on PED website)• NAEP Questions Tool (on NAEP website)

EoC Questions?

EoC DataJudy Harmon

judith.harmon@state.nm.us

EoC ProceduresJoslyn Overby

joslyn.overby@state.nm.us