End Of Course Assessments Field Test Spring 2008

End Of Course Assessments

Field Test Spring 2008

By The Missouri Department of Elementary and Secondary Education

Assessment SectionApril 2008

PowerPoint OverviewPowerPoint OverviewThis presentation is geared toward the This presentation is geared toward the Spring 2008 Field Test and contains the Spring 2008 Field Test and contains the following information:following information:

• A brief history of End-Of-Course (EOC)A brief history of End-Of-Course (EOC)• EOC guidelines and calibrations with No Child EOC guidelines and calibrations with No Child

Left Behind (NCLB)Left Behind (NCLB)• An overview of the test development processAn overview of the test development process• The design of the EOC Field TestThe design of the EOC Field Test• The EOC BlueprintsThe EOC Blueprints• Information about ancillaries and reference Information about ancillaries and reference

sheetssheets

Why End of Course?Why End of Course?

DESE responded to feedback from Missouri DESE responded to feedback from Missouri districts regarding large scale assessments districts regarding large scale assessments for secondary school.for secondary school.

EOC Assessments were created to adapt EOC Assessments were created to adapt testing to the needs of Missouri districts, testing to the needs of Missouri districts, schools, teachers and students while schools, teachers and students while meeting state and federal requirements.meeting state and federal requirements.

EOC Assessments were created in conjunction EOC Assessments were created in conjunction with national standards of excellence and Show-with national standards of excellence and Show-Me Standards.Me Standards.

Both Course Level Expectations (CLE’s) and the Both Course Level Expectations (CLE’s) and the EOC Assessments are a continuous collaborative EOC Assessments are a continuous collaborative effort with:effort with:

• Missouri teachersMissouri teachers• AdministratorsAdministrators• Regional Instructional FacilitatorsRegional Instructional Facilitators• The Missouri Department of Elementary and Secondary The Missouri Department of Elementary and Secondary

Education (DESE) Education (DESE) • Assessment Resource Center (ARC)Assessment Resource Center (ARC)• The Riverside Publishing CompanyThe Riverside Publishing Company

Missouri educators are playing a key Missouri educators are playing a key role in the development of EOC role in the development of EOC assessments:assessments:

• Authoring itemsAuthoring items• Reviewing and revising items for Reviewing and revising items for

content, sensitivity, and biascontent, sensitivity, and bias• Revising items, when neededRevising items, when needed• Approving items to be on the final Approving items to be on the final

operational test forms.operational test forms.

Specific GuidelinesSpecific GuidelinesIn order to ensure the best assessments possible, the advisory committee In order to ensure the best assessments possible, the advisory committee and DESE have set specific guidelines for the tests. and DESE have set specific guidelines for the tests.

The items must:The items must:1.1. Be in full compliance with the federal legislation, No Child Left Behind Be in full compliance with the federal legislation, No Child Left Behind

(NCLB)(NCLB)

1.1. Be aligned to the Missouri Course Level Expectations (CLE’s)Be aligned to the Missouri Course Level Expectations (CLE’s)

2.2. Be aligned to Norman Webb’s Depth of Knowledge (DOK) cognitive Be aligned to Norman Webb’s Depth of Knowledge (DOK) cognitive levelslevels

3.3. Include multiple choice and performance events. (Algebra I, Biology, Include multiple choice and performance events. (Algebra I, Biology, and English II are the only tests that will have performance events).and English II are the only tests that will have performance events).

4.4. Have both stand-alone and passage-based itemsHave both stand-alone and passage-based items

5.5. Be criterion-referenced testsBe criterion-referenced tests

Missouri State Board of EducationMissouri State Board of Education

The Missouri State Board of Education identified the The Missouri State Board of Education identified the following purposes for EOC:following purposes for EOC:

Measuring and reflecting student mastery toward Measuring and reflecting student mastery toward post-secondary readinesspost-secondary readiness

Identifying students’ strengths and weaknessesIdentifying students’ strengths and weaknesses Communicating expectations for all studentsCommunicating expectations for all students Serving as the basis of state and national Serving as the basis of state and national

accountability plansaccountability plans Evaluating programsEvaluating programs

CLE’s instead of GLE’sCLE’s instead of GLE’s

Course Level Expectations are goals Course Level Expectations are goals for a particular course rather than a for a particular course rather than a grade level. grade level.

A course such as Algebra I could be A course such as Algebra I could be delivered in 8delivered in 8thth grade or any grade grade or any grade level in secondary school.level in secondary school.

Districts can offer courses with Districts can offer courses with different titles which cover the same different titles which cover the same CLE’s.CLE’s.

The right test at the right timeThe right test at the right time The responsibility for testing students belongs to The responsibility for testing students belongs to

the school district.the school district.

The district has the responsibility to match its The district has the responsibility to match its course course contentcontent with the CLE’s that are being with the CLE’s that are being tested in the EOC Assessment.tested in the EOC Assessment.

When the content of the assessed information is When the content of the assessed information is covered, the test can be administered regardless covered, the test can be administered regardless of student grade level.of student grade level.

Examiners will deliver the EOC assessment at the Examiners will deliver the EOC assessment at the appropriate time for their students.appropriate time for their students.

11stst Field Test Field Test

Spring 2008 Spring 2008

Algebra IAlgebra IBiologyBiology

English IIEnglish II

A full census field test will be A full census field test will be administered in three content areas administered in three content areas in approximately 2000 Missouri in approximately 2000 Missouri schools in the Spring of 2008.schools in the Spring of 2008.

The first operational administration The first operational administration of EOC will follow during the 2008-of EOC will follow during the 2008-2009 school year.2009 school year.

Online VersionOnline Version

A small group of students will trial A small group of students will trial the online version of the Spring 2008 the online version of the Spring 2008 EOC Field Test. EOC Field Test.

An online version of each test will An online version of each test will also be available in the operational also be available in the operational test to school districts wishing to use test to school districts wishing to use this format.this format.

No reports will be produced for the No reports will be produced for the districts from this Spring 2008 Field districts from this Spring 2008 Field Test.Test.

The EOC Assessment Field Test will The EOC Assessment Field Test will be used to provide future operational be used to provide future operational tests.tests.

District ReportsDistrict Reports

Braille/Large PrintBraille/Large Print

For the field test, the EOC will not have a For the field test, the EOC will not have a Braille/Large Print edition. Therefore, Braille/Large Print edition. Therefore, students needing Braille/Large Print students needing Braille/Large Print editions may have all content areas, editions may have all content areas, including the English II assessment, read including the English II assessment, read to them without invalidating the test.to them without invalidating the test.

The Examiner must mark the new bubble The Examiner must mark the new bubble on the SIS for the above accommodation.on the SIS for the above accommodation.

FormsForms

EOC Field Test consists of ten forms EOC Field Test consists of ten forms for both Sessions I and II of testing.for both Sessions I and II of testing.

The forms are spiraled (distributed in The forms are spiraled (distributed in roughly equal numbers) within each roughly equal numbers) within each content area and should not be content area and should not be separated.separated.

Two SessionsTwo Sessions

The EOC Field Tests are not strictly timed and are The EOC Field Tests are not strictly timed and are designed to fit into a normal class period.designed to fit into a normal class period.

EOC Field Test consists of two sessions for each EOC Field Test consists of two sessions for each content area content area

Session I = 55 minutesSession I = 55 minutes Session II = 55 minutesSession II = 55 minutes

55-minute class period (times are approximate)55-minute class period (times are approximate)• 10 minutes for test distribution, directions, collection, 10 minutes for test distribution, directions, collection,

etc. and etc. and • 45 minutes for actual test taking45 minutes for actual test taking

Session ISession I

Selected-response items (also known Selected-response items (also known as multiple choice questions)as multiple choice questions)

Each item (question) has four Each item (question) has four response options – one correct response options – one correct answer and three plausible answer and three plausible distractersdistracters

Session IISession II

Performance Event Performance Event • Requires students to work through more Requires students to work through more

complicated itemscomplicated items• Makes allowances for more than one Makes allowances for more than one

correct answercorrect answer• Allows for insight to be garnered about Allows for insight to be garnered about

the student’s ability to apply their the student’s ability to apply their knowledge and understanding to real-knowledge and understanding to real-life situationslife situations

Writing PromptWriting Prompt

This is a special type of performance This is a special type of performance event that appears in the English II event that appears in the English II assessmentassessment

It is open-endedIt is open-ended

It is scored holisticallyIt is scored holistically

Field Test SpecificationsField Test Specifications

Essential content and statistical Essential content and statistical considerations are aligned to:considerations are aligned to:

• Depth of Knowledge cognitive levelsDepth of Knowledge cognitive levels• Content Strand/Course Level Content Strand/Course Level

ExpectationsExpectations

Algebra I Field Test SpecificationsAlgebra I Field Test Specifications

20 Forms20 Forms• 10 multiple-choice forms each with 18 10 multiple-choice forms each with 18

itemsitems• 10 performance-event forms each with 10 performance-event forms each with

one performance eventone performance event• Each student being field tested will take:Each student being field tested will take:

1 multiple-choice form (Session I) and 1 multiple-choice form (Session I) and 1 performance event form (Session II)1 performance event form (Session II)

Field to OperationalField to Operational

Field Tested Items will be reviewed Field Tested Items will be reviewed and selected for the Operational Test and selected for the Operational Test based on:based on:• Content appropriatenessContent appropriateness• Statistical performance (Appendix A)Statistical performance (Appendix A)• Depth of Knowledge alignmentDepth of Knowledge alignment

Operational TestOperational Test

The field test items will be processedThe field test items will be processed into 4 operation forms for the into 4 operation forms for the Operational EOC Assessment.Operational EOC Assessment.

Each operational form will consist of Each operational form will consist of 2 sessions:2 sessions:• Session I = 35 multiple-choice itemsSession I = 35 multiple-choice items• Session II = 1 performance eventSession II = 1 performance event

Operational Algebra I BlueprintOperational Algebra I BlueprintContent Content StrandStrand

Target Target # #

PointsPoints

10% 10% TolerancToleranc

ee

Point Point RangeRange

Target % Target % Total Total PointsPoints

Min Min EmphasisEmphasis

Max Max EmphasisEmphasis

Number & Number & OperationOperation

88 0.80.8 7-97-9 21%21% 19%19% 23%23%

Algebraic Algebraic RelationshipsRelationships

2323INCLUDES INCLUDES A 4 POINT A 4 POINT PERFORM-PERFORM-

ANCE ANCE EVENTEVENT

2.32.3 21-2521-25 58%58% 53%53% 63%63%

Geometric & Geometric & Spatial Spatial RelationshipsRelationships

00 00 00 0%0% 0%0% 0%0%

MeasurementMeasurement 00 00 00 0%0% 0%0% 0%0%

Data & Data & ProbabilityProbability

88 0.80.8 7-97-9 21%21% 19%19% 23%23%

TOTALSTOTALS 3939 100%100%

Guiding Principles for Test Guiding Principles for Test ConstructionConstruction

Each performance event will be aligned to Each performance event will be aligned to one CLE from the Algebraic Relationships one CLE from the Algebraic Relationships strand.strand.

On the operational tests, a balance of On the operational tests, a balance of items written for particular CLEs will be items written for particular CLEs will be sought.sought.

Particular CLEs may be only assessed once Particular CLEs may be only assessed once on a form while other forms may contain on a form while other forms may contain several items for that CLE.several items for that CLE.

Some CLEs may be assessed more Some CLEs may be assessed more frequently because the skills are integral frequently because the skills are integral to the course.to the course.

Operational English II BlueprintOperational English II BlueprintContent StrandContent Strand Target Target

# # PointsPoints


Target % Target % Total Total PointsPoints



Develop and apply Develop and apply skills and strategies to skills and strategies to the reading process.the reading process.

1212 10-1410-14 31%31% 26%26% 36%36%

Develop and apply Develop and apply skills and strategies to skills and strategies to comprehend, analyze comprehend, analyze and evaluate fictions, and evaluate fictions, poetry and drama.poetry and drama.

99 8-118-11 23%23% 23%23% 28%28%

Develop & apply skills Develop & apply skills & strategies to & strategies to comprehend, analyze comprehend, analyze & evaluate nonfiction.& evaluate nonfiction.

99 8-118-11 23%23% 23%23% 28%28%

Writing: Writing:

Apply a writing process Apply a writing process in composing textin composing text

Compose well-Compose well-developed textdeveloped text

Write effectively in Write effectively in various forms & types various forms & types of writingof writing

9*9**This *This strand strand includes includes the 4-point the 4-point writing writing prompt.prompt.

8-98-9 23%23% 23%23% 23%23%


Guiding Principles for Test Guiding Principles for Test ConstructionConstruction

Each form will contain 5 passages: Either 2 fiction passages and 3 Each form will contain 5 passages: Either 2 fiction passages and 3 non-fiction passages or vice versa.non-fiction passages or vice versa.

The writing form/type will vary depending on the writing prompt The writing form/type will vary depending on the writing prompt selected. Writing prompts will be aligned to a primary CLE. selected. Writing prompts will be aligned to a primary CLE. Multiple writing CLES may be assessed to reflect the holistic Multiple writing CLES may be assessed to reflect the holistic rubric.rubric.

Some CLEs can be used for both fiction and nonfiction passages.Some CLEs can be used for both fiction and nonfiction passages.

Particular CLEs may be only assessed once on a form while other Particular CLEs may be only assessed once on a form while other forms may contain several items for that CLE.forms may contain several items for that CLE.

The goal will be to balance the CLEs across the forms. Since The goal will be to balance the CLEs across the forms. Since English II is passage-based, including every assessable CLE in this English II is passage-based, including every assessable CLE in this round of operational forms may not be possible.round of operational forms may not be possible.

Operational Biology BlueprintOperational Biology BlueprintContent Content StrandStrand

Target # Target # PointsPoints

10% 10%

ToleranceTolerancePoint Point

RangeRangeTarget Target % Total % Total PointsPoints



1.Properties and 1.Properties and Principle of Matter Principle of Matter and Energy.and Energy.

00 00 00 0%0% 0%0% 0%0%

2.Properties and 2.Properties and Principles of Force Principles of Force and Motion. and Motion.

00 00 00 0%0% 0%0% 0%0%

3.Characteristic 3.Characteristic and Interactions of and Interactions of Living Organisms.Living Organisms.

2222 2.22.2 20-20-2424

40%40% 36%36% 44%44%

4.Changes in 4.Changes in Ecosystems and Ecosystems and Interactions of Interactions of Organisms with Organisms with their their Environments.Environments.

1313 1.31.3 12-12-1414

25%25% 22%22% 27%27%

5.Processes and 5.Processes and Interactions of the Interactions of the Earth’s systems Earth’s systems (Geosphere, (Geosphere, Atmosphere, & Atmosphere, & Hydrosphere)Hydrosphere)

00 00 00 0%0% 0%0% 0%0%

Continued on next slideContinued on next slide

Operational Biology BlueprintOperational Biology BlueprintContinued from previous slideContinued from previous slide

Content StrandContent Strand Target Target # Points# Points

10% 10% ToleranceTolerance


Target Target % Total % Total PointsPoints



6.Composition and 6.Composition and Structure of the Structure of the Universe and the Universe and the Motion of the Motion of the Objects within it.Objects within it.

00 00 00 0%0% 0%0% 0%0%

7. Scientific Inquiry7. Scientific Inquiry 20*20*This strand This strand

contains the contains the performance performance

eventevent

(2.0)(2.0) 2020 36%36% 36%36% 36%36%

8. Impact of 8. Impact of Science, Technology Science, Technology and Human Activityand Human Activity

00..

00 00 0%0% 0%0% 0%0%


Breakdown by DOK Breakdown by DOK for all content areas for all content areas

DOK levels for each CLE were DOK levels for each CLE were determined by groups of Missouri determined by groups of Missouri educators during an alignment educators during an alignment review, facilitated by Norman Webb.review, facilitated by Norman Webb.

At least 80% or more of the items on At least 80% or more of the items on the operational form should be at the the operational form should be at the designated DOK level.designated DOK level.

The other 20% may fall below the The other 20% may fall below the designated DOK level.designated DOK level.

Non-secure materialsNon-secure materials

Non-secure materials will be Non-secure materials will be delivered to the districts the week of delivered to the districts the week of April 14April 14thth

Non-secure materials include: Test Non-secure materials include: Test Coordinator Manuals, Examiners Coordinator Manuals, Examiners Manuals, barcode labels, return kits, Manuals, barcode labels, return kits, and answer documents for Session I and answer documents for Session I AssessmentsAssessments

Manipulatives for the field testManipulatives for the field test Algebra I:

• Students can use a calculator during both Session I and Session II. A four-function calculator is sufficient for the needs of this assessment.

• The testing materials include an Algebra I reference sheet for use during both Session I and Session II.

Biology: • Students are not provided nor do they need a reference sheet. • Students can NOT use a calculator.

English II, Session II only:• Students can use a standard dictionary, thesaurus, or grammar

handbook only for Session II. • These tools are not allowed in Session I.

NOTE: For students taking either the Algebra I or Biology test, their writing, spelling and grammar will not be evaluated.

LabelsLabels

Each content area will have 2 labels, Each content area will have 2 labels, one for Session I and the second for one for Session I and the second for Session IISession II

The labels will be pre-coded based on The labels will be pre-coded based on the MOSIS core datathe MOSIS core data

The labels are sorted by District, The labels are sorted by District, School, Grade, Last Name, First School, Grade, Last Name, First NameName

Secure MaterialsSecure Materials

Session I and Session II test books Session I and Session II test books will arrive in the Districts the week of will arrive in the Districts the week of April 21stApril 21st

RulesRules

The rules for EOC testing are The rules for EOC testing are basically the same as MAP testing basically the same as MAP testing regarding calculators, highlighters, regarding calculators, highlighters, erasures, all testing environments, erasures, all testing environments, etc.etc.

The same rules apply for test The same rules apply for test security and test administration in security and test administration in the EOC as in MAP.the EOC as in MAP.

GlossaryGlossary

Open-ended question - the student’s Open-ended question - the student’s answer can vary depending on the answer can vary depending on the student’s thought process. An example of student’s thought process. An example of an open-ended item might be to write a an open-ended item might be to write a friendly letter to a friend telling them friendly letter to a friend telling them about the sports event you just watched about the sports event you just watched

Holistically scored – Scored on the overall Holistically scored – Scored on the overall impression of how well a response reflects impression of how well a response reflects the scoring criteria.the scoring criteria.

Documents

End Of Course Assessments Field Test Spring 2008