Assessment Literacy Series

1

-Module 3-Item Specifications

Participants will:1. Examine multiple choice and constructed

response items/tasks.2. Develop items/tasks that fit the previously

created specifications and blueprint in terms of:

content accuracy; item type; cognitive load; and sufficiency.

2

Objectives

Participants may wish to reference the following:

Guides• Handout #4 – DoK Chart• Handout #5 – Item Examples

Other “Stuff”• DoK Chart II• “Smart Book”• Textbooks, teacher manuals, and other supplemental

materials

3

Helpful Tools

4

Outline of Module 3

Module 3: Item Specifications

Multiple Choice Items

Constructed Response

Items/Tasks

Short Constructed

Response

Extended Constructed

Response

Process Steps

MC items consist of a stem and answer options• Grades K-1 = 3 options, Grades 2-12 = 4 options

MC items contain only one correct answer Other options (distractors) are the same structure and length as the answer The distractors should be plausible (realistic) Balance the placement of the correct answerAvoid answer options that provide clues to the answer

5

Multiple Choice (MC) Guidelines

Answer options should be in ascending or descending order when possible

Avoid “All of the above” and “None of the above”

Directions state what to do, where and how to respond, and point values

Refrain from adding directions when test-takers are repeating the same behavior from item to item

6

MC Guidelines (cont.)

Constructed Response (CR) Guidelines

Language is appropriate to the age and experience of the students

Student expectations are clear-• Explain vs. Discuss• Describe vs. Comment

State the extent of the expected answer• Give three reasons vs. give some reasonsDirections state what to do, where and how

to respond, and point valuesRefrain from adding directions when test-

takers repeat the same behavior from item/task to item/task

7

Short Constructed Response (SCR) items/tasks:• One step to solve• Requires a brief response (2-5 minutes)• Worth up to 3 points

Extended Constructed Response (ECR) items/tasks:• Multiple steps to solve• Requires 5-10 minutes to answer• Worth 4 or more points

Note: ALL CR items/tasks require a well-developed, scoring rubric. 8

CR Guidelines (cont.)

Scenarios and passages should be:• Relatively short• Developmentally appropriate• Sensitive to readability

Performance expectations must state exactly what is to be observed and how it will be measured.

Items/tasks are considered secure even in draft form.

Copyrights must be handled appropriately.Images, graphs, and charts must be clear and of

sufficient size for interpretation.9

Helpful Hints

Each item/task requires a unique identifier. ◦ Tags contain information used to code and identify items/tasks. ◦ Item tags typically contain:• Item number• Subject• Grade• Post test• Item type• DoK level• Content standard number

(0008.MTH.GR4.POST.MC-LV2-4OA1) 10

Item Tag Codification

Item # Subject Grade Post Item Type

DoK Standard ID

1. Review specification tables, blueprint, and targeted standards.

2. Draft the item, passage, prompt, or scenarios. Insert any necessary tables, graphs, or images.

3. For MC items, create the single correct answer and then identify distractors, which reflect common errors/misinterpretations. For CR items/tasks, create a scoring “rubric” that articulates different levels of performance, including a sample response for each level.

4. Specify item/task unique ID: [e.g., item number. subject. grade. post. item type-DoK level-content standard number]

5. Repeat Steps 1-4 for the remainder of the items/tasks needed to complete the blueprint.

11

Process Steps

QA ChecklistTypes, point values, and DoKs match the

blueprint.There are sufficient items/tasks to sample the

targeted content.The items/tasks are developmentally appropriate

for the intended test-takers.The correct answers and/or expected responses

are clearly identified.Each item/task is assigned a unique ID.

12

Think-Pair-ShareBased on the discussed item specifications, decide what is wrong with the following questions.

1. Some scientists believe that Pluto is __________.A.an escaped moon of NeptuneB.usually the most distant planet in the solar systemC.the name of Mickey Mouse’s dogD.all of the above

2. The people of Iceland __________.a country located just outside the Arctic CircleA.work to keep their culture aliveB.claim to be descendants of the AztecsC.the capital, Reykjavik, where arms talks have been held

13

SummaryModule 3: Item Specifications•Developed performance measure items/tasks that match the applicable blueprints.

Next StepsModule 4: Scoring Keys and Rubrics•Given the items/tasks created, develop scoring keys and rubrics.

14

Summary & Next Steps