Part One: Reading Test Specifications One: Reading Test Specifications I. TEST PURPOSE The purpose of this test is to measure New Mexico eleventh-grade students’ level of proficiency

1

New Mexico High School Standards Assessment (NMHSSA) Test and Item Specifications

March 17, 2003

Part One: Reading Test Specifications

I. TEST PURPOSE The purpose of this test is to measure New Mexico eleventh-grade students’ level of proficiency in meeting the New Mexico Content Standards and Benchmarks for Language Arts as this document applies to reading. The test will provide valid, reliable, and useful information to New Mexico students, parents, educators, the business community, and institutions of higher learning on whether students have met the expectations set by the State Board of Education and New Mexico educators for performance on the state standards. This test will contain a wide variety of literary, informational, and functional reading passages with accompanying items designed to measure a student’s ability to comprehend, apply, analyze, synthesize, and evaluate information. II. GENERAL TEST CONSIDERATIONS 1. The operational Reading Test forms and practice Reading Test form will be designed so that

the administration time will be approximately 90 minutes, though students should be given as much time as necessary to complete the test.

2. The test content will be culturally diverse, well written, and appropriate for eleventh-grade students. The stimulus materials and items will be presented in order to gain a true picture of each student’s reading skills.

3. Items will not display unfair representations of gender, race, disabled individuals, or cultural or religious groups. Items will not contain material that may disadvantage a particular gender, race, cultural, religious, or disabled group. Across all forms, there will be an equal representation of gender and involvement in active and passive roles (i.e., avoiding gender stereotyping).

4. The testing instructions to the students will include sample items for each item type on the test. 5. Students may not use resource materials, such as dictionaries, during the administration of the

test.

2

III. TEST CONTENT Because the New Mexico Language Arts Content Standards and Benchmarks is a spiraling framework, the performance standards do not exist as isolated skills. The measurable objectives are drawn primarily from the tenth grade standards but include standards from other grade levels as well. The Reading Test does not directly assess performance standards that are best measured in the classroom (i.e., speaking and writing tasks/activities); however, it does assess the reading skills that a student would need to possess in order to complete the performance tasks successfully. Benchmark 1: Read, synthesize, and analyze text. (Focus: Objectives assess a student’s ability to comprehend, synthesize, and analyze text.)

Assessment Objectives:

1. Evaluate problems and solutions in a text by: • identifying the problem and relevant issues • understanding the significance of the problem • evaluating the effectiveness of the solution.

2. Recognize and evaluate a writer’s position and analyze the writer’s support to determine

if it is relevant, valid, and reliable.

Benchmark 2: Demonstrate critical thinking skills and evaluate text. (Focus: Objectives assess a student’s ability to use critical thinking and evaluative skills.) Assessment Objectives:

3. Critically interpret and evaluate experiences, literature, language, and ideas by making generalizations supported by specific references.

4. Identify critical questions that would lead to a broader understanding of a selection.

5. Understand complex literary terms and find examples in a text.

6. Read critically and independently to draw conclusions, make inferences, and

make predictions from text. 7. Independently apply the reading process and strategies to a variety of texts and use the

defining features and structures of those works to understand main elements, perspective, and style.

3

Benchmark 3: Apply knowledge of the reading process. (Focus: Objectives assess a student’s ability to use reading strategies and understand text features and literary techniques.) Assessment Objectives:

8. Analyze the ideas of others by identifying the ways in which writers: • introduce and develop a main idea • choose and incorporate significant, supporting, relevant details

9. Analyze the ideas of others by identifying ways in which writers: • relate structure/organization to the ideas • use effective word choice as a basis for coherence • achieve a sense of completeness and closure.

10. Differentiate among literal, figurative, and connotative meanings.

4

IV. READING PASSAGES Reading passages will range from 100 to 900 words and will be appropriate for eleventh-grade New Mexico students. Reading passages will either be commissioned or drawn from published sources. Special effort will be made to locate and select passages that reflect the diversity of the state of New Mexico. Reading passages will fall into three general categories:

• Literary Text may include poetry, short stories, vignettes, personal narratives, biographies, memoirs, diaries, scripts, novel excerpts, drama, myths and folklore, and literary essays.

• Informational Text may include newspaper and magazine articles, letters to an editor, informational essays, debates, reviews, historical documents, political commentaries, encyclopedias and other reference materials, pamphlets, book excerpts, and materials drawn from web sites.

• Functional Text includes work-related and task-oriented text. Work-related text will involve applications one might experience in a business or industry setting; it may include instructions, e-mail, memos, letters, forms, applications, schedules, or other texts that one might encounter in a work situation. Task-oriented text may include want ads, applications, tax forms and information, maps, recipes, instructions, advertisements, consumer reports, travel books, manuals, catalogs, yellow pages, and schedules.

Each form will have one passage that is fairly lengthy (up to 900 words). Each form will have a pair of passages that are thematically linked and will have items that refer to each passage and across both passages. Each form will have approximately two passages from each category (literary, informational, functional). Each form will have a minimum of one work-related text and one poetry or drama selection. The total word count of all passages on a single form of the Reading Test will be approximately 2500-3000 words.

General Reading Passage Considerations 1. So that the test is a fair and valid assessment, passages will not exhibit cultural or other forms

of bias that might disadvantage any group (or groups) of students and will avoid subject matter that might cause an emotional response from the students.

2. Reading passages will reflect New Mexico’s cultural diversity. Passages will be well written and appropriate for high school students. Informational passages will be factually correct. Where appropriate, the passages will include original illustrations, other graphics, sidebars, etc.

3. If a passage is an excerpt from a larger work, it will represent a complete thought or idea and be able to stand on its own.

4. Passages will not appear in textbooks or anthologies that are part of most high school reading/English curricula.

5

5. The passages will be rich enough to allow high-quality items to be written. 6. Permissions to use copyrighted material will be secured (including Internet rights for passages

used with released items). 7. The Content and Fairness Committees and New Mexico State Department of Education (SDE)

will judge the overall appropriateness of the passages.

Part Two: Reading Test Item Specifications V. TEST ITEMS

The test will consist of two question types: multiple choice (MC) and constructed response (CR).

• The operational test will contain 30 multiple-choice items, worth one point each. Students will select one correct answer from the four response choices provided. Each multiple-choice item will have only one correct answer. The answer and three distracters will be approximately the same length, have the same format, and be syntactically and semantically parallel.

• The operational test will contain 6 constructed-response items; two of these will be worth 4 points each, and four will be worth 2 points each. For constructed-response items, students will construct text-based responses using phrase(s), sentence(s), or paragraph(s).

General Test Item Considerations 1. The set of items accompanying a passage will assess as many assessment objectives as

possible. 2. Each test item will measure the assessment objective it is intended to measure. 3. The test items will focus on the important themes and details in the text and require higher-

order thinking skills; few items will require the student to merely recall/find information in the text.

4. To the greatest extent possible, the test items will be constructed so that the student must read the text in order to answer the item, minimizing the impact of prior knowledge.

5. The test items will be presented in a logical order (e.g., main idea items would appear first). 6. The constructed response items and multiple-choice items will be arranged in a logically

mixed order (i.e., all of the multiple choice items will not precede all constructed response items or vice versa).

7. To the greatest extent possible, no item on the test will “clue” the answer to any other item. 8. Each item will function independently; the answer to one item will not be required as a

condition for answering subsequent items. 9. The test items will be worded precisely and clearly so that the student understands what is

being asked. No items should be “trick” questions. 10. The test items will avoid language that shows bias or is otherwise likely to be offensive or to

disadvantage a particular group of students.

6

11. Each multiple-choice item will contain a question (or incomplete statement) and four answer (or completion) options, only one of which is correct. Questions are preferred, but open stems are permitted, especially where an item might be stated awkwardly as a question.

12. The test item distracters will be plausible but incorrect. 13. The answer options will be presented in a logical order (e.g., numerical, sequential, etc.) where

appropriate. 14. There will be roughly the same number of A, B, C, and D correct answers on each form. 15. Each constructed-response item will give a clear indication of what is required of students in

order to earn full credit. (e.g., Explain two ways . . . .) 16. Constructed-response items will not ask students to reveal personal information. 17. The Content and Fairness committees and SDE will judge the appropriateness of each item. VI. TEST ITEM DISTRIBUTION The following table shows item distribution by benchmark in the operational test.

Benchmark

Number of Assessment Objectives

Number of MC

(Approx. Range)

Number of SCR/ECR (Approx. Range)

1. Read, synthesize, and analyze text 2 6-11 0-2/0-1 2. Demonstrate critical thinking skills and

evaluate text 5 9-15 0-2/0-1

3. Apply knowledge of reading process 3 7-12 0-2/0-1 Totals 10 30 max. 6 max Note: MC = Multiple choice; SCR = Short constructed response; ECR = Extended constructed response.

7

VII. SCORING

The following table shows the distribution of score points by item type.

Type Number of Items Total Points Percent of the Total Score

Multiple choice 30 30 65% Short response (0-2) 4 8 17.5% Extended response (0-4) 2 8 17.5% Total 36 46 100% Note: Each multiple-choice item is worth 1 point, and each constructed-response item is worth 2 or 4 points.

For each constructed-response item, an individual scoring guide with exemplars will be developed to score the item on a scale of 0 to 2 or 0 to 4. Information from the pilot testing will be used to refine these scoring guides for use with the final forms of the test.

Responses will be scored with emphasis on communication of ideas. Conventions of writing (sentence structure, word choice, usage, grammar, and mechanics) will be disregarded unless they seriously interfere with communication. VIII. ITEM SPECIFICATIONS Organization: The item specifications are organized by benchmark. Within each benchmark, the assessment objectives, text categories, text content, item types, and response formats are defined and followed by example items and depth of knowledge classifications. Appendix A explains the Depth of Knowledge classifications. Benchmark 1: Read, synthesize, and analyze text. Assessment Objective 1: Evaluate problems and solutions in a text by:

• identifying the problem and relevant issues • understanding the significance of the problem • evaluating the effectiveness of the solution.

[The student will evaluate problems and solutions in a text by identifying the problem and relevant issues, understanding the significance of the problem, and evaluating the effectiveness of the solution.] Depth of Knowledge:

• identifying the problem and relevant issues—1 • understanding the significance of the problem—2 • evaluating the effectiveness of the solution—3

8

Text Categories: literary, informational Text Content: Text includes a problem and a solution. Item Types: multiple choice, short constructed response, extended constructed response Example Items: 1. Multiple-choice items: Stem: What problem does the author introduce in the text? Correct Response: The correct response accurately identifies the problem

in the text. Distracters: Incorrect responses may describe events/action in the text that are not problems.

Depth of Knowledge: Level 1—Recall 2. Short constructed-response items:

Stem: Identify one problem that the author introduces in the text and explain the problem’s significance. Use information from the text to support your answer. Correct Response: (2-point) The student correctly identifies one problem and explains the problem’s significance by using information from the text.

Depth of Knowledge: Level 2—Procedural Skill 3. Extended constructed-response items:

Stem: Describe one problem that the author introduces in the text and the solution he/she proposes. Tell whether you think this solution will be effective. Explain your response by using information from the text as well as your own opinion as support. Correct Response: (4-point) The student correctly identifies one problem and its solution presented in the text and explains whether this solution will be effective using information from the text and personal opinion as support.

Depth of Knowledge: Level 2—Procedural Skill

9

Benchmark 1: Read, synthesize, and analyze text. Assessment Objective 2: Recognize and evaluate a writer’s position and analyze the writer’s support to determine if it is relevant, valid, and reliable. [The student will recognize and evaluate a writer’s position and analyze the writer’s support to determine if it is relevant, valid, and reliable.] Depth of Knowledge: 2 Text Categories: literary, informational Text Content: Text will include enough information so that the writer’s position can be determined and the support analyzed for relevance, validity, and reliability. Item Types: multiple choice, short constructed response, extended constructed response Example Items: 1. Multiple-choice items: Stem: With which statement would the author/writer most likely agree? Correct Response: The correct response is a statement with which the

author/writer would likely agree based on information in the text. Distracters: Incorrect responses include statements that are incongruous with the author/writer’s position or are statements irrelevant to the passage.

Depth of Knowledge: Level 2—Procedural Skill 2. Short constructed-response items:

Stem: Describe the author/writer’s position on (main idea/concept from text) and explain one way he/she supports this position. Use information from the text to support your answer. Correct Response: (2-point) The student accurately describes the author’s position and explains, by using information from the text, how the author supports his/her position.

Depth of Knowledge: Level 2—Procedural Skill 3. Extended constructed-response items (for a paired set of passages):

Stem: Describe an example that each author provides in support of his/her argument/position on (main idea/concept from text). Which author’s support seems stronger? Explain your answer by using information from each selection. Correct Response: (4-point) The student describes an example that each author provides in support of his/her position and explains which author’s support seems stronger by using information from each selection. Depth of Knowledge: Level 2—Procedural Skill

10

Benchmark 2: Demonstrate critical thinking skills and evaluate text. Assessment Objective 3: Critically interpret and evaluate experiences, literature, language, and ideas by making generalizations supported by specific references.

[The student will critically interpret and evaluate experiences, literature, language, and ideas by making generalizations supported by specific references.] Depth of Knowledge: 2

Text Categories: literary, informational, functional Text Content: Text will offer substantial information so that generalizations can be made. Item Types: multiple choice, short constructed response, extended constructed response Example Items: 1. Multiple-choice items:

Stem: Which best states the writer/author’s purpose in writing (title of text)? Correct Response: The correct response offers a logical purpose. Distracters: Incorrect responses offer purposes that are not substantiated

by the text. Depth of Knowledge: Level 2—Procedural Skill 2. Short constructed-response items:

Stem: Write an example of a point on which the author contradicts him/herself. Explain how this contradiction affects the message the author is trying to convey. Correct Response: (2-point) The student correctly identifies a point on which the author contradicts him/herself and explains how the contradiction affects the message the author is trying to convey.

Depth of Knowledge: Level 3—Strategic Thinking 3. Extended constructed-response items (for a paired set of passages):

Stem: Write one example that shows each writer/author’s bias about (subject from text). Explain why each writer/author’s bias affects the writer/author’s message. Use information from each text to support your answer. Correct Response: (4-point) The student correctly identifies one example that shows each writer/author’s bias about (subject from text) and explains, by using support from the text, why that bias affects the writer/author’s presentation of information and effectiveness of argument.

Depth of Knowledge: Level 3—Strategic Thinking

11

Benchmark 2: Demonstrate critical thinking skills and evaluate text. Assessment Objective 4: Identify critical questions that would lead to a broader understanding of a selection. [The student will identify critical questions or information that would lead to a broader understanding of a selection.] Depth of Knowledge: 2 Text Categories: literary, informational Text Content: The text prompts questions or a need for additional information that would lead to a broader understanding. Item Types: multiple choice Example Items: 1. Multiple-choice items:

Stem: Which resource would best help a reader understand the ideas/concepts/information presented in the text? Correct Response: The correct response is a logical and reasonable type or title of a resource that would offer information to better understand the text (e.g., a book titled Birds of the Northeast). Distracters: Incorrect responses include types or titles of resources that would be unlikely to offer additional information on the ideas/concepts presented in the text.

Depth of Knowledge: Level 1—Recall

12

Benchmark 2: Demonstrate critical thinking skills and evaluate text. Assessment Objective 5: Understand complex literary terms and find examples in text. [The student will demonstrate understanding of complex literary terms and sound devices by finding examples in text and/or explaining their significance to the text.] Depth of Knowledge: 1 Text Categories: literary Text Content:

For complex literary devices, literary text must include examples of one or more of the following:

1) allusion 2) dialog 3) flashback 4) foreshadowing 5) humor 6) exaggeration (hyperbole) 7) imagery 8) irony 9) metaphor 10) mood 11) oxymoron 12) personification 13) simile 14) symbolism

For sound devices, literary text must include examples of one or more of the following:

1) alliteration 2) assonance 3) consonance 4) onomatopoeia

Item Types: multiple choice, short constructed response, extended constructed response Example Items: 1. Multiple-choice items:

Stem: Which of the following (phrases/sentences) from the text is an example of irony? Correct Response: The correct response is an example of irony from the text. Distracters: Incorrect responses include phrases/sentences from the text that are examples of literary devices other than irony and/or are not examples of any type of literary device.

Depth of Knowledge: Level 1—Recall

13

2. Short constructed-response items:

Stem: Write one example of alliteration from the text and explain how it helps the writer convey his/her ideas.

Correct Response: (2-point) The student writes one example of alliteration from the text and explains how it helps the writer convey his/her ideas.

Depth of Knowledge: Level 2—Procedural Skill 3. Extended constructed-response items:

Stem: Give one example of symbolism from the story and one from the poem. Use information from the story and the poem to explain how each example contributes to each overall work. Correct Response: (4-point) The student gives one example of symbolism from the story and one from the poem and uses information from the story and the poem to explain how each example contributes to the overall work.


14

Benchmark 2: Demonstrate critical thinking skills and evaluate text. Assessment Objective 6: Read critically and independently to draw conclusions, make inferences, and make predictions from texts. [The student will read critically and independently to draw conclusions, make inferences, and make predictions from text.] Depth of Knowledge: 2 Text Categories: literary, informational, functional Text Content: The text must be sufficiently developed so that the reader can reasonably draw conclusions, make inferences, or make predictions based on information stated or implied in the text. Item Types: multiple choice, short constructed response Example Items: 1. Multiple-choice items:

Stem: Based on the information in the (title of table in text), what will the (e.g., sales figures for Moore Manufacturing be next year)? Correct Response: The correct response is a reasonable and logical prediction based on information in the table. Distracters: Incorrect responses include unreasonable predictions based on information in the table or predictions that cannot be made based on information in the table.


Stem: What will (name of character) most likely do next? Support your prediction with information from the text. Correct Response: (2-point) The student makes a reasonable prediction based on the text and uses information/examples from the text to support the prediction.


15

Benchmark 2: Demonstrate critical thinking skills and evaluate text. Assessment Objective 7: Independently apply the reading process and strategies to a variety of texts and use the defining features and structures of those works to understand main elements, perspective, and style. [The student will apply the reading process and strategies to a variety of texts and use the defining text features and structures to understand main elements, perspective, and style.] Depth of Knowledge: 1 Text Categories: literary, informational, functional Text Content: Literary text must include one or more of the following:

1) plot: episodic development which may include the development and resolution of conflict

2) characters: people, animals, and/or inanimate objects that are personified 3) setting: time(s) and location(s) 4) theme: message(s) or central idea(s) 5) point of view: first-person, third-person, omniscient 6) conflict: the problem central to the development of the story 7) resolution: follows the climax and provides sense of wholeness,

completeness, and finality to the storyline Informational and functional text must include one or more of the following:

1) appendix 2) captions 3) glossary 4) graphic features 5) index 6) preface 7) table of contents 8) titles, headings, or other text dividers

16

Item Types: multiple choice, short constructed response, extended constructed response Example Items—Literary: 1. Multiple-choice items: Stem: How does (name of character)’s decision to (action taken by

character) affect the outcome in the passage? Correct Response: The correct response accurately states how the

character’s decision affects the outcome in the passage. Distracters: Incorrect responses include information not substantiated by the text, information unrelated to the character or outcome, and/or faulty reasoning.

Depth of Knowledge: Level 2—Prediction 2. Short constructed-response items: Stem: From what point of view is (title of passage) told? Use one example

from the text to support your answer. Correct Response: (2-point) The student correctly names the point of view

in the story and provides reasonable/logical support from the text. Depth of Knowledge: Level 1—Recall 3. Extended constructed-response items (for a paired passage set): Stem: Describe the setting of the poem and the story. Tell how the poem

might change if the setting were (change time/place) instead. Then tell how the story might change if the setting were (change time/place). Use information from the poem and story to support your answer. Correct Response: (4-point) The student correctly describes the setting of the poem and the setting of the story and explains how the poem might change and how the story might change if the settings were different.

Depth of Knowledge: Level 2—Procedural Skill Example Items—Informational or Functional: 1. Multiple-choice items:

Stem: How does the author’s use of (name graphic feature) help convey the message of the passage? Correct Response: The correct response accurately states how the graphic feature helps convey the message of the passage. Distracters: Incorrect responses may describe how another text feature helps convey the message.


17

2. Short constructed-response items: Stem: What is the purpose of (name graphic element) in this passage? What idea in the passage does it help clarify? Correct Response: (2-point) The student correctly describes the purpose of (name of graphic element) in the passage and explains the idea in the passage the element helps clarify.

Depth of Knowledge: Level 1—Recall 3. Extended constructed-response items:

Stem: Analyze the data in (title of table provided in text). Describe how the data are organized and provide three examples from the table that are helpful for understanding the information in the text. Correct Response: (4-point) The student accurately describes the organization of the data in the table and provides three examples from the table that are helpful for understanding the information in the text.


18

Benchmark 3: Apply knowledge of the reading process. Assessment Objective 8: Analyze the ideas of others by identifying the ways in which writers:

• introduce and develop a main idea • choose and incorporate significant, supporting, relevant details.

[The student will analyze the ideas of others by identifying the ways in which writers introduce and develop the main idea and select and incorporate significant, relevant supporting details.] Depth of Knowledge: 2 Text Categories: literary, informational Text Content: The text must include ample development of main ideas and include details that support the author’s development of the main idea. Item Types: multiple choice, short constructed response Example Items: 1. Multiple-choice items:

Stem: Which phrase/sentence best summarizes the writer/author’s main idea? Correct Response: The correct response accurately summarizes the author’s main idea. Distracters: Incorrect responses may include details from the text or inaccurately summarize the author’s main idea.

Depth of Knowledge: Level 2—Procedural Skill 2. Short constructed-response items:

Stem: (Title of text) describes (main idea from text). Write one supporting detail the writer/author uses to support his/her main point and describe how this detail contributes to the development of the main idea in the text. Correct Response: (2-point) The student correctly identifies one supporting detail from the text and offers reasonable explanation for how the selected detail contributes to the development of the main idea in the text.


19

Benchmark 3: Apply knowledge of the reading process Assessment Objective 9: Analyze the ideas of others by identifying the ways in which writers:

• relate the structure/organization to the ideas • use effective word choice as a basis for coherence (logical connectors) • achieve a sense of completeness and closure.

[The student will analyze the ideas of others by identifying the ways in which writers relate the structure/organization to the ideas, use effective word choice as a basis for coherence (including effective transitions), and achieve a sense of completeness and closure (including effective conclusion).] Depth of Knowledge: 2 Text Categories: literary, informational, functional Text Content: The text may include explicit or implicit structure and organization of ideas, text clues that allow one idea to progress to the next, and a reasonable conclusion. Item Types: multiple choice, short constructed response, extended constructed response Example Items: 1. Multiple-choice items: Stem: How does the writer/author organize ideas in the text? Correct Response: The correct response accurately names or describes the

organizational structure used by the writer/author (e.g. cause/effect, chronological, descending importance). Distracters: Incorrect responses may name or describe an organizational structure not used by the writer/author or loosely refer to the text without naming or describing the structure/organization of the text.


Stem: Identify one transition the author uses in the text and explain why the transition is effective. Correct Response: (2-point) The student writes one use of transition from the text and explains why the transition is effective.


20

3. Extended constructed-response items: Stem: What does the author conclude about (idea from text)? Explain how the author’s conclusion is supported in the text by citing three examples from the text. Correct Response: (4-point) The student writes the author’s conclusion and explains, by using three examples from the text, how the information from the text supports the conclusion.


21

Benchmark 3: Apply knowledge of the reading process. Assessment Objective 10: Differentiate among literal, figurative, and connotative meanings. [The student will differentiate among literal, figurative, and connotative meanings and/or explain the meaning.] Depth of Knowledge: 1 Text Categories: literary, informational, functional Text Content: The text must include (in the form of word, phrase, or expression) at least one example where sufficient context clues support meaning. Item Types: multiple choice Example Items:

1. Multiple-choice items: Stem: Why does the writer use “(word/phrase)” to describe (event/thing in text)? Correct Response: The correct response is a reasonable explanation of why the writer uses “(word/phrase)” to describe (event/thing in text). Distracters: Incorrect responses include interpretations that are illogical/unreasonable or not substantiated by the text.


22

Documents

Part One: Reading Test Specifications One: Reading Test Specifications I. TEST PURPOSE The purpose of this test is to measure New Mexico eleventh-grade students’ level of proficiency