Embed Size (px)
Citation preview
Washington Educational
Research AssociationConference
Washington Educational
Research AssociationConferenceBeth DorrBeth Dorr
Reading Assessment SpecialistReading Assessment [email protected]@k12.wa.us
Beth DorrBeth DorrReading Assessment SpecialistReading Assessment [email protected]@k12.wa.us
WERA WERA AgendaAgenda
• WASL Development, Test Matrix, and Targets
• Overview of 2006 Reading Performance Grades 3, 4, and 5
• General Guidelines for responses and scoring
• Introduction to the 2006 Reading Released Items
• Q & A
Reading Passage Selection and Copyrights
Research
Item Pilot Testing
Item Reviews for Content by OSPI Item Writing Training
and Item Writing
Pilot Rangefinding Pilot Scoring Training and Pilot Scoring
Operational Rangefinding
Operational Testing
Operational ScoringTeacher Scoring
Data Review
Standard Setting(First Year with Periodic
Reviews)
Develop Item and Test Specifications Passage Bias/Sensitivity
Reviews
Item Reviews for Content & Bias/Sensitivity by
Committee
We Want You to Participate!
To apply for Reading WASL Committees visit:
www.k12.wa.us/assessment/WASL/ReadingAssessment.aspx
Test Characteristics Matrix• What makes up each Operational Reading WASL?
(Refer to the Test Characteristics Matrix Handout)
Test MapTEST CHARACTERISTICS
Test Characteristics Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 High School
# of total items (MC, SA, ER) on test
28 29 31 32 34 35 37
# of total points possible on test 40 42 44 46 48 50 52
Number of operational passages on each test
6 6 6 6 6 6 6
Number of strands assessed per test
4 4 4 6 6 6 6
Minimum # of points per strand per test
6 6 6 6 6 6 6
# of multiple choice (MC) items on test
20 20 22 22 24 24 26
# of short answers (SA) on test 6 7 7 8 8 9 9
#of extended responses (ER) on test
2 2 2 2 2 2 2
Percent of multiple choice in total score
50% 48% 50% 48% 50% 48% 50%
Percent of constructed response in total score (short answer and extended response)*
50% 52% 50% 52% 50% 52% 50%
Total Number of Embedded Items 7 7 7 7 7 7 7
Strands and Targets
• The blue prints of the Reading WASL(Refer to the Strands and Targets Handout)
Grades 3, 4, 5 Strands and TargetsGRADES 3, 4, AND 5 READING TARGETS ASSESSED ON THE READING WASL
and Their Corresponding Grade Level Expectations
Literary Targets Informational Targets
LC01 Demonstrate understanding of theme or message and supporting details (Reading GLE 2.1.3)
IC11 Demonstrate understanding of major ideas and supporting details (Reading GLE 2.1.3)
LC02 Summarize with evidence from the reading (Reading GLE 2.1.7)
IC12 Summarize with evidence from the reading (Reading GLE 2.1.7)
LC03 Make inferences or predictions based on the reading (Reading GLE 2.1.5)
IC13 Make inferences or predictions based on the reading (Reading GLE 2.1.5)
LC04 Interpret vocabulary critical to the meaning of the text (Reading GLE 1.3.2)
IC14 Interpret vocabulary critical to the meaning of the text (Reading GLE 1.3.2)
LC05 Order steps, sequence, and/or events from the reading (Reading GLE 2.2.1)
IC15 Order steps, sequence, and/or process from the reading (Reading GLE 2.2.1)
LA06 Demonstrate understanding of literary elements (genres; story elements such as plot, character, setting; stylistic devices; problem/resolution) and graphic elements/illustrations
(Reading GLEs 2.2.2, 2.2.3, 2.3.3)
IA16 Demonstrate understanding of text features (titles, headings, and other information divisions, table of contents, captions) and graphic features
(Reading GLE 2.2.2)
LA07 Compare and contrast elements of text (Reading GLEs 2.3.1, 2.4.5, 2.4.6)
IA17 Compare and contrast information presented in text (Reading GLEs 2.3.1, 2.4.5, 2.4.6)
LA08 Make connections (cause and effect) within a text (Reading GLE 2.3.1)
IA18 Make connections (cause and effect) within a text (Reading GLE 2.3.1)
LA09 Analyze author's purpose and evaluate effectiveness for different audiences
(Reading GLEs 2.4.2, 2.4.3, 2.4.4)
IA19 Analyze author's purpose (including distinguishing between fact and opinion) and evaluate effectiveness for different audiences (Reading GLEs 2.4.2, 2.4.3, 2.4.4)
LA10 Extend information beyond text—apply information, give a response to reading, express insight gained from reading
(Reading GLEs 2.4.1, 2.4.5)
IA20 Extend information beyond text—apply information, give a response to reading, express insight gained from reading (Reading GLEs 2.4.1, 2.4.5)
N=3,899 N=10,448 N=32,239 N=26,748
Percent of Students at Each Level
46.5
38.6
44.5
41.4
40.0
31.1
35.9
27.9
27.0
21.5
22.4
17.7
15.6
16.836.0
28.3
25.3
19.4
14.0
27.4
27.0
31.2
34.6
11.4
6.4
4.7
4.9
5.2
4.9
5.1
7.3
7.5
42.7
43.3
43.4
24.0
1997 1998
1999 2000 2001 2002 2003 2004 2005
Did not meet standard Exceeded standardMet standard
Percentages do not total 100 as chart does not include untested students
Grade 4 Reading
Grades 3, 4, & 5 2006
Percent of Students at Each level Grades 3, 4, & 5 for 2006
34.5
46.4
37.3
33.4
34.4
38.717.3
14.6
20.79.8
3.4
5.5
1 2 3 4
3
4
5
2006
58,44413,550
14.6%3.4% 46.4% 34.4%
1997 36.0%11.4% 31.1% 16.8%
Did not meet standard Exceeded standardMet standard
29,626 29,134
Grade 4 ReadingMovement Across Achievement Levels
Percent of Students at Each Level* Percentages do not total 100 as chart does not include untested students
Grade 3
• High percent of blanks for level 1 students.• When presented with a choice of words to describe a character, students performed
low. (LA06)** Explanation: Any of these words could describe the character in the story.**
Choose the word that best describes the character in the story.Adj. Adj. Adj.
Use two details from the story to support your choice.• Author’s Purpose is difficult. (LA09/IA19)
Grade 4• Successful at word choice for (LA06).
Successful at Author’s Purpose.• Low areas
Cause/Effect (LA08)Compare/Contrast (LA07) especially in informational passages.
Grade 5• Struggle with:
Figurative LanguageSummaries and author’s purpose for level 1 student
Summarizing Students Who Met Standard on the Grade 4 Reading WASL
• Can respond to multiple choice and constructed response items for a variety of genres
• Use clear details from the text. Students do not use “vague” answers or simply repeat the question
• Can correctly select most of the multiple choice responses and cite appropriate text-based evidence for constructed response items
• Are familiar with reading assessment vocabulary (summary, prediction, compare and contrast, author’s purpose)
Summarizing Students Who Met Standard on the Grade 4 Reading WASL
• Refer to the text when answering both multiple choice and constructed response items.
• They know that evidence from the text is something they can “put their finger on” in the selection.
• Cite requested number of pieces of text-evidence in their responses (i.e., if asked to provide two differences, they provide two or more differences).
Constructed response items will follow these general guidelines:
• Short-answer (2 pts. with 9 lines) and extended-response (4pts. with 18 lines)
• Item-specific scoring guide is developed for each short-answer and extended-response item
• Extended-response items generally require longer, more detailed responses providing more evidence, information, or examples
• When an item has two parts, the parts sometimes appear separately (scaffolded), with lines following each part. This encourages students to notice and answer both parts of the item.
• Note: With paired passages, items follow each passage. The items requiring comparison between the passages appear last, after both passages. There should always be more than one item that compares the two passages. Constructed-response items should be asked about individual passages in addition to the passages as a pair.
Multiple choice items will follow these general guidelines:
• Each multiple-choice item contains three answer options, only one of which is correct.
• The choices are approximately the same length, have the same format, and are syntactically and semantically parallel; students should not be able to rule out a wrong answer or identify a correct response simply by virtue of its looking or sounding different.
• Distracters adopt the language and sense of the material in the selection so that students must think their way to the correct answer rather than simply identify incorrect responses by virtue of a distracter's obviously inappropriate nature.
• Distracters should always be plausible (but of course incorrect) in the context of the selection.
• To the greatest extent possible, no item or response choice "clues" the answer to any other item.
Scoring
• Anchor, Practice, and Qualifying sets are used to train in scoring.
• MC=1, SA=2, ER=4
• Cut Score
• Rubrics– Summarizing – Bullet points vs. Letters
Released Item Documents
• Items and Passages that were previously used on the 2006 Operational test, released to the field for teacher use.
• Released from 3, 5, 6 & 8
• Item-specific scoring guides, student responses, and annotations for these responses.
• The items were scored by teachers in the summer of 2006
• Contains school/district/state data table with state-level data already in place
Involving Students in Assessment
Have students complete the items Train students on the sample
student responses, then have students score their own response using the scoring guide (rubric)
Open discussion “What do the results tell us?”
Mini-lessons to follow up in area of concern
Train student to use WASL stems (questions)
How Can We Help You?
• OSPI @ www.k12.wa.us– Reading– Assessment
• Committee Application
• Support
[email protected] or 360-725-6442
What can I do for YOU? Website: www.k12.wa.us
Assessment
Reading Assessment
Click on Application
Click drop-down menu for all of the
dates
