Upload
marylou-dennis
View
216
Download
0
Embed Size (px)
Citation preview
Assessment
“We assess students not merely to evaluate them, but to improve the entire process of teaching and
learning.”- Douglas B. Reeves, Making Standards Work
Designing STAAR-Like Quality Assessments
Learning Objective
Participants will be able to create more rigorous assessments by analyzing the content and cognitive level of state released test questions.
Why Assessment Training?
Teacher and campuses are overwhelmed with the number of assessments they have to give
How do we target High Priority TEKS? How is a good unit assessment
constructed? Individual teacher assessments should
follow the guidelines of quality assessment
Agenda
Six Steps to Creating Quality Assessments
1. Data2. Content3. Processes4. Review Examples5. Item & Test Leveling6. Item Design
Look at the Data
Review the District Learning Profile.
Which SEs are the lowest?
Which SEs had the most questions?
Analyzing the content , concepts, processes, and skills that will be
assessed(Cognitive and Procedural Knowledge)
TEKS and Assessment: Things to Remember
The wording of the standard tells us: WHAT CONTENT will be assessed on STAAR
and AT WHAT LEVEL the standard will be assessed
on STAAR
Cognitive and Content Expectations Content
The content items for which students must demonstrate understanding at the appropriate cognitive level in order to adequately meet the standard.
Cognitive The level at which students are expected to perform in
order to adequately meet the standard. Determined by the verbs used in BOTH the Knowledge
and Skills statements and the Student Expectations
What Should Students be able to DO ?
Look at the cognitive level of the verb
• If the cognitive level of the SE is UNDERSTAND, what does that mean students have to be able to do?
Understanding: Constructing meaning from different types of
functions be they written or graphic messages activities like
interpreting, exemplifying, classifying, summarizing, inferring,
comparing, and explaining.
ELAR – Grade 8
8.4 Comprehension of Literary Text/Poetry. Students understand, make inferences and draw conclusions about the structure and elements of poetry and provide evidence from the text to support their understanding. The student is expected to:– (A) compare and contrast the relationship
between the purpose and characteristics of different poetic forms (e.g., epic poetry, lyric poetry). Supporting Standard
ELAR – Grade 8
Figure 19 Reading/Comprehension Skills.Students use a flexible range of metacognitive reading skills in both assigned and independent reading to understand an author’s message. Students will continue to apply earlier standards with greater depth in increasingly more complex texts as they become self-directed, critical readers. The student is expected to: – (D) make complex inferences about text
and use textual evidence to support understanding
Bundling and Dual Coding
Brain Research and Dual Coding
Application of Knowledge and Building Schema The brain learns new knowledge (content) by
attaching that knowledge to existing schema The brain builds schema by applying (do)
conceptual and content knowledge in a variety of novel ways
You can most effectively test conceptual knowledge through application questions
STAAR and Dual Coding
Math = 75% of items Science = 40% of items Social Studies = 30% English = 60%???
Figure 19
As we review the following examples, think about the implications of test and item design
ELAR: Dual Coding with Figure 19
ELAR: Dual Coding with Figure 19
Your Turn: ACTIVITY
Look at the STAAR released items in your handout.
Identify what content is being assessed. Identify the skill being assessed. Identify how the student has to apply
content with the selected skill. Discuss with your group.
STAAR Overview: Generalizations
Verbiage in the questions is complex and reflective of the TEKS
Supporting Standards REALLY are being tested! More questions at a higher level Inference from stimulus; no direct answers (no clues or
cues) Complex distracters - 4 viable answer choices Dual coding is prevalent Greater depth of content knowledge required to answer
each question
Creating an Assessment Blueprint
Step 5:Determining Cognitive and Procedural Difficulty Levels
Aligning Item and Test Difficulty to STAAR
Determining Difficulty Level
Cognitive Difficulty (verbs) EASY
Remember Understand
MEDIUM Apply Analyze
HARD Evaluate Create
Determining Difficulty Level
Procedural DifficultyThis is basically: How many mental
processing steps does the student have to go through to answer the question?
The greater the number of processing steps the higher the difficulty level
Determining Difficulty Level
Procedural Difficulty EASY
The item includes only the stem and the answer choices MEDIUM
The item includes a graphic, short reading selection, map etc. (Stimulus piece) The student only has to interpret the stimulus or pull information from it to select the correct answer.
HARD The item includes a graphic, short reading selection, map
etc. (Stimulus piece) The student has to infer, analyze, summarize, etc. and apply that to the stem or answer choices to select the correct answer.
Your Turn: Determining Item Difficulty
1. Look at the following items
2. Identify the cognitive difficulty of the item
3. Identify the procedural difficulty of the item
Determining Item Difficulty
Determining Item Difficulty
Item Writing Checklist
Demonstration of Learnig
After having determined the cognitive and procedural difficulty of a test item, learners will rewrite the question so that it is either up a level or down a level.
Creating an Assessment Blueprint
Identify the process/skills SE that you are pairing with the content SE
Write these on the Blueprint Use a variety of processes and skills on the
test and throughout the year.
Design Your Test So….
How many items per SE? Cognitive Level- Make 70% of the test easy-
medium and 30% difficult. Procedural Level- Make 60% easy
procedural and 40% difficult. Use same vocabulary throughout.
Next Steps: What are the implications of today’s learning for your campus?
What are things you will change on your assessments?
What are things you are still pondering?
What is something you want to know more about?
Bonus Slides: Examples of Higher Level Stems Which is an example of…
Who would most likely have written (asked, said…) followed by a
quote
Analogies Fill in the missing part of this graphic organizer
Which of the following does not belong?
What is the best category for the following?
Give a series of clues to the name of a person, place, etc. (riddle format)
Cloze passage
Hypothetical situation – analysis
Sequencing by Roman numeral
Who would have been helped by (law, invention, organization –
hypothetical)
Use a variety of stimuli
Textbox of primary source material Graphs and Charts Pictures or illustrations Spoke Diagrams Multiple visuals: comparison maps, charts Political Cartoons Time lines Flow Charts Graphic designs Headlines Speaker Questions
Qualities of a Good Multiple-Choice Item Effectively written stems Focused on one idea Clear and concise Phrased as a direct question Phrased positively Has only one best and correct answer Contains options that are plausible, but incorrect Has four choices that are homogeneous, parallel
in structure, and logical
Qualities of a BAD Multiple-Choice Item Tests trivial information Contains unnecessary/confusing information in the
stem or options Is tricky or cute Gives cues or clues to the correct answer Does not have plausible answers Poses a question for which many defensible
answers are possible Contains a bias toward or against a group of
individuals
Develop plausible distractors
Plausible distracters include… common misconceptions statements that sound logical but that are not based on the
material given in the stimulus and question common or familiar phrases statements that are true but that do not answer the
question
Be sure there is not more than one plausible answer.
Organization of Answer Choices Alphabetical Ascending or descending order Shortest to longest Same order as presented in stimulus