Embed Size (px)
Citation preview
Understanding Test Blueprints and Their Impact on Student Growth Holly Lavender and Cathryn Shaw
Learning Targets
1. Understand the definition and purpose of test blueprints
2. Know how to build a test blueprint
3. Understand how assessment blueprints connect to other initiatives
The Purpose of Test Blueprints
1
The Purpose of Test Blueprints • Define the parameters of an assessment before you
begin the creation process. • Review a locally- or vendor-developed assessment for
alignment to learning expectations, instruction, and cognitive complexity.
• Confirm in advance the information and knowledge you are planning to assess, which can guide instructional activities.
• Share blueprints with students to help make learning targets clear.
A Blueprint is a TOOL to ensure: alignment of assessment to standards
- content - depth of knowledge
alignment of assessment to instruction adequate “stretch” in assessment
Building a Test Blueprint
2
Where to Begin?
The End
Design the blueprint Create the assessment Plan for Instruc7on
Sample Test Blueprint LEARNING PRIORITIES ASSESSMENT
Learning Priori5es DOK Level 1-‐4
MethodSR, CR, P
DOK 1: Recall or Produce
DOK 2: Basic Applica5on
DOK 3: Strategic Thinking
DOK 4: Extended Thinking
Total Points
Weight
1
2
3
4
5
Total Points
Weight
IDENTIFIED STANDARDS
Learning Priorities Selection
Unit Test
Quarterly Assessment
SLO Pre/Post-Assessment
Identify Learning Priorities
Longevity
Leverage
Levels
Learning Priorities
Ainsworth, L. (2003). Power standards: Iden7fying the standards that maHer the most. Englewood, CO: Lead and Learn Press.
Learning Priorities: Longevity
• The knowledge and skills that are important today, tomorrow, and in the future.
• Example: Reading Comprehension Longevity
Learning Priorities: Leverage
• The knowledge and skills that are used in more than one content area at a specific grade level (horizontal).
• Example: Creating and analyzing charts and graphs
Leverage
Learning Priorities: Levels • The knowledge and
skills that are key to success in subsequent grade levels or courses.
• Example: Strong understanding of fractions to prepare for work with rational numbers
Levels
Learning Priorities
• Introduced early • Reinforced often • Multiple opportunities
for learning Learning Priorities
Identify Learning Priorities
Clear Focus
Improved Validity
SLO Checklist
Sample Test Blueprint LEARNING PRIORITIES ASSESSMENT
Learning Priori5es DOK Level 1-‐4
MethodSR, CR, P
DOK 1: Recall or Produce
DOK 2: Basic Applica5on
DOK 3: Strategic Thinking
DOK 4: Extended Thinking
Total Points
Weight
1. Describe how characters respond to challenges
2. Determine the lesson or morale of a story
3. Demonstrate understanding of key details in a text
4. Iden5fy differences in point of view
5. Compare and contrast two or more versions of a story
Total Points
Weight
IDENTIFIED STANDARDS
Alignment Content: Are the expectations consistent for both curriculum and assessments? Depth of Knowledge (Rigor): Are the cognitive demands of curricular aims and assessments the same?
– For individual items? – For the assessment as a whole?
Cognitive Complexity
• Common Core State Standards and Ohio’s New Learning Standards call for a higher level of cognitive complexity than previous assessments could measure
©2013. Battelle for Kids. All Rights Reserved.
Hess’ Cognitive Rigor Matrix
Bloom’s Taxonomy Webb’s Depth of Knowledge
DoK Level 1 Routine Thinking
Recall and Reproduction
Describe Measure Define Calculate Locate Memorize
Jen Jones (2013); helloliteracy.blogspot.com
DoK Level 2 Conceptual Thinking
Basic Application and Reasoning
Classify Categorize Summarize Graph Infer Interpret
Jen Jones (2013); helloliteracy.blogspot.com
DoK Level 3 Strategic Thinking
Higher Level Application
Discuss Revise
Investigate Assess Critique Argue/Justify
* Usually more than one right answer Jen Jones (2013); helloliteracy.blogspot.com
DoK Level 4 Extended Thinking
Real World Application
Synthesize Create Design Plan Develop Propose
* Usually over an extended period of time Jen Jones (2013); helloliteracy.blogspot.com
Is This Item Aligned? K.G2: Correctly name shapes regardless of their orientations or overall size. (CCSS)
• Show me the triangles. • Show me the pentagons. • Show me the cylinders.
Is This Item Aligned?
LS 5.2: Food webs can be used to identify the relationships among producers, consumers and decomposers in an ecosystem. (ONLS)
Is This Item Aligned?
Write the definitions for the following words:
1. Figure ______________________
2. Determine ______________________
3. Abode ______________________
3.L.4a: Use sentence-level context as a clue to the meaning of a word or phrase. (CCSS)
Is This Item Aligned?
Analyze the causes and consequences of major political, economic and social developments of the 1930’s with emphasis on the Dust Bowl. (OACS)
Is This Item Aligned?
Create a mixed media fantasy landscape that conveys a story, incorporating a well-known character from children’s fiction.
VA 6.4: Connect selected ideas, concepts and processes used in visual art with those used in other academic disciplines. VA 6.5: Use observations, life experiences and imagination as sources for visual symbols, images and creative expression.
Sample Test Blueprint LEARNING PRIORITIES ASSESSMENT
Learning Priori5es DOK Level 1-‐4
MethodSR, CR, P
DOK 1: Recall or Produce
DOK 2: Basic Applica5on
DOK 3: Strategic Thinking
DOK 4: Extended Thinking
Total Points
Weight
1. Describe how characters respond to challenges
2
2. Determine the lesson or morale of a story
2
3. Demonstrate understanding of key details in a text
1
4. Iden5fy differences in point of view
3
5. Compare and contrast two or more versions of a story
3
Total Points
Weight
IDENTIFIED STANDARDS
Determine Weight
What portion of the total test will be
assigned to each of the identified standards?
Sample Test Blueprint LEARNING PRIORITIES ASSESSMENT
Learning Priori5es DOK Level 1-‐4
MethodSR, CR, P
DOK 1: Recall or Produce
DOK 2: Basic Applica5on
DOK 3: Strategic Thinking
DOK 4: Extended Thinking
Total Points
Weight
1. Describe how characters respond to challenges
2
2. Determine the lesson or morale of a story
2
3. Demonstrate understanding of key details in a text
1
4. Iden5fy differences in point of view
3
5. Compare and contrast two or more versions of a story
3
Total Points
Weight
Weight
How is Weight Determined?
Learning targets that are broad in scope
The importance of the standard
Standards that are emphasized in classroom instruction
Why should I determine weight?
Ensures adequate coverage of content
Ensures emphasis is placed where intended
Sample Test Blueprint LEARNING PRIORITIES ASSESSMENT
Learning Priori5es DOK Level 1-‐4
MethodSR, CR, P
DOK 1: Recall or Produce
DOK 2: Basic Applica5on
DOK 3: Strategic Thinking
DOK 4: Extended Thinking
Total Points
Weight
1. Describe how characters respond to challenges
2 5
points 20%
2. Determine the lesson or morale of a story
2 3
points 12%
3. Demonstrate understanding of key details in a text
1 3
points 12%
4. Iden5fy differences in point of view
3 6
points 24%
5. Compare and contrast two or more versions of a story
3 8
points 32%
Total Points
Weight
Weight
Assessment Method
Many factors can affect the test format
How Much Time is Available?
Time is a factor when it comes to preparing and scoring tests
Consider collaborating with others to write and/or evaluate test items
What is to be Measured?
Choose the most appropriate format for
measuring the cognitive objectives of the test
FIP refers to this as the “target-method match”
Which Assessment Method? Identify the eight major planets in the solar system.
Selected Response
Constructed Response
Which Assessment Method? Make a line plot to display a data set of measurements in fractions of a unit (1/2, 1/4, 1/8).
Performance
Which Assessment Method? Write arguments to support claims with clear reasons and relevant evidence.
Constructed Response
Which Assessment Method? Work effectively in a group to complete a task or solve a problem for which the group is held accountable.
Performance
Sample Test Blueprint LEARNING PRIORITIES ASSESSMENT
Learning Priori5es DOK Level 1-‐4
MethodSR, CR, P
DOK 1: Recall or Produce
DOK 2: Basic Applica5on
DOK 3: Strategic Thinking
DOK 4: Extended Thinking
Total Points
Weight
1. Describe how characters respond to challenges
2
SR & CR 5
points 20%
2. Determine the lesson or morale of a story
2
CR 3
points 12%
3. Demonstrate understanding of key details in a text
1
SR 3
points 12%
4. Iden5fy differences in point of view
3
P 6
points 24%
5. Compare and contrast two or more versions of a story
3
P 8
points 32%
Total Points
Weight
Method
Stretch
Assessments should demonstrate stretch Include both high and low-achieving students
Sample Test Blueprint LEARNING PRIORITIES ASSESSMENT
Learning Priori5es DOK Level 1-‐4
MethodSR, CR, P
DOK 1: Recall or Produce
DOK 2: Basic Applica5on
DOK 3: Strategic Thinking
DOK 4: Extended Thinking
Total Points
Weight
1. Describe how characters respond to challenges
2
SR & CR 5
points 20%
2. Determine the lesson or morale of a story
2
CR 3
points 12%
3. Demonstrate understanding of key details in a text
1
SR 3
points 12%
4. Iden5fy differences in point of view
3
P 6
points 24%
5. Compare and contrast two or more versions of a story
3
P 8
points 32%
Total Points
Weight
Cogni5ve Complexity
Options to Demonstrate Stretch
Include items that assess content from adjacent grade levels
Include items that assess on-level standards at a greater or lesser depth
How Much Stretch?
At least 50% of
assessment items should be at or
above the target DOK level
Sample Test Blueprint LEARNING PRIORITIES ASSESSMENT
Learning Priori5es DOK Level 1-‐4
MethodSR, CR, P
DOK 1: Recall or Produce
DOK 2: Basic Applica5on
DOK 3: Strategic Thinking
DOK 4: Extended Thinking
Total Points
Weight
1. Describe how characters respond to challenges
2
SR & CR
2 points
3 points 5
points 20%
2. Determine the lesson or morale of a story
2
CR
1 point
2 points 3
points 12%
3. Demonstrate understanding of key details in a text
1
SR
3 points 3
points 12%
4. Iden5fy differences in point of view
3
P 2 points 4 points 6
points 24%
5. Compare and contrast two or more versions of a story
3
P
8 points 8
points 32%
Total Points 6 points 7 points 12 points 0 points
Weight 24% 28% 48% 0%
Cogni5ve Complexity
Rigor
Putting It All Together
3
Remember…. We are measuring the effects of teaching
Assessment must reflect what is taught
Teaching must reflect what is to be assessed
Remember… Look for and evaluate items from banks and collaborate with others to write and evaluate high-quality items Use the data to improve instruction and Increase student learning. The quality of the data is only as good as the quality of the assessment!
Social Media
@OHEducation
ohio-department-of-education
Ohio Families and Education Ohio Teachers’ Homeroom
OhioEdDept
storify.com/ohioEdDept
