Upload
ciara-conrad
View
40
Download
4
Tags:
Embed Size (px)
DESCRIPTION
Tools for Supporting Increased Rigor & High-Quality Assessment and Instruction. Verona Public Schools Karin K. Hess, Ed.D. www.karin-hess.com. Track your reflections as we work…. Ways I am refining my thinking about DOK/rigor… ? ?. - PowerPoint PPT Presentation
Citation preview
Tools for Supporting Increased Rigor & High-Quality Assessment
and Instruction
Verona Public Schools
Karin K. Hess, Ed.D.www.karin-hess.com
Track your reflections as we work…
Ways I am refining my thinking about DOK/rigor…•?•?
Scaffolding strategies for getting students to deeper thinking…•?•?
Before we begin…
• Take a minute to jot down words/phrases that come to mind when you think of “cognitive rigor” as it relates to instruction, learning, and/or assessment.
Let’s apply your rigor definitions
Your class has just read some version of Little Red Riding Hood.
• What is a basic comprehension question you might ask?
• What is a more rigorous question you might ask?
The Hess Cognitive Rigor Matrix integrates Bloom + Webb
Different states/schools/teachers use different models to describe cognitive rigor. Each addresses something different.
• Bloom – What type of thinking (verbs) is needed to complete a task?
• Webb – How deeply do you have to understand the content to successfully interact with it? How complex is the content?
Merging Bloom + Webb:
The thinking behind the development of the Hess Cognitive Rigor Matrix …
Bloom’s Taxonomy [1956 ] & Bloom’s Cognitive Process Dimensions [2001]
Knowledge -- Define, duplicate, label, list, name, order, recognize, relate, recall
Remember Retrieve knowledge from long-term memory, recognize, recall, locate, identify
Comprehension -- Classify, describe, discuss, explain, express, identify, indicate, locate, recognize, report, review, select, translate
Understand -- Construct meaning, clarify, paraphrase, represent, translate, illustrate, give examples, classify, categorize, summarize, generalize, predict…
Application -- Apply, choose, demonstrate, dramatize, employ, illustrate, interpret, practice, write
Apply -- Carry out or use a procedure in a given situation; carry out or use
/apply to an unfamiliar task Analysis -- Analyze, appraise, explain calculate, categorize, compare, criticize, discriminate, examine
Analyze -- Break into constituent
parts, determine how parts relate Synthesis -- Rearrange, assemble, collect, compose, create, design, develop, formulate, manage, write
Evaluate -- Make judgments based on criteria, check, detect inconsistencies/fallacies, critique
Evaluation -- Appraise, argue, assess, choose, compare, defend, estimate, explain, judge, predict, rate, core, select, support, value
Create -- Put elements together to form a coherent whole, reorganize elements into new patterns/ structures
Webb’s Depth-of-Knowledge Levels• DOK-1 – Recall & Reproduction - Recall of a fact, term, principle,
concept, or perform a routine procedure
• DOK-2 - Basic Application of Skills/Concepts - Use of information, conceptual knowledge, select appropriate procedures for a task, two or more steps with decision points along the way, routine problems applying 2+ concepts, organize/display data, interpret/use simple graphs
• DOK-3 - Strategic Thinking - Requires reasoning, developing a plan or sequence of steps to approach problem; requires some decision making and justification; abstract, complex, or non-routine; often more than one possible answer or approach
• DOK-4 - Extended Thinking - An original investigation or application to real world; requires time to research, problem solve, and process multiple conditions of the problem or task; non-routine manipulations, across disciplines/content areas/multiple sources
DOK Misconception #1:
All kids can’t do this; or Kids don’t need scaffolding to get “up” there.
Engaging in “a complex task” with supports/ scaffolding is an essential step along the way to proficiency (Vygotsky’s ZPD)
– Do it with others first; DOK 3 and 4 are not meant to only be done alone/independently, especially at first
– Oral language & meaningful discourse support deeper thinking and increase initial exposures to the content and student engagement. This is NOT cheating!
– One strategy: Plan questioning & formative probes from DOK 1-2-3-4 over the course of a lesson or unit of study. Consider all DOK levels in your planning.
Karin Hess (2008). Using learning progressions as a schema to monitor progress across grades.
Vygotsky: Zone of Proximal DevelopmentVygotsky: Zone of Proximal Development(What a child can do (What a child can do with assistancewith assistance today) today)
LEARNING PROGRESSIONS ZONE: Dynamic area
Causes development to move forwardSocial interaction essential (scaffolding)
Actual Development
Area
Potential Development
Area
The ZONE
What a child can do independently now: “ENTRY”
What a child can do independently tomorrow/future
DOK Misconception #2:
Webb’s DOK model is a taxonomy• Bloom’s is a taxonomy, intended to be a
hierarchy
• Primary Weaknesses of Bloom: generic verbs (void of content); same verbs at different levels
• Webb’s DOK model is nominative: – It names how you interact with content
– It differentiates varying levels of engagement with content and suggests what tasks might look like
– DOK 4 is not better than DOK 3, or DOK 2, or DOK 1
DOK Misconception #3:
Bloom verbs & levels = Webb DOKThe DOK “Wheel of Misfortune” implies that a DOK level is
indicated by a particular verb or set of verbs.
Norman Webb, “It’s what comes after the verb, that indicates the complexity of a task.”
DOK Misconception #4:
DOK is about difficulty.
• The intended student learning outcome determines the DOK level. What mental processing must occur? DOK = Complexity, not difficulty!
• While verbs may appear to point to a DOK level, it is what comes after the verb that is the best indicator of the rigor/DOK level and complexity of the task.– Describe the information contained in graphics or data tables in the text; or the
rule for rounding a number– Describe how the two story characters are alike and different.– Describe the data or text evidence that supports your solution, reasoning, or
conclusions– Describe varying perspectives on global climate change using supporting
scientific evidence, and identify the most significant effects it might have on the planet in 100 yrs.
“UGs”
Evaluate
Recall, locate basic facts, definitions, details and eventsSelect appropriate word when intended meaning is clear
DOK LEVEL 1Recall and Reproduction
Understand
DOK LEVEL 3Reasoning
DOK 2Skills and Concepts
DOK 4Extended Thinking
Create
Use context to find meaningObtain and use information in text features
Explain relationshipsSummarizeCentral ideas
AnalyzeApply
Use language structure or word relationships (synonyms/anto-nyms)
Remember
Analyze or interpret author’s craft (e.g., literary devices, viewpoint, or potential bias) to critique a text
Use concepts to solve non-routine problems and justify solutions with evidence
Explain, generalize or connect ideas using supporting evidence (quote, text, evidence, data, etc.)
Compare literary elements, facts, terms and events.Analyze format, organization & text structures
Identify information in a graphic, table, visual, etc.
Cite evidence and develop a logical argument for conjectures based on one text or problem.
Synthesize information across multiple sources Articulate a new voice, theme, perspective.
Evaluate relevancy, accuracy and completeness of information
Analyze multiple sources or textsAnalyze complex abstract themes.
Devise an approach among alternatives to research a novel problem
Explain how concepts relate to other content domains
Develop a complex model or approach for a given situationDevelop an alternative solution
Generate conjectures or hypotheses based on observations or prior knowledge
Brainstorm ideas, concepts, problems, or perspectives related to a topic
BloomBloom’’s Taxonomy s Taxonomy ++ Webb Webb’’s DOK = the Hess CRMs DOK = the Hess CRM 14
The Hess Cognitive Rigor Matrix Applies Webb’s DOK to Bloom’s Cognitive Process
DimensionsDepth + Thinking
Level 1Recall & Reproduction
Level 2Skills & Concepts
Level 3Strategic Thinking
Level 4Extended Thinking
Remember - Recall, locate basic facts, details, events
Understand - Select appropriate words to use when intended meaning is clearly evident
- Specify, explain relationships- summarize– identify main ideas
- Explain, generalize, or connect ideas using supporting evidence (quote, example, data …)
- Explain how concepts or ideas specifically relate to other content domains or concepts
Apply - Use language structure (pre/suffix) or word relationships (synonym/antonym) to determine meaning
– Use context to identify meaning of word- Obtain and interpret information using text features
- Use concepts to solve non-routine problems
- Devise an approach among many alternatives to research a novel problem
Analyze - Identify whether information is contained in a graph, table, text feature, etc.
– Compare literary elements, terms, facts, events– analyze format, organization, & text structures
- Analyze or interpret author’s craft (literary devices, viewpoint, or potential bias) to critique a text
– Analyze multiple sources- Analyze complex/abstract themes
Evaluate “UG” = unsubstantiated generalization
– Cite evidence and develop a logical argument for conjectures
- Evaluate relevancy, accuracy, & completeness of information
Create - Brainstorm ideas about a topic
- Generate conjectures based on observations or prior knowledge
- Synthesize information within one source, data set, or text
- Synthesize information across multiple sources or texts
DOK Misconception #5:
All DOK levels can be assessed with a multiple choice question
• That’s just dumb!• Weak DOK 3 multiple choice items are possible; but
does selecting the best option (e.g., locate supporting evidence for a theme) provide as much insight as seeing HOW a student formulates and reveals thinking?
• By their nature, DOK 3 and 4 questions/tasks are more open-ended, generally take longer to respond to/solve, and may have more than one “appropriate right answer”
2. The DOK Matrix Instructional Paths
Each standard has an assigned Depth of Knowledge.
The DOK determines the cognitive level of instruction.
Recall, locate basic facts, definitions, details, events
Select appropriate words for use when intended meaning is clearly evident.
DOK 1Recall and Reproduction
Remember
Understand
DOK 2Skills and Concepts
Apply
Explain relationshipsSummarizeState central idea
Use context for word meaningsUse information using text features
DOK 3Reasoning and
Thinking
Analyze
Analyze or interpret author’s craft (e.g., literary devices, viewpoint, or potential bias) to critique a text
Explain, generalize or connect ideas using supporting evidence (quote, evidence, data)
.
Cite evidence and develop a logical argument for conjectures based on one text or problem
Evaluate
Use concepts to solve non-routine problems and justify
DOK 4Extended Thinking
Synthesize across multiple sources/ textsArticulate a new voice, theme, or perspective
Evaluate relevancy, accuracy and completeness of information across texts or sources
Analyze multiple sources or multiple textAnalyze complex abstract themes
Devise an approach among many alternatives to research a novel problem
-Explain how concepts or ideas specifically relate to other content domains.
Develop a complex model or approach for a given situationDevelop an alternative solution
.Create
Instruction & Assessment Decisions…
17
Selected ResponseSelected ResponseSelected ResponseSelected Response
Constructed ResponseConstructed ResponseConstructed ResponseConstructed Response
Performance Performance TasksTasks
Performance Performance TasksTasks
Let’s practice using the CRM-back to Little Red Riding Hood
• Your sample questions – a basic and more rigorous question
• Handout #2: Linking Research, Tool #1: CRM template for Close Reading (page 4)
Depth + Thinking
Level 1Recall & Reproduction
Level 2Skills & Concepts
Level 3Strategic Thinking/ Reasoning
Level 4Extended Thinking
Remember What color was Red’s cape?Who is this story about?
Understand Who are the main characters?What was the story’s setting?
Retell or summarize the story in your own words.
What is the author’s message or theme? Justify your interpretation using text evidence.
Apply Identify words/phrases that helped you to know the sequence of events in the story.
Analyze Is this a realistic or fantasy story?
Compare the wolf character to the character of Red. How are they alike-different?
Is this a realistic or fantasy story? Justify your interpretation using text evidence.
Are all wolves (in literature) like the wolf in this story? Support your response using evidence from this and other texts.
Evaluate What is your opinion about the cleverness of the wolf? Justify your opinion using text evidence.
Which version has the most satisfying ending? (establish criteria first, then locate evidence)
Create Write text messages between Red & her mother explaining the wolf incident.
Some other content examples…
• Your class will be learning about…– Fractions/decimals; Data use– Data use; scientific investigation– Elements of art & principles of design
• Come up with a basic understanding and more rigorous question you could pose.
• Use a CRM Template to Plan Units
Depth + Thinking
Level 1Recall & Reproduction
Level 2Skills & Concepts
Level 3Strategic Thinking/ Reasoning
Level 4Extended Thinking
Remember What is slope?What is white space?
Understand Read, write, and represent these fractionsDescribe why white space is used.
Explain how you solved this problem.Why control variables in the investigation?
Construct an argument to show equivalence using area, set, and linear models
Apply Convert this fraction to a decimalAdd these fractions
Use these data to graph your solution
Conduct the investigation, interpret results, and support conclusions with data
Analyze What kind of graph or model is this?Which data point shows ____?Find examples…
Compare these methods.
Which graph shows how the data would be displayed?
Interpret what was happening in the event? Justify your interpretation using what you know about slope.
Analyze more than one product(same time period, medium,theme drawing from multiple contexts source materials for the analyses
Evaluate UG - Which team is the best?
How would you rank these ___? Justify your rankings using data that supports your criteria.
Some say the NFL settlement for player brain injury is not adequate. Evaluate both sides using data to determine the validity of this claim.
Create How would you demonstrate each technique?
Create a card game using fractions.Create scenario explained by a data display.
Integrate multiple source materials with intent to develop a product
DOK Misconception #6:
Higher order thinking = deeper learning
• What we have thought of as “higher order” (analysis, evaluation, creative thinking) might only be engaging or fun…and not always deeper
• Many critical thinking examples do not go deep or get to DOK 3 or 4 (e.g., interpret/solve and justify)
• Shift our thinking from “higher order” to deeper learning, and that can mean:– deeper understanding
– deeper application
– deeper analysis, etc.The Hess CRM illustrates this shift
Some general rules of thumb…• If there is one correct answer, it is probably level
DOK 1 or DOK 2– DOK 1: you either know it (can recall it, locate it, do it) or you don’t know it – DOK 2 (conceptual): apply one concept, then make a decision before going
on applying a second concept; express relationship (if-then; cause-effect)
• If more than one answer/approach, requiring evidence, it is DOK 3 or 4– DOK 3: Must interpret, provide supporting evidence and reasoning (not just
HOW solved, but WHY it works– explain reasoning for each step/decision made)
– DOK 4: all of “3” + use of multiple sources/data/ texts; initiate & complete an investigation
DOK Misconception #7:
Multi-step or longer tasks, multiple texts, or complex texts always means deeper thinking
• DOK 2 is not simply more than one step, it’s applying more than one concept; DOK 2 is still routine/typical (main idea, word problems, etc.)
• Simply reading more complex texts, but NOT delving deeply into the text’s meaning/style/etc., is likely to still be DOK 1 or 2
• DOK 3 requires some aspect of open-endedness and interpretation with justification or support; DOK 4 = drawing from multiple sources
Reading StandardsDepth + Thinking
Level 1Recall & Reproduction
Level 2Skills & Concepts
Level 3Strategic Thinking
Level 4Extended Thinking
Remember KEY DETAILSDecode, read orally
Understand KEY DETAILSWORD MEANINGS- fill in
CENTRAL IDEASSUMMARIZEpredict, infer
REASONING & SUPPORT – interpret theme, purpose, pt of view/perspective
REASONING & SUPPORT –multiple texts
Apply WORD STRUCTURE -roots, affixes, structure, RELATIONSHIPSsynonyms-antonyms
WORD MEANINGS-use in contextUSE TEXT STRUCTURES & FEATURES
TEXT STRUCTURES & FEATURES – how is the message structured/ presented?
REASONING & SUPPORT –multiple texts-compare treatments
Analyze USE of TEXT STRUCTURE or FEATURESLANGUAGE USE-identify non- literal usage
ANALYSIS & REASONING WITHIN TEXTSLANGUAGE USE-interpret non- literal usage
ANALYSIS & REASONING ACROSS TEXTS
Evaluate AUTHOR’s CRAFT WITHIN A TEXT (e.g., language use impact/intent; bias, reasoning)
EVALUATE AUTHOR’s PURPOSE or CRAFT ACROSS TEXTS
Create
Reading & Writing Depth + Thinking
Level 1Recall & Reproduction
Level 2Skills & Concepts
Level 3Strategic Thinking
Level 4Extended Thinking
Remember KEY DETAILS
Understand KEY DETAILSWORD MEANINGS- fill in
CENTRAL IDEASSUMMARIZEpredict, infer
REASONING & SUPPORT – DEVELOP theme, or pt of view/ perspective topic
REASONING & SUPPORT – use multiple texts - compare or elaborate
Apply WORD STRUCTURERELATIONSHIPS
EDIT/CLARIFYUSE TECHNOLOGYCITE SOURCES
WORD MEANINGS-LANGUAGE USEUSE OF TEXT STRUCTURES & FEATURES
INTEGRATE TEXT STRUCTURES & FEATURES into Compositions
REASONING & SUPPORT
Analyze USE of TEXT STRUCTURES or FEATURESLANGUAGE USE
ANALYSIS & REASONING WITHIN TEXTS – RESEARCH for WritingDevelop reasoning
ANALYSIS & REASONING ACROSS TEXTSRESEARCH for Writing;Comparing themes
Evaluate AUTHOR’s CRAFT WITHIN A TEXTEvaluate credibility of sources
EVALUATE AUTHOR’s PURPOSE or CRAFT ACROSS TEXTS
Create WRITE/EDIT BRIEF TEXTS
COMPOSE /REVISE FULL TEXTS
COMPOSE FULL TEXTS-sources
Math Content Standards & Math Practices
Depth + Thinking
Level 1Recall & Reproduction
Level 2Skills & Concepts
Level 3Strategic Thinking (support with data, equations, models, etc.)
Level 4Extended Thinking (cross domains)
Remember Know math facts, terms
Understand Attend to precisionEvaluate expressions, plot point
Model with mathematicsEstimate, predict, observe, explain relationships
Construct viable argumentsGeometry proof
Integrate concepts across domains
Apply Calculate, measure, make conversions
Make sense of routine problems
Make sense of non-routine problems
Design & conduct a project
Analyze Identify a patternLocate information in table
Use tools strategicallyClassify, organize data, extend a pattern
Reason abstractlyGeneralize a pattern
Analyze multiple sources ofevidence
Evaluate Critique the reasoning of others
Create Design a complex model
For each assessment task (or rubric)…ask
– What is its purpose? (What content/skill is being assessed? is there a ‘right’ answer?)
– What is the implied/intended rigor? (What mental processing would you expect students to engage in? Use the CRM to find descriptors.)
– Which standards does it REALLY assess? (content + intended rigor)
– Does the scoring guide/rubric match content + intended rigor?
– What would student responses tell a teacher if students could/could not do all or part of this task? (open-ended tasks, reasoning used) – next instructional decisions are clear
Take-Away Messages: Cognitive Rigor & Some Implications for Assessment
• Begin with daily DOK3 classroom discourse!• Assessing only at the highest DOK level (the
“ceiling”) will miss opportunities to know what students do & don’t know – go for a range; end “high” in selected/prioritized content
• Performance assessments can offer varying levels of DOK embedded in a larger, more complex task
• Planned formative assessment strategies and tools can/should focus on differing DOK levels
Some resources worth exploring…• www.nms.org – national math science initiative – gr 3 – HS; also has ELA and
SS performance tasks with DOK designations
• www.readworks.org – gr k-8; short literary & informational texts with CC Qs
• http://www.louisianabelieves.com/resources/library/common-core-state-standards - LA gr 3-HS math & ELA tasks (some math samples are weak and not all math DOK levels are correct, but generally good examples)
• http://ccgpsmathematicsk-5.wikispaces.com/K-5+Formative+Assessment+Lessons+%28FALs%29 - GA gr K-5 math
• www.exemplars.com – sample math PAs for k-12; science PAs for k-8
• Dan Meyer blog – math PAs for MS-HS; kids have to build the problems by deciding what’s needed to solve them – good strategic thinking required
• Hess & Gong (2014). Ready for college and career? Achieving the Common Core Standards and beyond through deeper, student-centered learning http://www.nmefoundation.org/resources/scl-2/ready-for-college-and-career
• Links on www.karin-hess.com: – Karin’s YouTube video – “Text Complexity Tools” (qualitative text analysis)
– Karin’s vimeo DOK video – http://vimeo.com/18281415
– Karin’s YouTube DOK video – excerpt from a CT workshop