Professional Development to Practice
The contents of this presentation were developed under a grant from the US Department of Education to the Missouri Department of Elementary and Secondary Education (#H323A120018). However, these contents do not necessarily represent the policy of the US Department of Education, and you should not assume endorsement by the Federal Government. Professional Development to Practice
Professional Development to Practice
Writing Constructed
Response Items
Common Formative Assessment
Professional Development to Practice
KEY:
Core Training Modules
Follow-up Training Modules
Precursors to Training
Once teams determine an EP to focus on, they can choose one or multiples of these focused modules. Each of the EP modules in this section will include: implementation guidance with tools and troubleshooting, and using data to determine effectiveness.
The contentsof this presentation weredeveloped under a grant fromtheUSDepartment of Education to theMissouri Department of Elementaryand Secondary Education(#H323A120018). However, thesecontents do not necessarily representthe policy of the US Department ofEducation, and youshould not assumeendorsement by the FederalGovernment.
Collaborative Data Teams (CDT)
Foundational Processes
AgendasCommunicationNormsRoles
Overview and Purpose
Collaborative Teams
Activity: Wrap Up/Overview of
Next Steps
Follow-Up Based on Data: Coaching and Revisiting PD
School-Based Implementation Coaching
Overview and Purpose of
Coaching for supporting
school-wide implementation
Critical skills of coaching
Coaching in Practice
Activity: Wrap Up/Overview of
Next Steps
Follow-Up Based on Data: Coaching and Revisiting PD
Collaborative Work Training
Follow-up to Training
Getting Started
Wrap Up Activity
Focus AreasIntroduction to Missouri Collaborative Work
Use Getting Started Guide to determine starting point and scope of learning
Data-Based Decision Making (DBDM)
Overview and Purpose of DBDM
Data Team Process Steps Sequence and Examples
1. Collect and Chart Data2. Analyze and Prioritize3. SMART Goal4. Instructional Decision Making5. Determine Results Indicators6. Ongoing Monitoring
Developing Meaningful
Learning Targets
Quality Assessment
Design
Performance Events
Constructed Response
Items
Selected Response
Items
Common Formative Assessment (CFA)
Overview and Purpose of CFA
AdvancedProcesses
Consensus Collaborative SkillsProtocols
Overview and Purpose of EP
Effective Teaching/ Learning Practices (EP)
Spaced versus
Massed
Feedback
Assessment Capable Learners
Reciprocal Teaching
Spaced versus
Massed
Assessment Capable Learners
September 2013
Activity: Wrap Up/Overview of
Next Steps
Activity: Wrap Up/Overview of
Next Steps
Activity: Wrap Up/Overview of
Next Steps
Professional Development to Practice
Writing Constructed
Response Items
Developing Meaningful Learning
Targets
Quality Assessment Design
Common Formative Assessment (CFA)
Overview and Purpose of CFA
Professional Development to Practice
What are Constructed-Response (CR) Questions?
Formulate own answersConstruct own responsesClosed and Open-Ended CRs
Professional Development to Practice
What do you think?
What is the difference between an open-ended constructed response item and a closed constructed response item?
Jana Scott, University of MO-Columbia, Revised Presentation 2013.
Professional Development to Practice
Constructed Response Questions
CLOSED CRs Measure knowledge and comprehension Elicit a right or wrong answerNo tolerance for diversityUsually allow for only one way to arrive at a correct answer
OPEN-ENDED CRs •Measure higher-level cognitive processes such as application, inference, analysis and synthesis•Elicit diverse answers•Mild to high tolerance for diversity•Usually allow for more than one way to arrive at an answer
Professional Development to Practice
Degrees of Open-Endedness
Closed CR Minimal
Open-Ended CR
Medium Open-Ended CR
Extended Open-Ended CR
Professional Development to Practice
Closed 0-1 Minimal Open Ended 0-2 Medium Open Ended 0-3 Extended Open-Ended 0-4
Closed CR items have two distinct levels of performance….right or wrong or acceptable and not acceptable.
The range of responses yielded by minimal constructed response items necessitates the use of three distinct levels of performance.
The range of responses yielded by medium constructed response items necessitates the use for four distinct levels of performance.
The wide range of responses yielded by extended constructed response items necessitates the use for five distinct levels of performance.(Sometimes referred called Extended Responses.)
Example: What is the setting of the story Little Red Riding Hood (LRRH)?
Example: What is the setting of the story LRRH? How do you know?
Example: What is the setting? Explain the impact on the story if the setting was changed?. Use two specific examples from the story to support your answer.
Example: Change the setting of LRRH to a city. Explain how the plot, characters, and major story events are impacted and how they are different than the original story.
Usual time for completion is about 2-3 minutes.
Usual time for completion is about 4-7 minutes.
Usual time for completion is about 8-11 minutes.
Usual time for completion is about 12-15 minutes.
Jana Scott, University of MO-Columbia, 2012.
Constructed Response Fall on a Spectrum of Open-Endedness (Handout)
Professional Development to Practice
Inquiry Activity What characteristics do open-
ended constructed response (OCR) questions possess?
Samples in handout packet.
Professional Development to Practice
OCR QuestionsApplication Complex cognitive processesElicit diverse answersMultiple ways to arrive at an
answerSupport or justification, More than one component piece.
Professional Development to Practice
OCR Questions Making an inference Interpreting information Logical responses based on
givensClear concise answer Completed anywhere from 2-15
minutes (depending on type
Professional Development to Practice
Different Names for Same Type of Item (Handout)
Missouri Terms Smarter Balanced Math
Smarter Balanced ELA
Assessment For Learning
Leadership and Learning Center
Closed Constructed Response Items 0-1
Referred to as fill-in-the-blank or short answer questions
Not referenced Referred to as short answer written response
Does not consider Fill-in the Blank to be CR Items
Referred to as fill-in the blank and short answer questions
Minimal Open-Ended Constructed Response 0-2
Medium Open-Ended Constructed Response Item 0-3
Referred to as brief constructed response questions
Usually part of the computer-adaptive section
Referred to as 2 Point CRs or 3-Point CRs
Referred to as short written response items
Referred to as short constructed response
Extended Open-Ended Constructed Response Items 0-4
Performance Events or Tasks
Any CR Item that is part of the Performance Task will be referred to as Extended Response (ER) items.
Performance tasks are a series of scaffolded CRs (called ERs) related to a theme.
Not referenced
Part II of Performance Task is a Writing Prompt
Referred to as extended written response items
Referred to as Performance Task
Referred to as extended response
Referred to as extended response
Professional Development to Practice
1. Forces students to demonstrate an in-depth understanding of content 2. Requires the use of complex cognitive processes or higher level thinking skills 3. Elicits diverse answers 4. Allows for multiple points of view and interpretations. 5. Requires support, justification, evidence, and/or explanation of reasoning 6. Requires the student to refer to specific places in the text that justify their conclusions
or claims 7. Requires students to “inference or interpret information 8. Is worded in a clear, concise manner 9. Prompts the student to do everything required by the rubric for the highest rating 10. Avoids teaching in the item stem (giving clues in the item stem) 11. If support is elicited, there is ample information (in story, graph, etc.) for students to use as support. 12. Is written in the simplest language possible.
Critique Form for OCRs (Handout) Except Mathematics
Professional Development to Practice
1. The problem allows for multiple solution processes or may be solved in more than one way. (This does not necessarily mean that there are multiple answers; instead there are multiple ways to arrive at an answer.)
2. The problem allows for application of knowledge and/or is set in a real world scenario. 3. The solution process or how one goes about finding an answer is not apparent. 4. The problem involves multiple steps. 5. The problem contains enough information to find a solution and have clarity; but not too
much to diminish the problem-solving approach to learning. 6. The problem is written is such a way to clearly elicit the desired evidence of a student’s
problem-solving approach. 7. When feasible, the problem promotes connections between and among concepts. 8. When feasible, the problem requires supporting reasoning/data and/or justification/
explanation. 9. When feasible, the problem allows for conjecture. 10. The problem accurately assesses the learning target. 11. The problem is appropriate in terms of grade-level difficulty, cognitive complexity, and
reading-level. 12. The content and terms used are mathematically accurate. 13. The problem asks students to show a process, communicate a process, and/or explain a
problem-solving approach. 14. The wording used is clear, concise, and to the point.
Critique Form for Higher-Level Math CRs (Handout)
Professional Development to Practice
Constructed Response Items from Smarter
Balanced Assessment Consortium
Professional Development to Practice
The table below shows the number of students in each third-grade class at Lincoln School.
There are 105 fourth-grade students at Lincoln School. How many more fourth-grade students than third-grade students are at Lincoln School? Show or explain how you found your answer.
The table below shows the number of students in each third-grade class at Lincoln School.
There are 105 fourth-grade students at Lincoln School. How many more fourth-grade students than third-grade students are at Lincoln School? Show or explain how you found your answer.
Students in Third-Grade
Class Number of Students
Mrs. Roy 24
Mr. Grant 21
Mr. Harrison 22
Ms. Mack 25
Professional Development to Practice
Professional Development to Practice
Scaffolded Set of
CRs Grade 7
Professional Development to Practice
Scaffolded Set of CRs Grade 8
Professional Development to Practice
Constructed ResponseExtended Response
Ms. McCrary wants to make a rabbit pen in a section of her lawn. Her plan for the rabbit pen includes the following:•It will be in the shape of a rectangle.•It will take 24 feet of fence material to make.•Each side will be longer than 1 foot.•The length and width will measure whole feet.
Part ADraw 3 different rectangles that can each represent Ms. McCrary’s rabbit pen. Be sure to use all 24 feet of fence material for each pen.
Use the grid below. Click the places where you want the corners of your rectangle to be. Draw one rectangle at a time. If you make a mistake, click on your rectangle to delete it. Continue as many times as necessary.
Use your keyboard to type the length and width of each rabbit pen you draw. Then type the area of each rabbit pen. Be sure to select the correct unit for each answer.
[Students will input length, width, and area for each rabbit pen. Students will choose unit from drop down menu.]
Pen 1: Length: (feet, square feet) Width: (feet, square feet) Area: (feet, square feet)
Part BMs. McCrary wants her rabbit to have more than 60 square feet of ground area inside the pen. She finds that if she uses the side of her house as one of the sides of the rabbit pen, she can make the rabbit pen larger.•Draw another rectangular rabbit pen. •Use all 24 feet of fencing for 3 sides of the pen.•Use one side of the house for the other side of the pen. •Make sure the ground area inside the pen is greater than 60 square feet.Use the grid below. Click the places where you want the corners of your rectangle to be. If you make a mistake, click on your rectangle to delete it.
Pen 2: Length: (feet, square feet) Width: (feet, square feet) Area: (feet, square feet)
Pen 3: Length: (feet, square feet) Width: (feet, square feet) Area: (feet, square feet)
Use your keyboard to type the length and width of each rabbit pen you draw. Then type the area of each rabbit pen. Be sure to select the correct unit for each answer.
Length: (feet, square feet) Width: (feet, square feet) Area: (feet, square feet)
Professional Development to Practice
Grade 4 OCR
Professional Development to Practice
Grade 4 ELA
Professional Development to Practice
Professional Development to Practice
Handout Packet
Professional Development to Practice
Items That Are Not Considered to Be Quality
Speculate the Reasons! 1. Create a pamphlet to advertise a country. Include the
country’s flag, anthem, imports, exports, etc.2. Write an essay to explain what the country of Canada
is like. 3. Create a PowerPoint Presentation about the novel Red
Badge of Courage. 4. Draw a chart showing the three branches of
government. 5. Draw a picture of a cell. Be sure to add labels. 6. Write five events that happened in the story. 7. Draw a picture of the food pyramid. 8. Explain the causes of the Civil War.
Professional Development to Practice
What are two major flaws in this OCR question?
How might it be repaired? In class, we learned that when supply
goes down, cost goes up. Explain what would happen to the price of hula-hoops if the largest supplier in the United States went out of business. Use four details to support your answer.
Professional Development to Practice
What are the flaws in these questions?Then, describe how to repair the
questions. 1. Did you like the story? Explain why or why not
using three details and/or examples from the story.
2. Explain, what you think will happen to John (the main character) after the end of the story.
3. Predict what will happen to Mary if she disobeys her mother? Use two details and/or examples from the story in your answer.
4. What are two historical reasons to explain the present-day appearance of our national flag?
Professional Development to Practice
Possible Stimulus Materials ELA Mathematics Other
Readings • video clips • audio clips • research topic/issue/ problem • graphs, charts, other visuals • etc.
Graphs Pictures Models Tables Figures Scenarios Data Bases Video Clips Maps Spreadsheets Photos Research Reports
Any items used in ELA and Mathematics.
Any items used in your subject area.
Professional Development to Practice
Locate Stimulus Materials • National Archives• Smithsonian Institute• Marco Polo website • Internet search for charts, graphs,
maps, excerpts from documents• Digital video clips from U-tube • Magazine articles • Newspaper articles • Reading A to Z • Project Gutenberg
Professional Development to Practice
Complete the graphic organizer to show the social, political, and economic impact of the Monroe Doctrine.
Mon
roe
Doc
trin
e
Social
Political
Economic
Before After
Professional Development to Practice
Complete the graphic organizer to show two cause-effect relationships contained in the article.
Cause Effect
Professional Development to Practice
Steps in Writing an
Open-Ended CR
See Handout
Professional Development to Practice
1. Work in a small group.
2. Choose any brochure from the center of your table to use as stimulus material.
3. Write an open-ended constructed response question related to any content area.
4. Be ready to share.
Apply Your Knowledge
Professional Development to Practice
Scoring Open-Ended
Constructed
Response Items
Professional Development to Practice
Scoring a Closed CR
Generic Rubric for Closed CR 1 PointThe response demonstrates the knowledge necessary to complete the prompted purpose and contains the correct or acceptable answer. O Points The response demonstrates little to no evidence of knowledge that is appropriate to the intent of the prompted purpose and does not contain the correct answer.
Professional Development to Practice
Requires a different approach than just grading responses right or wrong
OCR questions elicit responses that fall on a wide spectrum of possibilities
Criterion referenced scoring guide or rubric is used.
Generic or Task Specific
Response is rated based in its ability to meet specified criteria
Scoring Responses to OCRs
Professional Development to Practice
GenericGeneric Rubric for a Minimal CR
2 Points The response demonstrates proficient evidence of the knowledge necessary to complete the prompted purpose. The student’s strategy and execution meet the content (including concepts, technique, representations and connections), thinking processes and qualitative demands of the task. Minor omissions may exist, but do not detract from the correctness of the response. 1 PointThe response demonstrates partial knowledge necessary to complete the prompted purpose. It may contain overlooked issues, misleading assumptions, and/or errors in execution. Evidence in the response demonstrates that that the student can revise the work to accomplish the task with the help of written feedback. The student does not need dialogue or additional instructions. O Points The response lacks any evidence of knowledge that is appropriate to the intent of the prompted purpose.
Professional Development to Practice
2 Points: The response contains a main idea and two details as support.
1 Point: The response contains the main idea and one detail as support.
0 Points: The response shows little to no knowledge.
Task Specific
Professional Development to Practice
Steps in Writing an
Open-Ended CR
See Handout
Professional Development to Practice
Steps in Writing a Scoring Guide
See Handout
Professional Development to Practice
2 Points: The response contains an acceptable/correct answer and a valid solution process. The response shows an understanding of the process needed to find the answer.
1 Point: The response contains an acceptable/correct answer. OR 1 Point: The response contains a valid solution process but had minor computational errors. The response shows an understanding of the process needed to find the answer. 0 Points: The response shows severe misunderstanding.
Scoring Guide for an OCR Math Problem
Professional Development to Practice
Sample Scoring Guide for a Mathematics OCR from SBAC
Professional Development to Practice
Rubric for a 2-Point ELA OCR Item
Professional Development to Practice
44
Rubric for a 3-Point ELA OCR Item
Professional Development to Practice
Generic Rubric for Extended CR
Professional Development to Practice
Generic Rubric for Medium CR
Professional Development to Practice
Generic Rubrics for Minimal and Closed CRs
Professional Development to Practice
Work with a small group to write a scoring rubric for these two questions. Look at the steps in your handout packet.
1.What is the main idea of the article? Use two details and examples from the article to explain why you believe this to be the main idea.
2.Evaluate Sally’s experimental design. Identify two things Sally could have done differently to make her results more valid. Give reasoning for your suggestions.
Application Activity
Professional Development to Practice
Who wants to ask the
first question?
Qs and As
Professional Development to Practice
Out of all the things you learned about OCRs, what point stands out as being the “shining star”?
Reflection
Professional Development to Practice
Practice ProfileMissouri Collaborative Work Practice Profile
Foundations present in the implementation of each essential function: Commitment to the success of all students and to improving the quality of instruction. Common Formati ve Assessment
Essential Functions Exemplary Proficiency Ideal Implementation
Proficient
Close to Proficient
(Skill is emerging, but not yet to ideal
proficiency. Coaching is
recommended.)
Far from Proficient (Follow-up
professional development and coaching
is critical.)
Evidence
1
Educators develop clear and meaningful learning goals to guide instruction and student learning.
All of the following criteria are met. Learning goal is clearly connected to a big idea/essential
learning in the domain Learning goal develops deep understanding of
underlying concepts and/or acquisition of skills Learning goal clearly engages higher order thinking
processes Learning goal is clearly manageable and can be
accomplished in the course of a lesson or unit (may be several periods)
Learning target is clearly explained to students Connections between current learning goal and prior
learning are clearly made
At least 6 of the criteria are met.
At least 4 of the criteria are met.
Less than 4 of the criteria are met.
Common Formative Assessment Development & Implementation Template.
2
Educators establish clear and measureable student success criteria in a rubric, scoring guide, or checklist.
All of the following criteria are met. Success criteria are clearly and effectively aligned to
learning goals Success criteria clearly and effectively relate to what
students will say, do, make or write to show evidence of learning
Success criteria clearly and effectively reflect ways for students to indicate their current status relative to the learning goals
Success criteria are communicated in language student can fully understand
Success criteria are frequently referred to during the learning process.
At least 3 of the criteria are met.
At least 2 of the criteria are met.
Less than 2 of the criteria are met.
Common Formative Assessment Development & Implementation Template.
Professional Development to Practice
Implementation Fidelity
Yes Partially No If partially or no, please
explain. 1. Common formative assessment is linked to selected
learning standards.
2. Learning goal engages higher order thinking processes.
3. Learning goal can be accomplished in the course of a unit.
4. Learning target is written in language that students can clearly understand.
5. Learning target is clearly explained to students. 6. Success criteria are written in language that
students can clearly understand in a rubric or checklist.
7. Students receive feedback based on learning goal and their assessment results.
8. The quality of assessment items for measuring mastery is reviewed and items revised as needed.
Total