60
EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

Embed Size (px)

Citation preview

Page 1: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

EQuIP Rubric & Quality Review Training Session:

ELA/Literacy Grades 3-12

1

Page 2: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

2

Use the EQuIP quality review process to determine the quality and alignment of lessons and units to the Common Core State Standards (CCSS) in English language arts (ELA)/literacy

During this session, reviewers will:• Develop their abilities to use EQuIP criteria to provide observations about CCSS-

aligned instructional materials and make suggestions for improvement• Develop a common understanding of the EQuIP quality review process• Develop a common understanding of the rating scale and descriptors for the four

rubric dimensions and the rating categories and descriptors for overall ratings• Develop their abilities to use EQuIP criteria, rating scales and rating descriptors to

accurately rate instructional materials

Session Goals

Page 3: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

3

1. CCSS: Before beginning a review, all members of a review team are familiar with the CCSS.

2. Inquiry: Review processes emphasize inquiry rather than advocacy and are organized in steps around a set of guiding questions.

3. Respect & Commitment: Each member of a review team is respected as a valued colleague and contributor who makes a commitment to the EQuIP process.

4. Criteria & Evidence: All observations, judgments, discussions and recommendations are criterion and evidence based.

5. Constructive: Lessons/units to be reviewed are seen as “works in progress.” Reviewers are respectful of contributors’ work and make constructive observations and suggestions based on evidence from the work.

6. Individual to Collective: Each member of a review team independently records his/her observations prior to discussion. Discussions focus on understanding all reviewers’ interpretations of the criteria and the evidence they have found.

7. Understanding & Agreement: The goal of the process is to compare and eventually calibrate judgments to move toward agreement about quality with respect to the CCSS.

EQuIP Quality Review: Principles & Agreements

Page 4: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

4

EQuIP Quality Review ProcessThe EQuIP quality review process is a collegial process that centers on the use of criteria-based rubrics for English language arts (ELA)/literacy and mathematics. The criteria are organized into four dimensions:

The Four Dimensions1. Alignment to the depth of the CCSS; 2. Key shifts in the CCSS;3. Instructional supports; and4. Assessment.

As educators examine instructional materials against the criteria in each dimension, they are able to use common standards for quality and generate evidence-based commentary and ratings on the quality and alignment of materials.

EQuIP Quality Review: Process & Dimensions

Page 5: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

5

Using the Electronic Quality Review Rubric PDF Form

Page 6: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

6

Using the Quality Review Rubric PDF Form

For each dimension:

• Select the checkbox for each criterion for which clear and substantial evidence is found.

• Make observations and suggestions related to criteria and evidence.

• Determine a rating for each dimension based on checked criteria and observations.

For Dimension I:• Use alignment rating to

determine whether to proceed with review.

Page 7: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

Giving Feedback

Writing effective feedback is vital to the EQuIP Quality Review Process. Below are the four qualities of effective feedback. • Criteria-based: Written comments are based on the criteria used for review in each

dimension. No extraneous or personal comments are included. • Evidence Cited: Written comments suggest that the reviewer looked for evidence in the

lesson or unit that address each criterion of a given dimension. Examples are provided that cite where and how the criteria are met or not met.

• Improvement Suggested: When improvements are identified to meet criteria or strengthen the lesson or unit, specific information is provided about how and where such improvement should be added to the material.

• Clarity Provided: Written comment are constructed in a manner keeping with basic grammar, spelling, sentence structure and conventions.

Page 8: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

Example 1: ELA/Literacy

This unit clearly targets two standards, which are noted in the overview on p. 3. The activities throughout the unit correspond to this list of targeted standards. There are possibilities for alignment with other supporting standards, but the focus chosen by the developer is clear. The purpose of instruction is clearly stated and the unit contains multiple and well integrated opportunities for speaking, listening, reading and writing via discussion, worksheets and close readings. After reviewing the texts, it is clear that they are sufficient quality, complexity and scope for the purpose of the unit to make claims based on textual evidence. Too, the unit provides opportunities to build the students’ content knowledge and their understanding of reading and writing in social studies by weaving in reading from primary sources for the Women’s Suffrage movement.Is this feedback criteria-based?Was evidence cited?Was there an improvement suggested?Is clarity provided?

Page 9: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

Example 1: Feedback

Criteria-based: Yes

Evidence Cited: Yes

Improvement suggested: Partial

Clarity Provided: Yes• This is effective feedback. The reviewer mentions two standards and cites their evidence,

“which are noted in the overview.” It would be more effective if the reviewer named the standards specifically.

• The reviewer continues to use the alignment criteria to provide effective feedback about why the instructional materials meet the criteria: “The purpose of instruction is clearly stated and the unit contains multiple and well integrated opportunities for speaking, listening, reading, and writing via discussion, worksheets and close reading.”

• The single improvements mentioned, “ there are possibilities for other alignment,” could be more effective if it suggested at least one of those possibilities.

• The written comments are constructed in a manner in keeping with basic grammar, spelling, sentence structure and conventions.

Page 10: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

Example 2: ELA/Literacy

The lesson targets three standards, which are highlighted in the lesson cover page. The first activity targets the first standard listed. The second activity to compare and contrast character traits, although an instructive exercise, is not aligned to the standards listed and, in this context, the instructional purpose of the compare and contrast activity is unclear. The texts are very engaging and could be fun to do as a read-aloud as well.

Is this feedback criteria-based?

Was evidence cited?

Was there an improvement suggested?

Is clarity provided?

Page 11: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

Example 2: Feedback

Criteria-based: Partial

Evidence Cited: Partial

Improvement suggested: No

Clarity Provided: Partial• This could be more effective feedback. The reviewer mentions two standards and cites their

evidence. It would be more effective if the reviewer named the standards specifically. The reviewer also mentions personal comments that are not related to the criteria.

• The reviewer mentions the second activity and describes why it does not meet the criteria stating, “although an instructive exercise, is not aligned to the standards listed and, in this context, the instructional purpose of the compare and contrast activity is unclear. The texts are very engaging and could be fun to do as a read-aloud as well,” but fails to offer the improvement that would help to meet the criteria. A suggestion for improvement could be to revise the activity so that there is better alignment to the standard.

• The feedback could be more clear and the written comments are constructed in a manner in keeping with basic grammar, spelling, sentence structure and conventions.

Page 12: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

12

Quality Review Steps

Step 1. Review Materials • Record the grade and title of the lesson/unit on the Quality Review Rubric PDF• Scan to see what the lesson/unit contains and how it is organized • Read key materials related to instruction, assessment and teacher guidance• Study and measure the text(s) that serves as the centerpiece for the lesson/unit,

analyzing text complexity, quality, scope and relationship to instruction

Step 2. Apply Criteria in Dimension I: Alignment to the Depth of the CCSS• Identify the grade-level CCSS that the lesson/unit targets• Closely examine the materials through the “lens” of each criterion• Indicate each criterion for which clear and substantial evidence is found • Record input on specific improvements needed to meet criteria or strengthen alignment• Compare observations and suggestions for improvement

Page 13: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

13

Quality Review Steps

Step 3. Apply Criteria in Dimensions II–IV • Examine the lesson/unit through the “lens” of each criterion • Indicate each criterion met and record observations and feedbackWhen working in a group, individuals may choose to compare observations and suggestions for improvement after each dimension or wait until each person has rated and recorded all input for Dimensions II–IV.

Step 4. Apply an Overall Rating and Provide Summary Comments • Individually review comments for Dimensions I–IV, adding/clarifying comments as

needed • Individually write summary comments on the Quality Review Rubric PDFWhen working in a group, individuals should record summary comments prior to conversation.

Step 5. Compare Overall Ratings and Determine Next Steps • Note the evidence cited to arrive at summary comments and similarities and differences

among reviewers. Recommend next steps for the lesson/unit and provide recommendations for improvement to developers/teachers.

Page 14: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

14

EXAMPLE: Common Unit for Review — ELA/Literacy

Grade 8 — “Making Evidence-Based Claims”

This state is working with vendors to create full instructional units and, ultimately, comprehensive, year-long curricular materials aligned to the CCSS. This state uses the CCSS, the Tri-State Rubrics and the Publishers’ Criteria to guide development. Teachers will be able to use these materials for classroom instruction or model their own planning after these examples.

Note: This unit was submitted to EQuIP for review in October 2012 as an in-progress work. The unit has been revised by the developers in response to feedback from the EQuIP quality review process. Do not distribute the version of this unit used in EQuIP trainings. See the revised lesson/unit on the Engage NY website: www.engageny.org/resource/making-evidence-based-claims-units-ccss-ela-literacy-grades-6-12.

Page 15: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

15

EXAMPLE:Step 1. Review Materials• Record the grade and title of the lesson/unit on the Quality Review Rubric PDF

– Grade 8, “Making Evidence-Based Claims”• Scan to see what the lesson/unit contains and how it is organized

– Unit overview– Lessons with instructional notes and assessment opportunities– Checklists and rubrics– Student handouts– Model handouts

• Read key materials related to instruction, assessment and teacher guidance– Unit outline (p. 2 of overview)– Alignment (p. 3 of overview and pp. 1, 7, 15 and 15 of lesson)– Assessment (p. 3 of overview and at the end of each lesson)– Instructional notes (in each lesson)

Page 16: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

16

EXAMPLE:Step 1. Review Materials• Study and measure the text(s) that serves as the centerpiece for the lesson/unit,

analyzing text complexity, quality, scope and relationship to instruction– “Ain’t I a Woman” measures 770L– “Equal Rights for Women” measures 1170L– “Wimbledon Has Sent me a Message” measures 1130L– The texts contain complex ideas, figurative and academic language, and distinct

points of view

Page 17: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

11

The lesson/unit aligns with the letter and spirit of the CCSS:

1. Targets a set of grade-level ELA/literacy standards .

2. Includes a clear and explicit purpose for instruction.

3. Selects text(s) that measure within the grade-level text complexity band and are of sufficient quality and scope for the stated purpose (i.e., present vocabulary, syntax, text structures, levels of meaning/purpose, and other qualitative characteristics similar to CCSS grade-level exemplars in Appendices A and B).

A unit or longer lesson should:

4. Integrate reading, writing, speaking and listening so that students apply and synthesize advancing literacy skills.

5. (Grades 3–5) Build students’ content knowledge and their understanding of reading and writing in social studies, the arts, science or technical subjects through the coherent selection of texts.

Criteria for Dimension I: Alignment to the Depth of the CCSS

Page 18: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

18

Dimension Rating and Descriptive Scales To Synthesize Judgment

Rating Scale for Dimensions I–IV:

3: Meets most to all of the criteria in the dimension

2: Meets many of the criteria in the dimension

1: Meets some of the criteria in the dimension

0: Does not meet the criteria in the dimension

Descriptors for Dimensions I–IV:

3: Exemplifies CCSS Quality — meets the standard described by criteria in the dimension, as explained in criterion-based observations

2: Approaching CCSS Quality — meets many criteria but will benefit from revision in others, as suggested in criterion-based observations

1: Developing toward CCSS Quality — needs significant revision, as suggested in criterion-based observations

0: Not representing CCSS Quality — does not address the criteria in the dimension

Page 19: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

19

EXAMPLE:Step 2. Apply Criteria in Dimension I: Alignment

The lesson/unit aligns with the letter and spirit of the CCSS: 1. Targets a set of grade-level ELA/literacy standards. 2. Includes a clear and explicit purpose for instruction. 3. Selects text(s) that measure within the grade-level text complexity band and are of

sufficient quality and scope for the stated purpose (i.e., present vocabulary, syntax, text structures, levels of meaning/purpose, and other qualitative characteristics similar to CCSS grade-level exemplars in Appendices A and B).

A lesson or longer unit should: 4. Integrate reading, writing, speaking and listening so that students apply and synthesize

advancing literacy skills. 5. (Grades 3–5) Build students’ content knowledge and their understanding of reading

and writing in social studies, the arts, science or technical subjects through the coherent selection of texts.

NOTE: Reviewers should be able to refer to evidence to support pattern of

checkboxes selected.

Jennifer Taylor
This note is not in the math slides, should it be?
Page 20: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

20

EXAMPLE:Step 2. Apply Criteria in Dimension I: Alignment

Observations/Feedback and Rating

This unit clearly targets two standards on p. 4 of the overview. The activities throughout the unit correspond to this list of targeted standards (e.g., p. 8). There are possibilities for other alignment, but the focus chosen by the developer is clear.

The texts measure 770L, 1130L and 1170L. After reading the texts, it is clear that they are of sufficient quality and scope for the purpose of the unit — making claims based on textual evidence. The texts also cohere around the theme of women’s rights.

The purpose of instruction is clearly stated, and the unit contains multiple and well-integrated opportunities for speaking, listening, reading and writing (discussion, worksheets, close readings).

Rating: 3Approaching CCSS Quality — meets many criteria but will benefit from revision in others, as suggested in criterion-based observations

Page 21: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

21

EXAMPLE:Step 2. Apply Criteria in Dimension I: Alignment

Compare Criterion-Based Checks, Observations and Feedback• What is the pattern within our team in terms of the criteria we have checked?• Do our observations and feedback reference the criteria and evidence (or lack of

evidence) in the instructional materials?• Does our feedback include suggestions for improvement(s)?

Page 22: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

Criteria for Dimension II: Key Shifts in the CCSS

The lesson/unit addresses key shifts in the CCSS:

1. Reading Text Closely: Makes reading text(s) closely, examining textual evidence and discerning deep meaning a central focus of instruction.

2. Text-Based Evidence: Facilitates rich and rigorous evidence-based discussions and writing about common texts through a sequence of specific, thought-provoking and text-dependent questions (including, when applicable, questions about illustrations, charts, diagrams, audio/video and media).

3. Writing from Sources: Routinely expects that students draw evidence from texts to produce clear and coherent writing that informs, explains or makes an argument in various written forms (notes, summaries, short responses or formal essays).

4. Academic Vocabulary: Focuses on building students’ academic vocabulary in context throughout instruction.

Page 23: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

23

Criteria for Dimension II: Key Shifts in the CCSS

A unit or longer lesson should:

5. Increasing Text Complexity: Focus students on reading a progression of complex texts drawn from the grade-level band. Provide text-centered learning that is sequenced, scaffolded and supported to advance students toward independent reading of complex texts at the college- and career-ready level.

6. Building Disciplinary Knowledge: Provide opportunities for students to build knowledge about a topic or subject through analysis of a coherent selection of strategically sequenced, discipline-specific texts.

7. Balance of Texts: Within a collection of grade-level units a balance of informational and literary texts is included according to guidelines in the CCSS (p. 5).

8. Balance of Writing: Include a balance of on-demand and process writing (e.g., multiple drafts and revisions over time) and short, focused research projects, incorporating digital texts where appropriate.

Page 24: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

24

Dimension Rating and Descriptive Scales To Synthesize Judgment

Rating Scale for Dimensions I–IV:

3: Meets most to all of the criteria in the dimension

2: Meets many of the criteria in the dimension

1: Meets some of the criteria in the dimension

0: Does not meet the criteria in the dimension

Descriptors for Dimensions I–IV:

3: Exemplifies CCSS Quality — meets the standard described by criteria in the dimension, as explained in criterion-based observations

2: Approaching CCSS Quality — meets many criteria but will benefit from revision in others, as suggested in criterion-based observations

1: Developing toward CCSS Quality — needs significant revision, as suggested in criterion-based observations

0: Not representing CCSS Quality — does not address the criteria in the dimension

Page 25: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

25

EXAMPLE:Step 3. Apply Criteria in Dimension II: Key Shifts in the CCSS

The lesson/unit addresses key shifts in the CCSS:

1. Reading Text Closely: Makes reading text(s) closely, examining textual evidence and discerning deep meaning a central focus of instruction.

2. Text-Based Evidence: Facilitates rich and rigorous evidence-based discussions and writing about common texts through a sequence of specific, thought-provoking and text-dependent questions (including, when applicable, questions about illustrations, charts, diagrams, audio/video and media).

3. Writing from Sources: Routinely expects that students draw evidence from texts to produce clear and coherent writing that informs, explains or makes an argument in various written forms (notes, summaries, short responses or formal essays).

4. Academic Vocabulary: Focuses on building students’ academic vocabulary in context throughout instruction.

Page 26: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

26

EXAMPLE:Step 3. Apply Criteria in Dimension II: Key Shifts in the CCSS

A unit or longer lesson should:

5. Increasing Text Complexity: Focus students on reading a progression of complex texts drawn from the grade-level band. Provide text-centered learning that is sequenced, scaffolded and supported to advance students toward independent reading of complex texts at the college- and career-ready level.

6. Building Disciplinary Knowledge: Provide opportunities for students to build knowledge about a topic or subject through analysis of a coherent selection of strategically sequenced, discipline-specific texts.

7. Balance of Texts: Within a collection of grade-level units a balance of informational and literary texts is included according to guidelines in the CCSS (p. 5).

8. Balance of Writing: Include a balance of on-demand and process writing (e.g., multiple drafts and revisions over time) and short, focused research projects, incorporating digital texts where appropriate .

Page 27: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

27

EXAMPLE:Step 3. Apply Criteria in Dimension II: Key Shifts in the CCSS

Observations/Feedback and Rating

The unit is structured throughout to support students in reading texts closely. The questions and activities make text the focus of instruction and develop a clear lesson progression that develops skills and causes students to search for textual evidence. The texts work well together to deepen content knowledge about the women’s rights movement.

As students move from the EBC worksheets to writing a developed claim, they engage in a process of writing their ideas and refining and developing these ideas over time as they read several texts.

Importantly, there are missed opportunities to develop academic vocabulary.

Rating: 2Approaching CCSS Quality — meets many criteria but will benefit from revision in others, as suggested in criterion-based observations

Page 28: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

28

EXAMPLE:Step 3. Apply Criteria in Dimension II: Key Shifts in the CCSS

Compare Criterion-Based Checks, Observations/Feedback and Rating

• What is the pattern within our team in terms of the criteria we have checked?

• Do our observations and feedback reference the criteria and evidence (or lack of evidence) in the instructional materials?

• Do our ratings correspond to the rating and descriptors on the rubric?

Page 29: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

29

Criteria for Dimension III: Instructional Supports

The lesson/unit is responsive to varied student learning needs:

1. Cultivates student interest and engagement in reading, writing and speaking about texts.

2. Addresses instructional expectations and is easy to understand and use.

3. Provides all students with multiple opportunities to engage with text of appropriate complexity for the grade level; includes appropriate scaffolding so that students directly experience the complexity of the text.

4. Focuses on challenging sections of text(s) and engages students in a productive struggle through discussion questions and other supports that build toward independence.

5. Integrates appropriate supports in reading, writing, listening and speaking for students who are English language learners, have disabilities or read well below the grade level text band.

6. Provides extensions and/or more advanced text for students who read well above the grade level text band.

Page 30: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

30

Criteria for Dimension III: Instructional Supports

A unit or longer lesson should:

7. Include a progression of learning where concepts and/or skills advance and deepen over time (may be more applicable across the year or several units).

8. Gradually remove supports, requiring students to demonstrate their independent capacities (may be more applicable across the year or several units).

9. Provide for authentic learning, application of literacy skills, student-directed inquiry, analysis, evaluation and/or reflection.

10. Integrate targeted instruction in such areas as grammar and conventions, writing strategies, discussion rules, and all aspects of foundational reading for grades 3–5.

11. Indicate how students are accountable for independent reading based on student choice and interest to build stamina, confidence and motivation (may be more applicable across the year or several units).

12. Use technology and media to deepen learning and draw attention to evidence and texts as appropriate.

Page 31: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

31

Dimension Rating and Descriptive Scales To Synthesize Judgment

Rating Scale for Dimensions I–IV:

3: Meets most to all of the criteria in the dimension

2: Meets many of the criteria in the dimension

1: Meets some of the criteria in the dimension

0: Does not meet the criteria in the dimension

Descriptors for Dimensions I–IV:

3: Exemplifies CCSS Quality — meets the standard described by criteria in the dimension, as explained in criterion-based observations

2: Approaching CCSS Quality — meets many criteria but will benefit from revision in others, as suggested in criterion-based observations

1: Developing toward CCSS Quality — needs significant revision, as suggested in criterion-based observations

0: Not representing CCSS Quality — does not address the criteria in the dimension

Page 32: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

32

EXAMPLE:Step 3. Apply Criteria in Dimension III: Instructional Supports

The lesson/unit is responsive to varied student learning needs:

1. Cultivates student interest and engagement in reading, writing and speaking about texts.

2. Addresses instructional expectations and is easy to understand and use. 3. Provides all students with multiple opportunities to engage with text of appropriate

complexity for the grade level; includes appropriate scaffolding so that students directly experience the complexity of the text.

4. Focuses on challenging sections of text(s) and engages students in a productive struggle through discussion questions and other supports that build toward independence.

5. Integrates appropriate supports in reading, writing, listening and speaking for students who are English language learners, have disabilities or read well below the grade level text band.

6. Provides extensions and/or more advanced text for students who read well above the grade level text band.

Page 33: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

33

EXAMPLE:Step 3. Apply Criteria in Dimension III: Instructional Supports

A unit or longer lesson should:

7. Include a progression of learning where concepts and/or skills advance and deepen over time (may be more applicable across the year or several units).

8. Gradually remove supports, requiring students to demonstrate their independent capacities (may be more applicable across the year or several units).

9. Provide for authentic learning, application of literacy skills, student-directed inquiry, analysis, evaluation and/or reflection.

10. Integrate targeted instruction in such areas as grammar and conventions, writing strategies, discussion rules, and all aspects of foundational reading for grades 3–5.

11. Indicate how students are accountable for independent reading based on student choice and interest to build stamina, confidence and motivation (may be more applicable across the year or several units).

12. Uses technology and media to deepen learning and draw attention to evidence and texts as appropriate.

Page 34: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

34

EXAMPLE:Step 3. Apply Criteria in Dimension III: Instructional Supports

Observations/Feedback and Rating

The unit is likely to cultivate student interest and engagement as it communicates a clear purpose to students, uses engaging texts, and provides a mix of teacher modeling, pair/group work and independent work that will support student success with the unit’s activities and tasks.

The unit is exemplary in its commitment to having all students experience the text directly and targeting difficult/critical sections. Graphic organizers are used strategically within the instructional sequence to both support and assess students who may struggle with the reading. The quality of the questions as well as the clear progression toward independent discussion and writing are evident.

The unit would benefit from additional guidance about how to support students who are learning English and students with disabilities, as well as students who are performing above grade level. Consider including opportunities for students to make additional reading choices of their own.Rating: 2Approaching CCSS Quality — meets many criteria but will benefit from revision in others, as suggested in criterion-based observations

Page 35: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

35

EXAMPLE:Step 3. Apply Criteria in Dimension III: Instructional Supports

Compare Criterion-Based Checks, Observations/Feedback and Rating

• What is the pattern within our team in terms of the criteria we have checked?

• Do our observations and feedback reference the criteria and evidence (or lack of evidence) in the instructional materials?

• Do our ratings correspond to the rating and descriptors on the rubric?

Page 36: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

36

Criteria for Dimension IV: Assessment

The lesson/unit regularly assesses whether students are mastering standards-based content and skills:

1. Elicits direct, observable evidence of the degree to which a student can independently demonstrate the major targeted grade-level CCSS standards with appropriately complex text(s).

2. Assesses student proficiency using methods that are unbiased and accessible to all students.

3. Includes aligned rubrics or assessment guidelines that provide sufficient guidance for interpreting student performance.

A unit or longer lesson should:

4. Use varied modes of assessment, including a range of pre-, formative, summative and self-assessment measures.

Page 37: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

37

Dimension Rating and Descriptive Scales To Synthesize Judgment

Rating Scale for Dimensions I–IV:

3: Meets most to all of the criteria in the dimension

2: Meets many of the criteria in the dimension

1: Meets some of the criteria in the dimension

0: Does not meet the criteria in the dimension

Descriptors for Dimensions I–IV:

3: Exemplifies CCSS Quality — meets the standard described by criteria in the dimension, as explained in criterion-based observations

2: Approaching CCSS Quality — meets many criteria but will benefit from revision in others, as suggested in criterion-based observations

1: Developing toward CCSS Quality — needs significant revision, as suggested in criterion-based observations

0: Not representing CCSS Quality — does not address the criteria in the dimension

Page 38: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

38

EXAMPLE:Step 3. Apply Criteria in Dimension IV: Assessment

The lesson/unit regularly assesses whether students are mastering standards-based content and skills: 1. Elicits direct, observable evidence of the degree to which a student can independently

demonstrate the major targeted grade-level CCSS standards with appropriately complex text(s).

2. Assesses student proficiency using methods that are unbiased and accessible to all students.

3. Includes aligned rubrics or assessment guidelines that provide sufficient guidance for interpreting student performance.

A unit or longer lesson should: 4. Use varied modes of assessment, including a range of pre-, formative, summative and

self-assessment measures.

Page 39: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

39

EXAMPLE:Step 3. Apply Criteria in Dimension IV: Assessment

Observations/Feedback and Rating

The unit activities elicit observable evidence of students’ abilities to cite textual evidence while they are reading, speaking and writing through the Making EBCs worksheet and the final written claim that students produce.

The unit includes several tools to guide the interpretation of student performance including the Making EBCs Checklist, the Text Discussion Checklist and an Evidence-Based Writing Checklist. The notes labeled “Assessment Opportunities” provide additional guidance.

The unit assesses students using both writing and speaking.

Rating: 3

Exemplifies CCSS Quality — meets the standard described by criteria in the dimension, as explained in criterion-based observations

Page 40: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

40

EXAMPLE:Step 3. Apply Criteria in Dimension IV: Assessment

Compare Criterion-Based Checks, Observations and Feedback• What is the pattern within our team in terms of the criteria we have checked?

• Do our observations and feedback reference the criteria and evidence (or lack of evidence) in the instructional materials?

• Does our feedback include suggested improvement(s)?

Page 41: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

41

Overall Rating and Summary Comments

Overall Rating for the Lesson/Unit:

E: Exemplar – Aligned and meets most to all of the criteria in dimensions II, III, IV (total 11 – 12)

E/I: Exemplar if Improved – Aligned and needs some improvement in one or more dimensions (total 8 – 10)

R: Revision Needed – Aligned partially and needs significant revision in one or more dimensions (total 3 – 7)

N: Not Ready to Review – Not aligned and does not meet criteria (total 0 – 2)

Page 42: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

42

EXAMPLE:Step 4. Apply an Overall Rating and Provide Summary Comments

This unit is well aligned, but the instructional supports need improvement. The worksheets and questions are particularly well designed. However, it is not clear how the students learning English or students with disabilities might successfully engage in the activities. The developers might also consider ways to streamline the instructional notes for ease of use.

E/I: Exemplar if improved

Page 43: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

43

EXAMPLE:Step 5. Compare Summary Comments and Determine Next Steps

Guiding questions to synthesize criterion-based observations and suggestions: • How do the observations and suggestions for improvement compare?• Are the observations and suggestions criterion-based?• Does this example serve as a model of CCSS instruction? What are its strengths?

Areas for improvement?

Page 44: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

44

Reflection on Session Goals

• Did we use the EQuIP criteria to frame and explain evaluation of evidence found in instructional materials?

• Did we develop a common understanding of EQuIP criteria among reviewers?• Are there any criteria or evidence about which reviewers disagree?• Did we develop reviewers’ abilities to use EQuIP criteria, rating

scales/categorizations and rating descriptors to accurately rate instructional materials?

• To what degree were there differences among reviewers when checking criteria, assigning dimension ratings and assigning overall ratings? What do you think caused these differences?

Page 45: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

45

Achievewww.achieve.org1400 16th Street, NW / Suite 510 Washington, DC 20036

Page 46: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

46

Appendix:Slides for reviewing a lesson

Page 47: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

47

How To Use

The slides in this PowerPoint are currently set up to review the unit “Making Evidence-Based Claims.” To prepare for a review of the lesson “Part 1: Understanding EBCs – ‘Out of Kilter’” use the following slides that reference the lesson in the place of previous slides that reference the unit.

Page 48: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

48

EXAMPLE: Common Lesson for Review — ELA/Literacy

Grade 8 — “Ain’t I a Woman”

This state is working with vendors to create full instructional units and, ultimately, comprehensive, year-long curricular materials aligned to the CCSS. This state uses the CCSS, the Tri-State Rubrics and the Publishers’ Criteria to guide development. Teachers will be able to use these materials for classroom instruction or model their own planning after these examples.

Note: This lesson was submitted to EQuIP for review in October 2012. The lesson has been revised by the developer in response to feedback from the EQuIP quality review process. See the revised lesson/unit on the Engage NY site: www.engageny.org/resource/making-evidence-based-claims-units-ccss-ela-literacy-grades-6-12.

Page 49: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

49

EXAMPLE:Step 1. Review Materials• Record the grade and title of the lesson/unit on the Quality Review Rubric PDF

– Grade 8, “Ain’t I a Woman”• Scan to see what the lesson/unit contains and how it is organized

– Overview– A lesson with instructional notes and assessment opportunities– Checklists and rubrics– Student handouts– Model handouts

• Read key materials related to instruction, assessment and teacher guidance− Unit outline (p. 1-7)− Alignment (p. 8)− Instructional notes (p. 9–12)− Assessment (p. 12)− Text- “Ain’t I a Woman”− Student handouts (Forming EBCs, EBC Checklist, Making EBCs)

Page 50: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

50

EXAMPLE:Step 1. Review Materials• Study and measure the text(s) that serves as the centerpiece for the lesson/unit,

analyzing text complexity, quality, scope and relationship to instruction– “Ain’t I a Woman” measures 770L– The text contains complex ideas, figurative language and a distinct point of view

Page 51: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

51

EXAMPLE:Step 2. Apply Criteria in Dimension I: Alignment

The lesson/unit aligns with the letter and spirit of the CCSS: 1. Targets a set of grade-level ELA/literacy standards. 2. Includes a clear and explicit purpose for instruction. 3. Selects text(s) that measure within the grade-level text complexity band and are of

sufficient quality and scope for the stated purpose (i.e., present vocabulary, syntax, text structures, levels of meaning/purpose and other qualitative characteristics similar to CCSS grade-level exemplars in Appendices A and B).

A unit or longer lesson should: 4. Integrate reading, writing, speaking and listening so that students apply and synthesize

advancing literacy skills. 5. (Grades 3–5) Build students’ content knowledge and their understanding of reading

and writing in social studies, the arts, science or technical subjects through the coherent selection of texts.

*NOTE: Reviewers should be able to refer to evidence to support pattern of checkboxes selected.

Page 52: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

52

EXAMPLE:Step 2. Apply Criteria in Dimension I: Alignment

Observations/Feedback and Rating

This lesson clearly targets one standard (RI8.1) and includes supporting standards (RI8.2, RI8.6, SL8.1). The activities throughout the lesson correspond to this list of targeted and supporting standards.

The lesson communicates a clear purpose for instruction.

The text measure of 770L is in the 4–5 grade-level text complexity band. However, the text clearly is of sufficient quality and scope for the purpose of the unit — making claims based on textual evidence. The text is also of cultural and historical significance.

Rating: 3

Exemplifies CCSS Quality — meets the standard described by criteria in the dimension as explained by criterion-based observations

Page 53: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

53

EXAMPLE:Step 3. Apply Criteria in Dimension II: Key Shifts in the CCSS

The lesson/unit addresses key shifts in the CCSS:

1. Reading Text Closely: Makes reading text(s) closely, examining textual evidence and discerning deep meaning a central focus of instruction.

2. Text-Based Evidence: Facilitates rich and rigorous evidence-based discussions and writing about common texts through a sequence of specific, thought-provoking and text-dependent questions (including, when applicable, questions about illustrations, charts, diagrams, audio/video and media).

3. Writing from Sources: Routinely expects that students draw evidence from texts to produce clear and coherent writing that informs, explains or makes an argument in various written forms (notes, summaries, short responses or formal essays).

4. Academic Vocabulary: Focuses on building students’ academic vocabulary in context throughout instruction.

Page 54: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

54

EXAMPLE:Step 3. Apply Criteria in Dimension II: Key Shifts in the CCSS

Observations/Feedback and Rating

The lesson is structured throughout to support students in reading text closely. The questions and activities make text the focus of instruction and develop a clear lesson progression that develops skills and causes students to search for textual evidence.

The text-dependent questions are well designed, and the instructional notes set a standard for the richness and rigor that teachers should expect from student responses and dialogue. There are opportunities for students to discuss and write about common texts in response to these questions.

The lesson expects student to write from sources and draw evidence from the text. The "Forming Evidence-Based Claims" graphic organizer is very thorough and specifically guides students through the process of making a claim based on those ideas from the text.

There is little guidance about ways to develop academic vocabulary.

Rating: 2

Approaching CCSS Quality — meets many criteria but will benefit from revision in others, as suggested in criterion-based observations

Page 55: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

55

EXAMPLE:Step 3. Apply Criteria in Dimension III: Instructional Supports

The lesson/unit is responsive to varied student learning needs: 1. Cultivates student interest and engagement in reading, writing and speaking about

texts. 2. Addresses instructional expectations and is easy to understand and use. 3. Provides all students with multiple opportunities to engage with text of appropriate

complexity for the grade level; includes appropriate scaffolding so that students directly experience the complexity of the text.

4. Focuses on challenging sections of text(s) and engages students in a productive struggle through discussion questions and other supports that build toward independence.

5. Integrates appropriate supports in reading, writing, listening and speaking for students who are English language learners, have disabilities or read well below the grade level text band.

6. Provides extensions and/or more advanced text for students who read well above the grade level text band.

Page 56: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

56

EXAMPLE:Step 3. Apply Criteria in Dimension III: Instructional Supports

Observations/Feedback and Rating

The lesson is likely to cultivate student interest and engagement as it communicates a clear purpose to students, uses an engaging text, and provides a mix of teacher modeling, pair/group work and independent work that will support student success with the lesson’s activities.

The lesson is exemplary in its commitment to having all students experience the text directly and targeting difficult/critical sections of text. Graphic organizers are used strategically within the instructional sequence to both support and assess students who may struggle with the reading. The quality of the discussion questions as well as the clear progression toward independent discussion and writing are present in the unit.

While “Ain’t I a Woman” is a text that will support students reading well below the grade level, supports for English language learners, students with disabilities or students reading above grade level are not addressed.

Rating: 2

Approaching CCSS Quality — meets many criteria but will benefit from revision in others, as suggested in criterion-based observations

Page 57: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

57

EXAMPLE:Step 3. Apply Criteria in Dimension IV: Assessment

The lesson/unit regularly assesses whether students are mastering standards-based content and skills: 1. Elicits direct, observable evidence of the degree to which a student can independently

demonstrate the major targeted grade-level CCSS standards with appropriately complex text(s).

2. Assesses student proficiency using methods that are unbiased and accessible to all students.

3. Includes aligned rubrics or assessment guidelines that provide sufficient guidance for interpreting student performance.

A unit or longer lesson should: 4. Use varied modes of assessment, including a range of pre-, formative, summative and

self-assessment measures

Page 58: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

58

EXAMPLE:Step 3. Apply Criteria in Dimension IV: Assessment

Observations/Feedback and Rating

The lesson activities elicit observable evidence of students’ abilities to cite textual evidence while they are reading, speaking and writing through the Forming EBCs worksheet and the final written claim that students produce.

The lesson assessments use both speaking and writing to assess students, but accommodations for students learning English or with disabilities have not been identified. Developers should consider additional guidance regarding accessibility during assessment.

The lesson includes several tools to guide the interpretation of student performance including the Forming EBCs Checklist, the Text-Centered Discussion Checklist. The notes labeled “Assessment Opportunities” provide additional guidance.

Rating: 2

Exemplifies CCSS Quality — meets the standard described by criteria in the dimension, as explained in criterion-based observations

Page 59: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

59

EXAMPLE:Step 4. Apply an Overall Rating and Provide Summary Comments

This lesson is a strong example of CCSS alignment and quality. This lesson is well aligned, but the instructional supports need improvement. The worksheets and questions are particularly well designed. However, it is not clear how the students learning English or students with disabilities might successfully engage in the activities. The developers might also consider ways to streamline the instructional notes for ease of use.

E/I: Exemplar if improved

Page 60: EQuIP Rubric & Quality Review Training Session: ELA/Literacy Grades 3-12 1

60