FACILITATOR’S GUIDE DRAFT - Pearson Educationassets.pearsonschool.com › asset_mgr › versions › 2013-09 › 3FD6...Review 3 day agenda Review purpose of the training Provide

LeaderSetSet. Ready. Go.

EVALUATOR TRAINING Superintendent Certification

FACILITATOR’S GUIDEDRAFT


Set. Ready. Go! © 2013 New Leaders. All rights reserved. 07172013LeaderSetSet. Ready. Go.

2

Pearson School Achievement Services Evaluator Training © 2013 Facilitator Handbook

Published by Pearson School Achievement Services, a division of Pearson, Inc. 1900 E. Lake Ave., Glenview, IL 60025

© 2013 Pearson, Inc. All rights reserved. Printed in the United States of America.

DRAFT



3

Notes to the FacilitatorCourse OverviewThis two-and-a-half day course provides an overview of the New Leaders Principal Evaluation Model, a research-based framework for effective principal practice. The course outlines the components of the comprehensive evaluation system and grounds participants in a consistent process for the collection and analysis of observational data aligned to the New Leaders rubric. As a result of the course, evaluators will develop the necessary skills to become consistent, accurate, fair and reliable evaluators of administrators, using coaching and feedback skills and strategies for the continuous development and professional growth of administrator practice that reflects the performance expectations within the evaluation rubric.

OutcomesParticipants will be able to:

� Demonstrate the knowledge and skills of a knowledgeable

and reliable evaluator of principals

� Begin to build a clear and shared vision of effective

principal practice

� Learn skills and strategies in coaching and feedback for the

continuous development of principal practice that reflects the

performance expectations contained within the rubric

DRAFT



4

Time Topic Presentation Points

8:00 - 8:45 45 minutes

Welcome

Local Context

Welcome

Presentation by “local” leadership to provide context for the training (suggest we have pre-conference to help local leadership craft the context)

� Brief testimony, importance of the work

� Expectations of principal evaluators

� Restate importance of ongoing growth and development – not a ‘gotcha’

� Policy implications and restatement of existing policies

� Local leadership discusses how their responsibilities (visiting schools, etc.) need to be

realigned to reflect expectations of evaluation system

� Rubric and evaluation system include a focus on implementing college ready standards as

well as a focus on evaluating and developing teachers

8:45 – 9:20 35 minutes

Part 1

Purpose and Overview of Training

Review 3 day agenda

Review purpose of the training

� Develop the knowledge and skills of the participants so they can become knowledgeable

and reliable evaluators of principals

� Begin to build a clear and shared vision of effective principal practice

� Learn skills and strategies (coaching and feedback) for the continuous development of

principal practice that reflects the performance expectations contained within the rubric

Overview of the training:

� Provide high level overview of the New Leaders Principal Evaluation Model

� Review the core design principles of New Leaders Principal Evaluation Model

� Understand that the rubric is to be used for continuous development of principal

practice (brief table talk – reactions to rubric based evaluation; strengths, challenges,

wonderings). Look at performance levels on rubric to illustrate the continuum

� Understand structure and purpose of Evaluation handbookHighlight the key areas of each part of handbook

Part I: Core Design PrinciplesPart III: The Evaluation ProcessPart IV: Assigning Summative RatingsPart V: Tools and Resources

DRAFT



5


Part 2

Shifts in Practice

Discuss shifts from present practice for principal and for evaluator

� (This is what we did before [we used to] …; since we have the NL rubric [now we …..])

� Complete two-column T-chart; use different T-chart or different colors to indicate

whether the shift is for evaluator OR for principal

� Debrief at tables

Annual Review timeline (provide sample)

9:20 – 10:05 45 minutes

Break 10:05 – 10:20

10:20 – 12:00 100 minutes

Part 3

Introduction to Standards and Indicators

25 min

Introduction to standards and indicators Provide 5 standards on slide Read through text for each of the following standards and corresponding indicators

1: Learning and Teaching 2: Shared Vision, School Culture and Family Engagement 3: Strategic Planning and Operations 4: Talent Management 5: Personal Leadership and Growth

30 min

Participants record response to:

� What are the skills and knowledge of an effective principal in the five standards areas?

Review responses with table partner. (7 minutes per standard)

25 min

Review standards and indicators – common language to describe principal practices (objective way to think about principal practice)

� Standards are posted on walls around the room.

� Groups rotate through each standard.

� 5 small groups each start at a different standard to record responses to these two prompts.

� Similarities in this standard to my personal experiences and knowledge

� Questions and/or wonderings I have about this standard

DRAFT



6


9:20 – 10:05 45 minutes

Break 10:05 – 10:20

10:20 – 12:00 100 minutes

Part 4

Going Deeper into Standards and Indicators

15 min

� Debrief activity by asking which standards had the most similarities between rubric and

practice? Which had the most differences? Why? In what ways does the rubric challenge

your thinking about the attributes of effective principals?

60 min

Review 5 Standards and indicators

� Discuss performance descriptors (continuum of practice)

� Look at language of indicators on 1 standard to distinguish differences in performance

levels (guided practice for group)

� Share sample scoring matrix (understand how the ratings come together)

� Evidence mapping

� Identifying evidence

� Preponderance of the evidence necessary to make a sound judgment.

� Start to think about what kinds of evidence need to be collected if I were evaluating

a principal. Provide a few minutes at the table – each person (or team) works on a

different standard to identify samples of evidence; share results.

Break 2:00 – 2:15

2:15 – 3:30 75 minutes

Part 5

Principal snapshot

Closing and Homework

60 min

� Show 6 minute video of Monarch Teaching and Learning video

� Review a few artifacts that directly relate to the video, artifacts that provide more details

about agenda, pretest, etc.

� Participants discuss video and identify evidence from principal practice that they

observed around each or any of the standards.

� Using the video as the common experience, divide the group so that each small table

group works on one of the standards to identify additional examples of practice that

represents a more comprehensive snapshot of the principal.

� Groups will post the snapshot with the additional examples of practice for each

of the standards.

� Group completes gallery walk.

Identify any ‘grey’ areas in the snapshot where there are questions about how it relates to the rubric.

DRAFT



7


15 min

� Debrief gallery walk to check for understanding. How well do participants understand

the rubric?

� Ask participants to identify evidence they already collect and how the example of

observing practice is similar to and different from their current practice.

� Q and A

Day 1Time: 8:00 – 3:30 (includes two 15 minute breaks, 45 min lunch)

6 hours 15 min instructional time

Essential Questions � How does the New Leaders Principal Evaluation System support both accountability and growth for principal practice?

� What are the skills and attributes of effective principals?

� What is evidence and how do we use it purposefully?

Objectives � Define the core principles of the evaluation model.

� Identify the skills of effective principal evaluators.

� Explain the benefits of the continuous development of principal practice.

� Explain the components of the rubric.

Facilitator Preparation � Review state principal performance standards if they exist.

DRAFT



8

Topic Presentation Points

Welcome

Local Context

Slides 1-3

The local leadership (state superintendent, regional superintendent or directors, district superintendent) provides the context for the training. The leadership should state the importance of the work and describe the organization’s expectations of principal evaluators. If applicable, they should also discuss any policy implications. The local leadership should also discuss how their responsibilities will be realigned to reflect expectations of evaluation system.

Debrief Slide 2 Welcome Activity (this can occur before or after local leadership presentation.

Local leadership presentation should highlight where to access specific information about the system requirements and responsibilities and any necessary tools, forms, and technology links (such as a state/district website on evaluation or an evaluation handbook that they have received).

Part 1

Purpose and Overview of Training

Slides 4-7

� Review 3 day agenda

� Review purpose of the training

� Provide high-level overview of the New Leaders Principal Evaluation Model

� Explain that the rubric is to be used for continuous development of principal practice

� Share and Discuss: Engage in brief table talk. Ask participants to share their reactions to rubric based

evaluation; strengths, challenges, wonderings

� Show 4 performance levels on rubric to illustrate the continuum

� Review the core design principles of New Leaders Principal Evaluation Model

Summarize the following description as you discuss that the model is built on six core design principles that we (New Leaders and Pearson) believe make it applicable and meaningful to educators and leaders across the nation. Highlight the research points.

From New Leaders research:

WHY IT’S IMPORTANT TO HAVE BETTER EVALUATION SYSTEM FOR PRINCIPALS

In offering these recommendations for change, we hope that states and districts will follow a few basic principles in the design and implementation of evaluation systems. The first is to put the task of improving student outcomes at the center. Principals should be assessed primarily on whether they are successful in improving student achievement results and other critical student outcomes. The second principle is to highlight the role of principals in improving teacher effectiveness. Because boosting the effectiveness of their teaching corps is central to what effective principals do, principals should be judged on their success in increasing the percentage of effective teachers working on behalf of their students. The third principle is to commit to the ongo¬ing professional growth of school leaders. While more rigorous evaluation will have substantial consequences for some principals

DRAFT



9


(ranging from promotion to dismissal), the vast majority of principals should experience the system as driving their effectiveness and learning, helping them to adopt and perfect the practices of successful leaders.

More details on each of these principles follow:

1. Evaluate principals on practice and outcomes:

Principals are responsible for a great many things, but no evaluation system can measure every principal responsibility. Our model proposes that a principal’s effectiveness should be measured by: (1) the actions that they demonstrate to drive higher levels of student achievement and (2) the degree to which they are successful in improving student outcomes. We call these “practice” and “outcomes.”

2. Use research on principal practice to prioritize things that matter most:

In addition to being thorough consumers of research on principal effectiveness, we have undertaken careful direct study of the practices of effective principals and we have partnered with several states and districts to support their leadership development strategies. Distilling from all of these sources and experiences, this model represents the most current knowledge on the school leader practices that are most closely tied to positive student outcomes.

3. Clearly differentiate levels of performance:

A good evaluation system makes bright-line distinctions among multiple levels of performance. We choose four levels because it allows for one category (Exemplary) to describe performance to emulate, one (Proficient) to describe performance that we expect to translate into strong results, one (Basic) to describe performance needing improvement toward proficiency, and one (Unsatisfactory) to describe performance that cannot be allowed to continue.

4. Demand a lot from evaluators:

It is our contention that good evaluation centers on powerful conversations between a principal and her manager about how to continually improve practice and about how to get better results. This, of course, is hard work and it requires evaluators to be in schools a lot – working with principals to set good goals, observing practice, and giving feedback. In many districts, that means changing time allocation for busy superintendents and assistant superintendents; from our point of view, having these “principal managers” spend more and better time evaluating and providing feedback to principals is one of the most important aspects of our model.

DRAFT



10


5. Attend to both development and accountability:

The model does two things at once. First, it serves as an accountability tool, defining effectiveness and assessing whether principals measure up to that definition. Second, it serves as a development tool, providing a structure for principals to get timely feedback and improve their practice. What binds these two things together is a significant emphasis on growth over time, whether that’s in the structure for supporting principals to continually improve practice or in setting ambitious student outcome targets.

6. Make the system ready for use:

This model is built so that any state or district can adopt it with relatively minimal additional design and, notably, with minimal cost. It is aligned to and supports the current work of principals and the supervisors who evaluate them.

� Review structure and purpose of Evaluation handbook.

Highlight the key areas of each part of handbook

Part I: Core Design Principles describes the core design principles that underlie the model. Districts and states should begin by reviewing these to assess alignment with their own priorities for educator evaluation.

Part II: Two Components outlines the two things that we recommend are measured when assessing principals and supporting their improvement: leadership practice and student outcomes. These categories encompass the most important principal actions and the most important results of their work.

Part III: The Evaluation Process outlines the evaluation process that districts should follow, from setting goals for the year to determining whether those goals were met. Also focuses on making this a “developmental” process, through more frequent school-site observations, more frequent feedback, a mid-year formative evaluation, and asking the principal to complete a self-assessment.

Part IV: Assigning Summative Ratings describes the process for assigning a rating at the end of the annual evaluation cycle.

Part V: Tools and Resources provides tools and resources to support effective implementation of the model. All of these tools and resources can be downloaded for use from our website: www.newleaders.org

� Pause and Reflect: How will you use the handbook as a resource in your work to support both growth and

accountability for principal practice?

DRAFT



11


Part 2

Shifts in Practice

Slides 8-9

� Discuss shifts from present practice for principal and for evaluator

Draw two columns T-Chart on chart paper, smart board, and laptop. Heading on the left columns (This is what we did before [we used to]) Heading on the right column (Since we have the NL rubric [now we …..])

Share and Discuss: Have participants in small groups complete two-column T-chart. Suggest they use different T-chart or different colors to indicate whether the shift is for evaluator OR for principal. Tables could divide the group in half; one group completes T-chart as shifts for evaluator; other group completes T-chart as shifts for principal.

Debrief at tables. Have groups compare results on T-chart. Solicit sample responses from each table group. Highlight key responses by recording on the T-chart at the front of the room. Focus on what is really different now – you highlight key responses and summarize the activity.

Some themes should emerge from these discussions: 1. Having clear and specific common language around principal practice 2. The need for more frequent observation and providing feedback to principals 3. Seeing evaluation as a developmental system for continuous improvement of principal practice.

Summarize activity.

Before introducing the sample timeline, make sure you have noted any comments from the T-chart debrief that address the evaluation timeline sample.

� Review sample of Annual Review timeline. Discuss how this timeline is similar to and different from the current

evaluation cycle the district uses. Reinforce any ideas that were discussed during the T-chart activity debrief.

Part 3

Introduction to Standards and Indicators

Slides 10-13

� Introduction to standards and indicators

� Slide 10 describes the components of the rubric.

Provide 5 standards on slide

Read through text for each of the following standards and corresponding indicators to get an initial sense of what is included. (Don’t go deep here – just a brief introduction. As you read the standards, think about ‘unwrapping’ the standards, highlighting the verbs – the skills – and the nouns and noun phrases – knowledge, concepts, etc.)

1: Learning and Teaching 2: Shared Vision, School Culture and Family Engagement 3: Strategic Planning and Operations 4: Talent Management 5: Personal Leadership and Growth

DRAFT



12


In the following activity, we ask participants to draw on their beliefs and experiences to describe the skills and knowledge of an effective principal in categories that align to the five standards. Next, we have participants closely read each standard in the rubric and understand how their vision of effective leadership compares to the skills and knowledge described in the rubric.

Use Slide 13 to reinforce the purpose of the standards and indicators:

� Provides lens for evaluating principals.

� Articulates new standards for proficient leadership.

� Provides common language to describe principal practice.

� Puts focus on evidence to describe level of performance.

� Orients feedback toward professional growth with descriptors.

Begin the activity by asking participants to bring their experiences and background knowledge to the effective principal conversation. Do not have participants refer to the New Leaders Rubric. Have them generate responses based on their professional experience.

Participants will brainstorm and record response to:

� What is your vision of an effective principal? What are the skills and knowledge of an effective principal in the

five standards areas? Review responses with table partner. (Allow 7 minutes per standard for this activity.)

� It may be helpful for participants to think of principals they consider effective and then to determine what

indicators they can identify that cause them to make that assessment.

Remind participants that the standards and indicators are a common language to describe principal practices and provide an objective way to think about principal practice.

� Standards (with indicators) are posted on walls around the room. Two prompts are posted: The first prompt is

on the chart with the standard (with indicators). The second prompt is on an adjacent chart paper.

1. Similarities in this standard to my personal experiences and knowledge 2. Questions and/or wonderings I have about this standard

� Taking the notes generated in the activity, groups rotate through each standard

� 5 small groups each start at a different standard to record responses to these prompts.

1. Similarities in this standard to my personal experiences and knowledge 2. Questions and/or wonderings I have about this standard

DRAFT



13


If you need additional questions to prompt conversation, consider the following:

1. In which standards did you find the most similarities to my personal experiences and knowledge? Why do you think that is?

2. In which standards did you find the most differences? Why do you think that is?

Close the activity by asking: How do the differences you discovered challenge you to think differently about principal practice? Remember that at this point, the participants are only reviewing their comments against the standards and indicators. The gallery activity continues with another round. Share a few key points to illustrate that their experiences have aligned with some qualities contained within the rubric but that additional clarity and specificity is necessary to fully benefit from the new evaluation model.

Facilitator key points:

� Many of the attributes we already use to define effective principals are found in the New Leaders rubric.

� We often use different language to describe effective principals; the New Leaders rubric gives us a common

language AND focuses primarily on evidence of performance.

� The New Leaders rubric does not list traditional principal responsibilities. This is meant to reflect the shift in

focus from principal actions to student outcomes.

� In reviewing leadership frameworks as part of the development of the Principal Effectiveness Rubric, the goal

was not to create a principal evaluation that would try to be all things to all people. Rather, the rubric focuses

unapologetically on evaluating the principal’s role as driver of student growth and achievement through their

leadership skills and ability to manage teacher effectiveness in their buildings.

� This focus reflects a strong belief that if a principal is evaluated highly on this particular instrument, he/she will

likely be effective in areas not explicitly touched upon in the rubric, such as school safety or school operations.

� This is not to say that principals should not be evaluated in these other areas. In fact, schools and districts that

elect to utilize the rubric are encouraged to add or develop additional indicators.

� Any additions should supplement, not supplant, the indicators already outlined in the rubric.

DRAFT



14


Part 4

Going Deeper into Standards and Indicators

Slides 14 - 25

Debrief Learning Activity

� Show the group a slide that contains the Standards, indicators, elements, and four levels of performance. While

we don’t expect them to be able to read all of the words on the screen, we are peeling back another layer of the

rubric, highlighting the standards and indicators that the participants used in the last activity as well as highlighting

the elements and the four levels of performance, as we prepare for another cycle of the gallery walk.

� Post the proficient description for each of the elements. The ‘posters’ on the wall now are: Standards, Indicators,

participants’ response to the last activity, and now we have added the element and proficient descriptions.

� Now that participants have explored their beliefs about principal practice, they need to become familiar with

the proficient indicator language in each leadership standard and then calibrate the practice described in the

rubric with the skills and knowledge that they brainstormed.

Each group should do the following:

� Go to one standard

� Read the proficient description for the indicators and elements in the standard.

� Explain the activity by asking “Where do you see links between the skills and knowledge the participants

identified and the proficient descriptions (from the New Leaders rubric) that you just read?” Do the same as

you move from standard to standard. Read responses that others have written before you so that you build on

the similarities and questions for each standard.

� Explain rubric components (slide 15).

� Review overall performance descriptors for Exemplary, Proficient, Basic, and Unsatisfactory.

� Share and Discuss: Allow participants a few minutes to reflect upon the descriptors. How do the descriptors

compare with their own beliefs? How do they anticipate their principals will react?

� Slide 21 – see example from Learning and Teaching standard to illustrate.

� (Slide 21) Now that the group has looked at all of the standards, you will ‘unwrap’ indicators on 1 standard

to understand the relationship between standards, indicators, elements, principal actions. Do this as a guided

practice for group.

� Note the following example from the Learning and Teaching standard. The important words and phrases are

highlighted in bold. Read through each indicator, element, and performance description, focusing on the words

in bold.

� Some examples to note:

� Principal actions to drive DRAMATIC student gains

� Support EFFECTIVE teaching

� RIGOROUS curricula (how do we define rigorous?)

� Curricula ALIGNED to college readiness standards

� Supports IMPLEMENTATION of standards-based lesson and unit plans

DRAFT



15


� Leads ANALYSES of standards, curricula, and assessments LINKED TO YEAR END GOALS.

� Standard 1: Learning and Teaching

� The actions a principal takes to drive dramatic student achievement gains though the development

and support of effective teaching.

Indicator: Implements rigorous curricula and assessments tied to both state and college-readiness standards

Element: Implements curricula aligned to state or national college readiness standards

Performance Level (Proficient) – Engages the leadership team and other key staff in developing, adapting,

and implementing curriculum aligned to CC and state standards to meet student learning needs

Element: Supprts the development and implementation of standard-based lesson and unit plans

Performance Level (Proficient) – Leads analyses of standards, curricula, and aligned assessments to develop

and implement standards-based lesson and unit plans linked to year-end goals; oversees revisions to unit and

lesson plans based on student outcomes.

� ‘Unwrap’ other indicators on remaining standards if time allows.

� Share sample scoring matrix (understand how the ratings come together). Briefly comment on the way an

overall evaluation score will be determined.

Review purpose and process for evidence mapping. Note the different types of evidence that are illustrated

in the rubric. Discuss the fact that a preponderance of the evidence is necessary to make a sound judgment.

Just because information or action steps are included in a plan doesn’t mean it’s actually happening. We need

to look for evidence from multiple sources (artifacts and observation) whenever possible. If no evidence

for a standard/indicator exists after first few observations, this should become a real focus for subsequent

school visits and artifact requests.

Participants will start to think about what kind of evidence needs to be collected if they were evaluating

a principal. Provide a few minutes at the table – each person (or team) works on a different standard to

identify samples of evidence.

Have tables briefly share results.

DRAFT



16


Part 5

Principal snapshot

Closing and Homework

Slides 25-27

Show 6 minute video of Monarch Teaching and Learning video (3-4 on rubric - Data talk meeting; principal observation of 3rd grade teacher).

Note: Remind audience that we would never rely on just one piece of evidence to make a judgment about a standard. We rely on a collection of evidence and then evaluate performance based on a preponderance of the evidence.

Review a few artifacts that directly relate to the video, artifacts that provide more details about agenda, pretest, etc.

Give participants an opportunity to talk through the video and identify as much evidence as possible from principal practice in the video that they observed around each or any of the standards.

Highlight the fact that the principal is effective and demonstrates proficient behaviors as indicated on the rubric.

Using the video as the common experience, divide the group so that each table works on one of the standards to identify additional examples of practice that represents a more comprehensive snapshot of the principal.

Point of clarification – they are NOT identifying additional evidence from the video and artifacts. Rather they are brainstorming possible additional evidence that they imagine could be coming from a more complete evaluation of this principal from Monarch.

The groups will complete the gallery walk.

Identify evidence you already collect. How are the examples of observing principal practice similar to and different from your current practice?

� Groups will post the additional examples of practice for each of the standards.

� Complete gallery walk.

� Identify any ‘grey’ areas in the snapshot where there are questions about how it relates to the rubric.

DRAFT



17


3:15 – 3:30

Debrief gallery walk to check for understanding. How well do participants understand the rubric and evidence of principal practice?

End the day with an opportunity for Q and A

Review HW Assignment (Slide 27)

� Participants are expected to read through the entire rubric, highlighting any questions or comments they may

want to share.

� Each table is assigned 1 standard to become an ‘expert’.

� Read through indicators, elements, and principal actions.

� Read through 4 performance levels.

� For Day 2, you will need to be prepared to discuss the kinds of principal actions expected in the standard in

response to questions from your colleagues regarding their highlighted comments.

� As we begin Day 2, each table will have an ‘expert’ that reviewed one of the standards. Other participants will

share their questions at the table and the ‘expert’ from that standard will respond to the questions.DRAFT



18

Time Topic Agenda Items

8:00 – 8:40 40 minutes

8:40 – 9:00 20 minutes

Part 1

Understanding Collection and Calibration of Evidence

Review Feedback from previous day

Complete HW assignment

� As we begin Day 2, each table will have an ‘expert’ that reviewed one of the standards.

Other participants will share their questions at the table and the ‘expert’ for that stan-

dard will respond to the questions.

� Debrief the HW assignment by answering any lingering questions.

� Review of artifacts, labeled by standard

� Process and use of common language

� Rationale

� Review of “Identifying Evidence graphic organizer”

� Calibration across evaluators – why this is important

� Review continuum of evidence across sample indicators for one standard (10 min)

� Looking for patterns within and across the evidence collected.

� How to connect the evidence to patterns of teaching and learning and how leader-

ship actions are connected to these outcomes.

� Evidence comes from a variety of sources – observation, indirect observation, arti-

facts, student data, etc. Case studies will illustrate the multiple data points to build a

robust picture of a principal’s practice.

� Share sample scoring matrix to understand how the ratings come together

Observation and Data Gathering (artifacts)

� Direct Observation

� Indirect Observation

� Review sample artifacts

Overview of actionable feedback (Discuss the purpose of actionable feedback)

9:00 – 9:10 10 minutes

Part 2

Guided Practice #1

9:00 – 10:40 one hour and 40 minutes

Guided and Independent Practice

� Explain the processes - Case studies will be analyzed to align evidence to performance

levels on rubrics; Reconcile differences in scoring the case studies; Purpose of calibration.

� Small groups of participants score case study, enter ratings, debrief and calibrate

DRAFT



19


9:10 – 9:40 30 minutes

9:40 – 10:00 20 minutes

Break 10:05 – 10:20 15 minutes

10:20 – 10:40 20 minutes

Guided Practice (Small groups of participants score case study, enter ratings, debrief and calibrate) Observation #1 (Video vignette - Talent Management – Basic – Beach Court Elementary– collection of evidence)

Debrief Observation (15 minutes) Record rating (score the quality of the evidence observed in the video using rating on rubric)

Observation #1 debriefing and rating calibration

10:40 – 11:25 45 minutes

11:25 – 11:30 5 minutes

11:30 – 11:45 15 minutes

11:45 – 12:00 15 minutes

Part 3

Guided Practice #2

10:40-12:00 80 minutes

Observation #2 (Vignette on Vision, Culture, & Families – Proficient - including artifacts, Indirect Observation – need to confirm this once case is selected.)

Observation #2 data entry (rating on rubric)

Observation #2 debriefing and rating calibration

Reflection on process – what’s working well, where do participants have questions?

12:00 – 12:45 LUNCH

Lunch (this may be a working lunch so folks begin to review the first case study with plenty of time, as much as 90 minutes).

DRAFT



20


12:45 – 1:45 60 minutes

1:45 – 2:15 30 minutes

Break 2:15-2:30 15 minutes

2:30 – 2:40 10 minutes

2:40 – 3:30 50 minutes

Part 4

Independent Practice #1

12:45-3:45

Independent Practice (Individual participants score case study, enter ratings, debrief and calibrate to reconcile differences in scoring the case studies)

Observation #3 (Full Case Study – evidence from all 5 standards at an overall performance level (proficient) but different standards may represent different performance levels.)

Case Study #1 Read case study. Participants should think about the following questions:

What principal leadership actions do I ‘see’ in the case study? Where do the leadership actions align with the rubric? How does the evidence help me assess the principal’s practice?

Map evidence from case study using Case Study Evidence Map Calibrate evidence – model best practice for evidence collection


Reveal master score Observation #3 debriefing and rating calibration

3:30 – 3:45 15 minutes

End of day reflection, Wrap up and next steps

DRAFT



21



Essential Questions � What is evidence and how do we use it meaningfully and purposefully?

� Why are multiple forms and sources of data necessary for a fair and rigorous evaluation?

� Why is it important that we calibrate across evaluators?

Objectives � Define the types of evidence and why each is important in the evaluation process

� Understand the key principles for evidence collection

� Define the characteristics of effective actionable feedback

� Evaluate and calibrate evidence for sample case studies

DRAFT



22


Part 1

Understanding Collection and Calibration of evidence

Slide 28 – 40

60 minutes

Review Feedback from previous day Solicit sample responses from each table from homework:

� What was their response to the kinds of principal actions expected in the standard they reviewed?

� Have participants offer responses as well as the facilitator.

Understanding Collection of evidence

Understanding Collection of evidence (slide 29)

� Rationale

� It is important for evaluators and principals to gather a preponderance of evidence to make fair and accurate

judgments about a principal’s practice.

� It’s important to consider how to connect the evidence collected to teaching and learning patterns, and how

leadership actions are connected to these outcomes.

� We recommend multiple data sources, gathered over time, to build a robust and comprehensive picture of a

principal’s practice.

� Calibration of evidence and use of common language

� Evaluators need to use a common language to describe principal practice Once we can use common lan-

guage to describe what has been collected as evidence, then analysis of the evidence—looking for patterns

within and across the evidence collected—should follow logically.

� You may want to provide some examples of evidence. Here are some for the standard, Shared Vision,

School Culture & Family Engagement. (NOTE – THIS WILL BE INCLUDED IN PARTICIPANT HAND-

BOOK SO THEY CAN CATEGORIZE THE EXAMPLES OF EVIDENCE.)

� Written values and beliefs reflect high expectations for all students

� School vision is clearly articulated and understood by all staff

� School vision includes a focus on student academic achievement and health

social/emotional development

� There is visible alignment between school goals, the instructional program, and the vision

� School-wide code of conduct aligned with district and school priorities is in place and consistently implemented across all classrooms

DRAFT



23


Observation and Data Gathering (slide 30)

� Evidence collection comes in different forms. It can be notes from an observation, indirect observation, artifacts,

school data, etc. This is brief description of the types of evidence collected as part of the evaluation process.

Detailed information is provided on slide #32. Evidence collected in these four areas should inform both the

professional growth needs of a principal and the performance ratings as part of an evaluation system. This is just

the general introduction – we go deeper on Direct Observation shortly.

� A comprehensive effort to gather evidence of leadership practice includes four things:

1. Direct observation of principal practice 2. Indirect observation of principal practice 3. Artifacts 4. School data

Key principles for evidence collection (slide 31)

� The discipline of description is the core practice for any observation and it must be practiced.

� Observable evidence is mapped to the rubric to demonstrate the level of principal effectiveness.

� Evidence should include multiple forms and a variety of sources to develop a complete picture of the

principal’s practice.

� Search for a cause and effect relationship between what we observe the principal and teachers doing and the

student achievement outcomes of those observable practices.

� Take written notes when observing.

Slide 32 A comprehensive effort to gather evidence of leadership practice includes four things:

1. Direct observation of principal practice occurs when the evaluator is physically present in the school or venue where the principal is present and leading. These observations include but are not limited to leadership team meetings, principals observing teacher practice, or principal to teacher feedback conversations.

2. Indirect observation of principal practice occurs when the evaluator is observing or reviewing systems or processes that have been developed and implemented by the principal but operate without the principal present. These indirect observations include but are not limited to attending teacher team meetings or collaboration sessions (where the principal is not present) or observing teacher practice across multiple classrooms.

3. Artifacts documenting principal practice include but are not limited to the strategic school plan, documentation of the school’s instructional framework, and communications to families and community members.

DRAFT



24


4. School data are concrete results of a principal’s work, including but not limited to leading indicators, direct evidence of student performance, and all stakeholder feedback. Evidence collected in these four areas should inform both the professional growth needs of a principal and summative ratings as part of an evaluation system (see the following description of using the rubric for both development and accountability).

Slide 33 Illustrates another key principle for evidence collection. (the other key principles are on slide 31). This is an example of strong and weak evidence. Example supports Learning and Teaching, Monitors multiple forms of student level data to inform instructional and intervention decisions. Note the P = Principal; T1 = Teacher 1, T2 = Teacher 2. Ask participants which type of evidence this represents.

Slide 34 Illustrates the importance of note taking during observations. It then illustrates making a judgment about the observation. Present a different note taken during an observation such as “principal discusses data with teachers”. How would this note lead to a different judgment?

Highlight the description of judgment – and the purpose of judgment in providing feedback to improve performance.

Judgment is a conclusion based on what you see. Ultimately you will make a judgment, but you need specific evidence to give principals concrete, actionable feedback.

Slide 35 Show the Identifying Evidence graphic organizer that we will use during the case study. The purpose of this is to illustrate the idea that we ‘map’ evidence to the standards. Evidence may support more than one standard.

DRAFT



25


Slide 36 Review continuum of evidence across sample indicators for one standard, Learning and Teaching

� Ask participants to look for patterns within and across the evidence collected.

� It’s important to consider how to connect the evidence collected to teaching and learning patterns, and how

leadership actions are connected to these outcomes.

� Evidence comes from a variety of sources – observation, indirect observation, artifacts, school data, etc. Case

studies will illustrate the multiple data points to build a robust picture of a principal’s practice.

Slide 37 Presents 3 misconceptions about principal evaluation. Ask participants, with a shoulder partner, to discuss why each of the three is a misconception.

Some key points to mention:

1. More than a portfolio of evidence – the evidence needs to illustrate principal actions as described in the rubric. A prepondernance of evidence is needed to make a judgement about quality of performance.

2. Principal evaluation is different from teacher evaluation. A principal’s practice is seen in multiple settings – providing feedback to teachers, establishing leadership teams, creating structures to monitor ongoing progress, etc. Principals interact with all teachers in the school, students, parents, district personnel, etc.

3. It will be difficult for principal evaluators to observe every indicator and element. They are not in the school every day. However, a preponderance of evidence is used to make an informed judgment about performance level for each standard.

We mentioned earlier that evidence collection comes from a variety of sources. Direct Observation, Indirect Observation, Reviewing sample artifacts and school data.

Slide 38 Presents the role that direct observation plays in the evaluation process. Observation is one way of collecting evidence about the instructional leadership work of the principal. Direct observation occurs when the evaluator is physically present in the school or venue where the principal is present and leading. These observations include but are not limited to leadership team meetings, principals observing teacher practice, or principal to teacher feedback conversations.

Later in the training, we’ll have an opportunity to practice a direct observation, using a video to calibrate our observation.

DRAFT



26


Slide 40 Overview of actionable feedback (Discuss the purpose of actionable feedback)

One shift in evaluator practice we noted yesterday was the fact that principal evaluators will be spending more time in schools conducting high-quality observations of practice and providing actionable feedback to principals. This shift is practice and priorities will result in rich, professional conversations between the evaluator and the principal, and it will help to ensure that the evaluation process does not devolve into a compliance-driven process.

� Providing frequent, clear, and specific feedback is one of the most powerful tools to support the practice of

a school leader.

� Frequent feedback helps to establish performance expectations that are grounded in the rubric.

� The principal evaluation process we have been discussing creates the conditions conducive to providing

meaningful feedback to principals systematically.

� Actionable feedback includes ongoing monitoring for the changed practice.

� Stress the point that frequent, actionable feedback – with ongoing monitoring for changed practice – is a key

driver of improvement, whether we’re talking about students, teachers, or principals.

� Actionable feedback –

� It’s not overly general.

� It’s not critical of the person but is rather being specific about a behavior they can change.

� The feedback results in measurable and/or observable action steps.

� The ‘grain size’ is just right and doable; it’s not too large of a grain size.

Insert table of examples of actionable feedback versus feedback that is not actionable/effective.

� For feedback to be meaningful, it must be:

1. Timely 2. Specific 3. Actionable 4. Continuous throughout the year

Slide 41 5 Common Mistakes in providing feedback. Left column is the common mistake made when providing feedback. Right column describes the more effective way to provide feedback.

DRAFT



27


Slide 42 Balance accountability and growth

� Deep examination is accomplished through deeper collection of evidence throughout the year that addresses

particular areas of need for the principal, coaching, and support.

� Broad assessment of practice - principals are evaluated against all 5 standards with an expectation for continuous

growth and improvement, holding everyone accountable to the new evaluation model.

Slide 43 Continuum: Identifying Problems and Specific Actions

� Once the principal supervisor has asked some probing questions, the supervisor has a variety of approaches

based on the responsiveness and capacity of the principal.

Slide 44 Feedback Continuum

� Reflective dialogue with evidence at the center of the conversation is the most effective way to support growth

in principal practice.

� One of the characteristics of effective feedback is that it is directed at high priority needs of the principal: it

involves the amount of information the principal can use rather than the amount we (as supervisors or principal

managers) would like to give. This concept is important when we are working with a principal who has many and

varied needs.

� Especially when we work with principals who are struggling in multiple areas of the performance expectations

of the rubric, it is important that we don’t overwhelm them; but rather, provide them with concrete, actionable

feedback they can use to improve in the highest priority areas to meet the needs of their schools. If we give

them too much to work on and overwhelm them, they might end up not improving in any areas.

� At the same time, we sometimes support principals who are strong across all areas. In a continuous growth-re-

lated evaluation system, it is important that we provide high-performing principals with areas in which they can

get even better than they are currently.

� Common Core implementation is a good example. Right now, as we begin to implement Common Core across

districts, there are shifts in the expectations for principal practice. What was “proficient” performance in the

past may not be enough to keep up with the new demands of overseeing Common Core implementation.

Let participants know that we will model the feedback process tomorrow.

DRAFT



28


Summarize this section with some of these key points:

� Direct observation is an important part of principal evaluation.

� Observing a principal’s practice is not the same as observing a teacher.

� Direct observation is one of several ways to collect evidence of a principal’s practice.

� First collect evidence, and then determine the level of performance.

� After any observation, provide actionable feedback to the principal in a timely manner.

� Feedback is a key lever for effective principal evaluation and development.

� Well planned, clear feedback models the feedback expected from principals in teacher evaluation.

� Making feedback a priority sends a message that the work of the principal is important and that your intention is

to support success.

� Feedback must be delivered in a timely manner and in a format that is understandable and clear to the person

for whom it is designed.

� Feedback can be collaborative: engage those you supervise in problem solving in areas where you would like to

see improvement.

� Feedback should be focused on communicating progress toward goals.

� Providing feedback can help principals engage more deeply in their practice and improve their understanding

regarding their own personal improvement.

Part 2

Guided Practice #1

9:00 – 10:40 one hour and 40 minutes

Guided and Independent Practice

� Explain the processes - Case studies will be analyzed to identify evidence and then align the evidence to

standards and to performance levels on rubrics.

� As part of the learning process, we will have a conversation that initially asks participants to agree to the

evidence that was collected.

� Then, we will reconcile differences in scoring the evidence and principal performance in the case studies.

� Purpose of calibration.

� Small groups of participants score case study, enter ratings, debrief and calibrate.

Slide 48 Only mentions guided practice but as you explain the processes, let participants know that we will engage in both guided and independent practice to build their familiarity with the rubric.

� Case studies (large and small) will be analyzed to align evidence to performance levels on rubrics. First we

will look at two small case studies, or vignettes, where we only look at principal performance related to one

standard. Then we will look at two full case studies, one today and one tomorrow, where we will evaluate

principal practice in all of the standards.

DRAFT



29


Slide 49 Visually describes the calibration process.

� Vignettes will be analyzed to align evidence to the rubric (this is the outcome)

� Watch video and/or review artifacts

� Discuss quality evidence (we begin without judging the evidence)

� Map the evidence to the rubric

� Record the rating

� Calibrate the evidence and the rating

Slide 50 Presents the actual process we will follow as we rate the vignette. Vignette #1, Talent Management Standard, Master Score of Basic – Beach Court Elementary– video plus collection of evidence.

The video is an example of Direct Observation.

� While viewing the video, it is to serve as simulation of a principal evaluator “visit” where the evaluator is observ-

ing the principal’s practice as direct observation. When watching these videos, try to focus on what the princi-

pal’s leadership actions are that promote teacher effectiveness

Participants will take notes on observation tool while viewing the video.

Use the “Video Viewing Tool” to take notes. As they complete the form, remind participants that it is critical to use the common language from the rubric that describes the principal practice that is observed.

Any notes they record will reflect language from the rubric that describes the principal practice that is observed.

What did you see? Come to agreement about what you observed, but do not make any judgments about the quality. The first priority is to become very clear on what was observed.

Participants will discuss the evidence collected until they agree on what they collectively observed. Before we ‘rate’ the principal’s performance, we need to agree on what was observed.

DRAFT



30


Note that we separate the agreement on the evidence and then move to judgements and scoring.

At their tables, they are to try and reach agreement on evidence observed. As facilitator, you will be monitoring the conversations and interjecting where necessary.

Now that you agree on what you have observed, map the evidence to the New Leaders Principal Evaluation Rubric. First identify where the evidence belongs (remember we are only looking at the Talent Management Standard).

Next assign the evidence a rating.

Then at their tables, they will discuss the evidence they observed in the video.

To assign a rating of principal leadership practice, the evaluator takes the following steps:

1. Review all evidence collected from watching the video

2. For the Talent Management Standard, determine the rating (exemplary, proficient, basic, or unsatisfactory) that matches the preponderance of evidence observed from the video.

� Then individually, they will “Rate” Principal practice (slide 51)

� You will inform them of the ‘master score’ and explain that the ‘master score’ was derived by a team of

educational experts with deep familiarity with the New Leaders rubric and principal evaluation.

� Debrief. Ask where their judgment of the evidence was different from the master score. Talk through the

differences, always relying on the language of the rubric to describe the evidence.

Note some possible misconceptions:

Since this collection is centered around teacher-generated pre-planning, peer observation, and peer debrief, the most common misconception might be that the collection of artifacts doesn’t really provide evidence of principal practice (there’s nothing from/about the principal in the documents). As facilitator, you should guide the discussion to how artifacts like this are evidence of systems the principal has put in place to build the capacity to increase teacher effectiveness through professional learning structures (second indicator) and develop high-performing instructional leadership team (4th indicator).

As the participants viewed the video, what did they notice about the principal’s tone and style? What kind of evidence can be extracted from how the principal conducts feedback sessions?

DRAFT



31


As you summarize, here are some key points to consider:

� Observation is one way of collecting evidence about the instructional leadership work of the principal.

� The purpose of this activity is to provide a common viewing experience to increase their familiarity with one of

the standards of the rubric and the overall rubric language and structure. The common “observation” serves as

the basis for the calibration discussion.

� The use of the rubric helps to ensure that as principal evaluators, they conduct VALID and RELIABLE observations

� Valid = accurate

� Reliable = consistent

Part 3

Guided Practice #2

10:40 – 12:00

100 minutes including 15 minutes for reflection

We will engage in a second guided practice where as the facilitator, you lead the conversation and help increase their familiarity with another standard, Vision, Culture and Families.

Slide 52 Presents the actual process we will follow as we rate the second vignette. Vignette #2 on Vision, Culture, & Families – Proficient Including artifacts, Indirect Observation

� The video is supposed to be an example of indirect observation.

� While viewing the video, it is to serve as simulation of a principal evaluator. “Indirect observation” is where the

evaluator is observing or reviewing systems or processes that have been developed and implemented by the

principal but operate without the principal present.

� Participants will take notes on observation tool while viewing the video.

� Use the “Video Viewing Tool” to take notes. As they complete the form, remind participants that the

notes they record should use the common language from the rubric to describes the principal practice

that is observed.

� What did you see? Come to agreement about what you observed, but do not make any judgments about the

quality. The first priority is to become very clear on what was observed.

� Participants will discuss the evidence collected until they agree on what they collectively observed. Before we

‘rate’ the principal’s performance, we need to agree on what was observed.

� At their tables, they are to try and reach agreement on evidence observed. As facilitator, you will be monitoring

the conversations and interjecting where necessary.

� Now that you agree on what you have observed, map the evidence to the New Leaders Principal Evaluation

Rubric. First identify where the evidence belongs (remember we are only looking at the Vision, Culture &

Families Standard).

� Next assign the evidence a rating.

DRAFT



32


� Then at their tables, they will discuss the evidence they observed in the video.

� To assign a rating of principal leadership practice, the evaluator takes the following steps:

1. Review all evidence collected from watching the video

2. For the Vision, Culture, and Families standard, determine the rating (exemplary, proficient, basic, or unsatisfactory) that matches the preponderance of evidence observed from the video.

� The individually, they will “Rate” Principal practice (slide 53)

� You will inform them of the ‘master score’ and explain that the ‘master score’ was derived by a team of

educational experts with deep familiarity with the New Leaders rubric and principal evaluation.

� Debrief. Ask where their judgment of the evidence was different from the master score. Talk through the

differences, always relying on the language of the rubric to describe the evidence.

Facilitator Key Points: You may make some of these here after the second vignette, and some after the first full case study.

� Knowing the rubric at the indicator level and knowing all your evidence sources in advance will help to remove

uncertainty and subjectivity.

� Some evidence may support multiple indicators. A lack of evidence may also support a rating.

� Intentional, regular practice of evidence collection and analysis is essential to quality evaluations.

� The more practice an evaluator has with this activity, the greater the calibration will be over time among

evaluators. It’s a good idea to do this work in pairs when possible. Suggest participants consider partnerships and

the creation of critical friends group.

� Evidence must be observable and captured in non-subjective ways. As the body of evidence grows, analyzing

and mapping it to the rubric becomes a much more rigorous evaluation process. Slowing down and analyzing

evidence pushes us to deepen our shared understanding of effective principal practice.

� Every context is full of nuanced differences, and it is important to identify and collect concrete evidence

(examples) to justify ratings for each indicator. Knowing your evidence sources and, who is responsible for each,

will streamline the process.


uncertainty and subjectivity, and help to maintain a focus on description.

� Now that you have taken the participants through two observations or vignettes, with examples of evidence, we

need to provide them some time to reflect on the process.

� Ask, what’s working well in relation to the guided practice?

� Where do participants have questions? Let them know that after lunch we will look at an entire case study.

DRAFT



33


Part 5


� Independent Practice

� Explain the processes - Case studies will be analyzed to identify evidence and then align the evidence to

standards and to performance levels on rubrics

� As part of the learning process, they are continuing to practice their observation skills, note taking,

aligning evidence to the rubric, rating the case study evidence, and calibrating their responses with the

master score.

� They have looked at two small case studies or vignettes, where they rated principal performance on one

standard. Now they will look at one of the two full case studies where we will evaluate principal practice in

all of the standards.

Slide 55 Once again visually describes the calibration process.

� Case studies will be analyzed to align evidence to the rubric (this is the outcome)

� Read through the entire case study.

� Map the evidence to the rubric

� Record the rating

� Calibrate the evidence and the rating

Case Study #1 Read case study. As they read the case study, participants should think about the following questions:


Participants will take notes as they review the case study.

Encourage them to use a highlighter and identify evidence that supports the rubric as they read through the case study. As they read, they should refer to the evidence map and the rubric. Remind participants to read the narrative twice: first, to get a complete picture and, second, to look for specific evidence. They can use post-it notes, highlighters, and pens - whatever it takes to assist in the collection of evidence and to facilitate a close reading of the case.

Then, they will map the evidence on the “case study evidence map’.

Then individually, they will assign a preliminary rating for each of the standards.

DRAFT



34


Remind them of the process to assign a rating of principal leadership practice, the evaluator takes the following steps:

1. Review all evidence collected in the case study.

2. For each of the 5 standards, determine the rating (exemplary, proficient, basic, or unsatisfactory) that matches the preponderance of evidence observed from the video.

Give them the opportunity to calibrate evidence at their table using language from the rubric that describes the principal practice that is observed and reach agreement on evidence observed.

Participants, at their tables, should share and discuss their evidence for each of the elements for each of the standards.

Reaching agreement relative on each indicator will require the identification of evidence from the case study.

It can be assumed that if it is not mentioned in the profile, it is not being done.

As facilitator, you will be monitoring the conversations and interjecting where necessary. Help the group reach agreement on the evidence.

Slide 58 Now that they have had the opportunity to discuss the evidence and justify their rating, give them an opportunity to individually review their preliminary rating for each of the standards, and they can revise if necessary. Then they need to assign an overall rating for the principal.

Remind them of the scoring matrix they looked at earlier.

Exemplary (4) Proficient (3) Basic (2) Unsatisfactory (1)

� Exemplary on at least

3 Standards

� No rating below

Proficient on any

Standard

� At least Proficient on

at least 3 Standards

� No rating below Basic

on any Standard

� At least Basic on at

least 4 Standards

� Unsatisfactory on at

least 2 Standards

DRAFT



35


You will inform them of the ‘master score’.

Debrief. Slide 59. Ask where their judgment of the evidence was different from the master score.

Note from master scoring session:

For Personal Leadership, the master scorers all identified the tone of the principal communication in letters

to teachers as evidence of proficient practice. Did participants see this evidence under Learning and Teaching

(supporting staff) and Talent Management? Did anyone make the connection that they also provide evidence

of personal leadership?

Ask participants at their tables to select the Standards and/or Indicators that they would like to discuss:

� Those that have the largest discrepancy

� Those that you are confused on the final rating

Make a tally mark by the Standard and/or Indicator on the chart paper.

Then discuss the justification for the rating for the Standard(s) and/or Indicator(s) with the most tallies.

Facilitator Key Points: Highlight the remaining key points that you did not mention earlier.


uncertainty and subjectivity.

� Some evidence may support multiple indicators. A lack of evidence may also support a rating.

� Intentional, regular practice of evidence collection and analysis is essential to quality evaluations.

� The more practice an evaluator has with this activity, the greater the calibration will be over time among

evaluators. It’s a good idea to do this work in pairs when possible. Suggest participants consider partnerships and

the creation of critical friends group.

� Evidence must be observable and captured in non-subjective ways. As the body of evidence grows, analyzing,

and mapping it to the rubric becomes a much more rigorous evaluation process. Slowing down and analyzing

evidence pushes us to deepen our shared understanding of effective principal practice.

� Every context is full of nuanced differences, and it is important to identify and collect concrete evidence

(examples) to justify ratings for each indicator. Knowing your evidence sources and, who is responsible for each,

will streamline the process.


uncertainty and subjectivity, and help to maintain a focus on description.

End of day reflection, Wrap up and next steps.

DRAFT



36


8:00 – 8:10 10 minutes

Introduction Opening comments, Q and A

8:10 – 9:00

50 minutes

9:00 – 9:40 40 minutes

9:40 – 10:00 20 minutes

Part 1


110 minutes

Independent Practice (Individual participants score case study, enter ratings, debrief and calibrate to reconcile differences in scoring the case studies)

Observation #4 (Full Case Study – evidence from all 5 standards at an overall performance level (proficient) but different standards may represent different performance levels.)

Case Study #2

Read case study.

Participants should think about the following questions:


Map evidence from case study using Case Study Evidence Map


Observation #4 debriefing

One misconception to be prepared for is the perception that the video provides 100% positive evidence for personal leadership but there is also some mixed evidence in there. Be prepared to tease out if the principal in video is being as open to feedback and accepting responsibility, etc., as appears on surface.

Another misconception is possible with the mid-year self-assessment. Master scorers identified that the principal missed the opportunity to identify root causes but participants may see the self-assessment in very positive way (principal showing great resiliency and self-awareness).

While the H-OC Data Discussion Protocol could be seen as evidence of very proficient practice for Learning and Teaching (protocols in place for reflecting on data-master scorers also saw it as evidence that the students were being repeatedly blamed and that there wasn’t a link to teaching.

DRAFT



37


10:00 – 10:15 BREAK

10:15 – 10:30 15 minutes

Reveal master score Rating calibration Evidence calibration

10:30 – 11:00 30 minutes

11:00 – 11:45 45 minutes

11:45 – 12:15 30 minutes

12:15 – 1:00 LUNCH

Part 2 Actionable Feedback

Discuss the purpose of actionable feedback (formative and summative)

Review 3-step approach to feedback:

1. Make a clarifying statement 2. Express the value in statements or questions 3. Provide questions or possibility statements

Watch video of evaluator providing actionable feedback to principal

Review sample of evaluation provided to principal in Case study #3

Participants highlight specific examples of feedback; identify standards, indicators and elements (we may limit the scope of standards, indicators, and elements for this activity) that were used to provide the feedback

Critique feedback

Calibrate responses

Participants generate samples of actionable feedback for case study #4. Small groups will generate feedback for specific standards.

Share examples. Offer suggestions for improvement.

DRAFT



38


1:00 – 3:00 2 hours

Option A: If this video exists, we can debrief actionable feedback. Sample actionable feedback/debrief (from EPIC); in module 7-definition of feedback and practice opportunities

Option B: In a small group fishbowl setting, participants will ‘practice’ providing actionable feedback to principal.

Option C: Model ‘providing feedback’ section on certain elements in the case study that is the solid 2. TBD.

Debrief the feedback sessions

Complete evaluation form

Complete certification assessment

DRAFT



39



Essential Questions � What are the characteristics of actionable feedback?

� What are the steps in conducting quality school visits?

� What are some strategies you will use when you deliver this information to your colleagues?

Objectives � Define the characteristics of actionable feedback

� Model the process to provide actionable feedback

� Review sample timelines for the evaluation process

� Evaluate and calibrate evidence for case study

DRAFT



40


Part 1


110 minutes

Opening comments, Q and A

Reflect upon how our learning of the last few days is more than a checklist.

How will results in our district be improved using this new evaluation system?

Case Study #2 Read case study. As they read the case study, participants should think about the following questions:


Participants will take notes as they review the case study.

Encourage them to use a highlighter and identify evidence that supports the rubric as they read through the case study. As they read, they should refer to the evidence map and the rubric. Remind participants to read the narrative twice: first, to get a complete picture and, second, to look for specific evidence. They can use post-it notes, highlighters, and pens - whatever it takes to assist in the collection of evidence and to facilitate a close reading of the case.

Then, they will map the evidence on the “case study evidence map’.

Then individually, they will assign a rating for each of the standards.

Remind them of the process to assign a rating of principal leadership practice, the evaluator takes the following steps:

1. Review all evidence collected in the case study.

2. For each of the 5 standards, determine the rating (exemplary, proficient, basic, or unsatisfactory) that matches the preponderance of evidence observed from the video

Give them the opportunity to calibrate evidence at their table using language from the rubric that describes the principal practice that is observed and reach agreement on evidence observed. As facilitator, you may decide to do this as a whole group for one or two of the standards that were the topic of the conversation at the end of the day yesterday, slide 59, discussion of standard ratings.

DRAFT



41


Participants, at their tables, should come to agreement for each of the elements for each of the standards

Reaching agreement on each indicator will require the identification of evidence from the case study.

It can be assumed that if it is not mentioned in the profile, it is not being done.

As facilitator, you will be monitoring the conversations and interjecting where necessary.

Slide 67 They need to assign an overall rating for the principal.

Remind them of the scoring matrix they looked at earlier.

You will inform them of the ‘master score’.

Debrief. Slide 68 Ask where their judgment of the evidence was different from the master score.

Ask participants at their tables to select the Standards and/or Indicators that they would like to discuss:

� Those that have the largest discrepancy

� Those that you are confused on the final rating

Make a tally mark by the Standard and/or Indicator on the chart paper.

Then discuss the justification for the rating for the Standard(s) and/or Indicator(s) with the most tallies.

We have completed the independent practice. They will rate one more case study on their own as part of the certification at the end of the day.

Exemplary (4) Proficient (3) Basic (2) Unsatisfactory (1)

� Exemplary on at least

3 Standards

� No rating below

Proficient on any

Standard

� At least Proficient on

at least 3 Standards

� No rating below Basic

on any Standard

� At least Basic on at

least 4 Standards

� Unsatisfactory on at

least 2 StandardsDRAFT



42


Summary and Key points:

The rubric and the entire evaluation system have two purposes: Development and Accountability.

1. It informs professional growth: The standards and rubric can be used as developmental tools to help principals identify priority areas for their own professional growth that are aligned to the needs of their school. The rubric contains a detailed continuum of performance for every indicator of principal practice to serve as a guide and resource for school leaders and evaluators to talk about practice, identify specific areas for growth and development, and have language to use in describing what improved practice would be.

2. It is the basis for assigning leadership practice ratings: The standards and rubric should also be used to review all evidence of principal practice and assign ratings of the leader’s practice in relation to each standard. Evaluators and principals will review principal practice and specifically the evidence from the 4 categories (direct observation, indirect observation, artifacts, and school data). Evaluators will then complete evaluation detail at the standard level, using the detailed indicators as supporting information as needed.

Part 2 Actionable Feedback and Quality Site visits

2 hours

Slides 69 - 81

Slide 69

Discuss the purpose of actionable feedback

Actionable feedback serves formative and summative purposes. The evaluation timeline illustrates opportunities for formative and summative feedback. (see slide 76)

As part of the new evaluation system, we encourage evaluators to prioritize the evaluation process by spending more time in schools conducting high-quality observations of practice and providing actionable feedback to principals. This will enable deep, professional conversations between the evaluator and the principal, and it will ensure the evaluation process does not devolve into a compliance-driven process.

Feedback is important for learning and development.

� Be sincere and avoid sugarcoating your comments

� Focus on positive areas before discussing areas for improvement

� Focus on behaviors and performance, not the person

� Be descriptive and specific and provide examples. Avoid vague comments

� Turn negative feedback about performance into a new goal or challenge and look for evidence in the future.

� Explain the impact of the behavior on teaching and learning

DRAFT



43


Slide 70 Watch video of evaluator providing actionable feedback to principal

Record examples of:

� Probing questions from supervisor

� Who identified the problem of practice?

� How were the action steps discussed?

Slide 71 Providing clear and specific feedback is one of the most powerful tools to support the practice of a school leader.

Feedback helps to establish performance expectations and ensures that performance is grounded in the rubric.

The principal evaluation process creates the conditions conducive to provide meaningful feedback to principals systematically. For feedback to be meaningful, it must be focused, clear, and actionable.

Well-planned, clear feedback to Principal:

� Models the feedback principals provide in teacher evaluation.

� Sends a message that the work of the principal is important and that your intention is to support success.

� Must be timely, understandable and clear to the recipient

� Can be collaborative: engage those you supervise in problem solving in areas where you would like to

see improvement.

� Focused on communicating progress toward goals.

� Helps principals engage more deeply in their practice and improve their understanding regarding how to

construct pathways to improvement.

Slide 72 Review sample of evaluation provided to principal in Case study #3.

DRAFT



44


Slide 73 Participants will highlight specific examples of feedback.

They will identify standards, indicators and elements that were used to provide the feedback in Case Study #3.

At their tables, give participants an opportunity to critique the feedback provided to the principal.

Calibrate responses – identify a few areas where the group is in agreement about the quality of the feedback or an area where the feedback needs to be revised.

Slide 74 Have participants look at Case Study #4, their notes, and the evidence ratings.

Identify standards for feedback that would provide the most critical information to the principal.

In small groups, participants generate samples of actionable feedback for case study #4.

Each small group will generate feedback for specific standards.

Share examples with the larger group.

Critique and offer suggestions for improvement.

Slide 75 Option A: If this video exists, we can debrief actionable feedback. Sample actionable feedback/debrief (from EPIC); in module 7-definition of feedback and practice opportunities.

Option B: In a small group fishbowl setting, participants will ‘practice’ providing actionable feedback to principal.

Option C: Model ‘providing feedback’ section on certain elements in the case study that is the solid 2.

Debrief the feedback sessions.

Complete evaluation form.

DRAFT



45


In the video, did you observe that feedback was delivered in an encouraging and supportive manner? Note that there are a variety of coaching approaches, from reflective (best for learning if coach and coachee have the necessary skill and will or desire to change) to directive (if there is performance that must be corrected immediately, or coachee needs real support in identifying and addressing areas for improvement).

Insert matrix of different coaching approaches. If time allows, have participants quickly identify a circumstance under which each coaching approach would be beneficial.

Each feedback/coaching opportunity should close with agreements about next steps. For example, what kind of support will the evaluator (coach) provide to the principal? What have you and the principal agreed to about next steps? This needs to be clearly stated. Also, the evaluator must commit to observing for improved practice moving forward and it would be very effective to state some upcoming opportunities for a school visit to observe this improved practice.

Was this evident in the video?

Slide 76 Illustrates a sample timeline of the evaluation process.

Slide 77 Describes the four-step process for conducting quality school visits.

Evaluators gather evidence of principal practice through school site visits and observations of principals in action, especially prior to each milestone step in the continuous improvement cycle illustrated on the slide.

Further, evaluators conduct additional school site observations for principals who are new to their district, school, or the profession, or who have previously received ratings below Proficient.

Periodic, purposeful school visits offer critical opportunities for evaluators to observe, collect evidence, and analyze the work of principals to facilitate ongoing feedback, dialogue about principal practice, and continuous improvement.

Evaluators provide timely feedback after each observation or collection of evidence.

The process described on slide 77 ensures that evaluators can make the most of their school site visits and collect the evidence needed to rate principal practice and support their leadership development.

DRAFT



46


This process is designed to follow the through line from school and principal learning priorities to implementation of interventions/learning opportunities where evidence is collected and ultimately leads to professional conversation between evaluator and principal during the mid-year formative review.

The most critical step in this process is observing and recording evidence of principal practice which we have practiced throughout this training.

The evaluator should be intentional in gathering detailed, relevant evidence that specifically aligns to the principal’s student learning priorities and leadership practice priorities. For example, if the evaluator is observing classrooms for the quality of instruction and planning, evidence collection should include a review of lesson plans, professional learning plans, records of observations, and feedback for teachers. The evaluator’s follow-up should include monitoring teachers’ progress in implementing the instructional strategies recommended by the principal.

Slide 78 Provides an illustration of a sample administrator growth plan. Note the specificity of the Focus area and the SMART goal, The bottom row outlines 5 steps to be completed.

Slide 79 Planning for Presentation

NOTE – THE PARTICIPANT MATERIALS WILL PROVIDE OPPORTUNITIES FOR THE PRINCIPALS TO RESPOND TO THE MATERIAL, IDENTIFY QUESTIONS THEY MIGHT ANTICIPATE FROM THEIR COLLEAGUES, AND RECORD IDEAS THEY WANT TO REMEMBER WHEN THEY PRESENT THE CONTENT TO THEIR COLLEAGUES.

� Review training materials.

� Return to T-chart activity from Day 1 to identify key points for your presentation.

� Identify your timeline for presenting key points of the new evaluation system.

� Generate questions you anticipate from your principals.

� Share ideas with colleagues.

The following research and recommendations come directly from the New Leaders Principal Evaluation Paper. Highlight the key points that will resonate most with your audience.

DRAFT



47


WHY IT’S IMPORTANT TO HAVE BETTER EVALUATION SYSTEM FOR PRINCIPALS

In offering these recommendations for change, we hope that states and districts will follow a few basic principles in the design and implementation of evaluation systems. The first is to put the task of improving student outcomes at the center. Principals should be assessed primarily on whether they are successful in improving student achievement results and other critical student outcomes. The second principle is to highlight the role of principals in improving teacher effectiveness. Because boosting the effectiveness of their teaching corps is central to what effective principals do, principals should be judged on their success in increasing the percentage of effective teachers working on behalf of their students. The third principle is to commit to the ongoing professional growth of school leaders. While more rigorous evaluation will have substantial consequences for some principals (ranging from promotion to dismissal), the vast majority of principals should experience the system as driving their effectiveness and learning, helping them to adopt and perfect the practices of successful leaders. (NL paper p 10)

KEY FINDINGS FROM RESEARCH AND THE FIELD THAT IDENTIFIED FOUR PROBLEMS WITH CURRENT EVALUATION SYSTEMS

1. Principal evaluation systems tend not to focus enough on the right things.

First and foremost, it is uncommon for districts to directly tie principal evaluation ratings to student achievement outcomes or teacher effectiveness ratings (Reeves 2009). In addition, Goldring and her colleagues (2009) found that evaluation instruments most frequently assessed “general management,” along with several other categories of principal action: “implementing vision, relationship with parents and communities, data-based decisions, and communication skills.” By contrast, “the critical behaviors that principals perform to influence student achievement do not receive emphasis” in the vast majority of principal evaluation systems (Goldring et al., 2009, p. 34).

2. Principal evaluation systems tend not to be based on clear performance standards.

Reeves’ review of evaluation documents identifies two prevailing problems: “Either the standards themselves were ambiguous or the performance expectations were unclear” (2009, p. 4). Likewise, Goldring and her colleagues find that “assessments of principals are conducted with no clear norms or performance standards” (p. 34).

DRAFT



48


3. Principal evaluation systems are limited in the rigor of their design.

The research speaks to this point primarily in relation to evaluation instruments. The Learning Point scan, for example, found that most available instruments for assess¬ing principal performance have two problems (Condon & Clifford, 2010). First, they do not appear to have been tested for important psychometric properties, or at least there is not documentation to that effect. Second, they are at least a decade old and, therefore, are not constructed with the latest research on principal leadership in mind.

4. Principal evaluation systems are limited in the rigor of their implementation.

Moving beyond instruments, Goldring (2009) offers a broader critique of evaluation systems: “There is little discussion of psychometric properties, evaluation procedures, or evaluator training among the sampled assessment instruments and procedures… There is little consistency in how assessments are developed, which leadership standards are used, and if the measures are reliable and valid” (p. 35). Building on this observation, we know from the field and from some case study research that principal evaluation systems often suffer from the same problems that have been documented about teacher evaluation: namely, that principals and their supervisors do not invest in evaluation as a tool for systematic improvement and learning.

Based on our observations from research and practice, we offer four ideas for improving principal evaluation systems:

1. Make student outcomes and teacher effectiveness outcomes 70% of a principal’s evaluation, and base the remaining 30% on the leadership actions shown to drive better results.

2. Base the evaluation of principal managers and other central office staff primarily on student outcomes and principal effectiveness, and give principal managers the tools and skills they need to effectively balance principal accountability with professional support and development.

3. Develop performance expectations that are universally high and differentiated in ways that drive continuous improvement.

4. Ensure that the evaluation system is informed by principals and other experts and is adapted over time to reflect new understandings of the practices that contribute to increased student achievement.

Complete certification assessment

DRAFT

Documents

FACILITATOR’S GUIDE DRAFT - Pearson Educationassets.pearsonschool.com › asset_mgr › versions › 2013-09 › 3FD6...Review 3 day agenda Review purpose of the training Provide