In the “Eye of the Tiger” It is (mostly) about

Improvement

Improving Teaching: What we would like to happen

Num

ber o

f Te

ache

rs

“Teacher Effectiveness”

How far will personnel evaluation

take us?

Suppose We Were Evaluating 10,000 Teachers: Ideal Result

2500

2500

2500

2500

1 2 3 4

Observational rubric designation

4 3 2 1

Valu

e ad

ded

des

igna

tion

What We Will Likely See in Practice: 10,000 teachers with ρ =.30

940 680 520 350

680 690 620 530

520 620 690 670

350 530 670 940

1 2 3 4

Observation rubric designation4 3 2 1

Valu

ed a

dded

d

esig

natio

n

Lots of mixed patterns of evidence

Empirical Teacher Effectiveness Distributions: A small percent outliers… a huge group in the middle

So can we use teacher evaluation rankings to improve teaching?

• Simple, but naïve idea— “You’re category 2…improve!”

• Not mostly a problem of effort, intentions or attention.

• For most teachers it is a question of “Well, exactly how do I improve?”

• Classic Edward Deming observation…

You cannot fatten the cow by weighing it!

So what in the teacher evaluation data might actually inform improvement?

• Value added—some use for targeting but even here pretty unreliable.

• Classroom observation protocols should be of value.

• But, even here there is a complication.

Measurement For Improvement vsMeasurement for Evaluation

Measurement for personnel evaluation– Reliability is paramount– Low inference premium, clearly observable behaviors– Generality for all

• Measurement for improvement– Focus on signaling what we value– Timely, woven into the web of work– High inference OK within professional community, low

stakes– Aligned to specific work of instruction

An Instructional-System Specific Protocol

• Text Selection• Text Introduction• During the Reading• After Reading• Word Work

Guided Reading: A High Leverage Daily Activity in Grade 1-3 Comprehensive Literacy Classrooms

Five Elements Form the Practice:

Guided Reading—After Reading(one of two rubric elements)

__Does not engage

students in discussion of

the text.

__Engages children in

some discussion, but

talk is often unfocused or

off topic.

__Engages children in

some discussion of text. Some

student comments

indicate they are thinking

about the text.

__Engages children in a

rich discussion of the meaning of the text that

is evident in students

comments about their

thinking.

A General Standards-Based ProtocolInstruction: Feedback to Students

__Teacher’s feedback to students is of poor quality and not provided in a timely manner.

__Teacher’s feedback to students is uneven and its timeliness is inconsistent.

__Teacher’s feedback to students is timely and of consistently high quality.

__Teacher’s feedback to students is timely and of consistently high quality, and students make use of the feedback in their learning.

So the good news…we are beginning to focus on the right problem

–Moving out problematic teachers must happen

—truncate off the extreme negative tail.

–And it is also about improving teaching

–move the overall distribution.

But…

• It is far from clear that we are equipping districts with the right observational tools to inform such improvements.

For in the end, it is all about…