New Zealand Council for Educational Research Annual Conference

Teachers Make a DifferenceWhat is the research evidence?

John Hattie

The University of AucklandOctober 2002

What are the attributes ofWhat are the attributes ofWhat are the attributes ofWhat are the attributes ofexcellent teachers?excellent teachers?excellent teachers?excellent teachers?

Identifying that which mattersIdentifying that which mattersIdentifying that which mattersIdentifying that which mattersIdentifying that which mattersIdentifying that which mattersIdentifying that which mattersIdentifying that which matters

Percentage of Achievement Variance

Students

Teachers

Home

PeersSchools Principal

Distribution of Effect-sizesDistribution of Effect-sizes

0

50

100

150

200

250

300

350

400

-1.3 -1 -0.8 -0.5 -0.3 -0.1 0.1 0.3 0.5 0.7 0.9 1.1 1.3 1.5 1.7 1.9 2.1 2.3 2.6 3

Effect-size

Ave

rage

(.4

)

Influence Effect Size Source of Influence Feedback 1.13 TeacherStudents prior cognitive ability 1.04 StudentInstructional quality 1.00 TeacherDirect instruction .82 TeacherRemediation/feedback .65 TeacherStudents disposition to learn .61 StudentClass environment .56 TeacherChallenge of Goals .52 TeacherPeer tutoring .50 TeacherMastery learning .50 TeacherParent involvement .46 HomeHomework .43 TeacherTeacher Style .42 TeacherQuestioning .41 TeacherPeer effects .38 PeersAdvance organisers .37 TeacherSimulation & games .34 TeacherComputer-assisted instruction .31 TeacherTesting .30 TeacherInstructional media .30 TeacherAims & policy of the school .24 SchoolAffective attributes of students .24 StudentPhysical attributes of students .21 StudentProgrammed instruction .18 TeacherAbility grouping .18 SchoolAudio-visual aids .16 TeacherIndividualisation .14 TeacherFinances/money .12 SchoolBehavioural objectives .12 TeacherTeam teaching .06 TeacherPhysical attributes (e.g., class size) -.05 SchoolTelevision -.12 HomeRetention -.15 School

Difference between ExpertDifference between Expertand Experienced Teachers:and Experienced Teachers:

The ReviewThe Review

Five major dimensions ofFive major dimensions ofFive major dimensions ofFive major dimensions ofFive major dimensions ofFive major dimensions ofFive major dimensions ofFive major dimensions ofexcellent teachersexcellent teachersexcellent teachersexcellent teachersexcellent teachersexcellent teachersexcellent teachersexcellent teachers

Expert teachers can:

Identify essential representations of theirsubject;Guide learning through classroominteractions;Monitor learning and provide feedback;Attend to affective attributes; andInfluence student outcomes.

A.A. Can identify essential representationsCan identify essential representationsof their subject(s)of their subject(s)

A1. Expert teachers have deeperrepresentations about teaching andlearning.

A2. Expert teachers adopt a problem-solving stance to their work.

A3. Expert teachers can anticipate, plan,and improvise as required by thesituation.

A4. Expert teachers are better decision-makers and can identify what decisionsare important and which are lessimportant decisions.

B.B. Guiding learning through classroomGuiding learning through classroominteractionsinteractions

B5. Expert teachers are proficient atcreating an optimal classroom climatefor learning.

B6. Expert teachers have amultidimensionally complex perceptionof classroom situations.

B7. Expert teachers are more context-dependent and have high situationcognition.

C.C. Monitoring learning and provideMonitoring learning and providefeedbackfeedback

C8. Expert teachers are more adept atmonitoring student problems andassessing their level of understandingand progress, and they provide muchmore relevant, useful feedback.

C9. Expert teachers are more adept atdeveloping and testing hypothesesabout learning difficulties orinstructional strategies.

C10. Expert teachers are more automatic.

D.D. Attending to affective attributesAttending to affective attributes

D11. Expert teachers have high respect forstudents.

D12. Expert teachers are passionate aboutteaching and learning.

E.E. Influencing student outcomes.Influencing student outcomes.

E13. Expert teachers engage students inlearning and develop in their studentsself-regulation, involvement in masterylearning, enhanced self-efficacy, andself-esteem as learners.

E14. Expert teachers provide appropriatechallenging tasks and goals forstudents.

E15. Expert teachers have positiveinfluences on students’ achievement.

E16. Expert teachers enhance surface anddeep learning.

The Difference betweenThe Difference betweenExpert and ExperiencedExpert and Experienced

Teachers: The StudyTeachers: The Study

Vice Chancellor’s Lecture SeriesVice Chancellor’s Lecture Series

Strategic Directions in New Zealand Education

30 October 2002University of Auckland

9.00am - 4.00 pm

Strategic Directions for New Zealand Schools - A New Framework for Public Education in theNext Decade Brian Caldwell - Dean of Education, University of Melbourne

Strategic Directions for Excellence in Teaching in NZ SchoolsDr Gary Galluzzo, Executive Vice-President of the National Board for

Professional Teaching Standards, USA

7.00 - 9.00 pm

A proposal to identify New Zealand’s best teachers. What is an accomplished teacher andhow would we know if we saw one? Dr Gary Galluzzo

Contact: mt.millet@auckland.ac.nz

The TeachersThe Teachers

Experienced teachersGroup 1 17 scored – 1.25 standard deviations from the cut scoreGroup 2 17 scored between .25 and .75 below the cut score

Expert teachersGroup 3 15 scored between .25 and .75 above the cut scoreGroup 4 16 scored + 1.25 standard deviations from the cut score

Dimension Les

son

Tra

nscr

ipt

& O

bse

rva

tion

Co

din

g

Te

ache

r In

terv

iew

Stu

den

tIn

terv

iew

s

Pre

-Ob

serv

atio

nQ

ues

tion

s

Ass

ignm

ent

Log

Stu

den

tQ

ues

tion

nair

es

Stu

den

t W

ork

Sam

ples

Wri

ting

Sa

mp

le

Use of KnowledgeDeep RepresentationsProblem SolvingImprovisationChallenge of ObjectivesClassroom ClimateMultidimensional PerceptionSensitivity to ContextMonitoring Learning and ProvidingFeedbackTest HypothesesRespect for StudentsPassion for Teaching and LearningMotivation and Self-EfficacyOutcomes of Lessons: Surface andDeepOutcomes of Lessons: Achievement

Dimensions of expertise in teaching andDimensions of expertise in teaching andmeasures for each dimension.measures for each dimension.

Teacher InterviewsTeacher Interviews

Before What did you think about as you planned? What factors influenced your planning? If one of your students had difficultly understanding (specific content from lesson

observed), what are some suggestions you could generate for helping him/her tomake connections?

After What were the most important decisions you made during today’s lessons? What influenced your lesson planning? What expectation do you have for [student’s name] How does [student’s name] approach to learning vary from day to day? Would you rate this lesson as successful? Why or why not? How else could the lesson have gone? What particular things do you want to accomplish as teacher?

Lesson TranscriptsLesson Transcripts

Analyze to determine teacher’s ability to use classroom data to define and addresslearning.

Determine the degree with which questions were used to assess skill, obtaincontrol, or exercise management in the classroom.

Determine how teachers generate specific modifications to activities that addressthe changing social and cognitive needs of students.

Coded independently based on surface and deep learning opportunities, teacherquestions and student responses to teacher, to each other, and to concepts.

Classroom ObservationClassroom Observation

Code students’ off- and on-task behaviours.

Student engagement in lesson.

Class groupings.

Management vs. instructional time.

Nature of classroom activity (e.g., development of new content, review, practice,enrichment, assessment, homework, transitional, lesson close, assigning tasks,relationships)

Code feedback – amount and nature, and from whom to whom.

Determine teacher’s ability to identify events occurring simultaneously in theclassroom.

ScenariosScenarios

It is five weeks into the school year, and you have just been assigned a new Englishclass, because the previous teacher left abruptly. The previous teacher left a gradebook with grades and attendance recorded, student information cards containingdemographic information on one side and teacher comments about the student on theother, corrected tests and homework assignments, and the text book. Question:Imagine that you have no more than 4-5 minutes before you meet the class for thefirst time, what would you plan to do in the first lesson?

Student InterviewsStudent Interviews

Tell me what you did during this lesson [Probe for examples[ What do you think your teacher wanted you to learn today? What expectations do you believe the teacher has of you?

Student SurveysStudent Surveys

Tell me what you did during this lesson [Probe for examples] What do you think your teacher wanted you to learn today? What expectations do you believe the teacher has of you?

Artifacts from the lessonsArtifacts from the lessons

Samples of student work, coded independently based on surface and deeplearning/outcomes

Grade appropriate writing prompts developed by the research team

SOLO (Structure of theSOLO (Structure of theLearning Outcome) TaxonomyLearning Outcome) Taxonomy

Unistructural Multistructural Relational Extended AbstractSurface or reproducing levelsDepend on an intention that is extrinsic to thereal purpose of the taskThese approaches can increase one’sknowledge and involve memorization andreproducing as well as the application of factsand procedures in different contexts.

Deep or transforming levelsReflect an intention to gain understanding byrelating to the task in a way that is personallymeaningful, or that links up with existingknowledge.Aim is to understand, see something in a differentway, and/or change as a person.

Capacity Student encodes the given informationand may use a recall strategyto provide an answer.

Student thinks about morethings at once.

Relationship Student responseinvolvesgeneralizing only interms of one aspect;thus, there is little orno relationshipinvolved.

Student responseinvolves relationship interms of a few limitedand independent aspects.

Student generalizeswithin a given orexperienced context.

Student generalizes tosituations not experienced.

Consistency andclosure

Student often seizeson immediate recallinformation.

Student finds and recordsfacts, can make decisionsbased on facts; studentcan use several pieces ofinformation, but thepieces remain discreet.Student can compare andcombine information.

Student works towardconsistency; student isable to see and createcoherent wholes,integrating informationinto moderatelycomplex combinations.

Student leaves room forinconsistency acrosscontext (comfortable withambiguity).Motivation is a deep-seatedconcern to achieve;involves willingness toinvest time/energy;inclination to getsatisfaction from obtaininginsights.

Structure Student uses onerelevant piece ofinformation toapproach the task.

Student uses severalrelevant pieces ofinformation to approachthe task.

Student makes use ofan underlyingconceptual structure toapproach the task.

Student demonstratesability (or even need) togeneralize concepts beyondthe original context.

Differences in Means between ExpertsDifferences in Means between Expertsand Experienced Teachersand Experienced Teachers

Percentage of Student Work classified as Surface or Deep

0

10

20

30

40

50

60

70

80

Experienced Experts

Surface Deep Surface Deep

Percentage of Student WorkPercentage of Student Workclassified as Surface or Deepclassified as Surface or Deep

Deep R

epre

senta

tions

Problem

Sol

ving

Anticip

ate a

nd Pla

n

Bette

r Dec

ision

mak

ers

Class

room

Clim

ate

Multi

dimen

sional

Per

spec

tives

Sensit

ivity

to C

onte

xt

Feedbac

k & M

onito

ring L

earn

ing

Test H

ypot

hesis

Autom

atici

ty

Respec

t for

Stu

dents

Passio

n

Engage

in le

arnin

g

Set ch

allen

ging t

asks

Enhance

surf

ace a

nd dee

p lear

ning

Effect-size0

0.2

0.4

0.6

0.8

1

1.2

Effect-sizes of differences between Expert and Experienced Teachers

Essential Representations

Guiding learningMonitoring and

FeedbackAffective attributes Influencing Student

Outcomes

Effect-sizes of differences betweenEffect-sizes of differences betweenExpert and Experienced TeachersExpert and Experienced Teachers

ChallengeChallengeSTUDENTS’ LEVELS OF CHALLENGE

Expert teachers set demanding goals rather than “do your best” goals on the basis of students’present competencies. Experts not only set challenging goals, but also structure situations so thatstudents can reach them. While competent teachers may challenge some students some of thetime, experts find ways to challenge all students to “stretch” their understanding of ideas.Students perceive assignments and activities as thought provoking as well as engaging.

Level 4: A teacher atthis level:

Level 3: A teacher atthis level:

Level 2: A teacher atthis level:

Level 1: A teacher atthis level:

• systematically andconsistentlychallengesstudents to think

• regularly promotesvaried andappropriateassignments thatare demandingand engaging foreveryone

• structures activitiesand lessons toassure that tudentscan meet thesechallenges

• often challengesstudents to thinkbut does soinconsistently

• promotes someassignments asinteresting, or asinteresting to somestudents

• offers generalSupport tostudents whoattempt to meetchallenges

• only occasionallychallengesstudents to think,some more thanothers

• promotes aperception ofassignments asnecessary, if notinteresting

• offers little supportto students to helpthem meetchallenges

• rarely challengesstudents to think

• promotes aperception ofassignments astime-consumingand tedious

• promotes a “sink orswim” mentality

Monitoring and FeedbackMonitoring and FeedbackGATHERING AND USING INFORMATION

Expert teachers monitor students’ learning by examining their responses to instruction to assess their current levels of understanding.Based on this monitoring, experts give students feedback: they offer them information about their understanding that guides them tohigher levels of comprehension. Feedback, in this dimension of expertise, is more than positive reinforcement. It is information aboutunderstanding. While competent teachers interpret student responses, experts can detect when students are not understanding. Theycan diagnose students’ interpretations and tailor the feedback they give to correct students’ misunderstandings or to help them createnew learning connections.

Level 4: A teacher at thislevel:

Level 3: A teacher at thislevel:

Level 2: A teacher at thislevel:

Level 1: A teacher at thislevel:

• consistently monitorsstudents’ engagement andredirects disengagementwith strategies thatpromote re-engagement

• regularly and accuratelyassesses students’understanding throughcareful observation,detailed analysis of theirwork, varied questions,and attentive listening totheir comments

• regularly offers feedbackthat corrects students’misunderstandings andguides students to higherlevels of understanding

• generally monitorsstudents’ engagementwith a focus on learning,encouraging students toget back on task

• interprets students’understanding throughobservation of their workand listening to theircomments

• generally reportscorrect/incorrectresponses to students andoffers feedback that helpssome students advancetheir learning

• monitors students’engagement with a focuson uniformity, maintainssmooth-runningclassrooms with a primaryemphasis on compliance

• interprets students’misunderstandings onlywhen they becomeobvious

• limits feedback to thereporting ofcorrect/incorrectresponses

• monitors students’engagement haphazardly

• often fails to recognizestudents’misunderstanding

• provides inconsistentresponses Íto students

Expert Teachers make theExpert Teachers make theDifferenceDifference

have excellent monitoring,

They challenge,

have deeper representations of classrooms,

and provide good feedback.