What Can Districts and Schools Do to Make Professional Development Work?

Andy PorterVanderbilt University

June, 2004

2

Study Design

• District and IHE Interviews – 363 district coordinators (88%); 92 IHE coordinators (87%)

• Teacher Activity Survey – teachers’ professional development experiences– 1025 teachers (72%), representing 657 activities

in 358 districts and IHEs

• Case Studies– 10 in-depth in 5 states; 6 exploratory

• Longitudinal Study of Teacher Change – teacher classroom practice– 30 schools, 10 districts, 5 states

3

Study Timeline

FallSummerSpringWinter SummerSpring WinterFall Spring Summer FallWinter Winter

Interviews

Wave 2

Survey 1

Wave 1 Wave3

1st ReportExploratory Cases

Survey 2

3rd ReportMostly

Impact onTeachers

2nd ReportMostly National

Description

19981997 1999 2000

District andIHE Interviews

Case Studies

Teacher ActivitySurvey

LongitudinalStudy ofTeacher Change

Reports

Exploratory Cases In-depth Cases In-depth Cases

4

This Presentation

• Characteristics of Effective Professional Development (PD)

• Effects of Professional Development on Changes in Teaching Practice

• District Management and Implementation Strategies Related to Effective Professional Development

5

Characteristics of Effective PD

• Identify key features of professional development

• examine the relationship between features of professional development and teacher outcomes

6

Teachers’ PD Experiences

• Enhanced knowledge and skills, and change in practice are the expected outcomes of teachers’ PD experiences

• What features might be related to these outcomes?– Sponsorship

• District vs. IHE– Structural Features

• Type: traditional vs. reform• Duration: amount (hours) and time span• Collective participation

– Core Features• Focus on content knowledge• Active learning: discussing classroom implementation, being observed, sharing

knowledge, evaluating student work• Coherence: continuity of learning, alignment, building professional community

7

Teachers are More Likely to Participate in Traditional

than Reform Types of PD

Activity Types

District

IHE/NPO

Per

cent

of

Tea

cher

s

0

5

10

15

20

25

30

35

40

45

50

In-district workshop or institute

College course

Out-of-district workshop or institute

Out-of-district conference

Study group

Teacher network

Mentoring Committee or task force

Resourcecenter

OtherInternship

Traditional Reform

3

20

30

15

24

11

0 1

36 7

2 25

2 13

1 1

9 8

45

8

Compared to Districts, IHE Activities are Longer

0

10

20

30

40

50

60

70

80

90

100

Districts

Traditional Reform

IHE/NPO

Traditional Reform

23

37

59 62

H

ours

9

Emphasize Content Knowledge More,

0

50

100

District IHE/NPO

Activity Type

Per

cen

t of

tea

cher

s re

por

tin

g m

ajor

em

ph

asis

on

con

ten

t k

now

led

ge

Reform

Traditional49 50

6865

10

Offer More Opportunities for Active Learning

Practi

ced i

n sim

ulat

ed co

nditi

ons

Receiv

ed co

achi

ng

Leade

r obs

erve

d tea

chin

g

Other

s obs

erve

d tea

chin

g

Commun

icated

with

lead

er

Stud

ents'

wor

k rev

iewed

Held fo

rmal

mee

tings

Held in

form

al m

eetin

gs

Develo

ped l

esso

n pla

ns

Types of Opportunities for Active Learning

Per

cen

t of

Tea

cher

s

District

IHE/NPO31

95

10

35

11

33

45

29

41

16 1513

54

25

55

43

53

0

50

100

11

and Are More Coherent with Teachers’ Goals and Other Activities

81

54

69

3633

88

0

10

20

30

40

50

60

70

80

90

100

Consistent with goals Builds upon earlier activities Followed-up with additional activities

Continuity of learning

Percent of Teachers

District

IHE/NPO

12

But District Activities Are More Likely To Be School-Based

18

13

11 11

0

10

20

30

40

50

All teachers in department or grade level All teachers in school

School-based participation

Percent of Teachers

District

IHE/NPO

13

PD and Teacher Outcomes

Sponsorship Structure Quality Outcomes

ControlsSchools % PovertySchool % MinorityTeacher’s GenderSubject (Math & Science)Grade Level (El, Middle, High)In-field CertificationTeaching Experience

Sponsor(IHE vs District)

Time Span

Contact Hours

Coherence

Active Learning

Type (Traditional vs

Reform)

Changein Teaching

Practice

Collective Participation

EnhancedKnowledge &

Skill

Focus onContent

Knowledge

.27

.27

.21

.08

.10

.30.33

.06

.14

.05

.09

.21

.25

.10

.08.-13

.30

.13.42

.08

.44

.10

.08

-.11

14

Summary from National Data: 6 Features of Quality Make PD More Effective

• Six Features of the Quality of Professional Development Predict Self-reported Change in Teachers Knowledge, Skills, and Teaching Practice

Structural Features Core Features

reform type (vs. tradition) focus on content knowledge

duration active learning

collective participation coherence

15

Structural Features Core Features Teacher Change

• Teacher knowledge and skills are enhanced by professional development that emphasizes content, provides opportunities for active learning, and is coherent with teaching goals, standards and assessments, and professional community.

• Longer, reform type, and school-based professional development activities are associated with content emphasis, active learning, and coherence.

16

Longitudinal Study of Teacher Change

Do teachers’ experiences in professional development activities contribute to changes in teaching practice?

17

Longitudinal Study of Teacher Change

30 schools, 10 districts, in 5 states

1 elementary, 1 middle, and 1 high school in each district

Schools were selected based on:

participation in Eisenhower-assisted activities over the 1997-98 year

diverse approaches to professional development

All mathematics and science teachers surveyed in each school

18

Sample for the Longitudinal Study of Teacher Change

17 of the sample schools, or 57%, are high-poverty, compared to the national average of 25%

287 teachers responded to all three waves of the survey

Among responding teachers:

• 74% are female and 18% are minority

• 12% of mathematics and 18% of science teachers are novices

19

Overview of Longitudinal Teacher Survey

The longitudinal survey provides...

three years of data on teaching practices (1996-97, 1997-98, and 1998-99)

• content covered

• specific teaching strategies for higher-order learning

two years of data on participation in professional development activities (1997-98 and 1998-99)

• specific teaching strategies for higher-order learning

• six quality features

20

Effects of Participation in PD on Teaching Practices: Approach

We measured: baseline teaching practice in 1996-97 characteristics of a professional development activity

teachers participated in during 1997-98

teaching practice again in 1998-99

We looked at effects on: specific teaching practices

21

Effects of Participation in PD Focused on Specific Teaching Strategies for Higher-

order Learning We asked teachers about:

Frequency of use of specific teaching strategies in mathematics and science instruction

Whether the professional development activity focused on using the strategies

We examined the effects of professional development in three areas related to higher-order learning:

Use of technology Instructional methods Student assessment

22

Technology Use for Higher-order Learning

Calculators or computers to develop models or simulations

Computers to write reports

Calculators or computers for data collection and analysis

Computers to access the Internet

23

Instructional Methods for Higher-order Learning

Work on independent, long-term (at least one week) projects Work on problems for which there is no immediately

obvious method or solution

Develop technical or mathematical writing skills, including using equations, graphs, tables, and text together

Work on interdisciplinary lessons (e.g., writing journals in class)

Debate ideas or otherwise explain their reasoning

24

Student Assessment for Higher-order Learning

Essay tests

Performance tasks or events

Systematic observation of students

Math/science reports

Math/science projects

Portfolios

25

Independent Variables

For each of the teaching strategies, we modeled the effects of three variables describing the professional development teachers received:

Whether the professional development focused on the strategy (relative focus);

Whether the professional development activity also focused on closely related topics (mean focus); and

The quality features of the professional development experience.

26

Effects on the Use of Calculators and Computers to Develop Models of Professional Development Focused on Specific

Technology-Use Strategies, and Active Learning

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

No focus on strategy, no active learning Focus on set of technology-use strategiesno active learning

Focus on set of technology-usestrategies, high active learning

Characteristics of professional development activity

Teacher's use of calculators or computers to develop models

0 = almost never1 = some lessons2 = most lessons3 = every lesson

27

Effects on the Use of Calculators and Computers to Develop Models of PD Focused on Specific Technology-Use Strategies

and Collective Participation

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1.1

1.2

1.3

1.4

1.5

No focus on strategy, no collective

participation

Focus on set of technology-use

strategies, no collective participation

Focus on set of technology-use

strategies, collective participation

Characteristics of professional development activity

Teacher's use of calculators or computers to develop models

0 = almost never1 = some lessons2 = most lessons3 = every lesson

28

Effects of Professional Development on Teaching Practice

Well Designed Professional Development is Effective

Professional development activities that focus on a specific, higher-order teaching strategy increase teachers’ use of this strategy in the classroom.

Features of high quality—reform type, collective participation, active learning, and coherence—increase the impact of professional development activities that focus on specific higher-order teaching strategies.

29

District Management and Implementation of PD

• identify key features of district management and implementation of professional development

• examine the relationship between district management and implementation and the quality of professional development

30

What Shapes District-sponsored Professional Development?

• Building a vision for professional development:– Alignment with standards and assessments– Coordination with other programs: working together and co-funding

• Implementation:– Continuous improvement: indicators, needs

assessments, evaluation, and guidance to schools and providers– Teacher participation in planning

31

Building a Vision Implementation Portfolio Features

Effects of Management and Implementation on PD Quality

ContinuousImprovement

Alignment

Targeting

Participation in “Reform”

Types Activities

Coordination(Co-funding)

Opportunities forActive Learning

Teacher Participationin Planning

Opportunities forCollective

Participation

Average Duration

Controls

District Poverty District Size Cluster Status Consortium Status

.16

.15

.15

.12

.16

.14

.40

.16

.19

.17

.20

.20

32

Summary from National District Data: Building a Vision and Implementing It• Alignment with standards and assessments and co-funding with other programs

are associated with reform types of professional development, portfolios of higher quality activities, and greater targeting.

• Continuous improvement is important for translating a vision into practice, but districts may have insufficient capacity.

• Teacher planning is linked to more opportunities for active learning and more targeting of teachers of special needs students.

33

Implications from our National Data for Improving PD

• Encourage content focus, duration, opportunities for active learning, coherence, and school-based professional development.

• Continue to emphasize alignment with standards and assessments, and co-funding with activities funded by other programs.

• Build district capacity to foster continuous improvement.

34

Summary

Large-scale change in teaching practice would require districts and schools to specify the areas of teaching in which change is desired.

Then, districts and schools would need to plan and provide professional development activities that focus on these areas, that are aligned with other reforms, and that have the 6 characteristics of effective professional development: longer, reform type, school-based activities that focus on content, provide opportunities for active learning and are coherent with teachers’ goals, standards and assessments and professional community.

35

36

Conclusions

We know what makes professional development work, and it is expensive, so it can’t be used for all teachers.

But, superintendents and principals want to reach all of their teachers, and this creates a dilemma.

There are two options:

(1) provide more money for professional development or

(2) develop a targeting strategy.

37

Citations

Porter, A.C., Garet, M.S., Desimone, L.M., & Birman, B.F. (2003, Spring). Providing effective professional development: Lessons from the Eisenhower Program. Science Educator 12(1), 23-40.

Garet, M.S., Porter, A.C., Desimone, L., Birman, B.F., & Yoon, K.S. (2001, Winter). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal 38(4), 915-945.

Desimone, L., Porter, A.C., Garet, M., Suk Yoon, K., & Birman, B. (2002, Summer). Effects of professional development on teachers’ instruction: Results from a three-year longitudinal study. Educational Evaluation and Policy Analysis 24(2), 81-112.

Desimone, L., Porter, A.C., Birman, B.F., Garet, M.S., & Suk Yoon, K. (2002, October). How do district management and implementation strategies relate to the quality of the professional development that districts provide to teachers? Teachers College Record 104(7), 1265-1312.

Desimone, L., Garet, M., Birman, B., Porter, A., & Suk Yoon, K. (in press). Improving teachers’ in-service professional development in mathematics and science: The role of postsecondary education. Education Policy.

Birman, B., Desimone, L., Porter, A.C., & Garet, M. (2002). Designing professional development that works. Educational Leadership 57(8), 28-33.

Birman, B.F., & Porter, A.C. (2002). Evaluating the effectiveness of education funding streams. Peabody Journal of Education 77(4), 59-85.