52
1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The Urban Institute

1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

Embed Size (px)

Citation preview

Page 1: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

1

National Board Certification:Who Becomes NBPTS Certified and What

Are the Consequences for Students?

Dan Goldhaber

University of Washington and The Urban Institute

Page 2: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

2

History of the National Board• Grew out of the Carnegie Forum’s A Nation

Prepared: Teachers for the 21st Century (1986)

• Founded in 1987 with the stated mission to:– Establish high and rigorous standards for what

accomplished teachers should know and be able to do– Develop and operate a national voluntary system to

assess and certify teachers who meet these standards – Advance related education reforms for the purpose of

improving student learning in American schools

Page 3: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

3

Increase in NBPTS-Certified Teachers

303676

1064

131

705

1448

2037

0

500

1000

1500

2000

1997 1998 1999 2000

ApplicantsCertified

41%

43%

47%

52%

Page 4: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

4

Why Should We Care AboutNBPTS Certification?

• Teachers matter!• Unmeasured characteristics - Goldhaber et al. (1999), Rivkin, Hanushek, and

Kain (2001), Sanders (1997)• Academic Skills - Ehrenberg and Brewer (1994), Strauss and Sawyer (1986),

Wayne (2001), Ferguson (1991, 1998) - 1 standard deviation increase in teacher licensure scores raises student scores by .17 standard deviations

– NBPTS designed to identify “effective” teachers

• Significant expenditures on NBPTS• Rewards associated with NBPTS certification are a

significant departure from the single salary schedule– Potential long-term influence on teachers’ recruitment and career

paths

Page 5: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

5

Certification Process• Two major components of assessment:

– Portfolio: Classroom practices, teacher reflection & community involvement

• Videotaped classroom interactions, collected student work (includes assessments, instructional materials), documented work with families, community, colleagues, and professional organizations

– Assessment center exercises: Content & pedagogical knowledge tests

• Constructive responses (essays) to challenging classroom situations

• Must meet performance standard of 275 points• Certification process takes 6-12 months and is

valid for 10 years

Page 6: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

6

Expenditure on NBPTS

• By 2002, NBPTS had received over– $109 million in federal funds– $106 million from non-federal sources

• State and local expenditures– Reimbursement of assessment fees– Release time and preparation materials– One time bonus or annual salary increase

Page 7: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

7

State and Local Expenditures on NBPTS(North Carolina Example)

$2,300Reimbursement ofassessment fee

1st year Cost for Individual Certification

+Annual salary

increase $5,389

Total cost 1st year = $7,689

10 Year Cost for Individual

Certification$56,190

Total Cost to NC

(x 2,037 applicants) = $4,685,100

(x 1,064 certified) = $5,706,951

Total cost 1st year =

+

$10,392,051

10 Year Cost to NC = $61,754,610

Page 8: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

8

Some Key Questions• What factors are associated with the decision to seek

NBPTS certification?

• What factors are associated with the success of applicants to NBPTS?

• Can we judge the effectiveness of NBPTS based on student outcomes– Identification of effective teachers

– NBPTS process as professional development

• How does NBPTS certification affect teachers’ career paths– Length of time that teachers remain in the teaching profession

– Distribution across districts, schools, and students?

Page 9: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

9

Key Findings• Influences on probability of application

– Teacher demographics and test scores– Districts’ financial incentives for successful applicants

• Influences on probability of certification– Major effect of teacher race– Strong positive effects of teacher test scores

• Simulations suggest major reason for increased certification rates is change in applicant pool (stronger applicants)

• Uneven student outcome results– Stronger evidence favoring NBPTS certified teachers for younger

students and in math

Page 10: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

10

Previous Studies (1): Reasons for Seeking Certification and Perceptions of Effects

• CPRE case studies– Finds applicants in districts were motivated by:

• Desire to validate their teaching techniques and obtain leadership position, monetary incentive

• Widespread belief in benefits of NBPTS certification to schools, teachers and students

• NBPTS surveys– Most NBPTS teachers report certification process:

• Is excellent professional development (80%)• Has positively affected teaching practices (91%)• Has resulted in positive impacts on student engagement,

motivation, and achievement (69%)

Page 11: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

11

Previous Studies (2): Student Outcomes

• Bond study– NBPTS teachers outperform non-certified peers in 11 of 13

“key teaching dimensions”• Does not use pre- and post-year measures of student performance or

adjust for differences among students

• Stone study– NBPTS certified teachers are not particularly effective as

measured by TVAAS scores• No comparison group or control for student background factors,

reliance on 115% as a benchmark for “high performing”

Page 12: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

12

Data Sources

• N. C. Department of Public Instruction– Teacher and student records

• NBPTS/Educational Testing Service– NBPTS candidate teacher records

• Common Core of Data– Community and school district characteristics

• Survey of administrative offices of N.C. school districts– Incentives for NBPTS candidates and teachers

Page 13: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

13

Why North Carolina Data?

• Comprehensive accountability system allows for student teacher links– Student level (about 1 million obs/year)

• Background: race, parent education level• Subject and year-end test scores: grades 3-12

– Teacher level (over 70,000 obs/year)• Background: race, age, license, years experience• Candidates’ scores on NBPTS assessments

• Strong backing of NBPTS by Gov. Hunt– NC has about a quarter of the NBPTS teachers in U.S.

Page 14: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

14

Teacher Characteristics

TEACHERSCHARACTERISTICS

OverallSample

Did NotApply

AppliedApplied,

NotCertified

Applied,Certified

Age 41.47(10.62)

41.46**(10.65)

41.80**(8.81)

.42.13**(8.77)

41.47**(8.83)

Black .14(.35)

.14*(.35)

.13*(.35)

.21**(.41)

.04**(.20)

White .83(.38)

.83**(.38)

.85**(.36)

.76(.43)

.94(.24)

Hispanic .005(.07)

.005**(.07)

.001**(.04)

.00(.04)

.00(.05)

Average TeacherTest Score

-.009(.92)

-.015**(.92)

.17**(.90)

-.10**(.89)

.46**(.81)

Highest Degree isMasters

.26(.44)

.25**(.43)

.43**(.50)

.40(.49)

.47(.50)

*p<.10 **p<.05

Page 15: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

15

School CharacteristicsSCHOOLCHARACTERISTICS

OverallSample

Did NotApply

AppliedApplied,NotCertified

Applied,Certified

% Students Performing at orabove Grade-Level

70.39(11.63)

70.35**(11.62)

71.53**(11.77)

70.10**(12.21)

73.05**(11.08)

Exceeded Expected Growthin Student Performance

.52(.50)

.52**(.50)

.55**(.50)

.51**(.50)

.59**(.49)

Most Improved in StudentPerformance

.01(.12)

.01**(.12)

.02**(.14)

.02(.13)

.02(.15)

% of Minority Students 37.77(24.67)

37.80**(24.68)

36.71**(24.33)

39.98**(26.12)

33.23**(21.75)

Student/Teacher Ratio 14.49(5.81)

14.49**(5.84)

14.82**(4.19)

14.71*(4.92)

14.94*(3.24)

% Students on Free Lunch 31.84(19.34)

31.87**(19.34)

31.02**(19.50)

34.21**(20.36)

27.63**(17.93)

*p<.10 **p<.05

Page 16: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

16

District and Community Characteristics

DISTRICT CHARACTERISTICS OverallSample Did Not Apply Applied Applied, Not

CertifiedApplied,Certified

Current Expenditure (per pupil)5413.22(571.42)

5409.15**(570.80)

5598.46**(557.77)

5554.99**(554.52)

5644.61**(557.63)

% In districts withone-time bonus

2.90(16.78)

2.86**(13.68)

5.46**(22.72)

5.30(22.42)

5.63(23.06)

Mean value of one-time bonus15.17

(91.14)14.97**(90.50)

29.44**(127.58)

29.48(130.28)

29.39(124.68)

% In districts with salaryincrement

4.03(19.66)

4.01**(19.63)

4.98**(21.75)

4.72(21.21)

5.25(22.30)

Mean value of salary increment33.72

(226.99)33.37**(225.43)

58.38**(317.22)

53.68(305.19)

63.36(329.60)

% In districts with financialincentives

6.16(24.03)

6.10**(23.94)

9.72**(29.63)

9.39(29.18)

10.07(30.10)

% Children in Poverty14.89(7.55)

14.90*(7.55)

14.69*(7.65)

15.51**(8.20)

13.82**(6.91)

% Non-White Children22.86

(18.80)22.89**(18.81)

21.85**(18.21)

24.24**(19.66)

19.30**(16.15)

% BachelorÕs or Higher Degree17.44

(11.53)17.43

(11.51)17.25

(11.87)15.93**(11.26)

18.66**(12.34)

Median Value Housing ($)66667.31

(16918.90)66643.27*(16914.19)

67133.21*(16825.28)

65005.49**(16227.91)

69393.86**(17154.60)

*p<.10 **p<.05

Page 17: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

17

Analytic Approach

• OLS and logit models of the factors associated with– Application for NBPTS certification – The likelihood of successful certification – NBPTS assessment scores

• Fixed and random effects models to account for potential correlation of error terms across schools/districts & over time

• Bivariate probit to account for potential correlation of error terms between applicant and certification models

Page 18: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

18

Application CertificationNBPTS Assessment

Score

Teacher Test (z-) Score0.32* (0.14)

0.31** (0.14)

21.50***(6.93)

20.46***(7.38)

23.86***(3.53)

22.76***(3.52)

Black0.28***(0.079)

0.28***(0.079)

-32.51***(3.38)

-32.09***(3.41)

-25.04***(1.82)

-24.65***(1.86)

Male-1.055*** (0.078)

-1.06*** (0.079)

-13.40***(2.96)

-13.43***(3.00)

-13.35***(1.88)

-13.71***(1.89)

19982.08*** (0.14)

2.14** *(0.14)

0.16(6.27)

-2.63(5.93)

3.11(3.61)

2.88(3.41)

19992.60*** (0.16)

2.76*** (0.13)

0.25(6.83)

2.23(5.84)

3.26(4.00)

6.32*(3.35)

20002.87*** (0.19)

2.98*** (0.14)

3.03(7.69)

7.21(6.01)

6.64(4.59)

11.42***(3.45)

Yrtest980.14

(0.15)0.15

(0.15)-1.37(7.41)

-0.84(7.82)

-7.76**(3.78)

-6.95*(3.78)

Yrtest990.15

(0.14)0.17

(0.14)-6.27(7.14)

-5.19(7.59)

-10.79***(3.65)

-9.74***(3.65)

Yrtest00-0.038 (0.14)

-0.028 (0.14)

-6.78(7.09)

-5.31(7.56)

-11.72***(3.62)

-10.49***(3.62)

Selected Coefficient Estimates

Page 19: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

19

Application Model Results

• Teacher variables– Those who are younger, African-American, and/or female are

more likely to apply– Those fully licensed, and receiving a license from a North

Carolina approved teacher training institution more likely to apply

– Teacher test score and year dummy variables are significant

• Salary incentive, but not bonus, has an impact

Page 20: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

20

Certification Model Results

• Those who are younger, female, and/or with Masters degrees are more likely to obtain certification

• African-Americans far less likely to obtain certification• Teacher test score is significant, but no significant

difference in slope or intercepts across years– Suggests certification is not becoming easier

• NBPTS assessment model results parallel results from certification models

Page 21: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

21

Simulations

• Application and certification probabilities, and NBPTS assessment scores over time– For a hypothetical teacher with the same characteristics as

the 1997 sample mean– With changes in teacher test (z-) scores– For teachers who switches from an affluent to a

disadvantaged school/district

• Application probabilities with changes in district incentive variables

• Use these results to simulate costs and benefits

Page 22: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

22

Probability of Application(Teacher with 1997 Mean Characteristics)

Page 23: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

23

Simulated Application Ratesby Z-Score and Year

Page 24: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

24

Simulated Conditional Certification Rates by Z-Score and Year

Page 25: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

25

Mean Characteristics of Affluent and Disadvantaged Schools

Variable School A School B

Teacher Characteristics

% Black 11.3% 34.8%

Teacher Test (z-) Score 0.23 -0.40

% with MasterÕs 38.3% 12.4%

School Characteristics% Students Performing At orAbove Grade-Level

78.2% 43.5%

% Minority Students 24% 70%

% Free-Lunch Students 11% 30%

Community Characteristics

% Children in Poverty 13.1% 29.4%

% BachelorÕs or Higher Degree 30.8% 7.8%

Median Value Housing ($) $86,348 $45,488

Page 26: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

26

Simulated Application and Certification Rates by School and Teacher Type

75.40%

2.13%0.29%

17.50%

2.16%

43.70%

0%

10%

20%

30%

40%

50%

60%

70%

80%

Application Rate Certification Rate

School A, School A Teachers School B, School B Teachers School B, School A Teachers

Page 27: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

27

Simulated Marginal Effects of Local Salary Incentives

Page 28: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

28

So is NBPTS IdentifyingEffective Teachers?

(Using NC test scores as measure of student gains to measure teacher effectiveness)

• What are the “Right” Student Outcome Comparisons?– Certified versus non-certified teachers

• Relative effectiveness of NBPTS certified teachers

– Non-certified applicants versus non-applicants• Evidence of teacher “motivation”

– Non-certified applicants versus certified applicants• NBPTS ability to discern teacher effectiveness

– Certified applicants over time• NBPTS process as professional development

• Multiple grades and subjects

Page 29: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

29

Preliminary Evidence: 19973rd Grade Means

(Standard Deviation in Parenthesis)Reading Math

Non-ApplicantCertifiedApplicant

Non-CertifiedApplicant

Non-ApplicantCertifiedApplicant

Non-CertifiedApplicant

Beginningof Grade

Score

**138.02(8.65)

141.05**(9.01)

134.65**(8.35)

**130.15(8.12)

132.05**(8.78)

126.19**(7.11)

End ofGradeScore

**144.22(9.78)

145.96*(10.21)

139.19**(8.65)

**143.01(11.23)

144.95(11.53)

136.22**(10.62)

Growth inGradeScore

**6.28(6.74)

4.76**(5.54)

4.88(6.34)

**12.95(7.36)

13.15(7.71)

9.83**(7.61)

SampleSize (N)

74268 81 73 74466 82 73

4th Grade Means(Standard Deviation in Parenthesis)

Reading Math

Non-ApplicantCertifiedApplicant

Non-CertifiedApplicant

Non-ApplicantCertifiedApplicant

Non-CertifiedApplicant

Beginningof Grade

Score

144.07(10.06)

148.11(8.46)

142.10(10.22)

142.18(11.80)

146.56**(9.31)

140.14**(12.74)

End ofGradeScore

148.49(9.60)

151.85(7.69)

146.73(9.30)

**149.54(10.80)

150.44(8.80)

146.12**(10.86)

Growth inGradeScore

*4.56(6.05)

3.60**(5.36)

5.34**(4.95)

7.58(7.27)

4.16**(6.76)

6.58**(7.21)

SampleSize (N)

65020 62 80 65224 62 79

5th Grade Means(Standard Deviation in Parenthesis)

Reading Math

Non-ApplicantCertifiedApplicant

Non-CertifiedApplicant

Non-ApplicantCertifiedApplicant

Non-CertifiedApplicant

Beginningof Grade

Score

148.84(9.73)

151.70**(8.94)

149.41**(7.76)

148.69(10.96)

152.48**(10.46)

149.23**(9.69)

End ofGradeScore

153.05(8.91)

155.74**(9.59)

153.75*(7.92)

156.51(10.39)

160.92**(10.79)

157.62**(10.10)

Growth inGradeScore

4.32(5.72)

3.69(4.78)

4.51(4.72)

8.05(6.29)

8.10(5.05)

8.72(5.35)

SampleSize (N)

63476 89 144 63494 90 144

** p<.05 * p<.1

Page 30: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

30

Preliminary Evidence: 19983rd Grade Means

(Standard Deviation in Parenthesis)Reading Math

Non-ApplicantCertifiedApplicant

Non-CertifiedApplicant

Non-ApplicantCertifiedApplicant

Non-CertifiedApplicant

Beginning ofGrade Score

**137.86(8.50)

139.36**(8.76)

136.75**(8.08)

**130.34(7.97)

132.15**(8.21)

129.11**(7.68)

End of GradeScore

**145.78(9.68)

148.35**(8.98)

144.68**(9.19)

**142.47(11.13)

145.27**(10.86)

140.54**(11.22)

Growth inGrade Score

7.90(6.91)

8.85**(6.59)

7.61**(6.86)

**12.23(7.37)

13.19**(7.09)

11.07**(7.13)

Sample Size(N)

72555 391 629 72874 396 631

4th Grade Means(Standard Deviation in Parenthesis)

Reading Math

Non-ApplicantCertifiedApplicant

Non-CertifiedApplicant

Non-ApplicantCertifiedApplicant

Non-CertifiedApplicant

Beginning ofGrade Score

144.57(9.53)

145.50**(9.68)

145.21(9.07)

**143.47(10.90)

144.07(11.09)

145.11(10.28)

End of GradeScore

149.44(9.56)

150.37**(10.24)

149.85(9.30)

151.62(10.76)

152.76**(11.40)

152.11(10.27)

Growth inGrade Score

4.93(5.44)

4.59(5.11)

4.60(5.37)

**8.32(6.45)

8.48(7.10)

7.03**(6.10)

Sample Size(N)

58101 392 376 58291 392 380

5th Grade Means(Standard Deviation in Parenthesis)

Reading Math

Non-ApplicantCertifiedApplicant

Non-CertifiedApplicant

Non-ApplicantCertifiedApplicant

Non-CertifiedApplicant

Beginning ofGrade Score

**148.77(9.43)

150.46**(9.22)

150.08(8.86)

**149.93(10.59)

151.75**(10.18)

151.67(10.02)

End of GradeScore

**154.39(8.99)

155.80**(8.81)

155.34(8.27)

**157.59(10.08)

158.94**(10.37)

158.63(10.42)

Growth inGrade Score

*5.73(5.14)

5.86(5.29)

5.31*(4.93)

**7.87(5.79)

7.95(5.60)

7.20**(5.48)

Sample Size(N)

57206 491 543 57442 492 548

** p<.05 * p<.1

Page 31: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

31

Preliminary Evidence: 19993rd Grade Means

(Standard Deviation in Parenthesis)Reading Math

Non-ApplicantCertifiedApplicant

Non-CertifiedApplicant

Non-ApplicantCertifiedApplicant

Non-CertifiedApplicant

Beginningof Grade

Score

138.20(8.46)

139.26**(8.57)

137.54**(8.28)

**130.89(7.91)

132.01**(8.19)

130.50**(7.80)

End ofGradeScore

*146.44(9.69)

148.56**(9.54)

145.99**(9.21)

**142.97(11.08)

145.18**(10.71)

142.07**(10.72)

Growth inGradeScore

8.24(6.97)

9.27**(6.99)

8.24**(7.04)

*12.01(7.36)

13.11**(7.26)

11.28**(7.20)

SampleSize (N)

78766 1250 1178 79156 1256 1181

4th Grade Means(Standard Deviation in Parenthesis)

Reading Math

Non-ApplicantCertifiedApplicant

Non-CertifiedApplicant

Non-ApplicantCertifiedApplicant

Non-CertifiedApplicant

Beginningof Grade

Score

146.04(9.49)

147.14**(9.40)

145.95**(8.96)

142.89(10.86)

143.79**(10.86)

143.02(10.09)

End ofGradeScore

*149.56(9.35)

150.60**(9.20)

149.08**(8.75)

**152.39(10.28)

153.58**(10.44)

151.32**(9.50)

Growth inGradeScore

3.51(5.54)

3.31(5.20)

3.35(5.26)

**9.51(6.33)

9.62(6.01)

8.46**(5.68)

SampleSize (N)

48141 557 543 48414 557 544

5th Grade Means(Standard Deviation in Parenthesis)

Reading Math

Non-ApplicantCertifiedApplicant

Non-CertifiedApplicant

Non-ApplicantCertifiedApplicant

Non-CertifiedApplicant

Beginningof Grade

Score

149.51(9.43)

151.63**(9.50)

149.79**(9.32)

151.73(10.63)

153.36**(10.54)

152.05**(10.34)

End ofGradeScore

154.39(8.50)

155.62**(8.52)

154.27**(8.40)

159.38(10.01)

160.56**(9.73)

159.03**(10.08)

Growth inGradeScore

4.83(5.12)

3.95**(5.14)

4.51*(5.10)

**7.68(5.71)

7.18**(5.73)

7.19(5.31)

SampleSize (N)

47160 612 600 47408 618 601

** p<.05 * p<.1

Page 32: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

32

Student Achievement ResultsRelative to students of non-certified applicants in:• 18 of 18 cases students of NBPTS teachers had

higher end-of-year scores• 17 of 18 cases students of NBPTS teachers had

higher beginning-of-year scores• 9 of 18 cases students of NBPTS teachers had

higher growth scores6 of 9 cases in

math, across grades3 of 9 cases in reading, across grades

5 of 6 cases in 3rd grade2 of 6 cases in 4th grade2 of 6 cases in 5th grade

Page 33: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

33

Conclusions

• Strong correlation (across time) between teacher test scores and application, certification, and NBPTS assessment scores– Significant differences by race

• Clear increase over time in the inclination to apply, but little evidence that the likelihood of certification has gone up over time– More academically skilled applicants explain majority of increase

in certification rate

• Uneven student outcome results– Stronger evidence favoring NBPTS certified teachers for younger

students and in math

Page 34: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

34

Next Steps

• Multivariate analysis of effectiveness of NBPTS assessment process based on student outcomes– Identification of NBPTS teachers– NBPTS process as professional development

• Assess impact of NBPTS on teachers’ career paths– Measure the influence of NBPTS certification on length of time in

teaching profession and length of time teaching in particular districts or schools

– Determine the types of districts, schools, and students are NBPTS-certified teachers most likely to teach

– Measure the affect of district-specific NBPTS-related incentives on the mobility and distribution of NBPTS-certified teachers within the profession

Page 35: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

35

BACKUP SLIDES

Page 36: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

36

NBPTS Assessment Model Results

• Generally parallel results from certification models

• Interactions between teacher pretest score and year dummies are statistically significant– Implies a reduced correlation between pretest

scores and NBPTS assessment scores in years following 1997

Page 37: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

37

Probability of Certification (Teacher with 1997 Mean Characteristics)

33.7%

33.0%

33.8%

35.4%

32%

32%

33%

33%

34%

34%

35%

35%

36%

36%

1997 1998 1999 2000

Pro

bab

ility

*

Page 38: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

38

NBPTS Assessment Score (Teacher with 1997 Mean Characteristics)

265

265

264

268

261

262

263

264

265

266

267

268

269

1997 1998 1999 2000

Page 39: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

39

Growth in Certification Areas

7

17

2426

31

12

0

5

10

15

20

25

30

35

1998 1999 2000 2001 2002* 2003*Year

Num

ber

of C

erti

fica

tion

Are

as

*Projected

Page 40: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

40

CPRE Study

• Qualitative study to assess effects of NBPTS certification– Conducted interviews with 30 teachers, NBPTS candidates,

administrators in 5 districts in 5 different states that vary by level of incentive offered

• Finds applicants in districts were motivated by:– Desire to validate their teaching techniques– Obtain leadership position– Monetary incentive

• Strong widespread belief in benefits of NBPTS certification to schools, teachers and students

Page 41: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

41

Bond Study

• Compares NBPTS certified to non-certified teachers (a total of 65) in 2 states and D.C.– Uses teacher surveys, observations and student work to

measure 13 areas of teacher effectiveness and 2 areas of student learning

• Finds NBPTS teachers outperform non-certified peers in 11 of these “key teaching dimensions,” and produce higher achieving students as measured by student work samples– Does not use pre- and post-year measures of student

performance or adjust for differences among students

Page 42: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

42

Stone Study

• Compares NBPTS-certified teachers to non-NBPTS-certified teachers in Tennessee– The study includes only 16 NBPTS-certified teachers

• Finds that NBPTS certified teachers were not exceptionally successful in improving student achievement– The study does not use conventional techniques for

demonstrating statistical significance

– The Value-Added system in Tennessee used here to measure student achievement does not directly account for differences in students’ backgrounds

Page 43: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

43

NBPTS Surveys

• Most NBPTS teachers report certification process:– Is excellent professional development (80%)

– Has positively affected teaching practices (91%)

– Has resulted in positive impacts on student engagement, achievement, and motivation (69%)

• No comparison group

Page 44: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

44

Fixed and Random Effects Application Models

• Specification tests– Random effects models preferred over OLS (Lagrange

multiplier test)– Reject the null hypothesis of joint significance of school and

district fixed-effects being zero

• Coefficients are quite robust across model specifications– Unobserved school and district characteristics important

predictors of application, but uncorrelated with included explanatory variables

Page 45: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

45

Effect Size of Select Variables on Application

Page 46: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

46

Effect Size of Select Variables on Conditional Certification

-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

Ag

e

Page 47: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

47

Effect Size of Select Variables on Unconditional Certification

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

Ag

e

Page 48: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

48

NBPTS: Governance & Structure

• Governed by a 63-member Board of Directors– Classroom teachers– District administrators and school board members– State officials– Business and community leaders– Union officials

• Certification Areas– Offers certificates in 24 areas– Most common in generalist certificates for teachers of

multiple subjects (early childhood, middle childhood, and early adolescence)

Page 49: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

49

Demographic Information 19973rd Grade Means

(Standard Deviation in Parenthesis)Non-Applicant Certified

ApplicantNon-Certified

Applicant% Minority Students per

Teacher36.90

(25.64)36.78

(19.88)52.70

(29.53)% Title 1 Students

per Teacher36.66

(40.90)12.62

(14.59)61.70

(42.06)Current Per Pupil

Expenditure4936.62(408.71)

4933.70(180.89)

5165.83(448.74)

Sample Size (N) 83445 84 784th Grade Means

(Standard Deviation in Parenthesis)% Minority Students per

Teacher36.65

(25.66)37.94

(20.46)54.18

(26.54)% Title 1 Students

per Teacher34.41

(41.01)41.27

(44.46)62.17

(46.18)Current Per Pupil

Expenditure4941.68(412.94)

5446.41(963.83)

5189.93(181.10)

Sample Size (N) 82057 71 1165th Grade Means

(Standard Deviation in Parenthesis)% Minority Students per

Teacher35.66

(25.19)25.16

(16.32)31.74

(28.18)% Title 1 Students

per Teacher34.29

(41.86)24.42

(27.89)9.02

(25.89)Current Per Pupil

Expenditure4937.88(406.78)

5023.88(326.37)

4979.70(319.68)

Sample Size (N) 79943 99 156

Page 50: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

50

Demographic Information 19983rd Grade Means

(Standard Deviation in Parenthesis)Non-Applicant Certified

ApplicantNon-Certified

Applicant% Minority Students per

Teacher36.99

(25.59)29.97

(16.17)42.04

(30.98)% Title 1 Students

per Teacher37.79

(41.57)26.51

(31.20)44.41

(40.77)Current Per Pupil

Expenditure5264.41(454.04)

5373.49(671.08)

5291.44(386.58)

Sample Size (N) 87694 454 7194th Grade Means

(Standard Deviation in Parenthesis)% Minority Students per

Teacher36.63

(25.60)30.04

(25.05)38.87

(29.26)% Title 1 Students

per Teacher35.82

(41.70)39.45

(46.78)36.42

(39.70)Current Per Pupil

Expenditure5265.45(451.51)

5254.80(396.24)

5400.53(561.06)

Sample Size (N) 83965 496 5475th Grade Means

(Standard Deviation in Parenthesis)% Minority Students per

Teacher35.72

(25.38)29.42

(21.30)32.66

(23.45)% Title 1 Students

per Teacher34.68

(41.93)22.35

(35.54)29.59

(40.74)Current Per Pupil

Expenditure5266.30(455.39)

5285.69(643.07)

5420.57(615.36)

Sample Size (N) 81648 640 677

Page 51: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

51

Demographic Information 19993rd Grade Means

(Standard Deviation in Parenthesis)Non-Applicant Certified

ApplicantNon-CertifiedApplication

% Minority Students perTeacher

37.78(26.18)

31.66(22.23)

46.64(30.02)

% Title 1 Studentsper Teacher

40.16(42.80)

32.11(40.33)

54.40(42.57)

Current Per PupilExpenditure

5659.74(484.83)

5800.59(70.43)

5513.34(392.90)

Sample Size (N) 89115 1377 13354th Grade Means

(Standard Deviation in Parenthesis)% Minority Students per

Teacher37.27

(25.98)30.88

(21.79)40.71

(30.06)% Title 1 Students

per Teacher38.06

(43.15)24.13

(35.12)47.06

(42.87)Current Per Pupil

Expenditure8665.11(491.63)

5705.74(498.07)

5772.50(426.65)

Sample Size (N) 87337 964 10025th Grade Means

(Standard Deviation in Parenthesis)% Minority Students per

Teacher36.54

(25.90)29.91

(22.06)38.85

(22.47)% Title 1 Students

per Teacher36.90

(43.08)25.04

(40.14)38.61

(42.32)Current Per Pupil

Expenditure5660.66(489.69)

5762.28(521.07)

5717.51(691.81)

Sample Size (N) 85459 994 1034

Page 52: 1 National Board Certification: Who Becomes NBPTS Certified and What Are the Consequences for Students? Dan Goldhaber University of Washington and The

52

Simulated NBPTS Assessment Scoresby Z-Score and Year

200

220

240

260

280

300

320