Alan Spicciati, Ed.D. Seattle Pacific University, Class of 2008

spicciad@hsd401.org

The difference between teachers one SD above and below the mean is one year’s worth of achievement (Hanushek, 1992)

Teacher effects are cumulative; three years with top vs. bottom quintile teachers opens a 54 percentile gap (Sanders & Rivers, 1996)

Rowan, Correnti, & Miller’s (2002) comprehensive study of teacher measurement methodology concluded 52%-72% of student mathematics variance lies between classrooms, with the rest between students and between schools.

A one SD increase in teacher effectiveness is equal to a reduction in class size from 25 to 15 (Nye, Konstantopoulos, & Hedges, 2004)

Experience Experience has a curvilinear relationship

with achievement. Achievement rises with experience for

between 2 and 5 years, with “on-the-job training”, then levels off (Ferguson, 1991; Darling-Hammond, 2000; Rockoff, 2004; Rivkin, Hanushek, Kane, 2005).

Advanced Degrees Master’s degrees are important in mathematics

and science in secondary (Goldhaber & Brewer, 1997; Wenglinsky, 2000).

Findings on advanced degrees are split for elementary.◦ Many studies find that advanced degrees do not relate

to elementary mathematics achievement...(Hanushek, 1986; Rivkin, Hanushek, Kain, 2005; Clotfelter, Ladd, & Vigdor, 2007).

◦ However, some reputable studies find a positive, significant relationship (Ferguson & Ladd, 1996; Greenwald, Hedges, Laine, 1996; Nye, Konstantopolous, & Hedges, 2004).

College Selectivity A teacher’s academic ability, particularly

verbal ability, is among the most established teacher variables in relation to student achievement (Hanushek, 1986; Rice, 2003).

College selectivity, often measured by Barron’s rankings, is a proxy for academic ability that is moderately related to student achievement (Wayne & Youngs, 2003).

Mathematics Courses Mathematics content knowledge, as measured

by tests of teachers, relates to achievement (Harbison & Hanushek, 1992; Hill, Rowan, & Ball, 2005).

Mathematics courses relate to math achievement in secondary (Monk & King, 1994).

However, Hill, Rowan, & Ball (2005) found there is little empirical evidence examining math courses and achievement at the elementary level, and their findings were not significant.

Teacher effectiveness. The present study is focused on “teachers,” as opposed to “teaching.” In this context, “teacher effectiveness” is defined by the mathematics achievement of a teacher’s students, as measured by growth on the Measures of Academic Progress (MAP) test, compared to expected growth. While teacher effectiveness is a term used in the literature, this will be a correlational study and will not imply effects.

1. In terms of descriptive statistics, what is the distribution of achievement growth at the classroom level?

2. Is there a significant relationship between advanced degrees, experience, college selectivity, or total mathematics courses taken at the university level and growth in mathematics achievement?

3. What combinations of the above teacher variables best explain the variance in student growth?

4. Since poor and minority communities generally attract and retain less qualified and experienced teachers than other communities, would the achievement of diverse classes be significantly higher if they had equal or even equitable access to teachers with experience and advanced degrees?

3,558 students◦ 70.7% of all students in grades 3-6◦ 84.2% of all students with complete scores,

excluding self-contained classes 156 teachers

◦ 68.7% of all teachers in grades 3-6◦ 89.7% of all eligible teachers

Required teacher variable data was located for all teachers

Measures of Academic Progress (MAP)◦ Published by Northwest Evaluation Association (NWEA)◦ Computer adaptive; item response theory◦ Multiple choice; typically 40 items◦ Measures the content strands found on the math WASL◦ Administered fall, winter, and spring

Reliability and Validity◦ Test-retest reliability: r = .88 to r = .93 ◦ Marginal reliability: r = .94◦ Concurrent validity (with state tests): r = .79 to r = .89

Permission granted by superintendent and SPU Institutional Review Board

Gathered existing data◦ MAP scores accessed in “raw” format from district

database◦ Teacher data accessed from Human Resources

Degree database contained universities and degrees Highly Qualified Teacher database contained record

of course taking Samples double checked against actual transcripts

Independent Variables Demographic

◦ Class Percent Non White (CPNW)◦ School Free and Reduced Lunch Percentage (SFRL)◦ Class Percent of English Language Learners (CPEL)

Teacher◦ Experience (EXP)◦ Experience Dichotomized (EXPDI)◦ Degree (DEGR)◦ College Selectivity (COLL)◦ Number of Mathematics Courses, Content and Pedagogy (MC)◦ Math Courses Dichotomized (MCDI)

Dependent Variable Class Percent of Expected Growth (CPEG)

◦ Fall to spring student level MAP growth, divided by NWEA expected (normal) growth, aggregated to class level

Descriptive Statistics◦ Overall◦ Disaggregated by quartile level of diversity

Correlation Multiple Regression

◦ Identification of best model for this dataset◦ Regression equation used to estimate results with

various staffing scenarios

Means and standard deviations for each variable

N Mean SD

1. Class Percentage of Expected Growth (CPEG) 156 98.9 28.3

2. Class Percentage Non-White (CPNW) 156 62.3 21.1

3. School Free and Reduced Lunch Percentage (SFRL) 156 59.5 18.7

4. Class Percentage of English Language Learners (CPEL) 156 11.8 11.5

4. Experience (EXP) 156 9.3 8.3

5. Exp. Dichotomized (EXPDI) (0 = 0-5 years; 1 = 6+ years) 156 .57 -

6. Degree (DEGR) (0 = BA; 1 = MA) 156 .58 -

7. College Selectivity (COLL) (0 = low; 1 = high) 156 .51 -

8. Num. of Math Courses, Content, and Pedagogy (MC) 156 3.4 2.1

9. Math Courses Dichotomized (MCDI) (0 = 0-3; 1 = 3.5+) 156 .45 -

Performance Least diverse

quartile grew most

Demographics Poverty and ELL

highly related to diversity

Teachers Low diversity

classes taught by more experienced teachers

Other variables have weaker relationships

1st Quartile

CPNW 5-48

M (SD)

2nd Quartile

CPNW 48-68

M (SD)

3rd Quartile

CPNW 68-78

M (SD)

4th Quartile

CPNW 79-100

M (SD)

1. CPEG 111.0 (32.0) 96.8 (28.0) 96.0 (27.0) 91.7 (22.7)

2. CPNW 32.9 (10.1) 57.1 (6.4) 73.3 (3.2) 86.2 (5.6)

3. SFRL 35.1 (13.4) 58.4 (12.3) 69.1 (8.2) 75.4 (7.7)

4. CPEL 1.7 (4.3) 9.7 (9.7) 14.8 (10.2) 21.0 (10.9)

5. EXP 13.0 (8.5) 10.6 (8.5) 6.3 (6.8) 9.2 (7.7)

6. EXPDI .77 .64 .44 .44

7. DEGR .69 .49 .51 .64

8. COLL .51 .44 .59 .49

9. MC 3.1 (2.2) 3.9 (2.4) 3.2 (1.7) 3.3 (2.0)

10. MCDI .38 .51 .41 .49

Overall n = 156; each quartile n = 39

Diversity level only explains 9% of growth

Large range of growth at every level of diversity

Many highly diverse classes outperform expected growth

Explanation Each color

represents a diversity quartile

Each bar represents 9 or 10 classrooms, grouped by growth, with average CPEG shown

Interpretation Top classrooms

in every diversity quartile outperform the average non diverse class

Variable 1 2 3 4 5 6 7 8 9 10

1. CPEG ___ -.30** -.26** -.25** .05 .17* .32** .05 -.03 .07

2. CPNW -.30** ___ .85** .63** -.35** -.30** -.02 -.01 .02 .05

3. SFRL -.26** .85** ___ .61** -.29** -.24** -.02 -.16* .07 .10

4. CPEL -.25** .63** .61** ___ -.30** -26** -.06 -.15 -.08 -.08

5. EXP .05 -.35** -.29** -.30** ___ .75** .06 -.04 .08 .09

6. EXPDI .17* -.30** -.24** -.26** .75** ___ .11 -.11 .19* .13

7. DEGR .32** -.02 -.02 -.06 .06 .11 ___ .13 -.10 -.02

8. COLL .05 -.01 -.16* -.15 -.04 -.11 .13 ___ -.10 -.01

9. MC -.03 .02 .07 -.08 .08 .19* -.10 -.10 ___ .78**

10. MCDI .07 .05 .10 -.08 .09 .13 -.02 -.01 .78** ___

** Correlation is significant at the 0.01 level (2-tailed).

* Correlation is significant at the 0.05 level (2-tailed).

Diversity explains 9% of CPEG scores

Advanced degrees and experience explain an additional approximately 9%

Experience does not significantly explain CPEG scores beyond advanced degrees

R R2 Adj. R2 R2

Change

B Sig.

Constant 110.206 .000

Set One .299 .090 .084 .090

CPNW -.372 .000

Set Two .435 .189 .173 .100

DEGR 17.575 .000

EXPDI 2.923 .507

The reduced model includes only diversity and advanced degrees

Advanced degrees explain more than 9% of variance in CPEG scores beyond what diversity explains

The model as a whole explains about 18% of the variance in CPEG scores

R R2 Adj. R2 R2

Change

B Sig.

Constant 112.998 .000

Set One .299 .090 .084 .090

CPNW -.393 .000

Set Two .433 .187 .176 .097

DEGR 17.870 .000

Explanation Using Beta weights

from the multiple regression equation, achievement levels are simulated using different allocation methods

Interpretation An equitable approach

could close achievement gap between Q1 and Q4 from 21% (in the status quo model) to 7%◦ See limitations

This approach would be more powerful with a stronger measure of teacher quality or a characteristic that varies more greatly across schools

Scenario 1st Qtile. 2nd Qtile. 3rd Qtile. 4th Qtile. Overall

State Average CPEG 111.1 101.6 95.2 86.2 99.5

DEGR .62 .62 .62 .62 .62

Equality (District Avg.) CPEG 110.4 100.9 94.6 89.5 98.9

DEGR .58 .58 .58 .58 .58

Equitable Distribution CPEG 104.2 98.4 96.2 97.0 98.9

DEGR .23 .44 .67 1.00 .58

Status Quo* CPEG 112.4 99.3 93.3 90.6 98.9

DEGR .69 .49 .51 .64 .58

* “Status Quo” is based on actual CPNW and DEGR, but computed CPEG.

Q1: Distribution of Achievement by Classroom◦ 1 SD of classroom effectiveness = nearly 3 months of growth

Q2: Teacher Characteristics and Math Achievement Growth◦ Advanced degrees

Different findings may be attributable to small “n” of colleges, local bargaining context, or methodology that cannot link individual teachers with their characteristics.

◦ Experience Findings herein consistent with research.

◦ College selectivity Data lacks variability to show results.

◦ Mathematics courses Content knowledge appears to matter, based on Hill, Rowan, & Ball

(2005), but coursework is a poor proxy. Q3: Combinations of Variables

◦ No significant interactions. Q4: Equal or Equitable Distribution of Teachers

◦ Simulations of this nature may be needed to encourage policy.

Knowing and acting on data◦ Disaggregating achievement data by classroom◦ Using responsible and ethical assessment and HR

practices◦ Engaging in courageous conversations and

leadership actions Teacher distribution and assignment

◦ Monitor teacher characteristics data to prevent neediest schools from having disproportionately inexperienced/less qualified teachers

◦ Referee student assignment to avoid repeated exposure to low performing classrooms (Sanders)

Methodology◦ Gain scores◦ Small student “n” size per teacher◦ Multiple regression vs. HLM

Internal Validity◦ Does measuring classrooms = measuring teachers?◦ Unidentified covariates

External Validity◦ Ability to generalize◦ Assumptions that teachers would perform similarly

in different situations

1. A multi-state study. 2. A study of teachers that lasted more than

one year. 3. A study of other forms of mathematics

content acquisition. 4. A study that includes variables for teachers

who took a remedial mathematics course or who failed a mathematics course.

5. A qualitative study of teachers whose students significantly outperform.

Alan Spicciati, Ed.D. Seattle Pacific University, Class of 2008

spicciad@hsd401.org