J Child Fam StudDOI 10.1007/s10826-017-0932-0

ORIGINAL PAPER

Traditional vs. Alternative Teacher Preparation Programs: AMeta-Analysis

Denise K. Whitford 1● Dake Zhang2 ● Antonis Katsiyannis3

© Springer Science+Business Media, LLC 2017

Abstract A meta-analysis is presented of the academicachievement effects on students taught by teachers fromalternative teacher preparation (ATP) programs, comparedto students taught by teachers from traditional teacher pre-paration (TTP) programs. The literature has indicated mixedresults on the student-level academic outcomes of ATPprograms. Findings from this meta-analysis indicate anoverall statistically significant, yet small, difference in ATPand TTP programs (g= 0.03; 95% CI= 0.01, 0.04, p=0.001), with the mean achievement of students who hadATP teachers was about 0.03 standard deviations above thatof students who had TTP teachers. Further, there weredifferences in student achievement by type of ATP pro-gram, school level, and academic subject area. Theseresults, as well as implications for policy and practice arediscussed.

Keywords Meta-analysis ● Teacher preparation ●

Traditional ● Alternative ● Student academic achievement

Introduction

In an era of educational reform and increased emphasis onaccountability, teacher preparation standards and improvedstudent performance are of critical importance. Indeed, NoChild Left Behind (NCLB; U.S. Department of Education2002) mandated strict accountability measures to ensure thatall students become proficient in reading and mathematics,as well as ensure that teachers are highly qualified to teach intheir area of certification. Regarding student outcomes,scores from the National Assessment of Educational Pro-gress (NAEP) indicate that NCLB has had a positive impacton student math achievement in 4th and 8th grade, but, noimprovement has been demonstrated in 4th grade readingachievement (Dee and Jacob 2011). Specifically, accordingto the National Center for Education Statistics (The Nation’sReport Card 2015), among the 49.8 million students enrolledin public schools in 2013, the percentages of students at orabove proficient in mathematics were 42% for 4th grade and35% for 8th grade; reading scores were 35 and 36%,respectively.

One argument as to the cause of poor student proficiencylevels is the lack of enough highly qualified teachers, a keyrequirement of NCLB. Though the reauthorization of NCLB,through its successor, the Every Student Succeeds Act(ESSA), eliminates this requirement, the need for strongteachers in every classroom is still one of several crucialelements to the success of students (e.g., Nye et al. 2004;Stronge et al. 2011). For example, in a large randomizedstudy, Nye et al. (2004) found substantial differences amongteachers in the ability to produce student achievement gains(see also Aaronson et al. 2007). Further, recent studies ofteacher effects at the classroom level using the TennesseeValue-Added Assessment System (TVAAS) and a similardatabase in Dallas, Texas, have indicated that differential

* Denise K. Whitforddwhitford@purdue.edu

1 Department of Educational Studies, Purdue University, 100N.University Street, West Lafayette, IN 47907, USA

2 Department of Educational Psychology, Rutgers University, NewBrunswick, USA

3 Department of Education and Human Development, ClemsonUniversity, Clemson, USA

teacher effectiveness is a strong determinant of differences instudent learning; far outweighing the effects of differences inclass size and composition (Wright et al. 1997; see alsoStronge et al. 2011). Similarly, reports indicate that teacherpreparation is one of the strongest links between studentachievement in reading and mathematics, regardless of socio-economic and language status (Darling-Hammond 2000).

The White House press release on ESSA (2015)addressed the need for effective teachers working withminority and low-income students and has listed specificinitiatives taken by the U.S. Department of Education toaddress this issue. However, the issue appears to be morecentral to the quality of teacher training, particularly in lightof the influx of alternative teacher preparation (ATP) pro-grams, which promote the training of highly qualified tea-chers in a fraction of the time traditional teachers areprepared (U.S. Department of Education, Office of Post-secondary Education 2013). ESSA allows states to set upnew degree-granting academies for teachers outside of tra-ditional higher education systems and the creation of resi-dency programs, in which pre-service teachers work withveterans for a year of in-classroom training in addition totheir coursework (ESSA 2015).

Traditional teacher preparation (TTP) programs havetypically served undergraduate students who have no priorteaching or work experience, and generally lead to at least abachelor’s degree, although some programs may lead to ateaching credential, without a degree (Boyd et al. 2007;Shuls and Trivitt 2015; U.S. Department of Education,Office of Postsecondary Education 2013). In contrast,alternative teacher preparation (ATP) programs primarilyserve teachers already in classrooms as they participate inaccelerated programs as outlined by individual states (Boydet al. 2007; Shuls and Trivitt 2015; U.S. Department ofEducation, Office of Postsecondary Education 2013). Fur-ther distinguishing ATP from TTP programs, ATP pro-grams provide in-service teachers with full access toteaching students, with minimal supervision, while TTPprograms provide pre-service teachers with scheduledaccess to teaching students, with continuous supervision(Boyd et al. 2007; Shuls and Trivitt 2015).

ATP programs were created to diversify and increase theteaching force by allowing those without certification toswitch careers and expedite entry into K-12 classrooms (Boydet al. 2007; Shuls and Trivitt 2015). Currently, every state andthe District of Columbia have adopted alternative certificationprograms (NEA 2015). In recent years as many as a third ofnew hires have used an alternative route to certification andhave begun teaching before completing all certificationrequirements (Constantine et al. 2009). In 2014, there were26,595 teacher preparation programs, with the overwhelmingmajority housed in institutions of higher education (IHE);70% (n= 18,514) of TTP programs were in IHEs, 20%

(n= 5331) of ATP programs were in IHEs, and 10% (n=2750) of ATP programs were outside of IHEs. Further, of the500,129 individuals enrolled in teacher preparation programsin 2012–2013, 89% (n= 447,445) were enrolled in TTPprograms, 5% (n= 25,135) were enrolled in ATP programs atIHEs, and 6% (n= 27,549) were enrolled in ATP programsthat were not based at IHEs (Dalton and Henderson 2015).

The proliferation of alternative routes to certification canalso be attributed to shortages in critical subject areas andlow numbers of male and culturally and linguisticallydiverse teacher candidates, along with NCLB highly qua-lified teacher-related requirements (see Henry et al. 2014).Indeed, the inadequate supply of teachers, has been achronic concern in certain subject areas, such as STEMeducation (Hutchison 2012), special education (Zhang et al.2014), and in rural regions (Brownell et al. 2005). Under thecontext of teacher shortages, beginning in the 1980s, pol-icymakers in many states and school districts createdalternative certification as a means to meet the demand formore teachers (U.S. Department of Education, Office ofPostsecondary Education 2002). Today, ATP routes arecommonplace and becoming an important addition to thetraditional teacher preparation routes throughout the UnitedStates (Humphrey and Wechsler 2007). Additionally, giventhe importance of ATP programs as ascribed by ESSA, it isunderstandable that we should evaluate the impact ATPprograms have on student achievement in comparison toTTP programs, which are not a major focus of ESSA whenit comes to teacher preparation (ESSA 2015).

Unfortunately, investigating the differential effects onstudent achievement based on a teacher’s preparation uponentering the profession is both complicated and challenging topinpoint. First, there has not been an agreed upon definitionfor what constitutes alternative certification (Humphrey andWechsler 2007; Wilson et al. 2001, 2002), though theseprograms typically encompass attaining certification outsidethe traditional four or five year undergraduate program. Dif-ferent states have their own ranges of ATP programs, andsome national programs, such as Teach for America (TFA)and Troops to Teachers, that recruit candidates into teachingand provide support, but whose participants must also com-plete other certification programs, make the within-ATPprogram category even more heterogeneous. Second, teachersprepared via ATP programs seem to yield very differentteaching effectiveness at different grade levels and for dif-ferent subjects. For example, Henry et al. (2014) comparedthe academic achievement of students from traditionallyprepared teachers in North Carolina state public universities tothose from out-of-state and ATP program backgrounds. Theyfound that (a) out-of-state TTP teachers were less effective inelementary grades and high school, (b) ATP teachers wereless effective in high school, and (c) TFA corps memberswere more effective in teaching STEM subjects and

J Child Fam Stud

secondary grades. Third, disadvantaged students, such asthose in special education, students of color, students in ruralregions, and those in poorer school districts, are more likely tohave teachers prepared through ATP programs, because thesestudents are more vulnerable to teacher shortages (Mason-Williams 2015). The differences in student populations makeit even more difficult to make comparisons of teachingeffectiveness of teachers prepared via widely different routes.It has been under criticism that serving these disadvantagedstudents by teachers with emergent or intern certification has,or will, worsen student achievement (Mason-Williams 2015).

Alternative certification is generally viewed negativelyrelated to student achievement vs. full certification which isviewed positively related to student achievement (Bettset al. 2003; Goldhaber and Brewer 2000). Although ATPprograms have been criticized for lack of strong pedagogy-related coursework and limited field based experiences,TTP programs have also been under recent criticism forlacking content-specific expert guidance during candidatetraining (Feuer et al. 2013).

Regarding impact of TTP and ATP programs on aca-demic achievement, Wilson et al. (2001, 2002) conductedearly studies on teacher preparation for the U.S. Departmentof Education. Their findings indicated that: (a) a positivecorrelation existed between pre-service teacher subject-matter preparation and student performance, but it wasunclear how that content area knowledge was obtained (i.e.,specific coursework, an academic major, or other); and (b)pedagogical preparation had an impact on student perfor-mance, but it was unclear which aspects of pedagogicalpreparation had the greatest impact.

In light of the continued emphasis on student outcomesand the central role teachers play in the process, the issue ofteacher preparation and its impact on student achievementwarrants further examination. This is particularly pertinentas policymakers across states are releasing details addres-sing ESSA related requirements and also implementingevaluation systems that link teacher performance to studentperformance (Stronge and Associates 2013; U.S. Depart-ment of Education, Race to the Top 2009).

Therefore, the purpose of the present meta-analysis wasto expand on the small portion of the work in Wilson et al.(2001, 2002) and assess the quantifiable impact of ATPprograms, compared to TTP programs, on student academicachievement.

Method

Search Strategy

The authors followed guidelines from the PreferredReporting Items for Systematic reviews and Meta-Analyses

(PRISMA; Moher et al. 2009). The four primary electronicdatabases in education research (Academic Search Premier,Education Full Text, ERIC, and PsychInfo) were searchedfor studies published from 1 January 1990 through 31August 2016. The basal year of 1990 was selected for tworeasons. First, as discussed in Wilson et al. (2001, 2002),their findings included studies as far back as 1979, and asthey noted, alternative programs were in their infancy at thattime making comparisons with traditional programs not themost advantageous. Second, alternative preparation pro-grams began to increase rapidly with the introduction ofTeach for America in 1989. These two factors combined ledus to review published research the year after TFA began.Authors used the Boolean phrase “Teacher AND (prepara-tion OR certification OR credential OR authorization) AND(alternative OR traditional) AND student AND academicAND (achievement OR performance OR effect OR impactOR quality)”.

Selection Criteria

Quantitative studies that measured the academic achieve-ment of students taught by teachers from ATP and TTPprograms, were considered for this meta-analysis. Studieshad to be (a) research articles from English peer-reviewedjournals, or (b) reports reviewed by the What WorksClearinghouse. Additionally, studies must have includedeffect sizes demonstrating the comparison of academicachievement from students with teachers from ATP andTTP programs, with accompanying standard errors, orenough information to calculate effect sizes and theiraccompanying standard errors.

Studies that provided unstandardized regression coeffi-cients and effect sizes for independent groups, but notenough information to determine standard errors or a dis-aggregation of the number of TTP and ATP in the analyses(e.g., Darling-Hammond et al. 2005) were excluded fromanalysis, because when the sampling distribution and var-iance are unknown, weighting and appropriate confidenceintervals could not be computed. Also, when one or morearticles were located that had what strongly appeared to bereports of identical data (e.g., Gimbert et al. 2007; Gimbertet al. 2007), we opted to include only the article with themost descriptive information. Furthermore, this meta-analysis did not consider qualitative studies (with theexception of mixed methodology), literature reviews, the-oretical articles and position papers, conference papers andpresentations, or working papers.

Quality Indicators

Although multiple resources and recommendations areavailable to assess the quality of studies (e.g., Deeks et al.

J Child Fam Stud

2003), studies included in this meta-analysis were assessedfor quality using the essential quality indicators for groupexperimental and quasi-experimental research recom-mended by Gersten et al. (2005), based on author familiaritywith the instrument. Studies were assessed on their ability tomeet the 10 essential quality indicators and the 8 desirablequality indicator questions. The authors predetermined abasic minimum of quality aligned with those described byGersten et al. (2005); acceptable if the study meets all butone essential indicator and at least one desirable indicator.

Operational Definitions

For descriptive purposes and assessment of potential mod-erators, the studies were coded for the following char-acteristics: settings, demographics, preparation programattributes, and outcome variables related to the currentreview.

Settings

Settings were recorded as region, community, and schoollevel. Region was classified using the four regions of theUnited States as outlined by the U.S. Census Bureau(2016); West, Midwest, South, and Northeast, communitywas recorded as urban, suburban, and/or rural, while schoollevel was recorded as elementary school, middle school,and/or high school.

Demographics

Teacher demographics were recorded as gender, race/eth-nicity, and age. Gender was recorded as the percentage ofmales, race/ethnicity was recorded as the percentage ofWhite teachers, and age was recorded as the mean age ofteachers. Student demographics were not consistentlyreported and therefore were not coded.

Preparation program attributes

Attributes were recorded as ATP, TFA, and TTP. Forpurposes of succinct categorization, studies which reportedresults for alternative program, uncertified, under-certified,provisional license, temporary license, or emergency licensewere coded as ATP. TFA data was purposefully analyzedseparate from ATP data because research suggests that TFAtends to produce stronger academic results than those fromother ATP programs (Henry et al. 2014).

Outcome variables

Outcome variables were recorded as the size of academiceffect from baseline to the end of the academic year.

Academic effect was further classified into English/Lan-guage Arts/Reading, Mathematics, Science, Social Studies,and General Academics. General Academics was a categoryused for studies that did not disaggregate data by contentarea. Student achievement outcomes were measured usingstandardized subject-based assessments (e.g., CAT-5, ITBS,SAT-9, VASOL), end of course and end of year tests, andgenerally defined test scores. In some instances studies hadoutcome variables that were not relevant to the presentreview and therefore were not recorded.

Inter-rater agreement

In educational research the standard interrater agreementchecks are at 33% or more (see Losinski et al. 2014).Overlapping inter-rater reliability was conducted for 36% ofthe studies to ensure accuracy of extracted data, accuracy ofthe interpretation of intervention results, and accuracy ofopinions on quality indicators. Initial agreement was at98.74%. Authors met to resolve discrepancies and inter-rater reliability was met at 100% agreement.

Summary Measures

The meta-analysis included an assessment of teacher pre-paration program effectiveness based on a three-step pro-cess. In the first step, sample estimates of the standardizedmean differences (Cohen’s d effect size estimate) and theiraccompanying standard errors were retrieved from indivi-dual studies or calculated based on author-reported infor-mation. We utilized two different formulas for calculatingeffect sizes for studies that adopted matching procedures (n= 5) and those that did not (n= 7). For independent groups(n= 86 effect sizes), d was calculated by subtracting TTPacademic means from ATP (or TFA) means and thendividing by the within-groups standard deviation, pooledacross ATP (or TFA) and TTP (Borenstein et al. 2009).

d ¼ XATPðor TFAÞ � XTTP

SDwithin groups

Next, the variance was calculated by dividing the sum ofthe same sizes by the product of the sample sizes and thenadding result to the squared effect size divided by two timesthe sum of the sample sizes; the standard error was thencalculated by square rooting the variance.

SE ¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffinATPðor TFAÞ þ nTTPnATP ðor TFAÞnTTP

þ d2

2ðnATPðor TFAÞ þ nTTPÞ

s

For matched groups (n= 32 effect sizes), d was calcu-lated by subtracting TTP academic means from ATP (orTFA) means and then dividing by the standard deviation of

J Child Fam Stud

the difference.

d ¼ XATPðor TFAÞ � XTTP

SDof he difference

Next, the variance was calculated by multiplying the sumof one divided by the sample size and the effect size squareddivided by two times the sample size, and two times oneminus the correlation between ATP (or TFA) and TTP; thestandard error was then calculated by square rooting thevariance.

SE ¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi1nþ d2

2n

� �2 1� rð Þ

s

For the second step, we entered Cohen’s d effect sizeestimates into Comprehensive Meta-Analysis 3.3.070(CMA 2014). To reduce the chance of bias due to the dif-ference in sample sizes across studies we then convertedCohen’s d to Hedges’ g (Hedges and Olkin 1985). Effectsize estimates were entered for comparison between theacademic assessment results of students with ATP teachersvs. those with TTP teachers, and for those with TFA tea-chers vs. those with TTP teachers. A negative g indicatednegative mean outcomes of students with ATP (or TFA)teachers as compared to those with TTP teachers. A positiveg indicated positive mean outcomes of students with ATP(or TFA) teachers, as compared to those with TTP teachers.

For the third step, we separated the effect sizes by clas-sification (ATP or TFA) and then by school level (i.e.,elementary school, middle school, or high school). Further,we separated the effect sizes by subject area. We obtainedpooled preparation program effect estimates (i.e., weightedaverages of the program effect sizes). Disparities in samplesize were compensated by weighting the effect sizes basedon sample sizes (Borenstein et al. 2009).

CMA (2014) calculated the inverse variance weightedmean effect size for both fixed and random effects foracademic achievement, with accompanying standard errors,in order to develop confidence intervals and p-values and todetermine precision of the estimates and strength of thepreparation program for each outcome classification. Thereare pros and cons to both the fixed effects and randomeffects error models. The fixed effects model tends to pro-duce smaller confidence intervals and has a greater potentialfor allowing us to make inferences from the results, butthose results should not be generalized to the larger popu-lation (Borenstein et al. 2009; Wilson 2011). The randomeffects model, on the other hand, tends to produce largerconfidence intervals and has less of a potential for allowingus to make inferences from the results, but those results maybe generalized to the larger population (Borenstein et al.2009; Wilson 2011). We opted to report results for randomeffects as they tend to be the standard in meta-analytic

reporting and there may have been an impact on our resultsif we had assumed the studies were functionally identical(Borenstein et al. 2009; Wilson 2011). Although it has lowpower in meta-analyses of this size, and caution should betaken with its use, we relied on the Q test statistic in order todetermine the presence or absence of heterogeneity and thel2 index to identify the amount of any such heterogeneity ona relative scale (Borenstein et al. 2009).

Publication bias

To address the potential for publication bias we conducted avisual examination of funnel plots produced in CMA (2014)to evaluate the symmetry of our dataset. The test of sym-metry is a subjective task and not reliable for a smallsample, such as the one in this meta-analysis, therefore, wealso provided a Fail-Safe N for each overall analysis byschool level. The Fail-Safe N is a probability estimation ofthe number of non-published studies averaging non-significant results, which must be in existence before thepooled effect of the programs would no longer be statisti-cally significant (Borenstein et al. 2009; Cooper 1979;Rosenthal 1979).

Results

Study Selection

As outlined in the PRISMA Flow Diagram (Fig. 1), thesearch yielded an initial 2338 records and an ancestralsearch yielded an additional 14 records. Upon removal ofduplicates, 719 records remained. The studies were nar-rowed down by the relevance of their titles and abstracts. Insome cases the relevance of the study could not be deter-mined solely on the title or abstract and in those instancesthe content of the article was examined to determineinclusion, leading to the removal of 688 records. Theauthors assessed the remaining 31 full-text records,screening for the inclusion and exclusion criteria previouslydetailed, ultimately excluding 13 for failure to meet our apriori inclusion and exclusion criteria. The remaining 18records were subsequently assessed for quality. The authorsexcluded 6 articles that did not meet a predetermined basicminimum of quality, including those that lacked key resultsor statistical information (e.g., Darling-Hammond 2000;Darling-Hammond et al. 2005), lacked clarity on the type ofresults reported (e.g., Gansle et al. 2012), and those thatappeared to use duplicate data of an included study (e.g.,Gimbert et al. 2007), leaving a total of 12 articles includedin this meta-analysis, including one study that was alsoincluded in Wilson et al. (2001) under the topic arearegarding the effects of pedagogical preparation.

J Child Fam Stud

Study Characteristics

The characteristics of each included study are presented inTable 1. The studies ranged from 1998 to 2015 withreported sample sizes ranging from 12 to 29,934 (M=3313; SD= 8890) teachers. The subject area of analysis fellinto five categories across the studies. Authors reportedresults for mathematics (n= 11); English Language Arts(ELA) and/or reading (n= 8), science (n= 5), social studies(n= 2), and a general category of academics (n= 1). Datacame from elementary schools (n= 7 articles), middleschools (n= 4 articles), and high schools (n= 7 articles).

Meta-Analysis

Data extraction yielded a total of 118 independent effectsizes, which are reported in this meta-analysis. Separatetables for elementary school, middle school, and highschool, as well as all combined school levels have beenprovided for the results below. Each table includes thenumber of effect sizes (k), the inverse variance weightedmean effect size (g), the standard error (SE), the p-value (p),

and the 95% confidence interval (CI). Also included are theresults for heterogeneity; the Q-value (Q), the p-value (p),and the percentage of variation in outcomes acrossstudies (l2).

Elementary school

As reported in Table 2, we found a small, but statisticallysignificant effect (g= 0.03, SE= 0.01, p= 0.018, CI=0.01, 0.05), that indicates that the mean achievement ofstudents who had ATP teachers (whether TFA or anotherprogram) was about 0.03 standard deviations above that ofstudents who had TTP teachers. The sample indicatedmoderate heterogeneity, Q(31)= 51.77 (l2= 40.12, p=0.011); about 40% of the total variability observed could beattributed to factors other than random chance. Addition-ally, we found a small, but statistically significant effect(g= 0.04, SE= 0.02, p= 0.006, CI= 0.00, 0.08), thatindicates that the mean mathematics achievement of stu-dents who had ATP teachers (whether TFA or anotherprogram) was about 0.04 standard deviations above that ofstudents who had TTP teachers. The sample indicated

Records iden�fied through database searching

(n = 2,338)

Scre

enin

gIn

clud

ed

Elig

ibili

ty

Iden

�fica

�on

Addi�onal records iden�fied through other sources

(n = 14)

Records a�er duplicates removed(n = 719)

Records screened(n = 719)

Records excluded(n = 688)

Full-text ar�cles assessed for eligibility

(n = 31)

Full-text ar�cles excluded, with reasons

(n = 13)

Studies included in quality coding (n = 18)

Studies included in quan�ta�ve synthesis

(meta-analysis) (n = 12)

Fig. 1 Flow diagram. Source:Moher et al. (2009)

J Child Fam Stud

Tab

le1

Study

characteristics

Study

Program

(s)of

stud

yCon

tent

area

Measurementtool(s)

Sum

maryof

results

Antecol

etal.(201

3)TFA

vs.TTP

Reading

NormalCurve

Equ

ivalent(NCE)testscores

“While

wegenerally

find

that

TFA

teachers

neith

erhelp

norhu

rtstud

entsin

term

sof

readingtestscores,wefind

positiv

eandstatistically

sign

ificant

effectsof

TFA

across

themathachievem

entdistribu

tionforthefullsampleandtheeffectsarefairly

uniform”(abstract)

Mathematics

Clark

etal.(201

5)TFA

vs.TTP

Reading

End

-of-year

(EOY)testscores

“Wefoun

dthat

TFA

teachers

inlower

elem

entary

grades

(prekind

ergarten

throug

hgrade2)

hadapo

sitiv

e,statistically

sign

ificant

effecton

stud

ents’reading

achievem

ent

…Wedidno

tfind

statistically

sign

ificant

impactsforothersubg

roup

sof

TFA

teachers

that

weexam

ined”(p

xi)

Mathematics

Clotfelteret

al.(201

0)ATPvs.TTP

Eng

lish

Sub

ject-specificend-of-cou

rse(EOC)

exam

s“…

stud

entstaug

htby

teachers

with

[analternative]

license

average0.06

standard

deviations

lower

than

thosetaug

htby

teachers

with

aregu

larlicense”(p

670)

Mathematics

Science

Gim

bertet

al.(200

7)ATPvs.TTP

Mathematics

DistrictQuarterly

Assessm

ents(D

Q)

“The

results

ofthisstud

yafford

evidence

that

thestud

entstaug

htby

1st-year,

alternativelyprepared

teachersachieved

aswellas

orbetterthan

theirpeerstaug

htby

tradition

ally

certified

1st-year

teachers,accordingto

stud

entachievem

entin

mathematics,specifically

Algebra

I”(abstract)

End

ofYearStand

ards

ofLearning

Assessm

ent(V

ASOL)

Glazerm

anet

al.(200

6)TFA

vs.TTP

Reading

IowaTestof

Basic

Skills

(ITBS)

“Wefoun

dthat

TFA

teachers

hadapo

sitiv

eim

pact

onmathachievem

entandno

impact

onreadingachievem

ent”(abstract)

Mathematics

NormalCurve

Equ

ivalent(NCE)testscores

Goldh

aber

andBrewer

(200

0)ATPvs.TTP

Mathematics

Sub

ject-based

Tests

“Inmathematics,wefind

teacherswho

have

astandard

certificatio

nhave

astatistically

sign

ificant

positiv

eim

pact

onstud

enttestscores

relativ

eto

teachers

who

either

hold

privatescho

olcertificatio

nor

areno

tcertified

intheirsubjectarea.Con

traryto

conv

entio

nalwisdo

m,mathematicsandsciencestud

entswho

have

teachers

with

emergencycredentialsdo

noworse

than

stud

entswho

seteachers

have

standard

teaching

credentials”(abstract)

Science

Henry

etal.(201

4)ATPvs.TTP

Eng

lish

End

-of-Grade

(EOG)tests

“Com

paredwith

undergradu

ate-prepared

teachers

from

in-state

public

universities…

alternativeentryteachers

areless

effectivein

high

scho

ol,and…

Teach

For

America

corpsmem

bersaremoreeffectivein

STEM

subjectsandsecond

arygrades”(abstract)

Reading

Mathematics

TFA

vs.TTP

Science

End

-of-Cou

rse(EOC)tests

SocialStudies

Miller

etal.(199

8)ATPvs.TTP

Reading

IowaTestof

Basic

Skills

(ITBS)

“There

was

nodifference

inaveragestud

entachievementtestscore

levels,b

ased

upon

whether

stud

entshadbeen

taug

htby

[alternative]

or[traditio

nal]teachers”(pp

171–

172)

Mathematics

NormalCurve

Equ

ivalent(NCE)testscores

Nun

nery

etal.(200

9)ATPvs.TTP

Reading

Florida

Com

prehensive

Assessm

entTest

(FCAT)

“Resultsindicatedthat

stud

entsserved

byTroop

steachers

performed

abou

tequally

wellin

readingandachieved

asm

allbu

tstatistically

sign

ificant

advantagein

mathematicswhencomparedwith

allteachers

…”(abstract)

Mathematics

J Child Fam Stud

moderate heterogeneity, Q(12)= 33.79 (l2= 64.48, p=0.001); about 64% of the total variability observed could beattributed to factors other than random chance. However,based on the statistically significant findings regarding TFAin Table 2, it may be inferred that TFA had a greater impacton mathematics achievement than ATP in general.

Middle school

As reported in Table 3, we found a small, but statisticallysignificant effect (g= 0.09, SE= 0.02, p< 0.001, CI=0.04, 0.13), that indicates that the mean achievement ofstudents who had ATP teachers (whether TFA or anotherprogram) was about 0.09 standard deviations above that ofstudents who had TTP teachers. The sample indicated highheterogeneity, Q(45)= 251.02 (l2= 82.07, p < 0.001);about 82% of the total variability observed could beattributed to factors other than random chance. Addition-ally, we found a small, but statistically significant effect (g= 0.11, SE= 0.04, p= 0.015, CI= 0.02, 0.19), that indi-cates that the mean English achievement of students whohad ATP teachers (whether TFA or another program) wasabout 0.11 standard deviations above that of students whohad TTP teachers. The sample indicated high heterogeneity,Q(12)= 103.91 (l2= 88.45, p < 0.001); about 88% of thetotal variability observed could be attributed to factors otherthan random chance. We also found a small, but statisticallysignificant effect (g= 0.12, SE= 0.05, p= 0.022, CI=0.02, 0.22), that indicates that the mean mathematicsachievement of students who had ATP teachers (whetherTFA or another program) was about 0.12 standard devia-tions above that of students who had TTP teachers. Thesample indicated high heterogeneity, Q(28)= 110.41 (l2=74.64, p < 0.001); about 75% of the total variabilityobserved could be attributed to factors other than randomchance.

High school

Interestingly, student achievement from ATP and TFAteachers appear to cancel each other out. We did not find astatistically significant result for total academic achievementat the high school level. However, statistically significantresults were observed when the programs were dis-aggregated by ATP and TFA. As reported in Table 4, wefound a small, but statistically significant effect (g=−0.07,SE= 0.02, p= 0.001, CI=−0.12, −0.03), that indicatesthat the mean achievement of students who had ATP tea-chers (excluding TFA) was about 0.07 standard deviationsbelow that of students who had TTP teachers. The sampleindicated high heterogeneity, Q(19)= 180.38 (l2= 89.47,p < 0.001); about 89% of the total variability observedcould be attributed to factors other than random chance.T

able

1continued

Study

Program

(s)of

stud

yCon

tent

area

Measurementtool(s)

Sum

maryof

results

Piroet

al.(201

0)ATPvs.TTP

Reading

Tennessee

Com

prehensive

Assessm

ent

Program

(TCAP)

“Resultsshow

edthatalternativelycertified

teachersperformed

aswellas

tradition

ally

certified

teachers

atallgradelevelsandcontent,except

insecond

arymath,

where

tradition

ally

trainedteachersevidencedhigh

erstud

entg

rowth

andin

second

arysocial

stud

ies,where

alternativelytrainedteachers

evidencedhigh

erstud

entgrow

th”

(abstract)

ELA

Mathematics

Science

NormalCurve

Equ

ivalent(NCE)testscores

SocialStudies

Sharkey

andGoldh

aber

(200

8)ATPvs.TTP

Mathematics

Testscores

“Wefind

that…

privatescho

olstud

entsof

fully

certified

12th

grademathandscience

teachers

dono

tappear

toou

tperform

stud

entsof

privatescho

olteachers

who

areno

tfully

certified”(abstract)

Science

Xuet

al.(201

1)TFA

vs.TTP

Eng

lish

Sub

ject-specificend-of-cou

rse(EOC)

exam

s“W

efind

that

TFA

teachers

tend

tohave

apo

sitiv

eeffect

onhigh

scho

olstud

enttest

scores

relativ

eto

non-TFA

teachers,includ

ingthosewho

arecertified

infield”

(abstract)

Mathematics

Science

J Child Fam Stud

Furthermore, we found a small, but statistically significanteffect (g= 0.04, SE= 0.02, p= 0.013, CI= 0.01, 0.08),that indicates that the mean achievement of students whohad TFA teachers was about 0.02 standard deviations abovethat of students who had TTP teachers. The sample indi-cated high heterogeneity, Q(19)= 933.40 (l2= 97.96, p <0.001); about 98% of the total variability observed could beattributed to factors other than random chance.

Overall effect

We combined all data reported in Tables 2–4 in order toreport an overall effect across K-12 schools combined. Asreported in Table 5, we found a small, but statisticallysignificant effect (g= 0.03, SE= 0.01, p= 0.001, CI=0.01, 0.04), that indicates that the mean achievement ofstudents who had ATP teachers (whether TFA or anotherprogram) was about 0.03 standard deviations above that ofstudents who had TTP teachers. The sample indicated highheterogeneity, Q(117)= 1631.18 (l2= 92.83, p < 0.001);about 93% of the total variability observed could beattributed to factors other than random chance. Addition-ally, we found a small, but statistically significant effect (g= 0.04, SE= 0.02, p= 0.021, CI= 0.01, 0.08), that

indicates that the mean mathematics achievement of stu-dents who had ATP teachers (whether TFA or anotherprogram) was about 0.04 standard deviations above that ofstudents who had TTP teachers. The sample indicated highheterogeneity, Q(58)= 749.53 (l2= 92.62, p < 0.001);about 93% of the total variability observed could beattributed to factors other than random chance.

Sensitivity analyses

Separation of TFA from ATP, and subject areas from eachother, as well as school levels from one another served asindividual sensitivity analyses (see Tables 2–5). Addition-ally, by utilizing the one study removed function withinCMA (2014), we conducted sensitivity analyses to identifythe undue influence of any one study at each school leveland K-12 combined (see Table 6). We found our analyses tobe robust, as there were no changes in the overall outcomeestimates.

Publication bias

Visual analysis of our funnel plot, Fig. 2, indicates a fairlysymmetrical distribution around the weighted mean effects.

Table 2 Impact of teacherpreparation program onacademic achievement atelementary schools

Effect size Heterogeneity

k g SE p CI Q p l2

ATP vs. TTP

English/ELA/Reading 9 0.00 0.07 0.991 −0.14, 0.14 10.91 0.207 26.64

Mathematics 7 0.04 0.04 0.375 −0.05, 0.12 8.72 0.190 31.15

Science 2 −0.02 0.02 0.319 −0.06, 0.02 0.01 0.913 0.00

Social Studiesa 1 0.11 0.37 0.766 −0.62, 0.84 0.00 1.000 0.00

General academics 0 – – – – – – – –

Total achievement 19 0.00 0.01 0.705 −0.02, 0.02 20.61 0.300 12.65

TFA vs. TTP

English/ELA/Reading 6 0.02 0.02 0.139 −0.01, 0.06 3.59 0.609 0.00

Mathematics 6 0.08 0.02 <0.001 0.04, 0.12 5.69 0.337 12.14

Sciencea 1 0.01 0.05 0.003 −0.09, 0.11 0.00 1.000 0.00

Social Studies 0 – – – – – – – –

General academics 0 – – – – – – – –

Total achievement 13 0.05 0.01 0.001 0.02, 0.08 14.83 0.251 19.07

ATP and TFA vs. TTP

English/ELA/Reading 15 0.01 0.01 0.391 −0.02, 0.04 16.04 0.311 12.70

Mathematics 13 0.04 0.02 0.006 0.00, 0.08 33.79 0.001 64.48

Science 3 −0.02 0.02 0.395 −0.05, 0.02 0.32 0.852 0.00

Social Studiesa 1 0.11 0.37 0.766 −0.62, 0.84 0.00 1.000 0.00

General academics 0 – – – – – – – –

Total achievement 32 0.03 0.01 0.018 0.01, 0.05 51.77 0.011 40.12

a Denotes studies used for the analysis of Total Achievement only

J Child Fam Stud

Additionally, our Fail Safe N indicated that 1550 statisti-cally significant studies matching the criteria of the studiesidentified for this meta-analysis would need to be in exis-tence to nullify the overall effects we have reported. This islarger than the suggested threshold (5k+ 10) (Rosenthal1979), therefore we are confident in the reportedfindings.

Discussion

The present study examined empirical research on theimpact the type of teacher preparation program (alternativevs. traditional) has on the academic achievement of stu-dents. Though there was a plethora of articles in profes-sional journals regarding traditional and alternativecertification related information, the paucity of empiricalstudies was a surprise for an issue that has been of criticalimportance to professionals and the public alike. Specifi-cally, as indicated earlier, No Child Left Behind placed anemphasis on a highly qualified teaching force as a means toimprove educational outcomes, and attempts were made tohold teachers accountable for students’ academic achieve-ment (e.g., Race to the Top). Though one may argue thatfindings based on this limited number of studies are

inconclusive, along with recognizing the complexities oflinking teacher preparation to student achievement (seeStronge et al. 2011), our results indicate that, in general,students who have teachers from ATP programs, particu-larly, Teach for America (TFA), tend to do slightly betterthan those from TTP programs. However, these findingsdiffer depending on school level (elementary school, middleschool, or high school), as well as academic subject area(English, mathematics, science, or social studies). Specifi-cally, students with ATP teachers tend to do moderatelybetter in English at the middle school level, but show nostatistically significant differences in elementary school andhigh school. Furthermore, students with TFA mathematicsand science teachers tend to do better, albeit just slightly,than students with TTP math and science teachers at allschool levels.

Related research on ATP and TTP programs has beenmixed, with researchers, like Gimbert et al. (2007) indi-cating favorable student outcomes for ATP teachers com-pared to TTP teachers, and researchers, like Clotfelter et al.(2010) indicating favorable student outcomes for TTP tea-chers compared to ATP teachers. While others, for exampleXu et al. (2011), report more favorable outcomes for ATP-specific teachers, like those from TFA as providing thegreatest student gains. Moreover, there are yet others who

Table 3 Impact of teacherpreparation program onacademic achievement at middleschools

Effect size Heterogeneity

k g SE p CI Q p l2

ATP vs. TTP

English/ELA/Reading 12 0.53 0.21 0.011 0.12, 0.93 99.57 <0.001 88.95

Mathematics 28 0.14 0.07 0.051 0.00, 0.29 78.82 <0.001 65.75

Science 2 −0.03 0.02 0.479 −0.07, 0.01 0.01 0.921 0.00

Social Studies 0 – – – – – – – –

General academics 0 – – – – – – – –

Total achievement 43 0.10 0.03 0.002 0.04, 0.16 187.53 <0.001 77.60

TFA vs. TTP

English/ELA/Readinga 1 0.02 0.01 0.000 0.00, 0.04 0.00 1.000 0.00

Mathematicsa 1 0.12 0.02 0.000 0.08, 0.16 0.00 1.000 0.00

Sciencea 1 0.19 0.04 0.000 0.11, 0.27 0.00 1.000 0.00

Social Studies 0 – – – – – – – –

General academics 0 – – – – – – – –

Total achievement 3 0.10 0.05 0.030 0.01, 0.20 33.39 <0.001 94.01

ATP and TFA vs. TTP

English/ELA/Reading 13 0.11 0.04 0.015 0.02, 0.19 103.91 <0.001 88.45

Mathematics 29 0.12 0.05 0.022 0.02, 0.22 110.41 <0.001 74.64

Science 3 0.08 0.11 0.302 −0.13, 0.28 24.20 <0.001 91.74

Social Studies 0 – – – – – – – –

General academics 0 – – – – – – – –

Total achievement 46 0.09 0.02 <0.001 0.04, 0.13 251.02 <0.001 82.07

a Denotes studies used for analysis of Total Achievement only

J Child Fam Stud

report no substantial general differences in programs (Milleret al. 1998), and others reporting no substantial differences,depending on subject area or grade level (Clark et al. 2015).Our results, however, indicated that the differences in stu-dent academic achievement between the three programs(TTP, ATP, and ATP-TFA) are negligibly small, with theexception of ATP teachers in middle school English. Thisexception to the results may be due to confounding factorsthat were not reported or available in the three studies thatcomprised the data in this category (Henry et al. 2014;Nunnery et al. 2009; Piro et al. 2010).

Variability within Alternative Preparation Programs

Variables beyond ATP vs. TTP preparation program mayhave a greater impact than the programs themselves (Henryet al. 2014; Wilson et al. 2001, 2002). Specifically, wefound higher academic performance when we comparedTFA to TTP, than when we compared ATP to TTP. TFArecruits recent college graduates and professionals, oftenfrom top universities with students majoring in math andscience, to teach for a minimum of two years in urban andrural communities. TFA has been an important source ofteachers in high poverty U.S. schools, especially for subject

areas experiencing chronic shortages (TFA 2016). TFA alsodistinguishes itself from typical ATP programs and TTPprograms with much more selective standards and screeningprocedures for teacher candidates, and on-going mentoringand support throughout their first two years of in-serviceteaching (TFA 2016). However, TFA programs have beencriticized for the relatively short commitment required ofTFA trained teachers, which does not allow for the gaugingof teaching effectiveness over time (Heilig and Jez 2010)and the lack of extensive college-based pedagogy-related(Darling-Hammond et al. 2005).

Implications for Policy and Practice

Given that teachers from each preparation program gen-erally do not present discernable differences in effective-ness, yet student achievement in the U.S. is still belowaverage, in comparison to the rest of the world (OECD2015), alternative and traditional programs must focus onskill development that result in sustained and improvedstudent academic performance. Subject matter preparationis also critical as teachers play a more significant role instudent learning of mathematics than in reading (Henryet al. 2014). Though more teachers are proficient in ELA/

Table 4 Impact of teacherpreparation program onacademic achievement at highschools

Effect size Heterogeneity

k g SE p CI Q p l2

ATP vs. TTP

English/ELA/Readinga 1 0.00 0.01 1.000 −0.02, 0.02 0.00 1.000 0.00

Mathematics 10 −0.18 0.14 0.201 −0.46, 0.10 141.84 <0.001 93.66

Science 6 −0.14 0.07 0.046 −0.28, −0.00 8.05 0.153 37.92

Social Studiesa 1 −0.03 0.01 0.003 −0.05, −0.01 0.00 1.000 0.00

General academics 2 −0.07 0.01 <0.001 −0.08, −0.05 0.50 0.480 0.00

Total achievement 20 −0.07 0.02 0.001 −0.12, −0.03 180.38 <0.001 89.47

TFA vs. TTP

English/ELA/Reading 3 −0.03 0.03 0.315 −0.09, 0.03 33.89 <0.001 94.10

Mathematics 7 0.06 0.03 0.079 −0.01, 0.12 397.53 <0.001 98.49

Science 9 0.05 0.02 0.026 0.01, 0.10 327.22 <0.001 97.56

Social Studiesa 1 0.09 0.04 0.024 0.01, 0.17 0.00 1.000 0.00

General academics 0 – – – – – – – –

Total achievement 20 0.04 0.02 0.013 0.01, 0.08 933.40 <0.001 97.96

ATP and TFA vs. TTP

English/ELA/Reading 4 −0.02 0.02 0.359 −0.07, 0.03 52.77 <0.001 94.32

Mathematics 17 −0.01 0.03 0.832 −0.07, 0.05 599.86 <0.001 97.33

Science 15 0.03 0.02 0.217 −0.02, 0.07 354.62 <0.001 96.05

Social Studies 2 0.03 0.06 0.691 −0.09, 0.14 8.47 0.004 88.19

General academics 2 −0.07 0.01 <0.001 −0.08, −0.05 0.50 0.480 0.00

Total achievement 40 0.00 0.02 0.762 −0.03, 0.03 1299.75 <0.001 97.00

a Denotes studies used for analysis of Total Achievement only

J Child Fam Stud

reading content knowledge, fewer are proficient in mathe-matics and science content areas (Ball et al. 2005; Hill et al.2005; Wilson et al. 2001, 2002), let alone possessing theability to teach math and science. Indeed, research hasshown that effective mathematics and science teachers notonly have to be proficient content knowledge, but also inpedagogy (Van de Walle et al. 2012). Further, mathematics,science, and special education are typically the most hard-to-staff subject areas, especially in urban/high need schools(Gimbert et al. 2007). How we ensure that both TTP andATP teachers are well prepared for pedagogy and contentremains a challenging task (Wilson et al. 2001, 2002), butthe first step should include a strategic focus on content areaknowledge at all primary and secondary grades.

Regarding the achievement of students with TFA tea-chers suggest the possibility of building an effective systemof recruiting non-education majors to teach students in highneeds schools and specific subject areas. We foresee thatthis would require all future teachers to earn, at a minimum,a bachelor’s degree from an accredited IHE in their desiredcontent area (e.g., English, mathematics, science, socialstudies). After the bachelor’s is obtained, teacher candidateswould then attend a rigorous and selective, extended orexpedited, certification program that focuses on highly-effective pedagogy and interventions in elementary, sec-ondary, special education, and/or gifted education settings.Again, this will likely require incentives by means of fed-eral funding.

Table 5 Impact of teacherpreparation program onacademic achievement at K-12schools combined

Effect size Heterogeneity

k g SE p CI Q p l2

ATP vs. TTP

English/ELA/Reading 22 0.04 0.03 0.163 −0.01, 0.09 110.50 <0.001 81.00

Mathematics 45 0.01 0.03 0.834 −0.05, 0.07 235.43 <0.001 81.31

Science 10 −0.03 0.01 0.005 −0.06, −0.01 9.33 0.408 3.49

Social Studies 3 0.31 0.33 0.340 −0.33, 0.96 7.30 0.026 72.60

General academics 2 −0.07 0.01 <0.001 −0.08, −0.05 0.50 0.480 0.00

Total achievement 82 −0.01 0.01 0.282 −0.04, 0.01 463.69 <0.001 82.53

TFA vs. TTP

English/ELA/Reading 10 0.00 0.02 0.996 −0.04, 0.04 83.66 <0.001 89.24

Mathematics 14 0.08 0.02 0.002 0.03, 0.12 418.21 <0.001 96.89

Science 11 0.06 0.02 0.006 0.02, 0.10 342.15 <0.001 97.08

Social Studiesa 1 0.09 0.04 0.024 0.01, 0.17 0.00 1.000 0.00

General academics 0 – – – – – – – –

Total achievement 36 0.05 0.01 <0.001 0.03, 0.08 988.17 <0.001 96.46

ATP and TFA vs. TTP

English/ELA/Reading 32 0.01 0.01 0.537 −0.02, 0.03 213.25 <0.001 85.46

Mathematics 59 0.04 0.02 0.021 0.01, 0.08 749.53 <0.001 92.62

Science 21 0.03 0.02 0.135 −0.01, 0.06 386.98 <0.001 94.83

Social Studies 4 0.06 0.08 0.447 −0.09, 0.20 15.68 <0.001 80.87

General academics 2 −0.07 0.01 <0.001 −0.08, −0.05 0.50 0.480 0.00

Total achievement 118 0.03 0.01 0.001 0.01, 0.04 1631.18 <0.001 92.83

a Denotes studies used for analysis of Total Achievement only

Table 6 One study removedsensitivity analyses

Effect size Heterogeneity

k g SE p CI Q p l2

Elementary School 32 0.03 0.01 0.018 0.01, 0.05 51.77 0.011 40.12

Middle School 46 0.09 0.02 <0.001 0.04, 0.13 251.02 <0.001 82.07

High School 40 0.00 0.02 0.762 −0.03, 0.03 1299.75 <0.001 97.00

K-12 Combined 118 0.03 0.01 0.001 0.01, 0.04 1631.18 <0.001 92.83

J Child Fam Stud

Limitations and Future Research

Due to several limitations, these results should be inter-preted with caution. First, our use of random effects modelsassumes that the true effect obtained from the analyses mayin fact fluctuate from one study to the next. Our results, interms of those with statistically significant Q-values and l2,revealed that the majority had high heterogeneity, as definedby Higgins et al. (2003). High heterogeneity allows us tospeculate reasons for the variance and then employ sub-group analyses or meta-regressions to explain our findings(Borenstein et al. 2009). We ran subgroup analyses, asreported in Tables 2–5, however due to inconsistentreporting of characteristics across studies, meta-regressionswere not a practical option for the current study. Con-founding variables were not reported or available in some ofthe studies that comprised the dataset and there wasinconsistent reporting in others. The absence, variability,and lack of detail regarding statistical matching proceduresutilized across studies limited the comparison of distinctschool, teacher, and student characteristics and any con-founding variables that we could have accounted for over-all. We encourage future researchers to include an extensivereport on informational characteristics and confoundingvariables regarding schools, teachers, and students.

Second, as previously discussed, student achievementoutcomes were measured using standardized subject-basedassessments, end of course and end of year tests, andmeasurements generally defined as test scores. Due to lackof information in half of the studies regarding whether ornot end of course, end of year, and tests in general werestandardized we could not conclude the impact of standar-dized achievement outcomes vs. non-standardized out-comes in our results. Again, we encourage futureresearchers to include detailed information regarding studycharacteristics, particularly instruments used to measurestudent achievement.

Third, although this is a very important and controversialtopic, to date there has been very limited literature

empirically examining the effectiveness of TTP programs,particularly in comparison to ATP programs, in addition toa scarcity of studies that report key statistical informationfor comparison. A good faith effort was made to identify asmany peer reviewed empirically-supported studies as pos-sible, however, there has been even less research thataddresses this topic by comparing teacher training programeffectiveness with student achievement as an outcomevariable. We unfortunately only identified 12 studies thatmet our inclusion criteria, and this small number of existingstudies limit the power of the current research. It is plausiblethat different findings will be discovered once moreresearch is conducted in this area. Certainly, the need forresearchers to conduct more studies in this field that areempirical, high quality, and transparent in regards to thereporting of their data is critical. Indeed, given NCLBmandates in the past and federal state grants, states havedeveloped accountability systems that allow linking tea-chers (and their preparation) to student performance.Examining these data on a large, national scale has thepotential to provide insights on the effects of teacher pre-paration on student achievement.

Compliance with Ethical Standards

Conflict of Interest The authors declare that they have no compet-ing interests.

References

* Identifies studies included in the meta-analysis.Aaronson, D., Barrow, L., & Sander, W. (2007). Teachers and student

achievement in the Chicago Public High Schools. Journal ofLabor Economics, 25(1), 95–135.

*Antecol, H., Eren, O., & Ozbeklik, S. (2013). The effect of Teach forAmerica on the distribution of student achievement in primaryschool: evidence from a randomized experiment. Economics ofEducation Review, 37, 113–125. https://doi.org/10.1016/j.econedurev.2013.08.004.

Ball, D.L., Hill, H.C., & Bass, H. (2005). Knowing mathematics forteaching: who knows mathematics well enough to teach thirdgrade, and how can we decide? American Educator, 29(1),14–17, 20–22, 43–46. https://deepblue.lib.umich.edu/bitstream/handle/2027.42/65072/Ball_F05.pdf?sequence=4&isAllowed=y.

Betts, J. R., Zau, A. C., & Rice, L. A. (2003). Determinants of studentachievement: new evidence from San Diego. San Francisco, CA:Public Policy Institute of California.

Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R.(2009). Introduction to meta-analysis. West Sussex, UK: JohnWiley & Sons, Ltd.

Boyd, D., Goldhaber, D., Lankford, H., & Wyckoff, J. (2007). Theeffect of certification and preparation on teacher quality. TheFuture of Children, 17, 45–68.

Brownell, M. T., Bishop, A. M., & Sindelar, P. T. (2005). NCLB andthe demand for highly qualified teachers: challenges and solutions

Fig. 2 Funnel plot of the 118 effect sizes for teacher preparationprogram student academic achievement outcomes

J Child Fam Stud

for rural schools. Rural Special Education Quarterly, 24(1),9–15. https://acres-sped.org/journal.

*Clark, M.A., Isenberg, E., Liu, A.Y., Makowsky, L., Zukiewicz, M.(2015). Impacts of the Teach for America investing in innovationsscale-up. (Ref #: 06889.740). Princeton, NJ: Mathematica PolicyResearch.https://www.mathematica-mpr.com/~/media/publications/pdfs/education/tfa_investing_innovation.pdf.

*Clotfelter, C. T., Ladd, H. F., & Vigdor, J. L. (2010). Teacher cre-dentials and student achievement in high school: a cross-subjectanalysis with student fixed effects. Journal of Human Resources,45, 655–681. https://doi.org/10.1353/jhr.2010.0023.

Comprehensive Meta-Analysis (Version 3) [CMA; Computer Soft-ware] (2014) Englewood, NJ: Biostat.

Constantine, J., Player, D., Silva, T., Hallgren, K., Grider, M., & Deke,J. (2009). An evaluation of teachers trained through differentroutes to certification (NCEE 20094043). U.S. Department ofEducation, Institute of Education Sciences, National Center forEducation Evaluation and Regional Assistance. https://ies.ed.gov/ncee/pubs/20094043/pdf/20094044.pdf

Cooper, H. M. (1979). Statistically combining independent studies: ameta-analysis of sex differences in conformity research. Journalof Personality and Social Psychology, 37, 131–146. https://doi.org/10.1037/0022-3514.37.1.131.

Dalton, S.S., & Henderson, A. (2015). HEA title II accountability:impact of alternative route teacher preparation. Paper presented atthe 25th Annual National Association for Alternative Certifica-tion Route Teacher Preparation, Chicago, IL. https://alt-teachercert.org/HEA%20Title%20II%20.pdf

Darling-Hammond, L. (2000). Teacher quality and student achieve-ment: a review of state policy evidence. Education Policy Ana-lysis Archives, 8(1), 1–44. https://epaa.asu.edu/ojs/article/view/392/515.

Darling-Hammond, L., Holtzman, D. J., Gatlin, S. J., & Heilig, J. V.(2005). Does teacher preparation matter? Evidence about teachercertification, Teach for America, and teacher effectiveness. Edu-cation Policy Analysis Archives, 13(42), 1–47. https://doi.org/10.14507/epaa.v13n42.2005.

Dee, T. S., & Jacob, B. (2011). The impact of No Child Left Behind onstudent achievement. Journal of Policy Analysis and Manage-ment, 30, 418–446. https://doi.org/10.1002/pam.20586.

Deeks, J. J., Dinnes, J., D’Amico, R., Sowden, A. J., Sakarovitch, C.,Song, F., … European Trial Collaborative Group (2003). Evalu-ating non-randomised intervention studies. Health TechnologyAssessment, 7(27), 1–173.

Every Student Succeeds Act [ESSA], Text - S.1177 - 114th Congress(2015–2016). https://www.congress.gov/bill/114th-congress/senate-bill/1177/text.

Feuer, M. J., Floden, R. E., Chudowsky, N., & Ahn, J. (2013). Eva-luation of teacher preparation programs: purposes, methods, andpolicy options. Washington, DC: National Academy ofEducation.

Gansle, K. A., Noell, G. H., & Burns, J. M. (2012). Do studentachievement outcomes differ across teacher preparation pro-grams? An analysis of teacher education in Louisiana. Journal ofTeacher Education, 63, 304–317. https://doi.org/10.1177/0022487112439894.

Gersten, R., Fuchs, L. S., Compton, D., Coyne, M., Greenwood, C., &Innocenti, M. S. (2005). Quality indicators for group experi-mental and quasi-experimental research in special education.Exceptional Children, 71, 149–164.

Gimbert, B., Bol, L., & Wallace, D. (2007). The influence of teacherpreparation on student achievement and the application ofnational standards by teachers of mathematics in urban secondaryschools. Education and Urban Society, 40, 91–117. https://doi.org/10.1177/0013124507303993.

*Gimbert, B., Cristol, D., & Sene, A. M. (2007). The impact of teacherpreparation on student achievement in algebra in a “hard-to-staff”urban preK–12–university partnership. School Effectiveness andSchool Improvement, 18, 245–272. https://doi.org/10.1080/09243450601147528.

*Glazerman, S., Mayer, D., & Decker, P. (2006). Alternative routes toteaching: the impacts of Teach for America on student achieve-ment and other outcomes. Journal of Policy Analysis and Man-agement, 25, 75–96. https://doi.org/10.1002/pam.20157.

*Goldhaber, D. D., & Brewer, D. J. (2000). Does teacher certificationmatter? High school teacher certification status and studentachievement. Educational Evaluation and Policy Analysis, 22,129–145. https://doi.org/10.3102/01623737022002129.

Hedges, L. J., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.

Heilig, J. V., & Jez, S. J. (2010). Teach for America: a review of theevidence. Boulder, CO: Education and the Public Interest Center.Retrieved from National Education Policy Center website http://epicpolicy.org/publication/teach-for-america.

*Henry, G. T., Purtell, K. M., Bastian, K. C., Fortner, C. K.,Thompson, C. L., Campbell, S. L., & Patterson, K. M. (2014).The effects of teacher entry portals on student achievement.Journal of Teacher Education, 65, 7–23. https://doi.org/10.1177/0022487113503871.

Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G.(2003). Measuring inconsistency in meta-analyses. British Med-ical Journal, 327, 557–560. https://doi.org/10.1136/bmj.327.7414.557.

Hill, H. C., Rowan, B., & Ball, D. L. (2005). Effects of teachers’mathematical knowledge for teaching on student achievement.American Educational Research Journal, 42, 371–406. https://doi.org/10.3102/00028312042002371.

Humphrey, D., & Wechsler, M. (2007). Insights into alternative cer-tification: initial findings from a national study. Teachers CollegeRecord, 109, 483–530. https://www.tcrecord.org

Hutchison, L. F. (2012). Addressing the STEM teacher shortage inAmerican schools: ways to recruit and retain effective STEMteachers. Action in Teacher Education, 34, 541–550. https://doi.org/10.1080/01626620.2012.729483.

Losinski, M., Maag, J. W., Katsiyannis, A., & Parks Ennis, R. (2014).Examining the effects and quality of interventions based on theassessment of contextual variables: a meta-analysis. ExceptionalChildren, 80, 407–422. https://doi.org/10.1177/0014402914527243.

Mason-Williams, L. (2015). Unequal opportunities: a profile of thedistribution of special education teachers. Exceptional Children,81, 247–262. https://doi.org/10.1177/0014402914551737.

*Miller, J. W., McKenna, M. C., & McKenna, B. A. (1998). Acomparison of alternatively and traditionally prepared teachers.Journal of Teacher Education, 49, 165–176. https://doi.org/10.1177/0022487198049003002.

Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G., The PRISMAGroup (2009). Preferred reporting items for systematic reviewsand meta-analyses: the PRISMA statement. PLoS Med, 6(7),e1000097 https://doi.org/10.1371/journal.pmed1000097.

National Education Association [NEA] (2015). Research spotlight onalternative routes to teacher certification. https://www.nea.org/tools/16578.htm.

No Child Left Behind Act of 2001, 20 U.S.C. 70 § 6301 et seq. (2002).Retrieved from https://www2.ed.gov/policy/elsec/leg/esea02/107-110.pdf.

*Nunnery, J., Kaplan, L., Owings, W. A., & Pribesh, S. (2009). Theeffects of troops to teachers on student achievement: one state’sstudy. NASSP Bulletin, 93, 249–272. https://doi.org/10.1177/0192636509359338.

J Child Fam Stud

Nye, B., Konstantopoulos, S., & Hedges, L. V. (2004). How large areteacher effects. Educational Evaluation and Policy Analysis, 26,237–257.

Organization for Economic Co-Operation and Development [OECD](2015). Universal basic skills: what countries stand to gain.Washington, DC: OECD Publishing. https://doi.org/10.1787/9789264234833-en.

*Piro, J. S., Shutt, T., & Stewart, G. (2010). Value added studentachievement in alternative and traditional teacher preparationpathways. The Journal of Academic Administration in HigherEducation, 6(2), 1–8. https://jwpress.com/JLHE/JLHE.htm.

Rosenthal, R. (1979). The “file drawer problem” and tolerance for nullresults. Psychological Bulletin, 86, 638–641. https://doi.org/10.1037/0033-2909.86.3.638.

*Sharkey, N. S., & Goldhaber, D. (2008). Teacher licensure status andstudent achievement: lessons from private schools. Economics ofEducation Review, 27, 504–516. https://doi.org/10.1016/j.econedurev.2007.09.009.

Shuls, J. V., & Trivitt, J. R. (2015). Teacher effectiveness: an analysisof licensure screens. Educational Policy, 29(4), 645–675. https://doi.org/10.1177/0895904813510777.

Stronge & Associates (2013). Stronge teacher evaluation system: avalidation report. Oshkosh, WI: Strong & Associates EducationalConsulting, LLC. Retrieved from http://www.cesa6.org/effective-ness_project/Validation-Report-of-Stronge-Evaluation-System.pdf.

Stronge, J. H., Ward, T. J., & Grant, L. W. (2011). What makes goodteachers good? A cross-case analysis of the connection betweenteacher effectiveness and student achievement. Journal of Tea-cher Education, 62, 339–355.

Teach for America [TFA] (2016). https://www.teachforamerica.org/.The White House (2015). Every Student Succeeds Act: a progress

report on elementary and secondary education. Executive Officeof the President (December). https://www.whitehouse.gov/sites/whitehouse.gov/files/documents/ESSA_Progress_Report.pdf.

U.S. Census Bureau (2016). Census regions and divisions of theUnited States. https://www2.census.gov/geo/pdfs/maps-data/maps/reference/us_regdiv.pdf.

U.S. Department of Education (2002). New No Child Left Behind.https://www.gpo.gov/fdsys/pkg/PLAW-107publ110/html/PLAW-107publ110.htm.

U.S. Department of Education, Office of Postsecondary Education(2002). Meeting the highly qualified teachers challenge: thesecretary’s annual report on teacher quality. https://www2.ed.gov/about/reports/annual/teachprep/2002title-ii-report.pdf.

U.S. Department of Education, Office of Postsecondary Education(2013). Preparing and credentialing the nation’s teachers: thesecretary’s ninth report on teacher quality. https://title2.ed.gov/TitleIIReport13.pdf.

U.S. Department of Education, Race to the Top (2009). Race to the topprogram executive summary. https://www2.ed.gov/programs/racetothetop/executive-summary.pdf.

Van de Walle, J. A., Karp, K. S., & Bay-Williams, J. M. (2012).Elementary and middle school mathematics: teaching devel-opmentally (8th ed.). New York, NY: Pearson.

Wilson, D.B. (2011). Effect size calculation and basic meta-analysis.The Campbell Collaboration [Training video]. https://www.campbellcollaboration.org.

Wilson, S. M., Floden, R. E., & Ferrini-Mundy, J. (2002). Teacherpreparation research: an insider’s view from the outside. Journalof Teacher Education, 53(3), 190–204.

Wilson, S.M., Floden, R.E., & Ferrini-Mundy, J. (2001). Teacherpreparation research: current knowledge, gaps, and recommen-dations. Document R-01-3. https://www.education.uw.edu/ctp/sites/default/files/ctpmail/PDFs/TeacherPrep-WFFM-02-2001.pdf.

Wright, S. P., Horn, S. P., & Sanders, W. L. (1997). Teacher andclassroom context effects on student achievement: implicationsfor teacher evaluation. Journal of Personnel Evaluation in Edu-cation, 11, 57–67.

*Xu, Z., Hannaway, J., & Taylor, C. (2011). Making a difference? Theeffects of Teach for America in high school. Journal of PolicyAnalysis and Management, 30, 447–469. https://doi.org/10.1002/pam.20585.

Zhang, D., Wang, Q., Losinski, M., & Katsiyannis, A. (2014). Anexamination of preservice teachers’ intentions to pursue careers inspecial education. Journal of Teacher Education, 65, 156–171.https://doi.org/10.1177/0022487113510743.

J Child Fam Stud