This article was downloaded by: [Stony Brook University]On: 01 November 2014, At: 02:18Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,37-41 Mortimer Street, London W1T 3JH, UK
The Journal of Educational ResearchPublication details, including instructions for authors and subscription information:http://www.tandfonline.com/loi/vjer20
Middle School Mathematics Teachers’ ProfessionalDevelopment and Student AchievementJames A. Telese aa University of Texas, Brownsville/Texas Southmost CollegePublished online: 06 Feb 2012.
To cite this article: James A. Telese (2012) Middle School Mathematics Teachers’ Professional Development and StudentAchievement, The Journal of Educational Research, 105:2, 102-111, DOI: 10.1080/00220671.2010.521209
To link to this article: http://dx.doi.org/10.1080/00220671.2010.521209
PLEASE SCROLL DOWN FOR ARTICLE
Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) containedin the publications on our platform. However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of theContent. Any opinions and views expressed in this publication are the opinions and views of the authors, andare not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon andshould be independently verified with primary sources of information. Taylor and Francis shall not be liable forany losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoeveror howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use ofthe Content.
This article may be used for research, teaching, and private study purposes. Any substantial or systematicreproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions
The Journal of Educational Research, 105:102–111, 2012Copyright C© Taylor & Francis Group, LLCISSN: 0022-0671 print / 1940-0675 onlineDOI:10.1080/00220671.2010.521209
Middle School Mathematics Teachers’Professional Development and Student
AchievementJAMES A. TELESEUniversity of Texas, Brownsville/Texas Southmost College
ABSTRACT. Middle school mathematics teacher quality isquestionable because the number of certified mathematicsteachers considered highly qualified is low (Birman et al.,2009). The author examined Grade 8 data from the 2005National Association of Educational Progress mathematicsassessment. The purposes of the study were to (a) determinethe impact of middle school mathematics teachers’ contentknowledge and teachers’ mathematics pedagogical knowledgeon student achievement and (b) compare the effect of the de-gree to which teachers received reform-oriented professionaldevelopment activities on student achievement. The resultsindicated that mathematics content knowledge has a largerrole in predicting student achievement than mathematics ped-agogical knowledge. Also, teachers who reported participat-ing in fewer professional development activities had studentswith higher scores than those students whose teachers re-ported either participating in more professional development.Results for various professional development activities arealso presented.
Keywords: middle school mathematics teachers, National As-sociation of Educational Progress (NAEP), professional de-velopment
I n the era of the No Child Left Behind Act of 2001(NCLB; 2002), a focus has been placed on teacherquality, whereby the High Objective Uniform State
Standard of Evaluation provision permits states to allow theirteachers to demonstrate content knowledge through experi-ence, college coursework, or professional development (Bir-man et al., 2009). The intent of the program is that NCLBwill provide for effective and knowledgeable teachers.
Professional development of teachers is seen as an avenueto help young people learn complex and analytical skills nec-essary for the 21st century, which requires education systemsto provide more effective professional learning than whathas been made available in the past (Darling-Hammond, &Richardson, 2009). There is a potential to positively influ-ence student outcomes when teacher professional develop-ment focuses on student learning and pedagogical contentknowledge (Blank, de las Alas, & Smith, 2007). It is thoughtthat professional development of teachers will lead to moreeffective teachers. There has long been a debate as to what
constitutes effective teaching. One measure involves studentperformance. In the present article I present findings relatedto middle school mathematics teachers’ content knowledge,professional development, and student achievement.
Teacher Knowledge
What type of knowledge is necessary for effective teach-ing? According to Darling-Hammond (2001), teachers needknowledge related to the understanding of human develop-ment and learning in general as well as specific domains,the effects of curricular approaches and teaching strategiesfor special instances and circumstances, and assessment re-sulting in insight into students’ understanding. Similarly,10 principles were established by the Interstate TeacherSupport and Assessment Consortium (1992) for beginningteachers, which include understandings related to how chil-dren learn and how children differ in their approaches tolearning, using various instructional strategies that fostercritical thinking and problem solving, and understanding ofhow to use formal and informal assessment strategies. Be-cause content knowledge is a necessary component of effec-tive teaching, the level of middle school teachers’ contentknowledge plays an important role in how reform efforts areimplemented (Ball, Lubienski, & Spangler-Newborn, 2001).
Under the NCLB program, Birman et al. (2009) reportedthat in 2006–2007 most mathematics teachers in the UnitedStates met the requirements to be deemed highly qualified,even though there were a variety of definitions used by thestates. However, the percentage of teachers not highly qual-ified was higher for middle school teachers (Birman et al.,2009). NCLB calls for middle and secondary teachers topass rigorous state certification tests in mathematics or havea major in mathematics.
Yet, in a review of 57 studies conducted by Wilson, Flo-den, and Ferrini-Mundy (2002), which focused on research
Address correspondence to James A. Telese, Department of Teach-ing, Learning, and Innovation,University of Texas, Brownsville/TexasSouthmost College, 80 Fort Brown, Brownsville, TX 78520, USA.(E-mail: [email protected])
Dow
nloa
ded
by [
Ston
y B
rook
Uni
vers
ity]
at 0
2:18
01
Nov
embe
r 20
14
The Journal of Educational Research 103
in teacher preparation, there were no reports identified thatdirectly related prospective teachers’ subject matter knowl-edge with student achievement. In their review, four of sevenstudies were identified that related to mathematics teachers’subject matter knowledge. Wilson et al. found that a positiverelationship existed between teachers’ subject matter knowl-edge and higher student achievement. They concluded thateducation coursework is an essential ingredient in teacherperformance. Regarding subject matter coursework, there islittle effect on student achievement when teachers reporthaving more than four to six courses. However, there is athreshold effect: Monk (1994) found little improvement instudent achievement when teachers took more than fiveundergraduate mathematics courses and that mathematicseducation courses contributed more to student achievementgains than undergraduate mathematics courses. These find-ings suggest that undergraduate mathematics content andmathematics education courses are necessary to positivelyaffect student achievement, with mathematics educationcourses having a greater impact.
Moreover, Wilson et al. (2002) noted that researchon pedagogical preparation is very scarce, with few orno studies having been conducted on the relationshipbetween pedagogical preparation and student learningor teacher behavior. They concluded that because ofinadequate measurements, it is unclear as to the degreeof this association, showing some benefit for pedagogicalpreparation, which includes instructional methods, learningtheories, and educational psychology.
There are very few studies that have examined the rela-tionship between teachers’ knowledge and student achieve-ment. Darling-Hammond (2000) examined National As-sociation of Educational Progress (NAEP) data and founda positive and statistically significant relationship amongteachers’ certification status, degree in the field, and studentoutcomes. The important influence of pedagogical prepa-ration was identified in an interpretive study in whichsecondary teachers with no pedagogical preparation werelimited in their ability to engage students in instruc-tion (Darling-Hammond, 2000; Felter, 1999; Goldhaber &Brewer, 2000). Teacher education courses experienced byveteran teachers were thought to be of little or inconsequen-tial use to them in practice (Kagan, 1992). Many teachersviewed them as irrelevant and had to learn how to teachon their own in their school (Zeichner, 1993). The value ofthe impact of teacher education coursework is reported tobe inconclusive due to research methods used and the smallsample sizes in interpretive studies (Wilson et al., 2002).Hence, the research on the influence of teacher educationon student achievement appears to be scarce in quality.
Teacher Professional Development
NCLB dictates that states ensure their teachers re-ceive high-quality professional development without defin-ing high-quality professional development (Borko, 2004).
Little (1987) defined professional development as “any activ-ity that is intended partly or primarily to prepare paid staffmembers for improved performance in present or future rolesin the school districts” (p. 491). Assuming that professionaldevelopment should focus on aspects of improving teachers’knowledge of content and pedagogy, it follows that profes-sional development for middle school mathematics teach-ers should hinge on topics that enhance teachers’ contentknowledge and instructional techniques.
A key component in teachers’ lifelong learning processis continual professional development. However, it is oftenviewed as being fragmented, on an as-needed basis, andrelatively superficial (Loucks-Horsley, Love, Stiles, Mundry,& Hewson, 2003). Professional development activities thatmay improve teachers’ knowledge and skills range fromformal, structured topic-specific workshops to informal dis-cussions in hallways (Desimone, 2009). There is a trend inteachers’ professional development to connect it to studentlearning with an ultimate goal of closing achievement gapsamong student groups (Desimone, 2009; Loucks-Horsleyet al., 2003), as reported in Figure 1, which presents a coreconceptual framework for studying the effects of professionaldevelopment on teachers and students (Desimone, 2009).
The context includes teacher characteristics, such as ex-perience, knowledge, beliefs, and attitudes (e.g., Borko &Putman, 1996; Franke, Carpenter, Levi, & Fennema, 2001),and student characteristics, such as achievement and so-cioeconomic status (Darling-Hammond & Sykes, 1999). Inthe model, context has the potential to influence the corecomponents of professional development and its outcomesrelated to the teachers and students.
Criticism of teacher professional development stems fromits expense and unseen benefits. In 2004–2005, local, state,and federal agencies spent about $1.5 billion on teacher pro-fessional development (Birman et al., 2007). Professional de-velopment for inservice teachers is an expensive endeavor,and teachers may experience workshops that do not directlyaddress their needs, making them a waste of time and money(Cohen & Hill, 2000; Wilson, Lubienski, & Mattson, 1996).Professional development is rarely considered developmen-tal because there few programs that address teachers’ learningand the practices they are to enact, and associated math-ematical practices (Heaton, 2000). Although professionaldevelopment is an expensive endeavor, it is a critical aspectof teachers’ professional life.
The National Council of Teachers of Mathematics(NCTM) established standards for the professional devel-opment of mathematics teachers (Martin, 2007). The coun-cil contended that professional development should focuson five standards (which parallels Desimone’s [2009] coreconceptual framework): (a) knowing mathematics contentand school mathematics, (b) knowing students as learners ofmathematics, (c) knowing mathematics pedagogy, and (d)developing as a mathematics teacher (Martin, 2007). Theprevious list suggests that teachers, through professional de-velopment, become reflective in their practice. Teachers
Dow
nloa
ded
by [
Ston
y B
rook
Uni
vers
ity]
at 0
2:18
01
Nov
embe
r 20
14
104 The Journal of Educational Research
FIGURE 1. Core conceptual framework for studying the effects of professional development on teachers and students (adaptedfrom Desimone, 2009).
ought to examine and revise their assumptions about thenature of mathematics, how it should be taught, how stu-dents learn mathematics, and analyze the effectiveness oftheir teaching (Loucks-Horsley et al., 2003; Martin, 2007).
Professional development should be designed to enhancemiddle school mathematics teachers’ knowledge of math-ematics and their ability to effectively teach mathematicsto culturally and socially diverse students (Stevens, Harris,Aguirre-Munoz, & Cobbs, 2009). Loucks-Horsley et al.(2003) also considered the context of teacher professionaldevelopment to be important and included several con-textual factors, including students, standards, and studentlearning needs, practices regarding curriculum, instruction,and assessment, and national, state and local policies.Hence, it is apparent that teachers must possess contentknowledge and knowledge of how to effectively implementmathematics instruction.
The research on teacher learning is a relatively youngfield; a knowledge base is beginning to form on the topic ofthe impact of teacher professional development on studentoutcomes (Borko, 2004). There are few studies that squarelyattacked this issue (e.g., Desimone, 2009). When studies onteacher learning are conducted, they tend to focus on whatthe teacher has learned and the extent to which the pro-gram was implemented (Borko, 2004). For example, Penuel,Fishman, Yamaguchi, and Gallagher (2007) examined theimplementation of the GLOBE earth science program bysurveying their teachers about their knowledge, how they
changed and continued professional development, and theextent the program aligned to the teachers’ districts stu-dent learning goals. Smith, Desimone, and Ueno (2005)conducted a study using the 2000 NAEP data set that exam-ined the relationship between teacher quality and the use ofreform-oriented instructional strategies. The present studydiffers from Smith et al.’s study, in that Smith et al. usedprofessional development as a whole, examining the rela-tionship between the number of workshops and professionaldevelopment hours with other teacher credential variables,whereas in the present study I examined the relationshipbetween the extent of different professional developmentactivities and student achievement.
The question remains, how much professional develop-ment is enough (Desimone, 2009)? The question has re-mained unanswered since Stout (1996) pointed out that“no evidence exists to allow a sensible policy decision aboutthe amount of staff development needed to accomplish anygiven purpose” (p. 6). Moreover, Desimone (2009) con-cluded that more work in connecting changes in teach-ing practices through professional development to studentachievement is needed.
The purposes of this study were twofold: (a) to de-termine which type of knowledge, content knowledge ormathematics education knowledge, is the best predictorof student achievement and (b) to determine the impactof various professional development activities on studentachievement.
Dow
nloa
ded
by [
Ston
y B
rook
Uni
vers
ity]
at 0
2:18
01
Nov
embe
r 20
14
The Journal of Educational Research 105
The research questions were the following:
Research Question 1: What is the relationship between stu-dent achievement and the number of advanced mathe-matics courses taken as either an undergraduate or gradu-ate student?
Research Question 2: What is the relationship between stu-dent achievement and the number of mathematics educa-tion courses taken as either an undergraduate or graduatestudent?
Research Question 3: What type of professional developmentactivity for middle school mathematics teachers has thegreatest impact on student achievement?
Research Question 4: What level of professional developmentfor middle school mathematics teachers allows for greaterstudent achievement?
Method
Data
Data were compiled using the 2005 NAEP database. Theanalysis was conducted during a training workshop sponsoredby the National Center of Educational Statistics (NCES).The teacher questionnaire was administered as part of NAEPmathematics assessment. The questionnaire was given toteachers of students who were randomly selected. NAEPrandomly selects 100 schools from each jurisdiction andthen randomly selects 60 students from each school (NCES,2009c). From the NAEP online glossary, the term jurisdictionis defined as
any government-defined geographic area sampled in theNAEP assessment e.g., a state, the District of Columbia, aUnited States territory, a Trial Urban District, the Depart-ment of Defense Domestic Dependent Elementary and Sec-ondary Schools (DDESS), a subdivision within a state orcounty. (NCES, 2009a)
The NAEP is administered to students in Grades 4, 8, and12, resulting in a national sample of approximately 2,800students for the Grade 8 data set (NCES, 2009c). This studyused the Grade 8 data set of over 100,000 eighth-grade stu-dents and their teachers, resulting in a stratified nationalprobability sample. The teacher survey in NAEP is oneof the few nationally representative data sets that makesinquires about teacher characteristics, such as educationalbackground, preparation to teach specific content, partici-pation in professional development activities, and using awide range of teaching strategies. Although NAEP is notdesigned to gauge attributes of the teacher population, theresponses from teachers regarding each of their classes inwhich a student was sampled makes it possible to examineparticular teacher characteristics and their professional de-velopment experiences in relation to student achievement(NCES, 2009b).
Sample
NAEP employs a complex multistage sampling design.Schools are sampled from each jurisdiction across the coun-try including Washington, DC, and Puerto Rico, and stu-dents are sampled from each selected school. Oversamplingof some schools is used to ensure that data include theseschools with smaller numbers of certain categories (NCES,2009c). To generalize findings to the population of Grade 8mathematics teachers of students who were sampled, sam-pling weights are used to adjust for the over sampling in theanalyses (NCES, 2009d). These are known as full sampleweights. In addition, replicate weights are created, whichare used to calculate the variances of survey estimates usinga jackknife repeated replication method. Thus, the processproduces unbiased estimates of sampling variances. In ad-dition, the various weighting procedures were repeated oneach set of replicate weights to appropriately reflect the im-pact of the weighting adjustments on the sampling varianceof a survey estimate (NCES, 2009d). The teacher file wasmerged with the student file so that data from students whoseteachers were surveyed could be analyzed.
Measures
Variables in this study were chosen to reflect the math-ematics teachers’ knowledge and skills and related profes-sional development activities that the literature reports asimportant factors contributing to student achievement. Thecodes for ethnicity were White (0), Black (1), Hispanic (2),Asian or Pacific Islander (3), and American Indian or AlaskaNative (4). The codes for gender were male (0) and female(1). Mathematics content knowledge was represented by thefollowing variable: “As part of either your undergraduateor graduate coursework, how many advanced mathematicscourses have you taken?” The options for responses includednone (coded as 0), one or two courses (coded as 1), three orfour courses (coded as 2), and five courses or more (coded as3). Teacher pedagogical content knowledge was representedby the following item: “As part of either your undergraduateor graduate coursework, how many mathematics educationcourses did you take?” The response options and the codingwere the same as for the number of content courses taken.The survey also asked teachers to indicate the type of pro-fessional development activities that they had experiencedwithin the 2 years of taking the survey: “Consider all ofthe professional development activities you participated induring the last 2 years. To what extent did you learn abouteach of the following topics?” The response options included“not at all,” “small extent,” “moderate extent,” or “large ex-tent,” coded as 0, 1, 2, and 3, respectively. Parenthetically,the 2005 NAEP teacher survey did not specify definitionsfor the terms “small extent,” “moderate extent,” or “largeextent.”
The dependent variable was the composite mathematicsplausible value score. Plausible values are derived variables
Dow
nloa
ded
by [
Ston
y B
rook
Uni
vers
ity]
at 0
2:18
01
Nov
embe
r 20
14
106 The Journal of Educational Research
in response to the complex sampling design with the intentto provide valid estimates of population characteristics(NCES, 2009c). The process involves calculating marginalmaximum likelihood estimates, the matrix of effects, andthe residual covariance matrix, resulting in five sets ofdistributional draws called plausible or imputed values, forall sampled students (NCES, 2009c).
The mathematics composite score is a measure of overallperformance. NAEP provides scale scores in the followingcontent areas of mathematics: (a) geometry, (b) measure-ment, (c) algebra, (d) data analysis and probability, and (e)number properties and operation. The scale ranges from 0to 500. There are three levels, basic (262 to 299), proficient(300 to 333), and advanced (334 to 500; NCES, 2009c).
The professional development categories used in this studywere chosen based on the literature suggesting that teacherswho receive professional development in content knowl-edge and pedagogical knowledge are more effective (e.g.,Darling-Hammond & Richardson, 2009). Those providedon the 2005 NAEP teacher survey included training in howstudents learn, learning about mathematics theory, train-ing in the use of curriculum materials, training related toinstructional strategies, use of manipulatives, using calcula-tors, training in assessment, training in how to teach diversestudents, and training on state mathematics assessment, allof which were used as dependent variables for t tests. Notethat the 2005 NAEP Grade 8 mathematics teacher surveyasked about the extent that teachers attended professionaldevelopment workshops or seminars, using the categories:(a) not at all, (b) small extent, (c) moderate extent, and (d)large extent, without listing specific number of hours.
Results
Bonferroni Corrections
Data were analyzed using AM Statistical Software Beta(Version 0.06.03) from the American Institutes for Research(American Institutes for Research & Cohen, n.d.). To makeaccurate comparisons of multiple groups, the software em-ploys Bonferroni adjustments to control Type I experiment-wise error rate inflation. This means that some possibly sig-nificant differences may not be found, but that the overallexperiment- or family-wise error rate will not become exor-bitant (NCES, 2009b).
Regression Results
Multiple regression and t tests, with Bonferroni adjust-ments, were conducted. The t tests were used to determinedifferences in mean performance for students whose teachersexperienced various levels of professional development ac-tivities. Table 1 presents the results of a multiple regressionanalysis using gender, ethnicity, the number of mathemat-ics content, and mathematics education courses as predic-tors, and the composite plausible value mathematics scoreas the criterion variable. The variables were dummy coded;
TABLE 1. Multiple Regression Model Using Gender,Ethnicity, Number of Advanced Mathematics Courses,and Number of Mathematics Education Courses asPredictors of Student Achievement (N = 135,681)
Parameter Estimate SE z p > z
Constant 281.90 0.89 315.98 .001Female −1.88 0.26 −7.31 .001Black −34.23 0.53 −64.86 .001Hispanic −24.90 0.36 −68.88 .001Asian American 3.29 0.92 3.59 .001American Indian −25.30 1.00 −25.35 .001One or two advanced
mathematics courses4.44 0.78 5.69 .001
Three or four advancedmathematics courses
8.89 0.73 12.12 .001
Five advancedmathematics coursesor more
13.07 0.66 19.93 .001
One or twomathematicseducation courses
−0.66 0.77 −0.88 .380
Three or fourmathematicseducation courses
−0.14 0.80 −0.17 .870
Five mathematicseducation courses ormore
0.49 0.78 0.64 .530
Note. For the model, R2 = .18, p < .001.
for example, the category “White” was coded as 0 so that acomparison was made between this category and the otherethnic categories. A similar coding method was used for thenumber of mathematics courses and mathematics educationcourse which were collapsed into the present categories inwhich 1 represented one or two courses, 2 signified three orfour courses, and 3 reflected five or more courses. The modelproduced an R2 value of .18, which is typical for this type ofanalysis of NAEP data, with several other variables.
Female eighth-grade students had lower performancethan their male counterparts. Asian American students hadgreater achievement levels than any other ethnic group. Theassociation between mathematics course taking, whetherfrom one to two, with a beta value of 4.44; three or four,with a beta value of 8.89; five or more courses with, a betavalue of 13.07; and student achievement; each of whichwas found to be statistically significant at p < .001. Thisindicates within the context of the final model more ad-vanced mathematics classes being taken predicted higherstudent achievement. However, the association between thesame numbers of mathematics education courses as previouswas not statistically significant, regardless of the number ofcourses. The number of mathematics education courses hada slightly negative or no impact on the model.
The survey listed 12 different professional developmentactivities involving learning about mathematics content
Dow
nloa
ded
by [
Ston
y B
rook
Uni
vers
ity]
at 0
2:18
01
Nov
embe
r 20
14
The Journal of Educational Research 107
TABLE 2. Results of a Multiple Regression ModelUsing the Extent of Professional Development andStudent Achievement
Parameter Estimate SE z p > z
Constant 280.73 0.75 374.80 .001Small extent 0.48 0.89 0.54 .600Moderate extent −1.24 0.83 −1.50 .130Large extent −2.11 1.00 −2.11 .040
Note. For the model, R2 = .01, p < .001.
and pedagogical strategies. Respondents were asked thefollowing: “Consider all of the professional developmentactivities you participated in during the last 2 years. Towhat extent did you learn about each of the followingtopics?” Response choices were “not at all,” “small extent,”“moderate extent,” and “large extent.” A multiple regressionwas conducted, using the level of professional developmentas the predictor variable, and was dummy coded, when com-pared with no professional development, with the criterionvariable the composite mathematics plausible value.
Overall, there was a negative prediction for a large ex-tent of professional development and student achievement,with a beta value of −2.11 (p < .05). Hence, students whoseteachers reported receiving a large extent of professional de-velopment training were associated with lower achievementscores. The association between teachers whose students re-ported experiencing a small to moderate extent of profes-sional development was weak in relation to achievement(see Table 2).
Results of t Test
To determine the impact of various professional develop-ment activities on student achievement, t tests were con-ducted, using Bonferroni adjustments, that compared thelevels of professional development in the specific activitywith composite mean scores of students whose teachers re-ported no professional development or a small extent, invarious categories, to those students’ composite mean scoreswhose teachers’ reported attending professional develop-ment in a small, moderate, or large extent. To limit family-wise error, the preceding variables were dummy coded withthe variable “no professional development” as the referencecategory.
Knowledge of student learning and mathematics theory. Theresults indicate that students of teachers reporting a smallextent of training in how students learn mathematics scoredhigher (M = 281.21, SD = 35.50) than those students whoseteachers reported receiving professional development at amoderate (M = 279.49, SD = 35.56) or large extent (M =278.63, SD = 36.24; see Table 3). There was no difference
in student achievement for those students whose teachersreported receiving no professional development when com-pared with a small extent of professional development inmathematics theory or applications. However, there was astatistically significant difference among those students withteachers reporting “not at all” (M = 280.97, SD = 35.38),“moderate extent” (M = 278.93, SD = 36.00), and “large ex-tent” (M = 278.00, SD = 36.49). Hence, students of teacherswho reported more than a small amount of professional de-velopment in mathematics theory or applications performedworse than students whose teachers reported a moderate orlarge extent of professional development in mathematicstheory or applications. Regarding professional developmentin mathematics content standards, the mean scores werehigher for students whose teachers attended workshops oncontent standards at each level when compared with thosestudents’ teachers who received no training in content stan-dards. However, there was only a statistically significant dif-ference, t(108) = 3.28, p < 0.001 between “not at all” (M= 277.61, SD = 35.96) and “small extent” (M = 281.88,SD = 36.11).
Curriculum and instructional methods. A small extent oftraining in curriculum materials produced greater achieve-ment than for students of teachers who received no trainingin curriculum materials (not at all: M = 278.19, SD = 35.84;small extent: M = 281.16, SD = 35.36), t(119) = 2.59,p < 0.5. The differences were not statistically significantfor a moderate extent or large extent. The students’ perfor-mance tended to decrease as the extent increased. Profes-sional development in instructional methods was found tohave little or no impact on student achievement regardlessof the reported levels of professional development. In thisstudy, students of teachers who attended various levels oftraining in instructional methods performed statistically atsimilar levels.
Instructional tools. Students of teachers who reported asmall extent of professional development in the use of ma-nipulatives (M = 281.34, SD = 35.47) performed at thesame level as those not receiving professional developmentin manipulatives (M = 282.02, SD = 35.11). Those studentswhose teachers reported a moderate extent (M = 278.49,SD = 35.80), t(120) = 3.89, p < 0.001, and a large ex-tent (M = 278.25, SD = 36.41) of professional develop-ment performed at a lower level, t(110) = 3.96, p < 0.001,than did those who teachers received no professional de-velopment in manipulative use. With regard to training inthe use of calculators, students of teachers who reportedno professional development (M = 278.20, SD = 35.15)had a statistically lower mean score than did those stu-dents whose teachers reported a small extent (M = 280.42,SD = 35.22), moderate extent (M = 280.84, SD = 35.80),or a large extent (M = 280.55, SD = 37.55) of professionaldevelopment.
Dow
nloa
ded
by [
Ston
y B
rook
Uni
vers
ity]
at 0
2:18
01
Nov
embe
r 20
14
108 The Journal of Educational Research
TABLE 3. Mean NAEP Grade 8 Mathematics Composite Score for Various Professional Development (PD) Activities
Type of PD activity Weighted N M SE SD t Cohen’s d
How students learnNot at all 365,583 280.73 0.75 35.93Small extent 1,032,758 281.21 0.41 35.50Moderate extent 1,336,295 279.49 0.37 35.56 3.17∗∗∗ .03Large extent 596,287 278.63 0.62 36.24 3.50∗∗∗ .06
Mathematics theory or applicationsNot at all 662,465 280.97 0.52 35.38Small extent 1,213,848 281.09 0.35 35.30Moderate extent 1,053,952 278.93 0.43 36.00 2.92∗∗∗ .06Large extent 393,017 278.00 0.77 36.49 3.25∗∗∗ .08
Content standards in mathematicsNot at all 167,091 277.61 1.10 35.96Small extent 471,329 281.88 0.64 36.11 3.28∗∗∗ .12Moderate extent 1,181,142 279.74 0.44 35.85Large extent 1,501,197 279.98 0.35 35.40
Curricular materials available in mathematics (units, texts)Not at all 269,893 278.19 0.82 35.84Small extent 813,490 281.16 0.48 35.36 2.59∗ .08Moderate extent 1,293,115 279.53 0.42 35.56Large extent 946,590 279.97 0.43 36.17
Instructional methods for teaching mathematicsNot at all 260,416 279.53 0.83 35.68Small extent 838,017 281.14 0.44 35.10Moderate extent 1,336,200 280.33 0.36 35.74Large extent 898,936 278.63 0.47 36.19
Effective use of manipulatives in mathematics instructionNot at all 536,594 282.02 0.69 35.11Small extent 1,167,385 281.34 0.37 35.47Moderate extent 1,058,275 278.49 0.50 35.80 3.89∗∗∗ .10Large extent 570,374 278.25 0.63 36.41 3.96∗∗∗ .11
Effective use of calculators in mathematics instructionNot at all 798,696 278.20 0.45 35.15Small extent 1,179,591 280.42 0.37 35.22 3.69∗∗∗ .06Moderate extent 872,523 280.84 0.51 35.80 3.65∗∗∗ .07Large extent 482,301 280.55 0.71 37.55 2.53∗ .06
Methods for assessing students in mathematicsNot at all 478,616 280.80 0.65 35.88Small extent 1,181,116 281.80 0.38 35.50Moderate extent 1,163,750 279.04 0.39 35.45 2.25∗ .05Large extent 507,600 277.33 0.63 36.43 3.73∗∗∗ .10
Strategies for teaching mathematics to students from diversebackgrounds (including English language learners)
Not at all 1,184,775 283.65 0.39 34.69Small extent 1,199,608 280.66 0.39 35.52 5.33∗∗∗ .09Moderate extent 686,241 275.73 0.52 36.11 12.93∗∗∗ .22Large extent 260,479 272.14 0.93 37.54 10.68∗∗∗ .32
Preparation of students for district and state testsNot at all 384,499 282.77 0.69 35.90Small extent 739,045 280.58 0.48 36.25 2.55∗ .06Moderate extent 1,100,533 278.84 0.41 35.81 5.15∗∗∗ .11Large extent 1,103,223 279.81 0.39 35.12 3.78∗∗∗ .08
Note. NAEP = National Association of Educational Progress; PD = Professional development.∗p < .05. ∗∗∗p < .001.
Assessment training. Students of teachers who reporteda small extent of professional development in assessmentof students in mathematics (M = 281.80, SD = 35.50)
performed at a statistically similar level as those teacherswho did not receive professional development in assess-ment (M = 280.80, SD = 35.88). The students whose
Dow
nloa
ded
by [
Ston
y B
rook
Uni
vers
ity]
at 0
2:18
01
Nov
embe
r 20
14
The Journal of Educational Research 109
teachers who received either a moderate (M = 279.04, SD= 35.45) or large extent (M = 277.33, SD = 36.43) ofprofessional development in methods for assessing studentshad lower performance than those students whose teachersdid not receive training. This indicates that some trainingin methods for assessing students can have a positive effecton student achievement. In addition, professional develop-ment in preparing students for district and state tests tend tolower achievement as teachers of students who were admin-istered NAEP reported increased extent. Students of thoseteachers who reported “not at all” had a mean of 282.77(SD = 35.90), whereas those who reported a small extenthad a mean of 280.58 (SD = 36.25), a moderate extent hada mean of 278.84 (SD = 35.81), and a large extent had amean of 279.81 (SD = 35.12). Each of the previous meanswas statistically different from the mean for the professionaldevelopment category “not at all.”
Teaching culturally diverse students. Students of teachers,regardless of the extent of professional development,performed worse than those students whose teach-ers received no training in teaching diverse students.The mean for students whose teachers did not re-ceive the professional development was higher (M =283.65, SD = 34.69) than the means for those studentswhose teachers reported a small (M = 280.66, SD = 35.52),moderate (M = 275.73, SD = 36.11), and large extent(M = 272.14, SD = 37.54).
Discussion
It is important to consider that the NAEP mathematicsscale makes it possible to examine relationships between stu-dents’ performance and various factors measured by NAEP.In this study, teacher variables were merged with studentvariables to determine relationships between teacher vari-ables and student variables. A limitation of the study is thatalthough a relationship may exist between achievement andanother variable, the underlying cause is not revealed andmay be influenced by a number of other variables. I didnot examine the unmeasured variables. The results are mostuseful when they are considered in combination with otherknowledge about the student population and the educationalsystem (NCES, 2009c).
The results of this study indicated that the number ofmathematics teachers’ content courses was a better predictorof student achievement than the number of mathematicseducation courses. This finding corroborates early research(e.g., Darling-Hammond, 2001), and it is contrary to whatWilson et al. (2002) reported in their review of researchthat education coursework was a better predictor for teachersuccess than subject matter coursework, if teacher successis predicated on student achievement. Future researchersshould address this relationship through the use of quasi-experimental or casual comparative methods.
The variables selected for this study were chosen becausethey represent aspects or categories of activities for profes-sional development that are thought to improve teacherperformance as reflected by student achievement. This studyprovides a first step toward examining the questions raised byDesimone (2009) regarding how much professional devel-opment is enough and what the focus areas of professionaldevelopment activities should be. Professional developmentmay be conducted in a variety of ways, such as study groups,curriculum development, or mentoring (Loucks-Horsleyet al., 1998), but it is commonly in a form of workshops,seminars, or college coursework (Garet, Birman, Porter,Yoon, & Desimone, 2001). The findings of this studyregarding middle school mathematics teacher preparation,including professional development, raised more questions.It appears that, overall, middle school students whose math-ematics teachers receive only some extent of professionaldevelopment performed better on the NAEP than thosewho had received moderate to large extent of training. Thisfinding begins to shed some light on stated concerns, such ashow much professional development is enough and what thefocus of the professional development should be. Perhapsreceiving too much professional development may reducestudent achievement. The findings show that althoughcontent knowledge is important for teachers to possess,any more than a small extent of professional developmentin this area was associated with lower achievement whencompared to teachers not receiving any training.
The study revealed that certain topics for professionaldevelopment may be more effective than others in raisingstudent achievement. Professional development in how stu-dents learn tends to adversely affect student achievement.Professional development topics that include training incontent standards, the available curriculum materials, in-structional methods for teaching mathematics, and effectiveuse of calculators in mathematics instruction where foundto be positively related to student achievement when com-pared with no professional development as long as teach-ers received a small extent of professional developmentin these areas. A surprising result is students of teacherswho received a small extent of professional developmentin methods for assessing students performed at the samelevel as teachers receiving no training at all, whereas stu-dents whose teachers received more than a small extentof training were found to have lower achievement. Thismay imply professional development in methods of assess-ment may be effective if it does not go beyond a certainpoint, yet to be determined. Another surprising result wasprofessional development in strategies for teaching mathe-matics to students from diverse backgrounds produced stu-dent achievement levels lower than if teachers received notraining at all, regardless of the extent of the training theyreceived.
In summary, the survey did not clearly specify what wasmeant by the levels of professional development (i.e., smallextent); I assume that it was left up to the respondent to make
Dow
nloa
ded
by [
Ston
y B
rook
Uni
vers
ity]
at 0
2:18
01
Nov
embe
r 20
14
110 The Journal of Educational Research
that judgment within his or her own situation. Yet, the find-ings suggest that when planning professional developmentfor middle school mathematics teachers, the plan should in-volve a small extent and on the topics that have potential toraise student achievement. It appears that going to a mod-erate level can have the potential to lower student achieve-ment. These results seem to answer the concern of how muchprofessional development is enough, and yet it raises morequestions. Why did students of teachers who reported receiv-ing professional development in teaching diverse studentsscore worse than those who did not receive this training? Per-haps those teachers were teaching that particular group (e.g.,low-performing students). Another possible explanationaligns with Desimone’s (2009) core conceptual framework,which includes teachers’ beliefs and attitudes, which werenot examined in this study but may offer a possible explana-tion for these results. For example, could it be that teacherswho chose to participate in more professional developmentactivities deem themselves not to be highly qualified, andare in need of more help? Likewise, teachers who think ofthemselves as highly qualified may feel that they do not needto participate in professional development to become betterteachers and, thus, have students with higher achievement.This raises yet another question: Why do teachers partici-pate in professional development? Further research is neededto shed light on the questions such as how much professionaldevelopment is enough? In what professional developmenttopics should middle school mathematics teachers haveexperience?
ACKNOWLEDGMENTS
The author would like to thank the anonymous reviewer for thethoughtful and constructive comments and Bruce Thompson at TexasA&M University, College Station, for his review of an earlier draft of themanuscript.
REFERENCES
American Institutes for Research & Cohen, J. (n.d.). AM statistical softwarebeta (Version 0.06.03) [Computer software]. Washington, DC: AmericanInstitutes of Research. Retrieved from http://am.air.org/default.asp
Ball, D., Lubienski, S., & Newborn, D. (2001). Research on teaching math-ematics: The unsolved problem of teachers’ mathematical knowledge.In V. Richardson (Ed.), Handbook of research on teaching (4th ed., pp.433–456). Washington, DC: American Education Research7break; As-sociation.
Birman, B., Boyle, A., Le Floch, K., Elledge, A., Holtzman, D., Tomsen, K.,. . . Yoon, K.S. (2009). State and local implementation of the No Child LeftBehind Act: Vol. 8. Teacher quality under NCLB: Final report. Washington,DC: U.S. Department of Education.
Birman, B., Le Floch, K. C., Klekotka, A., Ludwig, M., Taylor, J., Walters,K., . . . O’Day, J. (2007). State and local implementation of the No ChildLeft Behind Act: Vol. 2. Teacher quality under NCLMB: Interim Report.Washington, DC: U.S. Department of Education; Office of Planning,Evaluation, and Policy Development; Policy and Program Studies Ser-vices.
Blank, R. K., de las Alas, N., & Smith, C. (2007). Analysis of the qualityof professional development programs for mathematics and science teachers.Washington, DC: Council of Chief State School Officers.
Borko, H. (2004). Professional development and teacher learning: Mappingthe terrain. Educational Researcher, 33(8), 3–15.
Borko, H., & Putman, R. (1996). Learning to teach. In D. C. Berliner & R.C. Calfee (Eds.), Handbook of educational psychology (pp. 673–708). NewYork, NY: Macmillan.
Cohen, D. K., & Hill, H. (2000). State policy and classroom perfor-mance: Mathematics reform in California. Teachers College Record, 10,294–343.
Darling-Hammond, L. (2000). Teacher quality and student achievement:A review of sate policy evidence. Education Policy Analysis Archives, 8,Article 1. Retrieved from http://epaa.asu.edu/epaa/v8n1/2000
Darling-Hammond, L. (2001). Standard setting in teaching: Changes inlicensing, certification, and assessment. In V. Richardson (Ed.), Hand-book of research on teaching (4th ed., pp. 433–456). Washington, DC:American Education Research Association.
Darling-Hammond, L., & Richardson, N. (2009). Teacher learning: Whatmatters? Educational Leadership, 66(5), 46–53.
Darling-Hammond, L., & Sykes, G. (Eds.). (1999). Teaching as the learningprofession: Handbook of policy and practice. San Francisco, CA: Jossey-Bass.
Desimone, L. M. (2009). Improving impact studies of teachers’ professionaldevelopment: Toward better conceptualizations and measures. Educa-tional Researcher, 38, 181–199.
Felter, M. (1999). High school staff characteristics and mathematics testresults. Education Policy Analysis Archives, 7, Article 9. Retrieved fromhttp://epaa.asu.edu/epaa/v7n9.html
Franke, M. L., Carpenter, T. P., Levi, L., & Fennema, E. (2001). Cap-turing teachers’ generative change: A follow-up study of professionaldevelopment in mathematics. American Educational Research Journal, 38,653–689.
Garet, M., Birman, B., Porter, A., Yoon, K., & Desimone, L. (2001).What makes professional development effective? Analysis of a nationalsample of teachers. American Educational Research Journal, 38, 915–945.
Goldhaber, D. D., & Brewer, D. J. (2000). Does teacher certifica-tion matter? High school teacher certification status and studentachievement. Educational Evaluation and Policy Analysis, 22, 129–145.
Heaton, R. M. (2000). Teaching mathematics to the new standards: Relearningthe dance. New York, NY: Teachers College Press.
Interstate New Teacher Support and Assessment Consortium. (1992).Model standards for beginning teacher licensing and development: A resourcefor state dialogue. Washington, DC: Council of Chief State School Offi-cers.
Kagan, D. M. (1992). Professional growth among preservice and beginningteachers. Review of Educational Research, 62, 129–169.
Little, J. W. (1987). Teachers as colleagues. In V. Richardson-Koehler(Ed.), Educators’ handbook: A research perspective (pp. 491–518). NewYork, NY: Longman.
Loucks-Horsley, S., Love, N., Stiles, K. E., Mundry, S., & Hewson, P.W. (2003). Designing professional development for teachers of science andmathematics. Thousand Oaks, CA: Corwin Press.
Martin, T. (Ed.). (2007). Mathematics teaching today: Improving practice,improving student learning (2nd ed.). Reston, VA: National Council ofTeachers of Mathematics.
Monk, D. H. (1994). Subject area preparation of secondary mathematicsand science teachers and student achievement. Economics of EducationReview, 13, 125–145.
National Center for Education Statistics. (2009a). The NAEP glossary ofterms. Retrieved from http://nces.ed.gov/nationsreportcard/glossary.asp#j
National Center for Education Statistics. (2009b). NAEP technical report:Processing assessment materials. Retrieved from http://nces.ed.gov/nationsreportcard/tdw/process/materials/2004-2005/counts2004/partic.asp
National Center for Education Statistics. (2009c). NAEP technical re-port: Sample design. Retrieved from http://nces.ed.gov/nationsreportcard/tdw/sample design/index.asp
National Center for Education Statistics. (2009d). NAEP technical re-port: Weighting. Retrieved from http://nces.ed.gov/nationsreportcard/tdw/weighting/2004 2005/weighting 2005.asp
No Child Left Behind Act of 2001, Pub. L. No. 107-110, 2204 (2002).Penuel, W. R., Fishman, B. J., Yamaguchi, R., & Gallagher L. (2007).
What makes professional development effective? Strategies that fostercurriculum implementation. American Educational Research Journal, 44,921–958.
Smith, T., Desimone, L., & Ueno, K. (2005). “Highly qualified” to do what?The relationship between NCLB teacher quality mandates and the use of
Dow
nloa
ded
by [
Ston
y B
rook
Uni
vers
ity]
at 0
2:18
01
Nov
embe
r 20
14
The Journal of Educational Research 111
reform-oriented instruction in middle school mathematics. EducationalEvaluation Policy and Analysis, 27, 75–109.
Stevens, T., Harris, G., Aquirre-Munoz, Z., & Cobbs, L. (2009). A casestudy approach to increasing teachers’ mathematics knowledge for teach-ing and strategies for building students’ mathematics self efficacy. Inter-national Journal of Mathematical Education in Science and Technology, 40,903–914.
Stout, R. T. (1996). Staff development policy: Fuzzy choices in an imperfectmarket. Education Policy Analysis Archives, 4(2), 1–15.
Wilson, S. M., Floden, R. E., & Ferrini-Mundy, J. (2002). Teacher prepa-ration research: An insider’s view from the outside. Journal of TeacherEducation, 53, 190–204.
Wilson, S. M., Lubienski, S., & Mattson, S. (1996, April). Where’s the math-ematics?: The competing commitments of professional development. Paperpresented at the annual meeting of the American Educational ResearchAssociation, New York, NY.
Zeichner, L. M. (1993). Reflections on the career-long preparation of teachersin Wisconsin. Madison, WI: University of Wisconsin.
AUTHOR NOTE
James A. Telese is a Professor of Mathematics Educa-tion in the Department of Teaching, Learning and Inno-vation. He leads programs for the professional developmentof K–12 mathematics teachers and conducts program eval-uation. His research interests include teacher professionaldevelopment, learning of algebra and geometry, and themathematics learning of English language learners.
Dow
nloa
ded
by [
Ston
y B
rook
Uni
vers
ity]
at 0
2:18
01
Nov
embe
r 20
14
Recommended
