Student Achievement and School and Teacher Accountability

  • Published on

  • View

  • Download

Embed Size (px)


<ul><li><p>Journal of Personnel Evaluation in Education 12:3 257267, 1998</p><p># 1998 Kluwer Academic Publishers, Boston Manufactured in The Netherlands</p><p>Student Achievement and School andTeacher Accountability</p><p>ROBERT L. MENDRO</p><p>Dallas Public Schools</p><p>Abstract</p><p>This article is part of a set of papers generated from a keynote presentation by Dr. Jack Frymier at the 1997</p><p>CREATE annual meeting. Dr. Frymier dealt with several reasons, that, as he saw it, invalidate the use of student</p><p>achievement data in teacher accountability systems. This article rst notes problems with Dr. Frymier's</p><p>conception of accountability. Next, it summarizes some of the recent evidence showing the strong connection</p><p>between school and teacher effectiveness measures and student achievement. It then notes some of the benets of</p><p>school and teacher effectiveness measures external to their function as measures of performance. Next, policy</p><p>issues arising from the use of student data and the associated research are considered. Finally, it concludes with</p><p>some cautions about using effectiveness measures in teacher accountability systems.</p><p>Dr. Frymier's Conception of Accountability</p><p>Dr. Jack Frymier's paper on ``Accountability and Student Learning'' presents an</p><p>interesting but untenable position. He draws what seem to be preordained conclusions</p><p>regarding accountability that just do not hold in the light of evidence (Frymier, 1997). This</p><p>article rst argues for the link between accountability and student performance in an</p><p>overall response to Dr. Frymier. It then moves to a discussion of the existing data</p><p>regarding tying school and teacher performance to student performance in a value-added</p><p>context. Finally, it concludes with a discussion of issues that come to the forefront in light</p><p>of this research, including some of the many benets that can be derived from value-added</p><p>measures and some cautions with regard to their use in teacher accountability systems.</p><p>Dr. Frymier basically holds that teachers can be held accountable only for their own</p><p>performance and not for the performance of their students. Regardless of the many ways</p><p>he restates his position, it is always a reiteration of this basic theme. There are three</p><p>arguments presented in support of this theme. The rst is the argument of tradition.</p><p>According to his illustrations, the British tried payment according to student outcome in</p><p>the 1800s and rejected it, and, therefore, so should we. Also, the ancient Greeks believed</p><p>individuals were responsible only for their own behavior and this, perforce, excludes</p><p>teachers from being held accountable for any part of their students' behavior. Next, the</p><p>idea of teacher accountability through student achievement is rejected on a supposed</p><p>violation of legal principles. Legally, he claims, even parents cannot be held responsible</p><p>for the behavior of their own children, so teachers cannot be held responsible for their</p><p>students' behavior. Finally, he produces an unusual twist of the concept of locus of control</p></li><li><p>for an argument based on psychology. Holding teachers responsible for student outcomes</p><p>will give students an external locus of control, which will turn them into mindless</p><p>automatons. As he has applied them, all three are false positions. Let us examine them</p><p>each in a little more detail.</p><p>The rst argument, that of tradition, falls apart on analysis. Regardless of who has tried</p><p>it and failedthe British, the ancient Greeks, or anyone elsenew information based on</p><p>solid research can invalidate tradition or, at a minimum, indicate that tradition needs to be</p><p>challenged. As is demonstrated below, such information now exists. Regardless of what</p><p>the Greeks held personal responsibility to be, we are dealing with organizations and</p><p>individuals given the specic task of inuencing the learning of students. Their</p><p>responsibility for doing so and their accountability for the task are based on traditions,</p><p>laws, and information available since the time of the ancient Greeks, thus rendering the</p><p>opinion of the Greeks irrelevant. Of course, there are other problems with these analogies.</p><p>To consider just a few with the British example, their school system was entirely different</p><p>from a modern system, and the status of employees was legally different. The curriculum</p><p>was much less well dened, and there were entirely different ways of measuring the</p><p>outcomes of schooling.</p><p>The Greeks we also consider regarding the legal argument for parental responsibility.</p><p>Here, Dr. Frymier has ignored a number of precedents where parents are being heldresponsible for their children's truancy and a number of other individual behaviors.</p><p>Further, responsibility for the entire range of behavior of an individual is confused with the</p><p>limited responsibility of a teacher for, among other things, the learning, social behavior in</p><p>school, and academic performance of students. For example, from a legal standpoint, the</p><p>Texas legislature (and, no doubt, others) certainly holds the legal position that teachers are</p><p>responsible for their students' academic behavior, since, in 1995, it passed a law that</p><p>mandated the inclusion of the academic performance of a teacher's students as a part of all</p><p>teacher evaluation systems in the state (Senate Bill 95-1, 1995). The ancient Greeks may</p><p>or may not have disagreed with this stance, but since their time, new data regarding</p><p>behavior have come to light. Certainly, knowledge of the ability to manipulate behavior</p><p>through positive and negative reinforcement (Skinner, 1953) and the in loco parentisfunction of teachers and the schools relative to their students behavior and well-being</p><p>affects these conclusions. When these are combined with the teacher's obligation to spend</p><p>a minimum amount of time manipulating academic behavior, the conception of the Greeks</p><p>delivering schools and teachers from responsibility for student achievement collapses.</p><p>Locus of control has been stood on its head in order to absolve teachers of responsibility</p><p>for their students' achievement. Locus of control is a part of attribution theory, which</p><p>posits that individuals perceive (general) achievement-related situations as falling along a</p><p>continuum from uncontrollable to totally controllable. Recent research indicates that the</p><p>teacher can be directly responsible for instilling an internal locus of control in a student</p><p>(U.S. Department of Education, 1992; Freeman &amp; Sokoloff, 1995). Thus, teachers are</p><p>shown to have responsibility for developing locus of control in their students as part of</p><p>enhancing student achievement. Regardless of the distortion of locus of control research, a</p><p>false dichotomy is offered with the concept. Locus of control is situation specic, and it is</p><p>possible for a student to perceive a different locus of control with each task, assignment, or</p><p>258 R.L. MENDRO</p></li><li><p>goal. Thus, locus of control is not the either/or situation argued. Finally, one of the</p><p>demonstrated tenets of the standards-based education movement is that students feel quite</p><p>comfortable with rational, achievable standards as long as they are allowed to have a</p><p>degree of control over how they reach the standard (Cross &amp; Joftus, 1997; Mendro, 1997).</p><p>Thus, it is possible for students to allow a different locus of control for different parts of a</p><p>larger complex task as opposed to the all or nothing alternative presented to the reader.</p><p>Data in Support of Using Student Achievement in School andTeacher Accountability</p><p>The real issue is to determine what data are available that bear on the issue of school and</p><p>teacher accountability. The use of student achievement data as a part of school and teacher</p><p>accountability has been debated for some time, but there have been few practical</p><p>applications free from known biases until the last decade (Millman, 1997). The debate has</p><p>centered around the need for including multiple outcome variables and for controlling</p><p>exogenous inuences. The use of multiple-outcome variables is a two-part problem. The</p><p>rst part centers on the need for multiple-outcome variables that are related to important</p><p>educational goals (see, for example, Murname, 1987; David, 1987). Here goals must be</p><p>decided with input from stakeholders. The second part of the multiple-outcome question</p><p>is mainly a question of resources on the part of the district involved (Webster &amp;</p><p>Schuhmacher, 1973). Resources are needed to maintain the extensive databases and to take</p><p>the multiple measures required.</p><p>The variables used in the school effectiveness model used in the Dallas Public Schools</p><p>are chosen by a group known as the Accountability Task Force. This group consists</p><p>of principals, teachers, parents, community members, business representatives, and</p><p>administrators. The Task Force makes the nal decisions on all variables to be included in</p><p>the School Effectiveness Indices. In addition, the District has performance awards for</p><p>those schools that excel as measured by the School Effectiveness Indices. The Task Force</p><p>sets the rules under which a school will receive an award (for example, the school must test</p><p>at least 95 per cent of their non-special education students) and serves as the nal arbiter</p><p>for all appeals regarding the indices and the awards. The awards serve as one type of</p><p>employee bonuses. Each professional employee in an awarded school receives a $1,000</p><p>bonus. Each support employee receives $500, and the school receives $2,000 for its</p><p>activity fund. The variables currently included in the indices as designated are included in</p><p>table 1. The table includes the weights assigned to the variables as well. The variables</p><p>include primarily test data when weights are considered. For example, in an elementary</p><p>school with grades 1 through 6, test scores have weights totaling 89, attendance totals 6,</p><p>and promotion rate 1. The Accountability Task Force intended these weights to serve as the</p><p>relative value of the outcome measures for these schools. Indeed, where parents and</p><p>community members discuss goals, our experience has been that more emphasis is placed</p><p>on achievement as the primary legitimate outcome of schooling.</p><p>The stumbling block to the analysis of these data has been the known inuence of</p><p>variables outside the control of the school or teacher (Webster &amp; Mendro, 1997). For</p><p>STUDENT ACHIEVEMENT AND SCHOOL AND TEACHER ACCOUNTABILITY 259</p></li><li><p>Tabl</p><p>e1.</p><p>Wei</p><p>gh</p><p>tin</p><p>go</p><p>fC</p><p>rite</p><p>rio</p><p>nM</p><p>easu</p><p>res</p><p>by</p><p>the</p><p>Acc</p><p>ounta</p><p>bil</p><p>ity</p><p>Tas</p><p>kF</p><p>orc</p><p>eof</p><p>the</p><p>Dal</p><p>las</p><p>Publi</p><p>cS</p><p>chools</p><p>.</p><p>Ele</p><p>men</p><p>tary</p><p>Cri</p><p>teri</p><p>aM</p><p>iddl</p><p>eSc</p><p>hool</p><p>Cri</p><p>teri</p><p>aH</p><p>igh</p><p>Scho</p><p>olC</p><p>rite</p><p>ria</p><p>Cri</p><p>teri</p><p>on</p><p>Wei</p><p>gh</p><p>tC</p><p>rite</p><p>rion</p><p>Wei</p><p>ght</p><p>Cri</p><p>teri</p><p>on</p><p>Wei</p><p>ght</p><p>ITB</p><p>S</p><p>Rea</p><p>din</p><p>g4</p><p>/Gra</p><p>de</p><p>ITB</p><p>S</p><p>Rea</p><p>din</p><p>g4/G</p><p>rade</p><p>TA</p><p>P</p><p>Rea</p><p>din</p><p>g,</p><p>Gra</p><p>de</p><p>96</p><p>ITB</p><p>S</p><p>Mat</p><p>hem</p><p>atic</p><p>s4</p><p>/Gra</p><p>de</p><p>ITB</p><p>S</p><p>Mat</p><p>hem</p><p>atic</p><p>s4/G</p><p>rade</p><p>TA</p><p>P</p><p>Mat</p><p>hem</p><p>atic</p><p>s,G</p><p>rade</p><p>96</p><p>TA</p><p>AS</p><p>R</p><p>ead</p><p>ing</p><p>5/G</p><p>rad</p><p>eT</p><p>AA</p><p>S</p><p>Rea</p><p>din</p><p>g5/G</p><p>rade</p><p>TA</p><p>AS</p><p>R</p><p>eadin</p><p>g,</p><p>Gra</p><p>de</p><p>10</p><p>12</p><p>TA</p><p>AS</p><p>M</p><p>ath</p><p>emat</p><p>ics</p><p>4/G</p><p>rad</p><p>eT</p><p>AA</p><p>S</p><p>Mat</p><p>hem</p><p>atic</p><p>s4/G</p><p>rade</p><p>TA</p><p>AS</p><p>M</p><p>athem</p><p>atic</p><p>s,G</p><p>rade</p><p>10</p><p>12</p><p>TA</p><p>AS</p><p>W</p><p>riti</p><p>ng</p><p>,G</p><p>rade</p><p>45</p><p>TA</p><p>AS</p><p>W</p><p>riti</p><p>ng,</p><p>Gra</p><p>de</p><p>85</p><p>TA</p><p>AS</p><p>W</p><p>riti</p><p>ng,</p><p>Gra</p><p>de</p><p>10</p><p>12</p><p>Att</p><p>end</p><p>ance</p><p>1/G</p><p>rad</p><p>eT</p><p>AA</p><p>S</p><p>Sci</p><p>ence</p><p>,G</p><p>rade</p><p>81</p><p>AC</p><p>P</p><p>Lan</p><p>guag</p><p>e8/S</p><p>chool</p><p>Pro</p><p>mo</p><p>tio</p><p>nR</p><p>ate</p><p>1/S</p><p>choo</p><p>lT</p><p>AA</p><p>S</p><p>Soci</p><p>alS</p><p>tudie</p><p>s,G</p><p>rade</p><p>81</p><p>AC</p><p>P</p><p>Mat</p><p>hem</p><p>atic</p><p>s8/S</p><p>chool</p><p>Att</p><p>endan</p><p>ce1/G</p><p>rade</p><p>AC</p><p>P</p><p>Sci</p><p>ence</p><p>8/S</p><p>chool</p><p>Pro</p><p>moti</p><p>on</p><p>rate</p><p>1/S</p><p>chool</p><p>AC</p><p>P</p><p>Soci</p><p>alS</p><p>tudie</p><p>s8/S</p><p>chool</p><p>Dro</p><p>pout</p><p>rate</p><p>1/S</p><p>chool</p><p>AC</p><p>P</p><p>Rea</p><p>din</p><p>gim</p><p>p,</p><p>Gra</p><p>de</p><p>92</p><p>Per</p><p>centa</p><p>ge</p><p>inhonors</p><p>cours</p><p>es2/S</p><p>chool</p><p>AC</p><p>P</p><p>Honors</p><p>/Advan</p><p>ced</p><p>3/S</p><p>chool</p><p>AC</p><p>P</p><p>Fore</p><p>ign</p><p>Lan</p><p>guag</p><p>e2/S</p><p>chool</p><p>SA</p><p>T/A</p><p>CT</p><p>,G</p><p>rade</p><p>12</p><p>4</p><p>PS</p><p>AT</p><p>V</p><p>erbal</p><p>1/S</p><p>chool</p><p>PS</p><p>AT</p><p>M</p><p>athem</p><p>atic</p><p>s1/S</p><p>chool</p><p>Gra</p><p>duat</p><p>ion</p><p>rate</p><p>5/S</p><p>chool</p><p>Per</p><p>centa</p><p>ge</p><p>takin</p><p>gS</p><p>AT</p><p>/AC</p><p>T5/S</p><p>chool</p><p>Per</p><p>centa</p><p>ge</p><p>takin</p><p>gP</p><p>SA</p><p>T3/S</p><p>chool</p><p>Per</p><p>centa</p><p>ge</p><p>inhonors</p><p>cours</p><p>es5/S</p><p>chool</p><p>Per</p><p>centa</p><p>ge</p><p>inA</p><p>Pco</p><p>urs</p><p>es4/S</p><p>chool</p><p>Per</p><p>centa</p><p>ge</p><p>takin</p><p>gA</p><p>Pan</p><p>dpas</p><p>sing</p><p>1/S</p><p>chool</p><p>260 R.L. MENDRO</p></li><li><p>example the Dallas Public Schools system controls for the effects of a combined ethnicity</p><p>and language prociency variable, student gender, four variables measuring socio-</p><p>economic status (free or reduced lunch participation, census family income, census</p><p>poverty level, and census college participation), and the complete interactions of ethnicity</p><p>or language, gender, and free lunch.</p><p>Although regression analysis has been proposed for years as a remedy for controlling</p><p>these inuences (see, for example, Felter &amp; Carlson, 1985; Kirst, 1986), standard multiple</p><p>linear regression has shown problems controlling known inuences in the higher levels of</p><p>multilevel data (Webster &amp; Olson, 1988; Webster, Mendro, Orsak &amp; Weerasinghe, 1998).</p><p>Only with the widespread availability of software and techniques for the analysis of</p><p>sophisticated multilevel models such as hierarchical linear modeling (HLM) (Raudenbush</p><p>&amp; Bryk, 1989) or with the statistical developmental efforts commissioned by at least one</p><p>state Department of Education (Sanders &amp; Horn, 1993) have school personnel had the</p><p>statistical tools available to control inuences on multiple levels of variables and their</p><p>associated bias. For example, at the school level, the Dallas Public Schools model</p><p>simultaneously controls the effects of mobility, crowding, average family income, average</p><p>family educational level, average family poverty level, per cent on free lunch, per cent</p><p>SOL students, per cent African American students, per cent Hispanic students, per cent</p><p>minority students, and per cent of teacher days for vacant positions, while controlling the</p><p>student variables listed above.</p><p>With the initiation of sophisticated models for multilevel data analysis, several carefully</p><p>implemented systems for determining school and teacher effectiveness have been</p><p>developed and implemented (Sanders, Saxton &amp; Horn, 1997; Webster &amp; Mendro, 1997).</p><p>Further, Webster et al. (1998) have laid out clear criteria for the evaluation of the level of</p><p>bias in these models. They note that the nal criterion must be the degree to which these</p><p>models control the correlation of variables known to be related to the outcome variables</p><p>but not under control of the school or teacher (SES, for example) and the effectiveness</p><p>outcomes derived from the models. The degree to which these correlations are nonzero is</p><p>the degree to which a model is biased. Furthermore, these correlations must be controlled</p><p>at both the student and school or classroom level.</p><p>Research by Sanders and Rivers (1996), Jordan, Mendro, and Weerasinghe (1997), and</p><p>Bembry, Jordan, Gomez, Anderson, and Mendro (1998) has demonstrated the effects of</p><p>teachers on student achievement. They show that there are large additive components in</p><p>the longitudinal effects of teachers, that these effects are much larger than expected, and</p><p>that the least effective teachers have a long-term inuence on student achievement that is</p><p>not fully remediated for up to three years later. Finally, both Dallas studies show a</p><p>selection bias where lower-achieving students are more likely to be put with lower</p><p>effectiveness teachers and vice-versa. Thus, the negative effects of less effective teachers</p><p>are being visited on students who probably need the most help.</p><p>Further, the research done in Tennessee by Sanders and Rivers and that done in Dal...</p></li></ul>


View more >