Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
EVALUATING SCHOOL-LEVEL AND STUDENT-LEVEL EFFECTS ON STUDENT
ACHIEVEMENT: EVIDENCE FROM THE EDUCATION LONGITUDINAL STUDY OF
2002
by
JIANG HE
(Under the Direction of Eric Houck)
ABSTRACT
This dissertation is grounded in the empirical and theoretical literature on student
achievement. Using the Education Longitudinal Study of 2002 (ELS:2002), this dissertation
sought to contribute to a better understanding of what factors explain the educational outcomes
in an education production function.
This dissertation aims to examine the relationship between educational expenditure and
student achievement, accounting for student, school, and family characteristics. Another aim of
this dissertation is to take a close look at students and their educational behaviors in relation to
achievement. This research model brings empirical evidence to test the effects of student
motivation and attitudes toward learning.
Findings indicate that student socioeconomic status and expectation and attitude about
learning are the three strongest factors positively related to student achievement. These effects
are significant on both the student and the school levels. Comparatively, school resource, teacher
quality, and school poverty variables do not show significant impact on learning outcomes. An
important finding derived from hierarchical linear modeling is that low performing students may
benefit greatly when they are in schools where expectation and attitude about learning are high.
These effects are typically strong for Black and Hispanic students.
INDEX WORDS: Student Achievement, Educational Expenditure, Student Motivation,
Education Production Function, Hierarchical Linear Modeling
EVALUATING SCHOOL-LEVEL AND STUDENT-LEVEL EFFECTS ON STUDENT
ACHIEVEMENT: EVIDENCE FROM THE EDUCATION LONGITUDINAL STUDY OF
2002
by
JIANG HE
B.S., Peking University, China, 2003
M.P.A., University of North Carolina at Pembroke, 2004
A Dissertation Submitted to the Graduate Faculty of The University of Georgia in Partial
Fulfillment of the Requirements for the Degree
DOCTOR OF PHILOSOPHY
ATHENS, GEORGIA
2009
© 2009
Jiang He
All Rights Reserved
EVALUATING SCHOOL-LEVEL AND STUDENT-LEVEL EFFECTS ON STUDENT
ACHIEVEMENT: EVIDENCE FROM THE EDUCATION LONGITUDINAL STUDY OF
2002
by
JIANG HE
Major Professor: Eric Houck
Committee: Sally Zepeda Catherine Sielke
Electronic Version Approved:
Maureen Grasso Dean of the Graduate School The University of Georgia August 2009
iv
ACKNOWLEDGEMENTS
I wish to extend my deep gratitude to my major professor, Dr. Eric Houck. His constant
support made this dissertation possible. I also want to thank Dr. Catherine Sielke, who critiqued
my thinking and writing while encouraging me to press on. I am thankful for Dr. Sally Zepeda’s
patience, unfailing support, and tremendous editing efforts. Her investment further helped to
improve the research.
This dissertation would never have been completed without the assistance from the
Lifelong Education, Administration, and Policy Department for providing secure office for my
data. I am also indebted to the National Center for Education Statistics for allowing me to use the
Education Longitudinal Study: 2002 to conduct my research.
I want to thank my mother and my mother-in-law for their unselfish help whenever we
needed the most. Their child support was a huge relieve on us so that we can focus on our
studies. A special thanks to my wife, Dr. Lei Cheng, who always stands by me from the
beginning to the end of my doctoral journal.
v
TABLE OF CONTENTS
Page
ACKNOWLEDGEMENTS ........................................................................................................... iv
LIST OF FIGURES ...................................................................................................................... vii
LIST OF TABLES ....................................................................................................................... viii
CHAPTER
1 INTRODUCTION .......................................................................................................... 1
Statement of the Problem ......................................................................................... 1
Definition of Key Terms .......................................................................................... 3
Conceptual Framework ............................................................................................ 4
Purposes of the Study ............................................................................................... 5
Research Questions .................................................................................................. 7
Methods .................................................................................................................... 8
Significance of the Study ....................................................................................... 10
2 REVIEW OF THE LITERATURE .............................................................................. 12
Overview ................................................................................................................ 12
Education Production Function .............................................................................. 12
Issues and Future Directions of Education Production Function ........................... 28
Student Learning Motivation Theories ................................................................... 32
Empirical Findings on Student Achievement.......................................................... 44
Other Factors Affecting Student Motivation ........................................................... 48
vi
3 DATA AND METHODS.................................................................................................. 55
Overview .................................................................................................................... 55
Data ............................................................................................................................ 55
Hierarchical Linear Modeling .................................................................................... 60
Methods ....................................................................................................................... 63
4 FINDINGS OF THE STUDY.......................................................................................... 68
Overview .................................................................................................................... 68
Data Description ......................................................................................................... 68
Factor Analysis ........................................................................................................... 70
Treating Missing Values ............................................................................................ 75
Variables..................................................................................................................... 81
Hierarchical Linear Modeling .................................................................................... 84
Summary of Findings ............................................................................................... 112
5 CONCLUSION............................................................................................................... 119
Review of Findings .................................................................................................. 119
Research Implications .............................................................................................. 124
Policy Implications ................................................................................................... 126
Data Limitations ....................................................................................................... 127
Future Steps .............................................................................................................. 128
vii
LIST OF FIGURES
Page
Figure 1. Conceptual Framework of Education Production Function ............................................ 5
Figure 2. Math and reading scores vs. Black plot with focus of mean expectation ...................... 91
Figure 3. NAEP scores vs. Hispanic Student plot with focus of mean expectation ..................... 98
Figure 4. Range of plausible values for math and reading test scores ........................................ 113
Figure 5. Range of plausible values for NAEP test scores ......................................................... 114
Figure 6. Range of plausible values for SAT scores ................................................................... 114
Figure 7. Range of plausible values for academic GPA ............................................................. 114
Figure 8. Percent of Variance in Achievement at Student and School Levels by Four Measures
..................................................................................................................................................... 117
viii
LIST OF TABLES
Page
Table 1. Research Findings on Educational Expenditure and Achievement ................................ 18
Table 2. Research Findings on Class Size and Achievement ....................................................... 24
Table 3. Selected Variable Description ........................................................................................ 58
Table 4. Summary of Variables and Factor Loadings matrix of FEXP ........................................ 72
Table 5. Summary of Variables and Factor Loadings matrix of FMOV ...................................... 73
Table 6. Summary of Variables and Factor Loadings matrix of FATD ....................................... 74
Table 7. Summary of Variables and Factor Loadings matrix of FPAR ........................................ 74
Table 8. Results of Bartlett's and Kaiser-Meyer-Olkin Tests ....................................................... 75
Table 9. Case Numbers of Factors ................................................................................................ 75
Table 10. Factor Analysis Missing Values Check (FEXP) ........................................................... 76
Table 11. Factor Analysis Missing Values Check (FMOV) ......................................................... 76
Table 12. Factor Analysis Missing Values Check (FATD) .......................................................... 77
Table 13. Factor Analysis Missing Values Check (FPAR) .......................................................... 77
Table 14. ANOVA test of Factors with and without Missing Values .......................................... 79
Table 15. Means, Standard Deviations, and Descriptions of Student-Level Variables ................ 82
Table 16. Means, Standard Deviations, and Descriptions of School-Level Variables ................. 83
Table 17. Results for Four Model (Standard Coefficient) on Student Math, Reading Test Score 92
Table 18. Results for Four Model Estimates (Standard Coefficient) on Student NAEP scores ... 99
Table 19. Results for Four Model Estimates (Standard Coefficient) on Student SAT Score ..... 105
Table 20. Results for Four Model Estimates (Standard Coefficient) on Academic GPA .......... 111
Table 21. Two-Level HLM Estimates (Standard Coefficient) over Four Outcome Measures ... 116
1
CHAPTER 1
INTRODUCTION
Statement of the Problem
The relationship between educational expenditure through tax dollars and student
academic achievement is not a new topic, and the results so far are mixed (Goldschmidt &
Eyermann, 1999). Theoretically, increasing educational expenditure should improve school
conditions, teacher salaries, student/teacher ratios, and educational facilities; thus, granting
students a better educational opportunity. Coleman et al. (1966) first adopted education
production functions to explain the relation between expenditure and achievement. They found
that educational resources have very little impact on student achievement in general, except for
the pupil/teacher ratios. Rather, family characteristics had a much greater influence on student
achievement. Hanushek (1989, 1994, 1996, 2003), through a serious of individual studies and
meta-analyses, concluded that public spending only slightly improved student performance.
Although this point-of-view faced challenges from other scholars (Hedges, Laine, & Greenwald,
1994; Hedges & Greenwald, 1996; Greenwald, Hedges, & Laine, 1996a, 1996b; Goldschmidt &
Eyermann, 1999), the field of education finance gained increased attention from both educational
researchers and practitioners (Rolle & Houck, 2004).
Further evidence has emerged from studies using state and local data which examines the
role of educational expenditures on student academic achievement (Marlow, 1999; Tobier,
2001). Education finance studies often lead to a unique situation in that the effects of money are
not quite the same on different demographic groups even within one particular region
(Rumberger & Palardy, 2005). However, blurred as the picture appears to be, expenditures in
general have not yet made an impressive impact, which drives scholars to ponder whether things
2
are left out of the theoretical framework or whether money falls to the right spot, such as teacher
salary or school facilities.
The dissertation develops a new model accounting for student behaviors on learning (i.e.,
motivation, attitude, and expectation). Several researchers have laid the ground work of how
motivation toward learning works for students. Lareau (2003) and her research team observed
different patterns from the children of various social classes and racial groups. The key finding
was that self-entitlement, such as the pursuits of individual preference, helped student learn
better. Sirin’s meta-analysis (2005) implied that, learning motivation of students influenced by
parental education level and family structure, leads to high academic achievement; however, the
relationship was not very strong. Rothstein (2000) argued that students were affected by the
educational institution, which was eventually affected by students themselves. This loop
emphasized the importance of peer effect and how students perceived themselves and each other.
In explaining how motivation relates with achievement from students of different racial
background, a similar idea raised by Ogbu (2004) suggested that the lack of achievement from
Black students was caused by the burden of “acting White.” Black students “lacked the
motivation to study hard in order not to be viewed by their peers” as “acting White.” Mickelson
(1990) also observed the education achievement disparities between White students and Black
students controlling for other factors. The survey findings suggested that Black students had a
dual-value system. What every student in general believed was that education was a good way to
a better life and it should provide a vital opportunity to everyone for success. But this “abstract
attitude” or “external attitude” conflicted with the real world obstacles that Black students face
and truly believe, namely the “concrete attitude” or “internal attitude.” Further, Mickelson
3
(1990) found that the concrete attitude contributed more than the abstract attitude with regard to
student academic achievement.
Few have investigated how educational resources affect student groups characterized by
attitudes toward learning. Controlling for race, gender, and other demographic information, the
author believes that student attitude may endogenously affect academic achievement more than
other exogenous factors such as teacher quality, class size, and school characteristics. This is
important, in particular, because the education psychology literature has proved that student
motivation and other behavioral factors significantly affect student achievement (Duncan,
Featherman, & Duncan, 1972; Hanson, 1994). Incorporating these motivational factors into the
education production function and testing it will lead to the answer of the classic question: what
matters the most to student achievement? The author hopes this dissertation can help to develop
a new line of inquiry in the field of school finance indicating how educational resources make
the most sense to the most students according to their learning attitudes.
Definition of Key Terms
The following terms are used in this dissertation. This section provides the explanations
of these terms.
Education Production Function: A mathematical construct that models the relationship between a
given set of educational resource inputs and a set of educational outputs.
Input Variables: Variables or factors that are potentially impacting education process that
ultimately impacting educational outputs.
Output Variables: Variables or measurements that are the outcome of the education process.
Teacher Quality: A term capturing teachers’ characteristics based primarily on teacher education
and experience (Grubb, 2008).
4
Student/teacher Ratio: The number of students divided by the number of certified FTE staff in
one educational unit.
Student Achievement: High school standardized tests, NAEP equalized tests, high school
academic GPA, SAT scores.
Conceptual Framework
The conceptual model guiding this study draws from literature on education production
functions. The input measurements of the function are on the left side of the Figure 1. On the
right side, there are output variables such as high school grade point average (GPA), Scholastic
Aptitude Test (SAT), and math and reading test score.
In this model, student-level independent variables such as gender, race, and
socioeconomic status (SES) will be examined. School-level characteristics such as teacher
quality, student/teacher ratio, school type, and percent minority will also be examined. Student
attitude and motivational factors will be created through factor analysis on a number of
independent variables taken from Education Longitudinal Study of 2002 (ELS:2002) dataset.
These factors will be applied to both student-level and school-level in the hierarchical linear
modeling analysis.
Figure 1. Conceptual Framework of Education Production Function
Education, with its complex structure and high stakes, is always
researchers. The U.S. public education system has drawn more and more attention from
educators, school administrators, and politicians over the last
that education generally prepares human capital in various
(Becker, 1992). Abundant education
advantages for the potential employees
5
. Conceptual Framework of Education Production Function
Purposes of the Study
Education, with its complex structure and high stakes, is always attractive
researchers. The U.S. public education system has drawn more and more attention from
educators, school administrators, and politicians over the last few decades. First, it is recognized
that education generally prepares human capital in various sectors of the macro economy
educational experience and an advanced degree generate comparative
potential employees entering the labor market. According to the
attractive to policy
researchers. The U.S. public education system has drawn more and more attention from
, it is recognized
macro economy
advanced degree generate comparative
ccording to the Digest of
6
Education Statistics, men with bachelor degrees earn $24,230 more annually than men with high
school degrees; women with college degrees earn $18,530 more annually than women with high
school degrees; and the earning gaps have increased steadily since 1990 (Snyder, Dillow, &
Hoffman, 2009). Therefore, it is important to understand theoretically and empirically what the
gaps represent in the education process, especially in the early stages of education.
Secondly, public education is funded largely by tax revenues at state, local, and federal
levels. Many taxpayers, if not all, expect that their money is spent effectively on education. Thus,
not only students and parents, but also the general public, has reason to care about public
education. It is in the interest of the majority to make sure that students learn from certain
educational programs and interventions funded by taxpayers. This dissertation, in its unique way,
examines a national sample of students, teachers, parents, and school administrators to identify
what factors work best to improve student achievement, and under what condition(s), either
institutional or individual, that affects these factors.
The current focus of educational reform is on school accountability and student
achievement as the No Child Left Behind ACT1 progresses along. It is reasonable to believe that
external incentive or sanction stimulates teachers to teach close to the tested curriculum and that
students learn about the materials that are being tested. One issue with this type of schooling is
that students tend to ignore other useful elements of learning such as social cohesion and other
progressive perspectives of education. On the other hand, one alternative to this external reform
model is internal motivation of student learning. The hypotheses being tested in this dissertation
include three student behavioral constructs, specifically, locus of control (Rotter, 1966),
motivation (Pintrich & Schunk, 2002), and self-concept and self-esteem (Rosenberg, 1986).
1 ED.gov at http://www.ed.gov/policy/elsec/guid/states/index.html, accessed on 2009/03/26
7
Theoretically, these variables all show positive correlations with student achievement. However,
very few studies have included all three constructs and fewer are in the realm of education
production function. This dissertation adds to this research area by using sophisticated research
methods and examining large-scale dataset in the analysis.
A recent development in the field of education finance is that educational researchers
noticed a primary objection with education production function, which is a single variable or few
variables research design (Carpenter, 2000; Ilon & Normore, 2006; Odden, Borman, &
Fermanich, 2004). Contemporary education research tends to study student achievement based
on student, classroom, or school variables in isolation from other important factors. Such
research from the field of education finance generates critical findings. For example, Lee and
Smith (1997) investigated the NELS:88 dataset using Hierarchical Linear Modeling (HLM)
methods to find that students achieve the maximum test score gains at schools enrolling between
600 and 900 students. Glass and Smith (1979) undertook an extensive meta-analysis of class-size
research and identified non-linear effect of class size of 20 students is the tipping point for this
class size effect. Coleman et al. (1966) concluded that family characteristics of students are most
influential to student achievement. Most of the existing research paid attention to control
variables to identify the key variables that have effects on student learning; however, research
rarely covered the whole picture of educational processes. This dissertation advances research
through multiple level analysis that takes consideration of individual student factors and teacher
and school factors. Moreover, each level of analysis consists of a host of variables and factors
generated from factor analysis.
Research Questions
Specifically, this dissertation attempts to address the following research questions:
8
1. In the realm of education production function, what factors affect student academic
achievement the most?
2. Specifically, how do the motivation and attitude affect student academic
achievement?
3. Do these factors have the same effect at the student level and the school level?
4. What portion of variance do factors from each of the two level of analysis explain?
Methods
The analysis was built on data from the ELS:2002. To obtain key school resource
variables, this dissertation acquired the restricted data license which contained certain variables
that are linked from the F-33 Common Core of Data to the ELS dataset. Since ELS:2002
includes both the individual student information, and aggregate level information from teachers
and schools, this type of dataset was ideal for multi-level modeling.
ELS:2002 is on-going, and is also the fourth longitudinal study conducted by the NCES
Longitudinal Studies Program. The first three waves of surveys were finished by 2006. Base-
year data collection for the study was in 2002, with approximately 20,000 10th grade students
selected from 750 public and private high schools. Policy issues to be studied through ELS:2002
include the identification of school attributes associated with achievement; the influence of
parent and community involvement on student achievement; and the transition of different racial-
ethnic, gender, and socioeconomic groups from high school to postsecondary institutions and the
labor market (Bozick & Lauff, 2007).
Math and reading test score composites from the base year study is included in this
dissertation analysis, specifically, high school standardized tests. Each of the next two follow-up
studies had special contributions as well, especially for the student achievement measurement.
9
The first ELS follow-up study was conducted in 2004, when high school transcripts were
collected. The second follow-up study, which was administrated in 2006, contains college
enrollment information of the same cohort in 2002. The ELS:2002 datasets allow a number of
input and output measures which are used in the education production functions for this
dissertation. Coming from the two follow-up studies, high school GPA, dropout information,
SAT scores, ACT scores, and college enrolment information are also used as output
measurements.
Compared with other datasets, ELS:2002 had apparent advantages for this dissertation.
First, ELS:2002 is relatively new to the research arena, in both the time of data release, and the
infrequent usage. A search through academic databases revealed that over 10,000 publications
were written from previous NCES studies. For example, the NELS series, ELS:2002, is seldom
seen in the academic fields. Secondly, as the NCES longitudinal studies get more sophisticated in
data collection, many of the sampling issues were taken into consideration when the datasets are
released to researchers. Thirdly, the large-scale dataset yields tremendous statistical power to the
analysis.
The common trend of existing research holds educational expenditure variables as
exogenous factors that exert influences from outside of the student learning process. However,
synthesizing literature from education theories of public spending, this study sought to add
critical endogenous factors such as student attitude toward class and school, and self-esteem
effects into the education production function.
These factors exert influences from inside of the student learning process. Adopting
appropriate statistical techniques with the ELS dataset, the author believes that the empirical
results should support the hypotheses this dissertation raises. The new research direction is to
10
add the peer effect, student attitude factors, as well as the public expenditures into the traditional
model to predict student academic achievement. With feasible data, the new model will illustrate
how the educational expenditure has the desirable effect on student academic achievement.
The dependent variables are standardized test composite scores in reading and math,
NAEP equalized test scores, student SAT scores, and student academic GPA. These variables
also take into consideration the weight variables representing the national sample. The
explanatory variables include gender, race, socioeconomic status composites, attitudes about
oneself, future educational expectation, teacher experience, teacher credentials, school level
spending, poverty, school size, school type. The motivation and attitude variables are reduced to
a small number of factors by factor analysis.
Based on the nested nature of the education process, where students are grouped in
classrooms, and classrooms are grouped in schools, HLM analysis is most appropriate. This
research will run the HLM analysis on student and school level.
Significance of the Study
The essential contribution of this dissertation to policy makers is to refresh the focus of
educational expenditures. This research may indicate new directions for effective school
spending that improves student academic achievement. In addition, the new theoretical
framework sorts out other influencing factors related to educational investment. Furthermore, the
application of HLM onto the ELS:2002 dataset examines the issue from both the individual and
institutional perspectives. The large dataset and sophisticated quantitative research are among the
new directions for school finance (Plecki, 2006). This will provide scientific based evidence to
back the funding decisions for specific education spending and could be used to guide either
greater investment, or less investment, in some particular students' education. Specifically, this
11
dissertation points out different ways it could or would change people's perceptions about
education, since motivation is itself a factor that many see as beyond the immediate reach of
public policy.
Another implication from this research is that it helps to evaluate existing educational
policies. For example, in coping with the No Child Left Behind (NCLB) Act, many local
education authorities develop strategies to improve student academic achievement. Some of
these may involve extra spending that may be futile to the current system. Combined with both
attitude and monetary concerns, this research should inform policy makers about what
components a good policy should include.
In general, school curricula do not necessarily mandate teachers to motivate students to
learn, although many good teachers do so anyway. One of the implications for policy makers is
that educational expenditure could be spent on encouraging teachers to help students build a
positive attitude. Policy makers should take the results from this research to prioritize the usage
of educational expenditure. The public education system is all about making choices with limited
resources. Scientific based strategies are set to start policy-makers to observe from multiple
angles to the matter of education. Improving student attitudes, on the one hand, may improve
student achievement. On the other hand, it may accomplish the same end result at a lower cost,
compared with traditional spending patterns.
Building on this theoretical framework, the author believes that the fundamental question
lies within the social value of education. If the stakes for education are high enough, students and
parents should have higher motivation to seek better education opportunities and therefore push
for higher educational standards. The motivation to learn is a reflection of social value of
education that is definitely worth investing in the long run.
12
CHAPTER 2
REVIEW OF THE LITERATURE
Overview
This chapter reviews prior literature on various aspects of the education process related to
student achievement. First, this chapter introduces education production functions and the
applications over time. The next section summarizes specific issues that emerged when using
these functions. This chapter then turns to the work of student motivation theories before
synthesizing findings from prior studies. The last section discusses other important factors
affecting student motivation which also potentially affect student achievement.
Education Production Function
This section summarizes the field of education finance with a concentration on student
achievement. The focus of this literature is the education production function and its various
applications. The first section defines and describes education production function. The
following sections break down the literature by the key research area and common variables used
to conduct these studies.
Education is a complicated process because there are many people involved – students,
parents, teachers, school administrators, and educational policy makers. Each part of the process
has a certain amount of input to the outcome, which usually is measured in terms of student
academic achievement. Among all the inputs, there is money that pays for the education process
and covers the expenses from school where education of all forms occurs. Needless to say,
money is important, in both the amount and the way it is spent. Many educators and policy
makers study the dynamic of how to make the best use of every dollar on education, and others
13
focus on how the amount of money is distributed among different groups of students (Rolle &
Houck, 2004).
At school, students learn from teachers and interact with peers, but after school, students
spend a significant amount of time with their family members. Therefore, parents and family
characteristics can certainly shape students’ attitude toward the learning process. Many
researchers have been investigating the impact from various factors such as students’ family
characteristics, parents’ education background, family socioeconomic status (SES), school class
characteristics, and school characteristics on students’ academic achievements (Coleman et al.,
1966). Most of the outcome measures are characterized by test sores.
Such research studies are broadly viewed as education production function which
contains the input factors to predict the outcomes of education. The most often seen research
methods are regression analyses of various patterns. There is a long lasting investigation of the
relationship between educational inputs and output as illustrated in equation 2-1.
Equation 2-1 O → F (S, X, C) + V
O represents outputs of various forms. S, X, and C are inputs of school characteristics,
student characteristics, and community characteristics. Since equations are only proxies to the
real world, V captures other information as well as errors.
The outputs from education production function are student achievement. Test scores are
the commonly used measurement from many existing studies. Other researchers use graduation
rate, dropout rate, percentage of students going to college, or the passage of one particular exam
as output measurements (Rowan-Kenyon, 2007; Rumberger, 1995). Some studies targeting the
long term effects of education use average salaries in the workforce as outcome of prior
education. There are also abstract variables used to capture student achievement, such as the
14
cognitive ability measurement that is not always in the form of a test. Comparatively, the inputs
can be a series of different measures. The general categories for inputs in education production
function are student inputs, teacher inputs, and school inputs.
Student input variables may include innate student ability (intellectual quotient, academic
attitude, etc.), demographic information (race, gender, etc.), parental background (family living
patterns, parental education level, etc.), and the socioeconomic status (family income,
neighborhood information, etc.) of the student. Teacher input variables may include teachers’
degree and education, years of experience, teachers’ licensures, teachers’ professional
development, teachers’ classroom practices, and teachers’ salaries. School input variables may
include classroom size, school size, school type (public, private, catholic, or charter), school
leadership status, school expenditure, geographic location of the school, and wealth of the school
district. The teacher and school input variables can be aggregated to classroom or school level in
a multi-level environment.
Other important input variables that are directly related with student, teacher, and school
are also commonly seen in the education production function. Based on the specific research
agenda, these variables may be federal expenditure on education (such as Title-I funding), state
and local investment, or one particular policy implementation (such as the NCLB Act). No
matter what variables are in place, people seldom include all the input and output measurements
due to availability of information and time and resource considerations. The variables that are
directly related with the research questions are the explanatory variables, and the ones that are
important but not being studied directly are kept in the education production function as control
variables.
15
Educational Expenditure
Various scholars had studied the relationship between public education expenditure
through tax dollars and student academic achievement (Coleman et al., 1966; Hanushek, 1989,
1994, 1996, & Hedges & Greenwald, 1996). The results were mixed. Theoretically, increasing
education expenditure should improve the school condition, teacher salary, teacher/student ratio,
and educational facilities. Thus, the students can have a better environment to study. Coleman et
al. (1966) first used educational production function to explain the relationship between
expenditure and achievement. After surveying students from grades 1, 3, 6, 9, and 12, they
concluded that the resources had very little impact on student achievement in general, except that
the student-teacher ratio had negative significant effect.
Hanushek (1989) had long studied this issue and found little evidence to support public
spending enhanced student achievement. Based on Hanushek’s research, Hedges et al. (1994)
conducted a meta-analysis of education production function model and found positive
relationships between educational resources inputs and student outcomes, which started this
academic debate. Hanushek (1994) soon wrote a reply mentioning different viewpoints of his,
and he also pointed out some of the statistical errors that Hedges et al. (1994) had made. The
disagreement included that the quantity of educational expenditure did not matter much unless it
was spent efficiently. Hedges and Greenwald (1996) claimed that the vote-counting technique
used in Hanushek’s meta-analysis was not appropriate and their combined significance tests were
better.
Greenwald et al. (1996a) conducted a meta-analysis on the effect of school resources on
student achievement. They found a broad range of resources that are positively related to student
outcomes such as per-pupil expenditure, teacher ability, teacher salary, teacher/student ratio, and
16
class size. Hanushek (1996) published his article toward the same issue and directly pointed out
several statistical misinterpretations. He confirmed his previous finding that lack of resources is
not the largest problem facing schools and that more fundamental reforms are needed in schools.
Greenwald et al. (1996b) extended the debate with a rejoinder confirming the accuracy of their
former research.
In more recent years, Hanushek (2003) reviewed the U.S. and international evidence on
the effectiveness of educational policies and found that increased educational expenditures, and
that student achievement did not show significant improvement. In addition, he also mentioned
that the variation in teacher quality caused the variation in student performance. The common
problem with Hanushek’s work and his debaters are that aggregate measures for student
achievement and public expenditure seem plausible because the amount of unobserved
heterogeneity is hard to control (Goldschmidt & Eyermann, 1999).
On a separate level, Tobier (2001) conducted analysis on New York City’s public schools
and found that students are still underperforming despite the increased funds and personnel. The
author did point out the problem could be the way resources are spent, in particular, teacher
training should be a key factor in improving student achievement. By applying Oklahoma data
into the education production function, Jacques and Brorsen (2002) found that different
expenditure patterns have different impact on student test scores, but the most effective approach
is to target the spending toward classroom instruction.
Analyzing district level and student level data, Ferguson and Ladd (1996) investigated
Alabama schools using education production function and found that instructional spending had
a large effect on test scores. They also concluded the effect is non-linear, in that the most
17
significant effect accrued to districts where per-pupil instructional spending is the lowest, and no
effect to districts where instructional spending is above the median.
Other scholars have found that money hardly makes any difference or even makes the
matter worse. Marlow (1999) studied spending on public schools in California about school
structure and public education quality. He also found that the public expenditure had negative
impacts on student achievement when the expenditure was defined as spending per pupil or
personnel salaries. He also pointed out that school size matters because it costs more for smaller
schools to raise student achievement than that of bigger schools.
Applying education production function with eight large Texas school districts, Clark
(1998) found that teacher salaries are the biggest factor for resource difference. However, the
strongest predictor for student achievement is not directly related with expenditure, but the non-
monetary variables such as student SES that are directly related with the expenditure. This tells
us that only looking at money spent in the educational institutions may not be sufficient. At least
we need to look for better measurement of educational expenditure.
Overall, the findings of educational expenditure research is disunity (See Table 1).
Researchers have different opinions on which variables to use to capture the educational
expenditure information. Meta-analysis showed discrepancy, mainly caused by different research
methods. One theme coming out of the education production function that is close to an
agreement, though, is the strong impact of student family characteristic.
18
Table 1
Research Findings on Educational Expenditure and Achievement
Author Data Method Variables Used Funding
Coleman et al. (1966)
Students of 1,3,6,9, and 12 grades were surveyed and tested for achievements
Education Production Function, Regression
Student/teacher ratio, SES, school resources
Resources made little difference to student achievement
Hanushek (1989)
38 studies with 187 estimates of relationships between resources and achievement
Vote counting Per pupil expenditure, teacher experience, teacher education, teacher salary, teacher student ratio, administrative input, and facilities
No strong, or systematic relationship between educational expenditure and academic achievement
Hedges, Laine, & Greenwald (1994a)
Same as Hanushek (1989)
Meta-Analysis, Inverse chi-square test
Same as Hanushek (1989)
School spending and achievement were associated with one another
Greenwald, Hedges, & Laine (1996a)
Updated data from Hanushek’s (1989) research
Meta-Analysis, Combined significant testing and effect magnitude estimation
Per pupil expenditure, teacher experience, teacher education, teacher salary, teacher student ratio, administrative input, and facilities
A broad range of resources were positively related with student outcomes
Ferguson & Ladd (1996)
Alabama schools Education Product Function, Regression
Instructional spending Instructional spending had a larger but non-linear effect on test scores
Marlow (1999)
California public schools
Education Product Function, Regression
Spending on school infrastructure, school enrollment
Spending defined as spending per pupil or personnel income had negative impacts
Tobier (2001)
New York City’s public schools
Education production function, Regression
Funding, personnel, and teacher’s training
teacher’s training is a key factor in improving achievement
Jacques & Brorsen
Oklahoma School Districts
Education production
Expenditures on instruction,
Spending toward classroom
19
(2002) function, Regression
instructional support, administration, and facilities
instruction on student is most effective.
Student Factors
Existing research shows that the socio-economic status (SES) is an important factor to
predict student achievement. Research finds that students from low-SES background typically
underachieve compared to students from high-SES backgrounds (Nye & Hedges, 2001; White,
1982). Coleman (1996) argued that SES was the single dominant factor affecting student
learning. Forty years later, research still holds that production function for education needs more
variables for student learning dynamics such as SES and the interaction between home and
school (Stiefel, 2006).
Lareau (2003) and her research team observed different patterns of children from
different social classes. Regardless of race, rich children have a sense of entitlement which
means that they have the rights to pursue individual preference such as asking questions in class,
and sharing information with others. Children from poor families have shown pattern of sense of
constraint in the interactions with others in an institutional setting. These children are reluctant to
try new things and to ask questions. Lareau also concludes that the children with the sense of
entitlement are more individualist than the ones with the sense of constraint.
Sirin (2005) reviewed literatures published between 1990 and 2000 on SES and academic
achievement. At first the author introduces three new trends of the research direction. Firstly, the
range of the SES indicators has expanded. Specific factors such as mother’s education
background and family structure are more likely to be included in the measurements. Secondly, a
greater percentage of children are living with parents who are better educated with strong family
structures over the past 20 years. This might be because of the entire education level of the
20
nation has increased. Thirdly, many moderate factors such as race/ethnicity, neighborhood
characteristics, student grade level, etc. have been included into the model in recent years.
Needless to say, the more variables in the model, the more accurate the model is supposed to be.
But the many factors have moderated the magnitude of the relationship between SES and
academic achievement as Sirin (2005) concludes in his meta-study.
Analyzing the Illinois School Report Card, Sutton and Soderstrom (1999) discovered that
low income and percentage minority are the two most significant predictors for student test
scores with the same direction, while average class size, elementary pupil-teacher ratio, teacher
salary, teacher experience, and expenditure per pupil have weak relationships with the dependent
variable. These results are similar to Coleman et al. study (1966).
Similarly, Okpala, Okpala, and Smith (2001) applied the education production function to
North Carolina data and revealed that poverty and socioeconomic status are good predictors for
mathematics scores of elementary students. The greater the percentage of free and reduced-price
lunch students in schools, the lower the student test scores are. There is an inverse relationship
between socioeconomic status and student achievement. The study further finds that there is no
significant correlation between the instructional supplies expenditures and the test scores.
Race is another widely studied factor with education production function. There is a
commonly observed gap between the White students and the minority students in terms of
academic achievement. This pattern has been reflected from a number of studies using national
represented samples (Coleman et al., 1966; Campbell, Hombo, & Mazzeo, 2000; Jencks &
Philips 1980).
Gender difference is one unique aspect in academic achievement. Research finds that
girls tend to have higher educational attainment than boys in general (Lee et al., 2008); however,
21
boys tend to have better practical abilities than girls and have better scores in mathematics and
science subjects (Hedges & Nowell, 1995). Other research shows that male students have more
apparent advantages over their female counterparts to exceed in class as grade levels increased
(Ethington, 1992; Linn & Hyde, 1989).
In summary, student level factors have significant influences on student achievement.
Socioeconomic status and parental education background are among the strongest variables in
education production functions. White students in general perform better than minority students.
Gender is another strong predictor of student achievement; however, the relationship seems to be
non-linear.
Class Size
Education takes place in classrooms. Students learn from the teacher as well as from their
fellow students within the classrooms. The peer effect on student learning is reflected by class
size. Class size not only determines how students are learning and interacting with each other but
also how teachers are interacting with students. Teacher can spend more time on each student if
the class size is small and less time per student if the class has many students.
Glass and Smith (1979) undertook an extensive meta-analysis of class-size research and
sorted from more than 100 articles on class size. The findings did confirm that small class size is
beneficial for students. The fewer students in calss (e.g. 20), students do better. But after 20
students, the difference between a class of 20 and a class of 40 does not very significantly impact
student achievement. This non-linear effect also implies 20 students are the critical point for
class size effect.
Hanushek (1986) reviewed previous studies using education production function to study
the effects of class size on student achievement, and he found that out of the 112 estimates, only
22
23 had statistically significant relationships with student outcomes, and only 9 of them showed
positive impacts. But most of the measurements were using pupil/teacher ratio instead of real
class size. There is a sharp distinction between the two terms. According to Finn (2002), “The
‘pupil/teacher ratio’ is the ratio of the number of students in an educational unit to the number of
full-time-equivalent professionals assigned to that unit” (p. 557). Therefore ,the class size which
is the average number of students per classroom is a better measure, and Hanushek’s conclusions
can be challenged.
Elliot (1998) took a similar approach using HLM and investigated with NELS:88 on the
direct effects of school expenditures on math and science achievement. The findings suggest that
unlike in the direct uses of simply hiring more teachers and spending on capital investments,
resources are most effective in promoting student achievement when they are targeted to
encourage teachers to use effective teaching strategies which emphasize higher order thinking
and inquiry skills.
Using Hierarchical Linear Modeling (HLM), Nye and Hedges (2001) studied class size
effects on student learning with the Tennessee Project STAR randomized experiment and found
positive effects on reading and mathematics achievement across all grade levels. They also
concluded that there are cumulative effects on academic achievement for small classes, which
indicates the earlier small class setting will benefit students over the course of early education
process.
Applying similar research methods, Nye and Hedges (2004) analyzed the Tennessee
Project STAR again and found four years of positive lasting benefit on minority students in small
classes in reading. However, there were five years of negative lasting benefit for girls in small
classes in mathematics. Based on the Tennessee two year longitudinal dataset, Finn and Achilles
23
(1990) also found that small class size yields lasting advantages for minority and for
economically disadvantaged students. These findings indicate that class size can have
differentiated effects among various student populations within the classroom. Class size effects
may also vary by different subjects.
In reviewing the studies on class size, especially the ones with the Tennessee Project
STAR, Hanushek (1999) confirmed the class size effect should be non-linear at large and the
benefit does not appear until the class size decreases to a certain point. The implication of his
finding led to the discussion of the difference that teacher quality makes on student learning. And
policy makers should not just focus on reducing class size but also pay attention to the teacher
and the instruction quality. Rice (1999) confirmed this implication and based on the data from
National Education Longitudinal Study: 1988 (NELS:88), her findings suggested that class size
is positively correlated with the use of class time on non-instructional tasks such as maintaining
class order.
More recently, Borland, Howsen, and Trawick (2005) found evidence to confirm the non-
linear relationship between class size and student achievement based on Kentucky schools. This
study further concluded that the optimal class size is different based on subject areas, therefore
the impact of one class size can be positive for one subject and negative for the other. The other
important finding is that as the class size increases, peer effects actually help student to compete
for better achievement. But the effects may differ from one type of student to another.
Many state and local educational authorities have implemented class size reduction
programs, such as the Project STAR in Tennessee, the Project Prime Time in Indiana, the Project
SAGE in Wisconsin, the class size reduction program in California, the class size reduction
program in Florida, and the class size reduction project in Burke County, North Carolina.
24
Researchers have been looking into what the results from these experiments (Biddle & Berliner,
2002; Finn 2002; Ilon & Normore, 2006). The current knowledge about class size is that if
funded adequately, small classes benefit students in their early stage of schooling and the effect
can carry over to higher grade levels. The magic number is around 20 which mean classes with
less than 20 students tend to improve student learning. The benefits from small class are greater
for minorities and economically disadvantaged students. In addition, small class size improves
teacher morale, and gives teachers more individual time with students (Biddle & Berliner, 2002;
Finn, 2002; Ilon & Normore, 2006).
The rationales for the class size reduction in earlier grades are that students in their early
age of education require more attention and behavior assistance than when they are older and
more acquainted with the class setting. The research does shed lights on where to spend the
money. As Hoxby (2000) noted in her study of class size effect, some of the benefits from class
size reduction are simply too costly for most schools to consider, because the policy
implementation for cutting class size also means hiring more teachers, and building more
classrooms, which ultimately means increasing the budget of the schools. This ties closely to the
impact that schools bring to the education production function since the inputs of education are
the independent variables which affect the outputs of education. The overall summary of this
section can be seen in the Table 2.
Table 2
Research Findings on Class Size and Achievement
25
Author Data Method Variables Used Finding
Glass & Smith (1979)
More than 100 articles on class size
Meta-Analysis Class size of 20 Small class (less than 20) size is beneficial for students, non-linear relationship after 20 students
Hanushek (1986)
1960-1980, 112 estimates from 147 studies of the relationships between pupil/teacher ratio and achievement
Vote counting of regression estimates of the partial effect of given input
Pupil-teacher ratio, teacher education, teacher experience, teacher salary, and expenditure/pupil
Few pupil-teacher ratio variables showed significant relationships but the directions are different
Elliot (1998)
NELS: 88 Dataset Education production function, HLM
Promoting student achievement, and effective teaching strategies
Teachers’ encouragement has positive impact on student learning
Rice (1999) NELS: 88 Dataset Education production function, Regression
Class size, teacher experience, instructional time, and non-instructional time
Class size has positive impact on class time; teacher experience has negative impact on student learning
Nye & Hedges (2001)
Tennessee Project STAR
Education Product Function, Randomized experiment
Class size Small class has positive effect on student learning and math achievement
Nye & Hedges (2004)
Tennessee Project STAR
Education Product Function, Randomized experiment
Class size Small class has positive effect on minority student but negative effect on girls
Borland, Howsen, & Trawick (2005)
Kentucky Department of Education data
Education Product Function, Regression
Class grade, rank, experience, salary, and student innate ability
non-linear relationship between class size and achievement
School Effects
One of the strongest indicators of school effects on student learning is the fiscal ability of
the school district. The implication is that high SES schools can spend the money on improving
26
the teacher quality and reduce classroom size. Studies on school effects have pointed out that
school funding in part depends on the historical heritage of the school district (Bowles &
Bosworth, 2002). Their study calculated the costs of 17 Wyoming school districts and revealed
that on average it costs more in smaller schools than in larger schools for students to achieve
similar outcomes. But this research used ordinary least square regression on mainly school level
data, and the characteristic of students and teachers were not presented to get the more complete
picture.
A number of studies have found that school level resources do affect student
achievement. Wenglinsky (1997) conducted a study based on selected nationally datasets, which
provided empirical evidence that school level expenditures in some areas matter to student
achievement. The study showed that more money used to hire more teachers leads to greater
student achievement, while increased spending on school administration or raising teachers’
salary did not.
Using a national representative data, Ram (2004) found that school level resources are
positively correlated with mathematics and verbal SAT scores. Jacques and Brorsen (2002)
studied Oklahoma school districts and used education production function to find that school
level spending is positively related with student achievement, especially when the money is spent
on instruction related areas.
Another trend in the research of school resource is to look at school size effect on student
achievement. To answer this complex question, Lee and Smith (1997) investigated the NELS:88
dataset using HLM methods to find that school size exerts influences in two ways. One comes
from the theory of economies of scale in that large schools promote specialty in curriculum
skills. In other words, these skills are positively correlated with school size. Therefore, student
27
achievement goes up as the size of the school increases. School size also exert influence on that
schools also function as social communities, and as the size of school increases, teachers and
school administrators have reduced the ability to teach all the students and provide
individualized care. Therefore, as the school size increases, students may suffer. The two
competing effects influence students at the same time to make the optimal student number
between 600 and 900, which means students at such schools can enjoy fairly good instruction
and social care to make the most achievement (Lee & Smith, 1997).
Teacher Effects
Teachers are the most important factors in the schooling process: children can somehow
learn without books, or even without classrooms, but they can hardly learn without a teacher.
Indeed, research has proven that teacher and classroom effects have the biggest impact on
student learning (Goldhaber, 2002; Goldhaber & Brewer, 1997, Xu & Gulosino, 2006).
There are different perspectives of teacher and classroom effects. Teachers are
categorized in terms of teaching degrees, teaching experience, and teaching certifications. There
is research showing that the teachers’ degrees have a non-linear impact on student learning.
Summarizing from existing research, Odden, Borman, and Fermanich, (2004) found that the
impact increases as teacher degree goes up, and then the effect fades to a flat line after teachers
getting a bachelor degree. Years of experience have little impact except that the teachers with
three or more years of teaching showed stronger impact than teachers with less than three years
of teaching. The findings for teacher licensure indicated the certification has a positive impact on
student learning (Odden et al., 2004).
On the one side, there is research arguing that teachers have substantial impacts on
student learning and that there is a large variation in the impact of individual teachers across
28
classrooms even within the same school (Odden et al., 2004). Therefore, educational spending
targeting these teacher characteristics should contribute to student achievement.
However, there is other research saying the opposite. Podgursky and Springer (2007)
summarized past literature of teacher effects and found that the type of teaching certification,
teacher education, licensing exam scores, and experience are generally weak predictors of
student achievement. Ballou and Podgursky (2000) claimed that most of the teacher preparation
and licensing research are misrepresented which Darling-Hammond (2000) found to be untrue
due to the different research methods they used. Later, Darling-Hammond (2007) pointed out
that qualified teachers are the key to student success, and she also called for more federal and
local support for highly effective teachers.
Teacher’s salary seems to matter little to student achievement. Through the research on
teacher pay and focusing nationally and in 66 metropolitan areas, Greene and Winters (2007)
concluded that even though on average public school teachers are paid more than their private
school counterparts and most white collar workers, increasing their pay did not increase student
achievement.
In summary, research has shown that a number of student, school, and teacher factors
influence student achievement. However, discrepancies exist within the application of education
production function. One reason for that is the imperfect nature of the research method, which is
introduced in the next section.
Issues and Future Directions of Education Production Function
The fundamental problem with the educational production function is that many
researchers have been applying single-variable designs with the function. As Odden, Borman,
and Fermanich (2004) pointed out, too much previous research has tended to examine student
29
achievement based on student, classroom, and school variables in isolation from other important
factors. Others (Carpenter, 2000; Ilon & Normore, 2006) noted that research designs that only
used one variable to emphasize the educational outcome was one of the problems of the recent
education reforms.
Those who use education production functions to study the relationship between
expenditures and student achievement, often commit a common mistake in that they often over
look the critical process variables such as curriculum setting, and classroom instruction. This
leads to the criticism of the over simplification of the education production function (Odden &
Picus, 1992). There are a number of procedures in between having the money to spend on
education and the students learning more or gaining better scores. If these variables representing
teacher and classroom behaviors are missing, the education production function is definitely
misused.
Odden and Picus (1992) also pointed out another potential threat to education production
function that is the mismatch between educational goal that the school is carrying along and the
assessment for which students are held accountable. Most of the existing education finance
research that deals with data and analysis does not always address whether the curriculum
assignment and the tests are in the mission statement of the school district. If the tests are not a
priority of the school agenda, students low test scores may not lead directly to the conclusion that
“money does not matter.”
The solution of these critiques may not be very complicated. One way to improve the
usage of education production function is to increase more variables in the model. For example,
examining the relationship between educational expenditure and student academic achievement,
it would be problematic if the researchers only used expenditure variables as explanatory
30
variables. There should be at least some control variables such student race, gender, and socio-
economic status, among some of the instructional characteristics. The premise for this strategy is
the availability of dataset with ample variables, either through the requesting of large national
datasets or the collections of first hand dataset by education researchers.
Problems with the education production function may also occur in the misrepresenting
the relationships between the explanatory variables and the dependent variable. In a recent study,
Harris (2007) examined the Trends in International Mathematics and Science Study (TIMSS)
1999 dataset and found educational expenditure has a diminishing effect to student achievement.
This non-linear relationship between input and output is not new. Ferguson (1991) analyzed
school expenditures in Texas school districts and found among elementary school teachers, the
positive teacher effects turned insignificant at around five years of experience, and additional
years of teaching experience did not improve student test scores. Similar effects occur in other
studies from the Alabama experience (Ferguson & Ladd, 1996).
Since most of the existing researchers use some sort of linear regression methods along
with the education production function, determining whether there is a linear association
between the input and output can be problematic. The interpretation of the findings must be
careful and the insignificant impact from explanatory variables to student outcome can tell one of
the two things. One, money of different forms does not matter to student achievement. Two,
money matters in some non-linear way, such as teaching experience having a positive correlation
with student learning under the first five years and the effect fades out after five years of
teaching experience (Ferguson, 1991). One solution for this problem is a careful assessment of
the true relationship before concluding whether money matters or not. The other solution is
adopting a more sophisticated statistical approach to the education production function to capture
31
the subtle change in the direction of the relationship between educational expenditure and
outcome.
One other issue with the education production function is from the application of this
model in analyzing student achievement. Fowler and Monk (2001) pointed out that expenditure
is not the best indicator for student learning outcomes using the education production function. It
is the actual cost associated with the educational activities that should be used to predict student
achievement. They also noted that educational cost is the minimum expenditure spent on
education. The more appropriate approach to study education finance it to use cost function
instead of education production function.
Looking from the cost side of education, one interesting implication is that like many
other public institutions, public schools tend to spend as much of the available resources they
have in one fiscal year. Therefore, the reality is that more spending does not necessarily lead to
better outcomes (Hanushek, 1989). In the education production function, if the spending
variables are included, the results may be skewed because these spending variables are likely to
be more than the actual cost of education production.
However, the superiority of cost function also brings the challenge of measuring the real
cost of education. One must have known the amount for adequate funding for as certain degree
of education to claim the amount of cost. Anyone studying in the field of education finance
understands that it is difficult to define adequate funding.
Even though the application of the education production function may include a range of
student, teacher, school, and other variables, sometimes the relationship between the explanatory
variables and the dependent variable can be spurious if all the variables are on the same level of
analysis (Odden & Picus, 1992). For example, individual and aggregate students’ social-
32
economic status may have different effect on student achievement. In another instance, teacher
experience has unique influence to student learning, but using average experience in one school
and treating all the teachers as one variable reduces the explainatory power from individual
teacher experience.
Furthermore, incorporating all the student level and school level variables still does not
suffice to capture a full picture of student achievement. In addition to these exogenous factors of
schooling, education production functions should be more powerful in predicting student
academic achievement by adding endogenous variables which are innate or closely related with
family characteristics of the students. Student behavior characteristics such as locus of control,
motivation, and attitude within the education production function should explain part of the
schooling myth.
Student Learning Motivation Theories
Distilling from the literature on student behavioral research, there are three major
concepts that are believed to have influence on student achievement: locus of control,
motivation, and attitude. Locus of control, motivation, and attitude have different effects on
student learning. And they also have internal influence with each other. This section reviews the
relevant theories on each of the three concepts and the next section draws empirical evidence that
these theories are based. Most of the findings are generated from the education production
function settings.
Locus of Control
The locus of control construct originated from Rotter’s (1966) social learning theory,
which was defined as a person’s belief in the amount of control a person has in life, either in
general or in a specific area. Locus of control has impact on learning, motivation, and behavior
33
(Pintrich & Schunk, 2002). There are two types of orientation for locus of control. An internal
locus of control refers to a belief or expectancy that one’s behavior or stable personal
characteristic will control specific events or outcomes (in other words, self-belief). External
locus of control refers to a belief or expectancy that events or outcomes are controlled by
external forces to oneself (such as luck or other people’s power).
When talking about influence on something, people usually regard that as the internal
locus of control. Previous research generally shows that internal locus of control has positive
impact on student achievement and that internal locus of control is a powerful variable both in
predicting and explaining educational achievement (Shepherd, Owen, Fitch, & Marshall, 2006;
Sterbin & Rakow, 1996). James Coleman’s widely cited Equality of Educational Opportunity
Study proved that locus of control is a one of the strongest predictors for student achievement
(Coleman et al., 1966).
In addition, internal locus of control influences individual motivation to some extent
(Graham, 1994), because locus of control is also an element of attribution theory besides social
learning theory (Autry & Langenbach, 1985). The causality, in the form of many successes or
failures, is attributed to the choices of different behaviors. although the two theories all make
predictions regarding the influence of causal factors on the expectancy of success, there are sharp
differences between social learning theory and attribution theory in terms of theoretical origins.
Social learning theory is more abstract; while attribution theory uses concrete concepts (Weiner,
1976). The two theories will be discussed in detail in the motivation section.
Motivation
Derived from the Latin verb movere (to move), “motivation is the process whereby goal-
directed activity is instigated and sustained” (Pintrich & Schunk, 2002, p. 5). The central element
34
for motivation is an anticipated goal, which provides impetus for and direction to action. Goals
may not be well formulated and may change with experience. Goals are something individuals
have in mind that they try to achieve or avoid. Motivation also requires activity, either mental or
physical. Physical activities include effort, persistence, and other overt actions; while mental
activities involve cognitive actions such as planning, rehearsing, organizing, monitoring, making
decisions, solving problems, and assessing progress (Pintrich & Schunk).
As Tollefson (2004) described, motivation comes from cognitive psychology, in which
people believe cognitions cause behavior. Cognitive psychology deals with the process of how
people obtain information and how they interpret and give personal meaning to situations or
events in their lives. Cognitive theories of motivation start from the premise that people try to
bring order into their lives by developing personal theories about why things happen the way
they evolve. There are three major cognitive theories explaining motivation: expectancy value
theory, attribution theory, and social learning theory.
Expectancy value theory hypothesizes that the degree to which people will devote effort
at certain things, such as learning is a function of two variables: expectancy construct and value
components. Expectancy construct reflects individual’s beliefs and judgments about his or her
capabilities to finish the task successfully. Value components of motivation are linked to the
results of the task; they are the rewards on completion of the task (Pintrich & Schunk, 2002;
Tollefson, 2004).
According to Tollefson (2004), there are two types of rewards that are commonly
associated with completion of the task. One is external reward, which is the rewards from other
people such as teachers, parents, and peers. The other reward is internal reward, which is a sense
of self-fulfillment.
35
In the case of education, it is similar to a two-part test. A student will have a good sense
of motivation if he or she first anticipates whether the learning process is feasible as planned by
the teacher or curriculum. Second, the student understands what kind of consequences he or she
will receive from successful completion of the learning process. Most of the time, the
consequences are good, although sometimes students are motivated to learn to avoid bad
consequences such as being punished for bad grades. These are external rewards. The internal
rewards may be that the student feels something is accomplished, and he or she is mentally
happy or has a sense of pride with the educational achievement.
The second motivational theory is social learning theory which is generated from Rotter’s
(1954) and Bandura’s (1986) work. As Rotter pointed out, “The major or basic modes of
behaving are learned in social situations and are inextricably fused with needs requiring for their
[sic] satisfaction the mediation of other persons” (p. 84). The theory comprises four basic
variables: behavior potential in a given situation, expectancy of a consequence following a
specific behavior, reinforcement value of outcome, and the psychological situation in which the
context of behavior is captured (Pintrich & Schunk, 2002)
In the field of education, there are also two premises for this theory. One is that students
make personal interpretations according to their past accomplishments and failures to set goals
based on these interpretations. The second premise is that students set individual goals which
become their personal standards for evaluating their performance (Tollefson, 2004).
An important element in the motivational process is self-efficacy, which Bandura defined
as “People’s judgments of their capabilities to organize and execute courses of action requires to
attain designated types of performances” (1986, p. 391). There are four principal sources from
which people develop their personal sense of efficacy: “performance accomplishment,
36
observation of the performance of others, verbal persuasion and related types of social influence,
and states of physiological arousal from which people judge their personal capabilities and
vulnerability” (Tollefson, 2004, p. 209).
In the case of education, for example, a student may take a course and the student will
have an anticipated grade based on information such as how hard the course is, and what the past
achievement is for the student. After first setting the goal for the course, the student will use this
as this goal to assess the current endeavor.
Social learning theory shares some similarity from the expectancy value theory. The
social learning theory argues that an individual undertakes self evaluation first, and then sets up a
goal based on the evaluation. The next step is to work toward the goal. In expectancy value
theory, the individual understands the goal first. Based on all the information of the goal and
what the expected reward is, the individual then decides whether to work on the goal or not. The
major difference is that in the expectancy value theory, incentive plays an important role for the
individual, whereas in for the social learning theory, the individual is more likely to receive an
internal incentive of meeting the goal during the process of working toward the goal.
Expectancy value theory and social learning theory all point out that individuals are
motivated to perform certain tasks based on the expectancies for success and the values
connected to the success. These theories assume that individuals are rational thinkers who plan
things before they seek out the task challenges. These theories also assume individuals have
good sense of their ability in certain task areas. However, in the realm of education of school
children, these assumptions are not always satisfied.
The third motivational theory is the attribution theory which is concerned with
perceptions of causality and with the consequences of these perceptions (Tollefson, 2004).
37
Attribution theory believes individual human beings who are trying to understand their
environment and the causal determinants of their own behavior as well as the behaviors of others
(Pintrich & Schunk, 2002).
Attribution theory also looks at both thoughts and feelings as motivators of human
learning. The key for the attribution theory is on the individual’s search for understanding the
conditions for success or the reason for failure. These conditions can be personal or
environmental. Environmental conditions include specific information as well as social norms.
Personal conditions include a variety of schemas, prior experience, and knowledge of the
individual’s ability. Furthermore, prior knowledge and beliefs, and individual differences are the
two major factors to influence the attribution process (Pintrich & Schunk, 2002).
Derived mostly from Weiner’s (1972) work, attribution theory has four elements that
people commonly use to explain why they have been successful or have failed at an academic
task. These causes are ability, effort, task difficulty, and luck. Weiner also developed dimensions
to categorize these causes. On the one dimension, the ability and task difficulty causes are stable,
while the effort and luck causes are unstable. On the other dimension, the ability and effort
causes are internal, while the task difficulty and luck causes are external. Empirically, Uguak,
Elias, Uli, and Suandi (2007) found that the majority of students contribute their causes of
success as internal rather than external.
In a piece by Weiner (1972), the attribution theory was applied to the study of the
educational process. Weiner specifically pointed out that attribution theory can be used to
examine the influence of causal beliefs on both teacher and student behaviors. Students may seek
to understand why they failed or passed an exam; a teacher may attempt to understand the
reasons why students have different behavior and performance over a variety of tasks in class.
38
The teachers have two instruments that they can use to motivate students: rewards and
punishments. So do the students to themselves. Achievement motivation is developed through
the instruments and the ability for the students to understand the causal linkages from striving to
the achievement activities – either reward or punishment.
Besides the three major theoretical frameworks, there are other terms used to express the
meaning of motivation such as aspiration and expectation. Academic expectation means the level
of education that students expect to achieve, while academic aspiration means the education they
hope to achieve (Hanson, 1994). Educational aspiration is among the most significant predictors
of educational attainment (Duncan, Featherman, & Duncan, 1972). Parallel to this, student’s
academic expectation, in the next step, is an important variable both in predicting and explaining
educational attainment (Hanson, 1994).
Human and financial capital from parents can provide a cognitive environment and
physical resources for the child that aid performance and aspirations (Qian & Blair, 1999; Sewell
& Shah, 1968). Expectation, like aspiration, belongs to the category of motivation (Hanson,
1994; Trusty, 2000). Trusty (2000) used the NELS:88 dataset and logistic regression to find that
there are a number of factors influencing student academic expectation such as socioeconomic
status, ethnic group, prior achievement, parental expectation, locus of control, parental
involvement of school extracurricular activities.
Similar findings emerge from other research. Flowers, Milner, and Moore (2003) visited
the issue of locus of control and its impact to student aspiration. The authors argued that to
examine the relationship, the theoretical framework must include three aspects of education:
family characteristic, student characteristic, and school characteristics. The family aspect is
important because parents are vital to provide academic support and motivation. In addition,
39
successful parents are in a good position to stay active in their children’s education process and
get involved with students and schools when needed. The student aspect of education process is
that when they have good attitude in learning, they tend to enjoy many of the school activities
which in return help them learn more. The school aspect of education is usually connected to the
resource of various sorts that schools are able to allocate. The school facilities and in-school
programs are important to stimulate student learning.
Locus of control is the top level of behavioral concept and it is believed to have influence
on motivation. Employing regression analysis to the NELS dataset and the first two follow-up
surveys, and controlling for family characteristic, student characteristic, and school
characteristics, Flowers, Milner, and Moore (2003) found that African American high school
seniors with higher levels of locus of control were more likely to have higher educational
aspirations than African American high school seniors with lower levels of locus of control.
Synthesizing from existing learning frameworks, Walberg (1984) studied nearly three
thousand research articles in the U.S. and internationally and concluded that nine important
factors are influencing student learning: 1) the ability or prior achievement, 2) the childhood
development, 3) the student motivation, or self-concept, 4) the amount of time students engage in
learning, 5) the quality of the instructional experience, both psychological and curricular wise, 6)
the home factor, 7) the classroom social group, 8) the peer group outside the school, and 9) the
out-of-school time on studying. These findings suggested that not only these nine factors have
significant impact on student learning, but they also have impact to each other.
Walberg’s (1984) famous Educational Productivity Model divides these nine factors into
three categories. First, the attitude, motivation, and prior learning ability are internally affecting
student learning outcomes. Another category consists of other factors, the classroom social
40
group, the peer group outside the school, and the out-of-school time on studying as
environmental factors that are affecting student learning externally. Third, the amount of time in
learning and instructional quality are the school factors which are usually unalterable compared
with the other two categories. The research concluded with that these three major categories all
have a positive impact on student achievement.
Attitude
Attitude has a much broader meaning than motivation and locus of control, from the field
of psychology to education, and it usually consists of the following components: persistence or
perseverance, and self-esteem or self-concept.
Self-concept means “the totality of the individual’s thoughts and feelings having
reference to himself as an object” (Rosenberg, 1986, p. 7). Rosenberg went on to clarify the
definition by providing what the self-concept is not. On the one hand, self-concept is not as
cognitive as Freud’s ego, which consists of a set of intellectual processes enabling the individual
to think, perceive, remember, reason, and attend to deal with reality. Ego does work to protect
and enhance the self-concept; however, it does not constitute the self-concept. On the other hand,
self-concept is not as real as the “real self,” since the person may have a disposition to love, to
write, to compose, or to excel. In other words, the self concept is only the picture of the real self
(Rosenberg, 1986).
There are a few motives for self-concept. Self-esteem as the first motive is “the wish to
think well of oneself” (Rosenberg, 1986, p. 53). Rosenberg did suggest that self-concept and
self-esteem vary across a range of factors, such as age, race, social class, birth order, sex,
religion. Therefore it will be useful to put these two concepts into specific context.
41
Self-esteem or self-concept is the factor often studied in the educational field about how
students view themselves in regard to study. Self-concept consists of the strength of a student's
sense of self-worth and overall self-esteem (Lee, Daniels, Puig, Newgent, & Nam, 2008, p. 309).
Slavin (1997) stated that self-concept includes the way people perceive the self strengths,
weaknesses, abilities, attitudes and values. Similarly, self-esteem is about how people evaluate
self skills and abilities.
One of the most studied aspects in attitude is the race factor, that students of different
races should have different educational attitude. Since Ogbu and Fordham’s (Fordham, 1988;
Fordham & Ogbu, 1986; Ogbu, 1978; 1983; 1991) series of writings about the ethnic-
achievement dilemma in the U.S. education system, they also developed the oppositional culture
theory. The theory states that some African Americans purposefully choose to expect less effort
than they would otherwise do because of the opposition to the behavior of the majority White
students. As Ogbu (2004) pointed out, the lack of achievement from Black students can be
explained by the burden of “acting White.” The oppositional culture theory views this type of
behavior as blocked opportunity.
To explain why some minorities succeed while others tend to underperform, Ogbu (2004)
distinguishes between two types of minorities: voluntary minorities and involuntary minorities.
Voluntary minorities are those who migrated more or less voluntarily from their homeland to
America. Asian Americans are often cited as an example of immigrant minorities since a
significant portion of the first generation migrations came to the U.S. in seeking better
opportunities.
On the contrary, involuntary minorities are those who did not originally choose to arrive
at American land. Instead, they unwillingly left home and to come to the U.S. These differences
42
of entry into American society result in differences in adaptation and differences in the
perception of cultural and social barriers usually confronted by new migrants. Voluntary
minorities are more willing to adapt to American society as a means for upward mobility and
they believe the possibility to achieve better social status through hard work like the Asian
Americans (Qian & Blair, 1999). Involuntary minorities, however, perceive American culture as
a threat to their own identity, and view the historical and contemporary social structures as
barriers for them to move up the social ladders. They usually have lower educational aspirations
compared with voluntary minorities.
More specifically, Ogbu (2004) reasoned two types of burden the involuntary African
Americans may face. One type of the burden is from the past history. During the slavery era,
Blacks were forced to do whatever the Whites told them to do. As part of the history which can
never be easily forgotten, the Black people now are looking at this past period as bitter
experience. Traditionally, Black people do not want to follow the mainstream people because
they do not want to comply with the Whites. The second type of burden of “acting White” is
more practical in the modern society. It is commonly observed that Blacks are rewarded a lot less
than their Whites counterparts with the same skill, education and ability. Perhaps this occurs
because real segregation still exists. Black students learn these facts in the real world or they are
told by their parents who suffer from this kind of unequal treatment will be reluctant to invest
their full strength at school work. They do not believe education will benefit them the same as it
will benefit to the White students (Ogbu, 2004).
Past research has successfully explained the reason why students of different race have
different achievement levels holding other things constant. Mickelson (1990) observed the
education achievement disparities between White and Black students and found the achievement
43
gap was closely associated with student attitudes toward school. Through a series of surveys, the
findings suggested that the Black students have a dual-value system. What every student in
general believes is that education is the only way to a better life and it will provide an equal
opportunity to everyone. But the belief conflicts with the real world observation that Black
students face. The fact is that every student may have a good attitude toward education
regardless of color, however, the Black student has another attitude that education will not work
for them.
According to Mickelson (1990), the first type of attitude is abstract attitude or external
attitude, and the second type of attitude is concrete attitude or internal attitude. The research
results showed that the concrete attitude outweighed abstract attitude when the common belief
conflicts with the individual observation. This explains the education achievement disparity very
well. Associate with this concept, the current educational achievement can be explained partially
by student attitude. Because students also spend out-of-school time at home. If they have a
positive attitude toward studying, they may work hard at home. This improves their academic
achievement but it has nothing to do with educational expenditure.
Mickelson (1990) also suggested that attitudes are less effective in improving
achievement when they are too abstract. Abstract attitudes are formed from the prevalent belief
that hard work leads to better pay, and ultimately leads to higher social status. In school, this
kind of abstract ideology is translated that anyone can become successful by studying hard in
school. Mickelson views these concepts as too abstract, and they are not linked to the reality
because one man’s case can be completely different to another’s. Therefore, such belief in hard
work does not automatically lead to student achievement. Concrete attitudes, on the other hand,
are formed from students’ daily experiences. They see the life of the people around them. In the
44
case of African Americans, they are most likely to interact with other African Americans. If they
see real examples around them and still fail to achieve desired goals through hard work, the
concrete attitude outweighs the abstract attitude and the students may lose faith in working hard.
Theoretically, it is reasonable to believe that if a student holds a positive attitude of him
or herself, the student should routinely exhibit positive or persisting behaviors toward learning,
such as attending school regularly, participating in extracurricular activities, and completing
required work in and out of school. These positive or persisting behaviors should give the student
more chance to achieve more academically.
Empirical research has demonstrated that the influence from attitude is directly correlated
with student achievement. Applying the educational productivity theory, Walberg, Fraser, and
Welch (1986) studies the factors affecting student performance. The findings showed that
student achievement was related persistence, and persistence is positively correlated with
motivation, among other things.
In addition, research has proved that self-concept is among the most significant
predictors for student achievement in mathematics (O’Conner & Miranda, 2002; Reynolds &
Walberg, 1992; Thomas, 2000). Reynolds and Walberg (1992) also found that parents have
greater amount of influence to children’s self-concept.
Empirical Findings on Student Achievement
Many of the prior studies used the National Education Longitudinal Study of 1988
(NELS:88) and its follow-up studies to examine the relationship between student educational
behaviors and achievement of various measurements (Cook & Ludwig, 1997; Finn & Rock,
1997; Lan & Lanthier, 2003; Lee, Daniels, Puig, Newgent, & Nam, 2008; Muller, Stage, &
45
Kinzie, 2001; O’Conner & Miranda, 2002; Reis & Park, 2001; Singh, Granville, & Dika, 2002;
Sterbin & Rakow, 1996; Thomas, 2000).
Sterbin and Rakow (1996) applied path analysis and regression analysis to investigate the
impact from locus of control and self-esteem on student achievement. Specifically, they used
three variables to compose the locus of control scale:
“good luck is more important than hard work for success;
every time I try to get ahead, something or somebody stops me;
planning only makes a person unhappy, since plans hardly ever work out anyway.”
For self-esteem, they used four variables:
“I take a positive attitude toward myself;
I feel I am a person of worth, on an equal plane with others;
I am able to do things as well as most other people;
on the whole, I am satisfied with myself.”
The results showed that both locus of control and self-esteem are significantly correlated with
standardized test scores. In addition, they found that self-esteem and locus of control are highly
correlated with each other.
Another way to study the student behavior impacts to achievement is the examination of
different effects by gender. Reis and Park (2001) investigated the relationship between self-
concept, locus of control and math and science achievement with NELS:88 dataset. The factor
analysis sorted eight variables into the self-concept factor and five other variables into the locus
of control variable. The regression results indicated that the best predictor for distinguishing
between high-achieving males and females is locus of control. Finn and Rock (1997) used
identical dataset and similar statistical procedures except that they looked at particular at-risk
46
students. Their findings showed that the self-esteem and locus of control factors distinguish
students in terms of academic success.
Using the similar strategy to break down the student population by race and gender
subgroups, Muller, Stage, and Kinzie (2001) found locus of control to be strongly related to
students’ science achievement for all subgroups except Asian American males.
Cook and Ludwig (1997) took the issue of unequal motivation toward school wok
between Black and White students and tested there hypotheses. First, do Black students
experience greater alienation toward school than non-Hispanic Whites? Second, do Blacks incur
social penalties from their peers for succeeding academically? And third, if yes, are these
achievement penalties greater than those for the Whites? The research found all three questions
have negative answers after controlling for socioeconomic status and other family characteristics.
However, the findings did not provide evidence to verify that Black students would be rewarded
for “acting White” either. If they see no benefit for working harder for good grades or lack of
“concrete attitude (Mickelson, 1990),” even they would not be punished for succeeding. In other
words, they might still choose to be less motivated toward school work.
Applying a different outcome measurement to the NELS datasets, Lan and Lanthier
(2003) detected very similar results. The authors found that the student motivation composite had
a significant relationship with dropout decisions while the self-esteem composite did not. The
locus of control composite failed to show significant impact either. This research confirmed
some of the prior findings that prior academic performance, relationship with teachers, and
participation in school activities were significantly related with student attainment. Lee et al.
(2008) studied the relationship between motivation and postsecondary educational attainment
among low socioeconomic status students. Building on the categorical regression analysis and
47
path analysis, their research found no direct or indirect impact from student attitude toward
educational attainment. However, high academic expectation does have a significant effect on
student attainment (Lan & Lanthier, 2003).
In another study used the NELS:88 dataset, Singh, Granville, and Dika (2002) tested the
influence of motivation, attitude, and academic engagement on mathematics and science
achievement. They applied confirmative factor analysis and incorporated two motivation factors,
one attitude factor, and one academic engagement factor into the structural equation models to
predict achievement. In both the science and mathematics models, they found positive results
from all these factors. The authors further implied that since the three aspects of student learning
inputs are relatively alterable compared with fiscal spending and family characteristics, it may be
a wise choice to improve these motivation and attitude factors to improve student achievement.
Combining the NELS:88 and its first two follow-up studies, O’Conner and Miranda
(2002) selected 10 variables as proxies of student attitude of mathematics. Six of these items
were composed into the self-concept factor, and the other four were composed into the effort
factor. The regression analysis showed that these two attitude factors were the strongest
predictors of student mathematics performance. Using exactly the same datasets and regression
analysis, Thomas (2000) also found that motivation and attitude measurements are positively
correlated with student science achievement.
In the same study, Thomas (2000) found that attitude affects science achievement. In
observing the achievement gap among students of different ethnicity, his research also revealed
that the impact sizes of attitude among the major student ethnic groups are inseparable, however
the way attitude affects student learning is not the same across ethnic groups. The explanation of
this may come from Ogbu’s study (1983), which recognized the difference expectation of their
48
future among each ethnic group. Asian Americans are optimistic where as African Americans are
pessimistic about their future, because Asian Americans immigrants compare their current
conditions with their peers in their home county, in which case they are better off, but African
Americans compare theirs with the White peers in the United States, in which case they are
worse off. This different reference group theory gives the two ethnic population different
attitudes toward education since they view their future differently.
The other study used ELS:2002 dataset aimed to look at locus of control, motivation, and
attitude simultaneously in regard to student achievement (Sciarra & Seirup, 2008). Teacher and
student survey items were composed into factors to predict math achievement scores. The
findings showed that among the five major ethnicity groups locus of control and attitude explain
more variance of student achievement than motivation does.
Other Factors Affecting Student Motivation
From the previous discussion it is summarized that motivation and other behavioral
characteristics toward learning are heavily correlated with student race. In addition, these factors
are also influenced by gender, peer effect, prior ability, and parental role. This section briefly
discusses these factors in relation to student achievement.
Gender
Other than the race factor which has been studied to a great extent, gender, on the other
hand, is also an important factor in student motivation toward learning. Evidence shows that girls
and boys have different reactions toward motivation. Marsh, Martin, and Cheng (2008) studied
964 Year 8 and 10 high school students in their mathematics, English, and science classes in 5
49
Australian coeducational government schools. Their analysis of classroom motivation suggested
that girls are easier motivated than boys, and teacher’s gender has little impact on student
motivation.
However, O’Conner and Miranda (2002) summarized prior research and concluded a
mixed message: “gender differences in self-concept overwhelmingly report that males’
mathematics self-concepts were higher than females’ mathematics self-concepts and females’
verbal self-concepts exceed that of males’ although females had higher achievement grades in
both core areas” (p. 77). Achievement wise, boys do have apparent advantages over girls,
especially in mathematics and science. Another finding is that girls in higher grades are less
interested in taking mathematics and problem-solving courses, compared with their male
counterparts.
Goldsmith (2004) studied eighth graders from NELS:88 and found that female students
tend to have greater educational and occupational aspirations than male students. The findings
also showed that Black and Latino students are more likely than White students to have high
educational and occupational aspirations. Combining these two factors, Goldsmith (2004) found
an even bigger gap between White males and Black females. The model also predicts that in
terms of educational aspirations, Black females have a greater advantage over White females
than Black males have over White males.
Research on gender-achievement issues borrows the concept of gender difference and to
explain the disparities between the girls and boys. Lundy (2003) selected student samples from
the NELS:88 dataset and analyzed the race and gender effect on student achievement. The
findings show that African American students are less likely to express hard working behavior
50
than the White and Asian American students. although there is evidence that African American
students have pro-education attitudes, over all, they still achieve less than other ethnic groups.
More importantly, Lundy’s (2003) study found that there is a clear gender factor among
high school students, in that male students appear to be more resistant to school work, and they
receive poorer grades than female students. Lundy offered a hypothesis that male students view
hard-working female students as girlish, and girls resist the image of girlish by not study hard
intentionally. Furthermore, the study has controlled parent and other family characteristics
because these factors are often associated with race but not gender (Lundy 2003).
Peer Effect
Most of research about the peer effects of learning has been focused on the race and
gender factors. Others have been looking further at the sub-groups of ethnicity and gender
combined (O’Conner & Miranda, 2002). Still there is important reason to understand the peer
effect by controlling race, gender, socioeconomic status, among other factors. Rothstein (2000)
wrote that students are not only affected by the educational institution, but also affected by their
fellow students. Peer effect cannot be neglected when studying the student learning process. Peer
effect coming from students of similar background usually contributes to the achievement and
underachievement of students. A positive peer environment has been found to have a positive
impact on attitude and small to moderate impact on learning outcomes (Walberg, 1984). Reis and
McCoach (2000) found that high-achieving students had a positive influence on gifted high
school students who began to under-achieve.
On the other hand, students with low academic performance usually find themselves with
negative attitude, and this attitude in return is among the strongest predictors of student
underachievement (Clasen & Clasen, 1995). Their study also showed that about two thirds of the
51
participating students mentioned peer pressure or attitude of the other kids is the primary force
against them obtaining good grades.
One intriguing fact is that the notion of high-achieving and low-achieving are always
relative in a specific context. For example, in a school of hundreds of students, no matter where
to draw the achievement line is drawn, it is reasonable to assume that there are always students
above the line and students below the achievement bar. When studying the impact of one group
of students to the other group of students, the separation criteria is critical to the results,
especially if the students are from a homogeneous population.
Prior Ability
Researchers have tried to detect whether attitudes influence achievement or prior
achievement influences attitudes. Adopting the second follow-up survey (NELS:92), Qian and
Blair (1999) studied the factors that were influencing student motivation which ultimate by
influencing student achievement. They revealed that prior educational performance has a strong
impact on student motivation. Through their research, O’Conner and Miranda (2002) found that
prior ability is highly correlated with student attitude toward learning, and this holds true for any
demographic subgroup: gender or ethnicity. Similar effects were found among African American
male and female students. Prior academic performance is the strongest predictor for education
expectations, and then the high expectations tend to lead to high educational attainment (Trusty,
2002).
Wilson (1983) conducted a meta-analysis covering research on kindergarten through
college students and found that successful achievement leads to positive attitudes. Using
structural equation modeling, Reynolds and Walberg (1992) studied science achievement and
attitudes among eleventh graders. Their findings confirmed prior research in that prior
52
achievement is among the factors that influence attitudes about science. In addition, Reynolds
and Walberg’s (1992) results showed that science achievement influences science attitudes but
not the other way. Another important finding from this research is that high motivation also leads
to good attitude toward learning (Reynolds & Walberg, 1992).
Examining both the prior achievement and attitude together, research has identified that
prior achievement plays a more important role than learning attitude or socioeconomic status
(Ma & Williams, 1999). The fact that student’s prior ability explains present achievement is
intuitively understandable. If the effect is direct, the lag effect shows that student performance
does not change dramatically unless unexpected things happen. On the contrary, if the effect
from prior ability to student achievement is indirect, and then theoretically there may be a loop
between achievement and motivation. Prior academic achievement enhances student self-esteem,
therefore enhances student motivation toward learning. As research has shown, enhanced
motivation correlates with current student achievement.
Parental Role
Typically, educational aspirations from parents have been captured as parental
expectation of their children’s highest education level. Trivette and Anderson (1995) concluded
that this is one of the ways parents can improve student motivation. Parental characteristics and
other social variables also have high a correlation with student motivation, and research
suggested the effects come through parental involvement in students’ school activities (Qian &
Blair, 1999). In addition, the findings further suggest that the role of parental status is stronger
for White students than for African Americans in educational attainment.
George and Kaplan (1998) conducted research with the NELS:88 dataset. Their path
analysis discovered that parental involvement has significant direct as well as indirect effects on
53
student science attitudes. The indirect effect is mediated through science activities and
library/museum visits. The study also concluded that science activities have a significant direct
effect on science attitudes toward learning (George & Kaplan, 1998). Evidence suggests that
other formats of parent involvement of school activities are also helping to improve student
achievement. These activities include active involvement in parent-teacher organizations, and
home-based partnerships or projects (Trivette & Anderson, 1995).
These findings agree with some of the empirical studies on student behavior patterns
(Qian & Blair, 1999; Sewell, & Shah, 1968; Smith-Maddox, 2000), and the reason that parents
matter is not only because the high parental expectation serves as motivation that passes to the
students (Fan, 2001; Fan & Chen; 2001), but also because parents with high levels of education
are able to provide financial and human capital to support students to learn more (Hanson, 1994;
Trusty, 2000). Trusty (2002) later studies this issue with NELS:88 dataset and found that
parental education level is a proxy for socioeconomic status factor to influence student
motivation on learning.
In addition to the parental aspirations to improve student achievement and parental
participation in school activities to enhance student achievement, Trivette and Anderson (1995)
found two other components of parental involvement that are useful for students learning.
Summarizing from prior research, they identified that parent-child communication about school
and school activities increases student aspirations and expectations positively. Such
communication can be simply a verbal interaction, or cues of problem-solving strategies.
The other important component is home structure or environment. The research on the
relationship between home environment and cognitive development has shown that parents are
able to set the “hidden” curriculum for success. Children coming out of a study-friendly home
54
environment tend to spend more time on studying. This type of motivation is unconsciously
working toward children overtime. On the other hand, if parents spend most of the time watching
TV at home or the home environment is noisy, children will not feel motivated to study at home
(Trivette & Anderson, 1995).
One other way parents can influence their children come from the influence they have on
the teachers. In a study about teacher quality, Xu and Gulosino (2006) found evidence to support
that the positive feelings about students’ parents not only encourages teachers to establish a
better teacher parent partnership, but also boosts teacher morale in classroom teaching.
In summary of this chapter, education production function provides a desirable platform
to analyze the relationships between various input variables and output measures. Most of the
methodological approach is regression on student and school data. The findings on school
expenditure, teacher quality, and various other student or school variables do not paint the same
picture as results from empirical studies are not consistent. Research using education production
function is not new; however, very few people conducted this type of research with the addition
of student behavior variables. The current understanding of the research is that to reach sound
results, the education production function needs to embody a broader than ever set of factors,
both the traditional ones and the new ones that this dissertation proposes. Student expectation,
motivation, and expectation on learning as education psychology proves, is among the best
predictors of student academic achievement. The addition of student behavioral factors
potentially leads to new finding of how to improve student learning through school expenditure.
The next chapter provides the detailed scope of the research.
55
CHAPTER 3
DATA AND METHODS
Overview
This chapter reviews the data and research strategies this dissertation uses to address the
research questions:
1. In the realm of education production function, what factors affect student academic
achievement the most?
2. Specifically, how do the motivation and attitude affect student academic
achievement?
3. Do these factors have the same effect at the student level and the school level?
4. What portion of variance do factors from each of the two level of analysis explain?
The analytical procedures of this dissertation are mainly building on the Education
Longitudinal Study of 2002 (ELS:2002) acquired from the National Center for Education
Statistics (NCES). Due to the nested nature of the data, the Hierarchical Linear Modeling (HLM)
is the most appropriate research methods for this dissertation. HLM has two level of regression
analysis: one on student level and the other on school (district) level. In sorting out of the
multiple independent variables, factor analysis of various formats is performed.
Data
ELS:2002 Description
The Education Longitudinal Study of 2002 (ELS:2002) is the fourth in the NCES
national longitudinal high school cohort series and is designed to build on the multiple policy
objectives of NLS-72, HS&B, and NELS:88. Base-year data collection for the study was in
56
2002, with approximately 20,000 10th grade students selected from 750 public and private high
schools. Policy issues to be studied through ELS:2002 include the identification of school
attributes associated with achievement; the influence of parent and community involvement on
student achievement; the dynamics and determinants of dropping out of the educational system;
changes in educational practices over time; and the transition of different racial-ethnic, gender,
and socioeconomic groups from high school to postsecondary institutions and the labor market
(Bozick & Lauff, 2007).
Like the earlier NCES studies, ELS:2002 examines students’ values and goals,
investigates factors affecting risk and resiliency, and gathers information about participation in
social and community activities. The study also contains teacher evaluations of the effort and
ability of each student, school administrator questionnaires, school library and media center
questionnaires, and parent questionnaires. In the ELS:2002 series, the first follow-up was
administrated in 2004, and high school transcripts were collected covering the span of the high
school years (Bozick & Lauff, 2007).
As in NELS:88, ELS:2002 includes measures of school climate, each student’s native
language and language use, student and parental educational expectations, attendance at school,
course and program selection, use of technology, planning for college, interactions with teachers
and peers, perceptions of safety in school, parental income, resources, and home education
support system (Bozick & Lauff, 2007).
The longitudinal study is also designed to support both longitudinal and cross-cohort
analyses and to provide a basis for important descriptive cross-sectional analyses as well.
However, priority was given to the longitudinal aspects of the study, with survey items chosen
for their usefulness in predicting or explaining future outcomes as measured in later survey
57
waves. The ELS:2002 content is also designed to provide comparability, where possible, to the
prior NCES high school studies to facilitate cross-cohort comparisons. For example, trends over
time can be examined by comparing the data from 1980, 1990, and 2002 high school
sophomores, collected with HS&B, NELS:88, and ELS:2002, respectively; or data from 1972,
1980, 1982,8 1992, and 2004 high school seniors, collected from NLS-72, HS&B, NELS:88, and
ELS:2002 (Bozick & Lauff, 2007).
There are seven survey components of the base-year design: assessments of students in
mathematics and reading; a survey of students; a survey of parents, teachers, school
administrators, and librarians; and a facilities checklist (completed by survey administrators,
based on their observations at the school). The student assessments measured achievement in
mathematics and reading and were based on the test frameworks used in NELS:88. The
assessments designed for ELS:2002 used items from NELS:88, the National Assessment of
Educational Progress (NAEP), and the Program for International Student Assessment (PISA).
The baseline scores from these assessments can serve as a covariate or a control variable for later
analyses. Mathematics achievement was reassessed in the first follow-up, so that achievement
gain over the last two years of high school can be measured and related to school processes and
mathematics course taking. The student questionnaire gathered information about the student’s
background, school experiences and activities, plans and goals for the future, employment
experience, in and out-of-school experiences, language background, and orientation toward
learning (Ingels, 2007).
The ELS:2002 also contains transcripts and grade point average (GPA) information that
were collected from sample members in late 2004 and early 2005, about 6 months to 1 year after
most students had graduated from high school. Collecting the transcripts in the 2004–05
58
academic year allowed for more complete high school records. Transcripts, both academic and
general, were collected from the school that the students were originally sampled from in the
base year (which was the only school for most sample members) and from their last school of
attendance if it was learned during the first follow-up student data collection that they had
transferred. Transcripts were collected for regular graduates, dropouts, early graduates, and
students who were homeschooled after their sophomore year. School course offerings data were
also collected (for base-year schools only) (Ingels, 2007).
Data and Variables from ELS:2002
Table 3 illustrates the selected variables the author will use and the hypotheses related to
student academic achievement. Among all the variables, the author particularly interested in the
attitude and motivation variables (e.g. how much students like school, how far in school students
think will get, and how far in school parents expect). The author want to test how attitude toward
education will affect academic achievement with controlling for other factors such as family
background, race, and gender. The author also interested in the relationship between educational
expenditures and student academic achievement. In this study, the educational expenditure
variable is represented by teacher’s education and teaching experience.
Table 3
Selected Variable Description
Variable Name Variable Coding Hypothetic
Relation
Explanations
Standardized test score (Math, Reading)
From low to high in a 100-point scale
N/A N/A
High School GPA Continuous number on a 4.00 scale
N/A N/A
Gender of student Male=1, Female=0 – Males usually have higher scores in secondary schools
Race of student Asian, Black, Hispanic, and White
? Asian and White students tend to perform better than black
59
students students
SES composition From low to high + The rich have more to spend on education than the poor
How much students like school
From “not at all” to “a great deal”
+ The more students like school, they tend to study harder to get better grades
How far in school student think will get
From high school degree to Ph.D. degree
+ The higher the expectation from students, the harder students may try to study
How far in school parents expect
From high school degree to Ph.D. degree
+ The higher the expectation from parents, the harder students may try to study
Discuss school related things with parents
From never to often + More discussion about things in school will help students understand school materials better
Teacher has master degree or above
Master=1, not=0 + The higher degree, the higher credentials teachers have to teach students
Teacher’s experience From 1 year to 40 years
+ The more years teachers have taught, the better the teachers might be
School size From low to high ? Not sure about the relationship
Free and reduced-price lunch student percentage
Free and reduced-price lunch student percentage increases
– The higher the percentage students are in free and reduced-price lunch, the less money the school may have to spend
Is the school in urban area Urban = 1 – Urban schools may have lower performance than rural schools
Percentage of minority students
Between 0 and 1 – The higher minority students in school, potentially the lower the overall achievement
ELS:2002 has more than 100 student behavior variables. Based on the literature and prior
research, the author use factor analysis to sort these variables into two or three factors. Factor
analysis is a typical way to reduce large number of variables into a few factors that make sense
of which variables each factor includes. The original variables load onto factors based on the
pattern of correlations. In this dissertation, these factors were categorized as “motivation” or
“attitude” depending on the factor loadings on each of them. The regression analysis in the later
stage incorporated these factors along with other explanatory variables.
60
Hierarchical Linear Modeling
One particular focus of the dissertation was to bring in the student motivation aspect of
education input. The next section states the problems associated with the design, application, and
interpretation of education production function before offers a number of solutions. To
demonstrate one of the solutions, the final section discussed hierarchical linear modeling, its
merits as well as its potential problems.
A better way to refine the education production function is to employ the Hierarchical
Linear Modeling (HLM). Education takes place in classroom with teachers; therefore, teacher
quality, experience, and certification have direct impact on student learning. Classrooms are in
schools buildings and each school has its own unique characteristic such as school leadership,
school type, school size, district fiscal ability, and geographical location of the school. These
factors also have direct or indirect impact on student learning. Based on the nested nature of
education, it will be helpful not only to include all these variables, but also to build them into
different layers.
Using HLM had distinct advantages over other statistical procedures. Cohen (1983) was
one the first few scholars that brought the attention of the multiple level modeling in education
research. Cohen (1983) called for a statistical framework that incorporated measurements of
multilevel structure of schools. Starting from the 1980s, more researchers had been considering
conducting education research using the nested model. As the techniques became more mature,
scholars recognized the importance of setting student, teacher, and school effects in a multilevel
framework by which to examine the improvements in student learning emerge from these three
different levels.
61
Burstein (1980) stated schooling activities occur within hierarchical groups, and
individuals belong to every level of such groups. Each of these groups has a specific
arrangements such as the learning group within classrooms, the classrooms within schools, the
schools within districts, the families within communities, and schools within communities. These
groups also have a specific influence on the group members in terms of thoughts, behavior, and
feelings of their members. In other words, members in one group may be quite different from
members in other groups even though they are on the same hierarchical level. This is the case for
students taught by different teachers; the teacher factor may have unequal contribution to student
achievement, even they are in the same school and same community.
To explain the complex influences of group settings on individual student behavior, the
hierarchical structure brings the chance to evaluate the effects of higher level organizational units
(such as schools) on their lower level subgroups (such as classrooms) (Burstein, 1980). Two
important advantages rise from this specific research technology: “(1) variables can have
different meanings at different levels of analyses, and (2) measures of group outcomes other than
the group mean warrant careful attention in analyses involving group-level outcomes” (Burstein,
1980, p. 161).
Raudenbush and Bryk (1986) summarized the merits of using HLM was the application
of slopes as outcomes. Take the usage of educational expenditure to explain the student
achievement for example: the first step will be running regression on student achievement based
on a spending variable within each school. By applying slopes as outcomes, the estimated slopes
from this step will be the outcomes in the second step which is the analysis of variances in slopes
on the basis of other factors within classrooms.
62
Raudenbush and Bryk (1986) also pointed some technical difficulties in conducting such
procedures. First, regression coefficients usually have greater sampling variability than sample
means. If the sample size is small, the regression coefficient may result in a large sampling error.
Therefore the beginning of the second step has less statistical power. Second, the sampling
precision of the estimated slopes varies across units depending on the data collection design used
within each unit. The regression analysis that is used in the second-stage usually assumes equal
variances across cases on the dependent variable. For example, the unequal variance at
classroom level, may bias the prediction at school level.
As the techniques mature, researchers are able to solve some of the difficulties to use
HLM. Beyond two-level hierarchical modeling, the emergence of three-level equations extended
the understanding of education from within school effects, between school effects, to within
classroom effects (Bryk & Raudenbush, 1988). Therefore the whole picture of education, and in
statistical terms, levels of the education system – the student, the classroom, and the school are
all under a closer examination.
Hierarchical linear modeling was used to analyze a range of education issues, one of
which being the relationship of educational expenditure and student achievement. Archibald
(2006) studied elementary schools of one school district in Nevada and applied three-level HLM
to students, teachers, and schools. She found that most of the per-pupil expenditure effects on
student achievement were explained at student level, and a small portion of the variance was at
school level while a much smaller portion was from the teachers.
A number of studies were carried out with the NELS:88 dataset because of the very
nested structure of the information collected. School finance has been an important topic for
educators for years. The application of NELS:88 in this area has been tremendous. After 20 years
63
of the first wave of data collection, researchers are still using these datasets. Grubb (2008)
summarized previous knowledge about the field of education finance and pointed out one of the
often neglected aspect of research is not paying attention to the education process. Using the
NELS:88 datasets, his study investigated various aspects of school finance and found that the
most significant factor to influence student achievement is family characteristics such as SES
and parental involvement in education. In addition, the research discovered from students’
perspective that motivation and engagement, such as doing home work on time and liking school
are important to student outcomes.
There are two bold policy implications coming out of this recent study. First, the simple
statement of “money matters to education; or, money does not matter to education” is not the
right way to describe the relationship between education and expenditure, even though the results
may come out of credible empirical evidence. Based on the fact that many factors are involved in
the education process, researchers should emphasize the process of determining where money is
spent and how the money is spent before making the conclusion. Second, because almost every
school finance research has found that family characteristics are important to student
achievement, the direction for school finance reform may turn to ways to improve these family
characteristics, such as housing policies to improve family stability and welfare policies to
guarantee income (Grubb, 2008).
Methods
This dissertation employed primarily HLM analysis on the relationship among various
factors to student academic achievement. The dependent variables are a standardized test
composite with reading and mathematics, and high school GPA. These variables are also
properly weighted to represent the sample population before being used. The explanatory
64
variables include gender, Black or White, SES, whether students have positive feeling about
themselves, negative feeling about themselves, time spent on study, attitude toward class, future
educational expectation, school expenditure, and student-teacher ratio. The student motivation
variables are reduced to a few factors using factor analysis.
The ELS:2002 data are nested with two levels; therefore, HLM is appropriate for the
analysis. Developed in the early 1980s, HLM is useful in this context because it allows the
researcher to use teacher and school level variables to explain variation in the individual level
parameters and provides a test for main effects and interactions between and within levels or
groups (Bryk & Raudenbush, 1988). For instance, at the student level, a basic question of interest
is whether the number of science courses taken by students positively predicts science
achievement within schools. At the school level, interest may focus on learning whether the level
of curriculum implementation within schools predicts mean science achievement across schools.
Another possibility concerns cross-level inferences, such as testing whether the level of
curriculum implementation within schools has any different effects across schools.
The first step of the dissertation research is the HLM analysis of student and school
levels. The results of the multiple linear regressions and the HLM models will be compared since
one of the critiques of education production function analysis with multiple linear regressions is
that it overlooks the nested nature of the education process, where students are grouped in
classrooms, and classrooms are grouped in schools.
Basic Regression Model
Yi = β0 + βi Xi + ri (1)
The intercept β0 is the expected value of Y when the starting point of independent
variable X is zero. βi is the parameter estimate of the independent variable X. The error term r is
65
the unexplained variance by the regression function. The “i” behind each explanatory variable
and the dependent variable represents the observation number of the dataset.
Level One Model
Most people use level one model for the individual level analysis, and more often than
not, that will be student level regression. Of course, there can be more than one independent
variable shown in this single regression model. Then the regression model with “l” independent
variables will be:
Yi = β0 + β1i X1i + β2i X2i + β3i X3i + . . . +βli Xli + ri (2)
There are several forms of the models depending on the research question and the data
availability. The simplest possible hierarchical model can be the random-intercept model in
which only the intercept parameter is in the level one model. Compared with model (1), the βi in
this model is simply zero now, and it is equivalent to a one-way ANOVA which tests the random
effect.
Yi = β0 + ri (3)
Level Two Model
The next level should be the groups that the individual cases reside in. One of the merits
of HLM is that HLM estimates a set of random effects associated with each higher level unit. In
the realm of education, the second level regression can be the analysis of school characteristics.
ELS:2002 dataset provides adequate school information so the author choose the second level to
be school analysis.
66
If the explanatory variables of the student level and the school level are exclusive, the
second level of regression will decompose the β0 of the level one model. For example, if the first
level is examining the student demographic impact to student test scores, the regression model
will have race (RACE), gender (GEND), socioeconomic status (SES), and student motivation
(MOV). Therefore the model will look like:
Yi = β0 + β1i (RACE)i + β2i (GEND)i + β3i (SES)i + β4i (MOV)i + ri (4)
And if the school level regression is examining the teachers’ average experience (EXP)
within school, teachers’ average education level (EDU) within school, and school funding
(FUND), the level-two model will look like:
β0 = γ00 + γ1j (FUND)j + γ2j (EXP)j + γ3j (EDU)j + rj (5)
Here the γ0 designates the expected value of β0 when all the school parameters are zero.
In other words, if the school has certain amount of funding, and default years of teaching
experience and default level of education, the value of γ0 will be the explained variance this level
two model has for the β0. The other γs are the regression coefficients of each of the level two
independent variables. The “j” means the number of schools in this dataset.
However, if the level two and level one model have overlapping explanatory variables,
for example, the research also examines the school level SES (M_SES), race (M_RACE), and
motivation (M_MOV), the level two model will be a bit complex. Beside the model (5), there
should be another model to decompose the β3 using the mean SES at school level to explain the
variance at the student level. The model should look like:
β1 = γ01 + γ1j (FUND)j + γ2j (EXP)j + γ3j (EDU)j + γ4j (M_SES)j + γ5j (M_RACE)j + γ6j
(M_MOV)j + rj (6)
67
Depending on the specific nature of the research, the level two model(s) can be as many
as the independent variables in the level one model plus the intercept. After writing down all the
level two regression models, take them to the level one model and use the parts to the right of the
equations to replace the parameter estimates in the level one model. This will be the combined
model for two levels:
Yi = [γ01 + γ1j (FUND)j + γ2j (EXP)j + γ3j (EDU)j + γ4j (M_SES)j + γ5j (M_RACE)j + γ6j
(M_MOV)j + rj ]i (RACE)i + [γ02 + γ1j (FUND)j + γ2j (EXP)j + γ3j (EDU)j + γ4j (M_SES)j + γ5j
(M_RACE)j + γ6j (M_MOV)j + rj ]i (GEND)i + [γ03 + γ1j (FUND)j + γ2j (EXP)j + γ3j (EDU)j + γ4j
(M_SES)j + γ5j (M_RACE)j + γ6j (M_MOV)j + rj ]i (SES)i + [γ04 + γ1j (FUND)j + γ2j (EXP)j + γ3j
(EDU)j + γ4j (M_SES)j + γ5j (M_RACE)j + γ6j (M_MOV)j + rj ]i (MOV)i + γ01 + γ1j (FUND)j +
γ2j (EXP)j + γ3j (EDU)j + γ4j (M_SES)j + γ5j (M_RACE)j + γ6j (M_MOV)j + ri + rj (7)
In summary, the main research technique this dissertation employs is HLM on student
and school levels. The ELS:2002 dataset contains numerous variables, and the independent
variables of interest are student demographic, teacher quality, school characteristics, and other
control variables. Generated by factor analysis, student behavior factors will be included in the
model. The dependent variables are learning outcome measurements such as math and reading
test score, NAEP score, SAT score, and high school academic GPA.
68
CHAPTER 4
FINDINGS OF THE STUDY
Overview
The purpose of this study was to explore the relationship between student behavioral
factors to student academic achievement in the realm of education production function. Other
factors of interest are school expenditure variables. The research employed Hierarchical Linear
Modeling (HLM) which has student level predictors and school level predictors. The adoption of
HLM enabled the research to look at a more complete picture of the education production
process as it happens with students and in schools.
To follow the research method discussion in the previous chapter, this chapter carries out
as the following: the first section described the processes of cleaning up the dataset, including
selecting variables, coding and recoding values, and checking missing values. The second
section focused on the procedures of creating factors through a set of factor analyses. The final
section rested on the HLM analysis with a series of different models which center around the
different outcome measurements.
Data Description
Data for this dissertation came from the Education Longitudinal Study of 2002
(ELS:2002) restricted version. The difference between the restricted and publicly available
datasets is that the restricted version has more extensive race/ethnicity categorization. Variables
used in this study, such as student/teacher ratio, percent free and reduced-price lunch, and
number FTE teachers are only available from the restricted-use dataset. One of the apparent
advantages is that the restricted dataset contains school resources, student enrollment, and school
69
poverty information which were drawn directly from the Common Core of Data (CCD).
Therefore there is no need to merge the ELS:2002 with CCD to get the necessary variables.
Education Longitudinal Study currently has three waves in 2002, 2004, and 2006,
respectively. Some of the outcome measurements in this dissertation are taken from the 2004
follow-up (F1) and 2006 follow-up (F2) studies. These variables are the NAEP-equated
ELS:2002 math score (F1), GPA for all academic courses (F1), student ever dropped out (F2),
and highest level of education attempted (F2). Most of the independent variables are drawn from
the base year study. The sample weight variable is the student expanded sample weight from the
base year restricted dataset. This variable has a more accurate reflection of the entire student
population then the regular sample weight from the public dataset, although these two weight
variables do not show significant difference statistically. The adoption of the restricted weight
variable enable the dataset to represent the actual student population nationally.
The data for the present study has a sample of about 16,000 students. These students are
nested in more than 700 schools, which have an average of 23 students per school. The average
student number is usable, and the school number is big enough for multilevel models (Bickel,
2007). Due to the nature of the survey, there are 5%-30% missing values for the variables this
dissertation uses. During the data preparation stage with SPSS software, the statistical procedures
choose pairwise technique whenever possible to maximize the number of students. At the stage
of factor analysis, some cases are dropped because of the missing value, and basic statistical tests
are adopted to show that the cases in use and the cases excluded from the factor analysis do show
significant difference in terms of mean value and standard deviation. Linear trend at a point
imputation technique is used to replace missing values so that the new factors are statistically
alike to the original ones.
70
Factor Analysis
As Meyers, Gamst, and Guarino (2006) pointed out, factor analysis has two distinct
advantages. The usage of factor analysis can reduce a relatively large number of variables to a
much smaller number of factors. The final factors, through appropriate factor analysis
procedures, are also the constructs that best represent the inventory of the set of variables.
Furthermore, factor analysis is a useful tool for researchers to organize or conceptualize a set of
measures. The final version of factors should have meanings that best represent the research
purposes.
Factor analysis separates the total variance of each variable into two parts. The first part
is the common or shared variance with other variables in the analysis. The second part is the
variance unique to the variable itself. During the extraction process, factor analysis adopts only
the commonly shared variance. Multiple factors constitute a factor matrix, which displays how
much each variable has its variance loaded on the factors. A good measure of the magnitudes of
factors extracted from the process is the amount of common variance each factor accounts for.
Each variable has a portion of variance captured by the factor. This portion of variance is the
loading score of a factor matrix (Kim & Mueller, 1979). The dissertation uses the Principal Axis
Factoring method in SPSS to create all the four groups of factors.
There are a number of criteria to gauge the quality of factors produced by factor analysis.
The first criterion is the problem of multicollinearity, which is tested by looking at the initial
communalities. The rule of thumb is to make sure all the initial communalities are all less than
0.8 to clear the potential multicollinearity problem (Kim & Mueller, 1979). The second criterion
is the problem of outlier, which is also checked from the initial communalities. The rule of
thumb is that none of the initial communalities is less than 0.1 to clear the potential outlier
71
problem (Kim & Mueller, 1979). The third test is the factorability of a set of variables. This test
is to check the correlation matrix and if some of the correlations are greater than 0.3, then the
factor analysis is a good method for this set of variables (Kim & Mueller).
Bartlett's Test of Sphericity is an important test in factor analysis. It calculates the
determinate of the matrix of the sums of products and cross-products from which the inter-
correlation matrix is derived. The determinant of the matrix is converted to a chi-square statistic
and tested for significance (Kim & Mueller, 1979). The null hypothesis is that the inter-
correlation matrix comes from a population in which the variables are non-collinear (i.e. an
identity matrix, which is a matrix repeats itself diagonally). If the significance level is less than
0.05, the null hypothesis is rejected (Kim & Mueller).
It is also important to check the Kaiser-Meyer-Olkin (KMO) test for the final factor(s). It
measures the degree of common variance among the variables. Usually, the closer the number is
to one, the better the factor(s) may be. In other words, the factors extracted account for a
substantial amount of variance when KMO is high (Kim & Mueller, 1979). The last criterion is
to check the final factor matrix and examine the factor loadings of each of the variables included
in the factor. The factor loading is the portion of variance the factor captures from each of the
variables. If the loading is greater than 0.5, that means the variable is properly loaded onto the
factor. Otherwise the variable should be dropped from the factor. If there are multiple factors
extracted, it is suggested to go with the factor that has the strongest loading (Kim & Mueller,
1979).
This dissertation included four sets of factor analysis. The target factors were in the area
of expectation on learning, motivation on learning, attitude on learning, and parental
involvement on student learning. Each factor analysis used about 15 variables, and only the
72
variables with a factor loading above 0.45 were kept in the factor. The following sections are
factor analysis processes carried out exactly according to the six criteria discussed above.
Meyers, Gamst, and Guarino (2006) mention that no factor should have fewer than three
variables, and it is not the case here for the four sets of factor analysis.
Table 4 shows the results of the first set of factor analysis on student expectation on
learning. There were two factors formulated based on the variables used. Comparing the two
factors, the first one captured much more variable information in terms of factor loadings.
Therefore, the final factor for expectation was only the first factor.
Table 4
Summary of Variables and Factor Loadings matrix of FEXP
Variables Used in Student Educational
Expectation Factor (FEXP)
Factor Loadings Initial
Communalities 1 2
Can do excellent job on math tests .656 -.487 .670
Can understand difficult math texts .653 -.476 .708
Can understand difficult English texts .635 .412 .635
Can learn something really hard .684 .039 .496
Can understand difficult English class .705 .459 .700
Remembers most important things when studies .654 .175 .473
Can do excellent job on English assignments .725 .485 .724
Can do excellent job on English tests .719 .470 .724
Can understand difficult math class .708 -.482 .717
Can master skills in English class .729 .445 .699
Can get no bad grades if decides to .683 .022 .499
Can get no problems wrong if decides to .679 -.134 .485
Can do excellent job on math assignments .746 -.477 .744
Can learn something well if wants to .752 .009 .602
Can master math class skills .752 -.457 .739
Extraction Method: Principal Axis Factoring. 2 factors extracted. 5 iterations required.
73
Table 5 shows the results of the second set of factor analysis on student motivation on
learning. There were three factors formulated based on the variables used. Comparing the three
factors, the first one captured much more variable information in terms of factor loadings.
Therefore, the final factor for motivation is only the first factor.
Table 5
Summary of Variables and Factor Loadings matrix of FMOV
Variables Used in Student Educational
Motivation Factor (FMOV)
Factor Loadings Initial
Communalities 1 2 3
How much likes school .656 -.487 .318 .318
Classes are interesting and challenging .653 -.476 .470 .470
Satisfied by doing what expected in class .635 .412 .470 .470
Education is important to get a job later .684 .039 .276 .276
Importance of good grades to student .705 .459 .398 .398
Importance of getting good education .654 .175 .303 .303
Gets totally absorbed in mathematics .725 .485 .225 .225
Mathematics is important .719 .470 .289 .289
Studies to get a good grade .708 -.482 .497 .497
Studies to increase job opportunities .729 .445 .581 .581
Studies to ensure financial security .683 .022 .534 .534
Extraction Method: Principal Axis Factoring. 3 factors extracted. 23 iterations required.
Table 6 shows the results of the second set of factor analysis on student attitude on
learning. There were four factors formulated based on the variables used. Comparing the three
factors, the first one captured much more variable information in terms of factor loadings.
Therefore, the final factor for attitude was only the first factor.
74
Table 6
Summary of Variables and Factor Loadings matrix of FATD
Variables Used in Student Attitude Toward
Learning Factor (FATD)
Factor Loadings Initial
Communalities 1 2 3 4
How many times late for school .496 -.036 .442 .070 .496
How many times cut/skip classes .478 -.058 .406 .085 .478
How many times got in trouble .467 -.006 .188 .301 .467
How often student completes homework (English) .710 .257 -.144 -.169 .710
How often student completes homework (math) .704 -.328 -.208 -.116 .704
How often student is absent (English) .525 .151 .122 -.365 .525
How often student is absent (math) .511 -.207 .121 -.293 .511
How often student is tardy (English) .561 .248 .090 -.020 .561
How often student is tardy (math) .560 -.170 .078 .045 .560
How often student is attentive in class (English) .666 .395 -.189 -.053 .666
How often student is attentive in class (math) .688 -.380 -.253 .005 .688
How often student is disruptive in class (English) .540 .301 -.140 .315 .540
How often student is disruptive in class (math) .543 -.156 -.177 .323 .543
Extraction Method: Principal Axis Factoring. 4 factors extracted. 9 iterations required.
Table 7 shows the results of the second set of factor analysis on student attitude on
learning. There were just one factor formulated based on the variables used. The factor loadings
were relatively high comparing with 0.45. This factor was also the final factor for parental
involvement on student learning.
Table 7
Summary of Variables and Factor Loadings matrix of FPAR
Variables Used in Student Parental School Involvement
Factor (FPAR)
Factor Loading Initial
Communalities 1
How often discussed school courses with parents .714 .461
How often discussed school activities with parents .705 .457
How often discuss things studied in class with parents .740 .475
How often discussed grades with parents .647 .383
How often discussed prep for ACT/SAT with parents .594 .331
75
How often discussed going to college with parents .687 .429
How often discussed current events with parents .595 .313
How often discussed troubling things with parents .561 .283
Extraction Method: Principal Axis Factoring. 1 factors extracted. 4 iterations required.
For all the four factors, the initial communalities show that there was no multicollinearity
problem or outlier problem. All the four sets of the factor analyses were factorable, which meant
that the correlation matrix had at least one correlation that is greater than 0.3. The results of the
Bartlett's Test of Sphericity and the Kaiser-Meyer-Olkin test were shown in Table 8. The results
showed that these factor analyses were valid and statistically solid.
Table 8
Results of Bartlett's and Kaiser-Meyer-Olkin Tests
Factor Bartlett's KMO
FEXP 0.000 0.938
FMOV 0.000 0.850
FATD 0.000 0.866
FPAR 0.000 0.899
Treating Missing Values
Pairwise case selection is adopted to maximize the most number of cases in the process
of factor analysis. The overall cases left in the factors are shown in Table 9.
Table 9
Case Numbers of Factors (rounded to closest hundreds)
Factor FEXP FMOV FATD FPAR
Valid Cases 10000 9900 9200 12000
The case numbers indicate that there are significant amount of cases left out of the
factoring process. To make sure the cases used and the cases not used do not have statistical
76
difference, descriptive analysis of these cases against the other variables, and T-test are
performed. The results are shown in Table 10 – Table 13.
Table 10
Factor Analysis Missing Values Check (FEXP) (rounded to closest hundreds)
Variables N (In) N (Out)
Mean (In)
Mean (Out)
SD (In) SD (Out)
T-Test
Gender 10000 5400 0.53 0.46 0.499 0.498 0.000 Race 10000 5300 6.34 5.72 2.294 2.376 0.000
Total years teaching/K-12 7300 3700 14.67 14.38 7.858 7.831 0.064
Highest degree earned by the teacher
7400 3800 3.99 3.98 0.772 0.785 0.767
Type of certification held 7300 3700 4.55 4.5 0.913 0.958 0.017
Standardized test math and reading
10000 6000 52.00 48.39 9.717 9.737 0.000
NAEP-equated ELS:2002 math score
8500 5100 156.89 145.99 32.512 34.255 0.000
GPA for all academic courses
8100 4800 14.03 13.32 28.125 27.415 0.158
Most recent SAT composite score
3900 1900 1051.66 997.31 200.662 218.735 0.000
Table 11
Factor Analysis Missing Values Check (FMOV) (rounded to closest hundreds)
Variables N (In) N (Out)
Mean (In)
Mean (Out)
SD (In) SD (Out)
T-Test
Gender 10000 5400 0.53 0.46 0.499 0.498 0.000 Race 10000 5300 6.34 5.72 2.294 2.376 0.000
Total years teaching/K-12 7300 3700 14.67 14.38 7.858 7.831 0.037
Highest degree earned by the teacher
7400 3800 3.99 3.98 0.772 0.785 0.234
Type of certification held 7300 3700 4.55 4.5 0.913 0.958 0.001
Standardized test math and reading
10000 5900 52.00 48.39 9.717 9.737 0.000
NAEP-equated ELS:2002 math score
8500 5100 156.89 145.99 32.512 34.255 0.000
GPA for all academic 8100 4800 14.03 13.32 28.125 27.415 0.251
77
courses
Most recent SAT composite score
3900 1900 1051.66 997.31 200.662 218.735 0.000
Table 12
Factor Analysis Missing Values Check (FATD) (rounded to closest hundreds)
Variables N (In) N
(Out)
Mean
(In)
Mean
(Out)
SD (In) SD
(Out)
T-Test
Gender 10000 5400 0.53 0.46 0.499 0.498 0.226
Race 10000 5300 6.34 5.72 2.294 2.376 0.000
Total years teaching/K-12 7300 3700 14.67 14.38 7.858 7.831 0.968
Highest degree earned by the teacher
7400 3800 3.99 3.98 0.772 0.785 0.654
Type of certification held 7300 3700 4.55 4.5 0.913 0.958 0.000
Standardized test math and reading
10000 5900 52.00 48.39 9.717 9.737 0.000
NAEP-equated ELS:2002 math score
8500 5100 156.89 145.99 32.512 34.255 0.000
GPA for all academic courses
8100 4800 14.03 13.32 28.125 27.415 0.001
Most recent SAT composite score
3900 1900 1051.66 997.31 200.662 218.735 0.000
Table 13
Factor Analysis Missing Values Check (FPAR) (rounded to closest hundreds)
Variables N (In) N
(Out)
Mean
(In)
Mean
(Out)
SD (In) SD
(Out)
T-Test
Gender 10000 5400 0.53 0.46 0.499 0.498 0.000
Race 10000 5300 6.34 5.72 2.294 2.376 0.000
Total years teaching/K-12 7300 3700 14.67 14.38 7.858 7.831 0.000
Highest degree earned by the teacher
7400 3800 3.99 3.98 0.772 0.785 0.395
Type of certification held 7300 3700 4.55 4.5 0.913 0.958 0.019
Standardized test math and reading
10000 6000 52.00 48.39 9.717 9.737 0.000
NAEP-equated ELS:2002 math score
8500 5100 156.89 145.99 32.512 34.255 0.000
GPA for all academic courses
8100 4800 14.03 13.32 28.125 27.415 0.055
78
Most recent SAT composite score
3900 1900 1051.66 997.31 200.662 218.735 0.000
These tables show that missing values caused about 30% – 35% of cases to be excluded
from the four factors created by factor analysis. The results of these tests, especially the T-test
showed that there were significant differences between the cases in the four factors and the cases
out of the four factors. In other words, the cases in and out of the factors had different means and
standard deviations, according to some of the variables used in the analysis. This could be a
potential bias to the reliability of these factors.
To improve the quality of these four factors, the problem of missing values needed to be
addressed. The easiest method to fix missing values is the deletion method, which included
listwise deletion and pairwise deletion. The author viewed deletion as the last resort of treating
missing values; therefore, this dissertation did not use either of them to fix the missing value
problems for the factors.
Another strategy to solve the missing value problem is to use the single imputation
technique, which replaces the missing value with something else. McKnight, McKnight, Sidani,
and Figuerudo (2007) discussed three types data replacement: constant, random, and nonrandom.
The first type may include the mean or median values of the variable or even zero. Random
replacement is related with a random function. Nonrandom replacement brings in part or all of
the other cases in the variable to be fixed. The missing value can be replaced by a mathematical
calculation of adjacent values or by a regression model using all of the non-missing values. This
dissertation adopts the regression replacement method to improve the missing value conditions
of the factors.
SPSS offers a number of single imputation strategies to replace missing values. The
series mean strategy replaces missing values with the mean of the entire variable. The mean of
79
nearby points replaces missing values with the mean of valid surrounding values, which are the
valid values above and below the missing value. Similarly, median of nearby points replaces
missing values with the median of valid surrounding values. Linear interpolation replaces
missing values using a linear interpolation that is based on the last valid value before the missing
value and the first valid value after the missing value. Linear Trend at Point (LTP) method
replaces missing values with the linear trend for that point based on the regression of the entire
series. The missing values are replaced with their predicted values.2
The first four strategies rely heavily on either the mean value of some or part of the data,
or the median of part of the data. The LTP method projects the missing value be regression and
on the entire variable. Therefore, the LTP method is adopted to fill in the missing values in this
study. The new factors with no missing values and the original factors are compared by ANOVA
test. The results are shown in Table 14.
Table 14
ANOVA test of Factors with and without Missing Values
Factor F Value Significance
FEXP 0.054 0.947
FMOV 0.154 0.858
FATD 0.000 0.995
FPAR 1.028 0.358
These results indicated that the four new factors were statistically inseparable from the
four original factors. The new factors still had the identities of valid factor analysis but were
immune from the missing value problem.
2 Source: SPSS Help Function. Topic: Estimation Methods for Replacing Missing Values.
80
The technique to handle missing value problems employed in this dissertation was by no
means the most advanced. Besides single imputation method which was used in this study and
deletion method discussed briefly in the previous section, there were also model-based
procedures to correct missing value. According to McKnight et al. (2007), model-based
procedures include maximum likelihood, expectation maximization, Markov chain Monte Carlo,
and other adjustments. These procedures are usually derived from underlying distribution,
probability, or theoretical models; and these procedures treat the missing data as if they can be
observed to produce robust parameter estimates (McKnight et al., 2007). The model-based
procedures are complex in nature, and some techniques can be quite cumbersome for analyses
using multiple iterations. Therefore, this study did not adopt these procedures.
Another option is the multiple imputation (MI), which is available from SAS software.
Although this analysis did not use this method, but considering the importance of its application,
some of the usages from multiple imputations to deal with missing values were explained.
Multiple imputation (MI) on the other hand, is viewed as the “gold standard” of current
research (Treiman, 2009). The critical feature of MI is the ability to estimate the influence of the
missing data on parameter estimation (McKnight et al., 2007). Proposed by Rubin (1977), MI is
a method of generating multiple simulated values for each incomplete dataset, and then
iteratively analyzing datasets with each simulated value substituted in turn. The purpose is to
generate estimates that better reflect true variability and uncertainty in the data than do
regression methods. MI procedure replaces each missing value with a set of plausible values that
represent the uncertainty about the right value to impute. These multiple imputed data sets are
then analyzed by using standard procedures for complete data and combining the results from
these analyses (Yuan, 2000).
81
Variables
To fulfill the research purposes, the data were used to construct a comprehensive set of
individual-level and school-level variables to measure the various effects on student
achievement. The way most of the variables in ELS:2002 were coded are not directly useable for
dissertation analysis; therefore, additional steps were needed to clean up the raw data. These
additional steps included adjusting missing values, reversing values to proper order, and
conducting factor analysis to create necessary factors.
Student-level variables were shown in Table 15. There were three types of variables:
continuous variables, dichotomous variable, and factors. Several classes of student-level
variables were constructed. The first was demographic including gender, race, and
socioeconomic status (SES) variables. The SES variable was a composite measure developed by
NCES that captures father’s education level, mother’s education level, family income, father’s
job type, and mother’s job type3.
The next class of factors was created through factor analysis (See factor analysis section
for the detailed procedures and lists of variables used to generate the factors). There were four
type of student learning behavior factors. The first factor was the student educational expectation
(FEXP) that measured how much students want to pursue higher level of education. The second
factor was the student educational motivation (FMOV) that measures how motivated students
were to learn. The third factor was the student attitude toward learning (FATD) that measures
how much effort students put in learning. The last factor was the parental school involvement
(FPAR) that measured the degree of involvement of parents to students learning. The difference
between this factor and the SES composite variable was that the SES was about the financial
3 Source: ELS Electronic Code Book variable description.
82
capital and educational background of the family, while the FPAR was about the behavior of
parents on student learning. The inclusion of FPAR intends to covered a broader range of family
characteristics.
The third class of variables was output measures. They included standardized test
composite math and reading, NAEP-equated ELS:2002 math score, GPA for all academic
courses, and most recent SAT composite score. Each of them became a separate dependent
variable in the later part of the analysis. Overall, the school-level variables showed median range
variance.
Table 15
Means, Standard Deviations, and Descriptions of Student-Level Variables
Variable Mean SD Type Description from ELS:2002
Demographic Variables
GENDR 0.50 0.500 D Female = 1
ASIAN 0.09 0.289 D Asian = 1
BLACK 0.13 0.339 D Black = 1
HISPN 0.15 0.353 D Hispanic = 1
WHITE 0.17 0.495 D White = 1
SESC 0.42 0.743 C Socioeconomic Status Composite
Student Learning Behavior Factors (missing value treated)
FEXP 0.02 0.768 F Student Educational Expectation Factor *
FMOV 0.00 0.741 F Student Educational Motivation Factor *
FATD 0.05 0.703 F Student Attitude Toward Learning Factor *
FPAR 0.01 0.802 F Parental School Involvement Factor *
Student Academic Achievement Variables
TCMR 50.66 9.880 C Standardized test composite math and reading
NAEP 152.81 33.591 C NAEP-equated ELS:2002 math score (F1)
GPA 2.57 0.835 C GPA for all academic courses (F1)
SATC 1034.2 208.161 C Most recent SAT composite score
C = Continuous Variable, D = Dichotomous Variable, and F = Factor * See Table 4 - 7 for the lists of variables used in the factor analysis
School-level variables are shown in Table 16. There were three types of variables:
continuous variables, dichotomous variable, and factors. Several classes of school-level variables
83
were constructed. The first was student composite which were the mean values of student-level
variables: Mean Socioeconomic Status Composite (MSESC), Mean Student Educational
Expectation Factor (MFEXP), Mean Student Educational Motivation Factor (MFMOV), Mean
Student Attitude Toward Learning Factor (MFATD), and Mean Parental School Involvement
Factor (MFPAR).
The next class of variables was measures of teacher characteristics. They were the mean
total years teaching/K-12, the mean type of certification held, and the mean highest degree
earned by the teacher. These variables were also proxies for part of school expenditures.
The last class of variable is school structural characteristics which included school type
(public or private), school urbanicity (urban or rural), total school enrollment, percent minority,
student/teacher ratio, % full-time certified teachers, percent free and reduced-price lunch. Among
these, student/teacher ratio, % full-time certified teachers were two of the school expenditure
proxies. All the school-level variables were included in the second level model of the HLM
analysis (see HLM section for more technical details).
Table 16
Means, Standard Deviations, and Descriptions of School-Level Variables
Variable Mean SD Type Description from ELS:2002
Student Composites
MSESC 0.04 0.434 C Mean Socioeconomic Status Composite
MFEXP 0.02 0.227 F Mean Student Educational Expectation Factor
MFMOV 0.01 0.212 F Mean Student Educational Motivation Factor
MFATD 0.05 0.245 F Mean Student Attitude Toward Learning Factor
MFPAR 0.01 0.255 F Mean Parental School Involvement Factor
Teacher Characteristics
MTEXP 14.60 5.661 C Mean total years teaching/K-12
MTCRT 4.51 0.824 C Mean type of certification held
MTDEG 4.00 0.601 C Mean highest degree earned by the teacher
Structural Characteristics
SURB 0.33 0.471 D School urbanicity (Urban = 1)
84
SENR 1245.86 846.904 C Total school enrollment
SPMN 34.05 31.471 C Percent minority
SSTR 16.46 4.367 C Student/teacher ratio
SPTC 91.21 19.917 C % full-time teachers are certified
SPFL 24.91 19.462 C Percent free and reduced-price lunch
C = Continuous Variable, D = Dichotomous Variable, and F = Factor
Hierarchical Linear Modeling
Schreiber and Griffin (2004) reviewed academic articles using multilevel modeling in
The Journal of Educational Review for 10 years (1992-2002). They generalized a common
framework of carrying out multilevel modeling and a standardized reporting style. This
dissertation referred to their modeling and reporting approaches in the HLM stage. Schreiber and
Griffin (2004) introduced three models of HLM. The first model is the basic model with no
predictors in either level. It is also referred as the one-way ANOVA model or the “fully
unconditional model” (Raudenbush & Bryk, 2002). This model provides the critical information
of whether multilevel research is necessary by checking the Intraclass Correlation Coefficient
(ICC). “ICC measures the proportion of the total variance that occurs systematically between
groups (Raudenbush & Bryk, 2002, p. 36).” Multi-level methods need to be used when the ICC
is greater than 10%, which means there is more than 10% of the total variance coming from
between groups (Lee, 2000). Later in this analysis the results demonstrate that across all four
outcome measures, each of them satisfies the threshold of ICC, in that more than 10% of the total
variation is from the school level (see Figure 8 for details).
Referred to as the means-as-outcome model (Raudenbush & Bryk, 2002), model 2 has
only school-level predictors and leaves the student level empty. The purposes of this model is to
test whether controlling for school-level variables, ICC shows any change. The anticipated
change is for ICC to decrease, which means the school-level predictors are capturing significant
variances in explaining the outcome variables.
85
Although not included by Schreiber and Griffin (2004), this dissertation adds a third
model for each of the outcome variables. Model 3 is also called the random coefficient model
(Raudenbush & Bryk, 2002). This model only has student-level variables to test the effects of the
individual predictors on the outcome measures.
The final model is an intercepts & slopes-as-outcome model (Raudenbush & Bryk, 2002).
Model 4 contains both student-level and school-level predictors. The four models altogether not
only show the proportion of variance on each level, but also show the change in the explanatory
power of model overall as more variables are added to the models. The following sections are
description of the 4 models by each outcome measures: math and reading test score, NAEP
score, high school GPA, and SAT score.
Math and Reading Test Score
One-way ANOVA Model (Model I)
Model I is a one-way ANOVA model. There is no predictor at both student level and
school level. The dependent variable is math and reading standardized test score composite. This
test score is a norm-referenced measurement of achievement based on the estimate of
achievement relative to the population4. This model is to partition the total variance in the
student achievement into within and between school components, and to see if there is
significant difference in test scores among schools.
The results show that the grand mean student achievement is 50.06. The reliability of the
sample means for the true school mean is 0.855, which indicates that the sample means are quite
reliable as indicators of the true school mean.
4 Source: ELS Electronic Code Book variable description.
86
Results also show that the variability exist across schools (23.33) is statistically
significant at 0.000 level, so that the null hypothesis that the variance across schools equals 0 is
rejected. The intraclass correlation coefficient ρ = τ00 / (τ00 + σ2) = 23.33 / (23.33 + 77.03) =
0.23. This suggests that 23% of the total variance in math and reading standardized test score
exists across schools. Since this number is greater than 0.1, the dataset qualifies for multilevel
analysis. In other words, the results of this empty model show that math and reading test score
has enough variability at the student and school levels for further analysis with HLM.
Means-as-outcome Model (Model II)
Model II is means-as-outcome model. To account for the variance across schools, the
Mean Socio-Economic Status (MSES), the Mean Student Expectation (MFEXP), the Mean
Motivation (MFMOV), the Mean Attitude (MFATD), Mean Parent Involvement (MFPAR),
Mean Teacher Experience (MTEXP), Mean Teacher Certification (MTCRT), Mean Teacher
Highest Degree (MTDEG), School Urbanicity (SURB), School Enrollment (SENR), School
Percent Minority (SPMN), School Student-Teacher Ratio (SSTR), School Percent Teacher
Certified (SPTC), and School Percentage of Student Receive Free and reduced-Price Lunch
(SPFL) variables are included in the level 2 model. All these school-level variables are grand
mean centered. The results show that the grand mean of math and reading standardized test score
composite is 49.37, which is relatively the same as in Model I.
MSES, MFEXP, and MFATD all show significant effects on school mean test scores at
.000 level. MFMOV and MFPAR show insignificant effect on school mean test scores. These
results mean that higher mean SES, educational expectation, and attitude toward learning, tend to
have higher level of mean test score composite. Among the teacher characteristics and school
characteristics variables, only the SPMN and SPFL show significant impact to student
87
achievement. These relationships are both negative meaning that the higher the percentages of
minority and free and reduced-price lunch students in the school, the lower the students score in
the tests.
The reliability 0.523 is a conditional reliability with which we can discriminate among
schools with the same mean predictors at the school level. Controlling these school level
variables, the variance in mean student achievement among schools decreases dramatically,
which is also suggested by the decreased intraclass correlation coefficient ρ = τ00 / (τ00 + σ2) =
4.08 / (4.08 + 79.22) = 0.05 (was 0.23 in Model 1). Comparing with the one-way ANOVA
model, adding school-lever predictors decreases the variance among schools from 23.33 to 4.08.
In other words, 82.5% of the variance in mean math and reading standardized test score
composite between schools is explained by the school-level variables {[τ00(ANOVA) -
τ00(current)] / τ00(ANOVA) = (23.33 – 4.08) / 23.33 = 0.825}. Overall, Model II has more
predicting power then Model I at school level.
Random Coefficient (RC) Model (Model III)
Model III is a random coefficient model. In this model, students’ Socio-Economic Status
Composite (SESC), Expectation Factor (FEXP), Motivation Factor (FMOV), Attitude Factor
(FATD), Parent Involvement Factor (FPAR), Student Gender (GENDR), and racial variables are
included as predictors at student level. All these variables are group centered. There is no school-
level predictor in this model. The average intercept is quite reliable with reliability at 0.878.
The grand mean math and reading standardized test score composite is a bit higher at
50.08. All the student-level predictors, except for the gender variable, are significant at 0.000
level. The results show that, on average, students who have higher SES tend to do better in the
tests. Students who have higher expectations of their education and higher attitude tend to have
88
higher test scores. However, surprisingly, the results also show that students who have stronger
motivations of education tend to have lower test scores. The parent involvement factor is
significant too. White and Asian students tend to have better test scores than Black and Hispanic
students. Overall, Black students have significantly lower test scores than others. Student SES,
expectation, and attitude all show very strong positive impact on test scores.
The inclusion of student level variables decreases variances at student level from 77.03 to
52.21. That is, 33.2% of the variance at student level is explained by these student level variables
altogether {[σ2 (ANOVA) - σ2 (current)] / σ2 (ANOVA) = (77.03 – 52.21) / 77.03 = 0.332}.
Comparing the proportions of variances explained by these variables at student level (33.2%) in
the current model and at school level (82.5%) in the means-as-outcome model, it seems these
variables have stronger effects at school level than at student level. The mean math and reading
standardized test score composite still varies significantly across schools at 0.000 level.
The random part of the student level model as a whole shows that mean levels of math
and reading scores still vary across schools after controlling for the ten student-level variables.
Although insignificant, the random part of gender variable does indicate heterogeneity of
regression slopes exists for gender. Similarly, regression slopes vary across school for Hispanic
and all the student behavior variables.
Intercepts & Slopes-as-outcome Model (Model IV)
Model IV is an intercepts & slopes-as-outcome model. In this model, school level
predictors MSES, MFEXP, MFMOV, MFATD, MFPAR, MTEXP, MTCRT, MTDEG, SURB,
SENR, SPMN, SSTR, SPTC, and SPFL are included. The conditional reliability of the average
intercept is 0.595, and the conditional reliability of the other slopes is around 0.10. This number
is low because these predictors as a whole have explained much variance in the achievement
89
slope, making it more difficult to detect additional variance; hence the individual slopes are not
very high. Student level predictors SESC, FEXP, FMOV, FATD, FPAR, GENDR, and four
racial variables are included.
Overall the results reflect the means-as-outcome model and the random coefficient model
in terms of the significance of school-level and student-level predictors. On the school level, the
results show that that the grand mean math and reading standardized test score composite is
49.42. Schools with higher mean SES tend to have higher level of student achievement (γ01
=6.33). Students in schools where there are higher mean learning expectations tend to have
higher level of achievement (γ02 =4.04). Students in schools of higher mean learning attitude tend
to show higher level of learning outcome (γ04 =4.26). Students in schools with higher percentage
minority students tend to have lower level of student achievement, but the relationship is weak
(γ011 = – 0.04). Schools with higher percentage of free and reduced-price lunch tend to have a
lower level of student achievement, but again the relationship is weak (γ014 = – 0.03). The other
school level predictors do not present statistical significance.
On the student level, the Asian Student (ASIAN) variable turns to be insignificant. White
Student (WHITE), Black Student (BLACK), and Hispanic Student (HISPN) variables remain
significant, and the directions of the relationships are the same as in the random coefficient
model. SESC, FEXP, FMOV, FATD, and FPAR variables are all significant. The school-level
composites under each student-level variable in general show little impact since very few of
them are statistically significant. The heterogeneity of regression slopes still exists for the
student-level variables, although the scale decreases for most of these variables. This is an
indication that certain amount of variance picked up by the decomposition of school-level
predictors.
90
Since there are two levels of inferences, student-level variables are modeled with the
school-level variables in Model IV. Eight variables – Black student, Hispanic student, White
student, expectation, motivation, attitude, and parental involvement – were observed to vary
among schools, and because of this variability, these eight variables can be modeled with school-
level variables. This decomposition means that each randomly varying coefficient at student-
level becomes a model. The school-level variables that are observed to be significantly related to
the random coefficients, or in other words, some school-level variables show influences to
student-level slopes.
Among the eight significant student-level variables, five of them were observed to have
significant school-level predictors. Take Black student variable for example, school mean
expectation and mean teacher experience accounted some of the variability in BLACK slope.
The result implies that, on average, the impact of Black students on math and reading scores
increases from negative to positive if the students are in schools where students have higher
expectations and teachers have a bit more experience, although the impact from mean teacher
experience is smaller than that of student expectation. Figure 2 illustrates the fact that although
being a Black student tends to have a negative impact on achievement, however, Black students
in a school of high education expectation usually have better chances of learn more.
91
Figure 2. Math and reading scores vs. Black plot with focus of mean expectation
Also, the impact that student expectation has toward achievement tends to drop a little if
students are in schools where more teachers have certification, while the impact may increase a
little if students are in schools where there are more minority students. Among all the student-
level variables that are modeled by school-level variables, BLACK shows the strongest effect
from mean expectation variable. Examining the random effects, 14.4% the proportion of
variance of the BLACK coefficient is explained by school mean expectation {τ03 (Model III) -
τ03 (Model IV) / τ03 (Model IV) = (8.83 – 7.56) / 7.56 = 0.144}.
The school level predictors as a whole explain 79.9% of the variance in student math and
reading standardized test score composite among schools {[τ00(RC) - τ00(current)] / τ00(RC) =
(24.02 – 4.82) / 24.02 = 0.799}. All the results from the four models are shown in Table 17.
92
Table 17
Results for Four Model (Standard Coefficient) on Student Math, Reading Test Score
I II III IV Fixed Effects School Mean, γ00 50.06*** 50.74*** 50.08*** 49.42*** School-level Variables Mean SES, γ01 6.24*** 6.33*** Mean Expectation, γ02 4.14*** 4.04*** Mean Motivation, γ03 -0.71 -0.26 Mean Attitude, γ04 3.97*** 4.26*** Mean Parent Involvement, γ05 -0.53 0.74 Mean Teacher Experience, γ06 0.02 0.03 Mean Teacher Certification, γ07 -0.05 -0.05 Mean Teacher Degree, γ08 0.16 0.08 School Urbanicity, γ09 -0.20 -0.22 School Enrollment, γ010 -0.00 -0.00 School % Minority, γ011 -0.05*** -0.04*** School Student/Teacher, γ012 -0.05 -0.06 School % Teacher Certified, γ013 -0.00 0.00 School Free Reduced Lunch, γ014 -0.03* -0.03 Student-level Variables Gender, γ10 -0.06 0.06 Asian Student, γ20 1.81*** 0.94 Black Student, γ30 -3.16*** -3.31*** Mean Expectation, γ32 6.66* Mean Teacher Experience, γ36 0.19* Hispanic Student, γ40 -1.98*** -1.92*** White Student, γ50 2.16*** 2.18*** SES, γ60 3.14*** 3.19*** Expectation, γ70 2.91*** 2.80*** Mean Teacher Certification, γ77 -0.93* School % Minority, γ79 0.75* Motivation, γ80 -0.92*** -0.92*** Mean Teacher Degree, γ88 -0.64* School Urbanicity, γ89 -1.14** Attitude, γ90 2.36*** 2.38*** School % Minority, γ911 -0.01* Parent Involvement, γ100 0.43*** 0.52*** Mean Expectation, γ102 -1.63*
Random Effects τ00 23.33*** 4.08*** 24.02*** 4.82*** τ01 1.81* 1.10* τ02 5.84 6.60 τ03 8.83 7.56 τ04 13.87* 11.53* τ05 11.81 13.54 τ06 2.27*** 1.65** τ07 1.28*** 1.18** τ08 1.96*** 2.17*** τ09 1.25** 0.73* τ10 1.04* 0.92* σ
2 77.03 79.22 52.21 52.78 ρ 0.23 0.05
93
*** p<0.000, ** p<0.01, * p<0.05; All school-level predictors are grand mean centered; All student-level predictors are group mean centered.
NAEP Scores
One-way ANOVA Model (Model I)
Model I is a one-way ANOVA model. There is no predictor at both student level and
school level. The dependent variable is student NAEP score. This test score information is from
the first follow-up survey in 2004. This model is to partition the total variance in the student
achievement into within and between school components, and to see if there is significant
difference in test scores among schools.
The results show that the grand mean student achievement is 149.78. The reliability of
the sample means for the true school mean is 0.817, which indicates that the sample means are
quite reliable as indicators of the true school mean.
Results also show that the variability exists across schools (236.56) is statistically
significant at 0.000 level, so that the null hypothesis that the variance across schools equals 0 is
rejected. The intraclass correlation coefficient ρ = τ00 / (τ00 + σ2) = 236.56 / (236.56+ 887.14) =
0.21. This suggests that 21% of the total variance in NAEP score exists across schools. Since this
number is greater than 0.1, the dataset qualifies for multilevel analysis. In other words, the
results of this empty model show that NAEP scores have enough variability at the student and
school levels for further analysis with HLM.
Means-as-outcome Model (Model II)
Model II is means-as-outcome model. To account for the variance across schools, the
Mean Socio-Economic Status (MSES), the Mean Student Expectation (MFEXP), the Mean
Motivation (MFMOV), the Mean Attitude (MFATD), Mean Parent Involvement (MFPAR),
94
Mean Teacher Experience (MTEXP), Mean Teacher Certification (MTCRT), Mean Teacher
Highest Degree (MTDEG), School Urbanicity (SURB), School Enrollment (SENR), School
Percent Minority (SPMN), School Student-Teacher Ratio (SSTR), School Percent Teacher
Certified (SPTC), and School Percentage of Student Receive Free and reduced-Price Lunch
(SPFL) variables are included in the level 2 model. All these school-level variables are grand
mean centered. The results show that the grand mean of NAEP score is 147.19, which is
relatively the same as in Model I.
MSES, MFEXP, and MFATD all show significant effects on school mean NAEP scores
at .000 level. MFMOV and MFPAR show insignificant effect on school mean test scores. These
results mean that higher mean SES, educational expectation, and attitude toward learning, tend to
have higher level of mean test score composite. Among the teacher characteristics and school
characteristics variables, only the SPMN and SPFL show significant impact to student
achievement. These relationships are both negative meaning that the higher the percentages of
minority and free and reduced-price lunch students in the school, the lower the students score in
the tests.
The reliability 0.473 is a conditional reliability with which we can discriminate among
schools with the same mean predictors at the school level. Controlling these school level
variables, the variance in mean student achievement among schools decreases dramatically,
which is also suggested by the decreased intraclass correlation coefficient ρ = τ00 / (τ00 + σ2) =
45.73 / (45.73 + 908.69) = 0.05 (was 0.21 in Model I). Comparing with the one-way ANOVA
model, adding school-lever predictors decreases the variance among schools from 236.56 to
45.73. In other words, 80.7% of the variance in mean NAEP score between schools is explained
95
by the school-level variables {[τ00(ANOVA) - τ00(current)] / τ00(ANOVA) = (236.56 – 45.73) /
236.56 = 0.807}. Overall, Model II has more explanatory power than Model I at school level.
Random Coefficient (RC) Model (Model III)
Model III is a random coefficient model. In this model, students’ Socio-Economic Status
Composite (SESC), Expectation Factor (FEXP), Motivation Factor (FMOV), Attitude Factor
(FATD), Parent Involvement Factor (FPAR), Student Gender (GENDR), and racial variables are
included as predictors at student level. All these variables are group centered. There is no school-
level predictor in this model. The average intercept is quite reliable with reliability at 0.854.
The grand mean NAEP score is a at 150.16. All the student-level predictors, except for
the parental involvement variable, are significant at 0.000 level. The results show that, on
average, students who have higher SES tend to do better in the tests. Students who have higher
expectations of their education and higher attitude tend to have higher test scores. However, the
results also show that students who have stronger motivations of education tend to have lower
test scores. White and Asian students tend to have better test scores than Black and Hispanic
students. Overall, Black students have significant lower NAEP scores than otherwise. Student
SES, expectation, and attitude all show very strong positive impact on NAEP scores.
The inclusion of student level variables decreases variances at student level from 887.14
to 589.26. That is, 33.6% of the variance at student level is explained by these student level
variables altogether {[σ2 (ANOVA) - σ2 (current)] / σ2 (ANOVA) = (887.14 – 589.26) / 887.14 =
0.336}. Comparing the proportions of variances explained by these variables at student level
(33.6%) in the current model and at school level (80.7%) in the means-as-outcome model, it
seems these variables have stronger effects at school level than at student level. The mean NAEP
score still varies significantly across schools at 0.000 level.
96
The random part of the student level model as a whole shows that mean levels of NAEP
scores still vary across schools after controlling the 10 student-level variables. Although
insignificant, the random part of parental involvement variable does indicate heterogeneity of
regression slopes exists for parental involvement. Similarly, regression slopes vary across school
for HISPAN, WHITE, SES, motivation, and attitude variables.
Intercepts & Slopes-as-outcome Model (Model IV)
Model IV is an intercepts & slopes-as-outcome model. In this model, school level
predictors MSES, MFEXP, MFMOV, MFATD, MFPAR, MTEXP, MTCRT, MTDEG, SURB,
SENR, SPMN, SSTR, SPTC, and SPFL are included. The conditional reliability of average
intercept is 0.543, and the conditional reliability of the other slopes are around 0.10. This number
is low because these predictors as a whole have explained much variance in the achievement
slope, making it more difficult to detect additional variance; hence the individual slopes are not
very high. Student level predictors SESC, FEXP, FMOV, FATD, FPAR, GENDR, and four
racial variables are included.
Overall the results reflect the means-as-outcome model and the random coefficient model
in terms of the significance of school-level and student-level predictors. On the school level, the
results show that that the grand mean NAEP score is 147.73. Schools with higher mean SES tend
to have higher level of student achievement (γ01 =20.61). Students in schools where there are
higher mean learning expectation tend to have higher level of achievement (γ02 =13.92). Students
in schools of higher mean learning attitude tend to show higher level of learning outcome (γ04
=10.56). Schools with higher percentage of minority students tend to have lower level of student
achievement (γ011 = – 0.11). Similarly, schools with more free and reduced-price lunch students
97
tend to have lower level of student achievement (γ014 = – 0.14). The other school level predictors
are not statistically significant.
On the school level, only some of the variables are significant. Students in schools where
there is higher mean learning expectation tend to have a higher level of achievement (γ02 =4.04).
Students in schools of higher mean learning attitude tend to show higher level of learning
outcome (γ04 =4.26). In schools with higher percentage minority students, students on average
tend to have lower level of student achievement, although the relationship is weak (γ011 = – 0.04).
Schools with higher percentage of free and reduced-price lunch tend to have lower level of
student achievement, and the relationship is weak too (γ014 = – 0.03). The other school level
predictors do not show statistical significance.
On the student level, gender, the Asian Student (ASIAN), White Student (WHITE),
Black Student (BLACK), and Hispanic Student (HISPN) variables remain significant, and the
directions of the relationships are the same as in the random coefficient model. SESC, FEXP,
FMOV, and FATD variables are all significant. FPAR is not significant as shown in Model III.
The school-level composites under each student-level variables in general show little impact
since very few of them are statistically significant. The heterogeneity of regression slopes still
exist for the student-level variables, although the scale decreases for most of these variables. This
is an indication that certain amount of variance picked up by the decomposition of school-level
predictors.
Since there are two levels of inferences, student-level variables are modeled with the
school-level variables in Model IV. Nine out of the 10 variables–gender, Asian student, Black
student, Hispanic student, White student, SES, expectation, motivation, and attitude–were
observed to vary among schools, and because of this variability, these nine variables can be
98
modeled with school-level variables. This decomposition means that each randomly varying
coefficient at student-level becomes a model. The school-level variables that are observed to be
significantly related to the random coefficients, or in other words, some school-level variables
show influences to student-level slopes.
Among the nine significant student-level variables, five of them were observed to have
significant school-level predictors. For Asian, Black, Hispanic, and White students, school mean
expectation accounted a great amount of the variability in these racial variable slopes, and the
directions are positive. For Black and White students, school mean motivation accounted a great
amount of the variability in these racial variable slopes, and the directions are negative. Figure 3
illustrates that although Hispanic students overall have a negative impact on NAEP score,
attending a school with high educational expectation increases the chance of having better NAEP
scores than being in a school with low expectation.
Figure 3. NAEP scores vs. Hispanic Student plot with focus of mean expectation
99
Student SES can also be modeled be school urbanicity and student/teacher ratio,
however, the relationships are relatively small. Among all the student-level variables that are
modeled by school-level variables, school-level expectation shows the strongest effect toward
Hispanic student variable. Examining the random effects for Hispanic students, 18.8% the
proportion of variance of the HISPAN coefficient is explained by school mean expectation {τ03
(Model III) - τ03 (Model IV) / τ03 (Model IV) = (177.99 – 149.85) / 149.85 = 0.188}.
The school-level predictors as a whole explain 79.8% of the variance in student NAEP
score composite among schools {[τ00(RC) - τ00(current)] / τ00(RC) = (248.14 – 50.19) / 248.14 =
0.798}. This number is quite close to the number using math and reading scores as outcome
measure (79.9). All the results from the four models are shown in Table 18.
Table 18
Results for Four Model Estimates (Standard Coefficient) on Student NAEP scores
I II III IV Fixed Effects School Mean, γ00 149.78*** 147.19*** 150.16*** 147.73*** School-level Variables Mean SES, γ01 20.15*** 20.61*** Mean Expectation, γ02 13.75*** 13.92*** Mean Motivation, γ03 -5.36 -3.90 Mean Attitude, γ04 9.66*** 10.56*** Mean Parent Involvement, γ05 -2.11 -2.34 Mean Teacher Experience, γ06 0.06 0.09 Mean Teacher Certification, γ07 0.75 0.46 Mean Teacher Degree, γ08 0.52 0.18 School Urbanicity, γ09 -0.61 -0.51 School Enrollment, γ010 0.00 0.00 School % Minority, γ011 -0.13*** -0.11*** School Student/Teacher, γ012 -0.04 -0.07 School % Teacher Certified, γ013 -0.02 -0.04 School Free Reduced Lunch, γ014 -0.11* -0.14** Student-level Variables Gender, γ10 -5.07*** -4.74*** Asian Student, γ20 8.95*** 8.08** Mean Expectation, γ22 30.90* Black Student, γ30 -11.93*** -10.89*** Mean Expectation, γ32 38.82** Mean Motivation, γ33 -34.04* Hispanic Student, γ40 -5.89** -4.3**
100
Mean Expectation, γ42 46.13*** White Student, γ50 5.77*** 6.29** Mean Expectation, γ52 33.74** Mean Motivation, γ53 -34.88** SES, γ60 10.35*** 10.92*** School Urbanicity, γ69 -3.43* School Student/Teacher, γ612 -0.40* Expectation, γ70 9.78*** 9.49*** Motivation, γ80 -1.88*** -2.31*** Attitude, γ90 9.68*** 9.76*** Parent Involvement, γ100 0.35 0.59
Random Effects τ00 236.56*** 45.73*** 248.14*** 50.19*** τ01 33.84** 18.64*** τ02 122.96 107.18 τ03 129.99 126.40 τ04 177.99** 149.85** τ05 164.46* 166.92* τ06 33.96*** 25.82*** τ07 6.91 4.65 τ08 13.27* 19.08* τ09 16.40* 10.60* τ10 8.03* 6.93* σ
2 887.14 908.69 589.26 598.70 ρ 0.21 0.05
*** p<0.000, ** p<0.01, * p<0.05; All school-level predictors are grand mean centered; All student-level predictors are group mean centered.
SAT
One-way ANOVA Model (Model I)
Model I is a one-way ANOVA model. There is no predictor at both student level and
school level. The dependent variable is student SAT score. This model is to partition the total
variance in the student achievement into within and between school components, and to see if
there is significant difference in test scores among schools.
The results show that the grand mean student achievement is 981.77. The reliability of
the sample means for the true school mean is 0. 857, which indicates that the sample means are
quite reliable as indicators of the true school mean.
Results also show that the variability exists across schools (10526.85) is statistically
significant at 0.000 level, so that the null hypothesis that the variance across schools equals 0 is
101
rejected. The intraclass correlation coefficient ρ = τ00 / (τ00 + σ2) = 10526.85 / (10526.85 +
31899.76) = 0.25. This suggests that 25% of the total variance in SAT scores exists across
schools. Since this number is greater than 0.1, the dataset qualifies for multilevel analysis. In
other words, the results of this empty model show that SAT scores have enough variability at the
student and school levels for further analysis with HLM.
Means-as-outcome Model (Model II)
Model II is means-as-outcome model. To account for the variance across schools, the
Mean Socio-Economic Status (MSES), the Mean Student Expectation (MFEXP), the Mean
Motivation (MFMOV), the Mean Attitude (MFATD), Mean Parent Involvement (MFPAR),
Mean Teacher Experience (MTEXP), Mean Teacher Certification (MTCRT), Mean Teacher
Highest Degree (MTDEG), School Urbanicity (SURB), School Enrollment (SENR), School
Percent Minority (SPMN), School Student-Teacher Ratio (SSTR), School Percent Teacher
Certified (SPTC), and School Percentage of Student Receive Free and reduced-Price Lunch
(SPFL) variables are included in the level 2 model. All these school-level variables are grand
mean centered. The results show that the grand mean of SAT score is 1000.09, which is
relatively the same as in Model I.
MSES, MFEXP, and MFATD all show significant effects on school SAT scores at .000
level. These results mean that higher mean SES, educational expectation, and attitude toward
learning, tend to have higher level of mean test score composite. Among the teacher
characteristics and school characteristics variables, only mean teacher experience, school
student/teacher ratio, and school free and reduced-price lunch variables show significant impact
to student achievement. However, these relationships are weak compared with MSES, MEXP,
and MATD variables.
102
The reliability 0.507 is a conditional reliability with which we can discriminate among
schools with the same mean predictors at the school level. Controlling these school level
variables, the variance in mean student achievement among schools decreases dramatically,
which is also suggested by the decreased intraclass correlation coefficient ρ = τ00 / (τ00 + σ2) =
3193.49 / (3193.49 + 32355.19) = 0.09 (was 0.25 in Model I). Comparing with the one-way
ANOVA model, adding school-lever predictors decreases the variance among schools from
10526.85 to 3193.49. In other words, 69.7% of the variance in mean SAT between schools is
explained by the school-level variables {[τ00(ANOVA) - τ00(current)] / τ00(ANOVA) =
(10526.85 – 3193.49) / 10526.85 = 0.697}. Overall, Model II has more explanatory power then
Model I at school level.
Random Coefficient (RC) Model (Model III)
Model III is a random coefficient model. In this model, students’ Socio-Economic Status
Composite (SESC), Expectation Factor (FEXP), Motivation Factor (FMOV), Attitude Factor
(FATD), Parent Involvement Factor (FPAR), Student Gender (GENDR), and racial variables are
included as predictors at student level. All these variables are group centered. There is no school-
level predictor in this model. The average intercept is quite reliable with reliability at 0.841.
The grand mean SAT score is 981.41. All the student-level predictors, except for the
parental involvement variable, are significant. The results show that, on average, students who
have higher SES tend to do better in the tests. Students who have higher expectations of their
education and higher attitude tend to have higher test scores. However, the results also show that
students who have stronger motivations of education tend to have lower test scores. White and
Asian students tend to have better test scores than Black and Hispanic students. Overall, Black
103
students have significant lower SAT scores than otherwise. Student SES, expectation, and
attitude all show very strong positive impact on SAT scores.
The inclusion of student level variables decreases variances at student level from 887.14
to 589.26. That is, 35.9% of the variance at student level is explained by these student level
variables altogether {[σ2 (ANOVA) - σ2 (current)] / σ2 (ANOVA) = (32355.19 – 20728.06) /
32355.19 = 0.359}. Comparing the proportions of variances explained by these variables at
student level (35.9%) in the current model and at school level (69.7%) in the means-as-outcome
model, it seems these variables have stronger effects at school level than at student level. The
mean SAT score still varies significantly across schools at 0.000 level.
The random part of the student level model as a whole shows that mean levels of SAT
scores still vary across schools after controlling the ten student-level variables. Among the ten
student-level variables, only motivation indicates heterogeneity of regression.
Intercepts & Slopes-as-outcome Model (Model IV)
Model IV is an intercepts & slopes-as-outcome model. In this model, school level
predictors MSES, MFEXP, MFMOV, MFATD, MFPAR, MTEXP, MTCRT, MTDEG, SURB,
SENR, SPMN, SSTR, SPTC, and SPFL are included. The conditional reliability of average
intercept is 0.607, and the conditional reliability of the other slopes are around 0.09. This number
is low because these predictors as a whole have explained much variance in the achievement
slope, making it more difficult to detect additional variance; hence the individual slopes are not
very high. Student level predictors SESC, FEXP, FMOV, FATD, FPAR, GENDR, and four
racial variables are included.
Overall the results reflect the means-as-outcome model and the random coefficient model
in terms of the significance of school-level and student-level predictors. On the school level, the
104
results show that that the grand mean SAT score is 147.73. Schools with higher mean SES tend
to have higher level of student achievement (γ01 =126.96). Students in schools with higher mean
expectation tend to have higher level of student achievement (γ02 =81.59). Similarly, students in
schools with higher mean attitude toward learning tend to learn more (γ04 =32.37). Interestingly,
schools with more experienced teachers may lead to lower level of student achievement,
although the relationship is weak (γ06 = 1.97). Schools with higher percentage of minority
students and higher level of free and reduced-price lunch students tend to have lower level of
student achievement, again with weak relationships (γ11 = -0.97, γ014 = -1.09). The other school
level predictors are not statistically significant.
On the student level, gender, Asian Student (ASIAN), White Student (WHITE), Black
Student (BLACK), and Hispanic Student (HISPN) variables remain significant, and the
directions of the relationships are the same as in the random coefficient model. SESC, FEXP,
FMOV, and FATD variables are all significant. FPAR is not significant as shown in Model III.
The school-level composites under each student-level variable in general show little impact since
very few of them are statistically significant. The heterogeneity of regression slopes becomes
more for the student-level variables.
Since there are two levels of inferences, student-level variables are modeled with the
school-level variables in Model IV. Nine out of the 10 variables–gender, Asian student, Black
student, Hispanic student, White student, SES, expectation, motivation, and attitude–were
observed to vary among schools, and because of this variability, these nine variables can be
modeled with school-level variables. This decomposition means that each randomly varying
coefficient at student-level becomes a model. The school-level variables that are observed to be
105
significantly related to the random coefficients, or in other words, some school-level variables
show influences to student-level slopes.
Among the nine significant student-level variables, eight of them were observed to have
significant school-level predictors. Mean student behavior variables remain strong predictors on
school level, although student motivation has a different direction from student expectation and
attitude. Among all the student-level variables that are modeled by school-level variables,
school-level expectation shows the strongest effect toward Black student variable. Examining the
random effects for Black students, 41.8% the proportion of variance of the HISPAN coefficient
is explained by school mean expectation {τ03 (Model III) - τ03 (Model IV) / τ03 (Model IV) =
(8815.53 – 5126.19) / 8815.53 = 0.4188}.
The school-level predictors as a whole explain 69.0% of the variance in student NAEP
score among schools {[τ00(RC) - τ00(current)] / τ00(RC) = (11233.76 – 3476.79) / 11233.76 =
0.690}. All the results from the four models are shown in Table 19.
Table 19
Results for Four Model Estimates (Standard Coefficient) on Student SAT Score
I II III IV Fixed Effects School Mean, γ00 981.77*** 1000.09*** 981.41*** 959.88*** School-level Variables Mean SES, γ01 126.84*** 126.96*** Mean Expectation, γ02 62.94* 81.59** Mean Motivation, γ03 14.29 6.21 Mean Attitude, γ04 36.73* 32.37* Mean Parent Involvement, γ05 -16.50 -23.62 Mean Teacher Experience, γ06 2.36** 1.97* Mean Teacher Certification, γ07 -15.92 -3.83 Mean Teacher Degree, γ08 2.87 -1.92 School Urbanicity, γ09 -13.22 -12.18 School Enrollment, γ010 -0.01 0.00 School % Minority, γ011 -0.24 -0.97*** School Student/Teacher, γ012 4.28** 2.00 School % Teacher Certified, γ013 -0.65 -0.50 School Free Reduced Lunch, γ014 -1.83*** -1.09** Student-level Variables Gender, γ10 -14.25** -10.98*
106
Mean Teacher Experience, γ16 -3.01** Mean Teacher Degree, γ16 20.55* Free Reduced Lunch, γ114 1.11* Asian Student, γ20 37.31** 36.15** Mean Attitude, γ24 222.57** School Student/Teacher, γ212 8.80* Black Student, γ30 -110.31*** -108.25*** Mean Expectation, γ32 210.14* Mean Attitude, γ34 138.90* Hispanic Student, γ40 -31.81** -27.08 Mean Expectation, γ42 246.10** Mean Attitude, γ44 154.25* Free Reduced Lunch, γ414 3.16* White Student, γ50 31.17** 36.46** Mean Expectation, γ52 170.68* Mean Motivation, γ53 -218.76* SES, γ60 57.24*** 56.86*** Expectation, γ70 68.84*** 68.23*** Mean Expectation, γ72 48.15* School % Minority, γ711 -0.57** Motivation, γ80 -22.87*** -22.25*** Mean Attitude, γ84 38.94* Attitude, γ90 58.19*** 55.94*** Mean Teacher Experience, γ96 2.40 * Mean Teacher Degree, γ98 -19.13* Parent Involvement, γ100 4.15 5.94 Mean Expectation, γ102 -44.79* School Student/Teacher, γ1012 -2.48 *
Random Effects τ00 10526.85*** 3193.49*** 11233.76*** 3476.79*** τ01 1604.43 644.28** τ02 826.75 1415.48** τ03 8815.53 5126.19** τ04 3693.65 3508.49*** τ05 3696.62 4784.87** τ06 1638.90 1568.82*** τ07 404.95 504.83*** τ08 842.02** 703.32*** τ09 1116.84 1527.82** τ10 803.19 848.19** σ
2 31899.76 32355.19 20728.06 20696.96 ρ 0.25 0.09
*** p<0.000, ** p<0.01, * p<0.05; All school-level predictors are grand mean centered; All student-level predictors are group mean centered.
High School GPA
One-way ANOVA Model (Model I)
Model I is a one-way ANOVA model. There is no predictor at both student level and
school level. The dependent variable is student high school GPA for all academic courses. The
107
GPA information is from the first follow-up survey in 2004. This model is to partition the total
variance in the student achievement into within and between school components, and to see if
there is significant difference in student GPA among schools.
The results show that the grand mean student achievement is 2.51. The reliability of the
sample means for the true school mean is 0.762, which indicates that the sample means are quite
reliable as indicators of the true school mean.
Results also show that the variability exists across schools (0.12) is statistically
significant at 0.000 level, so that the null hypothesis that the variance across schools equals 0 is
rejected. The intraclass correlation coefficient ρ = τ00 / (τ00 + σ2) = 0.12 / (0.12 + 0.60) = 0.17.
This suggests that 17% of the total variance in student GPA exists across schools. Since this
number is greater than 0.1, the dataset qualifies for multilevel analysis. In other words, the
results of this empty model show that student GPA has enough variability at the student and
school levels for further analysis with HLM.
Means-as-outcome Model (Model II)
Model II is means-as-outcome model. To account for the variance across schools, the
Mean Socio-Economic Status (MSES), the Mean Student Expectation (MFEXP), the Mean
Motivation (MFMOV), the Mean Attitude (MFATD), Mean Parent Involvement (MFPAR),
Mean Teacher Experience (MTEXP), Mean Teacher Certification (MTCRT), Mean Teacher
Highest Degree (MTDEG), School Urbanicity (SURB), School Enrollment (SENR), School
Percent Minority (SPMN), School Student-Teacher Ratio (SSTR), School Percent Teacher
Certified (SPTC), and School Percentage of Student Receive Free and reduced-Price Lunch
(SPFL) variables are included in the level 2 model. All these school-level variables are grand
108
mean centered. The results show that the grand mean of high school GPA is 2.48, which is
relatively the same as in Model I.
MSES, MFATD, SENR, SPMN, SSTR, SPTC, and SPFL all show significant effects on
school mean GPA at .000 level. MFEXP, MFMOV, MFPAR, SURB, and the teacher
characteristics variables show insignificant effect on school mean GPA. These results mean that
higher mean SES and mean educational expectation tend to have higher level of mean GPA.
SENR, SPMN, SSTR, SPTC, and SPFL show significant impact to student achievement, but the
relationships are very small. These relationships indicate that bigger schools and schools with
more minority students tend to decrease student GPA, while schools with higher student/teacher
ratio, more teachers are certified, and schools with higher percentages free and reduced-price
lunch tend to increase student GPA.
The reliability 0.522 is a conditional reliability with which we can discriminate among
schools with the same mean predictors at the school level. Controlling these school level
variables, the variance in mean student achievement among schools decreases dramatically,
which is also suggested by the decreased intraclass correlation coefficient ρ = τ00 / (τ00 + σ2) =
0.03 / (0.03 + 0.62) = 0.05 (was 0.17 in Model I). Comparing with the one-way ANOVA model,
adding school-lever predictors decreases the variance among schools from 0.12 to 0.03. In other
words, 75% of the variance in high school GPA between schools is explained by the school-level
variables {[τ00(ANOVA) - τ00(current)] / τ00(ANOVA) = (0.12 – 0.03) / 0.12 = 0.75}. Overall,
Model II has more explanatory power then Model I at school level.
Random Coefficient (RC) Model (Model III)
Model III is a random coefficient model. In this model, students’ Socio-Economic Status
Composite (SESC), Expectation Factor (FEXP), Motivation Factor (FMOV), Attitude Factor
109
(FATD), Parent Involvement Factor (FPAR), Student Gender (GENDR), and racial variables are
included as predictors at student level. All these variables are group centered. There is no school-
level predictor in this model. The average intercept is quite reliable with reliability at 0.853.
The grand mean GPA score is a bit higher at 2.53. All the student-level predictors, except
for the Hispanic student variable, are significant at 0.000 level. The results show that, on
average, students who have higher SES tend to do better in the tests. Students who have higher
expectations of their education and higher attitude tend to have higher GPA. Girls in general tend
to have higher GPA than boys. White and Asian students tend to have better test scores than
Black students. The strongest predictor at student level is the attitude factor, and one unit
increase of student attitude tends to increase GPA by 0.45.
The inclusion of student level variables decreases variances at student level from 0.60 to
0.32. That is, 46.7% of the variance at student level is explained by these student level variables
altogether {[σ2 (ANOVA) - σ2 (current)] / σ2 (ANOVA) = (0.60 – 0.32) / 0.60 = 0.467}.
Comparing the proportions of variances explained by these variables at student level (46.7%) in
the current model and at school level (75%) in the means-as-outcome model, it seems these
variables have stronger effects at school level than at student level. The mean GPA still varies
significantly across schools at 0.000 level.
The random part of the student level model as a whole shows that mean levels of NAEP
scores still vary across schools after controlling the ten student-level variables. Although
insignificant, the random part of Hispanic student variable does indicate heterogeneity of
regression slopes exists for Hispanic student. Similarly, regression slopes vary across school for
GENDR, BLACK, WHITE, SES, expectation, motivation, attitude, and parental involvement
variables.
110
Intercepts & Slopes-as-outcome Model (Model IV)
Model IV is an intercepts & slopes-as-outcome model. In this model, school level
predictors MSES, MFEXP, MFMOV, MFATD, MFPAR, MTEXP, MTCRT, MTDEG, SURB,
SENR, SPMN, SSTR, SPTC, and SPFL are included. The conditional reliability of average
intercept is 0.686, and the conditional reliability of the other slopes are around 0.08. This number
is low because these predictors as a whole have explained much variance in the achievement
slope, making it more difficult to detect additional variance; hence the individual slopes are not
very high. Student level predictors SESC, FEXP, FMOV, FATD, FPAR, GENDR, and four
racial variables are included.
Overall the results reflect the means-as-outcome model and the random coefficient model
in terms of the significance of school-level and student-level predictors. On the school level, the
results show that the grand mean GPA is 2.50. Schools with higher mean SES tend to have
higher level of student achievement (γ01 =0.38). Students in Schools with higher learning attitude
tend to have higher level of achievement (γ04 =0.47). Although SENR, SPMN, SSTR, and
MTCRT variables are statistically significant, the relationships are minimal to student GPA (γs
=0.00).
On the student level, gender, Asian Student (ASIAN), White Student (WHITE), and
Black Student (BLACK) variables remain significant, and the directions of the relationships are
the same as in the random coefficient model. SESC, FEXP, FMOV, FATD, and FPAR variables
are all significant. Hispanic student is not significant as shown in Model III. The school-level
composites under each student-level variables in general show little impact since very few of
them are statistically significant. The heterogeneity of regression slopes still exist for the student-
111
level variables, although the scale decreases for most of these variables. This is an indication that
certain amount of variance picked up by the decomposition of school-level predictors.
Since there are two levels of inferences, student-level variables are modeled with the
school-level variables in Model IV. Nine out of the 10 variables–gender, Black student, White
student, SES, expectation, motivation, and attitude–were observed to vary among schools, and
because of this variability, these nine variables can be modeled with school-level variables. This
decomposition means that each randomly varying coefficient at student-level becomes a model.
The school-level variables that are observed to be significantly related to the random
coefficients, or in other words, some school-level variables show influences to student-level
slopes.
Among the nine significant student-level variables, three of them were observed to have
significant school-level predictors. Student expectation can be modeled by school mean teacher
certification; in that students with high expectation have lower GPA in schools that more
teachers are certified. Students with high attitude will have higher GPA is they are in schools
with high mean motivation and attitude.
The school-level predictors as a whole accounts for 58.3% of the variance in student high
school GPA among schools {[τ00(RC) - τ00(current)] / τ00(RC) = (0.12 – 0.05) / 0.12 = 0.583}.
All the results from the four models are shown in Table 20.
Table 20
Results for Four Model Estimates (Standard Coefficient) on Student Academic GPA
I II III IV Fixed Effects School Mean, γ00 2.51*** 2.48*** 2.53*** 2.50*** School-level Variables Mean SES, γ01 0.38*** 0.38*** Mean Expectation, γ02 0.03 0.03 Mean Motivation, γ03 0.11 0.13
112
Mean Attitude, γ04 0.47*** 0.47*** Mean Parent Involvement, γ05 -0.04 -0.05
Mean Teacher Experience, γ06 -0.00 -0.00 Mean Teacher Certification, γ07 0.02 0.03 Mean Teacher Degree, γ08 -0.04 -0.04 School Urbanicity, γ09 -0.04 -0.04 School Enrollment, γ010 -0.00*** -0.00*** School % Minority, γ011 -0.00*** -0.00*** School Student/Teacher, γ012 0.01* 0.01** School % Teacher Certified, γ013 0.00** 0.00* School Free Reduced Lunch, γ014 0.00* 0.00 Student-level Variables Gender, γ10 0.27*** 0.27*** Asian Student, γ20 0.32*** 0.31*** Black Student, γ30 -0.17*** -0.13** Hispanic Student, γ40 -0.01 -0.00 White Student, γ50 0.17*** 0.19*** SES, γ60 0.24*** 0.23*** Expectation, γ70 0.13*** 0.12*** Mean Teacher Certification, γ77 -0.08** School % Minority, γ79 -0.00* Motivation, γ80 0.08*** 0.08*** Attitude, γ90 0.45*** 0.45*** Mean Motivation, γ93 0.16* Mean Attitude, γ94 0.17*** Parent Involvement, γ100 0.04*** 0.05*** School Student/Teacher, γ1012 -0.01*
Random Effects τ00 0.12*** 0.03*** 0.12*** 0.05*** τ01 0.02** 0.01* τ02 0.08 0.07 τ03 0.09** 0.06** τ04 0.13** 0.08* τ05 0.10** 0.10* τ06 0.02*** 0.01** τ07 0.01** 0.01** τ08 0.01*** 0.01** τ09 0.01 0.00 τ10 0.00*** 0.00** σ
2 0.60 0.62 0.32 0.32 ρ 0.17 0.05
*** p<0.000, ** p<0.01, * p<0.05; All school-level predictors are grand mean centered; All student-level predictors are group mean centered.
Summary of Findings
School-level Effects versus Student-level Effects
One of the interesting findings is related to the achievement differences on individual and
school levels can be observed by calculating the plausible value range. The plausible value range
can be used to gauge the magnitude of the variance among students or among schools in terms of
113
student mean achievement (Raudenbush & Bryk, 2002). Figure 4 through Figure 7 illustrate the
95% confidence interval of each of the four outcome measures on school and student levels. It is
obvious that the school mean values of student achievement have a large range. Comparing with
the mean value of each outcome measures, the plausible range for the math and reading test
scores is 38%, the NAEP scores is 40%, the SAT is 41%, and the academic GPA is 54%.
Of course the plausible value range for students is larger than that of schools. These
results together suggest that where students attend high school has a great to do with how much
they learn, although these estimates do not account for the student background, teacher and
school characteristics.
Figure 4. Range of plausible values for math and reading test scores
20
30
40
50
60
70
80
School Student
114
Figure 5. Range of plausible values for NAEP test scores
Figure 6. Range of plausible values for SAT scores
Figure 7. Range of plausible values for academic GPA
60
110
160
210
School Student
200
400
600
800
1000
1200
1400
School Student
0
1
2
3
4
5
6
School Student
115
Student Behavior Factors versus School Resource Variables
The results of all the Model IVs are compiled into Table 21. This table also summaries
the answers to the first research question this study raises–what factors affect student academic
achievement the most? Overall, the results from the four outcome measures are quite close. The
school-level variables show consistent patterns in that the mean SES, mean student expectation,
and mean student attitude are the three strongest predictors to student achievement. Just like
some of the existing literatures found that SES has the most impact on student achievement, this
dissertation suggests that SES has the strongest positive influence on student learning on school
level. This means that students in schools of high SES tend to have higher academic achievement
than those students in schools of low SES. Similarly, students in schools of high educational
expectation and attitude tend to have higher test scores than those students in schools of low
educational expectations and attitude.
The results also show that teacher characteristics including mean teacher experience,
mean teacher certification, and teacher education background do not have significant impact on
student achievement. School urbanicity, school enrollment, student/teacher ratio, and percent
teacher certification do not have significant impact on student achievement either. School
percent minority and percent free and reduced-price lunch show significant relationships on
some outcome measures, however, the coefficients are small and limited.
On the student level, almost all the predictors have significant relationships with student
achievement measures. Racial variables including Asian, Black, Hispanic, and White all have
strong impacts to student learning outcomes. Asian and White students on average tend to have
higher achievement than Black and Hispanic students. Similar to school-level mean SES,
student-level SES proved to the strongest positive variable among all the predictors. Student
116
behavior factors also have significant impact to student achievement. Expectation and attitude
have strong positive influences while motivation has weak negative influence on student learning
outcomes. Parental involvement shows inconsistent relationships with 2 of the 4 outcome
measures, and the influences are not strong.
Table 21
Two-Level HLM Estimates (Standard Coefficient) over Four Outcome Measures
Math & Reading
NAEP Score Academic GPA
SAT Score
Model V Model V Model V Model V Demographic Variables Gender, γ10 0.06 -4.74*** 0.27*** -10.98* Asian Student, γ20 0.94 8.08** 0.31*** 36.15** Black Student, γ30 -3.31*** -10.89*** -0.13** -108.25*** Hispanic Student, γ40 1.92*** -4.3** -0.00 -27.08 White Student, γ50 2.18*** 6.29** 0.19*** 36.46** SES, γ60 3.19*** 10.92*** 0.23*** 56.86*** Mean SES, γ01 6.33*** 20.61*** 0.38*** 126.96*** School Resource Variables Mean Teacher Experience, γ06 0.03 0.09 -0.00 1.97* Mean Teacher Certification, γ07 -0.05 0.46 0.03 -3.83 Mean Teacher Degree, γ08 0.08 0.18 -0.04 -1.92 School Urbanicity, γ09 -0.22 -0.51 -0.04 -12.18 School Enrollment, γ010 -0.00 0.00 -0.00*** 0.00 School % Minority, γ011 -0.04*** -0.11*** -0.00*** -0.97*** School Student/Teacher, γ012 -0.06 -0.07 0.01** 2.00 School % Teacher Certified, γ013 0.00 -0.04 0.00* -0.50 School Free Reduced Lunch, γ014 -0.03 -0.14** 0.00 -1.09** Educational Behavioral Variables Expectation, γ70 2.80*** 9.49*** 0.12*** 68.23*** Motivation, γ80 -0.92*** -2.31*** 0.08*** -22.25*** Attitude, γ90 2.38*** 9.76*** 0.45*** 55.94*** Parent Involvement, γ100 0.52*** 0.59 0.05*** 5.94 Mean Expectation, γ02 4.04*** 13.92*** 0.03 81.59** Mean Motivation, γ03 -0.26 -3.90 0.13 6.21 Mean Attitude, γ04 4.26*** 10.56*** 0.47*** 32.37* Mean Parent Involvement, γ05 0.74 -2.34 -0.05 -23.62
*** p<0.000, ** p<0.01, * p<0.05; All school-level predictors are grand mean centered;
All student-level predictors are group mean centered;
117
Where Students Go to School Matters
Using both the school-level and student-level variables, these models across the four
outcome measures indicate that more variance exist on school level than on student level. In
other words, the school-level variables altogether predict more proportion of the student
achievement than the student-level variables combined. To address the research question 4 –
What portion of variance do factors from each of the two level of analysis explain? The results
presented in Figure 8 demonstrate the percent variance in four different student achievement
measures between student and school levels. Between 58.3% and 79.9% of the total variability in
achievement is related to the differences among schools the students attend, and between 20.1%
and 41.7% is due to the differences among students themselves. This finding implies that where
students go to school and who the students go to school with has tremendous impact on student
achievement. Students learn by themselves, however, the school environment, teachers and other
students who they interact everyday also related to their learning outcomes.
Figure 8. Percent of Variance in Achievement at Student and School Levels by Four Measures
20.1 20.231.0
41.7
79.9 79.869.0
58.3
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Math and
Reading Scores
NAEP Scores SAT Scores High School
Academic GPA
118
Combining the findings from Table 21, these school variation come mostly the school-
level SES, student expectations of education, and student attitudes of learning. This directly
address the research questions 2 this dissertation raises, in that student behavior factors have the
most contribution to student achievement, both individually, and collectively in schools.
Furthermore, to answer the research question 3, the mean school-level student behavioral factors
have more influences on student learning than student-level factors.
The results from the four sets of HLM analyses contribute to the understanding of which
factors at the school level influence the student level predictors, in other words, the changes
school-level to the student-level slopes. Across the four output measures, mean student
expectation has the strongest positive effect on a number of student-level variables including the
racial and student behavior variables. School-level mean SES, on the other hand, has very little
influence to any of the student-level variables. Some of the school characteristics variables, such
as student/teacher ratio, teacher experience, and school enrollment have limited impact on
student-level predictors.
119
CHAPTER 5
CONCLUSION
Review of Findings
This dissertation raised four research questions:
1. In the realm of education production function, what factors affect student academic
achievement the most?
2. Specifically, how do the motivation and attitude affect student academic
achievement?
3. Do these factors have the same effect at the student level and the school level?
4. What portion of variance do factors from each of the two level of analysis explain?
Findings from the HLM analysis indicated that on the student level, race and SES had
strong relationships with student achievement. Asian and White students tended to have better
test scores than Black and Hispanic students. Students with higher SES were more likely to have
higher achievements. Gender variable had significant impact to student learning over three of the
four outcome measures. The pattern was not consistent on whether boys or girls have better
achievements. The three student behavior variables demonstrated significant influences on all
four learning outcomes. Expectation and attitude had positive impact while motivation had
negative impact to students. Parental involvement had a weak positive influence on two of the
four outcome measures, (i.e. math and reading test scores and academic GPA).
The results also illustrated that school-level variables were showing consistent patterns
over four different outcome measures. Mean SES, mean student expectation, and mean student
attitude were the top three positive predictors of student achievement. School characteristic
120
variables such as school urbanicity, school enrollment, student teacher ratio, percent teacher
certified, and school free and reduced-price lunch eligibility possessed very little impact, if any,
to student learning. The school percent minority variable was consistent over the four models
with negative impact to student achievement; however, the relationship was weak. Teacher
quality variables including teacher experience, teacher degree, and teacher certification all had
minimal influence on student learning. Similar to student level results, school level mean SES,
expectation, and attitude displayed strong positive impact to student learning. Mean student
motivation on school level was insignificant.
These findings across the four outcome measures showed that more variation exists at the
school level than at the student level, which means that school-level variables collectively
predicted more proportion of the student achievement than the student-level variables
collectively do. Between 58.3% and 79.9% of the total variability in achievement was related to
the differences among schools the students attend, and between 20.1% and 41.7% was due to the
differences among students themselves.
Furthermore, strong school effects may alter the impact of student-level variable.
Evidence emerged from the second outcome measure–the NAEP score. Hispanic students tend to
have low achievement, but Hispanic students in high educational expectation schools
outperformed their counterparts in low educational expectation schools significantly. This
pattern also held true for Black students using NAEP and SAT as outcome measures. On the
other hand, Asian and White students tended to have even higher achievement if they were
placed in schools where students had higher expectation and attitude of learning.
121
The sections that follow address other areas that are not directed related with the research
questions in this dissertation, however; these results relate back to the literature this dissertation
draws from and the findings are of important implications.
Student Behavior Factors
Among the three student behavioral factors, expectation and attitude on learning showed
strong and positive impact to student learning on student and school levels across the four output
measurements. Motivation factor, on the other hand, displayed inconsistent patterns. On the
student level, motivation was negatively associated with three of the four student achievement
variables (except GPA). All the relationships were weak compared with expectation and attitude
factors. The fact that high level of motivation on learning did not lead to high achievement is
contrary to student behavior literature. Unlike on the student level, the motivation factor on the
school level became insignificant across all four outcome measures. The discrepancy on student
and school levels was an indication that motivation factor in this dataset might not fully capture
the concept of motivation on learning.
Parental involvement, the fourth factor in this dissertation, had limited influence on
student learning. This factor was significant on math and reading scores and student GPA on the
student level, and the size of the effects is small. Parental involvement was insignificant on all
four outcome measures on the school level.
Teacher quality in this study consisted of three variables: teacher experience, teacher
certification, and teacher degree. All the three variables were on the school level, and all three
variables were insignificant across four student achievement variables except teacher experience
that had a weak positive effect on SAT scores. This consistent lack of impact from teacher
122
quality variables indicated that school resources spent on these areas might not lead to desirable
results, at least compared with student expectation and attitude on learning factors.
Gender
The effect from the gender variables in this study fluctuated across different outcome
variables. It was insignificant on math and reading test scores, negatively associated with NAEP
scores and SAT scores, and positively associated with academic GPA. The size of the effects
ranged from medium to large. The inconsistency reflected the fact that the effects were mixed on
different tests and different group of students.
Race
Most of the race variables showed significant influence to student learning. In general,
Asian and White students tended to have higher achievement, while Black and Hispanic students
tended to have lower achievement. These effects ranged from medium to large depending on the
outcome variable. Socioeconomic status had the strongest positive impact on student
achievement on both student and school levels. Take math and reading test score for example, on
average, one unit increase of student SES tended to increase the test score by 3.14 points, and
one unit increase of school SES tended to increase school mean score by 6.24 points.
School Characteristics
School level variables included school urbanicity, school enrollment, percent minority
students, student/teacher ratio, percent teacher certified, and percent free and reduced-price lunch
students. Urban schools in general had lower student achievement than non-urban schools; the
relationship was not significant though. Student enrollment variable only became significant at
the GPA measurement, and the relationship was very weak. Similar to that was student/teacher
and percent teacher certified variables.
123
The only school level variables had certain influence on student learning were the percent
minority students, and percent free and reduced-price lunch students variables. High levels of
minority and free and reduced-price lunch students were both associated with low level of
student achievement. Still the strength of the relationship was weak compared with school level
SES variable and expectation factor.
Since HLM allows two levels of analysis, the school level decomposition in this study
yielded some interesting findings. The student-level race variables were further explained by
school-level learning behavior factors multiple times across the four outcome measures. The
decomposition occurred mostly with the race variables.
School Level Decomposition on Math and Reading Test Score
On the math and reading test score outcome measure, Black students in general tend to
have a 3.31 points disadvantage compared with otherwise. However, Black students in high
expectation schools tend to gain 6.66 points advantage - that is a net 3.35 points benefit. There
are other pairs of variables involves school-level decomposition. Similarly, Black students in
schools with experienced teachers, their test score then to increase by 0.19 points. This small
increase means that experienced teachers help Black students learn a bit more. There are other
school-level decompositions, but due to the fact that they are not significant at school level, these
decompositions are not as important as from the student behavior factors.
School Level Decomposition on NAEP Test Score
On the NAEP test score outcome measure, all four race variables had decompositions
from student behavior factors. Students in schools with high expectations on learning, regardless
of the race, the students tended to have test scores gain of 30 to 40 points. Motivation factor
tended to reduce the test scores by about 34 points.
124
School Level Decomposition on SAT Test
On the SAT score outcome measure, the student behavior variables remained the patterns
as shown in previous outcome measures. Students in schools with high expectation, and attitude
had a significant score gain regardless of race. Motivation had an opposite impact on White
students, however; this factor was not significant on school level. Teacher experience showed a
little positive impact on students in schools with high attitude on learning. Teacher degree had a
bigger negative impact than teacher experience. This indicated that resources spent on
experienced teachers may generate better learning outcomes than spent on well-educated
teachers. However, either teacher experience or teacher degree did not present significant
relationships with student achievement on the school level.
School Level Decomposition on Academic GPA
On the high school academic GPA outcome measure, student expectation had
decomposition by teacher certification. Students with high expectation on learning tended to
have high achievement, however; these students might have a significant GPA drop if they were
in schools with high level of teacher certification. Again, teacher certification was not significant
on student level.
Research Implications
Lee (2000) stated that the school environment, in which students learn, had a great deal
of influence on the learning outcome. “Specifically, these school factors are school type, school
effectiveness, and school organizational style” (Lee, p126). The results from this dissertation
have reinforced the argument that schools make a difference. Across the four outcome measures,
the results proved once again that where students attend school impacts how much they learn.
The school contribution does not come from school resources or from teacher quality variables.
125
What really matters is student educational expectation on both the individual level and the school
level. This implies that to help students achieve more, the solution may not come from reforming
the education system but may exist from the ways in which students view their educational
opportunities and the interaction with peers and teachers.
Another implication from this research is that it helps to evaluate existing educational
policies, especially school finance policies. The findings in this study add to the long-lasting
debate of whether money matters to student learning. If educational resources can be used to
increase student educational expectations, the impact is considerable.
Along with school resource allocation policies, findings from this dissertation imply that
student assignment policies may include more elements. One recent Supreme Court case5 ruled
that race cannot be used alone in student assignment policies. To achieve student diversity,
policy makers may consider student expectation or attitude on learning. These factors are as
strong as race or SES factors in predicting student achievement.
This study has also shown that the attempts to increase validity by including four
different outcome measures are working. All the four student achievement variables yielded
similar results from the same set of student-and school-level variables. In addition, two of the
four student achievement variables are from the first follow-up survey: NAEP score and
academic SAT. These two variables were constructed 2 years after the math and reading test
score, which was extracted from the base year survey when all the students were 10th graders at
the time. The similarity of results from outcome measures of different times may suggest that
student achievement of high school students tend to have lasting effects.
5 Parents Involved in Community Schools v. Seattle School District No. 1, 127 S. Ct. 2738 (2007)
126
Policy Implications
“Learning is a product of schools, but also of families, communities, and peers”
(Rothstein, 2000, p. 5). The education production function has at least five inputs: peer effects
from fellow students, school and teacher characteristics, family SES and parental involvement,
community properties such as minority and urbanicity, and student themselves, specifically the
learning behavior factors. All five aspects have been included in this analysis to predict effects
on student achievement. As expected, the strongest variables are SES and student expectation
and attitude on learning. None of these are within school systems and none of them are directly
related to school expenditure. SES has been an important predictor to student achievement since
the Coleman et al. study (1966). Student behavior variables, as significant as they are in this
study, pave the way to a new focus of educational policy.
One policy implication from this study is that examining relations between school
characteristics and student academic achievement could help parents to choose schools that
would be most appropriate for their children. In addition, administrators could make more
accurate decisions regarding which schools in the district best serve students based on
examination of the relationship between school characteristics and student academic
achievement.
Since there is no empirical linkage from school resource and teacher quality variables,
another policy implication from this study is that school finance policy should not be formulated
principally on the amount of money to spend to project student learning. This is not by any
means to claim that school expenditures of various sorts are negligible. In fact, school finance
policy may be framed toward creative ways to increase student learning behaviors (i.e., student
educational expectation and learning attitude).
127
It comes as no surprise that SES has the most positive impact to student achievement on
both the student and school levels. School policy has very limited influence on student SES. The
results also show that learning occurs at school but learning outcome is not only related to school
factors. In this study, SES is not merely a measure of family capital; it is a composite of parents’
education background, parents’ working status, and family income.
Data Limitations
As with all research, there are a number of limitations which may undermine the results
of this study. First, significant numbers of cases were dropped out during the factor analysis
stage due to missing values. The factor analysis uses linear trend at a point regression method to
replace missing data. However, as pointed out in chapter 4, further steps of this research should
adopt more sophisticated techniques such as multiple imputation to address the missing value
problem.
Secondly, since this is a longitudinal study, it will be useful to test the lasting effects of
all the variables used here in relation to student achievement. For example, the ELS:2002 is still
on-going, and there will be future data on college academic achievement. Testing the earlier (10th
grade) student demographic and educational behavior variables indicates what kind of impact
these variables had onto students’ later achievement.
Additionally, students’ present academic achievement is related with initial achievement
to a certain degree, and prior ability is also highly associated with achievement (Qian & Blair,
1999; Trusty, 2002). Therefore the current student achievement from this study may possibly be
overestimated. One approach to make a better inference is to use growth modeling to account for
the prior ability in the future research design.
128
Teacher salary was not included in this dissertation. The reason is that ELS:2002 is a
national data and pure dollar amount may not reflect the difference among different living
standards of the school locations. The inclusion of teacher salary without controlling the income
difference may incur bias into the analysis. However, teacher salary takes up a considerable
amount of total school expenditures,6 not including benefits and other type of spending on
teaching staff. Taking this variable into the consideration is believed to increase the explanatory
power of the study. A better way to handle this problem is to link the current ELS:2002 data to
external data to include the necessary variables.
Future Steps
This study shows that student educational expectation and attitude are among the
strongest predictors of academic achievement. In the future, this research may point to student
expectation and attitude into more depth. First, whether or not prior ability has any correlation
with either expectations and motivations or learning outcomes was not addressed in this research.
It will be beneficial to incorporate prior ability into the model as a control variable. Therefore the
relationship from expectation and attitude can be further purified. Second, ELS:2002 is a
longitudinal study with multiple survey points. This research may test the student expectation
and attitude effects over time to see whether or not schools alter these student-level variables,
and how.
6 Source for raw data: National Education Association, "Rankings & Estimates" published December 2008.
http://www.nea.org/home/29402.htm
129
REFERENCES
Archibald, S. (2006). Narrowing in on educational resources that do affect student achievement.
Peabody Journal of Education, 81(4), 23-42.
Autry, L. B., & Langenbach, M. (1985). Locus of control and self-responsibility for behavior.
Journal of Educational Research, 79(2), 76-84.
Ballou, D., & Podgursky, M. (2000). Reforming teacher preparation and licensing: What is the
evidence? Teachers College Record, 102(1), 5-27.
Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory.
Englewood Cliffs, NJ: Prentice Hall.
Becker, G. S. (1992). The economic way of looking at life. Nobel Lecture, December 9, 1992.
Biddle, B. J., & Berliner, D. C. (2002) Small class size and its effects: What does the evidence
say about the effects of reducing class size? Educational Leadership, 59(5), 12-23.
Bickel, R. (2007). Multilevel analysis for applied research: It’s just regression. New York: The
Guilford Press.
Borland, M. V., Howsen, R. M., & Trawick, M. W. (2005). An investigation of the effect of class
size on student academic achievement. Education Economics, 13(1), 73-83.
Bowles, T. J., & Bosworth, R. (2002). Scale economics in public education: Evidence from
school level data. Journal of education finance, 28(2), 285-300.
Bozick, R., & Lauff, E. (2007). Education Longitudinal Study of 2002 (ELS:2002): A first look
at the initial postsecondary experiences of the sophomore class of 2002 (NCES 2008-
308). National Center for Education Statistics, Institute of Education Sciences, U.S.
Department of Education. Washington, DC.
Bryk, A. S., & Raudenbush, S. W. (1988). Toward a more appropriate conceptualization of
research on school effects: A three-level hierarchical linear model. American Journal of
Education, 97(1), 65-108.
Burstein, L. (1980). The analysis of multilevel data in educational research and evaluation.
Review of Research in Education, 8, 158-233.
Campbell, J. R., Hombo, C. M., & Mazzeo, J. (2000). NAEP 1999 trends in academic progress.
Washington, DC: U.S. Department of Education, National Center for Education
Statistics.
Carpenter, W. A. (2000). Ten years of silver bullets: Dissenting thoughts on education reform.
Phi Delta Kappan, 81(5), 383-389.
130
Clark, C. (1998). Using school-level data to explore resources and outcomes in Texas. Journal of
Education Finance, 23(3), 374-398.
Clasen, D. R., & Clasen, R. E. (1995). Underachievement of highly able students and the peer
society. Gifted and Talented International, 10(2), 67-75.
Cohen, M. (1983). Instructional, management and social conditions in effective schools. In A.
Odden & L. D. Webb (Eds.), School finance and school improvement linkages for the
1980s (pp. 17-50). Cambridge, MA: Ballinger.
Coleman, J. S., Campbell, E. Q., Hobson, C. J., McPartland, J., Mood, A. M., Weinfeld, F. D., &
Work, R. L. (1966). Equality of educational opportunity. Washington, DC: U.S.
Government Printing Office.
Cook, P. J., & Ludwig, J. (1997). Weighing the “Burden of ‘Acting White’”: Are there race
differences in attitudes toward education? Journal of Policy Analysis and Management,
16(2), 256-278.
Darling-Hammond, L. (2000). Reforming teacher preparation and licensing: Debating the
evidence. Teachers College Record, 102(1), 28-56.
Darling-Hammond, L. (2007). Race inequality and educational accountability: The irony of ‘No
Child Left Behind’. Race Ethnicity and Education, 10(3), 245-260.
Duncan, O. D., Featherman, D. L., & Duncan, B. (1972). Socioeconomic Background and
Achievement. New York: Seminar Press.
Elliot, M. (1998). School finance and opportunities to learn: Does money well spent enhance
students achievement? Sociology of Education, 71(3), 223-245.
Ethington, C. A. (1992). Gender differences in psychological model of mathematics
achievement. Journal for Research in Mathematics Education, 23(2), 166-181.
Fan, X. (2001). Parental involvement and students’ academic achievement: A growth modeling
analysis. The Journal of Experimental Education, 70(1), 27-61.
Fan, X., & Chen, M. J. (2001). Parental involvement and students’ academic achievement: A
meta-analysis. Educational Psychology Review, 13(1), 1-22.
Ferguson, R. F. (1991). Paying for public education: New evidence on how and why money
matters. Harvard Journal on Legislation, 28(2), 465-498.
Ferguson, R. F., & Ladd, H. F. (1996). How and why money matters: An analysis of Alabama
schools. In H. Ladd, (Ed.), Holding schools accountable: Performance-based reform in
education (pp. 265-298). Washington, DC: The Brooking Institution.
131
Finn, J. D. (2002). Small classes in American schools: Research, practice, and politics. Phi Delta
Kappan, 83(7), 551-560.
Finn, J. D., & Achilles, C. M. (1990). Answers and questions about class size: A statewide
experiment. American Educational Research Journal, 27(3), 557-577.
Finn, J. D., & Rock, D. A. (1997). Academic success among students at risk for school failure.
Journal of Applied Psychology, 82(2), 221–234.
Flowers, L. A., Milner, H. R., & Moore, J. L. (2003). Effects of locus of control on African
American high school seniors’ educational aspirations: Implications for preservice and
inservice high school teachers and counselors. High School Journal, 87(1), 39-50.
Fordham, S. (1988). Racelessness as a factor in black students' school success: Pragmatic
strategy or pyrrhic victory? Harvard Educational Review, 58(1), 54-84.
Fordham, S., & Ogbu, J. U. (1986). Black students' school success: Coping with the 'burden of
acting white'. The Urban Review, 18(3), 176-206.
Fowler, W. J., Jr., & Monk, D. H. (2001). A primer for making cost adjustments in education.
Research and Development Report, Washington, DC: National Center for Education
Statistics, February.
George, R., & Kaplan, D. (1998). A structural model of parent and teacher influences on science
attitudes of eighth graders: Evidence from NELS:88. Science Education, 82(1), 93-109.
Glass, G. V., & Smith, M. E. (1979). Meta-analysis of research on class size and achievement.
Educational evaluation and Policy Analysis, 1(1), 2-16.
Goldhaber, D. D. (2002). The mystery of good teaching. Education Next, 2(1), 50-55.
Goldhaber, D. D., & Brewer, D. J. (1997). Evaluation the effect of teacher degree level on
educational performance. In W. J. Fowler (Ed.), Developing in school finance (pp. 197-
210). Washington, DC: National Center for Education Statistics, U.S. Department of
Educaiton.
Goldhaber, D. D., & Brewer, D. J. (2000). Does teacher certification matter? High school teacher
certification status and student achievement. Educational Evaluation and Policy Analysis,
22(2), 129-145.
Goldschmidt, P., & Eyermann. T. S. (1999). International educational performance of the United
States: Is there a problem that money can fix? Comparative Education, 35(1), 27-43.
Goldsmith, P. A. (2004). Schools’ racial mix, students’ optimism, and the black-white and
Latino-white achievement gaps. Sociology of Education, 77(2), 121-147.
132
Graham, S. (1994). Motivation in African Americans. Review of Educational Research, 64(1),
55-117.
Greene, J. P., & Winters, M. A. (2007). How much are public school teachers paid? Civic
Report, January No. 50. Manhattan Institute.
Greenwald, R., Hedges, L. V., & Laine, R. D. (1996a). The effect of school resources on student
achievement. Review of Educational Research, 66(3), 361-396.
Greenwald, R., Hedges, L.V., & Laine, R. D. (1996b). Interpreting research on school resources
and student achievement: A rejoinder to Hanushek. Review of Educational Research,
66(3), 411-416.
Grubb, W. N. (2008). Multiple resources, multiple outcomes: Testing the “improved” school
finance with NELS88. American Educational Research Journal, 45(1), 104-144.
Hanson, S. L. (1994). Lost talent: Unrealized educational aspirations and expectations among
U.S. youths. Sociology of Education, 67(3), 159-183.
Hanushek, E. A. (1986). The economics of schooling: Production and efficiency in public
schools. The Journal of Economic Literature, 24(3), 1141-1177.
Hanushek, E. A. (1989). The impact of differential expenditures on school performance.
Educational Researcher, 18(4), 45-65.
Hanushek, E. A. (1994). An exchange: Part II: Money might matter somewhere: A response to
Hedges, Laine, and Greenwald. Educational Researcher, 23(4), 5-8.
Hanushek, E. A. (1996). A more complete picture of school resources policies. Review of
Educational Research, 66(3), 397-409.
Hanushek, E. A. (1999). Some findings from an independent investigation of the Tennessee
STAR experiment and from other investigations of class size effects. Educational
Evaluation and Policy Analysis, 21(2), 143-164.
Hanushek, E. A. (2003). The failure of input-based schooling policies. Royal Economic Society,
Blackwell Publishing, Oxford, F64-F98.
Harris, D. N. (2007). Diminishing marginal returns and the production of education: An
international analysis. Education Economics, 15(1), 31-53.
Hedges, L. V., & Greenwald, R. (1996). Have times changed? The relation between school
resources and student performance. In G. Burtless (Ed.), Does money matter? The effects
of school resources on student achievement and adult success (pp. 74-92). Washington,
DC: Brookings Institution.
133
Hedges, L. V., Laine, R. D., & Greenwald, R. (1994a). An exchange: Part I: Does money matter?
A meta-analysis of studies of the effects of differential school inputs on student
outcomes. Educational Researcher, 23(3), 5-14.
Hedges, L. V., Laine, R. D., & Greenwald, R. (1994b). Money does matter somewhere: A reply
to Hanushek. Educational Researcher, 23(3), 9-10.
Hedges, L. V., & Nowell, A. (1995). Sex differences in mental test scores, variability, and
numbers of high-scoring individuals. Science, 269, 41-45.
Hoxby, C. M. (2000). The effects of class size on student achievement: New evidence from
population variance. Quarterly Journal of Economics, 15(4), 1239-1285.
Ilon, L., & Normore, A. H. (2006). Relative Cost-effectiveness of School Resources in
Improving Achievement. Journal of Education Finance, 31(3), 238-254.
Ingels, S. J., Pratt, D. J., Wilson, D., Burns, L. J., Currivan, D., Rogers, J. E., and Hubbard-
Bednasz, S. (2007). Education Longitudinal Study of 2002: Base-year to second follow-
up data file documentation (NCES 2008-347). U.S. Department of Education.
Washington, DC: National Center for Education Statistics.
Jacques, C., & Brorsen, B. W. (2002). Relationship between school district expenditures in
various categories and student performance. Unpublished manuscript.
Jencks, C., & Philips, M. (1980). The Black-White test score gap. Washington, DC: The
Brookings Institute.
Kim, J., & Mueller, C. W. (1979). Introduction to factor analysis: What it is and how to do it.
Beverly Hills, CA: SAGE Publications.
Lan, W., & Lanthier, R. (2003). Changes in students’ academic performance and perceptions of
school and self before dropping out of schools. Journal of Education for Students Placed
at Risk, 8(3), 309-332.
Lareau, A. (2003). Unequal childhoods: Class, race, and family life. University of California
Press.
Lee, S. M., Daniels, M. H., Puig, A., Newgent, R. A, & Nam, S. K. (2008). A data-based model
to predict postsecondary educational attainment of low-socioeconomic-status students.
Professional School Counseling, 11(5), 306-316.
Lee, V. E. (2000). Using hierarchical linear modeling to study social contexts: The case of
school effects. Educational Psychologist, 35(2), 125-141
134
Lee, V. E., & Smith. J. B. (1997). High school size: Which works best and for whom?
Educational Evaluation and Policy Analysis, 19(3), 205-227.
Linn, M., & Hyde, J. (1989). Gender, mathematics, and science. Educational Researcher, 18(8),
17-19, 22-27.
Lundy, G. F. (2003). School resistance in American high schools: The role of race and gender in
oppositional culture theory. Evaluation and Research in Education, 17(1), 6-30.
Ma, X., & Williams, J. D. (1999). Dropping out of advanced mathematics: How much do
students and schools contribute to the problem? Educational Evaluation and Policy
Analysis, 21(4), 365-383.
Marlow, M. L. (2000). Spending, school structure, and public education quality: Evidence from
California. Economics of Education Review, 19, 89-106.
Marsh, H. W., Martin, A. J., & Cheng, J. H. S. (2008). A multilevel perspective on gender in
classroom motivation and climate: Potential benefits of male teachers for boys? Journal
of Educational Psychology, 100(1), 78-95.
McKnight, P. E., McKnight, K. M., Sidani, S., & Figuerudo, A. J. (2007). Missing data: A
gentle introduction. New York: The Guilford Press.
Meyers, L. S., Gamst, G., & Guarino, A. J. (2006). Applied multivariate research: Design and
interpretation. Thousand Oaks, CA: SAGE Publications.
Mickelson, R. A. (1990). The attitude-achievement paradox among black adolescents. Sociology
of Education, 63(1), 44-61.
Muller, P. A, Stage, F. K., & Kinzie, J. (2001). Science achievement growth trajectories:
Understanding factors related to gender and racial-ethnic differences in precollege
science achievement. American Educational Research Journal, 38(4), 981-1012.
Nye, B., & Hedges, L. V. (2001). Are effects of small classes cumulative? Evidence from a
Tennessee experiment. The Journal of Educational Research, 94(6), 336-345.
Nye, B., & Hedges, L. V. (2004). Do minorities experience larger lasting benefits from small
classes? The Journal of Educational Research, 98(2), 94-100.
O’Conner, S. A., & Miranda, K. (2002). The linkages among family structure, self-concept,
effort, and performance on mathematics achievement of American high school students
by race. American Secondary Education, 31(1), 72-95.
Odden, A., Borman, G., & Fermanich, M. (2004). Assessing teacher, classroom, and school
effects, including fiscal effects. Peabody Journal of Education, 79(4), 4-32.
135
Odden, A. R., & Picus, L. O. (1992). School finance: A policy perspective. New York: McGraw
Hill.
Ogbu, J. U. (1978). Minority education and caste: The American system in cross-cultural
perspective. New York: Academic Press.
Ogbu, J. U. (1983). Minority status and schooling in plural societies. Comparative Education
Review, 27(2), 168-190.
Ogbu, J. U. (1991). Minority coping responses and school experiences. The Journal of
Psychohistory, 18(4), 433-456.
Ogbu, J. U. (2004). Collective identity and the burden of “Acting White” in Black history,
community, and education, The Urban Review, 36(1), 1-35.
Okpala, C. O., Okpala, A.O., & Smith, F. E. (2001). Parental involvement, instructional
expenditures, family socioeconomic attributes, and student achievement. The Journal of
Educational Research, 95(2), 110-115.
Pintrich, P. R., & Schunk, D. H. (2002). Motivation in education. 2nd ed. Upper Saddle River,
NJ: Merrill/Prentice Hall.
Plecki, M. (2006). Reflects on the field of school finance: Enduring questions and future
directions, Education Finance and Policy, 1(2), 1-9.
Podgursky, M., & Springer, M. G. (2007). Credentials versus performance: Review of the
teacher performance pay research. Peabody Journal of Education, 82(4), 551-573.
Qian, Z., & Blair, S. L. (1999). Racial/ethnic differences in educational aspirations of high
school seniors. Pacific Sociological Association, 42(4), 605-625.
Ram, R. (2004). School expenditures and student achievement: Evidence for the United States.
Education Economics, 12(2), 169-176.
Raudenbush, S. W., & Bryk, A. S. (1986). A hierarchical model for studying school effects.
Sociology of Education, 59, 1-17.
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data
analysis methods. (2nd Edition). Thousand Oaks, CA: SAGE Publications.
Reis, S. M., & McCoach, D. B. (2000). The underachievement of gifted students: Who do we
know and where do we go? Gifted Child Quarterly, 44(3), 152-170.
Reis, S. M., & Park, S. (2001). Gender differences in high-achieving students in math and
science. Journal for the Education of the Gifted, 25(1), 52-73.
136
Reynolds, A. J., & Walberg, H. J. (1992). A structural model of science achievement and
attitude: An extension to high school. Journal of Educational Psychology, 84(3), 371–
382.
Rice, J. K. (1999). The impact of class size on instructional strategies and the use o ftime in high
school mathematics and science courses. Educational Evaluation and Policy Analysis,
21(2), 215-229.
Rolle, A. & Houck, E. A. (2004). Introduction to the Peabody Journal of Education’s Special
Issue on the future of school finance research. Peabody Journal of Education, 79(3), 1-6.
Rosenberg, M. (1986). Conceiving the self. Malabar, FL: Krieger Publishing Company.
Rothstein, R. (2000). Finance fungibility: Investing relative impacts of investments in schools
and non-school educational institutions to improve student achievement. In Improving
educational achievement. A volume exploring the role of investment in schools and other
supports and services for families and communities. Finance Project, Washington, DC.
Center on Education Policy.
Rotter, J. B. (1954). Social learning and clinical psychology. Englewood Cliffs, N.J.: Prentice-
Hall.
Rotter, J. B. (1966). Generalized expectancies for internal versus external control of
reinforcement. Psychological Monographs, 80(1, Whole No. 609)
Rowan-Kenyon, H. T. (2007). Predictors of delayed college enrollment and the impact of
socioeconomic status. The Journal of Higher Education, 78(2), 188-214.
Rubin, D. B. (1977). Formalizing subjective notion about the effect of nonrespondents in sample
surveys. Journal of the American Statistical Association, 72, 538-543.
Rumberger, R. W. (1995). Dropping out of middle school: A multilevel analysis of students and
schools. American Educational Research Journal, 32(3), 583-625.
Rumberger, R. W., & Palardy, G. J. (2005). Does segregation still matter? The impact of student
composition on academic achievement in high school. Teachers College Record, 107(9),
1999-2045.
Schreiber, J. B., & Griffin, B. W. (2004). Review of multilevel modeling and multilevel studies
in The Journal of Educational Review (1992-2002). The Journal of Educational Review,
98(1), 24-33.
Sciarra, D. T., & Seirup, H. J. (2008). The multidimensionality of school engagement and math
achievement among racial groups. Professional School Counseling, 11(4), 218-228.
137
Sewell, W. H., & Shah, V. P. (1968). Parents’ education and children’s educational aspirations
and achievements. American Sociological Review, 33(2), 191-209.
Shepherd, S., Owen, D., Fitch, T. J., & Marshall, J. L. (2006). Locus of control and academic
achievement in high school students. Psychological Report, 98(2), 318-322.
Singh, K., Granville, M., & Dika, S. (2002). Mathematics and science achievement: Effects of
motivation, interest, and academic engagement. The Journal of Educational Research,
95(6), 323-332.
Sirin, S. R. (2005). Socioeconomic status and academic achievement: A Meta-analytic review of
research 1990-2000. Review of Educational Research, 75(3), 417-453.
Slavin, R. E. (1997). Educational psychology: Theory and practice (5th ed.). Needham Heights,
MA: Allyn & Bacon.
Smith-Maddox, R. (2000). Educational aspirations of African American Eighth Graders. Race,
Gender & Class in Education, 7(3), 58-80.
Snyder, T. D., Dillow, S. A., and Hoffman, C. M. (2009). Digest of Education Statistics 2008
(NCES 2009-020). National Center for Education Statistics, Institute of Education
Sciences, U.S. Department of Education. Washington, DC.
Sterbin, A., & Rakow, E. (1996). Self esteem, locus of control, and student achievement. Paper
presented at the Annual Meeting of the Mid-South Educational Research Association.
Stiefel, L. (2006). Insight from hindsight: The new education finance of the next decade.
Education Finance and Policy, 1(4), 1-13.
Sutton, A., & Soderstrom, I. (1999). Predicting elementary and secondary school achievement
with school-related and demographic factors. The Journal of Educational Research,
92(6), 330-338.
Thomas, J. P. (2000). Influences on mathematics learning and attitudes among African American
high school students. Journal of Negro Education, 69(3), 165-183.
Tobier, E. (2001). New York City’s public schools: The facts about spending and performance.
Civic Bulletin, 26.
Tollefson, N. (2004). Cognitive theories of motivation: Their application to the classroom. In M.
F. Shaughnessy (Ed.), Education in the new millennium: Not like your grandmother’s
schoolhouse. (pp. 203-224). Lanham, MD: University Press of America, Inc.
Treiman, D. J. (2009). Quantitative data analysis: Doing social research to test ideas. San
Francisco, CA: Jossey-Bass.
138
Trivette, P., & Anderson, E. (1995). The effects of four components of parental involvement on
eight-grade students achievement: Structural analysis of NELS-88 data. School
Psychology Review, 24(2), 299-318.
Trusty, J. (2000). High educational expectations and low achievement: Stability of educational
goals across adolescence. Journal of Educational Research, 93(6), 356-365.
Trusty, J. (2002). African Americans’ educational expectations: Longitudinal causal models for
woman and men. Journal of Counseling and Development, 80(3), 332-345.
Uguak, U. A., Elias, H. B., Uli, J., & Suandi, T. (2007). The Influence of Causal Elements of
Locus of Control on Academic Achievement Satisfaction. Journal of Instructional
Psychology, 34(2), 120-128.
Walberg, H. J. (1984). Improving the productivity of American’s schools. Educational
Leadership, 41(8), 19-27.
Walberg, H. J., Fraser, B. J., & Welch, W. (1986). A test of a model of educational productivity
among senior high school students. Journal of Educational Research, 79(3), 133-139.
Weiner, B. (1972). Attribution theory: A motivation and educational process. Review of
Educational Research, 42(2), 202-215.
Weiner, B., Nierenberg, R., & Goldstein, M. (1976). Social learning (locus of control) versus
attributional (casual stability) interpretations of expectancy of success. Journal of
Personality, 44(1), 52-68.
Wenglinsky, H. (1997). How money matters: The effect of school district spending on academic
achievement. Sociology of Education, 70(July), 221–237.
White, C. R. (1982). The relationship between socioeconomic status and academic achievement.
Psychological Bulletin, 91, 461-481.
Wilson, V. L. (1983). A meta-analysis of the relationship between science achievement and
science attitude: Kindergarten through college. Journal of Research in Science Teaching,
20(9), 839–850.
Xu, Z., & Gulosino, C. A. (2006). How does teacher quality matter? The effect of teacher –
parent partnership on early childhood performance in public and private schools.
Education Economics, 14(3), 345-367.
Yuan, Y. C. (2000). Multiple imputation for missing data: Concepts and new development.
Proceedings of the Twenty-Fifth Annual SAS Users Group International Conference
(Paper, No. 267). Cary, NC: SAS Institute. Retrieved July 29, 2008, from
http://www.ats.ucla.edu/stat/sas/library/multipleimputation.pdf