Direct Effects of Principal Effect
Ali Korkmaz, Ph.D.Strategic Data Project FellowLong Beach
Unified School District
Ahmet Uludag, Ph.D.Accord Institute for Education
ResearchInstructional Leadership Direct Effecton Student
OutcomesBackgroundPrincipal effects on student outcomesPast
ResearchSmall IndirectMediated through other school factorsRecent
studies (Education Next 2013, Educational Administration Quarterly
2012, Econ Papers 2011, School Effectiveness and School Improvement
2010)Direct and indirectNCLB (accountability) effects on the role
of principals
Purpose of the studiesTo understand principal attention impact
on student growthTo understand the direct effects of
principal-student discussions on student achievement and growth To
understand students reactions and actions based on these
discussions
First Study (1st Year)Student Fall Reading & Math
TestStudent Spring Reading & Math TestStudent
SurveyInstructional practicesStudy habits, motivationHome
environmentSchool learning environmentFirst Study Design
ElementsNorthwest Evaluation Association (NWEA) MAP TestConsortium
on Chicago School Research (CCSR) Student SurveyLongitudinal growth
for reading
NWEA MAP TestsA computer adaptive test tool to measure student
levels, placement and differentiate instruction to meet student
needsto guide curriculum and instructional decisionsto measure
student growth over timeof accountability to see how well we have
done our jobs; to measure the effectiveness of curriculum and
instruction
So what do we have as a tool:6
Fall RIT Spring Target RIT Target Growth = 10 192 202
Fall RIT 210Spring Target RIT 215Target Growth = 5Target Growth
for two 4th Grade StudentsStudent SurveyConsortium on Chicago
School Research (CCSR) SurveysMore than 10 years in the fieldIRT
based (Rasch) survey items and constructsIt is designed mainly for
school level analysisStudent survey items include:Academic
Engagement, Study Habits, Classroom Behavior, Learning Climate
(Expectation, Relationships)Subject area specific questionsSchool
and home environment questions
Data (1st Study)846 students at 9 CA middle schools55% male50%
free-reduced lunch11% African-American45% White44% Hispanic
9Data AnalysesStudents are grouped into 6 groups based on Fall
proficiency and Fall to Spring Growth (progress in the same
year)Effect-sizeNon-ProficientProficientHigh DeclineTypicalHigh
IncreaseHigh DeclineTypicalHigh IncreaseEffect-sizeWhich of the
following has the most impact on READING growth (for NON-PROFICIENT
students)?Principal attentionTeacher personal supportTeacher
personal attention(ENGLISH)Academic press(ENGLISH)
Which of the following has the most impact on reading growth
(for non-proficient students)?Effect-size differences for most
improving vs. most decliningPrincipal attention (.50)Teacher
personal support (.37)Teacher personal attention(ENGLISH)
(.34)Academic press(ENGLISH) (.34)
Which of the following has the most impact on READING growth
(for PROFICIENT students)?Principal attentionIncidence of
disciplinary actionHours reading outside schoolParental support for
student learning
Which of the following has the most impact on reading growth
(for proficient students)?Effect-size differences for most
improving vs. most decliningPrincipal attention (.37)Incidence of
disciplinary action (-.41)Hours reading outside school
(.48)Parental support for student learning (.27)
Summary for ReadingGroupsScales Effect-sizeNon -
ProficientPrincipal AttentionTeacher Personal SupportTeacher
Personal Attention (Eng)Academic Press (Eng)Student-Teacher
TrustReading Hours pwParent Support for Student LearningRigorous
Study Habits.50.37.34.34.31.27.26.20ProficientReading Hours
pwIncidence of Disciplinary ActionPrincipal AttentionParent Support
for Student Learning.48-.41.37.27Design of the 2nd Study (2nd
Year)Students (6th-8th grades) who had Basic from Fall and Winter
MAP Math test results eligible for the study (53 students)27
randomly identified and invited to participate15 accepted (parent
consent)26 are control groupPrincipal meets with students in the
SpringReviews CSTs, MAP testsGets student reactions to his/her
resultsStudents set goals for themselvesResearcher interviews
participating studentsStudents reactions to principal
discussionWhat students did afterwards
School ProfileMiddle school in an urban areaStudent body66% Free
or reduced lunch71% Hispanic or Latino12% White6% African
AmericanData and AnalysesNorth West Evaluation Association (NWEA)
MAP tests results (student level)Target and growth comparisons with
Norm dataCST Math results (2012 and 2013)Semi-structured interviews
with studentsConstant comparative method (Glaser, 1965)
MethodsMixed Methods Explanatory Sequential Design(Creswell,
2003)Qualitative methods used to help explain quantitative
findings
MATH MAP Growth & MAP Proficiency
CST 2012 and 2013 Proficiency
How Did Students Perceive?Participating experiment groupExcept 1
all were positiveStudents felt empoweredI start believing more in
myself.It encouraged me to do more.Principal was counting on me to
be successful.It pushed me to do better what I want to achieve.I
felt I needed to better. I should do better. Maybe a little
better.
How Did Students React? Better awareness of what is available to
themMotivated me to go to tutoring. I needed help in extra
things.Using the available resourcesTeacher tutored me about my
problems. It is not confusing.I participated more in class. I asked
for help. There was a lot of help.
How Did Students React?More strategic studyingMy teacher gave me
more exercise on my problem areas.Pure more studyingI studied more
for the test. I Disciplined myself. I really worked hard to do
better in future.I tried my best afterwards. My tests (referring to
course finals) were better.
ImplicationsSo what More work on principals?
Maybe more focused
Maybe more programmaticThroughout the year (Couple times a
year)
Assistant principal/academic deanLimitations1st StudyStudent
surveyTeacher input on classroom practices (teacher survey)No
teacher observationStudent growth High stakes, any rewards etc.
involved2nd study Small groupOne time event vs. established program
BiasSelectionStudents could have reflected only on all
positives
ContactAli Korkmaz, Ph.D. [email protected]
Ahmet Uludag, Ph.D. [email protected]
CERA December 6th, 2013
