View
3
Download
0
Category
Preview:
Citation preview
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS
EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
___________________________________
By
R. Kent Yocum
___________________________________
A DISSERTATION
Submitted to the faculty of the Graduate School of the Creighton University in Partial
Fulfillment of the Requirements for the degree of Doctor of Education in the Department
of Interdisciplinary Leadership
_________________________________
Omaha, NE
(August 7, 2015)
Copyright 2015, R. Kent Yocum
This document is copyrighted material. Under copyright law, no part of this document may be reproduced without the expressed permission of the author.
iii
Abstract
The mixed methods convergent transformative research study examined the alignment
between student academic achievements over time against the beliefs of English
Language Learner (ELL) teachers’ perceptions of students’ achievement after
participation in a seven-week summer intervention. Three types of archived achievement
data were analyzed: reading comprehension, English language acquisition and state
standardized assessment. The quantitative data results suggested students that participated
in the intervention made higher trend growth over time in reading comprehension and
English language acquisition assessments when compared to the control group.
Standardized assessment scores were inconclusive. The results of qualitative interviews
with ELL teachers supported the beliefs the students that participated in the intervention
had increased achievement and the intervention helped slow or stop summer learning
regression. Together these findings indicate the intervention was a tool that can be
implemented in other districts to slow summer learning regression and close the
achievement gap for ELL students with the most language need.
Keywords: English language learner (ELL), summer learning regression,
achievement gap, English language intervention, early language intervention
iv
Dedication
Dedicated to Tak
You help me be a better person
You help me realize what is really important
You make me whole
I was proud to witness you BECOME an American
v
Acknowledgements
This project would not have been possible without the love, support and
encouragement of many individuals behind the scenes being my cheerleader. In my
personal life, Tak lived the dissertation writing process as much as I did. That required a
high level of patience, problem-solving ability, a willingness to listen to endless hours of
talking out loud and sometimes just listening to the grumbling. My mom knew when to
offer advice and when to not offer an opinion. My aunt Barb encouraged, encouraged,
and encouraged. You will never know how much it meant to have you come to
commencement. You have always been one of my biggest supporters and most powerful
motivators.
My dissertation committee gave me invaluable advice and guidance. Dr. Julie
Gaddie, my dissertation chair, endless thanks for going through this with me. I wouldn’t
be done without your help. You never let me off the hook or deviate when it got tough. I
picked you for a reason and never questioned the selection. I don’t think you knew what
you were in for and I can never thank you enough for your guidance. Dr. Jeff Ehrlich, a
brilliant man committed to education with the perfect question at the perfect time. Dr.
Jennifer Moss-Breen for taking on double duty of advising and dissertation committee
member. Dr. Donna Ehrlich, she knew my dissertation before I did and stepped in last
minute. That’s dedication.
In the professional realm, I have to thank Dr. Leah Copeland, my principal
through the whole process and receptive to all the questions about the process. Laura
Lukens, Lezlie Paden, and the entire ELL department for constant support, quick
responses to specific question and access to volumes of data and resources. Dr. Michael
vi
Pragman and Cindy Hedrick: the data gurus of the world. You have amazing minds and
you both always responded with “how can I help?” when I needed data insight. Mr.
Sutton, for granting me access. Dr. Todd White, the superintendent that had the vision for
a project that broke all the traditions/rules about what summer education looked like and
could be. He lives the phrase “doing what is right for kids” and inspires me to make the
same choices. Deyrle Wallace for letting his building be used for a grand experiment.
Shelly Sanders: my mental chiropractor that helped keep my head on straight. The
Winnwood teachers—the best teachers there are! They move kids—what more could you
ask! A special thanks to Danielle Bentley, a teacher extraordinaire and you know the
special place you hold in my heart. I can’t thank enough all the ELL teachers that
willingly gave their time in order to participate in the research study. In memory of
Lynda Callan, a soul who worked tirelessly to ensure kids were protected and
championed the cause of education among immigrant children.
vii
Table of Contents
Page
Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
Dedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .v
Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvi
CHAPTER ONE: INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Background of the Problem . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Statement of the Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Purpose of the Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Mixed Methods Purpose Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Research Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Method Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Definition of Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Delimitations and Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Significance of the Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
CHAPTER TWO: REVIEW OF THE LITERATURE . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Summer Learning Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
viii
Regression in the General Student Population . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Poverty and Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
The “Investment Model” . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Socio-Economic Status and Summer Learning Regression Outside the United States . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
Long-term Impact of Regression on English Language Learner Students . . . . . 20
Other Contributing Factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
The Achievement Gap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Three Views of the Achievement Gap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
A Historical Perspective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Student Tracking and the Achievement Gap . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Influence of Positive Culture on Gaps in Achievement . . . . . . . . . . . . . . . . . . . . 30
Intervention . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
Early Interventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Early Intervention in Language and Literacy . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
First Grade Intervention in Language and Literacy . . . . . . . . . . . . . . . . . . . . . . . 35
Native and English Early Oral Language Intervention and Long-term Growth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 Trends . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 Accountability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 Academic Calendar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 Time spent on Learning as a Factor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 Assessment as a Measure of Academic Achievement . . . . . . . . . . . . . . . . . . . . . . 45 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
ix
CHAPTER THREE: METHODOLOGY. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
Research Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Description/Rationale of Participants/Sample. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Population . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
Sample. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Inclusion and/or Exclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Instrumentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
Variables/The Researcher’s Role . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
Data Collection Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Data Analysis Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Assumptions/Quality and Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
Ethical Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
CHAPTER FOUR: FINDINGS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Review of the Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 67
Data Analysis Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
Quantitative Data Analysis Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
Qualitative Data Analysis Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
Quantitative Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
x
Reading Comprehension – Fountas and Pinnell . . . . . . . . . . . . . . . . . . . . . . . . . 78
2011 cohort groups: Fountas and Pinnell reading comprehension . . . . . .79
2012 cohort groups: Fountas and Pinnell reading comprehension . . . . . .84
2013 cohort groups: Fountas and Pinnell reading comprehension . . . . . .85
2014 cohort groups: Fountas and Pinnell reading comprehension . . . . . .87
English Language Acquisition – ACCESS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .88
2011 cohort groups: English language acquisition . . . . . . . . . . . . . . . . . .88
2012 cohort groups: English language acquisition . . . . . . . . . . . . . . . . . .90
2013 cohort groups: English language acquisition . . . . . . . . . . . . . . . . . .91
Standardized English Language Arts Content – MAP . . . . . . . . . . . . . . . . . . . . .92
2011 cohort groups: English language arts . . . . . . . . . . . . . . . . . . . . . . . 93
2012 cohort groups: English language arts . . . . . . . . . . . . . . . . . . . . . . . 95
2013 cohort groups: English language arts . . . . . . . . . . . . . . . . . . . . . . . 95
Qualitative Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
Summer Learning Regression and the Achievement Gap Theme . . . . . . . . . . . . 96
Access to Educational Resources Theme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
The Academic Calendar Theme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
Curriculum and Program Goal of Developing English Language Proficiency Theme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 Teacher Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
Teacher A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
Summer learning loss . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .100
Academic calendar and curriculum connection . . . . . . . . . . . . . . . . . . . . .101
Intervention impact on assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
xi
Teacher B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
Eliminating summer regression by teaching forward . . . . . . . . . . . . . . . 103
Confluence of time and educational resources . . . . . . . . . . . . . . . . . . . . 104
Thematic learning units . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
Teacher C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
Summer learning regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
The calendar and servicing more students . . . . . . . . . . . . . . . . . . . . . . . . 107
Thematic units . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
Teacher D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
Summer regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
Instructional calendar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Intervention and assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Teacher E . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
Program goals and learning loss . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
Trajectory of students after intervention . . . . . . . . . . . . . . . . . . . . . . . . . 111
Teacher F . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
Reading comprehension and intervention participation . . . . . . . . . . . . . 113
Impact on other assessments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
Curriculum connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
Teacher G . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 114
Intervention impact on assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
Filling educational need during the summer . . . . . . . . . . . . . . . . . . . . . . 115
Calendar and intervention time frame . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
xii
Teacher H . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
Supporting students through the summer . . . . . . . . . . . . . . . . . . . . . . . . 117
Intervention and assessment relationship . . . . . . . . . . . . . . . . . . . . . . . . . 118
Teacher I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
The intervention goal and learning regression . . . . . . . . . . . . . . . . . . . . . 119
The intervention and the calendar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
The intervention and curriculum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
The intervention and assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
CHAPTER FIVE: CONCLUSIONS AND RECOMMENDATIONS . . . . . . . . . . . . . 124
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
Summary of the Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
Summary of the Findings . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .127
Central Question . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
Quantitative Question . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
Reading comprehension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
English language acquisition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
Standardized assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
Qualitative Question. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
Summer learning regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
Access to resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
Academic calendar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
Intervention goals and curriculum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
xiii
Transformative Question . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
Implications for Action/Recommendations for Further Research . . . . . . . .. . . . . . . . . 135
Implications for Action . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
Recommendations for Action/Future Research . . . . . . . . . . . . . . . . . . . . . . . . . . 136
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
Appendices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
xiv
List of Tables
Page
Table 2.1. Characteristics of Academic Calendar Types . . . . . . . . . . . . . . . . . . . . . . . . 40
Table 3.1. Archived Assessment Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
Table 3.2. WIDA ACCESS Language Level Ranges . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 Table 4.1. 3rd Grade ELA Missouri Assessment Program (MAP) Proficiency
by Subgroup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 Table 4.2. 4th Grade ELA Missouri Assessment Program (MAP) Proficiency by Subgroup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 Table 4.3. 5th Grade ELA Missouri Assessment Program (MAP) Proficiency by Subgroup . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . 66 Table 4.4. Fountas and Pinnell Reading Comprehension Proficiency Level by Grade Level . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . 69 Table 4.5. Assessment by Grade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 Table 4.6. Interviewee Participation in ELL Intervention . . . . . . . . . . . . . . . . . . . . . . . . 74 Table 4.7. Correlation and Interpretation of R-squared Value . . . . . . . . . . . . . . . . . . . . . 78 Table 4.8. 2012 Cohort Groups: Reading Comprehension . . . . . . . . . . . . . . . . . . . . . . . 84 Table 4.9. 2013 Cohort Groups: Reading Comprehension . . . . . . . . . . . . . . . . . . . . . . . 86 Table 4.10. 2014 Cohort Groups: Reading Comprehension . . . . . . . . . . . . . . . . . . . . . . 88 Table 4.11. 2011 Cohort Groups: English Language Acquisition . . . . . . . . . . . . . . . . . . 89 Table 4.12. 2012 Cohort Groups: English Language Acquisition . . . . . . . . . . . . . . . . . . 91 Table 4.13. 2013 Cohort Groups: English Language Acquisition . . . . . . . . . . . . . . . . . . 92 Table 4.14. 2011 Cohort Groups: English Language Arts Content . . . . . . . . . . . . . . . . . 94 Table 4.15. 2012 Cohort Groups: English Language Arts Content . . . . . . . . . . . . . . . . . 95 Table 4.16. 2013 Cohort Groups: English Language Arts Content . . . . . . . . . . . . . . . . . 96
xv
Table 4.17. Percentage ELL, Free and Reduced Lunch and Mobility . . . . . . . . . . . . . . 97 Table 5.1. Reading Comprehension Growth Trends . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
xvi
List of Figures
Page
Figure 3.1. Transformative framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 Figure 4.1. Cohort 2011: Kindergarten Fountas and Pinnell benchmark reading comprehension assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
Figure 4.2. Cohort 2011: First grade Fountas and Pinnell benchmark reading comprehension assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
Figure 4.3. Cohort 2011: Second grade Fountas and Pinnell benchmark reading comprehension assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 Figure 4.4. Cohort 2011: Third grade Fountas and Pinnell benchmark reading comprehension assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
Running head: ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED 1 METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
CHAPTER ONE: INTRODUCTION
Background of the Problem
From a historical perspective, summer learning regression has been of interest to
educational researchers for more than 100 years. The first definitive research conducted
by Heyns in the 1970’s acknowledged differences in family and community resources as
a leading gauge of summer learning regression and ultimately to the academic gap
between students of students of color and their white peers (Heyns, 1987; Cooper, Nye,
Charlton, Lindsay, & Greathouse, 1996; McCombs et al., 2011). As demographics of the
student population within the United States have changed, an increased number of
students learning English as a second language are falling into the lower quartile of
family income and adding an extra layer to an already complex issue (United States
Department of Education, 2014). At the same time, the Federal government has increased
accountability measures to ensure all students are meeting grade level academic goals
(NCLB, 2002; MSIP 5, 2011).
The summer vacation break is a fertile time period to examine when discussing
programs that bridge school years and staunch summer learning regression. McCombs et
al. (2014) noted, “students who attend summer programs have better outcomes than
similar peers who do not attend these programs” (p. xv) and suggested the effects can last
as long as two years after program participation. As the numbers of English Language
Learners increase, the cycle of regression impacts larger number of students annually.
Effective academic interventions for English Language Learners are an integral part of
building language and achievement competencies. The study will examine the
effectiveness of an extended summer intervention for English Language Leaners,
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
2
examining academic outcomes and educator perceptions of effectiveness. As the English
Language Learner population increases, so does the need for a broader base of research
and options for educators and policy makers to make programmatic and accountability
decisions. Summer interventions can provide a valuable scaffold and access to English
and grade level content to ensure students thrive through the rest of the academic year.
Lack of English language communication skills has been treated like a cognitive issue
and “just because you don’t speak English doesn’t mean you can’t think” (Checkley,
2014, para 4). In addition to adding to the knowledge base on English Language Learner
interventions, the population sampled in the study will provide depth of diversity in the
native language of participants, age ranges of participants, and the number of years
receiving English Language Learner services, that has not been available in the past.
The literature review will examine the summer learning regression and the
connection to the cumulative of gaps in achievement, in particular, among students
learning English as a second language. Other important aspects of the dialogue involve
early interventions, trends in summer programing and accountability as a reaction to the
research base already in existence. This combination of factors has led to the study on the
longitudinal trajectory of a seven-week summer intervention for English Language
Learner students and comparing learning outcomes of participants with the perceptions of
the English Language Learner teachers.
Statement of the Problem
The United States Department of Education (2104) noted for the 2014-2015
school year, minority students will outnumber their white peers and the number of
English Language Learners (ELL) is expected to increase over the next decade. Estimates
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
3
suggest, Hispanic, multi-racial, and Asian/Pacific Islander students will make the largest
categorical percentage gains over the next ten years with Spanish speaking students, both
immigrant and United States born, comprising the largest segment of the English
Language Learner population (United States Department of Education, 2014).
English instruction, as part of a comprehensive academic curriculum, has the
potential to develop approximately 4.7 million second language learners (United States
Department of Education, 2013) in the public school system into literate, multilingual
employees. The alternative is students that struggle with literacy in their native language
and second language. When looking at the research, McCombs et al. (2011) identified
minority and English Language Learners as disproportionately represented among low-
income families and the “subsequent inequities in educational attainment, in which
students from the bottom quartile of the income distribution are more than twice as likely
to drop out of high school as students from the top quartile” (p. 1). The potential for
dropping out can be traced directly to summer learning regression and gaps in academic
performance (McCombs et al., 2011).
The cumulative and disproportionate rate with which summer learning regression
impacts low-income students is a leading indicator of long-term academic growth
(McCombs et al., 2014). Among English Language Learners, the United States
Department of Education (2013) recognized English Language Learner students were on
average 36 points behind their English-speaking peers on fourth national normed reading
assessments and that gap increased to 44 points by eighth grade. In many communities,
limited access to quality educational resources through the summer is a major
contributing factor (Heyns, 1987; Cooper et al., 1996; McCombs et al., 2011).
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
4
Beyond meeting the academic needs of English Language Learners, the research
on longitudinal growth of English Language Leaners is limited and with increasing
accountability requirements, there is a need for research that contributes to the discussion
on effective summer learning interventions for English Language Leaner students.
Current research is focused on the identifying the root causes of summer learning
regression and the gap in academic achievement between students from low-income
families, with a particular emphasis on students that speak Spanish as a primary
language.
Purpose of the Study
The purpose of the study was to explore the longitudinal reading comprehension,
English language acquisition and standardized assessment outcomes of English Language
Learners that participated in a seven-week summer intervention with the perceptions of
English Language Learners teachers of the participating students. A mixed methods study
was conducted due to the application of a transformative lens and benefits of quantitative
and qualitative data during data analysis.
Mixed Methods Purpose Statement
The study investigated the longitudinal learning outcomes of a seven-week
extended summer intervention for English Language Learners through a convergent
mixed-methods research design with a transformative theoretical perspective (Creswell &
Plano Clark, 2011). A transformative theoretical perspective was used in which advocacy
(Sweetman, Badiee, & Creswell, 2010) was the overarching framework for the study.
This lens was selected due to the dual social components of language and educational
equity addressed by the study (Sweetman, Badiee, & Creswell, 2010). The study included
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
5
quantitative and qualitative data gathered concurrently. The quantitative data was used to
examine the affect a seven-week extended summer intervention had on longitudinal
reading comprehension, language acquisition and statewide normed assessment
performance of English Language Learners in an K – 5 elementary setting. The
qualitative data explored the perceptions of English Language Leaner teachers of students
that participated in the extended summer intervention and whose archived data was
collected in the quantitative phase (Creswell & Plano Clark, 2011). A mixed methods
approach was used to determine the correlation between perceptions and achievement,
while adding to the discussion on policy and instructional decisions on programing for
English Language Learners.
Research Questions
The research questions provided a focus for each phase of the study, including
driving data collection and analysis during the quantitative and qualitative phases,
merging and analyzing the two data sets as a mixed methods study and addressing the
transformative framework. All research questions were inductive and were not
considered null or directional hypothesis based on previous research. The central research
question synthesized the results of the quantitative and qualitative questions and provided
the overarching guide to research.
Central Research Question: What are the congruencies between archived English
Language Learner students’ achievement data in reading comprehension, English
language acquisition and standardized assessment and the beliefs of English Language
Learner teachers’ about student academic outcomes after students participated in a seven-
week summer intervention?
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
6
Additional questions were developed to guide specific aspects of the study. For
this study, the quantitative question examined student academic achievement outcomes
after participating in a summer program against the academic achievement outcomes of
students that did not participate. Treatment groups based on family selected level of
student participation in the intervention were the variable and the control group was non-
participation in the intervention among a homogeneous group of students. The qualitative
research question was used to identify themes from interview responses conducted with
English Language Learner teachers of students that participated in the summer
intervention. The theoretical and philosophical lens of the fourth question used the
synthesized data to provide policy recommendations for a growing and underserved
student population.
Quantitative Research Question: Does a seven-week summer intervention for
English Language Learners influence longitudinal growth (growth toward grade level
proficiency standards over multiple years) in reading comprehension, English language
acquisition and statewide normed assessment outcomes?
Qualitative Research Question: What were the English Language Learner
teachers’ perceptions of students’ academic outcomes after students participated in a
seven-week summer intervention?
Transformative Research Question: What educational policy considerations can
be derived from the outcomes of this research?
Method Overview
The type of study design that best suit the research was a convergent
transformative mixed methods design with predetermined research questions. Creswell
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
7
and Plano Clark (2011) noted, a convergent model is incorporated into transformative
design with attention to appropriate use of social justice language. This study design was
convergent transformative in the merger of quantitative and qualitative during the data
analysis phase and in the enhanced understanding of inequities in educational
interventions for English Language Learners. Mixed methods were defined as research
where the “researcher mixes or combines quantitative and qualitative research
techniques, methods, approaches, concepts or language into a single study” (Ponterotto,
Mathew, & Raughley, 2004, p. 43). The language of social justice is explained as
“investigators striving to simultaneously promote human development and the common
good through addressing challenges related to both individual and distributive justice.
Social justice research includes empowerment of the individual as well as the active
confrontation of injustice and inequality in society because they affect research
participants as well as those in their systematic contexts” (Ponterotto, Mathew, &
Raughley, 2004, p. 44).
A convergent transformative mixed methods study was selected to address the
lack of research that merged qualitative and qualitative data on the topic. A mixed
methods research design brings a richness and depth to the conversation that cannot be
accomplished through empirical or qualitative data alone. A transformative-based
theoretical framework is used to advance the conversation on educational outcomes for
English Language Learners, with specific recommendations for changes based on the
research outcomes. Creswell and Clark Plano (2014) further explained, triangulation of
the qualitative and quantitative data sets provides greater validity through “mutually
corroborated” (p. 62) findings. A mixed methods study offers a completeness or
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
8
comprehensive explanation of results that would not be accomplished through a single
methodology.
A challenge of this research method, and in particular this study, were ethical
considerations in addressing language and culture of participants in a neutral method of
discourse. To accomplish this, many definitions and descriptions were taken from the
Encyclopedia of Cross-Cultural School Psychology (Clauss-Ehlers, 2010), to provide a
broadly accepted reference tool to discuss the many dimensions of cross-cultural
language experience in the context of a school setting. In addition, to overcome bias or
leading interview questions, the qualitative data questions were drafted and reviewed by
an advisory board, including English Language Learner professionals. These challenges
related to the qualitative data phase, but attrition was a potential problem during the
quantitative phase, due to the high mobility of the population studied.
Definition of Terms
This research, the educational field, and in particular English Language Learner
education, has an abundance of technical and academic jargon that is specific to this field
of study. The following terms were used operationally within this study and included:
Assessing Comprehension and Communication in English State-to-State for
English Language Learners (ACCESS for ELLs): “a secure large-scale
English language proficiency assessment given to Kindergarten through 12th
graders who have been identified as English Language Learners (ELLs). It is
given annually in WIDA consortium member states to monitor students' progress
in acquiring academic English” (WIDA, ACCESS for ELLs Summative
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
9
Assessment, para. 1). For the purposes of this research, the scope will be limited
students in kindergarten through fifth grade.
Convergent Mixed Methods: research design method that merges data collected
from the quantitative and qualitative phases during data analysis to respond to the
research questions (Creswell, 2012, Creswell & Plano Clark, 2011).
English Language Learners (ELL): a student whose home or first language
(L1) is a language other than English and English language acquisition is a second
language (L2) that needs to be developed along with their first language. “ELLs
are individuals whose language proficiency limits their access to education”
(Solano-Flores, 2010, p. 427). Synonyms, as they may be used in the literature
review, include: second language learners, English Learners (EL), limited English
proficient, language minority, non-English speaking.
Fountas and Pinnell Benchmark Assessment System: researchers Fountas and
Pinnell developed a continuum of specific reading and comprehension
“understandings that students must be acquire to become highly effective users of
oral and written language” (Fountas & Pinnell, 2007, p. 1). The Fountas and
Pinnell Benchmark Assessment System is a common, grade level reading
comprehension assessment to measure the skills proficient readers have a specific
grade levels. After analyzing assessment results, educators have targeted skills to
use for guiding student instruction.
Intervention: any educational strategy used to improve the academic and/or
language skills of students that have not met grade level proficiency or mastery on
specified skills or instructional standards. Interventions can occur during any
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
10
point of a child’s academic career and can occur during the school day,
afterschool, during the summer, can be mandatory or voluntary with the intent of
improving academic or language achievement.
Language acquisition: movement toward English language proficiency in the
four language domains: reading, writing, speaking and listening, as assessed
through standardized protocols across multiple content areas aligned with grade
level academic content vocabulary.
Longitudinal learning outcomes: researcher collects data about trends and
changes in learning outcomes within the same population over a one year, two
year, three year or four year time period (Creswell, 2012).
Missouri Assessment Program (MAP): “Missouri Assessment Program
assesses students’ progress toward mastery of the Show-Me Standards which are
the educational standards in Missouri” (Missouri Department of Elementary and
Secondary Education, Grade level section, para 1). For the purposes of this
research, the assessment will include third through fifth grade English Language
Arts scores, represented as a scaled score and proficiency level.
Norm-referenced Assessment: assessments used to “measure a person’s
performance in relation to the performance of others on the same test” (p. 689)
and are reported as age or grade equivalents and as percentiles (Chang & Kim,
2010).
Reading comprehension: the full range of skills necessary to develop literacy,
including, but not limited to: letter/sound correspondence, decoding, phonemic
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
11
awareness, fluency, word recognition and vocabulary, and strategies for creating
understanding, using oral language skills as the foundation for literacy skills.
Transformative theoretical/philosophical lens: a methodological framework or
perspective designed “for advancing the needs of underrepresented or
marginalized populations” (p. 96) while being sensitive to the cultural and
linguistic needs of the participating population with the results impacting social
justice for the group studied (Creswell & Plano Clark, 2011).
World-Class Instructional Design and Assessment (WIDA): a consortium of
31 states for the purpose of advancing “academic language development and
academic achievement for linguistically diverse students through high quality
standards, assessments, research, and professional development for educators”
(WIDA, Mission and the WIDA story, para. 3). WIDA designed a standardized
English language proficiency assessment of students speaking, reading, writing,
and listening skills in grade level content and social skills.
Assumptions
The first assumption was the data collected for the research was a representative
sample of the archived data from the population of English Language Learner students in
the study. In addition, there was an assumption that responses received from the English
Language Learner teachers participating in the qualitative phase accurately reflected their
professional opinions and perceptions of students’ past academic performance. Lastly,
there was an assumption that all participants answered openly and honestly to the
interview questions.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
12
Delimitations and Limitations
The primary delimitation of the study is “findings are true for all people in all
times and places” (Bryant, 2004, p. 57). For the quantitative phase, the assessments
represent the archival data of students comprising the various cohort groups, the specific
assessments, and the time period of the investigation. In addition, student cohort groups
were established based on the year of program participation and does not reflect the
number of years participating the intervention. The quantitative data does not project
future trajectory for other English Language Learner students. Data collected during the
quantitative phase reflected the current perception of interview participants at that
moment in time. A limitation of this mixed methods design, in particular the qualitative
phase of the study, are human memory and individual perceptions of events that have
occurred in the past. Since some of the educators interviewed were also teachers in the
intervention, there may be an unconscious tendency to provide responses with a positive
perception.
Significance of the Study
One point of significance rests in adding to the research on intervention
effectiveness for English Language Learners from diverse linguistic and cultural
backgrounds since the majority of previous research has been conducted using Spanish-
speaking English Language Leaners. In addition, the study broadens the base of
knowledge on summer interventions for English Language Leaners. Examining
longitudinal achievement of students’ reading, language acquisition and norm-referenced
assessments after participating in a seven-week summer intervention advances the
discussions about strategies for closing the learning gaps between second language
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
13
learners and their peers. In addition, combining the voice of classroom practitioners with
student performance data offers alternatives for educators and policy makers that work as
advocates for student populations that tend to be marginalized by language and family
resources.
Summary
In summary, the study examined the actual educational outcomes of English
Language Learners that participated in a seven-week summer intervention with the
perceived academic outcomes from English Language Learner teachers. The central
question guided the merger and synthesis of the quantitative and qualitative data phases
to determine commonalities and trends. The research was also guided by independent
quantitative and qualitative research questions. A transformative question was used to
direct the recommendations. The purpose of this study was to explore the relationship
between the learning outcomes of English Language Learners that participated in a
seven-week summer intervention and the learning trajectory after participation. In
addition, the research set out to determine the trends and commonalities between student
learning outcomes and educator perceptions of student performance.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
14
CHAPTER TWO: LITERATURE REVIEW
Introduction
Prior to initiating the seven-week summer intervention for English Language
Learner (ELL) students, the district participating in the study noted distinct academic
trends associated with the academic achievement of English Language Leaner students.
The first trend observed was the regression of English language and reading skills in the
fall following summer break, even if students participated in a traditional summer school
program (Lukens, personal communication, January 2014). If students participated in
summer school, the summer learning break was still seven to eight weeks long. The break
in learning spanned as much as 12 weeks if the students did not attend summer school.
Lukens (personal communication, January 2014) explained, Kansas City was
designated a primary destination for refugee families. As a result, an influx of Burmese,
Iraqi and African refugee families moved into the metropolitan area within a three-year
time period, nearly doubling the districts English Language Learner population. During
that time period, observed anecdotal evidence suggested a second trend among the
English language Learner students: families had limited economic resources and access
to educational resources within the community and were hindered by a cultural and
language divide or limited access to education prior to arriving in the United States
leading to a gap in achievement between English Language Learner students and their
white peers (Lukens, personal communication, January 2014).
In addition, there was a rise in the number of preschool and kindergarten age
children that were not prepared with the English language and academic skills necessary
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
15
to start school (Lukens, personal communication, January 2014). Early academic and
language interventions constitute the third local observed trend.
These three locally observed trends were used as the starting point for the themes
in the literature review. Additional themes examined the trends in accountability
measures, research on instructional calendar models and assessment as a method of
gauging English Language Learners students’ academic success. Each theme will move
from the global perspective of significance on students as a whole and progress to the
themes’ specific significance on English Language Learner students. A level of
interdependence exists between the themes, not in a linear, sequential or concentric
manner, but in the fact each theme can be viewed as variable within the other.
For the purpose of this literature review, electronic databases were queried using
key words from broad themes related to the topic of academic achievement of English
Language Learners, included, but not limited to: summer learning regression or summer
learning loss, the achievement gap, early intervention, modified calendar, accountability
trends and assessment.
Summer Learning Regression
For more than 100 years, researchers have examined the question of summer
learning regression and the answers are as essential in modern education as they were a
century ago (White, 1906 as cited by McLaughlin & Smink, 2010). For the purpose of
this study, summer learning regression or summer learning loss, was defined as the loss
of learning associated with the break in formal instruction that occurs during the summer
as measured by grade level assessments from the end of one academic year to the start of
the next academic school year (Marinez-Lora & Quintana, 2010). A vast knowledge base
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
16
on socio-economic status and its implication on summer learning regression have been
collected (Cooper et al., 1996). A century ago, in the United States, the majority of
immigrants were of European decent and relatively homogeneous (Hobbs & Stoops,
2002). In the interim, the population of the United States has more than tripled and the
population center has shifted westward and southward to reflect the patterns of internal
migration and new waves of immigration (Hobbs & Stoops, 2002). At the start of the 20th
century, “one out of eight Americans was of a race other than white; at the end of the
century, the ratio was one out of four” (Hobbs & Stoops, 2002, p. 71), adding to the racial
and language diversity of the country. With new populations of second language learners
impacting classrooms nation-wide, a new area of study has been added to the existing
knowledge base: summer learning regression among English Language Leaners (ELL)
students. Progressing through the review of literature on summer learning regression, it
was evident how difficult it is to isolate the factors of economic status, race, ethnicity and
native language and the relationship they have with one another.
Regression in the General Student Population
Starting with the perspective of summer learning regression among all students,
the research team of Cooper et al. (1996) noted the average student lost one month of
reading and math skills as gaged on benchmark assessments at the end of the school year
in the spring to the benchmark assessments at the start of school in the fall. Among
students from lower socio-economic backgrounds, however, summer learning loss was
more pronounced. This paralleled the findings in Summer Learning and the Effects of
Learning (Heyns, 1987), that summer could be used as a temporal control for the effect
of family background and cognitive development. As such, all children learned slower
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
17
during the summer, but the “effects of family background and race increase substantially”
(p. 1156) the cognitive inequality and discrepancies in cognitive growth existing in every
school. Heyns (1987) findings aligned with the findings of the Sustaining Effects Study
(SES) and “examined summer learning patterns without the lens of compensatory
education” (p. 1156).
Poverty and Regression
By comparison, the reading scores of middle-class students stayed relatively
stable while the reading scores of students from low socio-economic backgrounds fell
three months behind their more affluent peers. When the data collected by the National
Center for Educational Statistics (2013) on school age children living in poverty is added,
the depth of the issue can be seen in greater detail. “In 2012, approximately 21 percent of
school-age children in the United States were in families living in poverty. The
percentage of school-age children living in poverty ranged across the United States from
11 percent in North Dakota to 32 percent in the Mississippi”…and the “percentage was
higher than it was two decades earlier in 1990” (National Center for Educational
Statistics, Children Living in Poverty section, 2013, para. 1) with much of the poverty
concentrated in the rural south, the urban core and border states aligning with the
immigration patterns that established over the last century (Hobbs & Stoops, 2002).
Poverty and equitable access to educational resources are intertwined in the realm of
public education.
The “Investment Model”
In 2005, Borman, Benson, and Overman applied Becker’s (1981) theoretical
economic research on the “investment model” to summer learning regression. The
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
18
investment model suggests, parents apply various amount of capital: time, money, skills
and beliefs to their children, thus impacting a child’s cognitive development. High-
poverty families, compared to middle-class families, “tend to have fewer educational
resources within their homes and communities to provide opportunities to practice
reading and to learn new literacy skills” (Borman et al., 2005, p. 132), making in-school
learning time more valuable and increasing the potential for summer learning regression.
The findings of Borman et al. (2005) support the theory that children of disadvantaged
backgrounds suffer summer learning regression at higher levels and highlights the
necessity of having access to “material resources, social and psychological capital”
(p. 146) in addition to quality summer programs to create better learning outcomes.
The “investment model” researched by Borman et al., substantiated the “faucet
theory” of access to learning resources previously identified by Entwisle, Alexander and
Olsen (2001) in their research seeking to explain academic achievement differences
between students from middle class and poor families. When school is in session,
students from differing economic backgrounds make academic gains at a similar pace.
During the school year, the school resource faucet is turned on equally for all students
and turned off during the summer months, with poor families lacking the ability to
provide the same level of learning resources as provided by the school. A misconception
noted by the Entwisle et al. (2001) team was family resources only make a significant
difference in academic progress through the summer months when family resources are
needed to fill in the gap.
When looking at the influence of socio-economic status on summer learning as
reflected in the transition from kindergarten to first grade, the research follows a similar
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
19
pattern (Burkam, Ready, Lee, & LoGerfo, 2004). For sociologist, the relationship
between the start of formal education and family socio-economic status are inseparable
factors in the rapid cognitive growth of young children (Burkam et al., 2004). The
research of Burkam et al. (2004) identified family socio-economic status was strongly
associated with achievement and students from higher socio-economic backgrounds
outperformed their peers and participated in a greater number of activities as the family
socio-economic status increased. In references to the social justice lens of the
transformational research question, Burkam et al. (2004) identified less than 5% of the
higher socio-economic status and 16% of the lower socio-economic status students as
non-English speaking and a combined 60% of the lower economic status students were
black or Hispanic, strongly associating home-language and race with socio-economic
status.
The disadvantage can be seen in areas as simple as the number of words spoken to
children. Reardon (as cited in O’Brien, 2014) noted affluent families spend more time
talking with their children during some of the most critical developmental years, helping
build essential vocabulary skills. Often, up to three hours per week more than lower
income families. These statistics underscored the connection between race, economic and
language status and the subsequent impact of summer learning regression. In addition,
even among the youngest school age children, the link between access to resources
outside of school is evident in academic and language progress and growth. Allington
(2012) noted a similar discrepancy among struggling readers. Struggling readers did not
read the same high volume as more proficient readers, identifying limited home and
school resources as the source for reading deficiency. Allington (2012) expressed that
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
20
middle-income families averaged 199 books at home and 392 books in classroom library
while lowest income families averaged 0.4 books at home and 47 book in classroom
libraries (p. 87) drawing a stark contrast in the allocation of family resources available for
students learning over the summer and how limited volume of reading can exacerbate
summer learning regression.
Socio-Economic Status and Summer Learning Regression Outside the United States
The impact of socio-economic status on summer learning is a phenomenon
observed outside the borders of the United States as well. In a study of the summer
learning effect conducted in New Zealand, the research team of Jesson, McNaughton, and
Kolose (2014) noted, after five years of compulsory primary school education, the
cumulative effect of summer accounts for “more than half of the overall differences in
literacy achievement between low and high and contribute to on-going inequity in high
school placement, dropouts and college attendance” (p. 1). Recognizing summer learning
regression leads to the question of summer learning regression and the long-term
academic trajectory of ELL students.
Long-Term Impact of Regression on English Language Learner Students
The United States Department of Education (2013) noted the growth in the
number of students receiving English Language Learner services across the United
States. During the 2002-2003 school year, approximately 4.1 million students received
additional instructional support to learn English. Less than a decade later during the
2010-2011 academic years, nearly 4.7 million second language learners received
additional language support services. In Missouri, the English Language Learner student
population increased over 60% during the same time period (National Center for
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
21
Educational Statistics, 2013). In addition, reading scale score data collected by the
National Association of Educational Progress (NAEP) for the Department of Education
from 2002 to 2011 showed ELL students are 36 points behind their non-English
Language Learner peers in fourth grade and the gap increased to 44 points by the time
students reach eighth grade, suggesting summer learning regression has the same impact
on English Language Learner students as economically disadvantaged youth.
The research team of Patton and Reschly (2013) examined beginning of the year
and end of the year Reading Curriculum Based Measure (R-CBM) scores of 317 students
in second through fifth grades. The team disaggregated data by ethnicity, language,
family income and special education participation. The findings indicated a significant
reading level loss for students in second and third grade across all subgroups. “Even a
small drop over the summer could require weeks or even months to recover, which
creates a lag when students start school in the fall to when they exhibit growth beyond
their previous spring level” (Patton & Reschly, 2013, p. 748). English Language Learner
students start with a deficit due to language. In addition, limited language experience
paired with the lack of opportunity to practice the emergent language can intensify
learning loss over the summer months (Borman et al., 2005).
The literature pertaining to summer learning regression suggested learning loss
has compounded negative effects in long-term academic outcomes (Entwisle et al., 2001;
Alexander, Entwisle, & Olson, 2007b). Among English Language Learner students, the
setbacks tend to be greater in part to inadequate access to reading or other educational
resources within the community. In a report by The Education Trust (2014, June), Latino
students are the largest segment of the English Language Learner population, but perform
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
22
below their non-white peers. The report highlighted data from the National Association
of Educational Progress (NAEP), that “in 2013, 19% of Latino students read at advanced
or proficient level on NAEP, compared with 45% of white fourth graders” and “white
eighth-graders were over twice as likely as Latino students to be proficient or advanced in
math” (p. 6). The Education Trust further explained, that while gains are being made,
only three out of ten Latino students took advanced placement math, only four out of ten
took advanced placement science, and less than half of the high school graduates met
college readiness benchmarks on the ACT.
Research conducted by Rojas and Iglesias (2013) on language acquisition
trajectory of Spanish-speaking English Language Learner students using summer break
as one “time-varying predictor” (p. 631) noted, “contrasting patterns of change over time
resulted in trajectories with periods of discontinuous growth during the summer” (p. 631).
The research tracked primary (Spanish) and second (English) language learning
simultaneously and the outcomes noted similar growth rates in both language and
reduced growth in English during the summer.
In contrast, Kohn (2012) expressed summer regression is a result of
misrepresentation and misidentification of a number of factors and are the result of a
flawed, “traditional” education process. First, students from lower socio-economic
backgrounds demonstrated learning loss when they are compared against their more
affluent peers, rather than comparing against a similar peer group. Kohn (2012) also
suggested relationship between summer learning regression and achievement gap
research is biased due to the continued reliance on standardized assessments and
assessments that focus on factual and process-based knowledge. The traditional education
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
23
model is broken, as noted by Kohn (2012) and a reliance on lecture as the primary
method of instructional delivery, outdated textbooks, inconsistent alignment between
grading practices and curriculum, outmoded assessments and incongruous homework
bears the responsibility for creating summer learning regression, not the time off
associated with the break.
In general, the research suggests there is a relationship between a two-month
summer break and cognitive learning loss in many different academic content areas. The
effect of summer learning regression is compounded due to educational resource equity
in the English Language Learner students’ community and schools. Across the literature
reviewed, the ability or inability to fully access instruction due to English language
proficiency has lasting long-term implications and leads to gaps between second
language learners and their English-speaking peers.
Other Contributing Factors
One contributing factor has been access to summer school programs. Summer
school has long been either voluntary or designed as a limited remediation program with
uneven distribution and access. “Racial differences are also apparent; most studies find
that the racial group most likely to attend is white children, followed by African-
American children, with Latino children attending at lower rates” (Miller, 2007, p. 8). In
addition, the quality of summer learning experiences in and out of school, access to
educational resources within the community, family socio-economic status, and the level
of student/family engagement in learning, subject matter and age of student (Heyns,
1987; Cooper et al., 1996; Alexander et al., 2007a; Borman et al., 2005; Fairchild &
Boulay, 2002) play a role in summer learning regression.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
24
A report by Miller (2007) for the Nellie Mae Educational Foundation noted that
students in all socio-economic groups learned at about the same rate through the school
year, but inequities outside of school contribute to the learning regression gap as a result
of poverty, violence and discrimination within the community. As well, unintended
institutional racism, language barriers, low teacher expectations for achievement, and
lack of same-race or same language role models account for achievement differences in
elementary students, allowing students to get further behind in the early developmental
years and leading to fewer high level education opportunities and increased dropout rates
(Miller, 2007). For English Language Learners, these factors, in addition to cultural
beliefs and language add additional complexity to the issue (Graves, 2004; Burkam et al.,
2004; Rojas & Iglesias, 2013). For English Language Learners, the consequences of
summer learning loss are often visible in the academic achievement differences among
English Language Learner students and their white peers.
The Achievement Gap
Socio-economic and second language learner status has been linked in research as
contributing factors to summer learning regression. The body of research on the
achievement gap has only recently moved beyond historic notions of socio-economic
status and ethnicity as the only factors impacting the achievement gap. More recently,
acculturation, cultural beliefs and other psychological factors of associated with
belonging and fitting into in a new culture have become factors informing the
conversation about the achievement gap among English Language Learners (Hernandez,
2010). In the literature, however, the factors of socio-economic status, second language
learning and ethnicity are difficult to isolate, so for the purpose of this research, the term
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
25
“achievement gap” will be referred to and defined as differences or gap in academic
achievement among minority and/or English Language Learner students in comparison to
their white, native English speaking peers using a variety of academic achievement
measures.
Three Views of the Achievement Gap
The academic achievement gap theme will be viewed from three distinct
perspectives. First, Nisbett’s (2011) research examined the historic context of intelligence
testing and current system of tracking academic performance by race as a result of No
Child Left Behind legislation. Nisbett (2011) explored existing research and systems that
have been created to eliminate the achievement gap.
Second, researcher Chambers (2009) noted poor, minority, low achieving and
second language learner students do not receive the same psychological support as their
higher achieving peers, resulting in an accepted “culture of separation” that contributes to
the achievement gap. Students that receive support services, remediation, English
language services, etc., get used to being separated from their higher performing peers
and start to believe they are not as capable and do not receive the same affirmations about
what is necessary to be a successful student.
Finally, the research of Cohen (2006) added another dimension to the
psychological thread that runs through the other two factors in the achievement gaps:
positive affirmation for low performing students. The research suggests the link between
issue of under-performance and the relationship between group-identity in an educational
setting. Group identity is a component of acculturation among English language learners
(Hernandez, 2010).
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
26
A Historical Perspective
Nisbett’s (2011) research focused on the historical research of IQ testing and the
race/ethnicity misconceptions that were created by this early research. He asserted the
idea of the achievement gap has been perpetuated by these misconceptions, but it is
highly probable it can be eliminated in a very short period of time, three years, with
minimal change to the status quo.
Nisbett (2011) noted early IQ testing supported the idea that Black IQ was lower
than White IQ by about 15 points and the basis for this difference is genetic. Over time,
research examined the possibility of differences in socio-economic status for the disparity
in IQ scores between races. As reported in Nisbett (2011), several studies were conducted
to determine the effect of home environment on IQ. These included Black and Multi-
ethnic children adopted by Black and White families to see if there was a difference. For
the most part, there was not a difference in IQ. Moore (as cited in Nisbett, 2011) noted
Black and Multi-ethnic children adopted by middle-class families had increased IQ,
irrespective of the race of families. One of the key findings was identifying “within each
race, prior knowledge predicted learning and reasoning, but between races, only prior
knowledge differed, not reasoning ability” (p. 93).
This evidence led to the research of political scientist James Flynn. Flynn (as cited
in Nisbett, 2011) discovered that the IQ of the developed world had increased by 18
points from the mid-twentieth century to the 2000. Flynn expressed, “Our genes could
not have changed enough over such a brief period to account for the shift. The only
plausible explanation is that it was the result of powerful environmental factors” (Nisbett,
2011, p.93). There were big gains in all subgroups, such as English Language Learner,
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
27
during this period as well. During that time, the environmental influences for everyone
improved, and the research by the National Assessment of Education Progress (NAEP,
2009) supports that academic performance for Black students has improved at higher
rates than white students during this time period. This dispels the myth of black genetic
inferiority created by early IQ tests results. In the NAEP (2009) study, the Black student
subgroup encompasses native English speaking and non-English speaking students.
In addition, Nisbett (2011) examined several programs that have been successful
in narrowing the achievement gap, in particular, Jaime Escalante’s math program in East
Los Angeles, the Achievement First Schools and KIPP, a not-for-profit school system
that focus on students in low-income and minority neighborhoods, frequently with high
immigrant and English language learner populations. Nisbett (2011) also examined
voucher systems and the limited effect they had on student performance. Stanford
University psychologist, Dweck (as cited in Nisbett, 2011) identified the reason for the
limited effect of vouchers as voucher students were not part of the dominant culture due
to ethnic, racial, language or socio-economic differences and did not feel like they fit in.
The social cognitive development research of Olson and Dweck (2008) identified that
intelligence had been viewed as a fixed, innate trait. Olson and Dweck (2008) study
proposed that achievement motivation is highly influenced by positive environmental
factors, such as positive praise of effort rather than personal characteristics and impacted
life-long perception about achievement ability in an affirmative manner in poor, minority
students.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
28
Student Tracking and the Gap in Achievement
The phrase “achievement gap” is loaded with assumptions about the educational
outputs and connotations of Black and Latino students’ inability to achieve in an
academic setting (Chambers, 2009). Chambers (2009) expressed the phrase “achievement
gap” puts the emphasis on the students’ lack of performance without placing any
responsibility for the learning process on the educators. In order to change that
perception, Chambers (2009) endorsed the notion of using the expression “receivement
gap” in order to understand the disparity in school finance, “lack of quality teachers,
residential segregation, access to technology, quality pre-school preparation, and home
support” (Chambers, p. 418) for Black and Latino students as indicators of a lack of
inputs that contribute to the gap in learning.
Chambers (2009) further suggested the tracking system established by many
districts and schools, often starting in elementary school, accentuates the dominant
culture divide by normalizing student segregation through English Language Learners
(ELL) pull-out services (when students are pulled from general education classes to
receive English language remediation or interventions), reading support for struggling
readers and special programs offerings for gifted education. The “normalization of
isolation” process actively discourages assimilation due to a “label” and serves to
heighten institutional differences in classroom environments and teacher management
styles (Chambers, 2009).
In addition, Chambers (2009) conducted a case study of a high school with an
informal three-track system: the bridge group, composed of students in an alternative
educational environment in the main campus building; a high-track group (frequently
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
29
referred to as college prep, honors, advanced or gifted course work in high school); and a
regular-track group (general education classes that lack advanced levels of rigor, but meet
the state requirements for graduation). Over one school year, the researcher interviewed
students and recorded observational notes on a weekly basis, as well as conducted
document analysis of student handbooks, registration materials and yearbooks to
substantiate peer group associations. During analysis of the data, one particular theme
became apparent in the conversations between participants. Assimilation or lack of
assimilation was a relevant component of the achievement gap. “This theme details a
process in which some students were actively encouraged to become part of or assimilate
to the school culture through the participation in special academic programs with elite
teachers, while others were discouraged or denied participation” (Chambers, p. 422). This
assertion was substantiated through accounts detailed by students. It was noted the
normalization of separation by educational programs, such as students in English
Language Learner programs, caused students to continuously move in isolated groups
socially and the stratification accelerated throughout high school.
Chambers (2009) conclusions included: (a) test results as achievement markers
alone are not adequate indicators of academic performance; (b) students have little
influence in selecting their track and early decisions about the resources students receive
impact their view of their ability; (c) focusing on standardized test scores of students
offers a post-mortem view of achievement; and (d) the effect is compounded yearly and
creates a self-fulfilling prophecy of failure. The conclusions align with the research of
Howard (2006) with respect to white privilege in public education and the students’
ability or decision to assimilate to with the majority culture. This research implicated the
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
30
reliance on standardized testing as the primary metric for gauging academic success and
normalization of segregation through specialized educational programs as the leading
reasons for identification of the achievement gap (Chambers, 2009).
Influence of Positive Culture on Gaps in Achievement for ELLs
The normalization of separation for students that receive educational support
services, such as English language support, reinforced negative impressions of the
students’ own ability and institutionalized impressions of student ability (Chambers,
2009). However, can positive social-psychological interventions aimed at eliminating
negative stereotypes reaffirm self-adequacy and close the achievement gap? Timely and
targeted self-affirmations lessen perceived negative threats from testing and raise overall
student academic performance among African American students in data collected by
Cohen, Garcia, Apfel, and Master (2006). Cohen et al. (2006) conducted “two
randomized, double-blind field experiments and a second replication with different
cohorts of students” (p. 1307). In the study, students were asked to indicate their most
important value and wrote why these values were important to them. “Based on their
official transcripts, African Americans in the affirmation condition earned higher fall-
term grades in the targeted course than did those in the control condition” (p. 1308).
Cohen et al. (2006) noted no change in European American students grades in either the
affirmation group or the control group. The average benefit for African American
students, averaged over both studies, was nearly 70% improvement (Cohen et al., 2006).
The previously poor performing students had the greatest gain in performance. The
research also showed affirmation group students performed better not only in the targeted
courses, but is all courses.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
31
The research of Cohen et al. (2006) revealed several significant implications
• there is a negative cycle of poor performance associated with perceived
testing threat of failure;
• a small reduction in perceived testing threat leads to sustained performance
gains;
• small improvements have a cumulative effect; the psychological effect of
gains has a positive impact on social experiences and elimination of perceived
environmental bias; and
• with necessary resources, a brief affirmation event starts a chain reaction of
positive social-psychological changes permitting sustainable academic
growth.
Educational outcomes for English Language Learners, faced with learning
English along with grade level academic content, face a gap in achievement in
comparison to their English-speaking peers. The Migration Policy Institute (Flores,
Batalova, & Fix, 2012) conducted a research study of “Ever-ELL” students in Texas for
the purpose of examining learning trajectory. “Ever-ELL” was defined as “students who
were ever classified as English Language Learner students who entered Texas schools in
the first grade and who advanced to the 12th grade” (Flores et al., 2012) at a rate of one
grade per year or “on time” to be compared against non-English Language Learner peers.
What the team discovered for the population studied was Hispanic, Asian, and
Black students were more likely to be economically disadvantaged with over 90% of the
Hispanic student receiving free and reduced lunch (Flores et al., 2012). This is significant
since English Language Learner students scored lower than their non-English Language
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
32
Learner peers in reading and math, at every grade from third through eleventh, on state
mandated accountability assessments of grade level proficiency. While there are points in
the English Language Learners academic careers where they are close to closing the
academic gap, they lag their non-English Language Learner peers their entire academic
career and putting students at risk for dropping out (Flores at al., 2012). The study also
examined academic performance in reading and math based on length of time receiving
English Language Learner services.
The research demonstrated the academic trajectory of students that exited English
Language Learner services early, within three years, was higher than their non-English
Language Learner peers in both math and reading. The longer the time frame a student
received services, the lower the performance, with sporadic periods of growth and decline
across their academic career (Flores et al., 2012). In the study, race and ethnicity of
English Language learners had a high correlation to academic performance, high school
graduation, and post-high school outcomes. One notable outcome was that English
Language Learner classification does not translate to poor academic outcomes over the
entirety of an academic career.
Intervention
Educational interventions take many forms: summer school, after-school, and
during the day. They also occur for a variety of reasons, including remediation or
enrichment. However, several factors with the most impact are the quality of the
programming and intervening at the earliest possible age (Heyns, 1987; McCombs,
Augustine & Schwartz, 2010; Alexander et al., 2007a; Miller, 2007; Allington et al.,
2010) will be discussed as part of this theme.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
33
High quality early educational interventions lead to better long-term academic
gains and increased cognitive development among children living in poverty, but low
income families are less likely to access educational programs for their children (The
Annie E. Casey Foundation, 2013). Meeting developmental milestones are essential for
success in school and life, with below average language and cognitive skills likely to
follow a student through their academic career (The Annie E. Casey Foundation, 2013).
Data collected by The Annie E. Casey Foundation (2013) demonstrated the level of delay
among third graders across the country: 48% of white, 14% of black, 19% of Hispanic,
and 19% of low income students meet cognitive knowledge and skills criteria in reading,
math and science by third grade. By fourth grade, 83% of black, 81% of Hispanic, 49%
of Asian/Pacific Islander, 78% of American Indian/Native American, 61% of multi-
racial, and 93% of dual-language learners score below proficient in reading (The Annie
E. Casey Foundation, 2014), demonstrating the need for early intervention in cognitive
and language skills.
Early Interventions
Data collected by the Casey Foundation (2014), as mentioned in the previous
paragraph, noted the speed in which students from low income families fell behind their
more affluent peers, emphasizing the opportune time to intervene is while the academic
gaps are smallest and students are at the youngest age possible. Poverty is a factor
limiting emergent language skills and early literacy (Snow, Burns, & Griffin, 1998) in
both native and second language acquisition. Children living in poverty and acquiring a
new language have an increased likelihood of reading difficulties and negative academic
outcomes (August & Hakuta, 1997). To increase academic preparedness in student
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
34
entering kindergarten, in 2002, the United States Department of Education instituted the
Early Reading First and Reading First programs, as part of the No Child Left Behind Act
of 2001, with the goals of developing centers to serve low-income families in preparing
“early language, cognitive and pre-reading skills for continued school success” (Program
Description section, para. 1) in pre-school and kindergarten age students. Entwisle and
Alexander (1998) described the importance of early intervention for pre-school students
to reach cognitive development benchmarks. In addition, the researchers noted the role
school environment plays in students achieving critical academic milestones, especially
among young students.
Early Intervention in Language and Literacy
The research team of Wilson, Dickinson and Rowe (2013) examined the language
and literacy outcomes of monolingual English speaking and non-English speaking
students from similar socio-economic backgrounds that participated in the Early Reading
First program. The Early Reading First program was established as an initiative under the
No Child Left Behind Act of 2001, to provide targeted funding to support the language
development needs of low-income and non-English speaking families prior to entering
school. The program provided professional development for staff, an evidence based
literacy-focused curricula for students and materials to combat the issues of inequity in
socio-economic background, cultural differences in beliefs about education, variances in
community or school resources and home language. Wilson et al. (2013) found English
Language Learner pre-school students that participated in the program “began
kindergarten at the same level (or better) than the non-English Language Learners”
(p. 586) using the Woodcock-Johnson Letter-Word Identification skill assessment and in
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
35
the three cohort groups of English Language Learners, the “program produced
statistically significant effects on letter-word identification” (p. 586).
Beyond increases in letter-word identification, the Early Reading First program
demonstrated improvements in English Language Learner students’ receptive vocabulary
(aural identification), expressive vocabulary (oral vocabulary) and spelling. Only one of
the three cohorts demonstrated significant improvement in oral comprehension. Overall,
the research findings of Wilson et al. (2013) would support the notion of including pre-
kindergarten students in an extended summer program for ELL students with a high-
quality, literacy rich curriculum prior to entering kindergarten. The research revealed the
success of early intervention in pre-school, but is the impact intervention as significant
when an ELL student is a low reader in first grade?
First Grade Intervention in Language and Literacy
Wilson et al. (2013) noted evidence of significant gains with pre-school
intervention through the Early Reading First program. However, as students age, are
academic and language interventions as impactful? Kelly, Gomez-Bellange, Chen, and
Schultz (2008) examined the issues for early interventions for English Language Learner
below reading level in first grade. Using Reading Recovery®, a commonly used
intervention system, the researchers used pre and post assessments measures to track
literacy outcomes among native and non-native English speakers. The measures tracked
phonemic awareness, one of the skills necessary for early reading skills
August and Hakuta (1997) suggested early intervention is more important than
language instruction for non-English fluent students and improves overall language
proficiency. The research results from Kelly et al. (2008) indicated that first grade
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
36
English Language Learner students receiving early intervention “reached average reading
levels relative to their classroom peers” (p. 252). In addition, the results were consistent
with students learning English as a second language in England and New Zealand.
Researchers conducted “in England and in New Zealand reported similar rates of success
for English Language Learners as for native English speaking children” (Kelly et al., p.
240). As well, Entwisle and Alexander (1998) described pre-school through first grade as
critical for language and cognitive development as students are instructed in increasingly
larger groups as they approach intermediate levels of elementary school, coinciding with
immense changes in students’ social environment and self-perception.
Native and English Early Oral Language Intervention and Long-term Growth
Spanish is one of the dominant native languages within most school districts
(United States Department of Education, 2014). As a result, an understanding of the
connection between early oral language and reading development among Spanish-
speaking English Language Learner students and long-term academic growth is a
consideration for instructional policy decisions. Alexander et al. (2007b) noted the lasting
impact of learning regression among non-English fluent students and the need for
opportunities that neutralize the effect of limited educational resources outside of the
classroom and early instructional intervention in oral language is one potential
opportunity for educators to equalize educational outcomes. Kieffer (2012) explored this
connection between early oral language development among Spanish-speaking students
and long-term growth.
The National Early Literacy Panel (2008) identified oral language, in both first
and second languages, as one of the best predictors of reading ability as student mature.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
37
Kieffer (2012) tested the oral language predictability assertion, exploring Spanish and
English language oral proficiency relationships against later reading comprehension
levels. Kieffer’s (2012) findings, controlling for socio-economic status, noted the effect
size (.20-.29) indicated a small to moderate relationship between early oral language
acquisition, in pre-school and kindergarten, and reading ability in later grades and is
comparable to native English speaking peers (p. 153). This held true for oral language
skills in native language as well as in the second language of English, but English
language expressive vocabulary had the highest correlation to success. This indicates the
need to ensure productive oral language opportunities are part of kindergarten and
preschool curriculum to cultivate oral language skills of native English speakers and
second language learners.
Trends
Nationally and locally, educators and policy makers are responding to trends in
accountability legislation and new instructional standards, increasing instructional
minutes through differentiated calendar structures, and assessment as a tool for gauging
academic achievement. Accountability has taken on new importance with measures to
ensure achievement for all students. Suzuki, Ngo, and Kuger (2010) noted the historical
context of using linguistic and cultural “disadvantage” as a method of explaining subpar
performance of English Language Learners and ethnic minority students. In addition,
English Language learner students had been excluded from accountability measures or
were not held to the same academic standards as their peers as a result of educators’
thinking about second language learners (Coltrane, 2002).
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
38
Accountability
Accountability for student achievement has been at the forefront of educational
policy with approval of the No Child Left Behind Act (2002), but it is not a new
discussion. The purpose of the legislation is to “ensure that all children have a fair, equal,
and significant opportunity to obtain a high-quality education and reach, at a minimum,
proficiency on challenging State academic achievement standards and state academic
assessments” (NCLB, 2002). Key features of the legislation include: common
expectations for all students, meeting the needs students in high-poverty neighborhoods,
English language learners and students with disabilities, and closing the gap between
minority students and non-minority students through the use of high quality, rigorous
assessments of the states’ design. Despite the efforts of NCLB, “during 2005-2006, 71%
of all eight grade English Language Learners who were assessed achieved below basic
levels, 24% scored at basic levels, and 4% scored at proficient levels in standardized
reading assessments” (DaSilva Iddings, 2010, p. 970). More recently, states have been
adopting the Common Core State Standards (CCSS) or a local version based on the CCSS,
as the model to increase accountability, uniformity and consistency across grade level and
content curriculum from state to state and as a curricular mandate, is largely outside of the
scope of this research and the literature review.
Missouri’s new accountability standards are Missouri School Improvement Plan 5
(MSIP 5) and include some significant changes. MSIP 5 (2011) changed two of the
accountably criteria to reflect the federal criteria, focusing efforts on academic
achievement and subgroup performance. MSIP 5 (2011) requires school districts to
demonstrate improvement over time on the state department of education assessment and
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
39
meet growth goals on assessments. Missouri also created a super subgroup comprised of
students qualifying for free and reduced lunch, English Language Learner students,
students with disabilities, Latino/Hispanic and Black students. In the past, if subgroup
categories were under a certain threshold, the subgroup did not have to be counted and
some students were counted under multiple subgroups. With the current accountability
measures, all students are counted once if they meet any subgroup criteria. All students
are expected to meet the same grade level performance standards, with the exception of
English Language Learner students in the United States less the one-year.
The literature on accountability dates back to the turn of the twentieth century and
as noted in NCLB (2002), closing the gap in academic performance between white and
non-white students is one of the goals.
Academic Calendar
As noted earlier in the research of Entwisle, Alexander and Olsen (2001),
examining the academic calendar relates to the analogy of education as a “faucet” that is
turned off for many students during the summer. Cooper et al. (1996) identified the
structure of the current academic calendar as a result of combining the different
community based school calendars in use before the turn of the twentieth century. School
calendars were designed to meet the needs of the community. A rural school’s academic
calendar was fashioned around harvest and planting seasons and students would attend
school for five to six months of the year. The urban school schedules during the same
time period were typically on an 11-month instructional calendar (Cooper et al., 1996).
However, shortly after the turn of the century, “the implementation of standardized grade
level curricula created pressure to also standardize the amount of time students spent in
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
40
school. The present nine-month calendar, under which schools are closed in summer,
emerged as the norm when 85% of Americans were involved in agriculture. Today, about
3% of Americans’ livelihood is tied to the agriculture cycle, but the school calendars
have not changed” (p. 228).
The current 180-day calendar adopted by most states and districts, for the purpose
of this discussion will be defined as the traditional school calendar. Johnson and Spradlin
(2007) noted, “the traditional 180-day school year is largely a given. Some states vary on
the exact number of days, but most are reasonably close to the 180-day gold standard,
with 29 states and the District of Columbia requiring exactly 180-days” (p. 2). Year-
round school, as defined by Johnson and Spradlin (2007), “redistributes the school days
throughout the year, eliminating a long summer vacation in lieu of shorter breaks” (p. 3).
In essence, both year-round calendar and traditional calendar models consist of 180 days
with instructional and vacation days spread at various intervals throughout the year. The
definition of extended year calendar is adding additional instructional days. Delineation
of calendar characteristics by type is detailed in table 2.1.
Table 2.1
Characteristics of Academic Calendar Types
Calendar type Number of Instructional days Length of vacation calendar days before break Traditional 180 days varied 1 long (30–60 days) Multiple short (1-2 wks) Multi-track (45/15) 180 days 45 days 15 days Multi-track (60/20) 180 days 60 days 20 days Multi-track (90/30) 180 days 90 days 30 days Extended year 210 days varied multiple short (1-2 wks)
Note. Adapted from “Year-Round Education Program Guide,” by California Department
of Education (2013).
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
41
The research around the success of year-round education is mixed. In one study
conducted by Bradley McMullen (2001) in a statewide evaluation of North Carolina
found student performance in year-round schools was no higher than student performance
in traditional calendar schools. “Initial analyses indicated no statistically significant
differences in either reading or mathematics achievement” (p. 72) but acknowledged that
socio-economic homogeneity of multi-track schools created differences in academic
achievement.
A longitudinal study conducted in California found single-track and multi-track
year-round calendars had significant negative impact on standardized test scores of low
socio-economic status, limited English proficient, Hispanic and Latino, and African
American students. Graves (2010) noted the “importance of understanding unintended
impacts of school programs on academic achievement of disadvantaged and minority
students” (p. 1281) and the decision for year-round school calendars should not be based
solely on combating summer regression since the total time spent outside of school
remains the same as traditional calendars. Other considerations are academic
environments, including the quality of classroom instruction and effective use of time
resources, such as interventions. “A common thread that runs through the motivation
behind each of these studies on disadvantaged student subgroups is their different
exposure to better or worse academic environments” (Graves, 2010, p. 1283). Both of
these studies acknowledge that there may not have been gains for students in year-round
schools, and factors other than the calendar structure could contribute to this result
(Graves, 2010; McMullin, 2001).
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
42
In respect to meeting the diverse learning needs of students, the California
Department of Education After School Division released the Quality Standards for
Expanded Learning (2014) as part of the state’s education strategic plan. In the plan,
California defines expanded learning as “before, and after school, summer, and
intersession learning programs” (p. 3) to meet all the need social and educational needs of
students through quality, student-centered, research based school and community
learning partnership. A key component is acknowledging the differentiated needs of
students and providing the opportunities with sustainable and fiscally responsible
choices.
Learning in Afterschool and Summer (n.d.) noted extended learning aligns with
recent brain research and learning goals associated with the common core and workforce
skills that will be needed in the coming years. The number of out-of-school hours
presents both challenges and opportunities (Carnegie Corporation, 1992; Learning in
After School and Summer, n.d.) for developing students into productive members of
society and overcoming the limitations of the environment in the community. In
particular, Carnegie (1992) noted young people “want more regular contact with adults
who care about and respect them” (p.11) while socializing with their peers and avoiding
loneliness.
The school can be the stabilizing factor for students, when you consider the whole
learning environment for shaping student success. The brain-based learning research of
Cave, Ludwar, and Williams (n.d.) explained the importance of acknowledging home and
community as key aspects of a students’ learning environment. More time at school
mitigates community factors.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
43
Research conducted by Stanat, Becker, Baumert, Ludtke and Eckhardt (2012) in
Germany concluded that immigrant students learning German as a second language
performed significantly better in grammar, vocabulary and reading after participating in a
summer school program focusing on explicit and implicit instruction on second language
usage. The research of Cornelius and Semmel (1982) identified students with learning
disabilities regress in reading when they have extended breaks, but a five-week summer
school reading intervention help recoup the summer learning loss. Cornelius and Semmel
(1982) conclude the placement of the summer program at the beginning of the break had
greater impact than at the end of the summer. The research of Cornelius and Semmel
(1982) points to importance of interventions. The literature on the effectiveness of the
calendar structure is inconclusive, but it does highlight the need for quality educational
programs, instructional practices and effective use of time as ingredients for successful
programing with an emphasis on maximizing cumulative time spent engaged in active
learning.
Time Spent Learning as a Factor
The purpose of attending summer school is to increase the amount of time spent
involved in active learning. Miller (2007) explained “increasing the time that children
spend reading is the single most powerful strategy for improving literacy skills in
fluency, vocabulary and comprehension” (p. 9). Building programs that focus on building
relationships and developing strong racial/ethnic identities are just as important for
counteracting achievement gaps by race and language as countering income related
achievement gaps (Miller, 2007).
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
44
Kelly Gallagher (2009) noted some facts demonstrating the decline in the amount
of time spent on reading: less than one-third of 13 year olds are daily readers, a 14%
decline from twenty years earlier; and students age 15 to 24 spend about two hours a day
watching television, but only seven minutes a day reading. (p. 41). Educational
researcher, Richard Allington (2012) found a similar pattern for learners that are
struggling. Further noted, more time spent reading, at home or in school correlated with
higher reading scores. Increased reading volume, counted as more words, more pages or
more books or more time, resulted in better reading comprehension. Allington (2012)
explained, in the classroom with effective reading instruction, students had upwards of 60
minutes of uninterrupted reading time and summer reading programs with intensive,
expert instruction can minimize summer learning regression and gaps in achievement
while accelerating literacy. Allington (2012) further suggested summer last “eight to ten
weeks” (p. 191) for maximum effectiveness. Research conducted by the Center for
Applied Linguistics (2007) reported on the effects of increased instructional hours for
English Language Learners.
The focus of the Center for Applied Linguistics (2007) study was the relationship
between instructional hours and intensity of instruction on oral language proficiency
gains using the BEST Plus, as English language proficiency assessment tool. The results
of the study indicated, “the greater the number of instructional hours, the higher the
percentage the percentage of students who made level gains” (Young, 2007, p.1) among
students at all proficiency levels. It was also noted, intensity of instruction had the
greatest impact on student at the lowest level of proficiency (Young, 2007).
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
45
Assessment as a Measure of Academic Achievement
Assessment of students from diverse linguistic and cultural backgrounds offers a
number of challenges. One challenge is test and item bias to the extent items or test are
“culturally loaded” in favor of the dominant culture or the test taker would be required to
have a minimum level of knowledge or understanding of cultural values to be successful
(Suzuki, Ngo, & Kuger, 2010). Norming is a second challenge to testing. Students from
diverse backgrounds are not represented in the norming sample in the same proportion as
the actual population being assessed (Suzuki, Ngo, & Kuger, 2010). Both of these
challenges are attributed to cultural variables of acculturation, educational background or
experience, language, and socio-economic status (Suzuki, Ngo, & Kuger, 2010). Some
researchers argue it is necessary to move beyond examining bias and moving toward
implementation of culturally competent assessment.
Culturally competent assessment, in the frame of English Language Learners,
involves using assessment tools that are designed for fluent English speaking peers.
Many testing tools rely heavily on English language ability and fluency, “as a result, the
validity of these instruments may be influenced by a child’s level of acculturation”
(Perez, Harris, Martinez, & Ridley, 2010, p. 336) and does not reflect the student’s ability
to achieve academically.
In a report by the Center for Applied Linguistics (2013) on the implementation of
the Common Core State Standards (CCSS) and assessment of the standards, the report
identified the opportunities presented. One benefit identified was the “attention to
language in the content areas for all students, thus English learners may be more likely to
experience language-focused instruction” (Duguay, Massoud, Tabuku, Himmel, &
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
46
Sugarman, 2013, p. 17). In addition, CAL (2013) noted, English Language Learner
students can meaningfully participate in CCSS instructional activities and assessments,
with proper linguistic support and time, since missing rigorous grade level content
instruction would hinder opportunities for engagement in intellectually challenging
engagement with their English speaking peers.
Summary
The literature review examined summer learning regression, gaps in academic
achievement and early academic interventions among English Language Learner
students, based on the locally observed trends that guided the creation of the seven-week
extended summer intervention. In addition, trends in accountability, school year calendar
structure and second language learning as a factor for gauging the language and overall
academic performance of English Language Learners were reviewed. The literature
supports the cumulative effect of summer learning loss and gaps in learning achievement
between English Language Learners and their native English-speaking peers. The
research suggests early interventions can be a tool in combating summer learning
regression and filling the linguistic and cognitive skill needs of students. Schools are key
in bridging summer learning and the achievement gap by filling the educational resource
void, but only if filled with quality instruction in programs that complement and align
with learning through the rest of the year (Harris & Wallace, 2012). The literature
suggests the ideas that increased accountability standards and changes to calendar
structure are possible tools to fill the summer resource void for English Language
Learner students. The literature review leads to the examination of the learning trajectory
of English Language Learner students that have participated in a seven-week summer
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
47
intervention and the perceptions of academic outcomes of the English Language Learner
teachers.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
48
CHAPTER THREE: METHODOLOGY
Introduction
The intention of the study was to gain insights into the effectiveness of a seven-
week summer intervention for English Language Learners through three performance
measures (English language proficiency, reading comprehension, statewide norm
referenced assessments) and English Language Learner teacher interviews about
perceptions of student performance after participation in the intervention. The purpose of
the convergent transformative mixed methods study was to converge the students’
quantitative assessment data with interview data from English Language Learners
teachers’ about their perceptions of student academic performance to make
recommendations to educators and policy makers for an at-risk student population. In this
approach, archived student assessment data and teacher interviews triangulated the
statistical data with the humanistic data. In addition, a transformative lens was applied for
the purpose of informing policy makers and educators on anticipated outcomes of the
seven-week summer intervention to drive sustainable English language learner academic
performance.
Research Questions
The study was guided by four research questions with each question used to focus
a specific aspect of the study. As a mixed methods design, the central research question,
was answered through the analysis and synthesis of the data collected during the
quantitative and qualitative data phases. In addition, quantitative and qualitative data
collection was directed by specific research questions. There was an overarching
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
49
transformative framework and research question to frame the discussion and conclusion
of the study. The following is a summary the research questions used to guide the study:
Central, mixed methods research question: What were the congruencies between
archived student achievement data and the beliefs of ELL teachers about student
academic outcomes after participation in a seven-week summer intervention?
Quantitative research question: What were the multi-year academic outcomes on
reading comprehension, English language acquisition and standardized state assessments
for English Language Learners students that participated in a seven-week summer
intervention?
Qualitative research question: What were the English Language Learner
instructors’ perceptions of students’ academic outcomes after students participated in a
seven-week summer intervention?
Transformative research question: What are the educational policy
recommendations that can be derived from the research outcomes?
The research questions, in conjunction with the review of literature played a
foundational role in determining the research methodology used in the study. Roberts,
(2010) explained, blending qualitative and quantitative approaches “generally allows
greater depth of understanding and insight than what is possible using just one approach.
Plus, blending helps overcome the biases inherent in each method” (p. 142) while
complementing each other.
Method
Research around extended summer learning interventions of English Language
Learners is limited and revolves around socio-economic status and its impact on
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
50
academic performance (Heyns, 1987; Alexander et al., 2007a, & 2007b; Allington et al.,
2010; Cooper et al., 1996). In addition, many studies focused on either the quantitative or
qualitative side of the research equation, neglecting the complex understanding that
students are a composition of the statistical and human aspects of learning needs. As
such, a targeted study, focused on academic indicators in conjunction with teacher
perceptions would be best suited to a convergent transformative mixed methods design.
Creswell and Plano Clark (2011) noted, a convergent model is incorporated into
transformative design with attention to appropriate use of social justice language. This
study design is convergent transformative due to the extent the quantitative and
qualitative data converge and provide an enhanced understanding of inequities in
educational interventions for English Language Learners. Mixed methods are defined as
research where the “researcher mixes or combines quantitative and qualitative research
techniques, methods, approaches, concepts or language into a single study” (Ponterotto,
Mathew, & Raughley, 2004, p. 43). The language of social justice is explained as
“investigators striving to simultaneously promote human development and the common
good through addressing challenges related to both individual and distributive justice.
Social justice research includes empowerment of the individual as well as the active
confrontation of injustice and inequality in society because they affect research
participants as well as those in their systematic contexts” (Ponterotto, Mathew, &
Raughley, 2004, p. 44).
A convergent transformative mixed methods study was selected to address several
research issues, including the lack of research that merges qualitative and qualitative data
on the topic. A mixed methods research design brings a richness and depth to the
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
51
conversation that cannot be accomplished through empirical or qualitative data alone. A
transformative-based theoretical framework is used to advance the conversation on
educational outcomes for English Language Learners, with specific recommendations for
changes based on the research outcomes. Creswell and Clark Plano (2014) further
explain, triangulation of the qualitative and quantitative data sets provides greater validity
through “mutually corroborated” (p. 62) findings. A mixed methods study offers a
completeness or comprehensive explanation of results that would not be accomplished
through a single methodology. Figure 3.1 delineates the convergent transformative
design.
Figure 3.1. Transformative framework
F
Qualitative Data Collection and
Analysis
Quantitative Data Collection
and Analysis
Compare or Relate
Convergent Transformative Design adapted from Creswell’s (2012) mixed methods design types, incorporating a convergent parallel design with the transformative framework to give equal weight to the QUAN and QUAL aspects of the research.
Interpret Data
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
52
Description/Rationale of Participants/Sample
Quantitative and qualitative data collection phases occurred concurrently and
answered parallel research questions from independent data sources. The quantitative and
qualitative data results were merged to respond to the central and transformative research
questions. The quantitative phase examined four years of English Language Learner
archived assessment data from reading comprehension, English language acquisition and
standardized assessments. An original instrument will be designed to record quantitative
data and included assessment scores, home language, intervention participation level,
grade-level and growth expectations for comparison. Three types of archival assessment
data were used to neutralize the instrumentation effect (Gerring, 2012). The quantitative
data collection was designed to answer: How does a seven-week summer program for
English Language Learners influence growth toward grade level proficiency standards
over multiple years in reading comprehension, language acquisition and norm referenced
achievement?
The qualitative data were collected through one-on-one interviews with a
randomized, representative population of English Language Learners teachers that
instructed the intervention participants during the regular calendar year. These
participants were chosen to “synthesize information on the topic from different levels of
participants” (Creswell & Plano Clark, 2011, p. 180). For example, some of the
interviewees had been instructors of the students during the intervention and some were
the instructor during the school year and had not worked with the students during the
summer intervention. In addition, the instructors interviewed had worked with student for
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
53
various lengths of times, ranging from up to four years to less than one year. The
questions used in the qualitative phase were reviewed with the assistance of an advisory
board, including an ELL Coordinator, Deputy Director of Elementary Education/Federal
Programs and the Director of Research, Evaluation and Accountability to eliminate
biased or leading questions. All survey questions were developed with consideration
toward the research questions of the study. All qualitative data was collected through
semi-structured, open-end, one-on-one interviews. Jacob and Furgerson (2012)
recommend an interview protocol that used these steps: create questions guided by the
qualitative research questions and the literature review, beginning to end scripting to
ensure details or questions were not omitted, open-ended questions to allow participants
to fully expound on their responses, questions started with the basics and moved to more
complicated questions as the interviewer and interviewee relationship has developed, and
additional prompts to illicit expounded upon responses. The same protocol was used for
all qualitative interviews.
The responses were collected by audio recording, transcribed and responses coded
to identify themes. In addition, the researcher collected written notes during the
interviews. The qualitative research addressed: What are the perceptions of student
academic success from the English Language Learner teachers of participants in a seven-
week extended summer program for English Language Learners?
Meta-inferences were drawn from the merged quantitative and qualitative data
sets to answer the mixed methods central research question: What are the congruencies
between archived English Language Learner student assessment data outcomes and the
beliefs of ELL teachers about students academic outcomes after participation in a seven-
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
54
week summer intervention? Merged data sets were used to answer the transformative
research question: What educational policy considerations can be derived from the
outcomes of the research?
Population
The quantitative population was selected with the intent to be a probabilistic
sample of English Language Learner populations across the United States. The
qualitative sample was purposefully selected as populations that have varying levels of
experience with the central phenomena of the study: participation in a seven-week
summer intervention for English Language Learners. The population researched in the
quantitative phase were K-5 elementary English Language Learner students from a large,
suburban, mid-western school district adjoining a large urban district. The population
encompassed students from diverse socio-economic, cultural, ethnic and linguistic
backgrounds. For consistency, English Language Learners students were defined as a
student whose home or first language (L1) is a language other than English and the
English language is a second language (L2) that needs to be developed along with the
students’ first language. “English Language Learners are individuals whose language
proficiency limits their access to education” (Solano-Flores, 2010, p. 427).
The qualitative phase included elementary English Language Learners teachers
from the same district population sampled during the quantitative phase of the study. The
surveyed population was drawn from a potential pool of 19 elementary teachers that work
with ELL students during the regular calendar year. English Language Learner teachers
had a high level of direct and consistent instructional interaction with English Language
Learner students during the school year, with the joint responsibility of monitoring
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
55
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
56
reading comprehension, and language acquisition assessment and measuring growth
toward overall grade level content goals.
Sample
The sample population for the quantitative study was English Language Learner
students that scored below 3.5 on the World-Class Instruction Design and Assessment
(WIDA) Assessing Comprehension and Communication in English State-to-State for
English Language Learners (ACCESS for ELLs) English language proficiency
assessment and were invited to participate in a summer English language intervention
program. The proficiency range were 1.0 for entering to 6.0 for bridging. WIDA
ACCESS for English Language Learners was an English language proficiency
assessment given to kindergarten through 12th graders on an annual basis to monitor
English language acquisition of grade level academic and social content in four language
domains. The assessment is used by a consortium of 34 states as part of state
accountability measures established under the No Child Left Behind Act of 2001 (World-
Class Instruction and Design, 2014). After the initial screening, students were divided
into class groups based on grade level. Participation level was determined after the
intervention was completed. Student archival achievement data was later sorted into two
groups based on level of intervention attendance to create cohort groups. Student cohort
groups were established based on the year of program participation and does not reflect
the number of years participating the intervention. Each annual cohort group consists of
one treatment group with a seven-week summer intervention and a control group of
invited students that did not attend and received no intervention.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
57
Archived reading comprehension data, English language proficiency and state
standardized assessment data were collected and analyzed for each cohort group. Table
3.1 laid out the assessment data collected by grade level. The first annual cohort group
was tracked over four years, the second annual cohort group was tracked over three years,
and the third annual cohort group was tracked over two years. The fourth annual cohort
examined end of the academic year to start of the next academic year reading
comprehension data, since it was most recent intervention session, no long-term data for
the full year was available. The tracking time period signifies and coincides with the
number of years since participation in the intervention. For example, the first cohort
participated in the intervention during the summer of 2011 and has fours years of trailing
data after intervention participation. There were approximately 350 students that were
sampled in the study.
Table 3.1
Archived Assessment Data
Assessment Purpose Grade Level Fountas and Pinnell Benchmark Assessment Reading Comprehension Kinder to Fifth System WIDA ACCESS English Proficiency Kinder to Twelfth Missouri Assessment Program (MAP) Standardized Third to Twelfth Note. Table shows the assessment and grade levels covered by the assessments.
The qualitative population sample consisted of nine English Language Learners
teachers. The potential pool of staff interviewees was 19 teachers. Consent from all
interview participants was be gained in advance. The participants were selected randomly
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
58
to give a balanced perspective on the outcomes. In addition, the range of time spent as an
instructor for the intervention participants varies as staff allocations change, with the
range of contact time of less than a year to a high of four years.
Inclusions and/or Exclusions
WIDA ACCESS for ELLs provides a common assessment administered to all
kindergarten through 12th grade English Language Learner students and allows a focus on
students with the most English language need. A score of 3.5 represents the middle range
between newcomer and near proficient language usage. Table 3.2 represents the language
ranges and a description of the language levels.
Table 3.2
WIDA ACCESS Language Level Ranges
Language Level Range Entering 1.0 – 1.9 Beginning 2.0 – 2.9 Developing 3.0 – 3.9 Expanding 4.0 – 4.9 Bridging 5.0 – 5.9 Reaching 6.0 Note. Adapted from World Class Design and Instruction (WIDA) resource guide: The
relationship among WIDA’s strands of model performance indicators, ELP standards,
CAN DO descriptors and performance definitions (n.d.). Retrieved from
https://www.wida.us/standards/RG_Performance%20Definitions.pdf.
All English Language Learner students at 3.5 or lower on the WIDA ACCESS for
English Language Learners were invited to attend with families self-selecting attendance
participation level. As students’ language skills progressed and were formally exited from
ELL program services, academic growth continued to be tracked. For instance, if a
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
59
student exited ELL services, student achievement continued to be tracked on all three
assessments.
Instrumentation
Due to the nature of the study, there were a variety of instruments used to collect
data. Data collected during the quantitative phase was collected using a researcher
developed tool and will include student identification codes for tracking purposes, but
does not reveal personal details of any particular students. The primary purpose of the
tool is to be able to sort, collect, and analyze the achieved reading comprehension,
English language acquisition and norm-referenced state assessments. In addition, the tool
will collect student grade level, intervention participation level, home language and other
cultural/ethnic descriptors for demographic background on the diversity of the population
sample.
The researcher used multiple performance measures in the quantitative phase of
the study. The Fountas and Pinnell Benchmark Assessment was the instrument used for
reading comprehension measurement. The reading comprehension assessment is a
criterion-referenced assessment used to measure a students’ ability to comprehend fiction
and non-fiction texts based on criteria indicative of proficient readers. The instrument
used to measure English language proficiency is the WIDA ACCESS, a criterion-
referenced assessment. The state norm-referenced assessment, Missouri Assessment
Program (MAP) is a measure of how well students did in comparison to all other students
at the same grade level.
The qualitative phase of research used random sampling of English Language
Learner teachers from the district participating in the study. The purpose of the random
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
60
sample was to ensure diversity in the interview participants, in terms of the teachers’
exposure to students participating in the intervention.
Variables/The Researcher’s Role
In the quantitative phase of the research, the dependent variables were scores on
reading comprehension, English language proficiency and norm-referenced state
assessments. The independent variable was the effect of the intervention on assessment
scores. There was one treatment group, students that participated in seven-week
participation. Students that participated in half of the intervention were included in this
cohort group. In addition, there was one control group of students that met the initial
participation criteria, but did not attend the intervention. The researcher did not choose
the treatment group participation level. The participation level was self-selected by
families and applied to the quantitate data during collection. The intervention and control
groups were drawn from the same homogeneous pool of invited students and that meet a
minimum threshold language ability.
A confounding variable is the effect of the regular school year instruction. After
participating in the intervention, students returned to their regular academic instruction in
21 different schools within the school district participating in the study. While the district
curriculum was uniform, the level of implementation was not within the scope of this
study. As well, variations in instructional model, ranging from pull-out to integrated
English instruction and number of English Language Learner service minutes are not
within the scope of the study. In a pull-out instructional model, English Language
Learner students leave the general education classroom to receive English language
support services for specified periods of time “and is intended to provide language
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
61
support in a protective environment” (Kim, McLellan, & Asbell, 2010, p. 426).
Alignment to grade level curriculum can vary. The integrated model is characterized by
inclusion with English Language Learner support staff following the English Language
Learner student into the general education classroom to assist the student with language
support. A push-in model of integration is categorized by limited collaboration or
planning between the general education teacher and the English Language Learner
teacher (New York Collective of Radical Educators, n.d.; Honigsfeld and Dove (2008)
described co-teaching integrated model as a fully collaborative model of instructional
practice with the general education practitioner and special service provider taking equal
responsibility for instructional planning and implementation and has a higher level of
grade level content alignment.
In the qualitative phase, the central phenomena were English Language Learner
teachers’ perceptions of program effectiveness in creating academic success. The
researcher’s role was to accurately document interview participants’ responses. This was
accomplished through audio-recording and transcribing responses, in addition to taking
hand-written notes. The researcher was the facilitator during the interviews and used a
predetermined list of questions to avoid influencing interview responses. However, the
semi-structured nature of the interviews allowed for the opportunity to delve deeper into
responses to specific questions. In addition, interview participants confirmed accuracy of
transcript prior to inclusion.
Data Collection Procedures
The data collection process for the qualitative phase included one-on-one
structured interviews with English Language Learner teachers of students that
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
62
participated in the seven-week intervention in the school or other location that was
convenient for the interviewee. The interview questions examined the teachers’
perception about effectiveness of the intervention and the perceptions of immediate and
long-range academic outcomes of the students that participated in the intervention. Each
interview was audiotaped and transcribed verbatim for data analysis. The researcher
conducted each interview using standardized protocol of open-ended questions designed
to investigate the teachers’ perceptions. In addition, interviewees were probed for
additional information, to clarify responses and to elicit elaboration upon the responses
given.
Interview participants were sent the transcript of their interview to validate and
confirm responses. The transcripts were sent via email to participants, with participants
afforded the opportunity to comment on the accuracy of the transcription and obtain final
approval for direct personal quotes, even though the identity of the participant is
anonymous.
Creswell (2012) defines the unit of analysis as “the unit (e.g., individual, family,
school, school district) the researcher uses to gather the data” (p. 630). For the purposes
of the quantitative phase of this study, the unit of analysis will be cohort groups by year
of intervention participation and grade level.
Data Analysis Plan
A linear regression analysis was used to determine the effect of participation in an
intervention on academic achievement over time. The explanatory variable is the effect of
participation, either full participation, partial participation and non-participation in the
intervention on a homogeneous group of students. For this study, the term explanatory
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
63
variable is used, since the variable may not be statistically independent, as a result of
variances in instruction throughout the rest of the academic year and fidelity of reading
comprehension assessment implementation, which cannot be accounted for during this
study.
The qualitative data was analyzed by going from the detailed data of the
transcribed interviews to the coded data and themes. Due to the nature of qualitative
research, Creswell (2012) explained, analyzing and collecting occurs simultaneously as
major themes are identified. However, the first step in data analysis was to conduct a
preliminary exploration to develop an overall sense of the data. Next, the data was coded
and reduced to major and minor themes in a layered qualitative analysis.
Assumptions/Quality and Verification
The qualitative data was validated through member checking. Creswell (2012)
described member checking as the process in which interview participants approve the
transcript as a fair and accurate representation of their responses. In addition, the mixed
methods design triangulated data through qualitative and quantitative sources of data.
Ethical Considerations
Ethical considerations included protecting the identities of students under the age
of consent. During the quantitative phase of data collection, numerical identifiers were
attached to archival data sources. For the qualitative phase, all participants will be
informed the nature and purpose of the research in advance. As well, all participants are
to be identified with pseudonyms, for example, Teacher A represents teacher one of the
teachers, but does not reflect the sequence in which the teacher was interviewed.
References to specific students, by name or school of attendance were removed to protect
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
64
students’ identities. This was applied to references to district employees. Other ethical
considerations include ensuring the interview was conducted in a comfortable, neutral
location that had a minimum number of distractions with the researcher maintaining a
neutral stance throughout and respecting the research site.
Summary
The purpose of selecting a convergent, transformative mixed methods study was
to add a depth to the conversation about the effectiveness of a seven-week summer
intervention for English Language Learner students. A quantitative study on the
educational outcomes of the intervention contributes to the policy dialogue and effective
use or limited resources for a vulnerable population. However, the converged quantitative
student achievement data with qualitative data from educator surveys provided a fuller
picture for making policy recommendations and understanding outcomes.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
65
CHAPTER FOUR: FINDINGS
Introduction
The purpose of this chapter was to describe and interpret the data collected in a
convergent transformative mixed methods study to determine the congruencies between
student data outcomes after participation in a seven-week summer intervention with the
ELL teachers’ perceptions of student academic performance to determine the impact of
the intervention on student academic performance. The reason for the research is to add
to the knowledge base on ELL policy and programs as more and more students from
diverse backgrounds enter the public schools. ELL students make up the fastest growing
segment of public school population in the United States and are “vastly outpacing the
growth in the overall school population” (Heritage et al., 2015, p. 1). It is imperative for
schools and communities to uncover promising techniques and programs to help bridge
the academic divide between students that speak English as a first language and second
language learners. However, all education programs produce an effect, positive or
negative, on student achievement (Hattie, 2011; Wiggins, 2012). Determining which
influences produce the highest positive effect on student achievement is the key.
In 2011, assessment scores on the Missouri Assessment Program (MAP), the
state’s primary academic indicator for determining student performance and movement
toward grade level proficiency, reflected English Language Leaners scores were
substantially lower than English speaking peers (Missouri Department of Elementary and
Secondary Education, 2014). The discrepancy among ELL students and their peers, Table
4.1, Table 4.2 and Table 4.3 highlighted the need for programing to close the
achievement gap and increase ELL academic performance. In addition, grade level
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
66
disaggregated data showed the gap increased as the students progressed through their
academic careers. While that data reflected lower achievement in students learning
English as a second language, it by no means should be interpreted to mean ELL students
have lower capabilities as research suggested all student have the same growth rate
during the school year (Entwisle, Alexander, & Olson, 2001; Heyns, 1978). An additional
factor noted previously, the “receivement gap” (Chambers, 2009) underscored
deficiencies in the educational structures and access to educational resources that allow
all students to achieve at the highest levels. Table 5.3 delineated third grade
disaggregated data by ethnicity on the Missouri Assessment Program Assessment (MAP),
the standardized assessment used by the state of Missouri for accountability to record
student progress toward grade level mastery of standards. This data represented students
in the district participating in the research.
Table 4.1
3rd Grade ELA Missouri Assessment Program (MAP) Proficiency by Subgroup
Subgroup 2011 2012 2013 2014 All students 42.6% 45.4% 45.5% 41.6% Asian 42.2% 44.7% 41.8% 33.4% Black 26.8% 28.0% 29.1% 20.7% Hispanic 28.3% 35.8% 59.3% 33.4% Multi-racial 43.3% 39.4% 40.8% 35.4% White 48.8% 50.7% 48.9% 48.7% ELL 16.8% 22.8% 24.3% 22.5% Note. Data represents 3rd grade disaggregated data by ethnicity for the district of interest
(Missouri Department of Elementary and Secondary Education, 2014).
Table 4.2 represented the disaggregated data by ethnicity for fourth grade students
for the MAP assessment.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
67
Table 4.2
4th Grade ELA Missouri Assessment Program (MAP) Proficiency by Subgroup
Subgroup 2011 2012 2013 2014 All students 53.9% 49.6% 52.3% 46.6% Asian 46.2% 39.7% 52.8% 48.5% Black 42.0% 37.7% 35.1% 27.7% Hispanic 44.7% 38.1% 44.2% 47.5% Multi-racial 45.4% 50.7% 51.7% 43.1% White 58.1% 55.2% 56.9% 50.7% ELL 25.2% 26.5% 25.5% 27.1% Note. Data represents 4th grade disaggregated data by ethnicity for the district of interest
(Missouri Department of Elementary and Secondary Education, 2014).
The data in table 4.3 represented the disaggregated data by ethnicity for fifth
grade students in the district that participated in the research project. The data was
collected from the Missouri Department of Elementary and Secondary Education for the
MAP assessment.
Table 4.3
5th Grade ELA Missouri Assessment Program (MAP) Proficiency by Subgroup
Subgroup 2011 2012 2013 2014 All students 53.6% 53.5% 48.9% 52.4% Asian 52.9% 57.2% 47.2% 44.2% Black 35.6% 36.7% 35.9% 37.3% Hispanic 44.3% 37.6% 34.4% 44.4% Multi-racial 50.8% 61.1% 45.2% 58.3% White 59.4% 59.1% 54.7% 57.5% ELL 13.3% 24.3% 11.5% 19.8% Note. Data represents 5rd grade disaggregated data by ethnicity for the district of interest
(Missouri Department of Elementary and Secondary Education, 2014).
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
68
Review of the Methodology
A convergent transformative mixed methods study design was determined to be
the appropriate methodology to determine the effectiveness of the seven-week summer
intervention for English Language Learner students. Describing and displaying the
qualitative data collected through interviews with ELL teachers along with the
quantitative archived assessments data of the students provided an opportunity to bring
depth the to data that could not be accomplished through a single methodology. Creswell
and Plano Clark (2014) noted triangulation of qualitative and quantitative data through a
mixed methodology provides greater validity to the findings through the completeness of
the results.
All research questions were guided by the overarching transformative framework
of determining the effectiveness of the seven-week summer intervention as tool in closing
the achievement gap and staunching the summer learning regression among ELL
students. The central research question focused on the congruencies between archived
student assessment data and the beliefs of ELL teachers about student academic
performance? The quantitative research was dedicated the impact of a seven-week
summer intervention on multi-year assessment performance on reading comprehension,
English language proficiency and standardized state assessments of students that
participated in the intervention. ELL teachers’ perceptions of students’ academic
outcomes after participation in the intervention was the basis for the qualitative question.
Lastly, the transformative question addressed educational policy recommendations based
on the outcomes of the research.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
69
Data Analysis Procedures
The quantitative and qualitative data collected in this study were collected
concurrently. The quantitative data included the archived assessment data assessment of
English language learner students over a multi-year period. The qualitative data consisted
of one-on-one interviews with ELL teacher of students that have participated in the
seven-week summer intervention for ELL students.
Quantitative Data Analysis Procedures
For the quantitative phase of the study, the researcher examined archived student
data from three unique assessments. The data was archived in the district data storehouse
known as Data Director. The purpose of the archived student data was to respond to the
quantitative research question: What were the multi-year academic outcomes on reading
comprehension, English language acquisition and normed state assessments for ELL
students that participated in a seven-week summer intervention? The first measure was
the Fountas and Pinnell Benchmark Assessment System 1 and 2, used to assess reading
comprehension. The second assessment, WIDA ACCESS, measured English language
acquisition in the language domains of speaking, listening, reading and writing within
grade level content clusters. The Missouri Assessment Program (MAP), examined
English Language Arts normed for grade level content standards proficiency. Gerring
(2012) noted using multiple types of data neutralized the instrument effect.
The first assessment data examined was the Fountas and Pinnell Benchmark
Assessment System 1 and 2; a research based reading comprehension assessment
(efficacy studies can be viewed at http://www.heinemann.com/fountasandpinnell/
researchLLI.aspx) of leveled common assessments to identify the probable reading
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
70
behaviors of proficient readers at a specific reading level. The levels range from pre-A
for a pre-reading student to a level Z. A pre-reading student would generally include
students in the first half of the kindergarten year and ELL students that would be
considered “newcomers” or that have recently arrived in the United States. The
breakdowns of corresponding reading levels with grade level expectations are listed in
table 4.4. As well, the rate of anticipated annual growth for each grade level is shown.
Fountas and Pinnell Benchmark Assessment System 1 and 2 were used for students in
kindergarten through fifth grade. The assessment was administered three times per year,
at the beginning, middle and end of the academic year. When students reach middle
school, a different reading comprehension assessment was used, so student data is limited
to elementary years exclusively.
Table 4.4
Fountas and Pinnell Reading Comprehension Proficiency Level by Grade Level
Grade Level Fall Winter Spring Expected Annual Growth Kindergarten B D 4 levels First Grade C/D G J 7 levels Second Grade I/J L/M N 4 levels Third Grade M/N P/Q R 4 levels Fourth Grade Q/R S U 3 levels Fifth Grade T/U V W 2 levels Note. Grade level reading proficiency expectations for students in the district of interest.
Kindergarten does not have a proficiency goal for the fall benchmarking period. When
two levels are listed, both levels are considered proficient for the benchmarking period.
Dual fall reading levels account for summer regression from the end of the previous
instructional year.
English language acquisition was measured through the WIDA ACCESS. The
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
71
domains of speaking, listening, reading and writing were assessed in grade level content
clusters. The grade level clusters are: (a) kindergarten; (b) first and second grade; (c)
third through fifth grade; (d) grades six to eight; and (e) ninth grade to 12th grade. The
content areas included the language of social skills and the language of math, science,
language arts and social sciences. Reports on the validity of the assessment can be
viewed at https://www.wida.us/assessment/access/TechReports/index.aspx. The use of a
consistent, district-wide, language acquisition assessment, ACCESS, allowed for student
data to be collected as students progressed through their academic careers. ACCESS was
administered to kindergarten through twelfth grade. Assessment data beyond elementary
were able to be included as part of the data set.
The assessment data for the third measure were the English Language Arts scaled
scores for the Missouri Assessment Program (MAP). The MAP is a state normed, grade
level content assessment for students in grade three to 12. The data is reported in two
forms, a proficiency level and scaled score. Proficiency levels, from low to high, were
below basic, basic, proficient and advanced. However, for the purpose of this research,
the scaled score provided a numeric method of tracking change over time. MAP was
administered to third grade through twelfth grade students and assessment data as
students transitioned beyond elementary were included as part of the data set. For
example, fifth grade students from the 2011 cohort, have MAP data available for fifth,
sixth, seventh and eighth grade.
The initial criteria for participation in the seven-week summer intervention was
based on students with the highest language need. The highest language need was
determined to be students that scored a 3.5 or lower composite score on the WIDA
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
72
ACCESS language proficiency exam. Archived assessment data was retrieved from a
data storehouse belonging to the district of interest. The time period covered included
spring 2011 through winter 2015. Some outliers, students with higher baseline scores,
were included for participation in the intervention in order to keep sibling groups intact.
Assessment data points were collected into a spreadsheet, grouped by assessment in time-
sequenced order and included the three assessments identified. Student data was then
grouped by level of intervention participation. One group participation level was invited
to participate in the intervention, but did not attend. The second group participation level
was invited to intervention and did attend the intervention.
The initial group of invited students provided a homogeneous student population
and simultaneously provided a control group, differentiated by non-participation (control)
or participation (intervention) in the seven-week summer intervention. The cohort groups
examined in the research will be identified in the following manner to distinguish the
year of participation in the intervention and level of participation:
• 2011 control cohort and 2011 intervention cohort;
• 2012 control cohort and 2012 intervention cohort;
• 2013 control cohort and 2013 intervention cohort; or
• 2014 control cohort and 2014 intervention cohort.
In addition, table 4.5 identified the assessments and the year assessment data points were
collected for each cohort group.
ACCESS is the language acquisition assessment measure used for the purposes of
this study. MAP is the state standardized assessment of English Language Arts grade
level content. Students in middle school continue to be assessed on the ACCESS and
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
73
MAP, so middle school scores on those assessments were included when available.
Table 4.5
Assessment by Grade
Grade Level Fountas and ACCESS MAP Pinnell Kindergarten X X (at end of 3rd grade) First Grade X X (at end of 3rd grade) Second Grade X X (at end of 3rd grade) Third Grade X X X Fourth Grade X X X Fifth Grade X X Middle School X X Note. Chart represents the grade level when assessments are given. The Fountas and
Pinnell Benchmark Assessment System 1 and 2 are used for reading comprehension.
Data was collected in a spreadsheet, creating a data grid for input for analysis.
Each cohort year was grouped into a separate spreadsheet. Next a tab was created for
each assessment and students were sorted by non-participation or participation in the
seven-week summer intervention. Fountas and Pinnell data was collected as a letter from
pre-A (A-) through Z. The letters were converted into a numeric equivalent. Pre-A was
the equivalent of one, A had the numeric value two, and so on, with Z converted to a 27.
Spot checks were conducted to ensure accuracy. The database was cleaned through a
visual inspection. In addition, a high student mobility rate required students with missing
data in years following the intervention, to be removed.
A linear regression was conducted on each cohort year and cohort assessment
using Excel. A third party conducted the linear regression and each calculation was run
three times. On outlier result, data was rechecked and the regression was conducted again
to ensure accuracy and determine if there were input errors. Student names and
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
74
identification numbers, as well as any other identifiers were removed for anonymity. The
only evident identifier attached to the data was grade level.
A level of reliability is built into the assessment by having a standardized testing
timeframe for all students across the district. The Fountas and Pinnell Benchmark
assessment is given in the fall (September), winter (December) and spring (May). The
statewide ACCESS administration timeline is the month of February and the statewide
administration timeline for the MAP assessment is May. In addition, all assessments have
a consistent administration protocol and script to add to the standardization of
administration and were a consistent measurement tool across the intervention time span.
Each assessment provides research on the reliability and validity of the assessment, even
though the public debate on the validity of standardized assessment continues. Each data
point represents is a single-item assessment score.
Qualitative Data Analysis Procedures
Qualitative data was collected through one-on-one interviews with teachers of
English language learner students with a purpose of responding to the qualitative research
question: What were the English language learner teachers’ beliefs of students’ academic
outcomes after students participated in a seven-week summer intervention? A total of
nine out of 19 ELL teachers were interviewed. Teachers were randomly selected and had
various levels of ELL instructional experience within the district, as well as various levels
of participation in the summer intervention. Teachers were randomly selected, providing
a maximal variation sample of teacher experiences with students that have participated in
the summer intervention (Creswell, 2012). Table 4.6 noted the years of ELL instructional
experience within the district of interest, as well and number of years of participation as
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
75
an instructor for the seven-week ELL summer intervention, if applicable.
Table 4.6 Interviewee Participation in ELL Intervention
Teacher Building Number ELL Intervention Number of Years ELL Teacher of Years Teacher Intervention Teacher Teacher A X 10 Teacher B X 4 X 2 Teacher C X 10 Teacher D X 9 Teacher E X 11 X 2 Teacher F X 1 X 3 Teacher G X 1 Teacher H X 3 X 3 Teacher I X 8 X 3
Note. Demographic data was collected by the researcher as part of the questioning on the
interviewees on their relationship to the program.
To provide an additional layer of anonymity to the interview participants, the
interviewees were randomly assigned a letter of the alphabet to be used as their identifier.
Interview respondents will be referred to as teacher A, teacher B, etc. and this identifier
has no relationship to the sequence in which they were interviewed. The interview
participants were randomly selected from elementary teachers of ELL students within the
participating district. At the time the interviews were conducted, 19 full-time ELL
teachers worked within the district’s elementary schools. Teachers were contacted by
email and/or by phone initially, following a scripted protocol (appendix A) ten teachers
were contacted and agreed to participate in one-on-one interviews. One initially agreed,
but acquiesced due to schedule conflicts. In all, a total of nine interviews were conducted.
The interviews were conducted in person for seven participants and two were
conducted by phone at a place and time convenient for the interviewee. Interviews were
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
76
semi-structured, with a set of questions (appendix B) asked to each participant, but also
allowed flexibility for a natural conversation and the ability to explore lines of
questioning at a deeper level as needed. All interviews were digitally recorded and
professionally transcribed. Prior to conducting the interview, all interview participants
reviewed the informed consent document with the researcher and the document was
given to the individual. If candidates were interviewed by phone, the same procedure was
followed and a copy of the informed consent document was sent to the participant.
After transcription, the researcher reviewed the transcripts to verify any inaudible
words or phrases and double check names of assessments and acronyms. A list of
possible unfamiliar acronyms, assessment names, or English language learner specific
vocabulary and terms were supplied to the transcriptionist. The transcriptions were
checked by the researcher due to familiarity with the content and context of the
responses. This also provided an additional level of accuracy prior to sending the
transcripts for member checking. For instance, the word math and name of the MAP
assessment were easy to misunderstand. In addition, the researcher edited out any proper
names of individuals or schools that could be used to identify the interviewee. As well,
names of students and other individuals employed by the district of interest were
removed from the text. For the purpose of the member checking, words and phrases like
“um” were left in the initial transcript, but were edited out of transcript excerpts included
in the text of the dissertation.
After the transcription and researcher verification process was completed,
transcripts were emailed to interview participants as part of the member check process.
Participants were asked to verify the content of the transcript and give approval that the
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
77
content accurately represented their perceptions of ELL student outcomes after
participating in the seven-week summer intervention. All participants responded in the
affirmative, acknowledged receipt of the transcript and accepted the content as
transcribed. The member checking process, or respondent validation, improved the
accuracy and validity of interview responses through the interview participant being able
to analyze and respond to the comments (Creswell, 2012; Creswell & Plano Clark, 2011;
Herr & Anderson, 2015).
Once the interview participants accepted the transcripts, the researcher conducted
several readings of the transcripts to get an overall feel and to develop an understanding
of possible thematic connections that might be developed across all interviews. The
researcher created an Excel document and recorded key words and phrases from each
interview question. A row was created for each teacher and columns were created for
each question. The Excel document served as a tool for identifying and sorting words or
phrases that could be used to identify specific themes and the context under which the
subject referred to the theme. For example, the theme of summer learning loss was
referenced as a goal of the seven-week summer intervention, but was also referenced in
response to teacher perceptions about student outcomes on various assessments, which
provided a context around the response.
Results
The sequence of the results will start with the display of the quantitative results
grouped by assessment type: reading comprehension using the Fountas and Pinnell
Benchmark Assessment System 1 and 2, English language acquisition measured by the
WIDA ACCESS, and standardized state assessment of grade level English language arts
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
78
content assessed by the MAP. Within each assessment type, the results will be separated
by cohort group year, cohort grade level and cohort group participation level. For the
qualitative results, results were sequenced by the major themes identified from the one-
on-one ELL teacher interviews. Five broad themes were identified through the interviews
conducted with ELL teachers of the students that have participated in the seven-week
summer intervention. The themes included (a) summer learning regression and the
achievement gap, (b) lack of resources available at home or the community, (c) closing
the achievement gap, (d) the academic calendar; and (e) the curriculum and program goal
of language proficiency. The first four themes align with themes in the review of
literature. The fifth theme, curriculum, while outside the scope of the study, was
referenced with a high enough frequency to be addressed and is tied directly to the
program goal of increasing English language proficiency. In addition, a summary of
responses about student outcomes on the three assessments, reading comprehension,
English language proficiency and state standardized assessments, after participation in
the intervention in the seven-week intervention for ELLs.
Quantitative Data
The qualitative data encompassed three assessment measures: reading
comprehension, English acquisition and English language arts grade level content. A
linear regression was conducted using the data. First, a scatter plot of the data was created
for each type of assessment, each cohort grade level, and level of participation. Next, a
linear regression trend line was established. The relationship was represented by the
equation y = Mx + b. The independent variable was time and the dependent variable was
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
79
student assessment scores. An R-squared coefficient calculation was used to determine
the fit of the data values. Interpretation of the value is represented in table 4.7.
Table 4.7
Correlation and Interpretation of R-squared Value
Correlation Value Interpretation + 1.00 perfect upward linear relationship + 0.80 to 0.99 strong upward linear relationship + 0.60 to 0.79 moderate upward linear relationship + 0.30 to 0.59 modest upward linear relationship + 0.11 to 0.29 low upward linear relationship + 0.01 to 0.10 weak upward linear relationship 0.0 no linear relationship - 0.01 to 0.10 weak downward linear relationships - 0.11 to 0.29 low downward linear relationships - 0.30 to 0.59 modest downward linear relationships - 0.60 to 0.79 moderate downward linear relationship - 0.80 to 0.99 strong downward linear relationships -1.00 perfect downward linear relationships Note. Compiled from Muijs (2004) and Rumsey (n.d.)
Reading Comprehension – Fountas and Pinnell
Fountas and Pinnell Benchmark Assessment System 1 and 2 data were used as the
assessment tool for reading comprehension. The assessment was administered in the fall,
winter and spring of each school year. Scores are recorded as letter gradients, starting
with A- for an early emergent reader to an independent reader at level Z. The
characteristics of an early emergent reader include developing letter sound
correspondence, learning some high frequency word and developing left-right text
tracking. A kindergarten student in the first months of school would be an early emergent
reader and as such, assessment for kindergarten students does not start until the winter
semester. In addition, a pre-reader would be categorized as “entering” for ELL students
and would be identified as a newcomer, or student that has recently started learning
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
80
English as a second language. Characteristics of a level Z reader include the ability to
process texts across a range of genres, the ability to interpret abstract texts, and process
through a heavy load of context-specific vocabulary while applying schema. For the
purpose of this research, alpha scores were converted to numeric equivalents. For
example, pre-A was the equivalent of one, A had the numeric value two, and so on, with
Z converted to a 27.
2011 cohort groups: Fountas and Pinnell reading comprehension.
Looking at the data from figure 4.1, the 2010 school year was used as the benchmark year
prior to the intervention and represents the kindergarten. The first data point for 2011 is
the fall assessment period and the student would have transitioned to first grade. Only
student data from the benchmark period through winter 2015 were included in the
analysis. The baseline mean for the 2011 kindergarten control cohort was M = 2.0 and the
baseline mean for the 2011 kindergarten intervention cohort was M = 1.07.
The final mean at the end of the intervention for the 2011 kindergarten control
cohort (n = 4) was M = 15.25, 2011 kindergarten intervention cohort (n = 14) was M =
16.21 at the end of the winter benchmark during 4th grade. The trends were a positive
1.95 for the control cohort, a positive 4.21 for the intervention cohort. Strength of
association values, R2 = 0.963 demonstrated a strong correlation for the predictability of
the intervention cohort.
Compared against the district’s grade level reading expectations at the start of first
grade, the expected reading level would C/D (numeric equivalent were 4/5) the ELL
students started on average, over half a grade lower. At the middle of 4th grade, the
current grade of the cohort groups, the grade level reading expectation is for
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
81
Figure 4.1. Cohort 2011: Kindergarten Fountas and Pinnell benchmark reading
comprehension assessment
students to be a level S reader. The intervention cohort remained one-year behind their
peers in reading comprehension, but with the projected growth rate, could be within a
half a year at the end of 5th grade. Students in the control group, currently at closer to one
and a half years behind their peers, would be approximately 2 years behind at the end of
5th grade.
Next, looking at the same cohort year and reading comprehension for first grade
students the R-squared values showed a higher correlation. For the 2011 first grade
control cohort the values were R2 = 0.85, the intervention cohort values were R2 = 0.81,
signifying a strong correlation of scores.
02468
101214161820222426
2009 2010 2011 2012 2013 2014
Aver
age
Scor
e
Year Control cohort Intervention cohort
Linear (Control cohort) Linear (Intervention cohort)
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
82
Figure 4.2. Cohort 2011: First grade Fountas and Pinnell benchmark reading
comprehension assessment
The starting reading comprehension mean score for first graders as they entered
second grade were M = 4.7 for the control cohort and M = 7.63 for the intervention
cohort. This puts the control group one grade level behind their peers and the group of
students that participated in the intervention at about a half-year behind grade level
expectations. At current grade level expectations, all cohort groups are at the grade level
expectations.
The second grade cohort, as it transitioned to third grade, started with an M = 9.08
for the control group of students that were invited to participate in the intervention, but
did not attend. The intervention group were M = 10.00, approximately one grade level
below their peers.
02468
101214161820222426
2009 2010 2011 2012 2013 2014
Aver
age
Scor
e
Year Control Cohort Intervention cohort
Linear (Control Cohort) Linear (Intervention cohort)
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
83
Figure 4.3. Cohort 2011: Second grade Fountas and Pinnell benchmark reading
comprehension assessment
The last reading comprehension data for this cohort group is the end of 5th grade
and as a result, there were no data points for 2014. However, at the end of 5th grade
combined cohort had a mean reading comprehension score of M = 21.44 which would
put then at the reading comprehension equivalent for starting fifth grade.
Reading comprehension data for the 2011 third grade cohort, as the group started
4th grade, showed a similar trend line as the previous grade levels. However, as a group,
the mean starting point was two years behind their peers. By the end of 5th grade, as a
combined group, continued to lag and the gap grew to over 2 years behind grade level
expectations.
02468
101214161820222426
2009 2010 2011 2012 2013 2014
Aver
age
Scor
e
Year Control Cohort Intervention cohort
Linear (Control Cohort) Linear (Intervention cohort)
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
84
Figure 4.4. Cohort 2011: Third grade Fountas and Pinnell benchmark reading
comprehension assessment
The 2011 fourth grade cohort, starting fifth grade immediately after the
intervention experience, had a mean reading comprehension starting point of M = 16.76
for the control cohort and M = 12.13 for the intervention cohort. At the end of the data
collection period, as a combined group, students moved from the middle of second grade
at the low end of the range to a collective middle of fourth grade reading comprehension
level. The reliability of the growth projections were difficult due to the small number of
participants and the limited number of data points. The fifth grade group were starting
sixth grade after the intervention, so no data was collected.
02468
10121416182022242628
2009 2010 2011 2012 2013 2014
Aver
age
Scor
e
Year
Control cohort Intervention cohort Linear (Intervention cohort)
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
85
2012 cohort groups: Fountas and Pinnell reading comprehension. The
2012 cohort had similar results. Table 4.8 showed the regression for each grade level and
high R-squared values at each grade level cohorts, with the second grade control group
the exception.
Table 4.8
2012 Cohort Groups: Reading Comprehension
Grade n Start M End M Trend Line R2
Slope Pre-Kinder Control 6 2.00 12.50 5.41 0.92 Intervention 20 2.19 12.76 5.29 0.94 Kindergarten Control 2 3.5 14.50 4.63 0.92 Intervention 29 4.11 17.59 5.51 0.96 First Control 7 8.43 17.00 3.63 0.95 Intervention 31 9.00 16.19 3.54 0.92 Second Control 2 7.00 16.00 3.81 0.40 Intervention 12 10.08 19.00 3.41 0.96 Third Control 0 Intervention 5 13.40 19.20 3.13 0.92 Fourth Control 1 16.00 21.00 4.33 Intervention 14 14.60 17.93 3.13 0.87 Note. Pre-kinder students were included in the intervention for this year; therefore
reading comprehension scores were available for this cohort year. The mean starting and
end scores provide the mean growth range for the cohort and the regression gives the
mean year over year growth. The kinder, second, third grade and fourth grade control
cohorts were small cohorts to start and attrition through mobility further reduced the size
of the cohorts. The entire cohort was small before the data was cleaned.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
86
Pre-kinder students participated in the intervention prior to the start of
kindergarten and the mean for the first benchmark placed them one level below grade
level expectations. However, at the end of the data collection period, the group mean
placed slightly below second grade winter benchmark expectations, meaning the students
were above grade level goals. For the 2012 kindergarten cohort, the starting mean was on
par with grade level expectations. At the end of the data collection period, the
kindergarten-combined cohort was slightly more than 2 reading levels above the grade
level expectation.
The 2012 combined first grade cohort started one reading level below the grade
level expectation and finished one level below grade level goals. For the second grade
control group, the starting point was seven levels below their peers and finished within
two levels of their peers. At the beginning of the year, the second grade intervention
cohort started one full grade level below expectations, but was able to move within a
half-year of their peers. Comparing against grade level reading comprehension goals,
both third grade and fourth grade cohorts were over one and a-half years behind, but were
able to move within one year of the grade level goal.
2013 cohort groups: Fountas and Pinnell reading comprehension. The reading
comprehension data for the cohort 2013 (table 4.9) demonstrated students in the
kindergarten intervention cohort started below peer expectations by almost two reading
levels. At the end of the intervention, this cohort has advanced almost eight reading
levels, ending one level behind.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
87
Table 4.9
2013 Cohort Groups: Reading Comprehension
Grade n Start M End M Trend Line R2
Slope Kindergarten Control 0 Intervention 11 1.17 9.00 5.60 0.87 First Control 13 10.31 14.92 3.09 0.91 Intervention 45 6.65 14.48 4.38 0.88 Second Control 10 14.00 18.10 2.65 0.81 Intervention 20 10.85 16.35 3.41 0.91 Third Control 4 13.00 16.75 4.05 0.68 Intervention 11 9.64 16.82 6.31 0.15 Fourth Control 5 14.00 18.10 5.33 0.89 Intervention 5 10.85 16.35 5.48 0.99 Note. The mean starting and end scores provide a growth range for the cohort and the
regression gives the mean year over year growth. The kinder control cohort was a small
cohort to start and attrition through mobility further reduced the size of the cohorts.
The first grade control group data (table 5.9) started slightly below reading level
goals and the intervention group started close one year behind. By the end of the data
collection period, the end of second grade, both cohorts had made similar progress and
end within one reading level of the peers. The second grade cohorts followed a similar
pattern with the control group starting higher than peers and the intervention cohort
started at one grade level behind. At the end of the data collection period, both second
grade cohorts were only slightly below end of the year expectations. In both cases, the
students that participated in the intervention had higher average growth than students that
did not participate.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
88
At third grade, the control cohort started 5 levels below grade level expectation
and the intervention cohort started at a reading level deficit of 10 levels, the equivalent of
mid-year in first grade. Both cohorts trended higher at a rate almost double their peers,
finishing about the same reading level as students mid-year in 3rd grade. The fifth grade
cohort followed a similar pattern and experienced a growth trend over double the
expectation of their peers. In upper elementary grades, third through fifth, due to the
complexity of texts, students’ progress through reading levels at a slower pace. During
the fourth grade year, the average student moves three levels and in fifth grade the rate
decreases to two levels.
2014 cohort groups: Fountas and Pinnell reading comprehension. The data
collected for the 2014 cohort (table 4.10) covered one academic year. The trend is harder
to identify, since the timeframe and the number of sequential data points is limited.
Starting mean for the control group is slightly below the expectation and the intervention
cohort is slightly above expectation. At the end of the year, both groups averaged about
two reading levels growth, below the grade level expectation of seven levels. In essence,
the cohorts fell behind their peers with a growth trend of approximately 2.5 points versus
the grade level growth expectation of seven levels throughout the year.
Following a similar pattern, the first grade growth trend was below the peer
expectation, but the intervention cohort experienced a higher trend growth rate. Data for
the second grade cohorts reflected a comparable configuration, with both the control
cohort and intervention cohort ended the data collection period with a wider gap between
current reality and anticipated grade level outcomes. The exception for the 2014 cohort
group was the fourth grade intervention group.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
89
Table 4.10
2014 Cohort Groups: Reading Comprehension
Grade n Start M End M Trend Line R2
Slope Kindergarten Control 22 3.73 5.50 2.44 0.60 Intervention 27 4.48 6.48 2.64 0.50 First Control 11 8.00 10.18 2.84 0.43 Intervention 17 7.76 10.53 3.77 0.74 Second Control 10 10.67 11.83 1.50 0.38 Intervention 9 7.30 8.30 1.68 0.40 Third Control 5 13.8 16.00 3.05 0.88 Intervention 6 9.67 11.67 2.65 0.54 Fourth Control 3 20.00 19.67 0.58 0.16 Intervention 11 14.30 18.30 5.59 0.93
Note. The mean starting and end scores provide the mean growth range for the cohort and
the regression gives the mean year over year growth. The shorter time span increased the
number of cohort members that remained and had complete data sets.
English Language Acquisition – ACCESS
WIDA Access is the measurement tool used for determining students English
language acquisition in elementary through high school. The range is from one, for a
student at the entering level to a six, bridging, for students that have reached the
maximum proficiency level equivalent to native English speakers. On average, students
are expected to move one level, or one whole point, in one academic year.
2011 cohort groups: English language acquisition. For the 2011 cohort group,
data form the initial benchmark was followed by three additional years of scores. Third
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
90
and fourth grade students’ scores followed into middle school and fifth grade scores into
high school were available, since the same assessment is used.
Table 4.11
2011 Cohort Groups: English Language Acquisition
Grade n Start M End M Trend Line R2
Slope Kindergarten Control 3 1.27 4.33 1.04 0.96 Intervention 14 1.04 4.86 1.31 0.97 First Control 5 4.00 5.24 0.46 0.87 Intervention 9 2.44 5.18 0.95 0.86 Second Control 11 3.33 5.50 0.75 0.92 Intervention 15 1.41 4.94 1.21 0.88 Third Control 4 1.68 4.86 1.12 0.91 Intervention 9 1.90 5.53 1.29 0.86 Fourth Control 4 2.40 5.10 0.93 0.95 Intervention 4 2.03 5.35 1.08 0.92 Fifth Control 2 1.55 3.45 0.65 0.89 Intervention 5 2.03 5.35 1.08 0.92
Note. The mean starting and end scores provide the mean growth range for the cohort and
the regression gives the mean year-over-year growth.
Four control groups trended lower than one point per year: first grade (0.46),
second grade (0.75), fourth (0.93) and fifth grade (0.65). In comparison, the only grade
level that made less than one-point growth in the intervention cohort was first grade
(0.95). At the starting mean for most cohorts, one-point growth annually, would take
students 4 years to reach a language proficiency level high enough to exit ELL services
and to be considered English proficient. A student that made a 0.48 annual growth rate
would take double the time to meet proficiency.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
91
2012 cohort groups: English language acquisition. A pre-kinder cohort was
added to this year and as suggested in the findings of a Dutch study, age of acquisition of
vocabulary effect “proved to be the largest in the youngest subject group” (Assink, van
Well, & Knuijt, 2003) and sited the cumulative effect of word and vocabulary usage over
time as the possible reason. In addition, the district had noticed the incoming pre-kinder
ELL students and those ELL students that did not have pre-kinder were coming in at far
lower skill levels than students.
The first item to note about the data from the 2012 cohort groups ACCESS
scores, is the frequency of “perfect” data sets. Statistician Joseph Morse, an independent
third party used to check data, noted the fit is a result of the number of data points
available for this cohort (personal conversation, June 30, 2015). In addition, several
cohort groups were lost due to mobility.
Data collected for the 2012 cohorts in English acquisition (Table 5.12) showed
higher starting language acquisition scores than the 2011 cohorts. However, as a group,
the end mean was not as high and the cohorts had lower trend lines.
In particular, both fourth grade cohorts were half of the expected growth. Among
the control cohorts, the pre-kinder and second grade groups made the expected growth
rate. First grade was the only intervention grade level that made the expected growth,
with pre-kinder and kinder slightly below.
2013 cohort groups: English language acquisition. The 2013 cohort year added
pre-kinder students during the summer intervention. The data from the 2013 cohort group
(table 4.13) had a limited number of data points across time creating a distorted
correlation between data points. This cohort year had a high starting mean. First grade,
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
92
second grade, third grade, fourth grade and fifth grade groups started above a 3.0, a mid-
level English acquisition score. However, the trend growth for fourth and fifth would put
them on a slow pace for reaching proficiency and delayed time frame for exiting ELL.
Table 4.12
2012 Cohort Groups: English Language Acquisition
Grade n Start M End M Trend Line R2
Slope Pre-Kinder Control 8 2.49 3.69 1.20 1.00 Intervention 20 3.02 3.97 0.95 1.00 Kindergarten Control 2 2.75 4.40 0.85 0.96 Intervention 30 2.42 4.42 0.99 0.95 First Control 7 4.10 4.87 0.77 1.00 Intervention 31 4.02 5.12 1.10 1.00 Second Control 2 2.25 4.45 1.10 0.86 Intervention 12 3.09 4.80 0.85 0.88 Third Control Intervention 5 3.48 4.96 0.74 0.70 Fourth Control 2 3.35 4.10 0.42 0.25 Intervention 13 3.74 4.91 0.58 0.57 Fifth Control Intervention 3 2.80 4.97 0.85 0.96
The end mean for all groups, except the pre-kinder group, are in the 4 range. The
trend for all groups, except fifth grade is close to the one level of growth expectation per
year. These data points, starting mean and ending mean, would reflect a benchmark score
and one data point, or one-year trend.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
93
Table 4.13
2013 Cohort Groups: English Language Acquisition
Grade n Start M End M Trend Line R2
Slope Pre-Kinder Control 8 2.49 3.69 1.20 1.00 Intervention 20 3.02 3.97 0.95 1.00 Kindergarten Control 2 2.75 4.40 0.85 0.96 Intervention 30 2.42 4.42 0.99 0.95 First Control 15 3.28 4.32 1.04 1.00 Intervention 46 3.13 3.99 0.86 1.00 Second Control 11 3.77 4.87 1.10 1.00 Intervention 21 3.55 4.55 1.03 1.00 Third Control 4 3.83 4.08 0.25 1.00 Intervention 12 3.04 4.30 1.26 1.00 Fourth Control 6 3.90 4.63 0.73 1.00 Intervention 15 3.84 4.74 0.90 1.00 Fifth Control 1 4.70 4.90 0.20 1.00 Intervention 13 4.52 4.47 -0.05 1.00
Standardized English Language Arts Content – MAP
ELL status is highly scrutinized and in the state of Missouri, comprised one
student population that falls within the super subgroup. The super subgroups were a
category of students that have traditionally lagged the academic achievement of their
peers. Heritage et al. (2015) explained, “English language learners (ELL) group status,
unlike that of other student groups, is meant to be temporary. Federal and state policies
anticipate that ELL’s will leave this category as a result of language instruction and
academic support services that they are entitled to receive” (p. 109). The Missouri
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
94
Assessment Program (MAP) is a primary state level data-tracking tool to gage annual
growth for ELL students. Summative assessments, such as the MAP, provided a view of
student learning after a specific period of time. The assessment covered the content
taught to evaluate the student learning of the content. These types of assessment are
fraught with controversy and have become an inescapable part of the educational and
assessment landscape and cannot be excluded as one piece of an ELL student overall
academic picture.
Student data for the MAP assessment is presented in two ways. First, a numeric
scaled score is assigned for each assessment content area and these scores are normed
annually to provide a proficiency level. The range changes yearly. The proficiency
ranges, from lowest to highest, were below basic, basic, proficient, and advanced. For the
purpose of this study, the scaled scores was used to track student data. MAP assessments
were administered to students in grade three to twelve.
2011 cohort groups: English language arts. Looking at table 4.14, this
described the data that is included in the table. First grade students were assessment on
the MAP for the first time at the end of third grade and fourth grade, giving only two data
points and an artificial “perfect” correlation. A third grade student for example, would
have been tested for the first time and would have trailing data that includes the next
three years and into the first year of middle school. A fifth grade student would have a
similar sequence with three trailing data points into high school. Kindergarten data were
not included since the students were assessed once.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
95
Table 4.14
2011 Cohort Groups: English Language Arts Content
Grade n Start M End M Trend Line R2
Slope First Control 6 624.60 651.33 26.73 1.00 Intervention 8 618.13 633.13 15.00 1.00 Second Control 12 604.67 648.08 21.70 0.84 Intervention 15 618.93 660.43 20.75 0.99 Third Control 4 606.25 639.75 12.22 0.63 Intervention 8 572.44 641.63 23.57 0.94 Fourth Control 3 622.67 649.00 7.66 0.62 Intervention 8 618.00 688.25 20.60 0.83 Fifth Control 2 632.00 657.50 10.65 0.57 Intervention 4 612.50 647.00 11.22 0.96 The grade level groups tend to be smaller with the MAP assessment data than
with reading comprehension and ACCESS, in part to student language levels. ELL
students in the United States less than one calendar year were waived from the ELA
portion of the MAP assessment, but were not waived from the other assessments used in
this research.
The data for second, third, fourth, and fifth grade reflected a higher trend line for
the intervention cohort than the control cohort. In addition, the intervention grade level
cohorts stated at a lower mean score and ended the data collection period a higher mean
than the control cohorts. The first grade cohorts were the exception, with the intervention
cohort making lower trend growth than the control. As well, the end mean for the first
grade intervention cohort was lower than the control cohort.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
96
2012 cohort group: English language arts. Kinder and first grade data were not
included in table 4.15 since two consecutive data points were accumulated. Student
transiency and assessment requirements caused a high attrition rate for the cohort year.
However, all groups with data available made positive growth. In comparing to the
previous cohort year, 2011, the growth was smaller.
Table 4.15 2012 Cohort Groups: English Language Arts Content
Grade n Start M End M Trend Line R2 Slope Second Control Intervention 12 649.00 667.00 9.00 0.46 Third Control Intervention 5 632.20 659.60 13.70 0.63 Fourth Control 1 639.00 665.00 13.00 0.83 Intervention 15 620.07 644.07 12.00 0.77 Fifth Control Intervention 9 649.00 667.00 9.00 0.46
2013 cohort groups: English language arts. For this cohort year, the first grade
level with two consecutive data points was third grade. With only two data points, the R-
squared values showed an unlikely high correlation. For fifth grade, the same is true with
the control cohort due to one student comprising the cohort.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
97
Table 4.16
2013 Cohort Groups: English Language Arts Content
Grade n Start M End M Trend Line R2
Slope Third Control 4 589.25 616.50 27.25 1.00 Intervention 11 580.83 621.27 40.44 1.00 Fourth Control 5 605.00 649.00 22.00 0.99 Intervention 5 585.50 605.75 10.12 0.51 Fifth Control 1 639.00 -16.00 1.00 Intervention 11 609.33 636.72 13.65 0.93
Qualitative Data
The primary themes discovered through the analysis of the participant interviews
and the connection to the literature review where appropriate. The participants’ responses
will follow, sequenced by individual teacher and comments by theme. The themes used
are (a) summer learning regression and the achievement gap, (b) access to educational
resources, (c) the academic calendar, and (d) program goals and the curriculum.
Summer Learning Regression and the Achievement Gap Theme
Referenced in the review of literature, summer learning regression negatively
impacted the learning of all students, but had a particularly negative effect on English
language learners’ retention of knowledge. Entwisle et al. (2001) and Heyns (1978) noted
students from all socio-economic backgrounds have remarkably similar learning growth
trajectories during the school year, but out of school time during the summer resulted in a
“faucet effect” (Entwisle et al., 2001), or turning off opportunities to learn for students
that lack access to learning resources during the summer months. In particular, the
summer after first grade was the time when learning losses started to accumulate. Over
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
98
the course of their elementary career, students in the study from low-SES backgrounds
gained less than one point in reading and high-SES peers gained 47 points (Entwisle et
al., 2001) when the researchers looked exclusively at assessments used to gage summer
learning patterns. While the outcomes of the Entwisle et al. (2001) research focused on
differences in socio-economic status, within the district of interest, the buildings with the
highest ELL populations also had the highest free and reduced lunch population and
highest mobility rates. The free and reduced lunch rate is a primary indicator of socio-
economic levels. Table 4.17 details the five elementary school populations with the
highest percentage of students receiving free and reduced lunch, the mobility rate and the
percentage of the building receiving ELL services.
Table 4.17
Percentage ELL, Free and Reduced Lunch and Mobility
Building % ELL Free and Reduced Lunch Mobility Rate 1 26.62 77.94% 61.2% 2 20.17 77.74% 57.2% 3 13.09 74.11% 60.9% 4 13.65 72.70% 55.5% 5 16.53 70.00% 47.0% Note. Data compiled from District Free and Reduced Lunch Rankings (2015), Mobility
Trend Report (2015) and Ethnicity Report (2015).
The responses of interview participants frequently mingled the idea of summer
learning regression and the achievement gap. As such, and due to the connection between
the two concepts, summer learning regression and the achievement gap will be addressed
simultaneously, but separated out when possible. The idea of summer learning regression
was mentioned in the ELL teacher interview responses a total of 61 times, with a mean of
6.7 per respondent. Summer learning regression was mention by all participants, with
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
99
four references to summer learning loss being the fewest and 17 uses of the phrase as the
highest. The following phrases were a sample of the references attributed to summer
learning regression: (a) summer regression, (b) summer loss, (c) bridging the summer, (d)
stayed the same over the summer or (e) advanced (in reference to assessments conducted
in the spring and again in the fall after the intervention), (f) academic loss and (g) close
the gap.
Access to Educational Resources Theme A second theme frequently referenced in the teacher interviews was student and
family access to education or learning resources outside of the traditional school year.
The research of Entwisle et al. (2000) noted the effect of access to learning resources as a
primary cause of summer learning regression. The lack of resources was a theme
regularly references by the teachers of English language learner students, with 41 specific
mentions, with a mean 4.5, about the seven-week summer as an opportunity to provide
resources to students through the summer months. The phrases that were counted
included: (a) extra support, (b) field trips or learning experiences, (c) language support at
home or home language, (d) income gap and (e) building schema.
The responses of the educators acknowledged a number of potential causes of
summer learning regression and no matter the reason; educators believed summer
learning loss had a significant impact on the learning of ELL students. The responses also
suggested the summer intervention played a role in limiting the impact of summer
learning regression since students continued to have opportunities to learn and practice
English through the traditional vacation time. The connection between summer learning
regression and the seven-week summer intervention led to the next theme: the impact of
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
100
the academic calendar on student achievement.
The Academic Calendar Theme
The academic calendar was another frequent theme, in terms of structure for the
summer intervention, increasing the program to provide more language or instructional
support for all English language learner students year-round, not just for students with the
most need. General thoughts on instructional calendar included the luxury of time the
summer intervention afforded to develop concepts in more depth. Overall, the ELL
teachers’ responses favored the idea of modifying the calendar. However, the form and
structure of the calendar were not identified with consistency. In addition, the responses
suggested the modified calendar was good for ELL students, but the teachers themselves
may not be interested in working in the situation.
Curriculum and Program Goal of Developing English Language Proficiency Theme
The theme of curriculum is outside the parameters of the study, but as a frequent
response and tied to program goals of language acquisition, it was not easy to separate. In
particular, interviewees spoke to the notion of the thematic units or project-based learning
used in the summer intervention was tied to student achievement.
Within the district, the expectation is for teachers to use the Readers and Writers
Workshop model, with appropriate modification for ELL students to be able to access the
content. During the summer intervention, the workshop model continued to be the
expectation, but with the latitude to incorporate more hands on, experiential and thematic
units to full develop student language skills.
Teacher Interviews
Teacher interviews consisted of the recorded responses of the ELL teachers that
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
101
had students that participated in the seven-week summer intervention. The interviews are
grouped by individual and contain brief descriptions of the interview situation and
environment. As well, content for the response, such as the question the respondent was
answering, has been included to create fuller picture of the response, without interpreting
the response.
Teacher A To interview teacher A, the researcher contacted the teacher by email and a time
was selected that was mutually convenient. The interview was conducted at the work site
of the interview candidate and the first order of business was to review the informed
consent agreement. The interview transcript was recorded digitally and sent for
transcription. The researcher reviewed the transcript, removed names and other
identifiers, and changed acronyms as necessary that were not understood by the
transcriber. After this checking process, the transcript was emailed to the participant for
member checking and approval for use.
Summer learning loss. In response to a question about reading comprehension,
Teacher A commented on anecdotal differences between students that have participated
in the intervention and students that did not participate:
The students that I have sent there and have come back, what I have noticed is
that they did have better reading. They didn’t have the summer loss of reading
which was great. And some of my students who did not do well during the year,
eventually went on to get it, so that may have been, you know, helpful for them,
but they still needed that extra support through more than I could give them.
My students, I was really impressed with last year some of the data that I was
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
102
given after the program, that none of my students had any loss at all over the
summer. So that was a really great thing, I thought, and then that kind of kept
them going for the reading and they were excited about reading more than they
had been before.
In essence, the teacher’s perception of student success through the year hinged on the
additional support the student received through the summer and had assessment data to
further support the assertion of student success. As well, Teacher A defined the purpose
of the intervention and the association between the summer intervention in these terms:
Comparing students who, when looking at the data, comparing students who
attended the program compared to students who have not attended the program
and without remembering the exact percentages, has consistently been, the
students have, ok, the overall group of students have maintained their reading
level or grown or increased their reading level at higher percentage than the
students who did not attend summer school. That’s the piece we’ve all been
grasping this whole, that’s how we know that it works in closing the gap for our
ELL students.
An impassioned description of summer learning regression and the positive impact the
intervention has had on closing the gap for ELL students.
Academic calendar and curriculum connection. These responses reflect the
complexity and the interconnectedness of many of the issues explored in the review of
literature. The responses addressed the pace of instruction during the school year versus
the summer school. In addition, the link between the summer school curricula versus
school year curricula was covered. Teacher A explained thinking about the calendar in
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
103
these terms:
When I taught ELL summer school in the past the kids really had a chance to
delve into a subject area and learn a lot about it and learn the vocabulary and
really feel like they had gotten something. And they really enjoyed that. I think
that is something that I noticed as a teacher in the summer ELL program that I
thought worked really well.
The first half of the response delved into the pace of having additional instructional time
in the summer, while the second half of the response is directed specifically toward the
instructional pedagogy. Teacher A claimed:
I think that themed learning really works for kids, because when I taught ELL
summer school in the past, we had themes for the kids and so then they really had
a chance to delve into a subject area and learn a about it.
The educator also noted, student comments about the instructional activities. While the
information is relayed through a third party, it helps establish some additional context
surrounding the response. This was the student view through the teacher: The one thing I
did hear from the students was “it was more like hands-on learning, like group activities,
where you weren’t sitting down doing worksheets.” Learning in a different kind of way,
that is more helpful for them.
Intervention impact on assessment. In response to a question on standardized
assessment, the educator’s perceptions of student outcomes after participation in the
seven-week summer intervention included a belief that students made positive gains, with
the exception being a student that was having struggles in areas beyond language
acquisition. Teacher A pointed out:
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
104
I believe that my students increased on those assessments. All of them except one
did not and that was because the student had some other issues going on that I
don’t really think should have been included in that group because the student is
gone now. But, the rest of the kids did really well. Most of my kids were at least
proficient on the MAP test. I had really good scores last year. I was really proud
of my kiddos, so the kids that did go, I think it definitely helped them.
In all, the interviewee’s responses were thoughtful and focused on the needs of students.
As noted previously, many of the topics intertwine and could be included in multiple
categories.
Teacher B
Teacher B was interviewed at an off-campus location that was convenient for the
interviewee after the initial email contact. The research followed the same procedure of
providing informed consent, conducting the interview, transcribing the interview,
checking the interview for accuracy, member checking and approval was used.
Eliminating summer regression through teaching forward. The take of this
teacher was to combat summer learning regression by pre-teaching to the next grade,
rather than trying to catch up missed skills and language. Teacher B explained why this
summer program was beneficial for student learning:
I think that during the summer program, I was able to kind of frontload them or
prepare them for the following school year. So what I saw and for my students
who participated in it, they were more prepared for some of the concepts that they
needed to learn in the next grade level. So I would say the overall time impact
was highly positive.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
105
The process of pre-teaching is used as an ELL instructional strategy for building schema
for upcoming units or days of lessons, but is not typically applied to the next grade level.
¡Colorín Colorado! (2007), a bilingual website for ELL professionals and families of
ELL students explained pre-teaching as an essential vocabulary skills necessary for
reading comprehension that can be accomplished through a variety of visual learning
clues tied to the specific English language vocabulary (2007). The concept of pre-
teaching toward the next school year guided thoughts to the confluence of time and
building context by offering student experiences the might not otherwise have access to.
Confluence of time and educational resources. Teacher B responded in a
similar manner other interview participants while answering the question about program
purpose. There was insight into the connection between access to educational resources
that could be provided to the students participating in the intervention and the impact of
additional instructional time. The response included, “they get to participate and
experience things that they wouldn’t otherwise have. We just provided them experiences
that we couldn’t during the school year and their families wouldn’t through have
otherwise during the summer. When asked about the impact of time on student learning,
Teacher B stated:
In terms of short term versus long term learning goals, I think that that’s true for
any kid, regardless of their English language abilities. Being a regular classroom
teacher and having an entire year with students versus those six weeks, I think
that with ELL, it does take a longer time, a longer amount of time for them to
acquire a new concept because language is conceptual and so I think they have to
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
106
be just constantly exposed to that content and to the language of that content and
the summer helps.
Thematic learning units. As with many of the teachers interviewed, Teacher B
believed the thematic units of the program helped students make connections between
their prior knowledge and new learning and maintained:
So the summer learning program is really thematic, which obviously allows for
students to make connections between their prior knowledge and something new.
So integrating just different disciplines, having a culminating experience
connected to that theme really solidified the content that they were working on.
It’s too bad the rest of the school year can’t be more thematic, like the summer
program is.
Teacher C
Teacher C was interviewed in person at the employee’s work site after school was
dismissed for the day. This teacher was initially contacted by email and subsequent
scheduling conversations were conducted by phone. All other procedures remained the
same.
Summer learning regression. When Teacher C responded to the question about
English language acquisition of students that participated in the intervention, these were
the reflections about summer learning regression:
There is a definite notice that the slide that typically occurs with our ELL students
isn’t as big. The gap isn’t as large for students that participated. The students
don’t lose their second language skills they had developed before leaving in the
summer. Which would normally happen if they didn’t have the seven-week
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
107
program.
The theme of summer learning regression and the direct impact on English language
acquisition skills in evident. The teacher’s perception was the students that participated in
the intervention and had the opportunity to continue language practice maintained their
skills. In addition, the impression of students’ reading comprehension scores mirrored the
perception of language acquisition and included, “assessment wise, if students didn’t
attend the intervention you would see a bigger drop in the reading level scores at the
beginning of the year than you do in the students that have the opportunity to attend the
seven-week program.” Asked about the overall success of ELL students that participated
in the intervention, the response referenced student reading comprehension scores on the
Fountas and Pinnell Benchmark Assessment System 1 and 2 and articulated the
following:
I’m not sure I can answer the question fully, other than what I’ve noticed in the
Fountas and Pinnell scores, in that they don’t lose their ability to comprehend
what they’ve read because they continue to read in English and they continued to
speak to people in English throughout the summer.
The perception of student outcomes on the state standardized assessment, the MAP test,
after participation in the seven-week summer intervention followed a different trajectory
due to the composition of the student population. Either the students were to young to
participate in MAP testing or they had not meet the one-year minimum residency
requirements established by the state of Missouri to participate in the assessment.
Teacher C illustrated the point this way:
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
108
Honestly I don’t have an answer for that because many of our, our lowest level
learners that attended the program have not taken the standardized test yet, either
they are not old enough or they are exempt due to the length of time in the
country. So I do not have an answer to that.
The calendar and servicing more students. Questioned about the impact of
length of time for the intervention, Teacher C commented by comparing the district’s
former summer school model:
Our district’s traditional summer school model and that kind of timeframe isn’t
long enough, based on typically being Monday through Thursday and a shortened
day and only being four weeks long. The added three weeks and the opportunities
and the skills simply facilitate their language learning and academic skills that
they would normally continue. I would love to see it expanded and not simply be
for the lowest level learners, but also for the upper level learners as a way to work
on mastery and simply delving into the concepts more deeply. I would love to see
it bigger and more availability.
Thematic units. The perception of thematic units aligned with the thinking of
many other interview participants. Teacher C thought themes added value to the language
instruction and added:
I understand the program focuses on units and themes to focus on language
instruction. It involves a lot of field trips to give them schema and real life
experiences to give students background knowledge. All the experiences are tied
to the weekly themes.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
109
In addition to the focus on thematic units, Teacher C responded in a manner that also
heighted the opportunity for providing educational experiences to students that might not
have them otherwise.
Teacher D Teacher D was contacted by email. The researcher followed up in person through
a conversation with the participant. This interview was conducted by phone. Prior to the
start of the interview, the participant was read the informed consent agreement. A copy of
the consent agreement was sent by district mail to the teacher. Other procedures remained
the same.
Summer regression. Summer learning regression was identified by Teacher D
when asked about the impact of length of time had on students that partook in the
intervention. Initially, the respondent did not think the length of time played a role, but
while continuing to expound upon the response realized:
I think the length of time wouldn’t necessarily play a role in the students’
achievement. But the type of program is what played a role. Their scores, on their
reading score, language acquisition scores, I think that having the seven weeks
helps so there is not summer regression.
Furthermore, Teacher D mentioned the perceived connection between access to
educational activities outside of the school year and offered this response:
I would tell her what the kids need as far if they come from a, if their family life,
if they have a poor family life, you know, then they would give those kids a little
more TLC, but I also think about giving those kids more experiences, to provide
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
110
them with opportunities. With learning opportunities they wouldn’t have if they
were not in summer school, if they were not in the summer learning program.
As well, the educator added views comparing the opportunities the students have during
the summer intervention and the opportunities available during the regular school year.
The teacher injected, “the summer learning program has a field trip every week to build
background. In regular education, we need to have more field trips and more project
based learning, again adding additional thinking on the perceived differences in
curriculum.
Instructional calendar. Teacher D had a dissenting opinion about modifying
instructional calendars and the impact on student success and suggested:
I don’t necessarily think we need a longer school day. I don’t necessarily think we
need to add more days to the calendar. I think need to be out in the community
more and giving our students more real world experiences.
The teacher elaborated about the hands-on learning experiences students are able to have
with science activities and personal experiences with their child.
Intervention and assessment. Asked about the impact of the intervention on
English language acquisition assessment, Teacher D indicated:
Honestly, I can only think of three students off the top of my head and I don’t
remember their scores from before and after, but if English language acquisition
in any way mirrors their achievement, I mean, in reading and writing, then I
would say it didn’t have much effect on these students. Right now maybe it
didn’t, but then I think down the line it will. If it hasn’t right now, it will later.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
111
In response to student outcomes on the state standardized assessment, the MAP test, after
participation in the seven-week summer intervention, Teacher D rationalized:
I honestly don’t know, but would assume that their score either (a) stayed the
same or (b) went up a little bit. I mean, maybe those kids are still in the basic
range, but maybe they’re mid-basic rather than low basic. Or, I honestly don’t
know off the top of my head, but just, by thinking of the students, you know,
without looking at their score for almost one year, without having those right in
front of me. Like I said, I would imagine it would raise their scores a little bit at
least.
Teacher E
Teacher E was contacted by email, and follow up was conducted through text
message to schedule an interview time. The interview was conducted at an off-campus
location. The interviewee was provided with the informed consent, interviewed and
provided the occasion to member check after the transcription.
Program goals and learning loss. Teacher E explained the perception of the
correlation between program goals, language skills and summer learning regression:
That it’s to improve the English language acquisition of our students and the goal
is to, with that, is to close the gap that tends to happen during the summertime.
We leave in the spring and come back at the beginning of the school year the
following year, there’s always a learning loss. In addition to a learning loss, our
students also have—so an academic loss—but they also have a language loss if
they go home to their families and speak their native language with their families,
they lose a lot of English they’ve been developing.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
112
The response explained one of the purposes of the program, responding to learning loss
for in a high need student population, and identified a potential reason for the learning
loss among ELL students. Teacher E’s response examined the connection of native
language use during the summer vacation time and the impact on student’s second
language learning. The thoughts of Teacher E elaborated on the use of English at home
during the summer months:
They also have a language loss if they go home to their families and speak their
native language with their families; they lose a lot of the English they’ve been
developing. The goal is to have them in school more time so they are continuing
to develop English in the four domains of speaking, listening, reading and writing.
In another answer, Teacher E’s response highlighted the linkage between educational
resources, curriculum and the calendar.
Trajectory of students after intervention. This response expounds on the
benefit of time a change in the calendar has on ELL student learning, addressing access to
educational resources and how these ideas impact learning regression. Teacher E noted:
It is a huge piece of what our district needs to explore, like the seven-week
program for all students. I would say the seven-week program indeed has a
positive impact. I would say, if we could afford it, or the state could afford it, or
however it works, I think other districts will look at what we are doing and see it
works. But I certainly don’t think that twelve weeks or eleven weeks or whatever
it comes down to in the summer for students who don’t go to summer school or
don’t have other learning experiences with their families, I certainly don’t think
twelve weeks is detrimental and if you look at the research they have shown.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
113
Like one student, you know, in a certain income bracket, the family is in a certain
income bracket, compared to a student in a higher income bracket, they start at the
same starting block. But each year, that they go through the school year, and then
one has summer learning experiences with their family and the other one does not,
by third grade their gap is so huge, with their reading level and their vocabulary
range.
If that gap is there it is very hard to close. Another example, student A qualifies,
but for some reason doesn’t come to the summer program and student B qualifies
with the same data as students A, but student B attends the summer program. If all
the circumstances were very similar, Student B would exit ELL services would
more quickly and would advance more rapidly through the progression of the
proficiency standards.
In summary, the response addressed the end goal of ELL services; exit ELL students
once they have gained adequate English language skills to perform at a level equivalent
to their native speaking peers. In addition, identified the purpose of the study: does the
intervention have an impact on ELL student academic achievement?
Teacher F
Teacher F was contacted by email and through a follow up phone conversation an
interview time was scheduled. The interview was conducted in the participant’s office
after students were dismissed for the day. Informed consent was gained and the interview
was conducted. Next, transcription occurred and the participant gave final approval of the
content.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
114
Reading comprehension and intervention participation. In response to a
question about reading comprehension outcomes, Teacher F stated the facts presented to
the staff of the ELL department:
I know in some of our ELL department meetings, we looked at graphs that
showed how the students that were in the program compared, how their reading
scores compared to students that were not in the program. I know that showed that
it did make a difference. It made a positive difference for the students that were in
the program. So essentially, students that were in the program either retained their
reading level or increased and that reflects my thoughts on how that impacts the F
and P (Fountas and Pinnell) assessment.
In the perception of this ELL teacher, reading comprehension data presented during
departmental meetings supported the teacher’s thoughts that students participating in the
summer intervention made maintained or showed growth over the summer, lessening the
summer learning gap.
Impact on other assessments. Interviewees also responded on their perception
of student outcomes on the state standardized assessment, the MAP test, after
participation in the seven-week summer intervention. Teacher F illustrated thinking this
way:
Well, so this would tie back, so I believe the program is increasing their abilities
to connect to the units of study and I believe, it’s also improving their academic
language and their social language, then, I’m also of the belief it also will increase
their standardized test scores in the MAP test.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
115
The response connected language acquisition and usage to a broader range of
application, including standardized assessment. Moreover, in response to a question
about language acquisition surmised, “my perception is that certainly the program is
never going to hurt them…and you could literally see, you could really hear and see the
growth in their language skills” and ended the statement with the belief in the benefit of
the intervention.
Curriculum Connection. Teacher F echoed the thought of Teacher B for
preparing students in advance for the upcoming school year these points:
I look back at the purpose of the program, and I think, okay, these second graders
that I’m working with, as they move into their next year, these are the things they
are going to hear and need to know and they are going to have that background
knowledge. Students had more access to the curriculum and they had some
refinement to the previous units of study.
Teacher G
Teacher G was contacted twice by email and responded after the second request
for participants. The interview was conducted by phone. Informed consent was given
orally and the informed consent letter was sent by district mail. Other procedures in the
process remained the consistent for this participant as with the others.
Intervention impact on assessment. Teacher G was emphatic with a belief the
program did not impact English Language acquisition:
Through personal observation, I would say that students are able to articulate
some of the field trips they’ve gone on, they’re able to articulate some of the
experiences that they were a part of, but from a language, purely a language
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
116
output perspective, like I said, in the previous question, I’m not sure that I’ve
directly seen a correlation on an assessment.
Contemplation continued on the impact of the intervention upon on other
assessments and areas of student learning included these perceptions on standardized
assessment:
Well, I know that this is the first year my students, and I work primarily with third
graders in my build, this is their first exposure to the MAP test. It seems as if the
language is very, very rigorous for them this year. That the vocabulary, the
content of the story, the formatting of it on the computer was a little bit of a
learning curve for some of them in the look of the <inaudible>, the left side of the
screen, the stories were like in a sidebar. So the formatting of it is just kind of
created the perception for the students that it was a very long passage. In reality, it
was not that long, but it was the formatting of it. So, students I feel like struggled
with, you know, main ideas kind of weren’t concrete, so, I think it is very difficult
assessment for the language learner. If that first session was five questions long
and it requires a lot of stamina from the language learners especially.
Filling educational need during the summer. Teacher G did have a more
positive take on the intervention’s impact in filling the educational resource void students
may encounter during the summer. Teacher G, while responding to the purpose of the
program included, “They’re also given additional opportunities for exposure and
awareness by doing real-life field trips to places they wouldn’t usually go with their
families, to expose them to little bit more than they would be exposed to at home” and
summarized the purpose of the intervention to give students and intensive experience that
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
117
supplements the instruction of the school year. The intervention provided an opportunity
to “dig deeper with some of the language objectives” and was geared specifically to the
student’s current level.
Calendar and intervention time frame. Not modifying the instructional
calendar was on the mind of Teacher G and provided the following thoughts on the
impact of the intervention on reading comprehension:
I think from an idealist’s perspective; I want it to work. I’m not sure. I’m not sure
on paper how much I’ve seen that carry through with my students this year. I’ve
heard them talk about the field trips or we did this or we experienced that. I’m not
sure I see a direct carry over to their reading levels or their writing skills or their
math skills.
In addition, Teacher G voiced a strong opinion about the length of time for the
intervention. The educator articulated these thoughts:
I think overall, my perception is that the seven-week system is a little bit longer
time and a little bit more continuity maybe in carrying through some of the
learning objectives and structures that fit into their summer learning. I think, and
this is my opinion, obviously, but I think that the few students that go for both of
the sessions, if they were to go to both, that seems like that would maybe be a
little long and students would maybe tire of it easier and quicker.
The conclusion of these thoughts focused on the better training for teachers that work
with ELL students in the general education classroom and giving these teachers
additional resources to meet ELL student needs.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
118
Teacher H
Teacher H responded very quickly to the first call to participate. A few
subsequent emails were exchanged to schedule and meeting and there were a few
reschedules due to last minute conflicts. The interview occurred in the classroom of the
ELL teacher and a similar process was followed with this interview as with the previous
interviews.
Supporting students through the summer. Teacher H expressed many reasons
why providing extra support through the summer intervention is important to ELL
students. Queried about the purpose of the program, Teacher H responded:
I believe the purpose is to keep the children more involved in a school on a
regular schedule and expose them to many opportunities that they wouldn’t
normally have, they wouldn’t be exposed to by their families. In addition to
keeping skills sharp that they learned during the year, so they don’t regress over
the summer, I think one of the important things was the field trips they were able
to take and also to keep the skills sharp in terms of the units of study so that they
have a better understanding of the world around them and are constantly building
background in a way they would usually do without this program.
The reply showed the interwoven relationship between all the concepts and issues
surrounding ELL student education. This interviewee further explained when asked about
the impact of the length of time for the intervention:
I mean the schedule of school; they’re getting breakfast and lunch at least. We
provide transportation and maybe some of the kids, would not have, I don’t know,
quality daycare. Families get by the best they can and this is not to sound judgey
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
119
or anything like that, but I think that families really appreciate knowing their
children are in a quality program for the summer instead of them trying to
scramble and try to find someone to watch their children.
In addition, support through the summer extended to expanding learning opportunities.
Teacher H mentioned:
Any extra you can give them helps. I liked the seven-week program because it
gave the students more time and more opportunities to experience more things. I
think, I definitely think the program should be seven weeks and then they do get
two weeks break before school starts again. So they get a different experience, yet
a beneficial experience. I would like year round school, not to be confused with
extended calendar, for everyone. I think it just makes sense. You don’t have big
lags in education, you keep going and to me it just makes sense.
Through multiple questions, the responses came back to the idea of meeting students’
social, academic and physical needs to make up for lags in achievement.
Intervention and assessment relationship. All interviewees were asked about
the impact of the intervention on specific assessments ELL students were taking
throughout the school year. When asked about state standardized assessment, MAP,
Teacher H described it as:
Any, any extra you give them helps. I can’t think of a specific example, but of the
students I’m thinking of that did participate in the program, I’ve been really happy
with their progress. I don’t have a specific example with state assessments that I
can recall.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
120
In response to reading comprehension and the opportunities for field trips to build
schema, noted, “It certainly helps their comprehension. It certainly helps their
comprehension when you are reading about a reptile or amphibian and you can say
remember what we saw, so then the words make more sense.” A real life experience is
more impactful on reading comprehension than other visuals such as pictures or realia.
Teacher I
The interview for Teacher I was set up through an initial phone conversation and
conducted in the office of the interviewee. The same process of informed consent,
interview, transcription, member checking and approval was followed for this participant.
The intervention goal and learning regression. Teacher I, responded to the
question about the purpose and goals of the program and commented on summer learning
regression in the following manner:
So our students we invite to the program are the students with the lowest English
language proficiency and include newcomers within the previous two academic
years or are significantly low in reading or they are below basic on the state
assessment. So any of those criteria qualify a student for the program. In addition
to the fact that students typically have summer learning loss, a gap in learning that
occurs between June, July and August when they return to school. So our goal and
our purpose is to close that gap or slow it down for our students in the highest
need.
Teacher I identified one the primary indicators used by the district for initiating the
seven-week summer learning intervention for ELL students. The district had noticed the
ELL students with the most language need, were returning in the fall with some of the
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
121
largest learning regression.
The intervention and the calendar. Thoughts about summer learning regression
fomented into the length of time for the intervention and calendar structure. Teacher I
rationalized:
In addition to the fact everything was different, the schedule was different, the
teachers were different, I don’t think the duration of seven weeks, I think that
helps tremendously. I didn’t see, the students did not seem fatigued and have
academic fatigue at all. My perception is that, the seven-weeks, they can handle it,
they don’t get fatigued and from the data, it speaks that, it is a positive effect.
They don’t lose what they learned and in some cases even make gains.
The impression of this educator, the time was adequate and had a positive outcome for
student learning.
The intervention and curriculum. Teacher I elaborated the most on themes and
the connection to student learning and provided these details:
I think students are typically excited about summer school. I think that the
teachers overall enjoy it. I definitely think that has something to do with the
environment that’s created and the fact that there is, I think there’s something to
be said about having thematic units. I really think they are effective and I don’t
know what the pros and cons are really, really stacked for or against this type of
planning, but I, that holistic approach is very many positive outcomes as I can see
and really no negative outcomes that I can observe.
The way that this was developed or planned from the beginning, I wasn’t part of
the first year, but the idea and the research behind what is called thematic
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
122
learning, is a little bit different than what the students experience in the
classroom, day-to-day, throughout the regular school year. But the idea is that
you’re taking all the content areas, math English Language Arts, science, and
social studied and you are, perhaps you have one theme, for example life science.
So students are learning about life science through their ELA, science and it could
be scaled down like animals at the zoo or mammals or something like that.
So whatever the theme or topic is, it’s infused through all of those content areas at
the same time. They’re using speaking, listening, reading and writing throughout
the entire day. Intentional planning and intentional structure so that you have an
end goal in mind for what you want students to be able to produce in speech and
produce in writing.
That does not occur during the regular school year, although it would be nice if it
did because it makes all the learning in context, you know, instead of being an
isolated topic. Think about how much more you would gain if you were learning
whole themes or a whole concept throughout the entire day, but through different
lenses. It makes the most sense, but the research behind it shows that students
really retain more information that way.
The intervention and assessment. All interview participants were asked about
the impact of the intervention on student English language acquisition assessments.
Teacher I pondered the question with this thought replied, “my perception is, overall, an
overwhelming majority, that they all made gains,” but also suggested that due to when
the assessment is given during the school year, other factors in conjunction with the
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
123
intervention created the positive outcomes. When asked about the MAP assessment
explained:
I would say just a handful of our students that are invited are, really just a handful
of our students are below basic, which is the lowest measure and that’s the
measure and criteria we would use to invite students to qualify them. So getting
back to your question, I ascertain they make gains, but its kind of difficult to
measure, it’s apple to oranges on the assessment, but the positive, one good way
of looking at it that fewer are in the red, the below basic category.
When teachers answered questions about student outcomes on reading comprehension
after participation in the intervention, this was how Teacher I countered:
I would align it again to the F and P assessment, the reading scores so again, the
students who have attended all or part of the seven-week summer school
intervention program, a majority in the two years we have looked at, remained at
the same level or make increases in levels during the intervention.
Overall, the educator interviews provided specific details about their perception of
student outcomes after participation. In general, the perceptions seemed positive and at
the very least, would not have a negative effect.
Summary
This chapter presented a review of the study methodology and the research
questions. In addition, this chapter provided a detailed description of the data collection
procedures for the qualitative and qualitative phases of the study. Quantitative data
analysis provided the results of student achievement in four cohort groups and used three
measurement tools. The tools measure students’ assessed student reading comprehension,
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
124
English language acquisition and standardized assessment scores on English Language
Arts. Each cohort group was disaggregated by assessment type and grade level.
Qualitative data analysis provided the results of interviews on the perceptions of student
outcomes with ELL teachers that had students that participated in the intervention.
This mixed methods study used a linear regression to examine the quantitative
data and developed qualitative themes based the ELL teacher interviews. The quantitative
data analysis revealed early intervention had a cumulative effect; in particular, reading
comprehension intervention at the earliest age moved ELL students’ reading
comprehension scores to similar levels as their peers. Other reading comprehension
outcomes included trend lines indicating higher annual growth than expected for grade
level peers. The results from ACCESS score analysis, the English language acquisition
tool, the results suggested the intervention cohorts trended higher and the control cohorts
trended lower than the expected growth goals. MAP results are inconclusive, however,
data for the 2011 intervention cohort suggested higher trend lines at all grade levels and
students ended a higher mean.
The qualitative results suggested a common perception the intervention was
beneficial for students, with several interviewees citing assessment data, in addition to
anecdotal impressions. The primary themes included: slowing summer learning
regression, providing students with access to resources, the additional time provide
through extending the academic calendar, and the use of learning experiences that tie to
the curriculum.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
125
CHAPTER FIVE: CONCLUSIONS AND RECOMMENDATIONS
Introduction
English language learner (ELL) students are an increasing portion of the student
population in the United States and as noted by Heritage et al. (2015), “the number of
students who are ELLs has grown from two million to five million since 1990, vastly
outpacing the growth in the overall school population. However, in general, academic
outcomes for ELLs remain stubbornly low” (p. 1). The need for solutions to combat
summer learning regression and meet the language needs of ELL students is the purpose
of this study.
In this chapter, the conclusions, implications, and recommendations will be put
forth. The sequence of the chapter will include: first, a summary of the study; second, a
summary of the findings; and third, implications/recommendations for future research. A
recommendation for future study includes conducting research that not only features
student assessment data and qualitative data from teacher interviews, but also includes
the third aspect of student and parent perceptions of academic outcomes after
participating in the intervention.
Summary of the Study
This study was a convergent transformative mixed methods study. The
transformative framework was selected to provide the framework for improving
educational outcomes for a marginalized student population. Ponterotto et al. (2004)
explained, the social justice aspect of transformative theoretical research is to empower
the individual to confront injustice and inequality in systemic environments, such as the
educational setting. A mixed methods study was selected to optimize the quantitative and
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
126
qualitative data relationships between student data outcomes and educator perception of
student outcomes. A mixed methods study “mutually corroborates” the quantitative and
qualitative data sets to offer a complete and comprehensive explanation of the results
(Creswell & Plano Clark, 2014).
The quantitative data collection included archived student assessment data for
reading comprehension, English acquisition and standardized English language arts
assessments. Data was collected and sorted by cohort year, assessment type, grade level
and level of participation. Scores on the ACCESS English language acquisition
assessment initially screened students for inclusion in the study. This provided a
homogeneous pool of students. Attendance was the screener used to determine if the
student was in the control group (invited to attend the intervention, but did not attend) or
intervention group (invited and attended).
Assessment data was collected for cohort year 2011 (the first year of the
intervention), 2012 (the second year of the intervention), 2013 and 2014. The
intervention was a seven-week summer intervention, with the first cohort year being a
pilot with about 80 students in attendance from 5 different elementary schools. In
subsequent years, the intervention was made available to all district elementary ELL
students with the highest language need. After data was cleaned, a linear regression was
conducted on the data sets to determine if the intervention had an impact of student
learning outcomes.
The qualitative data was collected by conducting one-to-one in person or phone
interviews with ELL teachers of the students from the intervention. Interviewees were
selected at random and included nine out of 19 elementary teachers from the district of
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
127
interest. The researcher used semi-structured questions to ensure a consistent questioning
format, but also allowed the flexibility to pursue ideas that needed additional context,
further explanation through a natural conversational approach. Prior to starting the
interview, each participant was provided informed consent. All interviews were recorded
digitally and immediately sent for transcription. After transcription was complete, the
researcher removed personal identifiers of the interviewees, students, or other district
employees. Each interviewee member checked the transcription and gave approval for
use.
For the data analysis, each transcript was read for initial thoughts and to develop
broad themes. Next, key words and/or phrases were recorded in a spreadsheet, grouped
by participant and question for coding purposes and subsequently analyzed for themes
across all the interviews.
The research questions used to focus the study are tied to the mixed methodology,
with one question for each aspect. Envision a funnel with quantitative and qualitative data
being added through the wide mouth of the opening. As the funnel narrows, the
quantitative and qualitative data are distilled down through the narrow opening with the
results being a response to the central question. Finally, the transformative resulted from
the mixture of all the parts into a recommendation for the betterment of ELL education.
The questions that focused the study were:
1. Central Research Question - What are the congruencies between archived
English Language Learner students’ assessment data in reading
comprehension, English language acquisition and standardized assessment
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
128
and the beliefs of English Language Learner teachers’ about student academic
outcomes after students participated in a seven-week summer intervention?
2. Quantitative Research Question - Does a seven-week summer intervention for
English Language Learners influence longitudinal growth (growth toward
grade level proficiency standards over multiple years) in reading
comprehension, English language acquisition and statewide normed
assessment outcomes?
3. Qualitative Research Question - What were the English Language Learner
teachers’ perceptions of students’ academic outcomes after students
participated in a seven-week summer intervention?
4. Transformative Research Question - What educational policy considerations
can be derived from the outcomes of this research?
The research questions guided every aspect of the study form the formulation of the
design and methodology, through the research of literature, collection and analysis of
data, and finally, the summarization of findings.
Summary of the Findings
Central Question
The central question is focused on the commonalities between the findings that
emerge in ELL student assessment data and ELL teacher perceptions of student
achievement after participation in the seven-week summer intervention. First, student
assessment data revealed reading comprehension growth aligned with the teacher
impressions of student outcomes after participation in the intervention. Second, teacher
impressions about student outcomes in English language acquisition on the ACCESS
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
129
assessment intimated that the intervention influenced student learning in a positive
manner and that thinking was borne out on the student assessment data findings. In
addition, the inconclusiveness of the MAP assessment data associates with the educators’
opinion that it was hard to determine of the intervention had an impact on the MAP
assessment.
In summary, teachers believed the intervention offered an opportunity for students
to continue learning and provide educational resources through the summer in ways
families may not be able to provide. Slowing or stopping summer learning regression was
one of the purposes of the intervention, along with closing the achievement gap. The
student data implies these opinions of student outcomes were substantiated.
Quantitative Question
The purpose of the quantitative phase of the research was to determine if the
seven-week summer intervention had an impact on the academic achievement of students
after participation in the intervention. Three types of assessments were, examined. The
Fountas and Pinnell Benchmark Assessment System 1 and 2 were used to assess reading
comprehension. English language acquisition was assessed through the WIDA ACCESS
and the standardized ELA content was assessed with the Missouri Assessment Program
(MAP) annual test. These are the findings for the quantitative phase of the study for each
assessment measure. It is important to note, both the control and intervention students
were derived from a homogeneous pool of students that scored below a minimum
threshold (below a 3.5) on the WIDA ACCESS English language assessment. Families
decided to participate or not participate at the time of the intervention and the
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
130
differentiation into control or intervention group occurred at the at the time of the study
by the researcher, based on attendance registers.
Reading comprehension. The 2011 intervention cohort had four out of five
grade levels trend higher than the control group. The intervention cohorts’ average trend
was 5.83 levels of growth compared to an average trend growth of 3.23 levels among the
control group.
Reading comprehension for the 2012 cohorts, the intervention cohorts had higher
trend than the control for one out of the six grade level groups. The average trend for the
intervention cohorts was 4.11 levels of growth versus an average trend growth of 4.36
levels for the control group.
For the 2013 cohort groups, all intervention cohorts had higher trend lines than
control cohorts for reading comprehension. The intervention cohort trend average was
5.03 levels of growth, while the control cohorts’ average was 3.78 levels of growth.
In comparing reading comprehension trends for the 2014 cohort groups, four out
of the five grade level intervention cohorts had higher trends than the control group. The
average growth for the intervention cohorts was 3.23 levels and the control cohorts
averaged 2.08 levels of growth.
Table 5.1
Reading Comprehension Growth Trends
Cohort Years 2014 2013 2012 2011 Levels of Growth
Intervention Cohort 3.26 5.03 4.11 5.83 Control Cohort 2.08 3.78 4.36 3.23 Note. Adapted from data previously presented in Chapter Four
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
131
The data in table 5.1 suggests that reading comprehension growth trend rates
accelerate for students that have participated in the intervention. The farther away, the
higher the growth trend, indicating the intervention does have a positive correlation to
reading comprehension. This growth trend moves the students that participated in the
intervention closer to their peers.
English language acquisition. Students participating in the 2011 intervention
cohort realized higher trend growth than the control groups at all grade levels in English
Language acquisition. The trend average for the intervention cohort was 1.15 compared
to 0.825 for the control cohort. The goal for students is advance by one level per year
until they are able to exit ELL language services. If two students started at the same
point, the intervention student would gain 1/3 of a year toward that goal over the student
that did not participate in the intervention, in effect, adding one year more toward the
goal.
ACCESS trends for the 2012 cohorts demonstrated that three out of five of the
intervention cohorts surpassed the trend of the control cohorts. Two grade levels did not
have a comparison. Both the intervention and control cohorts reported 0.86-trend growth,
putting both slightly behind expectations.
English language acquisition trends for the 2013 cohorts showed that three out of
the seven intervention cohorts had higher trend lines that the control group. The average
trend growth for the intervention cohort was 0.84 and the control cohort experienced 0.81
trend growth. Both are slightly below the anticipated annual growth.
Overall, 12 out of 18 intervention grade level cohorts trended higher than the
control cohort peers, indicating students that participated in the intervention had a
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
132
positive growth outcome and have the potential to exit ELL services faster. In addition,
the data indicated students in pre-kinder, kinder and first grade had higher average trend
growth.
Standardized assessment. For students that participated in the 2011
intervention, three out of the five grade level cohorts trended higher than their control
group peers. The intervention cohort averaged 18.24 scaled score trend growth versus
15.79 scaled score trend growth for the control cohort. For the other two years, 2012, and
2013, there was not adequate data to draw conclusions. Overall, the standardized
assessment data is inconclusive.
Qualitative Question
The qualitative phase of the research was framed by the question: what were the
ELL teachers’ perceptions of student academic outcomes after English language learner
students participated in a seven-week summer intervention. In the responses from
teachers of ELL students that participated, the teachers’ interest in improving outcomes
for ELL students, whether students participated in the intervention or not, was apparent.
All questions elicited a common frame of thinking around improving students’ prospects
for academic success through enhancing the learning experience during the school year or
the intervention. But the idea of academic success proved to be very complex, tied to the
families’ financial resources and ability to expose their children to learning experiences,
provide a trip to the library for book and sometimes ensure adequate food through the
break.
However, improving the students’ opportunities offered a multi-layer approach,
including expanding leaning prospects through the summer. This concept flowed into the
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
133
curriculum of the intervention. While the impact of the curriculum was outside the scope
of the study, the frequency with which it came up, required it be addressed. All of these
opinions highlighted the key purpose the starting the intervention, slowing down summer
learning loss.
Summer learning regression. The responses from educators made apparent
their desire to improve summer learning regression for ELL students. Their perception
was the intervention had a positive impact on overall student achievement and reading
comprehension of the students that participated in the intervention. In terms of the impact
on standardized assessment, the MAP, the consensus was less clear, but at the very least,
it would not hurt student outcomes and over time would be a benefit. A similar line of
thinking was evident in the findings for the English language acquisition assessment,
ACCESS. In particular, teachers identified the instruction through the school year as
having the most impact on the ACCESS scores. For both MAP and ACCESS, the length
of time between the assessments, once annually, made it hard for teachers to make an
inference as to the impact of the intervention on these assessments.
Access to resources. A second thematic idea that became evident was the ability
of the intervention to provide academic as well as social support for students. A frequent
retort was that the intervention gave students the opportunity to have experiences they
might not have otherwise. As part of the intervention, students had weekly field trips that
tied to the instructional units. For example, a social studies unit was focused on pioneer
life in Missouri and the students’ field trip included visiting a working pioneer village
and wagon rides on westward leading trails, direct link between instruction, vocabulary
and experience.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
134
In addition, being in school provided daily chances for students to have access to
books and reading instruction through a model similar to the traditional classroom. Not
only did instruction continue, the social network of peers encouraged many students to
attend and have the chance to maintain English through daily practice with friends and
teachers. Many teachers also described the parents’ perspective of knowing their child
was in a safe, caring environment and were going to get two meals a day.
Academic calendar. The educator interviews also identified the calendar as a
theme. Three sub-themes emerged: time to develop the curriculum and teach at a deeper
level, a time to maintain skills, and the extra time can be used to pre-teach the content of
the next grade. Looking at these three subthemes, it is noticeable the educators did not
mention the traditional approach of interventions, identify a skill deficit and teaching/re-
teach to mastery. The approach of developing a concept at a deeper level, drawing on the
schema already accessible to students, enabling students to create new information and a
different understanding of the content. The luxury of time that is not available through
the school year as educators are trying to teach the standards at a prescribed pace that
may not match student learning pace.
Skill maintenance was the second subtheme. Several teachers specifically
mentioned the idea of refining student knowledge. Again, this notion builds from a place
of using students’ schema. The third concept of pre-teaching for the upcoming year is a
valued ELL instructional strategy applied to vocabulary instruction. In this case, the
teacher is building schema across the entire school year, not just in the isolated case of a
daily lesson. All three suggestions offer perspective that can be applied to future
interventions.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
135
Intervention goals and curriculum. The intervention goal, staunching summer
learning loss, was frequently tied to the curriculum. This came through the descriptions
of the activities the students were able to participate in and the ability to include project
based or thematic learning. This was hard to separate out from access to resources and the
calendar, but teachers frequently mentioned the goal of lessening summer learning
regression with the type of learning student did in the program. In reality, the pedagogy
was modeled after the district reading and writing workshop model. The only addition, in
effect, was the weekly field trip and the opportunity to carry the unit theme through all
the content area. Perception is reality, since almost all teachers interviewed, both teachers
that have taught in the intervention and those that have not been intervention instructors,
voiced the same perception about the curriculum of the intervention.
Transformative Question
The transformative aspect of the research was guided by the question: what are
the educational policy suggestions that can be derived from the results and the findings.
The responses of the teachers indicate a willingness to be proactive and advocate for ELL
students in the most need and these are the necessary agreements they advocated for.
First, the district’s responsiveness to implementing a disruptive intervention that breaks
with summer school tradition. Second, parents and students committed to participating in
a non-traditional calendar. Third, educators dedicated to student success by putting
student achievement above their own comfort. Last, an educational community
enthusiastically behind delivering transportation, food services, building maintenance,
and all other essential services necessary to run an educational enterprise. It requires risk
takers.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
136
The program showed evidence, both through the teacher perceptions of student
achievement outcomes and through the assessments data for students that participated in
the assessment of being successful in creating long-term growth for ELL students. This
intervention was started during a time when state, federal and local budgets were being
cut, but the intervention moved forward to produce results that demonstrated good
stewardship of fiscal resources and a true commitment to doing what is in the best
interest of students over the comfort of adults.
Implications for Action/Recommendations for Further Research
Implications for Action
There are several implications that can be made from the findings. The first is the
importance of identifying viable and sustainable solutions that can be implemented to
counteract summer learning loss in the English language learner population, a population
already starting with a language disadvantage. With an eye toward the literature and the
impact of summer learning regression on other vulnerable student populations, can be
used in the broader educational settings.
An implication for reading comprehension is that the intervention has the ability
to close the gap in reading and the earlier the intervention, the better for improving
student learning outcomes. Based on the study findings, the intervention had a lasting and
needed impact on the learning of a group of students that is expanding population in the
public education realm. In addition, as noted previously in the research of Heritage et al.,
(2015), this student population has a legacy of lagging native English speaking peers, so
it is important to identify programs that can bridge learning through the summer and
close the achievement gap.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
137
The implications of the English acquisition findings suggest that an extended
summer intervention assisted students with the process of English language acquisition.
Again, the indication is that early intervention accelerates learning and has a compounded
effect over time. The suggestion would be to provide an extended summer language
intervention to students with the most need, at the earliest age. This concept is supported
by the research of Entwisle and Alexander (1998) in that schools play a role in sustaining
and the cognitive development of students by providing early academic interventions to
students in the most need.
Recommendations for Further Research
The most immediate recommendation would be to continue data collection with
current cohorts of students and add data for the most recent round of assessments. Both
MAP and ACCESS have recent year-end data that was not included in the current
research. Additional data points for every cohort will further support/dispel trends that
have developed and substantiate trends that have limited data points. Along the same line,
implementing a similar intervention (seven weeks during the summer) with a larger
population sample, either by adding more students within the same district or in a district
with a large ELL student population would help with the attrition that occurred through
the high levels of transiency.
Another area of future research would be to include student and family
perceptions about the learning outcomes after participating in the intervention, along with
teachers’ perceptions and student assessment data. The third prong of data would add an
additional layer of support. As well, there would be a benefit to exploring families’
reasons for non-participation since there were no barriers for participation other than a
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
138
commitment of time. This information would provide useful insights that could be
adopted and used for planning purposes of other organizations that are implementing this
type of intervention.
Another area of research that could be pursued, but was not part of this study is
the role of curriculum in the success of students. This theme was a heavily referenced
throughout the teacher interviews.
Summary
The convergent transformative mixed methods study examined the confluence of
student data after participation in a seven-week summer intervention for English
language learners with the ELL teachers’ perceptions about students’ achievement. The
quantitative themes identified in the teacher interviews included: a perception the seven-
week summer intervention helped lessen summer learning regression through the
opportunity to continuously practice English; the seven-week intervention provided an
opportunity for students to have access to educational resources and learning
opportunities that student might not otherwise have opportunities to experience; a
perception that the extended summer gave students and teachers the time to delve more
deeply into the content of the school year and refine learning; a perception that the
curriculum with project based and theme based learning assisted in meeting the
intervention goals; and an overall impression that the intervention was effective in
increasing student achievement. The educators’ perceptions were borne out with the
student reading comprehension and English acquisition assessment findings. The findings
of the study revealed that the majority of the students that participated in the intervention
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
139
had higher trend lines than the control group and accelerated as the students’ progress
through their academic careers.
The recommendations for future study included continuing the study with the
current population of students to further substantiate trends. In addition, future study
should include students’ perceptions of their academic success after participation in the
intervention. From the transformative point of view, the results supported educators,
districts and policy makers implementing a quality summer learning situation that would
bridge the entire summer. The seven-week intervention fills a number of educational
needs of English language learner students and as a burgeoning student population,
educational policies need to proactively address this student population.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
140
References
Alexander, K. L., Entwisle, D. R., & Olson, L. S. (2007a). Lasting consequences of the
summer learning gap. American Sociological Review, 72, 167-180. doi:
http://dx.doi.org/10.1177/000312240707200202
Alexander, K. L., Entwisle, D. R., & Olson, L. S. (2007b). Summer learning and its
implications: Insights from the beginning school study. New Directions for Youth
Development, 107, 11-32. doi: 10.1002/yd
Allington, R. L. (2012). What really matters for struggling readers: Designing research-
based programs (3rd ed.). Boston, MA: Pearson Education, Inc.
Allington, R. L., McGill-Franzen, A., Camilli, G., Williams, L., Graff, J., Zeig, J.,
…Nowak, R. (2010). Addressing summer reading setback among economically
disadvantaged elementary students. Reading Psychology, 31(5), 411-427. doi:
10.1080/02702711.2010.505165
The Annie E. Casey Foundation. (2013). The first eight years: Giving kids a foundation
for lifetime success. Retrieved from http://www.aecf.org/resources/the-first-eight-
years-giving-kids-a-foundation-for-lifetime-success/
The Annie E. Casey Foundation. (2014). Early reading proficiency in the United States.
Retrieved from http://www.aecf.org/resources/early-reading-proficiency-in-the-
united-states/
Assink, E., van Well, S., & Knuijt, P. N. A. (2003). Age of acquisition effects in native
speakers and second-language learners. Memory and Cognition, 31(8), 1218-
1228. doi: 10.3758/BF03195805
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
141
August, D., & Hakuta, K. (Eds.). (1997). Improving schooling for language-
minority children: A research agenda. Washington, DC: National Research
Council Committee on Developing a Research Agenda on the Education of
Limited-English-Proficient and Bilingual Students, National Academy Press.
Becker, G. S., & Tomes, N. (1986). Human capital and the rise and fall of families.
Journal of Labor Economics, 4, 1-39. doi: http://dx.doi.org/10.1086/298118
Borman, G. D., Benson, J., & Overman, L. T. (2005). Families, schools and summer
learning. The Elementary School Journal, 106, 131-150. Retrieved from
http://www.jstor.org/stable/10.1086/499195
Bryant, M. T. (2004). The portable dissertation advisor. Thousand Oaks, CA: Corwin
Press.
Burkam, D. T., Ready, D. D., Lee, V. E., & LoGerfo, L. F. (2004). Social-class
differences in summer learning between kindergarten and first grade: Model
specification and estimation. Sociology of Education, 77, 1-31.
doi: 10.1177/003804070407700101
California Department of Education, (2014). Quality standards for expanded learning in
California: Creating and implementing a shared vision of quality. Retrieved
from http://www.afterschoolnetwork.org/sites/main/files/file-
attachments/quality_standards_for_expanded_learning_in_calfornia.pdf
California Department of Education. (2013). Year-round education program guide.
Retrieved from www.cde.ca.gov/ls/fa/yr/guide.asp
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
142
Carnegie Corporation. (1992). Risk and opportunity in the non-school hours: Report of
the taskforce on youth development and community programs. Retrieved
from http://files.eric.ed.gov/fulltext/ED355007.pdf
Cave, T., Ludwar, J., & Williams, W. (n.d.). Brain-based Learning. Retrieved from
http://education.alberta.ca/apps/aisi/literature/pdfs/bbased_learning.pdf
Chambers, T. V. (2009). The “receivement gap”: School tracking policies and the fallacy
of the “achievement gap”. The Journal of Negro Education, 78, 417-431.
Checkley, K. (2014). Setting ELLs up for success. Education Update, 56(10). Retrieved
from http://www.ascd.org/publications/newsletters/education-
update/oct14/vol56/num10/Setting-ELLs-Up-for-Success.aspx
Clauss-Ehlers, C. S. (Ed.). (2010). Encyclopedia of Cross-Cultural School Psychology.
York, NY: Springer Science+Business Media, LLC.
Cohen, G. L., Garcia, J., Apfel, N., & Master, A. (2006). Reducing the racial
achievement gap: A social-psychological intervention. Science, 313 (September
1), 1307-1310. doi:10.1126/science.1128317
¡Colorin Colorado!. (2007). Vocabulary development [Web post]. Retrieved from
http://www.colorincolorado.org/educators/teaching/vocabulary/
Coltrane, B., (2002). English language learners and high-stakes tests: An overview of the
issues. Center for Applied Linguistics, November 2002. Retrieved from
http://www.ericdigests.org/2003-4/high-stakes.html
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
143
Cooper, H., Nye, B., Charlton, K., Lindsay, J., & Greathouse, S., (1996). The effects of
summer vacation on achievement test scores: A narrative and meta-analytic
review. Review of Educational Research, 66, 227-268.
doi:10.3102/00346543066003227
Cornelius, P., & Semmel, M. I. (1982). Effects of summer instruction on reading
achievement regression of learning disabled students. Journal of Learning
Disabilities, 15, 409-413. Retrieved from https://login.cuhsl.creighton.edu/login?
url=http://search.ebscohost.com/login.aspx?direct=true&db=a2h&AN=4729671&
site=ehost-live
Creswell, (2012). Educational research: Planning, conducting, and evaluating
quantitative and qualitative research (4th ed.). Boston, MA: Pearson Education,
Inc.
Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods
research. Thousand Oaks, CA: SAGE Publications.
DaSilva Iddings, A. C. (2010) Teaching of English as a second language (TESOL). In C.
Clauss-Ehlers (Eds.), Encyclopedia of cross-cultural school psychology (pp. 66-
71). New York, NY: Springer Science+Business Media, LLC.
Duguay, A., Massoud, L., Tabuku, L., Himmel, J., & Sugarman, J. (2013). Implementing
the common core for English learners: Responses to common questions. Center
for Applied Linguistics, September. Retrieved from http://www.cal.org/resource-
center/briefs-digests/briefs/implementing-the-common-core-for-english-learners
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
144
The Education Trust. (2014, June 23). The state of education for Latino students.
Retrieved from http://www.edtrust.org/sites/edtrust.org/files/TheStateofEducation
forLatinoStudents_EdTrust_June2014.pdf
Entwisle, D. R., Alexander, K. L., & Olson, L. S. (n.d.). Keep the faucet flowing:
Summer learning and home environment. American Educator. Retrieved from
http://www.aft.org/periodical/american-educator/fall-2001/keep-faucet-flowing
Entwisle, D.R., Alexander, K. L., & Olson, L. S. (2005). First grade and educational
attainment by age 22: A new story. American Journal of Sociology, 110, 1458-
1502. doi:10.1086/428444
Entwisle, D. R., Alexander, K. L. (1998). Facilitating the transition to first grade: The
nature of transition and research on factors affecting it. The Elementary School
Journal, 98, 351-364. Retrieved from http://www.jstor.org/stable/1002192
Entwisle, D. R., Alexander, K. L., & Olson, L. S. (2001). Keep the faucet flowing.
American Educator. Retrieved from http://www.aft.org/newspubs/periodicals/
ae/fall2001/entwisle.cfm
Flores, S. M., Batalova, J., & Fix, M. (2012). The educational trajectories of English
language learners in Texas. Migration Policy Institute. Retrieved from
http://www.migrationpolicy.org/research/educational-trajectories-english-
language-learners-texas
Fountas, I. C., & Pinnell, G. S. (2007). The continuum of literacy learning grades K-2: A
guide to teaching. Portsmouth, MH: Heinemann.
Gallagher, K. (2009). Readicide: How schools are killing reading and what you can do
about it. Portland, ME: Stenhouse Publishers.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
145
Gerring, J. (2012). Social science methodology: A unified framework (2nd ed.). New
York, NY: Cambridge University Press.
Graves, J. (2010). Effects of year-round schooling on disadvantaged students and the
distribution of standardized test performance. Economics of Education Review,
30(2011), 1281-1305. Retrieved from http://www.sciencedirect.com/science/
article/pii/S0272775711000586
Halle, T., Hair, E., Wander, L., McNamara, M.., & Chien, N. (2012). Predictors and
outcomes of early versus later English language proficiency among English
language learners. Early Childhood Research Quarterly, 27(1), 1-20.
doi.org.cuhsl.creighton.edu /10.1016/j.ecresq.2011.07.004
Hardin, B. J., Sandstrom, H., & Chazan-Cohen, R. (2012). Early head start and African
American families: Impacts and mechanisms of child outcomes. Early Childhood
Research Quarterly, 27(1), 572-581. doi.org/10.1016/j.ecresq.2012.07.006
Harris, E., & Wallace, A. (2012). Year round learning: Continuity in education, across
settings and time through expanded learning opportunities (Brief No. 3).
Retrieved from Harvard Family Research Project, Harvard Graduate School of
Education website: http://www.hfrp.org/publications-resources/browse-our-
publications/year-round-learning-continuity-in-education-across-settings-and-
time-through-expanded-learning-opportunities
Hattie, J. (2011). Visible learning for teachers: Maximizing impact for learning. [Nook
Reader version]. Retrieved from barnesandnoble.com
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
146
Hernandez, A. (2010). Academic achievement in minority children. In C. Clauss-Ehlers
(Eds.), Encyclopedia of cross-cultural school psychology (pp. 66-71). New York,
NY: Springer Science+Business Media, LLC.
Heritage, M., Walqui, A., & Linquanti, R. (2015). English language learners and the new
standards: Developing language, content knowledge, and analytical practices in
the classroom. Cambridge, MA: Harvard Education Press.
Herr, K., & Anderson, G. L. (2015). The action research dissertation: A guide for
students and faculty (2nd ed.). Thousand Oaks, CA: Sage Publications.
Heyns, B. (1978). Summer learning and the effects of schooling. New York, Academic
Press, Inc.
Heyns, B. (1987). Schooling and cognitive development: Is there a season for learning.
Child Development 58(5), 1151-1160. doi: 10.2307/1130611
Hobbs, F., & Stoops, N. (2002, November). Demographic trends in the 20th century:
Census 2000 special reports. United States Department of Commerce: Economics
and Statistical Administration: United States Census Bureau. Retrieved from
http://www.census.gov/prod/2002pubs/censr-4.pdf
Honigsfeld, A., & Dove, M. (2008). Co-teaching in the ESL classroom. Retrieved from
http://coteachingforells.weebly.com/uploads/8/0/6/6/8066516/2008
_dkg_article_co-teaching_copy.pdf
Howard, G. A., Banks, J. A., & Nieto, S. (2006). We can't teach what we don't know:
White teachers, multiracial schools. New York, NY: Teachers College Press.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
147
Jacob, S. A., & Furgerson, P. S. (2012). Writing interview protocols and conducting
interviews: Tips for students new to the field of qualitative research. The
Qualitative Report, 17, 1 -10. Retrieved from http://files.eric.ed.gov/fulltext/
EJ990034.pdf
Jesson, R., McNaughton, S., & Kolose, T. (2014). Investigating the summer learning
effect on low SES schools. Australian Journal of Language and Literacy, 37,
45-54.
Johnson, S. P., & Spradlin, T.E. (2007). Alternatives to the traditional school-year
calendar. Center for Evaluation and Education Policy, 5(3), 1–11. Retrieved from
http://ceep.indiana.edu/projects/PDF/PB_V5N3_Spring_2007_EPB.pdf
Kelly, P. R., Gomez-Bellange, F., Chen, J., & Schulz, M. M. (2008). Learner outcomes
for English Language Learner low readers in an early intervention. Teachers of
English to Speakers of Other Languages (TESOL) Journal, 42, 235-260.
Retrieved from http://www.jstor.org/stable/40264449
Kieffer, M. J. (2012). Early oral language and later reading development in Spanish-
speaking English language learners: Evidence from a nine-year longitudinal
study. Journal of Applied Developmental Psychology, 33, 146-157. doi:
10.1016/j.appdev.2012.02.003
Kim, Y., McLellan, M., & Asbell, M. E. (2010). English as a second language instruction
(ESL). In C. Clauss-Ehlers (Eds.), Encyclopedia of cross-cultural school
psychology (pp. 424-427). New York, NY: Springer Science+Business Media,
LLC.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
148
Kohn, A. (2012, July 20). Lowering the temperature on claims of summer learning loss:
Why are we so nervous about giving kids a break from school? Psychology
Today. Retrieved from http://www.psychologytoday.com/blog/the-homework-
myth/201207/lowering-the-temperature-claims-summer-learning-loss
Learning in Afterschool and Summer. (2014). Our position statement [Website].
Retrieved from http://www.learninginafterschool.org/position.htm
Lukens, L., Paden, L., Yocum, K. (2013). [ELL summer program: 2011, 2012, & 2013].
Unpublished raw data.
Marinez-Lora, A. M., & Quintana, S. M. (2010). Summer learning loss. In C. Clauss-
Ehlers (Eds.), Encyclopedia of cross-cultural school psychology (pp. 427–428).
New York, NY: Springer Science+Business Media, LLC.
McCombs, J. S., Augustine, C. H., Schwartz, H. L., Bodilly, S. J., McInnis, B., Lichter,
D. S., Cross, A. B. (2011). Making summer count: How summer programs can
boost children’s learning. The Rand Corporation. Retrieved from
http://www.rand.org/content/dam/rand/pubs/monographs/2011
/RAND_MG1120.pdf
McLaughlin, B., & Smink, J. (2010, May). Why summer learning deserves a front-row in
the education reform arena. The Johns Hopkins University New Horizons for
Learning. Retrieved from http://education.jhu.edu/PD/newhorizons/Journals/
spring2010/why-summer-learning/
McMullin, B.J. (2001). A statewide evaluation of academic achievement in year-round
schools. The Journal of Educational Research, 95, 67-74. Retrieved from
http://www.jstor.org.cuhsl.creighton.edu/stable/27542361
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
149
Miller, B. M. (2007). The learning season: The untapped power of summer to advance
student achievement: Executive summary. The Nellie Mae Education Foundation.
Retrieved from http://www.nmefoundation.org/getmedia/17ce8652-b952-4706-
851b-bf8458cec62e/Learning-Season-ES?ext=.pdf
Missouri Department of Elementary and Secondary Education. (n.d.). Grade level
assessment. Retrieved from http://dese.mo.gov/college-career-readiness/
assessment/grade-level
Missouri Department of Elementary and Secondary Education. (n.d.). Raw data: 2014
district level MAP and End–of–Course (EOC) [Data file]. Retrieved from
http://mcds.dese.mo.gov/quickfacts/Pages/State-
Assessment.aspxMissouriSchoolImprovementPlan5 .
Missouri Department of Elementary and Secondary Education. (2011). Comprehensive
guide to Missouri School Improvement Plan. Retrieved from
http://dese.mo.gov/qs/documents/MSIP-5-comprehensive-guide-3-13.pdf
Muijs, Daniel. (2004). Doing quantitative research in education with SPSS. Thousand
Oaks, CA: Sage Publications.
National Early Literacy Panel. (2008). Developing early literacy: Report of the national
early literacy panel. Washington, DC: National Institute of Literacy. Retrieved
from http://lincs.ed.gov/publications/pdf/NELPReport09.pdf
Nisbett, R. E. (2011). The achievement gap: Past, present and future. Deadalus, The
Journal of the American Academy of Arts and Sciences, 140, 90-100.
doi:10.1162/DAED_a_00079
No Child Left Behind Act of 2001. Pub. L. No 107-110, § 101, 115 Stat. 1425 (2002).
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
150
New York Collective of Radical Teachers. (n.d.). A guide for educators of English
language learners. Retrieved from http://www.nycore.org/newsite/wp-
content/uploads/NYCORE-ELLteacher_handbook.pdf
O’Brien, M. (2014, October 18). Poor kids who do everything right don’t do better than
rich kids who do everything wrong. The Washington Post. Retrieved from
http://www.washingtonpost.com/blogs/wonkblog/wp/2014/10/18/poor-kids-who-
do-everything-right-dont-do-better-than-rich-kids-who-do-everything-wrong/
Olson, K. R., & Dweck, C. S. (2008). A blueprint for social cognitive development.
Association for Psychological Science, 3, 193 – 202. doi: 10.1111/j.1745-
6924.2008.00074.x
Patton, K. L., & Reschly, A. L. (2013). Using curriculum-based measurement to examine
summer learning loss. Psychology in the Schools, 50, 738–753. doi: 10.1002/pits
Perez, B., Harris, B., Martinez, R. S., & Ridley, C. R. (2010). Culturally competent
assessment of English language learners. In C. Clauss-Ehlers (Eds.),
Encyclopedia of cross-cultural school psychology (pp. 336-338). New York, NY:
Springer Science+Business Media, LLC.
Ponterotto, J. G., Mathew, J. T., & Raughley, B. (2013). The value of mixed methods
designs to social justice research in counseling and psychology. Journal for Social
Action in Counseling and Psychology, 3, 42–68. Retrieved from
http://www.psysr.org/jsacp/Ponterotto-V5N2-13_42-68.pdf
Roberts, C. M. (2010). The dissertation journey: A practical and comprehensive guide to
planning, writing, and defending your dissertation. Thousand Oaks, CA: Corwin.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
151
Rojas, R., & Iglesias, A. (2013). The language growth of Spanish-speaking English
language learners. Child Development, 84(2), 630-646. doi:10.1111/j.1467-
8624.2012.01871.x
Rumsey, D. (n.d.). How to interpret a correlation coefficient r. Retrieved from
http://www.dummies.com/how-to/content/how-to-interpret-a-correlation-
coefficient-r.html
Snow, C. E., Burns, M. S., & Griffin, P. (Eds.). (1998). Preventing reading difficulties in
young children. Washington, DC: National Research Council, National Academy
Press. Retrieved from http://www.nap.edu/readingroom/books/reading/
Solano-Flores, G. (2010). English Language Learners. In C. Clauss-Ehlers (Eds.),
Encyclopedia of cross-cultural school psychology (pp. 427–428). New York, NY:
Springer Science+Business Media, LLC.
Stanat, P., Becker, M., Baumert, J., Ludtke, O., & Eckhardt, A. G. (2012). Improving
second language skills of immigrant students: A field trial study evaluating the
effects of a summer learning program. Learning and Instruction, 22, 159-170.
Retrieved from http://dx.doi.org.cuhsl.creighton.edu/10.1016
/j.learninstruc.2011.10.002
Sweetman, D., Badiee, M., & Creswell, J. W. (2010). Use of the transformative
framework in mixed methods studies. Qualitative Inquiry, 16, 441-454. doi:
10.1177/1077800410364610
Suzuki, L. A., Ngo, L., & Kuger, J. (2010). Assessment of culturally diverse children. In
C. Clauss-Ehlers (Eds.), Encyclopedia of cross-cultural school psychology
(pp. 124-131). New York, NY: Springer Science+Business Media, LLC.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
152
United States Department of Education. (2012, April 26). Early reading first. Retrieved
from http://www2.ed.gov/programs/earlyreading/index.html
United States Department of Education, National Center for Educational Statistics.
(2009). National assessment of educational progress: The nation’s report card:
Trends in academic progress in reading and mathematics 2008. Retrieved from
http://nces.ed.gov/nationsreportcard/pubs/main2008/2009479.asp#section2
United States Department of Education, National Center for Educational Statistics.
(2013). Fast facts: English language learners. National Center for Educational
Statistics. Retrieved from http://nces.ed.gov/fastfacts/display.asp?id=96
United States Department of Education, National Center for Educational Statistics.
(2014). English language learners. Retrieved from http://nces.ed.gov/programs
/coe/indicator_cgf.asp
United States Department of Education, National Center for Educational Statistics.
(2014). The condition of education: Children living in poverty. Retrieved from
http://nces.ed.gov/programs/coe/indicator_cce.asp
Wiggins, G. (2012). What works in education: Hattie’s list of the greatest effects and why
it matters [Blog post]. January 7, 2012. Retrieved from https://grantwiggins.
wordpress.com/2012/01/07/what-works-in-education-hatties-list-of-the-greatest-
effects-and-why-it-matters/
Wilson, S. J., Dickinson, D. K., Rowe, D. W. (2013). Impact of an early reading first
program on the language and literacy achievement of children from diverse
language backgrounds. Early Childhood Research Journal, 28, 578-592. doi:
http://dx.doi.org/10.1016/j.ecresq.2013.03.006
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
153
World-Class Instruction and Design. (2014). ACCESS for ELLs summative assessment.
Retrieved from https://www.wida.us/assessment/ACCESS/
World-Class Instruction and Design. (2014). Mission and the WIDA story. Retrieved
from https://www.wida.us/aboutUs/mission.aspx
World Class Instruction and Design. (n.d.). The relationship among WIDA’s strands of
model performance indicators, ELP standards, CAN DO descriptors and
performance definitions (n.d.). Retrieved from
https://www.wida.us/standards/RG_Performance%20Definitions.pdf
Young, S. (2007). Effects of instructional hours and intensity of instruction on NRS level
gain in listening and speaking. CAL Digest, December. Retrieved from
http://www.cal.org/resource-center/briefs-digests/digests/(offset)/30
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
154
Appendix A
Email/phone Invitation to Participate
Dear Potential Study Participant, My name is Kent Yocum. I am a doctoral student candidate in the Interdisciplinary Leadership program at Creighton University. I am exploring teachers’ perceptions of the academic success of English Language Learner students that have participated in a seven-week summer intervention. Your experience working with these students in conjunction with your perceptions of student performance after participation in the intervention will provide valuable insight. Participation in this anonymous study is voluntary. Phase one includes a one-on-one interview conducted by the researcher and last approximately one hour. Phase two includes a focus group to further develop themes identified during the one-on-one interviews. All responses will be confidential. Participants will have the opportunity to review transcripts of responses to verify accuracy If you are interested in participating in this important study, through one-on-one interviews, focus study group or both phases, please respond to this email to schedule a time. Thank you in advance for your participation. Respectfully, R. Kent Yocum Doctoral Candidate Creighton University RYocum@Creighton.edu 816-960-7035
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
155
Appendix B
Interview Question Protocol One-on-one interview script
1. What is your relationship with the seven-week summer intervention for ELL students?
2. Describe what you believe are the purpose and/or goals of the seven-week
summer intervention.
3. Describe your perception of how the seven-week summer intervention meets the program goals.
4. What is your perception of students’ academic outcomes on English language
acquisition after participation in the intervention?
5. Describe your perception of students’ academic outcomes on reading comprehension assessments after participation in the intervention?
6. Describe your perception of students’ academic outcomes on state
standardized assessments after participation in the intervention?
7. What is your impression of the length of time of the intervention and the impact of student academic outcomes?
8. What changes would you recommend to improve student academic
outcomes?
9. What additional information would you like to add about the seven-week intervention and your perceptions of student academic outcomes?
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
156
Appendix C
Letter of Agreement
Dr. J Michael Pragman Director of Research, Evaluation, and Accountability
North Kansas City Schools Email:
Michael.Pragman@nkcschools.org
Letter of Agreement February 11, 2015
To the Creighton University IRB: We are familiar with R. Kent Yocum’s research project entitled English Language Learner (ELL) Achievement: A Mixed-Methods Examination of a Seven-Week Summer Intervention. I understand North Kansas City Schools’ involvement will be…
• to allow the researcher access to the archived academic data of kindergarten through fifth grade English Language Learners that participated in a seven week summer intervention. The archived data consists of reading comprehension benchmarks (Fountas and Pinnell Benchmark Assessment System), English Language acquisition annual benchmarks (WIDA ACCESS) and Missouri Assessment Program (MAP) assessment annual results from the time period starting Spring 2011 through Winter 2015. MAP scores for students that have transitioned to middle school will also be part of the data collected.
• to allow researcher the opportunity to conduct one-to-one interviews with
five to seven English Language Learner (ELL) instructional staff members. Participation will be voluntary.
• to allow the researcher the opportunity to conduct a voluntary focus group
comprised of the elementary English Language Learner teachers. Questions for the focus group will be facilitated by a neutral, third-party not part of the research team or employee of the district. Questions will be determined through responses from one-on-one interviews.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
157
We understand that this research will be conducted following sound ethical principles, that participant involvement in this research study is strictly voluntary, and that confidentiality of participants’ research data is ensured, as described in the protocol.
In compliance with Department of Education regulation, the North Kansas City School District Board of Education is committed to maintaining a workplace and educational environment that is free from discrimination and harassment in admission or access to, or treatment or employment in, its programs, services, activities and facilities. In accordance with law, the district strictly prohibits discrimination and harassment against employees, students or others on the basis of race, color, religion, sex, national origin, ancestry, disability, age, genetic information or any other characteristic protected by law. The North Kansas City School District is an equal opportunity employer.
Therefore, as a representative of North Kansas City Schools, I agree that R. Kent Yocum’s research project may be conducted at our agency, pending final IRB approval. Respectfully Submitted, Dr. J. Michael Pragman Dr. J Michael Pragman Director of Research, Evaluation, and Accountability Dept. of Academic Services & School Accountability North Kansas City Schools 2000 NE 46th Street Kansas City, MO 64116 Office 816-413-5141 Fax 816-413-5095 Cell 816-719-3138 Email Michael.Pragman@nkcschools.org
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
158
Appendix D
Participant Informed Consent
Project Title: English Language Learner Achievement: A Mixed-Methods
Examination of a Seven-Week Summer Intervention
Purpose of the Study: I am conducting a research study on the academic
achievement of students that have participated in a seven-week summer
English Language intervention for the purpose of improving educational
outcomes of ELL students and to better understand the academic
achievement of students that have participated in the intervention. The
results of the research will inform educational policy for underserved
student populations.
Methods and Procedures: I am asking you to participate in a one-on-one
interview and/or focus group about your perceptions of English Language
Learner student learning outcomes after participating in the seven-week
summer intervention.
Time to Participate: The interview will take approximately 1-hour at a location
that is convenient and comfortable for you. I will audio record the interview
to ensure accuracy and provide you will a transcript for approval. I will also
take some hand written notes. The focus group will take approximately 1-
hour at (location to be determined, such as meeting room at the public
library). The focus group will be audio-recorded and facilitated by a neutral,
third party.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
159
Risks and Discomforts: There are no know risks or discomforts associated with
participation and participant identities will not be revealed. Transcripts will
be provided for your approval of accuracy and provide an opportunity to
remove any personal identifiers to ensure confidentiality.
Benefits: Information gathers will be used to inform educational policy decisions of
students learning English and create the best possible learning outcomes.
Confidentiality: Identities of participants will not be revealed. All responses will be
coded and tracked using pseudonyms. Records will be stored in a secured
electronic and/or physical location that is only accessible by the researcher.
Results will be reported in a manner to provide participants with anonymity.
Compensation: Participants will receive no monetary compensation.
Opportunity to Ask Questions: Questions or concerns about participation and
your rights can be directed to the Institutional Review Board of Creighton
University.
Freedom to Withdraw: Participation is optional and voluntary. You have the right
to withdraw your consent to participate in the research interview at any
point during the interview or focus group without adverse personal effect.
Consent: Your signature signifies consent to be interviewed and acknowledges you
have read and understand the information presented above. A copy of the
consent form will be provided for your records.
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
160
Appendix E
IRB Exempt Letter
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
161
Appendix F
Additional 2011 Cohort Graphs – ACCESS
00.5
11.5
22.5
33.5
44.5
55.5
66.5
2010 2011 2012 2013 2014
Aver
age
Scor
e
Year
Cohort 1: Kindergarten Access
No Participation Participation
Linear (No Participation) Linear (Participation)
00.5
11.5
22.5
33.5
44.5
55.5
66.5
2010 2011 2012 2013 2014
Aver
age
Scor
e
Year
Cohort 1: First Grade Access
No Participation Participation
Linear (No Participation) Linear (Participation)
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
162
00.5
11.5
22.5
33.5
44.5
55.5
66.5
2010 2011 2012 2013 2014
Aver
age
Scor
e
Year
Cohort 1: Second Grade Access
No Participation Participation
Linear (No Participation) Linear (Participation)
00.5
11.5
22.5
33.5
44.5
55.5
66.5
2010 2011 2012 2013 2014
Aver
age
Scor
e
Year
Cohort 1: Third Grade Access
No Participation Participation
Linear (No Participation) Linear (Participation)
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
163
00.5
11.5
22.5
33.5
44.5
55.5
66.5
2010 2011 2012 2013 2014
Aver
age
Scor
e
Year
Cohort 1: Fourth Grade Access
No Participation Participation
Linear (No Participation) Linear (Participation)
00.5
11.5
22.5
33.5
44.5
55.5
66.5
2010 2011 2012 2013 2014
Aver
age
Scor
e
Year
Cohort 1: Fifth Grade Access
No Participation Participation
Linear (No Participation) Linear (Participation)
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
164
Appendix G
Additional 2011 Cohort Graphs - MAP
560570580590600610620630640650660
2010 2011 2012 2013 2014
Aver
age
Scor
e
Year
Cohort 1: First Grade MAP
No Participation ParticipationLinear (No Participation) Linear (Participation)
570580590600610620630640650660670
2010 2011 2012 2013 2014
Aver
age
Scor
e
Year
Cohort 1: Second Grade MAP
No Participation Participation
Linear (No Participation) Linear (Participation)
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
165
560570580590600610620630640650660
2010 2011 2012 2013 2014
Aver
age
Scor
e
Year
Cohort 1: Third Grade MAP
No Participation Participation
Linear (No Participation) Linear (Participation)
610620630640650660670680690700
2010 2011 2012 2013 2014
Aver
age
Scor
e
Year
Cohort 1: Fourth Grade MAP
No Participation Participation
Linear (No Participation) Linear (Participation)
ENGLISH LANGUAGE LEARNER ACHIEVEMENT: A MIXED-METHODS EXAMINATION OF A SEVEN-WEEK SUMMER INTERVENTION
166
600
610
620
630
640
650
660
2010 2011 2012 2013 2014
Aver
age
Scor
e
Year
Cohort 1: Fifth Grade MAP
No Participation Participation
Linear (No Participation) Linear (Participation)
Recommended