Embed Size (px)
Citation preview
This article was downloaded by: [Northeastern University]On: 10 November 2014, At: 15:24Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Advances in School Mental HealthPromotionPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/rasm20
Three-year longitudinal study of schoolbehavior and academic outcomes:results from a comprehensiveexpanded school mental healthprogramBrian P. Dalya, Mark A. Sanderb, Elizabeth G. Nichollsa, AmanuelMedhaniec, Eric Vanden Berkc & James Johnsonc
a Department of Psychology, Drexel University, Philadelphia, PA,USAb Hennepin County/Minneapolis Public Schools, Minneapolis, MN,USAc Minneapolis Public Schools, Minneapolis, MN, USAPublished online: 19 Dec 2013.
To cite this article: Brian P. Daly, Mark A. Sander, Elizabeth G. Nicholls, Amanuel Medhanie,Eric Vanden Berk & James Johnson (2014) Three-year longitudinal study of school behavior andacademic outcomes: results from a comprehensive expanded school mental health program,Advances in School Mental Health Promotion, 7:1, 24-41, DOI: 10.1080/1754730X.2013.867712
To link to this article: http://dx.doi.org/10.1080/1754730X.2013.867712
PLEASE SCROLL DOWN FOR ARTICLE
Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoeveror howsoever caused arising directly or indirectly in connection with, in relation to orarising out of the use of the Content.
This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
15:
24 1
0 N
ovem
ber
2014
Advances in School Mental Health Promotion, 2014 Vol. 7, No. 1, 24–41, http://dx.doi.org/10.1080/1754730X.2013.867712
Three-year longitudinal study of school behavior and academic outcomes: results from a comprehensive expanded school mental
health program
Brian P. Dalya*, Mark A. Sanderb, Elizabeth G. Nichollsa, Amanuel Medhaniec,Eric Vanden Berkc and James Johnsonc
a bDepartment of Psychology, Drexel University, Philadelphia, PA, USA; Hennepin County/Minneapolis Public Schools, Minneapolis, MN, USA; cMinneapolis Public Schools,
Minneapolis, MN, USA
(Received 18 March 2013; accepted 15 November 2013)
While there has been encouraging growth in the number of expanded school mental health programs (ESMH) across the country, few programs rigorously evaluate long-term academic outcomes associated with receipt of these services. This study examined the effects of services from an ESMH program on school behavior (number of out-ofschool suspensions and attendance rates), and academic outcomes (standardized test scores in reading and math). Participants were 89 students from Kindergarten to 8th grade who received ESMH services and 89 students from a matched comparison group. Results revealed that ESMH services (i.e., treatment) did not have a statistically significant association with any of the school behavior or academic outcome variables. Findings are discussed in the context of theoretical and methodological challenges associated with program evaluations of ESMH. Implications for practitioners and researchers regarding program evaluation and response to stakeholders are addressed.
Keywords: expanded school mental health; program evaluation; longitudinal; academic outcomes
Introduction
Expanded school mental health (ESMH) programs are characterized by collaborative partnerships between schools and mental health community organizations in which the goal is to provide a broad range of therapeutic services and learning support to students in general and special education in the school setting (Flaherty & Osher, 2003;Weist, 1997;Weist, Evans, & Lever, 2003). ESMH programs emphasize and utilize collaborative and interprofessional relationships to enhance access to youth mental health care (Juszczak, Melinkovich, & Kaplan, 2003; Weist, Meyers, Hastings, Ghuman, & Ham, 1999), and ameliorate gaps in service provision while also improving existing services (Flaherty & Weist, 1999; Weist & Albus, 2004). As such, comprehensive ESMH programs focus on delivering effective mental health prevention, promotion, and treatment services (Elias, Gager, & Leon, 1997; Weare, 2000), as well as augmenting, integrating, and coordinating with existing school-based supportive services (e.g., nursing, school psychology, school counseling, and social work) to provide the most comprehensive level of care for students and families (Weist, 1997).
Findings from program evaluations of school mental health services reveal multiple positive outcomes including improvements in student emotional and behavioral functioning
*Corresponding author. Email: [email protected]
q 2013 The Clifford Beers Foundation
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
15:
24 1
0 N
ovem
ber
2014
25 Advances in School Mental Health Promotion
(Armbruster & Lichtman, 1999; Hussey & Guo, 2003; Nabors & Reynolds, 2000) and aspects of the school climate (Bruns, Walrath, Siegel, & Weist, 2004). These program evaluations also demonstrated reductions in stigma for help-seeking behavior (Nabors & Reynolds, 2000) and declines in inappropriate referrals to special education services for students that accessed school mental health services (Bruns, Walrath, Siegel, & Weist, 2004).
In addition to improving student outcomes in the area of social-emotional development, one central goal of ESMH is to fully integrate the mental health agenda with the educational agenda to also improve student academic outcomes (Paternite, 2005; Stephan, Brandt, Lever, Acosta-Price, & Connors, 2012). The rationale for integrating these unique agendas is that attention and support for mental health needs are considered necessary for children to achieve their full potential in multiple areas of functioning, including education (Owens & Murphy, 2004). This contention is primarily supported by two lines of evidence that indicate learning is significantly compromised by emotional and behavioral health problems, and that there is a positive association between mental health and academic success (Bishop et al., 2004; Catalano, Haggerty, Oesterle, Fleming, & Hawkins, 2004; Klern & Connell, 2004; Zins, Bloodworth, Weissberg, & Walberg, 2004). Although the alignment between the mental health and education agendas seems a natural fit, the true integration of these systems has proven difficult, in part due to long-standing differences in established priorities between the schools and school mental health programs (Atkins, Hoagwood, Kutash, & Seidman, 2010). For instance, schools have often been singularly focused on improving student learning, while the traditional core function for school mental health programs has been to engage in efforts that enhance the social-emotional adjustment of students.
However, national education reform efforts during the last two decades that emphasize academic accountability and more defined links between school systems and mental health systems (e.g., Race to the Top, No Child Left Behind, President’s New Freedom Commission on Mental Health; Atkins et al., 2010) mean that priorities between the systems are becoming more integrated. In line with these reform efforts, many program funders, school administrators, school districts, and other stakeholders now mandate that ESMH programs systematically evaluate not only psychosocial outcomes, but also academic and educational outcomes for youth who receive behavioral health services in school (Daly et al., 2006). Thus, while the primary targets of ESMH programs are to increase access to and usage of mental health treatment services and improve student social and emotional outcomes, there also is an expectation of positive program impact on academic and educational outcomes such as attendance, disciplinary violations, and standardized test scores (Kutash, Duchnowski, & Lynn, 2006; Weist & Christodulu, 2000). However, although ESMH programs are already present in many school settings and continue to grow across the country (Weist, Lowie, Flaherty, & Pruitt, 2001), relatively few ESMH programs have systematically investigated the effectiveness of their own services on educational outcomes (Hoagwood & Erwin, 1997). Instead, much of the published research associated with ESMH has focused on the evaluation of specific evidence-based prevention or intervention programs, typically with younger students.
For instance, a critical review of empirically based studies of school mental health interventions found that 38% of the programs reviewed (n ¼ 24/64) examined both mental health and educational outcomes (Hoagwood et al., 2007). Importantly, of this sample, only 15 of the 24 studies (62%) found a positive outcome for both mental health and educational outcomes, and impacts were modest and generally not sustained over time (Hoagwood et al., 2007). On the other hand, a recent meta-analysis of school-based,
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
15:
24 1
0 N
ovem
ber
2014
26 B.P. Daly et al.
universal social, and emotional learning programs reported gains for participants in social and emotional skills, grades, and standardized test scores (11 percentile points) as compared to control group participants (Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011). It is notable that the Hoagwood et al. (2007) review included empirically based programs which were predominantly preventive in design, while the Durlak et al. (2011) analysis only included studies that targeted students without identified adjustment or learning problems.
Although universal prevention strategies are an important component of the ESMH framework (Weist & Christodulu, 2000), many ESMH providers spend the majority of their time delivering early intervention and treatment services to students who are already demonstrating mental health, behavioral, and/or academic concerns (Foster et al., 2005). In addition, students referred to ESMH programs often present with multiple comorbidities and different levels of severity, characteristics that are often in contrast to the sample of students included in prevention studies. Moreover, the ability to use highly structured, manualized, evidence-based approaches delivered under tightly controlled conditions is not always feasible for ESMH practitioners, given the realities and competing demands faced in the educational setting (Wilson, 2007). Thus, while studies such as those by Hoagwood et al. (2007) and Durlak et al. (2011) make important contributions to the research base for evidence-based school mental health, the utility of these findings in terms of comprehensive ESMH programs and their everyday ‘real-world’ practice is limited. As such, there is a compelling need to expand research efforts to include program evaluations that examine educational outcomes associated with ESMH everyday clinical practice conditions (Weist, Nabors, Myers, & Armbruster, 2000).
There have been some published studies and technical reports on programs that used elements of ESMH and examined educational and academic outcomes. For instance, Jennings, Pearson, and Harris (2000) examined the academic impact of the Dallas Public Schools Youth and Family Centers and reported a 31% decrease in course failures among those children who accessed services in the school-based clinics. Additional findings from this program, published in a technical report, revealed significant decreases in discipline referrals and improvements in rates of attendance (Hall, 2000). Analysis of schools participating in the Children First Plan program revealed lower suspension and truancy rates in the majority of schools participating in the program as compared to schools not utilizing school-based mental health initiatives (Children First Plan, 2002). Finally, in the 1998–1999 school year, grade point average (GPA) improved from 1.8 to 2.1 among students who received four or more mental health services from the University of Maryland’s Mental Health Program (University of Maryland, 1999).
Several recent studies have utilized more methodologically rigorous program evaluations that employed propensity score matching to evaluate program impact. The first study evaluated a comprehensive ESMH program in Baltimore and sought to link ESMH service use data to academic outcomes and school functioning (Anthony & Sebian, 2011). The study sample included children in first grade to high school. Results for attendance were generally positive, with those children who received ESMH services demonstrating an increase in attendance as compared to a decline in the matched comparison group. Findings related to number of suspensions were more nuanced. For instance, students receiving ESMH services were at higher risk for suspensions relative to other students. However, when analyses included matched comparison groups, children who accessed ESMH services generally had a higher number of suspensions across most grade levels. In terms of standardized test scores,
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
15:
24 1
0 N
ovem
ber
2014
27 Advances in School Mental Health Promotion
both the ESMH and matched comparison groups demonstrated significant decreases in Maryland State Assessment (MSA) reading and math scores. Notably, the pattern of decline did not significantly differ between the groups (Anthony & Sebian, 2011). In another study, Walker, Kerns, Lyon, Bruns, and Cosgrove (2010) employed a well-controlled longitudinal model to evaluate the impact of School-Based Health Centers (SBHC) on academic outcomes for high school students. Findings revealed improvements in GPA over time for students who accessed the SBHC for mental health services as compared to students who did not access the SBHC. Additional results revealed no significant program impact on rates of attendance or number of disciplinary events.
With the exception of these studies, most of the program evaluations on comprehensive ESMH programs contain notable methodological shortcomings. First, the research efforts largely focus on single time points rather than longitudinal impact of treatment. Second, the majority of the studies use school-level analyses to evaluate the effect of these programs instead of comparing treatment group participants to non-treatment students matched on similar demographic characteristics. Third, the investigations usually do not examine other important educational outcomes, such as standardized reading and math test scores. In a continuing effort to expand research on comprehensive ESMH program evaluations, this study seeks to make a contribution to the literature by exploring the longitudinal impact of a comprehensive Minneapolis ESMH program on school behaviors and academic achievement.
Study aims
The purpose of this study was to evaluate the effects of ESMH services on school behavior and academic outcomes. The research question investigated in this study was the following: Is receiving ESMH services associated with improvement in school behavior (reduced number of suspensions and improved attendance rates) and academic outcomes (improved standardized test scores in Math and Reading)?
With regard to school behavior, it is hypothesized that ESMH services will be associated with a decrease in out of school suspensions and an increase in attendance. The reason for this hypothesis is that ESMH services are intended to address pertinent emotional and/or behavioral issues (e.g., depression, anxiety, disruptive behavior disorders, and anger management) that impede a student’s ability or desire to attend school. In addition, ESMH services seek to address and ameliorate disruptive behaviors that frequently result in suspension.
Unlike school behavior, it is hypothesized that receiving EMSH services will have little to no effect on academic outcomes as measured by MAP reading and MAP math scores. The reason for this hypothesis is that standardized testing assesses cumulative knowledge acquired over many years and therefore does not provide a relevant proximal measure of ESMH impact that occurred during a select period of service delivery (Anthony & Sebian, 2011).
Methods
Description of the Minneapolis Public Schools (MPS) comprehensive ESMH program
Planning for the Minneapolis ESMH program began in 2004, and the program formally began in winter 2005, partially funded through a federal Safe Schools Healthy Students grant. The program was originally implemented in five K-5 and K-8 schools with collaborations with Watercourse Counseling and Washburn Center for Children.
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
15:
24 1
0 N
ovem
ber
2014
28 B.P. Daly et al.
Currently, the ESMH program serves 25 schools with collaborations occurring across four agencies, including the Minneapolis City Health Department, Washburn Center for Children, Watercourse Counseling, NorthPoint Health and Wellness Center, and Guadalupe Alternative Programs. At almost all of the schools participating in the ESMH program, collaborating agencies place a full-time, licensed mental health professional in the school to deliver clinical mental health treatment to students as well as treatment-related consultation to school staff and care coordination (Sander, Everts, & Johsnon, 2011). The ESMH therapists provide the same type of treatment services as they would at their agencies outpatient offices, but instead of being located at the agencies’ office they are located full time at a school. The therapists provide diagnostic assessments, and individual, family, and group therapy (see participation in treatment section for more details). ESMH therapists also deliver classroom presentations to students and training on mental health issues to school staff. When this study was initiated, the MPS ESMH program was in 12 schools. ESMH schools were selected on the basis of having low attendance rates, a large percentage of students struggling academically, and high percentages of students with risk factors including limited English proficiency (LEP) and receiving free or reduced priced lunch service.
Students are identified as possibly eligible for ESMH services by parents, teachers, school administrators, or school staff. Parents are able to refer their children to ESMH if their child attended one of the schools receiving services and they felt that their child could benefit from such services. School staff in the participating schools received training on identifying students who might be eligible for ESMH and were instructed to discuss students exhibiting behavioral problems, demonstrating inattention or having difficulty in class, struggling socially, or exhibiting mood problems (anxiety or depression) with the school social worker who would then contact the parents/guardian if appropriate. If the parent or guardian was interested in ESMH for their son or daughter, the ESMH therapist would meet with the family and obtain informed consent to treat from the parent or legal guardian prior to initiating services. ESMH clinicians would then conduct a diagnostic assessment with the family, and based on the findings and input of the student and their parents/guardian, a determination was made on whether individual, family, group therapy, or some combination of these modalities was most appropriate. Clinicians see students on a weekly basis or possibly more frequently depending on their needs and the treatment plan. Treatment plans are reviewed with parent and guardians every 90 days. ESMH clinicians typically have an active caseload of 20–25 students at any one time and over the course of a school year may work with approximately 30–40 students.
Student participants
While ESMH services were provided to students of all age and grade ranges, the sample for this study consisted of students in grades Kindergarten through 8th grade in 12 MPS schools. Students in grades 9–12 were not included in this sample because a majority of these students were missing their student identification number in the clinical database. For the purposes of this study, the 2008–2009 school year was considered a baseline year (i.e., prior to ESMH services). As such, students who received four or more hours of treatment through the ESMH program between October of 2008 (the earliest treatment record pulled from the clinical database) and 1 July 2009 were excluded from this study. The decision to exclude students with four or more hours was made on the basis of prior research, suggesting that this level of treatment corresponds with active client engagement
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
15:
24 1
0 N
ovem
ber
2014
29 Advances in School Mental Health Promotion
Table 1. Descriptive statistics on original ESMH sample, treatment, and matched-comparison students.
Original ESMH sample Comparison Treatment Variable (N ¼ 216) (N ¼ 89) (N ¼ 89)
American Indian 9.3% 11.2% 10.1% African American 62.5% 64% 61.8% Asian 1.4% 1.1% 1.1% Hispanic 18.1% 21.3% 24.7% White 8.8% 2.2% 2.2% LEP 13.9% 20.2% 20.2% Free/reduced priced lunch 94.4% 96.6% 96.6% Average percentage attendance 89.7% 92.1% 91% Average math scale score 191.7 191.7 190.3 Average reading scale score 184.4 184.1 183.0
and an adequate dosage of mental health therapy to possibly produce a treatment effect (Center for School Mental Health Assistance, 2003).
The selection criteria resulted in an original ESMH sample size of 216 students who received ESMH services. The sample of 216 students who received ESMH services was further restricted to a sample of 89 students who had complete data on the following variables: attendance percentage, grade, LEP status, free/reduced priced lunch status, special education status, whether the student was identified as homeless or highly mobile, the number of times a student was suspended, gender, ethnicity, and initial reading and mathematics score (in the Fall of 2009, prior to treatment) for matching purposes (described below). Descriptive statistics on the original ESMH sample, treatment, and matched-comparison students is presented in Table 1.
The multivariate matching algorithm implemented was performed using the R (R Core Team, 2013) package ‘Matching’ (Sekhon, 2011). Prior to implementing the multivariate matching procedure, the two unmatched samples (ESMH students and non-ESMH students) differed in terms of the rate of students receiving free or reduced priced lunch (Tx: 96.6%, control: 66.6%), special education flag (Tx: 43.8%, control: 16.1%), the proportion of African American students (Tx: 61.8%, control: 36.3%), and the proportion of Asian students (Tx: 1.1%, control: 8.0%). After implementing the multivariate matching procedure, the resulting matched sample (89 ESMH students and 89 matched non-ESMH students) did not differ significantly in terms of the variables used in the matching algorithm (see Table 1). Therefore, the matching procedure resulted in an analytical sample size of 178 students (89 ESMH and 89 matched non-ESMH students). This was the sample used to answer the research questions in this study.
The significant decrease from 216 students to 89 was primarily due to students being eliminated from the sample because they did not take the reading and/or the mathematics assessment in the Fall of 2009 (because they were not in the district in the Fall of 2009). The sampling of students that did not receive ESMH services was performed by obtaining a matched sample of students that did not receive ESMH services through the school district’s administrative database. The selection criterion was that these non-ESMH students be similar to the 89 SMH students on the set of variables: attendance percentage, grade, LEP status, free/reduced priced lunch status, special education status, whether the student was flagged as homeless or highly mobile, the number of times a student was suspended, gender, ethnicity, and initial reading and mathematics score (in the Fall of 2009, prior to treatment). The population of non-ESMH students considered for matching
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
15:
24 1
0 N
ovem
ber
2014
30 B.P. Daly et al.
consisted of students in Kindergarten through eighth grade in the Fall of 2009 who attended any school in the district (N . 15,000).
Procedure
Data spanned the 2008–2009 school year through the 2011–2012 school year and included data on demographic characteristics of students as well as measures of educational, behavior, and academic outcomes. With regard to demographic characteristics, data included student race/ethnicity, gender, whether the student received free or reduced priced lunch services (FRL), whether the student was flagged as having LEP, and whether the student was receiving special education services.
Measures
With regard to measures of school behavior, data included the percentage of attendance (i.e., the percentage of days a student attended school out of the days they were enrolled in the school) and the number of suspensions a student received. With regard to measures of academic outcomes, data included performance on the measures of academic progress (MAP) (http://www.nwea.org/) in both reading and mathematics. The MAP is a standardized computerized adaptive test administered in the district both in the Fall and Spring of each academic year.
The mathematics and reading scores, as well as the attendance outcome were analyzed longitudinally (mathematics and reading scores by semester, and attendance by year). When the outcome of interest was mathematics or reading performance as measured by the MAP, time was represented by an integer which ranged from zero to five where zero represented the Fall of 2008, one represented the Spring of 2009 and so forth until five represented the Spring of 2012, the last time point considered in this study. When the outcome of interest was attendance, the measure of time was an integer which reflected yearly progression through time where zero represented the 2008–2009 school year, one represented the 2009–2010 school year and so forth until three represented the 2011– 2012 school year, the last time point considered in this study.
The independent variable of interest in this study was a dichotomous variable indicating whether the student received ESMH services. This variable was created using data obtained from a record keeping system tracking the interactions of ESMH practitioners with students and their families. This record keeping system tracked many interactions practitioners had with students, even those the authors did not consider to be forms of treatment. To ensure that the treatment variable (whether the student received ESMH services) accurately reflected whether the student received treatment, the authors only considered some of the types of interactions ESMH clinicians had with students and their families. The interactions that were counted as treatment were family therapy with or without client, individual therapy 30 min, individual therapy 45 min, individual therapy 60 min, individual therapy 90 min, and psychiatric services and psychological testing.
After each interaction was coded as either a treatment or non-treatment interaction, the number of minutes were summed and divided by 60 min in order to determine the number of treatment hours students received from ESMH services. The treatment hours were then converted to a dichotomous treatment variable reflecting whether a student received four or more hours of treatment after July 2009 (0 ¼ no, 1 ¼ yes). Students in the matched non-ESMH sample were automatically coded as zero for the treatment variable (reflecting that by definition these individuals did not receive any treatment services).
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
15:
24 1
0 N
ovem
ber
2014
31 Advances in School Mental Health Promotion
Results
School behavior
Percentage attendance
The attendance data were modeled using a generalized linear model using the R (R Core Team, 2013) generalized estimating equation package ‘geepack’ (Hojsgaard, Halekoh, & Yan, 2006). Yearly attendance was dichotomized ($90% ¼ 1, ,90% ¼ 0). The authors used a binomial link function for the model with the dichotomized repeated measures attendance data nested within students. This longitudinal model included the independent variables grade in the 2009–2010 school year, LEP status, free/reduced priced lunch status, ethnicity (American Indian, African American, Asian, Hispanic, and white), and treatment (whether the student received four or more hours of ESMH services). While these variables were included in the multivariate matching procedure implemented for this study, these variables were also included in the models to attempt to control for any residual error, and because there is evidence that the outcome (attendance) differs by these demographic characteristics in this particular school district. All of the independent variables in the model with the exception of treatment were based on values observed in the database system for the 2009–2010 school year. The results of this analysis are presented in Table 2.
The results from the Attendance model above suggest that treatment was unrelated to whether a student attended school at least 90% of the time in a given school year. Specifically, the coefficient reflecting the relationship between treatment status and whether a student had an initial attendance rate of at least 90% was not statistically significant (b ¼ 20.18, p . 0.05). Additionally, the coefficient reflecting the relationship between treatment status and the linear trajectory of the log odds of a student attending school at least 90% of the time was not statistically significant (b ¼ 0.11, p . 0.05). This suggests that treatment and control students had similar trajectories in the likelihood of attending school at least 90% of the time. Additionally, the non-significant linear term suggests that this group of treatment and matched control students remained relatively flat in their likelihood of attending at least 90% of the time within the time frame observed in this study (from the 2008/2009 school year to the 2011/2012 school year). As expected, students in higher grades tended to be less likely to attend school at least 90% of the time, after controlling for the other variables in the model (b ¼ 20.09, p , 0.05).
Table 2. Fixed effects for longitudinal model of attending school 90% or higher.
Effect Estimate Standard error
Intercept 3.39*** 0.99 Grade 20.09 0.07 LEP 1.58 0.88 Free/reduced price lunch 21.61** 0.57 American Indian 21.05 0.83 African American 20.97 0.77 Asian 37.42*** 1.19 Hispanic 20.92 1.13 Linear 20.04 0.12 Treatment 20.18 0.28 Treatment £ linear 0.11 0.15
Notes: N ¼ 178 students. Linear term reflects school year. Treatment reflects either receiving ESMH services at any time after July 2009 or not receiving ESMH services. *p , 0.05, **p , 0.01, and ***p , 0.001.
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
15:
24 1
0 N
ovem
ber
2014
32 B.P. Daly et al.
Figure 1. Last observed number of suspensions by treatment and comparison group.
Suspensions
The suspension data were not able to be analyzed using a longitudinal model because suspension data were missing for the entire sample in the 2011 school year (as a consequence of data sharing issues). Instead, the data were analyzed using a cross-sectional negative binomial regression model in which the outcome of interest was the number of suspension during their last observed school year. This negative binomial regression was fit to the data using the R (R Core Team, 2013) package ‘pscl’. Figure 1 depicts the distribution of the last observed number of suspensions (i.e., the last year students were observed in the school district data capturing system, prior to the end of the 2011/2012 school year) by treatment group.
The majority of students in both the treatment and comparison groups had zero suspensions. Additionally, the comparison group appeared to have more variable counts of suspensions, whereas the counts of suspensions for the treatment group tended to be below 3 with the exception of a few students having a count of 10. The suspension data were analyzed using a negative binomial regression model in which this outcome was regressed on treatment hours, grade, LEP status, free/reduced priced lunch status, and the first observed number of suspensions. Ethnicity was removed from the model because the coefficients for ethnicity had extremely large standard errors due to the homogeneity of the sample with respect to ethnicity. The results from this model are presented in Table 3.
The results from the model fit to the suspension data suggests that treatment was not related to suspensions. Specifically, the coefficient of 20.02 was not statistically significant at a ¼ 0.05 suggesting that the log of the last observed number of suspensions was unrelated to treatment. As one would expect, the first observed number of suspensions was predictive of the last observed number of suspen
sions (b ¼ 0.14, p , 0.001). To examine whether the results were susceptible to
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
15:
24 1
0 N
ovem
ber
2014
Advances in School Mental Health Promotion 33
Table 3. Results of negative binomial regression for suspensions.
Effect Estimate Standard error
Intercept 21.48 1.09 Treatment 20.02 0.39 Grade 0.09 0.10 LEP 21.08 0.56 Free/reduced price lunch 0.70 0.77 First observed # of suspensions 0.14*** 0.04
Notes: N ¼ 178 students. Treatment reflects either receiving ESMH services at any time after July 2009 or not receiving ESMH services. *p , 0.05, **p , 0.01, and ***p , 0.001.
misspecification of the model, the outcome (last observed number of suspensions) was dichotomized (0 ¼ no suspensions) and the model was refit to the data using a logistic regression. The results were similar in that treatment was unrelated to the log odds of receiving at least one suspension, after controlling for the first observed number of suspensions.
Academic outcomes
The academic outcomes used in this study consisted of the MAP) assessments in both mathematics and reading. To examine the potential impact of ESMH services on academic
Figure 2. Average MAP score in mathematics by treatment and comparison groups.
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
15:
24 1
0 N
ovem
ber
2014
34 B.P. Daly et al.
outcomes, this study used the dichotomous treatment variable to predict the trajectories of MAP test.
Mathematics
Figure 2 depicts the relationship between time (i.e., semester) and average mathematics scale scores for the sample used in this study.
It is important to note that while Figure 2 reflects average mathematics performance of both students who received ESMH services and those that did not receive services, the graph reflects averages of students who were in various grades in a given year. For example, the Spring 2010 (S10) semester contains students that were in any grade between 2nd and 8th (the MAP assessments were only administered to students in grades 2–8), and the Fall 2010 (F10) semester contains students that were in any grade between 3rd and 8th grade. It is for this reason that the grade of a student in the 2009–2010 school year is included as a covariate in the longitudinal analysis.
In order to model the longitudinal nature of the mathematics scores, the authors built a linear mixed model starting with a random intercepts model containing grade as a predictor of the intercepts (to allow for heterogeneity in intercepts due to students in the sample being in different grades in 2009–2010). Model comparisons were performed after adding random effects for the linear trajectory (i.e., allowing students’ estimated trajectories to grow at different rates). The model with random intercepts and slopes fit the data better than a model with random intercepts and fixed slopes (based on the fit indices, x 2, AIC, and BIC).
The final model fit to the mathematics data consisted of a linear mixed model with the student’s grade in the 2009–2010 school year, LEP status, free/reduced priced lunch status, ethnicity (American Indian, African American, Asian, Hispanic, and white), percentage attendance, and the independent variable of interest, treatment. All of the covariates in the model were based on values observed in the database system for the 2009–2010 school year. This was done to attempt to statistically control for pre-existing differences between treatment and non-treatment students. The results of this analysis are presented in Table 4 along with the analysis of reading data.
Table 4. Fixed effects for longitudinal model of MAP mathematics and reading scale scores.
Mathematics Reading
Effect Estimate Standard error Estimate Standard error
Intercept 144.03*** 15.80 141.47*** 17.47 Treatment 22.17 1.96 22.00 2.16 Grade 6.18*** 0.51 6.08*** 0.56 LEP 25.47 6.12 214.72* 6.75 Free/reduced price lunch 27.49 6.71 221.06** 7.28 American Indian 11.24 7.08 11.41 7.81 African American 3.45 6.58 2.50 7.27 Asian 24.36 13.29 219.74 14.17 Hispanic 5.84 8.59 13.33 9.48 Percentage attendance 0.27* 0.13 0.37* 0.14 Linear 3.44*** 0.32 3.83*** 0.29 Treatment £ linear 0.30 0.45 20.04 0.40
Notes: N ¼ 178 students. Linear term reflects semester. Treatment reflects either receiving ESMH services at any time after July 2009 or not receiving ESMH services. *p , 0.05, **p , 0.01, and ***p , 0.001.
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
15:
24 1
0 N
ovem
ber
2014
35 Advances in School Mental Health Promotion
The relationship between the treatment variable and the mathematics score was not statistically significant at a ¼ 0.05. Specifically, the students that received ESMH services did not differ from those that did not receive ESMH services in terms of their initial mathematics scores (b ¼ 22.17, p . 0.05) or the linear trajectory of mathematics scores across semesters (b ¼ 0.30, p . 0.05). The average linear trajectory for this group of students was positive (b ¼ 3.44, p , 0.001) with students increasing an average of about 3.4 points per semester on the mathematics assessment.
Reading
Figure 3 depicts the relationship between time (i.e., semester) and average reading scale scores for the sample used in this study. The model fit to these data was the same as the model fit to the mathematics scale scores. While a linear mixed model with a quadratic time trend fit the reading data better (as measured by fit indices) than a model with only a linear time trend (as measured by model fit indices), a linear model was chosen in favor of ease of interpretation. The results were similar between the quadratic and linear models resulting in no change in the inferences made about the estimate of the relationship between treatment and reading scores.
As was observed for the mathematics data, the results from the longitudinal model of reading scale scores suggested that the relationship between treatment and reading scores was not statistically significant at a ¼ 0.05, after having controlled for the other variables. Specifically, students that received ESMH services, and those that did not, tended to start at about the same place in terms of reading scores (b ¼ 22.0, p . 0.05), and changed at
Figure 3. Average MAP score in reading by treatment and comparison groups.
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
15:
24 1
0 N
ovem
ber
2014
36 B.P. Daly et al.
about the same rate (b ¼ 20.04, p . 0.05). The average linear trajectory for this group of students was positive (b ¼ 3.83, p , 0.001) with students increasing an average of about 3.8 points per semester on the reading assessment.
The authors also fit the models above (for mathematics, reading, attendance, and suspensions) using a continuous treatment variable (reflecting the amount of hours of ESMH services students received). For the longitudinal models (mathematics, reading, and attendance), treatment hours were time-varying covariates (reflecting either the number of treatment hours received during each semester, for mathematics and reading, or year, for attendance). For the cross-sectional model of suspensions, treatment hours were summed across the entire time interval in between the pre- and post-observations of suspensions. The results using treatment hours as a continuous variable were similar to the results observed using treatment as a dichotomous variable. In both cases, treatment (as measured by hours or dichotomously) was unrelated to the outcomes of interest.
Discussion
This study examined the longitudinal effects of ESMH services on relevant school behavior and academic outcomes. While many program evaluations of ESMH focus on proximal and distal academic and educational outcomes, the study endpoint is usually the conclusion of one academic year. In contrast, this study is unique in that it examined data over a three-year time period to investigate the longitudinal effects of ESMH services on both proximal and distal outcomes.
Contrary to the hypothesis, findings from the current study revealed a slight decrease in rates of attendance for both the ESMH treatment group and the matched comparison group from school year 2008–2009 to school year 2011–2012. These findings are consistent with the results from other studies that demonstrated decreases in attendance as children age into adolescence – regardless of whether they receive mental health treatment (Baker, Sigmon, & Nugent, 2001; Corville-Smith, Ryan, Adams, & Dalicandro, 1998). Importantly, results from the study also revealed relatively similar rates of attendance over time for both the ESMH group and the comparison group. These findings are consistent with those of recent program evaluations of ESMH services which demonstrated no impact on rates of attendance (Kang-Yi, Mandell, & Hadley, 2013; Michael et al., 2013), as well as results from a Family Support Program that also indicated no impact on rates of attendance as compared to a no-treatment group (Pullmann, Weathers, Hensley, & Bruns, 2013). On the other hand, Anthony and Sebian (2011) found that children who received mental health treatment services in Baltimore’s ESMH program demonstrated an increase in attendance as compared to a decline in the matched comparison group. Notably, though, these results also suggested that rates of attendance were lowest for ESMH and comparison group students during school transition years such as the 6th and 9th grades. Given this finding, one recommendation is that ESMH programming that targets mental health conditions and other stressors that may impact attendance might demonstrate the most impact if delivered during the important school transition years of elementary to middle school and middle school to high school.
Although our hypothesis was not supported, it is notable that the rates of attendance in the current study for many of the students in the ESMH group and the matched comparison group were over 90% across the three school years, suggesting that both groups of students demonstrated satisfactory annual attendance. For instance, in school year 2011/2012, 62% of the treatment group and 64% of the comparison group attended over 90% of the school days in that academic year. This finding is important because it highlights that a large
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
15:
24 1
0 N
ovem
ber
2014
37 Advances in School Mental Health Promotion
portion of students in need of ESMH services demonstrated academic resiliency in that they attended the overwhelming majority of days of school. As such, directly targeting improvements in attendance as an outcome may not be appropriate for most students receiving ESMH services, given that this is not an area of need. However, 38% of the students in our study were struggling to consistently attend school (,90%). For these students, clinicians may want to engage in data monitoring of attendance and then implement more specific interventions as part of the treatment plan to reduce number of days absent from school (Lyon, Borntrager, Nakamura, & Higa-McMillan, 2013).
For out-of-school suspensions, the majority of students in the treatment and comparison groups did not receive any suspensions. Similar to the results for attendance, this finding suggests that most students receiving ESMH services in this study were not engaged in disruptive behavior significant enough to warrant an out-of-school suspension. Interestingly, for those students who were suspended, the comparison group appeared to have more variable counts of suspensions and had more students receiving 11 or more out of school suspensions. Alternatively, for the most part, the counts of suspensions for the treatment group tended to be less than three. This finding again highlights the importance of initial assessment of suspension data for clients receiving ESMH services followed by continual data monitoring to determine whether this is a variable worthy of intervention. Caution is warranted for interpreting these findings, however, as the results are cross-sectional and therefore prevent us from making any cause and effect determinations.
This study also evaluated distal academic outcomes (i.e., standardized test scores). Our hypothesis was supported in that EMSH treatment did not significantly impact academic outcomes as measured by MAP reading and MAP math scores. Notably, though, standardized test scores did improve over time for both the ESMH and matched comparison group. Recent evaluations of school-based supportive services also have failed to reveal statistically significant program impact on standardized test scores (Anthony & Sebian, 2011; Pullmann et al., 2013). While a positive trend in scores emerged in our study, results from the Anthony and Sebian (2011) evaluation indicated that students in both the ESMH and matched comparison group demonstrated decreases in standardized reading and math scores over time. But, similar to our study results, the pattern of change over time was consistent between groups. An important finding from the Anthony and Sebian study (2011) was that exposure to ESMH services positively impacted the percentage of students improving on math and English benchmark assessments over time for almost all grade levels (1st–8th). Because benchmark assessments are given to students on a quarterly basis, the authors interpreted these findings to suggest that these scores may be a more sensitive and proximal measure of ESMH impact on academic progress as compared to annual or bi-annual standardized test scores which probably better measure cumulative knowledge acquired over a long period of time.
Limitations
The contribution of our investigation must be interpreted within the notable limitations of this study design. First, the extant literature has demonstrated that clinicians typically account for more variance in client outcomes relative to differences in treatments or client baseline characteristics (Crits-Christoph & Mintz, 1991; Luborsky, McLellan, Diguer, Woody, & Seligman, 1997). Thus, an important limitation is that our models did not account for nesting by clinician or school even though treatment effects may vary by clinician or school.
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
15:
24 1
0 N
ovem
ber
2014
38 B.P. Daly et al.
Additional limitations speak to the methodological challenges associated with using matched comparison groups when conducting analyses. First, selection criteria used to form matched comparison groups frequently result in significant reductions in sample size – an occurrence that was true in the current study. For example, the first inclusion/exclusion criteria, that students in the ESMH group must have received at least one mental health treatment service in school year 2010, resulted in a sample size of roughly 900 students. As additional criteria were applied, the sample size dropped, resulting in a final sample of 89 students in the ESMH group. Given the relatively small sample size and the unique characteristics of the sample (high percentage of free/reduced price lunch, high percentage not meeting state reading and math standards), these results are applicable to a relatively restricted sample of students and cannot be generalized to the larger school population.
Another limitation of this study is that, while the authors controlled for a set of covariates thought to be proxies for need of ESMH services, data on need (as measured by mental health assessments) were not available for students who did not receive ESMH services, and as such, the authors were unable to control for mental health need. To the extent that need is not evenly distributed between the treatment and the matched-control group, the inferences about the relationship between ESMH services and the outcomes of interest in this study may be biased. Additionally, the multivariate matching procedure did not take into account the school a student attended when the referral for ESMH services was made, because the authors did not always know exactly which school the student attended when the referral was made. To the extent that the ESMH referral process varied by school, and mental health need differed by ESMH eligible and ESMH non-eligible schools, the estimates of the treatment effect of ESMH services may be biased.
As noted previously, another methodological limitation is that in order to form a group of ‘treatment naive’ students for the ESMH group, which is necessary to run a clean comparison between treatment and comparison groups, we excluded students from the ESMH group if they had received 4 h or more of treatment services in the prior year (2008–2009). This level of treatment hours was chosen as a cut-off based on the suggestion that four or more services are necessary for students to receive a meaningful quantity of therapeutic services (Center for School Mental Health Assistance, 2003). That being said, the extant literature is decidedly mixed with regard to the dose–response relationship (number of treatment hours received and changes in symptoms or outcomes) for child and adolescent mental health services. For example, Bickman, Andrade, and Lambert (2002) evaluated the dose–response relationship for youth in a community setting and reported findings that demonstrated no statistically significant dose–response. These results led the authors to conclude that no recommendations are consistent enough to guide clinicians, researchers, administrators, or policy makers regarding the dose– response relationship for youth mental health (Bickman et al., 2002). Thus, while we selected 4 or more hours as qualifying for meaningful treatment, which then resulted in the exclusion of many subjects from the ESMH group, it may be that this cut-off was not appropriate. Overall, because the effects of dosage on program outcomes remain complex and understudied, future research should design studies that are able to disentangle the impact of dosage on ESMH services.
Implications and future directions
Further research should seek to evaluate which ‘outcomes’ are sensitive enough to change from ESMH services and determine, as much as possible, what measures will remain constant over the duration of the program evaluation (e.g., annual standardized testing).
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
15:
24 1
0 N
ovem
ber
2014
39 Advances in School Mental Health Promotion
Related to this point, clinicians and researchers need to be careful about what ‘outcomes’ are expected to change in a positive direction. Additionally, researchers should think about using standardized testing that measures expected and actual growth and not just performance and proficiency. For instance, academic variables that have not received much attention in program evaluations, but which may be sensitive to ESMH impact, include passing status, credit readiness, and graduation readiness. Setting or system variables that also may be important as outcome or mediators of ESMH impact include school climate, school connectedness, and school engagement. Finally, future studies should more specifically investigate the effects of ESMH services on those students considered most at-risk by virtue of low attendance, high rates of suspension, poor academic grades, and/or low standardized test scores.
The ability of ESMH programs to demonstrate ‘lasting impact’ on school behavior and academic variables represents an important, as well as methodologically challenging, next step for the school mental health research agenda. Conducting well-controlled research studies of comprehensive school mental health programs is difficult, given the many system and ethical barriers associated with employing a no-treatment control group study design that necessarily restricts students from receiving needed mental health services. Utilizing matched comparison groups or propensity score matching research designs represent promising strategies for effectively measuring the effects of ESMH services on mental health, behavioral, and academic outcomes. Our study contributes to the developing school mental health program evaluation literature by underscoring that there is more work to be done in demonstrating lasting impact on educational variables. However, as the research of ESMH and SMH programs becomes more sophisticated through the use well-controlled, longitudinal research designs, there is hope that the results will substantially add to our understanding of which behavioral and academic variables are most sensitive to effects of ESMH treatment.
References Anthony, B. J., & Sebian, J. K. (2011). Baltimore expanded school mental health: Report of the
2008 – 2009 program evaluation. Washington, DC: Georgetown University Center for Child and Human Development.
Armbruster, P., & Lichtman, J. (1999). Are school-based mental health services effective? Evidence from 36 inner city schools. Community Mental Health Journal, 35, 493–504.
Atkins, M., Hoagwood, K., Kutash, K., & Seidman, E. (2010). Towards the integration of education and mental health in schools. Administration and Policy in Mental Health and Mental Health Services Research, 37, 40–47.
Baker, M. L., Sigmon, J. N., & Nugent, M. E. (2001). Truancy reduction: Keeping students in school. Washington, DC: Office of Juvenile Justice and Delinquency Prevention. Juvenile Justice Bulletin. Retrieved March, 2013 from http://www.ncjrs.org/html/ojjdp/jjbul2001_9_1/contents. html
Bickman, L., Andrade, A. R., & Lambert, W. (2002). Dose response in child and adolescent mental health services. Journal of Mental Health Services Research, 4, 57–70.
Bishop, J. H., Bishop, M., Bishop, M., Gelbwasser, L., Green, S., Peterson, E., . . . Zuckerman, A. (2004). Why we harass nerds and freaks: A formal theory of student culture and norms. Journal of School Health, 74, 235–251.
Bruns, E. J., Walrath, C., Glass-Siegel, M., & Weist, M. D. (2004). School-based mental health services in Baltimore: Association with school climate and special education referrals. Behavior Modification, 28, 491–512.
Catalano, R. F., Haggerty, K. P., Oesterle, S., Fleming, C. B., & Hawkins, J. D. (2004). The importance of bonding to school for health development: Findings from the Social Development Research Group. Journal of School Health, 74, 252–261.
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
15:
24 1
0 N
ovem
ber
2014
40 B.P. Daly et al.
Center for School Mental Health Assistance. (2003). Outcomes of expanded school mental health programs. Baltimore, MD: Author.
Children First Plan. (2002). Children First Plan: Evaluation report for year four. Cincinnati, OH: Author.
Corville-Smith, J., Ryan, B. A., Adams, G. R., & Dalicandro, T. (1998). Distinguishing absentee students from regular attenders: The combined influence of personal, family, and school factors. Journal of Youth and Adolescence, 27, 629–649.
Crits-Christoph, P., & Mintz, J. (1991). Implications of therapist effects for the design and analysis of comparative studies of psychotherapies. Journal of Consulting and Clinical Psychology, 59, 20–26.
Daly, B. P., Burke, R., Hare, I., Mills, C., Owens, C., Moore, E., & Weist, M. D. (2006). Enhancing school mental health – No child left behind connections. Journal of School Health, 76, 446–451.
Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82, 405–432.
Elias, M. J., Gager, P., & Leon, S. (1997). Spreading a warm blanket of prevention over all children: Guidelines for selecting substance abuse and related prevention curricula for use in the schools. Journal of Primary Prevention, 18, 41–69.
Flaherty, L. T., & Osher, D. (2003). History of school-based mental health services in the United States. In M. D. Weist, S. W. Evans, & N. A. Lever (Eds.), Handbook of school mental health: Advancing practice and research (pp. 11–22). New York, NY: Kluwer Academic/Plenum Publishers.
Flaherty, L. T., & Weist, M. D. (1999). School-based mental health services: The Baltimore models. Psychology in the Schools, 36, 379–389.
Foster, S., Rollefson, M., Doksum, T., Noonan, D., Robinson, G., & Teich, J. (2005). School Mental Health Services in the United States, 2002 – 2003. DHHS Pub. No. (SMA) 05-4068 Rockville, MD: Center for Mental Health Services, Substance Abuse and Mental Health Services Administration.
Hall, S. (2000). Final report youth and family centers program: 1999 – 2000. Dallas, TX: Dallas Public Schools Division of Evaluation, Accountability, and Information Systems.
Hoagwood, K., & Erwin, H. (1997). Effectiveness of school-based mental health services for children: A 10-year research review. Journal of Child and Family Studies, 6, 435–451.
Hoagwood, K. E., Olin, S. S., Kerker, B. D., Kratochwill, T. R., Crowe, M., & Saka, N. (2007). Empirically based school interventions targeted at academic and mental health functioning. Journal of Emotional and Behavioral Disorders, 15, 66–92.
Hojsgaard, S., Halekoh, U., & Yan, J. (2006). The R package geepack for generalized estimating equations. Journal of Statistical Software, 15, 1–11.
Hussey, D., & Guo, S. (2003). Measuring behavior change in young children receiving intensive school-based mental health services. Journal of Community Psychology, 31, 629–639.
Jennings, J., Pearson, G., & Harris, M. (2000). Implementing and maintaining school-based mental health services in a large urban school district. Journal of School Health, 70, 201–296.
Juszczak, L., Melinkovich, P., & Kaplan, D. (2003). Use of health and mental health services by adolescents across multiple delivery sites. Journal of Adolescent Health, 32, 108–118.
Kang-Yi, C. D., Mandell, D. S., & Hadley, T. (2013). School-based mental health program evaluation: children’s school outcomes and acute mental health service use. Journal of School Health, 83, 463–472.
Klern, A. M., & Connell, J. P. (2004). Relationships matter: Linking teacher support to student engagement and achievement. Journal of School Health, 74, 262–273.
Kutash, K., Duchnowski, A. J., & Lynn, N. (2006). School-based mental health: A guide for decision-makers. Tampa, FL: University of South Florida Louis de la Parte Florida Mental Health Institute, Research and Training Center for Children’s Mental Health.
Luborsky, L., McLellan, A. T., Diguer, L., Woody, G., & Seligman, D. A. (1997). The psychotherapist matters: Comparison of outcomes across twenty-two therapists and seven patient samples. Clinical Psychology: Science and Practice, 4, 53–65.
Lyon, A. R., Borntrager, C., Nakamura, B., & Higa-McMillan, C. (2013). From distal to proximal: Routine educational data monitoring in school-based mental health. Advances in School Mental Health Promotion, 6, 263–279.
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
15:
24 1
0 N
ovem
ber
2014
41 Advances in School Mental Health Promotion
Michael, K. D., Albright, A., Jameson, J. P., Sale, R., Massey, C., Kirk, A., & Egan, T. E. (2013). Does cognitive behavioural therapy in the context of a rural school mental programme have an impact on academic outcomes? Advances in School Mental Health Promotion, 6, 247–262.
Nabors, L. A., & Reynolds, M. W. (2000). Program evaluation activities: Outcomes related to treatment for adolescents receiving school-based mental health services. Children’s Services: Social Policy, Research, and Practice, 3, 175–189.
Owens, J. S., & Murphy, C. E. (2004). Effectiveness research in the context of school-based mental health. Clinical Child and Family Psychology Review, 7, 195–209.
Paternite, C. E. (2005). School-based mental health programs and services: Overview and introduction to the special issue. Journal of Abnormal Child Psychology, 33, 657–663.
Pullmann, M. D., Weathers, E. S., Hensley, S., & Bruns, E. J. (2013). Academic outcomes of an elementary school-based family support programme. Advances in School Mental Health Promotion, 6, 231–246.
R Core Team (2013). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from http://www.R-project.org/
Sander, M. A., Everts, J., & Johnson, J. (2011). Using data to inform program design and implementation and make the case for school mental health. Advances in School Mental Health Promotion, 4, 13–21.
Sekhon, J. S. (2011). Multivariate and propensity score matching software with automated balance optimization: The matching package for R. Journal of Statistical Software, 42, 1–52.
Stephan, S., Brandt, N., Lever, N., Acosta-Price, O., & Connors, E. (2012). Key priorities, challenges, and opportunities to advance an integrated mental health-education research agenda. Advances in School Mental Health Promotion, 5, 125–138.
University of Maryland, School Mental Health Program (1999). SMHP: Annual report for 1998 – 1999. Baltimore, MD: Author.
Walker, S. C., Kerns, S., Lyon, A. R., Bruns, E. J., & Cosgrove, T. J. (2010). Impact of school-based health center use on academic outcomes. Journal of Adolescent Health, 46, 251–257.
Weare, K. (2000). Promoting mental, emotional and social health: A whole school approach. London: Routledge.
Weist, M. D. (1997). Expanded school mental health services: A national movement in progress. In T. H. Ollendick & R. J. Prinz (Eds.), Advances in clinical child psychology (pp. 319–352). New York, NY: Plenum Press.
Weist, M. D., & Albus, K. E. (2004). Expanded school mental health: Exploring program details and developing the research base. Behavior Modification, 28, 463–471.
Weist, M. D., & Christodulu, K. V. (2000). Expanded school mental health programs: Advancing reform and closing the gap between research and practice. Journal of School Health, 70, 195–201.
Weist, M. D., Myers, C. P., Hastings, E., Ghuman, H., & Han, Y. L. (1999). Psychosocial functioning of youth receiving mental health services in the schools versus community mental health. Community Mental Health Journal, 35, 69–81.
Weist, M. D., Nabors, L. A., Myers, C. P., & Armbruster, P. (2000). Evaluation of expanded school mental health programs. Community Mental Health Journal, 36, 395–412.
Weist, M. D., Lowie, J. A., Flaherty, L. T., & Pruitt, D. (2001). Collaboration among the education, mental health, and public health systems to promote youth mental health. Psychiatric Services Journal, 52, 1348–1351.
Weist, M. D., Evans, S. W., & Lever, N. A. (2003). Introduction: Advancing mental health practice and research in schools. In M. D. Weist, S. W. Evans, & N. A. Lever (Eds.), Handbook of school mental health: Advancing practice and research (pp. 1–7). New York, NY: Kluwer Academic/ Plenum Publishers.
Wilson, G. T. (2007). Manual-based treatment: Evolution and evaluation. In T. A. Treat, R. R. Bootzin, & T. B. Baker (Eds.), Psychological clinical science: Papers in honor of Richard M. McFall. Modern pioneers in psychological science (pp. 105–132). New York, NY: Psychology Press.
Zins, J. E., Bloodworth, M. R., Weissberg, R. P., & Walberg, H. J. (2004). The scientific base linking social and emotional learning to school success. In J. Zins, R. Weissberg, M. Wang, & H. J. Walberg (Eds.), Building academic success on social and emotional learning: What does the research say? (pp. 3–22). New York, NY: Teachers College Press.
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
15:
24 1
0 N
ovem
ber
2014
