Embed Size (px)
Citation preview
ORIGINAL PAPER
Enhancing Student Attitudes via a Brief, Universal-Level BullyingPrevention Curriculum
Tyler L. Renshaw • Shane R. Jimerson
Published online: 8 December 2011
� Springer Science + Business Media, LLC 2011
Abstract Bullying is a salient challenge for children and
schools around the world, appearing to be especially pre-
valent at the middle-school level. Contemporary research
reveals an exigent need for systematic programing that
aims to reduce and prevent bullying via promoting
awareness, attitudes, and skills that facilitate the develop-
ment of positive peer relationships and positive school
climates. Considering this need, the present quasi-experi-
mental study examined the effects of a relatively new,
brief, universal-level bullying prevention curriculum on
middle-school students’ general attitudes toward bullying
and perceptions of their school bullying supports. Results
indicated statistically significant enhancement of prosocial
attitudes for intervention-group students compared to
control-group students, with small effect sizes. Feasibility
and social validity outcomes indicated that the intervention
was implemented with adequate fidelity and that its goals,
procedures, and outcomes were perceived as appropriate.
The interpretation, limitations, and implications of these
results for practice are discussed herein.
Keywords Bullying � Universal-level intervention �Attitudes � Social validity � Aggression
Introduction
Given that most severe forms of aggression and victim-
ization (e.g., assault with a weapon) take place in the home
and other community settings, schools are often considered
one of the safest places for youth (Jimerson, Morrison,
Pletcher, & Fulrong, 2006). Nevertheless, given the amount
of mildly aggressive acts perpetrated on campus, schools
have also been conceptualized as epicenters for lower-level,
non-severe violence (e.g., verbal harassment; Furlong,
Pavelski, & Saxton, 2002). Among these less-severe-yet-
more-common forms of violence perpetrated at school,
bullying has received increasing attention during the past
decade, as it is becoming more prevalent, is associated with
a host of deleterious outcomes for students and school
climates, and is posited to serve as a developmental path-
way toward more severe forms of antisocial behavior
(Spivak & Prothrow-Stith, 2001). In response, researchers
from various disciplines have undertaken bullying schol-
arship in an attempt to better understand the nature of the
phenomenon and to develop interventions that mitigate its
negative effects on youth and schools (e.g., Espelage &
Swearer, 2004; Jimerson, Swearer, & Espelage, 2010).
Bullying is generally operationalized as a three-dimen-
sional construct: an (1) intentional, malicious form of
victimization that (2) occurs repeatedly over time (3)
between parties wherein there exists a relative power dif-
ferential (Olweus, 2010). Students may participate in bul-
lying relationships via several roles: as bullies, victims,
bully-victims, and bystanders (Alsaker and Gutzwiller-
Helfenfinger 2010). To date, each of these participatory
roles has been identified in every nation that has under-
taken a descriptive study of this three-dimensional con-
struct, suggesting that bullying is problematic for students
and schools around the world (Jimerson et al., 2010). For
T. L. Renshaw � S. R. Jimerson
Department of Counseling/Clinical/School Psychology,
University of California at Santa Barbara, Santa Barbara, CA,
USA
T. L. Renshaw (&)
Special Education Department, Alpine School District, 575 N.
100 E., American Fork, UT 84003, USA
e-mail: [email protected]
123
School Mental Health (2012) 4:115–128
DOI 10.1007/s12310-011-9069-2
example, within the United States alone, it is estimated that
around 30% of students in sixth through tenth grades are
involved in bullying—either as a bully, victim, or both (Nansel
et al., 2001). Moreover, within the past decade, several
initiatives studying bullying from a cross-cultural perspec-
tive continue to reveal increasing prevalence rates within
both Western and Eastern nations (e.g., Murray-Harvey,
Slee, & Taki, 2010). Such international studies indicate
that school bullying in all countries, including the United
States, is highly associated with pervasive, negative school
climate effects, including an environment characterized
by increased interpersonal tension and intimidation as
well as decreased school connectedness and academic
engagement (e.g., Kosciw, Greytak, Diaz, & Bartkiewicz,
2010).
Beyond elevated prevalence rates, both national and
cross-national studies reveal that bullying is highly asso-
ciated with deleterious physical, social, educational, and
psychological outcomes for victims, perpetrators, and
bystanders. Specifically, being bullied has been shown to
be related to enhanced likelihood for acquiring generalized
and socialized anxiety, depression, loneliness, posttrau-
matic stress, low self-esteem and self-competence, poor
school adjustment and academic performance, poor phys-
ical health, self-perceived unpopularity, externalizing
problem behaviors (e.g., intentional rule-breaking), and
attention difficulties (Dioguardi & Theodore, 2006).
Moreover, in a review of research investigating the nega-
tive correlates for all parties involved in bullying (Swearer,
Grills, Haye, & Cary, 2004), several noteworthy psycho-
social outcome trends were highlighted: (a) both bullies
and victims report lower levels of self-esteem than unin-
volved students; (b) bully-victims report lower levels of
self-esteem than all other involved parties; (c) youth
involved in bullying more frequently, no matter what their
role, report lower self-esteem than those who are involved
less often; (d) depression is highly associated with all roles
in the bullying process, with bully-victims showing the
highest levels; (e) bully-victims are the most at-risk for
suicide of all involved parties; and (f) victims and bully-
victims manifest greater general anxiety than bullies and
uninvolved parties. Taken together, these data suggest that
bullying involvement at any level is associated with dele-
terious psychosocial outcomes, while bully-victims are at
the highest risk for mental health problems.
Given the negative effects of bullying on school climates
and youth’s well-being, a large portion of bullying schol-
arship has focused on evaluating the effectiveness of sys-
tematic prevention and intervention efforts employed
within school settings. Although all bullying prevention
programs share a similar overarching aim—namely,
reducing bullying behavior—they seek to accomplish this
aim via different means. For example, some programs focus
on identifying, targeting, and remediating bullies, while
others focus on identifying and empowering victims or
bystanders; some focus on improving the social skills of all
students, while others focus primarily on changing the
school culture and environment. Within the past 5 years,
several meta-analyses have synthesized the national and
international state-of-the-art surrounding these efforts.
Taken together, these studies have indicated mixed find-
ings, suggesting that, overall, bullying prevention and
intervention efforts may have negligible to strong effects on
student outcomes, depending on what indicators are being
evaluated. For example, Smith, Schneider, Smith and
Ananiadou (2004) reported negligible effects for both self-
reported victimization and perpetration outcomes, while
Ttofi, Farrington and Baldry (2008) reported large, positive
effects for observed bullying and self-reported emotional
and attitudinal outcomes. Yet another meta-analysis, con-
ducted by Merrell, Guelder, Ross and Isava (2008), reported
small to moderate, positive effects for attitudinal outcomes
but primarily negligible effects for behavioral outcomes.
Although the types of bullying interventions evaluated
in these meta-analyses seek to reduce bullying via different
means, and although there is little agreement among pro-
fessionals regarding the best approach for effectively pre-
venting bullying (Rigby & Bauman, 2010), a review of the
most popular intervention programs suggests that current
prevention efforts—and thus those evaluated within the
meta-analyses described above—tend to share several core
administrative characteristics: large-scale year-to-multi-
year implementation periods, general resource intensive-
ness (i.e., constant assistance from extra-school support
stuff and substantial training needed for local educators), a
school-ecology intervention approach (i.e., involving
individual, classroom, and schoolwide components, which
target staff, student, and policy changes), and an orientation
toward antisocial behavior and attitude reduction (Jimerson
& Huai, 2010). Given this administrative homogeneity
among bullying prevention programs, and considering the
mixed effectiveness of such efforts to date, there appears to
be a substantial warrant for the development and evaluation
of different kinds of bullying interventions, including those
characterized by smaller-scale and briefer implementation
periods, lower resource intensiveness (e.g., little involve-
ment from outside support staff as well as little training
requirements), reduced and isolated intervention compo-
nents (e.g., classroom-only efforts), and an orientation
toward prosocial behavior and attitude development. In
addition, there is also a warrant for the development and
evaluation of bullying interventions that are informed by
contemporary research regarding the core characteristics of
effective prevention programs.
As a result of the mixed effectiveness of bullying
interventions, much scholarship has also been devoted to
116 School Mental Health (2012) 4:115–128
123
identifying the core characteristics of successful bullying
prevention programing. But similar to the meta-analyses,
empirical studies in this area have also revealed differing
findings. For instance, Ttofi et al. (2008) noted eight core
components associated with programs that demonstrated
effective reductions in reported bullying behavior: parent
training, improved playground supervision, enhanced dis-
ciplinary methods, frequent school conferences, provision
of information for parents, enhanced classroom rules,
improved classroom management, and inclusion of video-
based instructional materials—and five core components
associated with programs that demonstrated effective
reductions in reported victimization experiences: enhanced
disciplinary methods, initiated work with peers, parent
training, cooperative group work, and improved play-
ground supervision. Moreover, Rigby (2006) observed that
effective bullying interventions are typically comprised of
at least three core components: the development of uni-
versal-level antibullying policies; the integration of a
social-cognitive classroom curriculum to raise awareness
of bullying, promote prosocial attitudes, and teach social-
emotional learning skills; and the establishment of proce-
dures for handling bullying incidences at school. Further-
more, Hazler and Carney (2006) concluded that the most
effective bullying interventions seem to share several
characteristics: enhancing the empathic involvement of
students, increasing school connectedness, raising aware-
ness regarding bullying and its deleterious effects, estab-
lishing anti-bullying school policies, and developing
student social skills.
In addition to centering the development and imple-
mentation of interventions around the characteristics of
effective bullying programs (described above), scholars
also acknowledge that consideration of contextual and
feasibility elements—which are typically ignored in studies
identifying effective program characteristics—is equally
important, as striking a balance between empirical rigor
and social validity is necessary to promote the sustain-
ability of bullying interventions at the local site level
(Larson & Busse, 2006). To date, as noted above, the most
widely used bullying interventions are characterized by
administrative characteristics that are likely to negatively
impact social validity. Specifically, large-scale, multi-year,
resource-intensive, multicomponent prevention efforts are
likely to tax the motivation of school personnel and
diminish the overall implementation fidelity of the inter-
vention. This is especially likely in school contexts wherein
substantial extra-school resources, such as monetary sup-
port and additional administrative personnel, are limited or
absent. Therefore, in addition to research evaluating
empirically promising interventions that are grounded in
contemporary theory and evidence, there is also a sub-
stantial warrant for research evaluating the social validity
and feasibility of these efforts (Olweus, 2010). Without this
sort of scholarship, the literature will remain ambiguous
regarding the capacity of school personnel to effectively
sustain effective bullying prevention programing at the
local site level, without the assistance of significant extra-
school resources. Such local sustainability seems necessary
if bullying prevention is ever to take root as a core com-
ponent of educators’ practice and schools’ curriculum.
Beyond the social validity and feasibility of bullying
interventions, there is another conspicuous characteristic
that has yet to receive much empirical attention: dosage-
response effects (Merrell et al., 2008). In the greater world
of educational research, dosage-response effects have been
frequently investigated and demonstrated to be a significant
moderator of intervention effectiveness. For example,
several studies show that measures of cognitive and aca-
demic achievement increase as intervention duration
increases (e.g., Breit-Smith, Justice, McGinty, & Kaderavek,
2009), while other studies indicate that increased inter-
vention duration is significantly associated with increased
positive behavioral outcomes in the classroom and other
school-based settings (e.g., Zelezny, 1999). Furthermore,
educational scholarship demonstrates that the maintenance
of positive academic and behavioral outcomes is also sig-
nificantly moderated by the original intervention duration
(e.g., Reynolds, Temple, Robertson, & Mann, 2001).
Generalizing from these and similar findings, it seems
reasonable that intervention duration may be a key mod-
erating characteristic of the effectiveness of bullying
prevention programing. So far, however, most studies
reporting on bullying prevention span long intervention
durations—ranging from 1 year to multiple years—while
other educational interventions, such as those described
above, are typically characterized by multi-week to multi-
month durations. Moreover, there is no evidence to indicate
that bullying interventions need to be this long, nor that
briefer interventions (i.e., multi-week to multi-month) may
not be just as effective. In fact, given the strain on
resources and social validity that can accompany long-term
programing efforts, and recognizing the mixed findings
currently yielded by efforts of this nature, there appears to
be a substantial warrant for investigating the influence of
dosage-response effects on the utility of briefer bullying
interventions in school settings.
That said, the purpose of the present study was to further
bullying intervention scholarship by investigating the ini-
tial effects of a relatively new, brief, empirically promis-
ing, universal-level antibullying curriculum: Promoting
Positive Peer Relationships (P3R; Faull, Swearer, Jimer-
son, Espelage, & Ng, 2008). P3R is grounded in a trans-
actional-ecological model of youth’s development, which
views all behavior, including antisocial acts such as bul-
lying, within a functional-contextual framework—where
School Mental Health (2012) 4:115–128 117
123
habits of action are engaged and maintained because they
help youth better adapt to their environments (Sameroff,
2000). Working from this theoretical perspective, P3R
seeks to alter youth’s developmental trajectories toward
more adaptive and healthier outcomes by raising awareness
regarding bullying-related issues and promoting prosocial
cognitive-behavioral habits (Faull et al., 2008). Unlike the
majority of current universal-level bullying interventions,
P3R is amenable to implementation within smaller-scale
and briefer implementation periods (i.e., 5 weeks to
2 months), is characterized by low-resource intensiveness
(i.e., requires little involvement from outside support staff
as well as little training and support materials), is amenable
to investigation of isolated intervention components (i.e.,
classroom-based intervention only), and is characterized by
an orientation toward prosocial behavior and attitude
development (i.e., cultivation of knowledge, empathy, and
problem-solving skills). Considering the various areas of
warranted investigation within bullying prevention
research (as noted above), the present study aimed to
evaluate more than just general intervention efficacy—
extending the inquiry to target dosage-response effects and
social validity considerations. Specifically, the following
research questions were investigated by utilizing P3R
within a junior-high school context:
1. Is P3R a feasible and socially valid intervention when
used within a general education classroom and imple-
mented by a general education teacher?
2. Is P3R an effective intervention for enhancing stu-
dents’ general attitudes toward bullying as well as their
perceptions of their school’s bullying supports?
3. Does the implementation duration of P3R (i.e., 1-, 5-,
and 8-week iterations) have dosage-response effects on
students’ attitudinal outcomes?
Method
School Context and Participants
The public junior-high school targeted in this study was
located on the south-central coast of California, near a public
university with which both of the authors were affiliated.
During the beginning of the academic school year in which
the study was conducted, the administration of the junior-
high school and the two authors formed an applied research
partnership. The administration noted that the school was
experiencing increased bullying-related behavior, as evi-
denced by the nature of rising disciplinary referrals, and
requested that the two authors help facilitate an appropriate
schoolwide intervention. The authors agreed to provide the
school with the necessary resources to implement and
evaluate P3R during the coming year, given that the
administration allowed the authors to use the de-identified
data for research purposes. Therefore, during the year in
which the study was conducted, the junior-high school
adopted P3R as a mandatory element of the exploratory
Health class taken by all seventh-grade students. Thus, all
intervention-group students were seventh-graders who
received the curriculum as part of their exploratory Health
class curriculum during one of three trimesters (i.e., fall,
winter, or spring), while all control-group students were 8th
graders enrolled in alternative elective courses during the
same trimesters.
Given this school context, all participants attended the
same public junior-high school described above. The total
sample was comprised of 636 seventh-grade and eighth-
grade students, of which 320 were in the intervention group
and 316 were in the control group. Although the entire
student population was targeted as part of either the inter-
vention group or the control group, only data from
approximately 75% of student body were used herein—due
to unusable data resulting from failure to complete surveys,
incomplete survey responses, or questionable survey
response patterns. As part of this study, demographics for
each participating student were not collected; however,
given that the majority of the school’s students participated
in the study and that no particular demographics were
biased through participant selection, schoolwide demo-
graphics were deemed to be an appropriate proxy measure.
At the time of this study, then, 46% of the student popula-
tion at this particular school were identified as White, 40%
as Hispanic or Latino, 6% as Asian, 2% as African Amer-
ican, and 6% as multiple or other ethnicities. Furthermore,
31% of students were classified as socioeconomically dis-
advantaged—as evidenced by their qualification for free or
reduced-price lunch—16% as English-language learners,
and 15% as students with disabilities. Demographics for
participating students were assumed to be similar.
P3R Curriculum
P3R is comprised of a series of psychoeducational film-
based resources for supporting students, educators, and the
surrounding community in addressing the problem of bul-
lying in schools. The full curriculum is comprised of three
separate yet interrelated programs—including classroom,
professional development, and community education
resources—that can be used in tandem or in isolation.
Development of these resources was a multiyear project
resulting from the collaborative efforts of a professional
film director and a team of academic school psychologists
(of which the second author was a member), who were
guided by empirical literature, best practices, and consul-
tative feedback from various students, teachers, school
118 School Mental Health (2012) 4:115–128
123
administrators, parents, and other child-service providers
from across the United States (Faull et al., 2008). Given
that P3R was created specifically for school-based use, its
program features are designed to be feasibly implemented
within school settings by either school-based mental health
professionals (e.g., school psychologists) or educators (e.g.,
general and special education teachers). Since the public
release of P3R in 2008, several small-scale empirical
studies have been undertaken in various locations around
the United States to evaluate the effects of the intervention
on various students’, parents’, and educators’ bullying-
related attitudes (e.g., empathy toward victims) and
behaviors (e.g., bullying perpetration). To date, however,
none of these studies have been published in professional
journals; therefore, there are no public results available
regarding the preliminary effects of P3R. (For more com-
prehensive information regarding P3R and its develop-
ment, see www.storiesofus.com.)
Although P3R is a multicomponent and ecologically
oriented intervention, the present study examined only the
effects of the Classroom Resource on middle-school stu-
dents’ prosocial attitudes. The limited scope of this
investigation was constrained by practical considerations,
as the P3R Classroom Resource is the only schoolwide
intervention component and, at the time of the study, there
were not enough human resources available to sponsor
robust implementations of the other P3R component pro-
grams. The P3R Classroom Resource is an intervention
comprised of lessons designed to be implementable by
educators and flexibly integrated within classroom struc-
tures and schedules. Each lesson is planned to last
approximately 30–50 min and is accompanied by the P3R
Teacher’s Guide (Faull et al., 2008), which is a standard-
ized, semi-scripted curriculum that serves to structure each
lesson implementation. For each lesson, this guide provides
the teacher with a bullet-pointed outline of the lesson’s
core objectives, the materials needed to conduct the lesson,
and a detailed overview of the various classroom activities
to be carried out during the lesson.
All P3R lessons follow the same basic pedagogical
format, consisting of a few core components: (a) viewing
film segments portraying the initiation and development of
bullying relationships for one male and one female victim
within a middle-school context; (b) facilitating class dis-
cussions regarding these film segments—considering the
students’ interpersonal relationship dynamics, identifying
and working through related emotions, and employing
group problem-solving strategies to develop positive
behavioral solutions; and (c) providing psychoeducational
information before, during, and following class activities to
inform students of current school policy as well as supports
available to assist them in handling bullying situations in a
prosocial manner (e.g., seeking mediation via local school
administrators or counselors). Taken together, these lessons
provide youth with opportunities to cultivate greater
empathic responsiveness and prosocial cognitive-behav-
ioral habits by, first, exposing them to realistic portrayals of
bullying and its deleterious effects; second, facilitating a
safe space for students to explore their emotional, cogni-
tive, and behavioral reactions to these portrayals as well as
their real-life experiences; and, third, providing guidance
and supervision for students as they work individually and
collectively to synthesize their emotional, cognitive, and
behavioral experiences into prosocial responses that can be
enacted in the real-world. In this way, the Classroom
Resource lessons facilitate the development of empathy
and positive cognitive-behavioral habits through a combi-
nation of semi-structured reflective (i.e., past oriented),
experiential (i.e., present oriented), and forward-looking
(i.e., future oriented) activities that compel students to
confront and engage their emotions, cognitions, and
behavior. To compliment the classroom activities, each
lesson is accompanied by suggestions for optional home-
work assignments (none of which were employed in the
present study). Moreover, the P3R Teacher’s Guide con-
tains both five-lesson-plan and eight-lesson-plan versions
of the curriculum, allowing for more or less intensive
implementations, depending on need and context.
The P3R Classroom Resource is grounded in a trans-
actional-ecological model of youth’s development of
antisocial behavior, which views student development
within a contextual framework wherein all behaviors,
including negative ones, serve a functional purpose for the
individual—helping one better adapt to one’s environment
(Sameroff, 2000). The transactional-ecological model rec-
ognizes that youth development is a complex process,
comprised of macroregulations (i.e., environmental chan-
ges during key developmental periods), miniregulations
(i.e., daily interactions between the child and her caregiv-
ers, siblings, or peers, which shape the development of
behavior), and microregulations (i.e., brief interactions
between the child and caregiver during which their
responses are in sync; Sameroff, 1995). Given that this
model emphasizes the dynamic nature of development,
‘‘children are not viewed as sponges, merely soaking up
information, but rather as individuals who give and receive
information that subsequently influences future develop-
mental opportunities’’ (Jimerson et al., 2006, p. 10). Thus,
there is a reciprocal relationship between children and
contexts: contexts shape their behaviors and, in turn, their
behaviors reshape those contexts (Sameroff, 2009).
Grounded in a transactional-ecological perspective, the
P3R Classroom Resource aims to raise students’ awareness
regarding the functional nature of bullying-related behav-
iors, how those behaviors are shaped by youth’s contexts,
and how they negatively impact their and other’s well-
School Mental Health (2012) 4:115–128 119
123
being. In addition to establishing such awareness, as
mentioned above, the Classroom Resource also provides
youth with opportunities to develop empathy and cogni-
tive-behavioral skills that serve to prevent bullying-related
behavior, promote prosocial behaviors, and therefore
enhance the well-being of all students (Faull et al., 2008).
Thus, by promoting positive internal and external actions,
the Classroom Resource aims to alter youth’s develop-
mental trajectories toward more adaptive and healthier
outcomes.
Viewed in light of the scholarship regarding key char-
acteristics of effective bullying prevention programs (dis-
cussed above), P3R’s program features appear promising.
In particular, P3R’s film segments, class discussions, and
pscyhoeducational elements are designed to promote the
empathic involvement of students and emphasize the
importance of school connectedness (cf. Hazler & Carney,
2006), include cooperative groupwork elements that aim to
empower students with social problem-solving skills (cf.
Rigby, 2006), and provide engaging opportunities for stu-
dents to learn via an interactive, video-based medium (cf.
Ttofi et al., 2008). However, it is noteworthy that over-
arching intention of P3R is fundamentally different from
the majority of other current bullying prevention programs
on the market. While other programs seek to directly and
immediately reduce the prevalence of bullying behavior
and victimization present within a school, P3R aims to
raise awareness, enhance knowledge, and improve attitudes
that are likely to facilitate more positive peer relationships
and increased well-being at school. This scope appears
both appropriate and reasonable in light of previous find-
ings regarding the effectiveness and potential effects of
bullying programing (e.g., Merrell et al., 2008) as well as
developmental considerations regarding children’s cogni-
tive and behavioral maturation (Jimerson et al., 2010).
Research Design and Variables
The questions and associated hypotheses of this study were
investigated via a quasi-experimental, pretest–posttest
research design, in which the seventh-graders in the Health
classes served as the intervention group (n = 320) and the
eighth-graders in the other elective courses served as the
control group (n = 316). Overall, the study was designed
to test one main effect as well as one interaction effect: the
main effect of Group (i.e., intervention vs. control status)
and the interaction effect of Group 9 Duration (i.e.,
1-week vs. 5-week vs. 8-week implementation iterations,
as a function of group status). Thus, P3R and its various
duration iterations were conceptualized as the independent
variables, while two attitudinal outcomes were conceptu-
alized as the dependent variables: students’ general atti-
tudes toward bullying and their perceptions of school
bullying supports. Intervention feasibility and teacher
perceptions of social validity were also conceptualized as
independent variables; however, these outcomes were
linked with the teacher, not the students, and were therefore
only evaluated descriptively.
Measures
Bullying Attitudinal Scale: Short Form
The Bullying Attitudinal Scale (Song, Swearer, Haye, &
Bandalos, 2001) is one of the few existing measures for
assessing student attitudes related to bullying. It is com-
prised of 14 items arranged on a five-point Likert scale
(i.e., ‘‘totally false’’ to ‘‘totally true’’) and is scored by
summing the responses to produce a composite score, with
higher scores indicating more prosocial attitudes toward
bullying. At the time of the present study, limited psy-
chometric support for this measure was available, with only
two studies reporting acceptable levels of internal consis-
tency (Cronbach’s a = .71 and .78) and only one study
reporting a moderate, 1-year test–retest reliability coeffi-
cient (r = .54; Song et al.). Given the sparse psychometric
data, and considering that the BAS had yet to be used for
evaluating attitudinal change in response to intervention,
the scale was descriptively analyzed for problematic items.
As a result of this critical review, seven of the BAS items
were deemed to be potentially problematic, as they did not
appear to measure prosocial attitudinal change and there-
fore could not be expected to have an optimal response
(e.g., ‘‘I would be friends with a bully’’). Thus, these seven
items were deemed inappropriate for the purposes of the
present study and were selected out, while the seven
remaining items were retained and used as a single, short-
form measure: the Bullying Attitudinal Scale: Short Form
(BAS-SF).
Perceptions of School Bullying Supports Scale
Prior to this study, there were no available measures for
assessing students’ perceptions of their local school bul-
lying supports, a construct conceptualized as a proxy for
school climate regarding bullying issues. Thus, for the
purposes of this study, a brief 4-item measure was devel-
oped by the authors to assess this particular construct: the
Perceptions of School Bullying Supports Scale (PSBSS).
The few items comprising the PSBSS were designed to
have practical utility and face validity for evaluating the
underlying construct of student perceptions of school bul-
lying supports (i.e., perceptions of efforts, policies, and
relationships). Similar to the BAS-SF, each item was
arranged on the same five-point Likert scale, and the
responses were summed to produce a composite score, with
120 School Mental Health (2012) 4:115–128
123
higher scores indicating more positive perceptions of local
school climate regarding bullying. No pilot testing was
conducted with this measure prior to its use in the present
study.
P3R Implementation Fidelity Checklists
The P3R Implementation Fidelity Checklists (P3R-IFC)
were created by the authors for the present study to assess
the Health teacher’s implementation fidelity of P3R and to
serve as a proxy for intervention feasibility. Because dif-
ferent lesson plan versions were implemented during dif-
ferent trimesters, separate checklists were created for both
the five-lesson and eight-lesson versions. One checklist
was developed per lesson, consisting of brief restatements
of the briefing procedures, lesson activity components, and
debriefing components—summarized directly from the
P3R Teacher’s Guide (Faull et al., 2008). Directions on
each checklist instructed raters to mark ‘‘X’’ if the proce-
dure or component was observed during the intervention
session and ‘‘O’’ if the procedure or component was not
observed.
Intervention Rating Profile for Teachers: P3R Adaptation
Intervention social validity has generally been conceptu-
alized as a threefold evaluation of the social significance of
the goals, social appropriateness and feasibility of the
procedures, and social importance of the outcomes of an
intervention, as perceived by its consumers (i.e., imple-
menters or clients; Wolf, 1978). Given that P3R is a uni-
versal, social-cognitive intervention targeting groups of
students, none of the currently available self-report social
validity measures were deemed appropriate for assessing
its social validity. Thus, the Intervention Rating Profile for
Teachers, which has been deemed to have high internal
consistency (Cronbach’s a = .91) and one primary factor
loading (15-item loadings ranging from .82 to .95; Martens,
Witt, Elliot, & Darveaux, 1985), was modified by the
authors for use in the present study. The scale’s original
6-point Likert rating scale, ranging from ‘‘strongly dis-
agree’’ to ‘‘strongly agree,’’ was maintained, but item
wording was altered to align with the aims of the P3R
program (i.e., focus on attitudinal as opposed to behavioral
changes.) This new, modified measure was called the
Intervention Rating Profile for Teachers: P3R Adaptation
(IRP-P3R).
P3R Social Validity Questionnaire
To provide a more robust assessment of social validity, the
P3R Social Validity Questionnaire (P3R-SVQ) was created
by the authors to assess the Health teacher’s perceptions of
the goals, procedures, and outcomes of the P3R imple-
mentations. Specifically, the questionnaire was comprised
of eight open-ended questions that surveyed the teacher’s
perceptions of (a) the need for bullying intervention for the
target students, (b) the feasibility of intervention proce-
dures and durations, (c) the outcomes produced by the
interventions, and (d) her general compliments and criti-
cisms regarding the various intervention implementations
as well as the overall P3R intervention.
Intervention and Data Collection Procedures
Given that the P3R Classroom Resource was officially
adopted by the school’s administration as a mandatory part
of the exploratory Health class curriculum, the adminis-
tration chose to test three different iterations of the inter-
vention—one during each trimester—to see whether one
proved more effective than another, and thus to inform
their future usage of the curriculum. Such iteration testing
was amenable to the design of the Classroom Resource, as
the curriculum included two different lesson plan versions
within the Teacher’s Guide, both of which could be
implemented via various durations. The administration and
the authors decided, therefore, that the 5-lesson plan ver-
sion would be implemented during a 1-week duration,
followed by 5-lesson plan implementation during a 5-week
duration, and concluding with an 8-lesson plan imple-
mentation during an 8-week duration. This implementation
schedule would provide data on the comparative effec-
tiveness of approximately 1-week, 1-month, and 2-month
P3R interventions, respectively, and would run throughout
the academic school year.
During the fall trimester of 2008, the five-lesson plan
version of P3R was implemented in the exploratory Health
course, which consisted of four class periods comprised of
22–28 students each (n = 107). One lesson was adminis-
tered per day, Monday through Friday, and therefore the
entire intervention was completed within 1 week. Follow-
ing, during the winter trimester, the five-lesson plan ver-
sion of P3R was again implemented in the exploratory
Health course, with similar class sizes (n = 113). This
time, however, only one lesson was administered per week,
and thus the entire intervention was completed within
5 weeks. Finally, during the spring trimester, the eight-
lesson plan version of P3R was implemented in the
exploratory Health course, again with similar class sizes
(n = 100). Similar to the previous trimester, one lesson
was administered per week, and therefore the intervention
was completed in 8 weeks. Control-group students
(n = 105, 108, and 103 for the fall, winter, and spring
trimesters, respectively) received ‘‘treatment as usual’’
during the same time periods in their exploratory elective
courses. During each of these P3R Classroom Resource
School Mental Health (2012) 4:115–128 121
123
implementations, the Health teacher facilitated the inter-
vention using the Teacher’s Guide (described above). Prior
to each trimester’s implementation, the first author met
with the teacher, checked to see whether she was oriented
to the correct section of the curriculum that would be
administered during the present trimester and answered any
questions regarding procedures or content. The teacher
typically had few, if any, questions—reporting that the
Teacher’s Guide was self-explanatory and comprehensible.
As previously described, the present study employed a
quasi-experimental research design, with repeated-mea-
sures (i.e., pre-posttest) used to assess students’ general
attitudes toward bullying and perceptions of their local
school bullying supports. To accomplish this, all items
from the BAS-SF and the PSBSS were combined into a
single outcome measure—entitled Quick Questions About
Bullying—that was administered to both control-group and
intervention-group students the day prior to beginning the
intervention (i.e., pretest) as well as the day following the
completion of the intervention (i.e., posttest) during each
trimester. The teacher of the exploratory Health class
administered these assessments for the intervention-group
students, while the teachers of the various elective courses
administered these assessments for the control-group stu-
dents. During the course of the study, the control-group
elective courses consisted of art, keyboarding, and wood-
shop—none of which included curriculum related to social,
emotional, or behavioral development in youth. To
enhance administration fidelity of the surveys and prevent
against confounding teacher effects, all teachers were
provided with a one-page, scripted handout consisting of an
administration protocol, which was reviewed with them by
the first author. The teachers reported that they followed
the protocol appropriately; however, no observational
implementation fidelity was taken. For both pretest and
posttest administrations, and across control and interven-
tion groups, all students completed these assessments in
less than 10 min. Following each test administration (i.e.,
pre and post) during each trimester, the assessments were
retrieved from all participating teachers and entered into an
electronic database.
To measure implementation fidelity and therefore feasi-
bility of the P3R, the Health teacher was provided with the
appropriate P3R-IFC for each implementation, depending
on the duration (i.e., 5-lesson or 8-lesson versions),
and completed self-ratings immediately following each
intervention session. Given that four classes received the
intervention each trimester, the teacher completed 20
checklists the first trimester (for five lessons), 20 more the
second trimester (for five lessons), and 32 the last trimester
(for eight lessons)—resulting in a total of 72 self-completed
fidelity checklists. To enhance the validity of the teacher’s
self-reports, the first author conducted implementation
fidelity observations using the P3R-IFC for approximately
25% of the intervention lessons during each trimester.
Moreover, following the conclusion of all interventions, the
Health teacher completed the IRP-P3R and the P3R-SVQ.
Data Analyses
Preliminary Analyses
Given that the BAS-SF and PSBSS were original with this
study, preliminary analyses were conducted to explore their
initial psychometric properties, using a subsample of 105
students from the first-trimester control group. One-week
test–retest reliability was assessed for each of the seven
items comprising the BAS-SF and each of the four items
comprising the PSBSS, to ensure that the items amassing
both composites were tenable. Following, test–retest reli-
ability was also assessed for the BAS-SF and PSBSS
composites. Internal consistency analyses were also con-
ducted for the items comprising each measure, to ensure
that it represented a unified underlying construct. Finally,
to further confirm the reliability results, exploratory factor
analyses—employing the Maximum Likelihood extraction
method with a Promax (oblique) rotation—were conducted
to determine the statistical dimensionality of both
measures.
Beyond these analyses, the appropriateness of the pro-
jected usage of two-way factorial ANCOVA for some of
the primary analyses was investigated by checking the
assumptions for this particular analysis against the
parameters of the current data set. Also, given that two
separate ANCOVA analyses were projected, a Bon-
feronni’s adjustment was performed to prevent inflation of
family-wise error, and power analyses were carried out to
ascertain whether sufficient power would be obtained to
identify statistically significant effects, if present.
Primary Analyses
The primary analyses of this study were conducted to
investigate the three aforementioned research questions.
Implementation fidelity for each intervention iteration was
assessed via a two-stage quantitative descriptive analysis:
first, fidelity for each lesson was assessed via summing the
number of implemented components and dividing that by
the total number of components per lesson, resulting in a
percentage; second, a fidelity subtotal for each iteration
implementation was ascertained and reported by summing
all of the resulting percentages from each lesson within
each duration iteration and then determining the mean
percentage for each. Moreover, teacher perceptions of
social validity were assessed by quantitative and qualitative
122 School Mental Health (2012) 4:115–128
123
means: first, all responses to items on the IRP-P3R were
summed to produce a composite score, which was then
judged against an associated descriptor indicating a level of
perceived social validity (e.g., composite scor-
e [75 = high social validity); second, a general qualitative
descriptive analysis of the content of the teacher’s
responses to the P3R-SVQ was used to provide evidence
for or against positive perceptions of the goals, procedures,
and outcomes of the three intervention iterations as well as
the overall intervention.
To assess the general intervention efficacy as well as
possible dosage-response effects on students’ general atti-
tudes toward bullying, an ANCOVA was conducted using
the BAS-SF pretest composite group means as a covariate
(i.e., to neutralize any confounding variables that may have
effected baseline reports between seventh- and eighth-
graders or between students in the health class and those in
the other exploratory classes), the BAS-SF posttest com-
posite group means as a dependent variable, and the Group
(i.e., control or intervention) and Duration (i.e., 1-, 5-,
8-week) statuses as the between-subjects fixed-factor
independent variables. Specifically, a Group main effect
and a Duration 9 Group interaction effect were evaluated
for potentially statistically significant difference between
BAS-SF posttest means, followed by a descriptive mean
evaluation to confirm the directionality of any noted effect
as well as an effect size analysis to determine the educa-
tional import of the effect. Similarly, to assess the general
intervention efficacy as well as possible dosage-response
effects on students’ perceptions of their school’s bullying
supports, a separate ANCOVA was conducted using the
PSBSS pretest composite group means as a covariate, the
PSBSS posttest composite group means as a dependent
variable, and the Group and Duration statuses as the
between-subjects fixed-factor independent variables.
Again, a Group main effect and a Duration 9 Group
interaction effect were evaluated for potentially statistically
significant difference between PSBSS posttest means, fol-
lowed by a descriptive mean evaluation to confirm the
directionality of any noted effect as well as an effect size
analysis to determine the educational import of the effect.
Results
Preliminary Outcomes
Psychometric Explorations
For both the BAS-SF and the PSBSS, results from the test–
retest reliability analyses revealed at least moderate, sta-
tistically significant bivariate correlations for all items,
r [ .50, as well as both composites, r [ .65, suggesting
adequate overall reliability for attitudinal measurements
that may be somewhat inherently variable. Furthermore,
the internal consistency analysis indicated acceptable
results for both the BAS-SF and the PSBSS, Cronbach’s
a = .88 and .73, respectively, suggesting each composite
adequately represents a single underlying attitudinal con-
struct. Moreover, results from the BAS-SF exploratory
factor analysis indicated that only one factor was extracted,
which accounted for approximately 50% of the total vari-
ance (Eigen value = 3.94) and had moderate to high
loadings across all seven items, [.55, suggesting unidi-
mensionality. Similarly, results from the PSBSS explor-
atory factor analysis indicated that only one factor was
extracted, which accounted for approximately 42% of the
total variance and had moderate to high loadings across all
four items, [.57, also suggesting unidimensionality. Taken
together, results from these analyses suggest that the BAS-
SF and the PSBSS had adequate psychometric properties to
justify using their composite scores as representations of
attitudinal constructs.
ANCOVA Assumption Checks
All ANCOVA assumptions—including (a) independence
of observations, (b) homogeneity of variance, (c) multi-
variate normality across each group distribution, (d) ade-
quate and equal group sample sizes, (e) low measurement
error for the covariate, (f) a linear relationship between the
covariate and the dependent variable, (g) homogeneity of
covariate regression coefficients, (h) lack of substantial
covariate outliers, (g) general additivity, and (h) indepen-
dence of the error term—were checked and deemed to be
met or, if not met, deemed to be robust in spite of such
violations. For example, given that the present study was
quasi-experimental and thus lacked random group assign-
ment, a significant Group main effect was found at baseline
for both the BAS-SF composite distribution, F(1) = 48.27,
p \ .001, and the PSBSS composite distribution,
F(1) = 18.41, p \ .001. Thus, the error term could not be
considered independent of the covariates and the categor-
ical independent variables because systematic, statistically
significant difference was identified between the groups at
baseline. Nevertheless, despite this violation, ANCOVA
remains the most appropriate analysis for repeated-mea-
sures, pre-post quasi-experimental designs (cf. Tabachnick
and Fidell 1996) and was thus considered appropriate for
the primary analyses.
Bonferonni’s Adjustment and Power Analyses
Bonferonni’s adjustment, which divided the original sig-
nificance level, a = .05, by the total number of planned
analyses (i.e., two), resulted in a modified significance
School Mental Health (2012) 4:115–128 123
123
level of a = .025. Considering this adjusted significance
level with the total number of groups within each projected
analysis (i.e., six), and assuming that at least a moderate
effect size (partial g2 C .06) would be obtained for each
analysis, Power analyses indicated that all current group
sample sizes, n C 100, were more than adequate for
securing adequate power, C.80, for all F-tests.
Primary Outcomes
Intervention Feasibility and Social Validity
Results from the quantitative descriptive analyses indicated
that the teacher’s mean self-reported fidelity rating for
proportion of intervention components administered per
lesson was high: 87% for the 1-week implementation, 92%
for the 5-week implementation, 95% for the 8-week
implementation, and 91% for total lesson administrations.
The trustworthiness of these ratings was corroborated via
high inter-rater reliability established between the teacher’s
and independent observer’s fidelity ratings, Cohen’s
j = .90, suggesting adequate overall implementation
fidelity. Furthermore, the teacher’s total IRP-P3R com-
posite score was 79, putting the average item response in
the ‘‘agree’’ to ‘‘strongly agree’’ range, M = 5.27, indi-
cating a very favorable perception of the overall social
validity of P3R as a classroom intervention within the
particular school and classroom context. Moreover, the
teacher’s qualitative responses on the P3R-SVQ confirmed
her very favorable perception of goals, procedures, and
outcomes of the P3R intervention. Specifically, she
remarked that (a) students in her classes needed the inter-
vention because they lacked awareness and knowledge
related to bullying dynamics; (b) the overall intervention
was feasibly implemented within her classes under typical
classroom conditions; (c) the content of the intervention
was perceived as realistic by both herself and the students;
and (d) P3R appeared to enhance students’ mindfulness of
their actions, words, and attitudes related to bullying—as
evidence by informal observations of changes in student
social interactions in the classroom during and following
the intervention. Furthermore, the Health teacher also
remarked on the P3R-SVQ that (e) all intervention dura-
tions seemed equally feasible to implement and that (f) she
did not perceive any dosage-response effects—or differ-
ences in student attitude or behavior changes—between the
1-, 5-, and 8-week iterations.
General Intervention Efficacy
Results from the two-way factorial ANCOVA using the
BAS-SF pretest scores as a covariate and the BAS-SF
posttest scores as the dependent variable indicated a
statistically significant Group main effect, F(1) = 9.93,
p = .002, with a small effect size, partial g2 = .02. A
descriptive evaluation of the resulting estimated marginal
means from this analysis indicated that the total posttest
marginal mean for the intervention group was approxi-
mately two mean points higher than the total posttest
marginal mean for the control group, indicating positive
improvements in intervention-group students’ general
attitudes toward bullying. Moreover, results from the two-
way factorial ANCOVA using the PSBSS pretest scores as
a covariate and the PSBSS posttest scores as the dependent
variable also revealed a significant Group main effect,
F(1) = 7.25, p = .007, with a small effect size, partial
g2 = .01. A descriptive evaluation of the resulting esti-
mated marginal means from this analysis indicated that the
total posttest marginal mean for the intervention group was
approximately one mean point higher than the total posttest
marginal mean for the control group, indicating positive
attitudinal improvements in intervention-group students’
perceptions of their school’s bullying supports.
Dosage-Response Effects
Results from the same two-way factorial ANCOVAs
described above, using the BAS-SF and PSBSS pretest
scores as covariates and the BAS-SF and PSBSS posttest
scores as the dependent variables, indicated nonsignificant
Group 9 Duration interaction effects for both general
attitudes toward bullying, F(2) = .67, p = .51, as well as
perceptions of the school’s bullying supports, F(2) = .85,
p = .43, which suggests no dosage-response effects were
observed for either outcome measure (see Table 1 for a
summary of all primary ANCOVA results).
Discussion
Interpretation of Results
The present study investigated the feasibility and social
validity, general intervention efficacy, and dosage-response
effects of teacher-implemented P3R Classroom Resource
Table 1 Summary of primary ANCOVA results
Measure Source df F p g2
BAS-SFa Group 1 9.93 .002 .02
Group 9 duration 2 .67 .51 .00
PSBSSb Group 1 7.25 .007 .01
Group 9 duration 2 .85 .43 .00
a Bullying attitudinal scale—short formb Perceptions of school bullying supports scale
124 School Mental Health (2012) 4:115–128
123
interventions on attitudinal outcomes of junior-high school
students. Findings from the analyses of resulting data
suggest that the intervention was feasible for the Health
teacher to implement independently and that it was socially
valid within the particular school and classroom contexts—
as evidenced by high implementation fidelity and positive
teacher perceptions of P3R’s goals, procedures, and out-
comes across implementations. Moreover, findings from
this study suggest that P3R marginally enhanced students’
attitudes toward bullying as well as their perceptions of
their school’s bullying supports, resulting in more prosocial
attitudes overall—as evidenced by statistically significant
changes in mean attitudinal scores accompanied by small
effect sizes. Furthermore, findings from this study suggest
no dosage-response effects among the 1-, 5-, and 8-week
iterations of P3R, indicating relatively equivalent effec-
tiveness of implementations—as evidenced by statistically
non-significant differences in mean attitudinal scores
across durations. Taken together, these findings suggest
that, as used in the present study, the P3R Classroom
Resource was feasibly implemented, socially valid, and
marginally effective as an intervention for enhancing stu-
dents’ general attitudes toward bullying as well as their
attitudes toward their school’s bullying supports.
As noted previously herein, the P3R Classroom
Resource differs from the majority of mainstream uni-
versal-level bullying interventions (cf. Jimerson & Huai,
2010), as it is amenable to implementation within smaller-
scale and briefer implementation periods (e.g., 1 week to
2 months), is characterized by low-resource intensiveness
(i.e., requires little involvement from outside support staff
as well as little training and support materials), is amenable
to investigation of isolated intervention components (i.e.,
classroom-based intervention only), and is characterized by
an orientation toward prosocial behavior and attitude
development (i.e., cultivation of knowledge, empathy, and
problem-solving skills). Considering this, the positive
findings resulting from the present study enhance the
warrant for further research into a new wave of bullying
intervention and prevention programing, which, similar to
P3R, can be feasibly implemented by educators at the local
site level and still facilitate positive student outcomes.
Intervention efforts of this nature are likely to be highly
sustainable (cf. Larson & Busse, 2006), but much empirical
research will be needed to prove their efficacy.
Nevertheless, given the current dearth of studies inves-
tigating the feasibility and social validity of bullying
interventions (Olweus, 2010), findings from the present
study are encouraging: providing a case example of a
teacher who accurately and effectively implemented P3R
amidst her typical school duties. Yet given the dearth of
studies investigating possible dosage-response effects for
bullying interventions (Merrell et al., 2008), findings from
this study are essentially neutral: providing no evidence for
preferring shorter implementation durations (i.e., 1-week)
over longer ones (i.e., 5-, 8-week). Moreover, given the
preponderance of evidence indicating only marginal
changes in student outcomes resulting from bullying
interventions (e.g., Smith et al., 2004), findings from the
present study serve to strengthen the status quo of student
outcomes: providing another example of small, positive
intervention effects on students’ social attitudes. Given that
the content and method of P3R is comprised of several
empirically promising program features (e.g., video-based
curriculum, social problem-solving orientation, empathic
involvement of students; e.g., Hazler & Carney, 2006), this
last finding may seem surprising. However, a reconsider-
ation of the bullying intervention literature suggests at least
four possible limitations that may account for such mar-
ginal effects: (a) an internal-potency limitation of the
intervention itself; (b) an overall dosage-response limita-
tion; (c) a problematic measurement limitation; and
(d) sampling bias and research design limitations.
Limitations and Future Research
Internal-Potency
The effectiveness of the P3R Classroom Resource to
enhance students’ prosocial attitudes may have been lim-
ited by its internal-potency, meaning that the intervention’s
contents and methods may have not been potent enough to
facilitate more remarkable student outcomes. A critical
evaluation of P3R’s content and methods in relation to the
literature regarding effective characteristics of bullying
programs suggests that although its method is characterized
by several empirically promising features (e.g., video-
based instruction and a social problem-solving orientation),
its content is lacking one fundamental component that has
been associated with stronger student outcomes: direct
social-skills instruction (cf. Hazler & Carney, 2006). Thus,
the potency of P3R may be limited by the exclusion of a
skills acquisition component. To test this hypothesis, future
studies could investigate the effectiveness of the interven-
tion as is—or without social-skills training—compared to
the effectiveness of the intervention enhanced—or with a
social-skills training component. However, by adding a
social-skills training component to P3R, the program is
likely to become more similar to other preexisting bullying
interventions and may, as a result, become more difficult
for general educators and other school-based practitioners
to implement accurately and independently. Recognizing
that ‘‘any method, regardless of its empirical merit, is only
as valid as the probability that school-based personnel will
use it in a reliable and effective manner’’ (Scott et al., 2004,
p. 190), we recommend that future research seek to strike a
School Mental Health (2012) 4:115–128 125
123
balance between empirical rigor and social validity—
aiming for sustainability at the local site level.
Dosage-Response
The present study may also be limited by an overall dos-
age-response threshold. Although part of the original
intention of the study was to investigate various brief
dosage-response effects (i.e., 1-, 5-, 8-week), it is possible
that each intervention duration was too brief to facilitate
more remarkable student outcomes, failing to meet a dos-
age-response threshold. Thus, the same intervention, if
administered over longer periods of time—such as
3–5 months, as opposed to 1 or 2 months—may be capable
of facilitating more remarkable student outcomes. This
hypothesis is supported by a large body of educational
research indicating the significant role of duration in
enhancing the effectiveness of interventions (e.g., Zelezny,
1999); however, it remains largely uninvestigated as per-
taining to bullying interventions (Olweus, 2010). Thus, we
recommend that future research is warranted to investigate
the possibility of a duration-response threshold for the P3R
Classroom Resource and similar bullying interventions,
which may require stretching their recommended imple-
mentation periods.
Problematic Measurement
Another possible limitation of the present study is prob-
lematic measurement. In the present study, post hoc anal-
yses revealed that baseline results from both measures
revealed significantly non-normal, positively skewed dis-
tributions. Such results suggest substantial ceiling effects
for baseline responding, in which large proportions of
students were already responding prosocially to both out-
come indicators prior to receiving the intervention—
resulting in a significantly lower likelihood of identifying
statistically significant results with moderate to large effect
sizes at posttest (cf. McHorney & Tarlov, 1995). Thus,
although it is possible that students in the present study
already possessed extremely prosocial attitudes toward
bullying and positive perceptions of their local school
bullying supports at baseline, it is also possible that the
suggested measurement ceiling effect is attributable to a
social-desirability response effect. Given all of the items in
both measures used some form of the word ‘‘bullying,’’
which carries an inherently negative connotation, students
may have been primed to respond in an overly prosocial
manner at baseline. If this was the case, it could have
blurred the descriptive comparison between posttest
responses, which may have been more honest or accurate
than those at baseline and thus lessened the integrity of
construct validity (cf. Furlong, Sharkey, Felix, Tanigawa,
& Green, 2010). Given that concerns with problematic
measurement have been described as the ‘‘Achilles heel of
bullying prevention efforts’’ (Cornell, Sheras, & Cole,
2006, p. 193), we recommend that future research is war-
ranted to develop psychometrically sound measures with
high comparative and practical utility.
Sampling Bias and Research Design
Finally, a general sampling bias and other unintended
research design features may have also limited the present
study. For example, given that this study utilized the nat-
ural classroom groupings of students to serve as the bases
for determining group membership, it is possible that such
nonprobabilistic or convenience sampling may be con-
founded by other characteristics unique to students in the
Health class intervention group (cf. Heppner, Wampold, &
Kivlighan, 2007). Specifically, because the P3R Classroom
Resource was presented in a Health class, it is possible that
class content or activities presented prior to or in con-
junction with P3R may have confounded the intervention
effects. Although such sampling bias is undesirable, it was
largely unavoidable in the present study, as the school’s
administration selected the intervention for inclusion with
the exploratory Health class only. Nevertheless, future
studies within similar lines of research may compensate for
this possible confound by arranging for implementations of
P3R or other bullying interventions across multiple class-
room settings (e.g., Mathematics, Language Arts, and
Technology), thereby creating a categorical classroom
variable that can be investigated as a random-effect within
a larger general linear model (cf. Hancock & Mueller,
2010). Moreover, given that the present study lacked fol-
low-up measurements of students’ attitudes post-interven-
tion, we recommend that future bullying intervention
studies utilize longitudinal research designs to test whether
student outcomes decline, maintain, or improve following
intervention paired with typical developmental maturation.
Implications for Practitioners
A couple of important implications for school-based
practice can be derived from the present study. First,
findings from the present study suggest that classroom
teachers and, by extension, other educational professionals
(e.g., school psychologists) may be capable of imple-
menting P3R with high fidelity and resulting in high social
validity as a brief, universal-level bullying intervention.
However, because only one teacher’s implementations
were evaluated in the present study, more studies with
more teachers in different contexts are needed prior to
making any substantive generalizations. Second, findings
from the present study suggest that P3R had positive
126 School Mental Health (2012) 4:115–128
123
effects on students’ prosocial attitudes regarding bullying
and school climate and thus may be an empirically prom-
ising school-based intervention. Nevertheless, because this
is one of the first empirical investigations of P3R and only
small effect sizes resulted in the present study, more
studies with more compelling evidence are needed before
P3R can be recommended as an evidence-based practice.
Moreover, given current educational policy and the general
scientific zeitgeist regarding selecting best practices (e.g.,
Slavin, 2002), if P3R is chosen for school-based use as a
pilot intervention, we strongly recommend that school-
based practitioners inform all involved parties of the status
of its current empirical validation.
Conclusion
The present study investigated the effects of a new, brief,
universal-level bullying intervention—the P3R Classroom
Resource. Findings indicated that the intervention had small,
positive effects on student attitudes and was feasibly
implemented by the classroom teacher, with high percep-
tions of social validity. Given the preliminary nature of these
findings, we hope that scholars and school-based practitio-
ners will collaborate in validating P3R and other similar
bullying interventions—continually striving to balance
empirical rigor with social validity in pursuit of enhancing
youth’s mental health and well-being through school-based
prevention and intervention efforts. Ultimately, we hope that
efforts in this line of applied research will increasingly move
toward developing and evaluating bullying interventions
that are sustainable at the local site level—empowering
school communities with the tools and skills necessary to
efficiently prevent bullying-related problems.
References
Alsaker, F. D., & Gutzwiller-Helfenfinger, E. (2010). Social behavior
and peer relationships of victims, bully-victims, and bullies in
kindergarten. In S. R. Jimerson, S. M. Swearer, & D. L. Espelage
(Eds.), Handbook of bullying in schools (pp. 87–100). New
York, NY: Routledge.
Breit-Smith, A., Justice, L. M., McGinty, A. S., & Kaderavek, J.
(2009). How often and how much? Intensity of print referencing
intervention. Topics in Language Disorders, 29, 360–369.
Cornell, D., Sheras, P. L., & Cole, J. C. (2006). Assessment of
bullying. In S. R. Jimerson & M. J. Furlong (Eds.), Handbook ofschool violence and school safety: From research to practice(pp. 191–210). Mahwah, NJ: Erlbaum.
DioGuardi, R. J., & Theodore, L. A. (2006). Understanding and
addressing peer victimization among students. In S. R. Jimerson
& M. J. Furlong (Eds.), Handbook of school violence and schoolsafety (pp. 339–352). Mahwah, NJ: Erlbaum.
Espelage, D. L., & Swearer, S. M. (Eds.). (2004). Bullying inAmerican schools. Mahwah, NJ: Erlbaum.
Faull, C., Swearer, S. M., Jimerson, S. R., Espelage, D. L., & Ng, R.
(2008). Promoting positive peer relationships: Middle school
bullying prevention program—Classroom resource. Adelaide,
Australia: Readymade Productions.
Furlong, M. J., Pavelski, R., & Saxton, J. (2002). The prevention of
school violence. In S. E. Brock, P. J. Lazarus, & S. R. Jimerson
(Eds.), Best practices in school crisis prevention and interven-tion (pp. 131–149). Bethesda, MD: National Association of
School Psychologists.
Furlong, M. J., Sharkey, J. D., Felix, E., Tanigawa, D., & Green, J. G.
(2010). Bullying assessment: A call for increased precision of
self-reporting procedures. In S. R. Jimerson, S. M. Swearer, & D.
L. Espelage (Eds.), Handbook of bullying in schools (pp.
329–346). New York, NY: Routledge.
Hancock, G. R., & Mueller, R. O. (2010). The reviewer’s guide toquantitative methods in the social sciences. New York, NY:
Routledge.
Hazler, R. J., & Carney, J. V. (2006). Critical characteristics of
effective bullying prevention programs. In S. R. Jimerson & M.
J. Furlong (Eds.), Handbook of school violence and school safety(pp. 275–292). Mahwah, NJ: Erlbaum.
Heppner, P. P., Wampold, B. E., & Kivlighan, D. M. (2007).
Research design in counseling (2nd ed.). New York, NY:
Brooks.
Jimerson, S. R., & Huai, N. (2010). International perspectives on
bullying prevention and intervention. In S. R. Jimerson, S.
M. Swearer, & D. L. Espelage (Eds.), Handbook of bullying inschools: An international perspective (pp. 571–592). New York,
NY: Routledge.
Jimerson, S. R., Morrison, G. M., Pletcher, S. W., & Fulrong, M. J.
(2006). Youth engaged in antisocial behaviors: Who are they? In
S. R. Jimerson & M. J. Furlong (Eds.), Handbook of schoolviolence and school safety (pp. 3–20). Mahwah, NJ: Erlbaum.
Jimerson, S. R., Swearer, S. M., & Espelage, D. L. (Eds.). (2010).
Handbook of bullying in schools. New York, NY: Routledge.
Kosciw, J. G., Greytak, E. A., Diaz, E. M., & Bartkiewicz, M. J.
(2010). The 2009 school climate survey: The experiences oflesbian, gay, bisexual, and transgender youth in our nation’sschools. Washington, DC: Gay, Lesbian and Straight Education
Network. Retrieved from http://www.glsen.org.
Larson, J., & Busse, R. T. (2006). A problem-solving approach to
school violence prevention. In S. R. Jimerson & M. J. Furlong
(Eds.), Handbook of school violence and school safety: Fromresearch to practice (pp. 73–88). Mahwah, NJ: Erlbaum.
Martens, B. K., Witt, J. C., Elliot, S. E., & Darveaux, D. X. (1985).
Teacher judgments concerning the acceptability of school-based
interventions. Professional psychology: Research and practice,16, 191–198.
McHorney, C. A., & Tarlov, A. R. (1995). Individual-patient
monitoring in clinical practice: Are available health status
surveys adequate? Quality of Life Research, 4, 293–307.
Merrell, K. W., Guelder, B. A., Ross, S. W., & Isava, D. M. (2008).
How effective are school bullying intervention programs? A
meta-analysis of intervention research. School PsychologyQuarterly, 23, 26–42.
Murray-Harvey, R., Slee, P. T., & Taki, M. (2010). Comparative and
cross-cultural research on school bullying. In S. R. Jimerson, S.
M. Swearer, & D. L. Espelage (Eds.), Handbook of bullying inschools (pp. 35–48). New York, NY: Routledge.
Nansel, T. R., Overpeck, M., Pilla, R. S., Ruan, W. J., Simons-
Morton, B., & Scheidt, P. (2001). Bullying behaviors among US
youth: Prevalence and association with psychosocial adjustment.
Journal of the American Medical Association, 285, 2094–2100.
Olweus, D. (2010). Understanding and researching bullying. In S.
R. Jimerson, S. M. Swearer, & D. L. Espelage (Eds.), Handbookof bullying in schools (pp. 9–34). New York, NY: Routledge.
Reynolds, R. J., Temple, J. A., Robertson, D. L., & Mann, E. A.
(2001). Long-term effects of an early childhood intervention on
School Mental Health (2012) 4:115–128 127
123
educational achievement and juvenile arrest. The Journal of theAmerican Medical Association, 285, 2339–2346.
Rigby, K. (2006). What we can learn from evaluated studies of
school-based programs to reduce bullying in schools. In S.
R. Jimerson & M. J. Furlong (Eds.), Handbook of schoolviolence and school safety (pp. 335–338). Mahwah, NJ:
Erlbaum.
Rigby, K., & Bauman, S. (2010). How school personnel tackle cases
of bullying: A critical examination. In S. R. Jimerson, S.
M. Swearer, & D. L. Espelage (Eds.), Handbook of bullying inschools: An international perspective (pp. 455–468). New York,
NY: Routledge.
Sameroff, A. J. (1995). General systems theories and developmental
psychopathology. In D. Cicchetti & D. Cohen (Eds.), Develop-mental psychopathology (pp. 659–689). New York, NY: Wiley.
Sameroff, A. J. (2000). Dialectical processes in developmental
psychopathology. In A. J. Sameroff, M. Lewis, & S. M. Miller
(Eds.), Handbook of developmental psychopathology (2nd ed.,
pp. 23–40). New York, NY: Plenum.
Sameroff, A. J. (Ed.). (2009). The transactional model of develop-ment: How children and contexts shape each other. Washington,
DC: American Psychological Association.
Scott, T. M., Bucalos, A., Liaupsin, C. J., Nelson, C. M., Jolivette, K.,
& DeShea, L. (2004). Using functional behavioral assessment in
general education settings: Making a case for effectiveness and
efficiency. Behavioral Disorders, 29, 189–201.
Slavin, R. E. (2002). Evidence-based education policies: Transform-
ing educational practice and research. Educational Researcher,31(2), 15–21.
Smith, J. D., Schneider, B. H., Smith, P. K., & Ananiadou, K. (2004).
The effectiveness of whole-school antibullying programs: A
synthesis of evaluation research. School Psychology Review, 33,
547–560.
Song, S., Swearer, S. M., Haye, K. M., & Bandalos, D. L. (2001,
August). Development of a comprehensive survey for bullyingand victimization: Psychometric analyses. Poster presented at
the annual conference of the American Psychological Associa-
tion, San Francisco, CA.
Spivak, H., & Prothrow-Stith, D. (2001). The need to address bullying:
An important component of school violence prevention. Journalof the American Medical Association, 285, 2131–2132.
Swearer, S. M., Grills, A. E., Haye, K. M., & Cary, P. T. (2004).
Internalizing problems in students involved in bullying and
victimization: Implications for intervention. In D. L. Espelage &
S. M. Swearer (Eds.), Bullying in American schools: A social-ecological perspective on prevention and intervention (pp.
63–84). Mahwah, NJ: Erlbaum.
Tabachnick, B. G., & Fidell, L. S. (1996). Using multivariatestatistics (3rd ed.). New York, NY: Harper Collins.
Ttofi, M. M., Farrington, D. P., & Baldry, A. C. (2008). Effectivenessof programmes to reduce school bullying: A systematic review.
The Swedish National Council for Crime Prevention. Retrieved
from http://www.bra.se/extra/faq/?module_instance=2&action=
question_show&id=474&category_id=9.
Wolf, M. M. (1978). Social validity: The case for subjective
measurement or how applied behavior analysis is finding its
heart. Journal of Applied Behavior Analysis, 11, 203–214.
Zelezny, L. C. (1999). Educational interventions that improve
environmental behaviors: A meta-analysis. The Journal ofEnvironmental Education, 31, 5–14.
128 School Mental Health (2012) 4:115–128
123
