The Social Science Journal 46 (2009) 671–688

Available online at www.sciencedirect.com

Effect of relational bullying on attitudes, behavior andinjury among adolescent bullies,

victims and bully-victims

Richard L. Dukes a,∗, Judith A. Stein b, Jazmin I. Zane c

a Department of Sociology, University of Colorado-Colorado Springs,1420 Austin Bluffs Parkway, Colorado Springs, CO 80918, USA

b Department of Psychology, University of California, Los Angeles, USAc Department of Social Welfare, University of California, Los Angeles, USA

Received 6 October 2008; received in revised form 2 May 2009; accepted 10 May 2009

Abstract

This study examined whether dysfunctional correlates of relational bullying parallel those that havebeen found for physical bullying. We used data from the population of 2,494 respondents in grades7–12 from a school district in Colorado to compare latent variables of self-esteem, attitudes towardschool, delinquent behavior, physical injury, physical bullying, and physical victimization among 291relational bullies, 303 victims, 213 bully-victims, and 1,687 neutrals. We used gender as a controlvariable. Neutrals had the best school attitudes, least problem behavior, least injury, and were lesslikely to be physical bullies or victims. Bully-victims reported worse outcomes on the latent vari-ables than either neutrals or victims, and they were more likely to be physically victimized thanbullies. These outcomes parallel findings among physical bullies in a similar sample. Results showthe need for improved intervention, heightened awareness, and particular attention to relational bully-victims.© 2009 Western Social Science Association. Published by Elsevier Inc. All rights reserved.

Bullying among adolescents has raised public concern both nationally and internation-ally, and it is considered to be an abusive, sometimes violent form of harassment (Crawford,2007; Gini, 2004; Olweus, 2003; Yang, Kim, Kim, Shin, & Yoon, 2006). Involvementin bullying, whether as a bully or a victim, is associated with a wide array of negative

∗ Corresponding author. Tel.: +1 719 598 6277; fax: +1 719 262 4450.E-mail address: rdukes@uccs.edu (R.L. Dukes).

0362-3319/$ – see front matter © 2009 Western Social Science Association. Published by Elsevier Inc. All rights reserved.doi:10.1016/j.soscij.2009.05.006

672 R.L. Dukes et al. / The Social Science Journal 46 (2009) 671–688

outcomes. These outcomes include difficulty with rules, negative attitudes about school,poor school adjustment, excessive substance and alcohol use, physical injury, depres-sion, psychiatric symptoms later in life, and suicide ideation (Kaltiala-heino, Rimpela,Rantanen, & Rimpela, 2000; Klomek, Marrocco, Kleinman, Schonfeld, & Gould, 2007;Kumpulainen & Räsänen, 2000; Nansel et al., 2004; Olweus, 1993; Stein, Dukes, & Warren,2007).

Bullying is characterized by three defining conditions. First, bullying involves negative ormalicious behavior intended to cause harm or distress. Second, this behavior occurs repeatedlyover time. Third, the relationship between the parties involved in bullying is characterized byan imbalance in strength or power (American Psychological Association, 2005; Gini, 2004;Nansel et al., 2001; Olweus, 1993).

Two types of bullying have been identified. Direct bullying involves hitting, kicking or otherforms of physical aggression. Indirect bullying includes repeated teasing or verbal insults,circulation of rumors, or exclusion from a peer-group (Olweus, 1993, 2003). Indirect bullyingis also known as relational bullying, and it is the focus of the current paper.

1. Relational bullying

Whereas relational bullying is not as overt as direct bullying, it is insidious, and itcan have long-term negative effects on attitudes and behavior of both bullies and vic-tims (Crick & Grotpeter, 1995; Crozier & Skliopidou, 2002; Newman, Holden, & Delville,2005). The Virginia Tech and Columbine shootings have highlighted the importance ofinvestigating relational bullying among children and adolescents. Reports indicate that priorto the shootings, the perpetrators at both the Virginia Tech and Columbine had experi-enced teasing and mocking from their peers (Crawford, 2007; Kleinfield, 2007; Rook,2007).

Victims of name-calling have reported higher rates of anger, embarrassment, shame, andunhappiness. Often these victims have coped with relational bullying by withdrawing fromfriendships and school activities (Crozier & Skliopidou, 2002). Although these feelings dimin-ished with time, individuals who reported higher rates of being bullied tended to have long-termpersonality problems, a linkage that could impact school violence. According to the SafeSchool Initiative of the Secret Service (Vossekuil, Reddy, & Fein, 2001), school shootingsare not impulsive; they are planned beforehand, sometimes over a long period. Furthermore,it was common for school shooters to have experienced bullying and harassment that waslongstanding and severe (Vossekuil et al., 2001).

Relational bullying tends to occur more among girls (Raskauskas & Stoltz, 2004), whereasphysical bullying occurs more among boys (Björkqvist, Lagerspetz, & Kaukiainen, 1992).However, it is possible that this gender difference is due to underreporting by boys. Boys aremore likely to manifest violent behavior in response to long-term emotional bullying. Crozierand Skliopidou (2002) found that adults who reported feeling the “most hurt” by relationalbullying were those who experienced bullying at an early age, and who also endured name-calling and physical bullying. Hence, examining correlates of relational bullying may revealimportant aspects to target for intervention for both genders.

R.L. Dukes et al. / The Social Science Journal 46 (2009) 671–688 673

2. Bully group classification

Four distinct types of bullying groups have been studied (Haynie et al., 2001; Schwartz,2000; Stein et al., 2007; Woods & White, 2005). Pure bullies are individuals who bully otherchildren but who are not victims of bullying. Pure victims are children who are victimized bybullies but who do not bully others. Bully-victims are children who bully other children andare also victims of bullying. Neutrals are children who are neither bullies nor victims.

Past research shows clearly that physical bully-victims tend to be most at risk for psy-chosocial problems and aggressive behaviors against their peers (Stein et al., 2007; Unnever,2005). Other problems that are associated with bully-victims include poor social adjustment(Nansel et al., 2001), social isolation (Juvonen, Graham, & Schuster, 2003; Veenstra et al.,2005), alcohol use (Nansel et al., 2004; Stein et al., 2007), mental health symptoms (Juvonenet al., 2003; Kaltiala-heino et al., 2000), behavior disorders (Kokkinos & Panayiotou, 2004;Schwartz, 2000), personality difficulties (Kaltiala-heino et al., 2000), and relationship difficul-ties (Kumpulainen & Räsänen, 2000). Findings on pure bullies and pure victims show negativeeffects, but they are not as strong as the effects that have been observed for bully-victims.Children who are victimized, furthermore, often report poor attitudes toward school (Olweus,2003; Stein et al., 2007).

3. Multiple problem behaviors

Problems among adolescents often do not occur singly. Rather, problems occur in sets.This co-occurrence of problem behaviors, such as those reviewed above, has been representedby a construct, the problem behavior syndrome (Donovan & Jessor, 1985; Jessor & Jessor,1977). First using factor analysis, and later using Structural Equation Modeling, researchershave demonstrated the existence of the syndrome (Ary et al., 1999; Willoughby, Chalmers,& Busseri, 2004). The components of multiple problem behaviors most often include useof alcohol, tobacco and other drugs, as well as delinquency and other anti-social behavior.Observed intercorrelations among problem behaviors likely are due to their relation to anunderlying, latent variable of general deviance (Osgood, Johnston, O’Malley, & Bachman,1988). Since ample evidence above suggests that bullying is related to various adolescentproblems, it is highly likely to be related to multiple problem behaviors as well.

4. Current study

In a prior study, using data gathered in 2003, we examined psychosocial and behavioralproblems associated with physical bullying among a large sample of adolescent boys (Steinet al., 2007). We found evidence for substantive differences among those boys classified asneutrals, bullies, victims, and bully-victims. The bully-victims were greatly disadvantagedcompared to the other boys, and the neutrals reported the most positive outcomes. In thecurrent research, we examine the extent to which we can extrapolate our findings to a differenttype of bullying, relational bullying. We create the same four types of groups that are standard

674 R.L. Dukes et al. / The Social Science Journal 46 (2009) 671–688

in the literature. This time we include adolescent girls and boys in our analyses, and weassess the extent to which relational bully-victims will be disadvantaged on the same outcomesthat we addressed in the prior study. In addition, we examine physical bullying and physicalvictimization as further correlates of relational bullying.

Numerous studies have examined the effects of status as a physical neutral, bully, vic-tim, or bully-victim, but few have examined the connection between these types of bullyinggroups and negative effects associated with relational bullying. It is important to investigatecorrelates of relational bullying among bullies, victims and bully-victims because this area isunderstudied, and because of the violence or other adverse sequelae that sometimes are theresult of bullying among youth. Especially important is the investigation of a link betweenrelational bullying and injury because of the possible connection between verbal harassmentand violence. Knowledge about relational bullying may be expected to play an importantrole in the development and improvement of high-quality bullying interventions. If dysfunc-tional outcomes for physical bullying and relational bullying are similar, programs that aredesigned to curtail physical bullying would be encouraged to place equal emphasis on relationalbullying.

From previous research we hypothesize that adolescents who are not involved in relationalbullying (neutrals) will have the highest self-esteem, best school attitudes, least multipleproblem behavior, less involvement with physical bullying, and least injury. By contrast,we hypothesize that relational bully-victims will have the lowest self-esteem, worst schoolattitudes, most multiple problem behavior, most injury, and the greatest association with phys-ical bullying and victimization. Furthermore, we hypothesize that girls will engage in morerelational bullying than boys.

5. Method

5.1. Participants

The entire population of students from the four middle schools and two high schools in asuburban school district in Colorado Springs, Colorado, was eligible to participate in the surveyin April 2007. A total of 2,724 respondents participated in the survey. The response rate was76%. Students may not have participated in the survey because of absence from class duringthe testing session, moving or dropping out of school after the school census date but beforetesting. Finally students may have refused to participate on the day of the testing. Demographicsof responding students closely matched official monthly enrollment statistics for the district.

Complete data were available for 2,494 participants. Based on responses to a series ofquestions about bullying and victimization (described below) we were able to classify therespondents into four sets of respondents: 1,687 neutrals, 291 relational bullies, 303 relationalvictims, and 213 relational bully-victims.

The final sample was 51% male. Nineteen percent of the sample was white, 18% wasAfrican American, 31% was Hispanic, 24% described themselves as mixed (parents of differentethnicity), nearly 6% were Asian, and 3% were Native Americans. Twenty percent of the samplereported that their parents had less than a high-school education, 27% of the parents were high

R.L. Dukes et al. / The Social Science Journal 46 (2009) 671–688 675

school graduates, 24% of the parents had attended college or a trade school, and over 29%were college graduates or had advanced degrees.

5.2. Instrumentation and procedures

Anonymous responses to a 112-item questionnaire, written in English, were recorded on amachine-scored form. Respondents completed the survey in an average time of 25 min. Schooladministrators informed parents about the survey in March 2007, and they posted copies ofthe questionnaire in each school building. Although parents could have requested that theirchildren not participate in the survey, no refusals were reported. Written instructions on theenvelope for each classroom requested that students collect completed surveys and seal them inthe envelope for delivery to the central office of the school. School personnel sent the envelopesto the district office for transmittal to the researchers. On the instrument, written instructionspromised anonymity to the student participants and invited them to decline to answer any itemsto which they objected or to which they felt their parents might object. The research obtaineduniversity IRB approval via an expedited process because the questionnaire was anonymousand answering posed minimal risk. The school district assumed passive parental consent afterinforming parents about the survey.

5.3. Measures

5.3.1. Classification into bullying categoriesFour pairs of items based upon Crick and Grotpeter (1995) assessed relational bullying and

relational bullying victimization. Relational bullying items asked, “During the last 12 months,how often have you . . .” (1) “. . .told friends you would stop liking them unless they did whatyou said?” (2) “. . .told lies about someone to make other kids not like them anymore?” (3)“. . .tried to keep certain people from being in your group?” and (4) “. . .spread embarrassinginformation about someone?”

In addition, four items were paired with the ones above. They assessed being a victim ofrelational bullying by asking, “During the last 12 months, how often has someone. . .” (1)“. . .told you they would stop liking you unless you did what they said?” (2) “. . .told lies aboutyou to make other kids not like you anymore?” (3) “. . .tried to keep you from being in theirgroup?” and (4) “. . .spread embarrassing information about you?”

Responses were recorded on five-point scales: “Never (1),” “Once (2),” “Twice (3),” “Threeor four times (4),” “Five or more times (5).” Neutrals were classified as respondents whoanswered “1” to all eight items or those who did not fall into the more extreme categoriesdescribed below (N = 1,687). Bullies were classified as respondents who answered “1” or “2”to at least one victimization item and “4” or “5” to the paired bullying item (N = 291). Victimswere classified as respondents who answered “1” or “2” to at least one bullying item and “4”or “5” to the paired victimization item (N = 303). Bully-victims responded “4” or “5” to atleast one pair of bullying and victimization items (N = 163). In addition, bully-victims alsowere classified as bullies on one pair of items and then as victims on at least one other pair ofitems (N = 150). The total number of bully-victims was 213 (163 + 50 = 213). We opted for thecreation of discrete bullying groups due to their heuristic value when they were contrasted on

676 R.L. Dukes et al. / The Social Science Journal 46 (2009) 671–688

the key constructs included in this study. Contrasting these nosological groups has provideduseful information previously (e.g., Stein et al., 2007).

5.3.2. DemographicsRespondents were asked “What was your average grade on your last report card?” Responses

were: mostly A’s, mostly B’s, etc. scored 1–5 (mean score = 3.0). Parental education wasassessed by an item that asked, “How much education was completed by your parent who wentto school longer?” Response categories and numerical codes ranged from: “Grade school orless,” coded “1,” to “Graduate school (doctor, lawyer, Ph.D.),” coded “6.”

5.4. Latent variables

Self-esteem was indicated by five positively worded items from the Rosenberg Self-esteemScale (1965). Items included: (1) “I am satisfied with myself.” (2) “I feel that I have a numberof good qualities.” (3) “I am able to do things as well as most people.” (4) “I feel that I am aworthwhile person,” and (5) “I take a positive attitude toward myself.” The five-point responsescales were anchored by “strongly disagree” (1) and “strongly agree” (coefficient α = .84). Thetreatment of five or more ordinal response categories as being at the interval/ratio level ofmeasurement has been standard procedure in social science research because of its utility andstatistically sound approach (Bollen & Barb, 1981; Labovitz, 1970). An underlying continuousdistribution is assumed.

Positive attitudes toward school were indicated by three items. The first item said, “I enjoygoing to school.” The second item was reverse-scored. It said, “I do not care how I do in school.”The response scales for these two items were identical to the one above. The third item asked,“How important is it to you to get good grades in school?.” The five-point response scale wasanchored by “not important at all” (1) to “very important” (coefficient α = .57).

Problem behavior was indicated by four items that are means of responses to questionsconcerned with (1) Weapon possession: the weapon items assessed possession of a gun orother weapons such as knives or clubs at school, at school-sponsored activities, and while outwith friends. Responses were scored on five-point scales identical to the ones described abovefor bullying and victimization (coefficient α = .87); (2) delinquent behavior: the heading stated,“During the past twelve months how often have you”: example items include: “Gotten into aserious physical fight,” “Hurt someone enough to need bandages or a doctor,” “Taken somethingfrom a store without paying for it,” “Damaged property just for fun (such as breaking windows,scratching cars, painting graffiti),” “Gotten into trouble with the police,” and “Violated curfew.”Responses to all delinquency items were scored on a five-point scale identical to the onesfor bullying and victimization (coefficient α = .79). (3) Alcohol use: items measured use ofbeer, wine or wine coolers, and hard liquor (coefficient α = .86); (4) hard drug use includedamphetamines and barbiturates, psychedelics (LSD, PCP, or other psychedelics), cocaine (orcrack), “club” drugs, heroin, steroids, and inhalants. Responses for both alcohol and hard drugswere scored on a seven-point scale: “Never tried (1),” “Tried once or twice in past but quit(2),” “Occasionally but not in the last 30 days (3),” “Once/twice in the last 30 days (4),” “Threeto five times in the last 30 days (5),” “Six to fifteen times in the last 30 days (6),” and “Overfifteen times in the last 30 days (7).” Coefficient α for hard drugs was .94.

R.L. Dukes et al. / The Social Science Journal 46 (2009) 671–688 677

Injury was indicated by three items. A header asked, “During the last twelve months,how many times have you been injured by someone enough to need bandages or a doc-tor?.” Individual items stated, “At school,” “Going to and from school,” and “While out withfriends.” Responses were recorded on five-point scales identical to those used for bullying andvictimization (coefficient α = .76).

Physical bullying was indicated by three items. A header stated, “During the past 12 monthshow often have you. . .” “(1) . . .hit or kicked someone who is weaker than you are?”; (2)“. . .grabbed or shoved someone who is weaker than you are?”; and (3) “. . .threatened someonewho was weaker than you are?.” Responses were recorded on five-point scales identical to thoseused for relational bullying and victimization (coefficient α = .87).

Physical victimization was indicated by three items. A header stated, “During the past 12months how often have you. . .” (1) “. . .been hit or kicked by someone who is stronger thanyou are?”; (2) “. . . been grabbed or shoved by someone who is stronger than you are?”; and (3)“. . . been threatened by someone who was stronger than you are?” Responses were recordedon five-point scales identical to those used for relational bullying and victimization (coefficientα = .88).

5.5. Analyses

5.5.1. Assessments based on group membershipCrosstabulations were used to assess associations between the four bullying and victimiza-

tion groups and gender, age, grade point average and parental education. Initially, to ascertainwhether we needed to do a multilevel model based on school differences, we also examinedthe intraclass correlations for all variables used in the model based on school membership. Nointraclass correlations even approached significance, so we concluded there were no schooleffects for relevant variables in this model.

5.5.2. Latent variable analysesWe performed all latent variable analyses (Ullman, 1996) using the EQS structural equa-

tions modeling program (Bentler, in press; Byrne, 2006). Latent variables are hypothesizedunderlying constructs that explain the shared variance among indicator variables. They areconsidered to be error-free, and they represent a higher order of abstraction than measuredvariables.

Because we have a categorical independent variable, the four bullying groups, and inter-val/ratio dependent variables, our analyses most closely parallel multivariate analyses ofvariance (MANOVA). Structural Equation Modeling (SEM) gives a much clearer picture ofhow the items of the dependent variable are related to each other. Use of a bully-victim groupcaptures the joint effects of being a bully and being a victim. Structural Equation Modelingis more parsimonious than either regression or analysis of variance; therefore, it has a largeadvantage in the analysis of multivariate data such as ours. In addition, unlike regression oranalysis of variance, Structural Equation Modeling provides several single summary measuresof the overall fit of the model to the data. The availability of fit indices gives Structural EquationModeling a decisive advantage over the competing techniques.

678 R.L. Dukes et al. / The Social Science Journal 46 (2009) 671–688

5.5.3. Tests of significance in SEMGoodness-of-fit of the models was evaluated using the maximum-likelihood chi-square

statistic (ML χ2). This test of significance tests the null hypothesis that the saturated covariancematrix of all possible relations is identical to the covariance matrix of the restricted modelused in the research. Ideally, the researcher would like the maximum-likelihood chi-squareto be nonsignificant because it would mean that the more parsimonious, restricted model hadcaptured nearly the same amount of covariance as the saturated matrix. We also report the RootMean Square Error of Approximation (RMSEA). The RMSEA is a measure of fit per degreesof freedom, controlling for sample size, and values less than .06 indicate a relatively good fitbetween the hypothesized model and the observed data (Hu & Bentler, 1999). The RMSEAcan be used like more familiar standard error statistics to create a confidence interval arounda fit index.

In addition, we employed several other fit indices. We used the Comparative Fit Index (CFI),and robust fit statistics: the Satorra–Bentler χ2 (S–B χ2) and the Robust Comparative Fit Index(RCFI). We used the S–B χ2 in addition to the maximum-likelihood fit statistics because thedata were multivariately kurtose. This condition is common in research on deviant behaviorbecause most of the respondents report little or no deviance. The CFI and RCFI, which rangefrom 0 to 1, report the improvement in fit of the hypothesized model over a model of completeindependence adjusted for sample size. Values on the CFI and RCFI of at least .90 represent awell-fitting model, and values of at least .95 represent a model that fits the data very well (Hu& Bentler, 1999).

5.5.4. Confirmatory factor analysesSeparate confirmatory factor analyses were performed on the dependent variables for each

relational bullying group: neutrals, bullies, victims, and bully-victims. These analyses testedthe plausibility of the measurement model within each group and provided the correlationsamong the latent variables. To improve fit, we planned to add a few correlated error residualssuggested by the Lagrange Multiplier (LM) Test provided that we could rationalize thesecorrelated error residuals as plausible and logical (Bentler, in press).

5.5.5. Multisample analysesWe contrasted the four groups (neutrals, bullies, victims, and bully-victims) through the

use of multisample constrained models. First, after assessing a baseline unrestrained model,multiple-group latent variable models tested the equivalence (invariance) of the measurementmodel among the groups (Byrne, Shavelson, & Muthén, 1989; Stein, Lee, & Jones, 2006).The factor loading of each measured variable on its latent factor was constrained to equalityacross the four groups. The LM test in this context reports constraints that are untenable.We then contrasted the covariances between the variables in the model to determine whetherthese relationships were similar for the groups. The plausibility of the equality constraints wasdetermined with chi-square difference tests.

Next, we assessed whether there were significant group differences in the latent means of thelatent constructs in the model. We used the bully-victims as the reference group for one latentmeans analysis because we had hypothesized that they would show the lowest self-esteem,worst school attitudes, most problem behavior, most injury, and the most physical bullying and

R.L. Dukes et al. / The Social Science Journal 46 (2009) 671–688 679

victimization among the four groups. This analysis constrains the item means to equality, andthe LM test reports which of these constraints are not tenable. In an additional analysis, wealso used the neutral group as the reference group to assess whether respondents in this groupheld an advantage over respondents in the other groups.

6. Results

6.1. Crosstabulations of relational bullying by demographic variables

Boys were more likely to be neutrals (71%) than girls (66%), and girls were more likelyto be relational bully-victims (16%) than boys (10%; χ2 = 21.40, 3 d.f.; p < .001). In addition,a higher percentage of A-students were neutrals (72%) than F-students (59%), and a higherpercentage of F-students were bully-victims (15%) than A-students (9%; χ2 = 27.70, 12 d.f.;p < .01). There were no significant effects of age or parental education.

6.2. Confirmatory factor analyses

After minimal model modification (and in some cases no model modification at all) the CFAshad an excellent fit in all groups (neutrals: ML χ2 = 712.26, 189 d.f.; CFI = .95, RMSEA = .041;relational bullies: ML χ2 = 327.49, 187 d.f.; CFI = .95, RMSEA = .053; relational bully-victims:ML χ2 = 258.96, 189 d.f.; CFI = .96, RMSEA = .044; relational victims: ML χ2 = 285.05, 187d.f.; CFI = .95, RMSEA = .043). Robust statistics were equally good. All hypothesized factorloadings for all groups were significant (p ≤ .001). Two supplementary correlated errors wereadded to the models for the bully group (weapons and hard drugs, and delinquent behaviorand injury while out with friends) and the victims group (two of the self-esteem indicators,and the two threat items from the physical bullying and victimization latent variables). Table 1presents the factor loadings, means, and standard deviations of the measured variables for eachgroup. Table 2 reports the correlations among all of the latent variables, and it includes thecorrelations with gender as well.

6.3. Multisample analyses

Before the means of the four samples were contrasted, we tested whether there was rea-sonable factorial invariance between the groups. A baseline model provided the benchmarkfor further comparisons. We used the robust statistics to determine the χ2-differences dueto the high multivariate kurtosis across the groups. The baseline model had an outstanding fit(ML χ2 (756, N = 2,494) = 1,662.32; CFI = .95; RMSEA = .045; S–B χ2 = 1,429.81; RCFI = .95,RMSEA = .039). Adding the invariance constraints on the full measurement model produceda significant decrement in fit (adjusted χ2-difference = 178.11/44 d.f.) although all of the fitindexes were still excellent. After dropping three constraints as suggested by the LM test, theadjusted difference was nonsignificant (p > .05) (χ2-difference = 53.72/41 d.f.; critical valuefor χ2, p = .05, 41 d.f. = 56.84). Fit indexes were excellent. Equality constraints that equatedthe factor loadings of relational bully-victims and the neutrals on alcohol use and delinquent

680R

.L.D

ukesetal./T

heSocialScience

Journal46(2009)

671–688

Table 1Means, standard deviations, and factor loadings for each group.

Variable (see text for complete wording of items) Neutral Bully Victim Bully-victim

Mean(S.D.)

Factorloading

Mean(S.D.)

Factorloading

Mean(S.D.)

Factorloading

Mean(S.D.)

Factorloading

Problem behaviorDelinquent behavior (1 = low, 5 = high) 1.79 (0.87) 0.86 2.72 (1.16) 0.86 1.93 (0.85) 0.84 2.64 (1.10) 0.67Weapon-carrying (gun, knife; 1 = never, 5 = 5 or more) 1.17 (0.50) 0.51 1.50 (0.87) 0.68 1.21 (0.48) 0.35 1.59 (1.06) 0.89Hard drugs (1 = never, 7 = 15+ times in last month) 1.10 (0.44) 0.31 1.35 (0.87) 0.52 1.13 (0.40) 0.36 1.53 (1.33) 0.81Alcohol (beer, wine, liquor) 0.29 (0.45) 0.49 0.49 (0.50) 0.46 0.34 (0.48) 0.46 0.50 (0.50) 0.39

Self-esteemSatisfied with self 4.06 (0.90) 0.67 4.03 (1.07) 0.80 3.78 (1.05) 0.79 3.74 (1.09) 0.72Number of good qualities 4.09 (0.82) 0.74 3.96 (1.04) 0.75 3.99 (0.97) 0.75 3.89 (1.11) 0.82Does things well as most 4.08 (0.88) 0.61 4.11 (1.02) 0.66 3.90 (0.95) 0.49 3.86 (1.15) 0.72I am a worthwhile person 4.06 (0.91) 0.73 4.03 (1.12) 0.82 3.90 (1.09) 0.62 3.98 (1.19) 0.80Positive attitude toward self 3.98 (1.00) 0.70 3.91 (1.16) 0.78 3.69 (1.15) 0.70 3.75 (1.24) 0.69

Injury (1 = never, 5 = 5 or more)At school 1.16 (0.52) 0.62 1.40 (0.94) 0.79 1.36 (0.89) 0.74 1.65 (1.22) 0.73Going to/from school 1.13 (0.47) 0.69 1.34 (0.86) 0.86 1.26 (0.78) 0.82 1.54 (1.18) 0.84While out with friends 1.28 (0.77) 0.64 1.61 (1.15) 0.72 1.45 (0.96) 0.49 1.83 (1.32) 0.71

Positive attitudes toward schoolEnjoys school 3.26 (1.12) 0.54 2.80 (1.21) 0.59 3.18 (1.13) 0.43 2.88 (1.25) 0.57Importance of good grades 4.33 (0.88) 0.64 4.00 (1.10) 0.76 4.41 (0.89) 0.78 4.02 (1.18) 0.72Cares about school performance 4.12 (1.24) 0.40 3.79 (1.40) 0.59 4.31 (1.15) 0.52 3.98 (1.35) 0.60

Physical bullyingHit or kicked someone weaker 1.68 (1.13) 0.83 2.67 (1.54) 0.79 1.90 (1.27) .82 2.91 (1.63) 0.83Grabbed or shoved someone weaker 1.71 (1.14) 0.90 2.70 (1.53) 0.94 1.94 (1.24) .84 3.03 (1.63) 0.83Threatened someone weaker 1.53 (1.04) 0.71 2.68 (1.62) 0.77 1.82 (1.32) .63 2.87 (1.60) 0.76

Physical victimizationHit or kicked by someone stronger 1.96 (1.27) 0.85 2.48 (1.51) 0.85 2.69 (1.60) .88 3.23 (1.53) 0.83Grabbed or shoved by someone stronger 1.85 (1.24) 0.92 2.26 (1.49) 0.90 2.55 (1.58) .87 3.10 (1.56) 0.94Been threatened by someone stronger 1.71 (1.17) 0.73 2.10 (1.44) 0.74 2.48 (1.58) .76 3.02 (1.58) 0.76

R.L

.Dukes

etal./The

SocialScienceJournal46

(2009)671–688

681

Table 2Correlations among latent variables and gender.

Problem behavior Self-esteem Injury School attitude Physical bullying Physical victimization

Relational neutrals (N = 1,687)Problem behaviorSelf-esteem −.05Injury .40*** −.06School attitudes −.46*** .48*** −.21***Physical bullying .47*** −.01 .23*** −.18***Physical victimization .39*** −.08** .28*** −.15*** .43***Female −.16*** .00 −.04 .18*** −.08** −.05*

Relational bullies (N = 291)Problem behaviorSelf-esteem −.10Injury .28*** −.07***School attitudes −.49*** .36*** −.13Physical bullying .58*** .05 .13 −.15*Physical Victimization .25*** −.01 .08 .05 .36***Female −.30*** −.09 −.10 .14* −.14* −.06

Relational bully-Victims (N = 213)Problem behaviorSelf-esteem −.26***Injury .70*** −.11School attitudes −.59*** .57*** −.47***Physical bullying .39*** −.01 .28*** −.32***Physical victimization .17* −.09 .22** −.05 .54***Female −.27*** .06 −.24*** .23** −.12 −.18*

Relational victims (N = 303)Problem behaviorSelf-esteem −.04Injury .15* −.16*School attitudes −.25*** .41*** −.08Physical bullying .45*** −.05 .09 .09Physical victimization .30*** −.27*** .33** −.10 .42***Female −.27*** .03 −.14* .17* −.07 −.15*

*p < .05, *p < .01, ***p < .001.

682 R.L. Dukes et al. / The Social Science Journal 46 (2009) 671–688

behavior were reported as untenable by the LM test, and they were dropped. Also, responsetendencies were unalike on one item on the self-esteem instrument for bully-victims versusvictims: this item said, “I am able to do things as well as most people.” Due to the droppingof the three constraints, there is some very slight partial measurement invariance in the mul-tisample model. However, this very minimal degree of partial invariance did not preclude usfrom taking the next steps in the analyses, the comparison of the covariances and the testingof the means (Byrne et al., 1989).

When we constrained the covariances of the latent variables in the four groups to equal-ity, several differences in the magnitude of their associations were statistically significant inthe multivariate test (p < .001, χ2

crit = 7.87). We also constrained the associations betweengender and the latent variables to equality, but none of the differences was significant. Thegreatest differences were found for the covariances between physical bullying and physicalvictimization contrasting relational bully-victims (standardized r = .54) versus neutrals (.43)and relational bully-victims (.54) versus relational victims (.42), problem behavior and injurycontrasting relational bully-victims (.70) versus neutrals (.40) and relational bully-victims (.70)versus relational victims (.15), physical victimization and self-esteem for neutrals (−.08) ver-sus relational victims (−.27), problem behavior and physical bullying for relational victims(.45) versus relational bullies (.58), and relational bullies (.58) versus relational bully-victims(.39), and injury and physical victimization for relational bully-victims (.22) and relationalvictims (.33).

6.4. Latent means comparisons

Because we hypothesized that the relational bully-victims would report the most dysfunc-tional attitudes and behavior, they were used as the reference group in the first latent meansanalysis (see Table 3). Gender was included as a covariate. As expected, latent means scoresof the relational bully-victims were the least favorable. We found various differences in latentmeans as indicated by the z-scores. First, large differences favored the neutral adolescents.As reported in Table 3, the bully-victims reported significantly more problem behavior, lessself-esteem, more injury, worse school attitudes, more physical bullying, and more physical vic-timization. All probabilities were <.001 except for self-esteem (p < .05). Second, bullies wereless likely to be physical victims than bully-victims (p < .001). Other differences were non-significant but generally they favored the bullies over the bully-victims. Third, there were largedifferences in favor of the relational victims. Relational victims reported less problem behavior,less injury, better school attitude, less physical bullying, and less physical victimization thanbully-victims. All probabilities were <.001 except for injury (p < .01).

As an additional analysis to test the degree of dysfunction among the relational victimsand bullies, we used the neutrals as the reference group in another latent means analysis (seecolumns 4 and 5 in Table 3). In addition to the considerable mean differences reported abovebetween the bully-victims and the neutrals (column 1), we found further differences favoringthe neutrals. Bullies reported more problem behavior, more injury, worse school attitudes,more physical bullying, and they were more likely to have been physically victimized than theneutral adolescents (all p < .001). Victims reported more problem behavior, less self-esteem,more injury, more physical bullying and more victimization than the neutral adolescents. All

R.L

.Dukes

etal./The

SocialScienceJournal46

(2009)671–688

683

Table 3z-Scores in four-group latent means comparisons among adolescent relational neutrals (N = 1,687), relational bullies (N = 291), relational bully-victims (N = 213),and relational victims (N = 303).

Variable Relationalbully-victims versusrelational neutralsa

Relationalbully-victims versusrelational bulliesa

Relationalbully-victims versusrelational victimsa

Relational neutralsversus relationalbulliesb

Relational neutralsversus relationalvictimsb

Problem behavior 9.74*** 1.64 7.00*** 10.67*** 2.30*Self-esteem 2.42* 1.39 .35 .82 3.85***Injury 5.60*** 1.44 2.63** 4.64*** 3.75***School attitudes 3.16*** 1.80 3.51*** 5.81*** 1.49Physical bullying 11.12*** 1.52 8.07*** 10.52*** 2.64**Physical victimization 11.38*** 5.90*** 3.69*** 4.97*** 7.91***

*p < .05, **p < .01, ***p < .001.a Positive score indicates higher scores for relational bully-victims.b Positive score indicates higher scores for relational neutrals.

684 R.L. Dukes et al. / The Social Science Journal 46 (2009) 671–688

probabilities were less than .001 except for probabilities for problem behavior (p < .05) andphysical bullying (.01).

7. Discussion

Results showed that relational bullying was similar to physical bullying in its negativeconsequences for adolescents (Stein et al., 2007). Neutrals in the current study had the highestself-esteem, best school attitudes, least problem behavior, least injury, and were less likely toengage in physical bullying or be physically bullied. By contrast, the relational bully-victimshad the lowest self-esteem, worst school attitudes, most problem behavior and most injury.They were also likely to engage in more physical bullying and be physically victimized. Purebullies and pure victims were between the extremes of neutrals and relational bully-victimsbut nonetheless reported considerably poorer outcomes than the neutrals.

Our findings support clearly the negative impacts of relational bullying. They extend recentfindings on physical bullying among a similar group of boys (Stein et al., 2007). Thus, we canextrapolate our prior results to another dimension of bullying behavior: relational bullying, andour findings on relational bullying apply to both boys and girls. Those who are perpetrators ofrelational bullying while being victims of such bullying reported just as many dysfunctionalattitudes and behaviors as those in the earlier study who reported being physical bully-victims.Thus far, relational bullying has not been considered to have negative consequences as extensiveas those for physical bullying, but the present research shows that relational bullying is aproblem of equal importance because of the negative consequences to bullies and victims andbecause of a possible linkage to violent behavior and injury. Future research should explorethese findings in more depth.

Demographic patterns found in our prior study of physical bullying were not found in thisstudy. This inconsistency may due in part to the presence of boys and girls rather than justboys. For relational bullying, boys were more likely to be neutrals than girls, and girls weremore likely to be victims; however, both boys and girls participated in relational bullying. Thepattern for age and relational bullying was similar to the pattern for physical bullying foundin our earlier study. Victims tended to be younger, and neutrals and bully-victims tended to beolder, but results were not statistically significant.

Past research on relational bullying has focused more on girls than boys (e.g., Björkqvistet al., 1992). This emphasis may be due to the cultural norm that girls are expected to be lessphysically aggressive than boys. In addition, girls, may value their friendships more highlythan boys; hence, many conflicts among girls stem from emotions regarding these friendships(Besag, 2006). However, as stated above, it is clear that relational bullying is occurring amongboth boys and girls.

Weapon-carrying was part of problem behaviors, so analyses did not examine it specifically.However, more fine-grained analyses indicated that relational bully-victims reported moreweapon-carrying than neutrals, bullies or victims of relational bullying (ETA = .30; p < .001).This finding provides a possible connection to the violence that has on occasion been pre-ceded by relational bullying, and it is consistent with those from qualitative research by Leary,Kowalski, Smith and Phillips (2003). These researchers found that access to – and fascination

R.L. Dukes et al. / The Social Science Journal 46 (2009) 671–688 685

with – firearms and explosives facilitated the connection between bullying and a subsequentdeadly rampage aimed at retribution and restoration of respect (Kimmel & Mahler, 2003; Learyet al., 2003; Newman Nguy, 2004; Vossekuil, Fein, Reddy, Borum, & Modzeleski, 2002).

7.1. Policies and programs on bullying

Bullying will not disappear spontaneously (Zimring, 1998), so schools and communitiesshould attempt to deal with underlying causes of youth behavior problems (Jones, Fisher,Greene, Hertz, & Pritzl, 2007). However, anti-bullying policies without programs in primaryschools have been reported to have no effect on the amount of physical bullying Ironically,anti-bullying policies may drive bullying underground, so that direct bullying may give wayto relational bullying, a process that is more subtle, insidious, and less likely to be observed byadults (Woods & Wolke, 2003); however as our research shows relational bullying can resultin negative outcomes that are just as serious as those of direct bullying.

Most researchers agree that the school is an appropriate place to implement anti-bullyingprograms. Parental involvement, strong support and discipline at school, as well as an emphasison academics, appear to discourage bullying (Juvonen & Graham, 2004; Samples, 2004).Recent research suggests that fostering empathy and self-efficacy among adolescents can behelpful in the creation of group resistance to bullying (Gini, Albiero, Benelli, & Altoè, 2008;Jolliffe & Farrington, 2006). Future research should investigate the extent to which this relationholds for relational bullying.

Overall, evaluations of anti-bullying programs have been encouraging (DeRosier, 2004;Juvonen & Graham, 2004; Samples, 2004; Tolan, 2001; Wilson, Lipsey, & Derzon, 2003).Unfortunately, most anti-bullying programs (and few evaluations of anti-bullying programs;Woods and Wolke (2003) could find only two) have not focused on relational bullying. Ourresearch emphasizes that anti-bullying programs and program evaluations should focus on bothphysical and relational bullying.

7.2. Limitations

Data reported here were from a crosssectional survey. Strengths and limitations of thismethod are well-known. Nevertheless, caveats are in order. First, we treated the relationalbullying variables as independent variables and negative outcomes as dependent variables.Our crosssectional survey research design cannot determine cause and effect, or even time-order (see Carlyle & Steinman, 2007 for a similar caveat), and even if we could determinetime-order, other antecedent variables could be the cause of both independent and dependentvariables—a classic case of explanation (Kendall & Lazarsfeld, 1950). Longitudinal studies ofbullying could go a long way toward untangling the time-order of variables in bullying studies.

Second, we collapsed the relational bullying measures into the four categories of neutral,bully, victim and bully-victim. This collapsing currently is the most standard method of dealingwith the additive and multiplicative effects of bullying (see Alikasifoglu et al., 2007; Carlyle& Steinman, 2007 for the recent examples). Future research should investigate the advantagesof treating bullying variables as continuous, as in a regression approach.

686 R.L. Dukes et al. / The Social Science Journal 46 (2009) 671–688

Third, the instrument was written in English, so students who spoke only Spanish couldnot complete it. This problem probably is a very small one, as the school district reports thatless than 16 students in each grade (7 through 12) are English language learners, and aboutthree-quarters of these students are at least partially proficient in English.

Acknowledgments

Partial support for this research was provided by school districts in the Pikes Peak regionof Colorado, and partial support was provided by a grant from the National Institute on DrugAbuse (DA01070-35). Views expressed in this paper do not necessarily reflect those of theNational Institute on Drug Abuse or the school districts. The authors gratefully acknowledgethe secretarial and administrative assistance of Gisele Pham.

References

Alikasifoglu, M., Mujgan, E., Ethem, E., Oya, U., Uysal, O., & Albayrak-Kaymak, D. (2007). Bullying behaviorsand psychosocial health: Results from a cross-sectional survey among high school students in Istanbul, Turkey.European Journal of Pediatrics, 166, 1253–1260.

American Psychological Association (2005). Bullying. Retrieved February 22, 2006 from: http://www.apa.org/ppo/issues/bullying.html.

Ary, D., Duncan, T., Biglan, A., Metzler, C., Noell, J., & Smolkowski, K. (1999). Development of adolescentproblem behavior. Journal of Abnormal Child Psychology, 27, 141–150.

Bentler, P. M. (in press). EQS 6 Structural Equations Program Manual. Encino, CA: Multivariate Software, Inc.Besag, V. (2006). Understanding girls’ friendships fights and feuds: A practical approach to girls’ bullying.

Columbus, OH: Open University Press.Björkqvist, K., Lagerspetz, K. M. J., & Kaukiainen, A. (1992). Do girls manipulate and boys fight? Developmental

trends in regard to direct and indirect aggression. Aggressive Behavior, 18, 117–127.Bollen, K., & Barb, K. (1981). Pearson’s r and coarsely categorized measures. American Sociological Review, 46,

232–239.Byrne, B. (2006). Structural equation modeling with EQS. Mahwah, NJ: Lawrence Erlbaum Associates.Byrne, B. M., Shavelson, R. J., & Muthén, B. (1989). Testing for the equivalence of factor covariance and mean

structures: The issue of partial measurement invariance. Psychological Bulletin, 105, 456–466.Carlyle, K., & Steinman, K. (2007). Demographic differences in the prevalence, co-occurrence and correlates of

adolescent bullying at school. Journal of School Health, 77, 623–629.Crawford, T. (2007, April 3). Bullying has spread like wildfire, educator says: Latest waves of violence involve

children as young as 4 years old, conference told. Retrieved May 28, 2009 from http://www.thestar.com/article/198803.

Crick, N. R., & Grotpeter, J. K. (1995). Relational aggression, gender, and social psychological adjustment. ChildDevelopment, 66, 710–722.

Crozier, W. R., & Skliopidou, E. (2002). Adult recollections o name-calling at school. Educational Psychology, 22,113–124.

DeRosier, M. E. (2004). Building relationships and combating bullying: Effectiveness of a school-based socialskills group intervention. Journal of Clinical Child & Adolescent Psychology, 33, 196–201.

Donovan, J., & Jessor, R. (1985). Structure of problem behavior in adolescence and young adulthood. Journal ofConsulting and Clinical Psychology, 53, 890–904.

Gini, G. (2004). Bullying in Italian schools: An overview of intervention programmes. School Psychology Interna-tional, 25, 106–116.

R.L. Dukes et al. / The Social Science Journal 46 (2009) 671–688 687

Gini, G., Albiero, P., Benelli, B., & Altoè, G. (2008). Determinants of adolescents’ active defending and passivebystanding in bullying. Journal of Adolescence, 31, 93–105.

Haynie, D. L., Nansel, T., Eitel, P., Crump, A. D., Saylor, K., Yu, K., et al. (2001). Bullies, victims, and bully-victims:Distinct groups of at-risk youth. Journal of Early Adolescence, 21, 29–49.

Hu, L., & Bentler, P. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteriaversus new alternatives. Structural Equation Modeling, 6, 1–55.

Jessor, R., & Jessor, S. (1977). Problem behavior and psychological development: A longitudinal study of youth.New York: Academic Press.

Jolliffe, D., & Farrington, D. (2006). Development and validation of the Basic Empathy Scale. Journal of Adoles-cence, 29, 589–611.

Jones, S., Fisher, C., Greene, B., Hertz, M., & Pritzl, J. (2007). Healthy and safe school environment, Part I. Resultsfrom the school health Policies and programs study 2006. Journal of School Health, 77, 522–543.

Juvonen, J., & Graham, S. (2004). Research-based interventions on bullying. In C. E. Sanders, & G. D. Phye (Eds.),Bullying: Implications for the classroom (pp. 229–255). San Diego, CA: Elsevier Academic Press.

Juvonen, J., Graham, S., & Schuster, M. A. (2003). Bullying among young adolescents: The strong, the weak, andthe troubled. Pediatrics, 112, 1231–1237.

Kaltiala-heino, R., Rimpela, M., Rantanen, P., & Rimpela, A. (2000). Bullying at school—An indicator of adoles-cents at risk for mental disorders. Journal of Adolescence, 23, 661–674.

Kendall, P., & Lazarsfeld, P. (1950). Problems of survey analysis. In R. Merton, & P. Lazarsfeld (Eds.), Continuitiesin social research: Studies in the scope and method of “The American soldier” (pp. 133–196). New York: TheFree Press.

Kimmel, M. S., & Mahler, M. (2003). Adolescent masculinity, homophobia and violence: Random school shootings1982–2001. American Behavioral Scientist, 46, 1439–1458.

Kleinfield, N. R. (2007, April 22). Before deadly rage, a life consumed by a troubling silence. New York Times,.Retrieved October 1, 2007, from http://www.nytimes.com.

Klomek, A. B., Marrocco, F., Kleinman, M., Schonfeld, I. S., & Gould, M. S. (2007). Bullying, depression, andsuicidality in adolescents. American Academy of Child and Adolescent Psychiatry, 46, 40–49.

Kokkinos, C. M., & Panayiotou, G. (2004). Predicting bullying and victimization among early adolescents: Asso-ciations with disruptive behavior disorders. Aggressive Behavior, 30, 520–533.

Kumpulainen, K., & Räsänen, E. (2000). Children involved in bullying at elementary school age: Their psy-chiatric symptoms and deviance in adolescence: An epidemiological sample. Child Abuse and Neglect, 24,1567–1577.

Labovitz, S. (1970). The assignment of numbers to rank order categories. American Sociological Review, 35,515–524.

Leary, M., Kowalski, R., Smith, L., & Phillips, S. (2003). Teasing, rejection, and violence: Case studies of theschool shootings. Aggressive Behavior, 29, 202–214.

Nansel, T. R., Craig, W., Overpeck, M. D., Saluja, G., Ruan, W. J., & The Health Behavior in School-aged ChildrenBullying Analyses Working Group. (2004). Cross-national consistency in the relationship between bullyingbehaviors and psychosocial adjustment. Archives of Pediatric Adolescent Medicine, 158, 730–736.

Nansel, T. R., Overpeck, M. O., Pilla, R. S., Ruan, W. J., Simons-Morton, B., & Scheidt, P. (2001). Bullying behaviorsamong U.S. youth: Prevalence and association with psychosocial adjustment. Journal of the American MedicalAssociation, 16, 2094–2100.

Newman, M., Holden, G., & Delville, Y. (2005). Isolation and the stress of being bullied. Journal of Adolescence,28, 343–357.

Newman Nguy, K. S. (2004). Rampage: The social roots of school shootings. New York: Basic Books.Olweus, D. (2003). Bullying is not a fact of life. Retrieved October 5, 2007, from http://download.ncadi.samhsa.gov/

ken/pdf/SVP-0052/SVP-0052.pdf.Olweus, D. (1993). Bullying at school: What we know and what we can do. Oxford, England: Cambridge, Blackwell.Osgood, D., Johnston, L., O’Malley, P., & Bachman, J. (1988). The generality of deviance in late adolescence and

early adulthood. American Sociological Review, 53, 81–93.Raskauskas, J., & Stoltz, A. D. (2004). Identifying and intervening in relational aggression. The Journal of School

Nursing, 20, 209–215.

688 R.L. Dukes et al. / The Social Science Journal 46 (2009) 671–688

Rook, K. (2007, April 18). Gunman was likely ‘a bullied bully,’ experts says, ‘relentlessly tormented’?Retrieved May 27, 2009 from http://www.nationalpost.com/news/story.html?id=d7d6bc5a-0ba0-4040-b4b4-71b0c89dd86d&p=1.

Rosenberg, M. (1965). Society and the adolescent self-image. Princeton, NJ: Princeton University Press.Samples, F. L. (2004). Evaluating curriculum-bases intervention programs: An examination of preschool, primary

and elementary school intervention programs. In C. E. Sanders, & G. D. Phye (Eds.), Bullying: Implications forthe classroom (pp. 203–227). San Diego, CA: Elsevier Academic Press.

Schwartz, D. (2000). Subtypes of victims and aggressors in children’s peer groups. Journal of Abnormal ChildPsychology, 28, 181–192.

Stein, J. A., Dukes, R. L., & Warren, J. I. (2007). Adolescent male bullies, victims, and bully-victims: A comparisonof psychosocial and behavioral characteristics. Journal of Pediatric Psychology, 32, 273–282.

Stein, J. A., Lee, J. W., & Jones, P. S. (2006). Assessing cross-cultural differences through the use of multiple groupinvariance analysis. Journal of Personality Assessment, 87, 249–258.

Tolan, P. (2001). Youth violence and its prevention in the United States: An overview of current knowledge. InjuryControl and Safety Promotion, 8, 1–12.

Ullman, J. (1996). Structural Equation Modeling. In B. Tabachnick, & L. Fidell (Eds.), Using multivariate statistics(third edition, pp. 709–811). New York: HarperCollins.

Unnever, J. D. (2005). Bullies, aggressive victims, and victims: Are they distinct groups? Aggressive Behavior, 31,153–171.

Veenstra, R., Lindenberg, S., Oldehinkel, A. J., De Winter, A. F., Verhulst, F. C., & Ormel, J. (2005). Bully-ing and victimization in elementary schools: A comparison of bullies, victims, bully-victims, and uninvolvedpreadolescents. Developmental Psychology, 41, 672–682.

Vossekuil, B., Reddy, M., & Fein, R. (2001). The secret services’ safe school initiative. Education Digest, 66, 4–10.Vossekuil, B., Fein, R., Reddy, M., Borum, R., & Modzeleski, W. (2002). The final report and findings of the Safe

Schools Initiative: Implications for the prevention of school attacks in the United States. Washington, D. C.:United States Secret Service and United States Department of Education.

Willoughby, T., Chalmers, H., & Busseri, M. (2004). Where is the syndrome? Examining co-occurrence amongmultiple problem behaviors in adolescence. Journal of Consulting and clinical Psychology, 72, 1022–1037.

Wilson, S. J., Lipsey, M. W., & Derzon, J. H. (2003). The effects of school-based intervention programs on aggressivebehavior: A meta-analysis. Journal of Consulting and Clinical Psychology, 71, 136–149.

Woods, S., & White, E. (2005). The association between bullying behaviour, arousal levels and behaviour problems.Journal of Adolescence, 28, 381–395.

Woods, S., & Wolke, D. (2003). Does the content of anti-bullying policies inform us about the prevalence of directand relational bullying behavior in primary schools? Educational Psychology, 23, 381–401.

Yang, S.-J., Kim, J.-M., Kim, S.-W., Shin, I.-S., & Yoon, J.-S. (2006). Bullying and victimization behaviors inboys and girls at South Korean primary schools. Journal of the American Academy of Child and AdolescentPsychiatry, 45, 69–77.

Zimring, F. E. (1998). American youth violence. New York: Oxford University Press.