Upload
dinhthu
View
220
Download
4
Embed Size (px)
Citation preview
Promising Results from the National Evaluation of the Gang Resistance Education and Training (G.R.E.A.T.)
Program
Finn EsbensenE. Desmond Lee Professor of Youth Crime &
ViolenceChair, Department of Criminology and
Criminal JusticeUniversity of Missouri-St. Louis
Presentation OverviewWhy is there interest in youth violence and gang
prevention?
How do we know what works?
What do terms like “evidence-based”, “model programs” and “effective” really mean?
Use the G.R.E.A.T. program as an example of how question of program effect is addressed.
NYGS: Jurisdictions Reporting Youth Gang Problems in One or More Years, 1999-2001(N=>1400)
Gang ViolenceWidely documented finding that gang
members are responsible for a disproportionate amount of crime.
Thornberry and Burch (1997), for example,
reported that gang members accounted for 86 percent of all serious offenses in the Rochester Youth Development Study.
Youths have higher rates of offending during active gang membership than they do either before or after gang involvement
How do we know what works?Anecdotal evidence is NOT sufficient
Good intentions are NOT sufficient
Saying that something works or that something is effective, no matter how many times you say it, does NOT make it so
Rigorous evaluation is required
Promising Programs forViolence Prevention The Blue Prints Program : Center for the Study and
Prevention of Violence at CU-Boulder Reviewed over 600 programs • 11 were identified as model programs
proven scientifically to be effective in reducing youth aggression, violence, other delinquent behavior, and substance abuse;
An additional 21 designated as “promising” That means 570 were not effective or not evaluated
Model Program DesignationSound program or theoretical model
Random assignment
Sustained program effect (12 month post treatment)
Replicated in at least one study
What is known about responses to gangs?
Very littleNo shortage of programs - lack of evaluations of
existing programs or lack of positive outcomes
G.R.E.A.T. as example
What is G.R.E.A.T.?13 week in-school general prevention program
Taught by uniformed law enforcement officer
Original program developed in 1991 by Phoenix area officers – 9 lessons and mostly didactic
Several efforts to review the curriculumnull findings from longitudinal study
Curriculum review conducted from 1999 – 2000
What is G.R.E.A.T.? - 2Program revision 2001 – piloted in 2002
Seattle Social Development Model - classroom management, cooperative learning, interactive teaching
Life Skills Training – self-management and social skills development
Interactive and skills building focusEspecially important that program be taught as
designed – lessons build on each otherImportant to assess program fidelity
Implementation of Revised G.R.E.A.T. in 2003
G.R.E.A.T. LESSONSLesson 1 – Welcome to G.R.E.A.T. - acts as the introduction
to the program and introduces the relationship between gangs, violence, drug abuse, and crime.
Lesson 2 – What’s the Real Deal - consists of message analysis skills and “facts and fictions” about gangs.
Lesson 3 – It’s About Us - focuses on different communities and how students are a part of these, including their responsibilities to their community or communities.
Lesson 4 – Where Do We Go From Here - introduces students to the concept of goals and how to set realistic and achievable aspirations.
Lesson 5 – Decisions, Decisions, Decisions - focuses on decision-making, in which students learn the G.R.E.A.T. decision making model and the impact their decisions have on their goals; students are able to practice making positive decisions.
G.R.E.A.T. LESSONS - IILesson 6 – Do You Hear What I’m Saying? - teaches the
importance of listening to others and the difference between verbal and non-verbal communication.
Lesson 7 – Walk in Someone Else’s Shoes - instructs students in active listening skills and how to identify others’ emotional states through empathy-building techniques.
Lesson 8 – Say It Like You Mean It - teaches refusal skills so students may resist peer-pressure to engage in deviant or delinquent acts; this includes learning about body language and tone of voice.
Lesson 9 – Getting Along Without Going Along - consists of recognizing peer pressure and other influences that may push students into delinquency.
Lesson 10 – Keeping Your Cool - teaches students to keep calm in the face of anger with anger management tips and practicing the “cooling-off” technique.
G.R.E.A.T. LESSONS - IIILesson 11 – Keeping It Together - consists of
recognizing anger in others and learning to calm them. Lesson 12 – Working It Out - teaches students to work
through problems without fighting and provides tips for conflict resolution, practice of such, and information about where to go for help in their communities.
Lesson 13 – Looking Back - consists of a program review and the presentation and discussion of student final projects.
http://www.great-online.org/
Program GoalsMeasuring Explicit Goals of Program:
Avoid gang membership Do you consider your group of friends to be a gang? Are you now in a gang?
Prevent violence and criminal activity Self-reported delinquency measures
Develop positive relationships w/law enforcement “Global” attitudinal scale G.R.E.A.T.-specific attitudes
Lesson Specific OutcomesImpulsivity and risk
takingAnger managementSelf centerednessPeer pressureNegative peer
commitmentPeer delinquencyNeutralization (hitting)Conflict resolution
•Refusal skills•Empathy•Active listening•Problem solving•Collective efficacy•Attitudes about gangs•Altruism
Risk FactorsAnalogy with medical fieldGenetic and lifestyle factors associated with
likelihood of particular illnessPresence of risk factors does not mean that
person will be ill; only increased probabilityAdopt this approach with violent offending
and gang membershipRisk factors in 5 domains: community, school,
family, peer, & individual
Risk factorsThe more risk factors and in multiple
domains; the greater the risk of gang joining.
No silver bullet – not all youth living in high-risk neighborhoods join gangs.
Addressing even one of these risk factors reduces the odds of gang joining and violent offending.
Cumulative Effect of Multiple Risk Factors
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
0 1-5 6-10 11+
Non-violentGang Membership
Number of Risk Factors
Does the program work?
Is the program implemented with fidelity?
Can the program be adopted in a variety of settings?
Is the program effective?
This research was made possible, in part, by the support and participation of seven school districts, including the School District of Philadelphia. This project was supported by Award No. 2006-JV-FX-0011 awarded by the National Institute of Justice, Office of Justice Programs, U.S. Department of Justice. The opinions, findings, and conclusions or recommendations expressed in this presentation are those of the authors and do not necessarily reflect the views of the Department of Justice or of the seven participating school districts.
Finn Esbensen, Ph.D., Principal Investigator University of Missouri-St. Louis
Terrance J. Taylor, Ph.D., Investigator University of Missouri-St. Louis
Dana Peterson, Ph.D., Investigator University at Albany
Wayne Osgood, Ph.D., InvestigatorPennsylvania State University
Kristy Matsuda, Ph.D., Research Associate University of Missouri-St. Louis
Adrienne Freng, Ph.D., Site CoordinatorUniversity of Wyoming
Dena Carson, Site CoordinatorUniversity of Missouri-St. Louis
Site Selection Site selection:
Geographic and demographic diversity Nature of community gang problems Law enforcement agency and Program size G.R.E.A.T. taught at least one year No program saturation Obtained agreements from law enforcement and public
school districts
Process: Created preliminary list of 56 cities (identified by at least
one RA, IIR or BJS staff member) Contacted agencies about G.R.E.A.T. program Resulted in final list of 7 – hoped to recruit 6
Albuquerque, NM ; Chicago, IL; DFW area city, TX; Greeley, CO; Nashville, TN; Philadelphia, PA; Portland, OR
Process EvaluationTo what extent is G.R.E.A.T. delivered as
intended? If effect found, can it be attributed to the program? If no effect found, is it a case of program failure or failed
program?
TriangulationGOT observationsClassroom observationsOfficer & Supervisor interviewsG.R.E.A.T. Officer surveySchool personnel questionnaires
GOT ObservationsObservations of Training
Observed nine GOTs (1 week and 2 week sessions)Purpose:
Learn new curriculum as officers are learning it; Assess quality and consistency of training within and
across GOTs
Well-designed and implementedReasonable to expect that officers are prepared to teach
the program with fidelity
Classroom Observations (2006-2007 school year)
Purpose:Assess program delivery regarding:
Dosage Adherence to intended lessons Quality of instruction
How?502 in-class observations of program delivery108 non-G.R.E.A.T. classes observed
Control classrooms and G.R.E.A.T. when program not taught
33 different officersEach lesson observed between 27 and 49 times – allows
us to assess lesson quality in addition to officer implementation fidelity
Findings: Implementation We were able to provide feedback on some important
lesson-related issues (e.g., time management, student interest, role of teacher, lesson content)
Importantly, of 33 total officers, 27 implemented the lessons in average/above average manner3 officers were judged not to have delivered the program
Evidence of program fidelity so if effects are found, they can be reasonably attributed to the program.
Outcome Evaluation
To what extent is G.R.E.A.T. achieving its stated goals?Determine both short- and long-term effectsCompare students who receive program to students who
do not, in an experimental designMultiple, diverse locations
Outcome EvaluationSeven cities
Albuquerque, Chicago, DFW area, Greeley, Nashville, Philadelphia, Portland
31 schools total 4 schools in five cities, 5 in Portland, 6 in Chicago
195 classes total (102 GREAT, 93 Control)24 - 35 classes per city3 - 12 classes in each school (half receive G.R.E.A.T.)
3,820 students457 - 614 students per cityBetween 59 and 186 students per school
Outcome EvaluationRandom Assignment at classroom level
Equivalent comparison groupsNo bias in assignment
Active Parental Consent
Students are the program target, so we need their assessments. How do we do this?Confidential self-report questionnairesQuestionnaires read aloud in group-administered settingResearchers ensure privacy, answer questionsPre- and post-tests4 annual follow-up surveys
Outcome – Student Surveys Site selection
Aug and Sept 2006Active consent process
Sept – Oct 2007; Jan & Feb 2008 (two added schools)Survey administration
Pre-tests - Sept 2006 – Feb 2007; Feb & March 2008Post-tests - Nov 2006 – May 2007; May 2008Wave 3 – Oct 2007 – March 2008; Dec 2008 – Feb 2009Wave 4 – Oct 2008 – March 2009; Nov 2009 – Feb 2010 Wave 5 – Oct 2009 – March 2010; Nov 2010 – March
2011Wave 6 – Oct 2010 – March 2011; October 2011
Data entry (and verification)Preparation of analysis file and outcome analyses
Completion RatesActive consent rate = 78% (N=3,820)
Pretest completion = 98%
Post-tests = 95%
Wave 3 = 87%
Wave 4 = 83%
Wave 5 = 75%
Wave 6 = 73% Wave 5 surveyed students in 216 difference schools 66 different schools in Philly
Outcome Results - ICompare G.R.E.A.T. students with non-G.R.E.A.T.
studentsWave 2 (post-test) andWave 3 (one year after program)
A number of significant differences were found:11 of 33 significant at p<.053 significant at p<.10
Program Goals - OutcomesAvoid gang membership
39% lower odds of gang membership
Develop positive relationships w/law enforcementMore positive attitudes to police (ES=.076)Teaching specific role of police (ES=.204)
Prevent violence and criminal activityNo significant differences
Proximate outcomesFavorable G.R.E.A.T. outcomes p<.05
Less positive attitudes about gangs (ES=.114)More us of refusal skills (ES=0.90)Resistance to peer pressure (ES=.079)Higher collective efficacy (ES=.125)Less hitting neutralizations (ES=.105)Fewer delinquent peers (ES=.083)Less self centeredness (ES=.054)Less anger (ES=.057)
Proximate outcomesFavorable G.R.E.A.T. outcomes p<.10
Less use of lie neutralizationMore pro-social peersMore pro-social involvement
So, why are these proximate outcomes important?
Remember, reducing one or two risk factors reduces odds of gang joining
Bottom LineAt 12 month post treatment, 2 of the three program
goals were met (reduction in gang joining and improvement in attitudes towards police)
A number of proximate factors suggested G.R.E.A.T. has a positive effect
But, effect sizes are modest & no effect on delinquency
Will these effects be sustained across time?Are these findings replicated across cities?Are there differential effects based on initial level of
risk?
ReportsWebsite:
http://www.umsl.edu/~ccj/html_files/great_evaluation.html
Reports to Schools & Communities: 2007: Evaluation Overview. 2008: Anti-Social Norms among a Sample of Middle-School Students. 2009: Program Implementation and Preliminary Outcome Results. 2010 Report to Schools and Communities: School Safety and
Victimization.
Results from Surveys and Interviews with G.R.E.A.T.-trained Officers.
Observing the Implementer: Description of Findings, Recommendations, and Conclusions from GREAT Program Implementation Observations.
School Personnel Survey Report.
G.R.E.A.T. Officer Training (GOT) Report.
That’s all, folksThanks for listeningAny questions?