Cure Violence Evaluation Plan

By Jeffrey A. Butts and Caterina Roman

Submitted to the Robert Wood Johnson Foundation — August 26, 2013

by the Research & Evaluation Center at John Jay College of Criminal Justice and

the Department of Criminal Justice at Temple University

BACKGROUND

With a planning grant from the Robert Wood Johnson Foundation (RWJF), researchers at John Jay

College of Criminal Justice (led by Dr. Jeffrey Butts) and Temple University (led by Dr. Caterina Roman)

recently collaborated to investigate the feasibility of a rigorous evaluation of the Cure Violence model of

violence reduction. The goal of the planning project was to identify possible indicators of successful

implementation and outcomes of the Cure Violence model in existing program sites— especially those

thought to be performing well— and to consider potential methods for conducting a rigorous evaluation

of the model that could add to the knowledge base about the effectiveness of Cure Violence. In addition,

the research team also observed operations of Cure Violence in three cities—New York, Philadelphia and

Chicago, to assess the potential for using one or more cities for the next evaluation of Cure Violence

(CV). During an initial meeting with RWJF and the evaluation Advisory Committee in October 2012, the

research team discussed a number of key questions that needed to be answered before a rigorous and cost-

effective evaluation could be undertaken. With funds provided by the planning grant, we sought to answer

those questions, which are summarized below.

Recommendations

Reject Randomization

One of the first issues to decide is random assignment. Is it feasible given the CV theory of change

and how it might be perceived by communities implementing the CV model? If sufficient funds were

available, could an evaluation actually randomize neighborhoods? How many neighborhoods would be

needed to ensure statistical (explanatory) power? If an evaluation wanted to randomize individuals instead

of communities, would this even be possible given the operating principles of the CV model?

To build sound knowledge about CV, researchers should use the most rigorous methods possible.

Ideally, this would include an experimental design. There would, of course, be many challenges in setting

up a randomized field experiment of CV while still maintaining the integrity of the program itself.

Page 2 of 25

Without an experimental design, internal threats to validity (as discussed during the Advisory Committee

meeting) will cast doubt on the findings of any evaluation. Previous studies have shown that it is possible

to implement randomized studies in field settings (e.g., one recent example is the Philadelphia Foot Patrol

Experiment), but it is not clear that a true experimental design is possible in an evaluation of the CV

model.

The consensus of the Advisory Committee was that to have a rigorous design, an evaluation would

need a “few dozen” neighborhoods in each condition (i.e., CV versus control). Given the scope of

implementation challenges in the CV model, as well as likely contamination issues (innovations are likely

to spread from neighborhood to neighborhood), we do not believe it is feasible to do random assignment.

Even clustered random assignment (with closely matched pairs) is impractical because so many facets of

social life would have to be measured inside every neighborhood. We discussed the most likely option of

recruiting six neighborhoods and randomly assigning three to treatment and three to control, but the

Advisory Board emphasized that this would be insufficient for attaining the statistical power needed to

test program effects at the neighborhood level and would only help to control some selection issues (e.g.,

the capacity of neighborhoods to implement CV).

We recommend that RWJF not pursue a randomized design at this time. True random assignment is

not feasible at the individual level, and random assignment of a few neighborhoods would not result in

robust findings.

Implement a Quasi-Experimental Evaluation

The next best option after a true experiment is a quasi-experimental design. In a quasi-experiment,

the evaluation could collect data in two sets of communities: one set implementing the CV program and

one set serving as a comparison group. Of course, many important challenges would remain. How many

sites around the country are willing to host an evaluation, and do those sites have the administrative

capacity to sustain the research design and to support the required data collection tasks?

We believe the evaluation must be undertaken in a city or set of cities that already have some CV

structure in place and an existing set of neighborhood programs capable of expanding their CV

operations. It takes a lot of effort to begin a CV program from scratch. Our guess is that it would take a

year of close implementation study before a brand new site could feasibly be ready for an evaluation. We

believe that the study sites should be located in “new” neighborhoods within cities that already have

established CV teams and experienced CV leadership.

Page 3 of 25

Based on these considerations and our review of data collected during the planning project (see the

Appendix), we recommend that the most rigorous outcome evaluation of the Cure Violence model

possible at this time is a quasi-experimental comparison design that includes the following features:

1. The study should rely on prospective data collection in at least three "new" program sites

chosen for (a) population size, (b) historical patterns of violence (especially frequency of

gun violence), (c) willingness to implement the Cure Violence model, and (d) proximity to

an existing and successful Cure Violence program. The optimal number of sites, given

unlimited resources, would be around 12 program sites (likely more) if we wanted to answer

many of the theory of change questions posed by CV leadership during the planning process.

Even with three sites, however, implementing a successful design with effective data

collection protocols will be challenging. Data should also be collected concurrently in

selected comparison jurisdictions (as many as possible). These sites should be selected to

match as closely as possible all pertinent characteristics of the implementation jurisdictions.

2. Implementation sites should be recruited using RWJF procedures for grant awards, but with

the close cooperation of the evaluation team and the Cure Violence national leadership team.

Participating sites must know that they are being recruited as sites for an evaluation and not

simply a programmatic initiative. We would recommend that the program sites be direct

grantees of the Foundation.

3. Implementation sites should be launched using procedures that are more intensive than the

methods usually managed by the Cure Violence national office (visits, training, and

technical assistance). Each start-up should be led and managed by at least one experienced

staff person from a nearby, successful Cure Violence program. None of the sites should start

"cold," with staff members that are all new to Cure Violence.

4. The evaluation team should begin its work in each implementation site and each comparison

site before CV implementation in order to collect baseline data.

5. The evaluation must include the thorough documentation of program implementation in each

CV site, including utilization of the databases already maintained by Cure Violence national

office. The Cure Violence staff, however, must be willing to support (and sometimes assist)

the evaluation team to implement any additional implementation measures in the three study

sites.

Page 4 of 25

6. Before any sites are selected, RWJF, CV staff and the research team should discuss

revamping the CV database. There are handful of "fixes" that could be made to the database

that would better enable the research team to track implementation and allow the staff the

flexibility to input the important measures related to CV inputs, outputs and outcomes. (See

the attached Appendix on recommended changes to the existing database.)

7. Implementation measures should encompass regular recording of all program activities,

including conflict mediations, outreach contacts, focus groups, hospital-based contacts, all

forms of community events and public education efforts. Hence, we need to make decisions

about automated data collection versus paper data collection

8. Finally, the evaluation should measure a range of CV program outcomes using several

strategies designed to fit each outcome. The strategies will include:

(a) analysis of official crime data from each study site and each comparison site,

both historically and during the study period (at a minimum shootings and all

crimes with firearms);

(b) analysis of the intensity of program engagement by Cure Violence participants

in each of the study sites, using the Cure Violence database but also continual

and systematic interviews and/or surveys with program staff who will be asked

to report the engagement of individual participants on at least a monthly basis;

(c) analysis of values, beliefs and attitudes about violence among high-risk males,

ages 18-35 (perhaps adding in the younger age group if IRB obstacles can be

overcome), in the study sites and comparison sites using Respondent-Driven

Sampling methods and subject samples approximately twice as large as those

likely to be required in simple random sampling, following the recommendation

of Salganik (2006);1

(d) analysis of the values, beliefs, attitudes, and behavior of CV participants measured

at least twice, but ideally three times during the course of the study with interviews.

Select Sites to Maximize Evaluation Design

How should the evaluation select its sites? There are several important questions that must be

addressed. What would the RFP look like? What should be required from site applicants? What is the

minimal level of existing gun violence necessary to qualify a community or neighborhood to host a

1 Salganik, M.J. (2006). Variance estimation, design effects, and sample size calculations for respondent-driven

sampling. Journal of Urban Health, 83(Supp 1): 98-112. (http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1705515/)

Page 5 of 25

randomized experiment? How much gun violence in an area (and for how long over time) is enough to

warrant an intervention like CV?

If RWJF chooses to use an RFP process to recruit neighborhoods/cities for new sites and for the

evaluation, we would suggest the following:

• Neighborhoods (average population size 10,000) should report at least 40 shootings a year.

This should be possible in most large cities. In Chicago, for example, there are 269 police

beats, and the average beat population is 9,980 (all ages).

• The cities hosting evaluation sites should have at least 6 to 9 months of previous

implementation experience, with stable OW staff, VIs and program manager in place.

• Sites should be capable of monthly data submissions to assess fidelity.

• Sites should commit to bi-weekly phone calls to discuss implementation status and

challenges.

For each site under consideration, the evaluation team should assess its strengths and weaknesses,

and these should be reviewed by an Advisory Committee. We would suggest obtaining answers to the

following questions:

• How faithfully is the CV model being implemented in the neighborhood? What are some

local aspects of the program that might deviate from the CV strategy?

• What implementation challenges are already being encountered by the host agencies and

other stakeholders?

• Are there organizational attributes of the program/partnership that may adversely impact the

implementation process? Could these attributes be altered before the beginning of the

evaluation?

• How likely is it that the key leaders will remain in their positions throughout the

implementation of the CV strategy and any planned evaluation?

• How might contextual factors (characteristics of the local economy, the community chosen

for enforcement activities, and the local target population) influence implementation? Can

current funding levels for Cure Violence be sustained?

Page 6 of 25

• Does the site have the capacity to collect performance data to monitor implementation? Is

the site using the Chicago Database? Is the site doing any additional data collection beyond

that required for the CV database? How is the site collecting mediation data? Are these data

consistent and reliable?

• Can the jurisdiction's existing data systems be used to diagnose implementation problems?

Will all the same data be available across all CV targeted neighborhoods, as well as any

control or comparison neighborhoods?

• Do the key agencies involved in the CV strategy have actual experience with collecting

performance data?

• Has the site added any new or additional program components, sanctions or services that

might contribute to a reduction in shootings but appear to be outside the CV strategy, such as

the prevention focused strategy in Crown Heights?

• Is the local police department willing to share crime incident data at the address/point level?

Is there a separate or different database that catalogs all shootings as opposed to using the

NIBRS categories for assault and homicides? What are the challenges to acquiring these

data?

• What do we know about the availability of hospital data for shootings? Is there a central

repository?

• Are weekly or monthly data available on police patrol resources by some aggregate policing

unit that is meaningful? Any data available that would show any changes in police

enforcement over time by local law enforcement?

• Ideally, an evaluation would have access to some form of consistent data for violent

incidents occurring at least 60 months prior and 30 months after to CV implementation. Is

this possible in the CV and comparison communities?

• Do partnerships between Federal, state and local law enforcement exist that might help an

evaluation team to obtain information about other violence-reduction initiatives in treatment

and control neighborhoods that might muddy the waters for a rigorous quasi-experimental

design?

Page 7 of 25

Evaluation sites should be selected after obtaining answers to these questions. At this time, however,

we think the best candidates for such a study would be neighborhoods in Chicago, New York City, and

Philadelphia.

Philadelphia Strengths:

• Reliable crime data and other data; data needed to cluster neighborhoods and assign is

already gathered (shootings, all census data; social and health data, parcel data).

• An entire section of Philadelphia (West Philly) could benefit from CV, but there is also a

large footprint already established by programs implementing the focused deterrence model

(i.e., David Kennedy) in South Philadelphia.

• There is a strong survey team at Temple University’s Institute for Survey Research (ISR).

Philadelphia Weaknesses:

• It is possible that Philadelphia will expand the Kennedy model throughout the city.

• The City appears to be more interested in expanding a competing model (YVRP) rather than

CV. Neither the State nor the City has expressed sustained interest in expanding/funding CV.

• Philadelphia does not have a particularly strong CV infrastructure at this time and needs

more staff. The program is still working through data collection consistency by OWs after 5

months.

New York Strengths

• Given the size of New York, there are many neighborhoods in which to place new

intervention programs and many available comparison areas.

• Both the City and the State are committed to the CV model and continue to provide financial

resources to support programs and evaluations of the model.

• A number of programs in the City have solid experience with the model; staff members have

been trained by the CV national office and it could be relatively easy to add new sites.

• John Jay College is based in New York, providing easy access to sites and few travel costs.

New York Weaknesses

• Some of the existing program organizations do not have strong relationships with law

enforcement.

Page 8 of 25

• New York politics are constant and complex and it could be difficult to control what happens

in comparison neighborhoods.

• Local organizational and political culture can vary significantly among communities.

Chicago Strengths

• There is strong CV leadership and the national office could provide almost daily support.

• New sites would most likely have the fewest startup problems compared to Philadelphia and

New York. This support is central to successful implementation.

• Local sites would likely have relatively quick startup times.

Chicago Weaknesses

• CV is well established. New sites might be contiguous to existing sites; contamination may

be likely given that staff already follow conflicts out of their set boundaries.

• Research team would have large travel costs (but could possible minimize these by using

local research staff and survey specialists—i.e., NORC).

• It could be difficult to find suitable comparison sites given the number of neighborhoods

already implementing the CV model

• Focused deterrence model is also gaining strength and is favored by current CPD leadership.

Once the final sites are selected, we propose that RWJF support an evaluation meeting or series of

meetings in the selected city(ies). These meetings should include key RWJF staff, several members of the

Advisory Committee, the Chicago CV staff, the evaluation team, and the leadership of local CV agencies,

city government, and police departments. The meetings should include a focus on the procedures for

limiting cross-jurisdiction contamination. Within 90 days of each meeting, the evaluation team should

draw up MOUs and have all documents signed and distributed to all partners. We also propose

developing a plan of action (or reflection) that provide an ongoing process for assessing serious threats to

validity as the programs are being implemented. We should have a pre-specified contingency plan in

place, and agreed upon by RWJF and the Advisory Group, for any case in which it appears that new or

large law enforcement efforts are getting ready to take place in the treatment areas.

Page 9 of 25

Measurement Issues

The Cure Violence approach to violence reduction and violence prevention stems from a public

health understanding of social problems. The Cure Violence model posits that violent behavior —much

like any other behavior— is learned, which means that individuals can unlearn it given the right

opportunity in the appropriate context. The principal way that people learn behavior is observation.

Violence is transmitted from one person to another by observation, interpretation, and imitation. Each

person’s support and acceptance of violence is derived from their experience of whatever social norms

appear to govern the use of violence, and the strongest influences over any one individual’s behavior is

the behavior that person observes in the immediate social environment. In other words, people behave

violently when they believe that violence is natural, acceptable, or even expected by their friends, family

and neighbors. If violence is normal, why avoid it? The Cure Violence (CV) model attempts to harness

the forces that shape behavior in a way that reduces the normality of violence and stops its transmission.

The CV model relies on three key elements to stop the transmission of violent behavior:2

1. interrupting transmission directly;

2. identifying and changing the thinking of the highest potential transmitters; and

3. changing group norms regarding violence.

Program Staff and Organizational Milieu

The three key components of the CV model are reflected in the type of staff working in CV program

sites. Some of the workers in a CV program site are hired to stop violent incidents through direct

intervention. These individuals, known as “violence interrupters” (VIs), are selected for their experience

and backgrounds. Often former offenders, former gang members, and/or formerly incarcerated persons,

VI’s work to establish relationships with the most high-risk young people in the community, usually

young men between the ages of 15 and 30. By forming relationships with high-risk and gang-involved

young men, VI’s monitor ongoing disputes and try to learn about potential acts of retaliation before they

happen. When one person is injured or shot, the victim's friends and known associates are likely to seek

revenge. The VI’s from Cure Violence sites seek out those associates and try to “talk them down,” or

persuade them that there are other ways to negotiate the conflict without engaging in more violence that

could risk their liberty and even their own lives.

2 See the website: http://cureviolence.org/

Page 10 of 25

Violence Interrupters must be carefully recruited. They need to be seen as “credible messengers” by

the most high-risk young people in the community. They need to know about the daily life of people who

are involved in criminal lifestyles. They cannot be judgmental, or be perceived as outsiders. Ideally, VI’s

will come from the same neighborhoods in which they are working, and they will have had some personal

experience with the justice system, perhaps as an offender or gang member or a formerly incarcerated

person.

Another key job in every CV program site is called an “outreach worker” (OW). Outreach workers

are similar to case managers. Like a VI, the OW needs to have trusting relationships with the most high-

risk individuals in the community, and it helps if the OW has also had prior justice system involvement.

Both the VI and the OW need to be seen as credible by the young people in the community living high-

risk lives. The daily tasks of an OW, however, are not as focused as those of a VI on monitoring threats of

violence and intervening directly to stop the transmission of violence. Instead, the OW uses his or her

relationships with program participants to help connect individuals with opportunities and resources in the

community. This would include things like employment, housing, recreational activities, and educational

opportunities. Outreach workers provide high-risk young people with positive alternatives to violent

lifestyles. The central goal of an OW is to facilitate the process by which potentially violent individuals

learn to think about violence differently and to change their behavior accordingly.

Outreach workers and violence interrupters work in teams along with their supervisors and program

directors. They meet on a regular basis —often once a day—to review their interactions around the

neighborhood and to discuss those individuals who are thought to present the greatest current risk of

violence. They compare notes on potential incidents of violence and the needs and interests of program

participants in order to match those participants with resources and opportunities in a positive way that

may draw them out of the violent lifestyle. The observations of the workers in a CV site are organized in

case review meetings and case planning sessions, and much of the information is recorded in a

continually updated database— albeit without identifiable information about individual participants.

High-risk individuals are referred to by the CV program using pseudonyms (e.g., Individual A, Individual

B) in order to preserve their anonymity and their cooperation.

While the VI and OW teams focus their efforts on young people who are most at risk of transmitting

violence, the Cure Violence program also pursues the third key element of the model. The staff works to

change social norms about violence in the broader community. A variety of program activities are

designed to focus on social norms, including media campaigns, signs and billboards, and public events

such as anti-violence marches and post-shooting vigils. Essentially, the CV program is willing to pursue

Page 11 of 25

any activity that exposes the larger community to an effective anti-violence message and that builds a

general social consensus against acts of violence. In this way, the Cure Violence model works at both

ends of the spectrum of behavioral transmission—both the senders and receivers of social messages

related to violence and the acceptance of violence.

Prior Research on Cure Violence

Evidence of the effectiveness of the Cure Violence model can be found in several recent studies that

documented the implementation issues and outcomes associated with the program in Chicago (still known

as Chicago CeaseFire), Baltimore (the Safe Streets Program), and New York City (Save Our Streets,

Brooklyn). A number of publications are available about these studies (e.g., Boyle et al. 2010; Engle et

al. 2011; Pichard-Fritsche and Cerniglia 2013; Ritter 2009; Skogan 2008; Skogan et al. 2009; Spergel

2007; Webster et al. 2012; Wilson et al. 2010).

CeaseFire-Chicago

In the first rigorous study of Cure Violence, researchers worked with a number of sites in Chicago to

track both process and outcomes of the program (Skogan et al. 2009). Process measures focused on

issues such as the site selection process, the selection and managerial quality of the host organization, and

staff training. Outcomes included shooting incident patterns in intervention neighborhoods compared with

trends in nearby areas that matched the characteristics of the selected sites. In addition, social network

analysis was utilized to document and assess changes in the density of gang violence and reciprocity of

homicides across gangs. Geographic analyses assessed changes in the size and intensity of hot spot areas

before and after pre and post intervention.

In all, 21 local sites classified as “high need” communities were selected for the study. The sites

faced implementation obstacles almost immediately. Neighborhoods were selected based on their high

rates of violence, but the communities tended to be high-poverty areas as well. Skogan and his colleagues

noted that the selected neighborhoods were inevitably harder to serve due to high levels of

disorganization and a dearth of community leaders willing to serve as hosts for violence-reduction work.

For many neighborhoods, the CeaseFire initiative was a new program, taking a new approach to the

reduction of violence. Some community leaders were cynical about CeaseFire from the very beginning of

the initiative. Cure Violence leaders vetted agencies to determine the most appropriate to be the host

agency in each neighborhood.

Program participants (i.e., clients) were recruited through outreach workers in each neighborhood.

The focus was on high risk individuals who were involved in street violence. A survey of program

participants by the research team indicated that outreach staff were successful in reaching their targeted

Page 12 of 25

clientele. More than four of every five (82%) clients had been arrested previously. Nearly half (45%) had

five or more prior arrests and more than half (56%) reported spending time in jail in the past. Nearly all

participants (96%) were racial or ethnic minorities (70% Black; 26% Hispanic).

CeaseFire participants believed the program was “very important” despite the large number of

obstacles and pressures they faced in their day to day lives (Skogan et al. 2009). The participants viewed

the violence interrupter staff, in particular, as essential to the program. As former offenders, they brought

a level of credibility with the local population that the outreach workers did not always have. The

interrupters were often more successful in getting across the program’s anti-violence message to

participants, and the study suggested that they were very helpful in diffusing the type of violent

confrontations that often lead to retaliation. Moreover, the personal networks of violence interrupters

sometimes crossed geographic boundaries and their collaborations revealed the underlying dynamics of

conflicts that would have been missed by an exclusive focus on the violence within one neighborhood

alone. Skogan and his colleagues estimated that as much as 70 percent of the individuals and 60 percent

of the conflicts mediated by CeaseFire workers originated outside of the target areas designated for

particular interrupters.

The results of the Chicago impact study were mixed. On one hand, shootings and homicides declined

in most of the intervention sites. On the other hand, there were a number of viable alternatives that could

explain the degree of change and the scale of program effects. Crime was decreasing across the board and

continued to do so, not only in the intervention sites but elsewhere. Only seven of the 21 program sites

were able to produce enough data to examine crime trends and possible covariates in any detail during the

study time period. Researchers were unable to obtain data with enough detail to address why and how

crime fell in each neighborhood. Furthermore, inconsistencies in the actual time of program

implementation in specific sites made trend analysis problematic and not reliable. Possible variations in

the strength of programs across different sites could not be measured accurately given the level of

resources devoted to the evaluation. Finally, the violence interrupter component of the program could not

be tracked to a specific target location because —as noted above— the work of the interrupters spilled

over geographic borders.

Other important limitations of the evaluation include the following:

• The study did not evaluate changes in community norms regarding gun violence;

• The study did not assess the impact of program involvement on behavior change among

participants;

Page 13 of 25

• The study did not study the behavior change process and norm change process through

ethnography of some type of in-depth qualitative study;

• The study did not link monthly implementation data to changes in shootings; and

• The study did not systematically compare intermediate outcomes or performance measures

with outcomes.

Safe Streets-Baltimore

Following the Chicago evaluation, another study emerged from Baltimore where public health

researchers at Johns Hopkins University measured the effects of the “Safe Streets” intervention program.

Safe Streets was a replication of the Cure Violence strategy and the elements of the study were essentially

similar, with the main components including: 1) a review of implementation data; 2) an analysis of

differences in homicides and non-fatal shootings between implementation areas and nearby comparison

areas; 3) surveys of community attitudes toward gun violence; and 4) surveys of program participants that

measured their opinions and perceptions about the program and its impact on their lives (Webster et al.

2012; 2013).

Safe Streets attempted to gain traction in five neighborhoods, but in one (Union Square) the effort

was unsuccessful as it proved too difficult to establish a reliable group of outreach workers (Webster et al.

2012). Ultimately, the intervention program was implemented in four neighborhoods, all located in

Southwest and East Baltimore. As part of the evaluation, border neighborhoods were examined for trends

associated with crime mobility and spatial lag effects, but only informally. The study’s formal comparison

neighborhoods were dispersed throughout the larger Baltimore area. Project Safe Streets began by

soliciting proposals from community organizations in Baltimore’s most violent neighborhoods. Learning

from some of the issues that were problematic in the Chicago study, the Baltimore program staff was

instructed to keep detailed records of each conflict mediated by the program. The results indicated that

over a 3.5 year period, 276 incidents were successfully mediated; 88 percent of the conflicts involved

individuals with violent histories and 75 percent involved known gang members. Webster and his

colleagues reported that outreach workers believed that nearly all (85%) of the mediations involved

incidents that were “likely” or “very likely” to have resulted in shootings had they not been interrupted.

To evaluate the Baltimore program, shooting and homicide data from the intervention sites were

compared with data from other precincts that ranked in the top 25 percent of city precincts for number of

shootings during the 2003-2006 period. Similar to the Chicago results, the findings of the Safe Streets

evaluation were mixed. Overall, there was very little deviation from general trends during the

Page 14 of 25

implementation of the project. In some cases, however, homicides spiked at the same time that non-fatal

shootings decreased significantly. Ultimately, this mixed pattern was seen as the norm in the final project

evaluation. In all four neighborhoods, one of the two gun violence indicators improved, but not always

the same indicator. The researchers believed that the inconsistent pattern was likely due to bias from

unmeasured confounders and they offered a series of anecdotal explanations for the pattern. In the end,

the final conclusion from the Baltimore project was similar to that of the Chicago study—there were some

promising trends, but they could not be attributed reliably to the intervention and they could not be

disaggregated from the ongoing national decline in violence.

Save Our Streets (SOS), Brooklyn

The Cure Violence strategy was implemented in Brooklyn, NY, by a program known as Save Our

Streets (or, S.O.S.). The S.O.S. program was launched by the Crown Heights Community Mediation

Center and the Center for Court Innovation (CCI) in early 2010 and later evaluated by a team of CCI

researchers (Picard-Fritsche and Cerniglia 2013). The S.O.S. evaluation set out to follow the same quasi-

experimental design as the Chicago study in order to replicate the findings. The intervention model,

however, was not exactly the same. In the Brooklyn project, the outreach workers also performed the

work of violence interrupters. The research team believed that the alteration did not affect the quality of

the conflict mediation activities in a negative way.

The Brooklyn evaluation found evidence to suggest that gun violence decreased during the

intervention period while it increased in proximate comparative neighborhoods, but the reductions were

not statistically significant. Still, the fact that gun crime appeared to decline in the intervention area while

not declining elsewhere was seen as a promising result, especially when coupled with the fact that

Brooklyn as a whole saw increases in shootings during that time. The community measures from the

study were also encouraging, although they were not obtained with highly rigorous methods. Surveys of

community residents showed an increase in awareness of the project’s mobilization against gun violence

(up from 27% to 73%) and increased confidence in the SOS program itself (from 29% to 55%). However,

the surveys indicated that community residents did not feel any safer and they actually supported the right

to carry a gun if they had witnessed a gun-related crime in the past.

Following the Logic of the Cure Violence Model

The CV model is different than other models of violence reduction. It is a public health approach that

relies less on the coercive power of law enforcement and more on the normative power of the social

environment. The CV approach is to: 1) identify the source of unwanted behavior (i.e., individuals

engaged in violence); 2) work intensively and without judgment to encourage alternate forms of behavior

Page 15 of 25

among those individuals; and 3) spread awareness and knowledge among the larger community about the

negative impact of the behavior as well the effectiveness and benefits of methods known to reduce it. The

evaluation design must follow the logic of the Model itself.

The key workers in Cure Violence programs —outreach workers and violence interrupters— reduce

the social harm of violence by forming relationships with individuals already engaged in violence and

those most likely to become involved. They rely on these relationships to learn about and then stop

potential retaliatory acts of violence, and they expose violent participants to different ways of thinking

about violent behavior and alternative methods of solving disputes and conflicts. The evaluation design

must attempt to measure the quality and intensity of these relationships.

Of course, the fact that the CV approach requires a staff that includes “credible messengers”

introduces complications. By hiring staff members with extensive justice experience, there is a chance

that they may still be involved in criminal behavior— and whether they are still involved or not, the fact

of their prior criminal records increases the scrutiny of policymakers and funding authorities. Public

officials and government agencies are likely to react very strongly to even the slightest impropriety

involving a Cure Violence program. In addition, and perhaps because of this added scrutiny, the staff

members of CV are likely to be more skeptical of outsiders, including the evaluation project.

Without the inclusion of former offenders and formerly incarcerated persons as staff, however, the

Cure Violence strategy would likely not be as effective. Violence interrupters and outreach workers are

able to obtain information about high-risk communities and the residents of those communities that would

be unreachable by conventional entities, such as law enforcement authorities and social service agencies.

The workers in a CV program have more trusting relationships with community residents and they can

identify ongoing conflicts that would be invisible to the workers in more conventional program models.

This feature, however, is also a serious obstacle for evaluation. If the effectiveness of the model

depends on the informality and confidential relationship of CV staff and program participants, it is

challenging to conceive of a method for measuring program activities and outcomes at the individual

level without generating negative reactions and suspicions from both the staff and clients of CV

programs. The natural instinct of evaluators in this circumstance would be to avoid the potential for

conflict and to treat program activities as a black box while measuring outcomes only at the level of

neighborhoods and communities. In fact, this is what previous studies of the Cure Violence approach

have done.

Page 16 of 25

The key challenge in evaluating Cure Violence is to create a study design that can measure all the

key components of the program and their effects on the model’s intended outcomes. The program

hypothesizes that community violence can be reduced through two causal pathways: 1) by changing the

violent behavior of individuals (e.g., active criminal participants, gang members, and high-risk youth);

and 2) by changing the broader social norms that perpetuate violence. The activities pursued by Cure

Violence programs are designed to focus on both of these causal pathways. Staff work actively with

violent individuals and gang members to prevent ongoing violence, and at same time they participate in

public education campaigns, post-shooting vigils, and other public demonstrations of support for

nonviolence. The program basically attempts to achieve violence reduction by working at both ends of the

causal spectrum— both individual behavior change and the broader area of social norms.

The following theoretical framework was developed by the research team in order to facilitate

analysis of the Cure Violence model and to plan for an effective evaluation design.

Theoretical Framework for Cure Violence (CV) Model

Source: Research & Evaluation Center, John Jay College of Criminal Justice

Page 17 of 25

The theoretical framework portrays the two principal pathways of hypothesized program effect. The

path in shades of blue depicts how program activities lead to changes in individual behavior, both among

CV program participants as well as other high-risk youth from the same neighborhoods. The path in

varying shades of green describes the way that the CV model is hypothesized to affect social norms.

Evaluation Challenges

The key challenge facing an evaluation of Cure Violence is how to measure both of these causal

pathways at the same time, with sufficient detail, and in a way that is cost feasible. Measuring individual-

level behavior change can be very expensive if a study is designed to collect primary data from

individuals themselves rather than relying on imperfect proxy measures from bureaucracies (e.g., policy

and courts). To collect detailed information about personal behavior, the study will have to conduct

interviews (or surveys) with active participants and with repeated measurements in order to detect

behavior change.

This is an expensive and labor-intensive process even with very cooperative and accessible research

subjects. The participants in CV programs, however, are not likely to be motivated to collaborate with

researchers and an evaluation team cannot simply call up and get a list of local gang members and send

them e-mail invitations. Some primary data collection from high-risk individuals is essential in an

evaluation of Cure Violence, however, because the program is designed to intervene in behaviors,

attitudes and norms that are often unknown and may never be known to the formal bureaucracies of social

services and law enforcement.

The Cure Violence evaluation will most likely have to use some form of interview. In addition to the

literacy problems that would likely prevent the use of written surveys in such a high-risk population, the

participants in a Cure Violence program will not likely want to share their real identities and contact

information with program staff. The research team will have to be very careful to establish their

independence from the program. Moreover, the researchers will need to devise some way to make

appointments with interview subjects, and if the evaluation team relied on the program to coerce

participants into appointments, they would be undermining the very nature of the program strategy they

were trying to evaluate. Thus, interview contacts between researchers and participants will have to happen

completely separately from the program. Interviews may also have to take place during more than one

interaction. Participants in a Cure Violence program are unlikely to be willing to sit through a long

protocol of questions. The research team may have to collect interview data in short, intermittent

encounters with participants. This approach will involve considerable labor costs, as the interviewers

Page 18 of 25

would have a lot of “downtime” when they would be on the job but without continual contact with

research subjects.

An attractive alternative to face-to-face interviews would be to use Internet-based surveys with touch

screens and sound. This could help to overcome literacy problems and might be more engaging for the

respondents. This strategy, of course, would add to the project costs and it also presents challenges.

Where would the subjects complete the survey? Could they use portable devices such as smart phones or

tablet computers? Who would maintain and control access to the equipment? How much theft would

occur? Even if the process was successful, how would the study ensure the identity of each subject when

the person signed on to the survey site? There would still be high labor costs in using the survey method.

A number of sampling challenges would also confront such an evaluation. How does one identify a

program participant? Would any young person who has had any contact at all with a violence interrupter

qualify, for example, or would they have to be more deeply involved and have a longer relationship with

an outreach worker? If a known gang member is well acquainted with other program participants, but has

never interacted with any staff from the Cure Violence program, that person should qualify as a research

subject and be eligible for the interview portion of the study, but they should be considered a non-

participant, high-risk subject.

The Cure Violence Model is intended to affect the behavior and attitudes of not only program

participants, but the broader members of their social networks as well. A sampling design for using

interviews and surveys in an evaluation of Cure Violence will need to distinguish at least three types of

research subjects: 1) program participants; 2) other high-risk individuals who are known to and socially

networked with program participants; and 3) the broader resident populations of high-risk communities.

Surveying or interviewing each of these three groups presents differing issues related to complexity and

cost.

Data Strategies

The evaluation project will need to use different data collection strategies for several areas within the

theoretical framework. For example, in the social norm pathway (Green), many of the intermediate

outcomes and final outcome are subjective perceptions and beliefs of community residents. One of the

intermediate outcomes refers to the residents of a CV community developing the “self-efficacy” (or

confidence in one’s own ability) to bring about change in public safety and violence. To measure this

construct with enough precision to detect change, the evaluation will have to conduct a probability sample

Page 19 of 25

of neighborhood residents, asking questions about perceived community safety and the respondent’s

belief that conditions could ever improve.

Social Norm Outcomes

The activities leading up to these outcomes could be largely measured using objective data. The role

of “community outreach and focus groups,” for example, could be estimated with simple counts of these

events, provided that CV program sites were diligent about recording and reporting such events.

Similarly, the evaluation could track the frequency of public education campaigns by recording how often

and where they occur. The bottom two activities in the social norm pathway (coordination with other

community groups and relationships with law enforcement and political leaders) could be measured with

a combination of objective data (counts of communications and meetings) and subjective data (surveys of

agency staff and local officials).

Social Norm Activities

In the behavior change pathway, some of the measurements would be relatively straightforward and

several are already tracked routinely by the databases of the CV national program office. The evaluation

could count the frequency of contacts between outreach workers and CV participants with the database,

for example. The evaluation team, of course, would have to work with each evaluation site and the CV

national staff to ensure that the counts were done regularly and reliably.

Could be

measured

with program

data

Will require

surveys of

local

agencies

Page 20 of 25

The same approach could be used to monitor the street mediations, hospital interventions and other

groups and meetings with CV participants and high-risk youth. Each CV program site is likely already

recording these events. The evaluation team would just have to make sure that everything was recorded

using consistent methods and that the data were captured on regular basis.

Behavior Change Activities

Measuring outcomes in the behavior change pathway would be more complicated. It would also be

the most costly portion of the study. The first consideration would be to decide whether the contribution

of program activities should be measured at the individual level in order to establish their empirical

relationship with individual behavior. In other words, would it be sufficient to know that the behavior of

individuals changed more in jurisdictions with stronger CV implementation, or should the evaluation

track each individual’s engagement with the various activities in order to establish their association at the

level of the individual? To do the latter would add significant complexity to the data collection contacts

with CV participants and other high-risk youth.

Behavior Change Outcomes — CV Participants

If data collection tasks for behavior change outcomes did not involve the measurement of program

activities, the tasks involved would focus on some relatively conventional measures and some more

challenging measures. For example, the two outcomes referring to CV participants’ improved access to

Could be

measured

with program

data

Could be

measured

with program

data

Will require

youth

interviews

Page 21 of 25

educational and employment opportunities would be fairly simple. During the interviews with youth, the

research team could ask a series of questions about these topics. In addition, the evaluation study could

ask the CV staff to include regular reports about whatever knowledge they had of participant involvement

in educational and employment activities.

The third behavior change outcome from the top, “CV participants form pro-social bonds and

relationships,” could be measured using a fairly easy method (by asking CV staff to report on the quality

and frequency of their contacts with youth) or a more complex method (including questions about

relationships and social bonds in the youth interviews). The more complex method, however, is preferred

required if the logic of the CV theoretical framework is to be followed. This particular casual pathway is

expected to have a secondary influence on the other high-risk youth in the community (i.e., non CV

participants). Thus, this construct should be measured at the highest level of precision possible.

The framework requires a parallel data collection effort for two additional outcomes among both CV

participants and other high-risk youth who are not CV participants. The evaluation design must provide

for the measurement of these outcomes (learning “nonviolent conflict skills,” and adoption of the “goals

of Cure Violence”) using face-to-face interviews, as there will be no other source of data about these

outcomes for non-participants.

Behavior Change Outcomes — Non-Participants

Finally, the last three outcomes in the behavior change pathway present different levels of difficulty.

One of the penultimate outcomes (“CV participants avoid situations involving the risk of violence”) could

be measured during youth interviews, but it could also be estimated using law enforcement or court

referral data, as one could assume that avoiding the risk of violence would result in fewer contacts with

the justice system. Of course, measuring this outcome would require identifiable data, which could

introduce complications. The other penultimate outcome (“High-risk youth apply nonviolent approaches

to conflict resolution”) must be measured through the youth interviews.

Will require

youth

interviews

Page 22 of 25

The last and final outcome in the behavior change pathway is the most traditional outcome which is

measured by all violence reduction evaluations. The hypothesized reduction in violence would be

measured using official data about gun crimes, shootings, and homicides. While conceptually easy to

imagine, measuring this outcome will still require considerable effort on the part of the study team. Few

cities are both able and willing to generate crime data at the level of neighborhoods or even census tracts.

The study team will have to negotiate access to these data in each CV site. (Note: The R&E Center at

John Jay College already has this information for New York City from 2007 to the present day).

Behavior Change Outcomes — Final Outcomes

Data Analysis

The analysis methods to be used in the evaluation will likely focus on interrupted time series analysis

and a “difference in difference” analysis to detect the differences between CV sites and comparison sites

over time. Before finalizing the analytic plan, we would also work the RWJF evaluation Advisory Panel

to design the overall approach. The study would have to be flexible and consider a range of analyses, but

the main tests of CV effectiveness would involve a number of designs and analytical options, recognizing

that it must be possible to examine the differences in violence in CV sites before and after

implementation, relative to the same difference in comparison sites. The evaluation team would hope to

have at least 60 months of administrative data pre-intervention, and 30-36 months post CV

implementation, along with the study’s own interview, surveys, and observational data. This would be

consistent with previous studies. The Chicago evaluation, for example, relied on data representing an

average of 59 months for the post-implementation period (range 33-79 months) and 192 months overall.

The analysis plan will also have to incorporate monthly performance data, as well as any time-series data

on resources or outside interventions that might influence the effects of Cure Violence interventions.

Requires interviews

Administrative data or interviews

Administrative

data

Page 23 of 25

Establishing the Potential Value of the Cure Violence Model

At its heart, Cure Violence is a community-based violence intervention—but much of its theory of

change is focused on changing the behavior of individuals. Community-based interventions that are

expressly intended for community-level change are typically measured by examining hypothesized

reductions in aggregated criminal incidents across some geographic unit (e.g., neighborhood, police beat,

etc). Past evaluations of Cure Violence and other popular violence reduction strategies have largely

focused on such changes in aggregate neighborhood units and have not examined behavior change across

individuals affected by the program. These studies leave many questions unanswered about the workings

of the model. Is violence affected at the community level because a large number of individuals are

directly influenced by the program to stop shooting, or are others in the community hearing and seeing the

message and their values and attitudes eventually alter group and larger social units?

The central purpose of doing an evaluation of Cure Violence is to develop sound methods for

replicating the program in other jurisdictions. The evaluation study will need to document enough detail

about the program and its implementation to facilitate the transfer of that knowledge to new communities

and new jurisdictions. It may not be sufficient to demonstrate that the Cure Violence strategy worked in a

few sites. An evaluation would have to show exactly how the program reduces violence and to point out

which practices are and which are not related to the program’s effectiveness.

The findings of the three most prominent studies of Cure Violence to date are mixed. All three of the

evaluations revealed at least some evidence that supports the approach, but none of the studies could

clearly disentangle the results from national and regional trends in violent crime, and there were always

“confounding” effects from factors related to sample design, participant selection, and variations in

implementation. Of course, this same criticism could (and is) leveled at the evaluations of other

comprehensive violence reduction programs, even those widely perceived as successful (e.g., the Boston

Gun Project, the Comprehensive Gang Model, etc).

As described in the Department of Justice’s “crimesolutions.gov,” the database that chronicles and

synthesizes evidence on criminal justice-related prevention and intervention programs, the Cure Violence

approach currently merits the label “promising” rather than effective, but it has something else that no

other well-known violence reduction model has—it can be very cost-efficient and it places much less

demand on the political and administrative resources of law enforcement and the larger criminal justice

system. For this reason alone, the model deserves additional investment and investigation.

Page 24 of 25

Examples of disruptive innovations

Disruptor Disruptee

Personal computers Mainframe and mini computers

Cellular phones Fixed line telephony

Community colleges Four-year colleges

Discount retailers Full-service department stores

Retail medical clinics Traditional doctor’s offices

Source: www.claytoncchristensen.com

The Cure Violence model may be viewed as a type of “disruptive innovation.” Popularized by the

business academic, Clayton Christensen, the concept of disruptive innovation describes changes in a

marketplace that introduce new products that alter traditional assumptions (Christensen et al. 2006). Not

all innovations are disruptive; most are “sustaining.” Christensen suggests that disruptive innovations are

not simply improvements to existing arrangements —i.e., the proverbial “better mousetrap.” Disruptive

innovations are produced by market environments in which an existing product or solution has become so

entrenched, so expensive, and so complicated, that fewer and fewer people or organizations are able to

afford it or access it. Rather than simply joining the competition to serve an increasingly smaller niche of

elite consumers (i.e., with ever-better sustaining innovations), a disruptive innovation emerges to bring an

entirely new method or product to what has become a much larger, underserved market.

In the market for violence

prevention and violence reduction, the

sustaining innovations that long

dominated the market would be more

effective approaches to law enforcement

as well as the growing number of

individualized, therapeutic treatments.

For more than thirty years, policymakers

in the U.S. have been willing to spend

considerable amounts of money to pursue

stronger and more certain law

enforcement (e.g., policing, prosecution, and imprisonment). At the same time, the helping professions

have been working closely with researchers to expand upon the increasingly expensive array of research-

based programs for treating violent behavior at the individual level.

The Cure Violence model, on the other hand, may be the disruptive innovation that offers a valuable

and desired product (safety) to larger segments of the market (poor communities) that have been under-

served and out-priced by suppliers focused on meeting the demands of a smaller, elite market. Of course,

even a disruptive innovation has to be effective in order to succeed. Thus far, researchers have found

sufficient evidence for the efficacy of Cure Violence (it could work under ideal conditions). The next task

is to establish its effectiveness (it actually does work in practice).

Page 25 of 25

References

Boyle, D.J., Lanterman, J.L., Pascarella, J.E., & Cheng, C.C. (2010). The Impact of Newark’s

Operation CeaseFire on Trauma Center Gunshot Wound Admissions. Justice Research and Policy, 12(2),

105-123.

Christensen, C.M., Baumann, H., Ruggles, R. & Sadtler, T.M. (2006). Disruptive Innovation for

Social Change. Harvard Business Review, December 2006: 2-8.

Engel, R.S., Tillyer, M.S., Corsaro, N. (2011). Reducing Gang Violence Using Focused Deterrence:

Evaluating the Cincinnati Initiative to Reduce Violence (CIRV). Justice Quarterly,

DOI:10.1080/07418825.2011.619559.

Picard-Fritsche, S., & Cerniglia, L. (2013). Testing a Public Health Approach to Gun Violence. New

York: Center for Court Innovation.

Skogan, W.G. (2008). Brief summary: An evaluation of CeaseFire-Chicago. Evanston, IL: Institute

for Policy Research, Northwestern University.

Skogan, W.G., Hartnett, S.M., Bump, N., & Dubois, J. (2009). Evaluation of CeaseFire-Chicago.

Washington, D.C.: U.S. Department of Justice, Office of Justice Programs, National Institute of Justice.

Spergel, I.A. (2007). Reducing Youth Violence: The Little Village Gang Project in Chicago. Lanham,

MD. Rowman Altamira.

Webster, D.W., Whitehill, J.M., Vernick, J.S., & Parker, E.M. (2012). Evaluation of Baltimore’s

Safe Streets Program: Effects on Attitudes, Participants’ Experiences, and Gun Violence. Baltimore, MD:

Johns Hopkins Center for the Prevention of Youth Violence.

Webster, D.W., Whitehill, J.M., Vernick, J.S., & Curriero, F.C. (2013). Effects of Baltimore’s Safe

Streets Program on Gun Violence: A Replication of Chicago’s CeaseFire Program. Journal of Urban

Health, 9(1): 27–40.

Weisburd, D. and J. Eck. (2004). “What Can Police Do to Reduce Crime, Disorder, and Fear?”

Annals of the American Academy of Political and Social Science, 593: 42 – 65.

Wilson, J.M., S. Chermak, & E.F. McGarrell. 2010. Community-based Violence Prevention: An

Assessment of Pittsburgh’s One Vision One Life Program. Santa Monica, CA: RAND.