Performance Measurement for Justice Integration Projects The Illinois and Cook County Experience Mark Myrent Illinois Criminal Justice Information Authority

Performance Measurement for Performance Measurement for Justice Integration ProjectsJustice Integration Projects

The Illinois and Cook The Illinois and Cook County ExperienceCounty Experience

Mark MyrentMark MyrentIllinois Criminal Justice Information AuthorityIllinois Criminal Justice Information Authority

June 20, 2005 2IL Criminal Justice Information

Authority

What is Performance What is Performance Measurement?Measurement?

Performance Performance measurementmeasurement can be can be

defined as the process defined as the process of routinely measuring of routinely measuring

the outputs and the outputs and outcomes produced by outcomes produced by

a project, thereby a project, thereby allowing one to assess allowing one to assess

the effectiveness of the effectiveness of project investments and project investments and

activitiesactivities


Authority

Why Measure Why Measure Performance?Performance?

Build consensus & commitment Build consensus & commitment within the justice communitywithin the justice community

Obtain & allocate resourcesObtain & allocate resources Plan and manage project Plan and manage project

executionexecution Demonstrate success & Demonstrate success &

improve accountabilityimprove accountability


Authority

A Method for Measuring A Method for Measuring SuccessSuccess

Statement of business Statement of business problemproblem

GoalsGoals ObjectivesObjectives

Project objectivesProject objectivesFunctional objectivesFunctional objectivesBusiness objectivesBusiness objectives


Authority

Developing Business Developing Business ObjectivesObjectives

Identify a BASIC MEASUREIdentify a BASIC MEASURE Indicate DIRECTION of the measureIndicate DIRECTION of the measure Identify the OBJECT of the measureIdentify the OBJECT of the measure Identify the expected VALUE of the Identify the expected VALUE of the

measuremeasure Identify WHERE the measurement will Identify WHERE the measurement will

occuroccur Identify WHEN the measure will be Identify WHEN the measure will be

obtainedobtained


Authority

The LOGIC MODEL APPROACH

ACTIVITIES OUTCOMESINPUTS OUTPUTS

INPUTS are your resources, while OUTPUTS are the product of an activity

OUTCOMES are not what the program or initiative itself does, but the consequences of the program/initiative efforts -- changes in conditions, attitudes or behavior of individuals or outcomes foragencies and communities.

The STRATEGIC PLAN and THEORY OF CHANGE links these together


Authority

GOALS

In order to solve these

problems

Activities

We will do the followingactivities and

events

Targets

For thesepeople and

for this amount of

time

Initial/Short &Intermediate

Term Outcomes

We will know these changeshave occurred

If:___________

THEORYOf

CHANGE

This activitywill lead to changes in

these factors__________,which in turnwill lead to

solving theseproblems

LONGTERM

OUTCOMES

We will knowwe are

reachingour goals

If:___________

Thinking Through a Logic Model and Articulating a

Theory of Change


Authority

Developing Business Developing Business ObjectivesObjectives

Identify a BASIC MEASUREIdentify a BASIC MEASURE Indicate DIRECTION of the measureIndicate DIRECTION of the measure Identify the OBJECT of the measureIdentify the OBJECT of the measure Identify the expected VALUE of the Identify the expected VALUE of the

measuremeasure Identify WHERE the measurement will Identify WHERE the measurement will

occuroccur Identify WHEN the measure will be Identify WHEN the measure will be

obtainedobtained


Authority

The Illinois Experience: The Illinois Experience: Statement of Statement of

Business ProblemBusiness Problem

The absence of standardized The absence of standardized arrest booking procedures arrest booking procedures creates inconsistencies in the creates inconsistencies in the criminal history reporting criminal history reporting process, which contributes to process, which contributes to breakdowns in the quality of breakdowns in the quality of Illinois rap sheets.Illinois rap sheets.


Authority

The Illinois Experience: The Illinois Experience: Strategic Goal for Strategic Goal for Business ProblemBusiness Problem

Recommend, and implement Recommend, and implement mechanisms and processes mechanisms and processes to inventory, develop, adopt, to inventory, develop, adopt, publish, disseminate, and publish, disseminate, and maintain standards that maintain standards that apply to justice information apply to justice information sharing.sharing.


Authority

Ilinois Misstep: Ilinois Misstep: Performance Measures Performance Measures

from Strategic Planfrom Strategic PlanObjective:Objective: By July 2003, Standards By July 2003, Standards Committee to the Governing Board will Committee to the Governing Board will identify procedures for evaluating, identify procedures for evaluating, developing, approving, disseminating, and developing, approving, disseminating, and maintaining standards. maintaining standards.

Performance measures:Performance measures: Percentage of stakeholders represented & Percentage of stakeholders represented &

providing input on advisory committeeproviding input on advisory committee Number of standards adopted by IIJIS Gov. Bd.Number of standards adopted by IIJIS Gov. Bd. Percentage of IL agencies employing justice Percentage of IL agencies employing justice

information standardsinformation standards


Authority

ProjectInitiation

StrategicPlanning

TacticalPlanning

Project

Project

Project

Strategic Planning is only the Start of Integrated

Justice Planning


Authority

Performance measures Performance measures derived from strategic derived from strategic planning are different planning are different

from those derived from from those derived from tactical planningtactical planning

Strategic plans are often written at a a very high Strategic plans are often written at a a very high level in order to convey the conceptual level in order to convey the conceptual framework (particularly in state-level plans)framework (particularly in state-level plans)

Strategic performance measures often address Strategic performance measures often address pre-implementation “process” activities, thus pre-implementation “process” activities, thus producing a preponderance of “outputs” rather producing a preponderance of “outputs” rather than “outcomes” than “outcomes”


Authority

New Illinois Direction: New Illinois Direction: Business Objectives as Business Objectives as Performance Measures Performance Measures

(from Tactical Plan)(from Tactical Plan)Proposed Solution for Business Problem:

Expand the pilot project in Kane County in which:

1. Individual police departments submit data file containing arrestee charges, demographics, and tracking number to state CHRI repository

2. After file is posted to CHRI database, it is instantaneously sent electronically to court clerk’s office, along with fingerprint ID number, and is downloaded to clerk’s database, without the need for additional data entry.


Authority

Developing Business Developing Business Objectives as Objectives as

Performance MeasuresPerformance Measures

1. Identify a basic measure:

Actual Measure: Percentage of offender records in CHRI system having final dispositions


Authority



2. Indicate direction of the measure:

Actual Measure: Increase the percentage of offender records in CHRI system having final dispositions


Authority



3. Identify the object of the measure:

Actual Measure: Increase the percentage of felony arrests in CHRI system having final dispositions


Authority



4. Identify the expected value of the measure of the measure:

Actual Measure: Increase the percentage of felony arrests in CHRI system having final dispositions to 95%


Authority



5. Identify where the measurement will occur:

Actual Measure: Increase the percentage of felony arrests in CHRI system having final dispositions to 95% in the target county


Authority


Performance MeasuresPerformance Measures6. Identify when the measurement will be

obtained:

Actual Measure: Increase the percentage of felony arrests in CHRI system having final dispositions to 95% in the target county within the 1st year after project implementation


Authority

Lessons from the Field: Lessons from the Field: Performance measures Performance measures cannot be objectively cannot be objectively selected; stakeholders selected; stakeholders must reach consensusmust reach consensus

Stakeholders, depending on their role within the Stakeholders, depending on their role within the justice enterprise, are likely to have strongly justice enterprise, are likely to have strongly divergent views divergent views

Higher-level goals and outcomes will usually be Higher-level goals and outcomes will usually be easier to agree upon than specific performance easier to agree upon than specific performance measuresmeasures

Performance measure selection, however, is not Performance measure selection, however, is not entirely subjective, and some measures are entirely subjective, and some measures are likely to be agreed upon by alllikely to be agreed upon by all


Authority

Lessons from the Field: It Lessons from the Field: It is nearly impossible to is nearly impossible to definitively establish definitively establish

causal linkages to public causal linkages to public safety outcomessafety outcomes

Outcomes become more diffuse the further out Outcomes become more diffuse the further out you get from actual processesyou get from actual processes

Improved information sharing is only one of a Improved information sharing is only one of a number of factors affecting public safetynumber of factors affecting public safety

A single performance measure may be inadequate A single performance measure may be inadequate to indicate that a system is working (or not)to indicate that a system is working (or not)

A carefully selected family of measures is A carefully selected family of measures is preferable, and should be thoroughly tested before preferable, and should be thoroughly tested before being deployed.being deployed.


Authority

Lessons from the Field: Lessons from the Field: Public sector outcome Public sector outcome measures are driven by measures are driven by

different factors than for different factors than for the private sector the private sector

Greater integration success by private industry due to Greater integration success by private industry due to clearer lines of accountabilityclearer lines of accountability

While the private sector is driven by the bottom line, While the private sector is driven by the bottom line, government is driven by the competing needs of many government is driven by the competing needs of many agencies with differing missions and purposes; lines of agencies with differing missions and purposes; lines of accountability are therefore more diffuse.accountability are therefore more diffuse.

Government is accountable not only to the governor, Government is accountable not only to the governor, legislature, and public officials, but to the public as welllegislature, and public officials, but to the public as well

Concern that the public believes that the justice system Concern that the public believes that the justice system works as it is portrayed on television creates another works as it is portrayed on television creates another type of accountabilitytype of accountability


Authority

Performance Measurement Performance Measurement CaveatsCaveats

Most people (including your employees and Most people (including your employees and consultants) can learn to make the measures come consultants) can learn to make the measures come out the way they think out the way they think youyou want them to, while want them to, while exerting a minimum of effort in actually improving a exerting a minimum of effort in actually improving a processprocess

Always question the measures you’ve defined, Always question the measures you’ve defined, keeping in mind that the people applying them could keeping in mind that the people applying them could find ways of boosting the measures without really find ways of boosting the measures without really improving anythingimproving anything

Test each measure to determine if it operates as Test each measure to determine if it operates as expected. Does it always go one way when things expected. Does it always go one way when things get better and the other when things get worse?get better and the other when things get worse?


Authority

Performance Performance Measurement MustsMeasurement Musts

Early and often, measure and evaluate Early and often, measure and evaluate progress toward the goals and objectives that progress toward the goals and objectives that have been defined by the governing bodyhave been defined by the governing body

Continually verify that defined measures Continually verify that defined measures actually correspond with the achievement of actually correspond with the achievement of goals and objectivesgoals and objectives

Resist overreaching : measures of long-term, Resist overreaching : measures of long-term, global outcomes tend to be unreliable.global outcomes tend to be unreliable.

Performance measurement is complex and Performance measurement is complex and requires significant expertiserequires significant expertise


Authority

Resources for Assistance Resources for Assistance with Performance with Performance

Measures (& Overall Measures (& Overall Justice Integration Justice Integration

Planning)Planning) SEARCH SEARCH

www.search.orgwww.search.org

Center for Society, Law, and Justice Center for Society, Law, and Justice (University of New Orleans)(University of New Orleans)

www.cslj.netwww.cslj.net

Documents

Performance Measurement for Justice Integration Projects The Illinois and Cook County Experience Mark Myrent Illinois Criminal Justice Information Authority