- Rosell “If I include you, you will be my partner. If I exclude you, you will be my judge.”

- Rosell

“If I include you,you will be my partner.

If I exclude you,you will be my judge.”

PERFORMANCE

ACCOUNTABILITY

Results-Based Accountabilityis made up of two parts:

Performance Accountabilityis about the well-being of

CUSTOMER POPULATIONSfor Programs – Agencies – Service Systems

Population Accountabilityis about the well-being of

WHOLE POPULATIONSfor Communities – Cities – Counties – States - Nations

“All performance measures

that have ever existed

for any program

in the history of the universe

involve answering two sets of

interlocking questions.”

HowMuchdid we do?

( # )

HowWell

did we do it?

( % )

Quantity Quality

Performance Measures

EffortHow hard did we try?

EffectIs anyone better off?


Effort

Effect

HowMuch

HowWell


How much service did we deliver?


How welldid we

deliver it?

How much change / effect

did we produce?

What quality of change / effect

did we produce?

Quantity Quality

Effe

ct

Effo

rt

O

utpu

t

In

put

How much did we do?


How welldid we do it?

Is anyonebetter off?

Quantity Quality

Effe

ct

Effo

rt

# %

How much did we do?

Education

How well did we do it?

Is anyone better off?

Quantity Quality

Effe

ct

E

ffort Number of

studentsStudent-teacher

ratio

Number ofhigh schoolgraduates

Percent ofhigh schoolgraduates

How much did we do?

Education



Quantity Quality

Effe

ct

E

ffort Number of

studentsStudent-teacher

ratio

Percent of 9th graders who

graduate on timeand enter college or

employment after graduation

Number of 9th graders who

graduate on timeand enter college or

employment after graduation

How much did we do?

Pediatric Practice



Number ofchildrentreated

Percent ofpatients waiting

less than30 min in thewaiting room

Quantity Quality

Effe

ct

E

ffort

#children

fullyimmunized

(in the practice)

%children

fullyimmunized

(in the practice)

How much did we do?

Drug/Alcohol Treatment Program



Number ofpersonstreated

Percent ofstaff withtraining/

certification

Number of clientsoff of alcohol & drugs - at exit - 12 months after exit

Percent of clientsoff of alcohol & drugs - at exit - 12 months after exit

Quantity Quality

Effe

ct

E

ffort

How much did we do?

Fire Department



Number ofresponses

ResponseTime

Quantity Quality

Effe

ct

E

ffort

# of fireskept to

room of origin

% of fireskept to

room of origin

How much did we do?

General Motors



# of production hrs

# tons of steel

Employees pervehicle

produced

# of cars sold

$ Amount of Profit

$ Car value after 2 years

Quantity Quality

Effe

ct

E

ffort

Source: USA Today 9/28/98

% Market share

Profit per share

% Car value after 2 years

How much did we do?

Not All Performance Measures Are Created Equal



LeastImportant

Quantity Quality

Effe

ct

E

ffort

MostImportant

Least

Most

AlsoVery Important

Quality

RBA Categories Account for All Performance Measures(in the history of the universe)

Quantity Quality

Efficiency, Admin overhead, Unit costStaffing ratios, Staff turnoverStaff morale, Access, Waiting time, Waiting lists, Worker safety

Customer Satisfaction(quality service delivery& customer benefit)

Cost / Benefit ratioReturn on investment

Client results or client outcomes

EffectivenessValue addedProductivity

Benefit value

Product Output Impact

Process Input

Effe

ctEf

fort

Cost

Total Quality Mgmt (TQM)

Effectiveness

Efficiency Admin overhead, Unit cost

Quantity Quality






Benefit value

Process Input

Effe

ctEf

fort

Cost




Quantity Quality






Benefit value

Process Input

Effe

ctEf

fort

Cost

1. Did we treat you well?

2. Did we help you with your problems?

*



* World’s simplest completecustomer satisfaction survey


Quantity Quality






Benefit value

Process Input

Effe

ctEf

fort

Cost


OBA Categories Account for All Performance Measures(in the history of the universe)


How much did we do?

The Matter of Control



Quantity Quality

Effe

ct

E

ffort

LeastControl

PARTNERSHIPS

MostControl

Advanced Baseline Display

Your Baseline

Comparison Baseline

Goal (line)

Target or Standard

Instead:Count anything better than baseline as progress.

Avoid publicly declaringtargets by year if possible.

●

Create targets only when they are:

FAIR & USEFUL

x

AspirationNOT

Punishment

© The Farside 1989

" Bummer of a Birthmark Hal "

How much did we do?

Program Performance Measures

How welldid we do it?

Is anyonebetter off?

Quantity Quality

Effe

ct

Effo

rt

# %

How much did we do?

Performance AccountabilityTypes of Measures found in each Quadrant



# Clients/customers served

# Activities (by type of activity)

% Common measurese.g. client staff ratio, workload ratio, staffturnover rate, staff morale, % staff fully trained, % clients seen in their own language,worker safety, unit cost

% Skills / Knowledge (e.g. parenting skills)

#

% Attitude / Opinion (e.g. toward drugs)

#

% Behavior (e.g.school attendance)

#

% Circumstance (e.g. working, in stable housing)

#

% Activity-specific measures

e.g. % timely, % clients completing activity, % correct and complete, % meeting standard

Point in Timevs.

2 Point Comparison

# %

How much did we do?

Service: __________________________________



Primary customers# studentspatientspersons trained

Primary activity# hours of

instructiondiagnostic testsjob coursesalarms responded to

If your service works really well,how are your customer's better off?

# students who graduatepatients who fully recoverfires kept to room of origin

%

Unit cost

Workload ratio

% of ___x___ that happen on time

persons who get jobs

School Hospital Job Training Fire Department

How much did we do?

Choosing Headline Measures and the Data Development Agenda



Quantity Quality

E

ffect

Effo

rt

# Measure 1 ----------------------------

# Measure 2 ----------------------------

# Measure 3 ----------------------------

# Measure 4 ----------------------------

# Measure 5 ----------------------------

# Measure 6 ----------------------------

# Measure 7 ----------------------------

#1 Headline

#2 Headline

#3 Headline

#1 DDA

#2 DDA

#3 DDA% Measure 8 ----------------------------

% Measure 9 -----------------------------

% Measure 10 ---------------------------

% Measure 11 ---------------------------

% Measure 12 ---------------------------

% Measure 13 ---------------------------

% Measure 14 ---------------------------

# Measure 15 ----------------------------

# Measure 16 ----------------------------

# Measure 17 ----------------------------

# Measure 18 ----------------------------

# Measure 19 ----------------------------

# Measure 20 ----------------------------

# Measure 21 ----------------------------

% Measure 15 ----------------------------

% Measure 16 ----------------------------

% Measure 17 ----------------------------

% Measure 18 ----------------------------

% Measure 19 ----------------------------

% Measure 20 ----------------------------

% Measure 21 ----------------------------

Select 3 to 5 Performance MeasuresACROSS THE BOTTOM OF THE ORG CHART

3 - 5 3 - 5 3 - 5 3 - 5 3 - 5 3 - 53 - 5

3 - 5 3 - 5 3 - 5

3 - 5

20 – 60 – 20 Rule

LR

UR

Primary v. Secondary

Direct v. Indirect

Internal v. External

Baseline & Story

New Mexico Christus St. Vincents Medical Center

How Population

&Performance

AccountabilityFIT TOGETHER

Contributionrelationship

Alignmentof measures

Appropriateresponsibility

THE LINKAGE Between POPULATION and PERFORMANCE

POPULATION ACCOUNTABILITY

Healthy Births Rate of low birth-weight babiesStable Families Rate of child abuse and neglect Children Ready for School Percent fully ready per K-entry assessment

CUSTOMERRESULTS

# ofwomenserved

%attendance

Unit cost

# low birth-weight births (for

participating women)

PERFORMANCE ACCOUNTABILITY

POPULATIONRESULTS

Pre-natal Nutrition Program

% low birth-weight births (for

participating women)

Performancemeasure

PopulationIndicator

Every timeyou present

your program,

Use atwo-part

approach.

Result: to which you contribute to most directly.

Indicators:

Story:

Partners:

What would it take?:

Your Role: as part of a larger strategy.

Population Accountability

Program:Performance measures:

Story:

Partners:

Action plan to get better:

Performance Accountability

Your Role


Indicators:

Story:

Partners:


Your Role: within the larger strategy.



Story:

Partners:



Your Role


your program,

Use atwo-part

approach.


your program,

Use atwo-part

approach.


Indicators:

Story:

Partners:


Your Role: as part of a larger strategy.



Story:

Partners:



Your Role

Shortcut

The Special Case

of

Prevention Programs

(Read page 100)

Adult Cigarette Smoking New Mexico and United States, 2001-2010

0

5

10

15

20

25

30

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

Sm

oki

ng P

reva

lence

New Mexico United States

Result: HEALTHY PEOPLE.

Indicator:

Story:

Partners:

What would it take to significantly reduce this rate?

Your Role = Smoking Prevention Program +



Story:

Partners:


Smoking Prevention Program

Your Role

Adult Cigarette Smoking New Mexico and United States, 2001-2010

0

5

10

15

20

25

30

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

Sm

oki

ng P

reva

lence

New Mexico United States

- % youth in HS assembly who change attitudes about tobacco

- Ad campaign effectiveness in hitting target audience (# reached by % target demographic)

Performance AccountabilityContribution

Different Kinds of Progress

1. Data

a. Population indicators Actual turned curves: movement for the better away from the baseline.

b. Program performance measures: customer progress and better service: How much did we do?

How well did we do it? Is anyone better off?

2. Accomplishments: Positive activities, not included above.

3. Stories behind the statistics that show how individuals are better off.

Board of Directors MeetingAGENDA

1. New data

2. New story behind the curves

3. New partners

4. New information on what works.

5. New information on financing

6. Changes to action plan and budget

7. Adjourn

1. New data

2. New story behind the curves

3. New partners

4. New information on what works.

5. New information on financing

6. Changes to action plan and budget

7. Adjourn

SUMMARY

3 - kinds of performance measures. How much did we do? How well did we do it? Is anyone better off?

RBA in a Nutshell2 – 3 - 7

2 - kinds of accountability Population accountability Performance accountability

7 - questions from ends to means in less than an hour.

plus language discipline Results & Indicators Performance measures

Baselines & Turning the Curve

“If you do what you always did,

you will get what you always got.”

Kenneth W. JenkinsPresident, Yonkers NY NAACP

“We will succeed together…

or fail separately.”

- Winston Churchill or

- Sonny and Cher

THANK YOU !

Book - DVD Ordersamazon.com

resultsleadership.org

“Data is theunblinking eye

of reform.”

Barbary Curley,Area Director, Hyde Park Office, Boston

EXERCISES

Creating a Working Baselinefrom Group Knowledge

Now

Indicator or Performance Measure

65% Not OK?

BackcastingForecasting

Turn the Curve Exercise: Program Performance

5 min: Starting Points - timekeeper and reporter - identify a program to work on - two hats (yours plus partner’s)

10 min: Performance measure baseline - choose 1 measure to work on – from the lower right quadrant - forecast (to 2017) – OK or not OK?

15 min: Story behind the baseline - causes/forces at work - information & research agenda part 1 - causes

15 min: What works? (What would it take?) - what could work to do better? - each partners contribution - no-cost / low-cost ideas - information & research agenda part 2 – what works

10 min: Report convert notes to one page

Two pointers to action

Program: _______________Performance Measure

(Lay definition)PerformanceMeasureBaseline

Story behind the baseline --------------------------- --------------------------- (List as many as needed)

Partners --------------------------- --------------------------- (List as many as needed)

Three Best Ideas – What Works 1. --------------------------- 2. --------------------------- 3. ---------No-cost / low-cost

ONE PAGE Turn the Curve Report: Performance

SharpEdges

4. --------- Off the Wall

4. --------- Off the Wall

Turn the Curve Exercise – LessonsTalk to Action in an hour

1. How was this different from other processes? What worked and what didn’t work?

2. Why did we ask for:

a. Results before indicators?b. Forecast?c. Story? d. No cost / low cost?e. Two hats?f. Crazy idea?

g. Only 3 best ideas?

3. Do you think a lay audience could understand the reports?

4. How many think you could lead this exercise with a small group? (2+ curves at the same time)

Next Steps:

1. What’s one thing I could personally do with what I learned today?

2. What’s one thing I would like to ask someone else to do to support this work?

Resources



www.raguide.org

www.resultsaccountability.com

RBA Facebook Group

My question is: Are we making an impact? My question is: Are we making a contribution?

- Dave Barry

Never be afraid to try something new. Remember that a lone amateur

built the Ark.

A large group of professionals built the Titanic.

THANK YOU !



Criteria for

Choosing Indicatorsas Primary vs. Secondary Measures

Communication Power

Proxy Power

Data Power

Does the indicator communicate to a broad range of audiences?

Does the indicator say something of central importance about the result?

Does the indicator bring along the data HERD?

Quality data available on a timely basis.

Choosing IndicatorsWorksheet

Outcome or Result_______________________

Candidate IndicatorsCommunication

PowerProxyPower

DataPower

H M L

H

Measure 1

Measure 2

Measure 3

Measure 4

Measure 5

Measure 6

Measure 7

Measure 8

HData

Development

Agenda

Safe Community

H M L H M L

H H

H L

Three Part Indicator List for each Result

Part 1: Primary Indicators

Part 2: Secondary Indicators

Part 3: Data Development Agenda

● 3 to 5 “Headline” Indicators● What this result “means” to the community● Meets the Public Square Test

● Everything else that’s any good (Nothing is wasted.)● Used later in the Story behind the Curve

● New data● Data in need of repair (quality,timeliness etc.)

Alternative to Traditional Evaluation Methods:DEMONSTRATING a CONTRIBUTION

to complex change efforts… requires 3 elements:

A Curve to Turn

1 We tried a bunch of stuff that had a credible

chance of making a difference …

2 …and it had a timely relationship to….

3 …. a turn in the curve.

c FPSI

The Matter of Use

1. The first purpose of performance measurement is to IMPROVE PERFORMANCE.

2. Avoid the performance measurement equals punishment trap.

● Create a healthy organizational environment.

● Start small.

● Build bottom-up and top-down simultaneously.

1. To Ourselves Can we do better than our own history?

2. To Others When it is a fair apples/apples comparison.

3. To Standards When we know what good performance is.

Comparing Performance




1. To Ourselves First

Can we do better than our own history?

Using a BaselineCHART ON THE

WALL

CHARTS on the WALL

CHARTS ON THE WALL


1. To Ourselves First Can we do better than our own history?


Reward?

Punish?


1. To Ourselves First Can we do better than our own history?




Next Generation ContractingContract Provisions

Provision 1. Specify the 3 to 5 most important performance measures (from the How well did we do it? and Is anyone better off? categories).

Provision 2. Specify that the contractor will use a continuous improvement process (the RBA 7 Questions).

Provision 3. Specify how the funder and contractor will work

in partnership to maximize LR customer results (quarterly

meetings using the 7 questions as the agenda).Provision 4. Specify that the funder will work with the funding

community to simplify and standardize contracting and performance reporting.

Provision 5: : Clear articulation of role in population/community well-being using the language of contribution not attribution.

Provision 6: 10% for quality management and administration.

Provision 7: Multi-year funding using 3 year rolling contracts

Provision 8: Use of targets that are fair and useful.

Provision 9: Fund flexibility and virtual funding pool: transfer of up to 10% across line items and program lines.

Provision 10: Request for Results: Getting past the sometimes negative effects of competitive RFP contracting or tendering.

Next Generation ContractingContract Provisions

Documents

- Rosell “If I include you, you will be my partner. If I exclude you, you will be my judge.”