110
Outcome Based Accountability The Fiscal Policy Studies Institute Santa Fe, New Mexico Websites raguide.org resultsaccountability.com Book - DVD Orders amazon.com resultsleadership.org

Outcomes based accountability 101 (2009)

Embed Size (px)

DESCRIPTION

Outcomes based accountability 101 (2009)

Citation preview

Page 1: Outcomes based accountability 101 (2009)

Outcome BasedAccountability

The Fiscal Policy Studies InstituteSanta Fe, New Mexico

Websitesraguide.org

resultsaccountability.com

Book - DVD Ordersamazon.com

resultsleadership.org

Page 2: Outcomes based accountability 101 (2009)

SIMPLE

COMMON SENSE

PLAIN LANGUAGE

MINIMUM PAPER

USEFUL

Page 3: Outcomes based accountability 101 (2009)

Outcome Based Accountabilityis made up of two parts:

Performance Accountabilityabout the well-being of

CUSTOMER POPULATIONSFor Programs – Agencies – and Service Systems

Population Accountabilityabout the well-being of

WHOLE POPULATIONSFor Communities – Cities – Counties – States - Nations

Page 4: Outcomes based accountability 101 (2009)

Results and Performance Accountability

COMMON LANGUAGE

COMMON SENSE

COMMON GROUND

Page 5: Outcomes based accountability 101 (2009)

THE LANGUAGE TRAPToo many terms. Too few definitions. Too little discipline

Benchmark

Target

Indicator Goal

Result

Objective

Outcome

Measure

Modifiers Measurable Core Urgent Qualitative Priority Programmatic Targeted Performance Incremental Strategic Systemic

Lewis Carroll Center for Language DisordersMeasurable urgent systemic indicatorsCore qualitative strategic objectivesYour made up jargon here

Page 6: Outcomes based accountability 101 (2009)

DEFINITIONS

Being Healthy - Staying Safe - Enjoying and achievingMaking a Positive Contribution - Economic Well-being

Rate of low-birthweight babies - Rate of confirmed child abuse Percent 16 & 19 yr. olds with 5 A-C GCSE’s - Rate of volunteering Rate of child poverty

1. How much did we do? 2. How well did we do it? 3. Is anyone better off?

OUTCOME

INDICATOR

PERFORMANCE MEASURE

A condition of well-being for children, adults, families or communities.

A measure which helps quantify the achievement of an outcome.

A measure of how well a program, agency or service system is working. Three types:

= Customer Outcome

Popu

latio

nPe

rfor

man

ce

Safe Communities - Clean Environment

Crime rate - Air quality index

Page 7: Outcomes based accountability 101 (2009)

From Ends to Means

ENDS

MEANS

From Talk to ActionPo

pula

tion

Perf

orm

ance

OUTCOME

INDICATOR

PERFORMANCEMEASURE

Customer outcome = EndsService delivery = Means

From Talk to Action

Page 8: Outcomes based accountability 101 (2009)

1. Safe Community

2. Crime Rate

3. Average Police Dept response time

4. An educated workforce

5. Adult literacy rate

6. People have living wage jobs and income

7. % of people with living wage jobs and income

8. % of participants in job training who get living wage jobs

IS IT A OUTCOME, INDICATOR OR PERFORMANCE MEASURE?

OUTCOME

INDICATOR

PERF. MEASURE

OUTCOME

INDICATOR

OUTCOME

INDICATOR

PERF. MEASURE

Page 9: Outcomes based accountability 101 (2009)

Results – Indicators – Performance Measures in

Amharic, Cambodian, Laotian, Somali, Spanish, Tigrigna, Vietnamese

Page 10: Outcomes based accountability 101 (2009)
Page 11: Outcomes based accountability 101 (2009)

Translation Guide/Rosetta StoneNot the Language Police

Ideas

1. A condition of well-being for children, adults, families & communities

2.

3.

etc.

Group 1 Group 2 Group 3 etc.

OUTCOME RESULT GOAL

TRANSLATION

Back to the Idea

Page 12: Outcomes based accountability 101 (2009)

POPULATIONACCOUNTABILITY

Fiscal Policy Studies InstituteSanta Fe, New Mexicowww.resultsaccountability.comwww.raguide.org

For Whole Populationsin a Geographic Area

Page 13: Outcomes based accountability 101 (2009)

Community Outcomesfor Christchurch, NZ

1. A Safe City

2. A City of Inclusive and Diverse Communities

3. A City of People who Value and Protect the Natural Environment

4. A Well-Governed City

5. A Prosperous City

6. A Healthy City

7. A City for Recreation, Fun and Creativity

8. City of Lifelong Learning

9. An Attractive and Well-Designed City

Page 14: Outcomes based accountability 101 (2009)

Outcomes forChildren, Families and Communities

A Working List● Healthy Births

● Healthy Children and Adults

● Children Ready for School

● Children Succeeding in School

● Young People Staying Out of Trouble

● Stable Families

● Families with Adequate Income

● Safe and Supportive Communities

Page 15: Outcomes based accountability 101 (2009)

Georgia Policy Councilfor Children and Families

RESULTS

● Healthy Children

● Children Ready for School

● Children Succeeding in School

● Strong Families

● Self Sufficient Families

Page 16: Outcomes based accountability 101 (2009)

Every Child Matters – Children Act

Outcomes for Children and Young People

Being Healthy: enjoying good physical and mental health and living a healthy lifestyle.

Staying Safe: being protected from harm and neglect and growing up able to look after themselves.

Enjoying and Achieving: getting the most out of life and developing broad skills for adulthood.

Making a Positive Contribution: to the community and to society and not engaging in anti-social or offending behaviour.

Economic Well-being: overcoming socio-economic disadvantages to achieve their full potential in life.

Page 17: Outcomes based accountability 101 (2009)

1. Health & Emotional Well-being

2. Quality of life

3. Making a positive contribution

4. Exercising choice & control

5. Freedom from discrimination & harassment

6. Economic well being

7. Personal dignity & respect

Our Health, Our Care, Our Say – White Paper

Outcomes for Adults

Source: A New Outcomes Framework for Performance Assessment of Adult Social Care 2006 - 07

Note that 8. Effective leadership and (9. Effective commissioning are means and not ends

Page 18: Outcomes based accountability 101 (2009)

Georgia

Lehigh Valley, PADayton, OH

Santa Cruz, CA

REPO

RT C

ARDS

Page 19: Outcomes based accountability 101 (2009)

New Zealand Kruidenbuurt Tilburg, Netherlands

Portsmouth, UK

Country Neighborhood

City

Page 20: Outcomes based accountability 101 (2009)
Page 21: Outcomes based accountability 101 (2009)

Leaking Roof(Outcomes thinking in everyday life)

Experience

Measure

Story behind the baseline (causes)

Partners

What Works

Action Plan

Inches of WaterBASELINE

? Fixed

Not OK

Turning the Curve

Action Plan #2

Page 22: Outcomes based accountability 101 (2009)

Outcomes

Page 23: Outcomes based accountability 101 (2009)
Page 24: Outcomes based accountability 101 (2009)

Criteria for

Choosing Indicatorsas Primary vs. Secondary Measures

Communication Power

Proxy Power

Data Power

Does the indicator communicate to a broad range of audiences?

Does the indicator say something of central importance about the outcome?

Does the indicator bring along the data HERD?

Quality data available on a timely basis.

Page 25: Outcomes based accountability 101 (2009)

Choosing IndicatorsWorksheet

Outcome _______________________________

Candidate IndicatorsCommunication

PowerProxyPower

DataPower

H M L

H

Measure 1

Measure 2

Measure 3

Measure 4

Measure 5

Measure 6

Measure 7

Measure 8

HData

Development

Agenda

Safe Community

H M L H M L

H H

H L

Page 26: Outcomes based accountability 101 (2009)

Three Part Indicator List for each Outcome

Part 1: Primary Indicators

Part 2: Secondary Indicators

Part 3: Data Development Agenda

● 2 or 3 or 4 “Headline” Indicators● What this outcome “means” to the community● Meets the Public Square Test

● Everything else that’s any good (Nothing is wasted.)● Used later in the Story behind the Curve

● New data● Data in need of repair (quality,timeliness etc.)

Page 27: Outcomes based accountability 101 (2009)

The Matter of Baselines

Baselines have two parts: history and forecast

H

M

L

History Forecast

Turning the CurvePoint to Point

OK?

Page 28: Outcomes based accountability 101 (2009)

The Cost of Bad Outcomes

Invest in prevention to reduce or avoid out-year costs.

The costs of remediating problems after they occur

InvestmentTrackCost

$300 billion

Revenue

Convergenceof Cost & Revenue

Page 29: Outcomes based accountability 101 (2009)

2005

COST OF BAD OUTCOMESUnited States 1970 to 2010

Page 30: Outcomes based accountability 101 (2009)

6.00%

8.00%

10.00%

12.00%

14.00%

16.00%

18.00%

20.00%

Ncle 14.5 14.5 16.8 14.5 17 15 11.9 10.6 9.5 9.3

1998 1999 2000 2001 2002 2003 2004 2005 2006 2007

Source: Connexions Tyne and Wear, UK

Newcastle, UK

Revised 9 Nov 2007

Page 31: Outcomes based accountability 101 (2009)

Figure 5: Pre and Post Turning the Curve ExerciseBreakdown of offences committed for the data periods 01/01/04 -

31/03/04 and 01/01/05 - 31/03/05

0255075

100125150175200225250

01/01/04 - 31/03/04 01/01/05 - 31/03/05

Criminal Damage Public Order Theft Overall Decrease

Paulsgrove, Portsmouth, UK

Youth Nuisance

Page 32: Outcomes based accountability 101 (2009)

MADD

Page 33: Outcomes based accountability 101 (2009)

Rebound

Page 34: Outcomes based accountability 101 (2009)
Page 35: Outcomes based accountability 101 (2009)
Page 36: Outcomes based accountability 101 (2009)

Alternative to Traditional Evaluation Methods:DEMONSTRATING a CONTRIBUTION

to complex change efforts… requires 3 elements:

Given a Curve to Turn

1 We tried a bunch of stuff that had a credible

chance of making a difference …

2 …and it had a timely relationship to….

3 …. a turn in the curve.

c FPSI

Page 37: Outcomes based accountability 101 (2009)

- Rosell

“If I include you,you will be my partner.

If I exclude you,you will be my judge.”

Page 38: Outcomes based accountability 101 (2009)

Performance Accountability

For Services, Agencies and Service Systems

Fiscal Policy Studies InstituteSanta Fe, New Mexicowww.resultsaccountability.comwww.raguide.org

Page 39: Outcomes based accountability 101 (2009)

“All Performance Measures

that have ever existed

for any service

in the history of the universe

involve answering two sets of

interlocking questions.”

Page 40: Outcomes based accountability 101 (2009)

HowMuchdid we do?

( # )

HowWell

did we do it?

( % )

Quantity Quality

Program Performance Measures

Page 41: Outcomes based accountability 101 (2009)

EffortHow hard did we try?

EffectIs anyone better off?

Performance Measures

Page 42: Outcomes based accountability 101 (2009)

Effort

Effect

HowMuch

HowWell

Program Performance Measures

Page 43: Outcomes based accountability 101 (2009)

How much service did we deliver?

Program Performance Measures

How welldid we

deliver it?

How much change / effect

did we produce?

What quality of change / effect

did we produce?

Quantity Quality

Effe

ct

Effo

rt

O

utpu

t

In

put

Page 44: Outcomes based accountability 101 (2009)

How much did we do?

Program Performance Measures

How welldid we do it?

Is anyonebetter off?

Quantity Quality

Effe

ct

Effo

rt

# %

Page 45: Outcomes based accountability 101 (2009)

How much did we do?

Education

How well did we do it?

Is anyone better off?

Quantity Quality

Effe

ct

E

ffort Number of

studentsStudent-teacher

ratio

Number of 16 olds with 5 A to C

GCSE’s

Number of fixedterm exclusions

Percent of 16 yr olds with 5 A to C

GCSE’s

Rate of fixedterm exclusions

Page 46: Outcomes based accountability 101 (2009)

How much did we do?

Education

How well did we do it?

Is anyone better off?

Quantity Quality

Effe

ct

E

ffort Number of

studentsStudent-teacher

ratio

Number of 16 olds with 5 A to C

GCSE’s

Number with goodschool attendance

Percent of 16 yr olds with 5 A to C

GCSE’s

Percent with goodschool attendance

Page 47: Outcomes based accountability 101 (2009)

How much did we do?

Pediatric Practice

How well did we do it?

Is anyone better off?

Number ofpatientstreated

Percent ofpatients treated

in less than1 hour

Quantity Quality

Effe

ct

E

ffort

#children

fullyimmunized

(in the practice)

%children

fullyimmunized

(in the practice)

Page 48: Outcomes based accountability 101 (2009)

How much did we do?

Drug/Alcohol Treatment Program

How well did we do it?

Is anyone better off?

Number ofpersonstreated

Percent ofstaff withtraining/

certification

Number of clientsoff of alcohol & drugs - at exit - 12 months after exit

Percent of clientsoff of alcohol & drugs - at exit - 12 months after exit

Quantity Quality

Effe

ct

E

ffort

Page 49: Outcomes based accountability 101 (2009)

How much did we do?

Home Care Services

How well did we do it?

Is anyone better off?

# of persons served

# visits

# visits per week

Ave length of visit

% service users who say the service is reliable, flexible, respectful, friendly (SPRU))

Rate entry into hospital, residential or nursing care

% carers report the service helped the service user remain at home

# entry into hospital, residential or nursing care

# carers report the service helped the service user remain at home

Page 50: Outcomes based accountability 101 (2009)

How much did we do?

Fire Department

How well did we do it?

Is anyone better off?

Number ofresponses

ResponseTime

Quantity Quality

Effe

ct

E

ffort

# of fireskept to

room of origin

% of fireskept to

room of origin

Page 51: Outcomes based accountability 101 (2009)

How much did we do?

General Motors

How well did we do it?

Is anyone better off?

# of production hrs

# tons of steel

Employees pervehicle

produced

# of cars sold

$ Amount of Profit

$ Car value after 2 years

Quantity Quality

Effe

ct

E

ffort

Source: USA Today 9/28/98

% Market share

Profit per share

% Car value after 2 years

Page 52: Outcomes based accountability 101 (2009)

How much did we do?

Not All Performance Measures Are Created Equal

How well did we do it?

Is anyone better off?

LeastImportant

Quantity Quality

Effe

ct

E

ffort

MostImportant

Least

Most

AlsoVery Important

Quality

Page 53: Outcomes based accountability 101 (2009)

RBA Categories Account for All Performance Measures(in the history of the universe)

Quantity Quality

Efficiency, Admin overhead, Unit costStaffing ratios, Staff turnoverStaff morale, Access, Waiting time, Waiting lists, Worker safety

Customer Satisfaction(quality service delivery& customer benefit)

Cost / Benefit ratioReturn on investment

Client results or client outcomes

EffectivenessValue addedProductivity

Benefit value

Product Output Impact

Process Input

Effe

ctEf

fort

Cost

TQM

Effectiveness

Efficiency

Page 54: Outcomes based accountability 101 (2009)

Quantity Quality

Efficiency, Admin overhead, Unit costStaffing ratios, Staff turnoverStaff morale, Access, Waiting time, Waiting lists, Worker safety

Customer Satisfaction(quality service delivery& customer benefit)

Cost / Benefit ratioReturn on investment

Client results or client outcomes

EffectivenessValue addedProductivity

Benefit value

Process Input

Effe

ctEf

fort

Cost

TQM

Product Output Impact

RBA Categories Account for All Performance Measures(in the history of the universe)

Page 55: Outcomes based accountability 101 (2009)

Quantity Quality

Efficiency, Admin overhead, Unit costStaffing ratios, Staff turnoverStaff morale, Access, Waiting time, Waiting lists, Worker safety

Customer Satisfaction(quality service delivery& customer benefit)

Cost / Benefit ratioReturn on investmentClient results or client outcomes

EffectivenessValue addedProductivity

Benefit value

Process Input

Effe

ctEf

fort

Cost

TQM

1. Did we treat you well?

2. Did we help you with your problems?

*

Product Output Impact

RBA Categories Account for All Performance Measures(in the history of the universe)

* World’s simplest completecustomer satisfaction survey

Page 56: Outcomes based accountability 101 (2009)

Quantity Quality

Efficiency, Admin overhead, Unit costStaffing ratios, Staff turnoverStaff morale, Access, Waiting time, Waiting lists, Worker safety

Customer Satisfaction(quality service delivery& customer benefit)

Cost / Benefit ratioReturn on investment

Client results or client outcomes

EffectivenessValue addedProductivity

Benefit value

Process Input

Effe

ctEf

fort

Cost

TQM

Product Output Impact

RBA Categories Account for All Performance Measures(in the history of the universe)

Page 57: Outcomes based accountability 101 (2009)

How much did we do?

The Matter of Control

How well did we do it?

Is anyone better off?

Quantity Quality

Effe

ct

E

ffort

LeastControl

PARTNERSHIPS

MostControl

Page 58: Outcomes based accountability 101 (2009)

The Matter of Use

1. First Purpose is to Improve Performance as a contribution to improving results

2. Avoid the Performance Measurement Equals Punishment Trap

● Acknowledge the experience as real.

● Work to create a healthy organizational environment

● Start small.

● Build bottom-up and top-down simultaneously.

Page 59: Outcomes based accountability 101 (2009)

1. To Ourselves First Can we do better than our own history?

2. To Others When it is a fair apples/apples comparison.

3. To Standards When we know what good performance is.

Comparing Performance

Page 60: Outcomes based accountability 101 (2009)

2. To Others When it is a fair apples/apples comparison.

3. To Standards When we know what good performance is.

Comparing Performance

1. To Ourselves First

Can we do better than our own history?

Using a BaselineCHART ON THE

WALL

Page 61: Outcomes based accountability 101 (2009)

Comparing Performance

1. To Ourselves First Can we do better than our own history?

2. To Others When it is a fair apples/apples comparison.

Reward?

Punish?

3. To Standards When we know what good performance is.

Page 62: Outcomes based accountability 101 (2009)

1. To Ourselves First Can we do better than our own history?

2. To Others When it is a fair apples/apples comparison.

Comparing Performance

3. To Standards When we know what good performance is.

Page 63: Outcomes based accountability 101 (2009)

The Matter of Standards

Quantity

Effe

ctEf

fort

1. Quality of Effort Standards aresometimes WELL ESTABLISHED

● Child care staffing ratios● Application processing time● Handicap accessibility● Child abuse response time

2. Quality of Effect Standards arealmost always EXPERIMENTAL

● Hospital recovery rates ● Employment placement and retention rates● Recidivism rates

3. Both require aLEVEL PLAYING FIELD and an ESTABLISHED RECORDof what good performance is.

BUT

AND

Page 64: Outcomes based accountability 101 (2009)

Advanced Baseline Display

Your Baseline

Comparison Baseline

Goal (line)Target or Standard

Instead:Count anything better than baseline as progress.

Avoid publicly declaringtargets by year if possible.

Create targets only when they are:

FAIR & USEFUL

Page 65: Outcomes based accountability 101 (2009)

How much did we do?

Program Performance Measures

How welldid we do it?

Is anyonebetter off?

Quantity Quality

Effe

ct

Effo

rt

# %

Page 66: Outcomes based accountability 101 (2009)

Lay

Definition

All Data have two Incarnations

Technical

Definition

Graduation Rate % enrolled June 1 who graduate June 15

% enrolled Sept 30 who graduate June 15

% enrolled 9th grade who graduate in 12th grade

Page 67: Outcomes based accountability 101 (2009)

How much did we do?

Separating the Wheat from the ChaffTypes of Measures Found in Each Quadrant

How well did we do it?

Is anyone better off?

# Clients/customers served

# Activities (by type of activity)

% Common measurese.g. client staff ratio, workload ratio, staffturnover rate, staff morale, % staff fully trained, % clients seen in their own language,worker safety, unit cost

% Skills / Knowledge (e.g. parenting skills)

#

% Attitude / Opinion (e.g. toward drugs)

#

% Behavior (e.g.school attendance)

#

% Circumstance (e.g. working, in stable housing)

#

% Activity-specific measures

e.g. % timely, % clients completing activity, % correct and complete, % meeting standard

Point in Time vs. Point to Point

Improvement

Page 68: Outcomes based accountability 101 (2009)

How much did we do?

Choosing Headline Measures and the Data Development Agenda

How well did we do it?

Is anyone better off?

Quantity Quality

E

ffect

Effo

rt

# Measure 1 ----------------------------

# Measure 2 ----------------------------

# Measure 3 ----------------------------

# Measure 4 ----------------------------

# Measure 5 ----------------------------

# Measure 6 ----------------------------

# Measure 7 ----------------------------

#1 Headline

#2 Headline

#3 Headline

#1 DDA

#2 DDA

#3 DDA% Measure 8 ----------------------------

% Measure 9 -----------------------------

% Measure 10 ---------------------------

% Measure 11 ---------------------------

% Measure 12 ---------------------------

% Measure 13 ---------------------------

% Measure 14 ---------------------------

# Measure 15 ----------------------------

# Measure 16 ----------------------------

# Measure 17 ----------------------------

# Measure 18 ----------------------------

# Measure 19 ----------------------------

# Measure 20 ----------------------------

# Measure 21 ----------------------------

% Measure 15 ----------------------------

% Measure 16 ----------------------------

% Measure 17 ----------------------------

% Measure 18 ----------------------------

% Measure 19 ----------------------------

% Measure 20 ----------------------------

% Measure 21 ----------------------------

Page 69: Outcomes based accountability 101 (2009)

Select 3 to 5 Performance Measuresat each level of the organization

3 - 5 3 - 5 3 - 5

3 - 5 3 - 5 3 - 5

3 - 5

Pick the 3 – 5 most important of the 9 – 15 measures or create composites.

“Get over it!”Be disciplined about what’s most important. Don’t get distracted.

?

. .

Page 70: Outcomes based accountability 101 (2009)

Customer Outcomes

Page 71: Outcomes based accountability 101 (2009)

LR

UR

Page 72: Outcomes based accountability 101 (2009)
Page 73: Outcomes based accountability 101 (2009)
Page 74: Outcomes based accountability 101 (2009)

How Population

&Performance Accountability

FIT TOGETHER

Page 75: Outcomes based accountability 101 (2009)

Contributionrelationship

Alignmentof measures

Appropriateresponsibility

THE LINKAGE Between POPULATION and PERFORMANCE

POPULATION ACCOUNTABILITY

Being Healthy Rate of low birth-weight babiesStaying Safe Rate of child abuse and neglect Enjoying and Achieving Percent with 5 A-C GCSE’s

CUSTOMEROUTCOMES

# ofinvestigations

completed

% initiatedwithin 24 hrs

of report

# repeatAbuse/Neglect

% repeatAbuse/Neglect

PERFORMANCE ACCOUNTABILITY

POPULATIONOUTCOMES

Child Protection Service

Page 76: Outcomes based accountability 101 (2009)

Contributionrelationship

Alignmentof measures

Appropriateresponsibility

THE LINKAGE Between POPULATION and PERFORMANCE

POPULATION ACCOUNTABILITY

Healthy Births Rate of low birth-weight babiesChildren Ready for School Percent fully ready per K-entry assessmentSelf-sufficient Families Percent of parents earning a living wage

CUSTOMEROUTCOMES

# personsreceivingtraining

Unit costper person

trained

# who getliving wage jobs

% who getliving wage jobs

PERFORMANCE ACCOUNTABILITY

POPULATIONOUTCOMES

Job Training Program

Page 77: Outcomes based accountability 101 (2009)
Page 78: Outcomes based accountability 101 (2009)

Every timeyou make a

presentation,

use atwo-part

approach

Outcome: to which you contribute to most directly.

Indicators:

Story:

Partners:

What would it take?:

Your Role: as part of a larger strategy.

Population Accountability

Service:Performance measures:

Story:

Partners:

Action plan to get better:

Performance Accountability

Your Role

Page 79: Outcomes based accountability 101 (2009)

Every timeyou make a

presentation,

use atwo-partformat

Result: to which you contribute to most directly.

Indicators:

Story:

Partners:

What would it take?:

Your Role: within the larger strategy.

Population Accountability

Service:Performance measures:

Story:

Partners:

Action plan to get better:

Performance Accountability

Your Role

Page 80: Outcomes based accountability 101 (2009)

Different Kinds of Progress

1. Data

a. Population indicators Actual turned curves: movement for the better away from the baseline.

b. Service performance measures: customer progress and better service: How much did we do?

How well did we do it? Is anyone better off?

2. Accomplishments: Positive activities, not included above.

3. Anecdotes: Stories behind the statistics that show how individuals are better off.

Page 81: Outcomes based accountability 101 (2009)
Page 82: Outcomes based accountability 101 (2009)
Page 83: Outcomes based accountability 101 (2009)
Page 84: Outcomes based accountability 101 (2009)

Population Results

1. Population2. Results (Outcomes, Goals)3. Indicators (Benchmarks) Data Development Agenda Report Card4. Baseline5. Story behind the baseline Cost of Bad Results Research Agenda Part 1

6. Partners 7. What works Research Agenda Part 2

8. Action Plan (strategy)9. Funding Plan (budget)

Program Performance

1. Customers (Clients)2. Performance measures Customer results Quality of Effort Quantity of Effort Data Development Agenda3. Baseline4. Story behind the baseline Research Agenda Part 1

5. Partners6. What works Agency/program actions Partner's actions Research Agenda Part 2

7. Action Plan (strategy)8. Funding Plan

Framework:

__________

Framework Crosswalk Analysis

(For Population Well-being, Across Communities, Across Systems)

(For Programs, Agencies and Service Systems)

Example

Input

Activity

Output

Outcome

Goal

Logic Model

Page 85: Outcomes based accountability 101 (2009)

Board of Directors MeetingAGENDA

1. New data

2. New story behind the curves

3. New partners

4. New information on what works.

5. New information on financing

6. Changes to action plan and budget

7. Adjourn

1. New data

2. New story behind the curves

3. New partners

4. New information on what works.

5. New information on financing

6. Changes to action plan and budget

7. Adjourn

Page 86: Outcomes based accountability 101 (2009)

What’s Next?A Basic Action Plan for Outcome Based Accountability

TRACK 1: POPULATION ACCOUNTABILITY

● Establish outcomes● Establish indicators, baselines and charts on the wall● Create an indicators report card● Set tables (action groups) to turn curves

TRACK 2: PERFORMANCE ACCOUNTABILITY

● Performance measures, and charts on the wall for programs, agencies and service systems● Use 7 Questions program by program in management, budgeting and strategic planning

Page 87: Outcomes based accountability 101 (2009)

IN CLOSING

Page 88: Outcomes based accountability 101 (2009)

“If you do what you always did,

you will get what you always got.”

Kenneth W. JenkinsPresident, Yonkers NY NAACP

Page 89: Outcomes based accountability 101 (2009)

------Book - DVD Orders------amazon.com

resultsleadership.org

THANK YOU !Websites

raguide.orgresultsaccountability.com

Page 90: Outcomes based accountability 101 (2009)

EXERCISES

Fiscal Policy Studies InstituteSanta Fe, New Mexicowww.resultsaccountability.comwww.raguide.org

Page 91: Outcomes based accountability 101 (2009)

30

35

40

45

50

55

60

65

70

2001 2002 2003 2004 2005 2006 2007 2008

North West

England

North West incl Eng & Maths

England incl Eng & Maths

2008 National Target

GCSE 5+ A*-C

All Children and Young People in the NW Succeed in Literacy and Numeracyand Can Lead Productive Lives

PSA 10

This is our baseline

How do we maintain

momentum?

Page 92: Outcomes based accountability 101 (2009)

20

25

30

35

40

45

50

55

1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

Und

er 1

8 co

ncep

tion

rate

s pe

r 10

00, 1

5-17

yea

r ol

d fe

mal

es

North West

England

2010 National Target

Under 18 Conception Rates

All Children and Young People in the NW Have Healthy Lifestylesand Productive Lives

PSA 3

This is our baseline

How do we turn the curve?

Page 93: Outcomes based accountability 101 (2009)

Manchester

North West

England

2

4

6

8

10

12

14

16

2004 2005 2006 2007 2008 2009 2010

Manchester

North West

England

Percentage NEET

All Children and Young People in the NW Can Lead Productive Lives

PSA 12Projected reduction required to meet 2010 target Actual Data

Page 94: Outcomes based accountability 101 (2009)

60

65

70

75

80

2004 2005 2006 2007 2008

North West

England

2006 National Target

Percentage of 19 year olds with at least Level 2 qualification

All Children and Young People in the NW Can Lead Productive Lives

PSA 11

This is our baseline

How do we maintain momentum?

Page 95: Outcomes based accountability 101 (2009)

5

6

7

8

9

10

2002-03 2003-04 2004-05 2005-06 2006-07 2007-08 2008-09 2009-10

North West

England

Percentage care leavers with 5+ GCSEs A*-C

All children and young people in the NW live in stable and safe environments and Can Lead Productive Lives

PSA

This is our baseline

Page 96: Outcomes based accountability 101 (2009)

Turn the Curve Exercise: Population Well-being

5 min: Starting Points - timekeeper and reporter - geographic area - two hats (yours plus partner’s)

10 min: Baseline - pick an outcome, and an indicator curve to turn - forecast – OK or not OK?

15 min: Story behind the baseline - causes/forces at work - information & research agenda part 1 - causes

15 min: What works? (What would it take?) - what could work to do better? - each partners contribution - no-cost / low-cost ideas - information & research agenda part 2 – what works

10 min: Report convert notes to one page

Two pointers to action

Page 97: Outcomes based accountability 101 (2009)

ONE PAGE Turn the Curve Report: Population

Outcome: _______________Indicator

(Lay Definition)IndicatorBaseline

Story behind the baseline --------------------------- --------------------------- (List as many as needed)

Partners --------------------------- --------------------------- (List as many as needed)

Three Best Ideas – What Works 1. --------------------------- 2. --------------------------- 3. ---------No-cost / low-cost

SharpEdges

4. --------- Off the Wall

4. --------- Off the Wall

Page 98: Outcomes based accountability 101 (2009)

The first step in performance accountability is to

DRAW A FENCEAround something that has

ORGANIZATIONAL OR FUNCTIONAL IDENTITY

The Whole Organization

Division ADivision B

Unit

Division C

FunctionUnit

1

Page 99: Outcomes based accountability 101 (2009)

Creating a Working Baselinefrom Group Knowledge

Now

Graduation Rate

75% Not OK?

BackcastingForecasting

Page 100: Outcomes based accountability 101 (2009)

Turn the Curve Exercise: Program Performance

5 min: Starting Points - timekeeper and reporter - identify a program to work on - two hats (yours plus partner’s)

10 min: Performance measure baseline - choose 1 measure to work on – from the lower right quadrant - forecast – OK or not OK?

15 min: Story behind the baseline - causes/forces at work - information & research agenda part 1 - causes

15 min: What works? (What would it take?) - what could work to do better? - each partners contribution - no-cost / low-cost ideas - information & research agenda part 2 – what works

10 min: Report Convert notes to one page

Two pointers to action

Page 101: Outcomes based accountability 101 (2009)

Service: _______________Performance Measure

(Lay definition)PerformanceMeasureBaseline

Story behind the baseline --------------------------- --------------------------- (List as many as needed)

Partners --------------------------- --------------------------- (List as many as needed)

Three Best Ideas – What Works 1. --------------------------- 2. --------------------------- 3. ---------No-cost / low-cost

ONE PAGE Turn the Curve Report: Performance

SharpEdges

4. --------- Off the Wall

4. --------- Off the Wall

Page 102: Outcomes based accountability 101 (2009)

THANK YOU !

WEBSITES:

www.raguide.orgwww.resultsaccountability.com

BOOK ORDERS: www.trafford.comwww.amazon.com

Page 103: Outcomes based accountability 101 (2009)

2 customers

1 measure

1 measure

Discuss how we’re doing

2 partners

2 what works ideas

(1 nc/lc)

Discuss how to implement

20 MinuteExercise

40

Page 104: Outcomes based accountability 101 (2009)

1 population

1 result

1 indicator

Discuss how we’re doing

2 partners

2 what works ideas

(1 nc/lc)

Discuss how to implement

20 MinuteExercise

1. What population are we concerned about? (e.g. U.S. population)

2. What condition(s) do we want for this population? (Americans are free of cancer.)

3. How could we measure these conditions? ( (e.g. cancer rates)

4. How are we doing on the most important

of these measures? (baseline history)5. Who are the partners with a role to play in doing better?6. What works – what would it take – to do better? (What is our role?)7. What do we propose to do?

POPULATION ACCOUNTABILITY

Page 105: Outcomes based accountability 101 (2009)

Some datafor the

Turn the CurveExercises

Page 106: Outcomes based accountability 101 (2009)

30

35

40

45

50

55

60

65

70

2001 2002 2003 2004 2005 2006 2007 2008

North West

England

North West incl Eng & Maths

England incl Eng & Maths

2008 National Target

GCSE 5+ A*-C

All Children and Young People in the NW Succeed in Literacy and Numeracyand Can Lead Productive Lives

PSA 10

This is our baseline

How do we maintain

momentum?

Page 107: Outcomes based accountability 101 (2009)

20

25

30

35

40

45

50

55

1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

Und

er 1

8 co

ncep

tion

rate

s pe

r 10

00, 1

5-17

yea

r ol

d fe

mal

es

North West

England

2010 National Target

Under 18 Conception Rates

All Children and Young People in the NW Have Healthy Lifestylesand Productive Lives

PSA 3

This is our baseline

How do we turn the curve?

Page 108: Outcomes based accountability 101 (2009)

Manchester

North West

England

2

4

6

8

10

12

14

16

2004 2005 2006 2007 2008 2009 2010

Manchester

North West

England

Percentage NEET

All Children and Young People in the NW Can Lead Productive Lives

PSA 12Projected reduction required to meet 2010 target Actual Data

Page 109: Outcomes based accountability 101 (2009)

60

65

70

75

80

2004 2005 2006 2007 2008

North West

England

2006 National Target

Percentage of 19 year olds with at least Level 2 qualification

All Children and Young People in the NW Can Lead Productive Lives

PSA 11

This is our baseline

How do we maintain momentum?

Page 110: Outcomes based accountability 101 (2009)

5

6

7

8

9

10

2002-03 2003-04 2004-05 2005-06 2006-07 2007-08 2008-09 2009-10

North West

England

Percentage care leavers with 5+ GCSEs A*-C

All children and young people in the NW live in stable and safe environments and Can Lead Productive Lives

PSA

This is our baseline