GET READY1 Task 1: Assemble and Orient an Outcome Measurement Workgroup
Workgroup:
Task 3: Develop Timeline
Task 4: Distribute your game plan to Key Players
Task 2: Decide which programs to start with. Program:
Communication Plan:
Inputs Activities Outputs
Initial & IntermediateIncreasedImprovedReduction
GainCreate
AdmittedReceive
SeeDevelopAttractBegin
ExpandEnhance
Client SatisfactionEnroll
ParticipateRecruitIdentifyClassifyAttendReportMeet
FollowSign UpUtilize
Common Action Verbs Used for Outcome Statements
Long TermAre MetSatisfiedMaintainImprove
IncreasedAchieveSustain
Client SatisfactionContinuedEstablishedGraduate
Moved to Next LevelAvoidRetainReduce
Positive ResponseImplement
CHOOSE THE OUTCOMES YOU WANT TO MEASURE 2
Knowledge/Learning/Attitude
a) Skills (knowledge, learning)Common Indicators: % increase in scores after attending % that believe skills were increased after attending
% increase in knowledge (before/after program)b) Attitude
Common Indicators: % improvement reported by parent, teacher, co-worker, other % improvement as reported by participantc) Readiness (qualification)
Common Indicators: % feeling well-prepared for a particular task/undertaking % meeting minimum qualifications for next level/undertaking
Behavior a) Incidence of bad behavior
Common Indicators: Incidence rate Relapse/recidivism rate % reduction in reported behavior frequency b) Incidence of desirable activity
Common Indicators: Success rate % that achieve goal Rate of improvement c) Maintenance of new behavior
Common Indicators: Number weeks/months/years continued % change over time % moving to next level/condition/status % that do not reenter the program/system
Condition/Status a) Participant social status
Common Indicators: % with improved relationships % who graduate % who move to next level/condition/status % who maintain current level/condition/status % who avoid undesirable course of action/behaviorb) Participant economic condition
Common Indicators: % who establish career/employment % who move to long term housing % who maintain safe and permanent housing % enrolled in education programs % who retain employment % with increased earnings c) Participant health condition
Common Indicators: % with reduced incidence of health problem % with immediate positive response % that report positive response post-90 days
Common Framework of Outcomes Building A Common Outcomes Framework to Measure Non-Profit Performance – Urban Institute
CHOOSE THE OUTCOMES YOU WANT TO MEASURE 2
Who/What (the target subject)
Change desired (action verb)
In what (expected results)
By when
Imagine your client in program or day after leaving program. What new knowledge, attitude and skills are seen?
Initi
al
Imagine your client 3-9 months after leaving program. What improved attitudes or behaviors are seen?
Inte
rme
diat
e
Imagine your client 6+ months after leaving program. How has condition or status improved? Ideal, ultimate goal
Lon
g -T
erm
EXAM
PLE
SPEC
IFY
IND
ICAT
ORS
FO
R YO
UR
OU
TCO
MES
3
*Ref
er to
Com
mon
Out
com
e an
d Pe
rfor
man
ce In
dica
tor P
acke
t
Task 2: Decide what factors could influence participant Outcomes
SPECIFY INDICATORS FOR YOUR OUTCOMES3
Demographics (age, gender, education,income level, disability, single parent…)
Level of Difficulty (very difficult to help, moderate difficulty, minor difficulty)
Level of Involvement (high, moderate, low participation)
Organizational unit (if more than one service delivery facility)
Service Delivery (group session vs 1-on-1,live vs taped)
PREPARE TO COLLECT DATA ON YOUR INDICATORS 4
_ Task 1: Identify data sources for your indicators_ Task 2: Data Collection_ Task 3: Pretest your data collection instruments and procedures
TRY OUT YOUR OUTCOME MEASUREMENT5
Task 1: Describe your trial strategy
Task 2: Who are your data collectors?
How will you train data collectors on instrument used?(How often do you survey/pre-post?, How long between, How to approach participant,What demographics are we really needing to ensure are represented…)
Task 3: Track and collect outcome data (create a data collection spreadsheet – refer to sample)
Sam
ple
Dat
a Co
llecti
on F
orm
ANALYZE AND REPORT YOUR FINDINGS 6
_ Task 1: Enter the data and check for errors
_ Task 2: Tabulate the data
_ Task 3: Analyze data broken out by key characteristics
_ Task 4: Provide Explanatory Information related to your findings
_ Task 5: Present your data in Clear and Understandable form
Tips for Formatting Your Reports• Consider the needs of your audience:
what information are they looking for?• Keep it Simple• Include a summary of major points• Don’t crowd too much on a page• Define unfamiliar terms• Define each outcome indicator• Highlight points of interest with bold type, circles or arrows• Use color to help highlight key findings• Label charts and tables clearly – titles, rows, columns, axes…• Identify source and date of the data and note limitations• Provide context (history or comparisons)• Add variety to data presentation by using bar or pie charts• Internal repots should be much more detailed than external
IMPROVE YOUR SYSTEM7
Task 1: Review Your Trial Run Experience, Make Necessary Adjustments, and Start Full-Scale Implementation
Workgroup Review Questions Yes No
Did you get all of the data you needed?
Did you measure what you intended to measure?
Does what you measured still seem to represent important outcomes for which your program should be held accountable?
Task 2: Monitor and Review your system periodically
Reviewed? Aspects to Review
Data Collection Instruments
Training of data collectors
Data collection procedures
Data entry procedures
Time and Cost in collecting and analyzing data
Monitor procedures used during trial run
USE YOUR FINDINGS 8
_ Detect Needed Improvements_ Motivate Staff, Volunteers and Clients_ Use in Program Planning_ Report to Board_ Report to Funders _ Report to Community
Make Your Data Pay Off