Upload
charla-bruce
View
214
Download
0
Embed Size (px)
Citation preview
Copyright 2004 ROI Institute, Inc.
how me the money!how me the money!
Moving from Impact to ROIMoving from Impact to ROI
Patti Phillips, Ph.D.
Copyright 2004 ROI Institute, Inc.
In this session participants will:
• Identify program benefits
• Convert benefits to monetary value
• Tabulate fully-loaded costs
• Identify intangible benefits
• Calculate an ROI
Copyright 2004 ROI Institute, Inc.
ROI is reported as
BCR = Program Benefits
Program Costs
ROI = Net Program Benefits
Program CostsX 100
Copyright 2004 ROI Institute, Inc.
ROI is reported in the context of five levels of evaluation.
Level
1. Reaction & Planned Action
2. Learning
3. Application
4. Impact
5. Return on Investment
Measurement Focus
Measures participant satisfaction with the program and captures planned actions
Measures changes in knowledge, skills, and attitudes
Measures changes in on-the-job behavior
Measures changes in critical outcomes
Compares program benefits to the costs
Copyright 2004 ROI Institute, Inc.
Moving from Impact to ROI requires five steps.
Level 4
Level 5
1. Identify program benefits
2. Convert benefits to monetary value
3. Tabulate program costs
4. Identify intangible benefits
5. Compare monetary benefits to program costs
Copyright 2004 ROI Institute, Inc.
Level 4 1.Identify program benefits
Copyright 2004 ROI Institute, Inc.
Level 4 measures are defined as
… the consequence of doing something different as a result of the
program
Copyright 2004 ROI Institute, Inc.
Poll Question
A. Decrease error rates
B. Increase use of disciplinary discussion skills
C. Initiate at least three cost reduction projects
D. Achieve a 2:1 benefit to cost ratio
E. Improve teamwork
Which of the following measures represent Level 4 Impact measures?
Copyright 2004 ROI Institute, Inc.
A. Decrease error rates
E. Improve teamwork
Which of the following measures represent Level 4 Impact measures?
Copyright 2004 ROI Institute, Inc.
Hard Data
versus
Soft Data
Level 4 measures include two type of data.
Copyright 2004 ROI Institute, Inc.
Decrease error rates
versus
Improve teamwork
Level 4 measures include two type of data.
Copyright 2004 ROI Institute, Inc.
Tangible
versus
Intangible
Level 4 measures include two type of data.
Copyright 2004 ROI Institute, Inc.
All data can be converted.
• Productivity
• Quality
• Time
• Cost Savings
• Revenue
Convert data to cost savings
Sales converted to profit margin
Copyright 2004 ROI Institute, Inc.
Poll QuestionWhich one of the following is the most
credible item?
A. Vulcan Materials Company produced 195 million tons of crushed stone during 2001
Source: Annual Report
B. Wachovia Bank receives a 932% ROI in a training program for relationship managers.
Source: Measuring Return on Investment, Vol. 3 In Action Series, Patti Phillips (editor) ASTD, 2001, p. 81.
C. Verizon receives a -85% ROI in a telephonic customer service skills program.
Source: Measuring Return on Investment, Vol. 3 In Action Series, Patti Phillips (editor) ASTD, 2001, p. 131.
Copyright 2004 ROI Institute, Inc.
Credibility of data is influenced by
• Reputation of the source– Source of data– Source of study
• Bias– Researcher bias– Audience bias
• Methodology– Assumptions made in the analysis– Type of data– Scope of analysis– Realism of the outcome data
Copyright 2004 ROI Institute, Inc.
Data are converted by• Converting output to contribution - standard value• Converting the cost of quality - standard value• Converting employee’s time – standard value• Using historical costs• Using internal and external experts• Using data from external databases• Linking with other measures• Using participants’ estimates• Using supervisors’ and managers’ estimates• Using staff estimates
Copyright 2004 ROI Institute, Inc.
Historical Costs and Expert Input are Used to Convert Data
Actual Costs Legal Fees, Settlements,from Records Losses, Material,
Direct Expenses
Additional Estimated EEO/AA Staff Time, Costs from Staff Management Time
The Cost of A Sexual Harassment Complaint
35Complain
ts
$852,000Annually
Cost per Complaint = $24,343
$852,000
35
Copyright 2004 ROI Institute, Inc.
External Databases are Used to Convert Data
Cost of one turnover*
Middle Manager $70,000 annual salary
Cost of turnover 150%
Total cost of turnover $105,000
* External data – value obtained from industry related study
Copyright 2004 ROI Institute, Inc.
Supervisor Estimates Used to Convert Absenteeism
Unit of Measure: AbsenteeismData Collection Technique: Focus GroupsSource of Data: Supervisors
1. What happens when someone is unexpectedly absent?2. In monetary terms, how much does it cost you when a
person is unexpectedly absent for one day?1. What is your basis for this estimate?
3. On a scale of 0% - 100% (0% = no confidence; 100% = certainty) how confident are you in your estimated value?
Copyright 2004 ROI Institute, Inc.
Supervisor Estimates Used to Convert Absenteeism
Supervisor Est. Per Day Cost
% Confidence Adjusted Per Day
Cost
1 $1,000 70% $700
2 $1,500 65% $975
3 $2,300 50% $1,150
4 $2,000 60% $1,200
5 $1,600 80% $1,280
$5,305
Average adjusted per day cost of 1 absence $1,061
Copyright 2004 ROI Institute, Inc.
Estimated Value of Absenteeism
$0
$500
$1,000
$1,500
$2,000
$2,500
1 2 3 4 5
Supervisors
Est
imat
ed V
alue
s
Estimated Value Adjusted Value
Copyright 2004 ROI Institute, Inc.
To Convert or Not Convert
Use the 4-Part Test Is there a standard value? Is there a method to get
there? Can we get there with
minimum resources? Can we convince our
executive in 2 minutes that the value is credible?
Copyright 2004 ROI Institute, Inc.
Data conversion requires 5 steps:
1. Focus on a unit of measure2. Determine the value (V) of each unit3. Calculate the improvement in the
measure (∆P)4. Determine the annual improvement
in the measure (A∆P)5. Calculate the total monetary value of
the improvement (A∆P x V)
Copyright 2004 ROI Institute, Inc.
Grievances Converted by Internal Experts
Step 1: 1 Grievance
Step 2: V = $6,500
Step 3: ∆P = 7 out of 10 grievances prevented per month due to program
Step 4: A∆P =
Step 5: A∆P x V =
Copyright 2004 ROI Institute, Inc.
Grievances Converted by Internal Experts
Step 1: 1 Grievance
Step 2: V = $6,500
Step 3: ∆P = 7 out of 10 grievances prevented per month due to program
Step 4: A∆P =
Step 5: A∆P x V = 84 x $6,500 = $546,000
7 x 12 = 84
Copyright 2004 ROI Institute, Inc.
A BOperating CostsSupport Costs
Administrative CostsParticipant Compensation and Facility CostsClassroom Costs
C DProgram Development CostsAdministrative CostsClassroom CostsParticipant Costs
Analysis CostsDevelopment CostsDelivery CostsOverhead CostsEvaluation Costs
Poll QuestionWhich cost categories represent the fully-loaded costs of a program?
Copyright 2004 ROI Institute, Inc.
Recommended Cost Items• Analysis Costs (Prorated)• Development Costs (Prorated)• Program Materials• Instructor/Facilitator Costs• Facilities Costs• Travel/Lodging/Meals• Participant Salaries and Benefits• Administrative/Overhead Costs • Evaluation Costs
Copyright 2004 ROI Institute, Inc.
Potential Intangible Benefits
• Increased Job Satisfaction
• Increased Organizational Commitment
• Improved Teamwork
• Improved Customer Service
• Reduced Complaints
• Reduced Conflicts
• Reduced Stress
Copyright 2004 ROI Institute, Inc.
ROI is reported one of two ways:
BCR = Program Benefits
Program Costs
ROI = Net Program Benefits
Program CostsX 100
Copyright 2004 ROI Institute, Inc.
Healthcare Inc.
• Program Costs– Fully loaded to include needs assessment, development,
coordination, participant salaries and benefits, and evaluation– Total Costs = $277,987
• Monetary Benefits– Complaint Reduction
• Value of one internal complaint = $24,343• Annual improvement related to program – 14.8 complaints
prevented
– Turnover Reduction• Value of one turnover statistic = $20,887• Annual improvement related to program = 136 turnovers prevented
Your assignment: Calculate the BCR and the ROI
Copyright 2004 ROI Institute, Inc.
Healthcare Inc.
• Program Costs = $277,987• Monetary Benefits
– Complaint Reduction• Value of one internal complaint = $24,343• Annual improvement related to program – 14.8 complaints
prevented• 14.8 X $24,343 = $360,276
– Turnover Reduction• Value of one turnover statistic = $20,887• Annual improvement related to program = 136 turnovers
prevented• 136 X $20,887 = $2,840,632
Copyright 2004 ROI Institute, Inc.
Healthcare, Inc.
Program Benefits =
Program Costs =
BCR = =
ROI = x 100 =
$3,200,908
$277,98711.51
$2,922,921
$277,9871,051.5%
$360,276 + $2,840,632
$277,987
Copyright 2004 ROI Institute, Inc.
What is a good ROI?
• Set the value as with other investments, e.g. 15%
• Set slightly above other investments, e.g. 25%
• Set at break even - 0%
• Set at client expectations
Copyright 2004 ROI Institute, Inc.
During this session we
Defined program benefitsConverted benefits to monetary valueIdentified fully-loaded costsIdentified intangible benefitsCalculated an ROI
Copyright 2004 ROI Institute, Inc.
Remember
Data ConversionWhen in doubt, leave it out!
Use the most conservative alternative
Use the most credible data course
Use the four-part test! Use only first-year
benefits for short-term programs
Cost TabulationWhen in doubt, leave it in!
Tabulate the fully-loaded costs
Develop a consistent cost profile
Use the most conservative alternative
Use the most credible data source
Copyright 2004 ROI Institute, Inc.
Go forth and….Go forth and….
how them the how them the money!money!
t