Embed Size (px)
Citation preview
• This is a Title Presentation of Three Lines Long
• Ju
Show Me the Money! Demonstrating the ROI in Learning and Performance Improvement
June 28, 2017
Stay until theend for yourfree gift.
+
Upon completion you will know
• how to calculate the ROI;
• the difference between the benefit-
cost ratio and the ROI percentage;
• how to balance ROI with other
measures of performance; and
• which programs are candidates for
evaluation up to ROI.
What is ROI?
BCR = Program Benefits
Program Costs
ROI = Net Program Benefits
Program CostsX 100
+
Try it!
Retail Merchandise implemented an
Effective Selling Skills Program targeting increase in sales
on products in the electronics department.
• Benefits = $71,760
• Program Costs = $32,984
• ROI = ???
What is ROI?
BCR = $71,760
$32,984
ROI = $71,760 - $32,984
$32,984X 100
What is ROI?
BCR = $71,760
$32,984
ROI = $71,760 - $32,984
$32,984X 100
= 2.18
= 118%
+
Five Critical Success Factors
To make measurement and evaluation work, you need a systematic,
yet, seamless approach.
Levels of Evaluation Measurement Focus
0. Inputs and IndicatorsThe input into the project in terms of scope,
volume, efficiencies, costsParticipants, hours, costs, timing
1. Reaction & Perceived
Value
Reaction to the project or program, including
the perceived value
Relevance, importance, usefulness,
appropriateness, Intent to use, motivation to
take action
2. Learning
Learning to use the content and materials,
including the confidence to use what was
learned
Skills, Knowledge, Capacity, Competencies,
Confidence, Contacts
3. Application &
Implementation
Use of content and materials in the work
environment, including progress with actual
items and implementation
Extent of use, task completion, behavior
change, frequency of use, actions completed,
success with use, barriers to use, enablers to
use
4. Business Impact
The consequences of the use of the content
and materials expressed as business impact
measures
Improvement in measures of output, quality,
cost, time, customer satisfaction, job
satisfaction, work habits, innovation
Typical Measures
5. ROIComparison of monetary benefits from
program to program costs
Benefit-Cost Ratio (BCR), ROI%, Payback
Period
+
A sample of 235 ROI Methodology users considers the following criteria when
selecting programs for evaluation to impact and ROI.
Criteria For Selecting Programs
Cost of the program 52%
Importance of program to strategic objectives 50%
Executive interest in the evaluation 48%
Visibility of the program 45%
Linkage of programs to operational goals 29%
Life cycle of the program 14%
Investment of time required 7%
Size of target audience 6%
+
ROI Methodology
Process Model
ReportingData AnalysisData CollectionEvaluation Planning
Develop Objectives of
Solution(s)
Develop Evaluation Plans and Baseline
Data
Collect Data During Solution Implementation
Develop Report and
Communicate Results
• Reaction • Learning
• ROI• Input/ • Indicators
Isolate the Effects of Solution
Convert Data to
MonetaryValues
IdentifyIntangibles
CaptureCosts
of Solution
Calculating the Return on
Investment
Collect Data After Solution
Implementation
• Application• Business
Impact
© 2017 Copyright ROI Institute, Inc.
+
Techniques to Isolate Program Effects
Method1. Control Groups2. Trend Line Analysis3. Forecasting Methods4. Participant Estimates5. Manager Estimates6. Sr. Manager Estimates7. Expert Input8. Customer Input
*Survey of Users, N = 235
0%
10%
20%
30%
40%
50%
60%
1 2 3 4 5 6 7 8
32%29%
5%
55%
37%
11%13%
11%
Frequency
+
Guiding Principles
1. Report the complete story
2. Conserve evaluation resources
3. Use the most credible sources of data
4. Choose the most conservative
alternatives
5. Isolate the effects of the program
6. Assume no data, no improvement
7. Adjust estimates for error
8. Throw out the extreme and unsupported
claims
9. Use first year benefits for short-term
programs
10. Include fully-loaded costs
11. Report intangible benefits
12. Communicate results to all stakeholders
12
+
Case Application
Safety Leadership for Construction Project Leaders
• Global Engineering and Construction
Company (GEC)
• Designs and builds large commercial
projects (e.g. chemical plants, paper
mills, municipal water systems)
• 35,000 full-time employees
• 100,000 contractors at construction
sites
• Safety performance unacceptable and
costly
+
Needs Assessment
• Lack of knowledge about
different tools and techniques
available to improve safety
performance
• Project safety leaders not
operating on a proactive basis
• Safety meetings need more
content, better planning, and
improved coordination
• Project safety leaders need to
use available tools for
investigation, causal analysis,
and corrective action
Safety Leadership for Construction Project Leaders
+
• Two-day safety leadership
workshop
• 25 project safety leaders (e.g.
safety manager, safety engineer,
safety superintendent)
• Action planning was a key
component
• Participants identified three
safety and health performance
measures
Solution
Safety Leadership for Construction Project Leaders
+
Level Measurement Focus
1. Reaction Obtain favorable reaction to program and materials on:
• Need for program
• Relevance to project
• Importance to project success
• Identify planned actions
2. Learning After attending this program, participants should be able to:
• Establish safety audits
• Provide feedback and motivate employees
• Investigate accidents
• Solve safety problems3. Application and
Implementation
• Use knowledge, skills, and tools routinely in appropriate
situations
• Complete all steps of action plan
4. Business Impact • Improve at least three safety and health measures
5. Return on Investment 20%
Objectives
Safety Leadership for Construction Project Leaders
+
Data Collection Plan
Safety Leadership for Construction Project Leaders
+
Data Collection Plan
Safety Leadership for Construction Project Leaders
+
ROI Analysis Plan
Safety Leadership for Construction Project Leaders
+
Topic Rating
Need for the program 4.3
Relevance to construction project 4.5
Importance to project success 4.5
Delivery of the program 4.2
Facilitation of the program 4.2
Planned actions developed 100%
1= Unsatisfactory 5 = Exceptional
Target: 4.0
Reaction
Safety Leadership for Construction Project Leaders
+
Topic Rating
Establish safety audits 4.2
Provide feedback and motivation to employees 4.0
Investigate accidents 4.9
Follow safety procedures and standards 4.2
Counsel problem employees 3.9
Conduct safety meetings 4.8
1 = Cannot do this 5 = Can do this extremely well
Target: 4.0
Learning
Safety Leadership for Construction Project Leaders
+
Success with Rating
Conducting safety audits 4.1
Providing feedback to employees 3.9
Investigating accidents 4.8
Solving safety problems 4.9
Following safety procedures and standards 4.7
Counseling problem employees 4.2
Conducting safety meetings 4.6
1= Unsuccessful 5 = Very SuccessfulTarget: 4.0
Application
Safety Leadership for Construction Project Leaders
+
Business Impact
Safety Leadership for Construction Project Leaders
+
Business Impact
Safety Leadership for Construction Project Leaders
+
Needs Assessment (Prorated over three sessions) $1,667
Program Development (Prorated over three sessions) 2,500
Program Materials–25 @ $100 2,500
Travel and Lodging –25 @ $2000 37,500
Facilitation and Coordination 50,000
Facilities and Refreshments – 2 days @ $700 1,400
Participants Salaries Plus Benefits 28,000
ROI Evaluation 4,500
Total $128,067
Program Costs
Safety Leadership for Construction Project Leaders
What is ROI?
BCR = Program Benefits
Program Costs
ROI = Net Program Benefits
Program CostsX 100
What is ROI?
BCR = $539,939
$128,067
ROI = $539,939 - $128,067
$128,067X 100
= 4.22:1
= 322%
+
• Improved productivity
• Improved teamwork
• Improved work quality
• Improved job satisfaction
• Improved job engagement
• Enhanced image
• Reduced stress
Intangible Benefits
Safety Leadership for Construction Project Leaders
+
Credibility of DataSafety Leadership for Construction Project Leaders
• Data came directly from participants (safety
leaders)
• Improvement in measures could be audited
• Only first-year benefits were taken into
consideration
• Monetary value of improvement reflected only
that attributable to the program
• Estimate of contribution was adjusted for error
• Program costs were fully loaded
• Business impact reflects improvement in
measures identified in action plans; it does not
include improvement in other measures that
happened to improve as a result of the program
+
Transnet Coach Business South AfricaSupervisor coaching program focused on improving beat rate and sales targets. ROI = 32.64%
PolyWrigton USAWork engagement intervention focused on controllable waste, rework, and engagement. ROI = 399%
Industrial Company GermanyGlobal sales training focused on improving sales. ROI = 161%
Baptist Hospital USASexual harassment prevention program focused on reduction in complaints, turnover, and bad behavior.ROI = 1051%
Verizon USACall center representative training focused on call escalations.ROI = - 85%
Case Studies
+
Assess Situation
Analyze Maturity Integrate Results
Define Analytical Approach
Invest & Evaluate
Execute
Find Cause
Human Capital Strategy
The Conference Board Human Capital Analytics Research Working Group reports that organizations must have an ecosystem supportive of human capital
analytics implementation in order to fully integrate and sustain the practice. Fitz-enz, J., Phillips, P. P., and Ray, R. (2012). Human Capital Analytics: A Primer. The
Conference Board. Research Report R-1500-12-RR, p. 23.
Seamless Integration
+
1. Assess your readiness for
ROI
2. Identify stakeholders and
their data needs
3. Determine the purpose of
your evaluation practice
4. Identify programs suitable
for ROI
5. Build capability in the
ROI Methodology
Steps to Move Your Practice Forward
We will draw from the contact list of those who attended live!
+
Remember:
All roads lead to ROI!
