View
1.026
Download
7
Embed Size (px)
DESCRIPTION
Citation preview
Measuring ROI in Training
Reaction Objectives
1) Perceive it to be relevant to your work
2) Find it important to your success on the job
3) Find the case studies relevant and useful
Learning Objectives
1) Identify the different evaluation levels
2) Align programs to business needs
3) Identify latest 3 ways to collect data
4) Describe at least 2 ways to isolate the effects of the program
5) Calculate ROI given benefits & cost
Application Objectives
1) Develop an evaluation plan for a specific program
Impact Objectives
1) Improve the effectiveness of the programs
2) Redesign ineffective programs
3) Show ROI on your programs
Six months after completing the workshop, you should:
Case Study [ Reliance Insurance Organization ]
Page 1.4
Discussion Questions
1) Is this situation typical?
2) What are the basic issues in the case?
3) How could this situation be avoided in the future?
4) What should Marge do? Be specific?
5) What should Frank be expecting in the future?
Activity Vs Results
Activity Vs Results
No business need for the program Program linked to specific business needs
No effort to prepare the work environment to support program
Environment prepared to support program
Reporting on program is input focused
Reporting on program is output focused
Objectives are defined at lower levels
Objectives are defined at higher levels
Training Evaluation
is
? ART Science
evolution of evaluation…
1954
Peter Drucker
If you can’t measure it
You can’t manage it
1959
Donald Kirkpatrick
1. Reaction
2. Learning
3. Behavior
4. Results
1970
Jack Phillips
5. ROI
Kirkpatrick Model Phillips ROI Model
Reaction Level 1
Level 2
Level 3
Level 4
Accountability
Process Model
Standards
Publications
Usage
Dr. Donald Kirkpatrick Dr. Jack Phillips Level 1 Developed
Inputs / Indicators Level 0
Level 5
Reaction , Satisfaction
Reaction , Satisfaction & Planned action
Kirkpatrick Model Phillips ROI Model
Reaction Reaction, Satisfaction & Planned Action Level 1
Level 2
Level 3
Level 4
Accountability
Process Model
Standards
Publications
Usage
Dr. Donald Kirkpatrick Dr. Jack Phillips Level 1 Developed
Inputs / Indicators Level 0
Level 5
Learning Learning
Kirkpatrick Model Phillips ROI Model
Reaction Reaction, Satisfaction & Planned Action
Learning Learning
Behaviour Application
Level 1
Level 2
Level 3
Level 4
Accountability
Process Model
Standards
Publications
Usage
Dr. Donald Kirkpatrick Dr. Jack Phillips Level 1 Developed
Inputs / Indicators Level 0
Level 5
Learning
Application
If the participants know the L3 behaviors are being tracked they will be more likely to apply
Kirkpatrick Model Phillips ROI Model
Reaction Reaction, Satisfaction & Planned Action
Learning Learning
Behaviour Application
Results Business Impact
Level 1
Level 2
Level 3
Level 4
Accountability
Process Model
Standards
Publications
Usage
Dr. Donald Kirkpatrick Dr. Jack Phillips Level 1 Developed
Inputs / Indicators Level 0
Level 5
Customer satisfaction
Sales
Units manufactured
Students graduated
increase
Costs
Quality complaints
Time to project completion
Ineffective programs
decrease
Cost avoidance
Kirkpatrick Model Phillips ROI Model
Reaction Reaction, Satisfaction & Planned Action
Learning Learning
Behaviour Application
Results Business Impact
Isolating the effects of the program
Level 1
Level 2
Level 3
Level 4
Accountability
Process Model
Standards
Publications
Usage
Dr. Donald Kirkpatrick Dr. Jack Phillips Level 1 Developed
Inputs / Indicators Level 0
Level 5
Incentives Systems/ Procedures Training
Management Attention
Incentives Systems/ Procedures Training
Management Attention
Kirkpatrick Model Phillips ROI Model
Reaction Reaction, Satisfaction & Planned Action
Learning Learning
Behaviour Application
Results Business Impact
Isolating the effects of the program
Level 1
Level 2
Level 3
Level 4
Accountability
Process Model
Standards
Publications
Usage
Dr. Donald Kirkpatrick Dr. Jack Phillips Level 1 Developed
Inputs / Indicators Level 0
ROE / ROI Level 5
Return on Expectations
(ROE)
Show Me the Money
Return on Investment
(ROI)
Defining BCR & ROI
Benefits/ Cost Ratio = Program Benefits
Program Cost
ROI = Net Program Benefits
Program Costs
X 100
Program Benefits from Participants (1st year) = ` 12,30,000 Costs per Program (25 participants) = ` 6,88,000
Benefits/ Cost Ratio = 12, 30, 000
6,88, 000
= 1.79
ROI = 12,30,000 – 6,88,000
6,88, 000
= 79% X 100
Cause & Effect for 5 Levels
Lost the Evidence
Kirkpatrick Model Phillips ROI Model
Reaction Reaction, Satisfaction & Planned Action
Learning Learning
Behaviour Application
Results Business Impact
Isolating the effects of the program
Level 1
Level 2
Level 3
Level 4
Accountability
Process Model
Standards
Publications
Usage
Dr. Donald Kirkpatrick Dr. Jack Phillips Level 1 Developed
10 step by step of collecting, analyzing, reporting, using systematic & methodic
90–100%
70–80%
20–30%
10–20%
01–05%
Inputs / Indicators Level 0 100%
Return on Investment Level 5
Develop Objectives of Solution (s)
Develop Evaluation Plans and
Baseline Data
Collect Data During
Solution Implementation
Collect Data After Solution
Implementation
Level 2: Learning Level 4: Business Impact
Level 1: Reaction, Satisfaction, and Planned Actions
Level 3: Application/ Implementation
Evaluation Planning Data Collection
THE PROCESS MODEL
Isolate the Effects
Convert Data to Monetary
Value
Calculate the Return on Investment
Tabulate Costs of Solution
Identify Intangible Measures
Generate Impact Study
Level 5: ROI
Intangible Measures
Data Analysis Reporting
Kirkpatrick Model Phillips ROI Model
Reaction Reaction, Satisfaction & Planned Action
Learning Learning
Behaviour Application
Results Business Impact
Isolating the effects of the program
Level 1
Level 2
Level 3
Level 4
Accountability
Process Model
Standards
Publications
Usage
Dr. Donald Kirkpatrick Dr. Jack Phillips Level 1 Developed
10 step by step of collecting, analyzing, reporting, using systematic & methodic
12 guiding principles to be conservative in approach
90–100%
70–80%
20–30%
10–20%
01–05%
Inputs / Indicators Level 0 100%
Return on Investment Level 5
12 Guiding Principles
Job Aid
Kirkpatrick Model Phillips ROI Model
Reaction Reaction, Satisfaction & Planned Action
Learning Learning
Behaviour Application
Results Business Impact
Isolating the effects of the program
Level 1
Level 2
Level 3
Level 4
Accountability
Process Model
Standards
Publications
Usage
Dr. Donald Kirkpatrick Dr. Jack Phillips Level 1 Developed
10 step by step of collecting, analyzing, reporting, using systematic & methodic
12 guiding principles to be conservative in approach
5 books 40 books,3000 case studies
Slowly phasing out 3000 organizations in 50 countries
90–100%
70–80%
20–30%
10–20%
01–05%
Inputs / Indicators Level 0 100%
Return on Investment Level 5
Characteristics of Evaluation Levels
Chain of Value of Customer Frequency Difficulty of Impact Information Focus of Use Assessment Satisfaction Lowest Consumer Frequent Easy Learning Application Impact ROI Highest Client Infrequent Difficult
All program are ‘not evaluated’ at all
levels…
Criteria for L4 and L5
Level Measurement category Current status Evaluation
Targets
0 Inputs and Indicators: 100%
1 Reaction and planned action: 100%
2 To measure what the participants have learned in the program-
90%
3 Application and Implementation: 50%
4 Impact and consequences: 30%
5 Return on investment: 10%
Level Measurement category Current status Evaluation
Targets
0 Inputs and Indicators: Less then 25% of the programs 100%
1 Reaction and planned action: 100% of the workshops conducted. 100%
2 To measure what the participants have learned in the program-
Done for more than 60-70% 90%
3 Application and Implementation: Less than 10% of the training 50%
4 Impact and consequences: Not done for any training 30%
5 Return on investment: Not done for any training 15%
Tata Sky
What is your organization’s
current status?
Evaluation Planning
Category 1
Develop Objectives of Solution (s)
Develop Evaluation Plans and
Baseline Data
Evaluation Planning
Hard Data
Output
Time
Costs
Quality
Page 3.5
Soft Data
Work habits
Climate/ Satisfaction
Initiative/ Innovative
Customer Service
Development
Image
Page 3.7
Baseline Data
Initial collection of data which serves as a basis for
COMPARISON with the subsequently acquired data.
125 Kgs 86 Kgs
Existing Expected
Evaluation Planning Worksheet
Page 3.2 (pre-work)
Analysis
Design
Development
Implementation
Evaluation } ADDIE
Project Business Alignment and Forecasting The ROI Process Model
Business Alignment Model
Learning Needs
Preference Needs
Measurement and Evaluation
Reaction
Learning
Application
Impact
ROI
Reaction Objectives
Learning Objectives
Application Objectives Performance Needs
Impact Objectives Business Needs
Payoff Needs ROI Objectives
End Here Start Here 5
4
3
2
5
4
3
2
1 1
ROI Needs Analysis
Do an ROI based needs
Project
Learning Needs
Preference Needs Reaction
Learning
Application
Impact
ROI
Reaction Objectives
Learning Objectives
Application Objectives Performance Needs
Impact Objectives Business Needs
Payoff Needs ROI Objectives
End Here Start Here 5
4
3
2
5
4
3
2
1 1
2 things
1. Different Hats
Business Head
Trainer
Program Evaluator
2. Cause & Effect
( Training Need)
primary
heuristic
coherent
Stripping Uncertainty in Needs Analysis
Learning Needs
Preference Needs
Performance Needs
Business Needs
Payoff Needs 5
4
3
2
1
Absenteeism is causing 15,00,000 monthly
Unplanned absenteeism is 9%
Discussion between team members and supervisor are not occurring
Counseling and discussion skills
Counseling skills must be practical and should happen immediately
Develop Objectives of Solution (s)
Develop Evaluation Plans and
Baseline Data
Collect Data During
Solution Implementation
Collect Data After Solution
Implementation
Evaluation Planning Data Collection
ROI Objectives
Learning Needs
Preference Needs
Performance Needs
Business Needs
Payoff Needs 5
4
3
2
1
Discussion between team members and supervisor are not occurring
Counseling and discussion skills
Counseling skills must be practical and should happen immediately
4/5 on the practicality of the program and relevance to job
Participants will learn 3 key counseling skills
Counseling discussion conducted in 95% situations when an unexpected absence occurs
Reduce absenteeism to 5%
ROI of 20%
Reaction Objectives
Learning Objectives
Application Objectives
Impact Objectives
Absenteeism is causing 15,00,000 monthly
Unplanned absenteeism is 9%
The best way to Predict a future is to … create it.
“ ”
Develop Objectives of Solution (s)
Develop Evaluation Plans and
Baseline Data
Collect Data During
Solution Implementation
Collect Data After Solution
Implementation
Evaluation Planning Data Collection
V Model
Learning Needs
Preference Needs Reaction
Learning
Application
Impact
ROI
Reaction Objectives
Learning Objectives
Application Objectives Performance Needs
Impact Objectives Business Needs
Payoff Needs ROI Objectives
End Here Start Here 5
4
3
2
5
4
3
2
1 1
Absenteeism is causing `15,00,000 monthly
Unplanned absenteeism is 9% and increasing.
Discussion between team member and supervisor are not occurring
Deficiency in counseling and discussion skills
Counseling skills must be practical and should happen immediately
4/5 on the practicality of the program and relevance to job
Participants will learn 3 key counseling skills
Counseling discussion conducted in 95% situations when an unexpected absence occurs
Reduce absenteeism to 5%
ROI of 20%
Reaction Questionnaire at the end of the program
Practice session during program and facilitator assessment
Follow up questionnaire to participants to check the frequency of discussions
Monitor absenteeism records for 6 months
Calculate ROI, BCR
V Model – Tata Sky
Learning Needs
Preference Needs Reaction
Learning
Application
Impact
ROI
Reaction Objectives
Learning Objectives
Application Objectives Performance Needs
Impact Objectives Business Needs
Payoff Needs ROI Objectives
End Here Start Here 5
4
3
2
5
4
3
2
1 1
Increase Sales Increase EPRS
Increase walkin conversion
Profiling of the customer, Standardization in customer demo, & transaction
Greeting Asking right questions to the prospects to understand the wants and needs, Match the needs with features & benefits
ILT and role plays 4/5 on the practicality of the program and relevance to job
Participants will learn key interaction & influencing skills
Use the skills consistently, which is 100% of the times
+ve ROI
Track business dashboards every month
Calculate ROI, BCR
Increase walk-in conversion ratio Increase the subscription recharge by 10%
Increase Sales by 10%
Action plan activity sheet
Facilitator assessment through role plays
Reaction and planned action through a questionnaire at the end of the program
Learning Needs
Preference Needs
Performance Needs
Business Needs
Payoff Needs 5
4
3
2
1
Increase Sales Increase EPRS
Profiling of the customer, Standardization in customer demo, & transaction
Greeting understand the wants and needs Influencing
ILT and role plays
4/5 on the practicality and relevance
Participants will learn key interaction & influencing skills
Use the skills consistently, which is 100% of the times
+ve ROI
Increase Sales by Increase the subscription recharge by
V Model – Tata Sky
Learning Needs
Preference Needs Reaction
Learning
Application
Impact
ROI
Reaction Objectives
Learning Objectives
Application Objectives Performance Needs
Impact Objectives Business Needs
Payoff Needs ROI Objectives
End Here Start Here 5
4
3
2
5
4
3
2
1 1
Increase Sales Increase EPRS
Profiling of the customer, Standardization in customer demo, & transaction
Greeting Asking right questions to the prospects to understand the wants and needs, Match the needs with features & benefits
ILT and role plays 4/5 on the practicality of the program and relevance to job
Participants will learn key interaction & influencing skills
Use the skills consistently, which is 100% of the times
+ve ROI
Track business dashboards every month
Calculate ROI, BCR
Increase the subscription recharge by 10% Increase Sales by 10%
Action plan activity sheet
Facilitator assessment through role plays
Reaction and planned action through a questionnaire at the end of the program
V Model – Publishing Firm
Learning Needs
Preference Needs Reaction
Learning
Application
Impact
ROI
Reaction Objectives
Learning Objectives
Application Objectives Performance Needs
Impact Objectives Business Needs
Payoff Needs ROI Objectives
End Here Start Here 5
4
3
2
5
4
3
2
1 1
Potential project loss of `1,25,00,000 a year
-Improve quality of emails -Decrease supervisors monitoring time
Excellent email communication skills for a global audience
Structure, tone, style and grammar and global culture
Blended – ILT, LMS and coaching Overall score of 90%
Participants will learn email skills
Use appropriate email writing skills 100% of the times while writing emails
Error rates to zero Decrease managers monitoring time to nil
ROI of 50%
Online reaction Questionnaire
Online pre and post assessment
Email scorecard
Performance records & Interview
Calculate ROI, CBR
Each level includes: • Data Sources • Data Collection • Key Questions • Key Issues
Collect Data After Solution
Implementation
Isolate the Effects
of the Solution
Convert Data To Monetary
Value
Calculate The Return On
Investment
Tabulate Costs of Solution
Identify Intangibles
Significant Influences • Policy Statement • Procedures and Guidelines • Staff Skills • Management Support • Technical Support • Organizational Culture
Phillips ROI Methodology TM
Develop
Report and Communicate
Results
Level 4 Level 3 Level 3 and 2 Level 1
Level 3 Level 4
Level 5
Level 5
Results-Based Approach
Intangible Benefits
Specify Skill/ Knowledge
Deficiencies of Affected Population
Training Required
?
Level 2
Problem/ Opportunity Present or Anticipated
Identify Job Performance
Needs, Gaps, and Why
Identify
Solutions
Identify
Preferences
Develop Objectives/ Evaluation Strategy
Identify Transfer Strategy Options and
L-2 & L-3 Support For All Stakeholders
Identify Business, Needs, Gaps, and Stakeholders
Design Solution and Stakeholder Components
Consider Resources/ Logistics Delivery
Develop Content/ Materials
Conduct/ Implement Solution
Level 1 Level 2
Implement Pre-activity
Implement Transfer Strategy
- Solution - Pre-work Activity/Transfer Strategy
Page 3.9
Page 3.12 ( More Examples )
Exercise 3.16
Develop Objectives of Solution (s)
Develop Evaluation Plans and
Baseline Data
Collect Data During
Solution Implementation
Collect Data After Solution
Implementation
Evaluation Planning Data Collection
Step 2
1. Data collection Plan
2. ROI analysis plan
Evaluation Planning ( 2 docs)
Page 3.24 ( sample )
Level Objectives Measures/Data Data
Collection method
Data Source Timings Responsibility
1 Positive reaction An overall average rating of 4/5
Questionnaire Participants of the program
At the end of each workshop
Facilitator
2 Will be able to demonstrate all the 3 skills during the role play
Facilitators rating Observation of skill practice by facilitator
Facilitator During the workshop
Facilitator
3
(1)Will be able to assess all the EZs for adherence to the taught processes (2)Will be able to bring all the employees to a consistent level of display of learned skills.
Completion of Action plan
Action plan sheet
Retail Operation Incharges(ROIs) Retail Operation Managers(ROMs)
30 – 45 days after the workshop
Regional Managers Retail Operations
4
(1)The walk in conversion ratio will increase (2)The sales will increase (3)The customer retention will increase
Monthly data from the Retail Operation Vertical of the Experience Zones
Business performance monitoring
Dashboard, Retail Operations
45 – 90 days after the workshop
Regional Managers Retail Operations
5 ROI No Objective was set for this level in the beginning. The expectation was to have a positive ROI.
Data Collection Plan– Tata Sky
Level Objectives Measures/Data Data
Collection method
Data Source Timings Responsibility
1
Reaction to the program, content and facilitator.
An overall average rating of 4.5/5
Online questionnaire
Participants of the program
At the end of each workshop
Facilitator
2 Assess the email writing skills Score above 90% in the post assessment
Online test Participant At the end of each workshop
Facilitator
3 Demonstrate appropriate structure, tone, style and grammar in emails
100% score on the score card
Score card Participant
Weekly monitoring of emails for 3 months
Facilitator
4 Improve quality of emails Decrease managers monitoring time
Decrease email errors to zero Decrease manager’s monitoring time to nil
Performance records, Interview
Managers 3 months after training
Facilitator
5 ROI 50% ROI was set.
Data Collection Plan– Publishing
Data Items (Usually Level 4
Method for Isolation
Methods of Converting Data to Monetary Values
Cost Categories Intangible Benefits
Communication Targets for Final Report
Other Influences/ Issues During Application
Monthly conversion ratio
Participant estimate (ROMs & ROI)
Standard Value.
(1)Facilitation Fees (2)Program Materials (3)Market survey Fees (4)Facility costs (5)Participants salaries
ROMs ROIs Exec. V P Retail operations
Any change in market factor
ROI Analysis Plan– Tata Sky
Baseline Data
Tata Sky – Sales Mystery Shopping
• Ambience ( 4 parameters ) • Grooming ( 4 parameters ) • Opening the call (15 parameters ) • Developing/ Identifying the need ( 8 parameters ) • Proposing a solution ( 4 parameters ) • Objection Handling ( 5 parameters ) • Closing (4 parameters ) • Follow up ( 2 parameters ) • Process adherence ( 4 parameters ) • Telephone Related (1 parameters )
Baseline Data
Publishing Firm – Score from emails, reports
Parameters Pre-Assessment
Grammar 63.8 %
Structure 85.4%
Tone & Style 58.3%
Average Score 70.2%
Data Collection
Category 2
Develop Objectives of Solution (s)
Develop Evaluation Plans and
Baseline Data
Collect Data During
Solution Implementation
Collect Data After Solution
Implementation
Level 2: Learning
Level 4: Business Impact
Level 1: Reaction, Satisfaction, and Planned Actions
Level 3: Application/ Implementation
Evaluation Planning Data Collection
Step 3 Step 4
1. Relevance to work
2. Important to success
3. Would recommend to others
4. Amount of new information
5. Method of delivery
6. Learning Environment
7. Planned Action
Level 1 Target Areas
Page 4.3
1. Skills
2. Knowledge
3. Capacity
4. Capability
5. Awareness
Level 2
1. Test
2. Simulation
3. Business games
4. In-basket
5. Case Studies
6. Role playing
7. Assessment center
8. Self, team, facilitator assessment
Techniques for Measuring Learning
Develop Objectives of Solution (s)
Develop Evaluation Plans and
Baseline Data
Collect Data During
Solution Implementation
Collect Data After Solution
Implementation
Level 4: Business Impact
Level 3: Application/ Implementation
Evaluation Planning Data Collection
Step 4
1. Skills - Demonstration of the new skills on the job
2. Knowledge - Take appropriate actions/ decisions based on the
knowledge acquired
3. Capacity – Demonstrate ability on the job
4. Awareness – Demonstrate the required behaviors on the job
Level 3 Target Areas
Increase / Decrease / Cost avoidance in:
1. Hard data
2. Soft Data
Level 4 Target Areas
A. Survey
B. Test
C. Questionnaire
D. Interview
E. Focus Group
F. Observation
G. Performance Records
Data Collection Instruments
S.No Instrument Level 1 Level 2 Level 3 Level 4
1 Survey
2 Test
3 Questionnaire
4 Interview
5 Focus Group
6 Observation
7 Performance Contracts
8 Action Planning
Data Collection Instruments
Page 4.8
Follow up Questionnaire content checklist
Page 4.10
How do you improve the response rate for the questionnaire?
Page 4.21
Page 4.23
Action Planning
Data Analysis
Category 3
Isolate the Effects
Convert Data to Monetary
Value
Calculate the Return on Investment
Tabulate Costs of Solution
Identify Intangible Measures
Generate Impact Study
Level 5: ROI
Intangible Measures
Data Analysis Reporting
Step 5
1. Trend line analysis of performance data
2. Use of a control group arrangement
3. Estimate of the program’s impact (percent)
• Participants
• Supervisor
• Experts
4. Use of experts/previous studies
Techniques to Isolate the effects of Programs
20% 10% J F M A M J J A S O N D J
REJECT RATES
18.5% Pre Program Six-Month Average
Projected Average — Using Pre Data as a Base
14.5%
7% Post Program Six-Month
Average
CPI Program Conducted
MONTHS
Using Trend Analysis
Two Conditions
1. If the training program would not been implemented, would this trend continue on the same path established before the training?
2. No other variables and influences entered the process after the training was conducted
Control Group
Sales & Customer Service Training
Retail Chain (130 stores)
No Program
Program
Measurement
Measurement
Control Group
Experimental
Using Estimates
• Participants estimate
• Supervisor estimate
• Customer Input
• Experts Inputs
Wisdom of Crowds
Francis Galton
Ox Weighted – 1,198 pounds
Crowd guessed– 1,197 pounds
Sales % Attributable to
training Confidence Level
ERPS % Attributable to
training
Confidence Level
Total Average for the Group 42% 86% 42% 88%
Corrected Group average for sales
36% of the increase in Sales of Boxes
Corrected Group average for Subscription Recharge(EPRS)
37% of Increase in Monthly Recharge( EPRS)
Tata Sky used Supervisors Estimates
Data Source
Time Savings in Hours
Attribution Level to Training
Confidence Level
Adjusted Value
Decrease in Supervisor monitoring
time
Supervisors 1.2 hours / day 60% 100% 21.5 days /
year
Project & Strategic
Head 2 hours / week 60% 100% 7.8 days / year
Decrease in escalations of poor quality
Project Head - 60% 100% 4.2 escalations
Publishing Firm used Supervisors & Project Head Estimates
Case Study National Bank
Page 5.10
Exercise 5.5
Isolate the Effects
Convert Data to Monetary
Value
Calculate the Return on Investment
Tabulate Costs of Solution
Identify Intangible Measures
Generate Impact Study
Level 5: ROI
Intangible Measures
Data Analysis Reporting
Step 6
• Converting output to contribution - standard value (profit/savings)
• Converting the cost of quality - standard value
• Converting employee’s time (using compensation)
• Using historical costs/savings
• Using internal and external experts
• Using data from external databases/studies
Methods of converting data to monetary value
Service/Production Workers - Hourly (e.g. Courier) 40 - 70%
Skilled Hourly (e.g. Machinist) 75 - 100% Clerical/Administrative (e.g. Scheduler) 50 - 80% Professional (e.g. Sales Representative, Nurse, Accountant) 75 - 125%
Technical (e.g. Computer Technician) 100 - 150%
Engineers (e.g. Chemical Engineer) 200 - 300% Specialists (e.g. Computer Software Designer) 200 - 400%
Supervisors/Team Leaders (e.g. Section Supervisor) 100 - 150%
Middle Managers (e.g. Department Manager) 125 - 200%
Turnover Summary
Exit cost of previous employee Lost productivity
Recruiting cost Quality problems
Employee cost Customer dissatisfaction
Orientation cost Loss of expertise/knowledge
Training cost Supervisor’s time for turnover
Wages and salaries while training Temporary replacement costs
Parameters of a Turnover
Converting Data to Monetary Values:
Step 1 Define the unit of measure
(Increase in sales, decrease in absenteeism, time saving)
Step 2 Determine the value of each unit One unit of sales, one cost of an absentee, one hour of time saving)
Step 3 Calculate the change in performance Change during the reporting period
Step 4 Determine the annual amount of change
Step 5 Calculate the total value of improvement The value of one unit of improvement multiplied by the amount of annual change during the reporting period
Converting Data to Monetary Values:
Evaluation of a Team-Building Program in a Manufacturing Plant Step 1 Define the unit of measure
Unit of measure = one less grievance
Step 2 Determine the value of each unit Using internal experts on the labour relations staff, the cost of an average grievance was estimated to be 18,000, considering time and direct cost
Step 3 Calculate the change in performance Six months after the program, total grievances per month had declined by 10. Supervisors isolated the effects of the program, determining that 7 of the 10 grievance reductions were because of the program
Step 4 Determine the annual amount of change Using the adjusted value of 7 per month yields an annual improvement of 84
Step 5 Calculate the total value of improvement Annual value = 84 x 18,000 = 15, 12,000
Converting Data to Monetary Values: Winning Every Walk-in – Tata Sky
Step 1 Define the unit of measure
One new customer, Subscription recharge for a month
Step 2 Determine the value of each unit Average revenue earned is Rs. 300 for a set top box(Finance dept. worked out). Increased subscription recharge per month= Rs. 21,27,878 (after deduction 70% cost)
Step 3 Calculate the change in performance Average increment in sales boxes is 1016. Out of which 36% was attributed to training. (1016 x 36)/ 100 = 365 The percentage increase in subscription recharge attributable to training is 37%. (2127878 x 37)/ 100 = 7,87,315
Step 4 Determine the annual amount of change Annual value of increased sales in the set top box : 300 x 365 x 12 = 13,14,000 Annual value of increased in subscription recharge : 7,87,315 x 12 = 94,47,780
Step 5 Calculate the total value of improvement Total benefit : 13,14,000 + 94,47,780 = 1,07,61,780
Exercise 5.21
TURNOVER COSTS SUMMARY
Entry Level - Hourly, Non Skilled (e.g. Fast Food Worker) 30 - 50%
Service/Production Workers - Hourly (e.g. Courier) 40 - 70%
Skilled Hourly (e.g. Machinist) 75 - 100% Clerical/Administrative (e.g. Scheduler) 50 - 80% Professional (e.g. Sales Representative, Nurse, Accountant) 75 - 125%
Technical (e.g. Computer Technician) 100 - 150%
Engineers (e.g. Chemical Engineer) 200 - 300% Specialists (e.g. Computer Software Designer) 200 - 400%
Supervisors/Team Leaders (e.g. Section Supervisor) 100 - 150%
Middle Managers (e.g. Department Manager) 125 - 200%
To Convert or Not to Convert
No
Is there a standard value?
Yes
Is there a Method to get there?
Move to Intangible Benefits
Can we get there with minimum resources?
No
Yes
No No
Can we convince
executive of credibility in 2 minutes?
Convert data and add to
numerator Yes Yes
Add to
numerator
Move to Intangible Benefits
Move to Intangible Benefits
Isolate the Effects
Convert Data to Monetary
Value
Calculate the Return on Investment
Tabulate Costs of Solution
Identify Intangible Measures
Generate Impact Study
Level 5: ROI
Intangible Measures
Data Analysis Reporting
Step 7
Fully load the cost
• Needs Analysis cost
• Development cost
• Delivery cost
• Evaluation cost
• Others
S. No The Cost category Value( In Indian Rupees)
1. Facilitation fees for the Master trainer 2,00,000 2. The cost for the content 12,00,000 3. The cost for the Market survey 25,000
4.
Participants Salaries(As per managerial grades): M1-2 Participants( @ Rs.10000 per day) 40,000 M2 – 2 Participants( @ 7000 per day) 28,000 M3 – 2 Participants( @ 4000 per day) 16,000 M4 – 10 Participants( @ 2800 per day) 56,000 M5 – 22 Participants ( @ 1400 per day) 61,600
5. Cost of internal venue for the six batches done for the Company staff (ROIs and the ROMs). This included rent of venue and food.
1,50,000
6. Cost of the 25 training batches for the Experience Zone Employees (@ Rs.25000 per batch).This included rent of venue and food.
6,25,000
7. Cost of 6 refresher batches – required to tackle the employee churn (@ Rs.25000 per batch).
1,50,000
Total Costs for the training program 25,51,600
Fully loaded Cost – Tata Sky
Isolate the Effects
Convert Data to Monetary
Value
Calculate the Return on Investment
Identify Intangible Measures
Generate Impact Study
Level 5: ROI
Intangible Measures
Data Analysis Reporting
Step 8
Benefits/ Cost Ratio = Program Benefits
Program Cost
ROI = Program Benefits – Program Costs
Program Costs X 100
Calculate BCR & ROI
Program Benefits = 1,07,58,691 Program Costs = 25,51,600
Benefits/ Cost Ratio = 1,07,58,691
2551600
= 4.21
ROI = 1,07,58,691 – 25,51,600
25,51,600 = 321% X 100
Isolate the Effects
Convert Data to Monetary
Value
Calculate the Return on Investment
Tabulate Costs of Solution
Identify Intangible Measures
Generate Impact Study
Level 5: ROI
Data Analysis Reporting
Step 9
Common Intangibles
§ Brand Awareness § Job satisfaction § Organizational Commitment § Work Climate § Employee Engagement § Employee Stress § Employee Innovation § Employee Networking § Employee Collaboration § Employee partnering § Customer Satisfaction /
Dissatisfaction § Reputation § Image
§ Customer Complaints § Customer Response Time § Customer Loyalty § Teamwork § Cooperation
§ Conflict § Decisiveness § Communication § Accountability § Alliances
§ Awards § Capability § Capacity
§ Creativity § Clarity § Caring § Corporate Social
Responsibility
§ Human Life § Intellectual Capital § Leadership § Poverty
§ Sustainability § Work/Life Balance
• The confidence of the Experience zone employee increased due to the training. This reflected in all the activities of the store. The morale of the experience zone employee as well that of the owner increased. The effect was noticeable but since we could not measure the difference in confidence pre and post training, this was reported as an intangible benefit.
• The better customer management at the stores reduced the incoming call load at the contact center. As we could not use any method to isolate the effect we decided to keep it as an intangible method.
• Better handling of customers during the time of sale also ensured that they recharged for the first month more regularly. This reduced the casual churn of the customers due to not recharging their accounts in time. Here again since we could not find a way to isolate this particular effect of training we decided to list this as an intangible benefit.
• The ROIs got skilled on the job coaching. This will help in better channel management.
Intangibles – Tata Sky
Isolate the Effects
Convert Data to Monetary
Value
Calculate the Return on Investment
Tabulate Costs of Solution
Identify Intangible Measures
Generate Impact Study
Level 5: ROI
Intangible Measures
Data Analysis Reporting
Step 10
• General information • Methodology for impact study • Data analysis • Costs • Results • Barriers and enablers • Summary of findings • Conclusions and recommendations • Exhibits
Complete Report
Project Implementation Issues
Shyam Sunder Cofounder & Principal [email protected] www.linkedin.com/in/shyamsundera www.twitter.com/shyamsunder www.greenbooks.co.in
get started