Measuring ROI in Training

Reaction Objectives

1)  Perceive it to be relevant to your work

2)  Find it important to your success on the job

3)  Find the case studies relevant and useful

Learning Objectives

1)  Identify the different evaluation levels

2)  Align programs to business needs

3)  Identify latest 3 ways to collect data

4)  Describe at least 2 ways to isolate the effects of the program

5)  Calculate ROI given benefits & cost

Application Objectives

1)  Develop an evaluation plan for a specific program

Impact Objectives

1)  Improve the effectiveness of the programs

2)  Redesign ineffective programs

3)  Show ROI on your programs

Six months after completing the workshop, you should:

Case Study [ Reliance Insurance Organization ]

Discussion Questions

1)  Is this situation typical?

2)  What are the basic issues in the case?

3)  How could this situation be avoided in the future?

4)  What should Marge do? Be specific?

5)  What should Frank be expecting in the future?

Activity Vs Results

No business need for the program Program linked to specific business needs

No effort to prepare the work environment to support program

Environment prepared to support program

Reporting on program is input focused

Reporting on program is output focused

Objectives are defined at lower levels

Objectives are defined at higher levels

Training Evaluation

? ART Science

evolution of evaluation…

Peter Drucker

If you can’t measure it

You can’t manage it

Donald Kirkpatrick

1.  Reaction

2.  Learning

3.  Behavior

4.  Results

Jack Phillips

5. ROI

Kirkpatrick Model Phillips ROI Model

Reaction Level 1

Level 2

Level 3

Level 4

Accountability

Process Model

Standards

Publications

Dr. Donald Kirkpatrick Dr. Jack Phillips Level 1 Developed

Inputs / Indicators Level 0

Level 5

Reaction , Satisfaction

Reaction , Satisfaction & Planned action

Reaction Reaction, Satisfaction & Planned Action Level 1

Level 2

Level 3

Level 4

Accountability

Process Model

Standards

Publications

Level 5

Learning Learning

Reaction Reaction, Satisfaction & Planned Action

Learning Learning

Behaviour Application

Level 1

Level 2

Level 3

Level 4

Accountability

Process Model

Standards

Publications

Level 5

Learning

Application

If the participants know the L3 behaviors are being tracked they will be more likely to apply

Learning Learning

Results Business Impact

Level 1

Level 2

Level 3

Level 4

Accountability

Process Model

Standards

Publications

Level 5

Customer satisfaction

Units manufactured

Students graduated

increase

Quality complaints

Time to project completion

Ineffective programs

decrease

Cost avoidance

Learning Learning

Isolating the effects of the program

Level 1

Level 2

Level 3

Level 4

Accountability

Process Model

Standards

Publications

Level 5

Incentives Systems/ Procedures Training

Management Attention

Incentives Systems/ Procedures Training

Management Attention

Learning Learning

Level 1

Level 2

Level 3

Level 4

Accountability

Process Model

Standards

Publications

ROE / ROI Level 5

Return on Expectations

Show Me the Money

Return on Investment

Defining BCR & ROI

Benefits/ Cost Ratio = Program Benefits

Program Cost

ROI = Net Program Benefits

Program Costs

Program Benefits from Participants (1st year) = ` 12,30,000 Costs per Program (25 participants) = ` 6,88,000

Benefits/ Cost Ratio = 12, 30, 000

6,88, 000

= 1.79

ROI = 12,30,000 – 6,88,000

6,88, 000

= 79% X 100

Cause & Effect for 5 Levels

Lost the Evidence

Learning Learning

Level 1

Level 2

Level 3

Level 4

Accountability

Process Model

Standards

Publications

10 step by step of collecting, analyzing, reporting, using systematic & methodic

90–100%

70–80%

20–30%

10–20%

01–05%

Inputs / Indicators Level 0 100%

Return on Investment Level 5

Develop Objectives of Solution (s)

Develop Evaluation Plans and

Baseline Data

Collect Data During

Solution Implementation

Collect Data After Solution

Implementation

Level 2: Learning Level 4: Business Impact

Level 1: Reaction, Satisfaction, and Planned Actions

Level 3: Application/ Implementation

Evaluation Planning Data Collection

THE PROCESS MODEL

Isolate the Effects

Convert Data to Monetary

Calculate the Return on Investment

Tabulate Costs of Solution

Identify Intangible Measures

Generate Impact Study

Level 5: ROI

Intangible Measures

Data Analysis Reporting

Learning Learning

Level 1

Level 2

Level 3

Level 4

Accountability

Process Model

Standards

Publications

12 guiding principles to be conservative in approach

90–100%

70–80%

20–30%

10–20%

01–05%

12 Guiding Principles

Job Aid

Learning Learning

Level 1

Level 2

Level 3

Level 4

Accountability

Process Model

Standards

Publications

12 guiding principles to be conservative in approach

5 books 40 books,3000 case studies

Slowly phasing out 3000 organizations in 50 countries

90–100%

70–80%

20–30%

10–20%

01–05%

Characteristics of Evaluation Levels

Chain of Value of Customer Frequency Difficulty of Impact Information Focus of Use Assessment Satisfaction Lowest Consumer Frequent Easy Learning Application Impact ROI Highest Client Infrequent Difficult

All program are ‘not evaluated’ at all

levels…

Criteria for L4 and L5

Level Measurement category Current status Evaluation

Targets

0 Inputs and Indicators: 100%

1 Reaction and planned action: 100%

2 To measure what the participants have learned in the program-

3 Application and Implementation: 50%

4 Impact and consequences: 30%

5 Return on investment: 10%

Level Measurement category Current status Evaluation

Targets

0 Inputs and Indicators: Less then 25% of the programs 100%

1 Reaction and planned action: 100% of the workshops conducted. 100%

2 To measure what the participants have learned in the program-

Done for more than 60-70% 90%

3 Application and Implementation: Less than 10% of the training 50%

4 Impact and consequences: Not done for any training 30%

5 Return on investment: Not done for any training 15%

Tata Sky

What is your organization’s

current status?

Evaluation Planning

Category 1

Baseline Data

Evaluation Planning

Hard Data

Output

Quality

Soft Data

Work habits

Climate/ Satisfaction

Initiative/ Innovative

Customer Service

Development

Baseline Data

Initial collection of data which serves as a basis for

COMPARISON with the subsequently acquired data.

125 Kgs 86 Kgs

Existing Expected

Evaluation Planning Worksheet

2 (pre-work)

Analysis

Design

Development

Implementation

Evaluation } ADDIE

Project Business Alignment and Forecasting The ROI Process Model

Business Alignment Model

Learning Needs

Preference Needs

Measurement and Evaluation

Reaction

Learning

Application

Impact

Reaction Objectives

Learning Objectives

Application Objectives Performance Needs

Impact Objectives Business Needs

Payoff Needs ROI Objectives

End Here Start Here 5

ROI Needs Analysis

Do an ROI based needs

Project

Learning Needs

Preference Needs Reaction

Learning

Application

Impact

Reaction Objectives

Learning Objectives

2 things

1. Different Hats

Business Head

Trainer

Program Evaluator

2. Cause & Effect

( Training Need)

primary

heuristic

coherent

Stripping Uncertainty in Needs Analysis

Learning Needs

Preference Needs

Performance Needs

Business Needs

Payoff Needs 5

Absenteeism is causing 15,00,000 monthly

Unplanned absenteeism is 9%

Discussion between team members and supervisor are not occurring

Counseling and discussion skills

Counseling skills must be practical and should happen immediately

Baseline Data

Collect Data During

Implementation

ROI Objectives

Learning Needs

Preference Needs

Performance Needs

Business Needs

Payoff Needs 5

Discussion between team members and supervisor are not occurring

Counseling and discussion skills

4/5 on the practicality of the program and relevance to job

Participants will learn 3 key counseling skills

Counseling discussion conducted in 95% situations when an unexpected absence occurs

Reduce absenteeism to 5%

ROI of 20%

Reaction Objectives

Learning Objectives

Application Objectives

Impact Objectives

Absenteeism is causing 15,00,000 monthly

Unplanned absenteeism is 9%

The best way to Predict a future is to … create it.

“ ”

Baseline Data

Collect Data During

Implementation

V Model

Learning Needs

Learning

Application

Impact

Reaction Objectives

Learning Objectives

Absenteeism is causing `15,00,000 monthly

Unplanned absenteeism is 9% and increasing.

Discussion between team member and supervisor are not occurring

Deficiency in counseling and discussion skills

4/5 on the practicality of the program and relevance to job

Participants will learn 3 key counseling skills

Counseling discussion conducted in 95% situations when an unexpected absence occurs

Reduce absenteeism to 5%

ROI of 20%

Reaction Questionnaire at the end of the program

Practice session during program and facilitator assessment

Follow up questionnaire to participants to check the frequency of discussions

Monitor absenteeism records for 6 months

Calculate ROI, BCR

V Model – Tata Sky

Learning Needs

Learning

Application

Impact

Reaction Objectives

Learning Objectives

Increase Sales Increase EPRS

Increase walkin conversion

Profiling of the customer, Standardization in customer demo, & transaction

Greeting Asking right questions to the prospects to understand the wants and needs, Match the needs with features & benefits

ILT and role plays 4/5 on the practicality of the program and relevance to job

Participants will learn key interaction & influencing skills

Use the skills consistently, which is 100% of the times

+ve ROI

Track business dashboards every month

Calculate ROI, BCR

Increase walk-in conversion ratio Increase the subscription recharge by 10%

Increase Sales by 10%

Action plan activity sheet

Facilitator assessment through role plays

Reaction and planned action through a questionnaire at the end of the program

Learning Needs

Preference Needs

Performance Needs

Business Needs

Payoff Needs 5

Greeting understand the wants and needs Influencing

ILT and role plays

4/5 on the practicality and relevance

+ve ROI

Increase Sales by Increase the subscription recharge by

V Model – Tata Sky

Learning Needs

Learning

Application

Impact

Reaction Objectives

Learning Objectives

Greeting Asking right questions to the prospects to understand the wants and needs, Match the needs with features & benefits

ILT and role plays 4/5 on the practicality of the program and relevance to job

+ve ROI

Track business dashboards every month

Calculate ROI, BCR

Increase the subscription recharge by 10% Increase Sales by 10%

Action plan activity sheet

Facilitator assessment through role plays

Reaction and planned action through a questionnaire at the end of the program

V Model – Publishing Firm

Learning Needs

Learning

Application

Impact

Reaction Objectives

Learning Objectives

Potential project loss of `1,25,00,000 a year

-Improve quality of emails -Decrease supervisors monitoring time

Excellent email communication skills for a global audience

Structure, tone, style and grammar and global culture

Blended – ILT, LMS and coaching Overall score of 90%

Participants will learn email skills

Use appropriate email writing skills 100% of the times while writing emails

Error rates to zero Decrease managers monitoring time to nil

ROI of 50%

Online reaction Questionnaire

Online pre and post assessment

Email scorecard

Performance records & Interview

Calculate ROI, CBR

Each level includes: •  Data Sources •  Data Collection •  Key Questions •  Key Issues

Implementation

Isolate the Effects

of the Solution

Convert Data To Monetary

Calculate The Return On

Investment

Identify Intangibles

Significant Influences •  Policy Statement •  Procedures and Guidelines •  Staff Skills •  Management Support •  Technical Support •  Organizational Culture

Phillips ROI Methodology TM

Develop

Report and Communicate

Results

Level 4 Level 3 Level 3 and 2 Level 1

Level 3 Level 4

Level 5

Results-Based Approach

Intangible Benefits

Specify Skill/ Knowledge

Deficiencies of Affected Population

Training Required

Level 2

Problem/ Opportunity Present or Anticipated

Identify Job Performance

Needs, Gaps, and Why

Identify

Solutions

Identify

Preferences

Develop Objectives/ Evaluation Strategy

Identify Transfer Strategy Options and

L-2 & L-3 Support For All Stakeholders

Identify Business, Needs, Gaps, and Stakeholders

Design Solution and Stakeholder Components

Consider Resources/ Logistics Delivery

Develop Content/ Materials

Conduct/ Implement Solution

Level 1 Level 2

Implement Pre-activity

Implement Transfer Strategy

- Solution -  Pre-work Activity/Transfer Strategy

12 ( More Examples )

Exercise 3.16

Baseline Data

Collect Data During

Implementation

Step 2

1.  Data collection Plan

2.  ROI analysis plan

Evaluation Planning ( 2 docs)

24 ( sample )

Level Objectives Measures/Data Data

Collection method

Data Source Timings Responsibility

1 Positive reaction An overall average rating of 4/5

Questionnaire Participants of the program

At the end of each workshop

Facilitator

2 Will be able to demonstrate all the 3 skills during the role play

Facilitators rating Observation of skill practice by facilitator

Facilitator During the workshop

Facilitator

(1)Will be able to assess all the EZs for adherence to the taught processes (2)Will be able to bring all the employees to a consistent level of display of learned skills.

Completion of Action plan

Action plan sheet

Retail Operation Incharges(ROIs) Retail Operation Managers(ROMs)

30 – 45 days after the workshop

Regional Managers Retail Operations

(1)The walk in conversion ratio will increase (2)The sales will increase (3)The customer retention will increase

Monthly data from the Retail Operation Vertical of the Experience Zones

Business performance monitoring

Dashboard, Retail Operations

45 – 90 days after the workshop

Regional Managers Retail Operations

5 ROI No Objective was set for this level in the beginning. The expectation was to have a positive ROI.

Data Collection Plan– Tata Sky

Level Objectives Measures/Data Data

Collection method

Data Source Timings Responsibility

Reaction to the program, content and facilitator.

An overall average rating of 4.5/5

Online questionnaire

Participants of the program

At the end of each workshop

Facilitator

2 Assess the email writing skills Score above 90% in the post assessment

Online test Participant At the end of each workshop

Facilitator

3 Demonstrate appropriate structure, tone, style and grammar in emails

100% score on the score card

Score card Participant

Weekly monitoring of emails for 3 months

Facilitator

4 Improve quality of emails Decrease managers monitoring time

Decrease email errors to zero Decrease manager’s monitoring time to nil

Performance records, Interview

Managers 3 months after training

Facilitator

5 ROI 50% ROI was set.

Data Collection Plan– Publishing

Data Items (Usually Level 4

Method for Isolation

Methods of Converting Data to Monetary Values

Cost Categories Intangible Benefits

Communication Targets for Final Report

Other Influences/ Issues During Application

Monthly conversion ratio

Participant estimate (ROMs & ROI)

Standard Value.

(1)Facilitation Fees (2)Program Materials (3)Market survey Fees (4)Facility costs (5)Participants salaries

ROMs ROIs Exec. V P Retail operations

Any change in market factor

ROI Analysis Plan– Tata Sky

Baseline Data

Tata Sky – Sales Mystery Shopping

•  Ambience ( 4 parameters ) •  Grooming ( 4 parameters ) •  Opening the call (15 parameters ) •  Developing/ Identifying the need ( 8 parameters ) •  Proposing a solution ( 4 parameters ) •  Objection Handling ( 5 parameters ) •  Closing (4 parameters ) •  Follow up ( 2 parameters ) •  Process adherence ( 4 parameters ) •  Telephone Related (1 parameters )

Baseline Data

Publishing Firm – Score from emails, reports

Parameters Pre-Assessment

Grammar 63.8 %

Structure 85.4%

Tone & Style 58.3%

Average Score 70.2%

Data Collection

Category 2

Baseline Data

Collect Data During

Implementation

Level 2: Learning

Level 4: Business Impact

Level 1: Reaction, Satisfaction, and Planned Actions

Step 3 Step 4

1.  Relevance to work

2.  Important to success

3.  Would recommend to others

4.  Amount of new information

5.  Method of delivery

6.  Learning Environment

7.  Planned Action

Level 1 Target Areas

1.  Skills

2.  Knowledge

3.  Capacity

4.  Capability

5.  Awareness

Level 2

1.  Test

2.  Simulation

3.  Business games

4.  In-basket

5.  Case Studies

6.  Role playing

7.  Assessment center

8.  Self, team, facilitator assessment

Techniques for Measuring Learning

Baseline Data

Collect Data During

Implementation

Level 4: Business Impact

Step 4

1.  Skills - Demonstration of the new skills on the job

2.  Knowledge - Take appropriate actions/ decisions based on the

knowledge acquired

3.  Capacity – Demonstrate ability on the job

4.  Awareness – Demonstrate the required behaviors on the job

Increase / Decrease / Cost avoidance in:

1.  Hard data

2.  Soft Data

A. Survey

B.  Test

C. Questionnaire

D.  Interview

E.  Focus Group

F.  Observation

G. Performance Records

Data Collection Instruments

S.No Instrument Level 1 Level 2 Level 3 Level 4

1 Survey

2 Test

3 Questionnaire

4 Interview

5 Focus Group

6 Observation

7 Performance Contracts

8 Action Planning

Data Collection Instruments

Follow up Questionnaire content checklist

How do you improve the response rate for the questionnaire?

Action Planning

Data Analysis

Category 3

Isolate the Effects

Level 5: ROI

Intangible Measures

Step 5

1.  Trend line analysis of performance data

2.  Use of a control group arrangement

3.  Estimate of the program’s impact (percent)

•  Participants

•  Supervisor

•  Experts

4.  Use of experts/previous studies

Techniques to Isolate the effects of Programs

20% 10% J F M A M J J A S O N D J

REJECT RATES

18.5% Pre Program Six-Month Average

Projected Average — Using Pre Data as a Base

7% Post Program Six-Month

Average

CPI Program Conducted

MONTHS

Using Trend Analysis

Two Conditions

1.  If the training program would not been implemented, would this trend continue on the same path established before the training?

2.  No other variables and influences entered the process after the training was conducted

Control Group

Sales & Customer Service Training

Retail Chain (130 stores)

No Program

Program

Measurement

Control Group

Experimental

Using Estimates

•  Participants estimate

•  Supervisor estimate

•  Customer Input

•  Experts Inputs

Wisdom of Crowds

Francis Galton

Ox Weighted – 1,198 pounds

Crowd guessed– 1,197 pounds

Sales % Attributable to

training Confidence Level

ERPS % Attributable to

training

Confidence Level

Total Average for the Group 42% 86% 42% 88%

Corrected Group average for sales

36% of the increase in Sales of Boxes

Corrected Group average for Subscription Recharge(EPRS)

37% of Increase in Monthly Recharge( EPRS)

Tata Sky used Supervisors Estimates

Data Source

Time Savings in Hours

Attribution Level to Training

Confidence Level

Adjusted Value

Decrease in Supervisor monitoring

Supervisors 1.2 hours / day 60% 100% 21.5 days /

Project & Strategic

Head 2 hours / week 60% 100% 7.8 days / year

Decrease in escalations of poor quality

Project Head - 60% 100% 4.2 escalations

Publishing Firm used Supervisors & Project Head Estimates

Case Study National Bank

Exercise 5.5

Isolate the Effects

Level 5: ROI

Intangible Measures

Step 6

•  Converting output to contribution - standard value (profit/savings)

•  Converting the cost of quality - standard value

•  Converting employee’s time (using compensation)

•  Using historical costs/savings

•  Using internal and external experts

•  Using data from external databases/studies

Methods of converting data to monetary value

Service/Production Workers - Hourly (e.g. Courier) 40 - 70%

Skilled Hourly (e.g. Machinist) 75 - 100% Clerical/Administrative (e.g. Scheduler) 50 - 80% Professional (e.g. Sales Representative, Nurse, Accountant) 75 - 125%

Technical (e.g. Computer Technician) 100 - 150%

Engineers (e.g. Chemical Engineer) 200 - 300% Specialists (e.g. Computer Software Designer) 200 - 400%

Supervisors/Team Leaders (e.g. Section Supervisor) 100 - 150%

Middle Managers (e.g. Department Manager) 125 - 200%

Turnover Summary

Exit cost of previous employee Lost productivity

Recruiting cost Quality problems

Employee cost Customer dissatisfaction

Orientation cost Loss of expertise/knowledge

Training cost Supervisor’s time for turnover

Wages and salaries while training Temporary replacement costs

Parameters of a Turnover

Converting Data to Monetary Values:

Step 1 Define the unit of measure

(Increase in sales, decrease in absenteeism, time saving)

Step 2 Determine the value of each unit One unit of sales, one cost of an absentee, one hour of time saving)

Step 3 Calculate the change in performance Change during the reporting period

Step 4 Determine the annual amount of change

Step 5 Calculate the total value of improvement The value of one unit of improvement multiplied by the amount of annual change during the reporting period

Converting Data to Monetary Values:

Evaluation of a Team-Building Program in a Manufacturing Plant Step 1 Define the unit of measure

Unit of measure = one less grievance

Step 2 Determine the value of each unit Using internal experts on the labour relations staff, the cost of an average grievance was estimated to be 18,000, considering time and direct cost

Step 3 Calculate the change in performance Six months after the program, total grievances per month had declined by 10. Supervisors isolated the effects of the program, determining that 7 of the 10 grievance reductions were because of the program

Step 4 Determine the annual amount of change Using the adjusted value of 7 per month yields an annual improvement of 84

Step 5 Calculate the total value of improvement Annual value = 84 x 18,000 = 15, 12,000

Converting Data to Monetary Values: Winning Every Walk-in – Tata Sky

Step 1 Define the unit of measure

One new customer, Subscription recharge for a month

Step 2 Determine the value of each unit Average revenue earned is Rs. 300 for a set top box(Finance dept. worked out). Increased subscription recharge per month= Rs. 21,27,878 (after deduction 70% cost)

Step 3 Calculate the change in performance Average increment in sales boxes is 1016. Out of which 36% was attributed to training. (1016 x 36)/ 100 = 365 The percentage increase in subscription recharge attributable to training is 37%. (2127878 x 37)/ 100 = 7,87,315

Step 4 Determine the annual amount of change Annual value of increased sales in the set top box : 300 x 365 x 12 = 13,14,000 Annual value of increased in subscription recharge : 7,87,315 x 12 = 94,47,780

Step 5 Calculate the total value of improvement Total benefit : 13,14,000 + 94,47,780 = 1,07,61,780

Exercise 5.21

TURNOVER COSTS SUMMARY

Entry Level - Hourly, Non Skilled (e.g. Fast Food Worker) 30 - 50%

Service/Production Workers - Hourly (e.g. Courier) 40 - 70%

Skilled Hourly (e.g. Machinist) 75 - 100% Clerical/Administrative (e.g. Scheduler) 50 - 80% Professional (e.g. Sales Representative, Nurse, Accountant) 75 - 125%

Technical (e.g. Computer Technician) 100 - 150%

Engineers (e.g. Chemical Engineer) 200 - 300% Specialists (e.g. Computer Software Designer) 200 - 400%

Supervisors/Team Leaders (e.g. Section Supervisor) 100 - 150%

Middle Managers (e.g. Department Manager) 125 - 200%

To Convert or Not to Convert

Is there a standard value?

Is there a Method to get there?

Move to Intangible Benefits

Can we get there with minimum resources?

Can we convince

executive of credibility in 2 minutes?

Convert data and add to

numerator Yes Yes

Add to

numerator

Move to Intangible Benefits

Isolate the Effects

Level 5: ROI

Intangible Measures

Step 7

Fully load the cost

•  Needs Analysis cost

•  Development cost

•  Delivery cost

•  Evaluation cost

•  Others

S. No The Cost category Value( In Indian Rupees)

1. Facilitation fees for the Master trainer 2,00,000 2. The cost for the content 12,00,000 3. The cost for the Market survey 25,000

Participants Salaries(As per managerial grades): M1-2 Participants( @ Rs.10000 per day) 40,000 M2 – 2 Participants( @ 7000 per day) 28,000 M3 – 2 Participants( @ 4000 per day) 16,000 M4 – 10 Participants( @ 2800 per day) 56,000 M5 – 22 Participants ( @ 1400 per day) 61,600

5. Cost of internal venue for the six batches done for the Company staff (ROIs and the ROMs). This included rent of venue and food.

1,50,000

6. Cost of the 25 training batches for the Experience Zone Employees (@ Rs.25000 per batch).This included rent of venue and food.

6,25,000

7. Cost of 6 refresher batches – required to tackle the employee churn (@ Rs.25000 per batch).

1,50,000

Total Costs for the training program 25,51,600

Fully loaded Cost – Tata Sky

Isolate the Effects

Level 5: ROI

Intangible Measures

Step 8

Benefits/ Cost Ratio = Program Benefits

Program Cost

ROI = Program Benefits – Program Costs

Program Costs X 100

Calculate BCR & ROI

Program Benefits = 1,07,58,691 Program Costs = 25,51,600

Benefits/ Cost Ratio = 1,07,58,691

2551600

= 4.21

ROI = 1,07,58,691 – 25,51,600

25,51,600 = 321% X 100

Isolate the Effects

Level 5: ROI

Step 9

Common Intangibles

§  Brand Awareness §  Job satisfaction §  Organizational Commitment §  Work Climate §  Employee Engagement §  Employee Stress §  Employee Innovation §  Employee Networking §  Employee Collaboration §  Employee partnering §  Customer Satisfaction /

Dissatisfaction §  Reputation §  Image

§  Customer Complaints §  Customer Response Time §  Customer Loyalty §  Teamwork §  Cooperation

§  Conflict §  Decisiveness §  Communication §  Accountability §  Alliances

§  Awards §  Capability §  Capacity

§  Creativity §  Clarity §  Caring §  Corporate Social

Responsibility

§  Human Life §  Intellectual Capital §  Leadership §  Poverty

§  Sustainability §  Work/Life Balance

•  The confidence of the Experience zone employee increased due to the training. This reflected in all the activities of the store. The morale of the experience zone employee as well that of the owner increased. The effect was noticeable but since we could not measure the difference in confidence pre and post training, this was reported as an intangible benefit.

•  The better customer management at the stores reduced the incoming call load at the contact center. As we could not use any method to isolate the effect we decided to keep it as an intangible method.

•  Better handling of customers during the time of sale also ensured that they recharged for the first month more regularly. This reduced the casual churn of the customers due to not recharging their accounts in time. Here again since we could not find a way to isolate this particular effect of training we decided to list this as an intangible benefit.

•  The ROIs got skilled on the job coaching. This will help in better channel management.

Intangibles – Tata Sky

Isolate the Effects

Level 5: ROI

Intangible Measures

Step 10

•  General information •  Methodology for impact study •  Data analysis •  Costs •  Results •  Barriers and enablers •  Summary of findings •  Conclusions and recommendations •  Exhibits

Complete Report

Project Implementation Issues

Shyam Sunder Cofounder & Principal ss@greenbooks.co.in www.linkedin.com/in/shyamsundera www.twitter.com/shyamsunder www.greenbooks.co.in

get started

Measuring ROI in Training

Education

Measuring RoI for Transmedia

Measuring ROI of Training.ppt

Social Media Marketing - Measuring ROI

Measuring ROI on Inﬂuencer Marketing

B2B Marketing ROI: Measuring Success

Measuring Sponsor ROI in Europe

Measuring the ROI of Learning

Measuring Your Marketing ROI

Measuring ROI

The Real ROI of Leadership Training - MeasureUp … · Further, when it comes to measuring the powerful effects of leadership training, ... HOW TO MEASURE THE ROI OF LEADERSHIP TRAINING

Take the Guesswork Out of Measuring ROI for Sales Training

Measuring ROI of Training

Measuring ROI to Make the Case for Training

Measuring the ROI of Software Training - Eton メンズ ...versitas.com/wp-content/uploads/2015/09/Versitas-ROI-eBook.pdf · Measuring the ROI of Software Training ... learning the

Measuring digital roi

Measuring and Maximising Training ROI Training · Business Strategy: the Need for Long-term Planning for Future Skills and Competences Establishing Training Needs Aligning Training

Measuring the ROI of HR

Measuring & Maximising Training ROI - AZTECHaztechtraining.com/.../2016/01/Measuring-Maximising-Training-ROI.pdfMeasuring & Maximising Training ROI An HR ... • Monitoring and evaluating

Measuring Success and ROI in Corporate Training · PDF fileMeasuring Success and ROI in Corporate Training Journal of Asynchronous Learning Networks, Volume 14: Issue 29! A. Instructional

Measuring ROI in Training with Case Studies