Show Me The Money!

JJaacckk JJ.. PPhhiilllliippss,, PPhh..DD..

PPaattttii PP.. PPhhiilllliippss,, PPhh..DD..

Phone (205) 678-8101 Fax (205) 678-8102

www.roiinstitute.net © 2007 ROI Institute, Inc.

Learning Objectives • Identify the major reasons for the growth and use of ROI as

a measurement tool • Identify the key steps and principles in the ROI

Methodology • Identify the issues in a successful implementation and

examine the trends for use of ROI in the future

1

The Value Evolution

“Show me the money!” We have heard it before, sometimes as a serious request and other times as a half-hearted appeal to “prove it or go away.” Either way, it represents the newest value statement. These days the concept of value of projects, programs, and processes has changed dramatically. Consider the following:

• Unilever seeks a financial return on its investment in the arts. • The US Air Force has developed the return on investment in data security to prevent intrusion

into its databases. • Eskom, a large electric utility based in South Africa is interested in the financial impact of a major

transition program in its transmission division. • The BBC in London is interested in knowing the financial impact of its major leadership program. • Businessweek.com has developed an ROI award for Hollywood’s big earners comparing the net

benefit divided by the cost of the investment in actors. • Apple Computer has calculated the ROI for investing in process improvement teams. • Sprint/NEXTEL has developed the ROI on its diversity program. • The Australian Capital Territory Community Care agency forecast the ROI for the

implementation of a client relationship management (CRM) system. • GlaxoSmithKline is interested in showing the value of providing faster internet connections for its

pharmaceutical sales representatives. • Accenture has calculated the ROI on a new sales platform for its consultants. • Wachovia has developed the forecast and actual ROI for its negotiations program. • A major hotel chain has calculated the financial value and ROI of its coaching program. • The cities of New York, San Francisco, and Phoenix are showing the monetary value of

investing in projects and programs to reduce the number of homeless citizens on the streets. • The US Navy has developed the financial ROI on a succession program for top leaders. • Cisco Systems is measuring the ROI for its key meetings and events. • A major US Defense Department agency has developed the ROI for a masters degree program

offered by a major university. Questions for Discussion: 1. Any surprises here?

2. How do you define/describe value?

The Many Applications of the ROI Methodology

Communications Consulting/OD

Policies/Procedures Procurement/Supply Chain

Educational Systems Public Policy Programs Human Resources/Human Capital Public Relations / Public Affairs Leadership and Coaching Quality / Six Sigma Learning and Development Research and Development Marketing/Advertising Social Programs Meetings and Events Technology / Systems

2

The “New” Definition of Value Value Must:

• Be balanced, with qualitative and quantitative data • Contain financial and non-financial perspectives • Reflect strategic and tactical issues • Represent different time frames • Satisfy all key stakeholders • Be consistent in collection and analysis • Be grounded in conservative standards • Come from credible sources • Reflect efficiency in its development • Create a call for action

The Changing Face of Value

Old New

Activity cost center Results profit center Expense control Maximize Value Human Resources as expenses Human capital as investment Rule centered Client centered We value what IBM and GE value Our value systems are unique Tolerate overhead Outsource or automate overhead Add value with small pieces Add value with integrated applications Most of us are in a support role All of us are in a sales role Our measures are based on benchmarking Our measures are based on what we need We view value from one perspective We view value from everyone’s perspective Just another days work in the cubicle We do something of value A job A performance Treat old ideas as new ideas Treat old ideas as old ideas

The “Show Me” Evolution

Term Issue Show Me! Collect Impact Data

Show Me the Money!

And Convert Data to Money

Show Me the Real Money!

And Isolate the Effects of the Project

Show Me the Real Money, And Make Me Believe it!

And Compare the Money

to the Cost of the Project

3

How “Show Me the Money” Connects

Varieties of Terms and Applications

Name Example Program A leadership development process for senior executives Project A re-engineering project for the plastics division System A fully networked system for all branches Initiative A faith-based initiative to reduce recidivism Policy A new pre-school policy for the disadvantaged

Procedure A new scheduling procedure for truck drivers Event A golf outing for customers

Meeting US Coast Guard innovations conference Process Quality sampling process People Staff additions in the customer care center

Tool A new selection tool for the hotel staff

SHOW

ME THE

MONEY!!

Organizational Measures (Sponsor)

Personal Measures

(Stakeholders)

Financial Measures

(CFO)

4

The Results*

Consider your most important project or program (i.e., a strategic, expensive, high-profile project that attracts management attention). You conduct an impact study to measure the success of the program. You discover that four months after the program was completed, the participants:

• reacted very positively to the program and found it to be very relevant to their work;

• learned new skills and gained new insights about themselves; • utilized the skills and insights routinely with their teams, although they had some

difficulty in a few areas; • improved several important work unit measures, with some measures improving

as much as 28%; • achieved an impressive 105% return on investment; and • reported an increase in job satisfaction in the work unit.

Questions for discussion: 1. What are your concerns and issues about the data?

2. What would these results mean for the program?

3. If the above items were negative, what would it mean for the program?

4. How should the results be used?

Key Issues with This Level of Analysis

• Objectives? • Credibility of data? • Source of data? • Consistent methodology? • Scope? • Standards? • Use of data? • Cost of process? • Fear of data?

5

Project or Program Value Chain

Level Measurement Focus

0. Input

Measures input such as volume and efficiencies

1. Reaction & Planned Action

Measures participant reaction to the program and captures planned actions

2. Learning Measures changes in knowledge, skills, and attitudes

3. Application Measures changes in on-the-job behavior or actions and progress with actions

4. Business Impact Captures changes in business impact measures

5. ROI Compares program benefits to the costs

Global Trends in Measurement and Evaluation*

• Organizations are moving up the value chain in their evaluation strategies • Investment is increasing to 3-5% of the budget • Increased focus is driven by clients and sponsors • ROI is the fastest growing metric • Evaluation data is used to drive improvement and secure funding • Evaluation is addressed early and often in the implementation cycle • Processes are systematic and methodical, often designed into the delivery and

implementation processes • Technology is significantly enhancing processing

*Based on benchmarking with over 200+ organizations using a comprehensive measurement and evaluation process.

The Measurement Landscape is confusing…

Balanced Scorecard PROGRAM IMPACT

Evaluation Shareholder Value

Strategic Accountability

Benefits vs Costs ECONOMIC

VALUE ADDED

BOTTOM LINE CONTRIBUTION Profitability

Effectiveness Vital Signs

ROI Value Based

PERFORMANCE STANDARDS

6

ROI Dilemma

Shifting Paradigms Activity Based Results Based

• No business need for the program • Program linked to specific business • No assessment of performance issues • Assessment of performance effectiveness • No specific measurable objectives • Specific objectives for behavior and

business impact • No effort to prepare program participants

to achieve results • Results expectations communicated to

participants • No effort to prepare the work

environment to support application • Environment prepared to support

application • No efforts to build partnerships with key

managers • Partnerships established with key

managers and clients • No measurement of results or ROI

analysis • Measurement of results and ROI analysis

• Planning and reporting are input focused • Planning and reporting are outcome focused

The ROI Process Collects Six Types of Data

1. Reaction and Planned Action 2. Learning and Confidence 3. Application and Implementation 4. Business Impact 5. Return on investment 6. Intangible benefits

. . . and includes a technique to isolate the effects of the program or solution.

The use of the ROI Process is Impressive:

• Process refined over a 25-year period • Thousands of impact studies conducted each year • More than 100 case studies published on ROI use • More than 15,000 have attended a two day ROI workshop • More than 3,000 individuals certified to implement the ROI Process • More than 16 books developed to support the process • ROI Process adopted by hundreds of organizations in manufacturing, service, non-profit, and

government settings in 44 countries - See the ROI Fact Sheet –

15-20% of organizations are currently using ROI

Wish List ROI ----- ----- ----- -----

Use List ----- ----- ----- ----- ROI

HIGH

LOW

70-80% of organizations want to use ROI

Why the gap?

7

Why Use Impact and ROI Analysis? Reactive

• Show contributions of selected programs • Justify/defend budgets • Identify inefficient programs that need to be

redesigned or eliminated

Proactive • Aligns learning to business needs • Earn respect of senior management /

administrators • Improve support for projects • Enhance design and implementation processes • Identify successful programs that can be

implemented in other areas • Earn a “seat at the table”

Applications • Learning and Development • Career Development • Competency Systems • Diversity Programs • E-Learning • Executive Coaching • Gainsharing • Meetings and Events • Leadership Development

• Organization Development • Orientation Systems • Recruiting Strategies • Safety & Health Programs • Self-Directed Teams • Skill-Based/Knowledge-Based

Compensation • Technology Implementation • Quality Programs • Wellness/Fitness Initiatives

Who is Using the Methodology?

• Accenture • Apple Computer • AT&T • Bank of America • British Telecom • Caremark Rx • Coors Brewing Company • Eli Lilly • Federal Express • Hewlett-Packard

• Hilton Hotels • IBM • Microsoft • Nokia • PricewaterhouseCoopers • SAP • Scotia Bank • Verizon Communications • Wachovia • Wal-Mart

Over 2,000 private sector organizations

Who is Using the Methodology? • US Department of Defense • US Department of Labor • US Department of Veteran Affairs • US Government Accountability Office • US Office of Personnel Management • NASA • EPA

• European Patent Office • British Government • New Zealand • Singapore • Italy • Australia • Canada • Poland

Over 200 Government Units

8

ROI Quiz

True or False? Please choose the answer you feel is most correct

T F

1. The ROI Methodology generates just one data item, expressed as a percentage.

2. A program with monetary benefits of $200,000 and costs of $100,000 translates into a 200% ROI.

3. The ROI Methodology is a tool to strengthen and improve the projects, programs, and processes.

4. After reviewing a detailed ROI impact study, senior executives will usually require ROI studies on all programs.

5. ROI studies should be conducted very selectively, usually involving 5-10% of programs.

6. While it may be a rough estimate, it is always possible to isolate the effects of a program on impact data

7. A program costing $100 per participant, designed to teach basic skills with job related software, is an ideal program for an ROI impact study.

8. Data can always be converted to monetary value, credibly.

9. The ROI Methodology contains too many complicated formulas.

10. The ROI Methodology can be implemented for about 3-5% of my budget.

11. ROI is not future oriented; it only reflects past performance.

12. ROI is not possible for soft skills programs.

13. If an ROI impact study, conducted on an existing program, shows a negative ROI, the client is usually already aware of the program’s lack of results.

14. The best time to consider an ROI evaluation is three months after the program is completed.

15. In the early stages of implementation, the ROI Methodology is a process improvement tool and not a performance evaluation tool for the staff.

16. If senior executives are not asking for ROI, there is no need to pursue the ROI Methodology.

9

So, how did you do?

Now that the answers to the quiz have been explained, see how you fared. Tally your scores. Based on the interpretations below, what is your ROI acumen? No. of Correct Responses Interpretation

14-16 You could be an ROI consultant 10-13 You could be a speaker at the next ROI Conference 7-9 You need a copy of a thick ROI book 4-6 You need to attend a two-day ROI workshop 1-3 You need to attend the ROI certification

10

Basic Elements

An EvaluationFramework

Case Applicationsand Practice

A ProcessModel

OperatingStandards and

Philosophy

Implementation

Chain of Impact

Reaction & Planned Action

Learning & Confidence

Application & Implementation

Impact

ROI

Isolate the Effects of the Program

Intangible Benefits

Needs Program

Assessment Objectives Evaluation

Business Impact BusinessNeeds Objectives Impact

Job Performance Application ApplicationNeeds Objectives

Skills/Knowledge Learning LearningNeeds Objectives

SatisfactionPreferences Objectives Reaction

4 4

3 3

2 2

1 1

Potential ROI ROIPayoffs Objectives5 5

How often does this alignment exist?

How many of these elements do you have in place?

11

ISOLATEEFFECTS

OF SOLUTION

CALCULATETHE RETURN ON

INVESTMENT

CAPTURECOSTS OFSOLUTION

IDENTIFYINTANGIBLEMEASURES

DEVELOP EVALUATION PLANS AND

BASELINE DATA

GENERATEIMPACT STUDY

REPORT

COLLECT DATADURING

SOLUTION IMPLEMENTATION

DATA COLLLECTIONEVALUATION PLANNING DATA ANALYSIS REPORTING

INTANGIBLEBENEFITS

5. ROI

DEVELOP/REVIEWOBJECTIVES

OF SOLUTION

COLLECT DATAAFTER SOLUTION IMPLEMENTATION

3. APPLICATION/IMPLEMENTATION

4. BUSINESS IMPACT

1. SATISFACTION/REACTION

2. LEARNING

EVALUATION FRAMEWORKLevel Measurement Focus

1. Reaction, Satisfaction Measures participant satisfaction with the& Planned Action program/project and captures planned action.

2. Learning Measures changes in knowledge, skills,and attitudes.

3. Application Measures implementation and changes in behavior in the performance setting.

4. Business Impact Measures changes in business impactvariables.

5. Return on Investment Compares benefits to the costs.

HR SolutionsTraining Solutions/E-LearningMajor ProjectsTechnology SolutionsOrganization Development SolutionsChange Initiatives

THE ROI PROCESSTHE ROI PROCESS™™

CONVERT DATA TO

MONETARYVALUE

Calculating the Return on Investment of Performance Solutions

ROI = = 0.87 x 100 = 87%

ROI Calculation - Example

Net Program/Project BenefitsROI =

Program/Project Costs

$430,000 - $230,000

$230,000

Costs of program/project $230,000

Benefits of program/project (1st year) $430,000

12

A comprehensive m easurem ent and evaluation process that generates six types of m easures:

Reaction, Satisfaction, and Planned Action

Learning

Application and Im plementation

Business Impact

Return on Investment

Intangible Measures

This balanced approach to measurement includes a technique to isolate the effect of the program or solution.

THE ROI PROCESSTHE ROI PROCESSGuiding PrinciplesGuiding Principles

Results-based SolutionsPerformance solutions/projects are initiated, developed and delivered with the end in mind.

Participants understand their responsibility to obtain results with programs/solutions.

Support groups (management, supervisors, co-workers, etc.) help to achieve results from performance solutions.

A comprehensive measurement and evaluation system is in place for each program/project.

Variety of approaches are utilized to measure contribution, representing a balanced viewpoint.

Follow-up evaluations (Application, Impact and ROI) are developed for targeted solutions/projects and results are reported to a variety of stakeholders.

1. W hen a higher level evaluation is conducted, data must be collected at lower levels.

2. W hen an evaluation is planned for a higher level, the previous level of evaluation does not have to be comprehensive.

3. W hen collecting and analyzing data, use only the most credible sources.

4. W hen analyzing data, select the most conservative alternative for calculations.

5. At least one method must be used to isolate the effects of the solution/project.

6. If no improvement data are available for a population or from a specific source, it is assumed that little or no improvement has occurred.

7. Estimates of improvements should be adjusted for the potential error of the estimate.

8. Extreme data items and unsupported claims should not be used in ROI calculations.

9. Only the first year of benefits (annual) should be used in the ROI analysis of short term solutions.

10. Costs of a solution, project, or program should be fully loaded for ROI analysis

11. Intangible measures are defined as measures that are purposely not converted to monetary values.

12. The results from the ROI methodology must be communicated to all key stakeholders.

13

Collecting Follow-UP Data

Type of Data Method Level 3 Level 4

Surveys/Questionnaires Observation on the Job Interviews Follow-Up Focus Group Action Planning Performance Contracting Performance Monitoring

Isolating the Effects of the Program Best Practice

Method1 Use2

1. Comparison Group Analysis 35% 2. Trend/Forecasting Analysis 20% 3. Expert Estimation50% 4. Other 20%

1 Listed in order of credibility 2 Percentages exceed 100%

Converting Data to Money

Credibility Resources Needed

Standard values Records / Reports analysis Databases Expert estimation

High High

Moderate Low

Low High

Moderate Low

Tabulating Program Costs

Direct • Program Materials • Facilitator Costs • Facilities • Travel

Indirect

• Needs Assessment • Program Development • Participant Time • Administrative Overhead • Evaluation

Intangible Benefits • Complaints • Conflicts • Teamwork • Customer service

• Commitment • Stress • Job satisfaction • Engagement

14

Defining the Return on Investment

The ROI is calculated the same as the financial ROI for a building.

Benefits/Cost Ratio =Monetary Benefits

Program Costs

ROI (%) = Net Monetary Benefits Program Costs X 100

Example Costs for project (25 participants) $80,000 Benefits from program (1st year) $240,000 BCR = ROI = x 100 = ______%

ROI Target Options 1. Set the value at the same level as other investments – 15% 2. Set slightly above other investments – 25% 3. Set at break even – 0% 4. Set at client expectations

Private sector organizations usually go with option #2; public sector organizations prefer #3.

ROI Best Practices

1. The ROI methodology is implemented as a process improvement tool and not a performance

evaluation tool for the staff. 2. ROI impact studies are conducted very selectively, usually involving 5-10% of projects and

programs. 3. A variety of data collection methods are used in ROI analysis. 4. For a specific ROI evaluation, the effects of the program are isolated from other influences. 5. Business impact data are converted to monetary values. 6. ROI evaluation targets are developed, showing the percent of programs evaluated at each level. 7. The ROI methodology generates a micro level scorecard. 8. ROI methodology data are being integrated to create a macro scorecard for the learning/

development function 9. The ROI methodology is being implemented for about 3-5% of the budget. 10. ROI forecasting is being implemented routinely. 11. The ROI methodology is used as a tool to strengthen/improve the programs and processes.

*Based on benchmarking with over 200 organizations using ROI routinely

Cost-Saving Approaches to ROI

• Plan for evaluation early in the process • Build evaluation into the process • Share the responsibilities for evaluation • Require participants to conduct major

steps • Use short-cut methods for major steps

• Use sampling to select the most appropriate programs for ROI analysis

• Use estimates in the collection and analysis of data

• Develop internal capability to implement the ROI process

• Streamline the reporting process • Utilize web-based software to reduce time

A great ROI? Depends on the target. ________________________________________

________________________________________

________________________________________

________________________________________

________________________________________

15

Evaluation Targets

Criteria for Selecting Programs – Level 4 & 5 Evaluation

What happens if we maintain a status quo?

ROI Reality

• Impact/ROI information is desired by clients/ executives • The impact/ROI process provides a balanced, credible approach with six types of data • All types of organizations are routinely using impact/ROI • The impact/ROI process can be implemented without draining resources • The impact/ROI process is a long-term goal for many organizations.

Impact/ROI Standards*

1. When a higher-level evaluation is conducted, data must be collected at lower levels. 2. When an evaluation is planned for a higher level, the previous level of evaluation does not have to be

comprehensive. 3. When collecting and analyzing data, use only the most credible source. 4. When analyzing data, choose the most conservative among the alternatives. 5. At least one method must be used to isolate the effects of the solution. 6. If no improvement data are available for a population or from a specific source, it is assumed that little

or no improvement has occurred. 7. Estimates of improvements should be adjusted for the potential error of the estimate. 8. Extreme data items and unsupported claims should not be used in ROI calculations. 9. Only the first year of monetary benefits should be used in the ROI analysis for short-term solutions. 10. Costs of the solution should be fully loaded for ROI analysis. 11. Intangible measures are defined as measures that are purposely not converted to monetary values. 12. The results from the ROI methodology must be communicated to all key stakeholders.

* Developed by the ROI Institute with input from users. The standards are used by more than 2,000 organizations in 41

countries.

ROI

Impact

Application

Learning

Reaction and Satisfaction

Level 5: 5-10%

Level 1: 90-100%

Level 2: 40-60%

Level 3: 30%

Level 4: 10-20%

• Expected life cycle of the program • The importance of the program in meeting organization’s goals • The cost of the program • Visibility of the program • The size of the target audience • Extent of management interest

• Budget? • Influence? • Support? • Other Issues?

16

Southeast Corridor Bank

Program Title: Managing Retention Target Group: Bank Tellers Solution: Skills Based Pay System to Reduce Employee Turnover

RESULTS

Level 1: Reaction

Level 2: Learning

Level 3: Application

Level 4: Impact

Level 5: ROI

Intangible Benefits

Composite Rating: 4.2 out of 5

Positive self assessment on program understanding

Positive self assessment on each course with few exceptions (only two failed to be promoted because of performance in training)

95% participation rate

86 requests for training compared to 46 the year before

138 review situations

257 promotions compared to 139 the year before

Turnover reduced from 71% to 35%

Staffing level reduced by

4%

BCR: 3.58

ROI: 258%

Customer Satisfaction

Job Satisfaction

Product Sales

Cross Selling

Technique to Isolate Effects of Program: Estimates from branch managers and branch staff, adjusted for error Technique to Convert Data to Monetary Value: External studies at similar institutions and standard values (for staffing) Fully-loaded Program Costs: $857,196 First year; $433,200 Second year

17

Sprint/Nextel

Program Title: Diversity Target Group: Managers and Employees Solution: All-Inclusive Workforce Program (AIW)

RESULTS

Level 1: Reaction

Level 2: Learning

Level 3: Application

Level 4: Impact

Level 5: ROI

Intangible Benefits

Composite Rating: 4.39 out of 5 (for six items)

Averaged 4.28 out of 5 (for learning on six objectives)

Managers: Supports AIW (87%) Addresses Problems (81%) Encourages Staff (78%)

Employees:

Supports AIW (65%) Identifies Differences (63%) Encourages Staff (60%)

91% of Managers successful completed action plans

Attrition Rate Improvement

= 9.77%

BCR: 2.6

ROI: 163%

Employee Satisfaction

Communication

Cooperation

Diversity Mix

Teamwork

Technique to Isolate Effects of Program: Manager’s estimate, adjusted for error Technique to Convert Data to Monetary Value: Standard cost item ($89,000 per Turnover) Fully-loaded Program Costs: $1,216,836

18

Healthcare, Inc.

Program Title: Preventing Sexual Harassment Target Group: Primary First and Second Level Managers (655) Secondary: All employees through group meetings (6,844) Solution: Workshops and meetings on company policy and sexual harassment regulations

RESULTS

Level 1: Reaction

Level 2: Learning

Level 3: Application

Level 4: Impact

Level 5: ROI

Intangible Benefits

Overall rating 4.11 out of 5

93% provided action items

65% increase post-test versus pre-test

Skill practice demonstration

96% conducted meetings and completed meeting record

4.1 out of 5 on behavior change survey

68% report all action items complete

92% report some action items complete

Turnover reduction:

$2,840,632

Complaint reduction: $360,276

Total improvement: $3,200,908

1,051% Job Satisfaction

Absenteeism

Stress Reduction

Better Recruiting

Technique to Isolate Effects of Program: Trendline analysis; participant estimation Technique to Convert Data to Monetary Value: Historical costs; internal experts Fully-loaded Program Costs: $277,987

19

Sample of Published ROI Studies

Case Study Name Measuring the ROI: Key Impact Measures: ROI

Cracker Box Performance Management (Restaurant Chain)

A variety of measures, such as productivity, quality, time, costs, turnover, and absenteeism

298%1

Federal Information Agency

Internal Graduate Degree Program (Federal Agency)

Retention, individual graduate projects

153%4

Healthcare, Inc. Sexual Harassment Prevention (Health Care Chain)

Complaints, turnover, absenteeism, job satisfaction

1052%2

Imperial National Bank Executive Leadership Development (Financial)

Team projects, individual projects, retention

62%2

International Car Rental First Level Leadership Development (Auto Rental Company)

Various measures – at least two per manager

105%7

MetroTransit Absenteeism Control/Reduction Program (Major City)

Absenteeism, customer satisfaction 882%2

Midwest Electric Stress Management Program (Electric Utility)

Medical costs, turnover, absenteeism 320%2

National Crushed Stone Skill-Based Pay (Construction Materials Firm)

Labor costs, turnover, absenteeism 805%2

National Steel Safety Incentive Plan (Steel Company)

Accident frequency rate, accident severity rates

379%2

Nations Hotel Executive Coaching (Hotel Chain)

Cost reduction, sales growth, operating efficiency, retention, and customer satisfaction

221%5

Nextel Communications Diversity (Communications Company)

Retention, employee satisfaction 163%6

Southeast Corridor Bank

Retention Improvement (Financial Services)

Turnover, staffing levels, employee satisfaction

258%3

United Petroleum E-Learning (Petroleum) Sales 206%2

VA Sunshine Healthcare Network

Competency Development (Veteran’s Health Administration)

Time savings, work quality, faster response

159%4

20

References for Published Studies

1. In Action: Measuring Return on Investment, Volume 3. Patricia P. Phillips, Editor; Jack J. Phillips, Series Editor. American Society for Training and Development, Alexandria, VA, 2001.

2. The Human Resources Scorecard: Measuring the Return on Investment. Jack Phillips, Ron D. Stone, Patricia P. Phillips. Butterworth-Heinemann, Woburn, MA, 2001.

3. In Action: Retaining Your Best Employees. Patricia P. Phillips, Editor; Jack J. Phillips, Series Editor. American Society for Training and Development and the Society for Human Resource Management, Alexandria, VA, 2002.

4. In Action: Measuring ROI in the Public Sector. Patricia P. Phillips, Editor. American Society for Training and Development, Alexandria, VA, 2002.

5. ROI at Work: Best Practice Case Studies from the Real World. Jack J. Phillips and Patti P. Phillips, Editors. American Society for Training and Development, Alexandria, VA, 2005.

6. In Action: Implementing Training Scorecards. Lynn Schmidt, Editor; Jack J. Phillips, Series Editor. American Society for Training and Development, Alexandria, VA, 2003.

7. The Leadership Scorecard, Jack J. Phillips and Lynn Schmidt, Butterworth-Heinemann, Woburn, MA, 2004.

Additional Reading

Show Me the Money. Jack J. Phillips and Patricia P. Phillips. Berrett-Koehler, San Francisco, CA, 2007

Return on Investment Basics. Patricia P. Phillips and Jack J. Phillips. American Society for Training and Development, Alexandria, VA, 2005

Investing in Your Company’s Human Capital: Strategies to avoid too much or too little. Jack J. Phillips, New York, NY: Amacom 2005.

Proving the Value of HR: Why and How to Measure ROI. Jack J. Phillips and Patricia P. Phillips. Society for Human Resource Management, Alexandria, VA, 2005.

Return on Investment in Training and Performance Improvement Programs,2nd Edition, Jack J. Phillips. Butterworth-Heinemann, Woburn, MA, 2003.

Project Management Scorecard. Jack J. Phillips, Tim Bothell, G. Lynne Snead. Butterworth-Heinemann, Woburn, MA, 2002

The Bottomline On ROI. Patricia P. Phillips, Center for Effective Performance, 2002. How to Measure Training Results, A Practical Guide to Tracking the Six Key Indicators. Jack J.

Phillips, Ron D. Stone. McGraw-Hill Publishing, New York, NY, 2002. The Consultant’s Scorecard. Jack J. Phillips, McGraw-Hill Publishing, New York, NY, 2000. HRD Trends Worldwide: Shared Solutions to Compete in a Global Economy. Jack J. Phillips.

Butterworth-Heinemann, Woburn, MA, 1999. A New Vision for Human Resources. Jac Fitz-enz and Jack J. Phillips. Crisp Publications, San

Francisco, CA, 1998. Accountability in Human Resource Management. Jack J. Phillips. Butterworth-Heinemann, Woburn,

MA, 1996. In Action: Measuring Intellectual Capital. Patricia P. Phillips, Editor, Jack J. Phillips, Series Editor.

American Society for Training and Development, Alexandria, VA, 2002. In Action: Performance Analysis and Consulting. Jack J. Phillips, Editor & Series Editor, American

Society for Training and Development, Alexandria, VA, 1999. In Action: Coaching for Extraordinary Results. Darelyn J. Mitch, Editor; Jack J. Phillips, Series Editor.

American Society for Training and Development, Alexandria, VA, 2002.

21

Origin/Development • The ROI Methodology™ was developed by Dr. Jack J. Phillips in the 1970s, refined through application and use in

the 1980s, and implemented globally during the 1990s. • First impact study – 1973, Measuring the ROI in a Cooperative Education Program, for Lockheed-Martin • First public presentation on the methodology – 1978, ASTD Annual Conference • First book published to include methodology – 1983, Handbook of Training Evaluation and Measurement Methods,

Gulf Publishing (this was the first USA book on training evaluation) • First one-day public workshop –1991, Birmingham, Alabama • First two-day public workshop –1992, Johannesburg, South Africa • First case study book published – 1994, Measuring Return on Investment, ASTD • First international partnership established – 1994, Indonesia • First public certification workshop – 1995, Nashville, Tennessee • ROI Network organized - 1996 • First ROI Network Conference –1997, New Orleans, Louisiana • First international ROI Network conference – 2002, Toronto, Canada • First ROI in Government Conference – 2003, Gulfport, Mississippi, Co-sponsored by the University of Southern

Mississippi • First ROI software release – 2003, KnowledgeAdvisors • Online ROI certification launched – 2006, University Alliance • ROI Certification offered as part of Masters and Ph.D. degree – Capella University, 2006. Use • More than 3,000 organizations are using the ROI Methodology, through planned implementation. • 2,000 organizations have formally implemented the methodology through ROI Certification™ conducted by the ROI Institute. • Approximately 5,000 impact studies are conducted annually in learning and development and human resources. • At least 200 public sector governmental units are using the methodology. • ROI implementation was first pursued in manufacturing, then moved to service, healthcare, non-profits,

governments, and is now in educational systems Applications Typical applications include: • Apprenticeship • Career Management • Competency Systems • Diversity • E-Learning • Coaching • Information Assurance

• Gainsharing • Meetings and Events • Management Development • Leadership Development • Organization Development • Orientation • Recruiting

• Safety & Health Programs • Self-Directed Teams • Skill-Based/Knowledge-Based

Compensation • Technology Implementation • Total Quality Management • Wellness/Fitness Initiatives

Articles and Publicity • More than 60 articles have been published on the ROI Methodology in major publications in 20 countries. • The ROI Methodology has been a cover story on at least 15 publications, magazines, and journals. • At least 50 interviews in major global business and professional publications • More than 25 radio and TV interviews in different countries © 2006, ROI Institute, Inc.

The ROI Fact Sheet

Phone: 205-678-8101 ∗ Fax: 205-678-8102

Email: info@roiinstitute.net

22

Books • Sixteen books have been published on the ROI Methodology and its application (www.roiinstitute.net) • Primary reference – Return on Investment in Training and Performance Improvement Projects, 2nd Edition, Jack J.

Phillips, Butterworth-Heinemann, Woburn, MA, 2003 (originally published in 1997) • Award winning book – Bottomline on ROI, Patti P. Phillips, CEP Press, Atlanta, GA, 2002 (received ISPI award) Case Studies • More than 100 case studies published in books, journals, and industry publications • Four-volume set published by ASTD in 1994, 1997, 2001, and 2005 • First public sector case book – 2002, published jointly by the International Personnel and Management Association

and the American Society for Training and Development • First International casebook – 2005, Ireland published by Skillnets • International case studies under development in 12 countries Workshops (One-Day, Two-Day, and Three- Day) • Approximately 200 one-day workshops conducted with more than 8,000 participants • Approximately 500 two-day workshops conducted with more than 15,000 specialists and managers attending

(offered in almost every major international city) • Routine schedules of one-day, two-day, and three-day workshops offered in the USA by ASTD (www.astd.org) and

through partners around the world ROI Certification™ • Five-day workshop plus two work products lead to certification for ROI implementation • More than 3,000 professionals have attended certification, representing more than 2,000 organizations in at least 50

countries • Certifications offered routinely about 25 times per year both, internally and publicly by the ROI Institute

(www.roiinstitute.net) • Online certification begins every month-six months duration (www.roiinstituteonline.com) Global Implementation • First implementation of the ROI Methodology outside the USA – 1992, South Africa • First certification in non-English language – 1995, Italy • Implementation is accomplished through partners in various countries • Implementation is currently occurring in 44 countries, with additional implementations planned in other countries • Books published in 28 languages • Twelve international casebooks in development or in the planning stages

The ROI Fact Sheet

Phone: 205-678-8101 ∗ Fax: 205-678-8102

Email: info@roiinstitute.net

23

Jack J. Phillips, Ph.D.

As a world-renowned expert on accountability, measurement and evaluation, Dr. Phillips provides consulting services for Fortune 500 companies and major global organizations. The author or editor of more than 50 books, Phillips provides workshops and conference presentations throughout the world. His expertise in measurement and evaluation is based on more than twenty-seven years of corporate experience in five industries (aerospace, textiles, metals, construction materials, and banking). Phillips has served as training and development manager at two Fortune 500 firms, senior HR officer at two firms, president of a regional bank, and management professor at a major state university. His background led Phillips to develop the ROI Methodology⎯a revolutionary process that provides bottom-line figures and accountability for all types of learning, performance improvement, human resources, technology, and public policy programs. Phillips regularly consults with clients in manufacturing, service, and government organizations in 44 countries in North and South America, Europe, Africa, Australia, and Asia

Books most recently authored by Phillips include Show Me the Money, Berrett-Koehler, 2007; Investing in Your Company’s Human Capital: Strategies to Avoid Spending Too Much or Too Little, Amacom 2005; Proving the Value of HR: How and Why to Measure ROI, SHRM 2005; The Leadership Scorecard, Elsevier Butterworth-Heinemann 2004; The Human Resources Scorecard: Measuring the Return on Investment, Elsevier Butterworth-Heinemann 2001; Building a Successful Consulting Practice, ASTD 2002; The Consultant’s Scorecard, McGraw-Hill 2000; Managing Employee Retention, Elsevier Butterworth-Heinemann, 2003; Return on Investment in Training and Performance Improvement Projects, 2nd Edition Elsevier Butterworth-Heinemann 2003; The Project Management Scorecard, Elsevier Butterworth-Heinemann 2002; How to Measure Training Results, McGraw-Hill 2002; and Performance Analysis and Consulting, ASTD 2000. Phillips served as series editor for ASTD’s In Action casebook series, one of ASTD’s more ambitious publishing projects with 30 titles. Now, he serves as series editor for Elsevier Butterworth-Heinemann’s Improving Human Performance series and Pfeiffer’s new series on Measurement and Evaluation. Phillips has received several awards for his books and his work. The Society for Human Resource Management gave him its highest creative award for an ROI study and an award for one of his books. The American Society for Training and Development gave him its highest award, Distinguished Contribution to Workplace Learning and Development. Meeting News named Phillips one of the 25 most influential people in the Meetings and Events industry, based on his work on ROI for the industry. Phillips has undergraduate degrees in electrical engineering, physics, and mathematics; a master’s degree in decision sciences from Georgia State University; and a Ph.D. in human resource management from the University of Alabama. Jack Phillips has served on several boards of private businesses – including two NASDAQ companies – and several non-profits and associations, including the American Society for Training and Development. He is Chairman, ROI Institute, Inc. and can be reached at (205) 678-8101, or by e-mail at jack@roiinstitute.net.

24

Patti P. Phillips, Ph.D.

Dr. Patti P. Phillips is president of the ROI Institute, Inc., the leading source of ROI competency building, implementation support, networking, and research. She is also chair and CEO of The Chelsea Group, Inc., an international consulting organization supporting organizations and their efforts to build accountability into their training, human resources, and performance improvement programs with a primary focus on building accountability in public sector organizations. She helps organizations implement the ROI methodology in countries around the world—including South Africa, Singapore, Japan, New Zealand, Australia, Italy, Turkey, France, Germany, Canada, and the United States.

After a thirteen-year career in the electrical utility industry, Dr. Phillips took advantage of the opportunity to pursue a career in a growing consulting business where she was introduced to training, human resources, and performance improvement from a new perspective—a perspective that directly reflected her values of accountability, ROI evaluation. Since 1997, she has embraced the ROI methodology by committing herself to ongoing research and practice. To this end Dr. Phillips has implemented ROI in private sector and public sector organizations. She has conducted ROI impact studies on programs such as leadership development, sales, new-hire orientation, human performance improvement, K-12 educator development, educators’ National Board Certification mentoring, and faculty fellowship. Dr. Phillips is currently expanding her interest in public sector accountability through application of the ROI methodology in community- and faith-based initiatives, including Citizen Corps, AmeriCorps, and the Compassion Capital Fund.

Dr. Phillips teaches others to implement the ROI methodology through the ROI certification process, as a facilitator for ASTD’s ROI and Measuring and Evaluating Learning Workshops, and as adjunct professor for graduate-level evaluation courses. She speaks on the topic of ROI at conferences such as ASTD’s International Conference and Exposition and ISPI’s International Conference.

Dr. Phillips’ academic accomplishments include a Ph.D. in International Development and a master’s degree in Public and Private Management. She is certified in ROI evaluation and has earned the designation of Certified Performance Technologist. She has authored a number of publications on the subject of accountability and ROI, including Show Me the Money (Berrett-Koehler, 2007); The Value of Learning (Pfeiffer, 2007); Return on Investment Basics (ASTD, 2005); Proving the Value of HR: How and Why to Measure ROI (SHRM, 2005); Make Training Evaluation Work (ASTD, 2004); The Bottom Line on ROI (Center for Effective Performance, 2002), which won the 2003 ISPI Award of Excellence; ROI at Work (ASTD, 2005); the ASTD In Action casebooks Measuring Return on Investment Vol. III (2001), Measuring ROI in the Public Sector (2002), and Retaining Your Best Employees (2002); the ASTD Infoline series, including Planning and Using Evaluation Data (2003), Mastering ROI (1998), and Managing Evaluation Shortcuts (2001); and The Human Resources Scorecard: Measuring Return on Investment (Butterworth-Heinemann, 2001). Dr. Phillips’s work is published in a variety of journals. Dr. Phillips can be reached at patti@roiinstitute.net.