Introduction to ROI Methodology for Meeting Professionals Elling Hamso European Event ROI Institute...

Introduction to ROI Methodologyfor Meeting Professionals

Elling HamsoEuropean Event ROI Institute

©MPI Foundation/Fusion Productions 2005

Understand the basic ROI methodology

Understand the different evaluation levels

Isolate the effects of the meeting

Convert the effects of the meeting into $ values

Today’s Meeting Objectives

What is ROI?

Today’s Meeting Objectives

What is ROI?What is ROI?

% Return on Investment

X 100 50 % ROI=

What is ROI?What is ROI?

Sales Price € 150 000

Purchase Price € 100 000

= Profit € 50 000

€ 100 000

€ 50 000

Why do you need to know about ROI?

Today’s Meeting Objectives

Why ROI?

Sales Argument

Defend or get new budgetsSelling events through ROITalking the language of customerSponsor ROI

Competitive Advantage

Design events to improve ROIMore targeted events

Compare Improve

Agency offer ROI services to clients

To other eventsTo other marketing toolsBenchmarking

Show CompetenceTransparencyAccountabilityFinancial

“I need my people to think

like business people and

focus on the ‘business of

meetings’ – not the meeting

business.”

Chief Executive

Officer

Benefits of Meeting & Event ROI

MPI Pathways to Excellence

E

I

O

ducation

nfluence

pportunity

Provide education and recognition to the meeting planning profession

Make senior managers understand the value of meetings

Provide business opportunities for members

♫♫♫ Old MacDonald had a farm, ei, eio...

Can we measure everything in monetary values?

Questions about ROIQuestions about ROI

Is it difficult? Is it expensive?

Questions about ROIQuestions about ROI

Should we measure ROI for all meetings?

Questions about ROIQuestions about ROI

What is ’acceptable’ ROI?

Questions about ROIQuestions about ROI

Is it a well proven method?

Questions about ROIQuestions about ROI

• Refined over a 20-year period

• Thousands of ROI studies conducted every year

• 100 case studies published

• 2,500 individuals certified

• 30 books developed to support the model

• Adopted by hundreds of organizations in more than 40 countries

The Phillips ROI ModelThe Phillips ROI Model

Some Users of the Phillips ROI ModelSome Users of the Phillips ROI Model

AT&T Coca Cola General Motors Hewlett Packard Bristol-Myers Squibb Motorola PricewaterhouseCoopers Shell Oil Singapore Airlines

US Department of Defense NASA Government of Italy Government Canada

The Phillips ROI Model

Develop Evaluation

Plans

Develop Evaluation

Plans

EvaluationPlanning

Data Collection

Collect Data Before and

During Meeting

Collect Data Before and

During Meeting

Collect DataAfter MeetingCollect Data

After MeetingDevelop Meeting

Objectives

Develop Meeting

Objectives

© Copyright 2004 ROI Institute

1 2 3 4

CalculateCosts

of Meeting

CalculateCosts

of Meeting

Isolate theEffects of

the Meeting

Isolate theEffects of

the Meeting

Convert Data to Monetary

Value

Convert Data to Monetary

Value

Calculatethe Return On

Investment

Calculatethe Return On

Investment

CommunicateCommunicateresultsresults

CommunicateCommunicateresultsresults

Data Analysis

Identify IntangibleBenefits

Identify IntangibleBenefits

Reporting

5 6 8

7

9

10

© Copyright 2004 ROI Institute

Evaluation PlanningEvaluation Planning

Stakeholders’ needs

A stakeholder will say:

”It is important to me that this meeting is sucessful and meets its objectives”

Evaluation PlanningEvaluation Planning5 levels of measurement

Evaluation PlanningEvaluation Planning

The Chain of Impact

© Copyright 2004 ROI Institute

1. Participants react favorably to the meeting

2. Participants learn new skills or information

3. Participants apply new skills or information

4. A business impact is achieved

5. A return on investment is generated

Evaluation PlanningEvaluation Planning

Assigning objectives to levels of measurement

1. Satisfaction, reaction, planned action

2. Learning

3. Application

4. Impact

5. ROI

Objective:90% of attendees will indicate an intention to implement new sales strategies within two months after the meeting

Answer:Level 1 - Satisfaction, reaction, planned action(intentions are always level 1)

Evaluation PlanningEvaluation Planning

Assigning objectives to levels of measurement

1. Satisfaction, reaction, planned action

2. Learning

3. Application

4. Impact

5. ROI

Objective:Decrease the amount of time required to complete projects by 5% within 6 months of the meeting

Answer:Level 4 - Impact(time may be converted to money)

Evaluation PlanningEvaluation Planning

Assigning objectives to levels of measurement

1. Satisfaction, reaction, planned action

2. Learning

3. Application

4. Impact

5. ROI

Objective:Participants rate the relevance of the meeting to success in their jobs as 4.5 out of 5

Answer:Level 1 - Satisfaction, reaction, planned action

Evaluation PlanningEvaluation Planning

Assigning objectives to levels of measurement

1. Satisfaction, reaction, planned action

2. Learning

3. Application

4. Impact

5. ROI

Objective:Use the new customer interaction skills in 90% of situations where they are needed.

Answer:Level 3 - Application

Evaluation PlanningEvaluation Planning

Assigning objectives to levels of measurement

1. Satisfaction, reaction, planned action

2. Learning

3. Application

4. Impact

5. ROI

Objective:Score 75 out of 100 or better on the new strategy quiz

Answer:Level 2 - Learning

Evaluation PlanningEvaluation Planning

Assigning objectives to levels of measurement

1. Satisfaction, reaction, planned action

2. Learning

3. Application

4. Impact

5. ROI

Objective:Rate the meeting as a good investment for the company with an average of 4.3 out of 5.

Answer: Level 1 - Satisfaction, reaction, planned action

Evaluation PlanningEvaluation Planning

Assigning objectives to levels of measurement

1. Satisfaction, reaction, planned action

2. Learning

3. Application

4. Impact

5. ROI

Objective:Increase sales from existing customers by 5% within 9 months of the meeting

Answer:Level 4 - Impact

Evaluation PlanningEvaluation Planning

Assigning objectives to levels of measurement

1. Satisfaction, reaction, planned action

2. Learning

3. Application

4. Impact

5. ROI

Objective:Achieve a 25% return on investment within the 12 months of the meeting

Answer:Level 5 - ROI

Evaluation PlanningEvaluation Planning

Assigning objectives to levels of measurement

1. Satisfaction, reaction, planned action

2. Learning

3. Application

4. Impact

5. ROI

Objective:Successfully demonstrate new selling techniques during classroom role play

Answer:Level 2 - Learning

Surveys (“smile sheets”)

The Tools

Level 1 Reaction, Satisfaction, Planned Action

Data CollectionData Collection

© Copyright 2004 ROI Institute

• 80% of attendees would recommend the conference to others.• Receive a speaker rating from participants of at least 4.5 out of 5.0• 50% of attendees will indicate an intention to submit an RFP to one or more trade show vendors within 3 months of the meeting.

Tests, exams, audience response systems

Level 2 Learning

Data CollectionData Collection

© Copyright 2004 ROI Institute

• Name two major sponsors of the event• 50% of booth attendees will list three product features

The Tools

Follow up surveys, focus groups, observation

The Tools

Level 3 Application / Implementation

Data CollectionData Collection

© Copyright 2004 ROI Institute

• Conduct a meeting with committee members to implement new association policy within one month of chapter leadership conference• 50% of attendees will make an inquiry for the software within 6 months of product launch event

Follow up questionnaires, performance monitoring, scorecards

The Tools

Level 4 Business Impact

Data CollectionData Collection

© Copyright 2004 ROI Institute

• Decrease customer complaints by 30% within 9 months• Increase membership satisfaction level by 10% in 6 months• Generate 1 million in signed contracts within 9 months of trade show

Net Meeting Benefit

Total Cost

The Tools

Level 5 Return on Investment

Data CollectionData Collection

x 100 = ROI

© Copyright 2004 ROI Institute

• Achieve a 5% return on investment using first year benefits.

• Control groups

• Forecasting methods

• Trend analysis

• Participants’ estimates

• Supervisors’ estimates

• Outside experts

The Solution:The Problem:

OtherInfluences

MeetingImpact

Tota

l m

easu

red

Im

pact

Data AnalysisData Analysis

Isolating the effects of the meeting

© Copyright 2004 ROI Institute

Data AnalysisData Analysis

Using control groups to isolate the effects of the meeting

© Copyright 2004 ROI Institute

Retail Merchandise Company – Interactive Selling Skills ProgrammeTwo groups of 3 stores matched on:•Store size•Store location (median houshold income in the area)•Customer traffic levels•Previous store perfomance

Weekly sales 3 months after training ($)

Trained group 12,075Control group 10,449Increase due to training 1,626

(ROI was in this case calculated to be 118%)

Data AnalysisData Analysis

Using participants’ estimates to isolate the effects of the meeting

Elements influencing sales increase:

Influence factor

Sales kickoff meeting

Advertizing campaign

New incentive programme

Product improvements

Other factors

Total

20%

15%

30%

25%

10%

100%

Participants’ estimates of factors influencing sales increase following a sales kick-off meeting

Confidence of estimate

Adjusted influence factor

50% (i.e. 10 – 30%) 10%

• Profit from sales

• Converting employee’s time

• Historical costs

• Internal and external experts

• Participants’ estimates

• Supervisors’ and managers’ estimates

• Staff estimates

The Solution:The Problem:

Data AnalysisData Analysis

Converting data to monetary value

© Copyright 2004 ROI Institute

Data AnalysisData AnalysisConverting data to monetary value

using participants’ estimates

Basis for Value EstimatedValue

Confi-dence (%)

Adjusted Value ($)

Improvement in efficiency of group. $1,500 month x 12 (group estimate)

$18,000 85% $15,300

Turnover reduction: Two employees retained per year. Base salary x 1.5

$36,000 90% $32,400

15% improvement in customer response time. Estimated value $1.100/month

$13,200 60% $7,920

Absenteeism reduction (50 absences per year x $150 standard value)

$7,500 85% $6,375

Examples of business impact converted to money after leadership training seminar

With permission from Jack Phillips, ’Measuring Return on Investment’ Vol. 2. ASTD Press, 1997

Data AnalysisData Analysis

Assigning costs

• Marketing Costs• Developments Costs• Program Materials• Facilitator/Speaker Costs• Facilities Costs• Travel/Hotel/Meals• Participants Salaries & Benefits• Administrative/Overhead Costs• Evaluation Costs• Taxes

© Copyright 2004 ROI Institute

€ Net Meeting Benefit

€ Meeting Costsx 100 = ROI

Benefits of meeting (1st year) €750,000Costs of meeting €500,000

€ 250,000

€ 500,000x 100 = 50%

Data AnalysisData Analysis

Calculating ROI

• Improved Image

• Improved Customer Service

• Reduced Complaints

• Reduced Conflicts

• Reduced Stress

Data AnalysisData Analysis

Potential intangible benefits

© Copyright 2004 ROI Institute

The Phillips ROI Model

Develop Evaluation

Plans

Develop Evaluation

Plans

EvaluationPlanning

Data Collection

Collect Data Before and

During Meeting

Collect Data Before and

During Meeting

Collect DataAfter MeetingCollect Data

After MeetingDevelop Meeting

Objectives

Develop Meeting

Objectives

Level 1 – Reaction & SatisfactionLevel 2 – Learning

Level 3 – ApplicationLevel 4 – Impact

ReportingReporting

1 2 3 4

© Copyright 2004 ROI Institute

CalculateCosts

of Meeting

CalculateCosts

of Meeting

Isolate theEffects of

the Meeting

Isolate theEffects of

the Meeting

Convert Data to Monetary

Value

Convert Data to Monetary

Value

Calculatethe Return On

Investment

Calculatethe Return On

Investment

CommunicateCommunicateresultsresults

CommunicateCommunicateresultsresults

Data Analysis

Identify IntangibleBenefits

Identify IntangibleBenefits

Reporting

Level 5 - ROI

5 6 810

7

9

© Copyright 2004 ROI Institute

www.EventROI.com

www.ROIInstitute.net

www.mpiweb.org

Want to know more?Want to know more?

The Bottom Line on ROI. Patricia P. Phillips. Atlanta: CEP Press, 2002.

Return in Investment in Training and Performance Improvement Programs (2nd Edition). Jack J. Phillips. Boston: Butterworth-Heinemann, 2003.

Measuring Return on Investment, Volume 3. In In Action series, Patricia P. Phillips (editor) Jack J. Phillips (series editor). Alexandria: ASTD, 2001.

*Available from www.amazon.com

Books on ROIBooks on ROI

14 – 15 December 2005, Copenhagen (in English)

20 – 21 December 2005, London (in English)

11 – 12 January 2006, Frankfurt (in German)

18 – 19 January 2006, Stockholm (in Swedish)

8 – 9 February 2006, Amsterdam (in English)

More information and registration on www.EventROI.com

2-day ROI Training Seminars2-day ROI Training Seminars

Thank you!

European Event ROI Institute

Elling HamsoVagsgt. 43Postboks 171NO-4302 SandnesNorwayE-mail: Elling@EventROI.comTelephone: +47 90 12 24 18

Bernhard AggelerFlurstrasse 12CH-6332 Hagendorn ZGSwitzerlandE-mail: Bernhard@EventROI.com

Telephone: +41 79 443 63 49

www.EventROI.com