Developing Metrics that Demonstrate the Value of Technical Communications

May 2011

Developing Metrics that Demonstrate the Value of Technical Communications

Vic Passion

© 2011 TechProse All rights reserved. 2

• What are metrics?

• Kirkpatrick’s levels of evaluation

• The value of the baseline

• Choosing your metrics

• Calculating ROI

• Wrap-up

What we’ll cover …

2


Measures to demonstrate the value of technical communications

3

What are metrics?

Let’s explore why metrics are so important…


Depends on:

– Amount and complexity of content

– Type and number of deliverables

– Frequency of updates

– Translation requirements

– Amount of customization

– Tools and techniques

– Format of source material

– Other factors

4

What do technical communications cost?






• Calculating ROI

• Wrap-up


5


Source: *Kirkpatrick, Donald L, Evaluating Training Programs: The Four Levels

Kirkpatrick’s levels of evaluation and objectives

4. Results

3.Transfer

2.Learning

1.Reactions

Performance-based objectives

Satisfaction

Job task performance

Enterprise, division, and departmental objectives


Who wants to know the results?

4. Results

3.Transfer

2.Learning

1.Reactions

Supervisors, line managers

Technical communications managers

Managers

Department managers, execs, CEO


A transit provider has unacceptable outages for its subway escalators and elevators. Management has designated documentation funds to help reduce time out of operation. What would you measure for at each level?

Business case

4. Results

3.Transfer

2.Learning

1.Reactions






• Calculating ROI

• Wrap-up


9


Support Calls

0

50

100

150

200

250

300

350

Q1 Q2 Q3 Q4

Quarter 2007

Nu

mb

er o

f su

pp

ort

cal

ls

Compelling metrics show the delta between the baseline and the post-documentation data

The value of the baseline

Documentation and website launched in Q3


• Build all deliverables around what you want end users to be able to do, integrated with corporate strategy

• Define what they can do before the release—the baseline

– Number of errors

– Revenue generated

– Orders processed

– Stock shipped

– Stock returned

– Etc.

The ABCs of performance-based metrics


• You get what you measure

• Easy to capture…

– Performance after release (but what was it before?)

– Performance of groups with most access to documentation vs. groups with least access (but what about complexity of tasks?)

– Improvement/degradation over time (but what about upgrades?)

• Characteristics of good metrics

– A baseline

– Near-term benefits

– Meaningful, tangible

– Careful analysis

Choosing the right metrics






• Calculating ROI

• Wrap-up


13


• Break into groups.

• Read your case study and decide what you would measure to build a compelling case.

• Choose a group leader to present your measures.

Building a compelling case


Business cases

#1: After three severe injuries to machine operators, a national paper products company requires all machine

operators and managers to receive training on an updated user’s guide.

#2: A technology company delivers training to its employees and partners on a new enterprise software system for quoting and ordering.

#3: After a tax season in which the software company is unable to keep up with support calls, the company launches an online knowledge base.

#4: After getting feedback from clients and suppliers that translated materials and updates to documentation are taking too long, an international online retailer implements single sourcing.

10 mins


Wrap-Up






• Calculating ROI

• Wrap-up


17


Calculating ROI

Example: Reduce transit technician repair time by 30 minutes per repair

30 min x $100/hour = $50

$50 x 1,500 repairs per year = $75,000

Cost of documentation = $50,000

(Benefit – Cost)/Cost = ROI

($75,000-$50,000)= $25,000 savings

$25,000/$50,000 investment = 50% ROI


– Use conservative assumptions

– Focus on only one or two compelling metrics

– Keep it simple

– Document your assumptions

Building a case for a technical communications program


• Break into the same groups.

• Using the same case study, calculate the ROI.

• Document your assumptions.

• Choose a group leader to present your ROI.

Calculating ROI


Business cases

#1: After three severe injuries to machine operators, a national paper products company requires all machine

operators and managers to receive training on an updated user’s guide.

#2: A technology company delivers training to its employees and partners on a new enterprise software system for quoting and ordering.

#3: After a tax season in which the software company is unable to keep up with support calls, the company launches an online knowledge base.

#4: After getting feedback from clients and suppliers that translated materials and updates to documentation are taking too long, an international online retailer implements single sourcing.

15 mins


Wrap-Up


• Don’t take your budget for granted

• Use higher levels of evaluation to speak to execs in your organization

• Capture the baseline before measuring

• Capture meaningful, tangible metrics with near-term benefits

• Estimate proposed ROI by:

– Documenting your assumptions clearly

– Using conservative assumptions

– Focusing on only one or two compelling metrics

– Keeping it simple

• (Benefit – Cost)/Cost = ROI

Key points to take home


Vic PassionTechProse

Walnut Creek, CA, USA925.956.4231

[email protected]

Your turn!


Helping clients navigate change

www.techprose.com


Backup


Business case #1

After three severe injuries to machine operators, a national paper products company requires all machine operators and managers to receive training on an updated users guide.Goals:

– Fewer lawsuits against the company this year

– Lower employee absenteeism

– Lower insurance costs


Business case #2

A technology company delivers training to its employees and partners on a new enterprise software system for quoting and ordering. Goals:

– More efficient use of employee and partner time

– Better retention of partners

– Ability to see customers with expiring contracts


Business case #3

After a tax season in which the software company is unable to keep up with support calls, the company launches an online knowledge base.

Goals:

– Faster response time

– Better retention of end users

– Lower technical support costs


Business case #4

After hearing from clients and suppliers that translated materials and updates to documentation take too long, an international online retailer implements single sourcing. Goals:

– Shorter release cycle

– Better retention of end users

– Lower technical support costs