Using Talent Development Reporting Principles to Run Learning Like a Business

You can listen to today’s webinar using your computer’s speakers or you may dial into the teleconference.

If you would like to join the teleconference, please dial 1.408.600.3600 and enter access code 921 466 798 #

You will be on hold until the seminar begins.

#CLOwebinar

Using Talent Development Reporting Principles to Run Learning Like a

Business

Speaker: Dave VanceFormer PresidentCaterpillar University

Moderator: Deanna HartleySenior EditorChief Learning Officer magazine

#CLOwebinar


Business

• Q&A– Click on the Q&A icon on

your floating toolbar on the top of your screen.

– Type in your question in the space provided

– Click on “Send.”

Tools You Can Use

#CLOwebinar

• Polling

– The poll will appear on the right side of your screen

– Select the best option for each question

– Click on “Submit”

#CLOwebinar

Tools You Can Use

Frequently Asked QuestionsWill I receive a copy of the webinar recording?

YES

Will I receive a copy of the slides?

YES

Please allow up to 2 business days to receive these materials

#CLOwebinar

#CLOwebinar

Deanna HartleySenior EditorChief Learning Officer magazine


Business

#CLOwebinar

Dave VanceFormer PresidentCaterpillar University


Business

Chief Learning Officer Webinar

Sponsored by Skillsoft

September 13, 2012David Vance

The basics of running learning like a business

Talent Development Reporting Principles (TDRP)◦ What is it?◦ How can it help?

Q&A

September 13, 2012Chief Learning Officer Webinar 8

Help our organizations achieve their goals◦ Achieve their goals more quickly◦ Achieve their goals at lower cost

Best accomplished by running learning like a business focusing on◦ Outcomes (Are we doing the right things?)◦ Effectiveness (How well?)◦ Efficiency (How much and at what cost?)


1. Align learning to business goals2. Plan it carefully3. Execute and report with discipline4. Measure and evaluate


Proactive, strategic exercise◦ Meet with CEO or business unit leader◦ Learn about next year’s goals and priorities◦ Learn who the stakeholders areIf not done proactively, at least be sure the requested learning does support the highest-priority business goals◦ May mean saying “No” to some requests


A: Proactively to align learning to business goals

B: Align to business goals as requests for learning come in

C: No process to ensure alignment but most of our learning is aligned to business goals

D: No process and much of our learning is probably not aligned to business goals

E: We really don’t know what our business goals are


Meet with stakeholders to understand◦ Business need◦ Learning’s role (if any)Agree with stakeholder on◦ Specific programs, type of learning◦ Target audience, location, completion dates◦ Number of participants◦ Roles and responsibilities◦ Measures of success

Expected impact of learning on business goal◦ Costs


If done proactively◦ Develop a business case for key learning

All costsBenefits

◦ Create a business plan for learning includingStrategic alignmentExpected results from learningBusiness caseMeasurement and evaluation strategy

◦ Have plan approved by CEO and senior leaders


A: Written business plan for learning approved by the CEO, governing body or other senior leaders

B: Written business plan for learning but not approved by senior leaders

C: PowerPoint business plan for learning approved or shared with senior leaders

D: We have L&D goals but no formal plan. We respond to requests as they come in.


Manage your execution to deliver planned outcomes, effectiveness, and efficiencyReview progress monthly within L&D◦ Year-to-date results versus plan◦ Will you make plan?◦ If not, what is the forecast?◦ What action should you take?Review progress at least quarterly with CEO,senior leaders◦ Focusing on most important business goals, key measures


A: Report results to our CEO or top-level leader at least quarterly

B: Report results to our SVP of HR or other direct report to the top-level leader at least quarterly

C: Report semi-annually or annually

D: Don’t report results to others


Execute your measurement and evaluation strategy. For example,◦ Level 1 for all courses◦ Level 2 where appropriate◦ Level 3 for 10 key programs◦ Levels 4 and 5 for three key programsDid you achieve your goals?If not, what can you learn? What can you improve?If you exceeded goals, what can you learn?


Conceptually not difficultChallenging in practiceRequires new skills for some, confidence to apply them for othersSome common objections◦ No standards in place to provide guidance◦ No internal processes to support it◦ Not comfortable with business-like approachTDRp can help


Brings principles, standards, definitions and recommended statements and reports to talent developmentAnswers key questions:◦ What does the CLO need to run learning like a business?

What data should we collect?How are the measures defined?

◦ What do the CEO and senior leaders want to see and how should it be presented?


Began in fall 2010 by Kent Barnett (CEO, Knowledge Advisors) and Tamar Elkeles (VP of Learning and Organization Development, Qualcomm)

Engaged industry thought leaders like Fitz-enz, Bassi, Phillips, Brinkerhoff, Bersin and CLOs of major organizations

TDRp for L&D completed in March 2012. Now being extended to rest of HR.



Data Sources and Systems

Systems and processes for organizing data,

calculating measures

Specific methodology (e.g. Phillips, Brinkerhoff)

Statements

Standard Measures

Data

Reports

System Wide Analytics

Program/ Initiative Analytics

Standard Measures: • Outcome measures: What is the impact on the

business?• Efficiency measures: How much? How many? At what

cost? • Effectiveness measures: How well?






Statements

Standard Measures

Data

Reports



Three Statements:• Outcome Statement• Effectiveness

Statement• Efficiency Statement






Statements

Standard Measures

Data

Reports



Three Reports:• Summary Report• Program Report• Operations Report

25September 13, 2012Chief Learning Officer Webinar

• Three standard statements◦ Outcome◦ Effectiveness◦ Efficiency

• Standard measures are used but choice of measures depends on organization

• Summary statements show◦ Last year’s actual◦ Plan (or goal) for this year◦ Year-to-date results

• Detail statements show◦ Monthly, quarterly, trend data◦ Granularity◦ Without plan

Shows alignment of learning to goalsExpected impact of learning on those goals◦ Isolated impact of learning (level 4)

Participant’s forecastSponsor’s forecast, estimateStatistical estimate (after the training is completed)

◦ Application (level 3)◦ Proxies for impact

In some cases may be number of participants◦ Agreed upon with stakeholder upfront


September 13, 2012Chief Learning Officer Webinar

2011 For 2012 Priority Business Outcomes and Learning Impact Actual Plan Jun YTD % of Plan

1 Revenue: Increase Sales by 20% Corporate Goal or Actual % 10% 20% 17% 85% Application of Training: % Applying 3 key concepts % NA 80% 85% 106%

2 Leadership: Increase EOS (1) Leadership Score by 5 Points Corporate Goal or Actual Points 0 pts 5 pts 3 pts 60% Impact of Training: Time spent coaching direct reports Minutes/wk NA 60 50 83%

3 Safety: Reduce Injuries by 20% Corporate Goal or Actual % 10% 20% 15% 75% Impact of Training: 70% reduction in injuries due to L&D % 5% 14% 11% 79%

4 Call Center Satisfaction: Improve Satisfaction Score by 4 Points Corporate Goal or Actual Points 1.6 4.0 2.9 73% Impact of Training: High, Medium, Low High/Medium/Low NA High High

5 Comply with New Regulations Corporate Goal or Actual % in compliance 100% 100% 100% Impact of Training: Essential for compliance NA Essential Essential

6 Innovation: Generate 10 New Patentable Ideas Corporate Goal or Actual # 4 10 7 70% Impact of Training: None planned NA NA NA

Results through June

Kirkpatrick/Phillips five levels◦ Level 1: Satisfaction, Forecasts of levels 3-5

ParticipantSponsor

◦ Level 2: Amount learned◦ Level 3: Application rate◦ Level 4: Impact◦ Level 5: Value (net benefit or ROI)Some will have only L1, L2




• Volume◦ Participants◦ Courses, classes, hours- Offered, taken◦ By ILT, vILT, WBT, etc.

• Costs◦ Development, delivery,

management◦ Tuition◦ Opportunity◦ Cost reduction due to e-

learning

• Utilization◦ Courses, classes, instructors◦ By ILT, vILT

• Effort to develop and maintain• Reach• Cycle times

◦ Performance consulting, development, delivery

*See a complete list in the Definition of Terms and Measurements document



• Three levels of reports◦ Summary Report (for CEO)◦ Program Report (for CLO)◦ Operations Report (for CLO)

◦ Highly customized, pulling the most important measures from the statements

• Executive-level reports show◦ Last year’s actual◦ Plan (or goal) for this year◦ Year-to-date results◦ Forecast for this year

• Detailed reports for managers may show◦ Monthly, quarterly, trend data◦ Granularity◦ Without plan or forecast


2011Impact of Learning Actual Plan Jun YTD % Plan Forecast On Top Ten Business Goals (1) Sponsor Estimate of Impact On 5 point scale 2.3 pts 2.5 pts 2.4 pts 2.4 pts Application Rate % who applied it 65% 75% 71% 73% Total Participants Number 25,000 30,000 17,268 58% 32,000

On Top Three Business Goals Increase in Sales by 20% Corporate Goal or Actual % 2% 20% 17% 20% Application of Training (application rate) % NA 80% 85% 85% Total participants Number 386 1,000 826 83% 1,115

Increase Leadership Score by 5 points Corporate Goal or Actual points 0 pts 5 pts 3 pts 4 pts Impact of Training (time spent coaching) minutes/week NA 60 50 50 Total participants Number 0 400 168 42% 400

Reduce Injuries by 20% Corporate Goal or Actual % 10% 20% 9% 20% Impact of Training: 70% reduction in injuries due to L&D % 5% 14% 6% 43% 14% Total participants Number 4,216 11,000 5,826 53% 11,300

Total Participants in All Programs Number 40,121 50,000 22,013 44% 48,000 Unique Participants in All Programs Number 20,263 23,000 18,407 80% 24,000

Results Through June

For 2012


2011Actual Plan Jun YTD % Plan Forecast

Effectiveness Participant Feedback % favorable 80% 85% 87% 102% 85% Sponsor Feedback % favorable 75% 80% 77% 96% 78% Learning Score 78% 85% 83% 84% Application rate % who applied it 61% 75% 78% 79%

Efficiency % of courses developed on time % 82% 95% 88% 94% % of employees reached by L&D % 85% 88% 72% 88% % of ee's with development plan % 82% 98% 95% 96% E-learning course utilization rate % taken by 20+ 83% 97% 91% 97%

L&D Investment L&D Expenditure Million $ $15.8 $20.2 $9.9 49% $20.2 Opportunity Cost Million $ $3.4 $2.9 $1.3 45% $2.9 Cost Reduction (internal to L&D) Thousand $ $63 $295 $168 57% $325

Note: (1) Learning is expected to contribute to achievement of all ten top goals except Goal 8: Open Office in Beijing Key programs in support of the top ten goals include Marketing, Leadership, Safety, Fraud Detection, Business Acumen, Engineering, Performance Mgt, New Employee Orientation, and Innovation.

Results Through June

For 2012




Simple, consistent approach to reporting◦ Use standard definitions to extract and convert data into

three types of measuresOutcome, effectiveness, and efficiency

◦ Create three standard statements with these measuresOutcome, Effectiveness, and Efficiency

◦ Create three types of customized reports from these statements

Summary, Program (Initiative), and Operations



39Senior Executives Learning Executives

Executive Reports

Statements

Data Sets

Extract, convert and calculate Standard Measures

Data Sources

Executive Reporting ProcessG

uiding Principles

Financial Data Evaluation System

Learning Management System

Other Sources: HRIS, CRM, ERP

OutcomeStatement

Learning Effectivenes

sStatement

Learning EfficiencyStatement

Outcomes Effectiveness Efficiency

L&D Summary Report

Quarterly

L&D Program Report

Monthly

L&D Operational Report

Monthly

Step 1: Align learning to goals◦ => The Outcome StatementStep 2: Plan carefully◦ => The three statementsStep 3: Execute with discipline◦ => The three reportsStep 4: Measure and evaluate◦ => All statements and reports


Running learning like a business◦ The Business of Learning by David Vance◦ The Chief Learning Officer by Tamar Elkeles and Jack Phillips◦ Strategic Learning Alignment by Rita Smith

Resources Available at TDRprinciples.org◦ 56-page white paper, 18-page overview, 4-page summary◦ Definition of terms and measures for L&D◦ PowerPoints◦ Sample write-up of an L&D Summary Report for a CEO

Contact Dave Vance for more information:[email protected]


#CLOwebinar

Questions & Answers

Dave VanceFormer PresidentCaterpillar University

Join Our Next CLO Webinar

Virtual Training: Building aGlobal Training Strategy

Wednesday, September 19, 2012

CLO Webinars start at 2 p.m. Eastern / 11 a.m. Pacific

Register at www.clomedia.com/events

Join the CLO Network: http://network.clomedia.com/

#CLOwebinar

http://www.clomedia.com/events

Education

Using Talent Development Reporting Principles to Run Learning Like a Business