Upload
nguyenkhanh
View
217
Download
1
Embed Size (px)
Citation preview
BERSIN & ASSOCIATES © 2011
BERSIN & ASSOCIATES
Research Bulletin | 2011
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
Introduction : Talent Management Continues to Grow
Integrated talent management continues to be one of the most
important strategies in business today. Companies are integrating their
recruiting, talent development, succession and leadership strategies to
drive growth, productivity, and employee retention and engagement.
As our Talent Management Framework1 illustrates (see Figure 1), these
strategies include many initiatives and potential areas on which to
focus. Retailers, for example, which deal with high-volume recruiting
and high turnover rates, focus heavily on assessment, talent acquisition
and onboarding strategies to drive their business. Professional services
firms focus on deep skills development, knowledge-sharing and
organizational collaboration to drive strong levels of expertise and
client value. All organizations understand that investment in leadership
development at all levels is core to business success.2
1 For more information, The Talent Management Framework: A Modern Approach for Developing and Mobilizing Talent, Bersin & Associates / Josh Bersin, Stacey Harris, Kim Lamoureux, Madeline Laurano and David Mallon, May 2010. Available to research members at www.bersin.com/library.
2 For more information, High-Impact Leadership Development, Bersin & Associates / Laci Loew and Stacia Sherman Garr, October 2011. Available to research members at www.bersin.com/library or for purchase at www.bersin.com/hild.
November 29, 2011 Volume 6, Issue 59
BERSIN & ASSOCIATES, LLC180 GRAND AVENUE
SUITE 320OAKLAND, CA 94612
(510) [email protected]
KnowledgeAdvisors and Metrics that MatterTM: A Leading Platform for Learning and Talent Measurement
About the Author
Josh Bersin,Principal Analyst
BERSIN & ASSOCIATES © 2011
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES, LLC6114 LA SALLE AVENUE
SUITE 417OAKLAND, CA 94611
(510) [email protected]
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES © 2011
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
KnowledgeAdvisors and Metrics that MatterTM Josh Bersin | Page 2
Training and Capability Development Play a Foundational Role
As our Framework points out, training, coaching and what we call,
“capability development,” form a foundation to almost every piece of
the talent management puzzle. In fact, our research shows that, among
all the various talent management areas on which to focus, excellence
in development planning, career development and training drive the
highest value of all.3 Companies that excel in these areas actually drive
almost twice the revenue per employee over those which do not invest
in training and development planning.
3 For more information, Talent Management Factbook 2010: Best Practices and Bench-marks in U.S. Talent Management, Bersin & Associates / Karen O’Leonard and Stacey Harris, September 2010. Available to research members at www.bersin.com/library or for purchase at www.bersin.com/tmfactbook.
Source: Bersin & Associates, 2010.
Figure 1: Bersin & Associates Talent Management Framework®
Companies that excel
in development
planning, career
development and
training drive almost
twice the revenue
per employee over
those which do not.
KEY POINT
BERSIN & ASSOCIATES © 2011
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES, LLC6114 LA SALLE AVENUE
SUITE 417OAKLAND, CA 94611
(510) [email protected]
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES © 2011
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
KnowledgeAdvisors and Metrics that MatterTM Josh Bersin | Page 3
The spending on training and development is large. Companies spend
close to $50 billion a year in all areas of training and development
(around $680 per employee on average in 2010)4, and this investment
goes into a wide array of programs – onboarding, functional and
technical training, leadership development, certificate programs,
employee skills assessment, coaching, compliance training, performance
support, and social and informal learning. When any manager tries to
improve the performance of an employee, ultimately his / her challenge
is to assess performance, coach the employee to fill gaps and train them
to perform better.
So while training itself is often run by a training department, or
learning and development team, the work that training does is diverse
and broadly used throughout the organization. Our research continues
to show high-impact learning organizations outperform their peers
through their focus on the tools, technologies, skills and organizational
structure of their training function.
Training and Capability Measurement Continue to be a Challenge
Despite the proliferation of learning management systems (LMSs) and
talent management software platforms in the market5, companies
still struggle to measure the effectiveness, efficiency and alignment
of training. Because training is a highly customized and somewhat
“indirect” driver of business impact, there are many models for training
measurement (i.e., the Bersin Impact Measurement Model6, the
4 For more information, The Corporate Learning Factbook® 2011: Benchmarks, Trends and Analysis of the U.S. Training Market, Bersin & Associates / Karen O’Leonard, January 2011. Available to research members at www.bersin.com/library or for purchase at www.bersin.com/factbook.
5 For more information, Talent Management Systems Buyer’s Guide, Bersin & Associates / Josh Bersin and Katherine Jones, November 2011. Available to research members at www.bersin.com/library or for purchase at www.bersin.com/tms.
6 For more information, please read the following two publications: (1) High-Impact Talent Management: Trends, Best Practices and Industry Solutions, Bersin & Associates / Josh Bersin, May 2007, available to research members at www.bersin.com/library or for purchase at www.bersin.com/hitm; and, (2) The Training Measurement Book: Best Practices, Proven Methodologies, and Practical Approaches, Josh Bersin / Pfeiffer, 2008, available for purchase at www.bersin.com.
In 2010,
companies spent
close to $50
billion in all areas
of training and
development,
with around $680
per employee on
average in 2010.
KEY POINT
BERSIN & ASSOCIATES © 2011
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES, LLC6114 LA SALLE AVENUE
SUITE 417OAKLAND, CA 94611
(510) [email protected]
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES © 2011
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
KnowledgeAdvisors and Metrics that MatterTM Josh Bersin | Page 4
Kirkpatrick Model7, the Success Case Methodology8 and many others).
Our research shows that only six percent of HR organizations rate
themselves “highly skilled” at talent measurement, and 56 percent rate
themselves “poor” at understanding their organization’s skills and what
programs best drive business value.9
Not only is training measurement imperfect and complex, there are
many tools and systems to use. In order to adequately measure and
monitor capability development programs, organizations must use
survey tools and assessments – and then match this data against many
data elements in the LMS and HR management systems. So while
nearly every learning and HR software vendor sells a measurement and
analytics module to their systems, companies typically tie together as
many as seven different systems to build an integrated approach to
training measurement.10
Enter KnowledgeAdvisors: A Company Laser Focused on Learning and Talent Measurement
Enter KnowledgeAdvisors. Founded in 1999, this company developed a
platform and related services to help organizations to solve the puzzling
problem of creating an integrated solution for learning measurement.
KnowledgeAdvisor’s product, Metrics that MatterTM, was originally
designed as a highly scalable system to distribute employee surveys
and assessments to people, and then collect, analyze and manage this
information around a given training program and across the entire
training function.
7 Donald Kirkpatrick’s four-level Measurement Model has been widely published in many articles and its terminology is well-known to most training professionals. The origi-nal model was published in Training and Development Handbook, R. L. Craig, McGraw-Hill, 1976.
8 Source: The Success Case Method: Find Out Quickly What’s Working and What’s Not, Robert O. Brinkerhoff, Ed.D. / Berrett-Koehler Publishing Inc., 2003.
9 For more information, The High-Impact HR Organization: Top 10 Best Practices on the Road to Excellence, Bersin & Associates / Stacey Harris, January 2011. Available to research members at www.bersin.com/library or for purchase at www.bersin.com/hihr.
10 For more information, Talent Management Factbook 2010: Best Practices and Benchmarks in U.S. Talent Management, Bersin & Associates / Karen O’Leonard and Stacey Harris, September 2010.
Our research
shows that only
six percent of HR
organizations rate
themselves “highly
skilled” at talent
measurement, and
56 percent rate
themselves “poor”
at understanding
their organization’s
skills and what
programs best drive
business value.
KEY POINT
BERSIN & ASSOCIATES © 2011
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES, LLC6114 LA SALLE AVENUE
SUITE 417OAKLAND, CA 94611
(510) [email protected]
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES © 2011
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
KnowledgeAdvisors and Metrics that MatterTM Josh Bersin | Page 5
Because the company originally focused primarily on the measurement
of training, KnowledgeAdvisors built a wide array of reports,
dashboards and tools to make training measurement easy. The system
includes many predefined measurement templates that implement
the most modern training measurement models in the market. Data is
collected and aggregated precisely the way a CLO or training manager
wants to see it; reports and dashboards give users the ability to measure
program efficiency and effectiveness, manager satisfaction and value,
learning transfer, and a wide variety of other measures. Since training
measurement has a number of workflow elements (e.g., end-of-
course surveys, follow-up surveys, manager surveys, etc.), the platform
is also designed to address all of the various models for collecting
measurement data.
As KnowledgeAdvisors grew, the company started to realize that, if
customers used standard assessment questions, they could benchmark
various learning measures against other companies. So over the
years, KnowledgeAdvisors has built a large database of training-
related metrics which let companies compare their training programs’
effectiveness against similar programs in companies of their size and
industry. This type of benchmarking may not specifically tell a company
what they are doing well or poorly, but it gives them tremendous
insights into what possible elements of a training program they may
need to radically improve.
As the company’s customer base grew, KnowledgeAdvisors realized that
businesses need much more than a tool set – they need advice, standard
reports and consulting to build the right training and capability
measurement solution. So KnowledgeAdvisors built up its consulting
team and worked closely with companies, like Bersin & Associates,
to identify new models for measurement and to help their clients
implement them. Today KnowledgeAdvisors and Bersin and Associates
have worked together at many large companies to help organizations
build an end-to-end training and talent measurement strategy.
Our experience with KnowledgeAdvisors over the years has shown
that the company’s platform and services provide an invaluable service
to any midsize or large organization that wants to better understand
how to understand, manage, improve and benchmark its training
Our experience with
KnowledgeAdvisors
over the years has
shown that the
company’s platform
and services provide
an invaluable
service to any
midsize or large
organization that
wants to better
understand how
to understand,
manage, improve
and benchmark its
training investment.
A N A LY S I S
BERSIN & ASSOCIATES © 2011
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES, LLC6114 LA SALLE AVENUE
SUITE 417OAKLAND, CA 94611
(510) [email protected]
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES © 2011
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
KnowledgeAdvisors and Metrics that MatterTM Josh Bersin | Page 6
investment. Cognizant, a global IT consulting firm, standardized on
Metrics that Matter for its corporate university which trained more than
100,000 employees in 2010. Cognizant now uses KnowledgeAdvisors’
product as a standard tool to measure feedback, learning effectiveness
and blended-learning design for all courses – and has improved its
learning effectiveness measures by more than 10 percent in the last
year. (Read our case study on Eaton University’s implementation of
Metrics that Matter for more details on how KnowledgeAdvisor’s
training measurement solutions work.11)
Evolving into a SaaS12 Talent Measurement Solution
As organizations have evolved their end-to-end talent management
strategies, they are now integrating the L&D function into many more
areas of HR. High-impact learning organizations typically create a
“federated” model13 for L&D – in which the team supports not only
operational training and skills development, but a wide variety of other
areas of the business.14 This includes providing support for onboarding,
leadership development, sales enablement, customer training and
general performance consulting. So, while “training measurement”
continues to be an important and highly complex problem,
KnowledgeAdvisor’s platform can be extended to measure these other
areas, as well.
11 For more information, Eaton University: Training Analytics Excellence in a Corporate University, Bersin & Associates, April 2004. Available to research members at www.bersin.com/library.
12 “Software as a Service” (or SaaS) refers to the business of selling software over the Internet as a web service. In this business model, the software vendor charges an annual “rental” fee and can predict recurring revenues far more reliably than with the licensed software model. As SaaS becomes proven and the delivery strategies more refined, most analysts agree that more and more enterprise software will be sold in this way.
13 A “federated model” has a small core team that manages some technology and corporate programs, and empowers business and functional units to run their own training programs.
14 For more information on the federated model, The High-Impact Learning Organization: WhatWorks® in the Management, Governance and Operations of Modern Corporate Training, Bersin & Associates / Josh Bersin, May 2008. Available to research members at www.bersin.com/library or for purchase at www.bersin.com/highimpact.
Three of
KnowledgeAdvisors’
most recent
extensions focus
on the areas
of onboarding,
leadership
development and
sales training.
K E Y P O I N T
BERSIN & ASSOCIATES © 2011
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES, LLC6114 LA SALLE AVENUE
SUITE 417OAKLAND, CA 94611
(510) [email protected]
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES © 2011
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
KnowledgeAdvisors and Metrics that MatterTM Josh Bersin | Page 7
Over the last several years, the company has extended its solution
to provide more specific application-level measurement solutions,
leveraging the same technology, workflow and benchmarking available
in its core training offering. Three of KnowledgeAdvisors’ most recent
extensions focus on the areas of onboarding (a critical part of talent
acquisition), leadership development and sales training.
• Onboarding – Since onboarding is essentially a workflow-based
training process, it has many of the same characteristics of a training
program. Employee onboarding requires a high degree of content
and workflow management and tools. (Vendors such as Taleo,
Oracle / PeopleSoft and SilkRoad offer very powerful automation
systems to manage the entire onboarding process.) But how do you
measure the effectiveness of various onboarding programs? How do
you determine the level of readiness of new employees when they
finish? Even more importantly, how do you continuously monitor
the effectiveness of the company’s onboarding after the initial
program is completed?
KnowledgeAdvisors now offers a solution in this area. Using much
of the same underlying infrastructure for training measurement,
KnowledgeAdvisors has now built a set of dashboards, reports
and models to help companies measure onboarding effectiveness
and impact.
Onboarding typically generates a very high return on investment –
but, since it is not formal training, it is often difficult to measure.
One of KnowledgeAdvisors’ clients showed 15,000 hours of
increased productivity to the business as a result of the increased
speed to competency. This is the type of analysis which can help to
both justify the program and also find areas for improvement.
• Leadership Development – Leadership development is a special
type of training process. It often includes a highly blended model,
which includes 360-degree assessments15, action learning16, formal
15 A “360-degree assessment” incorporates feedback from the worker, and his / her man-ager, peers, superiors, subordinates and customers.
16 “Action learning” is a process in which groups of learners collaborate to solve actual workplace problems. In this way, organizations benefit from gaining solutions to critical
Onboarding
typically generates
a very high return
on investment –
but, since it is not
formal training, it
is often difficult to
measure.
K E Y P O I N T
No two leadership
development
programs are
identical.
K E Y P O I N T
BERSIN & ASSOCIATES © 2011
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES, LLC6114 LA SALLE AVENUE
SUITE 417OAKLAND, CA 94611
(510) [email protected]
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES © 2011
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
KnowledgeAdvisors and Metrics that MatterTM Josh Bersin | Page 8
classroom and e-learning, developmental assignments, special
projects, simulations, books and special reading assignments, and
competency-based assessments. No two leadership development
programs are identical; companies change and improve these
programs regularly, and often use third-party consultants and
content on a regular basis. Again, KnowledgeAdvisors has extended
its platform, reports and dashboards to give companies easy-to-use
tools to measure the satisfaction, efficiency and effectiveness of
leadership development. Applying the concepts developed for the
training industry, KnowledgeAdvisors gives customers easy-to-use
reports to measure satisfaction, job impact, alignment, efficiency
and follow-up measures to quantify actual results.
Executive and leadership development is particularly hard
to measure. Many of our clients invest heavily in leadership
development but find that budgets are cut during tough times. The
KnowledgeAdvisors system would not only help to maintain that
ongoing investment, but would also give the program manager
detailed feedback on the satisfaction and effectiveness of each
element of the program across a broad range of leaders.
• Sales Training – Sales training, like leadership development, is
one of the most universal and important development investment
areas in talent management. Like any other form of training,
organizations want to measure the effectiveness, efficiency, business
alignment and impact of sales training, so that these programs can
be continuously improved. KnowledgeAdvisors has built special
reports and dashboards to make sales training easy to measure and
to help companies benchmark their sales training programs against
peers in their industry.
Sales training is also a program that we know organizations
need, but is often difficult to cost-justify directly. By using a tool
like Metrics that Matter, companies can diagnose different
modalities, audiences and geographies, as well understand which
elements work best and which elements of the program need the
most improvement.
challenges and participants benefit by learning from their experiences.
Companies change
and improve
these programs
regularly, and
often use third-
party consultants
and content on a
regular basis.
K E Y P O I N T
Sales training is also
a program that we
know organizations
need, but is often
difficult to cost-
justify directly.
K E Y P O I N T
BERSIN & ASSOCIATES © 2011
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES, LLC6114 LA SALLE AVENUE
SUITE 417OAKLAND, CA 94611
(510) [email protected]
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES © 2011
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
KnowledgeAdvisors and Metrics that MatterTM Josh Bersin | Page 9
Figure 2: Where KnowledgeAdvisors Impacts in Talent Management Areas
Source: Bersin & Associates, 2011.
The Value of a Single Measurement Platform across Many Types of Programs
All of the solution areas discussed here (including many other training-
related applications like customer service training, technical training,
etc.) are built in different ways. Some are highly instructor-led, some are
blended, some are highly informal or online, and some use many forms
of content. It is very inefficient (and usually impossible) for a company
to put in place separate measurement programs, tools and back-end
systems for each.
BERSIN & ASSOCIATES © 2011
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES, LLC6114 LA SALLE AVENUE
SUITE 417OAKLAND, CA 94611
(510) [email protected]
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES © 2011
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
KnowledgeAdvisors and Metrics that MatterTM Josh Bersin | Page 10
Our High-Impact Learning Measurement17 research (and also The
Training Measurement Book®18) describes a complete model for
measurement of any program – but as we point out in our research,
once you build a model you still need a way to manage this
measurement program. Surveys must be developed and they must be
deployed – and then lots of information must be collected, filtered,
sorted and analyzed. Ultimately, what makes measurement useful is
having the time to analyze the data (i.e., to view it by program, by
audience group, by program element, etc.).
KnowledgeAdvisors’ system can become this single measurement
platform for all of the various different types of development programs
in a company. Because the system has a standard toolset to build
surveys, develop reports and benchmark programs against many other
companies, organizations are beginning to “standardize” their talent
development measurement on the KnowledgeAdvisors platform.
Extensible beyond the LMS
How does KnowledgeAdvisors’ platform compare to a company’s
LMS? It turns out that most large companies (more than 70 percent
according to our research) have some type of learning management
system today and ideally this platform is designed to be the single
repository for training information.19 While LMS systems have built-in
testing and assessment tools, they have not been finely crafted with
the templates, reports, dashboards and SaaS-enabled benchmarks
available in Metrics that Matter™. In fact, during the last few years,
LMS vendors have shifted their technologies so heavily toward talent
17 For more information, High-Impact Learning Measurement: Best Practices, Models, and Business-Driven Solutions for the Measurement and Evaluation of Corporate Training, Bersin & Associates / Josh Bersin, November 2006. Available to research members at www.bersin.com/library or for purchase at www.bersin.com/measurement.
18 For more information, The Training Measurement Book: Best Practices, Proven Methodologies, and Practical Approaches, Josh Bersin / Pfeiffer, 2008, available for purchase at www.bersin.com.
19 For more information, Learning Management Systems 2011: The Definitive Buyer’s Guide to the Global Market for Learning Management Solutions, Bersin & Associates / David Mallon, December 2010. Available to research members at www.bersin.com/library or for purchase at www.bersin.com/lms.
According to our
research, more than
70 percent of large
companies have
some type of LMS
today and ideally
this platform is
designed to be the
single repository
for training
information.
KEY POINT
BERSIN & ASSOCIATES © 2011
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES, LLC6114 LA SALLE AVENUE
SUITE 417OAKLAND, CA 94611
(510) [email protected]
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES © 2011
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
KnowledgeAdvisors and Metrics that MatterTM Josh Bersin | Page 11
management that most have actually reduced their investments in
training measurement tools.
This gives KnowledgeAdvisors an even more important role in the
market. As companies build more diverse types of training programs
(e.g., informal and social learning, simulations, and gaming, as well
as e-learning and mobile applications), the need for an extensible,
easy-to-use measurement platform becomes bigger. Today, many
large organizations (such as NetworkAppliance, Defense Acquisition
University and Deloitte) use KnowledgeAdvisors in complement with
their learning management systems. The LMS manages employee
enrollment, course administration, and tracking and reporting
information – KnowledgeAdvisors serves as the “training analytics”
system, giving the organization all of the information it needs to
measure program-level effectiveness, efficiency, alignment and impact.
In these companies, KnowledgeAdvisors’ platform itself is the “training
analytics” solution and is totally complementary to the tools available
in most LMS systems. Data is often summarized and extracted from
KnowledgeAdvisors, and then moved into other HR analytics tools for
broader analysis.
Creating Standards for Training Measurement: TDRP – Talent Development Reporting Principles
This past year, KnowledgeAdvisors took the lead to pull together a
variety of organizations (including Bersin & Associates) to create an
open standard for the reporting of training and talent development
metrics. This program is called TDRP.20
Think for a minute about the accounting profession. There are
“generally accepted accounting principles (GAAP)” that spell out what
an income statement, balance sheet and cashflow statement should look
like. When finance professionals go about their work, they can capture
and analyze information, and then present it in a standard format.
20 For more information on the TDRP standards and program, please see the following report which includes am introduction by Josh Bersin, TDRP – The Beginning of Standards in Training Measurement, Bersin & Associates / Josh Bersin, July 21, 2011. Available to research members at www.berisn.com/library.
BERSIN & ASSOCIATES © 2011
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES, LLC6114 LA SALLE AVENUE
SUITE 417OAKLAND, CA 94611
(510) [email protected]
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES © 2011
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
KnowledgeAdvisors and Metrics that MatterTM Josh Bersin | Page 12
TDRP is trying to do this for the training profession. Today, TDRP
founding members include Qualcomm, Deloitte, Lowes, Lockheed
Martin, Cigna, NetApp, Sodexo, BP, CNA Insurance and ADP.
The initiative has focused on the following two major areas.
• First, create a set of standard “outcome statements” – similar
to GAAP. These include a business outcome statement, training
effectiveness statement and training efficiency statement.
◦ The business outcome statement shows how various L&D
investments tie to business outcomes.
◦ The effectiveness statement summarizes results in terms of
satisfaction, effectiveness, utility and alignment (some of
these measures come from the Bersin & Associates Impact
Measurement Framework – see Figure 3).
◦ The efficiency statement standardizes measures of cost,
volumes and utilization.
• Second, create a set of definitions (still a work in progress), so
that someone new to training measurement can review this to
understand what these various terms mean.
In the Bersin & Associates Impact Measurement Framework®, we find
four key elements to training measurement:
1. Effectiveness – How well does the L&D program or investment
actually work as designed;
2. Efficiency – How well is the L&D team using its resources efficiently
to build the right types of programs in the right blend;
3. Alignment – How well does the L&D investment align with current
urgent business priorities; and,
4. Impact – What real business impact is being driven by the programs.
We identified nine measurement areas – and recently added three more to
cover the effectiveness, efficiency and alignment of informal learning.21
21 For more information, The Impact Measurement Framework: An Update for the Measurement of Informal Learning, Bersin & Associates / Josh Bersin, May 3, 2011. Available to research members at www.bersin.com/library.
In the Bersin &
Associates Impact
Measurement
Framework, we
identified nine
measurement
areas – and recently
added three
more to cover
the effectiveness,
efficiency and
alignment of
informal learning.
KEY POINT
BERSIN & ASSOCIATES © 2011
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES, LLC6114 LA SALLE AVENUE
SUITE 417OAKLAND, CA 94611
(510) [email protected]
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES © 2011
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
KnowledgeAdvisors and Metrics that MatterTM Josh Bersin | Page 13
TDRP, which is an open initiative, borrows from some of this work to
build a set of “standards” that companies can use to get started with
their reporting programs.
Note that TDRP is a set of “reporting” principles, not “measurement”
principles. The idea of TDRP is to help companies create standard
reports, so that tools like Metrics that Matter (and others) can deliver
standard reports which business people can start to understand. This
makes TDRP very valuable to CLOs and training managers because,
if it succeeds, we can all compare more standardized outputs across
different organizations and different systems.
TDRP does not replace or compete with the Bersin & Associates
Measurement Framework (or any other model for that matter), because
it does not specify all the training measures a company needs, nor
does it try to describe how these measures are captured or many of the
details of an ongoing measurement program.
Figure 3: Bersin & Associates Impact Measurement Framework®
Source: Bersin & Associates, 2011.
BERSIN & ASSOCIATES © 2011
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES, LLC6114 LA SALLE AVENUE
SUITE 417OAKLAND, CA 94611
(510) [email protected]
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES © 2011
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
KnowledgeAdvisors and Metrics that MatterTM Josh Bersin | Page 14
As one of the leaders in training measurement (and by far the most
active company in this highly fragmented area), KnowledgeAdvisors is
taking on a very important role in making TDRP come true – and we do
hope that, over time, more and more companies see TDRP as a standard
way of reporting training effectiveness, efficiency and impact.
Summary: Measurement Really Matters
Because training and HR are somewhat “soft” areas of business,
organizations continually look for better ways to measure effectiveness,
efficiency and impact. Our research shows that companies do not
“implement” measurement programs – rather they “evolve” them. Early
on, companies start measuring things and they find that some of these
measures are highly effective and others are not, and then they improve
the measurement over time.
Our industry studies on The High-Impact Learning Organization22
and The High-Impact HR Organization23 both looked at the maturity
of measurement programs in many companies. What both of these
research programs found is that organizations which buckle down
and focus on measurement year after year significantly improve their
overall impact. As the old maxim goes, “You can’t improve what you
can’t measure,” which applies heavily to all areas of human resources.
KnowledgeAdvisors understands this – and focuses on giving companies
the tools they need to start fast and grow their measurement
sophistication over time.
In summary, this bulletin is designed to help organizations to better
understand the KnowledgeAdvisors offering, and to give readers the
Bersin & Associates perspectives on the company’s product, strategy and
direction. We believe that KnowledgeAdvisors plays a vital role in the
measurement of training and other talent development programs, and
that the company will continue to grow.
22 For more information, The High-Impact Learning Organization: WhatWorks® in the Management, Governance and Operations of Modern Corporate Training, Bersin & Associates / Josh Bersin, May 2008.
23 For more information, The High-Impact HR Organization: Top 10 Best Practices on the Road to Excellence, Bersin & Associates / Stacey Harris, January 2011.
Our research shows
that companies do
not “implement”
measurement
programs – rather
they “evolve”
them.
KEY POINT
BERSIN & ASSOCIATES © 2011
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES, LLC6114 LA SALLE AVENUE
SUITE 417OAKLAND, CA 94611
(510) [email protected]
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES © 2011
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
KnowledgeAdvisors and Metrics that MatterTM Josh Bersin | Page 15
While there are many talent platforms in the market (i.e., LMS
companies, talent management software companies, HRMS companies),
none do what KnowledgeAdvisors does. The company’s product is
highly complementary to these systems, and forms the basis for a
standard talent development measurement platform across many
training and development applications.
We believe that
KnowledgeAdvisors
plays a vital role in
the measurement of
training and other
talent development
programs, and that
the company will
continue to grow.
A N A LY S I S
BERSIN & ASSOCIATES © 2011
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES, LLC6114 LA SALLE AVENUE
SUITE 417OAKLAND, CA 94611
(510) [email protected]
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
BERSIN & ASSOCIATES
Research Bulletin | 2011
BERSIN & ASSOCIATES © 2011
THIS MATERIAL IS LICENSED TO KNOWLEDGEADVISORS FOR DISTRIBUTION ONLY.
KnowledgeAdvisors and Metrics that MatterTM Josh Bersin | Page 16
The Bersin & Associates Membership ProgramThis document is part of the Bersin & Associates Research Library. Our research is provided exclusively to organizational members of the Bersin & Associates Research Program. Member organizations have access to the largest library of learning and talent management related research available. In addition, members also receive a variety of products and services to enable talent-related transformation within their organizations, including:
• Research – Access to an extensive selection of research reports, such as methodologies, process models and frameworks, and comprehensive industry studies and case studies;
• Benchmarking – These services cover a wide spectrum of HR and L&D metrics, customized by industry and company size;
• Tools – Comprehensive tools for HR and L&D professionals, including tools for benchmarking, vendor and system selection, program design, program implementation, change management and measurement;
• Analyst Support – Via telephone or email, our advisory services are supported by expert industry analysts who conduct our research;
• Strategic Advisory Services – Expert support for custom-tailored projects;
• Member Roundtables® – A place where you can connect with other peers and industry leaders to discuss and learn about the latest industry trends and best practices; and,
• IMPACT® Conference: The Business Of Talent – Attendance at special sessions of our annual, best-practices IMPACT® conference.
• Workshops – Bersin & Associates analysts and advisors conduct onsite workshops on a wide range of topics to educate, inform and inspire HR and L&D professionals and leaders.
For more information about our membership program, please visit us at www.bersin.com/membership.