Upload
watershed
View
475
Download
3
Embed Size (px)
Citation preview
Research SpotlightLearning’s Big Data Problem: Measuring & Analyzing Impact
David Wentworth, Principal Analyst,
Brandon Hall Group@DavidMWentworth
Mike Rustici, CEO, Watershed
@mikerustici
Michael Rochelle, Chief Strategy
OfficerBrandon Hall Group
HCM Research and Advisory Firm
© Brandon Hall Group 2016 2
FOUNDED IN 1993
Community of 300,000+
10,000 Global Clients
Leading Independent HCM Research & Analyst Firm
Publishing 2+ Pieces of Research Each Day
Sample of Our Clients
How We Help You
© Brandon Hall Group 2016 3
MEMBERSHIP
STRATEGICCONSULTING
EXCELLENCEAWARDS
PROFESSIONAL DEVELOPMENT
Learning & Development
Talent Management
LeadershipDevelopment
Talent Acquisition
WorkforceManagement
Open Surveys
© Brandon Hall Group 2017 4
Visit www.brandonhall.com click Open Surveys
If you would like to join a panel of survey takers, please contact us at [email protected]
• Recruitment Marketing Practices 2017
• Extended Enterprise Learning 2017
• State of Talent Acquisition 2017
• HCM Technology 2017
How To Ask Questions
© Brandon Hall Group 2016 5
• Submit your questionsor comments about the discussion to our presenters using the Questions tab on your control panel.
• Presentation slides and giveaways can be found in the Handouts tab on your control panel.
Recording & Slides will also be sent out after the webinar.
Agenda
The State of Learning Measurement
Moving and Measuring Beyond Formal
How Do We Get Better?
The Role of Learning Standards
7
23%
27%
38%
40%
53%
66%
78%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%
To more strongly link learning and reduced turnover
To provide input to the performance management process
To determine the ROI of learning programs
To more strongly link learning and employee engagement
To more strongly link learning and individual performance
To more strongly link learning and organizational performance
To improve the effectiveness of learning programs
Drivers of Learning Measurement
14%
30%
37%
19%
Formal
8
51%
30%
17%
2%
Informal
43%
32%
16%
7%
Experiential
Not at all Somewhat
Moderately Highly
Effectiveness of Learning Measurement
• Formal Learning/Catalog
• Certification & Compliance
• Courses
• Groups Associated with Classes
• Impressions• Expertise Location
• Idea Engine• Real-Time
Collaboration• Groups &
Workspaces
9
Very Little Learning is Formal
The Blending Effect
4.9%
29.4%
46.1%
19.6%
Not at all
Small extent
Moderate extent
Great extent
Improved Link to Individual Performance
9.1%
37.4%
37.4%
16.2%
Not at all
Small extent
Moderate extent
Great extent
Improved Link to Organizational Performance
95% 91%
Source: Brandon Hall Group HCM Outlook 2017
How Do We Get Better?
© Brandon Hall Group 2016 11
“L&D is responsible for improving our most
important asset, our people. If we can improve
L&D, we can improve the world”
me, I said that
We can do better.
learning happenseverywhere.
Why connectwith xAPI?
To transfer learning records
Example: when you get a new learning platform
For products with different functions to work together
Example: your LMS and a social learning platform
To back up learning records
Example: in case your LMS breaks down
xAPI is a technical specification that makes it easier and
cheaper to connect learning technologies.
How far has xAPI adoption come?
xAPIadoption
Source: http://tincanapi.com/adopters/
Standards define
and confine an industry.
New modelOld model
eLearningLMS
2000
SCORM
eLearningLMS
MobileSocial
InformalSimulations
GamesMOOCs
ILTAdaptive
PersonalizedLRS
Learning Experience PlatformMany more……
2013
xAPI
xAPIdeployments
pilots
2015 – 16
prototypes
2013 – 14
production
2017 –
Why do I have to go beyond CMI-
5?
SCORM: Not evolving, but not dead
yet.
What’s a Learning Record Store?
Three typesof LRS
An Embedded LRS is typically
part of another system. It often
adds xAPI functionality to an
LMS or a sophisticated learning
experience provider. (Example:
Rustici Software’s Engine)
01
Embedded LRS
The main role of this LRS aligns
with the technical definition from
the xAPI spec; it receives,
stores, and provides access to
learning records via xAPI. These
systems are primarily technical
tools with an interface designed
for technical people. (Example:
Learning Locker)
02
Data Storage LRS
A Learning Analytics Platform
(LAP) is a tool that contains an
LRS, but layers a rich reporting
and analytics toolset on top of it.
It’s designed for learning
professionals and facilitates
robust analysis of all the data
collected by xAPI. (Example:
Watershed)
03
Learning Analytics
Platform
Tips for choosing an LRS
01Be on the same page.Make sure everyone involved in the process
shares a common definition for “learning record
store” and an understanding of the type of LRS
you need.
04Verify it meets expectations.Ask what you can do with the data in your LRS. Many
legacy LMS vendors have implemented an
embedded LRS to say they have xAPI, but, most
don’t have out-of-the-box ways to access that data for
reporting once it’s in the system.
02Ensure its xAPI compliant.ADL hasn’t released an xAPI certification
program, but there is a test suite coming soon
that you can run against any LRS to see how it
stacks up.
05Ensure unfettered access to data. One of the core tenants of xAPI is that data should
be freely portable across systems. Make sure your
LRS vendor will allow you to retrieve all the data you
store in your system.
03Join the xAPI community.The xAPI community is friendly and eager to
help. Get involved in discussions and ask more
about each provider. 06
Speak with client references. Make sure your vendor has deployed its product at a
scale and scope similar to your situation and speak
to an actual client to find out how the process and
deployment went.
Key Takeaways• Learning does not just exist within the confines of the
LMS.
• If we want learning to impact behavior and performance, we have to bring it closer to the point of need.
• Don’t let fear of measurement and tracking prevent you from getting started.
• xAPI is a standard that can help pull all the pieces of the puzzle together.
• AN LRS is the tool that lets you see the whole puzzle once it’s together.
Questions?
Submit your questions to the presenters via the
questions tab on your control panel
27
Type your questions here
Thank you for joining us today!
If you have any additional questions, please email us at [email protected].
Learn More About UsProducts and services that we offer
© Brandon Hall Group 2016 29
Membership Offers Tailored SupportRESEARCH ACCESS & EVENTS
• Reports • Case Studies, Frameworks & Tools• DataNow® & TotalTech®• Webinars and Research Spotlights• Annual HCM Conference
ADVISORY SUPPORT• Ask the Expert• 1 on 1 Consultations• Research Briefings• Benchmarking
CLIENT SUCCESS PLAN• Your Priorities• Executive Sponsor• Client Associate• Monthly Meetings
© Brandon Hall Group 2016 30
To learn more about becoming a member or to sign up for a free trial, please email us at [email protected]
Strategic Consulting Offers Expert Solution DevelopmentBENCHMARKING
• Competitive/Comparative
• Maturity Model
• Custom Research
STRATEGY• Business Case
• Planning
• Organization & Governance
TECHNOLOGY SELECTION• Vendor Selection
• Architecture Design
• Systems Evaluation
DEVELOPMENT & INTEGRATION• Program Design
• Assessment
• Survey
• Process Integration
© Brandon Hall Group 2016 31
To learn more about consulting, please email us at [email protected]
Why Brandon Hall Group
© Brandon Hall Group 2016 32
1
2
3
4
5
We offer flexible, high-value memberships that offer everything you need at a cost you can afford.
Our research is better – more targeted, easily digestible, focused on business results, predictive, prescriptive.
Our solutions gets better results – research-driven, focused on the business, simple, targeted, flexible approach.
Our case studies are unique for their diversity and focus on business results.
Our professional development is practical, tailoredto your needs, and equips you to take action.
We Do This Through…
Research and Analysis that Define New HCM Standards
Success Stories from Global Organizations that Show those Standards in Action
Professional Development that Assesses Current State and Enables Innovation
Flexible Services to Design and Implement Transformative Solutions
Recognition that Validates the Transformation
© Brandon Hall Group 2016 33
What Our Clients Have to Say
Brandon Hall Group’s Excellence Awards allows you to take a deeper dive and evaluate the programs that you’ve created at a level that you might not have the opportunity to do… They have always been a cornerstone for me.
- Katrina Williams, Director, Sales Capability
CDW
https://youtu.be/OgdA_tpM55U
© Brandon Hall Group 2016 34
What Our Clients Have to Say
I describe the attention and that customer service that I’ve experienced as being excellent. They are very attentive and wherever I need to be connected to whomever, they are there to help me.
- Joanne Veech, Global Talent Leader
PwC
https://youtu.be/YkY-v8gfW5k
© Brandon Hall Group 2016 35
HCMx Radio
© Brandon Hall Group 2016 36
THE ONLY PODCAST IN THE HCM ARENA THAT WEAVES CURRENT MARKET RESEARCH, HR TECHNOLOGY AND INDUSTRY LEADERS INTO EACH EPISODE
To catch up on the latest episodes, visit www.brandonhall.com/hcmxradio or subscribe on iTunes