Using Demographic Data to Gain Context and Insight

Housekeeping

• Asking Questions:

• Use the little hand on the side panel

• This alerts us there is a question

• Type question into the chat dialogue box

• The question will appear in the question pane

Housekeeping

Presentation

Q&A

Agenda

Housekeeping

Presenter

Brett HattenPrincipal Solution Architect at Explorance

Located in Chicago (USA)

THE MTM IMPACT COMMUNITY

The MTM Impact Community is a united family of Metrics That Matter users with a focus on connecting and learning to create a stronger community and an elevated impactful learning experience for all.

TOPICS 1. What does this mean?

2. Telling the right story

3. MTM application

4 Driving results and sustainability

5. Getting started

6. Questions

What does this mean?

Poll

Are your stakeholders actively asking you for the demographic distribution of your respondent data?

Demographics collected within MTM should be metadata that is important to business leaders.

Do not collect data simply… because.

We’re using breadcrumbs to lead us down the best path possible. Choose wisely.

Telling the right story

Poll

What types of demographic data are you currently collecting with your learning surveys?

(Role, Gender, Tenure, etc.)

MTM Application

MTM application

• Ready Reports

• Premium Dashboards

• Data Explorer

• *** Demographic Filtering ***

MTM application

Ready Reports

10+ Employment (Filtered by Question Answer)

MTM applicationPremium Dashboards

MTM application

Data Explorer

Less than 1 year 1+ years 3+ years 5+ years 7+ years 10+ years

Benchmark 6.06 6.06 6.06 6.06 6.06 6.06

Situational Leadership 6.30 6.03 - 4.76 - -

Delegating from the Middle 5.82 5.77 5.94 5.76 5.05 -

Managing Virtual Teams 5.74 5.08 6.14 6.16 5.86 -

Account Planning Fundamentals 6.46 5.25 5.95 - - -

Applied Business Strategy 6.03 5.94 6.02 5.17 6.18 6.03

Career Planning 5.71 5.80 6.04 5.42 5.47 5.83

Leadership 2020 5.83 5.73 6.22 6.24 5.68 6.61

Performance Management Foundations 5.76 5.95 5.43 5.54 4.93 -

Product Showcase 5.69 5.67 5.61 5.83 5.10 5.83

Transition to Management 5.91 6.03 - - - -

Questions?

Driving Results

Driving results and sustainability

Surface level data can be misleading or not reveal potential blind spots within feedback.

Look deeper to better understand your audience and develop a greater understanding of areas of success or opportunity.

Driving results and sustainability

• Stick to your plan

• Manage by exception

• See the forest through the trees

• Use a scalpel not a chainsaw

Driving results and sustainability:Stick to your plan

A sustainable/ repeatable reporting strategy that focuses upon stakeholder needs and data used to drive business decisions remains priority.

Role Class Trigger Monthly QuarterlyOngoing Monitoring /

Data Exploration

Instructor Instructor Class SummaryInstructor Performance

ReportInstructor Dashboard*

Instructor Manager

Instructor Class Summary

(triggered forlow-scoring classes)

Instructor Performance

Summary for team

Instructor Performance

Report for each Instructor

Data Explorer*,

Instructor Manager Dashboard*

Course Developer /

Instructional Designer

Course Summary Report

w/Course, Class & Instructor Level Details,

Learner Comment Analysis

Course Summary Report

w/Course, Class & Instructor Level Details

Data Explorer*,

Course Owner Dashboard*, Report Card

w/Key Driver Analysis

Instructional Design

Manager

Course Summary Report

w/Course, Class & Instructor Level Details

(for courses owned by

team)

Course Summary Report

w/Course, Class & Instructor Level Details

(for courses owned by

team)

Data Explorer*,

Course Owner Dashboard*, Report Card

w/Key Driver Analysis

Learning Analyst /

Measurement Champion

Report Card

w/Key Driver Analysis,Course Summary Report

w/Course, Class &

Instructor Level Details, Learner Comment Analysis

Report Card

w/Key Driver Analysis, Executive Summary Report

Data Explorer*, Dashboard*

Learning Consultant

Course Summary Report

w/Course, Class & Instructor Level Details,

Learner Comment Analysis

Key Metrics Report

(based on priority), Executive Summary Report

Data Explorer*, Dashboard*,

Report Card w/Key Driver Analysis (by demographic)

L&D LeaderReport Card

w/Key Driver Analysis

Executive Summary Report,

Data Analysis Report (prepared by Learning

Analyst)

Dashboard*

Driving results and sustainability:Manage by exception

Isolate demographic groups that are affecting overall performance.

Focus on both opportunities and success stories!

Less than 1 year 1+ years 3+ years 5+ years 7+ years 10+ years

Benchmark 6.06 6.06 6.06 6.06 6.06 6.06

Situational Leadership 6.30 6.03 - 4.76 - -

Delegating from the Middle 5.82 5.77 5.94 5.76 5.05 -

Managing Virtual Teams 5.74 5.08 6.14 6.16 5.86 -

Account Planning Fundamentals 6.46 5.25 5.95 - - -

Applied Business Strategy 6.03 5.94 6.02 5.17 6.18 6.03

Career Planning 5.71 5.80 6.04 5.42 5.47 5.83

Leadership 2020 5.83 5.73 6.22 6.24 5.68 6.61

Performance Management Foundations 5.76 5.95 5.43 5.54 4.93 -

Product Showcase 5.69 5.67 5.61 5.83 5.10 5.83

Transition to Management 5.91 6.03 - - - -

Driving results and sustainability:Seeing the forest through the trees

Presenting demographic success stories and opportunities supplement (not replace) overall portfolio, performance, and courseware measures.

Lose the battle, win the war – if necessary

Driving results and sustainability:Use a scalpel not a chainsaw

• Identify a potential area for corrective action

• Confirm if the hypothesis holds true

• Isolate the data to better understand steps necessary to manage change

Getting Started

Getting startedWhere to begin

Premium Dashboards

• Create a demographic dashboard

• Filter to high value/ high impact courses

• Focus on demographics with lowest/highest NPS, Scrap, and/or Benefit to Cost ratios

Data Explorer

• If you don’t have access to dashboards, use the Course Compare View with Demographic values as columns

• Use the Demographic topic to isolate the answer values which are performing at or below expectations

Questions to Ask

• Are certain demographic groups repeatedly affecting the overall score for a course or an instructor?

• Are the correct people attending the right training at the right time?

• Is that the target audience of the training event? If so, what qualitative feedback has been provided that may add clarity? • Use Demographic Answer filtering to generated a

targeted Learner Comment Analysis Report w/ Details.

Getting startedLeverage Question Answer Filtering in MTM Reporting

Getting startedLeverage Question Answer Filtering in MTM Reporting

Getting startedUse a demographic dashboard to save time!

“Report By” Demographic Widgets:

• Trend

• Ranking

• Activity

Getting startedData Explorer gets to the bottom of it!

Questions?

Thank You