Embed Size (px)
Citation preview
Insights & Ideas for
Improving Performance
Improvement InterventionsDenise Traicoff
Centers for Disease Control & Prevention
June 14, 2018
Objectives
Discover a ‘conventional wisdom’ that has recently been debunked
Consider Dr. Will Thalheimer’s brand new learning-transfer evaluation model to add rigor to your training evaluations
Identify useful tools for workplace, work and worker analysis
Advise the Atlanta chapter in continuous improvement
Move yourself from fixed mindset (you either
have it or you don’t) to growth mindset: you
can grow from making mistakes
Cheryl Lass, Skill director
Kaiser Permanente
Overview
Lessons from Dr. Will Thalheimer
Work Analytic tools from Danny & Kathleen W
Langdon
Other favorite insights
Brainstorm improvements for Atlanta chapter
Will Thalheimer
8 Level Model of Learning Evaluation
How confident are you that you know who
created the Four-Level method of
Learning evaluation?
A.Very confident
B.Somewhat confident
C.Not confident
Introducing the Katzell-Kirkpatrick
Evaluation Model
1. To determine how the trainees feel about the program.
2. To determine how much the trainees learn in the form of
increased knowledge and understanding.
3. To measure the changes in the on-the-job behavior of
the trainees.
4. To determine the effects of these behavioral changes
on objective criteria such as production, turnover,
absenteeism, and waste.
From “How to start an objective evaluation of your training program” D. Kirkpatrick 1956 and
attributed to “Raymond Katzell, a well-known authority in the field [of training evaluation]”
Join Will’s
Debunker’s
club!
What secret messages does Katzell-Kirkpatrick
send?
1-Reaction: instructor, facilities, delivery
2-Learning:written/oral tests, case studies, skill demo
3-Behavior: quality,
productivity, attitude
4-Results: impact on
organization-financial,
customer sat, emp sat,
process cycle time
Here’s a topic: Discuss
Change in knowledge and/or skills is the
only valid measurement of training
Research has found
Measuring:
Attendance
Course completion
Attention
Interest
Participation
Are inadequate ways to evaluate learning
Messaging our Evaluation Model Should Have: A Few Examples
• Just because learners ENGAGE IN LEARNING doesn’t mean they will have learned. Therefore, measuring attendance is an inadequate way of evaluating learning.
• Just because learners PAY ATTENTION doesn’t mean they learned. Measuring attention is an inadequate way of evaluating learning.
• Just because learners ACTIVELY PARTICIPATE in learning doesn’t mean they learned. Measuring participation is inadequate
• Just because learners say they LIKE A LEARNING EVENT doesn’t mean they learned. Therefore, surveying learners on their general satisfaction is an inadequate way of evaluating learning.
• Just because learners REPORT THEY HAVE EXPERIENCED EFFECTIVE LEARNING METHODS doesn’t guarantee they learned. Therefore, surveying learners on their experience with learning methods, must be augmented with objective measures of learning.
• Just because learners CAN RECITE FACTS AND TERMINOLOGY doesn’t mean they know what to do. Therefore, measuring knowledge recitation is an inadequate way of evaluating learning.
• Just because learners COMPREHEND A CONCEPT doesn’t mean they will be able to use that concept in a work situation. Therefore, measuring knowledge retention is an inadequate way of evaluating learning.
• Just because learners DEMONSTRATE COMPETENCY during a learning event doesn’t mean they’ll remember how to use the competency later. Therefore, measuring competency during or soon after a learning event is an inadequate way of evaluating learning.
• There are a NUMBER OF GOALS WE SHOULD HAVE as learning designers, including supporting our learners in building: comprehension, remembering, decision making competence, task competence, and perseverance in applying what they’ve learned to their job or other performance situations.
Understanding--------
Remembering
Understanding--------
Remembering
Understanding--------
Remembering
Work Performance
Performance In Learning
Compared to
Kirkpatrick-Katzell
Four-Level Model of Learning
EvaluationLevel 1 -- Reaction
Level 2 -- Learning
Level 3 -- Behavior
Level 4 -- Results
Two Ways to Use LTEM2018
2018
2019
1. Assessing Your Evaluations2. Learning Design & Development:
Working Backward from Your Goals
2020
Two Ways to Use LTEM1. Assessing Your Evaluations2. Learning Design & Development:
Working Backward from Your Goals
1. Sales Increase by 5%
2. Managers Coach Better
3. Simulated Coaching
4. Scenario Questions
5. IF-THEN Decisions
6. Perf-Focused Questions
Practice doesn’t make perfect:
Perfect practice makes perfect
Eric Barker
Author, “Barking up the wrong tree”
Danny Langdon & Kathleen Whiteside
Langdon
What’s the WAT I should be using?
Work Analytic Tools: Means to understanding,
analyzing, organizing, implementing and improving
performance
The Language of Work
1 32
Sample: Core Process model
Team first, then client, then growth
ISPI Conference Executive Buzz
How to Measure a Shift in CultureScott Weersing
Culture = “The way we do things around here”
1.Describe with adjectives (innovative, siloed,
collaborative, stressful….)
2.Break down to behaviors/outputs (encourage
risk-taking, reward individual achievement over
team….)
3. Identify metrics that can demonstrate
improvement
Steps to Shifting Culture
1.Define the desired culture
2.Connect to a strategy
3. Identify interventions processes that are already
in place
4.Look for metrics that already exist
5.Define additional data collection methods
6. Track change over time
ReSources
www.worklearning.com
www.debunker.club
Thallheimer: Performance Focused Smile Sheets, 2016
https://www.performanceinternational.com
Langdon & Whiteside: Language of Work trilogy, June, 2018
ISPI.org
https://www.td.org/atd-skill-tracker - talent development competencies self-
assessment
PMI.org – New 6th edition of PMBOK, 2017
Design Thinking isn’t new: Abraham Lincoln
used it when he built his Cabinet
ISPI Conference Executive Buzz
Help Us Be a More Dynamic Chapter!
Membership Experience
Communications
Frequency
Methods
Content
Strengthening the Society component of ISPI
Ground Rules
No idea is a bad idea
Be creative
Take risks
No criticism allowed
Brainstorming Instructions
1.Brainwrite or brainstorm
1.Discuss (or not) with your neighbors
2.Record every idea that you want to be
considered on a separate sticky note
3.Place your sticky note on the flip chart
2.Multivote
1.Use your 3 dots to vote on your favorite ideas
