Upload
phamliem
View
225
Download
0
Embed Size (px)
Citation preview
Large Scale, Blended Learning Development: Benchmark A Chapman Alliance, Research Study March 2013 By Bryan Chapman, Chief Learning Strategist [email protected] www.chapmanalliance.com
Abstract: In previous studies, we tackled development times and costs for isolated, learning-development tasks (ILT and eLearning). The purpose of this research is to analyze development of large-scale, often complex, blended learning courses with several different delivery formats bundled into a cohesive and unified course. We had 188 respondents who graciously shared information about their development practices including development times, costs, deliverables produced during the process, team structure and itemized information about each task in their work flow. Using these data we isolated a development ration of 49:1 (meaning that it takes 49 hours of development to create 1 finished hour of blended leaning (“seat time”). We also discovered that the per-finished-hour standard cost is $3,938 ($US Dollars). There are many variables that change the development ratio and cost that can be gleaned from the study with data you can use to benchmark your own practice. Now the good news! You are welcome to share this blended learning development research with anyone you’d like. Because we feel that this information is so important to the industry, we have released it under Creative Commons, meaning that you can share the whole deck or pull-out slides/charts and add them to your presentations, white papers, conference presentations, etc. The only restrictions are (1) you must ALWAYS list the source of information as “Chapman Alliance” and (2) you are NOT allowed to sell the results without written permission from Chapman Alliance. That’s it. Enjoy!
Source Citation: Chapman, B. (2013). Large Scale, Blended Learning Development: Benchmark [Research Study]. Published by Chapman Alliance LLC. www.chapmanalliance.com
Research Methodology How did we collect the data
The data contained in this research was collected by surveying 188 organizations involved in blended learning development. It was a daunting survey and we would like to thank those of you who participated, often sharing sensitive information to develop these metrics. The chart, above, shows the flow of the survey.
Demographics Dashboard Information about survey respondents and the blended courses represented in the results
Basic 188 – companies /organizations represented in the survey (courses) 7,025,269 – total # of employees across all companies/organizations 2,174,663 - # of learners reached by represented, blended courses
Located
83% - in the United States 17% - outside the US
By Industry
Information Technology/High Tech 12% Government/Military 10% Healthcare 9% Banking/Financial Services 9% Transportation/Automotive/Aviation 7% Professional Services 6% Energy 5% Industrial/Manufacturing 5% Higher Education 5% Pharmaceutical 5% Telecommunications 5% Educational Provider (sells learning content) 4% Other 3% Insurance 3% Learning Vendor 3% Association 3% Retail 2% General Business 2% NGO 2% K-12 1%
Learning Content Types
Job Specific/Technical Skills 39% Softskills (Leadership, Customer Service, etc.) 23% Computer Application Training 14% New Hire Orientation 10% Other 6% Product Knowledge & Sales 6% Compliance 2%
Other includes: Professional Licensure or Certificate, Mixture of Multiple Types, Academic Courses
Developed…
Internally 72% Outsource (External) 13% Hybrid (some of both) 15%
Duration of Blended Courses
53 hours (6.6 days) Average duration, blended courses in survey
24 hours (3 days) Median duration, blended courses in survey
45 minutes to 800 hours Range of blended courses represented in the survey
SMEs per course 4.3 Average number of
SME's used on blended learning projects
0 - 50 Range of the number of SME's listed on projects
7.3 Interesting fact: average # of SME's used on Software/Application blended Learning dev projects
How Long Does It Take to Create a Blended Learning Course “development time” per “finished hour” (seat time)
Across all 188 courses evaluated, the overall average development ratio came out at 49:1; in short, 49 hours of development for every 1 finished hour of final, blended course (“seat time”) Next, we’ll show you how this ratio changes based on specific variables…
per finished hour
So how long does it take to create a blended learning course? Because of the vast differences in course durations (ranging from 45 minute courses to 800 hours in seat time) we synthesized the information to one ratio….development time per “finished hour” (or seat time). Here’s what we found…
* $80.37 (US Dollars) estimated hourly rate * NOTE: survey respondents listed “person
hours” and overall cost. Displayed hourly rate is derived through calculation. In short, respondents did NOT provide this number. This is true for all “rates” shown throughout the summary.
How Long Does It Take to Create a Blended Learning Course Boil it down to “development time” per “finished hour” (seat time)
We cross-tabbed the data on outsource vs. internal vs. hybrid development. We also looked at how the development ratio and cost changes when developing “new” content vs. repurposing existing content.
per finished hour
per finished hour
Factor: Outsourced (Externally Developed)
- 18 + $1,158
per finished hour
Factor: Hybrid (mix of internal and outsource)
- 12 - $1,130
per finished hour
Factor: “New” Content (60% or more of course is “new” content)
+ 20 + 2,397
per finished hour
Factor: Repurposing Content (60% or more existing content)
- 27 - 2,217
How does “Length of Course” change development ratios? Course length does appear to have an effect on development times and cost
Another observation is that blended learning “course length” also changed the development ratio and cost. Advance to the next slide to see the results.
per finished hour
How does “Length of Course” change development ratios? Course length does appear to have an effect on development times and cost
Notice how significantly the development ratios and cost changes when creating smaller blended learning course (less than one day in length). We attribute this to economies of scale during development. .
per finished hour
per finished hour
Blended courses 2 to 3 days in length
- 7 - $743
per finished hour
Blended courses less the 1 day in length
+ 32 + $1,629
per finished hour
Blend courses 4 to 5 days in length
- 14 - $1,273
NOTE: the development ratio came out the same as 2 to 3 day courses in the study, with a slightly higher cost. This is not a typo
per finished hour
Blend courses 5 to 10 days in length
- 15 - $1,335
per finished hour
Blended courses longer than 10 days
- 16 - $1,631
Using Benchmark Data for your Projects Simple scenario, using core metrics to create high-level estimates
Setup: you’ve been tasked to create a 5-day, 40 hour, blended learning course
1. Multiply 40 (the number of hours in the course (“seat time”) by 49 (benchmark average, according to this study for blended learning course development)
2. The total is 1,960 “person hours” of overall development time, whether you are producing the course internally, outsourcing development, or using a hybrid approach with both internal and outsource development.
3. Next, apply cost. For default purposes, you are welcome to use the benchmark cost of $3,938 (average cost of development for 188 organizations per finished hour of course time). Multiply 40 (number of hours in course) by $3,938 (or any price adjustment that best matches your scenario).
4. Using benchmark data only, the estimated cost of development is $157,520 Remember, this is a benchmark average, meaning there are many factors that can change pricing including:
• Your choice to produce internally or outsource • New content vs. repurposed content • Development staff pay rate (NOTE: the US dollar amounts listed in this study is an average of
the 188 participants of this study. You’re rate of pay may have a significant impact on overall price)
• Deliverables included with the course (covered later in this study) • Development tasks that may be omitted depending on the overall blend (i.e. your blended
course doesn’t use eLearning) • etc.
Adjusted for
100K or more employees
How Long Does It Take to Create a Blended Learning Course “development time” per “finished hour” (seat time)
Across all 188 courses evaluated, the overall average development ratio came out at 49:1; in short, 49 hours of development for every 1 finished hour of final, blended course (“seat time”) Next, we’ll show you how this ratio changes based on specific variables…
per finished hour
So how long does it take to create a blended learning course? Because of the vast differences in course durations (ranging from 45 minute courses to 800 hours in seat time) we synthesized the information to one ratio….development time per “finished hour” (or seat time). Here’s what we found…
* $76.20 (US Dollars) estimated hourly rate
* NOTE: survey respondents listed “person hours” and overall cost. Displayed hourly rate is derived through calculation. In short, respondents did NOT provide this number. This is true for all “rates” shown throughout the summary.
How Long Does It Take to Create a Blended Learning Course “development time” per “finished hour” (seat time) Filtered for respondents with 100K or more employees
All Respondents 100K or more employees Time Cost cost per hour Time Cost cost per hour
Overall 49 $ 3,938 $ 80.37 54 $ 4,115 $ 76.20
Outsourced 31 $ 5,096 $ 164.39 50 $ 5,825 $ 116.50
Hybrid 37 $ 2,808 $ 75.89 41 $ 3,116 $ 76.00
New Content 69 $ 4,992 $ 72.35 70 $ 5,291 $ 75.59
Repurposing Content 22 $ 1,721 $ 78.23 23 $ 1,979 $ 86.04
1 day or less 81 $ 5,567 $ 68.73 85 $ 5,811 $ 68.36
2 to 3 days 42 $ 3,246 $ 77.29 43 $ 3,505 $ 81.51
4 to 5 days 35 $ 2,685 $ 76.71 39 $ 2,841 $ 72.85
6 to 10 days 34 $ 2,603 $ 76.56 34 $ 2,748 $ 80.82
10 days + 33 $ 2,307 $ 69.91 31 $ 2,459 $ 79.32
Adjusted for
100K or more employees
Budget Allocation How budget amounts change based on internal development, outsourcing, or using a hybrid approach
Internal Development
Outsourced Development
Hybrid Development
Average across all respondents
Internal Labor Cost = 87% Materials = 7% Outsourced Services = 4% Other = 2%
Internal Labor Cost = 9% Materials = 3% Outsourced Services = 88% Other = 0%
Internal Labor Cost = 48% Materials = 6% Outsourced Services = 46% Other = 0%
Internal Labor Cost = 71% Materials = 6% Outsourced Services = 21% Other = 1%
Lots of Different Kinds of Blends Observation: The study picked up lots of different blends. No two blends are the same. See random sample of blends from the actual survey below (actual listings by some respondents):
[add text here]
ILT
(face to face)
Virtual
Classroom
eLearning Hands-on Lab Mobile on
Smartphone
Mobile on
Tablet
Social
Collaboration
Informal
Learning
On-the-Job
(OJT)
Coaching
/Mentoring
Other
Respondent 1 80% 20%
Respondent 2 10% 5% 10% 5% 5% 60% 5%
Respondent 3 70% 10% 10% 5% 5%
Respondent 4 60% 25% 5% 5%
Respondent 5 15% 15% 60% 10%
Respondent 6 5% 85% 10%
Respondent 7 40% 20% 20% 20%
Respondent 8 60% 15% 5% 15% 5%
Respondent 9 75% 25%
Respondent 10 15% 65% 20%
Respondent 11 30% 5% 5% 5% 10% 15% 10% 10% 10%
Respondent 12 40% 30% 10% 10% 10%
Respondent 13 30% 10% 30% 15% 15%
Respondent 14 20% 60% 20%
Respondent 15 33% 67%
Respondent 16 45% 20% 20% 5% 5% 5%
Respondent 17 70% 20% 10%
Respondent 18 40% 40% 20%
Respondent 19 50% 20% 30%
Respondent 20 80% 10% 10%
Respondent 21 90% 5% 5%
Respondent 22 25% 5% 35% 5% 5% 20% 5%
Respondent 23 50% 50%
Respondent 24 40% 20% 20% 10% 10%
Blended Learning “Deliverables” Most common deliverables in blended learning development (sorted by most frequently used)
End-of-Course Evaluations 92% Instructor Guide (s) 84% Knowledge Test(s) (online or paper-based) 83% PowerPoint Deck(s )for Visuals 79% Student/Participant Guide(s) 78% Self-Paced eLearning (s) 68% Handouts or in class Activity Workbook(s) 62% Video (s) 54% Job Aide(s) 50% Virtual Classroom Material(s) 44% Reference Manual/Text Book (s) 41% Group Participation e-learning(s) 40% Pre-Reading (s) 32% Other 24% Poster(s) 18% Wiki(s) 13% Podcast(s) 12% Blog(s) 8%
Common Answers for “Other” includes: Simulations, Skill Assessments (more than just a knowledge test), Games, Virtual Labs and Lab Guides, Participatory Exercises, Certification Tests, Webinar, SharePoint Discussion Board, Social Learning Site, and Procedural Checklists)
Delivery Formats What modalities are used most often to deliver blended learning courses
In Class (face-to-face) 87% eLearning (self paced) 87% Virtual Classroom 58% Social Collaborative Learning 49% Coaching/Mentoring 49% Assigned On-the-Job Training (OJT) 42% Informal Learning & Knowledgebases 37% Mobile on Tablet 31% Mobile on Smartphone 28% Other 23%
Common Answers for “Other” includes: Self-Paced Video Training, Web Portal Resources, Teleconferences, Skype
Itemized Development Metrics
The next several slides provide detailed summaries of time spent across 30 developmental tasks through the ADDIE process, when creating large-scale blended learning courses. You can use this
information to further adjust the variables when calculating expected development times and costs (i.e. if eLearning is not part of your blend, remove the total hours for eLearning development
from your projections).
Analysis Phase – Development Tasks “Person Hours” spent in each task, organized by course length
Blended Learning Course Duration
Scoping and Project
Initiation
Kick Off Meetings
Front End Analysis and
Discovery Other
One day or less 23 11 37 12 2 - 3 days 39 12 67 12 4 - 5 days 48 17 107 14 6 - 10 days 52 27 134 20 Longer than 10 days 72 41 153 24
Total
83 person hours
130 person hours
186 person hours
233 person hours
290 person hours
* All times shown above are “person hours” spent in each task
Blended Learning Course Duration
Drafting Learning
Objectives
Course Outline
Content Analysis &
Data Mapping
Create Detailed Design
Document
Create Design
Protoype(s) Other
One day or less 16 10 22 69 34 5 2 - 3 days 17 19 35 80 53 7 4 - 5 days 22 23 61 109 117 7 6 - 10 days 36 38 72 129 139 10 Longer than 10 days 57 59 96 176 177 54
Design Phase – Development Tasks “Person Hours” spent in each task during the design phase, organized by course length
Total
156 person hours
211 person hours
339 person hours
424 person hours
619 person hours
* All times shown above are “person hours” spent in each task
Develop Phase – Development Tasks “Person Hours” spent in each task during the develop phase, organized by course length
Total
524 person hours
699 person hours
1228 person hours
2151 person hours
2956 person hours
Blended Learning Course Duration
Create Instructor Guide(s)
Create Student Guide(s)
Create PowerPoint (classroom
Visuals)
Create Handout(s) or In-Class
Activity Workbook
Create job Aide(s)
Create Pre-Reading(s)
Create Self-Paced
eLearning
eLearning Modules
for in-class exercises
Create Video
Create Knowledge
Test(s)
Create Course
Evaluation
Create Other Course
Materials
One day or less 41 38 39 16 23 16 165 76 50 33 15 12 2 - 3 days 63 53 57 18 28 17 204 96 100 35 14 14 4 - 5 days 172 103 79 37 57 18 409 102 140 70 16 25 6 - 10 days 295 109 181 51 75 30 1021 112 147 78 15 37 Longer than 10 days 482 170 232 86 101 36 1312 137 167 139 23 71
* All times shown above are “person hours” spent in each task
Implementation & Evaluation Phases – Development Tasks “Person Hours” spent in each task during implementation and Evaluation, organized by course length
Total
221 person hours
257 person hours
404 person hours
538 person hours
894 person hours
Blended Learning Course Duration
Set Up Virtual
Delivey of Learning
Publish Courses to the LMS
SME Reviews
Team Meetings
Course Walkthrough
Pilot Testing
Revision of Course Materials
Final Release
Debrief /After Action Report
One day or less 23 11 27 30 18 72 20 10 10 2 - 3 days 28 17 31 40 19 74 25 11 12 4 - 5 days 40 18 66 81 24 88 56 11 20 6 - 10 days 41 34 100 84 32 109 100 14 24 Longer than 10 days 58 26 151 130 76 267 116 39 31
* All times shown above are “person hours” spent in each task
Summary – Total Hours of Development This chart includes totals for the entire ADDIE process (sorted by course duration)
Blended Learning Course Duration Analysis Phase Design Phase Develop Phase Implement & Evaluate Phase
One day or less 83 156 524 221
2 - 3 days 130 211 699 257
4 - 5 days 186 339 1228 404
6 - 10 days 233 424 2151 538
Longer than 10 days 290 619 2956 894
Total Development
Time
984 person hours
1297 person hours
2157 person hours
3346 person hours
4759 person hours
Blended Learning Course Duration% % Analysis % Design % Develop % Implement &
Evaluate
One day or less 8% 16% 53% 22% 2 - 3 days 10% 16% 54% 20% 4 - 5 days 9% 16% 57% 19% 6-10 days 7% 13% 64% 16% Longer than 10 days 6% 13% 62% 19%
* All times shown above are “person hours” spent in each task
Development Time by Role (“person hours”) Use this information to benchmark how teams are structured for blended learning development
Project Manager(s) 13%
Lead Instructional Designer(s) 15%
Instructional Designer(s) 19%
Writer(s) 6%
Graphic Artist(s) 6%
Programming/Authoring Specialist(s) 14%
Audio/Video 3%
Quality Assurance 6%
Subject Matter Expert(s) 11%
Other 7%
Summary of Findings Key takeaways from this research
• On average, the development ratio for blended learning is 49:1, meaning it takes 49 hours to create one finished hour of blended learning (on average)
• Using the data from 188 survey respondents, the average price per finished hour of blended learning is $3,938
• Creating “new “content can increase your development ratio to 69:1
• Outsourcing content takes less time (31:1), but the cost per finished hour goes up by $1,158 per finished hour
• Shorter blended courses (less than 1 day in duration) increase the development ratio average (81:1)
• The median duration of a blended learning course is 3-days in length (24 hours), although they range in the survey from 45 minutes to 800 hours (100 days)
• There is NO STANDARD BLEND. There were literally 188 different blends, using a variety of delivery formats and integrating new, emerging technologies (such as Social, Mobile, and Informal learning)
About the Author Bryan Chapman is Chief Learning Strategist at Chapman Alliance; a provider of research-centric consulting solutions that assist organizations in defining, operating and optimizing their strategic learning initiatives. As a veteran in the industry, he has over 20 years experience and has worked with such organizations as American Express, Shell, Kodak, Sprint, Sharp Electronics, Honda, IBM, Microsoft, Avon, UNICEF, American Red Cross, The Food and Drug Administration, U.S. State Department, and many others; to help them optimize learning efficiency through the use of innovative learning techniques and technologies. Click here for a full bio. Please feel free to contact me if you need some assistance…
• working through learning management system (LMS) or any other learning technology selection project
• finding and applying learning industry research; custom research projects
• optimizing or establishing your learning governance approach
• exploring best practices for rapid learning development
• learning strategy “jam sessions” (available as a 2-hour block, full-day onsite, or as a workshop)
• Click here for more available services Contact information: Bryan Chapman Chief Learning Strategist Chapman Alliance (801) 568-7011 [email protected] www.chapmanalliance.com To participate in future research studies, Click here (add you name to our mailing list)