29
3 Challenges in Moving Your Distance Educaon Online and How to Solve Them A Special Report for Colleges and Universies

and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

1

3 Challenges in Moving Your Distance Education Onlineand How to Solve Them

A Special Report for Colleges and Universities

Page 2: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

2

CONTENTS

Overview

Challenge #1: Reducing costs to deliver courses

Challenge #2: Achieving engagement and success

Challenge#3:Selectingtherighttechnology

Technology:OpenSourceTeachingPlatforms

Technology:TraditionalLMS

Technology:Cloud-basedLMS

AboutWizIQ

AppendixA:WizIQLMSFeatures

Sources

©2016byWizIQInc.Allrightsreserved.Formoreinformation,[email protected]

3

4

7

12

16

19

22

24

25

27

Page 3: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

3

OVERVIEWAs colleges and universities are squeezed by rising costs, tuition fees continue to climb, student debt mounts, and more express doubts about the value of a college education.

Many academics believe it’s time to look at improving “educational productivity.” This means doing more with less, just like every other sector of the global economy. That doesn’t have to be a threat; it can be an opportunity.

“Productivity is a ratio between costs and outcomes,” concluded a seminal report from the U.S. Department of Education,”1 noting three ways to improve productivity:

• Reducing costs, while maintaining learning outcomes• Improving outcomes, while holding costs steady• Transforming processes to reduce costs and improve outcomes

To help boost productivity, many colleges and universities are turning to online education. In fact, 70% of Chief Academic Officers in the U.S. agree that online education is critical to their long-term strategy.2

Yet the answer is not simply to roll out online courses; the real solution is to do this as effectively as possible. Three main challenges face any college or university that considers moving courses online:

1. Reducing the cost to deliver courses2. Achieving learner engagement and success3. Selecting the right technology

Moving online may require a rethink of your campus legacy IT systems. There may be a better option than soldiering on with the status quo.

Moving courses online involves risky decisions, including how to manage course development, budget for added IT, set up an online payment system, create a mobile app, and launch promotions to attract new students.

To support online courses, colleges and universities generally choose between three categories of educational technology:

• Open source learning platforms, such as Moodle• Traditional learning management systems (LMS), such as Blackboard• Cloud-based LMS, such as WizIQ LMS

This white paper describes each of these categories, and how well it meets the challenges of moving online. This paper concludes that the best way to support online initiatives is with a cost-effective, integrated e-learning platform such as WizIQ LMS.

1.2.3.

Page 4: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

4

CHALLENGE #1: Reducing costs to deliver coursesThe first challenge for colleges and universities is to make sure that going online is a more cost-effective way to deliver courses. It’s an old challenge: How to do more with less. Researchers say that savings are possible, but not guaranteed. And they point to several promising pathways to reduce costs.

Student debt: a $1-trillion ticking time bomb

As you know, delivering traditional courses get more expensive every year, as costs for everything from salaries to snow removal creep up and up. Those rising costs continue to push up U.S. tuition fees, as shown in Figure 1.

Even the most affordable option, a public college in a student’s home state, has tripled since 1995—far outstripping inflation over that period.3

The result: The average U.S. graduate is saddled with $28,950 in student debt.4 Anyone who attended an Ivy League school or trained for certain professions carries much more. For example, dental school grads average $200,000 in debts.5

In fact, unpaid student loans were estimated at $1.3 trillion at the end of 2015,6 a situation that’s been called “America’s ticking time bomb.”7 Some worry that this indebtedness will spark a new economic crisis, with recent grads unable to afford cars or homes.

No wonder many students are asking, “Is going to college really worth it?”

And a recent survey by Kaplan Test Prep and Money magazine revealed that 4 out of 5 parents are asking the same question.8

Moving selected courses online promises many savings

Today's college and university administrators must find ways to contain spiraling costs and at the same time boost educational productivity.

Moving to online delivery for some courses holds a lot of promise.

For example, in an extensive literature survey, researchers with the U.S. Department of Education found nine possible pathways to savings with online courses.9

Four of the most promising are discussed below.

$40K

$25K

$30K

$35K

$20K

$15K

$10K

$5K

$0K1995 2000 2005 2010 2015

Average private tuition and fees

Average public out-of-state tuition and fees

Average public in-state tuition and fees

Figure 1: Rising U.S. Tuition Fees, 1995 through 2015

Source: U.S. News and World Report, 2015

Page 5: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

5

Increasing the rate of student learning

Online learning can actually speed up a student’s progress.

For example, one study found college students learned statistics about 50% faster online than in large lecture courses.10 Another found military training accelerated about 30% with online materials.11

Students in online courses may absorb material faster in their preferred time and space, rather than at a prescribed hour in an impersonal lecture hall. And of course, they save many hours commuting.

Since “time is money” the faster a student learns, the fewer academic resources they consume, and the sooner they graduate.

Making more effective use of teacher time

Online courses reduce or eliminate commuting, potentially freeing four hours or more a week for American college teachers.12 And online teaching can help instructors focus on adding more value.

For example, some routine activities such as distributing course materials and marking quizzes can be done by computers. And junior academics such as TAs or preceptors can sometimes substitute for senior academics.

All these measures can boost productivity measurably.

For a real-world example, Florida Gulf Coast University made a required course in arts appreciation 100% online. Over two years, this reduced the cost-per-student from $132 to $70.13 And testing showed that students who took the online course scored better in related knowledge and critical thinking.14

Reducing facilities costs

Shrinking the number of on-campus students can trim costs for classrooms, residences, and parking. For example, the University of British Columbia estimated that converting four large lecture-style courses to online delivery would reduce the number of classrooms needed at peak hours by 15%.15

Savings can also be generated on campus services like cleaning and printing, and on utilities like lighting and HVAC. But be warned: These savings can be offset by higher IT costs for on-premises hardware, software, and connectivity to support online courses.

Realizing economics of scale

The traditional model of in-person lectures and tutorials does not scale. The maximum throughput of students is limited by time, space, and personnel.

But once an online course is developed, lectures recorded, and discussion groups set up, these digital assets can be reused indefinitely to reach more students. And online distribution of assignments, marks, and comments is faster and less costly.

Page 6: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

6

This enables colleges and universities that move online to benefit from economies of scale that were impossible with the traditional model.

No simple answer, many clear hints

Can online courses save money for colleges and universities? There’s no simple answer. It all depends on the existing cost structures, the scope of the transition to online, and even the technology platform used.

But many researchers give online courses a thumbs-up.

For example, an extensive literature survey in 2013 concluded that, “Online learning can, under the right circumstances, help contain some post-secondary education costs without compromising quality.”16

As shown in Figure 2, at least six separate studies across 19 U.S. states17-22 have concluded that moving to online education saves money in higher education. The per-student savings ranged from 17% to 56% with an average of 37%.

And finally, the National Center for Academic Transformation (NCAT) aims to help U.S. post-secondary institutions improve learning outcomes while reducing costs. NCAT does this by redesigning large introductory courses to include more online components.

NCAT’s first course redesign saved an average of 37% across 30 institutions, totaling about $3 million a year. After completing 156 projects, NCAT redesigns have saved colleges an average of 34% on instructional costs.23

Though there’s no guarantee, moving courses online will likely improve your institution’s educational productivity and save money.

WisconsinStuiber et al

2010

$12,000

47%Drop

56%Drop

53%Drop

17%Drop

28%Drop

19%Drop

$10,000

$8,000

$6,000

$4,000

$2,000

$0

UtahSloan and Mackey

2009

Survey of 14 StatesCavanaugh

2009

FloridaTax Watch

2007

OhioWatson

2007

ColoradoWatson

2004

Figure 2: Per-Student Spending on Traditional and Online Courses

Source: Research cited by U.S. Department of Education, 2012

Face-To-Face Online

Page 7: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

7

CHALLENGE #2: Achieving engagement and success The second major challenge for any educator is how to boost engagement and learning outcomes with an online format. Some best practices here: Consider your students first, then pick an engaging format. Choose blended learning over self-paced or MOOCs. And always use analytics to measure, correct, and confirm the learning outcomes.

Picking the right format

Faced with moving courses online, too many educators start by trying to decide which format to use for their courses.

This can easily trigger an internal debate on the merits of different formats. Self-paced or live instructor-led? Synchronous or asynchronous? Totally online or blended?

In fact, the U.S. Department of Education created a conceptual framework that defines 12 different types of online education.24 As shown in Figure 3, this framework is based on a complex matrix of learning experiences, communications (synchronous or asynchronous), and objectives (to replace or enhance face-to-face interactions).

The details do not concern us here; the point is, there are categories of online learning. With all these possibilities, how can any educator pick the best one?

Figure 3: Conceptual Framework with 12 Types of Online Learning

Learning Experiences

Expository:devices transmit knowledge

Active:learners explore using digital artefacts

Interactive:learners collaborate using digital artefacts

Communications

Synchronous Type 1 Type 2

Type 6

Type 10

Type 5

Type 9

Synchronous

Synchronous

Asynchronous Type 3 Type 4

Type 8

Type 12

Type 7

Type 11

Asynchronous

Asynchronous

ReplaceFace-to-Face

EnhanceFace-To-Face

Source: U.S. Department of Education, 2010

Page 8: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

8

Today’s Millennial students are different

Start planning with the format of your online course. Instead, start with your intended students. Isn’t this what student-centered education is all about?

Most of today’s students are Millennials or so-called “digital natives” born between 1980 and the early 2000s, now aged 15 through 35.

While generalizations never encompass everyone, many Millennials share these basic characteristics:

• Grew up with the web and mobile devices• Need instant gratification, no tolerance for delays• Require ongoing feedback• Demand flexibility, like to “time and place-shift” activities• Spent years collaborating with peers in daycare, school, sports, and games

In terms of education, Millennials still want face-to-face interaction with teachers, but they also expect the speed, convenience, and flexibility of digital resources. And they would prefer the freedom to choose whichever suits them in the moment.25

In other words, today’s students need mobile learning, instant results, lots of interaction and feedback, the chance to work with peers, and a flexible learning experience where they don’t always have to show up at a certain time and place.

Eliminating less engaging formats

Since your students have changed, your courses must evolve to stay relevant and engaging. To begin, you must discard any approaches that don’t meet the expectations of today’s students.

For example, one format that higher educators can discount is self-paced online learning. A classic example of this approach is asking students to step through a slide deck while they listen to a voice-over.

This is certainly affordable to produce, and it might suffice for a company “coursinar” attended by captive employees.

But with no real-time interaction, no collaboration with peers, and no human feedback of any kind, many of today’s students will consider that format boring. So why pursue it?

The fatal flaw in even the best MOOCs

One format that attracts a lot of interest is the Massive Open Online Course (MOOC).

As you may know, edX is a non-profit founded by Harvard University and MIT in 2012 to develop MOOCs. Although the edX courses are among the most prestigious MOOCs available, they still have many flaws.

Page 9: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

9

“EdX does not offer the credentialing, networking, or communal experience of a traditional university,” says a review in PC Magazine by a university instructor and administrator. “Students have little contact with their professors, TAs, or colleagues.” And he found the online forums so old-fashioned, they reminded him of a bulletin board from the 1990s.27

Despite all this, do MOOCs work? Let’s look at edX’s own data.

In 2013, edX says 476,532 students took courses on anything from “Challenges of Global Poverty” to “Intro to Computer Science.” As shown in Figure 4, completion rates were abysmal, with a high of 7.5% and a median of only 3.7%.28

MOOCs sound exciting, but they suffer from a critical issue: low interaction, leading to low engagement, leading to low completion.

“While their scale is unprecedented, the underlying way that MOOCs are taught isn’t really new,” says a special report for Time magazine. “It’s very, very old—a system in which professors lecture to huge numbers of students with whom they seldom, if ever, interact.”29

There’s a simple reason for that. No institution can afford to let professors spend weeks interacting with students who pay no tuition. That’s why MOOCs are typically self-paced, based on videos of lectures and self-assessed quizzes. That’s just not enough for today’s demanding students.

0%

Challenges of Global Poverty

Health in Numbers

Elements of Structures

Justice

Intro to Biology

Intro to Solid State Chemistry

Circuits and Electronics

Mechanics

Intro to C5/Programming

Electricity and Magnetism

Human Health & Environment

The Ancient Greek Hero

Intro to Computer Science 0.8%

1.3%

1.8%

2.6%

3.0%

3.1%

3.7%

3.8%

3.9%

4.1%

4.4%

4.4%

7.5%

1% 2% 3% 4% 5% 6% 7%

Figure 4: Completion Rates of Harvard/MIT MOOCs, 2013

Source: EdX dataset, processed by Max Woolf

MIT courses

Harvard courses

Page 10: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

10

Blended learning: a viable approach

Blended learning combines elements of online teaching with more traditional face-to-face interaction to supplement but not replace interactions with the teacher.

This provides the best of both worlds: the flexibility of digital resources, plus the strengths of instructor-led classes.

“Blended learning acknowledges that we now live in a digital age,” says the website for the Macmillan English Campus. “People are now used to receiving a continuous stream of digital information at a fast pace… so why should the classroom be any different?”30

Today’s educators generally agree that blended learning achieves better learning outcomes than either classroom teaching or 100% online.

The U.S. Department of Education confirmed the power of blended instruction after sponsoring a meta-analysis of 176 studies into online education published between 1996 and 2008. (Another 346 studies were excluded for various reasons, including lack of statistics or rigor.)

“Students in online conditions performed modestly better, on average, than those learning the same material through traditional face-to-face instruction,” said the often-quoted findings, and “instruction combining online and face-to-face elements had a larger advantage.”31

By retaining some face-to-face interaction, blended learning helps teachers and students maintain a real-world connection, which builds engagement. So blended learning is the most viable approach to ensure positive learning outcomes.

Measure, correct, and improve with analytics

Of course, the best way to discover and track learning outcomes is to measure them. For online courses, this requires powerful analytics built into your e-learning platform.

Analytics that provide insights into student behavior make it easier to improve course curriculums and instructor effectiveness, boosting student engagement and learning outcomes.

Big data + analytics = better learning outcomes

Big data is a huge trend in IT that can be immensely useful in education.

“Higher education institutions, for the most part, are collecting more data than ever before,” found the EDUCAUSE association in a recent survey. “Many colleges and universities have demonstrated that analytics can help significantly advance an institution in such strategic areas as resource allocation [and] student success.”32

The ability to measure what’s working and what’s not in teaching and learning, coupled with the power to personalize instruction for individual learners, has been a major advance in education technology.

Page 11: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

11

Easy-to-view dashboards

Analytics are best viewed through an intuitive dashboard. An effective dashboard displays at-a-glance overviews or detailed reports on matters such as attendance, content consumption, heat maps, test scores, comparative score details, and course activity.

How long are learners viewing materials? Which content is being skipped?

By seeing what works and what does not, instructors can make refinements to improve the overall quality and impact of online courses.

Improving instructor effectiveness

Analytics give your instructors the evidence they need to help refine online courses and content to boost engagement. By tracking learner performance and progress, instructors can assess areas for improvement and increase the effectiveness of their pedagogy.

Knowing what’s working and what’s not for students, instructors can do quick course corrections. They can apply teaching tactics such as slowing down, speeding up, taking more questions, providing more written comments, doing more reviews, and so on.

With the right analytics, instructors can zero in on any learners who are struggling, and give them the individual attention they may need. This is vital in online learning, where an instructor may not have daily personal contact with each student. With insights into student behavior, instructors can intervene as appropriate to help ensure learner success.

Analytics make it easier for instructors

to improve, intervene as needed, and

boost learning outcomes

Page 12: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

12

CHALLENGE #3: Selecting the right technologyA final challenge faces all educators moving to online learning: Choosing the best technology to support your e-learning initiatives. This technology must support online teaching that is both affordable and pedagogically effective. Your legacy IT may not provide all the features you need, so look carefully at your options in three categories.

Some questions to ask

This challenge raises many questions, such as:

• What features do you absolutely need?• What unnecessary bells and whistles can you do without?• How can you ensure the technology delivers a seamless learner experience,

anywhere, anytime?• Should you consider building your own system, despite the high risks?• Should you carry on with your legacy IT regardless of its limitations?• Or should you choose a new system from the vast array on the market?• If so, how do you know which is the best solution for your needs?• And how cost-effective is each category?

These are complex questions. The answers depend to some extent on the specific needs of your institution.

Rethinking your legacy IT

When you consider moving courses online, your campus legacy IT may not be appropriate. After all, educators should focus on what you do best, not struggle with old-fashioned or limited technology that no longer suits your needs.

Instead of maintaining the status quo and using whatever your IT department selected years ago, you can redefine what you need for online success today.

Here are 10 key features that many educators moving to online education look for:

Rapid content authoring Virtual classroom Powerful test and assessment framework Flexible reports and analytics Support for mobile learning Independent branding Built-in monetization Quick and cost-effective deployment that works smoothly with existing ITScalability to support an unlimited number of studentsRelated services available

To be effective, the technology you use should provide as many of these key features as possible.

1.2.3.4.5.6. 7. 8. 9.

10.

Page 13: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

13

Three main categories of educational technology

In general, educators looking to support online learning choose from three main categories of educational technology:

• Open source learning platforms, such as Moodle• Traditional learning management systems (LMS), such as Blackboard• Cloud-based LMS, such as WizIQ LMS

The following pages describe each of these categories, highlighting how well they meet the key challenges in moving your courses online. Figure 5 provides a quick summary, and the following sections discuss each category in more detail.

Page 14: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

14

Category Definition, ExamplesChallenge #1:

Reducing costsChallenge #2:

Engagement and successChallenge #3:

The right technology

1. Open source learning platforms Free software distributed with no license fees. Organizations can configure the software as they wish. Third-party consulting firms support very popular open source software for a fee.

Example: Moodle

Strengths: Low initial cost

Drawbacks:• Total cost of ownership can

exceed commercial software• Ongoing IT support burden

Strengths: With sufficient budget, can build an engaging system

Drawbacks:• Success depends on budget

and coder expertise• Some features may not be

implemented due to costs• Poor usability due to outdated

software design. For example,frequent switching betweenscreens, modules, and

Strengths: • Some (not all) systems have

large community for support• Wide-open flexibility, you can

design as you wish

Drawbacks: • Massive learning curve• High costs to install, configure,

integrate, debug, document,and train

• Many features missing, suchas live instructor-led learning,quick content authoring, andpayment handling

• Some features can be boughtfrom third parties

• Total cost of ownership canexceed commercial software

• Ongoing IT support burden

2. Traditional LMS(Learning Management Systems)

Commercial software intended to handle every aspect of the learning process, from registering students to delivering content to tracking completion rates.

Example: Blackboard

Strengths: Industry standard, so many job applicants have Blackboard experience

Drawbacks:• Very costly, with high capex

for on-premise hardware, software, and networking

• Annual maintenance fees of20% or more

• Ongoing IT support burden

Strengths: Includes many tools for feedback, marking, and reporting

Drawbacks:Designed for CIOs, not educators

Strengths: • Some (not all) features now

available in cloud versions• Can scale up for popular online

courses

Drawbacks:• Very costly, with high capex

for on-premise hardware,software, and networking

• Ongoing IT support burden• Many non-essential• features included• Many bolt-ons added over

time yielding clunky learnerexperience

Figure 5: Main Categories of Educational Technology

functions can break thelearner experience

Page 15: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

15

Category Definition, ExamplesChallenge #1:

Reducing costsChallenge #2:

Engagement and successChallenge #3:

The right technology

3. Cloud-based LMS(Learning Management Systems)

Next-generation LMS running in the cloud. Provides all key features and benefits of traditional LMS at lower cost, and with smaller IT footprint.

Example: WizIQ LMS

Strengths: • No on-premise hardware or

software lowers capex• Funded from opex budgets• Lighter IT support burden,

with no development effort• No need to pay for extra bells

and whistles

Strengths: • Includes virtual classroom for

instructor-led learning• Supports mobile devices for

anywhere, anytime access• Learner analytics offer insights

into learner behavior leadingto better courseware andpedagogy

Strengths: • Quick course authoring• Easy to learn, simple to use• Lighter IT support burden,

with no development or setup• Supports mobile learning• Provides payment handling• White-labeling enables self-

branding of your academy• Learner analytics for deeper

actionable insights• Built-in test and assessment

features result in betterlearner performance trackingand outcomes

Page 16: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

16

TECHNOLOGY: Open Source Teaching PlatformsAs you know, “open source” is software in the public domain delivered with no license cost, but with no vendor to rely on for support or updates.

There is only one real benefit of using open source: a lower initial cost. Not paying a licensing fee or an incremental fee for every student who takes a course can seem like big savings.

But these savings are easily outweighed by the investment needed to build a functional e-learning system from the raw materials of open source.

Software projects are very risky

How risky is it to set up an open source teaching platform?

Consider the findings of The Standish Group, which has studied software projects since 1994. Their 2015 report analyzed more than 50,000 projects “ranging from tiny enhancements to massive systems re-engineering implementations.”33

The results were sobering: Only 29% of all projects came in on-time and on-budget with satisfactory results. All the other projects—more than 7 out of 10—were over-budget, over-schedule, or unsatisfactory.

As shown in Figure 6, this success rate hasn’t changed much in the past five years. Are those odds worth the risk to your institution, and your career?

About Moodle

The most popular open source teaching platform is Moodle, which claims “to provide educators, administrators, and learners with a single robust, secure, and integrated system to create personalized learning environments.”34

Among its users, Moodle counts Microsoft, Shell, and the UK’s Open University. For a major company like Shell that can budget a vast sum to implement Moodle, it can become an effective, full-featured teaching platform.

But for any college struggling to find resources, Moodle can be a black hole that consumes time and money but never delivers any satisfactory results.

2011 2012 2013 2014 2015Successful 29% 27% 31% 28% 29%Challenged 49% 56% 50% 55% 52%Failed 22% 17% 19% 17% 19%

Figure 6: Software Project Success Rates

Source: The Standish Group, 2015

Page 17: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

17

Challenge #1: Reducing costs

“Think of commercial software as a house,” suggested one software executive in Forbes, “and open source software as everything you need to build a house—raw lumber, nails, sheet rock, windows, plumbing fixtures, and the rest.

“You can spend your money and buy the house, or you can spend your time and build the house. Either way, you pay for your house.”35

In fact, the Total Cost of Ownership (TCO) for open source products can exceed the cost of commercial software.

TCO is a metric developed by consulting firm Gartner to measure all the costs for “software acquisition, management, and support, communications, end-user expenses and the opportunity cost of downtime, training, and other productivity losses.”36

By this more holistic measure, open source is much less of a bargain.

In head-to-head IT purchases for real-world users—not propeller-heads—open source seldom triumphs. For example, IT journal NetworkWorld says, “every time [UK] government people compare open source and Microsoft products, Microsoft products always come out cheaper in the long run.”37

And with open source, there are only three places to get support:

• From open source community volunteers• From your in-house IT team, who are already over-burdened• From third-party consultants

The last two options cost money. But can you really entrust the mission-critical system for delivering your online courses to volunteers scattered around the world?

Challenge #2: Building engagement and success

So, with enough money, you can build an engaging e-learning platform from open source. But your success will depend completely on your budget, and on the skills of your developers.

And some features may never be implemented, as found by The Standish Group.

Even if you get Moodle running smoothly, it provides what some educators call a “broken learner experience.”

Unlike commercial systems that give learners and instructors a single helpful dashboard, Moodle requires switching between different modules, functions, and screens to see essential items like schedules, content, and marks. This wastes time, creates confusion, and ruins the user experience.

Page 18: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

18

Challenge #3: Selecting the right technology

Not all open source projects attract a wide base of users, along with programmers willing to contribute. Fortunately, Moodle has a large community of support.

And it offers wide-open flexibility, so that any institution can design a learning platform however they wish.

But when you download open source, you get a bare shell that takes a massive learning curve to integrate into a full-featured system. Very few educators have the skills or the spare time to fiddle with open source code.

Instead, you must rely on your over-booked IT team or hire outside consultants to install, configure, integrate, debug, document, and train teachers how to use the system.

All this costs a significant amount and takes a long time. After all, there’s a lot to be done. Here is a short list of vital features missing from Moodle:

• Virtual classroom for live, instructor-led learning• Quick content authoring tool• Very limited features for mobile learning• Payment handling with a gateway and admin module• White-labeled apps ready to accept your branding• Player for video streaming (plug-in required)• Assignments module (plug-in required)

All this explains why educators asked to comment on Moodle call it “ugly,” “clunky,” “dated,” “bloated,” “difficult to learn,” and “quite the challenge to implement.” They say it has a “horrible interface” and “tortuous learning curve,” and that “training is hard to come by.”

In short, says one instructor who tried it, “Moodle is awful!”38

Page 19: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

19

TECHNOLOGY: Traditional LMSThe term Learning Management System (LMS) is often used, but seldom defined.

Two clear-thinking educators define LMS as “the framework that handles all aspects of the learning process.” This includes delivering content, assessing learning goals, tracking student progress, handling registration, doing administration, and producing reports.39

That’s a big job. And the biggest player in the field is Blackboard, the largest of all traditional LMS vendors.

About Blackboard Inc.

Founded in 1997, Blackboard offers a complex set of products and services, many of them acquired from other companies and “bolted on” to the system.

Despite being sold to many IT managers and installed at many schools, not everyone appreciates Blackboard. Some say it was innovative a decade ago, but has since fallen behind.

Too many bolt-ons give a clunky feel

“Blackboard knows it has an image problem—partly because its armada of tools and features developed over the years did not play well with one another,” notes one ed tech blog.40

To many educators, the more products Blackboard bolts on, the more unmanageable the system becomes.

For example, to complete a single assignment, students often need to switch back and forth between modules that aren’t linked together well. What should be a seamless experience becomes a herky-jerky annoyance.

“Many view Blackboard as the embodiment of everything wrong with education technology,” wrote Wired magazine, calling the software “frustrating, migraine-inducing, and burdensome.”

The same article quoted Blackboard’s own VP of Design calling it “a low-quality product that moves very slowly and does poor-quality work.”41

Many unhappy with outdated LMS

Is it any wonder that so many people want to replace Blackboard with something newer and better?

Page 20: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

20

For example, as shown in Figure 7, a recent report from Brandon Hall Group showed almost half (47.7%) of the 457 organizations surveyed want to replace their LMS. The main reasons for all this dissatisfaction were:

• Poor reporting• Difficult to use• Poor customer support• Out-of-date appearance• Can’t adapt to changing needs• Lack of mobile features

“In a nutshell,” the report concluded, “far too many organizations feel they are paying too much for systems that are difficult to use, out of date, and do not provide the data and analytics the companies need.”42

Challenge #1: Reducing costs

LMS in general, and Blackboard in particular, are industry standards, so many potential new hires have experience with these products. This reduces the need for costly consultants to configure and maintain an LMS.

However, this is small consolation for the many complaints noted above.

As legacy products designed in an earlier era, Blackboard and similar run on-premises as licensed software. They require each institution to pay a stiff license fee, buy hardware, and perhaps upgrade their networks.

On top of those capital expenses, colleges and universities continue to pay an annual maintenance fee running about 20% of the software license.

And your campus IT team must support the system, and get up in the middle of the night to fix it when it breaks.

By placing all these demands on an institution, traditional LMS is part of the infrastructure that drives up the cost of courses in the first place. How can something that’s part of the problem suddenly become part of the solution?

Challenge #2: Building engagement and success

Traditional LMS is now a mature category. Most systems provide many tools that support teaching processes, such as feedback, marking, and reporting.

Figure 7: Organizations Considering Replacing Their LMS

Source: Brandon Hall Group, 2014

YesNo52.3% 47.7%

Page 21: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

21

The problem is, these tools are not always linked together to create a smooth user experience for students, teachers, or administrators.

What’s more, the “look and feel” of traditional LMS seems old-fashioned. After all, Blackboard was designed before today’s petite mobile devices were even possible.

In effect, traditional LMS systems like Blackboard were designed for yesterday’s CIOs trying to automate schools, not for educators seeking to engage Millennial students.

Relying on a legacy LMS to support any new online courses could be a serious mismatch that limits student engagement and learning success.

Challenge #3: Selecting the right technology

Traditional LMS is reliable, with a solid record of up-time. Most of the serious bugs have by now been fixed. These systems scale up to handle many students, as long as you provide enough servers and connectivity.

Legacy vendors are slowly starting to offer cloud versions; for example, Blackboard has a cloud version of one major module. No word on when the other seven will be available; that could be years from now.

Now for the drawbacks.

Designed before today’s e-commerce systems were even on the drawing board, Blackboard lacks must-have features like an integrated payment system with invoicing for learners and a payment history for administrators.

Traditional LMS remains very costly, demanding on-premises hardware and high licensing fees. Support fees and IT costs add to the burden. These costs are irritating when educators see many features they don’t need but still have to pay for.

And the whole system has been bolted together with little apparent concern for the user experience.

Page 22: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

22

TECHNOLOGY: Cloud-based LMSThe next-generation of learning management system (LMS) runs in the cloud, not on campus. This eliminates the need to buy on-premises hardware, software, or networking, and provides a much more cost-effective a pay-as-you-go model.

As well, this significantly lightens the IT burden for support. Students, teachers, and administrators access the LMS through their browsers, the learning curve.

For any software running as a service (SaaS) all maintenance is done by the vendor. When an update is created or a bug fixed, the new version is automatically deployed and immediately visible.

Not every cloud-based LMS can match 100% of the features in a traditional LMS. But the features that are present are top priority for mainstream educators, not bells and whistles for a tiny minority.

And the more recent design means that cloud-based LMS offerings have likely been integrated to provide a seamless learner experience from desktop to tablet to smartphone.

About WizIQ LMS

WizIQ LMS can be considered a next-generation LMS that provides all the most popular “must have” features of a classic LMS.

What’s more, WizIQ LMS is an integrated e-learning platform that enables educators to blend recorded lectures with live interaction for the best of both worlds: self-paced online content plus live engagement with an instructor.

The WizIQ platform has been used by more than 400,000 educators to deliver online learning to more than 4.5 million students from 200 countries around the world. And every month, instructors use the platform to create another 100,000+ courses.

There’s a reason why the WizIQ LMS is so popular. It’s been designed from the ground up to support educators who want to deliver quality, cost-effective, successful online education.

Challenge #1: Reducing costs

The cloud deployment of SaaS turns the model for traditional LMS on its head. No licensing fee is extracted. No on-premises hardware or software is required. A vastly lighter burden is placed on IT, which no longer needs do any configuration or emergency fixes. And colleges don’t need to pay for unnecessary bells and whistles in the software itself.

All this rewrites the rules of software pricing, so that any department or school can afford to start running a cloud-based LMS and pay for it out of their department operating budget.

Page 23: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

23

Challenge #2: Building engagement and success

One good way to engage today’s students is with modern, well-designed software and a seamless user experience.

WizIQ LMS includes a virtual classroom suitable for live, instructor-led learning in a blended model. Earlier we saw that this is the most engaging and successful format for online education.

Of course, support for mobile devices is baked in, so students can enjoy anywhere, anytime access on a laptop, tablet, or mobile phone.

And finally, WizIQ LMS provides detailed learner analytics that provide insights that can lead to better courseware and pedagogy. Instructors no longer need to guess, they can easily measure student progress, and see where to intervene as necessary to provide more support.

Challenge #3: Selecting the right technology

WizIQ LMS provides a long list of “must-have” features, including quick course authoring and payment handling, two features missing from many traditional LMS systems.

White-labeled apps enable every college or university to add their own independent branding for a unified user experience.

Built-in test and assessment features include more than 50 different types of questions, and an easy-to-scan dashboard provides test results along with many powerful statistics and ratios.

Cloud-based systems can smoothly scale up to cover unexpected spikes in demand, such as first day/last day of term.

WizIQ provides all the key features and benefits of traditional LMS, plus more, at a lower cost and with a much smaller burden on your IT team.

For more on how WizIQ implements the top 10 “must have” features that educators look for when moving to online education, see Appendix A.

To find out more about how you can save costs and improve learning outcomes with the right technology, visit www.WizIQ.com or contact [email protected]

Page 24: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

24

About WizIQWizIQ is an easy-to-use, mobile-ready learning management platform. It enables education providers to deliver live instructor-led and self-paced learning by helping them create, manage, and market courses from their own self-branded online academies.

The WizIQ platform offers cutting-edge features such as live virtual classroom, rapid course authoring, mobile learning, test and assessment, learner insights and an integrated payment system.

Over 400,000 education and training service providers have leveraged the WizIQ platform to deliver online education to over 4.5 million learners in 200 countries.

In 2015, WizIQ was rated amongst the world’s top five learning management platforms onGetapp® and TrustRadius™ and was a winner at the Brandon Hall Excellence in Technology Awards.

For more information, visit www.WizIQ.com

Page 25: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

25

Appendix A: WizIQ LMS FeaturesThis appendix describes how WizIQ LMS implements each of the 10 must-have features that many educators look for to support their online learning.

1. Rapid content authoringProvides faster transition to online courses and faster ROI.

• No scripting language to learn, no coding headaches to troubleshoot• Includes cloud-based content library to securely host and deliver course

content• Supports all popular file formats: Word, Excel, PowerPoint, PDF, MP4• Convert PowerPoint slides with voiceover to MP4 video• Create your choice of self-paced, live instructor-led, or blended courses• Create, schedule, and deliver live audio/video lectures from desktop PC or

mobile device

2. Virtual classroomLive, instructor-led learning provides more interaction, more engagement, and more successful outcomes.

• Supports live audio/video, document sharing, and screen sharing• Includes interactive whiteboard to write notes, solve problems, draw

diagrams• Includes media player to deliver fast, full-screen video• Supports on-the-fly polls, questions, and peer-to-peer chat• Class recordings available to participants in video format• Powered by content delivery network that accelerates streaming and

downloads

3. Powerful test and assessment frameworkFeedback from learners helps to evaluate and improve learning outcomes.

• Provides customizable capabilities with more than 50 types of tests• Enables instructors to review each learner’s progress

4. Flexible reports and analyticsInsights into individual and group learning help educators improve the learner experience.

• Generate intuitive reports and dashboards• Provides reports on individual and group attendance, content views,

content preferences, and assignment completion• Gain a 360-degree view of your classroom, courses, and learners• Boost learner outcomes and increase ROI on your online courses

Page 26: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

26

5. Support for mobile learningProvides flexible learning options on various devices for students on the go.

• Students can use mobile apps to attend live classes, watch recordings,access courseware, and send in assignments

• Supports all mobile devices including Android, iPhone, and iPad• Get live notifications, chat or e-mail with other students or teacher• Out-of-the-box support for mobile learning with no coding or tweaking

6. Independent brandingEnables you to build an independent online brand for your institution.

• Easy to rework white-labeled modules with your branding• White-labeled modules include online academy, mobile apps, and

payments

7. Built-in monetizationSaves on development costs and helps generate new streams of revenue quickly.

• Delivers invoices to learners• Includes payment system for debit cards, credit cards, PayPal, ApplePay,

wire transfers, and more• Enables you to collect tuition fees instantly and securely• Provides individual payment history reports for admins

8. Quick and cost-effective deploymentEliminates headaches, reduces IT burden, saves costs, and generates faster ROI.

• Cloud-based solution means no capex on hardware, software, ornetworking

• Web access provides easy-to-learn, simple-to-use platform• Works smoothly with all existing IT

9. Smooth scalabilityEliminates technical headaches, saves costs, and supports popular online courses.

• Scales to support an unlimited number of learners• Supports scaling and access through multiple device and operating systems:

smartphones, tablets, or laptops

10. Related services availableVendor shares expertise in launching and delivering online courses.

• Fast, responsive technical support during course delivery• Free webinars and expert sessions to share best practices

Page 27: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

27

Sources1: Marianne Bakia, Linda Shear, Yukie Toyama, Austin Lasseter, “Understanding the Implications of Online Learning for Educational Productivity”, U.S. Department of Education, Office of Educational Technology, January 2012, page v, retrieved 15 December 2015 from http://tech.ed.gov/files/2013/10/implications-online-learning.pdf

2: Elaine Allen & Jeff Seaman, “Grade Level 2014: Tracking Online Education in the United States”, Babson Survey Research Group, Pearson, and Quahog Research Group, 2015, retrieved 25 November 2015 from http://www.onlinelearningsurvey.com/report/2014SurveyInfo.pdf

3: Travis Mitchell, “Chart: See 20 Years of Tuition Growth at National Universities,” US News & World Report, 29 July 2015, retrieved 15 December 2015 from http://www.usnews.com/education/best-colleges/paying-for-college/articles/2015/07/29/chart-see-20-years-of-tuition-growth-at-national-universities

4: “Student Loan Debt Averages $28,950 for Class of 2014”, The Institute for College Access & Success, 27 October 2015, retrieved 9 December 2015 from http://ticas.org/sites/default/files/pub_files/student_debt_and_the_class_of_2014_nr_0.pdf

5: Joel Diringer, Kathy Phipps, and Becca Carsel, “Critical Trends Affecting the Future of Dentistry: Assessing the Shifting Landscape,” American Dental Association, May 2013, page 8, retrieved 9 December 2015 from http://www.ada.org/~/media/ADA/Member%20Center/FIles/Escan2013_Diringer_Full.ashx

6: “Current Student Loan Debt”, Graphiq, retrieved 15 December 2015 from https://www.graphiq.com/wlp/7KVUt4zcqOh

7: Mike Patton, “Are Student Loans A Ticking Time Bomb?” Forbes.com, 25 November 2015, retrieved 8 December 2015 from http://www.forbes.com/sites/mikepatton/2015/11/25/are-student-loans-a-ticking-time-bomb/

8: Dian Schaffhauser, “Survey: 4 in 5 Parents Question Value Proposition of College”, T|H|E Journal, 7 December 2015, retrieved 14 December 2015 from https://thejournal.com/articles/2015/12/07/4-in-5-parents-question-value-proposition-of-college.aspx

9: Bakia et al, U.S. Department of Education, Office of Educational Technology, January 2012, pp 27 through 31

10: Marsha Lovett, Oded Meyer, and Candace Thille, “The Open Learning Initiative: Measuring the effectiveness of the OLI statistics course in accelerating student learning”, Journal of Interactive Media in Education, 2008, retrieved 15 December 2015 from http://oli.cmu.edu/wp-oli/wp-content/uploads/2012/05/Lovett_2008_Statistics_ Accelerated_Learning_Study.pdf

11: J. D. Fletcher and Ralph E. Chatham, “Measuring return on investments”, chapter in Human performance enhancement in high-risk environments, Paul O’Connor and Joseph Cohn (eds.), Greenwood Publishing Group, 2009, pp 106-128

12: According to the U.S. Census Bureau, the average American commute is 25.4 minutes each way, totaling 4 hours 10 minutes per week. This could be longer is congested areas where many colleges and universities are located. Confirm local commute times here http://project.wnyc.org/commute-times-us/embed.html#5.00/42.000/-89.500

13: “Impact on cost savings” from Final Report: Florida Gulf Coast University, National Center for Academic Transformation, 31 October 2003, retrieved 17 December 2015 from http://www.thencat.org/PCR/R3/FGCU/FGCU_FR2.htm

14: “Impact on Students” from Final Report: Florida Gulf Coast University, National Center for Academic Transformation, 31 October 2003, retrieved 17 December 2015 from http://www.thencat.org/PCR/R3/FGCU/FGCU_FR1.htm

Page 28: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

28

15: Tatiana Bourlova and Mark Bullen, “The Impact of E-Learning on the Use of Campus Instructional Space”, The University of British Columbia, Distance Education, 2005, retrieved 17 December 2015 from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.502.7644&rep=rep1&type=pdf

16: “Reducing Costs through Online Learning: Five Proven Strategies from the US, Canada, the UK and Australia”, Contact North, 2013, page 10, retrieved 15 December 2015 from http://teachonline.ca/sites/default/files/reducing_costs_through_online_learning.pdf

17: Paul Stuiber et al, “An evaluation of virtual charter schools”, Wisconsin Department of Public Instruction, February 2010, pp 40-43, retrieved 17 December 2015 from http://legis.wisconsin.gov/lab/reports/10-3full.pdf

18: James Sloan and Katherine Mackey, “VOISE Academy: Pioneering a blended-learning model in a Chicago public high school”, Innosight Institute, December 2009, retrieved 17 December 2015 from http://www.christenseninstitute.org/wp-content/uploads/2013/04/VOISE-Academy.pdf

19: Cathy Cavanaugh, “Getting students more learning time online: Distance education in support of expanded time in K–12 schools”, Center for American Progress, May 2009, page 12, retrieved 17 December 2015 from https://www.americanprogress.org/wp-content/uploads/issues/2009/05/pdf/distancelearning.pdf

20: “Final report: A comprehensive assessment of Florida Virtual School, 2007”, Florida Tax Watch Center for Educational Performance and Accountability, 2007, page 25, retrieved 17 December 2015 from https://www.flvs.net/docs/default-source/research/TaxWatch-Study.pdf

21: John Watson, “A national primer on K–12 online learning”, International Association for K–12 Online Learning (iNACOL), 2007, quoted in the second edition, 2010, page 15, retrieved 17 December 2015 from http://www.inacol.org/wp-content/uploads/2015/02/iNCL_NationalPrimerv22010-web1.pdf

22: John Watson, “Report to the Joint Budget Committee of the Colorado State Legislature on the cost of online education”, Colorado Department of Education, February 2004

23: “Who We Are”, The National Center for Academic Transformation website, not dated, retrieved 16 December 2015 from http://www.thencat.org/whoweare.html

24: Barbara Means, Yukie Toyama, Robert Murphy, Marianne Bakia, and Karla Jones, “Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies”, U.S. Department of Education, Office of Planning, Evaluation, and Policy Development, revised September 2010, page 5, retrieved 15 December 2015 from http://www2.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf

25: Borges, N.J., et al,“Comparing Millennial and Generation X Medical Students at One Medical School”, Academic Medicine, 81.6, 2006, pp 571-576, retrieved 15 December 2015 from http://www.med.wmich.edu/how-are-millennial-students-and-faculty-different-previous-generations

26: “About Us” and “News” pages, Edx.com, retrieved 11 December 2015 from https://www.edx.org/press/open-edx-powered-bitnami-now-availablehttps://www.edx.org/about-us

27: William Fenton, “EdX review”, PC Magazine, 29 May 2015, retrieved 11 December 2015 from http://www.pcmag.com/article2/0,2817,2484806,00.asp

28: Max Woolf, “Who Performs the Best in Online Classes?” Minimaxir blog, 10 July 2014, retrieved 11 December 2015 from http://minimaxir.com/2014/07/online-class-charts/

29: John Marcus, “All Hail MOOCs! Just Don’t Ask If They Actually Work”, Time, 12 September 2013, retrieved 11 December 11 2015 from http://nation.time.com/2013/09/12/all-hail-moocs-just-dont-ask-if-they-actually-work/

30: “Benefits of Blended Learning,” Macmillan English Campus website, no date, retrieved 15 December 2015 from http://www.macmillanenglishcampus.com/about-us/pedagogy/benefits-of-blended-learning/

Page 29: and How to Solve Them › uploads › attachment › ... · 2016-01-21 · mobile app, and launch ... online than in large lecture courses.10 Another found military training accelerated

29

31: Barbara Means, Yukie Toyama, Robert Murphy, Marianne Bakia, and Karla Jones, “Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies”, U.S. Department of Education, Office of Planning, Evaluation, and Policy Development, revised September 2010, pp xiv-xv, retrieved 15 December 2015 from http://www2.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf

32: Jacqueline Bichsel, “Analytics in Higher Education: Benefits, Barriers, Progress, and Recommendations”, EDUCAUSE Center for Applied Research, August 2012, page 3, retrieved 17 December 2015 from https://net.educause.edu/ir/library/pdf/ERS1207/ers1207.pdf

33: Shane Hastie and Stéphane Wojewoda, “Standish Group 2015 Chaos Report—Q&A with Jennifer Lynch”, InfoQ blog, 4 October 2015, retrieved 16 December 2015 from http://www.infoq.com/articles/standish-chaos-2015

34: “About Moodle,” Moodle website, retrieved 27 November 2015 from https://docs.moodle.org/30/en/About_Moodle

35: Raj Sabhlok , “Open Source Software: The Hidden Cost of Free”, Forbes.com, 18 July 2013, retrieved 17 December 2015 from http://www.forbes.com/sites/rajsabhlok/2013/07/18/open-source-software-the-hidden-cost-of-free/

36: “Total Cost of Ownership (TCO)”, Gartner IT Glossary, not dated, retrieved 17 December 0215 from http://www.gartner.com/it-glossary/total-cost-of-ownership-tco

37: Andy Patrizio, “Microsoft cheaper to use than open source software, UK CIO says”, NetworkWorld, 2 May 2014, retrieved 17 December 2015 from http://www.networkworld.com/article/2226844/microsoft-subnet/microsoft-cheaper-to-use-than-open-source-software--uk-cio-says.html

38: Justin Ferriman, “Why Moodle Is Becoming Irrelevant”, LearnDash blog post and comments, 16 June 2014, retrieved 27 November 2015 from http://www.learndash.com/why-moodle-is-becoming-irrelevant/

39: William R. Watson, Sunnie Lee Watson. “An argument for clarity: what are learning management systems, what are they not, and what should they become?” TechTrends, Springer Verlag, 2007, 51(2), pp.28-34, retrieved 7 December 2015 from https://hal.archives-ouvertes.fr/file/index/docid/692067/filename/Watson-2007.pdf

40: “Blackboard Turns to ‘New Learning Experience’ to Bust Its Bad Brand”, EdSurge blog, 21 July 2015, retrieved 7 December 2015 from https://www.edsurge.com/news/2015-07-21-blackboard-turns-to-new-learning-experience-to-bust-its-bad-brand

41: Issie Lapowsky, “The Reeducation Of Blackboard, Everyone’s Classroom Pariah, Wired, 21 July 2015, retrieved 7 December 2015 from http://www.wired.com/2015/07/blackboard-reinvention/

42: David Wentworth, “LMS Trends 2014: Satisfaction and Spending”, Brandon Hall Group, February 2014, page 4, retrieved 17 December 2015 from http://go.brandonhall.com/lms_trends_2014