Thompson NETg s Blended-Learning Model: The Next Generation … · 2015. 7. 29. · Elaboration Theory (Reigeluth, 1999b) is a model advocating a progression of successively more

Thomson NETg s Blended-Learning Model:The Next Generation of Corporate and

School-Based LeamningStacey L. T. Boyle, Kenneth Kolosh, James LAllier, andJudith Lambrecht

Abstract

As on-line learning technology becomes more advanced, it is worthwhile to re-examine the characteristics of effective instruction. Those characteristicsaddressed in this study include: creating realistic scenarios for learning; aligninglearning objects with realistic scenarios, using the actual software being taught assoon as possible; providing access to live on-line mentors; and providing on-site,instructor-led training responsive to individual learning differences. Three"blended models, " all incorporating scenario-based exercises, were found to resultin higher quality performance on authentic assessments than either an E-Learningmodel or a control group. E-Learning alone is more effective than no instruction.The lack of achievement differences among the blended models supports theassumption that effective instruction is less about media and more about instruc-tional design. Effective blended learning results when the rationale behind theblend is to (a) activate prior experience, (b) demonstrate skills, (c) apply the skills,and (d) integrate the skills into real-world activities.The authors would like to acknowledge the contribution of Dr. David Merrill, Utah StateUniversity, to every aspect of this study.

IntroductionSEVERAL FACTORS MAKE THIS STUDY ESSENTIAL as a response tocurrent technology training needs and the increased capabilities ofcontemporary training systems. These training systems include bothtraditional classrooms in public school settings as well as industry-basedtraining. As businesses continue to depend on information technology,more individuals must be supported in their efforts to master criticaltechnology applications. Further, ongoing technology change means thatnew software is regularly introduced and new technical capabilities areavailable to enhance business operations. These technological changesalso make it more feasible to create individually responsive learningsystems.

Research ProblemThe training system evaluated in this study is that developed by the

NETg corporation as part of its worldwide technology training offerings.Thomson NETg provides technology training using a blended model forinstruction. This study is a natural outgrowth of a continuous improve-ment effort at NETg. The mission is to identify the best blend of learning

DR. STACEY L. T. BOYLE is Lead Researcher, Thompson NETgMR. KENNETH KOLOSH is Researcher, Thompson NETgDR. JAMES L'ALLIER is Chief Learning Officer and Vice President, Research and Devel-opment, Thompson NETg, Naperville, IL 60563-9099DR. JUDITH J. LAMBRECHT is Professor, Business and Marketing Education, 420A VoTechBldg., 1954 Buford Avenue, University of Minnesota, St. Paul, MN 55108

Vol. XLV No. 3 Fall, 2003 145

STACEY L. T. BOYLE, KENNETH KOLOSH, JAMES L'ALLIER, AND JUDITHJ. LAMBRECHT

solutions that produce significant business impact and return oninvestment. The business impact is demonstrated in this study through aperformance assessment model using real-world task scenarios. Theimpact on return on investment is the subject of other studies extendingthe research reported here.

The primary goals of the study were:1. To determine if there are significant performance differences on

authentic assessments among learners who received an Instructor-Led Training (ILT) Blended Learning solution, Text Blend,Scenario-Based Exercise (SBE) Only Blend, E-Learning alone, orno training.

2. To determine if there are significant time performance differenceson authentic assessments among learners who received anInstructor-Led Training (ILT) Blended Learning solution, TextBlend, Scenario-Based Exercise (SBE) Only Blend, E-Learningalone, or no training.

Need to Evaluate Instructional ComponentsOne delivery mechanism within a learning system is on-line, com-

puter-based training designed to develop a wide range of technologyskills. As on-line learning technology becomes more advanced, it isworthwhile to re-examine the characteristics of effective instruction.Some features that may have been technically and economicallyunfeasible in the past may now, possibly, be incorporated into instruction.Those characteristics addressed in this study include the following:creating realistic scenarios for learning; aligning learning objects withrealistic scenarios; using the actual software being taught as soon aspossible; providing access to live on-line mentors; and providing on-site,instructor-led training responsive to individual learning differences. Thisstudy provides support to the importance of these characteristics ofeffective instruction. The following describes the need to evaluate thesespecific instructional components within the context of on-line training.Scenario-Based Exercises (SBEs)

Employed individuals, and those seeking eventual employment, needto learn technology skills in a context similar to actual use. Without sucha context, the skills attained are not likely to transfer to real world settings(Mayer & Wittrock, 1996). Much of current software instruction has beenfocused strictly on software operation in isolation from real-world uses.Learning software in an abstract, decontextualized way not only offers nomotivation for students (Carroll, 1998; and Collins, Brown, & Newman,1989; Merrill, 2002), but it prevents understanding actual software use. Ablended learning model needs to be empirically justified as providing abetter alternative to current technology-based training that providesexplanations with little real-world application.Aligning NETg Learning Objects (NLOs) with SBEs

Students need to learn in contexts that parallel the real world, andthey also need to spend their learning time on only those learning objec-tives that match their needs and also match employment requirements146 The Delta Pi Epsilon Joumnal

THOMSON NETG BLENDED-LEARNING MODEL

(Collins, Brown, & Newman, 1989; Mayer, 1992). Stand-alone NETglearning objects (NLOs) provide just such a path. This idea needs to beextended by validating a model that allows such stand-alone NETg objectsto be integrated into small case problems representing real-world tasks orScenario-Based Exercises (SBEs). Again, a blended learning model maydo this if it can be shown empirically to be better than the presentation ofNETg learning objects without such a real-world setting.Aligning Text Objects with SBEs

The use of textbooks has long been a cornerstone of academic study.However, in today's fast-paced business environment few employees havethe time or desire to study a textbook from start to finish. Instead, today'slearners need access to the specific information they need when theyneed it. Because of this, specific topics, or "Text Objects," from CourseTechnology textbooks were mapped to each step of the scenario-basedexercises. In this way learners could reference the specific content theyneeded when they needed it in order to successfully complete the exercise.Use of Actual Software

As students learn, they need eventually to use the real, full-featuredsoftware that is the object of instruction. In most on-line technologytraining settings, the software taught is simulated, not the actual desktopsoftware that is the focus of instruction. Real software is an importantsource of illustrating actual use in the job context. Further, using realsoftware provides essential feedback to confirm students' learning accom-plishments (Carroll, 1998). A way needs to be validated that confirms theintegration of real software as early in on-line instruction as possible.Use of a Mentor

In addition to the instruction and feedback provided through technology-based training, students need ways to get questions answered and prob-lems solved immediately in a way that is generally possible only throughlive intervention (Carroll, 1998; van der Meij & Carroll, 1998). Not everyquestion can be anticipated and provided for through formal technology-based instruction, and a real mentor is the best source of help to facilitatelearning. Use of a live expert or mentor during training may be a methodof validating such a model.Providing a Full Instructor-Led Environment

Instructor led training (ILT) has long been considered the premiummethod for delivering instruction. Beyond the availability of an on-linementor to answer questions, an instructor through interacting andobserving the learners may be best able to address the varying needs oflearners. Course Technology's Custom Course to integrate the ILTinstruction around the scenario-based exercises was used to deliverinstruction. The added benefit of ILT will be one of the instructionalcomponents tested in this study.

This study sought to address the aforementioned needs. The followingsections outline the specific research goals and provide the theoreticalrationale for the design that was implemented.

147Vol. XLV Na 3 Fall, 2003

STACEY L.T BOYLE, KENNETH KOLOSH, JAMES L'ALLIER, ANDJUDITHJ. LAMBRECHT

Defining Thomson NETg s Structured Blended Learning ModelRecent years have seen a proliferation of instructional design theories

and models. Tennyson, Schott, Seel, and Dijkstra (1997) and Reigeluth(1999a) summarized a number of these different positions. David Mer-rill's (2002) First Principles model was the current state-of-the-art instruc-tional solution. Merrill's model was often referred to as a blended learningapproach because it contained facets from a myriad of instructionalmodels ranging from traditional lecture to technology-based training.Merrill (2002) stated many current instructional models suggest the mosteffective learning environments are those that are problem-based, such asscenario-based exercises, and involve the student in four distinct phases oflearning: (a) activation of prior experience, (b) demonstration of skills,(c) application of skills, and (d) integration of these skills into real-worldactivities. The following outlines the instructional phases of Merrill's FirstPrinciples theory (Merrill, 2002):

* Learning is facilitated when learners are engaged in solving real-world problems. (Problem)

* Learning is facilitated when existing knowledge is activated as afoundation for new knowledge. (Activation)

* Learning is facilitated when new knowledge is demonstrated to thelearner. (Demonstration)

* Learning is facilitated when the learner applies new knowledge.(Application)

* Learning is facilitated when new knowledge is integrated into thelearner's world. (Integration)

Problem or Scenario-Based ExerciseMuch of the current work in cognitive psychology showed participants

learn better when engaged in solving problems. Problems should beauthentic, real world, and, if possible, personal. Problem-based learningwas well represented by a number of recent instructional modelsincluding: Collins et al (1989), Cognitive Apprenticeship; Schank,Berman, & Macperson (1999), Goal Based Scenarios; Jonassen (1999),Constructivist Learning Environments; Savery & Duffey (1995), Problem-Based Learning; Clark & Blake (1997), Novel Problem Solving; Carroll(1998), Minimalist Instructional Practices; and van Merrienboer (1997),Whole Task Practice in 4C/ID Model. Showing learners the task orproblem they will be able to solve is more effective than stating abstractlearning objectives.

To master a complex problem, learners must first start with a lesscomplex problem. After the first problem is mastered, learners are givena more complex problem. Through a progression of increasingly com-plex problems, the learners' skills and confidence gradually improveuntil they are able to solve complex problems. Elaboration Theory(Reigeluth, 1999b) is a model advocating a progression of successivelymore complex problems. Learning is best when there is a series ofproblems to solve and when coaching is gradually withdrawn for eachsucceeding problem.

148 The Delta Pi Epsilonjou-rnal


In this project, the problem is referred to as scenario-based exercises(SBEs). While the SBEs provide a real-world context for learningsoftware operation, instructional designers need to be prepared for thepossibility that learning problems may result from incorrectly assumingthat students understand the business problem. Thus, it is important tobe prepared to offer assistance with regard to both software operationand the current use as exemplified in the SBEs.Activation

It has long been a tenet of education to start where the learner is. Iflearners have had relevant experience, the first phase of learning is to besure that this relevant information is activated, ready for use as a founda-tion for the new knowledge. If learners have not had sufficient relevantexperience, the first phase of learning a new skill should be to provideexperiences that they can use as a foundation for the new knowledge(Andre, 1997). When learners feel that they already know some of thematerial to be taught, their existing experience can be activated by anappropriate opportunity to demonstrate what they already know. Thisactivity can be used to help direct learners to the yet-to-be-learned newmaterial and thus result in more efficient instruction. In this project.activation is accomplished by using SBEs that represent commonlyunderstood business scenarios and also by allowing the learners to useonly as much of the instruction as they need to complete the SBEs.Demonstration

Knowledge to be learned exists at two levels: the general level and thespecific level. Learning is best when learners are shown (examples)rather than told (generalities) (Gagne, 1985). One role of instruction isto provide appropriate learner guidance to facilitate learning. One formof guidance is to focus learners' attention on relevant information. Earlyin an instructional presentation, this attention focusing function facili-tates knowledge acquisition. However, as the instruction progresses, thisinformation focusing role should be faded and learners expected toattend to and focus their own attention on the relevant aspects of theinformation (Andre, 1997). In this project, demonstrations were prov-ided through on-line screen cams and within the technology-based portionof the instructional material.Application

Learning is facilitated when learners are required to use their newlyacquired knowledge or skill and when the practice is consistent with thelearning goal (Mayer, 1992). Making errors is a natural consequence ofproblem solving. Most learners learn from the errors they make, especiallywhen they are shown how to recognize the error, how to recover from theerror, and how to avoid the error in the future (van der Meij & Carroll,1998). Adequate practice provides mnultiple opportunities for learners touse their new knowledge or skill for a variety of problems (Spiro &Jehng,1990). In this project, the practice with the SBEs was performed in thelive Excel application and received the normal feedback about errorprovided by the software. Feedback was also available by providinglearners with model solutions to the practice problems.VoL XLV No. 3 Fall, 2003 149

STACEY L. T. BOYLE, KENNETH KOLOSH,JAMES L'ALLIER, AND JUDITH J. LAMBRECHT

IntegrationLearning is facilitated when learners can demonstrate skill improvement.Integration is facilitated not only by practice with feedback duringinstruction, but also by assessments. Realistic and valid assessments areimportant opportunities for integration of current learning. In thisproject the final performance assessments paralleled the SBEs in theirstructure. They were real-world problems, applying the same conceptsand skills that had been presented and practices in the SBEs.

MethodologyThe following sections describe the two major instructional models

compared, the Blended Learning Model and the E-Learning Model.Three variations of the Blended Learning Model were compared with theE-Learning Model and with a control group that received no training.The use of a control group without training is considered importantbecause sometimes persons who possess basic computing skills areexpected to learn on their own the desktop computer applicationsrequired in their work (Marquardt & Kearsley, 1999). While this viewmay not be stated publicly, it is often based on the assumption thatknowing the expectations of one's job is sufficient to determine whichfeatures of a software tool may be useful and that a person is able to learnsoftware procedures by self-learning and picking them up as time permits.It is important to ask whether persons with some computing skill canperform assessment made up of realistic business scenarios. Including acontrol no-training situation allowed comparison with persons receivingformal instruction.

Discussion of the two major types of instruction (Blended versus E-Learning) is followed by explanation of the study procedures: studyparticipants, study design, and data analysis.The Blended Learning Model

The Blended Learning Model presents five Scenario-Based Exercises(SBEs) in order of progressive difficulty. The SBEs were designed by aspreadsheet subject matter expert and finalized by internal instructionaldesign experts in conjunction with external reviews by business clients.

The Blended Learning Model was made available with three variations:

1. Blended without the Text Objects (from the textbook), ClassroomInstructor (ILT) or the ILT Custom Course. This will be calledBlended.

2. Blended with Text Objects (from the textbook), no ClassroomInstructor (ILT) or ILT Custom Course. This will be called TextBlended.

3. Blended with Text Objects (from the textbook), ClassroomInstructor, and ILT Customer Course. This will be called ILTBlended.

All three groups also had access to the NETg E-Learning materials,called NETg Learning Objects (NLOs). Table 1 summarizes how theSBEs were presented to study participants.

150 7'he Delta Pi F-psilonjournal


Table 1Components of Blended Learning Model

Training TrainingScenario Matrix Study Matrix to ILT-based to On-Line Manual Learner Learner Text Classroom Customexercise NLOs' Mentor' FAQs' Demo' Guidance' Feedback' Objects", Instructor' Course'

1 '1 1 X N N

2 V ai

3 V i N :

4 V i N i N V

5 V -1 - , V

'Available with all blended groups2Available w ith the Text Blend3Available with the ILT Blend

Training Matrix. The training matrix was a table of relevant learningactivities that learners could explore for in-depth instruction. The tasksthat comprise the SBEs were mapped to individual learning objects (forthis study, NETg Learning Objects, or NLOs, were employed) that sup-port the specific task the learners were being asked to perform. In thisway learners could access only the learning objects that they personallyneeded to successfully complete the SBE without having to review unneces-sary information. Learners used this matrix to locate specific learningobjects within a spreadsheet application fundamentals course. Theassumption made here is that learners in any setting, industry or schoolbased, need ready access to instruction on specific software features thatare either new to them or on which they are having difficulty. Suchresponsiveness to individual learning needs and differences is importantin any instructional setting.

Online Mentoring. On-line 24/7 mentoring provided access to certifiedsubject matter experts through the mentoring program. The mentorswere provided copies of the SBEs for reference and were directed tosupport the learning process rather than provide outright answers. Theonly restriction was that learners were not allowed to confer with anexpert when completing an assessment activity.

Frequently Asked Questions. Written Frequently Asked Questions (FAQs)were available to the study participants online and in the participants'study manual. Experts generated specific FAQs with the spreadsheetapplication for each scenario-based exercise based on the learningobjectives for that exercise. FAQs in the study manual were only availablefor the first two exercises as this form of additional support was elimi-nated as SBEs increased in difficulty.

Demonstration. Demonstrations were on-line screen-cams of the tasksthe learner was asked to complete. Demonstrations were provided onlyfor the first scenario-based exercise. Other explanations and demnonstrationsVol. XL>V No. 3 Fall, 2003 151

STACEY L. T BOYLE, KENNETH KOLOSH,JAMES L'ALLIER, ANDJUDITHJ.LAMBRECHT

of procedures were available when learners chose a learning objectassociated with one of the five scenarios. While not directly related to thespecific business content of the scenario, the learning objects (NLOs)were available through the scenario/learning object matrix. If learnerswere interested in viewing a demonstration of the required activity orseeing the associated learning object, they would access them on-line.Demonstration was offered only initially in an effort to build autonomouslearners. Thereafter, students needed to exercise their own judgmentabout whether a more detailed explanation/demonstration was necessary.

Guidance. In the first and second SBE, learners were provided detailedstep-by-step guidance. Learner Guidance was provided at the end of thethird and fourth SBEs for additional assistance. The five SBEs providedprogressively less assistance to the learners. Starting with the third SBE,detailed guidance was withdrawn from the body of instructions. To sup-port the learner's adjustment to the decrease in assistance, they wereprovided a Learner Guidance section at the end of the third and fourthSBEs. Learners were instructed to use the Learner Guidance section onlyif they needed additional assistance. All learner guidance was thenremoved for the fifth SBE.

Learner Feedback. Feedback was provided to the learners by associatingmodel solutions to each scenario (SBE). Feedback was also provided atthe conclusion of all five SBEs. A scoring macro built into the applicationspreadsheet provided the learners with task-specific feedback. Studyparticipants were trained to use the macro and how to interpret the results.

Text Objects. The tasks that comprise the SBEs were mapped toindividual topics from Course Technology's Microsoft Office 2000 Intro-ductory Concepts and Techniques textbook (Shelly, Cashman, & Vermaat,2001). Each topic supported the specific task the learner was being askedto perform.

Classroom Instructor. Participants in the ILT Blended group receivedthe same instructional materials as the Text Blend participants but withinthe context of an instructor led classroom. The basic instructional strategyused by the instructors consisted of three phases: (a) instruction inpreparation for a scenario exercise, (b) completion of the scenario as aself-directed classroom or homework exercise, and (c) guided classroomdiscussion reviewing the scenario exercise.

ILT Custom Course. The classroom instruction provided as part of theILT Blend was designed around the scenario exercises using CourseTechnology's ILT Custom Course 4.0. This application is a customizationengine that allows for the restructuring and creation of content to meetspecific training needs.

Taken together, the scenario-based exercise support features weremore comprehensive initially and gradually eliminated as recommendedby Merrill's First Principles model and to scaffold the learner to autonomy.

152 'Fhe Delta Pi Epsilonjournal


The E-Learning ModelTo measure software skill performance differences between Blended

Learning and E-Learning alone, this study relied on NETg E-Learning.This is a valid representation based on the ongoing, widespread globalusage of NETg E-Learning in corporate and school-based learning environ-ments where this technology training has been implemented.

NETg E-Learning was developed on the premise that the adult learnerhas a set of unique work-related needs that must be met if the instructionis to be accepted. There is a list of principles aimed at making these E-Learning courses both acceptable and relevant to this important popu-lation. The main adult learning principles addressed in the NETg E-Learning courses are: learner control, time sensitivity, real world applica-tion, and concrete examples. The E-Learning courses support theseprinciples by providing learners with the maximum control possible overtheir learning environment, allowing them to set their own pace, andallowing them to select the content they wish to pursue. The instructionis unit-based and allows learners, using pre-test guidance, to select onlythe material they need to learn at any given time, progressing at their ownpace.

Using established instructional design principles, NETg has createdmodular components of instruction, NETg Learning Objects (NLOs). InNETg E-Learning courses, these modules exist at the topic level. Eachtopic contains one measurable objective, a learning activity, and uniqueassessment items. This stand-alone structure allows NETg, based on pre-test results, to extract topics from a lesson or course and recombine theminto a new, customized learning solution on demand, creating theinstruction a learner requires. As with the Blended Learning Model,on-line mentoring and FAQs were available to the E-Learning aloneparticipants.Study Participants

Participants in this study included four corporate and three post-secondary educational sites. A self-selection sampling technique wasemployed in the corporate settings. The population for this study wasNETg clients in both corporate and post-secondary public school environ-ments. Global e-mails were sent to supervisors and employees at clientsites soliciting participation during targeted weeks. Those who respondedto the e-mail on a first-come first-served basis where given a short list ofthe spreadsheet objectives used in the study and asked to iderntify which,if any, they could currently perform. Participation was based on pos-sessing basic computing skills but not exceeding minimal skill levelson their self-report proficiency. Once the qualified participants wereselected, they were randomly assigned to one of the five groups. Thus,participants were self-selected, but randomly assigned to grou ps. Thoseassigned to the control group received training at the end of tlie researchsessions. Other volunteers who did not qualify for participation in theresearch sessions were not otherwise excluded from traininag in theircorporate settings. The goal was to have at least 30 participants in eachgroup. It was not possible to say how representative the final participantsVol. XLV No. 3 Fall, 2003 153

STACEY L.T BOYLE, KENNETH KOLOSH,JAMES L'ALLIER, ANDJUDITHJ.LAMBRECHT

are of employees or students among NETg clients.In school settings, entire classes were randomly assigned to one of

four instructional groups. The control group was a comparable group ofstudents not currently taking a spreadsheet course. In one school thecontrol group was comprised of a class that took the post-training assess-ment prior to instruction. The final assessments from this group were notincluded in the data analysis, only the assessments before instruction.Overall, the number of participants providing complete data sets was asfollows:

* Group One (ILT Blend) - 41* Group Two (Text Blend) - 31* Group Three (SBE Only Blended) -49 participants* Group Four (NETg) -49 participants* Group Five (Control) - 30 participants

Total - 200 participantsStudy Design

The study design, illustrated in Figure 1, measured all participants'skill prior to beginning fundamental spreadsheet training and thenapplied the prescribed learning approach. Following completion of thelearning process, study participants' ability to perform on authenticassessments was monitored and then compared.

Figure 1 Study Design

Prior To Training

DemographicI Questionnaire

Self Rating of ExcelI Proficiency

Training Post Training Assessment

Group OneILT Blended Learning

Group TwoText Blended Learning

Group ThreeSBE Only Blended

Learning

Group FourE-Learning +

Mentoring

Group Five-ControlNo Training

Authentic Excell Assessments I

A customized on-line learning environment (using NETg XtremeLearn-ing) was developed for this study and was used to deliver all instructionand assessments. The duration of the training was approximately five 8-hour days. School-based sessions lasted approximately three weeks meeting154 7'he Delta Pi EpsilonJournal


2 or 25 hours every other day. The study took place in either a corporateor academic computer lab.

The participants were given a manual containing the activities fortheir respective groups. All manuals introduced the on-line environmentand took the participants through the necessary steps to log on, set uppasswords, access E-Learning, access the on-line mentor, and completesurveys. The manuals for the 3 Blended Groups contained the fiveinstructional SBEs. The manual for all 5 groups also included the samethree authentic Excel assessments paralleling the instructional SBEs, forcompletion after training.

Authentic Assessments. Three post-training SBEs were developed inconjunction with the five instructional SBEs to represent commonbusiness applications. For example, the learners were asked to completea targeted spreadsheet requiring them to format cells, write a formula forcurrency exchange, and fill/copy cells. Spreadsheet subject-matterexperts and external business clients reviewed these tasks to ensurevalidity in relation to the instructional objectives. To assure accuratescoring of the assessment SBEs, two researchers independently scoredeach task using a set of standardized scoring sheets. The independentlyderived scores were then compared and any discrepancies resolved. Theresults indicated a very high level of agreement and provided strongsupport for the accuracy of the scoring process.

For this study, internal consistency reliability was concerned with theperformance of operations within each assessment SBE. The meaninternal consistency reliability coefficient for the three SBEs was aCronbach alpha of .94. This was an extremely high reliability coefficientand indicated the SBEs consistently measured what they purport tomeasure, which is authentic spreadsheet skills.Data Used for Analysis

All study participants receiving one of four instructional experiencescompleted SBEs at the conclusion of the instruction. The control groupcompleted the SBEs prior to instruction. These yielded accutracy scoresbased on the conformance of the student's solution with a key depictingthe model solution. Accuracy points were accumulated based on thematch of solution detail with this key. Final scores were expressed as apercentage of possible points.

Completion time for each SBE was recorded by a macro within thespreadsheet file used for each of the three tasks. Time in seconds wasrecorded for each task, and a total time for three tasks was converted tominutes for analysis.

Study ResultsThe first primary goal was to determine if there were significant

performance differences on SBEs among learners who received an ILTBlended learning solution, Text Blend, SBE-Only Blend, E-Learningalone, or no training. The research hypothesis tested was that the groupsdid not differ on either their accuracy or time scores attained on theauthentic assessment SBEs. Based on the conceptual base, it wasexpected the most inclusive ITL Blended learning solution would resultVol. XLV No. 3 F414 2003 155

STACEY L.T. BOYLE, KENNETH KOLOSH,JAMES L'ALLIER, AND JUDITH J. LAMBRECHT

in the highest achievement levels, followed by the Text Blend, SBE-OnlyBlend, and E-Learning, in this order. The control group was expected toperform least well. Table 2 provides a comparison of performance on theauthentic assessment SBEs by each group.

Table 2Quality Comparison of Performance on Authentic Assessment

Assessment StandardSBEs Deviation

Group N Mean as aPercent

1-ILT Blend 41 0.8985*** 0.06452-Text Blend 31 0.8665*** 0.10903-SBE Blended 49 0.8860*** 0.08594-E-Learning 49 0.6806*** 0.25665-Control 30 0.3416*** 0.2298

***F(4, 194) = 64.9 p < .001

Analysis of variance indicates that performance on the assessmentSBEs differed among the groups significantly, F(4, 194) = 64.91, p < .001.Planned contrasts indicated that performance differed significantlybetween the Blends (ILT, Text, SBE) and the control group (p < .001).Gabriel's (SPSS, 1999) pairwise comparisons test based on the Stu-dentized maximum modulus test [Post Hoc] indicated that the perform-ance of each of the blended groups (ILT, Text, and SBE) individuallydiffered from the control group's performance and from the E-Learninggroup's performance significantly (p < .001). Gabriel's Post Hoc tests(SPSS) also indicated that the E-Learning group significantly differedfrom the control group (p < .001). However, Gabriel's Post Hoc tests(SPSS) failed to show significant performance difference between thethree blended groups.

Table 3 highlights the comparisons of performance improvement onthe authentic assessment SBEs by the three Blended Models, E-Learning,and the Control Group.

Table 3Comparison of Performance Improvement on Authentic Assessment

PerformanceComparison Improvement

ILT Blend v. Control ...... ................ 163%SBE Only Blend v. Control .......... ............ 159%Text Blend v. Control ....... ............... 153%E-Learning v. Control ................. ...... ......... 99%ILT Blend v. E-Learning ......... ............. 32%SBE-Only Blend v. E-Leaming . ...................... 30%Text Blend v. E-Learning ......... ............. 27%ILT v. Text Blend ...................... 4%SBE-Only Blend v. Text Blend . ...................... 2%ILT v. SBE Only Blend ............................... I %

The Delta Pi Epsilonjournal156


As seen in Table 3, the blended groups performed between 153% and163% better than the control, and between 27% and 32% better than theE-Learning group. In addition, the E-Learning group perforrned 99%better than the control. Relatively small performance differences areshown among the three blended groups, from 1% to 4%.

The second primary goal was to determine if there were significanttime performance differences for the assessment SBEs. These results areshown in Table 4.

Table 4Completion Time on Authentic Assessment SBEs

Group Mean Time to StandardComplete Deviation

AssessmentSBEs

1-ILT Blend 23.68 minutes** 7.072-Text Blend 25.46 minutes** 5.283-SBE Only Blend 28.78 minutes"* 12.864-E-Learning 48.96 minutes** 19.885-Control (note)

**F(3, 140) = 26.73 p < .001

Note. Mean time performance was not analyzed for the control group because manyassessment SBEs were not successfully completed in this group.

Analysis of variance showed significant differences among the com-pletion times, F(3, 140) = 26.73, p = .001. Gabriel's (SPSS, 1999) pairwisecomparisons test based on the Studentized maximum modulus test [PostHoc] indicated that individually each of the blended groups (ILT, Text,and SBE) differed significantly from the E-Learning group's time (p <.001). However, Gabriel's (SPSS) pairwise comparisons test based on theStudentized maximum modulus test [Post Hoc] failed to show significanttime differences among the three blended groups.

Table 5 illustrates the percentage of time savings on the RWTs amongthe 4 groups. As seen there, the 3 blended groups completed RWTsbetween 41% and 51% faster than Group 4 (E-Learning). Relatively smalldifferences are shown among the 3 blended groups, from 7% to 18%.

Table 5Comparison of Time Improvement on Authentic Assessment SBEs

Time PerformanceComparison Improvement

ILT Blend v. E-Learning . ..................... 51%Text Blend v. E-Learning . ..................... 48%SBE Blend v. E-Learning ........................... 41%ILT Blend v. SBE Blend ....... .............. 18%Text Blend v. SBE Blend . ..................... 12%ILT Blend v. Text Blend ..................... 7%


STACEY L. T. BOYLE, KENNETH KOLOSH, JAMES L'ALLIER, AND JUDITH J. LAMBRECHT

In summary, the following findings supported the primary goals ofthis study:

* The 3 Blended Learning groups significantly out-performed othergroups in spreadsheet application performance.

* The 3 Blended Learning groups took significantly less time tocomplete the assessment SBEs than did the E-Learning alone group.

* The most unexpected finding was that the 3 Blended Learninggroups did not differ significantly from one another on performanceor time.

* As expected, the E-Learning alone group performed significantly bet-ter on the spreadsheet application assessments than the control group.

Conclusions and ImplicationsThe following conclusions can be drawn from the findings of the

blended learning study:* Structured Blended Learning Models do improve learner perform-

ance-speed and accuracy-over non-blended learning.* Blended Learning Models designed around SBEs achieve perform-

ance improvement from 27 to 32% over E-Learning alone for theseparticipants.

* E-Learning alone delivers nearly 100% improvements in learnerperformance in SBEs, indicating that formal instruction matters indeveloping software skills.

* The three different blended models did not differ from oneanother indicating the components common to the three blendsrepresent the essential components of a successful blended model.

* All of the blended groups did about 158% better on the authenticassessment score than the control group. Even the E-Learninggroup (no SBEs) came in at 99% higher than the control.

Comparing all of the blended groups to the E-Learning group (noSBEs), the learners in the blended groups did approximately 30% betteron their authentic assessment scores. The surprise in these comparisonswas that the learners in all of the blended groups completed the authenticassessment in an average of 46% less time. The findings demonstratedthat blended learning not only yielded better results over conventional,non-blended E-Learning, but it also produced a more efficient learner asmeasured in time.

If there were significant differences between the control group andthe blended groups, differences between the E-Learning group and theblended groups, and if media mix was not a statistically significant factor,what caused the increases in authentic assessment score and a decrease intime? After eliminating all other factors, it was found that the use of theSBEs accounted for these positive increases in the learners' effectivenessand efficiency. The SBEs were the common element in the blendedgroups that was absent from the E-Learning Group.

This caused NETg to redefine the term blended learning. It was lessabout media and more about instructional design. True blended learn-ing results when the rationale behind the blend is to: (a) activate priorexperience, (b) demonstrate skills, (c) apply the skills, and (d) integrate

158 7he Delta Pi Epsilon Journal


the skills into real-world activities (Merrill, 2002). These four attributeswere the basis for the SBEs used in each of the blended groups andrepresented the key to success of the learners in all of these groups.

All three of the blended models provided gradations in learningguidance and extent of feedback. In addition to the feedback that wasavailable to students by having a model solution available, use of theactual software was a key feature in giving students freedom to explore thesoftware, try to match the solution key, and get feedback to see whether amatch was present. Being able to use the full-featured version of thesoftware was an important aspect of what made the SBEs real and pre-pared students to successfully complete the assessment SBEs. A valuedattribute of the SBE experience of using real software was the freedom toaccomplish a task in a variety of ways and to try a procedure several times.

Since the SBEs have been judged to be both reliable and valid LevelIII Kirkpatrick measures (Kirkpatrick, 1998), researchers can have someconfidence in the differences and lack of differences that have beenobserved. Thus, it is possible to assess on-line training using SBEs and full-featured software.

In sum, the key implication of this research is that the structure usedfor the Blended Learning Models is one that should be emulated in bothcorporate and public school technology training settings if employment-ready skill capabilities are the instructional goal. The structure combinesscenario-based exercises, alignment of tasks in these scenarios with match-ing learningobjects, early use of the actual software, and readily availablementor support throughout the learning experience. While persons inschool-based settings have generally placed a premium on giving studentsaccess to the real, full-featured software, industry settings have often usedsimulated software incorporated in online technology training systems.Both groups, however, often present instruction largely focused onsoftware features as opposed to focusing instructional attention on thesefeatures embedded in realistic business problems. This research supportsthe inclusion of realistic business tasks/scenarios as an integral part of theinstruction. Future research should explore using SBEs in technologytraining settings to match a wider variety of software features and relatedbusiness content topics.

ReferencesAndre, T. (1997). Selected micro-instructional methods to facilitate

knowledge construction: Implications for instructional design. In R.D. Tennyson, F. Schott, N. Seel, & S. Dijkstra, Instructional design:International perspective: Theory, research, and models Vol. 1 (pp. 243-267).Mahwah, NJ: Lawrence Erlbaum Associates.

Carroll, J. M. (1998). Reconstructing minimalism. InJ. M. Carroll (Ed.),Minimalism beyond the Nurnbergfunnel (pp. 1-18). Cambridge, MA: TheMIT Press.

Clark, R. E., & Blake, S. B. (1997). Designing training for novel problem-solving transfer. In R. D. Clark, R., Building expertise: Cognitive methodsfor training and performance development. Washington DC: IniternationalSociety for Performance Improvement.


STACEY L.T. BOYLE, KENNETH KOLOSH,JAMES L'ALLIER, ANDJUDITHJ. LAMBRECHT

Collins, A., Brown, J. S., & Newman, S. E. (1989). Cognitive apprentice-ship: Teaching the crafts of reading, writing, and mathematics. In L.B. Resnick (Ed.), Knowing, learning and instruction: Essays in honor ofRobert Glaser (pp. 453494). Hillsdale, NJ: Lawrence Erlbaum Associates.

Gagne, R. M. (1985). The conditions of learning and theory of instruction (4thed.). NewYork: Holt, Rinehart and Winston.

Jonassen, D. (1999). Designing constructivist learning environments. InC. M. Reigeluth (Ed.), Instructional design theories and models: A newparadigm of instructional theory Vol. 2 (pp. 215-239). Mahwah, NJ:Lawrence Erlbaum Associates.

Kirkpatrick, D. L. (1998). Evaluating trainingprograms: Thefour levels (2nded.). San Francisco: Berrett-Koehler Publishers, Inc.

Marquardt, M. J., & Kearsley, G. (1999). Technology-based learning:Maximizing human performance and corporate success. Voca Raton, FL:American Society for Training & Development and St. Lucie Press.

Mayer, R. E. (1992). Thinking, problem solving, cognition (2nd ed.). NewYork: W. H. Freeman.

Mayer, R. E., & Wittrock, M. C. (1996). Problem solving transfer. In D.Berliner & R. Calfee (Eds.), Handbook of educational psychology (pp. 45-61). New York: Macmillan.

Merrill, M. D. (2002). First principles of instruction. Educational TechnologyResearch and Development, 50(3), 43-60.

Reigeluth, C. M. (1999a). Instructional design theories and models: A newparadigm of instructional theory Vol. 2. Mahwah, NJ: Lawrence ErlbaumAssociates.

Reigeluth, C. M. (1999b). The elaboration theory: Guidance for scopeand sequence decisions. In C. M. Reigeluth (Ed.), Instructional designtheories and models: A new paradigm of instructional theory Vol. 2 (pp. 425-453). Mahwah, NJ: Lawrence Erlbaum Associates.

Savery, J., & Duffy, T. (1995). Problem-based learning: An instructionalmodel and its constructivist framework. In B. G. Wilson (Ed.), Designingconstructivist learning environments (pp. 135-148). Englewood Cliffs:Educational Technology Publications.

Schank, R. C., Berman, T. R., & Macperson, K. A. (1999). Learning bydoing. In C. M. Reigeluth (Ed.), Instructional design theories andmodels: A new paradigm of instructional theory Vol. 2 (pp. 161-181).Mahwah, NJ: Lawrence Erlbaum Associates.

Shelly,G., Cashman, T., & Vermaat, M. (2001). Microsoft office 2000introductory concepts and techniques. Boston: Course Technology.

Spiro, R. J., & Jehng, J. C. (1990). Cognitive flexibility and hypertext:Theory and technology for the non-linear and multidimensionaltraversal of complex subject matter. In D. Nix & R. Spiro (Eds.),Cognition, education, and multimedia (pp. 163-205). Hillsdale, NJ:Lawrence Erlbaum Associates.

SPSS Advanced Models 10.0. (1999). Chicago: SPSS, Inc.Tennyson, R., Schott, F., Seel, N., & Dijkstra, S. (1997). Instructional

design: International perspective: Theory, research, and models Vol. 1.Mahwah, NJ: Lawrence Erlbaum Associates.

The Delta Pi Epsilonjournal160


van Merrienboer,J. J. G. (1997). Training complex cognitive skills. Engle-wood Cliffs, NJ: Educational Technology Publications.

van der Meij, H., & Carroll, J. M. (1998). Principles and heuristics fordesigning minimalist instruction. In J. M. Carroll (Ed.), Minimalismbeyond the Nurnbergfunnel (pp. 19-53). Cambridge: MIT Press.


COPYRIGHT INFORMATION

TITLE: Thomson NETg’s Blended-Learning Model: The NextGeneration of Corporate and School-Based Learning

SOURCE: Delta Pi Epsilon J 45 no3 Fall 2003WN: 0328803594002

The magazine publisher is the copyright holder of this article and itis reproduced with permission. Further reproduction of this article inviolation of the copyright is prohibited. To contact the publisher:http://www.dpc.org

Copyright 1982-2004 The H.W. Wilson Company. All rights reserved.

Documents

Thompson NETg s Blended-Learning Model: The Next Generation … · 2015. 7. 29. · Elaboration Theory (Reigeluth, 1999b) is a model advocating a progression of successively more