E-LEARNING AND STUDY EFFECTIVENESS

40 JournalofComputerInformationSystems Spring2009

E-LEARNINGANDSTUDYEFFECTIVENESS ˘ ˘ ˘ VIKTORIJASULCIC DUSANLESJAK UniversityofPrimorska UniversityofPrimorska SI-6140Koper,Slovenia SI-6140Koper,Slovenia

ABSTRACT

Aftertheinitiale-learningenthusiasm,wehavefinallyreachedastageofsobriety(similartotheburstofthedot.combubbleinbusiness). In thepaper, acaseofblended learningapproach inhigher education is presented that is part of a wider nationallyfinancedresearchprojectaboutICTande-learning.Theresearchresultspresented in thepaper showed that ICTper sedoesnotimprove e-learning effectiveness. Only the use of differentteachingstrategies(methodsofteachingandlearning)fromthoseusedintraditionaleducationcanimprovestudyeffectivenessine-learning. Keywords: Information and Communication Technology,HigherEducation,Europe,BlendedLearning

INTRODUCTION

E-learning is becoming increasingly interesting for societyand educational institutions because it supports the concept oflifelong learning [19, 21] and because knowledge is becomingmoreandmoreimportant,bothinSlovenia[15,16]andabroad[19] which increases demand for various educational formsandmeans. Increaseddemandfordifferenteducationprogramsworldwideiscateredforbyeducationalinstitutions,whichoffernewformsofeducationthatarefrequentlysupportedbyICT,and,aboveall,theInternet.Thee-learningmarketinNorthAmericaisthefastestgrowingmarket[7].Substantialgrowth(85%averageannualgrowth)ofexpensesfore-educationwasalsoforecastedbytheIDC[8]in2003. ICTinfrastructure,whichdiffersfromcountrytocountry[25],isoneoftheprerequisitesfore-learning.InthefirstpartofthepaperthedifferencesbetweentheUSAandtheEuropeanUnion(EU) countries are presented. The USA spends more on ICT,therefore, it is not surprising that there aremore InternetusersthanintheEUcountries.TheInternetisobligatoryife-learningisunderstoodasaweb-basedlearning.ButInternetpenetrationinthesocietycannotgiveustheassurancethatcomputerswouldbeusedmorefrequently,whichisconfirmedbythestatisticaldatainthepaperbelow. Even though it seems that e-learning could solve manyproblems of gaining knowledge [22], some researches statedseriousproblemsconnectedwithe-learning—e.g.highdrop-outrates[10,4],nosignificantdifferencesinacquiredknowledge[14,11]andunsuccessfulnessofe-learningprojects[12]. In the HE business school with no more than 3,000 post-graduate and undergraduate students, in a small country withno more than 2 million citizens1, a blended learning approachhasbeenimplementedandthroughtheevaluationprocesssomesignificantresultshavebeenfoundout.Theresearchmethodology,hypothesisanddataanalysesarepresentedinthesecondpartofthepaper. Our research presented in the paper proved that blendedlearning,whichinvolvesdifferentteachingstrategiesthanthosein

traditionaleducation,improvesstudyeffectivenessandrepresentsasuitablecoursedelivery forpart-timestudents,mainlydue totemporalandspatialadaptabilityofthestudyprocess.

ICTINSOCIETY

InternetPenetrationandICTExpenditures

TheactionplaneEurope[5]promotesintensiveICTusageinallEUcountriestomodifytheireconomiesintoknowledge-basedsocieties.E-learningbecomesmoreandmoreimportantbecausetheemployeescannotleavetheirworkplacesandreturnbacktothe schools and acquire the accurate knowledge that is neededfor active and successful participation in the knowledge-basedsociety. E-learning offers employees a flexible, time and placeindependent, way of study. The implementation of e-learningrequiressomeprerequisites—anappropriateICTinfrastructure,inadditiontocomputerandInternetliteracy,whichareobviouslytwo of the fundamental but not the most important factors forthe e-learning success. The suitable ICT infrastructure has tobeprovidedby a government or bybusinesses.The endeavorsof some national economies can be seen from the comparisonbetweenEuropeancountriesandtheUSpresentedbelow. AccordingtotheWDIdata[24]somesignificantdifferencesbetween Slovenia, the European Monetary Union (EMU)2countries and theUSAexisted in2005 (Table1). In theUSA,ICTexpendituresweremuchhigherthanintheEMUcountriesorinSlovenia.Thereforeitisnotsurprisingthatmorehouseholderswere connected to the Internet in the USA than in the EMUcountriesorSlovenia(Table1). Eurostatstatistic[6]reported55%oftheInternetaccessintheUSA,intheEMUcountries40%and43%inEU153in2003.In

1. In Slovenia, enrollment in higher education institutions be-tween 2000 and 2004 increased faster than in other EU states, and reached, on average, 5.6% annually (compared to the EU 25 with 3.3% and in the EU 15 with the average 2.4% annual increase) [6].

2. EMU — European Monetary Union or the euro area is the area comprising European Union Member States in which the Euro has been adopted as the single currency. In 2000, the euro area comprised Austria, Belgium, Fin-land, France, Germany, Ireland, Italy, Luxembourg, the Netherlands, Portugal, Spain. Greece became a member of the Euro area on 1 January 2001. Slovenia became a member of the Euro area on 1 January 2007.

3. The number of member countries in the European Union prior to the accession of ten candidate countries on 1 May 2004 (Aus-tria, Belgium, Denmark, Finland, France, Germany, Greece, Ireland, Italy, Luxembourg, Netherlands, Portugal, Spain, Swe-den, United Kingdom) [13].

Spring2009 JournalofComputerInformationSystems 41

thisyear,EurostatdidnotreportanydataforSlovenia.DataaboutICTexpenditurefromEurostat[6]differsfromWDIstatisticsabitaswell.AccordingtoEurostat[6]EU15countriesspent2.7%ofGDPonICT,inSlovenia2.2%andintheUSA3.3%ofGDPin2006althoughtheorderisnotchanged—theUSAportionofGDPforICTexpendituresisgreaterthanintheEUcountries. From 2000 to 2007 the significant Internet user growth isrecorded; the highest growth is stated in Slovenia (263.3%),thelowestinNorthAmerica(117.2%)wheretheportionoftheInternetusersinthemiddleof2007ismuchhigher(70.2%)thanin Slovenia (55.5%) at the same time. In the EU countries theInternetusergrowthis170.8%[10].Itisinterestingthatin2007

inSloveniamoreInternetusersarerecorded(55.5%)thanintheEUcountries(51.8%),whichwasnotthecasein2005.SloveniaisaheadoftheEUaveragewhenwecompareddataaboutstudentcomputerusageinthelast3months.AccordingtotheEurostat[6]. 86% of Slovenian students used computer in the last 3months compared to the 84% of the EU students. In Slovenia,adultpopulationwithhighereducationismostfrequentcomputerusers. Based on the Internet penetration and extensive computerusage among Slovenian students we supposed that there werenolimitationforcontinuinge-learningenlargementinSlovenia.There is one among many problems connected to e-learningimplementation that needs special attention, namely the wayofteachingandlearningneedtobechangedandadaptedtothee-learningenvironmentanditscharacteristics[18,20].

ICTinSlovenianHigherEducationInstitutions

In the academic year 2005/2006, an extended researchproject about ICT usage in Slovenian higher education (HE)institutionswasconducted[23].Thequestionnairethatfollowedthe requirementof theEurostatagencywasused forcollectingindicatorsoftheEU[23].ThequestionnairewassentviaregularmailtoallHEinstitutionsinSlovenia.Theresponsewas92.9%dueto2cyclesofadditionalcontactinacaseaHEinstitutiondidnotrespondtoregularmail. AccordingtoHEinstitutionsICTusagehadanimpactmoreon the improvement in the researcharea (M=4.1) andon theadministrationarea(M=4.0)thanonotherinstitution’sactivityareas(Table2). ICT can support educational processes in different ways.For some institutions web pages on which students can findneededinformationisinawaye-learning.Forothersonlywhenalearningenvironmentisused,e-learningcanbementioned.InSlovenia, a commercial learning management systems (LMS)suchasWebCTorBlackboardisusedandrecentlyopensourcecourse management system (CMS) Moodle became more andmorepopularamonginstitutionsatalleducationlevels[20].IntheTable3,theresponsesaboutwebpagesandLMS/CMSusagearepresented. PrivateHE institutionsandeconomicsandbusinessschoolsuseICTtosupporttheireducationalprocessesmoreintensivelythanthepubliconesandthosefromothersstudyareas.

TABLE1:InternetaccessandICTexpenditurein2005

Internetaccess ICTexpenditure

Internet Schools Users connectedto %of $per (%) theInternet(%) GDP capita

Slovenia 40.0 96 3.1 532EMU 43.9 NA 5.4 1,726USA 63.0 100 8.8 3,690

Legend: NA — not availableResource: WDI [24]

TABLE2:ImpactofICTusageintheSlovenianhighereducationinstitutions

ICTusageinfluencetheprocessimprovementonthe M SD

…researcharea 4.1 1.0…administrationarea 4.0 1.1…educationarea 3.9 1.0…managementarea 3.8 1.0…environmentrelationshiparea 3.8 1.0

Resource: Vehovar et al. 2006 [23]. Legend: M = average, SD = standard deviation: 1=a lower impact, 5=a huge impact.

TABLE3:WebpagesandlearningenvironmentsatSlovenianHEinstitutions

Webpages(%HEinstitutions) LearningHEInstitution/educationarea environments Withbasicinformation Up-to-dated Interactive (%HEinstitutions)

Property Public 70 54 25 10 Private 72 72 31 16

MedicineandHealth 45 55 4 3 Education SocialscienceandEducation 75 54 30 3 area EconomicsandBusinessscience 72 68 37 26 TechnicalandNaturalScience 71 60 27 11 Humanities 66 44 4 0

Average—2005/2006 70 58 26 12 Average—2004/2005 69 52 20 –

Resource: Vehovar et al. 2006 [23].


PROSANDCONSOFE-LEARNING

E-learning is learningduringwhich studentsget their studymaterials through electronic media (the Internet, intranet,extranet,satellites,audio/videoequipment,CDs)[9].Thestudybecome easily accessible to students who could not attendclassical lectures because of the distance or other daily duties(workandprivate—familyresponsibilities).BecauseoftheICT(e-)studybecomesmoretimeandspaceflexiblewayofstudythantraditionalstudy. But weaknesses of e-study (e-learning) should also bepointedout.Growinge-learningande-coursesmarketdoesnotmean that the education supply isbeing improvedand that thepossibilities for acquiring new knowledge are becoming morevaried. According to research carried out by Overton [14], forexample, 60% of projects related to e-learning introductionin British business environments were unsuccessful. Furtheron, Mungania [11] states in her research related to barriers ine-learningthat70%ofparticipantsine-learning,whichiscarriedout inAmericancompanies,areforvariousreasonsunlikely tofinish theire-learning.Withregard to technologicalcomplexityofe-learning such lackof success represents,on theonehand,negative promotion for e-learning in general and, on the otherhand, economically unjustified investment on the part of theeducationprovider. DespitethefactthatICTrepresentsthefundamentalmaterialconditionfore-learning,itdoesnothaveastatisticallysignificantinfluence on the effectiveness of e-learning. In 1983, Clark[3] expressed doubts by claiming that ICT represents only themediumforthematerialsdistributionandnotthemeansfortheimprovementoflearningoutcomes.SimilarresultsarefoundinthedetailedstudyofRussell [15,12]whofoundout that thereare no statistically significant differences between classicaland online learning. Therefore Russell suggested that classicalcontents (courses) should be adequately adapted for e-delivery[15]. Ally[1] foundout that teachingstrategies, i.e.methodsandwaysofteachingaswellastestingandassessmentmethodsaremuchmoreimportantthantheuseofICTineducation.Teachingstrategies depend on a number of factors, above all on theparticipantsineducation,thelevelofeducation,contentsandthepurposeofeducation(e.g.formalorinformaleducation). In their research on the reasons for huge dropout rates inonline education, Dagger and Wade [4] ascribed it to studentnon-participation in the learning process. Even the introduc-tion of adapted multimedia materials did not improve studentsuccess rates, which further confirms the importance of teach-ing methods in on-line education. Therefore teaching methodsand teaching strategies are investigated in our research aswell.

E-LEARNINGINPRACTICE—THERESEARCH

BlendedLearningApproachatHEInstitutions

E-learningdiffersfromtraditionaleducationinthatitincludesICTinthelearningprocess.Duetosomedisadvantagesofonlinelearning (feeling of isolation, lack of interaction with fellowstudentsandteachers,hugedropout)wedecidedtoblendonlinelearningwithtraditionalformsoflearning,knownasablendedlearningapproach. Inourcase,weusedane-classroom,which

wasthenamegiventotheweb-basedcoursemanagementsystemMoodle(www.moodle.org).Whenoutliningthecoursedeliveryweplanned tocombineface-to-facemeetingswithe-classroomactivities.Thefrequencyofface-to-facemeetingsdependsonthetype of course and the type of study. For compulsory courses,attendedbyfull-timestudents,weplanned50%of face-to-facesessions, whereas for part-time students we planned only twoface-to-face sessions in eight weeks, namely the introductorymeetingandthelastsession. Thecombinationofface-to-faceandonlinesessionswasnottheonlydifferenceintroducedinoureducationprocess.Wetriedtomakeadetailedplanofallstudentactivitieswhenpreparingtheplanforthecoursedelivery, takingintoaccountcriteriaforcreditassignmenttocoursesaccordingtoECTS(EuropeanCreditTransferandAccumulationSystem),whichstatesthatanaveragestudent(foranaveragegrade)shouldspendbetween25and30hoursoftheirworkfor1ECTScreditpoint.Thestudentworkloadincludes face-to-face sessions, independentworkathomeor atschool — the study of literature, home assignments, rehearsal,groupwork—planningandguidinggroupwork,fieldworkandotheractivitiesstatedinthedeliveryplanforacertaincourse,aswellasassessmentandtesting.

ResearchMethodologyandResearchHypothesis

The suitability of blended e-learning was studied usingdata collected during the course delivery and thus summarizesthe characteristics of the delivery itself. Next, we studied theinfluence of these characteristics for the course evaluation andthe opinion about the acquired knowledge as the output of theeducation process (the influence on the effectiveness of theeducationprocess). Data were collected by means of questionnaires, whichwere delivered to polled students electronically (through thee-classroom).Ourcontributionshowstheresultsofourresearchthatwascarriedoutduringtheacademicyear2005/2006amongthe students attending the E-business course (elective course,6 ECTS, 109 students enrolled, average response 94.5%) andamong the students attending the Business informatics course(compulsory course, 4 ECTS, 125 students enrolled, averageresponserate78.0%). Data gathered from e-questionnaires were entered in Exceland later imported in SPSS. During data processing we usedmethods of descriptive statistics and the method of correlationtogetherwiththemethodoflinearregression(Stepwisemethod).Becausetheblendedmethodofstudywasdealtwithfromvariousviewpoints, we grouped individual variables due to their largenumber.Ifcalculatedparametersallowed(Alpha<0.80;KMO<0.6;Bartletttest:Sig.>0.05),wegroupedvariablesbymeansofthemethodofmaincomponents. Inourresearch,thefollowinghypothesesweretested:

H1:E-learningrequiresarenewalofthelearningprocess— courses should be delivered by using differentteachingstrategies.

H2:Blended delivery enables students to acquiremore knowledge and different knowledge than intraditionallydeliveredcourses.

H3:Students, who have the opportunity to experiencemoreface-to-facesessionswiththeirfellowstudentsandteachersbecomemoreeffectivethantheirfellowstudents,whoaredeprivedofsuchopportunities.


Thesehypotheseswillbetestedonapopulationofstudents,enrolled in abusiness school in the tertiary levelof education.The researchwascarriedoutwith students studyingacompul-sorycourseaswellasanelectivecourse.Weshallonlydescribethepartofourresearch,whichrelatestothemodeofdeliveryofe-learning.

DataAnalysis

Activities in the e-classroom required from the students toaccessthee-classroomatleastthreetimes.Takingintoaccountthat all students do not have their own computer during theirstudies in their temporaryplaceof residence,we arranged thatstudentshadopenaccesstothecomputerroomatleastfortwohoursperday.Placeandfrequencyofe-classroomaccess,daysandtimeofon-lineaccesswascheckedbymeansoftheend-of-coursefeedback. AsseenfromtheTable4, themajorityofstudentsaccessede-classroom from their homes (elective course: 73.4% of stu-dentson full-timestudy,70.0%ofstudentsonpart-timestudy,compulsorycourse:68.9%),everyday,andregardlessofthedayoftheweekorthetimeoftheday.

Part-timestudentsattendingtheelectivecoursemostkeenlyacceptedthee-classroom.Studentsattendingtheelectivecoursebelieved that thee-classroomwassimple touse,welldesignedand represented a pleasant learning environment (Table 5).Differences are statistically significant (P < 0.05). Statisticallysignificantarealsothedifferencesbetweenstudentsonpart-timestudiesattendingtheelectivecourse(P<0.05),whichmeansthatthee-classroomwasmorekeenlyacceptedbystudentsattendingtheelectivecourseandbypart-timestudents. By using the method of principle components we formed anew variable, which explains a 79.35% variance of combinedvariables. Statistically significant differences among students can alsobenoticedwithregardtotheopinionaboutmaterials(Table6).Above all, there are differences among students studying full-timeandpart-timeandstudentsattendingelectiveandcompul-sory course. For students attending the compulsory coursematerialsweresignificantlylessunderstandablethanforstudentsattending the elective course, despite the fact that they werepreparedbythesameauthor.Inordertogetopinionsonmaterials,anewvariablewas introduced in theformofanaverageofallthreevariables.

TABLE4:Accesstoe-classroom

EC CCVariables Features FT PT FT # % # % # %

Accesstoe-classroom Onlyfromhome 10 16.7 11 27.5 10 11.1 Mainlyfromhome 34 56.7 17 42.5 52 57.8 Mainlyfromschool/office 12 20.0 11 27.5 17 18.9 Onlyfromschool/office 2 3.3 1 2.5 7 7.8 Fromelsewhere 2 3.3 4 4.4

Frequencyofe-classroomaccess Everyday 30 50.0 26 65.0 13 14.4 Morethan3timesaweek 24 40.0 13 32.5 48 53.4 1-3timesaweek 6 10.0 1 2.5 28 31.1 Onceaweek 1 1.1

Daysofe-classroomaccess Duringworkingdays 5 8.3 6 15.8 24 27.6 Duringweek-ends 6 10.0 1 2.6 10 11.5 Whenever 49 81.7 31 81.6 53 60.9

Timeofe-classroomaccess Inthemorning 4 6.7 5 12.5 6 6.7 Intheafternoon 13 21.7 4 10.0 17 18.9 Intheevening 11 18.3 9 22.5 13 14.4 Lateatnight 32 53.3 6 15.0 3 3.3 Regardlessofthetimeoftheday 51 56.7

Legend: EC – elective course, CC – compulsory course, FT – full-time students, PT – part-time students.

TABLE5:Opinionofstudentsaboute-classroom

EC CC PT:FT EC:CCOn-lineclassroomis... FT PT FT t-test–P t-test–P

...apleasantenvironment. 4.1 4.6 3.9 0.00 0.00

...easytouse. 4.3 4.4 4.0 0.04 0.01

...well-designed. 4.2 4.5 3.7 0.00 0.00

Average 4.2 4.5 3.8 0.00 0.00

Legend: EC – elective course, CC – compulsory course, FT – full time students, PT – part-time students


Similarly to the e-classroom acceptance, part-time studentsalso more keenly accepted study materials. Part-time students(3.0)weremoreinterestedininteractivematerialsthanfull-timestudents(2.7and2.2).Studentsattendingthecompulsorycoursewerenot interestedin interactivematerials(2.2).Datagatheredare even more interesting if we compare them with the shareofmaterialprintoutavailable throughtheonlineclassroom.Onaverage,studentsprintedout77.0%ofstudymaterials,whichislessthaninthepreviousresearch[16,17],whenstudentsprintedoutonaverage84%ofthem. Forthepurposeoffurtherresearch,weformed,bymeansofthemethodofmaincomponents,onevariablefromthevariablesgiveninTable6.Thenewvariable,whichwascalledmaterials,explains70.77%varianceofcombinedvariables. Course delivery in e-classroom differs from the traditionalclassroom course delivery. Initial enthusiasm, which wasnoticed when we introduced the e-classroom for the first time,disappeared.Students know in advancewhat is to be expectedin the on-line course delivery, because information about howon-linecoursesaredeliveredandstudentrequirementsareeasilyavailable in student forums. It is interesting to compare theanswersregardingthedifficultyofon-lineandtraditionalcoursedelivery.Itwasratherobviousthatstudentshadthefeelingthattheyhadtoparticipateactivelyandthattheyhadtoworkmuchharder than inothercourses.On theotherhand, they liked theflexiblewayofstudying.Suchcoursedeliverymadeiteasierforstudentstomanagetheirotherstudyandpersonalresponsibilities,because they could participate in the e-classroom at differenttimesandfromdifferentplaces.Thecomparisonofopinions isgiveninTable7. Blendedcoursedeliverywasexhaustingoratleastnoteasiermainly for students attending the compulsory course, whocomplained during office hours and in their e-mails that they

had to work too hard. Otherwise, average student assessmentsregarding the invested efforts do not differ statisticallysignificantly(Table7). Variablesrelatedtoon-linecoursedelivery,shownonTable4,weremergedintoonevariablebyusingthemethodofprinciplecomponents, which explains 57.32% variance of combinedvariables. The model of blended e-learning differed with regard toelective and compulsory course. Most likely, the differencesstemmed from the way of study. For part-time students, onlytwo face-to-face sessions were organized. The first session isforstudentstogetacquaintedwitheachotherandforpresentingthemwith thecoursedelivery.Thesecondsession—andalsothe last one — is dedicated to student presentations and to adiscussion about the course delivery. Students attending thecompulsory course were unanimous in saying that the numberof sessions is too high (6 in 12 weeks). Only 20% of studentsattendingthecompulsorycourseproposedmorethan6sessions.On average, students wanted 4.8 face-to-face sessions or 40%of planned traditional sessions. Students attending the electivecoursewanted,onaverage,2.9 sessions (part-time students)or3.4 sessions (full-time students). Taking into account that thecourselastedeightweeks,thisrepresentsbetween36.3to42.5%ofsessions.Webelievethatatleasttwosessionsarenecessary,butanadditionalsessionduringthecoursewouldproveuseful,because it may have a positive influence on the motivation ofstudents. AsseenfromTable8,full-timestudentswantedmoreface-to-facesessions,becauseface-to-facesessionsareanopportunitytosocializewiththeirfellowstudents.Ontheotherhand,part-timestudentsfinditdifficult toattendface-to-facesessions,becauseoftheirprofessionalandfamilyresponsibilities.Thedifferenceisstatisticallysignificant(P=0.00).

TABLE6:Opiniononmaterials

EC CC PT:FT EC:CCMaterialswere... FT PT FT t-test–P t-test–P

...understandable. 3.7 4.5 2.9 0.00 0.00

...well-designed. 3.4 4.3 3.5 0.00

...appropriatelycomprehensive. 3.4 3.7 2.5 0.00 0.00

Average 3.5 4.2 3.0 0.00 0.00

Wouldlikeinteractivematerials. 2.7 3.0 2.2 0.02 0.00%ofmaterialprintout. 60.8 91.9 81.1 0.00 0.00


TABLE7:Opiniononcoursedelivery

EC CC PT:FT EC:CCCoursedeliverywas... FT PT FT t-test–P t-test–P

...asexpected. 3.7 4.5 2.9 0.00 0.00

...easier. 3.4 4.3 3.5 0.00

...moreinteresting. 3.4 3.7 2.5 0.00 0.00

...moretiringthantraditionalstudy. 3.9 4.1 4.0

...cheaperthantraditionalstudy. 3.5 4.0 3.0 0.00 0.00



Intheelectivecourse,thefinalsuccessofstudentsdependedontheirsuccessduringplannedweeklyactivities.Theactivitieswere more keenly accepted by students attending the electivecoursethanbystudentsattendingthecompulsorycourse(Table9).Part-timestudentssawthepracticalorientationofthecourseintheweeklyactivities.Full-timestudentsaretoousedtolearnonlybeforetheexaminationsanddidnotliketobeinvolvedinsomanyactivitieseveryweek. By using the method of principle components, variablesconnectedwith study activitiesweremerged intoonevariable,whichexplains54.88%varianceofcombinedvariables. Theeffectivenessofthelearningprocesswastestedwiththecourseassessmentandtheopinionabouttheacquiredknowledge.The course assessment presents the assessment of all studyactivitiesinwhichstudentsparticipatedine-classroom.Becauseknowledge insocial sciences ismoredifficult tomeasure than,say, innatural sciencesor technology, students expressed theirviewsintermsofmoreorlessknowledgeacquiredduringblendedlearning.Theassessmentofacquiredknowledgewasdoneona5-level scale (5=acquiredmoreknowledge,1=acquired lessknowledge). Statistically significant differences cannot be perceived incourseassessment,butcanbefoundinstudents’opinionabout

theacquiredknowledge(P<0.05).Studentsattendingtheelectivecourseacquiredmoreknowledge. For the newly formed variables we tested the correlationof variables with course assessment and the opinion about theacquiredknowledge.

TABLE11.Thecorrelationbetweenthecharacteristicsofblendedlearning,courseassessment

andtheopinionabouttheacquiredknowledge

Variables 1. 2. 3. 4.

1.E-classroom 2.Materials 0.61** 3.Coursedelivery 0.63** 0.73** 4.Activities 0.71** 0.81** 0.77**5.Desired%offace-to-facesessions -0.20** -0.15*6.Courseassessment 7.Acquiredknowledge 0.54** 0.62** 0.69** 0.64**

Legend: ** statistically significant correlation at P=0.01,* statistically significant correlation at P=0.05

Students, who keenly accepted e-classroom and studymaterials,believed that theyacquiredmoreknowledge throughblendedlearning(0.54or0.62).Thesameopinionwasexpressedbystudents,whoweresatisfiedwith thecoursedelivery (0.69)and students, who did not have any problems with activities(0.64).Students,whodidnot likeeither thecoursedeliveryoractivities,desiredmoreface-to-facesessions.Ontheotherhand,students who liked the course delivery and activities desiredfewersessions(-0.20and-0.15),becausestudyinginanonlineclassroomdidnotcauseanyproblems. It is interesting, though, that in our analysis we could notfind any influences of individual course deliveries on course

TABLE8:Opinionaboutface-to-facesessions

EC CCVariables FT PT FT

Wewouldliketostudyinthisway. 3.6 4.2 3.0Thenumberofsessionscouldbesmaller. 3.4 2.9 5.0Desired%offace-to-facesessions. 42.5 36.3 40.0


TABLE9:Opinionofstudentsaboutstudyactivities

EC CC PT:FT EC:CCVariables FT PT FT t-test–P t-test–P

Igotusedtotheactivitieswithoutanyproblems. 3.8 4.1 3.0 0.00 0.00Activitieswerenottoodemanding. 2.9 3.4 2.2 0.00 0.00Activitieswereclearlydefined. 3.4 4.1 3.2 0.00 0.00Involvementinactivitiessuitedme. 3.9 4.3 3.4 0.00 0.00Groupworkdidnotcauseanyproblems. 3.7 3.8 3.1 0.00Therewerenottoomanyactivities. 3.3 4.1 2.5 0.00 0.00Activitiesarepracticallyoriented. 3.7 4.4 3.3 0.00 0.00

Average 3.5 4.0 3.0 0.00 0.00


TABLE10:Theassessmentofexamexercisesandtheopinionaboutacquiredknowledge

EC CC PT:FT EC:CCVariables FT PT FT t-test–P t-test–P

Exerciseassessment(examination). 8.2 8.4 8.0Acquiredmoreknowledge. 3.6 4.2 3.0 0.00 0.00



assessment, but only on the opinion about the acquired know-ledge. Next, we tested the direct influence of variables on theopinion about the acquired knowledge with the regressionanalysis(Table12). E-classroom, study materials and the delivery of e-learningexplain 52.4% variability of the opinion about the acquiredknowledge,whenotherconditionsareunchanged.Thegreatestinfluence on the opinion about the acquired knowledge can beascribed to the acceptability of e-learning delivery (it explains49.3%variabilityoftheopinionabouttheacquiredknowledge).

TABLE12.Blendedlearning(linearregressionanalysis;dependablevariable:opinionabouttheacquiredknowledge)

Includedvariables B t P

Deliveryofe-learning 0.556 5.807 0.000Materials 0.210 2.189 0.030E-classroom 0.173 2.033 0.044

F=62.318Sig.0.000R2=0.524

Excluded variables: activities, desired % of face-to-face session

CONCLUSION

The research regarding e-learning effectiveness was carriedout with the students studying a compulsory and an electivecourseataHEbusinessschool.Basedontheprosandconsstudy,ablendedlearningapproachwasimplemented.Differentteachingmethodswereusedtoinvestigatehowtheteachingmethodscouldimpactthestudyresults. At the beginning of the research three research hypotheseswereformulated.Throughtheresearchwe:

• provedthe hypothesis H1 that ICT introduction in educationrequiresdifferent teachingstrategies,becausecoursedeliveryhas a statistically significant influence on the opinion aboutthe acquired knowledge. A tutor supported weekly activitiesimplemented in the investigated courses resulted in betterresultsthanmethodsusedintraditionalcourses.

• provedthehypothesisH2thatstudentsacquiremoreknowledge,because statistically significant differences appeared withregardtotheopinionsabouttheacquiredknowledge.Especiallystudentswhoattendedelectivecoursesandwhostudiedpart-timeacquiredmoreknowledgeduringtheblendedlearningthanintraditionallearning.WithregardtoRussell[15,12],whoclaimsthattherearenostatisticallysignificantdifferencesinacquiredknowledge among traditional and online course delivery, ourfindingsmaybedifferentbecauseofthefactthatstudentsalsoacquired knowledge that was not directly connected with thecourse.TheyalsoacquiredskillsenabledbytheuseofICTineducation–groupwork,communicationskills,computerandInternetrelatedskills.

• didnot prove the hypothesis H3 that students, who had theopportunitytoexperiencemoreface-to-facesessionswiththeirfellow students and the teacher, became more effective thantheirfellowstudents,whoweredeprivedofsuchopportunities.Lackofface-to-facemeetingswassuccessfullycomplementedby tutor support that motivates students to accomplish theirstudyobligations.

Ablendedlearningapproachhasbeenfoundasamoresuitableapproachwheneducationisorganizedforpart-timestudentsthan

forfull-timestudents.Thefrequencyofface-to-facesessionsitselfdoesnothaveastatisticallysignificantinfluenceontheopinionabouttheacquiredknowledge,butweneverthelessbelievethatweshouldhaveface-to-facesessionswithstudentsat least twoor three times during the course. Our research proved that theacceptabilityofe-learningdelivery,theacceptabilityofmaterialsande-classroomhaveastatisticallysignificant influenceon theacquired knowledge, which means that these characteristicsshould be taken into consideration and, above all, thoroughlystudied. Itwouldbeinterestingtoseeiftheresearchresultswilldifferinthecaseofthestudylevelorinthecaseofstudyfield,thereforesomeadditionalresearchshouldbecarriedoutinthenearfuture.

REFERENCES

1.Ally, M. “Foundations of Educational Theory for OnlineLearning”.Anderson, Terry in Fathi Elloumni, 2004,3-31.

2.Anderson,T.andF.Elloumni.Theory and Practice of Online Learning.Athabasca:AthabascaUniversity,Canada,2004.

3.Clark, R. E. Media Will Never Influence Learning, 1993.http://www.usq.edu.au/material/unit/resource/clark/media.htm[9.6.2006].

4.Dagger,D.andV.P.Wade.Evaluation of Adaptive Course Construction Toolkit(ACCT),2004.http://wwwis.win.tue.nl/~acristea/AAAEH05/papers/6-a3eh_daggerd_IOS_format_v1.1.pdf[31.8.2006].

5. eEuroupe+2003.Progress Report — February 2004.http://europa.eu.int/information_society/eeurope/2005/doc/all_about/benchmarking/eeuropeplus_progress_report.pdf (4. 5.2006).

6.Eurostat — Statistical Office of the European Communi-ties. http://epp.eurostat.cec.eu.int/portal/page?_pageid=109 0 , 3 0 0 7 0 6 8 2 , 1 0 9 0 _ 3 0 2 9 8 5 9 1 & _ d a d = p o r t a l & _schema=PORTAL[21.1.2007].

7.HKGCC — Hong Kong General Chamber of Commerce.Edport.com. http://www.chamber.org.hk/info/member_a_week/edport.asp[2.6.2002].

8.IDC — Analyze for Future. Corporate Learning and Per-formance. An IDC Continuous Intelligence Service. http://www.idc.com/getdoc.jsp?containerId=IDC_P415 [19. 8.2006].

9.InternetTime Group. http://www.internettime.com/ [2. 6.2002].

10.InternetWorldStats.Usage and Population Statistics,2007.http://www.internetworldstats.com/[14.11.2007].

11.Mungania,P.Employees’ perceptions of barriers in e-learn-ing: The relationship among barriers, demographics, and e-learning self-efficacy — doctoral dissertation. Kentucky:UniversityofLousville,2004.

12.NSD—No Significant Difference Phenomenon.http://www.nosignificantdifference.org/[12.6.2006].

13.OECD.Glossary of Statistical Terms,2007.http://stats.oecd.org/glossary/(23.11.2007).

14.Overton,L.Linking Learning to business, Summary report 2004. BIZMedia. http://www.elearningage.co.uk/docs/LinkSummary.pdf[14.4.2006].

15.Russell, T. L. The No Significant Difference Phenomenon: A Comparative Research Annotated Bibliography on Tech-nology for Distance Education. IDECC — InternationalDistanceEducationCertificationCenter,2001.

16.SURS — Statistin̆i urad Republike Slovenije. Statistin̆i


letopis Republike Slovenije, 2002. http://www.sigov.si/zrs/leto02/kazalo02.htm[11.11.2003].

17.SURS — Statistin̆i urad RS. Vpis študentov na terciarno izobraževanje v študijskem letu 2005/06 — konc̆ni podatki. 13. 4. 2006. http://www.stat.si/novice_poglej.asp?ID=1027[29.30.2006].

18.Sulc̆ic̆, V., D. Lesjak, N. Trunk Širca and P. Purg. Keyissues of introducing blended learning in higher education.Proceedings of the IADIS international conference on Cognition and exploratory learning in digital age (CELDA2005), Porto, Portugal, December 14-16, 2005. [Porto]:IADISPress,cop.2005,pp.449-454.

19.Sulc̆ic̆, V. Is e-learning more suitable for full-time or forpart-timestudents?InTechnologies for Business Information Systems, ed. Abramowicz W., Mayr, H.C.. Heidelberg:SpringerVerlag,2007.

20.Sulc̆ic̆,V.Modelkombiniranegaelektronskegaizobraževanjavterciarnemizobraževanj—doctoraldissertation.UniversityofPrimorska,FacultyofmanagementKoper,2007.

21.TrunkŠirca,N.andV.Sulc̆ic̆.2003.LifelongLearningandHigherEducationInstitutions;fromStrategicPrinciplethroughImplemented Example to Systematic Solution? Knowledge

Society — Challenges to Management; Globalisation, Re-gionalism and EU Enlargement Process. 4th International Conference of the Faculty of Management Koper,Universityof Primorska. 20.-22. november 2003, Portorož. Koper:Fakultetazamanagement,pp.101-112.

22.Vasquez B. S. Linking Information Technology andPedagogical Innovation To Enchance ManagementEducation. ECIS 2000 — A Cyberspace Odyssey, vol. 2.Wien:Wirtscaftsuniversität.

23.Vehovar, V., V. Pehan, D. Lesjak and V. Sulc̆ic̆. RI — visokošolski in višješolski zavodi — e-izobraževanje 2005/ 2006, 2006. http://www.ris.org/index.php?fl=2&lact=1&bid=1293&parent=13[2.8.2006].

24.WDI—WorldDevelopmentIndicator.5.11TheInformationage. Pages 304-308, 2007. http://web.worldbank.org/WBSITE/EXTERNAL/DATASTATISTICS/ 0,,contentMDK:20394827~menuPK:1192714~pagePK:64133150~piPK:64133175~theSitePK:239419,00.html[14.11.2007].

25.Žižmond,E.andM.Novak.Controversiesoftechnologycon-vergencewithintheEuropeanUnion.Industrial management & data systems,2007,no.5,vol.107,pp.618-635.

Documents

E-LEARNING AND STUDY EFFECTIVENESS