Upload
others
View
9
Download
0
Embed Size (px)
Citation preview
MOOCRetentionRateandMotivationOndrejHenek
Thesis,M.Sc.ITProductDesign
1stJune2017 Supervisedby:SaluYlirisku
DepartmentofDesignandCommunicationUniversityofSouthernDenmark,Kolding
2
3
Thanksto
JakubRybář,SaluYlirisku,interviewparticipants,colleagues,teachers,friendsandfamilymembers
fortheirhelpandsupport.Printedby:Print&Sign,SDU
4
ABSTRACT......................................................................................................................................5
INTRODUCTION..............................................................................................................................6MASSIVEOPENONLINECOURSES.............................................................................................................7NOSTIS.................................................................................................................................................7VOCABULARY.........................................................................................................................................8
THERETENTIONRATE...................................................................................................................10MOOCSTUDENTSANDRETENTIONRATE...................................................................................................11REDEFININGRETENTIONRATE.................................................................................................................12
LONGTERMENGAGEMENTANDMOTIVATIONTHEORY...............................................................14UNDERSTANDINGMOTIVATION(MOTIVATIONTHEORIES,EXTRINSICANDINTRINSICMOTIVATION).....................14MOTIVATIONTYPESANDDIMENSIONS......................................................................................................14ENHANCINGMOTIVATIONUSINGGAMIFICATION........................................................................................16PITFALLSOFENHANCINGMOTIVATION.....................................................................................................18HOWOTHERSAPPROACHMOTIVATION?...................................................................................................18
METHODSANDDATA...................................................................................................................20THEFRAMINGOFTHISPROJECT...............................................................................................................20THEEXPLORATIONOFONLINEEDUCATION................................................................................................20THESURVEY.........................................................................................................................................21THEMOTIVATIONINTERVIEWS................................................................................................................21THEWORKSHOP...................................................................................................................................22OBSERVINGUSERBEHAVIOR...................................................................................................................22THEPROTOTYPETESTING.......................................................................................................................22
WHOARETHEMOOCUSERS?.......................................................................................................24USERSOFNOSTIS.................................................................................................................................24MOTIVATIONOFNOSTISUSERS...............................................................................................................27WHYDOPEOPLELEAVEANDNEVERCOMEBACK?.......................................................................................27
OPTIMIZATIONSFORRUSHERSANDSAMPLERS...........................................................................29THEEXPERIMENTRESULTS......................................................................................................................30
THEMOOCONBOARDINGEXPERIENCE........................................................................................32THEWIREFRAMEEXPLAINED...................................................................................................................32
THEMOTIVATIONHELPER............................................................................................................35THELACKOFTIMEPROBLEM...................................................................................................................35WHYMOTIVATIONHELPER.....................................................................................................................35
DISCUSSION.................................................................................................................................38THERESEARCHREFLECTION....................................................................................................................38OTHERCONCEPTSWORTHEXPLORING......................................................................................................38THEMOTIVATIONHELPER’SNEXTSTEP.....................................................................................................39
CONCLUSION................................................................................................................................40
REFERENCES.................................................................................................................................41
5
AbstractTheaverageretentionrateamongonlinecoursesisreportedtobearound10%,causingalotofcriticismofMOOCs.Buttheidea,thatonly10%ofusersaresuccessfulandtheremaining90%fails,iswrong.Thisthesisexploresthroughinterviewsandasurvey,whatdoesthe90%do, what are their motivations and behavior. Then, based on motivation theory andgamification, threeMOOCplatformoptimizations are designed: 1) Improved coursemainpage to convey all the important information the user needs to enroll. 2) Optimizedexperienceforpeoplefinishingacoursefasterthanusual.3)AMotivationHelperaimingtoenhancemotivation of users by a notification system. All these designs were tested andevaluated,theresultsandnextstepsarethendiscussed.
MOOCretentionrateogmotivering
Dengennemsnitligeopretholdelseomkringonlinekursererestimerettilomkring10%.Detteharmedført enmasse kritik omkringMOOCs. Ideenomkring at kun 10%af brugerneharsucces og at de resterende 90% fejler, er forkert. Gennem interviews og en rundspørge,udforskerdettespecialehvadderskermedderesterende90%.Hvaderderesmotivationerog adfærd? Baseret på motivations teorier og gamification, bliver tre MOOC platformsoptimalere designet. 1) Forbedre kursets forside så den giver al den vigtige informationbrugernebehøverforatdeltage.2)Enoptimeretoplevelseforpersonerderfærdiggøreetkursus hurtigere end normalt. 3) En motiverende hjælper der har til formål at forbedremotivationenhosbrugerneviaetnotifikationssystem.Alledissedesignserblevettestetogevalueret,resultaterneogdenæstenødvendigetrinbliverderefterdiskuteret.
6
IntroductionIn2014Ihaveco-createdtheNGONugisFinemwhichhasamotto“Thereisabetterlifeinmoreeducatedsociety”.Ourbiggestproject isthefirstpurelyCzechMassiveOpenOnlineCourseplatform–Nostis.DuringthetwoyearsofitsexistencewehavereceivedempoweringfeedbackfromouruserswhichkeptmeinvestingmytimeintoimprovingNostis.ButwhileIwasexploringthevastthemeofonlineeducation,IhaverepeatedlystumbledoveracritiqueofMOOCsbasedontheargumentthatinaverageonly10%ofenrolledstudentscompletethecourse.Thissoundedlikeanawfulnumberanditmademewonderwhatwashappeningwiththe90%.MorethanaresearchquestionIhaveconstructedaresearchdirection:
Whydopeoplesignupforanonlinecourse,whydotheyleaveandhow
canweencouragethemtostayandcompletethecourse?
ExploringandansweringthisquestionshouldhelpmycolleaguesinNugisFinemandotherMOOCcreatorsanddesignerstoprovidebetteruserexperiencetotheirusers.InthischapterIintroduceMOOCsandthenfocusonNostis.IalsoincludeavocabularyofthekeywordsusedintheMOOCworldandthisthesis.
ThesecondchapterisallaboutTheRetentionRate,whatitisandwhyitdoesnotmakesenseinthecurrentstate.ThefollowingchapterLongTermEngagementandMotivationTheorygivesatheoreticalintroductionintomotivationtheoriesandgamificationonwhichIbuiltmyenhancementsandprototypes.
Thenext chapterMethodsandData givesanoverviewofmethodsand tools I haveusedthroughoutmyresearchandhowIgatheredmydata.InchapterWhoAretheMOOCUsers?IdelveintotheworldofaMOOCstudent,exploringtheirmotivationsandbehaviorandwhypeopleleaveonlinecourses.
Following three chaptersOptimization for Rushers and Samplers, TheMOOC Onboarding
ExperienceandTheMotivationHelperintroducethewayIusedthemotivationtheoriesandtheresearchfindingsinordertoboostretentionrateandenhanceuserexperienceonNostis.IclosethisthesiswithaDiscussionevaluatingmyapproachanddescribingthenextstep,andConclusion,whereIprovideasummaryofthewholework.
“IfIciteanon-Englishtext,Iuseatranslationdirectlyinthetext,butthe
originalinnon-Englishcanstillbefoundinafootnote.”–Ondrej
Inthiskindofbox,Isharemyownexperiences.Whiletheydidn’tcontributedirectlytomyresearch,myownexperiencehasinfluencedmywork.
Chaptersummary
- Tomaintainanoverview,thereisaboxattheendofeverychapter,excepttheIntroduction,summarizingthecontents.
7
MassiveOpenOnlineCourses
MOOC1isaphenomenonofthe21stcentury,althoughtheideaofdistanteducationhasbeenaroundformorethanhundredyears.Theyear2012wascalledTheYearoftheMOOCbyTheNewYorkTimes(Pappano2012)becausetheleadinguniversitiesreleasedtheirfirstonlinecoursesandthepublicityoffreeonlineeducationboomed.Butaftertheinitialexcitement,alotofwell-groundedcriticismappeared.(J.Kim2012)
TheMOOCrevolutionwasmeanttodemocratizeeducationandbringtopknowledgetoallcornersoftheworld.Butthesehighgoalsarestillnotareality.Internetconnectionstableandfastenoughtoplayavideocourseisnotavailableinmanypartsoftheworldandthereis still the language barrier. Most of the popular courses are in English and althoughtranslations to various languages are sometimes present, the content is still notunderstandabletoeveryone.(Wildavsky2014;Glass,Shiokawa-Baklan,andSaltarelli2016)
AnotherconcernisaboutthequalityoftheeducationwithpeopleclaimingthattheMOOCversionofthecourseisinferiortotheon-campusone(Laurillard2014).Thecriticssay,thatfromapedagogicalperspective,MOOCsareonlyfocusedonthe“massive”andtheylackanypedagogicalinnovation.(Karsenti2013)
Anotherwaveofcriticismcamebecauseofverylowreportedretentionrate.Onlyaround10%ofsignedupstudentsfinishthecourse,sometimesdroppingtoastaggering2.5%(Hoetal.2014).Buttakingjustthisnumberwithoutthinkingaboutwhatitrepresentsisworthevenmorecriticism.MostoftheMOOCplatformsdonotofferanypreviewofthecontentwithoutsigningupandthesign-upbuttonalsoservesas“Remindmewhenthecoursestarts”givingtheactionseveralmeanings.
Nostis
NostisisaMOOCplatformpublishedinMay2015byNGONugisFinem2.ItsfirstcoursewasaboutFamilyLawandmorelaw-themedcoursesfollowed.Theplatformwasdesignedanddevelopedbymyself,thereforeIhavedeepinsightintohowitworksandaccesstoanydataavailable.Atthistime,wehave6coursesalready,5about lawand1aboutcreativity.WecollaboratewithprofessorsfromvariousCzechUniversitiesandexpertsinthefield.Asof23rdMay, we have 3404 registered users and 1237 certificates given among 4938 courseenrollments,thereforewecouldcalculate25%retentionrateacrossthe6courses.
1MassiveOpenOnlineCourseis:“AcourseofstudymadeavailableovertheInternetwithoutchargetoaverylargenumberofpeople.”(“MOOC”2017)2NugisFinemisaCzechNGOcreated in2015focusedoneducatinggeneralpublic.Czechwebsiteisathttps://www.nugisfinem.org
8
Figure1:DevelopmentofsignedupusersandcoursesafteracompleteredesigninJanuary2016untilMay2017.Everynew
coursebringsawaveofnewusers.
Vocabulary
SincethereareseveraltermsconnectedtoMOOCsthathavespecialmeaning,Iwouldliketointroducethemtothereadertoavoidanyconfusionfurtherinthetext.
MOOC
TheclassicalMOOCisservedonaninternetplatform,whereuserregistersandenrollsintoaMOOC.The coursehasa set startingdateandeveryweekanewblockof lecturesopenscontainingalsoaquiz.Afterscoringacertainnumberofpointsinquizzes,userisawardedbyacertificate.
Week/Block
Ablockisasetoflectures,oftencalledaweek,becauseusuallyeveryweekanewblockisaccessibletothestudent.
Lecture
ThebasicbuildingblockofMOOCs,usuallyinaformofvideo.Buttheselecturescanbealsographics,textsoradditionalmaterials.
Test/Quiz
Testoraquizisatypicalassessingtool.Userispresentedwithaquestionandmustchooseacorrectanswerfromgivenoptions.Aftersubmitting,user isawardedwithpointscountingtowardsacertificate.
Peerassessment
Peer assessment is another assessment tool with higher pedagogical value than a test(Kulkarnietal.2015).Ithasthreestages,oftenagaininaweeklyrhythm:Thefirstweekuserscompleteagiventaskbycreatingsomekindofcontent.(writinganessayorcommentary,
0
500
1000
1500
2000
2500
3000
3500
Nostistimeline
Users Courses(scaled)
9
drawingdiagrams, takingpictures,makingavideo…)The secondweek theygrade severalworksofotherpeersbygivencriteria.Andinthethirdandfinalstagetheusercanseetheirowncontentgradedbyseveralotherstudents.
Figure2:OneofthecoursesonNostis.Thecoreelementsaretitle,descriptionandanenrollbutton.Belowontheleftisalist
oflecturesdividedintoblocks.Ontheright,thereisanoverviewofpointsearnedshowingprogresstowardsthecertificate,
alistofcollaboratinginstitutionsandalistoflecturers.
10
TheRetentionRateTheretentionrateisformethemostinterestingphenomenonintheworldofMOOCs.Itisaveryloudlytalkedaboutbututterlymisunderstoodmetric.Asstatedintheintroduction,mostreports, papers and criticism isusing the value10% retention rate. (Hood, Littlejohn, andMilligan2015;BelangerandThornton2013;Adamopoulos2013)ManystudiesevendefinethetopicofMOOCswiththisnumber,copyingitfromothers,withoutconsideringhowthemetriciscreatedandtheimpactthishasondecisionmakersatuniversities.
“Notably,MOOCshavebeenplaguedbyextremelyhighdrop-outrates.”
(Zhengetal.2015)
The usual way to count retention rate is to take the number of successful students(completingcriteriaforacertificate)anddividethembythenumberofpeoplewhoenrolledto the course. The act of enrollingmeans clickingon the “Enroll” buttonbyuserwithoutknowingtheintentionsoftheuser.
Theretentionrateisoftencomparedtoenrollmenttoofflineuniversities,butthishasmanyproblems. Insheerabsolutenumbers,even1%retentionrate inaMOOC,where100000studentsenrolled,therefore1000studentsfinished,isagreatsuccesswhereclassicuniversitycourseneedsoftenseveralyearstoaccomplishthesame.WhereMOOCenrollmentrequiresemailaddressandoneclick,classicuniversitieshavetoughprerequisitesandvariousfees.ThereisalsonopenaltyinfailingaMOOC,whereasstudentsofclassicuniversitieshavetoinvestmuchmore timeand resources into thecourseand failing is thereforemuchmorecostly(Hoetal.2014).
From another perspective, continuous enrollment also skews the data. As those freshlyenrolledcannotobtainacertificateyet,buttheyappearinstatisticsnevertheless.Thishascausedatemporarydropinretentionratetoastunning2.5%onEdXplatform(Hoetal.2014).
Universitydecisionmakers
TheMOOCs on Nostis aremostlymadewith direct collaborationwith teachers.We hadproblemsnegotiatingwithauniversityasthewhole,becausethekeydecisionmakershavealmost no information aboutMOOCs and the easy to find criticism is often including thismythicalretentionratesasanargument.
11
Figure3:StudentpersistenceinBioelectricitycourseFall2012.(BelangerandThornton2013)Atypicaldistributionofstudent
engagementinaMOOC.
Moocstudentsandretentionrate
Theoriginalapproachtoretentionratehasverynarrowsuccessconditionstailoredjustforonekindofmotivation:Tocompletewholecourseandearnacertificate.Buttherecouldbemanykindsofmotivationtoclicktheenrollbutton.Thoseareforexample:
1. Watchlectures,doexercisesandearncertificate.(Theclassicalapproach.)2. Onlywatchlecturesandlearnwithoutexercises.3. Accessonlysomematerialswithinacourse(lecture,pdf,recommendation,forum…).4. Lookatwhatthecourseislike.5. LookatwhatMOOCslooklike.6. Beremindedofthestartofthecourseandthendecidewhetherdoitornot.
“Littleresearchattentionhasbeendirectedtothepopulationofstudents
whodonotfinishthecoursesinwhichtheyregister;asaresultitisdifficult
tofullycapturetheuserexperiencestiedtoretentionproblems,andmany
importantreflectionsaremissed.”(Zhengetal.2015)
Astudyofdisengagementfrom3coursesshowedaninterestingbehaviorofMOOCstudents.Kizilcec,Piech,andSchneider(2013)identified4basicsubpopulationsofMOOClearners:
1. ‘Completing’:learnerswhocompletedthemajorityoftheassessmentsofferedinthe
class.Thoughtheseparticipantsvariedinhowwelltheyperformedontheassessment,
theyallatleastattemptedtheassignments.Thisengagementpatternismostsimilar
toastudentinatraditionalclass.�
2. ‘Auditing’:learnerswhodidassessmentsinfrequentlyifatallandengagedinsteadby
watchingvideolectures.Studentsinthisclusterfollowedthecourseforthemajorityof
itsduration.Nostudentsinthisclusterobtainedcoursecredit.�
3. ‘Disengaging’:learnerswhodidassessmentsatthebe-ginningofthecoursebutthen
have a marked decrease in engagement (their engagement patterns look like
Completingatthebeginningofthecoursebutthenthestudenteitherdisappearsfrom
12
the course entirely or sparselywatches video lectures). Themoments atwhich the
learnersdisengagediffer,butitisgenerallyinthefirstthirdoftheclass.�
4. ‘Sampling’: learners who watched video lectures for only one or two assessment
periods(generallylearnersinthiscategorywatchjustasinglevideo).Thoughmany
learners“sample”atthebeginningofthecourse,therearemanyothersthatbriefly
explorethematerialwhentheclassisalreadyfullyunderway.
Thedistributionoftheseparticipantsvariedamongthe3coursesallwithdifferentdifficulty.
Figure4:Auserbehaviorbasedoncoursedifficulty.HS=highschool,UG=Undergraduate,GS=graduate
Mostinterestingformeisthesizeofsamplinggroupinrelationtocoursedifficulty.Thisresultsuggests, that over 74%probably just looked at several videos anddecided to leave thecourseforsomereason.Wecanalsoseethat inthedifficultcourse,morepeopleare justwatchinglecturesandtrytogettheknowledge(auditing),whilelesspeoplefinishthecoursewhilestayingontrack,completingassessmentsandgettingcertificate.
Thisbehaviorwasexploredfurtherbyanotherstudy(Zhengetal.2015)wheretheintentionbehindsigningupwasexploredbyconductinginterviews.Theresultssuggest,thattherearestudentswhowanttotakeonlypartoftheknowledgeinsideacourse,socalled:“learningondemand”.
“IamonlyinterestedintheAmericanhistoryduringtheColdWar,Ionly
studiedthoserelatedlecturesandskippedothers.ThatwaswhyIdidn’t
finishthecourse.Ihavealreadyreachedmygoal.”(Zhengetal.2015)
Redefiningretentionrate
Thoseexamplesofmotivationsandbehavior,different thanexpected linearadvancementtowards certificate, made me wonder whether the problem with retention rate is notsomethinginMOOCs,butthemetricitself.Althoughitmightsoundlikecheating,changingtherulesformeasuringretentionrateandsuccessofusersmakesalotofsenseinthiscase.
InspiredbyJobstobeDoneframework(Christensenetal.2016)Iwanttoselectandworkonly with those people, whose job to be done is to finish the whole course and earn
InlastfewyearsItriedtodo2courseswhereIhavefailedonpaper,butIhavefinishedthemanyway.IncoursesLearningHowtoLearnandGamification,IstoppeddoingexercisesatsomepointandthenIalsostoppedauditing,becauseIdownloadedtheremaininglecturestowatchitlater.Ileftthecoursebecauseoflackoftime,butIhavereturnedtothecontentlater.Thus,learningeverythingfromthecourse,butnotcompletingitinthegeneralsense.
13
certificate.Thenmyretentionratewouldshow,howmanypeoplehadtheintentiontofinishthecourseandthensucceeded.Andmyaimistohelpthoseuserstostayontrack.
Butwhatother jobstobedonearesatisfiedordissatisfiedbyNostis?AndshouldI ignorethose, who do not have the intention to earn certificate? To find an answer, I had tounderstandourusersandhowmotivationworks.
Theretentionratesummary:
- MOOCsareoftendefinedbylowretentionrate,whichmakesithardforourNGOtonegotiatesupportfromuniversities.
- Studies show, that MOOC users have different motivations and differentbehavior.
- Retentionrateiscalculatedbynumberofpeopleenrolleddividedbynumberofcertificatesgiven.Thisworksonlywithafractionofusersignoringmanydifferentmotivationsandbehaviors.
- Theretentionrateisflawed.Itdoesnotrepresentwhatitshouldbe.
14
LongtermengagementandmotivationtheoryEveryactionandbehaviorhavesomekindofmotivationbehindit,andenrollingintoacourseisnotanexception.Inthischapter,Ilookatthetheoreticalbasisofhowhumanmotivationworksandhowtoempowerit.ThenIdelveintogamificationtofindconcreteframeworksandtools,thatcanhelpmyusersachievetheirgoal.
Understandingmotivation(Motivationtheories,extrinsicandintrinsicmotivation)
Thefieldofmotivationaspartofthedisciplineofpsychologyhasmaturedovermanydecadesincludingwell-knownexamplesasPavlov’sconditionalpsychologyexperimentsorMaslow’spyramid.Motivation has as many definitions as points of view and various theorists areinterestedindifferentaspectsofmotivation.
Amotiveiswhatpromptsapersontoactinacertainwayoratleast
developaninclinationforspecificbehavior.–Kast’sandRosnzweig’sdefinition(Pardee1990)
Motivationisultimatelyderivedfromatensionthatresultswhenoneor
moreofourimportantneedsareunsatisfied.–Dessler’sdefinition(Pardee1990)
Tobemotivatedmeanstobemovedtodosomething.Apersonwhofeels
noimpetusorinspirationtoactisthuscharacterizedasunmotivated,
whereassomeonewhoisenergizedoractivatedtowardanendis
consideredmotivated.–(RyanandDeci2000)
IchosetousetheoriesofRichardM.RyanandEdwardL.Deci,whoareoftenmentionedincontext of gamification, motivation at work or in education. They respond well to theobjectionsofpreviousgenerationsofpsychologistsandtheirideasmadesenseincontextofmy research. (This chapter tries to explain a combination of Self-Determination Theory,Organismic Integration Theory and Cognitive Evaluation Theory. Together these theoriesdescribemotivation.)
Motivationcanbedividedintoextrinsicandintrinsicmotivation,whereintrinsicmotivationrefersto“doingsomethingbecauseitisinherentlyinterestingorenjoyable”whereasextrinsicmotivationrefersto“doingsomethingbecauseitleadstoaseparableoutcome”.(RyanandDeci2000)Motivationisnotflat,butamultidimensionalcombinationofvariousaspectsandminor motivators. It is a relation between an actor and activity and cannot be easilygeneralized.
Motivationtypesanddimensions
Intrinsicmotivationisoftenperceivedasthestrongestmotivation.Itwasfoundinpeopleandinanimalsanditisaboutlearning,trying,andplayfulcuriosity.Itcannotbecaused,butratheris facilitatedandcatalyzed.Thereareseveraltypesofextrinsicmotivation.Theyareoftenseenasthealienatedandalmost“negative”types,butthecaseisnotblackandwhite.Sincetheendofourchildhoodtheintrinsicmotivation(carelessexploration,purefunandjoy…)isoftenpushedasidebyextrinsicones(money,status,socialdemands,expectations),butthatdoesn’tmeanthatourlivesaredrivensolelybycarrotsandsticks.
15
Figure5:Visualizationofmotivationcategoriesincluding4typesoftheextrinsicmotivation.
OnFigure5wecanseethedistinctionbetweenamotivation,whichleadstonoaction,fourtypesofextrinsicmotivationandanintrinsicone.Ingeneral,themorewemovetotherightonthediagram,thestrongermotivatorsthereare.Amotivationtowardsanactivitycanshiftleftorrightintime.Thesemotivationtypesareusuallyinterwovenbecauseanactivitycanbedividedintosmallersub-activitieswithcombinationofmotivations.Forexample,anactivityof readingahistorybookcanbedivided into sub-activitieswithdifferentmotivations likesittingdownbecauseof tiredness, learningabouthistory for fun,acquiringknowledge forhistoryclass,finishingabooktocompetewithafriend’sreadingchallenge,etc.
Butmotivationismultidimensionalanditisstronglyconnectedtothethreefollowingaspectsthat play a critical role: Relatedness, Competence and Autonomy. These aspects aremaintainingintrinsicmotivationandenhancingextrinsicmotivationwhenpresent,theycan
Ifoundaniceanalogyofthemotivationtypesinreadingabook.Thiswillprobablyapplyonly to me personally, but I hope it will explain the theory better. I am intrinsicallymotivatedtoreadagoodfantasynovel,becauseIfinditfun.Thereisnoexternaloutcome,justajoy.Ialsoenjoyreadingaboutdesignwhichisstillfun,butpartlyIwanttobeabetterdesignerandchangetheworld(integratedgoal).IhavealsoreadabookaboutSpanish,whichisalanguageIwanttoimprove,becauseIwanttotravelanditwouldbequiteuseful(identification with self-set goal). I have started reading Lord of the Rings, becauseeverybodyaroundmehavealreadydidthatandIwantedtobeincludedinthediscussion(approvalfromothers).Ihavereadfewchaptersofabookfrommandatoryreadinglistinhighschooltonotfailanexam(extrinsicpunishment).Andfinally,IdidnotreadFiftyShadesofGreybecauseIhadnoinnerinterest,itdoesnotalignwithmyvalues,goals,mysocialcircledoesnotappreciateitandIwouldgetnorewardwhatsoever.
16
alsohindermotivationwhenabsent.TheRelatednessisinplaywhentheactivityisvaluedbysignificantothers,whotheactorfeelsorwouldliketofeelconnectedto.Itcouldbefamily,peergrouporasociety.Ifanactorcanrelatetosomebodywhoseesvalueintheactivity,themotivationgrows.ThesenseofCompetenceisaboutbalanceoffeelingefficacioustowardsanaction.Forexample,ifastudentfeelsthatanactivityistoohard(thestudentfeelsnotcompetentenough),themotivationtodoitdecreases.Butontheotherhand,ifthetaskistooeasy,themotivationalsodeclines.Thebiggerthegapbetweenperceivedcompetenceandthedifficultyofatask,thegreaternegativeimpactithasonmotivation.AnotheraspectisAutonomy,wheremoreautonomyleadstoaninternalizationandintrojectionofagoal,meaningthatthetypeofmotivationcanshifttowardstherightsideofthediagramandgetstronger.Iftheactorfeelsthatherbehavioriscontrolled,thenhermotivationismorelikelytodroptoweakertype.
Thismotivationtheoryexplainswellwhyandhowsomeactivitiesaresoengagingandfuntodo, but does not offer any direct tool for motivation enhancement. For this, I chose toresearchafieldofgamification,thatstudiestheengagingaspectsofgames.Itbuildsontopof the extrinsic and intrinsic motivation theories, but offers frameworks and tools forenhancingmotivationasgameelements.
Enhancingmotivationusinggamification
Intrinsicmotivation,asthestrongestmotivator,isallaboutfunandenjoyment,whichisthebasicbuildingblockofgames.BasedonRaphKoster,gamesarebasicallysystemsofpatterns,whichpeopleenjoy learningandmastering.Ourbrain iswired inaway, thattoachieveabettersurvival,itrewardsusforlearningandmasteringpatternsandskills.(Koster2013)
Figure6:JoKim'splayercategories(2012)
Butwhatmakesgamesmorefun,thanreadingatextbook?Thegamificationstudiesrevealthebasicideasbehindgamesandhowtheyworkwithhumanmotivations.Butnoteveryoneenjoyseverygame.AgamedesignerJoKimcreatedaframeworktodividepeopleintofour
17
categories based on their motivations in playing games. (A. J. Kim 2012) And comparingvariousMOOCmotivationstudies,IcanseealinktoMOOCs.AdesiretoCompeteissolvedby earning a certificate with a score mark, to Explore means having “collected” all theadditionalmaterialsandtoCollaboratemeansparticipationindiscussionandorganizationofstudygroups.Weareusuallytryingtocoveralltheaspectstoengageallplayertypes.ItseemsthatthereisaroomfortheExpressmotivatorpossiblyinassignmentsortasksinsideaMOOC.
EverygameisbuiltofGameElements,whichcouldbebrokendowntoapyramidof3levels.Inthehighestlevel,thereareGameDynamics,whichengagethecoremotivationofaplayer.It could be thought as a “grammar” of the game and it is connected to the player typesmentioned above. Dynamics could be about relationships, narratives, progression,exploration… In themiddle areGameMechanicswhich couldbe thought as “verbs” of agame.Theyaretheconcreterulesrepresentingagamedynamic.Thosecouldberewards,transactions,turns,challenges,cooperation,feedback.AndinthelowestlevelareconcreteGameComponentswhicharethemostbasicbuildingblocksofgames.Wecanthinkofthemas “nouns” which are representing and executing mechanics having the higher dynamicabove. These are for example: points, inventory, dice rolls, leaderboards, achievements,progressionbars,bossfights,maps,power-ups,superpowers,avatars,skills,health…
Figure7:Pyramidofgameelements
Engaging games are full of activity loops. These are either engagement loops, where amotivationleadstoanaction,thatgivesafeedbackwhichenhancesmotivation,seeFigure8.And then there is a progression loop, where player should have a sense of progressionthroughthecontent.Thesmallerstepstherearetotake,thelessoverwhelmingthechallengeis.Progressionloopisoftenimplementedasasetofsmallerchallengesdividedbycheckpoints(smallprogressionloops)andendingwitha“bossfight”.Afterdefeatingthelastchallenge,youprogresstoanotherlevel(biggerprogressionloops)andsoon.
18
Figure8:Engagementloop:Aninitialmotivationcausesandaction,whichresultsinafeedbackenhancingmotivationagain.
Therearemanyotherinterestingwaysinwhichgamesenhancemotivationofaplayer,thatcouldbetranslatedtoMOOCs.Forexamplestorytelling(Altarriba2014),whichcouldbeusedto link lectures together andmake the user curious about how it unfolds, progress barsshowinghowmuchtheuserhasaccomplished,collectibleachievementsorbadgesandmuchmore.Butinthecontextofthisthesis,Ichosetodescribeonlythemainprinciples,whichareimportanttounderstandmydesign.
Pitfallsofenhancingmotivation
Whilegamificationintroducestomyresearchaframingoftheconceptoffunandgivesmeaframeworktoenhancemotivation,therearealsopitfalls,thatgreatlydiminishthemotivationand must be avoided. As positive performance feedback enhances the more intrinsicmotivation(e.g.praising“goodjob”,givingthumbsup),anegativefeedbackjustmakestheuserfeellesscompetentandlowersthemotivation(e.g.emphasizingthewronganswersinatest result). Therefore, the activity loopsmustbeexecutedwell to support the feelingofmastery.
Lookingbackattheaspectofautonomy,therearemorethingstobecarefulabout.Examplesofcontrollingbehaviorwhichlimitsautonomyandthereforehindersmotivationare:tangiblerewards,directives,competitionpressure,threatsanddeadlines.Unfortunately,gamificationis often deployed only as points, badges and leaderboards (PBL) focusing on extrinsicmotivationandusingthesepotentiallydangerouscontrollingelements(Trenholm2015).Thiscanleadtoanoverjustificationeffect(Carlsonetal.2009),whereanintroductionofrewardsreplacesintrinsicmotivationbyanextrinsicone.Furthermore,withdrawaloftheserewardsdoesnotshift theextrinsicmotivationbacktoan intrinsicone,butoverall themotivationdiminishesastheserewardsareremoved.
Howothersapproachmotivation?
Gamificationisnowleadingtheonlinemotivationtrends(Deterding2012),hencethereisalotofresourcesavailable.Butitisnoteasytodothegamificationrightandtheinternetisfilledwithmanysimplifiedguidestogamificationwithoutdeeperunderstandingoftheissue.SomereportsexistongamificationincontextofMOOCs,butthisdevelopmentisveryrecentand studies are scarce. A review of several MOOC platforms (Hansch, Newman, andSchildhauer 2015) suggest, that PBL is the most widespread instrument of enhancingmotivation.AdetailedreportfromOpenHPIplatform(Willemsetal.2014)standsasagreatexampleoftheaforementionedreview.
19
TheapproachofOpenHPIplatformispartiallyoneofthose,whichItrytodifferentiatemyselffrom.Theiruseofrewardsisontheedgeofcontrollingbehaviorandthepoints,obtainedbyavarietyofverydifferentactions,arejustemptynumberswithoutanyfurthermeaning.Thepointsdonotrepresentamasteryofauser,theydonotrepresenthelpfulness,theyareablendofeverythingandnothing.Therefore,aleaderboardbasedonthesepointsdoesnotmake sense, too. Unfortunately, they have only designed their system and tested ittheoretically on personas. The actual result of their modifications, while being the mostinterestingpart,isnowheretobefoundevenaftertwoyearsafterpublishing.
AninterestingandinspiringuseofPBLhasStackOverflow3,oftenpraisedasagoodexampleofgamification(Deterding2011).Thepointsandbadgesrepresentdirectlyareputationofauser.Theonlywaytogetrewardsisbeinghelpfultoothersbyansweringquestionsthebestyoucan.Therefore,thegamificationhereemphasizesanaspectofrelatednessandamplifiesstrongertypesofmotivation.
“Likeanythingelse,gamificationcanbedonewellandcanbedonepoorly.Companiesthat
see the success of Foursquare or Farmville and just try to copy what they see, without
understandingthestakeholdersandtheirneeds,aredestinedtofail—asarethosewhothink
they can replace meaningful rewards and incentives with empty points and badges, and
manipulateusersintodoingthingstheydon’twanttodo.”-(Deterding2012)
3MoreonStackOverflowgamificationathttps://blog.codinghorror.com/the-gamification/
Themotivationtheoriessummary:
- Motivationofapersontowardsacertaintaskcouldbeextrinsic(4subtypes)orintrinsic(fun).Themotivationisstrongerthemoreintrinsicitis.
- Motivationismainlyinfluencedby:Relatedness,CompetenceandAutonomy.- Gamificationenhancesmotivationusingprincipleslike:
o Playertypes–differentfunfordifferentpeople.o Activityloops–toshowprogressandenhanceengagement.o Gameelements–frameworksandtoolstoenhancemotivation.
- CurrentresearchexploresuseofgamificationinMOOCplatforms,butoftenonlyPBLisused.Thismightleadtooverjustificationeffectandafailure.
20
MethodsanddataThischapterclarifiessome“how”and“why”questionsconcerningmyresearch.FirstIframethescopeofthisthesistomakeclearwhichaspectsofaMOOCIfocusonandwhichIneglect.Then Idescribemyapproachtotheresearch Ihavedoneandwhatmethods Ihaveused.While using variousmethods throughout the research, I havebeen analyzing thedatabywritingthistextandfindingrelevantliterature.
Theframingofthisproject
AsaprogrammerandwebsiteadministratorofNostis,IcanmakeanymodificationstotheplatformofNostis,thereforeinthecontextofthisworkIchosetoneglectthecontentandfocusontheplatform.Idonotdenytheimportanceofthecontent,contraryIthinkitisthemost important part ofMOOC, but I do not have access to the course lectors, the videoequipment,norIhavecompetencyincreatingaMOOC.Anotheressentialaspectofanonlinecourseisthelearning.Buttolearn,peoplemustvisitthecoursewebsitefirst.Therefore,myfocus is purely on motivation and encouraging particular behavior, while neglecting thepedagogical sideofMOOCs,which is still very interesting,butoutsideof thescopeofmythesis.
Figure9:Analysisofthe15interviewsbyextractinginterestingthemes.
Theexplorationofonlineeducation
Ihavestartedmyresearchatthebeginningof2016togetherwithmyclassmateJakubRybářwithadeskresearchandthen15exploratorysemi-structuredinterviews.Wewerelookingathow people learn online, what services they use and how they use them.Wewere alsotouchingtheirofflinelearningbehavior,butmostofourfocuswasontheirexperiencewithonlinelearningandMOOCs.Ourrespondentswerebetween20and28yearsold–universitystudents or fresh graduates.We conducted Skype interviewswith 9 of our Czech friends
21
rangingfrom30to80minutesand6shorterinterviewsfacetofacewithDanishstudentsorfriends.
Whilegatheringdataandre-listeningtotheinterviews,wecodedthereportedexperiencesintothemesandputthemonawall.Anaturalisticapproachofanalysis(Silverman2013)gaveusbasicoverviewofhowstudentsbehavewhilelearningonlineandtheirroughmotivations.
Thesurvey
TogetbetterunderstandingofusermotivationspecificallyonNostis,Icreatedasurveyforourcourseparticipantswheretheycouldselectmultiplereasonsfortheircourseenrollment,select their job status, field and education. This survey was based on our exploratoryinterviews andothers’ research in this area. (Breslowet al. 2013; Belanger and Thornton2013;Zhengetal.2015)IwantedtocompareouraudiencetootherMOOCsandresearchsoIincludedageintothesurvey.ItisveryeasytodeterminegenderofCzechsusingtheirnamesoIdidnotbotheruserswiththisquestion.
Figure10AlistofsurveyquestionstranslatedtoEnglish.
Respondentswere filling thesurveyduringacourse.Thesurveywasdisplayedat randomlecturehavingonly2questionsfromtheset.ThiswayItriedtominimizethedisruptionofuserflowandbyrandomizationinlecturestogetbalancedresultsoftheprogressionwithinthecourse.About1000usersparticipatedbutnoteveryonefilledallthequestions.Ianalyzedthemotivationquestionsafter158answersanddiscontinuedthispartofthesurveybecauseitdidnotmakesense.
Themotivationinterviews
The motivation results from the survey were fuzzy so I wanted to clear up my insights.BecausetheNostisusersarescatteredallaroundCzechRepublicIconducted4cellphoneinterviewswiththosewhohaverecentlyenrolledintoacourseandre-analyzeonepreviousinterview.While listening to the recordings, I took the constructivist approach, lookingat
- WhyamIhere?� Iwantacertificate� Iwantknowledge� Outofcuriosity� Becauseofwork� Becauseofstudies
- HowoldamI?- Myhighestachievededucationis:
o Primaryschoolo Highschoolo University
- WhatdoIdo?� Study� Work
[Inwhatfield?]
22
varioushints,pausesanddifferentrolesthattheparticipantstookthroughouttheinterview.Ifoundthatthesurveywastoostrictlyimposingourpre-definedcategoriesontotheusers.The motivations from the interviews were blending together having maybe one primarymotivation,butnotdenyingothermotivations.Intheinterviews,Iaskedifpeoplecoulddoanyonlinecoursewhatsoever,whatwoulditbeabout.Mostlytheytoldmeaboutatopicrelatedtotheirwork,whichtheyalsoenjoyedandfoundfuntolearnabout,thuscombiningallthemotivationcategoriestogether.
Theworkshop
IorganizedaworkshopwithstakeholdersfromNugisFinemwhereIwantedtomakesenseofmyresearchsofarandtogatherideasaboutthenextstep.Withthose5people,Istartedbypresentingmydatasofar,whileansweringtheirquestions.Onewastrulyinspiring:
“Dopeoplegothroughtheweeklyblockoflecturesinonesitting?Anddo
theywanttofinishawholecourseinoneweekend?”
WesharedourexperienceswithMOOCs,whatmadeusstayandwhatmadeusabandonit.Basedonourfindings,weideatedwaystomakepeoplestaywithacourse.WethenfocusedonNostisandwhatIcoulddointhescopeofthisthesis.
Observinguserbehavior
AbigpartofmystudyusesquantitativedatagatheredfromNostisdatabaseandanalyticaltools.Nostiscollectsdatesandtimesofeverycompletedlecture,course,earnedcertificateandtestattempt,thereforethereisahugeamountofdataforanalysis.Tomakesenseofallthisdata,IusedadirectSQLcommandstocountnumber,atrialversionofTableausoftwareandMicrosoftExceltovisualizeandanalyzethedata.Ialwaysposedaquestionandthenusedthedatatogetananswer:
- Howoftendopeoplewatchlecturesandinwhatpatterns?- Whichdaysofaweekandwhichtimesofaday?- Dothewatchmorelecturesinaroworjustfewlecturesperday?- Howquicklydopeoplefinishacourse?
Theprototypetesting
With the Motivation helper, I started by sketching the interface on a paper and thenintroducingthelastoneto2ofmyfriends.Imadethemreadthecontentandaskedthem,whatdotheythinktheyarelookingatandhowcouldtheyuseit.Thisusertestsgavemeapositiveresultregardingunderstandabilityandusability.
Ontheotherhand,oneofthemtoldmethatthefunctionalityisnotsufficientandshewouldneedadditionalfeaturesmoreresemblingacalendar.Thismademewonderwhoisexactlythetargetgroupofthishelperandhowbigitis.Idecidedtomakeafull-scaletest–toputtheprototypeonthelivewebsiteandseehowmanyuserswoulduseit.Unfortunately,Igotstuck in development and couldn’t introduce the tool. I describe the prototype in detailincludingthenextstepinthechapterMotivationhelperonpage35.
23
Figure11:Asketchofthemotivationhelperwithroughintroductionofthetoolandaninteractivetablewhereuserscan
selectthetimeofnotification.
Themethodsanddatasummary:
- This project is focused on improvements to MOOC platform Nostis to boostretentionrateusingmotivationtheories.
- Projectstartedwith15exploratoryinterviewsaboutonlineeducation- Thedemographicdataarebasedonasurveyof1000participants;Motivation
questionswereansweredby158people.- Iconducted4interviewstoclarifyusermotivation.- Anideationworkshopwith5stakeholdersshowedadirectionofmyprototypes.- UserbehaviorresultsarebasedondatafromdatabaseandGoogleAnalytics.- Itestedtheprototypesandimprovementswithabout10peopleinshortsessions
gettingtheirimmediateimpressionandfeedback.
24
WhoaretheMOOCusers?ToincreasetheretentionrateinthewayIhavespecified,Ihadtofamiliarizemyselfwithusersand understand their intentions. My main interest was their motivation and goals. Tocomparemy own findingswith different research reports in this area,mostwith Englishspeakingaudience,Iwasalsogatheringdemographicdata.Theresultsweresynthesizedfrom15exploratoryinterviews,anonlinesurveywithNostisusersand4follow-upinterviews.
UsersofNostis
AbriefanalysisofnamesofNostisusersshowsthattherearearound60%womenand40%menregistered.ThemedianageofNostisusersis26,andthehighestachievededucationisdominatedbyhigh school finishers (55%)and followedbyuniversitygraduates (36%)andpeople with basic knowledge have only 9%. But the most important discovery was thelocationofusers.
Figure12:MapofCzechRepublicshowingnumberofsessionsonnostis.org.Thedataarefromtheyear2016wherethe
audiencewasmostlyfromOlomoucarea,followedbySouthernMoravia.
Themaps(Figure12andFigure13)showinadequatedistributionacrossthecountry,whereprimaryregionsareOlomoucandSouthernMoravia. I ignorePragueonpurpose,becausemanyinternetserviceprovidersformobileconnectionarelocatedthere,hencethelocationofthesessionisalsoreportedasPrague.Thiscanhappenonpublicwifisinbusesandtrainsandmost importantly onmobile connections and similar networks without a permanentlocation.
TheregionsofOlomoucandSouthernMoraviaarealsotheregionswheremostoftheNGOmembers live andwork or study. SinceNostiswasmostly promoted by sharing on socialmediaamongfriendsandfamily,theresultsshowthelocationwhereNugisFinemfriendsandfamily are located. Also, Nugis Finem and its project Nostis are well known at PalackyUniversityofOlomouc,thereforealotofvisitsarefromOlomoucitself.
25
Figure13:Themapisfromperiodof16
thJanuary2017to16
thApril2017anditshowssessiondominanceinSouthernMoravia.
Atthebeginningofthisyear,welaunchedacoursewithMasarykUniversityinBrno(SouthernMoravia)asapartofanofficialuniversitycourse.Therefore,mostofthesessionsthisyeararefromBrno,wherestudentswereattendingthiscourseeveryweekfornewcontent.
Looking at the fields of occupation across both students and workers, next to theapproximately 20% of law students, we have a big group of users with social and civiloccupations,whicharestronglytiedtolaws.Iwasunabletofindanytraceofthisamountoflawstudentsinanyotherstudysuggesting,thatthethemeofacourseisbasicallydeterminingtheaudience.
Figure14:Surveyresultsforoccupation.Mostusersarefromthefieldoflaw,butmanyusersalsocomefromsocialorcivil
occupations,wherelawplaysabigrole.
To conclude the demographic study, I can say that the audience of Nostis courses isdeterminedbythetopicofthecourseandmarketingefforts.Thecomparisonwithstudiesfrom world-known platform containing various types of courses therefore did not make
0
10
20
30
40
50
60
70
Workonly Studyonly Work&study
Fieldsofworkorstudy(orboth)
Law Socialworks Civilservice Other Unknown
26
sense.Onepatterndiscovereddidnotmakemuch sense tome, though.Whydo the lawstudents do courses about law? The assumption was, that these users already have theknowledgeandthereforetheyarenotinterestedinNostis.Basedonthedata,Iknewwhatkindofrespondentstochooseforfurtherinterviewsandclarificationofmotivations.
Figure15:Weeklyhoursofcompletion.OnXaxisarehours,onYaxisweekdaysstartingwithSunday
TogetadeeperunderstandingofNostisusers,Ilookedattheirbehaviorpatterns.TheFigure15showspeeksofactivityonSundayafternoon,whendeadlinesareclosing,andonMondayandTuesdayafternoons,whennewcontentisavailable.Thursday,FridayandSaturdayarequitecalmincompletedlecturecount.Takingthistoanextlevel,Icouldobserveandidentifypeople,whoarehighlymotivatedtolearnandwatchlecturesrightwhentheyareavailable,orthose,whostruggletokeepwiththepace.TheFigure16suggests,thattherearetwoverydifferent learningstyles.Somepeoplespreadthe lecturesthroughentiredaysandweeks,completingonelectureatatime,andontheotherhandthereisagroupofpeople,whofinishawholeblockoflecturesinonesitting.
Figure16:Graphshowinglecturecompletionperhouronrandomsampleofusers.Bluelineisonelectureinanhour,yellow
2andthemorered,themorelectureswerecompletedperonehour.Thisdisprovesthehypothesis,thatmostpeoplewatch
severallecturesinarow,butbothbehavioralgroupsaremoreorlessequalindistribution.
27
MotivationofNostisusers
Inthesurvey,therewasamultiple-choicequestionaboutusermotivation.Thisattempttosortusers’motivationintocategoriesusedquestionsfromsimilarstudies(Zhengetal.2015).Asresults,39%ofresponsesincludeddesireforknowledge,24%wantedacertificate,20%wascuriosityand18%wasconnectedtostudyorwork.Myassumptionwas,thatthiswouldclearlydefineusergroups,butreportedmotivationswereblendedtogetherandnopatternwasclearlyvisible
Ahypothesisemergedthatthesurveycouldbemisinterpretedandtheresultsarenotcorrect.IrealizedthatevenIcouldn’tproperlyanswerthequestionandforeveryMOOCmyanswerwouldbeslightlydifferent.Therefore, Idecidedtoclarifythe ideasaboutmotivationwithmoreinterviewsonarandomsampleoffreshcourseparticipants.ThisgavemebetterinsightsintowhatpeoplewanttodoonNostis:
Dana–ApoliceofficermostlydoingpaperworkaboutlawsawNostisinTV,shelookeditupandtriedeverycourseavailable.Shefindsitinterestingbecauseofherworkbutshefindsitusefulinhereverydaylife.Shesuggeststhatit’sgoodtoknowthelawingeneral.
Renata–Formerjob-seeker,finishedschoolrecently,butfoundajobalready.Afriendfroma faculty of law suggested to do an online communismonNostis.Hermotivationwas toincreaseherchancestofindajob,butaftergettingemployed,themotivationhasfallen.
Robert–StudentofLaw,knowsNostisfromschool.Hejustwantedtorefreshhisknowledgesohestartedanonlinecourse.Hewouldliketotakeanothercourseforthesamereason–torefresh his memory to be more confident in his field. A certificate from law course ismeaningless for someone with law degree. This explains the motivation behind our bigpopulationoflawstudents.
Soňa–AteacherwhodidtheFamilyLawcoursebecauseshewastoldfromherdaughtertodo so. The themewas interesting to her and she has learned something throughout thecourse. The certificate has some kind of meaning for her; she has submitted it to heremployer.
Thisleavesmewithaconclusion,thatmotivationisnotbinary,ithasmanyaspectswhicharehardtodifferentiate.Ingeneral,peoplewantfromNostisusefulknowledgefortheirwork,study and daily life. The courses also satisfy curiosity about the topic. Any furthercategorizationwouldbebendingresultstofitartificialboxes.
Whydopeopleleaveandnevercomeback?
Theoverarchingthemeisthatthemoreapersonisinterestedinthetopicofthecourse,themorewillingsheistostickwithit.Andviceversaifthereisalmostnointerestinthetopic,thennothingcankeepthestudentaround.Fromthesideofthestudentitisoftenherlackoftime,procrastinationandvariousdistractions(Facebook,email,othertabsinbrowser…)thatkeepherfromcompleting(Zhengetal.2015).
FromtheMOOCpointofview,therearealsomanyaspectscrucialtotheretentionrate.Inthe exploratory workshop with stakeholders, we used our past experiences and currentresearch(Oakley,Poole,andNestor2016;Zhengetal.2015)tocomeupwithalistofaspects,thatarethemostcrucialtotheretentionrate:
28
Like in an ordinary lecture at school, the most influential and engaging is the lecturer(Adamopoulos2013).Whentheinformationispoorlypresented,itcanputoffeventhemostinterestedusersandviceversa;IftheuserisnotsoengagedwithathemeofaMOOC,thepresentationcanbestillsoentertaining,thatthereisnoreasontoleave.InmostMOOCs,there is a lecturer talking and possibly some kind of presentation in the background.Important is charisma of a lecturer, how clear he speaks and natural body language.Furthermore,betterthanonestaticcameraiswellthoughtmovementsandcutskeepingtheuser’sattentionalongwithapresentationandotherpost-productionimprovements.
Fromthetechnicalpointofview,audioqualityiscriticalbecauseofthespokeninformation,followedbyvideoqualityfortheadditionalnotesonthescreen.Ofcourse,ausermustfindtheMOOC,navigateto lecturesandfill testswithoutanymajorerrorsandfailures.ThesebasicwebsiterequirementsaresummarizedbyŘezáč‘spyramidofwebneeds(2016)basedonMaslow’spyramid.Thebasic idea is, thataservicemustfirstsatisfyneeds infollowingorder: usefulness, discoverability, availability, accessibility, usability and trustworthiness.Onlyaftermostlysatisfyingthoseaspects,theMOOCplatformcanbeusedwithoutcausingfrustration and can climb the pyramid higher to persuasiveness, happiness of usage andmakingabondwiththeuser.
Thetheoryoffun(Koster2013)offersanadditionalreasonswhythefuncandiminishandbecomereplacedbyboredom.FollowingisasummaryofthosecausesincontextofMOOCs–basicallyreplacing“player”by“user”and“game”by“course”:
- Theuserunderstandseverythingthetopiccanofferandthecoursebecomestooeasy.- Theuserdoesnotunderstandthetopicandcannotconnectthedots.Thecourseis
toohard.- Thepacingistooslowandnewchallengescometoolate.Theuserlosesinterestand
iseasilydistracted.- Thepacingistoofastandnewchallengescometooquickly.Theuserisgettinglost
andthecoursebecomesjustanoise.
TheMOOCUserssummary:
- UsersofNostisaredeterminedbymarketingactivitiesandcoursetopics.- Ifoundfollowinggroupsofusers:
o Lawstudentsrepeatingolderknowledgeo Peoplewhoseworkisrelatedtolaw.Theybenefitfromtheknowledgein
theirjobsanddailylives.o Therearealso peoplewhowant to impresspotentialemployerwitha
certificate,peoplewholookonlyforaspecificinformationandmore…- Userbehaviorisconnectedtovaryingmotivations,noteveryonewantstofollow
aMOOCintheclassicalsense.Therefore,alsoonNostisadifferentcalculationofretentionrateisneeded.
- Crucial aspects of engagement besides the MOOC topic are: student’s timecapacities and motivation, lecturer’s presentation abilities, audio and videoquality,camerawork,post-productionandbalanceddifficulty.
29
Optimizationsforrushersandsamplers
“SuchMOOCsshouldalsobeopentopublicatalltimes,allowingstudents
whohaveimmediatelearningneedstoaccessthecourseinstantlyandnot
waitforthenextsessionbeinglaunched.”-suggestedbyZhengetal.
(2015)
Theresearchshows,thatthereisasubstantialgroupofpeoplewhoarenotusingaMOOCinawayitwasdesigned.Iwantedtofindout,whetherthesegroupsappearalsoonNostisandifitwouldbeworthtooptimizetheuserexperiencefortheirgoals.AmongvarioususecaseswethoughtofinaworkshopwithNostisstakeholders,Ifoundtwohypothesistheeasiesttoexploretogetherbyoneexperiment:
- Rushers–people,forwhothecourseistooslowandtheywanttofinishitwholeinjustafewdays.
- Samplers–people,whoaresearchingontheinternetforaspecificinformation,whichcouldbesatisfiedbyasolelectureorasmallpartofawholeMOOC.(Kizilcec,Piech,andSchneider2013).
So far, these use caseswere not supported by Nostis, because by default, lectures fromupcoming blockswere not visible and they only appeared to a user after signing up andwaitingforaweekperblock.Therefore,notevensearchenginescouldseeallthecontentandmany lectureswerebasically invisible.Thereasonbehindthis is that learning ismoreeffective,whendoneinsmallerchunksthroughoutalongertimespan,sotheknowledgehastimeto“absorb”(Koller2012).
Inmyexperimentaldesign,Itriedtokeepthislong-termstudypattern,butalsogivetheuseranoptiontoskipthisrecommendationandtakeandtakeadifferentapproach.Idisplayeverylectureinacourse,souserscanhaveanoverviewofthecontentandsearchenginescanindexthoselectures.ButIaddedalockicontoshow,thatthecontentisprobablynotaccessible.Uponhovering,thelockedlockiconischangedtoanunlockedstate,suggestingapossibilityof interaction.Uponclickingona locked lecture,user ispresentedwithamessageaskingwhethertheuserwantstoholdontotherecommendedpace,orwhethershewantstounlockall content and proceed at her own pace. Every unlocked course was also saved to thedatabaseinordertomeasuretheusageanddeterminetheresultofthisexperiment.
Tomakethelecturesmoreeasilyfindable,inadditiontoservingthemtosearchengines,IaddedatextdescriptiontoeverypagewithlectureinoneofourMOOCs.Therefore,searchengines couldget keywords foreverypageand thendisplay the lecture in search results.Because Nostis already uses Google Analytics, a data about user acquisition could becompared from periods before and after this change and a result of the experimentdetermined.
30
Figure17:AscreenshotofanoriginalandoptimizedversionofNostis.Ontheleftthelecturesinblocksarehidden,onthe
right,therearelockiconsforaquickeraccesstotheMOOCcontent.
Theexperimentresults
TheexperimentwasrunningonNostisforthreemonthsandthedatafromdatabaseshow,thattheunlockfunctionwasused148timesoutof812newenrollments.Thatmeans18%ofusersdiscoveredandusedthisnewfeature.Buttheusercouldhavejustunlockedthecontentbut then follow in normal pace. Therefore, I did a deeper graphical analysis of lecturewatchingpatternsofthose,whohavealreadyearnedacertificate.BasedontheFigure18,Icansay,thatthisexperimentissuccessfullyprovinganexistenceofuserbase,whichwantstofinishacoursefasterthanthedefaultpace.
31
Figure18:During3months,32usersunlockedacourseandlaterearnedacertificate.Thisfigureshowshowmanylectures
werecompletedperdayforthoseusers.Everyrowisauserandtherearetwotimelines(becauseIwasunabletomergeit).
Ontheleft,thereareblacksquaresrepresentingearnedcertificates.Ontherightaretotallecturescompletedperdayincolor
gradients.Bluedotisonelecture,purpleisaround10,pink25,yellow40andthroughorangeto80lecturesinred.Wecan
seethat25peoplefinishedawholecourseinashortertime,thanthecoursewasdesignedfor,proving,thattheintroduced
optimizationmakessense.
Unfortunately,thesecondpartoftheexperiment,searchengineoptimization,didnotyieldasolidresult.ThedatainGoogleAnalyticsshowednoincreaseoftrafficfromsearchenginesdirectly to lectures.Thereasoncould lie in thechosenMOOCaboutbusiness lawandMSExcel. There is a lotof guides, tutorials, articles andothermaterials about this theme, soNostisisusuallynotinthetopresults.Toprovideclearandconclusivedata,thelecturesoftheMOOCneedstohaveauniquecontentandoptimizedtextualdescriptionincontextofpossiblesearchqueries.RecentlyaMOOCplatformCourserastartedofferingcontentsofaGamificationcoursetosamplersthroughsearchengineoptimizationandabackdooraccessproving,thatmyconceptisworthevaluating.
Optimizationsforrushersandsamplerssummary:
- Nostiswasoptimizedfortwouserbehaviorgroups:o Rushers–tounlockandfinishacoursesoonerthanrecommended.o Samplers–togetonlysomeinformation,notfinishingawholecourse.
- Theexperimentshowedapresenceofrushers,whounlockedandfinishedcourseearlier.
- Optimizationforsamplersdidnotworkasexpectedandisinconclusive.Ontheotherhand,Courseraintroducedsimilarfeatureshowingapotential.
- Nostisnowappealstomoreusergroupswithdifferentjobstobedone.
32
TheMOOCOnboardingExperience
“Bad[course],well…Itcanbeactuallygood,butthedescriptioncanbe
bad…”4–Tomáš
Toimprovetheretentionrate,Itackletheproblemofmisunderstandinginthebeginningofacourse.Usersmightleaveacourseforseveralreasons,butinspiredbyaquotefromTomáš,Iwanttominimizethedrop-outforfollowingreasons:
- Thecontentofthecourseisnotwhattheuserthoughtitis.- TheMOOCistoodifficultforausertograsp;theuserismissingsomepriorknowledge.- TheMOOCistootrivial;theuseralreadyhastheknowledge.- TheMOOCtakesmoretime,thanexpected.- Thelecturer,videooraudioqualityofthecourseisnotasexpected.
Byappealingtotherightaudienceanddeterringthewrongone,Iaimtolowerthenumberofenrolled studentsbut increase thenumberofhighlymotivatedones.Themorepeopleknowwhattoexpect,thelessfrustrationarisesandthemorepositiveemotionswillthebrandofNostisgain.Intheend,Icanonlyenhancemotivationofauser,whoisalreadymotivatedtofinishacourse.
ThewireframeinFigure19wasmadeagainwithJakubRybářbasedonourinitialexploratoryresearchandthentestedwithrandomstudentsallarounduniversitycampus.Weaskedthemaboutwhatforthemisimportanttoknowwhentheydecideforacourse(onlineoroffline),thenwepresentedthemoursketchtobesureaboutclarity.IhavethenslightlyimprovedthewireframebasedonthesynthesisofallthedataIhavegatheredincontextofthisthesis.
Thewireframeexplained
Thecoursepagewireframeisdividedintoseveralsectionsbyaline.Thetopsectionisthemost importantonecontainingatitle,veryshortclaimsaboutthecoursetoattractuser’sattention,pictureofthemainlecturerwithanameandherinstitutionandmostimportantlyaplaybuttoninthemiddlesuggestingthereisavideointhebackground.Afteractiviatingthevideo,itcomesintoforegroundandthelecturerexplainswhatthewholecourseisabout.Thisisnotonlyanattractivedescriptionofacourse,butalsoademonstrationofthepictureandsoundqualityof theMOOCandqualitiesof the lecturer. It is important for someusersasearchenginesnottoforgetthetextualdescriptionofacourse,whichIwouldpartiallyhidealsointhetopsection.
“IwanttoseethevideosoIknowthatitisnotuglyandthelecturerdoes
nothaveaboringvoice.5”–Eva
4Czechoriginal:“Blbej,třeba...Onmůžebýtprávědobrej,alemůžebýtjakoblběpopsanej.”5Czechoriginal:“No, jáchcividetotovideo,abytonebylohnusný,nebotenčlověknebylhnusnej,nebonemělblbejhlas.”
33
Figure19:Wireframeofimprovedcoursepage.
34
The second section of the wireframe pinpoints the most important data which are nowmissing on Nostis. There is a short explanation of who the course is for – law students,experiencedcivilengineers,orjustgeneralpublic.Thisisconnectedtoadifficultyleveljustbelow,whichcouldbecombinedtogether.Difficultyrepresentshowmuchconfidenceandknowledgeisrequiredfromtheuseringeneral.Andthethirdessentialinformationisabouttimeneededtocompletethecourse.Itisoptimaltodescribehowmanyhoursperweekthelecturesandassignmentswouldtake,andhowmanyweeksdoesthecourselast.Ontherightisthebuttontostartthecoursewhichemphasizesthecost–free.Somestudentssaid,thattheyhaveassociatedhighqualitycoursewithafee,andafreecoursemeanslowqualitytothem.Hence it is important toshowthequalityof thecourse first in thevideo,and thenemphasizetheno-cost.
Anothertwoblocksarededicatedtoearningthetrustoftheuser.Thereisasmallpreviewofa certificate, list of lecturers and a list of collaborating institutions. These displays ofauthoritieshaveabigimpactonuser’sdecisions.BelowaretestimonialsandratingsoftheMOOCfromusers.Inthebestscenario,thetestimonialsshowwouldbefromasimilaruserbasedonage,occupationorlocation.Themorethepeoplehaveincommon,thehighertrustthereis,nomatterhowrelevanttheinformationis(Cialdini2009).
Atthebottomofthepage,usercanfindlecturesthematicallydividedintoblocks.Thevideosshouldnotbe longer thanapproximately6minutes,becauseusers start to loseattentionaroundthattime.Also,everylectureisaprogressionloopenhancingmotivationasfastasthelectureends.Theselecturesandblocksformcheckpointsshowingtheuserhowmuchshehasaccomplishedandthatthenexttaskisjustanothereasysmallbite.
The wireframe has not yet been implemented, it should highlight the most importantinformationuserneedstosignupforacourse.Topickthemostimportantelement,itwouldbe the top video. User should be able to get all the required information from there;therefore,itiscrucialtomakeitright.
Theonboardingexperiencesummary:
- Usersoftenleaveacoursebecauseoflackoftime,boringlecturer,badaudioorvideoqualityorbecauseoftoodifficultoralreadyknowncontent.
- I created a wireframe tacking these problems by displaying all the requiredinformationupfrontinanintroductionvideoandalsotextuallyonthepage.
- Theaimistogiveenoughinformationtotheuserbeforeenrollingin.
35
TheMotivationhelperThethirdconceptIcreated,themotivationhelper,wasbasedonastakeholderexploratoryandideationworkshop.Other interestingconceptsthatemergedduringtheworkshoparedescribedintheDiscussionchapter.Themotivationhelperutilizesthemotivationtheoriesand ideas fromgamificationto increaseretentionrateofusers. Itwastestedwithseveralparticipants,unfortunatelyarealdeploymentandfollowingquantitativeevaluationwas intheendunsuccessful.
Thelackoftimeproblem
AstudybyNawrotandDoucet(2014)identifiedthebiggestthreattoMOOCengagement-lack of time (“bad time organization, conflicting real life responsibilities, too much timeconsumingcourse”).Myownunderstandingof“lackoftime”isgivingtheactivitylowpriority.IfIabsolutelywanttofinishmydegree,Iwillinvestmytimeintoit.ButifIdonothavemoneyforfood,Imightinvestmoretimeintojobseeking,ratherthanstudying.Inotherwords,Iaminthiscasemotivatedbymyprimaryneedtofeedmyself,ratherthanbeingmotivatedtolearnsomething(andpotentiallydiebyhunger).Therefore,Iseetherealchallengenotonlyas a “lack of time”, but also as a lack of motivation. This is nicely put by one of myinterviewees:
“IfIdonothavethetime,butIknowthatIREALLYneedthecourse,thenI
willjustcreatethetimeforit...”-Tomáš6
Totacklethisproblem,IdesignedaMotivationHelper–atimemanagement¬ificationtoolbasedonmotivationtheoriestoencourageuserstostayontrackwithacourse.Afterenrollingintoacourseandwatchingtwolectures,theuserispresentedwithacolor-codedtable(Figure20).ThereareallweekdaysandonYaxistimeperiodsofaday:beforenoon,afternoonandevening.Usersees3pre-selectedfieldsmeaningthathewillbenotifiedatthesetimesto“Keeptrackwithacoursematerialanddeadlines!”Theusercanmodifytheselectionandtimesofnotifications.
Thebasicideaofthistoolisthenotificationmanagement,whereuserisremindedthatthereisalearningtobedone.Itshouldservethosewithtimemanagementproblems,whohaveahardtimerememberingwhattheywantedtodo,orwhogetseasilydistracted.Butthereismuchmorethanthat.Toaddressthe“lackoftime”issue, Ialsowanttoforcetheusertoconsider,whethershehasenoughtimeforthecourse.Thetoolshowshowmuchtimethecoursewillrequireandthenaskstheusertoplanwhentheyhavetimetolearn.
Whymotivationhelper
Goingbacktothemotivationtheory,thetoolkeepsinmindthecoreprinciplesofamotivationenhancers. The user has a complete control over the notification settings, thereforeautonomyispreserved.Ialsotriedtobringintoplaytheaspectofrelatednessintheformofthecolorfulgrid.Usercansee,thatthedarkerthefieldis,themorepopularisthistimeamongotherparticipants.Therefore,usershouldgetthefeeling,thatsheisnotaloneandcanseetheactualbehaviorofherpeers.Thethirdaspectiscompetence,whichIdidnotfocusoninthistool.
6TranslatedfromCzech:“Kdyžnemámčasa...tenkurz,alevím,žefaktjakopotřebuju,taksitenčasjakobyudělám[…]”
36
Figure20:ImplementationoftheMotivationHelper.
Anotherpowerfulaspectissenseofcommitmenttooneself.Withthetool,theparticipantcommitsthatsheisgoingtolearnattheplannedtime.Andahumanbrainlikestosticktoplannedgoalsandstayconsistent.Thisisaremotepartofonetypeofextrinsicmotivation,butmuchdeeperexplanationandelaborationonthistopiccanbefoundinabookInfluenceby Robert B. Cialdini (2009). This commitment should also helpme in differentiating theparticipants,whowant to finish the course the classicalway, andother behavior groups.Although this approachof calculating retention ratehasother flaws, in combinationwithothermeasures,itcanshowinterestingpatternsindisengagement.
Themotivationhelperwasgreatly inspiredbyDuolingoandtheirmobileappnotificationsthatyouget,ifyoudonotpracticeduringaday.ButbecauseNostisdoesnothavenorneedto have a mobile application, I opted for a browser notifications. This way, the user isremindedwhilejustbrowsingtheinternetandifheisofflineandthebrowserisclosed,thenotificationisnotsorelevantanyway.
Figure21:Anexampleofnotificationpersuadingausertocomebacktothecourseandlearn.
37
The notification itself is trying to amplify the user’s engagement by displaying amessagerelevanttotheselectedcourse.Thetextgivestheuserreassurance,thatwhatsheisabouttodoisgoodforher.Totakethistoanextlevel,theusercouldfirstspecify,whatisthereasonbehind taking the course and what is her motivation. Then the message could be trulypersonalized. But that would add another step to the setup making it probably overlycomplicated.
Testingthistoolshowedinterestintheconceptwithsomeusers,otherrequestedadditionalfeatureswhichwouldresembleacalendar.Butthosepeoplearealreadyquiteorganizedandtheyalreadyusesometimemanagementtoolssobuildinganewoneforthemisnottherightwaytogo.Awidertestwouldbeneededtosee,whetherthere isalsoabiggroupof lessorganizedusersforwhothistoolwasdesignedfor.
TheNostismotivationhelpersummary:
- OneofthebiggestMOOCproblemsisusers’lackoftimeandlowmotivation.- Themotivationhelperusesmotivationtheoryandgamificationtoenhanceuser’s
motivation:autonomy,relatedness,commitment,activityloops…- Usersetsupnotificationsatspecifictimeremindinghertolearn.- Ratedpositivelyinqualitativetests,butquantitativetestneededtoprove,that
theconceptisusefulforawideraudience.
38
DiscussionAsineverywork,thereareaspectsthatcouldnotbecoveredmostlyduetotimeconstraintsandmycapabilities.Thischapterdiscussessomeotherperspectives,challenges,directionsandnextstepsofmywork.
Theresearchreflection
Ihaveencounteredacriticismofmyworkregardingonlyexplorativeresearch focusedonprovingordisprovinghypotheses,ratherthangettingexactnumbersandpercentages.Ifoundapproximationsandhintsfromqualitativeresearchassufficienttomakeaconclusion.ButinthecirclesofMOOCcreators, lecturersanduniversity stakeholders,adirectnumbersandprecisemeasurementswouldmakemuchbigger impact.Forexample,whileexploring thepopulationof rushersamongNostisusers, Ididnotknowhowtocomeupwithaprecisenumber.Howtoidentifythesepeople.Those,whounlockedacourse?Those,whounlockedandgotacertificate?Those,whowatchedlecturesfromdifferentblocksinonesession?
Inordertogetbetterarguments,IshouldhavesetupsomeKeyPerformanceIndicators(KPI)andthenmeasurethose.ButthenIwouldfallintothesameproblemsastheretentionratemetricIcriticize.Therefore,IhopethatthisthesiswillbeunderstoodasaworkbroadeningtheviewofMOOCs,notonedescribingtherealityofMOOCs.
Otherconceptsworthexploring
Tocomeupwithaconcepttoworkon,IinvitedthesecondNostisco-creator,theplatformmanagerand3MOOCcreators foran ideationworkshop. In this4-hoursession,wewentthroughcurrentobstaclesforcourseparticipantsandideatedaboutpossibleapproachestotakeinordertoenhanceusermotivation.WehadthreemainideasfromwhichIchosetoworkontheMotivationHelper.Theotherswere:
AdiscussionplatformforNostis,whereuserscouldaskquestionstooneanothercollaborateandsocialize.AlthoughoftenconsideredanessentialpartofMOOCs,Nostisdoesnotyethaveanydiscussionplatformwhatsoever. Inour first course, users couldput commentsundereverylecture,butthisfeaturewasbarelyusedandwestoppedsupportingit.ThefeelingofbeingsociallyconnectedandpartofagroupisapowerfulmotivationtoolinthetheoryofExtrinsicandIntrinsicMotivationandinGamificationandinadditionaparticipants’discussioncouldgiveusavaluablefeedbackonourMOOCs.
On the other hand, current research shows that discussion forums are also a source offrustration, when there are old unanswered questions (Oakley, Poole, and Nestor 2016).Therefore, for a discussion platform to works properly, it requires maintenance frommoderatorsandlecturers,theresourcesNostisdosnothave.Anotherstudypointsout,thatonly3%ofallparticipantsengagedinadiscussion.
Anotherideawasanauto-playfeaturewith…Itwasbuiltonahypothesis,thatmostofthepeoplewatchawholeblockoflecturesinarow.Inmyresearch,Ianalyzedthisbehaviorandbecauseitwasnotmajorityofusers,Idiscardedthisidea.Butasthisgroupofusersisstillaconsiderablepercentagesuggestingthisfeatureisworthtokeepinmind.Intheend,IchosethenotificationtoolalsobecauseIwasinterestedinthetechnology,notbecauseitwouldbeobjectivelythebestchoice.
39
Themotivationhelper’snextstep
Unfortunately,IcouldnottestthenotificationfunctionoftheMotivationHelperprototypeinalong-termexperiment,becausethecurrenttechnologyinwebbrowsersdoesnotsupporttimingofnotifications.Thereisaworkaround,whereattheselectedtime,theserverwouldsendapushnotificationtothebrowser,whichthendisplaysthenotification.Butthiswayismuchmoredemandingoninfrastructureandoutofmypossibilities.
Fromthefewtests,Icanseethatthereisapotentialinthistooltohelpsomeusersinplanningtheir time and finishing a course. But I do not know if potential users are inmajority orminority.Therefore,Isuggestasanextstepafake-doortest7–sendingnewusersanemailwithalinktotheprototype.Butsincethefunctionalityisnotthereyet,astheyclick“savenotifications”,theywouldbepresentedwithanapology,thatthisfeatureisjustatest.Inthisway,Icouldobservehowmuchwouldthistoolbeusedandhowdopeopleinteractwiththeinterface.
7Awaytotestwhetheraserviceiswantedornot.Wedescribetheservicetoauser(inemailoronwebpage)andmeasurehowmanyclickedonanactionbutton(likeregister,signuporsetupnotifications).
40
ConclusionMOOCusershavevariousmotivationstoenrollintoaMOOCanddifferentbehavioroncetheydo.From19interviewsandasurveystudy,Ifoundthatsomeuserswanttogothroughalllecturesandtestsandfinishthewholecoursewithacertificate,somedon’twanttobotherwithassignmentsandonlywatchlectures,andsomeareonlylookingforacertaininformationand then they leave. But retention rate, which is understood as a user success rate, iscalculatedasnumberofcertificatesdividedbynumberofusersenrolled.Theproblemis,thatthismeasurementisfocusedonlyonasubpopulationofMOOCusers,whowanttocompletethecourse.AlthoughausermightseeasampleofMOOC,gettheinformationneededandthenleavesatisfied,theretentionraterepresentsthisuserasfailure.Therefore,thewholeretentionrateisflawed.
Mydesign is focusedon improvingretentionrateonNostisusingmotivationtheoriesandgamification. The goal was to make a feasible prototype which could be then deployeddirectlytotheliveserverasanewfunctionality.IdesignedthreeimprovementstoNostis:Anoptimizationforrushersandsamplers,aredesignedcoursemainpageandtheMotivationHelper.
Rushersarethepeoplewhowanttofinishacourseinfewdays,insteadofweeks,rushingtowardsthecertificate.Sofaritwasimpossibleduetogradualunlockingofanewcontenteveryweek.Icreatedanexperiment,wherethewholecontentofacourseisshownandbyaquickinteractionuserscanunlockallthecontentofacourse.Thisexperimentprovedtobesuccessful showing that there is a subpopulation of users who unlocked the course andfinished sooner than usual. The samplers optimization is based on making all lecturesaccessible and searchable through search engines. But due to a poor set up of thisexperiment,Icouldnotconcludeexistenceornonexistenceofsamplersnormaketheiruserexperiencebetter.
Myresearchshowedthatthecoursemainpageisthepointwherepeopledecidewhethertoenrollintoacourse,butoftenthepagelacksnecessaryinformationwhichisdiscoveredafterenrollmentandleadstodroppingout. IsynthesizedthereasonswhypeopleleaveMOOCsandthekeyinformationtheyneedtomakeaninformeddecisionandredesignedthecoursemain page.User tests proved, that thewireframe is on the right track. Furthermore, if apersonknowsfromthestart,thatthecourseisnotwhattheyneed,theydonothavetimeforit,oritistoodifficult,theywillnotenroll.Thisgoesagainsttheideatogetasmanyusersaspossible,butwhat is thebenefitofattractingauserwhichgets frustrated lateronandleavesanyway?
TheMotivationHelperisanewfeatureofferingnotificationstouserstohelpthemstayontrack.Ausercansetupnotificationsatchosentimesforcinghertothinkaboutherweeklyscheduleandmakingacommitmenttoherself.Itisbuiltontheoryofextrinsicandintrinsicmotivation,emphasizingautonomyandrelatednesstostrengthenthe initialmotivation. Italsousesactionloopsfromthefieldofgamificationandsomegameelements.Althoughfewqualitative testsproved there isapotential,a full-scalequantitative test is stillneeded toassessthesuccessofthetool.
41
ReferencesAdamopoulos,Panagiotis.2013.“WhatMakesaGreatMOOC?AnInterdisciplinaryAnalysis
ofStudentRetentioninOnlineCourses.”In,2013:21.AIS.
Altarriba,Ferran.2014.“TheRevolutionofFun:AStudyofAppliedGamesandFuninNon-Entertainment Contexts.” BA (Hons) Interaction Design dissertation.http://www.ferranaltarriba.com/docs/papers/14_sp_rf.pdf.
Belanger,Yvonne,andJessicaThornton.2013.“Bioelectricity:AQuantitativeApproachDukeUniversity’s First MOOC (2013).”http://dukespace.lib.duke.edu/dspace/handle/10161/6216.
Breslow,Lori,DavidEPritchard,JenniferDeBoer,GlendaSStump,AndrewDHo,andDanielTSeaton.2013.“StudyingLearningintheWorldwideClassroom:ResearchintoedX’sFirst MOOC.” Research & Practice in Assessment 8.http://search.proquest.com/openview/27e5ba80e54284de770427a71f05bf8d/1?pq-origsite=gscholar.
Carlson, Neil R, Donald Heth, Harold Miller, John Donahoe, and G Neil Martin. 2009.Psychology:TheScienceofBehavior.Pearson.
Christensen, ClaytonM, TaddyHall, KarenDillon, andDavid SDuncan. 2016. “KnowYourCustomers’‘JobstoBeDone.’”HarvardBusinessReview9:54–62.
Cialdini,RobertB.2009.Influence:ScienceandPractice.5thed.Boston:PearsonEducation.
Deterding, Sebastian. 2011. “Meaningful Play. Getting »Gamification« Right.” January 31.https://www.slideshare.net/dings/meaningful-play-getting-gamification-right.
Deterding,Sebastian.2012.“Gamification:DesigningforMotivation.”Interactions19(4):14.doi:10.1145/2212877.2212883.
Glass, Chris R., Mitsue S. Shiokawa-Baklan, and Andrew J. Saltarelli. 2016. “Who TakesMOOCs?” New Directions for Institutional Research 2015 (167): 41–55.doi:10.1002/ir.20153.
Hansch,Anna,ChristopherNewman,andThomasSchildhauer.2015.“FosteringEngagementwithGamification:ReviewofCurrentPracticesonOnlineLearningPlatforms.”SSRNElectronicJournal.doi:10.2139/ssrn.2694736.
Ho,AndrewDean,JustinReich,SergiyONesterko,DanielThomasSeaton,TommyMullaney,JimWaldo,andIsaacChuang.2014.“HarvardXandMITx:TheFirstYearofOpenOnlineCourses, Fall 2012-Summer 2013.” SSRN Electronic Journal.doi:10.2139/ssrn.2381263.
Hood, Nina, Allison Littlejohn, and Colin Milligan. 2015. “Context Counts: How Learners’ContextsInfluenceLearninginaMOOC.”Computers&Education91(December):83–91.doi:10.1016/j.compedu.2015.10.019.
Karsenti,Thierry.2013.“WhattheResearchSays.” InternationalJournalofTechnologiesinHigherEducation10(2):23–37.
Kim,AmyJo.2012.“SocialEngagement:Who’sPlaying?HowDoTheyliketoEngage?”Amy
Jo Kim. September 19. http://amyjokim.com/blog/2012/09/19/social-engagement-whos-playing-how-do-they-like-to-engage/.
42
Kim,Joshua.2012.“PlayingtheRoleofMOOCSkeptic:7Concerns.”InsideHigherEd.May21.https://www.insidehighered.com/blogs/technology-and-learning/playing-role-mooc-skeptic-7-concerns.
Kizilcec,RenéF.,ChrisPiech, andEmily Schneider.2013. “DeconstructingDisengagement:AnalyzingLearnerSubpopulations inMassiveOpenOnlineCourses.” In ,170.ACMPress.doi:10.1145/2460296.2460330.
Koller,Daphne.2012. “DaphneKoller:WhatWe’reLearning fromOnlineEducation.”TED.June.https://www.ted.com/talks/daphne_koller_what_we_re_learning_from_online_education?language=en#t-672691.
Koster,Raph.2013.TheoryofFunforGameDesign.O’ReillyMedia,Inc.
Kulkarni,Chinmay,KohPangWei,HuyLe,DanielChia,KathrynPapadopoulos,JustinCheng,Daphne Koller, and Scott R Klemmer. 2015. “Peer and Self Assessment inMassiveOnlineClasses.”InDesignThinkingResearch,131–168.Springer.
Laurillard,Diana.2014.“FiveMythsaboutMOOCs.”TheTimesHigherEducation.
“MOOC.” 2017. Oxford Dictionaries. Oxford University Press. Accessed April 14.https://en.oxforddictionaries.com/definition/mooc.
Nawrot, Ilona, and Antoine Doucet. 2014. “Building Engagement for MOOC Students:IntroducingSupportforTimeManagementonOnlineLearningPlatforms.”In,1077–82.ACMPress.doi:10.1145/2567948.2580054.
Oakley,BarbaraAnn,DebraPoole,andMaryAnneNestor.2016.“CreatingaStickyMOOC.”OnlineLearning20(1).doi:10.24059/olj.v20i1.731.
Pappano,Laura.2012.“TheYearoftheMOOC.”TheNewYorkTimes2(12):2012.
Pardee,RonaldL.1990.“MotivationTheoriesofMaslow,Herzberg,McGregor&McClelland.A Literature Review of Selected Theories Dealing with Job Satisfaction andMotivation.”ERIC.https://eric.ed.gov/?id=ED316767.
Řezáč, Jan. 2016. Web ostrý jako břitva: návrh fungujícího webu pro webdesignery a
zadavateleprojektů.
Ryan, Richard M., and Edward L. Deci. 2000. “Intrinsic and Extrinsic Motivations: ClassicDefinitionsandNewDirections.”ContemporaryEducationalPsychology25(1):54–67.doi:10.1006/ceps.1999.1020.
Silverman,David.2013.DoingQualitativeResearch:APracticalHandbook.SAGEPublicationsLimited.
Trenholm,Zen.2015.“Points,Badges,andLeaderboards:TheGamificationFallacy.”Yu-KaiChou: Gamification & Behavioral Design. January 2.http://yukaichou.com/gamification-study/points-badges-and-leaderboards-the-gamification-fallacy/.
Wildavsky, Ben. 2014. “Evolving Toward Significance or MOOC Ado About Nothing?”InternationalEducator23(3):74–79.
Willems,C.,N.Fricke,S.Meier,R.Meissner,K.A.Rollmann,S.Voelcker,S.Woinar,andC.Meinel. 2014. “MOTIVATING THE MASSES - GAMIFIED MASSIVE OPEN ONLINE
43
COURSES ON OPENHPI.” In EDULEARN14 Proceedings, 4042–52. 6th InternationalConferenceonEducationandNewLearningTechnologies.Barcelona,Spain:IATED.
Zheng,Saijing,MaryBethRosson,PatrickCShih,andJohnMCarroll.2015.“UnderstandingStudentMotivation,BehaviorsandPerceptionsinMOOCs.”InProceedingsofthe18thACM Conference on Computer Supported Cooperative Work & Social Computing,1882–1895.ACM.