Trace-Based Microanalytic Measurement of Self-Regulated ... · Trace-based microanalytic measurement of self-regulated learning processes. ... 3.0) 183 Trace-Based Microanalytic Measurement

(2016). Trace-based microanalytic measurement of self-regulated learning processes. Journal of Learning Analytics, 3(1), 183-214.http://dx.doi.org/10.18608/jla.2016.31.11

ISSN1929-7750(online).TheJournalofLearningAnalyticsworksunderaCreativeCommonsLicense,Attribution-NonCommercial-NoDerivs3.0Unported(CCBY-NC-ND3.0)

183

Trace-BasedMicroanalyticMeasurementofSelf-RegulatedLearningProcesses

MelodySiadaty

SchoolofInteractiveArtsandTechnology,SimonFraserUniversity,Canada

DraganGaševićMorayHouseSchoolofEducationandSchoolofInformatics

UniversityofEdinburgh,UnitedKingdom

MarekHatalaSchoolofInteractiveArtsandTechnology,SimonFraserUniversity,Canada

[email protected]

ABSTRACT:Tokeeppacewithtoday’srapidlygrowingknowledge-drivensociety,productiveself-regulation of one’s learning processes are essential. We introduce and discuss a trace-basedmeasurement protocol to measure the effects of scaffolding interventions on self-regulatedlearning(SRL)processes.Itguidestracingoflearners’actionsinalearningenvironmentontheflyand translates thesedata into indicatorsofengagement in SRLprocesses that reflect learners’use of scaffolding interventions and contingencies between those events. Graphs of users’learningactionsinalearningenvironmentareproduced.Ourtrace-basedprotocoloffersanewmethodological approach to investigating SRL and new ways to examine factors that affectlearners’ use of self-regulatory processes in technology-enhanced learning environments. Ourapplicationof theprotocolwasdescribed inastudyaboutLearn-B,a learningenvironment forSRL in the workplace. The findings of the work presented in this paper indicate that futureresearchcangainsubstantiallybyusinglearninganalyticsbasedonusers’tracedataandmergingthem with other quantitative and qualitative techniques for researching SRL beliefs andprocesses.Keywords: Self-regulated learning, micro-level process, trace-based methodologies, learninganalytics,graphtheory,learningtechnology

1 INTRODUCTION Self-regulated learning (SRL) is acknowledged as an essential skill for lifelong learning in today’sknowledge-driven society (Klug,Ogrin,&Keller, 2011). It is often characterized as a process inwhichlearnerstakeinitiativetoidentifytheirlearninggoals;andchooseandregulatetheirlearningstrategies,cognitive resources,motivation, andbehaviour tooptimize their learningoutcomes (Boekaerts, 1997;Winne, 2010a; Zimmerman, 1990). Several research studies show that, in various learning contexts,learnersoftensub-optimallyregulatetheirlearningprocessesorsimplyhaveinadequatemodelsoftheirlearning process that leads them to misevaluating their learning (Bjork, Dunlosky, & Kornell, 2013;Margaryan,Milligan,&Littlejohn,2009;Winne,2005).



184

Toaddress thischallenge,self-regulatoryscaffoldsandpedagogicalaffordanceshaverecentlybecomeof interest to researchers as a means to support learners’ engagement in SRL processes (see, forexample,Dabbagh&Kitsantas,2005;Hadwin,Oshige,Gress,&Winne,2010;McLoughlin&Lee,2007;Zhou & Winne, 2012). Although empirical research has shown that scaffolding can foster learners’engagement insomeofthemainelementsofmetacognitionandSRL—suchasself-observation,self-reflection,orgoal-orientation(Greene&Azevedo,2010)—nomethodologicalframeworkhasyetbeendevelopedthatcanmeasuretheimpactofindividualscaffolds,embeddedtools,ordifferentelementsofSRL in thecontext inwhich theyareused.Thecontextalsoplaysan important role in theeffectofagivenscaffold.CurrentresearchhasshownthatSRLishighlycontextdependentandspecificfeaturesofalearningenvironmentcaninfluencewhetherlearnersengageinSRLprocessesandtheextentoftheirengagement (Boekaerts & Cascallar, 2006; Whipp & Chiarelli, 2004; Winne, 2010a, 2010b). In mostresearch studies, however, learner engagement in SRL processes is measured via self-reports thatrepresent learners’ perceptions of their own beliefs and abilities, statically and outside of the actuallearning environment, rather than indicating their dynamic cognitive processes and responses in theimmediate temporal context of scaffolding interventions. Although recent research has begun to paymoreattentiontomethodsforcollectingdataaboutlearningasithappensthroughso-calledtracedata(Hadwin, Nesbit, Jamieson-Noel, Code, &Winne, 2007;Winne, 2014),morework is needed showinghowtracedataare1)integratedintoresearchmethodsand2)alignedwiththeoreticalmodelsofSRLinwhicheffectsoftechnologicalscaffoldsforSRLarestudied.Tobeabletoinvestigatewhether,andtowhatextent,certaintheorizedSRLelementscanbeenhancedthroughscaffoldsprovidedbysoftwaretoolsandinterventionsinalearningenvironment,inthispaper,we introduce a trace-basedmethodology tomeasure use of SRL processes at bothmacro andmicrolevels, as well as the effect of tool components on certain theorized SRL elements. Macro-levelprocesses indicate categories or phases of SRL as defined in the underlying theory such as planning,monitoring or evaluation,whilstmicro-level processes indicatemore specific activitieswithin each ofthosephasesorcategoriessuchasgoalsettingwithintheplanningphase(Greene&Azevedo,2009).Inour research, we pursue an event-based conceptualization of SRL processes (Winne, 2014). Wedevelopedandappliedatrace-basedmicroanalyticmeasurementprotocolmethodologyto investigateusers’deploymentofSRLprocessesintheauthentic,dynamiccontextoflearning.Viathismethodology,learners’actionsarecapturedontheflyandintheauthenticcontextofoccurrence.Oneofthegreatestadvantages of trace-based methodologies is that they allow for grounding analyses and inferencesdrawn from analyses on accurate and authentic data that proximally identify learning events in theirveryowncontext(Azevedo,Moos,Johnson,&Chauncey,2010;Greene&Azevedo,2010;Winne,2010a,2010b;Winne&Perry,2000;Zhou,Xu,Nesbit,&Winne,2010).Thispaperdoesnotaimtoreportonthefindingsofanyspecificstudyorintroduceatheoreticalmodel.Rather, itaimstointroduceanovelmeasurementprotocolforuseindifferentstudiesthatlookattheeffectsoftechnologicalscaffoldinginterventionstosupportSRL.Theprotocolhasbeenappliedtotwo



185

different studies (Siadaty, Gašević, & Hatala, 2016a, 2016b) that offer 1) theoretical details for theadoptedSRLmodelandthechoiceanddesignoftechnologicalinterventions,2)hypothesizedeffectsofthetechnologicalinterventionsonspecificSRLphases,and3)findingsregardingtheproposedprotocoltobothtestthevalidityofthehypothesizedeffectsanddetectandexplorewhichinterventionshadthestrongesteffectsonspecificSRLprocesses.Weorganizethispaperasfollows:Modelsconceptualizingself-regulatedlearningprocessesandvariousmethods used in the literature to measure these processes are introduced in Section 0. Section 0describes our measurement protocol, starting with a discussion of the prerequisites for applying it,namely,formulatingtheunderlyingSRLmodelandtheintendedscaffoldinginterventions,followedbyadetailedexplanationofthestepsintheprotocolandexamplesofhowweappliedtheminourresearch.Section0discussestheimplicationsoftheproposedprotocolforbothresearchandpracticeandSection0concludesbyofferingdirectionsforfutureresearch.2 ASSESSMENT OF SRL PROCESSES In this section, we provide an overview of existing perspectives and models of SRL, and describemethods commonly applied in current empirical research for measuring SRL according to theseperspectives.2.1 Self-Regulated Learning: Conceptualization The concept of SRL emerged fromwithin educational psychology research in the 1980s and becameincreasingly popular as the concept of learning to learn found its way through educationalenvironments. Since then, it has been a subject of extensive study in different disciplines, such astraining,academiceducation,medicaleducation,andeducationalpsychology(Carmen&Torres,2004;Karoly,1993;Schunk&Zimmerman,1994;Winne&Perry,2000;Zimmerman,2001).Differentmodels posit alternative views on how learning is self-regulated (Boekaerts, 1997; Pintrich,2000;Winne&Hadwin,1998;Zimmerman&Schunk,1989).SRLmodelsingeneralaimtodescribehowlearnerstakecontrolofandmanagetheirlearningprocesses(Wolters,2010).Onewaytodifferentiatethese models is through different conceptualizations of SRL. One perspective offers an aptitude orcomponentconceptualization,whileanotherconceptualizesSRLintermsofeventsorprocesses(Dettori& Persico, 2008; Klug et al., 2011; Puustinen & Pulkkinen, 2001; Steffens, 2006;Winne, 2010b). Themodels using the componentoraptitudeperspective aremore trait-oriented (Klug et al., 2011); theycharacterizeSRLintermsofrelativelystable,decontextualizedindividualdifferences(e.g.,attitudesandbeliefs) and identify cognitive,meta-cognitive,motivational, andemotional aspectsof (self-regulated)learning.Animportantcommonalitywithinthiscategoryisthataptitudes,althoughrelativelyenduring,areconsideredadjustable, inthat learnerscanbetaughttodevelopthedesiredaptitudes,ortransfer



186

thelevelornatureofanaptitudeastheyprogressthroughlearningevents(Winne,2010a,2010b).Onthe other hand, theeventorprocess perspective ismore concernedwith theway learners approachproblemsandapplytheirlearningstrategies(Steffens,2006)insitu.ThesemodelsviewSRLasproactive,goal-orientedprocesses that learnersdeploy toacquireacademic skills andcompetencies (Klugetal.,2011;Steffens,2006). Suchprocessesare typicallyorganizedasa setof learningphaseswithinwhichlearners perform learning activities, e.g., tactics and strategies. These phases typically repeat duringlearningactivitiesandmayinfluenceoneanother.Another way to differentiate the existing models is by their theoretical underpinnings (Greene &Azevedo, 2007; Puustinen & Pulkkinen, 2001). The models proposed by Pintrich (2000), Winne andHadwin (1998), and Zimmerman and Schunk (1989) are most empirically supported in the currentliterature.While thesemodels share someoverlappingconceptualizations,discussed inSection0, thebiggestdifferentialisthetheoreticalbackgroundinwhichtheyaregrounded.Pintrich’s (2000) and Zimmerman and Schunk’s (1989) models are based on social-cognitive theory(Bandura, 1989), in that learning is shaped in terms of interactions between individual capacities,behaviours,andtheenvironment.Althoughthereisnouniversal,singledefinitionforSRL,manyrecentworkscite thedefinitionprovidedbyPintrich (2000,p.453): “anactive, constructiveprocesswherebylearnerssetgoalsfortheir learningandthenattempttomonitor,regulate,andcontroltheircognition,motivation, and behaviour, guided and constrained by their goals and the contextual features in theenvironment.” This definition reflects a goal-oriented perspective. Consistentwith this perspective, inZimmerman’s view, “students can be described as self-regulated to the degree that they are meta-cognitively, motivationally, and behaviourally active participants in their own learning process”(Zimmerman, 2001, p. 15). Self-regulation in Zimmerman’s social cognitivemodel is cyclic over threephases:forethought,performance,andself-reflection.Pintrich’sframeworkofself-regulationisdenotedviaa four-by-fourgridofphasesandareas.The fourphases include forethought,monitoring, control,andreflection.Theself-regulatoryactivitiesrelatedtoeachphaseoccurinfourdifferentareas,includingthecategoriesof1)cognitive,2)motivationalandaffective,3)behavioural,and4)contextualfeaturesoftheenvironment.Winne and Hadwin’s (1998) model, on the other hand, is inspired by information processing theory(IPT). In this model, SRL is identified in terms of events, and unfolds over four loosely sequenced,potentially recursive stages. These stages include 1) task definition, 2) goal setting and planning, 3)studying tactics,and4)meta-cognitivelyadapting studying techniques.TheacronymCOPES isused inthismodeltodescribeanIPT-basedstructure,intheformofeventunitscommonwithinthefourphases(Winne,Jamieson-Noel,&Muis,2002).ItstandsforConditions,Operations,Products,Evaluations,andStandards.Except foroperations, theother fourelements represent information that learners takeasinputto,orproduceasoutputoftheir learningprocess (Greene&Azevedo,2007).Conditions includeinternal(i.e.,cognitiveconditionssuchasbeliefs,domainknowledge,andmotivation)andexternal(i.e.,task conditions such as instructional cues and time available) information available to a learner that



187

influences how the task will be engaged.Operations represent the cognitive processes, tactics, andstrategiesthatlearnersperformtoaddressthetask.Productsaretheinformationgeneratedasaresultof operations, such as information recalled from memory. Evaluations are internal and externalfeedbackaboutproducts,whilestandardsare thecriteriaagainstwhichproductsandeffectivenessoftheoperationsusedaremonitored.In linewiththevariousmodelsofSRL inthe literature,thereexistdifferentapproachesformeasuringhow learning is self-regulated. In the following,we discuss themeasurementmethods available, andmostlyapplied,inthecurrentliterature.2.2 Self-Regulated Learning: Measurement ToprovideasuccinctyetinclusiveoverviewoftheavailableSRLmeasurementmethods,westartwithdescribingthetaxonomyprovidedbySchraw(2010).Thistaxonomyisbuiltuponthefourarticles inaspecialissueofEducationalPsychologistdedicatedtothetopicofmeasuringSRLconstructs(Greene&Azevedo,2010).Thetaxonomyrepresentsthebigpictureonthistopicwellsincethesearticlesforefrontchallengesandissuesraisedbypioneerresearchersinthisfield.In this taxonomy, Schraw (2010) divides applied measurement strategies into online and offlinemethods. Online methods are those to be applied during students’ active learning episodes, whileofflinemethods are takenbefore or after those episodes.Onlinemethods canbe eitherobtrusive orunobtrusive. Online, obtrusive methods require learners’ “conscious attention.” Such methods thusconsumeaportionof learners’“processingresources”andmightinterferewiththeirflowofcognition(not necessarily in a negative way). Among methods categorized under this label, think-alouds areperhapsthemost frequentlyapplied.Unobtrusivemethods,ontheotherhand,are indicators thatdonot require learners’ attention; they are gathered in the background. The most notable of thesemethods is trace logs. In general, traces can be any type of data collected from users’ actions in alearning environment, such as their clicks on hyperlinks or options selected from a palette. Readingtimes and eye-tracking strategies are other examples of online, unobtrusive methods (Greene &Azevedo,2010;Nesbitetal.,2006;Zhouetal.,2010).Schraw (2010) categorizes offline methods into self-reported beliefs, current abilities, and expectedperformance.Self-reportedbeliefsrepresentlearners’perceptionsabouttheirownbeliefsandabilities.These reportsmay concernmetacognition, epistemology, or self-efficacy, andmight bemeasured ingeneral or within a specific domain, e.g., self-efficacy for computer use. There exist a number ofdifferentself-reportquestionnairesintheliteraturethatareoftenusedinthisregard(Carmen&Torres,2004).For instance, theMotivationalStrategies forLearningQuestionnaire (MSLQ) isoneof themostwidely used self-report questionnaires (Credé & Phillips, 2011; Duncan & McKeachie, 2005). Thisquestionnaire includes 81 items, and aims to assess learners’ motivational orientation and use ofdifferent learning strategies relative to a specific course or subjectmatter (Pintrich, Smith, Garcia, &



188

McKeachie, 1991). The Learning and Study Strategy Inventory (LASSI) is another frequently used self-report questionnairedesignedmainly to assess the learning strategies thatuniversity students reportusing(Weinstein,Schulte,&Palmer,1987).Thisquestionnaire includes77 itemsgroupedaccordingtoten scales, including attitude, motivation, time-organization, anxiety, concentration, informationprocessing,selectionofmain ideas,useof techniquesandsupportmaterials,andself-assessmentandtestingstrategies.Currentabilitiesareexistingabilitiesandskillsthatlearnersbringwiththemtoalearningtask(Schraw,2010). These abilities might be general, such as students’ intelligence or reasoning skills, or moredomain-specific, such as their interest and past achievement in a given subject matter. Expectedperformance can be indicated through learners’ pre-judgement of learning (JOL) regarding a learningtask,aswell as learners’ verbal reportson theirplansand intended strategies for successful learning.Suchreportsindicatehowlearnersdefineandplantocarryouttheirlearninggoals,plusanarticulationoftheircriteriaforsuccessfullearning.Another useful way to examine existingmeasurementmethods is to look at how SRL constructs areconceptualizedintheunderlyingmodel.Winne(2010b)andWinneandPerry(2000)describeprotocolsformeasuringSRLcategorizedbywhethertheymeasureSRLaseventsorasaptitudes.Inventoriesandthink-alouds are themainprotocolsused in theexisting research tomeasureSRLasaptitude. In self-reportinventorieslearnersareusuallyaskedaboutsomecharacteristicoftheirlearningstrategies(e.g.,frequency,likelihood,ordifficulty),inalooselydefinedcontext(e.g.,whenyoustudy,inthiscourse,orfor exams). To answer such questions, learners rely on their memories and previous learningexperiences in similar situations. Their responses are typically limited to apre-defined setof options,e.g.,aLikert-scaleof1–5.Inthink-alouds, learnersareaskedtospeakaboutthoughtsanddecisionsasthey engage in learning. Contrary to self-reports, think-alouds do not require learners to recall frommemory or think in a particular way as directed by instructions in self-reports. Think alouds havepotentialtoindicatedynamicaspectsofSRL(Winne,Zhou,&Egan,2010).Nonetheless,theseindicatorsarelearners’interpretationsabouttheir“in-action”events,andwhodecideswhateventsaretakenintoaccountwhendescribingtheirchoicesanddecisions.Aswell,think-aloudscouldactivatecognitiveandmetacognitiveprocessesthatwouldnotbetriggeredotherwise.UnstructuredinterviewsareanotherprotocolusedtomeasureSRLasaptitudes(Cleary&Zimmerman,2012;Winne,2010b).Theyaresimilartothink-alouds inthatlearnerresponsesarenotlimitedtopre-definedanswersbutdifferentfromthink-alouds(andrathersimilartoself-reports)inthatlearnersareaskedabouttheirSRLprocessesafterthelearningsessionhasfinished,orabouttypicalbehaviourthattheyforeseeusinginfuturelearningsituations(see,forexample,Kitsantas&Zimmerman,2002).Traces(ortrace-logsasphrasedinSchraw’s2010taxonomy)arethemainprotocolused(andsuggestedtobeused) tomeasureSRLasevents (Azevedoetal.,2010;Greene&Azevedo,2010;Winne,2010a,2010b;Winne&Perry,2000).Tracescapturelearners’actionsonthefly,intheauthenticcontextwhere



189

theyhappen.Theyaredefinedas“observable indicatorsaboutcognition that studentscreateas theyengage with a task” (Winne & Perry, 2000, p. 551). For instance, a trace could be that the studenthighlightsatextbecausehe/shemetacognitivelymonitorsitasimportant,orclicksonahyperlinkbasedona judgment thatadditionalcontentwouldbeusefulat thismoment.Althoughtracesare themostsuitable, available method to measure SRL “as the dynamic, contextual and adaptive process it istheorizedtobe”(Winne,2010b,p.275),theycannotandshouldnotbeconsideredtheonlymethodformeasuring SRL constructs (Azevedo et al., 2010; Winne, 2010b). Self-reports and think-alouds stillprovideresearcherswithvaluableinformationonlearnerperceptionsaboutSRL,intentions,beliefs,andsomemotivationalconstructs.Asnotedtoearlier,theunderlyingmodelofSRLaffectshowitshouldbemeasured. Besides clarifying their underlyingmodel and conceptualization of SRL, it is important forresearcherstodecidewhichSRLconstructsorprocessesaremostimportanttotheirresearch,andthenusemeasurementprotocolsmostappropriatetomeasuringthoseprocesses/constructs.Affordances of technology-enhanced learning environments provide opportunities to measure SRLprocesses based on learners’ traces (Greene & Azevedo, 2010; Winne, 2010b; Zhou et al., 2010).Because traces are gathered within the learning environment and in their authentic context, theyempower researchers to track learners’ choices thoroughly and precisely, interactions with learningcontent, learning actions, and tactics applied on the fly. This method is often referred to as TraceMethodology in the contemporary research on measuring SRL features (Nesbit, Xu, Zhou, &Winne,2007;Winne& Perry, 2000; Zhou et al., 2010).Winne (Winne& Perry, 2000;Winne, 2010b) definestracesas“recordsofbehaviour,aformofperformanceassessment,thatprovidegroundsforinferringalearner’scognitiveandmetacognitiveactivities.”Inotherwords,whilethecognitiveandmetacognitivestates of learners might not be visible to researchers, traces are the “observable indicators aboutcognition that students create as they engage with a task” (Winne & Perry, 2000, p. 551). Unlikeinventories and think-aloud methods (see Section 0), trace methodologies enable researchers tounobtrusively track learners’ experiences through actual, in-action evidence of their cognitive andmetacognitive states rather than relying on learners’ perceptions of their use of these processes,sampled from memory (e.g., in self-reports), or the portion that they decide to share with theresearcher(e.g.,inthink-aloudorunstructuredinterviews).Recently,severalproposalshaveemergedfortheuseoftraces(Molenaar&Järvelä,2014;Winne,2014)to study SRL based on sound theoretical principles andwith a strong emphasis on advanced analysismethods suchasprocessmining (Reimann,Markauskaite,&Bannert,2014), graph theory (Hadwinetal., 2007), sequence mining (Zhou et al., 2010), and statistical discourse analysis (Molenaar & Chiu,2014). However, much less attention has been paid to developing a protocol that would guideresearchers inmeasuringandanalyzing SRLprocesses concomitantlywith theeffectsof technologicalinterventionsbasedontracedatacollectedbytechnology-enhancedlearningenvironments.



190

3 THE MEASUREMENT PROTOCOL In this section, first we describe prerequisites to employing our proposed trace-based,microanalyticmeasurement protocol (summarized in Figure 1), and then provide a systematic process to assistresearchersandpractitioners inemploying thisprotocol.Wesupplementeachstepwithmodels fromourpreviousresearch,illustratingcomprehensiveimplementationandapplicationcasesoftrace-based,microanalyticmeasurementprotocols.

Figure1:Trace-basedmicroanalyticmeasurementprotocol:

a)theprerequisites,b)themeasurementprocess.3.1 The Prerequisites ThemeasurementmethodappliedreflectshowSRListheorized.Tosetthestageforvalidinterpretationofmeasurementsandgeneralization, the selection,development, anddeploymentof ameasurementmethod(oracombinationofmethods)shouldalignwiththeunderpinningSRLmodelortheory(Greene&Azevedo,2010;Klugetal.,2011;Winne&Perry,2000;Winne,2010b).Thus,thefirstprerequisiteforthe application of this protocol is a precise and plain formulation of the SRL model underlying theresearchplusthetypeofconceptualizationpursued(e.g.,event,aptitude,orboth),andtheconstructsand/orprocesses (e.g., theplanningprocessormotivationconstructs)of interest to the researcher(s)and intended to be operationalized in a scaffolding intervention. Once the underlying SRL model isidentified,thesecondprerequisiteistotheorizehowscaffoldingsupportsprocesseswithinthismodel,whichleadtoformulatingtheresearchquestionsandhypotheses.Accordingly,thefirststepinapplyingthemeasurement protocol is to define the processes/constructs included in the SRLmodel at a fine-grained,micro level—seeFigure1.a.Thisenablesresearcherstotargethighlyspecificself-regulatorybeliefs and/orbehaviours, supported via a theorized setof scaffolding interventions, as theyoccur in



191

real time through individual engagement in specific (academic/non-academic) tasks across authenticcontexts(Cleary,2011).Weappliedtheproposedprotocolinourresearchtoinvestigatehowasetofscaffoldinginterventionscansupportknowledgeworkers’SRLprocessesintheworkplace(Siadaty,2013;Siadaty,Gašević,etal.,2012). These interventions were tailored to their particular workplaces. In our related work, weprovided a detailed description of the theoretical model along with the empirical and theoreticalgroundingfortheSRLscaffoldinginterventionsandhowtheyarehypothesizedtofacilitatedspecificSRLprocesses(Siadatyetal.,2016a).Inthefollowingsubsections,weillustratehowtheaboveprerequisitesaremetintheapplicationoftheprotocolinthetheorization,design,instrumentation,andmeasurementoftheLearn-Btool.Thesectionon the SRL model offers only an example of how it could be theorized and supported with thefunctionalitiesofa learningtool.Thedetailedtheoreticalunderpinningsofthemodeldiscussed inthefollowingsubsectioncanbefoundinrelatedwork(Siadatyetal.,2016a).3.1.1 TheSRLModelTomeetthefirstprerequisite,inthissectionweillustrateanddiscusstheformulationoftheunderlyingSRLmodelappliedinourresearch.Thismodelconsistsofthreephases:1)planning,2)engagement,and3)evaluationandreflection.TheymanifestthethreecommonphasesacrosstheprincipalSRLmodelsdiscussedinsection0,which,despitetheterminologyused,collectivelycharacterizeanumberofphasesproceedingfromaforethoughtorpreparatoryphase,throughataskperformanceorenactmentphase,toaself-reflectiveandevaluationphase(Dettori&Persico,2008;Puustinen&Pulkkinen,2001;Winters,Greene, & Costich, 2008). The underlying SRL model applied in our research is grounded on thesephases, as they encompass the need of contemporary knowledgeworkers to identify and plan theirlearning goals, apply strategies towardperforming these goals, and reflect on their learningpracticesthatwouldinfluencetheirsubsequentpreparatoryprocesses.Todefinethefine-grainedSRLprocessesin ourmodel, a set of specific self-regulatory activitieswere identified per each phase based on theexistingliterature(Dettori&Persico,2008;Greene&Azevedo,2009;Puustinen&Pulkkinen,2001).AscoinedbyGreene&Azevedo(2009),thethreecommonphasesinourmodelarethe“macro-level,”andthespecificactivitieswithineachare“micro-level”processes.Thesearediscussedinthenextsections.3.1.2 PlanningThis phase contains processes that precede acting and in particular includes activities related toanalyzingataskathand,settingrelatedgoalsandplanningstrategiesforachievingthosegoals(Dettori&Persico,2008;Greene&Azevedo,2009;Puustinen&Pulkkinen,2001;Zimmerman,2008).Entwinedwithtaskanalysis,goalsetting isoftenthepremierstepofaself-regulatory learningprocess.Learnersvarysignificantlyinthetypesandeffectivenessofthegoalstheysetforthemselves(Valleetal.,2009;Zimmerman, 2008). Nevertheless, that there are differences in the nature and goals of learningprocesses betweenworkplace context and formal education should be borne inmind. In educational



192

settings, learning is a goal in itself; inworkplace settings, it ismostly a by-product ofwork and taskperformance (Illeris, 2011; Ley, Kump, & Albert, 2010; Margaryan et al., 2009). Moreover, strategicplanningisalsocloselyassociatedwithgoalsetting,wherebylearnersidentifytheappropriatestrategiestoperforminordertoachievetheirdesiredlearninggoals.Inshort,duringthisphaselearnersanalyzeaspecificsituationand/oridentifytheneedtoenhancetheircompetencies,settheirlearninggoals,selectstrategies to reach them, judge their perceived capability to reach the goals, and take the expectedoutcomesintotheirconsideration.Table1describesthemicro-levelprocessesincludedinthisphase.3.1.3 EngagementTheEngagementphasereferstoprocessesoccurringduringtaskeffort.Thisphasefacilitatesafeedbackloop (Zimmerman& Schunk, 1989) in that learners engage in their learning strategies, observe theirperformance,compareitwithastandardbenchmark(e.g.,withintheorganization)oragoal,andapplyappropriate strategy changes in order to respond to the perceived differences (Dabbagh&Kitsantas,2004; see Table 1 for the micro-level processes included in this phase). This phase thus describeslearners’effortsnotonlytoenacttheirplansandstrategies,butalsotomonitorandtracktheirongoingprogresstowardalearninggoalandapplychangesintheirplannedstrategiesifneedbe.3.1.4 EvaluationandReflectionThis phase refers to processes occurring after a task ends or a goal is achieved. During this phase,learners compare their self-observedperformanceagainst somestandard, suchaspriorperformance,anotherperson’sperformance,oranabsolutestandardofperformance(e.g.,somecriteriaestablishedwithintheirorganization’sculture).Inaddition,learnersreviewandreflectontheirlearningexperience.Onekeyaspectofthisphaseisthegenerationofnewmeta-levelknowledgeaboutthewholelearningprocess, strategies,or self,which in turnaffects subsequentSRLprocesses (Wintersetal.,2008).Thetwomicro-levelprocessesincludedinthisphasearedescribedinTable1.Table1:Micro-levelprocessesincludedintheSRLmodelandtheirdescriptions(Siadaty,Gašević,etal.,2012).Macro-levelSRLprocess

Micro-levelSRLprocess Description

Planning

TaskAnalysisTo get familiarwith the learning context and the definition andrequirementsofa(learning)taskathand

GoalSetting Toexplicitlyset,define,orupdatelearninggoals

MakingPersonalPlans Tocreateplansandselect strategies forachievinga set learninggoal

EngagementWorkingontheTask To consistently engage with a learning task, using tactics and

strategiesApplyingStrategyChanges Toreviselearningstrategies,orapplyachangeintactics

Evaluation&Reflection

Evaluation Evaluatingone’slearningprocessandcomparingone’sworkwiththegoal

ApplyingStrategyChanges Reflectingonindividuallearningandsharinglearningexperiences



193

The theorized SRL model is supported by the scaffolding interventions described in the followingsubsection. The theoretical reasons for the use of specific scaffolds and the empirical results of theireffectsoneachofthetheorizedSRLprocessesarereportedelsewhere(Siadatyetal.,2016a).3.1.5 ScaffoldingInterventionsAs alluded to earlier, existing research has shown that SRL is inherently contextual and the specificfeatures of a learning environment can influence whether learners engage in SRL processes and theextentoftheirengagement(Boekaerts&Cascallar,2006;Whipp&Chiarelli,2004;Winne,2010).Thus,tomeasure theeffectivenessofcertainscaffolding interventionsonusers’SRLprocessing, thesecondprerequisitetoourproposedmeasurementprotocolistopreciselyformulatethecontext,i.e.,howthedesignedinterventionsareexpectedtosupporttheintendedSRLprocesses.Atabasiclevel,contextcanbemodeledviaasetofCondition(If)-Action(Then/Else)statements(Winne,2010b).Thismodelassistsresearcherstodelineatethecontextofusers’SRLprocessingviaatheorizedsetof If-Then transitions,where If’s represent the specific intervention conditions in the learning environment (i.e., inputinformationtousers’ learningprocess),uponwhich, i.e.,Then’s,userscouldchoosetooperateanSRLactionwithrespecttotheappliedSRLmodel.According to the SRL model underlying our research, we have designed and implemented a set ofscaffoldinginterventionstosupportusers’SRLprocessesintheworkplace.Inparticular,thedesignandimplementation of the various functionalities of these interventions were enhanced with 1) socialembeddedness elements to support the social nature of workplace learning and 2) support for theharmonization of individual learning goals with those of the organization to nurture the contextualdimensionoflearningintheworkplace.TheseinterventionsweredevelopedandintegratedinLearn-B,alearningsoftwarewedevelopedandappliedinourresearch.TheLearn-Benvironmentaimstoassistknowledgeworkerstocreate,maintain,andpursuetheirlearninggoalsaccordingtotheircompetencegaps. In the following, we introduce these interventions and formulate the respective contextualhypotheses, as required in the second pre-requisite. Figure 2 depicts these interventions and theirtheorized supporting effect on users’ SRL processingwithin the underlying SRLmodel. Greater detailaboutthedevelopmentand implementationofthese interventionsareprovided inSiadaty,Jovanović,etal.(2012)alongwithsampleeventsfromSiadaty,Gašević,etal.(2012).The notion of context used in the definition of interventions builds onwhatWinne and Hadwin callinternational and external conditions. Conditions are not shown in the theoreticalmodel used in thedesign of Learn-B, given in Table 1, as the model focuses on specific SRL micro-level processes.AccordingtoWinneandHadwin(1998),externalconditionsaresurroundingfactors(e.g., instructionaldesignandsocialnetworks)that influencelearners’decisionsassociatedwithdifferentmicro-levelSRLprocesses. For example, in the case of the studywith Learn-B, social contextwas supported throughdifferent interventions thatallow for social awareness, comparison,andco-operation in the followingsubsections. What Winne and Hadwin (1998) refer to as internal conditions (e.g., affective states,motivation, study skills) are not used in the studieswith the Learn-B software (Siadaty et al., 2016a,



194

2016b).Dataabout internal conditionscanbeobtained throughself-reportedmeasures,physiologicalmeasures,think-aloudprotocols,videoanalysis,discourseanalysis,andsoon(Azevedo,2015).3.1.6 UsageInformationDerivedfromcollectiveknowledge,thisinterventioninformsusersofvariouslearningresourceswithinthe organization and supports the functionality of Recommendation interventions (described in afollowingsection).Inotherwords,thisinterventionwasdevelopedtoinformusersofthesocialcontextof their organization around a particular learning resource. Hence, the respective hypothesis on theeffect of this intervention was that it helps users better understand the social context of theirorganizationregardingtheir learninggoals,andthussupportsusers intheplanningphaseof theirSRLprocesses.

Figure2.Thetheorizedsupportingeffectofthescaffoldinginterventionsonusers’SRLprocessingwithinthecontextoftheLearn-Benvironment.

3.1.7 SocialWaveTheSocialWaveinterventionwasdesignedtobringwavesofthelatestupdatestoknowledgeworkersabout their learninggoals,andthe learningresourcesassociatedwitheachspecificgoal,plusupdatesfromthelearningactivitiesoftheircolleagueswhomtheyfollow.Thefunctionalityofthisinterventionissimilartohavingaspecificnewsfeedforeachspecificlearninggoalorcolleagueaboutwhomtheuserisinterested in receiving updates. We hypothesized that Social Waves originating from users’ learningresourcesortheirfollowedcolleaguessupportusersintheirplanningandengagementprocesses.



195

3.1.8 Progress-o-metersWedevelopedthisinterventiontohelpknowledgeworkersmonitortheirownlearningprogresswithinthe context of their workplace. Being aware of one’s progress in achieving learning goals, observingoneself within the social context of the organization, and comparing personal learning progresswithcolleagues provide userswith the opportunity to gauge their learning strategies, apply the necessarychanges if need be, and reflect on their learning process. The contextual hypothesis was that thisinterventionassistsuserswiththeengagementandevaluation/reflectionphases.3.1.9 UserRecommendationsofLearningGoalsThis intervention enables users to recommend learning goals, alongwith all associated resources, totheircolleagues.Allowingcolleaguestorecommendlearninggoalshelpsuserswith initiatingtheirSRLcycles—i.e.,theplanningphaseinparticular—toperformtaskanalysisandgoalsetting.Userscanthusgaininsightintohowtodefinelearninggoalsthattargettheirlearningneeds,howsuchgoalscouldbeformulated,andhowothermembersoftheorganizationhaveapproachedthesegoals.3.1.10 OrganizationalRecommendationsofCompetencesandLearningPathsThis intervention informs users of the learning objectives and requirements of their organization,represented through a set of pre-defined and established competences, as well as recommendedlearningpathsforachievingthosecompetences.Thishelpslearnersharmonizetheirpersonalgoalswithorganizational needs (Siadaty, Jovanović, Gašević, & Jeremić, 2010). Thus informing workers of theorganizationalcompetencesofhigherimportanceandrelevancetothem,theirorganizationalpositions,andlevelofexpertise,alongwithpathsforattainingeachpotentialcompetence,helpsthemtoidentifytheirlearningneedsandthereforesupportsthemintheirplanning.3.1.11 KnowledgeSharingProfilesThrough knowledge sharing profiles, users canmonitor the extent towhich they share their learningexperienceswithin theirworkplace in terms of owned learning resources, and compare their sharingactivitieswith thoseof other userswithin the same group, project, or organization. This interventioninforms users of the social context of the organization in terms of sharing and contributing to thecollective,thussupportingusersinthereflectionphase.ThisinterventionismeanttohelpusersengageintheReflectionphasethroughincreasedself-awarenessandsocialcomparison.3.2 The Measurement Process OncetheunderlyingSRLmodelisdefinedatboththemacroandmicrolevelsandthesupportprovidedby the scaffolding interventions is contextualized, researchers can capture user traces in a learningenvironment into log files and utilize them tomeasure the effect of the interventions on users’ SRLprocesses/constructs of interest. We recognize three main steps within this measurement process(Figure 1.b). A complete reference for this process of identifying SRL and intervention events can befoundinSiadatyetal.(2016a).



196

3.2.1 IdentifyingSRL/InterventioneventsinthelearningenvironmentBeforecollectingusertracesintologfilesinaspecificlearningenvironment,thoseeventsthatrepresentthe enactment of the desired SRL (micro-level) processes must be identified. Perhaps the moststraightforward way to trace users’ learning activities is to log all their actions as they interact withdifferentmodulesandcomponentsinthelearningenvironment.However,oneofthemainchallengesinanalyzinglogdataformeasurementpurposes,especiallyinlearningenvironments,isthecomplexityandexcessivenessofthedetailcapturedinrawlogs.Logfilescancontainanabundanceoflowlevelevents,such as mouse clicks on different components, which do not necessarily correspond to the learningvariablesandconstructsofinteresttoaresearcher(Zhouetal.,2010).Toaddressthesechallenges,thescopeof tracedevents should includeonly those thatmanifest the researchers’ intendedSRLcontext(definedviathesecondpre-requisite,seeSection0)andthusareofinteresttotheirresearchquestionsandhypotheses.Accordingly, in the first stepof themeasurementprocess, researchersneed todefine the setsofSRLeventsandinterventioneventsthatcouldoccur.BySRLevents,wemeanthoseeventsthatindicatethattheuserhasenactedoneofthemicro-levelSRLprocessesinaccordancewiththeunderlyingSRLmodel,while interventionevents represent theactivationorusageofanyof the scaffolding interventions.Toidentifytheseevents,researchersshouldstartbythoroughlyexaminingtheaffordancesofthelearningenvironmentorsoftwareintermsoftheenvisionedSRLprocessesandscaffoldinginterventions.Descriptions ofmicro-level processes included in the underlying SRLmodel,which are elaborated onduringthefirstpre-requisitetothemeasurementprocess(seeSection0),areusedasareferenceguidefor identifying the relatedSRLevents available in the intended learningenvironment. Lookingup thisreferenceguide,researchersshouldexaminealltheavailablesoftwarecomponentsanduser interfaceelementspermicro-levelprocess,anddecidewhetherandhowanyoneofthesecomponentsallowsforenactingaparticularprocess.Forinstance,wehavedefinedtheGoalSettingmicro-levelSRLprocessastoexplicitlyset,define,orupdatelearninggoals.ExaminingtheLearn-Benvironment,wedocumentedthat users can define a new Learning Goal by “dragging and dropping an available competence to anew/existing learning goal” or update their learning goal by “removing a learning path from acompetence.”Sampleevents foreachmicro-levelSRLprocesswithin theLearn-BenvironmentcanbeseeninTable2.



197

Table2.Themicro-levelprocessesincludedintheunderlyingSRLmodelandsampleindicatoreventsfromtheLearn-Benvironment(Siadaty,Jovanović,etal.,2012).

Micro-levelSRLprocess SampleSRLEvents1

TaskAnalysisClicking on different competences under duties or projects related to the userSearchingforakeyword

GoalSettingDragging and dropping an available competence to a new/existing learning goalAddinganewlearningpathtoaneworexistingcompetence

MakingPersonalPlansChoosing an available learning path as the path for a certain competenceAssigningarecommendedlearningpathasthechosenpathforacompetence

WorkingontheTaskRequest collaboration for a competence, learning path, or learning activityMarkingalearninggoal,competence,orlearningactivityas“completed”

ApplyingStrategyChangesAddinganewactivitytoanexistinglearningpathUpdatingthepropertiesofalearningactivity,e.g.,itsname,startdate,expectedduration,visibility,rating,keywords,anduserprogress

EvaluationRating a learning path, learning activity, or knowledge assetAdding new keywords to or updating existing keywords of a learning goal,competence,learningpath,learningactivity,orknowledgeasset

ApplyingStrategyChangesAdding a comment for a competence, learning path, or learning activityRecommendingalearninggoaltoacolleague

Todefinethe interventionevents,researchersshouldfollowanapproachsimilartotheoneaboveforSRL events. Here, the reference guide is the hypothesized effect of each intended intervention (asdocumentedwithinthesecondpre-requisite;seeSection3.1formoredetails).Referringtothisguide,theresearchersthenscanthedifferentformsandfeaturesofeachscaffolding intervention,whichareimplementedandavailable inthe intendedsoftware,anddocumentthedifferentwaysthatuserscantriggeragivenscaffoldingintervention.Forinstance,UsageInformationisoneoftheinterventionsthatwetheorizedwouldassistuserswiththeirplanningpractices.ThisinterventionwasimplementedwithintheLearn-Benvironmentviathreedifferentfeatures:Analytics,SocialStream,andSocialStand,whichwere available for various resources such as competences, learning paths, and learning activities.Analytics, for example, provided users with statistical information such as the number of times thatcompetences required for a specific duty were added the learning goals of other users. Thus, wedocumentedthatuserscantriggerthisinterventionbyclickingthe“Achievementtab(underAnalytics)

1WithinthescopeoftheresearchpresentedinSiadaty,Jovanović,etal.(2012),userlearninggoalsaredefinedintermsofcompetences.Associatedwitheachcompetencecomesoneormorelearningpath(s).Eachlearningpathiscomprisedofoneormorelearningactivities,andleadstotheattainmentofaspecificcompetenceataspecificlevel. Each learning activity is accompanied by a set of metadata specifying its content, process and planninginformation (e.g. title, description, average time required, and deliverymode), and a set of knowledge assets.Knowledge assets can be in the form of learning contents such as documents, books, weblogs, videos, orpresentations;orhumanresourcessuchasaknowledgeablecolleaguewhohasalreadysuccessfullycompletedthislearningactivity.Finally,a(learning)resource istheumbrellatermusedtorefertoacompetence, learningpath,learningactivity,orknowledgeasset.



198

ofanavailablecompetence,learningpath,orlearningactivity”;or,byclicking“thecommentstabofacompetence, learningpath, learningactivity,or knowledgeasset”which represented theSocial Standdimension of this intervention. Table 3 provides sample events for the proposed interventionsimplementedwithinLearn-B,asdiscussedinSection3.1.2.AppendixA:SRLandInterventionEventsinthe Learn-B Environment provides the full list of all SRL and intervention events collected in ourresearch.Sucheventsneednotnecessarilybeunique;thatis,aneventcouldberepresentativeofoneor more micro-level SRL processes, or one or more scaffolding interventions. For instance, we hadconsideredtheevent“Addinganewcompetencetoanexistinglearninggoal”asatraceableactionforbothmicro-levelSRLprocessesofApplyingAppropriateStrategyChangesandGoalSetting,categorizedrespectivelyundertheEngagementandPlanningmacro-levelphases.

Table3.TheproposedscaffoldinginterventionandsampleeventsfromtheLearn-Benvironment(Siadaty,Jovanović,etal.,2012).

InterventionFeature SampleInterventionEvents

UsageInformation

ClickingontheAchievementtab(underAnalytics)ofanavailableCompetence,LearningPath,orLearningActivityClickingonthedatatabofaCompetence,LearningPath,LearningActivity,orKnowledgeAsset

SocialWaveClickingontheSocialWavetabofone’sLearningGoal,Competence,LearningPath,LearningActivity,orKnowledgeAssetClickingonDuties,Roles,Tasks,orProjectsfolder

Progress-o-meters

ClickingontheGoal-o-metertab(underAnalytics)ofone’sLearningGoalClickingontheProgress-o-metertab(underAnalytics)ofaLearningActivity

UserRecommendationsofLearningGoals

ClickingonasingleLearningGoalundertheRecommendedLearningGoalsfolder

OrganizationalRecommendationsofCompetencesandLearningPaths

ClickingonUserswhoareacquiring/havealreadyacquiredanavailablecompetenceClickingonaLearningPathforanavailablecompetenceClickingonthedatatabofanavailableLearningPath,LearningActivity,orKnowledgeAsset

KnowledgeSharingProfilesClickingonone’sAnalyticstab(theKnowledgeSharingProfilestabistheonlytab,sowillopenautomatically)

3.2.1 TranslatingTracestoSRLeventsOncethedesiredSRLandinterventioneventsaredefined,atrackingtoolcanbedevelopedtocaptureandrecordtheseeventsinlogfiles.However,basedontheaffordancesofagivenlearningenvironment,users’ actual, traceable actions are most often at a lower granularity than the desired events thatresearchers aim to track and capture. That is, the learning environment captures user interfaceelementsnotdirectlyrelatedtothe“semantics”ofSRLmicro-levelprocessesandinterventions.Often



199

anentiresequenceoftraceableactionsrepresentsasingleevent.Tofullycapturealltheoccurrencesofadesiredeventinaspecificlearningenvironment,therefore,researchersneedtofirstidentifytheseriesoflowlevelevents(e.g.,atthelevelofmouseclicks)thatrepresenteachofthedesiredSRL/interventionevents (e.g., creating a new learning goal/task). The previous step of the protocol (see section 0) ismeanttoassistresearchers in identifyingtheseseries.Accordingly,researcherscanstartbyexaminingthe primary paths — i.e., a sequence of actions within the learning environment — that lead toperformingaspecificevent.Oncetheinitialsetofsequentialactionsis identified,researchersneedtoexaminevarioususecases,suchasalternatepaths,forperformingthedesiredSRL/interventioneventstoensurethatallpossibleactionsleadingtothateventinthegivenlearningenvironmentareidentified.Atrackingtoolcanthenbedevelopedtocaptureandstorethese identifieduseractions into logfiles.After all the log files are collected, researchers need to develop aparsing algorithm to translate thefiner-graineduseractionsintochunksofcoarser-graineddesirableevents.Tothisend,researcherscancodeasequenceoflow-level,fine-grainedactionsintopatternsthatrepeatacrossusers’loggedactions.Next,patternmatchingmechanismssuchasregularexpressions(Mitkov,2003,p.784)canbeutilizedtodevelop an algorithm to mine users’ log files and locate the matching patterns as defined by theresearcher,translatingthemintothedesiredSRL/interventionevents.So far in this paper, we have focused on SRL events and described how they were defined,operationalized, and measured. In the following, we describe how this step of the measurementprotocolwasconductedinourresearch.The raw learning logs generated by the system during users’ interactions within the Learn-Benvironmentoftencontainedtwoormoreeventspermouseclick.Eacheventwascapturedasastand-alonerecordwithauniqueidentifier;somewerecommonacrossvarioususeractions,andsomewerespecifictooneormoreaffordancesofthesoftware.Forinstance,whenusersclickedondifferentnodesrepresenting various learning resources, two basic eventswere always fired: one selecting the node,whichwecalledaSelectNodeEvent,andoneopeningatabontheright-handsideofthetoolwheretheusercouldseemoredetailedinfoaboutthatresource,whichwecalledanOpenTabEvent.Ontheotherhand,creatingalearninggoal,auser-definedlearningpath,learningactivity,orknowledgeassetwereallrepresentedthroughtheCreateevent.Thus,theattributeeventType(showninFigure3)wasusedtodenotethetypeofeacheventasitwasrecordedbythesystem.Moreover,tocapturetheSRLeventsintheirauthenticsettings,itwasimportantthatusers’actionsbecaptured in their full context, containing all the events involved, as well as the detailed informationrelated to each event. This additional information, for instance, included the resourceType (shown inFigure3)onwhichtheeventwasapplied,theuniqueidentifiertoaccessthatresource,andadditionalinformationsuchasthenameofthenodeaccessedanditshierarchicallevelinthetool.Accordingly,asusersinteractedwiththedifferenttoolswithintheLearn-Benvironment,datawereloggedatthelevelof the abovementioned eventTypes, accompanied by additional information (if any), and written torecordsinthedatabasetablessetupinadvanceforthispurpose.



200

Figure3showsasnapshotof the tracedata,collectedby thesystemonthe fly.For instance,whenauser added a new keyword to a knowledge asset, the system recorded that a TaggingEvent wasperformedbythisspecificuser,onthislearningresource,followedbyanEditevent.Botheventsweretime-stamped to the full date and timeof their occurrence. Further information included typeof tag(e.g.,keyword),textofthetagitself(e.g.,“documentation”),thetypeandtitleofthelearningresourcebeingtagged(e.g.,“KnowledgeAsset”and“BasicsofSWdocumentation—standards,”respectively),aswell as the URIs (Uniform Resource Identifier, which indicates where a resource is stored) of all theentities involved, i.e.,theuser,theknowledgeasset,andtheuser-addedtag.Thebluebox inFigure3representsthisexample.

Figure3.AsnapshotofthelogfilesgeneratedwithintheLearn-Benvironment.

ToidentifypatternsindicatingoccurrenceoftheintendedSRLeventswithinthecollectedtracedata,inthesecondstepofthemeasurementprocess,wedevelopedapattern library.This libraryconsistedofpatterns of sequential event types corresponding to each of the previously defined SRL events.Wesystematically examined the Learn-B environment to identify these patterns, that is, sequences oflower-level events triggered, alongwith their specificdetails, and capturedby the tracking toolwhenusersperformedeachoftheSRLevents(seeTable2).Thisincludedrunningeachofthemicro-levelSRLprocessesviatheirindicatoreventsasdefinedinthepreviousstep,andrecordingeachtriggeredevent.For instance, one of the identified SRL events for task analysis micro-level process was “exploringcompetencesincludedinothercolleagues’learninggoals.”Performingthissingleactiontriggeredthreelow-leveleventsintheenvironment:



201

1) AnOpenTabEventcouldhavebeenfired,dependingonwhetherusersclickedontheUserstabintheLearn-Binterfaceordirectlyapproachedaspecificcompetence

2) ASelectNodeEvent(clickingonthedesiredcompetence)wouldhappen3) AnOpenTabEventwasfiredtoprovidemoreinformationaboutthatcompetence

Figure4.ashowsthesethreelow-leveleventsandthedetailedinformationspecifictoeachofthem.

Figure4.Asamplepatterninthepatternlibrary,denotingtheSRLevent“exploringcompetencesincludedinothercolleagues’learninggoals”:a)patternspecification,b)patternimplementedusingregularexpressions.

To implement the parsing process,we developed an analyzermodule called Log Parser. Thismodulereceived users’ raw log files as input andmatched them against thepatterns defined in the patternlibrary — see Figure 1.b.II. The input was in the form of textual collections of trace records acrossvariouslearningsessions,retrievedfromtherespectivedatabasesandaggregatedintooneconcretelogfile per user (a portion of such log files is shown in Figure 3). The patternswere defined in terms ofregular expressions. Regular expressions, or “regex” in short, arebasically stringsof characterswhichdenote a pattern and are oftenused to find a particular text, replace itwith other text or validate a



202

giveninput(Habibi,2004).Figure4bshowstheregeximplementationofthethree-eventpatternfortheSRLeventexploringothercolleagues’competencesasshownontheleftofFigure4.2

Thepattern-matchingalgorithmintheLogParserwasimplementedtofirstsearchforoccurrencesofallavailablepatternsdefinedinthepatternlibraryinusers’logfiles,andthenreplacethematchingeventrecordswithasinglerecordindicatoroftheoccurrencewhenasuccessfulmatchwasfound.Thus,theoutputoftheLogParserwasanewversionofeachuser’slogfileinthatextraneouseventsthatdidnothave amatching pattern in thepattern library were removed from user’s traces, and the output filecontained only coarser-grained SRL (and Intervention) events aggregated from users’ lower-leveltraceableactions.Figure5showsaportionofauser’srawlogfileastheinputtotheLogParser(a),andtheevent-izedlogfileastheoutputofthismodule(b).Ascanbeseeninthisfigure,theseriesoflower-level traces generated for the event record 15313, for example, are translated into the interventionevent User clicking on an Activity. This represents the intervention feature “OrganizationalRecommendations of Competences and Learning Paths.” The traces for event record 15315 are thentranslated intotheSRLeventUserclickingonacolleague’scompetences,whichrepresentsthemicro-levelSRLprocess“TaskAnalysis.”

Figure5.Asampleuser’sa)rawlogfile,b)event-izedlogfileastheoutputoftheLogParser.TheblueboxshowsthetraceablerecordsrelatedtotheSRLeventinFigure4.

2 Tools for pretty printing of regular expressions are available, but their discussion is beyond the scope of thispaper. Some relevant discussion can be found, for example, at http://stackoverflow.com/questions/5312301/is-there-some-good-visual-regular-expression-editor



203

3.2.2 CreatingContingencyrecordsandTransitiongraphs

Translatingusers’rawtracesintoevent-izedlogfilesasdescribedintheprevioussection(anddepictedin Figure 1.b.II) allows for operationalizing the event-based conceptualization of SRL in terms of anevent’soccurrence(Azevedoetal.,2010;Winne,2010b;Winne&Perry,2000).Anoccurrenceprovidesawindowfortheresearchertoobservetheevidence(orproduct)ofusercognitionoperations.Hence,anoccurrence is merely a tally of an observable state and does not convey any information about thecontext. As such, occurrences allow for performing frequency counts of users’ engagement in SRLprocesses,buttheyfailtocapturetransitionsbetweenthoseprocesses.Suchtransitionscanprovideafuller picture of the contextual, dynamic nature of self-regulated learning, especially in today’stechnology-enhancedlearningenvironments(Winne,2010a).

To include elements of context in the trace-basedmicroanalysis measurement of SRL processes, weneed to explore the contingencies between users’ actions, and build the transition graphs of thesecontingencies in the third step of the measurement process. This allows for going beyond simplefrequencycountsofactionsandoccurrencesandinsteadprobingintothecontextofsuchoccurrences,suchasthestatesprecedingasubsequentSRLevent.Contingencies ingeneralshowwhatsubsequenteventwasprecededbywhichpriorevent.Accordingly, they includesomefeaturesofcontext inthemand operationalize the event-based approach tomeasuring SRL at a higher level thanoccurrences. Acontingencycanbemodelledintermsofaconditional if-then (condition-action)relationship(Winne&Perry,2000;Winne,2010b),whereasetof if-then transitionscanbeusedtorepresentthecontextofusers’SRLprocessingasdiscussedearlier inSection3.1.Forexample,whenit isobserved inthetracedataof a givenuser thathe/she“includesa specific competence in their learninggoals” immediatelyafter“knowingaboutotherusers’commentsaboutthatcompetence,”theformereventrepresentsthecondition,andthelatterdemonstratestheaction.Here,theconditionisalearningeventindicatinguseoftherespectivescaffoldingintervention,whiletheactionisanSRLeventindicativeofthegoal-settingmicro-levelprocess.Forexample,ifoutof20competencesincludedinauser’slearninggoals,12wereincluded immediately after viewing other users’ comments,we candescribe a conditional probabilitythatthisusermetacognitivelyconsiderscollectiveintelligenceimportantwhencreatinghis/herlearninggoals(Pr[otherscomments|includedcompetence]=12/20or0.6).

Inadditiontothecontingenciesbetweenlearners’enactmentofSRLevents(andotherrelevantevents,such as scaffolding Intervention events), researchers can build transition graphs of users’ learningactionsandexaminethepredominanttransitionpatterns.Transitiongraphsillustrateusers’navigationbetween various events, e.g., using some scaffolding Interventions and performing SRL processes.Moreover,theycanrevealavarietyofquantitativemeasuresadaptedfromgraphtheory,whichcouldbe used in describing features of SRL processes. Centrality metrics, such as degree, betweenness,closeness,andeigenvectorcentrality,canbeusedtodenotetherelativeimportanceofanodewithinagraph (Borgatti, Mehra, Brass, & Labianca, 2009; Landherr, Friedl, & Heidemann, 2010; Yan & Ding,2009).Forexample,centralitymetricsenableustolookatthemoreinfluentialprocessesacrossusers’learningsessions,orexploreusefulpatternssuchasstrategiesmorepracticedbyuserswithacommon



204

background,organizationalposition,orprojectteam.AdetaileddiscussionoftheuseofgraphstatisticsforanalysisofdifferentaspectsofSRLisprovidedbyHadwinetal.(2007).

Tominetheexistingcontingenciesandbuildthetransitiongraphsinthethirdstepofthemeasurementprocess, researchersneed toperformanother levelofpatternmatchingonusers’event-ized log files.Theobjectiveofthisstepistotranslateallindicatorevents,i.e.,thoseincludedinthepatternlibrary,totheir respective SRL (and Intervention) events. Hence, the pattern library could be extended withadditionalsetsofhigherlevel,moregeneralpatternsthatshowwhichpatternsmanifestengagementin,orusageofwhichSRLorInterventionevent.Forexample,Figure6.aliststhetextualdescriptionofthesetofSRLeventsthatshowtheWorkingonTheTaskmicro-levelSRLprocess,asdescribedinAppendixA.Figure6bshowshowthismicro-levelSRLpatterncanalsobedefinedintermsofaregularexpressioncontainingthesetofitsrepresentativeSRLevents,addedtotheextendedpatternlibrary.Theblueboxin this figureshowsoneof theseSRLeventsgenerated fromauser’s low-levelmouseclicks into theirevent-izedlogfile,asshowninFigure5.

Figure6.ApatternintheextendedpatternlibrarythatdenotesalloftheSRLeventsindicativeofthe“WorkingontheTask”micro-levelSRLprocess:a)listoftheindicativeSRLeventsasdescribedinAppendixA;b)the

patternimplementedinREGEX.

It is important to note that depending on the underlying SRLmodel, each intervention or SRL eventcould be an occurrence indicator for one or more of the researcher’s intended micro-level SRLprocesses.Inthethirdstepofthemeasurementprocess,hence,whentranslatingtheSRL/interventioneventstohigher-levelmicro-levelSRLprocesses,eachoccurrenceofaneventshouldbetranslatedinto



205

asmanymicro-levelprocessesas it isdefined tobe representing (seeSection0). Forexample, inourresearchwithintheLearn-Benvironment,theSRLevent“addinganewlearningobject(e.g.,uploadingadocumentorabookmarktoawebpage)toanexisting learningactivity”wasdefinedasanSRLeventindicativeofthreedifferentmicro-levelSRLprocesses:1)GoalSetting,2)MakingPersonalPlans (bothrelatedtotheplanningprocess),and3)WorkingontheTask—relatedtotheengagementprocess(seeAppendixA).Accordingly, each instanceof theeventpattern “User.Activity.AddNewAsset” in a user’slogfilewastranslatedintoallthreeSRLevents(seeFigure7).

Contingencyrecords,builtfromuserdata,demonstratethesequentialstreamofauser’sactionsastheusers interactwith different entities in a learning environment. In the last step of themeasurementprocess, a graph analysis tool can be used to build the transition graphs out of users’ contingencyrecords.Oncethetransitiongraphsaregenerated,graphanalysistechniquescanbeappliedtoelucidategraph statistics and study users’ SRL processes and use of scaffolding interventions within a givenlearningenvironment.

Figure7:Asampleuser’sa)event-izedlogfile,b)event-izedlogfiletranslatedintocontingencyrecords.Theblueboxesshowhowoneeventpatterncouldbeindicativeof,andthustranslatedinto,twoormore

SRL/Interventionevents.

In our research within the Learn-B environment, we used the Gephi open source software (Bastian,Heymann,&Jacomy,2009)tobuildthetransitiongraphsfromusers’contingencyrecords,andanalyzethe generated graphs inorder topaint a fuller pictureof how theproposed scaffolding interventionsand/orengagedinSRLprocesseswereused.TheGephitoolsupportsmultiplefileformatsforimportingthegraphfile, includingCSV,GDF,andGEXF.WeusedtheCSVfileformattostoreusers’tracedatainterms of nodes and edges in a graph, a format that could then be imported into the Gephi tool togeneratethetransitiongraphs.Here,atransitiongraphshowsconditionalcontingencies,wherenodesrepresentuseractions—i.e.,performingSRLorInterventioneventswithintheLearn-Benvironment—andeachdirectionallinkbetweennodesrepresentsatransitionbetweentwoevents.



206

Figure8showsasampletransitiongraphgeneratedfromauser’stracedata.Foralessclutteredvisualrepresentation, only the links of interest to our research question,3 i.e., those directed from anInterventioneventtoothernodes(eitherSRLorInterventionnodes),areshowninthisgraph.Thebiggerthe size of a node, the more influential in users’ learning events, and the thicker a link, the morefrequent that contingency has appeared in a user’s parsed trace data. To investigate our researchquestion,wecalculatedthegraphtheoreticcentralitymeasures(Bastianetal.,2009)foreachproposedscaffolding intervention within the graph of user learning actions. Centrality denotes the relativeimportance of a nodewithin a graph and could be identified via degree, betweenness, closeness, oreigenvectorcentrality,themostcommonlyusedcentralitymeasuresinvariousdomains(Borgattietal.,2009; Freeman, Roeder, &Mulholland, 1979; Landherr et al., 2010; Yan & Ding, 2009). These graphstatistics canbeused to reveal the relative importanceof researchers’ desired interventionswithin anetworkofuserlearningactions.

Figure 8: A sample transition graph from a user’s log file translated into a time-stamped sequence ofcontingencies. Only the links starting at the Intervention nodes are shown in this graph. The size of a nodeindicatesitsinfluenceinthegraph;thethicknessofalinkshowshowfrequentlythatlinkoccurredinusertracedata.

3 In one of our relevant research questions,wewere particularly interested in investigating themost effectivescaffolding interventions (developedwithin the Learn-B environment) in supporting users’ SRL processeswithintheLearn-Benvironment.



207

InthecontextoftheLearn-Benvironment,wewereinterestedtofindoutwhether(andtowhatextent)usage of the proposed intervention could account for engagement in SRL processes within thisenvironment. Accordingly, in the context of our research, centralitywas considered to represent theimportanceofan interventionorSRLeventwithinthenetworkofuser learningactions intheLearn-Benvironment.Forinstance,interventioneventswithhigherdegreescouldindicateavarietyofdifferentusagesinthelearningprocess.InterventionnodeswithhigherclosenessvaluesindicatethatuserscouldeasilyperformtheirSRLpracticesorusetheotherinterventions.HighbetweennessvaluesspecifythoseinterventionsusedasabridgetoperformSRLpracticesorotherinterventions.Interventioneventswithhighereigenvaluecentralitiesdenotethoseusedbefore/afterotherwell-performedevents.Inadditionto centrality metrics, graph statistics can be correlated with more variables collected from users’learningactionsorviaother instruments,suchasself-reports, tostudytheeffectof thosecofoundingvariables (such as users’ computer skills or motivational strategies) on users’ SRL processes. Forexample, confounding variables such as general computer skills or familiarity with organizationalresponsibilities,collectedviaself-reports,couldpotentiallyaffectthefrequencyofuserengagementincertainSRLprocesses,whichcouldbeidentifiedviathecalculatedcentralitymetrics.

Inthispaper,wedonotreportonthefindingsoftheapplicationsofthisprotocolinthecontextoftheLearn-Benvironment,which insteadarecovered inSiadatyetal. (2016a,2016b).Weonly summarizethemain results here. As shown in Figure 8, through its central position and also confirmed throughfuturemultipleregressionanalyses,wehavefoundthatSocialWave interventionaccounts for68%ofthevariance inusers’ total frequencyofperformingall theSRLprocessesconsidered inourstudy(c.f.Table1). This interventionwas followedby another thatoffered system-generated recommendationsabout learningpaths, learningactivities,andknowledgeassetstostimulateengagementinmicro-levelprocesseswithintheforethoughtorpreparatoryphaseofSRL.Ouranalysisofcovariateinfluence,suchasusers’ computer skillsorexperience in theirorganizationalpositions,detected insignificanteffects.Siadaty et al. (2016a, 2016b) indicate that both the social and the organizational contexts should betaken into account when tailoring SRL interventions to support the forethought and engagementphases.

4 IMPLICATIONS FOR RESEARCH AND PRACTICE

Although the elements of our research strategy and the proposed measurement protocol are notcompletely new, together they suggest, in our view, a new, unique, forward-looking approach fordesigning scaffolding interventions and measuring their support for SRL processes in technology-enhanced learning environments. Self-regulated learning has become an essential skill in today’sknowledge-driven, rapidly changing society where individuals increasingly opt in to informal learningsettingsandhenceneedtoself-managetheirlearningprocesses.However,asresearchonlearningandmetacognitiveprocessessuggests,most learnersarepronetoassumptionsandbeliefsthatcan impairtheireffectivenessasself-managedlearners(Bjorketal.,2013);inotherwords,theysimplydonotknowhow to learn (Margaryan, Milligan, & Littlejohn, 2009). Self-regulatory interventions can thus assist



208

learnerstodefineandmanagetheirownlearningprocesses.AnimportantimplicationofthisprotocolisthatitguidesresearcherstoinvestigatetheeffectivenessoftheirdesiredSRLinterventionsgroundedinan explicit theoretical framework and in the authentic context of their application. Our proposedprotocol can be applied in any technology-enhanced learning environmentwhere the authenticity ofusers’ learningactions in their real context is important to researchers. Linkingpracticeand research,theproposedprotocolguides researchers to1) formulate theirhypotheses regarding the roleofeachdesigned intervention with regard to the underlying theoretical model and 2) avoid developinginterventions isolated from real practice by designing, implementing, and aligning the integrity,effectiveness, and measurement of those SRL interventions with the nature of the actual learningenvironment.

Anotherdirectimplicationoftheproposedmeasurementprotocolforbothresearchandpracticeisthat,through its prerequisites, it guides researchers to examine the provided support for SRL processes intheirentirety, includingthoserelatedtoallSRLphasesasdefinedintheunderlyingSRLmodel.Havingreviewed the existing literature (e.g., Dettori& Persico, 2008;Greene&Azevedo, 2009; Puustinen&Pulkkinen,2001;Sitzmann&Ely,2011),wehavearticulatedasetofmoregenericprocesseswithinthethreephasesofourunderlyingSRLmodelasmacro-levelprocesses,anddefinedthespecificactivitieswithineachofthesephasesasmicro-levelSRLprocesses.Thesemicro-andmacro-levelprocessescanbe(re-)usedasaguidebyotherresearcherstobuildtheirveryownunderlyingSRLmodel.Accordingly,examining the effect of the provided support at the level of the defined micro-processes enablesresearchers to provide amore accurate pictureof the role of their intended interventionswithin thelargerconstructofSRL.

Because SRL processes are dynamic and contextual, the proposed protocol pursues an event-basedconceptualizationofthoseprocessesandaimstomeasurethemasasequenceofevents(traces)intherealcontextwheretheyhappen.PioneeredbyWinneandassociates,tracingmethodologyhasstartedtofinditswayasanothermethodforexaminingself-regulatedlearningprocessesinformal,educationalsettings (Hadwin et al., 2007; Winne & Jamieson-Noel, 2002; Zhou & Winne, 2012). Compared toquestionnaires, trace data are not bonded to a certain point in time, and holistically operationalize“what users do as they do it” (Winne, 2010a, 2010b). In our view, the trace-basedmethodology, thecoreoftheproposedmeasurementprotocol,togetherwiththemicroanalyticalmeasurementmethod,provideadistinctivelensthroughwhichresearcherscanaccuratelymeasureandanalyzehowlearners’SRLprocessesaresupportedbytheintendedinterventions.Comparablewiththepotentialobjectiveofmicroanalyticalprotocolsinformaleducation(Cleary,Callan,&Zimmerman,2012),thiscombinationoftrace-based methodology and microanalytical measurement can guide researchers in interventionplanninganddevelopmentforvarioustechnology-enhancedlearningenvironments.

Some practical implications are also brought forth by the proposed protocol. First, application of theproposedmeasurementprotocolrequiresresearcherstodesignanddeveloptheirintendedscaffoldinginterventions in terms of software and log users’ learning actions indicative of their use of SRLprocesses/scaffolding interventions. This would require the research team to have access to



209

programmerstoimplementtheintendedinterventionsforthem,aswellastobuildparticular“hooks”intothe intendedlearningenvironmentstocollectandtraceusers’generatedSRL/interventionevents(seeSection0).Second,theapplicationoftheprotocolcanbeadditionallycostly,asimplementationoftheparsingalgorithmsfortheSRL/interventionpatternsrequiresresourcesfordevelopment,andgiventhe contextual nature of SRL and the potential use of different software tools in different learningenvironments,suchparsinglibrariesmaynotbeeasilytransferablefromoneenvironmenttoanother.Itshould be noted that the above implications, however, do not originate from our proposedmeasurementprotocol;rathertheyareinherenttoaneedforeffectivelysupportingthecollectionandanalysisoftracedatageneratedthroughsoftwaresystemsthataimtoscaffoldSRLprocessesforusers.

5 FUTURE DIRECTIONS

The proposed trace-based, microanalytic measurement protocol is built upon the event-basedconceptualizationofSRLusedtocaptureknowledgeworkers’SRLprocessesaccuratelyontheflyandintheirauthenticcontext.Asdiscussedearlier,anyappliedSRLmeasurementmethodologyshouldbe inaccordancewith the underlying conceptualization (Greene&Azevedo, 2010). In future research, thismethodology can be complemented with other forms of data that conceptualize SRL as an aptitude(Winne&Perry,2000;Winne,2010b).Although tracescanprovide researcheswithdetailed,valuableinformationonusers’learningactivities“inaction,”theyaresubjecttosomelimitationsandbiases,andthus,notinherentlythebestoronlymethodforgatheringdata(Winneetal.,2010a,2010b).Aptitudes,mostcommonlymeasuredviaself-reports,arealsoessentialtoresearchingSRL,astheyrepresentwhatusershave“inmind”whentheyengageinSRLprocesses.Together,tracesandself-reportscanbeusedtopaintamuchfullerandmoredetailedpictureofusers’actualengagementinSRLprocesses.Althoughself-reportedmeasurescanbeusedasanintrinsicpartoflearningenvironmentstomeasuresomeothermotivationalconstructs—forexample,goalorientationinacademicsettings(Zhou&Winne,2012)orperceived usefulness of the scaffolding — in our research within the Learn-B environment,measurements of some important SRL-based “aptitudes” can be done as by-products of such self-reports. In our future research, we aim to investigate the effective combinations and accordinglyelucidate the best research practices in combining our proposed measurement protocol with othermethods and instruments, such as self-reports or think-alouds, which could connect our proposedprotocolwiththeworksuggestedbyBannert,Reimann,&Sonnenberg(2014).

Additionally,werecommendthatfutureresearchinvestigatetheextenttowhichdifferenttools,aimedto provide scaffolding support, can support SRL processes. SRL is contextual; each software or tooldeliversdifferentcognitiveaffordances,accordingtowhichdifferenttracesofusers’SRLactivitiescouldbe expected and captured. These issues point to a need to design and evaluate howmore mature,different typesof toolscansupportdifferentmacro-andmicro-levelSRLprocesses.Methodologically,other modelling methods can be used instead of or in combination with graph theory (e.g., clusteranalysis,hiddenMarkovmodels,sequence,andprocessminingalgorithms).Questionstoprobeinclude1)whatcommonstrategieslearnersfollowwhenengaginginspecificSRLmicro-levelprocesses,2)what



210

is the probability of changing one learning strategy for another under the influence of specific SRLtechnological scaffolding interventions, and 3) what types of processes are mutually exclusive withdifferentprocess-miningalgorithms.

6 ACKNOWLEDGEMENTS

The authors are thankful to Professor Philip H. Winne of Simon Fraser University for his guidancethroughoutthedevelopmentofthemeasurementprotocolandhiscommentsofdraftversionsofthispaper.

REFERENCES

Azevedo, R. (2015). Defining and measuring engagement and learning in science: Conceptual,theoretical, methodological, and analytical issues. Educational Psychologist, 50(1), 84–94.http://dx.doi.org/10.1080/00461520.2015.1004069

Azevedo, R., Moos, D. C., Johnson, A. M., & Chauncey, A. D. (2010). Measuring cognitive andmetacognitive regulatory processes during hypermedia learning: Issues and challenges.EducationalPsychologist,45(4),210–223.http://dx.doi.org/10.1080/00461520.2010.515934

Bandura,A.(1989).Humanagencyinsocialcognitivetheory.AmericanPsychologist,44(9),1175–1184.http://dx.doi.org/10.1037/0003-066X.44.9.1175

Bannert,M., Reimann, P.,& Sonnenberg, C. (2014). Processmining techniques for analysing patternsandstrategies instudents’self-regulated learning.MetacognitionandLearning,9(2),161–185.http://dx.doi.org/10.1007/s11409-013-9107-6

Bastian, M., Heymann, S., & Jacomy, M. (2009). Gephi: An open source software for exploring andmanipulatingnetworks.InProceedingsofthe3rdInternationalAAAIConferenceonWeblogsandSocial Media (ICWSM ’09) (pp.361–362). San Jose, CA, USA. Retrieved fromhttp://aaai.org/ocs/index.php/ICWSM/09/paper/view/154/1009

Bjork,R.A.,Dunlosky,J.,&Kornell,N.(2013).Self-regulatedlearning:Beliefs,techniques,andillusions.Annual Review of Psychology, 64, 417–444. http://dx.doi.org/10.1146/annurev-psych-113011-143823

Boekaerts,M.(1997).Self-regulated learning:Anewconceptembracedbyresearchers,policymakers,educators, teachers, and students. Learning and Instruction, 7(2), 161–186.http://dx.doi.org/10.1016/S0959-4752(96)00015-1

Boekaerts,M., & Cascallar, E. (2006). How far havewemoved toward the integration of theory andpractice in self-regulation? Educational Psychology Review, 18(3), 199–210.http://dx.doi.org/10.1007/s10648-006-9013-4

Borgatti,S.,Mehra,A.,Brass,D.,&Labianca,G.(2009).Networkanalysisinthesocialsciences.Science,323(April),892–896.http://dx.doi.org/10.1126/science.1165821

Carmen, M., & Torres, G. (2004). Self-regulated learning: Current and future directions. ElectronicJournalofResearchinEducationalPsychology,2(1),1–34.

Cleary, T. J. (2011). Emergence of self-regulated learning microanalysis. In B. J. Zimmerman & D. H.Schunk(Eds.),Handbookofself-regulationoflearningandperformance(pp.329–345).London:Routledge.



211

Cleary, T. J., Callan, G. L., & Zimmerman, B. J. (2012). Assessing self-regulation as a cyclical, context-specificphenomenon:OverviewandanalysisofSRLmicroanalyticprotocols.EducationResearchInternational,2012,1–19.http://dx.doi.org/10.1155/2012/428639

Cleary, T. J., & Zimmerman, B. J. (2012). A cyclical self-regulatory account of student engagement:Theoretical foundations andapplications. In S. L. Christenson,A. L. Reschly,&C.Wylie (Eds.),Handbookofresearchonstudentengagement(pp.237–257).Boston,MA:Springer.

Credé, M., & Phillips, L. A. (2011). A meta-analytic review of the Motivated Strategies for LearningQuestionnaire. Learning and Individual Differences, 21(4), 337–346.http://dx.doi.org/10.1016/j.lindif.2011.03.002

Dabbagh,N.,&Kitsantas,A.(2004).Supportingself-regulationinstudent-centeredweb-basedlearningenvironments.InternationalJournalonE-Learning,3(1),40–46.

Dabbagh,N.,&Kitsantas,A. (2005).Usingweb-basedpedagogical toolsas scaffolds for self-regulatedlearning.InstructionalScience,33(5–6),513–540.http://dx.doi.org/0.1007/s11251-005-1278-3

Dettori,G.,&Persico,D. (2008).Detecting self-regulated learning inonline communitiesbymeansofinteraction analysis. IEEE Transactions on Learning Technologies, 1(1), 11–19.http://dx.doi.org/10.1109/TLT.2008.7

Duncan, T. G., & McKeachie, W. J. (2005). The making of the Motivated Strategies for LearningQuestionnaire. Educational Psychologist, 40(2), 117–128.http://dx.doi.org/10.1207/s15326985ep4002_6

Freeman, L. C., Roeder, D.,&Mulholland, R. R. (1979). Centrality in social networks: II. Experimentalresults.Socialnetworks,2(2),119–141.http://dx.doi.org/10.1016/0378-8733(79)90002-9

Greene,J.A.,&Azevedo,R.(2007).AtheoreticalreviewofWinneandHadwin’smodelofself-regulatedlearning: New perspectives and directions. Review of Educational Research, 77(3), 334–372.http://dx.doi.org/10.3102/003465430303953

Greene, J.A.,&Azevedo,R. (2009).Amacro-levelanalysisofSRLprocessesand their relations to theacquisition of a sophisticatedmental model of a complex system. Contemporary EducationalPsychology,34(1),18–29.http://dx.doi.org/10.1016/j.cedpsych.2008.05.006

Greene, J. A., & Azevedo, R. (2010). The measurement of learners’ self-regulated cognitive andmetacognitive processes while using computer-based learning environments. EducationalPsychologist,45(4),203–209.http://dx.doi.org/10.1080/00461520.2010.515935

Habibi,M.(2004).Javaregularexpressions:Tamingthejava.util.regexengine.Berkeley,CA:Apress.Hadwin,A.F.,Nesbit,J.C.,Jamieson-Noel,D.,Code,J.,&Winne,P.H.(2007).Examiningtracedatato

explore self-regulated learning. Metacognition and Learning, 2(2–3), 107–124.http://dx.doi.org/10.1007/s11409-007-9016-7

Hadwin, A. F.,Oshige,M.,Gress, C. L. Z.,&Winne, P.H. (2010). Innovativeways for using gStudy toorchestrate and research social aspects of self-regulated learning. Computers in HumanBehavior,26(5),794–805.http://dx.doi.org/10.1016/j.chb.2007.06.007

Illeris,K.(2011).Workplacesandlearning.InM.Malloch,L.Cairns,K.Evans,&B.N.O’Connor(Eds.),TheSAGEhandbookofworkplacelearning(pp.32–45).London:SAGE.

Karoly, P. (1993). Mechanisms of self-regulation: A systems view. Annual Review of Psychology,44(1993),23–52.http://dx.doi.org/10.1146/annurev.ps.44.020193.000323

Kitsantas, A., & Zimmerman, B. J. (2002). Comparing self-regulatory processes among novice, non-expert,andexpertvolleyballplayers:Amicroanalyticstudy.JournalofAppliedSportPsychology,14,91–105.http://dx.doi.org/10.1080/10413200252907761



212

Klug, J., Ogrin, S., & Keller, S. (2011). A plea for self-regulated learning as a process: Modelling,measuringandintervening.PsychologicalTestandAssessmentModeling,53(1),51–72.

Landherr, A., Friedl, B., & Heidemann, J. (2010). A critical review of centrality measures in socialnetworks. Business & Information Systems Engineering, 2(6), 371–385.http://dx.doi.org/10.1007/s12599-010-0127-3

Ley, T., Kump, B., & Albert, D. (2010). Amethodology for eliciting, modelling, and evaluating expertknowledge for an adaptive work-integrated learning system. International Journal of Human-ComputerStudies,68(4),185–208.http://dx.doi.org/10.1016/j.ijhcs.2009.12.001

Margaryan,A.,Milligan,C.,&Littlejohn,A.(2009).Self-regulatedlearningandknowledgesharingintheworkplace: Differences and similarities between experts and novices. Paper presented at the2009ResearchingWorkandLearning(RWL)Conference,Roskilde,Denmark.

McLoughlin,C.,&Lee,M.J.W.(2007).Socialsoftwareandparticipatory learning:PedagogicalchoiceswithtechnologyaffordancesintheWeb2.0era.InR.J.Atkinson,C.McBeath,S.K.A.Soong,&C.Cheers(Eds.),ICT:ProvidingChoicesforLearnersandLearning(ASCILITE2007)(pp.664–675).Singapore: Centre for Educational Development, Nanyang Technological University. Retrievedfromhttp://www.ascilite.org/conferences/singapore07/procs/

Mitkov, R. (Ed.). (2003). TheOxford handbook of computational linguistics. Oxford: OxfordUniversityPress.

Molenaar, I., & Chiu, M. M. (2014). Dissecting sequences of regulation and cognition: Statisticaldiscourse analysis of primary school children’s collaborative learning. Metacognition andLearning,9(2),137–160.http://dx.doi.org/10.1007/s11409-013-9105-8

Molenaar,I.,&Järvelä,S.(2014).Sequentialandtemporalcharacteristicsofselfandsociallyregulatedlearning.MetacognitionandLearning,9(2),75–85.http://dx.doi.org/10.1007/s11409-014-9114-2

Nesbit,J.C.,Winne,P.H.,Jamieson-Noel,D.,Code,J.,Zhou,M.,MacAllister,K.,…Hadwin,A.F.(2006).Using cognitive tools in gSTudy to investigate how study activities covary with achievementgoals. Journal of Educational Computing Research, 35(4), 339–358.http://dx.doi.org/10.2190/H3W1-8321-1260-1443

Nesbit,J.C.,Xu,Y.,Zhou,M.,&Winne,P.H.(2007).Advancingloganalysisofstudentinteractionswithcognitive tools. Presented at the 12th Biennial Conference of the European Association forResearchonLearningandInstruction(EARLI),28August–1September,Budapest,Hungary

Pintrich, P. R. (2000). The role of goal orientation in self-regulated learning. In M. Boekaerts, P. R.Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 451–502). San Diego, CA:AcademicPress.http://dx.doi.org/10.1016/B978-012109890-2/50043-3

Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1991). A manual for the use of theMotivatedStrategiesforLearningQuestionnaire(MSLQ).AnnArbor,MI:UniversityofMichigan.

Puustinen,M.,&Pulkkinen,L.(2001).Modelsofself-regulatedlearning:Areview.ScandinavianJournalofEducational,45(3),269–286.http://dx.doi.org/10.1080/00313830120074206

Reimann,P.,Markauskaite,L.,&Bannert,M.(2014).E-Researchandlearningtheory:Whatdosequenceandprocessminingmethodscontribute?BritishJournalofEducationalTechnology,45(3),528–540.http://dx.doi.org/10.1111/bjet.12146

Schraw, G. (2010). Measuring self-regulation in computer-based learning environments. EducationalPsychologist,45(4),258–266.http://dx.doi.org/10.1080/00461520.2010.515936

Schunk,D.,&Zimmerman,B. J. (Eds.). (1994).Self-regulationof learningandperformance: Issuesandeducationalapplications.Hillsdale,NJ:LawrenceErlbaumAssociates.



213

Siadaty,M. (2013). Semantic web-enabled interventions to support workplace learning. (UnpublishedDoctoralDissertation,SimonFraserUniversity,Vancouver,BC,Canada).

Siadaty, M., Gašević, D., & Hatala, M. (2016a). Associations between technological scaffolding andmicro-level processes of self-regulated learning: A workplace study. Computers in HumanBehavior,55,PartB,1007–1019.http://dx.doi.org/10.1016/j.chb.2015.10.035

Siadaty, M., Gašević, D., & Hatala, M. (2016b). Measuring the impact of technological scaffoldinginterventions on micro-level processes of self-regulated workplace learning. Submitted toComputersinHumanBehavior.

Siadaty,M.,Gašević,D., Jovanović, J., Pata,K.,Milikić,N.,Holocher-Ertl, T.,…Hatala,M. (2012). Self-regulatedworkplacelearning:Apedagogicalframeworkandsemanticweb-basedenvironment.EducationalTechnology&Society,15(4),75–88.

Siadaty, M., Jovanović, J., Gašević, D., & Jeremić, Z. (2010). Leveraging semantic technologies forharmonization of individual and organizational learning. In M. Wolpers, P. A. Kirschner, M.Scheffel,S.N.Lindstaedt,&V.Dimitrova(Eds.),SustainingTEL:FromInnovationtoLearningandPractice:5thEuropeanConferenceonTechnologyEnhancedLearning,EC-TEL2010(LectureNotesin Computer Science, Vol. 6383) (pp. 340–356). Barcelona, Spain: Springer.http://dx.doi.org/10.1007/978-3-642-16020-2_23

Siadaty,M.,Jovanović,J.,Gašević,D.,Milikić,N.,Jeremić,Z.,Ali,L.,…Hatala,M.(2012).Semanticwebandlinkedlearningtosupportworkplacelearning.InS.Dietze,M.D’Aquin,&D.Gašević(Eds.),Proceedingsofthe2ndInternationalWorkshoponLearningandEducationwiththeWebofData(LiLe2012atWWW’12).Retrievedfromhttp://ceur-ws.org/Vol-840/06-paper-29.pdf

Sitzmann, T., & Ely, K. (2011). Ameta-analysis of self-regulated learning inwork-related training andeducational attainment: What we know and where we need to go. Psychological Bulletin,137(3),421–442.http://dx.doi.org/10.1037/a0022777

Steffens,K.(2006).Self-regulatedlearningintechnology-enhancedlearningenvironments:LessonsofaEuropean peer review. European Journal of Education, 41(3–4), 353–379.http://dx.doi.org/10.1111/j.1465-3435.2006.00271.x

Valle, A., Núñez, J. C., Cabanach, R. G., González-Pienda, J. A, Rodríguez, S., Rosário, P.,… Cerezo, R.(2009).Academicgoalsandlearningqualityinhighereducationstudents.TheSpanishJournalofPsychology,12(1),96–105.

Weinstein,C.E.,Schulte,A.C.,&Palmer,D.R.(1987).Learningandstudystrategiesinventory(LASSI).Clearwater,FL:H&HPublishing.

Whipp, J., & Chiarelli, S. (2004). Self-regulation in a web-based course: A case study. EducationalTechnologyResearchandDevelopment,52(4),5–22.http://dx.doi.org/10.1007/BF02504714

Winne,P.H.(2005).Aperspectiveonstate-of-the-artresearchonself-regulatedlearning. InstructionalScience,33(5),559–565.http://dx.doi.org/10.1007/s11251-005-1280-9

Winne,P.H.(2010a).Bootstrappinglearner’sself-regulatedlearning.PsychologicalTestandAssessmentModeling,52(4),472–490.

Winne, P. H. (2010b). Improving measurements of self-regulated learning. Educational Psychologist,45(4),267–276.http://dx.doi.org/10.1080/00461520.2010.517150

Winne,P.H. (2014). Issues in researchingself-regulated learningaspatternsofevents.MetacognitionandLearning,9(2),229–237.http://dx.doi.org/10.1007/s11409-014-9113-3

Winne,P.H.,&Hadwin,A.F.(1998).Studyingasself-regulatedlearning.InD.J.Hacker,J.Dunlosky,&A.C.Graesser(Eds.),Metacognitionineducationaltheoryandpractice(pp.227–304).Hillsdale,NJ:LawrenceErlbaumAssociates.



214

Winne, P. H., & Jamieson-Noel, D. (2002). Exploring students’ calibration of self reports about studytactics and achievement. Contemporary Educational Psychology, 27, 551–572.http://dx.doi.org/10.1016/S0361-476X(02)00006-1

Winne,P.H.,Jamieson-Noel,D.,&Muis,K.(2002).Methodological issuesandadvancesinresearchingtactics,strategies,andself-regulatedlearning.InP.R.Pintrich,M.L.Maehr(Eds.),Advancesinmotivationandachievement:Newdirections inmeasures andmethods (Vol.12, pp. 121–155).Bingley,UK:EmeraldGroupPublishingLimited.

Winne,P.H.,&Perry,N.E.(2000).Measuringself-regulatedlearning.InM.Boekaerts,P.R.Pintrich,&M. Zeidner (Eds.), Handbook of self-regulation (pp. 531–566). San Diego, CA: AcademicPress.http://dx.doi.org/10.1016/B978-012109890-2/50045-7

Winne, P. H., Zhou, M., & Egan, R. (2010). Designing assessments of self-regulated learning. In G.Schraw,D.H.Robinson(Eds.),Assessmentofhigher-orderthinkingskills.Currentperspectivesoncognition,learningandinstruction(pp.89–121).NewYork:Routledge.

Winters,F. I.,Greene,J.A.,&Costich,C.M.(2008).Self-regulationof learningwithincomputer-basedlearning environments: A critical analysis. Educational Psychology Review, 20(4), 429–444.http://dx.doi.org/10.1007/s10648-008-9080-9

Wolters,C.A.(2010).Self-regulatedlearningandthe21stcenturycompetencies.[Onlinemanuscript]Retrievedfromhttp://www.hewlett.org/uploads/Self_Regulated_Learning__21st_Century_Competencies.pdf

Yan, E., & Ding, Y. (2009). Applying centrality measures to impact analysis: A coauthorship networkanalysis.JournaloftheAmericanSocietyforInformationScienceandTechnology,60(10),2107–2118.http://dx.doi.org/10.1002/asi.21128

Zhou,M.,&Winne,P.H. (2012).Modelingacademicachievementbyself-reportedversus tracedgoalorientation. Learning and Instruction, 22(6), 413–419.http://dx.doi.org/10.1016/j.learninstruc.2012.03.004

Zhou, M., Xu, Y., Nesbit, J. C., & Winne, P. H. (2010). Sequential pattern analysis of learning logs:Methodology and applications. In C. Romero (Ed.),Handbookof educational datamining (pp.107–121).BocaRaton,FL:CRCPress.

Zimmerman,B.J.(1990).Self-regulatedlearningandacademicachievement:Anoverview.EducationalPsychologist,25(1),3–17.http://dx.doi.org/10.1207/s15326985ep2501_2

Zimmerman,B. J. (2001). Theoriesof self-regulated learningandacademicachievement:Anoverviewand analysis. In B. J. Zimmerman&D.H. Schunk (Eds.),Self-regulated learning and academicachievement:Theoreticalperspectives(2nded.,pp.1–37).Mahwah,NJ:LawrenceErlbaum.

Zimmerman, B. J. (2008). Goal setting: A key proactive source of academic self-regulation. In D. H.Schunk&B.J.Zimmerman(Eds.),Motivationandself-regulatedlearning:Theory,researchandapplications(pp.267–290).NewYork:LawrenceErlbaum.

Zimmerman, B. J., & Schunk, D. H. (Eds.). (1989). Self-regulated learning and academic achievement:Theory,research,andpractice.NewYork:SpringerVerlag.



215

APPENDIX A: SRL AND INTERVENTION EVENTS IN THE LEARN-B ENVIRONMENT

SRLEvents

Macro-LevelSRLProcess:Planning

Micro-LevelSRLProcess SRLEventsinLearn-B

Task/Analysis ClickingonDuties,Roles,TasksorProjectsfolders

ClickingonasingleDutyundertheDutiesfolder

ClickingonasingleRoleundertheRolesfolder

ClickingonsingleTaskundertheTasksfolder

ClickingonsingleProjectundertheProjectsfolder

ClickingondifferentCompetencesrelatedtoaDuty,Role,TaskorProject

Exploringcompetencesincludedinothercolleagues’learninggoals

Searchingforakeyword

GoalSetting Creatinganewgoal

Dragginganddroppinganavailablecompetencetoaneworanexistinglearninggoal

AddinganewCompetencetoaneworanexistinglearninggoal

AddinganewLearningPathtoaneworanexistingcompetence

AddinganewLearningActivitytoaneworanexistinglearningpath

AddinganewKnowledgeAssettoaneworanexistinglearningactivity

RemovingaCompetencefromalearninggoal

DeletingaLearningPathfromacompetence

RemovingaLearningActivityfromalearningpath

RemovingaKnowledgeAssetfromanlearningactivity

SettingthepropertiesofaLearningGoale.g.itsname,deadline,visibility,priority,keywordsanduser’sprogress

SettingthepropertiesofaCompetence,e.g.itsname,deadline,visibility,currentuser’slevel,desiredlevel,keywordsanduser’sprogress

SettingthepropertiesofaLearningPath,e.g.itsname,expectedduration,



216

visibility,rating,keywordsanduser’sprogress

SettingthepropertiesofaLearningActivity,e.g.itsname,startdate,expectedduration,visibility,rating,keywordsanduser’sprogress

SettingthepropertiesofaKnowledgeAsset,e.g.itsname,URL,expectedduration,visibility,rating,keywordsanduser’sprogress

SharingaLearningGoalwitharecommendedcolleague

RequestingcollaborationforaCompetence,LearningActivityoraKnowledgeAsset

MakingPersonalPlans


AssigningarecommendedLearningPathasthechosenpathforacompetence


AddinganewLearningPathtoaneworanexistingcompetence

AddinganewLearningActivitytoaneworanexistinglearningpath

AddinganewKnowledgeAssettoaneworanexistinglearningactivity


Removingasub-Competencefromanuppercompetence

RemovingaLearningPathfromacompetence



SettingthepropertiesofaLearningPath,e.g.itsname,expectedduration,visibility,rating,keywordsanduser’sprogress

SettingthepropertiesofaLearningActivity,e.g.itsname,startdate,expectedduration,visibility,rating,keywordsanduser’sprogress

SettingthepropertiesofaKnowledgeAsset,e.g.itsname,URL,expectedduration,visibility,rating,keywordsanduser’sprogress

Macro-LevelSRLProcess:Engagement


WorkingontheTask

AssigningarecommendedLearningPathasthechosenpathforacompetence

RequestingcollaborationforaCompetence,LearningActivityoraKnowledge



217

Asset

MarkingaCompetenceas“favourite”

FollowingaCompetence


RecommendingaLearningGoaltoacolleague

Searchingforakeyword

MarkingaLearningGoal,Competence,orLearningActivityas“completed”

Leaving a comment for a Competence, Learning Path, Learning Activity orKnowledgeAsset

UpdatingthepropertiesofaLearningGoale.g. itsname,deadline,visibility,priority,keywordsanduser’sprogress

Updating thepropertiesof aCompetence, e.g. itsname,deadline, visibility,currentuser’slevel,desiredlevel,keywordsanduser’sprogress

UpdatingthepropertiesofaLearningPath,e.g.itsname,expectedduration,visibility,rating,keywordsanduser’sprogress

Updating the properties of a Learning Activity, e.g. its name, start date,expectedduration,visibility,rating,keywordsanduser’sprogress

Updating thepropertiesofaKnowledgeAsset,e.g. itsname,URL,expectedduration,visibility,rating,keywordsanduser’sprogress

Followingacolleague

CreatingalearninggroupforaCompetence

ApplyingappropriateStrategyChanges

AddinganewCompetencetoanexistinglearninggoal

Addinganewsub-Competencetoanexistingcompetence

UpdatingthepropertiesofaLearningGoale.g. itsname,deadline,visibility,priority,keywordsanduser’sprogress

Updating thepropertiesof aCompetence, e.g. itsname,deadline, visibility,currentuser’slevel,desiredlevel,keywordsanduser’sprogress

UpdatingthepropertiesofaLearningPath,e.g.itsname,expectedduration,visibility,rating,keywordsanduser’sprogress

Updating the properties of a Learning Activity, e.g. its name, start date,expectedduration,visibility,rating,keywordsanduser’sprogress

Updating thepropertiesofaKnowledgeAsset,e.g. itsname,URL,expectedduration,visibility,rating,keywordsanduser’sprogress



218


Removingasub-Competencefromanuppercompetence

Followingorunfollowingacompetence


AddinganewLearningActivitytoanexistinglearningpath

AddinganewKnowledgeAssettoanexistinglearningactivity

RemovingaLearningPathfromacompetence



Macro-LevelSRLProcess:Evaluation&Reflection


Evaluation

RatingaLearningPath,LearningActivityoraKnowledgeAsset

MarkingaLearningGoal,Competence,orLearningActivityas“completed”

LeavingacommentforaCompetence,LearningPath,LearningActivityorKnowledgeAsset

AddingnewkeywordstoorupdatingexistingkeywordsofaLearningGoal,competence,LearningPath,LearningActivityorKnowledgeAsset

Reflection

LeavingacommentforaCompetence,LearningPath,LearningActivityorKnowledgeAsset

AddingnewkeywordstoorupdatingexistingkeywordsofaLearningGoal,competence,LearningPath,LearningActivityorKnowledgeAsset

UpdatingthevisibilitypropertyofLearningGoal,competence,LearningPath,LearningActivityorKnowledgeAsset


RecommendingaLearningGoaltoacolleague

InterventionEvents

InterventionI:ProvidingUsageInformation

InterventionFeature InterventionEventsinLearn-B

Analytics ClickingontheAchievementtab(underAnalytics)ofanavailableCompetence,LearningPathorLearningActivity

ClickingonDutiesnode(thesummarytabwillshowintherightpanel)



219

SocialStream ClickingontheSocialWavetab(underAnalytics)ofanavailableCompetence,LearningPath,LearningActivityorKnowledgeAsset

SocialStand ClickingonthecommentstabofaCompetence,LearningPath,LearningActivityorKnowledgeAsset

ClickingonthedatatabofaCompetence,LearningPath,LearningActivityorKnowledgeAsset

InterventionII:SocialWave

InterventionFeature InterventionEventsinLearn-B

GenericSocialWave Clickingonone’sSocialWavetab

LearningResources’SocialWaves ClickingontheSocialWavetabofone’sLearningGoal,Competence,LearningPath,LearningActivityorKnowledgeAsset

BubbleSocialWaves ClickingontheSocialWaveBubblestab(underAnalytics)ofanavailableCompetence,LearningPath,LearningActivityorKnowledgeAsset

ClickingonDuties,Roles,TasksorProjectsfolder

ClickingonasingleDutyundertheDutiesfolder

ClickingonasingleRoleundertheRolesfolder

ClickingonsingleTaskundertheTasksfolder

ClickingonsingleProjectundertheProjectsfolder

InterventionIII:Progress-o-meters

ClickingontheGoal-o-metertab(underAnalytics)ofone’sLearningGoal

ClickingontheCompetence-o-metertab(underAnalytics)ofone’sCompetence

ClickingontheProgress-o-metertab(underAnalytics)ofaLearningPath

ClickingontheProgress-o-metertab(underAnalytics)ofaLearningActivity

InterventionIV:User-recommendedLearningGoals

ClickingonasingleLearningGoalundertheRecommendedLearningGoalsfolder

InterventionV:RecommendedavailableCompetences

ClickingondifferentCompetencesrelatedtoaDuty,Role,Task,orProject

ClickingonUserswhoareacquiring/havealreadyacquiredanavailablecompetence

InterventionVI:RecommendedavailableLearningPaths,LearningActivities,andKnowledgeAssets



220

ClickingonaLearningPathforanavailablecompetence

ClickingonaLearningActivitywithinanavailablelearningpath

ClickingonaKnowledgeAssetrelatedtoanavailablelearningactivity

ClickingonarecommendedLearningPath

ClickingonanabandonedLearningPath,i.e.apreviouslychosenrecommendedlearningpath

ClickingonthedatatabofanavailableLearningPath,LearningActivity,orKnowledgeAsset

InterventionVII:KnowledgesharingProfiles

Clickingonone’sAnalyticstab(theKnowledgeSharingProfilestabistheonlytab,sowillopenautomatically)

Documents

Trace-Based Microanalytic Measurement of Self-Regulated ... · Trace-based microanalytic measurement of self-regulated learning processes. ... 3.0) 183 Trace-Based Microanalytic Measurement