View
235
Download
0
Category
Preview:
Citation preview
Supporting Higher Education to Integrate Learning Analyticshttp://sheilaproject.eu/
LAPolicy:DevelopinganIns4tu4onalPolicyforLearningAnaly4csusingtheRAPIDOutcomeMappingApproach
LAK’17Workshop13thMarch2017
Yi-ShanTsai,DraganGašević,PedroJ.Muñoz-Merino,ShaneDawson,MarenScheffel,AlexanderWhitelock-Wainwright
Workshop scheduleTime Schedule
13.00-13.05 Welcome&introduc5on
13.05-14.30 Fivepresenta5ons&discussions1. Overview&LAinAustralia:ProfDraganGasevic(University
ofEdinburgh)2. LAinEurope(SHEILAProject):ProfPedroJ.Muñoz-Merino
(UniversidadCarlosIIIdeMadrid)3. SHEILA–groupconceptmapping:MarenScheffel(Open
UniversityoftheNetherlands)4. SHEILA–ins5tu5onalinterviews&survey:DrYi-ShanTsai
(UniversityofEdinburgh)5. SHEILA–studentsurvey:AlexanderWhitelock-Wainwright
(UniversityofLiverpool)
14.30-15.00 A^ernoonTea
15.00-16.00 Groupworkontheassessmentofins5tu5onalreadiness
16.00-17.00 LAPolicydra^ing
National project to benchmark LA status, policy and practices for Australian Universities
Dragan Gasevic (thanks Shane Dawson for the slides!)
Introduction
Massiveinterestandinvestmentindataandanaly5cs
• Academicperformance• Studentreten5on• Pastoralcare• Academicliteracies• Socialnetworks–collabora5ons
Aims
understand current LA practice in Australia unpack the challenges to institutional adoption identify practices that can aid the implementation of LA
Approach
2 complementary but separate studies
Study 1 – interviews with senior institutional leaders Study 2 – concept mapping with LA expert panel
Study 1
First study Interviews with 32 Universities:
Identification of current practice, methods and approaches Identification of key drivers for institutions, stage of development, process for implementation, project leads
Study 1
Much interest in LA Stated organisational priority LA projects were in the early phases of implementation and small scale (at time of interview July 2014) 2 distinct clusters across variables such as: implementation, conceptualisation, readiness
Cluster 1 (n=15) – Solutions focused Cluster 2 (n=17) – Process focused
Strategic capability
1. Solutions focused › LA to address a pressing need › Time sensitive
2. Process focused › Networked and integrated model › Minimal time pressures › Innovation and experimentation
Study 2
What are the ideal dimensions for long term sustainable uptake of LA?
Invite to Australian and international LA experts 28 completed the entire concept mapping phases Prompt: ‘for LA to make a continued impact on learning and teaching it would need to…’ 3 phases – brainstorming; sorting and ranking of statements
Bringing it together
Study 1 – 2 clusters Study 2 – 7 clusters
Essentially – how an organisation approaches its conceptualisation of LA underpins (2 clusters) the method for deployment and adoption (7 clusters)
Bringing it together
Challenges to be addressed: Leadership awareness Teams are seldom interdisciplinary IT driven and system focused Scale versus understanding Capabilities and skills deficit. Over reliance on current research – requires further validation across different contexts to demonstrate transportability of models
Complexity
Leveraging the outcomes of short term goals for long term gain How do we merge both models to gain both short and long term impact?
Conclusion
LA requires alternate models for implementation and leadership › Enabling leadership › Whole of organisation › Models that are agile and research informed
Working in complexity creates friction › Embrace the friction – generates innovation
Conclusion
A solutions based model can drive change – but need to be mindful of responding to changing organsational needs Process based model can drive innovation and interest – but need to be mindful of how to scale
Conclusion
Combined model framed in the organisational context › Small, diffuse pockets of innovation to build
capacity and build interest › View to scale adoption – demonstration of
impact (technical, pedagogical) › Distributed enabling leadership (complexity
leadership)
Conclusion
Any “successful” adoption of LA will be dependent on an institution’s ability to rapidly recognise and respond to the organisational culture and the concerns of all stakeholders.
SHEILAProject-Overview-
DraganGašević,Yi-ShanTsai,
MarenScheffel,ShaneDawson,
PedroJ.Muñoz-MerinoMarch,13,2017LAKSHEILAWorkshopVancouver,Canada
hgp://sheilaproject.eu/
SHEILAPartners
• UniversityofEdinburgh(UE)(Coordinator)• BrusselsEduca5onServices(BES)• OpenUniversityNetherlands(OUNL)• TallinnUniversity(TLU)• UniversidadCarlosIIIMadrid(UC3M)• EuropeanAssocia5onforQualityAssuranceinHigherEduca5on(ENQA)
• ErasmusStudentNetwork(ESN)
Policydevelopmentframework
Usedbydifferenthighereduca5onins5tu5onsRecognizesins5tu5onspecificdrivers
Promoteslearninganaly5csforforma5veassessment
hgp://sheilaproject.eu/
Projectapproach
Macfadyen,L.,Dawson,S.,Pardo,A.,Gašević,D.,(2014).Thelearninganaly5csimpera5veandthesociotechnicalchallenge:Policyforcomplexsystems.Research&Prac+ceinAssessment,9(Winter2014),17-28.
WP2
Projectapproach
Macfadyen,L.,Dawson,S.,Pardo,A.,Gašević,D.,(2014).Thelearninganaly5csimpera5veandthesociotechnicalchallenge:Policyforcomplexsystems.Research&Prac+ceinAssessment,9(Winter2014),17-28.
WP1andWP3
Projectapproach
Macfadyen,L.,Dawson,S.,Pardo,A.,Gašević,D.,(2014).Thelearninganaly5csimpera5veandthesociotechnicalchallenge:Policyforcomplexsystems.Research&Prac+ceinAssessment,9(Winter2014),17-28.
WP3
Projectapproach
Macfadyen,L.,Dawson,S.,Pardo,A.,Gašević,D.,(2014).Thelearninganaly5csimpera5veandthesociotechnicalchallenge:Policyforcomplexsystems.Research&Prac+ceinAssessment,9(Winter2014),17-28.
WP3
Projectapproach
Macfadyen,L.,Dawson,S.,Pardo,A.,Gašević,D.,(2014).Thelearninganaly5csimpera5veandthesociotechnicalchallenge:Policyforcomplexsystems.Research&Prac+ceinAssessment,9(Winter2014),17-28.
WP4
SHEILAapproach
Macfadyen,L.,Dawson,S.,Pardo,A.,Gašević,D.,(2014).Thelearninganaly5csimpera5veandthesociotechnicalchallenge:Policyforcomplexsystems.Research&Prac+ceinAssessment,9(Winter2014),17-28.
WP4
Tsai,Y-S.,Gašević,D.(2016).Adop+onofLearningAnaly+csinHigherEduca+on:State,Challenges,andPolicies(Execu+vesummary).SHEILAProjectReport,hgp://bit.ly/sheila_lr_es
Projectac5vi5es
LiteraturereviewInterviewswithdecisionmakers
Groupconceptmappingwithexperts
hgp://sheilaproject.eu/
Projectac5vi5es
LiteraturereviewInterviewswithdecisionmakers
GroupconceptmappingwithexpertsSurveysandfocusgroupswithstudentsandteachingstaff
hgp://sheilaproject.eu/
Expected5meline
Policydevelopmentframework(v1)–June2017Fourins5tu5onalpolicies(v1)–June2017
hgp://sheilaproject.eu/
Expected5meline
Policydevelopmentframework(v1)–June2017Fourins5tu5onalpolicies(v1)–June2017
Policydevelopmentframework(v2)–March2018Fourins5tu5onalpolicies(v2)–March2018
hgp://sheilaproject.eu/
Interestedinjoining?
Associatepartnersasearlyadoptersofthepolicydevelopmentframeworkarewelcome
hgp://sheilaproject.eu/
GroupConceptMappingPhase1:
Brainstorming
Phase2: Sor5ng
Phase3: Ra5ng
Anessen5alfeatureofahighereduca5onins5tu5on'slearninganaly5cspolicyshouldbe...
PointMap1 2
3
4 5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
2930
31
32
33
34
35
36
37
38
39 40
41
42
43
44
45
46
47
48
49
50
51
5253
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
7879 80
81
82
83
84
85
8687
88
89
90
91
92
93
94
95
96
97
9899
ClusterMap
1.privacy&transparency
2.roles&responsibili4es(ofallstakeholders)
3.objec4vesofLA(learnerandteachersupport)
4.risks&challenges
5.datamanagement
6.research&dataanalysis
Ra5ngMap–Importance
1.privacy&transparency
2.roles&responsibili4es(ofallstakeholders)
3.objec4vesofLA(learnerandteachersupport)
4.risks&challenges
5.datamanagement
6.research&dataanalysis
ClusterLegendLayerValue15.08to5.2725.27to5.4635.46to5.6545.65to5.8455.84to6.03
Ra5ngMap–Ease
1.privacy&transparency
2.roles&responsibili4es(ofallstakeholders)
3.objec4vesofLA(learnerandteachersupport)
4.risks&challenges
5.datamanagement
6.research&dataanalysis
ClusterLegendLayerValue13.79to4.1224.12to4.4534.45to4.7844.78to5.1155.11to5.44
Ra5ngLadderGraph
importance ease
privacy&transparency
privacy&transparency
risks&challenges
risks&challenges
roles&responsibili5es(ofallstakeholders)
roles&responsibili5es(ofallstakeholders)
objec5vesofLA(learnerandteachersupport)
objec5vesofLA(learnerandteachersupport)
datamanagement
datamanagement
research&dataanalysis
research&dataanalysis
3.79 3.79
6.03 6.03
r=0.66
GoZone–Privacy&Transparency
1 720 31
43862
10172445
64 65
8892
91556
6074
87
34
69
96
6.08
5.44
3.12
ease
3.83 6.03 6.59
importance
r=0.45
GoZone–Privacy&Transparency
1 720 31
43862
10172445
64 65
8892
91556
6074
87
34
69
96
6.08
5.44
3.12
ease
3.83 6.03 6.59
importance
r=0.45
2.transparency,i.e.clearlyinformingstudentsofhowtheirdataiscollected,usedandprotected88.acleardescrip5onofdataprotec5onmeasurestaken10.acleardescrip5onofdatausage17.beingclearaboutthepurposeforcollec5oncertaintypesofdata24.alignedwithdataprotec5onregula5ons(ins5tu5onal,na5onal,interna5onal)34. toassurethatthecollecteddataisusedonlyforthepurposeofimprovinglearningandinstruc5on96.anagreementbetweenlearners,teachersandpolicymakersonregula5ngaproperuseofdata
GoZone–Roles&Responsibili5es
5
38
62
11
1922
33
39 48
7091
25
28
37
40
55
61
6627
47 49
6.08
4.72
3.12
ease
3.83 5.48 6.59
importance
r=0.26
GoZone–Roles&Responsibili5es
5
38
62
11
1922
33
39 48
7091
25
28
37
40
55
61
6627
47 49
6.08
4.72
3.12
ease
3.83 5.48 6.59
importance
r=0.26
55.beingclearaboutthepurposeoflearninganaly5cs61.aclearar5cula5onofrolesandresponsibili5eswhenitcomestotheuseofins5tu5onaldata39.topromotebroadadop5onoflearninganaly5csbyspecifyingsuppor5veregula5onsandcaselaw
GoZone–Objec5vesofLA
3
80
84
99
12
7898
3550
93
4244
586376
79
94
6.08
3.79
3.12
ease
3.83 5.44 6.59
importance
r=-0.12
GoZone–Objec5vesofLA
3
80
84
99
12
7898
3550
93
4244
586376
79
94
6.08
3.79
3.12
ease
3.83 5.44 6.59
importance
r=-0.12
63.touselearninganaly5csforimprovingthequalityofteaching58.toallowforproac5vebehaviorofstudentstowardstheireduca5on,withsevngpersonalgoals44.tosupportlearnerslearningtolearnandimproveskills98.toencouragemeaningfulrela5onshipsbetweenstudents
GoZone–Risks&Challenges
18
41
8326
46
97
53
59
90
95
6.08
4.16
3.12
ease
3.83 5.52 6.59
importance
r=0.52
GoZone–Risks&Challenges
18
41
8326
46
97
53
59
90
95
6.08
4.16
3.12
ease
3.83 5.52 6.59
importance
r=0.52
90.todrawagen5ontowhatthe"bigpicture"endgoalofthedatauseis59.todemarcateclearlybetweendifferentusesofanaly5csinins5tu5onalsevngs95.toensurethebenefitstostudentsoutweightherisks53.toensureadequaterepresenta5onduringhigh-stakesdecisionprocesses18.integratedwitharewardsystemfortheuseofevidence-basedimprovementstoteachingandlearning41.todiscouragestudentsandfacultyfromgamingthesystem
GoZone–DataManagement
4
14
89
23
67
71
85
6.08
4.5
3.12
ease
3.83 5.16 6.59
importance
r=-0.54
23.todefineclearrulesforcollabora5onwithotherresearchers
GoZone–Research&DataAnalysis
6
30
57
6872
75
77
81
8
32
527382
1351
1621
29
36
54
6.08
4.24
3.12
ease
3.83 5.08 6.59
importance
r=0.27
GoZone–Research&DataAnalysis
6
30
57
6872
75
77
81
8
32
527382
1351
1621
29
36
54
6.08
4.24
3.12
ease
3.83 5.08 6.59
importance
r=0.27
57.toassurethatinstruc5onalinterven5onsarebasedonwell-studiedandempiricallyvalidatedanaly5calmethodsandalgorithms75.toencouragethedevelopmentofdashboardswithmeaningfulandunderstandableoutcomes68.torecognisethelimita5onsoflearninganaly5cs72.toprovideclearexamplesofhowtointerpretdata,especiallywhentheremaybeinterac5ons54.toenhancetheconnec5onbetweentheacademicandthesocietyandindustry,showingwhatisbeingdoneattheins5tu5onsisrelatedtotheeduca5onandtraining21.tomakeexplicitmathsfrominnova5vedevelopment/useofanaly5csinindividualclassroomstoscalingup
Institutional interviews• Methodology• Topicsoftheques5ons• Analysis• Preliminaryfindings
Researchers:Yi-ShanTsai(UniversityofEdinburgh)IoanaJivet(OpenUniversityoftheNetherlands)PedroManuelMorenoMarcos(UniversidadCarlosIIIdeMadrid)KairitTammets(TallinnUniversity)
Interviews – methodology • Purpose:tounderstandtheadop5onofLearningAnaly5csinhighereduca5onins5tu5onsthroughdirectconversa5onswithdecisionmakers.
• Sampling:deans,vice-deans,vice-rectors,viceprincipals,headsofIToreLearning,andseniorresearchersorprojectmanagersoflearninganaly5cs.
• Lengthofinterviews:25~60mins• Periodofdatacollec5on:August2016~January2017
Interviews – methodology • 64interviews• 51HEIs• 16countries
*Twoaddi5onalinterviewswerecarriedoutwithana5onalcollabora5veICTorganisa5oninNetherlandsandtheMinistryofEduca5onandScienceinEstonia.
Interviews – topics • LAprojects,scope,mo5va5ons,andgoals
• LAstrategy
• Progressandachievedgoals
• Challenges
• Ethicsandprivacy
Interviews – analysis
Macfadyen, L., Dawson, S., Pardo, A., Gašević, D., (2014). The learning analytics imperative and the sociotechnical challenge: Policy for complex systems. Research & Practice in Assessment, 9(Winter 2014), 17-28.
Interviews – preliminary findings
• Currentstateofadop4on
NOPLANS
INPREPARATION
IMPLEMENTED 9 7 5
12
18
THEADOPTIONOFLEARNINGANALYTICS
Ins5tu5on-wide Par5al/Pilots Dataexplora5on/cleaning N/A
Interviews – preliminary findings
• Step1–Mappoli4calcontext
LALearner
Teaching Ins5tu5onal
External
q Dataprotec5onregula5ons• 2018GDPR• Thestrictnessofexis5ngDP
regula5onsàAstopperq PressuretoadoptLAq Exis5ngsolu5onsfocuson
addressingreten5onproblems.
q Noone-size-fits-allsolu5ons
Interviews – preliminary findings
• Step2–Iden4fykeystakeholders
Managers
Students
Teachers
q 25ins5tu5onshaveestablishedformalworkinggroups.
q Notallins5tu5onshaveplannedtoprovideanaly5csdatatostudents.
• Concernsaboutdemo5va5on.
ProfessionalSupports
Interviews – preliminary findings
• Step3–Iden4fydesiredbehaviourchanges
LA
Learner
Teaching
Ins5tu5onal
q Interven5ons• E-mailalertsandpersonal
contactswithstudents(7HEIs)
• Teachingreports(1UKHEI)q Feedback• 3HEIsreceivedposi5ve
feedback• Lowresponserates
Howcanins5tu5onsengagestudentswithLA?
Nodefinedstrategy
Interviews – preliminary findings
• Step4–Developengagementstrategy
LA
Digitalisa5onstrategiesTeaching&learningstrategies
q Consulta4onswithprimarystakeholders
• 23HEIs• Surveys• focusgroups• Workshops• Annualstaffdevelopment
conference• Self-helptoolkitq Visualiseddashboard• 24HEIs
Dangerofbeingdatadriven
Interviews – preliminary findings
• Step5–Analyseinternalcapacitytoeffectchange
Managers
Students
Teachers ProfessionalSupports
Ins5tu5onalcontext
q Technologicalresources
q Humanresources
q Fundingq Ins5tu5onal
culture• Awareness• Buy-in• Common
understanding• Analy5csskills
Ø ClearvalueandrelevanceØ Ins5tu5onalpriori5esØ Workload
Ø TeachingstylesØ MonitoringØ Confidenceinhandlingnewtechnology&analy5csdata
Ø PassiveengagementwithstudiesØ Lackofinterestincertainsubjects
Ethics&Privacy
TrustinLA
Interviews – preliminary findings
• Step6–Establishmonitoringandlearningframeworks
• Fewwereabletotalkaboutplansforevalua5on.• Itisdifficulttoisolateanddefinethesuccessofalearninganaly5csprojectthatisimplementedalongsideotherprojectswiththesamegoalstoenhancelearningandteaching.
• 12ins5tu5onshavedevelopedorplannedtodevelopapolicytoaccommodatetheuseoflearninganaly5cs.
Conclusion• Successtodate:• Scaleduptheins5tu5onalcapacity(step5)• Begerunderstandingofchallenges• Gainedexperience
Siemens,G.,Dawson,S.,&Lynch,G.(2014).ImprovingtheQualityandProduc5vityoftheHigherEduca5onSector-PolicyandStrategyforSystems-LevelDeploymentofLearningAnaly5cs.Canberra,Australia:OfficeofLearningandTeaching,AustralianGovernment.
Institutional survey• Topicsinves4gated
ThedevelopmentofLA
• Currentadop5on• Ins5tu5onalinfrastructureandcapacity
• Strategyandpolicy• Legalandethicalconsidera5ons
• Evalua5on
Self-evalua5onofLAmaturityandIns5tu5onalreadiness
• LAmaturity• LAsuccess• Culture• Dataandresearchcapabili5es
• Legalandethicalconsidera5ons
• Trainingandcommunica5on
Institutional survey• Distribu4on:sixconsor5umsinEurope–EADTU(EuropeanAssocia5onofDistanceTeachingUniversi5es),EUA(EuropeanUniversityAssocia5on),HeLF(Headsofe-LearningForumintheUK),EUNIS(EuropeanUniversityInforma5onSystems),SNOLA(SpanishNetworkofLearningAnaly5cs)andtheeMadridNetwork.
• Survey4me:September2016~February2017• Returns:46ins5tu5ons(responserate:15%)EuropeanHEIsingeneralwerenewtoLAandhencedidnotfeelequippedwithsufficientexperienceorknowledgetoanswerthesurvey??
Institutional survey• 22countries
NOPLANS
INPREPARATION
IMPLEMENTED 2 13
15
16
Theadop4onofLAIns5tu5on-wide Smallscale N/A
Institutional survey• Mo4va4onsforadop4ngLAAmongthe11op5onsformo5va5onsspecifictolearningandteaching,thetop5driversare:§ Toimprovestudentlearningperformance(16%)§ Toimprovestudentsa5sfac5on(13%)§ Toimproveteachingexcellence(13%)§ Toimprovestudentreten5on(11%)§ Toexplorewhatlearninganaly5cscandoforourins5tu5on/staff/students(10%)
Institutional survey• Strategyandevalua5on• Amongins5tu5onsthathaveimplementedLA:§ Thereisastrategy:20%§ Intheprocessofdevelopingastrategy:20%§ Noclearstrategy:46.7%§ Havedevelopedsuccesscriteria:27%
Conclusion• ~HalfoftheEuropeancountriesinves5gatedhaveHEIsthatareeitherobservingthedevelopmentoflearninganaly5csorhaveengagedwithitprac5cally.
• Theearlyadoptersarelikelytoscaleupthecultureforlearninganaly5csinEurope.
• SHEILAwillassistwiththedevelopmentoflearninganaly5cspoliciesinfourins5tu5onsdrawingupontheoutputoftheproject.
• SHEILAwillpushthedevelopmentofalongtermlearninganaly5cspolicyagendaandcommunityamonghighereduca5onins5tu5ons.
What next?
• BecomeanassociatepartneroftheSHEILAproject?
• Visit:hgp://sheilaproject.eu/ Yi-ShanTsaiResearchassociate
UniversityofEdinburgh@yi_shan_tsai
yi-shan.tsai@ed.ac.uk
What do students want? Towards an instrument for students’ evalua6on of quality of learning analy6cs services
AlexanderWhitelock-Wainwright,DraganGašević,&
RicardoTejeiro
A.Wainwright@Liverpool.ac.uk
Aims
• Highlighttheimportanceofservicequalityinlearninganaly5cs.
• Developaninstrumenttoexplorestudentexpecta5onstowardslearninganaly5cs.
• Stepstowardsamodeloflearninganaly5csuse.
What is service quality?
• Subjec5veassessmentofthedegreetowhichaserviceuser’sneedsorexpecta5onsweremet(Parasuraman,Zeithaml,&Malhotra,2005).
Expecta5ons
ServiceUsage/Exposure
Percep5ons
Avtude
What is service quality? (Contd.)
• Encouragesuserstouseownserviceovercompe5tors(Parasuraman,Zeithaml,&Berry,1988).
• Servicequalityinhighereduca5on(Spooren,Brockx,&Mortelmans,2013).
• Ideologicalgap(Ng&Forbes,2009).
Service Quality in Learning Analy6cs • Learninganaly5csservicesdesignedtosupportlearning.
• Variousstakeholdergroupswithinlearninganaly5cs(e.g.,students,teachers,managers)(Clow,2012).
• Qualityindicatorsoflearninganaly5cstools(Scheffel,Drachsler,Stoyanov,&Specht,2014).
Ques6onnaire Development
• Measuringstudentexpecta5onsoflearninganaly5cs.
• Developmentofins5tu5onalpolicies(SHEILA).
• Theoriesofhumanbehaviour.
Ques6onnaire Development (Contd.) • Iden5fyingthemeswithinpastliterature(Ifenthaler&Schumacher,2016;Sclater,2016;West,Heath,&Huijser,2016):
• EthicsandPrivacy
• Meaningfulness
• Agency
• Interven5ons
Ques6onnaire Development (Contd.)
Created79Items PeerReview PilotStudy
with37Items
ExploratoryFactorAnalysis
Redistributethe
Ques5onnaire
Pilot Study Results
• Instrumentreducedto19items.
• Twofactorsolu5onforbothscales:
• Serviceexpecta5ons:o Desiresscale–0.88alpha.
o Predic5vescale–0.88alpha.
• Ethicalexpecta5ons:o Desiresscale–0.82alpha.
o Predic5vescale–0.86alpha.
Future Direc6ons
• Developthecorrespondingpercep5onsscale.
• Modelinten5onstowardsusinglearninganaly5cs.
Avtudes
SocialNorms
Inten5onstoUseLearningAnaly5cs
Perceived
Behavioural
Control
Workshop scheduleTime Schedule
13.00-13.05 Welcome&introduc5on
13.05-14.30 Fivepresenta5ons&discussions1. Overview&LAinAustralia:ProfDraganGasevic(University
ofEdinburgh)2. LAinEurope(SHEILAProject):ProfPedroJ.Muñoz-Merino
(UniversidadCarlosIIIdeMadrid)3. SHEILA–groupconceptmapping:MarenScheffel(Open
UniversityoftheNetherlands)4. SHEILA–ins5tu5onalinterviews&survey:DrYi-ShanTsai
(UniversityofEdinburgh)5. SHEILA–studentsurvey:AlexanderWhitelock-Wainwright
(UniversityofLiverpool)
14.30-15.00 A^ernoonTea
15.00-16.00 Groupworkontheassessmentofins5tu5onalreadiness
16.00-17.00 LAPolicydra^ing
Assessment of institutional readiness
• UsetheROMAframeworktomapyourprogresstowardsdeployinglearninganaly5cs.
Macfadyen, L., Dawson, S., Pardo, A., Gašević, D., (2014). The learning analytics imperative and the sociotechnical challenge: Policy for complex systems. Research & Practice in Assessment, 9 (Winter 2014), 17-28.
1) Mappoli4calcontext2) Iden4fykeystakeholders3) Iden4fydesiredbehaviour
changes4) Developengagement
strategy5) Analyseinternalcapacityto
effectchanges6) Establishmonitoringand
learningframeworks
• DiscussinyourgroupwhatLAprojectsanddevelopmentshavetaken/aretaking/willtakeplaceinyourorganisa5ons
LA projects in your organisations ~5mins
Keepyourgroupnoteshere:Group1:hgps://goo.gl/CcGlTkGroup2:hgps://goo.gl/zrMsyiGroup3:hgps://goo.gl/3WNr62Group4:hgps://goo.gl/WzLmQc
• Istheregovernment/teacher/student/administratorpressuretodeployLAinyourorganisa5on?Orpressurenottodeployit?
1) Map political context ~5mins
Keepyourgroupnoteshere:Group1:hgps://goo.gl/CcGlTkGroup2:hgps://goo.gl/zrMsyiGroup3:hgps://goo.gl/3WNr62Group4:hgps://goo.gl/WzLmQc
• Who/whichpeopleinsideandoutsideyourorganisa5onwouldbe/arethekeystakeholdersfortheuseofLA?
• WhowillbenefitfromtheuseofLA?
2) Identify key stakeholders ~5mins
Keepyourgroupnoteshere:Group1:hgps://goo.gl/CcGlTkGroup2:hgps://goo.gl/zrMsyiGroup3:hgps://goo.gl/3WNr62Group4:hgps://goo.gl/WzLmQc
• Whatareyourreasonsforadop5ngLA?• Whatbenefitsisitexpectedtobring/havebrought?
3) Identify desired behaviour changes ~5mins
Keepyourgroupnoteshere:Group1:hgps://goo.gl/CcGlTkGroup2:hgps://goo.gl/zrMsyiGroup3:hgps://goo.gl/3WNr62Group4:hgps://goo.gl/WzLmQc
• Howwellprepareddoyouthinkyourorganisa5onistodeployLA?
• Whatneedstobedonetoachievethedesiredchangesiden5fiedpreviously?Areastoconsiderareethical/legal,technical,pedagogical,andcultural.
4) Develop engagement strategy ~5mins
Keepyourgroupnoteshere:Group1:hgps://goo.gl/CcGlTkGroup2:hgps://goo.gl/zrMsyiGroup3:hgps://goo.gl/3WNr62Group4:hgps://goo.gl/WzLmQc
• Doesyourorganisa5onhasthecapacitytoimplementtheplannedstrategy?Doyouneedtoacquireanyskill-sets,funding,ortechnologicalsystems?
5) Analyse internal capacity to effect changes ~5mins
Keepyourgroupnoteshere:Group1:hgps://goo.gl/CcGlTkGroup2:hgps://goo.gl/zrMsyiGroup3:hgps://goo.gl/3WNr62Group4:hgps://goo.gl/WzLmQc
• Howwillyoutrackandevaluatetheprogress?Howwillthisprocessinformthenextstepofimplementa5on?Willtherebeanysuccesscriteria?
6) Establish monitoring and learning frameworks ~5mins
Keepyourgroupnoteshere:Group1:hgps://goo.gl/CcGlTkGroup2:hgps://goo.gl/zrMsyiGroup3:hgps://goo.gl/3WNr62Group4:hgps://goo.gl/WzLmQc
• Reviewthe6ROMAsteps.Canyouiden5fyanychallengesthatmaycomeupinanysteps?(Putyournotesinthesameboxes.)
Challenges ~5mins
Keepyourgroupnoteshere:Group1:hgps://goo.gl/CcGlTkGroup2:hgps://goo.gl/zrMsyiGroup3:hgps://goo.gl/3WNr62Group4:hgps://goo.gl/WzLmQc
• Chooseonecasetoworkonasagrouporworkindividuallyandshareideasinyourgroup.
Draft a LA policy
1. Audience2. Purpose3. Process§ Datacollec5on§ Datamanagement§ Stakeholderengagement§ Evalua5on4.Policyevalua5on
Keepyourgroupnoteshere:Group1:hgps://goo.gl/CcGlTkGroup2:hgps://goo.gl/zrMsyiGroup3:hgps://goo.gl/3WNr62Group4:hgps://goo.gl/WzLmQc
• 1.Audience• Whoisthispolicyfor?Considerwhoseworkingac5vi5esthepolicywillshape.
• 2.Purpose• Whatareyourpolicyobjec5ves?Consideryourobjec5vesforLAandthechangesyouseektoachieve.ThepolicywillsetoutasthereasonforintroducingLA.
Audience & Purpose ~5mins
Keepyourgroupnoteshere:Group1:hgps://goo.gl/CcGlTkGroup2:hgps://goo.gl/zrMsyiGroup3:hgps://goo.gl/3WNr62Group4:hgps://goo.gl/WzLmQc
• 3.Process• a)Datacollec5on• b)Datamanagement• c)Stakeholderengagement• d)Evalua5on
Process ~15mins
Keepyourgroupnoteshere:Group1:hgps://goo.gl/CcGlTkGroup2:hgps://goo.gl/zrMsyiGroup3:hgps://goo.gl/3WNr62Group4:hgps://goo.gl/WzLmQc
• 4.Policymanagement
Policy management ~10mins
Finalcheck-ups• WillthepolicyneedtostateanythingthatshouldNOTbedoneinrela5onto
LA?• Arethereanylimita5onsorpoten5alchallengesandrisksaboutLAthat
shouldbeaddressedinthepolicy?• Canyounarrowthepolicydowntoashortlistofprinciples?
Keepyourgroupnoteshere:Group1:hgps://goo.gl/CcGlTkGroup2:hgps://goo.gl/zrMsyiGroup3:hgps://goo.gl/3WNr62Group4:hgps://goo.gl/WzLmQc
Recommended