Upload
dragan-gasevic
View
6.204
Download
0
Tags:
Embed Size (px)
DESCRIPTION
The
Citation preview
Learning Analytics is about Learning
Dragan Gasevic@dgasevic
Growing demand for education!
Scalability is possible
Low effect size of class-sizeJohn Hattie
Delivery
Delivery
http://www.scientificamerican.com/article.cfm?id=massive-open-online-courses-transform-higher-education-and-science
Scientific American, March 13, 2013
MOTIVATION
Feedback loops between students and instructors
are missing!
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of educational research, 77(1), 81-112.
Learning and Collaborating
Educators
Learners
Registrations
Learning and Collaborating Networks
Videos/slides
Registrations
Mobile
Search
Educators
Learners
Networks
Learning and Collaborating Networks
Registrations
Mobile
Search
Networks
Educators
Learners
Videos/slides
DANGER
Predict-o-mania
The same predictive models for everything and everyone
Student diversity
http://www.census.gov/prod/2013pubs/acsbr11-14.pdf
Population Diversity
Female
s
Internati
onal stu
dents
Other lan
guag
e at h
ome
Living i
n non-urban
Part time s
tudent
Previously
enro
lled to
a co
urse
Early
acces
s
Did not acce
ss
Late a
ccess
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
ACCT 1 (n = 746)BIOL 1 (n = 220)BIOL 2 (n = 657)COMM 1 (n = 499)COMP 1 (n = 242)ECON 1 (n = 661)GRAP 1 (n = 192)MARK 1 (n = 723)MATH 1 (n = 194)
LMS Functionality Diversity
ACCT 1 BIOL 1 BIOL 2 COMM 1 COMP 1 ECON 1 GRAP 1 MARK 1 MATH 1
Light Box Gallery XForum X X X X X X X X XCourse X X X X X X X X XResource X X X X X X X X XTurn-it-in X X X X X XAssignment X X X X X X XBook X X XQuiz X X X XFeedback XMap XVirtual Classroom XLesson XGlossary XChat X
Predictive Power Diversity
All courses together
ACCT 1 BIOL 1 BIOL 2 COMM 1 COMP 1 ECON 1 * GRAP 1 MARK 1 MATH 10.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
Model 1MoodleModel 1 + Moodle
Model 1 – demographic and socio-economic variables* - not statistically significant
Retention is not the only challenge
It is important, of course!
But, where is learning?
How do we enhance learning if the focus is on outcomes only?
DIRECTION
Learning Analytics – What?
Measurement, collection, analysis, and reporting of data about
learners and their contexts
Learning Analytics – Why?
Understanding and optimising learning and the environments
in which learning occurs
Human agency is central to learning
Bandura, A. (1989). Human agency in social cognitive theory. American psychologist, 44(9), 1175-1184.
Modern Educational Psychology
Winne and Hadwin's model
of self-regulated learning
Knowledge society and knowledge economy
Why does it matter?!
ChallengeMetacognitive skills
Bjork, R. A., Dunlosky, J., & Kornell, N. (2013). Self-Regulated Learning: Beliefs, Techniques, and Illusions. Annual Review of Psychology, 64, 417-444. doi:10.1146/annurev-psych-113011-143823
Why does it matter?!
ChallengeInformation seeking skills
Judd, T., & Kennedy, G. (2011). Expediency-based practice? Medical students’ reliance on Google and Wikipedia for biomedical inquiries. British Journal of Educational Technology, 42 (2), 351-360. doi:10.1111/j.1467-8535.2009.01019.x
Why does it matter?!
ChallengeSensemaking paradox
Butcher, K. R., & Sumner, R. (2011). Self-Directed Learning and the Sensemaking Paradox. Human–Computer Interaction, 26(1-2), 123-159. doi:10.1080/07370024.2011.556552
Why does it matter?!
ChallengeAsking questions and critical thinking
Graesser, A. C., & Olde, B. (2003). How does one know whether a person understands a device? The quality of the questions the person asks when the device breaks down. Journal of Educational Psychology, 95(3), 524–536..
Process and context focus for learning analytics needed
to understand learning
OPPORTUNITIES
Learning Analytics
Effects of learning context
External conditions (e.g., instructional design)
Cognitive presence
the extent to which the participants in any particular configuration of a CoI are able to construct meaning via sustained communication
Garrison, D. R., Anderson, T., & Archer, W. (2001). Critical Thinking and Computer Conferencing: A Model and Tool to Assess Cognitive Presence. American Journal of Distance Education ,15(1), 7-23.
Effect size of the moderator role on critical thinking
Cohen’s d = 0.66
Effect size of an intervention on critical thinking in online discussions
d = 0.95 (non-moderators) and
d = 0.61 (moderators)
Cognitive Presence in Online Discussions – Association w/ Grades
Cognitive presence TMA1 TMA2 TMA3 TMA4 Final
Control group
Triggering event -.226 .005 -.046 -.050 -.010Exploration -.001 .141 .009 -.037 .048Integration .128 .060 .034 .043 .113Resolution .201 .027 -.023 -.054 .074Other -.028 .078 .113 .106 .154
** p < 0.01; * p < 0.05
Cognitive Presence in Online Discussions – Association w/ Grades
Cognitive presence TMA1 TMA2 TMA3 TMA4 Final
Control group
Triggering event -.226 .005 -.046 -.050 -.010Exploration -.001 .141 .009 -.037 .048Integration .128 .060 .034 .043 .113Resolution .201 .027 -.023 -.054 .074Other -.028 .078 .113 .106 .154
Intervention group
Triggering event .149 -.077 -.070 .000 .016Exploration .216 .197 .163 .223 .243
Integration .156 .396** .417** .338* .454**
Resolution -.041 .060 .154 .083 .129Other .219 .046 .050 .075 .088
** p < 0.01; * p < 0.05
Integration posts: effect on final grades
p < .001, Q1 vs. Q2; Q1 vs. Q3, Q1 vs. Q4
Q1 Q2 Q3 Q40
102030405060708090
100
Learning Analytics
Are students only driven by assessments?
Effects of external conditions
Self-reflections in video annotations
Course 1 (non-graded)
Course 2 (graded)
Course 3(graded)
Course 4 (non-graded)
Annotation total Annotation postion Q1
Annotation postion Q2
Annotation postion Q3
Annotation postion Q4
Annotation general0.00
20.00
40.00
60.00
80.00
100.00
120.00
Course 1 (non-graded)Course 2a (graded)Course 2b (graded)Course 3 (graded)Course 4 (non-graded)
Self-reflections in video annotations
Self-reflections in video annotations
Cognitive
proces
ses
Percep
tual pro
cesses
Positive
emotions
Negati
ve em
otions0
200
400
600
800
1000
1200
1400
1600
Course 1 (non-graded)Course 2a (graded)Course 2b (graded)Course 3 (graded)Course 4 (non-graded)
Tausczik, Y. R., & Pennebaker, J. W. (2010). The psychological meaning of words: LIWC and computerized text analysis methods. Journal of Language and Social Psychology, 29(1), 24-54.
Learning Analytics
Effects of students’ own decisions
Beyond external conditions
Learner profiles – use of LMS
Effect size .75 on critical thinking &academic success
34
Learner profiles – use of LMS
Effect size .75 on critical thinking and academic success
Cluster 1 Cluster 2 Cluster 3 Cluster 40
2
4
6
8
10
12
TriggeringExplorationIntegrationResolutionOther
CHALLENGES
Learning Analytics
What to measure?
We don’t need page access counts only!
Wilson, T.D. (1999). Models in information behaviour research. Journal of Documentation, 55(3), 249 - 270, doi:10.1108/EUM0000000007145
Instrumentation
About specific contexts and constructs
Instrumentation
Capturing interventionsPrevious learning and (memory of) experience
Social networks (e.g., communication, cross-class)Interaction types (e.g., transactional distances)
Motivation in Information Interaction
Zhou, M., & Winne, P. H. (2012). Modeling academic achievement by self-reported versus traced goal orientation. Learning and Instruction, 22(6), 413–419. doi:10.1016/j.learninstruc.2012.03.004
Motivation in Information Interaction
Zhou, M., & Winne, P. H. (2012). Modeling academic achievement by self-reported versus traced goal orientation. Learning and Instruction, 22(6), 413–419. doi:10.1016/j.learninstruc.2012.03.004
Achievement goal orientation (2x2)
Siadaty, M. (2013). Semantic Web-Enabled Interventions to Support Workplace Learning, PhD Thesis, Simon Fraser University, Surrey, BC, Canada.
Technology and process of self-regulated learning
Scaling up qualitative analysis
Temporal processesbeyond coding and counting
Longitudinal studies
Generating reports and nice visualization is
not enough
Building data-driven culture in institutions
Manyika, J., et al., Big Data: The Next Frontier for Innovation, Competition, and Productivity, 2011, McKinsey Global Institute, http://goo.gl/Lue3qs
Privacy and ethics
Data sharing and mobility
Thank you!