Tools and Methods to Enhance Information Seeking,
Sensemaking, and Learning
Dragan Gašević@dgasevic
https://semtech.athabascau.ca
Exciting times
MOOC-mania
http://www.scientificamerican.com/article.cfm?id=massive-open-online-courses-transform-higher-education-and-science
Scientific American, March 13, 2013
Aren’t they great?!
They offer good resources, but…
CHALLENGES
What skills to promote?
How about – critical thinking, creativity, research-intensive learning,
information seeking, sensemaking,self-directed and self-regulated learning,
…
“We teach what we can measure. If we don't measure what we care about,
it will never be taught.”
Paulo Blikinsein, LAK 2013
Three Generation of Distance Education Pedagogies
Anderson, T. & Dron, J. (2011). Three Generations of Distance Education Pedagogy, International Review of Research in Open and Distance Learning 12(3), 80-97, http://goo.gl/j3mRF
Why does it matter?!
ChallengeInformation seeking skills
Judd, T., & Kennedy, G. (2011). Expediency-based practice? Medical students’ reliance on Google and Wikipedia for biomedical inquiries. British Journal of Educational Technology, 42 (2), 351-360. doi:10.1111/j.1467-8535.2009.01019.x
Why does it matter?!
ChallengeSensemaking paradox
Butcher, K. R., & Sumner, R. (2011). Self-Directed Learning and the Sensemaking Paradox. Human–Computer Interaction, 26(1-2), 123-159. doi:10.1080/07370024.2011.556552
Why does it matter?!
ChallengeMetacognitive skills
Bjork, R. A., Dunlosky, J., & Kornell, N. (2013). Self-Regulated Learning: Beliefs, Techniques, and Illusions. Annual Review of Psychology, 64, 417-444. doi:10.1146/annurev-psych-113011-143823
How can we help?
Information seeking, sensemaking, & learning
Complex processes w/ dynamic feedback loops
APPROACH
Evidence-based Discipline
Integration of best research evidence with practitioner expertise and stakeholder values
The goal made up based on
Evidence-based discipline
Information seeking,sensemaking,
learning
Practice
Real world
Research
Evidence-based discipline
Information seeking,sensemaking,
learning
Practice
Real world
Research
Self-reports, laboratory, intrusive,
causality,…
Evidence-based discipline
Information seeking,sensemaking,
learning
Practice
Real world
Research
Self-reports, laboratory, intrusive,
causality,…
Users “on average”, GIGO, ..
GIGO – Garbage In Garbage Out
Intervene/instrument
Evidence-based discipline
Information seeking,sensemaking,
learning
Practice
Real world!
Research
Self-reports, laboratory, intrusive,
causality,…
Users “on average”,GIGO, …
GIGO – Garbage In Garbage Out
Intervene/instrument
Collect/Analyze
Learning Analytics – What?
Measurement, collection, analysis, and reporting of data about
learners and their contexts
Learning Analytics – Why?
Understanding and optimising learning and the environments
in which learning occurs
TOOLS
Data Collection
Importance of context
Tool and format independent Aggregates and integrates
Learning Context Ontology: LOCO
Jovanovic, J., Knight, C., Gasevic, D., Richards, G. (2007). Ontologies for Effective Use of Context in e-Learning Settings. Educational Technology & Society, 10(3), 47-59.
http://intelleo.eu
LOCO-Analyst
OAST and LOCO-Analyst
Ali, L., Hatala, M. Gašević, D., Jovanović, J., (2012). A Qualitative Evaluation of Evolution of a Learning Analytics Tool," Computers & Education, 58(1), 470-489
Learning Environment
LOCO-Analyst
Learning Analytics
Visual analytics requested (77.8%)
Ali, L., Hatala, M. Gašević, D., Jovanović, J. (2012). A Qualitative Evaluation of Evolution of a Learning Analytics Tool. Computers & Education, 58(1) 470-489.
LOCO-Analyst
LOCO-Analyst
LOCO-Analyst
LOCO-Analyst
Student comprehension
Student comprehension
Gašević, D., et al. (2011). An Approach to Folksonomy-based Ontology Maintenance for Learning Environments. IEEE Transactions on Learning Technologies, 4(4), pp. 301-314.
METHODS
Learning Analytics Acceptance Model
Ali, L., Asadi, M., Gašević, D., Jovanović, J., Hatala, M. (2013). Factors Influencing Beliefs for Adoption of a Learning Analytics Tool: An Empirical Study," Computers & Education, 62, 130–148.
Learning Analytics
What to measure?
We don’t need page access counts!
Wilson, T.D. (1999). Models in information behaviour research. Journal of Documentation, 55(3), 249 - 270, doi:10.1108/EUM0000000007145
Cognitive Presence in Online Discussions – Moderator Role
Cognitive presence Non-moderator Moderator t(df)
Control
group
Triggering event 4.29 (4.51) 1.05 (0.23) 4.18(37)*
Exploration 4.92 (4.12) 5.26 (3.43) -0.46(37)
Integration 1.42 (2.07) 2.68 (1.76) -3.05(37)**
Resolution 0.24 (0.49) 0.74 (1.01) -3.15(37)**
Other 0.89 (1.45) 0.89 (1.03) 0.00(37)
Interventi
on grou
p
Triggering event 1.43 (1.98) 1.02 (0.15) 1.35(43)
Exploration 3.64 (2.47) 2.82 (2.09) 1.83(43)
Integration 3.86 (3.00) 3.93 (2.32) -0.15(43)
Resolution 0.43 (0.85) 1.07 (1.50) -2.74(43)***
Other 0.86 (1.36) 0.61 (0.90) 1.21(43)
* p < 0.001; ** p < 0.005; *** p < 0.01
Cognitive Presence in Online Discussions – Association w/ Grades
Cognitive presence TMA1 TMA2 TMA3 TMA4 Final
Control group
Triggering event -.226 .005 -.046 -.050 -.010Exploration -.001 .141 .009 -.037 .048Integration .128 .060 .034 .043 .113Resolution .201 .027 -.023 -.054 .074Other -.028 .078 .113 .106 .154
Intervention group
Triggering event .149 -.077 -.070 .000 .016Exploration .216 .197 .163 .223 .243
Integration .156 .396** .417** .338* .454**
Resolution -.041 .060 .154 .083 .129Other .219 .046 .050 .075 .088
** p < 0.01; * p < 0.05
Analytics for Social Learning Environments
Automated content analysis and role miningOpen learner modeling
Recommendations
Self-regulated (Workplace) Learning
Trace-based Measurement Protocol
Siadaty, M. (2013). Semantic Web-Enabled Interventions to Support Workplace Learning, PhD Thesis, Simon Fraser University.
Siadaty, M. (2013). Semantic Web-Enabled Interventions to Support Workplace Learning, PhD Thesis, Simon Fraser University.
Siadaty, M. (2013). Semantic Web-Enabled Interventions to Support Workplace Learning, PhD Thesis, Simon Fraser University.
Siadaty, M. (2013). Semantic Web-Enabled Interventions to Support Workplace Learning, PhD Thesis, Simon Fraser University.
Siadaty, M. (2013). Semantic Web-Enabled Interventions to Support Workplace Learning, PhD Thesis, Simon Fraser University.
Siadaty, M. (2013). Semantic Web-Enabled Interventions to Support Workplace Learning, PhD Thesis, Simon Fraser University.
Self-regulated (Workplace) Learning
Self-regulated (Workplace) Learning
http://learningworksforkids.com/EF/metacognition.html
Information Interaction
Zhou, M., & Winne, P. H. (2012). Modeling academic achievement by self-reported versus traced goal orientation. Learning and Instruction, 22(6), 413–419. doi:10.1016/j.learninstruc.2012.03.004
Information Interaction
Zhou, M., & Winne, P. H. (2012). Modeling academic achievement by self-reported versus traced goal orientation. Learning and Instruction, 22(6), 413–419. doi:10.1016/j.learninstruc.2012.03.004
Achievement goal orientation (2x2)
Concept and relation filtering
Zouaq, A., Gašević, D., Hatala, M., "Towards Open Ontology Learning and Filtering," Information Systems, Vol. 36, No. 7, 2011, pp. 1064-1081.
Concept and relation filtering
Zouaq, A., Gašević, D., Hatala, M., "Towards Open Ontology Learning and Filtering," Information Systems, Vol. 36, No. 7, 2011, pp. 1064-1081.
FORWARD-LOOKING
Analyzing dynamic processes
http://www.processmining.org
Enough text obsession!
Forward-looking
George Siemens, LAK 2013
CLAS – Collaborative Lecture Annotation System
Multimodal analytics
Forward-looking
http://www.solaresearch.org
SUMMARY
Thank you!