Upload
others
View
9
Download
0
Embed Size (px)
Citation preview
Successful transition from
secondary to higher
education using learning
analyticsTINNE DE LAET, TOM BROOS (KU LEUVEN)
JAN-PAUL VAN STAALDUINEN (TU DELFT)
PHILIPP LEITNER, MARTIN EBNER (TU GRAZ)
1
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
Welcome!
Who are we?
Tinne De Laet ( KU Leuven)
Head of Tutorial Services of Engineering Science
promotor STELA
background = mechanical engineer
Tom Broos (KU Leuven)
Doctoral researcher STELA
Philipp Leitner (TU Graz)
Doctoral researcher STELA
2
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
Welcome
Would you please introduce yourself to the people on your table
Who are you?
How familiar are you with learning analytics?
Is your institute applying learning analytics?
3
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
Agenda for workshop
Welcome
What is learning analytics? (10 minutes)
STELA project:
Presentation of project goal & challenges (2 minutes)
Group discussion: how could learning analytics help? (10 minutes)
STELA project:
Presentation of project ideas (10 minutes)
Group discussion on project ideas (10 minutes)
Learning analytics technology (10 minutes)
Groups present their remarks, suggestions, … (10 minutes)
Wrap up, goodbye (3 minutes)
4
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
Agenda for workshop
…. so what we expect from you
participate in discussion in groups
each group gathers 3 ideas, thoughts, remarks, suggestions, ..
present them to each other at end of the workshop
5
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
What is learning analytics?
6
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
What is Learning Analytics? no universally agreed definition
“the measurement, collection, analysis and reporting of data about
learners and their contexts, for purposes of understanding and
optimizing learning and the environments in which it occurs” [1]
[1] Learning and Academic Analytics, Siemens, G., 5 August
2011, http://www.learninganalytics.net/?p=131
[2] What is Analytics? Definition and Essential Characteristics, Vol. 1, No. 5. CETIS Analytics Series,
Cooper, A., http://publications.cetis.ac.uk/2012/521
“the process of developing actionable insights through problem
definition and the application of statistical models and analysis against
existing and/or simulated future data” [2]
7
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
What is Learning Analytics?
no universally agreed definition
[3] Learning Analytics and Educational Data Mining, Erik Duval’s Weblog, 30 January 2012,
https://erikduval.wordpress.com/2012/01/30/learning-analytics-and-educational-data-mining/
“learning analytics is about
collecting traces that learners
leave behind and using those
traces to improve learning”
[Erik Duval, 3]
† 12 March 2016
8
What is Learning Analytics? How is learning analytics different from institutional data? [4]
High-level figures:
provide an overview for internal and external reports;
used for organisational planning purposes.
Academic analytics:
figures on retention and success, used by the institution to assess performance.
Educational data mining:
searching for patterns in the data.
Learning analytics:
use of data, which may include ‘big data’,
to provide actionable intelligence for learners and teachers.
[4] Learning analytics FAQs, Rebecca Ferguson, Slideshare, http://www.slideshare.net/R3beccaF/learning-
analytics-fa-qs
9
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
What is Learning Analytics?
Different level of analytics
adapted from http://www.slideshare.net/gsiemens/learning-analytics-educause
level beneficiaries
course-level learners, teachers, faculties
aggregate learners, teachers, tutors, counsellors, faculties
institutional administrators, funders, marketing
regional administrators, funders, policy makers
national and
international
national and international governments, policy
makers
10
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
What is Learning Analytics?
data visualization versus predictive analytics
is showing data enough?
how to show data to create sense-making and impact?
is predicting study success/drop out the only thing that matters?
can both be combined?
11
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
What is Learning Analytics?
learning analytics process model
[Verbert et al. 2013] Verbert K, Duval E, Klerkx J; Govaerts S, Santos JL (2013) Learning analytics dashboard
applications. American Behavioural Scientist, 10 pages. Published online February 201, doi:
10.1177/0002764213479363
12
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
What is Learning Analytics?
how to evaluate learning analytics?
is perceived usefulness enough?
is increased self-awareness enough? How will you measure this?
is increased sense-making enough? How will you measure this?
how could impact be measured?
13
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
What is Learning Analytics?
six critical dimensions of learning analytics
[Greller, W., & Drachsler, H. (2012). Translating Learning into Numbers: A Generic Framework for Learning
Analytics. Educational Technology & Society, 15 (3), 42–57. http://ifets.info/journals/15_3/4.pdf ]
14
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
What is Learning Analytics?
six critical dimensions of learning analytics
[Greller, W., & Drachsler, H. (2012). Translating Learning into Numbers: A Generic Framework for Learning
Analytics. Educational Technology & Society, 15 (3), 42–57. http://ifets.info/journals/15_3/4.pdf ]
Tutors
Policy makers
15
What is Learning Analytics?
data subjects
here: students
(could also be teachers)
data clients
students
tutors
academic administrators
policy makers
[Greller, W., & Drachsler, H. (2012). Translating Learning into Numbers: A Generic Framework for Learning
Analytics. Educational Technology & Society, 15 (3), 42–57. http://ifets.info/journals/15_3/4.pdf ]
16
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
What is Learning Analytics?
what are the objectives?
is it about reflection or prediction?
is showing data enough?
how to show data to create self-awareness, sense-making
and impact?
[Greller, W., & Drachsler, H. (2012). Translating Learning into Numbers: A Generic Framework for Learning
Analytics. Educational Technology & Society, 15 (3), 42–57. http://ifets.info/journals/15_3/4.pdf ]
17
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
What is Learning Analytics?
which data is available?
is it open? protected?
is it ethical to use the data?
[Greller, W., & Drachsler, H. (2012). Translating Learning into Numbers: A Generic Framework for Learning
Analytics. Educational Technology & Society, 15 (3), 42–57. http://ifets.info/journals/15_3/4.pdf ]
18
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
What is Learning Analytics?
Technology, algorithms, theories are at the basis of
learning analytics
open source or as close as possible to (proprietary)
university systems?
pedagogic theories for supporting students
[Greller, W., & Drachsler, H. (2012). Translating Learning into Numbers: A Generic Framework for Learning
Analytics. Educational Technology & Society, 15 (3), 42–57. http://ifets.info/journals/15_3/4.pdf ]
19
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
What is Learning Analytics?
[Greller, W., & Drachsler, H. (2012). Translating Learning into Numbers: A Generic Framework for Learning
Analytics. Educational Technology & Society, 15 (3), 42–57. http://ifets.info/journals/15_3/4.pdf ]
• conventions: ethics, personal privacy, and similar socially motivated limitations
• norms: restricted by laws or specific mandated policies or standard
ethics and privacy IS a big issue
20
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
What is Learning Analytics?
[Greller, W., & Drachsler, H. (2012). Translating Learning into Numbers: A Generic Framework for Learning
Analytics. Educational Technology & Society, 15 (3), 42–57. http://ifets.info/journals/15_3/4.pdf ]
• competences: application of learning analytics requires new higher-order competences to enable fruitful exploitation in learning and teaching
• acceptance: acceptance factors can further influence the application or decision making that follows an analytics process
21
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
Agenda for workshop
Welcome
What is learning analytics? (10 minutes)
STELA project:
Presentation of project goal & challenges (2 minutes)
Group discussion: how could learning analytics help? (10 minutes):
Presentation of project ideas (10 minutes)
Group discussion on project ideas (10 minutes)
Learning analytics technology (10 minutes)
Groups present their remarks, suggestions, … (10 minutes)
Wrap up, goodbye (3 minutes)
22
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
STELA project
Project goal & challenges
23
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
STELA project
focus: transition from secondary to higher education
(summer before entering university + 1st year experience)
goal: applying learning analytics to support transition
student-centered
additional focus to tutors, student counsellors
1. beyond identifying at-risk students
inclusive approach: feedback for every student
2. beyond single course
entire program
3. provide actionable feedback
ability to remediate based on feedback
24
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
STELA project
Group discussion: how could
learning analytics help?
(10 minutes)
25
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
Group discussion: how could learning
analytics help? (10 minutes)
What are challenges in the transition from secondary to higher
education?
Which actionable feedback could support students in the
transition?
How could learning analytics help in this matter?
26
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
Agenda for workshop
Welcome
What is learning analytics? (10 minutes)
STELA project:
Presentation of project goal & challenges (2 minutes)
Group discussion: how could learning analytics help? (10 minutes):
Presentation of project ideas (10 minutes)
Group discussion on project ideas (10 minutes)
Learning analytics technology (10 minutes)
Groups present their remarks, suggestions, … (10 minutes)
Wrap up, goodbye (3 minutes)
27
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
STELA project: project ideas
four areas of focus
1. academic performance
2. academic engagement
3. academic skills
4. academic well-being
How?
position students with respect to peers
show impact on future study pathway→ show how students with similar profile did in the past
28
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
STELA project: project ideas(1/4) academic performance
29
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
data (often) easily available
present at all universities
after the facts
strong relation performanceT-1 ~ performance
Source: ABLE project (2015-1-UK01-KA203-013767)
STELA project: project ideas(2/4) academic engagement
30
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
depends on context
• e.g. card swipes at Nottingham Trent University,
• learning platform activity at KU Leuven.
relation engagement ~ performance not always
straightforward.
data not always available
STELA project: project ideas(3/4) academic skills
31
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
Number of peers:
Motivation Score
Very high
Perf
orm
an
ce
An
xie
ty
High
Average
Low
Very Low
Very weak Weak Average High Very High
Recommendations
First aid for stress workshop at Student Health Center
Talk to a student counselor.
often self-reported
tests and questionnaires available form pedagogy
and psychology
• Time management, motivation, persistence, …
opportunity to move from information to
recommendation
STELA project: project ideas(4/4) academic well-being
32
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
How do you feel about this course now?
Last week, others felt like this:
opportunity to complement with continuous, bi-
directional feedback
tests and questionnaires available form pedagogy
and psychology
Agenda for workshop
Welcome
What is learning analytics? (10 minutes)
STELA project:
Presentation of project goal & challenges (2 minutes)
Group discussion: how could learning analytics help? (10 minutes):
Presentation of project ideas (10 minutes)
Group discussion on project ideas (10 minutes)
Learning analytics technology (10 minutes)
Groups present their remarks, suggestions, … (10 minutes)
Wrap up, goodbye (3 minutes)
33
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
Learning analytics
technology
34
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
Learning Analytics technology
Challenges
multiple and different kind of data-sources
lots of data
need to be combined and analyzed
scalable
real-time?
visualized
...
35
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
Learning Analytics technology 36
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
Concept
Learning Analytics technology(1/3) Data Collection: Logstash¹
37
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
flexbile and open source (Apache 2 Open Source License)
data collection, enrichtment and transportation pipeline
efficiently process log, event and unstructured data sources
¹) https://www.elastic.co/products/logstash
Learning Analytics technology(2/3) Search and Analytics Engine: Elasticsearch²
38
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
distributed, open source search and analytics engine
scalability, reliability, and easy management
developer-friendly query language covering structured,
unstructured, and time-series data
²) https://www.elastic.co/de/products/elasticsearch
Learning Analytics technology(3/3) Visualization: Kibana³ or Grafana4
39
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
3) https://www.elastic.co/products/kibana4) http://grafana.org/
open source data visualization platform
interact with data through powerful graphics
easy extensibility and a variety of panels
Agenda for workshop
Welcome
What is learning analytics? (10 minutes)
STELA project:
Presentation of project goal & challenges (2 minutes)
Group discussion: how could learning analytics help? (10 minutes):
Presentation of project ideas (10 minutes)
Group discussion on project ideas (10 minutes)
Learning analytics technology (10 minutes)
Groups present their remarks, suggestions, … (10 minutes)
Wrap up, goodbye (3 minutes)
40
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
Groups present their
remarks, suggestions, …
41
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD
Wrap up, goodbye
42
STELA - Erasmus+ Project
562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD