Upload
rebecca-ferguson
View
7.564
Download
2
Embed Size (px)
Citation preview
Learning design and learning analytics: building the links
EMMA Summer School, Ischia, July 2015
Rebecca Ferguson
• The Open University: largest in UK• Informal learning: iTunes, YouTube…• MOOCs on FutureLearn, OpenLearn
and elsewhere• Making use of big data for
more than 40 years• Learning analytics research / events• LACE project
2
Lead on MOOC evaluation at The Open University, UK
http://www.laceproject.eu/
Workshop overview
09.45 IntroductionLinking learning analytics, learning design and MOOCs
10.05 Group workFocusing on learning outcomes in MOOCs
10.25 Plenary
Discussing how these data might be used to support learning
10.40 Group work
Analytics, step by step
11.00 Plenary
What would MOOC learners and educators need to know to use these analytics?
11.15 Workshop end
3
You can view and download these slides athttp://www.slideshare.net/R3beccaF
What are learning analytics?
High-level figures
Brief overviews for internal and external reports
Academic analytics
Figures on retention and success, for the institution to assess performance
Learning analytics
Use of big data to provide actionable intelligence for learners and educators
4
Educators use analytics to
• Monitor the learning process• Explore student data• Identify problems• Discover patterns• Find early indicators for success• Find early indicators for poor marks or drop-out• Assess usefulness of learning materials• Increase awareness, reflect and self reflect• Increase understanding of learning environments• Intervene, supervise, advise and assist• Improve teaching, resources and the environment
5
Dyckhoff, A L, Lukarov, V, Muslim, A, Chatti, M A, & Schroeder, U. (2013). Supporting Action Research with Learning Analytics. Paper presented at LAK13.
Learners use analytics to
• Monitor their own activities and interactions
• Monitor the learning process
• Compare their activity with that of others
• Increase awareness, reflect and self reflect
• Improve discussion participation
• Improve learning behaviour
• Improve performance
• Become better learners
• Learn!
6
Dyckhoff, A L, Lukarov, V, Muslim, A, Chatti, M A, & Schroeder, U. (2013). Supporting Action Research with Learning Analytics. Paper presented at LAK13.
Analytics example: UK schools
7
• Aligned with clear aims• Huge and sustained effort• Agreed proxies for learning• Clear and standardised visualisation• Driving behaviour at every level
BUT• Stressed, unhappy learners• Analytics with little value for learners or educators• Omission of key areas, such as collaboration
Analytics example: Course Signals
Developed at Purdue University
8
Arnold, K E, & Pistilli, M (2012). Course Signals at Purdue: Using Learning Analytics To Increase Student Success. Paper presented at LAK12, Vancouver, Canada.
Analytics example: SNAPP
Network analysis
9
Analytics example: iSpot
Heading
10
Learning design in MOOCs
●Puts the learning journey at the heart of the design process●Provides a set of tools and information to support a learner-
activity based approach ●Helps to show the costs and performance outcomes of
design decisions●Enables the sharing of best practice●Helps MOOC designers to choose and integrate a coherent
range of media, technologies and pedagogies
●Enables a consistent and structured approach to review and analytics
11
Mor, Y, Ferguson, R, & Wasson, B. (2015). Editorial: learning design, teacher inquiry into student learning and learning analytics: a call for action. British Journal of Educational Technology, 46(2), 221-229.
MOOC learning design tools
• MOOC design template
• MOOC planner
• MOOC map
• Journey planner
12
Design template analytics
13
Learning outcome How this is assessed
1. Be able to define an ecosystem.
2. Have joined the iSpot community and obtained identifications for animals, plants or fungi.
1. Multiple choice. Week 1, question 5
2. Self report.
Analytics
1. How many attempted that question? How many got it right 1st / 2nd / 3rd time? How many followed the link back to resources?
2. Access to iSpot data. Use of MOOC hashtag. Persistence over time. Ethical implications of tracking off-site.
Short description of course and learning outcomes
MOOC planner
• Delivered• Reflection• Collaboration• Conversation• Networking• Browsing• Assessment
14
Blocking out types of learning activity
Conole, Gráinne. (2010). Learning design – making practice explicit. Paper presented at ConnectEd, Sydney, Australia. http://cloudworks.ac.uk/cloud/view/4001
MOOC planner analytics
Delivered Content (reading, watching, listening and observing)Analytics: amount of content viewed, dwell time
Reflection (thinking, considering and reflecting)Analytics: returns to the same material, reflection exercises completed, quality of reflection
Collaboration (constructing, collaborating, defining and engaging)Analytics: collaboration exercises completed, quality of collaboration
Conversation (debating, arguing, questioning, discussing…)Analytics: number and length of contributions, quality of discussion
Browsing (exploring, searching, finding and discovering)Analytics: Number of click-throughs to external links, number of visits, number of resources
Assessment (answering, presenting, demonstrating, critiquing…)Analytics: Assessments completed, scores, dwell time on hints, persistence in answering questions
15
MOOC map analytics
●How long did you expect learners to spend on these key elements?
●How long did learners actually spend on the key elements
●How many missed out these elements?
●How many jumped ahead to these elements?
●Which types of element are consistently (un)popular?
●How many left the MOOC at these points?
16
The MOOC map identifies key elements of the course
Ass
imila
tive
Info
rmat
ion
Han
dlin
g
Pro
duct
ive
Exp
erie
ntia
l
Ada
ptiv
e
Com
mun
icat
ive
Ass
essm
ent
Org
anis
atio
n
0
100
200
300
400
500
600
Min
ute
s
MOOC journey planner analytics
17
Relationships between tools, resources, activities & narrative
A framework for data collection
Analytics to solve problems
Analytics could filter discussions or group learners
18
You have been actively engaged in the discussions, which is excellent, thank you, but with more than 23,000 participants it
means that our responses and comments risk getting lost.
This will be primary school material for some of you and exactly the opposite for others. It is just not possible to tailor the
material to each of you […]Introduction to Forensic Science:University of Strathclyde
Start with the pedagogy
• How do people learn?
• How can we use data to facilitate that process in our MOOC?
• Which elements are learners struggling with?
• Which sections engage them the most?
• What prompts them to ask questions?
• Are they finding assessment challenging?
• What misconceptions have learners shown?
• Are there any accessibility issues?
• How can analytics be used to obtain desired learning outcomes?
19
Learning analytics and design
Learning design – helping to identify useful analytics●What do learners need to know in order to
network, collaborate, browse or reflect?●What do educators need to know to support them?
Learning design – helping to identify gaps in the data●What data do we need to collect? Learning design – helping to identify gaps in our toolkit●Which design elements can we look at easily?●Which ones still pose problems?
Learning design – helping to frame & focus analytics questions●What did they learn?… in relation to learning outcomes●Were they social?... when they were collaborating●Did they share links?... when encouraged to browse●Did they return to steps?... when encouraged to reflect
20
Making the links
Group activity20 minutes
Visit FutureLearn and register if necessaryhttps://www.futurelearn.com/
Select a MOOC currently open for registration.
Look at the introductory material and the first week – What are the expected
learning outcomes?
How will learners know they have achieved these?
21
Plenary15 minutes
What types of learning outcome are specified?
In what ways are learners assessed or could they be assessed?
What sorts of data and analytics could be used to support learners?
What sorts of data and analytics could be used to support
educators? 22
Group activity20 minutes
Take the first week of your chosen MOOC
Classify each step in terms of learning activity:(delivered content, reflection, collaboration,
conversation, browsing or assessment)
Note how long the step would be likely to take
Discuss which types of analytic would be useful to learners
and to educators
23
Plenary15 minutes
Share your findings
Which types of data and analytic could be used to support these types of
learning activity?
What would the educators need to know?
What would the learners need to know?
24