Upload
ralf-klamma
View
628
Download
3
Tags:
Embed Size (px)
Citation preview
Lehrstuhl Informatik 5 (Information Systems)
Prof. Dr. M. Jarke 1
Learning Layers
This slide deck is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
Technical Challenges for Realizing Learning Analytics
Ralf Klamma Advanced Community Information Systems (ACIS)
RWTH Aachen University, Germany [email protected]
LEARNTEC, Karlsruhe, Germany, January 27th, 2015
Lehrstuhl Informatik 5 (Information Systems)
Prof. Dr. M. Jarke 2
Learning Layers
RWTH Aachen University
• 512 professors, 4675 academic and 2443 non-academic colleagues
• Annual budget around 884 million Euros, 445 million Euros funded by third parties
• 1,250 spin-off businesses have created around 30,000 jobs in the greater Aachen region over the past 20 years
• 260 institutes in 9 faculties as Europe’s leading institutions for science and research
• Currently around 40,375 students are enrolled in over 130 academic programs
• Over 6,300 of them are international students hailing from 120 different countries
Lehrstuhl Informatik 5 (Information Systems)
Prof. Dr. M. Jarke 3
Learning Layers
Responsive Open
Community Information
Systems
Community Visualization
and Simulation
Community Analytics
Community
Support
Web Analytics W
eb E
ngin
eerin
g
Advanced Community Information Systems (ACIS) Group @ RWTH Aachen
Requirements Engineering
Lehrstuhl Informatik 5 (Information Systems)
Prof. Dr. M. Jarke 4
Learning Layers
Agenda
Lear
ning A
nalyt
ics
Comm
unity
Lear
ning A
nalyt
ics
Expe
rts in
Com
munit
y Info
rmati
on
Syste
ms
Over
lappin
g Com
munit
y Ide
ntific
ation
Conc
lusion
s & O
utloo
k
Lehrstuhl Informatik 5 (Information Systems)
Prof. Dr. M. Jarke 5
Learning Layers
LEARNING ANALYTICS
Lehrstuhl Informatik 5 (Information Systems)
Prof. Dr. M. Jarke 6
Learning Layers Self- and Community Regulated
Learning Processes
Based on [Fruhmann, Nussbaumer & Albert, 2010]
Learner profile information is
defined or revised
Learner finds and selects learning resources
Learner works on selected learning resources
Learner reflects and reacts on
strategies, achievements and usefulness
plan
learnreflect
The Horizon Report – 2011 Edition
Lehrstuhl Informatik 5 (Information Systems)
Prof. Dr. M. Jarke 7
Learning Layers The long tail of personal knowledge
in life-long learning
■ Zillions of new learning opportunities ■ Abundance of learning materials ■ But: Extremely challenging to find & navigate
High-quality, specially designed, learning materials like books or course material
Gaps in personal knowledge identified mostly by real-world practice
Lehrstuhl Informatik 5 (Information Systems)
Prof. Dr. M. Jarke 8
Learning Layers
Web 2.0 Competence Development Cultural and Technological
Shift by Social Software Impact on
Knowledge Work Impact on
Professional Communities
Web 1.0 Web 2.0 Microcontent Providing
commentary Personal knowledge
publishing Establishing personal
networks Testing Ideas
Social learning Identifying competences Emergent Collaboration
Trust & Social capital
personal website and content management
blogging and wikis User generated content Participation
directories (taxonomy) and stickiness
Tagging ("folksonomy") and syndication
Ranking Sense-making
Remixing Aggregation Embedding
Emergent Metadata Collective intelligence Wisdom of the Crowd Collaborative Filtering Visualizing Knowledge
Networks
Lehrstuhl Informatik 5 (Information Systems)
Prof. Dr. M. Jarke 9
Learning Layers
Personal Learning Environment (PLE) PLE describes the tools, communities, and services that constitute the individual educational platforms learners use to direct their own learning and pursue educational goals LMS – course-centric vs. PLE – learner-centric:
• Extension of individual research • Students in charge of their learning process
• self-direction, responsibility • Promotes authentic learning (incorporating expert feedback) • Student’s scholarly work + own critical reflection + the work and voice of others • Web 2.0 influence on educational process
• customizable portals/dashboards, iGoogle, My Yahoo! • Learning is a collaborative exercise in collection, orchestration, remixing, & integration of data into knowledge building • Emphasis on metacognition in learning
Lehrstuhl Informatik 5 (Information Systems)
Prof. Dr. M. Jarke 10
Learning Layers ROLE Approach to the Design
of Learning Experiences What is the impact of these findings from behavioral & cognitive psychology on
design of Personal Learning Environments?
learner profile information is defined and revised
learner finds and selects learning resources
learner works on selected learning resources
plan
learn reflect
learner input regarding goals, preferences, …
creating PLE
recommendations from peers or tutors
assessment and self-assessment
evaluation and self-evaluation
feedback (from different sources)
learner should understand and control own learning process
ROLE infrastructure should provide adaptive guidance
attaining skills using different learning events (8LEM)
learner reflects and reacts on strategies, achievements,
and usefulness
monitoring recommen-dations
be aware of
Lehrstuhl Informatik 5 (Information Systems)
Prof. Dr. M. Jarke 11
Learning Layers Learning Analytics Visualization –
Dashboards
Lehrstuhl Informatik 5 (Information Systems)
Prof. Dr. M. Jarke 12
Learning Layers Learning Analytics vs. Community
Learning Analytics Formal Learning Learning Analytics Community
Regulated Learning
Community Learning Analytics
Environment LMS EDM/Visual Analytics (VA) – xAPI??
Responsive Open Learning Environment (ROLE)
Data Mining / VA /Social Network Analysis / Role Mining
Tools Fixed LMS Specific Eco-System Tool Recommender
Activities Fixed Content Recommender
Dynamic Content Recommender / Expert Recommender
Goals Fixed Progress Dynamic Progress / Goal Mining / Refinement
Communities Fixed Not applicable Dynamic (Overlapping) Community Detection
Use Cases Courses Learning Paths Peer Production / Scaffolding
Semantic Networks of Learners / Annotations
Lehrstuhl Informatik 5 (Information Systems)
Prof. Dr. M. Jarke 13
Learning Layers
COMMUNITY LEARNING ANALYTICS
Lehrstuhl Informatik 5 (Information Systems)
Prof. Dr. M. Jarke 14
Learning Layers
Learning Communities Communication / Cooperation ?
Cultural heritage in Afghanistan
Database
Content input / request
Content retrieval
Surveying/ safeguarding
Sketch drawing
Photographing
Surveying/ recording
GPS positioning
Experiences imparting
Administration
UNESCO
Teaching/ presentation
Asia
ICOMOS
Standards defining
Research
RWTH Aachen
SPACH
www.bamiyan-development.org
Lehrstuhl Informatik 5 (Information Systems)
Prof. Dr. M. Jarke 15
Learning Layers Experts in
Learning Communities ■ In learning communities
many experts from different fields meet – Intergenerational learning – Interdisciplinary learning
■ New Openness for Amateur Contributions
■ Methods, Tools & CoP co-develop – Expert role models needed – Expert identification based
on complex media traces
Lehrstuhl Informatik 5 (Information Systems)
Prof. Dr. M. Jarke 16
Learning Layers
Communities of Practice ■ Communities of practice (CoP) are groups of people
who share a concern or a passion for something they do and who interact regularly to learn how to do it better (Wenger, 1998)
■ Characterization of experts in CoP – Shared competence in the domain – Shared practice over time by interactions – Expertise based on gaining and having reputation within the CoP – Being an expert vs. being a layman, a newcomer, an amateur etc. – Informal leadership – Identity as an expert depends on the lifecycle of the communities
Expertise in highly dynamic, locally distributed multi-disciplinary and heterogeneous communities?
Lehrstuhl Informatik 5 (Information Systems)
Prof. Dr. M. Jarke 17
Learning Layers Proposed Development of the
Community Learning Analytics Field ■ Will happen J Big Data by Digital Eco Systems (Quantitative Analysis)
– A plethora of targets (Small Birds) – Professional Communities are distributed in a long tail – Professional Communities use a digital eco system
– An arsenal of weapons (Big Guns) – A growing number of community learning analytics methods – Combined methods from machine intelligence and knowledge representation
■ May not happen L Deep Involvment with community (Qualitative Analysis) – Domain knowledge for sense making – Passion for community and sense of belonging – Community learns as a whole
→ Community Learning Analytics for the Community by the Community
Lehrstuhl Informatik 5 (Information Systems)
Prof. Dr. M. Jarke 18
Learning Layers Interdisciplinary Multidimensional
Model of Communities ■ Collection of CoP Digital Traces in a MediaBase
– Post-Mortem Crawlers – Real-time, mobile, protocol-based (MobSOS) – (Automatic) metadata generation by Social Network Analysis
■ Social Requirements Engineering with i* Framework for defining goals and dependencies in CoP
Social Software Cross-Media Social Network Analysis on Wiki, Blog, Podcast, IM, Chat, Email, Newsgroup, Chat …
Web 2.0 Business Processes (i*) (Structural, Cross-media)
Members (Social Network Analysis: Centrality,
Efficiency, Community Detection)
Network of Artifacts Content Analysis on Microcontent, Blog entry, Message,
Burst, Thread, Comment, Conversation, Feedback (Rating)
Network of Members
Communities of practice
Media Networks
Lehrstuhl Informatik 5 (Information Systems)
Prof. Dr. M. Jarke 19
Learning Layers Community Learning Analytics
in CoP ■ User-to-Service Communication
• CoP-aware Usage Statistics • Identification of successful CoP services • Identification of CoP service usage patterns
■ User-to-User Communication • CoP-aware Social Network Analysis • Identification of influential CoP members • Identification of CoP member interaction/learning patterns
+
Lehrstuhl Informatik 5 (Information Systems)
Prof. Dr. M. Jarke 20
Learning Layers
COMMUNITY LEARNING ANALYTICS – EXPERT IDENTIFICATION
Lehrstuhl Informatik 5 (Information Systems)
Prof. Dr. M. Jarke 21
Learning Layers
Space (shared by multiple users)
Video-Based Learning Framework
Web application (composed of widgets)
Widget (collaborative web component)
http://role-sandbox.eu/
Lehrstuhl Informatik 5 (Information Systems)
Prof. Dr. M. Jarke 22
Learning Layers ROLE Sandbox – Geospatial &
Temporal Access
§ Users: 1046 § Widgets: 523 § Spaces/Activities: 1377 § Shared Resources: 3764
Lehrstuhl Informatik 5 (Information Systems)
Prof. Dr. M. Jarke 23
Learning Layers YouTell - A Web 2.0 Service for
Collaborative Storytelling § Collaborative storytelling § Web 2.0 Service § Story search and
“pro-sumption”
§ Tagging § Ranking/Feedback § Expert finding § Recommending
Klamma, Cao, Jarke: Storytelling on the Web 2.0 as a New Means of Creating Arts Handbook of Multimedia for Digital Entertainment and Arts, Springer, 2009
Lehrstuhl Informatik 5 (Information Systems)
Prof. Dr. M. Jarke 24
Learning Layers Expert Finding – Computation of
Actual Knowledge ■ Data vector consists of
– Personal data vector – Competences, skills,
qualification profile – Self-entered data
– Story data vector – Visits of stories – Involvement in projects
– Expert data vector – Advice given – Advice received
– Value = #Keywords � Date Decay � Feedback
Motivation PESE: Web 2.0 –Anwen- dung für community- basiertes Storytelling Der PESE- Prototyp Evaluierung des Prototypen Zusammen- fassung Ausblick
Find the most appropriate expert
Data vector represents knowledge of the expert
Lehrstuhl Informatik 5 (Information Systems)
Prof. Dr. M. Jarke 25
Learning Layers Knowledge-Dependent
Learning Behaviour in Communities
Renzel, Cao, Lottko, Klamma: Collaborative Video Annotation for Multimedia Sharing between Experts and Amateurs, WISMA 2010, Barcelona, Spain, May 19-20, 2010
§ Expert finding algorithm: Knowledge value of community sorted by keywords § Community behavior: Experts spent more time on the services § Experts prefers semantic tags while amateurs uses “simple” tags frequently § Community tags: Experts use more precise tags
Lehrstuhl Informatik 5 (Information Systems)
Prof. Dr. M. Jarke 26
Learning Layers
Threads to Expert Finding ■ Compromising techniques
— Sybil attack [Douc 2002], Reputation theft, Whitewashing attack, etc.. — Compromising the input and the output of the expert identification algorithm
■ Example: Sybil attacks — Fundamental problem in open collaborative Web systems — A malicious user creates many fake accounts (Sybils) which all reference the user to
boost his reputation (attacker’s goal is to be higher up in the rankings)
Sybil region Honest region A0ack edges
Lehrstuhl Informatik 5 (Information Systems)
Prof. Dr. M. Jarke 27
Learning Layers
Conclusions & Outlook
■ Learning Analytics for Formal and Informal Learning – Challenges for data gathering and data management – Challenges for quantitative and qualitative analysis – Challenges for visual analytics, feedback and
interventions ■ Community Learning Analytics
– Responsive Open Learning Environments (ROLE) – Learning Layers – Learning Analytics as a Service – Social Network Analysis – Community Detection – Expert Identification