Embed Size (px)
DESCRIPTION
This presents the European Schoolnet's content enrichment, both by experts and end-users (folksonomic approach).
Citation preview
http://lre.eun.org
http://lre.eun.org
Collaborative Content Annotation in
Multilingual Europe
Riina Vuorikari, Frans Van AsscheEuropean Schoolnet
1st VARIAZIONI WorkshopBarcelona Nov 29 2007
http://lre.eun.org
http://lre.eun.org
Collaborative Content Annotation in
Multilingual Europe* motivation and context* federated architecture
* co-existence of LOM and
unstructured metadata* envisaged enrichment services
http://lre.eun.org
http://lre.eun.org
Collaborative Content Annotation in
Multilingual Europe
* what does multi-linguality mean for tags and how do
users deal with it?
http://lre.eun.org
http://lre.eun.org
Motivation of this work
European education, especially that of K-12 education,
is inherently multi-lingual and multi-cultural.
http://lre.eun.org
http://lre.eun.org
http://lre.eun.org
http://lre.eun.org European Schoolnet
Established a network among European Educational Authorities
in 1996
Currently 27 members from more than EU
http://lre.eun.org
http://lre.eun.org
...times are changing
http://lre.eun.org
http://lre.eun.org
Proliferation of web-based servicesfor sharing digital items
http://lre.eun.org
http://lre.eun.org
Proliferation of web-based servicesfor sharing digital items
http://lre.eun.org
http://lre.eun.org
New challenges for repositories
European teachers have access to multiple repositories of digital learning resources by – Educational Authorities, – publishers, – other teachers,.
Users become more demanding and expect services that are seen elsewhere (rss, personalised feed-selections, bookmarks, community rankings ..)
http://lre.eun.org
http://lre.eun.org
http://lre.eun.org
http://lre.eun.org
Since 1999 EUN's goal:
to facilitate the access to multi-lingual
learning resources repositories
http://lre.eun.org
http://lre.eun.org
EUN + Federation of LORs
=
Learning Resources Exchange
(LRE)
http://lre.eun.org
http://lre.eun.org
Long process in semantic interoperability since 1999
– First DC based Application Profile in 2001
Semantic interoperability for K-12
http://lre.eun.org
http://lre.eun.org
LOM based Application Profile v 3.0LOM based Application Profile v 3.0http://insight.eun.org/intern/shared/data/insight/lre/AppProfilev3p0.pdf
Semantic interoperability for K-12
http://lre.eun.org
http://lre.eun.org
Multi-lingual thesaurus in more than 15 languages
Semantic interoperability for K-12
http://lre.eun.org
http://lre.eun.org
http://lre.eun.org
http://lre.eun.org
Challenge for users
End-users (e.g. teachers) have difficulties to discover and find resources from educational repositories
– Metadata does not always match search terms
Locating content across linguistic and national borders within Europe has proven hard
– Despite the use of a multilingual Thesaurus and controlled vocabularies
http://lre.eun.org
http://lre.eun.org
New Mission Critical
MetadataEgology for Learning Technologies
=> co-existence of structured and unstructured metadata
=> by expert indexers and end-users
http://lre.eun.org
http://lre.eun.org
LRE Architecture
http://lre.eun.org
http://lre.eun.org
Architecture
http://lre.eun.org
http://lre.eun.org
Enrichment: by experts
By expert indexers:
“toolbox will be extended with tools for the development of taxonomies, and with tools for automatic translation and vocabulary management, as well as for collecting attention metadata to track what users actually do with the MELT infrastructure”
Original Metadata
Samgi generated Metadata
instance
Edited by expert indexer
Merged Metadata
http://lre.eun.org
http://lre.eun.org
Multi-lingual enrichment
http://lre.eun.org
http://lre.eun.org
Collaborative content enrichment in multilingual Europe
Addition to the traditional LOM
By users interacting with the portal, resources and other users
Four main tools:– social bookmarking and tags in multiple
languages – rating of usefulness – pedagogical annotations (used in learning events)– levels of user engagement when interacting with
the system (what is viewed, how many times,..)
http://lre.eun.org
http://lre.eun.org
http://lre.eun.org
http://lre.eun.org
Social bookmarking
To store, organise, share and search bookmarks of web pages. Keep found things found!
Bookmarks are usually public and shared
Users organise bookmarks with informal tags (instead of the traditional folders)
Find like-minded users with similar interest.
http://lre.eun.org
http://lre.eun.org
What is a tag?
Metadata externally applied to an item
Can be used for sorting or managing
A hook for aggregating
Provides identifier and/or description
Personal marker by Thomas Vander Wal
http://lre.eun.org
http://lre.eun.org
http://lre.eun.org
http://lre.eun.org
Bookmark is a triple (user, resource, {tags})
http://lre.eun.org
http://lre.eun.org
A great navigational aid to discover resources!
http://lre.eun.org
http://lre.eun.org
Follow those digital
traces!
http://lre.eun.org
http://lre.eun.org
More ways for social navigation
Discover resources through tags in
multiple languages
http://lre.eun.org
http://lre.eun.org
by Stiphy
“Social”
makes
trails
visible..
http://lre.eun.org
http://lre.eun.org
and
shows
where
to flock
http://lre.eun.org
http://lre.eun.org Can tags and bookmarks be used to
connect users cross national and
language borders?
http://lre.eun.org
http://lre.eun.org
Social bookmarking in multi-lingual environment
http://lre.eun.org
http://lre.eun.org
.. allows new ways to discover both resources and people!
http://lre.eun.org
http://lre.eun.org
How do users bookmark?
January 1 to October 31 2007
A post : a triple of (user, item, {tag}) 1022 posts to favourites 142 users 682 different learning resources (items) 1029 multilingual tags recorded to the system,
some of which were reused by users
http://lre.eun.org
http://lre.eun.org
How do users bookmark?
0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150
0
10
20
30
40
50
60
70
80
90
100
110
120
130
Distribution of posts and inverse Power Law
Column JColumn K
Num
ber
of B
ookm
arks
http://lre.eun.org
http://lre.eun.org
How do users tag?
Most posts consisted of only one tag
More than half of tags were used just once (54%)
Only about 10% of tags were reused more than twise
No guided tagging in this pilot! This may change.
Percent of tags/postingOnetag 79%2 tags 15%3 tags 4%4 tags 1%5 tags 0.39%6 tags 0.23%7 tags 0.08%
http://lre.eun.org
http://lre.eun.org
Are tags found useful?
LO Type Lang Keyword % of votes1 Tag En healthy meal 1 0 0%
1 Thesaurus En health education 7 7 %
2 Tag En EU 7 7 %
2 Tag Pl Europa 62 %
3 Thesaurus En physics 85%
3 Thesaurus En mathematics 92 %
4 Thesaurus En mathematics 92 %
4 Tag En -GeoGebra program 62 %
5 Thesaurus En Geography 92 %
5 Thesaurus En Island 69%
Vuorikari, R., Ochoa, X., Duval, E. Analysis of User Behavior on MultilingualTagging of Learning Resources. In Workshop proceedings of the EC-TEL conference: SIRTEL07 (EC-TEL ’07) (Crete, Greece, September 17-20, 2007)
http://lre.eun.org
http://lre.eun.org
Are tags found useful?
LO Type Lang Keyword % of votes1 Tag En healthy meal 1 0 0%
1 Thesaurus En health education 7 7 %
2 Tag En EU 7 7 %
2 Tag Pl Europa 62 %
3 Thesaurus En physics 85%
3 Thesaurus En mathematics 92 %
4 Thesaurus En mathematics 92 %
4 Tag En -GeoGebra program 62 %
5 Thesaurus En Geography 92 %
5 Thesaurus En Island 69%
Tags, produced with no outlay, show an encouraging and
potential gain in overall usefulness!
Vuorikari, R., Ochoa, X., Duval, E. Analysis of User Behavior on MultilingualTagging of Learning Resources. In Workshop proceedings of the EC-TEL conference: SIRTEL07 (EC-TEL ’07) (Crete, Greece, September 17-20, 2007)
http://lre.eun.org
http://lre.eun.org
Does the language matter?
Need for better ways to identify the language– Give rules (if the user first preferred languages is..,
then..)– Automate the recognition of languages– Out-source it to users
http://lre.eun.org
http://lre.eun.org
“Travel well” tags
About 15% of tags contain a general term, a name, place, etc. that is easily understood without translation
e.g. AIDS, software, EU, Euroopa, Europa, europe, Evropa, geograafia, Pythagoras, etc.
http://lre.eun.org
http://lre.eun.org
What's the point of travel well tags?
If those tags need no translation or language filtering to be understood, and
..if they can be identified:
We can be sure to show at least some tags to users – whose language preferences we don't know, and – in whose language there are no tags or keywords
available.
http://lre.eun.org
http://lre.eun.org
User engaging with resources
Steps taken:- views page- views metadata- bookmarks and
tags- rates
- what about the actual use?
http://lre.eun.org
http://lre.eun.org
Semantic analysis of tags
Factual tags 63%(Golder: item topics, kinds of item, category refinements)
Subjective tags 29%( Golder: item qualities)
Personal tags 3% (Golder: item ownership, self-reference, tasks organisation)
5% other
Sen et al. (2006).
http://lre.eun.org
http://lre.eun.org
Why tag categories? In Sen et al. (2006) it
was found that tags of different categories can be useful for different tasks
In our case it is too early to say anything, but ...we'll have an eye on it!
http://lre.eun.org
http://lre.eun.org
social bookmarks Metadata LOM tags folksonomy social tagging
multi-linguality social classificationthanks! for your attention
learning resources user communities
discover resources and itemsquestions?
teachers social navigationsocial tracespaths, trails
http://www.cs.kuleuven.ac.be/~hmdb/infovis/calibrate/calibrate.htmlflock
