Embed Size (px)
Citation preview
page 1
A Social Semantic Recommender for Learning
Soude Fazeli, PhD candidate Dr. Hendrik Drachsler Prof. Dr. Peter Sloep
page 2
• NELLL (Netherlands Laboratory for Lifelong Learning at the OUNL)
2
Run-time: 2011-2015
A socially-powered, multilingual open learning infrastructure to boost the adaptation of eLearning Resources in Europe
• Open Discovery Space (ODS)
The doctoral study is funded by
page 3
A social space for learning
page 4
page 5
Recommender systems?
page 6
Link to Learning Analytics (LA)
• Duval (2011) introduced recommenders as a solution • To deal with the “paradox of choice” • To turn the abundance from a problem into an asset for
learning • Several domains try to find patterns in a large amount of data
• Educational data mining, Big Data, and Web analytics • Recommender systems and personalization as an important part
of LA research, Greller and Drachsler (2012)
page 7
Based on the framework proposed by Manouselis & Costopoulou (2007) For more details, please refer to Fazeli, S., Drachsler, H., Brouns, F. and Sloep, P. (2012)
A proposed recommender system for learning
!
page 8
Sparsity!
Similarity
page 9
State-of-the-art educational recommenders
• Manouselis et al. (2010) • Testing multi-attribute recommenders within Learning Resource Exchange
(http://lreforschools.eun.org)
• Cechinel et al. (2012) • Several memory-based collaborative filtering algorithms on the MERLOT
repository (http://www.merlot.org)
• Koukourikos et al. (2012) • Using sentiment analysis techniques to enhance collaborative filtering
algorithms within MERLOT dataset
• Sparsity! • Verbert et al. (2011)
• Different algorithms on several datasets: MACE, Travel well, MovieLens
• Manouselis et al. (2012) • Organic.Edunet (http://portal.organic-edunet.eu/) and a synthetic dataset
including the real data plus some simulated data
page 10
(Golbeck, 2009; Kamvar et al., 2003; Ziegler & Golbeck, 2007; Massa & Avesani, 2004; Lathia et al., 2008; Fazeli et al., 2010)
page 11
A social recommender system: T-index approach (Fazeli et al., 2010)
• Creates trust relationships between users • Based on the ratings information
• Proposes T-index concept • To measure trustworthiness of users • To improve the process of finding the nearest neighbours
• Inspired by H-index • Used to evaluate the publications of an author
• Based on results, T-index improves • Prediction accuracy of generated recommendations • Structure of trust networks of users
page 12
Trust in recommender systems
• Trustworthy users == like-minded users
• A new trust relationship between two thus far unconnected users is inferred if and only if: • Condition 1:
• mutual trust value between intermediate users is higher than a certain threshold
• Condition 2: • The number of intermediate users is lower than an upper bound;
in this study the upper bound is 2
page 13
Social trust in recommender systems
Alice
Carol
Bob
rated rated
rated
rated
if A trusts B and B trusts C, then A trusts C if and only if condition 1 is met
and condition 2 is met
page 14
Social data
page 15
• RQ1: How to generate more accurate and thus,
more relevant recommendations by using the social data originating from social activities of users within an online environment?
• RQ2: Can the use of the inter-user trust
relationships that originally come from the social activities of users within an online environment, help user networks evolve?
page 16
Proposed research
1. Requirement analysis • Literature review • Interview study
2. Data-driven study 3. User evaluation study 4. Pilot study
page 17
1. Requirement analysis
• Goal • Investigating main needs and requirements of users in an online social environment
• Method • A summer school for European teachers in Greece, July 2012 • Asking the participants to fill in a questionnaire regarding
• The importance or usefulness of the activities within an online social environment • The use of recommender systems.
• Description • 33 teachers participated from 14 countries (Portugal, Germany, France, Finland,
Greece, Austria, Poland, Lithuania, Spain, Hungary, Romania, Cyprus, Ireland, Serbia and the US)
• “sharing content on Facebook, Twitter, etc. or by email” important, useful or not
• Expected outcomes • A list of the most important needs and requirements of teachers within an online social
environment like the ODS portal
page 18
1. Requirement analysis 1.1. Use case diagram
!
page 19
1. Requirement analysis 1.2. Results
!How much the teachers find the online social
activities important/useful
How much teachers find the detailed requirements important/useful
page 20
Proposed research
1. Requirement analysis • Literature review • Interview study
2. Data-driven study 3. User evaluation study 4. Pilot study
page 21
2. Data-driven study
• Goal • To find out the most suitable recommender system algorithm for a social
online platform like ODS platform
• Method • An offline empirical study of candidate algorithms including the extended T-
index algorithm • Datasets:
• TravelWell, Mace, OpenScout, MovieLens (as a standard dataset for comparison) • Mendeley, MERLOT
• Variables to be measured • Performance: Precision accuracy, recall, F-measure (F1) • Network analysis: degree centrality
• Expected outcomes • Which of the recommender algorithms best performs and thus, is suitable for
social online platforms like ODS platform
page 22
2. Data-driven study 2.1. F1 result
F1 of the extended T-index and Tanimoto algorithms for different datasets, based on the size of neighborhood
0"0.01"0.02"0.03"0.04"0.05"0.06"0.07"0.08"0.09"0.1"
3" 5" 7" 10"
F1@10%
size%of%neighborhood%(n)%
MACE%
Tanimoto4Jaccard"(CF1)"
Loglikelihood"(CF2)"
Euclidean"(CF3)"
Graph4based"(CF4)"
0"
0.02"
0.04"
0.06"
0.08"
0.1"
0.12"
0.14"
3" 5" 7" 10"
F1@10%
size%of%neighborhood%(n)%
OpenScout%
Tanimoto3Jaccard"(CF1)"
Loglikelihood"(CF2)"
Euclidean"(CF3)"
Graph3based"(CF4)"
0"
0.02"
0.04"
0.06"
0.08"
0.1"
3" 5" 7" 10"
F1@10%
size%of%neighborhood%(n)%
Travel%well%
Tanimoto3Jaccard"(CF1)"
Loglikelihood"(CF2)"
Euclidean"(CF3)"
Graph3based"(CF4)"
0"
0.05"
0.1"
0.15"
0.2"
0.25"
3" 5" 7" 10"
F1@10%
size%of%neighborhood%(n)%
MovieLens%
Tanimoto0Jaccard"(CF1)"
Loglikelihood"(CF2)"
Euclidean"(CF3)"
Graph0based"(CF4)"
page 23
2. Data-driven study 2.2. user network
page 24
2. Data-driven study 2.3. Degree centrality
Degree distribution of top-10 central users for different datasets
0"
50"
100"
150"
200"
250"
u1" u2" u3" u4" u5" u6" u7" u8" u9" u10"
degree%
Top)10%central%users%
MovieLens"
OpenScout"
MACE"
Travel"well"
page 25
Proposed research
1. Requirement analysis • Literature review • Interview study
2. Data-driven study 3. User evaluation study 4. Pilot study
page 26
3. User evaluation study
• Goal • To study usability of developed prototype by evaluating
users’ satisfaction
• Method • Questionnaire • Adapting the user-centric evaluation proposed by Pu et al.
(2011) in the context of recommender systems
• Variables to be measured • Quality of recommendations based on accuracy, novelty,
and usefulness
• Expected outcomes • Initial feedback by end-users on users’ satisfaction as an
input for pilot study
page 27
Proposed research
1. Requirement analysis • Literature review • Interview study
2. Data-driven study 3. User evaluation study 4. Pilot study
page 28
4. Pilot study
• Goal • To deploy the final release • To test it under realistic operational conditions with the end-users
• Method • Evaluating performance of the designed recommender system algorithm • Study the structure of the built users network
• Variables to be measured • Prediction precision and recall, and F-measure (F1) • Effectiveness in terms of total number of visited, bookmarked, or rated
learning objects for two groups of users (pre and post study) • Degree centrality distribution to study how the structure of users network
changes
• Expected outcomes • Empirical data on performance of the used recommender algorithm • The visualization of teachers’ networks
page 29
Conclusion
• The aim is to support user in social platforms to find the most suitable content or people
• Recommender systems as a solution • How to deal with the sparsity problem by use of
social data of users
page 30
Ongoing and Further work
• Data set study (May 2013) • Testing more datasets (Mendeley, MERLOT) • Testing other recommender algorithms (loglikelihood for implicit indicators,
Pearson, Euclidian for explicit indicators)
• Go online with the ODS platform (June 2013) • User evaluation study (September 2013)
page 31
Soude Fazeli PhD candidate Open University of the Netherlands Centre for Learning Sciences and Technologies (CELSTEC) PO-‐Box 2960 6401 DL Heerlen, The Netherlands email: [email protected]
