1. UMAP 08 July 2014 University of Leeds Using DBpedia as a Knowledge Source for Culture- Related User Modelling Questionnaires Dhaval Thakker (Uni. Of Leeds/UoL) Lydia Lau(UoL), Ronald Denaux(iSOCO, Spain), Vania Dimitrova(UoL), Paul Brna(UoL) and Christina Steiner(TU Graz, Austria)

2. UMAP 08 July 2014 University of Leeds Outline Definitions Problem: Modelling of learner's intercultural awareness Approach: interactive dialogue exploiting Linked data Evaluation Summary 3. UMAP 08 July 2014 University of Leeds t Kashima, Y. Conceptions of Culture and Person for Psychology. Journal of Cross-Cultural Psychology. 31, 1 (2000), 14-32. Gupta, V., Hanges, P.J., Dorfman, P. Cultural clusters: methodology and findings. Journal of World Business, 37,2 (2002) 11-15. Culture can be defined as a set of beliefs, values, behaviours and practices that characterise a given group of people [Kashima, 2000] Broad notion of culture Nationality and countries have been used as fairly reliable indicators for tackling cultural diversity [Gupta et al, 2002; Globe clusters; Hofstede dimensions] Narrowed scope: National Culture Definitions 4. University of LeedsUMAP 8 July 2014 Problem Cultural Awareness key 21st Century skills User-adaptive situational simulators are effective for training - first encounter BUT suffer from COLD START Situational simulators - Helps users in developing intercultural competencies - Provide User- Adaptive Learning Experience - Take into account learners knowledge of cultures 5. University of LeedsUMAP 8 July 2014 Main Challenge How to obtain a profile of a users knowledge of cultural aspects for different countries? Questionnaires and situational tests Do not scale Are not engaging Do not handle diversity Use the crowdwisdom from Wikipedia DBPedia semantic version of Wikipedia Large, multi-domain knowledge base 6. UMAP 8 July 2014 University of Leeds Our Approach Perico Dialogue Agent Knowledge Probing Quiz Generator User Profile GeneratorDialogue Planner User Model Cultural Exposure Score Knowledge Pool Challenge 2: Extract Knowledge Pool from DBpedia How to generate relevant knowledge pool from huge knowledge such as DBpedia? Challenge 1: Knowledge probing to assess learner's knowledge of a country 7. UMAP 8 July 2014 University of Leeds Extract Knowledge Pool from DBpedia Karanasios, S., Thakker, D., Lau, L., Allen, D., Dimitrova, V., & Norman, A. (2013). Making sense of digital traces: An activity theory driven ontological approach. Journal of the American Society for Information Science and Technology,(JASIST) 64(12), 2452-2467. 8. UMAP 08 July 2014 University of Leeds Distilling Intercultural Facts from DBpedia Identify DBpedia categories for seed terms (Clothing) Find specific categories and sub-categories (Loden_cape) Find broader categories (German_Culture; Germany) Generate facts Loden_cape is a Clothing Loden_cape occursIn Germany 9. UMAP 8 July 2014 University of Leeds Extracted Knowledge Pool 40K facts (OWL logical axioms) 565 items of clothing >4K items of food 88 gestures 159 currencies 288 languages 20K annotations (labels and depictions) Available at : http://imash.leeds.ac.uk/ontologies/amon/ 10. University of LeedsUMAP 8 July 2014 Probing and Modelling Users Knowledge Goal: Assess learner's (socio-political and intercultural) knowledge of country Ask facts (derived) from DBpedia Ask trick questions: close world around country Mark answers based on expected truth value and add to Country-awareness Profile Ronald Denaux, Vania Dimitrova, Lydia Lau, Paul Brna, Dhavalkumar Thakker, Christina M. Steiner: Employing linked data and dialogue for modelling cultural awareness of a user. IUI 2014: 241-246 11. University of LeedsUMAP 8 July 2014 Interaction with Perico http://imash.leeds.ac.uk:8080/perico/ https://www.youtube.com/watch?v=GTUtKSFjdLo 12. University of LeedsUMAP 8 July 2014 Evaluation Study Q1: Usability and interaction Is Perico usable and intuitive for the intended users; and what are the possible limitations of the interaction with Perico? Q2: UM accuracy Is the user model produced by Perico accurate against the users perception of his/her knowledge in the selected cultural aspects? 13. University of LeedsUMAP 8 July 2014 Evaluation Studies Participants - 22 participants (age mean=28), - Using GLOBE: - Group1 Narrow Cultural Exposure - Group 2 Broad Cultural Exposure Method - Online - Interaction with Perico four countries(none, low, med, high) familiarity - Total 92 questions in total User Model - Aggregated User model for each country and each topic - not-good, need- improvement, ok, very good - Participant rating -> accurate, under/over estimated Post-study - SUS usability questionaries' Q1: Usability and interaction Q2: UM accuracy 14. University of LeedsUMAP 8 July 2014 Usability three questions were tailored to Pericos interaction: (SUS11) The questions asked during the dialogue were easy to understand; (SUS12) The instructions provided during the dialogue were clear; and (SUS13) The assessment made by the dialogue was correct. Answering 92 questions in 30-45 minutes, the mean dialogue-score indicates good quality. System was easy to learn and did not require Additional support Integration & Consistency 15. University of LeedsUMAP 8 July 2014 Qualitative Feedback Deficiencies of DBpedia Pool - People in Cyprus use a garment called Icknield High School. - Frank is currency used in Germany Did not take globalisation into account - food, clothing, gestures have become common in countries which they did not originate from. - Inadequate assertions are hard to detect automatically. Possible solution - Allowing users to indicate that something is wrong - filtering or extending of the extracted DBpedia fact pool. 16. University of LeedsUMAP 8 July 2014 Interaction Feedback Limited content: - Some countries are prominent e.g. gestures coming largely from USA; - or that for some countries, e.g. Jordan, the dialogue presented mainly facts related to other countries. Possible solution - Expansion of topics: such as capital, population, climate, religion, festivals, popular sports, points of interest, were suggested. Lacking coherence: - interaction was jumping from question to question and lacked structure - due to the random selection from the pool of possible axioms Possible solution - to follow the GLOBE clusters - probing the knowledge on countries in the same cultural cluster, broadening, i.e. exploring countries from different clusters, - Or comparing, How does Italys income inequality compare to the UK higher/lower?). 17. University of LeedsUMAP 8 July 2014 UM Accuracy None or Low Exposure Medium or High Exposure Overestimation 17% 3% Underestimation 7% 12% Accurate 76% 86% 18. University of LeedsUMAP 8 July 2014 UM Feedback Answer indicated in the question -Indian Rupee, Japanese Yen, Bulgarian Lev, Polish Zloty Possible solution - use rdfs:label (name without the country) - use dbpprop:nickname (e.g. kint instead of Bulgarian lev). Answer given via knowledge elimination Able to answer the questions by knowing the facts about other countries they knew (canada -> china) (both cases) Possible solution (i) Explicitly ask if the user knew or guessed the answer; and (ii) ask for additional justification or explanation. Answer given using a clue in the question 19. University of LeedsUMAP 8 July 2014 Lessons Learnt Using Linked data for domain specific tasks Extensibility Missing or Dated Combine: give users opportunity to enter knowledge Gamifying questionnaires Engaging (>30 mis) Variety Examples Media Conversation-like Structure (GLOBE) Thanks! Dr.Dhaval Thakker Research Fellow Artificial Intelligence Group University of Leeds, UK http://dhavalthakker.wordpress.com/ Twitter: @dr_thakker