IntroductionSystem

Recommendation AlgorithmsExperimentsConclusions

SuggestBot: Using Intelligent Task Routing toHelp People Find Work in Wikipedia

Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl

presented by: Gianluca Demartini

July 6, 2007

Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing

IntroductionSystem

Recommendation AlgorithmsExperimentsConclusions

Outline

1 Introduction

2 System

3 Recommendation Algorithms

4 Experiments

5 Conclusions

Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing

IntroductionSystem

Recommendation AlgorithmsExperimentsConclusions

Scenario

Wikipedia lives thank to its contributorsIt is made of community-maintained artefacts of lasting value(CALV)Similar communities are iMDB, slashdot.org

Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing

IntroductionSystem

Recommendation AlgorithmsExperimentsConclusions

Wikipedia

A full dump (with history of pages) was 700GB in 2006Reasons for participating are similar to open source: learning,status, belongingBots: automated or semi-automated editing of pagesPeople tag articles they think need work

Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing

IntroductionSystem

Recommendation AlgorithmsExperimentsConclusions

Motivation

Member-maintained communities need contributionsReducing the cost of contribution increase motivationGoal: make it easy to find work to do

interestingthat need work

Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing

IntroductionSystem

Recommendation AlgorithmsExperimentsConclusions

Intelligent task routing definition

Intelligent task routingreduces the cost of finding workmatches people with tasks they are likely to care about

as a mechanism for increasing contribution

Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing

IntroductionSystem

Recommendation AlgorithmsExperimentsConclusions

Intelligent task routing on Wikipedia

With Intelligent task routing usinghistory of editstext matchinglink followingcollaborative filtering

people edit four time more often.

Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing

IntroductionSystem

Recommendation AlgorithmsExperimentsConclusions

Outline

1 Introduction

2 System

3 Recommendation Algorithms

4 Experiments

5 Conclusions

Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing

IntroductionSystem

Recommendation AlgorithmsExperimentsConclusions

SuggestBot

SuggestBot provides recommendations only on requestSuggestBot edits the user talk pages addingrecommendations

Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing

IntroductionSystem

Recommendation AlgorithmsExperimentsConclusions

Architecture

Four steps:Pre-processing WikipediaModelling user’s interestsFinding candidate articlesMake recommendations

Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing

IntroductionSystem

Recommendation AlgorithmsExperimentsConclusions

Figure: SuggestBot architecture

Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing

IntroductionSystem

Recommendation AlgorithmsExperimentsConclusions

Pre-processing

SuggestBot processes the wikipedia looking for users’ annotation.SuggestBot considers 6 types of work

Stub: short article needs more infoCleanup: need rewritingMerge: articles need to be combinedSource: need citationWikify: the text is not in the correct styleExpand: long article needs more info

Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing

IntroductionSystem

Recommendation AlgorithmsExperimentsConclusions

Modelling interests

The User’s Interests profileis build implicitly (no user participation)is the set of article titles that have been edited by the userdoes not contain more than 500 articlesdoes not consider “vandalism reversion” editsconsiders multiple edits as single edit

Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing

IntroductionSystem

Recommendation AlgorithmsExperimentsConclusions

Outline

1 Introduction

2 System

3 Recommendation Algorithms

4 Experiments

5 Conclusions

Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing

IntroductionSystem

Recommendation AlgorithmsExperimentsConclusions

Finding candidate articles

SuggestBot does not recommend already edited articlesSuggestBot finds related articles based on

similarity of text: user’s profile as query against full textconnections through linksconnections through co-editing

Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing

IntroductionSystem

Recommendation AlgorithmsExperimentsConclusions

Connections through links

SuggestBot uses links created by the users (citation network)SuggestBot ignores date-related linksALGO:

Initialize items in the profile to have a score of 1

Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing

IntroductionSystem

Recommendation AlgorithmsExperimentsConclusions

Remove items from original profile

Penalization function:

score :=score

log( #art|BestL−L|)

(1)

Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing

IntroductionSystem

Recommendation AlgorithmsExperimentsConclusions

Connections through co-editing

Find people whose history is similar usingJaccard metric for similarity between profilesGive credit to items based on the users’ similarity

Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing

IntroductionSystem

Recommendation AlgorithmsExperimentsConclusions

Connections through co-editing

Remove items edited by few others, or edited by the user t

Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing

IntroductionSystem

Recommendation AlgorithmsExperimentsConclusions

Filtering results

Both algorithms tend to recommend popular or controversialarticlesSuggestBot drops the top 1% of the most edited articles

Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing

IntroductionSystem

Recommendation AlgorithmsExperimentsConclusions

Outline

1 Introduction

2 System

3 Recommendation Algorithms

4 Experiments

5 Conclusions

Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing

IntroductionSystem

Recommendation AlgorithmsExperimentsConclusions

Setup

In 6 months 1200 people got recommendationsComparison based on number of edited recommendationsThree different experiments:

compared to random suggestions: 4 times more editedcomparing text-similarity, links, co-edit.Similar performances with differences:

text reco: focuses on rare wordlinks:biased by categories (link circles)co-edit: often edited articlesUsing meta-search techniques to combine results would help

removing noise: not considering minor edits does not improve

Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing

IntroductionSystem

Recommendation AlgorithmsExperimentsConclusions

Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing

IntroductionSystem

Recommendation AlgorithmsExperimentsConclusions

Outline

1 Introduction

2 System

3 Recommendation Algorithms

4 Experiments

5 Conclusions

Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing

IntroductionSystem

Recommendation AlgorithmsExperimentsConclusions

Conclusions

Simple algorithms gave strong resultsIntelligent task routing

can dramatically increase members’ contributionsis most useful where the tasks are numerous andheterogeneous

Future steps:incorporate meta-search techniquesremove noisegive to the user the possibility to edit the profile with low cost

Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing

IntroductionSystem

Recommendation AlgorithmsExperimentsConclusions

The End

Q&A

Dan Cosley, Dan Frankowski, Loren Terveen, John Riedl Intelligent Task Routing