Upload
marisa
View
42
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Human Computation Play a Game to Develop an Ontology. Peyman Nasirifard p+e+y+m+a+b-b+n dot sin(arcsin(lastname)) @ deri.org. Agenda. Introduction CAPTCHA Games with a purpose ESP game Peekaboom Verbosity Possible game for developing simple ontologies Play a game Conclusion. - PowerPoint PPT Presentation
Citation preview
11
Human ComputationHuman Computation
Play a Game to Develop an OntologyPlay a Game to Develop an Ontology
Peyman NasirifardPeyman Nasirifard
p+e+y+m+a+b-b+n p+e+y+m+a+b-b+n dotdot sin(arcsin(lastname)) sin(arcsin(lastname)) @@ deri.org deri.org
22
AgendaAgenda
IntroductionIntroduction CAPTCHACAPTCHA
Games with a purposeGames with a purpose ESP gameESP game PeekaboomPeekaboom VerbosityVerbosity
Possible game for developing simple ontologiesPossible game for developing simple ontologiesPlay a gamePlay a gameConclusionConclusion
33
IntroductionIntroductionHuman-based computationHuman-based computation is a technique when a computational is a technique when a computational process performs its function via outsourcing certain steps to process performs its function via outsourcing certain steps to humans.humans.
44
Back to HistoryBack to History
Yahoo! and Gmail are not interested to Yahoo! and Gmail are not interested to enable a bot to create thousands accounts enable a bot to create thousands accounts per day for sending spamper day for sending spam They use CAPTCHA to prevent itThey use CAPTCHA to prevent it
plus
55
CAPTCHACAPTCHA
Stands for “Completely Automated Public Stands for “Completely Automated Public Turing test to tell Computers and Humans Turing test to tell Computers and Humans Apart”Apart”Luis von AhnLuis von Ahn et al. coined the term in et al. coined the term in 20002000A Program that can tell A Program that can tell whether a user is a human whether a user is a human or a computeror a computerMany different techniquesMany different techniques
66
Some ExamplesSome Examples
77
Dog or Cat?Dog or Cat?
Human: mmm… dog
Computer: mmmmmmmmmmmmmmm…
88
Human ComputationHuman Computation
If we use people to break CAPTCHA, we If we use people to break CAPTCHA, we are doing human computationare doing human computation In some countries, some companies hire In some countries, some companies hire
people to break CAPTCHA and send spampeople to break CAPTCHA and send spam Some companies Some companies cleverlycleverly use humans to use humans to
break CAPTCHA and send spambreak CAPTCHA and send spam How?How?
99
Clever spammersClever spammers
1010
Clever SpammersClever Spammers
Type the word in the box if you want to see the next picture
Free Nude Photos
1111
Really?!Really?!Jan 2004: world without spam by 2006! Jan 2004: world without spam by 2006! Huge amount of investmentHuge amount of investment Bill Gates receives 4 million spams per dayBill Gates receives 4 million spams per day
1212
Nice QuoteNice QuoteLuis von Ahn: Luis von Ahn: Instead of hiring people and Instead of hiring people and pay them to solve our problems, we can pay them to solve our problems, we can design games and people will pay us to design games and people will pay us to play our games and solve our large-scale play our games and solve our large-scale problems!problems!
1313
The ESP gameThe ESP gameObject of the game: type the same wordObject of the game: type the same wordOnly thing in common is: an imageOnly thing in common is: an imagePlayersPlayers
Do not know each other (randomly paired)Do not know each other (randomly paired) Can not communicateCan not communicate
Advantages:Advantages: Two different sources labels the imageTwo different sources labels the image enjoyableenjoyable labels all images on labels all images on Google imageGoogle image in a short time in a short time Help to improve English!Help to improve English! There are many people that play over 20 hours a weekThere are many people that play over 20 hours a week
1414
The ESP gameThe ESP gamePlayer 1 Player 2
• CAR
• GIRL
• TREE
• WOMAN
• CAR
Agree: CAR
Get points
1515
Taboo wordsTaboo words
Taboo wordsTaboo words More difficult, but more funMore difficult, but more fun
• CAR
• WOMAN
1616
Single version of ESP gameSingle version of ESP game
• CAR
• GIRL
• TREE
• WOMAN
• CAR
• The engine records everything from previous players
• A single player will actually play with another player, but not at the same time
1717
Cheating and RepetitionCheating and Repetition
Problem: Agreement on cheatingProblem: Agreement on cheating Let’s label all images with “dog”Let’s label all images with “dog”Solution: At random, system gets players Solution: At random, system gets players test images to check whether they play test images to check whether they play honestly or nothonestly or not If they do not play honestly, the system will let If they do not play honestly, the system will let
them play, but nothing will be recordedthem play, but nothing will be recordedFor certainty, only labels which at least N For certainty, only labels which at least N pairs agreed upon will be storedpairs agreed upon will be stored
1818
The Limitations of ESPThe Limitations of ESP
The ESP Game can label images (and The ESP Game can label images (and consequently tell you what’s in them), but consequently tell you what’s in them), but it cannot:it cannot: Find the objects being labelledFind the objects being labelled Determine the way in which the object Determine the way in which the object
appears – does the label “car”appears – does the label “car”refer to the text “car” or an actual refer to the text “car” or an actual car in the image?car in the image?
1919
The place of objects in an imageThe place of objects in an image Such information would be extremely useful Such information would be extremely useful
for computer vision researchfor computer vision research
dog
man
2020
The Revealer clicks on parts of the image and shows them to the Guesser.
The Guesser guesses:•Flower•Petal•Butterfly
Server: Correct, Butterfly
2121
HintsHintsThe label “car” is ambiguous --
this is “car”
this is also “car”
The hints help distinguish the manner in which the label “car” appears:
this is the object “car”
this is the text “car”
2222
VerbosityVerbosityCollect common-sense factsCollect common-sense facts Water quenches thirstWater quenches thirst Sky is blueSky is blue Lions eat meatLions eat meat
We as human know hundreds of millions We as human know hundreds of millions common sense factscommon sense facts Computers do not knowComputers do not know
If know, potentially make them more intelligent If know, potentially make them more intelligent (e.g. search better)(e.g. search better)
2323
Common sense fact samplesCommon sense fact samples
MilkMilk
• It is liquid
• It is white
• it has lactose
• cereal is eaten with it
2424
VerbosityVerbosity
Narrator
MILK
is typically near cereal
is a liquid
Guesser
2525
VerbosityVerbosity
Narrator
MILK
Guesser
is typically near cereal
is a liquid
MILK
2626
VerbosityVerbosity
Narrator
Object
Common sense facts about the object
Guesser
2727
VerbosityVerbosity
Narrator
ObjectGuesser
Common sense facts about the object
Object
2828
TemplatesTemplates• ___ is a kind of ___. Allows for hierarchical categorization.• ___ is used for ___. Provides information about the purpose of a word.• ___ is typically near/in/on ___ (three templates). Provide spatial data.• ___ is the opposite of ___ / ___ is related to ___ (two templates). Provide data about basic relations between words.• ___. In the game, this is a “wildcard” that collects related words.
2929
Symmetric vs. AsymmetricSymmetric vs. AsymmetricVerbosity is a asymmetric game, whereas ESP game is a symmetric Verbosity is a asymmetric game, whereas ESP game is a symmetric game.game.
Symmetric games: constraint is number of outputs per inputSymmetric games: constraint is number of outputs per input
Asymmetric games: constraint is number of inputs that produces the Asymmetric games: constraint is number of inputs that produces the same outputsame output
3030
Possible game to build an ontologyPossible game to build an ontology
Several game should work togetherSeveral game should work together Images come from ESP gameImages come from ESP game Not always: only those images are selected Not always: only those images are selected
which have one object in itwhich have one object in it i.e. car, bike, monitor, mouse, house i.e. car, bike, monitor, mouse, house
These images are input to next game which These images are input to next game which tries to catch the properties of objectstries to catch the properties of objects
car has colour, car has wheels, car has car has colour, car has wheels, car has manufacture, car has owner, car has building year, manufacture, car has owner, car has building year, etc.etc.
3131
Possible game to build an ontologyPossible game to build an ontology
Cardinality will be caught by templates, as Cardinality will be caught by templates, as soon as we have properties.soon as we have properties. Car has four wheelsCar has four wheels Car has one plaqueCar has one plaque
These sentences will be transferred to These sentences will be transferred to OWL representation using a mediator. OWL representation using a mediator. The more pairs play the game, the more The more pairs play the game, the more complex the ontology will becomplex the ontology will be
3232
Contact me if you are interested to work on itContact me if you are interested to work on it
3333
This page intentionally left blankThis page intentionally left blank
3434
Guess what!Guess what!
It has usually It has usually fourfour wheels wheelsIt has usually It has usually oneone seat seatIt is kind of vehicleIt is kind of vehicleIt operates with human powerIt operates with human powerIt operates with batteriesIt operates with batteriesIt has a break systemIt has a break systemIt is a kind of chairIt is a kind of chair
3535
AnswerAnswer
3636
ConclusionConclusion
Games are enjoyable!Games are enjoyable!More than 9 billion Human-hours of More than 9 billion Human-hours of solitaire are played each yearsolitaire are played each yearWe may cleverly using humans to solve We may cleverly using humans to solve large-scale problems by designing large-scale problems by designing interesting gamesinteresting gamesMany people play word-guessing games Many people play word-guessing games to improve their Englishto improve their EnglishGo and play to promote science!Go and play to promote science!
3737
ReferencesReferences[1] Verbosity: A Game for Collecting Common-Sense Facts, http://www.cs.cmu.edu/~biglou/Verbosity.pdf [1] Verbosity: A Game for Collecting Common-Sense Facts, http://www.cs.cmu.edu/~biglou/Verbosity.pdf [2] Peekaboom: A Game for Locating Objects in Images, http://www.cs.cmu.edu/~biglou/Peekaboom.pdf[2] Peekaboom: A Game for Locating Objects in Images, http://www.cs.cmu.edu/~biglou/Peekaboom.pdf[3] Labeling Images with a Computer Game, http://www.cs.cmu.edu/~biglou/ESP.pdf [3] Labeling Images with a Computer Game, http://www.cs.cmu.edu/~biglou/ESP.pdf [4] Games with a Purpose, http://www.cs.cmu.edu/~biglou/ieee-gwap.pdf [4] Games with a Purpose, http://www.cs.cmu.edu/~biglou/ieee-gwap.pdf [5] Wikipedia, http://en.wikipedia.org/wiki/Human-based_computation [5] Wikipedia, http://en.wikipedia.org/wiki/Human-based_computation [6] We'll End Spam Within 2 Years, [6] We'll End Spam Within 2 Years,
http://www.connectedhomemag.com/Networking/Articles/Index.cfm?ArticleID=41587http://www.connectedhomemag.com/Networking/Articles/Index.cfm?ArticleID=41587http://news.bbc.co.uk/2/hi/business/3426367.stmhttp://news.bbc.co.uk/2/hi/business/3426367.stm
[7] CAPTCHA, [7] CAPTCHA, http://en.wikipedia.org/wiki/Captchahttp://en.wikipedia.org/wiki/Captcha, , http://www.captcha.nethttp://www.captcha.net
[8] ESP game, [8] ESP game, www.espgame.orgwww.espgame.org[9] Peekaboom game, [9] Peekaboom game, http://www.peekaboom.org/http://www.peekaboom.org/[10] Verbosity game, [10] Verbosity game, www.peekaboom.org/verbosity/www.peekaboom.org/verbosity/[11] Presentation, [11] Presentation, http://isandtcolloq.gsfc.nasa.gov/fall2006/presentations/Ahn.ppthttp://isandtcolloq.gsfc.nasa.gov/fall2006/presentations/Ahn.ppt[12] Presentation, [12] Presentation, www.aladdin.cs.cmu.edu/workshops/lamps05/Slides/Peekaboom.pptwww.aladdin.cs.cmu.edu/workshops/lamps05/Slides/Peekaboom.ppt
3838
Game OverGame Over
p+e+y+m+a+b-b+n p+e+y+m+a+b-b+n dotdot sin(arcsin(lastname)) sin(arcsin(lastname)) @@ deri.org deri.org