Upload
others
View
13
Download
0
Embed Size (px)
Citation preview
Motivation• How-tovideosarevaluableforteaching
• Searchingforhow-tovideosisdifficult.Usersoften
havetoskimmanyvideostofindtherightone
• Havingvideoannotationswouldimprovesearching,
browsing,andindexingvideos
ProblemGenerating video annotations is difficult
• Manuallyannotatingeachvideoiscumbersome
• Usingcomputervisionrequirestrainingdata
Research QuestionCan a crowdsourcing method be efficient and scalable enough to generate annotations for how-to videos?
• UsingPhotoshophow-tovideosasourexample
domain
• TestingourproposedmethodonAmazon
MechanicalTurk
Online Experiment
Future Work• Improveanalytictoolsfortheworkflow
•• Pipelinesubtaskstogetherautomatically
• Reducenumberofcrowdsourcingtasks/video
Acknowledgements• FundingfromQuantaComputer,Inc
• EECSSuperUROPprogram
Annotation Accuracy from a Mechanical Turk Experiment
• Thesizeofthevideoplayermadeithardforworkerstoseewhatisgoingon
• Workersstillhadtroubleknowingwheninstructorclicksontool
• Allvideosarelabeledcorrectlywithmajorityvoting
• Workershadtroubleselectingthecorrectlabelifmultiplelabelsseemsimilar(“Gradient”vs.“GradientMap”)
• Mostoftheincorrectbeforesandafterswereduetobadvideoclips(choosingbeforeimageofanincorrecttoolinclip)
• Someerrorswereduetodialogobstructions
Capturing Time Marks Labeling Tools Used Capturing Before/After Images
Procedure that
generates annotations
Task 2: Label Tools Used
Theworkerlabelsthetoolusedina20s
clipofahow-tovideowithdropdown
menus.
Task 3: Capture Before/After Images
Theworkerwatchesa20sclipofahow-
tovideoandcapturesthebeforeandafter
imagesforatooluse.
Generating Annotations for How-to Videos Using CrowdsourcingPhu Nguyen, Juho Kim, Rob Miller - MIT CSAIL
How-to Video Annotation Generator Annotations Interface
Task 1: Capture Time Marks
Theworkerwatchesa60sclipofahow-to
videoandclicksthe“ToolClicked”button
whenevertheinstructorclicksonatool.