Embed Size (px)
DESCRIPTION
We introduce a new algorithm for image segmentation based on crowdsourcing through a game : Ask'nSeek. The game provides information on the objects of an image, under the form of clicks that are either on the object, or on the background. These logs are then used in order to determine the best segmentation for an object among a set of candidates generated by the state-of-the-art CPMC algorithm. We also introduce a simulator that allows the generation of game logs and therefore gives insight about the number of games needed on an image to perform acceptable segmentation. Presented by Amaia Salvador in CrowdMM 2013 (http://crowdmm.org/). More info: https://imatge.upc.edu/web/publications/crowdsourced-object-segmentation-game-0
Citation preview
Crowdsourced object segmentation with a game
Vincent Charvillat
Axel Carlier
Amaia Salvador Aguilera
Xavier Giró i NietoOge Marques
Outline
• Motivation
• Object Segmentation
• Experiments
• Results
• Conclusions
• Ongoing work
2
Motivation
?
3
Motivation
?
4
Semi-Supervised object segmentation
• B. C. Russell, A. Torralba, K. P. Murphy, and W. T. Freeman. Labelme: A database and web-based tool for image annotation. IJCV, 2008
Rough segmentation
5
Semi-Supervised object segmentation
• P. Arbelaez and L. Cohen. Constrained image segmentation from hierarchical boundaries. In CVPR'08, 2008.
• 2) K. McGuinness and N. E. O'Connor. A comparative evaluation of interactive segmentationalgorithms.
6
Semi-Supervised object segmentation
Boring task for users!
7
8
Games with a purpose
• J. Steggink and C. Snoek. Adding semantics to image-region annotations with the name-it-game. Multimedia Systems, 2011.
• L. von Ahn, R. Liu, and M. Blum. Peekaboom: a game for locating objects in images. In CHI'06, 2006.
9
Ask’nSeek
A. Carlier, O. Marques, and V. Charvillat. Ask'nseek: A new game for object detection and labeling. In ECCV'12 Workshops 2012. 10
Motivation
?
11
Outline
• Motivation
• Object Segmentation
• Experiments
• Results
• Conclusions
• Next steps
12
Constrained parametric min-cuts for automatic object segmentation (CPMC)
J. Carreira and C. Sminchisescu. Constrained parametric min-cuts for automatic object segmentation. In CVPR'10, 2010.
13
Constrained parametric min-cuts for automatic object segmentation
14
Motivation
CPMC
15
Outline
• Motivation
• Object Segmentation
• Experiments
• Results
• Conclusions
• Ongoing Work
16
Experiments
How many clicks do we need to achieve a certain quality in the segmentation?
Test the algorithm for a large image dataset
17
Pascal VOC2010
1928 images divided in: Train (964) Validation (964)
18
Problem
Simulator
19
Simulator
• The simulator generates points using the ground truth of the image.
20
Simulator: Location of clicks
S. Goferman, L. Zelnik-Manor, and A. Tal. Context-aware saliency detection. PAMI, 2012.
21
Simulator: Foreground/Background ratio
22
Outline
• Motivation
• Object Segmentation
• Experiments
• Results
• Conclusions
• Ongoing Work
23
Jaccard index
Measure of similarity between the segmentation result and the ground truth mask
24
Results
Using Pascal VOC2010 (Validation)
25
Results
Using Pascal VOC2010 (Validation)
26
Outline
• Motivation
• Object Segmentation
• Experiments
• Results
• Conclusions
• Ongoing Work
27
Conclusions
• Realistic simulator to process large amounts of data.
• Estimation of the expected AVERAGE Jaccard index by clicks.
• Inter-class variance of results.
28
Ongoing Work
29
Ongoing Work
• Image segmentation
• CPMC candidates ● Label propagation throughhierarchical partitions (eg. UCM, BPT…)
● Grabcut + Superpixels
30
Ongoing Work
• Data collection• Awarded with $250 in CrowdMM Competition (ACM MM Barcelona 2013).
• Already more than 1500 games collected with 100 users
More on that in our poster!
31
Questions, suggestions…
Thank you for your attention
• Motivation
• Object Segmentation
• Experiments
• Results
• Conclusions
• Ongoing Work
32
![Dual Embedding Learning for Video Instance Segmentation · Video Object Segmentation. Most approaches [2,11, 8] for video object segmentation task rely on fine-tuning the first-frame](https://img.pdfslide.net/doc/110x75/5f79ec436259e82950543b11/dual-embedding-learning-for-video-instance-segmentation-video-object-segmentation.jpg)
