CrowdSearch: Exploiting Crowds for Accurate Real-time Image Search on Mobile Phones

Original work by Yan, Kumar & GanesanPresented by Shibo Li & Jian Yu

2

Problem Definition

• How to search information?

3

Problem Definition

• Mobile-based search will become more important in the future.– More than 70% of smart phone users perform searches.

• Expected to be more mobile searches than non-mobile searches soon

– Text-based mobile searches are easy as well…

• What about searching images?

4

Problem Definition

• Image search using mobile phones

5

Problem Definition

• Automatic searching

6

Idea

• Image searching based on crowd source.

CrowdSearch Algorithm

7

Challenges

• Automatic image search: – Delay↓, Cost ↓, Accuracy ↓

• People validation image search:– Delay ↑, Cost ↑, Accuracy ↑

8

CrowdSearch Algorithm

Overview

Implementation & Evaluations

Throughts & Criticisms

9

CrowdSearch: Overview

10

CrowdSearch: Overview

11

CrowdSearch: Overview

12

CrowdSearch: Overview

13

CrowdSearch: Overview

14

Challenge: Accuracy

15

Challenge: Accuracy

• Human validation improves accuracy 2-5 times.

• Majority(5) can achieve the highest accuracy up to 95%

So we send each image to 5 people to get the majority feedback.

16

Challenge: Delay & Cost tradeoff

17

Challenge: Delay & Cost tradeoff

• Parallel Scheme

18

Challenge: Delay & Cost tradeoff

• Serial Scheme

19

CrowdSearch: compromised scheme

20

CrowdSearch: compromised scheme

• Prediction requires delay and accuracy models

21

Delay Model

• Statistically, both of the delays follow the exponential distribution.

• Overall delay distribution is the convolution of the acceptance and submission delay.

22

Delay Prediction

23

Accuracy Prediction

24

Decision Engine

25

Overview

Implementation & Evaluations

Throughts & Criticisms

26

Implementation

27

Power Consideration

• Should some image processing occur on the local device or should it be outsourced to the server?

– Use remoteprocessing when WiFi is available.

– Use local processingwhen only 3G is available

28

Evaluation

• Delay model meets the exponential distribution

29

CrowdSearch Performance

CrowdSearch optimized algorithm

30

Overview

Implementation & Evaluations

Throughts & Criticisms

31

Thoughts/Criticism

• Only 1000 images in the backend database.– Would increasing the number of automated search images increase

total task time in a significant way?

• The evaluation only based on 4 categories.– Buildings, Books, Flowers and Faces

• Suggestion:• Internet database• Let the user to choose the categories

• Too many distractions in a single image

32

Thoughts/Criticism

• Too many disturbances in a single image

33

Q&A

Thank you!