A Crowdsourcing Based Mobile Image Translation and Knowledge Sharing Service

A Crowd-sourcing Based Mobile Image Translation and

Knowledge Sharing Service

Department of Computer ScienceWaseda University, Tokyo, Japan

1Helsinki Institute for Information Technology

2Eindhoven University of Technology

[email protected]

Yefeng Liu, Vili Lehdonvirta1, Mieke Kleppe2, Todorka Alexandrova, Hiroaki Kimura, Tatsuo Nakajima

mailto:[email protected]


A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus

• Introduction

• Human Mobile Image Translation

• Preliminary Study

• Discussion

• Future Directions

Outline

2


Introduction

A menu board outside a restaurant, Tokyo

3

“...I can’t wear tie here?? Should I

take off my tie?..”


Real World Problem

• Digital pocket translators or online translation services are useless if you don’t know how to input the characters.

4


(Typical) Mobile Image Translation

Image Text

OCR Optical Character Recognition

MT Machine Translation

Irregular fonts or formats, handwriting, etc.

Poor performance

English Text

5


Image Text

Question of the image

TranslatorCommunityOutsourcing

English Text

• Better quality in text recognition and translation

• Human worker can provide richer interpretations and responses in addition to literal answers.

Our Solution: Human Mobile Image Translation

6

Crowdsourcing


Image Based Translator + Mobile Q&A

• NOT only a translator

• But also a knowledge broker that allows users to share high level information pertinent to the situation at hand, e.g.

• advices

• instructions

• suggestions

7


Basic work-flow overview

Open call

etc.

Requester Translators Requester

Scoring

EnglishKanji

Best answer

8


Preliminary study

• A preliminary study and design research aims to

• verify the feasibility of the design

• identify real user requirements and design issues

9


Preliminary study - MethodCollected around a hundred pictures/questions from potential users

Fifteen characteristic cases were selected from the collected images

Interviewed the requesters what kind of answers they were expecting

Assigned questions to invited translators

Interviewed translators for their feedbacks

Compared the results with the requesters’ expectations

10

A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 11

Preliminary Study Cases - Example 1

“...how long do I have to wait?”

Information in the picture is insufficient to answer this question.

However, most of the repliers can still suggest an approximate waiting time according to their life experiences.


Preliminary Study Cases - Example 2

“What are the events between 5th and 8th?”

Poor question text.

Some translators misunderstood the question, thus provided useless answers.


Preliminary Study - Findings (1)

1. Communication between requester and worker.

Better communication Better understanding Better result

2. Question/Answer style

• Short, but clear (e.g clarify to what level of details is wanted);

• Question with choices is better;

• Asking for links (of image/web page/etc) is a good way to lower the difficulty and faster the response time.


Preliminary Study - Findings (2)

3. “Tweet” and “keywords” style answer is preferred

- Many translators use English as 2nd or 3rd language, they oftenface the problem of being unable to explain in long sentence.

a). “Pork, spicy, famous chinese food”

b). “Twice cooked pork (huiguo rou)”

- meaningless if don’t know the name


Discussion (1)

1. Quality of outcome

Misunderstanding between requester and worker strongly affects quality of outcome.

- Requesters may use unclear or too complicate English.- Workers often are not native English speaker.

suggests a single reply can hardly be trusted.

- Human always make mistakes.

- Malicious replies.


Discussion (2)

An additional proofreading phase.

open call

Requester Translators Requester

Scoring

EnglishKanji

Best answerProofreaders


Discussion (3)2. Different user types (user requirements)

Client Users

Short-term stay Long-term stay

Need immediate answer

Need immediate answer

Waitable Waitable

A B C D

may have different preference on the accuracy vs. timeliness trade-off


Future Directions (1)

1. Dynamical task allocation with real time requirement

i. capable for the task

ii. available for the task

In this study case, local context of the requester and background information of the worker is important to determine the capacity.

Not only about if the worker is free, but also involves other factors like expertise, properties of question, etc.

• Task is better be assigned to worker who is:


Future Directions (2)

2. Motivation and Incentive

Social and Intrinsic incentive: game play

A location-based mobile game is designed


Current Status

Early Test

Prototype Implementation

Redesign

Usability Test/On field studyRedesign

Preliminary Study

Design

This paper

(some images here..)

Thanks for your attention!

Distributed & Ubiquitous Computing Lab.Depart. of Computer Science, Waseda University

http://www.dcl.info.waseda.ac.jp/

Yefeng Liu, PhD candidate

[email protected]

http://www.dcl.info.waseda.ac.jp

http://www.dcl.info.waseda.ac.jp

