2009 International Symposium on Ubiquitous Computing System at Beijing, China Hiroaki Kimura, Sota Matsuzawa, and Tatsuo Nakajima Department of Computer

2009 International Symposium on Ubiquitous Computing System at Beijing, China

Hiroaki Kimura, Sota Matsuzawa, and Tatsuo NakajimaDepartment of Computer Science

Waseda University, Japan

UCS 2009 at Beijing, China 08/24/2009

OutlineOutline

1. Introduction

2. Approach

3. Experiment

4. Discussion

5. Conclusion


Interaction for Ubiquitous Computing

Support different kinds of interaction techniques. Not a pair of mouse and keyboard.

Provide high-quality user experiences.

Ideal interaction

Intuitive

High-learnability

Can be used by everyone

IntroductionIntroduction

The interaction using

Gesture Recognition

is one of the effectual choice.


IntroductionIntroduction

Nintendo Wii

Toshiba Qosmio G50


Introduction - IssueIntroduction - Issue An issue of designing gestural interface

Task ： play video

Shake left to right ？

Draw a triangle ？

Clap hands ？

…

What kind of gesture is the most suited?

For example...

No guideline, UI toolkit, reference point


Offer information for designers and developers.

1. Kind of task can be operated with gesture?

2. Type of gesture appropriated for a particular task?

To achieve our goal Observe subjects dealing with experimental

applications.

Extract some valuable elements.

Introduction - Our goalIntroduction - Our goal


ApproachApproach

We developed an experimental application.

Using a 3-axis accelerometer

Defining gestures by example

Setting corresponding gesture to each task


ApproachApproach

Circle...? Triangle!!!

Hmm...

Define Use re-Define

Subjects can re-define gestures as many times as needed.


(Re-)Define

gesturesCommand

task

Get

feedback

ApproachApproach Transition of subjects’ behavior

Before(at the

beginning)

After(at the end)

(Re-)Define

gesturesCommand

task

Get

feedback


ApproachApproach

IntuitiveHigh-

learnability

It must have some elements which subjects can imagine the gesture from corresponding task.

The common defined gesture is

suited for the taskcan be used for one of the primitive gestures.


ExperimentExperiment Target Applications

Questionnaire results

⇒ Movie Player / Picture Viewer

Analyzed these applications tasks.

Movie Player ： play, stop, pause, fast forward, rewind, volume up, volume down, full screen, download

Picture Viewer ： zoom in, zoom out, rotate, send (mail)

general tasks ： apply (OK), up, down, left, right

19 Tasks


ExperimentExperiment Implementation

Use 3-axis accelerometer for recognition. Assume on mobile phone.

We adopted Sun SPOT(Sun Microsystems, Inc.) as sensor device.


ExperimentExperiment Sensor ： Sun SPOT Data Manager ： Java Presentation ： Adobe AIR


ExperimentExperiment Screenshots


Subjects and procedure

8 university students, 4 days

Each subject has to ...

1.use the applications at least 15 minutes per day.

2.redefine hard to use gesture.

3.keep a log of shapes of redefined gestures.

ExperimentExperiment


ExperimentExperiment Questionnaire to each subject on the web.

(scale of 1-7)

Q1: Could you remember gestures which you defined corresponding to each task?

Q2: Do you think gestural interface is an intuitive way for using these applications?

Q3: Is gesture re-definition by example system necessary?

Q4: How comfortable are the applications?

Q5: Please tell us possible situation using with gestures


DiscussionDiscussion

Found common gestures in ... play, fast forward, rewind, volume up, volume down

zoom in, zoom out, rotate

ok, up, down, left, right

Not found common gestures in ... stop, pause, full screen, download

print, send mail



Fast Forward (movie player)

• Shake right twice ：５

• Imitation of icons

used in audio visual

equipments.



PLAY (movie player)

• Circle ：４

• Shake right ：３

• Simple and short

gesture for

frequently-used tasks



DOWNLOAD (movie player)

• No commonality

• Can not feel intuitive to

control this task with

gesture

• Need to improve design


DiscussionDiscussion Recognizing gestures

( 1 ) simple, ( 2 ) short, ( 3 ) uncompetitive

Show the recognition rate to users.

Learning cost

Users can remember their defined gestures easily.

However difficult to remember infrequently tasks.

Possible object/area using with gestures

Apply to 3D user interfaces

As temporal interaction in public space

User identification

e.g.


ConclusionConclusion

Observe how performed two casual applications using with gestures.

Questionnaire to subjects for acquiring qualitative data.

Common gestures corresponding each tasks, and extracted some valuable points for gestural interface.


Thank you very much.

Distributed and Ubiquitous Computing Lab.

http://www.dcl.info.waseda.ac.jp

Documents

2009 International Symposium on Ubiquitous Computing System at Beijing, China Hiroaki Kimura, Sota Matsuzawa, and Tatsuo Nakajima Department of Computer