Lecture Notes in Computer Science 5611...Lecture Notes in Computer Science 5611 Commenced Publication in 1973 Founding and Former Series Editors: Gerhard Goos, Juris Hartmanis, and

Lecture Notes in Computer Science 5611Commenced Publication in 1973Founding and Former Series Editors:Gerhard Goos, Juris Hartmanis, and Jan van Leeuwen

Editorial Board

David HutchisonLancaster University, UK

Takeo KanadeCarnegie Mellon University, Pittsburgh, PA, USA

Josef KittlerUniversity of Surrey, Guildford, UK

Jon M. KleinbergCornell University, Ithaca, NY, USA

Alfred KobsaUniversity of California, Irvine, CA, USA

Friedemann MatternETH Zurich, Switzerland

John C. MitchellStanford University, CA, USA

Moni NaorWeizmann Institute of Science, Rehovot, Israel

Oscar NierstraszUniversity of Bern, Switzerland

C. Pandu RanganIndian Institute of Technology, Madras, India

Bernhard SteffenUniversity of Dortmund, Germany

Madhu SudanMassachusetts Institute of Technology, MA, USA

Demetri TerzopoulosUniversity of California, Los Angeles, CA, USA

Doug TygarUniversity of California, Berkeley, CA, USA

Gerhard WeikumMax-Planck Institute of Computer Science, Saarbruecken, Germany

Julie A. Jacko (Ed.)

Human-ComputerInteraction

Novel InteractionMethods and Techniques

13th International Conference, HCI International 2009San Diego, CA, USA, July 19-24, 2009Proceedings, Part II

13

Volume Editor

Julie A. JackoUniversity of MinnesotaInstitute of Health InformaticsMMC 912, 420 Delaware Street S.E., Minneapolis, MN 55455, USAE-mail: [email protected]

Library of Congress Control Number: 2009929048

CR Subject Classification (1998): H.5, I.3, I.7.5, I.5, I.2.10

LNCS Sublibrary: SL 3 – Information Systems and Application, incl. Internet/Weband HCI

ISSN 0302-9743ISBN-10 3-642-02576-5 Springer Berlin Heidelberg New YorkISBN-13 978-3-642-02576-1 Springer Berlin Heidelberg New York

This work is subject to copyright. All rights are reserved, whether the whole or part of the material isconcerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting,reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publicationor parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965,in its current version, and permission for use must always be obtained from Springer. Violations are liableto prosecution under the German Copyright Law.

springer.com

© Springer-Verlag Berlin Heidelberg 2009Printed in Germany

Typesetting: Camera-ready by author, data conversion by Scientific Publishing Services, Chennai, IndiaPrinted on acid-free paper SPIN: 12707218 06/3180 5 4 3 2 1 0

Foreword

The 13th International Conference on Human–Computer Interaction, HCI Interna-tional 2009, was held in San Diego, California, USA, July 19–24, 2009, jointly with the Symposium on Human Interface (Japan) 2009, the 8th International Conference on Engineering Psychology and Cognitive Ergonomics, the 5th International Conference on Universal Access in Human–Computer Interaction, the Third International Confer-ence on Virtual and Mixed Reality, the Third International Conference on Internation-alization, Design and Global Development, the Third International Conference on Online Communities and Social Computing, the 5th International Conference on Augmented Cognition, the Second International Conference on Digital Human Model-ing, and the First International Conference on Human Centered Design.

A total of 4,348 individuals from academia, research institutes, industry and govern-mental agencies from 73 countries submitted contributions, and 1,397 papers that were judged to be of high scientific quality were included in the program. These papers ad-dress the latest research and development efforts and highlight the human aspects of design and use of computing systems. The papers accepted for presentation thoroughly cover the entire field of human–computer interaction, addressing major advances in the knowledge and effective use of computers in a variety of application areas.

This volume, edited by Julie A. Jacko, contains papers in the thematic area of Human–Computer Interaction, addressing the following major topics:

• Multimodal User Interfaces • Gesture, Eye Movements and Expression Recognition • Human–Robot Interaction • Touch and Pen-Based Interaction • Brain Interfaces • Language, Voice, Sound and Communication • Visualization, Images and Pictures

The remaining volumes of the HCI International 2009 proceedings are:

• Volume 1, LNCS 5610, Human–Computer Interaction––New Trends (Part I), edited by Julie A. Jacko

• Volume 3, LNCS 5612, Human–Computer Interaction––Ambient, Ubiqui-tous and Intelligent Interaction (Part III), edited by Julie A. Jacko

• Volume 4, LNCS 5613, Human–Computer Interaction––Interacting in Vari-ous Application Domains (Part IV), edited by Julie A. Jacko

• Volume 5, LNCS 5614, Universal Access in Human–Computer Interaction––Addressing Diversity (Part I), edited by Constantine Stephanidis

• Volume 6, LNCS 5615, Universal Access in Human–Computer Interaction––Intelligent and Ubiquitous Interaction Environments (Part II), edited by Constantine Stephanidis

VI Foreword

• Volume 7, LNCS 5616, Universal Access in Human–Computer Interaction––Applications and Services (Part III), edited by Constantine Stephanidis

• Volume 8, LNCS 5617, Human Interface and the Management of Information––Designing Information Environments (Part I), edited by Mi-chael J. Smith and Gavriel Salvendy

• Volume 9, LNCS 5618, Human Interface and the Management of Informa-tion - Information and Interaction (Part II), edited by Gavriel Salvendy and Michael J. Smith

• Volume 10, LNCS 5619, Human Centered Design, edited by Masaaki Kurosu • Volume 11, LNCS 5620, Digital Human Modeling, edited by Vincent G.

Duffy • Volume 12, LNCS 5621, Online Communities and Social Computing, edited

by A. Ant Ozok and Panayiotis Zaphiris • Volume 13, LNCS 5622, Virtual and Mixed Reality, edited by Randall

Shumaker • Volume 14, LNCS 5623, Internationalization, Design and Global Develop-

ment, edited by Nuray Aykin • Volume 15, LNCS 5624, Ergonomics and Health Aspects of Work with

Computers, edited by Ben-Tzion Karsh • Volume 16, LNAI 5638, The Foundations of Augmented Cognition: Neuro-

ergonomics and Operational Neuroscience, edited by Dylan Schmorrow, Ivy Estabrooke and Marc Grootjen

• Volume 17, LNAI 5639, Engineering Psychology and Cognitive Ergonomics, edited by Don Harris

I would like to thank the Program Chairs and the members of the Program Boards

of all thematic areas, listed below, for their contribution to the highest scientific quality and the overall success of HCI International 2009.

Ergonomics and Health Aspects of Work with Computers

Program Chair: Ben-Tzion Karsh

Arne Aarås, Norway Pascale Carayon, USA Barbara G.F. Cohen, USA Wolfgang Friesdorf, Germany John Gosbee, USA Martin Helander, Singapore Ed Israelski, USA Waldemar Karwowski, USA Peter Kern, Germany Danuta Koradecka, Poland Kari Lindström, Finland

Holger Luczak, Germany Aura C. Matias, Philippines Kyung (Ken) Park, Korea Michelle M. Robertson, USA Michelle L. Rogers, USA Steven L. Sauter, USA Dominique L. Scapin, France Naomi Swanson, USA Peter Vink, The Netherlands John Wilson, UK Teresa Zayas-Cabán, USA

Foreword VII

Human Interface and the Management of Information

Program Chair: Michael J. Smith

Gunilla Bradley, Sweden Hans-Jörg Bullinger, Germany Alan Chan, Hong Kong Klaus-Peter Fähnrich, Germany Michitaka Hirose, Japan Jhilmil Jain, USA Yasufumi Kume, Japan Mark Lehto, USA Fiona Fui-Hoon Nah, USA Shogo Nishida, Japan Robert Proctor, USA Youngho Rhee, Korea

Anxo Cereijo Roibás, UK Katsunori Shimohara, Japan Dieter Spath, Germany Tsutomu Tabe, Japan Alvaro D. Taveira, USA Kim-Phuong L. Vu, USA Tomio Watanabe, Japan Sakae Yamamoto, Japan Hidekazu Yoshikawa, Japan Li Zheng, P.R. China Bernhard Zimolong, Germany

Human–Computer Interaction

Program Chair: Julie A. Jacko

Sebastiano Bagnara, Italy Sherry Y. Chen, UK Marvin J. Dainoff, USA Jianming Dong, USA John Eklund, Australia Xiaowen Fang, USA Ayse Gurses, USA Vicki L. Hanson, UK Sheue-Ling Hwang, Taiwan Wonil Hwang, Korea Yong Gu Ji, Korea Steven Landry, USA

Gitte Lindgaard, Canada Chen Ling, USA Yan Liu, USA Chang S. Nam, USA Celestine A. Ntuen, USA Philippe Palanque, France P.L. Patrick Rau, P.R. China Ling Rothrock, USA Guangfeng Song, USA Steffen Staab, Germany Wan Chul Yoon, Korea Wenli Zhu, P.R. China

Engineering Psychology and Cognitive Ergonomics

Program Chair: Don Harris

Guy A. Boy, USA John Huddlestone, UK Kenji Itoh, Japan Hung-Sying Jing, Taiwan Ron Laughery, USA Wen-Chin Li, Taiwan James T. Luxhøj, USA

Nicolas Marmaras, Greece Sundaram Narayanan, USA Mark A. Neerincx, The Netherlands Jan M. Noyes, UK Kjell Ohlsson, Sweden Axel Schulte, Germany Sarah C. Sharples, UK

VIII Foreword

Neville A. Stanton, UK Xianghong Sun, P.R. China Andrew Thatcher, South Africa

Matthew J.W. Thomas, Australia Mark Young, UK

Universal Access in Human–Computer Interaction

Program Chair: Constantine Stephanidis

Julio Abascal, Spain Ray Adams, UK Elisabeth André, Germany Margherita Antona, Greece Chieko Asakawa, Japan Christian Bühler, Germany Noelle Carbonell, France Jerzy Charytonowicz, Poland Pier Luigi Emiliani, Italy Michael Fairhurst, UK Dimitris Grammenos, Greece Andreas Holzinger, Austria Arthur I. Karshmer, USA Simeon Keates, Denmark Georgios Kouroupetroglou, Greece Sri Kurniawan, USA

Patrick M. Langdon, UK Seongil Lee, Korea Zhengjie Liu, P.R. China Klaus Miesenberger, Austria Helen Petrie, UK Michael Pieper, Germany Anthony Savidis, Greece Andrew Sears, USA Christian Stary, Austria Hirotada Ueda, Japan Jean Vanderdonckt, Belgium Gregg C. Vanderheiden, USA Gerhard Weber, Germany Harald Weber, Germany Toshiki Yamaoka, Japan Panayiotis Zaphiris, UK

Virtual and Mixed Reality

Program Chair: Randall Shumaker

Pat Banerjee, USA Mark Billinghurst, New Zealand Charles E. Hughes, USA David Kaber, USA Hirokazu Kato, Japan Robert S. Kennedy, USA Young J. Kim, Korea Ben Lawson, USA

Gordon M. Mair, UK Miguel A. Otaduy, Switzerland David Pratt, UK Albert “Skip” Rizzo, USA Lawrence Rosenblum, USA Dieter Schmalstieg, Austria Dylan Schmorrow, USA Mark Wiederhold, USA

Internationalization, Design and Global Development

Program Chair: Nuray Aykin

Michael L. Best, USA Ram Bishu, USA Alan Chan, Hong Kong Andy M. Dearden, UK

Susan M. Dray, USA Vanessa Evers, The Netherlands Paul Fu, USA Emilie Gould, USA

Foreword IX

Sung H. Han, Korea Veikko Ikonen, Finland Esin Kiris, USA Masaaki Kurosu, Japan Apala Lahiri Chavan, USA James R. Lewis, USA Ann Light, UK James J.W. Lin, USA Rungtai Lin, Taiwan Zhengjie Liu, P.R. China Aaron Marcus, USA Allen E. Milewski, USA

Elizabeth D. Mynatt, USA Oguzhan Ozcan, Turkey Girish Prabhu, India Kerstin Röse, Germany Eunice Ratna Sari, Indonesia Supriya Singh, Australia Christian Sturm, Spain Adi Tedjasaputra, Singapore Kentaro Toyama, India Alvin W. Yeo, Malaysia Chen Zhao, P.R. China Wei Zhou, P.R. China

Online Communities and Social Computing

Program Chairs: A. Ant Ozok, Panayiotis Zaphiris

Chadia N. Abras, USA Chee Siang Ang, UK Amy Bruckman, USA Peter Day, UK Fiorella De Cindio, Italy Michael Gurstein, Canada Tom Horan, USA Anita Komlodi, USA Piet A.M. Kommers, The Netherlands Jonathan Lazar, USA Stefanie Lindstaedt, Austria

Gabriele Meiselwitz, USA Hideyuki Nakanishi, Japan Anthony F. Norcio, USA Jennifer Preece, USA Elaine M. Raybourn, USA Douglas Schuler, USA Gilson Schwartz, Brazil Sergei Stafeev, Russia Charalambos Vrasidas, Cyprus Cheng-Yen Wang, Taiwan

Augmented Cognition

Program Chair: Dylan D. Schmorrow

Andy Bellenkes, USA Andrew Belyavin, UK Joseph Cohn, USA Martha E. Crosby, USA Tjerk de Greef, The Netherlands Blair Dickson, UK Traci Downs, USA Julie Drexler, USA Ivy Estabrooke, USA Cali Fidopiastis, USA Chris Forsythe, USA Wai Tat Fu, USA Henry Girolamo, USA

Marc Grootjen, The Netherlands Taro Kanno, Japan Wilhelm E. Kincses, Germany David Kobus, USA Santosh Mathan, USA Rob Matthews, Australia Dennis McBride, USA Robert McCann, USA Jeff Morrison, USA Eric Muth, USA Mark A. Neerincx, The Netherlands Denise Nicholson, USA Glenn Osga, USA

X Foreword

Dennis Proffitt, USA Leah Reeves, USA Mike Russo, USA Kay Stanney, USA Roy Stripling, USA Mike Swetnam, USA Rob Taylor, UK

Maria L.Thomas, USA Peter-Paul van Maanen, The Netherlands Karl van Orden, USA Roman Vilimek, Germany Glenn Wilson, USA Thorsten Zander, Germany

Digital Human Modeling

Program Chair: Vincent G. Duffy

Karim Abdel-Malek, USA Thomas J. Armstrong, USA Norm Badler, USA Kathryn Cormican, Ireland Afzal Godil, USA Ravindra Goonetilleke, Hong Kong Anand Gramopadhye, USA Sung H. Han, Korea Lars Hanson, Sweden Pheng Ann Heng, Hong Kong Tianzi Jiang, P.R. China

Kang Li, USA Zhizhong Li, P.R. China Timo J. Määttä, Finland Woojin Park, USA Matthew Parkinson, USA Jim Potvin, Canada Rajesh Subramanian, USA Xuguang Wang, France John F. Wiechel, USA Jingzhou (James) Yang, USA Xiu-gan Yuan, P.R. China

Human Centered Design

Program Chair: Masaaki Kurosu

Gerhard Fischer, USA Tom Gross, Germany Naotake Hirasawa, Japan Yasuhiro Horibe, Japan Minna Isomursu, Finland Mitsuhiko Karashima, Japan Tadashi Kobayashi, Japan

Kun-Pyo Lee, Korea Loïc Martínez-Normand, Spain Dominique L. Scapin, France Haruhiko Urokohara, Japan Gerrit C. van der Veer, The Netherlands Kazuhiko Yamazaki, Japan

In addition to the members of the Program Boards above, I also wish to thank the

following volunteer external reviewers: Gavin Lew from the USA, Daniel Su from the UK, and Ilia Adami, Ioannis Basdekis, Yannis Georgalis, Panagiotis Karampelas, Iosif Klironomos, Alexandros Mourouzis, and Stavroula Ntoa from Greece.

This conference could not have been possible without the continuous support and advice of the Conference Scientific Advisor, Prof. Gavriel Salvendy, as well as the dedicated work and outstanding efforts of the Communications Chair and Editor of HCI International News, Abbas Moallem.

Foreword XI

I would also like to thank for their contribution toward the organization of the HCI International 2009 conference the members of the Human–Computer Interaction Laboratory of ICS-FORTH, and in particular Margherita Antona, George Paparoulis, Maria Pitsoulaki, Stavroula Ntoa, and Maria Bouhli.

Constantine Stephanidis

HCI International 2011

The 14th International Conference on Human–Computer Interaction, HCI Interna-tional 2011, will be held jointly with the affiliated conferences in the summer of 2011. It will cover a broad spectrum of themes related to human–computer interaction, in-cluding theoretical issues, methods, tools, processes and case studies in HCI design, as well as novel interaction techniques, interfaces and applications. The proceedings will be published by Springer. More information about the topics, as well as the venue and dates of the conference, will be announced through the HCI International Conference series website: http://www.hci-international.org/

General Chair Professor Constantine Stephanidis

University of Crete and ICS-FORTH Heraklion, Crete, Greece

Email: [email protected]

Table of Contents

Part I: Multimodal User Interfaces

Using Acoustic Landscapes for the Evaluation of Multimodal MobileApplications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Wolfgang Beinhauer and Cornelia Hipp

Modeling and Using Salience in Multimodal Interaction Systems . . . . . . . 12Ali Choumane and Jacques Siroux

Exploring Multimodal Interaction in Collaborative Settings . . . . . . . . . . . 19Lúıs Duarte, Marco de Sá, and Lúıs Carriço

Towards a Multidimensional Approach for the Evaluation ofMultimodal Application User Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

José E.R. de Queiroz, Joseana M. Fechine, Ana E.V. Barbosa, andDanilo de S. Ferreira

Multimodal Shopping Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39Jhilmil Jain, Riddhiman Ghosh, and Mohamed Dekhil

Value of Using Multimodal Data in HCI Methodologies . . . . . . . . . . . . . . . 48Jhilmil Jain

Effective Combination of Haptic, Auditory and Visual InformationFeedback in Operation Feeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

Keiko Kasamatsu, Tadahiro Minami, Kazuki Izumi, andHideo Jinguh

Multi-modal Interface in Multi-Display Environment for Multi-users . . . . 66Yoshifumi Kitamura, Satoshi Sakurai, Tokuo Yamaguchi,Ryo Fukazawa, Yuichi Itoh, and Fumio Kishino

Reliable Evaluation of Multimodal Dialogue Systems . . . . . . . . . . . . . . . . . 75Florian Metze, Ina Wechsung, Stefan Schaffer, Julia Seebode, andSebastian Möller

Evaluation Proposal of a Framework for the Integration of MultimodalInteraction in 3D Worlds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

Héctor Olmedo-Rodŕıguez, David Escudero-Mancebo, andValent́ın Cardeñoso-Payo

Building a Practical Multimodal System with a Multimodal FusionModule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

Yong Sun, Yu (David) Shi, Fang Chen, and Vera Chung

XVI Table of Contents

Modeling Multimodal Interaction for Performance Evaluation . . . . . . . . . 103Emile Verdurand, Gilles Coppin, Franck Poirier, andOlivier Grisvard

Usability Evaluation of Multimodal Interfaces: Is the Whole the Sumof Its Parts? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

Ina Wechsung, Klaus-Peter Engelbrecht, Stefan Schaffer,Julia Seebode, Florian Metze, and Sebastian Möller

Part II: Gesture, Eyes Movement and ExpressionRecognition

An Open Source Framework for Real-Time, Incremental, Static andDynamic Hand Gesture Learning and Recognition . . . . . . . . . . . . . . . . . . . . 123

Todd C. Alexander, Hassan S. Ahmed, andGeorgios C. Anagnostopoulos

Gesture-Controlled User Input to Complete Questionnaires onWrist-Worn Watches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

Oliver Amft, Roman Amstutz, Asim Smailagic, Dan Siewiorek, andGerhard Tröster

UbiGesture: Customizing and Profiling Hand Gestures in UbiquitousEnvironment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

Ayman Atia, Shin Takahashi, Kazuo Misue, and Jiro Tanaka

The Gestural Input System for Living Room Digital Devices . . . . . . . . . . 151Wen-Shan Chang and Fong-Gong Wu

Touchless Interaction-Novel Chances and Challenges . . . . . . . . . . . . . . . . . . 161René de la Barré, Paul Chojecki, Ulrich Leiner,Lothar Mühlbach, and Detlef Ruschin

Did I Get It Right: Head Gestures Analysis for Human-MachineInteractions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

Jürgen Gast, Alexander Bannat, Tobias Rehrl, Gerhard Rigoll,Frank Wallhoff, Christoph Mayer, and Bernd Radig

Interactive Demonstration of Pointing Gestures for Virtual Trainers . . . . 178Yazhou Huang and Marcelo Kallmann

Anthropometric Facial Emotion Recognition . . . . . . . . . . . . . . . . . . . . . . . . . 188Julia Jarkiewicz, Rafa�l Kocielnik, and Krzysztof Marasek

Real-Time Face Tracking and Recognition Based on Particle Filteringand AdaBoosting Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198

Chin-Shyurng Fahn, Ming-Jui Kuo, and Kai-Yi Wang

Table of Contents XVII

A Real-Time Hand Interaction System for Image Sensor BasedInterface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208

SeIn Lee, Jonghoon Seo, Soon-bum Lim, Yoon-Chul Choy, andTackDon Han

Gesture-Based Interface for Connection and Control of Multi-device ina Tabletop Display Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216

Hyunglae Lee, Heeseok Jeong, Joongho Lee, Ki-Won Yeom, andJi-Hyung Park

Shadow Awareness: Bodily Expression Supporting System with Use ofArtificial Shadow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226

Yoshiyuki Miwa, Shiorh Itai, Takabumi Watanabe, Koji Iida, andHiroko Nishi

An Approach to Glove-Based Gesture Recognition . . . . . . . . . . . . . . . . . . . 236Farid Parvini, Dennis McLeod, Cyrus Shahabi, Bahareh Navai,Baharak Zali, and Shahram Ghandeharizadeh

cfHMI: A Novel Contact-Free Human-Machine Interface . . . . . . . . . . . . . . 246Tobias Rehrl, Alexander Bannat, Jürgen Gast, Gerhard Rigoll, andFrank Wallhoff

Fly! Little Me: Localization of Body-Image within Reduced-Self . . . . . . . . 255Tatsuya Saito and Masahiko Sato

New Interaction Concepts by Using the Wii Remote . . . . . . . . . . . . . . . . . . 261Michael Schreiber, Margeritta von Wilamowitz-Moellendorff, andRalph Bruder

Wireless Data Glove for Gesture-Based Robotic Control . . . . . . . . . . . . . . 271Nghia X. Tran, Hoa Phan, Vince V. Dinh, Jeffrey Ellen, Bryan Berg,Jason Lum, Eldridge Alcantara, Mike Bruch, Marion G. Ceruti,Charles Kao, Daniel Garcia, Sunny Fugate, and LorRaine Duffy

PALMbit-Silhouette:A User Interface by SuperimposingPalm-Silhouette to Access Wall Displays . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281

Goshiro Yamamoto, Huichuan Xu, Kazuto Ikeda, and Kosuke Sato

Potential Limitations of Multi-touch Gesture Vocabulary:Differentiation, Adoption, Fatigue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291

Wendy Yee

Part III: Human-Robot Interaction

A Multimodal Human-Robot-Interaction Scenario: Working togetherwith an Industrial Robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303

Alexander Bannat, Jürgen Gast, Tobias Rehrl, Wolfgang Rösel,Gerhard Rigoll, and Frank Wallhoff

XVIII Table of Contents

Robotic Home Assistant Care-O-bot R© 3 – Product Vision andInnovation Platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312

Birgit Graf, Christopher Parlitz, and Martin Hägele

Designing Emotional and Interactive Behaviors for an EntertainmentRobot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321

Yo Chan Kim, Hyuk Tae Kwon, Wan Chul Yoon, andJong Cheol Kim

Emotions and Messages in Simple Robot Gestures . . . . . . . . . . . . . . . . . . . 331Jamy Li, Mark Chignell, Sachi Mizobuchi, and Michiaki Yasumura

Life with a Robot Companion: Video Analysis of 16-Days of Interactionwith a Home Robot in a “Ubiquitous Home” Environment . . . . . . . . . . . . 341

Naoko Matsumoto, Hirotada Ueda, Tatsuya Yamazaki, andHajime Murai

Impression Evaluation of a Conversational Robot Playing RAKUGO . . . 351Akihiro Ogino, Noritaka Moriya, Park Seung-Joon, andHirotada Ueda

Performance Assessment of Swarm Robots . . . . . . . . . . . . . . . . . . . . . . . . . . 361Ercan Oztemel, Cemalettin Kubat, Ozer Uygun, Tuba Canvar,Tulay Korkusuz, Vinesh Raja, and Anthony Soroka

A Robotic Introducer Agent Based on Adaptive Embodied EntrainmentControl . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368

Mutsuo Sano, Kenzaburo Miyawaki, Ryohei Sasama,Tomoharu Yamaguchi, and Keiji Yamada

Robot Helps Teachers for Education of the C Language Beginners . . . . . . 377Haruaki Tamada, Akihiro Ogino, and Hirotada Ueda

An Interactive Robot Butler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385Yeow Kee Tan, Dilip Limbu Kumar, Ridong Jiang, Liyuan Li,Kah Eng Hoe, Xinguo Yu, Li Dong, Chern Yuen Wong, andHaizhou Li

Part IV: Touch and Pen-Based Interaction

A Study on Fundamental Information Transmission Characteristics ofan Air-Jet Driven Tactile Display . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397

Takafumi Asao, Hiroaki Hayashi, Masayoshi Hayashi,Kentaro Kotani, and Ken Horii

VersaPatch: A Low Cost 2.5D Capacitive Touch Sensor . . . . . . . . . . . . . . . 407Ray Bittner and Mike Sinclair

Table of Contents XIX

From Implicit to Touching Interaction by Identification Technologies:Towards Tagging Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417

Jose Bravo, Ramon Hervas, Carmen Fuentes, Vladimir Villarreal,Gabriel Chavira, Salvador Nava, Jesus Fontecha, Gregorio Casero,Rocio Peña, and Marcos Vergara

VTouch: A Vision-Base Dual Finger Touched Inputs for LargeDisplays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 426

Ching-Han Chen and Cun-Xian Nian

Overview of Meta-analyses Investigating Vibrotactile versus VisualDisplay Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435

Linda R. Elliott, Michael D. Coovert, and Elizabeth S. Redden

Experimental Study about Effect of Thermal Information Presentationto Mouse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444

Shigeyoshi Iizuka and Sakae Yamamoto

Preliminary Study on Vibrotactile Messaging for Sharing BriefInformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451

Teruaki Ito

Orientation Responsive Touch Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . 461Jinwook Kim, Jong-gil Ahn, and Heedong Ko

Representation of Velocity Information by Using Tactile ApparentMotion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 470

Kentaro Kotani, Toru Yu, Takafumi Asao, and Ken Horii

Tactile Spatial Cognition by the Palm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479Misa Grace Kwok

A Study on Effective Tactile Feeling of Control Panels for ElectricalAppliances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 486

Miwa Nakanishi, Yusaku Okada, and Sakae Yamamoto

Facilitating the Design of Vibration for Handheld Devices . . . . . . . . . . . . . 496Taezoon Park, Jihong Hwang, and Wonil Hwang

Interaction Technique for a Pen-Based Interface Using FingerMotions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503

Yu Suzuki, Kazuo Misue, and Jiro Tanaka

A Basic Study of Sensory Characteristics toward Interaction with aBox-Shaped Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 513

Noriko Suzuki, Tosirou Kamiya, Shunsuke Yoshida, and Sumio Yano

TACTUS: A Hardware and Software Testbed for Research inMulti-Touch Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523

Paul Varcholik, Joseph J. Laviola Jr., and Denise Nicholson

XX Table of Contents

Low Cost Flexible Wrist Touch UI Solution . . . . . . . . . . . . . . . . . . . . . . . . . 533Bin Wang, Chenguang Cai, Emilia Koskinen, Tang Zhenqi,Huayu Cao, Leon Xu, and Antti O. Salo

Grasping Interface with Photo Sensor for a Musical Instrument . . . . . . . . 542Tomoyuki Yamaguchi and Shuji Hashimoto

Part V: Brain Interfaces

Ensemble SWLDA Classifiers for the P300 Speller . . . . . . . . . . . . . . . . . . . . 551Garett D. Johnson and Dean J. Krusienski

The I of BCIs: Next Generation Interfaces for Brain–ComputerInterface Systems That Adapt to Individual Users . . . . . . . . . . . . . . . . . . . . 558

Brendan Allison

Mind-Mirror: EEG-Guided Image Evolution . . . . . . . . . . . . . . . . . . . . . . . . . 569Nima Bigdely Shamlo and Scott Makeig

BEXPLORER: Computer and Communication Control Using EEG . . . . 579Mina Mikhail, Marian Abdel-Shahid, Mina Guirguis, Nadine Shehad,Baher Soliman, and Khaled El-Ayat

Continuous Control Paradigms for Direct Brain Interfaces . . . . . . . . . . . . . 588Melody Moore Jackson, Rudolph Mappus, Evan Barba,Sadir Hussein, Girish Venkatesh, Chetna Shastry, andAmichai Israeli

Constructive Adaptive User Interfaces Based on Brain Waves . . . . . . . . . . 596Masayuki Numao, Takayuki Nishikawa, Toshihito Sugimoto,Satoshi Kurihara, and Roberto Legaspi

Development of Symbiotic Brain-Machine Interfaces Using aNeurophysiology Cyberworkstation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 606

Justin C. Sanchez, Renato Figueiredo, Jose Fortes, andJose C. Principe

Sensor Modalities for Brain-Computer Interfacing . . . . . . . . . . . . . . . . . . . . 616Gerwin Schalk

A Novel Dry Electrode for Brain-Computer Interface . . . . . . . . . . . . . . . . . 623Eric W. Sellers, Peter Turner, William A. Sarnacki,Tobin McManus, Theresa M. Vaughan, and Robert Matthews

Effect of Mental Training on BCI Performance . . . . . . . . . . . . . . . . . . . . . . . 632Lee-Fan Tan, Ashok Jansari, Shian-Ling Keng, and Sing-Yau Goh

The Research on EEG Coherence around Central Area of LeftHemisphere According to Grab Movement of Right Hand . . . . . . . . . . . . . 636

Mincheol Whang, Jincheol Woo, and Jongwha Kim

Table of Contents XXI

Part VI: Language, Voice, Sound and Communication

A Speech-Act Oriented Approach for User-Interactive Editing andRegulation Processes Applied in Written and Spoken Technical Texts . . . 645

Christina Alexandris

Interacting with a Music Conducting System . . . . . . . . . . . . . . . . . . . . . . . . 654Carlos Rene Argueta, Ching-Ju Ko, and Yi-Shin Chen

Hierarchical Structure: A Step for Jointly Designing InteractiveSoftware Dialog and Task Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 664

Sybille Caffiau, Patrick Girard, Laurent Guittet, andDominique L. Scapin

Breaking of the Interaction Cycle: Independent Interpretation andGeneration for Advanced Dialogue Management . . . . . . . . . . . . . . . . . . . . . 674

David del Valle-Agudo, Javier Calle-Gómez,Dolores Cuadra-Fernández, and Jessica Rivero-Espinosa

SimulSort: Multivariate Data Exploration through an EnhancedSorting Technique . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 684

Inkyoung Hur and Ji Soo Yi

WeMe: Seamless Active and Passive Liquid Communication . . . . . . . . . . . 694Nicolas Masson and Wendy E. Mackay

Study of Feature Values for Subjective Classification of Music . . . . . . . . . 701Masashi Murakami and Toshikazu Kato

Development of Speech Input Method for Interactive VoiceWebSystems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 710

Ryuichi Nisimura, Jumpei Miyake, Hideki Kawahara, andToshio Irino

Non-verbal Communication System Using Pictograms . . . . . . . . . . . . . . . . 720Makiko Okita, Yuki Nakaura, and Hidetsugu Suto

Modeling Word Selection in Predictive Text Entry . . . . . . . . . . . . . . . . . . . 725Hamed H. Sad and Franck Poirier

Using Pictographic Representation, Syntactic Information and Gesturesin Text Entry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 735

Hamed H. Sad and Franck Poirier

Embodied Sound Media Technology for the Enhancement of the SoundPresence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 745

Kenji Suzuki

Compensate the Speech Recognition Delays for Accurate Speech-BasedCursor Position Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 752

Qiang Tong and Ziyun Wang

XXII Table of Contents

Effectiveness of the Text Display in Bilingual Presentation of JSL/JTfor Emergency Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 761

Shunichi Yonemura, Shin-ichiro Eitoku, and Kazuo Kamata

Part VII: Visualisation, Images, and Pictures

Specifying the Representation of Non-geometric Information in 3DVirtual Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 773

Kaveh Bazargan and Gilles Falquet

Prompter “•” Based Creating Thinking Support CommunicationSystem That Allows Hand-Drawing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 783

Li Jen Chen, Jun Ohya, Shunichi Yonemura, Sven Forstmann, andYukio Tokunaga

A Zoomable User Interface for Presenting Hierarchical Diagrams onLarge Screens . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 791

Christian Geiger, Holger Reckter, Roman Dumitrescu,Sascha Kahl, and Jan Berssenbrügge

Phorigami: A Photo Browser Based on Meta-categorization andOrigami Visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 801

Shuo-Hsiu Hsu, Pierre Cubaud, and Sylvie Jumpertz

Sphere Anchored Map: A Visualization Technique for Bipartite Graphsin 3D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 811

Takao Ito, Kazuo Misue, and Jiro Tanaka

Motion Stroke-A Tablet-Based Interface for Motion Design Tool UsingDrawing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 821

Haruki Kouda, Ichiroh Kanaya, and Kosuke Sato

Tooling the Dynamic Behavior Models of Graphical DSLs . . . . . . . . . . . . . 830Tihamér Levendovszky and Tamás Mészáros

Pattern Recognition Strategies for Interactive Sketch Composition . . . . . 840Sébastien Macé and Eric Anquetil

Specification of a Drawing Facility for Diagram Editors . . . . . . . . . . . . . . . 850Sonja Maier and Mark Minas

A Basic Study on a Drawing-Learning Support System in theNetworked Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 860

Takashi Nagai, Mizue Kayama, and Kazunori Itoh

Benefit and Evaluation of Interactive 3D Process Data Visualizationfor the Presentation of Complex Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . 869

Dorothea Pantförder, Birgit Vogel-Heuser, and Karin Schweizer

Table of Contents XXIII

Modeling the Difficulty for Centering Rectangles in One and TwoDimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 879

Robert Pastel

Composing Visual Syntax for Domain Specific Languages . . . . . . . . . . . . . 889Luis Pedro, Matteo Risoldi, Didier Buchs, Bruno Barroca, andVasco Amaral

The Effectiveness of Interactivity in Computer-Based InstructionalDiagrams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 899

Lisa Whitman

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 909

Part I

Multimodal User Interfaces

J.A. Jacko (Ed.): Human-Computer Interaction, Part II, HCII 2009, LNCS 5611, pp. 3–11, 2009. © Springer-Verlag Berlin Heidelberg 2009

Using Acoustic Landscapes for the Evaluation of Multimodal Mobile Applications

Wolfgang Beinhauer and Cornelia Hipp

Fraunhofer Institute for Industrial Engineering, Nobelstrasse 12, 70569 Stuttgart, Germany

{wolfgang.beinhauer,cornelia.hipp}@iao.fraunhofer.de

Abstract. Multimodal mobile applications are gaining momentum in the field of location based services for special purposes. One of them is navigation sys-tems and tourist guides for pedestrians. In some cases, when the visibility is limited or blind people are longing for guidance, acoustic landmarks are used for macro-navigation rather than visual landmarks. Likewise, micro-navigation supported by pedestrian navigation systems must comply to the user's expecta-tions. In this paper, we present an acoustic landscape that allows the emulation of arbitrary out-door situations dedicated for the evaluation of navigation sys-tems. We present the evaluation capabilities and limitations of the laboratory as well as an example of an evaluation of a pedestrian navigation system that uses acoustic and haptic feedback.

Keywords: Acoustic landscape, test bed, pedestrian navigation, haptic feed-back.

1 Introduction

User Centered Design (UCD) [1] is a widely accepted process of product develop-ment, in which specific emphasis is put on the active involvement of users at each step of the design process. In particular, the user centered design process comprises the iterative run through a cycle of product prototyping, real world tests with actual users, evaluation and refined prototyping until a satisfying design has been found. While UCD is well established as a method for the development of hardware devices or user interfaces for software solutions, it encounters difficulties in case of other types of applications, such as location based services, especially if those are in a pre-mature stage.

When it comes to mobile devices and location based services, a thorough evalua-tion in real world outdoor tests can be hardly practicable and difficult to manage. Ubiquitous network access is not given. The test setting is far off the development labs. In many cases, it is impossible to maintain equal test conditions due to rapidly changing environmental factors which are not under control. Quantitative evaluation data might be blurred by imperfect test conditions. In the worst case, the evaluation of premature prototypes in outdoor scenarios involving handicapped test persons might even put them at risk.

4 W. Beinhauer and C. Hipp

With the Fraunhofer Interaction Laboratories, we introduce a test environment that emulates realistic outdoor conditions without the drawbacks of public space. The laboratory contains several indoor installations for testing and prototyping, one of them being a large scaled multimodal immersive environment that combines a 3D stereo projection with a novel surround sound system. The sound system allows for enhanced spatial audio perception and serves as a test bed for the development and evaluation of new interaction techniques, notably mobile applications. Within the immersive environment, ubiquitous network access is given as well as camera sys-tems for easy evaluation and data processing, pathing the way for rapid design and short test circles.

Simple two-channel stereophony, quadrophonic or 5.1 sound systems fall short of the requirements for precise sound-based localization as introduced in the sequel. A more accurate and more flexible sound positioning is needed. The sound system pre-sented in this paper does not deliver the quality and accuracy of high-end acoustic installations such as wave field synthesis [2], but enhances conventional multi-channel sound generation in a way that it is easy to implement and suitable for test purposes. The system is capable of emulating real-world audio conditions in a labora-tory environment and creating repeatable test conditions for the evaluation of third services. This paper presents the laboratory itself and shows how the lab is used as an acoustic landscape for the development of alternative multimodal pedestrian navigation.

2 Related Work

Spatial hearing relies on interaural time difference, interaural intensity difference and head related transfer functions. Acoustic landscapes that build on these effects have been around for quite a while already. Most of previous work aimed at special audio effects. Mauney and Walker [3] introduced the idea of livable “soundscapes”, and Mynatt et al. [4] introduced the term of an “audio aura”. In distinction to those contri-butions, we focus on acoustic environments that can be used for the development and evaluation of interactive systems such as a navigation system.

The use of audio interfaces or other modalities for navigation tasks have been ex-amined by various groups. Loomis and Klatzky [5, 6] dealt with self-orientation and related spatial orientation and multimodal spatial representations and examined their relevance for sensory substitution.

Walker and Lindsay presented a rapid prototyping tool for auditory navigation dis-plays based on virtual reality [7]. An approach to interactive acoustic environments that are capable of streaming spatiotemporal audio data is introduced by Heimrich et al. [8].

3 The Acoustic Landscape

The idea of the lightweight acoustic landscape is to create a test environment that al-lows for spatial sound generation beyond stereophonic reproduction. Therefore, sev-eral independent 7.1 speaker systems are stacked on multiple layers, providing spatial

Using Acoustic Landscapes for the Evaluation of Multimodal Mobile Applications 5

perception in all three dimensions beyond psychoacoustic effects. A dedicated soft-ware controller enables the positioning of mono sounds within the laboratory area.

3.1 Installation

The physical installation of the acoustic landscape consists of two 7.1 speaker systems mounted at different height levels in a rectangular room. While the upper annulus is located at a height of y1 = 282cm, the lower one raises at y2 = 44cm, sandwiching the average ears’ elevation. The speakers are controlled by specific software that creates a spatial sound perception. The set-up emulates a cylindrical sound space with a base radius of 256cm. Due to the given room geometry and limited fixation possibilities, an aberration from the ideal cylindrical shape is inevitable and leads to some distor-tion in angular sound perception. This effect can be reduced by a fine calibration of the amplifiers. An empirical evaluation has proven that satisfying results are obtained within a larger sweet spot extending close to the boundaries of the room.

The speakers deployed are conventional high-end consumer devices (GigaSet S750) that feature no special directional characteristics. The configuration leaves a rectangular experimentation space of roughly 25m2. Additionally, at the right hand border, a stereoscopic display extending to the full width of the laboratory is avail-able, making up a multimodal immersive environment.

Fig. 1. The acoustic landscape consists of two 7.1 speaker systems mounted above each other (only the tweeters depicted). The set-up emulates a cylindrical sound space with a base radius of 256cm within the rectangular room. The aberration of the ideal cylindrical shape leads to some distorted angular sound perception. View from above; all dimensions are given in centi-meters (cm).


3.2 Control Software

In order to put the acoustic landscape into operation, a dedicated control software was developed that makes use of the speakers beyond stereophony, quadrophony or 5.1 deployment that build on pseudoacoustic effects in order to create the impres-sion of distance or movement. Besides the subwoofers, 14 tweeters have to be con-trolled in order to generate a sound composition that allows for near-reality test conditions.

One possible approach to control software would be direct control of each indi-vidual speaker. Sound samples could move along a given trajectory through the audio laboratory by hand-overs between the speakers. However, this approach is not practical, since it would require separate sound cards and amplifiers for each speaker.

The key to a simpler approach is the segmentation of the acoustic environment. Ex-isting libraries such as OpenAL by creative labs or Microsoft DirectSoundspace ease the development of psychoacoustic effects in the periphery of the soundscape. Refer-ring to the Fraunhofer installation, this refers to the space above and below the upper and lower plane of 7.1 systems, as well as to the outside of the cylindrical base model. Proper generation of pseudoacoustic effects requires an equidistant mounting of the speakers along the circumference of the cylinder – a requirement that is not perfectly fulfilled due to the given room architecture, but that has been partly compensated for by a calibration procedure.

In between the speaker levels, pseudoacoustic effects in vertical dimensions are eliminated and replaced by manual cross-fading. The same yields for the horizontal plane. Moreover, the Doppler Effect for moving sound sources is calculated and ap-plied to the sounds before their reproduction. This way, up to 256 sound samples (mono-recorded wav, 44.1kHz, 16bit) can be moved along predefined trajectories through the room. The situation is depicted in Fig 2. The system is extensible in case a neighboring system is installed, which would require cross-fading between the two systems.

Fig. 2. Segmentation of the room with two sound systems. Human ears will usualy be located in the middle between the two planes, allowing for the most realistic sound impression.


3.3 Building an Acoustic Landscape

Usability and modularity were important aspects in the design of the user interface of the authoring tool for sound composition. The generation of a sound landscape is a two-step process. Firstly, the trajectory and playback parameters of a single sound have to be defined. This can be done graphically as shown in figure 3. A sound file is selected and its positioning in space is defined. Additionally, the dimensions of the laboratory can be altered here in order to adapt the system to new geometries. Several sound sources or trajectories can be saved as macros and can be re-used for the gen-eration of more complex sounds.

The second step consists of the orchestration of the various sounds to a landscape. For this purpose, the preset sounds are triggered on a timeline and follow their respec-tive trajectory. Figure 4 shows an example of an audio scene composed of 18 different sounds.

Fig. 3. User interface for presetting sounds. A trajectory through the sound space is attached to each sound source (graphical tool to the right). Alternatively, a series of co-ordinate quadruples in space and time can be defined, which are approached consecutively and in between which a linear interpolation is performed.

Fig. 4. User interface for composing a landscape. In the example, a landscape consisting of 18 sound sources is created.


4 Discussion

4.1 Evaluation

Currently no detailed evaluation with professional audio measurement equipment has been performed. However, an empirical study has produced some first preliminary results. The relevant criteria for the functionality of the sound landscape are the direc-tion, where a certain sound is heard to come from and the perceived distance, where a certain sound is estimated to origin from. Moreover, it was tested how the transition of a moving sound source from the inner area of interpolated positioning to the outer area of psychoacoustic effects is perceived.

The precision in which positioned sounds are perceived respective to their set point was evaluated with a series of tests. Firstly, a test person was placed into the sweet spot, hence, in the centre of the enveloping cylinder. A sequence of pulsating sound signals was played, originating from different pre-defined set points. The test person pointed towards the direction from where she heard the sound. The series have been repeated with different persons and different sound sources.

Despite of the very inaccurate measurement and the inherent subjectivity of the procedure, the analysis delivered some first results: As expected, the anisotropy of the speaker installation leads to a distortion of the sound characteristics in horizontal di-mension. In the right hand section (see fig. 1) of the lab, a good precision of about 10° angle of aperture is reached, which is in line with the subjective perception tolerance. In the left part, where the installation deviates most from the ideal cylindrical set-up, the distortion tears down the accuracy to 30° approximately. In vertical dimension, the precision remains at 10° also in the distorted area.

The perceived distance could not be captured satisfyingly so far. However, no dis-continuity was recognized by the test persons when transitioning between two sound effect areas. The physically correct interpolation inside and outside psychoacoustic effects are integrated seamlessly.

4.2 Future Development

A precise evaluation of the system using acoustic equipment is in progress in order to minimize the current distortions. The next step will be an enhancement of the interac-tivity of the acoustic landscape. Currently, only predefined sound sources can be moved along predefined trajectories. It is planned to stream arbitrary sound sources to arbitrary positions in real time. This way, the localizable sound output could be com-bined with object tracking and create new possibilities of immersive interaction be-havior.

5 Exemplary Development

5.1 Test Conditions

As an example, we introduce a system that has been developed and evaluated in the acoustic landscape. The exemplary development makes use of the laboratories’ facili-ties and shows the benefits of an indoor test environment.


Navigation systems are nowadays broadly used. The use of pedestrian navigation systems is rather restricted to touristic areas or larger sports events like the Olympics. Other systems use tagging technologies like geoannotation [9]. However, these ap-proaches based on PDA computers fail when it comes to blind people who need pe-destrian navigation aids the most. In the sequel, we introduce a navigation system for the blind whose development and evaluation was supported by the creation of an acoustic landscape in the laboratory.

The acoustic landscape can be a powerful tool for the simulation of realistic test con-ditions, in particular in overlaying desired sound signals with unwanted background-noise. Those conditions are completely repeatable and ease evaluation procedures.

Background noise is not necessarily intrusive or distracting. It can be very informa-tive as well. Especially for visual impaired and blind people, sounds of the personal environment of a person are valuable indicators for orientation. Especially outdoors, blinded people use the background-sound in great amount for their macro-navigation. Steady noises from distant sound sources such as the roaring of a motorway are per-ceived as acoustic landmarks just like a spire would de for sighted people. By con-trast, micro-orientation is performed by aid of the immediate and direct environment within a short distance of a few meters. Compared to that, macro-navigation is aiming at the environment with a greater distance, e.g. the location of the next bus-station. Therefore the acoustic landscape can simulate a surrounding, which is used by blind people for the orientation in their macro-navigation. Additionally, disturbing sound sources can be blend such as the abrupt appearance of a rattling motor bike. In the sequel, two systems for blind people are tested in the acoustic landscape.

5.2 A Pedestrian Navigation System

Since the visual channel is not available, other modalities have to be found in order to create an effective navigation system for the blind. One means for accessing blind people and signaling hints for their orientation is the use of haptic interaction. An-other one would be audio feeds. Haptic interaction has advantages over audio signals especially in very noisy surroundings, such as pedestrian areas or at bus-stations. Also, haptic feedback is not distracting for third people. The first test case presents the idea of haptic navigation wristbands.

The guided person wears one wristband on each arm, equipped with vibration mo-tors, which can be controlled by a central navigation-system to direct him in the right or left direction, or to stop him in case of danger. Likewise, the person could wear a smart phone on the belt, which receives GPS information about the user’s location and subsequently calculates and sends correct signals to the wristband. To enhance the accuracy, a pedometer can be integrated as well.

This wristband is very useful for blind people, when the surrounding is noisy and they cannot listen properly to an audio navigational aid. To prove this and to enhance the wristband, a noisy situation can be simulated with the help of the acoustic land-scape and ensure, that in the test situation, there is always the same interfering noise, which makes audio-information less useful.

The second example represents a sound based navigation system for pedestrians. Especially for visual impaired and blind people an orientation with the help of their hearing is very important. Beside the usage of a speech-system which can tell


Fig. 5. The concept of haptic wristbands for pedestrian navigation. A GPS device attached to a belt receives localization data and guides the user by means of haptic stimuli transmitted by the wristbands.

information explicitly, information can be given via sounds. This can be less intrusive as speech commands because speech is intensely attention-grabbing. A continuously repeated command can be distracting and annoying, whereas a discrete sound might be much less intrusive. A comfortable short sound at this point can improve the audio-navigation system. Moreover, this idea was continued in order to improve the macro-navigation by sounds. In laboratory testing in the acoustic landscape it was found which kind of information blinds are interested in: information for orientation, safety-related information and temporary barriers like construction-sites. Furthermore, ade-quate sounds could be identified some of them as earcons and some as auditory icons. Earcons refer to an arbitrary melody, which were designed for information, which do not have a natural sound, like “stairs”. In contrast, the auditory icons refer to reality and therefore we used for example for the information “construction work” a re-cording of a real construction site.

The acoustic landscape proved to be a valuable tool to test the sounds, whether the system is well heard and understood actually and not be ignored, when other sounds of the real environment are interfering. Further more it can be tested, how often and exactly when the sounds should be played in interfering situations to make sure the user received the information. Additionally it can be investigated, whether auditory icons are always useful or whether users have problems to identify, that the auditory icon is coming from the earphones and not from the real surrounding.

Ongoing developments of smart phones and personal digital assistances tend to the continuous trend of mobile usage of systems. Therefore the demand of methodology and tools for testing devices and application in a mobile situation will increase. One solution therefore will be the acoustic landscape.

Acknowledgements. We would like to thank the all contributors to the development of the acoustic landscape, in particular Frank Falkenberg and Till Issler, Jan Roehrich


and Andreas Friedel. Moreover, thanks to Romy Kniewel and Brigitte Ringbauer who have been contributing decisively to the development of the pedestrian navigation system.

References

1. Norman, D.A.: The Design of Everyday Things. Perseus Books Group, New York (1990) 2. Brandenburg, K., Brix, S., Sporer, T.: Wave Field Synthesis: From Research to Applica-

tions. In: Proceedings of 12th European Signal Processing Conference (EUSIPCO), Vienna, Austria, August 7-10 (2004)

3. Mauney, B.S., Walker, B.N.: Creating functional and livable soundscapes for peripheral monitoring of dynamic data. In: Proceedings of ICAD 2004. The tenth international confer-ence on auditory display, Sydney, Australia (2004)

4. Mynatt, E.D., Back, M., Want, R., Baer, M., Ellis, J.B.: Designing audio aura. In: Proceed-ings of the SIGCHI conference on Human factors in computing systems, Los Angeles, Cali-fornia, United States, April 18-23, 1998, pp. 566–573 (1998)

5. Klatzky, R.L., Loomis, J.M., Beall, A.C., Chance, S.S., Golledge, R.G.: Spatial updating of self-position and orientation during real, imagined, and virtual locomotion. Psychological Science 9(4), 293–298 (1998)

6. Loomis, J.M., Klatzky, R.L.: Functional equivalence of spatial representations from vision, touch and hearing: Relevance for sensory substitution. In: Rieser, J., Ashmead, D., Ebner, F., Corn, A. (eds.) Blindness, brain plasticity and spatial function, pp. 155–184. Erlbaum, Mahwah (2007)

7. Walker, B.N., Lindsay, J.: Using virtual reality to prototype auditory navigation displays. Assistive Technology Journal 17(1), 72–81 (2005)

8. Heimrich, T., Reichelt, K., Rusch, H., Sattler, K., Schröder, T.: Modelling and Streaming Spatiotemporal Audio Data. In: Meersman, R., Tari, Z., Herrero, P. (eds.) OTM-WS 2005. LNCS, vol. 3762, pp. 7–8. Springer, Heidelberg (2005)

9. Beeharee, A., Steed, A.: Minimising Pedestrian Navigation Ambiguities Through Geoanno-tation and Temporal Tagging. In: Jacko, J. (ed.) Human-computer Interaction: Proceedings of the 12th International Conference, HCI International 2007, Beijing, China, July 22-27 (2007)

Documents

Lecture Notes in Computer Science 5611...Lecture Notes in Computer Science 5611 Commenced Publication in 1973 Founding and Former Series Editors: Gerhard Goos, Juris Hartmanis, and