Haptic Feedback on Mobile Touch Screens

Applications and Applicability

12.11.2008Sebastian Müller

Haptic Communication and Interaction in Mobile ContextUniversity of Tampere

12.11.2008 2

Outline

Motivation

Theory (haptic sense, technologies)‏

Practice (technologies, applications)‏

Design Issues

Future

Discussion

12.11.2008 3

Motivation

Keyboards being replaced by virtual buttons− worse entry speed and higher error rate

mobile = flexible context− appropriateness of feedback techniques

additional information channel = additional capacity

physical metaphor (”more real”)‏

12.11.2008 4

What information to deliver

actions− click, double-click, drag, rotate, zoom, scroll

reactions− events

scales− time, strength

target objects− GUI elements, on-screen keyboard

12.11.2008 5

Sensation space

Haptic: − frequency, amplitude, length, space (x2)‏− 5 times faster than vision [5]− 2mm two-point vibrotactile discrimination threshold at

fingertip− much smaller position shift identification threshold [7]− no ”privacy effect” on touchscreens

12.11.2008 6

Touchscreen interaction

Depending on technology− Capacitive TS-> skin

Depending on device sizeand layout

Depending on context− Noise, movement, task (accuracy required)‏

12.11.2008 7

Hoggan, Brewster & Johnston [8] (1/2) ‏

Investigating text input

Palm Treo 750 (physical keyboard) and Samsung i718 (virtual keyboard)‏

”over-”, ”home-” (F + J), click- and slip-events imitating physical properties

Lab vs. Subway

Physical vs. Virtual vs. Haptic

(Global vs. Local feedback)‏

12.11.2008 8

Hoggan, Brewster & Johnston (2/2)‏

Haptic feedback significantly improves performance, especially under mobile conditions

Two specialized actuators as good as a physical keyboard under mobile condition

12.11.2008 9

What I thought

12.11.2008 10

What I found

Snap-Crackle-Pop [4]

”off-the-shelf” piezo actuators

global feedback

4 applications− numerical keypad, text selection,

scrolling, drag and drop

Result: ”potential to improveusability and user experience”

Nokia Research Center, Helsinki (2006)‏

12.11.2008 11

Nokia Haptikos

two small piezo sensor pads

0.1mm movement

As you have seen in Nokia 770

“Funnily enough, although you think you’re typing faster than normal because of the feedback, in actual fact you’re not [...] There’s just some sort of mental

satisfaction that comes from typing with a tactile response.”-- Roope Takala, Senior Program Manager at Nokia’s research labs (after redferret.com [10])

12.11.2008 12

Poupyrev & Maruyama (2003) [5]

TouchEngine [12]

very thin actuators (~0.5mm)‏− embeddable inside screen

only the glass moves

haptically augmented GUI elements

various actuator sizes

12.11.2008 13

Haptic Pen ‏(1/2) [2]

low cost

multi-user

non-contact feedback

3D tracking allows feedback on any surface− drawable surfaces− feedback based on location

12.11.2008 14

Haptic Pen ‏(2/2)

via youtube.com

Haptic Pen Demonstration

12.11.2008 15

T-Bars

Finger gesture guide− Replacing buttons

Changes in intensity oftactile feedback

Can be invisible

Required size?

Direction??

12.11.2008 16

Design Issues - Hardware

energy consumption

material fatigue

where to put actuators− parallax problem [5]

synchronization

→ frequency, amplitude, (frequency of occurrence)‏

12.11.2008 17

Design Issues - Wetware

expressiveness limited

necessity to learn haptic icons ‏(?)− ”no [...] interaction between cognitive load and [...]haptic

feedback were found” [1]

subjectively useful/helpful

”confusing”, ”annoying” [1]

Inter-subject perception difference− Gender, culture, context

12.11.2008 18

Future

Blackberry Thunder RIM keyboard

United States Patent Application 0070247429 (Apple, Inc.) [11]− ”[...] a touch sensitive electrode circuit board disposed beneath the surface cover having a

plurality of holes disposed therein; an articulating frame disposed beneath the touchsensitive electrode circuit board having integral therewith a plurality of key edge ridgesaligned with the holes in the touch sensitive electrode”

− ”Articulating Frame Protrudes/Deforms Surface Cover at Key Edges During Typing”

− No technical details!

Same for Nokia− International Patent Application PCT/EP2006/009377

12.11.2008 19

Discussion

Haptic feed vs. haptic feedback− Fingers exploring touchscreen− A haptic feed piezo-style stylus

Can movement positions be averaged?

12.11.2008 20

Summary

Software-/Interaction-triggered actuators

”With the addition of this extra tactile feedback the performance of touchscreen keyboards can bebrought close to the level of real, physicalkeyboards. [...] the perceived quality of the buttonsis improved” [8]

Feedback moves from global to local

No real-time 3D printer

12.11.2008 21

Thank you.*

*Can somebody change being opponents with me on Dec. 10?

References ‏(1/3)[1] @conference{leung2007eha,title={{Evaluation of haptically augmented touchscreen gui elements under cognitive load}},author={Leung, R. and MacLean, K. and Bertelsen, M.B. and Saubhasik, M.},booktitle={Proceedings of the ninth international conference on Multimodal interfaces},pages={374–381}, year={2007}, organization={ACM New York, NY, USA}}

[2] @conference{lee2004hpt,title={{Haptic pen: a tactile feedback stylus for touch screens}},author={Lee, J.C. and Dietz, P.H. and Leigh, D. and Yerazunis, W.S. and Hudson, S.E.},booktitle={Proceedings of the 17th annual ACM symposium on UI software and technology},pages={291--294}, year={2004}, organization={ACM New York, NY, USA}}

[3] @conference{doerrer2001nao,title={{A New Approach to Operating Machines with High Functionality}},author={Doerrer, C. and Werthsch{\"u}tzky, R.},booktitle={Proceedings of Eurohaptics 2001}, pages={105--107}}

[4] @article{kaaresoja2006scp,title={{Snap-Crackle-Pop: Tactile Feedback for Mobile Touch Screens}},author={Kaaresoja, T. and Brown, L.M. and Linjama, J.},journal={Proceedings of Eurohaptics 2006}, pages={565--566}, year={2006}}

References ‏(2/3)[5] @article{geldard1960snp,title={{Some Neglected Possibilities of Communication}},author={Geldard, F.A.},journal={Science}, volume={131}, number={3413}, pages={1583--1588}, year={1960}}

[6] @conference{poupyrev2003tis,title={{Tactile interfaces for small touch screens}},author={Poupyrev, I. and Maruyama, S.},booktitle={Symposium on User Interface Software and Technology: Proceedings of the 16 th

annual ACM symposium on User interface software and technology},volume={2}, number={05}, pages={217--220}, year={2003}}

[7] @book{barfield1995vea,title={{Virtual Environments and Advanced Interface Design}},author={Barfield, W. and Furness, T.A.},year={1995}, publisher={Oxford University Press, USA}}

[8] @article{hoggan2008iet,title={{Investigating the effectiveness of tactile feedback for mobile touchscreens}},author={Hoggan, E. and Brewster, S.A. and Johnston, J.},year={2008}, publisher={ACM New York, NY, USA}}

References ‏(3/3)[9] @article{hall8tbt,title={{T-Bars: Towards Tactile User Interfaces for Touchscreen Mobiles}},author={Hall, M. and Hoggan, E. and Brewster, SA},journal={Proc MobileHCI'08}}

[10] Nokia perfects the clicky tactile touchscreen - iPhone gnashes teeth, swears revengehttp://www.redferret.net/?p=9533, accessed 06.11.2008

[11] United States Patent Application 0070247429http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&u=%2Fnetahtml%2FPTO%2Fsearch-adv.html&r=1&f=G&l=50&d=PG01&p=1&S1=20070247429 &OS=20070247429 &RS=20070247429, accessed 06.11.2008

[12] @conference{poupyrev2002ttd,title={{TouchEngine: a tactile display for handheld devices}},author={Poupyrev, I. and Rekimoto, J. and Maruyama, S.},booktitle={Conference on Human Factors in Computing Systems},pages={644--645}, year={2002}, organization={ACM Press New York, NY, USA}}