vSmiley : The imaging of vSmiley : The imaging of emotions through vibration emotions through vibration patterns. patterns.

Deepa Mathew

Department of Computer Sciences

University of Tampere, Finland

deepa.mathew@uta.fi

April, 2005

AAFG 2005

Deepa.Mathew p 02_18 18.04.2005

INTRODUCTION

Tactons are becoming a popular widget in various research programs to build new multi-model or multi-tasking user-interfaces, mainly because of it’s ability to directly interpret with human skin (cutaneous), which in turn gives an impression of ‘sense of feel’.

Various tactile patterns can be designed in a way, even to express emotion with sense of feel when it is presented as Smiley or in any

other forms like icons, graphics and so on.

Using tactons can even be a better option when building an user-interface for deaf/blind and also deaf-blind users

In the game “vSmiley” we have used the vibrating patterns (Tactons) for imaging of emotions. And this game is also intended for deaf or visually impaired children, using vibrations (Tactons) as the common root.

vSmiley

GOAL

•The goal of the project vSmileys is an empirical study of the

possibility to substitute emoticons by vibro-tactile patterns.

• Tactons could be a usesful means of communicating information

in user interfaces for the visually challenged and people with

hearing disabilities. So, the game could be a starting point to get

familiarised with the sematic patterns.

vSmiley

Deepa.Mathew p 03_18 18.04.2005

METHOD

Similar to the envelope of sound patterns (earcons) mechanical vibrations can be composed into the vibro-tactile patterns (tactons). Some earcons were produced and widely used to signify a semantically-completed messages and feedback cues in GUI. Vibro-tactile patterns are employed as a way of communication especially for and between people with sensorial deficit (deaf, blind, deaf-blind).

vSmiley

Deepa.Mathew p 04_18 18.04.2005

Figure1: Sound wave envelope

TECHNIQUES

The tactons were designed for 9 smileys signifying different emotions.

The aim of the pilot testing through the game was estimation of relative identification of the semantics of the constructed tactons (vSmileys) by comparing them with the emotion of the smileys presented visually.

Basic vibration patterns (Figure 2) have been used and combined to get the desired sample (vSmiley) which should contain semantics similar to the graphical prototype (smiley). Basic patterns can be combined to create composite vibro-tactile messages.

vSmiley

Deepa.Mathew p 05_18 18.04.2005

Figure2: Vibrating patters used in designing vSmileys, Immersion Studio 4.0

Figure 3: Logitech iFeel optical mouse

Apparatus and Procedure

Software: ‘Immersion Studio 4.0’ was used in designing vSlileys to transform smileys into composite vibro-tactile patterns.

Hardware: An Logitech iFeel optical mouse (Figure 4) connected via USB port was used to display the vibro-tactile patterns.

vSmiley

Deepa.Mathew p 06_18 18.04.2005

http://www.dcs.gla.ac.uk/~stephen/papers/WorldHaptics_Brown.pdf

Deepa.Mathew p 07_18 18.04.2005

vSmiley

Figure 6: C2 Tactor attached to finger.

There are some other devices as well that could be used. For example: Engineering Acoustics C2 Tactor (Figure 4) and TACTAID VBW32 transducer (Figure 5)

These devices was used in the research done by Brown et al [1]

Figure 5: TACTAID VBW32 transducer.

Figure 4: Engineering Acoustics C2 Tactor [2]

DESIGNING THE TACTONS

Before vSmileys have been created, the appropriate parameters for vibro-tactile patterns in which information can be encoded must be identified [1]. The most obvious parameters to use in vSmiley construction are the basic parameters of basic vibratory patterns such as frequency, amplitude, waveform (envelope: attack, sustain, decay) and duration of each segment/burst. However, in designing the vSmileys have been used only the frequency and duration, these parameters were changed to encode 9 smileys. Semantic information could be encoded by manipulating the duration of pulses. While duration alone could be used as a parameter, combining pulses of different durations to form rhythms would offer more flexibility [1].

vSmiley

Deepa.Mathew p 08_18 18.04.2005

TESTING PROCEDURE

Five participants, (2 females and 3 males) with normal vision and hearing took part in the pilot testing of the vSmileys game.

None of the participant had any previous experience in this kind of testing and especially had no idea about tactons.

They are in the age group of 23-28 (2 students and 3 employees)

Five minutes training time was given to each of them to memories vSmileys. If needed, then extra time was given.

After the training time was over the participants where asked to identify different vSmileys by feeling the vibratory patterns prsented with Logitech iFeel optical mouse with built-in shaking motor. The test consisted of 90 trials (9 vSmileys x 10 times) and the tactons where selected randomly. Data were recorded on the recognition of the correct smiley and stored in a log file after completion the game.

vSmiley

Deepa.Mathew p 09_18 18.04.2005

The game is played as explained below

• Click on each smiley as many times as needed to memories the vSmileys (vibro-tactile patterns).

• After player have memorised all the patterns, s/he can activate the testing phase by tapping the space bar.

• Tapping the spacebar produces the test-pattern. Player can repeat this vSmiley as many times as needed.

• The player has to identify the test-pattern and point by click the correct smiley having the same meaning.

vSmiley

Deepa.Mathew p 10_18 18.04.2005

Figure7: vSmiley game

RESULTS AND DISCUSSIONS

vSmiley

Deepa.Mathew p 11_18 18.04.2005

The players had some trouble choosing the right smiley in the

beginning but the results were gradually improved.

And they were able to distinguish the vibrating patterns and identify the

correct smiley after few trials.

But they had some trouble when identifying the somewhat-similar

vSmileys. They got easily confused and made more errors.

vSmiley

Deepa.Mathew p 12_18 18.04.2005

The average number of repeatitions per vSmiley in training phase for each player.

Each of the players had little difficulty in memorising the 6th and 9th vSmileys. Maybe because of its similarity.

0

2

4

6

(:| :(( :-O :D :) :-P :| X-( /:)

Smileys

Rep

etiti

on p

er im

age,

n

Series1 Series2 Series3 Series4 Series5

vSmiley

Deepa.Mathew p 13_18 18.04.2005

0

2

4

6

(:| :(( :-O :D :) :-P :| X-( /:)

Smileys

Rep

etiti

ons

per i

mag

e, n

Series1 Series2 Series3 Series4 Series5

The average number of repeatitions per vSmiley in testing phase for each player.

Deepa.Mathew p 14_18 18.04.2005

vSmiley

0

2

4

6

8

(:| :(( :-O :D :) :-P :| X-( /:)

Smileys

Wro

ng tr

ial,

n

Series1 Series2 Series3 Series4 Series5

The average number of wrong trials per vSmiley.

vSmiley

Deepa.Mathew p 15_18 18.04.2005

The average number of repetitions per image throughut the test for all the players.

0

1

2

3

4

5

6

(:| :(( :-O :D :) :-P :| X-( /:)

Smileys

Rep

etiti

ons

per i

mag

e, n

Repet. in training phase Repet. in testing phase

vSmiley

Deepa.Mathew p 16_18 18.04.2005

The average error rate during the test. Some vSmileys were easily distinguished and some were not.

0

1

2

3

4

5

6

(:| :(( :-O :D :) :-P :| X-( /:)

Smileys

Erro

rs, n

vSmileyCONCLUSIONS

Vibro-tactile patterns and semantic sequences can play a vital role for building new multi-model or multi-tasking user-interfaces especially for users having a sensorial deficit and also introduce a possible alternative way of communication.

Tactons were already been used in areas such as tele-operation or displays for blind people to provide sensory substitution [2]. But not to its true abilities.

More research should be done to know the true usefulness of the tactile patterns. And based on the pilot experiment it is obvious that it is easy to identify and differentiate different vibrating patterns. And with proper research and knowledge, tactons could replace speech or short speech messages which could be vital for the deaf users.

Deepa.Mathew p 17_18 18.04.2005

vSmiley

Click to go the first slide

References

[1] Lorna M. Brown, Stephen A. Brewster and Helen C. Purchase. A First Investigation into the Effectiveness of Tactons.Glasgow Interactive Systems Group, Department of Computing Science, University of Glasgow, UK

[2] Dr Steven Wall, An investigation of multimodal interaction with tactile displays. Department of Computing Science, University of Glasgow.

Deepa.Mathew p 18_18 18.04.2005

Thank you!

Any Questions?