On the semiotic appropriationof ICT tools

for people in loss of autonomy

N. Pignier+, D. Tsala-Effa+, L. Billonnet*, A. Geslin-Beyaert+ and J.M. Dumas*

University of Limoges

+ Centre de Recherches Sémiotiques (CeReS)Faculté des Lettres et Sciences Humaines

• Ecole Nationale Supérieure d’Ingénieurs de Limoges (ENSIL)

Outline

I. Introduction and context

II. Deployment of home automation and ICT packs

III. Major results

IV. Learning from experimentation…

V. What about semiotics?

VI. Methodology

VII. To a human-machine semiotic interface

I. Introduction and contextCreuse is one of the oldest population in Europe

« How Europe will stand within the next 20 years… »

The district of Guéret 19 towns 29000 inhabitants

http://www.cc-gueret.fr/

II. Deployment of homeautomation and ICT packs

Home automation and ICT healthcare solutions for: Motor disability Security Social link Actor’s coordination as a function

of patient troubles and housingEquipments packs :

Pack A : Comfort & Communication Pack B : Security Pack C : A + B Pack D : Enhanced B Pack E : A + D

III. Major resultsAppropriation and interest at home:

The major part of families wish to maintain the equipment and

imagine a system of coverage mutualized for a better financing.

The equipments of home automation are better accepted than the

“communicating equipments” (which did not work well).

Appropriation and interest in medical institution:

Persons with a single room take better benefit of the equipments.

The professionals show a better image of themselves.

Decline of the stress for people in charge of disoriented persons.

IV. Learning from experimentation…

Only 8 % of the calls come from a situation of loss of autonomy.

ICT: a wish but still an unsuitable system; feeling of disappointment.

Poor consideration of human being, user friendliness, personalization,

dignity and respect.

Feeling of intrusion and stigmatization. Training and information needed for a better appropriation of equipments.

1. Need for an ethical dimension.

2. Appropriation of ICT tools is a condition of success.

… to the semiotic approach

1. Definition Saussure (1916) : “The life of signs within the social life”. Today semiotics is a human science which analyzes systems of

signification, whatever the language which expresses them: texts, utterances, pictures, spaces, etc.

2. Semiotics of the ICT includes two main directions:

i.the multimedia, multimodal and hypertextual contents;

ii.the graphic and material interfaces.

In both cases, the semiotician questions the conditions of perception

of a text or an object on a digital support.

V. What about semiotics ?

V. What about semiotics ?

3. Semiotics of the ICT in the field of health

is interested in the way the user, with his current skills and the gestural, cultural experiences which he developed in the course of the practices of the everyday life perceives such or such digital object intended to help him.

As a consequence, the ICT object is acceptable only insofar as it interacts with the person through his experiences

(ex : failure of the light path that does not tell the story of the person).

Finally, the semiotician thinks out the coherence between design of a

numeric device and the perception the user can have of the interaction

with this device.

VI. MethodologyImportance of the gestural interaction

The specificity of the material interface is not only connected to the shape, the size, the color but it also includes the mode of gestural interaction. Gesture = any physical movement required by the digital interfaces of the objects to

activate an action

Gestures required by the material and graphic device to interact with the digital object constitute a modality that is a set of specific rules,based on a singular value system. Example of gestures : press on a button in the keyboard, pinch fingers on a touch-

sensitive screen to shrink an image, spread them to enlarge it, ...

VI. Methodology

Our hypothesis: the pleasure of the interaction

1. The pleasure bound to the use of a digital object can be of

diverse nature. It can be bound to the aesthetics, and\or to the performance,

and\or to the features…

but also to the gestural relation.

2. The pleasure bound to the interaction with the object can

appear from various modes of interaction, each based on a

particular system value.

VI. Methodology

The 3 modes of interaction

Proximity effectbetween the bodyand the digital object

Effect of similarity between gesturefor action on the digital object

SSSimulationOrange Gestures 2011SS

SS

LearningSpecific keyboard

HabitAF & SNCF terminals

VII. To the human-machine semiotic interface (HMSI)

Necessary conditions

To generate sense for the user the designer has to think of the graphic and material interface of the digital

object according to the specificity of the user (his gestural memory, lifestyle,

biography).

Under an HMSI situation, the relationship is unbalanced. The biographical elements taken into account are only those of the user, not

of the machine even for a humanoid.

VII. To the human-machine semiotic interface (HMSI)

Necessary conditions

The interaction User/numeric object : only a quest of result or also an art to make, a process?

To generate sense for the user the designer has to think the interaction not only as result, efficiency, because this point of view supposes that the body of the user is only an instrument which serves to activate commands but also as process, art to make.

In this case, the designer supposes also that the body of the user find, during the relation with the object, a dynamics, a cooperation on which bases itself the pleasure of the interaction.

VII. To the human-machine semiotic interface (HMSI)

Intuitive interfaces for a natural gesture

Several mono or multimodal interfaces techniques dematerialized interfaces

vocal interfaces (analysis and synthesis) virtual interfaces using gesture recognition

augmented reality and virtual assistive information haptic, visual or auditive augmentation as a an assistance haptic, visual or auditive augmentation as a validation

intuitive tactile sensitive processes as a projection of an intuitive gesture

On the semiotic appropriationof the ICT tools

for people in loss of autonomy

N. Pignier+, D. Tsala-Effa+, L. Billonnet*, A. Geslin-Beyaert+ and J.M. Dumas*

University of Limoges

+ Centre de Recherches Sémiotiques (CeReS)Faculté des Lettres et Sciences Humaines

• Ecole Nationale Supérieure d’Ingénieurs de Limoges (ENSIL)