Advanced HCI Affective Interaction

CG007: Advanced Human-Computer Interaction Lecture: Affective Interaction

Slide 1

Advanced HCI

Affective Interaction

Affect is dealt with in: Preece, Sharp and Rogers (Chapter 5)Benyon, Turner and Turner (Chapter 17)

This lecture reviews explanations of why and how interactive products influence users’ emotions, and through this, their attitudes and behaviour.


Slide 2

Performance & Emotion

• Early cognitive contributions to HCI focus on improving users’ performance

• Usability– Making tasks easier (preventing error,

providing help etc)– Lightening the “cognitive load”– Making tools intuitive

• Main aim in the late ‘80s: to improve design of computer-based work

– Users’ emotions only viewed as byproduct of unhindered cognitive processing

– Software/Hardware works = user satisfaction/acceptance

• Focus of computing changing: from workplace to lifestyle

– iPod, Xbox, Palm Pilot, games, chat, shopping, entertainment, leisure…

– Activities associated with pleasure (emotion)– Not just productivity

CHILL OUT!


Slide 3

Performance & Emotion

• What are emotions?• Why, and how, do they affect our performance? • In what ways can emotion be taken into account when

designing programs or devices?• 3 main research areas:

– Computer recognition of human emotion» Computer adapts its behaviour based on what it knows about

user– Computer simulation of human emotion

» Computer appears to display human characteristics– Computer evocation of human emotion

» Computer targets human emotions and encourages particular responses


Slide 4

What is an emotion?

• A neural impulse that moves an organism to action

– We cannot choose our emotions (see below)

– Emotions happen spontaneously in response to events in the environment

– Causes physiological changes visible in behaviour

» Facial expression, faster heartbeat, perspiration, aggression, crying etc.

– Deeply rooted survival mechanism enabling organism to adapt to environment

• cf. Method Acting– The actor “replicates” the

emotional conditions of the character in order to add realism to a dramatic performance

– Actor draws on own “emotional memory”

Pairs of emotions (Plutchik, 1980)Primary emotions can be combined to form secondary emotionsDisgust + Sadness = Remorse


Slide 5

Where do emotions come from?

Cerebral Cortex (Man)– Planning, language etc.

Limbic System (other mammals)– Emotions

Basal Ganglia– 2-way messaging system between

Cerebral Cortex & Limbic System Brain Stem (present in

reptiles)– Basic body functions– Hunt, kill, mate, defend territory…

Evolution of the brain

• Common belief was that rationality & emotion are separate cf. early cognitive view that treats user as an information processor

• Emotion plays some part in decision-making etc. Therefore affects performance, response etc.


Slide 6

Don Norman: Revised model of cognition

• Visceral level– Fast– Makes rapid judgements– Good/bad;

safe/dangerous– Signals motor system &

alerts brain– Inhibited/enhanced by

control systems• Behavioural level

– Learned behaviour coming from experience

– Largely automaticised– Can inhibit/enhance

visceral layer– Inhibited/enhanced by

reflective layer

Sensory

Motor

Visceral

Behavioural

Reflective

control

control

• Reflective level– No direct access to sensory

input or control of behaviour– Watches over, reflects upon,

& tries to influence behaviour level

– Responses slow based on assessment, judgement & decision-making

• Norman, 2004


Slide 7

Emotional responses to stimuli

stimulus event cognition feeling state overt behaviour

effect

threat “danger” fear escape safety

obstacle “enemy” anger attack destroy obstacle

gain of valued object

“possess” joy retain or repeat

gain resources

loss of valued object

“abandonment”

sadness cry reattach to lost object

member of one’s group

“friend” acceptance groom mutual support

unpalatable object

“poison” disgust vomit eject poison

new territory “examine” expectation map knowledge of territory

unexpected event

“what is it?” surprise stop gain time to orient


Slide 8

Emotional response to stimuli

• Ekman, Friesen & Ellsworth (1972)

• identified 6 basic emotions

• recognised & expressed in same way in all cultures

anger fear disgust

surprise happiness sadness

(Ekman, Friesen & Ellsworth, 1972)

Which emotions are these people displaying?


Slide 9

Relationships with Computers

• Humans respond to computers like real people

– We prefer computers that flatter us

– We prefer computers which have a similar personality to our own

– We are polite to computers

• Therefore, we seem to have social relationships with computers

– Emotions are an important component of social relationships…

• If computer has a friendly face, will we “like” it more?


Slide 10

Facial Action Coding System (FACS)

• Ekman and Friesen (1978) develop FACS

– Taxonomy of expressions– Defines expressions as 1 of

46 “Action Units”» Contraction or relaxation

of one or more muscles• Cowe & Johnston

(Psychology Dept. University College, London)

• system automatically generates a virtual puppet, or avatar, then drives it by mimicking the facial movements of an actor.

• Applications?– Recognising user expression– Generating appropriate

responses for avatars

actor

avatar


Slide 11

Recognizing emotions

• Tele-healthcare (Lisetti, et al. 2003)

• System models patient’s affective state using multiple inputs from wearable sensors and other devices such as camera

• Detected emotions mapped onto intelligent agents embodied as avatars

• Avatar able to chat to patient to confirm emotion

• 90% success recognizing sadness

• 80% success for anger

• 70% success for frustration


Slide 12

Expressing emotion: affective input to interactive systems

• Can humans express emotions to computers?• What if human wants to explicitly identify an emotion to influence the behaviour of a character in a game?• SenToy

– Doll-based interface to game– Gestures indicate emotions– Emotions affect game strategy

happiness sadness anger fear surprise gloating


Slide 13

Expressing emotion: affective input to interactive systems

• Users a little confused• Hard to understand opponent’s

emotions• BUT many related to doll

(projection)• Higher satisfaction

Bullock, A. and Gambäck, B. (2003)


Slide 14

The tenuous area of “Affective Computing”

• Rash of projects exploring “Affective Computing”• Example: Gustbowl project (Keller, van der Hoog, and Stappers, 2004)• “Digital communication technology is increasingly affecting the way people organize their social contacts. Product designers therefore must understand users’ needs, not only on a

functional level (what information they share) but also on an affective, experiential level (what emotions are involved). Technological communications solutions can easily fail because they reduce affective interactions to functional ones. Our team from Delft University of Technology’s ID-Studiolab (http://studiolab.io.tudelft.nl) designed the Gustbowl to promote and support informal, unobtrusive interactions in families whose members live apart. The Gustbowl helps families keep in touch, rather than just exchange information, by letting members be a part of each other’s daily routines.”

• Use scenario: son throws keys into the Gustbowl; mother notices the wobble and sees the picture appear in her Gustbowl.

• Can such projects really be classed as “affective computing”?


Slide 15

The tenuous area of “Affective Computing”

• Example: A Foundation for Emotional Expressivity (Stahl et al. )• “To express emotions to others in mobile text messaging in our view require designs that can both capture some of the ambiguity and subtleness that characterizes emotional

interaction and keep the media specific qualities.

Through the use of a body movement analysis and a dimensional model of emotion experiences, we arrived at a design for a mobile messaging service, eMoto.

The service makes use of the sub-symbolic expressions; colors, shapes and animations, for expressing emotions in an open-ended way.”

8 backgrounds each “expressing” a different emotion?


Slide 16

Subjective association between colour & emotion?

• Can we agree on emotion - colour linkage?

• What would you say your emotional state is at the moment?

• Don’t say “boredom”!


Slide 17

Computers as Persuasive Technology

• Captology (Fogg, 2002)• Interactive technology to persuade people to change ideas &

behaviour

• Computing technologies have some advantages over human persuaders. For example, computers can:

– Be more persistent than humans– Offer greater anonymity– Manage huge volumes of data– Go where humans cannot go or may not be welcome

• Computing technologies persuade in different ways, depending on their role:

– Tool– Social actor– Medium


Slide 18

Computers as Persuasive Technology

• A tool can be persuasive by:– Making target behaviour easier to do– Leading people through a process– Performing calculations or measurements that motivate people

• A social actor can be persuasive by:– Rewarding people by giving positive feedback– Modelling a target behaviour or attitude– Providing social support

• A medium can be persuasive by:– Allowing people to explore cause and effect relationships– Providing people with various experiences that motivate them– Helping people to rehearse a behaviour


Slide 19

Persuasive Technology: some difficulties

• Ethical issues– When and how am I being persuaded? Do I know?– Intrusive?

• Research – How easy is it to study persuasive technology?– If your bank is doing this, will they tell you?– How can you find out the effect?– Even if you can find out:– Aren’t people who have been “persuaded” likely to be defensive?


Slide 20

References

• Bullock, A. and Gambäck, B. (2003). Evaluating affective interaction in gaming. Presented at the 8th European Conference on Computer Supported Cooperative Work (ECSCW 03), 14th-18th September 2003, Helsinki, Finland.

• Ekman, P., Friesen, W.V. and Ellsworth, P. (1972). Emotion in the Human Face. Pergamon, NY.

• Ekman, P. and Friesen, W.V. (1978). The Facial Action Coding System. Consulting Psychologists’ Press, Palo Alto, CA.

• Fogg, B.J. (2002). Persuasive Technology: Using Computers to Change What We Think and Do. Morgan Kaufmann.

• Keller, I., Van der Hoog, W. and Stappers P.J. (2004). Gust of Me: Reconnecting Mother and Son. IEEE Pervasive Comp. 3,1(2004), 22-28.

• Lisetti, C., Nasoz, F., LeRouge, C., Ozyer, O. and Alvarez, K. Developing multimodal intelligent affective interfaces for tele-home health care. Int. J. Human-Computer Studies 59 (2003), 245-255.

• Norman, D. (2004). Emotional Design: Why we love (or hate) everyday things. Basic Books, NY.

• Plutchik, R. (1980). Emotion: A Psychobioevolutionary Synthesis. Harper and Row, NY.

• Reeves, B. and Nass, C. (1996) The Media Equation: How people treat computers, television, and new media like real people and places. Cambridge University Press, New York.

• Ståhl, A., Sundström, P., and Höök, K. (2005). A Foundation for Emotional Expressivity. In: Designing For User Experience, DUX 2005, 3-5 Nov, 2005, Fort Mason, San Francisco, CA.

Documents

Advanced HCI Affective Interaction