Theory of Affect

Part Two

MS2306

From rational machines to affective computing

• See IBM film from 1965 (first 2mins)

• See Royal Society Summer Exhibition (2mins)

This week

1. Defining affect2. The marketing of affect3. Affect and HCI

The Atmosphere of Affect

Insubstantiality of affect makes it difficult to

touch. It has no substance (form), but it

does have an influence… a force…

Nonrepresentation

What is in the box?

Theories of Affect

• What is Affect? (Thrift, 2008 pp. 175-182)

• No stable definition of affect

• Not just about feeling or emotion, more about motion, social interaction…

Affect is not the opposite of cognition

• Affect is not irrational

• Alternative way of thinking about the world – a different kind of intelligence (Thrift, 2008)

Emotion/Cognition

Four Approaches to Affect

Four Approaches to Affect 1. Darwin’s study of

emotions

• Universal emotion• Affective expression

evolutionary– Preparing humans and

animals for action• Omits communicative

affect

The Expression of the Emotions in Man and Animals in 1872

Slight Sadness Fear mixed with surprise

Happiness and contempt

Sadness and fear

Disgust and contempt

Fear and surprise Questioning surprise

Neutral

Four Approaches to Affect2. Psychoanalytical Affect

• Emotions part of the inner unconsciousness of the individual

• Linked to biological drives

• A biologically-derived affect

Is psychoanalysis dead or just unconscious?

Freud

Four Approaches to Affect3. Bodily States and

Processes

• Criticises idea of subjects “talking about” their emotion

• Source of emotions can come from outside the body

• Blushes, laughs, crying and anger are visceral responses to others

Four Approaches to Affect4. Manifold “Psychology”

of Affect

• Capacity to affect and be affected

• Like insects, swarms

• Emergent interactions driven by sensory environment

Crowd Theory

Feeling, Emotion, Affect

Feeling, Emotion, Affect by Eric Shouse (drawing on Brian Massumi)M/C Journal, Affect. Volume 8, Issue 6 Dec. 2005

Feeling• A sensation

• ‘Checked against previous experiences’

• Labeled (I feel happy, hurt, angry, scared…)

• ‘Personal and biographical’ – ‘every person has a distinct set of

previous sensations from which to draw when interpreting and labeling’

Feeling, Emotion, Affect by Eric Shouse M/C Journal, Affect. Volume 8, Issue 6 Dec. 2005

Emotion• A projection

• Display or broadcast of a feeling

• Can be either genuine or feigned

• Paul Ekman’s experiment

• American and Japanese subjects watched films depicting facial surgery

– When alone - displayed similar expressions

– When in groups - expressions were different

Feeling, Emotion, Affect by Eric Shouse M/C Journal, Affect. Volume 8, Issue 6 Dec. 2005

Affect• A non-conscious experience

of intensity

• A moment of unformed and unstructured potential - Always prior to and/or outside of consciousness (Massumi in Parables of the Virtual)

• The body’s way of preparing itself for action (decision making)

Feeling, Emotion, Affect by Eric Shouse M/C Journal, Affect. Volume 8, Issue 6 Dec. 2005

Putting Affect to Work?

• Thrift locates the “strategic” use of affect in political and corporate arenas…

• He also refers to its location in design

The marketing of affect

Look at how good Apple are at developing affective relations between users (consumers), brands and products

Computer design has not however always considered human emotion

IBM’s Design of Computers

IBM’s Design of Computers

IBM’s Design of Computers

Emotional Design?

The marketing of affect

Corporations in the business of making 'hormonal’splashes through increasing contact with consumers

(Thrift p. 247)

The marketing of affect

Affective Computing

Affective Computing

• Field concerns emotions and computers

• A major shift from traditional “rationale” computer research

But Jim, human emotions are irrational

Affective Computing

Main focus on the link between emotion and…

Decision making LearningMemory processes

Neuroscience

• Antonio Damasio (1994) located the importance of emotions in thinking processes

• I think therefore I am• I feel therefore I am

Descartes' Error?

Indecision• Contrary to Spock,

experiments with real people with brain disorders that produce a lack of emotion were found to be terrible at making decisions

• Consider all the possibilities

• Continue analyzing

• Unable to conclude

Affective Computing

1. Can computers express, recognize and understand emotions?

2. Is a computer able to feel?

See Eerik Vesterinen’s literature review - Locates prominent research questions in the field

How do computers express emotions?

• Like an actor controlling facial expressions, stance, voice tone, proximity etc…

Expressing emotions

• A computer can express emotions without really “having” emotions, or without really “feeling”

• Videos of MIT's Nexi MDS Robot: First Test of Expression

• - Official MDS Robot Video - First Test of Expressive Ability (Longer version)

How do computers recognize emotions?

• A computer must have senses

• Audio to hear vocal intonations

• Video to see facial expressions

• Try it out on the Affectiva website

• http://www.affectiva.com/affdex/?#pane_tryit

• Read about it: http://www.npr.org/2012/02/03/146343563/how-did-that-ad-make-you-feel-ask-a-computer

How do computers recognize emotions?

• Reading infrared body temperature and measuring electrothermal skin conductivity

• Measuring heart rate and respiration• Uses these inputs to infer an emotional state

How do computers recognize emotions?

• Testing a computer’s ability to guess the emotional state of a user

Joseph Weizenbaum’s Eliza (Java Version

Article by Noah Wardrip-Fruin

Eliza, Tale-Spin, and SimCity

“Jenn” at alaskaair.com

Alicebot.Org

Can Computers Feel?• Picard (1998) proposes a

model of five components that should all be present in a system if it is to have emotions.

1. Emergent Emotions2. Fast Primary Emotions3. Cognitive Emotions4. Emotional Experience5. Body-Mind Interactions

See also Picard’s AFFECTIVE COMPUTING FOR HCI

Four HCI Goals in Affective Computing

1. Reducing user frustration– Listening– Empathic– Sympathetic

2. Enabling comfortable communication of user emotion– Voice tone– Sense facial expressions

3. Developing infrastructure and applications to handle affective information

– Affective “wearables”4. Building tools that help develop social-emotional skills

– Autism Research

See Picard’s AFFECTIVE COMPUTING FOR HCI

Designing Sensory Environments

• See Neurosky• Affective Gaming• "The Sinking City of

Atlantis“• Firm adds smell to video

games

• Further viewing• Don Norman - Emotional Design

Tony’s Seminar

Sadness and fear

Slight Sadness

Disgust

Last Week

This week!

Affect Part Two

Textual Analysis of Affective Responses to Open Ended Questions

Watch this video – think of it as an experience

http://www.youtube.com/watch?v=KczxsvVvqGE

Part Two

TASK• Set up a simple user experience (watch a video, play a game, fill in a form…)• Ask at least three (reasonably) open ended questions about the experience and write down

the responses. – What concerns did you have during your experience?– How did you feel during the experience?

• Encourage the user to expand on points relating to how they felt.– Why were you excited?– Was there anything that frustrated you?

• Analyse key parts of the text using (a) a pen to circle positive and negative affective words and (b) using the below free software

• Emotional text recognition using Whissell’s Dictionary of Affective Language (DAL) http://sail.usc.edu/~kazemzad/emotion_in_text_cgi/DAL_app/

Part TwoEmotional text recognition using Whissell’s Dictionary of Affective Language (DAL)

Rating emotional content of words • Positive or Negative Pleasant

(1)>>>In Between (2)>>>Unpleasant (3) • Activation of emotion by size Passive

(1)>>>In Between (2)>>>Active (3)

• Hard to Imagine (1)>>>In Between (2)>>>Easy to Imagine (3)

See Cynthia Whissell academic homepage

distressed distrustful doubtful drowsy dumbfounded eager earnest elated enthusiastic embarrassed embittered envious exasperated excited ecstatic exuberant fascinated fed up forlorn frightened full of contempt full of hatred full of regrets full of reverence furious gloomy grateful grief-stricken guilty happy helpless hesitant homesick hopeful hopeless horrified

scared sensual serene shy sorry stimulated stubborn surprised tender tense tired touched triumphant undecided uneasy unhappy vengeful warmhearted weak worried

affectionate afraid agitated amazed amused angry anguished annoyed anxious apathetic ashamed astonished bored calm carefree cheerful compassionate confident confused contented courageous cowardly curious dejected delighted depressed desirous desperate despondent disappointed discontented discouraged disdainful disgruntled disgusted dismayed

hostile humble ill at ease impatient impressed in high spirits in love indecisive indifferent indignant inspired interested intimidated intolerant jealous joyful jubilant listless lively lonely melancholic nervous nostalgic offended outraged overexcited passionate proud puzzled relaxed relieved repelled resigned respectful sad satisfied

In K. R. Scherer (Ed.) (1988). Facets of emotion: Recent research (pp. 241-243). Hillsdale, NJ: Erlbaum. [Version revised by the members of the Geneva Emotion Research Group]. Copy retrieved [Date] from http://www.unige.ch/fapse/emotion/resmaterial/resmaterial.html

Pleasant (1)>>>In Between (2)>>>Unpleasant (3) Passive (1)>>>In Between (2)>>>Active (3) (activation)Hard to Imagine (1)>>>In Between (2)>>>Easy to Imagine (3)

What can this tell us about the user experience?

What did the user find pleasant?

What did the user find unpleasant?

What affective words elicit active or passive emotional responses?

Part Two

“I was impressed by the interface. However, I felt a little anxious to begin with. A little uneasy at first, kind of nervous, but I became more confident. The bit in the middle made me feel quite tense, but I was excited. Overall I’m very happy with my performance, although I am hated the music.”

Pleasant (1)>>>In Between (2)>>>Unpleasant (3)

Passive (1)>>>In Between (2)>>>Active (3) (activation)Hard to Imagine (1)>>>In Between (2)>>>Easy to Imagine (3)

Watch

• Don Norman: The three ways that good design makes you happy

• http://www.youtube.com/watch?v=RlQEoJaLQRA