Humaine Kaiser

8/3/2019 Humaine Kaiser

1/33

First HUMAINE Workshop, Geneva June 17-19,

2004

Possible solutions from models of emotional

expression ???

Susanne KaiserGeneva Emotion Research Group, University of Geneva

*This research was supported by grants from the Swiss National Science Foundation (FNRS

11-39551.93/049629.96 "Dynamic man-computer interactions as a paradigm in emotion

research") granted to S. Kaiser and T. Wehrle.
http://www.clicktoconvert.com/


2/33

Outline of the presentation

1. The complexity of facial expression

2. Ekman & Friesens Facial Action Coding

System

3. Discrete emotion theory and facial expression

4. Appraisal theory and facial expression

5. Examples of empirical studies in our research

group

6. Conclusions for animating facial expressions


3/33

The multi-functionality of facial behavior

A smile or a frown, for instance, can have different meanings.

It can be :

a speech-regulation signal

a speech-related signal

a means for signaling relationship

an indicator for cognitive processes

an indicator for an emotion.

A given facial expression can have several meanings at the

same time.


4/33

Expression controlDisplay rules:de-intensifying

affectless or neutral (poker face).

masking

over-intensifying.

Facial indicators of emotional processes are often verysubtle

Approaches to measure facial expressions objectively and

on a micro-analytic level are indispensable.


5/33

Facial Action Coding System (Ekman, P. & Friesen,W., 1978; Ekman, P., Friesen, W. V., & Hager, J. C., 2002)

http://face-and-emotion.com/dataface/facs/description.jsp

FACS is anatomically based and allows the reliable coding ofany facial action in terms of the smallest visible unit of muscular

activity. These smallest visible units are called Action Units

(AU), each referred to by a numerical code.

With FACS, data collection is independent of data interpretation.

There is no relation between the code and the meaning of thefacial action.

As a consequence, coding is independent of prior assumptions

about prototypical emotion expressions.


6/33

Source: Ekman, P., & Friesen, W. V. (1978). The Facial Action Coding System: A technique

for the measurement of facial movement . Palo Alto: Consulting Psychologist Press.


7/33


8/33

Discrete emotion theory and facial expression

Ekman & Friesen have presented emotion

prediction tables that list the Action Unit

combinations that build up the postulated

prototypical facial expressions of basic emotions

(EMFACS).


9/33

Examples for happiness, surprise, anger, and disgust

Source: Picard, R. W. (1997).Affective computing (p. 177). Cambridge: The MIT Press.


10/33

Major problems with the prototypical expression patterns

are:

These prototypes or major variants do only very rarely

occur during spontaneous interactions.

The meaning of a facial expression depends to a high

degree on the situational context (verbal, other

nonverbal channels, behavior of the interaction

partner, timing).

There exists a high inter-individual variability. Manyof the facial expressions shown by a person are

person-specific.


11/33

Componential Appraisal Theory

Appraisal theorists following a componential approach share

the assumption that:

a) emotions are elicited by a cognitive evaluation of

antecedent situations and events and that

b) the patterning of the reactions in the different response

domains (physiology, expression, action tendencies,

feeling) is determined by the outcome of this evaluation

process.


12/33

Componential Appraisal Theory

They argue that there are a large number of highly differentiated

emotional states, of which the current emotion labels capture

only clusters or central tendencies of regularly recurring ones,

referred to as modal emotions. In line with this reasoning, facial expressions are analyzed as

indicators of appraisal processes in addition to or as an

alternative to verbal report measures. Facial expressions are not

seen as the readout of motor programs but as indicators of

mental states and evaluation processes.


13/33

Appraisal

dimensions

Hot Anger AUs Elation AUs

Suddenness high AU1b + AU2b + AU25 high/medium AU1b + AU2b + AU25

Familiarity low AU4a + AU7

Predictability low AU4b low

Intrinsic

Pleasantness

high AU6 + 12b

Concern

relevance

high high

Outcome

probability

very high very high

Expectation low low

Conduciveness obstruct AU4b + AU7b + AU17b +

AU23b

very high AU6 + AU12d

Urgency high intensification, high tension low

Cause agent otherCause motive intent chance/intent

Control high

Power high AU17c + AU24

Adjustment high medium

External norms low AU10c high

Internal norms low high
http://www.clicktoconvert.com/http://www.clicktoconvert.com/


14/33

Wehrle, T., Kaiser, S., Schmidt, S. & Scherer, K. R (2000). Studying the dynamics of emotional expression via

synthesized facial muscle movements.Journal of Personality and Social Psychology , 78 (1), 105-119.

Hot Anger


15/33

The Geneva Appraisal Manipulation

Environment (GAME)

The Geneva Appraisal Manipulation Environment (GAME; Wehrle 1996)

is a tool for generating experimental computer games that translate

psychological theories into specific micro-world scenarios. The creation of micro world scenarios is based on theoretical predictions

concerning emotion antecedent appraisal and emotion specific action

tendencies as postulated by different componential appraisal theorists

(Scherer, 1988; Frijda, 1986).

Furthermore, game scenarios have been related to motivational

dispositions and corresponding coping strategies as postulated by Bischof

(1985).


16/33

Example of a maze like game level created with GAME


17/33

Can situational appraisal profiles differentiate

between different types of identically labeled

emotions?

We can describe at least 3 types of anger:

being angry as an reaction to an unfair event butwithout blaming anybody

being angry and blaming somebody else for

having caused it on purpose

being angry and blaming the other as well asoneself


18/33

An example of anger type I

We see the reaction of a subject to amigos

intervention reducing speed at the fast level.

She reports anger.


19/33


As can be seen in the following appraisal profile, the participant

evaluates the situation as very sudden, very new, very unpleasant,

and as not at all expected.

Additionally, she thinks that although the situation was difficult to

control - she had enough power to handle it, that she can easily

adjust to its consequences, and that her behaviour was adequate

(referred to as Selfin the figure).

She does not blame anybody or circumstances, however she

evaluates the situation as being very unfair (referred to as Norm

(-) in the figure).


20/33

being angry as an reaction to an unfair event but

without blaming anybody

Cognitive Appraisal of Subject 2 in Situation: Speed Reduction

Reported Emotion: Anger

0

1

2

3

4

5

6

Sudd

ennes

s

Nove

lty

Plea

santnes

s

Releva

nce

Outcom

e

Expect

ation

Obstru

ct

Urgen

cy

AgentSelf

IntentSelf

AgentO

ther

IntentO

ther

Cha

nce

Cont

rol

Po

wer

Adjustm

ent

Norm

(-)

Self

Appraisal Components

Value


21/33


As can bee seen in the next figure, she reacts by raising her

eyebrows (AU1 + AU2), only. Within an emotion-based approach,

this action unit combination could not be linked to anger.

The duration of AU1 and AU2, however, is rather long, i.e. 2.04

seconds. The mean duration of AU1 and AU2 in these studies is

1.04 seconds. Whereas the results show that short innervation of

AU1 and AU2 are linked to the appraisal dimension of

unexpectedness, in the case of the example shown in the figure, the

holding of the expression can be interpreted as an indicator ofappraising the situation as unfair.


22/33


23/33

An example of anger type II

We see the reaction of another subject to

amigos intervention reducing speed at the

fast level. She reports anger.


24/33

being angry and blaming somebody else for having

caused it on purpose

Cognitive Appraisal of Subject 6 in Situation 9: Speed Reduction


0

1

2

3

4

5

6

Sud

Nov

Plea

s

Relev

Outc

ome

Expe

ct

Obstruct

Urge

ncy

Agtslf

Intslf

Agtoth

Intoth

Chan

ce

Control

Powe

rAd

j

Norm

oth

Self


value

This watermark does not appear in the registered version - http://www.clicktoconvert.com


25/33

being angry and blaming somebody else for having

caused it on purpose

pp g p


26/33

An example of anger type III

In the following example, the subject is reacting to

JANUS, an animated figure in the game that up to

this moment had always helped the player byendowing her with the force to defeat the enemies.

Unexpectedly, and for the first time, JANUS

misbehaves and takes power away, thus making

the player's agent a sitting duck for the enemies.She reports anger.


27/33

being angry and blaming the other as well as

oneself

Cognitive Appraisal of Subject 21 in Situation 10: Janus Misbehaving


0

1

2

3

4

5

6

Sudd

enne

ss

Nove

lty

Plea

sant

ness

Relev

ance

Outco

me

Expe

ctatio

n

Obstr

uct

Urge

ncy

Agen

tSelf

Inte

ntSelf

Agen

tOth

er

Inte

ntOth

er

Chan

ce

Cont

rol

Powe

r

Adjus

tmen

t

Norm

(-)

Self


Value Subjects 21


28/33

An example of anger type III The rapid sequence of eyebrow raising, followed by an immediate

frown, can be explained as the output of the sequential processing of

the event as being novel/unexpected and as creating an obstruction.

The change from raised eyebrows to the furrowed brow occurs

within two frames (0.08 seconds) and the whole brow movement

lasts less than a second.

After one second the subject shows AU14 (dimpler), an Action Unit

that tightens the mouth corners and produces a dimplelike wrinkle

beyond the lip corners. This could be interpreted as an indicator of

frustration and as a reaction to the evaluation of JANUS behavior as

being unfair.

This watermark does not appear in the registered version - http://www.clicktoconvert.com


29/33

being angry and blaming the other as well as oneself


30/33

Conclusions for animating facial expressions

I tried to show how an appraisal-based approach might

help us to better understand how emotions are expressed

and perceived.

With respect to human-computer interactions, these twoprocesses refer to the synthesis of emotional agents and

the analysis of a users facial expression.


31/33


From the beginning of the early eighties appraisal theories

have given important inputs to emotion synthesis and

affective user modeling (e.g., OCC).

Appraisal theory is commonly used to implement thecognitive component of an emotional interface.

The outcome of the appraisal process is then mapped into

an emotion category, which determines the synthesis of the

respective facial expression pattern.


32/33


This might be an unnecessary and complicated procedure

that also reduces the available information.

Furthermore, linking facial expression and other

components of emotion directly to appraisal dimensionscan take advantage of the computational representations

that are commonly already defined in the respective

applications.


33/33

References

Kaiser, S. (2002). Facial expressions as indicators of "functional" and "dysfunctional"

emotional processes. In M. Katsikitis (Ed.), The Human Face: Measurement and Meaning (pp.235-254). Dordrecht: Kluwer.

Kaiser, S. & Wehrle, T. (2001a). The Role of Facial Expression in Intra-individual and Inter-individual Emotion Regulation. In D. Caamero (Ed.)Emotional and Intelligent II: TheTangled Knot of Cognition . Papers from the 2001 AAAI Fall Symposium. Technical Report FS-01-02 (pp. 61-66). Menlo Park, CA: AAAI Press.

Kaiser, S. & Wehrle, T. (2001b). Facial expressions as indicator of appraisal processes. In K. R.

Scherer, A. Schorr, & T. Johnstone (Eds.),Appraisal theories of emotions: Theories, methods,research (pp. 285-300). New York: Oxford University Press.

Wehrle, T. (1995/1999). The Facial Action Composing Environment (FACE) [Unpublishedcomputer software]. University of Geneva, Switzerland.

Wehrle, T. (1996). The Geneva Appraisal Manipulation Environment (GAME) [Unpublishedcomputer software]. University of Geneva, Switzerland.

Wehrle, T., Kaiser, S., Schmidt, S. & Scherer, K. R (2000). Studying the dynamics of emotionalexpression via synthesized facial muscle movements.Journal of Personality and SocialPsychology , 78 (1), 105-119.

For online publications and more details see:

http://www.unige.ch/fapse/emotion/members/kaiser/kaiser.html

Documents

Humaine Kaiser