1
Research ques+ons: Which spa*otemporal quali*es of arm gestures convey emo0on? How are these cues subjec*vely represented in percep*on? Are different cues related to different emo*on dimensions? Predic+ons: Major dimensions of emo0on expression can be perceived and influence different perceived spa0otemporal quali0es of arm gesture. High arousal ac*ve, fast Pleasant gesture percep*on fluent, located high in space High potency forceful, expansive What makes bodily expressions emo+onally salient? Perceptual ra+ng of emo+onal expressions Conclusions Scale Verbal description R SB Arousal Physical excitement in response to the situation 0.98 Potency Power/control over the situation 0.91 Valence Pleasantness of emotional experience 0.89 Intensity Impact of situation on person 0.97 Emo+on dimensions References [1] Dael, N., Mor0llaro, M., & Scherer, K. R., (in press). Emo0on expression in body ac0on and posture. Emo0on. [2] Pollick, F. E., Paterson, H. M., Bruderlin, A.,& Sanford, A. J. (2001). Perceiving affect from arm movement. Cogni0on, 82, B51B61. [3] Fontaine, J. R. J. Scherer, K. R., Roesch, E. B., & Ellsworth, P., C. (2007). The world of emo0ons is not twodimensional. Psychological Science, 18, 10501057. [4] Scherer, K. R. (2001). Appraisal considered as a process of mul0level sequen0al checking. In K. R. Scherer, A. Schorr and T. Johnstone (Eds.). Appraisal processes in emo0on: Theory, Methods, Research (pp. 92120). New York and Oxford: Oxford University Press. [5]Bänziger, T., Mor0llaro, M., & Scherer, K. R. (in press). Introducing the Geneva mul0modal expression corpus for experimental research on emo0on percep0on.Emo*on. S*muli Geneva Mul0modal Emo0on Portrayals corpus (GEMEP)[5] 12 emo0ons x 10 professional actors (N = 120 video clips) Felt experience enactment of social interac+on scenarios Mul0modal expression face: blurred voice: muted pseudo sentence body: cospeech gestures Raters 43 undergraduate students (10 males) Valence Potency high/low Posi0ve Nega0ve Arousal High Elated joy Amusement Pride Hot anger Panic fear Despair Low Pleasure Relief Interest Cold anger Anxiety Sadness Scale Verbal descrip*on and scale labels R SB Amount Amount of arm movement: ac0ve / passive 0.99 Speed Speed by which the arms are moving:slow / fast 0.99 Force Physical effort put in the arm movement: strong / weak 0.98 Size Spa0al extent or span of arm movement: compact / expansive 0.98 Fluency Smoothness of speed changes: abrupt / fluent 0.94 Height Ver0cal loca0on of the arms: low / high 0.96 Gesture dynamics 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 Nega0ve Posi0ve Mean Speed Valence High Potency Nega0ve Posi0ve Valence Low Potency High Arousal Low Arousal 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 Nega0ve Posi0ve Mean Force Valence High Arousal Nega0ve Posi0ve Valence Low Arousal High Potency Low Potency Both emo0on dimensions and gesture dynamics can be reliably perceived Arousal and potency strongly affect all perceived spa0otemporal quali0es of gesture Arousal boosts the effects of potency on gesture percep0on Valence has a subtle effect gesture that is difficult to disentangle from the effects of arousal and potency Gesture is an integrated part of nonverbal emo*on communica*on, not only serving cogni*velinguis*c but also affec*veexpressive func*ons Body movement (gait, dance, posture, everyday ac0on) conveys reliable informa0on about the emo0onal state of the expressor [1], which can be recognized by others [2]. Limle is known on the func0on of gesture to express and perceive emo0on. Emo+on is encoded in movement categories but also in dynamic quali0es [1, 2]. Arousal is posi0vely related to movement speed, accelera0on and jerk, but no other indicators have been found for emo0on dimensions or components [2]. At least three func+onal dimensions are relevant for emo0on expression and experience: Arousal/ac0va0on, Valence/pleasantness, and Potency (power/control) [3, 4]. Design Emo0ons grouped according to level of arousal (A), valence (V), and potency (P) Effects of A, V, P on ra0ngs of arm movement emo0on dimensions ANOVA with A, V, P as within subjects variables (2 levels for each scale) Table 1. Interrater reliability of emo0on scales Table 2. Interrater reliability of movement scales 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 Arousal Valence Potency Mean ra*ng of matching dimension high low high low pos neg 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 High potency Low potency Mean Height Valence High Arousal High potency Low potency Valence Low Arousal Nega0ve Posi0ve p< .001, η 2 = .93 p< .001, η 2 = .66 p< .001, η 2 = .44

ESCOP poster final...Research(quesons:" • Which"spa*otemporal"quali*es"of#arm#gestures#convey#emo0on?## • How#are#these#cues#subjecvely"represented"in"percep*on? • Aredifferentcues"related#to#differentemo*on"dimensions?

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: ESCOP poster final...Research(quesons:" • Which"spa*otemporal"quali*es"of#arm#gestures#convey#emo0on?## • How#are#these#cues#subjecvely"represented"in"percep*on? • Aredifferentcues"related#to#differentemo*on"dimensions?

Research  ques+ons:  •  Which  spa*otemporal  quali*es  of  arm  gestures  convey  emo0on?    •  How  are  these  cues  subjec*vely  represented  in  percep*on?    •  Are  different  cues  related  to  different  emo*on  dimensions?  

 Predic+ons:  Major  dimensions  of  emo0on  expression  can  be  perceived  and  influence  different  perceived  spa0otemporal  quali0es  of  arm  gesture.      

High  arousal        ac*ve,  fast  Pleasant              gesture  percep*on  fluent,  located  high  in  space  High  potency      forceful,  expansive  

What  makes  bodily  expressions  emo+onally  salient?  

Perceptual  ra+ng  of  emo+onal  expressions  

Conclusions  

Scale Verbal description RSB

Arousal Physical excitement in response to the situation 0.98

Potency Power/control over the situation 0.91

Valence Pleasantness of emotional experience 0.89

Intensity Impact of situation on person 0.97

Emo+on  dimensions  

References  [1]  Dael,  N.,  Mor0llaro,  M.,  &  Scherer,  K.  R.,  (in  press).  Emo0on  expression  in  body  ac0on  and  posture.  Emo0on.  

[2]  Pollick,  F.  E.,  Paterson,  H.  M.,  Bruderlin,  A.,&  Sanford,  A.  J.  (2001).  Perceiving  affect  from  arm  movement.  Cogni0on,  82,  B51-­‐B61.  

[3]  Fontaine,  J.  R.  J.  Scherer,  K.  R.,  Roesch,  E.  B.,  &  Ellsworth,  P.,  C.  (2007).  The  world  of  emo0ons  is  not  two-­‐dimensional.  Psychological  Science,  18,  1050-­‐1057.  

[4]  Scherer,  K.  R.  (2001).  Appraisal  considered  as  a  process  of  mul0level  sequen0al  checking.  In  K.  R.  Scherer,  A.  Schorr  and  T.  Johnstone  (Eds.).  Appraisal  processes  in  emo0on:  Theory,  Methods,  Research  (pp.  92-­‐120).  New  York  and  Oxford:  Oxford  University  Press.  

[5]Bänziger,  T.,  Mor0llaro,  M.,  &  Scherer,  K.  R.  (in  press).  Introducing  the  Geneva  mul0modal  expression  corpus  for  experimental  research  on  emo0on  percep0on.Emo*on.  

 

S*muli  Geneva  Mul0modal  Emo0on  Portrayals  corpus  (GEMEP)[5]  12  emo0ons  x  10  professional  actors  (N  =  120  video  clips)      Felt  experience  enactment  of  social  interac+on  scenarios    Mul0modal  expression  

 face:  blurred    voice:  muted  pseudo  sentence    body:  co-­‐speech  gestures  

Raters  43  undergraduate  students  (10  males)  

Valence  

Potency  high/low  

Posi0ve   Nega0ve  

Arou

sal  

High  

 Elated  joy  Amusement  

Pride    

 Hot  anger  Panic  fear  Despair  

 

Low  

 Pleasure  Relief  Interest  

 

 Cold  anger  Anxiety  Sadness  

 

Scale   Verbal  descrip*on  and  scale  labels   RSB  

Amount   Amount  of  arm  movement:  ac0ve  /  passive   0.99  

Speed   Speed  by  which  the  arms  are  moving:slow  /  fast   0.99  

Force   Physical  effort  put  in  the  arm  movement:  strong  /  weak   0.98  

Size   Spa0al  extent  or  span  of  arm  movement:  compact  /  expansive   0.98  

Fluency   Smoothness  of  speed  changes:  abrupt  /  fluent   0.94  

Height   Ver0cal  loca0on  of  the  arms:  low  /  high   0.96  

Gesture  dynamics  

0.00  

0.10  

0.20  

0.30  

0.40  

0.50  

0.60  

0.70  

0.80  

Nega0ve   Posi0ve  

Mean  Speed  

Valence  

High  Potency  

Nega0ve   Posi0ve  Valence  

Low  Potency  High  Arousal  

Low  Arousal  

0.00  

0.10  

0.20  

0.30  

0.40  

0.50  

0.60  

0.70  

0.80  

Nega0ve   Posi0ve  

Mean  Force  

Valence  

High  Arousal  

Nega0ve   Posi0ve  Valence  

Low  Arousal  High  Potency  

Low  Potency  

•  Both  emo0on  dimensions  and  gesture  dynamics  can  be  reliably  perceived  •  Arousal  and  potency  strongly  affect  all  perceived  spa0otemporal  quali0es  of  gesture  

•  Arousal  boosts  the  effects  of  potency  on  gesture  percep0on    

•  Valence  has  a  subtle  effect  gesture  that  is  difficult  to  disentangle  from  the  effects  of  arousal  and  potency  

•  Gesture  is  an  integrated  part  of  nonverbal  emo*on  communica*on,  not  only  serving  cogni*ve-­‐linguis*c  but  also  affec*ve-­‐expressive  func*ons  

Body  movement  (gait,  dance,  posture,  everyday  ac0on)  conveys  reliable  informa0on  about  the  emo0onal  state  of  the  expressor  [1],  which  can  be  recognized  by  others  [2].  Limle  is  known  on  the  func0on  of  gesture  to  express  and  perceive  emo0on.    Emo+on  is  encoded  in  movement  categories  but  also  in  dynamic  quali0es  [1,  2].  Arousal  is  posi0vely  related  to  movement  speed,  accelera0on  and  jerk,  but  no  other  indicators  have  been  found  for  emo0on  dimensions  or  components  [2].      At  least  three  func+onal  dimensions  are  relevant  for  emo0on  expression  and  experience:  Arousal/ac0va0on,  Valence/pleasantness,  and  Potency  (power/control)  [3,  4].    

Design  Emo0ons  grouped  according  to  level  of  arousal  (A),  valence  (V),  and  potency  (P)    Effects  of  A,  V,  P  on  ra0ngs  of  •  arm  movement  •  emo0on  dimensions    ANOVA  with  A,  V,  P  as  within-­‐subjects  variables  (2  levels  for  each  scale)  

Table  1.  Interrater  reliability  of  emo0on  scales  Table  2.  Interrater  reliability  of  movement  scales  

0.00  

0.10  

0.20  

0.30  

0.40  

0.50  

0.60  

0.70  

0.80  

Arousal   Valence   Potency  

Mean  ra*n

g  of  m

atching  dimen

sion

 

high   low   high   low  pos   neg  0.00  

0.10  

0.20  

0.30  

0.40  

0.50  

0.60  

0.70  

0.80  

High  potency   Low  potency  

Mean  He

ight  

Valence  

High  Arousal  

High  potency   Low  potency  

Valence  

Low  Arousal  Nega0ve  

Posi0ve  

p<  .001,  η2  =  .93    

p<  .001,  η2  =  .66    

p<  .001,  η2  =  .44