The use of eye movement in the study of emotionally expressive agent in multimedia environments

Enilda Romero-Hall, Ph.D. (@RomeroEnilda) Assistant Professor Instructional Design & Technology University of Tampa

Purpose To explain the theoretical and empirical studies which support the need for research on emotionally expressive animated agents using visual attention measures.

2

What are animated agents?

3

Benefits of Animated Agents ¡  Positive impact on learners’ motivation and how strongly they

value computer-based learning programs

¡  Help learners focus on important elements of learning materials

¡  Provide learners with context-specific learning strategies and advice

4

Benefits of Animated Agents

¡ Agent can demonstrate physical tasks, such as operation and repair of equipment

¡ Agents can serve as the learners’ attentional guides using the most common and natural methods: gaze and deictic gesture

5

Literature Review

Some studies have found that under certain conditions animated agents may distract learners, overload them cognitively, or even suppress learning

(Baylor & Ryu, 2003; Clark & Choi, 2007; Craig et al., 2002; Dirkin, Mishra, & Altermatt, 2005)

6

Literature Review

There are few studies that highlight the use of an animated agents as a factor contributing to higher learning scores and/or a more positive attitude towards the learning experience

(Arslan-Ari, 2010; Baylor, 2002; Clarebout & Elen, 2006; Kim et al., 2006; Moreno, 2001; Moreno et al., 2000; Ryokai, Vaucelle, & Cassell, 2003; Yung, 2009).

7

Literature Review

The results of the published empirical studies on animated agents are varied and the findings are inconclusive

8

Emotion and Animated Agents

9

Emotion and Animated Agents An entertaining and compelling animated agent must express emotion (Rickel et al., 2002).

The purpose is not just to add animated agents for prettiness, but to leverage learners’ natural tendencies to attribute humanness to the animated agent.

10

Visual Attention Measures in Animated Agent Research

11

Visual Attention Measures in Animated Agent Research

¡  Eye tracking methodology can help empirically substantiate theories of human-computer interactions that assume that humans are inclined to treat computers as social partners when interacting with them (Reeves & Nass, 1996)

¡ One study has investigated how the presence of an educational animated agent in a computer-mediated environment affected the learners’ visual attention (Louwerse et al., 2009)

12

Effects of Emotion in Visual Attention

13

Effects of Emotion in Visual Attention ¡ Visual attention might be directed to certain kinds of events

because of intrinsic meaning or salience (Hunt, 2007). Due to their adaptive relevance, emotional stimuli are expected to be detected quickly (Calvo et al., 2007).

¡  Research on the preferential visual processing of emotional scenes competing for attention resources with neutral scenes revealed that there was preferential processing of emotional scenes (Calvo et al., 2007).

14

Conclusion ¡  Research on emotionally expressive agents, using eye tracking

can provide unique information about what is visually attended to, in what order, and for how long (van Gog & Scheiter, 2010).

¡  Eye tracking data may enhance computer-meditated instruction by significantly improving their design based on knowledge of how learners process certain materials (Hyönä, 2010; Mayer, 2010; van Gog & Scheiter, 2010).

15

Questions or Comments? Contact

Email: eromerohall@ut.edu

LinkedIn: https://www.linkedin.com/in/romeroenilda

Twitter: @RomeroEnilda

16