1

MPML andMPML and SCREAM: Scripting the Bodies SCREAM: Scripting the Bodies and Minds of Life-Like Charactersand Minds of Life-Like Characters

Soft computing LaboratoryYonsei University

October 27, 2004

2

AgendaAgenda

sub-title

• Introduction

• Related work– Scripting the Bodies of Life-Like Characters– Scripting the Minds of Life-Like Characters

• Multi-modal Presentation Markup Language (MPML)

• Scripting Emotion-based Agent Minds (SCREAM)

• Conclusion

3

IntroductionIntroduction• Communication of humans

– Language– Body gesture– Facial expressions

• Character-based interfaces– An effective tools to enhance the interaction between humans

and computers

• Two categories in designing life-like characters– To instruct the “body” of an animated character– To design the mental make-up of a character

4

Scripting the Bodies of Life-Like CharactersScripting the Bodies of Life-Like Characters• Scripting

– Low level Manipulating a character’s facial and body motion models

– High level The synchronization of a character’s synthetic speech with appropriate

gestures The expression of emotions The synchronization of body motion sequences of an ensemble of

characters

• Existing scripting languages– CML (Character Markup Language)– VHML (Virtual Human Markup Language)– BEAT (Behavior Expression Animation Toolkit)– APML (Affective Presentation Markup Language)– Hard to use for non-experts

• MPML (Multi-modal Presentation Markup Language)– Designed for non-experts

5

Scripting the Minds of Life-Like CharactersScripting the Minds of Life-Like Characters• A task complementary to scripting a character’s visual

appearance– To author a character’s mental state– The way the character perceives its surrounding world– How the character reacts emotionally

• SCREAM (Scripting Emotion-based Agent Minds)– Similar to Reilly’s EM architecture– But more flexible in the sense– Emotion-based synthetic characters can be run in a web browser

like the Extermpo and IMP systems

6

Multi-Modal Presentation Markup LanguageMulti-Modal Presentation Markup Language• What is MPML?

– A scripting language to instruct the body-related behavior of animated agents

• The Features of MPML– Ease of use– Intelligibility– Believability– Extensibility– Easy Distribution

7

The MPML FamilyThe MPML Family

• MPML 2.2a– MPML 2.2a supports

sequential and parallel behavior of multiple charactersan interface to Macromedia Flash

• DWML– DWML is concerned with scripting dynamic relations between

web-based media objects

• MPML-VR– MPML-VR enables presentations in 3D space

8

MPML System ArchitectureMPML System Architecture• Fig 3

9

MPML System Architecture (cont.)MPML System Architecture (cont.)• MPML Script text file

– Containing the tagging structures specifying character and environment control

• MPML 3.0 Visual Editor– Script Loader module

Load the text fileCheck the script for syntactical errors

– Graph moduleVisualize the script by generating a graphical representation of

the presentation– Script Saver module

Convert the graph to a textual MPML script– Converter module

Transforms the MPML script to JavaScript

10

MPML SpecificationMPML Specification• Using DTD (document type definition) for readability• Element

– Consisted the name and child elements– If no child elements exist for a tag, it is called “empty”

• Attribute– Name-value pairs separated by “=”– An attribute value may be required or fixed, denoted by

#REQUIRED and #FIXED• Entity declarations

– (e1, e2, …, eN) : An ordered list of child elements– (e1 | e2 | … | eN) : An unordered list of child elements– e? : zero or one time– e+ : one or more times– e* : zero or more times– e : exactly on time

11

Tagging Structures for Character ControlTagging Structures for Character Control• DTD for character-related elements

12

Tagging Structures for Presentation ControlTagging Structures for Presentation Control• DTD for presentation-related elements

13

MPML Visual EditorMPML Visual Editor

14

Using MPML to Script an Interactive PresentationUsing MPML to Script an Interactive Presentation

15

Scripting Emotion-based Agent MindsScripting Emotion-based Agent Minds• What is SCREAM?

– A tool to script the interactive behavior of affective agents

• The key properties of SCREAM– Ease of use– Intelligibility– Believability– Extensibility– Granularity

16

SCREAM System ArchitectureSCREAM System Architecture

17

Communicative ActsCommunicative Acts• Emotion-based agents receive input in the form of

communicative acts

• Written as com_act (S, H, Concept, Sit)– S: speaker– H: the addressee (interlocutor)– Concept: the information conveyed by S to H– Sit: situation

SCREAM System ArchitectureSCREAM System Architecture

18

Emotion GenerationEmotion Generation• Appraisal Module

– Emotion types22 classes: joy, distress, happy for, sorry for, resent, angry at,

and so on. Intensity: δi ∊{0, …, 5}

– PersonalityTwo dimensions: agreeableness and extraversion ∊{-5, …, 5}

• Emotion Resolution Module– Select the active emotions in Sit– from a set {<E1, δ1, Sit>, <E2, δ2, Sit>, …, <En, δn, Sit>}

• Emotion Maintenance– Decrease the intensity by the agent’s personality

SCREAM System ArchitectureSCREAM System Architecture

19

Emotion RegulationEmotion Regulation• Social Variables

– Social powersocial-power(L2,L1,θP,Sit)

agent L2 is θP ranks higher than L1 in Sit, as perceived by L1

– social distancesocial-distance((L1,L2,θD,Sit)

The closeness between two agents(L1,L2)

Lower values indicate more closeness, as perceived by L1

SCREAM System ArchitectureSCREAM System Architecture

20

Emotion Regulation (cont.)Emotion Regulation (cont.)• Control Variables

– PersonalityAn agent is more likely to express negative emotions if it is either

disagreeable or extrovert

– Other agent’s personality If the agent assumes that the interlocutor’s personality is

disagreeable, it will rater not express a negative emotion

– Reciprocal feedbackThere are “negative” feedback loops where agents adjust to

linguistic unfriendly behavior

• Emotion Filter Module– ε = δ – (θ + )

SCREAM System ArchitectureSCREAM System Architecture

21

Emotion ExpressionEmotion Expression• Expressive responses

– Somatic responses (flushing)– Behavioral responses (fist-clenching, throwing objects)– Two types of communicative responses, verbal and non-verbal

(for instance, frowning)

• Information-processing responses– The agent’s diversion of attention and evaluations

SCREAM System ArchitectureSCREAM System Architecture

22

Agent ModelAgent Model• Character profile

• Affect processing

–

– ∊ { + , - }, L : agent, I : interlocutor

– If (δ+ - δ-) > 0, agent L like agent I

– Otherwise agent L dislike agent I

SCREAM System ArchitectureSCREAM System Architecture

n

SitSit

n

i

ILnIL

n

1

,,

)()(

23

Using SCREAM to Script a Conversational CharacterUsing SCREAM to Script a Conversational Character

24

Using SCREAM to Script a Conversational CharacterUsing SCREAM to Script a Conversational Character

• Dialogue for five consecutive rounds where the user never follows Djinn’s advice. The figure does not show the user’s interaction with the dealer

25

ConclusionConclusion• MPML

– A powerful and easy-to-use character scripting tool for web content experts who are non-programmers

• SCREAM– A practical technology for scripting the mental processes

underlying a character’s affective behavior

• Future research– The design of an architecture that allows for a tighter integration

of character and environment scripting– The development of a plot manager that produces narratively

meaningful interactive stories