1

Title

A Joint Intention Based Dialog Architecture

Rajah. A SubramanianUniversity of Rochester

2

Need for a Generic Dialog Engine Current spoken dialog systems

Domain based systems. Need to explicitly code all possible rules and conditions for

prompting, informing etc. Very difficult to port from one domain to another, breaks

easily and cannot be used for complex task-based scenarios.

We propose an architecture that can reason in a general manner and can replace many of the rules and condition that are explicit in current dialog systems. Handle over-answering, multi-agent interaction, and certain

dynamic changes in the environment. Build Systems that are robust and domain independent.

3

Dialog Process

We are concerned here in the Dialog Management Strategy.

LanguageUnderstanding

Dialog Management.Strategy

Back-end(various)

LanguageGeneration

SpeechRecognitio

n

SpeechSynthesis

4

Outline of the talk

Overview Joint Intentions Theory Semantics of Communicative Acts Belief Reasoner STAPLE interpreter

Room Reservation Demo

Summary, Future & Related work

5

Joint Intention Theory (JIT) - I Belief, Desire, Intention (BDI) Model –

Bratman/Israel/Pollack, Georgeff/Lansky … Joint Intention Theory (JIT) (Cohen, Levesque). Uses modal logic (beliefs, goals), dynamic

logic of actions (sequence, concurrent actions, etc.), temporal logic (eventually, always).

INTENTION – Choice with commitment. PGOAL x – Individual persistent goal of an

agent. Once adopted, the agent is committed to it until achieved, impossible or irrelevant.

6

Joint Intention Theory (JIT) - II PWAG x y – Joint goal between x and y.

PGOAL to achieve the goal relative to PWAG. if already achieved, impossible or irrelevant, then PGOAL

to mutually make it known to concerned agents. Teamwork: JPG x y – Two way PWAG’s/ X has a PWAG towards Y

and Y towards X. Prescribes how to execute actions jointly while being in a shared

mental state Requires starting mutual belief (MB) Requires MB about task completion, impossibility, or

irrelevance It predicts communication in the event of private beliefs

7

Semantics of Communicative Acts

Formal compositional semantics of commonly used speech acts using JIT constructs REQUEST and INFORM are the basic CA’s.

The goal of a request (x y a ): y eventually does the action a and also have a PWAG with respect to the requester x to do a.

The goal of an inform (x y p) is to establish MB between the informee (x) and the informed (y) that x believes the informed predicate (p).

The other CA’s such as ASK, AGREE, INFORMREF etc. are built from REQUEST and INFORM. For example, ASK is an REQUEST to INFORMREF.

KNOWREF – To know the referent of the description of a required value.

8

Communicative Actions (cont..)

Identified speech acts that can be used to create and discharge teams

Showed how to achieve mutual belief Explicated defeasible assumptions

about communication STAPLE (Sanjeev Kumar) –

Implements the constructs JI theory and interprets PGOAL’s, PWAG’s, Communicative Acts, etc.

Can you please

help me move the

table?

Sure

REQUEST + AGREE= Joint

Commitment

Back

9

STAPLE Architecture (Kumar 2006)

BeliefBase (BB)

TriggerM anager

ConsistencyChecker

BeliefR easoner

M AIN INTER PR ETER

INTEN DInterpreter

PG O ALInterpreter

PW AGInterpreter

AC TIO N / PLAN Library

RuleM anager

Belie fM ain-

tanence

EXTERN AL W O R LD

C om m .M anager

N etw orkO bserver

10

STAPLE Optimization Original version – Java only

Not efficient for unification, that constitutes more than 80% of execution time.

To optimize – combined Java and Prolog, completed as part of RPE project. Java for threads, stacks, triggers etc. Prolog for knowledge base, unification etc. More than 150 times faster for single-agent and 70 times faster for

multi-agents.

0

200

400

600

Normalized Execution Time per

Query

Multi-Agent Single-Agent

Execution Mode

Execution Speed Comparison

JavaJava & Prolog

11

Planning Relevant Questions – Snapshot of a Goal Stack in STAPLE

INTEND (agent, request (agent, user, action (informref (user, agent, i (D, date (Room,D)))*

PGOAL (agent, done (action (ask-ref ( agent, user, i(D, date (Room, D))* PGOAL (agent, (KNOW-REF agent, i (D,date(Room,D))) *

PGOAL(agent, Room (reserved(Room,Date,Time,Size))

PWAG(agent, user, user (done(reserved(Room, Date, Time,…))))

*The PGOAL is actually a bit more complex – the system wants to know the referent of the date that the user wants the room reservation; also the sub-goals are relative to the original goal.

12

STAPLE: Multi-agent Dialog Example

Room reservation domain Three Microsoft agent characters play the roles of User, System, &

Database agent

Peedy(SYSTEM)

Robby(USER)

Merlin(DATABASE AGENT)

13

Multi-Agent Dialog Example - Setup Robby (USER)

1. Initial state: - Believes that it wants a room at a particular date & time

2. Actions: - Cannot perform any actions other than communicative acts

Peedy (SYSTEM)1. Initial state:

- Believes that the user knows the date & time for reserving the room- Believes that the database agent can query for rooms and hold a room if

necessary.2. Actions:

- Can perform action reserve room that happens to be a named action expression (i.e., a plan)

Merlin (DATABASE AGENT)1. Initial state:

- No domain related beliefs.2. Actions:

- Can perform actions to query the database and to hold a room.

14

Handling Over-Answering

PGOAL (KNOW-REF i (D,date(Room,D)))

Agent: Please say the date you want the roomUser: I would like it on March 20th at Three PM

Bel(user, /\ ([March20 = i(Date, date (Room, Date))),3PM = i(Time, time (Room, Time)))])).

Bel(user, March20 = i(Date, date (Room, Date))))

Bel(user, 3PM = i(Time, time (Room, Time)))).

PGOAL (KNOW-REF i (T,time(Room,T)))

PGOAL (KNOW-REF i (S,size(Room,S)))

PGOAL Discharged

PGOAL Discharged

15

Multi-Agent Interaction

PGOAL(system-agent, done(action (getrooms, X,E),….

System Agent USEROriginal JPG to

reserve a room

Bel(system-agent, cando(getrooms, database-agent))

… (Action Expression Decomposition)

DatabaseAgent

16

Multi-Agent Interaction

PGOAL(system-agent, done(action (getrooms, X,E),….

INTEND(system, action(establish_jpg([system, database],done(action(getrooms(……relative to PWAG reserveroom

…..

INTEND(system, action(request(system,[system, database], action(getrooms, ……

DatabaseAgent

REQUEST

System Agent USEROriginal JPG to

reserve a room

17

Multi-Agent Interaction

System Agent USEROriginal JPG to

reserve a room

JPG to do getroomsAGREE

DatabaseAgent

REQUEST

Back

18

Handling Dynamic Changes

System Agent USER

JPG to do holdroom

DatabaseAgent

Original JPG to reserve a room

INTEND(database,action(holdroom,database)..Relative to original PWAG to reserve a room

JPG to do holdroom becomes impossible

PGOAL(database, mb([system, database],always(not(done(action(holdroom(…..

INTEND(database, done(action(inform(database,system,always(not(done(action(holdroom(…..

INFORM – not able to hold the room

Room is TAKEN!!!

Conversation between user and system to confirm the room

19

Handling Dynamic Changes

System Agent USER

JPG to do holdroom

DatabaseAgent

Original JPG to reserve a room

JPG to do holdroom becomes impossibe

Check if there are other rooms that are available or inform user that no rooms are available.

20

Demo

Back

21

The Point We get automatic team and task-related communication

behavior If agent believes that somebody else knows an answer then

initiates dialogue to find out that answer. If an agent believes that somebody else can do the action

that it wanted done then it establishes a team for that purpose

Dialogue about mutual belief establishment follows automatically.

No mutual observation leads to explicit communication, otherwise explicit communication is absent.

The above behavior can be chained to elicit even more interesting behavior. Example: A sub-dialogue may get started in the middle of

joint action execution to find the truth value of a proposition if it is the precondition for doing a part of the jointly committed action.

22

Future Work

In Progress Implicit Confirmations Explicit Clarifications

Still Mulling over… Partial and weak beliefs Belief update and revision

23

Related Work

Dialogue & Communication Collagen: MERL – Rich, Sidner, Lesh Artimis & Arcol: France Telecom - Sadek,

Panaget, Bretier,… TRIPS: University of Rochester – Allen,

Ferguson, … FIPA, KQML

Teamwork STEAM & Teamcore: USC – Tambe et al. Grate: Liverpool, UK - Jennings)

24

Appendix A: STAPLE Program – Multi Agent Interaction and Dynamic Changes bel(system,can_do(getrooms(Room,List), database)). bel(system,exists(D,bel(user,equals(D,i(Date,date(Room,Date)))))).

plan(reserveroom,4) :- [args: [Room, [Date, Time], E1,E2], achieve_precondition: true, effects: [(roomReserved(Room,[Date,Time]),1.0)], body: { action(roominfo([date(Room, Date), time(Room,

Time)]),system,E1)}].

plan(roominfo,1) :- [args: [[date(Room, Date), time(Room, Time)]], effects: [(roomsAvailable(Room, [Date, Time]),1.0)], body: { action(getrooms(Room, [Date, Time]),X, Ev1)}].

25

Appendix B: Belief Reasoner

(BEL self p) :- p(BEL α p) :- (BEL α □p)(BEL α p) :- (BEL α p)(BEL α (BEL α p)) :- (BEL α p)(BEL α p):- (BEL α p)(BEL α p) :- \+ (BEL α p)

(BEL α p) :- (BEL α p)(BEL α □□p):-(BEL α □p)(BEL α p):- (BEL α p)(BEL α p) :- (BEL α q=>p) /\ (BEL α q)(BEL α p/\q) :- (BEL α p) /\ (BEL α q)(BEL α □(p/\q)) :- (BEL α □p) (BEL α □q)

26

Appendix C: Mutual Belief Reasoner (BEL self p) :- (MB self y p) (BEL self (BEL y p)) :- (MB self y p) (BMB self y p) :- (MB self y p) (BMB y self p) :- (MB self y p) (BEL self (MB self y (BEL self p))) :- (MB self y p) (BEL self (MB self y (BEL y p))) :- (MB self y p) (BEL self (MB self y p/\q)) :- (BEL self (MB self y p)) /\ (BEL self (MB self y q)) (BEL self (MB self y □(p/\q))) :- (BEL self (MB self y □p)) (BEL self (MB self y □q))