1
Title
A Joint Intention Based Dialog Architecture
Rajah. A SubramanianUniversity of Rochester
2
Need for a Generic Dialog Engine Current spoken dialog systems
Domain based systems. Need to explicitly code all possible rules and conditions for
prompting, informing etc. Very difficult to port from one domain to another, breaks
easily and cannot be used for complex task-based scenarios.
We propose an architecture that can reason in a general manner and can replace many of the rules and condition that are explicit in current dialog systems. Handle over-answering, multi-agent interaction, and certain
dynamic changes in the environment. Build Systems that are robust and domain independent.
3
Dialog Process
We are concerned here in the Dialog Management Strategy.
LanguageUnderstanding
Dialog Management.Strategy
Back-end(various)
LanguageGeneration
SpeechRecognitio
n
SpeechSynthesis
4
Outline of the talk
Overview Joint Intentions Theory Semantics of Communicative Acts Belief Reasoner STAPLE interpreter
Room Reservation Demo
Summary, Future & Related work
5
Joint Intention Theory (JIT) - I Belief, Desire, Intention (BDI) Model –
Bratman/Israel/Pollack, Georgeff/Lansky … Joint Intention Theory (JIT) (Cohen, Levesque). Uses modal logic (beliefs, goals), dynamic
logic of actions (sequence, concurrent actions, etc.), temporal logic (eventually, always).
INTENTION – Choice with commitment. PGOAL x – Individual persistent goal of an
agent. Once adopted, the agent is committed to it until achieved, impossible or irrelevant.
6
Joint Intention Theory (JIT) - II PWAG x y – Joint goal between x and y.
PGOAL to achieve the goal relative to PWAG. if already achieved, impossible or irrelevant, then PGOAL
to mutually make it known to concerned agents. Teamwork: JPG x y – Two way PWAG’s/ X has a PWAG towards Y
and Y towards X. Prescribes how to execute actions jointly while being in a shared
mental state Requires starting mutual belief (MB) Requires MB about task completion, impossibility, or
irrelevance It predicts communication in the event of private beliefs
7
Semantics of Communicative Acts
Formal compositional semantics of commonly used speech acts using JIT constructs REQUEST and INFORM are the basic CA’s.
The goal of a request (x y a ): y eventually does the action a and also have a PWAG with respect to the requester x to do a.
The goal of an inform (x y p) is to establish MB between the informee (x) and the informed (y) that x believes the informed predicate (p).
The other CA’s such as ASK, AGREE, INFORMREF etc. are built from REQUEST and INFORM. For example, ASK is an REQUEST to INFORMREF.
KNOWREF – To know the referent of the description of a required value.
8
Communicative Actions (cont..)
Identified speech acts that can be used to create and discharge teams
Showed how to achieve mutual belief Explicated defeasible assumptions
about communication STAPLE (Sanjeev Kumar) –
Implements the constructs JI theory and interprets PGOAL’s, PWAG’s, Communicative Acts, etc.
Can you please
help me move the
table?
Sure
REQUEST + AGREE= Joint
Commitment
Back
9
STAPLE Architecture (Kumar 2006)
BeliefBase (BB)
TriggerM anager
ConsistencyChecker
BeliefR easoner
M AIN INTER PR ETER
INTEN DInterpreter
PG O ALInterpreter
PW AGInterpreter
AC TIO N / PLAN Library
RuleM anager
Belie fM ain-
tanence
EXTERN AL W O R LD
C om m .M anager
N etw orkO bserver
10
STAPLE Optimization Original version – Java only
Not efficient for unification, that constitutes more than 80% of execution time.
To optimize – combined Java and Prolog, completed as part of RPE project. Java for threads, stacks, triggers etc. Prolog for knowledge base, unification etc. More than 150 times faster for single-agent and 70 times faster for
multi-agents.
0
200
400
600
Normalized Execution Time per
Query
Multi-Agent Single-Agent
Execution Mode
Execution Speed Comparison
JavaJava & Prolog
11
Planning Relevant Questions – Snapshot of a Goal Stack in STAPLE
INTEND (agent, request (agent, user, action (informref (user, agent, i (D, date (Room,D)))*
PGOAL (agent, done (action (ask-ref ( agent, user, i(D, date (Room, D))* PGOAL (agent, (KNOW-REF agent, i (D,date(Room,D))) *
PGOAL(agent, Room (reserved(Room,Date,Time,Size))
PWAG(agent, user, user (done(reserved(Room, Date, Time,…))))
*The PGOAL is actually a bit more complex – the system wants to know the referent of the date that the user wants the room reservation; also the sub-goals are relative to the original goal.
12
STAPLE: Multi-agent Dialog Example
Room reservation domain Three Microsoft agent characters play the roles of User, System, &
Database agent
Peedy(SYSTEM)
Robby(USER)
Merlin(DATABASE AGENT)
13
Multi-Agent Dialog Example - Setup Robby (USER)
1. Initial state: - Believes that it wants a room at a particular date & time
2. Actions: - Cannot perform any actions other than communicative acts
Peedy (SYSTEM)1. Initial state:
- Believes that the user knows the date & time for reserving the room- Believes that the database agent can query for rooms and hold a room if
necessary.2. Actions:
- Can perform action reserve room that happens to be a named action expression (i.e., a plan)
Merlin (DATABASE AGENT)1. Initial state:
- No domain related beliefs.2. Actions:
- Can perform actions to query the database and to hold a room.
14
Handling Over-Answering
PGOAL (KNOW-REF i (D,date(Room,D)))
Agent: Please say the date you want the roomUser: I would like it on March 20th at Three PM
Bel(user, /\ ([March20 = i(Date, date (Room, Date))),3PM = i(Time, time (Room, Time)))])).
Bel(user, March20 = i(Date, date (Room, Date))))
Bel(user, 3PM = i(Time, time (Room, Time)))).
PGOAL (KNOW-REF i (T,time(Room,T)))
PGOAL (KNOW-REF i (S,size(Room,S)))
PGOAL Discharged
PGOAL Discharged
15
Multi-Agent Interaction
PGOAL(system-agent, done(action (getrooms, X,E),….
System Agent USEROriginal JPG to
reserve a room
Bel(system-agent, cando(getrooms, database-agent))
… (Action Expression Decomposition)
DatabaseAgent
16
Multi-Agent Interaction
PGOAL(system-agent, done(action (getrooms, X,E),….
INTEND(system, action(establish_jpg([system, database],done(action(getrooms(……relative to PWAG reserveroom
…..
INTEND(system, action(request(system,[system, database], action(getrooms, ……
DatabaseAgent
REQUEST
System Agent USEROriginal JPG to
reserve a room
17
Multi-Agent Interaction
System Agent USEROriginal JPG to
reserve a room
JPG to do getroomsAGREE
DatabaseAgent
REQUEST
Back
18
Handling Dynamic Changes
System Agent USER
JPG to do holdroom
DatabaseAgent
Original JPG to reserve a room
INTEND(database,action(holdroom,database)..Relative to original PWAG to reserve a room
JPG to do holdroom becomes impossible
PGOAL(database, mb([system, database],always(not(done(action(holdroom(…..
INTEND(database, done(action(inform(database,system,always(not(done(action(holdroom(…..
INFORM – not able to hold the room
Room is TAKEN!!!
Conversation between user and system to confirm the room
19
Handling Dynamic Changes
System Agent USER
JPG to do holdroom
DatabaseAgent
Original JPG to reserve a room
JPG to do holdroom becomes impossibe
Check if there are other rooms that are available or inform user that no rooms are available.
20
Demo
Back
21
The Point We get automatic team and task-related communication
behavior If agent believes that somebody else knows an answer then
initiates dialogue to find out that answer. If an agent believes that somebody else can do the action
that it wanted done then it establishes a team for that purpose
Dialogue about mutual belief establishment follows automatically.
No mutual observation leads to explicit communication, otherwise explicit communication is absent.
The above behavior can be chained to elicit even more interesting behavior. Example: A sub-dialogue may get started in the middle of
joint action execution to find the truth value of a proposition if it is the precondition for doing a part of the jointly committed action.
22
Future Work
In Progress Implicit Confirmations Explicit Clarifications
Still Mulling over… Partial and weak beliefs Belief update and revision
23
Related Work
Dialogue & Communication Collagen: MERL – Rich, Sidner, Lesh Artimis & Arcol: France Telecom - Sadek,
Panaget, Bretier,… TRIPS: University of Rochester – Allen,
Ferguson, … FIPA, KQML
Teamwork STEAM & Teamcore: USC – Tambe et al. Grate: Liverpool, UK - Jennings)
24
Appendix A: STAPLE Program – Multi Agent Interaction and Dynamic Changes bel(system,can_do(getrooms(Room,List), database)). bel(system,exists(D,bel(user,equals(D,i(Date,date(Room,Date)))))).
plan(reserveroom,4) :- [args: [Room, [Date, Time], E1,E2], achieve_precondition: true, effects: [(roomReserved(Room,[Date,Time]),1.0)], body: { action(roominfo([date(Room, Date), time(Room,
Time)]),system,E1)}].
plan(roominfo,1) :- [args: [[date(Room, Date), time(Room, Time)]], effects: [(roomsAvailable(Room, [Date, Time]),1.0)], body: { action(getrooms(Room, [Date, Time]),X, Ev1)}].
25
Appendix B: Belief Reasoner
(BEL self p) :- p(BEL α p) :- (BEL α □p)(BEL α p) :- (BEL α p)(BEL α (BEL α p)) :- (BEL α p)(BEL α p):- (BEL α p)(BEL α p) :- \+ (BEL α p)
(BEL α p) :- (BEL α p)(BEL α □□p):-(BEL α □p)(BEL α p):- (BEL α p)(BEL α p) :- (BEL α q=>p) /\ (BEL α q)(BEL α p/\q) :- (BEL α p) /\ (BEL α q)(BEL α □(p/\q)) :- (BEL α □p) (BEL α □q)
26
Appendix C: Mutual Belief Reasoner (BEL self p) :- (MB self y p) (BEL self (BEL y p)) :- (MB self y p) (BMB self y p) :- (MB self y p) (BMB y self p) :- (MB self y p) (BEL self (MB self y (BEL self p))) :- (MB self y p) (BEL self (MB self y (BEL y p))) :- (MB self y p) (BEL self (MB self y p/\q)) :- (BEL self (MB self y p)) /\ (BEL self (MB self y q)) (BEL self (MB self y □(p/\q))) :- (BEL self (MB self y □p)) (BEL self (MB self y □q))