An Agent Oriented Programming Language integrating Temporal
Planning and the Plan Coordination Mechanisms
Muhammad Adnan HASHMI
Thesis Supervisor: Amal EL FALLAH SEGHROUCHNI
2
Objectives
AOPL + Planning Planning + Coordination(Temporal Planning, Reactivity, Dynamic Environments) (Temporal Planning, Different Priority Goals)
AOPL integrating Planning, Plan Repairing and Coordination
3
Outline Context and Objectives
Plan Coordination Mechanisms Coordinated Planning Problem Proactive-Reactive Coordination Problem
P-CLAIM: AOP Language supporting Temporal Planning Language Definition Planning Mechanism
Conclusion and Perspectives
4
Outline Context and Objectives
Plan Coordination Mechanisms Coordinated Planning Problem Proactive-Reactive Coordination Problem
P-CLAIM: AOP Language supporting Temporal Planning Language Definition Planning Mechanism
Conclusion and Perspectives
5
AOP Languages Allow to program intelligent and autonomous agents
Main Characteristics Mental State: Beliefs, Goals, Commitments Reasoning Mechanism Capabilities, Services Communication Concurrence
Some Languages Agent-0 [Shoham 1993], 2APL [Dastani 2008], AgentSpeak (L)
[Rao 1996]
6
Objectives (AOP Languages) Current languages
Most AOP languages follow a reactive (PRS based) approach AOP languages do not support temporal planning
Only a few support planning
Problems Execution without planning may result in the goal failures
Agent can reach a dead end Conflicts can arise among different plans Real world actions take place over a timespan
7
Objectives (AOP Languages) Current languages
Most AOP languages follow a reactive (PRS based) approach AOP languages do not support temporal planning
Only a few support planning
Our aim Propose an extension to a programming language to endow it
with planning skills Has temporal planning Deals with uncertainty of the environment Incorporate reactivity by dealing with on the fly goals having
different priorities
8
Related Work CANPLAN [De Silva et al. 2005] combines JSHOP with
JACK Programmer’s responsibility to call the planner Doesn’t deal with uncertainty of the environment
X-BDI [Meneguzzi et al. 2004] incorporates classical planning in a BDI framework Efficiency concerns Loss of the domain information
CYPRESS [Myers et al. 1994] integrates SIPE-2 with PRS
9
Plans Coordination Planning for single agents
Computing plan from Initial State to Goal State
Coordination of plans Removing conflicts (negative interactions) Utilizing help relations (positive interactions)
a
b
c
A1 A2d
e
q
I G
10
Related Work (Plans Coordination) Coordination before planning
Social Laws [Shoham 1995], [Buzzing 2006]
Coordination during planning Partial Global Planning [Durfee and Lesser 1987] Incremental Plan Merging [Alami et al. 1994] Recursive Petri Nets [El Fallah Seghrouchni and Haddad 1996]
Coordination after planning Temporal Plan Merging [Tsamardinos et al. 2000] Merging Hierarchical Plans [von Martial 1992]
11
Objectives (Plans Coordination) Propose plans coordination mechanisms for the plans
having different priorities
Two different scenarios Coordination during planning
Coordinated Planning Problem (CPP) Coordination after planning
Proactive-Reactive Coordination Problem (PRCP)
12
Outline Context and Objectives
Plan Coordination Mechanisms Coordinated Planning Problem Proactive-Reactive Coordination Problem
P-CLAIM: AOP Language supporting Temporal Planning Language Definition Planning Mechanism
Conclusion and Perspectives
13
Assumptions Two agents α and β sharing the same environment
Agent α having higher priority (reactive) goals Agent β having normal priority (proactive) goals
Two possible conflicts among plans Causal link threat Parallel actions interference
14
Two Possible Conflicts Causal Link (A1, A2, p)
Action A1 adds an effect p Action A2 needs this effect No action between A1 and A2 adding p
Causal Link Threat If an action A deletes p and lies between A1 and A2, then A
threatens the causal link (A1, A2, p)
A1 A2p
A ¬p
Threat
15
Two Possible Conflicts Causal Link (A1, A2, p)
Action A1 adds an effect p Action A2 needs this effect No action between A1 and A2 adding p
Parallel Actions Interference Actions A1 and A2 lie in parallel Either one of them deletes the preconditions or add effects of the
other
A1
A2
¬p
p
A1
A2
p
¬p
16
Outline Context and Objectives
Plan Coordination Mechanisms Coordinated Planning Problem Proactive-Reactive Coordination Problem
P-CLAIM: AOP Language supporting Temporal Planning Language Definition Planning Mechanism
Conclusion and Perspectives
17
Coordinated Planning Problem Prerequisite:
Plan Pα of Agent α
Our Aim: Compute a Plan Pβ for Agent β
Has no conflict with Pα
Avails the cooperative opportunities offered by Pα
Solution: Non Temporal Domains µ-SATPLAN Temporal Domains Coordinated-Sapa
18
Coordinated-Sapa Our extension of the well-known temporal planner
Sapa [M. Do & S. Kambhampati 2001] Input
Initial State I A Set of Actions D A Set of Proactive Goals GP with deadlines A Reactive Plan PR
Output A plan PP from I to GP
PP is consistent with PR
OR Deadline Violated Failure + Goal name whose
deadline is violated OR No Solution Possible Failure
SAPA[Do and Kambhampati
2001]A Multi-Objective Heuristic Based
Temporal Planner
20
Sapa
Search through the space of time-stamped states
S=(P,M,,Q,t)
Set <pi,ti> of predicates pi and thetime of their last achievement ti < t.
Set of functions represent resource values.
Set of protectedpersistent conditions.
Event queue containing delayed effects.
Time stamp of S.
21
Sapa
Search through the space of time-stamped states
S=(P,M,,Q,t)
Set <pi,ti> of predicates pi and thetime of their last achievement ti < t.
Set of functions represent resource values.
Set of protectedpersistent conditions.
Event queue containing delayed effects.
Time stamp of S.
22
Main Flow-Diagram of SAPAS=(P,M,,Q,t)
S=(P = I, M, , Q =φ, t = 0)
23
Some Important Concepts Goal Satisfaction: S=(P,M,,Q,t) G if <pi,ti> G either:
<pi,tj> P, tj < ti and no event in Q deletes pi e Q that adds pi at time te < ti
Action Application: Action A is applicable in S =(P,M,,Q,t) if:
All preconditions of A are satisfied by P A’s effects do not interfere with Q
When A is applied to S =(P,M,,Q,t) : P is updated according to A’s instantaneous effects Delayed effects of A are put in Q.
Special Advance-Time Action Advances time to next earliest event in the queue, and adds
the event to P
S=(P,M,,Q,t)
Coordinated-Sapa
An Extension of Sapa that Finds Coordinated Plans in Temporal Domains
25
Handling Positive Interactions Before starting planning for Agent β
Create an event e = (st, et, effect) corresponding to every effect k of every action A in Pα
e.st = st(A) e.et = et(A) e.effect = k
Put all these events in the event queue Q
WHY? At every state, Coordinated-Sapa will take into account all the changes made by Agent α Advanced-Time action will add the effects generated by
Agent α, to P
26
Handling Negative Interactions Add another action applicability condition to the planning
mechanism of Agent β
Action Application: Action A is applicable in S=(P,M,,Q,t) if:
All preconditions of A are satisfied by P A’s effects do not interfere with Q A does not threaten any causal link at t
S=(P,M,,Q,t)
27
Example Plan Generated
Agent β drives person P4 in Car1 to city C2 So that P4 can board the plane Pl1 Agent α is bringing the plane Pl1 to C2 from C3
Agent β makes person P4 board the plane Pl1 at C2 Agent α flies plane Pl1 from C2 to C5
28
Experimental Results Domains and problems taken from 3rd International Planning Competition A multi-agent problem is generated by taking the original problem and dividing the
goals in two sets, one for each agent
29
Outline Context and Objectives
Plan Coordination Mechanisms Coordinated Planning Problem Proactive-Reactive Coordination Problem
P-CLAIM: AOP Language supporting Temporal Planning Language Definition Planning Mechanism
Conclusion and Perspectives
30
Proactive-Reactive Coordination Prerequisite:
Reactive plan Pα of Agent α
Proactive plan Pβ of Agent β
Our Aim: Modify plan Pβ such that:
It has no conflict with Pα
Avails the cooperative opportunities offered by Pα
Solution: Plan Merging Algorithm
31
Case Study Tasks of Rescue Agent
Rescue the Victims Tasks of Analyzer Agent
Analyze the goal cells Call the central agent
Constraints One agent in a cell Hyper energy cells
Needs fuel or energy to enter Agent should have key to open door
Rescue Agent : αAnalyzer
Agent : β
D
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
C
B
A
32
Conflict Resolution Threat-Repair Link (A1, A2, p)
Action A1 deletes p A2 is a subsequent action and adds p
A1 is called Threat Action A2 is called Repair Action
B1 B2p
A1 -p A2 p
Threat
Repair
33
Valid and Possibly Valid Time Stamps Possibly Valid Time Slot for an action A
All preconditions are met No parallel actions interference
P[1] abh P[2] b
c-d P[3] c e P[4] e f P[5] f g
P[1] bd-h
P[6] g i P[7] i h g
Valid Time Slot for an action A All preconditions are met No parallel actions interference Either:
No causal link threat Repair Action exist before the deadline
P[1] abh P[2] b
c-d P[3] c e P[4] e f P[5] f g
P[1] bd-h
P[6] g i P[7] i h g
P[2] d k P[3] k h
34
Plan Merging Algorithm
Fix all the actions of Reactive Plan Pα on timeline
For every action CA of Proactive Plan
Search for the first Possibly Valid Time Slot T on timeline
Reason about the time slot T There could be 5 cases at T
35
Plan Merging AlgorithmCase 1: No causal link threat by CA at T
Assign Time Slot T to CA
EXAMPLE Current Action: Move(A1, A2)
Returned Time Slot: 0 - 5 Any Threat? : No Assign Time Slot 0 – 5 to CA
D
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
C
B
A
36
Plan Merging AlgorithmCase 2: CA threatens a Causal Link but Repair Action exist
Assign Time Slot T to CA Save a Possible Threat <ThreatAction, RepairAction, Deadline>
EXAMPLE Current Action: Move(A4, A5)
Time Slot: 20 - 25 Any Threat? : Yes (Agent α needs A5 at time 40-45) Repair Action: Move(A5, A6) Assign Time Slot 20 - 25 to Move (A4, A5) Save <Move(A4, A5), Move(A5, A6), 40>
D
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
C
B
A
37
Plan Merging AlgorithmCase 3: It is a Repair Action but can not meet a deadline of
some Threat Action Backtrack to the Threat Action,find another time stamp
EXAMPLE Current Action: Move (A8, A9)
Returned Time Slot: 50 - 55 Any Threat?: Yes (Agent α needs A9 at 85-110) Repair Action : Move (A9, B9) Save <Move(A8, A9), Move(A9, B9), 85>
Next Action: AnalyzeCell (A9) Time Slot Assigned: 55 - 70
D
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
C
B
A
38
Plan Merging Algorithm Next Action: CallCentral (A9)
Time Slot Assigned: 80 – 90 Next Action: Move (A9, B9)
Is it a Repair Action? : Yes Meet all deadlines?: No (Agent α needs A9 at 85) Backtrack to action Move(A8, A9)
Find another Time Slot New Time Slot: 110 – 115 (Valid Time Slot)
D
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
C
B
A
Attention
39
Plan Merging AlgorithmCase 4: All the effects of CA are already achieved
WHAT TO DO? Mark CA as redundant
POST PROCESSING Remove all redundant actions from the plan Recursively remove all actions which achieve only the
preconditions of removed action
40
Plan Merging AlgorithmEXAMPLE Current Action: OpenDoor (C11)
Returned Time Slot: 172 - 175 Redundant(OpenDoor(C11)) true
Because openedDoor(C11) is true at time 172
When the plan is returned Remove OpenDoor(C11) from plan Also remove TakeKey(C11, key1) from plan
D
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
C
B
A
41
Plan Merging AlgorithmCase 5: Action CA’s preconditions can not be achieved
Remove action CA from the plan and compute a plan to achieve effects of CA
I = State just before CA G = Effects (CA)
Plan should have no conflict with Reactive Plan Pβ and if CA is a repair action, repair effects must meet their deadline
ReplacementPlan = Coordinated-Sapa (I, G, Pβ)
If a plan is returned, replace the removed actions with the plan
If a deadline is violated, backtrack to the threat action
If no plan possible, then remove another action CA + 1
G = G U Effects (CA + 1) \ Pre (CA + 1)
Use Coordinated-Sapa
42
Plan Merging AlgorithmEXAMPLE Current Action: TakeEnergy(B13, energy1) Preconditions can not be achieved
Repair the plan
D
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
C
B
A
43
Plan Repair Algorithm Create a CPP by removing TakeEnergy(B13, energy1)
I = { at(β, B13), at(energy1, B13), at(energy2, B13) } G = { hasEnergy(β, energy), at(β, B13)}
Call Coordinated-Sapa to solve this CPP Coordinated-Sapa returns fail Why? energy2 is also needed by Agent α
D
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
C
B
A
44
Plan Repair Algorithm Create another CPP by removing Move(B13, A12)
I = { at(β, B13), at(energy1, B13), at(energy2, C15) } G = { at(β, A12) }
Call Coordinated-Sapa to solve this CPP A plan is returned to enter A12 by taking the fuel from D14
POST PROCESSING This plan will become a replacement for both
TakeEnergy(B13, energy1) and Move(B13, A12)
D
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
C
B
A
45
Outline Context and Objectives
Plan Coordination Mechanisms Coordinated Planning Problem Proactive-Reactive Coordination Problem
P-CLAIM: AOP Language supporting Temporal Planning Language Definition Planning Mechanism
Conclusion and Perspectives
46
An AOP language having: Cognitive aspects specific to intelligent agents Communication primitives Mobility primitives
CLAIM Agent: Is autonomous, intelligent and mobile Has a mental state containing knowledge, goals, and capabilities Is able to communicate with other agents Entails a reactive behaviour
CLAIM [Suna and El Fallah Seghrouchni 2005]
47
Agent Definition in CLAIM
defineAgent agentName{
parent = null | agentName ;
knowledge = null ;| {knowledge1; …; knowledgem}
goals = null ;| {goal1; … ; goaln}
capabilities = null ; {capability1 … capabilityo}
agents = null ; | {agName1, agName2, …, agNameq}
}
capability = name { message = null | message ; conditions = null | condition ; do { process } effects = null ;| { effect1 ; …; effectf }
}
48
An AOP language having: Cognitive aspects specific to intelligent agents Communication primitives Mobility primitives Temporal planning capability (New)
P-CLAIM Agent: Is autonomous, intelligent and mobile Has a mental state containing knowledge, goals, and capabilities Is able to communicate with other agents Entails a planning based behaviour (New) Achieves goals based on their priorities (New) Maintains the stability of the plan in the dynamic environments (New)
P-CLAIM
49
Agent Definition in P-CLAIM
Similar to CLAIM, but Added priorities to goals
High Preemptive (reactive): Should be immediately planned for and achieved
High and Normal (proactive): High priority goals should be achieved before normal priority goals
Capabilities in CLAIM (Activities + Actions) in P-CLAIM
Activities are short plans to achieve tasks Actions are primitive tasks with duration
50
Outline Context and Objectives
Plan Coordination Mechanisms Coordinated Planning Problem Proactive-Reactive Coordination Problem
P-CLAIM: AOP Language supporting Temporal Planning Language Definition Planning Mechanism
Conclusion and Perspectives
51
Agent Definition to Planning (Translator)
Translator(JavaCC)
Knowledge Goals Activities Actions
Initial State Goals Methods Operators
Problem File Domain File
Agent Description File
Planner
52
Agent Life Cycle
53
Messages Handler
54
Waits for messages from other agents
Message is a request to achieve a goal? Assigns priority to the goal High Preemptive puts it in Global Reactive Goals (GRG)
queue High or Normal puts it in Global Proactive Goals (GPG) priority
queue
Message is an information? Store the information in the knowledge base of the agent
Messages Handler
55
Goal Priority Assignment
Message Priority
Sender’s Importance
High Normal
High Preemptive High Preemptive High
High High Normal
Normal Normal Normal
56
Planner
57
Temporal Converter
Compute Plan
Main Algorithm
GRG GPG
1- Fetch goals one by one
from GRG and GPG and calls Compute_Plan to compute a plan for the
goal
2- GPG Goals are accessed
only when GRG is empty
3- Sends a suspension
signal to Executor if the
goal is reactive i.e. from GRG
Planner
58
Plan Computation
JSHOP2 algorithm [Nau et el. 2003] is used to compute a totally ordered plan for each goal
An HTN planning algorithm
Decomposes the task into sub-tasks by applying methods
Recursively applies the same procedure on every composite sub-task until there are only primitive tasks
T1
M1
T11 T12
M12M11
59
Temporal Converter
Input to procedure A totally ordered plan Actions’ information (Add, Del, Pre, Durations)
Output of procedure A position constrained parallel plan
Every action is assigned a time stamp Multiple actions can possibly lie in parallel
60
P[1]a
-a+d+e+f
P[2]f-f+g+h
P[3]e +i P[4]h-h+j
P[5]gi
+k P[6]k +l P[7]l +m P[8]m +c P[9]c m
-c P[10]e-k-c+h
I = {a, b, c}
Dur(P[1])=15 Dur(P[2])=15 Dur(P[3])=15 Dur(P[4])=15 Dur(P[5])=15 Dur(P[6])=15 Dur(P[7])=15 Dur(P[8])=15 Dur(P[9])=15 Dur(P[10])=15
Temporal Converter (Example)
61
P[1]a
-a+d+e+f
P[2]f-f+g+h
P[3]e +i
P[4]h-h+j
P[5]gi
+k
P[6]k +l P[7]l +m P[8]m+n+c
P[9]c mn
-c
P[10]e-k-c+h
MaxTPreT MinT
Forbidden Forbidden
142
5
MinOrderingT
3
67
Temporal Converter (Example) P[10]e
-k-c+h
Action Under Consideration:
62
P[1]a
-a+d+e+f
P[2]f-f+g+h
P[3]e +i
P[4]h-h+j
P[5]gi
+k
P[6]k +l P[7]l +m P[8]m+n+c
P[9]c mn
-c
P[10]e-k-c+h
P[1]a
-a+d+e+f
P[2]f-f+g+h
P[3]e +i P[4]h-h+j
P[5]gi
+k P[6]k +l P[7]l +m P[8]m +c P[9]c m
-c P[10]e-k-c+h
Input Plan
Output Plan
Makespan Gain: 30%
Temporal Converter (Example)
63
Experimental Results (Temporal Converter)
64
Merging the New Plan to Global Plan
65
Merging the New Plan to Global Plan
Planner
Append at the end of PexecMerge at the start of Pexec
Proactive GoalReactive Goal
Plan Under Execution (Pexec)
66
Schedule Handler and Executor
67
Schedule Handler and Executor
68
Plan Mender
69
Plan Mender
SP(Pexec[1]) SP(Pexec[2])SP(Pexec[3])
GSP(Pexec[4])
SW
Compute Plan Usin
g Sapa
No Plan
Compute Plan Usin
g Sapa
ReplacementPlan ContinuationPlan
Pexecnew = ReplacementPlan + ContinuationPlan
70
Experimental Results (Plan Mender) To perform tests, we explicitly modified the knowledge base of
the agent to cause plan failure
71
Outline Context and Objectives
Plan Coordination Mechanisms Coordinated Planning Problem Proactive-Reactive Coordination Problem
P-CLAIM: AOP Language supporting Temporal Planning Language Definition Planning Mechanism
Conclusion and Perspectives
72
Conclusion An agent oriented programming language supporting:
Temporal Planning Plan Repairing Dealing with different priority goals
Coordinated Planning Problem Computing plan while coordinating it with another plan
SATPLAN µ-SATPLAN Sapa Coordinated-Sapa
Proactive-Reactive Coordination Problem Modifying a plan to remove conflicts with a higher priority
plan Plan Merging Algorithm
73
Conclusion Properties of the temporal conversion algorithm
Soundness (Proof: By construction) Termination
Properties of the plan merging algorithm Soundness (Verified by experimental evaluation)
Computational Complexity Temporal Converter: Quadratic Plan Merging Algorithm:
Worst Case: Exponential Average Case: Quadratic
74
Perspectives Coordination of plans for same priority goals
Negotiation based strategy
Planning with incomplete information Information gathering actions
Improvement in the computational complexity of plan merging algorithm Propose efficient heuristics to reduce the number of backtracks
75
Publications1. A. El Fallah Seghrouchni, M. A. Hashmi, “Multi-Agent Planning”, Book
chapter in Software Agents, Agent Systems and Their Applications. M. Essaaidi et al. (Eds.), 2012, IOS Press.
2. Y. Dimopoulos, M. A. Hashmi, P. Moraitis, “µ-SATPLAN: Multi-Agent Planning as Satisfiability”, Knowledge Based Systems Journal, 2011
3. M. A. Hashmi, A. El Fallah Seghrouchni, “Merging of Temporal Plans supported by Plan Repairing”, Proceedings of 22nd IEEE International Conference on Tools with Artificial Intelligence (ICTAI 2010), Aras, France
4. M. A. Hashmi, A. El Fallah Seghrouchni, “Coordination of Temporal Plans for the Reactive and Proactive Goals”, Proceedings of IEEE/WIC/ACM International Conference on Intelligent Agent Technology (IAT’ 10), Toronto, Canada
5. Y. Dimopoulos, M. A. Hashmi, P. Moraitis, “Extending SATPLAN to Multiple Agents”, Proceedings of 30th SGAI International Conference on Artificial Intelligence 2010, Cambridge, UK.
6. M. A. Hashmi, A. El Fallah Seghrouchni, “Temporal Planning in Dynamic Environments for P-CLAIM Agents”, In proceedings of Languages, Methodologies and Development Tools for Multi-Agent Systems (LADS’09), Torino, Italy, Springer-Verlag.
76
Publications7. M. A. Hashmi, A. El Fallah Seghrouchni, “Temporal Planning in Dynamic
Environments for Mobile Agents”, Proceedings of International Conference on Frontiers of Information Technology (FIT’ 09), Abottabad, Pakistan, Publisher ACM press.
8. M. A. Hashmi, “A Planning Component for CLAIM Agents”, In the proceedings of 17th International Conference on Control Systems and Computer Science, Bucharest, Romania, Volume 2, pages 485-492, Politehnica Press
77
Thanks for your attention!!!