An Agent Oriented Programming Language integrating Temporal Planning and the Plan Coordination Mechanisms Muhammad Adnan HASHMI Thesis Supervisor: Amal

An Agent Oriented Programming Language integrating Temporal

Planning and the Plan Coordination Mechanisms

Muhammad Adnan HASHMI

Thesis Supervisor: Amal EL FALLAH SEGHROUCHNI

2

Objectives

AOPL + Planning Planning + Coordination(Temporal Planning, Reactivity, Dynamic Environments) (Temporal Planning, Different Priority Goals)

AOPL integrating Planning, Plan Repairing and Coordination

3

Outline Context and Objectives

Plan Coordination Mechanisms Coordinated Planning Problem Proactive-Reactive Coordination Problem

P-CLAIM: AOP Language supporting Temporal Planning Language Definition Planning Mechanism

Conclusion and Perspectives

4





5

AOP Languages Allow to program intelligent and autonomous agents

Main Characteristics Mental State: Beliefs, Goals, Commitments Reasoning Mechanism Capabilities, Services Communication Concurrence

Some Languages Agent-0 [Shoham 1993], 2APL [Dastani 2008], AgentSpeak (L)

[Rao 1996]

6

Objectives (AOP Languages) Current languages

Most AOP languages follow a reactive (PRS based) approach AOP languages do not support temporal planning

Only a few support planning

Problems Execution without planning may result in the goal failures

Agent can reach a dead end Conflicts can arise among different plans Real world actions take place over a timespan

7

Objectives (AOP Languages) Current languages

Most AOP languages follow a reactive (PRS based) approach AOP languages do not support temporal planning

Only a few support planning

Our aim Propose an extension to a programming language to endow it

with planning skills Has temporal planning Deals with uncertainty of the environment Incorporate reactivity by dealing with on the fly goals having

different priorities

8

Related Work CANPLAN [De Silva et al. 2005] combines JSHOP with

JACK Programmer’s responsibility to call the planner Doesn’t deal with uncertainty of the environment

X-BDI [Meneguzzi et al. 2004] incorporates classical planning in a BDI framework Efficiency concerns Loss of the domain information

CYPRESS [Myers et al. 1994] integrates SIPE-2 with PRS

9

Plans Coordination Planning for single agents

Computing plan from Initial State to Goal State

Coordination of plans Removing conflicts (negative interactions) Utilizing help relations (positive interactions)

a

b

c

A1 A2d

e

q

I G

10

Related Work (Plans Coordination) Coordination before planning

Social Laws [Shoham 1995], [Buzzing 2006]

Coordination during planning Partial Global Planning [Durfee and Lesser 1987] Incremental Plan Merging [Alami et al. 1994] Recursive Petri Nets [El Fallah Seghrouchni and Haddad 1996]

Coordination after planning Temporal Plan Merging [Tsamardinos et al. 2000] Merging Hierarchical Plans [von Martial 1992]

11

Objectives (Plans Coordination) Propose plans coordination mechanisms for the plans

having different priorities

Two different scenarios Coordination during planning

Coordinated Planning Problem (CPP) Coordination after planning

Proactive-Reactive Coordination Problem (PRCP)

12





13

Assumptions Two agents α and β sharing the same environment

Agent α having higher priority (reactive) goals Agent β having normal priority (proactive) goals

Two possible conflicts among plans Causal link threat Parallel actions interference

14

Two Possible Conflicts Causal Link (A1, A2, p)

Action A1 adds an effect p Action A2 needs this effect No action between A1 and A2 adding p

Causal Link Threat If an action A deletes p and lies between A1 and A2, then A

threatens the causal link (A1, A2, p)

A1 A2p

A ¬p

Threat

15

Two Possible Conflicts Causal Link (A1, A2, p)

Action A1 adds an effect p Action A2 needs this effect No action between A1 and A2 adding p

Parallel Actions Interference Actions A1 and A2 lie in parallel Either one of them deletes the preconditions or add effects of the

other

A1

A2

¬p

p

A1

A2

p

¬p

16





17

Coordinated Planning Problem Prerequisite:

Plan Pα of Agent α

Our Aim: Compute a Plan Pβ for Agent β

Has no conflict with Pα

Avails the cooperative opportunities offered by Pα

Solution: Non Temporal Domains µ-SATPLAN Temporal Domains Coordinated-Sapa

18

Coordinated-Sapa Our extension of the well-known temporal planner

Sapa [M. Do & S. Kambhampati 2001] Input

Initial State I A Set of Actions D A Set of Proactive Goals GP with deadlines A Reactive Plan PR

Output A plan PP from I to GP

PP is consistent with PR

OR Deadline Violated Failure + Goal name whose

deadline is violated OR No Solution Possible Failure

SAPA[Do and Kambhampati

2001]A Multi-Objective Heuristic Based

Temporal Planner

20

Sapa

Search through the space of time-stamped states

S=(P,M,,Q,t)

Set <pi,ti> of predicates pi and thetime of their last achievement ti < t.

Set of functions represent resource values.

Set of protectedpersistent conditions.

Event queue containing delayed effects.

Time stamp of S.

21

Sapa

Search through the space of time-stamped states

S=(P,M,,Q,t)

Set <pi,ti> of predicates pi and thetime of their last achievement ti < t.

Set of functions represent resource values.

Set of protectedpersistent conditions.

Event queue containing delayed effects.

Time stamp of S.

22

Main Flow-Diagram of SAPAS=(P,M,,Q,t)

S=(P = I, M, , Q =φ, t = 0)

23

Some Important Concepts Goal Satisfaction: S=(P,M,,Q,t) G if <pi,ti> G either:

<pi,tj> P, tj < ti and no event in Q deletes pi e Q that adds pi at time te < ti

Action Application: Action A is applicable in S =(P,M,,Q,t) if:

All preconditions of A are satisfied by P A’s effects do not interfere with Q

When A is applied to S =(P,M,,Q,t) : P is updated according to A’s instantaneous effects Delayed effects of A are put in Q.

Special Advance-Time Action Advances time to next earliest event in the queue, and adds

the event to P

S=(P,M,,Q,t)

Coordinated-Sapa

An Extension of Sapa that Finds Coordinated Plans in Temporal Domains

25

Handling Positive Interactions Before starting planning for Agent β

Create an event e = (st, et, effect) corresponding to every effect k of every action A in Pα

e.st = st(A) e.et = et(A) e.effect = k

Put all these events in the event queue Q

WHY? At every state, Coordinated-Sapa will take into account all the changes made by Agent α Advanced-Time action will add the effects generated by

Agent α, to P

26

Handling Negative Interactions Add another action applicability condition to the planning

mechanism of Agent β

Action Application: Action A is applicable in S=(P,M,,Q,t) if:

All preconditions of A are satisfied by P A’s effects do not interfere with Q A does not threaten any causal link at t

S=(P,M,,Q,t)

27

Example Plan Generated

Agent β drives person P4 in Car1 to city C2 So that P4 can board the plane Pl1 Agent α is bringing the plane Pl1 to C2 from C3

Agent β makes person P4 board the plane Pl1 at C2 Agent α flies plane Pl1 from C2 to C5

28

Experimental Results Domains and problems taken from 3rd International Planning Competition A multi-agent problem is generated by taking the original problem and dividing the

goals in two sets, one for each agent

29





30

Proactive-Reactive Coordination Prerequisite:

Reactive plan Pα of Agent α

Proactive plan Pβ of Agent β

Our Aim: Modify plan Pβ such that:

It has no conflict with Pα

Avails the cooperative opportunities offered by Pα

Solution: Plan Merging Algorithm

31

Case Study Tasks of Rescue Agent

Rescue the Victims Tasks of Analyzer Agent

Analyze the goal cells Call the central agent

Constraints One agent in a cell Hyper energy cells

Needs fuel or energy to enter Agent should have key to open door

Rescue Agent : αAnalyzer

Agent : β

D

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

C

B

A

32

Conflict Resolution Threat-Repair Link (A1, A2, p)

Action A1 deletes p A2 is a subsequent action and adds p

A1 is called Threat Action A2 is called Repair Action

B1 B2p

A1 -p A2 p

Threat

Repair

33

Valid and Possibly Valid Time Stamps Possibly Valid Time Slot for an action A

All preconditions are met No parallel actions interference

P[1] abh P[2] b

c-d P[3] c e P[4] e f P[5] f g

P[1] bd-h

P[6] g i P[7] i h g

Valid Time Slot for an action A All preconditions are met No parallel actions interference Either:

No causal link threat Repair Action exist before the deadline

P[1] abh P[2] b

c-d P[3] c e P[4] e f P[5] f g

P[1] bd-h

P[6] g i P[7] i h g

P[2] d k P[3] k h

34

Plan Merging Algorithm

Fix all the actions of Reactive Plan Pα on timeline

For every action CA of Proactive Plan

Search for the first Possibly Valid Time Slot T on timeline

Reason about the time slot T There could be 5 cases at T

35

Plan Merging AlgorithmCase 1: No causal link threat by CA at T

Assign Time Slot T to CA

EXAMPLE Current Action: Move(A1, A2)

Returned Time Slot: 0 - 5 Any Threat? : No Assign Time Slot 0 – 5 to CA

D

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

C

B

A

36

Plan Merging AlgorithmCase 2: CA threatens a Causal Link but Repair Action exist

Assign Time Slot T to CA Save a Possible Threat <ThreatAction, RepairAction, Deadline>

EXAMPLE Current Action: Move(A4, A5)

Time Slot: 20 - 25 Any Threat? : Yes (Agent α needs A5 at time 40-45) Repair Action: Move(A5, A6) Assign Time Slot 20 - 25 to Move (A4, A5) Save <Move(A4, A5), Move(A5, A6), 40>

D

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

C

B

A

37

Plan Merging AlgorithmCase 3: It is a Repair Action but can not meet a deadline of

some Threat Action Backtrack to the Threat Action,find another time stamp

EXAMPLE Current Action: Move (A8, A9)

Returned Time Slot: 50 - 55 Any Threat?: Yes (Agent α needs A9 at 85-110) Repair Action : Move (A9, B9) Save <Move(A8, A9), Move(A9, B9), 85>

Next Action: AnalyzeCell (A9) Time Slot Assigned: 55 - 70

D

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

C

B

A

38

Plan Merging Algorithm Next Action: CallCentral (A9)

Time Slot Assigned: 80 – 90 Next Action: Move (A9, B9)

Is it a Repair Action? : Yes Meet all deadlines?: No (Agent α needs A9 at 85) Backtrack to action Move(A8, A9)

Find another Time Slot New Time Slot: 110 – 115 (Valid Time Slot)

D

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

C

B

A

Attention

39

Plan Merging AlgorithmCase 4: All the effects of CA are already achieved

WHAT TO DO? Mark CA as redundant

POST PROCESSING Remove all redundant actions from the plan Recursively remove all actions which achieve only the

preconditions of removed action

40

Plan Merging AlgorithmEXAMPLE Current Action: OpenDoor (C11)

Returned Time Slot: 172 - 175 Redundant(OpenDoor(C11)) true

Because openedDoor(C11) is true at time 172

When the plan is returned Remove OpenDoor(C11) from plan Also remove TakeKey(C11, key1) from plan

D

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

C

B

A

41

Plan Merging AlgorithmCase 5: Action CA’s preconditions can not be achieved

Remove action CA from the plan and compute a plan to achieve effects of CA

I = State just before CA G = Effects (CA)

Plan should have no conflict with Reactive Plan Pβ and if CA is a repair action, repair effects must meet their deadline

ReplacementPlan = Coordinated-Sapa (I, G, Pβ)

If a plan is returned, replace the removed actions with the plan

If a deadline is violated, backtrack to the threat action

If no plan possible, then remove another action CA + 1

G = G U Effects (CA + 1) \ Pre (CA + 1)

Use Coordinated-Sapa

42

Plan Merging AlgorithmEXAMPLE Current Action: TakeEnergy(B13, energy1) Preconditions can not be achieved

Repair the plan

D

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

C

B

A

43

Plan Repair Algorithm Create a CPP by removing TakeEnergy(B13, energy1)

I = { at(β, B13), at(energy1, B13), at(energy2, B13) } G = { hasEnergy(β, energy), at(β, B13)}

Call Coordinated-Sapa to solve this CPP Coordinated-Sapa returns fail Why? energy2 is also needed by Agent α

D

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

C

B

A

44

Plan Repair Algorithm Create another CPP by removing Move(B13, A12)

I = { at(β, B13), at(energy1, B13), at(energy2, C15) } G = { at(β, A12) }

Call Coordinated-Sapa to solve this CPP A plan is returned to enter A12 by taking the fuel from D14

POST PROCESSING This plan will become a replacement for both

TakeEnergy(B13, energy1) and Move(B13, A12)

D

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

C

B

A

45





46

An AOP language having: Cognitive aspects specific to intelligent agents Communication primitives Mobility primitives

CLAIM Agent: Is autonomous, intelligent and mobile Has a mental state containing knowledge, goals, and capabilities Is able to communicate with other agents Entails a reactive behaviour

CLAIM [Suna and El Fallah Seghrouchni 2005]

47

Agent Definition in CLAIM

defineAgent agentName{

parent = null | agentName ;

knowledge = null ;| {knowledge1; …; knowledgem}

goals = null ;| {goal1; … ; goaln}

capabilities = null ; {capability1 … capabilityo}

agents = null ; | {agName1, agName2, …, agNameq}

}

capability = name { message = null | message ; conditions = null | condition ; do { process } effects = null ;| { effect1 ; …; effectf }

}

48

An AOP language having: Cognitive aspects specific to intelligent agents Communication primitives Mobility primitives Temporal planning capability (New)

P-CLAIM Agent: Is autonomous, intelligent and mobile Has a mental state containing knowledge, goals, and capabilities Is able to communicate with other agents Entails a planning based behaviour (New) Achieves goals based on their priorities (New) Maintains the stability of the plan in the dynamic environments (New)

P-CLAIM

49

Agent Definition in P-CLAIM

Similar to CLAIM, but Added priorities to goals

High Preemptive (reactive): Should be immediately planned for and achieved

High and Normal (proactive): High priority goals should be achieved before normal priority goals

Capabilities in CLAIM (Activities + Actions) in P-CLAIM

Activities are short plans to achieve tasks Actions are primitive tasks with duration

50





51

Agent Definition to Planning (Translator)

Translator(JavaCC)

Knowledge Goals Activities Actions

Initial State Goals Methods Operators

Problem File Domain File

Agent Description File

Planner

52

Agent Life Cycle

53

Messages Handler

54

Waits for messages from other agents

Message is a request to achieve a goal? Assigns priority to the goal High Preemptive puts it in Global Reactive Goals (GRG)

queue High or Normal puts it in Global Proactive Goals (GPG) priority

queue

Message is an information? Store the information in the knowledge base of the agent

Messages Handler

55

Goal Priority Assignment

Message Priority

Sender’s Importance

High Normal

High Preemptive High Preemptive High

High High Normal

Normal Normal Normal

56

Planner

57

Temporal Converter

Compute Plan

Main Algorithm

GRG GPG

1- Fetch goals one by one

from GRG and GPG and calls Compute_Plan to compute a plan for the

goal

2- GPG Goals are accessed

only when GRG is empty

3- Sends a suspension

signal to Executor if the

goal is reactive i.e. from GRG

Planner

58

Plan Computation

JSHOP2 algorithm [Nau et el. 2003] is used to compute a totally ordered plan for each goal

An HTN planning algorithm

Decomposes the task into sub-tasks by applying methods

Recursively applies the same procedure on every composite sub-task until there are only primitive tasks

T1

M1

T11 T12

M12M11

59

Temporal Converter

Input to procedure A totally ordered plan Actions’ information (Add, Del, Pre, Durations)

Output of procedure A position constrained parallel plan

Every action is assigned a time stamp Multiple actions can possibly lie in parallel

60

P[1]a

-a+d+e+f

P[2]f-f+g+h

P[3]e +i P[4]h-h+j

P[5]gi

+k P[6]k +l P[7]l +m P[8]m +c P[9]c m

-c P[10]e-k-c+h

I = {a, b, c}

Dur(P[1])=15 Dur(P[2])=15 Dur(P[3])=15 Dur(P[4])=15 Dur(P[5])=15 Dur(P[6])=15 Dur(P[7])=15 Dur(P[8])=15 Dur(P[9])=15 Dur(P[10])=15

Temporal Converter (Example)

61

P[1]a

-a+d+e+f

P[2]f-f+g+h

P[3]e +i

P[4]h-h+j

P[5]gi

+k

P[6]k +l P[7]l +m P[8]m+n+c

P[9]c mn

-c

P[10]e-k-c+h

MaxTPreT MinT

Forbidden Forbidden

142

5

MinOrderingT

3

67

Temporal Converter (Example) P[10]e

-k-c+h

Action Under Consideration:

62

P[1]a

-a+d+e+f

P[2]f-f+g+h

P[3]e +i

P[4]h-h+j

P[5]gi

+k

P[6]k +l P[7]l +m P[8]m+n+c

P[9]c mn

-c

P[10]e-k-c+h

P[1]a

-a+d+e+f

P[2]f-f+g+h

P[3]e +i P[4]h-h+j

P[5]gi

+k P[6]k +l P[7]l +m P[8]m +c P[9]c m

-c P[10]e-k-c+h

Input Plan

Output Plan

Makespan Gain: 30%

Temporal Converter (Example)

63

Experimental Results (Temporal Converter)

64

Merging the New Plan to Global Plan

65

Merging the New Plan to Global Plan

Planner

Append at the end of PexecMerge at the start of Pexec

Proactive GoalReactive Goal

Plan Under Execution (Pexec)

66

Schedule Handler and Executor

67

Schedule Handler and Executor

68

Plan Mender

69

Plan Mender

SP(Pexec[1]) SP(Pexec[2])SP(Pexec[3])

GSP(Pexec[4])

SW

Compute Plan Usin

g Sapa

No Plan

Compute Plan Usin

g Sapa

ReplacementPlan ContinuationPlan

Pexecnew = ReplacementPlan + ContinuationPlan

70

Experimental Results (Plan Mender) To perform tests, we explicitly modified the knowledge base of

the agent to cause plan failure

71





72

Conclusion An agent oriented programming language supporting:

Temporal Planning Plan Repairing Dealing with different priority goals

Coordinated Planning Problem Computing plan while coordinating it with another plan

SATPLAN µ-SATPLAN Sapa Coordinated-Sapa

Proactive-Reactive Coordination Problem Modifying a plan to remove conflicts with a higher priority

plan Plan Merging Algorithm

73

Conclusion Properties of the temporal conversion algorithm

Soundness (Proof: By construction) Termination

Properties of the plan merging algorithm Soundness (Verified by experimental evaluation)

Computational Complexity Temporal Converter: Quadratic Plan Merging Algorithm:

Worst Case: Exponential Average Case: Quadratic

74

Perspectives Coordination of plans for same priority goals

Negotiation based strategy

Planning with incomplete information Information gathering actions

Improvement in the computational complexity of plan merging algorithm Propose efficient heuristics to reduce the number of backtracks

75

Publications1. A. El Fallah Seghrouchni, M. A. Hashmi, “Multi-Agent Planning”, Book

chapter in Software Agents, Agent Systems and Their Applications. M. Essaaidi et al. (Eds.), 2012, IOS Press.

2. Y. Dimopoulos, M. A. Hashmi, P. Moraitis, “µ-SATPLAN: Multi-Agent Planning as Satisfiability”, Knowledge Based Systems Journal, 2011

3. M. A. Hashmi, A. El Fallah Seghrouchni, “Merging of Temporal Plans supported by Plan Repairing”, Proceedings of 22nd IEEE International Conference on Tools with Artificial Intelligence (ICTAI 2010), Aras, France

4. M. A. Hashmi, A. El Fallah Seghrouchni, “Coordination of Temporal Plans for the Reactive and Proactive Goals”, Proceedings of IEEE/WIC/ACM International Conference on Intelligent Agent Technology (IAT’ 10), Toronto, Canada

5. Y. Dimopoulos, M. A. Hashmi, P. Moraitis, “Extending SATPLAN to Multiple Agents”, Proceedings of 30th SGAI International Conference on Artificial Intelligence 2010, Cambridge, UK.

6. M. A. Hashmi, A. El Fallah Seghrouchni, “Temporal Planning in Dynamic Environments for P-CLAIM Agents”, In proceedings of Languages, Methodologies and Development Tools for Multi-Agent Systems (LADS’09), Torino, Italy, Springer-Verlag.

76

Publications7. M. A. Hashmi, A. El Fallah Seghrouchni, “Temporal Planning in Dynamic

Environments for Mobile Agents”, Proceedings of International Conference on Frontiers of Information Technology (FIT’ 09), Abottabad, Pakistan, Publisher ACM press.

8. M. A. Hashmi, “A Planning Component for CLAIM Agents”, In the proceedings of 17th International Conference on Control Systems and Computer Science, Bucharest, Romania, Volume 2, pages 485-492, Politehnica Press

77

Thanks for your attention!!!

Documents

An Agent Oriented Programming Language integrating Temporal Planning and the Plan Coordination Mechanisms Muhammad Adnan HASHMI Thesis Supervisor: Amal