Automated Planning & Artificial Intelligence - Planning graphs

Automated Planning & Artificial IntelligencePlanning graphs & Graphplan algorithm

Humbert Fiorino

[email protected]://membres-lig.imag.fr/fiorino

Laboratory of Informatics of Grenoble – MAGMA team

March 2011

Planning graph search Mutual exclusion relation Graphplan

1 Planning graph search

2 Mutual exclusion relation

3 Graphplan

Humbert Fiorino Automated Planning & Artificial Intelligence 2 / 46


A simplified Dock-Worker Robot problem

loc1 loc2

robr

robq

conta

contb



DWR operators

(operator MOVE(params (<r> ROBOT) (<l> LOCATION) (<l’>

LOCATION))(preconds

(at <r> <l>)(adjacent <l> <l’>))(effects

(at <r> <l’>)(del at <r> <l>)))



DWR operators

(operator LOAD(params (<r> ROBOT) (<l> LOCATION) (<c>

CONTAINER))(preconds

(at <r> <l>)(in <c> <l>)(unloaded <r>))(effects

(loaded <r> <c>)(del in <c> <l>)(del unloaded <r>)))

(operator UNLOAD(params (<r> ROBOT) (<l> LOCATION) (<c>

CONTAINER))(preconds

(at <r> <l>)(loaded <r> <c>))(effects

(unloaded <r>)(in <c> <l>)(del loaded <r> <c>)))



To simplify representation, let us denote atoms by propositionalsymbols :

r1 and r2 stand for (at robr loc1) and (at robr loc2)q1 and q2 stand for (at robq loc1) and (at robq loc2)a1, a2, ar and aq stand for (int conta loc1), (in conta loc2), (loadedconta robr) and (loaded conta robq)b1, b2, br and bq stand for (int contb loc1), (in contb loc2), (loadedcontb robr) and (loaded contb robq)



Let us also denote the 20 actions in A :Mr12 is the action move(robr, loc1, loc2), Mr21 is the opposite, andMq12 and Mq21 are the similar move action for robot robqLar1 is the action load(conta, robr, loc1), Lar2, Laq1 and Laq2 are theother load actions for conta in loc2 with contb. Lbr1, Lbr2, Lbq1 andLbq2 are the load actions for contbUar1, Uar2, Uaq1, Uaq2, Ubr1, Ubr2, Ubq1, and Ubq2 are the unloadactions



Reachability tree

r1q2a1b2uruq

r2q2a1b2uruq

r1q1a1b2uruq

r1q2arb2uq

r1q2a1bqur

Mr12 Mq21 Lar1 Lbq2

r1q2a1b2uruq

r2q1a1b2uruq

r2q2a1brur

r2q2a1bquq

r2q2a1bqur

r1q1a1bqur

r1q2arbq

r1q2a1b2uruq

...Mr12Mq21 Lbr2 Lbq2

Mr12Mq21 Lar1 Ubq2



Reachabilty graph

r1q2a1b2uruq

r2q2a1b2uruq

r1q1a1b2uruq

r1q2arb2uq

r1q2a1bqur

Mr12 Mq21 Lar1 Lbq2

r2q1a1b2uruq

r2q2a1brur

r2q2a1bqur

r1q1a1bqur

r1q2arbq

Mr12

Mq21

Lbr2

Lbq2Mr12

Mq21

Lar1

Mr12

Ubq2

r1q1aqb2ur

Laq1 Lbq2

r1q1arb2uq

r2q2arb2uq

Lar1

Mq21

Mr12



Planning graph

r1q2a1b2uruq

r1r2q1q2a1arb2bquruq

r1r2q1q2a1araqb2bquruq

r1r2q1q2a1a2araqb1b2bquruq

Mr12

Mq21

Lar1

Lbq2

Mr12

Mr21

Mq12

Mq21

Lar1

Uar1

Uar2

Uaq1

Lbr2

Laq1

Lbq2

Ubq1

Ubq2

Ubr2

Mr12

Mr21

Mq12

Mq21

Lar1

Lbr2

Laq1

Lbq2

Uar1

Ubq2

P0 A1 P1 A2 P2 A3 P3

positive effectnegative effect



Planning graph

A planning graph is a directed layered graph : arcs are permitted onlyfrom one layer to the nextNodes in level 0 correspond to the set P0 of propositions denoting theinitial state s0 of the planning problemLevel 1 contains two layers :

1 an action level A1 that is the set of actions (ground instance ofoperators) whose preconditions are nodes in P0

2 a proposition level P1 that is defined as the union of P0 and the sets ofeffects of actions in A1



Planning graph

An action node in A1 is connected with :an incoming precondition arcs from its preconditions in P0outgoing arcs to its positive effects and to its negative effects in P1

Outgoing arcs are labeled positive or negative. Note that negativeeffects are not deleted from P1, thus P0 ⊆ P1

This process is iterated from one level to the next



Persistence actions

Inclusion of proposition levels is a way to represent inaction as well.Ai levels contain actions that could occur and "dummy" actions –No-op – representing the fact that some world properties(propositions) persist in different situationsTo each proposition p corresponds a no-op αp :

αp =

precond(αp) = {p},effect+(αp) = {p},effect−(αp) = {}.



Reachability analysis with planning graph

A major contribution of Graphplan is a relaxation of the reachabiityanalysis → incomplete condition of reachabilityA goal is reachable from s0 only if it appears in some node of theplanning graphWeak reachability condition is compensated for by a low complexity :the planning graph is of polynomial size and can be built inpolynomial time in the size of the input



Layered plan

A plan associated with a planning graph is no longer a sequence ofactions : a plan is a sequence of set of actions

Π = [π1, . . . , πk ]

i.e. a layered planπi ⊆ Ai . π1 is a subset of independent actions in A1 that can beapplied in any order to the initial state and can lead to a state that isa subset of P1 etc.πk actions lead to a state meeting the goal.



Independent actions

DefinitionTwo actions (a, b) are independent iff :

effects−(a) ∩ [precond(b) ∪ effects+(b)] = ∅effects−(b) ∩ [precond(a) ∪ effects+(b)] = ∅

A set of actions π is independent when every pair of π is independent.



r1q2a1b2uruq




Mr12

Mq21

Lar1

Lbq2

Mr12

Mr21

Mq12

Mq21

Lar1

Uar1

Uar2

Uaq1

Lbr2

Laq1

Lbq2

Ubq1

Ubq2

Ubr2

Mr12

Mr21

Mq12

Mq21

Lar1

Lbr2

Laq1

Lbq2

Uar1

Ubq2

P0 A1 P1 A2 P2 A3 P3

I cannot move and at the same moment load a

container



r1q2a1b2uruq




Mr12

Mq21

Lar1

Lbq2

Mr12

Mr21

Mq12

Mq21

Lar1

Uar1

Uar2

Uaq1

Lbr2

Laq1

Lbq2

Ubq1

Ubq2

Ubr2

Mr12

Mr21

Mq12

Mq21

Lar1

Lbr2

Laq1

Lbq2

Uar1

Ubq2

P0 A1 P1 A2 P2 A3 P3

Ok to swap places!



Applicable actions

DefinitionA set π of independent actions is applicable to a state s iffprecond(π) ⊆ s. The result of applying the set π to s is defined as :

γ(s, π) = (s − effects−(π)) ∪ effects+(π)

whereeffects−(π) =

⋃{effects−(a)|∀a ∈ π}

effects+(π) =⋃{effects+(a)|∀a ∈ π}

precond(π) =⋃{precond(a)|∀a ∈ π}



TheoremIf a set π of independent actions is applicable to s then, for anypermutation {a1, . . . , ak} of the elements of π, this permutation isapplicable to s and the state resulting from the application of π to s issuch that γ(s, π) = γ(. . . γ(γ(s, a1), a2) . . . , ak).



DefinitionA layered plan is a sequence of action sets. The layered planΠ = [π1, . . . , πn] is a solution to a problem (O, s0, g) iff :

1 each set πi ∈ Π is independent2 the set π1 is applicable to s0, π2 is applicable to γ(s0, π1) etc.3 g ⊆ γ(. . . γ(γ(s, π1), π2) . . . , πn).



Mutex

The incompatibility relations between actions and betweenpropositions in a planning graph, also called mutual exclusions ormutex relations are :

Interfering actions,Inconsistent effects,Actions having competing needs or inconsistent support.



Inconsistent effects

r1q2a1b2uruq




Mr12

Mq21

Lar1

Lbq2

Mr12

Mr21

Mq12

Mq21

Lar1

Uar1

Uar2

Uaq1

Lbr2

Laq1

Lbq2

Ubq1

Ubq2

Ubr2

Mr12

Mr21

Mq12

Mq21

Lar1

Lbr2

Laq1

Lbq2

Uar1

Ubq2

P0 A1 P1 A2 P2 A3 P3



Interference

r1q2a1b2uruq




Mr12

Mq21

Lar1

Lbq2

Mr12

Mr21

Mq12

Mq21

Lar1

Uar1

Uar2

Uaq1

Lbr2

Laq1

Lbq2

Ubq1

Ubq2

Ubr2

Mr12

Mr21

Mq12

Mq21

Lar1

Lbr2

Laq1

Lbq2

Uar1

Ubq2

P0 A1 P1 A2 P2 A3 P3



Inconsistent support & competing needs

r1q2a1b2uruq




Mr12

Mq21

Lar1

Lbq2

Mr12

Mr21

Mq12

Mq21

Lar1

Uar1

Uar2

Uaq1

Lbr2

Laq1

Lbq2

Ubq1

Ubq2

Ubr2

Mr12

Mr21

Mq12

Mq21

Lar1

Lbr2

Laq1

Lbq2

Uar1

Ubq2

P0 A1 P1 A2 P2 A3 P3



DefinitionTwo actions a and b in level Ai are mutex if either a and b aredependent or if a precondition of a is mutex with a precondition of b.Two propositions p and q in Pi are mutex if every action in Ai thathas p as a positive effect (including no-op actions) is mutex withevery action that produces q, and there is no action in Ai thatproduces both p and q.



Two dependent actions are necessarily mutex,However, two independent actions can be mutex !



We will denote the set of mutex pairs in Ai as µAi , and the set ofmutex pairs in Pi as µPi .

TheoremIf two propositions p and q are in Pi−1 and (p, q) /∈ µPi−1 then (p, q) /∈ µPi

If two actions a and b are in Ai−1 and (a, b) /∈ µAi−1 then (a, b) /∈ µAi

According to this result, propositions and actions monotonicallyincrease from one level to the next, while mutex pairs monotonicallydecrease.These monotonicity properties are essential to the complexity andtermination of the planning-graph techniques.



Proof

Every proposition p in a level Pi is supported by at least its no-opaction αp. Two no-op actions are necessarily independent. If p and qin Pi are such that (p, q) /∈ µPi−1 then (αp, αq) /∈ µAi . Hence, anonmutex pair of propositions remains nonmutex in the followinglevel.Similarly, if (a, b) /∈ µAi−1, then a and b are independent and theirpreconditions in Pi−1 are not mutex ; both properties remain valid atthe following level.



A solution plan

r1q2a1b2uruq




Mr12

Mq21

Lar1

Lbq2

Mr12

Mr21

Mq12

Mq21

Lar1

Uar1

Uar2

Uaq1

Lbr2

Laq1

Lbq2

Ubq1

Ubq2

Ubr2

Mr12

Mr21

Mq12

Mq21

Lar1

Lbr2

Laq1

Lbq2

Uar1

Ubq2

P0 A1 P1 A2 P2 A3 P3

no-op



TheoremA set g of propositions is reachable from s0 only if there is in thecorresponding planning graph a proposition layer Pi such that g ∈ Pi andno pair of propositions in g are in µPi .

The Graphplan performs a procedure close to iterative deepening,discovering a new part of the search space at each iteration ;It iteratively expands the planning graph by one level, then it searchesbackward from the last level of this graph for a solution ;The first expansion proceeds to a level Pi in which all of the goalpropositions are included and no pairs of them are mutex ;The iterative loop of graph expansion and search is pursued untileither a plan is found or a failure termination condition is met.



r1q2a1b2uruq

P0




r1q2a1b2uruq


Mr12

Mq21

Lar1

Lbq2

P0 A1 P1




r1q2a1b2uruq



Mr12

Mq21

Lar1

Lbq2

Mr12

Mr21

Mq12

Mq21

Lar1

Lbr2

Laq1

Lbq2

Uar1

Ubq2

P0 A1 P1 A2 P2




r1q2a1b2uruq




Mr12

Mq21

Lar1

Lbq2

Mr12

Mr21

Mq12

Mq21

Lar1

Uar1

Uar2

Uaq1

Lbr2

Laq1

Lbq2

Ubq1

Ubq2

Ubr2

Mr12

Mr21

Mq12

Mq21

Lar1

Lbr2

Laq1

Lbq2

Uar1

Ubq2

P0 A1 P1 A2 P2 A3 P3


Goal propositions found!



Expand([P0,A1, µA1,P1, µP1, . . . ,Ai−1, µAi−1,Pi−1, µPi−1])

Ai ← {a ∈ A | precond(a) ⊆ Pi−1 ∧ precond2(a) ∩ µPi−1 = ∅};1Pi ← {p | ∃a ∈ Ai , p ∈ effects+(a)};2µAi ← {(a, b) ∈ A2

i | effects−(a) ∩ [precond(b) ∪ effects+(b)] 6= ∅3∧ effects−(b) ∩ [precond(a) ∪ effects+(b)] 6= ∅∧ ∃(p, q) ∈ µPi−1, p ∈ precond(a) ∧ q ∈ precond(b)};foreach a ∈ Ai do4

link a with preconditions arcs to precond(a) ∈ Pi−1;5link positive arcs to effects+(a) and negative arcs to effects−(a) ∈ Pi ;6

return [P0,A1, µA1,P1, µP1, . . . ,Ai , µAi ,Pi , µPi ];7



The size of a planning graph down to level k and the time required toexpand it to that level are polynomial in the size of the planningproblem.Every graph G has a fixed-point level κ, which is the smallest k suchthat Pk−1 = Pk and µPk−1 = µPk .



r1q2a1b2uruq




Mr12

Mq21

Lar1

Lbq2

Mr12

Mr21

Mq12

Mq21

Lar1

Uar1

Uar2

Uaq1

Lbr2

Laq1

Lbq2

Ubq1

Ubq2

Ubr2

Mr12

Mr21

Mq12

Mq21

Lar1

Lbr2

Laq1

Lbq2

Uar1

Ubq2

P0 A1 P1 A2 P2 A3 P3

no-op



r1q2a1b2uruq




Mr12

Mq21

Lar1

Lbq2

Mr12

Mr21

Mq12

Mq21

Lar1

Uar1

Uar2

Uaq1

Lbr2

Laq1

Lbq2

Ubq1

Ubq2

Ubr2

Mr12

Mr21

Mq12

Mq21

Lar1

Lbr2

Laq1

Lbq2

Uar1

Ubq2

P0 A1 P1 A2 P2 A3 P3

no-op



r1q2a1b2uruq




Mr12

Mq21

Lar1

Lbq2

Mr12

Mr21

Mq12

Mq21

Lar1

Uar1

Uar2

Uaq1

Lbr2

Laq1

Lbq2

Ubq1

Ubq2

Ubr2

Mr12

Mr21

Mq12

Mq21

Lar1

Lbr2

Laq1

Lbq2

Uar1

Ubq2

P0 A1 P1 A2 P2 A3 P3

no-op

Solution found!



The search procedure looks for a set πi ∈ Ai of nonmutex actionsthat achieves all goal propositions.Preconditions of elements in πi become the new goal for level i − 1and so on.A failure to meet the goal of some level j leads to a backtrack overother subsets of Aj+1.If level 0 is successfully reached, then the corresponding sequence[π1, . . . , πi ] is a solution plan.



The extraction of a plan corresponds to a search in an AND/ORsubgraph of the planning graph :

From a proposition in goal g , OR-branches are positive arcs from allactions in the preceding action level that support this proposition ;From an action node, AND-branches are its preconditions arcs.



Graphplan(O, s0, g)

i ← 0 ; ∇ ← ∅ ; P0 ← s0 ; G ← [P0];1isSolvable ← g ⊆ Pi ∧ g2 ∩ µPi = ∅ ; isExpandable ← not (isSolvable ∨ isFixedPoint);2while IsExpandable do3

i ← i + 1;4G ← Expand(G);5isExpandable ← not (isSolvable ∨ isFixedPoint);6

if isSolvable = false then ⊥;7Π← Extract(G, g, i);8if isFixedPoint then9η ← |∇(κ)|10

else11η ← 012

while Π = ⊥ do13i ← i + 1;14G ← Expand(G);15Π← Extract(G, g, i);16if Π = ⊥ ∧ isFixedPoint then17

if η = |∇(κ)| then return ⊥18η ← |∇(κ)|;19

return Π;20Humbert Fiorino Automated Planning & Artificial Intelligence 37 / 46


The mutex relation between propositions provides only forbiddenpairs, not tuples.The search may show that a tuple corresponding to an intermediatesubgoal fails.Because of the backtracking and the iterative deepening, the searchmay have to analyze that same tuple more than once. Recording thetuples that failed may save time in future searches.This recording is performed by the Extract procedure into a nogoodhash-table denoted ∇.This hash table is indexed by the level of the failed goal because agoal g can fail at level i and succeed at j > i .



Extract takes as input a planning graph G , a current set of goalpropositions g and a level i ;It extracts a set of actions πi ⊆ Ai that achieves propositions of g byrecursively calling the GP-Search procedure ;If it succeeds in reaching level 0, then it returns an empty sequence,from which pending recursions successfully return a solution plan ;It records failed tuples into ∇ table, and it chekcs each current goalwith respect to recorded tuples ;a tuple g is added to the nogood table at level i only if the call toGP-Search fails to establish g at this level from mutex and othernogoods found or established at the previous level.



Extract(G , g , i)

if i = 0 then return [];1if g ∈ ∇(i) then return ⊥;2πi ← GP− Search(G , g , ∅, i);3if πi 6= ⊥ then return (πi);4∇(i)← ∇(i) ∪ {g};5return ⊥;6



GP-search(G , g , πi , i)

if g = ∅ then1Π← Extract(G ,

⋃{precond(a) | ∀a ∈ πi}, i − 1);2

if Π = ⊥ then return ⊥;3return Π.[πi ];4

else5select any p ∈ g ;6resolvers← {a ∈ Ai | p ∈ effects+(a) ∧ ∀b ∈ πi , (a, b) /∈ µAi};7if resolvers = ∅ then return ⊥;8Nondeterministically choose a ∈ resolvers;9return GP-Search(G , g − effects+(a), πi ∪ {a}, i);10



The GP-Search procedure selects each goal proposition p at a time insome heuristic order ;Among the resolvers of p, it nondetermnistically chooses one action athat tentatively extends the current subset πi through a recursive callat the same level ;This is performed on a subset of goals minus p and minus all positiveeffects of a in g ;As usual, a failure for this nondeterministic choice is a backtrackpoint over other alternatives for achieving p, if any, or a backtrackingfurther up if all resolvers of p have been tried ;When g is empty, then πi is complete ; the search recursively tries toextract a solution for level i − 1.



Graphplan performs an initial graph expansion until either it reaches alevel containing all goal propositions without mutex or it arrives at afixed-point level ;If this fixed-point is achieved first, the goal is not achievable ;Otherwise, a search for a solution is performed. If no solution is foundat this stage, the planning graph is iteratively expanded and explored ;This iterative deepening is pursued even after a fixed-point has beenreached, until success or until the termination condition is met :

the number of nogood in ∇(κ) at the fixed-point level κ stabilizes aftertwo successive failures.



Count operators

(operator UN(params )(preconds (go))(effects (one)(del two)(del three)))

(operator DEUX(params )(preconds (go))(effects (two)(del three)))

(operator TROIS(params )(preconds (go))(effects (three)))



Count Problem Definition

(preconds(go))

(effect(one)(two)(three))



go

onetwo

threego

UN

DEUX

TROIS

P0 A1 P1

{one, two}{one, three}{two, three}

µ



go

onetwo

threego

UN

DEUX

TROIS

P0 A1 P1 A2 P2

UN

DEUX

TROIS

onetwo

threego


µ



go

onetwo

threego

UN

DEUX

TROIS

P0 A1 P1 A2 P2


UN

DEUX

TROIS

onetwo

threego

{one, two, go}{one, three, go}{two, three, go}

{one, two, three}∇

µ



onetwo

threego

go

onetwo

threego

UN

DEUX

TROIS

P0 A1 P1 A2 P2


UN

DEUX

TROIS

onetwo

threego

UN

DEUX

TROIS

A3 P3

µ





onetwothreego

go

onetwo

threego

UN

DEUX

TROIS

P0 A1 P1 A2 P2


UN

DEUX

TROIS

onetwo

threego

UN

DEUX

TROIS

A3 P3

µ





onetwothreego

go

onetwo

threego

UN

DEUX

TROIS

P0 A1 P1 A2 P2


UN

DEUX

TROIS

onetwo

threego

UN

DEUX

TROIS

A3 P3

µ





onetwothreego

go

onetwo

threego

UN

DEUX

TROIS

P0 A1 P1 A2 P2


UN

DEUX

TROIS

onetwo

threego

UN

DEUX

TROIS

A3 P3

µ





onetwothreego

go

onetwo

threego

UN

DEUX

TROIS

P0 A1 P1 A2 P2


UN

DEUX

TROIS

onetwo

threego

UN

DEUX

TROIS

A3 P3

µ





onetwothreego

go

onetwo

threego

UN

DEUX

TROIS

P0 A1 P1 A2 P2


UN

DEUX

TROIS

onetwo

threego

UN

DEUX

TROIS

A3 P3

µ





onetwothreego

go

onetwothreego

UN

DEUX

TROIS

P0 A1 P1 A2 P2

UN

DEUX

TROIS

onetwothreego

UN

DEUX

TROIS

A3 P3


Documents

Automated Planning & Artificial Intelligence - Planning graphs