Introduction to Automated Task PlanningTDDC17/info/slides/2011/TDDC17_Fo8_9_plan_4sl.pdf · Miconic 10 Elevators 5 jonkv@ida Schindler Miconic 10 elevators People enter their destination

Introduction to

Automated Task

PlanningJonas Kvarnström

Automated Planning and Diagnosis Group

Artificial Intelligence and Integrated Computer Systems (AIICS) / IDA

3

jonk

v@ida

What is Planning?

Planning is thinking ahead

Not just reacting to what happens!

Thinking about possible actions and their effects,

formulating a complete plan that describes what to do and when,

in order to achieve a goal

…so when do we want computers to do that?

4

jonk

v@ida

Shakey

� Classical example: Shakey� Used the STRIPS planner

▪ Stanford Research InstituteProblem Solver

▪ One of the first planners (1969)

5

jonk

v@ida

Miconic 10 Elevators

� Schindler Miconic 10 elevators� People enter their destination

before they board an elevator

� Many elevators, many floors

� The goal: For everyone to be at their destination

� The plan tells us:Which elevators should go to whichfloors, in which order?

� The gain: Less time to wait for an elevator!

6

jonk

v@ida

Sheet Metal Bending

� Robots bending sheet metal� Goal: Bend a flat sheet to a specific shape

� Constraints: The piece must not collide with anything when moved!(Geometric reasoning required)

� Optimized operation saves a lot of time = money!

7

jonk

v@ida

NASA Earth Observing-1 Mission

� Earth Observing-1 Mission � Satellite in low earth orbit

▪ Can only communicate 8 x 10 minutes/day

▪ Operates for long periods without supervision

� CASPER software:Continuous Activity Scheduling, Planning, Execution and Replanning

� Dramatically increases science returns

▪ Interesting events are analyzed(volcanic eruptions, …)

▪ Targets to view are planneddepending on previous observations

▪ Plans downlink activities:Limited bandwidth

� http://ase.jpl.nasa.gov/

8

jonk

v@ida

Unmanned Aerial Vehicles

� Unmanned aerial vehicles� Multiple agents to coordinate

� Actions executed concurrently

9

jonk

v@ida

UAV 1: Photogrammetry

� Photograph buildings, to generate 3D models

10

jonk

v@ida

UAV 2: Traffic Monitoring

� Monitor traffic / find possible routes for emergency vehicles

11

jonk

v@ida

UAV 3: Finding Forest Fires

� Patrol large areas searching for forest fires, day after day after day…

12

jonk

v@ida

UAV 4: Emergency Services Logistics

� Assist in emergency situations

▪ Deliver packages of food, medicine, water

13

jonk

v@ida

Why Planning?

� Can’t we just solve these problems ourselves?

� Manual planning can be boring and inefficient

▪ Who wants to spend all day guiding elevators?

� Automated planning may create higher quality plans

▪ Software can systematically optimize, can investigate millions of alternatives

� Automated planning can be applied where the agent is

▪ Satellites cannot always communicate with ground operators

▪ Spacecraft or robots on other planets may be hours away by radio

Domain-Specific vs.

Domain-Independent Planning

15

jonk

v@ida

Introduction

� Let’s say we are interested in the photogrammetry domain� Multiple UAVs available

� Goal: To take pictures of buildings

� How do you create a photogrammetry plan?

16

jonk

v@ida

Domain-Specific Planning

� Domain-specific planner: Created for a particular domain� Input: A map containing buildings

▪ Everything else is implicit,since the planner already knowswhat actions are available, etc (hardcoded!)

� Implementation:

▪ Calculate suitable UAV positions

▪ Call a Travelling Salesman Problem solver

▪ Generate flight commands

PG Problem PG-specificPlanner

PG Plan

Sounds simple, and can be very efficient! But…

17

jonk

v@ida

Domain-Specific Planning

� Specialization means less flexibility! What if…� you want to deliver a couple of crates at the same time?

▪ Generalize input…

� you have to consider refueling?

▪ Different algorithm!

� you have two UAVs and a UGV (ground vehicle)?

▪ Differentalgorithm!

� you want to survey an area (send a video feed of the ground)?

� you have dynamic no-fly-zones (”don’t fly there at 15:00-16:00”)?

PG +DeliveryPlanner

PGPlannerw/ fuel

Multi-TSP

planner

18

jonk

v@ida


� We want a generic planning system!� Capable of taking a high-level description of any problem domain

and then solving problems within the domain!

� A single planner

▪ Improvements to the planner � all domains benefit

� High-level domain and problem definitions

▪ Easier to specify these than to write specialized algorithms

▪ Easier to change than a hard-coded optimized implementation

▪ Also useful for rapid prototyping

Satellite

DomainGeneralPlanner

Satellite

problem instance

PlanUAV

DomainGeneralPlanner

Photogrammetry

problem instance

Plan

19

jonk

v@ida

What is Common?

What is common to all of these problems?

20

jonk

v@ida

This is Common!

� The world contains various types of objects� Buildings, cards, aircraft, switches, people, …

� We are generally interested in properties of these objects� Location of a card, whether we have a picture of a building or not, …

� Goals can be described as desired properties� We should have a picture of each building

� Actions modify properties (simplified!)� takeoff(uav) � now the UAV is in the air

� photograph(uav, building) � now we have a picture of the building

� Can only be executed if we can see the building

21

jonk

v@ida

Example 1

� In the Emergency Services Logistics domain:� 1. What objects exist?

▪ Helicopters! heli = { surv1, surv2, …, cargo1, cargo2, … }

▪ Boxes!box = { box1, box2, … }

▪ Locations!loc = { basestation, person1, person2, …,depot1, depot2, … }

▪ Let’s generalize:Both helicopters and boxesare objects as well.

22

jonk

v@ida

Example 2

� 2. What properties can different types of objects have?(Defined using state variables)

▪ An object is or isn’t at a certain location: at(object, loc)A helicopter is or isn’t at a certain location: at(heli, loc)A helicopter is or isn’t carrying a certain box: carrying(heli, box)

23

jonk

v@ida

Example 3

� 3. What properties do the objects of the current problem have now?

▪ Every box is at a certain location:at(box1, depot1), at(box2, depot1), …

▪ Every helicopter is at a certain location:at(surv1, basestation), at(surv2, person1), …

▪ Some helicopters are carrying boxes:carrying(cargo1, box4)

� 4. What properties should they have?This is our goal!

▪ at(box1, person1), at(box2, person2), …

Situationwe want to

achieve

Situationright now

24

jonk

v@ida

Example 4

� 4. What can we do about it? Actions!� Example: fly(heli, loc1, loc2)

▪ Preconditions:at(heli, loc1)loc2 is free – no other helicopter therefuel(heli) > dist(loc1, loc2) * fuelUsage(heli)

▪ Effects:at(heli, loc1) is no longer trueat(heli, loc2) is true insteadfuel(heli) decreases by dist(loc1, loc2) * fuelUsage(heli)

▪ Time requirements:distance(loc1, loc2) / speed(heli)

� Take off / land

� Hover at the current location

� Position the camera at angle θ

� Take a visual picture / Take an infrared picture

� Lift a package using a winch / Deliver a package using winch

L1 L2

25

jonk

v@ida


Input 1: Planning domain

Input 2: Problem instance

Object Types: There are UAVs, boxes …

Properties: Every UAV has a maxSpeed, …

Actions: Definition of fly, pickup, …

Objects: Current UAVs are {UAV1,UAV2}

Initial State: Box locations, …

Goal: Box b1 at location l1, …

But what language should we use?

27

jonk

v@ida

Desirable Properties

� Desirable properties of a domain-independent planner:� Should be as general as possible

▪ Handle a wide variety of domains

� Should be as efficient as possible

▪ Generate plans quickly

� Should generate plans of the highest quality

▪ Fewer actions, lower cost, faster to execute, …

� Should support the user as much as possible

▪ Provide useful high-level structures such as actionsthat a user can easily specify

Many of these properties are in direct conflict!

28

jonk

v@ida

Classical Planning

� For efficiency, planners generally exploit domain structure� This implies constraining the expressivity of the planner

and its input language!

� Classical planning uses a common set of constraints…� Sometimes called "STRIPS planning"

� Demonstrated using a simple domain: The Blocks World

� Also introduces PDDL, the Planning Domain Definition Language

▪ General language

▪ Subsets are supported by most modern planners

29

jonk

v@ida

Blocks World

� What you can do:� Pick up a block

▪ unless something is on top of it

� Put a block that you’re holding…

▪ on the table (space for all blocks)

▪ on another block,unless something is on top of it

� Which actions will achieve your goals?

Your greatest desireWhat you knowYou

C B

A

C

B

A

30

jonk

v@ida

PDDL: Domain and Problem Definition

� PDDL: Lisp-like language, expressions in parentheses� Every domain is named,

associated with explicit list of expressivity requirements

▪ (define (domain blocksworld)(:requirements

:adl ;; Conditional effects, quantification, …:equality ;; Use of “=”…)

;; Remaining domain information goes here!)

� Problem instance has a name, belongs to a domain

▪ (define (problem MyThreeBlockProblem)(:domain blocksworld)…

)

Colon before many keywords,in order to avoid collisionswhen new keywords are added

31

jonk

v@ida

1: Objects

� In classical planning, there is a finite set of objects� Objects are generally declared in the problem instance

▪ Different objects in different instances!

▪ (define (problem MyThreeBlockProblem)(:domain blocksworld)(:objects A B C)…

)

� We can choose to model the table as a special block

▪ Always present in all instances � defined in the domain

▪ (define (domain blocksworld)(:requirements :adl …)(:constants Table)…

)

C B

A

32

jonk

v@ida

2: Finite States

� Classical planning: World modeled using finite states� A finite set of state variables (predicates or non-boolean fluents),

with values from a finite domain

▪ � A finite (but gigantic!) set of possible states of the world

▪ Each state gives every state variable a specific value

� Many ways to model any domain! One example for blocks world:

▪ We must know if a block is on something else, or not

▪ We must know whether an object is the table,which we modeled as a ”special” block

▪ (define (domain blocksworld)(:requirements :adl …)(:predicates (on ?X ?Y) (istable ?X))…

)

C B

A

Variables are prefixed with “?”

33

jonk

v@ida

2b: State Example

� Three different statesin the Blocks World� Can be described graphically

� Can be described in text(the following 3 slides!)

C

B

A

C B

A

C B

A

34

jonk

v@ida

3: Complete Initial Information

� We assume complete information about the initial state� All state variables are given values in the problem instance

▪ BW: on(A,B), on(B,Table), on(C,Table)… etc.

C B

A

35

jonk

v@ida

3b: Initial State in PDDL

� How do we completely specify the initial BW state?▪ (and

(not (on A A)) (not (on A B)) (on A C) (not (on A Table)(not (on B A)) (not (on B B)) (not (on B C)) (on B Table)(not (on C A)) (not (on C B)) (not (on C C)) (on C Table)(not (on Table A)) (not (on Table B))(not (on Table C)) (not (on Table Table))(not (istable A)) (not (istable B)) (not (istable C))(istable Table))

� Observation: Given complete info, most facts are false!

▪ A block can only be on one other block:Given any ?x, at most one instance of (on ?x ?y) is true(Given 1000 blocks, ?x is on 1, not on 999)

C B

A

36

jonk

v@ida

3c: Conventions for Initial States

� Convention: Only specify what is true in the initial state

▪ (define (problem MyThreeBlockProblem)(:domain blocksworld)(:objects A B C)(:init (on A C) (on C Table) (on B Table) (istable Table))

…)

� This is a shorthand notation, and is only used in :init.

▪ PDDL is for planners that cannot handle unknown initial information,so for convenience,an atom that is not mentioned is by definition false

C B

A

37

jonk

v@ida

4: Operators and Actions

� A set of single-step operators defines what you can do� Part of the domain description

� Many ways of modeling any domain! One example:

▪ “put” should take a block from where it is, and put it on another block

� Each operator has parameters: put(x,y)

▪ Instantiated into actions: put(A,B), put(A,C), …

� Each operator has a precondition

▪ “There must be no other block on top of x”, …

� Each operator has effects

▪ Afterwards, x will be on top of y

�� put(A,B) ��

C B

A

C B

A

38

jonk

v@ida

4b: Operators and Actions

� Each action is executed as a single step� If the precondition holds in the current state:

▪ The action can be executed

▪ A single new state results, where the effects hold.

� UAV domain:

▪ Flying from (100,100) to (300,300)is basically modeled as “teleporting”in a single step

▪ We don’t model the factthat the UAV moves throughintervening locations

▪ Sufficient for sequential plans�� put(A,B) ��

C B

A

C B

A

39

jonk

v@ida

4c: Operator Preconditions

� Precondition: Must hold for an action to be applicable

▪ Classical planning: We must know what an action requires!

▪ (define (domain blocksworld) …(:action put

:parameters (?moved ?dest):precondition (and

;; Don't put the table on top of something(not (istable ?moved)) ;; Don't put a block on top of itself(not (= ?moved ?dest));; There is nothing on top of the block we move(not (exists (?other) (on ?other ?moved))) ;; Either the destination is the table,;; or there is nothing on top of it(or (istable ?dest) (not (exists (?other) (on ?other ?dest))))

)

Connectives:(and …)(or …)(imply …)(not …)

Quantifiers:(exists (?v) …)(forall (?v) …)

Operators are called actions in PDDL…

40

jonk

v@ida

4d: Effects

� Effects: What changes if you perform an action?

▪ Classical planning: We must know all effects!

▪ Can’t state that an action may fail: (on x y) or (on x Table)

▪ Can’t model outcome probabilities: 95% (on x y), 5% (on x Table)

▪ Instead, complete information: Only (and …) is possible!

▪ Include atoms that become true: (on ?moved ?dest)

▪ Include negated atoms that become false: (not (on ?moved ?other))

:effect (and;; ?moved is now on ?dest.(on ?moved ?dest);; If ?moved was on some ?other, it no longer is.(forall (?other)

(when (on ?moved ?other) (not (on ?moved ?other))))

Uses quantified and conditional effects (ADL):(when (condition-in-invocation-state)

(effects-in-result-state))

41

jonk

v@ida

5: No Other Agents

� Classical planning assumes the state of the world can only be modifiedby actions generated by the planner

▪ No agents are moving blocks aroundunless we say so in the plan!

▪ � When considering different potential plans,the planner can predict the exact end result of every action in the planwithout worrying about someone "disturbing" its execution

42

jonk

v@ida

6: Goals

� Goals only constrain the final state of a plan� Specify the end result, not how you get there

� Specified in the problem instance

� Goal for our example instance� One possibility:

▪ (define (problem ABC)(:domain blocksworld)…(:goal (and (on A B) (on B C) (on C Table))))

C

B

A

43

jonk

v@ida

6b: Goals

� A complete goal specification is generally not required� Some predicates can "follow from others"

▪ (define (problem ABC)(:domain blocksworld)…(:goal (and (on A B) (on B C) (on C Table))))

� Here we don't specify that (on A C) must be false

▪ This just follows from the fact thata block can't be on two different blocks at the same time

C

B

A

44

jonk

v@ida

6c: Goals

� A goal specification can usually correspond to many states� An arbitrary conjunction is allowed

� We do not assume that predicates left out must be false!

▪ (define (problem ABC)(:domain blocksworld)…(:goal (and (on B C) (on C Table))))

▪ We don't say whether (on A B) is true,so either of the two following states would be considered goal states!

� Sometimes disjunctions may be allowed as well

C

B

A

C

B

A

Using Object TypesSame Expressivity, More Convenient

46

jonk

v@ida

Example Domain: Logistics

� The "traditional" logistics domain� Unlike the blocks world, many object types

▪ We have a number of packages at given locations

▪ Move packages within a city using trucks

▪ Move packages between cities using airplanes

▪ Airplanes can only move between certain airport locations

▪ Goal: Packages should be at their destinations

47

jonk

v@ida

Logistics Domain in PDDL

� First attempt at a logistics domain:� (define (domain logistics)

(:predicates(incity ?L ?C) (at ?X ?L) (in ?P ?V))

(:action load:parameters (?P ?V ?L):precondition (and (at ?P ?L) (at ?V ?L)):effect (and (in ?P ?V) (not (at ?P ?L)))

(:action drive-truck:parameters (?T ?L1 ?L2 ?C):precondition (and (incity ?L1 ?C) (incity ?L2 ?C) (at ?T ?L1)):effect (and (not (at ?T ?L1)) (at ?T ?L2)))

…)Problem: No type checking!• load plane7 onto city5 at truck2• drive city9 from truck1 to airplane2

48

jonk

v@ida

Type Predicates

� Solution 1: Type predicates� (define (domain logistics)

(:predicates(incity ?L ?C) (at ?X ?L) (in ?P ?V)(location ?L) (airport ?A) (vehicle ?V) (truck ?T)(airplane ?P) (package ?P))

(:action load:parameters (?P ?V ?L):precondition (and (package ?P) (vehicle ?V) (location ?L)

(at ?P ?L) (at ?V ?L)):effect (and (in ?P ?V) (not (at ?P ?L))))

…)

� (define (problem MyLogisticsProblem)(:domain logistics)(:objects plane1 plane2 truck1 truck2 truck3 loc1 loc2 …)(:init (incity loc1 linköping) …

(plane plane1) (plane plane2)(location loc1) (location loc2)…))

49

jonk

v@ida

Explicit Types

� Solution 2: "True" types (many modern planners)� (define (domain logistics)

(:requirements :typing)(:types truck airplane – vehicle

airport – location package vehicle location city

(:predicates (incity ?L – location ?C – city)(at ?X – package ?L – location)(in ?P – package ?V – vehicle))

(:action load:parameters (?P – package ?V – vehicle ?L – location):precondition (and (at ?P ?L) (at ?V ?L)):effect (and (in ?P ?V) (not (at ?P ?L)))

…)

� (define (problem MyLogisticsProblem)(:domain logistics)(:objects plane1 plane2 – airplane

truck1 truck2 truck3 – truckloc1 loc2 – location …)

• Unusual syntax:subtype subtype … - supertype

• All “top-level” types must be declared last!

51

jonk

v@ida

The 4-Operator Blocks World

� Planning will be demonstrated using the Blocks World� A more common variation, with 4 operators

▪ (pickup ?x) – takes ?x from the table

▪ (putdown ?x) – puts ?x on the table

▪ (unstack ?x ?y) – takes ?x from on top of ?y

▪ (stack ?x ?y) – puts ?x on top of ?y

▪ Plans are twice as long, but each operator is less complex

� Additional predicates used to avoid quantification

▪ (ontable ?x) – ?x is on the table

▪ (clear ?x): – ?x is an ordinary block whose top is free

▪ (holding ?x) – the robot is holding block ?x

▪ (handempty) – the robot is not holding a block

pickup(B)putdown(A)unstack(A,C) stack(B,C)

C B

A

52

jonk

v@ida

State Space

� Any classical planning problem corresponds to a state space� This state space is a directed graph

� Each node is a world state

� Each directed edge corresponds to an executable action

A

BC D

A

BC

D AB

C D

A

BC D

A

BC D

A

BC D

ABC D

A small part of a BW state space,

for a tiny problem instance.

Notice that the BW lacks dead ends.

This is not truefor all domains!

53

jonk

v@ida

Sequential Plans

� For simple sequential (not concurrent or timed) plans:� A plan is a path in the state space

from the initial stateto any of the goal states (recall that many states may satisfy the goal)!

A

BC D

A

BC

D AB

C D

A

BC D

A

BC D

A

BC D

ABC D

So howdo we find

such a path?

Forward State-Space Search

(Progression Search)

55

jonk

v@ida

Progression Search 1

� Progression (Forward-chaining):� “If I’m at this state now,

what actions are applicable,and where will they take me?”

� Begin at the initial state

� Search forward, hoping to find a goal state

� Example:� Initial state Goal state

C B

A

D C

B

A

D

56

jonk

v@ida


� Four potential ways of achieving the goal:� 1) We may already have achieved the goal (not true here)

� 2) Unstack(A,C), then continue

� 3) Pickup(B), then continue

� 4) Pickup(D), then continue

A

B

C D

A

BC D AB

C D

A

BC

D

A

BC D

57

jonk

v@ida


� Hypothetically: Suppose we unstack(A,C). What then?▪ 1) Already achieved the goal? No.

▪ 2) May stack(A,C) – cycle, skip this.

▪ 3) May stack(A,B), continue

▪ 4) May stack(A,D), continue

▪ 5) May putdown(A), continue

A

B

C D

A

BC D AB

C D

A

BC

D

A

BC D

A

BC D

A

BC D

A

BC D

ABC D

58

jonk

v@ida


� Result: A search tree structure� One start state, possibly many goal states

▪ Each node is associated with a search state (in this case = world state)

▪ Branching rule: Given a node, there is one outgoing branch for each applicable action

GG

59

jonk

v@ida

Depth First

� Remaining choice: search algorithm� Decides which actions to “simulate”, in which order

� One possibility: Depth first

cycle cycle

GG

60

jonk

v@ida

Breadth First

� Another possibility: Breadth first

▪ Same search tree, different search order

▪ Plans will be optimal in the number of actions

▪ Sometimes faster, sometimes slower, usually too much memory

▪ Can also use iterative deepening and other blind search methods

GG

The Difficulty of Planning

62

jonk

v@ida

Forward-chaining Search

� Seems intuitive – but what are the results?� Movie 4 / no-rules

� This search procedure is “blind”: It just tries all variations!

� Is this really a problem? Computers are fast!� What if we have 100 possible actions in each step, not 4 or 5?

� What if a plan requires 100 steps?

▪ 100 plans having 1 step

▪ 10 000 plans having 2 steps

▪ 1 000 000 plans having 3 steps

▪ 100 000 000 plans having 4 steps

▪ 10 000 000 000 plans having 5 steps

▪ …

� Emergency Service Logistics / 20 helicopters, 100 locations 200 boxes:

▪ 7 * 10415 potential plans…

70000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 plans – but computers are fast…

Assume 1020 (100 billion billion) plans per second: 20000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 years – but we have multiple cores…

Suppose every particle in the universe is a computer:2000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 years

63

jonk

v@ida

Hopeless?

� Is there still hope for planning?� Of course there is!

� Our trivial planning method uses blind search – tries everything!

� We wouldn’t choose such silly actions – so why should the computer?

� Planning is part of Artificial Intelligence!� Developing methods to judge what actions are promising given goals

� Rapid progress in the last decade

▪ We are still far from true human-level understanding

▪ But planners are thousands of times faster than 10-15 years ago,even on the same hardware,despite being much more expressive

▪ A very active research area

Heuristic Progression Search

and Domain-Dependent Guidance

65

jonk

v@ida

Heuristic Search (1)

� One approach: heuristic functions� For each node, (under)estimates the cost to reach the goal

� Usually not perfect (can’t always show the best way to go)

▪ Example: Can’t always choose the optimal block to unstack

▪ Would be equivalent to solving the original planning problem!

� …together with heuristic search methods� A*, hill climbing, many specialized methods

66

jonk

v@ida


� A very simple domain-independent heuristic:� Count the number of predicates with the “wrong” value

▪ Note: Only local information used (current state)

A

BC AB

C

A

BC

7

ontable(C)on(A,C)

¬on(C,A)¬on(A,B)clear(A)clear(B)

¬clear(C)

C

A

B

- on(A,C)- clear(A)

- ¬clear(C)+ holding(A)+ handempty

- clear(B)+ ¬ontable(B)+ holding(B)+ handempty

9

A

BC

4

ABC

6

- ¬on(A,B)- holding(A)- handempty

+ clear(A)

- holding(A)- handempty

+ clear(A)+ ontable(A)

6

67

jonk

v@ida


� A perfect solution? No!▪ Often useful compared to blind search

▪ Modern heuristics are more sophisticated – can still be "misleading"

A

BC D AB

C D

A

BC

D

A

BC D

A

B

C D

5

8

5

9

ontable(B)¬on(A,B)on(A,C)¬on(B,C)clear(B)

B

A

C D

6

BA

C D

6

- on(A,C)+ ¬clear(A)+ clear(C)

+ ¬handempty+ holding(A)

68

jonk

v@ida


� Hand-tailored domain-dependent heuristics:▪ 2 actions for any block that is not "on" its destination,

or that is above some other block that is not on its destination

▪ 1 action for any block we are holding, to put it down

▪ If we hold a block whose destination is not yet ready,we must put it on the table – and later, 2 actions to move it again

A

BC D AB

C D

A

BC

D

A

BC D

A

B

C D

Must move B; holding A2*1 + 1+0 = 3

Must move A; holding B; dest not ready2*1+ 1+2 = 5

Must move A, B; holding D2*2 + 1+2 = 7

Must move A, B2*2 + 0+0 = 4

69

jonk

v@ida

Domain-Dependent Guidance

� Other forms of domain-specific guidance:� Rules that help the planner recognize "stupid" actions

▪ "Don't unload packages before you reach the destination"

▪ "Don't drive to a destination where there's nothing to pick up"

� Sometimes a small set of common sense rules will suffice

▪ Examples later – TALplanner

A

BC D AB

C D

A

BC

D

A

BC D

A

B

C D

Moves a block that had to be moved,in order to put something else on C.GOOD.

B was moved, but it did not block anything and its destination is not yet ready. BAD.

As above. BAD.

Backward State-Space Search

(Regression Search)

71

jonk

v@ida

Regression Search (1)

� Search method #2: Regression� “If this is the goal,

what action could I use to achieve part of the goal,and what would I have to do before that?”

� Begin at the goal

� Search backwards, hoping to find the initial state

� Example:� Initial state Goal

C B

A

D C

B

A

D

72

jonk

v@ida


� Three potential ways of achieving the goal:� 1) It might already have been achieved – not in this case

� 2) The last action was stack(A,B)

▪ Before this, we’d have the same situation,except A would be held.

� 3) The last action was putdown(D)

▪ Before, D would have been held.

A

B

C D

A

BC D

A

B

C D

A

B

C

D

73

jonk

v@ida


� How may we achieve the state where we’re holding A?� pickup(A)

� unstack(A,D)

� unstack(A,B)▪ Cycle!

A

B

C D

A

BC D

A

B

C D

A

B

C

D

A

B

C D

AB

C D

A

B

C D

74

jonk

v@ida

No Current State during Search!

� Forward search has a "current state"

� Backwards search has a "current goal"

goalputdown(D)unstack(D,F) g1g2

initial s2stack(A,C)unstack(A,B) s1

75

jonk

v@ida

Backward and Forward Search

FORWARD SEARCH

� Problematic when:

� There are many applicable actionsin the "average state"� high forward branching factor� need guidance

� You know if an action is applicable,but not if it will contributeto the goal

BACKWARD SEARCH

� Problematic when:

� There are many relevant actionsfor the "average goal"� high backward branching factor� need guidance

� You know if an action contributes to the goal,but not if you can achieve its preconditions

� Blind backward searchis generally better thanblind forward search:Relevance tends to providebetter guidance than applicability

� But heuristics are still needed

77

jonk

v@ida

Background 1

� State-space planning commits immediately to action order� Selects an action that seems “useful”

▪ According to heuristics

▪ According to relevance, in regression planning

� Puts it in its final place in the plan

� Continues from the resulting state (progression)…

� … Or from the resulting goal (regression)



78

jonk

v@ida

Background 2: Example

� Heuristic forward-chaining might begin as follows:

� Luckily, the heuristic always selected useful actions

▪ Picking up b1 is necessary, driving is necessary, putting b1 down is necessary

� But we still have a problem!

▪ At some time, the robot also has to pick up block b2

▪ The best plan would be to do that right after pickup(b1,A):The robot has two arms

▪ Can't insert a new action there in state space search!

put(b1, B)effects:at(b1,B),

¬holding(b1)

We want:

at(b1,B)at(b2,B)

We have:robot-at(A)

at(b1,A)at(b2,A)

drive(A,B)effects:

¬robot-at(A),robot-at(B)

pickup(b1,A)effects:

holding(b1),¬at(b1,A)

robot-at(A)holding(b1)

at(b2,A)

robot-at(B)holding(b1)

at(b2,A)

robot-at(B)at(b1,B)at(b2,A)

79

jonk

v@ida

Background 3: Alternative A

� The planner can backtrack, wasting some of its effort,or continue from where it is:

put(b1, B)effects:

at(b1,B),¬holding(b1)

We want:

at(b1,B)at(b2,B)

We have:robot-at(A)

at(b1,A)at(b2,A)

put(b2, B)effects:


drive(A,B)effects:






drive(B,A)effects:

¬robot-at(B), robot-at(A)

drive(A,B)effects:

¬robot-at(A), robot-at(B)


at(b2,A)


at(b2,A)


robot-at(A)at(b1,B)at(b2,A)

robot-at(A)at(b1,B)

holding(b2)

robot-at(B)at(b1,B)

holding(b2)

robot-at(B)at(b1,B)at(b2,B)

Results in a suboptimalplan! We could have done it in 5 actions.

80

jonk

v@ida

Background 4: What We Prefer

We want:

at(b1,B)at(b2,B)

We have:robot-at(A)

at(b1,A)at(b2,A)



Since the first three actions we found were actually useful for the goal,we would prefer if we could just slide the last ones over…


at(b2,A)

put(b1, B)effects:


drive(A,B)effects:



at(b2,A)




robot-at(A)at(b1,B)

holding(b2)

…and then insert another pickup action inbetween!

This is no longer state-space search!Requires different techniques…

Generating Partial-Order Plans

82

jonk

v@ida

No Current State during Search!

� Forward search has a "current state"

� Backwards search has a "current goal"

� With partial-order plans: No “current” state or goal!� What is true after stack(A,B) below?

▪ Depends on the order in which other actions are executed

▪ Changes if we insert new actions before stack(A,B)!

stack(A,B)

putdown(D)unstack(D,F)



goalactioninitaction

83

jonk

v@ida

Search Nodes are Partial Plans

� No current state � search nodes can’t correspond to states!� A node in the search space has to be a partial plan

� Each successor is a modified, refined partial plan

� Use search strategies, backtracking, heuristics, ... to search this space!

initial search node

initaction goalaction


putdown(D)


unstack(B,C)


putdown(D)

putdown(K)

This is one form of ”plan-space” planning!

84

jonk

v@ida

POCL vs Forward-Chaining

� What is the initial search node?� Forward state space search: The initial state

� POCL: The initial plan, consisting of:

▪ A special initial action whose effects specify the initial state

▪ A special goal action whose preconditions specify the goal

▪ A precedence constraint: the initial action is before the goal action

at(b1, B)

at(b2, B)

goalactioninitaction

at(b1, A)

at(b2, A)

robot-at(A)

85

jonk

v@ida

POCL vs Forward-Chaining

� What are the successors of a search node?� Progression: One successor per action executable in the current state

▪ Preconditions satisfied, effects consistent, …

� POCL: One successor for every way that a flaw in the plan can be fixed

▪ Every way of supporting an open goal

▪ Every way of removing a threat

initial search node



putdown(D)


unstack(B,C)

86

jonk

v@ida

Flaw Type 1: Open Goals

� Open goal:� An action a has a precondition p that we haven’t

decided how to establish(it has no incoming causal link)

� For example, the preconditions of the goal action!

� To resolve the flaw:� Find an action b that causes p to become true

▪ Can be a new action

▪ Can be an action already in the plan,if we can make sure it precedes a

� Create a causal link

Partial order! This was not possible in backward search…

Even if there is already an action that causes p,you can still add a new action that also causes p!

at(b1, B)

at(b2, B) goalaction

87

jonk

v@ida

Resolving Open Goals 1

� In this initial Blocks World plan we have six open goals� We can choose to find support for clear(A):

▪ From the initaction

▪ From a new instance of unstack(B,A), unstack(C,A), or unstack(D,A)

▪ From a new instance of stack(A,B), stack(A,C), stack(A,D), or putdown(A)

� We can choose to find support for on(A,B):

▪ Only from a new instance of stack(A,B)

� …

on(A,B)

on(B,C)

on(C,D)

ontable(D)

handempty

clear(A)clear(A)

ontable(A)

clear(B)

ontable(B)

clear(C)

on(C,D)

ontable(D)

handempty

init

act

ion

go

ala

ctio

n

We haven't decided howto achieve any of these

six goals!

88

jonk

v@ida


� Suppose we add stack(A,B) to support (achieve) on(A,B)� Must add a causal link for on(A,B)

▪ Dashed line

� Must also add precedence constraints

� The plan looks totally ordered

▪ Because it actually only has one “real” action…

holding(A)

clear(B)

¬holding(A)

clear(A)

ontable(A)

clear(B)

ontable(B)

clear(C)

on(C,D)

ontable(D)

handempty

init

act

ion

on(A,B)

on(B,C)

on(C,D)

ontable(D)

handempty

clear(A)

go

ala

ctio

n

¬clear(B)

on(A,B)

handempty

clear(A)

sta

ck(A

,B)

Causal link says:This instance of stack(A,B)

is responsible forachieving on(A,B)for the goalaction

89

jonk

v@ida


� Now we have 7 open goals� We can choose to find support for clear(A):

▪ From the initaction

▪ From the instance of stack(A,B) that we just added

▪ From a new instance of stack(A,B), stack(A,C), stack(A,D), or putdown(A)

▪ From a new instance of unstack(B,A), unstack(C,A), or unstack(D,A)

� … holding(A)

clear(B)

¬holding(A)

clear(A)

ontable(A)

clear(B)

ontable(B)

clear(C)

on(C,D)

ontable(D)

handempty

init

act

ion

on(A,B)

on(B,C)

on(C,D)

ontable(D)

handempty

clear(A)

go

ala

ctio

n

¬clear(B)

on(A,B)

handempty

clear(A)

sta

ck(A

,B)

90

jonk

v@ida

Flaw Type 2: Threats

� Second flaw type: A threat▪ initaction supports clear(B) for stack(A,B) – there’s a causal link

▪ pickup(B) deletes clear(B), and may occur between initaction and stack(A,B)

▪ So we can’t be certain that clear(B) still holds when stack(A,B) starts!

holding(A)

clear(B)

¬holding(A)

clear(A)

ontable(A)

clear(B)

ontable(B)

clear(C)

on(C,D)

ontable(D)

handempty

init

act

ion

on(A,B)

on(B,C)

on(C,D)

ontable(D)

handempty

clear(A)

go

ala

ctio

n

¬clear(B)

on(A,B)

handempty

clear(A)

sta

ck(A

,B)

holding(B)

clear(C)

¬holding(B)

¬clear(C)

on(B,C)

handempty

clear(B)

sta

ck(B

,C)

handempty

clear(B)

¬handempty

¬clear(B)

¬ontable(B)

holding(B)pic

ku

p(B

)

91

jonk

v@ida

Resolving Threats 1

� How to make sure that clear(B) holds when stack(A,B) starts?� Alternative 1: The action that disturbs the precondition

is placed after the action that has the precondition

▪ Only possible if the resulting partial order is consistent (acyclic)!

holding(A)holding(A)holding(A)holding(A)

clear(B)clear(B)clear(B)clear(B)

¬holding(A)¬holding(A)¬holding(A)¬holding(A)

clear(A)clear(A)clear(A)clear(A)

ontableontableontableontable(A)(A)(A)(A)


ontableontableontableontable(B)(B)(B)(B)

clear(C)clear(C)clear(C)clear(C)

on(C,D)on(C,D)on(C,D)on(C,D)

ontableontableontableontable(D)(D)(D)(D)

handemptyhandemptyhandemptyhandempty

init

act

ion

on(A,B)on(A,B)on(A,B)on(A,B)

on(B,C)on(B,C)on(B,C)on(B,C)





go

ala

ctio

n

¬clear(B)¬clear(B)¬clear(B)¬clear(B)




sta

ck(A

,B)

holding(B)holding(B)holding(B)holding(B)


¬¬¬¬holding(B)holding(B)holding(B)holding(B)

¬¬¬¬clear(C)clear(C)clear(C)clear(C)




sta

ck(B

,C)



¬¬¬¬handemptyhandemptyhandemptyhandempty


¬¬¬¬ontableontableontableontable(B(B(B(B))))

holding(B)holding(B)holding(B)holding(B)pic

ku

p(B

)

92

jonk

v@ida

Resolving Threats 2

� Alternative 2:

▪ The action that disturbs the preconditionis placed before the action that supports the precondition

▪ Only possible if the resulting partial order is consistent – not in this case!












init

act

ion







go

ala

ctio

n





sta

ck(A

,B)








sta

ck(B

,C)







ku

p(B

)

93

jonk

v@ida

Resolving Threats 3

� Only causal links can be threatened!� Below, pickup(B) does not threaten the precond clear(B) of stack(A,B)

▪ We haven’t decided yet how to achieve clear(B): No incoming causal link

▪ So we can’t claim that its achievement is threatened!












init

act

ion







go

ala

ctio

n





sta

ck(A

,B)








sta

ck(B

,C)







ku

p(B

)

94

jonk

v@ida

The PSP Procedure

� Plan-Space Planning:� PSP(π)

flaws � OpenGoals(π) ∪ Threats(π)if flaws = ∅ then return π

select any flaw φ ∈ flawsresolvers � FindResolvers(φ, π)if resolvers = ∅ then return failure // Backtrack

nondeterministically choose a resolver ρ ∈ resolversπ’ � Refine(ρ, π) // Actually apply the resolverreturn PSP(π’)

end

� Call PSP(the initial plan)

� PSP is both sound and complete

� It returns a partially ordered solution plan

▪ Any total ordering of this plan will achieve the goals

Not a backtracking point! Resolving one flaw cannot prevent us from resolving other flaws.

This is a backtracking point. For example, a resolver might add an action that solves thislocal f law, but that cannot be part of a solution.

The plan is complete exactly when there are no remaining flaws (no open goals, no threats)

95

jonk

v@ida

Conclusions

� Partial-order planning delays commitment to action ordering� Lower branching factor

� More efficient in some situations

� Many POP planners still assume sequential execution� The intention was to find plans quickly,

not to find partially constrained plans

� POCL planning was long considered more efficient than FC� With new heuristics, forward-chaining planners took the lead

� At this point, many of the fastest planners are forward-chaining,but partial-order planners have also improved

96

jonk

v@ida

Other Search Spaces for Planning

� Many other search spaces:

� Parallel progression and regression

▪ Start from both ends; hope to meet in the middle

� Encoding: Transform into another problem,

e.g. SAT (boolean satisfiability)

� …

� Consult the literature for more information!

98

jonk

v@ida

Sequential Plans: Blocks World

� Sequential plans are OK for blocks world� There is only one robot…

pickup(B)

putdown(A)

unstack(A,C)

stack(B,C)

99

jonk

v@ida

Sequential Plans: Logistics

� Consider sequentialplanning for Logistics:� Two plans with the same

number of actions

� Do you see the difference?

load(P1,T1)

load(P2,T1)

load(P3,T1)

drive(T1,A,B)

unload(P1,T1)

drive(T1,B,C)

unload(P2,T1)

drive(T1,C,D)

unload(P3,T1)

load(P1,T1)

load(P2,T2)

load(P3,T3)

drive(T1,A,B)

unload(P1,T1)

drive(T2,A,C)

unload(P2,T2)

drive(T3,A,D)

unload(P3,T3)

100

jonk

v@ida

The Linear Plan: Logistics

� The first plan uses three trucks� Could easily be executed concurrently

� This is what we want!

� How should sequential planners seethat we prefer the first plan?

load(P1,T1)

load(P2,T1)

load(P3,T1)

drive(T1,A,B)

unload(P1,T1)

drive(T1,B,C)

unload(P2,T1)

drive(T1,C,D)

unload(P3,T1)

load(P1,T1) load(P2,T2) load(P3,T3)

drive(T1,A,B)

unload(P1,T1)

drive(T2,A,C)

unload(P2,T2)

drive(T1,A,D)

unload(P3,T3)

101

jonk

v@ida

Plan Structures

� The plan structure should depend on:� The abilities of the execution system

▪ If it can perform many tasks concurrently, we want concurrent plans

▪ Otherwise, we might be satisfied with simple sequences

� The desired solution properties

▪ Is timing important, or can it be ignored?

� The abilities of the algorithms

▪ Most algorithms are tightly bound toplan structures

▪ May settle for a less expressive plan structure,if this allows you to use an algorithmwhich is better in other respects

102

jonk

v@ida

Temporal Concurrent Plans

� Temporal concurrent plans: Explicit timing� Predict the amount of time required for each action

[0,10] load(P1,T1) [10,570] drive(T1,A,B) [570,580] unload(P1,T1)

[0,10] load(P2,T2) [10,420] drive(T2,C,D) [420,430] unload(P2,T2)

[0,10] load(P3,T3) [10,845] drive(T3,E,F) [845,855] unload(P3,T3)

103

jonk

v@ida

Action Durations in PDDL

� PDDL supports action durations� …And some planners implement this, generating temporal plans.

▪ (:durative-action turn:parameters (?current-target ?new-target - target):duration (= ?duration (/ (angle ?current-target ?new-target)

(turn-rate))):condition (and (at start (pointing ?current-target))

(at start (not (controller-in-use)))):effect (and (at start (not (pointing ?current-target)))

(at start (controller-in-use))(at start (vibration))(at end (not (controller-in-use)))(at end (not (vibration)))(at end (pointing ?new-target))))

105

jonk

v@ida

TALplanner

� TALplanner: A planner developed at IDA� Forward-chaining and uses depth-first search

▪ Allows us to use highly expressive models

▪ Arbitrary preconditions

▪ Conditional effects

▪ Timing, concurrency and resources

� Depth first requires guidance and goal-directedness

▪ No built-in heuristics

▪ Uses domain-dependent rules to guide towards the goal

▪ Can also specify domain-dependent heuristics

� Did well in international competitions

106

jonk

v@ida

TALplanner Example Domain

� Example Domain: ZenoTravel� Planes move people between cities (board, debark, fly)

� Planes have limited fuel level; must refuel

� Example instance:

▪ 70 people

▪ 35 planes

▪ 30 cities

107

jonk

v@ida

ZenoTravel Problem Instance

� A smaller problem instance

108

jonk

v@ida

What Just Happened?

� No additional domain knowledge specified yet!

� Pure depth first…initial node

one of thegoal nodes

109

jonk

v@ida

Control Rules

� TALplanner uses control rules� The plan generates a sequence of states

▪ Plan: step 1: board(p1, plane1, city4),step 2: board(p2, plane1, city4)

▪ States: time 0: initial statetime 1: after p1 boardingtime 2: after p2 boarding

� Control rules are formulasthat must hold in this state sequence

� Rules can also refer to the goal

▪ goal(f) true if f must hold in all goal states

at(p1, city4)at(p2, city4)

board(p1, plane1, city4)

board(p2, plane1, city4)

in(p1, plane1)at(p2, city4)

in(p1, plane1)in(p2, plane1)

110

jonk

v@ida

Multiple Uses of Control Rules

� Control rules have multiple uses� Prune parts of the tree that do not contain solutions▪ No need to investigate dead ends

▪ Some domains, like ZenoTravel, have no dead ends

� Prune parts of the tree that contain “bad” solutions▪ Plans that achieve the goal, but are unnecessarily expensive

▪ Applicable to this domain!

� Prune parts of the tree that contain redundant solutions▪ In optimizing planning, we don’t stop when the first plan is found

▪ No point in visiting many plans with equal cost

111

jonk

v@ida

Control Rules

� First problem in the example:� Passengers debark whenever possible.

� Rule: "At any timepoint, if a passenger debarks, he is at his goal.”

� #control :name "only-debark-when-in-goal-city"forall t, person, aircraft [

[t] in(person, aircraft) →[t+1] in(person, aircraft) ∨exists city [

[t] at(aircraft, city) ∧goal(at(person, city)) ] ]

112

jonk

v@ida

Control Rules

� Second problem in the example:� Passengers board planes, even at their destinations

� Rule: "At any timepoint, if a passenger boards a plane, he was not at his destination.”

� #control :name "only-board-when-necessary"forall t, person, aircraft [

([t] !in(person, aircraft) ∧[t+1] in(person, aircraft)) →

exists city1, city2 [[t] at(person, city1) ∧goal(at(person, city2)) ∧city1 != city2 ] ]

113

jonk

v@ida

Zeno Travel, second attempt 114

jonk

v@ida

What'sWrong This Time?

� Only constrained passengers

� Forgot to constrain airplanes

� Which cities are reasonable destinations?

� 1. A passenger’s destination

� 2. A place where a person wants to leave

� 3. The airplane’s destination

115

jonk

v@ida

Control Rules

� #control :name "planes-always-fly-to-goal“forall t, aircraft, city [

[t] at(aircraft, city) →([t+1] at(aircraft, city)) |exists city2 [

city2 != city &([t+1] at(aircraft, city2)) &[t] reasonable-destination(aircraft, city2) ]]

� #define [t] reasonable-destination(aircraft, city):[t] has-passenger-for(aircraft, city) |exists person [

[t] at(person, city) &[t] in-wrong-city(person) ] |

goal(at(aircraft, city)) &[t] empty(aircraft) &[t] all-persons-at-their-destinations-or-in-planes ]

116

jonk

v@ida

Zeno Travel, third attempt

117

jonk

v@ida

Positive Side Effects

� Positive effects of using domain knowledge:� Better performance than domain-independent heuristics▪ When you’ve found the right control rules!

� Can use more detailed domain models▪ With control rules, this has a smaller impact on performance

� Can generate better plans▪ Can prune non-solutions, but also bad solutions

� Negative effects of using domain knowledge:� Requires more knowledge

� Requires more domain engineering

� Rules are sometimes easy to find, sometimes difficult

� As always: A tradeoff!

Documents

Introduction to Automated Task PlanningTDDC17/info/slides/2011/TDDC17_Fo8_9_plan_4sl.pdf · Miconic 10 Elevators 5 jonkv@ida Schindler Miconic 10 elevators People enter their destination