Presented by: Robert T. Effinger

May 12, 2004

Massachusetts Institute of Technology

Presented by: Robert T. Effinger

January 25, 2005

Qualifying Exam Research Presentation

May 12, 2004


“Enabling Fast Flexible Planning through Incremental Temporal Reasoning with

Conflict Extraction”

I’shiang Shu, Robert Effinger, Prof. Brian WilliamsModel-Based Embedded and Robotic Systems Group

2005 International Conference on Automated Planning and Scheduling

3

Presentation Summary

• Importance of Automated Planning Research

• Three Main Types of Automated Planners– A Simple Example– Generative Planner– Path Planner– Task Network Planner

• Research Objectives of the Kirk Planner– Fast and Flexible Planning

• Key Ideas and Concepts of the Kirk Planner– Simple Temporal Network (STN)– Incremental Update Algorithm (ITC)

• Experimental Data

• Key Assumptions and Limitations

• Conclusions

4

Importance of Automated Planning

• Airline Timetables• Assembly Lines• Radio Frequency Planning• Military Convoys• ISS Life Support Rations• MER Rover Planning • NASA’s Deep Space Probes

5

Three Main Types of Automated Planners

– Generative Planners

– Path Planners

– Task Network Planners

6

A Simple Planning Example

Football Game:

20

10

G

QB

C R

Goal: Touchdown

Start Condition: 20 yardline

Library of Actions: Center (C):

- Hike- Block

Reciever (R):- Run to Endzone- Run Slant- Catch Football

QuarterBack (QB)- Pass to Endzone- Slant Pass- QB Sneak

7

Generative Planner

Football Game:

20

10

G

QB

R

Goal: Touchdown



- Hike- Block

Reciever (QB):- Run to Endzone- Run Slant- Catch Football


20 yardline

Generate Forward until Goal Condition Found: (Graphplan, FF)

(C) Hike

Start Condition: (R) Run to Endzone

(QB) Pass to Endzone(R) Catch Football

Goal:

Generate Backward until Start Condition Found: (PoP)

Touchdown !

C

Generative Planning Advantage: Very Expressive

Generative Planning Disadvantage: Often Intractable (exponential growth) Can Generate Infeasible Plans

Activity Times are not Flexible

Path Planner

8

Task Network Planner

Football Game:

20

10

G

QB

C R


Goal: Touchdown


- Hike- Block



9

Task Network Planning

Goal: Touchdown



- Hike- Block



Activity Plans:

20

10

G

QB

C R

Hail Mary Slant Pass

20

10

G

QB

C R

20

10

G

QB

C R

QB Sneak

20 yardline

Hail Mary Activity Plan:

(C) Hike(R) Run to Endzone

(QB) Pass to Endzone(R) Catch Football

Touchdown !

10

Task Network Planning

Goal: Touchdown



- Hike- Block



Activity Plans:

20

10

G

QB

C R

Hail Mary Slant Pass

20

10

G

QB

C R

20

10

G

QB

C R

QB Sneak

Task Network Advantages: Tractable in real-time Activity times can be flexible

Task Network Disadvantage: Less expressive

11

Research Objectives of the Kirk Planner

“To Enable Fast and Temporally Flexible Planning”

Translates into two technical research objectives:

2.) When replanning, try to reuse previous planning computations

1.) Allow flexible instead of fixed execution times on activities

Kirk’s Main Research Objective:

12

Two Key Ideas

“To Enable Fast and Temporally Flexible Planning”

2.) When replanning, try to reuse previous planning computations

1.) Allow flexible instead of fixed execution times on activities

Kirk’s Main Research Objective:

- Kirk supports a lower and upper time-bound on each activity

- an incremental update algorithm (ITC) that utilizes previous computations

Translates into two technical research objectives:

13

1st Key Idea: Flexible Execution Times

A Simple Temporal Network (STN):

Kirk supports flexible lower and upper time-bounds on activities

Touchdown !

20 yardline (C) Hike(R) Run to Endzone

(R) Catch Football(QB) Pass to Endzone

Another Representation of the Hail Mary Activity Plan:

20

10

G

QB

C R

Hail Mary:

(C) Hike [0,5]

( Touchdown ! )

(QB) Pass to Endzone [5,8]

(R) Run to Endzone [7,9]

(R) Catch Football [0,2]

14

Determining Temporal Consistency

To determine temporal consistency, an STN is converted into a distance graph

(C) Hike [0,5]

(20 yardline) ( Touchdown ! )




STN:

Distance Graph:

5

-0

Then the FIFO Label Correcting Algorithm is run on the new distance graph.

There are two possible outcomes:

1.) If the algorithm terminates, then the STN is temporally consistent

2.) If the algorithm enters an infinite loop, then the STN is temporally inconsistent - an infinite loop is detected as soon as a nodes cost drops below zero

- the infinite loop will contain the timing constraints in conflict.

-0

0

-00

8

-5

9

-7

0

-00

-0

2

-0

15

0

14

5

155

5

13

13

13

5

0

0

0

00

8

-5

9

-7

0

00

0

2

0

Determining Temporal Consistency

Update

FIFO Label-Correcting Algorithm(Graph G)

{01} for all s V(G) {02} d(s) = ∞{03a} d(sstart) = 0{03b} insert(Sstart){04a} while !Q.empty(){04b} u = Q.pop()

{05a} for v Succ(u){05b} dval = Update(u,v){06} if(dval) < 0{07} return false;{08} return true;

value Update(p,x){09} if (d(x) > d(p) + c(p,x)){10} d(x) := d(p) + c(p,x);{11} Q.Insert(x, d(x));{12} return d(x);

Check for Violating Arcs

Initialize

Run FIFO Label-Correcting Algorithm:

Original STN:

(C) Hike [0,5]





Terminates, so STN is consistent!!

16

0

20

5

5

5

13

1313

5

0

0

0

00

8

-5

9

-7

0

00

0

2

0

Detecting a Conflict

Update

FIFO Label-Correcting Algorithm(Graph G)

{01} for all s V(G) {02} d(s) = ∞{03a} d(sstart) = 0{03b} insert(Sstart){04a} while !Q.empty(){04b} u = Q.pop()

{05a} for v Succ(u){05b} dval = Update(u,v){06} if(dval) < 0{07} return false;{08} return true;

value Update(p,x){09} if (d(x) > d(p) + c(p,x)){10} d(x) := d(p) + c(p,x);{11} Q.Insert(x, d(x));{12} return d(x);

Check for Violating Arcs

InitializeRun FIFO Label-Correcting Algorithm:

Modified STN:

(C) Hike [0,5]





Node cost dropped below zero! STN is Inconsistent !!

(R) Walk to Endzone [12,15]

-12

15

1

1

1 9

9

9-3

Following the backpointers reveals conflict!

17

2nd Key Idea: Incremental Update Algorithm (ITC)

Three Arc Update Rules:

1.) Arc Change With No Effect to Shortest-Path

2.) Arc Change Improves Shortest-Path

3.) Arc Change Invalidates Shortest-Path

Use an Incremental Update Algorithm (ITC) to retain previous computations

0

14

5

155

5

13

13

13

5

0

0

0

00

8

-5

9

-7

0

00

0

2

0

Hail Mary Distance Graph:

Hail Mary STN:

(C) Hike [0,5]





18

1.) Arc Change Without Effect to Shortest-Path

Rule 1: If an arc change doesn’t affect the shortest-path, then no update is needed.

0

14

5

155

5

13

13

13

5

0

0

0

00

8

-5

9

-7

0

00

0

2

0

Hail Mary STN:

(C) Hike [0,5]






11


19

2.) Shortest-Path Improvement

Rule 2: If an arc change improves shortest-path, then just propagate the improved costs

0

14

5

155

5

13

13

13

5

0

0

0

00

8

-5

9

-7

0

00

0

2

0

Hail Mary STN:

(C) Hike [0,5]






7

12

1412

12


20

3.) Invalidation of Shortest-Path

Rule 3: If an arc change invalidates shortest-path:

First, reset all invalidated nodes to infinity, and then propagate improved costs from the changed arc’s head node.

0

14

5

155

5

13

13

13

5

0

0

0

00

8

-5

9

-7

0

00

0

2

0

Hail Mary STN:

(C) Hike [0,5]






9Hail Mary Distance Graph:

14

14

14

16

21

Glass Box Behavior of ITC

Rule 1: No effect to shortest path

Rule 2: Shortest path improvement

Rule 3: Invalidation of shortest path

START END

22

Experimental Results

The Incremental Update Algorithm (ITC) was tested on:

- UAV Scenarios - Randomly Generated STNs

NFZ1

NFZ2

WaterA2

WaterA1

WaterB1

WaterB2

Fire1

Fire2

Seeker UAV

Water UAV

No-Fly Zone

Legend:

Fire

Water

UAV Base

UAV Base

Plan Goal: Extinguish All FiresVehicles: Two Seeker UAVs

One Water UAV Resources: Fuel & Water

23

Experimental Results: UAV Scenarios

Comparison of Algorithm Runtime

0

2

4

6

8

10

12

1 10 19 28 37 46 55 64 73 82 91

Number of Activities

Alg

orith

m R

untim

e (s

ec)

Non-incrementalAlgorithm

IncrementalAlgorithm

Comparison of Algorithm Runtime

0

2

4

6

8

10

12

1 10 19 28 37 46 55 64 73 82 91

Number of Activities

Alg

orith

m R

untim

e (s

ec)

Non-incrementalAlgorithm

IncrementalAlgorithm

24

Experimental Results: Random STNs

Planner Performance Tests(on 100 randomly generated TPNs)

0

50

100

150

200

250

300

350

400

0 20 40 60 80 100 120

Th

ou

san

ds

# of Choice Nodes

# o

f T

emp

ora

l Co

nsi

sten

cy C

hec

ks inChronological BTw ithout ITC

Chrono BT w ith ITC

25

Key Assumptions and Limitations

•Kirk’s pre-defined operators are sufficiently expressive - Desired plan is expressible with Kirk’s pre-defined operators

•Activities are controllable- dispatcher can pick any specific time between the lower bound and upper bound of an activity (e.g. Hike [0,5] )

• There is no innovative problem solving or logical deduction when planning fails

26

Conclusions

• Automated Planning is Important to Aerospace Engineering

• Kirk allows flexibility in planning through flexible time-bounds

•With Incremental Update Rules, Kirk can utilize previous computations

• Experimental Data shows an order of magnitude improvement

May 12, 2004


Questions ?

May 12, 2004


Additional Slides

29

Origins of Kirk

• Deep Space 1 mission– NASA’s first fully autonomous spacecraft demonstration

• Used the HSTS planner– Used flexible time (Temporal Constraint Graph)

• Developed into Europa and then Kirk

30

Hail Mary: ( sequence ( C: hike ) ( parallel ( R: Run to Endzone ) ( QB: Throw to Endzone ) ) ( R: Catch Football ) )

Another Key Idea: Reducing Complexity

Kirk restricts activity plans to predefined operators.

Touchdown !

20 yardline (C) Hike(R) Run to Endzone

(R) Catch Football(QB) Pass to Endzone

Another Representation of the Hail Mary Activity Plan:

20

10

G

QB

C R

Hail Mary:

31

Using Conflicts to Guide the Search

• A Simple Temporal Network formed from the pre-defined operators can be transformed into a Conditional CSP.

• Then, Conflict-Directed Search Algorithms from the CSP literature such as Dynamic Backtracking and Conflict- Directed A* can be used to guide the search.

• Sometimes the conflicts returned by ITC aren’t focused enough:Planner Performance Tests

(on 100 randomly generated TPNs)

0

50

100

150

200

250

300

350

400

0 20 40 60 80 100 120

Th

ou

san

ds

# of Choice Nodes# o

f T

emp

ora

l Co

nsi

sten

cy C

hec

ks

in

Chronological BTw ithout ITC

Chrono BT w ith ITC

Dynamic BT w ith ITC

~ 3 sec

~ 0.5sec

Structured Test to show the Advantage of Dynamic Bactracking with ITC

0

20

40

60

80

100

120

2 4 6 8 10 12 14

Th

ou

sa

nd

s

# of parallel decision nodes

# o

f te

mp

ora

l co

nsi

sten

cy c

hec

ks in

ChronologicalBT w ithout ITC

Chrono BTw ith ITC

Dynamic BTw ith ITC

32

ITC Pseudo-codevoid Initialize(){01} Q := {02} for all sV(G) {03} d(s) = ∞;{04} p(s) = unknown;{05} d(sstart) = 0;{06} Q.Insert(sstart);

Conflict CheckTemporalConsistency(G){07} while !Q.empty(){08} u = Q.pop(){09} for v Succ(u){10} dval = Update(u,v){11} if(dval) < 0 {12} c = CompletedCycle(v);{13} if(c){14} return ExtractConf(c, );{15} return 0;

value Update(p,x){16} if (d(y) > d(x) + c(x,y)){17} d(y) := d(x) + c(x,y);{18} p(y) := x;{19} Q.Insert(y);{20} return d(y);

NodeCompletedCycle(v){21} if L.contains(v){22} return v;{23} else{24} L.add(v){25} return 0;

voidModifyConstraint(x,y,l,u){26} ModifyArc(arc(y,x),-l){27} ModifyArc(arc(x,y),u)

ConflictExtractConflict(c,l){28} if l.contains(c){29} return l;{30} else{31} l.add{c};{32} ExtractConflict(p(c),l);

voidModifyArc(arc,c){33} setCost(arc,c);{34} if (d(arc.head > d(arc.tail) + c){35} d(arc.head) := d(arc.tail) + c;{36} p(arc.head) := arc.tail; {37} Insert(arc.head);{38} elseif (d(arc.head) < d(arc.tail) + c)

AND (p(arc.head) == arc.tail)){39} d(arc.head) := ∞;{40} p(arc.head) := unknown;{41} InvalidateSupports(arc.head);{42} InsertParents(arc.head);

void InsertParents(n){43} for all m Pred(n){44} Insert(m);{45} if(p(m) == n){46} if(m == sstart){47} d(m) := 0; {48} else {49} d(m) := ∞;{50} p(m) := unknown;{51} InsertParents(m);

voidInvalidateSupports(n){52} for s Succ(n){53} if(p(s) == n){54} d(n) := ∞;{55} p(n) := unknown;{56} InsertParents(s);{57} InvalidateSupports(s);

Figure 6 ITC Pseudo-Code

33

ITC Arc Update Rules (Pseudocode)

{01} if (d(arc.head) < d(arc.tail) + c) AND{02} p(arc.head) == arc.tail{03} d(arc.head) := ∞;{04} p(arc.head) := unknown{05} InvalidateSupports(arc.head);{06} InsertParents(arc.head);

{01} if (d(arc.head) > d(arc.tail) + c){02} d(arc.head) := d(arc.tail) + c;{03} p(arc.head) := arc.tail; {04} Insert(arc.head);

{01} if (d(arc.head) < d(arc.tail) + c) ANDp(arc.head) ≠ arc.tail)

{02} return;

Three Arc Update Rules:

1.) Arc Change With No Effect to Shortest-Path

2.) Arc Change Improves Shortest-Path

3.) Arc Change Invalidates Shortest-Path

34

References

• R. Ahuja, T. Magnanti, J. Orlin. Network Flows: Theory, Algorithms, and Applications. Prentice Hall, 1993.• R. Bellman. On a Routing Problem. Quaterly of Applied Mathematics, vol. 16, pp. 87 90, 1958.• N. Chandrachoodan, S. Bhattachryya, and K.J. Liu. Adaptive Negative Cycle Detection in Dynamic Graphs. In Proc. of

the International Symposium on Circuits and Systems, May 2001. • R. Dechter, I. Meiri, J. Pearl. Temporal Constraint Networks. Artificial Intelligence, 49:61-95, May 1991. • T. Estlin, G. Rabideau, D. Mutz, S. Chien. Using Continuous Planning Techniques to Coordinate Multiple Rovers.

Electronic Transactions on Artificial Inttligence, 4:45-57, 2000.• P. Kim, B. Williams, and M. Abrahmson. Executing Reactive, Model-based Programs through Graph-based Temporal

Planning. In Proceedings of IJCAI-2001, Seattle, WA, 2001.• S. Koenig and M. Likhachev. Incremental A*. In Adv. in Neural Information Processing Systems 14, 2001.• D. McAllester. Truth Maintenance. In Proceedings of AAAI-90, 1990, 1109-1116.• G. Rabideau, R. Knight, S. Chien, A. Fukunaga, A. Govindjee. Iterative Repair Planning for Spacecraft Operations in

the ASPEN System. ISAIRAS, Noordwijk, The Netherlands, June 1999.• Tsmardinos, N. Muscettola, and P.Morris. Fast transformation of temporal plans for efficient execution. In AAAI-98,

1998.• B.C. Williams and R.J. Ragno. Conflict-directed A* and its role in model-based embedded systems. Journal of Discrete

Applied Math, • A. Stentz. Optimal and efficient path planning for partially known environments. In Proceedings of IEEE ICRA, May

1994.• N. Muscettola, P. Morris, B. Pell, and B. Smith. Issues in temporal reasoning for autonomous control systems. In

Autonomous Agents, 1998.• G. Verfaillie and T. Schiex. Dynamic backtracking for dynamic constraint satisfaction problems. In Proceedings of the

ECAI'94 Workshop on Constraint Satisfaction Issues Raised by Practical Applications, Amsterdam, The Netherlands, pages 1-8, 1994.

• Matthew L. Ginsberg and David A. McAllester. Gsat and dynamic backtracking. In Principles of Knowledge Representation and Reasoning: Proceedings of the Fourth International Conference (KR `94), 1994.

• P. Prosser, Hybrid algorithms for the constraint satisfaction problem, Computational Intelligence 3 (1993) 268—299.

Documents

Presented by: Robert T. Effinger