Problem Solving as State Space Search Assignmentsdspace.mit.edu/bitstream/handle/1721.1/71858/16-410-fall-2005/con… · – Problem solving as state space search • Mathematical

Problem Solving as State Space Search

Brian C.Williams 16.410-13 Session 3

Slides adapted from: 6.034 Tomas Lozano Perez, Russell and Norvig AIMA Brian Williams, Fall 05 1

Assignments

• Remember: Problem Set #1: Simple Scheme and Search Out today, due Session 4.

• Reading: – Solving problems by searching: AIMA Ch. 3

Brian Williams, Fall 05 2

Course Objective 1: Agent Architectures

To understand the major types of agents and common architectures used to develop agents.

– Mission-oriented Agents – Self-repairing Agents – Mobile Agents – Agile Agents – Communicating Agents

Brian Williams, Fall 05 3 Brian Williams, Fall 05 4

Plan

ExecuteMonitor & Diagnosis

Locate in World

Plan Routes

Map

Maneuver and Track

Communicate and Interpret

Course Objective 1: Agent Architectures

Course Objective 2: Principles of Agents

16.410/13: To learn the modeling and algorithmic building blocks for creating reasoning and learning agents:

1. To formulate reasoning problems in an appropriate formal representation.

2. To describe, analyze and demonstrate the application of reasoning algorithms to solve these problem formulations.


Agent Building Blocks

• Activity Planning • Path Planning • Execution/Monitoring • Localization • Diagnosis • Map Building • Repair • Trajectory Design • Scheduling • Policy Construction • Resource Allocation

�Most reasoning problems, like these, may be formulated as state space search.


1

Course Objective 3: Implementing Agents

16.413: To appreciate the challenges of building a state of the art autonomous explorer:

Fall 03: • Mars Exploration Rover shadow mode demonstration. Fall 04: • Gnu Robot competition. Fall 05: • Model-based autonomy toolbox • The virtual solar system • Stay tuned for more.


Complex missions must carefully:

�Most AI problems, like these, may be formulated as state space search.

• Plan complex sequences of actions

• Schedule actions

• Allocate tight resources

• Monitor and diagnose behavior

• Repair or reconfigure hardware.

Courtesy of Kanna Rajan, NASA Ames. Used with permission.

Outline

• Problem Formulation – Problem solving as state space search

• Mathematical Model – Graphs and search trees

• Reasoning Algorithms – Depth and breadth-first search


Problem Solving as State Space Search

• Formulate Goal – State

• Astronaut, Fox, Goose & Grain across river

• Formulate Problem – States

• Location of Astronaut, Fox, Goose & Grain at top or bottom river bank

– Operators • Astronaut drives rover and 1 or 0 items

to other bank.

• Generate Solution – Sequence of Operators (or States)

• Move(goose,astronaut), Move(astronaut), . . . Brian Williams, Fall 05 11


Early AI: What are the universal problem solving methods?

Astronaut Goose Grain Fox

Rover

Can the astronaut get its supplies safely across the Martian canal?

allowed in the rover.

Simple Trivial

• Astronaut + 1 item

• Goose alone eats Grain • Fox alone eats Goose

Courtesy of NASA.


Astronaut Goose Grain

Fox


Fox

Grain Fox

Astronaut Goose

Goose Grain

Astronaut Fox

Goose Fox

Astronaut Grain

Goose Grain Fox

Astronaut

2

05 05Brian Williams, Fall 13


Fox

Grain Fox

Astronaut Goose


Goose Fox

Astronaut Grain

Astronaut Grain Fox

Goose

Astronaut Goose

Grain Fox

Goose Grain

Astronaut Fox

Astronaut Goose Fox

Grain

Grain

Astronaut Goose Fox

Fox


Goose

Astronaut Fox

Grain

Goose Grain

Fox

Astronaut

Astronaut Grain

Goose Fox

Astronaut Fox

Goose Grain

Astronaut

Goose Grain Fox


Fox

Brian Williams, Fall 14


Grain Fox

Astronaut Goose

Astronaut Grain Fox

Goose

Goose

Astronaut Fox

Grain

Astronaut Goose

Grain Fox


Grain

Astronaut Goose Fox

Fox


Goose Fox

Astronaut Grain

Goose Grain

Astronaut Fox

Astronaut Goose Fox

Grain

Astronaut Grain

Goose Fox

Astronaut Fox

Goose Grain

Goose Grain Fox

Astronaut

Astronaut

Goose Grain

Fox


Fox


• States: • Operators: • Goal Test:

5 4

6 1

7 3

8

2

1 2

8

3

7 6

4

5

Start Goal

integer location for each tile AND … move empty square up, down, left, right goal state as given

Planning as State Space Search: STRIPS Operator Representation

Initial state: (and (hose a)

(clamp b)

(clear a) (clear b) (clear c) (empty arm))

goal (partial state): (and (connected a b)

(attached b a)))

precondition: (and (clear hose) table hose)

(empty arm))

effect : (and (not (clear hose)) (not (on -table hose)) (not (empty arm))

(holding arm hose)))

pickup hose

}

Available actions Strips operators

A Plan is an operator sequence from the initial state to the goal. Brian Williams, Fall 05 18

STRIPS Action Assumptions

• Atomic time.

• Agent is omniscient (no sensing necessary).

• Agent is sole cause of change.

• Actions have deterministic effects.

• No indirect effects.

precondition: (and (clear hose)

(empty arm))

effect : (and (not (clear hose)) (not (on (not (empty arm))

(holding arm hose)))

pickup hose

Formulation Example: 8-Puzzle

(hydroxide-unit c) (on-table a) (on-table b) (on-table c)

(on-

(on-table hose)

-table hose))

Example: Planning Discrete Actions

• Swaggert & Lovell work on Apollo 13 emergency riglithium hydroxide unit. – Assembly

• Mattingly works in groundsimulator to identify newsequence handling severe power limitations. – Planning & Resource Allocation

• Mattingly identifies novel reconfiguration, exploitingLEM batteries for power.

courtesy of NASA – Reconfiguration and Repair


Courtesy of Kanna Rajan, NASA Ames. Used with permission.

allow derived effects;

� Effects specify how to change the set of assertions.


you must be complete!

Note: strips doesn’t

3

Succinct Encodings: STRIPS Action Schemata

• Instead of defining: pickup-hose and pickup-clamp and …

• Define a single pickup schema (with variables ?v): (:operator pick-up

:parameters ((hose ?ob1)) :precondition (and (clear ?ob1)

(on-table ?ob1) (empty arm))

:effect (and (not (clear ?ob1)) (not (on-table ?ob1)) (not (empty arm)) (holding arm ?ob1)))


Outline





Problem Formulation: A Graph

Operator State

de

Edge Vertex

Tail Vertex of Edge

Head Vertex of Edge

neighbors (adjacent)

UndirectedDirected Incident to edge GraphGraph(one-way streets) (two-way streets)


Problem Formulation: A Graph

In Degree (2) Degree (1)

Out Degree (1) b b

Directed Graph (one-way streets)

Undirected Graph (two -way streets)


Problem Formulation: A Graph Strongly connected graph Connected graph

Directed path between all vertices. Path between all vertices.

Complete graph All vertices are adjacent.

a c

e d

b

a c

d e

b

Sub graph Subset of vertices edges between vertices in Subset

Undirected A complete subgraph GraphGraph (All vertices are adjacent). (two-way streets) (one-way streets)

Directed Clique


Examples of Graphs

Roadmap

Planning Actions Put B on C

(graph of possible states of the world)

Put C on A

Put C on A

A B

C A

B

C

A B C

Put A on C Put C on B

A B

C

Brian Williams, Fall 05

A

B

C

24

San Fran

Boston

LA Dallas

Wash DC

4

Formal Representation of a Graph

SFO

Bos

LA Dallas

Wash DC

Airline Routes

A Graph G is represented as an (ordered) pair <V,E>, where:

• V is a set of vertices {v1 …} < {Bos, SFO, LA, Dallas, Wash DC}

• E is a set of (directed) edges {e1, …} {<SFO, Bos>,

An edge is a pair <v1, v2> of vertices, where <SFO, LA>

• v2 is the head of the edge, <LA, Dallas>

•and v1 is the tail of the edge <Dallas, Wash DC>

Brian Williams, Fall 05 . . .} > 25 Note: A set has no duplicate elements.

end

A (directed) path is a sequence of vertices <v1, … vn> <S, A, D, C> such that each successive pair <vi,vi+1> is a (directed) edge in G

A simple path has no repeated vertices. A cycle is a subpath where start = end.


A Solution is a State Sequence: Problem Solving Searches Paths

Given Graph G, with start vertex S, and goal G, a solution is a simple path from S to G.

C

S

B

G A

D start

S

D

A

C



C

S

B

G A

D

S

D

A

C G

C

S

B

G A

D

S

D

BA

C G

C G

D

C G

Represent searched paths using a tree.


Represent searched paths using a tree.


Search Trees

Branch Node(Edge) (Vertex)

Root


Search Trees

Parent

Siblings

Child


5


Search Trees

Ancestors

Descendants

Outline





Classes of Search

Blind Depth-First Systematic exploration of whole tree

(uninformed) Breadth-First until the goal is found.

Iterative-Deepening


Search Trees

A tree T is a directed Graph, such that there exists exactly one directed path between any pair of vertices.


Classes of Search

Blind Depth-First Systematic exploration of whole tree

(uninformed) Breadth-First until the goal is found.

Iterative-Deepening

Heuristic Hill-Climbing Uses heuristic measure of goodness

(informed) Best -First of a node,e.g. estimated distance to.

Beam goal.

Optimal Branch&Bound Uses path “length” measure. Finds

(informed) A* “shortest” path. A* also uses heuristic


Depth First Search (DFS) Idea: After visiting node •Visit children, before siblings •Visit siblings left to right

1 S

D

BA

C G

C G

D

C G

S

A

D

C G

C

B

D

C G

G

2 7

3 6 8 11

4 5 9 10


6

Breadth First Search (BFS) Idea: After visiting node • Visit siblings, before children • Visit relatives left to right (top to bottom)

1 S

D

BA

C G

C G

D

C G

S

A

D

C G

C

B

D

C G

G

2 3

4 5 6 7

8 9 10 11


Outline

• Problem Formulation: State space search • Model: Graphs and search trees • Reasoning Algorithms: DFS and BFS

– A generic search algorithm – Depth-first search example – Handling cycles – Breadth-first search example


Simple Search Algorithm • S denotes the start node • G denotes the goal node. • A partial path is a path from S to some node D,

• e.g., <D, A, S> note reverse order. S

D

BA

C G

C G

D

C G

• The head of a partial path is the most recent visited node of the path, • e.g., D.

• The Q is a sequence of partial paths, • e.g. (<D, A, S>, <C, A, S> …>.


Elements of Algorithm Design Description: (today)

– stylized pseudo code, sufficient to analyze and implement the al gorithm (implementation next Wednesday).

Analysis: (following Monday) • Soundness:

– when a solution is returned, is it guaranteed to be correct? • Completeness:

– is the algorithm guaranteed to find a solution when there is one?

• Time complexity: – how long does it take to find a solution?

• Space complexity: – how much memory does it need to perform search?


Simple Search Algorithm Going Meta:

How do we maintain the search state? Search is State Space Search

• A set of partial paths explored thus far. State Space

• An ordering, specifying which partial path to expand next • (called aqueue Q.)

How do we perform search? • Repeatedly:

S

D

BA

C G

C G

D

C G

• Select next partial path from Q. • Expand it. Operator • Add expansions to Q.

• Terminate when goal found. Goal-Test Brian Williams, Fall 05 40

Simple Search Algorithm Let Q be a list of partial paths, Let S be the start node and Let G be the Goal node.

1. Initialize Q with partial path <S>

2. If Q is empty, fail. Else, pick a partial path N from Q

3. If head(N) = G, return N (goal reached!)

4. Else:

a) Remove N from Q

b) Find all children of head(N) and create all one-step extensions of N to each child.

c) Add all extended paths to Q

d) Go to step 2. Brian Williams, Fall 05 42

7

Depth First Search (DFS)Outline

Idea: • Visit children, then siblings

• Problem Formulation: State space search • Visit siblings left to right, (top to bottom).

• Model: Graphs and search trees 1

• Reasoning Algorithms: DFS and BFSS

D

BA

C G

C G

D

C G

S

A

D

C G

C

B

D

C G

G

2 7

– A generic search algorithm 3 6 8 11

– Depth-first search example 4 5 9 10 – Handling cycles – Breadth-first search example Assuming that we pick the first element of Q,

Then where do we add path extensions to the Q?


Simple Search Algorithm Depth-First Let Q be a list of partial paths, Let S be the start node and Pick first element of Q; Add path extensions to front of QLet G be the Goal node.

1. Initialize Q with partial path (S)



Q

1 (S)

2

3

4

5

1

C

S

B

GA

D

4. Else:

a) Remove N from Q

b) Find all children of head(N) and create all the one-step extensions of N to each child.


d) Go to step 2.Brian Williams, Fall 05 45 Brian Williams, Fall 05 46





Q

1 (S)

2

3

4

5

1

C

S

B

GA

D

4. Else:

a) Remove N from Q




8

C

S

B

G A

D

Depth-First Depth-First Pick first element of Q; Add path extensions to front of Q Pick first element of Q; Add path extensions to front of Q

Q

1 (S)

2 (A S)

3

4

5

1

C

S

B

GA

D

C

S

B

GA

D1

Added paths in blue Added paths in blue



1. Initialize Q with partial path (S) Q

1 (S)

2 (A S) (B S)

3

4

5


3. If head(N) = G, return N (goal reached!) 1

4. Else:

a) Remove N from Q

b) Find all children of head(N) and create all the one-step extensions of N to each child. Added paths in blue



Q

1 (S)

2 (A S) (B S)

3

4

5

Depth-First Pick first element of Q; Add path extensions to front of Q

Depth-First Pick first element of Q; Add path extensions to front of Q

Q

1 (S)

2 (A S) (B S)

3 (C A S) (D A S) (B S)

4

5

C

S

B

G A

D1

2

C

S

B

G A

D

Q

5

4

3

2

1

(A S) (B S)

(S)

1

2

(C A S) (D A S) (B S)

Added paths in blue


Added paths in blue


9

C

S

B

G A

D

2

4

C

S

B

G A

D

Q

5

4

3

2

1

(C D A S)(G D A S) (B S)

(D A S) (B S)

(A S) (B S)

(S)

1

2

4

(C A S) (D A S) (B S)

C

S

B

G A

D

2

4

C

S

B

G A

D

6

Q

5

4

3

2

1


(D A S) (B S)

(A S) (B S)

(S)

1

2

4

(G D A S) (B S)

(C A S) (D A S) (B S)


Q

1 (S)

2 (A S) (B S)

3 (C A S) (D A S) (B S)

4

5

C

S

B

GA

D

2

C

S

B

GA

D1

2

3 3

1

Added paths in blue Added paths in blue



Q

1 (S)

2 (A S) (B S)

3 (C A S) (D A S) (B S)

4 (D A S) (B S)

5

3 3

1

Added paths in blue



Q

1 (S)

2 (A S) (B S)

3 (C A S) (D A S) (B S)

4 (D A S) (B S)

5 (C D A S)(G D A S) (B S)

3 3

1


Q

1 (S)

2 (A S) (B S)

3 (C A S) (D A S) (B S)

4 (D A S) (B S)

5

10

6

5

4

3

2

1


C

S

B

G A

D

6

Q

5

4

3

2

1


(D A S) (B S)

(A S) (B S)

(S)

1

2

3

4

Pick first element of Q; Add path extensions to front of Q






4.

a) Remove N from Q

c) Add all extended paths to Q (G D A S) (B S)

(C A S) (D A S) (B S)

Depth-First

Else:


d) Go to step 2.


Outline



C

S

B

G A

D


Issue: Starting at S and moving top to bottom, will depth-first search ever reach G?


C

S

B

G A

D

Q


(D A S) (B S)

(A S) (B S)

(S)

1

2

3

4

How much wasted effort can be incurred in the worst case?

Effort can be wasted in more mild cases


How Do We Avoid Repeat Visits? Idea:

• Keep track of nodes already visited.

• Do not place visited nodes on Q.

Does this maintain correctness?

• Any goal reachable from a node that was visited a second time would be reachable from that node the first time.

Does it always improve efficiency?

• Visits only a subset of the original paths, such that each node appears at most once at the head of a path in Q.

(G D A S) (B S)

(C A S) (D A S) (B S)

Depth-First

• C visited multiple times • Multiple paths to C, D & G

11

How Do We Modify The Simple Search Algorithm?

Let Q be a list of partial paths, Let S be the start node and Let G be the Goal node.

1. Initialize Q with partial path (S) as only entry;

2. If Q is empty, fail. Else, pick some partial path N from Q


4. Else

a) Remove N from Q


c) Add to Q all the extended paths;

d) Go to step 2. Brian Williams, Fall 05 67

Testing for the Goal • This algorithm stops (in step 3) when head(N) = G.

• We could have performed this test in step 6 as each extended path is added to Q. This would catch termination earlier and be perfectly correct for all the searches we have covered so far.

• However, performing the test in step 6 will be incorrect for th e optimal searches we look at later. We have chosen to leave the test in step 3 to maintain uniformity with these future searches.


Breadth First Search (BFS) Idea: • Visit siblings before their children • Visit relatives left to right

1 S

D

BA

C G

C G

D

C G

S

A

D

C G

C

B

D

C G

G

2 3

4 5 6 7

8 9 10 11

Assuming that we pick the first element of Q,Then where do we add path extensions to the Q?



1. Initialize Q with partial path (S) as only entry; set Visited = {}

2. If Q is empty, fail. Else, pick some partial path N from Q


4. Else

a) Remove N from Q

b) Find all children of head(N) not in Visited and create all the one-step extensions of N to each child.

c) Add to Q all the extended paths;

d) Add children of head(N) to Visited Brian Williams, Fall 05 68

e) Go to step 2.

Outline




Breadth-First w Visited Pick first element of Q; Add path extensions to end of Q

Q Visited

1 (S) S

2

3

4

5

6

C

S

B

G A

D1


12

Breadth-First w Visited Breadth-First w Visited Pick first element of Q; Add path extensions to end of Q Pick first element of Q; Add path extensions to end of Q

Q Visited

1 (S) S

2

3

4

5

6

C

S

B

G A

D1

C

S

B

G A

D

6

VisitedQ

5

4

3

2

1

(A S) (B S)

S(S)

1

A,B,S



Q Visited

1 (S) S

2 (A S) (B S) A,B,S

3

4

5

6

C

S

B

G A

D1

2

C

S

B

G A

D

6

VisitedQ

5

4

3

2

1

(A S) (B S)

S(S)

1

2

C,D,B,A,S (B S) (C A S) (D A S)

A,B,S



Q Visited

1 (S) S

2 (A S) (B S) A,B,S

3 (B S) (C A S) (D A S) C,D,B,A,S

4

5

6

C

S

B

G A

D1

2

3

C

S

B

G A

D

6

VisitedQ

5

4

3

2

1

(A S) (B S)

S(S)

1

2

3

G,C,D,B,A,S (C A S) (D A S) (G B S)*

C,D,B,A,S (B S) (C A S) (D A S)

A,B,S

* We could stop here, when the first path to the goal is generated. Brian Williams, Fall 05 77 Brian Williams, Fall 05 78

13


Q Visited

1 (S) S

2 (A S) (B S) A,B,S

3 (B S) (C A S) (D A S) C,D,B,A,S

4 (C A S) (D A S) (G B S)* G,C,D,B,A,S

5

6

C

S

B

G A

D1

2

3

4 C

S

B

G A

D

6

VisitedQ

5

4

3

2

1

G,C,D,B,A,S(D A S) (G B S)

G,C,D,B,A,S

C,D,B,A,S

A,B,S(A S) (B S)

S(S)

1

2

3

4

5

(C A S) (D A S) (G B S)*

(B S) (C A S) (D A S)

* We could stop here, when the first path to the goal is generated. Brian Williams, Fall 05 79 Brian Williams, Fall 05 80


Q Visited

1 (S) S

2 (A S) (B S) A,B,S

3 (B S) (C A S) (D A S) C,D,B,A,S

4 (C A S) (D A S) (G B S)* G,C,D,B,A,S

5 (D A S) (G B S) G,C,D,B,A,S

6 (G B S) G,C,D,B,A,S

C

S

B

G A

D1

2

3

4

5

6 C

S

B

G A

D

G,C,D,B,A,S(G B S)6

VisitedQ

5

4

3

2

1

G,C,D,B,A,S(D A S) (G B S)

G,C,D,B,A,S

C,D,B,A,S

A,B,S(A S) (B S)

S(S)

1

2

3

4

5

6

(C A S) (D A S) (G B S)*

(B S) (C A S) (D A S)


Depth-first with visited list Depth First Search (DFS) S

D

BA

C G

C G

D

C G

Pick first element of Q; Add path extensions to front of Q Depth-first:

Add path extensions tofront of Q

Q Visited

1 (S) S

2 (A S) (B S) A, B, S

3 (C A S) (D A S) (B S) C,D,B,A,S

4 (D A S) (B S) C,D,B,A,S

5 (G D A S) (B S) G,C,D,B,A,S

C

S

B

GA

D4

3 Pick first element of Q

52

Breadth First Search (BFS)1 S

D

BA

C G

C G

D

C G

Breadth-first:

Add path extensions to back of Q

Pick first element of Q

For each search type, where do we place the children on the queue? Brian Williams, Fall 05 83 Brian Williams, Fall 05 84

14

What You Should Know • Most problem solving tasks may be

formulated as state space search. • Mathematical representations for search are Appendix

graphs and search trees. • Depth-first and breadth-first search may be

framed, among others, as instances of a generic search strategy.

• Cycle detection is required to achieve efficiency and completeness.


Breadth-First (without Visited list) Breadth-First (without Visited list) Pick first element of Q; Add path extensions to end of Q Pick first element of Q; Add path extensions to end of Q

Q

1 (S)

2

3

4

5

6

7

C

S

B

G A

D1

C

S

B

G A

D

7

6

Q

5

4

3

2

1

(A S) (B S)

(S)

1

2

Added paths in blue


Breadth-First (without Visited list) Breadth-First (without Visited list) Pick first element of Q; Add path extensions to end of Q Pick first element of Q; Add path extensions to end of Q

Q

1 (S)

2 (A S) (B S)

3 (B S) (C A S) (D A S)

4

5

6

7

C

S

B

G A

D1

2

3

C

S

B

G A

D

7

6

Q

5

4

3

2

1

(A S) (B S)

(S)

1

2

3

4

(C A S) (D A S) (D B S) (G B S)*

(B S) (C A S) (D A S)

Added paths in blue Added paths in blue Revisited nodes in pink * We could have stopped here, when the first path to the goal was generated.


15

Breadth-First (without Visited list) Pick first element of Q; Add path extensions to end of Q


Q

1 (S)

2 (A S) (B S)

3 (B S) (C A S) (D A S)

4 (C A S) (D A S) (D B S) (G B S)*

5 (D A S) (D B S) (G B S)

6

7

C

S

B

G A

D1

2

3

4

5

C

S

B

G A

D1

2

3

4

5 6

Q

1 (S)

2 (A S) (B S)

3 (B S) (C A S) (D A S)

4 (C A S) (D A S) (D B S) (G B S)*

5 (D A S) (D B S) (G B S)

6 (D B S) (G B S) (C D A S) (G D A S)

7



Q

1 (S)

2 (A S) (B S)

3 (B S) (C A S) (D A S)

4 (C A S) (D A S) (D B S) (G B S)*

5 (D A S) (D B S) (G B S)

6 (D B S) (G B S) (C D A S) (G D A S)

7 (G B S) (C D A S) (G D A S)

C

S

B

G A

D1

2

3

4

5 6

7



16

Documents

Problem Solving as State Space Search Assignmentsdspace.mit.edu/bitstream/handle/1721.1/71858/16-410-fall-2005/con… · – Problem solving as state space search • Mathematical