Department of Computer Science Lecture 3: Uninformed Search

Department of Computer Science

CSC-470: ARTIFICIAL INTELLIGENCE

Lecture 3: Uninformed Search

Lecturer

Muhammad Tariq Siddique

https://sites.google.com/site/mtsiddiquecs/ai

© M. Tariq Siddique 2015 Depart of Computer Science | Bahria University 2

Uninformed search Search strategies Problem formulation Uniform Search Strategies

Depth First SearchBreadth First SearchUniform cost SearchDepth-Limit Search Iterative Deepening SearchBidirectional Search

Outline


A strategy is defined by picking the order of node expansion

Strategies are evaluated along the following dimensions:

completeness – does it always find a solution if one exists?optimality – does it always find a least-cost solution?time complexity – number of nodes generated/expandedspace complexity – maximum number of nodes in memory

Time and space complexity are measured in terms ofb – maximum branching factor of the search treed – depth of the least-cost solutionm – maximum depth of the state space (may be infinite)

Search Strategies


The effectiveness of a search algorithm may only consist of search cost (depends on time complexity)

OR It may also include memory usage.OR Total cost, which combine the search cost and

the path cost of the solution found.

Search Strategies


For example:For the path finding problem from Arad to Bucharest, the

search cost is the amount of time by the search and the solution cost is the total length of the path in Kilometers.

Total cost = KM + Milliseconds. (officially no exchange rate)

So KM convert into milliseconds by estimating the car average speed (b/c time is what the agent cares about).

Thus it enable the agent to find an optimal tradeoff point for further computation to find a shorter path.

Search Strategies


Uninformed (blind) strategies use only the information available in the problem definition

vs. informed search techniques which might have additional information (e.g. a compass).1. Breadth-first search2. Depth-first search3. Uniform-cost search4. Depth-limited search5. Iterative deepening search6. Bidirectional SearchAll uninformed search can generate successors and distinguish a goal state from a nongoal state.

Uninformed Search Strategies


Root node is expanded first successor of root node their successors

All nodes are expanded at a given depth in the search tree before any nodes at the next level are expanded

Easy to implement by using FIFO queue

Breadth-First Search (BFS)


A

B CD

E F G H I J

K L M N O P Q R

S T U

Initial state

Goal state

Fringe: A (FIFO)

Successors: B,C,D

Visited:



A

B CD

E F G H I J

K L M N O P Q R

S T U

Fringe: B,C,D (FIFO)

Successors: E,F

Visited: A

Next node



A

B CD

E F G H I J

K L M N O P Q R

S T U

Fringe: C,D,E,F (FIFO)

Successors: G,H

Visited: A, B

Next node



A

B C D

E F G H I J

K L M N O P Q R

S T U

Fringe: D,E,F,G,H (FIFO)

Successors: I,J

Visited: A, B, C

Next node



A

B CD

E F G H I J

K L M N O P Q R

S T U

Fringe: E,F,G,H,I,J (FIFO)

Successors: K,L

Visited: A, B, C, D

Next node



A

B CD

E F G H I J

K L M N O P Q R

S T U

Fringe: F,G,H,I,J,K,L (FIFO)

Successors: M

Visited: A, B, C, D, E

Next node



A

B CD

E F G H I J

K L M N O P Q R

S T U

Fringe: G,H,I,J,K,L,M (FIFO)

Successors: N

Visited: A, B, C, D, E, F

Next node



A

B CD

E F G H I J

K L M N O P Q R

S T U

Fringe: H,I,J,K,L,M,N (FIFO)

Successors: O

Visited: A, B, C, D, E, F, G

Next node



A

B CD

E F G H I J

K L M N O P Q R

S T U

Fringe: I,J,K,L,M,N,O (FIFO)

Successors: P,Q

Visited: A, B, C, D, E, F, G, H

Next node



A

B CD

E F G H I J

K L M N O P Q R

S T U

Fringe: J,K,L,M,N,O,P,Q (FIFO)

Successors: R

Next node

Visited: A, B, C, D, E, F, G, H, I



A

B CD

E F G H I J

K L M N O P Q R

S T U

Fringe: K,L,M,N,O,P,Q,R (FIFO)

Successors: S

Next node

Visited: A, B, C, D, E, F, G, H, I, J



A

B CD

E F G H I J

K L M N O P Q R

S T U

Fringe: L,M,N,O,P,Q,R,S (FIFO)

Successors: T

Next node

Visited: A, B, C, D, E, F, G, H, I, J, K



A

B CD

E F G H I J

K L M N O P Q R

S T U

Fringe: M,N,O,P,Q,R,S,T (FIFO)

Successors:

Next node

Visited: A, B, C, D, E, F, G, H, I, J, K, L



A

B CD

E F G H I J

K L M N O P Q R

S T U

Fringe: N,O,P,Q,R,S,T (FIFO)

Successors:

Next node

Visited: A, B, C, D, E, F, G, H, I, J, K, L, M

Goal state achieved



Observations– Very systematic– If there is a solution breadth first search is

guaranteed to find it– If there are several solutions then breadth first

search will always find the shallowest goal state first and if the cost of a solution is a non-decreasing function of the depth then it will always find the cheapest solution

Evaluating Breadth-First Search


Completeness: Yes, if b is finite Time complexity: 1+b+b2+…+bd = O(b d), i.e.,

exponential in d Space complexity: O(b d) Optimality: Yes (assuming cost = 1 per step)

b – maximum branching factor of the search treed – depth of the least-cost solutionm – maximum depth of the state space (may be infinite)

Properties of Breadth-First Search


Search Pattern: spread before dive Fringe in red Visited in blue Size of fringe

– O(bd)– exponential

…

root

outline of tree

Breadth-First Search


Depth Nodes Time Memory0 1 1 millisecond 100 kbytes2 111 0.1 second 11 kilobytes4 11,111 11 seconds 1 megabyte6 106 18 minutes 111 megabytes8 108 31 hours 11 gigabytes

10 1010 128 days 1 terabyte12 1012 35 years 111 terabytes14 1014 3500 years 11,111 terabytes

Time and memory requirements for breadth-first search, assuming a branching factor of 10, 100 bytes per node and

searching 1000 nodes/second

Exponential Growth


Although breadth-first search is optimal when all step costs are equal -

O(bd) is scary for both time and memory required

The memory requirements are a bigger problem for breadth-first search than is the execution time.

Time is a problem, it will take 3500 years to find a solution with depth 14.



Expands the deepest node in the current frontier of the search tree until the nodes have no successors, then backs up to the next deepest node that still has unexplored successors.

Use LIFO queue The most recently generated node is chosen for

expansion, the deepest unexpanded node Use recursive

Depth-First Search (DFS)


A

B CD

E F G H I J

K L M N O P Q R

S T U

Initial state

Goal state

Fringe: A (LIFO)

Successors: B,C,D

Visited:



A

B CD

E F G H I J

K L M N O P Q R

S T U

Fringe: B,C,D (LIFO)

Successors: E,F

Visited: A



A

B CD

E F G H I J

K L M N O P Q R

S T U

Fringe: E,F,C,D (LIFO)

Successors: K,L

Visited: A, B



A

B CD

E F G H I J

K L M N O P Q R

S T U

Fringe: K,L,F,C,D (LIFO)

Successors: S

Visited: A, B, E



A

B CD

E F G H I J

K L M N O P Q R

S T U

Fringe: S,L,F,C,D (LIFO)

Successors:

Visited: A, B, E, K



A

B CD

E F G H I J

K L M N O P Q R

S T U

Fringe: L,F,C,D (LIFO)

Successors: T

Backtracking

Visited: A, B, E, K, S



A

B CD

E F G H I J

K L M N O P Q R

S T U

Fringe: T,F,C,D (LIFO)

Successors:

Visited: A, B, E, K, S, L



A

B CD

E F G H I J

K L M N O P Q R

S T U

Fringe: F,C,D (LIFO)

Successors: M

Backtracking

Visited: A, B, E, K, S, L, T



A

B CD

E F G H I J

K L M N O P Q R

S T U

Fringe: M,C,D (LIFO)

Successors:

Visited: A, B, E, K, S, L, T, F



A

B CD

E F G H I J

K L M N O P Q R

S T U

Fringe: C,D (LIFO)

Successors: G,H

Backtracking

Visited: A, B, E, K, S, L, T, F, M



A

B CD

E F G H I J

K L M N O P Q R

S T U

Fringe: G,H,D (LIFO)

Successors: N

Visited: A, B, E, K, S, L, T, F, M, C



A

B CD

E F G H I J

K L M N O P Q R

S T

Fringe: N,H,D (LIFO)

Successors:

Finished search

U

Goal state achieved

Visited: A, B, E, K, S, L, T, F, M, C, G



Only needs to store the path from the root to the leaf node as well as the unexpanded nodes. For a state space with a branching factor of b and a maximum depth of m, DFS requires storage of bm nodes

Time complexity for DFS is bm in the worst case

If DFS goes down a infinite branch it will not terminate if it does not find a goal state.

If it does find a solution there may be a better solution at a lower level in the tree. Therefore, depth first search is neither complete nor optimal.

Depth-First Search - Observations


Keeps O(bm) nodes in memory.

Requires a depth limit to avoid infinite paths (limit is 4 in the figure).

Incomplete (is not guaranteed to find a solution)

Not optimal Linear space complexity

(good) Exponential time complexity

Image from Russel & Norvig, AIMA, 2003

Black nodes are removed from memory

b = branching factor, m = depth



Search Pattern: dive before spread Fringe in red Visited in blue Size of fringe

O(bd) linear

root

outline of tree…



Much better than BFS in space complexity Graph-search : no advantage A depth-first tree search – need to store only a

single path from root to leaf, and remaining unexpanded sibling nodes . Once expanded, it can be removed from memory.



• Completeness: No, fails in infinite state-space(yes if finite state space)

• Time complexity: O(b m)• Space complexity: O(bm)• Optimality: No

Remember:

b = branching factor

m = max depth of search tree

Properties of Depth-First Search


In increasing order of effectiveness and computational overhead:

do not return to state we come from, i.e., expand function will skip possible successors that are in same state as node’s parent.

do not create paths with cycles, i.e., expand function will skip possible successors that are in same state as any of node’s ancestors.

do not generate any state that was ever generated before, by keeping track (in memory) of every state generated, unless the cost of reaching that state is lower than last time we reached it.

Avoiding repeated states


BFS: better if the branching factor of the graph is small Pros: No dead end, Complete and optimum Cons: time & memory consuming

DFS: better if tree is deep but a solution is known to be shallow in the graph Pros: small amount of memory, accidentally can

find solution in short time Cons: incomplete possibility (tree-search allow

redundant path)

Comparing BFS and DFS


Extension from breadth-first search Instead of expanding the shallowest node,

expand the node with the lowest path cost g(n)

Using priority queue ordered by g Uniformed cost search care about total cost of the

search rather than number of steps the paths has. May go to infinite loop if it ever expand a node

with cost zero leading back to the same state.

Uniform-Cost Search (UCS)


Let g(n) be path cost of node n. Expand least-cost unexpanded node Implementation:

QueueingFn = insert in order of increasing path length

Arad

Arad Oradea

7571

Arad Lugoj

118111

140

Zerind Sibiu Timisoara

75 118

75 140 118

150 146 236 229

Uniform-Cost Search (Dijkstra, 1959)


25

1 7

4 5

[5] [2]

[9][3]

[7] [8]

1 4

[9][6]

[x] = g(n)

path cost of node n

Goal state

Uniform Cost Search (UCS)


25

[5] [2]



25

1 7

[5] [2]

[9][3]



25

1 7

4 5

[5] [2]

[9][3]

[7] [8]



25

1 7

4 5

[5] [2]

[9][3]

[7] [8]

1 4

[9][6]



25

1 7

4 5

[5] [2]

[9][3]

[7] [8]

1 4

[9]

Goal state path cost g(n)=[6]



25

1 7

4 5

[5] [2]

[9][3]

[7] [8]

1 4

[9][6]



Complete: Yes if arc costs > 0. Optimal: Yes Time: Space:

Note: Breadth first is equivalent to Uniform-Cost Search if step cost is equal.

g = goaln = node

Properties of Uniform-Cost Search


Is a depth-first search with a predetermined depth limit l

Implementation:

Nodes at depth l are treated as if they have no successors.

Complete: if cutoff chosen appropriately then it is guaranteed to find a solution.

Optimal: it does not guarantee to find the least-cost solution (If choosing l >d )

Depth-Limited Search (DLS)


Given the following state space (tree search), give the sequence of visited nodes when using DLS (Limit = 2):

A

B C ED

F G H I J

K L

O

M N

Limit = 0

Limit = 1

Limit = 2



A,

A

B C ED

Limit = 2



A,B,

A

B C ED

F GLimit = 2



A,B,F,

A

B C ED

F GLimit = 2



A,B,F, G,

A

B C ED

F GLimit = 2



A,B,F, G, C,

A

B C ED

F G HLimit = 2



A,B,F, G, C,H,

A

B C ED

F G HLimit = 2



A,B,F, G, C,H, D, A

B C ED

F G H I JLimit = 2



A,B,F, G, C,H, D,I A

B C ED

F G H I JLimit = 2



A,B,F, G, C,H, D,I J,

A

B C ED

F G H I JLimit = 2



A,B,F, G, C,H, D,I J, E

A

B C ED

F G H I JLimit = 2



A,B,F, G, C,H, D,I J, E, Failure

A

B C ED

F G H I JLimit = 2



DLS algorithm returns Failure (no solution) The reason is that the goal is beyond the limit (Limit =2): the

goal depth is (d=4)

A

B C ED

F G H I J

K L

O

M N

Limit = 2



E.g. Romania case, 20 cities The longest is 19, so l can be 19 From study, cities can be reached by other

cities by 9 steps (diameter) If diameter is known, more efficient depth-

limited search

Depth limit search can be terminated by ◦ Failure no solution◦ Cutoff no solution within the depth limit

How to choose depth limit l


It’s a Depth First Search, but it does it one level at a time, gradually increasing the limit, until a goal is found.

Combine the benefits of depth-first and breadth-first search

like DFS, modest memory requirements O(bd) like BFS, it is complete when branching factor

is finite, and optimal when the path cost is a nondecreasing function of the dept of the node

Iterative Deepening Search


May seem wasteful because states are generated multiple times

but actually not very costly, because nodes at the bottom level are generated only once

In practice, however, the overhead of these multiple expansions is small, because most of the nodes are towards leaves (bottom) of the search tree:

thus, the nodes that are evaluated several times (towards top of tree) are in relatively small number.

Iterative depending is the preferred uninformed search method when the search space is large and the depth of the solution is unknown



Iterative Deepening Search l =0








Combines the best of breadth-first and depth-first search strategies.

• Completeness: Yes,• Time complexity: O(b d)• Space complexity: O(bd)• Optimality: Yes, if step cost = 1



Complete? Yes (b finite) Time? d b1 + (d-1)b2 + … + bd = O(bd)

Space? O(bd)

Optimal? Yes, if step costs identical

Properties of Iterative Deepening Search


Both search forward from initial state, and backwards from goal.

Stop when the two searches meet in the middle. Motivation: bd/2 + bd/2 is much less than bd

Implementation Replace the goal test with a check to see whether

the frontiers of the two searches intersect, if yes solution is found

GoalStart

Bidirectional Search


S

A D

B D A E

C E E B B F

D F B F C E A C G

G C G F

14

19 19 17

17 15 15 13

G 25

11

Forward

Backwards



Not always optimal, even if both searches are BFS Check when each node is expanded or selected for

expansion Can be implemented using BFS or iterative

deepening (but at least one frontier needs to be kept in memory)

Significant weakness Space requirement

Time Complexity is good



Bidirectional Search Problem: how do we search backwards from goal??

predecessor of node n = all nodes that have n as successor

this may not always be easy to compute! if several goal states, apply predecessor function to

them just as we applied successor (only works well if goals are explicitly known; may be difficult if goals only characterized implicitly).

for bidirectional search to work well, there must be an efficient way to check whether a given node belongs to the other search tree.

select a given search algorithm for each half.


• Completeness: Yes,• Time complexity: 2*O(b d/2) = O(b d/2)

• Space complexity: O(b m/2)

• Optimality: Yes

• To avoid one by one comparison, we need a hash table of size O(b m/2)

• If hash table is used, the cost of comparison is O(1)



Comparison Uninformed Search Strategies


Interesting problems can be cast as search problems, but you’ve got to set up state space

Problem formulation usually requires abstracting away real-world details to define a state space that can be explored using computer algorithms.

Once problem is formulated in abstract form, complexity analysis helps us picking out best algorithm to solve problem.

Variety of uninformed search strategies; difference lies in method used to pick node that will be further expanded.

Conclusions


Problem Solving Agents. What is a problem? Problem formulation, a general search algorithm. Read Chapter 3 from Russell & Norvig

Chapter 3 Exercises 3.1, 3.3,3.7

Reading Assignments

Documents

Department of Computer Science Lecture 3: Uninformed Search