Rutgers CS440, Fall 2003 Heuristic search Reading: AIMA 2 nd ed., Ch. 4.1-4.3

Rutgers CS440, Fall 2003

Heuristic search

Reading: AIMA 2nd ed., Ch. 4.1-4.3


(Blind) Tree search so far

• Strategies:– BFS: fringe = FIFO

– DFS: fringe = LIFO

• Strategies defined by the order of node expansion

function TreeSearch(problem, fringe)n = InitialState(problem);fringe = Insert(n);

while (1)If Empty(fringe) return failure;n = RemoveFront(fringe);If Goal(problem,n) return n;fringe = Insert( Expand(problem,n) );

endend


Example of BFS and DFS in a maze

s0

G

s0

G


Informed searches: best-first search

• Idea:

Add domain-specific information to select the best path to continue searching along.

• How to do it? – Use evaluation function, f(n), which selects the most “desirable”

(best-first) node to expand.

– Fringe is a queue sorted in decreasing order of desirability.

• Ideal case: f(n) = true cost to goal state = t(n)• Of course, t(n) is expensive to compute (BFS!)

• Use an estimate (heuristic) instead


maze example

• f(n) = straight-line distance to goal

4

s0

4

3

3

3

4

2

2

2

3

4

1

1

2

3

G1

2


Romania example


Greedy search

• Evaluation function f(n) = h(n) --- heuristic= estimate of cost from n to closest goal

• Greedy search expands nodes that appear to be closest to goal


Example of greedy search

Aradh = 366

Sibiuh = 253

Timisoarah = 329

Zerindh = 374

Aradh = 366

Fagarash = 176

Oradeah = 380

Rimnicu V.h = 193

Sibiuh = 253

Bucharesth = 0

Aradh = 366

Sibiuh = 253

Fagarash = 176

Bucharesth = 0

Is this the optimal solution?


Properties of greedy search

• Completeness:– Not complete, can get stuck in loops, e.g., Iasi -> Neamt -> Iasi ->

Neamt -> …

– Can be made complete in finite spaces with repeated-state checking

• Time complexity:– O(bm), but good heuristic can give excellent improvement

• Space complexity:– O(bm), keeps all nodes in memory

• Optimality:– No, can wander off to suboptimal goal states


A-search

• Improve greedy search by discouraging wandering-off• f(n) = g(n) + h(n)

g(n) = cost of path to reach node nh(n) = estimated cost from node n to goal nodef(n) = estimated total cost through node n

• Search along most promising path, not node

• Is A-search optimal?


Optimality of A-search

• Theorem:

Let h(n) be at most higher than t(n), for all n. Than, the A-search will be at most “steps” longer than the optimal search.

• Proof:

If n is a node on the path found by A-search, then

f(n) = g(n) + h(n) g(n) + t(n) + = optimal +


A* search

• How to make A-search optimal?• Make = 0.• This means heuristic must always underestimate the true cost

to goal. h(n) has to be an Optimistic estimate.• 0 h(n) t(n) is called an admissible heuristic.

• Theorem:

Tree A*-search is optimal if h(n) is admissible.


Example of A* search

Sibiu393 = 140+253

Timisoara477 = 118+329

Zerind449 = 75+374

Arad646 = 280+366

Fagaras415 = 239+176

Oradea671 = 291+380

Rimnicu V.413 = 220+193

Craiova526 = 366+160

Sibiu553 = 300+253

Pitesti417 = 317+100

Bucharest450 = 450+0

Sibiu591 = 338+253

Rimnicu V.607 = 414+193

Craiova615 = 455+160

Bucharestf = 418+0

Aradh = 366

Arad

Sibiu

Pitesti

Fagaras

Bucharestf = 418+0

Rimnicu V.


A* optimality proof

• Suppose there is a suboptimal goal G’ on the queue. Let n be an unexpanded node on a shortest path to optimal goal G.

s

G’n

G

• f(G’) = g(G’) + h(G’) = g(G’) since h(G’)=0 > g(G) since G’ is suboptimal f(n) = g(n) + h(n) since h is admissible

• Hence, f(G’) > f(n), and f(G’) will never be expanded

• Remember example on the previous page?


Consistency of A*

• Consistency condition:

h(n) - c(n,n’) h(n’) where n’ is any successor of n

• Guarantees optimality of graph search (remember, the proof was for tree search!)

• Also called monotonicity:

f(n’) = g(n’) + h(n’) = g(n) + c(n,n’) + h(n’) g(n) + h(n) = f(n)f(n’) f(n)

• f(n) is monotonically non-decreasing


Example of consistency/monotonicity

• A* expands nodes in order of increasing f-value• Adds “f-contours” of nodes• BFS adds layers


Properties of A*

• Completeness:– Yes (for finite graphs)

• Time complexity:– Exponential in | h(n) - t(n) | x length of solution

• Memory requirements:– Keeps all nodes in memory

• Optimal:– Yes


Admissible heuristics h(n)

• Straight-line distance for the maze and Romanian examples• 8-puzzle:

– h1(n) = number of misplaced tiles

– h2(n) = number of squares from desired location of each tile (Manhattan distance)

• h1(S) = 7

• h2(S) = 4+0+3+3+1+0+2+1 = 14


Dominance

• If h1(n) and h2(n) are both admissible and h1(n) h2(n), then h2(n) dominates over h1(n)

• Which one is better for search?– h2(n), because it is “closer” to t(n)

• Typical search costs

d = 14, IDS = 3,473,941 nodesA*(h1) = 539A*(h2) = 113

d = 24, IDS ~ 54 billion nodesA*(h1) = 39135A*(h2) = 1641


Relaxed problems

• How to derive admissible heuristics?• Can be derived from exact solutions to problems that are

“simpler” (relaxed) versions of the problem one is trying to solve

• Examples:– 8-puzzle:

• Tile can move anywhere from initial position (h1)• Tiles can occupy same square but have to move one square at a time

(h2)

– maze:• Can move any distance and over any obstacles

• Important:

The cost of optimal solution to relaxed problems is no greater than the optimal solution to the real problem.


Local search

• In many problems one does not care about the path, rather one wants to find the goal state (based on a goal condition).

• Use local search / iterative improvement algorithms:

– Keep a single “current” state, try to improve it.


Example

• N-queens problem:

from initial configuration move to other configurations such that the number of conflicts is reduced


Hill-climbingGradient ascent / descent

• Goal:

n* = arg max Value( n )

function HillClimbing(problem)n = InitialState(problem);

while (1)neighbors = Expand(problem,n);n* = arg max Value(neighbors);If Value(n*) < Value(n), return n;n = n*;

endend


Hill climbing (cont’d)

• Problem:

Depending on initial state, can get stuck in local maxima (minima), ridges, plateaus


Beam search

• Problem of local minima (maxima) in hill-climbing can be alleviated by starting HC from multiple random starting points

• Or make it stochastic (Stochastic HC) by choosing successors at random, based on how “good” they are

• Local beam search: somewhat similar to random-restart HC:– Start from N initial states.

– Expand all N states and keep N best successors.

• Stochastic beam search: stochastic version of LBS, similar to SHC.


Simulated annealing

• Idea: – Allow bad moves, initially more, later fewer

– Analogy with annealing in metallurgy

function SimulatedAnnealing(problem,schedule)n = InitialState(problem);t = 1;

while (1)T = schedule(t);neighbors = Expand(problem,n);n’ = Random(neighbors);V = Value(n’) - Value(n);If V > 0, n* = n;Else n* = n’ with probability exp(V/T);t = t + 1;

endend


Properties of simulated annealing

• At fixed temperature T, state occupation probability reaches Boltzman distribution, exp( V(n)/T )

• Devised by Metropolis et al. in 1953


Genetic algorithms

Documents

Rutgers CS440, Fall 2003 Heuristic search Reading: AIMA 2 nd ed., Ch. 4.1-4.3