1

CS 2710, ISSP 2610

Chapter 4, Part 2Heuristic Search

2

Beam Search

• Cheap, unpredictable search• For problems with many solutions,

it may be worthwhile to discard unpromising paths

• Greedy best first search that keeps a fixed number of nodes on the fringe

3

Beam Search

def beamSearch(fringe,beamwidth): while len(fringe) > 0: cur = fringe[0] fringe = fringe[1:] if goalp(cur): return cur newnodes = makeNodes(cur, successors(cur)) fringe=sortByH(newnodes, fringe) fringe = fringe[:beamwidth]return []

4

Beam Search

• Optimal? Complete?• Hardly!• Space?• O(b) (generates the successors)• Often useful

5

Generating Heuristics

• Exact solutions to different (relaxed problems)

• H1 (# of misplaced tiles) is perfectly accurate if a tile could move to any square

• H2 (sum of Manhattan distances) is perfectly accurate if a tile could move 1 square in any direction

6

Relaxed Problems (cont)

• If problem is defined formally as a set of constraints, relaxed problems can be generated automatically

• A tile can move from square A to square B if:– A is adjacent to B, and– B is blank

• 3 relaxed problems by removing one or both constraints

• Absolver (Prieditis, 1993)

7

Combining Heuristics

• If you have lots of heuristics and none dominates the others and all are admissible…

• Use them all!• H(n) = max(h1(n), …, hm(n))

8

Generating Heuristics

• Use the solution cost of a subproblem. E.g., get tiles 1 through 4 in the right location, ignoring the others.

• Pattern databases: store exact solution costs for every possible subproblem instance. – The heuristic function looks up the value in

the DB – DB constructed by searching back from goal

and recording cost of each new pattern • Do the same for tiles 5,6,7,8 and take

max

9

Other Sources of Heuristics

• Ad-hoc, informal, rules of thumb (guesswork)• Approximate solutions to problems (algorithms

course)• Learn from experience (solving lots of 8-

puzzles). – Each optimal solution is a learning example

(node,actual cost to goal)– Learn heuristic function, E.G. H(n) = c1x1(n)

+ c2x2(n) x1 = #misplaced tiles; x2 = #adj tiles also adj in the goal state. c1 & c2 learned (best fit to the training data)

10

Remaining Search Types

• Recall we have…– Backtracking state-space search– Local Search and Optimization – Constraint satisfaction search

11

Local Search and Optimization

• Previous searches: keep paths in memory, and remember alternatives so search can backtrack. Solution is a path to a goal.

• Path may be irrelevant, if the final configuration only is needed (8-queens, IC design, network optimization, …)

12

Local Search

• Use a single current state and move only to neighbors.

• Use little space• Can find reasonable solutions in

large or infinite (continuous) state spaces for which the other algorithms are not suitable

13

Optimization

• Local search is often suitable for optimization problems. Search for best state by optimizing an objective function.

14

Visualization

• States are laid out in a landscape• Height corresponds to the

objective function value• Move around the landscape to find

the highest (or lowest) peak• Only keep track of the current

states and immediate neighbors

15

Local Search Alogorithms

• Two strategies for choosing the state to visit next– Hill climbing– Simulated annealing

• Then, an extension to multiple current states:– Genetic algorithms

16

Hillclimbing (Greedy Local Search)

• Generate nearby successor states to the current state based on some knowledge of the problem.

• Pick the best of the bunch and replace the current state with that one.

• Loop

17

Hill-climbing search problems

• Local maximum: a peak that is lower than the highest peak, so a bad solution is returned

• Plateau: the evaluation function is flat, resulting in a random walk

• Ridges: slopes very gently toward a peak, so the search may oscillate from side to side

Local maximum Plateau Ridge

18

Random restart hill-climbing

• Start different hill-climbing searches from random starting positions stopping when a goal is found

• If all states have equal probability of being generated, it is complete with probability approaching 1 (a goal state will eventually be generated).

• Best if there are few local maxima and plateaux

19

Random restart hill-climbingHmm…

• If all states have equal probability of being generated, it is complete with probability approaching 1 (a goal state will eventually be generated).

• If it is restarted enough times, we considered… • Well, hill-climbing stops when no neighbors are

better than current• It stops on a local optimum (whether or not it

is a global optimum). • So, “more iterations” does not seem to be the

point.

20

Random restart hill-climbingHmm…

• If all states have equal probability of being generated, it is complete with probability approaching 1 (a goal state will eventually be generated). Any way to see this as true?

• R&N,p. 111: A complete local search algorithm always finds a goal if one exists; an optimal algorithm always finds a global minimum/maximum.

• So, “goal” means local minimum/maximum

21

Simulated Annealing

• Based on a metallurgical metaphor– Start with a temperature set very

high and slowly reduce it.– Run hillclimbing with the twist that

you can occasionally replace the current state with a worse state based on the current temperature and how much worse the new state is.

22

Simulated Annealing

• Annealing: harden metals and glass by heating them to a high temperature and then gradually cooling them

• At the start, make lots of moves and then gradually slow down

23

Simulated Annealing

• More formally…– Generate a random new neighbor

from current state.– If it’s better take it.– If it’s worse then take it with some

probability proportional to the temperature and the delta between the new and old states.

24

Simulated annealing

• Probability of a move decreases with the amount ΔE by which the evaluation is worsened

• A second parameter T is also used to determine the probability: high T allows more worse moves, T close to zero results in few or no bad moves

• Schedule input determines the value of T as a function of the completed cycles

25

function Simulated-Annealing(problem, schedule) returns a solution stateinputs: problem, a problem

schedule, a mapping from time to “temperature”local variables: current, a node

next, a nodeT, a “temperature” controlling the

probability of downward stepscurrent ← Make-Node(Initial-State[problem])for t ← 1 to ∞ do

T ← schedule[t]if T=0 then return currentnext ← a randomly selected successor of currentΔE ← Value[next] – Value[current]if ΔE > 0 then current ← nextelse current ← next only with probability eΔE/T

26

Local Beam Search

• Keep track of k states rather than just one, as in hill climbing

• In comparison to beam search we saw earlier, this alg is state-based rather than node-based.

27

Local Beam Search

• Begins with k randomly generated states

• At each step, all successors of all k states are generated

• If any one is a goal, alg halts• Otherwise, selects best k

successors from the complete list, and repeats

28

Local Beam Search

• Successors can become concentrated in a small part of state space

• Stochastic beam search: choose k successors, with probability of choosing a given successor increasing with value

• Like natural selection: successors (offspring) of a state (organism) populate the next generation according to its value (fitness)

29

Genetic Algorithms

• Variant of stochastic beam search• Combine two parent states to

generate successors (sexual versus asexual reproduction)

30

Fun GA (pop, fitness-fn)Repeat new-pop = {} for i from 1 to size(pop): x = rand-sel(pop,fitness-fn) y = rand-sel(pop,fitness-fn) child = reproduce(x,y) if (small rand prob): child mutate(child) add child to new-pop pop = new-popUntil an indiv is fit enough, or out of timeReturn best indiv in pop, according to fitness-fn

31

Fun reproduce(x,y) n = len(x) c = random num from 1 to n return:

append(substr(x,1,c),substr(y,c+1,n)

32

Example: n-queens

• Put n queens on an n × n board with no two queens on the same row, column, or diagonal

•

33

Genetic AlgorithmsNotes

• Representation of individuals– Classic approach: individual is a string over a finite

alphabet with each element in the string called a gene– Usually binary instead of AGTC as in real DNA

• Selection strategy– Random– Selection probability proportional to fitness– Selection is done with replacement to make a very fit

individual reproduce several times

• Reproduction– Random pairing of selected individuals– Random selection of cross-over points– Each gene can be altered by a random mutation

34

Genetic AlgorithmsWhen to use them?

• Genetic algorithms are easy to apply

• Results can be good on some problems, but bad on other problems

• Always good to give a try before spending time on something more complicated