Escaping Local Optima

Where are we?Optimization methodsOptimization methods

Complete solutionsComplete solutions Partial solutionsPartial solutions

Exhaustive searchHill climbingRandom restartGeneral Model of Stochastic Local SearchSimulated AnnealingTabu search

Exhaustive searchHill climbingRandom restartGeneral Model of Stochastic Local SearchSimulated AnnealingTabu search

Exhaustive searchBranch and boundGreedyBest firstA*Divide and ConquerDynamic programmingConstraint Propagation

Exhaustive searchBranch and boundGreedyBest firstA*Divide and ConquerDynamic programmingConstraint Propagation

Escaping local optima

Stochastic local search many important algorithms address

the problem of avoiding the trap of local optima(possible source of project topics)

Michalewicz & Fogel focus on two only simulated annealing tabu search

Formal model of Stochastic Local Search (SLS): Hoos and Stützle

goal:abstract the simple search subroutines from high level

control structureexperiment with various search methods

systematically

Formal model of Stochastic Local Search (SLS): Hoos and Stützle

Generalized Local Search Machine (GLSM) MM =(Z, z0, M, m0, Δ, σZ, σΔ, ΤZ, ΤΔ)

Z: set of states (basic search strategies)z0 ∈ Z : start state

M: memory spacem0 ∈ M : start memory state

Δ ⊆ Z×Z : transition relation (when to switch to another type of search)

σZ : set of state types; σΔ : set of transition types

ΤZ : Z ➝ σZ associate states with types

ΤΔ : Δ ➝ σΔ associate transitions with types

(0) Basic Hill climbing

determine initial solution swhile s not local optimum

choose s’ in N(s)such that f(s’)>f(s)

s = s’return s

M =(Z, z0, M, m0, Δ, σZ, σΔ, ΤZ, ΤΔ)

Z: {z0, z1}

M: {m0} //not used in this model

Δ ⊆ Z×Z :{ (zo, z1), (z1, z1)}

σZ : { random choice, select better neighbour }

σΔ : { Det}

ΤZ : ΤZ (z0) = random choice,

ΤZ(z1) = select better neighbour

ΤΔ : ΤΔ ((zo, z1)) = Det, ΤΔ((z1, z1)) = Det

M =(Z, z0, M, m0, Δ, σZ, σΔ, ΤZ, ΤΔ)

Z: {z0, z1}

M: {m0} //not used in this model

Δ ⊆ Z×Z :{ (zo, z1), (z1, z1)}

σZ : { random choice, select better neighbour }

σΔ : { Det}

ΤZ : ΤZ (z0) = random choice,

ΤZ(z1) = select better neighbour

ΤΔ : ΤΔ ((zo, z1)) = Det, ΤΔ((z1, z1)) = Det

z0z0 z1

z1

Det

Det

(1) Randomized Hill climbing

determine initial solution s; bestS = swhile termination condition not satisfied

with probability p choose neighbour s’ at random

else //climb if possiblechoose s’ with f(s’) > f(s)

s = s’; if (f(s) > f(bestS)) bestS = sreturn bestS

(1) Randomized Hill climbingM =(Z, z0, M, m0, Δ, σZ, σΔ, ΤZ, ΤΔ)

Z: {z0, z1, z2}

M: {m0}

Δ ⊆ Z×Z :{(zo, z1),(z1, z1),(zo, z2),(z1, z2),(z2, z1),(z2, z2)}

σZ : { random choice, select better neighbour,select any neighbour }

σΔ : { Prob(p), Prob(1-p)}

ΤZ : ΤZ (z0) = random choice,

ΤZ(z1) = select better neighbour

ΤZ(z2) = select any neighbour

ΤΔ : ΤΔ ((zo, z1)) = Prob(p), ΤΔ((z1, z1)) = Prob(p)

ΤΔ ((zo, z2)) = Prob(1-p), ΤΔ((z1, z2)) = Prob(1-p)

ΤΔ ((z2, z1)) = Prob(p), ΤΔ((z2, z2)) = Prob(1-p)

Prob(p)z0z0

z1z1Prob(p)

Prob(p)

z2z2

Prob(1-p)

Prob(1-p)

Prob(1-p)

(2) Variable Neighbourhooddetermine initial solution si = 1repeat

choose neighbour s’ in Ni(s) with max(f(s’))

if ((f(s’) > f(s))s = s’i = 1 // restart in first neighbourhood

elsei = i+1 // go to next neighbourhood

until i > iMaxreturn s

*example using memory to track neighbourhood definition

z0z0 z1

z1

Prob(1)i=1

NewBest(T) i = 1

NewBest(F) i++

Hoos and Stützle terminology

transitions: Det deterministic CDet(R), CDet(not R)

conditional deterministic on R Prob(p), Prob(1-p) probabilistic CProb(not R, p), CProb(not R, 1-p)

conditional probabilistic

Hoos and Stützle terminology

search subroutine(z states): RP random pick (usual start) RW random walk (any neighbour) BI best in neighbourhood

Some examples

RPRP BIBIDet

CDet(not R)

CDet(R)

1.

Cprob(not R, p)

RPRP

BIBICDet(R)

RWRW

Prob(p)

Prob(1-p)

Cprob(not R, p)

Cprob(not R, 1-p)

Cprob(not R, 1-p)

CDet(R)

2.

Simulated annealing metaphor:

slow cooling of liquid metalsto allow crystal structureto align properly

“temperature” T is slowly loweredto reduce random movementof solution s in solution space

Simulated Annealingdetermine initial solution s; bestS = sT = T0

while termination condition not satisfiedchoose s’ in N(s) probabilisticallyif (s’ is “acceptable”) // function of T and f(s’)

s = s’if (f(s) > f(sBest)) bestS = s

update Treturn bestS RPRP SA(T)SA(T)

Det:T = T0

Det:update(T)

Accepting a new solution

- acceptance more likely if f(s’) > f(s)- as execution proceeds,probability of acceptanceof s’ with f(s’) < f(s) decreases(becomes more like hillclimbing)

determine initial solution s; bestS = sT = T0

while termination condition not satisfiedchoose s’ in N(s) probabilisticallyif (s’ is “acceptable”) // function of T

s = s’if (f(s) > f(sBest)) bestS = s

update Treturn bestS

determine initial solution s; bestS = sT = T0

while termination condition not satisfiedchoose s’ in N(s) probabilisticallyif (s’ is “acceptable”) // function of T

s = s’if (f(s) > f(sBest)) bestS = s

update Treturn bestS

the acceptance function

€

p =1

1+ ef (s' )− f (s)

T

T e

volv

es

*sometimes p=1 when f(s’)-f(s)> 0 *sometimes p=1 when f(s’)-f(s)> 0

Simulated annealing with SAT

algorithm p.123 SA-SAT

propositions: P1,… Pn

expression: F = D1D2…Dk

recall cnf: clause Di is a disjunction of propositions and negative props

e.g., Px ~Py Pz ~Pw

fitness function: number of true clauses

Inner iteration – SA(T) in GLSMassign random truth set TFFTTFFTrepeat for i=1 to 4

flip truth of prop i FFFFTFFTevaluate FFTTFTFTdecide to keep (or not) FFFFTTTT

changed value FFTFFTFF

FFTTFFTT reduce T

RPRP SA(T)SA(T)

Det:T = T0

Det:update(T)

Tabu search (taboo)

always looks for best solutionbut some choices (neighbours) are

ineligible (tabu)ineligibility is based on recent moves:

once a neighbour edge is used, it cannot be removed (tabu) for a few iterations

search does not stop at local optimum

Symmetric TSP example

set of 9 cities {A,B,C,D,E,F,G,H,I}neighbour definition based on 2-opt*(27 neighbours)current sequence:

B - E - F - H - I - A - D - C - G - Bmove to 2-opt neighbour

B - D - A - I - H - F - E - C - G - Bedges B-D and E-C are now tabui.e., next 2-opt swap cannot involve these

edges

*example in book uses 2-swap, p 131

TSP example, algorithm p 133how long will an edge be tabu? 3 iterationshow to track and restore eligibility?

data structure to store tabu statusof 9*8/2 = 36 edges

B - D - A - I - H - F - E - C - G - B

recency-basedmemory

A B C D E F G H

I 0 0 0 0 0 0 0 2

H 0 0 0 0 0 1 0

G 0 1 0 0 0 0

F 0 0 0 0 0

E 0 0 3 0

D 2 3 0

C 0 0

B 0

procedure tabu search in TSPbegin

repeat until a condition satisfiedgenerate a tourrepeat until a (different) condition satisfied

identify a set T of 2-opt movesselect best admissible (not tabu) move from Tmake moveupdate tabu list and other varsif new tour is best-so-far

update best tour informationend

This algorithm repeatedly starts with a random tour of the cities.Starting from the random tour, the algorithm repeatedly moves to the best admissible neighbour; it does not stop at a hilltop but continues to move.

applying 2-opt with tabu

from the table, some edges are tabu:B - D - A - I - H - F - E - C - G - B 2-opt can only consider:

AI and FE AI and CG FE and CG

A B C D E F G H

I 0 0 0 0 0 0 0 2

H 0 0 0 0 0 1 0

G 0 1 0 0 0 0

F 0 0 0 0 0

E 0 0 3 0

D 2 3 0

C 0 0

B 0

importance of parameters

once algorithm is designed, it must be “tuned” to the problem selecting fitness function and

neighbourhood definition setting values for parameters

this is usually done experimentally

procedure tabu search in TSPbegin

repeat until a condition satisfiedgenerate a tourrepeat until a (different) condition satisfied

identify a set T of 2-opt movesselect best admissible (not tabu) move from Tmake moveupdate tabu list and other varsif new tour is best-so-far

update best tour informationend

Choices in ‘tuning’ the algorithm:what conditions to control repeated executions:

counted loops, fitness threshhold, stagnancy (no improvement)how to generate first tour (random, greedy, ‘informed’)how to define neighbourhoodhow long to keep edges on tabu listother vars: e.g., for stagnancy