Heuristic Optimization

Thomas Stutzle

IRIDIA, CoDEUniversite Libre de Bruxelles

stuetzle@ulb.ac.beiridia.ulb.ac.be/~stuetzle

iridia.ulb.ac.be/~stuetzle/Teaching/HO

Example problems

I imagine a very good friend from Germany visits you and hewants to visit all 146(?) breweries in Belgium during his oneweek stay

Is this feasible? If yes, which route to take?

The shortest certainly helps

I at brewery No. 49 your friend o↵ers to pay all beers you takeon the trip if you solve the following riddle

‘Last week my friends Anne, Carl, Eva, Gustaf and I went out for dinnerevery night, Monday through Friday. I missed the meal on Friday becauseI was visiting my sister and her family. But otherwise, every one of us hadselected a restaurant for a particular night and served as a host for thatdinner. Overall, the following restaurants were selected: a French bistro,a sushi bar, a pizzeria, a Greek restaurant, and the Brauhaus. Eva tookus out on Wednesday. The Friday dinner was at the Brauhaus. Carl, whodoesn’t eat sushi, was the first host. Gustaf had selected the bistro for

the night before one of the friends took everyone to the pizzeria. Tell me,who selected which restaurant for which night?

Heuristic Optimization, 2018 2

How to solve it?

I many possible approaches

I systematic enumeration is probably not realistic

I some people may eliminate certain assignments or partialtours through careful reasoning

I other intuitive approach: start with some good guess and thentry to improve it iteratively

The latter is an example of a heuristic approach to optimization

Heuristic Optimization, 2018 3

Optimization

Optimization refers to choosing the best element from some set ofavailable alternatives.

Optimization problems . . .

I arise in a wide variety of applications

I arise in many di↵erent forms, e.g., continuous, combinatorial,multi-objective, stochastic, etc.

I here we focus mainly on combinatorial problems

I range from quite easy to hard ones

I here we focus on the hard ones!

Heuristic Optimization, 2018 4

.. an easy one

find the best (most valuable) element from the set of alternatives

Heuristic Optimization, 2018 5

.. a more di�cult (but still “easy”) one

find best (shortest) route from A to B in an edge-weighted graph

Heuristic Optimization, 2018 6

.. a harder one

find best (shortest) round trip through some cities, aka TravelingSalesman Problem (TSP)

Heuristic Optimization, 2018 7

find best (shortest) round trip through some cities, aka TravelingSalesman Problem (TSP)

(see also http://www.math.uwaterloo.ca/tsp/maps/)

Heuristic Optimization, 2018 8

Practical applications of the TSP

Heuristic Optimization, 2018 9

.. and a large instance

Heuristic Optimization, 2018 10

A more real-life like problem

TSP arises as sub-problem, e.g., in vehicle routing problems(VRPs)

Heuristic Optimization, 2018 11

I realistic problems can involve many complicating details

I examples in VRP case are

I time windows, access restrictions, priorities, split delivery, . . .

I capacity restrictions, di↵erent costs of vehicles, . . .

I working time restrictions, breaks, . . .

I stochastic travel times or demands, incoming new requests, . . .

I in lecture: focus on simplified models of (real-life) problems

I useful for illustrating algorithmic principles

I they are “hard” and capture essence of morecomplex problems

I are treated in research to yield more general insights

Heuristic Optimization, 2018 12

Optimization problems arise everywhere!

Most such problems are computationally very hard (NP-hard!)Heuristic Optimization, 2018 13

Solving (combinatorial) optimization problems

I systematic enumeration

I problem specific, dedicated algorithms

I generic methods for exact optimization

I heuristic methods

Heuristic Optimization, 2018 14

Heuristic methods

Heuristic methods intend to compute e�ciently, good solutions toa problem with no guarantee of optimality

I range from rather simple to quite sophisticated approaches

I inspiration often fromI human problem solving

I rules of thumb, common sense rulesI design of techniques based on problem-solving experience

I natural processesI evolution, swarm behaviors, annealing, . . .

I usually used when there is no other method to solve theproblem under given time or space constraints

I often simpler to implement / develop than other methods

Heuristic Optimization, 2018 15

Goals of this course

Provide answers to these questions:

I Which heuristic methods are availableand what are their features?

I How can heuristic methods be used to solvecomputationally hard problems?

I How should heuristic methods be studied andanalysed empirically?

I How can heuristic algorithms be designed,developed, and implemented?

Heuristic Optimization, 2018 16

Contents

Basics:

I introduction, SLS framework

I iterative improvement algorithms

I simple SLS methods

I hybrid and population-based SLS methods

I empirical analysis of SLS algorithms

I search space analysis

Additional topics:

I tuning, algorithm configuration

I complex problem features

Heuristic Optimization, 2018 17

Heuristic Optimization field

SLS

Appli-cations

ComputerScience

OperationsResearch

Statis-tics

HO

Heuristic Optimization, 2018 18

Organizational matters

I webpages

iridia.ulb.ac.be/~stuetzle/Teaching/HO

http://www.sls-book.net/

I lectures and exercisesI Wednesday, 08:10 to 09:40 and 10:00 to 11:30 in IRIDIA’s

seminar room (C.5.130)

I lecture dates (preliminary schedule; check for updates)I February 28 (two)I March 7 (one), 14 (one), 21 (one), 28 (one)I April 18 (one), 25 (two)I May 2 (two), 9 (one)

Heuristic Optimization, 2018 19

I exercises and implementation tasks

I five exercise sessionsI exercise dates (preliminary schedule; check for updates)

I Mar 7, 21, 28, April 25, May 9

I two implementation exercises (second builds on first one)

I First: March 14 with short introductory lectureI Second: April 18

Heuristic Optimization, 2018 20

I evaluation

I precondition for passing course: successful completion of bothimplementation tasks (� 10 for each; if necessary corrections)

I oral exam at the end of semester (counts 60%)

I implementation exercises (counts 40%)

I final mark: weighted average of implementation exercises andoral exam (0.4⇥mark

impl

+ 0.6⇥markoral

)

I course material, literature

I slides

I H. H. Hoos and T. Stutzle. Stochastic Local Search:Foundations and Applications. Morgan Kaufmann Publishers,2005.

I additional literature will be given during the course

Heuristic Optimization, 2018 21