1

Lagrangean Relaxation

--- Bounding through penalty adjustment

2

Outline

• Brief introduction

• How to perform Lagrangean relaxation

• Subgradient techniques

• Example: Set covering

• Decomposition techniques and Branch and Bound

3

Introduction

Lagrangian Relaxation is a technique which has

been known for many years:• Lagrange relaxation is invented by (surprise!) La

grange in 1797 !• This technique has been very useful in conjuncti

on with Branch and Bound methods.• Since 1970 this has been the bounding techniqu

e of choice ...... until the beginning of the 90’ies (branch-and-price)

4

Linear Program

5

How can we calculate lower bounds ?

We can use heuristics to generate upper bounds, but getting (good) lower bounds is often much harder !

The classical approach is to create a relaxation.

6

7

8

The classical branch-and-bound algorithm use the

LP relaxation. It has the nice feature of being

general, i.e. applicable to all MIP models.

9

Linear Program

Min:

s.t.

10

This is called the Lagrangean Lower Bound

Program (LLBP).

11

Note:

• For any λ0, the program LLBP provides a lower bound to the original problem.

• To get the tightest lower bound, we should solve

}1,0{

..

)(min

max0

x

dBxts

Axbcx

(This problem is called Lagrangean Dual program.)

• Ideally, the optimal value of Lagrangear Dual program is equal to the optimal value of the original problem. If not, a duality gap exists.

12

Lagrangian Relaxation

What can this be used for ?• Primary usage: Bounding ! Because it is a relaxa

tion, the optimal value will bound the optimal value of the real problem !

• Lagrangian heuristics, i.e. generate a “good” solution based on a solution to the relaxed problem.

• Problem reduction, i.e. reduce the original problem based on the solution to the relaxed problem.

13

Two Problems

Facing a problem we need to decide:

• Which constraints to relax (strategic choice)

• How to find the best lagrangean multipliers, (tactical choice)

14

Which constraints to relax

Which constraints to relax depends on two things:• Computational effort:

– Number of Lagrangian multipliers– Hardness of problem to solve

• Integrality of relaxed problem: If it is integral, we can only do as good as the straightforward LP relaxation !

(Integrality: the solution to relaxed problem is guaranteed to be integral.)

15

Multiplier adjustment

Two different types are given:

• Subgradient optimisation• Multiplier adjustment

Of these, subgradient optimisation is the method of choice. This is general method which nearly always works !

Here we will only consider this Method, although more efficient (but much more complicated) adjustment methods have been suggested.

16

Lagrangian Relaxation

17

Problem reformulation

Note: Each constraint is associated with a multiplier i.iijj ij bxa

18

The subgradient

(subgradient of multiplier i)

19

Subgradient Optimization

20

Example: Set covering

21

Relaxed Set coverint

How can we solve this problem to optimality ???

22

Optimization Algorithm

The answer is so simple that we are reluctant calling it an optimization algorithm: Choose all x’s with negative coefficients !

What does this tell us about the strength of the relaxation ?

23

Rewritten: Relaxed Set covering

where

24

Lower bound

25

Comments

• This is actually quite interesting: The algorithm is very simple, but a good lower bound is found quickly !

• This relied a lot on the very simple LLBP optimization algorithm.

• Usually the LLBP requires much more work ....

• The subgradient algorithm very often works ...

26

So what is the use ?

This is all very nice, but how can we solve our problem ?

• We may be lucky that the lower bound is also a feasible and optimal solution.

• We may reduce the problem, performing Lagrangian problem reduction.

• We may generate heuristic solutions based on the LLBP.

• We may use LLBP in lower bound calculations for a Branch and Bound algorithm.

27

Homework

Starting with ZUB=6,=2 and i=0 for i=1,2,3, perform 3 iterations of the subgradient optimization method on the previous example problem. Please show the lower bound and multiplier values of each iteration.