1

Branch-and-Price(Column Generation)

Solving Integer ProgramsWith a

Huge Number of Variables

CP-AI-OR’02School on Optimization

Le Croisic, FranceMarch 2002

2

Reasons for Considering Formulations with a Huge Number of Variables

• A compact formulation of a MIP may have a weak LP relaxation. Frequently the relaxation can be tightened by a reformulation that involves a huge number of variables.

• A compact formulation of a MIP may have symmetric structure that causes branch-and-bound to perform poorly because the problem barely changes after branching. A reformulation with a huge number of variables can eliminate this symmetry.

• Column generation provides a decomposition of the problem into master and sub problems. This decomposition may have a natural interpretation in the contextual setting allowing for the incorporation of additional important constraints.

• A formulation with a huge number of variables may be the only choice.

3

Classic Example

The Cutting Stock or Bin Packing Problem

Input:1. A set of n items of sizes a1, a2, … an,2. A set of demands di for item i, i=1,…,n3. Bins of size L

Output: A packing of the items into a minimum number of bins

4

Example

i 1 2 3 4

45 36 31 14

97 610 395 211

ia

id

L = 100

Solution with FFD

45 45

45 36 14

36 36 14 14

36 36

36 31 31

31 31 31

x 48

x 1

x 1

x 131

x 105

x 199

= 485There are 37 patterns

n

aiidaL 1

1Lower bound rounded up

= 416 in example

5

Compact Formulation

and integer

K is an upper bound on the number of bins

The LP relaxation of this formulation is very bad since it only gives the trivial lower bound mentioned above.

Solution won’t change after branching because of symmetries.

We need a better formulation for which the LP gives a good bound and thesymmetries are eliminated.

0

1,0

,...,1,

,...,1,

min1

ik

k

kiki

i

ik

ik

K

kk

x

y

KkLyxa

nidx

y

6

Formulation with an Exponential Number of Columns

tp is the tth pattern

itp is the number of item i in tth pattern

In example 45 45 tp

0

0

0

2

ty number of times tth pattern is used

and integer

The difficulty is that T can be very large

0

min

1

1

t

t

T

tt

T

tt

y

dyp

y

7

LP bound for example is 452.25

5.197

0

2

1

0

75.100

0

0

2

0

5.105

2

0

2

0

5.48

0

0

0

2

roundup gives 454

round down gives 450 and then 3 more for residual

1

1

3

1

this is optimal because of the LP bound

8

LP Column Generation

1. We solve the LP relaxation over a small subset of columns.

2. We then need to find a column with negative reduced cost or prove that none exists implying the current solution is optional.

tc reduced cost of column t it

n

ii p

1

1

where is an optimal dual solution

0

max

max1min

1

1

1

it

it

n

ii

it

n

ii

it

n

ii

tt

t

p

Lpa

p

pc

integer

This is a knapsack problem that can be solved by dynamic programming or branch-and-bound.

9

LP Column Generation

Solve Master LP

Solve Sub problem

dual solution

optimal

CP may be useful here

new columns

After solving the LP relaxation why can’t we apply standard branch-and-boundTo solve the IP.

Difficulty: ty fractional

Branch:

It is difficult to enforce this constraint on the knapsack problem.

tf

tt fy

10

Routing Problems

• Several Routes are needed

• Individual Routes are Constrained

• Master Problems: Puts together feasible routes at minimum cost

• Sub Problem: Generates routes based on minimum cost and dual variables

• Vehicle routing

• Crew scheduling

• Traveling tournament problem

11

Airline Crew Scheduling

• Input: A schedule of a fleet

• Objective: Find a set of crew itineraries (pairings) that partition all of the legs at minimum cost.

ATL

MIA

JAX

Monday (duty 1) Tuesday (duty 2)8:00 17:00 10:00 13:00

12

Pairings

• Restrictions

– Connection times

– Duty elapsed time

– Flying time in a duty

– 8-in-24 rule

• Duty cost is the maximum of 3 quantities.

– Flying time, fraction of elapsed time, minimum guarantee

• Pairing cost is the maximum of 3 quantities.

– Sum of duty costs, elapsed time, minimum guarantee

• Daily problems

– 100 flights: 90,000 pairings

– 250 flights: 2.5 million pairings

– 1000 flights: more than a billion pairings

13

Airline Crew Scheduling

• A set partitioning problem: rows correspond to flights and columns to pairings

– aij=1 if flight i is in pairing j.

– xj=1 if pairing j is selected.

– n is very large, in billions.

n,,1j1,0x

m,,1i1xa

xcmin

j

n

1jjij

j

n

1jj

14

Delayed Column Generation

1. We solve the set partitioning problem over a subset of pairings.

2. The dual variables give reduced costs for the pairings.

3. We solve a constrained shortest path sub problem to generate new pairings or to show that the current LP solution is optimal.

4. CP may be a good way to solve the sub problem because of the large number of constraints.

15

Delayed Column Generation

16

Standard Branching

• A pairing has a fractional value in an optimal LP solution

1 ,* fofy j

• Two cases 1, fof

1f is easy, fix the pairing and knock out the flights it covers

of is hard, the pairing must be eliminated from the shortest path problem

17

Follow-on Branching

Given a fractional solution to the LP there are fractional pairings r and ssuch that ry and sy are fractional.

In r, flight k follows flight iIn s, flight k appears but flight i doesn’t

1 1

0 1ik

• branch 1: flight k follows flight i in some pairing. Eliminate all pairings that contain i or k or both, but not consecutively.[in the network delete all connections from i except the one to k] .oys

• branch 2: k cannot follow i. Eliminate all pairings in which k immediately follows i.[in the network, delete the connector arc (i,k)] .oyr

18

0-1 Cutting Stock Branching

A pattern contains are most one unit of an item, nidi ,...,1,1

Branching is the same as crew pairing.

A pattern with items r and s has a fractional value in an optimal LP solution.

• branch 1: define a new item rs of size sr aa (they must go together)

• branch 2: add at most one of item r and s to the knapsack problem

19

General Cutting Stock Branching

A pattern may contain several units of different items

0

2

1

0

0

0

2

0

2

0

2

0

0

0

0

2

48.5 105.5 100.75 197.5

• branch 1: patterns containing 2 or more units of item 2 207

• branch 2: 206

These are simple constraints to add to the master.

20

Column Management

• We can add multiple problems from the sub problem to the master.

• It can be a good idea to sets of columns that yield feasible integer solutions to the master.

• Columns no longer active in the LP solution can be deleted from the master.

21

Branch-and-Price Strategic Issues

1. Column generation at a node can be terminated before proving LP optimality.

2. If the LP value is above the upper bound, then we should continue column generation with the hope of fathoming the node.

3. Strong branching may be helpful in difficult problems.

4. We may only want to follow the branches in which we are fixing the solution and stop after a good feasible solution is obtained.