Algebraic and Geometric ideas in the theory of Discrete Optimization

Algebraic and Geometric ideas in the theory of DiscreteOptimization

Jesus A. De Loera, UC Davis

Three Lectures based on the book:

Algebraic & Geometric Ideas in the Theory of Discrete Optimization(SIAM-MOS 2013)

By J. De Loera, R. Hemmecke & M. Koppe

July 15, 2013

() July 15, 2013 1 / 25

Alg

ebr

Aic A

nd g

eom

etric id

eAs

in the t

heo

ry o

f disc

rete o

ptimizA

tion

MO14

Jesús A

. De Lo

eraR

aymo

nd

Hem

mecke

Matth

ias Kö

pp

e

This book presents recent advances in the mathematical theory of discrete optimization, particularly those supported by methods from algebraic geometry, commutative algebra, convex and discrete geometry, generating functions, and other tools normally considered outside the standard curriculum in optimization.

Algebraic and Geometric Ideas in the Theory of Discrete Optimization • offersseveralresearchtechnologiesnotyetwellknownamongpractitioners

of discrete optimization,

• minimizesprerequisitesforlearningthesemethods,and

• providesatransitionfromlineardiscreteoptimizationtononlineardiscrete optimization.

This book can be used as a textbook for advanced undergraduates or beginning graduate students in mathematics, computer science, or operations research or as a tutorial for mathematicians, engineers, and scientists engaged in computation whowishtodelvemoredeeplyintohowandwhyalgorithmsdoordonotwork.

Jesús A. De Loera is a professor of mathematics and a member of the Graduate Groups in Computer Science and Applied Mathematics at University of California, Davis.HisresearchhasbeenrecognizedbyanAlexandervonHumboldtFellowship,theUCDavisChancellorFellowaward,andthe2010INFORMSComputingSocietyPrize. He is an associate editor of SIAM Journal on Discrete Mathematics and Discrete Optimization.

Raymond Hemmecke is a professor of combinatorial optimization at Technische Universität München. His research interests include algebraic statistics, computer

algebra, and bioinformatics.

Matthias Köppe is a professor of mathematics and a member of the Graduate Groups in Computer Science and Applied Mathematics at University of California, Davis. He is an associate editor of Mathematical Programming, Series A and Asia-Pacific Journal of Operational Research.

MO14

SocietyforIndustrial and Applied Mathematics

3600MarketStreet,6thFloorPhiladelphia,PA19104-2688USA

+1-215-382-9800•[email protected]•www.siam.org

MathematicalOptimizationSociety3600MarketStreet,6thFloor

Philadelphia,PA19104-2688USA+1-215-382-9800x319 Fax+1-215-386-7999

[email protected]•www.mathopt.org

ISBN 978-1-611972-43-6

MOS-SIAM Series on Optimization

AlgebrAic And geometric ideAs in the theory

of discrete optimizAtion

Jesús A. De LoeraRaymond HemmeckeMatthias Köppe

J.A.DeLoera,R.Hemmecke,M.Köppe

MO14_DeLoera-Koeppecover09-24-12.indd 1 10/12/2012 10:35:59 AM

() July 15, 2013 2 / 25

() July 15, 2013 3 / 25

Menu for Lectures

For Lecture ONE: Motivation and Main Statements

Non-Linear Polynomials and Discrete Optimization.

Some Theorems on Non-Linear Discrete Optimization

For Lecture TWO: A taste of the Math

Generating Function Methods

Graver Bases Methods

For Lecture THREE: Using Polynomials when one does not expect them!

Hilbert Nullstellensatz and Colorability problems.

Central Path of Interior Point Methods.

() July 15, 2013 4 / 25

Menu for Lectures










() July 15, 2013 4 / 25

Menu for Lectures










() July 15, 2013 4 / 25

Menu for Lectures










() July 15, 2013 4 / 25

Menu for Lectures










() July 15, 2013 4 / 25

Menu for Lectures










() July 15, 2013 4 / 25

Polynomials and

Discrete OptimizationHistorical Transitioning from linearto non-linear constraints in models

() July 15, 2013 5 / 25

Now we have INTEGER or BINARY variables

max f (x)

subject to gi (x) ≤ 0, i = 1, 2, . . . , k ,

hj(x) = 0, j = 1, 2, . . . ,m,

x ∈ Rn1 × Zn2 .

(1)

Here the objective function f and the constraint functions gi , hj are assumed to bearbitrary real-valued functions.

KEY POINT: The study of these problems requires more ideas from Algebra,Geometry and Topology.

LET US GO BACK IN TIME....

() July 15, 2013 6 / 25

A Classical Example from the beginning of DiscreteOptimization

Initial work by Kantorovich (1939), T.C Koopmans (1941), von Neumann(1947).The Transportation problem: A company builds laptops in four factories,each with certain supply power. Four cities have laptop demands. There is acost ci,j for transporting a laptop from factory i to city j . What is the bestassignment of transport in order to minimize the cost?

ON FOUR CITIES

DEMANDS

220

215

93

64

108

286

71

127

SUPPLIES

BY FACTORIES

A silly way to solve this: run through all possibilities! Well how do I do this??Not so easy... If supply and demand are all ONE and if number of cities andfactories is n = 35, and a computer took 10−9 seconds to check onepossibility, it would take 200,000 years to solve!

() July 15, 2013 7 / 25



ON FOUR CITIES

DEMANDS

220

215

93

64

108

286

71

127

SUPPLIES

BY FACTORIES


() July 15, 2013 7 / 25



ON FOUR CITIES

DEMANDS

220

215

93

64

108

286

71

127

SUPPLIES

BY FACTORIES


() July 15, 2013 7 / 25

Modeling with LINEAR equations and inequalities

Let xi,j be a variable indicating number of laptops factory i provides to city j .xi,j can only take non-negative integer values, xi,j ≥ 0.

Then Since factory i produces ai laptops we have

n∑j=1

xi,j = ai , for all i = 1, . . . , n.

and since city j needs bj laptops

n∑i=1

xi,j = bj , for all j = 1, . . . , n.

Now we minimize∑

ci,jxi,j .

() July 15, 2013 8 / 25




n∑j=1

xi,j = ai , for all i = 1, . . . , n.


n∑i=1

xi,j = bj , for all j = 1, . . . , n.

Now we minimize∑

ci,jxi,j .

() July 15, 2013 8 / 25




n∑j=1

xi,j = ai , for all i = 1, . . . , n.


n∑i=1

xi,j = bj , for all j = 1, . . . , n.

Now we minimize∑

ci,jxi,j .

() July 15, 2013 8 / 25




n∑j=1

xi,j = ai , for all i = 1, . . . , n.


n∑i=1

xi,j = bj , for all j = 1, . . . , n.

Now we minimize∑

ci,jxi,j .

() July 15, 2013 8 / 25

Overview LINEAR Discrete Optimization (circa 1990)

Efficient computation with Convex Sets & Lattices ⇐⇒ Efficient Optimization

() July 15, 2013 9 / 25

At the beginning there was...

Linear programs

max c>x

s.t. Ax ≤ b

max c>

Easy(polynomial-time

solvable)

Special integer programs

max c>x

s.t. Ax ≤ b

all xi integer

Matrix A is SPECIAL!

Medium(can be easy or hard)

Network problemsFixed dimension

knapsacks0-1 matrices

Integer programs

max c>x

s.t. Ax ≤ b

all xi integer

max c>

Hard(NP-hard)

() July 15, 2013 10 / 25


Linear programs

max c>x

s.t. Ax ≤ b

max c>


solvable)


max c>x

s.t. Ax ≤ b

all xi integer





Integer programs

max c>x

s.t. Ax ≤ b

all xi integer

max c>

Hard(NP-hard)

() July 15, 2013 10 / 25


Linear programs

max c>x

s.t. Ax ≤ b

max c>


solvable)


max c>x

s.t. Ax ≤ b

all xi integer





Integer programs

max c>x

s.t. Ax ≤ b

all xi integer

max c>

Hard(NP-hard)

() July 15, 2013 10 / 25


Linear programs

max c>x

s.t. Ax ≤ b

max c>


solvable)


max c>x

s.t. Ax ≤ b

all xi integer





Integer programs

max c>x

s.t. Ax ≤ b

all xi integer

max c>

Hard(NP-hard)

() July 15, 2013 10 / 25

Integer Linear Programming: The state of the art

Traditional Algorithms

Dual (polyhedral) techniques

max c>

x2

x1

x0max c>

x2

x0

x1

Cutting plane algorithms– based on polyhedral theory

Enumeration

max c> x0max c> x0max c> x0

Branch-and-bound

Adhoc methods

special structure(e.g. network,matroids, etc.)

Mathematical modelling – Strong initial IP formulation

() July 15, 2013 11 / 25




max c>

x2

x1

x0max c>

x2

x0

x1


Enumeration


Branch-and-bound

Adhoc methods



() July 15, 2013 11 / 25




max c>

x2

x1

x0

max c>

x2

x0

x1


Enumeration


Branch-and-bound

Adhoc methods



() July 15, 2013 11 / 25




max c>

x2

x1

x0

max c>

x2

x0

x1


Enumeration


Branch-and-bound

Adhoc methods



() July 15, 2013 11 / 25




max c>

x2

x1

x0

max c>

x2

x0

x1


Enumeration

max c> x0

max c> x0max c> x0

Branch-and-bound

Adhoc methods



() July 15, 2013 11 / 25




max c>

x2

x1

x0

max c>

x2

x0

x1


Enumeration

max c> x0

max c> x0

max c> x0

Branch-and-bound

Adhoc methods



() July 15, 2013 11 / 25




max c>

x2

x1

x0

max c>

x2

x0

x1


Enumeration

max c> x0max c> x0

max c> x0

Branch-and-bound

Adhoc methods



() July 15, 2013 11 / 25




max c>

x2

x1

x0

max c>

x2

x0

x1


Enumeration

max c> x0max c> x0

max c> x0

Branch-and-bound

Adhoc methods



() July 15, 2013 11 / 25




max c>

x2

x1

x0

max c>

x2

x0

x1


Enumeration

max c> x0max c> x0

max c> x0

Branch-and-bound

Adhoc methods



() July 15, 2013 11 / 25

MANY CHALLENGES!!

LIFE IS NON-LINEAR!!

() July 15, 2013 12 / 25

Example: Non-linear transportation polytopes

1 In the traditional transportation problem cost at an edge is a constant. Thuswe optimize a linear function.

2 but due to congestion or heavy traffic or heavy communication load thetransportation cost on an edge could be a non-linear function of the flow ateach edge.

3 For example cost at each edge is fij(xij) = cij |xij |aij for suitable constant aij .This results on a non-linear function

∑fij which is much harder to minimize.

() July 15, 2013 13 / 25






() July 15, 2013 13 / 25






() July 15, 2013 13 / 25

Reality is NON-LINEAR and worse!!

Non-linear Discrete Optimization

max/min f (x1, . . . , xd)

subject to gj(x1, . . . , xd) ≤ 0,

for j = 1 . . . s, and with

with xi integer

with f , gj Non-Linear

WHAT CAN BE DONE IN THISGENERAL CONTEXT??

Prove good theorems? Are thereefficient algorithms?

BAD NEWS: The problem isINCREDIBLY HARDTheorem It is UNDECIDABLEalready when f ,gi ’s arearbitrary polynomials (Jeroslow,1979).

EVEN WORSE Theorem: Itundecidable even with number ofvariables=10. (Matiyasevich andDavis 1982).

No theorem or algorithm performance canbe proved without ASSUMPTIONS

Let us see two nice theorems that wereproved in the last years

() July 15, 2013 14 / 25



max/min f (x1, . . . , xd)



with xi integer








() July 15, 2013 14 / 25



max/min f (x1, . . . , xd)



with xi integer








() July 15, 2013 14 / 25



max/min f (x1, . . . , xd)



with xi integer








() July 15, 2013 14 / 25



max/min f (x1, . . . , xd)



with xi integer








() July 15, 2013 14 / 25

How about polyhedral constraints non-linear objective??

Let f be a multivariate polynomial function,

max f(x)

s.t. Ax ≤ b

Hard(NP-hard)

Special programs

max f(x)

s.t. Ax ≤ b

all xi integer


???We study TWO

special cases

max f(x)

s.t. Ax ≤ b

all xi integer

Hard(NP-hard)

() July 15, 2013 15 / 25



max f(x)

s.t. Ax ≤ b

Hard(NP-hard)

Special programs

max f(x)

s.t. Ax ≤ b

all xi integer


???We study TWO

special cases

max f(x)

s.t. Ax ≤ b

all xi integer

Hard(NP-hard)

() July 15, 2013 15 / 25



max f(x)

s.t. Ax ≤ b

Hard(NP-hard)

Special programs

max f(x)

s.t. Ax ≤ b

all xi integer


???We study TWO

special cases

max f(x)

s.t. Ax ≤ b

all xi integer

Hard(NP-hard)

() July 15, 2013 15 / 25



max f(x)

s.t. Ax ≤ b

Hard(NP-hard)

Special programs

max f(x)

s.t. Ax ≤ b

all xi integer


???

We study TWOspecial cases

max f(x)

s.t. Ax ≤ b

all xi integer

Hard(NP-hard)

() July 15, 2013 15 / 25



max f(x)

s.t. Ax ≤ b

Hard(NP-hard)

Special programs

max f(x)

s.t. Ax ≤ b

all xi integer


???We study TWO

special cases

max f(x)

s.t. Ax ≤ b

all xi integer

Hard(NP-hard)

() July 15, 2013 15 / 25

MAIN DISH

TWO EXAMPLES OF

ALGEBRAIC-GEOMETRIC

IDEAS FOR

DISCRETE OPTIMIZATION

() July 15, 2013 16 / 25

Problem type

max f (x1, . . . , xd)

subject to (x1, . . . , xd) ∈ P ∩ Zd ,

where

P is a polytope (boundedpolyhedron) given by linearconstraints,

f is a (multivariate)polynomial functionnon-negative over P ∩ Zd ,

the dimension d is fixed.

Prior Work

Integer Linear Programming can besolved in polynomial time

(H. W. Lenstra Jr, 1983)

Convex polynomials f can beminimized in polynomial time

(Khachiyan and Porkolab, 2000)

WHAT CAN BE PROVED IN THISCASE??

Lemma Optimizing an arbitrary degree-4polynomial f over the lattice points of apolygon is NP-hard

NP-complete to decide whether, giventhree positive integers a, b, c , there existsa positive integer x < c such that x2 iscongruent with a modulo b.

() July 15, 2013 17 / 25

Problem type

max f (x1, . . . , xd)


where




Prior Work








() July 15, 2013 17 / 25

Problem type

max f (x1, . . . , xd)


where




Prior Work








() July 15, 2013 17 / 25

Problem type

max f (x1, . . . , xd)


where




Prior Work








() July 15, 2013 17 / 25

Problem type

max f (x1, . . . , xd)


where




Prior Work








() July 15, 2013 17 / 25

Problem type

max f (x1, . . . , xd)


where




Prior Work








() July 15, 2013 17 / 25

Theorem (FPTAS for Integer Polynomial Maximization)JDL, Hemmecke, Koppe, Weismantel, 2006

Let the dimension d be fixed. There exists an algorithm whose input data area polytope P ⊂ Rd , given by rational linear inequalities, anda polynomial f ∈ Z[x1, . . . , xd ] with integer coefficients and maximum totaldegree D that is non-negative on P ∩ Zd

with the following properties.1 For a given k , it computes in running time polynomial in k, the encoding size

of P and f , and D lower and upper bounds Lk ≤ f (xmax) ≤ Uk satisfying

Uk − Lk ≤(

k

√|P ∩ Zd | − 1

)· f (xmax).

2 For k = (1 + 1/ε) log(|P ∩ Zd |), the bounds satisfy

Uk − Lk ≤ ε f (xmax),

and they can be computed in time polynomial in the input size, the totaldegree D, and 1/ε.

3 By iterated bisection of P ∩ Zd , it constructs a feasible solution xε ∈ P ∩ Zd

with ∣∣f (xε)− f (xmax)∣∣ ≤ εf (xmax).

() July 15, 2013 18 / 25





Uk − Lk ≤(

k

√|P ∩ Zd | − 1

)· f (xmax).






() July 15, 2013 18 / 25





Uk − Lk ≤(

k

√|P ∩ Zd | − 1

)· f (xmax).






() July 15, 2013 18 / 25





Uk − Lk ≤(

k

√|P ∩ Zd | − 1

)· f (xmax).






() July 15, 2013 18 / 25

Example : Multiobjective Transportation Problems

1 In the traditional transportation problem one cost per edge. Thus weoptimize ONE linear function.

2 but the cost of an edge for the company may not be the same as for anenvironmentalist or the government!! We face many cost functions at thesame time.

3 So we get three or more costs per edge and we are looking to find pointswhere three or more linear functionals are “minimized”.

4 The three objective functions induce a partial order over the lattice points inthe feasible region.

5 The multiobjective optimization approach is to find the minimal elements ofa partially ordered set, the Pareto Optima.

() July 15, 2013 19 / 25







() July 15, 2013 19 / 25







() July 15, 2013 19 / 25







() July 15, 2013 19 / 25







() July 15, 2013 19 / 25

Main resultsJDL, Hemmecke, Koppe, 2010

Theorem (Counting and enumeration theorem)

Let the dimension n and the number k of objective functions be fixed.Using the input data A ∈ Zm×n, an m-vector b, and linear functionsf1, . . . , fk ∈ Zn,

(i) there exists a polynomial-time algorithm to exactly count the Pareto optimaand the Pareto strategies;

(ii) there exists a polynomial-space polynomial-delay prescribed-orderenumeration algorithm to generate the full sequence of Pareto optimaordered lexicographically.

(iii) There exists a polynomial-time algorithm to find a Pareto optimum v thatminimizes the distance ‖v − v‖ from a prescribed point v ∈ Zk for anarbitrary polyhedral norm.

(Again in the spirit of Lenstra’s 1983 polynomial-time algorithm for ILP in fixeddimension.)

() July 15, 2013 20 / 25

Theorem: Convex Integer Optimization on TransportationPolytopesJDL, Hemmecke, Onn, Rothblum, Weismantel, 2009

Problem: Convex function c : Rd −→ R, find a nonnegative integer vectorx ∈ Nn maximizing

max {c(w1x , . . . ,wdx) : Ax = b, x ∈ Nn} .

INTERPRETATION: Given d linear objective functions w1, . . . ,wd , want tomaximize their “convex balancing” c(w1x , . . . ,wdx) over feasible lattice integerpoints.

Theorem (convex balancing on transportation problems) For any fixed d , p thereis a polynomial oracle-time algorithm that, given n, arrays w1, . . . ,wd ∈ Zp×n, andconvex c : Rd −→ R given by comparison oracle, solves the convex integertransportation problem with p many suppliers.

max{ c(w1x , . . . ,wdx) : x ∈ Np×n ,

p∑i

xi,j = zj ,

n∑j

xi,j = vi}

() July 15, 2013 21 / 25






max{ c(w1x , . . . ,wdx) : x ∈ Np×n ,

p∑i

xi,j = zj ,

n∑j

xi,j = vi}

() July 15, 2013 21 / 25






max{ c(w1x , . . . ,wdx) : x ∈ Np×n ,

p∑i

xi,j = zj ,

n∑j

xi,j = vi}

() July 15, 2013 21 / 25

Integer Optimization over N-fold Systems

Fix any pair of integer matrices A and B with the same number of columns, ofdimensions r × q and s × q, respectively. The n-fold matrix of the ordered pairA,B is the following (s + nr)× nq matrix,

[A,B](n) := (1n ⊗ B)⊕ (In ⊗ A) =

B B B · · · BA 0 0 · · · 00 A 0 · · · 0...

.... . .

......

0 0 0 · · · A

.

N-fold systems DO appear in applications! Transportation problems with fixednumber of suppliers are examples!Example: Consider the matrices A = [1 1] and B = I2.

[A,B](4) =

1 0 1 0 1 0 1 00 1 0 1 0 1 0 11 1 0 0 0 0 0 00 0 1 1 0 0 0 00 0 0 0 1 1 0 00 0 0 0 0 0 1 1

.

() July 15, 2013 22 / 25



[A,B](n) := (1n ⊗ B)⊕ (In ⊗ A) =

B B B · · · BA 0 0 · · · 00 A 0 · · · 0...

.... . .

......

0 0 0 · · · A

.


[A,B](4) =

1 0 1 0 1 0 1 00 1 0 1 0 1 0 11 1 0 0 0 0 0 00 0 1 1 0 0 0 00 0 0 0 1 1 0 00 0 0 0 0 0 1 1

.

() July 15, 2013 22 / 25



[A,B](n) := (1n ⊗ B)⊕ (In ⊗ A) =

B B B · · · BA 0 0 · · · 00 A 0 · · · 0...

.... . .

......

0 0 0 · · · A

.


[A,B](4) =

1 0 1 0 1 0 1 00 1 0 1 0 1 0 11 1 0 0 0 0 0 00 0 1 1 0 0 0 00 0 0 0 1 1 0 00 0 0 0 0 0 1 1

.

() July 15, 2013 22 / 25

Theorem

Fix two integer matrices A,B of sizes r × q and s × q, respectively. Then

there is a polynomial time algorithm that, given any n, an integer vectors b, acost vector c , solves the corresponding n-fold integer programming problem.

max{cx : [A,B](n)x = b, x ∈ Nnq} .

Similarly, Given a constant number of cost vectors c1, . . . , ck , a convexfunction f , then there is a polynomial time algorithm that given any n, aninteger vectors b, a cost vector c ,

max{f (c1x , c2x , . . . , ckx) : [A,B](n)x = b, x ∈ Nnq} .

Recent Advances: Extensions to convex minimization, stochastic integeroptimization, by Hemmecke, J. Lee, S. Onn, M. Koppe,

Note: Compare to polynomial-time of network flow integer programs (E.Tardos )!!

() July 15, 2013 23 / 25

Theorem








() July 15, 2013 23 / 25

Theorem








() July 15, 2013 23 / 25

WHAT IS THEMATH INSIDE?

Two clever algebraic encodingsof the lattice points in Polyhedra!!

Stay tuned for the second lecture!

() July 15, 2013 24 / 25




() July 15, 2013 24 / 25




() July 15, 2013 24 / 25

Thank you

Danke

Merci

Gracias

() July 15, 2013 25 / 25

Documents

Algebraic and Geometric ideas in the theory of Discrete Optimization