Convex Relaxations of Non-Convex Mixed Integer Quadratically Constrained Problems

Convex Relaxations of Non-Convex Mixed Integer Quadratically

Constrained Problems

Dr. Anureet SaxenaAssociate, Research

Axioma Inc.

(Joint Work with Pierre Bonami and Jon Lee)

Dedicated to Prof. Egon Balas

2Anureet Saxena, Axioma Inc.

MIQCP

min aT0xst

xTA ix + aTi x + bi · 0; i = 1 : : :mxj 2 Z; j 2 N I

l · x · u

Integer Constrained VariablesSymmetric Matrices

NOT necessarily positive semidefinite


MIQCP

min aT0xst

A i:Y + aTi x + bi · 0; i = 1 : : :mxj 2 Z; j 2 N I

l · x · uY = xxT

yi j = xi xj


Research Question?

Determine lower bounds on the optimal value of MIQCP by constructing strong convex relaxations of MIQCP.

Disjuncti

ve Programming


Disjunctive Programming

P = f x j Ax ¸ bg

DisjunctionPolyhedral Relaxation

Separation Problem

Given x2P show that x2PD or find an inequality which is satisfied by all points in PD and is violated by x.

6


Anureet Saxena, Axioma Inc.

CGLP

T heorem: x 2 PD if and only if the optimalvalue of the following cut generating linear pro-gram (CGLP ) is non-negative.

min ®x ¡ ¯s:t

®= utA + vtD t 8t = 1 : : :q

¯ · utb+ vtdt 8t = 1 : : :q

ut;vt ¸ 0 8t = 1 : : :q

P qt=1(u

t»+ vt»t) = 1



P = f x j Ax ¸ bg


Outer Approximation of MIQCP defined by the incumbent solution



P = f x j Ax ¸ bg


What are the sources of non-convexity in

MIQCP?



P = f x j Ax ¸ bg


• xj2 Z j2 NI

• Elementary 0-1 disjunction

(xj · 0) OR (xj ¸ 1)

• Split Disjunctions

• GUB Disjunctions

Integrality Constraints

?

Y=xxT


Y=xxT

Y=xxT

All eigenvalues of Y-xxT are equal to zero.

Eigenvectors of Y-xxT associated with non-zero eigenvalues can be used as sources of cuts


Y=xxT

Ohh!!I don’t like fractional components. I can use them to get good cuts

MILP


Y=xxT

MIQCP

Ohh!!I don’t like non-zero eigenvalues. I can use them to get good cuts


Negative Eigenvalues of Y-xxT

If (Y ¡ xxT )c = ¸c where ¸ < 0 then

² (cTx)2 · Y:ccT is a convex quadratic cutwhich cuts o®(x; Y )

² equivalent to imposing the SDP conditionY ¡ xxT ¸ SDP 0 by SOCP cuts.


Positive Eigenvalues of Y-xxT

If (Y ¡ xxT )c = ¸c where ¸ > 0 then

Y:ccT · (cTx)2

is a non-convex quadratic cut which cuts o®(x; Y ).

Univariate non-convex expression

Y:ccT · t2

t = cTx



min(x;Y )2OA cTx max(x;Y )2OA cTx

cTx

Y:ccT · (cTx)2

(cTx)2



cTx

p(cTx) + q

Y.ccT· p(cTx) + q

SecantApproximation



cTx

p1(cTx) + q1 p2(cTx) + q2

µµL µU



cTx

"µL (c) · cTx · µ

Y:ccT · p1(cTx) + q1

#W

"µ · cTx · µU(c)

Y:ccT · p2(cTx) + q2

#

19

Cutting Plane Algorithm


(x; Y )

(cTx)2 · Y:ccT Y:ccT · (cTx)2

Derive Disjunction

Derive DisjunctiveCut

CGLP

Convex Quadratic Cut

¸ > 0¸ < 0

Extract E igenvaluesand E igenvectors ofY ¡ xxT .

20

Sequential ConvexificationT heorem Let c1; : : : ; cn denote a set of mutually-orthogonal unit vectors in Rn, and let

S0 =

8><

>:(x;Y )

¯¯¯¯¯¯¯

A i:Y + aTi x + bi · 0 i = 1 : : :ml · x · u

Y ¡ xxT ¸ SDP 0

9>=

>;

Sj = clconv³Sj ¡ 1 \

n(x;Y ) j Y:cj cTj · (cTj x)

2o´

f or j = 1 : : :n

Sn+j = clconv³Sn+j ¡ 1 \

n(x;Y ) j xj 2 f 0;1g

o´f or j = 1 : : :p

T he following statements hold true:

Sn = clconv

8><

>:(x;Y )

¯¯¯¯¯¯¯

A i:Y + aTi x + bi · 0 i = 1 : : :ml · x · u

Y ¡ xxT = 0

9>=

>;

Sn+p = clconv

8>>>><

>>>>:

(x;Y )

¯¯¯¯¯¯¯¯¯¯

A i:Y + aTi x + bi · 0 i = 1 : : :ml · x · u

Y ¡ xxT = 0xj 2 f 0;1g j = 1 : : :p

9>>>>=

>>>>;

Y.ccT · (cT x)2

Can we improve the disjunctive cuts by choosing c more

carefully?

21

We are searching for vectors c which satisfy,

1. Y :ccT > (cT x)2

2. max(x;Y )2OAcTx ¡ min(x;Y )2OAc

Tx is assmall as possible

Improving Disjunctions?


This condition is always satisfied if c belongs to vector space spanned by eigenvectors of Y-xxT associated with positive eigenvalues.

This can be calculated by solving a linear program whose right hand side is a linear function of c

22

We are searching for vectors c which satisfy,

1. Y :ccT > (cT x)2

2. max(x;Y )2OAcTx ¡ min(x;Y )2OAc

Tx is assmall as possible

Improving Disjunctions?


This condition is always satisfied if c belongs to vector space spanned by eigenvectors of Y-xxT associated with positive eigenvalues.

This can be calculated by solving a linear program whose right hand side is a linear function of c

This problem can be formulated as a mixed integer linear program!!

Univariate Expression Generating Mixed Integer Program (UGMIP)

23

Cutting Plane Algorithm


Extract E igenvaluesand E igenvectors ofY ¡ xxT .

(x; Y )

(cTx)2 · Y:ccT Y:ccT · (cTx)2

Derive Disjunction

Derive DisjunctiveCut

CGLP

Convex Quadratic Cut

¸ > 0¸ < 0

UGMIP

24

MIQCP Reformulations


MIQCP (x)

MIQCP (x,Y)MIQCP (x,Y)RLT + SDP

Disjunctive Cuts

B & B

MIQCP (x)Projected Ineq

Lifting

Strengthening

Strengthening ?

Heavy Relaxation

Light Relaxation

Projection

25

MIQCP Reformulations


MIQCP (x)

MIQCP (x,Y)MIQCP (x,Y)RLT + SDP

Disjunctive Cuts

B & B

MIQCP (x)Projected Ineq

Lifting

Strengthening

Strengthening ?

Heavy Relaxation

Light Relaxation

Projection

Convex Relaxations of Non-Convex Mixed Integer Quadratically Constrained Problems: Projected Formulations

A. Saxena, P. Bonami and J. Lee

Anureet Saxena, Axioma Inc. 26

Projecting the RLT Formulation

Ak:Y + aTkx + bk · 0 k = 1 : : :m

Yi j ¡³lixj + uj xi ¡ liuj

´· 0 8i; j

Yi j ¡³uixj + lj xi ¡ ui lj

´· 0 8i; j

Yi j ¡³uixj + uj xi ¡ uiuj

´¸ 0 8i; j

Yi j ¡³lixj + lj xi ¡ li lj

´¸ 0 8i; j

RLT Inequalities

y¡ij (x) = max f uixj + uj xi ¡ uiuj ; lixj + lj xi ¡ li lj g 8i; j

y+ij (x) = min f lixj + uj xi ¡ liuj ;uixj + lj xi ¡ ui lj g 8i; j



Ak:Y + aTk x + bk · 0 k = 1 : : :m

y¡ij (x) · Yi j · y+ij (x) 8i; j

P(x;Y )

Qx =nx j 9Y s.t. (x;Y ) 2 P(x;Y )

o

Separation Problem

Given x show that x2Qx or find an inequality which is satisfied by all points in Qx and is violated by x.


min ´

aTk x + bk · ¡ Ak:Y + ´ k = 1 : : :m


ProjLP

T heorem x 2 Qx if and only if the optimalvalue of P rojLP is non-positive.

Anureet Saxena, Axioma Inc 28


min ´

aTk x + bk · ¡ Ak:Y + ´ k = 1 : : :m


Dual Solution(u, B, C)

X

i ;j

¡B i j y¡i j (x) ¡ Ci j y+i j (x)

¢+X

k2M

uk¡aTk x+bk

¢· 0

Projected Inequality



min ´

aTk x + bk · ¡ Ak:Y + ´ k = 1 : : :m


• A linear programming separation algorithm

• Handles large number O(n2) of RLT inequalities as bound

constraints

• No of constraints = No of quadratic constraints in the original problemAnureet Saxena, Axioma Inc 30

Surrogate Constraints

xTA1x + aT1x + b1 · 0

xTAmx + aTmx + bm · 0

u1

um

xTAx + aTx + b· 0 Surrogate Constraint

X

i ;j

¡B i j y¡i j (x) ¡ Ci j y+i j (x)

¢+X

k2M

uk¡aTk x+bk

¢· 0

A = B – CB, C ¸ 0

Surrogate Constraint

y¡ij (x) · xixj · y+ij (x)

Can we extract the convex part of the surrogate constraint

Surrogate Constraints

xTA1x + aT1x + b1 · 0


u1

um


A = B + C – DB ¸SDP 0C, D ¸ 0

xT Bx+X

i ;j

¡Ci j y¡i j (x) ¡ Di j y+i j (x)

¢+X

k2M

uk¡aTk x+bk

¢· 0

What happens if we add all such convex

quadratic cuts?

Projecting the SDP Formulationmin ´s.t.¡ Ak:Y +´ ¸ aTk x +bk; 8k 2 My¡i j (x) · Yi j · y+i j (x); 8i; j 2 NY +´I ¡ xxT ¸ SD P 0

xT Bx+X

i ;j

¡Ci j y¡i j (x) ¡ Di j y+i j (x)

¢+X

k2M

uk¡aTk x+bk

¢· 0

Dual Solution

(u, B, C, D)

T heorem x 2 Q+x if and only if the optimal

value of P rojSDP is non-positive.

ProjSDP

Separation Problem is a SDP


Projecting the SDP Formulation

T heorem x 2 Q+x if and only if the optimal

value of the following piecewise linear convexoptimzation problem is non-positive.

maxf F (u;B ) j u 2 § M ; B ¸ SDP 0g;

where § M = fu jPk2M uk = 1; u ¸ 0g and

F (u;B ) =Pi;j

³ Pk2M ukA

kij ¡ B i j

´+ ³y¡ij (x) ¡ xi xj

´

+Pi;j

³ Pk2M ukA

kij ¡ B i j

´¡ ³y+ij (x) ¡ xi xj

´

+Pk2M uk(x

TAkx) +Pk2M uk

³aTk x + bk

´

Unconstrained Convex Optimization Problem over the Cartessian product of a simplex and cone of PSD

matrices


35

Projecting the SDP Formulation


xTAx + aTx + b· 0A = B + C ¡ DB ¸ SDP 0; C;D ¸ 0

Projected Sub Gradient Heuristic

1. Initialize B = Projection of A to the cone of PSD matrices

2. Compute a sub gradient of F(u,B) at B

3. Perform line search along the sub gradient direction

4. Update B and goto 2

36

Limitations of Projection Theorems


xTA1x + aT1x + b1 · 0


u1

um


Once the surrogate constraint has been produced very little global information is used in the convexification process

37



min x3s.t.x1x2 ¡ x1 ¡ x2 ¡ x3 · 0¡ 6x1+ 8x2 · 33x1 ¡ x2 · 30 · x1;x2 · 1:5

• st_e23 instances from GlobalLib

• OPT = -1.08

• RLT = -3

• SDP + RLT = -1.5

• x = ( 0.811, 0.689, -1.500)

P1 = clconv

(

x

¯¯¯¯¯x1x2 ¡ x1 ¡ x2 ¡ x3 · 0

0 · x1;x2 · 1:5

)

(1:5;0; ¡ 1:5) 2 P1 (0;1:5; ¡ 1:5) 2 P10:5407 (1:5;0; ¡ 1:5) + 0:4593 (0;1:5; ¡ 1:5) = x0:5407 + 0:4593 = 1

The non-convex quadratic constraint and the bound constraints cannot cut off x

38



min x3s.t.x1x2 ¡ x1 ¡ x2 ¡ x3 · 0¡ 6x1+ 8x2 · 33x1 ¡ x2 · 30 · x1;x2 · 1:5

• st_e23 instances from GlobalLib

• OPT = -1.08

• RLT = -3

• SDP + RLT = -1.5

• x = ( 0.811, 0.689, -1.500)

P1 = clconv

(

x

¯¯¯¯¯x1x2 ¡ x1 ¡ x2 ¡ x3 · 0

0 · x1;x2 · 1:5

)

(1:5;0; ¡ 1:5) 2 P1 (0;1:5; ¡ 1:5) 2 P10:5407 (1:5;0; ¡ 1:5) + 0:4593 (0;1:5; ¡ 1:5) = x0:5407 + 0:4593 = 1

Global Information

We need a technique for engaging additional constraints in the

problem during the convexification process

39



x1x2 ¡ x1 ¡ x2 ¡ x3 · 0

12 (x1 + x2)

2 ¡ x1 ¡ x2 ¡ x3 ·12 (x1 ¡ x2)

2

Spectral Decomposition of"0 0:50:5 0

#

Univariate non-convex expression



min (x1 ¡ x2)s:t¡ 6x1+ 8x2 · 33x1 ¡ x2 · 30 · x1;x2 · 1:5

: : : · (x1 ¡ x2)2

(x1 ¡ x2)

(x1 ¡ x2)2

max (x1 ¡ x2)s:t¡ 6x1+ 8x2 · 33x1 ¡ x2 · 30 · x1;x2 · 1:5



SecantApproximation

(x1 ¡ x2)

(x1 ¡ x2)2 · 0:625(x1 ¡ x2) + 0:375

42



x1x2 ¡ x1 ¡ x2 ¡ x3 · 0

12 (x1 + x2)

2 ¡ x1 ¡ x2 ¡ x3 ·12 (x1 ¡ x2)

2

12 (x1+ x2)

2¡ x1¡ x2¡ x3 ·12 (0:625(x1 ¡ x2) + 0:375)

Spectral Decomposition

Secant ApproximationCuts off the incumbent

solution

43

Eigen Reformulation


xTAx + aTx + b· 0 A =Pj ¸ j vj v

Tj

P¸k>0 ¸k

³vTk x

´2+ aTx + b+

P¸k<0 ¸ksk · 0

yk = vTk x 8 k : ¸k < 0

sk = y2k 8 k : ¸k < 0

44

Eigen Reformulation


xTAx + aTx + b· 0 A =Pj ¸ j vj v

Tj

P¸k>0 ¸k

³vTk x

´2+ aTx + b+

P¸k<0 ¸ksk · 0

yk = vTk x 8 k : ¸k < 0

sk = y2k 8 k : ¸k < 0

Directions of maximal non-convexity

45

Eigen Reformulation


min aT0xs.t.xTAkx + aTk x + bk · 0 ; 8k 2 Mxj 2 Z ; 8j 2 N1

P¸kj>0 ¸kj

³vTkj x

´2+ aTk x + bk +

P¸kj<0 ¸kj skj · 0 ; 8k 2 M

ykj = vTkj x ; 8 j : ¸kj < 0; k 2 Mskj = y2kj ; 8 j : ¸kj < 0; k 2 ML kj · ykj · Ukj ; 8 j : ¸kj < 0; k 2 M :

Geometric correlations along

directions of maximal non-convexity

46

Eigen Reformulation


y1

y2

• st_glmp_kky instances from GlobalLib

• OPT = -2.5

• RLT = RLT+SDP = -3.0

x4x5+ x6x7 ¡ x3 ¡ z · 0y1 =

1p2x4 ¡

1p2x5

y2 =1p2x6 ¡

1p2x7

Projection along y1 and y2 Can we exploit these correlations in deriving strong cutting planes?


Polarity Cuts

1. Projection

2. Determine Extreme Points

3. Lifting

4. Convexification

y1

s1 · y21

L U

s1 ¡ (L + U)y1 ¡ LU · 0


Polarity Cuts

1. Projection


3. Lifting

4. Convexification

y1

Projection

min y1 max y1


Polarity Cuts

1. Projection


3. Lifting

4. Convexification

y1L U


Polarity Cuts

1. Projection


3. Lifting

4. Convexification

y1L U

(L ;L 2)

(U;U2)


Polarity Cuts

1. Projection


3. Lifting

4. Convexification

y1L U

(L ;L 2)

Facet

min ®kyk + ¯ksk ¡ °s.t.®kL k + ¯k(L k)

2 ¡ ° ¸ 0®kUk + ¯k(Uk)

2 ¡ ° ¸ 0®k ¡ ®+k + ®¡k = 0®+k + ®¡k ¡ ¯k = 1®+k ¸ 0; ®¡k ¸ 0; ¯k · 0

(U;U2)

Polar Program

Polar Program

Convex Relaxation

Extreme pointsof the projected set

minP

k2S (®k yk +¯k sk) ¡ °s.t.P

k2S

³®kytk +¯k (ytk)

2´¡ ° ¸ 0;8t

¯k · 0; 8k 2 S®k ¡ ®+k +®¡k =0; 8k 2 SP

k2S

¡®+k +®¡k ¡ ¯k

¢= 1

®+k ¸ 0; ®¡k ¸ 0

Polarity

• Additional problem constraints induce geometric correlations along directions of maximal non-convexity

• Projection mechanism identifies such correlations

• Polarity uses these correlations to derive strong cutting planes for MIQCP


53

Eigen Reformulation


y1

y2


• OPT = -2.5


x4x5+ x6x7 ¡ x3 ¡ z · 0y1 =

1p2x4 ¡

1p2x5

y2 =1p2x6 ¡

1p2x7


54

Eigen Reformulation


y1

y2


• OPT = -2.5


x4x5+ x6x7 ¡ x3 ¡ z · 0y1 =

1p2x4 ¡

1p2x5

y2 =1p2x6 ¡

1p2x7


Polarity cuts close 99.62% of the duality gap!!


Relaxations of MIQCP

RLT + SDPDisjunctive

CutsLow Width

Disjunctions

Projection RLT

Projection RLT + SDP

EigenReformulation

Polarity

56

Computational Results


Solvers

• Convex Relaxations – IpOpt • Eigenvalue Computations – LAPACK • Linear Programs & Mixed Integer Programs– CPLEX 11• COIN-OR / Bonmin based implementation

Test Bed

• 160 GlobalLIB Instances• 4 Chemical Process Design instances from Lee & Grossman• 90 Box QP Instances

57

Computational Results


Experiment Setup

• 1 Hour Time limit on each instance

Duality Gap=opt(F inal Relaxation) ¡ RLT

Opt(M I QCP ) ¡ RLT£ 100

58

GlobalLIB Instances


160 = 129 + 24 + 7

• All MIQCP Instances with upto 50 variables

• x1 x2 x3 x4 x5

• (x1+x2)/x3 ¸ 2x1

• x0.75

Numerical Problems

Zero Duality Gap

Non-Zero Duality Gap

59

Computational Results (Extended Formulations)


V1 SDP

V2 Disjunctive CutsV1

Y:ccT · (cTx)2

V3 UGMIPV2

60

GlobalLIB Instances


Summary of % Duality Gap Closed (129 Instances with non-zero Duality Gap)

V1 V2 V3>99.99 % 16 23 2398-99.99 % 1 44 5275-98 % 10 23 2125-75 % 11 22 200-25 % 91 17 13

Average Gap Closed 24.80% 76.49% 80.86%

61

GlobalLIB Instances


Summary of % Duality Gap Closed (129 Instances with non-zero Duality Gap)

V1 V2 V3>99.99 % 16 23 2398-99.99 % 1 44 5275-98 % 10 23 2125-75 % 11 22 200-25 % 91 17 13

Average Gap Closed 24.80% 76.49% 80.86%

62

Version (2,3) vs Version 1


% Duality Gap ClosedInstance V1 V2 V3

st_qpc-m3a 0.00% 98.10% 99.16%st_ph13 0.00% 99.38% 98.80%st_ph11 0.00% 99.46% 98.19%ex3_1_4 0.00% 86.31% 99.57%st_jcbpaf2 0.00% 99.47% 99.61%st_ph12 0.00% 99.49% 99.62%ex2_1_9 0.00% 98.79% 99.73%prob05 0.00% 99.78% 99.49%st_glmp_kky 0.00% 99.80% 99.71%st_e24 0.00% 99.81% 99.81%st_ph15 0.00% 99.83% 99.81%st_bsj4 0.00% 99.86% 99.80%st_ph14 0.00% 99.85% 99.86%st_e08 0.00% 99.81% 99.89%st_ht 0.00% 99.81% 99.89%st_pan2 0.00% 68.54% 99.91%ex2_1_1 0.00% 72.62% 99.92%st_fp1 0.00% 72.62% 99.92%st_pan1 0.00% 99.72% 99.92%ex5_2_4 0.00% 79.31% 99.92%st_e02 0.00% 99.88% 99.95%st_kr 0.00% 99.93% 99.95%


st_e33 0.00% 99.94% 99.95%st_z 0.00% 99.96% 99.95%st_qpc-m0 0.00% 99.96% 99.96%st_phex 0.00% 99.96% 99.96%st_e26 0.00% 99.96% 99.96%st_m1 0.00% 99.96% 99.96%ex2_1_6 0.00% 99.95% 99.97%st_fp6 0.00% 99.92% 99.97%st_e07 0.00% 99.97% 99.97%st_glmp_kk92 0.00% 99.98% 99.98%st_ph3 0.00% 99.98% 99.98%st_ph20 0.00% 99.98% 99.98%st_qpk1 0.00% 99.98% 99.98%st_bsj2 0.00% 99.98% 99.96%st_ph2 0.00% 99.98% 99.98%st_ph1 0.00% 99.98% 99.98%ex2_1_5 0.00% 99.98% 99.99%st_fp5 0.00% 99.98% 99.99%ex3_1_3 0.00% 99.99% 99.99%st_bpv2 0.00% 99.99% 99.99%st_qpc-m1 0.00% 99.99% 99.98%st_qpc-m3b 0.00% 100.00% 100.00%

63

Version (2,3) vs Version 1



st_qpc-m3a 0.00% 98.10% 99.16%st_ph13 0.00% 99.38% 98.80%st_ph11 0.00% 99.46% 98.19%ex3_1_4 0.00% 86.31% 99.57%st_jcbpaf2 0.00% 99.47% 99.61%st_ph12 0.00% 99.49% 99.62%ex2_1_9 0.00% 98.79% 99.73%prob05 0.00% 99.78% 99.49%st_glmp_kky 0.00% 99.80% 99.71%st_e24 0.00% 99.81% 99.81%st_ph15 0.00% 99.83% 99.81%st_bsj4 0.00% 99.86% 99.80%st_ph14 0.00% 99.85% 99.86%st_e08 0.00% 99.81% 99.89%st_ht 0.00% 99.81% 99.89%st_pan2 0.00% 68.54% 99.91%ex2_1_1 0.00% 72.62% 99.92%st_fp1 0.00% 72.62% 99.92%st_pan1 0.00% 99.72% 99.92%ex5_2_4 0.00% 79.31% 99.92%st_e02 0.00% 99.88% 99.95%st_kr 0.00% 99.93% 99.95%


st_e33 0.00% 99.94% 99.95%st_z 0.00% 99.96% 99.95%st_qpc-m0 0.00% 99.96% 99.96%st_phex 0.00% 99.96% 99.96%st_e26 0.00% 99.96% 99.96%st_m1 0.00% 99.96% 99.96%ex2_1_6 0.00% 99.95% 99.97%st_fp6 0.00% 99.92% 99.97%st_e07 0.00% 99.97% 99.97%st_glmp_kk92 0.00% 99.98% 99.98%st_ph3 0.00% 99.98% 99.98%st_ph20 0.00% 99.98% 99.98%st_qpk1 0.00% 99.98% 99.98%st_bsj2 0.00% 99.98% 99.96%st_ph2 0.00% 99.98% 99.98%st_ph1 0.00% 99.98% 99.98%ex2_1_5 0.00% 99.98% 99.99%st_fp5 0.00% 99.98% 99.99%ex3_1_3 0.00% 99.99% 99.99%st_bpv2 0.00% 99.99% 99.99%st_qpc-m1 0.00% 99.99% 99.98%st_qpc-m3b 0.00% 100.00% 100.00%

Observation

Either version 2 or version 3 closes >99% of the duality gap on 44 instances on which version 1 is unable to close any gap.

The relaxation obtained by adding disjunctive cuts can be substantially stronger than the SDP relaxation!!

66

Linear Complementarity Disjunctions


• Some problems have linear complementarity constraints

xi xj = 0

• These constraints can be used to derive the linear

complementarity disjunctions

(xi=0) OR (xj=0)

which can be used with the medley of other disjunctions to derive

disjunctive cuts

67

Linear Complementarity Disjunctions


Without Using LCD Using LCDInstance V2 V3 V2 V3

ex9_1_4 0.00% 1.55% 100.00% 99.97%ex9_2_1 60.04% 92.02% 99.95% 99.95%ex9_2_2 88.29% 98.06% 100.00% 100.00%ex9_2_3 0.00% 47.17% 99.99% 99.99%ex9_2_4 99.87% 99.89% 99.99% 100.00%ex9_2_6 87.93% 62.00% 80.22% 92.09%ex9_2_7 51.47% 86.25% 99.97% 99.95%

ObservationLinear Complementarity conditions can be exploited effectively within a disjunctive programming framework to derive strong cuts


Y=xxT

Y=xxT

All eigenvalues of Y-xxT are equal to zero.

Eigenvectors of Y-xxT associated with non-zero eigenvalues can be used as sources of cuts

What is the effect of these disjunctive cuts on the spectrum of

Y-xxT ?


Spectrum of Y-xxT

% Duality Gap closed by

V1 V2 Instance Chosen < 10 % > 90 % st_jcbpaf2 > 40% < 60% ex9_2_7 < 10% < 10% ex7_3_1


Version 1, 0% Gap Closed

0 5 10 15 20 25 30 35-0.60

-0.40

-0.20

0.00

0.20

0.40

0.60

0.80

1.00

1.20

Sum of Positive Eigen Values of Y-xxT

Sum of NegativeEigen Values of Y-xxT


Version 2, 99.47% Gap Closed

0 50 100 150 200 250 300-0.60

-0.40

-0.20

0.00

0.20

0.40

0.60

0.80

1.00

1.20


Version 3, 99.61% Gap Closed

0 20 40 60 80 100 120-0.60

-0.40

-0.20

0.00

0.20

0.40

0.60

0.80

1.00

1.20

78

Computational Results (Projected Formulations)


W1 ProjLP

W2

Disjunctive Cuts

W1 PolarityCuts

All experiments were done using the eigen reformulation

Computational Results: GlobalLib (Projected)

Disjunctive Cuts Extended

ProjLP ProjLP + PolarLP ProjLP ProjLP +

PolarLP SDP + RLT SDP + RLT + Dsj

>99.99 % gap closed 19 23 19 23 16 2398-99.99 % gap closed 22 31 5 21 1 4475-98 % gap closed 35 33 17 18 10 2325-75 % gap closed 34 23 26 32 11 220-25 % gap closed 14 14 57 30 91 170-(-0.22) % gap closed 4 4 4 4 0 0Average Gap Closed 70.65% 76.06% 40.92% 60.48% 24.80% 76.49%Average Time taken (sec) 4.616 19.462 0.893 0.814 198.043 978.140



















ObservationWe can generate relaxations in the space of x

variables which are almost as strong as those in the extended space even though our computing

times are 100 times smaller on average.







16%

30%







16%

30%

ObservationPolarity cuts can capture a portion of

strengthening derived from disjunctive cuts

Global Information at work!!


Computational Results: Box QP (Projected)

W3 ProjLPProjected GradientHeuristic

W3-SDP ProjLP

All experiments were done using the eigen reformulation



% Duality Gap Closed Time Taken (sec)Instances W3 W3-SDP W3 W3-SDP W3 (Adjusted)spar20* 94.60 - 99.97 91.54 - 99.91 2.49 - 408.36 0.84 - 2.46 0.51 - 1.60spar30* 89.87 - 99.99 51.41 - 98.79 12.33 - 565.88 1.74 - 14.38 3.32 - 14.49spar40* 87.85 - 99.60 21.78 - 89.63 35.77 - 134.8 4.16 - 65.28 13.75 - 49.76spar50* 87.88 - 97.53 11.38 - 50.15 50.22 - 180.96 8.76 - 99.13 28.95 - 76.19spar60* 85.78 - 90.99 0.00 - 0.00 121.83 - 226.11 111.07 - 127.47 86.28 - 141.77spar70* 89.78 - 99.36 0.00 - 53.67 191.12 - 693.28 22.02 - 202.98 92.63 - 143.35spar80* 88.13 - 97.49 2.94 - 56.23 257.62 - 892.96 34.77 - 67.66 121.62 - 230.53spar90* 89.44 - 96.60 5.73 - 50.13 408.73 - 991.04 46.98 - 95.66 184.63 - 294.92spar100* 92.15 - 96.46 8.17 - 51.79 538.03 - 1509.96 75.49 - 112.69 279.41 - 385.64Average 95.19% 50.01% 280.50 37.89 101.57







ObservationConvex quadratic cuts generated using the projected gradient heuristic can capture a

substantial portion of strengthening derived from the SDP+RLT relaxations




ObservationConvex quadratic cuts generated using the projected gradient heuristic can capture a

substantial portion of strengthening derived from the SDP+RLT relaxationsJust using the eigen

reformulation with ProjLP closes 50% of the duality gap !!







spar100-075-1 Instance

• 95.84% gap closed in 1509 sec• 94.84% gap closed in 366 sec

Assessing the Tailing off Behaviour



% Time spent onCut Generation

Instances W3 W3-SDPspar20* 26.28 - 95.77 0.12 - 0.24spar30* 17.78 - 91.48 0.00 - 0.21spar40* 27.5 - 78.37 0.01 - 0.13spar50* 51.02 - 79.73 0.01 - 0.11spar60* 46.29 - 56.61 0.10 - 0.12spar70* 71.13 - 87.7 0.01 - 0.11spar80* 76.37 - 84.44 0.01 - 0.02spar90* 73.44 - 88.25 0.01 - 0.02spar100* 77.49 - 92.3 0.01 - 0.23Average 66.05% 0.04%

Time Spent on Cut Generation

• Increases with version W3 reaching 75% for larger instances

• Remains less than 0.25% for all instances with W3-SDP

• ProjLP can be solved very efficiently


Computational Results: Box QP (Projected)Time to solve

% Duality Gap Closed Time Taken (sec) last relaxationInstances SDPLR SDPA Proj SDPLR SDPA Proj Projspar20* 99.67 - 100 99.67 - 99.99 94.6 - 99.97 0.97 - 56.37 1.98 - 3.39 2.48 - 408.35 0.05 - 0.32spar30* 97.81 - 100 97.81 - 99.99 89.87 - 99.99 3.57 - 243.3 16.66 - 29.33 12.33 - 565.88 0.06 - 0.89spar40* 96.6 - 100 96.6 - 99.99 87.85 - 99.6 10.3 - 515.73 105.68 - 157.83 35.77 - 134.8 0.16 - 1.18spar50* 95.55 - 100 95.55 - 99.99 87.88 - 97.53 41.72 - 926.15 438.77 - 589.17 50.21 - 180.95 0.13 - 0.86spar60* 98.69 - 100 98.69 - 99.99 85.78 - 90.99 88.05 - 532.45 1150.06 - 1408.32 121.83 - 226.1 0.53 - 1.55spar70* 98.46 - 100 98.46 - 99.99 89.78 - 99.36 133.07 - 3600.75 2769.98 - 3721.34 191.11 - 693.27 0.48 - 1.1spar80* 97.85 - 100 97.84 - 99.99 88.13 - 97.49 965.18 - 5413.02 6618.79 - 8285.12 257.61 - 892.95 0.56 - 2.02spar90* 97.83 - 99.99 97.83 - 99.99 89.44 - 96.6 2403.62 - 7049.49 12838.46 - 17048.98 408.73 - 991.04 0.77 - 1.51spar100* 98.17 - 99.38 98.17 - 99.38 92.15 - 96.46 5355.2 - 10295.88 23509.13 - 28604.12 538.02 - 1509.96 0.82 - 2Average 99.40% 99.40% 95.19% 1741.20 5247.04 280.50 0.67













No. ConstraintsNo. Variables Linear Convex (Non-Linear) Computing Time (sec) % Duality Gap Closed

Instances SDP Proj SDP Proj SDP Proj SDP Proj SDP Projspar100-025-1 5151 203 20201 156 1 119 5719.42 1.14 98.93% 92.36%spar100-025-2 5151 201 20201 151 1 95 10185.65 1.52 99.09% 92.16%spar100-025-3 5151 201 20201 150 1 114 5407.09 1.24 99.33% 93.26%spar100-050-1 5151 201 20201 150 1 98 10139.57 1.07 98.17% 93.62%spar100-050-2 5151 201 20201 150 1 113 5355.20 1.26 98.57% 94.13%spar100-050-3 5151 201 20201 150 1 97 7281.26 0.82 99.39% 95.81%spar100-075-1 5151 201 20201 150 1 131 9660.79 2.00 99.19% 95.84%spar100-075-2 5151 201 20201 150 1 109 6576.10 1.23 99.18% 96.47%spar100-075-3 5151 199 20201 147 1 90 10295.88 0.87 99.19% 96.06%



No. ConstraintsNo. Variables Linear Convex (Non-Linear) Computing Time (sec) % Duality Gap Closed

Instances SDP Proj SDP Proj SDP Proj SDP Proj SDP Projspar100-025-1 5151 203 20201 156 1 119 5719.42 1.14 98.93% 92.36%spar100-025-2 5151 201 20201 151 1 95 10185.65 1.52 99.09% 92.16%spar100-025-3 5151 201 20201 150 1 114 5407.09 1.24 99.33% 93.26%spar100-050-1 5151 201 20201 150 1 98 10139.57 1.07 98.17% 93.62%spar100-050-2 5151 201 20201 150 1 113 5355.20 1.26 98.57% 94.13%spar100-050-3 5151 201 20201 150 1 97 7281.26 0.82 99.39% 95.81%spar100-075-1 5151 201 20201 150 1 131 9660.79 2.00 99.19% 95.84%spar100-075-2 5151 201 20201 150 1 109 6576.10 1.23 99.18% 96.47%spar100-075-3 5151 199 20201 147 1 90 10295.88 0.87 99.19% 96.06%

Very little computational overheads at the nodes of the enumeration tree



ObservationStrengthened relaxations produced by our code are almost as strong as the SDP+ RLT relaxations and can be solved in less than 2 sec; state of art SDP

solvers can take upto a couple of hours to solve these relaxations in the extended space


Research Question?

1978-1988 •Data Structures•Theoretical Computer Science

1988-1998 •Linear Programming

1998-2008 •Mixed Integer Linear Programming

2008-2018 • ?Anureet Saxena, Axioma Inc 100

Research Question?

1978-1988 •Data Structures•Theoretical Computer Science

1988-1998 •Linear Programming

1998-2008 •Mixed Integer Linear Programming

2008-2018• Mixed Integer Non-Linear

Programming


Go Global for Global Optimization

Documents

Convex Relaxations of Non-Convex Mixed Integer Quadratically Constrained Problems