A Tighter Piecewise McCormick
Relaxation for Bilinear Problems
Pedro M. CastroCentro de Investigação Operacional
Faculdade de CiΓͺncias
Universidade de Lisboa
MINLP 2014
Problem definition (NLP)
June 3, 2014 Tighter Piecewise McCormick Relaxation 2
ππππ- parameters
π β vector of continuous variablesπ΅πΏπ - (π, π) index set
π β π β strictly bilinear problemsπ = π β can be allowed (quadratic problems)βπ(π₯)- linear function in π₯
Relaxation provides a lower bound
min π0(π₯)
ππ(π₯) =
(π,π)βπ΅πΏπ
πππππ₯ππ₯π + βπ(π₯) π β π
β’ Bilinear program
0 β€ π₯πΏ β€ π₯ β€ π₯π
π₯ β βπ
ππ(π₯) β€ 0 π β π\{0}
(P)
MINLP 2014
Introduction
β’ Bilinear problems occur in a variety of applicationsβ Process network problems (Quesada & Grossmann, 1995)
β Water networks (Bagajewicz, 2000)
β Pooling and blending (Haverly, 1978)
β’ Non-convex, leading to multiple local solutionsβ Gradient based solvers unable to certify optimality
β’ Need for global optimization algorithmsβ What do they have in common?
β’ Linear (LP) or mixed-integer linear (MILP) relaxation of (P) β LBβ LP: Standard McCormick envelopes (1976)
β MILP: Piecewise McCormick envelopes (Bergamini et al. 2005)
β MILP: Multiparametric disaggregation (Teles et al. 2013; Kolodziej et al. 2013)
β’ Solution of (P) with fast local solver β UBβ Using LB as starting point
β’ Tight relaxation critical to ensure convergenceβ Relative optimality gap (UB-LB)/LB<
June 3, 2014 3Tighter Piecewise McCormick Relaxation
MINLP 2014
Generalized Disjunctive Program (Balas, 1979; Raman & Grossmann, 1994)
Piecewise McCormick relaxation
β’ Domain of divided into π uniform partitions
June 3, 2014 4Tighter Piecewise McCormick Relaxation
π€ππ = π₯ππ₯π
Optimization will
pick single partition. . .
π₯ππΏ
π₯ππ
π₯π,1πΏ π₯π,2
πΏ π₯π,ππΏ
π₯π,πππ₯π,πβ1
ππ₯π,2ππ₯π,1
ππ = 1 π = 2 π = π
π¦π,1 = π‘ππ’π π¦π,2 = π‘ππ’π π¦π,π = π‘ππ’πβ¨ β¨ β¨. . .
π₯πππ₯π
πΏ π₯π
π₯π
Convex
hull
1.25
1.45
1.65
1.85
2.05
2.25
2.45
200 250 300 350 400
π = 2
1.25
1.45
1.65
1.85
2.05
2.25
2.45
200 250 300 350 400
π = 20
Standard McCormickπ = 1
β¨π
π¦ππ
π€ππ β₯ π₯π β π₯πππΏ + π₯π
πΏ β π₯π β π₯ππΏ β π₯ππ
πΏ
π€ππ β₯ π₯π β π₯πππ + π₯π
π β π₯π β π₯ππ β π₯ππ
π
π€ππ β€ π₯π β π₯πππΏ + π₯π
π β π₯π β π₯ππ β π₯ππ
πΏ
π€ππ β€ π₯π β π₯πππ + π₯π
πΏ β π₯π β π₯ππΏ β π₯ππ
π
π₯πππΏ β€ π₯π β€ π₯ππ
π
π₯ππΏ β€ π₯π β€ π₯π
π
π₯πππΏ = π₯π
πΏ + (π₯ππ β π₯π
πΏ) β (π β 1)/π
π₯πππ = π₯π
πΏ + (π₯ππ β π₯π
πΏ) β π/π
(PR-MILP)
MINLP 2014
New tighter piecewise relaxation
β’ Basic ideaβ Use partition-dependent
bounds also for variable π₯πβ’ π₯πππ
πΏ /π₯ππππ lower/upper bound
when π₯π constrained to π
β Better bounds tighter relaxation lower gap
June 3, 2014 5Tighter Piecewise McCormick Relaxation
β¨π
π¦ππ
π€ππ β₯ π₯π β π₯πππΏ + π₯πππ
πΏ β π₯π β π₯ππππΏ β π₯ππ
πΏ
π€ππ β₯ π₯π β π₯πππ + π₯πππ
π β π₯π β π₯ππππ β π₯ππ
π
π€ππ β€ π₯π β π₯πππΏ + π₯πππ
π β π₯π β π₯ππππ β π₯ππ
πΏ
π€ππ β€ π₯π β π₯πππ + π₯πππ
πΏ β π₯π β π₯ππππΏ β π₯ππ
π
π₯ππππΏ β€ π₯π β€ π₯πππ
π
π₯πππΏ β€ π₯π β€ π₯ππ
π
π₯πππΏ = π₯π
πΏ + (π₯ππ β π₯π
πΏ) β (π β 1)/π
π₯πππ = π₯π
πΏ + (π₯ππ β π₯π
πΏ) β π/π
(PRT-MILP)
Convex
hull
min π§π = π0 π₯ =
π,π βπ΅πΏ
πππ0π€ππ + β0 π₯
ππ π₯ =
π,π βπ΅πΏ
πππππ€ππ + βπ π₯ β€ 0 βπ β π\{0}
π₯π = π π₯ππ π π¦ππ = 1
β {π|(π, π) β π΅πΏ}
π₯πππΏ = π₯π
πΏ +π₯ππ β π₯π
πΏ β π β 1
π
π₯πππ = π₯π
πΏ +π₯ππ β π₯π
πΏ β π
ππ₯πππΏ β π¦ππ β€ π₯ππ β€ π₯ππ
π β π¦ππ
β π π, π β π΅πΏ , π
π€ππ β₯ π
π₯πππ β π₯πππΏ + π₯πππ
πΏ β π₯ππ β π₯ππππΏ β π₯ππ
πΏ β π¦ππ
π€ππ β₯ π
π₯πππ β π₯πππ + π₯πππ
π β π₯ππ β π₯ππππ β π₯ππ
π β π¦ππ
π€ππ β€ π
π₯πππ β π₯πππΏ + π₯πππ
π β π₯ππ β π₯ππππ β π₯ππ
πΏ β π¦ππ
π€ππ β€ π
π₯πππ β π₯πππ + π₯πππ
πΏ β π₯ππ β π₯ππππΏ β π₯ππ
π β π¦ππ
π₯π = π π₯πππ
β (π, π)
π₯ππππΏ β π¦ππ β€ π₯πππ β€ π₯πππ
π β π¦ππ β (π, π) β π΅πΏ, π β {1, β¦ , π}
MINLP 2014
How to generate bounds π₯ππππΏ & π₯πππ
π ?
β’ Optimality bound contraction with McCormick envelopesβ Solve multiple instances of LP problem (BC), two per:
β’ Non-partitioned variable π₯πβ’ Partitioned variable π₯πβ’ Partition π
β Focus on regions that can actually improve current UBβ’ π§Β΄ obtained from solving (P) with local solver
β (BC) infeasible? Remove partition πβ of πβ from (PRT-MILP)
β Computation can be time consuming
June 3, 2014 6Tighter Piecewise McCormick Relaxation
π€ππ β₯ π₯π β π₯ππΏ + π₯π
πΏ β π₯π β π₯ππΏ β π₯π
πΏ
π€ππ β₯ π₯π β π₯ππ + π₯π
π β π₯π β π₯ππ β π₯π
π
π€ππ β€ π₯π β π₯ππΏ + π₯π
π β π₯π β π₯ππ β π₯π
πΏ
π€ππ β€ π₯π β π₯ππ + π₯π
πΏ β π₯π β π₯ππΏ β π₯π
π
β (π, π) β π΅πΏ
π₯πβπΏ = π₯πβπβ
πΏ β€ π₯πβ β€ π₯πβπβπ = π₯πβ
π
π₯πΏ β€ π₯ β€ π₯π
ππ(π₯) =
π,π βπ΅πΏπ
πππππ€ππ + βπ(π₯) β€ 0 βπ β π\{0}
π₯πβπβπβπΏ βΆ= min π₯πβ (π₯πβπβπβ
π βΆ= max π₯πβ
(BC)
π0 π₯ =
π,π βπ΅πΏ0
πππ0π€ππ + β0(π₯) β€ π§Β΄
MINLP 2014
New optimization algorithm
β’ Key features
β User selects:
β’ π partitions
β’ Partitioned
variables π₯π
β Preliminary bound
contraction stage
β Bounds updated at
different levels
β LB from
MILP relaxation
β UB from NLP (single
starting point)
β Computes an
optimality gap
June 3, 2014 Tighter Piecewise McCormick Relaxation 7
Given:
partitions
, , , , ,
SOLVE (P)with local solver
upper bound
SOLVE (standard bound contract)
update, , ,
Initialization
SOLVE (BC)
Feasible?remove partitionNOtighter bounds
,
YES
YESupdate bounds
NO
YESupdate bounds
NO
NO
SOLVE(PRT-MILP)
YES
OutputOptimal solution
Optimality gap
Time Elapsed
lower bound
SOLVE (P)with local solver
upper bound
Given:
partitions
, , , , ,
SOLVE (P)with local solver
upper bound
SOLVE (standard bound contract)
update, , ,
Initialization
SOLVE (BC)
Feasible?remove partitionNOtighter bounds
,
YES
YESupdate bounds
NO
YESupdate bounds
NO
NO
SOLVE(PRT-MILP)
YES
OutputOptimal solution
Optimality gap
Time Elapsed
lower bound
SOLVE (P)with local solver
upper bound
MINLP 2014
Illustrative example P1
June 3, 2014 Tighter Piecewise McCormick Relaxation 8
min π0 π₯ = βπ₯1 + π₯1π₯2 β π₯2
β6π₯1 + 8π₯2 β€ 33π₯1 β π₯2 β€ 3
0 β€ π₯1, π₯2 β€ 1.5 Standard optimality bound contraction
0.357143 β€ π₯1 β€ 1.375
0 β€ π₯2 β€ 1.26
Partitioned variable: π₯1π΅πΏ = {(2,1)}
Non-partitioned variable: π₯2
Partition π π₯1ππΏ π₯1π
πΌ π₯21ππΏ π₯21π
πΌ
1 0.357143 0.696429 LP is infeasible
2 0.696429 1.035714 0 1.137609
3 1.035714 1.375 0.107143 1.195150
Partition-dependent bound contraction π = 3
π§ = π0 π₯ = β1.083333
Global optimum
π§π = β1.185207
(PR-MILP) relaxation
π§π = β1.169619
(PRT-MILP) relaxation
Gap
8.60%
7.38%
MINLP 2014
How about using more partitions?
β’ Up to two orders of magnitude reduction in optimality gapβ Major increase in total computational time
β Still, new approach performs best!
June 3, 2014 9Tighter Piecewise McCormick Relaxation
Relaxation Partitions (π) 15 150 1500 7500
(PR-MILP)Total CPUs 0.93 1.60 3.63 302
Optimality gap 1.67% 0.188% 0.0189% 0.0038%
(PRT-MILP)Optimality gap 0.77% 0.010% 0.0002%
Total CPUs 3.63 27.8 267
P1
Relaxation Partitions (π) 9 90 900 2700
(PR-MILP)Total CPUs 0.83 1.2 7.76 478
Optimality gap 8.41% 0.84% 0.0830% 0.0278%
(PRT-MILP)Optimality gap 0.84% 0.02% 0.0002%
Total CPUs 3.33 24.0 223
P2
min 6π₯12 + 4π₯2
2 β 2.5π₯1π₯2
π₯1π₯2 β 8 β₯ 0
1 β€ π₯1, π₯2 β€ 10
MINLP 2014
Results for larger test problems
β’ 16 easiest water-using design problemsβ New (PRT-MILP) approach leads to lower gaps at termination (3600 CPUs)
β Faster in 56% of the cases (finest partition level)
β Single failure when finding best-known solution for (P)
June 3, 2014 10Tighter Piecewise McCormick Relaxation
Number of variables (PR-MILP) (PRT-MILP)
Non-partitioned |πΌ| Partitioned |π½| Partitions (π) Solution Total CPUs Gap Gap Total CPUs Solution
Ex2 20 1610
74.469918.8 0.1250% 0.0484% 141
74.469950 3609 0.1245% 0.0103% 1536
Ex6 30 2510
142.081644.7 0.0435% 0.0110% 205
142.081650 3614 0.1379% 0.0041% 1090
Ex8 30 2010
164.489818.8 0.7134% 0.2132% 207
164.4898100 3613 1.9508% 0.0130% 1803
Ex10 15 910
169.11736.69 0.0160% 0.0008% 45.3
169.1173100 809 0.0023% 0.0000% 323
Ex14 32 2410 329.9689 282 1.2259% 1.0499% 447
329.569820 329.9566 3615 2.3368% 0.9206% 4270
Ex15 40 3010 361.5177 231 0.7963% 0.7427% 723 361.5177
20 361.6856 3618 0.9361% 0.7779% 4486 361.6856
Ex16 24 1610
285.934317.4 1.1105% 0.8861% 212
285.934350 3610 1.5755% 0.0526% 2042S
ub
se
t o
f p
rob
lem
s s
ele
cte
d fo
r ill
ustr
atio
n
MINLP 2014
How to determine the number of partitions?
β’ Quality of relaxation increases with πβ Careful not to generate intractable MILPs (Gounaris et al. 2009)
β’ Several works identify most appropriate value for a particular problem(Misener et al. 2011; Wittmann-Holhbein and Pistikopoulos, 2013; 2014)
β No method to estimate π as a function of problem complexity
β Needed to provide a fair comparison with commercial solvers
β’ Proposed formula meets important criteria:β Returns an integer value
β Ensures minimum of two partitionsβ’ So as to benefit from piecewise relaxation scheme
β Reduces π with increase in problem sizeβ’ Roughly inversely proportional (r2=0.76)
June 3, 2014 11Tighter Piecewise McCormick Relaxation
π = 1 +πΌ
|πΌ| β |π½|πΌ = 1.8πΈ4
MINLP 2014
Comparison to commercial solvers
β’ Performance profiles(Dolan & MorΓ©, 2002)
β KPI: Optimality gap
June 3, 2014 12Tighter Piecewise McCormick Relaxation
0
0.2
0.4
0.6
0.8
1
0 1 2 3 4 5 6 7 8 9 10
Cum
ula
tive d
istr
ibution f
unction
(PR-MILP)
(PRT-MILP)
GloMIQO 2.3
BARON 12.3.3
GAMS 24.1.3, CPLEX 12.5.1, GloMIQO 2.3,
BARON 12.3.3, Intel Core i7-3770 (3.07 GHz),
8 GB RAM, Windows 7 64-bit
Termination criteria: gap=0.0001%, Time=3600 CPUs
β’ (PRT-MILP) better (PR-MILP)
β’ GloMIQO outperforms BARON
# Failures finding best-known solution (34 problems)
Algorithm GloMIQO (PRT-MILP) BARON (PR-MILP)
Suboptimal 3 5 2 7
No solution 0 0 4 0
Total 3 5 6 7
MINLP 2014
Why not bivariate partitioning?
β’ Domain of π₯π and π₯π known a priori for each 2-D partition
β Slightly better performance than univariate partitioning for P1
β Gap for 1252 partitions one order of magnitude larger vs. (PRT-MILP)
June 3, 2014 13Tighter Piecewise McCormick Relaxation
π=1
π
πΒ΄=1
π
π¦ππππΒ΄
π€ππ β₯ π₯π β π₯ππΒ΄πΏ + π₯ππ
πΏ β π₯π β π₯πππΏ β π₯ππΒ΄
πΏ
π€ππ β₯ π₯π β π₯ππΒ΄π + π₯ππ
π β π₯π β π₯πππ β π₯ππΒ΄
π
π€ππ β€ π₯π β π₯ππΒ΄πΏ + π₯ππ
π β π₯π β π₯πππ β π₯ππΒ΄
πΏ
π€ππ β€ π₯π β π₯ππΒ΄π + π₯ππ
πΏ β π₯π β π₯πππΏ β π₯ππΒ΄
π
π₯πππΏ β€ π₯π β€ π₯ππ
π
π₯ππΒ΄πΏ β€ π₯π β€ π₯ππΒ΄
π
β(π, π)
(PR
-BV
-MIL
P)
Relaxation Partitions (π) 16 144 2500 10000
Univariate
(PR-MILP)
Total CPUs 0.88 1.42 9.11 429
Optimality gap 1.70% 0.197% 0.0114% 0.0029%
π΅ 4 12 50 100 125
Bivariate
(PR-BV-MILP)
Optimality gap 1.28% 0.188% 0.0078% 0.0029% 0.0018%
Total CPUs 0.93 0.91 7.55 217 469
MINLP 2014
Conclusions
β’ Bilinear problems tackled through piecewise relaxation (PR)
β’ Novel approach with partition-dependent boundsfor all bilinear variablesβ Significant improvement in relaxation quality
β Need for extensive optimality-based bound contraction (BC)β’ Total computational time is sometimes lower
β’ New algorithm outperforms state-of-the-art global solversβ Specific class of water-using network design problems
β’ PR and BC schemes should be used to greater extentβ Integrating with spatial B&B subject of future work
β’ See CACE paper for further details
β’ Acknowledgments:β Luso-American Foundation (2013 Portugal-U.S. Research Networks)
β Fundação para a CiΓͺncia e Tecnologia (Investigador FCT 2013)
June 3, 2014 14Tighter Piecewise McCormick Relaxation