View
9
Download
0
Category
Preview:
Citation preview
A Tighter Piecewise McCormick
Relaxation for Bilinear Problems
Pedro M. CastroCentro de Investigação Operacional
Faculdade de Ciências
Universidade de Lisboa
MINLP 2014
Problem definition (NLP)
June 3, 2014 Tighter Piecewise McCormick Relaxation 2
𝑎𝑖𝑗𝑞- parameters
𝒙 – vector of continuous variables𝐵𝐿𝑞 - (𝑖, 𝑗) index set
𝑖 ≠ 𝑗 – strictly bilinear problems𝑖 = 𝑗 – can be allowed (quadratic problems)ℎ𝑞(𝑥)- linear function in 𝑥
Relaxation provides a lower bound
min 𝑓0(𝑥)
𝑓𝑞(𝑥) =
(𝑖,𝑗)∈𝐵𝐿𝑞
𝑎𝑖𝑗𝑞𝑥𝑖𝑥𝑗 + ℎ𝑞(𝑥) 𝑞 ∈ 𝑄
• Bilinear program
0 ≤ 𝑥𝐿 ≤ 𝑥 ≤ 𝑥𝑈
𝑥 ∈ ℝ𝑚
𝑓𝑞(𝑥) ≤ 0 𝑞 ∈ 𝑄\{0}
(P)
MINLP 2014
Introduction
• Bilinear problems occur in a variety of applications– Process network problems (Quesada & Grossmann, 1995)
– Water networks (Bagajewicz, 2000)
– Pooling and blending (Haverly, 1978)
• Non-convex, leading to multiple local solutions– Gradient based solvers unable to certify optimality
• Need for global optimization algorithms– What do they have in common?
• Linear (LP) or mixed-integer linear (MILP) relaxation of (P) → LB– LP: Standard McCormick envelopes (1976)
– MILP: Piecewise McCormick envelopes (Bergamini et al. 2005)
– MILP: Multiparametric disaggregation (Teles et al. 2013; Kolodziej et al. 2013)
• Solution of (P) with fast local solver → UB– Using LB as starting point
• Tight relaxation critical to ensure convergence– Relative optimality gap (UB-LB)/LB<
June 3, 2014 3Tighter Piecewise McCormick Relaxation
MINLP 2014
Generalized Disjunctive Program (Balas, 1979; Raman & Grossmann, 1994)
Piecewise McCormick relaxation
• Domain of divided into 𝑁 uniform partitions
June 3, 2014 4Tighter Piecewise McCormick Relaxation
𝑤𝑖𝑗 = 𝑥𝑖𝑥𝑗
Optimization will
pick single partition. . .
𝑥𝑗𝐿
𝑥𝑗𝑈
𝑥𝑗,1𝐿 𝑥𝑗,2
𝐿 𝑥𝑗,𝑁𝐿
𝑥𝑗,𝑁𝑈𝑥𝑗,𝑁−1
𝑈𝑥𝑗,2𝑈𝑥𝑗,1
𝑈𝑛 = 1 𝑛 = 2 𝑛 = 𝑁
𝑦𝑗,1 = 𝑡𝑟𝑢𝑒 𝑦𝑗,2 = 𝑡𝑟𝑢𝑒 𝑦𝑗,𝑁 = 𝑡𝑟𝑢𝑒∨ ∨ ∨. . .
𝑥𝑗𝑈𝑥𝑗
𝐿 𝑥𝑗
𝑥𝑖
Convex
hull
1.25
1.45
1.65
1.85
2.05
2.25
2.45
200 250 300 350 400
𝑁 = 2
1.25
1.45
1.65
1.85
2.05
2.25
2.45
200 250 300 350 400
𝑁 = 20
Standard McCormick𝑁 = 1
∨𝑛
𝑦𝑗𝑛
𝑤𝑖𝑗 ≥ 𝑥𝑖 ⋅ 𝑥𝑗𝑛𝐿 + 𝑥𝑖
𝐿 ⋅ 𝑥𝑗 − 𝑥𝑖𝐿 ⋅ 𝑥𝑗𝑛
𝐿
𝑤𝑖𝑗 ≥ 𝑥𝑖 ⋅ 𝑥𝑗𝑛𝑈 + 𝑥𝑖
𝑈 ⋅ 𝑥𝑗 − 𝑥𝑖𝑈 ⋅ 𝑥𝑗𝑛
𝑈
𝑤𝑖𝑗 ≤ 𝑥𝑖 ⋅ 𝑥𝑗𝑛𝐿 + 𝑥𝑖
𝑈 ⋅ 𝑥𝑗 − 𝑥𝑖𝑈 ⋅ 𝑥𝑗𝑛
𝐿
𝑤𝑖𝑗 ≤ 𝑥𝑖 ⋅ 𝑥𝑗𝑛𝑈 + 𝑥𝑖
𝐿 ⋅ 𝑥𝑗 − 𝑥𝑖𝐿 ⋅ 𝑥𝑗𝑛
𝑈
𝑥𝑗𝑛𝐿 ≤ 𝑥𝑗 ≤ 𝑥𝑗𝑛
𝑈
𝑥𝑖𝐿 ≤ 𝑥𝑖 ≤ 𝑥𝑖
𝑈
𝑥𝑗𝑛𝐿 = 𝑥𝑗
𝐿 + (𝑥𝑗𝑈 − 𝑥𝑗
𝐿) ⋅ (𝑛 − 1)/𝑁
𝑥𝑗𝑛𝑈 = 𝑥𝑗
𝐿 + (𝑥𝑗𝑈 − 𝑥𝑗
𝐿) ⋅ 𝑛/𝑁
(PR-MILP)
MINLP 2014
New tighter piecewise relaxation
• Basic idea– Use partition-dependent
bounds also for variable 𝑥𝑖• 𝑥𝑖𝑗𝑛
𝐿 /𝑥𝑖𝑗𝑛𝑈 lower/upper bound
when 𝑥𝑗 constrained to 𝑛
– Better bounds tighter relaxation lower gap
June 3, 2014 5Tighter Piecewise McCormick Relaxation
∨𝑛
𝑦𝑗𝑛
𝑤𝑖𝑗 ≥ 𝑥𝑖 ⋅ 𝑥𝑗𝑛𝐿 + 𝑥𝑖𝑗𝑛
𝐿 ⋅ 𝑥𝑗 − 𝑥𝑖𝑗𝑛𝐿 ⋅ 𝑥𝑗𝑛
𝐿
𝑤𝑖𝑗 ≥ 𝑥𝑖 ⋅ 𝑥𝑗𝑛𝑈 + 𝑥𝑖𝑗𝑛
𝑈 ⋅ 𝑥𝑗 − 𝑥𝑖𝑗𝑛𝑈 ⋅ 𝑥𝑗𝑛
𝑈
𝑤𝑖𝑗 ≤ 𝑥𝑖 ⋅ 𝑥𝑗𝑛𝐿 + 𝑥𝑖𝑗𝑛
𝑈 ⋅ 𝑥𝑗 − 𝑥𝑖𝑗𝑛𝑈 ⋅ 𝑥𝑗𝑛
𝐿
𝑤𝑖𝑗 ≤ 𝑥𝑖 ⋅ 𝑥𝑗𝑛𝑈 + 𝑥𝑖𝑗𝑛
𝐿 ⋅ 𝑥𝑗 − 𝑥𝑖𝑗𝑛𝐿 ⋅ 𝑥𝑗𝑛
𝑈
𝑥𝑖𝑗𝑛𝐿 ≤ 𝑥𝑖 ≤ 𝑥𝑖𝑗𝑛
𝑈
𝑥𝑗𝑛𝐿 ≤ 𝑥𝑗 ≤ 𝑥𝑗𝑛
𝑈
𝑥𝑗𝑛𝐿 = 𝑥𝑗
𝐿 + (𝑥𝑗𝑈 − 𝑥𝑗
𝐿) ⋅ (𝑛 − 1)/𝑁
𝑥𝑗𝑛𝑈 = 𝑥𝑗
𝐿 + (𝑥𝑗𝑈 − 𝑥𝑗
𝐿) ⋅ 𝑛/𝑁
(PRT-MILP)
Convex
hull
min 𝑧𝑅 = 𝑓0 𝑥 =
𝑖,𝑗 ∈𝐵𝐿
𝑎𝑖𝑗0𝑤𝑖𝑗 + ℎ0 𝑥
𝑓𝑞 𝑥 =
𝑖,𝑗 ∈𝐵𝐿
𝑎𝑖𝑗𝑞𝑤𝑖𝑗 + ℎ𝑞 𝑥 ≤ 0 ∀𝑞 ∈ 𝑄\{0}
𝑥𝑗 = 𝑛 𝑥𝑗𝑛 𝑛 𝑦𝑗𝑛 = 1
∀ {𝑗|(𝑖, 𝑗) ∈ 𝐵𝐿}
𝑥𝑗𝑛𝐿 = 𝑥𝑗
𝐿 +𝑥𝑗𝑈 − 𝑥𝑗
𝐿 ∙ 𝑛 − 1
𝑁
𝑥𝑗𝑛𝑈 = 𝑥𝑗
𝐿 +𝑥𝑗𝑈 − 𝑥𝑗
𝐿 ∙ 𝑛
𝑁𝑥𝑗𝑛𝐿 ⋅ 𝑦𝑗𝑛 ≤ 𝑥𝑗𝑛 ≤ 𝑥𝑗𝑛
𝑈 ⋅ 𝑦𝑗𝑛
∀ 𝑗 𝑖, 𝑗 ∈ 𝐵𝐿 , 𝑛
𝑤𝑖𝑗 ≥ 𝑛
𝑥𝑖𝑗𝑛 ⋅ 𝑥𝑗𝑛𝐿 + 𝑥𝑖𝑗𝑛
𝐿 ⋅ 𝑥𝑗𝑛 − 𝑥𝑖𝑗𝑛𝐿 ⋅ 𝑥𝑗𝑛
𝐿 ⋅ 𝑦𝑗𝑛
𝑤𝑖𝑗 ≥ 𝑛
𝑥𝑖𝑗𝑛 ⋅ 𝑥𝑗𝑛𝑈 + 𝑥𝑖𝑗𝑛
𝑈 ⋅ 𝑥𝑗𝑛 − 𝑥𝑖𝑗𝑛𝑈 ⋅ 𝑥𝑗𝑛
𝑈 ⋅ 𝑦𝑗𝑛
𝑤𝑖𝑗 ≤ 𝑛
𝑥𝑖𝑗𝑛 ⋅ 𝑥𝑗𝑛𝐿 + 𝑥𝑖𝑗𝑛
𝑈 ⋅ 𝑥𝑗𝑛 − 𝑥𝑖𝑗𝑛𝑈 ⋅ 𝑥𝑗𝑛
𝐿 ⋅ 𝑦𝑗𝑛
𝑤𝑖𝑗 ≤ 𝑛
𝑥𝑖𝑗𝑛 ⋅ 𝑥𝑗𝑛𝑈 + 𝑥𝑖𝑗𝑛
𝐿 ⋅ 𝑥𝑗𝑛 − 𝑥𝑖𝑗𝑛𝐿 ⋅ 𝑥𝑗𝑛
𝑈 ⋅ 𝑦𝑗𝑛
𝑥𝑖 = 𝑛 𝑥𝑖𝑗𝑛
∀ (𝑖, 𝑗)
𝑥𝑖𝑗𝑛𝐿 ⋅ 𝑦𝑗𝑛 ≤ 𝑥𝑖𝑗𝑛 ≤ 𝑥𝑖𝑗𝑛
𝑈 ⋅ 𝑦𝑗𝑛 ∀ (𝑖, 𝑗) ∈ 𝐵𝐿, 𝑛 ∈ {1, … , 𝑁}
MINLP 2014
How to generate bounds 𝑥𝑖𝑗𝑛𝐿 & 𝑥𝑖𝑗𝑛
𝑈 ?
• Optimality bound contraction with McCormick envelopes– Solve multiple instances of LP problem (BC), two per:
• Non-partitioned variable 𝑥𝑖• Partitioned variable 𝑥𝑗• Partition 𝑛
– Focus on regions that can actually improve current UB• 𝑧´ obtained from solving (P) with local solver
– (BC) infeasible? Remove partition 𝑛∗ of 𝑗∗ from (PRT-MILP)
– Computation can be time consuming
June 3, 2014 6Tighter Piecewise McCormick Relaxation
𝑤𝑖𝑗 ≥ 𝑥𝑖 ⋅ 𝑥𝑗𝐿 + 𝑥𝑖
𝐿 ⋅ 𝑥𝑗 − 𝑥𝑖𝐿 ⋅ 𝑥𝑗
𝐿
𝑤𝑖𝑗 ≥ 𝑥𝑖 ⋅ 𝑥𝑗𝑈 + 𝑥𝑖
𝑈 ⋅ 𝑥𝑗 − 𝑥𝑖𝑈 ⋅ 𝑥𝑗
𝑈
𝑤𝑖𝑗 ≤ 𝑥𝑖 ⋅ 𝑥𝑗𝐿 + 𝑥𝑖
𝑈 ⋅ 𝑥𝑗 − 𝑥𝑖𝑈 ⋅ 𝑥𝑗
𝐿
𝑤𝑖𝑗 ≤ 𝑥𝑖 ⋅ 𝑥𝑗𝑈 + 𝑥𝑖
𝐿 ⋅ 𝑥𝑗 − 𝑥𝑖𝐿 ⋅ 𝑥𝑗
𝑈
∀ (𝑖, 𝑗) ∈ 𝐵𝐿
𝑥𝑗∗𝐿 = 𝑥𝑗∗𝑛∗
𝐿 ≤ 𝑥𝑗∗ ≤ 𝑥𝑗∗𝑛∗𝑈 = 𝑥𝑗∗
𝑈
𝑥𝐿 ≤ 𝑥 ≤ 𝑥𝑈
𝑓𝑞(𝑥) =
𝑖,𝑗 ∈𝐵𝐿𝑞
𝑎𝑖𝑗𝑞𝑤𝑖𝑗 + ℎ𝑞(𝑥) ≤ 0 ∀𝑞 ∈ 𝑄\{0}
𝑥𝑖∗𝑗∗𝑛∗𝐿 ∶= min 𝑥𝑖∗ (𝑥𝑖∗𝑗∗𝑛∗
𝑈 ∶= max 𝑥𝑖∗
(BC)
𝑓0 𝑥 =
𝑖,𝑗 ∈𝐵𝐿0
𝑎𝑖𝑗0𝑤𝑖𝑗 + ℎ0(𝑥) ≤ 𝑧´
MINLP 2014
New optimization algorithm
• Key features
– User selects:
• 𝑁 partitions
• Partitioned
variables 𝑥𝑗
– Preliminary bound
contraction stage
– Bounds updated at
different levels
– LB from
MILP relaxation
– UB from NLP (single
starting point)
– Computes an
optimality gap
June 3, 2014 Tighter Piecewise McCormick Relaxation 7
Given:
partitions
, , , , ,
SOLVE (P)with local solver
upper bound
SOLVE (standard bound contract)
update, , ,
Initialization
SOLVE (BC)
Feasible?remove partitionNOtighter bounds
,
YES
YESupdate bounds
NO
YESupdate bounds
NO
NO
SOLVE(PRT-MILP)
YES
OutputOptimal solution
Optimality gap
Time Elapsed
lower bound
SOLVE (P)with local solver
upper bound
Given:
partitions
, , , , ,
SOLVE (P)with local solver
upper bound
SOLVE (standard bound contract)
update, , ,
Initialization
SOLVE (BC)
Feasible?remove partitionNOtighter bounds
,
YES
YESupdate bounds
NO
YESupdate bounds
NO
NO
SOLVE(PRT-MILP)
YES
OutputOptimal solution
Optimality gap
Time Elapsed
lower bound
SOLVE (P)with local solver
upper bound
MINLP 2014
Illustrative example P1
June 3, 2014 Tighter Piecewise McCormick Relaxation 8
min 𝑓0 𝑥 = −𝑥1 + 𝑥1𝑥2 − 𝑥2
−6𝑥1 + 8𝑥2 ≤ 33𝑥1 − 𝑥2 ≤ 3
0 ≤ 𝑥1, 𝑥2 ≤ 1.5 Standard optimality bound contraction
0.357143 ≤ 𝑥1 ≤ 1.375
0 ≤ 𝑥2 ≤ 1.26
Partitioned variable: 𝑥1𝐵𝐿 = {(2,1)}
Non-partitioned variable: 𝑥2
Partition 𝑛 𝑥1𝒏𝐿 𝑥1𝒏
𝑼 𝑥21𝒏𝐿 𝑥21𝒏
𝑼
1 0.357143 0.696429 LP is infeasible
2 0.696429 1.035714 0 1.137609
3 1.035714 1.375 0.107143 1.195150
Partition-dependent bound contraction 𝑁 = 3
𝑧 = 𝑓0 𝑥 = −1.083333
Global optimum
𝑧𝑅 = −1.185207
(PR-MILP) relaxation
𝑧𝑅 = −1.169619
(PRT-MILP) relaxation
Gap
8.60%
7.38%
MINLP 2014
How about using more partitions?
• Up to two orders of magnitude reduction in optimality gap– Major increase in total computational time
– Still, new approach performs best!
June 3, 2014 9Tighter Piecewise McCormick Relaxation
Relaxation Partitions (𝑁) 15 150 1500 7500
(PR-MILP)Total CPUs 0.93 1.60 3.63 302
Optimality gap 1.67% 0.188% 0.0189% 0.0038%
(PRT-MILP)Optimality gap 0.77% 0.010% 0.0002%
Total CPUs 3.63 27.8 267
P1
Relaxation Partitions (𝑁) 9 90 900 2700
(PR-MILP)Total CPUs 0.83 1.2 7.76 478
Optimality gap 8.41% 0.84% 0.0830% 0.0278%
(PRT-MILP)Optimality gap 0.84% 0.02% 0.0002%
Total CPUs 3.33 24.0 223
P2
min 6𝑥12 + 4𝑥2
2 − 2.5𝑥1𝑥2
𝑥1𝑥2 − 8 ≥ 0
1 ≤ 𝑥1, 𝑥2 ≤ 10
MINLP 2014
Results for larger test problems
• 16 easiest water-using design problems– New (PRT-MILP) approach leads to lower gaps at termination (3600 CPUs)
– Faster in 56% of the cases (finest partition level)
– Single failure when finding best-known solution for (P)
June 3, 2014 10Tighter Piecewise McCormick Relaxation
Number of variables (PR-MILP) (PRT-MILP)
Non-partitioned |𝐼| Partitioned |𝐽| Partitions (𝑁) Solution Total CPUs Gap Gap Total CPUs Solution
Ex2 20 1610
74.469918.8 0.1250% 0.0484% 141
74.469950 3609 0.1245% 0.0103% 1536
Ex6 30 2510
142.081644.7 0.0435% 0.0110% 205
142.081650 3614 0.1379% 0.0041% 1090
Ex8 30 2010
164.489818.8 0.7134% 0.2132% 207
164.4898100 3613 1.9508% 0.0130% 1803
Ex10 15 910
169.11736.69 0.0160% 0.0008% 45.3
169.1173100 809 0.0023% 0.0000% 323
Ex14 32 2410 329.9689 282 1.2259% 1.0499% 447
329.569820 329.9566 3615 2.3368% 0.9206% 4270
Ex15 40 3010 361.5177 231 0.7963% 0.7427% 723 361.5177
20 361.6856 3618 0.9361% 0.7779% 4486 361.6856
Ex16 24 1610
285.934317.4 1.1105% 0.8861% 212
285.934350 3610 1.5755% 0.0526% 2042S
ub
se
t o
f p
rob
lem
s s
ele
cte
d fo
r ill
ustr
atio
n
MINLP 2014
How to determine the number of partitions?
• Quality of relaxation increases with 𝑁– Careful not to generate intractable MILPs (Gounaris et al. 2009)
• Several works identify most appropriate value for a particular problem(Misener et al. 2011; Wittmann-Holhbein and Pistikopoulos, 2013; 2014)
– No method to estimate 𝑁 as a function of problem complexity
– Needed to provide a fair comparison with commercial solvers
• Proposed formula meets important criteria:– Returns an integer value
– Ensures minimum of two partitions• So as to benefit from piecewise relaxation scheme
– Reduces 𝑁 with increase in problem size• Roughly inversely proportional (r2=0.76)
June 3, 2014 11Tighter Piecewise McCormick Relaxation
𝑁 = 1 +𝛼
|𝐼| ∙ |𝐽|𝛼 = 1.8𝐸4
MINLP 2014
Comparison to commercial solvers
• Performance profiles(Dolan & Moré, 2002)
– KPI: Optimality gap
June 3, 2014 12Tighter Piecewise McCormick Relaxation
0
0.2
0.4
0.6
0.8
1
0 1 2 3 4 5 6 7 8 9 10
Cum
ula
tive d
istr
ibution f
unction
(PR-MILP)
(PRT-MILP)
GloMIQO 2.3
BARON 12.3.3
GAMS 24.1.3, CPLEX 12.5.1, GloMIQO 2.3,
BARON 12.3.3, Intel Core i7-3770 (3.07 GHz),
8 GB RAM, Windows 7 64-bit
Termination criteria: gap=0.0001%, Time=3600 CPUs
• (PRT-MILP) better (PR-MILP)
• GloMIQO outperforms BARON
# Failures finding best-known solution (34 problems)
Algorithm GloMIQO (PRT-MILP) BARON (PR-MILP)
Suboptimal 3 5 2 7
No solution 0 0 4 0
Total 3 5 6 7
MINLP 2014
Why not bivariate partitioning?
• Domain of 𝑥𝑖 and 𝑥𝑗 known a priori for each 2-D partition
– Slightly better performance than univariate partitioning for P1
– Gap for 1252 partitions one order of magnitude larger vs. (PRT-MILP)
June 3, 2014 13Tighter Piecewise McCormick Relaxation
𝑛=1
𝑁
𝑛´=1
𝑁
𝑦𝑖𝑛𝑗𝑛´
𝑤𝑖𝑗 ≥ 𝑥𝑖 ⋅ 𝑥𝑗𝑛´𝐿 + 𝑥𝑖𝑛
𝐿 ⋅ 𝑥𝑗 − 𝑥𝑖𝑛𝐿 ⋅ 𝑥𝑗𝑛´
𝐿
𝑤𝑖𝑗 ≥ 𝑥𝑖 ⋅ 𝑥𝑗𝑛´𝑈 + 𝑥𝑖𝑛
𝑈 ⋅ 𝑥𝑗 − 𝑥𝑖𝑛𝑈 ⋅ 𝑥𝑗𝑛´
𝑈
𝑤𝑖𝑗 ≤ 𝑥𝑖 ⋅ 𝑥𝑗𝑛´𝐿 + 𝑥𝑖𝑛
𝑈 ⋅ 𝑥𝑗 − 𝑥𝑖𝑛𝑈 ⋅ 𝑥𝑗𝑛´
𝐿
𝑤𝑖𝑗 ≤ 𝑥𝑖 ⋅ 𝑥𝑗𝑛´𝑈 + 𝑥𝑖𝑛
𝐿 ⋅ 𝑥𝑗 − 𝑥𝑖𝑛𝐿 ⋅ 𝑥𝑗𝑛´
𝑈
𝑥𝑖𝑛𝐿 ≤ 𝑥𝑖 ≤ 𝑥𝑖𝑛
𝑈
𝑥𝑗𝑛´𝐿 ≤ 𝑥𝑗 ≤ 𝑥𝑗𝑛´
𝑈
∀(𝑖, 𝑗)
(PR
-BV
-MIL
P)
Relaxation Partitions (𝑁) 16 144 2500 10000
Univariate
(PR-MILP)
Total CPUs 0.88 1.42 9.11 429
Optimality gap 1.70% 0.197% 0.0114% 0.0029%
𝑵 4 12 50 100 125
Bivariate
(PR-BV-MILP)
Optimality gap 1.28% 0.188% 0.0078% 0.0029% 0.0018%
Total CPUs 0.93 0.91 7.55 217 469
MINLP 2014
Conclusions
• Bilinear problems tackled through piecewise relaxation (PR)
• Novel approach with partition-dependent boundsfor all bilinear variables– Significant improvement in relaxation quality
– Need for extensive optimality-based bound contraction (BC)• Total computational time is sometimes lower
• New algorithm outperforms state-of-the-art global solvers– Specific class of water-using network design problems
• PR and BC schemes should be used to greater extent– Integrating with spatial B&B subject of future work
• See CACE paper for further details
• Acknowledgments:– Luso-American Foundation (2013 Portugal-U.S. Research Networks)
– Fundação para a Ciência e Tecnologia (Investigador FCT 2013)
June 3, 2014 14Tighter Piecewise McCormick Relaxation
Recommended