Upload
others
View
20
Download
0
Embed Size (px)
Citation preview
Nonlinear Programming Formulation ofChance-Constraints
Andreas Wächterjoint with Alejandra Peña-Ordieres and Jim Luedtke
Department of Industrial Engineering and Management SciencesNorthwestern University
Evanston, IL, [email protected]
US & Mexico Workshop on Optimization and Its ApplicationsHuatulco, Mexico
January 9, 2018
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Problem Statement
minx∈X
f (x)
s.t. Pξ[c(x , ξ) ≤ 0] ≥ 1− α
f (x) : Rn → Rc(x , ξ) : Rn × Ξ→ Rm
I Random variable ξ with support ΞI Assume f (x) and c(x , ξ) are sufficiently smooth for all ξ ∈ ΞI X ⊆ Rn: Captures additional constraints
I Goal: Formulate as continuous NLPI Do not want to assume particular probability distributionI Do not want to assume convexity (or convex approximations)I Want to avoid combinatorial approachI Use powerful NLP algorithms and techniques
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Problem Statement
minx∈X
f (x)
s.t. Pξ[c(x , ξ) ≤ 0] ≥ 1− α
f (x) : Rn → Rc(x , ξ) : Rn × Ξ→ Rm
I Random variable ξ with support ΞI Assume f (x) and c(x , ξ) are sufficiently smooth for all ξ ∈ ΞI X ⊆ Rn: Captures additional constraintsI Goal: Formulate as continuous NLP
I Do not want to assume particular probability distributionI Do not want to assume convexity (or convex approximations)I Want to avoid combinatorial approachI Use powerful NLP algorithms and techniques
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Constraint in Probability SpaceImpose p(x) = P[c(x , ξ) ≤ 0] ≥ 1− α
-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5-0.5
0
0.5
1
1.5
( - 1) - p(x)
y = 0
c(x , ξ) = x2 − 2 + ξξ ∼ N (0, 1)
IssuesI Linearization is poor approximationI Always nonconvexI Used in [Hu, Hong, Zhang 13], [Bremer, Henrion, Möller 15]
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Quantile Formulation
-3 -2 -1 0 1 2 3
x
0
0.2
0.4
0.6
0.8
1
Cu
mu
lative
Dis
trib
utio
n F
un
ctio
n
Nomal CDF
I Let Y be a real-valued random variableI (1− α)-quantile:
QY1−α = inf{y ∈ R : P[Y ≤ y ] ≥ 1− α}
p(x) ≥ 1− α ⇐⇒ q1−α(x) = Qc(x ,ξ)1−α ≤ 0
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Quantile Formulation
-3 -2 -1 0 1 2 3
x
0
0.2
0.4
0.6
0.8
1
Cu
mu
lative
Dis
trib
utio
n F
un
ctio
n
Nomal CDF
I Let Y be a real-valued random variable (Y = c(x , ξ))I (1− α)-quantile:
Qc(x ,ξ)1−α = inf{y ∈ R : P[c(x , ξ) ≤ y ] ≥ 1− α}
p(x) ≥ 1− α ⇐⇒ q1−α(x) = Qc(x ,ξ)1−α ≤ 0
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Quantile Formulation
-3 -2 -1 0 1 2 3
x
0
0.2
0.4
0.6
0.8
1
Cu
mu
lative
Dis
trib
utio
n F
un
ctio
n
Nomal CDF
I Let Y be a real-valued random variable (Y = c(x , ξ))I (1− α)-quantile:
Qc(x ,ξ)1−α = inf{y ∈ R : P[c(x , ξ) ≤ y ] ≥ 1− α}
p(x) ≥ 1− α ⇐⇒ q1−α(x) = Qc(x ,ξ)1−α ≤ 0Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Choice of Formulation
p(x) ≥ 1− α ⇐⇒ q1−α(x) ≤ 0
-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5-0.5
0
0.5
1
1.5
( - 1) - p(x)
q1 -
(x)
y = 0
I Quantile formulation more suitable for NLP solver
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Sample Average Approximation (SAA)
minx∈X
f (x)
s.t. Pξ[c(x , ξ) ≤ 0] ≥ 1− α
f (x) : Rn → Rc(x , ξ) : Rn × Ξ→ R
I Finite scenario set Ξ̂N = {ξ̂1, . . . , ξ̂N} with ξ̂i ∈ Ξ chosen i.i.d.I For simplicity: ci (x) = c(x , ξ̂i )
SAA approximation of quantileI Ensure that M = d(1− α)Ne constraints are satisfiedI Scenario vector: C(x) = (c1(x), . . . , cN(x))I Order constraint values: c[1](x) ≤ c[2](x) ≤ . . . ≤ c[N](x)I Empirical quantile: q̃1−α(C(x)) = c[M](x) ≤ 0
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Sample Average Approximation (SAA)
minx∈X
f (x)
s.t. Pξ[c(x , ξ) ≤ 0] ≥ 1− α
f (x) : Rn → Rc(x , ξ) : Rn × Ξ→ R
I Finite scenario set Ξ̂N = {ξ̂1, . . . , ξ̂N} with ξ̂i ∈ Ξ chosen i.i.d.I For simplicity: ci (x) = c(x , ξ̂i )
SAA approximation of quantileI Ensure that M = d(1− α)Ne constraints are satisfied
I Scenario vector: C(x) = (c1(x), . . . , cN(x))I Order constraint values: c[1](x) ≤ c[2](x) ≤ . . . ≤ c[N](x)I Empirical quantile: q̃1−α(C(x)) = c[M](x) ≤ 0
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Sample Average Approximation (SAA)
minx∈X
f (x)
s.t. Pξ[c(x , ξ) ≤ 0] ≥ 1− α
f (x) : Rn → Rc(x , ξ) : Rn × Ξ→ R
I Finite scenario set Ξ̂N = {ξ̂1, . . . , ξ̂N} with ξ̂i ∈ Ξ chosen i.i.d.I For simplicity: ci (x) = c(x , ξ̂i )
SAA approximation of quantileI Ensure that M = d(1− α)Ne constraints are satisfiedI Scenario vector: C(x) = (c1(x), . . . , cN(x))
I Order constraint values: c[1](x) ≤ c[2](x) ≤ . . . ≤ c[N](x)I Empirical quantile: q̃1−α(C(x)) = c[M](x) ≤ 0
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Sample Average Approximation (SAA)
minx∈X
f (x)
s.t. Pξ[c(x , ξ) ≤ 0] ≥ 1− α
f (x) : Rn → Rc(x , ξ) : Rn × Ξ→ R
I Finite scenario set Ξ̂N = {ξ̂1, . . . , ξ̂N} with ξ̂i ∈ Ξ chosen i.i.d.I For simplicity: ci (x) = c(x , ξ̂i )
SAA approximation of quantileI Ensure that M = d(1− α)Ne constraints are satisfiedI Scenario vector: C(x) = (c1(x), . . . , cN(x))I Order constraint values: c[1](x) ≤ c[2](x) ≤ . . . ≤ c[N](x)
I Empirical quantile: q̃1−α(C(x)) = c[M](x) ≤ 0
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Sample Average Approximation (SAA)
minx∈X
f (x)
s.t. Pξ[c(x , ξ) ≤ 0] ≥ 1− α
f (x) : Rn → Rc(x , ξ) : Rn × Ξ→ R
I Finite scenario set Ξ̂N = {ξ̂1, . . . , ξ̂N} with ξ̂i ∈ Ξ chosen i.i.d.I For simplicity: ci (x) = c(x , ξ̂i )
SAA approximation of quantileI Ensure that M = d(1− α)Ne constraints are satisfiedI Scenario vector: C(x) = (c1(x), . . . , cN(x))I Order constraint values: c[1](x) ≤ c[2](x) ≤ . . . ≤ c[N](x)I Empirical quantile: q̃1−α(C(x)) = c[M](x) ≤ 0
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Empirical QuantileI Feasible region for c(x , ξ) = ξ1x1 + ξ2x2 − 1 ξ1, ξ1 ∼ N (0, 1)
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
N = 2000 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
N = 5000 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
N = 2000
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Empirical Quantile
I Feasible region for c(x , ξ) = ξ1x1 + ξ2x2 − 1 ξ1, ξ1 ∼ N (0, 1)
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
N = 2000 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
N = 5000 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
N = 2000
Observations:I Approximation improves as N increasesI Rough boundary of feasible region results in spurious local minimaI A lot of variance for small N as ξi are resampledI q̃1−α(C(x)) is non-differentiable
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Variance Reduction using Kernels
0.9 0.91 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99 1
0
0.5
1
1.5
2
I Y[1], . . . ,Y[N] ordered realizations of a random variable (c(x , ξi ))I Empirical quantiles: For p ∈ [0, 1] define
Q̃Yp = Y[j], where j ∈ {1, ..,N} withj−1N < p ≤
jN
I Kernel smoothing [Parson 79]
QY ,N,h1−α =∫ 1
0Q̃Yp 1hK
(α−p
h
)dp
=N∑
j=1Y[j]
=:wN,hj︷ ︸︸ ︷∫ jN
j−1N
1hK
(α−p
h
)dp
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Variance Reduction using Kernels
0.9 0.91 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99 1
0
0.5
1
1.5
2
I Y[1], . . . ,Y[N] ordered realizations of a random variable (c(x , ξi ))I Empirical quantiles: For p ∈ [0, 1] define
Q̃Yp = Y[j], where j ∈ {1, ..,N} withj−1N < p ≤
jN
I Kernel smoothing [Parson 79]
QY ,N,h1−α =∫ 1
0Q̃Yp 1hK
(α−p
h
)dp
=N∑
j=1Y[j]
=:wN,hj︷ ︸︸ ︷∫ jN
j−1N
1hK
(α−p
h
)dp
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Variance Reduction using Kernels
0.9 0.91 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99 1
0
0.5
1
1.5
2
I Y[1], . . . ,Y[N] ordered realizations of a random variable (c(x , ξi ))I Empirical quantiles: For p ∈ [0, 1] define
Q̃Yp = Y[j], where j ∈ {1, ..,N} withj−1N < p ≤
jN
I Kernel smoothing [Parson 79]
QY ,N,h1−α =∫ 1
0Q̃Yp 1hK
(α−p
h
)dp =
N∑j=1
Y[j]
=:wN,hj︷ ︸︸ ︷∫ jN
j−1N
1hK
(α−p
h
)dp
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Kernel Estimation of QuantilesI Approximate q1−α(C(x)) = QY1−α with Yi = c(x ; ξi ) by kernel
estimate
qN,h1−α(C(x)) :=N∑
i=1wN,hi c[i](x) ≤ 0
I The weights depend only on N and h and can be precomputed
I Epanechnikov kernel: K (u) = 34(1− u2)1|u|
Kernel Estimation of QuantilesI Approximate q1−α(C(x)) = QY1−α with Yi = c(x ; ξi ) by kernel
estimate
qN,h1−α(C(x)) :=N∑
i=1wN,hi c[i](x) ≤ 0
I The weights depend only on N and h and can be precomputedI Epanechnikov kernel: K (u) = 34(1− u
2)1|u|
Feasible Region of Kernel-Quantile Formulationc(x , ξ) = ξ1x1 + ξ2x2 − 1 ξ1, ξ1 ∼ N (0, 1) h ∈ {0.05, 0.01}
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
N = 2000 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
N = 5000 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
N = 2000
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Feasible Region of Kernel-Quantile Formulationc(x , ξ) = ξ1x1 + ξ2x2 − 1 ξ1, ξ1 ∼ N (0, 1) h ∈ {0.05, 0.01}
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
N = 2000 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
N = 5000 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
N = 2000Observations:
I “Looks” smoother and convexI Reduces variance as ξi are resampledI Large h creates biasI qN,h1−α(C(x)) has more points of non-differentiabilityI Usually “solved” well with Knitro, but no proper termination
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Smoothed Empirical CDF
-3 -2 -1 0 1 2 3
x
0
0.2
0.4
0.6
0.8
1
Cu
mu
lative
Dis
trib
utio
n F
un
ctio
n
Nomal CDF
I Quantile QYp = inf{y ∈ R : Φ(y) ≥ p} where Φ(y) is cdf of Y
I Y1, . . . ,YN (unordered) realizations of a random variableI Empirical cdf:
Φ̃(y) = 1N
N∑i=1
1≥0(y − Yi )I Smoothed cdf [Azzalini 81]: (K�(t) ≈ 1≥0(t) differentiable)
ΦN,�(y) = 1N
N∑i=1
K�(y − Yi )
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Smoothed Empirical CDF
-2 -1.5 -1 -0.5 0
c(x, )
0
0.2
0.4
0.6
0.8
1
Cum
ula
tive D
istr
ibution F
unction
Smoothing of empirical CDF
Empirical CDF
I Quantile QYp = inf{y ∈ R : Φ(y) ≥ p} where Φ(y) is cdf of YI Y1, . . . ,YN (unordered) realizations of a random variableI Empirical cdf:
Φ̃(y) = 1N
N∑i=1
1≥0(y − Yi )
I Smoothed cdf [Azzalini 81]: (K�(t) ≈ 1≥0(t) differentiable)
ΦN,�(y) = 1N
N∑i=1
K�(y − Yi )
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Smoothed Empirical CDF
-2 -1.5 -1 -0.5 0
c(x, )
0
0.2
0.4
0.6
0.8
1
Cum
ula
tive D
istr
ibution F
unction
Smoothing of empirical CDF
Empirical CDF
epsilon = 0.1
epsilon = 0.5
I Quantile QYp = inf{y ∈ R : Φ(y) ≥ p} where Φ(y) is cdf of YI Y1, . . . ,YN (unordered) realizations of a random variableI Empirical cdf:
Φ̃(y) = 1N
N∑i=1
1≥0(y − Yi )I Smoothed cdf [Azzalini 81]: (K�(t) ≈ 1≥0(t) differentiable)
ΦN,�(y) = 1N
N∑i=1
K�(y − Yi )
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Differentiable Quantile Estimates
-2 -1.5 -1 -0.5 0
c(x, )
0
0.2
0.4
0.6
0.8
1
Cum
ula
tive D
istr
ibution F
unction
Smoothing of empirical CDF
Empirical CDF
epsilon = 0.1
epsilon = 0.5
I Yi = c(x , ξi )I Smoothed cdf: ΦN,�(y) = 1N
∑Ni=1 K�(y − ci (x))
I Resulting quantile estimate q̂N,�p (C(x)) is the solution y∗ of
p = ΦN,�(y)
I Implicit function theorem guarantees that q̂N,�p (C(x)) exists and isdifferentiable
(for p ∈ {0, 1N ,2N , . . . ,
N−1N })
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Differentiable Quantile Estimates
-2 -1.5 -1 -0.5 0
c(x, )
0
0.2
0.4
0.6
0.8
1
Cum
ula
tive D
istr
ibution F
unction
Smoothing of empirical CDF
Empirical CDF
epsilon = 0.1
epsilon = 0.5
I Yi = c(x , ξi )I Smoothed cdf: ΦN,�(y) = 1N
∑Ni=1 K�(y − ci (x))
I Resulting quantile estimate q̂N,�p (C(x)) is the solution y∗ of
p = 1N
N∑i=1
K�(y − ci (x))
+ 12N
I Implicit function theorem guarantees that q̂N,�p (C(x)) exists and isdifferentiable
(for p ∈ {0, 1N ,2N , . . . ,
N−1N })
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Differentiable Quantile Estimates
-2 -1.5 -1 -0.5 0
c(x, )
0
0.2
0.4
0.6
0.8
1
Cum
ula
tive D
istr
ibution F
unction
Smoothing of empirical CDF
Empirical CDF
epsilon = 0.1
epsilon = 0.5
I Yi = c(x , ξi )I Smoothed cdf: ΦN,�(y) = 1N
∑Ni=1 K�(y − ci (x))
I Resulting quantile estimate q̂N,�p (C(x)) is the solution y∗ of
p = 1N
N∑i=1
K�(y − ci (x)) +12N
I Implicit function theorem guarantees that q̂N,�p (C(x)) exists and isdifferentiable (for p ∈ {0, 1N ,
2N , . . . ,
N−1N })
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Final Estimate
1. Kernel estimate using empirical quantiles c[i](x)
qN,h1−α(C(x)) =N∑
i=1wN,hi c[i](x)
2. Smoothed empirical quantile q̂N,�p (C(x)), solution of
p = 1N
N∑i=1
K�(y − ci (x)) +12N
3. Final estimate
qN,h,�1−α (C(x)) :=N∑
i=1wN,hi q̂
N,�iN
(C(x))
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Final Estimate
1. Kernel estimate using empirical quantiles c[i](x)
qN,h1−α(C(x)) =N∑
i=1wN,hi c[i](x)
2. Smoothed empirical quantile q̂N,�p (C(x)), solution of
p = 1N
N∑i=1
K�(y − ci (x)) +12N
3. Final estimate
qN,h,�1−α (C(x)) :=N∑
i=1wN,hi q̂
N,�iN
(C(x))
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Final Estimate
1. Kernel estimate using empirical quantiles c[i](x)
qN,h1−α(C(x)) =N∑
i=1wN,hi c[i](x)
2. Smoothed empirical quantile q̂N,�p (C(x)), solution of
p = 1N
N∑i=1
K�(y − ci (x)) +12N
3. Final estimate
qN,h,�1−α (C(x)) :=N∑
i=1wN,hi q̂
N,�iN
(C(x))
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Feasible Region with Smoothing (N = 100)Smoothing with =1, 0.5, 0.1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Kernel h=0.01 with smoothing = 0.06
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
x1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
x2
Smoothing with =1, 0.5, 0.1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
x1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
x2
Kernel h=0.01 with smoothing = 0.06
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
x1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
x2
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Example: Portfolio Optimization
maxx∈Rn
E[rT x ] = µT x
s.t. Pr [rT x ≥ −0.05] ≥ 0.95n∑
i=1xi = 1, x ≥ 0
I r ∼ N (µ,Σ), fixed µ ∈ R100 and Σ ∈ R100×100 pos. def.I 100 replications
Iterations with Knitro
N h min iter mean iter max iter500 0.05 15 26.7 54
10,000 0.005 17 29.3 45
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Example: Portfolio Optimization
maxx∈Rn
E[rT x ] = µT x
s.t. Pr [rT x ≥ −0.05] ≥ 0.95n∑
i=1xi = 1, x ≥ 0
I r ∼ N (µ,Σ), fixed µ ∈ R100 and Σ ∈ R100×100 pos. def.I 100 replications
Iterations with Knitro
N h min iter mean iter max iter500 0.05 15 26.7 54
10,000 0.005 17 29.3 45
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Portfolio Optimization
100 500 1000 5000 10000
Number of samples
0.89
0.9
0.91
0.92
0.93
0.94
0.95
0.96
0.97
P(c
(x,
))
h = 5.00e-02
100 500 1000 5000 10000
Number of samples
-3
-2
-1
0
1
2
3
Re
lative
fu
nctio
n e
rro
r (%
)
10-3 h = 5.00e-02
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Portfolio Optimization
100 500 1000 5000 10000
Number of samples
0.87
0.88
0.89
0.9
0.91
0.92
0.93
0.94
0.95
0.96
0.97
P(c
(x,
))
h = 1.00e-02
100 500 1000 5000 10000
Number of samples
-5
-4
-3
-2
-1
0
1
2
3
Re
lative
fu
nctio
n e
rro
r (%
)
10-3 h = 1.00e-02
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Portfolio Optimization
100 500 1000 5000 10000
Number of samples
0.87
0.88
0.89
0.9
0.91
0.92
0.93
0.94
0.95
0.96
0.97
P(c
(x,
))
h = 1.00e-03
100 500 1000 5000 10000
Number of samples
-3
-2
-1
0
1
2
3
Re
lative
fu
nctio
n e
rro
r (%
)
10-3 h = 1.00e-03
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Portfolio Optimization
100 500 1000 5000 10000
Number of samples
0.87
0.88
0.89
0.9
0.91
0.92
0.93
0.94
0.95
0.96
0.97
P(c
(x,
))
h = 1.00e-04
100 500 1000 5000 10000
Number of samples
-3
-2
-1
0
1
2
3
Re
lative
fu
nctio
n e
rro
r (%
)
10-3 h = 1.00e-04
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Joint Chance Constraints
I Original Problem
minx∈X
f (x)
s.t. Pξ[c(x , ξ) ≤ 0] ≥ 1− α
I Scenario vector
C(x) = (c(x , ξ1), . . . , c(x , ξN))
I NLP formulation
minx∈X
f (x)
s.t. qN,h,�1−α (C(x)) ≤ 0
q(Ĉ(x)) := qN,h,�1−α (Ĉ(x))
Note: Constraint no longer differentiable
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Joint Chance Constraints
I Original Problem
minx∈X
f (x)
s.t. Pξ[cj(x , ξ) ≤ 0, j = 1, . . . ,m] ≥ 1− α
I Scenario vector
C(x) = (c(x , ξ1), . . . , c(x , ξN))
I NLP formulation
minx∈X
f (x)
s.t. qN,h,�1−α (C(x)) ≤ 0
q(Ĉ(x)) := qN,h,�1−α (Ĉ(x))
Note: Constraint no longer differentiable
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Joint Chance Constraints
I Original Problem
minx∈X
f (x)
s.t. Pξ[ĉ(x , ξ) ≤ 0] ≥ 1− αĉ(x , ξ) := maxj cj(x , ξi )
I Scenario vector
C(x) = (c(x , ξ1), . . . , c(x , ξN))
I NLP formulation
minx∈X
f (x)
s.t. qN,h,�1−α (C(x)) ≤ 0
q(Ĉ(x)) := qN,h,�1−α (Ĉ(x))
Note: Constraint no longer differentiable
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Joint Chance Constraints
I Original Problem
minx∈X
f (x)
s.t. Pξ[ĉ(x , ξ) ≤ 0] ≥ 1− αĉ(x , ξ) := maxj cj(x , ξi )
I Scenario vector
Ĉ(x) = (ĉ(x , ξ1), . . . , ĉ(x , ξN))
I NLP formulation
minx∈X
f (x)
s.t. qN,h,�1−α (Ĉ(x)) ≤ 0
q(Ĉ(x)) := qN,h,�1−α (Ĉ(x))
Note: Constraint no longer differentiableAndreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Joint Chance Constraints
I Original Problem
minx∈X
f (x)
s.t. Pξ[ĉ(x , ξ) ≤ 0] ≥ 1− αĉ(x , ξ) := maxj cj(x , ξi )
I Scenario vector
Ĉ(x) = (ĉ(x , ξ1), . . . , ĉ(x , ξN))
I NLP formulation
minx∈X
f (x)
s.t. q(Ĉ(x)) ≤ 0q(Ĉ(x)) := qN,h,�1−α (Ĉ(x))
Note: Constraint no longer differentiableAndreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Exact Penalty Function
minx∈X
f (x)
s.t. q(Ĉ(x)) ≤ 0
I Exact penalty function:
φρ(x) = f (x) + ρmax{q(Ĉ(x)), 0} (+penalty for X )
I Model of scenario vector Ĉ(x) = (. . . ,maxj cj(x , ξi ), . . .)
mĈ (xk , d) =(. . . ,max
j{cj(xk , ξi ) +∇cj(xk , ξi )T d}, . . .
)I Model of penalty function
mφ(xk , d) =f (xk) +∇f (xk)T d + 12dT Bkd+
ρmax{q(Ĉ(xk)) +∇q(Ĉ(xk))T (mĈ (xk , d)− Ĉ(xk)), 0}
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Exact Penalty Function
minx∈X
f (x)
s.t. q(Ĉ(x)) ≤ 0
I Exact penalty function:
φρ(x) = f (x) + ρmax{q(Ĉ(x)), 0} (+penalty for X )I Model of scenario vector Ĉ(x) = (. . . ,maxj cj(x , ξi ), . . .)
mĈ (xk , d) =(. . . ,max
j{cj(xk , ξi ) +∇cj(xk , ξi )T d}, . . .
)
I Model of penalty functionmφ(xk , d) =f (xk) +∇f (xk)T d + 12d
T Bkd+ρmax{q(Ĉ(xk)) +∇q(Ĉ(xk))T (mĈ (xk , d)− Ĉ(xk)), 0}
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Exact Penalty Function
minx∈X
f (x)
s.t. q(Ĉ(x)) ≤ 0
I Exact penalty function:
φρ(x) = f (x) + ρmax{q(Ĉ(x)), 0} (+penalty for X )I Model of scenario vector Ĉ(x) = (. . . ,maxj cj(x , ξi ), . . .)
mĈ (xk , d) =(. . . ,max
j{cj(xk , ξi ) +∇cj(xk , ξi )T d}, . . .
)I Model of penalty function
mφ(xk , d) =f (xk) +∇f (xk)T d + 12dT Bkd+
ρmax{q(Ĉ(xk)) +∇q(Ĉ(xk))T (mĈ (xk , d)− Ĉ(xk)), 0}
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Trust Region S`1QP Algorithm
mφ(xk , d) =f (xk) +∇f (xk)T d + 12dT Bkd+
ρmax{q(Ĉ(xk)) +∇q(Ĉ(xk))T (mĈ (xk , d)− Ĉ(xk)), 0}
I Can show: |φρ(xk)−mφ(xk , d)| = O(‖d‖2)I Standard S`1QP-type trust region algorithm can be applied
I QP Subproblem
mind ,z,t
f (xk) +∇f (xk)T d + 12dT Bkd + ρ t
s.t. q(Ĉ(xk)) +∇q(Ĉ(xk))T Z ≤ t [Z = (z1, . . . , zn)]cj(xk , ξi ) +∇cj(xk , ξi )T d ≤ zi for all i , j
t ≥ 0, xk + d ∈ X̃ , ‖d‖∞ ≤ ∆k
I Size of QP proportional to number of nonzeros in ∇q(Ĉ(xk)), not N
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Trust Region S`1QP Algorithm
mφ(xk , d) =f (xk) +∇f (xk)T d + 12dT Bkd+
ρmax{q(Ĉ(xk)) +∇q(Ĉ(xk))T (mĈ (xk , d)− Ĉ(xk)), 0}
I Can show: |φρ(xk)−mφ(xk , d)| = O(‖d‖2)I Standard S`1QP-type trust region algorithm can be appliedI QP Subproblem
mind ,z,t
f (xk) +∇f (xk)T d + 12dT Bkd + ρ t
s.t. q(Ĉ(xk)) +∇q(Ĉ(xk))T Z ≤ t [Z = (z1, . . . , zn)]cj(xk , ξi ) +∇cj(xk , ξi )T d ≤ zi for all i , j
t ≥ 0, xk + d ∈ X̃ , ‖d‖∞ ≤ ∆k
I Size of QP proportional to number of nonzeros in ∇q(Ĉ(xk)), not N
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints
Summary
I Goal: Formulate chance-constrained problem as NLPI Constrain the quantile q1−α(x) ≤ 0
I Better linearization than p(x) ≥ (1− α)I Sample-based empirical quantile
I No need to know probability distributionI Variance reduction using kernel
I Improves approximation of feasible region, “smoother boundary”I Quantile estimates using smoothed empirical cdf
I Constraint becomes differentiableI Joint chance constraints
I Exact merit functionI S`1QP-type trust region algorithm
Andreas Wächter Nonlinear Programming Formulation of Chance-Constraints