LCoP Part V – Modeling and Applications€¦ · Modeling and Applications Content Weber Problem Matrix Optimization Approximating Solutions of Linear Equations Minimum of a Univariate

LCoP Part V – Modeling and Applications

Shu-Cherng Fang

Department of Industrial and Systems EngineeringGraduate Program in Operations Research

North Carolina State University

Conic Programming 1 / 34

Modeling and Applications

Content• Weber Problem• Matrix Optimization• Approximating Solutions of Linear Equations• Minimum of a Univariate Polynomial of Degree 2n

• Stochastic Queue Location Problem• Conic Reformulations of QCQP and Extensions• Robust Optimization


Weber Problem

In 1909, the German economist Alfred Weber introduced the problem offinding a best location for the warehouse of a company, such that the totaltransportation cost to serve the customers is minimum. Suppose thatthere are m customers needing to be served. Let the location ofcustomer i be ai ∈ R2, i = 1, . . . ,m. Suppose that customer may havedifferent demands, to be translated as weight ωi for customer i,i = 1, . . . ,m. Denote the desired location of the warehouse to be x.Then, the optimization problem becomes

Minm∑i=1

ωiti

s.t.

[x− aiti

]∈ L3, i = 1, . . . ,m


Matrix Optimization

Given A0, A1, . . . , Am, determine if there is y ∈ Rm such that

A0 +∑mi=1 yiAi � 0

which is equivalent to minimizing

λmax(A0 +∑mi=1 yiAi)

Notice that

t ≥ λmax(A0 +∑mi=1 yiAi) ⇔ tI −A0 −

∑mi=1 yiAi � 0

Equivalent Problem (SDP)

Min ts.t. tI −A0 −

∑mi=1 yiAi � 0

t ∈ R, y ∈ Rm


Approximating Solutions of LinearEquations

Given A ∈ Rm×n, b ∈ Rm, solve

Ax = b

Approximation:• l1 norm:

minx‖Ax− b‖1 ⇔ min

∑mi=1 ti

s.t. −ti ≤ Ai·x− bi ≤ ti, i = 1, . . . ,m.

• l2 norm:

minx‖Ax− b‖2 ⇔

min t

s.t.

[Ax− b

t

]∈ Lm+1


Approximating Solutions of LinearEquations

• l∞ norm:

minx‖Ax− b‖∞ ⇔ min t

s.t. −t ≤ Ai·x− bi ≤ t, i = 1, . . . ,m.

• Logarithm approximation: (b >Rm+

0)

minx

max1≤i≤m

| log(Ai·x)−log bi| ⇔min ts.t. 1/t ≤ Ai·x/bi ≤ t, i = 1, . . . ,m,

t > 0

⇔

min t

s.t.

t−Ai·x/bi 0 00 Ai·x/bi 10 1 t

� 0


Minimum of a Univariate Polynomial

Consider the problem of finding the minimum of a univariate polynomialof degree 2n:

Min x2n + a1x2n−1 + · · ·+ a2n−1x+ a2n

s.t. x ∈ R

This problem is equivalent to

Max ts.t. x2n + a1x

2n−1 + · · ·+ a2n−1x+ a2n − t ≥ 0 for all x ∈ R


Minimum of a Univariate Polynomial

It is well known that a univariate polynomial is nonnegative over the realdomain if and only if it can be written as sum of squares (SOS), which isequivalent to saying that there must be a positive semidefinite matrixX ∈ Sn+1

+ such that

x2n+a1x2n−1+· · ·+a2n−1x+a2n−t = (1, x, x2, · · · , xn)X(1, x, x2, · · · , xn)T .

Hence, this problem can be cast as an SDP:

Max ts.t. a2n − t = X11

a2n−k =∑

i+j=k+2

Xij , k = 1, . . . , 2n− 1

X(n+1),(n+1) = 1X ∈ Sn+1

+


Stochastic Queue Location Problem

BackgroundSuppose there are m potential customers to serve in the region.Customers’ demands are random, and once a customer calls for service,then the server in the service center will need to go to the customer toprovide the required service. In case the server is occupied, then thecustomer will have to wait. The goal is to find a good location for theservice center in order to minimize the expected waiting time of service.



Assumptions and notationsSuppose that the service calls from the customer are identicallyindependent distributed, and the demand process follows the Poissondistribution with overall arrival rate λ, and the probability that any servicecall is from customer i is assumed to be pi for i = 1, . . . ,m. The queueingprinciple is First Come First Service, and there is only one server in theservice center. This model can be regarded as M/G/1 queue as inQueueing theory, and the expected service time, including waiting timeand traveling, can be explicitly computed. To this end, denote the velocityof the server to be v, and the location of customer i is ai, i = 1, . . . ,m,and the location of the service center to be x.



Problem formulationThe expected waiting time for customer i is given by

ωi(x) =

(2λ/v2)m∑j=1

pj‖x− aj‖22

1− (2λ/v)m∑j=1

pj‖x− aj‖22+

1

v‖x− ai‖2,

where the first term in the expected term is the expected waiting time forthe server to be free and the second the term is the waiting time for theserver to travel after his departure at the service center.



Observing the fact that

‖x‖22/s ≤ t, s > 0⇐⇒∣∣∣∣∣∣∣∣[ x

t−s2

]∣∣∣∣∣∣∣∣2

≤ t+ s

2

We can formulate this problem as an SOCP:

Min (2mλ/v2)m∑i=1

piti + (1/v)m∑i=1

t0i

s.t.

[x− ait0i

]∈ L3,

x− aiti−s2

ti+s2

∈ L4, i = 1, ...,m

s ≤ 1− (2λ/v)m∑i=1

pisi,

[x− aisi

]∈ L3, i = 1, ...,m.


Quadratically Constrained QuadraticProgramming (QCQP)

QCQP:

min xTA0x+ 2bT0 x+ c0

s.t. xTAix+ 2bTi x+ ci ≤ 0, i = 1, . . . ,m,

Ai ∈ Sn, bi ∈ Rn, ci ∈ R, i = 0, 1, . . . ,m.

Homogeneous QCQP(HQCQP):

min xTA0x

s.t. xTAix+ ci ≤ 0, i = 1, . . . ,m


Convex QCQP

Convex QCQP: A0, . . . , Am � 0Assume rank(Ai) = di, then Ai = BTi Bi, Bi ∈ Rdi×n, i = 0, . . . ,m.

xTAix+ 2bTi x+ ci ≤ 0 ⇔

Bix−1/2− bTi x− ci/21/2− bTi x− ci/2

∈ Ldi+2

Reformulating Convex QCQP as SOCP:

min u

s.t.

B0x−1/2− bT0 x+ u/2− ci/21/2− bT0 x+ u/2− ci/2

∈ Ld0+2

Bix−1/2− bTi x− ci/21/2− bTi x− ci/2

∈ Ldi+2 i = 1, . . . ,m


SDP Relaxation for QCQP

Recall that

xTAix+ 2bTi x+ ci =

[1x

]T [ci bTibi Ai

] [1x

]=

[ci bTibi Ai

]•[

1 xT

x xxT

]SDP Relaxation

min

[c0 bT0b0 A0

]• Y

s.t.

[ci bTibi Ai

]• Y ≤ 0, i = 1, . . . ,m

Y11 = 1Y � 0


SDP Relaxation for HQCQP

Recall thatxTAix = Ai • xxT

SDP Relaxation

min A0 •Xs.t. Ai •X ≤ −ci, i = 1, . . . ,m

X � 0


Exact Solutions from SDP Relaxation

TheoremWhen m = 1, if the optimal solution of the SDP relaxation for QCQPexists, then there exists a rank one optimal solution Y ∗ of the SDP

relaxation for QCQP. Furthermore, let Y ∗ =

[1 (x∗)T

x∗ x∗(x∗)T

], then x∗ is

optimal for QCQP.



LemmaLet G be a symmetric matrix and X be a positive semidefinite matrix withrank d. Suppose that G •X ≤ (=)0. Then there exists a rank-onedecomposition X =

∑di=1 xix

Ti such that xTi Gxi ≤ (=)0, i = 1, . . . , d.

TheoremWhen m = 2, suppose there exists x such that xTAix+ 2bTi x+ ci < 0,i = 1, 2, and there exists y1 ≥ 0, y2 ≥ 0 such that[c0 bT0b0 A0

]+ y1

[c1 bT1b1 A1

]+ y2

[c2 bT2b2 A2

]� 0. If, for the optimal solution Y ∗,

either[c1 bT1b1 A1

]• Y ∗ < 0 or

[c2 bT2b2 A2

]• Y ∗ < 0, then there is no gap

between QCQP and its SDP relaxation and at least one optimal solutioncan be obtained from the rank-one decomposition of Y ∗.



Theorem(HQCQP)When m = 2, suppose there exists x, such that xTAix+ ci < 0, i = 1, 2,and there exists y1 ≥ 0 and y2 ≥ 0 such that A0 + y1A1 + y2A2 � 0. Thenthere is no gap between HQCQP and its SDP relaxation and at least oneoptimal solution can be obtained from the rank-one decomposition of X∗.


One Extension of QCQP

Ext-QCQP

min xTA0x+ 2bT0 x+ c0

s.t. xTAix+ 2bTi x+ ci ≤ 0, i = 1, . . . ,m,(x21, . . . , x

2n)T ∈ X

(Ext-QCQP)

where X is a closed convex set.SDP Relaxation:

min

[c0 bT0b0 A0

]• Y

s.t.

[ci bTibi Ai

]• Y ≤ 0, i = 1, . . . ,m

(Y22, Y33, . . . , Yn+1n+1)T ∈ XY11 = 1Y � 0



TheoremSuppose there is σ ∈ {−1, 1}n+1, such that all the off-diagonal elements

of Λσ

[ci bTibi Ai

]Λσ, 0 ≤ i ≤ m, are nonpositive. If Y ∗ is an optimal solution

for the SDP relaxation for Ext-QCQP, then

x∗ = σ1(σ2√Y ∗22, σ3

√Y ∗33, . . . , σn+1

√Yn+1n+1)T

is optimal for Ext-QCQP.


More Results on Ext-QCQP

A Special Case of Ext-QCQP

min xTAx

s.t. (x21, . . . , x2n)T ∈ X

where X is a closed convex set.SDP Relaxation:

min A •Xs.t. (X11, X22, . . . , Xnn)T ∈ X

X � 0


Approximation by Randomized Algorithm

• Suppose Z � 0, Zii = 1, i = 1, . . . , n. If ξ ∼ N(0, Z) andσ ∈ {−1, 1}n with

σi =

{1 ξi ≥ 0−1 ξi < 0

then E(σσT ) = 2π arcsinZ.

• Suppose Z � 0, Zii = 1, i = 1, . . . , n. Then arcsinZ � Z � 0.• 2

π arcsin t ≤ 1− α+ αt, ∀t ∈ [−1, 1] with α = 0.87856 . . ..



Given X � 0, let Z = D+XD+ + D, where D, D+, D are diagonalmatrices defined by

Dii =√Xii, D+

ii =

{(√Xii)

−1 Xii 6= 00 Xii = 0

, Dii =

{0 Xii 6= 01 Xii = 0

• X = DZD

• E(A • (DσσTD)) = A • (DE(σσT )D) = A • ( 2πD arcsin(Z)D).



TheoremSuppose A � 0, and X∗ is the minimizer of SDP relaxation. Let Z, D,D+, and D be define from X∗, and ξ ∼ N(0, Z) with corresponding σ.Then

E(A • (DσσTD)) ≤ 2

πmin{xTAx|(x21, . . . , x2n)T ∈ X}

Proof:E(A • (DσσTD))

= A • ( 2πD arcsin(Z)D)

≤ A • ( 2πDZD)

= 2πA •X

∗

≤ 2π min{xTAx|(x21, . . . , x2n)T ∈ X}



TheoremAssume that A � 0, Aij ≥ 0 for all i 6= j, and X∗ is the minimizer of SDPrelaxation. Let Z, D, D+ and D be defined from X∗ and ξ ∼ N(0, Z) withcorresponding σ. Then

E(A • (DσσTD)) ≤ αmin{xTAx|(x21, . . . , x2n)T ∈ X}

Proof:E(A • (DσσTD))

= A • ( 2πD arcsin(Z)D)

=∑i 6=j AijDiiDjj(

2π arcsinZij) +

∑ni=1AiiD

2ii

≤∑i 6=j AijDiiDjj(1− α+ αZij) +

∑ni=1AiiD

2ii

= (1− α)∑i,j AijDiiDjj + αA •X∗

≤ αA •X∗ ≤ αmin{xTAx|(x21, . . . , x2n)T ∈ X}Conic Programming 26 / 34

Example: MAX CUT Problem

Given a graph G = (V,E,W ), find an optimal partition of the node set intotwo subsets V1 and V2 such that the weighted cut is maximized.

max∑ni=1

∑nj=1 wij

1−xixj

2

s.t. x2i = 1, i = 1, . . . , n(MAX − CUT )

Reformulation as Ext-QCQP:

Aij =

{wij i 6= j

−∑k 6=i wik i = j

, X = {e}

TheoremIf wij ≥ 0, ∀i 6= j. Then the expected value of randomized algorithm is atleast α ≈ 0.878 times the value of the maximum cut.


Robust Optimization

min f0(x)

s.t. fi(x, ui) ≤ 0

∀ui ∈ Ui, i = 1, . . . ,m.

Motivation

• The parameters are inexact.• The parameters cannot be foreseen.• The parameters may vary with time and other environment factors.


Example: Robust Linear Program

min cTx

s.t. aTi x ≤ bi

∀ai ∈ {a0i +∑pj=1 uja

ji | ‖u‖2 ≤ 1},

i = 1, . . . ,m

For any x

maxai aTi x = maxu(a0i )

Tx+∑pj=1 uj(a

ji )Tx

= (a0i )Tx+ ‖((a1i )Tx, . . . , (a

pi )Tx)T ‖2


Example: Robust Linear Program

Therefore

aTi x ≤ bi,∀ai ⇔ ‖((a1i )Tx, . . . , (api )Tx)T ‖2 ≤ bi − (a0i )

Tx

Equivalent SOCP

min cTx

s.t.

(a1i )

Tx...

(api )Tx

bi − (a0i )Tx

∈ Lp+1, i = 1, . . . ,m


Robust Convex QCQP

min fTx

s.t. xTATAx ≤ 2bTx+ c

∀(A, b, c) ∈ {(A0, b0, c0) +∑pj=1 uj(A

j , bj , cj) | ‖u‖2 ≤ 1},

Let U(x) = (A0x,A1x, . . . , Apx) and

V (x) =

c0 + 2(b0)Tx 1

2c1 + (b1)Tx · · · 1

2cp + (bp)Tx

12c

1 + (b1)Tx 0...

. . .12cp + (bp)Tx 0


Robust Convex QCQP

xTATAx ≤ 2bTx+ c

⇔[

1u

]TUT (x)U(x)

[1u

]−[

1u

]TV (x)

[1u

]≤ 0,∀‖u‖2 ≤ 1

Lemma (S-lemma)Let P and Q be symmetric matrices and yTPy > 0 for some y. Then theimplication

zTPz ≥ 0 ⇒ zTQz ≥ 0

is valid if and only if Q− λP � 0 for some λ ≥ 0.


Robust Convex QCQP

From S-lemma,[1u

]TUT (x)U(x)

[1u

]−[

1u

]TV (x)

[1u

]≤ 0,∀‖u‖2 ≤ 1

⇔ V (x)− UT (x)U(x)− λ[1−I

]� 0,∃λ ≥ 0

⇔

V (x)− λ[1−I

]UT (x)

U(x) I

� 0,∃λ ≥ 0

(by Schur complementary theorem)


Robust Convex QCQP

Equivalent SDP

min fTx

s.t.

c0 + 2(b0)Tx− λ 1

2c1 + (b1)Tx · · · 1

2cp + (bp)Tx (A0x)T

12c

1 + (b1)Tx λ (A1x)T

.... . .

...12cp + (bp)Tx λ (Apx)T

A0x A1x · · · Apx I

� 0


Documents

LCoP Part V – Modeling and Applications€¦ · Modeling and Applications Content Weber Problem Matrix Optimization Approximating Solutions of Linear Equations Minimum of a Univariate