Report_Convex approximations in stochastic programming by semidefinite programming

8/6/2019 Report_Convex approximations in stochastic programming by semidefinite programming

1/47

Convex approximations in stochastic

programming by semidefiniteprogramming

Istvan Deak, Imre Polik, Andras

Prekopa, Tamas Terlaky


2/47


3/47

Outline

Stochastic Programming

Semidefinite Programming

Basic Models of Stochastic Programming Proposed Method

Experimental Results

Conclusion


4/47


A framework for modeling optimization

problems that involve uncertainty.

Real world problems almost invariably include

some unknown parameters.

Behavioural Ecology

Transportation

Economic Application


5/47

Example


6/47

You can not know the blue line.


7/47

But you know that probability distribution of the

noise data as a prior term.


8/47

Try to find a function to approximate the sample

data with the prior term.


9/47

Outline



Basic Models of Stochastic Programming

Proposed Method


Conclusion


10/47

Semidefinite Programming(SDP)

One minimizes a linear function subject to the

constraint that an affine combination of

symmetric matrices is positive semidefinite.


11/47

General Form of SDP

matricessymmetric1andtorcolumn vecaGiven nnmRcm

v

Find,...,, .10nn

FFFv

X

T

xcmin

subject to],...,,[for 21mT

m Rxxxx !

!

um

i

piiFxF

1

0 0

it.minimizetry toand

.0subject to),...,,,(SDPaiven1

010n !

um

i

piim FxFFFFc


12/47

Example

}1,0|{isregionfeasibletheThus

.enonnegativare1andbothtesemidefiniis

1

1thatNote

}01

1|{

}010

00

00

01

01

10|{isregionfeasibleThe

)10

00,

00

01,

01

10,

1

0(SDPConsider

211

2

1

211

2

1

2

1

2

1

21

2

1

2

uu

u

!

u

xxxx

x

xxx

x

x

x

x

x

x

xxx

x

p

p


13/47

Linear Programming is also a SDP

problem.

!

u

!!!

!

u

m

i

pii

ii

FxF

miadiagFbdiagF

bAxdiagxF

SDP

bAx

xc

LP

T

1

0

0

0

,...,1),(),(

)()(

:

0subject to

minimize

:


14/47


15/47

0

4000

0300

00200001

44000

03400

00240

00014

4

43000

03300

00230

00013

3

42000

03200

00220

00012

2

41000

03100

00210

00011

x1subject to

u

b

b

b

b

a

a

a

a

x

a

a

a

a

x

a

a

a

a

x

a

a

a

a


16/47

Goal

Given the data

xi, i, i = 1, , N

Try to determine an optimal function in some

sense by semidefinite programming to

approximate those.


17/47

Outline




Proposed Method


Conclusion


18/47


19/47

Basic Models of

Stochastic Programming(2/2)


20/47

Outline




Proposed Method


Conclusion


21/47

Proposed Method(1/2)


22/47

Proposed Method(2/2)

To solve the quadratic regression problem, to solve

the unconstrained optimization problem

The system may be over or underdetermined, so it

will use least-squares(LS) approach to solve.


23/47

Problem of least-squares approach(1/3)


24/47


Even if the points are in general position and

the function to approximate is convex, The

approximation may not be convex.


25/47



26/47

Semidefinite optimization


27/47

Quadratic Problem to SDP

t

xd

xc

bx

tt

bx

xd

xc

T

T

T

T

e

u

u

2

2

)(

0sub ect to

bound)upperanis(minimize

:

0sub ect to

)(minimize

:Q


28/47

0

0

0

00)(

subject to

bound)upperanis(minimize

u

xdxc

xct

bAxdiag

tt

TT

T


29/47


30/47

Assume the norm is maximum distance norm, theoptimal approximation is a solution of thefollowing problem

Pu

re semidefinite optimization problem. N nonnegative variables.

2N linear inequalities.


31/47


32/47

We can transform this linear programming to

SDP. So it is pure semidefinite optimization

problem.


33/47

An equivalent formulation

A mixed second-order-semidefinite optimization problem.

N linear equalities.


34/47

Outline





Conclusion


35/47

Experimental Results(1/11)

Matlab code with SeDuMi and Yalmip

F: constraints

OBJ: equations to be solved


36/47


Running times of different sizes

n: dimensions

N: # points in x

2-norm costs little memory

and running time


37/47


The increase of N is less sensitive to the

increase of n

29

1.6


38/47

Success rates for different types of datasets

Cube, Ball, Ball-noisy

Degenerate

Ill-conditioned: Degenerate + noise

Degenerate

The points span a lower dimensional subspace

The dimension of the subspace is roughly half

of the dimension of the space

Examples:Degenerate CircleRadius=0 A single point

Radius=Inf A straight line


Least square method

A OT handle degenerate

or ill-conditioned datasets


39/47


40/47


Success rates for different functions

Using: uniformly distributed unit-ball

General: positive semidefinte quadratic function

Definite: strictly convex quadratic function

Deficient: not strictly convex, some values equal 0

Four methodswork well


41/47


42/47


Matlab code we use

M

atlabM

atlab screenshotscreenshotample Resultsample Results

Error value

ost time


43/47


Running time (Using PC)

n N 1-norm 2-norm Inf -norm

2 6 0.062 0.046 0.0314 15 0.063 0.031 0.063

8 45 0.094 0.063 0.078

16 153 0.578 0.359 0.063

32 561 20.609 6.547 21.828

64 2145 NaN NaN NaN

2-norm is the fastest


44/47


Error values (Using PC)

n N 1-norm 2-norm Inf -norm

2 6 6.61E-09 8.77E-09 9.46E-11

4 15 1.85E-07 3.76E-09 9.48E-08

8 45 1.09E-06 1.89E-07 4.77E-09

16 153 2.75E-04 3.15E-08 5.85E-0732 561 2.59E-02 4.95E-07 9.02E-07

64 2145 NaN NaN NaN

Error is almost

equals to zero

n N 1-norm 2-norm Inf -norm LS

2 6 6.61E-09 8.77E-09 9.46E-11 358.0074


45/47


46/47

Outline





Conclusion


47/47

Conclusion

Least-squares approximation works well onlywhen the quadratic function is strictly convexand the noise is fairly small

The most efficient method is L2-normapproximation

Our experiments(using Matlab): Ifusing randomly generated datasets, the error ofLS

would be very large

L1-norm might have apparent error when n is fixedand N is large

Documents

Report_Convex approximations in stochastic programming by semidefinite programming