Semidefinite ProgrammingSemidefinite Programming

Based on slides by Based on slides by Uri ZwickUri Zwick

Lecture 8Lecture 8::

Magnus M. HalldorssonMagnus M. Halldorsson

Outline of talkOutline of talk

Semidefinite programming

MAX CUT (Goemans, Williamson ’95)

MAX 2-SAT and MAX DI-CUT (FG’95, MM’01, LLZ’02)

MAX 3-SAT (Karloff, Zwick ’97)

-function (Lovász ’79)

MAX k-CUT (Frieze, Jerrum ’95)

Colouring k-colourable graphs (Karger, Motwani, Sudan ’95)

Positive Semidefinite MatricesPositive Semidefinite Matrices

A symmetric nn matrix A is PSDPSD iff:

• xTAx 0 , for every xRn.

• A=BTB , for some mn matrix B.

• All the eigenvalues of A are non-negative.

Notation: A 0 iff A is PSD

Linear Linear ProgrammiProgrammi

ngngmax c x

s.t. ai x bi

x 0

Semidefinite Semidefinite ProgrammingProgramming

max CX

s.t. Ai X bi

X 0

Can be solved exactly

in polynomial time

Can be solved almost exactly

in polynomial time

LP/SDP algorithmsLP/SDP algorithms

• Simplex method (LP only)

• Ellipsoid method

• Interior point methods

Semidefinite Semidefinite ProgrammingProgramming(Equivalent formulation)

max cij (vi vj)

s.t. aij(k) (vi vj) b(k)

vi Rn

X ≥ 0 iff X=BTB. If B = [v1 v2 … vn] then xij = vi · vj .

Lovász’s Lovász’s -function-function(one of many formulations)

max JX

s.t. xij = 0 , (i,j)E

I X = 1

X 0

Orthogonal representation

of a graph:vi vj = 0 ,

whenever (i,j)E

The Sandwich TheoremThe Sandwich Theorem(Grötschel-Lovász-Schrijver ’81)

)G()G()G(

Size of max clique

Chromaticnumber

The The MAX CUTMAX CUT problem problem

Edges may be weighted

The The MAX CUTMAX CUT problem: problem:

motivationmotivation Given: n activities, m persons.

Each activity can be scheduled either in the morning or in the afternoon.

Each person interested in two activities.

Task: schedule the activities to maximize the number of persons that can enjoy both activities.

If exactly n/2 of the activities have to be held in the morning, we get MAX BISECTIONMAX BISECTION..

The The MAX CUTMAX CUT problem: problem: statusstatus

• Problem is NP-hard

• Problem is APX-hard (no PTAS unless P=NP)

• Best approximation ratio known, without SDP, is only ½. (Choose a random cut…)

• With SDP, an approximation ratio of 0.878 can be obtained! (Goemans-Williamson ’95)

• Getting an approximation ratio of 0.942 is NP-hard! (PCP theorem, …, Håstad’97)

A quadratic integer A quadratic integer programming formulation of programming formulation of

MAX CUTMAX CUT

}1,1{ s.t.2

1Max

i

jiij

x

xxw

An SDP Relaxation of An SDP Relaxation of MAX MAX CUTCUT

(Goemans-Williamson ’95)

1||||, s.t.

2

1Max

in

i

jiij

vRv

vvw

An SDP Relaxation of An SDP Relaxation of MAX CUT – MAX CUT –

Geometric intuitionGeometric intuition

Embed the vertices of the graph on the unit sphere such that vertices that are joined by edges are far apart.

Random hyperplane Random hyperplane roundingrounding

(Goemans-Williamson ’95)(Goemans-Williamson ’95)

To choose a random hyperplane,

choose a random normal vector

r

If r = (r1 , r2 , …, rn), andr1, r2 , … , rn N(0,1), then the direction of r

is uniformly distributed over the n-dimensional

unit sphere.

The probability that two vectors are separated

by a random hyperplane

vi

vj

Analysis of the Analysis of the MAX CUTMAX CUT Algorithm Algorithm (Goemans-Williamson (Goemans-Williamson

’95)’95)

1

1

1

1

exp

sdp

ratio min

1

2

2

1

cos

0.8785

( )

cos (.

)6. .

ii

jj

ijj

x

i

w

v v

x

v

w

v

x

Is the analysis tight?Is the analysis tight?

Yes!Yes!

(Karloff ’96) (Feige-Schechtman ’00)

The The MAX Directed-CUTMAX Directed-CUT problemproblem

Edges may be weighted

The The MAX 2-SATMAX 2-SAT problem problem

747354

435362

435221

xxxxxx

xxxxxx

xxxxxx

A Semidefinite A Semidefinite Programming Relaxation Programming Relaxation

of of MAX 2-SATMAX 2-SAT(Feige-Lovász ’92, Feige-Goemans ’95)

1

1

201s.t.4

3Max 00

||||,

,

,,,

in

i

iin

kjkiji

jijiij

vRv

nivv

nkjivvvvvv

vvvvvvw

Triangle constraints

The probability that a clause xi xj is satisfied is :

2ijj0i0

ij

iv

jv

0v

Approximability and Approximability and Inapproximability resultsInapproximability results

ProblemApprox.

RatioInapprox.

RatioAuthors

MAX CUT0.87816/17 0.941

Goemans Williamson ’95

MAX DI-CUT0.87412/13 0.923

GW’95, FW’95 MM’01, LLZ’01

MAX 2-SAT0.94121/22 0.954

GW’95, FW’95 MM’01, LLZ’01

MAX 3-SAT7/87/8Karloff

Zwick ’97

What else can we What else can we do with do with SDPSDPs?s?

• MAX BISECTION MAX BISECTION (Frieze-Jerrum ’95)

• MAX MAX kk-CUT-CUT (Frieze-Jerrum ’95)

• (Approximate) Graph colouring (Karger-Motwani-Sudan’95)

(Approximate) Graph (Approximate) Graph colouringcolouring

• Given a 3-colourable graph, colour it, in polynomial time,

using as few colours as possible.• Colouring using 4 colours is still NP-hard. (Khanna-Linial-Safra’93 Khanna-Guruswami’01)

• A simple combinatorial algorithm can colour, in polynomial time, using about n1/2 colours.

(Wigderson’81)

• Using SDP, can colour (in poly. time) using n1/4 colours (KMS’95), or even n3/14 colours (BK’97).

Vector Vector kk-Coloring-Coloring((Karger-Motwani-Sudan ’95)

A vector k-coloring of a graph G = (V,E) is a sequence of unit vectors v1 , v2 , … , vn

such that if (i,j)E then vi · vj = -1/(k-1).

The minimum k for which G is vector k-colorable is ( )G

A vector k-coloring, if one exists, can be found using SDP.

Lemma: If G = (V,E) is k-colorable, then it is also vector k-colorable.

Proof: There are k vectors v1 ,v2 , … , vk

such that vi · vj = -1/(k-1), for i ≠ j.

k = 3 :

Finding large independent Finding large independent setssets

((Karger-Motwani-Sudan ’95)Let r be a random normally distributed vector in Rn. Let .

I’ is obtained from I by removing a vertex from each edge of I.

lnlnln 31

32c

}|{ crvViI i

Constructing a large Constructing a large ISIS

riv

jv

Colouring Colouring kk-colourable -colourable graphsgraphs

Colouring k-colourable graphs using min{ Δ1-2/k , n1-3/(k+1) } colours.

(Karger-Motwani-Sudan ’95)

Colouring 3-colourable graphs using n3/14 colours.

(Blum-Karger ’97)

Colouring 4-colourable graphs using n7/19 colours.

(Halperin-Nathaniel-Zwick ’01)

Open problemsOpen problems

• Improved results for the problems considered.

• Further applications of SDP.