A Cutting Plane SDP Method

for Maxcut Problems

John E. Mitchell

Department of Mathematical Sciences

Rensselaer Polytechnic Institute

Troy, NY 12180 USA

mitchj@rpi.edu

http://www.rpi.edu/˜mitchj

Joint work with Kartik Krishnan.

MOPTA 02

McMaster University, Hamilton

3 August 2002

SDP for Maxcut 1

Abstract

We investigate solution of the maximum cut problem

using a semidefinite programming approach. The dual of

the well-known SDP relaxation of maxcut is formulated

as a semi-infinite linear programming problem, which is

solved with a cutting plane algorithm. Cutting planes

based on the polyhedral theory of the maxcut problem

are then added to the primal problem in order to improve

the SDP relaxation. Computational results are provided.

SDP for Maxcut 2

Contents

1. Overview

2. SDP relaxation of Maxcut

3. Semidefinite programming

4. A semi-infinite linear program

5. Cutting plane algorithm for SDP

6. Maxcut problems

7. Computational results

8. Conclusions

SDP for Maxcut 3

1 Overview

• The Maxcut problem: divide the nodes of a graph

into two sets so as to maximize the weight of the

edges that have exactly one endpoint in each set.

• Maxcut can be relaxed to give a semidefinite

programming problem (SDP).

• An SDP is a convex programming problem.

• We approximately solve the dual of this problem

using a cutting plane approach.

• The SDP relaxation is then improved by adding

constraints to the primal problem.

• The process is repeated as needed.

SDP for Maxcut 4

2 SDP relaxation of Maxcut

Example: Each edge has weight one.Divide vertices into two setsto cut as many edges as possible

vA v A

v B

vB v A

HHHHHHHHHH

����������

����������������Q

QQ

QQ

QQ

QQ

QQ

QQ

QQ

There are many feasible solutions with value 4.

Optimal solution has value 5.

SDP for Maxcut 5

Formulation of Maxcut

• Given: graph G = (V, E) with edge weights wij.

• Let xi =

+1 if vertex i is on one side of cut

−1 if vertex i is on the other side

• Let L be the Laplacian matrix for the graph, so

Lij =

−wij if i 6= j∑

k 6=i wik if i = j

• The value of a cut is 14xTLx.

• Formulation of Maxcut:

max1

4xTLx subject to xi = ±1, for each vertex i.

SDP for Maxcut 6

SDP Formulation of Maxcut

• Formulation of Maxcut:

max1

4xTLx subject to xi = ±1, for each vertex i.

• Equivalently: setting X = xxT , we get

max 14trace(LX)

subject to Xii = 1 ∀i ∈ V

X is rank one and symmetric.

• Relax the rank-one requirement:

max 14trace(LX)

subject to Xii = 1 ∀i ∈ V

X is positive semidefinite

and symmetric.

• Optimal value of Maxcut is at least 0.878 of optimal

value of SDP relaxation.

• In practice, the relaxation can be tightened by

adding linear constraints.

Eg: triangle inequality in earlier example.

SDP for Maxcut 7

3 Semidefinite programming

Standard form:

min C • X

subject to A(X) = b (SDP )

X � 0,

with dual

max bTy

subject to ATy + S = C (SDD)

S � 0

• C, X and S are square symmetric matrices.

• Require X and S be positive semidefinite (psd).

• Inner products are the Frobenius inner product,

so C • X = trace(CX).

• AX =

A1 • X...

Ak • X

, ATy =

∑ki=1 yjAj, where each Aj

is a symmetric matrix.

SDP for Maxcut 8

Methods for solving SDPs

• Primal-dual interior point approaches are

limited in the size of problems they can solve.

• Alternatives include:

– The bundle method of Helmberg, Rendl, and

Kiwiel.

– The nonlinear programming approach of

Burer, Monteiro, and Zhang.

SDP for Maxcut 9

4 A semi-infinite linear program

• (SDP ) and (SDD) are convex programming

problems.

• Only nonlinearity is the positive semidefiniteness (PSD)

requirement.

• Replace PSD constraint by linear constraints.

��

��

��

��

����������

Feasible region

@

@

@

@

@

@

SDP for Maxcut 10

Dual formulation:

max bTy (LDD)

subject to dT (C −ATy)d ≥ 0 ∀d with ||d||2 = 1.

• If (SDP ) has k constraints, then (LDD) has k vari-

ables.

• Need to be selective with the vectors d included in

the constraints. For example, use a cutting plane

approach to select vectors d.

• Typically, we will need some vectors d that are dense,

leading to a dense linear programming problem.

• Have far fewer variables than in a primal formulation,

so get smaller linear programs (although they are still

dense).

SDP for Maxcut 11

Regaining a primal solution

• Because the linear programs are smaller, it was more

efficient to work with the dual formulation.

• We only use a finite number of vectors d, so we are

solving a relaxation of the dual:

max bTy (LDR)

subject to dTi (C −ATy)di ≥ 0

for vectors di, i = 1, . . . , m.

• The optimal value to (LDR) gives an upper bound

on the optimal value of (SDP ).

SDP for Maxcut 12

• (LDR) is a relaxation of (SDD). The dual linear

program to (LDR) is a constrained version of

(SDP ).

• (LDR) can be rewritten as

max bTy (LDR)

subject to∑k

j=1 yj(dTi Ajdi) ≤ dT

i Cdi

for vectors di, i = 1, . . . , m.

• Writing the dual to (LDR) directly gives

min∑m

i=1(dTi Cdi)xi

subject to∑m

i=1(dTi Ajdi)xi = bj for j = 1, . . . , k

x ≥ 0.

• This can be rewritten as

min C • (∑m

i=1 xididTi )

subject to A(∑m

i=1 xididTi ) = b (LPR)

x ≥ 0.

• Thus, any feasible solution to (LPR) gives a feasi-

ble solution to (SDP), X =∑m

i=1 xididTi

SDP for Maxcut 13

A perfect set of constraints

Let X∗ solve (SDP ) and y∗, S∗ solve (SDD).

• X∗ has an eigendecomposition:

X∗ = [P Q]

Λ 0

0 0

PT

QT

• X∗ and S∗ are simultaneously diagonalizable:

S∗ = [P Q]

0 0

0 Γ

PT

QT

• Let p1, . . . , pr be the columns of P . Then the optimal

solutions to

max bTy

subject to piT (C −ATy)pi ≥ 0, i = 1, . . . , r

and its dual give the optimal solutions to (SDD)

and (SDP ): set x = λ, where λ = diag(Λ).

• Pataki: expect r ≈ O(√

k).

SDP for Maxcut 14

5 Cutting plane algorithm for (SDP )

• Choose an initial set of constraints for (LDR).

• Solve (LDR) and (LPR) approximately using an

interior point method.

• Get trial point y.

• If C−AT y is not psd, find vectors d and correspond-

ing violated constraints dT (C −ATy)d ≥ 0.

• Modify (LDR) and (LPR), and repeat.

SDP for Maxcut 15

Finding violated constraints

• We have a vector y from (LDR).

• Calculate S = C −AT y.

• Solve the quadratic subproblem

mind dT Sd

subject to ||d|| ≤ 1

• Subproblem only needs to be solved approximately:

just need to find a solution with negative objective

function value to cut off the current solution y.

• Can use different norms:

– 2-norm: Optimal solution is the eigenvector with

most negative eigenvalue. Can use eigenvalue

schemes such as Lanczos to find eigenbases for

several negative eigenvalues simultaneously.

– ∞-norm: Solutions d may contain a number of

±1 components. Useful if we are solving a com-

binatorial optimization problem such as Maxcut.

SDP for Maxcut 16

Complexity of the algorithm

• Theoretically, could use a volumetric barrier cut-

ting plane algorithm to solve (SDD).

• Want to get within ε of optimality.

• Requires O(k log(1ε)) calls to the quadratic sub-

problem and a similar number of Newton steps.

• Each solution of the subproblem requires O(n3)

work if the QR algorithm is used to find the solutions

to the subproblems. It also requires O(kn2) work to

calculate S.

• Each Newton step in (LDR) requires O(k3) work.

• Thus, for k ≥ n, total number of arithmetic

operations is of the order of O(k4 log(1ε )).

• A primal dual SDP method requires O(kn3 +

k2n2 + k3) arithmetic operations at each iteration

and O(√

n log(1ε)) iterations. So overall complexity

is O(k2n2.5 log(1ε)) for n ≤ k ≤ n2. (Complexity can

be reduced slightly if constraint matrices have special

structure.)

SDP for Maxcut 17

Dimension of added face

Assume:

1. The constraints A(X) = b imply trace(X) = a for

some fixed a.

2. Given y and S = C −AT y, the vector d is an eigen-

vector of S corresponding to the most negative

eigenvalue λmin of S. Let p be the multiplicity of

this eigenvalue.

Then:

1. There exists y feasible in (SDD) satisfying the con-

straint dT (C −ATy)d ≥ 0 at equality.

2. The hyperplane dT (C−ATy)d = 0 induces a tangent

space of dimension at least k − p.

SDP for Maxcut 18

Constraint satisfied at equality:

• Since trace(X) = a, there exists y with AT y = I .

• Let y := y + λminy.

• Then S := C −AT y is psd, and dT Sd = 0.

��

��

��

��

��

��

�

Feasible region in y-spacefor (SDD)

v

yvy

�����y

ss

ss

s

dT (C −ATy)d =dT (C −AT y)d

dT (C −ATy)d = 0

SDP for Maxcut 19

Dimension of tangent space

��

��

��

��

��

��

�

Feasible region in y-spacefor (SDD)

vy

vy + εv v y + εv + ε2w

dT (C −ATy)d = 0

�A(ddT )

SDP for Maxcut 20

Tangent space:

• Move distance O(ε), still within O(ε2) of feasible re-

gion.

• Define eigendecomposition of S:

S = [P Q]

0 0

0 Γ

PT

QT

• If v satisfies PT (AT v)P = 0, then y + εv − βy is

feasible in (SDD), for some β that is O(ε2).

• The dimension of the set of such v is k − p, since P

has rank p.

• So, if the minimum eigenvalue has multiplicity one,

then the tangent space is full-dimensional.

SDP for Maxcut 21

6 Maxcut problems

Divide the vertices of a graph into two sets so that the

number of edges with one endpoint in each set is as large

as possible. SDP relaxation:

max L4• X

subject to diag(X) = e

X � 0,

with dual

min eTy

subject to −Diag(y) + S = −L4

S � 0

Here L = Diag(Ae) − A is the Laplacian matrix of the

graph, where A is the weighted adjacency matrix. Note

that trace(X) = n, the number of nodes in the graph.

SDP for Maxcut 22

Initial relaxation (LDR):

Set d = ei , ∀i = 1 . . . n, where ei is the ith standard

basis vector for IRn. This generates the constraint yi ≥Lii4 , ∀i = 1 . . . n.

Solve (SDD) approximately:

Use the cutting plane approach.

Add good cut vectors:

The LP relaxation can be improved by using heuristics

such as Goemans-Williamson rounding and Kernighan-

Lin local improvement to generate incidence vectors of

good cuts. These are added as dual constraints.

This allows the primal solution to use these good so-

lutions — recall X =∑

i xididiT .

Add primal cutting planes:

The SDP relaxation is improved by adding cutting

planes.

SDP for Maxcut 23

Primal cutting planes

Cycle inequalities: Given a cycle C and a subset F ⊆ C

of odd cardinality, we have the inequality:

∑

(i,j)∈C\FXij − ∑

(i,j)∈FXij ≤ |C| − 2

• Defines a facet if the cycle is chordless.

• Violated inequalities of this type can be found in

polynomial time using an algorithm due to Barahona

and Mahjoub.

• Heuristic routines can be used to speed up the search

for violated constraints.

SDP for Maxcut 24

Overview

PRIMAL SDP CONE

MAX CUTpolytope

odd cycleinequality

violated

b

c

de

a e

f

g

h

optimal solutioni

abcde : Initial LP relaxationabcgh : LP region after adding violated odd cyclesabefgh : after the interior point sdp cutting plane schemeMax Cut Polytope : Shaded in redi : Optimal integer cut vector

SDP for Maxcut 25

7 Computational results

Name n m Maxcut 4 SDP1 5 SDP2 6 UB 7

gpp100 1 100 269 210 221.69 211.27 212.56

gpp1241 1 124 149 137 141.91 137 137.45

gpp1242 1 124 318 256 269.64 257.66 258.60

gpp1243 1 124 620 446 467.68 458.35 458.90

gpp1244 1 124 1271 834 864.26 848.92

gpp2501 1 250 331 305 317.23 305 307.83

gpp2502 1 250 612 502 531.78 511.12 520.19

gpp5001 1 500 625 574 598.11 576.64 8 584.91

toruspm-8-50 2 512 1536 456 527.81 464.70 8

torusg3-8 2 512 1536 412.75 9 457.36 417.69 8 428.18

ising10 3 100 200 70 79.22 70 70.70

ising2013 400 800 282 314.25 282 288.04

ising2023 400 800 268 305.63 268 271.82

ising30 3 900 1800 630 10 709.00 632

0All results for 10 iterations of the SDP scheme1SDPLIB2DIMACS3Random Ising Spin glass problems (Mitchell)4Best integer solution from the SDP scheme (Optimal value unless stated)5Over the elliptope6Over the elliptope and odd cycles7Upper bounds from our scheme8Best results obtained using an interior point approach9Optimal value: 416.84

10Optimal value: 632

SDP for Maxcut 26

Settings for the computational results

• We ran 10 cutting plane iterations in all.

• In our LP subroutine for the SDP, we add 5 cutting

planes in each iteration.

• Our starting tolerance for the relative duality gap

is 1.

• This is lowered by 0.9 in each iteration

• Initially, we solve our SDP relaxations very cheaply,

i.e. perform only 5 LP cutting plane iterations, and

we increase this number if we are unable to improve

the best incidence cut vector.

• We add the 100 most violated odd cycle in-

equalities returned by the Barahona-Mahjoub ap-

proach in each iteration.

• Upper bounds obtained by solving the final pri-

mal SDP relaxation with the odd cycle inequalities

to optimality using SDPT3 (Toh et al).

SDP for Maxcut 27

Observations

Each iteration of the cutting plane approach is faster

than an interior point iteration, i.e. solve the SDP relax-

ations using a warm start procedure, and call the sepa-

ration oracle to find violated odd cycle inequalities. This

is especially true if the graph is sparse.

We were able to obtain the optimal integer solution

in most cases (except problems torusg3-8 and ising30).

The former problem is not easy, since the maxcut solution

is not integer, whereas the latter problem is currently as

big as we can possibly handle.

For most of the problems we do not have a proof of

optimality, since the SDP relaxation over the elliptope

and odd cycles is not enough, and there is yet some gap

involved.

SDP for Maxcut 28

8 Conclusions

1. Our scheme is entirely a polyhedral LP approach

— a cutting plane approach within another cutting

plane approach.

2. Can find the optimal maxcut incidence vector

quickly, but the certificate of optimality takes time.

3. Our technique for computing upper bounds per-

forms poorly — subject of ongoing research.

4. The main computational task in our approach is the

time taken by SDP separation oracle. Another

difficulty is that the oracle returns cutting planes that

lead to dense LP relaxations. Ongoing research: try

to use matrix completion ideas to speed up the

oracle and get sparser constraints.

5. Helmberg shows that better SDP relaxations can be

obtained by considering cycles that include edges

not in the original graph. (In the LP case,

adding these redundant edges does not improve the

relaxation.)

SDP for Maxcut 29

References

[1] S. Burer, R. D. C. Monteiro, and Y. Zhang. Solving a class of semidefinite

programs via nonlinear programming. Technical report, School of ISyE,

Georgia Tech, Atlanta, GA, September 1999.

[2] C. Helmberg. Semidefinite programming for combinatorial optimization.

Technical Report ZR-00-34, TU Berlin, Konrad-Zuse-Zentrum, Berlin,

October 2000. Habilitationsschrift.

[3] C. Helmberg. A cutting plane algorithm for large scale semidefinite re-

laxations. Technical Report 01–26, TU Berlin, Konrad-Zuse-Zentrum,

Berlin, October 2001.

[4] C. Helmberg and K. Kiwiel. A spectral bundle method with bounds. Tech-

nical Report SC-99-37, Konrad-Zuse-Zentrum fuer Informationstechnik,

Berlin, December 1999.

[5] C. Helmberg and F. Rendl. A spectral bundle method for semidefinite

programming. SIAM Journal on Optimization, 10(3):673–696, 2000.

[6] K. C. Toh, M. J. Todd, and R. Tutuncu. SDPT3 — a Matlab software

package for semidefinite programming. Optimization Methods and Soft-

ware, 11:545–581, 1999.