Lasserre Hierarchy,Higher Eigenvalues and

Approximation Schemes for Graph Partitioning and PSD QIP

Ali Kemal Sinop(joint work with Venkatesan Guruswami)

Carnegie Mellon University

Outline

• Introduction– Sample Problem: Minimum Bisection– Approximation Algorithms – Our Motivation and Results Overview

• Results– Graph Spectrum– Related Work and Our Results

• Case Study: Minimum Bisection– Lasserre Hierarchy Formulation– Rounding Algorithm– Analysis 210:13 PM

Minimum Bisection• Given graph G=(V,E,W), find subset of size n/2

which cuts as few edges as possible.

• Canonical problem for graph partitioning by allowing arbitrary size:– Small Set Expansion (weight each node by its degree)– Uniform Sparsest Cut (try out all partition sizes in

small increments)– Etc…

• NP-hard. 3

1 2 3 4 1 2 3 4Cost=2

µ

10:13 PM

Approximation Algorithms

• Find an α-factor approximation. – If minimum cost = OPT, • Algorithm always finds a solution with value ≤ α OPT.

• (This work) Round a convex relaxation.

OPT Algorithm α OPT0 Relaxation

4

1

10:13 PM

Motivation

• For many graph partitioning problems (including minimum bisection), huge gap between hardness and approximation results.

• Best known algorithms have factor • Whereas no 1.1 factor hardness is known.• We want to close the gap.

510:13 PM

Our Results: Overview

• For graph partitioning problems including:– Minimum bisection,– Small set expansion,– Uniform sparsest cut,– Minimum uncut,– Their k-way generalizations, etc…

• We give approximation schemes whose running time is dependent on graph spectrum.

610:13 PM

Outline

• Introduction– Sample Problem: Minimum Bisection– Approximation Algorithms – Our Motivation and Results Overview

• Results– Graph Spectrum– Related Work and Our Results

• Case Study: Minimum Bisection– Lasserre Hierarchy Formulation– Rounding Algorithm– Analysis 710:13 PM

Graph Spectrum and Eigenvalues

1 2 3 4

rows and cols indexed by V

8

λ2: Measures expansion of the graph through Cheeger’s inequality .λr: Related to small set expansion [Arora, Barak, Steurer’10], [Gharan, Trevisan’11].

0 = λ1 ≤ λ2 ≤ … ≤ λn ≤2 and λ1 + λ2 + … + λn = n,

10:13 PM

Related Previous Work

• (Minimization form of) Unique Games (k-labeling with permutation constraints):– [AKKSTV’08], [Makarychev, Makarychev’10]

Constant factor approximation for Unique Games on expanders in polynomial time.

– [Kolla’10] Constant factor when λr is large.

• [Arora, Barak, Steurer’10] – For Unique Games and Small Set Expansion,

factor in time – For Sparsest Cut, factor assuming 910:13 PM

Our Results (1)• In time we obtain

• Why approximation scheme?– 0 = λ1 ≤ λ2 ≤ … ≤ λn ≤2 and λ1 + λ2 + … + λn = n,

10

Minimum Bisection*Small Set Expansion*Uniform Sparsest CutTheir k-way generalizations*

Independent Set

* Satisfies constraints within factor of

For r=n, λr >1, λn-r <1

Minimum Uncut

10:13 PM

Our Results for Unique Games

• For Unique Games, a direct bound will involve spectrum of lifted graph, whereas we want to bound using spectrum of original graph.– We give a simple embedding and work directly on

the original graph.• We obtain factor in time .• Concurrent to our work, [Barak, Steurer,

Raghavendra’11] obtained factor in time using a similar rounding.

1110:13 PM

Outline

• Introduction– Sample Problem: Minimum Bisection– Approximation Algorithms – Our Motivation and Results Overview

• Results– Graph Spectrum– Related Work and Our Results

• Case Study: Minimum Bisection– Lasserre Hierarchy– Rounding Algorithm– Analysis 1210:13 PM

Case Study: Minimum Bisection

• We will present an approximation algorithm for minimum bisection problem on d-regular unweighted graphs.

• We will show that it achieves factor .• Obtaining factor requires some additional

ideas.

1310:13 PM

Lasserre Hierarchy

• Basic idea: Rounding a convex relaxation of minimum bisection.

• [Lasserre’01] Strongest known SDP-relaxation.– (Relaxation of) For each subset S of size ≤ r and

each possible labeling of S, – An indicator vector which is 1 if S is labeled with f – 0 else.

• And all implied consistency constraints.

1410:13 PM

Previous Work on Lasserre Hierarchy

• Few algorithmic results known before, including:– [Chlamtac’07], [Chlamtac, Singh’08] nΩ(1) approximation for 3-coloring and

independent set on 3-uniform hypergraphs, – [Karlin, Mathieu, Nguyen’10] (1+1/r) approximation of knapsack for r-rounds.

• Known integrality gaps are:– [Schoenebeck’08], [Tulsiani’09] Most NP-hardness results

carry over to Ω(n) rounds of Lasserre.– [Guruswami, S, Zhou’11] Factor (1+α) integrality gap for

Ω(n) rounds of min-bisection and max-cut.• Not ruled out yet:

“5-rounds of Lasserre relaxation disproves Unique Games Conjecture.”

1510:13 PM

Why So Few Positive Results?

• For regular SDP [Goemans, Williamson’95] showed that with hyperplane rounding:

• Prior to our work, no analogue for Lasserre solution.

1610:13 PM

Consistency

Lasserre Relaxation for Minimum Bisection

17

• Relaxation for consistent labeling of all subsets of size < r:

Marginalization

Distribution

Partition SIze

Cut cost

10:13 PM

Rounding Algorithm

• Choose S with probability – [Deshpande, Rademacher, Vempala, Wang’06] Volume sampling.

• Label S by choosing f with probability .• Propagate to other nodes:– For each node v,• With probability include v in U.

– Inspired by [AKKSTTV’08] which used propagation from a single node chosen uniformly at random.

• Return U.

10:13 PM 18

Analysis

• Partition Size– Each node is chosen into U independently– By Chernoff, with high probability

• Number of Edges Cut– After arithmetization, we have the following

bound:

19

Normalized Vector for xS(f)

≤ OPT19

10:13 PM

Matrix ΠS

• Remember xS(f)f are orthogonal. is a projection matrix onto

spanxS(f)f .

• For any

20

Let PS be the corresponding projection

matrix.

10:13 PM

Low Rank Matrix Reconstruction

• The final bound is:

• For any S of size r this is lower bounded by:

• [Guruswami, S’11] Volume sampling columns yield

– And this bound is tight.

best rank-r approximation of X

best rank-r approximation of X

2110:13 PM

Relating Reconstruction Error to Graph Spectrum

• Best rank-r approximation is obtained by top r-eigenvectors.

• Using Courant-Fischer theorem,

• Therefore

2210:13 PM

Summary

• Gave a randomized rounding algorithm based on propagation from a seed set S so that:

• Related choosing S to low rank matrix reconstruction error.

• Bounded low rank matrix reconstruction error in terms of λr.

2310:13 PM

Questions?

• Thanks.

2410:13 PM