Upload
basil-austin
View
228
Download
0
Embed Size (px)
Citation preview
Arora: SDP + Approx Survey
Semidefinite Programming and Approximation Algorithms for NP-hard Problems: A Survey
Sanjeev AroraPrinceton University
Arora: SDP + Approx Survey
NP-completeness
Pragmatic Researcher
“Why the fuss? I am perfectly content with approximatelyoptimal solutions.” (e.g., cost within 10% of optimum)
Bad News: NP-hard for many problems. (“PCPs”)
Good news: Possible for a few problems. (“ApproximationAlgorithms”)
Thousands of problems are NP-complete (TSP, Scheduling, Circuit layout, Machine Learning,..)
Arora: SDP + Approx Survey
Talk Outline
• Defn of approximation and example
• SDP and its use in approximation
• Understanding SDPs <-> high dimensional geometry
• Faster algorithms (multiplicative update rule)
• Limitations of SDPs: local vs global issues
• Connections (a) metric spaces (b) avg case complexity(c) unique games conjecture
• Open problems
Arora: SDP + Approx Survey
Approximation Algorithms
MAX-3SAT: Given 3-CNF formula , find assignment maximizing the number of satisfied clauses.
An -approximation algorithm is one that for every formula, produces in polynomial time an assignment that satisfies at least OPT/ clauses. ( >= 1).
Good News: [KZ’97] An 8/7-approximation algorithm exists.
Bad News: [Hastad’97] If P NP then for every > 0, an(8/7 -)-approximation algorithm does not exist.
(Similar results for many other problems…)
Arora: SDP + Approx Survey
Good news (for me)
Status of many basic problems is still unresolved:
• Vertex Cover• Sparsest Cut and most graph partitioning problems• Graph coloring • Random instances of 3SAT
My feeling: Interesting algorithms remain undiscovered;
semidefinite programming (SDP) may be helpful.
SDP = Generalization of linear programming
Graph
Vector Representation
Arora: SDP + Approx Survey
Example: 2-approximation for Min Vertex Cover
G= (V, E)Vertex Cover = Set of vertices that touches every edge
“LP Relaxation”
Claim: Value at least OPT/2
Proof: “Rounding”
most
Proof: On Complete Graph Kn, OPT = n-1
but setting all xi = 1/2 gives feasible LP soln
Arora: SDP + Approx Survey
General Philosophy…
Interested in: NP-hard Minimization Problem
Value = OPT Write tractable relaxationvalue=
Round to get a solution of cost
= Approximation ratio = Integrality gap
Arora: SDP + Approx Survey
Main Idea in SDP: “Simulate” nonlinear programming
Nonlinear program for Vertex Cover Homogenized
SDP relaxation:New variable intended to stand for
Arora: SDP + Approx Survey
How do you understand thesevector programs?
Ans. Interesting geometric analysis
Arora: SDP + Approx Survey
Understanding SDPs <--> Understanding phenomena in high-dimensional geometry
computes c-approximationfor c < 2 iff following is true
Vertex Cover SDP
Every graph in this family has an independent set of size
Thm [Frankl-Rodl’87] False.
Vertices: n unit vectorsEdges: almost-antipodal pairs
Rn
Arora: SDP + Approx Survey
SDP rounding: The two generations
First generation: *Uses random hyperplane as in [GW]; * Edge-by-edge analysis
Max-2SAT and Max-CUT [GW’94] ;Graph coloring [KMS’95]; MAX-3SAT [KZ’97]; Algorithms for Unique Games;..
Second generation: Global rounding and analysisGraph partitioning problems [ARV’04],Graph deletion and directed partitioning problems [ACMM’05],New analysis of graph coloring [ACC’06]Disproof of UGC for expanding constraints [AKKSTV’08]
(Similarly, two generations of results showing limitson performance of SDPs)
Arora: SDP + Approx Survey
1st Generation Rounding: Ratio 1.13.. for MAX-CUT[Goemans-
Williamson’93]G = (V,E) Find that maximizes capacity .
Quadratic Programming Formulation
Semidefinite Relaxation [DP ’91, GW ’93]
Arora: SDP + Approx Survey
Randomized Rounding [GW ’93]
v6
v2
v3v5
Rn
v1
Form a cut by partitioning v1,v2,...,vn around a random hyperplane.
SDPOPT
vj
vi
ij
Old math rides to the rescue...
Arora: SDP + Approx Survey
Fact 1: No rounding algorithm can produce a bettersolution out of this SDP [Feige-Schectman]
Fact 2: If P NP then impossible to get 1.09-approximationby any efficient algorithm [Hastad’97]
Fact 3: If “unique games conjecture” is true, it is impossibleto get a better than 1.13-approximation.[KKMO’05]
(i.e., algorithm on prev. slide is optimal)
“Edges between all pairs of vectorsmaking an angle 138 degrees.”
Arora: SDP + Approx Survey
2nd Generation: for c-balanced separator
G= (V, E); constant c >0
Goal: Find cut s.t. each side contains atleast c fraction of nodes and minimized
1 -1
SDP:
“Triangle inequality”
Angle subtended by the line joiningtwo of them on the third is non-obtuse;“ “ condition.
Arora: SDP + Approx Survey
Rounding algorithm for -approximation [ARV’04]
1. Pick random hyperplane
2. Remove points in “slab” of width
3. Remove any pair (i, j) that lie on opp.sides of slab but
4. Call remaining sets S, T. Do BFS from S to T according to distance
5. Output level of BFS tree with least # of edges.
S
T
S T
Arora: SDP + Approx Survey
Geometric fact underlying the analysis (restatement of [ARV04] “Structure Theorem” by [AL06])
(“expander” : |(S)|¸ (|S|) )
Vertices: unit vectorssatisfying “triangle inequality”
Edges:
If then no graphin this family is an “expander.”
Proof is delicate and difficult
Arora: SDP + Approx Survey
Solving SDPs with m constraints takes time.
Issue of Running Time
m =n3 in some of these SDPs!
Next few slides: Often, can reduce running time: O(n2) or O(n3). [AHK’05], [AK’07]
Main idea: “Primal-dual schema.”Solve to approximate optimality; using insights from the rounding algorithms.
“Multiplicative Weight-Update Rule for psd matrices”
Arora: SDP + Approx Survey
Classical MW update rule (Example: predicting the market)
• N “experts” on TV• Can we perform as good as the best expert ?
1$ for correct prediction
0$ for incorrect
Thm[Going back to Hannan, 1950s] Yes.
Arora: SDP + Approx Survey
Weighted Majority Algorithm [LW’94]
“Predict according to the weighted majority”
• Maintain a weight for each expert. Initially• At step t, if expert i’s prediction was incorrect,
Claim: Expected Payoff of our algorithm
Similar algorithms discovered in a variety of areas:decision theory, learning theory (“boosting”),cryptography (“hardcore sets”), approx soln of LPs,..
(see survey [A, Hazan, Kale])
Arora: SDP + Approx Survey
Primal-dual approach for SDP relaxations [A., Kale’07]
At step t:
Primal player: PSD matrix Pt; candidate primal
Dual player: Let me run the rounding algorithm on Pt, get a primal
integer candidate and point out how pitiful it is.
“Feedback” matrix Mt
Primal player: Pt+1 = exp(- t Mt)
(Analysis uses formal analogy between real #s and symmetric matrics:[Other ingredients: flow computations, eigenvalues, dimension reduction tricks, etc.]
Arora: SDP + Approx Survey
Implications for geometric embeddings of metric spaces
x
(X, d): metric space
yd(x, y) f
f(x)
f(y)
C = distortion
Thm (Bourgain’85) For every X, there is f s.t. C= O(log n).
Open qs since then: is it possible to achieve smaller C forconcrete X, say X = ?
[CGR’05,ALN05]: Yes, C possible for X =
[KV06]: Cannot reduce C below
Arora: SDP + Approx Survey
Unique Games
Given: Number p, and m equations in n vars of the form:
Promise: Either there is a solution that satisfies fractionof constraints or no solution satisfies even fraction.
UGC [Khot’02]: Deciding which case holds is intractable.
Seems to capture our current limitations of thinking aboutSDPs; basis of many recent “hardness” results.
Arora: SDP + Approx Survey
Anatomy of a UGC-based hardness result
Interpret as a graph
EquationsVariables
Replace edges/verticeswith hypercube-like gadgets
EquationsVariables
Prove using harmonicanalysis that near--optimum solnscorrespond to goodSolution to theunique game
Arora: SDP + Approx Survey
Limitations of SDPs
For many problems, we know neither an NP-hardness result (via PCPs) nor a good SDP-based approach.
2nd Generation results: Large families of LPs or SDPs don’t work
[ABL’02], [ABLT’06]: “Proving integrality gaps withoutknowing the LP.”
Much subsequent work, especially on families obtained from “lift and project” ideas)
1st generation results: Specific SDPs don’t work
Can we show that known SDPs don’t work??
Arora: SDP + Approx Survey
“Lifted” SDP relaxations
Recall: SDP tries to “simulate” nonlinear programming;Variable for
Why not take it to the next level? Variablesfor products of up to k variables.
This is the main idea of Lovasz-Schrijver’91, Sherali-Adams,Lasserre etc.
“SDP as a proof system”: Integrality gaps proved in 2nd generation results.
Arora: SDP + Approx Survey
Main issue: Local versus Global
Example: [Erdos] There are graphs on n vertices that cannot be colored with 100 colors yetevery subgraph on 0.01 n vertices is 3-colorable.
LP relaxations or SDP relaxations concern localconditions.
How well do such local conditions capture global property in question?
Results for MAX-k-SAT [], [AAT’05], Vertex Cover[ABLT’06],[STT’07a+b] MAX-CUT, Vertex Coveretc. [CMM’08]
“Lifted SDPs.” Connections to Proof Complexity.
Arora: SDP + Approx Survey
Connections to Avg. Case Complexity(SDP used in reductions)
Recent development: Interreducibility among some“average case” problems of interest. [Feige’01][Alekh.03]
Problems like 3SAT seem difficult not only in the worstcase but also “on average.” (Needs careful definition!)
Theory of Avg Case complexity exists, but doesn’t usuallyapply to problems of practical interest.
SDP is used in the reduction!
Arora: SDP + Approx Survey
Open problems
• Techniques for proving lowerbounds on lifted SDPs.(difficult local-global results)
• New rounding algorithms• Clarify nature of connection to average case
complexity.• Resolve UGC (recently, disproof of UGC when the
constraint graph is an expander.[AKKSTV08]• SDP as a proof technique---apply to open problems of
circuit complexity, communication complexity etc.
Looking forward to many developments
THANK YOU!
Arora: SDP + Approx Survey
SDPs and MW Updates: Primal-dual algorithm
Known: MW Update rule --> Approx. solutions to LPs [PST’91, Y’95, GK’97,..etc.]
[AK’07] Matrix MW update rule that uses formal analogybetween psd matrices and nonnegative real #s.
(Spl. Case: LPs= SDPs with 0’s on offdiagonals)
[Other ingredients: flow computations, eigenvalues, dimension reduction tricks, etc.]
“experts” <-> constraints“payoffs” <-> “slack in constraint”
[Golden-Thompson]
Arora: SDP + Approx Survey
Embeddings and Cuts
Thm[LLR94, AR94]: Integrality gap for SDP for Nonuniform Sparsest Cut = Min distortion of any embedding of into
Rounding algorithm of [ARV04] gives insight into structureof ; basis of new embeddings
Hardness results for sparsest cut yielded insights at the heart of the embedding impossibility results.