Upload
brandon-stewart
View
216
Download
1
Embed Size (px)
Citation preview
Subexponential Algorithms for Unique Games and Related Problems
Barriers II Workshop, Princeton, August 2010
David SteurerMSR New England
Sanjeev Arora Princeton University & CCI
Boaz BarakPrinceton & MSR New England
U
Introduction
Small-Set Expansion
Unique Games
UNIQUE GAMESInput: list of constraints of form xi – xj = cij mod k
Goal: satisfy as many constraints as possible
Input: UNIQUE GAMES instance with k << log n (say)
Goal: Distinguish two cases
YES: more than 1 - ² of constraints satisfiableNO: less than ² of constraints satisfiable
Khot’s Unique Games Conjecture (UGC)For every ² > 0, the following is NP-hard:
UG(²)
Implications of UGCFor many basic optimization problems, it is NP-hard to beat current algorithms (based on simple LP or SDP relaxations)
Examples:
VERTEX COVER [Khot Regev’03], MAX CUT [KhotKindlerMosselO’Donnell’04,
MosselO’DonnellOleszkiewicz’05],any MAX CSP [Raghavendra’08], …
Implications of UGCFor many basic optimization problems, it is NP-hard to beat current algorithms (based on simple LP or SDP relaxations)
Examples:
VERTEX COVER [Khot Regev’03], MAX CUT [KhotKindlerMosselO’Donnell’04,
MosselO’DonnellOleszkiewicz’05],any MAX CSP [Raghavendra’08], …
Unique Games Barrier
Example: (®GW + ²)-approximation for MAX CUTat least as hard as UG(²’)
UNIQUE GAMES is common barrier for improving current algorithms of
many basic problems
Reductions show that beating current algorithms for these problems is harder than UNIQUE GAMES
®GW = 0.878…Goemans–Williamson
bound for Max Cut
Implications of UGCFor many basic optimization problems, it is NP-hard to beat current algorithms (based on simple LP or SDP relaxations)Examples:
VERTEX COVER [KR’03], MAX CUT [KKMO,’04 MOO’05],any MAX CSP [Raghavendra’08], …
Consequences for UGC (*)
Analog of UGC with subconstant ² (say ² = 1/log log n) is false(contrast: subconstant hardness for LABEL COVER [Moshkovitz-Raz’08])
Subexponential Algorithm for Unique Games
In particular: UG(²3) has exp(n²)-time algorithm
Given a UNIQUE GAMES instance with alphabet size ksuch that 1 - ² of constraints are satisfiable,can satisfy 1 - √²/¯
3 of constraints in time exp(k n¯)
NP-hardness reduction for UG(²) requires blow-up npoly(1/²)
rules out certain classes of reductions for proving UGC
(*) assuming 3 SAT does not have subexponential algorithms
poly(n) exp(n)
Concrete Complexity Landscape
2-SAT
MAX 3-SAT(7/8)MAX CUT (®GW)
* assuming Exponential Time Hypothesis [Impagliazzo-Paturi-Zane’01]( 3-SAT has no exp(o(n)) algorithm )
3-SAT (*)
FactoringGraph Isomorphism
exp(n1/2)exp(n1/3)exp(n²)
UG(²3)MAX 3-SAT(7/8+²)LABEL COVER(²)
[Moshkovitz-Raz’08+ Håstad’97]
If UGC true, UNIQUE GAMES is first CSP with intermediate complexity
MAX CUT(®GW + ²)?
UGC-based hardness does not rule out subexponential algorithms, Possibility: exp(n²)-time algorithm for MAX CUT(®GW + ²)
UNIQUE GAMES much easier than LABEL COVER
Implications of UNIQUE GAMES algorithm (*)
Introduction
Small-Set Expansion
Unique Games
d-regular graph Gd
vertex set S
Graph Expansion
expansion(S) = # edges leaving S
d |S|
volume(S) = |S||V|
d-regular graph Gd
vertex set S
Graph Expansion
expansion(S) = # edges leaving S
d |S|
volume(S) = |S||V|
S
expansion(S) = # edges leaving S
d |S|
Graph Expansion
volume(S) = |S||V|
SMALL-SET EXPANSION
Goal: find S with volume(S) < ± so as tominimize expansion(S)
Input: d-regular graph G, parameter ± > 0
Important concept in many contexts:derandomization, network routing, coding theory,Markov chains, differential geometry, group theory
close connection to UNIQUE GAMES [Raghavendra-S.’10]
S
expansion(S) = # edges leaving S
d |S|
Graph Expansion
volume(S) = |S||V|
Important concept in many contexts:derandomization, network routing, coding theory,Markov chains, differential geometry, group theory
Subexponential Algorithm for SMALL-SET EXPANSIONIf there exists S with volume(S) < ± and expansion(S) < ²,we can find S’ with volume(S’) < 2± and expansion(S’) < √²/¯
in time exp(n¯/±)
Subexponential Algorithm for SMALL-SET EXPANSIONIf there exists S with volume(S) < ± and expansion(S) < ²,we can find S’ with volume(S’) < 2± and expansion(S’) < √²/¯
in time exp(n¯/±)
1.few large eigenvalues 2.many large eigenvalues
Distinguish two cases:
large eigenvalues¸i > 1 - ´
Eigenvalues of the random walk matrix G:1 = ¸1 ¸ … ¸ ¸m ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1
(pseudorandom graph) (structured graph?)
Subexponential Algorithm for SMALL-SET EXPANSIONIf there exists S with volume(S) < ± and expansion(S) < ²,we can find S’ with volume(S’) < 2± and expansion(S’) < √²/¯
in time exp(n¯/±)
´ À ²(best: ´ = 100²,
simpler: ´ = ²0.75 )
1 - ´
Eigenvalues of the random walk matrix G:1 = ¸1 ¸ … ¸ ¸m ¸
Subexponential Algorithm for SMALL-SET EXPANSIONIf there exists S with volume(S) < ± and expansion(S) < ²,we can find S’ with volume(S’) < 2± and expansion(S’) < √²/¯
in time exp(n¯/±)
¸ ¸m+1 ¸ … ¸ ¸n ¸ -1
large eigenvalues¸i > 1 - ´
Case 1: Few large eigenvalues: (inspired by [Kolla–Tulsiani ’07] and [Kolla’10])
1 = ¸1 ¸ … ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1
Expander Mixing Lemma: indicator vector of S livesalmost completely in span of top m eigenvectors
Eigenvalues of the random walk matrix G:
can find set S’ close to S in time exp(m)
Enumerate this space in time exp(m)
Case 2: Many large eigenvalues (m > n¯/±)
can find small non-expanding set S’ around that vertexWill show: 9 vertex whose neighborhoods grow very slowly
Subexponential Algorithm for SMALL-SET EXPANSIONIf there exists S with volume(S) < ± and expansion(S) < ²,we can find S’ with volume(S’) < 2± and expansion(S’) < √²/¯
in time exp(n¯/±)
Case 1: Few large eigenvalues:
1 = ¸1 ¸ … ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1Eigenvalues of the random walk matrix G:
Case 1: Few large eigenvalues
Subspace enumeration (inspired by [Kolla–Tulsiani ’07] and [Kolla’10])
Eigenvalues of the random walk matrix G:
Suffices to show: indicator vector of S is ²/´-close to U
|S ¢ S’| < ²/´ |S [ S’|
For every set S with expansion(S) < ², can find S’ that is ²/´-close to S in time exp(m)
Algorithm: Let U be span of top-m eigenvectorsFor every vector u in ²-net of unit ball of U,
output all level sets of u
U
1 = ¸1 ¸ … ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1
Case 1: Few large eigenvalues
Subspace enumeration (inspired by [Kolla–Tulsiani ’07] and [Kolla’10])
Eigenvalues of the random walk matrix G:
Suffices to show: indicator vector of S is ²/´-close to U
(generalization of “easy direction” of Cheeger’s inequality)
hx, G xi > 1 - ² because expansion(S) < ²
x = normalized indicator vector of S
1 = ¸1 ¸ … ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1
For every set S with expansion(S) < ², can find S’ that is ²/´-close to S in time exp(m)
Case 1: Few large eigenvalues
Subspace enumeration (inspired by [Kolla–Tulsiani ’07] and [Kolla’10])
Eigenvalues of the random walk matrix G:
(generalization of “easy direction” of Cheeger’s inequality)
hx, G xi = hu, G ui + hw, G wi < hu,ui + (1 - ´)hw,wi = 1 - ´hw,wi
Suppose x = u + w for u in U and w orthogonal to U
hx, G xi > 1 - ² because expansion(S) < ²
x = normalized indicator vector of S
1 = ¸1 ¸ … ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1
Suffices to show: indicator vector of S is ²/´-close to U
For every set S with expansion(S) < ², can find S’ that is ²/´-close to S in time exp(m)
Eigenvalues of the random walk matrix G:1 = ¸1 ¸ … ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1
Case 2: Many large eigenvalues (m > n¯/±)
Eigenvalues of the random walk matrix G:1 = ¸1 ¸ … ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1
If m > 1, then 9 S with volume(S) < ½ and expansion(S) < √´and we can find S in poly(n)-time.
Compare: “hard direction” of Cheeger’s inequality
Case 2: Many large eigenvalues (m > n¯/±)
1 = ¸1 ¸ …………………… ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1Eigenvalues of the random walk matrix G:
If m > n¯/±, then 9 S with volume(S) < ± and expansion(S) < √´/¯
and we can find S in poly(n)-time
Number of large eigenvalues vs. small-set expansion
“higher eigenvalue Cheeger bound”
Heuristic: balls tend to be the least expanding sets in graphs
How can we find small non-expanding sets?
Case 2: Many large eigenvalues (m > n¯/±)
1 = ¸1 ¸ …………………… ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1Eigenvalues of the random walk matrix G:
If m > n¯/±, then 9 S with volume(S) < ± and expansion(S) < √´/¯
and we can find S in poly(n)-time
Number of large eigenvalues vs. small-set expansion
Suffices to show:9 vertex i such that volume( Ball(i, t) ) < ± for t = (¯/´) log(n)
Volume growth vs. small-set expansion
volume growth < 1+(´/¯) in intermediate step expansion( Ball(i,s) ) < (´/¯) for some s < t
Suppose volume( Ball(i, t) ) < ± for t = (¯/´) log(n)
Case 2: Many large eigenvalues (m > n¯/±)
1 = ¸1 ¸ …………………… ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1Eigenvalues of the random walk matrix G:
If m > n¯/±, then 9 S with volume(S) < ± and expansion(S) < √´/¯
and we can find S in poly(n)-time
Number of large eigenvalues vs. small-set expansion
Volume growth vs. small-set expansion
How can we relate eigenvalues and volume growth?
Case 2: Many large eigenvalues (m > n¯/±)
1 = ¸1 ¸ …………………… ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1Eigenvalues of the random walk matrix G:
If m > n¯/±, then 9 S with volume(S) < ± and expansion(S) < √´/¯
and we can find S in poly(n)-time
Number of large eigenvalues vs. small-set expansion
Suffices to show:9 vertex i such that volume( Ball(i, t) ) < ± for t = (¯/´) log(n)
Suffices to show:9 vertex i such that volume( Ball(i, t) ) < ± for t = (¯/´) log(n)
Heuristic:collision probability ¼ 1/|support|
Collision probability decay
|| Gt ei ||2 > 1/(±n)
collision probability of t-step random walk from i
Proof follows from local variant of Cheeger’s inequality(e.g. Dimitriou–Impagliazzo’98)
Case 2: Many large eigenvalues (m > n¯/±)
1 = ¸1 ¸ …………………… ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1Eigenvalues of the random walk matrix G:
If m > n¯/±, then 9 S with volume(S) < ± and expansion(S) < √´/¯
and we can find S in poly(n)-time
Number of large eigenvalues vs. small-set expansion
Volume growth vs. small-set expansion
Number of large eigenvalues vs. collision probability decay
Collision probability decay vs. small-set expansionSuffices to show:
9 vertex i such that || Gt ei ||2 > 1/(±n) for t = (¯/´) log(n)
= ihei, G2t eii = Trace(G2t) > m¢(1 - ´)2t> 1/±i||Gt ei||2
Case 2: Many large eigenvalues (m > n¯/±)
1 = ¸1 ¸ …………………… ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1Eigenvalues of the random walk matrix G:
If m > n¯/±, then 9 S with volume(S) < ± and expansion(S) < √´/¯
and we can find S in poly(n)-time
Number of large eigenvalues vs. small-set expansion
Case 1: Few large eigenvalues:
1 = ¸1 ¸ … ¸ ¸m > 1 - ´ ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1Eigenvalues of the random walk matrix G:
can find set S’ ²/´-close to S in time exp(m)
Subspace enumeration: discretize span of top m eigenvectors
Case 2: Many large eigenvalues (m > n¯/±)
can find small non-expanding set S’ around that vertex
Trace bound: 9 vertex where local random walk mixes slowlyU
Subexponential Algorithm for SMALL-SET EXPANSIONIf there exists S with volume(S) < ± and expansion(S) < ²,we can find S’ with volume(S’) < 2± and expansion(S’) < √²/¯
in time exp(n¯/±)
Introduction
Small-Set Expansion
Unique Games
UNIQUE GAMESInput: list of constraints of form xi – xj = cij mod k
Goal: satisfy as many constraints as possible
Constraint Graph Gvariable vertexconstraint edge i
j
xi – xj = cij
mod k
Subexponential Algorithm for Unique Games
In particular: UG(²3) has exp(n²)-time algorithm
Given a UNIQUE GAMES instance with alphabet size ksuch that 1 - ² of constraints are satisfiable,can satisfy 1 - √²/¯
3 of constraints in time exp(k n¯)
can solve UG(²3) in time exp(m)
Few large eigenvalues + strong L1 bounds on top-m eigenvectors
large eigenvalues
Eigenvalues of the random walk matrix of constraint graph G:1 = ¸1 ¸ … ¸ ¸m > 1 - ² ¸ ¸m+1 ¸ … ¸ ¸n ¸ -1
[Arora–Khot–Kolla– S.–Tulsiani–Vishnoi’08]
Expanding constraint graph (m=1)
can solve UG(²) in time poly(n)
[Kolla’10]
UNIQUE GAMES on pseudorandom instances is easy
Algorithms for special instances of Unique Games
Strategypartition general instance into pseudorandom instances by changing only a small fraction of edges [Trevisan’05]
Algorithms for general instances of Unique Games
general instance
pseudorandom instances
decomposition
few constraints between parts
[Arora–Impagliazzo– –Matthews–S.’10]
here: Leighton–Rao decomposition tree, constant depth, using higher
eigenvalue Cheeger boundhere: at most n¯ eigenvalues > 1-²
here: at most√²/¯
3 fraction
Subexponential Algorithm for Unique Games
few large eigenvalues: at most n¯ eigenvalues >1-²
Few-large-eigenvalues decomposition
Every regular graph G can be partitioned into components with few large eigenvalues by removing √²/¯
3 fraction of edges
Unique Games with few large eigenvalues
If every component of G has few large eigenvalues, can solve UG(²2) in time exp(n¯)
few large eigenvalues: at most n¯ eigenvalues >1-²
Unique Games with few large eigenvalues
If every component of G has few large eigenvalues, can solve UG(²2) in time exp(n¯)
few large eigenvalues: at most n¯ eigenvalues >1-²
Unique Games with few large eigenvalues
If every component of G has few large eigenvalues, can solve UG(²2) in time exp(n¯)
label-extended graph G*
ij
xi – xj = cij
mod k
constraint graph G
cloud jcloud i
(i,a) » (j,b) if a-b = cij mod k
assignment satisfying 1- ²2 of constraints
set with volume = 1/kand expansion < ²2
Subspace enumeration: can enumerate all nonexpanding sets if G* has few large eigenvalues
When does G* have herefew large eigenvalues?
Unique Games with few large eigenvalues
label-extended graph G*
ij
xi – xj = c ij
mod k
constraint graph G
cloud jcloud i
(i,a) » (j,b) if a-b = cij mod k
When does G* have herefew large eigenvalues?
•if G is an expander [Kolla–Tulsiani’07]•if G has few large eigenvalues and eigenvectors are well-spread [Kolla’10]•if G has few’ large’ eigenvalues (this work, by comparing collision probabilities)
If every component of G has few large eigenvalues, can solve UG(²2) in time exp(n¯)
few large eigenvalues: at most n¯ eigenvalues >1-²
Recall: (with slightly different parameters)
Analysis: For every subdivision S, charge its expansion to vertices in S
KIf component K has many large eigenvalues, then 9 S in K with |S| < |K|1-¯ and expansion(S) < √²/¯
and we can find S in poly(n)-time
Number of large eigenvalues vs. small-set expansion
Algorithm: Subdivide components until all components have few large eigenvalues
no vertex is charged more than log(1/¯)/¯ timesIf vertex is charged t times, its component has size < n(1-¯)t
few large eigenvalues: at most n¯ eigenvalues larger 1-²
Few-large-eigenvalues decomposition
Every regular graph G can be partitioned into components with few large eigenvalues by removing √²/¯
3 fraction of edges
Open Questions
Example: C-approximation for SPARSEST CUT in time exp(n1/C )
How many large eigenvalues can a small-set expander have?
Is Boolean noise graph the worst case? (polylog(n) large eigenvalues)
Thank you! Questions?
More Subexponential Algorithms
Similar approximation for MULTI CUT and d-TO-1 2-PROVER GAMES
Better approximations for MAX CUT and VERTEX COVER on small-set expanders
What else can be done in subexponential time?
Towards better-than-subexponential algorithms for UNIQUE GAMES
Better approximations for MAX CUT or VERTEX COVER on general instances?