Venkatesan Guruswami (CMU)Yuan Zhou (CMU)

Satisfiable CSPs

Theorem [Schaefer'78]

Only three nontrivial Boolean CSPs for which satisfiability is poly-time decidable• LIN-mod-2 -- linear equations mod 2e

• 2-SAT• Horn-SAT -- CNF formula where each clause consists of at most one unnegated literal e.g.

1x 2x 421 xxx , , ,

542 xxx 542 xxx (equivalent to )

Almost satisfiable CSPs

-satisfiable instance -- satisfiable by removing fraction of clauses

)1(

Finding almost satisfying assignments

satisfiable instance satisfying solution

"almost" satisfiable instance

"almost" satisfying solution

)1(opt ))1(1(alg ,no

robust version (against noise)

input output

Almost satisfiable CSPs

-satisfiable instance -- satisfiable by removing fraction of clauses

)1(

Finding almost satisfying assignmentsGiven a -satisfiable instance, can we efficiently find an assignment satisfying . constraints, where as . ?

)1(

))1()(1( nof 0)( f0

The answer...• No for LIN-mod-2

– vs. is NP-Hard [Håstad'01]• Yes for 2-SAT

– SDP-based alg. gives vs [Zwick'98]– Improved to vs [CMM'09]– Tight under Unique Games Conjecture [KKMO'07]

• Yes for Horn-SAT– LP-based alg. gives vs [Zwick'98]– For Horn-3SAT, Zwick's alg. gives vs

– Exponential loss -- is it tight?

)1( )2/1(

)1( )1( 3/1

)1(

1

1

log

loglog1

)1(

1log

11

)1( )1(

Approximability of almost satisfiable Horn-SAT

• Previously knownHorn 3-SAT

Approx. Alg. 1-1/(log 1/ε)[Zwick'98]

NP-Hardness 1-εc for some c < 1[KSTW'00]

UG-Hardness

Approximability of almost satisfiable Horn-SAT

• Previously knownHorn 3-SAT Horn 2-SAT

Approx. Alg. 1-1/(log 1/ε)[Zwick'98]

1-3ε[KSTW'00]

NP-Hardness 1-εc for some c < 1[KSTW'00]

1-1.36εfrom Vertex Cover

UG-Hardness 1-(2-δ)εfrom Vertex Cover

Approximability of almost satisfiable Horn-SAT

• Our result

• Comment. People need UGC to get sharp inapprox. result for most of problems

Horn 3-SAT Horn 2-SATApprox. Alg. 1-1/(log 1/ε)

[Zwick'98]1-2ε

NP-Hardness 1-εc for some c < 1[KSTW'00]

1-1.36εfrom Vertex Cover

UG-Hardness 1-1/(log 1/ε) 1-(2-δ)εfrom Vertex Cover

Proof framework of the hardness result

c vs. s dictatorship

test

c vs. s dictatorship

test

[KKMO'07,Rag'08] c vs. s UG-Hardness for

the CSP

c vs. s UG-Hardness for

the CSP

not clear how to construct a dictatorship test for HornSAT

Theorem. [Rag'08] There is a canonical SDP relaxation for SDP(Λ) each CSP Λ, such that c vs. s integrality gap => c-η vs. s+η dictator test.

MaxCut, Linear Equations, Max-2SAT, Vertex Cover ...

Proof framework of the hardness result

c vs. s dictatorship

test

c vs. s dictatorship

test

[KKMO'07,Rag'08] c vs. s UG-Hardness for

the CSP

c vs. s UG-Hardness for

the CSP

c vs. s integrality gap for

the "canonical SDP"

c vs. s integrality gap for

the "canonical SDP"

[Rag'08]

construct an SDP gap instance instead

MaxCut, Linear Equations, Max-2SAT, Vertex Cover ...

Theorem. [Rag'08] There is a canonical SDP relaxation for SDP(Λ) each CSP Λ, such that c vs. s integrality gap => c-η vs. s+η dictator test.

Our Theorem 1. There is a (1-2-k) vs. (1-1/k) gap instance for SDP(Horn-3SAT), for every k > 1.

Our Theorem 2. A tight gap instance for SDP(1-in-k HittingSet).

1-in-k HittingSet

• U : universe • C : collection of subsets of U of size <= k• Goal : a subset S of U intersecting maximum num

ber of sets in C at exactly one element

• Theorem 2. (1-1/k0.999) vs. 1/log k SDP gap.• Corollary. UG-Hard to approx. within O(1/log k).

• 1-in-Exact k HittingSet:

• Approximability of 1-in-EkHS: 1/e [GT05]

• C : collection of subsets of U of size k<=

=

1-in-k HittingSet

• U : universe • C : collection of subsets of U of size <= k• Goal : a subset S of U intersecting maximum num

ber of sets in C at exactly one element

• Theorem 2. (1-1/k0.999) vs. 1/log k SDP gap.• Corollary. UG-Hard to approx. within O(1/log k).

• Fact. An Ω(1/log k) approx. algorithm.• Theorem 3. A (1-1/2k) vs. 0.1 approx. algorithm.

c vs. s dictatorship

test

c vs. s dictatorship

test

[KKMO'07,Rag'08] c vs. s UG-Hardness for

the CSP

c vs. s UG-Hardness for

the CSP

c vs. s integrality gap for

the "canonical SDP"

c vs. s integrality gap for

the "canonical SDP"

[Rag'08]

MaxCut, Linear Equations, Max-2SAT, Vertex Cover ...

Horn-3SAT1-in-k HittingSet

The first work (and the only one so far) using Raghavendra's theorem to get sharp hardness result.

The canonical SDP:Lifted LP + semidefinite constraints

The lifted-LP (in Sherali-Adams system)• C: the set of clauses• For each CєC, set up local (integral) prob. distribu

tion πC on all truth-assignments {σ : XC -> {0, 1} }– Variables. πC(σ) >= 0 for each σ : XC -> {0, 1}– Constraints. Σσ πC(σ) = 1

maximize ECєC[Prσ~πC[C(σ)=1]]

Prσ~πC[σ(xi)=b1 Λ σ(xj)=b2] = X(xi,b1),(xj,b2)

for all CєC; xi, xj C; bє 1,b2 {0, 1}є

consistency of pairwise margins:

consistency of singleton margins: s.t. Prσ~πC[σ(xi)=b1] = X(xi,b1),(xi,b1)

linear expressions

The semidefinite constraints

• Vectors. Introduce v(x,0) and v(x,1) corresponding to the event x = 0 and x = 1.

• Constraints.– <v(x,0), v(x,1)> = 0 -- mutually exclusive events– v(x,0) + v(x,1) = I -- probability adds up to 1– Prσ~πC[σ(xi)=b1 Λ σ(xj)=b2] = <v(xi,b1),v(xj,b2)> -- pairwise marginals must be PSD

The gap instance for Horn-3SAT.

Instance Ik:

x0, y0x0 Λ y0 -> x1, x0 Λ y0 -> y1x1 Λ y1 -> x2, x1 Λ y1 -> y2x2 Λ y2 -> x3, x2 Λ y2 -> y3

xk Λ yk -> xk+1, xk Λ yk -> yk+1xk+1, yk+1

Step 0:Step 1:Step 2:Step 3:

Step k+1:Step k+2:

... ... ... ...

Observation. Ik is not satisfiable. Therefore OPT(Ik) < 1 - Ω(1/k) .

OPTLP(Ik) >= 1 - 1/2k

x0, y0x0 Λ y0 -> x1, x0 Λ y0 -> y1x1 Λ y1 -> x2, x1 Λ y1 -> y2x2 Λ y2 -> x3, x2 Λ y2 -> y3

xk Λ yk -> xk+1, xk Λ yk -> yk+1xk+1, yk+1

Step 0:Step 1:Step 2:Step 3:

Step k+1:Step k+2:

... ... ... ...

Observation. Clauses in different steps share at most one variable. No worry about pairwise margins between different steps.

OPTLP(Ik) >= 1 - 1/2k

x0, y0x0 Λ y0 -> x1, x0 Λ y0 -> y1x1 Λ y1 -> x2, x1 Λ y1 -> y2x2 Λ y2 -> x3, x2 Λ y2 -> y3

xk Λ yk -> xk+1, xk Λ yk -> yk+1xk+1, yk+1

Step 0:Step 1:Step 2:Step 3:

Step k+1:Step k+2:

... ... ... ...

x0(y0)10

πC(σ)1-δ

δ

x0Λy0->x1(y1)1 Λ 1 -> 10 Λ 1 -> 01 Λ 0 -> 0

πC(σ)1-2δ

δδ

loss = 2δ

OPTLP(Ik) >= 1 - 1/2k

x0, y0x0 Λ y0 -> x1, x0 Λ y0 -> y1x1 Λ y1 -> x2, x1 Λ y1 -> y2x2 Λ y2 -> x3, x2 Λ y2 -> y3

xk Λ yk -> xk+1, xk Λ yk -> yk+1xk+1, yk+1

Step 0:Step 1:Step 2:Step 3:

Step k+1:Step k+2:

... ... ... ...

x0Λy0->x1(y1)1 Λ 1 -> 10 Λ 1 -> 01 Λ 0 -> 0

πC(σ)1-2δ

δδ

x1Λy1->x2(y2)1 Λ 1 -> 10 Λ 1 -> 01 Λ 0 -> 0

πC(σ)1-4δ

2δ2δ

loss = 2δ

OPTLP(Ik) >= 1 - 1/2k

x0, y0x0 Λ y0 -> x1, x0 Λ y0 -> y1x1 Λ y1 -> x2, x1 Λ y1 -> y2x2 Λ y2 -> x3, x2 Λ y2 -> y3

xk Λ yk -> xk+1, xk Λ yk -> yk+1xk+1, yk+1

Step 0:Step 1:Step 2:Step 3:

Step k+1:Step k+2:

... ... ... ...

x2Λy2->x3(y3)1 Λ 1 -> 10 Λ 1 -> 01 Λ 0 -> 0

πC(σ)1-8δ

4δ4δ

x1Λy1->x2(y2)1 Λ 1 -> 10 Λ 1 -> 01 Λ 0 -> 0

πC(σ)1-4δ

2δ2δ

loss = 2δ

OPTLP(Ik) >= 1 - 1/2k

x0, y0x0 Λ y0 -> x1, x0 Λ y0 -> y1x1 Λ y1 -> x2, x1 Λ y1 -> y2x2 Λ y2 -> x3, x2 Λ y2 -> y3

xk Λ yk -> xk+1, xk Λ yk -> yk+1xk+1, yk+1

Step 0:Step 1:Step 2:Step 3:

Step k+1:Step k+2:

... ... ... ...

x2Λy2->x3(y3)1 Λ 1 -> 10 Λ 1 -> 01 Λ 0 -> 0

πC(σ)1-8δ

4δ4δ

xkΛyk->xk+1

xkΛyk->yk+11 Λ 1 -> 10 Λ 1 -> 01 Λ 0 -> 0

πC(σ)1-2k+1δ

2kδ2kδ

...

loss = 2δ

OPTLP(Ik) >= 1 - 1/2k

x0, y0x0 Λ y0 -> x1, x0 Λ y0 -> y1x1 Λ y1 -> x2, x1 Λ y1 -> y2x2 Λ y2 -> x3, x2 Λ y2 -> y3

xk Λ yk -> xk+1, xk Λ yk -> yk+1xk+1, yk+1

Step 0:Step 1:Step 2:Step 3:

Step k+1:Step k+2:

... ... ... ...

xk+1(yk+1)10

πC(σ)1-2k+1δ

2k+1δ

xkΛyk->xk+1

xkΛyk->yk+11 Λ 1 -> 10 Λ 1 -> 01 Λ 0 -> 0

πC(σ)1-2k+1δ

2kδ2kδ

loss = 2δ + 2(1-2k+1)δ = 1/2k (by taking δ = 1/2k+1)

Getting a good SDP solution

• No vectors corresponding to the previous LP solution– Because of the extra semidefinite constraints

• Solution: twist the LP solution in several ways

Summary of our results

• (1 - ε) vs (1 - 1/(log 1/ε)) UG-Hardness for Horn-3SAT

• (1 - 1/k0.999) vs 1/log k UG-Hardness for 1-in-k HittingSet

• (1 - ε) vs (1 - 2ε) algorithm for Horn-2SAT• (1 - 1/2k) vs 0.1 approximation algorithm for 1-in-k

HittingSet

Open directions

• NP-Hardness for approximating 1-in-k HittingSet. Ok(1)?

• For which CSPs does it suffice to show an LP integrality gap?

• Study finding almost satisfiable solutions for non-Boolean CSPs.– Conjecture. There are poly-time algorithms for al

most satisfiable CSPs that cannot express linear equations (i.e. "bounded width" CSPs, by [Barto-Kozik'09]).

The End.

Any questions?