PCP and Inapproximability Irit Dinur NEC. Example: the Minimum Vertex Cover functionExample: the Minimum Vertex Cover function Facts: 1. Best algorithm

PCP and Inapproximability

Irit Dinur

NEC

• Example: the Minimum Vertex Cover functionExample: the Minimum Vertex Cover function

• Facts: 1. Best algorithm runs in time (1.21)Facts: 1. Best algorithm runs in time (1.21)n n [Robson ‘86][Robson ‘86]• 2. VC is NP-hard. [Karp ’72]2. VC is NP-hard. [Karp ’72]• What about approximation.. Output a vertex cover that’s What about approximation.. Output a vertex cover that’s

“nearly” minimal!“nearly” minimal!

Minimum Minimum Vertex Vertex CoverCover

Given any function..Given any function..

How complicated is it to compute it?How complicated is it to compute it?

4

Vertex-Cover:Vertex-Cover: Given a network of roads. Each road requires a toll Given a network of roads. Each road requires a toll payment. payment.

Goal: Put up the smallest number of toll boothsGoal: Put up the smallest number of toll booths

7

What do we mean by approximation? What do we mean by approximation?

Each instance has many solutions, each has a value.Each instance has many solutions, each has a value.

In optimization, we are seeking the minimal. In optimization, we are seeking the minimal.

4 5

ApproximationApproximation

MINMIN

An approximation algorithm: finds a solution within a certain neighborhood of MIN

ApproxApprox

Example: An algorithm for Example: An algorithm for Approximating Vertex CoverApproximating Vertex Cover

1.1. Given G, find a maximal set of edges that do not touch each Given G, find a maximal set of edges that do not touch each other. other.

2.2. Add both vertices of each edge to the vertex cover.Add both vertices of each edge to the vertex cover.

74 5


ApproximationApproximationHow big is it? How big is it?

No more than No more than twice the twice the minimum!minimum!

This is This is aa solution: solution: all edges are all edges are coveredcovered






We’ve seen an approximation algorithm for Vertex-We’ve seen an approximation algorithm for Vertex-

Cover, with approximation factor 2.Cover, with approximation factor 2.

How big is it? How big is it?

No more than No more than twice the twice the minimum!minimum!




MINMIN

x 2

ApproxApprox

We’ve seen a factor 2 algorithm.We’ve seen a factor 2 algorithm.

Q: Is there a factor 1.99 algorithm?Q: Is there a factor 1.99 algorithm?

3/2 ?3/2 ?

4/3 ?4/3 ?

x 3/2 x 4/3 x 1.99

Not unless P=NP, [AS,ALMSS,BGS,Raz,Hastad,DS]

We don’t know


Finding the Approximation ThresholdFinding the Approximation Threshold

• Rich variety of approximation algorithms.Rich variety of approximation algorithms.• Various problems with vastly differing approximation Various problems with vastly differing approximation

ratios.ratios.• PCP theorem: breakthrough, we can prove hardness of PCP theorem: breakthrough, we can prove hardness of

approximation.approximation.• Moreover, it’s within reach to find the exact Moreover, it’s within reach to find the exact thresholdthreshold

where a problem changes from being easy to hard. where a problem changes from being easy to hard. (theoretic and practical importance)(theoretic and practical importance)

• Vertex Cover:Vertex Cover:• Upper bound: 2-o(1) Upper bound: 2-o(1) [BYE, MS, Hal][BYE, MS, Hal]

• Best hardness result:Best hardness result:Thm Thm [DS`02][DS`02]: NP-hard to approximate to within 1.36.: NP-hard to approximate to within 1.36.ConjectureConjecture: NP-hard to within 2- : NP-hard to within 2- >0>0

How to show hardness?How to show hardness?

Take any NP-hard problem, say SAT, and “reduce”Take any NP-hard problem, say SAT, and “reduce”

Translating a formula Translating a formula into a graph G, s.t. into a graph G, s.t.

VC(G) > kVC(G) > k

is SATis SAT VC(G)=kVC(G)=k

is unSAT is unSAT

The gap-version of the problemThe gap-version of the problemMIN = kMIN = k Approx <Approx < (1+c)(1+c)kk

Given a Given a 1+c1+c approximation algorithm, approximation algorithm,

If MIN = k then Approx < (1+c)If MIN = k then Approx < (1+c)kk..

A A 1+c1+c approximation algorithm could decide approximation algorithm could decide whether the minimum is whether the minimum is • below kbelow k or or• above (1+c)above (1+c)kk

In other words, it would solve the In other words, it would solve the gap problemgap problem::

Given a graph G, decide if the minimum VC isGiven a graph G, decide if the minimum VC is

How to show hardness of How to show hardness of approximation?approximation?

Take any NP-hard problem, say SAT, and “reduce”Take any NP-hard problem, say SAT, and “reduce”

Translating a formula Translating a formula into a graph G, s.t. into a graph G, s.t.

Decades, complexity of approximations remains mysteriousDecades, complexity of approximations remains mysterious

Rich variety of algorithms, hardly any lower-boundsRich variety of algorithms, hardly any lower-bounds

Why?Why?

VC(G) > kVC(G) > k


is unSAT is unSAT

VC(G) > (1+c)kVC(G) > (1+c)k


(x(x11,x,x22,…,x,…,xnn))0 1 00 1 0

Simple: Every SAT assignment translates to a size-k VC for G.

VC(G) > kVC(G) > k


is unSAT is unSAT

VC(G) > (1+c)kVC(G) > (1+c)k

(x(x11,x,x22,…,x,…,xnn))0 1 00 1 0

Simple: Every SAT assignment translates to a size-k VC for G.

And: Every size-k VC translates to a SAT assignment.

But, far from easy:

Given a VC of size < (1+c)k, how to decode it into a SAT assignment?

““Combinatorial Decoding”Combinatorial Decoding”

PProbabilistically robabilistically CCheckable heckable PProofsroofs

PCP – a referee’s dreamPCP – a referee’s dream

• Background context – Interactive ProofsBackground context – Interactive Proofs• Say you get a paper for refereeing. Say you get a paper for refereeing. • Select a few random lemmas and verify only them. Select a few random lemmas and verify only them. • Will this work? – in general, of course not!Will this work? – in general, of course not!• PCP TheoremPCP Theorem: Every proof can be compiled into : Every proof can be compiled into

“PCP-language” so that by reading only a few bits of “PCP-language” so that by reading only a few bits of the new PCP proof, correct verification can be the new PCP proof, correct verification can be achieved with high probability.achieved with high probability.

• Why is this amazing?Why is this amazing?• One tiny bug or hole or gap can cause much agony…One tiny bug or hole or gap can cause much agony…• In a proof usually the correctness of each step depends on In a proof usually the correctness of each step depends on

much of what happened before it…much of what happened before it…• It seems that if you only read a few bits, you can easily be It seems that if you only read a few bits, you can easily be

cheated!cheated!

x1

x2

x3

x4

x5

x6

x7

x8

xn-3

xn-2

xn-1

xn

x9

x10

x11

x12

x13

x14

x15

x16

……

• An NP statement can be written as a 3SAT formula An NP statement can be written as a 3SAT formula • A proof for its satisfiability is an assignment of 0/1:A proof for its satisfiability is an assignment of 0/1:• We can verify it clause by clause.We can verify it clause by clause.

1

0

1

0

0

0

1

0

1

0

0

1

1

1

0

0

1

0

1

1

((xx11vvxx33vvxx1212))((xx11vvxx22vvxx88))((xx77vvxxnnvvxx1010)…)…((xx11vvxx1515vvxx1414))

More ConcretelyMore Concretely

x1

x2

x3

x4

x5

x6

x7

x8

xn-3

xn-2

xn-1

xn

x9

x10

x11

x12

x13

x14

x15

x16

……

x1

x3

x12

x1

x2

x8

x7

xn

x10

x1

x14

x15

1

0

1

0

0

0

1

0

1

0

0

1

1

1

0

0

1

0

1

1

((xx11vvxx33vvxx1212))((xx11vvxx22vvxx88))((xx77vvxxnnvvxx1010)…)…((xx11vvxx1515vvxx1414))

• An NP statement can be written as a 3SAT formula An NP statement can be written as a 3SAT formula • A proof for its satisfiability is an assignment of 0/1:A proof for its satisfiability is an assignment of 0/1:• We can verify it clause by clause.We can verify it clause by clause.• Murphy’s law! we detect an “error” only on the last clauseMurphy’s law! we detect an “error” only on the last clause


x1

x2

x3

x4

x5

x6

x7

x8

x9

x10

x11

x12

x13

x14

x15

x16

==((xx11vvxx33vvxx1212)…)…((xx11vvxx1515vvxx1414))

Y1

Y2

Y3

Y4

Y5

Y6

Y7

Y8

Y9

Y10

Y11

Y12

Y13

Y14

Y15

Y16

Y17

Y18

Y19

Y20

Y21

Y22

Y23

Y24

Y25

Y26

Y27

Y28

Y29

Y30

Y31

Ym


The PCP theorem, gives a “compiler” translating The PCP theorem, gives a “compiler” translating into into

s.t. s.t. admits super-efficient verification.admits super-efficient verification.

If If is satisfiable, then so is is satisfiable, then so is If If is is notnot satisfiable, then every assignment satisfies at satisfiable, then every assignment satisfies at

most 99% of most 99% of ’s clauses. ’s clauses.

Y13

Y22

Y31

==(y(y11vv yy1313vv yy22)(y)(y1515vv yy1919vv yy2929)()(yy2222vv yy1313vv yy2121)…()…(yy44vv y y3131v v yy2424))

is unSATis unSAT is <99% SATis <99% SAT

is SATis SAT is SATis SAT

1

0

0

1

0

0

1

1

0

1

0

1

0

0

0

0

1

1

1

1

1

0

1

1

0

0

0

1

1

1

0

0

x1

x2

x3

x4

x5

x6

x7

x8

x9

x10

x11

x12

x13

x14

x15

x16

==((xx11vvxx33vvxx1212)…)…((xx11vvxx1515vvxx1414))

Y1

Y2

Y3

Y4

Y5

Y6

Y7

Y8

Y9

Y10

Y11

Y12

Y13

Y14

Y15

Y16

Y17

Y18

Y19

Y20

Y21

Y22

Y23

Y24

Y25

Y26

Y27

Y28

Y29

Y30

Y31

ym


Conclusion [AS,ALMSS] “gap-SAT is NP-hard”Conclusion [AS,ALMSS] “gap-SAT is NP-hard”

Given a 3-SAT formula Given a 3-SAT formula it is NP-hard to decide whether: it is NP-hard to decide whether: is satisfiableis satisfiable Every assignment satisfies at most 99% of Every assignment satisfies at most 99% of ’s clauses. ’s clauses.

==(y(y11vv yy1313vv yy22)(y)(y1515vv yy1919vv yy2929)()(yy2222vv yy1313vv yy2121)…()…(yy44vv y y3131v v yy2424))



This is an Inapproximability Result!This is an Inapproximability Result!

• Hardness for gap-3SAT, shows that it is NP-hard to approximate the Hardness for gap-3SAT, shows that it is NP-hard to approximate the following optimization problem:following optimization problem:Max-3-SATMax-3-SAT: : Given a 3SAT formula, Given a 3SAT formula, ==((yy11vvyy33vvyy1212)…)…((yy11vvyy1515vvyy1414),),find the maximal fraction of satisfiable clauses.find the maximal fraction of satisfiable clauses.

• A random assignment easily satisfies 7/8A random assignment easily satisfies 7/8• Håstad proved there’s no better algorithmHåstad proved there’s no better algorithm



Some Brief HistorySome Brief History

• Context: Interactive proofs Context: Interactive proofs • Connection to Approximation [FGLSS `91]Connection to Approximation [FGLSS `91]• PCP Theorem: [AS, ALMSS `92]PCP Theorem: [AS, ALMSS `92]• Immediately for many problems, e.g. vertex Immediately for many problems, e.g. vertex

cover, max-cut, metric-TSP, max-3SAT, cover, max-cut, metric-TSP, max-3SAT, bounded-degree-clique, … [PY `91]bounded-degree-clique, … [PY `91]

• Boom of hardness of approximation resultsBoom of hardness of approximation results• Emphasis on Emphasis on

• Better PCP parametersBetter PCP parameters • TightTight inapproximability results inapproximability results

Tighter ResultsTighter Results• […,BGLR,FK,BS]: […,BGLR,FK,BS]: Better reductions with explicit Better reductions with explicit

constantsconstants• [ BGS [ BGS ‘95‘95]:]: Introduced the Introduced the Long-CodeLong-Code. .

e.g. for VertexCover:e.g. for VertexCover: 1.0681.068, for Max-CUT: , for Max-CUT: 1.0141.014• [Håstad [Håstad 96-9796-97]:]: Clique is NP-hard to approximate to Clique is NP-hard to approximate to

within nwithin n1-1-; Optimal gap for 3-SAT and for Linear ; Optimal gap for 3-SAT and for Linear equations. equations.

• Using Fourier analysis of the “marvelous” Using Fourier analysis of the “marvelous” Long-Long-CodeCode, and a, and a Stronger PCP Stronger PCP [Raz [Raz ‘95‘95]]

Hardness factor for VertexCover :Hardness factor for VertexCover : 1.1661.166, for Max-, for Max-CUT:CUT: 1.0621.062

My WorkMy Work

• The Biased Long-Code: a generalization of the Long-Code. The Biased Long-Code: a generalization of the Long-Code. [D.,Safra[D.,Safra `02`02]]

• New perspectives on the Long Code yielding powerful new New perspectives on the Long Code yielding powerful new techniques.techniques.• Analysis of influence of variables on Boolean FunctionsAnalysis of influence of variables on Boolean Functions• Extremal Set Combinatorics. Extremal Set Combinatorics.

• New stronger enhancements of the PCP theorem, New stronger enhancements of the PCP theorem, [e.g. the [e.g. the Layered PCP in DGKR ’02, DRS ‘02]Layered PCP in DGKR ’02, DRS ‘02]

• Leading to best-known inapproximability results forLeading to best-known inapproximability results for• Vertex-Cover Vertex-Cover Hardness for vertex coverHardness for vertex cover 1.3671.367. . [D.,Safra[D.,Safra `02`02]]• Approximate Hypergraph Coloring Approximate Hypergraph Coloring Approximate hypergraph Approximate hypergraph

coloring. coloring. [D.,Regev,Smyth [D.,Regev,Smyth `02`02]]• Hypergraph Vertex Cover Hypergraph Vertex Cover [D.; D.,Guruswami,Khot; [D.; D.,Guruswami,Khot;

D.,Guruswami,Khot,Regev D.,Guruswami,Khot,Regev `02`02]]

• Proving hardness for gap-VC, we translate Proving hardness for gap-VC, we translate into G and into G and then prove 2 things:then prove 2 things:

I.I. II.II.

• The hard part of the proof is part The hard part of the proof is part IIII, showing that , showing that Every VC in G of size < (1+c)k can be Every VC in G of size < (1+c)k can be decodeddecoded into a into a satisfying assignment for satisfying assignment for . .

• In standard coding theory, we encode n bits by m bits (m>n), and In standard coding theory, we encode n bits by m bits (m>n), and are able to recover “somewhat corrupt” codewords.are able to recover “somewhat corrupt” codewords.

““Every word, if close enough, we can decode”Every word, if close enough, we can decode”

• In In combinatorial decodingcombinatorial decoding, we encode an assignment for , we encode an assignment for by a by a vertex cover in G and are able to recover “somewhat corrupt” vertex vertex cover in G and are able to recover “somewhat corrupt” vertex covers. covers.

““Every VC, if small enough, we can decode”Every VC, if small enough, we can decode”

VC(G) > (1+c)kVC(G) > (1+c)k


is < 99% SATis < 99% SAT

““Combinatorial Decoding”Combinatorial Decoding”

• Starting Point: the PCP theoremStarting Point: the PCP theorem1.1. Enhance itEnhance it

2.2. Apply the Long-Code on small sub-components.Apply the Long-Code on small sub-components.

• The hardest part of these works is the The hardest part of these works is the interplay combining these two partsinterplay combining these two parts

PCPPCPEnhancedEnhanced

PCPPCPLong-CodeLong-Code

The Underlying Structure The Underlying Structure

Vertex CoverVertex Cover

• A very loose outline of the construction:A very loose outline of the construction:

• A satisfying assignment can be A satisfying assignment can be encoded encoded into a vertex-cover.into a vertex-cover.• A vertex-cover for the graph is a vertex cover in each A vertex-cover for the graph is a vertex cover in each HH . .• DecodeDecode each small vertex cover in each small vertex cover in H H into a value for the into a value for the

underlying y variable. underlying y variable. • Then, show consistency between these values. Then, show consistency between these values. • Combinatorial Question:Combinatorial Question: Construct such a graph Construct such a graph H .H .

= = (y(y11vv yy1313vv yy22) (y) (y1515vv yy1919vv yy2929) () (yy2222vv yy1313vv yy2121) … () … (yy44vv y y3131v v yy2424))

yy11 yy22 yy33 yymm

Sub-Goal:Sub-Goal: Construct a graph Construct a graph HH

Such that, Such that,

1.1. Each Each value in value in {1,2,..,R} {1,2,..,R} correspondscorresponds to a small to a small vertex cover for vertex cover for HH (i.e. of size (i.e. of size k ~ ½k ~ ½¢¢VV).).

2.2. EveryEvery vertex cover for vertex cover for HH , if smaller than , if smaller than (2-(2-))¢¢kk roughly correspondsroughly corresponds to a single value in to a single value in {1,2,..,R}{1,2,..,R}..

• Technique:Technique: • Biased Long-Code, Biased Long-Code,

• Analysis of influence of variables on Boolean functions, Analysis of influence of variables on Boolean functions,

• Erdös-Ko-Rado theorems on intersecting families of subsets.Erdös-Ko-Rado theorems on intersecting families of subsets.

Long-Code of Long-Code of RR

R elements, can be most concisely R elements, can be most concisely

encoded by log R bits.encoded by log R bits. Seeking redundancy properties: we use Seeking redundancy properties: we use

many more bits in the encoding.many more bits in the encoding. The Long-Code is the most redundantThe Long-Code is the most redundant

way, using 2way, using 2RR bits. bits.

• One bit for every subset of One bit for every subset of [R][R]

Long-Code of Long-Code of R, R, LC:[R]LC:[R]{0,1}{0,1}22RR

11 22 RR. . .. . .

• One bit for every subset of One bit for every subset of [R][R]

• How do we encode the element How do we encode the element ii[R]?[R]?

(What’s the value of LC(i)?)(What’s the value of LC(i)?)

Long-Code of Long-Code of R, R, LC:[R]LC:[R]{0,1}{0,1}22RR

0 0 1 1 1

11 22 RR. . .. . .

• Endow the bits with the product distribution:Endow the bits with the product distribution:

For each subset For each subset FF, , pp((FF) = p) = p||FF||(1-p)(1-p)|R\|R\FF||

• Roughly: take only subsets whose size is pRoughly: take only subsets whose size is pR.R.

The p-The p-Biased Biased Long-CodeLong-Code

The Disjointness Graph of the The Disjointness Graph of the Biased Long-CodeBiased Long-Code

11 22 . . .. . . RR

What is a codeword?What is a codeword?

• A codeword is a vertex coverA codeword is a vertex cover• The complement of a vertex-cover is always an independent set. The complement of a vertex-cover is always an independent set. • In this graph, an independent set is an In this graph, an independent set is an intersecting family of subsets.intersecting family of subsets.• Claim: a long-code codeword, i.e. {all subsets containing i} is a Claim: a long-code codeword, i.e. {all subsets containing i} is a largest largest independent setindependent set, ,

and its complement, a and its complement, a smallestsmallest vertex cover vertex cover..• Maximal Intersecting Families of Subsets: Maximal Intersecting Families of Subsets: Erdös-Ko-Rado ’61Erdös-Ko-Rado ’61• Lemma: The Lemma: The pp size of an intersecting family is size of an intersecting family is p (proof using “shadows” [Kruskal `63, p (proof using “shadows” [Kruskal `63,

Katona `68])Katona `68])

• Much more difficult to prove: Any vertex cover whose size is < 1-pMuch more difficult to prove: Any vertex cover whose size is < 1-p22 is is “decodable”“decodable” into a into a value in 1,…,R. value in 1,…,R. (combinatorial decoding)(combinatorial decoding)

• Using: the complete characterization of maximal intersecting families by Using: the complete characterization of maximal intersecting families by Ahlswede and Ahlswede and Khachatrian ’97Khachatrian ’97, and, and Friedgut’s Theorem on when Boolean Functions are Juntas, etc.Friedgut’s Theorem on when Boolean Functions are Juntas, etc.

i i 22 R R

VC(G)=1-p = 2/3VC(G)=1-p = 2/3

i i 22 R R

VC(G)< (1-p2) = 8/9

We constructed a graph s.t., We constructed a graph s.t.,

1.1. Each Each value in value in {1,2,..,R} {1,2,..,R} correspondscorresponds to a small vertex to a small vertex cover for cover for HH (i.e. of size (i.e. of size kk).).

2.2. EveryEvery vertex cover for vertex cover for HH , if smaller than , if smaller than (4/3)(4/3)¢¢kk roughly roughly correspondscorresponds to a single value in to a single value in {1,2,..,R}{1,2,..,R}..

Now we can plug it into the whole construction… Now we can plug it into the whole construction…

(2-(2-)k)k ??? ???

i i 22 R R

VC(G)=1-p = 2/3VC(G)=1-p = 2/3

i i 22 R R

VC(G)< (1-p2) = 8/9

Future DirectionsFuture Directions

• Finding the true threshold: (stronger Finding the true threshold: (stronger “combinatorial decoding”)“combinatorial decoding”)• Factor 2 inapproximability for Vertex CoverFactor 2 inapproximability for Vertex Cover• Other problems: approximate Other problems: approximate ccoolloorriinngg, etc., etc.

• Simplification of PCP, locally testable codes.Simplification of PCP, locally testable codes.• Decoding in completely different contexts, with Decoding in completely different contexts, with

applications for database privacy.applications for database privacy.

ThanksThanks

Documents

PCP and Inapproximability Irit Dinur NEC. Example: the Minimum Vertex Cover functionExample: the Minimum Vertex Cover function Facts: 1. Best algorithm