Hardness of approximation

Carlo Lombardi, June 2008 Theoretical Computer Science

Hardness of Approximation

A brief survey of Inapproximability theory

for NP optimization problems

1


Overview

2

Optimization problem:

Definition

NP Optimization

•Approximability and Inapproximability

Approximation-Preserving Reduction

Gap Problems, Karp reduction, PCP Theorem

Probabilistic Verification


Optimization problem

3

),(min)(

),(max)(

)(

)(

yxFxOPT

yxFxOPT

xDy

xDy

“find the best solution from all feasible solution”

•x= instance of input•y= “witness” solution•D(x)= set of all feasible solution•F(x,y)=real-valued function wich assigns a “score” to y

If both y belongs to D(X)

F(x,y)are polynomial time computable then OPT(x) belongs to NPO

class


Approximation ratio

4

Consider a map:

)()(:..)( xDxAxtsxAx This map is said to approximate OPT(x) within a factor r(x)>=1 if:

• The best such r(x) is said approximation ratio• If A is p-time computable we say that OPT(x) is approximable within a factor of r(x)•If there are no A p-time computable under some complexity theoretic hipothesis then r(x) is said inapproximability ratio

OPT(x) <= r(x) F(x,A(x)) if OPT(x) is a MAX-P

F(x,A(x)) <= r(x) OPT(x) if OPT(x) is a MIN-P


Example: Set Cover Problem

5

Let be:

• x : a polynomial size set system S

•y : subsystem S1 S

• iff )(xDy SS 1

Find s.t. is minimized

)(xDy 1S

Feige has shown that that the set cover cannot be approximate within a factor of

)(0ln)1( loglog nnDTIMENPunlessPinforS

Sln

is the approximation boundary for Set Cover Problem

but unfortunately…this is only an inapproximability result for a specific problem…not a more general theory…


Question

6

In which way we can obtain inapproximability results

for optimization problems?

Roughly speaking, how we can find a lower bound for approximation ratio

of optimization problems?


Timeline for a more general inapproximability theory

7

• 1972 – Graham: exact bounds on the performance of various bin packing heuristics•1974 – Jhonson: Subest Sum, Set Cover, MAX k-SAT bounds

•1976 – Shani & Gonzalez: TSP problem bound Garey & Jhonson: Gap amplification technique for Chromatic Number of a graph bound

•1991 – Feige: MIP for MIN-Clique bound Papadimitriou & Yannakakis:Using L reduction (app-reserving)

•1992 – Arora et al.: Using PCPs for MAX-3SAT Problem bound

“EUREKA!!!”


Inapproximability results:the main ingredients

8


Approximation-Preserving Reduction (1/2)

9

If A Cook reduces to B we can state that

“the hardness of B follows from the hardness of A”

but we CANNOT state that

“if A is hard to approximate then B is hard to approximate”

To ensure reducing hardness of approximation we need a new definition of reduction…


Approximation-Preserving Reduction (2/2)

10

Let be:

• F1(x,y) and F2(x ’,y ’) to be optimized for y and y ‘ • OPT1(x) and OPT2(x ‘) the corresponding optimum

a KARP-LEVIN reduction involves two polynomial- time maps f and g s.t.:

12

21

)','(

)('

OPTyOPTyx

OPTxfxOPTxg

f

(Instance transformation)

(Witness transformation)

Let be:• opt1 = OPT1(x)• opt2 = OPT2(f(x))• appr1 = F1(x,g(f(x),y ’))• appr1 = F2(f(x),y ‘)

An approximation-preserving reduction scheme is a relation between this four entities that express the following:

“If appr2 well approximate opt2 then appr1 well approximate opt1 “


Gap Problem

11

Let be :

•OPT : a maximizationproblem•Tl(x) : a lower bound for OPT

•Tu(x) : an upper bound for OPT

•Both Tl(x) and Tu(x) are p-time computable in x

If we can efficiently approximate OPT(x) within a factor better than r(x)= Tu(x)/ Tl(x) then we can solve with only additive polynomial time also the so called GAP PROBLEM:

1 if OPT(x) >= Tu(x)

Gap(OPT, Tl, Tu ) 0 if OPT(x) <= Tl(x)

ANY if Tl(x)< OPT(x) < Tu(x)

If OPT is a minimization problem the roles of 0 and 1 in the above definition get exchanged


From languages to gap problem

12

• We can use Karp reduction to map a language into a Gap Problem

•A such reduction is a polynomial time computable map f from to input instances of OPT (max-problem) s.t.

*L

))(())((

))(())((

xfTxfOPTthenLxif

xfTxfOPTthenLxif

l

u

*

YN

Tu(f(x))

Tl(f(x))


PCP Theorem

13

Given an and a language there exists a polynomial-time computable function such that:

• if then f(x) is a formula in which all disjunctions are simultaneously satisfable

•if then f(x) is a formula in which one can satisfy at most 1-ε fraction of all clauses

0 NPL}3{: * formulasCNFf

Lx

Lx

•Considering we are discussing about optimization problem we can restate the Theorem:

Any language in NP is Karp reducible to Gap(Max-3SAT, 1-ε, 1)

where

Max-3SAT(φ)= maxy F(φ,y) being φ a 3CNF


Karp reduction & gap problem

14

• Karp reduction are approximation-preserving reduction

•We can reduce a gap problem G to a gap problem G’ preserving approximation results

Consider •two maximization problem OPT1 and OPT2 with bounds respectively T l , Tu , T’l , T’u and •A function f p-time computable from input instances of OPT1 to input instances of OPT2 s.t.:

1. if OPT1 (x)<= Tl (x) then OPT(f(x))<= Tl (f(x))2. if OPT1 (x)>= Tu(x) then OPT(f(x))>= Tu (f(x))

f is a Karp reduction from Gap(OPT1, T l, T l) to Gap(OPT2, T’ l, T’ l)

Gap(OPT1, T l, T u) is NP-Hard Gap(OPT2, T’ u, T’ u) is NP-Hard


The main philosophy of PCP Theory

15


Question

16

How it’s possible to compute efficiently gap problems?

Carlo Lombardi, June 2008 Theoretical Computer Science 17

Probabilistic Proof System

• A proof system consists of a verifier V and a prover P• Given a stastement x, such as “φ is satisfable”

•P produces a candidate proof y for the statement φ•V read the pair (φ,y) and either accepts or reject the proof y for φ

• Any proof system have two property:

•COMPLETENESS: if x is true exists y s.t. V(x,y) accepts

•SOUNDNESS: if x is false for every y V(x,y) rejects

A language L is in NP if there is a deterministic polynomial time verifier V and a polynomial p s.t.:

rejectsyxVxpytsyLx

acceptsyxVANDxpytsyLx

),())((..(

),()(..:


Probabilistic Oracle Machine (1/2)

•What’s happen if we allow V to be randomized?

•Let be:

My a probabilistic RAM with oracle y and random string r

M is said to accept a language L with completness α and soudness β (1>= α >β>0) iff

•if then ther is a y s.t. Probr(My(x,r)=1)>= α

•if then for every y it holds that Probr(My(x,r)=1)<=β

Lx

Lx


Parameters:

•Randomness: |r|•Query size: q•Completeness: α•Soundness: β

Probabilistic Oracle Machine (2/2)

The witness y written on a POM machine’s oracle tape is called Probabilistically Checkable Proof


Connection between POM and OPT problem

)(

)(

xOTPthenLxif

xOTPthenLxif

M

M

L Karp reduces to Gap(OPTM, β, α)

If L is NP-Hard, approximating OPTM within a factor better than α/β is also NP-Hard

Technology

Hardness of approximation