Carlo Lombardi, June 2008 Theoretical Computer Science
Hardness of Approximation
A brief survey of Inapproximability theory
for NP optimization problems
1
Carlo Lombardi, June 2008 Theoretical Computer Science
Overview
2
Optimization problem:
Definition
NP Optimization
•Approximability and Inapproximability
Approximation-Preserving Reduction
Gap Problems, Karp reduction, PCP Theorem
Probabilistic Verification
Carlo Lombardi, June 2008 Theoretical Computer Science
Optimization problem
3
),(min)(
),(max)(
)(
)(
yxFxOPT
yxFxOPT
xDy
xDy
“find the best solution from all feasible solution”
•x= instance of input•y= “witness” solution•D(x)= set of all feasible solution•F(x,y)=real-valued function wich assigns a “score” to y
If both y belongs to D(X)
F(x,y)are polynomial time computable then OPT(x) belongs to NPO
class
Carlo Lombardi, June 2008 Theoretical Computer Science
Approximation ratio
4
Consider a map:
)()(:..)( xDxAxtsxAx This map is said to approximate OPT(x) within a factor r(x)>=1 if:
• The best such r(x) is said approximation ratio• If A is p-time computable we say that OPT(x) is approximable within a factor of r(x)•If there are no A p-time computable under some complexity theoretic hipothesis then r(x) is said inapproximability ratio
OPT(x) <= r(x) F(x,A(x)) if OPT(x) is a MAX-P
F(x,A(x)) <= r(x) OPT(x) if OPT(x) is a MIN-P
Carlo Lombardi, June 2008 Theoretical Computer Science
Example: Set Cover Problem
5
Let be:
• x : a polynomial size set system S
•y : subsystem S1 S
• iff )(xDy SS 1
Find s.t. is minimized
)(xDy 1S
Feige has shown that that the set cover cannot be approximate within a factor of
)(0ln)1( loglog nnDTIMENPunlessPinforS
Sln
is the approximation boundary for Set Cover Problem
but unfortunately…this is only an inapproximability result for a specific problem…not a more general theory…
Carlo Lombardi, June 2008 Theoretical Computer Science
Question
6
In which way we can obtain inapproximability results
for optimization problems?
Roughly speaking, how we can find a lower bound for approximation ratio
of optimization problems?
Carlo Lombardi, June 2008 Theoretical Computer Science
Timeline for a more general inapproximability theory
7
• 1972 – Graham: exact bounds on the performance of various bin packing heuristics•1974 – Jhonson: Subest Sum, Set Cover, MAX k-SAT bounds
•1976 – Shani & Gonzalez: TSP problem bound Garey & Jhonson: Gap amplification technique for Chromatic Number of a graph bound
•1991 – Feige: MIP for MIN-Clique bound Papadimitriou & Yannakakis:Using L reduction (app-reserving)
•1992 – Arora et al.: Using PCPs for MAX-3SAT Problem bound
“EUREKA!!!”
Carlo Lombardi, June 2008 Theoretical Computer Science
Inapproximability results:the main ingredients
8
Carlo Lombardi, June 2008 Theoretical Computer Science
Approximation-Preserving Reduction (1/2)
9
If A Cook reduces to B we can state that
“the hardness of B follows from the hardness of A”
but we CANNOT state that
“if A is hard to approximate then B is hard to approximate”
To ensure reducing hardness of approximation we need a new definition of reduction…
Carlo Lombardi, June 2008 Theoretical Computer Science
Approximation-Preserving Reduction (2/2)
10
Let be:
• F1(x,y) and F2(x ’,y ’) to be optimized for y and y ‘ • OPT1(x) and OPT2(x ‘) the corresponding optimum
a KARP-LEVIN reduction involves two polynomial- time maps f and g s.t.:
12
21
)','(
)('
OPTyOPTyx
OPTxfxOPTxg
f
(Instance transformation)
(Witness transformation)
Let be:• opt1 = OPT1(x)• opt2 = OPT2(f(x))• appr1 = F1(x,g(f(x),y ’))• appr1 = F2(f(x),y ‘)
An approximation-preserving reduction scheme is a relation between this four entities that express the following:
“If appr2 well approximate opt2 then appr1 well approximate opt1 “
Carlo Lombardi, June 2008 Theoretical Computer Science
Gap Problem
11
Let be :
•OPT : a maximizationproblem•Tl(x) : a lower bound for OPT
•Tu(x) : an upper bound for OPT
•Both Tl(x) and Tu(x) are p-time computable in x
If we can efficiently approximate OPT(x) within a factor better than r(x)= Tu(x)/ Tl(x) then we can solve with only additive polynomial time also the so called GAP PROBLEM:
1 if OPT(x) >= Tu(x)
Gap(OPT, Tl, Tu ) 0 if OPT(x) <= Tl(x)
ANY if Tl(x)< OPT(x) < Tu(x)
If OPT is a minimization problem the roles of 0 and 1 in the above definition get exchanged
Carlo Lombardi, June 2008 Theoretical Computer Science
From languages to gap problem
12
• We can use Karp reduction to map a language into a Gap Problem
•A such reduction is a polynomial time computable map f from to input instances of OPT (max-problem) s.t.
*L
))(())((
))(())((
xfTxfOPTthenLxif
xfTxfOPTthenLxif
l
u
*
YN
Tu(f(x))
Tl(f(x))
Carlo Lombardi, June 2008 Theoretical Computer Science
PCP Theorem
13
Given an and a language there exists a polynomial-time computable function such that:
• if then f(x) is a formula in which all disjunctions are simultaneously satisfable
•if then f(x) is a formula in which one can satisfy at most 1-ε fraction of all clauses
0 NPL}3{: * formulasCNFf
Lx
Lx
•Considering we are discussing about optimization problem we can restate the Theorem:
Any language in NP is Karp reducible to Gap(Max-3SAT, 1-ε, 1)
where
Max-3SAT(φ)= maxy F(φ,y) being φ a 3CNF
Carlo Lombardi, June 2008 Theoretical Computer Science
Karp reduction & gap problem
14
• Karp reduction are approximation-preserving reduction
•We can reduce a gap problem G to a gap problem G’ preserving approximation results
Consider •two maximization problem OPT1 and OPT2 with bounds respectively T l , Tu , T’l , T’u and •A function f p-time computable from input instances of OPT1 to input instances of OPT2 s.t.:
1. if OPT1 (x)<= Tl (x) then OPT(f(x))<= Tl (f(x))2. if OPT1 (x)>= Tu(x) then OPT(f(x))>= Tu (f(x))
f is a Karp reduction from Gap(OPT1, T l, T l) to Gap(OPT2, T’ l, T’ l)
Gap(OPT1, T l, T u) is NP-Hard Gap(OPT2, T’ u, T’ u) is NP-Hard
Carlo Lombardi, June 2008 Theoretical Computer Science
The main philosophy of PCP Theory
15
Carlo Lombardi, June 2008 Theoretical Computer Science
Question
16
How it’s possible to compute efficiently gap problems?
Carlo Lombardi, June 2008 Theoretical Computer Science 17
Probabilistic Proof System
• A proof system consists of a verifier V and a prover P• Given a stastement x, such as “φ is satisfable”
•P produces a candidate proof y for the statement φ•V read the pair (φ,y) and either accepts or reject the proof y for φ
• Any proof system have two property:
•COMPLETENESS: if x is true exists y s.t. V(x,y) accepts
•SOUNDNESS: if x is false for every y V(x,y) rejects
A language L is in NP if there is a deterministic polynomial time verifier V and a polynomial p s.t.:
rejectsyxVxpytsyLx
acceptsyxVANDxpytsyLx
),())((..(
),()(..:
Carlo Lombardi, June 2008 Theoretical Computer Science 18
Probabilistic Oracle Machine (1/2)
•What’s happen if we allow V to be randomized?
•Let be:
My a probabilistic RAM with oracle y and random string r
M is said to accept a language L with completness α and soudness β (1>= α >β>0) iff
•if then ther is a y s.t. Probr(My(x,r)=1)>= α
•if then for every y it holds that Probr(My(x,r)=1)<=β
Lx
Lx
Carlo Lombardi, June 2008 Theoretical Computer Science 19
Parameters:
•Randomness: |r|•Query size: q•Completeness: α•Soundness: β
Probabilistic Oracle Machine (2/2)
The witness y written on a POM machine’s oracle tape is called Probabilistically Checkable Proof
Carlo Lombardi, June 2008 Theoretical Computer Science 20
Connection between POM and OPT problem
)(
)(
xOTPthenLxif
xOTPthenLxif
M
M
L Karp reduces to Gap(OPTM, β, α)
If L is NP-Hard, approximating OPTM within a factor better than α/β is also NP-Hard