Upload
callie-gutierrez
View
216
Download
0
Embed Size (px)
Citation preview
Average-case Complexity
Luca TrevisanUC Berkeley
Distributional Problem
<P,D>
P computational problem– e.g. SAT
D distribution over inputs– e.g. n vars 10n clauses
Positive Results:• Algorithm that solves P efficiently on most
inputs– Interesting when P useful problem, D
distribution arising “in practice”
Negative Results:• If <assumption>, then no such algorithm– P useful, D natural• guide algorithm design
– Manufactured P,D, • still interesting for crypto, derandomization
Positive Results:• Algorithm that solves P efficiently on most
inputs– Interesting when P useful problem, D
distribution arising “in practice”
Negative Results:• If <assumption>, then no such algorithm– P useful, D natural• guide algorithm design
– Manufactured P,D, • still interesting for crypto, derandomization
Holy Grail
If there is algorithm A that solves P efficiently on most inputs from D
Then there is an efficient worst-case algorithm for [the complexity class] P [belongs to]
Part (1)
In which the Holy Grail proves elusive
The Permanent
Perm (M) := Ss Pi M(i,s(i))
Perm() is #P-complete
Lipton (1990): If there is algorithm that solves Perm()
efficiently on most random matrices, Then there is an algorithm that solves it
efficiently on all matrices (and BPP=#P)
Lipton’s Reduction
Suppose operations are over finite field of size >n
A is good-on-average algorithm (wrong on < 1/(10(n+1)) fraction of matrices)
Given M, pick random X, compute A(M+X), A(M+2X),…,A(M+(n+1)X)
Whp the same as Perm(M+X),Perm(M+2X),…,Perm(M+(n+1)X)
Lipton’s Reduction
Given Perm(M+X),Perm(M+2X),…,Perm(M+
(n+1)X)
Find univariate degree-n polynomial p such thatp(t) = Perm(M+tX) for all t
Output p(0)
Improvements / Generalizations
• Can handle constant fraction of errors[Gemmel-Sudan]
• Works for PSPACE-complete, EXP-complete,…[Feigenbaum-Fortnow, Babai-Fortnow-Nisan-Wigderson]Encode the problem as a polynomial
Strong Average-Case Hardness
• [Impagliazzo, Impagliazzo-Wigderson] Manufacture problems in E, EXP, such that– Size-t circuit correct on ½ + 1/t inputs implies– Size poly(t) circuit correct on all inputs
Motivation:[Nisan-Wigderson]
P=BPP if there is problem in E of exponential average-case complexity
Strong Average-Case Hardness
• [Impagliazzo, Impagliazzo-Wigderson] Manufacture problems in E, EXP, such that– Size-t circuit correct on ½ + 1/t inputs
implies– Size poly(t) circuit correct on all inputs
Motivation:[Impagliazzo-Wigderson]
P=BPP if there is problem in E of exponential average worst-case complexity
Open Question 1
• Suppose there are worst-case intractable problems in NP
• Are there average-case intractable problems?
Strong Average-Case Hardness
• [Impagliazzo, Impagliazzo-Wigderson] Manufacture problems in E, EXP, such that– Size-t circuit correct on ½ + 1/t inputs implies– Size poly(t) circuit correct on all inputs
• [Sudan-T-Vadhan]– IW result can be seen as coding-theoretic
– Simpler proof by explicitly coding-theoretic ideas
Encoding Approach
• Viola proves that an error-correcting code cannot be computed in AC0
• The exponential-size error-correcting code computation not possible in PH
Problem-specific Approaches?
[Ajtai]
• Proves that there is a lattice problem such that:
– If there is efficient average-case algorithm
– There is efficient worst-case approximation algorithm
Ajtai’s Reduction
• Lattice Problem– If there is efficient average-case algorithm– There is efficient worst-case approximation
algorithm
The approximation problem is in NPIcoNPNot NP-hard
Holy Grail
• Distributional Problem:– If there is efficient average-case algorithm– P=NP
(or NP in BPP, or NP has poly-size circuits,…)
Already seen: no “encoding” approach works
Can extensions of Ajtai’s approach work?
A Class of Approaches
• L problem in NP, D distribution of inputs• R reduction of SAT to <L,D>:
• Given instance f of SAT,– R produces instances x1,…,xk of L, each distributed
according to D– Given L(x1),…,L(x1), R is able to decide f
If there is good-on-average algorithn for <L,D>, we solve SAT in polynomial time
[cf. Lipton’s work on Permanent]
A Class of Approaches
• L,W problems in NP, D (samplable) distribution of inputs
• R reduction of W to <L,D>
• Given instance w of W,– R produces instances x1,…,xk of L, each distributed
according to D– Given L(x1),…,L(x1), R is able to decide w
If there is good-on-average algorithm for <L,D>, we solve W in polynomial time;
Can W be NP-complete?
A Class of Approaches
• Given instance w of W,– R produces instances x1,…,xk of L, each
distributed according to D– Given L(x1),…,L(x1), R is able to decide w
Given good-on-average algorithm for <L,D>, we solve W in polynomial time;
If we have such reduction, and W is NP-complete, we have Holy Grail!
Feigenbaum-Fortnow: W is in “coNP”
Feigenbaum-Fortnow
• Given instance w of W,– R produces instances x1,…,xk of L, each
distributed according to D– Given L(x1),…,L(x1), R is able to decide w
• Using R, Feigenbaum-Fortnow design a 2-round interactive proof with advice for coW
• Given w, Prover convinces Verifier that R rejects w after seeing L(x1),…,L(x1)
Feigenbaum-Fortnow
• Given instance w of W,– R produces instances x of L distributed as in D– w in L iff x in L
Suppose we know PrD[ x in L]= ½
VP
w
R(w) = x1
R(w) = x2
. . .R(w) = xm
x1, x2,. . . , xm
(Yes,w1),No,. . . , (Yes, wm)
Accept iff all simulations of R rejectand m/2 +/- sqrt(m) answers are certified Yes
Feigenbaum-Fortnow
• Given instance w of W, p:= Pr[ xi in L]
– R produces instances x1,…,xk of L, each distrib. according to D
– Given L(x1),…,L(xk), R is able to decide wV
wR(w) -> x1
1,…,xk
1 . . .
R(w) -> x1m,
…,xkm
P
x11,…,xk
m
(Yes,w11),…,NO
Accept iff -pkm +/- sqrt(pkm) YES with certificates-R rejects in each case
Generalizations
• Bogdanov-Trevisan: arbitrary non-adaptive reductions
• Main Open Question:What happens with adaptive reductions?
Open Question 1
Prove the following:
Suppose: W,L are in NP, D is samplable distribution,
R is poly-time reduction such that– If A solves <L,D> on 1-1/poly(n) frac of
inputs– Then R with oracle A solves W on all inputs
Then W is in “coNP”
By the Way
• Probably impossible by current techniques:If NP not contained in BPPThere is a samplable distribution D and an NP
problem L Such that <L,D> is hard on average
By the Way
• Probably impossible by current techniques:If NP not contained in BPPThere is a samplable distribution D and an NP
problem L Such that for every efficient A A makes many mistakes solving L on D
By the Way
• Probably impossible by current techniques:If NP not contained in BPPThere is a samplable distribution D and an NP problem
L Such that for every efficient A A makes many mistakes solving L on D
• [Guttfreund-Shaltiel-TaShma] Prove:If NP not contained in BPPFor every efficient A There is a samplable distribution D Such that A makes many mistakes solving SAT on D
Part (2)
In which we amplify average-case complexity and we discuss a short paper
Revised Goal
• Proving“If NP contains worst-case intractable problems, then NP contains average-case intractable problems”
Might be impossible
• Average-case intractability comes in different quantitative degrees
• Equivalence?
Average-Case Hardness
What does it mean for <L,D> to be hard-on-average?
Suppose A is efficient algorithm Sample x ~ DThen A(x) is noticeably likely to be wrong
How noticeably?
Average-Case Hardness Amplification
Ideally:
• If there is <L,Uniform>, L in NP, such that every poly-time algorithm (poly-size circuit) makes > 1/poly(n) mistakes
• Then there is <L’,Uniform>, L’ in NP, such that every poly-time algorithm (poly-size circuit) makes > ½ - 1/poly(n) mistakes
Amplification
“Classical” approach: Yao’s XOR Lemma
Suppose: for every efficient APrD [ A(x) = L(x) ] < 1- d
Then: for every efficient A’ PrD [ A’(x1,…,xk) = L(x1) xor … xor L(xk) ]
< ½ + (1 - 2d)k + negligible
Yao’s XOR Lemma
Suppose: for every efficient APrD [ A(x) = L(x) ] < 1- d
Then: for every efficient A’ PrD [ A’(x1,…,xk) = L(x1) xor … xor L(xk) ]
< ½ + (1 - 2d)k + negligible
Note: computing L(x1) xor … xor L(xk) need not be in NP, even if L is in NP
O’Donnell Approach
Suppose: for every efficient APrD [ A(x) = L(x) ] < 1- d
Then: for every efficient A’ PrD [ A’(x1,…,xk) = g(L(x1), …, L(xk)) ]
< ½ + small(k, d)
For carefully chosen monotone function g
Now computing g(L(x1),…, L(xk)) is in NP, if L is in NP
Amplification (Circuits)
Ideally:• If there is <L,Uniform>, L in NP, such that every poly-time
algorithm (poly-size circuit) makes > 1/poly(n) mistakes• Then there is <L’,Uniform>, L’ in NP, such that every poly-
time algorithm (poly-size circuit) makes > ½ - 1/poly(n) mistakes
Achieved by [O’Donnell, Healy-Vadhan-Viola] for poly-size circuits
Amplification (Algorithms)
• If there is <L,Uniform>, L in NP, such that every poly-time algorithm makes > 1/poly(n) mistakes
• Then there is <L’,Uniform>, L’ in NP, such that every poly-time algorithm makes > ½ - 1/polylog(n) mistakes
[T]
[Impagliazzo-Jaiswal-Kabanets-Wigderson] ½ - 1/poly(n) but for PNP||
Open Question 2
Prove:
• If there is <L,Uniform>, L in NP, such that every poly-time algorithm makes > 1/poly(n) mistakes
• Then there is <L’,Uniform>, L’ in NP, such that every poly-time algorithm makes > ½ - 1/poly(n) mistakes
Completeness
• Suppose we believe there is L in NP, D distribution, such that <L,D> is hard
• Can we point to a specific problem C such that <C,Uniform> is also hard?
Completeness
• Suppose we believe there is L in NP, D distribution, such that <L,D> is hard
• Can we point to a specific problem C such that <C,Uniform> is also hard?
Must put restriction on D, otherwise assumption is the same as P != NP
Side Note
Let K be distribution such that x has probability proportional to 2-K(x)
Suppose A solves <L,K> on 1-1/poly(n) fraction of inputs of length n
Then A solves L on all but finitely many inputs
Exercise: prove it
Completeness
• Suppose we believe there is L in NP, D samplable distribution, such that <L,D> is hard
• Can we point to a specific problem C such that <C,Uniform> is also hard?
Completeness
• Suppose we believe there is L in NP, D samplable distribution, such that <L,D> is hard
• Can we point to a specific problem C such that <C,Uniform> is also hard?
Yes we can! [Levin, Impagliazzo-Levin]
Levin’s Completeness Result
• There is an NP problem C, such that
• If there is L in NP, D computable distribution, such that <L,D> is hard
• Then <C,Uniform> is also hard
Reduction
Need to define reduction that preserves efficiency on average
(Note: we haven’t yet defined efficiency on average)
R is a (Karp) average-case reduction from <A,DA> to <B,DB> if
1. x in A iff R(x) in B2. R(DA) is “dominated” by DB:
Pr[ R(DA)=y] < poly(n) * Pr [DB = y]
Reduction
R is an average-case reduction from <A, DA> to <B, DB> if
• x in A iff R(x) in B• R(DA) is “dominated” by DB:
Pr[ R(DA)=y] < poly(n) * Pr [DB = y]
Suppose we have good algorithm for <B, DB>
Then algorithm also good for <B,R(DA)>
Solving <A, DA> reduces to solving <B,R(DA)>
Reduction
If Pr[ Y=y] < poly(n) * Pr [DB = y]
and we have good algorithm for <B, DB >
Then algorithm also good for <B,Y>
Reduction works for any notion of average-case tractability for which above is true.
Levin’s Completeness Result
Follow presentation of [Goldreich]
• If <BH,Uniform> is easy on average
• Then for every L in NP, every D computable distribution, <L,D> is easy on average
BH is non-deterministic Bounded Halting: given <M,x,1t>,does M(x) accept with t steps?
Levin’s Completeness Result
BH, non-deterministic Bounded Halting: given <M,x,1t>,does M(x) accept with t steps?
Suppose we have good-on-average alg A
Want to solve <L,D>, where L solvable by NDTM M
First try: x -> <M,x, 1poly(n)>
Levin’s Completeness Result
First try: x -> <M,x, 1poly(n)>
Doesn’t work: x may have arbitrary distribution, we need target string to be nearly uniform (high entropy)
Second try: x -> <M’,C(x), 1poly(n)>Where C() is near-optimal compression alg,
M’ recover x from C(x), then runs M
Levin’s Completeness Result
Second try: x -> <M’,C(x), 1poly(n)>Where C() is near-optimal compression alg,
M’ recover x from C(x), then runs M
Works! Provided C(x) has length at mostO(log n) + log 1/PrD[x]
Possible if cumulative distribution function of D is computable.
Impagliazzo-Levin
Do the same but for all samplable distribution
Samplable distribution not necessarily efficiently compressible in coding theory sense. (E.g. output of PRG)
Hashing provides “non-constructive” compression
Complete Problems
BH with Uniform distribution
Tiling problem with Uniform distribution [Levin]
Generalized edge-coloring [Venkatesan-Levin] Matrix representability [Venkatesan-
Rajagopalan]Matrix transformation [Gurevich]. . .
Open Question 3
L in NP, M NDTM for L is specified by k bits
Levin’s reduction incurs 2k bits in fraction of “problematic” inputs(comparable to having 2k slowdown)
Limited to problems having non-deterministic algorithm of 5 bytes
Inherent?
More Reductions?
Still relatively few complete problems
Similar to study of inapproximability before Papadimitriou-Yannakakis and PCP
Would be good, as in Papadimitriou-Yannakakis, to find reductions between problems that are not known to be complete but are plausibly hard
Open Question 4
(Heard from Russell Impagliazzo)
Prove that
If 3SAT is hard on instances with n variables and 10n clauses,
Then it is also hard on instances with 12n clauses
See
• http://www.cs.berkeley.edu/~luca/average[slides, references, addendum to Bogdanov-T, coming soon]
• http://www.cs.uml.edu/~wang/acc-forum/ [average-case complexity forum]
• Impagliazzo A personal view of average-case complexityStructures’95
• Goldreich Notes on Levin’s theory of average-case complexityECCC TR-97-56
• Bogdanov-T. Average case complexityF&TTCS 2(1): (2006)