Complexity Theory

Lecture 6

Lecturer: Moni Naor

RecapLast week: • Probabilistic Complexity

1. Schwatz-Zippel2. Approximating Counting Problems

Plus• Alternation

This Week:• Non-Uniform Complexity Classes• Polynomial Time Hierarchy

Machines that Take Advice• Turing Machine with advice: extra read only tape

– Advice string• Machine M decides language L with advice A(n) if:

– X 2 L implies M(x,A(|x|) = `yes’– X L implies M(x,A(|x|) = `no’

Advice A(n) is specific to the length

Complexity Class with advice: C is a complexity class and f:N N. Language L 2 C/f(n) if there is a TM M and advice A(n)

that decides language L and |A(n)| · f(n)

Example: P/Poly(n) = [k P/nK

P/Poly• Every unary language is in P/poly

– There are undecidable languages in P/polyObservation: a language is in P/poly iff it has a

polynomial sized circuit– There exists a sequence of circuits C1, C2, … and a

polynomial p(n) such that:• circuit Cn computes f on inputs of size n • circuit Cn is of size at most p(n)

What is the power of P/polyis NP 2 P/poly?

Homework: show that if NP 2 P/log then P=NP

{1index(x) | xL} where L is undecidable

CircuitsLet B be a collection of Booleans functionsA Circuit for on n Boolean variables x1, x2, … xn over Basis B is a DAG with

– Each source is labeled with a literal– Unique sink– Each internal node is labeled with a function from B and has the appropriate indegree

• A circuit compute a function in the natural way• Which (families of) functions have circuits whose size is bounded by polynomial in the

input size?– B ={:, Ç, Æ}

Claim: all f 2 P have polynomial size circuitsProof: via the `usual’ tableau. Size of circuit T(n)S(n).

Using Oblivious Turing Machines T(n) log T(n)

Claim: most Boolean function f on n variables do not have polynomial sized circuitsFor any polynomial p(n) most functions need more p(n) gates

Need (2n/n) gates to compute all functions on n variables, which is tight

Can determine the inputs to a given gate by a log space machine

Polynomial Sized Circuits for BPP• Theorem: any f 2 BPP has a polynomial size circuit

– There exists a sequence of circuits C1, C2, … and a polynomial p(n) such that:

• circuit Cn computes f on inputs of size n • circuit Cn is of size at most p(n)

Several proofs (various derandomization methods):• Construction of hitting set (for f 2 RP)

– Claim that a sequence r1, r2, … rn exists where for each x 2 L at least one ri says `yes’

• Simulating large sample spaces by small ones• Amplification

– Reduce the error so that a single assignment will be good for all x

Derandomization ITheorem: any f 2 BPP has a polynomial size

circuitSimulating large sample spaces• Want to find a small collection of strings on

which the PTM behaves as on the large collection – If the PTM errs with probability at most , then

should err on at most + of the small collection• Choose m random strings• For input x event Ax is more than (+) of

the m strings fail the PTM

Pr[Ax] · e-22m < 2-2n

Pr[[x Ax] · x Pr[Ax] < 2n 2-2n=1

Good 1-

Bad

Collection that should resemble probability of success on ALL inputs

Chernoff

Derandomization

A major research question:• How to make the construction of

– Small Sample space `resembling’ large one– Hitting sets

Efficient.

Successful approach: randomness from hardness– (Cryptographic) pseudo-random generators– Complexity oriented pseudo-random generators

Alternation

• Non-determinism: a configuration leads to acceptance if there is an accepting leaf in the computation – Similar to Or

• Can also consider leads to acceptance if all leaves are accepting– Similar to And

• What if we alternate between the modes

Alternating Turing Machines

An Alternating Turing Machines (ATM) is a non-deterministic TM whose state S ={SAND [ SOR} are partitioned into two sets

• For input x consider the tree of computations where each node is a configuration

• The ATM is accepting if the root leads to an accepting configuration:– A leaf is accepting or not– A node in state s 2 SAND leads to acceptance if both its children

leads to acceptance– A node in state s 2 SOR leads to acceptance if one of its children

leads to acceptance

Alternating Time ClassesATIME(f(n))= class of language decided by an ATM where:• All computations halt after at most f(|x|) stepsASPACE(f(n))= class of language decided by an ATM where:• All computation halt • Never use more than f(|x|) cells

AL AL = ASPACE(log n)

Theorem: AL = PAL = PTheorem: by simulating circuitImportant point:Important point: if f 2 PP then the circuit is constructable in log space• Simulate the circuit from the output towards the inputs

Oracle MachinesRecall: Oracle Turing Machine (OTM) have a special query tape where can

ask membership queries and obtain answers• Oracle for A – can ask whether x 2 A• Oracle Machine with oracle for A: MA

Can consider time classes with oracle CA

• PA ={L|there is an OTM MA that decides L in polynomial time}• NPA ={L|there is an OTM MA that decides L in non deterministic

polynomial time}• Is PSAT = NPSAT ? Is PPSPACE = NPPSPACE ?

Can consider Time Classes with oracle from another time class• PNP ={L|there is an OTM M AND a language A2 L where MA

decides L in polynomial time}

Homework

Motivation: minimizing circuits• Central problem in logic synthesis:

• What is the complexity of this problem?– NP-hard? – Is it in NP, coNP, PSPACE?

• Complete for any of these classes?

x1 x2

x3 … xn

Given a Boolean circuit C

Is there a circuit C’ of size smaller than C computing the same function that C does?

The Polynomial-Time Hierarchy

• Can define many complexity classes using oracles• Concentrate on classes that

– have natural complete problems– have a natural interpretation in terms of alternating

quantifiers– help state consequences and containments

The Polynomial Time HierarchyVersion I: Oracles

• 0P = 0P = 0P =P

• i+1P = PiP

• i+1P = NPiP

• i+1P = coNP

iP

• PH = [i ¸0 iP

Notation sometimes i

P

Polynomial time hierarchyFirst level of the hierarchy:

• 1P = P

0P

= P

• 1P = NP

0P

= NP

• 1P = CoNP

0P

= CoNP

• If we believe the first level classes to be different from each other (and from the zero-level class(es)), are other levels also different?

• Does equality at one level imply one at the other levels? • Are there interesting problems higher than the first level?

Polynomial time hierarchy

Proposition: iPiP i+1P i+1Pi+1P

Proof: iP iP i+1P=P

iP Example NPcoNP PNP=2P

We have an oracle for L iP Ask the oracle and return answer for iPAsk the oracle and return the opposite for iP

(which is co-iP).

Polynomially Balanced RelationsA binary relation R µ {0,1}n £ {0,1}n is polynomially

balanced if – (x,y) 2 R, then |y|· |x|k for some k

Characterization of NP in terms of witnesses: Claim: L 2 NP iff there is a polynomially balanced relation

such that – R2 P– L={x| 9 y such that (x,y) 2 R}

Claim: L 2 CoNP iff there is a polynomially balanced relation R2 P such that

L={x| 8 y such that |y| · |x|k, (x,y) 2 R}

The Polynomial Time HierarchyVersion II: Quantifiers

Theorem: Let L be a language and i ¸ 1. L 2 iP iff there is a polynomially balanced relation R such that – R 2 i-1P

and– L={x| 9 y such that (x,y) 2 R}

Proof: by induction on iInteresting direction: show how to build a certificate checkable in

i-1P from computation of NP machine with i-1P oracle

Proof of Equivalence

• Proof of Theorem:– induction on i– Have seen base case

Oracle definition implies quantifier definition:• By definition ΣiP = NPΣi-1P = NPi-1P

• Given L={x| 9 y such that (x,y)2R} and R2i-1P to see why it is in ΣiP: Guess y, ask oracle if (x, y) R

Can do since ΣiP = NPi-1P

Proof of Equivalence

Quantifier definition implies oracle definition:

Given L Σi = NPΣi-1 decided by an Oracle NTM M running in time nk

• First try: R = {(x, y) | y describes a valid path of M’s computation}– Valid path = leading to qaccept

Problem: how to recognize valid computation path when it depends on results of oracle queries?

Proof of Equivalence

R = { (x, y) | y describes valid path of M’s computation}Try– valid path = step-by-step description including correct yes/no

answer for each A-oracle query zj (A Σi-1P)e.g: z1 A, z2 2 A, z3 A, … zm A

– verify “no” queries in i-1P:

e.g: z1 A z3 A … zm A – for each “yes” query zj , by induction. 9 wj, |wj| ≤ |zj|k

with (zj, wj) R’ for some R’ i-2P

For each “yes” query zj put wj in description of path y

Proof of Equivalence

Single relation R in i-1P :(x, y) R IFF – all “no” zj A and

– all “yes” zj have (zj, wj) R’ and

– y is a path leading to qaccept.

Key Point: the AND of i-1 predicates is in i-1.

The Polynomial Time HierarchyVersion II

Corollary: Let L be a language and i ¸ 1. L 2 iP iff there is a polynomially balanced relation R such that – {(x,y)|(x,y) 2 R} 2 i-1P and– L={x| 8y with |y| · |x|k we have (x,y) 2 R}

Proof: recall that i P is CoiP

Corollary: Let L be a language and i ¸ 1. L 2 iP iff there is a polynomially balanced and polynomially decidable (i+1)-ary relation R such that – L={x| 9 y1 8 y2 9 y3 …Q yi such that (x,y1, y2, … yi) 2 R}

Quantifier Q is 9 if i is odd and Q is 8 if i is even Complete problem for iP

Collapse of the Hierarchy

Theorem: If for some i ¸ 1 iPiP,

then for all j ¸ i jPjP

Proof: suffices to show iPiP implies iPi+1P

Corollary : If NP=CoNP then the polynomial hierarchy collapses to the first levelSimilarly if P=NP

Interesting Languages in the hierarchy

Minimum equivalent problems Minimum circuit: given a circuit C is it true that there

is no circuit C’ with fewer gates that computes the same function as C

Claim: Minimum circuit is in 2P: 8 C’, |C’| < |C| 9 x such that C(x) C’(x)

Minimum circuit is not known to be lower in the hierarchy or complete for 2P

Complete Problems for 2P

Minimum DNF: given DNF , integer k; is there a DNF ‘ of size at most k computing same function does?– Example:

x1x2x3 x1x3 x4 x1x2 x1x3 x4

Theorem: (Umans 98): Minimum DNF is Σ2P-complete

Complete Problems for 2P

Odd TSP: given a weighted graph G, is the length of the shortest TSP tour in G odd?

Theorem: Odd TSP is Δ2P-complete

Homework: Show Odd TSP is in Δ2P

Oracles vs. Algorithms• Given a polynomial time algorithm for SAT

– Can you solve Minimum Circuit efficiently?– What other consequences?

• Given an oracle for SATsame input/output behavior as algorithm!– Can you solve Minimum Circuit efficiently?

Key issue: is access to a program as a black box give you more power then given the code itselfRecent developments in cryptography show a difference (Barak

2001)

BPP in the Hierarchy• Do not know whether BPP µ NP

– Do not know whether BPP≠EXP

Theorem: BPP µ 2P 2P• Sufficient to show BPP µ 2P and then by symmetry BPP µ 2P

• L 2 BPP: Poly time PTM M: x L Pry[M(x,y) accepts] ≥ 2/3x L Pry[M(x,y) rejects] ≥ 2/3

• Consider a game between and players player wants to prove x L and player wants to prove x L

• Need to choose y cooperatively• First attempt:

player chooses ρ 2 {0,1}n

player chooses t 2 {0,1}n

– Set y = ρ t run M(x,y)

• There is a winning strategy for player as long as Pry[M(x,y) accepts] <1

BPP in the Hierarchy• Since game was unfair to player give him several chances

– Several (parallel) instances of the game– Force the player to use the same choice in all instances

• Second attempt: player chooses ρ1, ρ1,… ρm where each ρi 2 {0,1}n

player chooses t 2 {0,1}n

– Set yi = ρi t run M(x,y1), M(x,y2),…, M(x,ym) accept if any of the runs accepts

Each ρi invalidates 2/3rds of possible t 2 {0,1}n

• Invalidates = not a useful response for player • Due to randomization by Xor

By a hitting set argument there is a set of ρ1, ρ2,… ρm that invalidates all t 2 {0,1}n for m= n log3 2

But what happens when x L? There might be {ρ1, ρ2, ρ3 } such that

[i=1,2,3 { t | M(x,y) accepts for y = ρi t} = {0,1}n

Happens when bad witnesses are a coset of an ECC

Each of size at most 1/3

BPP in the Hierarchy– Can apply error reduction: Poly time PTM M’

• use n’ random bits (|y’| = n’)• fraction of strings y’ for which M’(x, y’) is incorrect is at most 1/n’

Now by the hitting set argument there is a set of ρ1, ρ2,… ρm that invalidates all t 2 {0,1}n for m= n’ /log n’

But when x L for any choice of {ρ1, ρ2,…, ρm }

1/ 2n’ |[i=1m { t | M(x,y) accepts for y = ρi t}|

< n’ /log n’ ¢ 1/n’

• Final form player chooses ρ1, ρ1,… ρm where each ρi 2 {0,1}n’

player chooses t 2 {0,1}n’

– Set yi = ρi t run M’(x,y1), M’(x,y2),…, M’(x,ym) accept if any of the runs accepts

clearly a 2P process

Each of size at most 1/n’

More on BPP in the HierarchySimultaneous Provers

• To prove BPP BPP ½½ 22PP we considered a game between and players player wants to prove x L and player wants to prove x L

• In 22PP setting first makes a move and then player (based on ’s move)

What if both players have to move simultaneously?• Want strategy where

player wins whenever x L even if player makes move after player player wins whenever x L even if player makes move after player

Homework:• Phrase the above requirements in terms of a polynomial time relation

R(x,y,z)• Show that for all L BPP BPP such a strategy exists

Polynomial sized circuits for SAT• We know that P P == NP NP implies SAT has poly-sized

circuits (SAT P/poly ).– Showing SAT does not have poly-size circuits is a

research direction for proving P ≠ NP

• Suppose SAT has poly-size circuits– Can we show implies P P == NP NP ?– Instead show: SAT P/poly PH collapses (to

second level)• similar implication as if SAT P

Self ReducibilityLemma: If SAT P/poly then there exists a polynomial p(n) and

family of circuits C1, C2 … such that Cn gets an n sized formula and outputs

• A satisfying assignment if is satisfiable• Null if is not satisfiable

Corollary: If SAT P/poly and R is polynomial time balanced relation, then for

L = { x | z (x,z) 2 R}

there is a family of circuits C1, C2 … such that L={x| |x|=n implies (x,Cn(x)) 2 R}

Can define new polynomial relation R’ such that (x,C) 2 R’ iff (x,C(x))2R and C is of the right size.

Collapse of The Hierarchy

Theorem: if SAT P/poly then PH collapses to the second level.

Proof: • Show that 2P µ Σ2P

– If L 2P then we know:L = { x | y z (x, y, z) 2 R }

for R P.– “z (x, y, z) R?” is in NP– Let Cn be the SAT solver – produces certificate z

z (x, y, z) R iff (x, y, Cn(x,y)) R

L = { x | Cn y (x, y, Cn(x,y)) 2 R } = { x | Cn y (x, y, Cn) 2 R’ } 2P

Approximate Counting is in the Hierarchy

Theorem: For any f 2 #P there is a g 2 Δ3P such that on input x and g(x, ) outputs a (1+ ) approximation to f(x)

(1-)g(x, ) · f(x) · (1+)g(x, )

Proof: key point – method for proving that sets are of certain size

The set in question: A = Accept(M(x)) µ {0,1}m

A function f is in #P#P if there exists a NTM M running in polynomial time where M(x) has f(x) accepting paths for all x 2 {0,1}n

Pair-wise Independent Hash FunctionsA family of hash functions

H={h|h:{0,1}m {0,1}ℓ }is pair-wise independent if: for all x ≠ y, for random h 2RH the random variables

h(x), h(y) are uniformly distributed and independent

In particular: Pr[h(x)=h(y)]= 1/2ℓ

Already saw the approximate version for distributed equality testing Pr[h(x)=h(y)]·

Constructing H: • Construction based on matrix multiplication.

– Treat x as a m £ 1 vector. – The function is defined by a random m£ ℓ matrix over GF[2].

H={hA|A 2 GF[2]m£ ℓ and hA(x)=Ax }

Methods for proving sizes of setsLet H be family of hash functions where for h 2 H h:{0,1}m {0,1}ℓ

• if h 2 H is 1-1 on A then clearly |A| · 2ℓ

– a bit difficult to expect full uniqueness• Relaxed property:

if there are h1, h2, … hℓ 2 H are such that– for all x 2 A there is an 1 · i · l where for all x’ 2 A\{x} we have hi(x) hi(x’) then we know |A| < ℓ 2ℓ

Claim: for a pair-wise independent family H, if |A|· 2ℓ-1 , then there exist h1, h2, … hℓ 2 H with the no collisions property. Denote with UH(A,ℓ).

Claim: for A=Accept(M) we have UH(A,ℓ) 2 Σ2PCan eliminate the quantifier on i by explicitly going over all possibilities!

Conclusion: with oracles calls to UH(A,ℓ) can find smallest ℓ for which UH(A,ℓ) holds

Yields a 2ℓ approximation for f(x)

• Can use amplification of f(x) by running M k times (from accepting positions).New machine M will have fk(x) by accepting paths

No collisions property

Proof of No Collisions Claim• For any x 2 A and any x’ 2 A\{x} if h is chosen at

random from HPr[h(x)=h(x’)]= 1/2ℓ

• By the Union bound Pr[h(x) is unique] ¸ |A| ¢ 1/2ℓ ¸ 1/2

• Hitting set: Construct h1, h2, … hℓ one by one, each time covering at least ½ of elements in A that have not been unique so far– Let A’ be the set not covered so far– For any x 2 A’ and any x’ 2 A\{x} for random h 2R H

Pr[h(x) is unique] ¸ |A| ¢ 1/2ℓ ¸ 1/2 Note: this is

thefull A

Approximate Counting and Uniform Generation

• For self reducible problems:– exact counting implies exact uniform generation of

witnesses/solutions– Approximate counting: almost uniform generation of

solutions

Theorem: if P=NP then given Boolean circuit C possible to sample almost uniformly at random satisfying assignment to C

The classes we discussedPSPACEPSPACE: :

Complete problems: TQBF, 2-person games, generalized geography, …

33rdrd level level: Complete problems: VC dimension for succinct sets

Containment: Approximate #PApproximate #P…

22ndnd level level: Complete problems: Min DNF, Succinct Set Cover

Containment: Min Circuit, BPP…

11stst level level: Complete problems: SAT, UNSAT,…lots…

Containment: factoring, graph isomorphism…

P

NP coNP

Σ3P Π3P

Δ3P

PSPACE

EXP

PH

Σ2P Π2P

Δ2P

#P

BPP

Some History• Polynomial Time Hierarchy: Meyer-Stockmeyer• BPP in the Hierarchy: Lautmann, Sipser• Approximate Counting in the Hierarchy: Stockmeyer • SAT P/poly implies collapse: Karp-Lipton, Sipser

• Stockmeyer’s papers available:http://www.geocities.com/stockmeyer@sbcglobal.net/

Survey on problems high in the hierarchy: Schaefer and Umanshttp://www.cs.caltech.edu/umans