Co-NP problems on random inputs

Paul Beame

University of Washington

2

Basic idea

NP is characterized by a simple property - having short certificates of membership

Show that co-NP doesn’t have this property would separate P from NP so probably

quite hard Lots of nice, useful baby steps towards

answering this question

3

Certifying language membership

Certificate of satisfiability Satisfying truth assignment Always short, SAT NP

Certificate of unsatisfiability ????? transcript of failed search for satisfying truth

assignment Frege-Hilbert proofs, resolution Can they always be short? If so then NP=co-NP.

4

Proof systems

A proof system for L is a polynomial time algorithm A s.t. for all inputs x x is in L iff there exists a certificate

P s.t. A accepts input (P,x)

Complexity of a proof system How big |P| has to be in terms of |x|

NP = {L: L has polynomial-size proofs}

5

Propositional proof systems

A propositional proof system is a polynomial time algorithm A s.t. for all formulas F F is unsatisfiable iff there exists a certificate P s.t.

A accepts input (P,F)

6

Sample propositional proof systems

Truth tablesAxiom/Inference systems, e.g.

modus ponens A, (A -> B) | B excluded middle | (A v ~A)

Tableaux/Model Elimination systems search through sub-formulas of input formula

that might be true simultaneously e.g. if ~(A -> B) is true A must be true and

B must be false

7

Frege Systems

Finite # of axioms/inference rulesProof of unsatisfiability of F - sequence

F1, …, Fr of formulas s.t. F1 = F

each Fj is an axiom or follows from previous ones via an inference rule

Fr = trivial falsehood

All of equivalent complexity up to poly

8

Resolution

Frege-like system using CNF clauses onlyStart with original input clauses of CNF FResolution rule

(A v x), (B v ~x) | (A v B)Goal: derive empty clause

Most-popular systems for practical theorem-proving

9

Davis-Putnam (DLL) Procedure

Both a proof system a collection of algorithms for finding proofs

As a proof system a special case of resolution where the

pattern of inferences forms a tree.The most widely used family of

complete algorithms for satisfiability

10

Simple Davis-Putnam Algorithm

Refute(F) While (F contains a clause of size 1)

set variable to make that clause truesimplify all clauses using this assignment

If F has no clauses thenoutput “F is satisfiable” and HALT

If F does not contain an empty clause thenChoose smallest-numbered unset variable x Run Refute( )Run Refute( )0xF

1xF splitting rule

11

Hilbert’s Nullstellensatz

System of polynomials Q1(x1,…,xn)=0,…,Qm(x1,…,xn)=0 over field K has no solution in any extension field of K iff there exist polynomials P1(x1,…,xn),…,Pm(x1,…,xn) in K[x1,…,xn] s.t.

1

QP ii

m

1i

12

Nullstellensatz proof system

Clause (x1 v ~x2 v x3) becomes equation (1-x1)x2(1-x3)=0

Add equations xi2-xi =0 for each

variable

Proof: polynomials P1,…, Pm+n proving unsatisfiability

13

Polynomial Calculus

Similar to Nullstellensatz except: Begin with Q1,…,Qm+n as before Given polynomials R and S can infer

a R + b S for any a, b in Kxi R

Derive constant polynomial 1 Degree = maximum degree of polynomial

appearing in the proof Can find proof of degree d in time nO(d) using

Groebner basis-like algorithm

14

Cutting Planes

Introduced to relate integer and linear programming: Clause (x1 v ~x2 v x3)

becomes inequality x1+1-x2+x3 1

Add xi 0 and 1-xi 0

Derive 0 1 using rules for adding inequalities and Division Rule:

acx+bcy d implies ax+by d/c

15

Some Proof System Relationships

Truth Tables

Davis-Putnam Nullstellensatz

Polynomial Calculus

Resolution

Cutting Planes

Frege

AC0-Frege

ZFC

P/poly-Frege

16

Random k-CNF formulas

Make m independent choices of one of the clauses of length k

= m/n is the clause-density of the formula

Distribution

k

nk2

kn,F

17

Threshold behavior of random k-SAT

18

Contrast with ...

Theorem [CS]: For every constant , random k-CNF formulas almost certainly require resolution proofs of size 2(n)

What is the dependence on ?

19

Width of resolution proofs

If P is a resolution proof width(P) = length of longest clause in P

Theorem [BW]: Every Davis-Putnam (DLL) proof of size S can be converted to one of width log2S

Theorem [BW]: Every resolution proof of size S can be converted to one of width

)logSnO(

20

Sub-critical Expansion

F - a set of clausess(F) - minimum size subset of F that is

unsatisfiable F - boundary of F - set of variables

appearing in exactly one clause of Fe(F) - sub-critical expansion of F =

max min { |G|:

G F, s/2 < |G| s} s s(F)

21

Width and expansion

Lemma [CS] : If P is a resolution proof of F then width(P) e(F).

s(F)

s/2 to s

G

containsG

22

Consequences

Corollaries: Any Davis-Putnam (DLL) proof of F

requires size at least 2e(F)

Any resolution proof of F requires size at least

n(F)2e2

23

s(F) and e(F) for random formulas

If F is a random formula from then s(F) is (n/1/(k-2)) almost certainly

e(F) is (n/2/(k-2)+) almost certainly

Proved for Hypergraph expansion

kn,F

24

Hypergraph Expansion

F - hypergraph F - boundary of F - set of degree 1

vertices of FsH(F) - minimum size subset of F that does

not have a System of Distinct Representatives

eH(F) - sub-critical expansion of F - max min { |G|: G F, s/2

< |G| s} s sH(F)

25

System of Distinct Representatives

sH(F) s(F) so eH(F) e(F)

variables/nodes

clauses/edges

26

Density and SDR’s

The density of a hypergraph is #(edges)/#(vertices)

Hall’s Theorem: A hypergraph F has a system of distinct representatives iff every subgraph has density at most 1.

27

Density and Boundary

A k-uniform hypergraph of density bounded below 2/k, say 2/k-has average degree bounded below 2 constant fraction of nodes are in

the boundary

28

Density of random formulas

Fix set S of vertices/variables of size r Probability p that a single edge/clause

lands in S is at most (r/n)k

Probability that S contains at least q edges is at most

q

1k-

1k-q

nre

qnpe

qp)n,B(Pr

29

s(F) for random formulas

Apply for q=r+1 for all r up to s using union bound:

for s = O(n/1/(k-2))

s

kr

1r

2k-

2k2

1r

1k-

1krs

kr

1r

1k-

1ks

kr

o(1)n

reenr

nre

rne

nre

r

n

30

e(F) for random formulas

Apply for q=2r/k for all r between s/2 and s using union bound:

for s = (n/2/(k-2))

s

s/2r

2r/k

k/2-1k-

k/21kk/21

2r/k

1k-

1krs

s/2r

2r/k

1k-

1ks

s/2r

o(1)n

re

nre

rne

nre

r

n

31

Hypergraph Expansion and Polynomial Calculus

Theorem [BI]: The degree of any polynomial calculus or Nullstellensatz proof of unsatisfiability of F is at least eH(F)/2 if the characteristic is not 2.

Groebner basis algorithm bound is only nO(eH(F))

32

k-CNF and parity equations

Clause (x1 v ~x2 v x3) is implied by x1+(x2+1)+x3 = 1 (mod 2) i.e. x1+x2+x3 = 0 (mod 2)

Derive contradiction 0 = 1 (mod 2) by adding collections of equations

# of variables in longest line is at least eH(F)

33

Parity equations and polynomial calculus

Given equations of form x1+x2+x3 = 0 (mod 2)

Polynomial equation yi2-1=0 for each variable

yi = 2xi-1

Polynomial equation y1 y2 y3-1=0

would be y1 y2 y3+1=0 if RHS were 1 Imply the old Nullstellensatz equations if

char(K) is not 2

34

Lower bounds

For random k-CNF chosen from almost certainly for any >0: Any Davis-Putnam proof requires size

Any resolution proof requires size

Any polynomial calculus proof requires degree

kn,F

2)2/(kn/2 Δ

2)4/(kn/2 Δ

2)2/(kn/Δ

35

Upper Bound

Theorem [BKPS]: For F chosen from and above the threshold, the simple Davis-Putnam (DLL) algorithm almost certainly finds a refutation of size

and this is a tight bound...

kn,F

O(1)n/O n22)1/(kΔ

36

Idea of proof

2-clause digraph (x v y)

Contradictory cycle: contains both x and xAfter setting O(n/1/(k-2)) variables,

> 1/2 the variables are almost certainly in contradictory cycles of the 2-clause digraph a few splitting steps will pick one almost certainly setting clauses of size 1 will finish things off

x

y

x

y

37

Implications

Random k-CNF formulas are provably hard for the most common proof search procedures.

This hardness extends well beyond the phase transition. Even at clause ratio =n1/3, current

algorithms on random 3-CNF formulas have asymptotically the same running time as the best factoring algorithms.

38

Random graph k-colourability

Random graph G(n,p) where each edge occurs independently with probability p Sharp threshold for whether or not

graph is k-colourable, e.g. p ~ 4.6/n for k=3

What about proofs that the graph is not k-colourable?

39

Lower Bound

Theorem [BCM 99]: Non-k-colourability requires exponentially large resolution proofs

Basic proof idea: same outline as before notion of boundary of a sub-graph

set of vertices of degree < k s(G) smallest non-k-colourable sub-graph

40

Challenges

Better bound for e(F) for random F Can it be (s(F)) ?

If so, the simple Davis-Putnam algorithm has asymptotically best possible exponent of any DP algorithm.

Extend lower bounds to other proof systems must be based on something other than expansion

since certain formulas with high expansion have small Cutting Planes proofs.

41

Challenges

Conjecture: Random k-CNF formulas are hard for Frege proofs

Extend to other random co-NP problems Independent Set?

Best algorithms only get within factor of 2 of the largest independent set in a random graph

42

Sources

[Cook, Reckhow 79] [Chvatal, Szemeredi 89] [Mitchell, Selman, Levesque 93] [Beame, Pitassi 97] [Beame, Karp, Pitassi, Saks 98] [Beame, Pitassi 98] [Ben-Sasson, Wigderson 99] [Ben-Sasson, Impagliazzo 99] [Beame, Culberson, Mitchell 99]

43

Circuit Complexity

P/poly - polysize circuitsNC1 - polysize formulasCNF - polysize CNF formulasAC0 - constant-depth polysize circuits

using and/or/not

AC0[m] - also = 0 mod m tests

TC0 - threshold instead

44

C-Frege Proofs

Given circuit complexity class C can define C-Frege proofs to be Frege-like proofs that manipulate circuits in C rather than formulas

Frege = NC1-FregeResolution = CNF-FregeExtended-Frege = P/poly-FregeAC0-FregeAC0[m]-FregeTC0-Frege