1

Ch 10 - NP-completeness

• Tractable and intractable problems• Decision/Optimization problems• Deterministic/NonDeterministic algorithms• Classes P and NP• Polynomial reductions• NP-complete problems

– Cook-Levin theorem– CLIQUE (via SAT CLIQUE reduction)– TSP (via HC TSP reduction)

2

Classifying problems

• Classify problems as tractable or intractable.

• Problem is tractable if there exists at least one polynomial bound algorithm to solve it.

• An algorithm is polynomial bound if its worst case growth rate can be bound by a polynomial p(n) in the size n of the problem

constant a is where...)( 01 kanananp kn

3

Intractable problems

• Problem is intractable if it is not tractable.• All algorithms that solve such a problem

are not polynomial bound. • It has a worst case growth rate f(n) which

cannot be bound by a polynomial p(n) in the size n of the problem.

• For intractable problems the bounds are:

f n c nn n( ) , ,log or etc.

4

Why is this classification useful?

• If problem is intractable, no point in trying to find an efficient algorithm

• All algorithms will be too slow for large inputs.

5

Hard practical problems• There are many practical problems for which

no one has yet found a polynomial bound algorithm.

• Examples: traveling salesperson, 0/1 knapsack, graph coloring, bin packing etc.

• Most design automation problems such as testing and routing.

• Many networks, database and graph problems.

6

The theory of NP completeness

• The theory of NP-completeness enables showing that these problems are at least as hard as NP-complete problems

• Practical implication of knowing problem is NP-complete is that it is probably intractable ( whether it is or not has not been proved yet)

• So any algorithm that solves it will probably be very slow for large inputs

7

Decision Problems vs. Optimization Problems

• Decision Problem: Yes/no answer

• Optimization Problem: Maximization or minimization of a certain quantity

• When studying NP-Completeness, it is easier to deal with decision problems than optimization problems

8

Element Uniqueness Problem• Decision Problem: Element Uniqueness

– Input: A sequence of integers S– Question: Are there two elements in S that are

equal?

• Optimization Problem: Element Count– Input: A sequence of integers S– Output: An element in S of highest frequency

• What is the algorithm to solve this problem? How much does it cost?

9

Coloring a Graph• Decision Problem: Coloring

– Input: G=(V,E) undirected graph and k, k > 0.– Question: Is G k-colorable?

• Optimization Problem: Chromatic Number– Input: G=(V,E) undirected graph – Output: The chromatic number of G,(G)

• i.e. the minimum number (G) of colors needed to color a graph in such a way that no two adjacent vertices have the same color.

10

Traveling Salesman

• Given a finite set C={c1,...,cm} of cities, a distance function d(ci, cj) of nonnegative numbers and a bound B

• Decision Problem– Is there a tour of all the cities (in which each city

is visited exactly once) with total length at most B?

• Optimization Problem– Find the length of the minimum distance tour

which includes every city exactly once

11

Cliques• Definition: A clique of size k in G, for some

+ve integer k, is a complete subgraph of G with k vertices.

• Decision Problem– Input:– Question:

• Optimization Problem– Input:– Output:

12

The relation between• If we have a solution to the optimization

problem we can compare the solution to the bound and answer “yes” or “no”.

• Therefore if the optimization problem is tractable so is the decision problem

• If the decision problem is “hard” the optimization problem is also “hard”– If the optimization was easy then the

decision problem is easy.

13

Deterministic Algorithms

• Definition: Let A be an algorithm to solve problem. A is called deterministic if, when presented with an instance of the problem , it has only one choice in each step throughout its execution.– If we run A again and again, is there a

possibility that the output may change?

• What type of algorithms did we have so far?

14

The Class P

• Definition: The class of decision problems P consists of those whose yes/no solution can be obtained using a deterministic algorithm that runs in polynomial time of steps, i.e. O(nk), where k is a non-negative integer and n is the input size.

15

Examples

• Sorting: Given n integers, are they sorted in non-decreasing order?

• Set Disjointness: Given two sets of integers, are they disjoint?

• Shortest path:• 2-coloring:

– Theorem: A graph G is 2-colorable if and only if G is bipartite if and only if G has no odd length cycle

16

Non-Deterministic Algorithms

• A non-deterministic algorithm A on input x consists of two phases:– Guessing: An arbitrary “string of characters y” is

(called certificate) generated. • It may or may not correspond to a solution• Not be in proper format of a solution• Differ from one run to another

– Verification: A deterministic algorithm verifies• The generated “string of characters y” is a

solution

17

Non-Deterministic Algorithms

• Definition: Let A be a nondeterministic algorithm for a problem . We say that A accepts an instance I of if and only if on input I, there exists a guess that leads to a yes answer.– Does it mean that if an algorithm A on a given

input I leads to an answer of no for a certain guess, that it does not accept it?

18

The Class NP• Definition: The class of decision problems NP

consists of those decision problems for which there exists a nondeterministic algorithm that runs in polynomial time.– The running time of a nondeterministic

algorithm, is the sum of the running times of the guessing and verification phases.

– Both phases need run in polynomial time.– In particular, the verification phase.

19

Example - TSP

• Show that the traveling salesman problem belongs to the class of NP problems

• Given a problem instance x for TSP and a certificate y– Check that y is indeed a cycle that includes

every vertex exactly once– Verify that the length of the cycle is at most B

• Is the verification algorithm polynomial?

20

P and NP Problems

• What is the difference between P problems and NP Problems?

• We can solve problems in P using deterministic algorithms that run in polynomial time

• We can check or verify the solution of NP problems in polynomial time using a deterministic algorithm

• What is the set relationship between the classes P and NP?

21

Summary: P and NP• Summary so far:

– P = problems that can be solved in poly time– NP = problems for which a solution can be verified

in polynomial time– P NP – Unknown whether P = NP (most suspect not)

• We’ve seen problems that belong to NP that may not belong to P– Hamiltonian path/cycle, TSP problems are in NP– Cannot solve in polynomial time– Easy to verify solution in polynomial time

22

NP-Complete Problems• We will see that NP-Complete problems are the

“hardest” problems in NP:– If any one NP-Complete problem can be

solved in polynomial time…– …then every NP-Complete problem can be

solved in polynomial time…– …and in fact every problem in NP can be

solved in polynomial time (showing P = NP)– Thus: solve hamiltonian-cycle in O(n100) time,

you’ve proved that P = NP. Retire rich & famous.

23

Reduction• The crux of NP-Completeness is reducibility

– Informally, a problem A can be reduced to another problem B if any instance of A can be “easily rephrased” as an instance of B , the solution to which provides a solution to the instance of A

• This rephrasing is called transformation– Intuitively: If A reduces to B, A is “no harder to

solve” than B • Total cost: cost of transformation + cost to solve B

24

Reduction: A reduces to B

• An example:– A: Given a set of Booleans, is at least one TRUE?– B: Given a set of integers, is their sum positive?– Transformation: (x1, x2, …, xn) = (y1, y2, …, yn) where

yi = 1 if xi = TRUE, yi = 0 if xi = FALSE

IA

A solver

IB OBOA

Transformer B solver

25

NP-Hard and NP-Complete

• If A is polynomial-time reducible to B, we denote this A p B

• Definition of NP-Hard and NP-Complete: – If all problems X NP are reducible to A, then A

is NP-Hard– We say A is NP-Complete if A is NP-Hard

and A NP• If A p B and A is NP-Complete, B is also

NP-Complete

26

Why Prove NP-Completeness?

• Though nobody has proven that P ≠ NP, if you prove a problem NP-Complete, most people accept that it is probably intractable

• Therefore it can be important to prove that a problem is NP-Complete– Don’t need to come up with an efficient

algorithm– Can instead work on approximation

algorithms

27

Proving NP-Completeness• What steps do we have to take to prove a problem A

is NP-Complete?– Pick a known NP-Complete problem B– Reduce B to A

• Describe a transformation that maps instances of B to instances of A, s.t. “yes” for A = “yes” for B

• Prove the transformation works• Prove it runs in polynomial time

– Oh yeah, prove A NP

28

Examples of NP-Complete problems

• Cook’s theorem – Satisfiability is NP-complete

• This was the first problem shown to be NP-complete

• Other problems– Graph coloring (= register allocation)– Hamiltonian path– Traveling salesman– Knapsack

29

Conjunctive Normal Form (CNF)

• A logical (Boolean) variable is a variable that may be assigned the value true or false (p, q, r and s are Boolean variables)

• A literal is a logical variable or the negation of a logical variable (p and q are literals)

• A clause is a disjunction of literals ( (pqs) and (q r) are clauses)

30

Conjunctive Normal Form (CNF)

• A logical (Boolean) expression is in Conjunctive Normal Form if it is a conjunction of clauses.

• The following expression is in conjunctive normal form:

(pqs) (q r) (p r) (r s) (psq)

31

The Satisfiability problem• Is there a truth assignment to the n variables of

a logical expression in Conjunctive Normal Form which makes the value of the expression true?– For the answer to be “yes”, all clauses must

evaluate to true– Otherwise the answer is “no”

• p=T, q=F, r=T and s=T is a truth assignment for:(pqs) (q r) (p r) (r s) (psq)

32

The Satisfiability problem

• Theorem: Satisfiability is an NP-Complete problem.

• Proof:– Satisfiability NP: Given a truth assignment

of a Boolean formula consisting of n literals, it is clear that we can evaluate it in poly time.

– Satisfiability NP-Hard: The proof involves reducing every problem in NP to Satisfiability in polynomial time. See Garey & Johnson.

33

Proving NP-CompletenessSAT

3-CNF-SAT

Subset-Sum

Clique Hamiltonian Cycle

Vertex-Cover Traveling Salesman

34

NP completeness of CLIQUE

• SAT reduces to CLIQUE– Given any input to SAT, we create a

corresponding input to CLIQUE that will help us solve the original SAT problem

– Specifically, for a SAT formula with K clauses, we construct a CLIQUE input that has a clique of size K if and only if the original Boolean formula is satisfiable

35

NP completeness of CLIQUE

• SAT reduces to CLIQUE– Associate a person to each variable

occurrence in each clause

36

NP completeness of CLIQUE

• SAT reduces to CLIQUE– Associate a person to each

variable occurrence in each clause– ”Two people” know each other

except if:• they come from the same clause• they represent t and t’ for some

variable t

37

NP completeness of CLIQUE• SAT reduces to CLIQUE

– Two people know each other except if:• they come from the same clause• they represent t and t’ for some variable t

– Clique of size 4 satisfiable assignment• set variable in clique to ”true”• (x, y, z) = (true, true, false)

38

NP completeness of CLIQUE• SAT reduces to CLIQUE

– Two people know each other except if:• they come from the same clause• they represent t and t’ for some variable t

– Clique of size 4 satisfiable assignment– Satisfiable assignment clique of size 4

• (x, y, z) = (false, false, true)• choose one true literal from

each clause

39

CLIQUE is NP-complete

• CLIQUE is NP-complete– CLIQUE is in NP– SAT is in NP-complete– SAT reduces to CLIQUE

• Hundreds of problems can be shown to be NP-complete that way…

40

NP completeness of CLIQUE• Show that the traveling salesman problem is

NP-complete, assuming that the Hamiltonian cycle problem is NP-complete.– Prove that TSP NP (already done)

• Guess: a sequence of cities• Verify: that it is a tour of cost less than B

– Reduce the undirected Hamiltonian cycle problem to the TSP• So if we had a TSP-solver, we could use it

to solve the Hamilitonian cycle problem in polynomial time

41

HamCycle p TSP?

• Transform input for HamCycle into input for TSP– HamCycle: Given unweighted graph G1,

does it have a Hamiltonian cycle?– TSP: Given weighted graph G2 and B, is

there a Hamiltonian cycle with total cost less than B?

• Must convert unweighted graph to weighted

42

HamCycle p TSP?• Convert unweighted graph to weighted

– Add edges to G1 to make a complete graph G2

– Add weights as follows:• wt(i,j) is 0 if edge i,j is in G1 (original graph

for HamCycle)• wt(i,j) is 1 if edge i,j is not in G1

• G1 has ham. cycle iff G2 has TSP with B=0– Can you see why? – Is this transformation polynomial?

43

The Main Question• Is P = NP?

– Is the original DECISION problem as easy as VERIFICATION?

• Most important open problem in theoretical computer science.

44

The Big Picture• Summarizing: it is not known whether NP

problems are tractable or intractable• But, there exist provably intractable problems

– Even worse – there exist problems with running times unimaginably worse than exponential!

• More bad news: there are provably non-computable (undecidable) problems– There are no (and there will not ever be!)

algorithms to solve these problems