Chapter 7Inefficiency and Intractability

CS 345

Spring Quarter, 2014

Towers of Hanoi Revisited

• We discovered that the time complexity of the Towers of Hanoi problem is 2N -1

• The original problem had 64 rings. If one ring could be moved every 10 seconds, it would take over 5 trillion years to solve the problem.

• Even moving 1 million rings a second takes over 1/2 million years

The Monkey Puzzle

• See pages 160-162 for an explanation of the puzzle

• Given N cards, a brute-force solution takes N! steps.

• Arranging 1 million cards a second still takes over 490 billion years for 25 cards.

Reasonable vs. Unreasonable Time

• The factorial function grows very rapidly.

• Factorial grows even faster than 2N.

• One of the fastest growing functions we know is Ackerman’s Function.

• For comparison, 2 slides that follow reproduce figures 7.3 (page 163) and 7.5 (page 167)

Ackerman’s Function

A(1, j) = 2j, for j 1

A(i, 1) = A(i-1, 2) for i 2

A(i, j) = A( i -1, A( i, j-1)) for i, j 2

How fast does this function grow? Note that

A(i, j+1) > A( i,j) ; A(i+1,j) > A(i,j) and A(3,4) = 2^2^2 …^2 (a stack of 65,536 2’s)

65,536

N 20 60 100 300 1000Func. 5N 100 300 500 1500 5000

N LogN 86 354 665 2469 9966

N2 400 3,600 10,000 90,000 1 mil

N3 8,000 216,000 1 mil 27 mil 1 bil

2N 1048576 19 digit 31 digit 91 digit 302 digit

N! 19 digit 82 digit 161 digit 623 digit big

NN 27 digit 107 digit 201 digit 744 digit real big

N 20 40 60 100 300f(N)

N2 1/2500ms 1/625ms 1/278ms 1/100ms 1/11ms

N3 1/300s 1/10s 78/100s 10s 40.5m

2N 1/1000s 18.3m 36.5Y 400bilC 72digC

NN 3.3bilY 46digC 89digC 182digC 725digC

Figure 7.5, page 164assume 1 instruction per nanosecond

Tractable Algorithms

• Examining the previous two slides indicate that some algorithms might be reasonable and others not reasonable.

• Algorithms whose run time is bounded from above by a polynomial function of N are called polynomial-time algorithms, are said to be reasonable, and are called tractable.

Intractable Algorithms

• An algorithm which has an exponential or super-exponential upper bound is said to be unreasonable and is called intractable.

• Note that an intractable algorithm has a solution. It just takes an unreasonably long time to find it.

Decision Problems

• Perhaps part of the long running time of an algorithm is that we are looking for a solution.

• What if we just asked, “Does a solution exist?”

• A decision problem is one which has a yes or no answer.

• Most problems have a decision problem version.

• For example: What is the minimum grade I can get on the final and still get an A in the class?

• The decision problem version is: If I get a (some number) on the final, will I still get an A in the class?

More on the Monkey PuzzlePage 166

1. Computers are becoming faster by the week. Over the last 10 years or so computer speed has increased roughly by a factor of 50. Perhaps obtaining a practical solution to the problem is just a question of awaiting an additional improvement in computer speed.

2. Doesn’t the fact that we have not found a better algorithm for this problem indicate our incompetence at devising efficient algorithms? Shouldn’t computer scientists be working at trying to improve the situation rather than spending their time writing books about it?

3. Haven’t people tried to look for an exponential-time lower bound on the problem, so that we might have a proof that no reasonable algorithm exists?

4. Maybe the whole issue is not worth the effort, as the monkey puzzle problem is just one specific problem. It might be a colorful one, but it certainly doesn’t look like a very important one.

• Reply to point 1: Examine figure 7.7 on page 166. Given an exponential algorithm, an improvement in computer speed by a constant factor--however large--will make only an insignificant improvement overall.

• Reply to point 4: It so happens that the “Monkey Puzzle” problem is not alone.

• There are close to 1000 diverse algorithmic problems which have intractable solutions. But, none are known not to have tractable solutions.

• In many cases, the best-known lower bounds are (N). That is, these problems have very large algorithmic gaps

The Class NP

• We denote these problems as NP.

• These are problems that have no known tractable solution, but we cannot prove that a tractable solution does not exist.

Some NP Problems

• Generalized jigsaw or arrangement

• Graph problems– Clique– TSP (the Decision Version)– Hamiltonian cycle and path– Eulerian Path: Königsberg bridges, 1736– Graph Colorings

Nondeterminism and Cook’s Theorem

• The theory of NP began with a remarkable theorem of Cook in 1971.

• The notion of nondeterminism is rather non-intuitive.

• One should think of a nondeterministic algorithm as an abstract notion, not as a realistic goal.

• Nondeterminism is more important to the development of the theory and the explanation of the existence of this class of problems than it is to the techniques for using the theory.

• A nondeterministic algorithm has, in addition to all the regular operations of a deterministic algorithm, a very powerful oracle.

• Problems solved by nondeterministic algorithms have the property that they can take an unacceptably long time to produce a solution. If a solution is arrived at, however, there is an easy way to convince someone that it is a solution.

• Nondeterministic algorithms are very powerful, but their power is not unlimited. Not all problems can be solved efficiently by nondeterministic algorithms.

• The class of problems that have a deterministic, polynomial time algorithm that solves them is called P.

• The class of problems that have a nondeterministic algorithm that can be checked in polynomial time is called NP.

• Are nondeterministic algorithms more powerful than deterministic ones????

• Its clear that P NP, but is P NP ???

• No one has yet demonstrated an algorithm that is in NP but not in P

• In contrast, to prove P = NP requires us to show that every problem in NP can be solved by a deterministic polynomial time algorithms. Nobody has done this either (and very few believe it is true)

• The question does P = NP? (first asked in 1971) is the most difficult open question in theoretical Computer Science.

Polynomial Reduction

• Algorithm X is reducible to Algorithm Y if there is a polynomial time algorithm which transforms an input to X into an input to Y in such a manner that Y always gives the same answer ( “yes” or “no”) using the transformed input as X does with the original input.

• If Alg X is reducible to Alg Y then, if Y can be solved with a polynomial time deterministic algorithm, so can X.

• This means that X is no harder than Y

• Definition: A problem X is called an NP-hard problem if every problem in NP is reducible to X. This means that no problem in NP is harder than X.

• Definition: A problem X is called an NP-complete problem if (1) it belongs to NP and (2) it is NP-hard.

• Note that from the definition of NP-hard, if any NP-hard problem is shown to be in P, then P = NP.

• Cook’s remarkable theorem proved that there exists NP-complete problems. In particular, he demonstrated the SAT was NP-complete.

• Cook’s theorem makes showing a problem is NP-complete easier.

• Definition: A problem X is an NP-complete problem if (1) X belongs in NP and (2) some problem Y reduces to X where Y is already known to be in NP-complete.

• Because all problems in NP reduce to each problem in NP-complete, if one NP-complete problem has a polynomial-time deterministic algorithm, then P = NP.

Imperfect Solutions to NP-Complete Problems

• NP-complete problems are usually taken as the decision problem (yes/no) version of optimization problems. E.g. TSP

• But, humans manage to solve these problems all the time in everyday life, and it doesn’t take centuries. Also, the human brain is orders of magnitudes slower than today’s computers.

• The human brain is massively parallel

• Approximation and Probabilistic Algorithms

• Heuristics

Provably Intractable Problems

• We are not able to prove that problems in NP have no tractable solution

• There are problems, however, such as the Towers Of Hanoi, that can be shown to have an exponential lower bound and that have no certificate.

• These problems belong to classes that are even harder than NP.

Complexity Classes

• Theoretical Computer Scientists talk about a hierarchy of complexity classes.

• These classes are for both time and space requirements.

• Dual classes or co- classes (co-NP): the yes or no answers are not the same. Is it true that p is a prime number? / Is it true that p is a composite number?

Research on Complexity Classes

• Some theorists work with questions such as: Does P = NP? or Does NP = co-NP? or Can we refine the complexity classes?

• Others work to develop approximate solutions or heuristic solutions or alternate models of computation: genetic algorithms, neural networks, etc..