Computability - Tractable, Intractable and Non-computable Function

Computability

Tractable, Intractable and

Non-computable functions

(C) 2010 Thomas J Cortina, Carnegie Mellon University

Efficiency

●A computer program should be totally correct, but it should alsooexecute as quickly as possible (time-efficiency)ouse memory wisely (storage-efficiency)

●How do we compare programs (or algorithms in general) with respect to execution time?ovarious computers run at different speeds due to different

processorsocompilers optimize code before executionothe same algorithm can be written differently depending on the

programming language used(C) 2010 Thomas J Cortina, Carnegie Mellon University

Order of Complexity

●For very large n, we express the number of operations

as the order of complexity.

●Order of complexity for worst-case behavior is often

expressed using Big-O notation:

Number of operations Order of Complexity

n O(n)

n/2 + 6 O(n)

2n + 9 O(n)

Usually doesn't

matter what the

constants are...

we are only

concerned about

the highest

power

of n.


O(n) ("Linear")

n

(amount of

data)

Number of

Operation

s

n

n/2 + 6

2n + 9


Order of Complexity


n2 O(n2)

2n2 + 7 O(n2)

n2/2 + 5n + 2 O(n2)

Usually doesn't

matter what the

constants are...

we are only

concerned about

the highest

power

of n.(C) 2010 Thomas J Cortina, Carnegie Mellon University

O(n2) ("Quadratic")

n

(amount of

data)

Number of

Operation

s

n2/2 + 5n + 22n2 + 7n2


Order of Complexity


log2n O(log n)

log10n O(log n)

2(log2n) + 5 O(log n)

The logarithm base

is not written in

big O notation

since all that matters

is that the function

is logarithmic.


O(log n) ("Logarithmic")

n

(amount of

data)

Number of

Operation

s log2 n

log10 n

2(log2 n) + 5


Comparing Big O Functions

n

(amount of

data)

Number of

Operation

s

O(2n)

O(1)

O(n log n)

O(log n)

O(n2)

O(n)


Searching & Sorting

WORST CASE Order Of Complexity on N data elements

●Linear Search O(N)

●Binary Search O(log N)

●Selection Sort O(N2)

●Bubble Sort O(N2)

●Merge Sort O(N log N)

●Quick Sort O(N2)

●Sort + Binary Search O(N log N) + O(log N)

= O(N log N)(C) 2010 Thomas J Cortina, Carnegie Mellon University

Comparing Algorithms

●Assume an algorithm processes n data values. If each

operation takes 1 s to execute, how many s will it take

to run the algorithm on 100 data values if the algorithm

has the following number of computations?

Number of Computations Execution Time

n 100 s

n • log2 n 665 s

n2 10,000 s

n3 1,000,000 s = 1 sec

2n > 1030 s

n! > 10160 s(C) 2010 Thomas J Cortina, Carnegie Mellon University

Decision Problems

●A specific set of computations are classified as decision problems.

●An algorithm describes a decision problem if its output is simply YES or NO, depending on whether a certain property holds for its input.

●Example:Given a set of n shapes,

can these shapes be

arranged into a rectangle?


Monkey Puzzle Problem

●Given:oA set of n square cards whose sides are imprinted with

the upper and lower halves of colored monkeys.on is a square number, such that n = m2.oCards cannot be rotated.

●Problem:oDetermine if an arrangement of the n cards in an

m X m grid exists such that each adjacent pair of cards display the upper and lower half of a monkey of the same color.

Source: www.dwheeler.com (2002)(C) 2010 Thomas J Cortina, Carnegie Mellon University

Example

Images from: Simonas Šaltenis, Aalborg University, [email protected]

1

2

3

4

5

6

7

8

9


Analysis

Simple algorithm:

●Pick one card for each cell of m X m grid.

●Verify if each pair of touching edges make a full

monkey of the same color.

●If not, try another arrangement until a solution is

found or all possible arrangements are checked.

●Answer "YES" if a solution is found. Otherwise,

answer "NO" if all arrangements are analyzed and

no solution is found. (C) 2010 Thomas J Cortina, Carnegie Mellon University

Analysis

If there are n = 9 cards (m = 3):

To fill the first cell, we have 9 card choices.

To fill the second cell, we have 8 card

choices left.

To fill the third cell, we have 7 card choices

remaining.

etc.The total number of unique arrangements for n = 9 cards is:

9 * 8 * 7 * 6 * 5 * 4 * 3 * 2 * 1 = 362,880(C) 2010 Thomas J Cortina, Carnegie Mellon University

Analysis

For n cards, the number of arrangements to examine

is n! (n factorial)

If we can analyze one arrangement in a microsecond:

n Time to analyze all arrangements

9 362,880 s = 0.36288 s

16 20,922,789,888,000 s

≈ 242 days

25 15,511,210,043,330,985,984,000,000 s

≈ 491,520,585,955 years(C) 2010 Thomas J Cortina, Carnegie Mellon University

Map Coloring

●Given a map of n territories, can the map be

colored using k colors such that no two

adjacent territories are colored with the same

color?

●k=4: Answer is always yes.

●k=2: Only if the map contains no point that is

the junction of an odd number of territories.


Map Coloring


Map Coloring

●Given a map of 48 territories, can the map be colored

using 3 colors such that no two adjacent territories are

colored with the same color?

oPick a color for California (3 choices)

oPick a color for Nevada (3 choices)

o...

●There are 348 = 79,766,443,076,872,509,863,361

possible colorings.

●No one has come up with a better algorithmic solution

that works in general for any map, so far.(C) 2010 Thomas J Cortina, Carnegie Mellon University

Classifications

●Algorithms that are O(nk) for some

fixed k are polynomial-time algorithms.oO(1), O(log n), O(n), O(n log n), O(n2)

oreasonable, tractable

●All other algorithms are super-polynomial-

time algorithms.oO(2n), O(n!), O(nn)

ounreasonable, intractable(C) 2010 Thomas J Cortina, Carnegie Mellon University

Traveling Salesperson

●Given: a weighted graph of nodes

representing cities and edges representing

flight paths (weights represent cost)

●Is there a route that takes the salesperson

through every city and back to the starting

city with cost no more than k?oThe salesperson can visit a city only once (except

for the start and end of the trip).(C) 2010 Thomas J Cortina, Carnegie Mellon University


A

B

D

C

G

E

F

12

64

59

810

7 11

3

7

7

Is there a route with cost at most 52? YES (Route above costs 50.)

Is there a route with cost at most 48? YES? NO?(C) 2010 Thomas J Cortina, Carnegie Mellon University


●If there are n cities, what is the maximum number of

routes that we might need to compute?

●Worst-case: There is a flight available between

every pair of cities.

●Compute cost of every possible route.oPick a starting city

oPick the next city (n-1 choices remaining)

oPick the next city (n-2 choices remaining)

o...

●Maximum number of routes: (n-1)! = O(n!)

how

to

build

a

route


P and NP

●The class P consists of all those decision problems

that can be solved on a deterministic sequential

machine (e.g. a computer) in an amount of time that

is polynomial with respect to the size of the input

●The class NP consists of all those decision

problems whose positive solutions can be verified in

polynomial time given the right information.

from Wikipedia


NP Complete

●The class NPC consists of all those problems in NP

that are least likely to be in P.oEach of these problems is called NP Complete.

oMonkey puzzle, Traveling salesperson, Hamiltonian path,

map coloring, satisfiability are all in NPC.

●Every problem in NPC can be transformed to another

problem in NPC.oIf there were some way to solve one of these problems in

polynomial time, we should be able to solve all of these

problems in polynomial time.(C) 2010 Thomas J Cortina, Carnegie Mellon University

Complexity Classes

NP Problems

P Problems

NP Complete

Problems

But does P = NP?

If P ≠ NP, then all decision problems can be broken

down into this classification scheme.

If P = NP, then all three classes are one and the same.The Clay Mathematics Institute is offering a $1M prize

for the first person to prove P = NP or P ≠ NP. (http://www.claymath.org/millennium/P_vs_NP/)

We know that P < NP, since

any problem that can be solved

in polynomial time can certainly

have a solution verified in

polynomial time.



It gets worse...

●Tractable ProblemsoProblems that have reasonable, polynomial-time

solutions

●Intractable ProblemsoProblems that have no reasonable, polynomial-time

solutions

●Noncomputable ProblemsoProblems that have no algorithms at all to solve

them(C) 2010 Thomas J Cortina, Carnegie Mellon University

Noncomputability and

Undecidability

●An algorithmic problem that has no algorithm is called noncomputable.

●If the noncomputable algorithm requires a yes/no answer, the problem is called undecidable.

●Example:oGiven any set of any number of different tile designs

(examples shown above), with an infinite number of each type of tile, can we tile any area with these tiles so that like colored edges touch?

oThis problem is undecidable!(C) 2010 Thomas J Cortina, Carnegie Mellon University

Tiling Problem

YES

Note the periodicity in the tiling.(C) 2010 Thomas J Cortina, Carnegie Mellon University

Tiling Problem

NO

For this 3 X 3 room, if we try all 39

tiling configurations, no tiling

works.(C) 2010 Thomas J Cortina, Carnegie Mellon University

Tiling Problem

●Possible algorithm: oIf we find a repeating pattern, report YES.oIf we find a floor we cannot tile, report NO.

●BUT: there are some tilings which have no repeating pattern!


Tiling Problem

●The only way to know if this set of tiles can tile every finite-sized floor is to evaluate every possible floor.

●BUT: there are an infinite number of finite-sized floors!oSo we could never answer YES in this case.

●This problem is undecidable.


Another Undecidable Problem:

The Barber Paradox

Suppose there is a town with one

male barber; and that every man in

the town keeps himself clean-shaven:

some shave themselves and some

are shaved by the barber. Only the

barber can shave another man. The

barber shaves all and only those men

who do not shave themselves.

Does the barber shave himself?(C) 2010 Thomas J Cortina, Carnegie Mellon University

Program Termination

●Can we determine if a program will terminate given

a valid input?

●Example:

1. Input x

2. While x is not equal to 1, do the following:

(a) Subtract 2 from x.●Does this algorithm terminate when x = 15105?

●Does this algorithm terminate when x = 2008?


Program Termination

●Another Example:

1. Input x

2. While x is not equal to 1, do the following:

(a) If x is even, divide x by 2.

(b) Otherwise, Set x to 3x + 1.●Does this algorithm terminate for x = 15?

●Does this algorithm terminate for x = 105?

●Does this algorithm terminate for any positive x?


The Halting Problem

●Can we write a general program Q that takes

as its input any program P and an input I and determines if

program P will terminate (halt) when run with input I?oIt will answer YES if P terminates successfully on input I.

oIt will answer NO if P never terminates on input I.

●This computational problem is undecidable!oNo such general program Q can exist!

oIt doesn’t matter how powerful the computer is.

oIt doesn’t matter how much time we devote to the computation.

oThe proof of this involves contradiction.(C) 2010 Thomas J Cortina, Carnegie Mellon University

start

end

Does this

algorithm end?

yes

no

Contradiction


Contradiction isn't just for

computer scientists...


Software

Computability - Tractable, Intractable and Non-computable Function