이산수학(Discrete Mathematics) - Kangwoncs.kangwon.ac.kr/~ysmoon/courses/2012_1/dm/11.pdf · 2016-06-02 · Discrete Mathematics Page 3 by Yang-Sae Moon Complexity Depends on

2012년 봄학기

강원대학교 컴퓨터과학전공 문양세

이산수학(Discrete Mathematics)

알고리즘의 복잡도

(Algorithm Complexity)

Discrete Mathematics by Yang-Sae Moon Page 2

What is Algorithm Complexity?

The algorithmic complexity of a computation is some

measure of how difficult it is to perform the computation.

(문제 계산(computation)이 얼마나 어려운가를 나타내는 측정치이다.)

Measures some aspect of cost of computation (in a general

sense of cost). (계산을 위한 비용에 대한 측정치이다.)

Common complexity measures:

• Time complexity: # of operations or steps required

• Space complexity: # of memory bits required

2.3 Algorithm Complexity


Complexity Depends on Input

Most algorithms have different complexities for inputs of

different sizes. (E.g. searching a long list takes more time

than searching a short one.)

(대부분의 알고리즘은 입력의 크기에 따라 복잡도가 달라진다. 당연!)

Therefore, complexity is usually expressed as a function of

input length.

(따라서, 복잡도는 입력의 크기/길이에 대한 함수로 표현한다.)

This function usually gives the complexity for the worst-

case input of any given length.

(복잡도를 나타내는 함수는 통상 입력 크기가 최악인 경우를 고려한다.)



Example: Max Algorithm

Problem: Find the exact order of growth () of the worst-

case time complexity of the max algorithm.

Assume that each line of code takes some constant time

every time it is executed.


procedure max(a1, a2, …, an: integers)

v := a1 {largest element so far}

for i := 2 to n {go thru rest of elems}

if ai > v then v := ai {found bigger?}

{at this point v’s value is the same as the largest

integer in the list}

return v


Complexity Analysis of Max Algorithm (1/2)

What’s an expression for the exact total worst-case time?

(Not its order of growth.) (최악의 경우, 정확한 수행 시간을 어떻게 될까?)



v := a1 t1

for i := 2 to n t2

if ai > v then v := ai t3

return v t4

Times for each

execution of each line.

(각 line을 하나의 수행으로

볼 때의 시간)


Complexity Analysis of Max Algorithm (2/2)

Worst case execution time:



v := a1 t1

for i := 2 to n t2

if ai > v then v := ai t3

return v t4

Times for each

execution of each line.

(각 line을 하나의 수행으로

볼 때의 시간)

)()()1()1()()1(

)1()1()1()1()1()1(

)()(

2

4

2

321

nnn

n

ttttnt

n

i

n

i


Example: Linear Search 2.3 Algorithm Complexity

procedure linear search x: integer, a1, a2, …, an: distinct integers)

i := 1 t1

while (i n x ai) t2

i := i + 1 t3

if i n then location := i t4

else location := 0 t5

return location t6

Worst case:

Best case:

Average case (if item is present):

)()()( 654

1

321 nttttttntn

i

)1()( 6421 ttttnt

)()()( 654

2/

1

321 nttttttntn

i


Example: Binary Search 2.3 Algorithm Complexity

procedure binary search (x:integer, a1, a2, …, an: distinct integers)

i := 1

j := n

while i<j begin

m := (i+j)/2

if x>am then i := m+1 else j := m

end

if x = ai then location := i else location := 0

return location

(1)

(1)

(1)

Key Question: How Many Loop Iterations?


Binary Search Analysis

Suppose n=2k.

Original range from i=1 to j=n contains n elements.

Each iteration: Size ji+1 of range is cut in half.

(매번 검색 범위(j-i+1)의 절반(½ )씩 줄여 나간다.)

Loop terminates when size of range is 1=20 (i=j).

(검색 범위의 변화: 2k 2k-1 2k-2 … 21 20, 결국 반복 횟수 = k)

Therefore, number of iterations is k = log2n

= (log2n)= (log n) (반복 횟수 = k이고, log2n = log22k이므로…)

Even for n2k (not an integral power of 2), time

complexity is still (log2n) = (log n).



Example: Bubble Sort 2.3 Algorithm Complexity

3

2

4

1

5

2

3

4

1

5

2

3

4

1

5

2

3

1

4

5

2

3

1

4

5

2

3

1

4

5

2

1

3

4

5

2

1

3

4

5

1

2

3

4

5

1

2

3

4

5

2nd pass 1st pass

3rd pass 4th pass

Consider # of compare operations only!

(n-1) + (n-2) + … + 2 + 1 = ((n-1)n)/2 = (n2)


Example: Insertion Sort 2.3 Algorithm Complexity

Also, consider # of compare operations only!

1 + 2 + … + (n-2) + (n-1) = ((n-1)n)/2 = (n2)

3

2

4

1

5

2

3

4

1

5

2

3

4

1

5

2

3

4

1

5

2

3

4

1

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

Then, are all sorting algorithm’s complexities (n2)?

NO! …, merge sort, heap sort, quick sort, …


Names for Some Orders of Growth

(1) Constant

(log n) Logarithmic

(n) Linear

(n log n) Polylogarithmic

(nc) Polynomial

(cn), c>1 Exponential

(n!) Factorial



Tractable versus Intractable

A problem or algorithm with at most polynomial time

complexity is considered tractable (or feasible). P is the

set of all tractable problems. (Polynomial 복잡도를 가지면, 풀만한(풀기 쉬운) 문제라 여기며, 이러한 문제들

의 집합을 P라 나타낸다.)

A problem or algorithm that has more than polynomial

complexity is considered intractable (or infeasible). (Polynomial 복잡도 이상이면 풀기 어려운 문제라 여긴다.)

But, note that

• n1,000,000 is technically tractable, but really impossible.

• clog log log n is technically intractable, but easy.



Unsolvable Problems

Alan Turing discovered in the 1930’s that there are

problems unsolvable by any algorithm.

(튜링은 “어떠한 알고리즘으로도 풀 수 없는 문제가 있음”을 밝혔다.)

Example: the halting problem.

• Given an arbitrary algorithm and its input, will that algorithm

eventually halt, or will it continue forever in an “infinite loop?”

(주어진 알고리즘이 결국 정지하는지의 여부를 판단하는 문제는 결국 풀 수가 없다.)

• We will handle the halting problem in $3.1 again.



P versus NP (1/3)

P is the set of all tractable problems. (i.e., the problems

have at most polynomial time complexity.)

(통상적으로 polynomial complexity를 가지면 P이다.)

NP is the set of problems for which there exists a tractable

algorithm for checking solutions to see if they are correct.

• In other words, NP is the set of decision problems for which a given

proposed solution for a given input can be checked quickly (in

polynomial time) to see if it really is a solution.

• 주어진 입력에 대해 제시된 솔루션이 바른 해인지의 여부를 빠르게(poly-

nomial time) 판단할 수 있는 알고리즘이 존재하는 문제들의 집합이 NP이다.

• 통상적으로 exponential/factorial complexity를 가지면 NP이다.



P versus NP (2/3)

NP-complete is the term used to describe decision problems that are

the hardest ones in NP in the sense that, if there were a polynomial-

bounded algorithm for an NP complete problem, then there would be a

polynomial-bounded algorithm for each problem in NP.

• NP중에서도 어려운 문제들이란 용어를 의미하며, 하나의 NP-complete 문제가 풀리면

(P가 되면), 관련된 수많은 모든 NP-complete 문제가 동시에 풀리게 된다.

• 많은 학자들은 어려운 많은 문제들에 대해서 NP-complete임을 증명(특정 문제가 다른

NP-complete 문제로 해석(변경)될 수 있음을 보임)하였다.



P versus NP (3/3)

We know PNP, but the most famous unproven conjecture in

computer science is that this inclusion is proper (i.e., that PNP

rather than P=NP).

Whoever first proves it will be famous!



Computer Time Examples 2.3 Algorithm Complexity

Assume time

= 1 ns (109

second) per

operation

#ops(n) n=10 n=106

log2 n 3.3 ns 19.9 ns

n 10 ns 1 ms

n log2 n 33 ns 19.9 ms

n2 100 ns 16 m 40 s

2n 1.024 s 10301,004.5

Gyr

n! 3.63 ms Ouch!

You should carefully design algorithms and write programs!


Homework #3 2.3 Algorithm Complexity

Documents

이산수학(Discrete Mathematics) - Kangwoncs.kangwon.ac.kr/~ysmoon/courses/2012_1/dm/11.pdf · 2016-06-02 · Discrete Mathematics Page 3 by Yang-Sae Moon Complexity Depends on