Fundamentals of the Analysis of Algorithm Efficiency (2)

DR. J IRABHORN CHAIWONGSAI

ดร.จิราพร ไชยวงศสายD E P A R T M E N T O F C O M P U T E R E N G I N E E R I N G

S C H O O L O F I N F O R M AT I O N A N D C O M M U N I C AT I O N T E C H N O L O G Y

U N I V E R S I T Y O F P H AYA O

• สิ่งที่ใชแทนความสัมพันธของฟงกชันในแงของอัตราการเติบโต

• ประเมินประสิทธิภาพของอัลกอริทึม

• A way of comparing functions that ignores constant factors and small input sizes

• มี 3 ตัวดังนี้ θ, O, และ Ω Big oh (O)

Big theta (θ)

Big omega (Ω)

**ใชกับฟงกชันที่ใหคาไมติดลบ

Asymptotic Notations (สัญกรเชิงเสนกํากับ)

2

• O(g(n)): class of functions f(n) that grow not faster than g(n)

• Θ(g(n)): class of functions f(n) that grow at same rate as g(n)

• Ω(g(n)): class of functions f(n) that grow at least as fast as g(n)

Asymptotic Notations

3

Big-oh

Definition: f(n) is in O(g(n)) if order of growth of

f(n) ≤ order of growth of g(n) (within constant multiple),

i.e., there exist positive constant c and non-negative

integer n0 such that

f(n) ≤ c g(n) for every n ≥ n0

Example1:

• 10n is in O(n2) ? 10n ϵ O(n2)

• 5n+20 is in O(n) ? 5n+20 ϵ O(n)

4

Big-oh (cont.)

5

Big-oh (cont.)

6

Definition: f(n) is in Ω(g(n)) if order of growth of f(n) ≥ order of growth of g(n) (within constant multiple),i.e., there exist positive constant c and non-negative integer n0 such that

f(n) ≥ c g(n) for every n ≥ n0

Example2:

• n3 is in Ω(n2) ?

• Show that (n/2)log(n/2) = Ω (nlog(n))

Big-omega

7

Big-omega (cont.)

8

Big-omega (cont.)

9

Definition: f(n) is in θ(g(n)) if order of growth of f(n) is bounded above and below order of growth of g(n) (within constant multiple),i.e., there exist positive constant c1 and c2 and some non-negative integer n0 such that

c2 g(n) ≤ f(n) ≤ c1 g(n) for every n ≥ n0

Example3: Prove that ϵ θ (n log n)

Big-theta

1

2� � − 1

10

Big-theta (cont.)

11

• Transitivity:

12

• Reflexivity

• Symmetry

))(()(

))(()(

))(()(

nfnf

nfOnf

nfnf

))(()())(()( nfngngnf

13

Properties of Asymptotic Notations

• Transpose Symmetry

))(()())(()( nfngngOnf

14

Properties of Asymptotic Notations

• เราสามารถลดรูปเพื่อใหอยูในรูปแบบที่งายขึ้น ดังนี้• กําหนดให f1(n) = O(g1(n)) และ f2(n) = O(g2(n))

f1(n) + f2(n) = O(g1(n) + g2(n)) f1(n) + f2(n) = O(max(g1(n), g2(n))) f1(n) • f2(n) = O(g1(n) • g2(n)) f1(n)k = O(g1(n)k)

• a

n

k

n

k

kfOkfO1 1

)())((

15

• ใชกับสวนของฟงกชันที่มีอัตราการเติบโตที่ชากวา

• ใชในกรณีที่ไมทราบรายละเอียด หรือรายละเอียดนั้นยืดยาวมาก

• ใชเขียนเปนสวนหนึ่งของความสัมพันธเวียนเกิด เพื่อแทนประสิทธิภาพของอัลกอริทึม

16

f(n) = 2n3+3n+7.5/n

f(n) = 2n3 + θ(n)

42 12012

1

2

1ln

nnnnH n

n

)/1(ln nnHn

17

Orders of growth of some important functions

• ใน Asymptotic Notations นั้น

• ไมวาจะเปน log ฐานอะไร ก็จะมีอัตราการเติบโตเทากัน เชน log2(n) = θ(log1000(n))

• เลขชี้กําลังภายใน log ไมมีผลตออัตราการเติบโตเชน log(n2) = θ(log n0.1)

18

• All logarithmic functions loga n belong to the same class θ(log n) no matter what the logarithm’s base a > 1 is

• All polynomials of the same degree k belong to the same class:

aknk + ak-1nk-1 + … + a0 (nk)

• Exponential functions an have different orders of growth for different a’s

• order log n < order n (>0) < order an < order n! < order nn

Orders of growth of some important functions

19

Basic asymptotic efficiency classes

1 constant

log n logarithmic

n linear

n log n n-log-n or linearithmic

n2 quadratic

n3 cubic

2n exponential

n! factorial

20

Time efficiency of non-recursive algorithms

General Plan for Analysis

• Decide on parameter n indicating input size

• Identify algorithm’s basic operation

• Determine worst, average, and best cases for input of size n

• Set up a sum for the number of times the basic operation is executed

21

Useful summation formulas and rules

• liu1 = 1+1+ ⋯ +1 = u - l + 1

In particular, liu1 = n - 1 + 1 = n (n)

• 1in i = 1+2+ ⋯ +n = n(n+1)/2 n2/2 (n2)

• 1in i2 = 12+22+ ⋯ +n2 = n(n+1)(2n+1)/6 n3/3 (n3)

• 0in ai = 1 + a + ⋯ + an = (an+1 - 1)/(a - 1) for any a 1

In particular, 0in 2i = 20 + 21 + ⋯ + 2n = 2n+1 - 1 (2n )

22

Mathematical Analysis of Non-recursive algorithms

Example 1: Maximum element

Problem Input size Basic operation

23

Example 2: Element uniqueness problem

Problem Input size Basic operation

24

Example 3: Matrix multiplication

Problem Input size Basic operation

25

Example 4: Counting binary digits

It cannot be investigated the way the previous examples are.

Problem Input size Basic operation

26

Time efficiency of recursive algorithms

Plan for Analysis of Recursive Algorithms

• Decide on a parameter indicating an input’s size.

• Identify the algorithm’s basic operation.

• Check whether the number of times the basic op. is executed may vary on different inputs of the same size. (If it may, the worst, average, and best cases must be investigated separately.)

• Set up a recurrence relation with an appropriate initial condition expressing the number of times the basic op. is executed.

• Solve the recurrence (or, at the very least, establish its solution’s order of growth) by backward substitutions or another method.

27

Example 1: Recursive evaluation of n!

Definition: n ! = 1 2 … (n-1) n for n ≥ 1 and 0! = 1

Recursive definition of n!: F(n) = F(n-1) n for n ≥ 1 and

F(0) = 1

Size:

Basic operation:

Recurrence relation:

28

Solving the recurrence for M(n)

M(n) = M(n-1) + 1, M(0) = 0

29

Example 2: The Tower of Hanoi Puzzle

1

2

3

Recurrence for number of moves:

30

Solving recurrence for number of moves

M(n) = 2M(n-1) + 1, M(1) = 1

31

Example 3: Counting #bits

32