Growth of Functions

2

Analysis of Bubble Sort

3

Time Analysis – Best Case

The array is already sorted – no swap operations are required

1 1 1

1 0 1

( ) ( (1)) ( )

( 1)1 2 ....( 1)

2

n i n

i j i

T n i

n nn

4

Time Analysis – Worst Case

Array is sorted in descending order –all swap operations will be executed

In both cases – O(n2)

1 1

1 0

1 1

1 0

( ) ( (1 ))

( 1)( 1) 1 ( 1)

2

n i

i j

n i

i j

T n k

n nk k

5

Asymptotic Notation

We are given:Two algorithms, A and B

The problem size: n

The running time for both algorithms: TA(n) and TB(n)

Need a way of comparing the functions TA(n) and TB(n), to determine which algorithm is better (more efficient)

6

Asymptotic NotationFor a given problem size k,if TA(k) < TB(k) then algorithm A is better than algorithm B

To choose between the two algorithms, we need to find the one that is (almost) always better

Want to show that TA(n) < TB(n) for all n

Solution: compare asymptotic behavior

7

Asymptotic Upper Bound – O

For a given function g(n),we define the group O(g(n)) as:

} allfor

)()(0

:such that and

constants positiveexist there:)({))((

0

0

nn

ncgnf

nc

nfngO

8

Asymptotic Upper Bound – O

f(n)

cg(n)

0n

0 ( ) ( )for all n n f n cg n

9

Example

2( 1)( ) , ( )

2

n nf n g n n

2( 1): ( )

2

n nprove O n

0 0

2

0

2

, 0 .

( 1)

2

1 1

( 1)

2

find c n s t for all n n

n ncn

for n and c

n nn

10

Asymptotic Lower Bound – Ω

For a given function g(n),we define the group Ω(g(n)) as:

} allfor

)( )(0

:such that and

constants positiveexist there:)({))((

0

0

nn

nfncg

nc

nfng

11

Asymptotic Lower Bound – Ω

cg(n)

f(n)

0n

0 ( ) ( )for all n n f n cg n

12

Example

2( 1)( ) , ( )

2

n nf n g n n

2( 1): ( )

2

n nprove n

0 0

2

0

2

0

2

, 0 .

( 1)

21

4( 1) 1

2 4

12

2 2

( 1) 1

2 4

find c n s t for all n n

n ncn

we choose c and find n

n nn

nn

n so for n

n nn

13

Asymptotic Tight Bound – Θ

For a given function g(n),we define the group Θ(g(n)) as:

} allfor

)()()(0

:such that and,

constants positiveexist there:)({))((

0

21

021

nn

ngcnfngc

n cc

nfng

14

Asymptotic Tight Bound – Θ

cg(n)

f(n)

0n

0 1 2( ) ( ) ( )for all n n c g n f n c g n

dg(n)

15

Example

2

2 2

2

2

( 1)( ) , ( )

2

1 ( 1) 1

2 2 4( 1)

( )2

( ) ( )

n nf n g n n

n nn n

n nn

f n n

16

Asymptotic Bounds - Summary

} allfor )()( :such that

andconstants positiveexist there:)({))((

0

0

nnncgnf

n c nfngO

} allfor )(0 :such that

and constants positiveexist there:)({))((

0

0

nnnfcg(n)

n cnfng

} allfor

)()(0 :such that

andconstants positiveexist there:)({))((

0

21

021

nn

ngcnfg(n)c

n, c cnfng

17

Asymptotic Notation

When we write f(n) = O(n)we actually mean: f(n) O(n)

When we write we mean that aside from the function , the expression includes an additional function from O(n) which we have no interest of stating explicitly

2 22 3 1 2 ( )n n n O n

22n

18

Example

Prove:f(n) = 8n + 128 = O(n2)

Let us choose c = 1, then: 2

2

2

0

( )

8 128

0 8 128

0 ( 8)( 16)

16

f n cn

n n

n n

n n

n

19

Conventions for Using Asymptotic Notation

Drop all but the most significant terms

Instead of:

we write:

Drop constant coefficientsInstead of:

we write:

3 2( 3 4)O n n 3( )O n

3(2 )O n3( )O n

20

Asymptotic Notation and Polynomials

Consider the following degree-d polynomial in n, where :

Use the definitions of asymptotic notation to prove the following properties:

0da

d

i

iinanp

0

)(

21

Asymptotic Notation and Polynomials

For any constant k:

If then:

If then:

If then:

dk )()( knOnp

dk

dk

)()( knnp

)()( knnp

22

Asymptotic Notation and Polynomials – Proof

If then: :

First, we mark:

Now:

By choosing c=b·d, we get:

dk )()( knOnp

d

i

ii nanp

0

)(

kncnp )(

kdd

i

dd

i

i ncndbnbnb 00

ii

ab max

23

Asymptotic Notation and Polynomials – Proof

If then: :

By choosing c=ad, we get:

dk )()( knnp

kncnp )(

0

( )d

i di d

i

p n a n a n

kk

d ncna

24

Asymptotic Notation and Polynomials – Proof

If then: :

Since k = d, also k ≤ d and k d

So, from the previous sections:

and

From this we get:

dk )()( knnp

)()( knOnp )()( knnp

)()( knnp

25

Example: Comparing the Functions and

We prove that :

Can we prove that ?

Assume, by way of contradiction, that:

!n

1 2 3 .... ....

! ! ( )n n

n n n n n

n n n O n

! ( )nn n

! ( )nn n

nn

! nn c n

26

Example: Comparing the Functions and

For any n such that :

Or:

From the assumption:

Contradiction:

11 2 ... ( 1) 1 ... 1 nn n n n n n 1! nn n

3n

1 !n nn n c n 1

nc

!n nn

27

Example

Compare the functions

The logarithmic base does not change the order of magnitude

log , loga bn n

, 1

loglog log (log )

logb

a b bb

a b

nn c n n

a

28

Properties of Asymptotic Notation – Transitivity

Prove: ( ) ( ( )) ( ) ( ( ))

( ) ( ( ))

g n O f n and f n O h n

g n O h n

1 1 1 1

2 2 2 2

1 2 1 1 2

( ) ( ( )) , . . : ( ) ( )

( ) ( ( )) , . . : ( ) ( )

max( , ), ( ) ( ) ( )

( ) ( ( ))

g n O f n c n s t n n g n c f n

f n O h n c n s t n n f n c h n

n n n g n c f n c c h n

g n O h n

29

Properties of Asymptotic Notation – Symmetry

Prove: ( ) ( ( )) ( ) ( ( ))g n f n f n g n

1 2 0

0 1 2

3 41 2

4 3

( ) ( ( )) , , :

, ( ) ( ) ( )

1 1,

( ) ( ) ( )

g n f n c c n such that

n n c f n g n c f n

let c cc c

c g n f n c g n

30

Properties of Asymptotic Notation – Reflexivity

Prove: ( ) ( ( )), ( ) ( ( )),

( ) ( ( ))

f n O f n f n f n

f n f n

1

( ) ( ) ( )

( ) ( ( ))

( ) ( ( )) , ( ) ( ( ))

for c and any n

cf n f n cf n

f n f n

f n O f n f n f n

31

Properties of Asymptotic Notation – Multiplication

If and

then:

By multiplying the equations, we get:

1( ) ( ( ))T n O f n 2 ( ) ( ( ))T n O g n

1 2( ) ( ) ( ( ) ( ))T n T n O f n g n

1 1 1 1 1 1

2 2 2 2 2 2

( ) ( ( )) , . . : : ( ) ( )

( ) ( ( )) , . . : : ( ) ( )

T n O f n c n s t n n T n c f n

T n O g n c n s t n n T n c g n

1 2 1 2 1 2max( , ) : ( ) ( ) ( ) ( )n n n T n T n c c f n g n

32

Exercises

For each of the following statements,prove it, or explain why it is false:

The claim is trueWe choose c = 2, and get:

12 (2 )n nO

12 2 2 2 (2 )n n n nc O

33

The claim is false:

Cannot find a constant c such that:

Exercises

22 (2 )n nO

22 2 2 2 2 (2 )n n n n n n nc O

2n c

34

Exercises

Show that

We prove the claim in two steps:

log( !) ( log )n n n

log( !) ( log )n O n n

log( !) ( log )n n n

35

log( !) ( log )n O n n

! 1 2 3...

log( !) log( ) log

( 1)

n

n

n n n

n n n n

c

Exercises

Prove:

36

log( !) ( log )n n n

2

2

! 1 2 ...2 2 2 2

log( !) log(( ) ) log( )2 2 2

(log log 2) (log )2 2 2

n

n

n n n nn n

n n nn

n n nn n

Exercises

Prove:

37

Exercises

Left to show:

We choose: c = ¼

(log ) log2 2

n nn c n n

0

log log

2 2 4log

4 2log 2 =4

n n n n n

n n n

n n