Upload
amelia-davidson
View
234
Download
0
Embed Size (px)
Citation preview
한양대학교 정보보호 및 알고리즘 연구실
2008. 2. 12이재준
담당교수님 : 박희진 교수님1
Chapter 3. Growth of functionChapter 4. Recurrences
Contents of Table
1. Technicalities for abbreviation
2. Recurrence solution methodsThe substitution method
2
2. Notation of asymptotically larger/smaller
o-notation ω-notation
3. Relationship between this notations
A. Growth of function
B. Recurrence
f(n) is asymptotically smaller than g(n) f(n) is asymptotically smaller than g(n)
• o-notation
Asymptotically smaller
- Definition for all n, n ≥ n0 , any positive
constant c, c>0, 0 ≤ f(n)< cg(n), the function f(n) is smaller to g(n) to within constant factor.
3
2. Asymptotically larger/smaller notation
f(n)
o(g(n))
n0
4
2. Asymptotically larger/smaller notation
• o-notation
- Definitiono(g(n)) = f(n) : for any positive constant c>0, there exist a constant
n0 >0 such that 0 ≤ f(n) < cg(n) for all n ≥ n0
0)(
)(lim
ng
nf
n
this limit shows the function f(n) becomes insignificant relative to g(n) as n approaches infinity
• o-notation Example
1. Asymptotic notation
Use the definition of o-notation to prove the following property.
f(n) = o(n2)
Use the definition of o-notation to prove the following property.
f(n) = o(n2)
nnf 2)(
6
2. Notation of asymptotically larger/smaller
• o-notation Solution to example
2n = o(n2)
If f(n) = o((g(n)) then
0)(
)(lim
ng
nf
n
f(n) is asymptotically larger than g(n) f(n) is asymptotically larger than g(n)
• ω -notation
Asymptotically larger
- Definition for all n, n ≥ n0 , any positive
constant c, c>0, 0 ≤ cg(n) < f(n), the function f(n) is larger to g(n) to within constant factor.
7
2. Notation of asymptotically larger/smaller
f(n)
ω(g(n))
8
2. Notation of asymptotically larger/smaller
• ω-notation
- Definition ω(g(n)) = f(n) : for any positive constant c>0, there exist a constant
n0 >0 such that 0 ≤ cg(n) < f(n) for all n ≥ n0
)(
)(lim ng
nf
n
if the limit exists. That is, f(n) becomes arbitrarily large relative to g(n) as n approaches infinity.
• ω -notation
- We use ω-notation to denote a lower bound that is not asymptotically tight.
- We use o-notation to denote an upper bound that is not asymptotically tight.
- f(n) ω(∈ g(n)) if and only if g(n) ∈ o(f(n)).
9
2. Notation of asymptotically larger/smaller
10
2. Notation of asymptotically larger/smaller
• ω -notation Example
Use the definition of o-notation to prove the following property.
f(n) = ω(n)
Use the definition of o-notation to prove the following property.
f(n) = ω(n)
2)(
2nnf
11
2. Notation of asymptotically larger/smaller
• ω -notation Solution to example
=ω(n)
If f(n) = ω((g(n)) then
2n
)(
)(lim ng
nf
n
12
3. Relationship between this notations
f(n)
O(g(n))
o(g(n))
ω(g(n))
Ω(g(n))
θ(g(n))
O(g(n)) Ω(g(n))
ω(g(n)) o(g(n))
13
3. Relationship between this notations
f(n)
O(g(n))
o(g(n))
ω(g(n))
Ω(g(n))
• Transitivity
• Transpose symmetry
f(n) = Θ(g(n)) and g(n) = Θ(h(n)) imply f(n) = Θ(h(n)) ,f(n) = O(g(n)) and g(n) = O(h(n)) imply f(n) = O(h(n)) , f(n) = Ω(g(n)) and g(n) = Ω(h(n)) imply f(n) = Ω(h(n)) ,f(n) = o(g(n)) and g(n) = o(h(n)) imply f(n) = o(h(n)) ,f(n) = ω(g(n)) and g(n) = ω(h(n)) imply f(n) = ω(h(n)).
f(n) = O(g(n)) if and only if g(n) = Ω(f(n)),f(n) = o(g(n)) if and only if g(n) = ω(f(n)).
14
3. Relationship between this notations
• Reflexivity
f(n) = Θ(f(n))f(n) = O(f(n))f(n) = Ω(f(n))
• Symmetry
f(n) = Θ(g(n)) if and only if g(n) = Θ(f(n)).
θ(g(n))
O(g(n)) Ω(g(n))
Recurrences
15
• Recurrence
When should we use the recurrence?
When algorithm contains a recursive call to itself, Its running time can often be described by a recurrence.
Recurrences
16
• Recurrence
- Definition Equation or inequality that describes a function in terms
of its value on smaller inputs.
Recurrences
17
• Recurrence Example
Worst-case running time of T(n) could be described by the recurrence
Merge-Sort procedure
Show that the solution of T(n) = Θ(n log n)
Merge-Sort procedure
Show that the solution of T(n) = Θ(n log n)
)()2/(2
)1()(
nnTnT
if n=1
if n>1
Recurrences
18
• Recurrence Solution method to example
- The substitution method
guess a bound and prove our guess correct for small input.
- The recursion-tree method
convert the recurrence into a tree whose nodes represent the costs incurred at various level of recursion
- The master method
determining asymptotic bounds for many simple recurrences of the form
)()/()( nfbnaTnT ( a ≥ 1, b ≥ 1, and f(n) is given function )
1. Technicalities for abbreviation
• Technicalities
- assumption of integerThe running time T(n) of algorithm is only defined when
n is an integer
- floors / ceilingsWe often omit floors and ceilings
- boundary conditionThe running time of an algorithm on a constant-sized
input is a constant that we typically ignore.
1. Technicalities for abbreviation
• Technicalities ExampleThe recurrence describing the worst-case running time of Merge-Sort
Omit assumption of integer
Omit boundary condition
)()2/(2
)1()(
nnTnT
if n=1
if n>1
if n=1
if n>1
)()2/()2/(
)1()(
nnTnTnT
)()2/(2)( nnTnT
)()2/(2
)1()(
nnTnT
if n=1
if n>1
Omit floors and ceilings
2. Recurrence solution methods
21
• Substitution Method
- DefinitionWhen recurrence is easy to guess form of the
answer withthe inductive hypothesis is applied to smaller
values,substitution of the guessed answer for the function
.
- It can be used to establish either upper or lower bounds on a recurrence.
2. Recurrence solution methods
22
• Substitution Method Steps
1. Guess the form of the solution
2. Use mathematical induction to find the constant and show that the solution works.
1. Guess the form of the solution
2. Use mathematical induction to find the constant and show that the solution works.
2. Recurrence solution methods
23
• Substitution Method - Making a good guess
prove loose upper and lower bounds on the recurrence
and then reduce the range of uncertainty.
- Guessing a solution takes : experience, occasionally, creativity
- we can use recursion tree
2. Recurrence solution methods
24
• Substitution Method - Subtleties
we can prove something stronger for given value by assuming something stronger for a smaller values.
1)2/()2/(2)( nTnTnTWe guess that the solution is O(n), and show that T(n) ≤ cn
12/2/)( ncncnT
1cnOvercome difficulty by subtracting a lower-order term from our previous guess
bcnnT )(
bcn
bcn
12
( b≥1, c must be chosen large enough )
2. Recurrence solution methods
25
• Substitution Method - Avoid pitfalls
nncnT )2/(2)(
)(
2
nO
ncn
)()2/(2)( nnTnT In recurrence we can falsely prove T(n)= O(n) by guessing T(n) ≤ cn
2. Recurrence solution methods
26
• Substitution Method Example
Let us determine an upper bound on the recurrence
Let us determine an upper bound on the recurrence
nnTnT )2/(2)(
2. Recurrence solution methods
27
• Substitution Method Solution to example
1. guess that the solution is 2. prove that ( c > 0 ) 1. guess that the solution is 2. prove that ( c > 0 )
)lg()( nnnT ncnnT lg)(
- Assume that this bound holds for ⌊n/2 , ⌋ that is, T (⌊n/2 ) ≤ ⌋ c ⌊n/2 ⌋lg(⌊n/2 ).⌋
T(n) ≤ 2(c ⌊n/2 lg(⌋ ⌊n/2 )) + ⌋ n
≤ cn lg(n/2) + n ( because, ⌊n/2 < ⌋ n/2 )
= cn lg n - cn lg 2 + n = cn lg n - cn + n ≤ cn lg n (as long as c ≥ 1)
We need to find constant c and n0
2. Recurrence solution methods
28
• Substitution Method Solution to example
- Find the constant n0
- we can take advantage of asymptotic notation,we can replace base case
T(1) ≤ cn lg n T(1) = 1 but, c1 lg1 = 0
- So, we only have to prove T (n) = cn lg n for n ≥ n0 for n (n0 =2)
Replace T(1) by T(2) and T(3) as the base cases by letting n0 =2.
Replace T(1) by T(2) and T(3) as the base cases by letting n0 =2.
2. Recurrence solution methods
29
• Substitution Method Solution to example
- Find the constant c
- choosing c large enough
T (2) = 4 and T (3) = 5.T (2) = 4 ≤ c ( 2 lg 2 ) T (3) = 5 ≤ c ( 3 lg 3 ).
Any choice of c ≥ 2 suffices for base case of n=2 and n =3 Any choice of c ≥ 2 suffices for base case of n=2 and n =3
2. Recurrence solution methods
30
• Substitution Method Example
Let us determine an upper bound on the recurrence
Let us determine an upper bound on the recurrence
nnTnT lg)(2)(
2. Recurrence solution methods
31
• Substitution Method Solution to example
- Simplify this recurrence by changing variable
nnTnT lg)(2)(
mTT mm )2(2)2( 2/
mmSmS )2/(2)(
Renaming m = lg n
S(m) = T(2m)
This new recurrence has same solution : S(m) = O(m log m)
2. Recurrence solution methods
32
• Substitution Method Solution to example
- Simplify this recurrence by changing variable mmSmS )2/(2)(
This new recurrence has same solution : S(m) = O(m log m)
Changing back from S(m) to T(n)
T(n) = T(2m) = S(m)= O(m lg m)= O( lg n lg(lg n) )