Embed Size (px)
Citation preview
2
Algorithms
• Design and Analysis of algorithms for a given problem P
• Lower Bound : 문제 자체가 가지는 복잡도의 분석
– P를 해결하기 위하여 소요되는 최소 연산의 수
– P자체의 기본성질 및 고유의 복잡도 분석
• Upper Bound: 문제를 해결하는 가장 좋은 방법
– P를 해결에 필요한 최대 연산의 수: 알고리즘
– 수학적인 분석을 통한 효율성의 증명 (효율적인 알고리즘)
– 구현 후 실제 수행시간 분석을 통한 효율성의 입증
An Example
35 25 15 65 7945 3 9 35 25 15 65 7945 3 9
3
• Finding the maximum
– Given n integer numbers, find the maximum !!
– How many comparisons are needed ?
• The lower bound : n-1
• Ex) 35, 25, 15, 65, 79, 45, 3, 9
Another Example
35 25 15 65 79 45 3 9
• Finding the 1st & 2nd maxima
– How many comparisons are needed ?
The lower bound : n –1 + log n - 1
4
5
Contents
• Asymptotic notation
– Θ-notation
– O-notation
– Ω-notation
– Other types of notation
• Standard notations and common functions
Asymptotic Notations
Θ(g(n)) = f(n) : there exist positive constants c1, c2, and n0
such that 0 ≤ c1g(n) ≤ f(n) ≤ c2g(n) for all n ≥ n0.
Θ-notation
• For all values of n to the right of n0, the value of f(n) lies at or above c1g(n) and at or below c2g(n).
• g(n) is called an asymptotically tight bound for f(n).
7
Θ-notation
• f(n) = Θ(g(n)) denotes f(n) ∈ Θ(g(n))
• Every member f(n) ∈ Θ(g(n)) is asymptotically nonnegative, that is, that f(n) be nonnegative whenever n is sufficiently large.
8
Θ-notation
• Example
• The right-hand inequality can be made to hold for any value of n ≥ 1 by choosing c2 ≥ ½.
• The left-hand inequality can be made to hold for any value of n ≥ 7 by choosing c1 ≤1/14.
• Thus, by choosing c1 = 1/14, c2 = 1/2, and n0 = 7, we can verify that 1/2n2 - 3n = Θ(n2).
9
.cn
cn
nn ncnnnc
n, ccn n n o
212
02
222
1
2122
3
2
1 yields by Dividing
. all for 32
1
that such and constants positive )(Θ32
1
Θ-notation
• Example
– Consider any quadratic function f(n) = an2 + bn + c, where a, b, and c are constants and a > 0.
– Throwing away the lower-order terms and ignoring the constant yields f(n) = Θ(n2).
– The reader may verify that 0 ≤ c1n2 ≤ an2 + bn + c ≤
c2n2 for all n ≥ n0.
– In general, for any polynomial
where the ai are constants and ad > 0, we have p(n) = Θ(nd).
10
d
i
iinanp
0)(
O-notation
11
• For all values n to the right of n0, the value of the function f(n) is on or below cg(n).
• g(n) is called an asymptotic upper boundof f(n).
O(g(n)) = f(n) : there exist positive constants c and n0
such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0.
O-notation
• f(n) = O(g(n)): A function f(n) is a member of the set O(g(n)).
• Note that f(n) = Θ(g(n)) implies f(n) = O(g(n)), since Θ-notation is a stronger notion than O-notation.
• Written set-theoretically, we have Θ(g(n)) ⊆ O(g(n)).
• Thus, our proof that any quadratic function an2 + bn + c, where a > 0, is in Θ(n2) also shows that any quadratic function is in O(n2).
12
Ω-notation
13
• For all values n to the right of n0, the value of f(n) is on or above cg(n).
• g(n) is called an asymptotic lower boundof f(n).
Ω(g(n)) = f(n) : there exist positive constants c and n0
such that 0 ≤ cg(n) ≤ f(n) for all n ≥ n0.
Theorem 3.1
For any two functions f(n) and g(n),
we have f(n) = Θ(g(n)) if and only if
f(n) = O(g(n)) and f(n) = Ω(g(n)).
14
Notation
• Notation in equations and inequalities
– n = O(n2)
– T(n) = 2T(n/2) + Θ(n)
– 2n2+Θ(n)=Θ(n2)
15
o-notation
• The asymptotic upper bound provided by O-notation may or may not be asymptotically tight.
• The bound 2n2 = O(n2) is asymptotically tight, but the bound 2n = O(n2) is not.
• We use o-notation to denote an upper bound that is not asymptotically tight.
• f(n) is called an asymptotically smaller than g(n).
16
o(g(n))= f(n) : for any positive constant c > 0, there exists
a constant n0 > 0 such that 0 ≤f(n) < cg(n) for all n ≥n0.
o-notation
• For example, 2n = o(n2), but 2n2 ≠ o(n2).
• The main difference
– f(n) = O(g(n)), the bound 0 ≤ f(n) ≤ cg(n) holds for some constant c > 0
– f(n) = o(g(n)), the bound 0 ≤ f(n) < cg(n) holds for all constants c > 0.
• Intuitively, in the o-notation, the function f(n) becomes insignificant relative to g(n) as n approaches infinity; that is,
17
0)(
)(lim
ngnf
n
ω-notation
• We use ω-notation to denote a lower bound that is not asymptotically tight.
• f(n) is called an asymptotically larger than g(n).
• One way to define it is by to f(n) ∈ ω(g(n))
if and only if g(n) ∈ o(f(n)).
18
ω(g(n)) = f(n) : for any positive constant c > 0, there exists
a constant n0 > 0 such that 0 ≤cg(n) <f(n) for all n ≥n0
ω-notation
• Example
– n2/2 = ω(n), but n2/2 ≠ ω(n2).
– The relation f(n) = ω(g(n)) implies that
if the limit exists.
– That is, f(n) becomes arbitrarily large relative to g(n) as n approaches infinity.
19
)(
)(lim ng
nf
n
Analogy
• f(n) = O(g(n)) ≈ a ≤ b,
• f(n) = Ω(g(n)) ≈ a ≥ b,
• f(n) = Θ(g(n)) ≈ a = b,
• f(n) = o(g(n)) ≈ a < b,
• f(n) = ω(g(n)) ≈ a > b.
20
Comparison of functions
• Comparison of functions
– Transitivity
– Reflexivity
– Symmetry
– Transpose symmetry
21
22
Transitivity & Reflexivity
• Transitivity
– f(n) = Θ(g(n)) and g(n) = Θ(h(n)) imply f(n) = Θ(h(n)),
– f(n) = O(g(n)) and g(n) = O(h(n)) imply f(n) = O(h(n)),
– f(n) = Ω(g(n)) and g(n) = Ω(h(n)) imply f(n) = Ω(h(n)),
– f(n) = o(g(n)) and g(n) = o(h(n)) imply f(n) = o(h(n)),
– f(n) = ω(g(n)) and g(n) = ω(h(n)) imply f(n) = ω(h(n)).
• Reflexivity
– f(n) = Θ(f(n))
– f(n) = O(f(n))
– f(n) = Ω(f(n))
23
Symmetry
• Symmetry
– f(n) = Θ(g(n)) if and only if g(n) = Θ(f(n)).
• Transpose symmetry
– f(n) = O(g(n)) if and only if g(n) = Ω(f(n)),
– f(n) = o(g(n)) if and only if g(n) = ω(f(n)).
Standard Notation & Common Functions
25
Standard Mathematical Functions
• Monotonicity
– m ≤ n → f(m) ≤ f(n) : A function f(n) is monotonically increasing
– m ≤ n → f(m) ≥ f(n) :A function f(n) is monotonically decreasing
– m < n → f(m) < f(n) : A function f(n) is strictly increasing
– m < n → f(m) > f(n) : A function f(n) is strictly decreasing
26
Floors and Ceilings (Definition)
• Definitions
– The floor of x : For any real number x, we denote the
greatest integer less than or equal to x by ⌊x⌋
– The ceiling of x : For any real number x, we denote the
least integer greater than or equal to x by ⌊x⌋
– For all real x,
1x x x 1x x
27
Floors and Ceilings (Properties)
• For any integer n,
⌈ n/2 ⌉ + ⌊ n/2 ⌋ = n (3.3)
• For any real number n ≥ 0 and integers a, b > 0,
– ⌈⌈n/a⌉ /b⌉ = ⌈n/ab⌉, (3.4)
– ⌊⌊n/a⌋ /b⌋ = ⌊n/ab⌋, (3.5)
– ⌈a/b⌉ ≤ (a+(b-1))/b, (3.6)
– ⌊a/b⌋ ≥ (a-(b_1))/b. (3.7)
• The floor function f(x) = ⌊x⌋ is monotonically increasing, as is the ceiling function f(x) = ⌈x⌉.
28
Modular Arithmetic
• For any integer a and positive integer n, the value a mod n is the remainder(or residue) of the quotient a/n: a mod n = a - ⌊a/n⌋ n.
• a is equivalent to b, modulo n.
– a ≡ b (mod n) ⇔ (a mod n) = (b mod n)
– : a is not equivalent to b modulo n
29
Polynomials
• Given a nonnegative integer d, a polynomial in n of degree d is a function p(n) of the form
• a0, a1, …., ad : coefficients of the polynomial and (ad≠ 0)
• A polynomial is asymptotically positive
if and only if ad > 0
• For an asymptotically positive polynomial p(n) of degree d, we have p(n) = Θ(n d )
d
i
iinanP
0
)(
30
Polynomially Bounded
• a ≥ 0 : the function na is monotonically increasing
• a ≤ 0: the function na is monotonically decreasing
• We say that a function f(n) is polynomially bounded if f(n) = O(nk) for some constant k.
31
Exponentials
• 00 = 1
• a0 = 1
• a1 = a
• a-1 = 1/a
• (am)n = amn
• (am)n = (an)m
• am an = am+n
• Real constant a and b, a > 1
(3.9) 0lim n
b
n an
32
Exponentials
• e = 2.71828…, ! denote the factorial function
• For all real x, we have the inequality ex≥1+x
• x=0, |x| ≤ 1, we have the approximation 1 + x ≤ ex ≤ 1 + x + x2
• When x → 0, the approximation of ex by 1 + x is quite good: ex = 1 + x + Θ(x2).
•
0
32
!...
!3!21
i
ix
ixxx
xe
xn
ne
nx
)1(lim
33
Logarithms
• lg n = log2 n (binary logarithm)
• ln n = loge n (natural logarithm)
• lgk n = (lg n)k (exponentiation)
• lg lg n = lg (lg n) (composition)
34
Logarithms
• For all real a > 0, b > 0, c > 0, and n,
•
•
(3.14) log
loglog
loglog
loglog)(log
log
ba
a
ana
baab
ba
c
cb
bn
b
ccc
ab
(3.15)
log
1log
log)1(log
loglog ac
ab
bb
bb ca
ba
aa
35
Logarithms
• x≤ 1 일 때, ln(1+x)
• x > -1:
• If f(n) = O(lgkn) : polylogarighmically bounds
...5432
)1ln(5432
xxxx
xx
(3.16) )1ln(1
xxx
x
0lg
lim)2(
lglim
lg
a
b
nna
b
n nnn
36
Factorials
• n ! (“n factorial”), integers n ≥ 0
• Stirling’s approximation
0 n if )!1(
0n if 1!
nnn
(3.17) 1
12!
nen
nnn
37
Factorials
• e is the base of the natural logarithm
• n! = Ο(nn)
n! = ω(2n)
lg(n!) = Θ(n lg n) (3.18)
•
•
(3.19) 2! neen
nnn
(3.20) 12
1
112
1
nn n
38
Functional Iteration
• f(i)(n) : the function f(n) iteratively applied i times to an initial value of n.
•
• For example, if f(n)=2n, then f(i)(n)=2in
0 if ))((
0if )(
)1(
)(
inff
innf
i
i
39
The Iterated Logarithm Function
• lg*n (“log star of n”) : the iterated logarithm
• lg(i)n is defined only if lg(i-1)n > 0
• The iterated logarithm function is defined aslg*n = min i ≥ 0 : lg(i) n ≤ 1 .
• The iterated logarithm is a very slowly growing function
lg*2=1lg*4=2lg*16=3lg*65536=4lg*(265536)=5
40
Fibonacci Numbers
• Definition
– F0 = 0F1 = 1Fi = Fi-1 + Fi-2 for i ≥ 2. (3.21)
• The golden ratio φ and to its conjugate
(3.23) 5
ˆ
61803- 2
51ˆ
(3.22) ...61803.1 2
51
ii
iF
....
