O, , and Notations

Lecture 39

Section 9.2

Wed, Mar 30, 2005

Notation

Let g : R R be a function. A function f : R R is “of order at least g,”

written “f(x) is(g(x)),” if there exist real numbers M, x0 such that

f(x) Mg(x)for all x x0.

“Big-Oh” Notation

A function f : R R is “of order at most g,” written “f(x) isO(g(x)),” if there exist real numbers M, x0 such that

f(x) Mg(x)for all x x0.

Notation

A function f : R R is “of order g,” written “f(x) is(g(x)),” if there exist real numbers M1, M2, x0 R such that

M1g(x) f(x) M2g(x)

for all x x0.

Growth Rates

If f(x) is O(g(x)), then the growth rate of f is no greater than the growth rate of g, and maybe less.

If f(x) is (g(x)), then the growth rate of f is no less than the growth rate of g, and maybe greater.

Growth Rates

Theorem: If f(x) is O(g(x)) and g(x) is O(f(x)), then f(x) is (g(x)).

Proof:If f(x) is O(g(x)), then there exist M1, x1 R

such that f(x) M1g(x) for all x x1.

If g(x) is O(f(x)), then there exist M2, x2 R such that g(x) M2f(x) for all x x2.

Growth Rates

Let x3 = max(x1, x2).Then

(1/M2)g(x) f(x) M1g(x)

for all x x3.Therefore, f(x) is (g(x)).

We call (f(x)) the growth rate of f.

Growth Rates

Theorem: f(x) is (g(x)) if and only if f(x) is (g(x)) and f(x) is O(g(x)).

Proof:Show that if f(x) is (g(x)), then g(x) is

O(f(x)).Then apply the previous theorem.

Common Growth Rates

Growth Rate Example

(1) Access an array element

(log2 x) Binary search

(x) Sequential search

(x log2 x) Quick sort

(x2) Bubble sort

(2x) Factor an integer

(x!) Traveling salesman problem

Transitivity of O(f)

Theorem: Let f, g, and h be functions from R to R. If f(x) is O(g(x)) and g(x) is O(h(x)), then f(x) is O(h(x)).

Proof:If f(x) is O(g(x)), then f(x) M1g(x) for all

x x1, for some M1 and x1.

If g(x) is O(h(x)), then g(x) M2h(x) for all x x2, for some M2 and x2.

Transitivity of O(f)

Let x3 = max(x1, x2).

Then f(x) M1M2h(x), for all x x3.Therefore, f(x) is O(h(x)).

Power Functions

Theorem: If 0 a b, then xa is O(xb), but xb is not O(xa).

Proof:Since a b, for all x > 1, xa – b < 1.Therefore, xa < xb for all x > 1.On the other hand, suppose xb Mxa, for

some M and for all x > x0, for some x0.

Power Functions

Then, xb – a M, for all x > x0.But b – a > 0, so xb – a increases without

bound.This is a contradiction, so xb is not O(xa).

Application

Any polynomial of degree d is(xm), if m < d.(xd).O(xn), if n > d.

For example, 16x3 – 10x2 + 3x – 12 is (x2), (x3), and O(x4).

Logarithmic Functions

Lemma: Proof:

It true by the definition of the loga function.Also, it follows from the fact that the

functions f(x) = ax and g(x) = loga x are inverses of each other.

.log and log xaxa xa

xa

Logarithmic Functions

Theorem: For all a, b 1, loga x is (logb x). Proof:

.logloglog

loglog log

log

axx

ax

ax

bab

xbb

x

a

a

Application

Since all logarithmic functions are of the same order, it doesn’t matter which base we use.

Logarithmic Functions vs. Power Functions

Theorem: For all a > 1, b > 0, loga x is O(xb), but xb is not O(loga x).

Proof (using Calc II):

.0ln

1lim

ln1

limlog

lim1

bxbxb

a

x xabbxax

x

x

Application

Every polynomial function has a faster growth rate than all logarithmic functions.

If a function mixes polynomial terms and logarithmic terms, then the polynomial tems “dominate.”3x2 + 20 log10 (5x + 2) is (x2).

Exponential Functions

Theorem: If 1 < a < b, then ax is O(bx), but bx is not O(ax).

Proof:Clearly, ax < bx for all x 1.Therefore, ax is O(bx).We must show that bx is not O(ax).Suppose that it is.Then bx Max for all x > x0 for some M, x0.

Exponential Functions

Then

But this contradicts that assumption that the inequality holds for all x > x0.

.lnln

ln

lnlnln

lnlnln

ab

Mx

Mabx

axMbx

Mab xx

Application

Any exponential function with base a is(bx), if b < a.(ax).O(bx), if b > a.

For example, 58x is (2x), (8x), and O(10x).

Power Functions vs. Exponential Functions

Theorem: For all a > 0, b > 1, xa is O(bx), but bx is not O(xa).

Application

If a function is a mix of polynomial terms, logarithmic terms, and exponential terms, the exponential term with the highest base “dominates.”5x3 + 2 log2 x + 1003x + 285x is (5x).

Benefits of O, , and

Clearly, the -notation is useful because it singles out the term that best represents the growth rate of the function (for large x).

The O-notation is useful when we can’t pin down the exact growth rate, but we can put an upper bound on it.

Similarly, the -notation expresses a lower bound.

Multiplicativity of O(f)

Theorem: Let f, g, h, and k be functions from R to R. If f(x) is O(h(x)) and g(x) is O(k(x)), then f(x)g(x) is O(h(x)k(x)).

Proof:Suppose f(x) M1h(x) for all x x1 and

g(x) M2k(x) for all x x2, for some M1, M2, x1, x2.

Then f(x)g(x) M1 M2h(x)k(x) for all x max(x1, x2).

Application

We know that log x is O(x) and that x is O(x), so x log x is O(x2).

More generally,xd is O(xd log x).xd log x is O(xd + 1).