Analysys & Complexity of Algorithms

Big Oh Notation

2

Complexity

• Two kinds of complexity– Space complexity

• Effects of data type choices on size of data• Effects of amount of data (input and variables)

– Time Complexity• Effects of choices of program design

• Given 2 algorithms, which is "the best"

3

Big Oh (or Big O)

• A measure of complexity related to "n" (problem size=amount of data) e.g.:– # of records to process– #of files– # of numbers or times through a loop

• This is asymptotic analysis– associates n, the problem size, with t, the processing

time required to solve the problem

4

Big O Examples• x=x+1;

– O(1) -> constant degradation• Binary Search of a SORTED set of n elements

– O(log n)• for (i=1;i<n;i++)

– O(n) -> linear degradation• selection sort, compare two 2D arrays, find duplicates in an

UNSORTED list– O(n2) -> quadratic degradation

• Generate all premutations of n symbols– O(an) -> 'a' is some constant independtent of 'n'

• O(l) < O(log n) < O(n) < O(n log n) < O(n2) < O(an)

5

3 cases

• Best case– minimum path lengths

• Average case– constant path length

• Worst case– maximum path length–most useful!! Leads to better design– answers question: will it be good enough

tomorrow??

6

Frequency Counting

• Examine a piece of code and predict the number of instructions to be executed

• e.g. predict how many times (max) each statement will run.

Inst # Code Freq count

1 for (int i=0; i< n ; i++) n+12 { printf ("%d",i); n3 p = p + i; n

} 3n+1

7

Order of magnitude• In the previous example:

– best case = average case = worst case – Example is based on iteration limit: n

• To convert Frequency Count to order of magnitude:– pick the most significant term if polynomial– discard constant terms (like the +1)– disregard coefficients (like the 3)– yields worst case path through algorithm

• Big O (represented as O(n))• O(n) for the previous example

8

Common growth rates

9

Big Oh - Formal Definition

• f(n)=O(g(n)),

• Thus, g(n) is an upper bound on f(n)• Note:

f(n) = O(g(n)) – f(n) has a complexity of g(n)"

this is NOT the same as O(g(n)) = f(n)

• The '=' is not the usual mathematical "=" operator (it is not reflexive)

• We will see more about this in chapter 6

iff ᴲ {c, n0 | f(n) <= c g(n) for all n >= n0}