Time complexity

CS 213

Data Structures and Algorithms

Second Semester, 2011-2012

Sum of Squares

int sumOfSquares(int n){

int sum = 0;

for(int i=1; i<=n; i++)

sum = sum + i*i;

return sum;

}T(n) = 5n + 4

Selection Sort

Given the following numbers in an array:53 23 10 34 2 17

2 23 10 34 53 17 2 10 23 34 53 17 2 10 17 34 53 23 2 10 17 23 53 34 2 10 17 23 34 53

Still on Selection Sortvoid selectionSort(int A[], int n){

for(int i=0; i<n; i++){ int min = i; for(int j=i+1; j<n; j++){

if(A[j] < A[min]) min = j;}int temp = A[i];A[i] = A[min];A[min] = temp;

}}

What is this function’s T(n)? What we need is to compute for T(n) from the inner loop going outwards.

We need to use the summation notation to solve the T(n).

The Summation Notation

= x1 + x2 + x3 + x4

Still on the summation notation

2)1(

1

nnn

i

i∑4

i=1

i = 1+2+3+4 = 10

∑5

i=1

3 = 3+3+3+3+3 = 15 ∑n

i=1

c =nc =15

Exercise for(int i=1; i<=5; i++) cout<<endl; for(int i=1; i<=n; i++) cout<<endl; for(int i=3; i<=m; i++) cout<<endl; for(int i=1; i<=6; i++)

for(int j=1; j<=8; j++) cout<<endl; for(int i=1; i<=n; i++)

for(int j=1; j<=n; j++) cout<<endl; for(int i=1; i<=m; i++)

for(int j=1; j<=m; j++) cout<<endl; for(int i=1; i<=n; i++)

for(int j=i; j<=n; j++) cout<<endl;

Going back to the Selection Sortvoid selectionSort(int A[], int n){

for(int i=0; i<n; i++){ int min = i; for(int j=i+1; j<n; j++){

if(A[j] < A[min]) min = j;}int temp = A[i];A[i] = A[min];A[min] = temp;

}}

∑n

j=i+1

c∑n

i=1

=∑n

i=1

c(n-i+1-1)

∑n

i=1

c(n-i) = c ∑n

i=1

n-i

∑n

i=1

n-i = ∑n

i=1

n ∑n

i=1

i-

Time Complexity

A function that maps problem size into the time required to solve the problem.

Typically, we are interested in the inherent complexity of computing the solution to problems in a particular class.

Lower Bound

We might want to know how fast we can sort a list of n items, initially in an arbitrary order, regardless of the algorithm used.

What is sought here is the lower bound, L(n), on sorting, a property of the sorting problem and not of any particular algorithm.

This says that no algorithm can do the job in fewer than L(n) time units for arbitrary inputs.

Upper Bound

We might also like to know how long it would take to sort such list using a known algorithm with a worst-case input.

What is sought here is the upper bound, U(n), which says that for arbitrary inputs we can always sort in time at most U(n).

Goal of Time Complexity Analysis

While there are apparently two complexity functions for problems, L(n) and U(n), the ultimate goal is to make these two bounds coincide.

This is the optimal algorithm which has L(n) = U(n). For some of the problems, this goal has not been

realized yet!

Invitation

Consider this, CS 213, as you journey into finding optimal solutions to classes of problems!

Who knows, you might win a million dollars ($$$) from Claymath Foundation!

Upper Bound Complexity

There are two ways in analyzing this bound:Counting instructionsSolving recurrences

Both are used to find the worst case of an algorithm.

Big O-Notation (O(g(n)))

The O-notation is used to describe the worst-case running time of an algorithm.

O(n) means that the growth of the running time of the algorithm is a function of n.

O-notation computes for the upper bound.

O-Notation Defined

O(g(n)) = {f(n): c>0, n0>0 s.t. 0 f(n) cg(n) for all n n0}.

Example: Check if (n2/2) – 3n O(n2)(n2/2) – 3n cn2

½ - 3/n cChoosing c = ½, n0 = 6 proves the claim.

Another Example

3n2 - 100n + 6 O(n2) ?3n2 - 100n + 6 cn2 3 – 100/n + 6/n2 cAt this point, we have to choose a c>0 and an

n0

What values will prove our claim?

Lower Bound Complexity

This is the more difficult of the bounds.There is no algorithm to analyze.(g(n)) is used to describe the lower bound of

the running time of the algorithm or minimum possible running time of the algorithm.

-Notation

(g(n)) = {f(n): c>0, n0>0 s.t. 0 cg(n) f(n) for all n n0}.

Example: Check if (n2/2) – 3n (n2)cn2 (n2/2) – 3n c ½ - 3/nChoosing c = 1/14, n0 = 7 proves the claim.

Another Example

Check if 3n2 - 100n + 6 (n)cn 3n2 - 100n + 6 c 3n – 100 + 6/n

At this point we need to find a c and an n0 that will prove our claim.

What values of c and n0 will suffice the inequality??

-Notation

Used to denote that the lower and upper bounds of the running time of the algorithm is tight, i.e. the growth rate of the upper and lower bounds are the same.

-Notation Defined

(g(n)) = {f(n): c1>0, c2>0, n0>0 s.t. 0 c1g(n) f(n) c2g(n) for all n n0}.

f(n) (g(n)) if f(n) O(g(n)) and f(n) (g(n))

Complexity Classes

Description O-notation

constant O(1)

logarithmic O(log n)

linear O(n)

n log n O(n log n)

quadratic O(n2)

cubic O(n3)

polynomial O(nk), k1

exponential O(an), a>1

Growth rate of complexity classes

class n=2 n=16 n=256 n=1024

1 1 1 1 1

log n 1 4 8 10

n 2 16 256 1024

n log n 2 64 2048 10240

n^2 4 256 65536 1048576

n^3 8 4096 16777216 1.07E+09

2^n 4 65536 1.16E+77 1.8E+308

Graph of the Growth Rates

log n

n

n log n

n 2̂

n 3̂

2 n̂

0

5

10

15

20

25

30

35

40

45

50

log n 0 1 1.58 2 2.32 2.58 2.81 3 3.17 3.32 3.46

n 1 2 3 4 5 6 7 8 9 10 11

n log n 0 2 4.75 8 11.61 15.51 19.65 24 28.53 33.22 38.05

n 2̂ 1 4 9 16 25 36 49 64 81 100 121

n 3̂ 1 8 27 64 125 216 343 512 729 1000 1331

2 n̂ 2 4 8 16 32 64 128 256 512 1024 2048

1 2 3 4 5 6 7 8 9 10 11

Bigger N

log nnn log nn 2̂

n 3̂

2 n̂

0

200

400

600

800

1000

1200

1400

1600

1800

2000

log n 0 1 1.58 2 2.32 2.58 2.81 3 3.17 3.32 3.46

n 1 2 3 4 5 6 7 8 9 10 11

n log n 0 2 4.75 8 11.61 15.51 19.65 24 28.53 33.22 38.05

n 2̂ 1 4 9 16 25 36 49 64 81 100 121

n 3̂ 1 8 27 64 125 216 343 512 729 1000 1331

2 n̂ 2 4 8 16 32 64 128 256 512 1024 2048

1 2 3 4 5 6 7 8 9 10 11

Still on the Graph

100*log n

50*n

20*n log n

10*n 2̂n 3̂

2 n̂

0

200

400

600

800

1000

1200

1400

1600

1800

2000

100*log n 0 100 158.5 200 232.19 258.5 280.74 300 316.99 332.19 345.94

50*n 50 100 150 200 250 300 350 400 450 500 550

20*n log n 0 40 95.1 160 232.19 310.2 393.03 480 570.59 664.39 761.07

10*n 2̂ 10 40 90 160 250 360 490 640 810 1000 1210

n 3̂ 1 8 27 64 125 216 343 512 729 1000 1331

2 n̂ 2 4 8 16 32 64 128 256 512 1024 2048

1 2 3 4 5 6 7 8 9 10 11