L1 Analysis

8/2/2019 L1 Analysis

1/28

Analysis of Algorithms

AlgorithmInput Output

An algorithm is a step-by-step procedure forsolving a problem in a finite amount of time.


2/28

Running Time (1.1)Most algorithms transform

input objects into outputobjects.

The running time of analgorithm typically grows

100

120

e

best case

average case

worst case

Analysis of Algorithms 2

with the input size.Average case time is oftendifficult to determine.

We focus on the worst case

running time. Easier to analyze

Crucial to applications such asgames, finance and robotics

0

20

40

60

RunningT

i

1000 2000 3000 4000

Input Size


3/28

Experimental Studies ( 1.6)

Write a programimplementing thealgorithm

Run the program with 6000

7000

8000

9000

)


npu s o vary ng s ze ancomposition

Use a method to get anaccurate measure of the

actual running timePlot the results

0

1000

2000

3000

4000

5000

0 50 100

Input Size

Time(m


4/28

Limitations of Experiments

It is necessary to implement thealgorithm, which may be difficult

Results may not be indicative of the


runn ng me on o er npu s no nc u ein the experiment.

In order to compare two algorithms, the

same hardware and softwareenvironments must be used


5/28

Theoretical Analysis

Uses a high-level description of thealgorithm instead of an implementation

Characterizes running time as a


function of the input size, n.Takes into account all possible inputs

Allows us to evaluate the speed of analgorithm independent of thehardware/software environment


6/28

Pseudocode (1.1)

High-level description

of an algorithmMore structured thanEnglish prose

Algorithm arrayMax(A,n)

Input arrayA ofn integers

Example: find max

element of an array


program

Preferred notation fordescribing algorithms

Hides program designissues

u pu max mum e emen o

currentMax A[0]

for i 1 to n 1 do

ifA[i] > currentMax then

currentMax A[i]

return currentMax


7/28

Pseudocode Details

Control flow if then [else ]

while do

repeat until

Method callmethod(arg [,arg])

Return valuereturn expression


for do Indentation replaces braces

Method declarationAlgorithmmethod(arg [,arg])

Input

Output

Expressions= or Assignment

==Equality testing

n2 Superscripts and other

mathematicalformatting allowed


8/28

The Random Access Machine

(RAM) Model

ACPU

A potentially unbounded


bank ofmemory cells,each of which can hold anarbitrary number or

character

012

Memory cells are numbered and accessingany cell in memory takes unit time.


9/28

Primitive Operations

Basic computations

performed by an algorithmIdentifiable in pseudocode

Lar el inde endent from the

Examples: Evaluating an

expression

Assigning a value


programming languageExact definition not important(we will see why later)

Assumed to take a constantamount of time in the RAMmodel

o a var a e

Indexing into anarray

Calling a method

Returning from amethod


10/28

Counting PrimitiveOperations (1.1)

By inspecting the pseudocode, we can determine the

maximum number of primitive operations executed byan algorithm, as a function of the input size

Algorithm arrayMax(A,n) # o erations


currentMax A[0] 2for i 1 to n 1 do 1 + n

ifA[i] > currentMax then 2(n 1)

currentMax A[i] 2(n 1)

{ increment counter i } 2(n 1)return currentMax 1

Total 7n 2


11/28

Estimating Running Time

AlgorithmarrayMax executes 7n 2 primitive

operations in the worst case. Define:

a = Time taken by the fastest primitive operation

=


Let T(n) be worst-case time ofarrayMax. Thena (7n 2) T(n) b(7n 2)

Hence, the running time T(n) is bounded by two

linear functions


12/28

Growth Rate of Running Time

Changing the hardware/ softwareenvironment

Affects T(n) by a constant factor, but


Does not alter the growth rate ofT(n)

The linear growth rate of the runningtime T(n) is an intrinsic property ofalgorithmarrayMax


13/28

Growth Rates

Growth rates offunctions:

Linear n

Quadratic n21E+18

1E+20

1E+22

1E+24

1E+261E+28

1E+30

Cubic

Quadratic

Linear


Cubic n3

In a log-log chart,the slope of the line

corresponds to thegrowth rate of thefunction

1E+0

1E+21E+4

1E+6

1E+8

1E+10

1E+12

1E+141E+16

1E+0 1E+2 1E+4 1E+6 1E+8 1E+10

n

T

(n

)


14/28

Constant Factors

The growth rate isnot affected by

constant factors or

lower-order terms 1E+161E+18

1E+20

1E+22

1E+24

1E+26

)

Quadratic

Quadratic

Linear

Linear


Examples 102n ++++ 105 is a linear

function

105n2 ++++ 108n is a

quadratic function 1E+01E+2

1E+4

1E+6

1E+8

1E+10

1E+12

+

1E+0 1E+2 1E+4 1E+6 1E+8 1E+10

n

T

(

n


15/28

Big-Oh Notation (1.2)Given functionsf(n) and

g(n), we say thatf(n) isO(g(n)) if there are

positive constantsc andn such that 100

1,000

10,000

3n

2n+10

n


f(n) cg(n) forn n0

Example: 2n + 10 is O(n)

2n + 10 cn

(c 2)n 10 n 10/(c 2)

Pickc = 3 andn0 = 10

1

10

1 10 100 1,000

n


16/28

Big-Oh Example

Example: the functionn2 is not O(n)

n2 cn

n c

10,000

100,000

1,000,000n^2

100n10n

n


The above inequalitycannot be satisfiedsincec must be a

constant

1

10

100

1,000

1 10 100 1,000n


17/28

More Big-Oh Examples7n-2

7n-2 is O(n)

need c > 0 and n0 1 such that 7n-2 cn for n n0this is true for c = 7 and n0 = 1

3 2


3n3 + 20n2 + 5 is O(n3)

need c > 0 and n0 1 such that 3n3 + 20n2 + 5 cn3 for n n0

this is true for c = 4 and n0 = 21

3 log n + log log n3 log n + log log n is O(log n)

need c > 0 and n0 1 such that 3 log n + log log n clog n for n n0this is true for c = 4 and n0 = 2


18/28

Big-Oh and Growth Rate

The big-Oh notation gives an upper bound on the

growth rate of a functionThe statement f(n) is O(g(n)) means that the growthrate off(n) is no more than the growth rate ofg(n)


We can use the big-Oh notation to rank functionsaccording to their growth rate

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more Yes No

f(n) grows more No Yes

Same growth Yes Yes


19/28

Big-Oh Rules

Iff(n) is a polynomial of degree d, thenf(n) isO(nd), i.e.,

-


.

2. Drop constant factors

Use the smallest possible class of functions

Say 2n is O(n) instead of 2n is O(n2)

Use the simplest expression of the class

Say 3n + 5 is O(n) instead of 3n + 5 is O(3n)


20/28

Asymptotic Algorithm AnalysisThe asymptotic analysis of an algorithm determinesthe running time in big-Oh notation

To perform the asymptotic analysis We find the worst-case number of primitive operations

executed as a function of the input size


e express s unc on w g- no a on

Example: We determine that algorithmarrayMax executes at most

7n 2 primitive operations

We say that algorithmarrayMaxruns in O(n) time

Since constant factors and lower-order terms areeventually dropped anyhow, we can disregard themwhen counting primitive operations


21/28

Computing Prefix AveragesWe further illustrate

asymptotic analysis withtwo algorithms for prefixaverages

The i-th prefix average of25

30

35

XA


an arrayXis average of thefirst (i + 1) elements ofX:

A[i] = (X[0] +X[1] + +X[i])/(i+1)

Computing the arrayA ofprefix averages of anotherarrayXhas applications tofinancial analysis

0

5

10

15

1 2 3 4 5 6 7


22/28

Prefix Averages (Quadratic)The following algorithm computes prefix averages inquadratic time by applying the definition

Algorithm prefixAverages1(X, n)

Input arrayXofn integers


A new array ofn integers nfor i 0 to n 1 do n

s X[0] n

for j 1 to i do 1 + 2 + + (n 1)

s s + X[j] 1 + 2 + + (n 1)

A[i] s / (i + 1) n

return A 1


23/28

Arithmetic Progression

The running time ofprefixAverages1 isO(1 + 2 + +n)

The sum of the first n


integers isn(n + 1) //// 2Thus, algorithmprefixAverages1 runs inO(n2) time


24/28

Prefix Averages (Linear)The following algorithm computes prefix averages inlinear time by keeping a running sum

Algorithm prefixAverages2(X, n)

Input arrayXofn integers


A new array ofn integers ns 0 1

for i 0 to n 1 do n

s s + X[i] n

A[i] s / (i + 1) n

return A 1

AlgorithmprefixAverages2 runs in O(n) time


25/28

properties of logarithms:

logb(xy) = logbx + logby

logb (x/y) = logbx - logby

Summations (Sec. 1.3.1)

Logarithms and Exponents (Sec. 1.3.2)

Math you need to Review


logbxa = alogbxlogba = logxa/logxb

properties of exponentials:a(b+c) = aba c

abc = (ab)c

ab/ac = a(b-c)

b = a logab

bc = a c*logabProof techniques (Sec. 1.3.3)

Basic probability (Sec. 1.3.4)


26/28

Relatives of Big-Ohbig-Omega

f(n) is (g(n)) if there is a constant c > 0

and an integer constant n0 1 such that

f(n) cg(n) for n n0big-Theta


s g ere are co s a s c a c a a

integer constant n0 1 such that cg(n) f(n) cg(n) for n n0little-oh

f(n) is o(g(n)) if, for any constant c > 0, there is an integerconstant n0 0 such that f(n) cg(n) for n n0

little-omega f(n) is (g(n)) if, for any constant c > 0, there is an integer

constant n0 0 such that f(n) cg(n) for n n0


27/28

Intuition for AsymptoticNotation

Big-Oh f(n) is O(g(n)) if f(n) is asymptotically less than or equal to g(n)

big-Omega


big-Theta f(n) is (g(n)) if f(n) is asymptotically equal to g(n)

little-oh

f(n) is o(g(n)) if f(n) is asymptotically strictly less than g(n)

little-omega f(n) is (g(n)) if is asymptotically strictly greater than g(n)


28/28

Example Uses of the

Relatives of Big-Oh

5n2 is n)

f(n) is (g(n)) if there is a constant c > 0 and an integer constant n0 1such that f(n) cg(n) for n n0

let c = 5 and n0 = 1

5n2 is (n2)


f(n) is (g(n)) if, for any constant c > 0, there is an integer constant n0 0 such that f(n) cg(n) for n n0

need 5n02 cn0 given c, the n0 that satifies this is n0 c/5 0

5n2 is (n)

f(n) is (g(n)) if there is a constant c > 0 and an integer constant n0 1such that f(n) cg(n) for n n0

let c = 1 and n0 = 1

Documents

L1 Analysis