1

Analysis of Algorithms

CS 105Introduction to Data

Structures and Algorithms

2

What is a good algorithm? It must be correct It must be efficient

Implementations of an algorithm must run as fast as possible

How is this measured? Running time

3

Running time of a program Amount of time required for a

program to execute for a given input

If measured experimentally, Dependent on hardware, operating

system and software Answer will be in milliseconds,

seconds, minutes, hours, …

4

Running time of an algorithm Answer will not be in seconds or minutes Instead, we count the number of

operations carried out Result will be a formula in terms of some

input size, n Takes into account all possible inputs

Example statement on running time: Running time of algorithm A is T(n) = 2n + 3

5

Describing algorithms First, we need to agree on how to

describe or specify an algorithm Note: algorithms are intended for

humans (programs are intended for computers)

Descriptions should be high-level explanations that combine natural language and familiar programming structures: Pseudo-code

6

Pseudo-code example

Algorithm arrayMax(A,n):Input: An array A storing n integers.Output: The maximum element in A. currentMax A[0] for i 1 to n - 1 do if currentMax < A[i] then currentMax A[i] return currentMax

7

Pseudo-Code conventions

General Algorithm Structure Statements Expressions Control Structures

8

Algorithm Structure

Algorithm heading

Algorithm name(param1, param2,...):Input : input elementsOutput : output elements

statements…

9

Statements Assignment: use instead of = Method or function call:

object.method(arguments) method(arguments)

Return statement: return expression

Control structures

10

Control Structures decision

structures while loops

repeat loops

for loop

if ... then ... [else ...]if ... then ... [else ...]

while ... dowhile ... do

repeat ... until ...repeat ... until ...

for ... dofor ... do

11

Expressions Standard math symbols

+ - * / ( )

Relational operators= > < <= >= !=

Boolean operatorsand or not

Assignment operator Array indexing A[i]

12

General rules on pseudo-code Should communicate high-level

ideas and not implementation details Syntax is not as tight (e.g.,

indentation and line-breaks take the place of ; and {} in Java)

Clear and informative

13

Back to running time Once an algorithm has been

described in pseudo-code, we can now count the primitive operations carried out by the algorithm

Primitive operations: assignment, calls, arithmetic operations, comparisons, array accesses, return statements, …

14

Back to arrayMax example Suppose the input array is

A = {42, 6, 88, 53}, (n = 4) How many primitive operations are

carried out? Some notes

The for statement implies assignments, comparisons, subtractions, and increments

the statement currentMax A[i] will sometimes not be carried out

15

What to count

currentMax A[0]: assignment, array access

for i 1 to n - 1 do:assignment,comparison,subtraction,increment (2 ops)

if currentMax < A[i] then: comparison, access

currentMax A[i]: assignment, access

return currentMax: return

16

Counting operations

currentMax A[0] 2 for i 1 to n - 1 do 15 if currentMax < A[i] then 6 currentMax A[i] 2 return currentMax 1

OPERATIONS CARRIED OUT 26

17

Handling all possible inputs In the example just carried out,

running time = 26 for the given input A more useful answer is to provide a

formula that applies in general Formula is in terms of n, the input or array

size Since there may be statements that do

not always execute, depending on input values, there are two approaches Assume worst-case Provide a range

18

Measuring running timecurrentMax A[0] 2 for i 1 to n - 1 do 1+2n+2(n-

1) if currentMax < A[i] then 2(n-1) currentMax A[i] 0 .. 2(n-1) return currentMax 1

RUNNING TIME (worst-case) 8n - 2

increment

comparison& subtraction

19

Nested loops

// prints all pairs from the set {1,2,…n}

for i 1 to n do for j 1 to n do print i, j

How many times does the print statement execute?

20

Nested loops

// prints all pairs from the set {1,2,…n}

for i 1 to n do for j 1 to n do print i, j n2 times

How about the operations implied in the for statement headers?

21

for statement revisited

for i 1 to n do…

1 assignment i 1n+1 comparisons i <= n

for each value of i in {1, 2, …, n+1}n increments = n assignments + n adds=> 3n+2 operations

22

Nested loops running time

for i 1 to n do 3n+2 for j 1 to n do n (3n+2) print i, j n2

TOTAL: 4n2 + 5n + 2

23

Nested loops example 2// prints all pairs from the set {1,2,…n}// excluding redundancies (i.e., 1,1 is not a // real pair, 1,2 and 2,1 are the same pair)

for i 1 to n-1 do for j i+1 to n do print i, j

How many times does the print statement execute?

24

Nested loops example 2for i 1 to n-1 do for j i+1 to n do print i, j

when i = 1, n-1 printswhen i = 2, n-2 prints…when i = n-2, 2 printswhen i = n-1, 1 print

25

Nested loops example 2

Number of executions of the print statement:

1 + 2 + 3 + … + n-1

= n(n-1)/2 = ½n2 – ½n

26

Nested loops example 2

for i 1 to n-1 do for j i+1 to n do print i, j

Take-home exercise: count the operations implied by the for statement headers

27

Running time function considerations In determining the running time of an

algorithm, an exact formula is possible only when the operations to be counted are explicitly specified

Different answers will result if we choose not to count particular operations

Problem: variations in the answers appear arbitrary

28

Classifying running time functions

We want to classify running times under particular function categories

Goal: some assessment of the running time of the algorithm that eliminates the arbitrariness of counting operations

Example: arrayMax runs in time proportional to n

29

Some function categories The constant function: f(n) = c The linear function: f(n) = n The quadratic function: f(n) = n2

The cubic function: f(n) = n3

The exponential function:f(n) = bn

The logarithm function: f(n) = log n The n log n function: f(n) = n

log n

30

Big-Oh notation Consider the following (running time) functions

f1(n) = 2n + 3

f2(n) = 4n2 + 5n + 2

f3(n) = 5n We place f1 and f3 under the same category

(both are proportional to n) and f2 under a separate category (n2 )

We say: 2n+3 is O(n), and 5n is O(n), while 4n2 + 5n + 2 is

O(n2) 2n+3 and 5n are linear functions,

while 4n2 + 5n + 2 is a quadratic function

31

Significance of Big-Oh Example for i 0 to n-1 do A[i] 0 Running time is

2n, if we count assignments & array accesses4n+2, if we include implicit loop

assignments/adds5n+3, if we include implicit loop comparisons

Regardless, running time is O(n)

32

Prefix Averages Problem Given an array X storing n numbers,

compute prefix averages or running averages of the sequence of numbers

A[i] = average of X[0],X[1],…,X[i] Example

Input: X = {1,3,8,2} Output: A = {1,2,4,3.5}

Algorithm?

33

Algorithm 1Algorithm prefixAverages1(A,n):Input: An array X of n numbers.Output: An array A containing prefix

averagesfor i 0 to n - 1 do sum 0 for j 0 to i do sum sum + X[j] A[i] sum/(i+1)return array A

34

Running time of Algorithm 1 Exercise: count primitive

operations carried out by Algorithm 1

Regardless of what you decide to count, the answer will be of the form:a n2 + b n + c

Observe that the answer is O(n2)

35

Algorithm 2Algorithm prefixAverages2(A,n):Input: An array X of n numbers.Output: An array A containing prefix

averagessum 0for i 0 to n - 1 do sum sum + X[j] A[i] sum/(i+1)return array A

36

Running time of Algorithm 2 Running time of Algorithm 2 will be

of the form: a n + b Observe that the answer is O(n) Assessment: Algorithm 2 is more

efficient than Algorithm 1, particularly for large input (observe the growth rate of n and n2)

37

Summary Algorithms are expressed in pseudo-code The running time of an algorithm is the

number of operations it carries out in terms of input size Answer depends on which operation(s) we

decide to count Big-Oh notation allows us to categorize

running times according to function categories

Next: formal definition of Big-Oh notation