Comparing Algorithms

8/8/2019 Comparing Algorithms

1/37

Comparing algorithms


2/37

Objectives

To cover:

What is an algorithm

Why we need to study algorithms Algorithm correctness

Computational complexity

Complexity of a problem

Order of growth of an algorithm

Big O notation and order ofcomplexity


3/37

What is an algorithm

Sequence of unambiguous instructions for

solving a problem


4/37

Algorithm essentials

Consider a mathematical problems: sum of

the first natural numbers.


5/37

One way: 1+2=3 then 3+4=7 then 7+5=12 and

so on and the algorithm for thatAlgorithm SumNaturalNumber(n)

//Implements sum of first n natural numbers

//Input: An integer n >=1

//Output: Sum

Sum 0

For i 1 To n

Do Sum Sum + i

Return Sum


6/37

Result for n = 100

Sum = 5050


7/37

Another way

Formula Sum= n(n+1)/2

Algorithm SumNaturalNumbersByFormula(n)

//Implements sum of first n natural numbers

//Input: An integer n >= 1//Output: Sum

Sum n*(n+1)/2

Return Sum

Result for n = 100, Sum = 5050


8/37

Observations

Each step in any algorithm must beunambiguous

The range of inputs for which an algorithmworks has to be specified

Several algorithm for solving the sameproblem may exist

Algorithm for the same problem can bebasedon very different ideas andcan solve theproblem with dramatically different speeds.


9/37

Performance

n Time (s)

SumNaturalNumbers(n) SumNaturalNumbersByFormula(n)

2000000000 9.05 Negligible

3000000000 13.59 Negligible

4000000000 17.98 Negligible

Both algorithms arecoded in a programming language andexecuted on an Intel

Celeron processor operating at 1.50 Ghz inside a laptop.

Estimation: 1/3 (9.05/2000000000 +13.59/3000000000+ 17.98/4000000000)

4.52*10^-9 seconds


10/37


11/37

Why do we need to study algorithms

Efficient use of resources like:

Time

Memory

Processor time


12/37

Algorithm correctness

We need to ensure that an algorithm

generates thecorrect output for all legitimate

inputs, not just for some. e.g:

SumNaturalNumbers(n)

Sum Sum + i

For n = 1 this step is executedjust once

Sum 0 + 1 = 1 therefore

SumNaturalNumbers(1) = 1


13/37

Next

SumNaturalNumbers(n+1) =

SumNaturalNumbers(n) + (n+1) si for n = 1 SumNaturalNumbers(2) =

SumNaturalNumbers(1) + (1+1) therefore

SumNaturalNumbers(2) = 1 + 2 =3


14/37


15/37

Computational complexity

How economical an algorithm is with time and

space

Depends on timecomplexity and spacecomplexity :

Timecomplexity: how long does the algorithm

takes

Spacecomplexity: how much memory does

the algorithm need


16/37

Spaceeffciency

PDAs andMobile phones have very limited

amount of memory so its very important to

develop programs that have a small memory

footprint


17/37

Timeefficiency

There are still problems for which exact

solutions take longer than human lifespan for

a very small range or size of input.


18/37

Trade-offbetween time and space

Suppose we need to compute

the factors ofevery integer

from 1 to 1000 Algorithm FindFactors(n)

//Finds every factor of a given natural

number, n

//Input: An integer >= 2

//Output: Factors

While n > 1Do

Factor LeastFactor(n)

Output Factor

n n Div Factor //Integer division

It uses this algorithm

Algorithm LeastFactor(n)

//Finds smallest of a given natural number, n

//Input: An integer n >= 2

//Output: smallest factor

i 1

Repeat

i i + 1

Until (n Mod i) = 0 //Integer remainder

division

Return i


19/37

It uses this algorithm


20/37


21/37

Input size

For many algorithm, the larger the input the

longer it runs.

Eg: it takes longer to reverse theelements of alist of 1000 numbers than a list of 10 numbers

so the timeefficiency or timecomplexity of an

algorithm is a function of some parameter n

indicating the algorithms input size.


22/37

Complexity of a problem

Worst-casecomplexity: when wechoose an inputthat requires the longest time or greatestworkload

Best-casecomplexity: when wechoose an inputthat requires the shortest time or smallestworkload

Average-casecomplexity: averagecomplexitycalculatedby averaging the times for everypossible input. Eg:


23/37

Units for measuring time

Theestimate wouldbedependent on:

the speed of thecomputer,

thequality of the program implementing thealgorithm

thecompiler used in generating the machinecode

Difficult to compareefficiency of algorithmsbecause of factors that can vary from acomputer system to computer system.


24/37

Basic operation

Fortunately its sufficient to identify theoperation contributing the most to the totalrunning time and to compute the number of

times this operation is executed.

Conculsion: the method for the analysis of an

algorithms timeefficiency is based oncounting the number of times the algorithmsbasic operation is executed on inputs of size n.


25/37


26/37

Order of growth

T(n)=bop C(n)

bop execution time of an algorithm basic

operationC(n) number of times this operation needs to be

executed for input n

T(n) estimated running time of a programimplementing this algorithm


27/37

What if input size is doubled

Suppose

For big values of n, the formula becomes

For n = 1000, n = 1 000 000

If we ignore the n term, we introduce an error of1000 in 1 000 000 which is 0.1%


28/37

So if the input sizedoubles


29/37


30/37

Assessing the order of growth

Algorithm RearrangeDiscs(LineOfDiscs, n)

//Rearranges a line of an equal number ofblack and whitediscs

//Inputs: Line ofblack and whitedisc n, the total no ofdiscs in line n 2

//Output: Rearranged line ofdiscs n n-1

For NumberOfPairs n DownTo 1Do

Forj 1 To NumberOfPairs

Do

If LineOfDiscs[j] = black And


31/37

Asymptoticbehaviour


32/37

n n


33/37


34/37

Big O notation


35/37


36/37


37/37

Order ofcomplexity

Exponential time

Polynomial time

Linear time

Documents

Comparing Algorithms