Upload
kritika-khanna
View
215
Download
0
Embed Size (px)
Citation preview
7/30/2019 C analysis
1/15
10-04-03 P.P.Chakrabarti, IIT Kharagpur 1
Analysis and Complexity:2
P. P. Chakrabarti
7/30/2019 C analysis
2/15
10-04-03 P.P.Chakrabarti, IIT Kharagpur 2
Analysis of Algorithms
? Quantifying the resources required.? Measures of resource utilization (efficiency):
? Execution time ? time complexity
? Memory space ? space complexity
? Observation :
? The larger the input data the more the resource
requirement: Complexities are functions of the
amount of input data (input size).
Refer to first few chapters of the book Introduction
To Algorithms by Cormen, Leiserson, Rivest and Stein
7/30/2019 C analysis
3/15
10-04-03 P.P.Chakrabarti, IIT Kharagpur 3
Yardstick?
?The same algorithm will run at different speedsand will require different amounts of space whenrun on different computers, differentprogramming languages, different compilers.
?Algorithms usually consume resources in somefashion that depends on the size of the problemthey solve.
?Need a machine independent complexity
measure that is a function of input size, n. (Alsowe are interested at asymptotic behaviour, thatis, when n becomes large.
7/30/2019 C analysis
4/15
10-04-03 P.P.Chakrabarti, IIT Kharagpur 4
Analyzing SelSort
? Computer 1:f1(n) = 0.0007772 n
2 + 0.00305 n +0.001
? Computer 2:
f2(n) = 0.0001724 n2 + 0.00040 n +
0.100Note: Both are quadratic functions
of n
? The shape of the curve thatexpresses the running time as a
function of the problem sizestays the same.
We say that Selection Sort is ofcomplexity Order n2 or O(n2)
int max_loc(int x[], int k, int size)
{ int j, pos;pos = k;
for (j=k+1; j x[pos])
pos = j;
return pos;}
int selsort (int x[], int size) {
int k, m, temp;
for (k=0; k
7/30/2019 C analysis
5/15
10-04-03 P.P.Chakrabarti, IIT Kharagpur 5
Complexity classes
? The running time for different algorithmsfall into different complexity classes.? Each complexity class is characterized by a
different family of curves.
?All curves in a given complexity class sharethe same basic shape. (In the Asymptotic
sense)
? The O-notationis used for talking aboutthe complexity classes of algorithms.
7/30/2019 C analysis
6/15
10-04-03 P.P.Chakrabarti, IIT Kharagpur 6
Order of Complexity, O- notation
? For the quadratic functionf(n) = an2 + bn + c
we will say that f(n) is O(n2).
? We focus on the dominant term, and ignore the lesserterms; then throw away the coefficient. [AsymptoticAnalysis]
? Since constants are finally not important we mayassume that each machine instruction takes one unit of
time when we analyze the complexity of an algorithm.? We may sometimes abstract this to unit time operators
like add, sub, multiply, etc and do an operator count.
? We can perform worst case or average case analysis.
7/30/2019 C analysis
7/15
10-04-03 P.P.Chakrabarti, IIT Kharagpur 7
How execution time is affected by various
complexity measures:
Assume speed S is 107 instructions per second.
size 10 20 30 50 100 1000 10000
n .001
ms
.002
ms
.003
ms
.005
ms
.01
ms
.1 ms 1 ms
nlogn
.003ms
.008ms
.015ms
.03ms
.07ms
1 ms 13 ms
n2 .01
ms.04ms
.09ms
.25ms
1 ms 100ms
10 s
n
3
.1
ms
.8
ms
2.7
ms
12.5
ms
100
ms
100 s 28 h
2n .1
ms.1 s 100 s 3 y 3x
1013
c
inf inf
C
O
MP
L
E
X
I
T
Y
7/30/2019 C analysis
8/15
10-04-03 P.P.Chakrabarti, IIT Kharagpur 8
Maximum size solvable within 1 hour
speedcomplexity
S 100 S 1000 S
n N1 =3.6x10
10
100 N1 1000 N1
n log n N2 =
1.2x109
85 N2 750 N2
n2 N3 =
2x105
10 N3 30 N3
2n N4 = 35 N4+7 N4+10
7/30/2019 C analysis
9/15
10-04-03 P.P.Chakrabarti, IIT Kharagpur 9
Formal Definition
? T(N) = O(f(N)) if there are positive constants c and n0 suchthat T(N) ? c f(N) when N ? n0.
Meaning : As N increases, T(N) grows no faster
than f(N).
The function T is eventually bounded by some multiple off(N). f(N) gives an upper bound in the behavior of T(N).
logen = O(n)
n2 = O(n2).
n2 = O(n3).nlogn = O(n2).
3 n2 + 5n + 1 = O(n2)
7/30/2019 C analysis
10/15
10-04-03 P.P.Chakrabarti, IIT Kharagpur 10
Some Worst Case Complexities:
? Linear Search in an array: ?
? Binary Search: ?
? Selection Sort: ?
? Mergesort: ?
? Quicksort: ?
? Stack ADT: Push and Pop are ?
? Queue using a singly linked list: ?? (What about a
doubly linked list??)
? Multiplying 2 square matrices:?
? The primes that we wrote in class:?
7/30/2019 C analysis
11/15
10-04-03 P.P.Chakrabarti, IIT Kharagpur 11
Some Worst Case Complexities:
? Linear Search in an array: O(n)
? Binary Search: O(log n)
? Selection Sort: O(n2)
?Mergesort:? T(n) = 2T(n/2) + cn
? Solves to O(n log n)
? T(n) = 2(2T(n/4) + cn/2) + cn
= 4T(n/4) + 2cn= 2kT(n/2k) + kcn [Note that T(2) = 1]
= O(nk) = O(n log n)
7/30/2019 C analysis
12/15
10-04-03 P.P.Chakrabarti, IIT Kharagpur 12
Some Worst Case Complexities:
?
Quicksort: O(n2
)[This occurs when every partition breaks
the array of n elements into 1 and n-1]
?
Stack ADT: Push and Pop are O(1)?Queue using a singly linked list:
? Insert (Enqueue): O(1)
?Delete (Dequeue): O(n)?Queue using a doubly linked list: All O(1)
7/30/2019 C analysis
13/15
10-04-03 P.P.Chakrabarti, IIT Kharagpur 13
Primes:
? For an integer of value k we loop at most sqrt(k)
times. So is the complexity O(sqrt(n))?
? Remember that the input for value n is in log2 k
bits.
? So the input size is n = log2 n
? The complexity in terms of input size for this
algorithm is therefore O(2n)
? The latest proof of Dr Manindra Agarwal et alproves that prime detection can be done in O(nk)
time for some constant k
7/30/2019 C analysis
14/15
10-04-03 P.P.Chakrabarti, IIT Kharagpur 14
More definitions
? T(N) = ? (g(N)) if there are positive constants c and n0 such that
T(N) ? c f(N) when N ? n0.
Meaning : As N increases, T(N) grows no slower than g(N) ;
T(N) grows at least as fast as g(N).
? T(N) = ?(h(N)) if and only if T(N) = O (h(N)) and T(N) = ? (h(N))Meaning : As N increases, T(N) grows as fast as h(N).
? T(N) = o(p(N)) if T(N) = O(p(N)) and T(N) ? ?(p(N))
Meaning : As N increases, T(N) grows slower than p(N).
lim n? ? T(N)/p(N) = 0.
logen = O(n)
n10 = o(2n)
3 n2 + 5n + 1 = ?(n2)
7/30/2019 C analysis
15/15
10-04-03 P.P.Chakrabarti, IIT Kharagpur 15
Space Complexity
? Measures the work space or space required in addition toinput storage
? Selection Sort: O(1)
? Binary Search:
?Recursive algorithm : O(log n) {Recursion Stack}
? Iterative algorithm: O(1)
? Mergesort: O(n) {Why: Additional array. Here recursion
stack is of depth log n}
? Quicksort: O(n) {Why: recursion stack in the worst case}