Upload
hno-al-sobhy
View
215
Download
0
Embed Size (px)
Citation preview
7/29/2019 1. Complexity Analysis
1/44
Complexity Analysis(newer version)
7/29/2019 1. Complexity Analysis
2/44
What is an algorithm?
The ideas behind computer programs Stays the same no matter
Which kind of hardware it is running on
Which programming language it is writtenin
Solves a general, well-specified problem
Is specified by Describing the set of instances (input) it
must work on
Describing the desired properties of theoutput
7/29/2019 1. Complexity Analysis
3/44
Important Properties of Algorithms
Correct
always returns the desired output for all
legal instances of the problem. Efficient
Can be measured in terms of
Time
Space
Time tends to be more important
7/29/2019 1. Complexity Analysis
4/44
Expressing Algorithms
English description
Pseudocode
High-levelprogramminglanguage
Moreprecise
More easilyexpressed
7/29/2019 1. Complexity Analysis
5/44
Pseudocode
A shorthand for specifying algorithms Leaves out the implementation details
Leaves in the essence of the algorithm
Algorithm ArrayMax(A,n)
Input: An array A storing n1 integers
Output: The maximum element in A.
currentMax A[0]
for i 1 to n-1 do
ifcurrentMax < A[i] then
currentMax A[i]
return currentMax
7/29/2019 1. Complexity Analysis
6/44
Analysis of Algorithms
Why analyze algorithms? Evaluate algorithm performance
Compare different algorithms
Analyze what about them? Running time, memory usage, solution
quality
Worst-case and typical case Analysis of algorithms compare
algorithms, not programs.
7/29/2019 1. Complexity Analysis
7/44
If each line takes constant time the whole algorithm (anyalgorithm) will take constant time, right?
Wrong!
Although some algorithms may take constant time, the
majority of algorithms varies its number of steps based onthe size of instance we're trying to solve
Therefore the efficiency of an algorithm is always normallystated as a function of the problem size
We generally use the variable n to represent the problem size
On the implementation, we could find out thatSuperDuper Sort takes 0.6n2 + 0.3n + 0.45 seconds on aPentium 3.
Plug a value for n and you have how long it takes
Analysis of Algorithms (Cont.)
7/29/2019 1. Complexity Analysis
8/44
Algorithm Complexity
Worst Case Complexity: The function defined by the maximum
number of steps taken on any instance ofsize n
Best Case Complexity:
The function defined by the minimumnumber of steps taken on any instance of
size n
Average Case Complexity:
The function defined by the average number
of steps taken on any instance of size n
7/29/2019 1. Complexity Analysis
9/44
Best, Worst, and Average CaseComplexity
Worst CaseComplexity
Average CaseComplexity
Best CaseComplexity
Number ofsteps
n(input size)
7/29/2019 1. Complexity Analysis
10/44
Doing the Analysis
Its hard to estimate the running time exactly
Best case depends on the input
Average case is difficult to compute
So we usually focus on worst case analysis
Easier to compute
Usually close to the actual running time
Strategy: try to find upperand lower bounds of
the worst case function.
Upper bound
Lower bound
Actual function
7/29/2019 1. Complexity Analysis
11/44
Running Time Analysis
7/29/2019 1. Complexity Analysis
12/44
Running Time Analysis
7/29/2019 1. Complexity Analysis
13/44
Comparing Algorithms
Establish a relative order amongdifferent algorithms in terms of
their relative rates of growth.The rates of growth are expressed
as functions, which are generally
in terms of the number of inputs n.
7/29/2019 1. Complexity Analysis
14/44
Asymptotic Analysis
Asymptotic analysis of an algorithm describesthe relative efficiency of an algorithm as n getvery large.
When you're dealing with small input size,
most of algorithms will do When the input size is very large things change
In the example it is easy to see that for verylarge n, g(n) grows faster than f(n) Take for instance the value n=20000000
Remember that the goal here is to comparealgorithms. In practice, if you're writing smallprograms, asymptotic analysis may not be thatimportant
7/29/2019 1. Complexity Analysis
15/44
A simple comparison
Let's assume that you have 3 algorithms to sort a list
f(n) = n log2n g(n) = n2
h(n) = n3
Let's also assume that each step takes 1 microsecond(10-6)
Most of the algorithms discussed here will be givenin terms of common functions: polynomials,logarithms, exponentials and product of thesefunctions
n n log n n^2 n^3
10 33.2 100 1000100 664 10000 1seg
1000 9966 1seg 16min
100000 1.7s 2.8 hours 31.7 years
7/29/2019 1. Complexity Analysis
16/44
Big-Oh Notation
T(n) = O(f(n)) if there are positive constants cand n0 such that T(n) cf(n) for n n0
This says that function T(n) grows at a rate no
faster thanf(n). Thus,f(n) is an upper
bound on T(n).
T(n) = O(f(n))
cf(n)
T(n)
7/29/2019 1. Complexity Analysis
17/44
n0
T(n) = O(f(n))
Big-Oh Upper Bound
7/29/2019 1. Complexity Analysis
18/44
Example
Prove that 7n3+2n2 = O(n3)
Since 7n3+2n2 < 7n3 +2n3 = 9n3 (for n 1)
Then 7n3+2n2 = O(n3) with c = 9 and n0 = 1
Similarly, we can prove that 7n3
+2n2
=O
(n4
). The first bound is tighter.
7/29/2019 1. Complexity Analysis
19/44
Tighter Upper Bound
n0
7/29/2019 1. Complexity Analysis
20/44
Big-Omega Notation
T(n) = (f(n)) if there are positive constants c and n0such that T(n) cf(n) for n n0
This says that function T(n) grows at a rate no slower
thanf(n).
Thus,f(n) is a lower bound
on T(n)
T(n) = ( (n))
cf(n)
T(n)
7/29/2019 1. Complexity Analysis
21/44
T(n) =
(f(n))
n0
Big-Omega Lower Bound
7/29/2019 1. Complexity Analysis
22/44
Example
Prove that 2n+5n2 = (n2)
Since 2n+5n2 > 5n2 > 1 n2 (for n 1)
Then 2n+5n2 = (n2) with c = 1 and n0 = 1 Similarly, we can prove that 2n+5n2 = (n).
The first bound is tighter.
7/29/2019 1. Complexity Analysis
23/44
Tighter Lower Bound
n0
h
7/29/2019 1. Complexity Analysis
24/44
Big-Theta Notation
T(n) = (f(n)) if and only ifT(n) = O(f(n)) andT(n) = (f(n))
This says that function T(n) grows at the same rate asf(n).
Put it another way:
T(n) = (f(n)) if there are
positive constants c1
, c2
,
and n0 such that
c1f(n) T(n) c2f(n) for n n0
T(n) = ( (n))
T(n)
c2f(n)
c1f(n)
7/29/2019 1. Complexity Analysis
25/44
Li l h
7/29/2019 1. Complexity Analysis
26/44
Little-oh
T(n) = o(f(n)) if and only if
T(n) = O(f(n)) and T(n) (f(n)) if
This says that function T(n) grows at a ratestrictly less thanf(n)
Hi h f G h R
7/29/2019 1. Complexity Analysis
27/44
Hierarchy of Growth Rates
c < log n < log2 n < logk n < n
7/29/2019 1. Complexity Analysis
28/44
7/29/2019 1. Complexity Analysis
29/44
7/29/2019 1. Complexity Analysis
30/44
7/29/2019 1. Complexity Analysis
31/44
7/29/2019 1. Complexity Analysis
32/44
7/29/2019 1. Complexity Analysis
33/44
7/29/2019 1. Complexity Analysis
34/44
7/29/2019 1. Complexity Analysis
35/44
7/29/2019 1. Complexity Analysis
36/44
36
7/29/2019 1. Complexity Analysis
37/44
7/29/2019 1. Complexity Analysis
38/44
7/29/2019 1. Complexity Analysis
39/44
7/29/2019 1. Complexity Analysis
40/44
7/29/2019 1. Complexity Analysis
41/44
7/29/2019 1. Complexity Analysis
42/44
7/29/2019 1. Complexity Analysis
43/44
7/29/2019 1. Complexity Analysis
44/44
44