View
16
Download
2
Category
Preview:
Citation preview
ANALYSIS OF ALGORITHM Detailed study of the basic notions of the design
of algorithms and the underlying data structures
Several measures of complexity are introduced
Emphasis on the structure, complexity, and efficiency of algorithms.
Prepare the Students for Analyzing algorithms solving same problems & making them capable of writing optimize algorithms
Instructor: Sadia Arshid
ANALYSIS OF ALGORITHM Text Book:
The Design And Analysis of Algorithm by Anany Levitin
Fundamentals of Algorithm by Richard Neopolitan
Reference Material:Algorithm Design by Micheal T. Good Rich
Introduction to Algorithms /2E, T. H. Cormen, C. E. Leiserson, and R. L. Rivest, MIT Press, McGraw-Hill, New York, NY, 2001.
Grading policy
Instructor: Sadia Arshid
Assignments 5%
Quizzes 10%
Project 15%
Midterm 20%
Final 50%
WHAT IS AN ALGORITHM? An algorithm is a set of rules for carrying out
calculation either by hand or on a machine.
An algorithm is a finite step-by-step procedure to achieve a required result.
An algorithm is a sequence of computational steps that transform the input into the output.
An algorithm is a sequence of operations performed on data that have to be organized in data structures.
An algorithm is an abstraction of a program to be executed on a physical machine (model of Computation).
Instructor: Sadia Arshid
WHAT IS AN ALGORITHM?
An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time.
Instructor: Sadia Arshid
ALGORITHM
• Problem
• problem instance and solution• Sort list of n numbers in ascending order
• Determine whether x is in list of n numbers
• Algorithm is a general solution which solves all instances of a problem.
Instructor: Sadia Arshid
• Generality
• Finiteness
• Non-ambiguity
• EfficiencyInstructor: Sadia Arshid
What properties an algorithm should have ?
• The algorithm applies to all instances of input data not only for particular instances
Example:
Let’s consider the problem of increasingly sorting a sequence of values.
For instance:
(2,1,4,3,5) (1,2,3,4,5)
input data resultInstructor: Sadia Arshid
Generality
Method:
Instructor: Sadia Arshid
Generality (cont’d)
2 1 4 3 5Step 1:
1 2 4 3 5
1 2 4 3 5
1 2 3 4 5
Step 2:
Step 3:
Step 4:
Description:
- compare the first two elements: if there are not in the desired order swap them- compare the second and the third element and do the same…..- continue until the last two elements were compared
The sequence has been ordered
GENERALITY (CONT’D)
• Is this algorithm a general one ? I meant, does it assure the ordering of ANY sequence of values ?
Instructor: Sadia Arshid
• Answer: NO Counterexample:
3 2 1 4 5 2 3 1 4 5 2 1 3 4 5 2 1 3 4 5
In this case the method doesn’t work, it isn’t a general sorting algorithm
FINITENESS
• An algorithm have to terminate, i.e. to stop after a finite number of steps
Example
Step1: Assign 1 to x;
Step2: Increase x by 2;
Step3: If x=10 then STOP;
else GO TO Step 2
How does this algorithm work ?
Instructor: Sadia Arshid
FINITENESS (CONT’D)
How does this algorithm work and what does it produce?
Step1: Assign 1 to x;
Step2: Increase x by 2;
Step3: If x=10
then STOP;
else Print x; GO TO Step 2;Instructor: Sadia Arshid
x=1
x=3 x=5 x=7 x=9 x=11
The algorithm generates odd numbers but it never stops !
FINITENESS (CONT’D)
The algorithm which generate all odd naturals smaller than 10:
Step1: Assign 1 to x;
Step2: Increase x by 2;
Step3: If x>=10
then STOP;
else Print x; GO TO Step 2Instructor: Sadia Arshid
NON-AMBIGUITY
The operations in an algorithm must be rigorously specified:• At the execution of each step one have to know exactly
which is the next step which will be executed
Example:
Step 1: Set x to value 0
Step 2: Either increment x with 1 or decrement x with 1
Step 3: If x[-2,2] then go to Step 2; else Stop.
As long as does not exist a criterion by which one can decide if
x is incremented or decremented, the sequence above cannot be
considered an algorithm.
Instructor: Sadia Arshid
NON-AMBIGUITY (CONT’D)
Let’s modify the previous algorithm as follows:
Step 1: Set x to value 0
Step 2: Flip a coin
Step 3: If one obtains head
then increment x with 1
else decrement x with 1
Step 3: If x[-2,2] then go to Step 2, else Stop.
• This time the algorithm can be executed but … different executions can be different
• This is a so called random algorithm
Instructor: Sadia Arshid
EFFICIENCY
• The need to design efficient algorithms
• What is efficiency?
• Example
• Sequential search Vs. Binary Search
Instructor: Sadia Arshid
ALGORITHM ANALYSIS
• How to analyze an algorithm?
• Predicting the resources that the algorithm requires.
• Memory
• Communications Bandwidth
• Logic gates etc
• Most important is Computational Time
Instructor: Sadia Arshid
ALGORITHM ANALYSIS
• Important thing before analysis
• Model of the machine upon which the algorithms is executed.
• Random Access Machine (RAM) (Sequential)
• Running Time: No. of primitive operations or “steps executed”.
Instructor: Sadia Arshid
RUNNING TIME
Most algorithms transform input objects into output objects.
The running time of an algorithm typically grows with the input size.
Average case time is often difficult to determine.
We focus on the worst case running time.Easier to analyzeCrucial to applications such
as games, finance and robotics
0
20
40
60
80
100
120
Runnin
g T
ime
1000 2000 3000 4000
Input Size
best caseaverage caseworst case
Instructor: Sadia Arshid
EXPERIMENTAL STUDIES
Write a program implementing the algorithm
Run the program with inputs of varying size and composition
Use a method in “Time.h” to get an accurate measure of the actual running time
Plot the results
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
0 50 100
Input Size
Tim
e (
ms)
Instructor: Sadia Arshid
LIMITATIONS OF EXPERIMENTS
• It is necessary to implement the algorithm, which may be difficult
• Results may not be indicative of the running time on other inputs not included in the experiment.
• In order to compare two algorithms, the same hardware and software environments must be used
Instructor: Sadia Arshid
THEORETICAL ANALYSIS
• Uses a high-level description of the algorithm instead of an implementation
• Characterizes running time as a function of the input size, n.
• Takes into account all possible inputs
• Allows us to evaluate the speed of an algorithm independent of the hardware/software environment
Instructor: Sadia Arshid
ALGORITHM ANALYSIS
• Running Time
• We need a measure that is independent of computer, programming language and the programmer.
• Concept of basic operation
• In sorting comparison is a basic operation.
Instructor: Sadia Arshid
ALGORITHM ANALYSIS
• How much time each construct / keyword of a pseudo code takes to execute. Assume it
takes ti (the ith construct)
• Sum / Add up the execution time of all the constructs / keywords. if there are m constructs then
Instructor: Sadia Arshid
m
iitT
1
ALGORITHM ANALYSIS
• That will be the execution time for a given input size (say n)
Instructor: Sadia Arshid
m
iin tT
1
Running time as the function of the input size T(n)
ALGORITHM ANALYSIS
• What are the constructs / Keywords.
• Time for each construct
• Total Time
• Total time as a function of input size
Instructor: Sadia Arshid
ALGORITHM ANALYSIS
• Construct:
• Sequence
• Selection• Iterations• Recursion
Instructor: Sadia Arshid
ALGORITHM ANALYSIS Sequence Statements: Just add the running time
of the statements If-Then-Else: if (condition) S1 else S2
Running time of the test plus the larger of the running times of S1 and S2.
Iteration is at most the running time of the statements inside the loop, including the times the number of iterations.
Nested Loops: Analyze these inside out. The total Running time of a statement inside a group of nested loops is the running time of the statement multiplied by the product of the size of all the loops.
Function Calls: Analyzing from inside to out. If there are function calls, these must be analyzed first.
Instructor: Sadia Arshid
EXAMPLE
Instructor: Sadia Arshid
Algorithm arrayMax(A, n)Input array A of n integersOutput maximum element of A
currentMax A[0]for i 1 to n 1 do
if A[i] currentMax thencurrentMax A[i]
return currentMax
Example: find max element of an array
COUNTING PRIMITIVE OPERATIONS
By inspecting the pseudocode, we can determine the maximum number of primitive operations executed by an algorithm, as a function of the input size
Algorithm arrayMax(A, n) # operationscurrentMax A[0] 2for i 1 to n 1 do 1 n
if A[i] currentMax then 2(n 1)
currentMax A[i] 2(n 1){ increment counter i } 2(n 1)return currentMax 1
Total 7n 2Instructor: Sadia Arshid
ESTIMATING RUNNING TIME• Algorithm arrayMax executes 7n 2 primitive operations in the
worst case.
• This notation is complex to use. To simplify it we will be using order of growth that is dominating term of expression; which is n (will be discuss in detail later)
• Instead of computing running time of every statement of the algorithm we can further simplify it by calculating running time of basic operation only(Time Complexity) (will be discuss in detail later)
• In This example • basic operation is comparison
• Executed n-1 number of times
• Order of growth is n
• Same as order of growth of computing running time of every statement.Instructor: Sadia Arshid
IMPORTANT PROBLEM TYPES
• Sorting
• Sorting keys
• Stable
• Inplace
• Searching
• String processing
Instructor: Sadia Arshid
IMPORTANT PROBLEM TYPES
• Graph problems
• Combinatorial Problems
• Geometric Problems
• Numerical Problems
Instructor: Sadia Arshid
Recommended