Unit 1 Notion

DESIGN AND ANALYSIS OF ALGORITHMS

Text Book:

Introduction to the Design and Analysis of Algorithms.

- Anany Levitin

UNIT I

• Introduction• Notion of Algorithm• Fundamentals of Algorithmic Solving• Important Problem Types• Fundamentals of the Analysis Framework• Asymptotic Notations • Basic Efficiency Classes

Introduction

• Algorithmics is more than a branch of computer science.

• It is the core of computer science, and, in all fairness, can be said to be relevant to most of science, business, and technology.

Notion of Algorithm

• An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time.

Notion of Algorithm

GCD of 2 Numbers

• Euclid’s algorithm for computing gcd(m, n)• Step 1: If n = 0, return the value of m as the

answer and stop; otherwise, proceed to Step 2.

• Step 2: Divide m by n and assign the value of the remainder to r.

• Step 3: Assign the value of n to m and the value of r to n. Go to Step 1.

GCD of 2 Numbers

ALGORITHM Euclid(m, n)

//Computes gcd (m, n) by Euclid’s algorithm

//Input: Two nonnegative, not-both-zero integers m and n

//Output: Greatest common divisor of m and n

while n != 0 do

r ←m mod n

m←n

n←r

return m

Notion of Algorithm

• Consecutive integer checking algorithm for computing gcd(m, n)

• Step 1: Assign the value of min{m, n} to t.

• Step 2: Divide m by t. If the remainder of this division is 0, go to Step 3;

otherwise, go to Step 4.

Notion of Algorithm

• Step 3: Divide n by t. If the remainder of this division is 0, return the value of t as the answer and stop; otherwise, proceed to Step 4.

• Step 4: Decrease the value of t by 1. Go to Step 2.

Notion of Algorithm

Middle-school procedure for computing gcd(m, n)

Step 1: Find the prime factors of m.

Step 2: Find the prime factors of n.

Notion of Algorithm

Middle-school procedure for computing gcd(m, n)

Step 3: Identify all the common factors in the two prime expansions found in Step

1 and Step 2.

Step 4: Compute the product of all the common factors and return it as the

greatest common divisor of the numbers given.

Notion of Algorithm

For the numbers 60 and 24, we get

60 = 2 . 2 . 3 . 5

24 = 2 . 2 . 2 . 3

gcd(60, 24) = 2 . 2 . 3 = 12.

Notion of Algorithm

Sieve of Eratosthenes method of generating prime numbers:

Notion of Algorithm

ALGORITHM Sieve(n)

//Implements the sieve of Eratosthenes

//Input: A positive integer n > 1

//Output: Array L of all prime numbers less than or equal to n

for p←2 to n do

A[p]←p

Notion of Algorithm

for p←2 to √ n do

if A[p] !=0

j ← p p∗while j ≤ n do

A[j ]←0

j ←j + p

end while

End for

Notion of Algorithm

//copy the remaining elements of A to array L of the primes

i ←0

for p←2 to n do

if A[p] = 0

L[i]←A[p]

i ←i + 1

return L

Notion of Algorithm

• Find gcd(31415, 14142) by applying Euclid’s algorithm.

Notion of Algorithm

• gcd(31415, 14142) • gcd(14142, 3131) • gcd(3131, 1618)• gcd(1618, 1513) • gcd(1513, 105) • gcd(1513, 105) • gcd(105, 43) • gcd(43, 19) = gcd(19, 5) = gcd(5, 4) • gcd(4, 1) = gcd(1, 0) = 1.

Fundamentals of Algorithmic Problem Solving

• Algorithms can be the procedural solutions to problems.

• These solutions are not answers but specific instructions for getting answers.


• Understanding the Problem

• Ascertaining the Capabilities of the Computational Device

• Choosing between Exact and Approximate Problem Solving

• Algorithm Design Techniques


• Designing an Algorithm and Data Structures

• Methods of Specifying an Algorithm

• Proving an Algorithm’s Correctness

• Analyzing an Algorithm

• Coding an Algorithm

Important Problem Types

• Sorting• Searching• String processing• Graph problems• Combinatorial problems• Geometric problems• Numerical problems

Examples

• Write an algorithm for the sorting problem that sorts an array by counting, for each of its elements, the number of smaller elements and then uses this information to put the element in its appropriate position in the sorted array.

• Apply this algorithm to sorting the list

60, 35, 81, 98, 14, 47.

Examples

ALGORITHM ComparisonCountingSort(A[0..n − 1])

//Sorts an array by comparison counting

//Input: Array A[0..n − 1] of orderable values

//Output: Array S[0..n − 1] of A’s elements sorted in nondecreasing order

Examples

for i ←0 to n − 1 do

Count[i]←0


for j ←i + 1 to n − 1 do

if A[i]<A[j ]

Count[j ]←Count[j ]+ 1

else Count[i]←Count[i]+ 1

Examples


S[Count[i]] = a[i]

Return S

Fundamentals of the Analysisof Algorithm Efficiency

• The Analysis Framework:

• There are two kinds of efficiency:

• Time efficiency

• Space efficiency.

Fundamentals of the Analysisof Algorithm Efficiency

• Time efficiency, also called time complexity, indicates how fast an algorithm in question runs.

• Space efficiency, also called space complexity, refers to the amount of memory units required by the algorithm in addition to the space needed for its input and output.

Measuring an Input’s Size

• Let’s start with the obvious observation that almost all algorithms run longer on larger inputs.

• For example, it takes longer to sort larger arrays, multiply larger matrices, and so on.

• Therefore, it is logical to investigate an algorithm’s efficiency as a function of some parameter n indicating the algorithm’s input size.

Units for Measuring Running Time

• We can simply use some standard unit of time measurement—a second, or millisecond, and so on—to measure the running time of a program implementing the algorithm.

• There are obvious drawbacks to such an approach.


• Dependence on the speed of a particular computer, dependence on the quality of a program implementing the algorithm and of the compiler used in generating the machine code, and the difficulty of clocking the actual running time of the program.

• Since we are after a measure of an algorithm’s efficiency, we would like to have a metric that does not depend on these extraneous factors.


• One possible approach is to count the number of times each of the algorithm’s operations is executed.

• The thing to do is to identify the most important operation of the algorithm, called the basic operation, the operation contributing the most to the total running time, and compute the number of times the basic operation is executed.


• As a rule, it is not difficult to identify the basic operation of an algorithm: it is usually the most time-consuming operation in the algorithm’s innermost loop.

• For example, most sorting algorithms work by comparing elements (keys) of a list being sorted with each other; for such algorithms, the basic operation is a key comparison.


• As another example, algorithms for mathematical problems typically involve some or all of the four arithmetical operations: addition, subtraction, multiplication, and division.

• Of the four, the most time-consuming operation is division, followed by multiplication and then addition and subtraction, with the last two usually considered together.


• Thus the established framework for the analysis of an algorithm’s time efficiency suggests measuring it by counting the number of times the algorithm’s basic operation is executed on inputs of size n.


• Let cop be the execution time of an algorithm’s basic operation on a particular computer, and let C(n) be the number of times this operation needs to be executed for this algorithm.

• Then we can estimate the running time T (n) of a program implementing this algorithm on that computer by the formula:

Orders of Growth

Worst-Case, Best-Case, and Average-Case Efficiencies

• The worst-case efficiency of an algorithm is its efficiency for the worst-case input of size n, which is an input (or inputs) of size n for which the algorithm runs the longest among all possible inputs of that size.

• The worst-case analysis provides very important information about an algorithm’s efficiency by bounding its running time from above.


• The best-case efficiency of an algorithm is its efficiency for the best-case input of size n, which is an input (or inputs) of size n for which the algorithm runs the fastest among all possible inputs of that size.


• However, that neither the worst-case analysis nor its best-case counterpart yields the necessary information about an algorithm’s behavior on a “typical” or “random” input.

• This is the information that the average-case efficiency seeks to provide.

• To analyze the algorithm’s average case efficiency, we must make some assumptions about possible inputs of size n.


Amortized Efficiency

• It applies not to a single run of an algorithm but rather to a sequence of operations performed on the same data structure.

• It turns out that in some situations a single operation can be expensive, but the total time for an entire sequence of n such operations is always significantly better than the worst-case efficiency of that single operation multiplied by n.

Amortized Efficiency

• So we can “amortize” the high cost of such a worst-case occurrence over the entire sequence in a manner similar to the way a business would amortize the cost of an expensive item over the years of the item’s productive life.

Asymptotic Notations and Basic Efficiency Classes

• The efficiency analysis framework concentrates on the order of growth of an algorithm’s basic operation count as the principal indicator of the algorithm’s efficiency.

• To compare and rank such orders of growth, computer scientists use three notations: • O(big oh), Ω(big omega), and θ(big theta).

Asymptotic Notations and Basic Efficiency Classes

• The idea of these definitions is to establish a relative order among functions.

• Given two functions, we compare their relative rates of growth.

• Example 1000n and n2

O - Notation

• A function t (n) is said to be in O(g(n)), denoted t (n) O(g(n)), if t (n) is bounded above by some ∈constant multiple of g(n) for all large n, i.e., if there exist some positive constant c and some nonnegative integer n0 such that

t (n) ≤ cg(n) for all n ≥ n0.

O - Notation

• Example:

• 100n + 5 O(n∈ 2)

O - Notation

Ω - Notation

• A function t (n) is said to be in (g(n)), denoted t (n) (g(n)), if t (n) is bounded below by ∈some positive constant multiple of g(n) for all large n, i.e., if there exist some positive constant c and some nonnegative integer n0 such that

t (n) ≥ cg(n) for all n ≥ n0.

Ω - Notation

• Example

• n3 ∈ Ω(n2)

Ω - Notation

Θ - Notation

• A function t (n) is said to be in (g(n)), denoted t (n) (g(n)), if t (n) is bounded both above ∈and below by some positive constant multiples of g(n) for all large n, i.e., if there exist some positive constants c1 and c2 and some nonnegative integer n0 such that

c2g(n) ≤ t (n) ≤ c1g(n) for all n ≥ n0.

Θ - Notation

• Example:

• 1/2 n (n-1) ∈ Θ (n2)

Θ - Notation

THEOREM

• If t1(n) O(g1(n)) and t2(n) O(g2(n)), then∈ ∈

t1(n) + t2(n) O(max{g1(n), g2(n)}).∈

PROOF

if a1 <= b1 and a2 <= b2, then

a1 + a2 < = 2 max{b1, b2}

Θ - Notation

RULES

If T1(n) = O(f(n)) and T2(n)=O(g(n)), then

(a) T1(n) + T2(n) = max (O(f(n)), O(g(n)))

(b) T1(n) * T2(n) = O(f(n) * g(n))

RULES

• If T(x) is a polynomial of degree n, then T(x) = θ(x n).

• Log k n = O(n) for any constant k. This tells us that logarithms grow very slowly.

RULES

• If two programs are expected to take

similar times, probably the best way to

decide which is faster is to code them both

up and run them.

Using Limits for Comparing Orders of Growth

• Though the formal definitions of O, , and are indispensable for proving their abstract properties, they are rarely used for comparing the orders of growth of two specific functions.

• A much more convenient method for doing so is based on computing the limit of the ratio of two functions in question.



• Compare the orders of growth of the following:

• ½ n(n − 1) and n2

• Log 2 n and √n

• n! And 2n

ASSIGNMENT

1. What is an algorithm? Explain the notion of an algorithm with a neat diagram.

2. Give and explain the important characteristics of an algorithm.

3. Write and explain the Euclid’s algorithm for computing GCD of any two integers. Hence find gcd(31415, 14142).

ASSIGNMENT

4. Explain with an example the consecutive integer checking algorithm for computing GCD of any two integers.

5. Implement Sieve’s algorithm to generate prime numbers between 1 and n. Trace the algorithm for n = 25.

6. Explain with a neat diagram, the sequence of steps to be followed in designing and analyzing an algorithm.

ASSIGNMENT

7. Write an algorithm for finding the distance between the two closest elements in an array of numbers.

8. Explain the important problem types. Give two examples for each.

9. Implement an algorithm that sorts an array by counting, for each of its elements, the number of smaller elements and then uses this information to put the element in its appropriate position in the sorted array. Apply this algorithm to sorting the list 60, 35, 81, 98, 14, 47.

ASSIGNMENT

10. Explain the terms: time and space efficiency.

11. Explain the units for measuring the running time.

12. Compare the orders of growth of various algorithms.

13. Define the following: worst case, best case and average case analysis of an algorithm.

ASSIGNMENT

14. Give the worst case, best case and average case analysis of sequential search algorithm.

15. What is amortized efficiency? Explain with an example.

ASSIGNMENT

16. For each of the following algorithms, indicate (i) a natural size metric for its inputs, (ii) its basic operation, and (iii) whether the basic operation count can be different for inputs of the same size:

a. computing the sum of n numbers

b. computing n!

c. finding the largest element in a list of n numbers

d. Euclid’s algorithm

e. sieve of Eratosthenes

f. pen-and-pencil algorithm for multiplying two n-digit decimal integers

ASSIGNMENT

17. Explain the different asymptotic notations with an example for each notation.

18. If If t1(n) O(g1(n)) and t2(n) O(g2(n)), then ∈ ∈show that t1(n) + t2(n) O(max{g1(n), g2(n)}).∈

19. Compare the orders of growth for the following:

a). ½ n(n − 1) and n2

b) Log 2 n and √n

c) n! And 2n

ASSIGNMENT

20. Indicate whether the first function of each of the following pairs has a smaller, same or larger order of growth than second function.

a. n (n+1) and 2000n2

b. 100n2 and 0.01n3

c. Log 2 n and ln n

d. 2n-1 and 2n

e. Log 2 n and log 2 n2

f. (n-1)! And n!

ASSIGNMENT

21. For each of the following functions, indicate the class (g(n)) the function belongs to. Prove your assertions.

22. Prove that if t1(n) Ω(g1(n)) and t2(n) Ω(g2(n)), ∈ ∈then t1(n) + t2(n) ∈ Ω(max{g1(n), g2(n)}).

ASSIGNMENT

23. Prove that every polynomial of degree k,

p(n) = aknk + ak−1nk−1+ . . . + a0

with ak > 0, belongs to θ (nk).

24.List the following functions according to their order of growth from the lowest to the highest:

ASSIGNMENT

25. Find the Big – oh notation for the following:a. Log n + √n

b. n + n log n

c. 6n + 2n4 + 4n5

26. With the help of a flowchart, explain the various stages of algorithm design and analysis process.

27. Explain the different problem types and give one algorithm for each type.

ASSIGNMENT

28. Use the informal definitions of O,, and to determine

whether the following assertions are true or false.

Documents

Unit 1 Notion