Program Design and Algorithm Analysis

INTRODUCTION TO PROGRAM DESIGN

• Understand the program development life cycle.

• Understand the algorithm and the notation to be used for expressing them.

• Understanding the various control structure

• Understanding the algorithms complexity• Understand the big O notation• Understand the recursion

PROGRAM DEVELOPMENT LIFE CYCLE.

The program development process is divided into following phases:

• Defining the problem.• Designing the problem.• Coding the problem.• Testing and debugging the problem.• Documenting the program.• Implementing and maintaining the program.

DEFINING THE PROBLEM.

• To form the program specification. Which includes:

1. Input data

2. Processing that should take place

3. Format of the output report

4. User interface

5. Handling of job by individual or team

DESIGNING THE PROBLEM

• Program design begins by focusing on the main goal that the program is going to achieve and breaking the program into manageable components each of which contribute to this goal.

• Approach program design is called top down program design or modular programming.

PROGRAM DESIGN TOOLS

• Structure charts

• Algorithms

• Flowcharts

• Pseudo codes

CODING THE PROBLEM

• Coding the program involves translating the algorithm into specific program language instructions.

• While writing the code, prefer to use well defined control structures.

• This technique of programming using only well defined control structures is known as structured programming.

TESTING AND DEBUGGING THE PROBLEM

• Programmer must find and correct logical errors by carefully examining the program output for a set of data for which results are already known, such type of data is known as test data.

• If the software developed is a complex one and consists of large number of modules then testing is done unit testing , system testing.

• Syntax errors and logical errors are collectively known as bugs. The process of identifying and eliminating these errors is known as debugging

DOCUMENTING THE PROGRAM

• After testing, the software is almost complete. The structure chart, pseudo codes, flow chart developed during the design phase become documentation for others who are associated with the software project.

• In addition more documentation done as the program are being coded such as list of variable names and definition, description of files that the program need to be work with and the format of the output that the program produces.

IMPLEMENTING AND MAINTAINING THE PROGRAM

• In the final phase, the program is installed at the user’s site. Here also, the program is kept under watch till the user gives green signal to it. User may discover the errors that were not caught in the testing phase.

• Even after the software project is complete, it needs to be maintained and evaluated regularly. In program maintenance, the programming team fixes program errors that user discover during day to day use.

INTRODUCTION TO ALGORITHM

An algorithm is a finite set of steps defining the solution of a particular problem.

Every algorithm must satisfy the following criteria:• Input: There are zero or more values which are

externally supplied.• Output: At least one value is produced• Definiteness: Each step must be clear and

unambiguous.• Finiteness: Must terminate after a finite number of

steps.• Effectiveness: Each step must be definite and must

be feasible.

EXAMPLE

• Given an one dimensional array a[0,n-1] of real values, and we want to find the location of its largest element.

• Algorithm1

Let a[0:n-1] be one dimensional array with n real values. This algorithm finds the location loc of its largest element. The variable i is used as index variable, max is temporary variable to store the current largest element.

ALGORITHMBegin

Read: n //enter value of nfor i=0 to (n-1) by 1 do

read: ai //enter elements of arrayendforset max=a0 //take first element as the largestset loc=0 //set location of the element to 0 for i=0 to (n-1) by 1 do

if(ai>max) then //compare the next element with maxset max=ai //take the element as the current largestset loc =I; //update location of the largest element

endifendforwrite: “location of the largest element=“,loc

end

ALGORITHM DESCRIPTION

Algorithm consists of two parts:

1. Describe the input data, the purpose of the algo. Identifies the variable used in algorithms.

2. Composed of sequence of instructions that lead to the solution of the problem.

SUMMARY OF CONVENTION USED FOR PRESENTING

ALGORITHMComments: which explain the purpose of instruction.

//this is a sample commentVariable Names: we use italicized lower case letters

such as max, loc etc.Assignment statement: the assignment statement will

use the notation as set max=ai

Input/output: data may be input and assigned to variables by means of a read statements with the following format

read: variable listOutput is done by write statement

write: message and/ or variable list

STRUCTURE PROGRAMMING CONSTRUCTS

• Sequential programming construct

• Selection programming construct

• Iterative programming construct

17

THE if SELECTION STRUCTURE

• if structure is a single-entry/single-exit structure

true

false

marks >= 40

print “Passed” 

A decision can be made on any expression.

zero - false

nonzero - true

Example:

3 - 4 is true

18

THE if/else SELECTION STRUCTURE

• Flow chart of the if/else selection structure

truefalse

print “Failed” print “Passed”

grade >= 40

19

Case 1

START

Case 2

Case 3

Case 4

STOP

Statement 1

Statement 1

Statement 1

Statement 1

Yes

Yes

Yes

YesNo

No

No

No

switch (choice)

{

case 1:

statement 1;

break;

case 2:

statement 2;

break;

case 3:

statement 3;

break;

case 4:

statement 4;

break;

}

SWITCH STATEMENT

20

THE WHILE REPETITION STRUCTURE

int number=1;while ( number <= 10){

printf(“number=%d”,number) ;

number=number+1;

}

• NUMBER=1

number <= 10 number = number+1true

false

21

THE do/while REPETITION STRUCTURE

true

false

action(s)

condition

22

FOR LOOP Syntax:for(initialization;condition;increment/decrement)

e.g for(i=0;i<10;i++) { printf(“%d”,i); }

What is Performance Analysis?

In order to be accessed, data is stored in a data structure. Often, there is a choice of data structures that can be used to store the data. Almost any data structure can be accessed by more than one algorithm.

Frequently, more than one algorithm performs the same action.

In order to chose between data structures and algorithms, the efficiency of competing data structures (in terms of space required) and competing algorithms (in terms of time used) must be compared.

Frequency Count Analysis • A frequency count of statements

executed is the most direct form of a priori analysis of the time used by an algorithm.

• Each statement in a program adds the value of 1 to the frequency count each time it is executed.

• int sum=0;

• for(int i=0; i<5; i++)

• sum += i;

• print (sum);

• The frequency count analysis of this fragment is:

• int sum=0; // add 1 to the time count, add 1 to the space count

• for(int i=0; i<5; i++) { // add 6 to the time count, add 1 to the

space count sum += i; // add 5 to the time count } Print ( sum); // add 1 to the time count

• This example has a frequency count for time of 13 and a frequency count for space of 2. Note that compound statements symbols are not counted. Note that the FOR statement requires 1 more iteration than the body of the for loop. This is so that the for variable (i) can reach 5 and trigger the loop halt.

• int b;

• for (int x=0; x<n; x++)

• for (int y=0; y<n; y++)

• {

• b = x * y;

• Print ( b);

• }

• int b; // add 1 to space

• for (int x=0; x<n; x++) // add 1 to space,

n +1 to time

• for (int y=0; y<n; y++) { // add 1 to space,

n times n + 1 to time

• {b = x * y; // add n times n to time

• Print ( b); // add n times n to time

• }

• This gives time and space complexity:

• Time Space

n+1 1

+ n (n+1) +1

+ n2 +1

+ n2

_____________ ______

3n2+2n+1 3

• Notice that one loop dependent on n gives a result in the form of (an + b), where the largest order of magnitude is 1.

• Two nested loops dependent on n give the result in the form (an2 + bn + c), where the largest order of magnitude is 2.

• In fact, the pattern continues. With a triply nested set of loops that depend on a limit of n, the result is in the form of (an3 + bn2 + cn + d), where the largest order of magnitude is 3. This pattern continues as nesting of loops continues.

Logarithmic loops• In linear loop, the loop update either adds or

subtracts.• In a logarithmic loop, the controlling variable is

multiplied or divided in each iteration.• Multiply loop:

for(i=1;i<1000:i*=2)application code

• Divide loop:for(i=1000;i>1:i/=2)application code

• To understand this , in next slide value of i for each iteration has been given.

Multiply Divide

iteration Value of i Iteration Value of i

1 1 1 1000

2 2 2 500

3 4 3 250

4 8 4 125

5 16 5 62

6 32 6 31

7 64 7 15

8 128 8 7

9 256 9 3

10 512 10 1

(exit) 1024 (exit) 0

Logarithmic loops• As you see the number of iteration is 10 in

both cases.• The reason is that in each iteration the

value of i doubles for multiply loop and is cut in half for divide loop.

• Thus the number of iterations is a function of the multiplier or divisor.

• Generalizing the analysis, we can say that the iterations in loops that multiply or divide are determined by the following formula:

F(n)=logn

ALGORITHM COMPLEXITY

Two aspects of computer programming:

1. Data organization i.e. the data structures to represent the data of the problem in hand.

2. Choosing the appropriate algorithm to solve the problem in hand.

ALGORITHM

The choice of particular algorithm depends on following considerations:

• Performance requirements, i.e. time complexity

• Memory requirements i.e. space complexity

• Programming requirements.

SPACE COMPLEXITY

The space complexity of an algorithm, is the amount of memory it needs to run to completion.

Some reason for studying space complexity are:1. If the program is to run on multy user system, it may

be required to specify the amount of memory to be allocated to the program.

2. We may be interested to know in advance that whether sufficient memory is available to run the program.

3. There may be several possible solutions with different space requirements.

4. Can be used to estimate the size of the largest problem that a program can solve.

SPACE COMPLEXITY

Space needed by a program consists of following components:

• Instruction Space- space needed to store the executable version of the program.

• Data space- space needed to store all constants, variable.

• Environment stack space

TIME COMPLEXITY

The time complexity of an algorithm is the amount of time it needs to run to completion.

Some of the reasons for studying time complexity are:

• We may be interested to know in advance that whether the program will provide a satisfactory real time response.

• There may be several possible solutions with different time requirements.

TIME SPACE TRADE-OFF

• The best algorithm, hence best program to solve a given problem is one that requires less space in memory and takes less time to complete its execution.

• In practice it is not always possible to achieve both of these objectives. we have to sacrifice one at the cost of other.

EXPRESSING SPACE AND TIME COMPLEXITY

The space and/or time complexity is usually expressed in the form of a function f(n); where n is the input size for a given instance of a problem being solved

Expressing time and space complexity as a function f(n) is important because of the following reason:

• We may be interested to predict the rate of growth of complexity as the size of the problem increases.

• To compare the complexities of two or more algorithms solving the same problem in order to find which is more efficient.

NOTE:

Since in modern computer, the memory is not severe constraint, therefore, our analysis of algorithms will be on the basis of time complexity.

BIG Oh NOTATION

Big Oh is a characterization scheme that allows to measure the properties of algorithms such as performance and/or memory requirements in general fashion.

The algorithm complexity can be determined ignoring the implementation dependent factor. Eliminating constant factors in the analysis of the algorithm. Basically these are the constant factor that differs from computer to computer.

Clearly, the complexity function f(n) of an algorithm increases as n increases. It is the rate of increase of f(n) that we want to examine.

BIG Oh NOTATION

Suppose , f(n) and g(n) are function defined on positive integer number n, then function f(n)=O(g(n)), read as “ f of n is big oh of g of n”, or as “ f(n) is of the order of g(n)”, if there exists positive constants c and n0 , such that

f(n)<=c x g(n) for all values of n>=n0

That is, for all sufficiently large amount of input data (n>n0), f(n) will grow no more than a constant factor than g(n).

THEOREM

If F(n)=amnm + am-1nm-1 + …+a2n2+a1n+a0 and am>0Then f(n)=O(nm)Proof:- m

f(n)<=∑|ai|ni

i=0

m

=nm∑|ai|ni-m

i=0

m

<=nm∑|ai| for n>=1

i=0

hence f(n)=O(nm)

EXAMPLES

G(n) F(n)=O(g(n))

20 O(1) i.e. constant

1/2n2+3 O(n2)

500n2-25 O(n2)

n3+n2-1 O(n3)

5n3+3n+6 O(n3)

Exact constant values do not matter and that the relationship f(n)<=c x g(n) may not hold small input sizes.

CATEGORIES OF ALGORITHMS

Based on the big Oh notation, the algorithms can be categorized as follows:

• Constant time (O(1)) algorithm.• Logarithmic time (O(logn)) algorithms.• Linear time (O(n)) algorithms.• O(n log n ) time functions• Quadratic (O(n2)) time functions• Polynomial time (O(nk), for k>1) algorithms.• Exponential time (O(kn), for k>1) algorithms

RATE OF GROWTH OF SOME STANDARD FUNCTION

f(n) Log2 n n n2 n3 2n nlog2n

n

5 3 5 25 125 32 15

10 4 10 100 1000 1000 40

100 7 100 104 106 1030 700

1000 10 103 106 109 10300 104

The logarithmic function log n grows most slowly, whereas the exponential function 2n grows most rapidly, and the polynomial function nk grows according to the exponential k.

LIMITATION OF BIG OH NOTATION

• It contains no consideration of programming effort.

• It masks (hides) potentially important constant.

For example one algorithm use 500000n2 time, and other n3 time. The first is O(n2), which implies that it will take less time than the other algorithm which is O(n3). However, second algorithm will be faster for n<500000, and this would be faster for many applications.