Design & Analysis of Algorithms

Introduction

Introduction

• Algorithms are the ideas behind computer programs.

• An algorithm is the thing which stays the same whether the program is in C++ running on a PC in New York or is in Android running on a Smartphone in Lahore!

• Sequence of steps that can be taken to solve a problem

• An algorithm is a sequence of unambiguous instructions for solving a problem in a finite amount of time.

• An Algorithm is well defined computational procedure that takes some value, or set of values, as input and produces some value, or set of values as output.

• More generally, an Algorithm is any well defined computational procedure that takes collection of elements as input and produces a collection of elements as output.

AlgorithmInput output

What is Algorithm?

• It must be correct.

• It must be composed of a series of concrete steps.

• The execution sequence of instructions should not be ambiguous.

• It must have finite number of instructions and steps.

• It must terminate.

4

Syntax & Semantics

An algo. is “correct” if its:– Semantics are correct– Syntax is correct

Semantics:The concept embedded in an algorithm (the soul!)

Syntax:The actual representation of an algorithm (the body!)

WARNINGS:

1. An algo. can be syntactically correct, yet semantically incorrect – very dangerous situation!

2. Syntactic correctness is easier to check as compared with semantic

Solving Problems (1)

When faced with a problem:

1.We first clearly define the problem

2.Think of possible solutions

3.Select the one that we think is the best under the prevailing circumstances

4.And then apply that solution

5. If the solution woks as desired, fine; else we go back to step 2

Early History:Search for a Generic Algorithm

• The study of algorithms began with mathematicians and was a significant area of work in the early years

• The goal of those early studies was to find a single, general algorithm that could solve all problems of a single type

Origin of the Term “Algorithm”

• The name derives from the title of a Latin book: Algoritmi de numero Indorum

• That book was a translation of an Arabic book: Al-Khwarizmi Concerning the Hindu Art of Reckoning

• That book was written by the famous 9-th century Muslim mathematician, Muhammad ibn Musa al-Khwarizmi

Al-Khwarzmi

• Al-Khwarizmi lived in Baghdad, where he worked at the Dar al-Hikma

• Dar al-Hikma acquired and translated books on science and philosophy, particularly those in Greek, as well as publishing original research

• The word Algebra has its origins in the title of another Latin book which was a translation of yet another book written by Al-Khwarzmi:

Kitab al-Mukhtasar fi Hisab al-Jabr wa'l-Muqabala

Al-Khwarizmi’s Golden PrincipleAll complex problems can be and must be solvedusing the following simple steps:

1. Break down the problem into small, simple sub-problems

2. Arrange the sub-problems in such an order that each of them can be solved without effecting any other

3. Solve them separately, in the correct order

4. Combine the solutions of the sub-problems to form the solution of the original problem

11

ProblemAlgorithmAlgorithm: A sequence of instructions describing how to do a task

• Algorithm• It is a method followed to solve a problem.• A mapping of input to output. As mentioned on

slide no. 2.• A problem can have many algorithms

• Computer program• It is a concrete representation of an algorithm in

some programming language• A set of instructions which the computer will follow t

o solve a problem

12

• Most basic and popular algorithms are– Sorting algorithms

– Searching algorithms

Which algorithm is best? • Mainly, it depends upon various factors, for

example in case of sorting – The number of items to be sorted

– The extent to which the items are already sorted

– Possible restrictions on the item values

– The kind of storage device to be used etc.

Popular Algorithms, Factors of Dependence

Problem• The statement of the problem specifies, in general

terms, the desired input/output relationship.

Algorithm• The algorithm describes a specific computational

procedure for achieving input/output relationship.

Example• One might need to sort a sequence of numbers

into non-decreasing order.

Algorithms• Various algorithms e.g. merge sort, quick sort,

heap sorts etc.

One Problem, Many Algorithms

• Brute Force– Straightforward, naive approach– Mostly expensive

• Divide-and-Conquer – Divide into smaller sub-problems

• Iterative Improvement– Improve one change at a time

• Decrease-and-Conquer– Decrease instance size

• Transform-and-Conquer– Modify problem first and then solve it

Important Designing Techniques

• Greedy Approach– Locally optimal decisions, can not change once made.– Efficient– Easy to implement– The solution is expected to be optimal– Every problem may not have greedy solution

• Dynamic programming– Decompose into sub-problems like divide and conquer– Sub-problems are dependant– Record results of smaller sub-problems– Re-use it for further occurrence– Mostly reduces complexity exponential to polynomial

There is a Tradeoffs b/w Space and Time – Use more space now to save time later

Important Designing Techniques

• Problem• Strategy• Algorithm

– Input– Output– Steps

• Analysis– Correctness– Time & Space – Optimality

• Implementation• Verification

Problem Solving Process