Contents of Chapter 1 Sections –1.1 What is an algorithm? –1.2 Algorithm specification –1.3...

전북대학교

전자정보공학부

Contents of Chapter 1

• Sections– 1.1 What is an algorithm?– 1.2 Algorithm specification– 1.3 Performance analysis– 1.4 Randomized algorithms– 1.5 References and readings

전북대학교

전자정보공학부

1.1 What is an Algorithm?

• The word algorithm– Comes from the name of a Persian author, Abu Ja’far

Mohammed ibn Musa al Khowarizmi (c. 825 A.D.)– Special significance in computer science referring to a

method that can be used by a computer for the solution of a problem

• Definition 1.1 [Algorithm]– A finite set of instructions that, if followed,

accomplishes a particular task– Satisfying the criteria:

• Input • Output• Definiteness• Finiteness• Effectiveness

전북대학교

전자정보공학부

1.1 What is an Algorithm?

• Algorithm vs. program– Program: expression of an algorithm in a programming

language– Program may never terminate (e.g, OS)

• Four distinct areas of studying algorithms– How to devise algorithms

• Art which may never be fully automated• Study of design strategies in this book is helpful

– How to validate algorithms• Algorithm validation program verification (two stages)

– How to analyze algorithms• Performance analysis on the best, worst, and average

cases– How to test a program

• Debugging (error correction) • Profiling (performance measurement)

(E. Dijkstra “debugging can only point to the presence of errors, but not to their absence”)

전북대학교

전자정보공학부

1.2 Algorithm Specification

• Algorithm vs. program– Use them interchangeably in this textbook

• Algorithm specification– Natural language – Flowchart– Programming language– (Programming language + natural language)

• Example 1.1 [selection sort]– Sort a collection of n1elements of type Type

“From those elements that are currently unsorted, find the smallest and place it next in the sorted list”

Not an algorithm

전북대학교

전자정보공학부

Selection Sort Algorithms

• Program 1.1

• Program 1.2

• Type implementation– 1) using typedef, or 2) using template

void SelectionSort(Type a[], int n) // Sort the array a[1:n] into nondecreasing order. {

for (int i=1; i<=n; i++) { int j = i; for (int k=i+1; k<=n; k++)

if (a[k]<a[j]) j=k; Type t = a[i]; a[i] = a[j]; a[j] = t;

} }

for (i=1; i<=n; i++) {

examine a[i] to a[n] and suppose

the smallest element is at a[j];

interchange a[i] and a[j];

}

전북대학교

전자정보공학부

• Recursion– Direct recursion: functions calling themselves directly

– Indirect recursion: functions calling indirectly via other functions

1.2.2 Recursive Algorithms

A(…) {

….A(…);

}

A(…) {

….B(…);

}

B(…){

….A(…);

}

전북대학교

전자정보공학부

1.2.2 Recursive Algorithms

• Example 1.2 [Tower of Hanoi]

• 2 disks– AC; AB; CB;

• 3 disks– AB; AC; BC; AB; CA; CB; AB;

• 4 disks– ?

Tower CTower A Tower B

전북대학교

전자정보공학부

1.2.2 Recursive Algorithms

• Program 1.3

#include<iostream.h>enum tower { A='A', B='B', C='C'};

void TowersOfHanoi(int n, tower x, tower y, tower z)

// Move the top n disks from tower x to tower y.

{ if (n) {

TowersOfHanoi(n-1, x, z, y);

cout << "move top disk from tower " << char(x)

<< " to top of tower " << char(y) << endl;

TowersOfHanoi(n-1, z, y, x);

}

}

전북대학교

전자정보공학부

1.2.2 Recursive Algorithms • 4 disks

– TOH(4,A,B,C)• TOH(3,A,C,B)

– TOH(2,A,B,C)» TOH(1,A,C,B) = AC;» AB;» TOH(1,C,B,A) = CB;

– AC;– TOH(2,B,C,A)

» TOH(1,B,A,C) = BA;» BC;» TOH(1,A,C,B) = AC;

• AB;• TOH(3,C,B,A)

– TOH(2,C,A,B)» TOH(1,C,B,A) = CB;» CA;» TOH(1,B,A,C) = BA;

– CB;– TOH(2,A,B,C)

» TOH(1,A,C,B) = AC;» AB;» TOH(1,C,B,A) = CB;

• Solution=AC;AB;CB;AC;BA;BC;AC;AB;CB;CA;BA;CB;AC;AB;CB

전북대학교

전자정보공학부

1.2.2 Recursive Algorithms

• Example 1.3: Permutation generator– Eg, permutation of (a,b,c,d)

• a followed by permutations of (b,c,d)• b followed by permutations of (a,c,d)• c followed by permutations of (a,b,d)• d followed by permutations of (a,b,c)

전북대학교

전자정보공학부

1.2.2 Recursive Algorithms

– Program 1.4

#include<iostream.h>

void Perm(Type a[], int k, int n) {

if (k==n) { // Output permutation. for (int i=1; i<=n; i++) cout << a[i] << " "; cout << endl;

} else // a[k:n] has more than one permutation. // Generate these recursively.

for (int i=k; i<=n; i++) { Type t=a[k]; a[k]=a[i]; a[i]=t; Perm(a, k+1, n); // All permutations of a[k+1:n] t=a[k]; a[k]=a[i]; a[i]=t;

} }

전북대학교

전자정보공학부

Programming Report

• Section 1.2 Exercises (pp.9-11)– 세 문제 이상을 선택하여 프로그래밍 (6, 7, 8, 11 중의 하나는 반드시

포함할 것 )– 제출일 : 2 주 후 ( 늦으면 하루 당 5% 씩 감점 )– 제출 방법

• report (hwp 파일 )• source code (p*.cpp 파일 )• outputs (p*.txt 파일 ) * 는 문제 번호• 모든 파일을 하나의 파일로 압축하여 TA 에게 email 로 제출 • 압축 파일명은 r1_ 학번 .zip

전북대학교

전자정보공학부

1.3 Performance Analysis

• Criteria for making evaluative judgments about software– Correctness– Documentation– Modularity– Readability– Etc

• Criteria for performance evaluation of algorithms– Definition 2.1:

• Space complexity: amount of memory the algorithm needs to run to completion

• Time complexity: amount of computer time the algorithm needs to run to completion

• Two kinds of performance evaluation– Performance analysis: a priori estimate – Performance measurement: a posteriori testing

전북대학교

전자정보공학부

1.3.2 Time Complexity

• One method– tp(n) = caADD(n) + csSUB(n) + cmMULT(n) …

– n: instance characteristic

– Impossible to estimate tp(n)

• Another method (counting the number of program steps)– Program step: syntactically or semantically meaningful

segment of a program that has an execution time being independent of the instance characteristics

– Eg, return(a+b+b*c+(a+b-c)/(a+b)+4.0);

전북대학교

전자정보공학부

– Program 1.8 (counting the steps of the function Sum)

– Each invocation of Sum executes a total of 2n+3 steps

1.3.2 Time Complexity

float Sum(float a[], int n) { float s = 0.0;

count++; // count is global for (int i=1; i<=n; i++) {

count++; // For `for' s += a[i]; count++; // For assignment

} count++; // For last time of `for' count++; // For the return return s;

}

전북대학교

전자정보공학부

– Program 1.10 (counting the steps of the function RSum)

– tRSum(n) = 2 if n=0,

2+tRSum(n-1) if n>0

– tRSum(n) = 2+tRSum(n-1)

= 2+2+tRSum(n-2) …. = 2n+2

1.3.2 Time Complexity

float RSum(float a[], int n) { count++; // For the if conditional

if (n <= 0) { count++; // For the return return 0.0;

} else { count ++; // For the addition, function // invocation and return return (RSum(a, n-1) + a[n]);

} }

전북대학교

전자정보공학부

1.3.2 Time Complexity

• Sum is linear time algorithm– 2n+2– If n is doubled, the run time also doubles approximately.– “The time complexity of Sum is linear in the input size

n.”

• Definition 1.3 [input size]– One of the instance characteristics – Number of elements needed to describe the instance– Example 1.9 (matrix addition)

• Program 1.11

• Input size = 2mn+2• Step count = 2mn+2m+1

void Add(Type a[][SIZE], Type b[][SIZE], Type c[][SIZE], int m, int n)

{ for (int i=1; i<=m; i++)

for (int j=1; j<=n; j++)

c[i][j] = a[i][j] + b[i][j];

}

전북대학교

전자정보공학부

1.3.2 Time Complexity

• Summary of time complexity– Sum, Rsum, Add is very simple examples– Not solely dependent on input sizes, but dependent on

actual values of inputs • Eg, sequential searching algorithm

– Best case: 1 comparison (query is the same as the first element)

– Worst case: n+1 comparisons (query is the same as the last element)

– Average case: n/2 – Which one is faster between n2+2n and 100n ?

전북대학교

전자정보공학부

1.3.3 Asymptotic Notation

• Definition 1.4 [Big “oh”]– f(n)=O(g(n)) iff there exist positive constants c and n0

such that f(n)c*g(n) for all n, nn0.

– Example 1.11• 3n+2 = O(n) (because 3n+24n for all n2)• 3n+3 = O(n) ?• 100n+6 = O(n) ?• 10n2+4n+2 = O(n2) ?• 1000n2+100n-6 = O(n2) ?• 6*2n+n2 = O(2n) ?

• 3n+3 = O(n2) ?• 10n2+4n+2 = O(n) ?

전북대학교

전자정보공학부

1.3.3 Asymptotic Notation

• Comparison of speed– O(1): constant– O(log n)– O(n): linear– O(n log n)– O(n2): quadratic– O(n3): cubic– O(2n): exponential

• In f(n)=O(g(n)), – g(n) is an upper bound of the value of f(n) for all n,

nn0.

– So, 3n+3=O(n2), 3n+3=O(2n) are correct– But, we say 3n+3=O(n) (almost never say 3n+3=O(n2),

3n+3=O(2n))

• Theorem 1.2– If f(n)=amnm +…+ a1n + a0, then f(n)=O(nm)

faster

전북대학교

전자정보공학부

1.3.3 Asymptotic Notation

• Definition 1.5 (Omega)– f(n)=(g(n)) iff there exist positive constants c and n0

such that f(n)c*g(n) for all n, nn0.

– Example 1.12• 3n+2 = (n) (because 3n+23n for all n1)• 3n+3 = (n) ?• 100n+6 = (n) ?• 10n2+4n+2 = (n2) ?• 1000n2+100n-6 = (n2) ?• 6*2n+n2= (2n) ?• 3n+3 = (n2) ?• 10n2+4n+2 = (n) ?

• In f(n)=(g(n)), – g(n) is a lower bound of the value of f(n) for all n, nn0.

– So, 3n+3= (1) is correct– But, we say 3n+3= (n) (almost never say 3n+3= (1))

전북대학교

전자정보공학부

1.3.3 Asymptotic Notation

• Definition 1.6 (Theta)– f(n)=(g(n)) iff there exist positive constants c1, c2, and

n0 such that c1*g(n) f(n)c2*g(n) for all n, nn0.

– Example 1.13• 3n+2 = (n)• 3n+3 = (n) ?• 100n+6 = (n) ?• 10n2+4n+2 = (n2) ?• 1000n2+100n-6 = (n2) ?• 6*2n+n2= (2n) ?• 3n+3 = (n2) ?• 10n2+4n+2 = (n) ?

• In f(n)=(g(n)), – g(n) is both an upper and lower bound of the value of

f(n) for all n, nn0.

– So, 3n+3=O(1) is not correct

전북대학교

전자정보공학부

1.3.3 Asymptotic Notation

• Example 1.17 [Magic square]– Figure 1.2

11182529

101219213

46132022

23571416

17241815

전북대학교

전자정보공학부

– Program 1.15

#include<iostream.h>#include<iomanip.h> void Magic(int n) // Create a magic square of size n, n being odd.

{ int square[SIZE][SIZE];

if (!(n%2)) {cout << "n is even" << endl; return;}

else {

for (int i=0; i<n; i++) // Initialize square to zero.

for (int j=0; j<n; j++) square[i][j] = 0;

square[0][(n-1)/2]=1; // Middle of first row

// (i, j) is the current position.

int j = (n-1)/2, k, l;

for (int key = 2; key <= n*n; key++) {// Move up and left. The next two if statements

// may be replaced by the % operator if -1%n is

전북대학교

전자정보공학부

– Time complexity: (n2)

// implemented to have value n-1.

k = (i) ? (i-1) : (n-1);

l = (j) ? (j-1) : (n-1);

if (square[k][l]) i = (i+1)%n;

else { // square[k][l] is empty.

i = k; j = l; }

square[i][j] = key; }

// Output the magic square.

for (i=0; i<n; i++)

for (j=0; j<n; j++) cout << setw(5) << square[i][j];

cout << endl; }

} }

전북대학교

전자정보공학부

1.3.3 Asymptotic Notation

• Example 1.18 [Computing xn]– Program 1.16

– Time complexity: (log n)

Type Exponentiate(Type x, int n) // Return xn for an integer n>=0 {

int m=n; Type power=1, z=x; while (m>0) {

while (!(m%2)) { m/=2; z*=z;

} m--; power*=z;

} return power;

}

전북대학교

전자정보공학부

1.3.4 Practical Complexities

• When program P has (n) and Q has (n2), P is faster than Q for sufficiently large n.

• Table 1.7

전북대학교

전자정보공학부

1.3.4 Practical Complexities

• Figure 1.3

전북대학교

전자정보공학부

1.3.4 Practical Complexities

• Table 1.8

전북대학교

전자정보공학부

1.3.5 Performance Measurement

• Time functions in C++#include <time.h>…

T(){

clock_t start_time, end_time; …..

start_time = clock(); …… /* program segment to measure the time

*/ end_time = clock(); printf("time elapsed=%f\n",

(float)(end_time-start_time)/CLOCKS_PER_SEC); ….}

전북대학교

전자정보공학부

1.3.5 Performance Measurement

• To time a short event,– Repeat it several times and divide the total time for the

event by the number of repetitions.

• Program 1.19

void TimeSearch()

{ // Repetition factors int r[21] = {0, 200000, 200000, 150000, 100000, 100000, 100000, 50000, 50000, 50000, 50000, 50000, 50000, 50000, 50000, 50000, 50000, 25000, 25000, 25000, 25000}; int a[1001], n[21]; for (int j=1; j<=1000; j++) a[j]=j; for (j=1; j<=10; j++) {

n[j]= 10*(j-1); n[j+10]=100*j; } cout << " n t1 t" << endl << endl; cout << setiosflags (ios::fixed); cout << setprecision(3);

전북대학교

전자정보공학부

1.3.5 Performance Measurement

for (j=1; j<=20; j++) {

int h = GetTime();

for (int i=1; i<=r[j]; i++)

int k = SeqSearch(a, 0, n[j]);

int h1 = GetTime();

int t1 = h1 - h;

float t = t1; t /= r[j];

cout << setw(5) << n[j] << << setw(5) << t1 << << setw(8)

<< t << endl;

}

cout << "Times are in milliseconds" << endl;

}

전북대학교

전자정보공학부

1.3.5 Performance Measurement

• Improper timing– Programs 1.20

float t = 0;

for (int i=1; i <= r[j]; i++) {

int h = GetTime();

int k = SeqSearch(a, 0, n[j]);

int h1 = GetTime();

t += h1 - h;

}

t /= r[j];

전북대학교

전자정보공학부

1.3.5 Performance Measurement

• Improper timing– Programs 1.21

int t = 0;

while (t < DESIRED_TIME) { int h = GetTime(); int k = SeqSearch(a, 0, n[j]) int h1 = GetTime(); t += h1 - h;

}

전북대학교

전자정보공학부

1.3.5 Performance Measurement

• Proper timing– Program 1.22

int h = GetTime(), t = 0;

while (t < DESIRED_TIME) {

int k = SeqSearch(a, 0, n[j]);

int h1 = GetTime();

t = h1 - h;

}