Chapter 3 The Efficiency of Algorithms

國立雲林科技大學 資訊工程研究所張傳育 (Chuan-Yu Chang ) 博士Office: ES 709TEL: 05-5342601 ext. 4337E-mail: chuanyu@yuntech.edu.tw

2

Properties of Algorithms

Necessary properties A well-ordered collection of unambiguous

and effectively computable operations that,

when, executed, produces a result and halts

in a finite amount of time.

What about desirable properties? Correctness

3

Attributes of Algorithms Correctness：

First, make it correct! Correct results may not be as straightforward

as it seems. May be giving correct results to the wrong problem

Second, the algorithm must give correct results for all possible input values

Third, Issue of accuracy of the result we are willing to accept as correct.

Attribute of an algorithm:

--Ease of understanding. --Clarity

--Ease of handling --Elegance (style)

--Efficiency

4

Example: adding 1+2+3+…+100 Solution 1:

Step 1: Set the value of sum to 0 Step 2: Set the value of x to 1 Step 3: While x is less than or equal to 100 do step 4 and 5 Step 4: Add x to sum Step 5: Add 1 to the value of x Step 6: Print the value of sum Step 7: Stop

Solution 2: Gauss’s Method 1+100=101, 2+99=101,…,50+51=101 50*101=5050

Solution 3: 梯形公式 (100/2)(100+1)=5050

聰明但不容易理解

不同的方法 (elegant)

5

Attributes of Algorithms (cont.) 一個演算法最好能同時具備容易理解 (ease of un

derstanding) 及優雅的形式 (elegant) 。 The Limited Resources

電腦科學家必須留意演算法所使用的資源 (Time & Spa

ce ) 。 Time and Space are not unlimited resources

“ Efficiency ” is the term used to describe an algorithm’s careful use of resources.

6

Measure Efficient Algorithm

How can we measure the time efficiency of an algorithm? Kinds of computer to be used ? Data from ? Data organization ?

Benchmark Use the same input data and running an algorithm on

different machines gives a comparison of machine speeds on identical tasks.

7

Formal Definition

Algorithm’s time efficiency An indication of amount of “work” required by the

nature of the algorithm itself and the approach it uses.

Count the number of instruction executions

It is the number of steps each algorithm requires, not the time the algorithm takes on a particular machine, that is important for comparing two algorithms that do the same task.

8

Data Cleanup Problem

We want to perform a “data cleanup” and remove the 0 entries from the list before the average is computed.

0 24 16 0 36 42 23 21 0 27 24 16 36 42 23 21 27

Legit: legitimate element

The value 0 will be removed

9

A Choice of Algorithms

The shuffle- left algorithm (Figure 3.1) 左手保存目前位置，且忽略非零值，當遇到 0 值時，以右

手邊第一個非零值取代之，且 0 右邊的所有值均左移一位。 The copy-over algorithm (Figure 3.2)

將每個 legitimate 值，複製到一個新的串列。 The converging-pointers algorithm (Figure 3.3)

左手指到串列的最左端，右手指到串列的最右端，當左手遇到非零值時，則往右移位；當遇到零值時，則以右手所指到的非零值來取代，右手往左移一位。

10

Figure 3.1 The shuffle- left algorithm 一開始 legit 的值設定成串列長度，當遇到一次

0 值，則 legit 減 1 。

0 右邊的所有項都被往左複製一次

11

Figure 3.2 The copy-over algorithm

12

Figure 3.3 The converging-pointers algorithm

13

Comparisons

可用 time- 和 space-efficient 來衡量？ Case by case

14

Measuring Efficiency Sequential search (Figure 3.1)

--Best Case ： 1 --Worst Case ： n --Average Case ： n/2

Usually interested not in the behavior of an algorithm on little problems but in its behavior as the size of a problem (n) gets vary large. In the New York City telephone directory, n may be large a

s 20000000. If the sequential search algorithm were executed on a computer that could do 50000 comparisons per seconds, it would require on the average about (20000000/2)*(1/50000)=200 sec

15

Figure 3.4 Sequential Search Algorithm

Peripheraltask

16

Order of Magnitude

The worst case behavior of the sequential search algorithm on a list n names requires n comparisons. If c is a constant factor representing the peripheral work, it requires cxn total work.

對於大的 n ，常數項 c 可忽略。 Order of magnitude n is written Θ(n) Sequential search is therefore an Θ(n) algorithm

in both worst case and average case.

17

Work = 2n for Various Values of c (figure 3.6)

以 sequential search 為例： n 筆資料，最多需要 n 次的比

較。 假設其週邊工作為 c 則需要 cn 的工作量。 右圖為 c=2 時，資料筆數 n 和

總工作量之間的關係圖。

18

Work = cn for Various Values of c (figure 3.7)

19

Work = cn for Various Values of c (figure 3.8)

20

Calling Information To write the calling information table out.

Pseudocode For each of rows 1 through 4 do the following

For each of columns 1 through 4 do the following Write the entry in this row and column

The number of write operations is 4x4=16 If there are n districts, the total works are nxn=n2.

1 2 3 4

1 243 187 314 244

2 215 420 345 172

3 197 352 385 261

4 340 135 217 344

21

A Comparison of n and n2 (Figure 3.10)

22

A comparison of n and n2 (figure 3.11)

23

Figure 3.12For large enough n, 0.25n2 has larger values than 10n

24

Figure 3.13 A Comparison of two extreme O(n2) and O(n) Algorithm

25

The tortoise and the hare Pentium Pro 200 具有 75 megaflops ，成本 $2000 Cray T3E 900 具有 670 gigaflops ，成本 $31 millions

n Pentium Pro O(n) Cray O(n2)

750 0.00001 0.00000084

7500 0.0001 0.000084

75000 0.001 0.0084

750000 0.01 0.84

7500000 0.1 84

75000000 1 8400sec =2.3hr

750000000 10 840000sec=9.7day

26

Analysis of Algorithms (data cleanup)

Shuffle- left Copy- over Converging

pointer

Time Space Time Space Time Space

Best Case O(n) n O(n) n O(n) n

Worst

Case

O(n2

) n O(n) 2n O(n) n

Average

Case

O(n2

) n O(n) n x ≦

2n≦O(n) n

Analysis of three data clean up algorithm

27

Analysis of Algorithms (Cont.)

Sort Selection Sort best case: n2, worst case: n2, average case: n2

28

Analysis of Algorithms (Cont.)

Selection Sort Ex:

Pass 0: 5 7 2 8 3 Pass 1: 5 7 2 3 8 Pass 2: 5 3 2 7 8 Pass 3: 2 3 5 7 8 Pass 4: 2 3 5 7 8

4321

比較次數

n-1n-2:1

比較總次數 =n(n-1)/2

比較總次數 =10

29

Exchange the value of X and Y 1. Copy the current

value at position Y into position X

2. Copy the current value at position X into position Y

Step 1

Step 2

30

Exchange the value of X and Y (cont.) 1. Copy the current valu

e at position X into position T

2. Copy the value at position Y into position X

3. Copy the current value at position T into position Y

Step 2

Step 3

Step 1

31

Binary Search Binary Search

The list being searched is already sorted. Search the midpoint of the list. best case: 1, worst case: (log n), average case: (log n)

32

Binary Search Tree

The basic shapes of n and log n

log n grows much more slowly than n

33

Pattern Matching

Finding all occurrences of a pattern of the form P1P2…Pm within text of the form T1T2…Tn.

Forward match At each position beginning an attempt to match each pattern c

haracter against the text characters. The process stops only after text position n-m+1 (n the length of the text string, m the length of the pattern strin

g) The best case: if the first character of the pattern is nowhere i

n the text. ( require n-m+1 comparisons) The worst case 1: if the pattern almost occurs everywhere in t

he text. (require nxm comparisons=> O(mxn)) The worst cases 2: the pattern is found at each location in the

text. (require nxm comparisons => O(mxn))

34

When Things Get Out Of Hand Polynomially bounded

Order of magnitude determines how quickly the values grow as n increases.

The work done by any of these algorithms is no worse than a constant multiple of n2, which is polynomial in n.

Non-polynomially bounded Is it possible to start at city A, go through every cit

y exactly once, and end up at A? A collection of nodes and connecting edges is call

ed a graph

35

When Things Get Out Of Hand (cont.) Hamiltonian circuit

A path through a graph that begins and ends at the same node and goes through all other nodes exactly once.

If there are n node in the graph, then a Hamiltonian circuit, if it exists, must have exactly n links.

A B

C D

36

When Things Get Out Of Hand (cont.)

The number of paths that must be examined is the number of nodes at the bottom level of the tree.

The bottom of the corresponding tree would be at level n, and there would be 2n paths to examine.

An O(2n) algorithm is called an exponential algorithm

37

When Things Get Out Of Hand (cont.) Exponential algorithm

Comparisons of log n, n, n2 and 2n

38

When Things Get Out Of Hand (cont.)

Comparisons of four orders of magnitude 假設單一指令的執行時間為 0.0001 sec ，則不同 order

所需執行時間，如下表所示：

Order 10 50 100 1000

log n 0.0003s 0.0006 s 0.0007 s 0.001 s

n 0.001 s 0.005 s 0.01 s 0.1 s

n2 0.01 s 0.25 s 1 s 1.67min

2n 0.1024s 3570 years 4*1016 centuries

Too big to compute

39

Intractable problem No polynomially bounded algorithm to solve. They are solvable, but the solution algorithms all require so

much work as to be virtually useless. bin-packing problem

Given an unlimited number of bins of volume 1 unit, and given n objects, all of volume between 0.0 and 1.0, find the minimal number of bins needs to store the n objects.

( 只能找出 Approximation algorithm)