Upload
patrick-baker
View
238
Download
2
Embed Size (px)
Citation preview
Invitation to Computer Science 6th Edition
Chapter 3
The Efficiency of Algorithms
Invitation to Computer Science, 6th Edition
Objectives
In this chapter, you will learn about:
• Attributes of algorithms
• Measuring efficiency
• Analysis of algorithms
• When things get out of hand
2
Invitation to Computer Science, 6th Edition
Introduction
• An important part of computer science– Finding algorithms to solve problems of interest
• Are some algorithms better than others?
3
Invitation to Computer Science, 6th Edition
Attributes of Algorithms
• We expect correctness from our algorithms
• Program maintenance– Fixes errors uncovered through repeated use – Extends program to meet new requirements
• Ease of understanding– Desirable characteristic of an algorithm
• Elegance– Algorithmic equivalent of style
4
Invitation to Computer Science, 6th Edition
Attributes of Algorithms (continued)
• Efficiency– Term used to describe an algorithm’s careful use of
resources
• Benchmarks– Useful for rating one machine against another and
for rating how sensitive a particular algorithm is with respect to variations in input on one particular machine
5
Invitation to Computer Science, 6th Edition
Measuring Efficiency
• Analysis of algorithms– The study of the efficiency of algorithms– An important part of computer science
• Efficiency measured as a function relating size of input to time or space used
• For one input size, best case, worst case, and average case behavior must be considered
• The notation captures the order of magnitude of the efficiency function
6
Invitation to Computer Science, 6th Edition 7
Invitation to Computer Science, 6th Edition 8
Invitation to Computer Science, 6th Edition
Sequential Search
• Search for NAME among a list of n names
• Start at the beginning and compare NAME to each entry until a match is found
9
Invitation to Computer Science, 6th Edition
Figure 3.1 Sequential Search Algorithm
10
Invitation to Computer Science, 6th Edition 11
Invitation to Computer Science, 6th Edition 12
Invitation to Computer Science, 6th Edition
Figure 3.2 Number of Comparisons to Find NAME in a List of n Names Using Sequential Search
13
Invitation to Computer Science, 6th Edition 14
Invitation to Computer Science, 6th Edition
Order of Magnitude - Order n
• Order of magnitude n– Anything that varies as a constant times n (and
whose graph follows the basic shape of n)
• Sequential search– An Θ(n) algorithm in both the worst case and the
average case
15
Invitation to Computer Science, 6th Edition 16
Invitation to Computer Science, 6th Edition 17
Invitation to Computer Science, 6th Edition
Figure 3.3 Work = 2n
18
Invitation to Computer Science, 6th Edition
Figure 3.4 Work = cn for Various Values of c
19
Invitation to Computer Science, 6th Edition
Figure 3.5 Growth of Work = cn for Various Values of c
20
Invitation to Computer Science, 6th Edition 21
Invitation to Computer Science, 6th Edition
Selection Sort
• Selection sort algorithm– Sorts in ascending order
• Subtask within selection sort– Task of finding the largest number in a list
• When selection sort algorithm begins– The largest-so-far value must be compared to all the
other numbers in the list– If there are n numbers in the list, n – 1 comparisons
must be done
22
Invitation to Computer Science, 6th Edition
Figure 3.6 Selection Sort Algorithm
23
Invitation to Computer Science, 6th Edition
Figure 3.7 Comparisons Required by Selection Sort
24
Invitation to Computer Science, 6th Edition
Selection Sort (continued)
• Selection sort algorithm – Does n exchanges, one for each position in the list to
put the correct value in that position
• Space efficiency of the selection sort– Original list occupies n memory locations– Storage is needed for:
• The marker between the unsorted and sorted sections
• Keeping track of the largest-so-far value and its location in the list
25
Invitation to Computer Science, 6th Edition
Figure 3.8 An Attempt to Exchange the Values at X and Y
26
Invitation to Computer Science, 6th Edition
Figure 3.9 Exchanging the Values at X and Y
27
Invitation to Computer Science, 6th Edition 28
Invitation to Computer Science, 6th Edition 29
Selection Sort (continued)
Count comparisons of largest so far against other values
Find Largest, given m values, does m-1 comparisons
Selection sort calls Find Largest n times,
Each time with a smaller list of values
Cost = n-1 + (n-2) + … + 2 + 1 = n(n-1)/2
Invitation to Computer Science, 6th Edition 30
Selection Sort (continued)
Time efficiency
Comparisons: n(n-1)/2
Exchanges: n (swapping largest into place)
Overall: (n2), best and worst cases
Space efficiency
Space for the input sequence, plus a constant number of local variables
Invitation to Computer Science, 6th Edition 31
Order of Magnitude – Order n2
All functions with highest-order term cn2 have similar shape
An algorithm that does cn2 work for any constant c is order of magnitude n2, or (n2)
Invitation to Computer Science, 6th Edition 32
Order of Magnitude – Order n2
Anything that is (n2) will eventually have larger values than anything that is (n), no matter what the constants are
An algorithm that runs in time (n) will outperform one that runs in (n2)
Invitation to Computer Science, 6th Edition
Order of Magnitude - Order n2
(continued)• Order of magnitude n2, or Θ(n2)
– An algorithm that does cn2 work for any constant c
• Selection sort– An Θ(n2) algorithm (in all cases)
• Sequential search – An Θ(n) algorithm (in the worst case)
33
Invitation to Computer Science, 6th Edition
Figure 3.10 Work 5 cn2 for Various Values of c
34
Invitation to Computer Science, 6th Edition
Figure 3.11 A Comparison of n and n2
35
Invitation to Computer Science, 6th Edition
Figure 3.12 For Large Enough n, 0.25n2 Has Larger Values Than 10n
36
Invitation to Computer Science, 6th Edition
Order of Magnitude - Order n2
(continued)• Part of the job of program documentation
– To make clear any assumptions or restrictions about the input size the program was designed to handle
• Comparing algorithm efficiency – Only makes sense if there is a choice of algorithms
for the task at hand
37
Invitation to Computer Science, 6th Edition
Figure 3.13 A Comparison of Two Extreme Q(n2) and Q(n) Algorithms
38
Invitation to Computer Science, 6th Edition 39
Analysis of Algorithms
Multiple algorithms for one task may be compared for efficiency and other desirable attributes
Data cleanup problem
Search problem
Pattern matching
Invitation to Computer Science, 6th Edition
Analysis of Algorithms
Given a collection of numbers, find and remove all zeros
Possible algorithms
Shuffle-left
Copy-over
Converging-pointers
40
Invitation to Computer Science, 6th Edition
The Shuffle-Left Algorithm
• Scans list from left to right
• When a zero is found, copy each remaining data item in the list one cell to the left
• Value of legit– Originally set to the length of the list– Is reduced by 1 every time a 0 is encountered
41
Invitation to Computer Science, 6th Edition
Figure 3.14 The Shuffle-Left Algorithm for Data Cleanup
42
Invitation to Computer Science, 6th Edition
The Shuffle-Left Algorithm (continued)
• To analyze time efficiency:– Identify the fundamental units of work the algorithm
performs• Examinations
• Copying numbers– Best case occurs when the list has no 0 values
because no copying is required– Worst case occurs when the list has all 0 values
43
Invitation to Computer Science, 6th Edition 44
The Shuffle-Left Algorithm (continued)
Time efficiency Count examinations of list values and copies Best case
No copies, n examinations (n)
Worst case Shift at each pass, n passes n2 copies plus n examinations (n2)
Invitation to Computer Science, 6th Edition
The Shuffle-Left Algorithm (continued)
• Space-efficient
– n slots for n values, plus a few local variables
– Only requires four memory locations to store the quantities n, legit, left, and right
45
Invitation to Computer Science, 6th Edition
The Copy-Over Algorithm
• Scans the list from left to right, copying every legitimate (non-zero) value into a new list that it creates
• Every list entry is examined to see whether it is 0
• Every non-zero list entry is copied once
46
Invitation to Computer Science, 6th Edition
Figure 3.15 The Copy-Over Algorithm for Data Cleanup
47
Invitation to Computer Science, 6th Edition 48
The Copy-Over Algorithm
Use a second list Copy over each nonzero element in turn
Time efficiency Count examinations and copies
Best case
All zeros
n examinations and 0 copies
(n)
Invitation to Computer Science, 6th Edition 49
The Copy-Over Algorithm (continued)
Time efficiency (continued) Worst case
No zeros
n examinations and n copies
(n)
Space efficiency 2n slots for n values, plus a few extraneous variables
Invitation to Computer Science, 6th Edition 50
The Copy-Over Algorithm (continued)
Time/space tradeoff
Algorithms that solve the same problem offer a tradeoff
One algorithm uses more time and less memory
Its alternative uses less time and more memory
Invitation to Computer Science, 6th Edition
The Converging-Pointers Algorithm
• Swap zero values from left with values from right until pointers converge in the middle
51
Invitation to Computer Science, 6th Edition
Figure 3.16 The Converging-Pointers Algorithm for Data Cleanup
52
Invitation to Computer Science, 6th Edition 53
The Converging-Pointers Algorithm
Time efficiency
Count examinations and swaps
Best case
n examinations, no swaps
(n)
Invitation to Computer Science, 6th Edition 54
The Converging-Pointers Algorithm (continued)
Time efficiency (continued)
Worst case
n examinations, n swaps
(n)
Space efficiency
n slots for the values, plus a few extra variables
Invitation to Computer Science, 6th Edition
Figure 3.17 Analysis of Three Data Cleanup Algorithms
55
Invitation to Computer Science, 6th Edition
The Converging-Pointers Algorithm (continued)
• In an Θ(n) algorithm:– The work is proportional to n
• In an Θ(n2) algorithm:– The work is proportional to the square of n
56
Invitation to Computer Science, 6th Edition 57
Invitation to Computer Science, 6th Edition 58
Binary Search Algorithm
Given ordered data
Search for NAME by comparing to middle element
If not a match, restrict search to either lower or upper half only
Each pass eliminates half the data
Invitation to Computer Science, 6th Edition
Binary Search
• Procedure– First looks for NAME at roughly the halfway point in
list– If name equals NAME, search is over– If NAME comes alphabetically before name at
halfway point, search is narrowed to the front half of list
– If NAME comes alphabetically after name at halfway point, search is narrowed to the back half of the list
– Algorithm halts when NAME is found
59
Invitation to Computer Science, 6th Edition
Figure 3.18 Binary Search Algorithm (list must be sorted)
60
Invitation to Computer Science, 6th Edition
Figure 3.19 Binary Search Tree for a 7-Element List
61
Invitation to Computer Science, 6th Edition 62
Binary Search Algorithm (continued)
Time Efficiency Best case
1 comparison
(1)
Worst case
lg n comparisons lg n: The number of times n can be divided by two
before reaching 1
(lg n)
Invitation to Computer Science, 6th Edition
Binary Search (continued)
• Logarithm of n to the base 2– Number of times a number n can be cut in half and
not go below 1
• As n doubles:– lg n increases by only 1, so lg n grows much more
slowly than n
• Worst case and average case– Θ(lg n) comparisons
• Works only on a list that has already been sorted
63
Invitation to Computer Science, 6th Edition
Figure 3.20 Values for n and lg n
64
Invitation to Computer Science, 6th Edition
Figure 3.21 A Comparison of n and lg n
65
Invitation to Computer Science, 6th Edition 66
Binary Search Algorithm (continued)
Tradeoff
Sequential search
Slower, but works on unordered data
Binary search
Faster (much faster), but data must be sorted first
Invitation to Computer Science, 6th Edition 67
Invitation to Computer Science, 6th Edition
Figure 2.16 Final Draft of the Pattern-Matching Algorithm
68
Invitation to Computer Science, 6th Edition 69
Pattern-Matching Algorithm
Analysis involves two measures of input size
m: length of pattern string
n: length of text string
Unit of work
Comparison of a pattern character with a text character
Invitation to Computer Science, 6th Edition
Pattern-Matching
• Usually involves a pattern length that is short compared to the text length– That is, when m is much less than n
Time Efficiency Best case
Pattern does not match at all n - m + 1 comparisons (n)
70
Invitation to Computer Science, 6th Edition
Pattern-Matching
Time Efficiency Worst case
Pattern almost matches at each point (m)(n - m + 1) comparisons (m x n)
71
• Text: AAAAAAAAA
• Pattern: AAAB
Invitation to Computer Science, 6th Edition 72
Invitation to Computer Science, 6th Edition
Figure 3.22 Order-of-Magnitude Time Efficiency Summary
73
Invitation to Computer Science, 6th Edition 74
When Things Get Out of Hand
Polynomially bound algorithms
Work done is no worse than a constant multiple of n2
Intractable algorithms
Run in worse than polynomial time
Examples
Hamiltonian circuit
Bin-packing
Invitation to Computer Science, 6th Edition
When Things Get Out of Hand
• Graph– A collection of nodes and connecting edges
• Hamiltonian circuit– A path through a graph that begins and ends at the
same node and goes through all other nodes exactly once
75
Invitation to Computer Science, 6th Edition
Figure 3.23 Four Connected Cities
76
Invitation to Computer Science, 6th Edition
Figure 3.24 Hamiltonian Circuits among All Paths from A in Figure 3.23 with Four Links
77
Invitation to Computer Science, 6th Edition
When Things Get Out of Hand (continued)
• Exponential algorithm– An Θ(2n) algorithm
• Brute force algorithm– One that beats the problem into submission by trying
all possibilities
• Intractable problem– No polynomially bounded algorithm exists
• Approximation algorithms– Do not solve the problem, but provide a close
approximation to a solution
78
Invitation to Computer Science, 6th Edition
Figure 3.27 A Comparison of Four Orders of Magnitude
79
Invitation to Computer Science, 6th Edition
Figure 3.25 Comparisons of lg n, n, n2, and 2n
80
Invitation to Computer Science, 6th Edition
Figure 3.26 Comparisons of lg n, n, n2, and 2n for Larger Values of n
81
Invitation to Computer Science, 6th Edition 83
Invitation to Computer Science, 6th Edition
Summary of Level 1
• Level 1 (Chapters 2 and 3) explored algorithms
– Chapter 2
• Pseudocode
• Sequential, conditional, and iterative operations
• Algorithmic solutions to various practical problems
– Chapter 3
• Desirable properties for algorithms
• Time and space efficiencies of a number of algorithms
84
Invitation to Computer Science, 6th Edition
Summary
• Desirable attributes in algorithms– Correctness– Ease of understanding (clarity)– Elegance– Efficiency
• Efficiency– An algorithm’s careful use of resources is extremely
important
85