Upload
amdileeyas
View
230
Download
0
Embed Size (px)
Citation preview
7/27/2019 04 Iterative Algorithms
1/177
Central Algorithmic Techniques
Iterative Algorithms
7/27/2019 04 Iterative Algorithms
2/177
COSC 3101, PROF. J. ELDER 2
CodeRepresentation of an Algorithm
class InsertionSortAlgorithm extends SortAlgorithm {
void sort(int a[]) throws Exception {
for (int i = 1; i < a.length; i++) {int j = i;
int B = a[i];
while ((j > 0) && (a[j-1] > B)) {
a[j] = a[j-1];
j--; }
a[j] = B;
}}
Pros and Cons?
7/27/2019 04 Iterative Algorithms
3/177
COSC 3101, PROF. J. ELDER 3
CodeRepresentation of an Algorithm
Runs on computers
Precise and succinct
I am not a computer
I need a higher level of intuition.
Prone to bugs
Language dependent
Pros: Cons:
7/27/2019 04 Iterative Algorithms
4/177
COSC 3101, PROF. J. ELDER 4
Two Key Types of Algorithms
Iterative Algorithms
Recursive Algorithms
7/27/2019 04 Iterative Algorithms
5/177
COSC 3101, PROF. J. ELDER 5
Iterative Algorithms
Take one step at a time
towards the final destination
loop (done)
take step
end loop
7/27/2019 04 Iterative Algorithms
6/177
COSC 3101, PROF. J. ELDER 6
Loop Invariants
A good way to structure many programs:
Store the key information you currently know in somedata representation.
In the main loop,
take a step forward towards destination
by making a simple change to this data.
7/27/2019 04 Iterative Algorithms
7/177
COSC 3101, PROF. J. ELDER 7
The Getting to School Problem
7/27/2019 04 Iterative Algorithms
8/177
COSC 3101, PROF. J. ELDER 8
Problem Specification Pre condition: location of home and school
Post condition: Traveled from home to school
7/27/2019 04 Iterative Algorithms
9/177
COSC 3101, PROF. J. ELDER 9
General Principle Do not worry about the entire computation.
Take one step at a time!
7/27/2019 04 Iterative Algorithms
10/177
COSC 3101, PROF. J. ELDER 10
A Measure of Progress
79 km
to school
75 kmto school
7/27/2019 04 Iterative Algorithms
11/177
COSC 3101, PROF. J. ELDER 11
Algorithm specifies from which locations
it knows how to step.
Safe Locations
7/27/2019 04 Iterative Algorithms
12/177
COSC 3101, PROF. J. ELDER 12
The computation is presently in a safe location.
May or may not be true.
Loop Invariant
7/27/2019 04 Iterative Algorithms
13/177
COSC 3101, PROF. J. ELDER 13
Defining Algorithm From every safe location,
define one step towards school.
7/27/2019 04 Iterative Algorithms
14/177
COSC 3101, PROF. J. ELDER 14
Take a step What is required of this step?
7/27/2019 04 Iterative Algorithms
15/177
COSC 3101, PROF. J. ELDER 15
Can we be assured that the computationwill always be in a safe location?
If the computation is in a safe location,it does not step into an unsafe one.
Maintain Loop Invariant
No. What if it is not initially true?
7/27/2019 04 Iterative Algorithms
16/177
COSC 3101, PROF. J. ELDER 16
From the Pre-Conditions on the input instancewe must establish the loop invariant.
Establishing Loop Invariant
7/27/2019 04 Iterative Algorithms
17/177
COSC 3101, PROF. J. ELDER 17
Maintain Loop Invariant
Can we be assured that the
computation will always bein a safe location?
By what principle?
7/27/2019 04 Iterative Algorithms
18/177
COSC 3101, PROF. J. ELDER 18
Maintain Loop Invariant By Induction the computation will
always be in a safe location.
(0)
, ( )
, ( ) ( 1)
S
i S i
i S i S i
7/27/2019 04 Iterative Algorithms
19/177
COSC 3101, PROF. J. ELDER 19
Ending The Algorithm Define Exit Condition
Termination: With sufficient progress,
the exit condition will be met.
When we exit, we know
exit condition is true
loop invariant is true
from these we must establish
the post conditions.
Exit
Exit
0 km Exit
7/27/2019 04 Iterative Algorithms
20/177
COSC 3101, PROF. J. ELDER 20
Lets Recap
7/27/2019 04 Iterative Algorithms
21/177
COSC 3101, PROF. J. ELDER 21
Designing an AlgorithmDefine Problem Define Loop Invariants Define Measure of
Progress
Define Step Define Exit Condition Maintain Loop Inv
Make Progress Initial Conditions Ending
km
79 km
to school
Exit
Exit
79 km 75 km
Exit
Exit
0 km Exit
7/27/2019 04 Iterative Algorithms
22/177
Simple Example
Insertion Sort Algorithm
7/27/2019 04 Iterative Algorithms
23/177
COSC 3101, PROF. J. ELDER 23
CodeRepresentation of an Algorithm
class InsertionSortAlgorithm extends SortAlgorithm {
void sort(int a[]) throws Exception {
for (int i = 1; i < a.length; i++) {
int j = i;
int B = a[i];
while ((j > 0) && (a[j-1] > B)) {
a[j] = a[j-1];
j--; }
a[j] = B;
}}
7/27/2019 04 Iterative Algorithms
24/177
COSC 3101, PROF. J. ELDER 24
Higher Level Abstract ViewRepresentation of an Algorithm
9 km
5 km
7/27/2019 04 Iterative Algorithms
25/177
COSC 3101, PROF. J. ELDER 25
Designing an AlgorithmDefine Problem Define Loop Invariants Define Measure of
Progress
Define Step Define Exit Condition Maintain Loop Inv
Make Progress Initial Conditions Ending
km
79 km
to school
Exit
Exit
79 km 75 km
Exit
Exit
0 km Exit
7/27/2019 04 Iterative Algorithms
26/177
COSC 3101, PROF. J. ELDER 26
Problem Specification Precondition: The input is a list of n values
with the same value possibly repeated. Post condition: The output is a list consistingof the same n values in non-decreasing order.
88 149825
62
52
79
3023
31 14,23,25,30,31,52,62,79,88,98
7/27/2019 04 Iterative Algorithms
27/177
COSC 3101, PROF. J. ELDER 27
88 14
9825 62
52
79
3023
31 14,23,25,30,31,52,62,79,88,98
14
9825 62
79
3023,31,52,88
Some subset of the elements are sorted
The remaining elements are off to the side.
Define Loop Invariant
7/27/2019 04 Iterative Algorithms
28/177
COSC 3101, PROF. J. ELDER 28
Defining Measure of Progress
14
9825 62
79
3023,31,52,88
6 elements
to school
7/27/2019 04 Iterative Algorithms
29/177
COSC 3101, PROF. J. ELDER 29
Define Step
Select arbitrary element from side.
Insert it where it belongs.
9825 62
79
23,31,52,88
14
9825
79
3023,31,52, 62 ,88
1430
7/27/2019 04 Iterative Algorithms
30/177
COSC 3101, PROF. J. ELDER 30
Making progress while Maintaining theloop invariant
9825 62
79
23,31,52,88
14
98
25
79
3023,31,52, 62 ,88
1430
79 km 75 km
5 elements
to school
6 elements
to school
Exit
Sorted sublist
B i i &
7/27/2019 04 Iterative Algorithms
31/177
COSC 3101, PROF. J. ELDER 31
88 14
9825 62
52
79
3023
31
88 14
98
25 62
52
79
3023
31 n elementsto school
14,23,25,30,31,52,62,79,88,98
14,23,25,30,31,52,62,79,88,980 elements
to school
Beginning &Ending
km Exit0 km Exit
7/27/2019 04 Iterative Algorithms
32/177
COSC 3101, PROF. J. ELDER 32
Running Time
Inserting an element into a list of size itakes (i) time.
Total = 1+2+3++n = (n 2)
9825 62
79
23,31,52,88
14
9825
79
3023,31,52, 62 ,88
1430
7/27/2019 04 Iterative Algorithms
33/177
COSC 3101, PROF. J. ELDER 33
OkI know you knew Insertion Sort
But hopefully you are beginning to appreciate
Loop Invariants
for describing algorithms
7/27/2019 04 Iterative Algorithms
34/177
Assertions
in Algorithms
7/27/2019 04 Iterative Algorithms
35/177
COSC 3101, PROF. J. ELDER 35
Purpose of Assertions
Useful for
thinking about algorithms developing
describing proving correctness
7/27/2019 04 Iterative Algorithms
36/177
COSC 3101, PROF. J. ELDER 36
Definition of Assertions
An assertion is a statement about thecurrent state of the data structure that is
either true or false.
eg. the amount in your bank account is notnegative.
7/27/2019 04 Iterative Algorithms
37/177
COSC 3101, PROF. J. ELDER 37
Definition of Assertions
It is made at some particular point during theexecution of an algorithm.
If it is false, then something has gone wrong in thelogic of the algorithm.
7/27/2019 04 Iterative Algorithms
38/177
COSC 3101, PROF. J. ELDER 38
Definition of Assertions
An assertion is not a task for the algorithm toperform.
It is only a comment that is added for thebenefit of the reader.
7/27/2019 04 Iterative Algorithms
39/177
COSC 3101, PROF. J. ELDER 39
Specifying a Computational Problem
Example of Assertions
Preconditions: Any assumptions that must betrue about the input instance.
Postconditions: The statement of what must
be true when the algorithm/program returns..
7/27/2019 04 Iterative Algorithms
40/177
COSC 3101, PROF. J. ELDER 40
Definition of Correctness
&
If the input meets the preconditions,
then the output must meet the postconditions.
If the input does not meet the preconditions, thennothing is required.
7/27/2019 04 Iterative Algorithms
41/177
COSC 3101, PROF. J. ELDER 41
An Example: A Sequence of Assertions
if( ) then
code else
code end if
if( ) then
code else
code end if
any code
How is this proved?
Definition of Correctness
Must check 2 r differentsettings of paths through the code.
Is there a faster way?
7/27/2019 04 Iterative Algorithms
42/177
COSC 3101, PROF. J. ELDER 42
An Example: A Sequence of Assertions
if( ) then
code else
code end if
if( ) then
code else
code end if
Step 1
code
code
7/27/2019 04 Iterative Algorithms
43/177
COSC 3101, PROF. J. ELDER 43
An Example: A Sequence of Assertions
if( ) then
code else
code end if
if( ) then
code else
code end if
Step 2
code
code
7/27/2019 04 Iterative Algorithms
44/177
COSC 3101, PROF. J. ELDER 44
An Example: A Sequence of Assertions
if( ) then
code else
code end if
if( ) then
code else
code end if
Step r
code
code
A S f A i
7/27/2019 04 Iterative Algorithms
45/177
COSC 3101, PROF. J. ELDER 45
A Sequence of Assertions
if( ) then
code else
code end if
if( ) then
code else
code end if
Step r
code
code
7/27/2019 04 Iterative Algorithms
46/177
7/27/2019 04 Iterative Algorithms
47/177
COSC 3101, PROF. J. ELDER 47
codeA loop
exit when codeBendloop codeC
Definition of Correctness?
Iterative AlgorithmsLoop Invariants
7/27/2019 04 Iterative Algorithms
48/177
COSC 3101, PROF. J. ELDER 48
codeA loop
exit when codeBendloop codeC
Definition of Correctness any code
How is this proved?
Iterative AlgorithmsLoop Invariants
7/27/2019 04 Iterative Algorithms
49/177
COSC 3101, PROF. J. ELDER 49
codeA loop
exit when
codeBendloop codeC
The computation may go aroundthe loop an arbitrary number of times.
any code
Is there a faster way?
Iterative AlgorithmsLoop Invariants
Definition of Correctness
7/27/2019 04 Iterative Algorithms
50/177
COSC 3101, PROF. J. ELDER 50
codeA loop
exit when codeBendloop codeC
Step 0codeA
7/27/2019 04 Iterative Algorithms
51/177
COSC 3101, PROF. J. ELDER 51
codeA loop
exit when codeBendloop codeC
Step 1 codeB
7/27/2019 04 Iterative Algorithms
52/177
COSC 3101, PROF. J. ELDER 52
codeA loop
exit when codeBendloop codeC
Step 2 codeB
7/27/2019 04 Iterative Algorithms
53/177
COSC 3101, PROF. J. ELDER 53
codeA loop
exit when codeBendloop codeC
Step 3 codeB
7/27/2019 04 Iterative Algorithms
54/177
COSC 3101, PROF. J. ELDER 54
codeA loop
exit when codeBendloop codeC
Step i codeB
7/27/2019 04 Iterative Algorithms
55/177
COSC 3101, PROF. J. ELDER 55
codeA loop
exit when codeBendloopcodeC
Last Step codeC
Iterative AlgorithmsLoop Invariants
Partial Correctness
7/27/2019 04 Iterative Algorithms
56/177
COSC 3101, PROF. J. ELDER 56
Partial Correctness
Proves that IF the program terminates then it works
&
codeC
Clean up loose ends
codeB
Maintaining Loop Invariant
codeA
Establishing Loop Invariant
Exit
Exit
7/27/2019 04 Iterative Algorithms
57/177
COSC 3101, PROF. J. ELDER 57
Algorithm Termination
km
79 km 75 km
Measure of progress
0 km
Exit
7/27/2019 04 Iterative Algorithms
58/177
COSC 3101, PROF. J. ELDER 58
Algorithm Correctness
Partial Correctness+ Termination Correctness
7/27/2019 04 Iterative Algorithms
59/177
7/27/2019 04 Iterative Algorithms
60/177
COSC 3101, PROF. J. ELDER 60
Don t start coding
You must design a working algorithm first.
7/27/2019 04 Iterative Algorithms
61/177
7/27/2019 04 Iterative Algorithms
62/177
COSC 3101, PROF. J. ELDER 62
Start with Small Steps
What basic steps might you follow to make some kind of progress towards the answer?
Describe or draw a picture of what the data structure mightlook like after a number of these steps.
7/27/2019 04 Iterative Algorithms
63/177
COSC 3101, PROF. J. ELDER 63
Picture from the Middle
Leap into the middle of the algorithm.
What would you like your data structure to look likewhen you are half done?
7/27/2019 04 Iterative Algorithms
64/177
COSC 3101, PROF. J. ELDER 64
Ask for 100%
Pretend that a genie has granted your wish. You are now in the middle of your computation and your
dream loop invariant is true.
7/27/2019 04 Iterative Algorithms
65/177
COSC 3101, PROF. J. ELDER 65
Ask for 100%
Maintain the Loop Invariant: From here, are you able to take some computational steps that
will make progress while maintaining the loop invariant?
7/27/2019 04 Iterative Algorithms
66/177
COSC 3101, PROF. J. ELDER 66
Ask for 100%
If you can maintain the loop invariant, great. If not,
Too Weak: If your loop invariant is too weak, then the genie hasnot provided you with everything you need to move on.
Too Strong: If your loop invariant is too strong, then you will notbe able to establish it initially or maintain it.
7/27/2019 04 Iterative Algorithms
67/177
COSC 3101, PROF. J. ELDER 67
Differentiating between Iterations
x=x+2 Meaningful as code
False as a mathematical statement
x = x i = value at the beginning of the iterationx = x i+1 = new value after going around the loop one more time.
x'' = x'+2
Meaningful as a mathematical statement
7/27/2019 04 Iterative Algorithms
68/177
Loop Invariantsfor
Iterative Algorithms
Three
Search Examples
7/27/2019 04 Iterative Algorithms
69/177
COSC 3101, PROF. J. ELDER 69
Define Problem: Binary Search
PreConditions Key 25
Sorted List
PostConditions Find key in list (if there).
3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95
3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95
7/27/2019 04 Iterative Algorithms
70/177
COSC 3101, PROF. J. ELDER 70
Define Loop Invariant
Maintain a sublist.
If the key is contained in the original list, then the key iscontained in the sublist.
key 25
3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95
7/27/2019 04 Iterative Algorithms
71/177
COSC 3101, PROF. J. ELDER 71
Define Step
Make Progress
Maintain Loop Invariant
key 25
3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95
7/27/2019 04 Iterative Algorithms
72/177
COSC 3101, PROF. J. ELDER 72
Define Step
Cut sublist in half.
Determine which half the key would be in.
Keep that half.
key 25
3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95
If key mid ,then key is inleft half.
If key > mid ,then key is inright half.
mid
7/27/2019 04 Iterative Algorithms
73/177
COSC 3101, PROF. J. ELDER 73
Define Step
It is faster not to check if the middle element is the key.
Simply continue.
key 43
3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95
If key mid ,then key is inleft half.
If key > mid ,then key is inright half.
7/27/2019 04 Iterative Algorithms
74/177
COSC 3101, PROF. J. ELDER 74
Make Progress
The size of the list becomes smaller.
3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95
3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95
79 km
75 km
79 km 75 km
Exit
7/27/2019 04 Iterative Algorithms
75/177
COSC 3101, PROF. J. ELDER 75
Initial Conditionskm
key 25
3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95
If the key is contained in theoriginal list,
then the key is contained in thesublist.
The sublist is theentire original list.
n km
7/27/2019 04 Iterative Algorithms
76/177
COSC 3101, PROF. J. ELDER 76
Ending Algorithm
If the key is contained in theoriginal list,
then the key is contained in thesublist.
Sublist contains one element.
Exit
Exit
3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95
0 km
If the key iscontained in theoriginal list,then the key is atthis location.
key 25
7/27/2019 04 Iterative Algorithms
77/177
COSC 3101, PROF. J. ELDER 77
If key not in original list
If the key is contained in theoriginal list,
then the key is contained in thesublist.
Loop invariant true,even if the key is notin the list.
If the key is contained inthe original list, then thekey is at this location.
3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95
key 24
Conclusion still solves the problem.Simply check this one locationfor the key.
7/27/2019 04 Iterative Algorithms
78/177
BinarySea rch(A[1..n], )e k y
7/27/2019 04 Iterative Algorithms
79/177
COSC 3101, PROF. J. ELDER 79
: A[1..n] is sorted in non-decreasing order: If is in A[1..n], algorithm returns
1,its location
loop-invariant>: If is inwhile
p q key
key q p
n
if [ ]
els
2
1
return( )
return("Key n
A[1..n], then
e
endendif [ ]
end
is in A[p..
ot in list")
q] p q mid
q mid
p mi
key A m
key
id
key A p
e
d
p lse
Algorithm Definition Completed
7/27/2019 04 Iterative Algorithms
80/177
COSC 3101, PROF. J. ELDER 80
g pDefine Problem Define Loop Invariants Define Measure of
Progress
Define Step Define Exit Condition Maintain Loop Inv
Make Progress Initial Conditions Ending
km
79 km
to school
Exit
Exit
79 km 75 km
Exit
Exit
0 km Exit
BinarySea rch(A[1..n], )e k y
7/27/2019 04 Iterative Algorithms
81/177
COSC 3101, PROF. J. ELDER 81
: A[1..n] is sorted in non-decreasing order: If is in A[1..n], algorithm returns
1,its location
loop-invariant>: If is inwhile
p q key
key q p
n
if [ ]
els
2
1
return( )
return("Key n
A[1..n], then
e
endendif [ ]
end
is in A[p..
ot in list")
q] p q mid
q mid
p mi
key A m
key
id
key A p
e
d
p lse
7/27/2019 04 Iterative Algorithms
82/177
COSC 3101, PROF. J. ELDER 82
Simple, right?
Although the concept is simple, binary search isnotoriously easy to get wrong.
Why is this?
7/27/2019 04 Iterative Algorithms
83/177
COSC 3101, PROF. J. ELDER 83
The Devil in the Details
The basic idea behind binary search is easy to grasp.
It is then easy to write pseudocode that works for atypical case.
Unfortunately, it is equally easy to write pseudocode thatfails on the boundary conditions .
7/27/2019 04 Iterative Algorithms
84/177
COSC 3101, PROF. J. ELDER 84
1
if [ ]
else
end
q mid
p
key A mid
mid
The Devil in the Details
1
if [ ]
else
end
q mid
p
key A mid
mid or
What condition will break the loop invariant?
7/27/2019 04 Iterative Algorithms
85/177
COSC 3101, PROF. J. ELDER 85
The Devil in the Details
key 36
3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95
mid
sC eod lek cey tA[m rige hid] t lf: ha
Bug!!
7/27/2019 04 Iterative Algorithms
86/177
COSC 3101, PROF. J. ELDER 86
1
if [ ]
else
end
q mid
p
key A mid
mid
The Devil in the Details
1
if [ ]
else
end
q mid
p
key A mid
mid
if < [ ]
else
end
1q mid
p
key A mid
mid
OK OK Not OK!!
7/27/2019 04 Iterative Algorithms
87/177
COSC 3101, PROF. J. ELDER 87
key 25
3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95
The Devil in the Details
mid2
p q
mid2
p q
or
Shouldnt matter, right? Select mid2
p q
7/27/2019 04 Iterative Algorithms
88/177
COSC 3101, PROF. J. ELDER 88
6 74
The Devil in the Details
key 25
9591888372605351494336252121181353
If key mid ,then key is inleft half.
If key > mid ,then key is inright half.
mid
7/27/2019 04 Iterative Algorithms
89/177
h l h l
Exit
7/27/2019 04 Iterative Algorithms
90/177
COSC 3101, PROF. J. ELDER 90
2513 74
The Devil in the Details
key 25
9591888372605351494336212118653
If key mid ,then key is inleft half.
If key > mid ,then key is inright half.
Another bug! No progress
toward goal:Loops Forever!mid
79 km 75 km
Exit
h l h l
7/27/2019 04 Iterative Algorithms
91/177
COSC 3101, PROF. J. ELDER 91
if [
mid
]2
1else
end
key A mi
p q
q mid
p mid
d
The Devil in the Details
if [
mid
]2
1else
end
key A mi
p q
q mid
p mid
d
if < [
mid2
1]
else
end
key A mid
p q
q mid
p mid
OK OK Not OK!!
H M P ibl Al i h ?
7/27/2019 04 Iterative Algorithms
92/177
COSC 3101, PROF. J. ELDER 92
if [
mid
]2
1else
end
key A mi
p q
q mid
p mid
d
How Many Possible Algorithms?
midr2
o ? p q
if < [or ?]key A mid
else
o
end
1r q mid
p mid
Alternative Algorithm: Less Efficient but More Clear
7/27/2019 04 Iterative Algorithms
93/177
COSC 3101, PROF. J. ELDER 93
: A[1..n] is sorted in non-decreasing order: If is in A[1..n], algorithm returns
1,its location
loop-invariant>: If is
BinarySea
in
rch(A[1..n],
whil
)
e p q
key
key q p
e
n
k y
2
retur
A[1..n
n(mid)
], then
if = [ ]
elseif root ,then key isin right half.
If key = root ,then key isfound
Algorithm Definition Completed
7/27/2019 04 Iterative Algorithms
100/177
COSC 3101, PROF. J. ELDER 100
Define Problem Define Loop Invariants Define Measure of Progress
Define Step Define Exit Condition Maintain Loop Inv
Make Progress Initial Conditions Ending
km
79 km
to school
Exit
Exit
79 km 75 km
Exit
Exit
0 km Exit
7/27/2019 04 Iterative Algorithms
101/177
End of Lecture 4
Sept 20, 2007
Card Trick
7/27/2019 04 Iterative Algorithms
102/177
COSC 3101, PROF. J. ELDER 102
A volunteer, please.
Card Trick
7/27/2019 04 Iterative Algorithms
103/177
Loop Invariantsfor
Iterative Algorithms
A Third
Search Example:
A Card Trick
7/27/2019 04 Iterative Algorithms
104/177
COSC 3101, PROF. J. ELDER 104
Pick a Card
Done
Loop Invariant:
7/27/2019 04 Iterative Algorithms
105/177
COSC 3101, PROF. J. ELDER 105
The selected card is one of these.
7/27/2019 04 Iterative Algorithms
106/177
COSC 3101, PROF. J. ELDER 106
Which column?
left
Loop Invariant:
7/27/2019 04 Iterative Algorithms
107/177
COSC 3101, PROF. J. ELDER 107
The selected card is one of these.
Selected column is placed
7/27/2019 04 Iterative Algorithms
108/177
COSC 3101, PROF. J. ELDER 108
in the middle
I will rearrange the cards
7/27/2019 04 Iterative Algorithms
109/177
COSC 3101, PROF. J. ELDER109
7/27/2019 04 Iterative Algorithms
110/177
7/27/2019 04 Iterative Algorithms
111/177
COSC 3101, PROF. J. ELDER111
Which column?
right
Loop Invariant:
7/27/2019 04 Iterative Algorithms
112/177
COSC 3101, PROF. J. ELDER112
The selected card is one of these.
Selected column is placed
7/27/2019 04 Iterative Algorithms
113/177
COSC 3101, PROF. J. ELDER113
in the middle
I will rearrange the cards
7/27/2019 04 Iterative Algorithms
114/177
COSC 3101, PROF. J. ELDER114
7/27/2019 04 Iterative Algorithms
115/177
COSC 3101, PROF. J. ELDER115
Which column?
left
Loop Invariant:
7/27/2019 04 Iterative Algorithms
116/177
COSC 3101, PROF. J. ELDER116
The selected card is one of these.
Selected column is placed h ddl
7/27/2019 04 Iterative Algorithms
117/177
COSC 3101, PROF. J. ELDER117
in the middle
7/27/2019 04 Iterative Algorithms
118/177
Algorithm Definition CompletedDefine Problem Define Loop Invariants Define Measure of
7/27/2019 04 Iterative Algorithms
119/177
COSC 3101, PROF. J. ELDER 119
Define Problem Define Loop Invariants Define Measure of Progress
Define Step Define Exit Condition Maintain Loop Inv
Make Progress Initial Conditions Ending
km
79 kmto school
Exit
Exit
79 km 75 km
Exit
Exit
0 km Exit
Ternary Search
7/27/2019 04 Iterative Algorithms
120/177
COSC 3101, PROF. J. ELDER 120
Ternary Search
How many iterations are required to guaranteesuccess?
Loop Invariant: selected card in central subset of cards
1Size of subset = / 3
wheretotal number of cards
iteration index
i n
n
i
7/27/2019 04 Iterative Algorithms
121/177
Loop Invariantsfor
Iterative Algorithms
A Fourth Example:
Partitioning
(Not a search problem:
can be used for sorting, e.g., Quicksort)
The Partitioning Problem
7/27/2019 04 Iterative Algorithms
122/177
COSC 3101, PROF. J. ELDER 122
g
88 149825
62
52
79
3023
31
Input:
14
2530
233188 98
6279
52
Output:x=52
Problem: Partition a list into a set of small values and a set of large values.
Precise Specification
7/27/2019 04 Iterative Algorithms
123/177
COSC 3101, PROF. J. ELDER 123
p
[ ... ] is an arbitrary list of values. [ ] is the pivoPrecondit .i ton: A p r x A r
p r
is rearranged such that [ ... 1] [ ] [ 1... ]for some q.Postcondition: A A p q A q x A q r
p r q
Loop Invariant
7/27/2019 04 Iterative Algorithms
124/177
COSC 3101, PROF. J. ELDER 124
3 subsets are maintained
One containing values lessthan or equal to the pivot
One containing valuesgreater than the pivot
One containing values yetto be processed
p
Maintaining Loop Invariant
7/27/2019 04 Iterative Algorithms
125/177
COSC 3101, PROF. J. ELDER 125
g p
Consider element at location j
If greater than pivot, incorporate into> set by incrementing j.
If less than or equal to pivot,incorporate into set by swapping
with element at location i+1 andincrementing both i and j.
Measure of progress: size of unprocessed set.
Maintaining Loop Invariant
7/27/2019 04 Iterative Algorithms
126/177
COSC 3101, PROF. J. ELDER 126
g p
Establishing Loop Invariant
7/27/2019 04 Iterative Algorithms
127/177
COSC 3101, PROF. J. ELDER 127
g p
Establishing Postcondition
7/27/2019 04 Iterative Algorithms
128/177
COSC 3101, PROF. J. ELDER 128
g
on exit j
Exhaustive on exit
Establishing Postcondition
7/27/2019 04 Iterative Algorithms
129/177
COSC 3101, PROF. J. ELDER 129
g
An Example
7/27/2019 04 Iterative Algorithms
130/177
COSC 3101, PROF. J. ELDER 130
Running Time
7/27/2019 04 Iterative Algorithms
131/177
COSC 3101, PROF. J. ELDER 131
Each iteration takes (1) time Total = (n)
or
Algorithm Definition CompletedDefine Problem Define Loop Invariants Define Measure of
7/27/2019 04 Iterative Algorithms
132/177
COSC 3101, PROF. J. ELDER 132
Progress
Define Step Define Exit Condition Maintain Loop Inv
Make Progress Initial Conditions Ending
km
79 kmto school
Exit
Exit
79 km 75 km
Exit
Exit
0 km Exit
7/27/2019 04 Iterative Algorithms
133/177
More Examples of Iterative Algorithms
Using Constraints on Input to Achieve Linear-Time Sorting
Recall: InsertionSort
7/27/2019 04 Iterative Algorithms
134/177
COSC 3101, PROF. J. ELDER 134
2
2
( 1)Worst case (reverse order): : 1 ( ) ( )
2
n
j j
n nt j j T n n
Recall: MergeSort
7/27/2019 04 Iterative Algorithms
135/177
COSC 3101, PROF. J. ELDER 135
( ) ( log )T n n n
Comparison Sorts
7/27/2019 04 Iterative Algorithms
136/177
COSC 3101, PROF. J. ELDER 136
InsertionSort and MergeSort are examples of (stable)Comparison Sort algorithms.
QuickSort is another example we will study shortly.
Comparison Sort algorithms sort the input by successivecomparison of pairs of input elements.
Comparison Sort algorithms are very general: theymake no assumptions about the values of the input
elements.
Comparison Sorts
7/27/2019 04 Iterative Algorithms
137/177
COSC 3101, PROF. J. ELDER 137
2InsertionSort is ( ).n
MergeSort is ( log ).n n
Can we do better?
Comparison Sort: Decision Trees
7/27/2019 04 Iterative Algorithms
138/177
COSC 3101, PROF. J. ELDER 138
Example: Sorting a 3-element array A[1..3]
Comparison Sort
7/27/2019 04 Iterative Algorithms
139/177
COSC 3101, PROF. J. ELDER 139
Worst-case time is equal to the height of the binarydecision tree.
The height of the tree is the log of the number of leaves.
The leaves of the tree represent all possible
permutations of the input. How many are there?
log ! ( log )n n nThus MergeSort is asymptotically optimal.
Linear Sorts?
7/27/2019 04 Iterative Algorithms
140/177
COSC 3101, PROF. J. ELDER 140
Comparison sorts are very general, but are ( log ) n n
Faster sorting may be possible if we can constrain the nature of the input.
Example 1. Counting Sort
7/27/2019 04 Iterative Algorithms
141/177
COSC 3101, PROF. J. ELDER 141
Counting Sort applies when the elements to be sortedcome from a finite (and preferably small) set .
For example, the elements to be sorted are integers inthe range [0k -1], for some fixed integer k.
We can then create an array V[0k -1] and use it tocount the number of elements with each value [0k -1].
Then each input element can be placed in exactly the
right place in the output array in constant time .
Counting Sort
7/27/2019 04 Iterative Algorithms
142/177
COSC 3101, PROF. J. ELDER 142
Input: N records with integer keys between [0k -1].
Output: Stable sorted keys.
Algorithm: Count frequency of each key value to determine transition
locations Go through the records in order putting them where they go.
Input:Output: 0 0 0 0 0 1 1 1 1 1 1 1 1 1 2 2 3 3 3
1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
CountingSort
I 0 0 0 0 0
7/27/2019 04 Iterative Algorithms
143/177
COSC 3101, PROF. J. ELDER 143
Stable sort: If two keys are the same, their order does not change.
Input:
Output:
1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
Thus the 4 th record in input with digit 1 must be
the 4 th record in output with digit 1.
It belongs at output index 8, because 8 records go before itie, 5 records with a smaller digit & 3 records with the same digit
0 0 0 0 0 1 1 1 1 1 1 1 1 1 2 2 2 33
Count These!
Index: 11109876543210 12 13 14 15 16 17 18
CountingSort
I 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
7/27/2019 04 Iterative Algorithms
144/177
COSC 3101, PROF. J. ELDER 144
Input:
Output:Index: 11109876543210 12 13 14 15 16 17 18
Value v:
# of records with digit v:
1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
32102395
N records. Time to count? (N)
CountingSort
I 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
7/27/2019 04 Iterative Algorithms
145/177
COSC 3101, PROF. J. ELDER 145
Input:
Output:Index: 11109876543210 12 13 14 15 16 17 18
Value v:# of records with digit v:
# of records with digit < v:
1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
32103395171450
N records, k different values. Time to count? (k)
CountingSort
I t 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
7/27/2019 04 Iterative Algorithms
146/177
COSC 3101, PROF. J. ELDER 146
Input:
Output:Index: 11109876543210 12 13 14 15 16 17 18
Value v:# of records with digit < v:
1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
3210171450
= location of first record with digit v.
0 0 0 0 0 1 1 1 1 1 1 1 1 1 2 2 2 33
CountingSort
I t 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
7/27/2019 04 Iterative Algorithms
147/177
COSC 3101, PROF. J. ELDER 147
Input:
Output:Index: 11109876543210 12 13 14 15 16 17 18
Value v:
1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
3210171450Location of first record
with digit v.
Algorithm: Go through the records in order putting them where they go.
10 ?
Loop Invariant
7/27/2019 04 Iterative Algorithms
148/177
COSC 3101, PROF. J. ELDER 148
The first i-1 keys have been placed in the correctlocations in the output array
The auxiliary data structure v indicates the location atwhich to place the i th key for each possible key value
from [1..k-1] .
7/27/2019 04 Iterative Algorithms
149/177
CountingSort
Input: 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
7/27/2019 04 Iterative Algorithms
150/177
COSC 3101, PROF. J. ELDER 150
Input:
Output:Index: 11109876543210 12 13 14 15 16 17 18
Value v:
1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
3210171460Location of next record
with digit v.
0
Algorithm: Go through the records in order putting them where they go.
1
CountingSort
Input: 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
7/27/2019 04 Iterative Algorithms
151/177
COSC 3101, PROF. J. ELDER 151
Input:
Output:Index: 11109876543210 12 13 14 15 16 17 18
Value v:
1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
3210171461Location of next record
with digit v.
0 0
Algorithm: Go through the records in order putting them where they go.
1
CountingSort
Input: 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
7/27/2019 04 Iterative Algorithms
152/177
COSC 3101, PROF. J. ELDER 152
Input:
Output:Index: 11109876543210 12 13 14 15 16 17 18
Value v:
1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
3210171462Location of next record
with digit v.
0 1
Algorithm: Go through the records in order putting them where they go.
10
CountingSort
Input: 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
7/27/2019 04 Iterative Algorithms
153/177
COSC 3101, PROF. J. ELDER 153
Input:
Output:Index: 11109876543210 12 13 14 15 16 17 18
Value v:
1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
3210171472Location of next record
with digit v.
0 1
Algorithm: Go through the records in order putting them where they go.
10 3
CountingSort
Input: 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
7/27/2019 04 Iterative Algorithms
154/177
COSC 3101, PROF. J. ELDER 154
Input:
Output:Index: 11109876543210 12 13 14 15 16 17 18
Value v:
1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
3210181472Location of next record
with digit v.
0 1
Algorithm: Go through the records in order putting them where they go.
10 1 3
CountingSort
Input: 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
7/27/2019 04 Iterative Algorithms
155/177
COSC 3101, PROF. J. ELDER 155
Input:
Output:Index: 11109876543210 12 13 14 15 16 17 18
Value v:
1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
3210181482Location of next record
with digit v.
0 1
Algorithm: Go through the records in order putting them where they go.
10 1 31
CountingSort
Input: 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
7/27/2019 04 Iterative Algorithms
156/177
COSC 3101, PROF. J. ELDER 156
Input:
Output:Index: 11109876543210 12 13 14 15 16 17 18
Value v:
1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
3210181492Location of next record
with digit v.
0 1
Algorithm: Go through the records in order putting them where they go.
10 331 1
CountingSort
Input: 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
7/27/2019 04 Iterative Algorithms
157/177
COSC 3101, PROF. J. ELDER 157
Input:
Output:Index: 11109876543210 12 13 14 15 16 17 18
Value v:
1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
3210191492Location of next record
with digit v.
0 1
Algorithm: Go through the records in order putting them where they go.
10 1 31 1 3
CountingSort
Input: 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
7/27/2019 04 Iterative Algorithms
158/177
COSC 3101, PROF. J. ELDER 158
Input:
Output:Index: 11109876543210 12 13 14 15 16 17 18
Value v:
1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
32101914102Location of next record
with digit v.
0 1
Algorithm: Go through the records in order putting them where they go.
10 0 1 1 1 3 3
CountingSort
Input: 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
7/27/2019 04 Iterative Algorithms
159/177
COSC 3101, PROF. J. ELDER 159
Input:
Output:Index: 11109876543210 12 13 14 15 16 17 18
Value v:
1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
32101914103Location of next record
with digit v.
0 1
Algorithm: Go through the records in order putting them where they go.
10 21 1 1 3 30
CountingSort
Input: 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
7/27/2019 04 Iterative Algorithms
160/177
COSC 3101, PROF. J. ELDER 160
Input:
Output:Index: 11109876543210 12 13 14 15 16 17 18
Value v:
1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
32101915103Location of next record
with digit v.
0 1
Algorithm: Go through the records in order putting them where they go.
10 11 1 1 3 30 2
7/27/2019 04 Iterative Algorithms
161/177
CountingSort
Input: 1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
7/27/2019 04 Iterative Algorithms
162/177
COSC 3101, PROF. J. ELDER 162
Input:
Output:Index: 11109876543210 12 13 14 15 16 17 18
Value v:
1 0 0 1 3 1 1 3 1 0 2 1 0 1 1 2 2 1 0
32101917145Location of next record
with digit v.
0 110 11 1 1 3 30 20 0 1 1 1 2 2
(N)Time = (N+k)Total =
7/27/2019 04 Iterative Algorithms
163/177
End of Lecture 5
Sept 25, 2007
Input:
Example 2. RadixSort 344125
125224
7/27/2019 04 Iterative Algorithms
164/177
COSC 3101, PROF. J. ELDER 164
Input: A of stack of N punch cards. Each card contains d digits. Each digit between [0k -1]
Output: Sorted cards.
125333134224334
143225325243
Digit Sort: Select one digit Separate cards into k piles
based on selected digit (e.g., Counting Sort).
225
325
333134
334
344143
243Stable sort: If two cards are the same for that digit,their order does not change.
RadixSort
344 125 125
7/27/2019 04 Iterative Algorithms
165/177
COSC 3101, PROF. J. ELDER 165
344
125333134224
334143225325243
Sort wrt whichdigit first?
The mostsignificant.
125
134 143 224225
243 344 333334325
Sort wrt whichdigit Second?
The next mostsignificant.
125
224225 325 134
333334 143 243 344
All meaning in first sort lost.
RadixSort
7/27/2019 04 Iterative Algorithms
166/177
COSC 3101, PROF. J. ELDER 166
344125333134
224334143225325243
Sort wrt whichdigit first?
Sort wrt whichdigit Second?
The leastsignificant.
333 143243 344
134224334 125225325
The next leastsignificant.
224125225325
333 134334 143243 344
7/27/2019 04 Iterative Algorithms
167/177
RadixSort
7/27/2019 04 Iterative Algorithms
168/177
COSC 3101, PROF. J. ELDER 168
Sort wrt i+1stdigit.
2 241 252 253 25
3 33 1 343 34 1 432 43 3 44
Is sorted wrtfirst i digits.
1 25 1 34 1 43
2 242 252 43
3 25 3 33 3 34 3 44
Is sorted wrtfirst i+1 digits.
i+1
These are in thecorrect order
because sortedwrt high order digit
RadixSort
7/27/2019 04 Iterative Algorithms
169/177
COSC 3101, PROF. J. ELDER 169
Sort wrt i+1stdigit.
2 241 252 253 25
3 33 1 343 34 1 432 43 3 44
Is sorted wrtfirst i digits.
1 25 1 34 1 43
2 242 252 43
3 25 3 33 3 34 3 44
i+1
These are in thecorrect order
because was sorted &stable sort left sorted
Is sorted wrtfirst i+1 digits.
Loop Invariant
7/27/2019 04 Iterative Algorithms
170/177
COSC 3101, PROF. J. ELDER 170
The keys have been correctly stable-sorted with respectto the i-1 least-significant digits.
Running Time
7/27/2019 04 Iterative Algorithms
171/177
COSC 3101, PROF. J. ELDER 171
Running time is ( ( ))Where
# of digits in each number# of elements to be sorted# of possible values for each digit
d n k
d n k
Example 3. Bucket Sort
7/27/2019 04 Iterative Algorithms
172/177
COSC 3101, PROF. J. ELDER 172
Applicable if input is constrained to finite interval, e.g.,[01).
If input is random and uniformly distributed, expected run time is (n).
Bucket Sort
7/27/2019 04 Iterative Algorithms
173/177
COSC 3101, PROF. J. ELDER 173
Loop Invariants
7/27/2019 04 Iterative Algorithms
174/177
COSC 3101, PROF. J. ELDER 174
Loop 1 The first i-1 keys have been correctly placed into buckets of
width 1/n .
Loop 2
The keys within each of the first i-1 buckets have been correctlystable-sorted.
PseudoCode
7/27/2019 04 Iterative Algorithms
175/177
COSC 3101, PROF. J. ELDER 175
(1)
(1)( )n
n
( )n
Examples of Iterative Algorithms
7/27/2019 04 Iterative Algorithms
176/177
COSC 3101, PROF. J. ELDER 176
Binary Search Partitioning
Insertion Sort
Counting Sort Radix Sort
Bucket Sort
Which can be made stable?
Which sort in place?
How about MergeSort?
7/27/2019 04 Iterative Algorithms
177/177
End of Iterative Algorithms