Embed Size (px)
Citation preview
Sorting andrecurrence analysis techniques
CSE 373: Data Structures and Algorithms
Thanks to Kasey Champion, Ben Jones, Adam Blank, Michael Lee, Evan McCarty, Robbie Weber, Whitaker Brand, Zora Fung, Stuart Reges, Justin Hsia, Ruth Anderson, and many others for sample slides and materials ...
Autumn 2018
Shrirang (Shri) [email protected]
Review
Given n comparable elements, rearrange them in an increasing order.Input- An array ! that contains n elements - Each element has a key " and an associated data- Keys support a comparison function (e.g., keys implement a Comparable interface)
Expected output- An output array ! such that for any # and $, - ![#] ≤ ![$] if # < $ (increasing order)- Array ! can also have elements in reverse order (decreasing order)
Sorting problem statement
CSE 373 AU 18 2
Review
Stable- In the output, equal elements (i.e., elements with equal keys) appear in their original order
In-place- Algorithm uses a constant additional space, !(1) extra space
Adaptive- Performs better when input is almost sorted or nearly sorted- (Likely different big-O for best-case and worst-case)
Fast. ! (% log %)
No algorithm has all of these properties. So choice of algorithm depends on the situation.
Desired properties in a sorting algorithm
CSE 373 AU 18 3
- ! "#- Insertion sort- Selection sort- Quick sort (worst)
- !(" log ")- Merge sort- Heap sort- Quick sort (avg)
- Ω(" log ") -- lower bound on comparison sorts- !(") – non-comparison sorts- Bucket sort (avg)
Sorting algorithms – High-level view
CSE 373 AU 18 4
Loop/step invariant: _______________________________________________________________________
Runtime: Worst ______________ Average ______________ Best ______________
Input: Worst ______________________ Best ______________________
Stable ____________________ In-place ____________________ Adaptive ____________________
Operations: Comparisons __________________________ Moves
Data structure ______________________________________________________
Some questions to consider when analyzing a sorting algorithm.
A framework to think about sorting algos
CSE 373 AU 18 5
> or < or approx. equal
What is the state of the data during each step while sorting
Yes/No/Can-be Yes/No/Can-be Yes/No
Which data structure is better suited for this algo
Loop/step invariant: _______________________________________________________________________
Runtime: Worst ______________ Average ______________ Best ______________
Input: Worst ______________________ Best ______________________
Stable ____________________ In-place ____________________ Adaptive ____________________
Operations: Comparisons __________________________ Moves
Data structure ______________________________________________________
Idea: At step !, insert the !"# element in the correct position among the first ! elements.
Insertion sort
CSE 373 AU 18 6
Visualization: https://visualgo.net/en/sorting
Loop/step invariant: _______________________________________________________________________Runtime: Worst ______________ Average ______________ Best ______________Input: Worst ______________________ Best ______________________Stable ____________________ In-place ____________________ Adaptive ____________________Operations: Comparisons __________________________ MovesData structure ______________________________________________________
Idea: At step !, find the smallest element among the not-yet-sorted elements (! … #) and swap it with the element at !.
Selection sort
CSE 373 AU 18 7
Visualization: https://visualgo.net/en/sorting
Loop/step invariant: _______________________________________________________________________Runtime: Worst ______________ Average ______________ Best ______________Input: Worst ______________________ Best ______________________Stable ____________________ In-place ____________________ Adaptive ____________________Operations: Comparisons __________________________ MovesData structure ______________________________________________________
Idea:
Heap sort
CSE 373 AU 18 8
Idea: 1. Treat initial array as a heap2. When you call removeMin(), that frees up a slot towards the end in the array. Put the extract min element there.3. More specifically, when you remove the !"# element, put it at $ % − !4. This gives you a reverse sorted array. But easy to fix in-place.
In-place heap sort
CSE 373 AU 18 9
Very important technique in algorithm to attack problemsThree steps:1. Divide: Split the original problem into smaller parts2. Conquer: Solve individual parts independently (think recursion)3. Combine: Put together individual solved parts to produce an overall solution
Merge sort and Quick sort are classic examples of sorting algorithms that use this technique
Design technique: Divide-and-conquer
CSE 373 AU 18 10
To sort a given array,
Divide: Split the input array into two halves
Conquer: Sort each half independently
Combine: Merge the two sorted halves into one sorted whole (HW3 Problem 6!)
Merge sort
CSE 373 AU 18 11
Visualization: https://visualgo.net/en/sorting
Merge sortSplit array in the middleSort the two halvesMerge them together
0 1 2 3 4
8 2 57 91 22
0 1
8 2
0 1 2
57 91 22
0
8
0
2
0
57
0 1
91 22
0
91
0
22
0 1
22 91
0 1 2
22 57 91
0 1
2 8
0 1 2 3 4
2 8 22 57 91
! " = $%& if " ≤ 12! "
2 + %-" otherwise
Review: Unfolding (technique 1)
CSE 373 AU 18 13
! " = $%& if " ≤ 12! "
2 + %-" otherwise
n
n2
n4
...
1 1
...
1 1
n4
...
1 1
...
1 1
n2
n4
...
1 1
...
1 1
n4
...
1 1
...
1 1
Technique 2: Tree method
CSE 373 AU 18 14
Level
Number of
Nodes at level
Work per Node
Work per Level
0
1
2
!
base
Last recursive level:
n
n2
n4
...
1 1
...
1 1
n4
...
1 1
...
1 1
n2
n4
...
1 1
...
1 1
n4
...
1 1
...
1 1
Technique 2: Tree method
CSE 373 AU 18 15
Level
Number of
Nodes at level
Work per Node
Work per Level
0
1
2
!
base
Last recursive level:
n
n2
n4
...
1 1
...
1 1
n4
...
1 1
...
1 1
n2
n4
...
1 1
...
1 1
n4
...
1 1
...
1 1
Technique 2: Tree method
CSE 373 AU 18 16
Level
Number of
Nodes at level
Work per Node
Work per Level
0
1
2
!
base
Last recursive level:
n
n2
n4
...
1 1
...
1 1
n4
...
1 1
...
1 1
n2
n4
...
1 1
...
1 1
n4
...
1 1
...
1 1
Technique 2: Tree method
CSE 373 AU 18 17
Level
Number of
Nodes at level
Work per Node
Work per Level
0
1
2
!
base
Last recursive level:
n
n2
n4
...
1 1
...
1 1
n4
...
1 1
...
1 1
n2
n4
...
1 1
...
1 1
n4
...
1 1
...
1 1
Technique 2: Tree method
CSE 373 AU 18 18
Level
Number of
Nodes at level
Work per Node
Work per Level
0
1
2
!
base
Last recursive level:
n
n2
n4
...
1 1
...
1 1
n4
...
1 1
...
1 1
n2
n4
...
1 1
...
1 1
n4
...
1 1
...
1 1
Technique 2: Tree method
CSE 373 AU 18 19
Level
Number of
Nodes at level
Work per Node
Work per Level
0 1 ! !
1 2!2 !
2 4!4 !
$ 2% !2% !
base 2&'()* 1 !
Last recursive level: log ! − 1
Combining it all together…
0 ! = ! + 3%45
&'(6 * 78! = ! + ! log !
Tree Method Practice
20
!n#
! n4
#
… …
! n4
#! n4
#
! n16
#! n16
#! n16
#! n16
#! n16
#! n16
#! n16
#! n16
#! n16
#
… … …… … …… … …… … …… … …… … …… … …… … ……
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
' ( =4 *ℎ,( ( ≤ 13' (
4 + !(# 01ℎ,2*34,
EXAMPLE PROVIDED BY CS 161 – JESSICA SUHTTPS://WEB.STANFORD.EDU/CLASS/ARCHIVE/CS/CS161/CS161.1168/LECTURE3.PDF
' (4 ' (
4 ' (4
' (4 + ' (
4 + ' (4 + !(#
Level (i)Number of
NodesWork per
NodeWork per
Level
0 1
1
2
3base
Tree Method Practice
21
!n#
!n
4
#
… …
!n
4
#!n
4
#
!n
16
#!
n
16
#!
n
16
#!
n
16
#!
n
16
#!
n
16
#
!n
16
#!
n
16
#!
n
16
#
… … …… … …… … …… … …… … …… … …… … …… … ……
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
' ( =4 *ℎ,( ( ≤ 1
3'(
4+ !(# 01ℎ,2*34,
EXAMPLE PROVIDED BY CS 161 – JESSICA SU HTTPS://WEB.STANFORD.EDU/CLASS/ARCHIVE/CS/CS161/CS161.1168/LECTURE3.PDF
'(
4 '(4 '
(
4
'(
4+ '
(
4+ '
(
4+ !(#
Level (i)Number of
NodesWork per
NodeWork per
Level
0 1 !(2 !(2
1 3 !(4
# 3
16!(#
2 9 !(16
# 9256
!(#
3 38 !(48
#316
8
!(#
base 39:;<= 4 4 > 39:;<=
Combining it all together…
' ( = 4 (9:;<? + @8AB
9:;C = DE3
16
8
!(#
Last recursive level: log< ( − 1
Technique 3: Master Theorem
22
! " =$ %ℎ'" " = 1)! "
* + ", -.ℎ'/%01'
Given a recurrence of the following form:
Where ), *, and 2 are constants, then !(") has the following asymptotic bounds
! " ∈ Θ ",log: ) < 2
log: ) = 2 ! " ∈ Θ ", log< "
log: ) > 2 ! " ∈ Θ ">?@A B
If
If
If
then
then
then
Apply Master Theorem
23
! " =1 %ℎ'" " ≤ 12! "
2 + " +,ℎ'-%./'
! " =0 %ℎ'" " = 1
1! "2 + "3 +,ℎ'-%./'
log7 1 = 8 ! " ∈ Θ "3 log; "log7 1 > 8 ! " ∈ Θ "=>?@ A
If
If
! " ∈ Θ "3log7 1 < 8If then
then
then
Given a recurrence of the form:
a = 2b = 2c = 1d = 1
log7 1 = 8 ⇒ log; 2 = 1
! " ∈ Θ "3 log; " ⇒ Θ "D log; "
Reflecting on Master TheoremThe case - Recursive case conquers work more quickly than it divides work- Most work happens near “top” of tree- Non recursive work in recursive case dominates growth, nc term
The case - Work is equally distributed across call stack (throughout the “tree”)- Overall work is approximately work at top level x height
The case - Recursive case divides work faster than it conquers work- Most work happens near “bottom” of tree- Leaf work dominates branch work
24
! " =$ %ℎ'" " = 1
)! "* + ", -.ℎ'/%01'
log5 ) = 6 ! " ∈ Θ ", log9 "log5 ) > 6 ! " ∈ Θ ";<=> ?
If
If
! " ∈ Θ ",log5 ) < 6If then
then
then
Given a recurrence of the form: log5 ) < 6
log5 ) = 6
log5 ) > 6
A')BC-/D ≈ $ ";<=> ?
ℎ'0Fℎ. ≈ log5 )*/)"6ℎC-/D ≈ ",log5 )
1. Unfolding method - more of a brute force method - Tedious but works2. Tree methods- more scratch work but less error prone
3. Master theorem- quick, but applicable only to certain type of recurrences - does not give a closed form (gives big-Theta)
Recurrence analysis techniques
CSE 373 AU 18 – SHRI MARE 25
