Upload
andres-mendez-vazquez
View
170
Download
4
Embed Size (px)
Citation preview
Analysis of AlgorithmsSorting
Andres Mendez-Vazquez
February 4, 2016
1 / 109
Images/ITESM.png
Outline1 Sorting problem
DefinitionClassic Complexities
2 Heap SortHeap Sort: DefinitionsHeap: Finding Parents and ChildrenMax-HeapifyBuild Max Heap: Using Max-HeapifyHeap Sort
3 Applications of Heap Data StructureHeap Sort: Exercises
4 QuicksortThe Divide and Conquer QuicksortQuicksort: Complexity AnalysisRandomized Quicksort
5 Lower Bounds of SortingLower Bounds of Sorting: Basic Concepts
2 / 109
Images/ITESM.png
Outline1 Sorting problem
DefinitionClassic Complexities
2 Heap SortHeap Sort: DefinitionsHeap: Finding Parents and ChildrenMax-HeapifyBuild Max Heap: Using Max-HeapifyHeap Sort
3 Applications of Heap Data StructureHeap Sort: Exercises
4 QuicksortThe Divide and Conquer QuicksortQuicksort: Complexity AnalysisRandomized Quicksort
5 Lower Bounds of SortingLower Bounds of Sorting: Basic Concepts
3 / 109
Images/ITESM.png
Sorting Problem
InputA sequence of n numbers 〈a1, a2, ..., an〉.
OutputA permutation (reordering) 〈a′1, a′2, ..., a′n〉 such that a′1 ≤ a′2 ≤ ... ≤ a′n
4 / 109
Images/ITESM.png
Sorting Problem
InputA sequence of n numbers 〈a1, a2, ..., an〉.
OutputA permutation (reordering) 〈a′1, a′2, ..., a′n〉 such that a′1 ≤ a′2 ≤ ... ≤ a′n
4 / 109
Images/ITESM.png
Outline1 Sorting problem
DefinitionClassic Complexities
2 Heap SortHeap Sort: DefinitionsHeap: Finding Parents and ChildrenMax-HeapifyBuild Max Heap: Using Max-HeapifyHeap Sort
3 Applications of Heap Data StructureHeap Sort: Exercises
4 QuicksortThe Divide and Conquer QuicksortQuicksort: Complexity AnalysisRandomized Quicksort
5 Lower Bounds of SortingLower Bounds of Sorting: Basic Concepts
5 / 109
Images/ITESM.png
Some Sorting Algorithms
Table of Sorting AlgorithmsAlgorithm Worst-case running time Expected running time
Insertion sort Θ(n2) Θ(n2)Merge sort Θ(n log n) Θ(n log n)Heapsort Θ(n log n) -Quicksort Θ(n2) Θ(n log n)(expected)
Countingsort Θ(k + n) Θ(k + n)Radix sort Θ(d(k + n)) Θ(d(k + n))Bucket sort Θ(n2) Θ(n)(average-case)
6 / 109
Images/ITESM.png
Some Sorting Algorithms
Table of Sorting AlgorithmsAlgorithm Worst-case running time Expected running time
Insertion sort Θ(n2) Θ(n2)Merge sort Θ(n log n) Θ(n log n)Heapsort Θ(n log n) -Quicksort Θ(n2) Θ(n log n)(expected)
Countingsort Θ(k + n) Θ(k + n)Radix sort Θ(d(k + n)) Θ(d(k + n))Bucket sort Θ(n2) Θ(n)(average-case)
6 / 109
Images/ITESM.png
Some Sorting Algorithms
Table of Sorting AlgorithmsAlgorithm Worst-case running time Expected running time
Insertion sort Θ(n2) Θ(n2)Merge sort Θ(n log n) Θ(n log n)Heapsort Θ(n log n) -Quicksort Θ(n2) Θ(n log n)(expected)
Countingsort Θ(k + n) Θ(k + n)Radix sort Θ(d(k + n)) Θ(d(k + n))Bucket sort Θ(n2) Θ(n)(average-case)
6 / 109
Images/ITESM.png
Some Sorting Algorithms
Table of Sorting AlgorithmsAlgorithm Worst-case running time Expected running time
Insertion sort Θ(n2) Θ(n2)Merge sort Θ(n log n) Θ(n log n)Heapsort Θ(n log n) -Quicksort Θ(n2) Θ(n log n)(expected)
Countingsort Θ(k + n) Θ(k + n)Radix sort Θ(d(k + n)) Θ(d(k + n))Bucket sort Θ(n2) Θ(n)(average-case)
6 / 109
Images/ITESM.png
Some Sorting Algorithms
Table of Sorting AlgorithmsAlgorithm Worst-case running time Expected running time
Insertion sort Θ(n2) Θ(n2)Merge sort Θ(n log n) Θ(n log n)Heapsort Θ(n log n) -Quicksort Θ(n2) Θ(n log n)(expected)
Countingsort Θ(k + n) Θ(k + n)Radix sort Θ(d(k + n)) Θ(d(k + n))Bucket sort Θ(n2) Θ(n)(average-case)
6 / 109
Images/ITESM.png
Some Sorting Algorithms
Table of Sorting AlgorithmsAlgorithm Worst-case running time Expected running time
Insertion sort Θ(n2) Θ(n2)Merge sort Θ(n log n) Θ(n log n)Heapsort Θ(n log n) -Quicksort Θ(n2) Θ(n log n)(expected)
Countingsort Θ(k + n) Θ(k + n)Radix sort Θ(d(k + n)) Θ(d(k + n))Bucket sort Θ(n2) Θ(n)(average-case)
6 / 109
Images/ITESM.png
Some Sorting Algorithms
Table of Sorting AlgorithmsAlgorithm Worst-case running time Expected running time
Insertion sort Θ(n2) Θ(n2)Merge sort Θ(n log n) Θ(n log n)Heapsort Θ(n log n) -Quicksort Θ(n2) Θ(n log n)(expected)
Countingsort Θ(k + n) Θ(k + n)Radix sort Θ(d(k + n)) Θ(d(k + n))Bucket sort Θ(n2) Θ(n)(average-case)
6 / 109
Images/ITESM.png
Outline1 Sorting problem
DefinitionClassic Complexities
2 Heap SortHeap Sort: DefinitionsHeap: Finding Parents and ChildrenMax-HeapifyBuild Max Heap: Using Max-HeapifyHeap Sort
3 Applications of Heap Data StructureHeap Sort: Exercises
4 QuicksortThe Divide and Conquer QuicksortQuicksort: Complexity AnalysisRandomized Quicksort
5 Lower Bounds of SortingLower Bounds of Sorting: Basic Concepts
7 / 109
Images/ITESM.png
Heap Sort: Definitions
DefinitionA heap is an array object that can be viewed as a nearly complete binarytree.
8
6 3
4 5 1
0 1 2 3 4 5 6 7
8 6 3 4 5 1
1
2 3
4 5 6
8 / 109
Images/ITESM.png
Heap: Basic Attributes
Given an array A, we have that length[A]It is the size of the storing array.
heap − size[A]Tell us how many elements in the heap are stored in the array.
Thus, we have
0 ≤ heap − size[A] ≤ length[A] (1)
9 / 109
Images/ITESM.png
Heap: Basic Attributes
Given an array A, we have that length[A]It is the size of the storing array.
heap − size[A]Tell us how many elements in the heap are stored in the array.
Thus, we have
0 ≤ heap − size[A] ≤ length[A] (1)
9 / 109
Images/ITESM.png
Heap: Basic Attributes
Given an array A, we have that length[A]It is the size of the storing array.
heap − size[A]Tell us how many elements in the heap are stored in the array.
Thus, we have
0 ≤ heap − size[A] ≤ length[A] (1)
9 / 109
Images/ITESM.png
Outline1 Sorting problem
DefinitionClassic Complexities
2 Heap SortHeap Sort: DefinitionsHeap: Finding Parents and ChildrenMax-HeapifyBuild Max Heap: Using Max-HeapifyHeap Sort
3 Applications of Heap Data StructureHeap Sort: Exercises
4 QuicksortThe Divide and Conquer QuicksortQuicksort: Complexity AnalysisRandomized Quicksort
5 Lower Bounds of SortingLower Bounds of Sorting: Basic Concepts
10 / 109
Images/ITESM.png
Heap Sort: Calculations given a Node i in the heap
Parent(i) - Parent NodeParent(i) =
⌊ i2⌋
Left Node Child: Left(i)Left(i) = 2i
Right Node Child: Right(i)Right(i) = 2i + 1
11 / 109
Images/ITESM.png
Heap Sort: Calculations given a Node i in the heap
Parent(i) - Parent NodeParent(i) =
⌊ i2⌋
Left Node Child: Left(i)Left(i) = 2i
Right Node Child: Right(i)Right(i) = 2i + 1
11 / 109
Images/ITESM.png
Heap Sort: Calculations given a Node i in the heap
Parent(i) - Parent NodeParent(i) =
⌊ i2⌋
Left Node Child: Left(i)Left(i) = 2i
Right Node Child: Right(i)Right(i) = 2i + 1
11 / 109
Images/ITESM.png
Max/Min Heap Properties
Given thatA [i] returns the value of the key, we have that
Max Heap propertyA [Parent(i)] ≥ A[i]
Min Heap propertyA [Parent(i)] ≤ A [i]
12 / 109
Images/ITESM.png
Max/Min Heap Properties
Given thatA [i] returns the value of the key, we have that
Max Heap propertyA [Parent(i)] ≥ A[i]
Min Heap propertyA [Parent(i)] ≤ A [i]
12 / 109
Images/ITESM.png
Max/Min Heap Properties
Given thatA [i] returns the value of the key, we have that
Max Heap propertyA [Parent(i)] ≥ A[i]
Min Heap propertyA [Parent(i)] ≤ A [i]
12 / 109
Images/ITESM.png
Outline1 Sorting problem
DefinitionClassic Complexities
2 Heap SortHeap Sort: DefinitionsHeap: Finding Parents and ChildrenMax-HeapifyBuild Max Heap: Using Max-HeapifyHeap Sort
3 Applications of Heap Data StructureHeap Sort: Exercises
4 QuicksortThe Divide and Conquer QuicksortQuicksort: Complexity AnalysisRandomized Quicksort
5 Lower Bounds of SortingLower Bounds of Sorting: Basic Concepts
13 / 109
Images/ITESM.png
What we want!!!
A function to keep the property of max or min heapAfter all, remembering Kolmogorov, we are acting in a part of the arraytrying to keep certain properties
Which ONE?
Remember
14 / 109
Images/ITESM.png
What we want!!!
A function to keep the property of max or min heapAfter all, remembering Kolmogorov, we are acting in a part of the arraytrying to keep certain properties
Which ONE?
RememberSingle nodes are always min heaps or max heaps
14 / 109
Images/ITESM.png
Heap Sort: Max-HeapifyAlgorithm (preserving the heap property) when somebody violates themax/min propertyMax-Heapify(A, i)
1 l = Left(i)2 r = Right(i)3 If l ≤ heap − size [A] and A [l] > A [i]4 largest = l5 else largest = i6 If r ≤ heap − size [A] and A [r ] > A [largest]7 largest = r8 if largest 6= i9 exchange A[i] with A[largest]10 Max-Heapify(A, largest)
Figure: A trickle down algorithm
15 / 109
Images/ITESM.png
Heap Sort: Max-HeapifyAlgorithm (preserving the heap property) when somebody violates themax/min propertyMax-Heapify(A, i)
1 l = Left(i)2 r = Right(i)3 If l ≤ heap − size [A] and A [l] > A [i]4 largest = l5 else largest = i6 If r ≤ heap − size [A] and A [r ] > A [largest]7 largest = r8 if largest 6= i9 exchange A[i] with A[largest]10 Max-Heapify(A, largest)
Figure: A trickle down algorithm
15 / 109
Images/ITESM.png
Heap Sort: Max-HeapifyAlgorithm (preserving the heap property) when somebody violates themax/min propertyMax-Heapify(A, i)
1 l = Left(i)2 r = Right(i)3 If l ≤ heap − size [A] and A [l] > A [i]4 largest = l5 else largest = i6 If r ≤ heap − size [A] and A [r ] > A [largest]7 largest = r8 if largest 6= i9 exchange A[i] with A[largest]10 Max-Heapify(A, largest)
Figure: A trickle down algorithm
15 / 109
Images/ITESM.png
Heap Sort: Max-HeapifyAlgorithm (preserving the heap property) when somebody violates themax/min propertyMax-Heapify(A, i)
1 l = Left(i)2 r = Right(i)3 If l ≤ heap − size [A] and A [l] > A [i]4 largest = l5 else largest = i6 If r ≤ heap − size [A] and A [r ] > A [largest]7 largest = r8 if largest 6= i9 exchange A[i] with A[largest]10 Max-Heapify(A, largest)
Figure: A trickle down algorithm
15 / 109
Images/ITESM.png
Heap Sort: Max-HeapifyAlgorithm (preserving the heap property) when somebody violates themax/min propertyMax-Heapify(A, i)
1 l = Left(i)2 r = Right(i)3 If l ≤ heap − size [A] and A [l] > A [i]4 largest = l5 else largest = i6 If r ≤ heap − size [A] and A [r ] > A [largest]7 largest = r8 if largest 6= i9 exchange A[i] with A[largest]10 Max-Heapify(A, largest)
Figure: A trickle down algorithm
15 / 109
Images/ITESM.png
Heap Sort: Max-HeapifyAlgorithm (preserving the heap property) when somebody violates themax/min propertyMax-Heapify(A, i)
1 l = Left(i)2 r = Right(i)3 If l ≤ heap − size [A] and A [l] > A [i]4 largest = l5 else largest = i6 If r ≤ heap − size [A] and A [r ] > A [largest]7 largest = r8 if largest 6= i9 exchange A[i] with A[largest]10 Max-Heapify(A, largest)
Figure: A trickle down algorithm
15 / 109
Images/ITESM.png
Heap Sort: Max-HeapifyAlgorithm (preserving the heap property) when somebody violates themax/min propertyMax-Heapify(A, i)
1 l = Left(i)2 r = Right(i)3 If l ≤ heap − size [A] and A [l] > A [i]4 largest = l5 else largest = i6 If r ≤ heap − size [A] and A [r ] > A [largest]7 largest = r8 if largest 6= i9 exchange A[i] with A[largest]10 Max-Heapify(A, largest)
Figure: A trickle down algorithm
15 / 109
Images/ITESM.png
Heap Sort: Max-HeapifyAlgorithm (preserving the heap property) when somebody violates themax/min propertyMax-Heapify(A, i)
1 l = Left(i)2 r = Right(i)3 If l ≤ heap − size [A] and A [l] > A [i]4 largest = l5 else largest = i6 If r ≤ heap − size [A] and A [r ] > A [largest]7 largest = r8 if largest 6= i9 exchange A[i] with A[largest]10 Max-Heapify(A, largest)
Figure: A trickle down algorithm
15 / 109
Images/ITESM.png
Heap Sort: Max-HeapifyAlgorithm (preserving the heap property) when somebody violates themax/min propertyMax-Heapify(A, i)
1 l = Left(i)2 r = Right(i)3 If l ≤ heap − size [A] and A [l] > A [i]4 largest = l5 else largest = i6 If r ≤ heap − size [A] and A [r ] > A [largest]7 largest = r8 if largest 6= i9 exchange A[i] with A[largest]10 Max-Heapify(A, largest)
Figure: A trickle down algorithm
15 / 109
Images/ITESM.png
Heap Sort: Max-HeapifyAlgorithm (preserving the heap property) when somebody violates themax/min propertyMax-Heapify(A, i)
1 l = Left(i)2 r = Right(i)3 If l ≤ heap − size [A] and A [l] > A [i]4 largest = l5 else largest = i6 If r ≤ heap − size [A] and A [r ] > A [largest]7 largest = r8 if largest 6= i9 exchange A[i] with A[largest]10 Max-Heapify(A, largest)
Figure: A trickle down algorithm
15 / 109
Images/ITESM.png
Example keeping the heap property starting at i = 1Here, you could imagine that somebody inserted a node at i = 13. If l ≤ heap − size [A] and A [l] > A [i]4 largest = l5 else largest = i6 If r ≤ heap − size [A] and A [r ] > A [largest]7 largest = r
16 10
4
14 7 9 3
8 1 1
i=1
l=2 r=3
4 5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
4 16 10
Violating Property
14 7 9 3 8 1 1
>? <?
16 / 109
Images/ITESM.png
Example keeping the heap property starting at i = 1
One of the children is chosen to be exchanged8. if largest 6= i9. exchange A[i] with A[largest]
16 10
4
14 7 9 3
8 1 1
4 5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
4 16 10
Violating Property
Larger
14 7 9 3 8 1 1
Exchange
l=2
i=1
r=3
17 / 109
Images/ITESM.png
Example: Now i = largest
Make the excahnge and call the Max-Heapify10. Max-Heapify(A, largest)
16
104
14 7 9 3
8 1 1
4 5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
416 10 14 7 9 3 8 1 1
i=2 3Violating property?
1
18 / 109
Images/ITESM.png
Example: Now i = largest
Keep going
16
104
14 7 9 3
8 1 1
4 5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
416 10 14 7 9 3 8 1 1
i=2 3Violating property?
1
19 / 109
Images/ITESM.png
Example: Now i = largest
Keep going
16
104
14 7 9 3
8 1 1
4 5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
416 10 14 7 9 3 8 1 1
i=2 3Violating property?
1
20 / 109
Images/ITESM.png
Example: Now i = largest
Keep going
16
104
14 7 9 3
8 1 1
l=4 r=5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
416 10 14 7 9 3 8 1 1
i=2 3Violating property?
1
>
21 / 109
Images/ITESM.png
Example: Now i = largest
Keep going
16
10
4
14
7 9 3
8 1 1
i=4 5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
416 10 7 9 3 8 1 1
2 3
1
Exchange
14
22 / 109
Images/ITESM.png
Example: Now i = largest
Keep going
16
10
4
14
7 9 3
8 1 1
i=4 5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
416 10 7 9 3 8 1 1
2 3
1
14
>
Violating property?
23 / 109
Images/ITESM.png
Example: Now i = largest
Keep going
16
10
4
14
7 9 38
1 1
4 5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
16 10 7 9 38 1 1
2 3
1
14 4
24 / 109
Images/ITESM.png
Complexity of Max-Heapify
For thisIt is possible to prove that the upper bound on the size of each children’ssubtrees is 2n
3 starting at the root (First Recursive Call!!!).
ThusIn addition, we use the idea of height from the root node (h = 0) to leaves(h = log n − 1).
25 / 109
Images/ITESM.png
Complexity of Max-Heapify
For thisIt is possible to prove that the upper bound on the size of each children’ssubtrees is 2n
3 starting at the root (First Recursive Call!!!).
ThusIn addition, we use the idea of height from the root node (h = 0) to leaves(h = log n − 1).
25 / 109
Images/ITESM.png
Then
We have that by using the nearly complete structure1 First for n = 1, we have that the size of children’s subtrees is 0 < 2
3 .2 For n = 2, we have that the size of children’s subtrees is at most
1 < 43 .
3 For n = 3, we have that the size of children’s subtrees is at most1 < 6
3 = 2.4 For n = 4, we have that the size of children’s subtrees is at most
2 < 83 .
5 etc...
26 / 109
Images/ITESM.png
Then
We have that by using the nearly complete structure1 First for n = 1, we have that the size of children’s subtrees is 0 < 2
3 .2 For n = 2, we have that the size of children’s subtrees is at most
1 < 43 .
3 For n = 3, we have that the size of children’s subtrees is at most1 < 6
3 = 2.4 For n = 4, we have that the size of children’s subtrees is at most
2 < 83 .
5 etc...
26 / 109
Images/ITESM.png
Then
We have that by using the nearly complete structure1 First for n = 1, we have that the size of children’s subtrees is 0 < 2
3 .2 For n = 2, we have that the size of children’s subtrees is at most
1 < 43 .
3 For n = 3, we have that the size of children’s subtrees is at most1 < 6
3 = 2.4 For n = 4, we have that the size of children’s subtrees is at most
2 < 83 .
5 etc...
26 / 109
Images/ITESM.png
Then
We have that by using the nearly complete structure1 First for n = 1, we have that the size of children’s subtrees is 0 < 2
3 .2 For n = 2, we have that the size of children’s subtrees is at most
1 < 43 .
3 For n = 3, we have that the size of children’s subtrees is at most1 < 6
3 = 2.4 For n = 4, we have that the size of children’s subtrees is at most
2 < 83 .
5 etc...
26 / 109
Images/ITESM.png
Then
We have that by using the nearly complete structure1 First for n = 1, we have that the size of children’s subtrees is 0 < 2
3 .2 For n = 2, we have that the size of children’s subtrees is at most
1 < 43 .
3 For n = 3, we have that the size of children’s subtrees is at most1 < 6
3 = 2.4 For n = 4, we have that the size of children’s subtrees is at most
2 < 83 .
5 etc...
26 / 109
Images/ITESM.png
Do you notice the following?
Imagine the following case
The maximum number of nodes in both children assuming a full treewith n nodes
21 + 22 + ... + 2dlog ne−2 + 2dlog ne−1 (2)
27 / 109
Images/ITESM.png
Do you notice the following?
Imagine the following case
The maximum number of nodes in both children assuming a full treewith n nodes
21 + 22 + ... + 2dlog ne−2 + 2dlog ne−1 (2)
27 / 109
Images/ITESM.png
Now
Imagine the following special case
The maximum number of nodes in one child is equal to21 + 22 + ... + 2dlog ne−2
2 + 2dlog ne−1
2 (3)
28 / 109
Images/ITESM.png
Now
Imagine the following special case
The maximum number of nodes in one child is equal to21 + 22 + ... + 2dlog ne−2
2 + 2dlog ne−1
2 (3)
28 / 109
Images/ITESM.png
The total number of elements in a child’s subtree
The total number of nodes in a CHILD is bounded
21 + 22 + ... + 2dlog ne−2
2 + 2dlog ne−2 = 1 + 22 + ... + 2dlog ne−3 + 2dlog ne−2
= 1− 2dlog ne−2
1− 2 + 2dlog ne−2
= 2dlog ne−2 − 1 + 2dlog ne−2
= 2× 2blog nc−2 − 1
= 2dlog ne−1 − 23 −
13
< 2dlog ne−1 − 23
29 / 109
Images/ITESM.png
The total number of elements in a child’s subtree
The total number of nodes in a CHILD is bounded
21 + 22 + ... + 2dlog ne−2
2 + 2dlog ne−2 = 1 + 22 + ... + 2dlog ne−3 + 2dlog ne−2
= 1− 2dlog ne−2
1− 2 + 2dlog ne−2
= 2dlog ne−2 − 1 + 2dlog ne−2
= 2× 2blog nc−2 − 1
= 2dlog ne−1 − 23 −
13
< 2dlog ne−1 − 23
29 / 109
Images/ITESM.png
The total number of elements in a child’s subtree
The total number of nodes in a CHILD is bounded
21 + 22 + ... + 2dlog ne−2
2 + 2dlog ne−2 = 1 + 22 + ... + 2dlog ne−3 + 2dlog ne−2
= 1− 2dlog ne−2
1− 2 + 2dlog ne−2
= 2dlog ne−2 − 1 + 2dlog ne−2
= 2× 2blog nc−2 − 1
= 2dlog ne−1 − 23 −
13
< 2dlog ne−1 − 23
29 / 109
Images/ITESM.png
The total number of elements in a child’s subtree
The total number of nodes in a CHILD is bounded
21 + 22 + ... + 2dlog ne−2
2 + 2dlog ne−2 = 1 + 22 + ... + 2dlog ne−3 + 2dlog ne−2
= 1− 2dlog ne−2
1− 2 + 2dlog ne−2
= 2dlog ne−2 − 1 + 2dlog ne−2
= 2× 2blog nc−2 − 1
= 2dlog ne−1 − 23 −
13
< 2dlog ne−1 − 23
29 / 109
Images/ITESM.png
The total number of elements in a child’s subtree
The total number of nodes in a CHILD is bounded
21 + 22 + ... + 2dlog ne−2
2 + 2dlog ne−2 = 1 + 22 + ... + 2dlog ne−3 + 2dlog ne−2
= 1− 2dlog ne−2
1− 2 + 2dlog ne−2
= 2dlog ne−2 − 1 + 2dlog ne−2
= 2× 2blog nc−2 − 1
= 2dlog ne−1 − 23 −
13
< 2dlog ne−1 − 23
29 / 109
Images/ITESM.png
The total number of elements in a child’s subtree
The total number of nodes in a CHILD is bounded
21 + 22 + ... + 2dlog ne−2
2 + 2dlog ne−2 = 1 + 22 + ... + 2dlog ne−3 + 2dlog ne−2
= 1− 2dlog ne−2
1− 2 + 2dlog ne−2
= 2dlog ne−2 − 1 + 2dlog ne−2
= 2× 2blog nc−2 − 1
= 2dlog ne−1 − 23 −
13
< 2dlog ne−1 − 23
29 / 109
Images/ITESM.png
The total number of elements in a child’s subtree
Notice the following
2dlog ne <43[2log n
](4)
When n = 2p − 1For the case that that n ≤ 2p − 1 we can use the fact thatdlog ne = p for some power of 2.
30 / 109
Images/ITESM.png
Induction to prove the previous statement
Step n = 1
2dlog 1e = 20 = 1 <43 × 2log 0 (5)
Assume is true for n
2dlog ne <43[2log n
](6)
31 / 109
Images/ITESM.png
Induction to prove the previous statement
Step n = 1
2dlog 1e = 20 = 1 <43 × 2log 0 (5)
Assume is true for n
2dlog ne <43[2log n
](6)
31 / 109
Images/ITESM.png
Induction to prove the previous statement
Now prove for n + 1
2dlog(n+1)e = 2dlog(2p−1+1)e
= 2dpe
= 2p
= 2log 2p
<43[2log 2p]
= 43[2log(n+1)
]
32 / 109
Images/ITESM.png
Induction to prove the previous statement
Now prove for n + 1
2dlog(n+1)e = 2dlog(2p−1+1)e
= 2dpe
= 2p
= 2log 2p
<43[2log 2p]
= 43[2log(n+1)
]
32 / 109
Images/ITESM.png
Induction to prove the previous statement
Now prove for n + 1
2dlog(n+1)e = 2dlog(2p−1+1)e
= 2dpe
= 2p
= 2log 2p
<43[2log 2p]
= 43[2log(n+1)
]
32 / 109
Images/ITESM.png
Induction to prove the previous statement
Now prove for n + 1
2dlog(n+1)e = 2dlog(2p−1+1)e
= 2dpe
= 2p
= 2log 2p
<43[2log 2p]
= 43[2log(n+1)
]
32 / 109
Images/ITESM.png
Induction to prove the previous statement
Now prove for n + 1
2dlog(n+1)e = 2dlog(2p−1+1)e
= 2dpe
= 2p
= 2log 2p
<43[2log 2p]
= 43[2log(n+1)
]
32 / 109
Images/ITESM.png
Induction to prove the previous statement
Now prove for n + 1
2dlog(n+1)e = 2dlog(2p−1+1)e
= 2dpe
= 2p
= 2log 2p
<43[2log 2p]
= 43[2log(n+1)
]
32 / 109
Images/ITESM.png
ThereforeWe have that
21 + 22 + ... + 2dlog ne−2
2 + 2dlog ne−2 < 2dlog ne−1 − 23
= 2dlog ne
2 − 23
<43[2log n−1
]− 2
3= 2
3[2× 2log n−1 − 1
]= 2
3[2log n − 1
]= 2
3 [n − 1]
<2n3
33 / 109
Images/ITESM.png
ThereforeWe have that
21 + 22 + ... + 2dlog ne−2
2 + 2dlog ne−2 < 2dlog ne−1 − 23
= 2dlog ne
2 − 23
<43[2log n−1
]− 2
3= 2
3[2× 2log n−1 − 1
]= 2
3[2log n − 1
]= 2
3 [n − 1]
<2n3
33 / 109
Images/ITESM.png
ThereforeWe have that
21 + 22 + ... + 2dlog ne−2
2 + 2dlog ne−2 < 2dlog ne−1 − 23
= 2dlog ne
2 − 23
<43[2log n−1
]− 2
3= 2
3[2× 2log n−1 − 1
]= 2
3[2log n − 1
]= 2
3 [n − 1]
<2n3
33 / 109
Images/ITESM.png
ThereforeWe have that
21 + 22 + ... + 2dlog ne−2
2 + 2dlog ne−2 < 2dlog ne−1 − 23
= 2dlog ne
2 − 23
<43[2log n−1
]− 2
3= 2
3[2× 2log n−1 − 1
]= 2
3[2log n − 1
]= 2
3 [n − 1]
<2n3
33 / 109
Images/ITESM.png
ThereforeWe have that
21 + 22 + ... + 2dlog ne−2
2 + 2dlog ne−2 < 2dlog ne−1 − 23
= 2dlog ne
2 − 23
<43[2log n−1
]− 2
3= 2
3[2× 2log n−1 − 1
]= 2
3[2log n − 1
]= 2
3 [n − 1]
<2n3
33 / 109
Images/ITESM.png
ThereforeWe have that
21 + 22 + ... + 2dlog ne−2
2 + 2dlog ne−2 < 2dlog ne−1 − 23
= 2dlog ne
2 − 23
<43[2log n−1
]− 2
3= 2
3[2× 2log n−1 − 1
]= 2
3[2log n − 1
]= 2
3 [n − 1]
<2n3
33 / 109
Images/ITESM.png
ThereforeWe have that
21 + 22 + ... + 2dlog ne−2
2 + 2dlog ne−2 < 2dlog ne−1 − 23
= 2dlog ne
2 − 23
<43[2log n−1
]− 2
3= 2
3[2× 2log n−1 − 1
]= 2
3[2log n − 1
]= 2
3 [n − 1]
<2n3
33 / 109
Images/ITESM.png
Complexity of Max-HeapifyKnowing that the number of nodes in any child is bounded by
2n3 (7)
Thus, given that T (n) represent the complexity of the Max-Heapify
T (n) = T (How many nodes will be touched by the recusrsion) + Θ (1)(8)
HereΘ (1) is the constant part of the algorithm before recursion.T (How many nodes will be touched by the recusrsion) =
T(∑ log2 n
2 −1i=1 3
).
How? 34 / 109
Images/ITESM.png
Complexity of Max-HeapifyKnowing that the number of nodes in any child is bounded by
2n3 (7)
Thus, given that T (n) represent the complexity of the Max-Heapify
T (n) = T (How many nodes will be touched by the recusrsion) + Θ (1)(8)
HereΘ (1) is the constant part of the algorithm before recursion.T (How many nodes will be touched by the recusrsion) =
T(∑ log2 n
2 −1i=1 3
).
How? 34 / 109
Images/ITESM.png
Complexity of Max-HeapifyKnowing that the number of nodes in any child is bounded by
2n3 (7)
Thus, given that T (n) represent the complexity of the Max-Heapify
T (n) = T (How many nodes will be touched by the recusrsion) + Θ (1)(8)
HereΘ (1) is the constant part of the algorithm before recursion.T (How many nodes will be touched by the recusrsion) =
T(∑ log2 n
2 −1i=1 3
).
How? 34 / 109
Images/ITESM.png
Complexity of Max-HeapifyKnowing that the number of nodes in any child is bounded by
2n3 (7)
Thus, given that T (n) represent the complexity of the Max-Heapify
T (n) = T (How many nodes will be touched by the recusrsion) + Θ (1)(8)
HereΘ (1) is the constant part of the algorithm before recursion.T (How many nodes will be touched by the recusrsion) =
T(∑ log2 n
2 −1i=1 3
).
How? 34 / 109
Images/ITESM.png
Complexity of Max-HeapifyKnowing that the number of nodes in any child is bounded by
2n3 (7)
Thus, given that T (n) represent the complexity of the Max-Heapify
T (n) = T (How many nodes will be touched by the recusrsion) + Θ (1)(8)
HereΘ (1) is the constant part of the algorithm before recursion.T (How many nodes will be touched by the recusrsion) =
T(∑ log2 n
2 −1i=1 3
).
How? 34 / 109
Images/ITESM.png
Complexity of Max-Heapify
The Recursion Idea
3
3
3
35 / 109
Images/ITESM.png
Complexity of Max-HeapifyThus
log2 n−12 −1∑i=1
3 = 3log2 n
2 − 13− 1 − 3
=
(3
12)log2 n
2 − 3
= nlog2
(3
12)
2 − 3
≤ n0.8
2
≤ 2n0.8
3≤ 2n
336 / 109
Images/ITESM.png
Complexity of Max-HeapifyThus
log2 n−12 −1∑i=1
3 = 3log2 n
2 − 13− 1 − 3
=
(3
12)log2 n
2 − 3
= nlog2
(3
12)
2 − 3
≤ n0.8
2
≤ 2n0.8
3≤ 2n
336 / 109
Images/ITESM.png
Complexity of Max-HeapifyThus
log2 n−12 −1∑i=1
3 = 3log2 n
2 − 13− 1 − 3
=
(3
12)log2 n
2 − 3
= nlog2
(3
12)
2 − 3
≤ n0.8
2
≤ 2n0.8
3≤ 2n
336 / 109
Images/ITESM.png
Complexity of Max-HeapifyThus
log2 n−12 −1∑i=1
3 = 3log2 n
2 − 13− 1 − 3
=
(3
12)log2 n
2 − 3
= nlog2
(3
12)
2 − 3
≤ n0.8
2
≤ 2n0.8
3≤ 2n
336 / 109
Images/ITESM.png
Complexity of Max-HeapifyThus
log2 n−12 −1∑i=1
3 = 3log2 n
2 − 13− 1 − 3
=
(3
12)log2 n
2 − 3
= nlog2
(3
12)
2 − 3
≤ n0.8
2
≤ 2n0.8
3≤ 2n
336 / 109
Images/ITESM.png
Complexity of Max-HeapifyThus
log2 n−12 −1∑i=1
3 = 3log2 n
2 − 13− 1 − 3
=
(3
12)log2 n
2 − 3
= nlog2
(3
12)
2 − 3
≤ n0.8
2
≤ 2n0.8
3≤ 2n
336 / 109
Images/ITESM.png
Complexity of Max-Heapify
Thus, if we assume that T is an increasing monotone function
T (n) = T
log2 n
2 −1∑i=1
3
+ Θ (1)
≤ T(2n
3
)+ Θ (1)
Algorithm Complexity
This is by the master the master theorem O (log2 n).
37 / 109
Images/ITESM.png
Complexity of Max-Heapify
Thus, if we assume that T is an increasing monotone function
T (n) = T
log2 n
2 −1∑i=1
3
+ Θ (1)
≤ T(2n
3
)+ Θ (1)
Algorithm Complexity
This is by the master the master theorem O (log2 n).
37 / 109
Images/ITESM.png
Complexity of Max-Heapify
Thus, if we assume that T is an increasing monotone function
T (n) = T
log2 n
2 −1∑i=1
3
+ Θ (1)
≤ T(2n
3
)+ Θ (1)
Algorithm Complexity
This is by the master the master theorem O (log2 n).
37 / 109
Images/ITESM.png
Outline1 Sorting problem
DefinitionClassic Complexities
2 Heap SortHeap Sort: DefinitionsHeap: Finding Parents and ChildrenMax-HeapifyBuild Max Heap: Using Max-HeapifyHeap Sort
3 Applications of Heap Data StructureHeap Sort: Exercises
4 QuicksortThe Divide and Conquer QuicksortQuicksort: Complexity AnalysisRandomized Quicksort
5 Lower Bounds of SortingLower Bounds of Sorting: Basic Concepts
38 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Algorithm Build-Max-HeapBuild-Max-Heap(A)
1 heap − size[A] = length[A]2 for i = blength[A]/2c downto 13 Max-Heapify(A, i)
Figure: Building a Heap
39 / 109
Images/ITESM.png
Question?Why from blength[A]/2c?Look at this
1
2 3
4 5 6 7
8 9 10 11
Thus, the nodes blength[A]/2c+ 1, blength[A]/2c+ 2, ..., nThey are actually leaves.This can be proved by induction on n!!!
I I leave this to you.40 / 109
Images/ITESM.png
Question?Why from blength[A]/2c?Look at this
1
2 3
4 5 6 7
8 9 10 11
Thus, the nodes blength[A]/2c+ 1, blength[A]/2c+ 2, ..., nThey are actually leaves.This can be proved by induction on n!!!
I I leave this to you.40 / 109
Images/ITESM.png
Question?Why from blength[A]/2c?Look at this
1
2 3
4 5 6 7
8 9 10 11
Thus, the nodes blength[A]/2c+ 1, blength[A]/2c+ 2, ..., nThey are actually leaves.This can be proved by induction on n!!!
I I leave this to you.40 / 109
Images/ITESM.png
Question?Why from blength[A]/2c?Look at this
1
2 3
4 5 6 7
8 9 10 11
Thus, the nodes blength[A]/2c+ 1, blength[A]/2c+ 2, ..., nThey are actually leaves.This can be proved by induction on n!!!
I I leave this to you.40 / 109
Images/ITESM.png
Question?
What about the loop invariance?Look at the Board!!!
41 / 109
Images/ITESM.png
Build Max Heap: Using Max-Heapify
Example
16
10
4
14
7
9
3
8
1
1
4 i=5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
161079 3 811
2 3
1
144
Exchange
42 / 109
Images/ITESM.png
Build Max Heap: Using Max-Heapify
Example
16 10
4
14 7
9
3
8
1
1
4 i=5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
16 10 79 3 811
2 3
1
144
Exchange
43 / 109
Images/ITESM.png
Build Max Heap: Using Max-Heapify
Example
16 10
4
14 7
9
3
8
1
1
i=4 5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
16 10 79 3 811
2 3
1
144
Exchange
44 / 109
Images/ITESM.png
Build Max Heap: Using Max-Heapify
Example
16 10
4
14
7
9
3
81
1
i=4 5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
16 10 79 3 811
2 3
1
144
Exchange
45 / 109
Images/ITESM.png
Build Max Heap: Using Max-Heapify
Example
16 10
4
14
7
9
3
81
1
4 5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
16 10 79 3 811
2 i=3
1
144
Exchange
46 / 109
Images/ITESM.png
Build Max Heap: Using Max-Heapify
Example
16
104
14
7
9 3
81
1
4 5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
1610 79 3 811
2 i=3
1
144
Exchange
47 / 109
Images/ITESM.png
Build Max Heap: Using Max-Heapify
Example
16
104
14
7
9 3
81
1
4 5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
1610 79 3 811
i=2 3
1
144
Exchange
48 / 109
Images/ITESM.png
Build Max Heap: Using Max-Heapify
Example
16 10
4
14 7 9 3
81
1
4 5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
16 10 7 9 3 811
i=2 3
1
14 4
49 / 109
Images/ITESM.png
Build Max Heap: Using Max-Heapify
Example
16 10
4
14 7 9 3
81
1
4 5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
16 10 7 9 3 811
2 3
i=1
14 4
Exchange
50 / 109
Images/ITESM.png
Build Max Heap: Using Max-Heapify
Example
16
10
4
14 7 9 3
81
1
4 5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
16 10 7 9 3 811
2 3
i=1
14 4
Exchange
51 / 109
Images/ITESM.png
Build Max Heap: Using Max-Heapify
Example
16
10
4
14
7 9 3
81
1
4 5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
16 10 7 9 3 811
2 3
i=1
14 4
Exchange
52 / 109
Images/ITESM.png
Build Max Heap: Using Max-Heapify
Example
16
10
4
14
7 9 38
1 1
4 5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
16 10 7 9 38 1 1
2 3
i=1
14 4
53 / 109
Images/ITESM.png
Height h of the Heap for Complexity of Build-Max-Heap
We can use the height of a three to derive a tight boundThe height h is the number of edges on the longest simple downwardpath from the node to a leaf.You have at most
⌈n
2h+1
⌉nodes at any height, where n is the total
number of nodes.
54 / 109
Images/ITESM.png
Height h of the Heap for Complexity of Build-Max-Heap
We can use the height of a three to derive a tight boundThe height h is the number of edges on the longest simple downwardpath from the node to a leaf.You have at most
⌈n
2h+1
⌉nodes at any height, where n is the total
number of nodes.
54 / 109
Images/ITESM.png
Height h of the Heap for Complexity of Build-Max-Heap
We can use the height of a three to derive a tight boundThe height h is the number of edges on the longest simple downwardpath from the node to a leaf.You have at most
⌈n
2h+1
⌉nodes at any height, where n is the total
number of nodes.
8
7 3
4 5 1
h=2
h=1
h=0
54 / 109
Images/ITESM.png
Cost of Building the Build-Max-Heap
Possible cost
O (n log2 n) (9)
55 / 109
Images/ITESM.png
Cost of Building the Build-Max-HeapWe have that you have
The number of nodes explored horizontally by the “for” loop can bebounded by ⌈ n
2h+1
⌉(10)
The depth of the Max-Heapify is
O (h) (11)
Therefore we have the following total tighter cost
blog nc∑h=0
⌈ n2h+1
⌉O(h) = O
nblog nc∑
h=0
h2h
56 / 109
Images/ITESM.png
Cost of Building the Build-Max-HeapWe have that you have
The number of nodes explored horizontally by the “for” loop can bebounded by ⌈ n
2h+1
⌉(10)
The depth of the Max-Heapify is
O (h) (11)
Therefore we have the following total tighter cost
blog nc∑h=0
⌈ n2h+1
⌉O(h) = O
nblog nc∑
h=0
h2h
56 / 109
Images/ITESM.png
Cost of Building the Build-Max-HeapWe have that you have
The number of nodes explored horizontally by the “for” loop can bebounded by ⌈ n
2h+1
⌉(10)
The depth of the Max-Heapify is
O (h) (11)
Therefore we have the following total tighter cost
blog nc∑h=0
⌈ n2h+1
⌉O(h) = O
nblog nc∑
h=0
h2h
56 / 109
Images/ITESM.png
Cost of Building the Build-Max-HeapWe have that you have
The number of nodes explored horizontally by the “for” loop can bebounded by ⌈ n
2h+1
⌉(10)
The depth of the Max-Heapify is
O (h) (11)
Therefore we have the following total tighter cost
blog nc∑h=0
⌈ n2h+1
⌉O(h) = O
nblog nc∑
h=0
h2h
56 / 109
Images/ITESM.png
Cost of Building the Build-Max-HeapWe have that you have
The number of nodes explored horizontally by the “for” loop can bebounded by ⌈ n
2h+1
⌉(10)
The depth of the Max-Heapify is
O (h) (11)
Therefore we have the following total tighter cost
blog nc∑h=0
⌈ n2h+1
⌉O(h) = O
nblog nc∑
h=0
h2h
56 / 109
Images/ITESM.png
ThusFrom (A.8) at Cormen’s
∞∑k=0
kxk = x(1− x)2
Thus, we have that
O
nblog nc∑
h=0
h2h
= O(
n∞∑
h=0
h2h
)
= O(
n∞∑
h=0h(1
2
)h)
= O
n
12(
1− 12
)2
= O (n)57 / 109
Images/ITESM.png
ThusFrom (A.8) at Cormen’s
∞∑k=0
kxk = x(1− x)2
Thus, we have that
O
nblog nc∑
h=0
h2h
= O(
n∞∑
h=0
h2h
)
= O(
n∞∑
h=0h(1
2
)h)
= O
n
12(
1− 12
)2
= O (n)57 / 109
Images/ITESM.png
ThusFrom (A.8) at Cormen’s
∞∑k=0
kxk = x(1− x)2
Thus, we have that
O
nblog nc∑
h=0
h2h
= O(
n∞∑
h=0
h2h
)
= O(
n∞∑
h=0h(1
2
)h)
= O
n
12(
1− 12
)2
= O (n)57 / 109
Images/ITESM.png
ThusFrom (A.8) at Cormen’s
∞∑k=0
kxk = x(1− x)2
Thus, we have that
O
nblog nc∑
h=0
h2h
= O(
n∞∑
h=0
h2h
)
= O(
n∞∑
h=0h(1
2
)h)
= O
n
12(
1− 12
)2
= O (n)57 / 109
Images/ITESM.png
ThusFrom (A.8) at Cormen’s
∞∑k=0
kxk = x(1− x)2
Thus, we have that
O
nblog nc∑
h=0
h2h
= O(
n∞∑
h=0
h2h
)
= O(
n∞∑
h=0h(1
2
)h)
= O
n
12(
1− 12
)2
= O (n)57 / 109
Images/ITESM.png
Outline1 Sorting problem
DefinitionClassic Complexities
2 Heap SortHeap Sort: DefinitionsHeap: Finding Parents and ChildrenMax-HeapifyBuild Max Heap: Using Max-HeapifyHeap Sort
3 Applications of Heap Data StructureHeap Sort: Exercises
4 QuicksortThe Divide and Conquer QuicksortQuicksort: Complexity AnalysisRandomized Quicksort
5 Lower Bounds of SortingLower Bounds of Sorting: Basic Concepts
58 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Heapsort AlgorithmHeapsort(A)
1 Build-Max-Heap(A)2 for i = length[A] downto 23 exchange A[1] with A[i]4 heap − size[A] = heap − size[A]− 15 Max-Heapify(A, 1)
Figure: Heapsort
59 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Heapsort AlgorithmHeapsort(A)
1 Build-Max-Heap(A)2 for i = length[A] downto 23 exchange A[1] with A[i]4 heap − size[A] = heap − size[A]− 15 Max-Heapify(A, 1)
Figure: Heapsort
59 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Heapsort AlgorithmHeapsort(A)
1 Build-Max-Heap(A)2 for i = length[A] downto 23 exchange A[1] with A[i]4 heap − size[A] = heap − size[A]− 15 Max-Heapify(A, 1)
Figure: Heapsort
59 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Heapsort AlgorithmHeapsort(A)
1 Build-Max-Heap(A)2 for i = length[A] downto 23 exchange A[1] with A[i]4 heap − size[A] = heap − size[A]− 15 Max-Heapify(A, 1)
Figure: Heapsort
59 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Heapsort AlgorithmHeapsort(A)
1 Build-Max-Heap(A)2 for i = length[A] downto 23 exchange A[1] with A[i]4 heap − size[A] = heap − size[A]− 15 Max-Heapify(A, 1)
Figure: Heapsort
59 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Heapsort AlgorithmHeapsort(A)
1 Build-Max-Heap(A)2 for i = length[A] downto 23 exchange A[1] with A[i]4 heap − size[A] = heap − size[A]− 15 Max-Heapify(A, 1)
Figure: Heapsort
59 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Heapsort AlgorithmHeapsort(A)
1 Build-Max-Heap(A)2 for i = length[A] downto 23 exchange A[1] with A[i]4 heap − size[A] = heap − size[A]− 15 Max-Heapify(A, 1)
Figure: Heapsort
59 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Example: Heapsort in action! By Moving the top element to thebottom position!!!
To be exchanged
16
10
4
14
7 9 38
1 1
4 5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
16 10 7 9 38 1 1
2 3
i=1
14 4
60 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Example: Heapsort in action! By Moving the top element to thebottom position!!!
16
10
4
14
7 9 38
1 1
4 5 6 7
8 9 i=10
0 1 2 3 4 5 6 7 8 9 10
10 7 9 38 1 1
2 3
1
144 16
Max-Heapify(A,1)
61 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Example: Heapsort in action! By Moving the top element to thebottom position!!!
16
104
14
7 9 38
1 1
4 5 6 7
8 9 i=10
0 1 2 3 4 5 6 7 8 9 10
10 7 9 38 1 1
2 3
1
14 4 16
To be exchanged
62 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Example: Heapsort in action! By Moving the top element to thebottom position!!!
16
10
4
14
7 9 3
8
1 1
4 5 6 7
8 9 i=10
0 1 2 3 4 5 6 7 8 9 10
10 7 9 38 1 1
2 3
1
14 4 16
63 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Example: Heapsort in action! By Moving the top element to thebottom position!!!
16
10
4
14
7 9 3
8
1 1
4 5 6 7
8 9 i=10
0 1 2 3 4 5 6 7 8 9 10
10 7 9 38 1 1
2 3
1
14 4 16
To be exchanged
64 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Example: Heapsort in action! By Moving the top element to thebottom position!!!
16
10
4
14
7 9 3
8
1
1
4 5 6 7
8 i=9 10
0 1 2 3 4 5 6 7 8 9 10
10 7 9 38 11
2 3
1
144 16
To be exchangedMax-Heapify(A,1)
65 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Example: Heapsort in action! By Moving the top element to thebottom position!!!
16
10
4
14
7 9 3
8
1
1
4 5 6 7
8 i=9 10
0 1 2 3 4 5 6 7 8 9 10
7 9 38 11
2 3
1
144 16
To be exchanged
10
66 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Example: Heapsort in action! By Moving the top element to thebottom position!!!
16
10
4
14
7
9
3
8
1
1
4 5 6 7
8 i=9 10
0 1 2 3 4 5 6 7 8 9 10
7 9 38 11
2 3
1
144 16
To be exchanged
10
67 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Example: Heapsort in action! By Moving the top element to thebottom position!!!
1610
4
14
7
9
3
8
1
1
4 5 6 7
i=8 9 10
0 1 2 3 4 5 6 7 8 9 10
7 9 381 1
2 3
1
144 16
To be exchanged
10
Max-Heapify(A,1)
68 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Example: Heapsort in action! By Moving the top element to thebottom position!!!
1610
4
14
7
9
3
8 1
1
4 5 6 7
i=8 9 10
0 1 2 3 4 5 6 7 8 9 10
79 38 11
2 3
1
144 16
To be exchanged
10
69 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Example: Heapsort in action! By Moving the top element to thebottom position!!!
1610
4
14
7
9
38
11
4 5 6 7
i=8 9 10
0 1 2 3 4 5 6 7 8 9 10
79 38 11
2 3
1
144 16
To be exchanged
10
70 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Example: Heapsort in action! By Moving the top element to thebottom position!!!
1610
4
14
7 9
38
1
1
4 5 6 i=7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
7 9381 1
2 3
1
144 16
To be exchanged
10
Max-Heapify(A,1)
71 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Example: Heapsort in action! By Moving the top element to thebottom position!!!
1610
4
14
7 9
3
8
1
1
4 5 6 i=7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
7 938 1 1
2 3
1
144 16
To be exchanged
10
72 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Example: Heapsort in action! By Moving the top element to thebottom position!!!
1610
4
14
7
9
3
8
1 1
4 5 6 i=7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
7 938 1 1
2 3
1
144 16
To be exchanged
10
73 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Example: Heapsort in action! By Moving the top element to thebottom position!!!
1610
4
14
7
9
3
81
1
4 5 i=6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
7 93 811
2 3
1
144 16
To be exchanged
10
Max-Heapify(A,1)
74 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Example: Heapsort in action! By Moving the top element to thebottom position!!!
1610
4
14
7
9
3
81
1
4 5 i=6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
7 93 811
2 3
1
144 16
To be exchanged
10
75 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Example: Heapsort in action! By Moving the top element to thebottom position!!!
1610
4
14
7
9
3
811
4 5 i=6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
7 93 811
2 3
1
144 16
To be exchanged
10
76 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Example: Heapsort in action! By Moving the top element to thebottom position!!!
1610
4
14
7 9
3
8
1
1
4 i=5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
7 93 81 1
2 3
1
144 16
To be exchanged
10
Max-Heapify(A,1)
77 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Example: Heapsort in action! By Moving the top element to thebottom position!!!
1610
4
14
7 9
3
8
1
1
4 i=5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
7 93 811
2 3
1
144 1610
To be exchanged
78 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Example: Heapsort in action! By Moving the top element to thebottom position!!!
1610
4
14
7 9
3
8
1
1
i=4 5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
7 93 81 1
2 3
1
144 1610
To be exchangedMax-Heapify(A,1)
79 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Example: Heapsort in action! By Moving the top element to thebottom position!!!
1610
4
14
7 9
3
8
1 1
i=4 5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
7 93 811
2 3
1
144 1610
To be exchanged
80 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Example: Heapsort in action! By Moving the top element to thebottom position!!!
1610
4
14
7 9
3
8
1
1
4 5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
7 93 81 1
2 i=3
1
144 1610
Max-Heapify(A,1)
81 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Example: Heapsort in action! By Moving the top element to thebottom position!!!
1610
4
14
7 9
3
8
1
1
4 5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
7 93 81 1
i=2 3
1
144 1610
Max-Heapify(A,1)
82 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
Example: Heapsort in action! By Moving the top element to thebottom position!!!
1610
4
14
7 9
3
8
1
1
4 5 6 7
8 9 10
0 1 2 3 4 5 6 7 8 9 10
7 93 81 1
2 3
i=1
144 1610
Max-Heapify(A,1)
83 / 109
Images/ITESM.png
Heap Sort: Using Max-Heapify
CostO(n log n)
84 / 109
Images/ITESM.png
Applications of Heap Data Structure
Priority QueuesHere, Heaps can be modified to support insert(), delete() and extractmax(),decreaseKey() operations in O(logn) time
This has direct applications1 Bandwidth management:
1 Many modern protocols for Local Area Networks include the concept ofPriority Queues at the Media Access Control (MAC).
2 Discrete Event Simulations3 Schedulers4 Huffman coding5 The Real-time Optimally Adapting Meshes (ROAM)
1 It computes a dynamically changing triangulation of a terrain using twopriority queues.
85 / 109
Images/ITESM.png
Applications of Heap Data Structure
Priority QueuesHere, Heaps can be modified to support insert(), delete() and extractmax(),decreaseKey() operations in O(logn) time
This has direct applications1 Bandwidth management:
1 Many modern protocols for Local Area Networks include the concept ofPriority Queues at the Media Access Control (MAC).
2 Discrete Event Simulations3 Schedulers4 Huffman coding5 The Real-time Optimally Adapting Meshes (ROAM)
1 It computes a dynamically changing triangulation of a terrain using twopriority queues.
85 / 109
Images/ITESM.png
Applications of Heap Data Structure
Priority QueuesHere, Heaps can be modified to support insert(), delete() and extractmax(),decreaseKey() operations in O(logn) time
This has direct applications1 Bandwidth management:
1 Many modern protocols for Local Area Networks include the concept ofPriority Queues at the Media Access Control (MAC).
2 Discrete Event Simulations3 Schedulers4 Huffman coding5 The Real-time Optimally Adapting Meshes (ROAM)
1 It computes a dynamically changing triangulation of a terrain using twopriority queues.
85 / 109
Images/ITESM.png
Applications of Heap Data Structure
Priority QueuesHere, Heaps can be modified to support insert(), delete() and extractmax(),decreaseKey() operations in O(logn) time
This has direct applications1 Bandwidth management:
1 Many modern protocols for Local Area Networks include the concept ofPriority Queues at the Media Access Control (MAC).
2 Discrete Event Simulations3 Schedulers4 Huffman coding5 The Real-time Optimally Adapting Meshes (ROAM)
1 It computes a dynamically changing triangulation of a terrain using twopriority queues.
85 / 109
Images/ITESM.png
Applications of Heap Data Structure
Priority QueuesHere, Heaps can be modified to support insert(), delete() and extractmax(),decreaseKey() operations in O(logn) time
This has direct applications1 Bandwidth management:
1 Many modern protocols for Local Area Networks include the concept ofPriority Queues at the Media Access Control (MAC).
2 Discrete Event Simulations3 Schedulers4 Huffman coding5 The Real-time Optimally Adapting Meshes (ROAM)
1 It computes a dynamically changing triangulation of a terrain using twopriority queues.
85 / 109
Images/ITESM.png
Applications of Heap Data Structure
Priority QueuesHere, Heaps can be modified to support insert(), delete() and extractmax(),decreaseKey() operations in O(logn) time
This has direct applications1 Bandwidth management:
1 Many modern protocols for Local Area Networks include the concept ofPriority Queues at the Media Access Control (MAC).
2 Discrete Event Simulations3 Schedulers4 Huffman coding5 The Real-time Optimally Adapting Meshes (ROAM)
1 It computes a dynamically changing triangulation of a terrain using twopriority queues.
85 / 109
Images/ITESM.png
Applications of Heap Data Structure
Heap Sort of ArraysClearly, if the list of numbers is stored in an array!!!
86 / 109
Images/ITESM.png
Outline1 Sorting problem
DefinitionClassic Complexities
2 Heap SortHeap Sort: DefinitionsHeap: Finding Parents and ChildrenMax-HeapifyBuild Max Heap: Using Max-HeapifyHeap Sort
3 Applications of Heap Data StructureHeap Sort: Exercises
4 QuicksortThe Divide and Conquer QuicksortQuicksort: Complexity AnalysisRandomized Quicksort
5 Lower Bounds of SortingLower Bounds of Sorting: Basic Concepts
87 / 109
Images/ITESM.png
Heap Sort: Exercices
From Cormen’s book6.1-16.1-46.1-76.2-56.2-66.3-36.4-26.4-36.4-4
88 / 109
Images/ITESM.png
Who invented Quicksort?
Imagine thisThe quicksort algorithm was developed in 1960 by Tony Hoare (He has apostgraduate certificate in Statistics) while in the Soviet Union, as avisiting student at Moscow State University.
Why?At that time, Hoare worked in a project on machine translation for theNational Physical Laboratory.
To doHe developed the algorithm in order to sort the words to be translated.
89 / 109
Images/ITESM.png
Who invented Quicksort?
Imagine thisThe quicksort algorithm was developed in 1960 by Tony Hoare (He has apostgraduate certificate in Statistics) while in the Soviet Union, as avisiting student at Moscow State University.
Why?At that time, Hoare worked in a project on machine translation for theNational Physical Laboratory.
To doHe developed the algorithm in order to sort the words to be translated.
89 / 109
Images/ITESM.png
Who invented Quicksort?
Imagine thisThe quicksort algorithm was developed in 1960 by Tony Hoare (He has apostgraduate certificate in Statistics) while in the Soviet Union, as avisiting student at Moscow State University.
Why?At that time, Hoare worked in a project on machine translation for theNational Physical Laboratory.
To doHe developed the algorithm in order to sort the words to be translated.
89 / 109
Images/ITESM.png
Outline1 Sorting problem
DefinitionClassic Complexities
2 Heap SortHeap Sort: DefinitionsHeap: Finding Parents and ChildrenMax-HeapifyBuild Max Heap: Using Max-HeapifyHeap Sort
3 Applications of Heap Data StructureHeap Sort: Exercises
4 QuicksortThe Divide and Conquer QuicksortQuicksort: Complexity AnalysisRandomized Quicksort
5 Lower Bounds of SortingLower Bounds of Sorting: Basic Concepts
90 / 109
Images/ITESM.png
The Divide and Conquer Quicksort
Divide Process1 Compute the index q as part of this partitioning procedure.2 Partition (rearrange) the array A[p, ..., r ] into two (possibly empty)
sub-arrays A[p, ..., q − 1] and A[q + 1, ..., r ]1 each element of A[p, ..., q − 1] is less than or equal to A[q].2 A[q] is less than or equal to each element of A[q + 1, ..., r ].
91 / 109
Images/ITESM.png
The Divide and Conquer Quicksort
Divide Process1 Compute the index q as part of this partitioning procedure.2 Partition (rearrange) the array A[p, ..., r ] into two (possibly empty)
sub-arrays A[p, ..., q − 1] and A[q + 1, ..., r ]1 each element of A[p, ..., q − 1] is less than or equal to A[q].2 A[q] is less than or equal to each element of A[q + 1, ..., r ].
91 / 109
Images/ITESM.png
The Divide and Conquer Quicksort
Divide Process1 Compute the index q as part of this partitioning procedure.2 Partition (rearrange) the array A[p, ..., r ] into two (possibly empty)
sub-arrays A[p, ..., q − 1] and A[q + 1, ..., r ]1 each element of A[p, ..., q − 1] is less than or equal to A[q].2 A[q] is less than or equal to each element of A[q + 1, ..., r ].
91 / 109
Images/ITESM.png
The Divide and Conquer Quicksort
Divide Process1 Compute the index q as part of this partitioning procedure.2 Partition (rearrange) the array A[p, ..., r ] into two (possibly empty)
sub-arrays A[p, ..., q − 1] and A[q + 1, ..., r ]1 each element of A[p, ..., q − 1] is less than or equal to A[q].2 A[q] is less than or equal to each element of A[q + 1, ..., r ].
91 / 109
Images/ITESM.png
The Divide and Conquer Quicksort
ConquerSort the two sub-arrays A[p, ..., q − 1] and A[q + 1, ..., r ] by recursive callsto quicksort.
CombineSince the sub-arrays are sorted in place, no work is needed to combinethem: the entire array A[p, ..., r ] is now sorted.
92 / 109
Images/ITESM.png
The Divide and Conquer Quicksort
ConquerSort the two sub-arrays A[p, ..., q − 1] and A[q + 1, ..., r ] by recursive callsto quicksort.
CombineSince the sub-arrays are sorted in place, no work is needed to combinethem: the entire array A[p, ..., r ] is now sorted.
92 / 109
Images/ITESM.png
Quicksort Algorithm
Quicksort AlgorithmQuicksort(A, p, r)
1 if p < r2 q = Partition (A, p, r)3 Quicksort(A, p, q − 1)4 Quicksort(A, q + 1, r)
93 / 109
Images/ITESM.png
Quicksort Algorithm
Quicksort AlgorithmQuicksort(A, p, r)
1 if p < r2 q = Partition (A, p, r)3 Quicksort(A, p, q − 1)4 Quicksort(A, q + 1, r)
93 / 109
Images/ITESM.png
Quicksort Algorithm
Quicksort AlgorithmQuicksort(A, p, r)
1 if p < r2 q = Partition (A, p, r)3 Quicksort(A, p, q − 1)4 Quicksort(A, q + 1, r)
93 / 109
Images/ITESM.png
Quicksort Algorithm
Quicksort AlgorithmQuicksort(A, p, r)
1 if p < r2 q = Partition (A, p, r)3 Quicksort(A, p, q − 1)4 Quicksort(A, q + 1, r)
93 / 109
Images/ITESM.png
Quicksort Algorithm
Quicksort AlgorithmQuicksort(A, p, r)
1 if p < r2 q = Partition (A, p, r)3 Quicksort(A, p, q − 1)4 Quicksort(A, q + 1, r)
93 / 109
Images/ITESM.png
Partition Algorithm
Quicksort PartitionPartition(A, p, r)
1 x = A[r ]2 i = p − 13 for j = p to r − 14 if A[j] ≤ x5 i = i + 16 exchange A[i] with A[j]7 exchange A[i + 1] with A[r ]8 return i + 1
94 / 109
Images/ITESM.png
Partition Algorithm
Quicksort PartitionPartition(A, p, r)
1 x = A[r ]2 i = p − 13 for j = p to r − 14 if A[j] ≤ x5 i = i + 16 exchange A[i] with A[j]7 exchange A[i + 1] with A[r ]8 return i + 1
94 / 109
Images/ITESM.png
Partition Algorithm
Quicksort PartitionPartition(A, p, r)
1 x = A[r ]2 i = p − 13 for j = p to r − 14 if A[j] ≤ x5 i = i + 16 exchange A[i] with A[j]7 exchange A[i + 1] with A[r ]8 return i + 1
94 / 109
Images/ITESM.png
Partition Algorithm
Quicksort PartitionPartition(A, p, r)
1 x = A[r ]2 i = p − 13 for j = p to r − 14 if A[j] ≤ x5 i = i + 16 exchange A[i] with A[j]7 exchange A[i + 1] with A[r ]8 return i + 1
94 / 109
Images/ITESM.png
Partition Algorithm
Quicksort PartitionPartition(A, p, r)
1 x = A[r ]2 i = p − 13 for j = p to r − 14 if A[j] ≤ x5 i = i + 16 exchange A[i] with A[j]7 exchange A[i + 1] with A[r ]8 return i + 1
94 / 109
Images/ITESM.png
Partition Algorithm
Quicksort PartitionPartition(A, p, r)
1 x = A[r ]2 i = p − 13 for j = p to r − 14 if A[j] ≤ x5 i = i + 16 exchange A[i] with A[j]7 exchange A[i + 1] with A[r ]8 return i + 1
94 / 109
Images/ITESM.png
Partition Algorithm
Quicksort PartitionPartition(A, p, r)
1 x = A[r ]2 i = p − 13 for j = p to r − 14 if A[j] ≤ x5 i = i + 16 exchange A[i] with A[j]7 exchange A[i + 1] with A[r ]8 return i + 1
94 / 109
Images/ITESM.png
Partition Algorithm
Quicksort PartitionPartition(A, p, r)
1 x = A[r ]2 i = p − 13 for j = p to r − 14 if A[j] ≤ x5 i = i + 16 exchange A[i] with A[j]7 exchange A[i + 1] with A[r ]8 return i + 1
94 / 109
Images/ITESM.png
Quicksort: What is the Invariance?
Loop Invariance1 If p ≤ k ≤ i, then A[k] ≤ x.2 If i + 1 ≤ k ≤ j − 1, then A[k] > x.3 If k = r , then A[k] = x.4 UNKNOWN
Proof of the Loop InvarianceLook at the Board.
95 / 109
Images/ITESM.png
Quicksort: What is the Invariance?
Loop Invariance1 If p ≤ k ≤ i, then A[k] ≤ x.2 If i + 1 ≤ k ≤ j − 1, then A[k] > x.3 If k = r , then A[k] = x.4 UNKNOWN
Proof of the Loop InvarianceLook at the Board.
95 / 109
Images/ITESM.png
Quicksort: What is the Invariance?
Loop Invariance1 If p ≤ k ≤ i, then A[k] ≤ x.2 If i + 1 ≤ k ≤ j − 1, then A[k] > x.3 If k = r , then A[k] = x.4 UNKNOWN
Proof of the Loop InvarianceLook at the Board.
95 / 109
Images/ITESM.png
Quicksort: What is the Invariance?
Loop Invariance1 If p ≤ k ≤ i, then A[k] ≤ x.2 If i + 1 ≤ k ≤ j − 1, then A[k] > x.3 If k = r , then A[k] = x.4 UNKNOWN
Proof of the Loop InvarianceLook at the Board.
95 / 109
Images/ITESM.png
Quicksort: What is the Invariance?
Loop Invariance1 If p ≤ k ≤ i, then A[k] ≤ x.2 If i + 1 ≤ k ≤ j − 1, then A[k] > x.3 If k = r , then A[k] = x.4 UNKNOWN
p i i+1 j-1
r
A[k]<x A[k]>x
Proof of the Loop InvarianceLook at the Board.
95 / 109
Images/ITESM.png
Quicksort: What is the Invariance?
Loop Invariance1 If p ≤ k ≤ i, then A[k] ≤ x.2 If i + 1 ≤ k ≤ j − 1, then A[k] > x.3 If k = r , then A[k] = x.4 UNKNOWN
p i i+1 j-1
r
A[k]<x A[k]>x
Proof of the Loop InvarianceLook at the Board.
95 / 109
Images/ITESM.png
Outline1 Sorting problem
DefinitionClassic Complexities
2 Heap SortHeap Sort: DefinitionsHeap: Finding Parents and ChildrenMax-HeapifyBuild Max Heap: Using Max-HeapifyHeap Sort
3 Applications of Heap Data StructureHeap Sort: Exercises
4 QuicksortThe Divide and Conquer QuicksortQuicksort: Complexity AnalysisRandomized Quicksort
5 Lower Bounds of SortingLower Bounds of Sorting: Basic Concepts
96 / 109
Images/ITESM.png
Quicksort: Complexity Analysis
Worst-case AnalysisPartition returns two arrays, one of size 0 and one of size n − 1. Then, wehave the recurrence:
T (n) = T (n − 1) + Θ(n) = O(n2). (12)
Best-case AnalysisPartition returns two arrays size n
2 and n2 − 1.
Then, we have the recurrence T (n) = 2T(n
2)
+ Θ(n).
97 / 109
Images/ITESM.png
Quicksort: Complexity Analysis
Worst-case AnalysisPartition returns two arrays, one of size 0 and one of size n − 1. Then, wehave the recurrence:
T (n) = T (n − 1) + Θ(n) = O(n2). (12)
Best-case AnalysisPartition returns two arrays size n
2 and n2 − 1.
Then, we have the recurrence T (n) = 2T(n
2)
+ Θ(n).
97 / 109
Images/ITESM.png
Quicksort: Complexity Analysis
Worst-case AnalysisPartition returns two arrays, one of size 0 and one of size n − 1. Then, wehave the recurrence:
T (n) = T (n − 1) + Θ(n) = O(n2). (12)
Best-case AnalysisPartition returns two arrays size n
2 and n2 − 1.
Then, we have the recurrence T (n) = 2T(n
2)
+ Θ(n).
97 / 109
Images/ITESM.png
Quicksort: Complexity Analysis
Worst-case AnalysisPartition returns two arrays, one of size 0 and one of size n − 1. Then, wehave the recurrence:
T (n) = T (n − 1) + Θ(n) = O(n2). (12)
Best-case AnalysisPartition returns two arrays size n
2 and n2 − 1.
Then, we have the recurrence T (n) = 2T(n
2)
+ Θ(n).
97 / 109
Images/ITESM.png
However for an Unbalanced Partition!!!
Unbalanced partitioning returns a O (n log n)After certain level, the total steps are ≤ than cn!!!
98 / 109
Images/ITESM.png
Quicksort: Complexity Analysis
Worst-case Analysis in detailFrom the recurrence: T (n) = max05q≤n−1(T (q) + T (n − q − 1)) + Θ(n)
By substitution, we can proveComplexity O(n2).
This can be proved as followsBLACKBOARD!
99 / 109
Images/ITESM.png
Quicksort: Complexity Analysis
Worst-case Analysis in detailFrom the recurrence: T (n) = max05q≤n−1(T (q) + T (n − q − 1)) + Θ(n)
By substitution, we can proveComplexity O(n2).
This can be proved as followsBLACKBOARD!
99 / 109
Images/ITESM.png
Quicksort: Complexity Analysis
Worst-case Analysis in detailFrom the recurrence: T (n) = max05q≤n−1(T (q) + T (n − q − 1)) + Θ(n)
By substitution, we can proveComplexity O(n2).
This can be proved as followsBLACKBOARD!
99 / 109
Images/ITESM.png
Outline1 Sorting problem
DefinitionClassic Complexities
2 Heap SortHeap Sort: DefinitionsHeap: Finding Parents and ChildrenMax-HeapifyBuild Max Heap: Using Max-HeapifyHeap Sort
3 Applications of Heap Data StructureHeap Sort: Exercises
4 QuicksortThe Divide and Conquer QuicksortQuicksort: Complexity AnalysisRandomized Quicksort
5 Lower Bounds of SortingLower Bounds of Sorting: Basic Concepts
100 / 109
Images/ITESM.png
Remember?
The use of uniform distributionTo get the average behaivior!!!
In many casesIt is better than the worst case scenario....
ThusWe can introduce randomization in the Quicksort.
101 / 109
Images/ITESM.png
Remember?
The use of uniform distributionTo get the average behaivior!!!
In many casesIt is better than the worst case scenario....
ThusWe can introduce randomization in the Quicksort.
101 / 109
Images/ITESM.png
Remember?
The use of uniform distributionTo get the average behaivior!!!
In many casesIt is better than the worst case scenario....
ThusWe can introduce randomization in the Quicksort.
101 / 109
Images/ITESM.png
Randomized Quicksort
RANDOMIZED-QUICKSORT(A,p,r)Randomized-Quicksort(A, p, r)
1 if p < r2 q =Randomized-Partition(A, p, r)3 Randomized-Quicksort(A, p, q − 1)4 Randomized-Quicksort(A, q + 1, r)
RANDOMIZED-PARTITION(A,p,r)
102 / 109
Images/ITESM.png
Randomized Quicksort
RANDOMIZED-QUICKSORT(A,p,r)Randomized-Quicksort(A, p, r)
1 if p < r2 q =Randomized-Partition(A, p, r)3 Randomized-Quicksort(A, p, q − 1)4 Randomized-Quicksort(A, q + 1, r)
RANDOMIZED-PARTITION(A,p,r)
102 / 109
Images/ITESM.png
Randomized Quicksort
RANDOMIZED-QUICKSORT(A,p,r)Randomized-Quicksort(A, p, r)
1 if p < r2 q =Randomized-Partition(A, p, r)3 Randomized-Quicksort(A, p, q − 1)4 Randomized-Quicksort(A, q + 1, r)
RANDOMIZED-PARTITION(A,p,r)
102 / 109
Images/ITESM.png
Randomized Quicksort
RANDOMIZED-QUICKSORT(A,p,r)Randomized-Quicksort(A, p, r)
1 if p < r2 q =Randomized-Partition(A, p, r)3 Randomized-Quicksort(A, p, q − 1)4 Randomized-Quicksort(A, q + 1, r)
RANDOMIZED-PARTITION(A,p,r)
102 / 109
Images/ITESM.png
Randomized QuicksortRANDOMIZED-QUICKSORT(A,p,r)Randomized-Quicksort(A, p, r)
1 if p < r2 q =Randomized-Partition(A, p, r)3 Randomized-Quicksort(A, p, q − 1)4 Randomized-Quicksort(A, q + 1, r)
RANDOMIZED-PARTITION(A,p,r)Randomized-Partition(A, p, r)
1 i =Random(p, r)2 exchange A[r ] with A[i]3 return Partition(A, p, r)
102 / 109
Images/ITESM.png
Quicksort: Randomized Quicksort
Expected running timeThe expected running time for the Randomized Quicksort algorithm arisesfrom the following lemma.
Lemma 7.1 (Cormen’s book)Let X be the number of comparisons performed in line 4 ofPARTITION algorithm over the entire execution of QUICKSORT onan n-element array.Then, the running time of QUICKSORT is O(n + X).
Now the proof of the expected running time.BLACKBOARD!
103 / 109
Images/ITESM.png
Quicksort: Randomized Quicksort
Expected running timeThe expected running time for the Randomized Quicksort algorithm arisesfrom the following lemma.
Lemma 7.1 (Cormen’s book)Let X be the number of comparisons performed in line 4 ofPARTITION algorithm over the entire execution of QUICKSORT onan n-element array.Then, the running time of QUICKSORT is O(n + X).
Now the proof of the expected running time.BLACKBOARD!
103 / 109
Images/ITESM.png
Quicksort: Randomized Quicksort
Expected running timeThe expected running time for the Randomized Quicksort algorithm arisesfrom the following lemma.
Lemma 7.1 (Cormen’s book)Let X be the number of comparisons performed in line 4 ofPARTITION algorithm over the entire execution of QUICKSORT onan n-element array.Then, the running time of QUICKSORT is O(n + X).
Now the proof of the expected running time.BLACKBOARD!
103 / 109
Images/ITESM.png
Therefore
It is possible to conclude thatThe Average Time Complexity of the Quicksort is O(n log n)
104 / 109
Images/ITESM.png
Applications
Sorting in Special EnvironmentsExample: Using Massive Parallel Stream Processors.
Multi-Objective OptimizationYes!!! Numerical Analysis using the Quick Sort!!!
Real-Time Visualization of Large Time-Varying MoleculesUse the distance of the atoms to the viewers - the Painters Algorithms!!!
105 / 109
Images/ITESM.png
Applications
Sorting in Special EnvironmentsExample: Using Massive Parallel Stream Processors.
Multi-Objective OptimizationYes!!! Numerical Analysis using the Quick Sort!!!
Real-Time Visualization of Large Time-Varying MoleculesUse the distance of the atoms to the viewers - the Painters Algorithms!!!
105 / 109
Images/ITESM.png
Applications
Sorting in Special EnvironmentsExample: Using Massive Parallel Stream Processors.
Multi-Objective OptimizationYes!!! Numerical Analysis using the Quick Sort!!!
Real-Time Visualization of Large Time-Varying MoleculesUse the distance of the atoms to the viewers - the Painters Algorithms!!!
105 / 109
Images/ITESM.png
Outline1 Sorting problem
DefinitionClassic Complexities
2 Heap SortHeap Sort: DefinitionsHeap: Finding Parents and ChildrenMax-HeapifyBuild Max Heap: Using Max-HeapifyHeap Sort
3 Applications of Heap Data StructureHeap Sort: Exercises
4 QuicksortThe Divide and Conquer QuicksortQuicksort: Complexity AnalysisRandomized Quicksort
5 Lower Bounds of SortingLower Bounds of Sorting: Basic Concepts
106 / 109
Images/ITESM.png
Lower Bounds of Sorting: Basic Concepts
Mergesort and HeapsortThey are algorithms that sort in O(n log n).It is more we can give a sequence such that Ω(n log n).
Property“These algorithms share an interesting property: the sorted order theydetermine is based only on comparisons between the input elements. Wecall such sorting algorithms comparison sorts.”
107 / 109
Images/ITESM.png
Lower Bounds of Sorting: Basic Concepts
Mergesort and HeapsortThey are algorithms that sort in O(n log n).It is more we can give a sequence such that Ω(n log n).
Property“These algorithms share an interesting property: the sorted order theydetermine is based only on comparisons between the input elements. Wecall such sorting algorithms comparison sorts.”
107 / 109
Images/ITESM.png
Lower Bounds of Sorting: Basic Concepts
Mergesort and HeapsortThey are algorithms that sort in O(n log n).It is more we can give a sequence such that Ω(n log n).
Property“These algorithms share an interesting property: the sorted order theydetermine is based only on comparisons between the input elements. Wecall such sorting algorithms comparison sorts.”
107 / 109
Images/ITESM.png
Lower Bounds of Sorting: Theorem and Corollary
TheoremAny comparison sort algorithm requires Ω(n log n) comparisons in theworst case.
CorollaryHeapsort and Mergesort are asymptotically optimal comparison sorts.
108 / 109
Images/ITESM.png
Lower Bounds of Sorting: Theorem and Corollary
TheoremAny comparison sort algorithm requires Ω(n log n) comparisons in theworst case.
CorollaryHeapsort and Mergesort are asymptotically optimal comparison sorts.
108 / 109
Images/ITESM.png
Exercises
Cormen’s Chapter 77.1-47.2-37.2-57.4-1
109 / 109