CS6045: Advanced Algorithms

Sorting Algorithms

Heap Data Structure

• A heap (nearly complete binary tree) can be stored as an array A– Root of tree is A[1]– Parent of – Left child of A[i] = A[2i]– Right child of A[i] = A[2i + 1]– Computing is fast with binary representation

implementation

Heaps

• Max-heap property:

• Min-heap property:

Maintain the Heap Property

• Running Time?

• O(log n)

Build a Heap

Correctness

• Loop invariant: At start of every iteration of for loop, each node i+1, i+2, …, n is the root of a max-heap

• Analysis running time?• O(n log n)• Tighter bound? O(n)

Heapsort

Analysis

• BUILD-MAX-HEAP: O(n)• for loop: n – 1 times• Exchange elements: O(1)• MAX-HEAPIFY: O(log n)• Time:

– O(n log n)

Min-Heap Operations 2

4

11

12

8

1310

6

97

3

2

4

3

12

8

1310

6

97

11

2

4

1112

8

1310

6

97

3

Insert(S, x): O(height) = O(log n)

Extract-min(S): return head, replace head key with the last, float down, O(log n)

2

6

11

12

8

1310

4

95

12

6

118

1310

4

95

4

6

118

1310

12

95

4

6

118

1310

5

912

Priority Queues • Priority Queue

– Maintains a dynamic set S of elements– Each element has a key --- an associated value

• Applications– job scheduling on shared computer– Dijkstra’s finding shortest paths in graphs– Prim’s algorithm for minimum spanning tree

Priority Queues • Operations supported by priority queue

– Insert(S, x) - inserts element with the pointer x– Minimum/Maximum(S) - returns element with the minimum key – Extract-Min/Max(S) - removes and returns minimum key– Increase/Decrease-Key(S,x,k) – increases/decreases the value of

element x’s key to the new value k

Comparison Sorting• The only operation that may be used to gain order

information about a sequence is comparison of pairs of elements

• Insertion sort, merge sort, quicksort, heapsort• Lower bound for comparison sorting?

Decision Tree Model• Abstraction of any comparison sort• Counting only comparisons• Abstract everything else: such as control and data

movement

• How many leaves on the decision tree?– >= n!

• What is the length of the longest path from root to leaf?– Depend on the algorithm

Lower Bound for Comparison Sorting

• A lower bound on the heights of decision trees in the lower bound on the running time of any comparison sort algorithm

• (n log n)– n! <= l <= 2h 2h >= n! h >= lg(n!) >= lg (n/e)n //Stirlling’s approximation = nlg(n/e) = nlgn – nlge = (n log n)

Non-comparison Sorts

• Counting Sort

Analysis

• O(n + k)

• How big a k is practical?

Radix Sort

Correctness

• Induction on number of passes (i in pseudocode).• Assume digits 1, 2, ……, i -1 are sorted.• Show that a stable sort on digit i leaves digits 1, 2,

……, i sorted:– If 2 digits in position i are different, ordering by

position i is correct, and positions 1, 2, …… , i - 1 are irrelevant.

– If 2 digits in position i are equal, numbers are already in the right order (by inductive hypothesis). The stable sort on digit i leaves them in the right order.

Analysis

• O(n + k) per iteration• d iterations• O(d(n + k)) total

Bucket Sort

• Idea:– Divide [0,1) into n equal-sized buckets– Distribute the n input values into the buckets– Sort each bucket– Go through buckets in order, listing elements in

each on