Lecture 7

Heaps and Priority Queues

Motivating Example

3 jobs have been submitted to a printer, the jobs have sizes 100, 10, 1 page.

Average waiting time for FIFO service, (100+110+111)/3 = 107 time units

Average waiting time for shortest job first service, (1+11+111)/3 = 41 time units

Need to have a queue which does insert and deletemin

Priority Queue

Common ImplementationLinked list

Insert in O(1)

Find the minimum element in O(n), thus deletion is O(n)

Search Tree (AVL tree)

Insert in O(log n)

Delete in O(log n)

Search Tree is an overkill as it does many other operations

Heaps

Almost complete binary tree

All levels are complete except the lowest one

In the last level empty spaces are towards the right.

(book says complete binary tree, I will define complete and almost complete whenever I use them)

4

2 5

61 3

If such a tree has height h, then it has between 2h-1 + 2 and 2h + 1 nodes. Thus h is O(log n)

Mistake in the expression in your book

Value of an element at a node is less than or equal to that of its descendants. (Heap property)

Mistake in your book

Section 6.1, 6.2, 6.3 in book

4

2 5

61 3

Not Heap

1

2 5

64 3

Heap

Array implementaionWe start from position 1 in the array.

The first element contains the root

Left child of element at position j is at position 2j, right child is at position 2j + 1

Parent of element at position j is at j/2

1

2 5

64 3

1 2 5 4 3 6

Need to know the size of the heap.

Would you recommend this implementation for any binary tree?

The array size can become 2n for n elements

Insertion

Find an empty position in the heap

If some nodes have one child, then the empty position is the other child

If all nodes have 0 or 2 children, then the empty position is the left child of a leaf.

Insert the element there.

Let the element be inserted at position j. If the parent of the element is more than the element, then interchange the parent and child

Interchange elements at j/2 and j.

1

2 5

64 3

1 2 5 4 3 6

If necessary, interchange elements at j/2 with j/2 /2

And so on till we reach the root.

For (k = empty_pos; k >1; k = k/2 )

If (Heap[k] < Heap[k/2 ])

interchange(Heap[k] , Heap[k/2 ]) ;

Complexity? O(log n)

1

2 5

64 3 2.5

1 2 5 4 3 6 2.5

1

2 2.5

64 3 5

1 2 2.5 4 3 6 5

We are maintaining the heap property at every stage.

DeleteMin

Delete the root.

Compare the two children of the root

Make the lesser of the two root.

An empty spot is created.

Bring the lesser of the two children of the empty spot to the empty spot.

A new empty spot is created.

Continue

1

2 5

64 3

1 2 5 4 3 6

2 5

64 3

2 5 4 3 6

2

5

64 3

2 5 4 3 6

2 3 5 4 6

2

3 5

64

What is the complexity?

Do we maintain the heap property in this procedure?

Completeness is not preserved

We can delete the last element in the heap, insert is at the empty spot at a leaf, and then move it upwards as necessary.

O(log n)

2

3 5

4 6

2 3 5 4 6

pseudocodeDelete root; k =1;

Do

{

If A[k] is empty, break;

If A[2k] < A[2k + 1] j = 2k, else j = 2k+1;

A[k] = A[j];

k = j;

}Insert A[currsize] at A[k];

Percolate up as necessary;

Heap Sort

You want to sort n real numbers.

Insert them in a heap;

Deletemin n times;

Complexity O(h), h is the depth of the tree

Increase Key

Need to increase the value of an element.

Increase the value of the element,

Interchange it with the lesser of its children if it is greater than any of its children,

Continue doing this till you reach a leaf

1

2 5

64 3

Fig 1

10

2 5

64 3

Increase Root

Fig 2

2

10 5

64 3

Fig 3

2

3 5

64 10

Fig 4

Decrease Key

Need to decrease the value of an element.

Decrease the value of the element,

Interchange it with its parent if it is less than its parent,

Continue doing this till you reach the root

1

2 5

64 3

Fig 1

10

2 5

3.14 3 Decrease 6 to 3.1

Fig 2

2

10 3.1

54 3

Fig 3

Tighter Analysis of Complexity of Build Heap

Suppose we have n elements.

We want to build a heap of n elements.

We present an O(n) algorithm to do so.

Insert these elements in n positions of an array.

Is an almost complete binary tree.

But does not satisfy the heap order

So we need to interchange these elements to obtain heap order

Parents are at position n/2 to 1.

If we percolate these down suitably, (interchange with lesser of the two children, if it is less than either of the two children, and so on), then we will get heap order.

For (j = n/2; j--; j >0)

Percolate Down (j);

Percolate Down (j)

{

If (2j > n) break;

If Heap[j] min(Heap[2j], Heap(2j+1), break;

If (Heap[2j] < Heap[2j + 1])

{ interchange(Heap[j], Heap[2j]);

Percolate Down (2j);}

else { interchange(Heap[j], Heap[2j+1]);

Percolate Down (2j+1);}

}

Complexity Analysis

Complexity of percolating down a single element is O(h) where h is the height of the element.

Overall complexity is O(sum of the heights of all elements)

We will show that sum of the heights of all elements is O(n).

For an almost complete binary tree,

There is 1 element at height h,

2 at height h-1

4 at height h-2……

2i at height h-i

The number at height 0 is between 1 and 2h

However, these elements do not contribute to the sum of the heights.

Thus sum of the heights is

j=0h-1 2j(h-j)

This is equal to 2h+1 - 1 -(h+1)

We have seen that the number of nodes n is greater than 2h-1 + 2 and 2h-1 + 2 is greater than (1/4) (2h+1 - 2 –h)

Thus sum of heights is less than 4n

Thus sum of heights is O(n)