COMP3600/6466 – Algorithms Abstract Data Structures Cont.:

Heaps + AVL Tree[CLRS 6.3-6.5], [Lev sec. 6.3]

Hanna Kurniawati

https://cs.anu.edu.au/courses/comp3600/Comp_3600_6466@anu.edu.au

A1• Marks out last Wednesday • Sent to your ANU email around noon time• Very good results• #submissions: 293• Average: 74.28 ; Median: 80 ; Max: 99.5• #HD: 147 ; #D: 53 ; #Cr: 34 ; #P: 27• Feedback sessions will be arranged for each

marking group in MS Teams• Marking group is stated in the email about your mark• Marking group ≠ Tutorial group• Watch out in piazza, we’ll release the schedule tonight• Solution example is in the class website

A2• All concept/theory needed to answer A2 have

been covered in lectures prior to week 5• C/C++ programming necessary has been

covered in tutorial in week 5• Programming component• Template for I/O of the program is in the class website• A few test cases are in the class website• Note: We will test your program with test cases we did

not release. Therefore, do NOT extrapolate specifications from the test cases. Specifications are as defined in the problem description

Due dates• Final Project – Milestone 1• Due: 7 Sep 23:59 (1st day of mid-term break)• Grace period ends: 8 Sep 13:00• A2• Due: 21 Sep 23:59 (1st day back after mid-term

break)• Grace period ends: 22 Sep 13:00• All time are in Canberra time• Submissions: Wattle, save as draft

Cheating is NOT Tolerated• Cheating includes intention to outsource

assignments• E.g.: Uploading assignment questions to a “tutoring

service” website, even if you end up not using their answers

• In A1, the majority of issues are found becauseof copying answers from “tutoring service(s)”, which happened to be wrong!!! • Dropping the class does not mean the student’s

record of misconduct will be removed. It will stayin the record!

Topics• Binary Search Tree• Heaps• AVL Tree• Red-black Tree

Topics• Binary Search Tree• What is it?• Tree walk & Querying• Insertion & Deletion• Heaps• What is it?• Heapify• Insertion & Building• Extract/Deletion• AVL Tree• Red-black Tree

TopicsüBinary Search Tree• HeapsüWhat is it?üHeapify• Insertion & Building• Extract/Deletion• Applications• AVL Tree• Red-black Tree

Time complexity for build-max-heap• We’ll compute the total time as the number of times max-

heapify is called multiply by the cost of each max-heapify:

TopicsüBinary Search Tree• HeapsüWhat is it?üHeapifyüInsertion & Building• Extract/Deletion• Applications• AVL Tree• Red-black Tree

ExtractMax• Intutitively:• Output the root node• Swap the root node with the last node in the heap (i.e.,

the bottom and right most leaf)• Decrease the heap size by 1, essentially removing the

last leaf node (which is now the same as the root node we just extracted)

• Heapify the new root of the tree

ExtractMax• Pseudo-code [CLRS pp.163]

• The algorithm only needs to traverse a path of the tree once. Hence, the complexity 𝑂(log 𝑛)

Example

TopicsüBinary Search Tree• HeapsüWhat is it?üHeapifyüInsertion & BuildingüExtract/Deletion• Applications• AVL Tree• Red-black Tree

Applications: HeapSort• Heapsort:• Build the heap and extract the heap element one by

one• Complexity: 𝑂(𝑛 log 𝑛)• Building the heap takes 𝑂(𝑛)• Heapifying the rest of the heap every time an

element in extracted takes a total 𝑂(𝑛 log 𝑛)• In practice, QuickSort (assuming proper

implementation) is faster than HeapSort• But, worst case of heapsort is 𝑛 log 𝑛 & in-place

Applications: Priority Queue• A priority queue is a data structure that:• Maintains a set S, where each element is

associated with a key• Has the following operations:• Insert(𝑆, 𝑥) : Inserts element x into the set S• Maximum(𝑆) : Returns an element of S with the

largest key• ExtractMax(𝑆) : Removes and returns an element of

S with the largest key• IncreaseKey (𝑆, 𝑥, 𝑘): Increase the key of 𝑥 to 𝑘

TopicsüBinary Search Tree• HeapsüWhat is it?üHeapifyüInsertion & BuildingüExtract/DeletionüApplications• AVL Tree• Red-black Tree

TopicsüBinary Search TreeüHeaps• AVL Tree• What is it?• Transformation• Insertion• Deletion• Red-black Tree

What is AVL Tree?• The first self-balancing binary search tree, invented in

1962 by Adel’son-Vel’skii and Landis (written in Russian)• Recall in binary search tree, Search, Min, Max, Successor,

Predecessor, Insert, Delete are guaranteed to be 𝑂(ℎ) where ℎ is the height of the tree. But, there’s no guarantee on how ℎrelates to 𝑛.

• A balance tree means the levels of the tree’s leaves are not that different, so as to ensure the height of the tree is 𝑂(log 𝑛), with 𝑛 being the #nodes in the tree

• Self-balancing tree: Automatically rebalance the tree when it becomes imbalanced.

• Intuitively, an AVL tree maintains balance by ensuring the height of the left and right sub-tree of all nodes in the tree are almost the same (i.e., differ by at most 1)

What is AVL Tree?• More formally, an AVL tree is a binary search tree

where • Each node is augmented with the balance factor of the node

Balance factor of node 𝑥:bf(𝑥) = height(leftSubtree(𝑥)) – height(rightSubtree(𝑥))

Height of a tree/sub-tree is the length of the longest path from the root of the tree/sub-tree to a leafConvention: The height of a tree with only a root node is 0, while the height of an empty tree (i.e., tree with no node) is -1

• The balance factor of each node in the tree must either be either 0, +1, or -1

• The balance requirement ensures that all operations of binary search tree with 𝑂(ℎ) complexity takes 𝑂(log 𝑛) in AVL tree

Example

The numbers above the nodes are the balance factor of the nodes

Is The Height of AVL Tree Really 𝑂(log 𝑛)? • Notice that the worst case in binary search tree, i.e.,

when ℎ = 𝑂(𝑛), happens when the number of nodes in the tree is the smallest possible, i.e., 𝑛 = ℎ + 1• To show that the height of an AVL tree with 𝑛 nodes is 𝑂 log 𝑛 , we’ll first compute the smallest number of nodes that an AVL tree of height ℎ could have • Suppose 𝑁! is the minimum number of nodes for an AVL

tree of height ℎ• In AVL tree, the minimum number of nodes is achieved

when every node in the tree have a balance factor of +/-1• Therefore, 𝑁! = 1 + 𝑁!"# + 𝑁!"$• Solve the recurrence

Solving the Recurrence• Solve 𝑁. = 1 + 𝑁./0 +𝑁./1• Let’s assume 𝑁! = 𝑁" = 1

TopicsüBinary Search TreeüHeaps• AVL TreeüWhat is it?• Transformation• Insertion• Deletion• Red-black Tree

Key to AVL• AVL tree relies on transformation operation,

called rotation, to re-balance the tree

Transformation: Rotation• Four types: 2 classes, 2 types in each class• Single Left Rotation (L-rotation)• L-rotation on node 𝑥: Rotate the edge connecting 𝑥 and its

left child

Transformation: Rotation• Four types: 2 classes, 2 types in each class• Single Left Rotation (L-rotation)• L-rotation on node 𝑥: Rotate the edge connecting 𝑥 and its

left child • Single Right Rotation (R-rotation): • Mirror image of L-rotation

Transformation: Rotation• Four types: 2 classes, 2 types in each class• Double Left Right Rotation (LR-rotation)• LR-rotation of node 𝑥: L-rotation on the root of 𝑥’s left

subtree followed by R-rotation on 𝑥

Transformation: Rotation• Four types: 2 classes, 2 types in each class• Double Left Right Rotation (LR-rotation)• LR-rotation of node 𝑥: L-rotation on the root of 𝑥’s left

subtree followed by R-rotation on 𝑥• Double Right Left Rotation (RL-rotation)• Mirror image of LR-rotation

Note• Obviously, the rotations hold when the

children are sub-trees too• An example on LR-rotation

Rotation – Summarised• bf(𝑥) = -2:• bf(𝑥.right) = -1: L-rotation• bf(𝑥.right) = +1: RL-rotation • bf(𝑥) = +2:• bf(𝑥.left) = +1: R-rotation• bf(𝑥.left) = -1: LR-rotation

TopicsüBinary Search TreeüHeaps• AVL TreeüWhat is it?üTransformation• Insertion• Deletion• Red-black Tree

Insertion• Essentially, perform binary search tree insertion and

then rebalance whenever necessary• To insert a node 𝑥 to an AVL tree T:• Insert 𝑥 as if T is a usual binary search tree• Let 𝑧 be the parent of the newly inserted node• While 𝑧 is not root • If 𝑧 violates the AVL property, rotate to restore AVL property• Update Balance Factor of 𝑧• Let 𝑧 = 𝑧. 𝑝𝑎𝑟𝑒𝑛𝑡

Example• Insert 9• Insert 15

Note• Insertion only requires one single/double rotation• If the tree becomes imbalanced due to the node being

inserted, then rotation at the new node’s lowest ancestor that became imbalanced will return the height of the tree before insertion takes place. Since we insert a node to an already balanced AVL tree, we can stop after this rotation

• You can also show this by enumerating the possible imbalanced caused by an insertion

• Since rotation takes constant time, AVL tree onlyrequires an additional constant time for insertion, compared to binary search tree

TopicsüBinary Search TreeüHeaps• AVL TreeüWhat is it?üTransformationüInsertion• Deletion• Red-black Tree

Deletion• Similar to insertion: Essentially, binary search tree

deletion and then rebalance whenever necessary• Recall binary search tree deletion has 3 cases based

on #children of the node being deleted• 0 children: Just delete à Nodes that might change height are

those in the path from deleted node to root• 1 child: Delete it, connect child to parent à Nodes that might

change height are those in the path from deleted node to root• 2 children: Replace the deleted node with its successor,

successor leaf à Nodes that might change height are those in the path from deleted successor leaf to root

Unlike Insertion,• In deletion, rebalancing may be needed along

the path where nodes might change height• Need to check nodes from the deleted node, along with

its parents and ancestors all the way to the root

Example• Delete 7• Delete 12

TopicsüBinary Search TreeüHeaps• AVL TreeüWhat is it?üTransformationüInsertionüDeletion• Red-black Tree

TopicsüBinary Search TreeüHeapsüAVL Tree• Red-black Tree

Next: Red-black Tree