Data Structures Brett Bernstein
Lecture 13: AVL Trees and Binary Heaps
Review Exercises
1.( ??) Interview question: Given an array show how to shu e it randomly so that any possible reordering is equally likely. static void shu e(int[] arr)
2. Write a function that given the root of a binary search tree returns the node with the largest value. public static BSTNode
3. Explain how you would use a binary search tree to implement the Map ADT. We have included it below to remind you. Map.java //Stores a mapping of keys to values public interface Map
10 5 15 1 12 18 16 17
1 Perform the following operations in order. (a) Remove 15. (b) Remove 10. (c) Add 13. (d) Add 8. 5. Suppose we begin with an empty BST. What order of add operations will yield the tallest possible tree?
Review Solutions 1. One method (popular in programs like Excel) is to generate a random double corre- sponding to each element of the array, and then sort the array by the corresponding doubles. Here is a dierent method that avoids sorting (used by Collections.shu e). RandomShu e.java import java.util.Random;
public class RandomShu e { static void swap(int[] arr, int i, int j) { int tmp = arr[i]; arr[i] = arr[j]; arr[j] = tmp; } //Iterative implementation public static void shu e(int[] arr) { Random ran = new Random(); for (int i = arr.length−1; i >= 1; −−i) swap(arr,i,ran.nextInt(i+1)); } //Recursive implementation public static void shu eRec(int[] arr) { sRHelp(arr, arr.length, new Random()); } public static void sRHelp(int[] arr, int len, Random ran) { if (len <= 1) return; swap(arr, len−1, ran.nextInt(len)); sRHelp(arr,len−1,ran); } }
2 Above we give an iterative and a recursive implementation. We rst randomly choose one of the elements and swap it into the nal position. Then we repeat the process on the rst n − 1 elements (i.e., randomly choose last element, and then randomly shu e rst n − 1 elements). 2. public static BSTNode
3. We require the keys of the Map to be Comparable or a Comparator to be provided. In each node instead of simply storing a value, store a key-value pair (i.e., an entry). The BST will be ordered by the keys. Here all operations above will be Θ(h) in the worst case (we don't have containsValue above which would always be Θ(n)). In a moment, we will show how to achieve Θ(lg n) height trees which gives an e cient Map implementation without needing a hash function (but we need ordered keys).
4. (a)
10 5 16 1 12 18 17
(b)
12 5 16 1 18 17
(c)
12 5 16 131 18 17
(d)
3 12 5 16 1318 18 17
5. Ascending or descending order (i.e., sorted or reverse sorted).
Data Structure: AVL Tree We have learned how to store values in a BST data structure that supports adding, removal, and searching in time Θ(h), where h is the height of the tree. We have also seen that in the worst case, the height can be Θ(n), the size of the tree. If we can keep the tree fairly balanced, then we can reduce the height to Θ(lg n) and obtain a fairly e cient data structure. To maintain a balanced structure we will use a technique invented by Adelson-Velskii and Landis (hence the name AVL tree). The idea is to keep a counter in every node (called a balance factor) that measures the dierence between the heights of the left and right subtrees (right minus left). We will add the following added constraint on our BST
No balance factor will be greater than 1 or less than -1.
Any BST that satis es this constraint will have Θ(lg n) height. Every time we add or remove a node we may violate this constraint. To return the tree to balance we will employ a technique called a rotation. Consider the following AVL tree:
10 +1
5 15 -1 +1
1 12 20 0 0 -1
17 0
Each node has the value and balance factor in it. Now suppose we add 16 (just as we would in a BST).
4 10 +2
5 15 -1 +2
1 12 20 0 0 -2
17 -1
16 0
As you can see, the tree now violates our constraints on the balance factors. To remedy the issue, we nd the closest (lowest) ancestor of the newly added node that is out of balance. In this case, it would be the node containing the 20. We then perform a rotation so that 17 takes the place of 20, and 20 becomes the right child of 17. The resulting tree is
10 +1 5 15 -1 +1 1 12 17 0 0 0 16 20 0 0
Now the tree satis es the constraint again. Let's generalize this example to all possible ways adding a node can imbalance the tree. First notice that if adding a value will cause a node to violate its balance factor constraint, then it must have been −1 or +1 already, since adding a single node can add at most 1 to the height of any subtree. Let's assume a node currently has balance factor −1 and some value X. We will model the possible ways that the node containing X will be the lowest node that violates the balance factor constraint (by adding values).
5 X -1 Y 0 C k
k A B k
Above Y < X and all of the labeled subtrees have the same height k ≥ 0. The subtree rooted at X has height k + 1. We rst handle the case that a node is added to subtree A that increases its height.
X -2 Y -1 Z C k
B k
A1 A2
Here we assumed the value added was smaller than Y . Note that the subtree rooted at X now has height k + 2. We then zoom in on the subtree A by drawing its root Z and subtrees A1 and A2. The possible heights of A1,A2 are (this will not eect our course of action though):
1. If k = 0 then A1,A2 are both empty. Z has a balance factor of 0
2. If k > 0 and the new value is smaller than Z then A1 has height k and A2 has height k − 1. Z has a balance factor of −1.
3. If k > 0 and the new value is greater than Z then A1 has height k − 1 and A2 has height k. Z has a balance factor of +1.
To x this we perform a rotation that puts Y where X is:
6 Y 0 ZX 0
A1 A2 BC k
The balance constraints are no longer violated. Note that the subtree rooted at Y has height k + 1, the same as the subtree rooted at X before a value was added. Thus all nodes above Y in the tree are now balanced as well. This is the so called left-left case since Z is the left child of Y which is the left child of X. The other left-right case occurs when we add a value that is larger than Y to our original tree (the picture with A, B, and C) above.
X -2 Y +1 W C k
k A
B1 B2
Here we have zoomed in on the subtree B. We have similar cases as above that determine the heights of B1,B2. Here we perform a rotation that puts W in place of Y :
X -2 W
Y C k
B2
k A B1
This gives us a case like the left-left situation. Thus we perform a second rotation as above putting W in place of X:
7 W 0 YX
k A B1 B2 C k
Again we have restored balance and the resulting tree has height k + 1. Thus all ancestors of W are balanced too. Using the operations above and their re ections (right-right and right-left cases) we can enforce the balance constraints on every add operation. The TreeMap and TreeSet in Java use a related data structure called a red-black tree that uses a slightly dierent constraint for balance, but also uses rotations to enforce the constraint.
AVL Tree Exercises 1. Consider the following BST.
10
5 15
6 12
(a) Add balance factors to all of the nodes. (b) Show what happens when we add the node 13 (treating the tree as an AVL tree). (c) Next add 7 to the tree (treating the tree as an AVL tree). (d)( ?) Next delete 5, 7, and 6 from the tree. Make sure to remedy any violated balance constraints.
2. Give a simple Θ(n lg n) sorting algorithm assuming you have access to an AVL Tree data structure.
3.( ??) The AVL Tree deletion algorithm is similar to the addition algorithm, but can cause as many as Θ(h) rotations to occur where h is the height of the tree. Can you explain why?
4.( ?) Let minNodes(h) denote the minimum number of nodes you need to have an AVL- Tree of height h. For h >= 2 give a recurrence relation satis ed by minNodes(h).
8 5. In the Priority Queue ADT elements have an ordering (Comparable or Comparator). When you dequeue an element, instead of removing the earliest added element, you remove the earliest element in the given ordering. You can imagine a situation where you are ordering tasks to work on, but each task has a priority governing when you must work on it. It has the following operations
(a) add: Adds an element to the Priority Queue (b) dequeue: Removes the smallest element (with respect to the ordering)
Describe an e cient implementation of this ADT.
AVL Tree Solutions 1. (a)
10 0
5 15 +1 -1
6 12 0 0
(b) We perform the left-right sequence of 2 rotations after adding 13.
10 0
5 13 +1 0
6 12 15 0 0 0
(c) After adding 7 we perform the right-right rotation which is simply the re ection of the left-left rotation.
9 10 0
6 13 0 0
5 7 12 15 0 0 0 0 (d) After the removal the root is unbalanced. We will treat this like the right-right case and rotate the 13 into the root.
13 -1
10 15 +1 0
12 0
2. Add all of the elements to the AVL Tree, then perform an inOrder traversal. If we want to handle duplicates, we can either extend our AVL Tree/BST to allow duplicates, or instead of storing values, we can store lists of values that are equivalent with respect to the ordering. This method of sorting isn't used.
3. Note that after an add operation, our rotations returned the eected subtree to the same height it was before the add operation. Thus no ancestors above the lowest unbalanced node had to be xed. After a remove operation the eected subtree may have a lower height than before, and thus ancestors of the lowest unbalanced node may need to be xed as well.
4. minNodes(h) = 1 + minNodes(h − 1) + minNodes(h − 2). To see why, note that we must have a subtree of height h − 1 so that the whole tree has height h. Secondly, the smallest we can make the other subtree is height h − 2 due to the balance constraint. This can be used to show that minNodes grows faster than the Fibonacci sequence, which grows exponentially. This in turn can be used to show that the height of an AVL tree is Θ(lg n). 5. Use an AVL Tree. Adds simply add nodes to the tree. To dequeue we simply remove the smallest value. Both operations require Θ(lg n) in the worst-case.
10 Data Structure: (Binary) Min Heap Above we saw how to implement a PriorityQueue using an AVL Tree. Here we will give a new data structure that can be used to implement a PriorityQueue and is way more e cient in practice (lower constants embedded in the Θ-terms). We will allow duplicates values. A min heap is a binary tree with the following two constraints:
1. Ordering: Every node is equal to or smaller than its children.
2. Completenees: Every level of the tree is full but the last level. In the last level the nodes ll up the leftmost positions.
The second constraint sounds odd, but it will enable a very e cient implementation. Instead of using binary tree nodes to store our values, we will just use an array containing all of the elements as they would appear in a level-by-level (breadth rst) traversal. Consider the following min heap:
4
57
9 129
We then store this in an ArrayList.
0 1 2 3 4 5
4 5 7 9 12 9
The nice thing about this format is that we can easily nd the children and parent of any node. Suppose you are at the node with index k in the array.
1. Left child: 2k + 1
2. Right child: 2k + 2
3. Parent: (k − 1)/2 (Java integer division; gives 0 on root) We will justify the left child formula. The rest will follow from that. Note that level of nodes at depth d contains the indices 2d − 1 through 2d+1 − 2. Thus the kth node in that level has index 2d − 1 + (k − 1). Applying the left child formula, we obtain index
2(2d − 1 + (k − 1)) + 1 = 2d+1 − 1 + 2(k − 1).
This is the index in level d + 1 just after the 2(k − 1) children of the nodes preceding our original node in level d.
11 The nice formulas above are made possible by the array storage format. Since our min heaps have the completeness property, the array format isn't wasteful. What remains is to implement the two main operations of the min heap: add and re- moveMin. The key to all heap operations is to rst guarantee that the completeness property holds. After the shape is correct, we then make a few updates to x the ordering constraint. Suppose we want to add the node 1 to our heap above. We rst add 1 in the next available spot in the lowest level:
4 57 9 1291
Next we need to x the ordering constraint. We use an operation called sift-up. Take the newly added node an compare it with its parent. If it is smaller, swap, and repeat the process on the parent. This is depicted below.
4 1 57 54 9 1291 9 1297
To remove the mininum we rst swap the top value with the last value. Then we can safely remove the last value and maintain the shape. Finally, we correct the ordering constraint by checking if the root is bigger than its smallest child. If so, swap and then repeat on the node you swapped with. This process is sometimes called sift-down.
Min Heap Exercises 1. Consider the following min heap.
5 96 14 1178 15 16 12 15
(a) What is the index of 11 in the corresponding array? (b) Add an 8 to the min heap. (c) Then add a 1 to the min heap. (d) Then removeMin from the min heap.
12 (e) Then removeMin from the min heap.
2.A max heap is just like a min heap, but each node must be larger than its children. Explain why a max heap data structure is unnecessary if you already have a min heap. 3. What are the worst-case runtimes of add and removeMin? 4. Assuming you have access to a min-heap, show how to sort a list of Comparable values in worst-case time Θ(n lg n). 5. How long does it take to nd a value in a min heap?
6.( ?) Sometimes it is useful to remove an arbitrary element from a min heap given its index. Explain how to do this in worst-case Θ(lg n) time. 7.( ??) Given an array of n comparable values, show how to turn it into a min heap. There is a Θ(n) worst-case implementation.
Min Heap Solutions 1. (a)4 (b) No sifting is required.
5 96 14 1178 15 16 12 158 (c) Here we must sift-up performing 3 swaps.
1 95 14 1168 15 16 12 1587 (d) We swap the 7 into the root, remove the 1, and then sift-down (swap 7 with 5 then 6).
5 96 14 1178 15 16 12 158
13 (e) We swap the 5 with the 8, remove the 5, and then sift-down (swap 8 with 6 then 7).
6 97 14 1188 15 16 12 15
2. Just use a min heap but reverse the ordering.
3. Both are Θ(lg n) since our tree is always well-balanced. 4. Add all elements to a min heap, and then repeatedly call removeMin to pull them out in order. This is called Heapsort (better than our AVL Tree sorting algorithm above due to the low constant on all heap operations).
5. Θ(n) in the worst-case (consider a really large value that isn't in the heap) since the heap ordering property doesn't aid in searches like the BST property does.
6. We use the following steps.
(a) Swap the item to be removed with the last item and then remove the last item. (b) The swap may have broken the ordering property so: i. Check if the swapped item is smaller than its parent. If so, do the sift-up procedure on it. ii. Otherwise, do the sift-down procedure on it.
7. The slow method is to just call add n times giving a worst-case Θ(n lg n) runtime. A better method is to loop backwards through the array and run the sift-down procedure on every value. To see why the runtime is Θ(n) we consider the work done by sift-down at every node. For simplicity, let's assume every level of the heap is full. Let the height of a node be the height of the subtree it is the root of. All nodes will require at most C steps to compare them with their children in the sift-down procedure. Nodes of height at least one will require at most an extra C steps, since they they could undergo a swap. Nodes of height at least two will require an extra C steps on top of that, and so forth. But, each time we increase the height we halve the number of nodes we consider. Since Cn + Cn/2 + Cn/4 + ··· = 2Cn = Θ(n) the result follows.
14