ECE750-TXB Lecture 7: Red-Black Trees, Heaps, and Treaps Todd L. ECE750-TXB Lecture 7: Red-Black Trees, Veldhuizen Heaps, and Treaps [email protected] Red-Black Trees Heaps Todd L. Veldhuizen Treaps [email protected] Bibliography Electrical & Computer Engineering University of Waterloo Canada February 14, 2007 ECE750-TXB Binary Search Trees Lecture 7: Red-Black Trees, Heaps, and Treaps Todd L. Veldhuizen [email protected] I Recall that in a binary tree of height h the time required to find or insert an element is O(h). Red-Black Trees Heaps I In the worst case h = n, the number of elements. Treaps I To keep h ∈ O(log n) one needs a balancing strategy. Bibliography I Balancing strategies may be either: I Randomized: e.g. a random insert order results in expected height of c log n with c ≈ 4.311. I Deterministic (in the sense of not random). I Today we will see an example of each: I Red-black trees: deterministic balancing I Treaps: randomized. Also demonstrate persistence and unique representation. ECE750-TXB Red-black trees Lecture 7: Red-Black Trees, Heaps, and Treaps Todd L. Veldhuizen [email protected] I Red-black trees are a popular form of binary search tree with a deterministic balancing strategy. Red-Black Trees Heaps I Nodes are coloured red or black. Treaps I Properties of the node-colouring ensure that the longest Bibliography path to a leaf is no more than twice the length of the shortest path. I This ensures height of ≤ 2 log2(n + 1), which implies search, min, max in O(log n) worst-case time. I Insert and Delete can also be performed in O(log n) worst-case time. I Invented by Bayer [2], red-black formulation due to Guibas and Sedgewick [9]. Other sources: [5, 10]. ECE750-TXB Red-Black Trees: Invariants Lecture 7: Red-Black Trees, Heaps, and Treaps Todd L. Veldhuizen [email protected] Red-Black Trees Heaps Treaps Balance invariants: I Bibliography 1. No red node has a red child. 2. Every path in a subtree contains the same number of black nodes. ECE750-TXB Red-Black Trees Lecture 7: Red-Black Trees, Heaps, and Treaps Todd L. Veldhuizen [email protected] Red-Black Trees Heaps Treaps Bibliography ECE750-TXB Red-Black Trees: Balance I Lecture 7: Red-Black Trees, Heaps, and Treaps Todd L. Veldhuizen [email protected] Red-Black Trees Let bh(x) be the number of black nodes along any path Heaps from a node x to a leaf, excluding the leaf. Treaps Bibliography Lemma The number of internal nodes in the subtree rooted at x is at least 2bh(x) − 1. Proof. ECE750-TXB Red-Black Trees: Balance II Lecture 7: Red-Black Trees, Heaps, and Treaps Todd L. Veldhuizen By induction on height: [email protected] 1. Base case: If x has height 0, then x is a leaf, and Red-Black Trees bh(x) = 0; the number of internal (non-leaf) Heaps bh(x) descendents of x is 0 = 2 − 1. Treaps 2. Induction step: assume the hypothesis is true for height Bibliography ≤ h. Consider a node of height h + 1. From invariant (2), the children have black height either bh(x) − 1 (if the child is black) or bh(x) (if the child is red). By induction hypothesis, each child subtree has at least 2bh(x)−1 − 1 internal nodes. The total number of internal nodes in the subtree rooted at x is therefore ≥ (2bh(x)−1 − 1) + 1 + (2bh(x)−1 − 1) = 2bh(x) − 1. ECE750-TXB Red-Black Trees: Balance Lecture 7: Red-Black Trees, Heaps, and Treaps Todd L. Veldhuizen [email protected] Theorem Red-Black Trees A red-black tree with n internal nodes has height at most Heaps Treaps 2 log2(n + 1). Bibliography Proof. Let h be the tree height. From invariant 1 (a red node must have both children black), the black-height of the root must be ≥ h/2. Applying Lemma 1.1, the number of internal nodes n of the tree satisfies n ≥ 2h/2 − 1. Rearranging, h ≤ 2 log2(n + 1). ECE750-TXB Red-Black Trees: Balance Lecture 7: Red-Black Trees, Heaps, and Treaps Todd L. Veldhuizen [email protected] I As with all non-randomized binary search trees, balance must be maintained when insert or delete operations are Red-Black Trees performed. Heaps Treaps I These operations may disrupt the invariants, so Bibliography rotations and recolourings are needed to restore them. I Insert for red-black tree: 1. Insert the new key as a red node, using the usual binary tree insert. 2. Perform restructurings and recolourings along the path from the newly added leaf to the root to restore invariants. 3. Root is always coloured black. ECE750-TXB Red-Black Trees: Balance Lecture 7: Red-Black Trees, Heaps, and Treaps I Four cases for red nodes with red children: Todd L. Veldhuizen [email protected] Red-Black Trees Heaps Treaps Bibliography I Restructure/recolour to correct: each of the above cases becomes ECE750-TXB Red-Black Trees: Example Lecture 7: Red-Black Trees, Heaps, and Treaps Todd L. Veldhuizen I Insertion of [1,2,3,4,5] into a red-black tree: [email protected] Red-Black Trees Heaps Treaps Bibliography I Implementation of rebalancing is straightforward but a bit involved. ECE750-TXB Heaps and Treaps Lecture 7: Red-Black Trees, Heaps, and Treaps Todd L. I Treaps are a randomized search tree that combine Veldhuizen TRees and hEAPS. [email protected] Red-Black Trees I First, let’s look at heaps. Heaps I Consider determining the maximum element of a set. Treaps I We could iterate through the array and keep track of Bibliography the maximum element seen so far. Time taken: Θ(n). I We could build a binary tree (e.g. red-black). We can obtain the maximum (minimum) element in O(h) time by following rightmost (leftmost) branches. If tree is balanced, requires O(n log n) time to build the tree, and O(log n) time to retrieve the maximum element. I A heap is a highly efficient data structure for maintaining the maximum element of a set. It is a rudimentary example of a dynamic algorithm/data structure. ECE750-TXB Dynamic Algorithms Lecture 7: Red-Black Trees, Heaps, and Treaps Todd L. I A static problem is one where we are given an instance Veldhuizen of a problem to solve, we solve it, and are done (e.g., [email protected] sort an array). Red-Black Trees I A dynamic problem is one where we are given a problem Heaps to solve, we solve it. Treaps I Then the problem is changed slightly and we resolve. Bibliography I ...ad infinitum. I The challenge goes from solving a single instance of a problem to maintaining a solution as the problem is modified. I It is usually more efficient to update the solution than recompute from scratch. I e.g., binary search trees can be viewed as a method for dynamically maintaining an ordered list as elements are inserted and removed. ECE750-TXB Heaps Lecture 7: Red-Black Trees, Heaps, and Treaps Todd L. Veldhuizen I A heap dynamically maintains the maximum element in [email protected] a collection (or, dually, the minimum element). A binary heap can: Red-Black Trees Heaps I Obtain the maximum element in O(1) time; Treaps I Remove the maximum element in O(log n) time; Bibliography I Insert new element in O(log n) time. Heaps are a natural implementation of the PriorityQueue ADT. I There are several flavours of heaps: binary heaps, binomial heaps, fibonacci heaps, pairing heaps. The more sophisticated of these support merging (melding) two heaps. I We will look at binary heaps. ECE750-TXB Binary Heap Invariants Lecture 7: Red-Black Trees, Heaps, and Treaps 1. A binary heap is a complete binary tree of height h − 1, Todd L. Veldhuizen plus a possibly incomplete level of height h filled from [email protected] left to right. Red-Black Trees 2. The key stored at each node is ≥ the key(s) stored in Heaps its children. Treaps Bibliography ECE750-TXB Binary Heap Lecture 7: Red-Black Trees, Heaps, and Treaps I A binary heap may be stored as a (1-based) array, where Todd L. Veldhuizen I Parent(j) = bj/2c [email protected] I LeftChild(i) = 2 ∗ i Red-Black Trees I RightChild(i) = 2 ∗ i + 1 Heaps I e.g., [17, 11, 13, 9, 6, 2, 12, 4, 3, 1] is an array Treaps representation of the heap: Bibliography ECE750-TXB Heap operations Lecture 7: Red-Black Trees, Heaps, and Treaps Todd L. Veldhuizen [email protected] I To insert a key k into the heap: I Place k at the next available position. Red-Black Trees I Swap k with its parent(s) until the heap invariant is Heaps satisfied. (Takes O(log n) time.) Treaps I The maximum element is just the key stored at the Bibliography root, which can be read off in O(1) time. I To delete the maximum element: I Place the key at the last heap position at the root (overwriting the current maximum), and decrease the size of the heap by one. I Choose the largest of the root and its two children, and make this the root; perform this procedure recursively until the heap invariant is satisfied. ECE750-TXB Heap: insert example Lecture 7: Red-Black Trees, Heaps, and Treaps Todd L. Veldhuizen [email protected] Red-Black Trees Heaps I Example: insert 23 into the heap and restore the heap Treaps invariant.