Summary of symbol-table implementations guarantee average case ordered Balanced Trees implementation search insert delete search insert delete iteration? unordered array N N N N/2 N/2 N/2 no ordered array lg N N N lg N N/2 N/2 yes unordered list N N N N/2 N N/2 no • 2-3-4 trees ordered list N N N N/2 N/2 N/2 yes • red-black trees BST N N N 1.39 lg N 1.39 lg N ? yes • B-trees randomized BST 7 lg N 7 lg N 7 lg N 1.39 lg N 1.39 lg N 1.39 lg N yes Randomized BSTs provide the desired guarantees probabilistic, with References: Algorithms in Java, Chapter 13 exponentially small Intro to Algs and Data Structs, Section 4.4 chance of error This lecture: Can we do better? Copyright © 2007 by Robert Sedgewick and Kevin Wayne. 1 3 Symbol Table Review Typical random BSTs Symbol table: key-value pair abstraction. ! Insert a value with specified key. ! Search for value given key. ! Delete value with given key. Randomized BST. ! Guarantee of ~c lg N time per operation (probabilistic). ! Need subtree count in each node. ! Need random numbers for each insert/delete op. N = 250 This lecture. 2-3-4 trees, red-black trees, B-trees. lg N ! 8 1.39 lg N ! 11 average node depth 2 4 Searching in a 2-3-4 Tree Search. ! Compare search key against keys in node. ! Find interval containing search key. ! Follow associated link (recursively). Ex. Search for L 2-3-4 trees red-black trees B-trees K R between K and R C E M O W smaller than K A D F G J L N Q S V Y Z found L 5 7 2-3-4 Tree Insertion in a 2-3-4 Tree 2-3-4 tree. Generalize node to allow multiple keys; keep tree balanced. Insert. ! Search to bottom for key. Perfect balance. Every path from root to leaf has same length. Allow 1, 2, or 3 keys per node. ! 2-node: one key, two children. ! 3-node: two keys, three children. Ex. Insert B ! 4-node: three keys, four children. K R K R smaller than K smaller than K larger than R between smaller than C C E M O W K and R C E M O W A D F G J L N Q S V Y Z A D F G J L N Q S V Y Z B not found 6 8 Insertion in a 2-3-4 Tree Insertion in a 2-3-4 Tree Insert. Insert. ! Search to bottom for key. ! Search to bottom for key. ! 2-node at bottom: convert to 3-node. ! 2-node at bottom: convert to 3-node. ! 3-node at bottom: convert to 4-node. Ex. Insert B Ex. Insert X K R K R smaller than K smaller than C C E M O W C E M O W A B D F G J L N Q S V Y Z A B D F G J L N Q S V X Y Z B fits here X fits here 9 11 Insertion in a 2-3-4 Tree Insertion in a 2-3-4 Tree Insert. Insert. ! Search to bottom for key. ! Search to bottom for key. Ex. Insert X Ex. Insert H K R K R larger than R smaller than K C E M O W C E M O W larger than W larger than E A D F G J L N Q S V Y Z A D F G J L N Q S V Y Z X not found H not found 10 12 Insertion in a 2-3-4 Tree Splitting 4-nodes in a 2-3-4 tree Insert. Idea: split 4-nodes on the way down the tree. ! Search to bottom for key. ! Ensures last node is not a 4-node. ! 2-node at bottom: convert to 3-node. ! Transformations to split 4-nodes: ! 3-node at bottom: convert to 4-node. ! 4-node at bottom: ?? Ex. Insert H K R C E M O W Invariant. Current node is not a 4-node. Consequence. Insertion at bottom is easy since it's not a 4-node. A B D F G J L N Q S V X Y Z H does not fit here! 13 15 Splitting a 4-node in a 2-3-4 tree Splitting 4-nodes in a 2-3-4 tree Idea: split the 4-node to make room C E G Local transformations that work anywhere in the tree C E Ex. Splitting a 4-node attached to a 2-node A B D F J A B D F G J H does fit here! H does not fit here D D Q C E G K Q W K W A-C A-C A B D F H J E-J L-P R-V X-Z E-J L-P R-V X-Z Problem: Doesn’t work if parent is a 4-node Solution 1: Split the parent (and continue splitting while necessary). Solution 2: Split 4-nodes on the way down. could be huge unchanged 14 16 Splitting 4-nodes in a 2-3-4 tree 2-3-4 Tree Local transformations that work anywhere in the tree Tree grows up from the bottom. Ex. Splitting a 4-node attached to a 3-node E tree height grows only when root splits X A D H D H Q K Q W K W M P A-C E-G A-C E-G I-J L-P R-V X-Z I-J L-P R-V X-Z L E could be huge unchanged 17 19 Splitting 4-nodes in a 2-3-4 tree 2-3-4 Tree: Balance Local transformations that work anywhere in the tree Property. All paths from root to leaf have same length. Splitting a 4-node attached to a 4-node never happens when we split nodes on the way down the tree. Invariant. Current node is not a 4-node. Tree height. ! Worst case: lg N [all 2-nodes] ! Best case: log4 N = 1/2 lg N [all 4-nodes] ! Between 10 and 20 for a million nodes. ! Between 15 and 30 for a billion nodes. 18 20 2-3-4 Tree: Implementation? Direct implementation is complicated, because: ! Maintaining multiple node types is cumbersome. ! Implementation of getChild() involves multiple compares. ! Large number of cases for split(), make3Node(), and make4Node(). 2-3-4 trees private void insert(Key key, Val val) { red-black trees Node x = root; while (x.getChild(key) != null) B-trees { x = x.getChild(key); if (x.is4Node()) x.split(); } if (x.is2Node()) x.make3Node(key, val); else if (x.is3Node()) x.make4Node(key, val); } fantasy code Bottom line: could do it, but say tuned for an easier way. 21 23 Summary of symbol-table implementations Red-black trees (Guibas-Sedgewick, 1979) Represent 2-3-4 tree as a BST. ! Use "internal" edges for 3- and 4- nodes. guarantee average case ordered “red” glue implementation search insert delete search insert delete iteration? unordered array N N N N/2 N/2 N/2 no ordered array lg N N N lg N N/2 N/2 yes unordered list N N N N/2 N N/2 no ordered list N N N N/2 N/2 N/2 yes BST N N N 1.38 lg N 1.38 lg N ? yes randomized BST 7 lg N 7 lg N 7 lg N 1.38 lg N 1.38 lg N 1.38 lg N yes ! Correspondence between 2-3-4 trees and red-black trees. 2-3-4 tree c lg N c lg N c lg N c lg N not 1-1 because 3-nodes can swing either way. constants depend upon implementation 22 24 Red-Black Tree Red-Black Tree: Splitting Nodes Represent 2-3-4 tree as a BST. Two easy cases. Switch colors. ! Use "internal" edges for 3- and 4- nodes. “red” glue Two hard cases. Use rotations. ! Disallowed: two red edges in-a-row. do single rotation do double rotation 25 27 Red-Black Tree: Splitting Nodes Rotations in a red-black tree Two easy cases. Switch colors. to insert G: change colors G does not fit here single rotation right rotate R ! double rotation single rotation left rotate E ! 26 G does fit here! 28 Red-Black Tree: Insertion Search implementation for red-black trees public Val get(Key key) { Node x = root; while (x != null) E { black tree height int cmp = key.compareTo(x.key); grows only when if (cmp == 0) return x.val; root splits else if (cmp < 0) x = x.left; else if (cmp > 0) x = x.right; } X A return null; } M P Search code is the same as elementary BST. Runs faster because of better balance in tree. L E 29 31 Red-Black Tree: Balance Insert implementation for red-black trees (skeleton) Property A. Every path from root to leaf has same number of black links. public class BST<Key extends Comparable, Val> implements Iterable { Property B. Never two red links in-a-row. private static final boolean RED = true; private static final boolean BLACK = false; private Node root; Property C. Height of tree is less than 2 lg N + 2 in the worst case. private class Node { Key key; Property D.