Trees & Tree-Based Data Structures Part 4: Heaps Dr. Christopher M. Bourke [email protected] Definition Example Definition 95 A (max) heap is a binary tree of depth d that satisfies the following properties. 93 35 1. It is full up to level d 1, every node at levels 0 through d 1 is present − − 60 72 15 21 2. At level d all nodes are full-to-the-left 3. Every node has a key value that is greater than both of its children. 35 53 10 70 12 14 19 7 20 30 I First two conditions: fullness property I Third condition: heap property Properties Example Min-Heap 1 I Heaps are not binary search trees I Keys are required to be unique, non-negative integers 5 30 I The maximum element is always at the top of the heap I The fullness property guarantees that 50 55 40 80 d O(log n) 60 53 72 70 65 45 90 95 ∈ I Equivalent definition for min-heaps: both children are greater than the parent 92 63 OperationsI OperationsII Restricted Access Data Structure Restricted Access Data Structure Both operations will: Heaps are restricted access data structures; core operations: I Preserve the heap properties I Preserve the fullness property I Get (and remove) maximum element (“getMax”) I Guaranteed to be efficient O(d) = O(log n) I Insert element I Insert is not arbitrary I Other general methods are not supported (arbitrary retrieve/remove) Insert Key Incorrect OperationI Example 100 Insert a new key: 95 I Must preserve the fullness property and the heap property I Heaps are not binary search trees 93 45 I We cannot treat them the same 60 72 15 21 35 53 Incorrect OperationII Incorrect Operation III Example Example 100 100 95 93 45 93 95 45 60 72 15 21 60 72 15 21 35 53 35 53 Incorrect OperationIV Incorrect OperationV Example Example 100 100 93 95 93 95 60 72 45 21 60 72 45 21 35 53 15 15 35 53 15 15 Insert Key Insert First Step 95 I Solution: preserve the fullness property first I Insert at the next available spot: 93 45 I At the deepest level I At the left-most available spot 60 72 15 21 I Start a new level if necessary 35 53 100 Insert Key InsertI Heapify 95 I Fullness property is preserved I Heap property may not be satisfied 93 45 I Fix the heap (“heapify”): exchange key values up the tree until the heap property is satisfied or we reach the root 60 72 15 21 35 53 100 InsertII Insert III Heapify Heapify 95 95 93 45 93 45 60 72 15 21 60 100 15 21 swap 35 53 100 35 53 72 InsertIV InsertV Heapify Heapify 95 95 93 45 93 swap 45 60 100 15 21 60 100 15 21 35 53 72 35 53 72 InsertVI Insert VII Heapify Heapify 95 95 100 45 100 45 60 93 15 21 60 93 15 21 35 53 72 35 53 72 Insert VIII InsertIX Heapify Heapify swap 95 100 100 45 95 45 60 93 15 21 60 93 15 21 35 53 72 35 53 72 Heapify Insertion Analysis Input : A heap H and an inserted node u 1 curr u I Assume that we can find the next available spot “for free” ← 2 while curr = H.root and curr.key > curr.parent.key do I Insertion: O(1) 6 3 swap curr.key, curr.parent.key I Heapify: O(d) comparisons and swaps in the worst case 4 curr curr.parent ← I Fullness property guarantees O(log n) Get Max Incorrect Operation: Get Max 95 I Heap property ensures that the maximum value is at the root I Variation: “peek” 93 45 I Get max generally removes (and returns) the root element 60 72 15 21 I Same problem as before: we need to fix the heap and satisfy both heap property and fullness property 35 53 Incorrect Operation: Get Max Incorrect Operation: Get Max 95 93 45 93 45 60 72 15 21 60 72 15 21 35 53 35 53 Incorrect Operation: Get Max Incorrect Operation: Get Max 93 93 45 45 60 72 15 21 60 72 15 21 35 53 35 53 Incorrect Operation: Get Max Get Max Correct Operation 93 I Remove (and save) the root element. 72 45 I Preserve the fullness property first 60 15 21 I Replace the root with the “last” element I Then heapify downward 35 53 Get Max Get Max Correct Operation Correct Operation 93 45 93 45 60 72 15 21 60 72 15 21 35 53 35 53 Get Max Get Max Correct Operation Correct Operation 53 53 93 45 93 45 60 72 15 21 60 72 15 21 35 53 35 Get Max Get Max Correct Operation Correct Operation swap 53 93 93 45 53 45 60 72 15 21 60 72 15 21 35 35 Get Max Get Max Correct Operation Correct Operation 93 93 53 45 53 45 swap 60 72 15 21 60 72 15 21 35 35 Get Max Algorithm Correct Operation 93 I Again: assume “free” access to the last element I Swap last and root 72 45 I From the root, swap the larger of the two children 60 53 15 21 I Repeat until both children are less or we’ve reached a leaf I Pseudocode: exercise 35 Implementations Array-Based Implementation I Since the tree is full, data can be stored contiguously I Suggests that an array can be used to store nodes (or node references) I Leverage other data structures and composition I Store root at index 1 I Array-based implementation I If a node u is stored at index i: I Tree-based implementation I Left child: 2i I Right child: 2i + 1 i I Parent: b 2 c I May use zero-indexing by subtracting 1. Array-Based Implementation Array-Based Implementation Relations Example 95 i 93 45 2 p 60 72 15 21 i 35 53 u 2i + 1 i 2i + 1 2i 0 1 2 3 4 5 6 7 8 9 2 i 2i r – 95 93 45 60 72 15 21 35 53 p u ` r ` ··· ··· ··· Array-Based Implementation Tree-Based Implementation Advantages I Familiar underlying data structure (arrays or array-based list) I Cannot (in general) use linked lists: no random access, no constant-time jumping I Possible to implement using a tree data structure (node with left/right/parent to left/right/parent references) I Simple, easy way to traverse the tree I Fundamental difference: we cannot “jump” to the last node or the next available I Fullness property guarantees data contiguity (in general, binary trees should not spot be implemented with arrays) I Still possible to access these in O(d) = O(log n) time I Get “free” (constant-time) access to: I Last element: at index n I Next available spot: at index n + 1 I Be sure to keep track of the size of the heap! Tree-Based Implementation Tree-Based Implementation Finding the next available spot: outline Finding the next available spot: outline r d 1 − 2k = 2d 1 I Given n: number of nodes in the heap − Xk=0 I Compute the depth: L R d = log (n) b 2 c Number of nodes level d level d at level d is I Start at root r m = n (2d 1) − − 2d 2d I Question: is the available spot in r’s left or right subtree? at most 2 at most 2 d d If m < 2 , the open If m 2 , the open 2 ≥ 2 slot is in the left slot is in the right subtree subtree Tree-Based Implementation Array-Based Implementation Finding the next available spot: outline Java Tutorial I Repeat analysis until you reach a single node tree with either its left or right child open I Design and implement a max-heap data structure (for Integer s) I Constant number of computations at each level I Use an ArrayList to simplify I O(d) = O(log n) I Support both insert and getMax operations I Doesn’t ruin overall running time I More complex, more operations than an array-based implementation Applications Priority Queue I Heaps are used as fundamental data structures in many algorithms I Elements are inserted (arbitrarily) I Two immediate applications: I But: elements are dequeued according to highest priority (not FIFO) I Efficient Priority Queue implementation I Heap provides both operations in an efficient manner, O(log n) I Sorting I Heap property based on element’s priority Heap Sort Heap Sort I Given n elements I Insert each one into a min/max heap I Remove each one until the heap is empty Smart Data Structures and dumb code are a lot better than I Removed order gives a sorted collection! the other way around. I Analysis: each insert/remove is at most O(log n) —Eric S. Raymond I n iterations of each insert/remove; overall O(n log n) I Java implementation.