Recall Queues CSE 326: Data Structures • FIFO: First-In, First-Out Priority Queues – Binary • Some contexts where this seems right? Heaps • Some contexts where some things should be allowed to skip ahead in the line? 1 2 Queues that Allow Line Jumping Priority Queue ADT • Need a new ADT 1. PQueue data : collection of data with • Operations: Insert an Item, priority Remove the “Best” Item 2. PQueue operations – insert – deleteMin 6 2 15 23 insert deleteMin 12 18 3. PQueue property: for two elements in the 45 3 7 queue, x and y, if x has a lower priority value than y, x will be deleted before y 3 4 Applications of the Priority Queue Potential Implementations • Select print jobs in order of decreasing length insert deleteMin • Forward packets on routers in order of urgency Unsorted list (Array) O(1) O(n) • Select most frequent symbols for compression Unsorted list (Linked-List) O(1) O(n) • Sort numbers, picking minimum first Sorted list (Array) O(n) O(1)* •Anythinggreedy Sorted list (Linked-List) O(n) O(1) 5 6 Recall From Lists, Queues, Binary Heap Properties Stacks • Use an ADT that corresponds to your 1. Structure Property needs 2. Ordering Property • The right ADT is efficient, while an overly general ADT provides functionality you aren’t using, but are paying for anyways • Heaps provide O(log n) worst case for both insert and deleteMin, O(1) average insert 7 8 Tree Review More Tree Terminology Tree T Tree T A A depth(B): root(T): leaves(T): B C height(G): B C children(B): degree(B): parent(H): D E F G D E F G siblings(E): branching factor(T): H I H I ancestors(F): descendents(G): subtree(C): J K L M N J K L M N 9 10 Brief interlude: Some Definitions: Heap Structure Property A Perfect binary tree – A binary tree with • A binary heap is a complete binary tree. all leaf nodes at the same depth. All Complete binary tree – binary tree that is completely filled, with the possible exception internal nodes have 2 children. of the bottom level, which is filled left to right. height h Examples: h+1 11 2 –1 nodes 2h –1 non-leaves 5 21 2h leaves 2 9 16 25 1 3 7 10 13 19 22 30 11 12 Representing Complete Why this approach to storage? Binary Trees in an Array 1 A From node i: 23B C 6 7 left child: 45D E F G 81011129 right child: H I J K L parent: implicit (array) implementation: ABCDEFGHI JKL 0 1 2 3 4 5 6 7 8 9 10 11 12 13 13 14 Heap Order Property Heap Operations Heap order property: For every non-root • findMin: node X, the value in the parent of X is • insert(val): percolate up. less than (or equal to) the value in X. • deleteMin: percolate down. 10 10 10 20 80 20 80 20 80 40 60 85 99 40 60 85 99 30 15 50 700 50 700 65 not a heap 15 16 Insert: percolate up Heap – Insert(val) 10 20 80 Basic Idea: 40 60 85 99 1. Put val at “next” leaf position 50 700 65 15 2. Percolate up by repeatedly exchanging node until no longer 10 needed 15 80 40 20 85 99 50 700 65 60 17 18 Insert Code (optimized) void insert(Object o) { int percolateUp(int hole, Heap – Deletemin assert(!isFull()); Object val) { while (hole > 1 && size++; val < Heap[hole/2]) Basic Idea: newPos = Heap[hole] = Heap[hole/2]; percolateUp(size,o); hole /= 2; } 1. Remove root (that is always the min!) Heap[newPos] = o; return hole; } } 2. Put “last” leaf node at root 3. Find smallest child of node 4. Swap node with its smallest child if runtime: needed. 5. Repeat steps 3 & 4 until no swaps needed. (Code in book) 19 20 DeleteMin: percolate down DeleteMin Code (Optimized) 10 Object deleteMin() { int percolateDown(int hole, assert(!isEmpty()); Object val) { 20 15 while (2*hole <= size) { returnVal = Heap[1]; left = 2*hole; right = left + 1; 40 60 85 99 size--; newPos = if (right ≤ size && Heap[right] < Heap[left]) 50 700 65 percolateDown(1, target = right; Heap[size+1]); else 15 Heap[newPos] = target = left; Heap[size + 1]; if (Heap[target] < val) { 20 65 return returnVal; Heap[hole] = Heap[target]; } hole = target; 40 60 85 99 } else 50 700 runtime: break; } 21 (code in book) return hole; 22 } Insert: 16, 32, 4, 69, 105, 43, 2 012345678 CSE 326: Data Structures Binary Heaps 23 24 Building a Heap Building a Heap 12 5 11 3 10 6 9 4 8 1 7 2 • Adding the items one at a time is O(n log n) in the worst case • I promised O(n) for today 25 26 BuildHeap: Floyd’s Method Working on Heaps 12 5 11 3 10 6 9 4 8 1 7 2 • What are the two properties of a heap? Add elements arbitrarily to form a complete tree. – Structure Property Pretend it’s a heap and fix the heap-order property! – Order Property 12 • How do we work on heaps? 5 11 – Fix the structure 3 10 6 9 – Fix the order 4 8 1 7 2 27 28 Buildheap pseudocode BuildHeap: Floyd’s Method private void buildHeap() { for ( int i = currentSize/2; i > 0; i-- ) 12 percolateDown( i ); } 5 11 3 10 6 9 4 8 1 7 2 runtime: 29 30 BuildHeap: Floyd’s Method BuildHeap: Floyd’s Method 12 12 12 5 11 5 11 5 11 3 10 2 9 3 10 2 9 3 1 2 9 4 8 1 7 6 4 8 1 7 6 4 8 10 7 6 31 32 BuildHeap: Floyd’s Method BuildHeap: Floyd’s Method 12 12 12 12 5 11 5 11 5 11 5 11 3 10 2 9 3 1 2 9 3 10 2 9 3 1 2 9 4 8 1 7 6 4 8 10 7 6 4 8 1 7 6 4 8 10 7 6 12 12 12 5 2 5 2 1 2 3 1 6 9 3 1 6 9 3 5 6 9 33 34 4 8 10 7 11 4 8 10 7 11 4 8 10 7 11 More Priority Queue Finally… Operations 1 • decreaseKey – given a pointer to an object in the queue, reduce its priority value 3 2 Solution: change priority and 4 5 6 9 ____________________________ 12 8 10 7 11 • increaseKey – given a pointer to an object in the queue, increase its priority value runtime: Why do we need a pointer? Why not simply data value? Solution: change priority and 35 36 _____________________________ More Priority Queue Operations Facts about Heaps Observations: • Remove(objPtr) • Finding a child/parent index is a multiply/divide by two – given a pointer to an object in the queue, • Operations jump widely through the heap remove the object from the queue • Each percolate step looks at only two new nodes • Inserts are at least as common as deleteMins Solution: set priority to negative infinity, percolate up to root and deleteMin Realities: • Division/multiplication by powers of two are equally fast • FindMax • Looking at only two new pieces of data: bad for cache! • With huge data sets, disk accesses dominate 37 38 Cycles to access: CPU A Solution: d-Heaps • Each node has d 1 Cache children • Still representable by 3 7 2 array Memory • Good choices for d: 4 8 5 12 11 10 6 9 – (choose a power of two for efficiency) 12 1 3 7 2 4 8 5 121110 6 9 – fit one set of children in Disk a cache line – fit one set of children on 39 40 a memory page/disk One More Operation CSE 326: Data Structures • Merge two heaps Priority Queues Leftist Heaps & Skew Heaps • Add the items from one into another? – O(n log n) • Start over and build it from scratch? –O(n) 41 42 New Heap Operation: Merge Leftist Heaps Given two heaps, merge them into one Idea: heap Focus all heap maintenance work in – first attempt: insert each element of the one small part of the heap smaller heap into the larger. runtime: Leftist heaps: 1. Most nodes are on the left – second attempt: concatenate binary heaps’ arrays and run buildHeap. 2. All the merging work is done on the right runtime: 43 44 Definition: Null Path Length Leftist Heap Properties null path length (npl) of a node x = the number of nodes between x • Heap-order property and a null in its subtree – parent’s priority value is ≤ to childrens’ priority OR npl(x) = min distance to a descendant with 0 or 1 children values – result: minimum element is at the root • npl(null) = -1 ? • npl(leaf) = 0 • npl(single-child node) = 0 ? ? • Leftist property – For every node x, npl(left(x)) ≥ npl(right(x)) Equivalent definitions: 0 1 ? 0 – result: tree is at least as “heavy” on the left as the right 1. npl(x) is the height of largest Are leftist trees… complete subtree rooted at x 0 0 0 2. npl(x) = 1 + min{npl(left(x)), npl(right(x))} complete? 45 balanced? 46 Right Path in a Leftist Tree is Short (#1) Are These Leftist? Claim: The right path is as short as any in the tree. 2 2 0 Proof: (By contradiction) 1 1 1 1 0 Pick a shorter path: D1 < D2 Say it diverges from right path at x x 0 1 0 0 1 0 0 0 1 npl(L) ≤ D1-1 because of the path of 0 0 0 0 0 0 0 0 length D1-1 to null L R D 1 D2 0 npl(R) ≥ D2-1 because every node on Every subtree of a leftist right path is leftist 0 tree is leftist! 0 47 Leftist property at x violated! 48 Right Path in a Leftist Tree is Short (#2) Why do we have the leftist Claim: If the right path has r nodes, then the tree property? has at least 2r-1 nodes.