Recall Queues CSE 326: Data Structures • FIFO: First-In, First-Out Priority Queues – Binary • Some contexts where this seems right? Heaps • Some contexts where some things should be allowed to skip ahead in the line?

1 2

Queues that Allow Line Jumping ADT • Need a new ADT 1. PQueue data : collection of data with • Operations: Insert an Item, priority Remove the “Best” Item 2. PQueue operations – insert – deleteMin 6 2 15 23 insert deleteMin 12 18 3. PQueue property: for two elements in the 45 3 7 queue, x and y, if x has a lower priority value than y, x will be deleted before y 3 4

Applications of the Priority Queue Potential Implementations

• Select print jobs in order of decreasing length insert deleteMin • Forward packets on routers in order of urgency Unsorted list (Array) O(1) O(n) • Select most frequent symbols for compression Unsorted list (Linked-List) O(1) O(n) • numbers, picking minimum first Sorted list (Array) O(n) O(1)*

•Anythinggreedy Sorted list (Linked-List) O(n) O(1)

5 6 Recall From Lists, Queues, Binary Properties Stacks • Use an ADT that corresponds to your 1. Structure Property needs 2. Ordering Property • The right ADT is efficient, while an overly general ADT provides functionality you aren’t using, but are paying for anyways • Heaps provide O(log n) worst case for both insert and deleteMin, O(1) average insert

7 8

Tree Review More Terminology Tree T Tree T A A depth(B): root(T): leaves(T): B C height(G): B C children(B): degree(B): parent(H): D E F G D E F G siblings(E): branching factor(T): H I H I ancestors(F): descendents(G): subtree(C): J K L M N J K L M N

9 10

Brief interlude: Some Definitions: Heap Structure Property A Perfect – A binary tree with • A is a complete binary tree. all leaf nodes at the same depth. All Complete binary tree – binary tree that is completely filled, with the possible exception internal nodes have 2 children. of the bottom level, which is filled left to right. height h Examples: h+1 11 2 –1 nodes 2h –1 non-leaves 5 21 2h leaves 2 9 16 25 1 3 7 10 13 19 22 30

11 12 Representing Complete Why this approach to storage? Binary Trees in an Array

1 A From node i: 23B C 6 7 left child: 45D E F G 81011129 right child: H I J K L parent:

implicit (array) implementation: ABCDEFGHI JKL 0 1 2 3 4 5 6 7 8 9 10 11 12 13

13 14

Heap Order Property Heap Operations Heap order property: For every non-root • findMin: node X, the value in the parent of X is • insert(val): percolate up. less than (or equal to) the value in X. • deleteMin: percolate down.

10 10 10 20 80 20 80 20 80 40 60 85 99 40 60 85 99 30 15 50 700 50 700 65 not a heap

15 16

Insert: percolate up Heap – Insert(val) 10 20 80 Basic Idea: 40 60 85 99 1. Put val at “next” leaf position 50 700 65 15 2. Percolate up by repeatedly exchanging node until no longer 10 needed 15 80 40 20 85 99 50 700 65 60

17 18 Insert Code (optimized) void insert(Object o) { int percolateUp(int hole, Heap – Deletemin assert(!isFull()); Object val) { while (hole > 1 && size++; val < Heap[hole/2]) Basic Idea: newPos = Heap[hole] = Heap[hole/2]; percolateUp(size,o); hole /= 2; } 1. Remove root (that is always the min!) Heap[newPos] = o; return hole; } } 2. Put “last” leaf node at root 3. Find smallest child of node 4. Swap node with its smallest child if runtime: needed. 5. Repeat steps 3 & 4 until no swaps needed. (Code in book) 19 20

DeleteMin: percolate down DeleteMin Code (Optimized)

10 Object deleteMin() { int percolateDown(int hole, assert(!isEmpty()); Object val) { 20 15 while (2*hole <= size) { returnVal = Heap[1]; left = 2*hole; right = left + 1; 40 60 85 99 size--; newPos = if (right ≤ size && Heap[right] < Heap[left]) 50 700 65 percolateDown(1, target = right; Heap[size+1]); else 15 Heap[newPos] = target = left; Heap[size + 1]; if (Heap[target] < val) { 20 65 return returnVal; Heap[hole] = Heap[target]; } hole = target; 40 60 85 99 } else 50 700 runtime: break; } 21 (code in book) return hole; 22 }

Insert: 16, 32, 4, 69, 105, 43, 2

012345678 CSE 326: Data Structures

Binary Heaps

23 24 Building a Heap Building a Heap 12 5 11 3 10 6 9 4 8 1 7 2 • Adding the items one at a time is O(n log n) in the worst case

• I promised O(n) for today

25 26

BuildHeap: Floyd’s Method Working on Heaps 12 5 11 3 10 6 9 4 8 1 7 2 • What are the two properties of a heap? Add elements arbitrarily to form a complete tree. – Structure Property Pretend it’s a heap and fix the heap-order property! – Order Property 12

• How do we work on heaps? 5 11 – Fix the structure 3 10 6 9 – Fix the order

4 8 1 7 2 27 28

Buildheap pseudocode BuildHeap: Floyd’s Method private void buildHeap() { for ( int i = currentSize/2; i > 0; i-- ) 12 percolateDown( i ); } 5 11

3 10 6 9

4 8 1 7 2 runtime:

29 30 BuildHeap: Floyd’s Method BuildHeap: Floyd’s Method 12 12 12

5 11 5 11 5 11

3 10 2 9 3 10 2 9 3 1 2 9

4 8 1 7 6 4 8 1 7 6 4 8 10 7 6

31 32

BuildHeap: Floyd’s Method BuildHeap: Floyd’s Method 12 12 12 12

5 11 5 11 5 11 5 11

3 10 2 9 3 1 2 9 3 10 2 9 3 1 2 9

4 8 1 7 6 4 8 10 7 6 4 8 1 7 6 4 8 10 7 6 12 12 12

5 2 5 2 1 2

3 1 6 9 3 1 6 9 3 5 6 9

33 34 4 8 10 7 11 4 8 10 7 11 4 8 10 7 11

More Priority Queue Finally… Operations 1 • decreaseKey – given a pointer to an object in the queue, reduce its priority value 3 2

Solution: change priority and 4 5 6 9 ______

12 8 10 7 11 • increaseKey – given a pointer to an object in the queue, increase its priority value runtime: Why do we need a pointer? Why not simply data value? Solution: change priority and 35 36 ______More Priority Queue Operations Facts about Heaps Observations: • Remove(objPtr) • Finding a child/parent index is a multiply/divide by two – given a pointer to an object in the queue, • Operations jump widely through the heap remove the object from the queue • Each percolate step looks at only two new nodes • Inserts are at least as common as deleteMins Solution: priority to negative infinity, percolate up to root and deleteMin Realities: • Division/multiplication by powers of two are equally fast • FindMax • Looking at only two new pieces of data: bad for cache! • With huge data sets, disk accesses dominate

37 38

Cycles to access: CPU A Solution: d-Heaps • Each node has d 1 Cache children • Still representable by 3 7 2 array Memory • Good choices for d: 4 8 5 12 11 10 6 9 – (choose a power of two for efficiency) 12 1 3 7 2 4 8 5 121110 6 9 – fit one set of children in Disk a cache line – fit one set of children on 39 40 a memory page/disk

One More Operation CSE 326: Data Structures

• Merge two heaps Priority Queues Leftist Heaps & Skew Heaps • Add the items from one into another? – O(n log n)

• Start over and build it from scratch? –O(n)

41 42 New Heap Operation: Merge Leftist Heaps

Given two heaps, merge them into one Idea: heap Focus all heap maintenance work in – first attempt: insert each element of the one small part of the heap smaller heap into the larger. runtime: Leftist heaps: 1. Most nodes are on the left – second attempt: concatenate binary heaps’ arrays and run buildHeap. 2. All the merging work is done on the right runtime: 43 44

Definition: Null Path Length Leftist Heap Properties null path length (npl) of a node x = the number of nodes between x • Heap-order property and a null in its subtree – parent’s priority value is ≤ to childrens’ priority OR npl(x) = min distance to a descendant with 0 or 1 children values – result: minimum element is at the root • npl(null) = -1 ? • npl(leaf) = 0 • npl(single-child node) = 0 ? ? • Leftist property – For every node x, npl(left(x)) ≥ npl(right(x)) Equivalent definitions: 0 1 ? 0 – result: tree is at least as “heavy” on the left as the right 1. npl(x) is the height of largest Are leftist trees… complete subtree rooted at x 0 0 0 2. npl(x) = 1 + min{npl(left(x)), npl(right(x))} complete? 45 balanced? 46

Right Path in a is Short (#1) Are These Leftist? Claim: The right path is as short as any in the tree. 2 2 0 Proof: (By contradiction) 1 1 1 1 0 Pick a shorter path: D1 < D2 Say it diverges from right path at x x 0 1 0 0 1 0 0 0 1

npl(L) ≤ D1-1 because of the path of 0 0 0 0 0 0 0 0 length D1-1 to null L R D 1 D2 0 npl(R) ≥ D2-1 because every node on Every subtree of a leftist right path is leftist 0 tree is leftist! 0 47 Leftist property at x violated! 48 Right Path in a Leftist Tree is Short (#2) Why do we have the leftist Claim: If the right path has r nodes, then the tree property? has at least 2r-1 nodes. Because it guarantees that: Proof: (By induction) • the right path is really short compared to Base case : r=1. Tree has at least 21-1 = 1 node the number of nodes in the tree Inductive step : assume true for r’< r. Prove for tree with right path at least r. • A leftist tree of N nodes, has a right path 1. Right subtree: right path of r-1 nodes of at most lg (N+1) nodes ⇒ 2r-1-1 right subtree nodes (by induction) 2. Left subtree: also right path of length at least r-1 (by previous slide) ⇒ r-1 left subtree nodes (by induction) 2 -1 Idea – perform all work on the right path Total tree size: (2r-1-1) + (2r-1-1) + 1 = 2r-1

49 50

Merging Two Leftist Heaps Merge two heaps (basic idea) • merge(T1,T2) returns one leftist heap containing all elements of the two • Put the smaller root as the new root, (distinct) leftist heaps T1 and T2 • Hang its left subtree on the left. merge T • Recursively merge its right subtree and 1 a a the other tree. merge

L1 R1 L1 a < b R1 T 2 b b

L R 51 2 2 L2 R2 52

Let’s do an example, but first… Merge Continued Other Heap Operations a a If npl(R’) > npl(L ) 1 • insert ?

’ ’ L1 R R L1 • deleteMin ? ’ R = Merge(R1, T2)

runtime:

53 54 Operations on Leftist Heaps Leftest Merge Example merge ? • merge with two trees of total size n: O(log n) 1 5 3 • insert with heap size n: O(log n) 0 merge 00 7 – pretend node is a size 1 leftist heap 10 12 1 ? 5 5 – insert by merging original heap with one node 0 1 heap 3 14 00 0 merge merge 10 12 10 0 00 12 7 8 0 0 8 8 0 • deleteMin with heap size n: O(log n) 14 – remove and return root – merge left and right subtrees 0 (special case) 8 merge 0 55 1256

Sewing Up the Example Finally…

? 3 ? 1 3 3 1 1 3 3 0 0 0 7 ? 7 1 7 5 1 5 5 0 0 7 5 1 5 1 7 0 0 0 0 0 0 14 0 14 0 0 14 10 8 10 10 8 0 0 0 0 0 0 8 14 10 8 10 8 14 0 0 12 0 12 0 0 12 12 12 Done?

57 58

Random Definition: Leftist Heaps: Summary Amortized Time am·or·tized time: Good Running time limit resulting from “writing off” expensive • runs of an algorithm over multiple cheap runs of the algorithm, usually resulting in a lower overall running time • than indicated by the worst possible case. If M operations take total O(M log N) time, amortized time per operation is O(log N) Bad • Difference from average time: •

59 60 Skew Heaps Merging Two Skew Heaps Problems with leftist heaps – extra storage for npl merge T – extra complexity/logic to maintain and check npl 1 a a – right side is “often” heavy and requires a switch merge

Solution: skew heaps L1 R1 L1 a < b R1 – “blindly” adjusting version of leftist heaps T – merge always switches children when fixing right 2 b b path – amortized time for: merge, insert, deleteMin = O(log L R n) 2 2 L2 R2 – however, worst case time for all three = O(n) Only one step per iteration, with children always switched 61 62

Example merge Code void merge(heap1, heap2) { 5 3 3 merge case { 7 5 7 heap1 == NULL: return heap2; 10 12 5 merge heap2 == NULL: return heap1; 14 10 14 3 10 12 12 heap1.findMin() < heap2.findMin(): 8 temp = heap1.right; 7 8 8 heap1.right = heap1.left; 3 heap1.left = merge(heap2, temp); 14 return heap1; 5 7 otherwise: 8 10 14 return merge(heap2, heap1); } 63 64 12 }

Runtime Analysis: Comparing Heaps Worst-case and Amortized • No worst case guarantee on right path • Binary Heaps • Leftist Heaps length! • All operations rely on merge

⇒ worst case complexity of all ops = • Probably won’t get to amortized analysis • d-Heaps • Skew Heaps in this course, but see Chapter 11 if curious. • Result: M merges take time M log n Still room for improvement! (Where?) 65 66 ⇒ amortized complexity of all ops = Yet Another : Binomial Queues CSE 326: Data Structures • Structural property Binomial Queues – Forest of binomial trees with at most one tree of any height What’s a forest?

What’s a binomial tree? • Order property – Each binomial tree has the heap-order property

67 68

The Binomial Tree, Bh h Binomial Queue with n elements •Bh has height h and exactly 2 nodes •Bis formed by making B a child of another B h h-1 h-1 Binomial Q with n elements has a unique structural • Root has exactly h children representation in terms of binomial trees! • Number of nodes at depth d is binomial coeff. – Hence the name; we will not use this last property Write n in binary: n = 1101 (base 2) = 13 (base 10) ⎛ h ⎞ ⎜ ⎟ ⎝d ⎠

1 B3 1 B2 No B1 1 B0 B0 B1 B2 B3

69 70

Properties of Binomial Queue Operations on Binomial Queue • At most one binomial tree of any height • Will again define merge as the base operation • n nodes ⇒ binary representation is of size ? – insert, deleteMin, buildBinomialQ will use merge ⇒ deepest tree has height ? ⇒ number of trees is ? • Can we do increaseKey efficiently? decreaseKey? Define: height(forest F) = maxtree T in F { height(T) } • What about findMin? Binomial Q with n nodes has height Θ(log n)

71 72 Merging Two Binomial Queues Example: Binomial Queue Merge Essentially like adding two binary numbers! H1: H2: 1. Combine the two forests 1 -1 3 5 2. For k from 0 to maxheight { 21 a. m ← total number of B ’s in the two BQs k 3 9 b. if m=0: continue; # of 1’s 7 2 1 6 c. if m=1: continue; 0+0 = 0 8 11 5 7 d. if m=2: combine the two Bk’s to form a1+0 = 1 Bk+1 1+1 = 0+c e. if m=3: retain one Bk and 1+1+c = 1+c 6 combine the other two to form a Bk+1 } Claim: When this process ends, the forest has at most one tree of any height 73 74

Example: Binomial Queue Example: Binomial Queue Merge Merge H1: H2: H1: H2:

1 -1 3 5 1 -1 5

7 2 1 3 21 9 6 7 3 2 1 3 9 6

8 11 5 7 21 8 11 5 7

6 6

75 76

Example: Binomial Queue Example: Binomial Queue Merge Merge H1: H2: H1: H2:

1 -1 -1

7 3 5 2 1 3 2 1 3 1

7 5 21 9 6 8 11 5 8 11 5 3

7 6 6 21 9 6

7

77 78 Example: Binomial Queue Merge Complexity of Merge H1: H2: Constant time for each height -1 Max number of heights is: log n 2 1 3 1

8 11 5 7 3 5 ⇒ worst case running time = Θ( )

6 21 9 6

7

79 80

Insert in a Binomial Queue deleteMin in Binomial Queue Insert(x): Similar to leftist or skew heap Similar to leftist and skew heaps….

runtime Worst case complexity: same as merge O( )

Average case complexity: O(1) Why?? Hint: Think of adding 1 to 1101

81 82

deleteMin: Example deleteMin: Example BQ 7 3 4 Result: 7 8 5 4 7 8 5

find and delete 7 smallest root merge BQ (without the shaded part) 8 5 and BQ’ BQ’ runtime: 7 83 84