Priority Queues Binary and Binomial Heaps Supports the following operations. ■ Insert element x. ■ Return min element. ■ Return and delete minimum element. ■ Decrease key of element x to k. Applications. These lecture slides are adapted from CLRS, Chapters 6, 19. ■ Dijkstra's shortest path algorithm. ■ Prim's MST algorithm. ■ Event-driven simulation. ■ Huffman encoding. ■ Heapsort. ■ . Princeton University • COS 423 • Theory of Algorithms • Spring 2002 • Kevin Wayne 2 Priority Queues in Action Priority Queues Heaps Operation Linked List Binary Binomial Fibonacci * Relaxed Dijkstra's Shortest Path Algorithm make-heap 1 1 1 1 1 PQinit() insert 1 log N log N 1 1 ∈ for each v V find-min N 1 log N 1 1 key(v) ←∞ PQinsert(v) delete-min N log N log N log N log N union 1 N log N 1 1 key(s) ← 0 decrease-key 1 log N log N 1 1 while (!PQisempty()) v = PQdelmin() delete N log N log N log N log N for each w ∈ Q s.t (v,w) ∈ E is-empty 1 1 1 1 1 if π(w) > π(v) + c(v,w) PQdecrease(w, π(v) + c(v,w)) Dijkstra/Prim O(|V|2) O(|E| log |V|) O(|E| + |V| log |V|) 1 make-heap |V| insert |V| delete-min |E| decrease-key 3 4 Binary Heap: Definition Binary Heap: Properties Binary heap. Properties. ■ Almost complete binary tree. ■ Min element is in root. – ■ filled on all levels, except last, where filled from left to right Heap with N elements has height = log2 N . ■ Min-heap ordered. – every child greater than (or equal to) parent 06 06 N = 14 Height = 3 14 45 14 45 78 18 47 53 78 18 47 53 83 91 81 77 84 99 64 83 91 81 77 84 99 64 5 6 Binary Heaps: Array Implementation Binary Heap: Insertion Implementing binary heaps. Insert element x into heap. ■ Use an array: no need for explicit parent or child pointers. ■ Insert into next available slot. – Parent(i) = i/2 ■ Bubble up until it's heap ordered. – Left(i) = 2i – Peter principle: nodes rise to level of incompetence – Right(i) = 2i + 1 06 06 1 14 45 14 45 2 3 78 18 47 53 78 18 47 53 4 567 83 91 81 77 84 99 64 83 91 81 77 84 99 64 42 next free slot 891011 12 13 14 7 8 Binary Heap: Insertion Binary Heap: Insertion Insert element x into heap. Insert element x into heap. ■ Insert into next available slot. ■ Insert into next available slot. ■ Bubble up until it's heap ordered. ■ Bubble up until it's heap ordered. – Peter principle: nodes rise to level of incompetence – Peter principle: nodes rise to level of incompetence 06 swap with parent 06 swap with parent 14 45 14 45 78 18 47 53 78 18 47 42 83 91 81 77 84 99 64 42 83 91 81 77 84 99 64 4253 9 10 Binary Heap: Insertion Binary Heap: Decrease Key Insert element x into heap. Decrease key of element x to k. ■ Insert into next available slot. ■ Bubble up until it's heap ordered. ■ Bubble up until it's heap ordered. ■ O(log N) operations. – Peter principle: nodes rise to level of incompetence ■ O(log N) operations. 06 stop: heap ordered 06 14 42 14 42 78 18 47 45 78 18 47 45 83 91 81 77 84 99 64 53 83 91 81 77 84 99 64 53 11 12 Binary Heap: Delete Min Binary Heap: Delete Min Delete minimum element from heap. Delete minimum element from heap. ■ Exchange root with rightmost leaf. ■ Exchange root with rightmost leaf. ■ Bubble root down until it's heap ordered. ■ Bubble root down until it's heap ordered. – power struggle principle: better subordinate is promoted – power struggle principle: better subordinate is promoted 06 53 14 42 14 42 78 18 47 45 78 18 47 45 83 91 81 77 84 99 64 53 83 91 81 77 84 99 64 06 13 14 Binary Heap: Delete Min Binary Heap: Delete Min Delete minimum element from heap. Delete minimum element from heap. ■ Exchange root with rightmost leaf. ■ Exchange root with rightmost leaf. ■ Bubble root down until it's heap ordered. ■ Bubble root down until it's heap ordered. – power struggle principle: better subordinate is promoted – power struggle principle: better subordinate is promoted 53 exchange with left child 14 exchange with right child 14 42 53 42 78 18 47 45 78 18 47 45 83 91 81 77 84 99 64 83 91 81 77 84 99 64 15 16 Binary Heap: Delete Min Binary Heap: Heapsort Delete minimum element from heap. Heapsort. ■ Exchange root with rightmost leaf. ■ Insert N items into binary heap. ■ Bubble root down until it's heap ordered. ■ Perform N delete-min operations. – power struggle principle: better subordinate is promoted ■ O(N log N) sort. ■ O(log N) operations. ■ No extra storage. 14 stop: heap ordered 18 42 78 53 47 45 83 91 81 77 84 99 64 17 18 Binary Heap: Union Priority Queues Union. Heaps ■ Combine two binary heaps H1 and H2 into a single heap. Operation Linked List Binary Binomial Fibonacci * Relaxed ■ No easy solution. make-heap 1 1 1 1 1 Ω – (N) operations apparently required insert 1 log N log N 1 1 ■ Can support fast union with fancier heaps. find-min N 1 log N 1 1 delete-min N log N log N log N log N H H 1 2 union 1 N log N 1 1 14 11 decrease-key 1 log N log N 1 1 delete N log N log N log N log N is-empty 1 1 1 1 1 78 18 53 62 41 91 81 77 84 99 64 19 20 Binomial Tree Binomial Tree Binomial tree. Useful properties of order k binomial tree Bk. k ■ Recursive definition: ■ Number of nodes = 2 . B Bk 0 ■ Height = k. Bk+1 ■ Degree of root = k. ■ Deleting root yields binomial Bk-1 trees Bk-1, … , B0. B1 B0 Bk-1 B2 Proof. Bk ■ By induction on k. B0 B1 B2 B3 B4 B0 B1 B2 B3 B4 21 22 Binomial Tree Binomial Heap A property useful for naming the data structure. Binomial heap. Vuillemin, 1978. k ■ B has nodes at depth i. ■ Sequence of binomial trees that satisfy binomial heap property. k i – each tree is min-heap ordered – 0 or 1 binomial tree of order k 4 = 6 2 depth 0 6 3 18 depth 1 8 29 10 44 37 depth 2 30 23 22 48 31 17 depth 3 45 32 24 50 depth 4 B4 55 B4 B1 B0 23 24 Binomial Heap: Implementation Binomial Heap: Properties Implementation. Properties of N-node binomial heap. ■ Represent trees using left-child, right sibling pointers. ■ Min key contained in root of B0, B1, . , Bk. – ■ ⋅ three links per node (parent, left, right) Contains binomial tree Bi iff bi = 1 where bn b2b1b0 is binary ■ Roots of trees connected with singly linked list. representation of N. – degrees of trees strictly decreasing from left to right ■ At most log2 N + 1 binomial trees. ≤ heap ■ Height log2 N . 6 3 18 6 3 18 6 3 18 t n re a P 37 29 10 44 37 29 R 8 29 10 44 37 ig ft h e t L 30 23 22 48 31 17 N = 19 48 31 17 48 10 # trees = 3 height = 4 45 32 24 50 binary = 10011 50 50 31 17 44 55 Binomial Heap Leftist Power-of-2 Heap B4 B1 B0 25 26 Binomial Heap: Union Binomial Heap: Union Create heap H that is union of heaps H' and H''. ■ "Mergeable heaps." ■ Easy if H' and H'' are each order k binomial trees. 6 3 18 – connect roots of H' and H'' – choose smaller key to be root of H 8 29 10 44 37 30 23 22 48 31 17 15 7 12 6 45 32 24 50 28 33 25 55 8 29 10 44 + 41 30 23 22 48 31 17 1 1 1 1 0 0 1 1 45 32 24 50 19 + 7 = 26 + 0 0 1 1 1 55 1 1 0 1 0 H' H'' 27 28 Binomial Heap: Union Binomial Heap: Union 12 18 6 3 18 6 3 18 8 29 10 44 37 8 29 10 44 37 30 23 22 48 31 17 30 23 22 48 31 17 15 7 12 15 7 12 45 32 24 50 45 32 24 50 28 33 25 28 33 25 55 55 + 41 + 41 29 30 3 12 3 3 12 7 37 18 15 7 37 7 37 18 25 28 33 25 25 41 6 3 18 6 3 18 8 29 10 44 37 8 29 10 44 37 30 23 22 48 31 17 30 23 22 48 31 17 15 7 12 15 7 12 45 32 24 50 45 32 24 50 28 33 25 28 33 25 55 55 + 41 + 41 12 12 18 18 31 32 3 3 12 3 3 12 15 7 37 7 37 18 15 7 37 7 37 18 28 33 25 25 28 33 25 25 41 41 6 3 18 6 3 18 8 29 10 44 37 8 29 10 44 37 30 23 22 48 31 17 30 23 22 48 31 17 15 7 12 15 7 12 45 32 24 50 45 32 24 50 28 33 25 28 33 25 55 55 + 41 + 41 3 12 6 3 12 15 7 37 18 8 29 10 44 15 7 37 18 28 33 25 30 23 22 48 31 17 28 33 25 41 45 32 24 50 41 33 55 34 Binomial Heap: Union Binomial Heap: Delete Min Create heap H that is union of heaps H' and H''.