www.getmyuni.com
Data Structures Mergeable Heaps www.getmyuni.com Summary of Heap ADT Analysis
Consider a heap of N nodes Space needed: O(N) Actually, O(MaxSize) where MaxSize is the size of the array Pointer-based implementation: pointers for children and parent Total space = 3N + 1 (3 pointers per node + 1 for size) FindMin: O(1) time; DeleteMin and Insert: O(log N) time BuildHeap from N inputs: What is the run time? N Insert operations = O(N log N) O(N): Treat input array as a heap and fix it using percolate down Thanks, Floyd! www.getmyuni.com Other Heap Operations
Find and FindMax: O(N) DecreaseKey(P,,H): Subtract from current key value at position P and percolate up. Running Time: O(log N) IncreaseKey(P,,H): Add to current key value at P and percolate down. Running Time: O(log N) E.g. Schedulers in OS often decrease priority of CPU-hogging jobs Delete(P,H): Use DecreaseKey (to 0) followed by DeleteMin. Running Time: O(log N) E.g. Delete a job waiting in queue that has been preemptively terminated by user www.getmyuni.com
But What About...
• Merge(H1,H2): Merge two heaps H1 and H2 of size O(N). • E.g. Combine queues from two different sources to run on one CPU. 1. Can do O(N) Insert operations: O(N log N) time 2. Better: Copy H2 at the end of H1 (assuming array implementation) and use Floyd’s Method for BuildHeap. Running Time: O(N) Can we do even better? (i.e. Merge in O(log N) time?) www.getmyuni.com
Binomial Queues
Binomial queues support all three priority queue operations Merge, Insert and DeleteMin in O(log N) time Idea: Maintain a collection of heap-ordered trees Forest of binomial trees Recursive Definition of Binomial Tree (based on height k): Only one binomial tree for a given height Binomial tree of height 0 = single root node
Binomial tree of height k = Bk = Attach Bk-1 to root of another Bk-1 www.getmyuni.com Building a Binomial Tree
• To construct a binomial tree Bk of height k: 1. Take the binomial tree Bk-1 of height k-1 2. Place another copy of Bk-1 one level below the first 3. Attach the root nodes • Binomial tree of height k has exactly 2k nodes (by induction)
B0 B1 B2 B3 www.getmyuni.com Building a Binomial Tree
• To construct a binomial tree Bk of height k: 1. Take the binomial tree Bk-1 of height k-1 2. Place another copy of Bk-1 one level below the first 3. Attach the root nodes • Binomial tree of height k has exactly 2k nodes (by induction)
B0 B1 B2 B3 www.getmyuni.com Building a Binomial Tree
• To construct a binomial tree Bk of height k: 1. Take the binomial tree Bk-1 of height k-1 2. Place another copy of Bk-1 one level below the first 3. Attach the root nodes • Binomial tree of height k has exactly 2k nodes (by induction)
B0 B1 B2 B3 www.getmyuni.com Building a Binomial Tree
• To construct a binomial tree Bk of height k: 1. Take the binomial tree Bk-1 of height k-1 2. Place another copy of Bk-1 one level below the first 3. Attach the root nodes • Binomial tree of height k has exactly 2k nodes (by induction)
B0 B1 B2 B3 www.getmyuni.com Building a Binomial Tree
• To construct a binomial tree Bk of height k: 1. Take the binomial tree Bk-1 of height k-1 2. Place another copy of Bk-1 one level below the first 3. Attach the root nodes • Binomial tree of height k has exactly 2k nodes (by induction)
B0 B1 B2 B3 www.getmyuni.com Building a Binomial Tree
• To construct a binomial tree Bk of height k: 1. Take the binomial tree Bk-1 of height k-1 2. Place another copy of Bk-1 one level below the first 3. Attach the root nodes • Binomial tree of height k has exactly 2k nodes (by induction)
B0 B1 B2 B3 www.getmyuni.com Building a Binomial Tree
• To construct a binomial tree Bk of height k: 1. Take the binomial tree Bk-1 of height k-1 2. Place another copy of Bk-1 one level below the first 3. Attach the root nodes • Binomial tree of height k has exactly 2k nodes (by induction)
B0 B1 B2 B3 www.getmyuni.com Why Binomial? • Why are these trees called binomial? • Hint: how many nodes at depth d?
B0 B1 B2 B3 www.getmyuni.com Why Binomial? • Why are these trees called binomial? • Hint: how many nodes at depth d? Number of nodes at different depths d for Bk = [1], [1 1], [1 2 1], [1 3 3 1], … Binomial coefficients of (a + b)k = k!/((k-d)!d!)
B0 B1 B2 B3 www.getmyuni.com
Definition of Binomial Queues
Binomial Queue = “forest” of heap-ordered binomial trees
B0 B2 B0 B1 B3 3 5 21 1 -1
9 6 7 2 1 3
7 8 11 5
Binomial queue H1 Binomial queue H2 6 5 elements = 101 base 2 11 elements = 1011 base 2
B2 B0 B3 B1 B0 www.getmyuni.com Binomial Queue Properties
Suppose you are given a binomial queue of N nodes 1. There is a unique set of binomial trees for N nodes 2. What is the maximum number of trees that can be in an N-node queue?
• 1 node 1 tree B0; 2 nodes 1 tree B1; 3 nodes 2 trees B0 and B1; 7 nodes 3 trees B0, B1 and B2 … 0 1 k k+1 • Trees B0, B1, …, Bk can store up to 2 + 2 + … + 2 = 2 – 1 nodes = N. • Maximum is when all trees are used. So, solve for (k+1). • Number of trees is log(N+1) = O(log N) www.getmyuni.com Binomial Queues: Merge
• Main Idea: Merge two binomial queues by merging individual binomial trees
• Since Bk+1 is just two Bk’s attached together, merging trees is easy • Steps for creating new queue by merging:
1. Start with Bk for smallest k in either queue. 2. If only one Bk, add Bk to new queue and go to next k.
3. Merge two Bk’s to get new Bk+1 by making larger root the child of smaller root. Go to step 2 with k = k + 1. www.getmyuni.com Example: Binomial Queue Merge
• Merge H1 and H2 H1: H2: 3 5 21 1 -1
9 6 7 2 1 3
7 8 11 5
6 www.getmyuni.com Example: Binomial Queue Merge
• Merge H1 and H2 H1: H2: 3 5 1 -1
21 9 6 7 2 1 3
7 8 11 5
6 www.getmyuni.com Example: Binomial Queue Merge
• Merge H1 and H2 H1: H2: 3 5 1 -1
21 9 6 7 2 1 3
7 8 11 5
6 www.getmyuni.com Example: Binomial Queue Merge
• Merge H1 and H2 H1: H2: 5 1 -1
9 6 7 3 2 1 3 21 7 8 11 5
6 www.getmyuni.com Example: Binomial Queue Merge
• Merge H1 and H2 H1: H2: 5 1 -1
9 6 7 3 2 1 3 21 7 8 11 5
6 www.getmyuni.com Example: Binomial Queue Merge
• Merge H1 and H2 H1: H2: 1 -1
7 3 5 2 1 3
21 9 6 8 11 5
7 6 www.getmyuni.com Example: Binomial Queue Merge
• Merge H1 and H2 H1: H2: 1 -1
7 3 5 2 1 3
21 9 6 8 11 5
7 6 www.getmyuni.com Example: Binomial Queue Merge
• Merge H1 and H2 H1: H2: -1
2 1 3 1
8 11 5 7 3 5 21 6 9 6
7 www.getmyuni.com Example: Binomial Queue Merge
• Merge H1 and H2 H1: H2: -1
2 1 3 1
8 11 5 7 3 5 21 6 9 6
7 www.getmyuni.com
Binomial Queues: Merge and Insert
• What is the run time for Merge of two O(N) queues? • How would you insert a new item into the queue? www.getmyuni.com
Binomial Queues: Merge and Insert
• What is the run time for Merge of two O(N) queues? • O(number of trees) = O(log N) • How would you insert a new item into the queue?
• Create a single node queue B0 with new item and merge with existing queue • Again, O(log N) time • Example: Insert 1, 2, 3, …,7 into an empty binomial queue www.getmyuni.com
Insert 1,2,…,7
1 www.getmyuni.com
Insert 1,2,…,7
1
2 www.getmyuni.com
Insert 1,2,…,7
1 3
2 www.getmyuni.com
Insert 1,2,…,7
1 3
2 4 www.getmyuni.com
Insert 1,2,…,7
1
2 3
4 www.getmyuni.com
Insert 1,2,…,7
1 5
2 3
4 www.getmyuni.com
Insert 1,2,…,7
1 5
2 3 6
4 www.getmyuni.com
Insert 1,2,…,7
1 5 7
2 3 6
4 www.getmyuni.com Binomial Queues: DeleteMin
• Steps:
1. Find tree Bk with the smallest root 2. Remove Bk from the queue 3. Delete root of Bk (return this value); You now have a new queue made up of the forest B0, B1, …, Bk-1 4. Merge this queue with remainder of the original (from step 2) • Run time analysis: Step 1 is O(log N), step 2 and 3 are O(1), and step 4 is O(log N). Total time = O(log N) • Example: Insert 1, 2, …, 7 into empty queue and DeleteMin www.getmyuni.com
Insert 1,2,…,7
1 5 7
2 3 6
4 www.getmyuni.com
DeleteMin
5 7
2 3 6
4 www.getmyuni.com
Merge
5
2 3 6
7 4 www.getmyuni.com
Merge
5
2 3 6
7 4 www.getmyuni.com
Merge
5
2 6
7 3
4 www.getmyuni.com
Merge
5
2 6
7 3
4 DONE! www.getmyuni.com Implementation of Binomial Queues
Need to be able to scan through all trees, and given two binomial queues find trees that are same size Use array of pointers to root nodes, sorted by size Since is only of length log(N), don’t have to worry about cost of copying this array At each node, keep track of the size of the (sub) tree rooted at that node Want to merge by just setting pointers Need pointer-based implementation of heaps DeleteMin requires fast access to all subtrees of root Use First-Child/Next-Sibling representation of trees www.getmyuni.com Other Mergeable Priority Queues: Leftist and Skew Heaps
Leftist Heaps: Binary heap-ordered trees with left subtrees always “longer” than right subtrees Main idea: Recursively work on right path for Merge/Insert/DeleteMin Right path is always short has O(log N) nodes Merge, Insert, DeleteMin all have O(log N) running time (see text) Skew Heaps: Self-adjusting version of leftist heaps (a la splay trees) Do not actually keep track of path lengths Adjust tree by swapping children during each merge O(log N) amortized time per operation for a sequence of M operations We will skip details… just recognize the names as mergeable heaps! www.getmyuni.com
Coming Up
• Some random randomized data structures • Treaps • Skip Lists www.getmyuni.com
BINOMIAL HEAP & FIBONACCI HEAP www.getmyuni.com index Binomial heap : 1.Find Minimum Key 2.Insert New Node 3.Delete a Node 4.Unite Two Binomial Heaps 5.Binomial Heap Union www.getmyuni.com Fibonacci heap: 1.insert 2.union 3.Delete min 4.Decrease key 5.Delete www.getmyuni.com
Binomial Heap www.getmyuni.com
Binomial Heap
• The binary heap data structure is fine for the simple operations of inserting, deleting and extracting elements, but other operations aren't so well supported. • One such operation is the Union operation, which joins two heaps together. • If the heaps are binary heaps then this requires building up a new heap from scratch, using the elements of the old heaps, which is expensive for large heaps. • Binomial heap presents the data structure, which supports Union operations more efficiently. www.getmyuni.com
Binomial Trees
• The binomial tree is the building block for the binomial heap. A binomial tree is an ordered tree that is, a tree where the children of each node are ordered.
• Binomial trees are defined recursively, building up from single nodes. A single tree of degree k is constructed from two trees of degree k - 1 by making the root of one tree the leftmost child of the root of the other tree. www.getmyuni.com
The Binomial Heap Properties
A binomial heap is a collection of binomial trees that satisfies the following binomial-heap properties: 1. No two binomial trees in the collection have the same size. 2. Each node in each tree has a key. 3. Each binomial tree in the collection is heap-ordered in the sense that each non-root has a key strictly less than the key of its parent.
• The number of trees in a binomial heap is O(log n). www.getmyuni.com
Operations on Binomial Heaps
• Creation of a new heap. • Search for the minimum key. • Uniting two binomial heaps. • Insertion of a node. • Removal of the root of a tree. • Decreasing a key. • Removal of a node. www.getmyuni.com
Search for the Minimum Key
• To do this we find the smallest key among those stored at the roots connected to the head of H. What's the cost of minimum-search? The cost is O(log n) because there are O(log n) heaps, in each tree the minimum is located at the root, and the roots are linked. www.getmyuni.com
Find Minimum Key www.getmyuni.com Insert New Node www.getmyuni.com
Delete a Node www.getmyuni.com www.getmyuni.com
Unite Two Binomial Heaps www.getmyuni.com
Binomial Heap Union
• Create heap H that is union of heaps H’ and H”. • “Mergeable heaps” • Easy if H’ and H” are each order k binomial trees Connect roots of H’ and H” Choose smaller key to be root of H • Running Time O(log N) www.getmyuni.com www.getmyuni.com www.getmyuni.com FIBONACCI HEAP
18 www.getmyuni.com
Fibonacci Heap Operations: Insert Union Delete min Decrease key Delete
19 www.getmyuni.com Fibonacci Heaps
Fibonacci heap history. Fredman and Tarjan (1986) • Ingenious data structure and analysis. • Original motivation: O(m + n log n) shortest path algorithm. • also led to faster algorithms for MST, weighted bipartite matching • Still ahead of its time.
Fibonacci heap intuition. • Similar to binomial heaps, but less structured. • Decrease-key and union run in O(1) time. • "Lazy" unions.
20 www.getmyuni.com
Fibonacci Heaps: Structure
Fibonacci heap. • Set of min-heap ordered trees.
min
17 24 23 7 3
30 26 46 marked 18 52 41
35 44 H 39 21 www.getmyuni.com
Fibonacci Heaps: Insert
Insert. • Create a new singleton tree. • Add to left of min pointer. • Update min pointer. Insert 21
21 min
17 24 23 7 3
30 26 46 18 52 41
35 44 H 39 22 www.getmyuni.com
Fibonacci Heaps: Insert
Insert. • Create a new singleton tree. • Add to left of min pointer. • Update min pointer. Insert 21
min
17 24 23 7 21 3
30 26 46 18 52 41
35 44 H 39 23 www.getmyuni.com
Fibonacci Heaps: Insert
Insert. • Create a new singleton tree. • Add to left of min pointer. • Update min pointer. Insert 21 Running time. O(1) amortized • Actual cost = O(1). • Change in potential = +1. min • Amortized cost = O(1).
17 24 23 7 21 3
30 26 46 18 52 41
35 44 H 39 24 www.getmyuni.com
Fibonacci Heaps: Union
Union. • Concatenate two Fibonacci heaps. • Root lists are circular, doubly linked lists.
min min
23 24 17 7 3 21
H' H'' 30 26 46 18 52 41
35 44 39 25 www.getmyuni.com
Fibonacci Heaps: Union
Union. • Concatenate two Fibonacci heaps. • Root lists are circular, doubly linked lists.
Running time. O(1) amortized • Actual cost = O(1). • Change in potential = 0. min • Amortized cost = O(1).
23 24 17 7 3 21
H' H'' 30 26 46 18 52 41
35 44 39 26 www.getmyuni.com
Fibonacci Heaps: Delete Min
Delete min. • Delete min and concatenate its children into root list. • Consolidate trees so that no two roots have same degree.
min
7 24 23 17 3
30 26 46 18 52 41
35 39 44
27 www.getmyuni.com
Fibonacci Heaps: Delete Min
Delete min. • Delete min and concatenate its children into root list. • Consolidate trees so that no two roots have same degree.
current min
7 24 23 17 18 52 41
30 26 46 39 44
35
28 www.getmyuni.com
Fibonacci Heaps: Delete Min
Delete min. • Delete min and concatenate its children into root list. • Consolidate trees so that no two roots have same degree. 0 1 2 3
current min
7 24 23 17 18 52 41
30 26 46 39 44
35
29 www.getmyuni.com
Fibonacci Heaps: Delete Min
Delete min. • Delete min and concatenate its children into root list. • Consolidate trees so that no two roots have same degree. 0 1 2 3
current min
7 24 23 17 18 52 41
30 26 46 39 44
35
30 www.getmyuni.com
Fibonacci Heaps: Delete Min
Delete min. • Delete min and concatenate its children into root list. • Consolidate trees so that no two roots have same degree.
0 1 2 3
min
7 24 23 17 18 52 41
39 44 30 26 46 current
35
31 www.getmyuni.com
Fibonacci Heaps: Delete Min
Delete min. • Delete min and concatenate its children into root list. • Consolidate trees so that no two roots have same degree.
0 1 2 3
min
7 24 23 17 18 52 41
39 44 30 26 46 current
35 Merge 17 and 23 trees.
32 www.getmyuni.com
Fibonacci Heaps: Delete Min
Delete min. • Delete min and concatenate its children into root list. • Consolidate trees so that no two roots have same degree.
0 1 2 3
current min
7 24 17 18 52 41
30 26 46 23 39 44
35 Merge 7 and 17 trees.
33 www.getmyuni.com
Fibonacci Heaps: Delete Min
Delete min. • Delete min and concatenate its children into root list. • Consolidate trees so that no two roots have same degree.
0 1 2 3
min current
24 7 18 52 41
26 46 17 30 39 44
35 23 Merge 7 and 24 trees.
34 www.getmyuni.com
Fibonacci Heaps: Delete Min
Delete min. • Delete min and concatenate its children into root list. • Consolidate trees so that no two roots have same degree. 0 1 2 3
min current
7 18 52 41
24 17 30 39 44
26 46 23
35 35 www.getmyuni.com
Fibonacci Heaps: Delete Min
Delete min. • Delete min and concatenate its children into root list. • Consolidate trees so that no two roots have same degree.
0 1 2 3
min current
7 18 52 41
24 17 30 39 44
26 46 23
35 36 www.getmyuni.com
Fibonacci Heaps: Delete Min
Delete min. • Delete min and concatenate its children into root list. • Consolidate trees so that no two roots have same degree.
0 1 2 3
min current
7 18 52 41
24 17 30 39 44
26 46 23
35 37 www.getmyuni.com
Fibonacci Heaps: Delete Min
Delete min. • Delete min and concatenate its children into root list. • Consolidate trees so that no two roots have same degree.
0 1 2 3
min current
7 18 52 41
24 17 30 39 44
26 46 23 Merge 41 and 18 trees.
35 38 www.getmyuni.com
Fibonacci Heaps: Delete Min
Delete min. • Delete min and concatenate its children into root list. • Consolidate trees so that no two roots have same degree.
0 1 2 3
min current
7 52 18
24 17 30 41 39
26 46 23 44
35 39 www.getmyuni.com
Fibonacci Heaps: Delete Min
Delete min. • Delete min and concatenate its children into root list. • Consolidate trees so that no two roots have same degree.
0 1 2 3
min current
7 52 18
24 17 30 41 39
26 46 23 44
35 40 www.getmyuni.com
Fibonacci Heaps: Delete Min
Delete min. • Delete min and concatenate its children into root list. • Consolidate trees so that no two roots have same degree.
min
7 52 18
24 17 30 41 39
26 46 23 44 Stop.
35 41 www.getmyuni.com
Fibonacci Heaps: Decrease Key
Decrease key of element x to k. • Case 0: min-heap property not violated. • decrease key of x to k • change heap min pointer if necessary
min 7 18 38
24 17 23 21 39 41
26 4645 30 52
Decrease 46 to 45. 35 88 72 42 www.getmyuni.com
Fibonacci Heaps: Decrease Key
Decrease key of element x to k. • Case 1: parent of x is unmarked. • decrease key of x to k • cut off link between x and its parent • mark parent • add tree rooted at x to root list, updating heap min pointer min 7 18 38
24 17 23 21 39 41
26 4515 30 52
Decrease 45 to 15. 35 88 72 43 www.getmyuni.com
Fibonacci Heaps: Decrease Key
Decrease key of element x to k. • Case 1: parent of x is unmarked. • decrease key of x to k • cut off link between x and its parent • mark parent • add tree rooted at x to root list, updating heap min pointer min 7 18 38
24 17 23 21 39 41
26 15 30 52
Decrease 45 to 15. 35 88 72 44 www.getmyuni.com
Fibonacci Heaps: Decrease Key
Decrease key of element x to k. • Case 1: parent of x is unmarked. • decrease key of x to k • cut off link between x and its parent • mark parent • add tree rooted at x to root list, updating heap min pointer min 15 7 18 38
72 24 17 23 21 39 41
26 30 52
Decrease 45 to 15. 35 88 45 www.getmyuni.com
Fibonacci Heaps: Decrease Key Decrease key of element x to k. • Case 2: parent of x is marked. • decrease key of x to k • cut off link between x and its parent p[x], and add x to root list • cut off link between p[x] and p[p[x]], add p[x] to root list • If p[p[x]] unmarked, then mark it. • If p[p[x]] marked, cut off p[p[x]], unmark, and repeat.
min 15 7 18 38
72 24 17 23 21 39 41
26 30 52
Decrease 35 to 5. 355 88 46 www.getmyuni.com
Fibonacci Heaps: Decrease Key Decrease key of element x to k. • Case 2: parent of x is marked. • decrease key of x to k • cut off link between x and its parent p[x], and add x to root list • cut off link between p[x] and p[p[x]], add p[x] to root list • If p[p[x]] unmarked, then mark it. • If p[p[x]] marked, cut off p[p[x]], unmark, and repeat.
min 15 5 7 18 38
72 24 17 23 21 39 41
parent marked 26 30 52
Decrease 35 to 5. 88 47 www.getmyuni.com
Fibonacci Heaps: Decrease Key Decrease key of element x to k. • Case 2: parent of x is marked. • decrease key of x to k • cut off link between x and its parent p[x], and add x to root list • cut off link between p[x] and p[p[x]], add p[x] to root list • If p[p[x]] unmarked, then mark it. • If p[p[x]] marked, cut off p[p[x]], unmark, and repeat.
min 15 5 26 7 18 38
72 88 24 17 23 21 39 41
parent marked30 52
Decrease 35 to 5. 48 www.getmyuni.com
Fibonacci Heaps: Decrease Key
Decrease key of element x to k. • Case 2: parent of x is marked. • decrease key of x to k • cut off link between x and its parent p[x], and add x to root list • cut off link between p[x] and p[p[x]], add p[x] to root list • If p[p[x]] unmarked, then mark it. • If p[p[x]] marked, cut off p[p[x]], unmark, and repeat. min 15 5 26 24 7 18 38
72 88 17 23 21 39 41
30 52
Decrease 35 to 5. 49 www.getmyuni.com
Fibonacci Heaps: Delete Delete node x. • Decrease key of x to -. • Delete min element in heap.
Amortized cost. O(D(n)) • O(1) for decrease-key. • O(D(n)) for delete-min. • D(n) = max degree of any node in Fibonacci heap.
50