CSE 548: Analysis of Algorithms
Lectures 11, 12 & 13 ( Binomial Heaps )
Rezaul A. Chowdhury Department of Computer Science SUNY Stony Brook Fall 2012 Mergeable Heap Operations MAKE -HEAP ( ): return a new heap containing only element INSERT ( ): insert element into heap , MINIMUM ( ): return a pointer to an element in containing the smallest key
EXTRACT -MIN ( ): delete an element with the smallest key from and return a pointer to that element UNION ( ): return a new heap containing all elements of , heaps and , and destroy the input heaps More mergeable heap operations:
DECREASE -KEY ( ): change the key of element of heap to , , assuming the current key of DELETE ( ): delete element from heap , Mergeable Heap Operations
Heap Binary Heap Binomial Heap Operation ( worst-case ) ( amortized )
MAKE -HEAP Θ Θ 1 1 INSERT Ο Θ log 1 MINIMUM Θ Θ 1 1 EXTRACT -MIN Ο Ο log log UNION Θ Θ 1 DECREASE -KEY Ο log DELETE Ο log Binomial Trees
A binomial tree is an ordered tree defined recursively as follows. − consists of a single node − For , consists of two ’s that are linked together so that the 0 root of one is the left child of the root of the other Binomial Trees
Some useful properties of are as follows. 1. it has exactly nodes 2 2. its height is 3. there are exactly nodes at depth 0,1,2,…, 4. the root has degree 5. if the children of the root are numbered from left to right by , then child 1, 2,…,0 is the root of a Binomial Trees Prove: has exactly nodes at depth . 0,1,2,…, Proof: Suppose has nodes at depth . ,
0 0 , , 1 0, , , .
, , , , , , 1 , , ,
, ,
Binomial Trees
0 0 , , 1 0, , , .
⇒ , 0 , , 0 Generating function: , , , … , , , , 0 0 , , 0
0 1 0 1 0, ⇒ 1 . 1 Equating the coefficient of from both sides: , Binomial Heaps
A binomial heap is a set of binomial trees that satisfies the following properties: Binomial Heaps
A binomial heap is a set of binomial trees that satisfies the following properties:
1. each node has a key
2. each binomial tree in obeys the min-heap property 3. for any integer , there is at most one binomial tree in whose root node 0 has degree
6 1 10
8 14 29 12 25
11 17 38 18 min
27 Rank of Binomial Trees
The rank of a binomial tree node , denoted , is the number of children of . The figure on the right shows the rank of each node in . 3 Observe that . 2 1 0 Rank of a binomial tree is the rank of 1 0 0 its root. Hence, 0
A Basic Operation: Linking Two Binomial Trees
Given two binomial trees of the same rank , say, two ’s, we link them in constant time by making the root of one tree the left child of the root of the other, and thus 6 producing a . 8 14 29 If the trees are part of a binomial 11 17 38 min-heap, we always make the root 27 with the smaller key the parent, and the one with the larger key the child.
Ties are broken arbitrarily. Binomial Heap Operations: UNION ( ) ,
8 12
11 17
27 min 6 1 18
14 29 25
38 min
min Binomial Heap Operations: UNION ( ) ,
8 12
11 17
27 min 6 1 18
14 29 25
38 min
12
18
min Binomial Heap Operations: UNION ( ) ,
8 12
11 17
27 min
6 1 18
14 29 25
38 min
12
18
min Binomial Heap Operations: UNION ( ) ,
8 12
11 17
27 min
6 1 18
14 29 25
38 min
1 12
12 25 18
18
min Binomial Heap Operations: UNION ( ) ,
8 12
11 17
27 min 6 1 18
14 29 25
38 min
1
12 25
18
min Binomial Heap Operations: UNION ( ) ,
8 12
11 17
27 min 6 1 18
14 29 25
38 min
6 1
8 14 29 12 25
11 17 38 18
27 min Binomial Heap Operations: UNION ( ) ,
8 12
11 17
27 min
6 1 18
14 29 25
38 min
, 6 1
8 14 29 12 25
11 17 38 18
27 min Binomial Heap Operations: UNION ( ) UNION works in exactly the same way as binary addition., , Let be the number of nodes in . 1,2 Then the largest binomial tree in is a , where . log Thus can be treated as a bit binary number , where bit is 1 if contains a , and 0 otherwise. 1 If , then can be viewed as a , bit binary number log , where . Binomial Heap Operations: UNION ( ) UNION works in exactly the same way as binary addition., , Initially, does not contain any binomial trees. Melding starts from ( LSB ) and continues up to ( MSB ). At each location , one encounters at most ∈ three 0, ( 3 ) ’s: − at most 1 from ( input ), − at most 1 from ( input ), and − if at most 1 from ( carry ) 0, Binomial Heap Operations: UNION ( ) UNION works in exactly the same way as binary addition., , When the number of ’s at location is: ∈ 0, − 0: location of is set to − 1: location of points to that − 2: the two ’s are linked to produce a which is stored as a carry at location of , and location is set 1 to − 3: two ’s are linked to produce a which is stored as a carry at location of , and the 3 rd is 1 stored at location Binomial Heap Operations: UNION ( ) UNION works in exactly the same way as binary addition., , Worst case cost of UNION is clearly Θ , where is the total number of nodes in , and . log Observe that this operation fills out locations of , where . 1 log It does only Θ work for each location. 1
Hence, total cost is Θ Θ . log Binomial Heap Operations: UNION ( )
One can improve the performance of UNION as follows., , W.l.o.g., suppose is at least as large as , i.e., . We also assume that has enough space to store at least up to , where, . log Then instead of melding and to a new heap , we can meld them in-place at . After melding till , we stop once the carry stops propagating. The cost is , but Ο . Worst-case cost is still Ο Ο . log Binomial Heap Operations: INSERT ( ) , Step 1: MAKE -HEAP ′ 5 ′Takes ← Θ time. 1 min ′ Step 2: UNION 8 12 ← ( in-place , at ′ ) 11 17 Takes Ο time, where 27 min is the number oflog nodes in .
Thus the worst-case cost of 8 5 11 17 12 INSERT is Ο , where 27 min is the , number oflog items already in the heap. Binomial Heap Operations: EXTRACT -MIN ( ) 6
Step 1: Step 2: remove the binomial tree with remove the smallest root from the input heap minimum min 6 10 element 8 14 29 12 25
11 17 38 18 27 10
12 25
Step 3: remove the root of the binomial 18 min Tree with the minimum element, and form a new binomial heap from the children of the removed root Step 4: UNION and update the min pointer ,
′ 8 14 29 8 14 29 11 17 38 10 11 17 38 27 min ′ 12 25 27 min 18 Binomial Heap Operations: EXTRACT -MIN ( ) 6
Step 1: Step 2: remove the binomial tree with remove the smallest root from the input heap minimum min 6 10 element Θ 8 14 29 12 25 Θ 1 11 17 38 18 1 27 10
12 25
Step 3: remove the root of the binomial 18 min Tree with the minimum element, and form a new binomial heap from the children of the removed root Step 4: UNION and update the min pointer Ο , log Ο log ′ 8 14 29 8 14 29 11 17 38 10 11 17 38 27 min ′ 12 25 27 min 18 Thus, the worst-case cost of EXTRACT -MIN is Ο log Binomial Heap Operations
Heap Worst-case Operation
MAKE -HEAP Θ 1 INSERT Ο log MINIMUM Θ 1 EXTRACT -MIN Ο log UNION Ο log Amortized Analysis ( Accounting Method ) We maintain a credit account for every tree in the heap, and always maintain the following invariant:
1 ∈ MAKE -HEAP ( ): actual cost, ( for creating the singleton heap ) extra charge, 1 ( for storing in the credit account 1 of the new tree ) amortized cost, Θ ̂ 2 1 Amortized Analysis ( Accounting Method ) We maintain a credit account for every tree in the heap, and always maintain the following invariant:
1 ∈ LINK ( ): actual, cost, ( for linking the two trees ) 1 We use pay for this actual work. Let be the newly created tree. We restore the credit invariant by transferring to . Hence, amortized cost, ̂ 1 1 0 Amortized Analysis ( Accounting Method ) We maintain a credit account for every tree in the heap, and always maintain the following invariant:
1 ∈ INSERT ( ): , Amortized cost of MAKE -HEAP is 2 Then UNION is simply a sequence of free LINK operations with only a constant , amount of additional work that do not create any new trees. Thus the credit invariant is maintained, and the amortized cost of this step is . 1 Hence, amortized cost of INSERT , Θ ̂ 2 1 3 1 Amortized Analysis ( Accounting Method ) We maintain a credit account for every tree in the heap, and always maintain the following invariant:
1 ∈ UNION ( ): , UNION includes a sequence of free LINK operations that maintain the, credit invariant. But it also includes Ο other operations that are not free ( e.g., consider meldinglog a heap with elements with one containing elements ). These operations 2 do not create new trees (and so do 1 not violate the credit invariant), and each cost Θ . 1 Hence, amortized cost of UNION , Ο ̂ log Amortized Analysis ( Accounting Method ) We maintain a credit account for every tree in the heap, and always maintain the following invariant:
1 ∈ EXTRACT -MIN ( ): Steps 1 & 2: The Θ actual cost is paid for by the credit released by the deleted tree. 1 Step 3: Exposes Ο new trees, and we charge 1 unit of extra credit for storing inlog the credit account of each such tree.
Step 4: Performs a UNION that has Ο amortized cost. log Hence, amortized cost of EXTRACT -MIN , Ο ̂ log Amortized Analysis ( Potential Method ) Potential Function, ( #trees in the data structure after the -th operation ), whereΦ is a constant. Clearly, ( no trees in the data structure initially ) and for allΦ , 0 ( #trees cannot be negative ) 0 Φ 0 MAKE -HEAP ( ): actual cost, ( for creating the singleton heap ) potential change, 1 Δ ( Φ as #trees Φ increases by 1 ) amortized cost, Θ ̂ Δ 1 1 Amortized Analysis ( Potential Method ) Potential Function, ( #trees in the data structure after the -th operation ), whereΦ is a constant. INSERT ( ): ,The number of trees increases by 1 initially. Then the operation scans ( say ) locations of the array of tree pointers. Observe that we use tree 0 linking times each of which reduces the number of trees by . 1 1 actual cost, potential change, 1 Δ Φ Φ 1 1 amortized cost, 1 For , we have, ̂ Δ 2 Θ 1 1 1 ̂ 2 1 Amortized Analysis ( Potential Method ) Potential Function, ( #trees in the data structure after the -th operation ), whereΦ is a constant. UNION ( ): Suppose , the operation scans locations of the array of tree pointers, and uses the link operation times. 0 Observe that . Each link reduces the number of trees by . 0 1 actual cost, potential change, amortized cost, Δ Φ Φ ̂ Δ Since Ο and Ο , we have, Οlog for any .log ̂ log Amortized Analysis ( Potential Method ) Potential Function, ( #trees in the data structure after the -th operation ), whereΦ is a constant. EXTRACT -MIN ( ): Let in Step 1: rank of the tree with the smallest key and in Step 4: #locations of pointer array scanned during UNION #link operations during UNION #trees in the heap after the UNION Then actual cost, 1 step 1 1 step 2 step 3 step 4: union step 4: update ptr 2 Amortized Analysis ( Potential Method ) Potential Function, ( #trees in the data structure after the -th operation ), whereΦ is a constant. EXTRACT -MIN ( ): Let in Step 1: rank of the tree with the smallest key and in Step 4: #locations of pointer array scanned during UNION #link operations during UNION #trees in the heap after the UNION potential change, Δ Φ Φ 1 removing element in step 1 removes 1 tree but creates new ones linkings in step 4 reduces #trees by Amortized Analysis ( Potential Method ) Potential Function, ( #trees in the data structure after the -th operation ), whereΦ is a constant. EXTRACT -MIN ( ): Let in Step 1: rank of the tree with the smallest key and in Step 4: #locations of pointer array scanned during UNION #link operations during UNION #trees in the heap after the UNION actual cost, potential change, 2 Then amortized cost, Δ Φ Φ 1 Since Ο , ̂ Ο Δ , 2 Ο & Ο , 1 welog have, Οlog for anylog . log ̂ log Binomial Heap Operations
Heap Worst-case Amortized Operation
MAKE -HEAP Θ Θ 1 1 INSERT Ο Θ log 1 MINIMUM Θ Θ 1 1 EXTRACT -MIN Ο Ο log log UNION Ο Ο log log Binomial Heaps with Lazy Union We maintain pointers to the trees in a doubly linked circular list ( instead of an array ), but do not maintain a pointer. Binomial Heap Operations with Lazy Union
We maintain the following invariant: 2 ∈ MAKE -HEAP ( ): Create a singleton heap as before. Hence, amortized cost Θ . LINK ( ): The1 two input trees have 4 units of saved credits of which 1, unit will be used to pay for the actual cost of linking, and 2 units will be saved as credit for the newly created tree. So, linking is still free, and it has 1 unused credit that can be used to pay for additional work if necessary. UNION ( ): Simply concatenate the two root lists into one, and update , the min pointer. Clearly, amortized cost Θ .
INSERT ( ): This is MAKE -HEAP followed by a UNION . Hence,1 amortized , cost Θ . 1 Binomial Heap Operations with Lazy Union
We maintain the following invariant: 2 ∈ EXTRACT -MIN ( ): Unlike in the array version, in this case we may have several trees of the same rank. We create an array of length with each location containing a pointer. We uselog this array 1 to transform the linked list version to array version. We go through the list of trees of , inserting them one by one into the array, and linking and carrying if necessary so that finally we have at most one tree of each rank. We also create a min pointer.
We now perform EXTRACT -MIN as in the array case. Finally, we collect the nonempty trees from the array into a doubly linked list, and return. Binomial Heap Operations with Lazy Union
We maintain the following invariant: 2 ∈ EXTRACT -MIN ( ): We only need to show that converting from linked list version to array version takes Ο amortized time. Suppose we start with trees, and performlog links. So, we spend Ο time overall. As each link decreases the number of trees by , after links we end up with trees. Since at that point we have1 at most one tree of each rank, we have . Thus Οlog 1 . The Ο part 2 can be paid for by the log extra credits from links. We only charge the Ο part to EXTRACT -MIN . log Binomial Heap Operations with Lazy Union We use exactly the same potential function as in the previous version, ( #trees in the data structure after the -th operation ), whereΦ is a constant. As before, clearly, and for all ,Φ 0 0 Φ 0 MAKE -HEAP ( ): actual cost, ( for creating the singleton heap ) potential change, 1 Δ ( Φ as #trees Φ increases by 1 ) amortized cost, Θ ̂ Δ 1 1 Binomial Heap Operations with Lazy Union We use exactly the same potential function as in the previous version, ( #trees in the data structure after the -th operation ), whereΦ is a constant.
UNION ( ): actual, cost, ( for merging the two doubly linked lists ) potential change, 1 Δ ( Φno new tree Φ is created 0 or destroyed ) amortized cost, Θ ̂ Δ 1 1 Binomial Heap Operations with Lazy Union We use exactly the same potential function as in the previous version, ( #trees in the data structure after the -th operation ), whereΦ is a constant. INSERT ( ): ,Constant amount of work is done by MAKE -HEAP and UNION , and MAKE -HEAP creates a new tree.
actual cost, potential change, 1 1 2 Δ Φ Φ amortized cost, Θ ̂ Δ 2 1 Binomial Heap Operations with Lazy Union We use exactly the same potential function as in the previous version, ( #trees in the data structure after the -th operation ), whereΦ is a constant.
EXTRACT -MIN ( ): Cost of creating the array of pointers is . Suppose we start with trees in the doublylog linked 1list, and perform link operations during the conversion from linked list to array version. So we perform work, and end up with trees. Cost of converting to the linked list version is . actual cost, potential change, log 1 2 log 1 Δ Φ Φ Binomial Heap Operations with Lazy Union We use exactly the same potential function as in the previous version, ( #trees in the data structure after the -th operation ), whereΦ is a constant.
EXTRACT -MIN ( ): actual cost, potential change, log 1 2 log 1 Δ Φ Φ amortized cost, ̂ Δ 2 log 1 2 But ( as we have at most one tree of each rank ) log 1 So, ̂ 3 log 3 ( assuming 2 ) Ο3 log 3 2 log Binomial Heap Operations
Heap Amortized Amortized Worst-case Operation ( Eager Union ) ( Lazy Union )
MAKE - Θ Θ Θ HEAP 1 1 1 INSERT Ο Θ Θ log 1 1 MINIMUM Θ Θ Θ 1 1 1 EXTRACT - Ο Ο Ο MIN log log log UNION Ο Ο Θ log log 1