26.03.2009

Heaps http://en.wikipedia.org/wiki/Category:Heaps_(structure) • Binary heap Advanced Algorithmics (4AP) – http://en.wikipedia.org/wiki/Binary_heap Heaps • Binomial heap Jaak Vilo – http://en.wikipedia.org/wiki/Binomial_heap 2009 Spring • Fibonacci heap – http://en.wikipedia.org/wiki/Fibonacci_heap

Jaak Vilo MTAT.03.190 Text Algorithms 1

Heap/Priority queue

• Find min/Delete; Insert;

• Decrease key (change value of the key)

• Merge two heaps …

Binomial heaps:

• Performance: All of the following operations work in O(log n) • http://www.cse.yorku.ca/~aaw/Jason/Fibonac time on a binomial heap with n elements: – Insert a new element to the heap ciHeapAnimation.html • – Find the element with minimum key http://net.pku.edu.cn/~course/cs101/resource/Intro2Algorithm/book6/ch ap20.htm – Delete the element with minimum key from the heap – Decrease key of a given element • http://net.pku.edu.cn/~course/cs101/resource/Intro2Algorithm/book6/ch – Delete given element from the heap ap21.htm – Merge two given heaps to one heap – Finding the element with minimum key can also be done in O(1) by using an additional pointer to the minimum. • http://www.jucs.org/jucs_7_5/animation_for_teaching_purposes/Lauer_T.html

1 26.03.2009

Binomial heaps, • CLRS: Fibonacci heaps, • http://net.pku.edu.cn/~course/cs101/resource/Intro2Algorithm/book6/chap20.htm and applica tions

http://www.cs.tau.ac.il/~dannyf/ds09/ds09a.htm Dan Feldman

7

Binomial trees Binomial trees

Bi B0

B1

......

B0

Bi B(i-1) B(i-2) B1

B(i-1)

B(i-1)

9 10

2 26.03.2009

Lemma 20.1 Properties of binomial trees

• For the binomial tree Bk, k k • 1. there are 2 nodes, 1) | Bk | = 2 = • 2. the height of the tree is k, 2) degree(root(Bk)) k   = • 3. there are exactly choose(i from k) nodes at depth i 3)d) dep th(Bk) k for i = 0, 1, . . . , k, and • 4. the root has degree k, which is greater than that of ==> The degree and depth of a binomial tree with at any other node; moreover if the children of the root most n nodes is at most log(n). are numbered from left to right by k ‐ 1, k ‐ 2, . . . ,0,

child i is the root of a subtree Bi. Define the rank of Bk to be k

14

Figure 20.4 The binomial tree B4 with nodes labeled in binary by a postorder walk. Binomial heaps (def) A collection of binomial trees at most one of every rank. Items at the nodes, heap ordered.

1 5 5

2 5 6 6

5 9 8 Possible rep: Doubly link roots 10 and children of every node. Parent pointers needed for delete. 16

Binomial heaps (operations)

Operations are defined via a basic operation, called linking, of binomial trees:

Produce a Bk from two Bk-1, keep heap order. 1

4 2 5 6

5 6 11 5 9 8

6 9 9 10

10

18

3 26.03.2009

Binomial heaps (ops cont.) The execution of BINOMIAL‐HEAP‐UNION.(a) Binomial heaps H1 and H2.

Basic operation is meld(h1,h2): Like addition of binary numbers.

B5 B4 B2 B1

h1: B4 B3 B1 B0 + h2: B4 B3 B0

B5 B4 B2

19

The execution of BINOMIAL‐HEAP‐UNION.(a) Delete min Binomial heaps H1 and H2. (continued)

Find min (=1)

Extract tree

Split tree, reverse

Merge/meld

Decrease key (y=26 => y=7) Binomial heaps (ops cont.)

Findmin(h): obvious

Insert(x,h) : meld a new heap with a single B0 containing x, with h deletemin(h) : Chop off the minimal root. Meld the subtrees with h. Update minimum pointer if needed . delete(x,h) : Bubble up and continue like delete-min decrease-key(x,h,) : Bubble up, update min ptr if needed All operations take O(log n) time on the worst case, except find-min(h) that takes O(1) time.

24

4 26.03.2009

Amortized analysis Amortized analysis (Cont.)

We are interested in the worst case running time of a On the worst case increment takes O(k). sequence of operations. k = #digits Example: binary counter What is the complexity of a sequence of increments (th(on the worst case) )? ? single operation -- increment Define a potential of the counter: 00000 00001 (c) = ? 00010  00011 00100 Amortized(increment) = actual(increment) +  00101

25 26

Amortized analysis (Cont.) Amortized analysis (Cont.)

Define a potential of the counter: Amortized(increment1) = actual(increment1) + 1- 0 Amortized(increment ) = actual(increment ) +  -  2 2 2 1  (c) = #(ones) … + … Amor tized(i ncremen t) = ac tua l(incremen t) + 

Amortized(increment ) = actual(increment ) + - n n n (n-1) Amortized(increment) = 1+ #(1 => 0) + 1 - #(1 => 0) = O(1)

==> Sequence of n increments takes O(n) time iAmortized(incrementi) = iactual(incrementi) + n- 0

iAmortized(incrementi) iactual(incrementi)

if n- 0  0 27 28

Binomial heaps ‐ amortized ana. Binomial heaps + lazy meld

 (collection of heaps) = #(trees) Allow more than one tree of each rank.

Meld (h1,h2) : Amortized cost of insert O(1) •Concatenate the lists of binomial trees . Amortized cost of other operations still O(log n) •Update the minimum pointer to be the smaller of the minimums

O(1) worst case and amortized.

29 30

5 26.03.2009

Binomial heaps + lazy meld Binomial heaps + lazy meld Possible implementation of delete-min is using an array As long as we do not do a delete-min our heaps are indexed by rank to keep at most one binomial tree of just doubly linked lists: each rank that we already traversed. Once we encounter a second tree of some rank we link 9 5 9 11 4 6 them and keep linking until we do not have two trees of the same rank. We record the resulting tree in the array Delete-min : Chop off the minimum root, add its children to the list of trees. Amortized(delete-min) = Successive linking: Traverse the forest keep = (#links + max-rank) - #links linking trees of the same rank, maintain a pointer to the minimum root. = O(log(n))

31 32

Fibonacci heaps (Fredman & Tarjan Dijkstra’s shortest path algorithm 84)

Let G = (V,E) be a weighted (weights are non-negative) Want to do decrease-key(x,h,) faster than delete+insert. undirected graph, let s  V. Want to find the distance (length of the shortest path), d(s,v) from s to every other Ideally in O(1) time. vertex. 3 3 Why ? s 2 1 3 2

33 34

Insert (left from root)

6 26.03.2009

Figure 21.3 The action of FIB‐ Finding the minimum node HEAP‐EXTRACT‐MIN.

A Fibonacci heap H. (b) The situation after the • The minimum node of a Fibonacci heap H is minimum node z is removed from the root list and its children are added to the root list. (c)- (e) The array A and the trees after each of the given by the pointer min[H], so we can find first three iterations of the for loop of lines 3- 13 of the procedure CONSOLIDATE. The root list is processed by starting at the minimum the minimum node in O(1) actual time. node and following right pointers. Each part shows the values of w and x at the end of an iteration. (f)-(h) The next iteration of the for Because the potential of H does not change, loop, with the values of w and x shown at the end of each iteration of the while loop of lines 6-12. Part (f) shows the situation after the first the amortized cost of this operation is equal time through the while loop. The node with key 23 has been linked to the node with key 7, which is now pointed to by x. In part (g), the to its O(1) actual cost. node with key 17 has been linked to the node with key 7, which is still pointed to by x. In part (h), the node with key 24 has been linked to the node with key 7. Since no node was previously pointed to by A[3], at the end of the for loop iteration, A[3] is set to point to the root of the resulting tree. (i)-(l) The situation after each of the next four iterations of the while loop. (m) Fibonacci heap H after reconstruction of the root list from the array A and determination of the new min[H] pointer.

Two calls of FIB‐HEAP‐DECREASE‐KEY.

(a) The initial Fibonacci heap. (b) The node with key 46 has its key decreased to 15. The node becomes a root, and its parent (with key 24), which had previously been unmarked, becomes marked. (c)-(e) The node with key 35 has its key decreased to 5. In part (c), the node, now with key 5, becomes a root. Its parent, with key 26, is marked, so a cascading cut occurs. The node with key 26 is cut from its parent and made an unmarked root in (d). Another cascading cut occurs, since the node with key 24 is marked as well. This node is cut from its parent and made an unmarked root in part (e). The cascading cuts stop at this point, since the node with key 7 is a root. (Even if this node were not a root, the cascading cuts would stop, since it is unmarked.) The result of the FIB-HEAP-DECREASE-KEY operation is shown in part (e), with min[H] pointing to the new minimum node.

Application #2 : Prim’s algorithm Application #2 : Prim’s algorithm for MST for MST Maintain the vertices out of T but adjacent to T in a heap.

Start with T a singleton vertex. The key of a vertex v is the weight of the lightest edge (v,w) where w is in the tree. Grow a tree by repeating the following step: Iteration: Do a delete-min. Let v be the minimum vertex Add the minimum cost edge connecting a vertex in T to a and(d (v,w )th) the lihtlightes t e dge as a bove. Add(Add (v,w )t) to T. For vertex out of T. each edge (w,u) where uT, if key(u) =  insert u into the heap with key(u) = w(w,u) if w(w,u) < key(u) decrease the key of u to be w(w,u).

With regular heaps O(m log(n)). With F-heaps O(n log(n) + m). 41 42

7 26.03.2009

2 5 5 Suggested implementation for decrease-key(x,h,): 5 3 6 6 If x with its new key is smaller than its parent, cut the subtree rooted at x and add it to the forest. Update 5 9 8 the minimum pointer if necessary. 10 2 1 5 5

5 6 8 6

5 9

10 43 44

Decrease‐key (cont.) Decrease‐key (cont.)

Does it work ? 2 Obs1: Trees need not be binomial trees any more..

Do we need the trees to be binomial ? 9 5 3 6 5 6 Where have we used it ? In the analysis of delete-min we used the fact that at most log(n) new trees are added to the forest. This was Such trees are now legitimate. obvious since trees were binomial and contained at most So our analysis breaks down. n nodes.

45 46

Fibonacci heaps (cont.) Fibonacci heaps (cont.) We shall allow non-binomial trees, but will keep the Decrease-key (x,h,): indeed cuts the subtree rooted by degrees logarithmic in the number of nodes. x if necessary as we showed. Rank of a tree = degree of the root. in addition we maintain a mark bit for every node. When we cut the subtree rooted by x we check the mark bit of Delete-min: do successive linking of trees of the same p(x). If it is set then we cut p(x) too. We continue this way rank and update the minimum pointer as before. until either we reach an unmarked node in which case Insert and meld also work as before. we mark it, or we reach the root. This mechanism is called cascading cuts.

47 48

8 26.03.2009

2 Fibonacci heaps (delete) 4 20 Delete(x,h) : Cut the subtree rooted at x and then 5 8 11 proceed with cascading cuts as for decrease key. Chop off x from being the root of its subtree and add the 9 6 14 subtrees rooted by its children to the forest

10 16 If x i s th e mi ni mum nod e d o success ive lin king

12 15

7 9 5 4 2

20 12 15 16 6 14 8 11 49 50

• In the amortized running time analysis we • The potential of a Fibonacci heap is given by pretend that very fast operations take a little • Potential = t + 2m where t is the number of bit longer than they actually do. trees in the Fibonacci heap, and m is the • This additional time is then later subtracted number of marked nodes. A node is marked if from the actual running time of slow at least one of its children was cut since this operations. node was made a child of another node (all • The amount of time saved for later use is roots are unmarked). measured at any given moment by a potential function.

Fibonacci heaps (analysis) Fibonacci heaps (analysis)

Want everything to be O(1) time except for delete What about delete and delete-min ? and delete-min. ==> cascading cuts should pay for themselves Cascading cuts and successive linking will pay for themselves. The only question is what is the maximum degree of a node ?  (lltifh(collection of heaps )#(t)) = #(trees) + 2#(mar ke d How many trees are being added into the forest nodes) when we chop off a root ?

Actual(decrease-key) = O(1) + #(cascading cuts) (decrease-key) = O(1) - #(cascading cuts) ==> amortized(decrease-key) = O(1) !

53 54

9 26.03.2009

Fibonacci heaps (analysis) Fibonacci heaps (analysis)

Lemma 1 : Let x be any node in an F-heap. Arrange the Corollary1 : A node x of rank k in a F-heap has at least k children of x in the order they were linked to x, from earliest descendants, where  = (1 + 5)/2 is the golden ratio. to latest. Then the i-th child of x has rank at least i-2. Proof: x Let sk be the minimum number of descendants of a node of rank k in a F-heap. k- 2 1 By Lemma 1 sk 2 i=0si + 2 x Proof: When the i-th node was linked it must have had at least i- 1 children. s =1, s = 2 Since then it could have lost at most one. 0 1 55 56

Fibonacci heaps (analysis) Proof (cont): Fibonaci numbers satisfy k Fk+2 = i=2Fi + 2, for k  2, and F2=1

so by induction sk  Fk+2 k It is well known that Fk+2  

It follows that the maximum degree k in a F-heap with n nodes is such that k  n so k  log(n) / log() = 1.4404 log(n)

58

• C code – http://www.cs.unc.edu/~bbb/foss/binheaps/iheap.h – http://www.cs.unc.edu/~bbb/#binomial_heaps

10 26.03.2009

More…

• http://www.cse.yorku.ca/~aaw/Jason/FibonacciHeapAnimation.html

11