Priority Queues

Fibonacci Heaps

Heaps Operation Binary Binomial Fibonacci † Relaxed make- 1 1 1 1 1 insert 1 log N log N 1 1 These lecture slides are adapted find-min N 1 log N 1 1 from CLRS, Chapter 20. delete-min N log N log N log N log N union 1 N log N 1 1 decrease-key 1 log N log N 1 1 delete N log N log N log N log N is-empty 1 1 1 1 1

†amortized this time Princeton University • COS 423 • Theory of Algorithms • Spring 2002 • Kevin Wayne 2

Fibonacci Heaps Fibonacci Heaps: Structure

Fibonacci heap history. Fredman and Tarjan (1986) Fibonacci heap.

■ Ingenious and analysis. ■ of min-heap ordered trees.

■ Original motivation: O(m + n log n) shortest path algorithm. – also led to faster algorithms for MST, weighted bipartite

■ Still ahead of its time.

Fibonacci heap intuition.

■ Similar to binomial heaps, but less structured.

■ Decrease-key and union run in O(1) time. min ■ "Lazy" unions.

17 24 23 7 3

30 26 46 marked 18 52 41

35 H 39 44

3 4 Fibonacci Heaps: Implementation Fibonacci Heaps: Potential Function

Implementation. Key quantities.

■ Represent trees using left-child, right sibling pointers and circular, ■ Degree[x] = degree of node x.

doubly linked list. ■ Mark[x] = mark of node x (black or gray). – can quickly splice off subtrees ■ t(H) = # trees.

■ Roots of trees connected with circular doubly linked list. ■ m(H) = # marked nodes. – fast union ■ Φ(H) = t(H) + 2m(H) = potential function. ■ Pointer to root of with min element. – fast find-min t(H) = 5, m(H) = 3 Φ(H) = 11 min degree = 3 min

17 24 23 7 3 17 24 23 7 3

30 26 46 30 26 46 18 52 41 18 52 41

35 35 H 39 44 H 39 44

5 6

Fibonacci Heaps: Insert Fibonacci Heaps: Insert

Insert. Insert.

■ Create a new singleton tree. ■ Create a new singleton tree.

■ Add to left of min pointer. ■ Add to left of min pointer.

■ Update min pointer. ■ Update min pointer.

Insert 21 Insert 21

21 min min

17 24 23 7 3 17 24 23 7 21 3

30 26 46 30 26 46 18 52 41 18 52 41

35 35 H 39 44 H 39 44

7 8 Fibonacci Heaps: Insert Fibonacci Heaps: Union

Insert. Union.

■ Create a new singleton tree. ■ Concatenate two Fibonacci heaps.

■ Add to left of min pointer. ■ Root lists are circular, doubly linked lists.

■ Update min pointer.

Running time. O(1) amortized Insert 21 ■ Actual cost = O(1).

■ Change in potential = +1.

■ Amortized cost = O(1). min min min

17 24 23 7 21 3 23 24 17 7 3 21

H' H'' 30 26 46 30 26 46 18 52 41 18 52 41

35 35 H 39 44 39 44

9 10

Fibonacci Heaps: Union Fibonacci Heaps: Delete Min

Union. Delete min.

■ Concatenate two Fibonacci heaps. ■ Delete min and concatenate its children into root list.

■ Root lists are circular, doubly linked lists. ■ Consolidate trees so that no two roots have same degree.

Running time. O(1) amortized

■ Actual cost = O(1).

■ Change in potential = 0. min ■ Amortized cost = O(1). min 7 24 23 17 3

23 24 17 7 3 21 30 26 46 18 52 41 H' H'' 30 26 46 18 52 41 35 39 44 35 39 44

11 12 Fibonacci Heaps: Delete Min Fibonacci Heaps: Delete Min

Delete min. Delete min.

■ Delete min and concatenate its children into root list. ■ Delete min and concatenate its children into root list.

■ Consolidate trees so that no two roots have same degree. ■ Consolidate trees so that no two roots have same degree.

0 1 2 3

current current min min

7 24 23 17 18 52 41 7 24 23 17 18 52 41

30 26 46 39 44 30 26 46 39 44

35 35

13 14

Fibonacci Heaps: Delete Min Fibonacci Heaps: Delete Min

Delete min. Delete min.

■ Delete min and concatenate its children into root list. ■ Delete min and concatenate its children into root list.

■ Consolidate trees so that no two roots have same degree. ■ Consolidate trees so that no two roots have same degree.

0 1 2 3 0 1 2 3

current min min

7 24 23 17 18 52 41 7 24 23 17 18 52 41

39 44 39 44 30 26 46 30 26 46 current

35 35

15 16 Fibonacci Heaps: Delete Min Fibonacci Heaps: Delete Min

Delete min. Delete min.

■ Delete min and concatenate its children into root list. ■ Delete min and concatenate its children into root list.

■ Consolidate trees so that no two roots have same degree. ■ Consolidate trees so that no two roots have same degree.

0 1 2 3 0 1 2 3

current min min

7 24 23 17 18 52 41 7 24 17 18 52 41

39 44 23 39 44 30 26 46 current 30 26 46

35 35 Merge 17 and 23 trees. Merge 7 and 17 trees.

17 18

Fibonacci Heaps: Delete Min Fibonacci Heaps: Delete Min

Delete min. Delete min.

■ Delete min and concatenate its children into root list. ■ Delete min and concatenate its children into root list.

■ Consolidate trees so that no two roots have same degree. ■ Consolidate trees so that no two roots have same degree.

0 1 2 3 0 1 2 3

min current min current

24 7 18 52 41 7 18 52 41

26 46 17 30 39 44 24 17 30 39 44

35 23 26 46 23 Merge 7 and 24 trees. 35 19 20 Fibonacci Heaps: Delete Min Fibonacci Heaps: Delete Min

Delete min. Delete min.

■ Delete min and concatenate its children into root list. ■ Delete min and concatenate its children into root list.

■ Consolidate trees so that no two roots have same degree. ■ Consolidate trees so that no two roots have same degree.

0 1 2 3 0 1 2 3

min current min current

7 18 52 41 7 18 52 41

24 17 30 39 44 24 17 30 39 44

26 46 23 26 46 23

35 35 21 22

Fibonacci Heaps: Delete Min Fibonacci Heaps: Delete Min

Delete min. Delete min.

■ Delete min and concatenate its children into root list. ■ Delete min and concatenate its children into root list.

■ Consolidate trees so that no two roots have same degree. ■ Consolidate trees so that no two roots have same degree.

0 1 2 3 0 1 2 3

min current min current

7 18 52 41 7 52 18

24 17 30 39 44 24 17 30 41 39

26 46 23 26 46 23 44 Merge 41 and 18 trees. 35 35 23 24 Fibonacci Heaps: Delete Min Fibonacci Heaps: Delete Min

Delete min. Delete min.

■ Delete min and concatenate its children into root list. ■ Delete min and concatenate its children into root list.

■ Consolidate trees so that no two roots have same degree. ■ Consolidate trees so that no two roots have same degree.

0 1 2 3

min current min

7 52 18 7 52 18

24 17 30 41 39 24 17 30 41 39

26 46 23 44 26 46 23 44 Stop. 35 35 25 26

Fibonacci Heaps: Delete Min Analysis Fibonacci Heaps: Delete Min Analysis

Notation. Is amortized cost of O(D(n)) good?

■ D(n) = max degree of any node in Fibonacci heap with n nodes.

■ t(H) = # trees in heap H. ■ Yes, if only Insert, Delete-min, and Union operations supported.

■ Φ(H) = t(H) + 2m(H). – in this case, Fibonacci heap contains only binomial trees since we only merge trees of equal root degree – this implies D(n) ≤log N Actual cost. O(D(n) + t(H)) 2

■ O(D(n)) work adding min's children into root list and updating min. – at most D(n) children of min node ■ Yes, if we support Decrease-key in clever way. – we'll show that D(n) ≤logφ N, where φ is golden ratio ■ O(D(n) + t(H)) work consolidating trees. – φ2 = 1 + φ – work is proportional to size of root list since number of roots decreases by one after each merging – φ = (1 + √5) / 2 = 1.618… – ≤ D(n) + t(H) - 1 root nodes at beginning of consolidation – limiting ratio between successive Fibonacci numbers!

Amortized cost. O(D(n))

■ t(H') ≤ D(n) + 1 since no two trees have same degree.

■ ∆Φ(H) ≤ D(n) + 1 - t(H).

27 28 Fibonacci Heaps: Decrease Key Fibonacci Heaps: Decrease Key

Decrease key of element x to k. Decrease key of element x to k.

■ Case 0: min-heap property not violated. ■ Case 1: parent of x is unmarked. – decrease key of x to k – decrease key of x to k – change heap min pointer if necessary – cut off link between x and its parent – mark parent – add tree rooted at x to root list, updating heap min pointer

min min 7 18 38 7 18 38

24 17 23 21 39 41 24 17 23 21 39 41

26 4645 30 52 26 4515 30 52

Decrease 46 to 45. Decrease 45 to 15. 35 88 72 35 88 72 29 30

Fibonacci Heaps: Decrease Key Fibonacci Heaps: Decrease Key

Decrease key of element x to k. Decrease key of element x to k.

■ Case 1: parent of x is unmarked. ■ Case 1: parent of x is unmarked. – decrease key of x to k – decrease key of x to k – cut off link between x and its parent – cut off link between x and its parent – mark parent – mark parent – add tree rooted at x to root list, updating heap min pointer – add tree rooted at x to root list, updating heap min pointer

min min 7 18 38 15 7 18 38

24 17 23 21 39 41 72 24 17 23 21 39 41

26 15 30 52 26 30 52

Decrease 45 to 15. Decrease 45 to 15. 35 88 72 35 88 31 32 Fibonacci Heaps: Decrease Key Fibonacci Heaps: Decrease Key

Decrease key of element x to k. Decrease key of element x to k.

■ Case 2: parent of x is marked. ■ Case 2: parent of x is marked. – decrease key of x to k – decrease key of x to k – cut off link between x and its parent p[x], and add x to root list – cut off link between x and its parent p[x], and add x to root list – cut off link between p[x] and p[p[x]], add p[x] to root list – cut off link between p[x] and p[p[x]], add p[x] to root list ! If p[p[x]] unmarked, then mark it. ! If p[p[x]] unmarked, then mark it. ! If p[p[x]] marked, cut off p[p[x]], unmark, and repeat. ! If p[p[x]] marked, cut off p[p[x]], unmark, and repeat.

min min 15 7 18 38 15 5 7 18 38

72 24 17 23 21 39 41 72 24 17 23 21 39 41

26 30 52 parent marked 26 30 52

Decrease 35 to 5. Decrease 35 to 5. 355 88 88 33 34

Fibonacci Heaps: Decrease Key Fibonacci Heaps: Decrease Key

Decrease key of element x to k. Decrease key of element x to k.

■ Case 2: parent of x is marked. ■ Case 2: parent of x is marked. – decrease key of x to k – decrease key of x to k – cut off link between x and its parent p[x], and add x to root list – cut off link between x and its parent p[x], and add x to root list – cut off link between p[x] and p[p[x]], add p[x] to root list – cut off link between p[x] and p[p[x]], add p[x] to root list ! If p[p[x]] unmarked, then mark it. ! If p[p[x]] unmarked, then mark it. ! If p[p[x]] marked, cut off p[p[x]], unmark, and repeat. ! If p[p[x]] marked, cut off p[p[x]], unmark, and repeat.

min min 15 5 26 7 18 38 15 5 26 24 7 18 38

72 88 24 17 23 21 39 41 72 88 17 23 21 39 41

parent marked30 52 30 52

Decrease 35 to 5. Decrease 35 to 5.

35 36 Fibonacci Heaps: Decrease Key Analysis Fibonacci Heaps: Delete

Notation. Delete node x.

■ t(H) = # trees in heap H. ■ Decrease key of x to -∞.

■ m(H) = # marked nodes in heap H. ■ Delete min element in heap.

■ Φ(H) = t(H) + 2m(H). Amortized cost. O(D(n))

Actual cost. O(c) ■ O(1) for decrease-key.

■ O(1) time for decrease key. ■ O(D(n)) for delete-min.

■ O(1) time for each of c cascading cuts, plus reinserting in root list. ■ D(n) = max degree of any node in Fibonacci heap.

Amortized cost. O(1)

■ t(H') = t(H) + c

■ m(H') ≤ m(H) - c + 2 – each cascading cut unmarks a node – last cascading cut could potentially mark a node

■ ∆Φ ≤ c + 2(-c + 2) = 4 - c.

37 38

Fibonacci Heaps: Bounding Max Degree Fibonacci Heaps: Bounding Max Degree

Definition. D(N) = max degree in Fibonacci heap with N nodes. Key lemma. In a Fibonacci heap with N nodes, the maximum degree of φ √ Key lemma. D(N) ≤ logφ N, where φ = (1 + √5) / 2. any node is at most logφ N, where = (1 + 5) / 2. Corollary. Delete and Delete-min take O(log N) amortized time. Proof of key lemma. degree(x) ■ For any node x, we show that size(x) ≥φ . Lemma. Let x be a node with degree k, and let y1, . . . , yk denote the children of x in the order in which they were linked to x. Then: – size(x) = # node in subtree rooted at x

– taking base φ logs, degree(x) ≤ logφ (size(x)) ≤ logφ N.  0 if i = 1 degree ( y ) ≥  ■ Let sk be min size of tree rooted at any degree k node. i  i − 2 if i ≥ 1 – trivial to see that s0 = 1, s1 = 2 = sk size (x*) Proof. – sk monotonically increases with k k = 2 + ∑ size( y ) ■ Let x* be a degree k node of size s , i ■ When yi is linked to x, y1, . . . , yi-1 already linked to x, k i=2 ⇒ degree(x) = i - 1 and let y1, . . . , yk be children in order k ⇒ that they were linked to x*. ≥ 2 + ∑ s degree(yi) = i - 1 since we only link nodes of equal degree deg[ yi ] i=2 ■ Since then, yi has lost at most one child k ≥ + ∑ – otherwise it would have been cut from x ≥ 2 si−2 Assume k 2 i=2 ■ Thus, degree(y ) = i - 1 or i - 2 k−2 i = + 2 ∑ si i=0 39 40 Fibonacci Facts Golden Ratio

 1 if k = 0  = = Definition. The Fibonacci sequence is: Fk  2 if k 1 Definition. The Fibonacci sequence is: 1, 2, 3, 5, 8, 13, 21, . . .  + ≥ φ √ ■ 1, 2, 3, 5, 8, 13, 21, . . .  Fk-1 Fk-2 if k 2 Definition. The golden ratio = (1 + 5) / 2 = 1.618…

• Slightly nonstandard definition. ■ Divide a rectangle into a square and smaller rectangle such that the smaller rectangle has the same ratio as original one. ≥φk φ √ Fact F1. Fk , where = (1 + 5) / 2 = 1.618…

k−2 Fact F2. ≥ = + ∑ For k 2, Fk 2 Fi = i=0 sk size (x*) k = + 2 ∑ size( yi ) i=2 Consequence. s ≥ F ≥φk. k k k degree(x) ≥ + ∑ ■ ≥φ 2 s This implies that size(x) deg[ yi ] i=2 for all nodes x. k ≥ + 2 ∑ si−2 i=2 k−2 = + 2 ∑ si i=0 Parthenon, Athens Greece 41 42

Fibonacci Facts Fibonacci Numbers and Nature

Pinecone

Cauliflower

43 44 Fibonacci Proofs On Complicated Algorithms

"Once you succeed in writing the programs for [these] complicated ≥φk algorithms, they usually run extremely fast. The computer doesn't Fact F1. Fk . F = F + F k+2 k k+1 need to understand the algorithm, its task is only to run the Proof. (by induction on k) ϕ k k+1 ≥ ϕ+ programs." ■ Base cases: ϕ k = (1+ ) – F = 1, F = 2 ≥φ. ϕ 0 1 = ϕ k 2 ϕ( ) φ2 φ ■ Inductive hypotheses: = + 1 = ϕ k+2 ≥φk ≥φk+1 – Fk and Fk+1

k−2 ≥ = + Fact F2. For k 2, Fk 2 ∑ Fi = Proof. (by induction on k) i 0 F + = F + F + ■ Base cases: k 2 k k 1 k−2 = + + – F2 = 3, F3 = 5 2 ∑ Fi Fk+1 i=0 ■ Inductive hypotheses: k R. E. Tarjan k−2 = + ∑ = + 2 Fk Fk 2 ∑ Fi i=0 i=0

45 46