Fibonacci Heaps

Home , Fibonacci heap, Priority queue

FIBONACCI HEAPS

‣ preliminaries ‣ insert ‣ extract the minimum ‣ decrease key ‣ bounding the rank ‣ meld and delete

Lecture slides by Kevin Wayne http://www.cs.princeton.edu/~wayne/kleinberg-tardos

Last updated on 7/25/17 11:07 AM Priority queues performance cost summary

Fibonacci heap operation linked list binary heap binomial heap †

MAKE-HEAP O(1) O(1) O(1) O(1)

IS-EMPTY O(1) O(1) O(1) O(1)

INSERT O(1) O(log n) O(log n) O(1)

EXTRACT-MIN O(n) O(log n) O(log n) O(log n)

DECREASE-KEY O(1) O(log n) O(log n) O(1)

DELETE O(1) O(log n) O(log n) O(log n)

MELD O(1) O(n) O(log n) O(1)

FIND-MIN O(n) O(1) O(log n) O(1)

† amortized

Ahead. O(1) INSERT and DECREASE-KEY, O(log n) EXTRACT-MIN.

2 Fibonacci heaps

Theorem. [Fredman-Tarjan 1986] Starting from an empty Fibonacci heap, any sequence of m INSERT, EXTRACT-MIN, and DECREASE-KEY operations involving n INSERT operations takes O(m + n log n) time.

this statement is a bit weaker than the actual theorem

Fibonacci Heaps and Their Uses in Improved Network Optimization Algorithms

MICHAEL L. FREDMAN

University of California, San Diego, L.a Jolla, California AND

ROBERT ENDRE TARJAN

AT&T Bell Laboratories, Murray HilI, New Jersey

Abstract. In this paper we develop a new data structure for implementing heaps (priority queues). Our structure, Fibonacci heaps (abbreviated F-heaps), extends the binomial queues proposed by Vuillemin and studied further by Brown. F-heaps support arbitrary deletion from an n-item heap in qlogn) amortized time and all other standard heap operations in o( 1) amortized time. Using F-heaps we are able to obtain improved running times for several network optimization algorithms. In particular, we obtain the following worst-case bounds, where n is the number of vertices and m the number of edges in the problem graph: ( 1) O(n log n + m) for the single-source shortest path problem with nonnegative edge lengths, improved from O(mlogfmh+2)n); (2) O(n*log n + nm) for the all-pairs shortest path problem, improved from O(nm lo&,,,+2,n); (3) O(n*logn + nm) for the assignment problem (weighted bipartite matching), improved from O(nmlog0dn+2)n); (4) O(mj3(m, n)) for the minimum spanning tree problem, improved from O(mloglo&,,.+2,n), where j3(m, n) = min {i 1log% 5 m/n). Note that B(m, n) 5 log*n if m 2 n. Of these results, the improved bound for minimum spanning trees is the most striking, although all the results give asymptotic improvements for graphs of appropriate densities.

Categories and Subject Descriptors: E.l [Data]: Data Structures--trees; graphs; F.2.2 [Analysis of Algorithms and Problem Complexity]: Nonnumerical Algorithms and Problems-computations on discrete structures; sorting and searching; G.2.2 [Discrete Mathematics]: Graph Theory-graph algo- 3 rithms; network problems; trees General Terms: Algorithms, Theory Additional Key Words and Phrases: Heap, matching, minimum spanning tree, priority queue, shortest Pati

’ A preliminary version of this paper appeared in the Proceedings of the 25th IEEE Symposium on the Foundations of Computer Science (Singer Island, Fla., Oct. 24-26). IEEE, New York, 1984, pp. 338- 346, 0 IEEE. Any portion of this paper that appeared in the preliminary version is reprinted with permission. This research was supported in part by the National Science Foundation under grant MCS 82-0403 1. Authors’ addresses: M. L. Fredman, Electrical Engineering and Computer Science Department, Uni- versity of California, San Diego, La Jolla, CA 92093; R. E. Tarjan, AT&T Bell Laboratories, Murray Hill, NJ 07974. Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Association for Computing Machinery. To copy otherwise, or to republish, requires a fee and/or specific permission. 0 1987 ACM 0004-541 l/87/0700-0596 $1.50

Journal ofthe Association for Computing Machinery, Vol. 34, No. 3, July 1987, Pages 596-615. Fibonacci heaps

Theorem. [Fredman–Tarjan 1986] Starting from an empty Fibonacci heap, any sequence of m INSERT, EXTRACT-MIN, and DECREASE-KEY operations involving n INSERT operations takes O(m + n log n) time.

History. ・Ingenious data structure and application of amortized analysis. ・Original motivation: improve Dijkstra’s shortest path algorithm from O(m log n) to O(m + n log n). ・Also improved best-known bounds for all-pairs shortest paths, assignment problem, minimum spanning trees.

4 FIBONACCI HEAPS

‣ structure ‣ insert ‣ extract the minimum ‣ decrease key ‣ bounding the rank ‣ meld and delete

SECTION 19.1 Fibonacci heaps

Basic idea. ・Similar to binomial heaps, but less rigid structure. ・Binomial heap: eagerly consolidate trees after each INSERT; implement DECREASE-KEY by repeatedly exchanging node with its parent.

・Fibonacci heap: lazily defer consolidation until next EXTRACT-MIN; implement DECREASE-KEY by cutting off node and splicing into root list.

Remark. Height of Fibonacci heap is Θ(n) in worst case, but it doesn’t use sink or swim operations.

6 Fibonacci heap: structure

・Set of heap-ordered trees.

each child no smaller than its parent

heap-ordered tree

17 24 23 7 3 root

30 26 46 18 52 41 heap H 35 39 44 7 Fibonacci heap: structure

・Set of heap-ordered trees. ・Set of marked nodes.

used to keep trees bushy (stay tuned)

min

17 24 23 7 3

30 26 46 18 52 41 heap H marked 35 39 44 8 Fibonacci heap: structure

Heap representation. ・Store a pointer to the minimum node. ・Maintain tree roots in a circular, doubly-linked list.

min

17 24 23 7 3

30 26 46 18 52 41 heap H 35 39 44 9 Fibonacci heap: representation

Node representation. Each node stores: ・A pointer to its parent. ・A pointer to any of its children. ・A pointer to its left and right siblings. ・Its rank = number of children. ・Whether it is marked.

min

17 24 23 7 3 rank = 3

30 26 46 18 52 41

children are in a heap H 35 circular doubly-linked list 39 44 10 Fibonacci heap: representation

Operations we can do in constant time: ・Determine rank of a node. ・Find the minimum element. ・Merge two root lists together. ・Add or remove a node from the root list. ・Remove a subtree and merge into root list. ・Link the root of a one tree to root of another tree.

min

17 24 23 7 3 rank = 3

30 26 46 18 52 41 heap H 35 39 44 11 Fibonacci heap: notation

notation meaning

n number of nodes rank(x) number of children of node x rank(H) max rank of any node in heap H trees(H) number of trees in heap H marks(H) number of marked nodes in heap H

n = 14 rank(H) = 3 trees(H) = 5 marks(H) = 3 min

rank = 1 17 24 23 7 3 rank = 3

30 26 46 18 52 41 heap H 35 39 44 12 Fibonacci heap: potential function

Potential function.

Φ(H) = trees(H) + 2 ⋅ marks(H)

Φ(H) = 5 + 2⋅3 = 11 trees(H) = 5 marks(H) = 3 min

17 24 23 7 3

30 26 46 18 52 41 heap H 35 39 44 13 FIBONACCI HEAPS

‣ preliminaries ‣ insert ‣ extract the minimum ‣ decrease key ‣ bounding the rank ‣ meld and delete

SECTION 19.2 Fibonacci heap: insert

・Create a new singleton tree. ・Add to root list; update min pointer (if necessary).

insert 21

min

17 24 23 7 3

30 26 46 18 52 41 heap H 35 39 44 15 Fibonacci heap: insert

・Create a new singleton tree. ・Add to root list; update min pointer (if necessary).

insert 21

min

17 24 23 7 21 3

30 26 46 18 52 41 heap H 35 39 44 16 Fibonacci heap: insert analysis

Actual cost. ci = O(1).

one more tree; Change in potential. ∆Φ = Φ(Hi) – Φ(Hi–1) = +1. no change in marks

Amortized cost. ĉi = ci + ∆Φ = O(1).

Φ(H) = trees(H) + 2 ⋅ marks(H)

min

17 24 23 7 21 3

30 26 46 18 52 41 heap H 35 39 44 17 FIBONACCI HEAPS

‣ preliminaries ‣ insert ‣ extract the minimum ‣ decrease key ‣ bounding the rank ‣ meld and delete

SECTION 19.2 Linking operation

Useful primitive. Combine two trees T1 and T2 of rank k. ・Make larger root be a child of smaller root. ・Resulting tree T ʹ has rank k + 1.

tree T1 tree T2 tree T′

15 3 3

56 24 33 18 52 41 15 18 52 41

77 39 44 56 24 33 39 44

77 still heap-ordered

19 Fibonacci heap: extract the minimum

・Delete min; meld its children into root list; update min. ・Consolidate trees so that no two roots have same rank.

min

7 24 23 17 3

30 26 46 18 52 41

35 39 44

20 Fibonacci heap: extract the minimum

・Delete min; meld its children into root list; update min. ・Consolidate trees so that no two roots have same rank.

min 7 24 23 17 18 52 41

30 26 46 39 44

21 Fibonacci heap: extract the minimum

・Delete min; meld its children into root list; update min. ・Consolidate trees so that no two roots have same rank.

current min 7 24 23 17 18 52 41

30 26 46 39 44

22 Fibonacci heap: extract the minimum

・Delete min; meld its children into root list; update min. ・Consolidate trees so that no two roots have same rank.

rank

0 1 2 3

current min 7 24 23 17 18 52 41

30 26 46 39 44

23 Fibonacci heap: extract the minimum

・Delete min; meld its children into root list; update min. ・Consolidate trees so that no two roots have same rank.

rank

0 1 2 3

current min 7 24 23 17 18 52 41

30 26 46 39 44

24 Fibonacci heap: extract the minimum

・Delete min; meld its children into root list; update min. ・Consolidate trees so that no two roots have same rank.

rank

0 1 2 3

current min 7 24 23 17 18 52 41

30 26 46 39 44

25 Fibonacci heap: extract the minimum

・Delete min; meld its children into root list; update min. ・Consolidate trees so that no two roots have same rank.

rank link 23 to 17 0 1 2 3

current min 7 24 23 17 18 52 41

30 26 46 39 44

26 Fibonacci heap: extract the minimum

・Delete min; meld its children into root list; update min. ・Consolidate trees so that no two roots have same rank.

rank link 17 to 7 0 1 2 3

current min 7 24 17 18 52 41

30 26 46 23 39 44

27 Fibonacci heap: extract the minimum

・Delete min; meld its children into root list; update min. ・Consolidate trees so that no two roots have same rank.

rank link 24 to 7 0 1 2 3

current min 24 7 18 52 41

26 46 17 30 39 44

35 23

28 Fibonacci heap: extract the minimum

・Delete min; meld its children into root list; update min. ・Consolidate trees so that no two roots have same rank.

rank

0 1 2 3

current min 7 18 52 41

24 17 30 39 44

26 46 23

35 29 Fibonacci heap: extract the minimum

・Delete min; meld its children into root list; update min. ・Consolidate trees so that no two roots have same rank.

rank

0 1 2 3

current min 7 18 52 41

24 17 30 39 44

26 46 23

35 30 Fibonacci heap: extract the minimum

・Delete min; meld its children into root list; update min. ・Consolidate trees so that no two roots have same rank.

rank

0 1 2 3

current min 7 18 52 41

24 17 30 39 44

26 46 23

35 31 Fibonacci heap: extract the minimum

・Delete min; meld its children into root list; update min. ・Consolidate trees so that no two roots have same rank.

rank link 41 to 18 0 1 2 3

current min 7 18 52 41

24 17 30 39 44

26 46 23

35 32 Fibonacci heap: extract the minimum

・Delete min; meld its children into root list; update min. ・Consolidate trees so that no two roots have same rank.

rank

0 1 2 3

current min 7 52 18

24 17 30 41 39

26 46 23 44

35 33 Fibonacci heap: extract the minimum

・Delete min; meld its children into root list; update min. ・Consolidate trees so that no two roots have same rank.

rank

0 1 2 3

current min 7 52 18

24 17 30 41 39

26 46 23 44

35 34 Fibonacci heap: extract the minimum

・Delete min; meld its children into root list; update min. ・Consolidate trees so that no two roots have same rank.

stop (no two trees have same rank)

min 7 52 18

24 17 30 41 39

26 46 23 44

35 35 Fibonacci heap: extract the minimum analysis

Actual cost. ci = O(rank(H)) + O(trees(H)). ・O(rank(H)) to meld min’s children into root list. ≤ rank(H) children ・O(rank(H)) + O(trees(H)) to update min. ≤ rank(H) + trees(H) – 1 root nodes O(rank(H)) + O(trees(H)) to consolidate trees. number of roots decreases by 1 after ・ each linking operation

Change in potential. ∆Φ ≤ rank(Hʹ) + 1 – trees(H). ・No new nodes become marked. ・trees(Hʹ) ≤ rank(Hʹ) + 1. no two trees have same rank after consolidation

Amortized cost. O(log n). ・ĉi = ci + ∆Φ = O(rank(H)) + O(rank(Hʹ)). ・The rank of a Fibonacci heap with n elements is O(log n).

Fibonacci lemma (stay tuned) Φ(H) = trees(H) + 2 ⋅ marks(H)

36 Fibonacci heap vs. binomial heaps

Observation. If only INSERT and EXTRACT-MIN operations, then all trees are binomial trees.

we link only trees of equal rank

B0 B1 B2 B3

Binomial heap property. This implies rank(H) ≤ log2 n.

Fibonacci heap property. Our DECREASE-KEY implementation will not preserve this property, but we will implement it in such a way that rank(H) ≤ logφ n.

37 FIBONACCI HEAPS

‣ preliminaries ‣ insert ‣ extract the minimum ‣ decrease key ‣ bounding the rank ‣ meld and delete

SECTION 19.3 Fibonacci heap: decrease key

Intuition for deceasing the key of node x. ・If heap-order is not violated, decrease the key of x. ・Otherwise, cut tree rooted at x and meld into root list.

decrease-key of x from 30 to 7

8 29 10 44

30 23 22 48 31 17

45 32 24 50

55 39 Fibonacci heap: decrease key

Intuition for deceasing the key of node x. ・If heap-order is not violated, decrease the key of x. ・Otherwise, cut tree rooted at x and meld into root list.

decrease-key of x from 23 to 5

6 7

8 29 10 44 45 32

23 22 48 31 17 55

24 50

40 Fibonacci heap: decrease key

Intuition for deceasing the key of node x. ・If heap-order is not violated, decrease the key of x. ・Otherwise, cut tree rooted at x and meld into root list.

decrease-key of 22 to 4 decrease-key of 48 to 3 decrease-key of 31 to 2 6 7 5 decrease-key of 17 to 1

8 29 10 44 45 32 24

22 48 31 17 55

41 Fibonacci heap: decrease key

Intuition for deceasing the key of node x. ・If heap-order is not violated, decrease the key of x. ・Otherwise, cut tree rooted at x and meld into root list. ・Problem: number of nodes not exponential in rank.

6 7 5 4 3 2 1

8 29 10 44 45 32 24 50

rank = 4, nodes = 5 55

42 Fibonacci heap: decrease key

Intuition for deceasing the key of node x. ・If heap-order is not violated, decrease the key of x. ・Otherwise, cut tree rooted at x and meld into root list. ・Solution: as soon as a node has its second child cut, cut it off also and meld into root list (and unmark it).

min

7 18 38

marked node: 24 17 23 21 39 41 one child already cut

26 46 30 52

35 88 72 43 Fibonacci heap: decrease key

Case 1. [heap order not violated] ・Decrease key of x. ・Change heap min pointer (if necessary).

decrease-key of x from 46 to 29 min

7 18 38

24 17 23 21 39 41

26 46 30 52

35 88 72 44 Fibonacci heap: decrease key

Case 1. [heap order not violated] ・Decrease key of x. ・Change heap min pointer (if necessary).

decrease-key of x from 46 to 29 min

7 18 38

24 17 23 21 39 41

26 29 30 52

35 88 72 45 Fibonacci heap: decrease key

Case 2a. [heap order violated] ・Decrease key of x. ・Cut tree rooted at x, meld into root list, and unmark. ・If parent p of x is unmarked (hasn’t yet lost a child), mark it; Otherwise, cut p, meld into root list, and unmark (and do so recursively for all ancestors that lose a second child). decrease-key of x from 29 to 15 min

7 18 38

24 17 23 21 39 41 p

26 29 30 52

35 88 72 46 Fibonacci heap: decrease key

7 18 38

24 17 23 21 39 41 p

26 15 30 52

35 88 72 47 Fibonacci heap: decrease key

7 18 38

24 17 23 21 39 41 p

26 15 30 52

35 88 72 48 Fibonacci heap: decrease key

72 24 17 23 21 39 41 p

26 30 52

35 88 49 Fibonacci heap: decrease key

72 24 17 23 21 39 41 p

26 30 52

35 88 50 Fibonacci heap: decrease key

Case 2b. [heap order violated] ・Decrease key of x. ・Cut tree rooted at x, meld into root list, and unmark. ・If parent p of x is unmarked (hasn’t yet lost a child), mark it; Otherwise, cut p, meld into root list, and unmark (and do so recursively for all ancestors that lose a second child). decrease-key of x from 35 to 5 min

15 7 18 38

72 2424 17 23 21 39 41

p 26 30 52

x 35 88 51 Fibonacci heap: decrease key

15 7 18 38

72 2424 17 23 21 39 41

p 26 30 52

x 5 88 52 Fibonacci heap: decrease key

x min 15 5 7 18 38

72 2424 17 23 21 39 41

p 26 30 52

88 53 Fibonacci heap: decrease key

x min 15 5 7 18 38

72 2424 17 23 21 39 41 second child cut

p 26 30 52

88 54 Fibonacci heap: decrease key

x min p 15 5 26 7 18 38

72 88 2424 17 23 21 39 41

30 52

55 Fibonacci heap: decrease key

x min p 15 5 26 7 18 38

72 88 p′ 2424 17 23 21 39 41

second child cut 30 52

56 Fibonacci heap: decrease key

x min p p′ p″ 15 5 26 24 7 18 38

but don’t 72 88 mark parent 17 23 21 39 41 if it’s a root

30 52

57 Fibonacci heap: decrease key analysis

Actual cost. ci = O(c), where c is the number of cuts. ・O(1) time for changing the key. ・O(1) time for each of c cuts, plus melding into root list.

Change in potential. ∆Φ = O(1) – c. ・trees(Hʹ) = trees(H) + c. marks(H ) ≤ marks(H) – c + 2. each cut (except first) unmarks a node ・ ʹ last cut may or may not mark a node ・ΔΦ ≤ c + 2 ⋅ (-c + 2) = 4 – c.

Amortized cost. ĉi = ci + ∆Φ = O(1).

Φ(H) = trees(H) + 2 ⋅ marks(H)

58 FIBONACCI HEAPS

‣ preliminaries ‣ insert ‣ extract the minimum ‣ decrease key ‣ bounding the rank ‣ meld and delete

SECTION 19.4 Analysis summary

Insert. O(1). Delete-min. O(rank(H)) amortized. Decrease-key. O(1) amortized.

Fibonacci lemma. Let H be a Fibonacci heap with n elements. Then, rank(H) = O(log n).

number of nodes is exponential in rank

60 Bounding the rank

Lemma 1. Fix a point in time. Let x be a node of rank k, and let y1, …, yk denote its current children in the order in which they were linked to x. Then: x 0 B7 i =1 rank(yi) i 2 B7 i 2 y1 y2 … yk

Pf. When was linked into , had at least children . ・ yi x x i – 1 y1, …, yi–1 ・Since only trees of equal rank are linked, at that time

rank(yi) = rank(x) ≥ i – 1. Since then, has lost at most one child (or would have been cut). ・ yi yi Thus, right now . ▪ ・ rank(yi) ≥ i – 2

61 Bounding the rank

Def. Let Tk be smallest possible tree of rank k satisfying property.

T0 T1 T2 T3 T4 T5

F2 = 1 F3 = 2 F4 = 3 F5 = 5 F6 = 8 F7 = 13

62 Bounding the rank

Def. Let Tk be smallest possible tree of rank k satisfying property.

T4 T5 T6

F6 = 8 F7 = 13 F8 = F6 + F7 = 8 + 13 = 21

63 Bounding the rank

Lemma 2. Let sk be minimum number of elements in any Fibonacci heap of th rank k. Then sk ≥ Fk+2, where Fk is the k Fibonacci number.

Pf. [by strong induction on k] ・Base cases: s0 = 1 and s1 = 2. Inductive hypothesis: assume for . ・ si ≥ Fi+2 i = 0, …, k – 1 As in Lemma 1, let let denote its current children in the order in ・ y1, …, yk which they were linked to x.

sk ≥ 1 + 1 + (s0 + s1 + … + sk–2) (Lemma 1)

≥ (1 + F1) + F2 + F3 + … + Fk (inductive hypothesis)

= Fk+2. ▪ (Fibonacci fact 1)

64 Bounding the rank

Fibonacci lemma. Let H be a Fibonacci heap with n elements.

Then, rank(H) ≤ logφ n, where φ is the golden ratio = (1 + √5) / 2 ≈ 1.618.

Pf. ・Let H is a Fibonacci heap with n elements and rank k. k ・Then n ≥ Fk+2 ≥ φ .

Lemma 2 Fibonacci Fact 2

Taking logs, we obtain . ▪ ・ rank(H) = k ≤ logφ n

65 Fibonacci fact 1

Def. The Fibonacci sequence is: 0, 1, 1, 2, 3, 5, 8, 13, 21, …

0 B7 k =0 F = 1 k =1 k B7 Fk 1 + Fk 2 B7 k 2

Fibonacci fact 1. For all integers k ≥ 0, Fk+2 = 1 + F0 + F1 + … + Fk. Pf. [by induction on k] Base case: . ・ F2 = 1 + F0 = 2 Inductive hypothesis: assume . ・ Fk+1 = 1 + F0 + F1 + … + Fk–1

Fk+2 = Fk + Fk+1 (definition)

= Fk + (1 + F0 + F1 + … + Fk–1) (inductive hypothesis)

= 1 + F0 + F1 + … + Fk–1 + Fk. ▪ (algebra)

66 Fibonacci fact 2

Def. The Fibonacci sequence is: 0, 1, 1, 2, 3, 5, 8, 13, 21, …

0 B7 k =0 F = 1 k =1 k B7 Fk 1 + Fk 2 B7 k 2 k Fibonacci fact 2. Fk+2 ≥ φ , where φ = (1 + √5) / 2 ≈ 1.618. Pf. [by induction on k] Base cases: , . ・ F2 = 1 ≥ 1 F3 = 2 ≥ φ Inductive hypotheses: assume k and k + 1 ・ Fk ≥ φ Fk+1 ≥ φ

Fk+2 = Fk + Fk+1 (definition)

≥ φk – 1 + φk – 2 (inductive hypothesis)

= φk – 2 (1 + φ) (algebra)

= φk – 2 φ2 (φ2 = φ + 1) = φk. ▪ (algebra)

67 Fibonacci numbers and nature

Fibonacci numbers arise both in nature and algorithms.

pinecone

cauliﬂower

68 FIBONACCI HEAPS

‣ preliminaries ‣ insert ‣ extract the minimum ‣ decrease key ‣ bounding the rank ‣ meld and delete

SECTION 19.2, 19.3 Fibonacci heap: meld

Meld. Combine two Fibonacci heaps (destroying old heaps).

Recall. Root lists are circular, doubly-linked lists.

min min

23 24 17 7 3 21

30 26 46 18 52 41

heap H1 heap H2 35 39 44 70 Fibonacci heap: meld

Meld. Combine two Fibonacci heaps (destroying old heaps).

Recall. Root lists are circular, doubly-linked lists.

min

23 24 17 7 3 21

30 26 46 18 52 41

35 heap H 39 44 71 Fibonacci heap: meld analysis

Actual cost. ci = O(1). Change in potential. ∆Φ = 0.

Amortized cost. ĉi = ci + ∆Φ = O(1).

Φ(H) = trees(H) + 2 ⋅ marks(H)

min

23 24 17 7 3 21

30 26 46 18 52 41

35 heap H 39 44 72 Fibonacci heap: delete

Delete. Given a handle to an element x, delete it from heap H. ・DECREASE-KEY(H, x, -∞). ・EXTRACT-MIN(H).

Amortized cost. ĉi = O(rank(H)). ・O(1) amortized for DECREASE-KEY. ・O(rank(H)) amortized for EXTRACT-MIN.

Φ(H) = trees(H) + 2 ⋅ marks(H)

73 Priority queues performance cost summary

Fibonacci heap operation linked list binary heap binomial heap †

MAKE-HEAP O(1) O(1) O(1) O(1)

IS-EMPTY O(1) O(1) O(1) O(1)

INSERT O(1) O(log n) O(log n) O(1)

EXTRACT-MIN O(n) O(log n) O(log n) O(log n)

DECREASE-KEY O(1) O(log n) O(log n) O(1)

DELETE O(1) O(log n) O(log n) O(log n)

MELD O(1) O(n) O(log n) O(1)

FIND-MIN O(n) O(1) O(log n) O(1)

† amortized

Accomplished. O(1) INSERT and DECREASE-KEY, O(log n) EXTRACT-MIN.

74 PRIORITY QUEUES

‣ binary heaps ‣ d-ary heaps ‣ binomial heaps ‣ Fibonacci heaps ‣ advanced topics Heaps of heaps

・b-heaps. ・Fat heaps. ・2–3 heaps. ・Leaf heaps. ・Thin heaps. ・Skew heaps. ・Splay heaps. ・Weak heaps. ・Leftist heaps. ・Quake heaps. ・Pairing heaps. ・Violation heaps. ・Run-relaxed heaps. ・Rank-pairing heaps. ・Skew-pairing heaps. ・Rank-relaxed heaps. Lazy Fibonacci heaps. ・ 76 Brodal queues

Q. Can we achieve same running time as for Fibonacci heap but with worst-case bounds per operation (instead of amortized)?

Theory. [Brodal 1996] Yes.

Practice. Ever implemented? Constants are high (and requires RAM model).

77 Strict Fibonacci heaps

Q. Can we achieve same running time as for Fibonacci heap but with worst-case bounds per operation (instead of amortized) in pointer model?

Theory. [Brodal–Lagogiannis–Tarjan 2012] Yes.

Strict Fibonacci Heaps Strict Fibonacci Heaps Gerth Stølting Brodal George Lagogiannis Robert E. Tarjan† MADALGO∗ Agricultural University Dept. of Computer Science Dept. of Computer Science of Athens Princeton University Aarhus University GerthIera Odos Stølting 75, 11855 Brodal Athens Georgeand Lagogiannis HP Labs Robert E. Tarjan† MADALGO∗ Agricultural University Dept. of Computer Science Åbogade 34, 8200 Aarhus N Greece 35 Olden Street, Princeton Dept. of Computer Science of Athens Princeton University Denmark [email protected] New Jersey 08540, USA [email protected] Aarhus University [email protected] Odos 75, 11855 Athens and HP Labs Åbogade 34, 8200 Aarhus N Greece 35 Olden Street, Princeton Denmark [email protected] New Jersey 08540, USA ABSTRACT [email protected]. INTRODUCTION [email protected] We present the first pointer-based heap implementation with Williams in 1964 introduced binary heaps [25]. Since then time bounds matching those of FibonacciABSTRACT heaps in the worst the design and analysis of heaps has1. been INTRODUCTION thoroughly inves- case. We support make-heap, insert, find-min, meld and tigated. The most common operations supported by the Williams in 1964 introduced binary heaps [25]. Since then decrease-key in worst-case O(1) time,We present and delete the andfirst delete- pointer-basedheaps heap in implementation the literature with are those listed below. We assume the design and analysis of heaps has been thoroughly inves- min in worst-case O(lg n)time,wheretime boundsn is the matching size of thethose of Fibonaccithat each heaps item storedin the worst contains an associated key. No item tigated. The most common operations supported by the heap. The data structure uses linearcase. space. We support make-heap,can insert, be in find-min, more than meld one and heap at a time. heaps in the literature are those listed below. We assume Aprevious,verycomplicated,solutionachievingthesamedecrease-key in worst-case O(1) time, and delete and delete- that each item stored contains an associated key. No item time bounds in the RAM model mademin essential in worst-case use ofO arrays(lg n)time,wheremakeheap()n is theCreate size of a new, the empty heap and return a ref- can be in more than one heap at a time. and extensive use of redundant counterheap. schemes The data to structure maintain uses linear space.erence to it. Aprevious,verycomplicated,solutionachievingthesame balance. Our solution uses neither. Our key simplification insert(H, i) Insert item i,notcurrentlyinaheap,intomakeheap() Create a new, empty heap and return a ref- is to discard the structure of thetime smaller bounds heap in when the RAM doing model made essential use of arrays and extensive use of redundant counterheap schemesH,andreturnareferencetowhere to maintain erence to it.i is stored ameld.Weusethepigeonholeprincipleinplaceofthe in H. redundant counter mechanism. balance. Our solution uses neither. Our key simplification insert(H, i) Insert item i,notcurrentlyinaheap,into is to discard the structure of the smaller heap when doing meld(H1,H2) Return a reference to aheap new heapH,andreturnareferencetowhere containing i is stored ameld.Weusethepigeonholeprincipleinplaceofthe Categories and Subject Descriptors all items in the two heaps H1 andin HH. 2 (H1 and H2 redundant counter mechanism. cannot be accessed after meld). E.1 [Data Structures]: Trees; F.2.2 [Theory of Compu- meld(H1,H2) Return a reference to a new heap containing tation]: Analysis of Algorithms andCategories Problem Complexity— and Subject Descriptorsfind-min(H) Return a reference to whereall items the in item the with two heaps H1 and H2 (H1 and H2 Nonnumerical Algorithms and Problems minimum key is stored in the heapcannotH. be accessed after meld). E.1 [Data Structures]: Trees; F.2.2 [Theory of Compu- delete-min(H) Delete the itemfind-min( with minimumH) Return key from a reference to where the item with tation]: Analysis of Algorithms and Problem Complexity— 78 General Terms Nonnumerical Algorithms and Problemsthe heap H. minimum key is stored in the heap H. Algorithms delete(H,e) Delete an item fromdelete-min( the heap HHgiven) Delete a ref- the item with minimum key from General Terms erence e to where it is stored. the heap H. Keywords Algorithms decrease-key(H,e,k) Decrease the key of the item given Data structures, heaps, meld, decrease-key, worst-case com- delete(H,e) Delete an item from the heap H given a ref- by the reference e in heap H to theerence newe keyto wherek. it is stored. plexity Keywords decrease-key(H,e,k) Decrease the key of the item given Data structures, heaps, meld, decrease-key,There are worst-case many heap com- implementations in the literature, by the reference e in heap H to the new key k. plexity with a variety of characteristics. We can divide them into two main categories, depending on whether the time bounds ∗Center for Massive Data Algorithmics, a Center of the Dan- There are many heap implementations in the literature, ish National Research Foundation. are worst case or amortized. Most of the heaps in the lit- with a variety of characteristics. We can divide them into †Partially supported by NSF grant CCF-0830676, US-Israel erature are based on heap-ordered trees, i.e. tree structures Binational Science Foundation Grant∗Center 2006204, for Massive and Data the Algorithmics,where the a Center item stored of the Dan-in a node hastwo a main key not categories, smaller depending than on whether the time bounds Distinguished Visitor Program ofish the National Stanford Research University Foundation.the key of the item stored in its parent.are worst Heap-ordered case or amortized. trees Most of the heaps in the lit- Computer Science Department. The†Partially information supported contained by NSF grantgive CCF-0830676, heap implementations US-Israel that achieveerature logarithmic are based ontime heap-ordered for trees, i.e. tree structures herein does not necessarily reflectBinational the opinion Science or policy Foundation of all Grant the operations. 2006204, and Early the exampleswhere are the the item implicit stored binary in a node has a key not smaller than the federal government and no offiDistinguishedcial endorsement Visitor should Program of the Stanford University the key of the item stored in its parent. Heap-ordered trees be inferred. Computer Science Department.heaps The information of Williams contained[25], the leftist heaps of Crane [5] as modi- fied by Knuth [20], and the binomialgive heaps heap of implementations Vuillemin [24]. that achieve logarithmic time for herein does not necessarily reflect the opinion or policy of all the operations. Early examples are the implicit binary the federal government and no offiThecial introduction endorsement of should Fibonacci heaps [15] by Fredman and be inferred. Tarjan was a breakthrough sinceheaps they achieved of WilliamsO(1) [25], amor- the leftist heaps of Crane [5] as modi- fied by Knuth [20], and the binomial heaps of Vuillemin [24]. Permission to make digital or hard copies of all or part of thisworkfor tized time for all the operations above except for delete and personal or classroom use is granted without fee provided that copies are delete-min, which require O(lg n) amortizedThe introduction time, where of Fibonaccin heaps [15] by Fredman and not made or distributed for profit or commercial advantage andthatcopies is the number of items in the heapTarjan and lg was the a base-two breakthrough log- since they achieved O(1) amor- bear this notice and the full citation on thePermission first page. To to cop makeyotherwise,to digital or hard copiesarithm. of all or The part drawback of thisworkfor of Fibonaccitized heaps time is for that all the they operations are above except for delete and republish, to post on servers or to redistributepersonal to lists, or re classroomquires prior use specific is granted withoutcomplicated fee provided compared that copies to are existingdelete-min, solutions and which not require as ef- O(lg n) amortized time, where n permission and/or a fee. not made or distributed for profit or commercialficient inadvantage practice andthatcopies as other, theoreticallyis the number less effi ofcient items sol inu- the heap and lg the base-two log- STOC.12, May 19–22, 2012, New York, Newbear York, this notice USA. and the full citation on the first page. To copyotherwise,to arithm. The drawback of Fibonacci heaps is that they are Copyright 2012 ACM 978-1-4503-1245-5/12/05 ...$10.00. tions. Thus, Fibonacci heaps opened the way for further republish, to post on servers or to redistribute to lists, requires prior specific complicated compared to existing solutions and not as ef- permission and/or a fee. STOC.12, May 19–22, 2012, New York, New York, USA. ficient in practice as other, theoretically less efficient solu- Copyright 2012 ACM 978-1-4503-1245-5/12/05 ...$10.00. tions. Thus, Fibonacci heaps opened the way for further Fibonacci heaps: practice

Q. Are Fibonacci heaps useful in practice? A. They are part of LEDA and Boost C++ libraries. (but other heaps seem to perform better in practice)

79 Pairing heaps

Pairing heap. A self-adjusting heap-ordered general tree.

Theory. Same amortized running times as Fibonacci heaps for all operations except DECREASE-KEY. ・O(log n) amortized. [Fredman et al. 1986] ・Ω(log log n) lower bound on amortized cost. [Fredman 1999] ・ 2 O (log log n ) amortized. [Pettie 2005]

80 Pairing heaps

Pairing heap. A self-adjusting heap-ordered general tree.

Practice. As fast as (or faster than) the binary heap on some problems. Included in GNU C++ library and LEDA.

RESEARCHCONlRlWlIONS

Algorithms and Data Structures Pairing Heaps: G. Scott Graham Editor Experiments and Analysis

JOHN T. STASKO and JEFFREY SCOTT VlllER

ABSTRACT: The pairing heap has recently been and practical importance from their use in solving a introduced as a new data structure for priority queues. wide range of combinatorial problems, including job Pairing heaps are extremely simple to implement and scheduling, minimal spanning tree, shortest path, seem to be very efficient in practice, but they are difficult and graph traversal. to analyze theoretically, and open problems remain. It Priority queues support the operations insert, has been conjectured that they achieve the same find-min, and delete-min; additional operations often amortized time bounds as Fibonacci heaps, namely, include decrease-key and delete. The insert(t, v) opera- O(log n) time for delete and delete-min and O(1) for tion adds item t with key value v to the priority all other operations, where n is the size of the priority queue. The find-min operation returns the item queue at the time of the operation. We provide empirical with minimum key value. The delete-min operation evidence that supports this conjecture. The most returns the item with minimum key value and promising algorithm in our simulations is a new variant removes it from the priority queue. The decrease- of the twopass method, called auxiliary twopass. We key(t, d) operation reduces item t’s key value by d. prove that, assuming no decrease-key operations are The delete(t) operation removes item t from the performed, it achieves the same amortized time bounds as priority queue. The decrease-key and delete opera- Fibonacci heaps. tions require that a pointer to the location in the priority queue of item t be supplied explicitly, since 1. INTRODUCTION priority queues do not support searching for arbi- A priority queue is an abstract data type for main- trary items by value. Some priority queues also sup- taining and manipulating a set of items based on port the merge operation, which combines two item- priority [I]. Prio’rity queues derive great theoretical disjoint priority queues. 81 We will concentrate on the insert, delete-min, and Support was provided in part by NSF research grant DCR-84-03613, an NSF Presidential Young Investigator Award, an IBM Faculty Development Award, decrease-key operations because they are the opera- and a Guggenheim Fellowship. tions that primarily distinguish priority queues from Part of this research was performed at Mathematical Sciences Research Insti- tute. Berkeley, Calif., and the Institut National de Recherche en Informatique other set manipulation algorithms and because they et en Automatique, Rocquencourt. France. are the critical operations as far as the time bounds 0 1987 ACM OOOl-0782/87/0300-0234 75a: are concerned.

Communications of the ACM March 1987 Volume 30 Number 3 Priority queues performance cost summary

binomial pairing Fibonacci Brodal operation linked list binary heap heap heap † heap † queue

MAKE-HEAP O(1) O(1) O(1) O(1) O(1) O(1)

IS-EMPTY O(1) O(1) O(1) O(1) O(1) O(1)

INSERT O(1) O(log n) O(log n) O(1) O(1) O(1)

EXTRACT-MIN O(n) O(log n) O(log n) O(log n) O(log n) O(log n)

DECREASE-KEY O(1) O(log n) O(log n) 2O(log log n) O(1) O(1)

DELETE O(1) O(log n) O(log n) O(log n) O(log n) O(log n)

MELD O(1) O(n) O(log n) O(1) O(1) O(1)

FIND-MIN O(n) O(1) O(log n) O(1) O(1) O(1)

† amortized

82 Priority queues with integer priorities

Assumption. Keys are integers between 0 and C.

Theorem. [Thorup 2004] There exists a priority queue that supports INSERT, FIND-MIN, and DECREASE-KEY in constant time and EXTRACT-MIN and DELETE-KEY in either O(log log n) or O(log log C) time.

ARTICLE IN PRESS

Journal of Computer and System Sciences 69 (2004) 330–353 http://www.elsevier.com/locate/jcss

Integer priority queues with decrease key in constant time and the single source shortest paths problem Mikkel Thorup AT&T Labs Research, Shannon Laboratory, 180 Park Avenue, Florham Park, NJ 07932, USA Received 22 July 2003; revised 8 March 2004 Available online 20 July 2004

Abstract

We consider Fibonacci heap style integer priority queues supporting ﬁnd-min, insert, and decrease key operations in constant time. We present a deterministic linear space solution that with n integer keys supports delete in O log log n time. If the integers are in the range 0; N ; we can also support delete in O log log N time. ð Þ ½ Þ ðEven for theÞ special case of monotone priority queues, where the minimum has to be non-decreasing, the 1= 3 e 1= 4 e best previous bounds on delete were O log n ð À Þ and O log N ð À Þ : These previous bounds used both randomization and amortization. Ourðð newÞ boundsÞ are deterministic,ðð Þ worst-case,Þ with no restriction to monotonicity, and exponentially faster. 83 As a classical application, for a directed graph with n nodes and m edges with non-negative integer weights, we get single source shortest paths in O m n log log n time, or O m n log log C if C is the maximal edge weight. The latter solves an open problemð þ of Ahuja,Þ Mehlhorn, Orlin,ð þ and TarjanÞ from 1990. r 2004 Elsevier Inc. All rights reserved.

Keywords: Integer priority queues; Decrease key; Fibonacci heaps; Single source shortest paths

1. Introduction

In 1984, Fredman and Tarjan introduced Fibonacci heaps [15], which is a priority queue over a dynamic ordered set H supporting the following operations: find-min H Returns an element from H with minimum key value in constant time. insert Hð; aÞ Adds the element a to H in constant time. dec-keyð HÞ; a; x Reduces the key value of element a to x in constant time. If the current key value of aðwas smallerÞ than x; it is an error.

E-mail address: [email protected].

Assumption. Keys are integers between 0 and C.

Theorem. [Thorup 2004] There exists a priority queue that supports INSERT, FIND-MIN, and DECREASE-KEY in constant time and EXTRACT-MIN and DELETE-KEY in either O(log log n) or O(log log C) time.

Corollary 1. Can implement Dijkstra’s algorithm in either O(m log log n) or O(m log log C) time.

Corollary 2. Can sort n integers in O(n log log n) time.

Computational model. Word RAM.

84 Soft heaps

Goal. Break information-theoretic lower bound by allowing priority queue to corrupt 10% of the keys (by increasing them).

elements inserted 0 11 22 44 55

88 11 22 99 44 33 77 66 0 55 57 66 77 88 99

soft heap corrupted

85 Soft heaps

Goal. Break information-theoretic lower bound by allowing priority queue to corrupt 10% of the keys (by increasing them).

Representation. ・Set of binomial trees (with some subtrees missing). ・Each node may store several elements. ・Each node stores a value that is an upper bound on the original keys. ・Binomial trees are heap-ordered with respect to these values.

86 Soft heaps

Goal. Break information-theoretic lower bound by allowing priority queue to corrupt 10% of the keys (by increasing them).

Theorem. [Chazelle 2000] Starting from an empty soft heap, any sequence of n INSERT, MIN, EXTRACT-MIN, MELD, and DELETE operations takes O(n) time and at most 10% of its elements are corrupted at any given time.

The Soft Heap: An Approximate Priority Queue with Optimal Error Rate

BERNARD CHAZELLE

Princeton University, Princeton, New Jersey, and NEC Research Institute

Abstract. A simple variant of a priority queue, called a soft heap,isintroduced.Thedatastructure supports the usual operations: insert, delete, meld, and findmin. Its novelty is to beat the logarithmic bound on the complexity of a heap in a comparison-based model. To break this information-theoretic barrier, the entropy of the data structure is reduced by artificially raising the values of certain keys. Given any mixed sequence of n operations, a soft heap with error rate ␧ (for any 0 Ͻ ␧Յ1/2) ensures that, at any time, at most ␧n of its items have their keys raised. The amortized complexity of each operation is constant, except for insert, which takes O(log 1/␧)time.Thesoftheapisoptimalforany value of ␧ in a comparison-based model. The data structure is purely pointer-based. No arrays are used and no numeric assumptions are made on the keys. The main idea behind the soft heap is to move items across the data structure not individually, as is customary, but in groups, in a data-structuring equivalent of “car pooling.” Keys must be raised as a result, in order to preserve the heap ordering of the data structure. The soft heap can be used to compute exact or approximate medians and percentiles optimally. It is also useful for approximate sorting and for computing minimum spanning trees of general graphs. Categories and Subject Descriptors: E.1 [Data Structures]: Nonnumerical Algorithms and Problems 87 General Terms: Theory Additional Key Words and Phrases: Amortization, heap, priority queue, soft heap

1. Introduction We design a simple variant of a priority queue, called a soft heap.Thedata structure stores items with keys from a totally ordered universe, and supports the operations:

ApreliminaryversionofthispaperasCHAZELLE,B.1998.Car-poolingasadatastructuringdevice: The soft heap. In Proceedings of the 6th Annual European Symposium on Algorithms,pp.35–42. This work was supported in part by National Science Foundation (NSF) Grants CCR 93-01254 and CCR 96-23768, ARO Grant DAAH04-96-1-0181, and NEC Research Institute. Author’s address: Department of Computer Science, Princeton University, 35 Olden Street, Prince- ton, NJ 08544-2087, e-mail: [email protected] or NEC Research Institute, e-mail: chazelle@ research.nj.nec.com. Permission to make digital/hard copy of part or all of this work for personal or classroom use is granted without fee provided that the copies are not made or distributed for profit or commercial advantage, the copyright notice, the title of the publication, and its date appear, and notice is given that copying is by permission of the Association for Computing Machinery (ACM), Inc. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. ©2000ACM0004-5411/00/1100-1012$05.00

Journal of the ACM, Vol. 47, No. 6, November 2000, pp. 1012–1027. Soft heaps

Goal. Break information-theoretic lower bound by allowing priority queue to corrupt 10% of the keys (by increasing them).

Q. Brilliant. But how could it possibly be useful? Ex. Linear-time deterministic selection. To find kth smallest element: ・Insert the n elements into soft heap. ・Extract the minimum element n / 2 times. ・The largest element deleted ≥ 4n / 10 elements and ≤ 6n / 10 elements. ・Can remove ≥ 4n / 10 of elements and recur. ・T(n) ≤ T(3n / 5) + O(n) ⇒ T(n) = O(n). ▪

88 Soft heaps

Theorem. [Chazelle 2000] There exists an O(m α(m, n)) time deterministic algorithm to compute an MST in a graph with n nodes and m edges.

Algorithm. Borůvka + nongreedy + divide-and-conquer + soft heap + …

AMinimumSpanningTreeAlgorithmwithInverse- Ackermann Type Complexity

BERNARD CHAZELLE

Princeton University, Princeton, New Jersey, and NEC Research Institute

Abstract. A deterministic algorithm for computing a minimum spanning tree of a connected graph is presented. Its running time is O(m␣(m, n)), where ␣ is the classical functional inverse of Ackermann’s function and n (respectively, m)isthenumberofvertices(respectively,edges).The algorithm is comparison-based: it uses pointers, not arrays, and it makes no numeric assumptions on the edge costs. Categories and Subject Descriptors: F.2.2 [Analysis of Algorithms and Problem Complexity]: Nonnumerical Algorithms and Problems. General Terms: Theory

Additional Key Words and Phrases: Graphs, matroids, minimum spanning trees 89

1. Introduction

The history of the minimum spanning tree (MST)problemislongandrich,going as far back as Boru˚vka’s work in 1926 [Boru˚vka 1926; Graham and Hell 1985; Nesˇetrˇil 1997]. In fact, MST is perhaps the oldest open problem in computer science. According to Nesˇetrˇil [1997], “this is a cornerstone problem of combinatorial optimization and in a sense its cradle.” Textbook algorithms run in O(m log n)time,wheren and m denote, respectively, the number of vertices and edges in the graph. Improvements to O(m log log n)weregivenindependently by Yao [1975] and by Cheriton and Tarjan [1976]. In the mid-eighties, Fredman and Tarjan [1987] lowered the complexity to O(m␤(m, n)), where ␤(m, n)is the number of log-iterations necessary to map n to a number less than m/n.In the worst case, m ϭ O(n)andtherunningtimeisO(m log* m). Soon after, the

ApreliminaryversionofthispaperappearedasCHAZELLE,B.1997.Afasterdeterministicalgorithm for minimum spanning trees. In Proceedings of the 38th Annual IEEE Symposium on Foundations of Computer Science.IEEEComputerSocietyPress,LosAlamitos,Calif.,pp.22–31. This work was supported in part by the National Science Foundation (NSF) Grants CCR 93-01254 and CCR 96-23768, ARO Grant DAAH04-96-1-0181, and NEC Research Institute. Author’s address: Department of Computer Science, Princeton University, 35 Olden Street, Prince- ton, NJ 083-44-2087, e-mail: [email protected] and NEC Research Institute, e-mail: [email protected]. Permission to make digital/hard copy of part or all of this work for personal or classroom use is granted without fee provided that the copies are not made or distributed for profit or commercial advantage, the copyright notice, the title of the publication, and its date appear, and notice is given that copying is by permission of the Association for Computing Machinery (ACM), Inc. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. ©2000ACM0004-5411/00/1100-1028$05.00

Journal of the ACM, Vol. 47, No. 6, November 2000, pp. 1028–1047.