FIBONACCI HEAPS ‣ Preliminaries ‣ Insert ‣ Extract the Minimum

Priority queues performance cost summary

FIBONACCI HEAPS operation linked list binary heap binomial heap Fibonacci heap †

MAKE-HEAP O(1) O(1) O(1) O(1) ‣ preliminaries IS-EMPTY O(1) O(1) O(1) O(1) ‣ insert INSERT O(1) O(log n) O(log n) O(1) ‣ extract the minimum EXTRACT-MIN O(n) O(log n) O(log n) O(log n) ‣ decrease key DECREASE-KEY O(1) O(log n) O(log n) O(1) ‣ bounding the rank ‣ meld and delete DELETE O(1) O(log n) O(log n) O(log n) MELD O(1) O(n) O(log n) O(1)

Lecture slides by Kevin Wayne FIND-MIN O(n) O(1) O(log n) O(1) http://www.cs.princeton.edu/~wayne/kleinberg-tardos † amortized

Ahead. O(1) INSERT and DECREASE-KEY, O(log n) EXTRACT-MIN.

Last updated on Apr 15, 2013 6:59 AM 2

Fibonacci heaps Fibonacci heaps

Theorem. [Fredman-Tarjan 1986] Starting from an empty Fibonacci heap, Theorem. [Fredman-Tarjan 1986] Starting from an empty Fibonacci heap, any sequence of m INSERT, EXTRACT-MIN, and DECREASE-KEY operations any sequence of m INSERT, EXTRACT-MIN, and DECREASE-KEY operations

involving n INSERT operations takes O(m + n log n) time. involving n INSERT operations takes O(m + n log n) time.

this statement is a bit weaker than the actual theorem History. Fibonacci Heaps and Their Uses in Improved Network ・Ingenious data structure and application of amortized analysis. Optimization Algorithms ・Original motivation: improve Dijkstra's shortest path algorithm MICHAEL L. FREDMAN

University of California, San Diego, L.a Jolla, California from O(m log n) to O(m + n log n). AND Also improved best-known bounds for all-pairs shortest paths, ROBERT ENDRE TARJAN ・ AT&T Bell Laboratories, Murray HilI, New Jersey assignment problem, minimum spanning trees.

Abstract. In this paper we develop a new data structure for implementing heaps (priority queues). Our structure, Fibonacci heaps (abbreviated F-heaps), extends the binomial queues proposed by Vuillemin and studied further by Brown. F-heaps support arbitrary deletion from an n-item heap in qlogn) amortized time and all other standard heap operations in o( 1) amortized time. Using F-heaps we are able to obtain improved running times for several network optimization algorithms. In particular, we obtain the following worst-case bounds, where n is the number of vertices and m the number of edges in the problem graph: ( 1) O(n log n + m) for the single-source shortest path problem with nonnegative edge lengths, improved from O(mlogfmh+2)n); (2) O(n*log n + nm) for the all-pairs shortest path problem, improved from O(nm lo&,,,+2,n); (3) O(n*logn + nm) for the assignment problem (weighted bipartite matching), improved from O(nmlog0dn+2)n); (4) O(mj3(m, n)) for the minimum spanning tree problem, improved from O(mloglo&,,.+2,n), where j3(m, n) = min {i 1log% 5 m/n). Note that B(m, n) 5 log*n if m 2 n. Of these results, the improved bound for minimum spanning trees is the most striking, although all the results give asymptotic improvements for graphs of appropriate densities.

Categories and Subject Descriptors: E.l [Data]: Data Structures--trees; graphs; F.2.2 [Analysis of Algorithms and Problem Complexity]: Nonnumerical Algorithms and Problems-computations on discrete structures; sorting and searching; G.2.2 [Discrete Mathematics]: Graph Theory-graph algo- 3 4 rithms; network problems; trees General Terms: Algorithms, Theory Additional Key Words and Phrases: Heap, matching, minimum spanning tree, priority queue, shortest Pati

’ A preliminary version of this paper appeared in the Proceedings of the 25th IEEE Symposium on the Foundations of Computer Science (Singer Island, Fla., Oct. 24-26). IEEE, New York, 1984, pp. 338- 346, 0 IEEE. Any portion of this paper that appeared in the preliminary version is reprinted with permission. This research was supported in part by the National Science Foundation under grant MCS 82-0403 1. Authors’ addresses: M. L. Fredman, Electrical Engineering and Computer Science Department, Uni- versity of California, San Diego, La Jolla, CA 92093; R. E. Tarjan, AT&T Bell Laboratories, Murray Hill, NJ 07974. Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Association for Computing Machinery. To copy otherwise, or to republish, requires a fee and/or specific permission. 0 1987 ACM 0004-541 l/87/0700-0596 $1.50

Journal ofthe Association for Computing Machinery, Vol. 34, No. 3, July 1987, Pages 596-615. Fibonacci heaps

Basic idea. FIBONACCI HEAPS ・Similar to binomial heaps, but less rigid structure. ・Binomial heap: eagerly consolidate trees after each INSERT; ‣ structure implement DECREASE-KEY by repeatedly exchanging node with its parent. ‣ insert ‣ extract the minimum ‣ decrease key ‣ bounding the rank ‣ meld and delete

・Fibonacci heap: lazily defer consolidation until next EXTRACT-MIN; SECTION 19.1 implement DECREASE-KEY by cutting off node and splicing into root list.

Remark. Height of Fibonacci heap is Θ(n) in worst case, but it doesn't use sink or swim operations.

Fibonacci heap: structure Fibonacci heap: structure

・Set of heap-ordered trees. ・Set of heap-ordered trees. ・Set of marked nodes.

each child no smaller than its parent used to keep trees bushy (stay tuned)

heap-ordered tree min

17 24 23 7 3 root 17 24 23 7 3

30 26 46 30 26 46 18 52 41 18 52 41 heap H heap H marked 35 35 39 44 39 44 7 8 Fibonacci heap: structure Fibonacci heap: representation

Heap representation. Node representation. Each node stores: ・Store a pointer to the minimum node. ・A pointer to its parent. ・Maintain tree roots in a circular, doubly-linked list. ・A pointer to any of its children. ・A pointer to its left and right siblings. ・Its rank = number of children. ・Whether it is marked.

min min

17 24 23 7 3 17 24 23 7 3 rank = 3

30 26 46 30 26 46 18 52 41 18 52 41

children are in a heap H heap H 35 35 circular doubly-linked list 39 44 39 44 9 10

Fibonacci heap: representation Fibonacci heap: notation

Operations we can do in constant time: notation meaning ・Find the minimum element. n number of nodes ・Merge two root lists together. ・Determine rank of a root node. rank(x) number of children of node x ・Add or remove a node from the root list. rank(H) max rank of any node in heap H ・Remove a subtree and merge into root list. trees(H) number of trees in heap H ・Link the root of a one tree to root of another tree. marks(H) number of marked nodes in heap H

n = 14 rank(H) = 3 trees(H) = 5 marks(H) = 3 min min

17 24 23 7 3 rank = 3 rank = 1 17 24 23 7 3 rank = 3

30 26 46 30 26 46 18 52 41 18 52 41 heap H heap H 35 35 39 44 39 44 11 12 Fibonacci heap: potential function

Potential function. FIBONACCI HEAPS

Φ(H) = trees(H) + 2 ⋅ marks(H) ‣ preliminaries ‣ insert ‣ extract the minimum ‣ decrease key ‣ bounding the rank Φ(H) = 5 + 2⋅3 = 11 trees(H) = 5 marks(H) = 3 min ‣ meld and delete

17 24 23 7 3 SECTION 19.2

30 26 46 18 52 41 heap H 35 39 44 13

Fibonacci heap: insert Fibonacci heap: insert

・Create a new singleton tree. ・Create a new singleton tree. ・Add to root list; update min pointer (if necessary). ・Add to root list; update min pointer (if necessary).

insert 21 insert 21

min min

17 24 23 7 3 17 24 23 7 21 3

30 26 46 30 26 46 18 52 41 18 52 41 heap H heap H 35 35 39 44 39 44 15 16 Fibonacci heap: insert analysis

Actual cost. ci = O(1). FIBONACCI HEAPS one more tree; Change in potential. ∆Φ = Φ(Hi) – Φ(Hi–1) = +1. no change in marks ‣ preliminaries Amortized cost. ĉi = ci + ∆Φ = O(1). ‣ insert ‣ extract the minimum

Φ(H) = trees(H) + 2 ⋅ marks(H) ‣ decrease key ‣ bounding the rank min ‣ meld and delete

17 24 23 7 21 3 SECTION 19.2

30 26 46 18 52 41 heap H 35 39 44 17

Linking operation Fibonacci heap: extract the minimum

Useful primitive. Combine two trees T1 and T2 of rank k. ・Delete min; meld its children into root list; update min. ・Make larger root be a child of smaller root. ・Consolidate trees so that no two roots have same rank. ・Resulting tree T ' has rank k + 1.

tree T1 tree T2 tree T'

15 3 3

min

7 24 23 17 3 56 24 33 18 52 41 15 18 52 41

77 39 44 56 24 33 39 44 30 26 46 18 52 41

77 still heap-ordered 35 39 44

19 20 Fibonacci heap: extract the minimum Fibonacci heap: extract the minimum

・Delete min; meld its children into root list; update min. ・Delete min; meld its children into root list; update min. ・Consolidate trees so that no two roots have same rank. ・Consolidate trees so that no two roots have same rank.

current min 7 24 23 17 18 52 41 min 7 24 23 17 18 52 41

30 26 46 39 44 30 26 46 39 44

35 35

21 22

Fibonacci heap: extract the minimum Fibonacci heap: extract the minimum

rank rank

0 1 2 3 0 1 2 3

current current min 7 24 23 17 18 52 41 min 7 24 23 17 18 52 41

30 26 46 39 44 30 26 46 39 44

35 35

23 24 Fibonacci heap: extract the minimum Fibonacci heap: extract the minimum

rank rank link 23 to 17 0 1 2 3 0 1 2 3

current current min 7 24 23 17 18 52 41 min 7 24 23 17 18 52 41

30 26 46 39 44 30 26 46 39 44

35 35

25 26

Fibonacci heap: extract the minimum Fibonacci heap: extract the minimum

rank rank link 17 to 7 link 24 to 7 0 1 2 3 0 1 2 3

current current min min 7 24 17 18 52 41 24 7 18 52 41

30 26 46 23 39 44 26 46 17 30 39 44

35 35 23

27 28 Fibonacci heap: extract the minimum Fibonacci heap: extract the minimum

rank rank

0 1 2 3 0 1 2 3

current current min min 7 18 52 41 7 18 52 41

24 17 30 39 44 24 17 30 39 44

26 46 23 26 46 23

35 35 29 30

Fibonacci heap: extract the minimum Fibonacci heap: extract the minimum

rank rank link 41 to 18 0 1 2 3 0 1 2 3

current current min min 7 18 52 41 7 18 52 41

24 17 30 39 44 24 17 30 39 44

26 46 23 26 46 23

35 35 31 32 Fibonacci heap: extract the minimum Fibonacci heap: extract the minimum

rank rank

0 1 2 3 0 1 2 3

current current min min 7 52 18 7 52 18

24 17 30 41 39 24 17 30 41 39

26 46 23 44 26 46 23 44

35 35 33 34

Fibonacci heap: extract the minimum Fibonacci heap: extract the minimum analysis

・Delete min; meld its children into root list; update min. Actual cost. ci = O(rank(H)) + O(trees(H)). ・Consolidate trees so that no two roots have same rank. ・O(rank(H)) to meld min's children into root list. ≤ rank(H) children ・O(rank(H)) + O(trees(H)) to update min. ≤ rank(H) + trees(H) – 1 root nodes ・O(rank(H)) + O(trees(H)) to consolidate trees. number of roots decreases by 1 after each linking operation stop (no two trees have same rank) Change in potential. ∆Φ ≤ rank(H') + 1 – trees(H). ・No new nodes become marked. ・trees(H') ≤ rank(H') + 1. no two trees have same rank after consolidation

min 7 52 18 Amortized cost. O(log n). ・ĉi = ci + ∆Φ = O(rank(H)) + O(rank(H')). ・The rank of a Fibonacci heap with n elements is O(log n). 24 17 30 41 39

Fibonacci lemma (stay tuned) 26 46 23 44 Φ(H) = trees(H) + 2 ⋅ marks(H)

35 35 36 Fibonacci heap vs. binomial heaps

Observation. If only INSERT and EXTRACT-MIN operations, then all trees are binomial trees. FIBONACCI HEAPS

we link only trees of equal rank ‣ preliminaries

B0 B1 B2 B3 ‣ insert ‣ extract the minimum ‣ decrease key ‣ bounding the rank ‣ meld and delete

Binomial heap property. This implies rank(H) ≤ log2 n. SECTION 19.3

Fibonacci heap property. Our DECREASE-KEY implementation will not preserve

this property, but we will implement it in such a way that rank(H) ≤ logφ n.

Fibonacci heap: decrease key Fibonacci heap: decrease key

Intuition for deceasing the key of node x. Intuition for deceasing the key of node x. ・If heap-order is not violated, decrease the key of x. ・If heap-order is not violated, decrease the key of x. ・Otherwise, cut tree rooted at x and meld into root list. ・Otherwise, cut tree rooted at x and meld into root list.

decrease-key of x from 30 to 7 decrease-key of x from 23 to 5

6 6 7

8 29 10 44 8 29 10 44 45 32

30 23 22 48 31 17 23 22 48 31 17 55

45 32 24 50 24 50

55 39 40 Fibonacci heap: decrease key Fibonacci heap: decrease key

decrease-key of 22 to 4 decrease-key of 48 to 3 decrease-key of 31 to 2 6 7 5 6 7 5 4 3 2 1 decrease-key of 17 to 1

8 29 10 44 45 32 24 8 29 10 44 45 32 24 50

rank = 4, nodes = 5 22 48 31 17 55 55

41 42

Fibonacci heap: decrease key Fibonacci heap: decrease key

Intuition for deceasing the key of node x. Case 1. [heap order not violated] ・If heap-order is not violated, decrease the key of x. ・Decrease key of x. ・Otherwise, cut tree rooted at x and meld into root list. ・Change heap min pointer (if necessary). ・Solution: as soon as a node has its second child cut, cut it off also and meld into root list (and unmark it).

decrease-key of x from 46 to 29

min min

7 18 38 7 18 38

marked node: 24 17 23 21 39 41 24 17 23 21 39 41 one child already cut

26 46 30 52 26 46 30 52

35 88 72 35 88 72 43 44 Fibonacci heap: decrease key Fibonacci heap: decrease key

Case 1. [heap order not violated] Case 2a. [heap order violated] ・Decrease key of x. ・Decrease key of x. ・Change heap min pointer (if necessary). ・Cut tree rooted at x, meld into root list, and unmark. ・If parent p of x is unmarked (hasn't yet lost a child), mark it; Otherwise, cut p, meld into root list, and unmark (and do so recursively for all ancestors that lose a second child). decrease-key of x from 46 to 29 decrease-key of x from 29 to 15

min min

7 18 38 7 18 38

24 17 23 21 39 41 24 17 23 21 39 41 p

26 29 30 52 26 29 30 52

x x

35 88 72 35 88 72 45 46

Fibonacci heap: decrease key Fibonacci heap: decrease key

Case 2a. [heap order violated] Case 2a. [heap order violated] ・Decrease key of x. ・Decrease key of x. ・Cut tree rooted at x, meld into root list, and unmark. ・Cut tree rooted at x, meld into root list, and unmark. ・If parent p of x is unmarked (hasn't yet lost a child), mark it; ・If parent p of x is unmarked (hasn't yet lost a child), mark it; Otherwise, cut p, meld into root list, and unmark Otherwise, cut p, meld into root list, and unmark (and do so recursively for all ancestors that lose a second child). (and do so recursively for all ancestors that lose a second child). decrease-key of x from 29 to 15 decrease-key of x from 29 to 15

min min

7 18 38 7 18 38

24 17 23 21 39 41 24 17 23 21 39 41 p p

26 15 30 52 26 15 30 52

x x

35 88 72 35 88 72 47 48 Fibonacci heap: decrease key Fibonacci heap: decrease key

min min x 15 7 18 38 x 15 7 18 38

72 24 17 23 21 39 41 72 24 17 23 21 39 41 p p

26 30 52 26 30 52

35 88 35 88 49 50

Fibonacci heap: decrease key Fibonacci heap: decrease key

Case 2b. [heap order violated] Case 2b. [heap order violated] ・Decrease key of x. ・Decrease key of x. ・Cut tree rooted at x, meld into root list, and unmark. ・Cut tree rooted at x, meld into root list, and unmark. ・If parent p of x is unmarked (hasn't yet lost a child), mark it; ・If parent p of x is unmarked (hasn't yet lost a child), mark it; Otherwise, cut p, meld into root list, and unmark Otherwise, cut p, meld into root list, and unmark (and do so recursively for all ancestors that lose a second child). (and do so recursively for all ancestors that lose a second child). decrease-key of x from 35 to 5 decrease-key of x from 35 to 5

min min

15 7 18 38 15 7 18 38

72 24 17 23 21 39 41 72 24 17 23 21 39 41

p 26 30 52 p 26 30 52

x 35 88 x 5 88 51 52 Fibonacci heap: decrease key Fibonacci heap: decrease key

x min x min 15 5 7 18 38 15 5 7 18 38

72 24 17 23 21 39 41 72 24 17 23 21 39 41 second child cut

p 26 30 52 p 26 30 52

88 88 53 54

Fibonacci heap: decrease key Fibonacci heap: decrease key

x min p x min p 15 5 26 7 18 38 15 5 26 7 18 38

72 88 24 17 23 21 39 41 72 88 p' 24 17 23 21 39 41

30 52 second child cut 30 52

55 56 Fibonacci heap: decrease key Fibonacci heap: decrease key analysis

Case 2b. [heap order violated] Actual cost. ci = O(c), where c is the number of cuts. ・Decrease key of x. ・O(1) time for changing the key. ・Cut tree rooted at x, meld into root list, and unmark. ・O(1) time for each of c cuts, plus melding into root list. ・If parent p of x is unmarked (hasn't yet lost a child), mark it;

Otherwise, cut p, meld into root list, and unmark Change in potential. ∆Φ = O(1) – c. (and do so recursively for all ancestors that lose a second child). ・trees(H') = trees(H) + c. each cut (except first) unmarks a node ・marks(H') ≤ marks(H) – c + 2. last cut may or may not mark a node decrease-key of x from 35 to 5 ・ΔΦ ≤ c + 2 ⋅ (-c + 2) = 4 – c. x min p p' p'' 15 5 26 24 7 18 38 Amortized cost. ĉi = ci + ∆Φ = O(1).

but don't 72 88 mark parent 17 23 21 39 41 if it's a root

30 52 Φ(H) = trees(H) + 2 ⋅ marks(H)

57 58

Analysis summary

Insert. O(1). FIBONACCI HEAPS Delete-min. O(rank(H)) amortized. Decrease-key. O(1) amortized. ‣ preliminaries ‣ insert

‣ extract the minimum Fibonacci lemma. Let H be a Fibonacci heap with n elements. ‣ decrease key Then, rank(H) = O(log n). ‣ bounding the rank number of nodes is ‣ meld and delete exponential in rank

SECTION 19.4

60 Bounding the rank Bounding the rank

Lemma 1. Fix a point in time. Let x be a node of rank k, and let y1, …, yk Lemma 1. Fix a point in time. Let x be a node of rank k, and let y1, …, yk denote its current children in the order in which they were linked to x. denote its current children in the order in which they were linked to x. Then: Then: x x 0 B7 i =1 0 B7 i =1 rank(yi) rank(yi) i 2 B7 i 2 i 2 B7 i 2 y1 y2 … yk y1 y2 … yk

Def. Let Tk be smallest possible tree of rank k satisfying property. Pf. T T T T T T 0 1 2 3 4 5 ・When yi was linked into x, x had at least i – 1 children y1, …, yi–1. ・Since only trees of equal rank are linked, at that time

rank(yi) = rank(x) ≥ i – 1.

・Since then, yi has lost at most one child (or yi would have been cut).

・Thus, right now rank(yi) ≥ i – 2. ▪ F2 = 1 F3 = 2 F4 = 3 F5 = 5 F6 = 8 F7 = 13

61 62

Bounding the rank Bounding the rank

Lemma 1. Fix a point in time. Let x be a node of rank k, and let y1, …, yk Lemma 2. Let sk be minimum number of elements in any Fibonacci heap of th denote its current children in the order in which they were linked to x. rank k. Then sk ≥ Fk+2, where Fk is the k Fibonacci number. Then: x 0 B7 i =1 Pf. [by strong induction on k] rank(yi) i 2 B7 i 2 ・Base cases: s0 = 1 and s1 = 2. y1 y2 … yk ・Inductive hypothesis: assume si ≥ Fi+2 for i = 0, …, k – 1.

・As in Lemma 1, let let y1, …, yk denote its current children in the order in which they were linked to x.

Def. Let Tk be smallest possible tree of rank k satisfying property.

sk ≥ 1 + 1 + (s0 + s1 + … + sk–2) (Lemma 1)

T4 T5 T6 ≥ (1 + F1) + F2 + F3 + … + Fk (inductive hypothesis)

= Fk+2. ▪ (Fibonacci fact 1)

F6 = 8 F7 = 13 F8 = F6 + F7 = 8 + 13 = 21

63 64 Bounding the rank Fibonacci fact 1

Fibonacci lemma. Let H be a Fibonacci heap with n elements. Def. The Fibonacci sequence is: 0, 1, 1, 2, 3, 5, 8, 13, 21, …

Then, rank(H) ≤ logφ n, where φ is the golden ratio = (1 + √5) / 2 ≈ 1.618. 0 B7 k =0 F = 1 k =1 Pf. k B7 Fk 1 + Fk 2 B7 k 2 ・Let H is a Fibonacci heap with n elements and rank k. k ・Then n ≥ Fk+2 ≥ φ .

Fibonacci fact 1. For all integers k ≥ 0, Fk+2 = 1 + F0 + F1 + … + Fk. Lemma 2 Fibonacci Pf. [by induction on k] Fact 2

・Base case: F2 = 1 + F0 = 2.

・Taking logs, we obtain rank(H) = k ≤ logφ n. ▪ ・Inductive hypothesis: assume Fk+1 = 1 + F0 + F1 + … + Fk–1.

Fk+2 = Fk + Fk+1 (definition)

= Fk + (1 + F0 + F1 + … + Fk–1) (inductive hypothesis)

= 1 + F0 + F1 + … + Fk–1 + Fk. ▪ (algebra)

65 66

Fibonacci fact 2 Fibonacci numbers and nature

Def. The Fibonacci sequence is: 0, 1, 1, 2, 3, 5, 8, 13, 21, … Fibonacci numbers arise both in nature and algorithms.

0 B7 k =0 Fk = 1 B7 k =1 Fk 1 + Fk 2 B7 k 2

k Fibonacci fact 2. Fk+2 ≥ φ , where φ = (1 + √5) / 2 ≈ 1.618. Pf. [by induction on k]

・Base cases: F2 = 1 ≥ 1, F3 = 2 ≥ φ. k k + 1 ・Inductive hypotheses: assume Fk ≥ φ and Fk+1 ≥ φ

Fk+2 = Fk + Fk+1 (definition)

≥ φk – 1 + φk – 2 (inductive hypothesis)

pinecone = φk – 2 (1 + φ) (algebra)

= φk – 2 φ2 (φ2 = φ + 1) = φk. ▪ (algebra) cauliﬂower

67 68 Fibonacci heap: meld

Meld. Combine two Fibonacci heaps (destroying old heaps). FIBONACCI HEAPS Recall. Root lists are circular, doubly-linked lists. ‣ preliminaries ‣ insert ‣ extract the minimum ‣ decrease key ‣ bounding the rank meld and delete ‣ min min

23 24 17 7 3 21 SECTION 19.2, 19.3

30 26 46 18 52 41

heap H1 heap H2 35 39 44 70

Fibonacci heap: meld Fibonacci heap: meld analysis

Meld. Combine two Fibonacci heaps (destroying old heaps). Actual cost. ci = O(1). Change in potential. ∆Φ = 0.

Recall. Root lists are circular, doubly-linked lists. Amortized cost. ĉi = ci + ∆Φ = O(1).

Φ(H) = trees(H) + 2 ⋅ marks(H)

min min

23 24 17 7 3 21 23 24 17 7 3 21

30 26 46 30 26 46 18 52 41 18 52 41

35 heap H 35 heap H 39 44 39 44 71 72 Fibonacci heap: delete Priority queues performance cost summary

Delete. Given a handle to an element x, delete it from heap H. operation linked list binary heap binomial heap Fibonacci heap † ・DECREASE-KEY(H, x, -∞). ・EXTRACT-MIN(H). MAKE-HEAP O(1) O(1) O(1) O(1)

IS-EMPTY O(1) O(1) O(1) O(1) Amortized cost. ĉi = O(rank(H)). INSERT O(1) O(log n) O(log n) O(1) ・O(1) amortized for DECREASE-KEY. ・O(rank(H)) amortized for EXTRACT-MIN. EXTRACT-MIN O(n) O(log n) O(log n) O(log n)

DECREASE-KEY O(1) O(log n) O(log n) O(1)

DELETE O(1) O(log n) O(log n) O(log n)

MELD Φ(H) = trees(H) + 2 ⋅ marks(H) O(1) O(n) O(log n) O(1)

FIND-MIN O(n) O(1) O(log n) O(1)

† amortized

Accomplished. O(1) INSERT and DECREASE-KEY, O(log n) EXTRACT-MIN.

73 74

Heaps of heaps

・b-heaps. PRIORITY QUEUES ・Fat heaps. ・2-3 heaps. ‣ binary heaps ・Leaf heaps. ‣ d-ary heaps ・Thin heaps. ・Skew heaps. ‣ binomial heaps ・Splay heaps. ‣ Fibonacci heaps ・Weak heaps. ‣ advanced topics ・Leftist heaps. ・Quake heaps. ・Pairing heaps. ・Violation heaps. ・Run-relaxed heaps. ・Rank-pairing heaps. ・Skew-pairing heaps. ・Rank-relaxed heaps. Lazy Fibonacci heaps. ・ 76 Brodal queues Strict Fibonacci heaps

Q. Can we achieve same running time as for Fibonacci heap but with Q. Can we achieve same running time as for Fibonacci heap but with worst-case bounds per operation (instead of amortized)? worst-case bounds per operation (instead of amortized) in pointer model?

Theory. [Brodal 1996] Yes. Theory. [Brodal-Lagogiannis-Tarjan 2002] Yes.

Strict Fibonacci Heaps Strict Fibonacci Heaps Gerth Stølting Brodal George Lagogiannis Robert E. Tarjan† MADALGO∗ Agricultural University Dept. of Computer Science Dept. of Computer Science of Athens Princeton University Aarhus University GerthIera Odos Stølting 75, 11855 Brodal Athens Georgeand Lagogiannis HP Labs Robert E. Tarjan† MADALGOGreece∗ Agricultural35 Olden Street, University Princeton Dept. of Computer Science Åbogade 34, 8200 Aarhus N Dept. of Computer Science of Athens Princeton University Denmark [email protected] New Jersey 08540, USA [email protected] Aarhus University [email protected] Odos 75, 11855 Athens and HP Labs Åbogade 34, 8200 Aarhus N Greece 35 Olden Street, Princeton Denmark [email protected] New Jersey 08540, USA ABSTRACT [email protected]. INTRODUCTION [email protected] We present the first pointer-based heap implementation with Williams in 1964 introduced binary heaps [25]. Since then time bounds matching those of FibonacciABSTRACT heaps in the worst the design and analysis of heaps has1. been INTRODUCTION thoroughly inves- case. We support make-heap, insert, find-min, meld and tigated. The most common operations supported by the Williams in 1964 introduced binary heaps [25]. Since then decrease-key in worst-case O(1) time,We present and delete the firstand delete-pointer-basedheaps heap inimplementation the literature with are those listed below. We assume the design and analysis of heaps has been thoroughly inves- min in worst-case O(lg n)time,wheretime boundsn is the matching size of those the of Fibonaccithat each heaps item instored the worst contains an associated key. No item tigated. The most common operations supported by the heap. The data structure uses linearcase. space. We support make-heap,can insert, be in find-min, more than meld one and heap at a time. heaps in the literature are those listed below. We assume Aprevious,verycomplicated,solutionachievingthesamedecrease-key in worst-case O(1) time, and delete and delete- that each item stored contains an associated key. No item time bounds in the RAM model mademin essentialin worst-case use ofO arrays(lg n)time,wheremakeheap()n is theCreate size of a new,the empty heap and return a ref- can be in more than one heap at a time. and extensive use of redundant counterheap. schemes The data to structure maintain uses linear space.erence to it. Aprevious,verycomplicated,solutionachievingthesame balance. Our solution uses neither. Our key simplification insert(H, i) Insert item i,notcurrentlyinaheap,intomakeheap() Create a new, empty heap and return a ref- is to discard the structure of thetime smaller bounds heap in when the RAM doing model made essential use of arrays and extensive use of redundant counterheap schemesH,andreturnareferencetowhere to maintain erence to it.i is stored ameld.Weusethepigeonholeprincipleinplaceofthe in H. redundant counter mechanism. balance. Our solution uses neither. Our key simplification insert(H, i) Insert item i,notcurrentlyinaheap,into is to discard the structure of the smaller heap when doing meld(H1,H2) Return a reference to aheap new heapH,andreturnareferencetowhere containing i is stored ameld.Weusethepigeonholeprincipleinplaceofthe Categories and Subject Descriptors all items in the two heaps H1 andin HH. 2 (H1 and H2 redundant counter mechanism. cannot be accessed after meld). Practice. Ever implemented? Constants are high (and requires RAM model). E.1 [Data Structures]: Trees; F.2.2 [Theory of Compu- meld(H1,H2) Return a reference to a new heap containing tation]: Analysis of Algorithms andCategories Problem Complexity— and Subject Descriptorsfind-min(H) Return a reference to whereall items the in item the with two heaps H1 and H2 (H1 and H2 Nonnumerical Algorithms and Problems minimum key is stored in the heapcannotH. be accessed after meld). E.1 [Data Structures]: Trees; F.2.2 [Theory of Compu- delete-min(H) Delete the itemfind-min( with minimumH) Return key from a reference to where the item with 77 tation]: Analysis of Algorithms and Problem Complexity— 78 General Terms Nonnumerical Algorithms and Problemsthe heap H. minimum key is stored in the heap H. Algorithms delete(H,e) Delete an item fromdelete-min( the heap HHgiven) Delete a ref- the item with minimum key from General Terms erence e to where it is stored. the heap H. Keywords Algorithms decrease-key(H,e,k) Decreasedelete( the keyH,e of the) Delete item givenan item from the heap H given a ref- Data structures, heaps, meld, decrease-key, worst-case com- by the reference e in heap H to theerence newe keyto wherek. it is stored. plexity Keywords decrease-key(H,e,k) Decrease the key of the item given Data structures, heaps, meld, decrease-key,There are worst-case many heap com- implementations in the literature, by the reference e in heap H to the new key k. plexity with a variety of characteristics. We can divide them into two main categories, depending on whether the time bounds Fibonacci heaps: practice Pairing heaps∗Center for Massive Data Algorithmics, a Center of the Dan- There are many heap implementations in the literature, ish National Research Foundation. are worst case or amortized. Most of the heaps in the lit- with a variety of characteristics. We can divide them into †Partially supported by NSF grant CCF-0830676, US-Israel erature are based on heap-ordered trees, i.e. tree structures two main categories, depending on whether the time bounds Binational Science Foundation∗ GrantCenter 2006204, for Massive and Data the Algorithmics,where the a Center item stored of the Dan-in a node has a key not smaller than Distinguished Visitor Program ofish the National Stanford Research University Foundation.the key of the item stored in its parent.are worst Heap-ordered case or amortized. trees Most of the heaps in the lit- Computer Science Department. The†Partially information supported contained by NSF grantgive CCF-0830676, heap implementations US-Israel that achieveerature logarithmic are based ontime heap-ordered for trees, i.e. tree structures Q. Are Fibonacci heaps useful in practice? Pairing heap.herein doesA notself-adjusting necessarily reflectBinational the opinion Scienceheap-ordered or policy Foundation of all Grant the operations. 2006204, general and Early the examples tree.where are the the item implicit stored binary in a node has a key not smaller than the federal government and no offiDistinguishedcial endorsement Visitor should Program of the Stanford University the key of the item stored in its parent. Heap-ordered trees be inferred. Computer Science Department.heaps The information of Williamscontained [25], the leftist heaps of Crane [5] as modi- fied by Knuth [20], and the binomialgive heaps heap of implementations Vuillemin [24]. that achieve logarithmic time for herein does not necessarily reflect the opinion or policy of all the operations. Early examples are the implicit binary A. They are part of LEDA and Boost C++ libraries. the federal government and no offiThecial introduction endorsement of should Fibonacci heaps [15] by Fredman and be inferred. Tarjan was a breakthrough sinceheaps they achieved of WilliamsO(1) [25], amor- the leftist heaps of Crane [5] as modi- tized time for all the operations abovefied by except Knuth for [20], delete and and the binomial heaps of Vuillemin [24]. Permission to make digital or hard copies of all or part of thisworkfor The introduction of Fibonacci heaps [15] by Fredman and (but other heaps seem to perform better in practice) personal or classroom use is granted without fee provided that copies are delete-min, which require O(lg n) amortized time, where n not made or distributed for profit or commercial advantage andthatcopies is the number of items in the heapTarjan and lg was the a base-two breakthrough log- since they achieved O(1) amor- bear this notice and the full citation on thePermission first page. To to cop makeyotherwise,to digital or hard copiesarithm. of all or The part drawback of thisworkfor of Fibonaccitized heaps time is for that all the they operations are above except for delete and republish, to post on servers or to redistributepersonal to lists, or re classroomquires prior use specific is granted withoutcomplicated fee provided compared that copies to are existingdelete-min, solutions and which not require as ef- O(lg n) amortized time, where n permission and/or a fee. not made or distributed for profit or commercialficient advantagein practice andthatcopies as other, theoreticallyis the number less effi ofcient items sol inu- the heap and lg the base-two log- STOC.12, May 19–22, 2012, New York, Newbear York, this notice USA. and the full citation on the first page. To copyotherwise,to tions. Thus, Fibonacci heaps openedarithm. the The way drawback for further of Fibonacci heaps is that they are Copyright 2012 ACM 978-1-4503-1245-5/12/05republish, ...$10.00. to post on servers or to redistribute to lists, requires prior specific complicated compared to existing solutions and not as ef- permission and/or a fee. STOC.12, May 19–22, 2012, New York, New York, USA. ficient in practice as other, theoretically less efficient solu- Copyright 2012 ACM 978-1-4503-1245-5/12/05 ...$10.00. tions. Thus, Fibonacci heaps opened the way for further

Theory. Same amortized running times as Fibonacci heaps for all operations except DECREASE-KEY. ・O(log n) amortized. [Fredman et al. 1986] ・Ω(log log n) lower bound on amortized cost. [Fredman 1999] ・ 2 O (log log n ) amortized. [Pettie 2005]

79 80 Pairing heaps Priority queues performance cost summary

Pairing heap. A self-adjusting heap-ordered general tree. binomial pairing Fibonacci Brodal operation linked list binary heap heap heap † heap † queue Practice. As fast as (or faster than) the binary heap on some problems. Included in GNU C++ library and LEDA. MAKE-HEAP O(1) O(1) O(1) O(1) O(1) O(1)

IS-EMPTY O(1) O(1) O(1) O(1) O(1) O(1)

INSERT RESEARCHCONlRlWlIONS O(1) O(log n) O(log n) O(1) O(1) O(1)

Algorithms and Data Structures Pairing Heaps: G. Scott Graham EXTRACT-MIN Editor Experiments and Analysis O(n) O(log n) O(log n) O(log n) O(log n) O(log n)

DECREASE-KEY O(1) O(log n) O(log n) 2O(log log n) O(1) O(1)

DELETE O(1) O(log n) O(log n) O(log n) O(log n) O(log n)

JOHN T. STASKO and JEFFREY SCOTT VlllER MELD O(1) O(n) O(log n) O(1) O(1) O(1)

ABSTRACT: The pairing heap has recently been and practical importance from their use in solving a introduced as a new data structure for priority queues. wide range of combinatorial problems, including job Pairing heaps are extremely simple to implement and scheduling, minimal spanning tree, shortest path, FIND-MIN O(n) O(1) O(log n) O(1) O(1) O(1) seem to be very efficient in practice, but they are difficult and graph traversal. to analyze theoretically, and open problems remain. It Priority queues support the operations insert, has been conjectured that they achieve the same find-min, and delete-min; additional operations often amortized time bounds as Fibonacci heaps, namely, include decrease-key and delete. The insert(t, v) opera- O(log n) time for delete and delete-min and O(1) for tion adds item t with key value v to the priority all other operations, where n is the size of the priority queue. The find-min operation returns the item † amortized queue at the time of the operation. We provide empirical with minimum key value. The delete-min operation evidence that supports this conjecture. The most returns the item with minimum key value and promising algorithm in our simulations is a new variant removes it from the priority queue. The decrease- of the twopass method, called auxiliary twopass. We key(t, d) operation reduces item t’s key value by d. prove that, assuming no decrease-key operations are The delete(t) operation removes item t from the performed, it achieves the same amortized time bounds as priority queue. The decrease-key and delete opera- Fibonacci heaps. tions require that a pointer to the location in the priority queue of item t be supplied explicitly, since 1. INTRODUCTION priority queues do not support searching for arbi- A priority queue is an abstract data type for main- trary items by value. Some priority queues also sup- taining and manipulating a set of items based on port the merge operation, which combines two item- priority [I]. Prio’rity queues derive great theoretical disjoint priority queues. 81 82 We will concentrate on the insert, delete-min, and Support was provided in part by NSF research grant DCR-84-03613, an NSF Presidential Young Investigator Award, an IBM Faculty Development Award, decrease-key operations because they are the opera- and a Guggenheim Fellowship. tions that primarily distinguish priority queues from Part of this research was performed at Mathematical Sciences Research Insti- tute. Berkeley, Calif., and the Institut National de Recherche en Informatique other set manipulation algorithms and because they et en Automatique, Rocquencourt. France. are the critical operations as far as the time bounds 0 1987 ACM OOOl-0782/87/0300-0234 75a: are concerned.

Communications of the ACM March 1987 Volume 30 Number 3 Priority queues with integer priorities Priority queues with integer priorities

Assumption. Keys are integers between 0 and C. Assumption. Keys are integers between 0 and C.

Theorem. [Thorup 2004] There exists a priority queue that supports INSERT, Theorem. [Thorup 2004] There exists a priority queue that supports INSERT, FIND-MIN, and DECREASE-KEY in constant time and EXTRACT-MIN and DELETE-KEY FIND-MIN, and DECREASE-KEY in constant time and EXTRACT-MIN and DELETE-KEY in either O(log log n) or O(log log C) time. in either O(log log n) or O(log log C) time.

ARTICLE IN PRESS Corollary 1. Can implement Dijkstra's algorithm in either O(m log log n) or O(m log log C) time.

Journal of Computer and System Sciences 69 (2004) 330–353 http://www.elsevier.com/locate/jcss Corollary 2. Can sort n integers in O(n log log n) time. Integer priority queues with decrease key in constant time and the single source shortest paths problem Mikkel Thorup Computational model. Word RAM. AT&T Labs Research, Shannon Laboratory, 180 Park Avenue, Florham Park, NJ 07932, USA Received 22 July 2003; revised 8 March 2004 Available online 20 July 2004

Abstract

We consider Fibonacci heap style integer priority queues supporting ﬁnd-min, insert, and decrease key operations in constant time. We present a deterministic linear space solution that with n integer keys supports delete in O log log n time. If the integers are in the range 0; N ; we can also support delete in O log log N time. ð Þ ½ Þ ðEven for theÞ special case of monotone priority queues, where the minimum has to be non-decreasing, the 1= 3 e 1= 4 e best previous bounds on delete were O log n ð À Þ and O log N ð À Þ : These previous bounds used both randomization and amortization. Ourðð newÞ boundsÞ are deterministic,ðð Þ worst-case,Þ with no restriction to monotonicity, and exponentially faster. 83 84 As a classical application, for a directed graph with n nodes and m edges with non-negative integer weights, we get single source shortest paths in O m n log log n time, or O m n log log C if C is the maximal edge weight. The latter solves an open problemð þ of Ahuja,Þ Mehlhorn, Orlin,ð þ and TarjanÞ from 1990. r 2004 Elsevier Inc. All rights reserved.

Keywords: Integer priority queues; Decrease key; Fibonacci heaps; Single source shortest paths

1. Introduction

In 1984, Fredman and Tarjan introduced Fibonacci heaps [15], which is a priority queue over a dynamic ordered set H supporting the following operations: find-min H Returns an element from H with minimum key value in constant time. insert Hð; aÞ Adds the element a to H in constant time. dec-keyð HÞ; a; x Reduces the key value of element a to x in constant time. If the current key value of aðwas smallerÞ than x; it is an error.

E-mail address: [email protected].

Goal. Break information-theoretic lower bound by allowing priority queue to Goal. Break information-theoretic lower bound by allowing priority queue to corrupt 10% of the keys (by increasing them). corrupt 10% of the keys (by increasing them).

Representation. ・Set of binomial trees (with some subtrees missing). ・Each node may store several elements. ・Each node stores a value that is an upper bound on the original keys. ・Binomial trees are heap-ordered with respect to these values.

elements inserted 0 11 22 44 55

88 11 22 99 44 33 77 66 0 55 57 66 77 88 99

soft heap corrupted

85 86

Soft heaps Soft heaps

Theorem. [Chazelle 2000] Starting from an empty soft heap, any sequence Q. Brilliant. But how could it possibly be useful? of n INSERT, MIN, EXTRACT-MIN, MELD, and DELETE operations takes O(n) time Ex. Linear-time deterministic selection. To find kth smallest element: and at most 10% of its elements are corrupted at any given time. ・Insert the n elements into soft heap. ・Extract the minimum element n / 2 times. ・The largest element deleted ≥ 4n / 10 elements and ≤ 6n / 10 elements. The Soft Heap: An Approximate Priority Queue with Optimal Error Rate ・Can remove ≥ 5n / 10 of elements and recur. ・T(n) ≤ T(3n / 5) + O(n) ⇒ T(n) = O(n). ▪ BERNARD CHAZELLE

Princeton University, Princeton, New Jersey, and NEC Research Institute

Abstract. A simple variant of a priority queue, called a soft heap,isintroduced.Thedatastructure supports the usual operations: insert, delete, meld, and findmin. Its novelty is to beat the logarithmic bound on the complexity of a heap in a comparison-based model. To break this information-theoretic barrier, the entropy of the data structure is reduced by artificially raising the values of certain keys. Given any mixed sequence of n operations, a soft heap with error rate ␧ (for any 0 Ͻ ␧Յ1/2) ensures that, at any time, at most ␧n of its items have their keys raised. The amortized complexity of each operation is constant, except for insert, which takes O(log 1/␧)time.Thesoftheapisoptimalforany value of ␧ in a comparison-based model. The data structure is purely pointer-based. No arrays are used and no numeric assumptions are made on the keys. The main idea behind the soft heap is to move items across the data structure not individually, as is customary, but in groups, in a data-structuring equivalent of “car pooling.” Keys must be raised as a result, in order to preserve the heap ordering of the data structure. The soft heap can be used to compute exact or approximate medians and percentiles optimally. It is also useful for approximate sorting and for computing minimum spanning trees of general graphs. Categories and Subject Descriptors: E.1 [Data Structures]: Nonnumerical Algorithms and Problems 87 88 General Terms: Theory Additional Key Words and Phrases: Amortization, heap, priority queue, soft heap

1. Introduction We design a simple variant of a priority queue, called a soft heap.Thedata structure stores items with keys from a totally ordered universe, and supports the operations:

ApreliminaryversionofthispaperasCHAZELLE,B.1998.Car-poolingasadatastructuringdevice: The soft heap. In Proceedings of the 6th Annual European Symposium on Algorithms,pp.35–42. This work was supported in part by National Science Foundation (NSF) Grants CCR 93-01254 and CCR 96-23768, ARO Grant DAAH04-96-1-0181, and NEC Research Institute. Author’s address: Department of Computer Science, Princeton University, 35 Olden Street, Prince- ton, NJ 08544-2087, e-mail: [email protected] or NEC Research Institute, e-mail: chazelle@ research.nj.nec.com. Permission to make digital/hard copy of part or all of this work for personal or classroom use is granted without fee provided that the copies are not made or distributed for profit or commercial advantage, the copyright notice, the title of the publication, and its date appear, and notice is given that copying is by permission of the Association for Computing Machinery (ACM), Inc. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. ©2000ACM0004-5411/00/1100-1012$05.00

Journal of the ACM, Vol. 47, No. 6, November 2000, pp. 1012–1027. Soft heaps

Theorem. [Chazelle 2000] There exists an O(m α(m, n)) time deterministic algorithm to compute an MST in a graph with n nodes and m edges.

Algorithm. Borůvka + nongreedy + divide-and-conquer + soft heap + …

AMinimumSpanningTreeAlgorithmwithInverse- Ackermann Type Complexity

BERNARD CHAZELLE

Princeton University, Princeton, New Jersey, and NEC Research Institute

Abstract. A deterministic algorithm for computing a minimum spanning tree of a connected graph is presented. Its running time is O(m␣(m, n)), where ␣ is the classical functional inverse of Ackermann’s function and n (respectively, m)isthenumberofvertices(respectively,edges).The algorithm is comparison-based: it uses pointers, not arrays, and it makes no numeric assumptions on the edge costs. Categories and Subject Descriptors: F.2.2 [Analysis of Algorithms and Problem Complexity]: Nonnumerical Algorithms and Problems. General Terms: Theory

Additional Key Words and Phrases: Graphs, matroids, minimum spanning trees 89

1. Introduction

The history of the minimum spanning tree (MST)problemislongandrich,going as far back as Boru˚vka’s work in 1926 [Boru˚vka 1926; Graham and Hell 1985; Nesˇetrˇil 1997]. In fact, MST is perhaps the oldest open problem in computer science. According to Nesˇetrˇil [1997], “this is a cornerstone problem of combinatorial optimization and in a sense its cradle.” Textbook algorithms run in O(m log n)time,wheren and m denote, respectively, the number of vertices and edges in the graph. Improvements to O(m log log n)weregivenindependently by Yao [1975] and by Cheriton and Tarjan [1976]. In the mid-eighties, Fredman and Tarjan [1987] lowered the complexity to O(m␤(m, n)), where ␤(m, n)is the number of log-iterations necessary to map n to a number less than m/n.In the worst case, m ϭ O(n)andtherunningtimeisO(m log* m). Soon after, the

ApreliminaryversionofthispaperappearedasCHAZELLE,B.1997.Afasterdeterministicalgorithm for minimum spanning trees. In Proceedings of the 38th Annual IEEE Symposium on Foundations of Computer Science.IEEEComputerSocietyPress,LosAlamitos,Calif.,pp.22–31. This work was supported in part by the National Science Foundation (NSF) Grants CCR 93-01254 and CCR 96-23768, ARO Grant DAAH04-96-1-0181, and NEC Research Institute. Author’s address: Department of Computer Science, Princeton University, 35 Olden Street, Prince- ton, NJ 083-44-2087, e-mail: [email protected] and NEC Research Institute, e-mail: [email protected]. Permission to make digital/hard copy of part or all of this work for personal or classroom use is granted without fee provided that the copies are not made or distributed for profit or commercial advantage, the copyright notice, the title of the publication, and its date appear, and notice is given that copying is by permission of the Association for Computing Machinery (ACM), Inc. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. ©2000ACM0004-5411/00/1100-1028$05.00

Journal of the ACM, Vol. 47, No. 6, November 2000, pp. 1028–1047.