Priority Queues Performance Cost Summary
Linked Binary Binomial Fibonacci Relaxed Operation Fibonacci Heaps List Heap Heap Heap † Heap
make-heap 1 1 1 1 1
is-empty 1 1 1 1 1
insert 1 log n log n 1 1
delete-min n log n log n log n log n
decrease-key n log n log n 1 1
delete n log n log n log n log n
union 1 n log n 1 1
Lecture slides adapted from: find-min n 1 log n 1 1
• Chapter 20 of Introduction to Algorithms by Cormen, Leiserson, Rivest, and Stein. n = number of elements in priority queue † amortized • Chapter 9 of The Design and Analysis of Algorithms by Dexter Kozen.
Theorem. Starting from empty Fibonacci heap, any sequence of
a1 insert, a2 delete-min, and a3 decrease-key operations takes
O(a1 + a2 log n + a3) time. COS 423 Theory of Algorithms • Kevin Wayne • Spring 2007 2
Priority Queues Performance Cost Summary Fibonacci Heaps
Linked Binary Binomial Fibonacci Relaxed History. [Fredman and Tarjan, 1986] Operation List Heap Heap Heap † Heap ! Ingenious data structure and analysis.
make-heap 1 1 1 1 1 ! Original motivation: improve Dijkstra's shortest path algorithm is-empty 1 1 1 1 1 from O(E log V ) to O(E + V log V ). V insert, V delete-min, E decrease-key insert 1 log n log n 1 1
delete-min n log n log n log n log n Basic idea. decrease-key n log n log n 1 1 ! Similar to binomial heaps, but less rigid structure. delete n log n log n log n log n ! Binomial heap: eagerly consolidate trees after each insert. union 1 n log n 1 1
find-min n 1 log n 1 1
n = number of elements in priority queue † amortized
Hopeless challenge. O(1) insert, delete-min and decrease-key. Why? ! Fibonacci heap: lazily defer consolidation until next delete-min.
3 4 Fibonacci Heaps: Structure Fibonacci Heaps: Structure
each parent larger than its children Fibonacci heap. Fibonacci heap.
! Set of heap-ordered trees. ! Set of heap-ordered trees.
! Maintain pointer to minimum element. ! Maintain pointer to minimum element.
! Set of marked nodes. ! Set of marked nodes. find-min takes O(1) time
roots heap-ordered tree min
17 24 23 7 3 17 24 23 7 3
30 26 46 30 26 46 18 52 41 18 52 41
Heap H 35 Heap H 35 39 44 39 44
5 6
Fibonacci Heaps: Structure Fibonacci Heaps: Notation
Fibonacci heap. Notation.
! Set of heap-ordered trees. ! n = number of nodes in heap.
! Maintain pointer to minimum element. ! rank(x) = number of children of node x.
! Set of marked nodes. ! rank(H) = max rank of any node in heap H.
! trees(H) = number of trees in heap H. use to keep heaps flat (stay tuned) ! marks(H) = number of marked nodes in heap H.
min trees(H) = 5 marks(H) = 3 n = 14 rank = 3 min
17 24 23 7 3 17 24 23 7 3
30 26 46 30 26 46 18 52 41 18 52 41
Heap H 35 marked Heap H 35 marked 39 44 39 44
7 8 Fibonacci Heaps: Potential Function
Insert !(H) !=!trees(H) + 2 " marks(H)
potential of heap H
trees(H) = 5 marks(H) = 3 !(H) = 5 + 2"3 = 11 min
17 24 23 7 3
30 26 46 18 52 41
Heap H 35 marked 39 44
9 10
Fibonacci Heaps: Insert Fibonacci Heaps: Insert
Insert. Insert.
! Create a new singleton tree. ! Create a new singleton tree.
! Add to root list; update min pointer (if necessary). ! Add to root list; update min pointer (if necessary).
insert 21 insert 21
21 min min
17 24 23 7 3 17 24 23 7 21 3
30 26 46 30 26 46 18 52 41 18 52 41
Heap H 35 Heap H 35 39 44 39 44
11 12 Fibonacci Heaps: Insert Analysis
Actual cost. O(1) !(H) !=!trees(H) + 2 " marks(H) Delete Min
Change in potential. +1 potential of heap H
Amortized cost. O(1)
min
17 24 23 7 21 3
30 26 46 18 52 41
Heap H 35 39 44
13 14
Linking Operation Fibonacci Heaps: Delete Min
Linking operation. Make larger root be a child of smaller root. Delete min.
! Delete min; meld its children into root list; update min.
! Consolidate trees so that no two roots have same rank.
larger root smaller root still heap-ordered
min 15 3 3
7 24 23 17 3 56 24 18 52 41 15 18 52 41
77 39 44 56 24 39 44 30 26 46 18 52 41 tree T tree T 1 2 77 35 39 44 tree T'
15 16 Fibonacci Heaps: Delete Min Fibonacci Heaps: Delete Min
Delete min. Delete min.
! Delete min; meld its children into root list; update min. ! Delete min; meld its children into root list; update min.
! Consolidate trees so that no two roots have same rank. ! Consolidate trees so that no two roots have same rank.
min min current
7 24 23 17 18 52 41 7 24 23 17 18 52 41
30 26 46 39 44 30 26 46 39 44
35 35
17 18
Fibonacci Heaps: Delete Min Fibonacci Heaps: Delete Min
Delete min. Delete min.
! Delete min; meld its children into root list; update min. ! Delete min; meld its children into root list; update min.
! Consolidate trees so that no two roots have same rank. ! Consolidate trees so that no two roots have same rank.
rank rank 0 1 2 3 0 1 2 3
min min current current
7 24 23 17 18 52 41 7 24 23 17 18 52 41
30 26 46 39 44 30 26 46 39 44
35 35
19 20 Fibonacci Heaps: Delete Min Fibonacci Heaps: Delete Min
Delete min. Delete min.
! Delete min; meld its children into root list; update min. ! Delete min; meld its children into root list; update min.
! Consolidate trees so that no two roots have same rank. ! Consolidate trees so that no two roots have same rank.
rank rank 0 1 2 3 0 1 2 3
min min
7 24 23 17 18 52 41 7 24 23 17 18 52 41
30 26 46 current 39 44 30 26 46 current 39 44
35 35 link 23 into 17
21 22
Fibonacci Heaps: Delete Min Fibonacci Heaps: Delete Min
Delete min. Delete min.
! Delete min; meld its children into root list; update min. ! Delete min; meld its children into root list; update min.
! Consolidate trees so that no two roots have same rank. ! Consolidate trees so that no two roots have same rank.
rank rank 0 1 2 3 0 1 2 3
min current
min 7 24 17 18 52 41 24 7 18 52 41
30 26 46 current 23 39 44 26 46 17 30 39 44
35 35 23 link 17 into 7 link 24 into 7
23 24 Fibonacci Heaps: Delete Min Fibonacci Heaps: Delete Min
Delete min. Delete min.
! Delete min; meld its children into root list; update min. ! Delete min; meld its children into root list; update min.
! Consolidate trees so that no two roots have same rank. ! Consolidate trees so that no two roots have same rank.
rank rank 0 1 2 3 0 1 2 3
current current
min min 7 18 52 41 7 18 52 41
24 17 30 39 44 24 17 30 39 44
26 46 23 26 46 23
35 35 25 26
Fibonacci Heaps: Delete Min Fibonacci Heaps: Delete Min
Delete min. Delete min.
! Delete min; meld its children into root list; update min. ! Delete min; meld its children into root list; update min.
! Consolidate trees so that no two roots have same rank. ! Consolidate trees so that no two roots have same rank.
rank rank 0 1 2 3 0 1 2 3
current current
min min 7 18 52 41 7 18 52 41
24 17 30 39 44 24 17 30 39 44
26 46 23 26 46 23 link 41 into 18 35 35 27 28 Fibonacci Heaps: Delete Min Fibonacci Heaps: Delete Min
Delete min. Delete min.
! Delete min; meld its children into root list; update min. ! Delete min; meld its children into root list; update min.
! Consolidate trees so that no two roots have same rank. ! Consolidate trees so that no two roots have same rank.
rank rank 0 1 2 3 0 1 2 3
current current
min min 7 52 18 7 52 18
24 17 30 41 39 24 17 30 41 39
26 46 23 44 26 46 23 44
35 35 29 30
Fibonacci Heaps: Delete Min Fibonacci Heaps: Delete Min Analysis
Delete min. Delete min. !(H) !=!trees(H) + 2 " marks(H) ! Delete min; meld its children into root list; update min. potential function ! Consolidate trees so that no two roots have same rank.
Actual cost. O(rank(H)) + O(trees(H))
! O(rank(H)) to meld min's children into root list.
! O(rank(H)) + O(trees(H)) to update min.
! O(rank(H)) + O(trees(H)) to consolidate trees.
min 7 52 18 Change in potential. O(rank(H)) - trees(H) ! trees(H' ) # rank(H) + 1 since no two trees have same rank.
! $!(H) # rank(H) + 1 - trees(H). 24 17 30 41 39 Amortized cost. O(rank(H))
26 46 23 44 stop 35 31 32 Fibonacci Heaps: Delete Min Analysis
Q. Is amortized cost of O(rank(H)) good? Decrease Key
A. Yes, if only insert and delete-min operations.
! In this case, all trees are binomial trees.
! This implies rank(H) # lg n. we only link trees of equal rank
B0 B1 B2 B3
A. Yes, we'll implement decrease-key so that rank(H) = O(log n).
33 34
Fibonacci Heaps: Decrease Key Fibonacci Heaps: Decrease Key
Intuition for deceasing the key of node x. Case 1. [heap order not violated]
! If heap-order is not violated, just decrease the key of x. ! Decrease key of x.
! Otherwise, cut tree rooted at x and meld into root list. ! Change heap min pointer (if necessary).
! To keep trees flat: as soon as a node has its second child cut, cut it off and meld into root list (and unmark it).
min min
7 18 38 7 18 38
marked node: one child already cut 24 17 23 21 39 41 24 17 23 21 39 41
26 46 30 52 26 4269 30 52
x
35 88 72 35 88 72 decrease-key of x from 46 to 29 35 36 Fibonacci Heaps: Decrease Key Fibonacci Heaps: Decrease Key
Case 1. [heap order not violated] Case 2a. [heap order violated]
! Decrease key of x. ! Decrease key of x.
! Change heap min pointer (if necessary). ! Cut tree rooted at x, meld into root list, and unmark.
! If parent p of x is unmarked (hasn't yet lost a child), mark it; Otherwise, cut p, meld into root list, and unmark (and do so recursively for all ancestors that lose a second child).
min min
7 18 38 7 18 38
24 17 23 21 39 41 24 17 23 21 39 41 p
26 29 30 52 26 2195 30 52 x x
35 88 72 decrease-key of x from 46 to 29 35 88 72 decrease-key of x from 29 to 15 37 38
Fibonacci Heaps: Decrease Key Fibonacci Heaps: Decrease Key
Case 2a. [heap order violated] Case 2a. [heap order violated]
! Decrease key of x. ! Decrease key of x.
! Cut tree rooted at x, meld into root list, and unmark. ! Cut tree rooted at x, meld into root list, and unmark.
! If parent p of x is unmarked (hasn't yet lost a child), mark it; ! If parent p of x is unmarked (hasn't yet lost a child), mark it; Otherwise, cut p, meld into root list, and unmark Otherwise, cut p, meld into root list, and unmark (and do so recursively for all ancestors that lose a second child). (and do so recursively for all ancestors that lose a second child).
min x min
7 18 38 15 7 18 38
24 17 23 21 39 41 72 24 17 23 21 39 41 p p
26 15 30 52 26 30 52
x
35 88 72 decrease-key of x from 29 to 15 35 88 decrease-key of x from 29 to 15 39 40 Fibonacci Heaps: Decrease Key Fibonacci Heaps: Decrease Key
Case 2a. [heap order violated] Case 2b. [heap order violated]
! Decrease key of x. ! Decrease key of x.
! Cut tree rooted at x, meld into root list, and unmark. ! Cut tree rooted at x, meld into root list, and unmark.
! If parent p of x is unmarked (hasn't yet lost a child), mark it; ! If parent p of x is unmarked (hasn't yet lost a child), mark it; Otherwise, cut p, meld into root list, and unmark Otherwise, cut p, meld into root list, and unmark (and do so recursively for all ancestors that lose a second child). (and do so recursively for all ancestors that lose a second child).
x min min
15 7 18 38 15 7 18 38
72 24 17 23 21 39 41 72 24 17 23 21 39 41 p mark parent
26 30 52 p 26 30 52
decrease-key of x from 29 to 15 decrease-key of x from 35 to 5 35 88 x 355 88 41 42
Fibonacci Heaps: Decrease Key Fibonacci Heaps: Decrease Key
Case 2b. [heap order violated] Case 2b. [heap order violated]
! Decrease key of x. ! Decrease key of x.
! Cut tree rooted at x, meld into root list, and unmark. ! Cut tree rooted at x, meld into root list, and unmark.
! If parent p of x is unmarked (hasn't yet lost a child), mark it; ! If parent p of x is unmarked (hasn't yet lost a child), mark it; Otherwise, cut p, meld into root list, and unmark Otherwise, cut p, meld into root list, and unmark (and do so recursively for all ancestors that lose a second child). (and do so recursively for all ancestors that lose a second child).
min min x
15 7 18 38 15 5 7 18 38
72 24 17 23 21 39 41 72 24 17 23 21 39 41
p 26 30 52 p 26 30 52
decrease-key of x from 35 to 5 decrease-key of x from 35 to 5 x 5 88 88 43 44 Fibonacci Heaps: Decrease Key Fibonacci Heaps: Decrease Key
Case 2b. [heap order violated] Case 2b. [heap order violated]
! Decrease key of x. ! Decrease key of x.
! Cut tree rooted at x, meld into root list, and unmark. ! Cut tree rooted at x, meld into root list, and unmark.
! If parent p of x is unmarked (hasn't yet lost a child), mark it; ! If parent p of x is unmarked (hasn't yet lost a child), mark it; Otherwise, cut p, meld into root list, and unmark Otherwise, cut p, meld into root list, and unmark (and do so recursively for all ancestors that lose a second child). (and do so recursively for all ancestors that lose a second child).
min min x x p
15 5 7 18 38 15 5 26 7 18 38
72 24 17 23 21 39 41 72 88 24 17 23 21 39 41 second child cut
p 26 30 52 30 52
88 decrease-key of x from 35 to 5 decrease-key of x from 35 to 5 45 46
Fibonacci Heaps: Decrease Key Fibonacci Heaps: Decrease Key
Case 2b. [heap order violated] Case 2b. [heap order violated]
! Decrease key of x. ! Decrease key of x.
! Cut tree rooted at x, meld into root list, and unmark. ! Cut tree rooted at x, meld into root list, and unmark.
! If parent p of x is unmarked (hasn't yet lost a child), mark it; ! If parent p of x is unmarked (hasn't yet lost a child), mark it; Otherwise, cut p, meld into root list, and unmark Otherwise, cut p, meld into root list, and unmark (and do so recursively for all ancestors that lose a second child). (and do so recursively for all ancestors that lose a second child).
min min x p x p p' p''
15 5 26 7 18 38 15 5 26 24 7 18 38
72 88 p' 24 17 23 21 39 41 72 88 don't mark 17 23 21 39 41 parent if it's a root
second child cut 30 52 30 52
decrease-key of x from 35 to 5 decrease-key of x from 35 to 5
47 48 Fibonacci Heaps: Decrease Key Analysis
Decrease-key. !(H) !=!trees(H) + 2 " marks(H) Analysis
potential function
Actual cost. O(c)
! O(1) time for changing the key.
! O(1) time for each of c cuts, plus melding into root list.
Change in potential. O(1) - c
! trees(H') = trees(H) + c.
! marks(H') # marks(H) - c + 2.
! $! # c + 2 " (-c + 2) = 4 - c.
Amortized cost. O(1)
49 50
Analysis Summary Fibonacci Heaps: Bounding the Rank
Insert. O(1) Lemma. Fix a point in time. Let x be a node, and let y1, …, yk denote Delete-min. O(rank(H)) † its children in the order in which they were linked to x. Then: Decrease-key. O(1) † x † amortized $ 0 if i =1 rank (yi ) " % & i # 2 if i "1 … y1 y2 yk
Key lemma. rank(H) = O(log n). ! number of nodes is exponential in rank Pf.
! When yi was linked into x, x had at least i -1 children y1, …, yi-1.
! Since only trees of equal rank are linked, at that time
rank(yi)!= rank(xi) % i - 1.
! Since then, yi has lost at most one child. ! Thus, right now rank(yi) % i - 2. or yi would have been cut
51 52 Fibonacci Heaps: Bounding the Rank Fibonacci Heaps: Bounding the Rank
Lemma. Fix a point in time. Let x be a node, and let y1, …, yk denote Lemma. Fix a point in time. Let x be a node, and let y1, …, yk denote its children in the order in which they were linked to x. Then: its children in the order in which they were linked to x. Then:
x x $ 0 if i =1 $ 0 if i =1 rank (yi ) " % rank (yi ) " % & i # 2 if i "1 & i # 2 if i "1 … … y1 y2 yk y1 y2 yk
! !
Def. Let Fk be smallest possible tree of rank k satisfying property. Def. Let Fk be smallest possible tree of rank k satisfying property.
F0 F1 F2 F3 F4 F5 F4 F5 F6
1 2 3 5 8 13
8 13 8 + 13 = 21
53 54
Fibonacci Heaps: Bounding the Rank
Lemma. Fix a point in time. Let x be a node, and let y , …, y denote 1 k Fibonacci Numbers its children in the order in which they were linked to x. Then:
x $ 0 if i =1 rank (yi ) " % & i # 2 if i "1 … y1 y2 yk
!
Def. Let Fk be smallest possible tree of rank k satisfying property.
k Fibonacci fact. Fk % & , where & = (1 + '5) / 2 ( 1.618.
golden ratio Corollary. rank(H) # log& n .
55 56 Fibonacci Numbers: Exponential Growth Fibonacci Numbers and Nature
Def. The Fibonacci sequence is: 1, 2, 3, 5, 8, 13, 21, …
# 1 if k = 0 % slightly non-standard definition Fk = $ 2 if k =1 % & Fk-1 + Fk-2 if k " 2
Lemma. F % &k, where & = (1 + '5) / 2 ( 1.618. ! k
Pf. [by induction on k] pinecone ! Base cases: F0 = 1 % 1, F1 = 2 % &. k k + 1 ! Inductive hypotheses: Fk % & and Fk+1 % & cauliflower
Fk+2 = Fk + Fk+1 (definition) k k+1 " # + # (inductive hypothesis) k = # (1 + #) (algebra) k 2 = # (# ) (&2 = & + 1) k+2 = # (algebra)
57 58
!
Fibonacci Heaps: Union
Union Union. Combine two Fibonacci heaps.
Representation. Root lists are circular, doubly linked lists.
min min
23 24 17 7 3 21
30 26 46 18 52 41
Heap H' 35 Heap H'' 39 44
59 60 Fibonacci Heaps: Union Fibonacci Heaps: Union
Union. Combine two Fibonacci heaps. Actual cost. O(1) !(H) !=!trees(H) + 2 " marks(H)
Representation. Root lists are circular, doubly linked lists. Change in potential. 0 potential function
Amortized cost. O(1)
min min
23 24 17 7 3 21 23 24 17 7 3 21
30 26 46 30 26 46 18 52 41 18 52 41
35 Heap H 35 Heap H 39 44 39 44
61 62
Fibonacci Heaps: Delete
Delete node x. Delete !(H) !=!trees(H) + 2 " marks(H) ! decrease-key of x to -). potential function ! delete-min element in heap.
Amortized cost. O(rank(H))
! O(1) amortized for decrease-key.
! O(rank(H)) amortized for delete-min.
63 64