Priority Queues Performance Cost Summary

Linked Binary Binomial Fibonacci Relaxed Operation Fibonacci Heaps List Heap Heap † Heap

make-heap 1 1 1 1 1

is-empty 1 1 1 1 1

insert 1 log n log n 1 1

delete-min n log n log n log n log n

decrease-key n log n log n 1 1

delete n log n log n log n log n

union 1 n log n 1 1

Lecture slides adapted from: find-min n 1 log n 1 1

• Chapter 20 of Introduction to Algorithms by Cormen, Leiserson, Rivest, and Stein. n = number of elements in † amortized • Chapter 9 of The Design and Analysis of Algorithms by Dexter Kozen.

Theorem. Starting from empty , any sequence of

a1 insert, a2 delete-min, and a3 decrease-key operations takes

O(a1 + a2 log n + a3) time. COS 423 Theory of Algorithms • Kevin Wayne • Spring 2007 2

Priority Queues Performance Cost Summary Fibonacci Heaps

Linked Binary Binomial Fibonacci Relaxed History. [Fredman and Tarjan, 1986] Operation List Heap Heap Heap † Heap ! Ingenious and analysis.

make-heap 1 1 1 1 1 ! Original motivation: improve Dijkstra's shortest path algorithm is-empty 1 1 1 1 1 from O(E log V ) to O(E + V log V ). V insert, V delete-min, E decrease-key insert 1 log n log n 1 1

delete-min n log n log n log n log n Basic idea. decrease-key n log n log n 1 1 ! Similar to binomial heaps, but less rigid structure. delete n log n log n log n log n ! : eagerly consolidate trees after each insert. union 1 n log n 1 1

find-min n 1 log n 1 1

n = number of elements in priority queue † amortized

Hopeless challenge. O(1) insert, delete-min and decrease-key. Why? ! Fibonacci heap: lazily defer consolidation until next delete-min.

3 4 Fibonacci Heaps: Structure Fibonacci Heaps: Structure

each parent larger than its children Fibonacci heap. Fibonacci heap.

! of heap-ordered trees. ! Set of heap-ordered trees.

! Maintain pointer to minimum element. ! Maintain pointer to minimum element.

! Set of marked nodes. ! Set of marked nodes. find-min takes O(1) time

roots heap-ordered min

17 24 23 7 3 17 24 23 7 3

30 26 46 30 26 46 18 52 41 18 52 41

Heap H 35 Heap H 35 39 44 39 44

5 6

Fibonacci Heaps: Structure Fibonacci Heaps: Notation

Fibonacci heap. Notation.

! Set of heap-ordered trees. ! n = number of nodes in heap.

! Maintain pointer to minimum element. ! rank(x) = number of children of node x.

! Set of marked nodes. ! rank(H) = max rank of any node in heap H.

! trees(H) = number of trees in heap H. use to keep heaps flat (stay tuned) ! marks(H) = number of marked nodes in heap H.

min trees(H) = 5 marks(H) = 3 n = 14 rank = 3 min

17 24 23 7 3 17 24 23 7 3

30 26 46 30 26 46 18 52 41 18 52 41

Heap H 35 marked Heap H 35 marked 39 44 39 44

7 8 Fibonacci Heaps: Potential Function

Insert !(H) !=!trees(H) + 2 " marks(H)

potential of heap H

trees(H) = 5 marks(H) = 3 !(H) = 5 + 2"3 = 11 min

17 24 23 7 3

30 26 46 18 52 41

Heap H 35 marked 39 44

9 10

Fibonacci Heaps: Insert Fibonacci Heaps: Insert

Insert. Insert.

! Create a new singleton tree. ! Create a new singleton tree.

! Add to root list; update min pointer (if necessary). ! Add to root list; update min pointer (if necessary).

insert 21 insert 21

21 min min

17 24 23 7 3 17 24 23 7 21 3

30 26 46 30 26 46 18 52 41 18 52 41

Heap H 35 Heap H 35 39 44 39 44

11 12 Fibonacci Heaps: Insert Analysis

Actual cost. O(1) !(H) !=!trees(H) + 2 " marks(H) Delete Min

Change in potential. +1 potential of heap H

Amortized cost. O(1)

min

17 24 23 7 21 3

30 26 46 18 52 41

Heap H 35 39 44

13 14

Linking Operation Fibonacci Heaps: Delete Min

Linking operation. Make larger root be a child of smaller root. Delete min.

! Delete min; meld its children into root list; update min.

! Consolidate trees so that no two roots have same rank.

larger root smaller root still heap-ordered

min 15 3 3

7 24 23 17 3 56 24 18 52 41 15 18 52 41

77 39 44 56 24 39 44 30 26 46 18 52 41 tree T tree T 1 2 77 35 39 44 tree T'

15 16 Fibonacci Heaps: Delete Min Fibonacci Heaps: Delete Min

Delete min. Delete min.

! Delete min; meld its children into root list; update min. ! Delete min; meld its children into root list; update min.

! Consolidate trees so that no two roots have same rank. ! Consolidate trees so that no two roots have same rank.

min min current

7 24 23 17 18 52 41 7 24 23 17 18 52 41

30 26 46 39 44 30 26 46 39 44

35 35

17 18

Fibonacci Heaps: Delete Min Fibonacci Heaps: Delete Min

Delete min. Delete min.

! Delete min; meld its children into root list; update min. ! Delete min; meld its children into root list; update min.

! Consolidate trees so that no two roots have same rank. ! Consolidate trees so that no two roots have same rank.

rank rank 0 1 2 3 0 1 2 3

min min current current

7 24 23 17 18 52 41 7 24 23 17 18 52 41

30 26 46 39 44 30 26 46 39 44

35 35

19 20 Fibonacci Heaps: Delete Min Fibonacci Heaps: Delete Min

Delete min. Delete min.

! Delete min; meld its children into root list; update min. ! Delete min; meld its children into root list; update min.

! Consolidate trees so that no two roots have same rank. ! Consolidate trees so that no two roots have same rank.

rank rank 0 1 2 3 0 1 2 3

min min

7 24 23 17 18 52 41 7 24 23 17 18 52 41

30 26 46 current 39 44 30 26 46 current 39 44

35 35 link 23 into 17

21 22

Fibonacci Heaps: Delete Min Fibonacci Heaps: Delete Min

Delete min. Delete min.

! Delete min; meld its children into root list; update min. ! Delete min; meld its children into root list; update min.

! Consolidate trees so that no two roots have same rank. ! Consolidate trees so that no two roots have same rank.

rank rank 0 1 2 3 0 1 2 3

min current

min 7 24 17 18 52 41 24 7 18 52 41

30 26 46 current 23 39 44 26 46 17 30 39 44

35 35 23 link 17 into 7 link 24 into 7

23 24 Fibonacci Heaps: Delete Min Fibonacci Heaps: Delete Min

Delete min. Delete min.

! Delete min; meld its children into root list; update min. ! Delete min; meld its children into root list; update min.

! Consolidate trees so that no two roots have same rank. ! Consolidate trees so that no two roots have same rank.

rank rank 0 1 2 3 0 1 2 3

current current

min min 7 18 52 41 7 18 52 41

24 17 30 39 44 24 17 30 39 44

26 46 23 26 46 23

35 35 25 26

Fibonacci Heaps: Delete Min Fibonacci Heaps: Delete Min

Delete min. Delete min.

! Delete min; meld its children into root list; update min. ! Delete min; meld its children into root list; update min.

! Consolidate trees so that no two roots have same rank. ! Consolidate trees so that no two roots have same rank.

rank rank 0 1 2 3 0 1 2 3

current current

min min 7 18 52 41 7 18 52 41

24 17 30 39 44 24 17 30 39 44

26 46 23 26 46 23 link 41 into 18 35 35 27 28 Fibonacci Heaps: Delete Min Fibonacci Heaps: Delete Min

Delete min. Delete min.

! Delete min; meld its children into root list; update min. ! Delete min; meld its children into root list; update min.

! Consolidate trees so that no two roots have same rank. ! Consolidate trees so that no two roots have same rank.

rank rank 0 1 2 3 0 1 2 3

current current

min min 7 52 18 7 52 18

24 17 30 41 39 24 17 30 41 39

26 46 23 44 26 46 23 44

35 35 29 30

Fibonacci Heaps: Delete Min Fibonacci Heaps: Delete Min Analysis

Delete min. Delete min. !(H) !=!trees(H) + 2 " marks(H) ! Delete min; meld its children into root list; update min. potential function ! Consolidate trees so that no two roots have same rank.

Actual cost. O(rank(H)) + O(trees(H))

! O(rank(H)) to meld min's children into root list.

! O(rank(H)) + O(trees(H)) to update min.

! O(rank(H)) + O(trees(H)) to consolidate trees.

min 7 52 18 Change in potential. O(rank(H)) - trees(H) ! trees(H' ) # rank(H) + 1 since no two trees have same rank.

! $!(H) # rank(H) + 1 - trees(H). 24 17 30 41 39 Amortized cost. O(rank(H))

26 46 23 44 stop 35 31 32 Fibonacci Heaps: Delete Min Analysis

Q. Is amortized cost of O(rank(H)) good? Decrease Key

A. Yes, if only insert and delete-min operations.

! In this case, all trees are binomial trees.

! This implies rank(H) # lg n. we only link trees of equal rank

B0 B1 B2 B3

A. Yes, we'll implement decrease-key so that rank(H) = O(log n).

33 34

Fibonacci Heaps: Decrease Key Fibonacci Heaps: Decrease Key

Intuition for deceasing the key of node x. Case 1. [heap order not violated]

! If heap-order is not violated, just decrease the key of x. ! Decrease key of x.

! Otherwise, cut tree rooted at x and meld into root list. ! Change heap min pointer (if necessary).

! To keep trees flat: as soon as a node has its second child cut, cut it off and meld into root list (and unmark it).

min min

7 18 38 7 18 38

marked node: one child already cut 24 17 23 21 39 41 24 17 23 21 39 41

26 46 30 52 26 4269 30 52

x

35 88 72 35 88 72 decrease-key of x from 46 to 29 35 36 Fibonacci Heaps: Decrease Key Fibonacci Heaps: Decrease Key

Case 1. [heap order not violated] Case 2a. [heap order violated]

! Decrease key of x. ! Decrease key of x.

! Change heap min pointer (if necessary). ! Cut tree rooted at x, meld into root list, and unmark.

! If parent p of x is unmarked (hasn't yet lost a child), mark it; Otherwise, cut p, meld into root list, and unmark (and do so recursively for all ancestors that lose a second child).

min min

7 18 38 7 18 38

24 17 23 21 39 41 24 17 23 21 39 41 p

26 29 30 52 26 2195 30 52 x x

35 88 72 decrease-key of x from 46 to 29 35 88 72 decrease-key of x from 29 to 15 37 38

Fibonacci Heaps: Decrease Key Fibonacci Heaps: Decrease Key

Case 2a. [heap order violated] Case 2a. [heap order violated]

! Decrease key of x. ! Decrease key of x.

! Cut tree rooted at x, meld into root list, and unmark. ! Cut tree rooted at x, meld into root list, and unmark.

! If parent p of x is unmarked (hasn't yet lost a child), mark it; ! If parent p of x is unmarked (hasn't yet lost a child), mark it; Otherwise, cut p, meld into root list, and unmark Otherwise, cut p, meld into root list, and unmark (and do so recursively for all ancestors that lose a second child). (and do so recursively for all ancestors that lose a second child).

min x min

7 18 38 15 7 18 38

24 17 23 21 39 41 72 24 17 23 21 39 41 p p

26 15 30 52 26 30 52

x

35 88 72 decrease-key of x from 29 to 15 35 88 decrease-key of x from 29 to 15 39 40 Fibonacci Heaps: Decrease Key Fibonacci Heaps: Decrease Key

Case 2a. [heap order violated] Case 2b. [heap order violated]

! Decrease key of x. ! Decrease key of x.

! Cut tree rooted at x, meld into root list, and unmark. ! Cut tree rooted at x, meld into root list, and unmark.

! If parent p of x is unmarked (hasn't yet lost a child), mark it; ! If parent p of x is unmarked (hasn't yet lost a child), mark it; Otherwise, cut p, meld into root list, and unmark Otherwise, cut p, meld into root list, and unmark (and do so recursively for all ancestors that lose a second child). (and do so recursively for all ancestors that lose a second child).

x min min

15 7 18 38 15 7 18 38

72 24 17 23 21 39 41 72 24 17 23 21 39 41 p mark parent

26 30 52 p 26 30 52

decrease-key of x from 29 to 15 decrease-key of x from 35 to 5 35 88 x 355 88 41 42

Fibonacci Heaps: Decrease Key Fibonacci Heaps: Decrease Key

Case 2b. [heap order violated] Case 2b. [heap order violated]

! Decrease key of x. ! Decrease key of x.

! Cut tree rooted at x, meld into root list, and unmark. ! Cut tree rooted at x, meld into root list, and unmark.

! If parent p of x is unmarked (hasn't yet lost a child), mark it; ! If parent p of x is unmarked (hasn't yet lost a child), mark it; Otherwise, cut p, meld into root list, and unmark Otherwise, cut p, meld into root list, and unmark (and do so recursively for all ancestors that lose a second child). (and do so recursively for all ancestors that lose a second child).

min min x

15 7 18 38 15 5 7 18 38

72 24 17 23 21 39 41 72 24 17 23 21 39 41

p 26 30 52 p 26 30 52

decrease-key of x from 35 to 5 decrease-key of x from 35 to 5 x 5 88 88 43 44 Fibonacci Heaps: Decrease Key Fibonacci Heaps: Decrease Key

Case 2b. [heap order violated] Case 2b. [heap order violated]

! Decrease key of x. ! Decrease key of x.

! Cut tree rooted at x, meld into root list, and unmark. ! Cut tree rooted at x, meld into root list, and unmark.

! If parent p of x is unmarked (hasn't yet lost a child), mark it; ! If parent p of x is unmarked (hasn't yet lost a child), mark it; Otherwise, cut p, meld into root list, and unmark Otherwise, cut p, meld into root list, and unmark (and do so recursively for all ancestors that lose a second child). (and do so recursively for all ancestors that lose a second child).

min min x x p

15 5 7 18 38 15 5 26 7 18 38

72 24 17 23 21 39 41 72 88 24 17 23 21 39 41 second child cut

p 26 30 52 30 52

88 decrease-key of x from 35 to 5 decrease-key of x from 35 to 5 45 46

Fibonacci Heaps: Decrease Key Fibonacci Heaps: Decrease Key

Case 2b. [heap order violated] Case 2b. [heap order violated]

! Decrease key of x. ! Decrease key of x.

! Cut tree rooted at x, meld into root list, and unmark. ! Cut tree rooted at x, meld into root list, and unmark.

! If parent p of x is unmarked (hasn't yet lost a child), mark it; ! If parent p of x is unmarked (hasn't yet lost a child), mark it; Otherwise, cut p, meld into root list, and unmark Otherwise, cut p, meld into root list, and unmark (and do so recursively for all ancestors that lose a second child). (and do so recursively for all ancestors that lose a second child).

min min x p x p p' p''

15 5 26 7 18 38 15 5 26 24 7 18 38

72 88 p' 24 17 23 21 39 41 72 88 don't mark 17 23 21 39 41 parent if it's a root

second child cut 30 52 30 52

decrease-key of x from 35 to 5 decrease-key of x from 35 to 5

47 48 Fibonacci Heaps: Decrease Key Analysis

Decrease-key. !(H) !=!trees(H) + 2 " marks(H) Analysis

potential function

Actual cost. O(c)

! O(1) time for changing the key.

! O(1) time for each of c cuts, plus melding into root list.

Change in potential. O(1) - c

! trees(H') = trees(H) + c.

! marks(H') # marks(H) - c + 2.

! $! # c + 2 " (-c + 2) = 4 - c.

Amortized cost. O(1)

49 50

Analysis Summary Fibonacci Heaps: Bounding the Rank

Insert. O(1) Lemma. Fix a point in time. Let x be a node, and let y1, …, yk denote Delete-min. O(rank(H)) † its children in the order in which they were linked to x. Then: Decrease-key. O(1) † x † amortized $ 0 if i =1 rank (yi ) " % & i # 2 if i "1 … y1 y2 yk

Key lemma. rank(H) = O(log n). ! number of nodes is exponential in rank Pf.

! When yi was linked into x, x had at least i -1 children y1, …, yi-1.

! Since only trees of equal rank are linked, at that time

rank(yi)!= rank(xi) % i - 1.

! Since then, yi has lost at most one child. ! Thus, right now rank(yi) % i - 2. or yi would have been cut

51 52 Fibonacci Heaps: Bounding the Rank Fibonacci Heaps: Bounding the Rank

Lemma. Fix a point in time. Let x be a node, and let y1, …, yk denote Lemma. Fix a point in time. Let x be a node, and let y1, …, yk denote its children in the order in which they were linked to x. Then: its children in the order in which they were linked to x. Then:

x x $ 0 if i =1 $ 0 if i =1 rank (yi ) " % rank (yi ) " % & i # 2 if i "1 & i # 2 if i "1 … … y1 y2 yk y1 y2 yk

! !

Def. Let Fk be smallest possible tree of rank k satisfying property. Def. Let Fk be smallest possible tree of rank k satisfying property.

F0 F1 F2 F3 F4 F5 F4 F5 F6

1 2 3 5 8 13

8 13 8 + 13 = 21

53 54

Fibonacci Heaps: Bounding the Rank

Lemma. Fix a point in time. Let x be a node, and let y , …, y denote 1 k Fibonacci Numbers its children in the order in which they were linked to x. Then:

x $ 0 if i =1 rank (yi ) " % & i # 2 if i "1 … y1 y2 yk

!

Def. Let Fk be smallest possible tree of rank k satisfying property.

k Fibonacci fact. Fk % & , where & = (1 + '5) / 2 ( 1.618.

golden ratio Corollary. rank(H) # log& n .

55 56 Fibonacci Numbers: Exponential Growth Fibonacci Numbers and Nature

Def. The Fibonacci sequence is: 1, 2, 3, 5, 8, 13, 21, …

# 1 if k = 0 % slightly non-standard definition Fk = $ 2 if k =1 % & Fk-1 + Fk-2 if k " 2

Lemma. F % &k, where & = (1 + '5) / 2 ( 1.618. ! k

Pf. [by induction on k] pinecone ! Base cases: F0 = 1 % 1, F1 = 2 % &. k k + 1 ! Inductive hypotheses: Fk % & and Fk+1 % & cauliflower

Fk+2 = Fk + Fk+1 (definition) k k+1 " # + # (inductive hypothesis) k = # (1 + #) (algebra) k 2 = # (# ) (&2 = & + 1) k+2 = # (algebra)

57 58

!

Fibonacci Heaps: Union

Union Union. Combine two Fibonacci heaps.

Representation. Root lists are circular, doubly linked lists.

min min

23 24 17 7 3 21

30 26 46 18 52 41

Heap H' 35 Heap H'' 39 44

59 60 Fibonacci Heaps: Union Fibonacci Heaps: Union

Union. Combine two Fibonacci heaps. Actual cost. O(1) !(H) !=!trees(H) + 2 " marks(H)

Representation. Root lists are circular, doubly linked lists. Change in potential. 0 potential function

Amortized cost. O(1)

min min

23 24 17 7 3 21 23 24 17 7 3 21

30 26 46 30 26 46 18 52 41 18 52 41

35 Heap H 35 Heap H 39 44 39 44

61 62

Fibonacci Heaps: Delete

Delete node x. Delete !(H) !=!trees(H) + 2 " marks(H) ! decrease-key of x to -). potential function ! delete-min element in heap.

Amortized cost. O(rank(H))

! O(1) amortized for decrease-key.

! O(rank(H)) amortized for delete-min.

63 64