CSE 548: Analysis of Algorithms

Lectures 14 & 15 ( Dijkstra’s SSSP & Fibonacci Heaps )

Rezaul A. Chowdhury Department of Computer Science SUNY Stony Brook Fall 2012 Fibonacci Heaps ( Fredman & Tarjan, 1984 ) A Fibonacci can be viewed as an extension of Binomial heaps which supports DECREASE -KEY and DELETE operations efficiently.

Binary Heap Heap Operation ( worst-case ) ( amortized )

MAKE -HEAP Θ Θ 1 1 INSERT Ο Θ log 1 MINIMUM Θ Θ 1 1 EXTRACT -MIN Ο Ο log log UNION Θ Θ 1 DECREASE -KEY Ο log DELETE Ο log Fibonacci Heaps ( Fredman & Tarjan, 1984 ) A can be viewed as an extension of Binomial heaps which supports DECREASE -KEY and DELETE operations efficiently.

Binary Heap Binomial Heap Heap Operation ( worst-case ) ( amortized )

MAKE -HEAP Θ Θ 1 1 INSERT Ο Θ log 1 MINIMUM Θ Θ 1 1 EXTRACT -MIN Ο Ο log log UNION Θ Θ Ο 1 DECREASE -KEY Ο ( worstlog case ) log Ο DELETE Ο ( worstlog case ) log Fibonacci Heaps ( Fredman & Tarjan, 1984 ) A Fibonacci heap can be viewed as an extension of Binomial heaps which supports DECREASE -KEY and DELETE operations efficiently.

Binary Heap Binomial Heap Fibonacci Heap Heap Operation ( worst-case ) ( amortized ) ( amortized )

MAKE -HEAP Θ Θ Θ 1 1 1 INSERT Ο Θ Θ log 1 1 MINIMUM Θ Θ Θ 1 1 1 EXTRACT -MIN Ο Ο Ο log log log UNION Θ Θ Θ Ο 1 1 DECREASE -KEY Ο Θ ( worstlog case ) log 1 Ο DELETE Ο Ο ( worstlog case ) log log Dijkstra’s SSSP Algorithm with a Min-Heap ( SSSP: Single-Source Shortest Paths ) Input: Weighted graph with vertex and edge set , a weight function , and a source , vertex . Output: For all , is set to the ∈ shortest distance from to . ∈ .

Dijkstra-SSSP ( ) , , , 1. for each do 2. ∈ . ← ∞ 3. . ← 0 { empty min-heap } ← 4. for each do INSERT ( ) 5. while ∈do , ∅ 6. EXTRACT -MIN ( ) 7. for ←each do ∈ 8. if then . . , 9. DECREASE -KEY ( ) , , . , 10. . ← . , Dijkstra’s SSSP Algorithm with a Min-Heap ( SSSP: Single-Source Shortest Paths ) Input: Weighted graph with vertex set and edge set , a weight function , and a source , vertex . Output: For all , is set to the ∈ shortest distance from to . ∈ .

Dijkstra-SSSP ( ) Let and , , , 1. for each do 2. ∈ . ← ∞ # INSERTS . ← 0 3. { empty min-heap } # EXTRACT -MINS ← 4. for each do INSERT ( ) # DECREASE -KEYS 5. while ∈do , ∅ 6. EXTRACT -MIN ( ) 7. for ←each do Total cost ∈ 8. if then . . , 9. DECREASE -KEY ( ) , , . , 10. . ← . , Dijkstra’s SSSP Algorithm with a Min-Heap ( SSSP: Single-Source Shortest Paths ) Input: Weighted graph with vertex set and edge set , a weight function , and a source , vertex . Output: For all , is set to the ∈ shortest distance from to . ∈ .

Dijkstra-SSSP ( ) Let and , , , 1. for each do 2. ∈ . ← ∞ For Binary Heap ( worst-case costs ): 3. . ← 0 { empty min-heap } ← 4. for each do INSERT ( ) Ο 5. while ∈do , logΟ ∅ 6. EXTRACT -MIN ( ) Οlog 7. for ←each do ∈ 8. if then log . . , 9. DECREASE -KEY ( ) Total cost ( worst-case ) , , . , 10. ∴ Ο . ← . , log Dijkstra’s SSSP Algorithm with a Min-Heap ( SSSP: Single-Source Shortest Paths ) Input: Weighted graph with vertex set and edge set , a weight function , and a source , vertex . Output: For all , is set to the ∈ shortest distance from to . ∈ .

Dijkstra-SSSP ( ) Let and , , , 1. for each do 2. ∈ . ← ∞ For Binomial Heap ( amortized costs ): 3. . ← 0 { empty min-heap } ← 4. for each do INSERT ( ) Ο 5. while ∈do , 1 Ο ∅ 6. EXTRACT -MIN ( ) Οlog 7. for ←each do ∈ ( worst-case ) 8. if then log . . , 9. DECREASE -KEY ( ) , , . , 10. Total cost ( worst-case ) . ← . , ∴ Ο log Dijkstra’s SSSP Algorithm with a Min-Heap ( SSSP: Single-Source Shortest Paths ) Input: Weighted graph with vertex set and edge set , a weight function , and a source , vertex . Output: For all , is set to the ∈ shortest distance from to . ∈ .

Dijkstra-SSSP ( ) Let and , , , 1. for each do Total cost 2. ∈ . ← ∞ 3. . ← 0 { empty min-heap } ← 4. for each do INSERT ( ) ∈ , 5. while do ∅ 6. EXTRACT -MIN ( ) Observation: 7. for ←each do Obtaining a worst-case bound for a ∈ 8. if then sequence of INSERTS , EXTRACT -MINS . . , 9. DECREASE -KEY ( ) , , . , and DECREASE -KEYS is enough. 10. . ← . , Amortized bound per operation is ∴sufficient. Dijkstra’s SSSP Algorithm with a Min-Heap ( SSSP: Single-Source Shortest Paths ) Input: Weighted graph with vertex set and edge set , a weight function , and a source , vertex . Output: For all , is set to the ∈ shortest distance from to . ∈ .

Dijkstra-SSSP ( ) Let and , , , 1. for each do Total cost 2. ∈ . ← ∞ 3. . ← 0 { empty min-heap } ← 4. for each do INSERT ( ) ∈ , 5. while do ∅ 6. EXTRACT -MIN ( ) Observation: 7. for ←each do For ∈ 8. if then the best possible bound is Θ . . . , 9. DECREASE -KEY ( ) , , . , ( else violates sorting lower bound log ) 10. , . ← . Perhaps can be improved to . o log Fibonacci Heaps from Binomial Heaps A Fibonacci heap can be viewed as an extension of Binomial heaps which supports DECREASE -KEY and DELETE operations efficiently. But the trees in a Fibonacci heap are no longer binomial trees as we will be cutting subtrees out of them.

However, all operations ( except DECREASE -KEY and DELETE ) are still performed in the same way as in binomial heaps. The rank of a is still defined as the number of children of the root, and we still link two trees if they have the same rank. Implementing DECREASE -KEY , , DECREASE -KEY ( ): One possible approach is to cut out the subtree rooted, at , from , reduce the value of to , and insert that subtree into the root list of . Problem: If we cut out a lot of subtrees from a tree its size will no longer be exponential in its rank. Since our analysis of EXTRACT -MIN in binomial heaps was highly dependent on this exponential relationship, that analysis will no longer hold. Solution: Limit #cuts among the children of any node to 2. We will show that the size of each tree will still remain exponential in its rank. When a 2nd child is cut from a node , we also cut from its parent leading to a possible sequence of cuts moving up towards the root. Analysis of Fibonacci Heap Operations

Recurrence for Fibonacci numbers : 0 0, 1 1, . We showed in a pervious lecture: , where and are the roots . 1 0 Analysis of Fibonacci Heap Operations

Lemma 1: For all integers , . 0 1 ∑ Proof: By induction on . Base case: . 1 1 0 1 1 ∑ Inductive hypothesis: for . 1 ∑ 0 1 Then . 1 ∑ 1 ∑ Analysis of Fibonacci Heap Operations

Lemma 2: For all integers , . 0 Proof: By induction on . Base case: and . 1 2 Inductive hypothesis: for . 0 1 Then 1 Analysis of Fibonacci Heap Operations Lemma 3: Let be any node in a Fibonacci heap, and suppose that . Let be the children of in the order in which they were linked, to, … , from the earliest to the latest. Then for . max 0, 2 1 Proof: Obviously, . For , when was linked to 0 , all of were children of . So, 1 . , , … , 1 Because is linked to only if , we must have had at that time. 1 Since then, has lost at most one child, and hence . 2 Analysis of Fibonacci Heap Operations Lemma 4: Let be any node in a Fibonacci heap with and . Then . log Proof: Let be the minimum possible size of any node of rank in any Fibonacci heap. Trivially, and . 1 2 Since adding children to a node cannot decrease its size, increases monotonically with . Let be a node in any Fibonacci heap with and . Analysis of Fibonacci Heap Operations Lemma 4: Let be any node in a Fibonacci heap with and . Then . log Proof ( continued ): Let be the children of in the order in which they were linked to, ,, …from , the earliest to the latest. Then 1 ∑ 1 ∑ , 2 ∑ We now show by induction on that for all integer . 0 Base case: and . 1 2 Inductive hypothesis: for . 0 1 Then 2 ∑ 2 ∑ 1 ∑ . Hence ⇒ log . Analysis of Fibonacci Heap Operations Corollary: The maximum degree of any node in an node Fibonacci heap is Ο . log Proof: Let be any node in the heap. Then from Lemma 4, Ο log log log . Analysis of Fibonacci Heap Operations All nodes are initially unmarked. We mark a node when − it loses its first child We unmark a node when − it loses its second child, or − becomes the child of another node ( e.g., LINK ed )

We extend the potential function used for binomial heaps: , Φ 2 3 where is the state of the after the operation, is the number of trees in the root list, and is the number of marked nodes. Analysis of Fibonacci Heap Operations We extend the potential function used for binomial heaps: , Φ 2 3 where is the state of the data structure after the operation, is the number of trees in the root list, and is the number of marked nodes. DECREASE -KEY ( ): Let #cascading cuts performed. , , Then the actual cost of cutting the tree rooted at is 1, and the actual cost of each of the cascading cuts is also 1.

overall actual cost, ∴ 1 Fibonacci Heaps from Binomial Heaps Potential function: DECREASE -KEY ( Φ ): 2 3 , , New trees: 1 tree rooted at , and 1 tree produced by each of the cascading cuts.

Marked∴ nodes: 1 node 1 unmarked by each cascading cut, and at most 1 node marked by the last cut/cascading cut.

Potential∴ drop, 1 Δ Φ Φ 2 3 2 1 3 1 5 Fibonacci Heaps from Binomial Heaps Potential function: DECREASE -KEY ( Φ ): 2 3 , , Amortized cost, ̂ Δ 1 5 6Ο 1 Fibonacci Heaps from Binomial Heaps Potential function: EXTRACT -MIN ( ): Φ 2 3 Let be the max degree of any node in an -node Fibonacci heap. Cost of creating the array of pointers is . Suppose we start with trees in the doubly linked1 list, and perform link operations during the conversion from to array version. So we perform work, and end up with trees. Cost of converting to the linked list version is . actual cost, Since no node is marked, 1 and each link reduces 2the #trees by 1 1, potential change, Δ Φ Φ 2 Fibonacci Heaps from Binomial Heaps Potential function: EXTRACT -MIN ( ): Φ 2 3 actual cost, 1 2 1 potential change, Δ Φ Φ 2 amortized cost, ̂ Δ 2 1 But ( as we have at most one tree of each rank ) 1 So, Ο . ̂ 3 3 log Fibonacci Heaps from Binomial Heaps Potential function: DELETE ( ): Φ 2 3

,STEP 1: DECREASE -KEY ( ) STEP 2: EXTRACT -MIN ( ,) , ∞ amortized cost, amortized cost of DECREASE -KEY ̂ amortized cost of EXTRACT -MIN Ο Ο Ο 1 log log