Chapter 4 Data Structures

Algorithm Theory WS 2012/13

Fabian Kuhn Examples

Dictionary: • Operations: insert(key,value), delete(key), find(key) • Implementations: – : all operations take time (: size of ) – Balanced binary : all operations take log time – : all operations take 1 times (with some assumptions) Stack (LIFO Queue): • Operations: push, pull • Linked list: 1 for both operations (FIFO) Queue: • Operations: enqueue, dequeue • Linked list: 1 time for both operations Here: Priority Queues (heaps), Union‐Find data structure Algorithm Theory, WS 2012/13 Fabian Kuhn 2 Dijkstra’s Algorithm

Single‐Source : • Given: graph ,with edge weights 0for ∈ source node ∈ • Goal: compute shortest paths from to all ∈

Dijkstra’s Algorithm: 1. Initialize , 0 and , ∞ for all 2. All nodes are unmarked 3. Get unmarked node which minimizes , : 4. For all , ∈ , , min , , , 5. mark node 6. Until all nodes are marked

Algorithm Theory, WS 2012/13 Fabian Kuhn 3 Example

1 ∞ 32 ∞ ∞ 3 9 10 4 23 13 ∞ ∞ 3 6 2 2 ∞ 1 ∞ 3 17 9 19 ∞ 8 1 2 20 ∞ 18 ∞ 1

Algorithm Theory, WS 2012/13 Fabian Kuhn 4 Example

1 ∞ 32 ∞ ∞ 3 9 10 4 23 13 ∞ 3 6 2 2 ∞ 1 3 17 9 19 8 1 2 20 18 1

Algorithm Theory, WS 2012/13 Fabian Kuhn 5 Example

1 ∞ 32 ∞ 3 9 10 4 23 13 ∞ 3 6 2 2 ∞ 1 3 17 9 19 8 1 2 20 18 1

Algorithm Theory, WS 2012/13 Fabian Kuhn 6 Example

1 32 ∞ 3 9 10 4 23 13 3 6 2 2 ∞ 1 3 17 9 19 8 1 2 20 18 1

Algorithm Theory, WS 2012/13 Fabian Kuhn 7 Example

1 32 3 9 10 4 23 13 3 6 2 2 ∞ 1 3 17 9 19 8 1 2 20 18 1

Algorithm Theory, WS 2012/13 Fabian Kuhn 8 Example

1 32 3 9 10 4 23 13 3 6 2 2 ∞ 1 3 17 9 19 8 1 2 20 18 1

Algorithm Theory, WS 2012/13 Fabian Kuhn 9 Example

1 32 3 9 10 4 23 13 3 6 2 2 1 3 17 9 19 8 1 2 20 18 1

Algorithm Theory, WS 2012/13 Fabian Kuhn 10 Example

1 32 3 9 10 4 23 13 3 6 2 2 1 3 17 9 19 8 1 2 20 18 1

Algorithm Theory, WS 2012/13 Fabian Kuhn 11 Implementation of Dijkstra’s Algorithm

Dijkstra’s Algorithm: 1. Initialize , 0 and , ∞ for all 2. All nodes are unmarked

3. Get unmarked node which minimizes , :

4. For all , ∈ , , min , , ,

5. mark node

6. Until all nodes are marked

Algorithm Theory, WS 2012/13 Fabian Kuhn 12 /

• Stores (key,data) pairs (like dictionary) • But, different of operations:

• Initialize‐Heap: creates new empty heap • Is‐Empty: returns true if heap is empty • Insert(key,data): inserts (key,data)‐pair, returns pointer to entry • Get‐Min: returns (key,data)‐pair with minimum key • Delete‐Min: deletes minimum (key,data)‐pair • Decrease‐Key(entry,newkey): decreases key of entry to newkey • Merge: merges two heaps into one

Algorithm Theory, WS 2012/13 Fabian Kuhn 13 Implementation of Dijkstra’s Algorithm

Store nodes in a priority queue, use , as keys: 1. Initialize , 0 and , ∞ for all 2. All nodes are unmarked

3. Get unmarked node which minimizes , :

4. mark node

5. For all , ∈ , , min , , ,

6. Until all nodes are marked

Algorithm Theory, WS 2012/13 Fabian Kuhn 14 Analysis

Number of priority queue operations for Dijkstra: • Initialize‐Heap:

• Is‐Empty: ||

• Insert: ||

• Get‐Min: ||

• Delete‐Min: ||

• Decrease‐Key: ||

• Merge:

Algorithm Theory, WS 2012/13 Fabian Kuhn 15 Priority Queue Implementation

Implementation as min‐heap:

 complete , e.g., stored in an array

• Initialize‐Heap: • Is‐Empty: • Insert: • Get‐Min: • Delete‐Min: • Decrease‐Key: • Merge (heaps of size and , ): Algorithm Theory, WS 2012/13 Fabian Kuhn 16 Better Implementation

• Can we do better?

• Cost of Dijkstra with complete binary min‐heap implementation: log

• Can be improved if we can make decrease‐key cheaper…

• Cost of merging two heaps is expensive

• We will get there in two steps:

Binomial heap  Fibonacci heap

Algorithm Theory, WS 2012/13 Fabian Kuhn 17 Definition: Binomial Tree

Binomial tree of order 0:

Algorithm Theory, WS 2012/13 Fabian Kuhn 18 Binomial Trees

Algorithm Theory, WS 2012/13 Fabian Kuhn 19 Properties

1. Tree has 2 nodes

2. Height of tree is

3. Root degree of is

4. In , there are exactly nodes at depth

Algorithm Theory, WS 2012/13 Fabian Kuhn 20 Binomial Coefficients

• Binomial coefficient: : # of element subsets of a set of size

• Property: Pascal triangle:

Algorithm Theory, WS 2012/13 Fabian Kuhn 21 Number of Nodes at Depth in

Claim: In , there are exactly nodes at depth

Algorithm Theory, WS 2012/13 Fabian Kuhn 22

• Keys are stored in nodes of binomial trees of different order

nodes: there is a binomial tree or order iff bit of base‐2 representation of is 1.

• Min‐Heap Property: Key of node keys of all nodes in sub‐tree of

Algorithm Theory, WS 2012/13 Fabian Kuhn 23 Example

• 10 keys: 2, 5, 8, 9, 12, 14, 17, 18, 20, 22, 25

• Binary representation of : 11 1011  trees , , and present

5 2 17

9 14 20 8

12 18 25

22

Algorithm Theory, WS 2012/13 Fabian Kuhn 24 Child‐Sibling Representation

Structure of a node: parent key degree child sibling

Algorithm Theory, WS 2012/13 Fabian Kuhn 25 Link Operation

• Unite two binomial trees of the same order to one tree:  12 ⨁ ⇒ 

• Time:  15 20 18  ⨁

25 40 22  

30 

Algorithm Theory, WS 2012/13 Fabian Kuhn 26 Merge Operation

Merging two binomial heaps: • For ,,…,: If there are 2 or 3 binomial trees : apply link operation to merge 2 trees into one binomial tree

Time:

∪

Algorithm Theory, WS 2012/13 Fabian Kuhn 27 Example

9 13 5 2 17

12 18 14 20 8

22 25

Algorithm Theory, WS 2012/13 Fabian Kuhn 28 Operations

Initialize: create empty list of trees

Get minimum of queue: time 1 (if we maintain a pointer)

Decrease‐key at node : • Set key of node to new key • Swap with parent until min‐heap property is restored • Time: log

Insert key into queue : 1. Create queue of size 1 containing only 2. Merge and • Time for insert: log

Algorithm Theory, WS 2012/13 Fabian Kuhn 29 Operations

Delete‐Min Operation:

1. Find tree with minimum root

2. Remove from queue  queue ′

3. Children of form new queue ′′

4. Merge queues ′ and ′′

• Overall time:

Algorithm Theory, WS 2012/13 Fabian Kuhn 30 Delete‐Min Example

2 5 17

9 14 20 8

12 18 25

22

Algorithm Theory, WS 2012/13 Fabian Kuhn 31 Complexities Binomial Heap

• Initialize‐Heap: • Is‐Empty: • Insert: • Get‐Min: • Delete‐Min: • Decrease‐Key: • Merge (heaps of size and , ):

Algorithm Theory, WS 2012/13 Fabian Kuhn 32 Can We Do Better?

• Binomial heap: insert, delete‐min, and decrease‐key cost log

• One of the operations insert or delete‐min must cost Ωlog : – Heap‐Sort: Insert elements into heap, then take out the minimum times – (Comparison‐based) sorting costs at least Ω log .

• But maybe we can improve decrease‐key and one of the other two operations?

• Structure of binomial heap is not flexible: – Simplifies analysis, allows to get strong worst‐case bounds – But, operations almost inherently need at least logarithmic time

Algorithm Theory, WS 2012/13 Fabian Kuhn 33 Fibonacci Heaps

Lacy‐merge variant of binomial heaps: • Do not merge trees as long as possible…

Structure: A Fibonacci heap consists of a collection of trees satisfying the min‐heap property.

Variables: • . : root of the tree containing the (a) minimum key • . : circular, doubly linked, unordered list containing the roots of all trees • . : number of nodes currently in

Algorithm Theory, WS 2012/13 Fabian Kuhn 34 Trees in Fibonacci Heaps

Structure of a single node :

parent right left key degree

child mark

• . : points to circular, doubly linked and unordered list of the children of • . , . : pointers to siblings (in doubly linked list) • . : will be used later… Advantages of circular, doubly linked lists: • Deleting an element takes constant time • Concatenating two lists takes constant time Algorithm Theory, WS 2012/13 Fabian Kuhn 35 Example

Figure: Cormen et al., Introduction to Algorithms

Algorithm Theory, WS 2012/13 Fabian Kuhn 36 Simple (Lazy) Operations

Initialize‐Heap : • . ≔ . ≔

Merge heaps and ′: • concatenate root lists • update .

Insert element into : • create new one‐node tree containing  H′ • merge heaps and ′

Get minimum element of : • return .

Algorithm Theory, WS 2012/13 Fabian Kuhn 37 Operation Delete‐Min

Delete the node with minimum key from and return its element:

1. ≔.; 2. if . 0 then 3. remove . from . ; 4. add . . (list) to . 5. . ;

// Repeatedly merge nodes with equal degree in the root list // until degrees of nodes in the root list are distinct. // Determine the element with minimum key

6. return

Algorithm Theory, WS 2012/13 Fabian Kuhn 38 Rank and Maximum Degree

Ranks of nodes, trees, heap:

Node : • : degree of

Tree : • : rank (degree) of root node of

Heap : • : maximum degree of any node in

Assumption (: number of nodes in ): – for a known function

Algorithm Theory, WS 2012/13 Fabian Kuhn 39 Merging Two Trees

Given: Heap‐ordered trees , ′ with • Assume: min‐key of min‐key of ′

Operation , : • Removes tree ′ from root list ′ and adds to child list of

• ≔1 • .≔ ′

Algorithm Theory, WS 2012/13 Fabian Kuhn 40 Consolidation of Root List

Array pointing to find roots with the same rank:

012 Consolidate: Time: 1. for ≔0to do ≔null; |.|. 2. while . null do 3. ≔“delete and return first element of . ” 4. while nulldo 5. ≔ ; 6. ≔; 7. ≔, 8. ≔ 9. Create new . and . Algorithm Theory, WS 2012/13 Fabian Kuhn 41 Consolidate Example

31 5 25 1 2 15 17 3

9 14 20 13 8 19 7

12 18 22

5 25 1 2 15 17 3

9 14 20 31 13 8 19 7

12 18 22

Algorithm Theory, WS 2012/13 Fabian Kuhn 42 Consolidate Example

5 25 1 2 15 17 3

9 14 20 31 13 8 19 7

12 18 22

5 1 2 15 17 3

9 14 20 25 13 8 19 7

12 18 22 31

Algorithm Theory, WS 2012/13 Fabian Kuhn 43 Consolidate Example

5 1 2 15 17 3

9 14 20 25 13 8 19 7

12 18 22 31

5 1 2 15 17

9 14 20 25 13 3 8

12 18 22 31 19 7

Algorithm Theory, WS 2012/13 Fabian Kuhn 44 Consolidate Example

5 1 2 15 17

9 14 20 25 13 3 8

12 18 22 31 19 7

1 2 15 17

5 25 13 3 8

9 14 20 31 19 7

12 18 22

Algorithm Theory, WS 2012/13 Fabian Kuhn 45 Consolidate Example

1 2 15 17

5 25 13 3 8

9 14 20 31 19 7

12 18 22 1 2 15

5 25 13 3 8 17

9 14 20 31 19 7

12 18 22

Algorithm Theory, WS 2012/13 Fabian Kuhn 46 Consolidate Example

1 2 15

5 25 13 3 8 17

9 14 20 31 19 7

12 18 22 1 2

5 25 13 3 8 15

9 14 20 31 19 7 17

12 18 22

Algorithm Theory, WS 2012/13 Fabian Kuhn 47 Operation Decrease‐Key

Decrease‐Key, : (decrease key of node to new value )

1. if .then return; 2. . ≔ ; update . ; 3. if ∈ . ∨ . . then return 4. repeat 5. ≔ . ; 6. . ; 7. ≔; 8. until . ∨ ∈ . ; 9. if ∉ . then . ≔ ;

Algorithm Theory, WS 2012/13 Fabian Kuhn 48 Operation Cut( )

Operation . : • Cuts ’s sub‐tree from its parent and adds to rootlist

1. if ∉ . then 2. // cut the link between and its parent 3. . ≔ . 1; 4. remove from . . (list) 5. . ≔ null; 6. add to . 1 2 15 1 3 2 15 25 13 3 8 25 13 19 7 8

31 19 7 31 Algorithm Theory, WS 2012/13 Fabian Kuhn 49 Decrease‐Key Example

• Green nodes are marked 5 1 2 15 17

9 14 20 25 13 3 8

12 18 31 19 7 Decrease‐Key, 22

5 18 9 1 2 15 17

14 20 22 12 25 13 3 8

31 19 7

Algorithm Theory, WS 2012/13 Fabian Kuhn 50 Fibonacci Heap Marks

History of a node :

is being linked to a node . ≔ a child of is cut . ≔ a second child of is cut .

• Hence, the boolean value . indicates whether node has lost a child since the last time was made the child of another node.

Algorithm Theory, WS 2012/13 Fabian Kuhn 51 Cost of Delete‐Min & Decrease‐Key

Delete‐Min: 1. Delete min. root and add . to . time: 1 2. Consolidate . time: length of . • Step 2 can potentially be linear in (size of ) Decrease‐Key (at node ): 1. If new key parent key, cut sub‐tree of node time: 1 2. Cascading cuts up the tree as long as nodes are marked time: number of consecutive marked nodes • Step 2 can potentially be linear in

Exercises: Both operations can take time in the worst case!

Algorithm Theory, WS 2012/13 Fabian Kuhn 52 Cost of Delete‐Min & Decrease‐Key

• Cost of delete‐min and decrease‐key can be Θ… – Seems a large price to pay to get insert and merge in 1 time

• Maybe, the operations are efficient most of the time? – It seems to require a lot of operations to get a long rootlist and thus, an expensive consolidate operation – In each decrease‐key operation, at most one node gets marked: We need a lot of decrease‐key operations to get an expensive decrease‐key operation

• Can we show that the average cost per operation is small?

• We can  requires

Algorithm Theory, WS 2012/13 Fabian Kuhn 53 Amortization

• Consider sequence ,,…, of operations (typically performed on some data structure )

• : execution time of operation • ≔ ⋯: total execution time

• The execution time of a single operation might

vary within a large range (e.g., ∈1, )

• The worst case overall execution time might still be small  average execution time per operation might be small in the worst case, even if single operations can be expensive

Algorithm Theory, WS 2012/13 Fabian Kuhn 54 Analysis of Algorithms

• Best case

• Worst case

• Average case

• Amortized worst case

What it the average cost of an operation in a worst case sequence of operations?

Algorithm Theory, WS 2012/13 Fabian Kuhn 55 Example: Binary Counter

Incrementing a binary counter: determine the bit flip cost: Operation Counter Value Cost 00000 1 00001 1 2 00010 2 3 00011 1 400100 3 5 00101 1 6 00110 2 7 00111 1 801000 4 9 01001 1 10 01010 2 11 01011 1 12 01100 3 13 01101 1

Algorithm Theory, WS 2012/13 Fabian Kuhn 56 Accounting Method

Observation: • Each increment flips exactly one 0 into a 1 001001111 ⟹ 001000000 Idea: • Have a bank account (with initial amount 0) • Paying to the bank account costs • Take “money” from account to pay for expensive operations

Applied to binary counter: • Flip from 0 to 1: pay 1 to bank account (cost: 2) • Flip from 1 to 0: take 1 from bank account (cost: 0) • Amount on bank account = number of ones  We always have enough “money” to pay!

Algorithm Theory, WS 2012/13 Fabian Kuhn 57 Accounting Method

Op. Counter Cost To Bank From Bank Net Cost Credit 0 0 0 0 0 10 0 0 0 1 1 20 0 0 1 0 2 30 0 0 1 1 1 40 0 1 0 0 3 50 0 1 0 1 1 60 0 1 1 0 2 70 0 1 1 1 1 80 1 0 0 0 4 90 1 0 0 1 1 10 0 1 0 1 0 2

Algorithm Theory, WS 2012/13 Fabian Kuhn 58 Potential Function Method

• Most generic and elegant way to do amortized analysis! – But, also more abstract than the others…

• State of data structure / system: ∈(state space)

Potential function : →

• Operation :

– : actual cost of operation – : state after execution of operation (: initial state) – ≔Φ: potential after exec. of operation – : amortized cost of operation :

≔

Algorithm Theory, WS 2012/13 Fabian Kuhn 59 Potential Function Method

Operation :

actual cost: amortized cost: Φ Φ Overall cost:

≔ Φ Φ

Algorithm Theory, WS 2012/13 Fabian Kuhn 60 Binary Counter:

• Potential function: :

• Clearly, Φ 0and Φ 0for all 0

• Actual cost : . 1 flip from 0 to 1

. 1flips from 1 to 0

• Potential difference: Φ Φ 1 1 2

• Amortized cost: Φ Φ 2

Algorithm Theory, WS 2012/13 Fabian Kuhn 61 Back to Fibonacci Heaps

• Worst‐case cost of a single delete‐min or decrease‐key operation is Ω

• Can we prove a small worst‐case amortized cost for delete‐min and decrease‐key operations?

Remark:

• Data structure that allows operations ,…,

• We say that operation has amortized cost if for every execution the total time is

⋅ ,

where is the number of operations of type Algorithm Theory, WS 2012/13 Fabian Kuhn 62 Amortized Cost of Fibonacci Heaps

• Initialize‐heap, is‐empty, get‐min, insert, and merge have worst‐case cost

• Delete‐min has amortized cost • Decrease‐key has amortized cost

• Starting with an empty heap, any sequence of operations with at most delete‐min operations has total cost (time)

.

• We will now need the marks…

• Cost for Dijkstra: || || log ||

Algorithm Theory, WS 2012/13 Fabian Kuhn 63 Fibonacci Heaps: Marks

Cycle of a node:

1. Node is removed from root list and linked to a node .

2. Child node of is cut and added to root list .

3. Second child of is cut node is cut as well and moved to root list

The boolean value .. indicates whether node has lost a child since the last time was made the child of another node.

Algorithm Theory, WS 2012/13 Fabian Kuhn 64 Potential Function

System state characterized by two parameters: • : number of trees (length of . ) • : number of marked nodes that are not in the root list

Potential function: ≔

Example: 5 18 9 1 2 15 17

14 20 22 12 25 13 3 8

31 19 7

• 7, 2  Φ11

Algorithm Theory, WS 2012/13 Fabian Kuhn 65 Actual Time of Operations

• Operations: initialize‐heap, is‐empty, insert, get‐min, merge actual time: 1 – Normalize unit time such that ,,,, 1

• Operation delete‐min: – Actual time: length of . – Normalize unit time such that

length of . • Operation descrease‐key: – Actual time: length of path to next unmarked ancestor – Normalize unit time such that

lengthof path to next unmarked ancestor

Algorithm Theory, WS 2012/13 Fabian Kuhn 66 Amortized Times

Assume operation is of type:

• initialize‐heap:

– actual time: 1, potential: Φ Φ 0 – amortized time: Φ Φ 1

• is‐empty, get‐min:

– actual time: 1, potential: Φ Φ (heap doesn’t change) – amortized time: Φ Φ 1

• merge:

– Actual time: 1 – combined potential of both heaps: Φ Φ – amortized time: Φ Φ 1

Algorithm Theory, WS 2012/13 Fabian Kuhn 67 Amortized Time of Insert

Assume that operation is an insert operation:

• Actual time: 1

• Potential function: – remains unchanged (no nodes are marked or unmarked, no marked nodes are moved to the root list) – grows by 1 (one element is added to the root list)

, 1 Φ Φ 1 • Amortized time:

Algorithm Theory, WS 2012/13 Fabian Kuhn 68 Amortized Time of Delete‐Min

Assume that operation is a delete‐min operation:

Actual time: . Potential function : • : changes from . to at most • : (# of marked nodes that are not in the root list) – no new marks – if node is moved away from root list, . is set to false  value of does not change!

, . Φ Φ |. | Amortized Time: Algorithm Theory, WS 2012/13 Fabian Kuhn 69 Amortized Time of Decrease‐Key

Assume that operation is a decrease‐key operation at node :

Actual time: length of path to next unmarked ancestor Potential function :

• Assume, node and nodes ,…, are moved to root list – ,…, are marked and moved to root list, . mark is set to true • marked nodes go to root list, 1node gets newly marked • grows by 1, grows by 1 and is decreased by

1, 1 Φ Φ 1 2 1 Φ 3 Amortized time:

Algorithm Theory, WS 2012/13 Fabian Kuhn 70 Complexities Fibonacci Heap

• Initialize‐Heap: • Is‐Empty: • Insert: • Get‐Min: • Delete‐Min: amortized • Decrease‐Key: • Merge (heaps of size and , ):

• How large can get?

Algorithm Theory, WS 2012/13 Fabian Kuhn 71 Rank of Children

Lemma:

Consider a node of rank and let ,…,be the children of in the order in which they were linked to . Then,

. Proof:

Algorithm Theory, WS 2012/13 Fabian Kuhn 72 Size of Trees

Fibonacci Numbers: 0, 1, ∀2: Lemma: In a Fibonacci heap, the size of the sub‐tree of a node with rank is at least . Proof:

• : minimum size of the sub‐tree of a node of rank

Algorithm Theory, WS 2012/13 Fabian Kuhn 73 Size of Trees

1, 2, ∀2: 2 • Claim about Fibonacci numbers:

∀ 0: 1

Algorithm Theory, WS 2012/13 Fabian Kuhn 74 Size of Trees

1, 2,∀2: 2, 1 • Claim of lemma:

Algorithm Theory, WS 2012/13 Fabian Kuhn 75 Size of Trees

Lemma: In a Fibonacci heap, the size of the sub‐tree of a node with rank is at least .

Theorem: The maximum rank of a node in a Fibonacci heap of size is at most . Proof: • The Fibonacci numbers grow exponentially: 1 1 5 1 5 ⋅ 5 2 2

• For , we need nodes.

Algorithm Theory, WS 2012/13 Fabian Kuhn 76 Summary: Binomial and Fibonacci Heaps

Binomial Heap Fibonacci Heap initialize insert get‐min delete‐min * decrease‐key * merge is‐empty ∗ amortized time

Algorithm Theory, WS 2012/13 Fabian Kuhn 77 Minimum Spanning Trees

Prim Algorithm:

1. Start with any node ( is the initial component) 2. In each step: Grow the current component by adding the minimum weight edge connecting the current component with any other node

Kruskal Algorithm:

1. Start with an empty edge set 2. In each step: Add minimum weight edge such that does not close a cycle

Algorithm Theory, WS 2012/13 Fabian Kuhn 78 Implementation of Prim Algorithm

Start at node , very similar to Dijkstra’s algorithm: 1. Initialize 0and ∞for all 2. All nodes are unmarked

3. Get unmarked node which minimizes :

4. For all , ∈ , min ,

5. mark node

6. Until all nodes are marked

Algorithm Theory, WS 2012/13 Fabian Kuhn 79 Implementation of Prim Algorithm

Implementation with Fibonacci heap: • Analysis identical to the analysis of Dijkstra’s algorithm:

insert and delete‐min operations

decrease‐key operations

• Running time:

Algorithm Theory, WS 2012/13 Fabian Kuhn 80 Kruskal Algorithm

1 1. Start with an 13 empty edge set 3 6 10 2. In each step: 4 23 14 Add minimum 21 weight edge 7 2 such that does 28 16 31 not close a cycle 17 9 19 8 12 25 20

18 11

Algorithm Theory, WS 2012/13 Fabian Kuhn 81 Implementation of Kruskal Algorithm

1. Go through edges in order of increasing weights

2. For each edge : if does not close a cycle then

add to the current solution

Algorithm Theory, WS 2012/13 Fabian Kuhn 82 Union‐Find Data Structure

Also known as Disjoint‐Set Data Structure…

Manages partition of a set of elements • set of disjoint sets

Operations:

• _: create a new set that only contains element

• : return the set containing

• , : merge the two sets containing and

Algorithm Theory, WS 2012/13 Fabian Kuhn 83 Implementation of Kruskal Algorithm

1. Initialization: For each node : make_set

2. Go through edges in order of increasing weights: Sort edges by edge weight

3. For each edge ,: if then add to the current solution ,

Algorithm Theory, WS 2012/13 Fabian Kuhn 84 Managing Connected Components

• Union‐find data structure can be used more generally to manage the connected components of a graph … if edges are added incrementally

• make_set for every node

• find returns component containing

• union, merges the components of and (when an edge is added between the components)

• Can also be used to manage biconnected components

Algorithm Theory, WS 2012/13 Fabian Kuhn 85 Basic Implementation Properties

Representation of sets: • Every set of the partition is identified with a representative, by one of its members ∈

Operations: • make_set: is the representative of the new set

• find: return representative of set containing

• union, : unites the sets and containing and and returns the new representative of ∪

Algorithm Theory, WS 2012/13 Fabian Kuhn 86 Observations

Throughout the discussion of union‐find: • : total number of make_set operations • : total number of operations (make_set, find, and union)

Clearly: • • There are at most 1unionoperations

Remark: • We assume that the make_setoperations are the first operations – Does not really matter…

Algorithm Theory, WS 2012/13 Fabian Kuhn 87 Linked List Implementation

Each set is implemented as a linked list: • representative: first list element (all nodes point to first elem.) in addition: pointer to first and last element

• sets: 1,5,8,12,43 , 7,9,15 ; representatives: 5, 9

Algorithm Theory, WS 2012/13 Fabian Kuhn 88 Linked List Implementation

_: • Create list with one element: time:

: • Return first list element: time:

Algorithm Theory, WS 2012/13 Fabian Kuhn 89 Linked List Implementation

, : • Append list of to list of :

Time:

Algorithm Theory, WS 2012/13 Fabian Kuhn 90 Cost of Union (Linked List Implementation)

Total cost for 1unionoperations can be Θ:

• make_set ,make_set ,…,make_set, union , ,union , ,…,union ,

Algorithm Theory, WS 2012/13 Fabian Kuhn 91 Weighted‐Union Heuristic

• In a bad execution, average cost per union can be Θ

• Problem: The longer list is always appended to the shorter one

Idea: • In each union operation, append shorter list to longer one!

Cost for union of sets and : min ,

Theorem: The overall cost of operations of which at most are make_set operations is .

Algorithm Theory, WS 2012/13 Fabian Kuhn 92 Weighted‐Union Heuristic

Theorem: The overall cost of operations of which at most are make_set operations is . Proof:

Algorithm Theory, WS 2012/13 Fabian Kuhn 93 Disjoint‐Set Forests

• Represent each set by a tree

• Representative of a set is the root of the tree

Algorithm Theory, WS 2012/13 Fabian Kuhn 94 Disjoint‐Set Forests

_: create new one‐node tree

: follow parent point to root (parent pointer to itself)

, : attach tree of to tree of

Algorithm Theory, WS 2012/13 Fabian Kuhn 95 Bad Sequence

Bad sequence leads to tree(s) of depth Θ

• make_set ,make_set ,…,make_set, union , ,union , ,…,union ,

Algorithm Theory, WS 2012/13 Fabian Kuhn 96 Union‐By‐Size Heuristic

Union of sets and : • Root of trees representing and : and • W.l.o.g., assume that || • Root of ∪: ( is attached to as a new child)

Theorem: If the union‐by‐rank heuristic is used, the worst‐case cost of a ‐operation is Proof:

Algorithm Theory, WS 2012/13 Fabian Kuhn 97 Union‐Find Algorithms

Recall: operations, of the operations are make_set‐operations

Linked List with Weighted Union Heuristic: • make_set: worst‐case cost 1 • find : worst‐case cost 1 • union : amortized worst‐case cost log

Disjoint‐Set Forest with Union‐By‐Size Heuristic: • make_set: worst‐case cost 1 • find : worst‐case cost log • union : worst‐case cost log

Can we make this faster?

Algorithm Theory, WS 2012/13 Fabian Kuhn 98 Path Compression During Find Operation

: 1. if .then 2. . ≔ find . 3. return .

Algorithm Theory, WS 2012/13 Fabian Kuhn 99 Complexity With Path Compression

When using only path compression (without union‐by‐rank):

: total number of operations • of which are find‐operations • of which are make_set‐operations  at most 1are union‐operations

Total cost: ⋅ ⋅ ⁄

Algorithm Theory, WS 2012/13 Fabian Kuhn 100 Union‐By‐Size and Path Compression

Theorem:

Using the combined union‐by‐size and path compression heuristic, the running time of disjoint‐set (union‐find) operations on elements (at most make_set‐operations) is

⋅ , ,

Where , is the inverse of the Ackermann function.

Algorithm Theory, WS 2012/13 Fabian Kuhn 101 Ackermann Function and its Inverse

Ackermann Function:

For , ℓ 1, ℓ, , ℓ , ℓ ≔ , , , ℓ , ,ℓ , ,ℓ

Inverse of Ackermann Function: , ≔ | , ⁄

Algorithm Theory, WS 2012/13 Fabian Kuhn 102 Inverse of Ackermann Function

• , ≔ min 1 | , ⁄ log

⟹, ⁄ , 1 ⟹ , min 1|, 1 log • 1, ℓ 2ℓ, , 1 1,2, , ℓ 1, , ℓ 1

Algorithm Theory, WS 2012/13 Fabian Kuhn 103