Chapter 4 Data Structures Algorithm Theory WS 2012/13 Fabian Kuhn Examples Dictionary: • Operations: insert(key,value), delete(key), find(key) • Implementations: – Linked list: all operations take time (: size of data structure) – Balanced binary tree: all operations take log time – Hash table: all operations take 1 times (with some assumptions) Stack (LIFO Queue): • Operations: push, pull • Linked list: 1 for both operations (FIFO) Queue: • Operations: enqueue, dequeue • Linked list: 1 time for both operations Here: Priority Queues (heaps), Union‐Find data structure Algorithm Theory, WS 2012/13 Fabian Kuhn 2 Dijkstra’s Algorithm Single‐Source Shortest Path Problem: • Given: graph ,with edge weights 0for ∈ source node ∈ • Goal: compute shortest paths from to all ∈ Dijkstra’s Algorithm: 1. Initialize , 0 and , ∞ for all 2. All nodes are unmarked 3. Get unmarked node which minimizes , : 4. For all , ∈ , , min , , , 5. mark node 6. Until all nodes are marked Algorithm Theory, WS 2012/13 Fabian Kuhn 3 Example 1 ∞ 32 ∞ ∞ 3 9 10 4 23 13 ∞ ∞ 3 6 2 2 ∞ 1 ∞ 3 17 9 19 ∞ 8 1 2 20 ∞ 18 ∞ 1 Algorithm Theory, WS 2012/13 Fabian Kuhn 4 Example 1 ∞ 32 ∞ ∞ 3 9 10 4 23 13 ∞ 3 6 2 2 ∞ 1 3 17 9 19 8 1 2 20 18 1 Algorithm Theory, WS 2012/13 Fabian Kuhn 5 Example 1 ∞ 32 ∞ 3 9 10 4 23 13 ∞ 3 6 2 2 ∞ 1 3 17 9 19 8 1 2 20 18 1 Algorithm Theory, WS 2012/13 Fabian Kuhn 6 Example 1 32 ∞ 3 9 10 4 23 13 3 6 2 2 ∞ 1 3 17 9 19 8 1 2 20 18 1 Algorithm Theory, WS 2012/13 Fabian Kuhn 7 Example 1 32 3 9 10 4 23 13 3 6 2 2 ∞ 1 3 17 9 19 8 1 2 20 18 1 Algorithm Theory, WS 2012/13 Fabian Kuhn 8 Example 1 32 3 9 10 4 23 13 3 6 2 2 ∞ 1 3 17 9 19 8 1 2 20 18 1 Algorithm Theory, WS 2012/13 Fabian Kuhn 9 Example 1 32 3 9 10 4 23 13 3 6 2 2 1 3 17 9 19 8 1 2 20 18 1 Algorithm Theory, WS 2012/13 Fabian Kuhn 10 Example 1 32 3 9 10 4 23 13 3 6 2 2 1 3 17 9 19 8 1 2 20 18 1 Algorithm Theory, WS 2012/13 Fabian Kuhn 11 Implementation of Dijkstra’s Algorithm Dijkstra’s Algorithm: 1. Initialize , 0 and , ∞ for all 2. All nodes are unmarked 3. Get unmarked node which minimizes , : 4. For all , ∈ , , min , , , 5. mark node 6. Until all nodes are marked Algorithm Theory, WS 2012/13 Fabian Kuhn 12 Priority Queue / Heap • Stores (key,data) pairs (like dictionary) • But, different set of operations: • Initialize‐Heap: creates new empty heap • Is‐Empty: returns true if heap is empty • Insert(key,data): inserts (key,data)‐pair, returns pointer to entry • Get‐Min: returns (key,data)‐pair with minimum key • Delete‐Min: deletes minimum (key,data)‐pair • Decrease‐Key(entry,newkey): decreases key of entry to newkey • Merge: merges two heaps into one Algorithm Theory, WS 2012/13 Fabian Kuhn 13 Implementation of Dijkstra’s Algorithm Store nodes in a priority queue, use , as keys: 1. Initialize , 0 and , ∞ for all 2. All nodes are unmarked 3. Get unmarked node which minimizes , : 4. mark node 5. For all , ∈ , , min , , , 6. Until all nodes are marked Algorithm Theory, WS 2012/13 Fabian Kuhn 14 Analysis Number of priority queue operations for Dijkstra: • Initialize‐Heap: • Is‐Empty: || • Insert: || • Get‐Min: || • Delete‐Min: || • Decrease‐Key: || • Merge: Algorithm Theory, WS 2012/13 Fabian Kuhn 15 Priority Queue Implementation Implementation as min‐heap: complete binary tree, e.g., stored in an array • Initialize‐Heap: • Is‐Empty: • Insert: • Get‐Min: • Delete‐Min: • Decrease‐Key: • Merge (heaps of size and , ): Algorithm Theory, WS 2012/13 Fabian Kuhn 16 Better Implementation • Can we do better? • Cost of Dijkstra with complete binary min‐heap implementation: log • Can be improved if we can make decrease‐key cheaper… • Cost of merging two heaps is expensive • We will get there in two steps: Binomial heap Fibonacci heap Algorithm Theory, WS 2012/13 Fabian Kuhn 17 Definition: Binomial Tree Binomial tree of order 0: Algorithm Theory, WS 2012/13 Fabian Kuhn 18 Binomial Trees Algorithm Theory, WS 2012/13 Fabian Kuhn 19 Properties 1. Tree has 2 nodes 2. Height of tree is 3. Root degree of is 4. In , there are exactly nodes at depth Algorithm Theory, WS 2012/13 Fabian Kuhn 20 Binomial Coefficients • Binomial coefficient: : # of element subsets of a set of size • Property: Pascal triangle: Algorithm Theory, WS 2012/13 Fabian Kuhn 21 Number of Nodes at Depth in Claim: In , there are exactly nodes at depth Algorithm Theory, WS 2012/13 Fabian Kuhn 22 Binomial Heap • Keys are stored in nodes of binomial trees of different order nodes: there is a binomial tree or order iff bit of base‐2 representation of is 1. • Min‐Heap Property: Key of node keys of all nodes in sub‐tree of Algorithm Theory, WS 2012/13 Fabian Kuhn 23 Example • 10 keys: 2, 5, 8, 9, 12, 14, 17, 18, 20, 22, 25 • Binary representation of : 11 1011 trees , , and present 5 2 17 9 14 20 8 12 18 25 22 Algorithm Theory, WS 2012/13 Fabian Kuhn 24 Child‐Sibling Representation Structure of a node: parent key degree child sibling Algorithm Theory, WS 2012/13 Fabian Kuhn 25 Link Operation • Unite two binomial trees of the same order to one tree: 12 ⨁ ⇒ • Time: 15 20 18 ⨁ 25 40 22 30 Algorithm Theory, WS 2012/13 Fabian Kuhn 26 Merge Operation Merging two binomial heaps: • For ,,…,: If there are 2 or 3 binomial trees : apply link operation to merge 2 trees into one binomial tree Time: ∪ Algorithm Theory, WS 2012/13 Fabian Kuhn 27 Example 9 13 5 2 17 12 18 14 20 8 22 25 Algorithm Theory, WS 2012/13 Fabian Kuhn 28 Operations Initialize: create empty list of trees Get minimum of queue: time 1 (if we maintain a pointer) Decrease‐key at node : • Set key of node to new key • Swap with parent until min‐heap property is restored • Time: log Insert key into queue : 1. Create queue of size 1 containing only 2. Merge and • Time for insert: log Algorithm Theory, WS 2012/13 Fabian Kuhn 29 Operations Delete‐Min Operation: 1. Find tree with minimum root 2. Remove from queue queue ′ 3. Children of form new queue ′′ 4. Merge queues ′ and ′′ • Overall time: Algorithm Theory, WS 2012/13 Fabian Kuhn 30 Delete‐Min Example 2 5 17 9 14 20 8 12 18 25 22 Algorithm Theory, WS 2012/13 Fabian Kuhn 31 Complexities Binomial Heap • Initialize‐Heap: • Is‐Empty: • Insert: • Get‐Min: • Delete‐Min: • Decrease‐Key: • Merge (heaps of size and , ): Algorithm Theory, WS 2012/13 Fabian Kuhn 32 Can We Do Better? • Binomial heap: insert, delete‐min, and decrease‐key cost log • One of the operations insert or delete‐min must cost Ωlog : – Heap‐Sort: Insert elements into heap, then take out the minimum times – (Comparison‐based) sorting costs at least Ω log . • But maybe we can improve decrease‐key and one of the other two operations? • Structure of binomial heap is not flexible: – Simplifies analysis, allows to get strong worst‐case bounds – But, operations almost inherently need at least logarithmic time Algorithm Theory, WS 2012/13 Fabian Kuhn 33 Fibonacci Heaps Lacy‐merge variant of binomial heaps: • Do not merge trees as long as possible… Structure: A Fibonacci heap consists of a collection of trees satisfying the min‐heap property. Variables: • . : root of the tree containing the (a) minimum key • . : circular, doubly linked, unordered list containing the roots of all trees • . : number of nodes currently in Algorithm Theory, WS 2012/13 Fabian Kuhn 34 Trees in Fibonacci Heaps Structure of a single node : parent right left key degree child mark • . : points to circular, doubly linked and unordered list of the children of • . , . : pointers to siblings (in doubly linked list) • . : will be used later… Advantages of circular, doubly linked lists: • Deleting an element takes constant time • Concatenating two lists takes constant time Algorithm Theory, WS 2012/13 Fabian Kuhn 35 Example Figure: Cormen et al., Introduction to Algorithms Algorithm Theory, WS 2012/13 Fabian Kuhn 36 Simple (Lazy) Operations Initialize‐Heap : • . ≔ . ≔ Merge heaps and ′: • concatenate root lists • update . Insert element into : • create new one‐node tree containing H′ • merge heaps and ′ Get minimum element of : • return . Algorithm Theory, WS 2012/13 Fabian Kuhn 37 Operation Delete‐Min Delete the node with minimum key from and return its element: 1. ≔.; 2. if . 0 then 3. remove . from . ; 4. add . (list) to . 5. ; // Repeatedly merge nodes with equal degree in the root list // until degrees of nodes in the root list are distinct. // Determine the element with minimum key 6. return Algorithm Theory, WS 2012/13 Fabian Kuhn 38 Rank and Maximum Degree Ranks of nodes, trees, heap: Node : • : degree of Tree : • : rank (degree) of root node of Heap : • : maximum degree of any node in Assumption (: number of nodes in ): – for a known function Algorithm Theory, WS 2012/13 Fabian Kuhn 39 Merging Two Trees Given: Heap‐ordered trees , ′ with • Assume: min‐key of min‐key of ′ Operation , : • Removes tree ′ from root list ′ and adds to child list of • ≔1 • .≔ ′ Algorithm Theory, WS 2012/13 Fabian Kuhn 40 Consolidation of Root List Array pointing to find roots with the same rank: 012 Consolidate: Time: 1. for ≔0to do ≔null; |.|. 2.