18.415 Advanced Algorithms Notes from Fall 2020. Last Updated: November 27, 2020. Contents Introduction 6 1 Lecture 1: Fibonacci Heaps7 1.1 MST problem review.................................7 1.2 Using d-heaps for MST................................8 1.3 Introduction to Fibonacci Heaps...........................8 2 Lecture 2: Fibonacci Heaps, Persistent Data Structs 12 2.1 Fibonacci Heaps Continued.............................. 12 2.2 Applications of Fibonacci Heaps........................... 14 2.2.1 Further Improvements............................ 15 2.3 Introduction to Persistent Data Structures...................... 15 3 Lecture 3: Persistent Data Structs, Splay Trees 16 3.1 Persistent Data Structures Continued........................ 16 3.2 The Planar Point Location Problem......................... 17 3.3 Introduction to Splay Trees.............................. 19 4 Lecture 4: Splay Trees 20 4.1 Heuristics of Splay Trees............................... 20 4.2 Solving the Balancing Problem............................ 20 4.3 Implementation and Analysis of Operations..................... 21 4.4 Results with the Access Lemma........................... 23 5 Lecture 5: Splay Updates, Buckets 25 5.1 Wrapping up Splay Trees............................... 25 5.2 Buckets and Indirect Addressing........................... 26 5.2.1 Improvements................................. 27 5.2.2 Formalization with Tries........................... 27 5.2.3 Laziness wins again.............................. 28 1 6 Lecture 6: VEB queues, Hashing 29 6.1 Improvements to the Denardo and Fox queue.................... 29 6.2 VEB queues...................................... 29 6.3 Hashing........................................ 31 7 Lecture 7: Hashing, Max Flows 33 7.1 Hashing with Less Space............................... 33 7.2 Perfect Hash Functions................................ 34 7.3 Combinatorical Optimization............................. 37 7.4 Introduction to the Max Flow Problem........................ 37 8 Lecture 8: Max Flows I 38 8.1 Max Flow Algorithms................................. 41 9 Lecture 9: Max Flows II 43 9.1 Improving on the Augmenting Paths Algorithm................... 43 9.2 Scaling Algorithms.................................. 44 9.3 Strongly Polynomial Algorithms........................... 45 10 Lecture 10: Max Flows III 46 10.1 Edmonds-Karp Algorithm............................... 46 10.2 Blocking Flows.................................... 47 11 Lecture 11: Max Flows IV 50 11.1 Improving Blocking Flows............................... 50 11.2 Min-Cost Max Flows................................. 50 12 Lecture 12: Max Flows V 52 12.1 Price Functions.................................... 52 12.2 Min-Cost Flow Algorithms.............................. 53 13 Lecture 13: Max Flows VI, Linear Programming I 55 13.1 Polynomial Min-cost Flow............................... 55 13.2 Complementary Slackness............................... 55 13.3 Linear Programming.................................. 56 14 Lecture 14: Linear Programming II 58 14.1 Verification and Size of Solutions........................... 58 14.2 Geometry of Solutions................................ 60 2 15 Lecture 15: Linear Programming III 63 15.1 Equivalence of Definitions Continued......................... 63 15.2 Complexity....................................... 64 15.3 Duality......................................... 64 16 Lecture 16: Linear Programming IV 68 16.1 Strong Duality..................................... 68 16.2 Taking a Dual..................................... 69 16.3 Duality - Shortest Path Problem........................... 70 17 Lecture 17: Linear Programming V 72 17.1 Duality - Max Flow Problem............................. 72 17.2 Complementary Slackness............................... 73 17.3 Duality - Min-Cost Circulation............................ 75 18 Lecture 18: Linear Programming VI 76 18.1 More on Basic Feasible Solutions........................... 76 18.2 Simplex Method.................................... 77 18.3 Simplex and Duality.................................. 78 19 Lecture 19: Linear Programming VII 79 19.1 Ellipsoid Method................................... 79 19.2 Interior Point Method................................. 82 20 Lecture 20: Approximation Algorithms I 83 20.1 Introduction to Approximation Algorithms...................... 83 20.2 Greedy Algorithms................................... 85 20.3 Scheduling Theory.................................. 87 21 Lecture 21: Approximation Algorithms II 88 21.1 Scheduling Theory Continued............................. 88 21.2 Approximation Schemes................................ 88 21.2.1 Knapsack Problem Revisited......................... 89 21.2.2 Enumeration Techniques........................... 90 22 Lecture 22: Approximation Algorithms III 91 22.1 Enumeration Techniques Continued......................... 91 22.2 MAX-SNP hard problems............................... 92 22.3 Relaxations...................................... 92 3 23 Lecture 23: Approximation Algorithms IV 94 23.1 Relaxation in Scheduling............................... 94 23.2 General Relaxation with Linear Programming.................... 95 23.2.1 Vertex Cover Problem............................ 95 23.2.2 Facility Location Problem.......................... 96 24 Lecture 24: Approximation Algorithms V, Parameterization 98 24.1 Max-SAT - Relaxation with Rounding........................ 98 24.2 Parametrized Complexity............................... 100 25 Lecture 25: Computational Geometry I 102 25.1 Introduction to Computational Geometry...................... 102 25.2 Orthogonal Range Queries.............................. 102 25.3 Sweep Algorithms................................... 104 25.3.1 Convex Hull Problem............................. 104 25.3.2 Halfspace Intersection Problem....................... 105 26 Lecture 26: Computational Geometry II 106 26.1 Voronoi Diagrams - Introduction........................... 106 26.2 Voronoi Diagrams - Construction........................... 107 27 Lecture 27: Online Algorithms I 110 27.1 Introduction to Online Algorithms.......................... 110 27.2 Ski-Rental....................................... 110 27.3 Selling Stuff...................................... 111 27.4 Online Scheduling................................... 112 27.5 Paging......................................... 112 28 Lecture 28: Online Algorithms II 113 28.1 Paging Continued................................... 113 28.2 Adversarial Inputs................................... 114 28.3 Randomized Paging.................................. 114 29 Lecture 29: Online Algorithms III 116 29.1 A general technique for lower bounds......................... 116 29.2 A lower bound on Paging............................... 117 29.3 The k-server problem................................. 118 4 30 Lecture 30: Online Algs. IV, External Memory Algs. I 119 30.1 Double Coverage Algorithm.............................. 119 30.2 Introduction to External Memory Algorithms.................... 120 31 Lecture 31: External Memory Algorithms II 121 31.1 Matrix Multiplication Continued........................... 121 31.2 Linked Lists...................................... 121 31.3 Search Trees...................................... 122 31.4 Sorting......................................... 123 32 Lecture 32: Parallel Algorithms I 124 32.1 Definitions and Models................................ 124 32.2 Addition........................................ 125 32.3 Parallel Processors/PRAM.............................. 126 33 Lecture 33: Parallel Algorithms II 128 33.1 MAX operation.................................... 128 33.2 Parallel Prefix..................................... 129 33.3 List Ranking...................................... 129 33.4 Binary Search and Sorting.............................. 130 5 Introduction These notes were typeset live from the online 18.415 lectures, taught by David Karger. Remark. Though the course was originally designed as ‘Advanced Algorithms,’ it is probably better to refer to it as ‘Classical Algorithms.’ The course doesn’t go into any cutting-edge algorithms; rather, it provides us a toolkit for algorithms that we are expected to know. We’ll mostly be looking at combinatorical problems, and introducing various models to be able to understand them as well as their efficiency. Most course information can be found at the following link: courses.csail.mit.edu/6.854. The core of the course are long and challenging PSETs, and is complemented by a peer grading assignment and a final project. Collaboration is encouraged and essential, though they should be small groups of size 3. Academic integrity is critical to the course, though in recent years people have always been caught. Due to current conditions, there are many experimental tactics in regards to lectures. Lectures will be recorded, though live lectures will allow for questions. Playing previous lectures may also happen, in where we can stop and ask questions. Self-study / giant office hours during lecture may also be possible. Problem set structure may also be changed, due to lack of facility of collaboration. Collaborators will be changed throughout the semester. There will be sufficient ‘slack points’ for late PSET submissions, but extensions will be granted as well if necessary. 6 1 Lecture 1: Fibonacci Heaps 1.1 MST problem review Fibonacci Heaps were developed by Fredman and Tarjan in 1985, and were done so specifically to solve the Shortest Path/Minimum Spanning Tree problems.