DOCSLIB.ORG

Advanced Algorithms, a Graduate-Level Course Taught by Anupam Gupta at Carnegie Mellon University in Fall 2020

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Advanced Algorithms, a Graduate-Level Course Taught by Anupam Gupta at Carnegie Mellon University in Fall 2020 ADVANCEDALGORITHMS notes for cmu 15-850 (fall 2020) lecturer: anupam gupta About this document This document contains the course notes for 15-850:Advanced Algorithms, a graduate-level course taught by Anupam Gupta at Carnegie Mellon University in Fall 2020. Parts of these notes were written by the students of previous versions of the class (based on the lectures) and then edited by the professor. The names of the student scribes will appear soon, as will missing details and bug fixes, more chapters, exercises, and (better) figures. The notes have not been thoroughly checked for accuracy, espe- cially attributions of results. They are intended to serve as study resources and not as a substitute for professionally prepared publica- tions. We apologize for any inadvertent inaccuracies or misrepresen- tations. More information about the course, including problem sets and references, can be found on the course website: https://www.cs.cmu.edu/~15850/ The style files (as well as the text on this page!) are mildly adapted from the ones developed by Yufei Zhao (MIT), for his notes on Graph Theory and Additive Combinatorics. As some of you may guess, the LATEX template used for these notes is called tufte-book. Contents I Discrete Algorithms 9 1 Minimum Spanning Trees 11 1.1 Minimum Spanning Trees: History............. 11 1.2 The Classical Algorithms.................. 13 1.3 Fredman and Tarjan’s O(m log∗ n)-time Algorithm... 15 1.4 A Linear-Time Randomized Algorithm.......... 18 1.5 Optional: MST Verification................. 21 1.6 The Ackermann Function.................. 25 1.7 Matroids............................ 25 2 Arborescences: Directed Spanning Trees 27 2.1 Arborescences......................... 27 2.2 The Chu-Liu/Edmonds/Bock Algorithm......... 28 2.3 Linear Programming Methods............... 30 2.4 Matroid Intersection..................... 35 3 Dynamic Algorithms for Graph Connectivity 37 3.1 Dynamic Connectivity.................... 37 3.2 Frederickson’s algorithm................... 39 3.3 An Amortized, Deterministic Algorithm.......... 43 3.4 A Randomized Algorithm for Dynamic Graph Connectiv- ity................................ 45 4 Shortest Paths in Graphs 49 4.1 Single-Source Shortest Path Algorithms.......... 49 4.2 The All-Pairs Shortest Paths Problem (APSP)....... 52 4.3 Min-Sum Products and APSPs............... 55 4.4 Undirected APSP Using Fast Matrix Multiplication... 58 4.5 Optional: Fredman’s Decision-Tree Complexity Bound. 61 5 Low-Stretch Spanning Trees 63 5.1 Towards a Definition..................... 63 5.2 Low-Stretch Spanning Tree Construction......... 66 4 5.3 Bartal’s Construction..................... 67 5.4 Metric Embeddings: a.k.a. Simplifying Metrics...... 72 6 Graph Matchings I: Combinatorial Algorithms 75 6.1 Notation and Definitions................... 75 6.2 Bipartite Graphs........................ 77 6.3 General Graphs: The Tutte-Berge Theorem........ 80 6.4 The Blossom Algorithm................... 81 6.5 Subsequent Work....................... 86 7 Graph Matchings II: Weighted Matchings 87 7.1 Linear Programming..................... 87 7.2 Weighted Matchings in Bipartite Graphs......... 90 7.3 Another Perspective: Buyers and sellers.......... 94 7.4 A Third Algorithm: Shortest Augmenting Paths..... 99 7.5 Perfect Matchings in General Graphs........... 100 7.6 Integrality of Polyhedra................... 101 8 Graph Matchings III: Algebraic Algorithms 105 8.1 Preliminaries: roots of low degree polynomials...... 105 8.2 Detecting Perfect Matchings by Computing a Determinant107 8.3 From Detecting to Finding Perfect Matchings....... 110 8.4 Red-Blue Perfect Matchings................. 113 8.5 Matchings in Parallel, and the Isolation Lemma..... 114 8.6 A Matrix Scaling Approach................. 115 II The Curse of Dimensionality, and Dimension Reduction 117 9 Concentration of Measure 119 9.1 Asymptotic Analysis..................... 120 9.2 Non-Asymptotic Convergence Bounds........... 121 9.3 Chernoff bounds, and Hoeffding’s inequality....... 124 9.4 Other concentration bounds................. 127 9.5 Application: Oblivious Routing on the Hypercube.... 130 10 Dimension Reduction and the JL Lemma 135 10.1 The Johnson Lindenstrauss lemma............. 135 10.2 The Construction....................... 136 10.3 Intuition for the Distributional JL Lemma......... 137 10.4 The Direct Proof of Lemma 10.2 .............. 138 10.5 Subgaussian Random Variables............... 140 10.6 JL Matrices using Rademachers and Subgaussian-ness. 142 10.7 Optional: Compressive Sensing............... 143 10.8 Some Facts about Balls in High-Dimensional Spaces... 145 5 11 Streaming Algorithms 147 11.1 Streams as Vectors, and Additions/Deletions....... 148 11.2 Computing Moments..................... 149 11.3 A Matrix View of our Estimator............... 152 11.4 Application: Approximate Matrix Multiplication..... 153 11.5 Optional: Computing the Number of Distinct Elements. 154 12 Dimension Reduction: Singular Value Decompositions 159 12.1 Introduction.......................... 159 12.2 Best fit subspaces of dimension k and the SVD...... 160 12.3 Useful facts, and rank-k-approximation.......... 163 12.4 Applications.......................... 164 12.5 Symmetric Matrices...................... 166 III From Discrete to Continuous Algorithms 167 13 Online Learning: Experts and Bandits 169 13.1 The Mistake-Bound Model.................. 169 13.2 The Weighted Majority Algorithm............. 170 13.3 Randomized Weighted Majority.............. 172 13.4 The Hedge Algorithm, and a Change in Perspective... 174 13.5 Optional: The Bandit Setting................ 176 14 Solving Linear Programs using Experts 179 14.1 (Two-Player) Zero-Sum Games............... 179 14.2 Solving LPs Approximately................. 182 15 Approximate Max-Flows using Experts 187 15.1 The Maximum Flow Problem................ 187 15.2 A First Algorithm using the MW Framework....... 188 15.3 Finding Max-Flows using Electrical Flows........ 190 15.4 An O(m3/2)-time Algorithm................. 195 15.5 Optional: An O(m4/3)-time Algorithm........... 196 e 16 The Gradient Descente Framework 203 16.1 Convex Sets and Functions................. 203 16.2 Unconstrained Convex Minimization........... 205 16.3 Constrained Convex Minimization............. 209 16.4 Online Gradient Descent, and Relationship with MW.. 211 16.5 Stronger Assumptions.................... 212 16.6 Extensions and Loose Ends................. 215 17 Mirror Descent 217 17.1 Mirror Descent: the Proximal Point View......... 217 17.2 Mirror Descent: The Mirror Map View........... 220 6 17.3 The Analysis.......................... 223 17.4 Alternative Views of Mirror Descent............ 225 18 The Centroid and Ellipsoid Algorithms 227 18.1 The Centroid Algorithm................... 227 18.2 The Ellipsoid Algorithm................... 229 18.3 Ellipsoid for LP Feasibility.................. 230 18.4 Ellipsoid for Convex Optimization............. 232 18.5 Getting the New Ellipsoid.................. 233 18.6 Algorithms for Solving LPs................. 236 19 Interior-Point Methods 237 19.1 Barrier Functions....................... 238 19.2 The Update Step....................... 240 19.3 The Newton-Raphson Method............... 244 19.4 Self-Concordance....................... 246 IV Combating Intractability 247 20 Approximation Algorithms 249 20.1 A Rough Classification into Hardness Classes...... 249 20.2 The Surrogate......................... 251 20.3 The Set Cover Problem.................... 251 20.4 A Relax-and-Round Algorithm for Set Cover....... 252 20.5 The Bin Packing Problem.................. 254 20.6 The Linear Grouping Algorithm for Bin Packing..... 255 20.7 Subsequent Results and Open Problems.......... 257 21 Approximation Algorithms via SDPs 259 21.1 Positive Semidefinite Matrices................ 259 21.2 Semidefinite Programs.................... 260 21.3 SDPs in Approximation Algorithms............ 262 21.4 The MaxCut Problem and Hyperplane Rounding... 262 21.5 Coloring 3-Colorable Graphs................ 265 22 Online Algorithms 271 22.1 The Competitive Analysis Framework........... 271 22.2 The Ski Rental Problem: Rent or Buy?........... 273 22.3 The Paging Problem..................... 277 22.4 Generalizing Paging: The k-Server Problem........ 279 V Additional Topics 281 23 Smoothed Analysis of Algorithms 283 7 23.1 The Traveling Salesman Problem.............. 284 23.2 The Simplex Algorithm................... 286 23.3 The Knapsack Problem.................... 289 24 Prophets and Secretaries 295 24.1 The Prophet Problem..................... 295 24.2 Secretary Problems...................... 301 Part I Discrete Algorithms 1 Minimum Spanning Trees 1.1 Minimum Spanning Trees: History In minimum spanning tree problem, the input is an undirected con- nected graph G = (V, E) with n nodes and m edges, where the edges have weights w(e) R. The goal is to find a spanning tree 2 of the graph with the minimum total edge-weight. If the graph G A spanning tree/forest is defined to be is disconnected, we get a spanning forest As a classic (and important) an acyclic subgraph T that is inclusion- wise maximal, i.e., adding any edge in problem, it’s been tackled many times. Here’s a brief, not-quite- G T would create a cycle. n comprehensive history of its optimization, all without making any assumptions on the edge weights other that they can be compared in constant time: • Otakar Bor ˚uvka 1 gave the first known MST algorithm in 1926; 1 it was independently discovered by Gustave Choquet, Georges Sollin, and others. VojteˇchJarník 2 gave his algorithm
Load more

Recommended publications
  • Planar Graph Perfect Matching Is in NC
    2018 IEEE 59th Annual Symposium on Foundations of Computer Science Planar Graph Perfect Matching is in NC Nima Anari Vijay V. Vazirani Computer Science Department Computer Science Department Stanford University University of California, Irvine [email protected] [email protected] Abstract—Is perfect matching in NC? That is, is there a finding a perfect matching, was obtained by Karp, Upfal, and deterministic fast parallel algorithm for it? This has been an Wigderson [11]. This was followed by a somewhat simpler outstanding open question in theoretical computer science for algorithm due to Mulmuley, Vazirani, and Vazirani [17]. over three decades, ever since the discovery of RNC matching algorithms. Within this question, the case of planar graphs has The matching problem occupies an especially distinguished remained an enigma: On the one hand, counting the number position in the theory of algorithms: Some of the most of perfect matchings is far harder than finding one (the former central notions and powerful tools within this theory were is #P-complete and the latter is in P), and on the other, for discovered in the context of an algorithmic study of this NC planar graphs, counting has long been known to be in problem, including the notion of polynomial time solvability whereas finding one has resisted a solution. P In this paper, we give an NC algorithm for finding a perfect [4] and the counting class # [22]. The parallel perspective matching in a planar graph. Our algorithm uses the above- has also led to such gains: The first RNC matching algorithm stated fact about counting matchings in a crucial way.
    [Show full text]
  • Formalizing Randomized Matching Algorithms
    Formalizing Randomized Matching Algorithms Dai Tri Man Leˆ and Stephen A. Cook Department of Computer Science, University of Toronto fledt, [email protected] Abstract—Using Jerˇabek’s´ framework for probabilistic rea- set to another. Using this method, Jerˇabek´ developed tools for soning, we formalize the correctness of two fundamental RNC2 describing algorithms in ZPP and RP. He also showed in [13], algorithms for bipartite perfect matching within the theory VPV [14] that the theory VPV+sWPHP(L ) is powerful enough to for polytime reasoning. The first algorithm is for testing if a FP bipartite graph has a perfect matching, and is based on the formalize proofs of very sophisticated derandomization results, Schwartz-Zippel Lemma for polynomial identity testing applied e.g. the Nisan-Wigderson theorem [23] and the Impagliazzo- to the Edmonds polynomial of the graph. The second algorithm, Wigderson theorem [12]. (Note that Jerˇabek´ actually used the due to Mulmuley, Vazirani and Vazirani, is for finding a perfect single-sorted theory PV1+sWPHP(PV), but these two theories matching, where the key ingredient of this algorithm is the are isomorphic.) Isolating Lemma. In [15], Jerˇabek´ developed an even more systematic ap- proach by showing that for any bounded P=poly definable set, I. INTRODUCTION there exists a suitable pair of surjective “counting functions” 1 There is a substantial literature on theories such as PV, S2 , which can approximate the cardinality of the set up to a poly- VPV, V 1 which capture polynomial time reasoning [8], [4], nomially small error. From this and other results he argued [17], [7].
    [Show full text]
  • Structural Results on Matching Estimation with Applications to Streaming∗
    Structural Results on Matching Estimation with Applications to Streaming∗ Marc Bury, Elena Grigorescu, Andrew McGregor, Morteza Monemizadeh, Chris Schwiegelshohn, Sofya Vorotnikova, Samson Zhou We study the problem of estimating the size of a matching when the graph is revealed in a streaming fashion. Our results are multifold: 1. We give a tight structural result relating the size of a maximum matching to the arboricity α of a graph, which has been one of the most studied graph parameters for matching algorithms in data streams. One of the implications is an algorithm that estimates the matching size up to a factor of (α + 2)(1 + ") using O~(αn2=3) space in insertion-only graph streams and O~(αn4=5) space in dynamic streams, where n is the number of nodes in the graph. We also show that in the vertex arrival insertion-only model, an (α + 2) approximation can be achieved using only O(log n) space. 2. We further show that the weight of a maximum weighted matching can be ef- ficiently estimated by augmenting any routine for estimating the size of an un- weighted matching. Namely, given an algorithm for computing a λ-approximation in the unweighted case, we obtain a 2(1+")·λ approximation for the weighted case, while only incurring a multiplicative logarithmic factor in the space bounds. The algorithm is implementable in any streaming model, including dynamic streams. 3. We also investigate algebraic aspects of computing matchings in data streams, by proposing new algorithms and lower bounds based on analyzing the rank of the Tutte-matrix of the graph.
    [Show full text]
  • The Polynomially Bounded Perfect Matching Problem Is in NC2⋆
    The polynomially bounded perfect matching problem is in NC2⋆ Manindra Agrawal1, Thanh Minh Hoang2, and Thomas Thierauf3 1 IIT Kanpur, India 2 Ulm University, Germany 3 Aalen University, Germany Abstract. The perfect matching problem is known to be in ¶, in ran- domized NC, and it is hard for NL. Whether the perfect matching prob- lem is in NC is one of the most prominent open questions in complexity theory regarding parallel computations. Grigoriev and Karpinski [GK87] studied the perfect matching problem for bipartite graphs with polynomially bounded permanent. They showed that for such bipartite graphs the problem of deciding the existence of a perfect matchings is in NC2, and counting and enumerating all perfect matchings is in NC3. For general graphs with a polynomially bounded number of perfect matchings, they show both problems to be in NC3. In this paper we extend and improve these results. We show that for any graph that has a polynomially bounded number of perfect matchings, we can construct all perfect matchings in NC2. We extend the result to weighted graphs. 1 Introduction Whether there is an NC-algorithm for testing if a given graph contains a perfect matching is an outstanding open question in complexity theory. The problem of deciding the existence of a perfect matching in a graph is known to be in ¶ [Edm65], in randomized NC2 [MVV87], and in nonuniform SPL [ARZ99]. This problem is very fundamental for other computational problems (see e.g. [KR98]). Another reason why a derandomization of the perfect matching problem would be very interesting is, that it is a special case of the polynomial identity testing problem.
    [Show full text]
  • 3.1 the Existence of Perfect Matchings in Bipartite Graphs
    15-859(M): Randomized Algorithms Lecturer: Shuchi Chawla Topic: Finding Perfect Matchings Date: 20 Sep, 2004 Scribe: Viswanath Nagarajan 3.1 The existence of perfect matchings in bipartite graphs We will look at an efficient algorithm that determines whether a perfect matching exists in a given bipartite graph or not. This algorithm and its extension to finding perfect matchings is due to Mulmuley, Vazirani and Vazirani (1987). The algorithm is based on polynomial identity testing (see the last lecture for details on this). A bipartite graph G = (U; V; E) is specified by two disjoint sets U and V of vertices, and a set E of edges between them. A perfect matching is a subset of the edge set E such that every vertex has exactly one edge incident on it. Since we are interested in perfect matchings in the graph G, we shall assume that jUj = jV j = n. Let U = fu1; u2; · · · ; ung and V = fv1; v2; · · · ; vng. The algorithm we study today has no error if G does not have a perfect matching (no instance), and 1 errs with probability at most 2 if G does have a perfect matching (yes instance). This is unlike the algorithms we saw in the previous lecture, which erred on no instances. Definition 3.1.1 The Tutte matrix of bipartite graph G = (U; V; E) is an n × n matrix M with the entry at row i and column j, 0 if(ui; uj) 2= E Mi;j = xi;j if(ui; uj) 2 E The determinant of the Tutte matrix is useful in testing whether a graph has a perfect matching or not, as the following claim shows.
    [Show full text]
  • Coded Sparse Matrix Multiplication
    Coded Sparse Matrix Multiplication Sinong Wang∗, Jiashang Liu∗ and Ness Shroff∗† ∗Department of Electrical and Computer Engineering yDepartment of Computer Science and Engineering The Ohio State University fwang.7691, liu.3992 [email protected] Abstract r×t In a large-scale and distributed matrix multiplication problem C = A|B, where C 2 R , the coded computation plays an important role to effectively deal with “stragglers” (distributed computations that may get delayed due to few slow or faulty processors). However, existing coded schemes could destroy the significant sparsity that exists in large-scale machine learning problems, and could result in much higher computation overhead, i.e., O(rt) decoding time. In this paper, we develop a new coded computation strategy, we call sparse code, which achieves near optimal recovery threshold, low computation overhead, and linear decoding time O(nnz(C)). We implement our scheme and demonstrate the advantage of the approach over both uncoded and current fastest coded strategies. I. INTRODUCTION In this paper, we consider a distributed matrix multiplication problem, where we aim to compute C = A|B from input matrices A 2 Rs×r and B 2 Rs×t for some integers r; s; t. This problem is the key building block in machine learning and signal processing problems, and has been used in a large variety of application areas including classification, regression, clustering and feature selection problems. Many such applications have large- scale datasets and massive computation tasks, which forces practitioners to adopt distributed computing frameworks such as Hadoop [1] and Spark [2] to increase the learning speed.
    [Show full text]
  • Chapter 4 Introduction to Spectral Graph Theory
    Chapter 4 Introduction to Spectral Graph Theory Spectral graph theory is the study of a graph through the properties of the eigenvalues and eigenvectors of its associated Laplacian matrix. In the following, we use G = (V; E) to represent an undirected n-vertex graph with no self-loops, and write V = f1; : : : ; ng, with the degree of vertex i denoted di. For undirected graphs our convention will be that if there P is an edge then both (i; j) 2 E and (j; i) 2 E. Thus (i;j)2E 1 = 2jEj. If we wish to sum P over edges only once, we will write fi; jg 2 E for the unordered pair. Thus fi;jg2E 1 = jEj. 4.1 Matrices associated to a graph Given an undirected graph G, the most natural matrix associated to it is its adjacency matrix: Definition 4.1 (Adjacency matrix). The adjacency matrix A 2 f0; 1gn×n is defined as ( 1 if fi; jg 2 E; Aij = 0 otherwise. Note that A is always a symmetric matrix with exactly di ones in the i-th row and the i-th column. While A is a natural representation of G when we think of a matrix as a table of numbers used to store information, it is less natural if we think of a matrix as an operator, a linear transformation which acts on vectors. The most natural operator associated with a graph is the diffusion operator, which spreads a quantity supported on any vertex equally onto its neighbors. To introduce the diffusion operator, first consider the degree matrix: Definition 4.2 (Degree matrix).
    [Show full text]
  • Algebraic Graph Theory
    Algebraic graph theory Gabriel Coutinho University of Waterloo November 6th, 2013 We can associate many matrices to a graph X . Adjacency: 0 0 1 0 1 0 1 B 1 0 1 0 1 C B C A(X ) = B 0 1 0 1 1 C B C @ 1 0 1 0 0 A 0 1 1 0 0 Tutte: 0 1 1 0 0 0 0 1 0 1 0 x12 0 x14 0 1 0 1 1 0 0 B C B −x12 0 x23 0 x25 C Incidence: B 0 0 1 0 1 1 C B C B C B 0 −x23 0 x34 x35 C B 0 1 0 0 0 1 C B C @ A @ −x14 0 −x34 0 0 A 0 0 0 1 1 0 0 −x25 −x35 0 0 What is it about? - the tools Matrices Problem 1 - (very) regular graphs Groups Problem 2 - quantum walks Polynomials Matrices Gabriel Coutinho Algebraic graph theory 2 / 30 Adjacency: 0 0 1 0 1 0 1 B 1 0 1 0 1 C B C A(X ) = B 0 1 0 1 1 C B C @ 1 0 1 0 0 A 0 1 1 0 0 Tutte: 0 1 1 0 0 0 0 1 0 1 0 x12 0 x14 0 1 0 1 1 0 0 B C B −x12 0 x23 0 x25 C Incidence: B 0 0 1 0 1 1 C B C B C B 0 −x23 0 x34 x35 C B 0 1 0 0 0 1 C B C @ A @ −x14 0 −x34 0 0 A 0 0 0 1 1 0 0 −x25 −x35 0 0 What is it about? - the tools Matrices Problem 1 - (very) regular graphs Groups Problem 2 - quantum walks Polynomials Matrices We can associate many matrices to a graph X .
    [Show full text]
  • Perfect Matching Testing Via Matrix Determinant 1 Polynomial Identity 2
    MS&E 319: Matching Theory Instructor: Professor Amin Saberi ([email protected]) Lecture 1: Perfect Matching Testing via Matrix Determinant 1 Polynomial Identity Suppose we are given two polynomials and want to determine whether or not they are identical. For example, (x 1)(x + 1) =? x2 1. − − One way would be to expand each polynomial and compare coefficients. A simpler method is to “plug in” a few numbers and see if both sides are equal. If they are not, we now have a certificate of their inequality, otherwise with good confidence we can say they are equal. This idea is the basis of a randomized algorithm. We can formulate the polynomial equality verification of two given polynomials F and G as: H(x) F (x) G(x) =? 0. ≜ − Denote the maximum degree of F and G as n. Assuming that F (x) = G(x), H(x) will be a polynomial of ∕ degree at most n and therefore it can have at most n roots. Choose an integer x uniformly at random from 1, 2, . , nm . H(x) will be zero with probability at most 1/m, i.e the probability of failing to detect that { } F and G differ is “small”. After just k repetitions, we will be able to determine with probability 1 1/mk − whether the two polynomials are identical i.e. the probability of success is arbitrarily close to 1. The following result, known as the Schwartz-Zippel lemma, generalizes the above observation to polynomials on more than one variables: Lemma 1 Suppose that F is a polynomial in variables (x1, x2, .
    [Show full text]
  • Algebraic Algorithms in Combinatorial Optimization
    Algebraic Algorithms in Combinatorial Optimization CHEUNG, Ho Yee A Thesis Submitted in Partial Fulfillment of the Requirements for the Degree of Master of Philosophy in Computer Science and Engineering The Chinese University of Hong Kong September 2011 Thesis/Assessment Committee Professor Shengyu Zhang (Chair) Professor Lap Chi Lau (Thesis Supervisor) Professor Andrej Bogdanov (Committee Member) Professor Satoru Iwata (External Examiner) Abstract In this thesis we extend the recent algebraic approach to design fast algorithms for two problems in combinatorial optimization. First we study the linear matroid parity problem, a common generalization of graph matching and linear matroid intersection, that has applications in various areas. We show that Harvey's algo- rithm for linear matroid intersection can be easily generalized to linear matroid parity. This gives an algorithm that is faster and simpler than previous known al- gorithms. For some graph problems that can be reduced to linear matroid parity, we again show that Harvey's algorithm for graph matching can be generalized to these problems to give faster algorithms. While linear matroid parity and some of its applications are challenging generalizations, our results show that the al- gebraic algorithmic framework can be adapted nicely to give faster and simpler algorithms in more general settings. Then we study the all pairs edge connectivity problem for directed graphs, where we would like to compute minimum s-t cut value between all pairs of vertices. Using a combinatorial approach it is not known how to solve this problem faster than computing the minimum s-t cut value for each pair of vertices separately.
    [Show full text]
  • Coded Computation for Speeding up Distributed Machine Learning
    CODED COMPUTATION FOR SPEEDING UP DISTRIBUTED MACHINE LEARNING DISSERTATION Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the Graduate School of the Ohio State University By Sinong Wang Graduate Program in Electrical and Computer Engineering The Ohio State University 2019 Dissertation Committee: Ness B. Shroff, Advisor Atilla Eryilmaz Abhishek Gupta Andrea Serrani c Copyrighted by Sinong Wang 2019 ABSTRACT Large-scale machine learning has shown great promise for solving many practical ap- plications. Such applications require massive training datasets and model parameters, and force practitioners to adopt distributed computing frameworks such as Hadoop and Spark to increase the learning speed. However, the speedup gain is far from ideal due to the latency incurred in waiting for a few slow or faulty processors, called \straggler" to complete their tasks. To alleviate this problem, current frameworks such as Hadoop deploy various straggler detection techniques and usually replicate the straggling task on another available node, which creates a large computation overhead. In this dissertation, we focus on a new and more effective technique, called coded computation to deal with stragglers in the distributed computation problems. It creates and exploits coding redundancy in local computation to enable the final output to be recoverable from the results of partially finished workers, and can therefore alleviate the impact of straggling workers. However, we observe that current coded computation techniques are not suitable for large-scale machine learning application. The reason is that the input training data exhibit both extremely large-scale targeting data and a sparse structure. However, the existing coded computation schemes destroy the sparsity and creates large computation redundancy.
    [Show full text]
  • Planar Graph Perfect Matching Is in NC
    Planar Graph Perfect Matching is in NC Nima Anari1 and Vijay V. Vazirani2 1Computer Science Department, Stanford University,∗ [email protected] 2Computer Science Department, University of California, Irvine,y [email protected] Abstract Is perfect matching in NC? That is, is there a deterministic fast parallel algorithm for it? This has been an outstanding open question in theoretical computer science for over three decades, ever since the discovery of RNC matching algorithms. Within this question, the case of planar graphs has remained an enigma: On the one hand, counting the number of perfect matchings is far harder than finding one (the former is #P-complete and the latter is in P), and on the other, for planar graphs, counting has long been known to be in NC whereas finding one has resisted a solution. In this paper, we give an NC algorithm for finding a perfect matching in a planar graph. Our algorithm uses the above-stated fact about counting matchings in a crucial way. Our main new idea is an NC algorithm for finding a face of the perfect matching polytope at which W(n) new conditions, involving constraints of the polytope, are simultaneously satisfied. Several other ideas are also needed, such as finding a point in the interior of the minimum weight face of this polytope and finding a balanced tight odd set in NC. 1 Introduction Is perfect matching in NC? That is, is there a deterministic parallel algorithm that computes a perfect matching in a graph in polylogarithmic time using polynomially many processors? This has been an outstanding open question in theoretical computer science for over three decades, ever since the discovery of RNC matching algorithms [KUW86; MVV87].
    [Show full text]