Partitioning, matching, and ordering: Combinatorial scientific computing with matrices and tensors Bora Uçar To cite this version: Bora Uçar. Partitioning, matching, and ordering: Combinatorial scientific computing with matrices and tensors. Computer Science [cs]. ENS de Lyon, 2019. tel-02377874 HAL Id: tel-02377874 https://hal.inria.fr/tel-02377874 Submitted on 24 Nov 2019 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. MEMOIRE´ D’HABILITATION A` DIRIGER DES RECHERCHES pr´esent´ele 19 septembre 2019 `al’Ecole´ Normale Sup´erieurede Lyon par Bora U¸car Partitioning, matching, and ordering: Combinatorial scientific computing with matrices and tensors (Partitionnement, couplage et permutation: Calcul scientifique combinatoire sur des matrices et des tenseurs) Devant le jury compos´ede : Rob H. Bisseling Utrecht University, the Netherlands Rapporteur Nadia Brauner Universit´eJoseph Fourier, Grenoble Examinatrice France Karen D. Devine Sandia National Labs, Albuquerque, Rapporteuse New Mexico, USA Ali Pınar Sandia National Labs, Livermore, Examinateur California, USA Yves Robert ENS Lyon, France Examinateur Denis Trystram Grenoble INP, France Rapporteur ii Contents 1 Introduction1 1.1 Directed graphs..........................1 1.2 Undirected graphs........................5 1.3 Hypergraphs...........................7 1.4 Sparse matrices..........................7 1.5 Sparse tensors...........................9 I Partitioning 13 2 Acyclic partitioning of directed acyclic graphs 15 2.1 Related work........................... 16 2.2 Directed multilevel graph partitioning............. 18 2.2.1 Coarsening........................ 18 2.2.2 Initial partitioning.................... 21 2.2.3 Refinement........................ 22 2.2.4 Constrained coarsening and initial partitioning.... 23 2.3 Experiments............................ 25 2.4 Summary, further notes and references............. 31 3 Distributed memory CP decomposition 33 3.1 Background and notation.................... 34 3.1.1 Hypergraph partitioning................. 34 3.1.2 Tensor operations.................... 35 3.2 Related work........................... 36 3.3 Parallelization........................... 38 3.3.1 Coarse-grain task model................. 39 3.3.2 Fine-grain task model.................. 43 3.4 Experiments............................ 47 3.5 Summary, further notes and references............. 51 iii iv CONTENTS II Matching and related problems 53 4 Matchings in graphs 55 4.1 Scaling matrices to doubly stochastic form........... 55 4.2 Approximation algorithms for bipartite graphs........ 56 4.2.1 Related work....................... 57 4.2.2 Two matching heuristics................. 59 4.2.3 Experiments....................... 69 4.3 Approximation algorithms for general undirected graphs... 75 4.3.1 Related work....................... 76 4.3.2 One-Out, its analysis, and variants.......... 77 4.3.3 Experiments....................... 82 4.4 Summary, further notes and references............. 87 5 Birkhoff-von Neumann decomposition 91 5.1 The minimum number of permutation matrices........ 91 5.1.1 Preliminaries....................... 92 5.1.2 The computational complexity of MinBvNDec .... 93 5.2 A result on the polytope of BvN decompositions....... 94 5.3 Two heuristics.......................... 97 5.4 Experiments............................ 103 5.5 A generalization of Birkhoff-von Neumann theorem...... 106 5.5.1 Doubly stochastic splitting............... 106 5.5.2 Birkhoff von-Neumann decomposition for arbitrary ma- trices........................... 108 5.6 Summary, further notes and references............. 109 6 Matchings in hypergraphs 113 6.1 Background and notation.................... 114 6.2 The heuristics........................... 114 6.2.1 Greedy-H ......................... 115 6.2.2 Karp-Sipser-H ....................... 115 6.2.3 Karp-Sipser-H-scaling ................... 117 6.2.4 Karp-Sipser-H-mindegree ................. 118 6.2.5 Bipartite-reduction .................... 119 6.2.6 Local-Search ........................ 120 6.3 Experiments............................ 120 6.4 Summary, further notes and references............. 128 III Ordering matrices and tensors 129 7 Elimination trees and height-reducing orderings 131 7.1 Background............................ 132 CONTENTS v 7.2 The critical path length and the elimination tree height... 134 7.3 Reducing the elimination tree height.............. 136 7.3.1 Theoretical basis..................... 137 7.3.2 Recursive approach: Permute ............. 137 7.3.3 BBT decomposition: PermuteBBT .......... 138 7.3.4 Edge-separator-based method.............. 139 7.3.5 Vertex-separator-based method............. 141 7.3.6 BT decomposition: PermuteBT ............ 142 7.4 Experiments............................ 142 7.5 Summary, further notes and references............. 145 8 Sparse tensor ordering 147 8.1 Three sparse tensor storage formats.............. 147 8.2 Problem definition and solutions................ 149 8.2.1 BFS-MCS ........................ 149 8.2.2 Lexi-Order ....................... 151 8.2.3 Analysis.......................... 154 8.3 Experiments............................ 156 8.4 Summary, further notes and references............. 160 IV Closing 165 9 Concluding remarks 167 Bibliography 170 vi CONTENTS List of Algorithms 1 CP-ALS for the 3rd order tensors................ 36 2 Mttkrp for the 3rd order tensors................ 38 3 Coarse-grain Mttkrp for the first mode of third order tensors at process p within CP-ALS................... 40 4 Fine-grain Mttkrp for the first mode of 3rd order tensors at process p within CP-ALS..................... 44 5 ScaleSK: Sinkhorn-Knopp scaling............... 56 6 OneSidedMatch for bipartite graph matching........ 60 7 TwoSidedMatch for bipartite graph matching........ 62 8 Karp-Sipser-MT for bipartite 1-out graph matching....... 64 9 One-Out for undirected graph matching............ 78 10 Karp-SipserOne-Out for undirected 1-out graph matching.. 80 11 GreedyBvN for constructing a BvN decomposition....... 98 12 Karp-Sipser-H-scaling for hypergraph matching......... 117 13 Row-LU(A)............................ 134 14 Column-LU(A)........................... 134 15 Permute(A) ........................... 138 16 FindStrongVertexSeparatorES(G) ............ 140 17 FindStrongVertexSeparatorVS(G) ............ 142 18 BFS-MCS for ordering a tensor dimension........... 150 19 Lexi-Order for ordering a tensor dimension.......... 153 vii viii LIST OF ALGORITHMS Chapter 1 Introduction This document investigates three classes of problems at the interplay of discrete algorithms, combinatorial optimization, and numerical methods. The general research area is called combinatorial scientific computing (CSC). In CSC, the contributions have practical and theoretical flavor. For all problems discussed in this document, we have the design, analysis, and implementation of algorithms along with many experiments. The theoretical results are included in this document, some with proofs; the reader is invited to the original papers for the omitted proofs. A similar approach is taken for presenting the experiments. While most results for observing theoretical findings in practice are included, the reader is referred to the original papers for some other results (e.g., run time analysis). The three problem classes are that of partitioning, matching, and or- dering. We cover two problems from the partitioning class, three from the matching class, and two from the ordering class. Below, we present these seven problems after introducing the notation and recalling the related defi- nitions. We summarize the computing contexts in which problems arise and highlight our contributions. 1.1 Directed graphs A directed graph G = (V; E) is a set V of vertices and a set E of directed edges of the form e = (u; v), where e is directed from u to v.A path is a sequence of edges (u1; v1) · (u2; v2) ··· with vi = ui+1. A path ((u1; v1) · (u2; v2) ··· (u`; v`)) is of length `, where it connects a sequence of `+1 vertices (u1; v1 = u2; : : : ; v`−1 = u`; v`). A path is called simple if the connected vertices are distinct. Let u v denote a simple path that starts from u and ends at v. We say that a vertex v is reachable from another vertex u, if a path connects them. A path ((u1; v1) · (u2; v2) ··· (u`; v`)) forms a (simple) cycle if all vi for 1 ≤ i ≤ ` are distinct and u1 = v`.A directed acyclic graph, DAG in short, is a directed graph with no cycles. A directed graph 1 2 CHAPTER 1. INTRODUCTION is strongly connected, if every vertex is reachable from every other vertex. A directed graph G0 = (V 0;E0) is said to be a subgraph of G = (V; E) if V 0 ⊆ V and E0 ⊆ E. We call G0 as the subgraph of G induced by V 0, denoted as G[V 0], if E0 = E \ (V 0 ×V 0). For a vertex set S ⊆ V , we define G − S as the subgraph of G induced by V − S, i.e., G[V − S]. A vertex set C ⊆ V is called a strongly connected component of G, if the induced subgraph G[C] is strongly connected, and C is maximal in the sense that for all C0 ⊃ C, G[C0] is not strongly connected. A transitive