kth royal institute of technology Doctoral Thesis in Computer Science Dynamic Matrix Algorithms and Applications in Convex and Combinatorial Optimization JAN VAN DEN BRAND Stockholm, Sweden 2021 Dynamic Matrix Algorithms and Applications in Convex and Combinatorial Optimization JAN VAN DEN BRAND Academic Dissertation which, with due permission of the KTH Royal Institute of Technology, is submitted for public defence for the Degree of Doctor of Philosophy on Wednesday the 9th of June 2021, at 3:00 p.m. in F3, Lindstedsvägen 26, Stockholm. Doctoral Thesis in Computer Science KTH Royal Institute of Technology Stockholm, Sweden 2021 © Jan van den Brand © Co-Authors: Yin Tat Lee, Yang P. Liu, Danupon Nanongkai, Richard Peng, Thatchaphol Saranurak, Aaron Sidford, Di Wang, Zhao Song ISBN 978-91-7873-867-0 TRITA-EECS-AVL-2021:31 Printed by: Universitetsservice US-AB, Sweden 2021 i Abstract Dynamic algorithms are used to efficiently maintain solutions to problems where the input undergoes some changes. This thesis studies dynamic algo- rithms that maintain solutions to linear algebra problems and we explore their applications and implications for dynamic graphs and optimization problems. Dynamic graph algorithms maintain properties of changing graphs, such as the distances in a graph that undergoes edge deletions and insertions. The main question is how to maintain the information without recomputing the solution from scratch whenever the graph changes. If maintaining the infor- mation without trivial recomputation is possible, the next natural question is how quickly the information can be maintained. This thesis makes progress on both questions: (i) We construct the first non-trivial fully dynamic graph algorithms for single-source shortest paths, diameter and other problems. This answers open questions stated in, e.g., [Demetrescu-Italiano’04]. (ii) We obtain matching upper and conditional lower bounds for the com- plexity of maintaining reachability, maximum matching, directed cycle detec- tion and many other graph properties. This settles the complexity for these problems and answers an open problem stated in [Abboud-V.Williams’14]. We get these results by reducing the dynamic graph problems to dynamic linear algebra problems for which we develop new algorithms. At the same time, conditional lower bounds for the dynamic graph problems thus imply lower bounds for dynamic linear algebra problems as well. We apply the developed techniques for dynamic linear algebra to algo- rithms for linear programs and obtain optimal (i.e. nearly-linear time) al- gorithms for dense instances of linear programs, Markov decision processes, linear L1 regression, and graph specific special cases thereof such as bipartite matching, minimum-cost flow, and (negative weight) shortest paths. For bi- partite matching on dense graphs, this is the first improvement since the clas- sic algorithms by [Dinic’70;Hopcroft-Karp’71;Karzanov’73;Ibarra-Moran’81]. The results are obtained by using that algorithms (i.e. interior point meth- ods) for these problems are iterative and must repeatedly solve linear systems and other linear algebra problems. By using techniques from dynamic lin- ear algebra (i.e. dynamic matrix algorithms), we are able to maintain the solution to these subproblems, reducing the time required per iteration. The construction of our algorithms relies on a joint analysis of the iterative algo- rithm and the dynamic matrix algorithms. On one hand, we develop robust interior point methods which are able to handle relaxations and approxi- mations to the linear algebra subroutines. On other hand, we develop fast dynamic matrix algorithms that are able to maintain the solution to these relaxed subproblems efficiently. Keywords: Dynamic Algorithm, Data Structure, Optimization, Linear Program, Bipartite Matching, Shortest Path, Maximum Flow, Minimum Cost Flow, Diameter ii Sammanfattning Dynamiska grafalgoritmer uppr¨atth˚alleregenskaper f¨or att ¨andra diagram, t.ex. avst˚anden i en graf som genomg˚arkantraderingar och ins¨attningar. Hu- vudfr˚agan ¨ar hur man uppr¨atth˚allerinformationen utan att ber¨akna l¨osningen fr˚angrunden n¨ar grafen ¨andras. Om det ¨ar m¨ojligt att bibeh˚allainformationen utan trivial omber¨akning ¨ar n¨asta naturliga fr˚agahur snabbt informationen kan uppr¨atth˚allas.Denna avhandling g¨or framsteg i b˚adafr˚agorna: (i) Vi ger de f¨orsta icke-triviala dynamiska grafalgoritmerna f¨or kortaste v¨agar, diameter och andra problem med en k¨alla. Detta besvarar ¨oppna fr˚agor fr˚ant.ex. [Demetrescu-Italiano’04]. (ii) Vi ger matchande ¨ovre och villkorade undre gr¨anser f¨or komplexiteten i att uppr¨atth˚allan˚abarhet, maximal matchning, riktad cykeldetektering och m˚angaandra grafegenskaper. Detta l¨oser komplexiteten f¨or dessa problem och svarar p˚a ¨oppna problem som anges i [Abboud-V.Williams’14]. Vi uppn˚ardessa resultat genom att reducera de dynamiska grafproble- men till dynamiska linj¨ar algebra-problem som vi utvecklar nya algoritmer f¨or. Samtidigt inneb¨ar villkorade undre gr¨anser f¨or dynamiska grafproblem s˚alundaundre gr¨anser f¨or dynamiska linj¨ara algebraproblem ocks˚a. Vi till¨ampar de utvecklade teknikerna f¨or dynamisk linj¨ar algebra p˚aal- goritmer f¨or linj¨ara program och uppn˚aroptimala (dvs. n¨astan linj¨ar tid) al- goritmer f¨or t¨ata instanser av linj¨ara program, MDP, linj¨ar L1-regression, och graf-specifika specialfall som bipartit matchning, fl¨ode av minimal kostnad, kortaste v¨agar (med negativa vikter). F¨or bipartit matchning i t¨ata grafer ¨ar detta den f¨orsta f¨orb¨attringen sedan de klassiska algoritmerna av [Dinic’70; Hopcroft-Karp’71; Karzanov’73; Ibarra-Moran’81]. Resultaten erh˚allsmed hj¨alp av att algoritmer (dvs. inrepunkts-metoder) f¨or dessa problem ¨ar iterativa och m˚aste l¨osa linj¨ara system och andra linj¨ara algebra-problem upprepade g˚anger.Genom att anv¨anda tekniker fr˚andyna- misk linj¨ar algebra kan vi bibeh˚allal¨osningen p˚adessa delproblem, vilket minskar den tid som kr¨avs per iteration. Konstruktionen av v˚araalgoritmer bygger p˚aen f¨orenad analys av den iterativa algoritmen och de dynamiska linj¨ara algebraalgoritmerna. A˚ ena sidan utvecklar vi robusta inre punktme- toder som kan hantera approximativa l¨osningar till de linj¨ara algebrasubruti- nerna. A˚ andra sidan utvecklar vi snabba dynamiska linj¨ara algebraalgoritmer som kan bibeh˚allal¨osningen p˚adessa approximativa delproblem effektivt. Acknowledgement My time at KTH was one of the most enjoyable stages of life so far. Clearly, the most influence during that time had my advisor Danupon Nanongkai and I am deeply grateful for his support. He provided me with a lot of freedom, e.g. I was able to select my own research questions, was allowed to select my own working hours, and he was always available online whenever I had questions. Even in the late evening or on weekends I could count on his help and advice. I would also like to express my sincere thanks to my major colaborators and inofficial co-advisors Thatchaphol Saranurak, Aaron Sidford, Yin Tat Lee, and Richard Peng. I would like to thank my other collaborators Aaron Bernstein, Joakim Blikstad, Maximilian Probst Gutenberg, Yang P. Liu, Sagnik Mukhopadhyay, Binghui Peng, Zhao Song, He Sun, Di Wang, and Omri Weinstein. I also thank Mikkel Thorup for inviting me to Copenhagen, the discussions we had, and the advice he has given while I was visiting. I thank Per Austrin for reading my thesis, helping with the Swedish summary, and for the enjoyable coffee and lunch breaks. I want to thank my fellow PhD students at KTH: Mohit, for repeatedly pushing me out of my comfort zone, e.g. to apply for the Google PhD Fellowship. Andreas, for lending me an ear whenever I felt like complaining about our landlords, and also for taking care of my apartment whenever I was traveling. Joseph, for introducing me to web fiction and the discus- sions we had. I also thank him, Kilian and Stephan for being available whenever I needed a break. Before coming to KTH, I studied at the Goethe University in Frankfurt and I want to thank faculty there for preparing me for my time as a PhD student. Thorsten Theobald, who was my Liaison Professor at the “Studienstiftung”, has offered me a lot of advice during my time at the Goethe University. He, Amin Coja-Oghlan and Rudolf Mester also offered a lot of advice near the end of my time at the Goethe University regarding how to proceed after graduation. I am also deeply grateful to Amin for allowing me to finish my Master’s Thesis while already working in Sweden as a PhD student. I thank Ronja D¨uffeland Hartwig Bosse for their support and advice related to teaching, and Hartwig’s advice on writing has been very helpful during my PhD. And finally, I thank my family and friends in Frankfurt. My friends always made time for me whenever I visited, even when the visits turned out to be unexpectedly iii iv frequent. (Again, thanks to Danupon for allowing me this freedom to frequently visit Frankfurt.) My parents who supported me all my life and who have taken great effort to provide me with the opportunity to focus on my education. At last, thank you, Stefanie, for brightening my life, no matter the distance between us. Contents Contents v I Thesis 1 1 Introduction 3 1.1 Dynamic Algorithms . 4 1.2 Optimization . 6 2 Publication List 11 3 Dynamic Linear Algebra 13 3.1 Unifying Matrix Data Structures . 13 3.2 Dynamic Matrix Inverse . 14 4 Dynamic Distances 21 4.1 New Dynamic Algorithms . 22 4.2 From Dynamic Linear Algebra to Dynamic Distances . 24 5 Convex and Combinatorial Optimization 27 5.1 Algorithmic Results . 28 5.2 From Dynamic Linear Algebra to Optimization . 31 II Included Papers 47 A Dynamic Matrix Inverse 49 A.1 Introduction .