Exact and Fast Algorithms for Mixed-Integer Nonlinear Programming vorgelegt von Dipl.-Math. Ambros M. Gleixner aus Landshut von der Fakultat¨ II – Mathematik und Naturwissenschaften der Technischen Universitat¨ Berlin zur Erlangung des akademischen Grades Doktor der Naturwissenschaften – Dr. rer. nat. – genehmigte Dissertation Promotionsausschuss Vorsitzender: Prof. Dr. Reinhold Schneider Berichter: Prof. Dr. Dr. h.c. mult. Martin Grotschel¨ Prof. Dr. Thorsten Koch Prof. Dr. Andrea Lodi Tag der wissenschaftlichen Aussprache: 24. Juni 2015 Berlin 2015 D 83 Abstract Mixed-integer nonlinear programming (MINLP) comprises the broad class of finite- dimensional mathematical optimization problems from mixed-integer linear program- ming and global optimization. The combination of the two disciplines allows us to construct more accurate models of real-world systems, while at the same time it in- creases the algorithmic challenges that come with solving them. This thesis presents new methods that improve the numerical reliability and the computational perfor- mance of global MINLP solvers. Since state-of-the-art algorithms for nonconvex MINLP fundamentally rely on solving linear programming (LP) relaxations, we address numerical accuracy directly for LP by means of LP iterative refinement: a new algorithm to solve linear programs to arbitrarily high levels of precision. The thesis is supplemented by an exact extension of the LP solver SoPlex, which proves on average 1.85 to 3 times faster than current state-of-the-art software for solving general linear programs exactly over the rational numbers. These methods can be generalized to quadratic programming. We study their application to numerically difficult multiscale LP models for metabolic networks in systems biology. To improve the computational performance of LP-based MINLP solvers, we show how the expensive, but effective, bound-tightening technique called optimization- based bound tightening can be approximated more efficiently via feasibility-based bound tightening. The resulting implementation increases the number of instances that can be solved and reduces the average running time of the MINLP solver SCIP by 17–19% on hard mixed-integer nonlinear programs. Last, we present branching rules that exploit the presence of nonlinear integer variables, i.e., variables both contained in nonlinear terms and required to be integral. The new branching rules prefer integer variables when performing spatial branching, and favor variables in nonlinear terms when resolving integer infeasibility. They reduce the average running time of SCIP by 17% on affected instances. Most importantly, all of the new methods enable us to solve problems which could not be solved before, either due to their numerical complexity or because of limited computing resources. i Acknowledgements This thesis would not have been possible without the inspiring environment and the technical and financial support I found at Zuse Institute Berlin. For their trust and for this support I wish to thank Martin Grotschel¨ and Thorsten Koch and my colleagues at the Department of Optimization and the administration of the institute. I am indebted to Andreas Bley, without whom I might never have learned about this special place, and to Gary Froyland, who encouraged me during my first steps in discrete optimization at the University of New South Wales. I thank Andrea Lodi for taking an interest in the results of this thesis and agreeing to be a member of my PhD committee, despite his numerous commitments. I am most grateful to the collaborators and co-authors of the research articles that were written in the course of this thesis. I want to thank Timo Berthold for his creative, critical, and ambitious mind, Dan Steffy for his initiative to study iterative refinement using SoPlex, Benjamin Muller¨ and Stefan Weltge for implementing and improving the OBBT components of SCIP, and Kati Wolter for her thorough testing and profiling of the exact extensions of SoPlex. I wish to thank Dennis Elbrachter,¨ Luca Fabbri, Wei Huang, Leif Naundorf, and Fr´ed´eric Pythoud, who supported me during their time as student assistants. The computational aspect of this thesis builds on the work of many people. I thank Roland Wunderling for creating SoPlex, Tobias Achterberg for creating SCIP, Stefan Vigerske for turning SCIP into an MINLP solver, and the entire team behind the SCIP Optimization Suite for developing this extraordinary code base. In particular, I want to thank Marc Pfetsch for our first debugging sessions in SoPlex, Stefan Heinz for his team spirit, Gregor Hendel for supporting me in the evaluation of the experiments in Chapters 6 and 7 by means of his excellent Ipet tool, and Gerald Gamrath and Matthias Miltenberger for freeing me from a lot of maintenance work on SCIP and SoPlex when I needed it. I am grateful to Ali Ebrahim for providing me with the problem instances which form the basis of the experiments in Chapter 4. Many thanks to Matthias Walter for testing the exact extensions of SoPlex. The quality of the manuscript was greatly improved by the thorough proofread- ing of Timo Berthold, Christina Burt, Ali Ebrahim, Gerald Gamrath, Gregor Hendel, Joshua Lerman, Steve Maher, Benjamin Muller,¨ Jonas Schweiger, Felipe Serrano, Dan Steffy, Charles Stewart, Stefan Vigerske, Matthias Walter, and Axel Werner. Thank you for your time and effort. I am particularly indebted to Ralf Borndorfer,¨ Steve Maher, and Marika Neumann for discussing the outline of the introduction. iii iv Acknowledgements My research was financially supported by the Berlin Mathematical 1School , the Re- search Campus MODAL2 funded by the German Ministry for Research and Education, and the COST Action TD12073 funded by the European Union. Finally, one of the most enriching facets of this time was certainly the opportunity to travel and see the world. I want to express my gratitude to everyone who hosted me during these years, in particular Andras´ Frank, Kristof´ B´erczi, Erika Kovacs,´ Laszlo´ V´egh, and the EGRES team for many kind invitations to Budapest, Gary Froyland for some Sydney nostalgia after the Discrete Optimization workshop 2010 at the University of New South Wales, Christina Burt, Pascal Van Hentenryck, Adrian Pearce, and the entire NICTA team for their continuing interest in SCIP and for their generous invitations to Melbourne, Katsuki Fujisawa, Yuji Shinano, and Hayato Waki for two wonderful trips to Japan, Yu-Hong Dai, Xin Liu, and Ya-Feng Liu for a very golden week in Beijing, Felipe Serrano and Carolina von Hausen and their families for a memorable time in Chile, Dan Steffy and Huizhu Crystal Wang for Detroit coffee and music, Joao˜ Pedro Pedroso for the beauty of Porto, Ruth Misener for an inspiring trip to London, Andrea Lodi, Luca Fabbri, and Jonas Schweiger for Bologna’s delicacies, Volker Kaibel, Matthias Walter, and Stefan Weltge for Magdeburg’s Christmas markets, and Michael Bussieck, Stefan Vigerske, and the GAMS team for the best Ethiopian coffee in Braunschweig. I am grateful to Maike Massierer and James MacLaurin for great times in Basel, Oxford, and Nice, and for continuing to share many common and distinct experiences since our undergraduate year in UNSW’s Honours room. 1 http://www.math-berlin.de/ 2 http://www.forschungscampus-modal.de/ 3 http://cost-td1207.zib.de/ Contents Abstract i Acknowledgements iii Introduction 1 1 Preliminaries 7 1.1 Notation and Abbreviations . 7 1.2 Mathematical Prerequisites . 8 1.3 Experimental Methodology . 10 1.3.1 Hardware and Software, 10 1.3.2 Averaging, 10 1.3.3 Statistical Significance, 11 1.4 Test Sets . 13 1.4.1 Linear Programming, 13 1.4.2 Metabolic Networks, 14 1.4.3 Mixed-Integer Nonlinear Programming, 14 2 Iterative Refinement for Linear Programming 15 2.1 Introduction to Linear Programming . 16 2.1.1 Basic Concepts, 16 2.1.2 Algorithms, 18 2.1.3 Numerics, 19 2.1.4 Iterative Refinement for Linear Systems of Equations, 22 2.2 An Iterative Refinement Scheme for Linear Programs . 23 2.2.1 The Basic Algorithm, 23 2.2.2 Convergence, 28 2.2.3 Oracle-Polynomial Running Time, 31 2.2.4 Arithmetic Precision, 33 v vi Contents 2.3 Handling LPs of General Form . 34 2.3.1 Variables with General Bounds, 34 2.3.2 Inequality Constraints, 35 2.4 Refining Infeasible and Unbounded LPs . 37 2.4.1 Testing Feasibility, 38 2.4.2 Approximate Proofs of Infeasibility, 40 2.4.3 Testing Unboundedness, 44 2.4.4 An Integrated Refinement Algorithm, 45 2.5 Computational Study . 47 2.5.1 Implementation, 47 2.5.2 Experimental Setup, 50 2.5.3 Computational Results, 51 2.6 Conclusion . 55 3 Exact Linear Programming over the Rational Numbers 57 3.1 Background and Previous Work . 57 3.1.1 Edmonds’ Q-Pivoting, 58 3.1.2 Gartner’s¨ Mixed-Precision Simplex Algorithm, 58 3.1.3 QSopt ex’s Incremental Precision Boosting, 59 3.1.4 Diophantine Approximation Algorithms, 60 3.1.5 Ellipsoid and Interior Point Methods, 62 3.1.6 Rigorous Bounds, 63 3.2 Exact Bases via Iterative Refinement . 64 3.2.1 A First Experiment, 64 3.2.2 Convergence to Optimal Bases, 65 3.2.3 Iterative Refinement with Basis Verification, 73 3.3 From Approximate to Exact Solutions . 74 3.3.1 Convergence to an Optimal Solution, 74 3.3.2 Interior Point Projections and Lattice Reduction, 76 3.3.3 Reconstruction of Rational Limit Points, 76 3.3.4 Exploiting Common Factors in Vector Reconstruction, 79 3.4 Computational Study . 81 3.4.1 Exact Solution of General Linear Programs, 81 3.4.2 Implementation, 81 3.4.3 Experimental Setup, 82 3.4.4 Computational Results, 83 3.5 Conclusion . 88 4 Accurate Optimization over Multiscale Metabolic Networks 91 4.1 Background . 91 4.1.1 Flux Balance Analysis, 92 4.1.2 Integrated Models for Metabolism and Gene Expression, 93 Contents vii 4.1.3 Increased Demand for Numerically Reliable LP Solvers, 93 4.2 Iterative Refinement for Metabolic Networks .