Advances in Robust and Adaptive Optimization: Algorithms, Software, and Insights by Iain Robert Dunning B.E.(Hons), University of Auckland (2011) Submitted to the Sloan School of Management in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY IN OPERATIONS RESEARCH at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY June 2016 ○c Massachusetts Institute of Technology 2016. All rights reserved. Author................................................................ Sloan School of Management May 13, 2016 Certified by. Dimitris Bertsimas Boeing Professor of Operations Research Thesis Supervisor Accepted by . Patrick Jaillet Dugald C. Jackson Professor, Department of Electrical Engineering and Computer Science Co-director, Operations Research Center 2 Advances in Robust and Adaptive Optimization: Algorithms, Software, and Insights by Iain Robert Dunning Submitted to the Sloan School of Management on May 13, 2016, in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY IN OPERATIONS RESEARCH Abstract Optimization in the presence of uncertainty is at the heart of operations research. There are many approaches to modeling the nature of this uncertainty, but this thesis focuses on developing new algorithms, software, and insights for an approach that has risen in popularity over the last 15 years: robust optimization (RO), and its extension to decision making across time, adaptive optimization (AO). In the first chapter, we perform a computational study of two approaches for solving RO problems: “reformulation” and “cutting planes”. Our results provide useful evidence for what types of problems each method excels in. In the second chapter, we present and analyze a new algorithm for multistage AO problems with both integer and continuous recourse decisions. The algorithm oper- ates by iteratively partitioning the problem’s uncertainty set, using the approximate solution at each iteration. We show that it quickly produces high-quality solutions. In the third chapter, we propose an AO approach to a general version of the process flexibility design problem, whereby we must decide which factories produce which products. We demonstrate significant savings for the price of flexibility versus simple but popular designs in the literature. In the fourth chapter, we describe computationally practical methods for solving problems with “relative” RO objective functions. We use combinations of absolute and relative worst-case objective functions to find “Pareto-efficient” solutions that combine aspects of both. We demonstrate through three in-depth case studies that these solutions are intuitive and perform well in simulation. In the fifth chapter, we describe JuMPeR, a software package for modeling RO and AO problems that builds on the JuMP modeling language. It supports many features including automatic reformulation, cutting plane generation, linear decision rules, and general data-driven uncertainty sets. Thesis Supervisor: Dimitris Bertsimas Title: Boeing Professor of Operations Research 3 4 Acknowledgments I would like to acknowledge and thank my supervisor Dimitris Bertsimas, who has been both a friend and mentor to me during my time at the ORC. I’ll always appre- ciate his patience during the early years of the degree, and his encouragement as I started to hit my stride. I am grateful to my doctoral committee members Juan Pablo Vielma and Vivek Farias, as well as Patrick Jaillet for serving on my general exami- nation committee and co-advising me during my first two years. I’m also grateful to Laura Rose and Andrew Carvalho for all their help over the years. I must especially thank Miles Lubin for being a constant friend throughout the last four years, and an outstanding collaborator. I’d also like to thank Joey Huchette for joining us on our Julia-powered optimization mission, and all the Julia contributors at both MIT and around the world. I’d like to thank everyone in my cohort for the conversations, the parties, the studying, and the support. I’d especially like to acknowledge John Silberholz, who has been an excellent friend and collaborator. I’d also like to acknowledge everyone who I worked with on the IAP software tools class, as well as all my co-TAs throughout the years. Special mention must go to Martin Copenhaver, Jerry Kung, and Yee Sian Ng, for all the conversations about life, the universe, and everything. Finally, I’d like to acknowledge my family for their support not only during my time at MIT, but over my whole life – I would not be where or what I am today without them. I’d like to thank my friends David and Alec who have been supporting me for almost 15 years now – looking forward to many more. Last, but not least, I’d like to acknowledge my wife Kelly, who changed my life and makes it all worthwhile. 5 6 Contents 1 Introduction 15 1.1 Overview of Robust Optimization . 17 1.2 Overview of Adaptive Optimization . 21 1.3 Overview of Thesis . 22 2 Reformulation versus Cutting-planes for Robust Optimization 27 2.1 Introduction . 27 2.2 Problem and Method Descriptions . 30 2.2.1 Cutting-plane method for RLO . 31 2.2.2 Cutting-plane method for RMIO . 32 2.2.3 Polyhedral uncertainty sets . 36 2.2.4 Ellipsoidal uncertainty sets . 38 2.3 Computational Benchmarking . 39 2.3.1 Results for RLO . 42 2.3.2 Results for RMIO . 43 2.3.3 Implementation Considerations . 47 2.4 Evaluation of Hybrid Method . 50 2.4.1 Results for RLO . 51 2.4.2 Results for RMIO . 52 2.4.3 Implementation Considerations . 53 2.5 Discussion . 56 7 3 Multistage Robust Mixed Integer Optimization with Adaptive Par- titions 59 3.1 Introduction . 59 3.2 Partition-and-bound Method for Two-Stage AMIO . 65 3.2.1 Partitioning Scheme . 66 3.2.2 Description of Method . 69 3.2.3 Calculating Lower Bounds . 73 3.2.4 Worked Example: Two-stage Inventory Control Problem . 74 3.2.5 Incorporating Affine Adaptability . 77 3.3 Analysis of the Partition-and-Bound Method . 78 3.3.1 Convergence Properties . 78 3.3.2 Growth in Partitioning Scheme . 80 3.4 Partition-and-Bound Method for Multistage AMIO . 83 3.4.1 Multistage Partitions and Nonanticipativity Constraints . 85 3.4.2 Multistage Method and Implementation . 88 3.5 Computational Experiments . 89 3.5.1 Capital Budgeting . 90 3.5.2 Multistage Lot Sizing . 92 3.6 Concluding Remarks . 97 4 The Price of Flexibility 99 4.1 Introduction . 99 4.2 Process Flexibility as Adaptive Robust Optimization . 103 4.2.1 Adaptive Robust Optimization Model . 104 4.2.2 Relative Profit ARO Model . 105 4.2.3 Solving the ARO Model . 108 4.2.4 Selecting the Uncertainty Set . 110 4.2.5 Extensibility of Formulation . 111 4.3 Pareto Efficiency in Manufacturing Operations . 113 4.3.1 Pareto Efficiency for Two-stage Problems . 114 8 4.3.2 Relaxed Pareto Efficiency . 118 4.4 Insights from Computations . 119 4.4.1 Tractability of Adaptive Formulation . 120 4.4.2 Impact of Parameter Selections . 122 4.4.3 Utility of Pareto Efficient Designs . 126 4.4.4 Quality of Designs . 128 4.4.5 Price of Flexibility . 130 4.5 Concluding Remarks . 132 5 Relative Robust and Adaptive Optimization 135 5.1 Introduction . 135 5.2 Offline Problem is a Concave Function of Uncertain Parameters . 141 5.2.1 Reformulation of Robust Regret Problem . 142 5.2.2 Reformulation of Competitive Ratio Problem . 144 5.2.3 Integer Variables in Offline Problem . 145 5.3 Offline Problem is a Convex Function of Uncertain Parameters . 147 5.3.1 Reformulating the Inner Problem as a MILO Problem . 148 5.4 Combining Objective Functions – Pareto-efficient Solutions . 151 5.4.1 Worked Example: Portfolio Optimization . 153 5.5 Computations and Insights . 155 5.5.1 Minimum-cost Flow . 156 5.5.2 Multistage Inventory Control . 162 5.5.3 Two-stage Facility and Stock Location . 166 5.6 Conclusions . 172 6 JuMPeR: Algebraic Modeling for Robust and Adaptive Optimiza- tion 177 6.1 Overview of JuMPeR . 180 6.1.1 Uncertain Parameters and Adaptive Variables . 182 6.1.2 Expressions and Constraints . 183 6.2 Uncertainty Sets . 184 9 6.3 Case Studies . 188 6.3.1 Portfolio Construction . 189 6.3.2 Multistage Inventory Control . 191 6.3.3 Specialized Uncertainty Set (Budget) . 194 6.4 Comparisons with Other Tools . 199 A Notes for “Multistage Robust MIO with Adaptive Partitions” 203 A.1 Comparison of partitioning schemes for a large gap example . 203 A.2 Admissability of rule for enforcing nonanticipativity . 205 A.3 Comparison of partitioning schemes for multistage lot sizing . 206 B Reformulation of Stock Location Problem from “Relative Robust” 211 10 List of Figures 2-1 Performance profiles for RLO instances . 48 2-2 Performance profiles for RMIO instances . 49 2-3 Hybrid method runtimes . 54 3-1 Partitioning of simple uncertainty sets using Voronoi diagrams . 69 3-2 Visualization of the partitions and active uncertain parameter tree across multiple iterations . 71 3-3 Partitioning for Example 2 without adjustments for multistage case . 85 3-4 Partitioning for Example 2 with adjustments for multistage case . 89 3-5 Capital budgeting: partitions with iteration, and improvement versus time . 91 3-6 Capital budgeting: improvement and bound gap versus time . 93 3-7 Multistage lot sizing problem: improvement and bound gap versus time 96 4-1 Examples of process flexibility designs . 101 4-2 Progress of a MIO solver for a flexibility instance . 122 4-3 Visualization of MIO gap versus time for flexibility problem . 123 4-4 Visualization of time to reach 90% MIO gap for flexibility problem . 123 4-5 Parameters (Γ,B) versus objective function value for “long chain” and ”dedicated” designs for flexibility problem . 125 4-6 Parameters (휌,B) versus objective function value for “long chain” and ”dedicated” designs for flexibility problem .