Dissertation Advanced acceleration techniques for Nested Benders decomposition in Stochastic Programming Christian Wolf, M.Sc. Schriftliche Arbeit zur Erlangung des akademischen Grades doctor rerum politicarum (dr. rer. pol.) im Fach Wirtschaftsinformatik eingereicht an der Fakultät für Wirtschaftswissenschaften der Universität Paderborn Gutachter: 1. Prof. Dr. Leena Suhl 2. Prof. Dr. Csaba I. Fábián Paderborn, im Oktober 2013 Acknowledgements This thesis is the result of working for nearly four years at the Decision Support & Opera- tions Research (DS&OR) Lab at the University of Paderborn. I would like to thank those people whose support helped me in writing my dissertation. First of all, I thank my supervisor Leena Suhl for giving me the possibility to pursue a thesis by offering me a job at her research group in the first place. Her support and guidance over the years helped me tremendously in finishing the dissertation. I also thank Achim Koberstein for introducing me to the field of Operations Research through the lecture “Operations Research A” and the opportunity to do research as a student in the field of stochastic programming. It is due to his insistence that I decided to write my computer science master’s thesis at the DS&OR Lab. I am very grateful to have Csaba Fábián as my second advisor. He not only gave valuable advice but also pointed me towards the on-demand accuracy concept. Collaborating with him has been both straightforward and effective. My present and past colleagues at the DS&OR Lab deserve a deep thank you for the enjoyable time I had at the research group in the last years. Discussions on scientific topics were as insightful as our non-work related discussions were humorous. I would like to thank Corinna Hallmann, Stefan Kramkowski, Daniel Rudolph, and Franz Wesselmann for their time and patience in discussing implementation details and other problems that have appeared out of nowhere during the implementation of solver software. I thank my office colleague Boris Amberg for funny and interesting conversations across two monitors and for leaving the office to me until lunch. Last but not least, I thank my family and friends, especially my parents, who believed in me and supported me. Special thanks go to my wife Pia. Finishing the thesis without her would have been much more difficult. Paderborn, October 2013 Christian Wolf iii Contents 1. Introduction 1 I. Fundamentals 5 2. Stochastic Programming Preliminaries 7 2.1. Mathematical Programs . .7 2.2. Stochastic Programs . 12 2.2.1. Basic Probability Theory . 13 2.2.2. Two-Stage Stochastic Programs . 14 2.2.3. Multi-Stage Stochastic Programs . 15 2.2.4. Basic Properties . 17 3. Solution Methods 21 3.1. Scenario Tree . 21 3.2. Deterministic Equivalent . 23 3.3. Benders Decomposition . 25 3.4. Lagrangean Relaxation . 29 3.5. Approximative Solution Methods . 31 3.5.1. Exterior Sampling . 32 3.5.2. Interior Sampling . 32 3.5.3. Scenario Tree Generation . 33 II. State-of-the-Art 35 4. Benders Decomposition 37 4.1. Notational Reconcilation . 37 4.2. Aggregates . 38 4.3. Stabilizing the master problem . 41 4.3.1. Regularized Decomposition . 42 4.3.2. Level Decomposition . 43 4.3.3. Trust-Region Method . 44 4.4. Cut Generation . 44 4.5. Solving Similar Subproblems . 45 5. Nested Benders Decomposition 47 5.1. Nested L-shaped method . 47 v Contents 5.2. Sequencing Protocols . 49 5.3. Parallelization . 54 5.4. Advanced Start . 58 5.5. Stage Aggregation . 58 6. Modeling Languages 61 6.1. Theoretical Concepts . 61 6.2. Practical Examples . 63 7. Required Work 67 7.1. Solver Development . 67 7.2. Modeling Languages . 68 III. Advanced Techniques and Computational Results 71 8. Accelerating the Nested Benders Decomposition 73 8.1. Cut Consolidation . 73 8.2. Dynamic Sequencing . 76 8.3. Parallelization . 77 8.4. Aggregation . 79 8.5. On-Demand Accuracy . 80 8.6. Level decomposition . 83 8.7. Extending techniques to the multi-stage case . 86 9. A Modeling Environment for Stochastic Programs 91 10.Computational Results 97 10.1. Test Instances . 97 10.2. Evaluation Techniques . 98 10.3. Implementation Aspects . 100 10.3.1. Implementation . 100 10.3.2. Solving a subproblem . 101 10.3.3. Warm Start . 102 10.3.4. Tolerances . 102 10.4. Computing environment . 102 10.5. Evaluation of Two-Stage Acceleration Techniques . 103 10.5.1. Cut Aggregation . 103 10.5.2. Cut consolidation . 105 10.5.3. Level decomposition . 106 10.5.4. On-demand accuracy . 119 10.5.5. Advanced start solution . 132 10.6. Effect of Parallelization . 133 10.7. Evaluation of Multi-Stage Acceleration Techniques . 136 10.7.1. Sequencing protocols . 136 10.8. SAA and Parallel Benders . 140 vi Contents 10.9. Conclusion . 140 11.Summary and Conclusion 145 Bibliography 147 A. Test problems 161 B. Test Results 171 List of Figures 183 List of Tables 185 List of Algorithms 187 vii 1 1. Introduction “Our new Constitution is now established, and has an appearance that promises permanency; but in this world nothing can be said to be certain, except death and taxes.” — Benjamin Franklin, Letter to Jean-Baptiste Leroy Real-world optimization problems are often modeled with traditional mathematical pro- gramming techniques. The implicit assumption, when using these tools, is that the un- derlying real-world problem is deterministic. Many real-world problems usually include uncertainty, such as uncertainty about future events, lack of reliable data, etc. The model could therefore be subject to uncertainty in its parameters or in itself, because the model is an approximation of the real-world problem and thus the optimal solution of the model may not be the optimal solution of the modeled problem. Attempts to investigate the effects of uncertainty with traditional methods like sensitivity analysis of optimal solutions or scenario analysis, e.g., solving a deterministic model with different parameters, do not suffice to take the effect of uncertainty into account (e.g., (Wallace, 2000) and (King & Wallace, 2012, p. 2ff)). Thus to determine if uncertainty is of importance for a particular model, it has to be checked by incorporating uncertainty into the optimization problem. Stochastic programming is a mathematical programming field that provides techniques to handle optimization under uncertainty. It is due to the early work of Dantzig (1955), Beale (1955) and Charnes & Cooper (1959). A common concept is that a decision has to be made here and now, and the uncertain future will reveal itself after that. A recourse decision can then be taken to react upon the new information. The key questions are (King & Wallace, 2012, p. 1) • What are the important uncertainties? • How can we handle them? • Can we deliver valuable solutions and insights? It is one of the main difficulties for many practical problems to deliver a solution at all, because incorporating uncertainty makes a model usually larger and harder to solve. The usage of specialized solution techniques, e.g., Benders decomposition, Progressive Hedging, Stochastic decomposition, Sample Average Approximation, etc., make practical decision problems tractable. The resulting solutions can then be examined for valuable insights. The theoretical development and practical implementation of solution techniques is therefore important to get people to use stochastic programming in the first place and thus improve their decision making capabilities. In addition, modeling tools that aid operation research 2 1. Introduction practitioners in modeling and analyzing stochastic programs are necessary to make the transition from modeling linear programs to stochastic programs possible. The importance and widespread applicability of stochastic programming is demonstrated by its variety of application areas, including electricity, finance, supply-chain, production, telecommunications and others (see the collection edited by Wallace & Ziemba (2005)). Two recent examples demonstrate that the usage of stochastic programming leads to better decisions. A strategic gas portfolio planning problem (Koberstein et al., 2011) determines the parameters of baseload and open contracts for gas delivery for the next gas year, where recourse actions are necessary to cover the demand during a year by using storages, open contracts and the spot market. The uncertainty of the problem is the demand, which correlates with the weather conditions. As gas is widely used for heating, colder winters generate more demand than warmer winters. Incorporating this uncertainty into the model results in an expected advantage of 5.9 million euro of the stochastic solution compared with the solution from the deterministic model. The expected solution value of the stochastic program is 182.2 million euro. A company that owns wind power plants and hydro power plants has to schedule the plants operationally. The goal is to optimize the profit of the company, while satisfying customer demand (Vespucci et al., 2012). Excess energy generated from wind power plants can be used to pump water into higher reservoirs that can later be used by the hydro power plants. The wind depends on uncertain weather conditions and thus the power generated from wind power plants is subject to uncertainty. Vespucci et al. (2012) analyze a stochastic programming model that takes weather forecast uncertainty into account and contrast this with a deterministic model where the forecast is taken at face value. The stochastic programming model results in significant savings compared to the deterministic model. Implementing these and other problems is easier with algebraic modeling languages that are capable of modeling stochastic programs directly. Using specialized solution methods directly after specifying the model results in either computing time savings or opens up the possibility to solve the resulting problems in the first place. Supporting and easing this process is the topic of this thesis. The thesis is structured in three parts. The first part deals with the fundamentals. It gives an understanding of stochastic programming in Chapter 2, along with mathematical properties of these problems from which solution methods can be derived.