Journal of Global Optimization manuscript No. (will be inserted by the editor) The Supporting Hyperplane Optimization Toolkit For Convex MINLP Andreas Lundell · Jan Kronqvist · Tapio Westerlund Received: June 10, 2020/ Accepted: June 10, 2020 Abstract In this paper, an open source solver for mixed-integer nonlinear programming (MINLP) problems is presented. The Supporting Hyperplane Optimization Toolkit (SHOT) combines a dual strategy based on polyhedral outer approximations (POA) with primal heuristics. The outer approximation is achieved by expressing the nonlinear feasible set of the MINLP problem with linearizations obtained with the extended supporting hyperplane (ESH) and extended cutting plane (ECP) algorithms. The dual strategy can be tightly integrated with the mixed-integer programming (MIP) subsolver in a so-called single-tree manner, i.e., only a single MIP optimization problem is solved, and the polyhedral linearizations are added as lazy constraints through callbacks in the MIP solver. This enables the MIP solver to reuse the branching tree in each iteration, in contrast to most other POA-based methods. SHOT has been released as a COIN-OR open source project, and it utilizes a flexible task-based structure making it easy to extend and modify. It is currently available as a solver in GAMS, and can be utilized in Pyomo and JuMP as well through its ASL interface. The main functionality and solution strategies available in SHOT are described in this paper, and their impact on the performance are illustrated through numerical benchmarks on all the 406 convex MINLP problems in the problem library MINLPLib. Many of the features introduced in SHOT can be utilized in other POA-based solvers as well. To show the overall effectiveness of SHOT, it is also compared to other state-of-the-art solvers on the same benchmark set. Keywords Supporting Hyperplane Optimization Toolkit (SHOT) · convex mixed-integer nonlinear programming (MINLP) · multi-tree vs single-tree algorithms · polyhedral outer approximation · extended supporting hyperplane (ESH) algorithm · extended cutting plane (ECP) algorithm Andreas Lundell Department of Information Technology and Department of Mathematics. Faculty of Science and Engineering, Åbo Akademi University, Turku, Finland. E-mail: andreas.lundell@abo.fi. Jan Kronqvist Department of Computing, Imperial College, London, UK. E-mail: [email protected]. Tapio Westerlund Process Design and Systems Engineering. Faculty of Science and Engineering, Åbo Akademi University, Turku, Finland. E-mail: tapio.westerlund@abo.fi. 2 Andreas Lundell et al. 1 Introduction The Supporting Hyperplane Optimization Toolkit (SHOT) is an open source solver for mixed- integer nonlinear programming (MINLP). It is based on the SHOT algorithm described later on in this paper, which is a combination of a dual and a primal strategy that, when considering a minimization problem, gives lower and upper bounds on the objective function value of an optimization problem; the terms primal and dual should not here be mixed up with duality of optimization problems, rather these are terms with special meaning in the context of MINLP. SHOT’s dual strategy is based on polyhedral outer approximation (POA) through supporting hyperplanes or cutting planes, of the integer-relaxed feasible set defined by the nonlinear constraints in the problem. The primal strategy consists of several deterministic primal heuristics based on, e.g., solving problems with fixed integer variables, root searches or utilizing solutions provided by the MIP solver in the dual strategy. Here the term ‘heuristic’ does not indicate that the procedures are stochastic, only that they are not guaranteed to succeed in finding a new better solution in each iteration. Although SHOT can be used for nonconvex problems as well, however without any guarantees of finding a solution, the convex functionality is the main focus of this paper. Convex MINLP is a subclass of the more general nonconvex MINLP problem class. MINLP combines the nonlinearity aspect from nonlinear programming (NLP) with the discreteness of mixed-integer programming (MIP). This allows for much flexibility when used as a modeling tool, since nonlinear phenomena and relations can be expressed in the same framework as discrete choices. Due to its generality, the solution process for problems in the MINLP class is often computationally very demanding, in fact NP hard, and even small problem instances may pose difficulties for solvers available today. This is one of the motivations behind considering the more specific convex MINLP problem type separately. The convexity assumption (either by manual determination or by automatic means) provide certain properties of convex functions that can be utilized to make the solution procedure more tractable. For example, the property that a linearization of a convex function in any point always give an underestimation of its value in all points, forms the theoretical foundation for many solution methods, e.g., those utilizing POA, such as Outer Approximation (OA) [17], as well as the Extended Cutting Plane (ECP) [69] and Extended Supporting Hyperplane (ESH) [35] algorithms. Other algorithms for convex MINLP include generalized Bender’s decomposition [24], quadratic OA [33, 61], decomposition based OA [54], and nonlinear branch-and-bound [15, 28]. However, due to the discrete nature of the MINLP problems, convexity is not enough to guarantee an efficient solution process, as the noncontinuous search-space caused by the discrete variables provides a very daunting challenge. The discreteness is normally handled by considering a so-called branch-and-bound tree that in its nodes considers smaller and easier subproblems where the discrete possibilities are gradually reduced [28]. Handling such a tree is difficult in itself as the number of nodes required are often very large, so creating an efficient MINLP solver based on the branch-and-bound methodology requires a significant development effort solely for managing the tree structure. MIP solvers such as CPLEX and Gurobi have also faced the problem of efficiently handling a large tree structure, and their performance has through the years improved steadily. The main motivation during the development of SHOT, i.e., to utilize the MIP solvers as much as possible, makes it directly benefit from the progress made in them. For example, SHOT takes direct advantage of the parallelism provided by modern computer processors; thus, since being able to perform several operations in parallel is nowadays mostly what drives the increase in computing performance, the performance of SHOT will continue to scale with The Supporting Hyperplane Optimization Toolkit 3 that of the MIP solvers. SHOT also takes full advantage of preprocessing, heuristics, integer cuts, and other features provided by the MIP solver. The MIP solver integration is mainly accomplished through so-called callbacks, which allows a third party to modify and affect the behavior of the MIP solution process. The nonlinearities in the MINLP problem are iteratively introduced into the MIP model through linear approximations with hyperplanes through the means of lazy constraints added through specialized callbacks. By adding the constraints as lazy, the MIP solver does not need to invalidate the previously built branching tree, and can continue without rebuilding it as if the constraint had always been part of the model. Also, this can efficiently be done in parallel on multiple threads. There are several different convex MINLP solving software available today. Many of these are implemented, or integrated with one or more of the mathematical modeling systems such as AIMMS [7], AMPL [19], GAMS [10] JuMP [16] or Pyomo [29]. Convex MINLP solvers available include AlphaECP [67], BONMIN [8], AOA [31], DICOPT [26], Juniper [32], Minotaur [49], Muriqui [50], Pajarito [42] and SBB [22]. Nonconvex global MINLP solvers include ANTIGONE [53], BARON [58], Couenne [4], LINDO [41], and SCIP [64]. Of the mentioned MINLP solvers, AlphaECP, AOA, BARON, BONMIN, DICOPT, SBB and SCIP are compared to SHOT in a numerical benchmark given in Sect. 7.1. Recent reviews over methods and solvers for convex MINLP are provided in [34, 63]; in [34] an extensive benchmark between convex MINLP solvers is also provided. Overviews over MINLP methods and software are also given in [13] and [25], while an older benchmark of MINLP solvers was conducted in [38]. In short, these are the major accomplishments presented in this paper: – An explanation of different strategies that can be used in POA-based solvers to enhance the performance. This includes a discussion on single- vs multi-tree approaches cf., Sect. 3 and general techniques, cf., Sect. 4. Many of these techniques are results of the research into the ESH algorithm and development of the SHOT solver. – A thorough description of the different steps taken by the SHOT solver, these are sum- marized in the SHOT algorithm, cf., Alg. 1. – Numerical comparisons of the impact of the different features developed for SHOT; many of these features are also applicable to other POA-based solvers. – An extensive benchmark of SHOT versus other state-of-the art solvers for convex MINLP. This paper is structured as follows: In Sect. 2, a short background about convex MINLP and some discussion about the terminology utilized in the paper is given. In Sect. 3, a discussion about the differences between single- and multi-tree methods is provided. The dual and primal strategies utilized in SHOT are described in Sects. 4 and 5, respectively, and in Sect. 6 the main SHOT algorithm and its solver implementation are discussed in detail. To provide insight into how different strategies in SHOT affect the performance of the solver, as well as to compare it other state-of-the-art MINLP solvers, an extensive benchmark is given in Sect. 7. Finally, some conclusions and ideas for future work are provided in Sect. 8. 2 Background SHOT is available as an open source COIN-OR project1, and is also included in GAMS (as of version 31.1). A simplified version is also available for Wolfram Mathematica2 [45]; 1 Project website: www.shotsolver.dev.