Domain Reduction Techniques for Global NLP and MINLP Optimization 2

arXiv:1706.08601v1 [cs.DS] 26 Jun 2017 omltduigti rmwr nldn water [ including framework design routinely this network are using ﬁelds A diverse formulated in subclasses. applications its of in variety (NLP) and programming (MIP) opti- programming nonlinear for linear programming representation mixed-integer linear (LP), includes general and very problems mization a is MINLP ecnie h olwn ie-nee nonlinear mixed-integer (MINLP): following problem programming the consider We Introduction 1 reduction domain tightening; opttoa nlsso h mato hs ehiuso h pe the problems. test on optimization 1740 techniques MINLP of these collection Keywords: and of a NLP on th impact solvers of the In global of reduction analysis domain problems. computational for optimization nonlin used nonconvex (MINLP) challenging techniques programming are for nonlinear techniques optimum reduction global integer mo Domain the including to techniques, techniques. convergence reduction presolve domain utilize and routinely solvers Optimization Abstract saalbeat available is http://creativecommons © the from Aside applications. for practical necessitated these is of constraints many or functions objective [ predictions seizure [ [ synthesis operations process chemical [ synthesis [ control [ management oanrdcintcnqe o lblNLP global for techniques reduction Domain 07 hsmnsrp eso smd vial ne h CC the under available made is version manuscript This 2017. 1 eateto hmclEgneig angeMlo Unive Mellon Carnegie Engineering, Chemical of Department 18 69 77 [ loss trim ], ,gsntok [ networks gas ], ,cytlorpi mgn [ imaging crystallographic ], s min . osritpoaain esblt-ae onstgtnn;opt tightening; bounds feasibility-based propagation; Constraint 44 t . ,poenflig[ folding protein ], https://link 94 148 ~ x ~ x g f , ( l ( ~ ∈ ~ ~ x 62 ≤ x 84 .Mdligvanonconvex via Modelling ]. n IL optimization MINLP and ) ) R ,hdoeeg systems energy hydro ], ≤ ~ x .Puranik Y. ,ha xhne network exchanger heat ], n . org/licenses/by-nc-nd/4 − ≤ 0 128 m ~ x 76 × u ,tasotto [ transportation ], . springer ,ceia process chemical ], Z itbrh A123 USA 15213, PA Pittsburgh, m        143 1 . ,robust ], 178 com/article/10 n ioasV Sahinidis V. Nikolaos and ,and ], 68 (1) ], n oe onigaedsge ostsycertain satisfy to designed [ are selection conditions bounding node branching, lower as and long as optimum global otenwsboan nate erhutlthe until search within terminate tree are to a algorithms known in Branch-and-bound converge. subdomains and applied bounds new lower recursively to the The are variables to procedures on subdomains. bounding into branching upper space by a the space as divide the exhaustively bound algorithm searches upper The the optimum. upper (near-)global with the Whenever tolerance, and terminates acceptable an lower algorithm process. within valid are search bounds a these the by throughout optimum bound global the t.Isie ybac-n-on o discrete for branch-and-bound by [ problems Inspired programming Problem solve ity. to exploited such minima of local optimization the multiple to problems. to challenge or a lead provide function and region objective feasible in the integer nonconvexities the by variables, introduced complexity combinatorial dpe o otnospolm yFl and Falk by problems continuous [ Soland for adapted . 0/ rnhadbud[ Branch-and-bound h omlpbiaino hsarticle this of publication formal The . . 1007/s10601-016-9267-5 58 .Teagrtmpoed ybounding by proceeds algorithm The ]. 89 .Frseilcss h reglobal true the cases, special For ]. frac fvroswdl available widely various of rformance a rgamn NP n mixed- and (NLP) programming ear seilyiprati peigup speeding in important especially e ipicto,reformulation simpliﬁcation, del swr,w uvytevarious the survey we work, is B-CN . license 4.0 -BY-NC-ND rbes eas rsn a present also We problems. 138 109 1 ae ehd a be can methods based ] ,bac-n-on was branch-and-bound ], ǫ acrc ( -accuracy mlt-ae bounds imality-based 1 . ogoa optimal- global to rsity, > ǫ )t a to 0) Domain reduction techniques for global NLP and MINLP optimization 2

optimum can be finitely achieved with branch-and- bound [173, 8, 38]. The success of branch-and-bound methods for global optimization is evident from the numerous software implementations available, including ANTIGONE [135], BARON [185], Couenne [25], LindoGlobal [117] and SCIP [3]. In this work, we survey the various domain reduction techniques that are employed within branch-and-bound algorithms. While these techniques are not necessary to ensure convergence to the global optimum, they typically speed up convergence. These techniques often exploit Reduced domain feasibility analysis to eliminate infeasible parts of the search space. Alternatively, the methods can also utilize optimality arguments to shrink the Original domain search space while ensuring at least one optimal solution is retained. Domain reduction techniques Figure 1: Reduction in domains leads to tighter constitute the major component of the solution methods for satisfiability problems through unit relaxations propagation [126] and for constraint programming (CP) through various filtering algorithms that achieve differing levels of consistencies [29]. They communities wherever applicable. We also present are also exploited in artificial intelligence (AI) [54] results on standard test libraries with different and interval analysis [142, 98]. Some of the other global solvers to demonstrate the impact of domain names used in the literature for these methods include reduction strategies on their performance. bound propagation, bounds tightening, bound The remainder of the paper is organized as strengthening, domain filtering, bound reduction and follows. Representation of general MINLPs through range reduction. factorable reformulations and directed acyclic graphs Mathematical-programming-based methods for is described in Section 2. Methods for constraint solving nonconvex MINLPs often rely on the solution propagation and bounds tightening in global opti- of a relaxation problem for finding a valid lower mization of MINLPs often rely on interval arithmetic. bound. The strength of the relaxations employed A brief introduction to this topic is provided in for lower bounding depends on the diameter of the Section 3. We introduce presolving for optimization feasible region. Smaller domains lead to tighter in Section 4. Domain reduction techniques that relaxations. For example, Figure 1 shows the convex rely on eliminating infeasible regions for the problem relaxation for a simple univariate concave function. are described in Section 5. Techniques that A convex relaxation for a given function defined on utilize optimality arguments for carrying out domain a nonempty convex set is a convex function that reduction are described in Section 6. Computational underestimates the given function on its domain. The tradeoffs in the implementation of some of the convex relaxation over the reduced domain provides advanced techniques are discussed in Section 7. The a better approximation for the univariate concave computational impact of many of these techniques function thereby providing better lower bounds. on the performance of widely available solvers is Domain reduction techniques not only reduce the investigated in Section 8. Finally, we conclude in diameter of the search space, but they also improve Section 9. the tightness of convex relaxations. Domain reduction techniques have been exten- 2 Representation sively studied in various communities including AI, CP, mathematical programming, interval analysis A crucial step in the branch-and-bound algorithm is and computer science where they can be viewed the construction of relaxations. One of the most as chaotic iterations [14]. The primary objective widely used methods for this purpose is the idea of this paper is to review key domain reduction of factorable reformulations. It involves splitting techniques applicable to (nonconvex) NLPs and a problem into basic atomic functions that are MINLPs and point to connections between methods utilized for computing the function, an idea exploited from constraint programming and interval arithmetic Puranik and Sahinidis 3

by McCormick [131], who developed a technique 167]. In this graph representation, variables (x,y,z) that constructed non-differentiable relaxations of and constants are leaf nodes, vertices are elementary optimization problems. Ryoo and Sahinidis [159] operations (+, −, ∗,/, log, exp, etc.) and the gain differentiability by introducing new variables functions to be represented are the root nodes. and equations for each of the intermediate functional Variable and constraint bounds are represented forms. These functional forms are simple in nature through suitable intervals for the root and leaf nodes. like the bilinear term. A similar idea was proposed Common subexpressions are combined to reduce the by Kearfott [97] where new variables and equations size of the graph as doing so is known to tighten are introduced to decompose nonlinearities to allow the resulting relaxations [185, Theorem 2]. Different for more accurate computations of interval Jacobian mathematical formulations can be generated from the matrices. same DAG depending on the needs of the solver. Consider the following example: Expressions are evaluated by propagating values from the leaves to the root node through the edges in a min 3x +4y forward mode. Backward propagation is utilized for x − y ≤ 4   the computation of slopes and derivatives. Slopes can xy ≤ 3  also be utilized to construct linear relaxations for the 2 2  (2) x + y ≥ 1  problem. Figure 2 represents the DAG for Problem 2. 1 ≤ x ≤ 5   1 ≤ y ≤ 5   3 Interval arithmetic A factorable reformulation can proceed by introducing a new variable for every nonlinearity occurring in Interval arithmetic is a system of arithmetic based on 2 2 the model. We replace z1 for x , z2 for y and z3 for intervals of real numbers [137]. An interval variable xy. is defined using a variable’s lower and upper bounds; A factorable reformulation of the model is thus the variable itself is restricted to lie between the given by: bounds. Consider the interval variables ~x = [~xl, ~xu] and ~y = [~yl, ~yu]. Addition on two intervals can be min 3x +4y defined as: x − y ≤ 4 ~x + ~y = [~xl + ~yl, ~xu + ~yu] (5) z3 ≤ 3 (3) z1 + z2 ≥ 1 The addition operator defined by equation 5 has the 2 z1 = x following property: 2 z2 = y ~x + ~y = {x + y | x ∈ ~x and y ∈ ~y} (6) z3 = xy (4) 1 ≤ x ≤ 5 Extensions to the definitions of other classic operators can be similarly defined: 1 ≤ y ≤ 5 ~x − ~y = [~xl − ~yu, ~xu − ~yl] (7) It is trivial to outer approximate the univariate terms l l l u u l u u of this model (cf. Figure 1), while the bilinear term ~x × ~y = [min(~x × ~y , ~x × ~y , ~x × ~y , ~x × ~y )] (8) in 4 may be outer approximated by its convex and 1 = [1/~xu, 1/~xl], 0 ∈/ [~xl, ~xu] (9) concave envelopes [131]. The combination of these ~x outer approximators provides a convex relaxation ~x = ~x × 1/~y, 0 ∈/ [~yl, ~yu] (10) of the original problem. In general, factorable ~y reformulations decompose the problem into simpler functional forms for which convex relaxations are These operators can be suitably defined to account known. Thus, a convex relaxation can be obtained by for infinities within the intervals and for the case reformulating a problem into its factorable form and when 0 lies in the interval of the denominator in relaxing each of the simple functional forms present a division operation [107]. Natural extensions of in the model. For example, see Algorithm Relax f(x) factorable functions can be computed by replacing in Tawarmalani and Sahinidis [185]. all the elementary operations involved in the Factorable reformulations can be conveniently computation of a factorable function with their represented through a directed acyclic graph [179, interval counterparts. A natural extension provides valid lower and upper bounds for the value of a Domain reduction techniques for global NLP and MINLP optimization 4

Ͳ1 *

Figure 2: Directed Acyclic Graph representation for Problem 2

function. functions. Numerically safe methods for computation Floating point arithmetic is susceptible to round- have also been developed by the CP community, and ing errors, which can lead to solutions being lost even are referenced in the remainder of the paper including for simple problems. Neumaier and Shcherbina [145] in Sections 5 and 6. provide an example of a simple MIP problem that leads to incorrect solutions by major commercial solvers due to roundoff errors. Interval arithmetic 4 Presolving optimization models methods utilize outward rounding to ensure no points are lost due to roundoff errors. These The idea of analyzing and converting an optimization methods are utilized to design rigorous branch-and- model into a form more amenable to fast solution is bound-based optimization and root finding methods old. Starting with the work of Brearley et al. [35], that ensure no solutions are lost due to roundoff a number of techniques have been used for analyzing errors [193, 15]. The methods bound function models in the operations research community. We values through interval arithmetic and carry out use the term presolve to denote all the techniques domain reduction through branching and fathoming used for simplification of optimization models. These tests based on monotonicity, convexity, infeasibility techniques have been developed extensively for linear as well as interval Newton type methods which programming models [189, 190, 93, 74] and for mixed- provide conditions for existence and uniqueness of integer linear programming models [157, 87, 141, solutions within a box [81, 142]. Interval-based 164, 121]. Bixby and Rothberg [31] observe that methods have been used for practical applications turning off root node presolve at the start of a like solvent blend design [177]. Interval-based solvers branch-and-bound search degrades performance of include ICOS [110], Globsol [99] and Numerica [193]. CPLEX 8.0 by a factor of 10.8, while turning off Interval-based solvers are often slower than their presolve at every node other than the root node nonrigorous counterparts. More recently, aspects of degrades the performance by a factor of 1.3 on safe computations are being introduced with different certain MIP models, demonstrating the importance parts of nonrigorous optimization algorithms. For of presolve. See Achterberg and Wunderling [6] for example, Neumaier and Shcherbina [145] describe an extensive computational analysis of the impact of methods to guarantee safety in the solution of MIPs various components of presolve algorithms for MIPs. through suitable preprocessing of the LP relaxations Some of the ideas for simplification of models include: and postprocessing of their solutions. Their methods • Elimination of redundant constraints lead to valid results even when the LP solver itself does not use rigorous methods for obtaining • Identification and elimination of dominated a solution. Borradaile and van Hentenryck [34] constraints (dominated constraints are con- provide ways of constructing numerically safe linear straints with a feasible region that is a superset underestimators for univariate and multivariate Puranik and Sahinidis 5

of the feasible region of other constraints in the the model expert to focus onto a smaller problem area model) within the model. The authors of [152] propose a deletion presolve procedure that exploits feasibility- • Elimination of redundant variables based bounds tightening techniques in order to • Assimilating singleton rows into bounds on accelerate the isolation of an IIS. variables While simplified models are often easier to solve than their original counterparts, once an optimal • Tightening bounds on dual variables solution has been obtained, the modelling system • Fixing variables at their bounds must return a solution that can be interpreted by the user with respect to the original model form • Increasing sparsity in the model that was specified. Restoration procedures to obtain • Rearrangement of variables and constraints to primal and dual solutions to the original problem are induce structure described by Andersen and Andersen [12] and Fourer and Gay [67]. Such procedures are not discussed Similar preprocessing operations can be derived for here. In the subsequent sections, we review domain nonlinear problems [10, 119, 63, 133, 75]. Some of the reduction techniques that are a major component of general guidelines include: all presolve algorithms. • Avoid potentially undefined functions • Reduce nonlinearity in the model 5 Reduction of infeasible domains • Improve scaling of model The methods described in this section eliminate • Increase convexity in the model through regions from the search space where no feasible reformulations points of Problem 1 can exist. Global optimization Amarger et al. [10] provide a software imple- algorithms maintain continuous and discrete variable mentation REFORM to carry out many of these domains through upper and lower bounds. Domain reformulations. Presolve techniques are usually reduction is achieved by making these bounds tighter. implemented at the solver level by developers of Therefore, domain reduction is also commonly various software for optimization. The modelling referred to as bounds tightening. systems AIMMS [7] and AMPL [11] also provide The tightest possible bounds based on feasibility dedicated presolve systems that are applied to of the constraints of Problem 1 can be obtained all optimization models [67, 91]. The success by solving the following problems for each of the n of presolve in mathematical programming has led variables (k =1,...,n): to efforts for its extension to general constraint min ±x programming such as automated reformulation of k ~  CP models [115, 147]. These methods can lead to s.t. g(~x) ≤ 0  l u  (11) considerable simplifications in the model, and can ~x ≤ ~x ≤ ~x Rn−m Zm also lead to a reduction in the memory required ~x ∈ ×   to solve the problem. Another advantage of these ±x in problem 11 denotes two optimization presolve methods is that they are often able to k problems, where x and −x are individually detect infeasibility in optimization models. If a k k minimized. Solution of these optimization problems presolved model is infeasible, then the original model returns the tightest possible bounds on the feasible is also infeasible. Presolve methods can also be region. If these bounds are tighter than the user- used to detect and correct the causes of infeasibilities specified bounds in the model, we can achieve for infeasible optimization models. Chinneck [46] domain reduction by using them. However, since the provides examples of simple cases where infeasibilities constraints of Problem 11 are potentially nonconvex can be correctly diagnosed by analyzing the sequence and due to the presence of integer variables, these are of reductions obtained with presolve. More recently, expensive global optimization problems, which might Puranik and Sahinidis [152] provide an automated be as hard to solve as Problem 1. A computationally infeasibility diagnosis methodology through the inexpensive way as compared to problem 11 to obtain isolation of irreducible inconsistent sets (IISs). An valid bounds for each of the variables is by solving IIS is defined as an infeasible set of constraints that the following problems for each of the n variables has every subset feasible. Isolating an IIS can help accelerate the process of model correction by allowing Domain reduction techniques for global NLP and MINLP optimization 6

(k =1,...,n): greater than bi, this also identifies infeasibility. In general, full solution of LP or NLP problems min ±xk for bounds tightening can be expensive. To balance  s.t. ~gconv(~x) ≤ 0 the computational effort involved with the reduction l u  (12) ~x ≤ ~x ≤ ~x  in bounds obtained, these techniques are usually n ~x ∈ R carried out only at the root node and/or for a   subset of variables. They are utilized only sparingly where ~gconv(~x) ≤ 0 refers to a convex relaxation through the rest of the search or not at all. In of the constraints. The convex relaxation can some cases, however, solving optimization problems be nonlinear. The integrality restrictions on the for tightening methods throughout the search has variables are relaxed as well. While bounds obtained been shown to provide significant computational from Problem 12 in general are weaker than the benefits [43]. bounds obtained from Problem 11, they require Note that reduction methods exploiting Prob- the solution of convex optimization problems and lems 11, 12 or 13 are often referred to in the literature are therefore obtained more efficiently. Convex as optimality-based bounds tightening. While relaxations are utilized in branch-and-bound methods these methods utilize the solution of optimization for obtaining valid lower bounds to the optimum, and problems, they only carry out reduction of infeasible are thus already available for bounds tightening. To regions from the search space. For this reason, exploit the efficiency and robustness of LP solvers, we prefer to classify them under feasibility-based these convex relaxations are linearized through outer reduction techniques. approximation to obtain linear relaxations [186]. Linear relaxations may provide weaker bounds than nonlinear ones, but can be solved very efficiently. 5.1 Bounds propagation techniques This linearization can also be utilized for obtaining Propagation-based bounds tightening techniques will tighter bounds on variables through the solution of be simply referred to as propagation in the remainder the following problems for each of the n variables of the paper. These techniques find their roots (k =1,...,n): in several works in the literatures of mathematical programming [35], constraint logic programming [47], min ±xk  s.t. ~glin(~x) ≤ 0 interval arithmetic [50], and AI [54]. These  (13) ~xl ≤ ~x ≤ ~xu  methods are often referred to as bounds propagation Rn techniques in CP and are a specialization of ~x ∈   constraint propagation. Davis [54] refers to a where ~glin(~x) ≤ 0 refers to a linearized outer constraint network with nodes (variables) which can approximation of the feasible region. The bounds take possible labels (domains) and are connected obtained from Problem 13 are in general weaker by constraints. He further refers to six different than the bounds obtained from Problem 12, but categories of constraint propagation based on the require the solution of linear instead of nonlinear type of information which is updated: programming problems and are therefore solved more • Constraint inference: New constraints are efficiently. inferred and added. Feasibility-based arguments for reduction can also be utilized for tightening constraints. Consider a • Label inference: Constraints are utilized to linear set of constraints Ax~ ≤ ~b. Tight bounds on restrict the sets of possible values for nodes. constraint i can be obtained with the solution of • Value inference: Nodes are partially initialized following optimization problems [164]: and constraints are utilized to complete T assignments for all nodes. min ±~ai ~x T  s.t. ~aj ~x ≤ bj j =1,...,i − 1,i +1,...,n  • Expression inference: Nodes are labelled with ~xl ≤ ~x ≤ ~xu  values expressed over other nodes. ~x ∈ Rn  • Relaxation: Nodes are assigned exact values (14) which may violate certain constraints. Problem 14 can help identify redundancy if the T maximum value of ~ai ~x is strictly less than bi. • Relaxation labelling: Nodes are assigned labels T Conversely, if the minimum value of ~ai ~x is strictly using probabilities. Updates in the network involve update of the probabilities. Puranik and Sahinidis 7

Propagation is equivalent to label inference in the AI updated to 0. From inequality 18, we can infer that community. Davis utilizes the Waltz algorithm [199] x2 ≤ 1 − min(−1, 1) = 2. Thus, the upper bound to describe bounds propagation in a constraint of x2 is updated to 2. By analyzing inequality 17 network. In the CP community, these methods again, we can update the lower bound of x1 to 2. are often utilized for solving discrete constraint The domains of the variables after these bounds satisfaction problems (CSP) [88] with backtracking- tightening steps are x1 ∈ [2, 4], x2 ∈ [0, 2] and x3 ∈ based search methods [192]. However, they are also [−1, 1]. Harvey and Schimpf [85] describe how utilized for continuous CSPs [28, 54]. bounds can be iteratively tightened in sublinear time Hager [79] discusses a reduce operator to eliminate for long linear constraints with many variables. regions outside the solution set for solving systems of Inequalities 15 and 16 indicate that only one of constraints. These methods have also been developed the bounds for a variable can be updated from extensively in the mathematical programming com- an inequality constraint based on the sign of its munity, first for LPs, and later for nonlinear program- coefficient. Achterberg [2] formalizes this through ming problems [187, 82, 80, 108, 132]. Mclinden and the concept of variable locks. Thus, inequality 17 Mangasarian [122] demonstrate inference of bounds down locks variables x1 and x2, since the lower for simple monotonic complementarity problems bounds for x1 and x2 cannot be moved arbitrarily and convex problems. Lodwick [118] analyzed without violating inequality 17. The concept of the relationship between the constraint propagation variable locks allows for efficient duality fixing of methods from AI and bound tightening methods variables. Note that there is no up lock for variable from mathematical programming. These methods x1 and no down lock for variable x3 based on the typically operate by systematically analyzing one inequalities 15 and 16. Thus, if the coefficient of x1 constraint at a time to infer valid bounds on variables. in the objective function is negative, x1 can be set Propagation techniques are computationally in- to its upper bound. Similarly, if the coefficient of x3 expensive. For example, consider a set of linear in the objective function is positive, x3 can be set to constraints: its lower bound. The concept of variables locks was n extended for CP to develop presolve procedures for a x ≤b i =1, ..., k cumulative constraints by Heinz et al. [86]. Duality X ij j i j=1 fixing is extended by Gamrath et al. [70] to allow for fixing of a singleton column. The authors also ~xl ≤ ~x ≤~xu define dominance between two variables for a MIP The following inequalities are implied by every linear and show how dominance information can be used constraint: for fixing variables at their bounds. Figure 3 considers three cases of propagation. In 1 U L Case 1, bounds for x and y are successfully tightened xh ≤ (bi − min aij xj ,aij xj ), aih > 0 aih X through iterative application of propagation, with no j6=h (15) further tightening possible via Problem 11. In Case 2, the bound obtained for x by propagation is the 1 U L tightest, however the bound for y can be tightened xh ≥ (bi − min aij xj ,aij xj ), aih < 0 aih X further by Problem 11. In Case 3, bounds for neither j6=h (16) x nor y can be tightened by propagation, whereas significant reduction is possible via Problem 11. If these inequalities imply tighter bounds on xh than the ones specified by the model, the bounds In general, reduction in domain through propa- can be updated. For example, consider the set of gation is not guaranteed. Consider the following inequalities: example:

x1 + x2 ≥ 0 x1 + x2 ≥ 4 (17) x1 + x2 ≤ 4 x2 + x3 ≤ 1 (18) −x1 + x2 ≥−2 x1 ∈[−2, 4] −x1 + x2 ≤ 2 x2 ∈[0, 4] x1 ∈[−3, 5] x3 ∈[−1, 1] x2 ∈[−3, 5] From inequality 17, we can infer x1 ≥ 4 − max(0, 4) = 0. Thus, the lower bound of x1 is The tightest possible domain for x1 and x2 Domain reduction techniques for global NLP and MINLP optimization 8

1. 2.

Figure 3: Domain reductions through linear propagation based on feasibility arguments is [−1, 3] and can be and the use of a constraint violation measure in obtained by solving Problem 11. However, analyzing the case of an infeasible subproblem to aggregate constraints one-at-a-time leads to no reduction of constraints. Their method shows computational bounds for this problem. The reason for this benefits in reducing the cluster effect (Section 6.4). drawback is that propagation only analyzes one The notion of global constraints in CP [154] is constraint at a time, whereas Problem 11 utilizes related. A global constraint provides a concise information from all the constraints in the model representation for a set of individual constraints. simultaneously. Filtering algorithms on a global constraint effectively Belotti [22] considers bound reduction by generat- carry out domain reduction by considering multiple ing a convex combination of two linear inequalities individual constraints simultaneously. Lebbah and utilizing this combination for bound reduction. et al. [111] present a global constraint for a The author proposes a univariate optimization set of quadratic constraints and describe filtering problem to determine the optimal value for the algorithms. Similar ideas have been extended to convex multipliers. However, the computational polynomial constraints [113]. complexity of considering all possible combinations Bounds can also be inferred for nonlinear of m constraints in n variables is O(m2n3). The constraints via propagation. Consider a bilinear term author proposes a heuristic scheme which, when of the form xi = xj xk. Then, implemented only on a subset of the nodes of L L L U U L U U the branch-and-bound tree, shows computational xi ≤ max{xj xk , xj xk , xj xk , xj xk } (19) benefits of using this method. For other presolve- based methods for mixed-integer programs analyzing and more than one constraint at a time, see Achterberg x ≥ min{xLxL, xLxU , xU xL, xU xU } (20) et al. [4]. Domes and Neumaier [55] propose i j k j k j k j k the generation of a new constraint by taking the represent valid bounds for variable xi and can be used linear combination or aggregation of all constraints to potentially tighten any user-specified bounds for for quadratic problems. This new constraint can this variable. uncover new relationships between the variables In the context of factorable reformulations, an and can lead to domain reduction. The authors optimization problem is already decomposed into propose the use of the dual multipliers of a simple functional forms, which can be readily utilized local solution in the case of a feasible subproblem Puranik and Sahinidis 9

for domain propagation. Consider, for instance, 5.3 Consistency the factorable reformulation for Problem 2. From Constraint programming algorithms for constraint constraint 3, we can infer that 0 ≤ z3 ≤ 3. From constraint 4, we can infer that 1 ≤ y ≤ 3. satisfaction problems rely on the formalized notion The iterative application of simple constraints such of consistency to propagate domains along constraint as 15–16 and 19–20 is referred to as poor man’s LPs networks and remove values from variable domains and poor man’s NLPs [173, 160] because this iterative that cannot be a part of any solution. Differing application is an inexpensive way to approximate levels of consistency exist, including arc consistency the solution of linear and nonlinear optimization and k-consistency, while various algorithms have problems that aim to reduce domains of variables. been proposed to achieve these consistencies [29]. Propagation based on these techniques can be applied These concepts were originally proposed for discrete not only to the original problem constraints but also problems but were also extended to continuous to any cutting planes and other valid inequalities that constraint satisfaction problems [116, 27, 162, 26, 49, might be derived by the branch-and-bound solution 30]. algorithm. Moreover, all these techniques can be The domain reduction algorithms typically em- implemented efficiently via the DAG representation. ployed for global optimization of MINLPs are usually If a bound on a variable x has changed, it can not iterated until they reach a certain level of be propagated to other variables that depend on x consistency because attempting to establish consis- through a simple forward propagation on the DAG tency can lead to a prohibitively large computational based on interval arithmetic. Similarly, the variables effort. However, domain reduction techniques are on which x depends can be updated through a not even necessary to prove convergence of branch- backward propagation. and-bound based methods for global optimization. In contrast to CP methods, the availability of relaxation methods for generating bounds allows 5.2 Convergence of propagation the mathematical-programming-based branch-and- bound algorithms to converge without establishing Propagation methods can be carried out iteratively any levels of consistency. as long as there is an improvement in variable bounds. However, these methods can fail to reach a fixed point finitely. Consider as an example [144]: 5.4 Techniques for mixed-integer linear pro- x1 + x2 = 0, x1 − qx2 = 0 with q ∈ (0, 1). grams Propagation will converge to (0, 0) at the limit, i.e., as the number of iterations approaches infinity. On Many techniques for presolving LPs can also be the contrary, any linear programming based method directly utilized for MIPs. However, specialized will terminate at (0, 0) quickly as the only feasible reduction methods can be developed for MIPs. For point. A necessary condition for nonconvergence of example, a number of methods have been developed propagation iterations is the presence of cycles in for the analysis of 0-1 binary programs [78, 51, expression graphs [59]. The problem of achieving 164]. Probing and shaving techniques are equivalent a fixed point with propagation iterations in the to the singleton consistency techniques from CP. presence of only integer variables has been shown to Singleton consistency techniques [150] proceed by be NP-complete [32, 33]. The fixed point obtained assigning a fixed value to a variable and carrying out is independent of the order in which the variables propagation. If this assignment leads to infeasibility are considered [41]. In practice, the iterations are of the constraint program, the value assigned to the usually terminated when the improvement in variable variable cannot be a part of any solution and can domains is insignificant or when an upper limit on the be eliminated. Probing techniques, that can be used number of iterations is reached. for binary programs, involve the fixing of a binary For linear inequalities in the presence of continuous variable xi to say 0. If basic preprocessing and variables alone, the fixed point can be obtained in other domain reduction methods are able to prove polynomial time by the solution of a large linear infeasibility for this subproblem, then the variable xi program [23]. For nonlinear problems and for can be fixed to 1. If fixing the binary variable xi to problems with integer variables, a linear relaxation both 0 and 1 lead to infeasibility, the problem can can be solved to obtain reduced bounds [24]. be declared as infeasible. For binary programs, if probing is carried on all binary variables, it leads to singleton consistency. Probing has also been utilized for satisfiability problems [120]. A similar technique Domain reduction techniques for global NLP and MINLP optimization 10

for continuous problems is called shaving [129, 144]. MIP formulation, lead to tighter LP relaxations The method proceeds by removing a fraction of the for the problem. Achterberg et al. [5] show how l u l u domain of a variable [xi, xi ] as either [xi + ǫ, xi ] or implications can be derived from conflicts (See l u [xi, xi − ǫ] and testing whether the reduced domain Section 5.5) and from knapsack covers. leads to provable infeasibility of the model. If Tighter formulations can also be obtained for a infeasibility is indeed proved, the domain of the problem through coefficient reduction. Consider the variable can be reduced to the complement of the example of a linear constraint on binary variables box used for testing. The author suggests using from [127]: u l ǫ =0.1×(xi −xi) for this method. Shaving is widely used in the sub-field of CP based scheduling [191]. −230x10 − 200x16 − 400x17 ≤−5 Belotti et al. [25] call this technique aggressive bounds tightening (ABT) and implement it in the The coefficients of this constraint can be reduced to: solver COUENNE. Once a fraction of the domain −x10 − x16 − x17 ≤−1 of a variable has been eliminated, propagation is invoked in the hopes of proving infeasibility in the While the set of binary values satisfying both of subproblem. However, ABT and shaving techniques the above constraints are the same, the reduced in general are expensive and reduction in the domain constraint has a tighter LP relaxation. Approaches is not guaranteed. Faria and Bagajewicz [62] propose for coefficient reduction are provided by [51, 164]. several variants of this shaving strategy for bilinear Andersen and Pochet [13] prove that, if no coefficients terms and utilize it in a branch-and-bound algorithm for an MIP system can be strengthened, then there for water management and pooling problems [61, 60]. does not exist a dominating constraint that can be Nannicini et al. [140] propose a similar strategy which used to replace an existing constraint in the MIP they refer to as aggressive probing. In contrast to system to tighten its relaxation. They also describe ABT technique, the nonconvex restrictions created an optimization formulation and an algorithmic by restricting a variable domain in their method are solution to the problem of strengthening a coefficient solved to global optimality with branch-and-bound in a constraint as much as possible. in order to carry out the maximum possible domain reduction. Implication-based reductions utilize relations be- 5.5 Conflict analysis tween the values of different variables that must be Branch-and-bound based algorithms often encounter satisfied at an optimal solution. For instance, if infeasibility in subproblems during the search for fixing two binary variables x and x to 0 leads to i j global optimum. Solvers for the satisfiability problem infeasibility, then we have an implication requiring (SAT) also utilize a backtracking-based branching that one of the variables must be nonzero, which scheme for their solution [139]. The SAT problem can be represented by the inequality x + x ≥ 1. i j consists of binary variables constrained by a set of Such relations can be efficiently represented through logical conditions. The SAT problem has a solution the use of conflict graphs [17]. Conflict graphs are if there exists an instantiation of the binary variables utilized to generate valid inequalities that strengthen satisfying all the logical conditions. SAT-based the MIP formulation. In general, if fixing a binary solvers learn and add conflict clauses from infeasible variable x to 0 implies that variable x must take i j subproblems [126]. These clauses are created value v, then the following inequalities are valid for by identifying the corresponding instantiations of the problem [121]: a subset of variables leading to infeasibility and u prohibiting them. This often leads to a reduction xj ≤ v + (xj − v)xi in the search tree. The idea of conflict analysis l xj ≥ v − (v − xj )xi is similar to the idea of no-good learning from the CP community [180, 114, 96, 100]. No-good is a Implications can be derived by carrying out generalization of a conflict clause from SAT to CP. probing by fixing more than one variable at The ideas of conflict analysis have been extended a time and/or through the analysis of problem for MIP [1, 163]. However, generation of conflict structure. Implications can also be utilized for clauses is more complicated for MIP due to the identifying and eliminating redundant constraints. presence of both continuous and general integer Inequalities derived from implications lead to variables. Infeasibility in SAT problems and CP automatic disaggregation for some constraints [164]. problems is detected through a chain of logical Disaggregated constraints, while redundant for the Puranik and Sahinidis 11

deductions caused by fixing some variables. However, be derived if, at the optimal solution L of the convex in MIP, infeasibility can be identified through either relaxation, a variable is at its lower bound, i.e., l such deductions or an infeasible LP relaxation. xj = xj , with the corresponding Lagrange multiplier Achterberg [1] proposes a generalized conflict graph λj > 0. The upper bound can then be potentially (termed implication graph in [163]) for representation tightened without losing optimal solutions by the of bound propagations that can be used to identify following cut: cause of infeasibility. Note that the conflict graph and the implication graph are defined differently than in u l U − L xj ≤ xj + (23) Section 5.4. In the case of an infeasible LP relaxation, λj Achterberg proposes identifying a minimum bound- cardinality IIS. Representation of conflict causes For variables that are not at their bounds at the for MIP requires use of disjunctive constraints relaxation solution, probing tests can be carried out which must be reformulated with additional binary by temporarily fixing variables at their bounds and solving the restricted problem. For example, Tests variables and inequalities. Limited computational u analysis demonstrates a reduction in the number of 3 and 4 in [158] work as follows. Set xj = xj and j nodes and time with conflict analysis. solve Problem 21. If the corresponding multiplier λ is positive, then the following constraint is valid:

6 Reduction of suboptimal domains l u U − L xj ≥ xj − (24) λj In contrast to the methods of the previous section, the A similar probing test can be developed by fixing a methods described here can lead to the elimination of variable at its lower bound [158, 159, 173]. Reduction feasible points from the domain under the condition by 22 and 23 only requires the solution of the that at least one globally optimal solution remains relaxation problem, and thus can be implemented within the search space. at every node of the branch-and-bound tree without Consider the following convex relaxation of much computational overhead. On the other hand, Problem 1: probing with fixing variables at their bounds requires min fconv(~x) the solution of 2n convex problems. Probing is  s.t. ~gconv(~x) ≤ 0 usually carried out only at the root node and  (21) ~xl ≤ ~x ≤ ~xu  then only at some nodes of the tree or only for Rn a subset of the variables. The probing tests ~x ∈   are a generalization of probing for the integer As discussed before, convex relaxations are often programming case. However, they differ from the constructed and linearized in a global branch-and- integer programming case in the sense that probing bound search for obtaining valid lower bounds. leads to variable fixing in the case of binary variables, Assume that the optimal objective for Problem 21 whereas in the continuous case it leads to reduction has value L and, at the optimal solution, a bound in the variable domains. u xj ≤ xj is active with a Lagrange multiplier λj > 0. Similarly, if at the optimal solution of the j Let U be a known valid upper bound for the optimal relaxation Problem 21, a constraint gconv(~x) ≤ 0 is objective function value of Problem 1. Then, the active with the corresponding multiplier µi ≥ 0, then following constraint does not exclude any optimal the following constraint does not violate any points solutions better than U (Theorem 2 in [158]): with objective values better than U [159]:

l u U − L j U − L xj ≥ xj − (22) gconv(~x) ≥− (25) λj µj

A geometric interpretation of this cut can be observed These inequalities can be appended to the original ∗ in Figure 4, where xj denotes the right-hand-side formulation. However, this process can lead to the of equation 22. The constraint excludes values of accumulation of a large number of constraints. Al- xj for which the convex relaxation is guaranteed to ternatively, these are often utilized for propagation- have its value function to be greater than or equal to based tests locally at the current node and then U. Consequently, the nonconvex problem also has its discarded. Lebbah et al. [112] provide a safe way of value function guaranteed to be greater than or equal implementing duality-based reduction techniques in to U in this domain. A corresponding cut can also the context of ﬂoating point arithmetic. It must be Domain reduction techniques for global NLP and MINLP optimization 12

Value function

Nonconvex problem Convex relaxation

Figure 4: Multipliers-based reduction geometrically

noted that suboptimal Lagrangian multipliers often can also be used for filtering in CP, see for provide greater reduction through constraints 22, example [66, 188]. 23, 24 and 25. A more detailed discussion on the Note that problem 26 can be solved iteratively to use of other than optimal multipliers is provided in obtain further tightening. However, convergence to a Tawarmalani and Sahinidis [185] and Sellmann [171]. fixed point can be slow. Caprara and Locatelli [41] If a valid upper bound U is available, it can also propose a different approach to carry out bounds be exploited by reformulating Problem 12 as follows: tightening called nonlinearities removal domain reduction (NRDR). NRDR relies on the solution of min ±xk parametric univariate optimization problems to find ~  s.t. g(~x) ≤ 0  tight bounds. The method reduces nonlinearities due  fconv(~x) ≤ U  (26) to a single variable at every iteration. The authors ~xl ≤ ~x ≤ ~xu further demonstrate that, under certain assumptions, ~x ∈ Rn  their domain reduction method is equivalent to   carrying out optimization-based bounds tightening Problem 26 differs from Problem 12 by the addition of iteratively until it reaches a fixed point. Caprara et a single constraint. In this constraint, fconv(~x) refers al. [42] show that the NRDR method leads to bounds to the convex relaxation of the objective function. It consistency for a special case of linear multiplicative can be replaced by a suitable linearization flin(~x) for programming problems with only two variables in the use in formulation 27. objective function.

min ±xk  s.t. ~glin(~x) ≤ 0  6.1 A unified theory for feasibility- and  flin(~x) ≤ U  (27) optimality-based tightening ~xl ≤ ~x ≤ ~xu Rn  ~x ∈  Tawarmalani and Sahinidis [185] provide a unified  theory of feasibility- and optimality-based bounds Zamora and Grossmann [201] refer to Problem 26 tightening techniques. This theory evolves around as the contraction subproblem. They define a Lagrangian subproblems created by the dualization contraction operation that uses the solution of of constraints. Consider the problem: Problem 26 along with multipliers-based reduction steps described in this section to carry out domain min f(~x)  reduction. Their algorithm is therefore referred to s.t. g(~~x) ≤ 0  (28) as branch-and-contract. Optimality-based arguments ~xl ≤ ~x ≤ ~xu  Rn ~x ∈   Puranik and Sahinidis 13

Deﬁne the Lagrangian subproblem as follows: be obtained as:

inf {−y0f(~x) − ~yg(~x)} µ =1/aih ~xl≤~x≤~xu ~λj = − max{aij /aih, 0} for all j 6= h The dual variables y0 and y are nonpositive. Suppose ~σj = min{aij /aih, 0} for all j 6= h an upper bound U is available for the optimal solution ~ of Problem 28. A domain reduction master problem λh =~σh =0 can be constructed as follows: T Here, µ is the dual multiplier corresponding to ~ai ~x ≤ inf h(µ0, ~µ) ~ u  bi, λj is the dual multiplier corresponding to ~xj ≤ ~xj , s.t. f(~x) ≤ µ0 ≤ U  (29) and ~σj is the dual multiplier corresponding to ~xj ≥ ~g(~x) ≤ ~µ ≤ b  l ~xj . The domain reduction master problem can be ~xl ≤ ~x ≤ ~xu  constructed as:  Using the linear objective function h(µ0, ~µ)= a0µ0 + min xh T ~ T T  ~a ~µ, Problem 29 can be equivalently stated as: s.t. −xh + µu + λ ~v − ~σ ~w ≤ 0   (32) T u ≤ bi  inf a0µ0 + ~a ~µ u  ~v ≤ ~x s.t. −y0(f(~x) − µ0)  ~w ≤ ~xl  T   −~y (~g(~x) − µ) ≤ 0 ∀(y0, ~y) ≤ 0  (30)  (µ0, ~µ) ≤ (U, 0) Equation 16 follows from Problem 32. Equation 15 ~xl ≤ ~x ≤ ~xu  can be similarly derived.   Tawarmalani and Sahinidis [185] also present a Solving Problem 30 can be as hard as solving Problem duality-based reduction scheme that utilizes dual 28. Instead, the lower bound for Problem 30 can be feasible solutions and a learning reduction heuristic. obtained from the solution of the following relaxed In branch-and-bound algorithms, branching is master problem: typically carried out by various heuristics for the T selection of the branching variable and the branching inf a0µ0 + ~a ~µ T point. If one of the nodes created due to a s.t. y0µ0 + ~y ~u   branching decision is proven to be inferior, the T ~  (31) + inf~xl≤~x≤~xu {−y0f(~x) − ~y g(x)}  learning reduction heuristic attempts to expand on ≤ 0 ∀(y0, ~y) ≤ 0  the region defined by this node that is proven to be (µ0, ~µ) ≤ (U, 0)   inferior by the construction of dual solutions for the Many domain reduction operations can be obtained other node. The authors remark that all dual feasible solutions can be utilized to carry out reductions. This by suitable choice of the coefficients (a0,~a) in the objective function for Problem 31. For example, idea is instantiated in the Lagrangian variable bounds propagation by Gleixner and coworkers [71, 72]. To equation 25 can be derived by setting (~a, a0) to ~ej in th avoid solving 2n optimization problems at every Problem 31, where ~ej is the j column of the identity matrix. node with Problem 27, valid Lagrangian variable Similarly, propagation for linear constraints inequalities can be generated by aggregating the described by equations 15 and 16 can derived by linear relaxation constraints with the dual solution application of duality theory to the relaxed problem of Problem 27. While redundant for the linear formed by relaxing all but the ith linear constraint relaxation, these inequalities approximate the effect under consideration: of solving Problem 27 locally at every node and can be used to infer stronger bounds on variables

min xh when variable bounds are updated through branching T or otherwise if a better upper bound is obtained. s.t. ~ai ~x ≤ bi Gleixner et al. [71] provides heuristics for ordering u ~x ≤ ~x the generated Lagrangian variable inequalities to ~x ≥ ~xl achieve maximum tightening. An aggressive ﬁltering strategy is proposed that involves the solution of LPs Assuming aih < 0 and ~xh is not at its lower bound, to determine variables for which bounds cannot be the optimal dual solution for the linear program can tightened. The optimization steps for these variables in Problem 27 can therefore be skipped, leading to Domain reduction techniques for global NLP and MINLP optimization 14

computational savings. The authors also provide the variables in the node to zero in another node. heuristics for ordering the LP solves in Problem 27 Thus, orbital branching implicitly prunes all the to allow for more eﬃcient warm starting and reduce other nodes which involve each of the other variables the number of simplex iterations. in the orbit ﬁxed to one.

6.2 Pruning 6.3 Exploiting optimality conditions If the lower bound obtained at a node of the branch- The Karush-Kuhn-Tucker [95, 106] conditions are and-bound search is worse than the best known upper necessary for a point to be locally optimal for a bound U, the current node can be fathomed. We nonlinear programming problem under certain con- are guaranteed that the globally optimal solution straint qualifications. See Schichl and Neumaier [168] cannot lie at this node, since all feasible solutions for a derivation and a general discussion of these at this node have an objective value greater than U. conditions. The conditions can be used to reduce This process is also referred to as pruning and is a the search space for a nonconvex NLP, since globally special case of a general concept called dominance; optimal points also satisfy them. Vandenbussche and node n1 dominates node n2 if, for every feasible Nemhauser generate valid inequalities for quadratic solution s in n2, there is a complete solution in programs with box constraints through the analysis of n1 that is as good or better than s. The idea optimality conditions [196] and also utilize them in a of dominance is old, first introduced by Kohler branch-and-cut scheme [195]. Optimality conditions and Steiglitz [105] and developed in more detail by have been extensively utilized in branch-and-bound Ibaraki [92]. Dominance relations can be utilized to algorithms for quadratic programs [83, 38, 39, 37, 45, speed up the search. For MIP problems, Fischetti 90]. Optimality conditions can also be utilized for and Toth [65] propose solution of an auxiliary MIP pruning of nodes. For example, for an unconstrained involving only fixed variables at a node to determine optimization problem, if 0 is not contained within the whether the current node is dominated by another interval inclusion function for the partial derivatives node which may or may not have been explored yet. of the objective function at a node, the corresponding However, the overhead of solving the auxiliary MIP node can be pruned [125]. This test is often referred is fairly large. Fischetti and Salvagnin [64] propose to as the monotonicity test. improvements to this scheme and demonstrate Sahinidis and Tawarmalani [161] have added a computational benefits for network loading problems modelling language construct for BARON which arising in telecommunication. Sewell et al. [172] allows for the specification of certain constraints propose a memory-based dominance rule for a as relaxation-only. Relaxation-only constraints are scheduling application. Memory-based dominance utilized for the construction of convex relaxations rules require storage of the entire search tree, and and for domain reduction, but are not utilized in their performance is dependent on the memory local search for obtaining upper bounds. The authors available. However, since information from the entire use first-order optimality conditions explicitly as tree is available, considerably stronger pruning rules relaxation-only constraints and observe improved can be determined. Memory-based search algorithms convergence for some univariate optimization prob- are related to the heuristic search algorithms from lems. Amaran and Sahinidis [9] use a similar AI [57, 182]. strategy and show significant computational benefits Problems involving integer variables often have a for parameter estimation problems. They analyze high degree of symmetry. Symmetry is detrimental their results and demonstrate that explicit use for branch-and-bound algorithms as it can lead to of optimality conditions aids in domain reduction repetitive work for the solver. Most symmetry steps of BARON leading to computational speedups. breaking approaches rely on exploiting information Puranik and Sahinidis [151] propose a strategy for about the specific problem being considered. A more carrying out implicit bounds tightening on optimality general technique called isomorphic pruning has been conditions for bound-constrained optimization prob- proposed by Margot [123, 124]. Isomorphic pruning lems. Their method does not require the generation relies on lexicographic tests to determine if a node of optimality conditions which can be time consuming can be pruned. Orbital branching [146] is another and lead to increase in memory requirements. For a method that can be utilized to tackle symmetry. large collection of test problems, this strategy leads to The branching method identifies orbits of equivalent computational speedups and reduction in the number variables. Orbital branching proceeds by fixing a of nodes. variable in the orbit to one at a node, and fixes all Puranik and Sahinidis 15

6.4 Cluster effect insights from Stergiou [181] that propagation events often occur in close clusters, Araya et al. [16] Branch-and-bounds methods often undergo repeated propose an adaptive strategy. Thus, if a constraint branching in the neighbourhood of the global solution propagation mechanism succeeds in carrying out before converging. This problem is referred to as domain reduction at a given node, it should the cluster effect and was first studied by Du and be exploited repetitively in other nodes that are Kearfott [56] in the context of interval-based branch- geometrically close to the node until the method fails. and-bound methods. Subsequent analysis [144, 169, Similar heuristics are also utilized for solution 200] also demonstrates the importance of convergence of MINLP problems. For example, the aggressive order of the bounding operation (see Definition 1 probing strategy of Nannicini et al. [140] solves in [200]). These results indicate that at least second- probing problems to global optimality and thus has order convergence is required to overcome the cluster a huge computational overhead. To avoid excessive effect. Thus, tighter relaxations can indeed help work, Nannicini et al. propose a strategy based on mitigate the cluster effect and their development is support vector machines [170] to predict when this the subject of extensive research in the area [174, aggressive probing strategy is likely to succeed based 175, 155, 176, 184, 134, 20, 102, 103, 183, 202, on the success of propagation operations. Aggressive 101, 19]. Neumaier and coworkers provide methods probing is only carried out when its chances of to construct exclusion regions for the solution of success are high. Couenne solves linear versions of systems of equations [166] and for the solution of Problem 11 in the branch-and-bound tree in all nodes optimization problems [165]. These exclusion regions up to a depth specified by a parameter L. For guarantee that no other solution can lie within the nodes at a depth d > L, the strategy is applied exclusion box around a local minimizer or a solution with a probability 2L−d. A similar strategy is for systems of equations. These boxes can then employed by ANTIGONE. The idea of propagation be eliminated from the search space. These boxes of Lagrangian variable bounds [72, 71] can also be are constructed through existence and uniqueness thought of as a means of balancing the computational tests based on Krawczyk operator or the Kantorovich effort for tightening based on solving variants of Theorem (see Chapter 1, [98]). Other methods to Problem 11. Vu et al. [198] show significant construct exclusion regions include back boxing [194] computational benefits by carrying out propagation and ǫ-inflation [130]. on a subset of the nodes and a partial subgraph of the DAG rather than the entire graph. Vu et 7 Implementation of domain reduction al. [197] present multiple strategies for combining techniques the various reduction schemes from constraint programming and mathematical programming for constraint satisfaction problems. Domain reduction strategies, if successful, typically lead to a reduction in the number of nodes searched in a branch-and-bound tree. Techniques like 8 Computational impact of domain re- propagation are computationally inexpensive and can duction be applied at every node without much overhead. However, they are not as efficient in carrying We demonstrate the computational impact of domain out domain reduction as the more computationally reduction techniques on three widely available intensive strategies like Problem 11 or probing. Thus, solvers: BARON [185], Couenne [25] and SCIP [3]. there exists a need to balance the effort involved in BARON is commercial [21] and also free through domain reduction in order to reduce the average effort the NEOS server [52]. SCIP is free for academics per node. Multiple heuristic or learning strategies and commercial for all others. Couenne is open- are employed for this purpose. These ideas are source and free software. All three solvers are important in constraint satisfaction problems where available under the GAMS modeling system [36] multiple filtering algorithms are available achieving and provide options that allow us to turn off their varying degrees of consistency. Stergiou [181] domain reduction algorithms. The global MINLP experimentally analysed the domain reduction events solvers Antigone [135] and LindoGlobal [117] are also through filtering techniques. Drawing insights available under GAMS but were not used in these from the clustering of this experimental data, experiments since they do not offer facilities that turn various heuristics are proposed for choosing the off their presolve routines. propagation algorithm on the fly. Based on The test libraries used in our tests are the Domain reduction techniques for global NLP and MINLP optimization 16

Global library [73], Princeton library [149], MINLP profile of the no-reduction version of BARON library [40] and the CMU-IBM library [48]. Global “catches up” with that of the reduction-based version and Princeton libraries consist of NLP problems, at the end of the profiles. This simply means that, while the MINLP and CMU-IBM libraries consist without reduction, BARON’s heuristics are still able of MINLP problems. While over 25% of the to come up with (near-)global solutions but the Princeton library are convex and the NLP relaxations branch-and-bound algorithm is not able to provide of the CMU-IBM library problems are all convex, sufficiently strong lower bounds in order to prove the Global and MINLP libraries contain mostly global optimality. For the MINLP case, the no- nonconvex problems. In the sequel we present results reduction-based algorithm is not even able to find for each library separately so as to demonstrate good feasible solutions. that the observed trends are not dominated by any Results in Table 3 show that turning off domain particular library, convexity, or integrality properties. reduction techniques leads to a huge increase in the Key statistics on the test sets are summarized in number of nodes required by BARON. The increase Table 1. in nodes is also accompanied by a significant increase All computational tests were run on a 64-bit Intel in computational time across all test libraries. Xeon X5650 2.66 Ghz processor running CentOS release 7. The tests were carried out with a time limit 8.2 Couenne of 500 seconds and absolute and relative optimality −6 tolerances set to 10 . All solvers were run under The solver options and their values utilized for tests two different settings: (1) default options and (2) with Couenne are summarized in Table 4. These turn default options with domain reduction turned off. off the various domain reduction techniques utilized Our main objective is to compare these two settings. in Couenne. The remaining options are utilized at Comparing the relative impact of the individual their default settings. reduction techniques on the performance of global Figures 9, 10, 11 and 12 demonstrate the solvers is beyond the scope of this work. Readers can performance of Couenne with and without domain refer to Kılın¸cand Sahinidis [104] for experiments reduction techniques employed on the Global, showing the effect of different bounds reduction Princeton, CMU-IBM and MINLP libraries. Similar strategies in BARON. In the results presented to BARON, turning off domain reduction techniques below, the suffix “nr” is used to denote a solver has a huge impact on the performance of Couenne. applied with domain reduction techniques turned off. Contrary to BARON, without reduction, this solver Comparisons are presented in terms of performance is unable to find good solutions for the NLP test profiles generated through PAVER [136]. In these libraries; as a result, the performance profiles for the profiles, a solver is considered to have solved a no-reduction-based algorithm do not catch up with problem if it obtains the best solution amongst the reduction-based algorithm. Table 5 indicates a the solvers compared in the profile within a given significant increase in computational time and the multiple of the time taken by the fastest solver that number of nodes explored in the branch-and-bound solves a problem. CAN SOLVE denotes the number search when reduction is turned off. It should be of problems in the library that can be solved with all pointed out that no comparisons are possible between the solvers compared in the profile. BARON and Couenne by looking at their respective performance profiles since these profiles depend solely 8.1 BARON on the solvers included in each figure. The solver options and their values utilized for tests 8.3 SCIP with BARON are summarized in Table 2. These turn off the various domain reduction techniques utilized The solver options and their values utilized for tests in BARON. The remaining options are utilized at with SCIP are summarized in Table 6. As before, the their default settings. remaining options are used at their default settings. Figures 5, 6, 7 and 8 demonstrate the performance Figures 13, 14, 15 and 16 demonstrate the huge of BARON with and without domain reduction impact of turning off domain reductions with SCIP techniques employed on the Global, Princeton, CMU- on the Global, Princeton, CMU-IBM and MINLP IBM and MINLP libraries. The profiles indicate libraries. Similar to Couenne, this solver is also not a huge deterioration in performance across all test able of finding good solutions for continuous problems libraries when reduction is turned off. Interestingly, when reduction is turned off. Results in Table 7 for the continuous test libraries, the performance Puranik and Sahinidis 17

Figure 5: Performance proﬁles for BARON on Global library

Figure 6: Performance proﬁles for BARON on Princeton library Domain reduction techniques for global NLP and MINLP optimization 18

Figure 7: Performance proﬁles for BARON on CMU-IBM library

Figure 8: Performance proﬁles for BARON on MINLP library Puranik and Sahinidis 19

Figure 9: Performance proﬁles for Couenne on Global library

Figure 10: Performance proﬁles for Couenne on Princeton library Domain reduction techniques for global NLP and MINLP optimization 20

Figure 11: Performance proﬁles for Couenne on CMU-IBM library

Figure 12: Performance proﬁles for Couenne on MINLP library Puranik and Sahinidis 21

Test Library Global Princeton CMU-IBM MINLP Number of problems 369 980 142 249 Avg. no. of continuous variables 1092 1355 369 346 Avg. no. of binary variables 0 0 139 235 Avg. no. of integer variables 0 0 0 24 Avg. no. of constraints 785 836 956 534

Table 1: Test set statistics

Option Value some of the more complex techniques requires the use LBTTDo 0 of smart heuristics to ensure that they are utilized MDo 0 only when domain reduction is likely. We have also presented computational results with BARON, OBTTDo 0 SCIP and Couenne on publicly available test libraries. PDo 0 The results show that domain reduction techniques TDo 0 have a significant impact on the performance of these solvers. Incorporation of domain reduction Table 2: BARON options used for tests within branch-and-bound leads to huge reductions in computational time and number of nodes required for solution. Future research in this area should focus indicate a significant deterioration in performance for on the development of domain reduction techniques SCIP as other solvers. based on sets of constraints (i.e., global filtering methods) for broad classes of structured problems but also for important science and engineering problems, 8.4 Relative solver performance such as pooling problems [153], protein folding [143], Figure 17 describes the performance of BARON, and network design problems [53, 156]. Couenne and SCIP on all test libraries aggregated together. Even without reduction, BARON References dominates over the other two solvers, suggesting that this solver has an edge over the other two solvers in [1] Achterberg, T.: Conflict analysis in mixed terms of its technology for relaxation construction, integer programming. Discrete Optimization branching schemes, and primal feasibility heuristics. 4, 4–20 (2007) However, the impact of domain reduction techniques is clear and rather substantial on all three solvers. [2] Achterberg, T.: Constraint Integer Program- Interestingly, the relative order of SCIP and Couenne ming. Ph.D. thesis, Technische Universität, changes when reduction is turned off, suggesting that Berlin (2009) SCIP is relying much more on domain reduction [3] Achterberg, T.: SCIP: solving constraint than Couenne does. Equivalently, SCIP enjoys a integer programs. Mathematical Programming substantial advantage over Couenne thanks to its Computation 1, 1–41 (2009) implementation of a more extensive set of reduction techniques. [4] Achterberg, T., Bixby, R.E., Gu, Z., Roth- berg, E., Weninger, D.: Multi-row presolve reductions in mixed integer programming. 9 Conclusions In: Proceedings of the Twenty-Sixth RAMP Symposium. Hosei University, Tokyo (2014) We have presented a review of the various domain reduction techniques proposed in literature [5] Achterberg, T., Sabharwal, A., Samulowitz, for the purpose of global NLP and MINLP H.: Stronger inference through implied literals optimization. These techniques vary in complexity from conflicts and knapsack covers. In: including simple ones like propagation to more C. Gomes, M. Sellmann (eds.) Proceedings computationally intensive ones that involve full of 10th International Conference on AI and solution of optimization subproblems. Application of OR Techniques in Constraint Programming for Domain reduction techniques for global NLP and MINLP optimization 22

Figure 13: Performance proﬁles for SCIP on Global library

Figure 14: Performance proﬁles for SCIP on Princeton library Puranik and Sahinidis 23

Figure 15: Performance proﬁles for SCIP on CMU-IBM library

Figure 16: Performance proﬁles for SCIP on MINLP library Domain reduction techniques for global NLP and MINLP optimization 24

Test Library Global Princeton CMU-IBM MINLP % increase in number of nodes 1180 261 546 802 % increase in computational time 67 70 75 47

Table 3: Performance deterioration for BARON when domain reduction is turned oﬀ

Figure 17: Performance proﬁles for all solvers on all libraries combined

Option Value [7] AIMMS: AIMMS Modeling Language (2015). aggressive fbbt no http://www.aimms.com/ branch fbbt no [8] Al-Khayyal, F.A., Sherali, H.D.: On ﬁnitely feasibility bt no terminating branch-and-bound algorithms for max fbbt iter 0 some global optimization problems. SIAM Journal on Optimization 10, 1049–1057 (2000) optimality bt no redcost bt no [9] Amaran, S., Sahinidis, N.V.: Global optimization of nonlinear least-squares problems by ﬁxpoint bt no branch-and-bound and optimality constraints. two implied bt no TOP 20, 154–172 (2012)

Table 4: Couenne options used for tests [10] Amarger, R.J., Biegler, L.T., Grossmann, I.E.: An automated modelling and reformulation system for design optimization. Computers & Combinatorial Optimization Problems, pp. 1– Chemical Engineering 16, 623–636 (1992) 11. Springer, Berlin (2013) [11] AMPL: AMPL Modeling Language. http:// [6] Achterberg, T., Wunderling, R.: Mixed integer www.ampl.com/ programming: Analyzing 12 years of progress. [12] Andersen, D.E., Andersen, K.D.: Presolving in In: M. J¨unger, G. Reinelt (eds.) Facets linear programming. Mathematical Program- of Combinatorial Optimization, pp. 449–481. ming 71, 221–245 (1995) Springer, Berlin (2013) Puranik and Sahinidis 25

Test Library Global Princeton CMU-IBM MINLP % increase in number of nodes 129 21 186 171 % increase in computational time 26 19 12 16

Table 5: Performance deterioration for Couenne when domain reduction is turned oﬀ

[13] Andersen, K., Pochet, Y.: Coefficient [23] Belotti, P., Cafieri, S., Lee, J., Liberti, L.: strengthening: A tool for reformulating mixed- Feasibility-based bounds tightening via fixed integer programs. Mathematical programming points. In: W. Wu, O. Daescu (eds.) 122, 121–154 (2010) Combinatorial Optimization and Applications, pp. 65–76. Springer, Berlin (2010) [14] Apt, K.R.: The essence of constraint propagation. Theoretical computer science [24] Belotti, P., Cafieri, S., Lee, J., Liberti, 221, 179–210 (1999) L.: On feasibility based bounds tightening (2012). http://www.optimization- [15] Araya, I., Reyes, V.: Interval Branch-and- online.org/DB HTML/2012/01/3325.html Bound algorithms for optimization and constraint satisfaction: A survey and prospects. [25] Belotti, P., Lee, J., Liberti, L., Margot, Journal of Global Optimization pp. 1–30 (2015) F., Wächter, A.: Branching and bounds tightening techniques for non-convex MINLP. [16] Araya, I., Soto, R., Crawford, B.: Adaptive Optimization Methods and Software 24, 597– filtering strategy for numerical constraint 634 (2009) satisfaction problems. Expert Systems with Applications 42, 8086–8094 (2015) [26] Benhamou, F., Goualard, F., Granvilliers, L., Puget, J.: Revising hull and box consistency. [17] Atamtürk, A., Nemhauser, G.L., Savelsbergh, In: Proceedings of the 1999 International M.W.P.: Conflict graphs in solving integer Conference on Logic Programming, pp. 230– programming problems. European Journal of 244. Massachusetts Institute of Technology Operational Research 121, 40 – 55 (2000) (1999) [18] Balakrishnan, V., Boyd, S.: Global op- [27] Benhamou, F., McAllester, D., Henten- timization in control system analysis and ryck, P.V.: CLP(intervals) revisited. In: design. In: C.T. Leondes (ed.) Control M. Bruynooghe (ed.) Proceedings of the 1994 and Dynamic Systems, Advances in Theory International Symposium on Logic program- and Applications. Academic Press, New York ming, pp. 124–138. MIT Press, Cambridge (1992) (1994) [19] Bao, X., Khajavirad, A., Sahinidis, N.V., [28] Benhamou, F., Older, W.J.: Applying interval Tawarmalani, M.: Global optimization of arithmetic to real, integer, and boolean nonconvex problems with multilinear in- constraints. The Journal of Logic Programming termediates. Mathematical Programming 32, 1–24 (1997) Computation 7, 1–37 (2015) [29] Bessiere, C.: Constraint propagation. In: [20] Bao, X., Sahinidis, N.V., Tawarmalani, M.: F. Rossi, P. van Beek, T. Walsh (eds.) Hand- Multiterm polyhedral relaxations for non- book of Constraint Programming, chap. 2, pp. convex, quadratically-constrained quadratic 29–83. Elsevier, Amsterdam (2006) programs. Optimization Methods and Software 24, 485–504 (2009) [30] Bessiere, C., Stergiou, K., Walsh, T.: Domain filtering consistencies for non-binary [21] BARON. constraints. Artificial Intelligence 172, 800–822 http://minlp.com/baron (2008) [22] Belotti, P.: Bound reduction using pairs [31] Bixby, R., Rothberg, E.: Progress in of linear inequalities. Journal of Global computational mixed integer programming–a Optimization 56, 787–819 (2013) look back from the other side of the tipping point. Annals of Operations Research 149, 37– 41 (2007) Domain reduction techniques for global NLP and MINLP optimization 26

[32] Bordeaux, L., Hamadi, Y., Vardi, M.Y.: [43] Castro, P.M., Grossmann, I.E.: Optimality- An analysis of slow convergence in interval based bound contraction with multiparametric propagation. In: C. Bessiere (ed.) Principles disaggregation for the global optimization of and Practice of Constraint Programming–CP mixed-integer bilinear problems. Journal of 2007, pp. 790–797. Springer-Verlag, Berlin Global Optimization 59, 277–306 (2014) (2007) [44] Catalão, J.P.S., Pousinho, H.M.I., Mendes, [33] Bordeaux, L., Katsirelos, G., Narodytska, N., V.M.F.: Hydro energy systems management in Vardi, M.Y.: The complexity of integer bound Portugal: Profit-based evaluation of a mixed- propagation. Journal of Artificial Intelligence integer nonlinear approach. Energy 36, 500– Research 40, 657–676 (2011) 507 (2011) [34] Borradaile, G., van Hentenryck, P.: Safe and [45] Chen, J., Burer, S.: Globally solving tight linear estimators for global optimization. nonconvex quadratic programming problems Mathematical Programming 102, 495–517 via completely positive programming. Math- (2005) ematical Programming Computation 4, 33–52 (2012) [35] Brearley, A.L., Mitra, G., Williams, H.P.: Analysis of mathematical programming prob- [46] Chinneck, J.W.: Feasibility and infeasibility in lems prior to applying the simplex algorithm. optimization. Springer, New York (2008) Mathematical programming 8, 54–83 (1975) [47] Cleary, J.G.: Logical arithmetic. Future [36] Brooke, A., Kendrick, D., Meeraus, A.: computing systems 2, 125–149 (1987) GAMS–A User’s Guide. The Scientific Press, [48] CMU-IBM open source MINLP project test Redwood City, CA (1988) set. [37] Burer, S., Chen, J.: Relaxing the optimality http://egon.cheme.cmu.edu/ibm/page.htm conditions of box QP. Computational [49] Collavizza, H., Delobel, F., Rueher, M.: Optimization and Applications 48, 653–673 Comparing partial consistencies. Reliable (2011) computing 5, 213–228 (1999) [38] Burer, S., Vandenbussche, D.: A finite branch- [50] Cornelius, H., Lohner, R.: Computing the and-bound algorithm for nonconvex quadratic range of values of real functions with accuracy programming via semidefinite relaxations. higher than second order. Computing 33, 331– Mathematical Programming 113, 259–282 347 (1984) (2008) [51] Crowder, H., Johnson, E.L., Padberg, M.: [39] Burer, S., Vandenbussche, D.: Globally solving Solving large-scale zero-one linear program- box-constrained nonconvex quadratic pro- ming problems. Operations Research 31, 803– grams with semidefinite-based finite branch- 834 (1983) and-bound. Computational Optimization and Applications 43, 181–195 (2009) [52] Czyzyk, J., Mesnier, M., Moré, J.: The NEOS server. IEEE Computational Science & [40] Bussieck, M.R., Drud, A.S., Meeraus, A.: Engineering 5, 68–75 (1998) MINLPLib–A collection of test models for mixed-integer nonlinear programming. IN- [53] DAmbrosio, C., Lodi, A., Wiese, S., Bragalli, FORMS Journal on Computing 15, 114–119 C.: Mathematical programming techniques in (2003) water network optimization. European Journal of Operational Research 243, 774788 (2015) [41] Caprara, A., Locatelli, M.: Global optimization problems and domain reduction [54] Davis, E.: Constraint propagation with strategies. Mathematical Programming 125, interval labels. Artificial intelligence 32, 281– 123–137 (2010) 331 (1987) [42] Caprara, A., Locatelli, M., Monaci, M.: [55] Domes, F., Neumaier, A.: Constraint Theoretical and computational results about aggregation for rigorous global optimization. optimality-based domain reductions. Compu- Mathematical Programming 155, 375–401 tational Optimization and Applications pp. 1– (2016) 21 (2016) Puranik and Sahinidis 27

[56] Du, K., Kearfott, R.B.: The cluster problem [68] Fügenschuh, A., Homfeld, H., Schülldorf, H., in multivariate global optimization. Journal of Vigerske, S.: Mixed-integer nonlinear problems Global Optimization 5, 253–265 (1994) in transportation applications. In: Proceedings [57] Edelkamp, S., Schroedl, S.: Heuristic search: of the 2nd International Conference on Theory and applications. Elsevier (2011) Engineering Optimization (CD-ROM), p. 14 (2010) [58] Falk, J.E., Soland, R.M.: An algorithm for separable nonconvex programming problems. [69] Furman, K.C., Sahinidis, N.V.: A critical Management Science 15, 550–569 (1969) review and annotated bibliography for heat exchanger network synthesis in the 20th [59] Faltings, B.: Arc-consistency for continuous century. Industrial & Engineering Chemistry variables. Artificial intelligence 65, 363–376 Research 41, 2335–2370 (2002) (1994) [70] Gamrath, G., Koch, T., Martin, A., Mil- [60] Faria, D.C., Bagajewicz, M.J.: Global tenberger, M., Weninger, D.: Progress in optimization of water management problems presolving for mixed integer programming. using linear relaxation and bound contraction Mathematical Programming Computation 7, methods. Industrial & Engineering Chemistry 367–398 (2015) Research 50, 3738–3753 (2011) [61] Faria, D.C., Bagajewicz, M.J.: Novel bound [71] Gleixner, A.M., Berthold, T., Müller, contraction procedure for global optimization B., Weltge, S.: Three enhancements for of bilinear MINLP problems with applications optimization-based bound tightening. ZIB to water management problems. Computers & Report pp. 15–16 (2016) Chemical Engineering 35, 446–455 (2011) [72] Gleixner, A.M., Weltge, S.: Learning and [62] Faria, D.C., Bagajewicz, M.J.: A new propagating Lagrangian variable bounds for approach for global optimization of a class of mixed-integer nonlinear programming. In: MINLP problems with applications to water Proceedings of 10th International Conference management and pooling problems. AIChE on AI and OR Techniques in Constraint Journal 58, 2320–2335 (2012) Programming for Combinatorial Optimization Problems, pp. 355–361. Springer, Berlin (2013) [63] Ferris, M.C., Munson, T.S.: Preprocessing complementarity problems. In: M.C. Ferris, [73] GLOBAL Library. O.L. Mangasarian, J. Pang (eds.) Com- http://www.gamsworld.org/global/ plementarity: Applications, Algorithms and globallib.htm Extensions, pp. 143–164. Springer, Dordrecht [74] Gondzio, J.: Presolve analysis of linear (2001) programs prior to applying an interior point [64] Fischetti, M., Salvagnin, D.: Pruning moves. method. INFORMS Journal on Computing 9, INFORMS Journal on Computing 22, 108–119 73–91 (1997) (2010) [75] Gould, N., Toint, P.L.: Preprocessing [65] Fischetti, M., Toth, P.: A new dominance for quadratic programming. Mathematical procedure for combinatorial optimization prob- Programming 100, 95–132 (2004) lems. Operations Research Letters 7, 181–187 [76] Grossmann, I.: Enterprise-wide optimization: (1988) A new frontier in process systems engineering. [66] Focacci, F., Lodi, A., Milano, M.: Cost- AIChE Journal 51, 1846–1857 (2005) based domain filtering. In: J. Jaffar (ed.) Proceedings of Principles and Practice of [77] Grossmann, I.E., Caballero, J.A., Yeomans, Constraint Programming: 5th International H.: Advances in mathematical programming Conference, pp. 189–203. Springer, Berlin for the synthesis of process systems. Latin (1999) American Applied Research 30, 263–284 (2000) [67] Fourer, R., Gay, D.M.: Experience with a [78] Guignard, M., Spielberg, K.: Logical reduction primal presolve algorithm. In: W. Hager, methods in zero-one programming-minimal D. Hearn, P. Pardalos (eds.) Large Scale preferred variables. Operations Research 29, Optimization: State of the Art, pp. 135–154. 49–74 (1981) Springer, Boston (1994) Domain reduction techniques for global NLP and MINLP optimization 28

[79] Hager, G.D.: Solving large systems of nonlinear [92] Ibaraki, T.: The power of dominance relations constraints with application to data modeling. in branch-and-bound algorithms. Journal of Interval Computations 3, 169–200 (1993) the ACM 24, 264–279 (1977) [80] Hamed, A.S.E., McCormick, G.P.: Calculation [93] Imbert, J., Hentenryck, P.V.: Redundancy of bounds on variables satisfying nonlinear elimination with a lexicographic solved form. inequality constraints. Journal of Global Annals of Mathematics and Artificial Intelli- Optimization 3, 25–47 (1993) gence 17, 85–106 (1996) [81] Hansen, E.R.: Global optimization using inter- [94] Jezowski, J.: Review of water network design val analysis. Pure and Applied Mathematics. methods with literature annotations. Industrial Marcel Dekker, New York (1992) & Engineering Chemistry Research 49, 4475– 4516 (2010) [82] Hansen, P., Jaumard, B., Lu, S.H.: An analytic approach to global optimization. Mathematical [95] Karush, W.: Minima of functions of several Programming 52, 227–254 (1991) variables with inequalities as side constraints. Master’s thesis, Department of Mathematics, [83] Hansen, P., Jaumard, B., Ruiz, M., Xiong, University of Chicago, Chicago, IL (1939) J.: Global minimization of indefinite quadratic functions subject to box constraints. Naval [96] Katsirelos, G.: Nogood processing in CSPs. Research Logistics (NRL) 40, 373–392 (1993) Ph.D. thesis, University of Toronto (2008) [84] Harjunkoski, I., Westerlund, T., Pörn, R., [97] Kearfott, R.B.: Decomposition of arithmetic Skrifvars, H.: Different transformations for expressions to improve the behavior of interval solving non-convex trim-loss problems by iteration for nonlinear systems. Computing 47, MINLP. European Journal of Operational 169–191 (1991) Research 105, 594–603 (1998) [98] Kearfott, R.B.: Rigorous Global Search: Con- [85] Harvey, W., Schimpf, J.: Bounds consistency tinuous Problems, Nonconvex Optimization techniques for long linear constraints. In: and Its Applications, vol. 13. Kluwer Academic Proceedings of TRICS: Techniques foR Imple- Publishers, Dordrecht (1996) menting Constraint programming Systems, pp. [99] Kearfott, R.B.: GlobSol user guide. Optimiza- 39–46 (2002) tion Methods and Software 24, 687–708 (2009) [86] Heinz, S., Schulz, J., Beck, J.C.: Using [100] Kell, B., Sabharwal, A., van Hoeve, W.: BDD- dual presolving reductions to reformulate guided clause generation. In: L. Michel cumulative constraints. Constraints 18, 166– (ed.) Integration of AI and OR Techniques 201 (2013) in Constraint Programming, pp. 215–230. [87] Hoffman, K.L., Padberg, M.: Improving LP- Springer (2015) representations of zero-one linear programs for [101] Khajavirad, A., Michalek, J.J., Sahinidis, branch-and-cut. ORSA Journal on Computing N.V.: Relaxations of factorable functions with 3, 121–134 (1991) convex-transformable intermediates. Mathe- [88] Hooker, J.N.: Integrated methods for optimiza- matical Programming 144, 107–140 (2014) tion. Springer Science & Business Media, New [102] Khajavirad, A., Sahinidis, N.V.: Convex en- York (2007) velopes of products of convex and component- [89] Horst, R., Tuy, H.: Global Optimization: wise concave functions. Journal of Global Deterministic Approaches, Third edn. Springer Optimization 52, 391–409 (2011) Verlag, Berlin (1996) [103] Khajavirad, A., Sahinidis, N.V.: Convex en- [90] Hu, J., Mitchell, J.E., Pang, J.: An LPCC velopes generated from finitely many compact approach to nonconvex quadratic programs. convex sets. Mathematical Programming 137, Mathematical programming 133, 243–277 371–408 (2013) (2012) [104] Kılın¸c, M., Sahinidis, N.V.: Solving MINLPs [91] Hunting, M.: A nonlinear presolve algorithm with BARON (2014). Presented at MINLP in AIMMS. An AIMMS white paper, Paragon Workshop, Pittsburgh http://http://minlp. Decision Technology, BV (2011) cheme.cmu.edu/2014/papers/kilinc.pdf Puranik and Sahinidis 29

[105] Kohler, W.H., Steiglitz, K.: Characterization [116] Lhomme, O.: Consistency techniques for and theoretical comparison of branch-and- numeric CSPs. In: International Joint bound algorithms for permutation problems. Conference on Artificial Intelligence, vol. 93, Journal of the ACM 21, 140–156 (1974) pp. 232–238 (1993) [106] Kuhn, H.W., Tucker, A.W.: Nonlinear [117] Lin, Y., Schrage, L.: The global solver in programming. In: J. Neyman (ed.) Proceedings the LINDO API. Optimization Methods and ofthe Second Berkeley Symposium on Mathe- Software 24, 657–668 (2009) matical Statistics and Probability, pp. 481–492. [118] Lodwick, W.A.: Constraint propagation, University of California Press, Berkeley (1951) relational arithmetic in AI systems and [107] Kulisch, U.W.: Complete interval arithmetic mathematical programs. Annals of Operations and its implementation on the computer. Research 21, 143–148 (1989) In: A. Cuyt, W. Krämer, W. Luther, [119] Lodwick, W.A.: Preprocessing nonlinear P. Markstein (eds.) Numerical Validation in constraints with applications to the pooling Current Hardware Architectures. Springer, problem. ORSA Journal on Computing 4, 119– Berlin (2009) 131 (1992) [108] Lamar, B.W.: An improved branch and bound [120] Lynce, I., Marques-Silva, J.: Probing-based algorithm for minimum concave cost network preprocessing techniques for propositional flow problems. Journal of Global Optimization satisfiability. In: Proceedings of 15th ICTAI, 3, 261–287 (1993) pp. 105–110 (2003) [109] Land, A.H., Doig, A.G.: An automatic method [121] Mahajan, A.: Presolving Mixed-Integer Linear for solving discrete programming problems. Programs. In: J.J. Cochran, L.A. Cox, Econometrica 28, 497–520 (1960) P. Keskinocak, J.P. Kharoufeh, J.C. Smith (eds.) Wiley Encyclopedia of Operations [110] Lebbah, Y.: ICOS: A branch and bound Research and Management Science. John Wiley based solver for rigorous global optimization. & Sons, Inc., New York (2010) Optimization Methods and Software 24, 709– 726 (2009) [122] Mangasarian, O.L., McLinden, L.: Simple bounds for solutions of monotone comple- [111] Lebbah, Y., Michel, C., Rueher, M.: A mentarity problems and convex programs. rigorous global filtering algorithm for quadratic Mathematical Programming 32, 32–40 (1985) constraints. Constraints 10, 47–65 (2005) [123] Margot, F.: Pruning by isomorphism in [112] Lebbah, Y., Michel, C., Rueher, M.: Using branch-and-cut. Mathematical Programming constraint techniques for a safe and fast 94, 71–90 (2002) implementation of optimality-based reduction. [124] Margot, F.: Exploiting orbits in symmetric In: Proceedings of the 2007 ACM symposium ILP. Mathematical Programming 98, 3–21 on Applied Computing, pp. 326–331 (2007) (2003) [113] Lebbah, Y., Michel, C., Rueher, M., Daney, [125] Markót, M.C., Schichl, H.: Bound constrained D., Merlet, J.P.: Efficient and safe global interval global optimization in the COCONUT constraints for handling numerical constraint Environment. Journal of Global Optimization systems. SIAM Journal on Numerical Analysis 60, 751–776 (2014) 42, 2076–2097 (2005) [126] Marques-Silva, J.P., Sakallah, K.A.: GRASP: [114] Lecoutre, C., Sais, L., Tabary, S., Vidal, A search algorithm for propositional satisfia- V.: Recording and minimizing nogoods from bility. IEEE Transactions on Computers 48, restarts. Journal on Satisfiability, Boolean 506–521 (1999) Modeling and Computation 1, 147–167 (2007) [127] Martin, A.: General Mixed Integer Pro- [115] Leo, K., Tack, G.: Multi-Pass High-Level gramming: Computational Issues for Branch- Presolving. In: Twenty-Fourth International and-Cut Algorithms. In: M. Jünger, Joint Conference on Artificial Intelligence D. Naddef (eds.) Computational Combinatorial (2015) Optimization: Optimal or Provably Near- Optimal Solutions, pp. 1–25. Springer, Berlin (2001) Domain reduction techniques for global NLP and MINLP optimization 30

[128] Martin, A., Möller, M., Moritz, S.: Mixed [140] Nannicini, G., Belotti, P., Lee, J., Linderoth, integer models for the stationary case of J., Margot, F., Wächter, A.: A probing gas network optimization. Mathematical algorithm for MINLP with failure prediction programming 105, 563–582 (2006) by SVM. In: T. Achterberg, J.C. Beck [129] Martin, P., Shmoys, D.B.: A new approach to (eds.) Integration of AI and OR Techniques computing optimal schedules for the job-shop in Constraint Programming for Combinatorial scheduling problem. In: H.W. Cunningham, Optimization Problems, pp. 154–169. Springer, T.S. McCormick, M. Queyranne (eds.) Inter- Berlin (2011) national Conference on Integer Programming [141] Nemhauser, G.L., Savelsbergh, M.P., Sigis- and Combinatorial Optimization, pp. 389–403. mondi, G.C.: MINTO, a mixed INTeger Springer, Berlin (1996) optimizer. Operations Research Letters 15, 47– [130] Mayer, G.: Epsilon-inflation in verification 58 (1994) algorithms. Journal of Computational and [142] Neumaier, A.: Interval methods for systems of Applied Mathematics 60, 147–169 (1995) equations. Cambridge university press (1990) [131] McCormick, G.P.: Computability of global so- [143] Neumaier, A.: Molecular modeling of proteins lutions to factorable nonconvex programs: Part and mathematical prediction of protein struc- I—Convex underestimating problems. Mathe- ture. SIAM review 39, 407–460 (1997) matical Programming 10, 147–175 (1976) [144] Neumaier, A.: Complete search in continuous [132] Messine, F.: Deterministic global optimization global optimization and constraint satisfaction. using interval constraint propagation tech- Acta numerica 13, 271–369 (2004) niques. RAIRO-Operations Research 38, 277– 293 (2004) [145] Neumaier, A., Shcherbina, O.: Safe bounds in linear and mixed-integer linear program- [133] Mészáros, C.S., Suhl, U.H.: Advanced pre- ming. Mathematical Programming 99, 283–296 processing techniques for linear and quadratic (2004) programming. OR Spectrum 25, 575–595 (2003) [146] Ostrowski, J., Linderoth, J., Rossi, F., Smriglio, S.: Orbital branching. Mathematical [134] Meyer, C.A., Floudas, C.A.: Convex envelopes Programming 126, 147–178 (2011) for edge-concave functions. Mathematical programming 103, 207–224 (2005) [147] O’Sullivan, B.: Automated Modelling and [135] Misener, R., Floudas, C.A.: ANTIGONE: Solving in Constraint Programming. In: Algorithms for coNTinuous/Integer Global Twenty-Fourth AAAI Conference on Artificial Optimization of Nonlinear Equations. Journal Intelligence (2010) of Global Optimization 59, 503–526 (2014) [148] Pardalos, P.M., Chaovalitwongse, W., [136] Mittelmann, H.D., Pruessner, A.: A server for Iasemidis, L.D., Sackellares, J.C., Shiau, automated performance analysis of benchmark- D., Carney, P.R., Prokopyev, O.A., Yatsenko, ing data. Optimization Methods and Software V.A.: Seizure warning algorithm based 21, 105–120 (2006) on optimization and nonlinear dynamics. Mathematical Programming 101, 365–385 [137] Moore, R.E., Kearfott, R.B., Cloud, M.J.: (2004) Introduction to interval analysis. Siam, Philadelphia (2009) [149] Princeton Library. http://www.gamsworld.org/performance/ [138] Morrison, D.R., Jacobson, S.H., Sauppe, J.J., princetonlib/princetonlib.htm Sewell, E.C.: Branch-and-bound algorithms: A survey of recent advances in searching, [150] Prosser, P., Stergiou, K., Walsh, T.: Singleton branching, and pruning. Discrete Optimization consistencies. In: R. Dechter (ed.) Proceedings 19, 79–102 (2016) of 6th International Conference, CP 2000 Singapore, pp. 353–368. Springer, Berlin (2000) [139] Moskewicz, M.W., Madigan, C.F., Zhao, Y., Zhang, L., Malik, S.: Chaff: Engineering an [151] Puranik, Y., Sahinidis, N.V.: Bounds tight- efficient SAT solver. In: Proceedings of the ening on optimality conditions for nonconvex 38th annual Design Automation Conference, bound-constrained optimization. Journal of pp. 530–535 (2001) Global Optimization (in preparation) Puranik and Sahinidis 31

[152] Puranik, Y., Sahinidis, N.V.: Deletion [163] Sandholm, T., Shields, R.: Nogood learning presolve for accelerating infeasibility diagnosis for mixed integer programming. In: Workshop in optimization models. INFORMS Journal on on Hybrid Methods and Branching Rules in Computing (in review) Combinatorial Optimization, Montréal, p. 138 (2006) [153] Rajagopalan, S., Sahinidis, N.V.: The pooling problem. In T. Terlaky, M. Anjos and [164] Savelsbergh, M.W.P.: Preprocessing and prob- S. Ahmed (eds.), Advances and Trends in ing for mixed integer programming problems. Optimization with Engineering Applications, ORSA Journal on Computing 6, 445–454 MOS-SIAM Book Series on Optimization, (1994) SIAM, Philadelphia (2017) [165] Schichl, H., Markót, M.C., Neumaier, A.: [154] Régin, J.C.: Global constraints: A survey. In: Exclusion regions for optimization problems. P. Van Hentenryck, M. Milano (eds.) Hybrid Journal of Global Optimization 59, 569–595 optimization: The Ten Years of CPAIOR, pp. (2014) 63–134. Springer, New York (2011) [166] Schichl, H., Neumaier, A.: Exclusion regions [155] Rikun, A.D.: A convex envelope formula for systems of equations. SIAM Journal on for multilinear functions. Journal of Global numerical analysis 42, 383–408 (2004) Optimization 10, 425–437 (1997) [167] Schichl, H., Neumaier, A.: Interval analysis on [156] R´ıos-Mercado, R.Z., Borraz-Sánchez, C.: directed acyclic graphs for global optimization. Optimization problems in natural gas trans- Journal of Global Optimization 33, 541–562 portation systems: A state-of-the-art review. (2005) Applied Energy 147, 536555 (2015) [168] Schichl, H., Neumaier, A.: Transposition [157] Roy, T.J.V., Wolsey, L.A.: Solving mixed theorems and qualification-free optimality integer programming problems using automatic conditions. SIAM Journal on Optimization 17, reformulation. Operations Research 35, 45–57 1035–1055 (2006) (1987) [169] Schöbel, A., Scholz, D.: The theoretical and [158] Ryoo, H.S., Sahinidis, N.V.: Global optimiza- empirical rate of convergence for geometric tion of nonconvex NLPs and MINLPs with branch-and-bound methods. Journal of Global applications in process design. Computers & Optimization 48, 473–495 (2010) Chemical Engineering 19, 551–566 (1995) [170] Scholkopf, B., Smola, A.J.: Learning with [159] Ryoo, H.S., Sahinidis, N.V.: A branch- kernels: Support vector machines, regulariza- and-reduce approach to global optimization. tion, optimization, and beyond. MIT press, Journal of Global Optimization 8, 107–139 Cambridge (2001) (1996) [171] Sellmann, M.: Theoretical foundations of CP- [160] Sahinidis, N.V.: Global optimization and based Lagrangian relaxation. In: M. Wallace constraint satisfaction: The branch-and-reduce (ed.) Proceedings of 10th International Confer- approach. In C. Bliek, C. Jermann, and ence, CP 2004, Toronto, Canada, pp. 634–647. A. Neumaier (eds.), Global Optimization Springer, Berlin (2004) and Constraint Satisfaction, Lecture Notes in [172] Sewell, E.C., Sauppe, J.J., Morrison, D.R., Computer Science, Vol. 2861, Springer, Berlin Jacobson, S.H., Kao, G.K.: A BB&R algorithm pp. 1–16 (2003) for minimizing total tardiness on a single [161] Sahinidis, N.V., Tawarmalani, M.: Acceler- machine with sequence dependent setup times. ating branch-and-bound through a modeling Journal of Global Optimization 54, 791–812 language construct for relaxation-specific con- (2012) straints. Journal of Global Optimization 32, [173] Shectman, J.P., Sahinidis, N.V.: A finite 259–280 (2005) algorithm for global minimization of separable [162] Sam-Haroud, D., Faltings, B.: Consistency concave programs. Journal of Global techniques for continuous constraints. Con- Optimization 12, 1–36 (1998) straints 1, 85–118 (1996) Domain reduction techniques for global NLP and MINLP optimization 32

[174] Sherali, H.D., Adams, W.P.: A hierarchy of [185] Tawarmalani, M., Sahinidis, N.V.: Global relaxations between the continuous and convex optimization of mixed-integer nonlinear pro- hull representations for zero-one programming grams: A theoretical and computational problems. SIAM Journal of Discrete Mathe- study. Mathematical Programming 99, 563– matics 3, 411–430 (1990) 591 (2004) [175] Sherali, H.D., Adams, W.P.: A hierarchy of [186] Tawarmalani, M., Sahinidis, N.V.: A poly- relaxations and convex hull characterizations hedral branch-and-cut approach to global for mixed- integer zero-one programming optimization. Mathematical Programming problems. Discrete Applied Mathematics 103, 225–249 (2005) 52(1), 83–106 (1994) [187] Thakur, L.S.: Domain contraction in nonlinear [176] Sherali, H.D., Adams, W.P.: A Reformulation- programming: Minimizing a quadratic concave Linearization Technique for Solving Discrete function over a polyhedron. Mathematics of and Continuous Nonconvex Problems, Noncon- Operations Research 16, 390–407 (1990) vex Optimization and its Applications, vol. 31. Kluwer Academic Publishers, Dordrecht (1999) [188] Thorsteinsson, E.S., Ottosson, G.: Linear relaxations and reduced-cost based propagation [177] Sinha, M., Achenie, L.E.K., Gani, R.: Blanket of continuous variable subscripts. Annals of wash solvent blend design using interval operations research 115, 15–29 (2002) analysis. Industrial & Engineering Chemistry Research 42, 516–527 (2003) [189] Tomlin, J.A., Welch, J.S.: Formal optimization of some reduced linear programming prob- [178] Smith, A.B., Sahinidis, N.V.: Optimization lems. Mathematical programming 27, 232–240 techniques for phase retrieval based on single- (1983) crystal X-ray diffraction data. In: C.A. Floudas, P.M. Pardalos (eds.) Encyclopedia of [190] Tomlin, L.A., Welch, J.S.: Finding duplicate Optimization, pp. 2858–2863. Springer, Boston rows in a linear programming model. Opera- (2009) tions Research Letters 5, 7–11 (1986) [179] Smith, E.M.B., Pantelides, C.C.: Global [191] Torres, P., Lopez, P.: Overview and possible optimisation of general process models. In: extensions of shaving techniques for job-shop I.E. Grossmann (ed.) Global Optimization problems. In: 2nd International Workshop in Engineering Design, pp. 355–386. Kluwer on Integration of AI and OR techniques in Academic Publishers, Boston (1996) Constraint Programming for Combinatorial Optimization Problems, pp. 181–186. Springer, [180] Stallman, R.M., Sussman, G.J.: Forward rea- Paderborn (2000) soning and dependency-directed backtracking in a system for computer-aided circuit analysis. [192] van Beek, P.: Backtracking search algorithms. Artificial intelligence 9, 135–196 (1977) In: F. Rossi, P. van Beek, T. Walsh (eds.) Handbook of Constraint Programming, [181] Stergiou, K.: Heuristics for dynamically chap. 4, pp. 85–134. Elsevier, Amsterdam adapting propagation in constraint satisfaction (2006) problems. AI Communications 22, 125–141 (2009) [193] Van Hentenryck, P., Michel, L., Deville, Y.: [182] Sturtevant, N.R., Felner, A., Likhachev, M., Numerica: A Modeling Language for Global Ruml, W.: Heuristic search comes of age. Optimization. The MIT Press, Cambridge, MA In: AAAI12: Proceedings of the 26th AAAI (1997) Conference on Artifical Intelligence (2012) [194] van Iwaarden, R.J.: An improved uncon- [183] Tawarmalani, M., Richard, J.P., Xiong, C.: strained global optimization algorithm. Ph.D. Explicit convex and concave envelopes through thesis, University of Colorado at Denver (1996) polyhedral subdivisions. Mathematical Pro- [195] Vandenbussche, D., Nemhauser, G.L.: A gramming 138, 531–577 (2013) branch-and-cut algorithm for nonconvex [184] Tawarmalani, M., Sahinidis, N.V.: Convex quadratic programs with box constraints. extensions and convex envelopes of l.s.c. Mathematical Programming 102, 559–575 functions. Mathematical Programming 93, (2005) 247–263 (2002) Puranik and Sahinidis 33

[196] Vandenbussche, D., Nemhauser, G.L.: A polyhedral study of nonconvex quadratic programs with box constraints. Mathematical Programming 102, 531–557 (2005) [197] Vu, X., Sam-Haroud, D., Faltings, B.: Enhanc- ing numerical constraint propagation using multiple inclusion representations. Annals of Mathematics and Artiﬁcial Intelligence 55, 295–354 (2009) [198] Vu, X., Schichl, H., Sam-Haroud, D.: Interval propagation and search on directed acyclic graphs for numerical constraint solving. Journal of Global Optimization 45, 499–531 (2009) [199] Waltz, D.: Understanding line drawings of scenes with shadows. In: P.H. Winston (ed.) The Pyschology of Computer Vision. McGraw- Hill, New York (1975) [200] Wechsung, A., Schaber, S.D., Barton, P.I.: The cluster problem revisited. Journal of Global Optimization 58, 429–438 (2014) [201] Zamora, J.M., Grossmann, I.E.: A branch and contract algorithm for problems with concave univariate, bilinear and linear fractional terms. Journal of Global Optimization 14, 217–249 (1999) [202] Zorn, K., Sahinidis, N.V.: Global optimization of general nonconvex problems with intermediate bilinear substructures. Optimization Methods and Software 29, 442–462 (2013) Domain reduction techniques for global NLP and MINLP optimization 34

Option Value presolving/maxrounds 0 presolving/components/maxrounds 0 presolving/convertinttobin/maxrounds 0 presolving/domcol/maxrounds 0 presolving/dualagg/maxrounds 0 presolving/dualinfer/maxrounds 0 presolving/gateextraction/maxrounds 0 presolving/implfree/maxrounds 0 presolving/implics/maxrounds 0 presolving/inttobinary/maxrounds 0 presolving/redvub/maxrounds 0 presolving/stuffing/maxrounds 0 presolving/trivial/maxrounds 0 presolving/tworowbnd/maxrounds 0 propagating/maxrounds 0 propagating/dualfix/maxprerounds 0 propagating/genvbounds/maxprerounds 0 propagating/obbt/freq -1 propagating/obbt/maxprerounds 0 propagating/obbt/tightintboundsprobing False propagating/probing/maxprerounds 0 propagating/pseudoobj/maxprerounds 0 propagating/redcost/maxprerounds 0 propagating/rootredcost/maxprerounds 0 propagating/vbounds/maxprerounds 0 conflict/useprop False conflict/useinflp False conflict/usepseudo False heuristics/bound/maxproprounds 0 heuristics/clique/maxproprounds 0 heuristics/randrounding/maxproprounds 0 heuristics/shiftandpropagate/freq -1 heuristics/shifting/freq -1 heuristics/vbounds/maxproprounds 0 misc/allowdualreds False misc/allowobjprop False

Table 6: SCIP options used for tests Puranik and Sahinidis 35

Test Library Global Princeton CMU-IBM MINLP % increase in number of nodes 174 152 417 56 % increase in computational time 24 33 141 18

Table 7: Performance deterioration for SCIP when domain reduction is turned oﬀ