Mixed Integer Nonlinear Programming (MINLP)

A Practical Guide to Mixed Integer Nonlinear Programming (MINLP)

Sven Leyffer Jeff Linderoth MCS Division ISE Department Argonne National Lab Lehigh University [email protected] [email protected]

SIAM Conference on Optimization Stockholm, Sweden May 15, 2005

Leyﬀer & Linderoth MINLP NLP MINLP

+ =6

New Math MI

Leyﬀer & Linderoth MINLP MINLP

New Math MI NLP

Leyﬀer & Linderoth MINLP New Math MI NLP MINLP

+ =6

Leyﬀer & Linderoth MINLP 2. Classical Solution Methods 3. Modern Developments in MINLP 4. Implementation and Software

MINLP Short Course Overview

1. Introduction, Applications, and Formulations

Leyﬀer & Linderoth MINLP 3. Modern Developments in MINLP 4. Implementation and Software

MINLP Short Course Overview

1. Introduction, Applications, and Formulations 2. Classical Solution Methods

Leyﬀer & Linderoth MINLP 4. Implementation and Software

MINLP Short Course Overview

1. Introduction, Applications, and Formulations 2. Classical Solution Methods 3. Modern Developments in MINLP

Leyﬀer & Linderoth MINLP MINLP Short Course Overview

1. Introduction, Applications, and Formulations 2. Classical Solution Methods 3. Modern Developments in MINLP 4. Implementation and Software

Leyﬀer & Linderoth MINLP Motivation Examples Tricks

Part I

Introduction, Applications, and Formulations

Leyﬀer & Linderoth MINLP • f, c smooth (convex) functions • X,Y polyhedral sets, e.g. Y = {y ∈ [0, 1]p | Ay ≤ b} • y ∈ Y integer ⇒ hard problem • f, c not convex ⇒ very hard problem

Motivation What Examples How Tricks Why? The Problem of the Day

Mixed Integer Nonlinear Program (MINLP)

 minimize f(x, y)  x,y subject to c(x, y) ≤ 0  x ∈ X, y ∈ Y integer

Leyﬀer & Linderoth MINLP • y ∈ Y integer ⇒ hard problem • f, c not convex ⇒ very hard problem

Motivation What Examples How Tricks Why? The Problem of the Day

Mixed Integer Nonlinear Program (MINLP)

 minimize f(x, y)  x,y subject to c(x, y) ≤ 0  x ∈ X, y ∈ Y integer

• f, c smooth (convex) functions • X,Y polyhedral sets, e.g. Y = {y ∈ [0, 1]p | Ay ≤ b}

Leyﬀer & Linderoth MINLP • f, c not convex ⇒ very hard problem

Motivation What Examples How Tricks Why? The Problem of the Day

Mixed Integer Nonlinear Program (MINLP)

 minimize f(x, y)  x,y subject to c(x, y) ≤ 0  x ∈ X, y ∈ Y integer

• f, c smooth (convex) functions • X,Y polyhedral sets, e.g. Y = {y ∈ [0, 1]p | Ay ≤ b} • y ∈ Y integer ⇒ hard problem

Leyﬀer & Linderoth MINLP Motivation What Examples How Tricks Why? The Problem of the Day

Mixed Integer Nonlinear Program (MINLP)

 minimize f(x, y)  x,y subject to c(x, y) ≤ 0  x ∈ X, y ∈ Y integer

• f, c smooth (convex) functions • X,Y polyhedral sets, e.g. Y = {y ∈ [0, 1]p | Ay ≤ b} • y ∈ Y integer ⇒ hard problem • f, c not convex ⇒ very hard problem

Leyﬀer & Linderoth MINLP • Equilibrium • Enthalpy

• Physical Processes and Properties

• Abstract Measures • Economies of But we all know the world is Scale nonlinear! • Covariance • Utility of decisions

Motivation What Examples How Tricks Why? Why the N?

An anecdote: July, 1948. A young and frightened George Dantzig, presents his newfangled “linear programming” to a meeting of the Econometric Society of Wisconsin, attended by distinguished scientists like Hotelling, Koopmans, and Von Neumann. Following the lecture, Hotellinga pronounced to the audience:

ain Dantzig’s words “a huge whale of a man”

Leyﬀer & Linderoth MINLP • Equilibrium • Enthalpy

• Physical Processes and Properties

• Abstract Measures • Economies of Scale • Covariance • Utility of decisions

Motivation What Examples How Tricks Why? Why the N?

ain Dantzig’s words “a huge whale of a man”

Leyﬀer & Linderoth MINLP • Equilibrium • Enthalpy

• Physical Processes and Properties

• Abstract Measures • Economies of Scale • Covariance • Utility of decisions

Motivation What Examples How Tricks Why? Why the N?

ain Dantzig’s words “a huge whale of a man”

Leyﬀer & Linderoth MINLP • Abstract Measures • Economies of Scale • Covariance • Utility of decisions

Motivation What Examples How Tricks Why? Why the N?

An anecdote: July, 1948. A young and frightened George Dantzig, presents his The world is indeed newfangled “linear programming” to a nonlinear meeting of the Econometric Society of Wisconsin, attended by distinguished • Physical Processes scientists like Hotelling, Koopmans, and and Properties Von Neumann. Following the lecture, • Equilibrium Hotellinga pronounced to the audience: • Enthalpy But we all know the world is nonlinear!

ain Dantzig’s words “a huge whale of a man”

Leyﬀer & Linderoth MINLP Motivation What Examples How Tricks Why? Why the N?

Leyﬀer & Linderoth MINLP • If the variable is associated with a physical entity that is indivisible, then it must be integer 1. Number of aircraft carriers to to produce. Gomory’s Initial Motivation 2. Yearly number of trees to harvest in Norrland

Motivation What Examples How Tricks Why? Why the MI?

Leyﬀer & Linderoth MINLP Motivation What Examples How Tricks Why? Why the MI?

• We can use 0-1 (binary) variables for a variety of purposes • Modeling yes/no decisions • Enforcing disjunctions • Enforcing logical conditions • Modeling ﬁxed costs • Modeling piecewise linear functions • If the variable is associated with a physical entity that is indivisible, then it must be integer 1. Number of aircraft carriers to to produce. Gomory’s Initial Motivation 2. Yearly number of trees to harvest in Norrland

Leyﬀer & Linderoth MINLP 1. Convince the user that he or she does not wish to solve a mixed integer nonlinear programming problem at all! 2. Otherwise, solve the continuous relaxation (NLP ) and round oﬀ the minimizer to the nearest integer. • Sometimes a continuous approximation to the discrete (integer) decision is accurate enough for practical purposes. • Yearly tree harvest in Norrland • For 0 − 1 problems, or those in which the |y| is “small”, the continuous approximation to the discrete decision is not accurate enough for practical purposes. • Conclusion: MINLP methods must be studied!

Motivation What Examples How Tricks Why? A Popular MINLP Method

Dantzig’s Two-Phase Method for MINLP Adapted by Leyﬀer and Linderoth

Leyﬀer & Linderoth MINLP 2. Otherwise, solve the continuous relaxation (NLP ) and round oﬀ the minimizer to the nearest integer. • Sometimes a continuous approximation to the discrete (integer) decision is accurate enough for practical purposes. • Yearly tree harvest in Norrland • For 0 − 1 problems, or those in which the |y| is “small”, the continuous approximation to the discrete decision is not accurate enough for practical purposes. • Conclusion: MINLP methods must be studied!

Motivation What Examples How Tricks Why? A Popular MINLP Method

Dantzig’s Two-Phase Method for MINLP Adapted by Leyﬀer and Linderoth 1. Convince the user that he or she does not wish to solve a mixed integer nonlinear programming problem at all!

Leyﬀer & Linderoth MINLP • Sometimes a continuous approximation to the discrete (integer) decision is accurate enough for practical purposes. • Yearly tree harvest in Norrland • For 0 − 1 problems, or those in which the |y| is “small”, the continuous approximation to the discrete decision is not accurate enough for practical purposes. • Conclusion: MINLP methods must be studied!

Motivation What Examples How Tricks Why? A Popular MINLP Method

Leyﬀer & Linderoth MINLP Motivation What Examples How Tricks Why? A Popular MINLP Method

Dantzig’s Two-Phase Method for MINLP Adapted by Leyﬀer and Linderoth 1. Convince the user that he or she does not wish to solve a mixed integer nonlinear programming problem at all! 2. Otherwise, solve the continuous relaxation (NLP ) and round oﬀ the minimizer to the nearest integer. • Sometimes a continuous approximation to the discrete (integer) decision is accurate enough for practical purposes. • Yearly tree harvest in Norrland • For 0 − 1 problems, or those in which the |y| is “small”, the continuous approximation to the discrete decision is not accurate enough for practical purposes. • Conclusion: MINLP methods must be studied!

Leyﬀer & Linderoth MINLP Motivation What Examples How Tricks Why?

Example: Core Reload Operation (Quist, A.J., 2000)

• max. reactor efficiency after reload subject to diffusion PDE & safety • diffusion PDE ' nonlinear equation ⇒ integer & nonlinear model • avoid reactor becoming sub-critical

Leyﬀer & Linderoth MINLP Motivation What Examples How Tricks Why?

Example: Core Reload Operation (Quist, A.J., 2000)

• max. reactor efficiency after reload subject to diffusion PDE & safety • diffusion PDE ' nonlinear equation ⇒ integer & nonlinear model • avoid reactor becoming overheated

Leyﬀer & Linderoth MINLP Motivation What Examples How Tricks Why?

Example: Core Reload Operation (Quist, A.J., 2000)

• look for cycles for moving bundles: e.g. 4 → 6 → 8 → 10 i.e. bundle moved from 4 to 6 ...

• model with binary xilm ∈ {0, 1} xilm = 1 ⇔ node i has bundle l of cycle m

Leyﬀer & Linderoth MINLP Motivation What Examples How Tricks Why? AMPL Model of Core Reload Operation

Exactly one bundle per node:

L M X X xilm = 1 ∀i ∈ I l=1 m=1

AMPL model: var x {I,L,M} binary ; Bundle {i in I}: sum{l in L, m in M} x[i,l,m] = 1 ; • Multiple Choice: One of the most common uses of IP • Full AMPL model c-reload.mod at www.mcs.anl.gov/~leyffer/MacMINLP/

Leyﬀer & Linderoth MINLP • All natural gas is imported from Norway, Holland, or Algeria. • Supply gas to all demand points in a network in a minimum cost fashion. • Gas is pumped through the network with a series of compressors • There are constraints on the pressure of the gas within the pipe

Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing

Gas Transmission Problem (De Wolf and Smeers, 2000)

• Belgium has no gas!

Leyﬀer & Linderoth MINLP • Gas is pumped through the network with a series of compressors • There are constraints on the pressure of the gas within the pipe

Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing

Gas Transmission Problem (De Wolf and Smeers, 2000)

• Belgium has no gas! • All natural gas is imported from Norway, Holland, or Algeria. • Supply gas to all demand points in a network in a minimum cost fashion.

Leyﬀer & Linderoth MINLP Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing

Gas Transmission Problem (De Wolf and Smeers, 2000)

• Belgium has no gas! • All natural gas is imported from Norway, Holland, or Algeria. • Supply gas to all demand points in a network in a minimum cost fashion. • Gas is pumped through the network with a series of compressors • There are constraints on the pressure of the gas within the pipe

Leyﬀer & Linderoth MINLP Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing Pressure Loss is Nonlinear

• Assume horizontal pipes and steady state ﬂows • Pressure loss p across a pipe is related to the ﬂow rate f as 1 p2 − p2 = sign(f)f 2 in out Ψ

• Ψ: “Friction Factor”

Leyffer & Linderoth MINLP • si, i ∈ N: supply at node i. • si > 0 ⇒ gas added to the network at node i • si < 0 ⇒ gas removed from the network at node i to meet demand • fij, (i, j) ∈ A: flow along arc (i, j) • f(i, j) > 0 ⇒ gas flows i → j • f(i, j) < 0 ⇒ gas flows j → i

Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing Gas Transmission: Problem Input

• Network (N,A). A = Ap ∪ Aa • Aa: active arcs have compressor. Flow rate can increase on arc • Ap: passive arcs simply conserve ﬂow rate

• Ns ⊆ N: set of supply nodes

• ci, i ∈ Ns: Purchase cost of gas

• si, si: Lower and upper bounds on gas “supply” at node i • p , p : Lower and upper bounds on gas pressure at node i i i

Leyﬀer & Linderoth MINLP Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing Gas Transmission: Problem Input

• Network (N,A). A = Ap ∪ Aa • Aa: active arcs have compressor. Flow rate can increase on arc • Ap: passive arcs simply conserve ﬂow rate

• Ns ⊆ N: set of supply nodes

• ci, i ∈ Ns: Purchase cost of gas

• si, si: Lower and upper bounds on gas “supply” at node i • p , p : Lower and upper bounds on gas pressure at node i i i • si, i ∈ N: supply at node i. • si > 0 ⇒ gas added to the network at node i • si < 0 ⇒ gas removed from the network at node i to meet demand • fij, (i, j) ∈ A: flow along arc (i, j) • f(i, j) > 0 ⇒ gas flows i → j • f(i, j) < 0 ⇒ gas flows j → i

Leyﬀer & Linderoth MINLP Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing Gas Transmission Model

X min cjsj

j∈Ns subject to X X fij − fji = si ∀i ∈ N j|(i,j)∈A j|(j,i)∈A 2 2 2 sign(fij)fij − Ψij(pi − pj ) = 0 ∀(i, j) ∈ Ap 2 2 2 sign(fij)fij − Ψij(pi − pj ) ≥ 0 ∀(i, j) ∈ Aa si ∈ [si, si] ∀i ∈ N p ∈ [p , p ] ∀i ∈ N i i i fij ≥ 0 ∀(i, j) ∈ Aa

Leyﬀer & Linderoth MINLP 2 sign(fij)fij − Ψij(ρi − ρj) = 0 ∀(i, j) ∈ Ap 2 fij − Ψij(ρi − ρj) ≥ 0 ∀(i, j) ∈ Aa ρ ∈ [pp , pp ] ∀i ∈ N i i i

• This trick only works because 2 1. pi terms appear only in the bound constraints 2. Also fij ≥ 0 ∀(i, j) ∈ Aa 2 • This model is nonconvex: sign(fij)fij is a nonconvex function

Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing Your First Modeling Trick

• Don’t include nonlinearities or nonconvexities unless necessary! 2 • Replace pi ← ρi

Leyﬀer & Linderoth MINLP • This trick only works because 2 1. pi terms appear only in the bound constraints 2. Also fij ≥ 0 ∀(i, j) ∈ Aa 2 • This model is nonconvex: sign(fij)fij is a nonconvex function

Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing Your First Modeling Trick

• Don’t include nonlinearities or nonconvexities unless necessary! 2 • Replace pi ← ρi

2 sign(fij)fij − Ψij(ρi − ρj) = 0 ∀(i, j) ∈ Ap 2 fij − Ψij(ρi − ρj) ≥ 0 ∀(i, j) ∈ Aa ρ ∈ [pp , pp ] ∀i ∈ N i i i

Leyﬀer & Linderoth MINLP Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing Your First Modeling Trick

• Don’t include nonlinearities or nonconvexities unless necessary! 2 • Replace pi ← ρi

2 sign(fij)fij − Ψij(ρi − ρj) = 0 ∀(i, j) ∈ Ap 2 fij − Ψij(ρi − ρj) ≥ 0 ∀(i, j) ∈ Aa ρ ∈ [pp , pp ] ∀i ∈ N i i i

• This trick only works because 2 1. pi terms appear only in the bound constraints 2. Also fij ≥ 0 ∀(i, j) ∈ Aa 2 • This model is nonconvex: sign(fij)fij is a nonconvex function

Leyﬀer & Linderoth MINLP Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing Dealing with sign(·): The NLP Way

• Use auxiliary binary variables to indicate direction of ﬂow

• Let |fij| ≤ F ∀(i, j) ∈ Ap 1 fij ≥ 0 fij ≥ −F (1 − zij) zij = 0 fij ≤ 0 fij ≤ F zij

• Note that sign(fij) = 2zij − 1 • Write constraint as

2 (2zij − 1)fij − Ψij(ρi − ρj) = 0.

Leyﬀer & Linderoth MINLP • fij ≤ F zij 2 • fij + Mzij ≤ M + Ψij(ρi − ρj)

Example

Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing Dealing with sign(·): The MIP Way

Model

2 2 fij ≤ Ψij(ρi − ρj) fij ≤ Ψij(ρj − ρi) fij > 0 ⇒ 2 fij < 0 ⇒ 2 fij ≥ Ψij(ρi − ρj) fij ≥ Ψij(ρj − ρi)

2 2 m ≤ fij − Ψ(ρi − ρj) ≤ M l ≤ fij − Ψ(ρj − ρi) ≤ L

Leyﬀer & Linderoth MINLP • fij ≤ F zij 2 • fij + Mzij ≤ M + Ψij(ρi − ρj)

Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing Dealing with sign(·): The MIP Way Model 2 2 fij ≤ Ψij(ρi − ρj) fij ≤ Ψij(ρj − ρi) fij > 0 ⇒ 2 fij < 0 ⇒ 2 fij ≥ Ψij(ρi − ρj) fij ≥ Ψij(ρj − ρi)

2 2 m ≤ fij − Ψ(ρi − ρj) ≤ M l ≤ fij − Ψ(ρj − ρi) ≤ L

Example

2 fij > 0 ⇒ zij = 1 ⇒ fij ≤ Ψ(ρi − ρj)

Leyﬀer & Linderoth MINLP 2 • fij + Mzij ≤ M + Ψij(ρi − ρj)

2 2 m ≤ fij − Ψ(ρi − ρj) ≤ M l ≤ fij − Ψ(ρj − ρi) ≤ L

Example

2 fij > 0 ⇒ zij = 1 ⇒ fij ≤ Ψ(ρi − ρj)

• fij ≤ F zij

Leyﬀer & Linderoth MINLP Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing Dealing with sign(·): The MIP Way Model 2 2 fij ≤ Ψij(ρi − ρj) fij ≤ Ψij(ρj − ρi) fij > 0 ⇒ 2 fij < 0 ⇒ 2 fij ≥ Ψij(ρi − ρj) fij ≥ Ψij(ρj − ρi)

2 2 m ≤ fij − Ψ(ρi − ρj) ≤ M l ≤ fij − Ψ(ρj − ρi) ≤ L

Example

2 fij > 0 ⇒ zij = 1 ⇒ fij ≤ Ψ(ρi − ρj)

• fij ≤ F zij 2 • fij + Mzij ≤ M + Ψij(ρi − ρj)

Leyﬀer & Linderoth MINLP Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing Dealing with sign(·): The MIP Way

• Wonderful MIP Modeling reference is Williams (1993) • If you put it all together you get...

fij ≤ F zij fij ≥ −F yij • zij ∈ {0, 1}: Indicator if ﬂow zij + yij = 1 is positive 2 fij + Mzij ≤ M + Ψij(ρi − ρj) • yij ∈ {0, 1}: Indicator if 2 fij + mzij ≥ m + Ψij(ρi − ρj) ﬂow is negative 2 fij + Lyij ≤ L + Ψij(ρj − ρi) 2 fij + lyij ≥ l + Ψij(ρj − ρi)

Leyﬀer & Linderoth MINLP • Neither way is how it is done in De Wolf and Smeers (2000). • Heuristic for ﬁnding a good starting solution, then a local optimization approach based on a piecewise-linear simplex method • Another (similar) approach involves approximating the nonlinear function by piecewise linear segments, but searching for the globally optimal solution: Special Ordered Sets of Type 2 • If the “multidimensional” nonlinearity cannot be removed, resort to Special Ordered Sets of Type 3

Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing Special Ordered Sets

• Sven thinks the NLP way is better • Jeﬀ thinks the MIP way is better

Leyﬀer & Linderoth MINLP • Heuristic for ﬁnding a good starting solution, then a local optimization approach based on a piecewise-linear simplex method • Another (similar) approach involves approximating the nonlinear function by piecewise linear segments, but searching for the globally optimal solution: Special Ordered Sets of Type 2 • If the “multidimensional” nonlinearity cannot be removed, resort to Special Ordered Sets of Type 3

Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing Special Ordered Sets

• Sven thinks the NLP way is better • Jeﬀ thinks the MIP way is better • Neither way is how it is done in De Wolf and Smeers (2000).

Leyﬀer & Linderoth MINLP Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing Special Ordered Sets

• Sven thinks the NLP way is better • Jeﬀ thinks the MIP way is better • Neither way is how it is done in De Wolf and Smeers (2000). • Heuristic for ﬁnding a good starting solution, then a local optimization approach based on a piecewise-linear simplex method • Another (similar) approach involves approximating the nonlinear function by piecewise linear segments, but searching for the globally optimal solution: Special Ordered Sets of Type 2 • If the “multidimensional” nonlinearity cannot be removed, resort to Special Ordered Sets of Type 3

Leyﬀer & Linderoth MINLP def • Markowitz: u(x) = −αT x + λxT Qx • α: Expected returns • Q: Variance-covariance matrix of expected returns • λ: Risk aversion parameter

Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing

Portfolio Management

• N: Universe of asset to purchase

• xi: Amount of asset i to hold • B: Budget

( ) X min u(x) | xi = B |N| x∈R+ i∈N

Leyﬀer & Linderoth MINLP Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing

Portfolio Management

• N: Universe of asset to purchase

• xi: Amount of asset i to hold • B: Budget

( ) X min u(x) | xi = B |N| x∈R+ i∈N

def • Markowitz: u(x) = −αT x + λxT Qx • α: Expected returns • Q: Variance-covariance matrix of expected returns • λ: Risk aversion parameter

Leyﬀer & Linderoth MINLP • Limit Names: |i ∈ N : xi > 0| ≤ K • Use binary indicator variables to model the implication xi > 0 ⇒ yi = 1 • Implication modeled with variable upper bounds:

xi ≤ Byi ∀i ∈ N P • i∈N yi ≤ K

Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing

More Realistic Models

|N| • b ∈ R of “benchmark” holdings def • Benchmark Tracking: u(x) = (x − b)T Q(x − b) • Constraint on E[Return]: αT x ≥ r

Leyﬀer & Linderoth MINLP Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing

More Realistic Models

|N| • b ∈ R of “benchmark” holdings def • Benchmark Tracking: u(x) = (x − b)T Q(x − b) • Constraint on E[Return]: αT x ≥ r • Limit Names: |i ∈ N : xi > 0| ≤ K • Use binary indicator variables to model the implication xi > 0 ⇒ yi = 1 • Implication modeled with variable upper bounds:

xi ≤ Byi ∀i ∈ N P • i∈N yi ≤ K

Leyﬀer & Linderoth MINLP • Round Lots: xi ∈ {kLi, k = 1, 2,...} • xi − ziLi = 0, zi ∈ Z+ ∀i ∈ N • Vector h of initial holdings

• Transactions: ti = |xi − hi| P • Turnover: i∈N ti ≤ ∆ P • Transaction Costs: i∈N citi in objective P 2 • Market Impact: i∈N γiti in objective

Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing

Even More Models

• Min Holdings:(xi = 0) ∨ (xi ≥ m) • Model implication: xi > 0 ⇒ xi ≥ m • xi > 0 ⇒ yi = 1 ⇒ xi ≥ m • xi ≤ Byi, xi ≥ myi ∀i ∈ N

Leyﬀer & Linderoth MINLP • Vector h of initial holdings

• Transactions: ti = |xi − hi| P • Turnover: i∈N ti ≤ ∆ P • Transaction Costs: i∈N citi in objective P 2 • Market Impact: i∈N γiti in objective

Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing

Even More Models

• Min Holdings:(xi = 0) ∨ (xi ≥ m) • Model implication: xi > 0 ⇒ xi ≥ m • xi > 0 ⇒ yi = 1 ⇒ xi ≥ m • xi ≤ Byi, xi ≥ myi ∀i ∈ N • Round Lots: xi ∈ {kLi, k = 1, 2,...} • xi − ziLi = 0, zi ∈ Z+ ∀i ∈ N

Leyﬀer & Linderoth MINLP Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing

Even More Models

• Min Holdings:(xi = 0) ∨ (xi ≥ m) • Model implication: xi > 0 ⇒ xi ≥ m • xi > 0 ⇒ yi = 1 ⇒ xi ≥ m • xi ≤ Byi, xi ≥ myi ∀i ∈ N • Round Lots: xi ∈ {kLi, k = 1, 2,...} • xi − ziLi = 0, zi ∈ Z+ ∀i ∈ N • Vector h of initial holdings

• Transactions: ti = |xi − hi| P • Turnover: i∈N ti ≤ ∆ P • Transaction Costs: i∈N citi in objective P 2 • Market Impact: i∈N γiti in objective

Leyﬀer & Linderoth MINLP Which is better?: Part III has the answer

Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing Making “Plays”

• Suppose that the stocks are partitioned into sectors S1 ⊆ N,S2 ⊆ N,...SK ⊆ N • The Fund Manager wants to invest all money into one sector “play” • P x > 0 ⇒ P x = 0 i∈Sk i j∈N\Sk j Modeling Choices

Aggregated : zk = 1 invest in sector k X X xi ≤ Bzk ∀k xj + Bzk ≤ B ∀k

i∈Sk j∈N\Sk

• Adds K variables and 2K constraints

Leyﬀer & Linderoth MINLP Which is better?: Part III has the answer

Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing Making “Plays”

Disaggregated : zi = 1 invest in asset i

xi ≤ uizi ∀i ∈ N xj + ujzi ≤ uj ∀j | i ∈ Sk, j 6∈ Sk

• Adds N variables and O(N 2) constraints

Leyﬀer & Linderoth MINLP Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing Making “Plays”

Disaggregated : zi = 1 invest in asset i

xi ≤ uizi ∀i ∈ N xj + ujzi ≤ uj ∀j | i ∈ Sk, j 6∈ Sk

• Adds N variables and O(N 2) constraints

Which is better?: Part III has the answer

Leyﬀer & Linderoth MINLP • Bi: Batch size of product i ∈ N • Vj: Stage j size: Vj ≥ SijBi ∀i, j • Nj: Number of machines at stage j • Ci: Longest stage time for product i: Ci ≥ tij/Nj ∀i, j

Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing

Multiproduct Batch Plants (Kocis and Grossmann, 1988)

• M: Batch Processing Stages • N: Diﬀerent Products • H: Horizon Time • Qi: Required quantity of product i • tij: Processing time product i stage j • Sij: “Size Factor” product i stage j

Leyﬀer & Linderoth MINLP Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing

Multiproduct Batch Plants (Kocis and Grossmann, 1988)

• Bi: Batch size of product i ∈ N • Vj: Stage j size: Vj ≥ SijBi ∀i, j • Nj: Number of machines at stage j • Ci: Longest stage time for product i: Ci ≥ tij/Nj ∀i, j

Leyﬀer & Linderoth MINLP Motivation Gas Transmission Examples Portfolio Management Tricks Batch Processing

Multiproduct Batch Plants

X βj min αjNjVj j∈M s.t.

Vj − SijBi ≥ 0 ∀i ∈ N, ∀j ∈ M CiNj ≥ tij ∀i ∈ N, ∀j ∈ M X Qi Ci ≤ H Bi i∈N Bound Constraints on Vj,Ci,Bi,Nj Nj ∈ Z ∀j ∈ M

Leyﬀer & Linderoth MINLP • Sometimes variable transformations work!

vj = ln(Vj), nj = ln(Nj), bi = ln(Bi), ci = ln Ci

X Nj +βj Vj min αje j∈M

s.t. vj − ln(Sij)bi ≥ 0 ∀i ∈ N, ∀j ∈ M ci + nj ≥ ln(τij) ∀i ∈ N, ∀j ∈ M X Ci−Bi Qie ≤ H i∈N (Transformed) Bound Constraints on Vj,Ci,Bi

Motivation Variable Transformation Examples MIQP Tricks Modeling Trick #2

• Horizon Time and Objective Function Nonconvex. :-(

Leyﬀer & Linderoth MINLP X Nj +βj Vj min αje j∈M

s.t. vj − ln(Sij)bi ≥ 0 ∀i ∈ N, ∀j ∈ M ci + nj ≥ ln(τij) ∀i ∈ N, ∀j ∈ M X Ci−Bi Qie ≤ H i∈N (Transformed) Bound Constraints on Vj,Ci,Bi

Motivation Variable Transformation Examples MIQP Tricks Modeling Trick #2

• Horizon Time and Objective Function Nonconvex. :-( • Sometimes variable transformations work!

vj = ln(Vj), nj = ln(Nj), bi = ln(Bi), ci = ln Ci

Leyﬀer & Linderoth MINLP Motivation Variable Transformation Examples MIQP Tricks Modeling Trick #2

• Horizon Time and Objective Function Nonconvex. :-( • Sometimes variable transformations work!

vj = ln(Vj), nj = ln(Nj), bi = ln(Bi), ci = ln Ci

X Nj +βj Vj min αje j∈M

s.t. vj − ln(Sij)bi ≥ 0 ∀i ∈ N, ∀j ∈ M ci + nj ≥ ln(τij) ∀i ∈ N, ∀j ∈ M X Ci−Bi Qie ≤ H i∈N (Transformed) Bound Constraints on Vj,Ci,Bi

Leyﬀer & Linderoth MINLP Motivation Variable Transformation Examples MIQP Tricks How to Handle the Integrality?

• But what to do about the integrality?

1 ≤ Nj ≤ N j ∀j ∈ M,Nj ∈ Z ∀j ∈ M

• nj ∈ {0, ln(2), ln(3), ...... } 1 n takes value ln(k) Y = j kj 0 Otherwise

K X nj − ln(k)Ykj = 0 ∀j ∈ M k=1 K X Ykj = 1 ∀j ∈ M k=1

• This model is available at http://www-unix.mcs.anl.gov/ ~leyffer/macminlp/problems/batch.mod

Leyﬀer & Linderoth MINLP Motivation Variable Transformation Examples MIQP Tricks MIQP: Modeling Tricks

• In 0-1 quadratic programming, we can always make quadratic forms convex. • Key: If y ∈ {0, 1}, then y = y2, so add a “large enough” constant to the diagonal, and subtract it from the linear term:

• y ∈ {0, 1}n consider any quadratic

q(y) = yT Qy + gT y = yT W y + cT y

where W = Q + λI and c = g − λe (e = (1,..., 1)) • If λ ≥(smallest eigenvalue of Q), then W 0.

Leyﬀer & Linderoth MINLP Motivation Variable Transformation Examples MIQP Tricks A Small Smattering of Other Applications

• Chemical Engineering Applications: • process synthesis (Kocis and Grossmann, 1988) • batch plant design (Grossmann and Sargent, 1979) • cyclic scheduling (Jain, V. and Grossmann, I.E., 1998) • design of distillation columns (Viswanathan and Grossmann, 1993) • pump conﬁguration optimization (Westerlund, T., Pettersson, F. and Grossmann, I.E., 1994) • Forestry/Paper • production (Westerlund, T., Isaksson, J. and Harjunkoski, I., 1995) • trimloss minimization (Harjunkoski, I., Westerlund, T., P¨orn, R. and Skrifvars, H., 1998) • Topology Optimization (Sigmund, 2001)

Leyﬀer & Linderoth MINLP Branch-and-Bound Outer Approximation Hybrid Methods

Part II

Classical Solution Methods

Leyﬀer & Linderoth MINLP Branch-and-Bound Outer Approximation Hybrid Methods Classical Solution Methods for MINLP

1. Classical Branch-and-Bound 2. Outer Approximation, Benders Decomposition et al. 3. Hybrid Methods • LP/NLP Based Branch-and-Bound • Integrating SQP with Branch-and-Bound

Leyﬀer & Linderoth MINLP Branch-and-Bound Deﬁnition Outer Approximation Convergence Hybrid Methods Variable & Node Selection Branch-and-Bound

Solve relaxed NLP (0 ≤ y ≤ 1 continuous relaxation) . . . solution value provides lower bound

integer feasible etc. • Branch on yi non-integral • Solve NLPs & branch until y = 1 etc. 1. Node infeasible ... • i 2. Node integer feasible ... dominated y = 0 ⇒ get upper bound (U) i by upper bound 3. Lower bound ≥ U ... N infeasible

Search until no unexplored nodes on tree

Leyﬀer & Linderoth MINLP All NLP problems solved globally & ﬁnite number of NLPs on tree ⇒ Branch-and-Bound converges ' complete enumeration at worst

Branch-and-Bound Deﬁnition Outer Approximation Convergence Hybrid Methods Variable & Node Selection Convergence of Branch-and-Bound

x y*=2.4 c(x)=0

y* y

Leyﬀer & Linderoth MINLP ' complete enumeration at worst

Branch-and-Bound Deﬁnition Outer Approximation Convergence Hybrid Methods Variable & Node Selection Convergence of Branch-and-Bound

x y*=2.4 c(x)=0

y* y<2 y>3 y

All NLP problems solved globally & ﬁnite number of NLPs on tree ⇒ Branch-and-Bound converges

Leyﬀer & Linderoth MINLP Branch-and-Bound Deﬁnition Outer Approximation Convergence Hybrid Methods Variable & Node Selection Convergence of Branch-and-Bound

x y*=2.4 c(x)=0

y* y<2 y>3 y

All NLP problems solved globally & ﬁnite number of NLPs on tree ⇒ Branch-and-Bound converges ' complete enumeration at worst

Leyﬀer & Linderoth MINLP 2. maximal fractional branching

max {min(1 − yˆi, yˆi)} i

1 ... ﬁnd yˆi closest to 2 ⇒ largest change in problem

Branch-and-Bound Deﬁnition Outer Approximation Convergence Hybrid Methods Variable & Node Selection Variable Selection for Branch-and-Bound

Assume yi ∈ {0, 1} for simplicity ... (ˆx, yˆ) fractional solution to parent node; fˆ = f(ˆx, yˆ) 1. user deﬁned priorities ... branch on most important variable ﬁrst

Leyﬀer & Linderoth MINLP Branch-and-Bound Deﬁnition Outer Approximation Convergence Hybrid Methods Variable & Node Selection Variable Selection for Branch-and-Bound

Assume yi ∈ {0, 1} for simplicity ... (ˆx, yˆ) fractional solution to parent node; fˆ = f(ˆx, yˆ) 1. user deﬁned priorities ... branch on most important variable ﬁrst 2. maximal fractional branching

max {min(1 − yˆi, yˆi)} i

1 ... ﬁnd yˆi closest to 2 ⇒ largest change in problem

Leyffer & Linderoth MINLP + − ... estimate ei , ei by keeping track of + ˆ − ˆ + fi − f − fi − f ei = and ei = 1 − yî yî

+/− where fi solution value after branching

Branch-and-Bound Deﬁnition Outer Approximation Convergence Hybrid Methods Variable & Node Selection Variable Selection for Branch-and-Bound

Assume yi ∈ {0, 1} for simplicity ... (ˆx, yˆ) fractional solution to parent node; fˆ = f(ˆx, yˆ) 3. pseudo-cost branching + − estimates ei , ei of change in f(x, y) after branching

n + − o max min(fˆ+ e (1 − yˆi), fˆ+ e yˆi) i i i

... ﬁnd yi, whose expected change of objective is largest

Leyﬀer & Linderoth MINLP Branch-and-Bound Deﬁnition Outer Approximation Convergence Hybrid Methods Variable & Node Selection Variable Selection for Branch-and-Bound

Assume yi ∈ {0, 1} for simplicity ... (ˆx, yˆ) fractional solution to parent node; fˆ = f(ˆx, yˆ) 3. pseudo-cost branching + − estimates ei , ei of change in f(x, y) after branching

n + − o max min(fˆ+ e (1 − yˆi), fˆ+ e yˆi) i i i

... find yi, whose expected change of objective is largest + − ... estimate ei , ei by keeping track of + ˆ − ˆ + fi − f − fi − f ei = and ei = 1 − yî yî

+/− where fi solution value after branching

Leyﬀer & Linderoth MINLP Branch-and-Bound Deﬁnition Outer Approximation Convergence Hybrid Methods Variable & Node Selection Variable Selection for Branch-and-Bound

Assume yi ∈ {0, 1} for simplicity ... (ˆx, yˆ) fractional solution to parent node; fˆ = f(ˆx, yˆ) 4. strong branching: solve all NLP child nodes:   minimize f(x, y) +/−  x,y fi ← subject to c(x, y) ≤ 0   x ∈ X, y ∈ Y, yi = 1/0

choose branching variable as

max min(f +, f −) i i i

... ﬁnd yi that changes objective the most

Leyﬀer & Linderoth MINLP Branch-and-Bound Deﬁnition Outer Approximation Convergence Hybrid Methods Variable & Node Selection Variable Selection for Branch-and-Bound

Assume yi ∈ {0, 1} for simplicity ... (ˆx, yˆ) fractional solution to parent node; fˆ = f(ˆx, yˆ) 5. MIQP strong branching: (Fletcher and Leyﬀer, 1998) parametric solution of QPs ... much cheaper than re-solve

x2 x’ solution of QP relaxation 2 step of the dual ASM

• step of dual active set method 1 x− f = −2.75 • factorization of KKT matrix f = −2.5556 x’ • ' multiple KKT solves x* f = −2.25 x • generalizes old MILP ideas 1 1 2

Leyﬀer & Linderoth MINLP Branch-and-Bound Deﬁnition Outer Approximation Convergence Hybrid Methods Variable & Node Selection Node Selection for Branch-and-Bound

Which node n on tree T should be solved next? 1. depth-ﬁrst search select deepest node in tree • minimizes number of NLP nodes stored • exploit warm-starts (MILP/MIQP only) 2. best lower bound choose node with least value of parent node fp(n) • minimizes number of NLPs solved

Leyﬀer & Linderoth MINLP Branch-and-Bound Deﬁnition Outer Approximation Convergence Hybrid Methods Variable & Node Selection Node Selection for Branch-and-Bound

Which node n on tree T should be solved next? 3. best estimate choose node leading to best expected integer solution    X + −  min fp(n) + min ei (1 − yi), ei yi n∈T   i:yifractional

where

• fp(n) = value of parent node +/− • ei = pseudo-costs summing pseudo-cost estimates for all integers in subtree

Leyﬀer & Linderoth MINLP Branch-and-Bound Deﬁnition Outer Approximation Convergence Hybrid Methods Benders Decomposition Outer Approximation (Duran and Grossmann, 1986)

Motivation: avoid solving huge number of NLPs • Exploit MILP/NLP solvers: decompose integer/nonlinear part Key idea: reformulate MINLP as MILP (implicit) • Solve alternating sequence of MILP & NLP

NLP subproblem yj ﬁxed: MILP  minimize f(x, yj)  x NLP(yj) subject to c(x, yj) ≤ 0  x ∈ X NLP(y j)

Main Assumption: f, c are convex

Leyﬀer & Linderoth MINLP Branch-and-Bound Deﬁnition Outer Approximation Convergence Hybrid Methods Benders Decomposition Outer Approximation (Duran and Grossmann, 1986)

f(x) • let (xj, yj) solve NLP(yj) • linearize f, c about (xj, yj) =: zj • new objective variable η ≥ f(x, y) • MINLP (P ) ≡ MILP (M)

 minimize η  z=(x,y),η  T (M) subject to η ≥ fj + ∇fj (z − zj) ∀yj ∈ Y T  0 ≥ cj + ∇c (z − zj) ∀yj ∈ Y  j  x ∈ X, y ∈ Y integer

SNAG: need all yj ∈ Y linearizations

Leyﬀer & Linderoth MINLP Branch-and-Bound Deﬁnition Outer Approximation Convergence Hybrid Methods Benders Decomposition Outer Approximation (Duran and Grossmann, 1986)

(Mk): lower bound (underestimate convex f, c) NLP(yj): upper bound U (ﬁxed yj)

NLP( y ) subproblem NLP gives linearization

MILP finds MILP master program new y

No MILP infeasible?

Yes STOP

⇒ stop, if lower bound ≥ upper bound

Leyﬀer & Linderoth MINLP Branch-and-Bound Deﬁnition Outer Approximation Convergence Hybrid Methods Benders Decomposition Convergence of Outer Approximation

Lemma: Each yi ∈ Y generated at most once.

Proof: Assume yi ∈ Y generated again at iteration j > i ⇒ ∃ xˆ such that (ˆx, yi) feasible in (Mj):

T η ≥ fi + ∇xfi (ˆx − xi) T 0 ≥ ci + ∇xci (ˆx − xi)

... because yi − yi = 0 Now sum with (1, λi) optimal multipliers of NLP(yi) T T ⇒ η ≥ fi + λi ci + (∇xfi + ∇xciλi) (ˆx − xi) T ... KKT conditions: ∇xfi + ∇xciλi = 0 & λi ci = 0 ⇒ η ≥ fi contradicts η < U ≤ fi upper bound ⇒ each yi ∈ Y generated at most once Refs: (Duran and Grossmann, 1986; Fletcher and Leyﬀer, 1994)

Leyﬀer & Linderoth MINLP Branch-and-Bound Deﬁnition Outer Approximation Convergence Hybrid Methods Benders Decomposition Convergence of Outer Approximation

f(x) 1. each yi ∈ Y generated at most once & |Y | < ∞ ⇒ ﬁnite termination 2. convexity ⇒ outer approximation ⇒ convergence to global min

Convexity important!!!

Leyﬀer & Linderoth MINLP Branch-and-Bound Deﬁnition Outer Approximation Convergence Hybrid Methods Benders Decomposition Outer Approximation & Benders Decomposition

n Take OA master ... z := (x, y) ... wlog X = R  minimize η  z=(x,y),η  T (M) subject to η ≥ fj + ∇fj (z − zj) ∀yj ∈ Y T  0 ≥ cj + ∇c (z − zj) ∀yj ∈ Y  j  y ∈ Y integer

∀j : sum 0 ≥ cj... weighted with multiliers λj of NLP(yj)

T T ⇒ η ≥ fj + λj cj + (∇fj + ∇cjλj) (z − zj) ∀yj ∈ Y

... is a valid inequality. References: (Geoﬀrion, 1972)

Leyﬀer & Linderoth MINLP Branch-and-Bound Deﬁnition Outer Approximation Convergence Hybrid Methods Benders Decomposition Outer Approximation & Benders Decomposition

Valid inequality from OA master; z = (x, y):

T T η ≥ fj + λj cj + (∇fj + ∇cjλj) (z − zj) use ﬁrst order conditions of NLP(yj) ...

T ∇xfj + ∇xcjλj = 0 & λj cj = 0

... to eliminate x components from valid inequality in y

T ⇒ η ≥ fj + (∇yfj + ∇ycjλj) (y − yj) T ⇔ η ≥ fj + (µj) (y − yj) where µj = ∇yfj + ∇ycjλj multiplier of y = yj in NLP(yj)

Leyﬀer & Linderoth MINLP Branch-and-Bound Deﬁnition Outer Approximation Convergence Hybrid Methods Benders Decomposition Outer Approximation & Benders Decomposition

⇒ remove x from master problem ... Benders master problem

 minimize η  y,η (M ) T B subject to η ≥ fj + (µj) (y − yj) ∀yj ∈ Y  y ∈ Y integer where µj multiplier of y = yj in NLP(yj)

• (MB) has less constraints & variables (no x!) • (MB) almost ILP (except for η) • (MB) weaker than OA (from derivation)

Leyﬀer & Linderoth MINLP Branch-and-Bound Deﬁnition Outer Approximation Convergence Hybrid Methods Benders Decomposition Extended Cutting Plane Method

Replace NLP(yi) solve in OA by η linearization about solution of (M ) j f(x) get cutting plane for violated constraint ⇒ no NLP(yj) solves ...... Kelley’s cutting plane method instead ⇒ slow nonlinear convergence: > 1 evaluation per y

References: (Westerlund, T. and Pettersson, F., 1995)

Leyﬀer & Linderoth MINLP Branch-and-Bound Deﬁnition Outer Approximation Convergence Hybrid Methods Benders Decomposition Disadvantages of Outer Approximation

f(y) = ( y - 1/8 ) 2

• MILP tree-search can be bottle-neck • potentially large number of iterations 1/4 1/8 1/2 1 1 2 minimize (y − 2n ) 1 subject to y ∈ {0, 2n ,... 1}

Second order master (MIQP): (Fletcher and Leyﬀer, 1994): • add Hessian term to MILP (Mi) becomes MIQP: 1 minimize η + (z − z )T W (z − z ) subject to ... 2 i i

Leyﬀer & Linderoth MINLP • interrupt MILP, when yj found ⇒ solve NLP(yj) get xj

• linearize f, c about (xj, yj) ⇒ add linearization to tree • continue MILP tree-search

... until lower bound ≥ upper bound

Branch-and-Bound LP/NLP Based Branch-and-Bound Outer Approximation Integrating SQP and Branch-and-Bound Hybrid Methods LP/NLP Based Branch-and-Bound

AIM: avoid re-solving MILP master (M)

• Consider MILP branch-and-bound

Leyﬀer & Linderoth MINLP • linearize f, c about (xj, yj) ⇒ add linearization to tree • continue MILP tree-search

... until lower bound ≥ upper bound

Branch-and-Bound LP/NLP Based Branch-and-Bound Outer Approximation Integrating SQP and Branch-and-Bound Hybrid Methods LP/NLP Based Branch-and-Bound

AIM: avoid re-solving MILP master (M)

integer • Consider MILP branch-and-bound feasible

• interrupt MILP, when yj found ⇒ solve NLP(yj) get xj

Leyﬀer & Linderoth MINLP • continue MILP tree-search

... until lower bound ≥ upper bound

Branch-and-Bound LP/NLP Based Branch-and-Bound Outer Approximation Integrating SQP and Branch-and-Bound Hybrid Methods LP/NLP Based Branch-and-Bound

AIM: avoid re-solving MILP master (M)

• Consider MILP branch-and-bound

• interrupt MILP, when yj found ⇒ solve NLP(yj) get xj

• linearize f, c about (xj, yj) ⇒ add linearization to tree

Leyﬀer & Linderoth MINLP Branch-and-Bound LP/NLP Based Branch-and-Bound Outer Approximation Integrating SQP and Branch-and-Bound Hybrid Methods LP/NLP Based Branch-and-Bound

AIM: avoid re-solving MILP master (M)

• Consider MILP branch-and-bound

• interrupt MILP, when yj found ⇒ solve NLP(yj) get xj

• linearize f, c about (xj, yj) ⇒ add linearization to tree • continue MILP tree-search

... until lower bound ≥ upper bound

Leyﬀer & Linderoth MINLP Branch-and-Bound LP/NLP Based Branch-and-Bound Outer Approximation Integrating SQP and Branch-and-Bound Hybrid Methods LP/NLP Based Branch-and-Bound

• need access to MILP solver ... call back ◦ exploit good MILP (branch-cut-price) solver ◦ (Akrotirianakis et al., 2001) use Gomory cuts in tree-search

• no commercial implementation of this idea • preliminary results: order of magnitude faster than OA ◦ same number of NLPs, but only one MILP • similar ideas for Benders & Extended Cutting Plane methods References: (Quesada and Grossmann, 1992)

Leyﬀer & Linderoth MINLP Branch-and-Bound LP/NLP Based Branch-and-Bound Outer Approximation Integrating SQP and Branch-and-Bound Hybrid Methods Integrating SQP & Branch-and-Bound

AIM: Avoid solving NLP node to convergence.

Sequential Quadratic Programming (SQP) → solve sequence (QPk) at every node

 T 1 T minimize fk + ∇fk d + d Hkd  d 2  T subject to ck + ∇c d ≤ 0 (QPk) k  xk + dx ∈ X   yk + dy ∈ Y.ˆ

Early branching: k+1 After QP step choose non-integral yi , branch & continue SQP References: (Borchers and Mitchell, 1994; Leyﬀer, 2001)

Leyﬀer & Linderoth MINLP Remedy: Exploit OA underestimating property (Leyﬀer, 2001): T • add objective cut fk + ∇fk d ≤ U − to (QPk) • fathom node, if (QPk) inconsistent ⇒ converge for convex MINLP

Branch-and-Bound LP/NLP Based Branch-and-Bound Outer Approximation Integrating SQP and Branch-and-Bound Hybrid Methods Integrating SQP & Branch-and-Bound

SNAG:(QPk) not lower bound ⇒ no fathoming from upper bound

T 1 T minimize fk + ∇fk d + d Hkd d 2 T subject to ck + ∇ck d ≤ 0 xk + dx ∈ X yk + dy ∈ Y.ˆ

NB: (QPk) inconsistent and trust-region active ⇒ do not fathom

Leyﬀer & Linderoth MINLP Branch-and-Bound LP/NLP Based Branch-and-Bound Outer Approximation Integrating SQP and Branch-and-Bound Hybrid Methods Integrating SQP & Branch-and-Bound

SNAG:(QPk) not lower bound ⇒ no fathoming from upper bound

T 1 T minimize fk + ∇fk d + d Hkd d 2 T subject to ck + ∇ck d ≤ 0 xk + dx ∈ X yk + dy ∈ Y.ˆ

Remedy: Exploit OA underestimating property (Leyﬀer, 2001): T • add objective cut fk + ∇fk d ≤ U − to (QPk) • fathom node, if (QPk) inconsistent ⇒ converge for convex MINLP

NB: (QPk) inconsistent and trust-region active ⇒ do not fathom

Leyﬀer & Linderoth MINLP Branch-and-Bound LP/NLP Based Branch-and-Bound Outer Approximation Integrating SQP and Branch-and-Bound Hybrid Methods Comparison of Classical MINLP Techniques

Summary of numerical experience 1. Quadratic OA master: usually fewer iteration MIQP harder to solve 2. NLP branch-and-bound faster than OA ... depends on MIP solver 3. LP/NLP-based-BB order of magnitude faster than OA . . . also faster than B&B 4. Integrated SQP-B&B up to 3× faster than B&B ' number of QPs per node 5. ECP works well, if function/gradient evals expensive

Leyﬀer & Linderoth MINLP Formulations Inequalities Dealing with Nonconvexity

Part III

Modern Developments in MINLP

Leyﬀer & Linderoth MINLP Formulations Importance of Relaxations Inequalities Big M Dealing with Nonconvexity Aggregation Modern Methods for MINLP

1. Formulations • Relaxations • Good formulations: big M 0s and disaggregation 2. Cutting Planes • Cuts from relaxations and special structures • Cuts from integrality 3. Handling Nonconvexity • Envelopes • Methods

Leyﬀer & Linderoth MINLP ∗ • If xT = arg minx∈T f(x) ∗ • And xT ∈ S, then ∗ • xT = arg minx∈S f(x)

Formulations Importance of Relaxations Inequalities Big M Dealing with Nonconvexity Aggregation Relaxations

def • z(S) = minx∈S f(x) def • z(T ) = minx∈T f(x) • Independent of f, S, T : z(T ) ≤ z(S)

Leyﬀer & Linderoth MINLP ∗ • xT = arg minx∈S f(x)

Formulations Importance of Relaxations Inequalities Big M Dealing with Nonconvexity Aggregation Relaxations

def • z(S) = minx∈S f(x) def • z(T ) = minx∈T f(x) • Independent of f, S, T : z(T ) ≤ z(S) ∗ • If xT = arg minx∈T f(x) S ∗ • And xT ∈ S, then

Leyﬀer & Linderoth MINLP Formulations Importance of Relaxations Inequalities Big M Dealing with Nonconvexity Aggregation Relaxations

def • z(S) = minx∈S f(x) def • z(T ) = minx∈T f(x) • Independent of f, S, T : z(T ) ≤ z(S) ∗ • If xT = arg minx∈T f(x) S ∗ • And xT ∈ S, then ∗ • xT = arg minx∈S f(x)

Leyﬀer & Linderoth MINLP Formulations Importance of Relaxations Inequalities Big M Dealing with Nonconvexity Aggregation A Pure Integer Program

T n z(S) = min{c x : x ∈ S},S = {x ∈ Z+ : Ax ≤ b}

2 S = {(x1, x2) ∈ Z+ : 6x1 + x2 ≤ 15, 5x1 + 8x2 ≤ 20, x2 ≤ 2} = {(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0)}

Leyﬀer & Linderoth MINLP Formulations Importance of Relaxations Inequalities Big M Dealing with Nonconvexity Aggregation How to Solve Integer Programs?

• Relaxations! • T ⊇ S ⇒ z(T ) ≤ z(S) • People commonly use the linear programming relaxation:

z(LP (S)) = min{cT x : x ∈ LP (S)} n LP (S) = {x ∈ R+ : Ax ≤ b}

• If LP (S) = conv(S), we are done. • Minimum of any linear function over any convex set occurs on the boundary • We need only know conv(S) in the direction of c. • The “closer” LP (S) is to conv(S) the better.

Leyﬀer & Linderoth MINLP Formulations Importance of Relaxations Inequalities Big M Dealing with Nonconvexity Aggregation Small M’s Good. Big M’s Baaaaaaaaaaaaaaaaaad!

• Sometimes, we can get a better relaxation (make LP (S) a closer approximation to conv(S)) through a diﬀerent tighter formulation • Let’s look at the geometry

P = {x ∈ R+, z ∈ {0, 1} : x ≤ Mz, x ≤ u}

LP (P ) = {x ∈ R+, z ∈ [0, 1] : x ≤ Mz, x ≤ u}

conv(P ) = {x ∈ R+, z ∈ {0, 1} : x ≤ uz}

Leyﬀer & Linderoth MINLP Formulations Importance of Relaxations Inequalities Big M Dealing with Nonconvexity Aggregation P

0 1 z

P = {x ∈ R+, z ∈ {0, 1} : x ≤ Mz, x ≤ u}

Leyﬀer & Linderoth MINLP Formulations Importance of Relaxations Inequalities Big M Dealing with Nonconvexity Aggregation LP Versus Conv

u LP (P ) = {x ∈ R+, z ∈ [0, 1] : x ≤ Mz, x ≤ u} x

0 1 z M

u conv(P ) = {x ∈ R+, z ∈ [0, 1] : x ≤ uz} x

0 1 z • KEY: If M = u, LP (P ) = conv(P ) • Small M’s good. Big M’s baaaaaaaad.

Leyﬀer & Linderoth MINLP • Which formulation is to be preferred? • I = J = 40. Costs random. • Formulation 1. 53,121 seconds, optimal solution. • Formulation 2. 2 seconds, optimal solution.

Formulations Importance of Relaxations Inequalities Big M Dealing with Nonconvexity Aggregation UFL: Uncapacitated Facility Location • Facilities: I • Customers: J X X X min fjxj + fijyij j∈J i∈I j∈J

X yij = 1 ∀i ∈ I j∈N X yij ≤ |I|xj ∀j ∈ J (1) i∈I OR yij ≤ xj ∀i ∈ I, j ∈ J (2)

Leyﬀer & Linderoth MINLP • I = J = 40. Costs random. • Formulation 1. 53,121 seconds, optimal solution. • Formulation 2. 2 seconds, optimal solution.

X yij = 1 ∀i ∈ I j∈N X yij ≤ |I|xj ∀j ∈ J (1) i∈I OR yij ≤ xj ∀i ∈ I, j ∈ J (2)

• Which formulation is to be preferred?

Leyﬀer & Linderoth MINLP • Formulation 1. 53,121 seconds, optimal solution. • Formulation 2. 2 seconds, optimal solution.

X yij = 1 ∀i ∈ I j∈N X yij ≤ |I|xj ∀j ∈ J (1) i∈I OR yij ≤ xj ∀i ∈ I, j ∈ J (2)

• Which formulation is to be preferred? • I = J = 40. Costs random.

Leyﬀer & Linderoth MINLP Formulations Importance of Relaxations Inequalities Big M Dealing with Nonconvexity Aggregation UFL: Uncapacitated Facility Location • Facilities: I • Customers: J X X X min fjxj + fijyij j∈J i∈I j∈J

X yij = 1 ∀i ∈ I j∈N X yij ≤ |I|xj ∀j ∈ J (1) i∈I OR yij ≤ xj ∀i ∈ I, j ∈ J (2)

• Which formulation is to be preferred? • I = J = 40. Costs random. • Formulation 1. 53,121 seconds, optimal solution. • Formulation 2. 2 seconds, optimal solution.

Leyﬀer & Linderoth MINLP • Thm: (Hahn-Banach). Let n S ⊂ R be a closed, convex set, T xˆ and letx ˆ 6∈ S. Then there exists π x = n π0 π ∈ R such that S πT xˆ > max{πT x} x∈S

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities Valid Inequalities

• Sometimes we can get a better formulation by dynamically improving it.

T • An inequality π x ≤ π0 is a valid inequality for S if T π x ≤ π0 ∀x ∈ S T • Alternatively: maxx∈S{π x} ≤ π0

Leyﬀer & Linderoth MINLP Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities Valid Inequalities

• Sometimes we can get a better formulation by dynamically improving it.

T • An inequality π x ≤ π0 is a valid inequality for S if T π x ≤ π0 ∀x ∈ S T • Alternatively: maxx∈S{π x} ≤ π0

• Thm: (Hahn-Banach). Let n S ⊂ R be a closed, convex set, T xˆ and letx ˆ 6∈ S. Then there exists π x = n π0 π ∈ R such that S πT xˆ > max{πT x} x∈S

Leyﬀer & Linderoth MINLP Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities Nonlinear Branch-and-Cut

Consider MINLP  minimize f T x + f T y  x,y x y subject to c(x, y) ≤ 0  y ∈ {0, 1}p, 0 ≤ x ≤ U

• Note the Linear objective • This is WLOG:

min f(x, y) ⇔ min η s.t. η ≥ f(x, y)

Leyﬀer & Linderoth MINLP 2 2 η ≥ (y1 − 1/2) + (y2 − 1/2)

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities It’s Actually Important!

• We want to approximate the convex hull of integer solutions, but without a linear objective function, the solution to the relaxation might occur in the interior. • No Separating Hyperplane! :-(

2 2 min(y1 − 1/2) + (y2 − 1/2) y2

s.t. y1 ∈ {0, 1}, y2 ∈ {0, 1}

(ˆy1, yˆ2)

Leyﬀer & Linderoth MINLP Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities It’s Actually Important!

• We want to approximate the convex hull of integer solutions, but without a linear objective function, the solution to the relaxation might occur in the interior. • No Separating Hyperplane! :-(

2 2 min(y1 − 1/2) + (y2 − 1/2) y2

s.t. y1 ∈ {0, 1}, y2 ∈ {0, 1}

(ˆy1, yˆ2) 2 2 η ≥ (y1 − 1/2) + (y2 − 1/2)

η y1

Leyﬀer & Linderoth MINLP Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities Valid Inequalities From Relaxations • Idea: Inequalities valid for a relaxation are valid for original • Generating valid inequalities for a relaxation is often easier.

xˆ S • Separation Problem over T:

π Givenx, ˆ T ﬁnd (π, π0) such T = T that π xˆ > π0, x T T π x ≤ π0∀x ∈ T

Leyﬀer & Linderoth MINLP • If P = {x ∈ {0, 1}n | Ax ≤ b}, then for any row i, n T Pi = {x ∈ {0, 1} | ai x ≤ bi} is a relaxation of P . • If the intersection of the relaxations is a good approximation to the true problem, then the inequalities will be quite useful. • Crowder et al. (1983) is the seminal paper that shows this to be true for IP. • MINLP: Single (linear) row relaxations are also valid ⇒ same inequalities can also be used

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities Simple Relaxations

• Idea: Consider one row relaxations

Leyﬀer & Linderoth MINLP • If the intersection of the relaxations is a good approximation to the true problem, then the inequalities will be quite useful. • Crowder et al. (1983) is the seminal paper that shows this to be true for IP. • MINLP: Single (linear) row relaxations are also valid ⇒ same inequalities can also be used

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities Simple Relaxations

• Idea: Consider one row relaxations • If P = {x ∈ {0, 1}n | Ax ≤ b}, then for any row i, n T Pi = {x ∈ {0, 1} | ai x ≤ bi} is a relaxation of P .

Leyﬀer & Linderoth MINLP • Crowder et al. (1983) is the seminal paper that shows this to be true for IP. • MINLP: Single (linear) row relaxations are also valid ⇒ same inequalities can also be used

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities Simple Relaxations

Leyﬀer & Linderoth MINLP • MINLP: Single (linear) row relaxations are also valid ⇒ same inequalities can also be used

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities Simple Relaxations

• Idea: Consider one row relaxations • If P = {x ∈ {0, 1}n | Ax ≤ b}, then for any row i, n T Pi = {x ∈ {0, 1} | ai x ≤ bi} is a relaxation of P . • If the intersection of the relaxations is a good approximation to the true problem, then the inequalities will be quite useful. • Crowder et al. (1983) is the seminal paper that shows this to be true for IP.

Leyﬀer & Linderoth MINLP Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities Simple Relaxations

Leyﬀer & Linderoth MINLP • If C ⊆ N is a cover, then the cover inequality X xj ≤ |C| − 1 j∈C

is a valid inequality for S • Sometimes (minimal) cover inequalities are facets of conv(K)

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities Knapsack Covers

K = {x ∈ {0, 1}n | aT x ≤ b}

P • A set C ⊆ N is a cover if j∈C aj > b • A cover C is a minimal cover if C \ j is not a cover ∀j ∈ C

Leyﬀer & Linderoth MINLP Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities Knapsack Covers

K = {x ∈ {0, 1}n | aT x ≤ b}

P • A set C ⊆ N is a cover if j∈C aj > b • A cover C is a minimal cover if C \ j is not a cover ∀j ∈ C

• If C ⊆ N is a cover, then the cover inequality X xj ≤ |C| − 1 j∈C

is a valid inequality for S • Sometimes (minimal) cover inequalities are facets of conv(K)

Leyﬀer & Linderoth MINLP Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities Example

7 K = {x ∈ {0, 1} | 11x1 + 6x2 + 6x3 + 5x4 + 5x5 + 4x6 + x7 ≤ 19}

7 LP (K) = {x ∈ [0, 1] | 11x1+6x2+6x3+5x4+5x5+4x6+x7 ≤ 19}

• (1, 1, 1/3, 0, 0, 0, 0) ∈ LP (K)

• CHOPPED OFF BY x1 + x2 + x3 ≤ 2 • (0, 0, 1, 1, 1, 3/4, 0) ∈ LP (K)

• CHOPPED OFF BY x3 + x4 + x5 + x6 ≤ 3

Leyffer & Linderoth MINLP Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities Other Substructures • Single node flow: (Padberg et al., 1985)    |N| |N| X  S = x ∈ R+ , y ∈ {0, 1} | xj ≤ b, xj ≤ ujyj ∀ j ∈ N  j∈N  • Knapsack with single continuous variable: (Marchand and Wolsey, 1999)    |N| X  S = x ∈ R+, y ∈ {0, 1} | ajyj ≤ b + x  j∈N  • Set Packing: (Borndörfer and Weismantel, 2000) n o S = y ∈ {0, 1}|N| | Ay ≤ e

A ∈ {0, 1}|M|×|N|, e = (1, 1,..., 1)T

Leyﬀer & Linderoth MINLP m 1. Choose nonnegative multipliers u ∈ R+ T T P T T 2. u Ay ≤ u b is a valid inequality ( j∈N u ajyj ≤ u b). P T T 3. j∈N bu ajcyj ≤ u b (Since y ≥ 0). P T T T 4. j∈N bu ajcyj ≤ bu bc is valid for S since bu ajcyj is an integer

m×n • Let the columns of A ∈ R be denoted by {a1, a2, . . . an} n • S = {y ∈ Z+ | Ay ≤ b}.

• Simply Amazing: This simple procedure suﬃces to generate every valid inequality for an integer program

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities The Chv´atal-Gomory Procedure

• A general procedure for generating valid inequalities for integer programs

n • S = {y ∈ Z+ | Ay ≤ b}.

• Simply Amazing: This simple procedure suﬃces to generate every valid inequality for an integer program

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities The Chv´atal-Gomory Procedure

• A general procedure for generating valid inequalities for integer programs m×n • Let the columns of A ∈ R be denoted by {a1, a2, . . . an}

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities The Chv´atal-Gomory Procedure

• A general procedure for generating valid inequalities for integer programs m×n • Let the columns of A ∈ R be denoted by {a1, a2, . . . an} n • S = {y ∈ Z+ | Ay ≤ b}.

Leyﬀer & Linderoth MINLP T T P T T 2. u Ay ≤ u b is a valid inequality ( j∈N u ajyj ≤ u b). P T T 3. j∈N bu ajcyj ≤ u b (Since y ≥ 0). P T T T 4. j∈N bu ajcyj ≤ bu bc is valid for S since bu ajcyj is an integer • Simply Amazing: This simple procedure suﬃces to generate every valid inequality for an integer program

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities The Chv´atal-Gomory Procedure

Leyﬀer & Linderoth MINLP P T T 3. j∈N bu ajcyj ≤ u b (Since y ≥ 0). P T T T 4. j∈N bu ajcyj ≤ bu bc is valid for S since bu ajcyj is an integer • Simply Amazing: This simple procedure suﬃces to generate every valid inequality for an integer program

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities The Chv´atal-Gomory Procedure

Leyﬀer & Linderoth MINLP P T T T 4. j∈N bu ajcyj ≤ bu bc is valid for S since bu ajcyj is an integer • Simply Amazing: This simple procedure suﬃces to generate every valid inequality for an integer program

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities The Chv´atal-Gomory Procedure

Leyﬀer & Linderoth MINLP • Simply Amazing: This simple procedure suﬃces to generate every valid inequality for an integer program

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities The Chv´atal-Gomory Procedure

• A general procedure for generating valid inequalities for integer programs m×n • Let the columns of A ∈ R be denoted by {a1, a2, . . . an} n • S = {y ∈ Z+ | Ay ≤ b}. m 1. Choose nonnegative multipliers u ∈ R+ T T P T T 2. u Ay ≤ u b is a valid inequality ( j∈N u ajyj ≤ u b). P T T 3. j∈N bu ajcyj ≤ u b (Since y ≥ 0). P T T T 4. j∈N bu ajcyj ≤ bu bc is valid for S since bu ajcyj is an integer

Leyﬀer & Linderoth MINLP Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities The Chv´atal-Gomory Procedure

Leyﬀer & Linderoth MINLP Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities

Extension to MINLP (C¸ezik and Iyengar, 2005)

• This simple idea also extends to mixed 0-1 conic programming  minimize f T z  def  z =(x,y) subject to Az b  K  y ∈ {0, 1}p, 0 ≤ x ≤ U

• K: Homogeneous, self-dual, proper, convex cone

• x K y ⇔ (x − y) ∈ K

Leyﬀer & Linderoth MINLP def • Dual Cone: K∗ = {u | uT z ≥ 0 ∀z ∈ K} • Extension is clear from the following equivalence:

T T Az K b ⇔ u Az ≥ u b ∀u K∗ 0

• Many classes of nonlinear inequalities can be • Go to other SIAM Short represented as Course to ﬁnd out about Semideﬁnite Programming Ax Kq b or Ax Ks b

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities

Gomory On Cones (C¸ezik and Iyengar, 2005)

n • LP: Kl = R+ • SOCP: Kq = {(x0, x¯) | x0 ≥ kx¯k} T • SDP: Ks = {x = vec(X) | X = X ,X p.s.d}

Leyffer & Linderoth MINLP • Many classes of nonlinear inequalities can be • Go to other SIAM Short represented as Course to find out about Semidefinite Programming Ax Kq b or Ax Ks b

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities

Gomory On Cones (C¸ezik and Iyengar, 2005)

n • LP: Kl = R+ • SOCP: Kq = {(x0, x¯) | x0 ≥ kx¯k} T • SDP: Ks = {x = vec(X) | X = X ,X p.s.d} def • Dual Cone: K∗ = {u | uT z ≥ 0 ∀z ∈ K} • Extension is clear from the following equivalence:

T T Az K b ⇔ u Az ≥ u b ∀u K∗ 0

Leyﬀer & Linderoth MINLP Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities

Gomory On Cones (C¸ezik and Iyengar, 2005)

T T Az K b ⇔ u Az ≥ u b ∀u K∗ 0

• Many classes of nonlinear inequalities can be • Go to other SIAM Short represented as Course to ﬁnd out about Semideﬁnite Programming Ax Kq b or Ax Ks b

Leyﬀer & Linderoth MINLP • LP (X) • conv(X) 1 • y ≤ bbc + 1−f x is a valid inequality for X

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities Mixed Integer Rounding—MIR

Almost everything comes from considering the following very simple set, and observation.

• X = {(x, y) ∈ R × Z | y ≤ b + x}

• f = b − bbc: fractional y • NLP People are silly and use f for the objective function

Leyﬀer & Linderoth MINLP • conv(X) 1 • y ≤ bbc + 1−f x is a valid inequality for X

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities Mixed Integer Rounding—MIR

Almost everything comes from considering the following very simple set, and observation.

• X = {(x, y) ∈ R × Z | y ≤ b + x}

• f = b − bbc: fractional y • NLP People are silly and use f for the objective function • LP (X)

Leyﬀer & Linderoth MINLP 1 • y ≤ bbc + 1−f x is a valid inequality for X

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities Mixed Integer Rounding—MIR

Almost everything comes from considering the following very simple set, and observation.

• X = {(x, y) ∈ R × Z | y ≤ b + x}

• f = b − bbc: fractional y • NLP People are silly and use f for the objective function • LP (X) • conv(X)

Leyﬀer & Linderoth MINLP Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities Mixed Integer Rounding—MIR

Almost everything comes from considering the following very simple set, and observation. • X = {(x, y) ∈ R × Z | y ≤ b + x} 1−f • f = b − bbc: fractional • NLP People are silly and use f 1−f for the objective function

• LP (X) b y <= floor(b) + x/(1−f) • conv(X) f 1 • y ≤ bbc + 1−f x is a valid floor(b) inequality for X

Leyﬀer & Linderoth MINLP Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities Extension of MIR

   + − 2 |N| X + − X2 = (x , x , y) ∈ R+ × Z | ajyj + x ≤ b + x  j∈N 

• The inequality

+ − X (fj − f) x b(a )c + y ≤ bbc + j 1 − f j 1 − f j∈N

is valid for X2 def + def • fj = aj − bajc, (t) = max(t, 0) • X2 is a one-row relaxation of a general mixed integer program • Continuous variables aggregated into two: x+, x−

Leyﬀer & Linderoth MINLP 1. Show X2 ⊆ P 2. Show simple (2-variable) MIR inequality is valid for P (with an appropriate variable substitution). 3. Collect the terms

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities It’s So Easy, Even I Can Do It

Proof:

• N1 = {j ∈ N | fj ≤ f}

• N2 = N \ N1 • Let

2 |N| P = {(x, y) ∈ R+ × Z | X X − X bajcyj + dajeyy ≤ b + x + (1 − fj)yj}

j∈N1 j∈N2 j∈N2

Leyﬀer & Linderoth MINLP Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities It’s So Easy, Even I Can Do It

Proof:

• N1 = {j ∈ N | fj ≤ f}

• N2 = N \ N1 • Let

2 |N| P = {(x, y) ∈ R+ × Z | X X − X bajcyj + dajeyy ≤ b + x + (1 − fj)yj}

j∈N1 j∈N2 j∈N2

1. Show X2 ⊆ P 2. Show simple (2-variable) MIR inequality is valid for P (with an appropriate variable substitution). 3. Collect the terms

Leyﬀer & Linderoth MINLP Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities Gomory Mixed Integer Cut is a MIR Inequality

• Consider the set ( ) = + − 2 |N| X + − X = (x , x , y0, y) ∈ R+ × Z × Z+ | y0 + aj yj + x − x = b j∈N which is essentially the row of an LP tableau • Relax the equality to an inequality and apply MIR • Gomory Mixed Integer Cut:

X f X fj − f f y + x+ + x− + (f − )y ≥ f j j 1 − f j 1 − f j j∈N1 j∈N2

Leyﬀer & Linderoth MINLP • Create Gomory mixed integer cuts from T η ≥ fj + ∇fj (z − zj) T 0 ≥ cj + ∇cj (z − zj) • Akrotirianakis et al. (2001) shows modest improvements • Research Question: Other cut classes? • Research Question: Exploit “outer approximation” property?

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities

Using Gomory Cuts in MINLP (Akrotirianakis et al., 2001)

• LP/NLP Based Branch-and-Bound solves MILP instances:  minimize η  def  z =(x,y),η  T k subject to η ≥ fj + ∇fj (z − zj) ∀yj ∈ Y  0 ≥ c + ∇cT (z − z ) ∀y ∈ Y k  j j j j  x ∈ X, y ∈ Y integer

Leyﬀer & Linderoth MINLP • Research Question: Other cut classes? • Research Question: Exploit “outer approximation” property?

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities

Using Gomory Cuts in MINLP (Akrotirianakis et al., 2001)

• Create Gomory mixed integer cuts from T η ≥ fj + ∇fj (z − zj) T 0 ≥ cj + ∇cj (z − zj) • Akrotirianakis et al. (2001) shows modest improvements

Leyﬀer & Linderoth MINLP Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities

Using Gomory Cuts in MINLP (Akrotirianakis et al., 2001)

• Create Gomory mixed integer cuts from T η ≥ fj + ∇fj (z − zj) T 0 ≥ cj + ∇cj (z − zj) • Akrotirianakis et al. (2001) shows modest improvements • Research Question: Other cut classes? • Research Question: Exploit “outer approximation” property?

Leyﬀer & Linderoth MINLP def • C = conv({x ∈ C | y ∈ {0, 1}p}) 0/1 def • Cj = {z ∈ C|yj = 0/1}   z = λ0u0 + λ1u1 def   let Mj(C) = λ0 + λ1 = 1, λ0, λ1 ≥ 0  0 1  u0 ∈ Cj , u1 ∈ Cj

⇒ Pj(C) := projection of Mj(C) onto z

⇒ Pj(C) = conv (C ∩ yj ∈ {0, 1}) and P1...p(C) = C

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities

Leyﬀer & Linderoth MINLP 0/1 def • Cj = {z ∈ C|yj = 0/1}   z = λ0u0 + λ1u1 def   let Mj(C) = λ0 + λ1 = 1, λ0, λ1 ≥ 0  0 1  u0 ∈ Cj , u1 ∈ Cj

⇒ Pj(C) := projection of Mj(C) onto z

⇒ Pj(C) = conv (C ∩ yj ∈ {0, 1}) and P1...p(C) = C

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities

Leyﬀer & Linderoth MINLP Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities

Disjunctive Cuts for MINLP (Stubbs and Mehrotra, 1999) Extension of Disjunctive Cuts for MILP: (Balas, 1979; Balas et al., 1993) def Continuous relaxation (z = (x, y)) x def • C = {z|c(z) ≤ 0, 0 ≤ y ≤ 1, 0 ≤ x ≤ U} def • C = conv({x ∈ C | y ∈ {0, 1}p}) integer 0/1 def feasible • Cj = {z ∈ C|yj = 0/1} set   y z = λ0u0 + λ1u1 def   let Mj(C) = λ0 + λ1 = 1, λ0, λ1 ≥ 0  0 1  u0 ∈ Cj , u1 ∈ Cj

⇒ Pj(C) := projection of Mj(C) onto z

⇒ Pj(C) = conv (C ∩ yj ∈ {0, 1}) and P1...p(C) = C

Leyﬀer & Linderoth MINLP Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities Disjunctive Cuts: Example

minimize x | (x − 1/2)2 + (y − 3/4)2 ≤ 1, −2 ≤ x ≤ 2, y ∈ {0, 1} x,y

x Givenz ˆ withy ˆj 6∈ {0, 1} ﬁnd separating hyperplane ( C0 C1 minimize kz − zˆk j j ⇒ z subject to z ∈ Pj(C) y

zˆ = (ˆx, yˆ)

Leyﬀer & Linderoth MINLP NONCONVEX

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities Disjunctive Cuts Example 0 1 Cj Cj z∗ def= arg min kz − zˆk

s.t. λ0u0 + λ1u1 = z λ0 + λ1 = 1 −0.16 0.66 ≤ u ≤ 0 0 1 −0.47 1.47 z∗ ≤ u ≤ 0 1 1 λ0, λ1 ≥ 0

zˆ = (ˆx, yˆ)

Leyﬀer & Linderoth MINLP NONCONVEX

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities Disjunctive Cuts Example 0 1 Cj Cj ∗ def 2 z = arg min kz − zˆk2

s.t. λ0u0 + λ1u1 = z λ0 + λ1 = 1 −0.16 0.66 ≤ u ≤ 0 0 1 −0.47 1.47 z∗ ≤ u ≤ 0 1 1 λ0, λ1 ≥ 0

zˆ = (ˆx, yˆ)

Leyﬀer & Linderoth MINLP NONCONVEX

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities Disjunctive Cuts Example 0 1 Cj Cj ∗ def z = arg min kz − zˆk∞

s.t. λ0u0 + λ1u1 = z λ0 + λ1 = 1 −0.16 0.66 ≤ u ≤ 0 0 1 −0.47 1.47 z∗ ≤ u ≤ 0 1 1 λ0, λ1 ≥ 0

zˆ = (ˆx, yˆ)

Leyﬀer & Linderoth MINLP Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities

0 Disjunctive Cuts Example1 Cj Cj

z∗ def= arg min kz − zˆk

s.t. λ0u0 + λ1u1 = z λ0 + λ1 = 1 −0.16 0.66 ≤ u ≤ 0 0 1 −0.47 1.47 ≤ u ≤ z∗ 0 1 1 λ0, λ1 ≥ 0

zˆ = (ˆx, yˆ) NONCONVEX

Leyﬀer & Linderoth MINLP • Perspective gives a convex reformulation of Mj(C): Mj(C˜), where   µci(z/µ) ≤ 0   C˜ := (z, µ) 0 ≤ µ ≤ 1

 0 ≤ x ≤ µU, 0 ≤ y ≤ µ 

• c(0/0) = 0 ⇒ convex representation

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities

What to do? (Stubbs and Mehrotra, 1999)

• Look at the perspective of c(z)

P(c(˜z), µ) = µc(˜z/µ)

• Think ofz ˜ = µz

Leyﬀer & Linderoth MINLP Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities

What to do? (Stubbs and Mehrotra, 1999)

• Look at the perspective of c(z)

P(c(˜z), µ) = µc(˜z/µ)

• Think ofz ˜ = µz

• Perspective gives a convex reformulation of Mj(C): Mj(C˜), where   µci(z/µ) ≤ 0   C˜ := (z, µ) 0 ≤ µ ≤ 1

 0 ≤ x ≤ µU, 0 ≤ y ≤ µ 

• c(0/0) = 0 ⇒ convex representation

Leyﬀer & Linderoth MINLP Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities Disjunctive Cuts Example

 2 2    µ (x/µ − 1/2) + (y/µ − 3/4) − 1 ≤ 0  x   −2µ ≤ x ≤ 2µ  C˜ =  y  0 ≤ y ≤ µ  µ   0 ≤ µ ≤ 1  x µ

x 0 Cj 0 C1 Cj j C0 C1 C1 y j j y yµ j µ x

Leyﬀer & Linderoth MINLP • Take v0 ← µ0u0 v1 ← µ1u1 min kz − zˆk Solution to example:

s.t. v0 + v1 = z x∗ −0.401 = µ0 + µ1 = 1 y∗ 0.780 ˜0 (v0, µ0) ∈ Cj ˜1 (v1, µ1) ∈ Cj µ0, µ1 ≥ 0 • separating hyperplane: ψT (z − zˆ), where ψ ∈ ∂kz − zˆk

Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities Example, cont.

˜0 ˜1 Cj = {(z, µ) | yj = 0} Cj = {(z, µ) | yj = µ}

Leyﬀer & Linderoth MINLP Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities Example, cont.

˜0 ˜1 Cj = {(z, µ) | yj = 0} Cj = {(z, µ) | yj = µ}

• Take v0 ← µ0u0 v1 ← µ1u1 min kz − zˆk Solution to example:

s.t. v0 + v1 = z x∗ −0.401 = µ0 + µ1 = 1 y∗ 0.780 ˜0 (v0, µ0) ∈ Cj ˜1 (v1, µ1) ∈ Cj µ0, µ1 ≥ 0 • separating hyperplane: ψT (z − zˆ), where ψ ∈ ∂kz − zˆk

Leyﬀer & Linderoth MINLP Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities 0 1 Cj Example, Cont. Cj

2x∗ + 0.5 ψ = 0.198 2y∗ − 0.75 x + 0 .061 y = z∗ 0.198x + 0.061y ≥ −0.032 −0.032

zˆ = (ˆx, yˆ)

Leyﬀer & Linderoth MINLP Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities

Nonlinear Branch-and-Cut (Stubbs and Mehrotra, 1999)

• Can do this at all nodes of the branch-and-bound tree • Generalize disjunctive approach from MILP • solve one convex NLP per cut • Generalizes Sherali and Adams (1990) and Lov´asz and Schrijver (1991) • tighten cuts by adding semi-deﬁnite constraint • Stubbs and Mehrohtra (2002) also show how to generate convex quadratic inequalities, but computational results are not that promising

Leyﬀer & Linderoth MINLP Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities

Generalized Disjunctive Programming (Raman and Grossmann, 1994; Lee and Grossmann, 2000) Consider disjunctive NLP  P minimize fi + f(x)  x,Y       Yi ¬Yi W subject to  ci(x) ≤ 0   Bix = 0  ∀i ∈ I   fi = γi fi = 0  0 ≤ x ≤ U, Ω(Y ) = true, Y ∈ {true, false}p Application: process synthesis • Yi represents presence/absence of units • Bix = 0 eliminates variables if unit absent

Exploit disjunctive structure • special branching ... OA/GBD algorithms

Leyﬀer & Linderoth MINLP Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities

ci(x) ≤ M(1 − yi) −Myi ≤ Bix ≤ Myi fi = yiγi Ω(Y ) converted to linear inequalities

Leyﬀer & Linderoth MINLP Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities

x = vi1 + vi0, λi1 + λi0 = 1 λi1ci(vi1/λi1) ≤ 0,Bivi0 = 0 0 ≤ vij ≤ λijU, 0 ≤ λij ≤ 1, fi = λi1γi

Leyﬀer & Linderoth MINLP Formulations Preliminaries Inequalities MILP Inequalities Applied to MINLP Dealing with Nonconvexity Disjunctive Inequalities Disjunctive Programming: Example

Y1 2 2 x1 + x2 ≤ 1 Y2 ∨ 2 2

(x1 − 4) + (x2 − 1) ≤ 1

¢ ¢ £

big−M£

¡ ¡

¢ ¢

£ £

¡ ¡ convex hull Y3 ∨ 2 2 (x1 − 2) + (x2 − 4) ≤ 1 ⇒

Leyﬀer & Linderoth MINLP • Underestimate nonconvex functions. Solve relaxation. Provides lower bound. • If relaxation is not exact, then branch

Formulations Diﬃculties Inequalities Envelopes Dealing with Nonconvexity Bilinear Terms Dealing with Nonconvexities

• Functional nonconvexity causes serious problems. • Branch and bound must have true lower bound (global solution)

Leyﬀer & Linderoth MINLP • If relaxation is not exact, then branch

Formulations Diﬃculties Inequalities Envelopes Dealing with Nonconvexity Bilinear Terms Dealing with Nonconvexities

• Functional nonconvexity causes serious problems. • Branch and bound must have true lower bound (global solution) • Underestimate nonconvex functions. Solve relaxation. Provides lower bound.

Leyﬀer & Linderoth MINLP Formulations Diﬃculties Inequalities Envelopes Dealing with Nonconvexity Bilinear Terms Dealing with Nonconvexities

• Functional nonconvexity causes serious problems. • Branch and bound must have true lower bound (global solution) • Underestimate nonconvex functions. Solve relaxation. Provides lower bound. • If relaxation is not exact, then branch

Leyﬀer & Linderoth MINLP Formulations Diﬃculties Inequalities Envelopes Dealing with Nonconvexity Bilinear Terms Dealing with Nonconvex Constraints

• If nonconvexity in constraints, may need to overestimate and underestimate the function to get a convex region

Leyﬀer & Linderoth MINLP • Concave Envelope (cavΩ(f)): Pointwise inﬁmum of concave overestimators of f over Ω.

Formulations Diﬃculties Inequalities Envelopes Dealing with Nonconvexity Bilinear Terms

Envelopes

f :Ω → R

• Convex Envelope (vexΩ(f)): Pointwise supremum of convex underestimators of f over Ω.

Leyﬀer & Linderoth MINLP Formulations Diﬃculties Inequalities Envelopes Dealing with Nonconvexity Bilinear Terms

Envelopes

f :Ω → R

• Convex Envelope (vexΩ(f)): Pointwise supremum of convex underestimators of f over Ω. • Concave Envelope (cavΩ(f)): Pointwise inﬁmum of concave overestimators of f over Ω.

Leyﬀer & Linderoth MINLP • General nonconvex functions f(x) can be underestimated over a region [l, u] “overpowering” the function with a quadratic function that is ≤ 0 on the region of interest

n X L(x) = f(x) + αi(li − xi)(ui − xi) i=1 Refs: (McCormick, 1976; Adjiman et al., 1998; Tawarmalani and Sahinidis, 2002)

Formulations Diﬃculties Inequalities Envelopes Dealing with Nonconvexity Bilinear Terms Branch-and-Bound Global Optimization Methods

• Under/Overestimate “simple” parts of (Factorable) Functions individually • Bilinear Terms • Trilinear Terms • Fractional Terms • Univariate convex/concave terms

Leyﬀer & Linderoth MINLP Formulations Diﬃculties Inequalities Envelopes Dealing with Nonconvexity Bilinear Terms Branch-and-Bound Global Optimization Methods

• Under/Overestimate “simple” parts of (Factorable) Functions individually • Bilinear Terms • Trilinear Terms • Fractional Terms • Univariate convex/concave terms • General nonconvex functions f(x) can be underestimated over a region [l, u] “overpowering” the function with a quadratic function that is ≤ 0 on the region of interest

n X L(x) = f(x) + αi(li − xi)(ui − xi) i=1 Refs: (McCormick, 1976; Adjiman et al., 1998; Tawarmalani and Sahinidis, 2002)

Leyﬀer & Linderoth MINLP Formulations Diﬃculties Inequalities Envelopes Dealing with Nonconvexity Bilinear Terms Bilinear Terms

The convex and concave envelopes of the bilinear function xy over a rectangular region

def 2 R = {(x, y) ∈ R | lx ≤ x ≤ ux, ly ≤ y ≤ uy} are given by the expressions

vexxyR(x, y) = max{lyx + lxy − lxly, uyx + uxy − uxuy} cavxyR(x, y) = min{uyx + lxy − lxuy, lyx + uxy − uxly}

Leyﬀer & Linderoth MINLP Formulations Diﬃculties Inequalities Envelopes Dealing with Nonconvexity Bilinear Terms Worth 1000 Words? xy

Leyﬀer & Linderoth MINLP Formulations Diﬃculties Inequalities Envelopes Dealing with Nonconvexity Bilinear Terms Worth 1000 Words?

vexR(xy)

Leyﬀer & Linderoth MINLP Formulations Diﬃculties Inequalities Envelopes Dealing with Nonconvexity Bilinear Terms Worth 1000 Words?

cavR(xy)

Leyﬀer & Linderoth MINLP Formulations Diﬃculties Inequalities Envelopes Dealing with Nonconvexity Bilinear Terms

Disaggregation Tawarmalani et al. (2002) Consider convex problem with bilinear objective  Pn minimize w i=1 cixi  w,x1,...,xn subject to (w, x) ∈ P Polyhedron  0 ≤ w ≤ v 0 ≤ x ≤ u

Formulation #1 Formulation #2 min z n s.t. (w, x) ∈ P X min z 0 ≤ z i n n i=1 X X s.t. (w, x) ∈ P ( ciui)w + v( cixi) i=1 i=1 0 ≤ zi ∀i n X ciuiw + vcixi −v( ciui) ≤ 0 −vciui ≤ 0 ∀i i=1 Leyﬀer & Linderoth MINLP Formulations Diﬃculties Inequalities Envelopes Dealing with Nonconvexity Bilinear Terms

Formulation #2

n X min zi Formulation #2 is better! i=1 s.t. (w, x) ∈ P 0 ≤ zi ∀i ciuiw + vcixi −vciui ≤ 0 ∀i

Leyﬀer & Linderoth MINLP Formulations Diﬃculties Inequalities Envelopes Dealing with Nonconvexity Bilinear Terms Summary

• MINLP: Good relaxations are important • Relaxations can be improved • Statically: Better formulation/preprocessing • Dynamically: Cutting planes • Nonconvex MINLP: • Methods exist, again based on relaxations • Tight relaxations is an active area of research • Lots of empirical questions remain

Leyﬀer & Linderoth MINLP Special Ordered Sets Parallel BB and Grid Computing Implementation & Software Issues

Part IV

Implementation and Software

Leyﬀer & Linderoth MINLP Special Ordered Sets Parallel BB and Grid Computing Implementation & Software Issues Implementation and Software for MINLP

1. Special Ordered Sets 2. Parallel BB and Grid Computing 3. Implementation & Software Issues

Leyﬀer & Linderoth MINLP Special Ordered Sets Special Ordered Sets of Type 1 Parallel BB and Grid Computing Special Ordered Sets of Type 2 Implementation & Software Issues Special Ordered Sets of Type 3 Special Ordered Sets of Type 1

P SOS1: λi = 1 & at most one λi is nonzero

Example 1: d ∈ {d1, . . . , dp} discrete diameters P ⇔ d = λidi and {λ1, . . . , λp} is SOS1 P P ⇔ d = λidi and λi = 1 and λi ∈ {0, 1}

... d is convex combination with coeﬃcients λi

Example 2: nonlinear function c(y) of single integer P P ⇔ y = iλi and c = c(i)λi and {λ1, . . . , λp} is SOS1

References: (Beale, 1979; Nemhauser, G.L. and Wolsey, L.A., 1988; Williams, 1993) . . .

Leyﬀer & Linderoth MINLP 3. ﬁnd t : at < a ≤ at+1

4. branch: {λt+1, . . . , λp} = 0 or {λ1, . . . , λt} = 0

Special Ordered Sets Special Ordered Sets of Type 1 Parallel BB and Grid Computing Special Ordered Sets of Type 2 Implementation & Software Issues Special Ordered Sets of Type 3 Special Ordered Sets of Type 1

P SOS1: λi = 1 & at most one λi is nonzero

Branching on SOS1

1. reference row a1 < . . . < ap e.g. diameters P 2. fractionality: a := aiλi

a < a a > a t t+1

P SOS1: λi = 1 & at most one λi is nonzero

Branching on SOS1

1. reference row a1 < . . . < ap e.g. diameters P 2. fractionality: a := aiλi

3. ﬁnd t : at < a ≤ at+1

4. branch: {λt+1, . . . , λp} = 0 or {λ1, . . . , λt} = 0 a < a a > a t t+1

Leyﬀer & Linderoth MINLP P • x = λixi P • z = λizi

• {λ1, . . . , λp} is SOS2 . . . convex combination of two breakpoints . . .

P SOS2: λi = 1 & at most two adjacent λi nonzero

Example: Approximation of nonlinear function z = z(x)

z(x) • breakpoints x1 < . . . < xp

• function values zi = z(xi) • piece-wise linear

P SOS2: λi = 1 & at most two adjacent λi nonzero

Example: Approximation of nonlinear function z = z(x)

z(x) • breakpoints x1 < . . . < xp

• function values zi = z(xi) • piece-wise linear P • x = λixi P • z = λizi

x • {λ1, . . . , λp} is SOS2 . . . convex combination of two breakpoints . . .

Leyﬀer & Linderoth MINLP 3. ﬁnd t : at < a ≤ at+1

4. branch: {λt+1, . . . , λp} = 0 or {λ1, . . . , λt−1}

P SOS2: λi = 1 & at most two adjacent λi nonzero

Branching on SOS2

1. reference row a1 < . . . < ap e.g. ai = xi P 2. fractionality: a := aiλi

x < a x > a t t

P SOS2: λi = 1 & at most two adjacent λi nonzero

Branching on SOS2

1. reference row a1 < . . . < ap e.g. ai = xi P 2. fractionality: a := aiλi

3. ﬁnd t : at < a ≤ at+1

4. branch: {λt+1, . . . , λp} = 0 or {λ1, . . . , λt−1} x < a x > a t t

Leyﬀer & Linderoth MINLP P • v = λijvi P • w = λijwj P • u = λijuij P 1 = λij is SOS3 . . .

Example: Approximation of 2D function u = g(v, w)

Triangularization of [vL, vU ] × [wL, wU ] domain

1. vL = v1 < . . . < vk = vU

2. wL = w1 < . . . < wl = wU w

3. function uij := g(vi, wj)

4. λij weight of vertex (i, j)

Leyﬀer & Linderoth MINLP P 1 = λij is SOS3 . . .

Example: Approximation of 2D function u = g(v, w)

Triangularization of [vL, vU ] × [wL, wU ] domain

1. vL = v1 < . . . < vk = vU

2. wL = w1 < . . . < wl = wU w

3. function uij := g(vi, wj)

4. λij weight of vertex (i, j) P • v = λijvi P • w = λijwj P • u = λijuij v

Example: Approximation of 2D function u = g(v, w)

Triangularization of [vL, vU ] × [wL, wU ] domain

1. vL = v1 < . . . < vk = vU

2. wL = w1 < . . . < wl = wU w

3. function uij := g(vi, wj)

4. λij weight of vertex (i, j) P • v = λijvi P • w = λijwj P • u = λijuij v P 1 = λij is SOS3 . . .

P SOS3: λij = 1 & set condition holds

w P 1. v = λijvi ... convex combinations P 2. w = λijwj P 3. u = λijuij

{λ11, . . . , λkl} satisﬁes set condition v ⇔ ∃ trangle ∆ : {(i, j): λij > 0} ⊂ ∆ violates set condn i.e. nonzeros in single triangle ∆

• compute centers: P vˆ = λijvi & P wˆ = λijwi • ﬁnd s, t such that vs ≤ vˆ < vs+1 & v ws ≤ wˆ < ws+1 • branch on v or w violates set condn

• compute centers: P vˆ = λijvi & P wˆ = λijwi • ﬁnd s, t such that vs ≤ vˆ < vs+1 & ws ≤ wˆ < ws+1 • branch on v or w X X vertical branching: λij = 1 λij = 1 L R

• compute centers: P vˆ = λijvi & P wˆ = λijwi • ﬁnd s, t such that vs ≤ vˆ < vs+1 & ws ≤ wˆ < ws+1 • branch on v or w X X horizontal branching: λij = 1 λij = 1 T B

Example: gas network from ﬁrst lecture . . . • pressure loss p across pipe is related to ﬂow rate f as q 2 2 −1 2 2 −1 2 pin − pout = Ψ sign(f)f ⇔ pin = pout + Ψ sign(f)f

where Ψ: “Friction Factor” • nonconvex equation u = g(v, w) . . . assume pressures needed elsewhere • (Martin et al., 2005) use SOS3 model . . . study polyhedral properties . . . solve medium sized problem

Leyﬀer & Linderoth MINLP Special Ordered Sets Parallel BB and Grid Computing Implementation & Software Issues Parallel Branch-and-Bound meta-computing platforms: • set of distributed heterogeneous computers, e.g. ◦ pool of workstations ◦ group of supercomputers or anything

• low quality with respect to bandwidth, latency, availability

• low cost: it’s free!!! ... huge amount of resources

... use Condor to “build” MetaComputer ... high-throughput computing

Leyﬀer & Linderoth MINLP Special Ordered Sets Parallel BB and Grid Computing Implementation & Software Issues Parallel Branch-and-Bound

Master Worker Paradigm (MWdriver) Object oriented C++ library on top of Condor-PVM

Master Condor−PVM Universe

GLOBAL DATA Task Pool

T1T2 T3 T4 T5 T6

Wid Wid 1 4 Wid Wid 2 3 W1 W4 PVM W2 comm W3

Condor−Workers

Fault tolerance via master check-pointing

Leyﬀer & Linderoth MINLP Special Ordered Sets Parallel BB and Grid Computing Implementation & Software Issues Parallel Branch-and-Bound

First Strategy: 1 worker ≡ 1 NLP ⇒ grain-size too small ... NLPs solve in seconds

New Strategy: 1 worker ≡ 1 subtree (MINLP) ... “streamers” running down tree Worker 1 Worker 2 Worker 3

Important: workers remove “small tasks” ... before returning tree to master

Leyﬀer & Linderoth MINLP Special Ordered Sets Parallel BB and Grid Computing Implementation & Software Issues Parallel Branch-and-Bound

Trimloss optimization with 56 general integers ⇒ solve 96,408 MINLPs on 62.7 workers ⇒ 600,518,018 NLPs

Wall clock time = 15.5 hours Cumulative worker CPU time = 752.7 hours ' 31 days work-time 752.7 eﬃciency := = = 80.5 work × job-time 62.7 × 15.5 ... proportion of time workers were busy

Leyﬀer & Linderoth MINLP Special Ordered Sets Parallel BB and Grid Computing Implementation & Software Issues Parallel Branch-and-Bound: Results

Leyﬀer & Linderoth MINLP Special Ordered Sets Detecting Infeasibility Parallel BB and Grid Computing Choice of NLP Solver Implementation & Software Issues MINLP Software Detecting Infeasibility

NLP node inconsistent (BB, OA, GBD) ⇒ NLP solver must prove infeasibility ⇒ solve feasibility problem: restoration

( minimize kc+(x, y)k (F ) x,y subject to x ∈ X, y ∈ Yˆ where c+(x, y) = max (c(x, y), 0) and k k any norm

If ∃ solution (ˆx, yˆ) such that kc+(ˆx, yˆ)k > 0 ⇒ no feasible point (if convex) in neighborhood (if nonconvex)

Leyﬀer & Linderoth MINLP Special Ordered Sets Detecting Infeasibility Parallel BB and Grid Computing Choice of NLP Solver Implementation & Software Issues MINLP Software Feasibility Cuts for OA et al.

Yˆ = {yˆ} singleton & c(c, y) convex (ˆx, yˆ) solves F (ˆy) with kc+(ˆx, yˆ)k > 0 ⇒ valid cut to eliminatey ˆ given by

x − xˆ 0 ≥ c+(ˆx, yˆ) +γ ˆT y − yˆ whereγ ˆ ∈ ∂kc+(ˆx, yˆ)k subdiﬀerential Polyhedral norms:γ ˆ = ∇cλˆ where P + 1. `∞ norm: λi = 1, and 0 ≤ λi ⊥ cˆi ≤ kcˆ k

2. `1 norm: 0 ≤ λi ≤ 1 ⊥ −cˆi ... λ multipliers of equivalent smooth NLP . . . easy exercise

Leyﬀer & Linderoth MINLP Special Ordered Sets Detecting Infeasibility Parallel BB and Grid Computing Choice of NLP Solver Implementation & Software Issues MINLP Software Geometry of Feasibility Cuts

y=3

feasible y=1 x y = 3 infeasible

y=3

feasible y=1 x solution to feasibility problem

y feasibility cuts y=3

feasible y=1 x feasibility cuts for OA

Leyﬀer & Linderoth MINLP Special Ordered Sets Detecting Infeasibility Parallel BB and Grid Computing Choice of NLP Solver Implementation & Software Issues MINLP Software Infeasibility in Branch-and-Bound

FilterSQP restoration phase

• satisﬁable constraints: J := {j : cj(ˆx, yˆ) ≤ 0} • violated constraints J ⊥ (complement of J)  X minimize cj(x, y)  x,y  j∈J⊥ subject to c (x, y) ≤ 0 ∀j ∈ J  j  x ∈ X, y ∈ Yˆ

• + + ﬁlter SQP algorithm on kcJ k and kcJ⊥ k ⇒ 2nd order convergence • adaptively change J

• similar to `1-norm, but λi 6≤ 1

Leyﬀer & Linderoth MINLP Special Ordered Sets Detecting Infeasibility Parallel BB and Grid Computing Choice of NLP Solver Implementation & Software Issues MINLP Software Choice of NLP Solver

MILP/MIQP branch-and-bound • (ˆx, yˆ) solution to parent node

• new bound: yi ≥ byˆic added to parent LP/QP ⇒ dual active set method; (ˆx, yˆ) dual feasible ⇒ fast re-optimization (MIQP 2-3 pivots!)

MILP exploit factorization of constraint basis ⇒ no re-factorization, just updates . . . also works for MIQP (KKT matrix factorization)

⇒ interior-point methods not competitive ... how to check λi > 0 for SOS branching ??? ... how to warm-start IPMs ???

Leyﬀer & Linderoth MINLP k k • factorized KKT system at (x , y ) to ﬁnd step (dx, dy) • NLP solution: k+1 k+1 k k (ˆx, yˆ) = (x , y ) = (x + αdx, y + αdy) but KKT system at (xk+1, yk+1) never factorized . . . SQP methods take 2-3 iterations (good active set) Outer Approximation et al. no good warm start (y changes too much) ⇒ interior-point methods or SQP can be used

Special Ordered Sets Detecting Infeasibility Parallel BB and Grid Computing Choice of NLP Solver Implementation & Software Issues MINLP Software Choice of NLP Solver

MINLP branch-and-bound • (ˆx, yˆ) solution to parent node

• new bound: yi ≥ byˆic added to parent NLP

Snag: ∇c(x, y), ∇2L etc. change ...

Leyﬀer & Linderoth MINLP • NLP solution: k+1 k+1 k k (ˆx, yˆ) = (x , y ) = (x + αdx, y + αdy) but KKT system at (xk+1, yk+1) never factorized . . . SQP methods take 2-3 iterations (good active set) Outer Approximation et al. no good warm start (y changes too much) ⇒ interior-point methods or SQP can be used

Special Ordered Sets Detecting Infeasibility Parallel BB and Grid Computing Choice of NLP Solver Implementation & Software Issues MINLP Software Choice of NLP Solver

MINLP branch-and-bound • (ˆx, yˆ) solution to parent node

• new bound: yi ≥ byˆic added to parent NLP

Snag: ∇c(x, y), ∇2L etc. change ... k k • factorized KKT system at (x , y ) to ﬁnd step (dx, dy)

Leyﬀer & Linderoth MINLP Special Ordered Sets Detecting Infeasibility Parallel BB and Grid Computing Choice of NLP Solver Implementation & Software Issues MINLP Software Choice of NLP Solver

MINLP branch-and-bound • (ˆx, yˆ) solution to parent node

• new bound: yi ≥ byˆic added to parent NLP

Snag: ∇c(x, y), ∇2L etc. change ... k k • factorized KKT system at (x , y ) to ﬁnd step (dx, dy) • NLP solution: k+1 k+1 k k (ˆx, yˆ) = (x , y ) = (x + αdx, y + αdy) but KKT system at (xk+1, yk+1) never factorized . . . SQP methods take 2-3 iterations (good active set) Outer Approximation et al. no good warm start (y changes too much) ⇒ interior-point methods or SQP can be used

Leyﬀer & Linderoth MINLP Special Ordered Sets Detecting Infeasibility Parallel BB and Grid Computing Choice of NLP Solver Implementation & Software Issues MINLP Software Software for MINLP

• Outer Approximation: DICOPT++

• Branch-and-Bound Solvers: SBB & MINLP

• Global MINLP: BARON & MINOPT

• Online Tools: MINLP World, MacMINLP & NEOS

Leyﬀer & Linderoth MINLP Special Ordered Sets Detecting Infeasibility Parallel BB and Grid Computing Choice of NLP Solver Implementation & Software Issues MINLP Software Outer Approximation: DICOPT++ Outer approximation with equality relaxation & penalty Reference: (Kocis and Grossmann, 1989) Features: • GAMS interface • NLP solvers: CONOPT, MINOS, SNOPT • MILP solvers: CPLEX, OSL2 • solve root NLP, or NLP(y0) initially • relax linearizations of nonlinear equalities: λi multiplier of ci(z) = 0 . . . T ≥ 0 if λi > 0 ci(ˆz) + ∇ci(ˆz) (z − zˆ) ≤ 0 if λi < 0

• heuristic stopping rule: STOP if NLP(yj) gets worse AIMMS has version of outer approximation Leyﬀer & Linderoth MINLP Special Ordered Sets Detecting Infeasibility Parallel BB and Grid Computing Choice of NLP Solver Implementation & Software Issues MINLP Software SBB: (Bussieck and Drud, 2000)

Features: • GAMS branch-and-bound solver • variable types: integer, binary, SOS1, SOS2, semi-integer • variable selection: integrality, pseudo-costs • node selection: depth-ﬁrst, best bound, best estimate • multiple NLP solvers: CONOPT, MINOS, SNOPT ⇒ multiple solves if NLP fails Comparison to DICOPT (OA): • DICOPT better, if combinatorial part dominates • SBB better, if diﬃcult nonlinearities

Leyﬀer & Linderoth MINLP Special Ordered Sets Detecting Infeasibility Parallel BB and Grid Computing Choice of NLP Solver Implementation & Software Issues MINLP Software MINLPBB: (Leyﬀer, 1998)

Features: • AMPL branch-and-bound solver • variable types: integer, binary, SOS1 • variable selection: integrality, priorities • node selection: depth-ﬁrst & best bound after infeasible node • NLP solver: ﬁlterSQP ⇒ feasibility restoration • CUTEr interface available

Leyﬀer & Linderoth MINLP Special Ordered Sets Detecting Infeasibility Parallel BB and Grid Computing Choice of NLP Solver Implementation & Software Issues MINLP Software Global MINLP Solvers

α-BB & MINOPT: (Schweiger and Floudas, 1998) • problem classes: MINLP, DAE, optimal control, etc • multiple solvers: OA, GBD, MINOS, CPLEX • own modeling language BARON: (Sahinidis, 2000) • global optimization from underestimators & branching • range reduction important • classes of underestimators & factorable NLP exception: cannot handle sin(x), cos(x) • CPLEX, MINOS, SNOPT, OSL • mixed integer semi-deﬁnite optimization: SDPA

Leyﬀer & Linderoth MINLP Special Ordered Sets Detecting Infeasibility Parallel BB and Grid Computing Choice of NLP Solver Implementation & Software Issues MINLP Software Online Tools

Model Libraries • MINLP World www.gamsworld.org/minlp/ scalar GAMS models ... diﬃcult to read • GAMS library www.gams.com/modlib/modlib.htm • MacMINLP www.mcs.anl.gov/~leyffer/macminlp/ AMPL models NEOS Server • MINLP solvers: SBB (GAMS), MINLPBB (AMPL) • MIQP solvers: FORTMP, XPRESS

Leyﬀer & Linderoth MINLP Special Ordered Sets Detecting Infeasibility Parallel BB and Grid Computing Choice of NLP Solver Implementation & Software Issues MINLP Software COIN-OR

http://www.coin-or.org • COmputational INfrastructure for Operations Research • A library of (interoperable) software tools for optimization • A development platform for open source projects in the OR community • Possibly Relevant Modules: • OSI: Open Solver Interface • CGL: Cut Generation Library • CLP: Coin Linear Programming Toolkit • CBC: Coin Branch and Cut • IPOPT: Interior Point OPTimizer for NLP • NLPAPI: NonLinear Programming API

Leyﬀer & Linderoth MINLP “MINLP solvers lag 15 years behind MIP solvers”

⇒ many research opportunities!!!

Special Ordered Sets Detecting Infeasibility Parallel BB and Grid Computing Choice of NLP Solver Implementation & Software Issues MINLP Software Conclusions

MINLP rich modeling paradigm ◦ most popular solver on NEOS

Algorithms for MINLP: ◦ Branch-and-bound (branch-and-cut) ◦ Outer approximation et al.

Leyﬀer & Linderoth MINLP Special Ordered Sets Detecting Infeasibility Parallel BB and Grid Computing Choice of NLP Solver Implementation & Software Issues MINLP Software Conclusions

MINLP rich modeling paradigm ◦ most popular solver on NEOS

Algorithms for MINLP: ◦ Branch-and-bound (branch-and-cut) ◦ Outer approximation et al.

“MINLP solvers lag 15 years behind MIP solvers”

⇒ many research opportunities!!!

Leyﬀer & Linderoth MINLP Part V

References

Leyﬀer & Linderoth MINLP References

C. Adjiman, S. Dallwig, C. A. Floudas, and A. Neumaier. A global optimization method, aBB, for general twice-differentiable constrained NLPs - I. Theoretical advances. Computers and Chemical Engineering, 22:1137–1158, 1998. I. Akrotirianakis, I. Maros, and B. Rustem. An outer approximation based branch-and-cut algorithm for convex 0-1 MINLP problems. Optimization Methods and Software, 16:21–47, 2001. E. Balas. Disjunctive programming. In Annals of Discrete Mathematics 5: Discrete Optimization, pages 3–51. North Holland, 1979. E. Balas, S. Ceria, and G. Corneujols. A lift-and-project cutting plane algorithm for mixed 0-1 programs. Mathematical Programming, 58:295–324, 1993. E. M. L. Beale. Branch-and-bound methods for mathematical programming systems. Annals of Discrete Mathematics, 5:201–219, 1979. B. Borchers and J. E. Mitchell. An improved branch and bound algorithm for Mixed Integer Nonlinear Programming. Computers and Operations Research, 21(4): 359–367, 1994. R. Borndörferand R. Weismantel. Set packing relaxations of some integer programs. Mathematical Programming, 88:425 – 450, 2000. M. R. Bussieck and A. Drud. SBB: A new solver for mixed integer nonlinear programming. In Recent Advances in Nonlinear Mixed Integer Optimization. INFORMS Fall, 2000. Invited talk. M. T. Çezik and G. Iyengar. Cuts for mixed 0-1 conic programming. Mathematical Programming, 2005. to appear. Leyffer & Linderoth MINLP References

H. Crowder, E. L. Johnson, and M. W. Padberg. Solving large scale zero-one linear programming problems. Operations Research, 31:803–834, 1983. D. De Wolf and Y. Smeers. The gas transmission problem solved by an extension of the simplex algorithm. Management Science, 46:1454–1465, 2000. M. Duran and I. E. Grossmann. An outer-approximation algorithm for a class of mixed–integer nonlinear programs. Mathematical Programming, 36:307–339, 1986. R. Fletcher and S. Leyffer. Solving mixed integer nonlinear programs by outer approximation. Mathematical Programming, 66:327–349, 1994. R. Fletcher and S. Leyffer. Numerical experience with lower bounds for MIQP branch–and–bound. SIAM Journal on Optimization, 8(2):604–616, 1998. A. M. Geoffrion. Generalized Benders decomposition. Journal of Optimization Theory and Applications, 10:237–260, 1972. I. E. Grossmann and R. W. H. Sargent. Optimal design of multipurpose batch plants. Ind. Engng. Chem. Process Des. Dev., 18:343–348, 1979. Harjunkoski, I., Westerlund, T., Pörn,R. and Skrifvars, H. Different transformations for solving non-convex trim-loss problems by MINLP. European Journal of Opertational Research, 105:594–603, 1998. Jain, V. and Grossmann, I.E. Cyclic scheduling of continuous parallel-process units with decaying performance. AIChE Journal, 44:1623–1636, 1998. G. R. Kocis and I. E. Grossmann. Global optimization of nonconvex mixed–integer nonlinear programming (MINLP) problems in process synthesis. Industrial Engineering Chemistry Research, 27:1407–1421, 1988. Leyffer & Linderoth MINLP References

G. R. Kocis and I. E. Grossmann. Computational experience with DICOPT solving MINLP problems in process systems engineering. Computers and Chemical Engineering, 13(3):307–315, 1989. S. Lee and I. Grossmann. New algorithms for nonlinear disjunctive programming. Computers and Chemical Engineering, 24:2125–2141, 2000. S. Leyffer. User manual for MINLP BB. University of Dundee, 1998. S. Leyffer. Integrating SQP and branch-and-bound for mixed integer nonlinear programming. Computational Optimization & Applications, 18:295–309, 2001. L. Lovászand A. Schrijver. Cones of matrices and setfunctions, and 0-1 optimization. SIAM Journal on Optimization, 1, 1991. H. Marchand and L. Wolsey. The 0-1 knapsack problem with a single continuous variable. Mathematical Programming, 85:15–33, 1999. A. Martin, M. Möller, and S. Moritz. Mixed integer models for the stationary case of gas network optimization. Technical report, Darmstadt University of Technology, 2005. G. P. McCormick. Computability of global solutions to factorable nonconvex programs: Part I—Convex underestimating problems. Mathematical Programming, 10:147–175, 1976. Nemhauser, G.L. and Wolsey, L.A. Integer and Combinatorial Optimization. John Wiley, New York, 1988. M. Padberg, T. J. Van Roy, and L. Wolsey. Valid linear inequalities for fixed charge problems. Operations Research, 33:842–861, 1985. Leyffer & Linderoth MINLP References

I. Quesada and I. E. Grossmann. An LP/NLP based branch–and–bound algorithm for convex MINLP optimization problems. Computers and Chemical Engineering, 16: 937–947, 1992. Quist, A.J. Application of Mathematical Optimization Techniques to Nuclear Reactor Reload Pattern Design. PhD thesis, Technische Universiteit Delft, Thomas Stieltjes Institute for Mathematics, The Netherlands, 2000. R. Raman and I. E. Grossmann. Modeling and computational techniques for logic based integer programming. Computers and Chemical Engineering, 18:563–578, 1994. N. V. Sahinidis. BARON: Branch and reduce optimization navigator. Technical report, University of Illinois at Urbana-Champaign, Department of Chemical Engineering, 2000. C. A. Schweiger and C. A. Floudas. MINOPT: A modeling language and algorithmic framework for linear, mixed-integer, nonlinear, dynamic, and mixed-integer nonlinear optimization. Technical report, Department of Chemical Engineering, Princeton University, 1998. H. D. Sherali and W. P. Adams. A hierarchy of relaxations between the continuous and convex hull representations for zero-one programming problems. SIAM Journal on Discrete Mathematics, 3:411–430, 1990. O. Sigmund. A 99 line topology optimization code written in matlab. Structural Multidisciplinary Optimization, 21:120–127, 2001. R. Stubbs and S. Mehrohtra. Generating convex polynomial inequalities for mixed 0-1 programs. Journal of Global Optimization, 24:311–332, 2002. Leyﬀer & Linderoth MINLP References

R. A. Stubbs and S. Mehrotra. A branch–and–cut method for 0–1 mixed convex programming. Mathematical Programming, 86:515–532, 1999. M. Tawarmalani, S. Ahmed, and N. Sahinidis. Product disaggregation in global optimization and relaxations of rational programs. Optimization and Engineering, 3:281–303, 2002. M. Tawarmalani and N. V. Sahinidis. Convexiﬁcation and Global Optimization in Continuous and Mixed-Integer Nonlinear Programming: Theory, Algorithms, Software, and Applications. Kluwer Academic Publishers, Boston MA, 2002. J. Viswanathan and I. E. Grossmann. Optimal feed location and number of trays for distillation columns with multiple feeds. I&EC Research, 32:2942–2949, 1993. Westerlund, T. and Pettersson, F. An extended cutting plane method for solving convex MINLP problems. Computers and Chemical Engineering Supplement, 19: S131–S136, 1995. ESCAPE–95. Westerlund, T., Isaksson, J. and Harjunkoski, I. Solving a production optimization problem in the paper industry. Report 95–146–A, Department of Chemical Engineering, Abo Akademi, Abo, Finland, 1995. Westerlund, T., Pettersson, F. and Grossmann, I.E. Optimization of pump conﬁgurations as MINLP problem. Computers & Chemical Engineering, 18(9): 845–858, 1994. H. P. Williams. Model Solving in Mathematical Programming. John Wiley & Sons Ltd., Chichester, 1993.

Leyﬀer & Linderoth MINLP