Artelys Knitro 11.0

Home , AMPL, Artelys Knitro, FICO Xpress

Artelys Knitro 11.0

SOCP, API, Preconditioner and much more!

June 29, 2018 www.artelys.com COMPANY OVERVIEW

Artelys We specialize in optimization, decision-support, modeling and deliver efficient solutions to complex business problems.

Domains of expertise Services Energy Auditing & Consulting Transport & Logistic On demand software Defense Distribution and support of Numerical and Combinatorial numerical optimization tools Optimization Training

June 29, 2018 www.artelys.com 2 NUMERICAL OPTIMIZATION TOOLS

KNITRO Industry leading solver for very large, difficult nonlinear optimization problems (NLP, MINLP)

FICO Xpress Optimization Suite High performance linear, quadratic and mixed integer programming solver (LP,MIP,QP)

Artelys Kalis Object-oriented environment to model and solve problems with constraints programming techniques

AMPL Comprehensive modeling language for Mathematical Programming

June 29, 2018 www.artelys.com 3 ARTELYS KNITRO: OVERVIEW

Background Created in 2001 by Ziena Optimization Spin-off of Northwestern University Now developed and supported by Artelys

Key features Efficient and robust solution on large scale problems (~105 variables) Four active-set and interior-point algorithms for continuous optimization MINLP algorithms and complementarity constraints for discrete optimization Many extra features based on customer feedbacks or project requirements Parallel multi-start method for global optimization. Easy to use and well documented: Online documentation

June 29, 2018 www.artelys.com 4 ARTELYS KNITRO: OVERVIEW

Widely used in academia… US Top Universities: Berkeley, Columbia, Harvard, MIT, Stanford… Worldwide Top Universities: ETH Zürich, LSE, NUS (Singapore), Melbourne…

… and industry Economic consulting firms Financial institutions Mechanical engineering companies Oil & Gas companies Regulator & Policy maker Software developers Used as a third-party optimization engine

More than 400 institutions in over 40 countries rely on Artelys Knitro

June 29, 2018 www.artelys.com 5 ARTELYS KNITRO: OVERVIEW

Interfaces Programming languages Modeling languages

Supported platforms

Windows 32-bit, 64-bit Linux 64-bit macOS 64-bit

June 29, 2018 www.artelys.com 6 ARTELYS KNITRO: OVERVIEW

Interfaces Programming languages Modeling languages

Julia Supported platforms

Windows 32-bit, 64-bit Linux 64-bit macOS 64-bit

June 29, 2018 www.artelys.com 6 PROBLEM CLASSES SOLVED BY KNITRO

min 푓 푥, 푦 푥∈ℝ푛,푦∈ℤ푚 s.t. 푔푖 푥, 푦 ≥ 0 푖 ∈ 퐼 ℎ푗 푥, 푦 = 0 푗 ∈ 퐽 2 0 ≤ 푥푘 ⊥ 푥푙 ≥ 0 푘, 푙 ∈ 퐶 ⊂ 1, 푛 푙푥 ≤ 푥 ≤ 푢푥 푙푦 ≤ 푦 ≤ 푢푦

• Variables (x and y): – Continuous or discrete – Bounded or unbounded • Objective (f) and constraints (g and h): – Linear or nonlinear – Smoothness: required, but may still work without it – Convexity: better but not required, local optimization or global optimization with multistart 풙풍 • Complementarity constraints: – 0 ≤ 푥푘 ⊥ 푥푙 ≥ 0 is equivalent to: 푥푘 ≥ 0 and 푥푙 ≥ 0 and 푥푘 = 0 or 푥푙 = 0 – Applications: strategic bidding, economic models, equilibrium constraints, disjunctions 풙풌 RELEASE 11.0

Artelys Knitro 11.0 new features: New SOCP Algorithm Detect conic constraints from quadratic structures Designed for general nonlinear problems with SOC constraints New C API Easier to use Allows passing problem structure (eg linear, quadratic, conic constraints) with dedicated API and without providing Hessian Preconditioning for all classes of problems Preconditioning can now be used for problems with equality and inequality constraints New parallel linear solvers HSL MA86 (non-deterministic) and MA97 (deterministic) Speedups on large scale problem with shared memory parallelism Performance improvements Very large speedup on QCQPs (including nonlinear QCQP) Speedups on general convex problems Speedups on MINLP algorithms June 29, 2018 www.artelys.com 9 RELEASE 11.0

Artelys Knitro 11.0 new features: New SOCP Algorithm 1 Detect conic constraints from quadratic structures Designed for general nonlinear problems with SOC constraints New C API Easier to use Allows passing problem structure (eg linear, quadratic, conic constraints) with dedicated API and without providing Hessian Preconditioning for all classes of problems Preconditioning can now be used for problems with equality and inequality constraints New parallel linear solvers HSL MA86 (non-deterministic) and MA97 (deterministic) Speedups on large scale problem with shared memory parallelism Performance improvements Very large speedup on QCQPs (including nonlinear QCQP) Speedups on general convex problems Speedups on MINLP algorithms June 29, 2018 www.artelys.com 9 RELEASE 11.0

Artelys Knitro 11.0 new features: New SOCP Algorithm Detect conic constraints from quadratic structures Designed for general nonlinear problems with SOC constraints New C API 2 Easier to use Allows passing problem structure (eg linear, quadratic, conic constraints) with dedicated API and without providing Hessian Preconditioning for all classes of problems Preconditioning can now be used for problems with equality and inequality constraints New parallel linear solvers HSL MA86 (non-deterministic) and MA97 (deterministic) Speedups on large scale problem with shared memory parallelism Performance improvements Very large speedup on QCQPs (including nonlinear QCQP) Speedups on general convex problems Speedups on MINLP algorithms June 29, 2018 www.artelys.com 9 RELEASE 11.0

Artelys Knitro 11.0 new features: New SOCP Algorithm Detect conic constraints from quadratic structures Designed for general nonlinear problems with SOC constraints New C API Easier to use Allows passing problem structure (eg linear, quadratic, conic constraints) with dedicated API and without providing Hessian Preconditioning for all classes of problems 3 Preconditioning can now be used for problems with equality and inequality constraints New parallel linear solvers HSL MA86 (non-deterministic) and MA97 (deterministic) Speedups on large scale problem with shared memory parallelism Performance improvements Very large speedup on QCQPs (including nonlinear QCQP) Speedups on general convex problems Speedups on MINLP algorithms June 29, 2018 www.artelys.com 9 SOCP

June 29, 2018 www.artelys.com 13 SECOND ORDER CONE CONSTRAINTS

Standard Second Order Cone (SOC) of dimension k is

푡 푢 ∈ 푅푘−1, 푡 ∈ 푅1, 푢 ≤ 푡 푢

For k=1 푡 푡 ∈ 푅1, 0 ≤ 푡

For k=2 t 푡 푢 ∈ 푅1, 푡 ∈ 푅1, 푢 ≤ 푡 푢

June 27, 2018 14 SECOND ORDER CONE PROGRAMS (SOCP)

Set 푢 = 퐴푥 + 푏 and 푡 = 푐T푥 + 푑 to create general second order cone constraints of form

퐴푥 + 푏 ≤ 푐푇푥 + 푑

Second Order Cone Program (SOCP):

min 푓T푥 푥 T s.t. 퐴푖푥 + 푏푖 ≤ 푐푖 푥 + 푑푖, i=1..m 퐺푇푥 + ℎ ≤ 0

Convex QP and QCQP (and more) can be converted to SOCP

June 27, 2018 15 APPLICATIONS

Applications Finance: portfolio optimization with loss risk constraints Facility location (e.g. antenna placement in wireless network) Robust optimization (under ellipsoid uncertainty) Robust least squares Grasping force optimization FIR filter design Truss design

See Applications of Second-Order Cone Programming, Lobo,Vandenberghe, Boyd, Lebret

Apr 16, 2018 16 EXAMPLES

Quadratic constraint 푥푇 푄푥 + 2푞푇푥 + 푟 ≤ 0 2 푄1/2푥 + 푄−1/2푞 + 푟 − 푞푇푄−1푞 ≤ 0 ||푄1/2푥 + 푄−1/2푞|| ≤ 푞푇푄−1푞 − 푟 1/2

Rotated cone constraint 푥푇푥 ≤ 푦푧, 푦 ≥ 0, 푧 ≥ 0 4푥푇푥 + 푦2 + 푧2 ≤ 4푦푧 + 푦2 + 푧2 4푥푇푥 + (푦 − 푧)2 ≤ 푦 + 푧 2푥 ≤ 푦 + 푧 푦 − 푧

Apr 16, 2018 17 KNITRO CONIC SOLVER

Knitro identifies the constraints in form 푛 2 2 σ푖=1 푎푖푥푖 ≤ 푎0푥0 , 푥0 ≥ 0 and 푛 2 σ푖=2 푎푖푥푖 ≤ 푎0푥0푥1, 푥0, 푥1 ≥ 0. as second order cone constraints, and internally puts them into the standard form

It also allows to input constraints in form 퐴푥 + 푏 ≤ 푐푥 + 푑 directly via the struct ‘L2norm’

Currently, it does second order conic constraint identification on the presolved problem

June 27, 2018 www.artelys.com 18 KNITRO CONIC SOLVER

Knitro conic solver moves beyond SOCP (more general)

min 푓T푥 푥 T s.t. 퐴푖푥 + 푏푖 ≤ 푐푖 푥 + 푑푖 퐺푇푥 + ℎ ≤ 0

min 푓(푥) 푥 T s.t. 퐴푖푥 + 푏푖 ≤ 푐푖 푥 + 푑푖 ℎ 푥 = 0 푔 푥 ≤ 0 • Handle any nonlinear problem with second order cone constraints, including non-convex • First specialized solver of this kind ! • Extension of existing Knitro Interior/Direct algorithm June 27, 2018 19 KNITRO CONIC SOLVER

It generalizes the operations on the slack variables in the existing Knitro/Direct algorithm using the algebra associated with second order cones

June 27, 2018 www.artelys.com 20 NLP FORM VS. CONIC FORM

Can always write the cone constraint as a general NLP constraint:

2 2 2 푢 ≤ 푡 → 푢1 + 푢2+. . +푢푘−1 ≤ 푡

This does not work well in general; constraint is non-differentiable as 푢 → 0 It is not uncommon that 푢 → 0 at the optimal solution

Can square the constraint, but then it is non-convex and degenerate at the solution if 푢 → 0 Can try to smooth or relax/perturb these constraints to avoid these issues, e.g.

2 2 2 2 푢1 + 푢2+. . +푢푘−1 + 휖 ≤ 푡 This works better sometimes but is still not robust or nearly as effective as dealing with them directly

June 27, 2018 21 NLP FORM VS. CONIC FORM

Consider the simple example:

min 0.5푥1 + 푥2 푥 s.t. 푥1 ≤ 푥2 The optimal solution is at (0,0) NLP form without conic detection can’t get dual feasible QCQP form (non-convex): min 0.5푥1 + 푥2 푥 2 2 s.t. 푥1 ≤ 푥2 , 푥2 ≥ 0 solves in 50 iterations Conic formulation solves in 4 iterations

In fact, development of the conic extension started after having troubles with such a problem Steiner_model / Steiner_model_100 on the next slide June 27, 2018 22 NLP FORM VS. CONIC FORM

Compare Knitro (with and without special treatment of cone constraints) and Xpress on small SOCP models (iteration comparison)

June 27, 2018 KNITRO CONIC SOLVER-MISOCP

Can utilize Knitro branch-and-bound algorithm to solve mixed- integer SOCP

… or more general mixed-integer models with SOC constraints

min 푓(푥, 푦) 푥,푦 T s.t. 퐴푖푥 + 푏푖 ≤ 푐푖 푥 + 푑푖, i=1..k ℎ 푥, 푦 = 0 푔 푥, 푦 ≤ 0 푦 integer

June 27, 2018 24 NEW KNITRO 11.0 API

June 27, 2018 www.artelys.com 25 NEW KNITRO 11.0 API – KEY FEATURES

Build optimization model piece-by-piece More flexible Easier problem modification More extendable (to multi-objective, statistical learning models, etc.)

Identify special structures (e.g. linear, quadratic, conic, etc) Identify more problem types (QCQP, SOCP, etc) Potential for more extensive presolve operations Faster (potentially parallel) evaluations of stored structures

Can combine exact and approximate derivatives

June 27, 2018 26 C EXAMPLE

ퟐ min 풇 풙 + ퟏ − 풙ퟎ