Artelys Knitro 11.0
Total Page:16
File Type:pdf, Size:1020Kb
Artelys Knitro 11.0 SOCP, API, Preconditioner and much more! June 29, 2018 www.artelys.com COMPANY OVERVIEW Artelys We specialize in optimization, decision-support, modeling and deliver efficient solutions to complex business problems. Domains of expertise Services Energy Auditing & Consulting Transport & Logistic On demand software Defense Distribution and support of Numerical and Combinatorial numerical optimization tools Optimization Training June 29, 2018 www.artelys.com 2 NUMERICAL OPTIMIZATION TOOLS KNITRO Industry leading solver for very large, difficult nonlinear optimization problems (NLP, MINLP) FICO Xpress Optimization Suite High performance linear, quadratic and mixed integer programming solver (LP,MIP,QP) Artelys Kalis Object-oriented environment to model and solve problems with constraints programming techniques AMPL Comprehensive modeling language for Mathematical Programming June 29, 2018 www.artelys.com 3 ARTELYS KNITRO: OVERVIEW Background Created in 2001 by Ziena Optimization Spin-off of Northwestern University Now developed and supported by Artelys Key features Efficient and robust solution on large scale problems (~105 variables) Four active-set and interior-point algorithms for continuous optimization MINLP algorithms and complementarity constraints for discrete optimization Many extra features based on customer feedbacks or project requirements Parallel multi-start method for global optimization. Easy to use and well documented: Online documentation June 29, 2018 www.artelys.com 4 ARTELYS KNITRO: OVERVIEW Widely used in academia… US Top Universities: Berkeley, Columbia, Harvard, MIT, Stanford… Worldwide Top Universities: ETH Zürich, LSE, NUS (Singapore), Melbourne… … and industry Economic consulting firms Financial institutions Mechanical engineering companies Oil & Gas companies Regulator & Policy maker Software developers Used as a third-party optimization engine More than 400 institutions in over 40 countries rely on Artelys Knitro June 29, 2018 www.artelys.com 5 ARTELYS KNITRO: OVERVIEW Interfaces Programming languages Modeling languages Supported platforms Windows 32-bit, 64-bit Linux 64-bit macOS 64-bit June 29, 2018 www.artelys.com 6 ARTELYS KNITRO: OVERVIEW Interfaces Programming languages Modeling languages Julia Supported platforms Windows 32-bit, 64-bit Linux 64-bit macOS 64-bit June 29, 2018 www.artelys.com 6 PROBLEM CLASSES SOLVED BY KNITRO min 푓 푥, 푦 푥∈ℝ푛,푦∈ℤ푚 s.t. 푔푖 푥, 푦 ≥ 0 푖 ∈ 퐼 ℎ푗 푥, 푦 = 0 푗 ∈ 퐽 2 0 ≤ 푥푘 ⊥ 푥푙 ≥ 0 푘, 푙 ∈ 퐶 ⊂ 1, 푛 푙푥 ≤ 푥 ≤ 푢푥 푙푦 ≤ 푦 ≤ 푢푦 • Variables (x and y): – Continuous or discrete – Bounded or unbounded • Objective (f) and constraints (g and h): – Linear or nonlinear – Smoothness: required, but may still work without it – Convexity: better but not required, local optimization or global optimization with multistart 풙풍 • Complementarity constraints: – 0 ≤ 푥푘 ⊥ 푥푙 ≥ 0 is equivalent to: 푥푘 ≥ 0 and 푥푙 ≥ 0 and 푥푘 = 0 or 푥푙 = 0 – Applications: strategic bidding, economic models, equilibrium constraints, disjunctions 풙풌 RELEASE 11.0 Artelys Knitro 11.0 new features: New SOCP Algorithm Detect conic constraints from quadratic structures Designed for general nonlinear problems with SOC constraints New C API Easier to use Allows passing problem structure (eg linear, quadratic, conic constraints) with dedicated API and without providing Hessian Preconditioning for all classes of problems Preconditioning can now be used for problems with equality and inequality constraints New parallel linear solvers HSL MA86 (non-deterministic) and MA97 (deterministic) Speedups on large scale problem with shared memory parallelism Performance improvements Very large speedup on QCQPs (including nonlinear QCQP) Speedups on general convex problems Speedups on MINLP algorithms June 29, 2018 www.artelys.com 9 RELEASE 11.0 Artelys Knitro 11.0 new features: New SOCP Algorithm 1 Detect conic constraints from quadratic structures Designed for general nonlinear problems with SOC constraints New C API Easier to use Allows passing problem structure (eg linear, quadratic, conic constraints) with dedicated API and without providing Hessian Preconditioning for all classes of problems Preconditioning can now be used for problems with equality and inequality constraints New parallel linear solvers HSL MA86 (non-deterministic) and MA97 (deterministic) Speedups on large scale problem with shared memory parallelism Performance improvements Very large speedup on QCQPs (including nonlinear QCQP) Speedups on general convex problems Speedups on MINLP algorithms June 29, 2018 www.artelys.com 9 RELEASE 11.0 Artelys Knitro 11.0 new features: New SOCP Algorithm Detect conic constraints from quadratic structures Designed for general nonlinear problems with SOC constraints New C API 2 Easier to use Allows passing problem structure (eg linear, quadratic, conic constraints) with dedicated API and without providing Hessian Preconditioning for all classes of problems Preconditioning can now be used for problems with equality and inequality constraints New parallel linear solvers HSL MA86 (non-deterministic) and MA97 (deterministic) Speedups on large scale problem with shared memory parallelism Performance improvements Very large speedup on QCQPs (including nonlinear QCQP) Speedups on general convex problems Speedups on MINLP algorithms June 29, 2018 www.artelys.com 9 RELEASE 11.0 Artelys Knitro 11.0 new features: New SOCP Algorithm Detect conic constraints from quadratic structures Designed for general nonlinear problems with SOC constraints New C API Easier to use Allows passing problem structure (eg linear, quadratic, conic constraints) with dedicated API and without providing Hessian Preconditioning for all classes of problems 3 Preconditioning can now be used for problems with equality and inequality constraints New parallel linear solvers HSL MA86 (non-deterministic) and MA97 (deterministic) Speedups on large scale problem with shared memory parallelism Performance improvements Very large speedup on QCQPs (including nonlinear QCQP) Speedups on general convex problems Speedups on MINLP algorithms June 29, 2018 www.artelys.com 9 SOCP June 29, 2018 www.artelys.com 13 SECOND ORDER CONE CONSTRAINTS Standard Second Order Cone (SOC) of dimension k is 푡 푢 ∈ 푅푘−1, 푡 ∈ 푅1, 푢 ≤ 푡 푢 For k=1 푡 푡 ∈ 푅1, 0 ≤ 푡 For k=2 t 푡 푢 ∈ 푅1, 푡 ∈ 푅1, 푢 ≤ 푡 푢 u June 27, 2018 14 SECOND ORDER CONE PROGRAMS (SOCP) Set 푢 = 퐴푥 + 푏 and 푡 = 푐T푥 + 푑 to create general second order cone constraints of form 퐴푥 + 푏 ≤ 푐푇푥 + 푑 Second Order Cone Program (SOCP): min 푓T푥 푥 T s.t. 퐴푖푥 + 푏푖 ≤ 푐푖 푥 + 푑푖, i=1..m 퐺푇푥 + ℎ ≤ 0 Convex QP and QCQP (and more) can be converted to SOCP June 27, 2018 15 APPLICATIONS Applications Finance: portfolio optimization with loss risk constraints Facility location (e.g. antenna placement in wireless network) Robust optimization (under ellipsoid uncertainty) Robust least squares Grasping force optimization FIR filter design Truss design See Applications of Second-Order Cone Programming, Lobo,Vandenberghe, Boyd, Lebret Apr 16, 2018 16 EXAMPLES Quadratic constraint 푥푇 푄푥 + 2푞푇푥 + 푟 ≤ 0 2 푄1/2푥 + 푄−1/2푞 + 푟 − 푞푇푄−1푞 ≤ 0 ||푄1/2푥 + 푄−1/2푞|| ≤ 푞푇푄−1푞 − 푟 1/2 Rotated cone constraint 푥푇푥 ≤ 푦푧, 푦 ≥ 0, 푧 ≥ 0 4푥푇푥 + 푦2 + 푧2 ≤ 4푦푧 + 푦2 + 푧2 4푥푇푥 + (푦 − 푧)2 ≤ 푦 + 푧 2푥 ≤ 푦 + 푧 푦 − 푧 Apr 16, 2018 17 KNITRO CONIC SOLVER Knitro identifies the constraints in form 푛 2 2 σ푖=1 푎푖푥푖 ≤ 푎0푥0 , 푥0 ≥ 0 and 푛 2 σ푖=2 푎푖푥푖 ≤ 푎0푥0푥1, 푥0, 푥1 ≥ 0. as second order cone constraints, and internally puts them into the standard form It also allows to input constraints in form 퐴푥 + 푏 ≤ 푐푥 + 푑 directly via the struct ‘L2norm’ Currently, it does second order conic constraint identification on the presolved problem June 27, 2018 www.artelys.com 18 KNITRO CONIC SOLVER Knitro conic solver moves beyond SOCP (more general) min 푓T푥 푥 T s.t. 퐴푖푥 + 푏푖 ≤ 푐푖 푥 + 푑푖 퐺푇푥 + ℎ ≤ 0 min 푓(푥) 푥 T s.t. 퐴푖푥 + 푏푖 ≤ 푐푖 푥 + 푑푖 ℎ 푥 = 0 푔 푥 ≤ 0 • Handle any nonlinear problem with second order cone constraints, including non-convex • First specialized solver of this kind ! • Extension of existing Knitro Interior/Direct algorithm June 27, 2018 19 KNITRO CONIC SOLVER It generalizes the operations on the slack variables in the existing Knitro/Direct algorithm using the algebra associated with second order cones June 27, 2018 www.artelys.com 20 NLP FORM VS. CONIC FORM Can always write the cone constraint as a general NLP constraint: 2 2 2 푢 ≤ 푡 → 푢1 + 푢2+. +푢푘−1 ≤ 푡 This does not work well in general; constraint is non-differentiable as 푢 → 0 It is not uncommon that 푢 → 0 at the optimal solution Can square the constraint, but then it is non-convex and degenerate at the solution if 푢 → 0 Can try to smooth or relax/perturb these constraints to avoid these issues, e.g. 2 2 2 2 푢1 + 푢2+. +푢푘−1 + 휖 ≤ 푡 This works better sometimes but is still not robust or nearly as effective as dealing with them directly June 27, 2018 21 NLP FORM VS. CONIC FORM Consider the simple example: min 0.5푥1 + 푥2 푥 s.t. 푥1 ≤ 푥2 The optimal solution is at (0,0) NLP form without conic detection can’t get dual feasible QCQP form (non-convex): min 0.5푥1 + 푥2 푥 2 2 s.t. 푥1 ≤ 푥2 , 푥2 ≥ 0 solves in 50 iterations Conic formulation solves in 4 iterations In fact, development of the conic extension started after having troubles with such a problem Steiner_model / Steiner_model_100 on the next slide June 27, 2018 22 NLP FORM VS. CONIC FORM Compare Knitro (with and without special treatment of cone constraints) and Xpress on small SOCP models (iteration comparison) June 27, 2018 KNITRO CONIC SOLVER-MISOCP Can utilize Knitro branch-and-bound algorithm to solve mixed- integer SOCP … or more general mixed-integer models with SOC constraints min 푓(푥, 푦) 푥,푦 T s.t. 퐴푖푥 + 푏푖 ≤ 푐푖 푥 + 푑푖, i=1..k ℎ 푥, 푦 = 0 푔 푥, 푦 ≤ 0 푦 integer June 27, 2018 24 NEW KNITRO 11.0 API June 27, 2018 www.artelys.com 25 NEW KNITRO 11.0 API – KEY FEATURES Build optimization model piece-by-piece More flexible Easier problem modification More extendable (to multi-objective, statistical learning models, etc.) Identify special structures (e.g.