Dr. Ampl a Meta Solver for Optimization

Dr. Ampl A Meta Solver for Optimization

Dominique Orban Bob Fourer École Polytechnique de Montréal Northwestern University

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 1/36 ■ Current software challenges ■ The NEOS Server ■ The AMPL modeling language ■ The DrAmpl meta solver ■ Numerical results ■ Future directions of research

Outline

● Outline ■ Numerical Optimization ● Numerical Optimization

Software

Model analysis

The DAG

Bounds

Convexity

Classiﬁcation

Preprocessing

Solver choice

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 2/36 ■ The NEOS Server ■ The AMPL modeling language ■ The DrAmpl meta solver ■ Numerical results ■ Future directions of research

Outline

● Outline ■ Numerical Optimization ● Numerical Optimization ■ Software Current software challenges

Model analysis

The DAG

Bounds

Convexity

Classiﬁcation

Preprocessing

Solver choice

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 2/36 ■ The AMPL modeling language ■ The DrAmpl meta solver ■ Numerical results ■ Future directions of research

Outline

● Outline ■ Numerical Optimization ● Numerical Optimization ■ Software Current software challenges

Model analysis ■ The NEOS Server

The DAG

Bounds

Convexity

Classiﬁcation

Preprocessing

Solver choice

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 2/36 ■ The DrAmpl meta solver ■ Numerical results ■ Future directions of research

Outline

● Outline ■ Numerical Optimization ● Numerical Optimization ■ Software Current software challenges

Model analysis ■ The NEOS Server The DAG ■ The AMPL modeling language Bounds

Convexity

Classiﬁcation

Preprocessing

Solver choice

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 2/36 ■ Numerical results ■ Future directions of research

Outline

● Outline ■ Numerical Optimization ● Numerical Optimization ■ Software Current software challenges

Model analysis ■ The NEOS Server The DAG ■ The AMPL modeling language Bounds ■ The DrAmpl meta solver Convexity

Classiﬁcation

Preprocessing

Solver choice

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 2/36 ■ Future directions of research

Outline

● Outline ■ Numerical Optimization ● Numerical Optimization ■ Software Current software challenges

Model analysis ■ The NEOS Server The DAG ■ The AMPL modeling language Bounds ■ The DrAmpl meta solver Convexity ■ Numerical results Classiﬁcation

Preprocessing

Solver choice

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 2/36 Outline

● Outline ■ Numerical Optimization ● Numerical Optimization ■ Software Current software challenges

Model analysis ■ The NEOS Server The DAG ■ The AMPL modeling language Bounds ■ The DrAmpl meta solver Convexity ■ Numerical results Classiﬁcation ■ Preprocessing Future directions of research

Solver choice

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 2/36 Numerical Optimization

● Outline A general problem, representable in a modeling language may ● Numerical Optimization be written as Software

Model analysis

The DAG minimize f(x) objective Bounds L U Convexity subject to c c(x) c general constraints L ≤ ≤U Classiﬁcation x x x explicit bounds ≤ ≤ Preprocessing

Solver choice n n m 2 Final notes where f : R R, c : R R . → → ∈ C

Bounds may be equal or inﬁnite, some ci(x) may be linear.

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 3/36 Current software challenges

● Outline ■ Myriad of solvers available for most classes of optimization ● Numerical Optimization problems Software ● Current software challenges ■ ● Consequences Competing solvers designed to tackle the same class of ● More software challenges ● The NEOS Optimization Server problems, with varying subtelties ● The AMPL modeling language ■ ● Some typical AMPL commands Competing general-purpose solvers ● DrAmpl is a meta solver ■ Model analysis Competing free and commercial codes ■ The DAG Competing modeling languages and environments Bounds ■ Often, solvers have limited compatibility with modeling Convexity environments or interfaces do not exist

Classiﬁcation

Preprocessing

Solver choice

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 4/36 Consequences

● Outline ■ Much confusion arising from so much choice ● Numerical Optimization ■ Software Much room for remote or local software offering guidance ● Current software challenges ● Consequences and bridging the abyssal gap between problem classes and ● More software challenges ● The NEOS Optimization Server solvers ● The AMPL modeling language ■ ● Some typical AMPL commands Much need for tools to help bridge this gap, by compiling lists ● DrAmpl is a meta solver of available software, classifying the possible problem Model analysis instances, linking them together The DAG ■ Bounds Much need for software able to associate a given problem instance with a general class and a general class with a few Convexity solvers. Classiﬁcation

Preprocessing

Solver choice

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 5/36 More software challenges

● Outline ■ Modeling languages are often commercial ● Numerical Optimization ■ Software Optimization software is often commercial ● Current software challenges ● Consequences ■ Software may be problematic to install locally ● More software challenges ● The NEOS Optimization Server ■ ● The AMPL modeling language Need to learn how to use the software ● Some typical AMPL commands ■ ● DrAmpl is a meta solver Trial versions are often available with various limitations

Model analysis

The DAG

Bounds

Convexity

Classiﬁcation

Preprocessing

Solver choice

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 6/36 The NEOS Optimization Server

● Outline ■ Remote optimization software repository ● Numerical Optimization ■ Software Problems modelled locally, solved remotely ● Current software challenges ● Consequences ■ Solution may be returned for later local examination ● More software challenges ● The NEOS Optimization Server ■ ● The AMPL modeling language Accepts problems in several modeling languages; AMPL, ● Some typical AMPL commands ● DrAmpl is a meta solver GAMS, SIF, SeDuMi, Sparse SDPA, C and Fortran ■ Model analysis 43 solvers for 12 classes of optimization problems The DAG ■ Optimization tree... last updated in March 1996. Bounds ■ 2723 submissions from May 2–May 9, up to 5430/week Convexity ■ Alleviates the need to install some solvers locally Classiﬁcation ■ Preprocessing Allows to test solvers before purchasing or downloading ■ Solver choice Free of charge Final notes ■ E-mail, Web, TCP/IP submission clients

How to best make use of all those resources?

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 7/36 The AMPL modeling language

● Outline ■ Fourer, Gay, Kernighan book ● Numerical Optimization ■ Software Flexible and powerful language to model linear, nonlinear, ● Current software challenges ● Consequences mixed-integer and constraint programming ● More software challenges ● The NEOS Optimization Server ■ Provides some level of preprocessing ● The AMPL modeling language ● Some typical AMPL commands ■ ● DrAmpl is a meta solver Provides automatic differentiation ■ Model analysis Size-limited student version available free of charge The DAG ■ Source code of AMPL library available on NetLib to facilitate Bounds interfacing with solvers and exploring the AMPL data Convexity structures

Classiﬁcation

Preprocessing

Solver choice

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 8/36 Some typical AMPL commands

● Outline Select a model, a solver and solve. ● Numerical Optimization

Software ● Current software challenges ● Consequences ampl: model mymodel.mod; ● More software challenges ● The NEOS Optimization Server ampl: data mymodel.dat; ● The AMPL modeling language ● Some typical AMPL commands ampl: option solver lancelot; ● DrAmpl is a meta solver ampl: solve; Model analysis

The DAG Bounds What is a solver to AMPL? Any program which can receive a Convexity model, possibly manipulate it and return some result. Classiﬁcation

Preprocessing

Solver choice Make a distinction between a solver and a meta solver.

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 9/36 DrAmpl is a meta solver

● Outline The purposes of DrAmpl are to ● Numerical Optimization

Software ● Current software challenges ● Consequences ■ ● More software challenges Analyze a given model ● The NEOS Optimization Server ■ ● The AMPL modeling language Provide bounds on expressions appearing in the model ● Some typical AMPL commands ● DrAmpl is a meta solver ■ Assess convexity of expressions appearing in the model Model analysis ■ Classify the model The DAG ■ Provide some level of "nonlinear preprocessing" Bounds ■ Convexity Assist in the choice of an appropriate solver

Classiﬁcation

Preprocessing

Solver choice

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 10/36 Analysis of a model

● Outline Problem statistics ● Numerical Optimization ------Software # Objectives: [1 ] Model analysis ● Analysis of a model # Nonlinear objectives: 0 ● Analysis of a model ● Analysis of a model # Linear objectives: 1 ● Analysis of a model The DAG Sparsity statistics Bounds ------Convexity # nnz in Jacobian: 4000 Classiﬁcation # nnz in obj. gradients: 800 Preprocessing

Solver choice

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 11/36 Analysis of a model

● Outline # Constraints: [1603 ] ● Numerical Optimization # Range: 801 Software # Equality: 0 Model analysis ● Analysis of a model # Inequality: 802 ● Analysis of a model ● Analysis of a model # Spurious: 0 ● Analysis of a model ------The DAG # Nonlinear general: 801 Bounds # Quadratic: 0 Convexity # Linear general: 802 Classiﬁcation # Nonlinear network: 0 Preprocessing # Linear network: 0 Solver choice # Complementarity cond: [0 ] Final notes # Linear: 0 # Nonlinear: 0 Provides initial point: Yes Provides initial dual point: Yes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 12/36 Analysis of a model

● Outline Variables statistics ● Numerical Optimization ------Software # Variables: [800 ] Model analysis ● Analysis of a model # Continuous: 800 ● Analysis of a model ● Analysis of a model # Binary: 0 ● Analysis of a model # Other integer: 0 The DAG # Variables Bounds bounded above only: 0 Convexity | below only: 0 Classiﬁcation |_ above and below: 800 Preprocessing fixed: 0 Solver choice free : 0 Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 13/36 Analysis of a model

● Outline Nonlinearity statistics ● Numerical Optimization ------Software # Variables appearing nonlinearly: [800 ] Model analysis ● Analysis of a model # in objectives only: 0 ● Analysis of a model ● Analysis of a model # in constraints only: 800 ● Analysis of a model # in both: 0 The DAG # Integer vars appearing nonlinearly: [0 ] Bounds # in objectives only: 0 Convexity # in constraints only: 0 Classiﬁcation

Preprocessing

Solver choice

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 14/36 Internals: The DAG

● Outline AMPL stores linear expressions in linked lists. Nonlinear ● Numerical Optimization expressions are stored in a Directed Acyclic Graph for Software evaluation and automatic differentiation. Model analysis

The DAG ● Internals: The DAG ● Walking the DAG

Bounds

Convexity

Classiﬁcation

Preprocessing

Solver choice

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 15/36 Internals: The DAG

The DAG 2 3 * ˆ 2 x[1] ˆ 3 + 1 x[2] ● Internals: The DAG x1(1 + x2) ● Walking the DAG

Bounds o2 #* * Convexity o5 #ˆ

Classiﬁcation v0 #x[1]

Preprocessing n2 ^ ^

Solver choice o5 #ˆ

Final notes o0 # + n1 2 x[1] 3 + v1 #x[2] n3 1 x[2]

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 15/36 Internals: The DAG

The DAG ● Internals: The DAG AMPL creates a DAG per objective and constraint function. ● Walking the DAG Bounds The recursive nature of this data structure allows us to infer Convexity useful information on nonlinear expressions. Classiﬁcation L U Preprocessing The key ingredient: x x x . ≤ ≤ Solver choice

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 15/36 Walking the DAG

● Outline By recursively exploring each node of the DAG, we are able to ● Numerical Optimization infer bounds on nonlinear expressions and assess their Software convexity. A prototype function looks like Model analysis

The DAG int OperatorProcessor( *node ) { ● Internals: The DAG ● Walking the DAG

Bounds switch( node->op ) { /* Represents operator */

Convexity case leaf: /* Variable or constant */ Classiﬁcation return appropriate_value;

Preprocessing Solver choice case other; Final notes OperatorProcessor( node->left ); /* left child */ OperatorProcessor( node->right ); /* right child */ process( node->op ); } }

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 16/36 ■ 2 1 + x 4 − ≤ 2 ≤ ■ 8 (1 + x )3 64 − ≤ 2 ≤ ■ 32 x2(1 + x )3 256. − ≤ 1 2 ≤

Bounds on expressions

● Outline Use interval arithmetic and the prototype function to propagate ● Numerical Optimization bounds. Software

Convexity ^ ^

Classiﬁcation

Preprocessing

Solver choice 2 x[1] 3 +

Final notes

1 x[2] Bounds on the variables are crucial!

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 17/36 ■ 8 (1 + x )3 64 − ≤ 2 ≤ ■ 32 x2(1 + x )3 256. − ≤ 1 2 ≤

Bounds on expressions

● Outline Use interval arithmetic and the prototype function to propagate ● Numerical Optimization bounds. Software

Model analysis 2 3 In the example x1(1 + x2) , assume we have the speciﬁed The DAG bounds 1 x1 2 and 3 x2 3. Bounds ≤ ≤ − ≤ ≤ ● Bounds on expressions * ● Simple sample AMPL example ■ 2 ● Objective bounds in COPS 1 x1 4 ● Redundant constraints ≤ ≤ ■ ● Fragment of camshape.mod 2 1 + x 4 − ≤ 2 ≤ Convexity ^ ^

Classiﬁcation

Preprocessing

Solver choice 2 x[1] 3 +

Final notes

1 x[2] Bounds on the variables are crucial!

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 17/36 ■ 32 x2(1 + x )3 256. − ≤ 1 2 ≤

Bounds on expressions

● Outline Use interval arithmetic and the prototype function to propagate ● Numerical Optimization bounds. Software

Preprocessing

Solver choice 2 x[1] 3 +

Final notes

1 x[2] Bounds on the variables are crucial!

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 17/36 Bounds on expressions

● Outline Use interval arithmetic and the prototype function to propagate ● Numerical Optimization bounds. Software

Solver choice 2 x[1] 3 +

Final notes

1 x[2] Bounds on the variables are crucial!

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 17/36 Simple sample AMPL example

● Outline model; ● Numerical Optimization var x >= -6, <= 8; Software minimize objective: Model analysis exp(sin(1/x)); The DAG

Bounds ● Bounds on expressions DrAmpl deduces the bounds ● Simple sample AMPL example ● Objective bounds in COPS ● Redundant constraints ● Fragment of camshape.mod sin(1/x) Convexity 0.36789 e 2.718282 ≤ ≤ Classiﬁcation

Preprocessing

Solver choice

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 18/36 Objective bounds in COPS

∗ ● Outline f = lowest value given in the COPS report ● Numerical Optimization f = inferred lower bound Software inf See www.mcs.anl.gov/˜more/cops Model analysis

The DAG Problem Parameter f ∗ inf f Bounds ● Bounds on expressions ● Simple sample AMPL example Bearing ny = 50 -1.54824e-1 ● Objective bounds in COPS −∞ ● Redundant constraints Camshape n = 800 4.23739e+0 3.974 ● Fragment of camshape.mod Catmix nh = 800 -4.80559e-2 Convexity −∞ Classiﬁcation Chain nh = 200 5.06891e+0 −∞ Preprocessing Channel nh = 200 1.00000e+0 1 Solver choice Elec np = 100 4.44835e+3 0 Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 19/36 Objective bounds in COPS

∗ ● Outline f = lowest value given in the COPS report ● Numerical Optimization f = inferred lower bound Software inf See www.mcs.anl.gov/˜more/cops Model analysis

The DAG Problem Parameter f ∗ inf f Bounds ● Bounds on expressions ● Simple sample AMPL example Gasoil nh = 200 5.23659e-3 0 ● Objective bounds in COPS ● Redundant constraints Glider nh = 200 1.24880e+3 0 ● Fragment of camshape.mod

Convexity Marine nh = 200 1.97465e+7 0

Classiﬁcation Methanol nh = 100 9.00563e-3 0 Preprocessing Minsurf ny = 50 2.51488e+0 1.02594 Solver choice Pinene nh = 50 1.93937e+1 0 Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 19/36 Objective bounds in COPS

∗ ● Outline f = lowest value given in the COPS report ● Numerical Optimization f = inferred lower bound Software inf See www.mcs.anl.gov/˜more/cops Model analysis

The DAG Problem Parameter f ∗ inf f Bounds ● Bounds on expressions ● Simple sample AMPL example Polygon nv = 50 6.57163e-1 0 ● Objective bounds in COPS ● Redundant constraints Robot nh = 200 9.14138e+0 0 ● Fragment of camshape.mod

Convexity Rocket nh = 400 1.01238e+0 1

Classiﬁcation Steering nh = 100 5.54594e-1 0 Preprocessing Torsion ny = 50 -4.18087e-1 -0.833013 Solver choice

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 19/36 Redundant constraints

● Outline If the inferred bounds L U are tighter than those ● Numerical Optimization γ ci(x) γ given in the constraint cL ≤ c x ≤ cU, this constraint is Software i i( ) i redundant. ≤ ≤ Model analysis The DAG Redundant constraints are detected in camshape. Bounds ● Bounds on expressions ● Simple sample AMPL example Nature Imposed lower Inferred lower ● Objective bounds in COPS ● Redundant constraints ------● Fragment of camshape.mod Inequality 0.997647 < 1 Convexity Inequality -2.00235 < -2 Classiﬁcation

Preprocessing Found 2 redundant constraints. Solver choice

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 20/36 Fragment of camshape.mod

● Outline param n := 800; ● Numerical Optimization param R min := 1.0; Software param R max := 2.0; Model analysis

The DAG param alpha := 1.5;

Bounds let d theta := 2*pi/(5*(n+1)); ● Bounds on expressions ● Simple sample AMPL example var r{1..n} <= R max, >= R min; ● Objective bounds in COPS ● Redundant constraints ● Fragment of camshape.mod subject to curvature edge1: Convexity -alpha*d theta <= (r[1] - R min) <= alpha*d theta; Classiﬁcation

Preprocessing subject to curvature edge2: Solver choice -alpha*d theta <= (R max - r[n]) <= alpha*d theta;

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 21/36 Fragment of camshape.mod

● Outline param n := 800; ● Numerical Optimization param R min := 1.0; Software param R max := 2.0; Model analysis

The DAG param alpha := 1.5;

Preprocessing subject to curvature edge2: Solver choice -alpha*d theta <= (R max - r[n]) <= alpha*d theta;

Final notes Offending constraints

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 21/36 Convexity of expressions

● Outline Similarly, using the prototype function, convexity may be ● Numerical Optimization assessed. Software Model analysis Constants and variables are convex expression. Recursively, The DAG use rules such as Bounds

Convexity ■ ● Convexity of expressions f and g convex implies f + g convex ● Two sides to convexity ■ ● Disproving convexity fg is convex when both have the same monotonicity and ● Final verdict on convexity ● Convexity in COPS f, g 0 and convex, or f, g 0 and concave ≥ ≤ Classiﬁcation ■ ef is convex if f is convex Preprocessing ■ cosh(f) is convex if f is linear or f is convex and Solver choice nonnegative, or f is concave and nonpositive Final notes ■ √f nonconvex in general; however √ex is convex ■ etc. for every operator implemented in AMPL

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 22/36 Two sides to convexity

● Outline Recursively walking the dag allows to prove ● Numerical Optimization convexity/concavity symbolically. It is an absolute statement. Software

Model analysis

The DAG

Bounds

Convexity ● Convexity of expressions ● Two sides to convexity ● Disproving convexity ● Final verdict on convexity ● Convexity in COPS

Classiﬁcation

Preprocessing

Solver choice

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 23/36 Two sides to convexity

Bounds

Convexity ● Convexity of expressions ● Two sides to convexity ● Disproving convexity ● Final verdict on convexity ● Convexity in COPS

Classiﬁcation

Preprocessing

Solver choice

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 23/36 Two sides to convexity

● Outline Recursively walking the dag allows to prove ● Numerical Optimization convexity/concavity symbolically. It is an absolute statement. Software Model analysis What if no rule applies and convexity cannot be proved? The DAG Bounds In such a case, attempt to disprove convexity/concavity Convexity ● Convexity of expressions numerically. Use an iterative method to attempt to discover ● Two sides to convexity ● Disproving convexity negative curvature in the Hessian matrix of the function being ● Final verdict on convexity ● Convexity in COPS examined.

Classiﬁcation

Preprocessing

Solver choice

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 23/36 Disproving convexity

● Outline Use GLTR [Gould et al.] to ● Numerical Optimization

Software T 1 T 2 minimized f(x) d + 2 d f(x)d Model analysis ∇ ∇ subject to d ∆. The DAG k k ≤ Bounds

Convexity ∆ truncates the Lanczos iterations to keep the work within ● Convexity of expressions ● Two sides to convexity reasonable limits. ● Disproving convexity ● Final verdict on convexity ● Convexity in COPS By default, select only x = x0 but several xi can be chosen on L U Classiﬁcation a latin hypercube in x x x . ≤ i ≤ Preprocessing If f is nonconvex at x, negative curvature may be discovered Solver choice inside the trust region. If not, convexity cannot be disproved. Final notes We choose the radius ∆ = max(10, 1 f(x) ). 10 k∇ k

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 24/36 Final verdict on convexity

● Outline ● Numerical Optimization Proving Disproving Verdict Software √ X Convex Model analysis X √ Nonconvex The DAG X X Inconclusive Bounds

Convexity √ √ Error! ● Convexity of expressions ● Two sides to convexity ● Disproving convexity ● Final verdict on convexity ● Convexity in COPS

Classiﬁcation

Preprocessing

Solver choice

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 25/36 Convexity in COPS

● Outline Problem Pre n m nb Class Objective Time (s) ● Numerical Optimization

Software Bearing 0 2704 208 2704 QP Convex 0.86

Model analysis 10 2500 0 2500 BQP Convex 0.80

The DAG Camshape 0 800 1603 800 QCLP Linear 0.74 Bounds

Convexity 10 800 1600 800 QCLP Linear 0.76 ● Convexity of expressions ● Two sides to convexity Catmix 0 2403 1602 801 QCLP Linear 4.87 ● Disproving convexity ● Final verdict on convexity ● Convexity in COPS 10 2401 1600 801 QCLP Linear 4.87

Classiﬁcation Chain 0 402 203 0 ECNLP Inconclusive 0.11

Preprocessing 10 400 201 0 ECNLP Inconclusive 0.09 Solver choice QE Final notes Channel 0 1600 1600 0 Constant 1.70

10 1598 1598 0 QE Constant 1.74

Elec 0 300 100 0 QCNLP Nonconvex 3.13

10 300 100 0 QCNLP Nonconvex 3.15

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 26/36 Convexity in COPS

● Outline Problem Pre n m nb Class Objective Time (s) ● Numerical Optimization

Software Gasoil 0 2003 2003 0 QCQP Convex 2.01

Model analysis 10 2001 1998 3 QCQP Convex 1.80

The DAG Glider 0 1006 1411 0 QCLP Linear 0.65 Bounds

Convexity 10 999 800 601 QCLP Linear 0.73 ● Convexity of expressions ● Two sides to convexity Marine 0 4815 4807 0 QCQP Convex 9.40 ● Disproving convexity ● Final verdict on convexity ● Convexity in COPS 10 4815 4792 15 QCQP Convex 9.17

Classiﬁcation Methanol 0 1205 1205 0 NCQP Convex 1.02

Preprocessing 10 1202 1197 5 NCQP Convex 0.75 Solver choice LCNLP Final notes Minsurf 0 2704 3696 0 Inconclusive 0.76

10 2500 0 2500 LCNLP Inconclusive 1.01

Pinene 0 1005 1005 0 QCQP Convex 0.85

10 1000 995 5 QCQP Convex 0.70

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 26/36 Convexity in COPS

● Outline Problem Pre n m nb Class Objective Time (s) ● Numerical Optimization

Software Polygon 0 100 1376 0 NLP Nonconvex 4.25

Model analysis 10 98 1273 98 ICNLP Nonconvex 3.17

The DAG Robot 0 1811 1213 1207 QCLP Linear 2.44 Bounds

Convexity 10 1799 1201 1202 NCLP Linear 2.51 ● Convexity of expressions ● Two sides to convexity Rocket 0 1605 2408 401 NCLP Linear 1.62 ● Disproving convexity ● Final verdict on convexity ● Convexity in COPS 10 1601 1200 1601 NCLP Linear 1.99

Classiﬁcation Steering 0 507 510 0 NCLP Linear 0.21

Preprocessing 10 500 401 103 NCLP Linear 0.25 Solver choice QP Final notes Torsion 0 2704 2704 0 Convex 0.70

10 2500 0 2500 BQP Convex 0.70

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 26/36 Problem classiﬁcation

● Outline Build a richer optimization tree for differentiable problems ● Numerical Optimization

Software

Model analysis

The DAG

Bounds

Convexity

Classification ● Problem classification ● Problem classification

Preprocessing

Solver choice

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 27/36 Problem classiﬁcation

NLPs MINLPs ● Outline ● Numerical Optimization Constrained Unconstrained MPECs NLPs NLPs Software Quadratically− Model analysis Quadratically− −constrained LPs −constrained NLPs Unconstrained Unconstrained The DAG LPs QPs

Bounds Linearly− Quadratically− Convexity LPs −constr. QPs −constrained Convex Noncnvx NLPs Classification ● Problem classification ● Problem classification Bound− Linearly− Nonlinearly− Preprocessing −constr. −constr. −constrained Convex Noncnvx NLPs QPs NLPs Solver choice

Final notes Bound− Equality− Inequality− −constr. Convex Noncnvx NCPs NLEs NLLS −constrained −constrained QPs NLPs NLPs

LCPs LLS Bound− −constr. LPs

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 27/36 Problem classiﬁcation

● Outline Specializing a problem moves it down the tree. ● Numerical Optimization

Software Problem type

Model analysis ------Problem has bounded variables The DAG -Problem has linear constraints Bounds -Problem has quadratic constraints Convexity

Classification ● Problem classification Analyzing problem using objective #0 ● Problem classification ------Preprocessing -This objective is linear Solver choice -Problem is a constrained NLP Final notes -Problem seems to be an LP -Problem seems to be a quadratically-constrained LP with inequality constraints with bounds

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 28/36 ■ Attempt to tighten bounds on variables ■ Examine each constraint in turn and decompose ci(x) = `(x) + αj xj + n(x)

■ Find bounds on `(x) and n(x) and obtain cL `U nU α x cU `L nL i − − ≤ j j ≤ i − −

■ L U May return tighter bounds than [xj , xj] ■ Keep an updated "tight" vector of bounds ■ Command-line option for (ﬁnite) number of passes ■ Tighter bounds on x means "no-looser" bounds on functions

"Nonlinear preprocessing"

● Outline ■ Build upon AMPL’s linear primal presolve ● Numerical Optimization

Software

Model analysis

The DAG

Bounds

Convexity

Classiﬁcation

Preprocessing ● "Nonlinear preprocessing" ● "Nonlinear preprocessing" in COPS

Solver choice

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 29/36 ■ Examine each constraint in turn and decompose ci(x) = `(x) + αj xj + n(x)

■ Find bounds on `(x) and n(x) and obtain cL `U nU α x cU `L nL i − − ≤ j j ≤ i − −

"Nonlinear preprocessing"

● Outline ■ Build upon AMPL’s linear primal presolve ● Numerical Optimization ■ Software Attempt to tighten bounds on variables

Model analysis

The DAG

Bounds

Convexity

Classiﬁcation

Preprocessing ● "Nonlinear preprocessing" ● "Nonlinear preprocessing" in COPS

Solver choice

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 29/36 ■ Find bounds on `(x) and n(x) and obtain cL `U nU α x cU `L nL i − − ≤ j j ≤ i − −

"Nonlinear preprocessing"

● Outline ■ Build upon AMPL’s linear primal presolve ● Numerical Optimization ■ Software Attempt to tighten bounds on variables

Model analysis ■ Examine each constraint in turn and decompose

The DAG c x ` x α x n x Bounds i( ) = ( ) + j j + ( )

Convexity

Classiﬁcation

Preprocessing ● "Nonlinear preprocessing" ● "Nonlinear preprocessing" in COPS

Solver choice

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 29/36 ■ L U May return tighter bounds than [xj , xj] ■ Keep an updated "tight" vector of bounds ■ Command-line option for (ﬁnite) number of passes ■ Tighter bounds on x means "no-looser" bounds on functions

"Nonlinear preprocessing"

● Outline ■ Build upon AMPL’s linear primal presolve ● Numerical Optimization ■ Software Attempt to tighten bounds on variables

Model analysis ■ Examine each constraint in turn and decompose

The DAG c x ` x α x n x Bounds i( ) = ( ) + j j + ( )

Convexity Classiﬁcation ■ Find bounds on `(x) and n(x) and obtain Preprocessing ● "Nonlinear preprocessing" ● L U U U L L "Nonlinear preprocessing" in COPS ci ` n αjxj ci ` n Solver choice − − ≤ ≤ − −

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 29/36 ■ Keep an updated "tight" vector of bounds ■ Command-line option for (ﬁnite) number of passes ■ Tighter bounds on x means "no-looser" bounds on functions

"Nonlinear preprocessing"

● Outline ■ Build upon AMPL’s linear primal presolve ● Numerical Optimization ■ Software Attempt to tighten bounds on variables

Model analysis ■ Examine each constraint in turn and decompose

The DAG c x ` x α x n x Bounds i( ) = ( ) + j j + ( )

Final notes ■ L U May return tighter bounds than [xj , xj]

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 29/36 ■ Command-line option for (ﬁnite) number of passes ■ Tighter bounds on x means "no-looser" bounds on functions

"Nonlinear preprocessing"

● Outline ■ Build upon AMPL’s linear primal presolve ● Numerical Optimization ■ Software Attempt to tighten bounds on variables

Model analysis ■ Examine each constraint in turn and decompose

The DAG c x ` x α x n x Bounds i( ) = ( ) + j j + ( )

Final notes ■ L U May return tighter bounds than [xj , xj] ■ Keep an updated "tight" vector of bounds

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 29/36 ■ Tighter bounds on x means "no-looser" bounds on functions

"Nonlinear preprocessing"

● Outline ■ Build upon AMPL’s linear primal presolve ● Numerical Optimization ■ Software Attempt to tighten bounds on variables

Model analysis ■ Examine each constraint in turn and decompose

The DAG c x ` x α x n x Bounds i( ) = ( ) + j j + ( )

Final notes ■ L U May return tighter bounds than [xj , xj] ■ Keep an updated "tight" vector of bounds ■ Command-line option for (ﬁnite) number of passes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 29/36 "Nonlinear preprocessing"

● Outline ■ Build upon AMPL’s linear primal presolve ● Numerical Optimization ■ Software Attempt to tighten bounds on variables

Model analysis ■ Examine each constraint in turn and decompose

The DAG c x ` x α x n x Bounds i( ) = ( ) + j j + ( )

Final notes ■ L U May return tighter bounds than [xj , xj] ■ Keep an updated "tight" vector of bounds ■ Command-line option for (ﬁnite) number of passes ■ Tighter bounds on x means "no-looser" bounds on functions

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 29/36 "Nonlinear preprocessing" in COPS

● Outline ● Numerical Optimization passes fmin fmax n/nvar Software 0 3.14159 6.28319 0/800 Model analysis 5 3.16109 5.44318 431/800 The DAG Problem camshape with presolve = 0 Bounds

Convexity passes fmin fmax n/nvar Classiﬁcation

Preprocessing 0 3.974 6.24433 0/800 ● "Nonlinear preprocessing" ● "Nonlinear preprocessing" in COPS 5 3.974 5.44305 425/800 Solver choice Problem camshape with presolve = 10 Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 30/36 Assistance in choosing a solver

● Outline ■ Once problem class is identiﬁed, query solver database ● Numerical Optimization ■ Software SQL queries assembled as classiﬁcation is carried out

Model analysis ■ Database contains 28 solvers for linear and nonlinear The DAG optimization Bounds ■ Solvers are/were on NEOS Convexity ■ Database links solvers to problem classes Classiﬁcation ■ Implemented with SQLite Preprocessing

Solver choice ● Assistance in choosing a solver ● Recommendation example

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 31/36 Recommendation example

● Outline Assume problem was deemed convex ● Numerical Optimization Software ### First query:: Soft parameters only ### Model analysis

The DAG SBB

Bounds TRON

Convexity BLMVM

Classiﬁcation MINLP

Preprocessing FortMP

Solver choice PathNLP ● Assistance in choosing a solver ● Recommendation example L-BFGS-B

Final notes First query ignores convexity requirement

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 32/36 Recommendation example

● Outline Assume problem was deemed convex ● Numerical Optimization Software ### Second query:: Include hard parameters ### Model analysis

The DAG OOQP

Bounds MOSEK

Convexity

Classiﬁcation Second query enforces convexity requirement, if detected

Preprocessing

Solver choice ● Assistance in choosing a solver ● Recommendation example

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 32/36 Recommendation example

● Outline Assume problem was deemed convex ● Numerical Optimization Software ### Matching general-purpose solvers ### Model analysis

The DAG Loqo

Bounds Knitro

Convexity Lancelot

Classiﬁcation

Preprocessing Last query returns matching general-purpose solvers

Solver choice ● Assistance in choosing a solver ● Recommendation example

Final notes

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 32/36 Statistics on DrAmpl

● Outline ■ About 7000 lines of code ● Numerical Optimization ■ Software Uses the AMPL library extensively

Model analysis ■ Written in ANSI C, should install on any platform where the The DAG AMPL library may be installed Bounds ■ Being developed and tested in Linux Convexity ■ Will (hopefully quite soon) be available for testing on NEOS Classiﬁcation

Preprocessing

Solver choice

Final notes ● Statistics on DrAmpl ● Other features ● Future features ● Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 33/36 ■ Prove that a nonlinear function is a polynomial ■ Prove that a function is DC ■ Assess convexity of feasible set

Other features

● Outline The following features are being implemented ● Numerical Optimization Software ■ Prove quadraticity Model analysis

The DAG

Bounds

Convexity

Classiﬁcation

Preprocessing

Solver choice

Final notes ● Statistics on DrAmpl ● Other features ● Future features ● Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 34/36 ■ Prove that a function is DC ■ Assess convexity of feasible set

Other features

● Outline The following features are being implemented ● Numerical Optimization Software ■ Prove quadraticity Model analysis ■ Prove that a nonlinear function is a polynomial The DAG

Bounds

Convexity

Classiﬁcation

Preprocessing

Solver choice

Final notes ● Statistics on DrAmpl ● Other features ● Future features ● Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 34/36 ■ Assess convexity of feasible set

Other features

● Outline The following features are being implemented ● Numerical Optimization Software ■ Prove quadraticity Model analysis ■ Prove that a nonlinear function is a polynomial The DAG ■ Bounds Prove that a function is DC

Convexity

Classiﬁcation

Preprocessing

Solver choice

Final notes ● Statistics on DrAmpl ● Other features ● Future features ● Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 34/36 Other features

Convexity ■ Assess convexity of feasible set

Classiﬁcation

Preprocessing

Solver choice

Final notes ● Statistics on DrAmpl ● Other features ● Future features ● Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 34/36 ■ Detect unbounded problems ■ Recommend a solver based on past experience ■ Include many more solvers in database ■ Treat problems with integer variables ■ Link with mProbe ■ Minimize the work in nonlinear presolve ■ Output updated nl ﬁle ■ Compare solving times with[out] presolving ■ IEEE directed roundings ■ Bounds on dual variables from KKT conditions ■ Hand model off to Kestrel

Future features

● Outline ■ Detect infeasible problems ● Numerical Optimization

Software

Model analysis

The DAG

Bounds

Convexity

Classiﬁcation

Preprocessing

Solver choice

Final notes ● Statistics on DrAmpl ● Other features ● Future features ● Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 35/36 ■ Recommend a solver based on past experience ■ Include many more solvers in database ■ Treat problems with integer variables ■ Link with mProbe ■ Minimize the work in nonlinear presolve ■ Output updated nl ﬁle ■ Compare solving times with[out] presolving ■ IEEE directed roundings ■ Bounds on dual variables from KKT conditions ■ Hand model off to Kestrel

Future features

● Outline ■ Detect infeasible problems ● Numerical Optimization ■ Software Detect unbounded problems

Model analysis

The DAG

Bounds

Convexity

Classiﬁcation

Preprocessing

Solver choice

Final notes ● Statistics on DrAmpl ● Other features ● Future features ● Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 35/36 ■ Include many more solvers in database ■ Treat problems with integer variables ■ Link with mProbe ■ Minimize the work in nonlinear presolve ■ Output updated nl ﬁle ■ Compare solving times with[out] presolving ■ IEEE directed roundings ■ Bounds on dual variables from KKT conditions ■ Hand model off to Kestrel

Future features

● Outline ■ Detect infeasible problems ● Numerical Optimization ■ Software Detect unbounded problems

Model analysis ■ Recommend a solver based on past experience

The DAG

Bounds

Convexity

Classiﬁcation

Preprocessing

Solver choice

Final notes ● Statistics on DrAmpl ● Other features ● Future features ● Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 35/36 ■ Treat problems with integer variables ■ Link with mProbe ■ Minimize the work in nonlinear presolve ■ Output updated nl ﬁle ■ Compare solving times with[out] presolving ■ IEEE directed roundings ■ Bounds on dual variables from KKT conditions ■ Hand model off to Kestrel

Future features

● Outline ■ Detect infeasible problems ● Numerical Optimization ■ Software Detect unbounded problems

Model analysis ■ Recommend a solver based on past experience The DAG ■ Include many more solvers in database Bounds

Convexity

Classiﬁcation

Preprocessing

Solver choice

Final notes ● Statistics on DrAmpl ● Other features ● Future features ● Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 35/36 ■ Link with mProbe ■ Minimize the work in nonlinear presolve ■ Output updated nl ﬁle ■ Compare solving times with[out] presolving ■ IEEE directed roundings ■ Bounds on dual variables from KKT conditions ■ Hand model off to Kestrel

Future features

● Outline ■ Detect infeasible problems ● Numerical Optimization ■ Software Detect unbounded problems

Model analysis ■ Recommend a solver based on past experience The DAG ■ Include many more solvers in database Bounds ■ Treat problems with integer variables Convexity

Classiﬁcation

Preprocessing

Solver choice

Final notes ● Statistics on DrAmpl ● Other features ● Future features ● Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 35/36 ■ Minimize the work in nonlinear presolve ■ Output updated nl ﬁle ■ Compare solving times with[out] presolving ■ IEEE directed roundings ■ Bounds on dual variables from KKT conditions ■ Hand model off to Kestrel

Future features

● Outline ■ Detect infeasible problems ● Numerical Optimization ■ Software Detect unbounded problems

Model analysis ■ Recommend a solver based on past experience The DAG ■ Include many more solvers in database Bounds ■ Treat problems with integer variables Convexity ■ Link with mProbe Classiﬁcation

Preprocessing

Solver choice

Final notes ● Statistics on DrAmpl ● Other features ● Future features ● Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 35/36 ■ Output updated nl ﬁle ■ Compare solving times with[out] presolving ■ IEEE directed roundings ■ Bounds on dual variables from KKT conditions ■ Hand model off to Kestrel

Future features

● Outline ■ Detect infeasible problems ● Numerical Optimization ■ Software Detect unbounded problems

Solver choice

Final notes ● Statistics on DrAmpl ● Other features ● Future features ● Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 35/36 ■ Compare solving times with[out] presolving ■ IEEE directed roundings ■ Bounds on dual variables from KKT conditions ■ Hand model off to Kestrel

Future features

● Outline ■ Detect infeasible problems ● Numerical Optimization ■ Software Detect unbounded problems

Solver choice ■ Output updated nl ﬁle

Final notes ● Statistics on DrAmpl ● Other features ● Future features ● Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 35/36 ■ IEEE directed roundings ■ Bounds on dual variables from KKT conditions ■ Hand model off to Kestrel

Future features

● Outline ■ Detect infeasible problems ● Numerical Optimization ■ Software Detect unbounded problems

Solver choice ■ Output updated nl ﬁle Final notes ■ ● Statistics on DrAmpl Compare solving times with[out] presolving ● Other features ● Future features ● Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 35/36 ■ Bounds on dual variables from KKT conditions ■ Hand model off to Kestrel

Future features

● Outline ■ Detect infeasible problems ● Numerical Optimization ■ Software Detect unbounded problems

Solver choice ■ Output updated nl ﬁle Final notes ■ ● Statistics on DrAmpl Compare solving times with[out] presolving ● Other features ■ ● Future features IEEE directed roundings ● Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 35/36 ■ Hand model off to Kestrel

Future features

● Outline ■ Detect infeasible problems ● Numerical Optimization ■ Software Detect unbounded problems

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 35/36 Future features

● Outline ■ Detect infeasible problems ● Numerical Optimization ■ Software Detect unbounded problems

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 35/36 Personal data

● Outline ● Numerical Optimization Contact [email protected]

Software Model analysis Homepage www.mgi.polymtl.ca/dominique.orban The DAG

Bounds Reports $ HOME /reports.html Convexity { } Classiﬁcation

Preprocessing

Solver choice

Final notes ● Statistics on DrAmpl ● Other features ● Future features ● Personal data

Dominique Orban, May 21, 2004 The DrAmpl meta solver - p. 36/36