Midwest Optimization Seminar Western Michigan University October 6, 2001

NEOS Optimization Solutions for Scientiﬁc Computing Problems

Jorge J. Mor´e Joint work with Liz Dolan, Bob Fourer, Todd Munson www-neos.mcs.anl.gov

Mathematics and Computer Science Division Argonne National Laboratory Outline

How do we solve optimization problems? How do we solve optimization problems with NEOS? NEOS users and uses Data mining the NEOS database Selected recent developments ? Kestrel ? Performance proﬁles ? GAMS to AMPL translator Solving Optimization Problems: The standard model

Develop code to deﬁne the optimization problem Decide on the software Retrieve the software Hand-code gradients, Hessians, sparsity patterns. Link the software. Compile, debug, compile, debug, . . . , run Solve. Solving Optimization Problems: The NEOS Model

Remote Solvers

NEOS Administration Server Tools

Web browser Email NST Kestrel (HTTP) (SMTP) (Sockets) (Corba)

Optimization Problem Solving Optimization Problems: The NEOS Model

Develop code to deﬁne the optimization problem Submit the problem to NEOS Interpret the results Repeat the process

Advantages ◦ Easy to use ◦ No need to retrieve the software ◦ Derivatives and sparsity patterns computed automatically ◦ Independent of computing resources NEOS Server Highlights

November 1994 – Demo at SC94. Version 1: September 1995 Version 2: May 1997 Version 3: March 4, 1999 Version 4: May 3, 2001

Processing rates Version 1: 590 jobs per month Version 2: 1280 jobs per month Version 3: 2810 jobs per month Version 4: 5750 jobs per month Design Guidelines for NEOS

Minimal problem speciﬁcation Uniform and automated software registration Fault tolerant Minimal maintenance Research Issues for NEOS

How do we add solvers? How are problems speciﬁed? How are problems submitted? How are problems scheduled for solution? How are the problems solved? Where are the problems solved? Remote Solvers

◦ Argonne National Laboratory ◦ Arizona State University ◦ National Taiwan University ◦ Northwestern University ◦ Technical University Aachen ◦ University of Wisconsin Modeling Languages: Shape Optimization of a Cam

Maximize the area of the valve opening in a convex cam,

n ! 1 X f(r) = πr2 r , v n i i=1 subject to the constraints rmin ≤ ri ≤ rmax on the radii ri. The convexity constraint is expressed as

2ri−1ri+1 cos(θ) ≤ ri(ri−1 + ri+1), i = 0, . . . , n + 1, and the curvature requirement is expressed by r − r −α ≤ i+1 i ≤ α, i = 0, . . . , n. θ NST: NEOS Submission Tool NST: Scheduling NST: Executing NST: Solution Shape Optimization of a Cam Uses of NEOS

Optimization solutions for scientiﬁc computing problems Education and information on optimization algorithms Classroom projects (beware of deadlines) Testing optimization software Comparing optimization software Prototype problem solving environment NEOS Users and Uses – 2000

Estimating the value at risk of ﬁnancial institutions Optimal shift schedules for ground-handling activities Unilateral contact problems in engineering mechanics Distance geometry and multidimensional scaling Portfolio selection and scheduling Designing a yagi antenna Resource requirements for broadband networks Image reconstruction of space objects Optimization of weights for routing in the Internet Designing large, distributed communication networks Circuit simulation at Philips Research Laboratories NEOS Users and Uses – 2001

Optimizing design parameters for high-voltage power systems Teaching network flow optimization Predicting globular protein folding Showing characteristics of multi-person co-operation Studying the brain’s representation system Modeling electricity markets Designing fractional delay filters for LAN/WAN’s Scheduling thermo and hydro energy resources Building a crew-scheduling system for an airline in Indonesia Solving the modified Cahn-Hilliard free energy equation Applying optimization to farming in Switzerland Data Mining the NEOS Database

Question Is there any worthwhile information on the NEOS database?

Number of jobs per time period Number of jobs per solver area Number of jobs per solver (in a given area) NEOS Solvers, part I

Nonlinearly Constrained Optimization ◦ CONOPT, DONLP2, FILTER , KNITRO, ◦ LANCELOT, LOQO, MINOS, MOSEK, SNOPT Mixed Integer Nonlinearly Constrained Optimization ◦ MINLP, SBB Mixed Integer Linear Programming ◦ BonsaiG, FORTMP, GLPK, XPRESS Semideﬁnite Programming ◦ CSDP, CirCut, DSDP, SDPA, SDPT3, SeDuMi Linear Programming ◦ BDMLP, FORTMP, HOPDM, MOSEK, PCx, XPRESS NEOS Solvers, part II

Bound Constrained Optimization ◦ BLMVM, L-BFGS-B, LANCELOT, TRON Nondiﬀerentiable Optimization ◦ ACCPM, APPS, DFO, NDA Unconstrained Optimization ◦ CGplus, NMTR, VMLM Linear Network Optimization ◦ NETFLO, RELAX4 Complementarity Problems ◦ MILES, PATH Stochastic Linear Programming ◦ AUGMENTED, MSLIP NEOS: October 1999 – July 2001

8000

7000

6000 1999 2000 2001 5000

4000 Submissions 3000

2000

1000

0 1234567891011 12 Month NEOS Jobs: June 2000 – June 2001

21,629

15,082 Jobs

5,530

1,277 1,429 1,619 318 507 542 610 615

SLP NDO ADMIN CP SDP BCO UCO LNO LP IP NCO

SLP Stochastic Linear Programming SDP Semidefinite Programming LP Linear Programming NDO Nondifferentiable Optimization BCO Bound Constrained Optimization IP Integer Programming ADMIN Administrative Solvers UCO Unconstrained Optimization NCO Nonlinearly Constrained CP Complementarity Problems LNO Linear Network Optimization Optimization Nonlinearly Constrained Optimization Solvers

6000

5000

4000

3000

2000

1000

L O L L T L L L L L S S S T P R P P O P P P P M M M P M P M M L M M M MP M A A A O A - A A E A A A A A G G N - L - - C ------G - S P T O T 2 O K T T S S N P R P O M O Q A E L R E P P O IN A L O L O T IT S O O N E L N IL N O N N I M C S F O N S M N D K M O A C L Integer Programming Solvers

8000

7000

6000

5000

4000

3000

2000

1000

0 MINLP- XPRESS- XPRESS- FORTMP XPRESS- BONSAIG- BONSAIG- BONSAIG MIQP AMPL MPMODEL AMPL INTEGER AMPL LP Linear Programming Solvers

2500

2000

1500

1000

500

X S L L X L P L M D P L R C M P P E P -L P D P M E E P A M M L M M P T D I P D A M R -G -A -A -A P - O P R O R P S D IM X M K H B O M A L S P S C P E F P - P S -B M E M S B -M S D R P S O S S B P B E M S E X R E R P R P X P X X Research Challenges

Metacomputing environments for optimization ◦ www.mcs.anl.gov/metaneos Development of solvers with C input Global optimization solvers An adaptive scheduler Data mine the database Benchmark solvers Flexible interfaces Application speciﬁc solvers Increase computational capacity Selected Recent Developments

Kestrel Performance proﬁles GAMS to AMPL translator

Notes ? Kestrel provides a ﬂexible interface between NEOS, and the mathematical programming languages AMPL and GAMS ? Performance proﬁles provide a rational framework for comparing solvers ? The GAMS to AMPL translator is courtesy of GAMS Development Corp. Kestrel Kestrel Solvers (Partial Listing)

AMPL GAMS

◦ BPMPD ◦ BDMLD ◦ DONLP2 ◦ CONOPT ◦ FORTMP ◦ MILES ◦ KNITRO ◦ MINOS ◦ L-BFGS-B ◦ PATH ◦ LANCELOT ◦ SBB ◦ LOQO ◦ SNOPT ◦ MINOS ◦ SNOPT ◦ TRON Benchmarking

A benchmark is a set P of test problems, a termination criterion T , and a set of S of solvers.

Choose a performance measure tp,s for each problem p ∈ P and solver s ∈ S.

Performance Measures Number of iterations Number of function evaluations Number of conjugate gradient iterations Computing time Performance Proﬁles

Deﬁnition. A performance proﬁle is the distribution function of a performance metric.

Compute the performance ratio

tp,s rp,s = . min{tp,s : 1 ≤ s ≤ ns}

Compute the probability distribution function 1 n o ρs(τ) = size p ∈ P : rp,s ≤ τ np Properties of Performance Proﬁles

The performance proﬁle ρs : R 7→ [0, 1] is a non-decreasing, piecewise constant function, continuous from the right at each breakpoint.

Not sensitive to the data on a small number of problems Not sensitive to small changes in the data Information on the size of improvement is provided Does not depend on the subjective choice of a parameter Can be used to compare more than two solvers. Benchmarking Tools

Compute average or cumulative totals of metric ◦ Sensitive to results on a small number of problems Rank solvers by the number of k-th place entries. ◦ No information on the size of improvement Compute the number of wins by a ﬁxed amount or percentage ◦ Dependent on the subjective choice of a parameter Compute medians or quartiles of a metric ◦ Information between quartiles is lost Performance Proﬁle for the COPS optimal control problems

Log Scaled Performance Profile on Subset of COPS 2 1

0.9

0.8 )

s 0.7 n ≤ s

≤ 0.6 : 1 τ

≤ 0.5 )) p,s 0.4 (r 2

0.3 P( (log

0.2 LANCELOT LOQO MINOS SNOPT 0.1

0 0 2 4 6 8 10 ...r τ M Performance Proﬁle for the full COPS problems

Log Scaled Performance Profile on Full COPS Set 2 1

0.9

0.8 )

s 0.7 n ≤

s 0.6 ≤ : 1 τ

≤ 0.5 ))

p,s 0.4 (r 2

0.3 P( (log

0.2 LANCELOT LOQO MINOS 0.1 SNOPT

0 0 2 4 6 8 10 ...r τ M Performance Proﬁle for Mittleman’s LP Data

Log Scaled Performance Profile for LP Solvers 2 1

BPMPD−2.14 0.9

0.8 MOSEK

0.7 )

s PCx n ≤ 0.6 s SOPLEXC ≤

LPABO : 1

τ 0.5 BPMPD−2.11

≤

)) COPL_LP

p,s 0.4 (r 2

0.3 P( (log

0.2

0.1

0 0 2 4 6 8 10 12 ...r τ M NEOS

A comprehensive guide to optimization algorithms and software

An environment for solving optimization problems over the Internet NEOS Team

? Michael Bussieck (GAMS) ? Chih-Jen Lin (Taiwan) ? Liz Dolan (Argonne/NU) ? Jeﬀ Linderoth (Axioma) ? Steve Dirkse (GAMS) ? Sven Leyﬀer (Dundee) ? Michael Ferris (Wisconsin) ? Hans Mittelmann (Arizona) ? Bob Fourer (Northwestern) ? Todd Munson (Argonne) ? David Gay (Lucent) ? Jorge Nocedal (Northwestern) ? Philip Gill (UC San Diego) ? Michael Saunders (Stanford) ? Nick Gould (Rutherford) ? Bob Vanderbei (Princeton) ? Jean-Pierre Goux (Artelys) ? Steve Wright (Wisconsin)

Emeritus: Mike Mesnier (Intel) Selected References

J. Czyzyk and M. Mesnier, and J. Moré, The NEOS Server, IEEE Journal on Computational Science and Engineering, 5 (1998), 68-75. R. Fourer and J.-P. Goux, Optimization as an Internet resource, Interfaces 31 (2001), 130-150. E. Dolan, NEOS Server 4.0 Administrative Guide, Technical Report ANL/MCS-TM-250, Mathematics and Computer Science Division, Argonne National Laboratory, May 2001. E. Dolan and J. Moré,Benchmarking optimization software with performance profiles, to appear in Mathematical Programming, 2001.