AMO - Advanced Modeling and Optimization Volume 1, Number 1, 1999

The TOMLAB Optimization

Environment in Matlab

Kenneth Holmstrom

Center for Mathematical Mo deling

Department of Mathematics and Physics

Malardalen University PO Box SE Vasteras Sweden

Abstract

TOMLAB is a general purp ose op en and integrated Matlab development environment for

research and teaching in optimization on Unix and PC systems One motivation for TOMLAB

is to simplify research on practical optimization problems giving easy access to all types of

solvers at the same time having full access to the p ower of Matlab The design principle is

dene your problem once optimize using any suitable solver In this pap er we discuss the design

and contents of TOMLAB as well as some applications where TOMLAB has b een successfully

applied

TOMLAB is based on NLPLIB TB a Matlab to olb ox for and pa

rameter estimation and OPERA TB a Matlab to olb ox for linear and discrete optimization

More than dierent algorithms and graphical utilities are implemented It is p ossible to call

solvers in the Matlab Optimization Toolb ox and generalpurp ose solvers implemented in For

tran or using a MEXle interface Currently MEXle interfaces have b een developed for

MINOS NPSOL NPOPT NLSSOL LPOPT QPOPT and LSSOL A problem is solved by

a direct call to a solver or by a general multisolver driver routine or interactively using a

graphical user interface GUI or a menu system If analytical derivatives are not available

automatic dierentiation is easy using an interface to ADMATADMIT TB Furthermore ve

types of numerical dierentiation metho ds are included in NLPLIB TB

A large set of standard test problems are included as well as example and demonstration

les New userdened problems are easily added Furthermore using MEXle interfaces

problems in the CUTE test problem data base and problems dened in the AMPL mo deling

language can b e solved TOMLAB currently solves small and medium size dense problems

where the available memory is the limit TOMLAB is free for academic purp oses and down

loadable at our home page

Keywords Mathematical Programming Nonlinear Programming Op erations Research Matlab

CUTE AMPL Graphical User Interface Software Engineering Mathematical Software Optimiza

tion Algorithms Nonlinear Least Squares

AMS Sub ject Classication C C C

1

Financed by the Malardalen University Research Board pro ject Applied Optimization and Modeling TOM

2

Email hkhmdhse URL httpwwwimamdhsetom

The TOMLAB Optimization Environment in Matlab

Introduction

Many scientists and engineers are using Matlab as a mo deling and analysis to ol but for the solution

of optimization problems the supp ort is weak That was one strong motivation for starting the de

velopment of TOMLAB a generalpurp ose op en and integrated development environment

in Matlab for research and teaching in optimization Another motivation was to provide p owerful

to ols for the research on the hardtosolve applied problems our research group are working on by

systematic development of a large set of usable optimization to ols This pap er presents the design

and contents of TOMLAB

To solve optimization problems traditionally the user has b een forced to write a co de that

calls some standard solver written as a Fortran subroutine For nonlinear problems the user must

also write subroutines computing the ob jective function value and the vector of constraint function

values The needed derivatives are either explicitly co ded computed by using numerical dierences

or derived using automatic dierentiation techniques

In recent years several mo deling languages are developed like AIMMS AMPL ASCEND

GAMS and LINGO The mo deling system acts as a prepro cessor The user describ es

the details of his problem in a very verbal language an opp osite to the concise mathematical

description of the problem The problem description le is normally mo died in a text editor with

help from example les solving the same type of problem Much eort is directed to the development

of more user friendly interfaces The mo del system pro cesses the input description le and calls

any of the available solvers For a solver to b e accessible in the mo deling system sp ecial types of

interfaces are developed

The mo deling language approach is suitable for many management and decision problems but may

not always b e the b est way for engineering problems which often are nonlinear with a complicated

problem description Until recently the supp ort for nonlinear problems in the mo deling languages

has b een crude This is now rapidly changing

For p eople with a mathematical background mo deling languages often seems to b e a very tedious

way to dene an optimization problem There has b een several attempts to nd languages more

suitable than Fortran or CC to describ e mathematical problems like the compact and p owerful

APL language Nowadays languages like Matlab has a rapid growth of users Matlab was

originally developed as a prepro cessor to the standard Fortran subroutine libraries in numerical

linear algebra LINPACK and EISPACK much the same idea as the mo deling languages

discussed ab ove Matlab of to day is an advanced and p owerful to ol with graphics animation and

advanced menu design p ossibilities integrated with the mathematics The Matlab language has

made the development of to olb oxes p ossible which serves as a direct extension to the language itself

Using Matlab as an environment for solving optimization problems oers much more p ossibilities

for analysis than just the pure solution of the problem

The concept of TOMLAB is to integrate all dierent systems getting access to the b est of all

worlds Furthermore TOMLAB provides wellimplemented solution algorithms for most common

optimization problems TOMLAB should b e seen as a complement to existing mo del languages

for the user needing more p ower and exibility than given by a mo deling system

Much eort has b een put on advanced design utilizing the p ower and features of MatlabThe design

principle is dene your problem once optimize using any suitable solver This is p ossible using

general gateway and interface routines global variables string evaluations and structure arrays

When solving advanced applied problems the design principle is of sp ecial imp ortance as it might

b e a huge task to rewrite the function constraint and derivative sp ecications

A stack strategy for the global variables makes recursive calls p ossible in eg separable nonlinear

least squares algorithms One structure holds all information ab out the problem and one holds the

results For the optimization algorithm developer and the applied researcher in need of optimization

to ols it is very easy to compare dierent solvers or do test runs on thousands of problems A to ol

is provided to automate the tedious work of making computational result tables from test runs

directly making LaTeX tables for inclusion in pap ers TOMLAB should b e seen as a prop osal for

The TOMLAB Optimization Environment in Matlab

a standard for optimization in Matlab

In Sweden TOMLAB has b een used for several years in optimization courses It is distributed free

of charge for academic purp oses During the rst months since the rst ocial release of TOMLAB

December ab out researchers and research groups have downloaded the software All

users have b een requested to answer what applications they have in mind for TOMLAB and it is

clear that most elds of applied mathematics are represented So there is an obvious need for go o d

optimization to ols in the scientic community

This pap er is organized as follows In Section we discuss the design of TOMLAB Overviews of

the two to olb oxes that are part of TOMLAB are given in Section NLPLIB TB and Section

OPERA TB The graphical user interface GUI is further discussed in The

MEXle interfaces to several solvers are discussed in Section and the interface to the Matlab

Optimization Toolb ox is discussed in Section Currently TOMLAB may run problems dened

in CUTE Section and AMPL Section Finally we end with some conclusions in

Section

The Design of TOMLAB

As the scop e of TOMLAB is large and broad there is a clear need of a welldesigned system It is

also necessary to use the p ower of the Matlab language to make the system exible and easy to use

and maintain We have used the concept of structure arrays and made use of b oth the ability in

Matlab to execute Matlab co de dened as string expressions and to execute functions sp ecied by

a string To b e able to live up to the basic design principle of TOMLAB dene your problem once

optimize using any suitable solver very many dierent interface routines as well as several interface

layers was needed It is a dicult task to provide the necessary information for a particular solver

esp ecially when the problem class is just a subset of the class of problems the particular solver is

able to handle To solve this design problem in a general and easily expandable way we use one

structure array that holds all information ab out the problem all input for the solution pro cess In

a similar way one structure array holds the all output from the optimization together with the

input structure array This makes advanced p ostpro cessing of results easy

The thought is that the routines in TOMLAB may b e put together into more advanced hybrid

algorithms or b eing subparts in more complex algorithms Therefore it is imp ortant to organize all

handling of global information to avoid conicts We use a stack strategy for the global variables

that works well when making recursive calls in separable nonlinear least squares algorithms This

algorithm is used to solve the chemical equilibrium analysis problems we currently are working with

The size of the system is quite large only the Matlab co de consists of more than mles and more

than lines of co de and it is rapidly growing This motivates a welldened naming convention

and design In the following we describ e this design which is fully developed in NLPLIB TB and

partly for in OPERA TB

TOMLAB solves a number of dierent types of optimization problems Currently we have dened

the types listed in Table The global variable probType is the current type to b e solved An

optimization solver is dened to b e of type solvType where solvType is any of the probType entries

in Table It is clear that a solver of a certain solvType is able to solve a problem dened to b e

of another type For example a constrained nonlinear programming solver should in principle b e

able to solve nearly all types of problems dened

Dene probSet to b e a set of dened optimization problems to b e solved Each probSet b elongs to a

certain class of optimization problems of type probType In Table the currently dened problem

sets are listed and new sets are easily added The probSet usr is dened in order to make the

inclusion of a few optimization problems of any type a simple and fast task This metho d is to

prefer when TOMLAB is used in optimization courses

A owsheet of the pro cess of optimization in TOMLAB is shown in Figure Normally a single

optimization problem is solved running any of the menu systems one for each solvType or using

The TOMLAB Optimization Environment in Matlab

Table The dierent types of optimization problems treated in NLPLIB TB

probType Number Description of the type of problem

uc Unconstrained optimization incl b ound constraints

qp

con Constrained nonlinear optimization

ls Nonlinear least squares problems incl b ound constraints

exp Exp onential mo del tting problems

cls Constrained nonlinear least squares problems

nts Nonlinear time series

lp Linear programming

glb Boxbounded

glc Global mixedinteger nonlinear programming

Table Dened test problem sets in TOMLAB

probSet probType Description of test problem set

uc Unconstrained test problems

qp Quadratic programming test problems

con Constrained test problems

ls Nonlinear least squares test problems

exp Exp onential mo del tting problems

cls Linear constrained nonlinear least squares problems

nls Nonlinear constrained nonlinear least squares problems

glb Boxbounded global optimization test problems

glc Global MINLP test problems

mgh More Garb ow Hillstrom nonlinear least squares problems

amp AMPL test problems as les

cto CUTE constrained test problems as d l lles

ctl CUTE large constrained test problems as d l lles

uto CUTE unconstrained test problems as d l lles

utl CUTE large unconstrained test problems as d l lles

nts Nonlinear time series

usr User dened problems of probType

the Graphical User Interface GUI When several problems are to b e solved eg in algorithmic

development it is inecient to use an interactive system This is symbolized with the Advanced

User b ox in the gure which directly runs the Optimization Driver The Interface Routines in

Figure are used to convert computational results to the form exp ected by dierent solvers The

Gateway Routines do es the b o okkeeping keep track of search directions and determines type of

dierentiation analytic automatic or any of the dierent types of numerical dierentiation The

Gateway Routines used for nonlinear least squares also implements dierent types of weighting

schemes as well as separable nonlinear least squares algorithms

It was necessary to make a layer with Gateway Routines to make the co de of the Low Level Routines

clean enough to make automatic dierentiation p ossible

Not shown in Figure is the p ossibility to directly call the TOMLAB solvers if an input structure

is rst created Tools to easily create this structure is provided

A large set of Matlab mles are needed to implement the chain of function calls for all solver types

and problem sets For the menu systems driver routines etc Table shows the naming convention

used eg when probType con routines conOpt conRun conDef and conSolve are available

The TOMLAB Optimization Environment in Matlab

The names of the problem setup routine and the low level routines are constructed as two parts

The rst part b eing the abbreviation used in Table for the probSet The second part denoting the

computed task as shown in Table An example of the pro cess of optimization and the function

names for the constrained nonlinear programming case solvType con probSet con is shown

in Figure

Table Names of main mle functions in NLPLIB TB

Generic variable Purp ose solvType is eg con

Opt Menu program

Run Multisolver optimization driver routine

Def Routine dening optimization parameters

Solve Prototype solver

Table Names on the low level mles in NLPLIB TB

Generic name Purp ose is any probSet eg amp

prob Dene string matrix with problems and a structure prob for each

problem

f Compute ob jective function f x

g Compute the gradient vector g x

H Compute the Hessian matrix H x

c Compute the vector of constraint functions cx

dc Compute the matrix of constraint normals cxdx

dc Compute the nd part of nd derivative matrix of the Lagrangian

P

2 2

c xdx function

i i

i

r Compute the residual vector r x

J Compute the Jacobian matrix J x

P

2 2

r x r xdx dr Compute the nd part of the Hessian matrix

i i

i

The problem setup routine has two mo des of op eration either return a string matrix with the

names of the problems in the probSet or a structure with all information ab out the selected problem

The structure named Prob is shown in Table and its subtables We refer to the Users Guide

for a full description of all dierent elds Using a structure makes it easy to add new items

without to o many changes in the rest of the system The menu systems and the GUI are using the

string matrix for user selection of which problem to b e solved

There are default values for everything that is p ossible to set defaults for and all routines are

written in a way that makes it p ossible for the user to just set an input argument empty and

get the default Automatic dierentiation is easy using an interface to ADMATADMIT TB and

ve types of numerical dierentiation are included The elds AutoDi and NumDi in the Prob

structure controls the use of dierentiation metho d If routines for derivatives are nonpresent

numerical dierentiation is automatically used by the solvers More information ab out ADMAT

TB is found at the URL httpsimoncscornelleduhomevermaAD

The results of the optimization attempts are stored in a structure array named Result The currently

dened elds in the structure are shown in Table The use of structure arrays make advanced

result presentation and statistics p ossible

The eld xState describ es the state of each of the variables In Table the dierent values are

describ ed The dierent conditions for linear constraints are dened by the state variable in eld

bState In Table the dierent values are describ ed

The TOMLAB Optimization Environment in Matlab



User

 



Menu system GUI

Advanced User

 

Dene names of functions

Optimization Driver

Setup Problem

computing f x g x etc

z

MEXle interface

j

z

NLPLIBOPERA solver Optimization Toolb ox Solver

MEXle Solver

R

Interface Routines

U

nlp f nlp g nlp H nlp c nlp dc nlp dc nlp r nlp J nlp dr

NLPLIB Gateway routines

P

2 2

Compute f x g x H x cx cxdx c xdx

i i

i

P

2 2

Low Level Routines

r xdx r x For NLLS residual r x Jacobian J x

i i

i

Figure The pro cess of optimization in TOMLAB

The TOMLAB Optimization Environment in Matlab



User

 

Either choose predened problem

conOpt GUI



from menu or silent setup

Advanced User

 

Dene names of functions con f

con prob

conRun

g con H con c con dc con dc con

z

MEX minos npsol or npopt

j

z

conSolve NLPLIB constr OPTIM

MINOS NPSOL NPOPT

R

Interface nlp fc nlp gdc nlp fg nlp cdcS nlp cdc

Eg Gateway Routine Probp fnlp f computes fx

U

f x Prob Call f fevalProbp

NLPLIB Gateway Routines

In nlp f f fevalProbUSERf x Prob

Low Level Routines

where ProbUSERfcon f

Figure Solution of constrained nonlinear problems in TOMLAB

The TOMLAB Optimization Environment in Matlab

To conclude the system design is exible and easy to expand in many dierent ways

The NLPLIB Toolb ox

The NLPLIB TB is a set of Matlab mles which implements optimization solvers test prob

lems and utilities for nonlinear optimization and nonlinear parameter estimation The fo cus is on

dense problems There are eight main elds unconstrained and constrained optimization quadratic

programming b oxbounded global optimization global mixedinteger nonlinear programming un

constrained and constrained nonlinear least squares and exp onential sum mo del data tting There

are four ways to solve a predened problem dening a problem input structure and calling the

solver algorithm calling a multisolver driver routine using a menu system or running the Graphi

cal User Interface GUI NLPLIB TB is describ ed in more detail in Also see the Users Guide

for TOMLAB which is updated regularly and available online

Currently NLPLIB TB consists of ab out lines of mle co de in les with algorithms

utilities and predened problems Included are new algorithms for the nonlinear parameter esti

mation problem of tting sums of exp onential functions to empirical data The TOMLAB solvers

all explicitly handle b ounds and linear constraints with an input mo del upp erlower b ound format

like NPSOL

The routine ucSolve implements a prototype algorithm for unconstrained optimization with simple

b ounds on the parameters It includes several of the most p opular search step metho ds for uncon

strained optimization Bound constraints are treated as describ ed in Gill et al The search step

metho ds for unconstrained optimization are the Newton metho d the quasiNewton BFGS

and inverse BFGS metho d the quasiNewton DFP and inverse DFP metho d the FletcherReeves

and PolakRibiere conjugategradient metho d and the Fletcher conjugatedescent metho d For the

Newton and the quasiNewton metho ds the co de is using a subspace minimization technique to han

dle rank problems see Lindstrom The quasiNewton co des also use safe guarding techniques

to avoid rank problem in the up dated matrix

For general nonlinear problems with nonlinear constraints a sequential quadratic program

ming SQP metho d by Schittkowski is implemented in the routine conSolve as well as an

HanPowell SQP metho d The routine nlpSolve implements the Filter SQP by Fletcher and Leyer

Another constrained solver in NLPLIB TB is the structural trust region algorithm imple

mented in sTrustR combined with an initial trust region radius algorithm The co de is based on

the algorithms in and and treats partially separable functions The constraints should b e

convex for the routine to work

Quadratic programs qp are solved with a standard activeset metho d implemented in the

routine qpSolve

The prototype nonlinear least squares algorithm lsSolve treats problems with b ound constraints in

a similar way as the routine ucSolve The prototype routine lsSolve implements four search metho ds

for nonlinear least squares problems the GaussNewton metho d the AlBaaliFletcher and the

FletcherXu hybrid metho d and the Huschens TSSM metho d If rank problems o ccur

lsSolve uses subspace minimization

The constrained nonlinear least squares solver clsSolve is based on lsSolve and its search steps

metho ds Currently clsSolve treats linear equality and inequality constraints using an activeset

strategy

The line search algorithm LineSearch used by the solvers conSolve lsSolve clsSolve and ucSolve

is a mo died version of the algorithm by Fletcher chap The use of quadratic and cubic

interpolation is p ossible in the line search algorithm For more details see

There are three dierent routines for global optimization in TOMLAB all using only function val

ues The routine glbSolve implements an algorithm for b oxbounded global optimization glb

ie problems that have nite simple b ounds on all the variables glbSolve implements the DIRECT

algorithm which is a mo dication of the standard Lipschitzian approach that eliminates the

The TOMLAB Optimization Environment in Matlab

need to sp ecify a Lipschitz constant For global mixedinteger nonlinear programming glc

glcSolve implements an extended version of DIRECT see For global optimization problems

with exp ensive function evaluations the routine ego implements the Ecient Global Optimization

EGO algorithm

The OPERA Toolb ox

The Matlab to olb ox OPERA TB is a collection of Matlab mles which solves many basic linear and

discrete optimization problems in op erations research and mathematical programming Currently

OPERA TB consists of ab out lines of Matlab mle co de in les with algorithms and

utilities and example les OPERA TB was originally presented in and the current version

is presented in Also see the Users Guide for TOMLAB which is updated regularly and

available online

There are several algorithms implemented for linear programs LP the standard revised simplex

algorithm and two p olynomial algorithms The dual simplex metho d usable when a dual feasible

solution is available instead of a primal feasible is also implemented

For transp ortation problems the transp ortation simplex metho d is included together with three

simple algorithms to nd a starting basic feasible solution

The implementation of the network programming algorithms are based on the forward and

reverse star representation technique describ ed in Ahuja et al pages Implemented are

routines for network search and nding the minimal spanning tree There are three algorithms to

solve the shortest path problem an algorithm to solve maximum ow problems using the Ford

Fulkerson augmenting path metho d and a network simplex algorithm to solve the minimum cost

network ow problem The symmetric traveling salesman problem is solved using Lagrangian

relaxation and the subgradient metho d with the Polyak rule I I an algorithm by Held and Karp

To solve mixed linear integer programs a branch and b ound algorithm from Nemhauser and

Wolsey chap and a cutting plane algorithm using Gomory cuts are implemented Balas

metho d for integer programs restricted to integer co ecients is also implemented

Included in the to olb ox are also some examples of dynamic programming and Lagrangian relaxation

techniques For linear programs a menu system and a multisolver driver routine is implemented in

the same way as in NLPLIB TB

For linear and b oth classro om solvers as well as more robust solvers are

available

The MEXle Interface

TOMLAB is an op en system with p ossibilities to interact with other program packages An opti

mization solver implemented in Fortran or C is called from TOMLAB using a MEXle interface

MEXle interfaces for b oth Fortran and C are easy to develop for Unix systems Interfaces to

many solvers are available on Unix On PC machines there has b een problems to make Fortran

MEXle interfaces that work prop erly We have developed general MEXle interfaces in C and

converted solvers written in Fortran to C using the Fortran to C converter fc This solution

is wellworking and it should b e easy to expand the list of available solvers to TOMLAB

Presently MEXle interfaces have b een developed for six generalpurp ose solvers available from

the Systems Optimization Lab oratory Department of Op erations Research Stanford University

California NPSOL NPOPT up dated version of NPSOL NLSSOL QPOPT

LSSOL and LPOPT Furthermore an interface to MINOS has b een

developed MEXle interfaces are available for b oth Unix and PC systems

The TOMLAB Optimization Environment in Matlab

The Matlab Optimization Toolb ox Interface

Included in TOMLAB is an interface to several of the solvers in the Matlab Optimization Toolb ox

OPTIM The solvers callable are listed in Table The TOMLAB optimization driver routine

checks if the routine is in the path and then calls the Matlab function feval to evaluate it Two

lowlevel interface routines have b een written The constr solver needs b oth the ob jective function

and the vector of constraint functions in the same call which nlp fc supplies Also the gradient

vector and the matrix of constraint normals should b e supplied in one call These parameters are

gdc returned by the routine nlp

The CUTE Interface

The Constrained and Unconstrained Testing Environment CUTE is a wellknown software

environment for nonlinear programming The distribution of CUTE includes a test problem data

base of nearly optimization problems b oth academic and reallife applications This data base

is often used as a b enchmark test in the development of general optimization software Using the

TOMLAB CUTE interface it is p ossible to run the huge set of CUTE test problems in NLPLIB TB

using any solver callable from the to olb ox

CUTE stores the problems in the standard input format SIF in les with extension sif There are

to ols to select appropriate problems from the data base Running CUTE a SIF deco der creates up

to ve Fortran les elfuns extern groups ranges and settyp and one ASCI I data le outsdifdat

or outsdifd The Fortran les are compiled and linked together with the CUTE library and a solver

routine Running the binary executable the problem is solved using the current solver During the

solution pro cedure the ASCI I data le outsdifdat or outsdifd is read

With the CUTE distribution follows a Matlab interface There are one MEXle interface routine

ctoolsf for constrained CUTE problems and one MEXle interface routine utoolsf for uncon

strained problems The MEXle interface routine is compiled and linked together with the Fortran

les generated by the SIF deco der and the Matlab MEX library to make a DLL Dynamic Link

Library le At runtime Matlab calls the DLL which will read the CUTE ASCI I data le for the

problem sp ecic information Also included in the CUTE distribution is a set of Matlab lowlevel

mle interface routines that calls the appropriate MEXle interface routine

For the TOMLAB CUTE interface we assume that the DLL les are already built and stored in

predened directories The name of the dll is the problem name used by CUTE eg rosenbrdl l

for the Rosenbrock banana function The ASCI I data le also has a unique name eg rosenbrdat

The CUTE Matlab lowlevel mle interface routines assumes the DLL le to b e named cto olsdll

and uto olsdll and the data le to b e called outsdifdat on PC TOMLAB calls the Matlab les

in the CUTE distribution but to solve the name conict uses sp ecial mles ctoolsm and utoolsm

to make a call to the correct DLL le The ASCI I data le must also b e copied to a temp orary le

with the necessary lename outsdifdat b efore executing the DLL

When using the TOMLAB interface the user either gets a menu of all DLL les in the CUTE

directory chosen or directly makes a choice of which problem to solve Precompiled DLL les for

the CUTE data set is available or the necessary les for the user to build his own DLL les

The AMPL Interface

Using interfaces b etween a mo deling language and TOMLAB could b e of great b enet and improve

the p ossibilities for analysis on a given problem As a rst attempt a TOMLAB interface to the

mo deling language AMPL was built The reason to choose AMPL was that it has a rudimentary

Matlab interface written in C that could easily b e used

AMPL is using ASCI I les to dene a problem The naming convention is to use the problem

name and various extensions eg rosenbrmod and rosenbrdat for the Rosenbrock banana function

The TOMLAB Optimization Environment in Matlab

These les are normally converted to binary les with the extension nl called nlles eg rosenbrnl

After solving the problem the solver creates a le with extension eg rosenbrsol containing a

termination message and the solution it has found

The current TOMLAB AMPL interface is an interface to the problems dened in the AMPL nl

format When using the TOMLAB interface the user either gets a menu of the nlles found or

directly makes a choice of which problem to solve The initialization routine in TOMLAB for AMPL

prob either calls amplfunc or spamfunc the two MEXle interface routines written problems amp

by Gay The TOMLAB low level routines amp f amp g etc calls the same MEXle interface

routines and dep endent on the parameters in the call the appropriate information is returned

Note that the design of the AMPL solver interface makes it easy to run the NLPLIB TB solvers

from AMPL using the Matlab Engine interface routines a p ossible extension in the future But

indeed any solver callable from NLPLIB TB may now solve problems formulated in the AMPL

language

Applications of TOMLAB

TOMLAB has already b een used in a wide range of applied mathematical pro jects Here we give a

few examples of its successful use

To nd unknown sp ecies formation constants and other unknown parameters in multiphase chem

ical equilibrium analysis we have formulated a separable nonlinear least squares algorithm The

separation aims at using the structure of a large illconditioned parameter estimation problem to

obtain a solvable set of optimization problems Each iteration in the separable algorithm consists

of a ma jor problem and the solution of hundreds or thousands of minor illconditioned problems

To obtain a practically useful to ol a combination of approximation algorithms to nd initial values

for the unknown parameters is needed together with robust constrained nonlinear least squares

solvers As we are using the weighted L norm sensitive to outliers all the minor problems

2

must b e solved with high accuracy and no failures Here the TOMLAB constrained nonlinear least

squares solver clsSolve is used in our new Matlab to olb ox LAKE TB Results so far are excellent

and clsSolve converges faster and gives b etter accuracy than the previous solver that is part of our

Fortran program package LAKE used in inorganic chemical research for more than ten years

Preliminary results are discussed in As TOMLAB handles recursive calls in a prop er way it

is p ossible to use clsSolve for b oth the ma jor and minor optimization problems

In a joint pro ject with Prof Jordan M Berg Texas Tech University Lubb o ck TX we want to nd

accurate loworder phenomenological mo dels of thin lm etching and dep osition pro cesses These

pro cesses are central to the manufacture of microelectronic devices We have developed algorithms

and software for parameter estimation using level sets ie the problem of selecting the member of

a parameterized family of curves that b est matches a given curve Level set metho ds oer several

attractive features for treating such problems The metho d is completely geometric there is no

need to introduce an arbitrary co ordinate system for the curves We have derived analytic results

necessary for the application of gradient descent algorithms and made numerical computations

using TOMLAB

In our research on prediction metho ds in computational nance we study the prediction of various

kinds of quantities related to sto ck markets like sto ck prices sto ck volatility and ranking measures

In one pro ject we instead of the classical time series approach used the more realistic prediction

problem of building a multistock articial trader ASTA The b ehavior of the trader is controlled

by a parameter vector which is tuned for b est p erformance The global optimization routine glbSolve

is used to nd the optimal parameters for the noisy functions obtained when running on a large

of Swedish sto ck market data

TOMLAB has also b een an essential to ol in the applied pro jects researched in the licentiate thesis

of my graduate students Erik Dotzauer Algorithms for ShortTerm Pro duction Planning of Cogen

eration Plants Joran Petersson Algorithms for Fitting Two Classes of Exp onential Sums to

Data and Thomas Hellstrom A Randow Walk through the Sto ck Market

The TOMLAB Optimization Environment in Matlab

Conclusions and Future work

TOMLAB is an op en and general optimization development environment in Matlab Many internal

algorithms are implemented in the two to olb oxes NLPLIB TB and OPERA TB as well as interfaces

to other algorithms in Matlab Fortran and C It is a exible to ol with menu programs a graphical

user interface and multisolver driver routines for many types of optimization problems

A large set of optimization problems are included as Matlab mle co de It is easy for the user to

co de new problems using the predened les as a mo del TOMLAB may also run problems dened

in the CUTE SIF language using DLL les or in the AMPL language after making nlles

TOMLAB is suitable for computer based learning in optimization courses b oth algorithmic and

applied and in computer exercises

TOMLAB is a p owerful to ol b oth in the development of generalpurp ose optimization software and

in the solution of applied optimization problems

Future plans include extension of TOMLAB to sparse optimization problems interface to the new

routines in MathWorks Optimization Toolb ox MEXle interfaces to more solvers like CFSQP

and interfaces to the commercial mo deling systems GAMS and XPRESSMP

Acknowledgements

I would like to thank Prob Philip E Gill for giving me access to the commercial co des NPSOL

NPOPT NLSSOL LPOPT QPOPT and LSSOL I am grateful to Prof Michael Saunders for

sending me the MINOS distribution and his continuous supp ort of the TOMLAB development

Prof David M Gay help ed making the AMPL interface work I would also thank Prof Yangquan

Chen for a lot of help and hard work in making the CUTE interface run on PC systems Prof Anders

Forsgren has provided Fortran MEXle interfaces which has b een the source of inspiration for the

development of the C MEXle interfaces Prof Arthur Jutan has given several go o d suggestions

concerning numerical dierentiation Dr Arun Verma has kindly supplied improved versions of

his ADMAT TB to cop e with the trouble of automatic dierentiation for the dierent TOMLAB

problems Many others around the world has given us interesting suggestions found bugs and

discussed the system and we hereby thank all p eople that have contributed

All our eorts on global optimization have only b een p ossible b ecause of Dr Donald R Jones

which has supplied us with preprints and pap ers and advice on implementation issues

My graduate students Mattias Bjorkman Erik Dotzauer and Joran Petersson have made many

imp ortant contributions to TOMLAB Many students have also contributed with bits and pieces

which I hereby acknowledge

References

LINGO The and Optimizer LINDO Systems Inc Chicago IL

R K Ahuja T L Magnanti and J B Orlin Network Flows Theory Algorithms and

Applications PrenticeHall Inc Kanpur and Cambridge

M AlBaali and R Fletcher Variational methods for nonlinear least squares J Op er

Res So c pp

J M Berg and K Holmstrom On Parameter Estimation Using Level Sets SIAM Journal

on Control and Optimization To b e published

J Bisschop and R Entriken AIMMS The Modeling System Paragon Decision Technol

ogy Haarlem The Netherlands

The TOMLAB Optimization Environment in Matlab

J Bisschop and A Meeraus On the development of a general algebraic modeling system

in a strategic planning environment Mathematical Programming Study pp

I Bongartz A R Conn N Gould and P L Toint CUTE Constrained and Un

constrained Testing Environment tech rep IBM T J Watson Research Center Yorktown

Heights NY September

I Bongartz A R Conn N I M Gould and P L Toint CUTE Constrained and

Unconstrained Testing Environment ACM Transactions on Mathematical Software

pp

M A Branch and A Grace Optimization Toolbox Users Guide Prime Park Way

Natick MA

A Brooke D Kendrick and A Meeraus GAMS A Users Guide The Scientic Press

Redwoo d City CA

A R Conn N Gould A Sartenaer and P L Toint Convergence properties of

minimization algorithms for convex constraints using a structured trust region SIAM Journal

on Scientic and Statistical Computing pp

J J Dongarra C B Moler J R Bunch and G W Stewart LINPACK Users

Guide SIAM

E Dotzauer Algorithms for ShortTerm ProductionPlanning of Cogeneration Plants Li

centiate Thesis No ISBN ISSN Division of Optimization De

partment of Mathematics Linkoping University Linkoping Sweden

E Dotzauer and K Holmstrom A Graphical User Interface for Nonlinear Programming

in Matlab Annals of Op erations Research Mo deling Languages and Approaches Sub

mitted

The TOMLAB Graphical User Interface for Nonlinear Programming Advanced Mo del

ing and Optimization

A S Drud Interactions between nonlinear programing and modeling systems Mathematical

Programming Series B pp

S I Feldman D M Gay M W Maimone and N L Schryer A FortrantoC

converter Tech Rep Computing Science Technical Rep ort No ATT Bell Lab oratories

May

R Fletcher Practical Methods of Optimization John Wiley and Sons New York nd ed

R Fletcher and S Leyffer Nonlinear programming without a penalty function Tech

Rep NA University of Dundee September

R Fletcher and C Xu Hybrid methods for nonlinear least squares IMA Journal of Nu

merical Analysis pp

R Fourer D M Gay and B WKernighan AMPL A Modeling Language for Math

ematical Programming The Scientic Press Redwoo d City CA

B S Garbow J M Boyle J J Dongara and C B Moler Matrix Eigensystem

RoutinesEISPACK Guide Extension in Lecture Notes in Computer Science Springer Verlag

New York

D M Gay Hooking your solver to AMPL technical rep ort Bell Lab oratories Lucent Tech

nologies Murray Hill NJ

The TOMLAB Optimization Environment in Matlab

P E Gill W Murray M A Saunders and M H Wright Users Guide for NPSOL

Version A Fortran package for nonlinear programming Department of Op erations Re

search Stanford University Stanford CA SOL

P E Gill W Murray and M H Wright Practical Optimization Academic Press

London

M Held and R M Karp The TravelingSalesman problem and minimum spanning trees

Part II Mathematical Programming pp

T Hellstrom A Random Walk through the Stock Market Licentiate Thesis ISSN

UMINF Department of Computing Science Umea University Sweden December

T Hellstrom and K Holmstrom Parameter Tuning in Trading Algorithms using ASTA

in Computational Finance Pro ceedings of the Sixth International Conference Leonard N

Stern School of Business January Y S AbuMostafa B LeBaron A W Lo and A S

Weigend eds Cambridge MA MIT Press

K Holmstrom NLPLIB TB A Matlab Toolbox for Nonlinear Optimization and Pa

rameter Estimation Technical Rep ort IMaTOM Department of Mathematics and

Physics Malardalen University Sweden

OPERA TB A Matlab Toolbox for Optimization Algorithms in Operations Research

Technical Rep ort IMaTOM Department of Mathematics and Physics Malardalen

University Sweden

TOMLAB A General Purpose Open Matlab Environment for Research and Teaching in

Optimization Technical Rep ort IMaTOM Department of Mathematics and Physics

Malardalen University Sweden Presented at the th International Symp osium on

Mathematical Programming Lausanne Switzerland August

TOMLAB An Environment for Solving Optimization Problems in Matlab in Pro ceed

ings for the Nordic Matlab Conference Octob er M Olsson ed Sto ckholm Sweden

Computer Solutions Europ e AB

Constrained Separable NLLS Algorithms for Chemical Equilibrium Analysis in Pro ceed

ings from the th Meeting of the Nordic Section of the Mathematical Programming So ciety

A Lkketangen ed ISBN Molde Division of Op erations Research Molde

University

K Holmstrom and M Bjorkman The TOMLAB NLPLIB Toolbox for Nonlinear Pro

gramming Advanced Mo deling and Optimization pp

K Holmstrom M Bjorkman and E Dotzauer The TOMLAB OPERA Toolbox for

Linear and Discrete Optimization Advanced Mo deling and Optimization

TOMLAB v Users Guide Technical Rep ort IMaTOM Department of

Mathematics and Physics Malardalen University Sweden

J Huschens On the use of product structure in secant methods for nonlinear least squares

problems SIAM Journal on Optimization pp

N Ingri I Andersson L Pettersson L Andersson A Yagasaki and K Holm

strom LAKE A Program System for Equilibrium Analytical Treatment of Multimethod Data

Especially Combined Potentiometric and NMR Data Acta ChemScand pp

K Iverson A Programming Language John Wiley and Sons New York

D R Jones DIRECT Encyclop edia of Optimization To b e published

The TOMLAB Optimization Environment in Matlab

D R Jones C D Perttunen and B E Stuckman Lipschitzian optimization without

the Lipschitz constant Journal of Optimization Theory and Applications pp

D R Jones M Schonlau and W J Welch Ecient global optimization of expensive

BlackBox functions Journal of Global Optimization pp

P Lindstrom Algorithms for Nonlinear Least Squares Particularly Problems with Con

straints PhD thesis Inst of Information Pro cessing University of Umea Sweden

D G Luenberger Linear and Nonlinear Programming AddisonWesley Publishing Com

pany Reading Massachusetts nd ed

C B Moler MATLAB An Interactive Matrix Laboratory Technical Rep ort Depart

ment of Mathematics and Statistics University of New Mexico

B A Murtagh and M A Saunders MINOS USERS GUIDE Technical Rep ort

SOL R Revised Feb Systems Optimization Lab oratory Department of Op erations

Research Stanford University Stanford California

G L Nemhauser and L A Wolsey Integer programming in Optimization G L

Nemhauser A H G R Kan and M J Todd eds vol of Handb o oks in Op erations

Research and Management Science ElsevierNorth Holland Amsterdam The Netherlands

J Petersson Algorithms for Fitting Two Classes of Exponential Sums to Empirical Data Li

centiate Thesis ISSN Opuscula ISRN HEVBIBOPSE Division of Optimiza

tion and Systems Theory Royal Institute of Technology Sto ckholm Malardalen University

Sweden December

P C Piela T G Epperly K M Westerberg and A W Westerberg ASCEND

An objectoriented computer environment for modeling and analysis The modeling language

Computers and Chemical Engineering pp

R P Polivka and S Pakin APL The Language and Its Usage Prentice Hall Englewoo d

Clis N J

A Sartenaer Automatic determination of an initial trust region in nonlinear programming

Technical Rep ort Department of Mathematics Facultes Universitaires ND de la Paix

Bruxelles Belgium

K Schittkowski On the Convergence of a Sequential Quadratic Programming Method with

an Augmented Lagrangian Line Search Function technical rep ort Systems Optimization lab

oratory Stanford University Stanford CA

B T Smith J M Boyle J J Dongarra B S Garbow Y Ikebe V C Klema

and C B Moler Matrix Eigensystem Routines EISPACK Guide Lecture Notes in Com

puter Science SpringerVerlag New York nd ed

The TOMLAB Optimization Environment in Matlab

Table Information stored in the problem structure Prob

Field Description

Name Problem name

P Problem number

probType TOMLAB problem type see Table

probFile Name of mle in which problem are dened

xName Name of each decision variable

cName Name of each general constraint

optParam Structure with sp ecial elds for optimization parameters see Table

Solver Structure with elds Name and Alg Name is the name of the solver and Alg

is the solver algorithm to b e used

uP User supplied parameters for the problem

uPName Problem name connected to the user supplied parameters

ExpFit Structure with sp ecial elds for exp onential mo del tting problems see Table

QP Structure with sp ecial elds for quadratic problems see Table

NLLS Structure with sp ecial elds for nonlinear least squares see Table

NTS Structure with sp ecial elds for nonlinear time series see Table

PartSep Structure with sp ecial elds for partially separable functions see Table

GLOBAL Structure with sp ecial elds for global optimization see Table

A Constraint matrix for linear constraints one constraint p er row

b L Lower b ounds on the linear constraints

U Upp er b ounds on the linear constraints b

c L Lower b ounds on the general constraints

U Upp er b ounds on the general constraints c

x L Lower b ounds on the variables

x U Upp er b ounds on the variables

Starting p oint x

N Problem dimension number of variables

f Low Lower b ound on function value Used in line search by Fletcher default

r eal max E



opt Optimal p oint x if known x



opt Optimal ob jective function value f x f

AutoDi If true use automatic dierentiation

NumDi Numerical approximation of derivatives If set to classical approach with

forward or backward dierences together with automatic step selection will b e

used If set to or the spline routines csapi csaps or spaps in SPLINE

Toolb ox will b e used If set to derivatives will b e estimated by use of

complex variables

p f Name of Gateway routine computing the ob jective function f x

p g Name of Gateway routine computing the gradient vector g x

H Name of Gateway routine computing the Hessian matrix H x p

p c Name of Gateway routine computing the vector of constraint functions cx

p dc Name of Gateway routine computing the matrix of constraint normals

cxdx

dc Name of Gateway routine computing the nd part of nd derivative matrix of p

P

2 2

cxdx the Lagrangian function

i

i

r Name of Gateway routine computing the residual vector r x p

p J Name of Gateway routine computing the Jacobian matrix J x

p dr Name of Gateway routine computing the second part of the Hessian for a

m

2

P

r (x)

i

r x nonlinear least squares problem ie

i

x x

j

k

i=1

USER Structure with user dened names of the mles computing the ob jective

gradient Hessian etc See Table These routines are called from the corre

sp onding Gateway Routine

x min Lower plot region parameters

x max Upp er plot region parameters

The TOMLAB Optimization Environment in Matlab

Table Information stored in the structure ProboptParam

Field Description

alg Optimization Algorithm Dep endent on type of problem Default

method Solver submetho d technique Default

PriLev Print level in optimization solver default

x Convergence tolerance in optimal solution x distance b etween suc eps

8

cessive x kx x k default

k +1 k

f Convergence tolerance on f Also used when testing on the directed eps

8

derivative default

8

eps dirg Convergence tolerance on the directed derivative default

6

eps c Constraint violation convergence tolerance default

LineAlg Line search algorithm quadratic interpolation cubic inter

p olation Default LineAl g

GradCheck Set to if you want to check usersupplied gradients default

MaxIter Maximum number of iterations default

DiGradMinChange Minimum change in variables for nite dierence gradients default

8

DiGradMaxChange Maximum change in variables for nite dierence gradients default

InitStepLength Initial step length default or less

6

g Gradient or reduced gradient convergence tolerance default eps

10

eps Rank Rank test tolerance default

wait Flag if to use pause statements after output default

eps absf Convergence tolerance on absolute function value default r eal min

PreSolve Flag if presolve analysis is to b e applied on linear constraints default

InitMatrix Initial matrix for QuasiNewton may b e set by the user When QN

QN InitMatrix is empty the identity matrix is used

LineSearch Structure with sp ecial elds for the line search see Table

Penalty Penalty parameter for constrained problems

L x L bT ol or x U bT ol x U x x on b ound xTol If x x

bTol Feasibility tolerance for linear constraints

cTol Feasibility tolerance for nonlinear constraints

09

fTol Accuracy in the computation of the function value default eps

size x Size at optimum for the variables x used in the convergence tests

Default

f Size at optimum for the function f used in the convergence tests size

Default

size c Size at optimum for the constraints c used in the convergence tests

Default

LowIts Number of iterations with low reduction b efore convergence

release al l Set to if not to release more than one variable at the time NOT

subalg Optimization sub algorithm Dep endent on type of problem Default

splineSmooth Smo othness parameter sent to the SPLINE Toolb ox routine csapsm

when computing numerical approximations of the gradient and the

Jacobian Default

splineTol Tolerance parameter sent to the SPLINE Toolb ox routine spapsm

when computing numerical approximations of the gradient and the

3

Jacobian Default

The TOMLAB Optimization Environment in Matlab

Table Information stored in the structure ProbExpFit

Field Description

p Number of exp onential terms default

wType Weighting type default

eType Type of exp onential terms default

infCR Information criteria for selection of b est number of terms default

dType Dierentiation formula default

geoType Type of equation default

qType Length q of partial sums default

p

sigType Sign to use in P QD in exp geo for p default

lambda Vector of dimension p intensities

alpha Vector of dimension p weights

xType Type of starting value algorithm

sumType Type of exp onential sum

eqdist Flag if data is equidistant in time t

Table Information stored in the structure ProbQP

Field Description

F Constant matrix the Hessian

c Constant vector

B Logical vector of the same length as the number of variables A one

corresp onds to a variable in the basis

Table Information stored in the structure ProbNLLS

Field Description

weightType Weighting type

No weighting

Weight with data in Yt If Y t the weighting is ie

deleting this residual element

Weight with weight vector or matrix in weightY If weightY is

a vector then weighting by w eig thY r elementwise multipli

cation If weightY is a matrix then weighting by w eig thY r

matrix multiplication

nlp r calls the routine weigthtY must b e a string with the

routine name to compute the residuals

weightY Either empty a vector a matrix or a string see weightType

t Time vector t

Yt Matrix with observations Y t

UseYt If U seY t compute residual as f x t Y t default other

wise Y t should b e treated separately and the residual routines just

return f x t

SepAlg If S epAl g use separable non linear least squares formulation

default

The TOMLAB Optimization Environment in Matlab

Table Information stored in the structure ProbNTS

Field Description

SepAlg If S epAl g use separable non linear least squares formulation

default

ntsModel Nonlinear mo del number

p The number of terms lags in the mo del

pL The number of nonlinear parameters

pA The number of linear parameters

ntsSeed Reset number for random generator or Time series number

N Total number of data p oints

t The starting p oint for the estimation

tN The end p oint for the estimation

gamma Exp onential weighting factor default

lambdaArt Nonlinear parameters used to create the articial data

alphaArt Linear parameters used to create the articial data

lambda Exp onential parameters in autoregressive mo dels

alpha Weights in autoregressive mo dels

Table Information stored in the structure ProbPartSep

Field Description

pSepFunc Number of partially separable functions

index Index for the partially separable function to compute ie if i

index compute f x If index compute the sum of all ie

i

M

P

f x f x

i

i=1

The TOMLAB Optimization Environment in Matlab

Table Information stored in the structure ProbGLOBAL

Field Description

iterations Number of iterations default

MaxEval Number of function evaluations default

Integers Set of integer variables

4

epsilon Globallo cal weight parameter default

K The Lipschitz constant Not used

tolerance Error tolerance parameter Not used

C Matrix with all rectangle centerpoints

D Vector with distances from centerpoint to the vertices

L Matrix with all rectangle side lengths in each dimension

F Vector with function values

d Row vector of all dierent distances sorted

d min Row vector of minimum function value for each distance

Split S pl iti j is the number of splits along dimension i of rectangle j

T T i is the number of times rectangle i has b een trisected

G Matrix with constraint values for each p oint

ignoreidx Rectangles to b e ignored in the rectangle selection pro cedure

L I Li j is the lower b ound for rectangle j in integer dimension I i I

I U I U i j is the upp er b ound for rectangle j in integer dimension I i

feasible Flag indicating if a feasible p oint has b een found

min Best function value found at a feasible p oint f

s s is used as s

s sj is the sum of observed rates of change for constraint j

t ti is the total number of splits along dimension i

Table Information stored in the structure ProbUSER

Field Description

f Name of mle computing the ob jective function f x

g Name of mle computing the gradient vector g x If ProbUSERg

is empty then numerical derivatives will b e used

H Name of mle computing the Hessian matrix H x

c Name of mle computing the vector of constraint functions cx

dc Name of mle computing the matrix of constraint normals cxdx

dc Name of mle computing the nd part of nd derivative matrix of

P

2 2

cxdx the Lagrangian function

i

i

r Name of mle computing the residual vector r x

J Name of mle computing the Jacobian matrix J x

dr Name of mle computing the nd part of the Hessian for nonlinear

m

2

P

r (x)

i

r x least squares problem ie

i

x x

j

k

i=1

The TOMLAB Optimization Environment in Matlab

Table Information stored in the structure ProboptParamLineSearch

Field Description

sigma Line search accuracy sig ma sig ma inaccurate line

search sig ma accurate line search default

rho Determines the line default

tau Determines how fast step grows in phase default

tau How near end p oint of a b default

tau Choice in a b phase default

7

eps Minimal length for interval a b default

12

eps Minimal reduction default

MaxIter Maximum number of line search iterations

The TOMLAB Optimization Environment in Matlab

Table Information stored in the global Matlab structure Result

Field Description

Iter Number of ma jor iterations

MinorIter Number of minor iterations

ExitFlag if convergence to lo cal min Otherwise errors

Inform Information parameter type of convergence

f k Function value at optimum

g k Gradient value at optimum

H k Hessian value at optimum

k QuasiNewton approximation of the Hessian at optimum B

x Starting p oint

Function value at start ie f x f

x k Optimal p oint

v k Lagrange multipliers

k Residual at optimum r

J k Jacobian matrix at optimum

c k Value of constraints at optimum

cJac Constraint Jacobian at optimum

xState State of each variable describ ed in Table

bState State of each linear constraint describ ed in Table

cState State of each general constraint

optParam Structure with sp ecial elds for optimization parameters see Table

Name Problem name

P Problem number

dx Matrix where each column is a search direction p

alphaV Matrix where row i stores the steplengths tried for the ith iteration

x min Lowest xvalues in optimization Used for plotting

x max Highest xvalues in optimization Used for plotting

X F X is a global matrix with rows iter no fx F

GLOBAL Structure with sp ecial elds for global optimization see Table

SepNLLS General result variable with elds z and Jz Used when running sepa

rable nonlinear least squares problems

Solver Solver used

SolverAlgorithm Solver algorithm used

CPUtime CPU time used

REALtime Real time elapsed

Nops Number of oating p oint op erations

probType TOMLAB problem type

solvType TOMLAB solver type

FuncEv Number of function evaluations needed

GradEv Number of gradient evaluations needed

ConstrEv Number of constraint evaluations needed

ResEv Number of residual evaluations needed

JacEv Number of Jacobian evaluations needed

Prob Problem structure see Table

plotData Structure with plotting parameters

The TOMLAB Optimization Environment in Matlab

Table The state variable xState for the variable

Value Description

A free variable

Variable on lower b ound

Variable on upp er b ound

Variable is xed lower b ound is equal to upp er b ound

Table The state variable bState for each linear constraint

Value Description

Inactive constraint

Linear constraint on lower b ound

Linear constraint on upp er b ound

Linear equality constraint

Table Information stored in the structure ResultGLOBAL

Field Description

C Matrix with all rectangle centerpoints in original co ordinates

D Vector with distances from centerpoint to the vertices

L Matrix with all rectangle side lengths in each dimension

F Vector with function values

d Row vector of all dierent distances sorted

min Row vector of minimum function value for each distance d

Split S pl iti j is the number of splits along dimension i of rectangle j

T T i is the number of times rectangle i has b een trisected

G Matrix with constraint values for each p oint

ignoreidx Rectangles to b e ignored in the rectangle selection pro cedure

I L I Li j is the lower b ound for rectangle j in integer dimension I i

I U I U i j is the upp er b ound for rectangle j in integer dimension I i

feasible Flag indicating if a feasible p oint has b een found

min Best function value found at a feasible p oint f

s s is used as s

s sj is the sum of observed rates of change for constraint j

t ti is the total number of splits along dimension i

Table Matlab Optimization to olb ox routines with a TOMLAB interface

Function Type of problem solved

constr Constrained minimization

leastsq Nonlinear least squares

fmins Unconstrained minimization using NelderMead

type simplex search metho d

fminu Unconstrained minimization using gradient

search

lp Linear programming

qp Quadratic programming