The Linear Convergence of a Successive Linear



Programming

y z

Michael C Ferris Sergei K Zavriev

December

Abstract

We present a successive algorithm for solving

constrained nonlinear optimization problems The algorithm employs

an Armijo pro cedure for up dating a radius We prove the

linear convergence of the metho d by relating the solutions of our sub

problems to standard trust region and gradient pro jection subproblems

and adapting an error b ound analysis due to Luo and Tseng Com

putational results are provided for p olyhedrally constrained nonlinear

programs

Introduction

In this pap er we develop a rst order technique for constrained optimization

problems based on subproblems with a linear ob jective function There have

b een many attempts to generate robust successive linear programming al

gorithms for see for example

This pap er sp ecies a new algorithm that has the same convergence

prop erties as the gradient pro jection metho d We mo dify the conditional

gradient metho d sometimes called the FrankWolfe metho d for

p olyhedrally constrained optimization problems to obtain a linearly con

vergent algorithm The only known linear rate of convergence is given in

but this is for problems where the constraints are not p olyhedral

but have p ositive curvature there are even simple examples p

This work is partially supp orted by the National Science Foundation under grant

CCR

y

Computer Sciences Department University of Wisconsin Madison WI

z

Op erations Research Department Faculty of Computational Mathematics and Cy

b ernetics Moscow State University Moscow Russia

that show the conditional gradient metho d has a sublinear rate of conver

gence

The main idea is to use a p olyhedral trust region constraint in the stan

dard linear programming subproblem of the conditional gradient metho d

This maintains the attractive computational feature that all subproblems are

linear programs for which a wealth of highly eective and ecient software

has b een developed It also guarantees that every subproblem we generate

has an optimal solution We develop an adaptive mechanism to up date the

trust region parameter that guarantees the linear convergence of the metho d

under the standard assumptions given by Luo and Tseng

Our analysis is somewhat involved and relies up on estimating descent

prop erties of the directions generated by our algorithm in terms of standard

quadratic subproblems arising for example in gradient pro jection

To our knowledge every linearly convergent technique for such problems is

based on such quadratic subproblems The main contribution of this pap er

is that the solutions of our linear subproblems are considered as approxi

mate solutions of the quadratic subproblems and analysis is carried out to

precisely quantify this relationship

We will consider the following optimization problem

P inf f x

xX

n

where X R is a nonempty closed convex set and the ob jective function f

is assumed to b e continuously dierentiable Furthermore it will b e assumed

that f has a Lipschitz continuous gradient rf on X that is

x y X krf x rf y k L kx y k

n

Here L is a p ositive scalar and kk represents the Euclidean norm on R

We also supp ose that

f inf f x

opt

xX

and the optimal solution set for P is not empty

Let N x denote the normal cone to X at x X The stationary set

X

for the problem P is the set of all p oints satisfying the rst order optimality

conditions for P that is

X fx X j rf x N xg

stat X

fx X j y X hrf x y xi g

Our algorithm will converge under standard assumptions to a stationary

p oint Conditions relating stationary p oints to solutions of P are well known

Our metho d resembles a trust region metho d see by virtue of the

fact that the subproblems are of the form

min hrf x hi

h

LP x r

sub ject to x h X

khk r

n

where kk represents an arbitrary norm on R and r is the trust region

radius We actually prove convergence of our algorithm based on X b eing

a convex set and the trust region constraint b eing determined by an arbi

trary norm When sp ecialized to the case where X p olyhedral and kk is

a p olyhedral norm the subproblems LP x r are linear programs and our

algorithm is a successive linear programming algorithm

There are two key approximations in our analysis The rst is that we

relate solutions of our linear programming descent problem LP x r to the

following subproblem

min hrf x hi khk

h

a

QP x a

sub ject to x h X

where a is a corresp onding stepsize When X is p olyhedral QP x a is

a problem It is imp ortant that we guarantee in

our algorithm that the trust region radius r is not to o small We therefore

employ a sp ecial Armijo type pro cedure for choosing r at each iteration The

essential diculty in the pro of relates to nding an expression for the inverse

ax r of r x a and this is explored in Section The key relationship is

b etween solutions of LP x r and QP x a app ears as Lemma

Usually rapid convergence techniques ie at least a linear rate for

solving P require the solution of a subproblem with a quadratic ob jective

function at each iteration k For example the gradient pro jection

algorithm solves subproblems of the form QP x a The second piece of

our analysis given as Theorem in Section of this pap er relates the

solutions of QP x r to solutions of the following standard trust region

problem

min hrf x hi

h

TP x r

sub ject to x h X

khk r

Note that the ob jective function of this problem is linear and that the trust

region constraint is dened in terms of the Euclidean norm

In Section we state the general form of the metho d which we call the

Sequential Linearization Metho d and prove its linear convergence to the

stationary set X in Section The sp ecial case of p olyhedral constraints

stat

is considered in Section The resulting algorithm is a linearly convergent

sequential linear programming SLP technique for which we also provide

some numerical results using Matlab Cplex and the Cute suite of problems

Technical Preliminaries

For every x X consider the following auxiliary problem

min hrf x hi

h

TP x r

sub ject to x h X

khk r

where r is a parameter of the problem Let Hx r and v x r denote

the optimal set and the optimal value of TP x r resp ectively We shall

relate this to the following problem

min hrf x hi

h

TP x

sub ject to x h X

Note that it is p ossible for v x and Hx

The standard optimality conditions see for the problem TP x r

provide the following lemma

n

Lemma i For every x X r and h R a and b are

equivalent

a h Hx r

b there exists such that

rf x h N x h

X

x h X

khk r khk r

n

ii For every x X and h R a and b are equivalent

a h Hx

b rf x N x h x h X

X

The following corollary is immediate from Lemma

Corollary For every x X and r the fol lowing assertion holds

x X v x r

stat

n

For a subset S R let x denote the set of all orthogonal pro jections

S

of x on S that is

x fy S j kx y k dist x S g

S

Here dist x S represents the distance from x to S

dist x S inf kx y k

y S

Let x b e a typical element of x it is clear that if S is closed and

S S

convex then

x f xg

S S

For x X and a dene

p x a x arf x x

X

Clearly for a p x a is the unique solution of the following problem

khk min hrf x hi

h

a

QP x a

sub ject to x h X

The following two simple results prove to b e useful in the sequel

Lemma Let x X and a be given The fol lowing assertions

hold

i h p x a if and only if

h N x h x h X rf x

X

a

ii hrf x p x ai kp x ak

a

n

Corollary Let h R be a limit point of a sequence fp x a g a

k k

then h Hx

We now dene a residual function r x R R by

r x a kp x ak

where R fx R j x g We collect some well known prop erties of

the function r x R R in the following lemma The pro ofs can b e

found in

Lemma For every x X the fol lowing properties hold

i r x C R

ii a a r x a r x a

r xa r xa

iii a a

a a

iv a r x a x X

stat

The following lemma b ounds the residual in terms of any solution of

TP x r

Lemma Let x X and suppose Hx Then for every h

Hx and every a

r x a kp x ak khk

Pro of The result is trivial for a Otherwise let us x arbitrary x X

h Hx and a Supp ose that

khk kp x ak

By the denition of Hx we have

hrf x hi hrf x p x ai

Hence applying we obtain

khk hrf x p x ai kp x ak hrf x hi

a a

which is a contradiction to the fact that p x a is the solution of QP x a

We now investigate some limiting prop erties of this residual function

For every x X we let

r x lim r x a

a

Obviously it is p ossible that r x for some x X The follow

ing lemma allows us to identify certain limits of the functions p x and

r x It is really just a technical result used for proving the ensuing result

Lemma Let x X and suppose for some a

r x a r x

Then

x a a a p x a p

In this case

lim p x a p x a Hx

a

Pro of Let a a Lemma ii and show that

x a kp x ak p

We now derive a contradiction to the supp osition that

x a p x a p

Since p x a and p x a are the unique solutions of the problems QP x a

and QP x a resp ectively we have

hrf x p x ai x a x a p kp x ak rf x p

a a

x a p x a rf x p

a

hrf x p x ai kp x ak

a

Hence by

x a hrf x p x ai hrf x p x ai rf x p

which is a contradiction and so is proved It now follows from

that lim p x a p x a so that Corollary implies p x a

a

Hx

For x X with Hx we dene p x as follows

p x arg min khk

hHx

In particular note the dierence b etween the denition of p x and

r x to that of p x a and r x a given ab ove

Lemma For every x X the fol lowing assertions hold

i r x Hx

ii If r x then

r x kp x k

and for al l r r x

Hx r Hx B

r

n

where B fx R j kxk r g

r

Pro of i Let Hx Applying Lemma we immediately get

that r x Conversely let r x Then for an

arbitrary sequence fa g the sequence fp x a g is b ounded and

k k

n

has a limit p oint p R By Corollary we have that p Hx

hence Hx

ii Supp ose that r x Lemma yields kp x k r x

On the other hand by Corollary

r x lim kp x ak min khk kp x k

a

hHx

For the nal assertion of the lemma let us x an r with

r r x

and take an arbitrary h Hx r We need to show that

h Hx

Applying Lemma i obtain that there exists such that

rf x h N x h

X

x h X

khk r khk r r r (x,a) 0

r*

1

a

0.5 2

Figure r x a a x r is essentially the inverse of this function

If then follows immediately from and Lemma ii

Assume therefore that Then implies khk r By

Lemma i obtain h p x Therefore r p x

r x Hence by and Lemma ii we have

a r x a r x

Now applying Lemma we obtain

We now dene a function a x r that acts like the inverse of the function

r x a see Figure The inverse is generally set valued see r hence

for every x X and r dene

a x r min fa j r x a r g

where we formally set min Note that this is imp ortant in the example

of Figure when r r Furthermore in that example note that r x a

for any a but that a x is well dened The main

prop erties of the function a x R R fg are presented in the

following lemma

Lemma For every x X the fol lowing properties hold

i For al l r r a x r a x r a x

ii r a x r if x X and a x r if x X

stat stat

iii r a x r Hx

iv For al l r r x a x r r

Pro of i Follows from Lemma ii

ii Follows from Lemma iv

iii Follows from Lemma i

iv Obviously for every x X and r we have the following if there

exists a such that r r x a then

r x a x r r

Otherwise r r x with r x and a x r

We now present an imp ortant relationship that exists b etween the problems

TP x r and QP x a Essentially it states that the solution of QP x a

is also a solution of TP x r

Theorem For every x X and r p x a x r Hx r

Pro of Let x X and r b e arbitrary We can supp ose that x X

stat

since otherwise p x a x r Hx r It follows that a x r

and we consider two p ossible cases

In the rst case supp ose that there exists a such that r r x a

Then a x r r x a x r r by and so kp x a x r k r

Also by Lemma i we have

rf x p x a x r N x p x a x r

X

a xr

x p x a x r X

Applying Lemma i we obtain p x a x r Hx r

Otherwise r r x a for all a It follows that r r x

r x Hx and a x r Now from Lemma ii

we have

p x a x r p x Hx r

completing the pro of of the theorem

The following result that provides a useful b ound on the optimal value of

TP x r is now easily established using Lemma and Lemma iv

Corollary For every x X n X and r the fol lowing bound

stat

holds

r v x r min hrf x hi

a x r

xhX khkr

where we formal ly set

The Sequential Linearization Metho d

n

Let kk b e an arbitrary norm in R It is well known that there are p ositive

n

constants c and c such that for all x R

c kxk kxk c kxk

Note that for every x X and r the following inclusions hold

fh j x h X khk c r g fh j x h X khk r g

fh j x h X khk c r g

For x X and r consider the following problem

min hrf x hi

h

LP x r

sub ject to x h X

khk r

Clearly when kk is a p olyhedral norm and X is a p olyhedral set LP x r is

a linear programming problem Denote by H x r and v x r the solution

set and the optimal value of LP x r resp ectively The inclusions imply

that

v x c r v x r v x c r

for every x X and r The next result follows immediately from

Corollary and

Lemma For every x X and r the fol lowing assertion holds

x X v x r

stat

In order to justify the technique that we use to up date the trust region

radius we will also need a simple technical result

r with Lemma For every and x X n X there exist r

stat

r r such that the fol lowing properties hold

r r h H x r f x f x h i r

ii r r h H x r f x f x h r

Pro of Let and x X n X

stat

q

i Dene r f x f so that for all r r

opt

r r f x f x h f x f

opt

ii Let

a c Lc

with a x c r a Take so that by Lemma there exists r

an arbitrary r r and h H x r By Lemma i

a a x c r a x c r

then Corollary implies

c

r c r v x c r

a x c r a

It follows from and that

c

hrf x hi v x r v x c r r

a

Therefore by

L

khk f x h f x hrf x hi

c Lc

r khk

a

Lc c

r r

a

c

Lc r

a

r

The algorithm we prop ose will have an Armijo style comp onent to it

To describ e this comp onent precisely let us x arbitrary parameters and

r with and r and dene

n o

k

R r j r r k

The next result now follows immediately from Lemma and Lemma

and shows there is an r R that satises the Armijo test given b elow in

Algorithm SLP but that r do es not

Corollary Let x X and for each r R let h x r Hx r The

fol lowing assertions hold

i If x X then

stat

r R v x r rf x h x r

ii If x X then

stat

r R v x r rf x h x r

and moreover there exists r R such that

r r

f x f x h x

and

f x f x h x r r

We now state the main algorithm The parameter k counts the number

of iterations while represents the number of subproblems that we have

k

solved at the k th iteration

Algorithm SLP

Parameters r

Input x X

Step Initialization Set k r r k

k stop

k

Step Set k k r r Solve the problem LP x r to

k

k k

obtain a solution h x r Hx r Set

D E

k k k

if v x r rf x h x r

k k k k

r if v x r and f x f x h x r

k k k k

if v x r and f x f x h x r r

k k

stop

x and exit If set k k x

stop

k

Step Set and r r Solve the problem LP x r to obtain

k k

a solution h x r Hx r

k k k

r or and Step If and f x f x h x r

k k k

f x f x h x r r then go to Step else go to Step

k

Step If set r r else r r Let and x

k k k

k k

x h x r and go to Step

k

Note that in Step the parameter can take on one of three values If

a stationary p oint has b een found and the algorithm terminates If

then sucient decrease was not achieved and thus the subproblem

needs to b e resolved with a smaller value of r If then sucient

decrease was achieved but the subproblem is solved again this time with a

larger value of r Both of these resolves take place in Step

If the algorithm terminates after a nite number of iterations we can

k k k

stop stop stop

dene the iterates x x as equal to x for the purp oses of

stating convergence results Furthermore it follows from Corollary that

k

for every sequence of iterates fx g generated by the algorithm

k

k and hence even if k r k Therefore

stop k

the algorithm is well dened We now pro ceed to prove the convergence and

asso ciated rates as outlined in the introduction

General Case Convergence Theory

k

It is easily seen that every innite sequence of iterates fx g of the algorithm

has the following prop erties

r r

k k

k k k

f x f x h x

k k k k k

f x f x f x f x h x r r

k

k

r k

k

We also note that

k k k k

x x h x r c h x r c r

k k k

Hence for k

k k k k

x x f x f x

c

To analyze convergence prop erties of this sequence we need the following

lemma

r

Lemma Suppose that for some x X n X r and h x

stat

r

the fol lowing condition holds H x

r r

f x f x h x

Then

Lc

v x r r

c

a x r a x c r a

Lc

r a r x a kx x rf xk

X

where c min c a min a

Pro of It follows from that

r r

f x f x h x

L

hrf x h x r i kh x r k

Lc

r hrf x h x r i

Hence

Lc

v x r hrf x h x r i r

and so holds

If a x c r then a x r and

r x r c r r x a

Thus and are valid

Supp ose that a x c r We also note from Lemma ii that

a x c r By Theorem p x a x c r Hx c r therefore

hrf x p x a x c r i v x c r v x c r

Applying and we obtain

Lc

r hrf x p x a x c r i v x c r v x r v x r

Now by Lemma ii we have

Lc

r hrf x p x a x c r i

kp x a x c r k

a x c r

r x a x c r

a x c r

c r

a x c r

the last inequality following from Lemma iv Hence by Lemma i

c

a x r a x c r a

Lc

and is proved

Lemma iv and Lemma ii imply

r r x a x r r x a

so applying Lemma ii and iii

a r x if a

r x a

r x if a

Thus is proved

Note that the following convergence theorem do es not require any as

sumptions apart from the solvability of P and the convexity of X

k

Theorem Every sequence of iterates fx g produced by SLP has the

property that

k k k k

x rf x x x r

X

Moreover if X is compact then every sequence of iterates converges to X

stat

Pro of It follows from and Lemma that for every sequence of

iterates pro duced by SLP

k k k k k

x rf x f x f x x a

X

k

for k However ff x g is b ounded b elow hence converges

implying the rst statement of the theorem If the second statement is

k

not valid then there is a subsequence fx g and an such that

k K

k k k

for k K Since fx g is b ounded fx g has a limit dist x X

k K stat

p oint x for which dist x X But this is a contradiction to the

stat

rst statement of the theorem

Our convergence analysis is based in part of that given by Luo and Tseng

The following assumptions are part of that work and will b e used in

our main convergence result

Assumption A For every f there exist scalars and

opt

such that

dist x X kx x rf xk

stat X

for all x X with f x and kx x rf xk

X

Assumption A For every f the set

opt

f fx X j f x g R

stat

is nite

Theorem of gives sucient conditions to guarantee that the ab ove

assumptions hold

To obtain more precise convergence prop erties of SLP we need the fol

lowing lemma This result essentially provides a lo cal error b ound

Lemma Let Assumption A hold Then for every v f and a

opt

L such that for al l r r x a there exists v a

f x p x a x r f x r

X

stat

for al l x X with f x and r x kx x rf xk

X

Pro of Let v f and a b e arbitrary and consider any x X with

opt

Fix an arbitrary f x v r x and r r x a

x x x

X X

stat stat

Note that X may not b e convex It follows from A that

stat

kx xk dist x X r x

stat

hence by Lemma iii

kx xk r x a x r kp x a x r k

where max a Obviously if x X then x x and

stat

x p x a x r x x

and the desired b ound holds

Supp ose therefore that x X then a x r it is p ossible that

stat

a x r For simplicity denote p x a x r by p Note that by

the denition of a x r and Lemma

r if r r x

kp k r x a x r

r if r r x

Thus

kp k r

Let us show that

hrf x x p xi kx xk

a

If a x r then p Hx and

hrf x p i hrf x x xi

which implies

If a x r then by the denition of p x a x r we have

kp k hrf x x xi kx xk hrf x p i

a x r a x r

hence

kx xk hrf x x p x i

a

It follows from and that

hrf x x p x i r

a

Applying the Mean Value Theorem we have

f x p f x hrf x p xi

for some x x p Clearly

k xk kx xk kp k r

where It now follows from that

f x p f x hrf x p xi

hrf rf x x p x i hrf x x p xi

L kx k kx xk kp k hrf x x p xi

Using the relations and we obtain

r f x p f x L a

as required

Lemma Let A hold Then for every v f there exists

opt

L such that if x X n X and r satises f x v v a

stat

r x and then

r f x f x

X

stat

Pro of Lemma and imply that

Lc

v x r r

so that Theorem shows

p x a x c r Hx c r

Applying we obtain

hrf x p x a x c r i v x c r v x c r

Lc

v x r v x r r

Therefore

f x p x a x c r f x

L

kp x a x c r k hrf x p x a x c r i

Lc Lc

r r

r

where

Lc Lc

By Lemma a x c r a so that Lemma iv implies

c r r x a x c r r x a

Applying Lemma we see that

f x p x a x c r f x r

X

stat

where v a L Combining and yields

f x f x r

X

stat

Now the pro of of the main convergence theorem is similar to that given

by Luo and Tseng Theorem for the linear convergence of a class of

descent techniques

Theorem Let A A hold Every innite sequence of iterates produced

by SLP converges to a point of x X Moreover for every v f

stat opt

there exist q q with q q such that for every innite sequence

k

fx g f x v the fol lowing estimations hold

k

f x f x

q lim sup

k

f x f x

k

k

k

lim sup q dist x X

stat

k

k

Every nite sequence of iterates fx g of SLP terminates in the stationary

k

stop

set x X

stat

Pro of Let v f b e chosen arbitrarily and consider the sequence of

opt

k k

stop

r iterates fx g of SLP with f x v If k then v x

k stop

stop

k

stop

and Lemma gives x X

stat

Supp ose therefore that k It is easily seen that for k

stop

k k

f f x f x v

opt

Hence there exists f f v such that

opt

k

lim f x f

k

It follows from and that

k k k

x lim f x f x lim r

k k

and hence that for some K

k k k k

x r x rf x x k K K

x

k

Since X may not b e convex let us denote an arbitrary pro jection of x

stat

k

onto X by x that is

stat

k k k

x x x k

X X

stat stat

Using and the assumption A implies that for k K K

k k k k

x x x dist x X r

stat

Therefore by

k k k

x x lim lim dist x X

stat

k k

It follows from and that

k k

lim x x

k

which combined with yields

k k

x x lim

k

The assumption A p ostulates that there is some f R and K K

stat

such that

k

f x f f X k K K

stat stat

The Mean Value Theorem implies that for k K

D E

k k k k k

f x f x rf x x

k k k

for some lying on the line segment joining x and x Hence

D E D E

k k k k k k k k

f f x rf rf x x x rf x x x

stat

D E

k k k k

As x X it follows that rf x x x so that

stat

k k k k k k k

f f x L x x x L x x

stat

k

Therefore by f lim f x f Using

stat k

and Lemma we have

k

r f x f k K K

stat

k

so that using

k k k

f x f k K K f x f x

stat

and

k k

f x f q f x f k K K

stat stat

where q max q Thus

k

lim f x f f

stat

k

and moreover

k

f x f

stat

q k K K

k

f x f

stat

Now it follows from that there exists c such that

p

k

k k

q k K K x x c

P

k k

Hence x x and there exists an x such that

k

k

lim x x X

k

Moreover by x X It follows from and that

stat

k k k

f x f f x f x r k

stat

k

and by Lemma and A that

a

k k

dist x X k K K f x f

stat stat

Taking into account we obtain that there exist K and q with

q such that

k k

K q k K K dist x X

stat

Sp ecial Case Polyhedral Constraints

In the case where X is dened by linear equalities and inequalities the

problem LP x r is a linear program and hence the algorithm SLP is a

successive linear programming algorithm

We have implemented SLP in Matlab using the Cplex linear

programming co de for the trust region subproblems LP x r Using the

interface b etween Cute and Matlab describ ed in we have tested the

algorithm on a reasonable class of linearly constrained test problems The

results are summarized in Tables and which arbitrarily separates the

problems in the Ho ckSchittkowski collection from the remaining ones

we tested from the Cute collection In the tables the number of general

linear constraints is denoted by m and n is used to denote the number

of variables Lower and upp er b ounds are treated explicitly by the linear

programming co de and thus are not rep orted here The computations were

carried out on a Sun SPARCstation with MB RAM running Matlab

version c The time rep orted in the table was derived using the cputime

function of Matlab No attempt was made to optimize the SLP co de to allow

for fast restarts in the Armijo test and so this is generally a considerable over

estimate of the time that would b e required for a sophisticated implementa

tion of the algorithm Of more imp ortance are the number of iterations and

the total number of LPs that needed to b e solved Note that the number

of LPs is guaranteed to b e at least twice the number of iterations due to

the Armijo test

We have excluded the problems BOOTH EXTRASIM HIMMELBA

HS HS HYDROELM HYDROELS STANCMIN SUPERSIM and

ZANGWIL from the tables of results for the sake of brevity since in these

examples a stationary p oint was found after one iteration with just two

linear program solutions Note that paradoxically more than one LP was

required to solve some of the linear programs in the Cute collection This is

due to our default choice on the initial trust region radius

The algorithm app ears to b e very robust Since the LP solver is very

fast the resulting NLP co de is also very ecient even in the Matlab imple

mentation

Note however that the metho d will only give linear convergence and thus

once an active set has b een identied the algorithm should apply a New

ton pro cess for solving the resulting reduced problem such as the linear

programming based technique given in Furthermore conditions under

which this metho d identies the correct active set after a nite number of

iterations could b e derived in a similar fashion to that given by Burke and

More Such a pro cedure would undoubtedly improve the p erformance

of the algorithm on problems HS and S Similar nitely terminat

ing successive linearization algorithms have b een used for machine learning

problems

References

T E Baker and L S Lasdon Successive linear programming at Exxon

Management Science

Problem m n CPU Iters LPs

HS

HS

HS

HS

HS

HSMOD

HS

HS

HS

HS

HSMOD

HS

HS

HS

HS

HSNEW

HS

HS

HS

HS

HS

HS

HS

HS

HS

HS

Table SLP on p olyhedrally constrained HS nonlinear programs

Problem m n CPU Iters LPs

DEGENLPA

DEGENLPB

GENHS

GOULDQP

GOULDQP

HAGER

HAGER

HAGER

HAGER

HATFLDH

HIMMELBI

HONG

ODFITS

PENTAGON

QPCBLEND

QPCBOEI

QPCBOEI

QPCSTAIR

QPNBLEND

QPNBOEI

QPNBOEI

QPNSTAIR

READING

S

S

SIMPLLPA

SIMPLLPB

SSEBLIN

TAME

TFI

TFI

Table SLP on remaining p olyhedrally constrained nonlinear programs

from Cute

K P Bennett and O L Mangasarian Bilinear separation of two sets

in nspace Computational Optimization and Applications

D P Bertsekas Nonlinear Programming Athena Scientic Belmont

Massachusetts

I Bongartz A R Conn N Gould and Ph L Toint CUTE Con

strained and unconstrained testing environment ACM Transactions

on Mathematical Software

P S Bradley O L Mangasarian and W N Street Feature selection

via mathematical programming Mathematical Programming Technical

Rep ort Computer Sciences Department University of Wisconsin

Madison Wisconsin

M A Branch Getting CUTE with Matlab Cornell Theory Center

Rep ort CTCTR Cornell University Ithaca New York

J V Burke A robust trust region metho d for constrained nonlin

ear programming problem SIAM Journal on Optimization

J V Burke and J J More On the identication of active constraints

SIAM Journal on Numerical Analysis

J V Burke J J More and G Toraldo Convergence prop erties of

trust region metho ds for linear and convex constraints Mathematical

Programming

P H Calamai and J J More Pro jected gradient metho ds for linearly

constrained problems Mathematical Programming

CPLEX Optimization Inc Incline Village Nevada Using the

CPLEXTM Linear Optimizer and CPLEXTM Mixed Integer Op

timizer Version

J C Dunn Rates of convergence for conditional gradient algorithms

near singular and nonsingular extremals SIAM Journal on Control and

Optimization

J C Dunn and E Sachs The eect of p erturbations on the con

vergence rates of optimization algorithms Applied Mathematics and

Optimization

M C Ferris Iterative linear programming solution of convex programs

Journal of Optimization Theory and Applications

R Fletcher and E Sainz de la Maza Nonlinear programming and

nonsmo oth optimization by successive linear programming Numerical

Analysis Rep ort NA Department of Mathematical Sciences The

University Dundee Scotland

M Frank and P Wolfe An algorithm for quadratic programming Naval

Research Logistics Quarterly

E M Gafni and D P Bertsekas Twometric pro jection metho ds for

constrained optimization SIAM Journal on Control and Optimization

W Ho ck and K Schittkowski Test Examples for Nonlinear Program

ming Codes volume of Lecture Notes in Economics and Mathemat

ical Systems Springer Verlag Berlin Germany

L S Lasdon A Waren S Sarkar and F Palacios Solving the p o ol

ing problem using generalized reduced gradient and successive linear

programming algorithms ACM SIGMAP Bul letin

ZQ Luo and P Tseng Error b ounds and convergence analysis of

feasible descent metho ds A general approach Annals of Operations

Research

O L Mangasarian Nonlinear Programming McGrawHill New York

SIAM Classics in Applied Mathematics SIAM Philadelphia

O L Mangasarian Machine learning via p olyhedral concave mini

mization In H Fischer B Riedmueller and S Schaeer editors

Applied Mathematics and Parallel Computing Festschrift for Klaus

Ritter pages PhysicaVerlag A SpringerVerlag Company Hei

delb erg

MATLAB Users Guide The MathWorks Inc

F PalaciosGomez L Lasdon and M Engquist Nonlinear optimiza

tion by successive linear programming Management Science

R T Ro ckafellar Convex Analysis Princeton University Press Prince

ton New Jersey

TH Shiau Iterative Linear Programming for Linear Complementarity

and Related Problems PhD thesis Computer Sciences Department

University of Wisconsin Madison Wisconsin Technical Rep ort

Ph L Toint Global convergence of a class of trust region metho ds for

nonconvex minimization in a Hilb ert space IMA Journal of Numerical

Analysis

J Zhang NH Kim and L Lasdon An improved successive linear

programming algorithm Management Science