Sequential Quadratic Programming

Sequential Quadratic Programming

Acta Numerica pp Sequential Quadratic Programming Paul T Boggs Applied and Computational Mathematics Division National Institute of Standards and Technology Gaithersburg Maryland USA Email boggscamnistgov Jon W Tolle Departments of Mathematics and Operations Research University of North Carolina Chapel Hil l North Carolina USA Email tol lemathuncedu CONTENTS Intro duction The Basic SQP Metho d Lo cal Convergence Merit Functions and Global Convergence Global to Lo cal Behavior SQP Trust Region Metho ds Practical Considerations References Intro duction Since its p opularization in the late s Sequential Quadratic Program ming SQP has arguably b ecome the most successful metho d for solving nonlinearly constrained optimization problems As with most optimization metho ds SQP is not a single algorithm but rather a conceptual metho d from whichnumerous sp ecic algorithms haveevolved Backed by a solid theoretical and computational foundation b oth commercial and public do main SQP algorithms have b een develop ed and used to solve a remarkably e large set of imp ortant practical problems Recently large scale versions hav b een devised and tested with promising results In this pap er we examine the underlying ideas of the SQP metho d and the theory that establishes it as a framework from which eective algorithms can Contribution of the National Institute of Standards and Technology and not sub ject to copyright in the United States Boggs and Tolle b e derived In the pro cess we will describ e the most p opular manifestations of the metho d discuss their theoretical prop erties and commentontheir practical implementations The nonlinear programming problem to b e solved is minimize f x x NLP sub ject to h x g x n n m n p where f R R h R R and g R R Such problems arise in avariety of applications in science engineering industry and management In the form NLP the problem is quite general it includes as sp ecial cases linear and quadratic programs in which the constraint functions h and g are ane and f is linear or quadratic While these problems are imp ortant and numerous the great strength of the SQP metho d is its abilitytosolve problems with nonlinear constraints For this reason it is assumed that NLP contains at least one nonlinear constraint function The basic idea of SQP is to mo del NLP at a given approximate solu k tion say x by a quadratic programming subproblem and then to use the k solution to this subproblem to construct a b etter approximation x This pro cess is iterated to create a sequence of approximations that it is hop ed will converge to a solution x Perhaps the key to understanding the p er formance and theory of SQP is the fact that with an appropriate choice of quadratic subproblem the metho d can b e viewed as the natural extension of Newton and quasiNewton metho ds to the constrained optimization set ting Thus one would exp ect SQP metho ds to share the characteristics of Newtonlike metho ds namely rapid convergence when the iterates are close to the solution but p ossible erratic b ehavior that needs to b e carefully con trolled when the iterates are far from a solution While this corresp ondence is valid in general the presence of constraints makes b oth the analysis and implementation of SQP metho ds signicantly more complex Two additional prop erties of the SQP metho d should b e p ointed out First SQP is not a feasiblep oint metho d that is neither the initial p oint nor any of the subsequent iterates need b e feasible a feasible p oint satises all of the constraints of NLP This is a ma jor advantage since nding a feasible p oint when there are nonlinear constraints may b e nearly as hard as solving NLP itself SQP metho ds can b e easily mo died so that lin ear constraints including simple b ounds are always satised Second the success of the SQP metho ds dep ends on the existence of rapid and accurate algorithms for solving quadratic programs Fortunately quadratic programs are easy to solve in the sense that there are a go o d pro cedures for their so lution Indeed when there are only equality constraints the solution to a quadratic program reduces to the solution of a linear system of equations Sequential Quadratic Programming When there are inequality constraints a sequence of systems mayhavetobe solved A comprehensive theoretical foundation do es not guarantee that a pro p osed algorithm will b e eective in practice In reallife problems hyp otheses may not b e satised certain constants and b ounds may not b e computable matrices maybenumerically singular or the scaling may b e p o or A suc cessful algorithm needs adaptive safeguards that deal with these pathologies The algorithmic details to overcome such diculties as wellasmoremun dane questionshowtocho ose parameters how to recognize convergence and how to carry out the numerical linear algebraare lump ed under the term implementation While a detailed treatment of this topic is not p os sible here we will take care to p oint out questions of implementation that p ertain sp ecically to the SQP metho d This survey is arranged as follows In Section we state the basic SQP metho d along with the assumptions ab out NLP that will hold throughout the pap er We also make some necessary remarks ab out notation and ter minology Section treats local convergence that is b ehavior of the iterates when they are close to the solution Rates of convergence are provided b oth in general terms and for some sp ecic SQP algorithms The goal is not to present the strongest results but to establish the relation b etween Newtons metho d and SQP to delineate the kinds of quadratic mo dels that will yield satisfactory theoretical results and to place currentvariations of the SQP metho d within this scheme The term global is used in two dierentcontexts in nonlinear optimiza tion and is often the source of confusion An algorithm is said to b e global ly convergent if under suitable conditions it will con verge to some lo cal so lution from any remote starting p oint Nonlinear optimization problems can havemultiple lo cal solutions the global solution is that lo cal solution corresp onding to the least value of f SQP metho ds like Newtons metho d and steep est descent are only guaranteed to nd a lo cal solution of NLP they should not b e confused with algorithms for nding the global solution which are of an entirely dierentavor To establish global convergence for constrained optimization algorithms away of measuring progress towards a solution is needed For SQP this is done by constructing a merit function a reduction in which implies that an acceptable step has b een taken In Section two standard merit functions are dened and their advantages and disadvantages for forcing global con vergence are considered In Section the problems of the transition from global to lo cal convergence are discussed In Sections and the emphasis is on line search metho ds Because trust region metho ds whichhavebeen found to b e eective in unconstrained optimization have b een extended to t into the SQP framework a brief description of this approachisgiven in Boggs and Tolle Section Section is devoted to implementation issues including those asso ciated with large scale problems Toavoid interrupting the ow of the presentation comments on related results and references to the literature are provided at the end of eachsec tion The numb er of pap ers on the SQP metho d and its variants is large and space prohibits us from compiling a complete list of references wehave tried to give enough references to each topic to direct the interested reader to further material Notes and References The earliest reference to SQPtyp e algorithms seems to have b een in the PhD thesis of Wilson at Harvard University in which he prop osed the metho d we call in Section the NewtonSQP algorithm The development of the secantorvariablemetric algorithms for unconstrained optimization in the late s and early s naturally led to the extension of these metho ds to the constrained problem The initial work on these metho ds was done by Mangasarian and his students at the University of Wisconsin GarciaPalomares and Mangasarian investigated an SQPtyp e algo rithm in which the entire Hessian matrix of the Lagrangian ie the matrix of second derivatives with resp ect to b oth to x and the multipliers was up dated at each step Shortly thereafter Han Han and Han provided further imp etus for the study of SQP metho ds In the rst pap er Han gave lo cal convergence and rate of convergence theorems for the PSB and BFGSSQP algorithms for the inequalityconstrained problem and in the second employed the merit function to obtain a global convergence theorem in the conv ex case In a series of pap ers presented at conferences Powell Powell a and Powell b Hans work was brought to the attention of the general optimization audience From that time there has b een a continuous pro duction of research pap ers on the SQP metho d As noted a signicantadvantage of SQP is that feasible p oints are not required at any stage of the pro cess Nevertheless a version of SQP that always remains feasible has b een develop ed and studied for example by Bonnans Panier Tits and Zhou The Basic SQP Metho d Assumptions and Notation As with virtually all nonlinear problems it is necessary to makesomeas sumptions on the problem NLP that clarify the class of problems for which the algorithm can b e shown to p erform well These assumptions as well as the consequent theory of nonlinear programming are also needed to

View Full Text

Details

  • File Type
    pdf
  • Upload Time
    -
  • Content Languages
    English
  • Upload User
    Anonymous/Not logged-in
  • File Pages
    52 Page
  • File Size
    -

Download

Channel Download Status
Express Download Enable

Copyright

We respect the copyrights and intellectual property rights of all users. All uploaded documents are either original works of the uploader or authorized works of the rightful owners.

  • Not to be reproduced or distributed without explicit permission.
  • Not used for commercial purposes outside of approved use cases.
  • Not used to infringe on the rights of the original creators.
  • If you believe any content infringes your copyright, please contact us immediately.

Support

For help with questions, suggestions, or problems, please contact us