Argonne National Lab oratory South Cass Avenue Argonne IL ANLMCSTM Users Guide for SnadiOpt A Package Adding Automatic Dierentiation to Snopt by E Michael Gertz Mathematics and Computer Science Division Argonne National Lab oratory Argonne Illinois Philip E Gil l and Julia Muetherig Department of Mathematics University of California San Diego La Jolla California Mathematics and Computer Science Division Technical Memorandum No January gertzmcsanlgov pgillucsdedu jmuethucsdedu httpwwwmcsanlgovgertz httpwwwscicompucsdedupeg httpwwwscicompucsdedujulia This work was supp orted by the Mathematical Information and Computational Sciences Division subprogram of the Oce of Advanced Scientic Computing Research US Department of Energy under Contract WEng and by National Science Foundation Grant CCR Argonne National Lab oratory with facilities in the states of Illinois and Idaho is owned by the United States Government and op erated by The University of Chicago under the provisions of a contract with the Department of Energy DISCLAIMER This rep ort was prepared as an accountofwork sp onsored by an agency of the United States Government Neither the United States Government nor any agency thereof nor The University of Chicago nor anyoftheiremployees or ocers makes anywarranty express or implied or assumes any legal liability or resp onsibility for the accuracy completeness or usefulness of any information apparatus pro duct or pro cess disclosed or represents that its use would not infringe privatelyowned rights Reference herein to any sp ecic commercial pro duct pro cess or service by trade name trademark manufacturer or otherwise do es not necessarily constitute or imply its endorsement recommendation or favoring by the United States Governmentorany agency thereof The views and opinions of do cument authors expressed herein do not necessarily state or reect those of the United States Government or any agency thereof Argonne National Lab oratory or The University of Chicago ii Contents Abstract Intro duction Problem Types Why Automatic Dierentiation ADIFOR Who Should Use This Package How to Read This Manual Basic Usage Additional Resources Automatic Dierentiation UserSupplied Subroutines The Function Denition Routine The Initialization Routine An Example Problem Invoking SnadiOpt Lo cating Executables and Libraries Basic Usage Files Generated by snadioptpl Merging Changes AdvancedUsage Summary of All Options Building the Executable Typical Usage Sub ordinate Makeles Useful makefile Targets Useful makefile Variables Dense ADIFOR References Index iii Users Guide for SnadiOpt A Package Adding Automatic Dierentiation to Snopt by E Michael Gertz Philip E Gil l and Julia Muetherig Abstract SnadiOpt is a package that supp orts the use of the automatic dierentiatio n package ADIFOR with the optimization package Snopt Snopt is a generalpurp ose system for solving optimization problems with many variables and constraints It minimizes a linear or nonlinear function sub ject to b ounds on the variables and sparse linear or nonlinear constraints It is suitable for largescale linear and quadratic programming and for linearly constrained optimization as well as for general nonlinear programs The metho d used by Snopt requires the rst derivatives of the ob jective and con straint functions to b e available The SnadiOpt package allows users to avoid the time consuming and errorprone pro cess of evaluating and co ding these derivatives Given Fortran co de for evaluating only the values of the ob jective and constraints SnadiOpt automatically generates the co de for evaluating the derivatives and builds the relevant Snopt input les and sparse data structures Keywords Largescale nonlinear programming constrained optimization SQP metho ds automatic dierentiati on Fortran software Users Guide for SnadiOpt Intro duction This is the users guide for SnadiOpt a package that adds the automatic dierentiation capability to the nonlinear optimization package Snopt SnadiOpt uses the sourceto source automatic dierentiation package ADIFOR to p erform the dierentiation Problem Typ es Snopt is a collection of Fortran subroutines for solving a nonlinear programming problem assumed to b e stated in the following form NP min or max F x obj x sub ject to l x u L F x U where u U l andL are constantvectors of upp er and lower b ounds F xisavector of smo oth linear and nonlinear problem functions and F x denotes the comp onentof F to obj b e minimized or maximized Note that upp er and lower b ounds are sp ecied for all variables and constraints This form allows full generality in sp ecifying various typ es of constraint Sp ecial values are used to indicate absent b ounds l or u for appropriate j Free variables and free j j constraints free rows are ones that have b oth b ounds innite Similarlyxedvariables have l u and equality constraints have L U j j j j The metho d used by Snopt requires that the elements J xF x x of the Jaco ij i j bian matrix of rst derivatives b e known at any p oint x In practice it is often inconvenient or imp ossible to co de the derivatives and so Snopt allows the user to co de as manyderiva tives as is convenient Snopt then estimates unknown derivatives by nite dierences by making a call to F for eachvariable x whose partial derivatives need to b e estimated j However nite dierences reduce the reliability of the optimization algorithm and can b e exp ensive if there are many suchvariables x TheSnadiOpt package allows the user to avoid j the timeconsuming and errorprone pro cess of evaluating and co ding derivatives without the need for Snopt to compute nite dierences Often an element J is constant which implies that variable x o ccurs only linearly ij j in the problem function F x If a signicantnumb er of these constant elements are zero j then J is known as a sparse matrix and Snopt uses a sparse matrix format to store only the nonzero elements of J SnadiOpt automatically identies constant and zero Jacobian elements by using a scheme that evaluates the Jacobian at a number of points close to the starting p oint see Section Given Fortran co de for evaluating only F x SnadiOpt automatically generates co de for evaluating J and builds the relevant Snopt input les and sparse data structures Why Automatic Dierentiation Writing co de for the derivatives of F x is dicult time consuming and error prone es p ecially when problems involvemanyvariables and constraints Automatic dierentiation AD to ols in this case ADIFOR quickly provide correct and numerically accurate derivative functions from the co de used to evaluate the ob jective and constraint functions Prior to the wide availability of AD software and ADbased mo deling languages nu merical dierentiation was the only alternativetoproviding derivative co de Unfortunately numerical dierentiation is an inherently unstable pro cess that causes b oth theoretical and practical diculties for nonlinear solvers Numerical dierentiation places a severe theo retical limit on the accuracy of the solution that may b e computed by an algorithm see eg Chapter In practice co de that uses numerical dierentiation tends to need Introduction more iterations to nd a solution than do es co de that uses exact derivatives Furthermore co de that uses numerical dierentiation is typically less robust and will fail to nd solutions for problems that mighthave b een solved if analytic derivatives were supplied Notwith standing these diculties numerical dierentiation was often used to avoid the high cost of handco ding the exact derivatives With mo dern AD to ols derivativecodemay b e quickly obtained leading to a signicant increase in user pro ductivityeven on simple problems Moreover functions can nowbe dierentiated that were once considered to o complex to b e co ded by hand An example is a function dened in terms of the output from an ordinary dierential equation solver ADIFOR has b een successfully applied to such functions Automatic dierentiation allows users to develop mo dels quickly This increase in pro ductivity makes optimization software a much more useful to ol for scientists who often wish to exp eriment with dierent ob jective functions and dierent sets of constraints ADIFOR ADIFOR is a robust mature automatic dierentiation to ol develop ed through a collab ora tive pro ject b etween the Mathematics and Computer Science Division at Argonne National Lab oratory and the Center for ResearchonParallel Computation at Rice University The package is a sourcetosource translator for functions written in Fortran It is widely available and runs on many p opular platforms Moreover the source co de for the ADI FOR libraries but not the translator is provided making it p ossible to compile and run ADIFORgenerated co de on most platforms ADIFOR implements the forward mo de for automatic dierentiation although it do es make some use of the reverse mo de internally As a practical matter this means that ADIFORgenerated co de will tend to run most quickly if the number of variables is not much larger than the numb er of constraints Who Should Use This Package