Optimization Toolbox for Use with MATLAB®

Optimization Toolbox For Use with MATLAB® User’s Guide Version 3 How to Contact The MathWorks: www.mathworks.com Web comp.soft-sys.matlab Newsgroup [email protected] Technical support [email protected] Product enhancement suggestions [email protected] Bug reports [email protected] Documentation error reports [email protected] Order status, license renewals, passcodes [email protected] Sales, pricing, and general information 508-647-7000 Phone 508-647-7001 Fax The MathWorks, Inc. Mail 3 Apple Hill Drive Natick, MA 01760-2098 For contact information about worldwide offices, see the MathWorks Web site. Optimization Toolbox User’s Guide © COPYRIGHT 1990 - 2004 by The MathWorks, Inc. The software described in this document is furnished under a license agreement. The software may be used or copied only under the terms of the license agreement. No part of this manual may be photocopied or repro- duced in any form without prior written consent from The MathWorks, Inc. FEDERAL ACQUISITION: This provision applies to all acquisitions of the Program and Documentation by, for, or through the federal government of the United States. By accepting delivery of the Program or Documentation, the government hereby agrees that this software or documentation qualifies as commercial computer software or commercial computer software documentation as such terms are used or defined in FAR 12.212, DFARS Part 227.72, and DFARS 252.227-7014. Accordingly, the terms and conditions of this Agreement and only those rights specified in this Agreement, shall pertain to and govern the use, modification, reproduction, release, performance, display, and disclosure of the Program and Documentation by the federal government (or other entity acquiring for or through the federal government) and shall supersede any conflicting contractual terms or conditions. If this License fails to meet the government's needs or is inconsistent in any respect with federal procurement law, the government agrees to return the Program and Documentation, unused, to The MathWorks, Inc. MATLAB, Simulink, Stateflow, Handle Graphics, and Real-Time Workshop are registered trademarks, and TargetBox is a trademark of The MathWorks, Inc. Other product or brand names are trademarks or registered trademarks of their respective holders. History: November 1990 First printing December 1996 Second printing For MATLAB 5 January 1999 Third printing For Version 2 (Release 11) September 2000 Fourth printing For Version 2.1 (Release 12) June 2001 Online only Revised for Version 2.1.1 (Release 12.1) September 2003 Online only Revised for Version 2.3 (Release 13SP1) June 2004 Fifth printing Revised for Version 3.0 (Release 14) October 2004 Online only Revised for Version 3.0.1 (Release 14SP1) Acknowledgements The MathWorks would like to acknowledge the following contributors to the Optimization Toolbox. Thomas F. Coleman researched and contributed the large-scale algorithms for constrained and unconstrained minimization, nonlinear least squares and curve fitting, constrained linear least squares, quadratic programming, and nonlinear equations. Dr. Coleman is Professor of Computer Science and Applied Mathematics at Cornell University. He is Director of the Cornell Theory Center and the Cornell Computational Finance Institute. Dr. Coleman is Chair of the SIAM Activity Group on Optimization, and a member of the Editorial Boards of Applied Mathematics Letters, SIAM Journal of Scientific Computing, Computational Optimization and Applications, Communications on Applied Nonlinear Analysis, and Mathematical Modeling and Scientific Computing. Dr. Coleman has published 4 books and over 70 technical papers in the areas of continuous optimization and computational methods and tools for large-scale problems. Yin Zhang researched and contributed the large-scale linear programming algorithm. Dr. Zhang is Associate Professor of Computational and Applied Mathematics on the faculty of the Keck Center for Computational Biology at Rice University. He is on the Editorial Board of SIAM Journal on Optimization, and is Associate Editor of Journal of Optimization: Theory and Applications. Dr. Zhang has published over 40 technical papers in the areas of interior-point methods for linear programming and computation mathematical programming. Contents Getting Started 1 What Is the Optimization Toolbox? . 1-2 Optimization Example . 1-3 The Problem . 1-3 Setting Up the Problem . 1-3 Finding the Solution . 1-4 More Examples . 1-5 Tutorial 2 Introduction . 2-3 Problems Covered by the Toolbox . 2-3 Using the Optimization Functions . 2-6 Medium- and Large-Scale Algorithms . 2-7 Examples That Use Standard Algorithms . 2-9 Unconstrained Minimization Example . 2-10 Nonlinear Inequality Constrained Example . 2-11 Constrained Example with Bounds . 2-13 Constrained Example with Gradients . 2-14 Gradient Check: Analytic Versus Numeric . 2-16 Equality Constrained Example . 2-17 Maximization . 2-18 Greater-Than-Zero Constraints . 2-18 Avoiding Global Variables via Anonymous and Nested Functions . 2-19 Nonlinear Equations with Analytic Jacobian . 2-23 Nonlinear Equations with Finite-Difference Jacobian . 2-26 Multiobjective Examples . 2-27 i Large-Scale Examples . 2-40 Problems Covered by Large-Scale Methods . 2-41 Nonlinear Equations with Jacobian . 2-44 Nonlinear Equations with Jacobian Sparsity Pattern . 2-47 Nonlinear Least-Squares with Full Jacobian Sparsity Pattern . 2-49 Nonlinear Minimization with Gradient and Hessian . 2-51 Nonlinear Minimization with Gradient and Hessian Sparsity Pattern . 2-52 Nonlinear Minimization with Bound Constraints and Banded Preconditioner . 2-54 Nonlinear Minimization with Equality Constraints . 2-58 Nonlinear Minimization with a Dense but Structured Hessian and Equality Constraints . 2-59 Quadratic Minimization with Bound Constraints . 2-63 Quadratic Minimization with a Dense but Structured Hessian . 2-65 Linear Least-Squares with Bound Constraints . 2-70 Linear Programming with Equalities and Inequalities . 2-72 Linear Programming with Dense Columns in the Equalities . 2-73 Default Options Settings . 2-76 Changing the Default Settings . 2-76 Displaying Iterative Output . 2-79 Most Common Output Headings . 2-79 Function-Specific Output Headings . 2-80 Calling an Output Function Iteratively . 2-86 What the Example Does . 2-86 The Output Function . 2-87 Creating the M-File for the Example . 2-88 Running the Example . 2-89 Optimizing Anonymous Functions Instead of M-Files . 2-93 Typical Problems and How to Deal with Them . 2-95 Selected Bibliography . 2-99 ii Contents Standard Algorithms 3 Optimization Overview . 3-3 Unconstrained Optimization . 3-4 Quasi-Newton Methods . 3-6 Line Search . 3-8 Quasi-Newton Implementation . 3-10 Hessian Update . 3-10 Line Search Procedures . 3-10 Least-Squares Optimization . 3-17 Gauss-Newton Method . 3-18 Levenberg-Marquardt Method . 3-19 Nonlinear Least-Squares Implementation . 3-21 Nonlinear Systems of Equations . 3-23 Gauss-Newton Method . 3-23 Trust-Region Dogleg Method . 3-23 Nonlinear Equations Implementation . 3-25 Constrained Optimization . 3-27 Sequential Quadratic Programming (SQP) . 3-28 Quadratic Programming (QP) Subproblem . 3-29 SQP Implementation . 3-30 Simplex Algorithm . 3-36 Multiobjective Optimization . 3-41 Introduction . 3-41 Goal Attainment Method . 3-47 Algorithm Improvements for Goal Attainment Method . 3-48 Selected Bibliography . 3-51 iii Large-Scale Algorithms 4 Trust-Region Methods for Nonlinear Minimization . 4-2 Preconditioned Conjugate Gradients . 4-5 Linearly Constrained Problems . 4-7 Linear Equality Constraints . 4-7 Box Constraints . 4-7 Nonlinear Least-Squares . 4-10 Quadratic Programming . 4-11 Linear Least-Squares . 4-12 Large-Scale Linear Programming . 4-13 Main Algorithm . 4-13 Preprocessing . 4-16 Selected Bibliography . 4-17 Function Reference 5 Functions — Categorical List . 5-2 Minimization . 5-2 Equation Solving . 5-2 Least Squares (Curve Fitting) . 5-3 Utility . 5-3 Demos of Large-Scale Methods . 5-3 Demos of Medium-Scale Methods . 5-4 Function Arguments . 5-5 Input Arguments . 5-5 Output Arguments . ..

Load more