Nonlinear Programming
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Lecture 3 1 Geometry of Linear Programs
ORIE 6300 Mathematical Programming I September 2, 2014 Lecture 3 Lecturer: David P. Williamson Scribe: Divya Singhvi Last time we discussed how to take dual of an LP in two different ways. Today we will talk about the geometry of linear programs. 1 Geometry of Linear Programs First we need some definitions. Definition 1 A set S ⊆ <n is convex if 8x; y 2 S, λx + (1 − λ)y 2 S, 8λ 2 [0; 1]. Figure 1: Examples of convex and non convex sets Given a set of inequalities we define the feasible region as P = fx 2 <n : Ax ≤ bg. We say that P is a polyhedron. Which points on this figure can have the optimal value? Our intuition from last time is that Figure 2: Example of a polyhedron. \Circled" corners are feasible and \squared" are non feasible optimal solutions to linear programming problems occur at \corners" of the feasible region. What we'd like to do now is to consider formal definitions of the \corners" of the feasible region. 3-1 One idea is that a point in the polyhedron is a corner if there is some objective function that is minimized there uniquely. Definition 2 x 2 P is a vertex of P if 9c 2 <n with cT x < cT y; 8y 6= x; y 2 P . Another idea is that a point x 2 P is a corner if there are no small perturbations of x that are in P . Definition 3 Let P be a convex set in <n. Then x 2 P is an extreme point of P if x cannot be written as λy + (1 − λ)z for y; z 2 P , y; z 6= x, 0 ≤ λ ≤ 1. -
Nonlinear Optimization Using the Generalized Reduced Gradient Method Revue Française D’Automatique, Informatique, Recherche Opé- Rationnelle
REVUE FRANÇAISE D’AUTOMATIQUE, INFORMATIQUE, RECHERCHE OPÉRATIONNELLE.RECHERCHE OPÉRATIONNELLE LEON S. LASDON RICHARD L. FOX MARGERY W. RATNER Nonlinear optimization using the generalized reduced gradient method Revue française d’automatique, informatique, recherche opé- rationnelle. Recherche opérationnelle, tome 8, no V3 (1974), p. 73-103 <http://www.numdam.org/item?id=RO_1974__8_3_73_0> © AFCET, 1974, tous droits réservés. L’accès aux archives de la revue « Revue française d’automatique, in- formatique, recherche opérationnelle. Recherche opérationnelle » implique l’accord avec les conditions générales d’utilisation (http://www.numdam.org/ conditions). Toute utilisation commerciale ou impression systématique est constitutive d’une infraction pénale. Toute copie ou impression de ce fi- chier doit contenir la présente mention de copyright. Article numérisé dans le cadre du programme Numérisation de documents anciens mathématiques http://www.numdam.org/ R.A.LR.O. (8* année, novembre 1974, V-3, p. 73 à 104) NONLINEAR OPTIMIZATION USING THE-GENERALIZED REDUCED GRADIENT METHOD (*) by Léon S. LÀSDON, Richard L. Fox and Margery W. RATNER Abstract. — This paper describes the principles and logic o f a System of computer programs for solving nonlinear optimization problems using a Generalized Reduced Gradient Algorithm, The work is based on earlier work of Âbadie (2). Since this paper was written, many changes have been made in the logic, and significant computational expérience has been obtained* We hope to report on this in a later paper. 1. INTRODUCTION Generalized Reduced Gradient methods are algorithms for solving non- linear programs of gênerai structure. This paper discusses the basic principles of GRG, and constructs a spécifie GRG algorithm. The logic of a computer program implementing this algorithm is presented by means of flow charts and discussion. -
Combinatorial Structures in Nonlinear Programming
Combinatorial Structures in Nonlinear Programming Stefan Scholtes¤ April 2002 Abstract Non-smoothness and non-convexity in optimization problems often arise because a combinatorial structure is imposed on smooth or convex data. The combinatorial aspect can be explicit, e.g. through the use of ”max”, ”min”, or ”if” statements in a model, or implicit as in the case of bilevel optimization where the combinatorial structure arises from the possible choices of active constraints in the lower level problem. In analyzing such problems, it is desirable to decouple the combinatorial from the nonlinear aspect and deal with them separately. This paper suggests a problem formulation which explicitly decouples the two aspects. We show that such combinatorial nonlinear programs, despite their inherent non-convexity, allow for a convex first order local optimality condition which is generic and tight. The stationarity condition can be phrased in terms of Lagrange multipliers which allows an extension of the popular sequential quadratic programming (SQP) approach to solve these problems. We show that the favorable local convergence properties of SQP are retained in this setting. The computational effectiveness of the method depends on our ability to solve the subproblems efficiently which, in turn, depends on the representation of the governing combinatorial structure. We illustrate the potential of the approach by applying it to optimization problems with max-min constraints which arise, for example, in robust optimization. 1 Introduction Nonlinear programming is nowadays regarded as a mature field. A combination of important algorithmic developments and increased computing power over the past decades have advanced the field to a stage where the majority of practical prob- lems can be solved efficiently by commercial software. -
Nonlinear Integer Programming ∗
Nonlinear Integer Programming ∗ Raymond Hemmecke, Matthias Koppe,¨ Jon Lee and Robert Weismantel Abstract. Research efforts of the past fifty years have led to a development of linear integer programming as a mature discipline of mathematical optimization. Such a level of maturity has not been reached when one considers nonlinear systems subject to integrality requirements for the variables. This chapter is dedicated to this topic. The primary goal is a study of a simple version of general nonlinear integer problems, where all constraints are still linear. Our focus is on the computational complexity of the problem, which varies significantly with the type of nonlinear objective function in combination with the underlying combinatorial structure. Nu- merous boundary cases of complexity emerge, which sometimes surprisingly lead even to polynomial time algorithms. We also cover recent successful approaches for more general classes of problems. Though no positive theoretical efficiency results are available, nor are they likely to ever be available, these seem to be the currently most successful and interesting approaches for solving practical problems. It is our belief that the study of algorithms motivated by theoretical considera- tions and those motivated by our desire to solve practical instances should and do inform one another. So it is with this viewpoint that we present the subject, and it is in this direction that we hope to spark further research. Raymond Hemmecke Otto-von-Guericke-Universitat¨ Magdeburg, FMA/IMO, Universitatsplatz¨ 2, 39106 Magdeburg, Germany, e-mail: [email protected] Matthias Koppe¨ University of California, Davis, Dept. of Mathematics, One Shields Avenue, Davis, CA, 95616, USA, e-mail: [email protected] Jon Lee IBM T.J. -
Extreme Points and Basic Solutions
EXTREME POINTS AND BASIC SOLUTIONS: In Linear Programming, the feasible region in Rn is defined by P := {x ∈ Rn | Ax = b, x ≥ 0}. The set P , as we have seen, is a convex subset of Rn. It is called a convex polytope. The term convex polyhedron refers to convex polytope which is bounded. Polytopes in two dimensions are often called polygons. Recall that the vertices of a convex polytope are what we called extreme points of that set. Recall that extreme points of a convex set are those which cannot be represented as a proper convex combination of two other (distinct) points of the convex set. It may, or may not be the case that a convex set has any extreme points as shown by the example in R2 of the strip S := {(x, y) ∈ R2 | 0 ≤ x ≤ 1, y ∈ R}. On the other hand, the square defined by the inequalities |x| ≤ 1, |y| ≤ 1 has exactly four extreme points, while the unit disk described by the ineqality x2 + y2 ≤ 1 has infinitely many. These examples raise the question of finding conditions under which a convex set has extreme points. The answer in general vector spaces is answered by one of the “big theorems” called the Krein-Milman Theorem. However, as we will see presently, our study of the linear programming problem actually answers this question for convex polytopes without needing to call on that major result. The algebraic characterization of the vertices of the feasible polytope confirms the obser- vation that we made by following the steps of the Simplex Algorithm in our introductory example. -
The Feasible Region for Consecutive Patterns of Permutations Is a Cycle Polytope
Séminaire Lotharingien de Combinatoire XX (2020) Proceedings of the 32nd Conference on Formal Power Article #YY, 12 pp. Series and Algebraic Combinatorics (Ramat Gan) The feasible region for consecutive patterns of permutations is a cycle polytope Jacopo Borga∗1, and Raul Penaguiaoy 1 1Department of Mathematics, University of Zurich, Switzerland Abstract. We study proportions of consecutive occurrences of permutations of a given size. Specifically, the feasible limits of such proportions on large permutations form a region, called feasible region. We show that this feasible region is a polytope, more precisely the cycle polytope of a specific graph called overlap graph. This allows us to compute the dimension, vertices and faces of the polytope. Finally, we prove that the limits of classical occurrences and consecutive occurrences are independent, in some sense made precise in the extended abstract. As a consequence, the scaling limit of a sequence of permutations induces no constraints on the local limit and vice versa. Keywords: permutation patterns, cycle polytopes, overlap graphs. This is a shorter version of the preprint [11] that is currently submitted to a journal. Many proofs and details omitted here can be found in [11]. 1 Introduction 1.1 Motivations Despite not presenting any probabilistic result here, we give some motivations that come from the study of random permutations. This is a classical topic at the interface of combinatorics and discrete probability theory. There are two main approaches to it: the first concerns the study of statistics on permutations, and the second, more recent, looks arXiv:2003.12661v2 [math.CO] 30 Jun 2020 for the limits of permutations themselves. -
Linear Programming
Stanford University | CS261: Optimization Handout 5 Luca Trevisan January 18, 2011 Lecture 5 In which we introduce linear programming. 1 Linear Programming A linear program is an optimization problem in which we have a collection of variables, which can take real values, and we want to find an assignment of values to the variables that satisfies a given collection of linear inequalities and that maximizes or minimizes a given linear function. (The term programming in linear programming, is not used as in computer program- ming, but as in, e.g., tv programming, to mean planning.) For example, the following is a linear program. maximize x1 + x2 subject to x + 2x ≤ 1 1 2 (1) 2x1 + x2 ≤ 1 x1 ≥ 0 x2 ≥ 0 The linear function that we want to optimize (x1 + x2 in the above example) is called the objective function.A feasible solution is an assignment of values to the variables that satisfies the inequalities. The value that the objective function gives 1 to an assignment is called the cost of the assignment. For example, x1 := 3 and 1 2 x2 := 3 is a feasible solution, of cost 3 . Note that if x1; x2 are values that satisfy the inequalities, then, by summing the first two inequalities, we see that 3x1 + 3x2 ≤ 2 that is, 1 2 x + x ≤ 1 2 3 2 1 1 and so no feasible solution has cost higher than 3 , so the solution x1 := 3 , x2 := 3 is optimal. As we will see in the next lecture, this trick of summing inequalities to verify the optimality of a solution is part of the very general theory of duality of linear programming. -
An Improved Sequential Quadratic Programming Method Based on Positive Constraint Sets, Chemical Engineering Transactions, 81, 157-162 DOI:10.3303/CET2081027
157 A publication of CHEMICAL ENGINEERING TRANSACTIONS VOL. 81, 2020 The Italian Association of Chemical Engineering Online at www.cetjournal.it Guest Editors: Petar S. Varbanov, Qiuwang Wang, Min Zeng, Panos Seferlis, Ting Ma, Jiří J. Klemeš Copyright © 2020, AIDIC Servizi S.r.l. DOI: 10.3303/CET2081027 ISBN 978-88-95608-79-2; ISSN 2283-9216 An Improved Sequential Quadratic Programming Method Based on Positive Constraint Sets Li Xiaa,*, Jianyang Linga, Rongshan Bia, Wenying Zhaob, Xiaorong Caob, Shuguang Xianga a State Key Laboratory Base for Eco-Chemical Engineering, College of Chemical Engineering, Qingdao University of Science and Technology, Zhengzhou Road No. 53, Qingdao 266042, China b Chemistry and Chemistry Engineering Faculty,Qilu Normal University, jinan, 250013, Shandong, China [email protected] SQP (sequential quadratic programming) is an effective method to solve nonlinear constraint problems, widely used in chemical process simulation optimization. At present, most optimization methods in general chemical process simulation software have the problems of slow calculation and poor convergence. To solve these problems , an improved SQP optimization method based on positive constraint sets was developed. The new optimization method was used in the general chemical process simulation software named optimization engineers , adopting the traditional Armijo type step rule, L-BFGS (Limited-Memory BFGS) algorithm and rules of positive definite matrix. Compared to the traditional SQP method, the SQP optimization method based on positive constraint sets simplifies the constraint number of corresponding subproblems. L-BFGS algorithm and rules of positive definite matrix simplify the solution of the second derivative matrix, and reduce the amount of storage in the calculation process. -
Appendix a Solving Systems of Nonlinear Equations
Appendix A Solving Systems of Nonlinear Equations Chapter 4 of this book describes and analyzes the power flow problem. In its ac version, this problem is a system of nonlinear equations. This appendix describes the most common method for solving a system of nonlinear equations, namely, the Newton-Raphson method. This is an iterative method that uses initial values for the unknowns and, then, at each iteration, updates these values until no change occurs in two consecutive iterations. For the sake of clarity, we first describe the working of this method for the case of just one nonlinear equation with one unknown. Then, the general case of n nonlinear equations and n unknowns is considered. We also explain how to directly solve systems of nonlinear equations using appropriate software. A.1 Newton-Raphson Algorithm The Newton-Raphson algorithm is described in this section. A.1.1 One Unknown Consider a nonlinear function f .x/ W R ! R. We aim at finding a value of x so that: f .x/ D 0: (A.1) .0/ To do so, we first consider a given value of x, e.g., x . In general, we have that f x.0/ ¤ 0. Thus, it is necessary to find x.0/ so that f x.0/ C x.0/ D 0. © Springer International Publishing AG 2018 271 A.J. Conejo, L. Baringo, Power System Operations, Power Electronics and Power Systems, https://doi.org/10.1007/978-3-319-69407-8 272 A Solving Systems of Nonlinear Equations Using Taylor series, we can express f x.0/ C x.0/ as: Â Ã.0/ 2 Â Ã.0/ df .x/ x.0/ d2f .x/ f x.0/ C x.0/ D f x.0/ C x.0/ C C ::: dx 2 dx2 (A.2) .0/ Considering only the first two terms in Eq. -
Maxlik: Maximum Likelihood Estimation and Related Tools
Package ‘maxLik’ July 26, 2021 Version 1.5-2 Date 2021-07-26 Title Maximum Likelihood Estimation and Related Tools Depends R (>= 2.4.0), miscTools (>= 0.6-8), methods Imports sandwich, generics Suggests MASS, clue, dlm, plot3D, tibble, tinytest Description Functions for Maximum Likelihood (ML) estimation, non-linear optimization, and related tools. It includes a unified way to call different optimizers, and classes and methods to handle the results from the Maximum Likelihood viewpoint. It also includes a number of convenience tools for testing and developing your own models. License GPL (>= 2) ByteCompile yes NeedsCompilation no Author Ott Toomet [aut, cre], Arne Henningsen [aut], Spencer Graves [ctb], Yves Croissant [ctb], David Hugh-Jones [ctb], Luca Scrucca [ctb] Maintainer Ott Toomet <[email protected]> Repository CRAN Date/Publication 2021-07-26 17:30:02 UTC R topics documented: maxLik-package . .2 activePar . .4 AIC.maxLik . .5 bread.maxLik . .6 compareDerivatives . .7 1 2 maxLik-package condiNumber . .9 confint.maxLik . 11 fnSubset . 12 gradient . 13 hessian . 15 logLik.maxLik . 16 maxBFGS . 17 MaxControl-class . 21 maximType . 24 maxLik . 25 maxNR . 27 maxSGA . 33 maxValue . 38 nIter . 39 nObs.maxLik . 40 nParam.maxim . 41 numericGradient . 42 objectiveFn . 44 returnCode . 45 storedValues . 46 summary.maxim . 47 summary.maxLik . 49 sumt............................................. 50 tidy.maxLik . 52 vcov.maxLik . 54 Index 56 maxLik-package Maximum Likelihood Estimation Description This package contains a set of functions and tools for Maximum Likelihood (ML) estimation. The focus of the package is on non-linear optimization from the ML viewpoint, and it provides several convenience wrappers and tools, like BHHH algorithm, variance-covariance matrix and standard errors. -
Math 112 Review for Exam Ii (Ws 12 -18)
MATH 112 REVIEW FOR EXAM II (WS 12 -18) I. Derivative Rules • There will be a page or so of derivatives on the exam. You should know how to apply all the derivative rules. (WS 12 and 13) II. Functions of One Variable • Be able to find local optima, and to distinguish between local and global optima. • Be able to find the global maximum and minimum of a function y = f(x) on the interval from x = a to x = b, using the fact that optima may only occur where f(x) has a horizontal tangent line and at the endpoints of the interval. Step 1: Compute the derivative f’(x). Step 2: Find all critical points (values of x at which f’(x) = 0.) Step 3: Plug all the values of x from Step 2 that are in the interval from a to b and the endpoints of the interval into the function f(x). Step 4: Sketch a rough graph of f(x) and pick off the global max and min. • Understand the following application: Maximizing TR(q) starting with a demand curve. (WS 15) • Understand how to use the Second Derivative Test. (WS 16) If a is a critical point for f(x) (that is, f’(a) = 0), and the second derivative is: f ’’(a) > 0, then f(x) has a local min at x = a. f ’’(a) < 0, then f(x) has a local max at x = a. f ’’(a) = 0, then the test tells you nothing. IMPORTANT! For the Second Derivative Test to work, you must have f’(a) = 0 to start with! For example, if f ’’(a) > 0 but f ’(a) ≠ 0, then the graph of f(x) is concave up at x = a but f(x) does not have a local min there. -
The Feasible Region for Consecutive Patterns of Permutations Is a Cycle Polytope
Algebraic Combinatorics Draft The feasible region for consecutive patterns of permutations is a cycle polytope Jacopo Borga & Raul Penaguiao Abstract. We study proportions of consecutive occurrences of permutations of a given size. Specifically, the limit of such proportions on large permutations forms a region, called feasible region. We show that this feasible region is a polytope, more precisely the cycle polytope of a specific graph called overlap graph. This allows us to compute the dimension, vertices and faces of the polytope, and to determine the equations that define it. Finally we prove that the limit of classical occurrences and consecutive occurrences are in some sense independent. As a consequence, the scaling limit of a sequence of permutations induces no constraints on the local limit and vice versa. (1,0,0,0,0,0) 1 1 (0; 2 ; 2 ; 0; 0; 0) 1 1 1 1 (0; 0; 2 ; 2 ; 0; 0) (0; 2 ; 0; 2 ; 0; 0) 1 1 (0; 0; 0; 2 ; 2 ; 0) arXiv:1910.02233v2 [math.CO] 30 Jun 2020 Figure 1. The four-dimensional polytope P3 given by the six pat- terns of size three (see Eq. (2) for a precise definition). We highlight in light-blue one of the six three-dimensional facets of P3. This facet is a pyramid with square base. The polytope itself is a four-dimensional pyramid, whose base is the highlighted facet. This paper has been prepared using ALCO author class on 1st July 2020. Keywords. Permutation patterns, cycle polytopes, overlap graphs. Acknowledgements. This work was completed with the support of the SNF grants number 200021- 172536 and 200020-172515.