DOCSLIB.ORG

Mathematical Optimization Techniques

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Mathematical Optimization Techniques MATHEMATICAL OPTIMIZATION TECHNIQUES S. Russenschuck CERN, 1211 Geneva 23, Switzerland Abstract From the beginning the ROXIE program was structured such that mathemati- cal optimization techniques can be applied to the design of the superconduct- ing magnets. With the concept of features it is possible to create the complex coil assemblies in 2 and 3 dimensions with only a small number of engineer- ing data which can then be addressed as design variables of the optimization problem. In this chapter some background information on the application of mathematical optimization techniques is given. 1 Historical overview Mathematical optimization including numerical techniques such as linear and nonlinear programming, integer programming, network flow theory and dynamic optimization has its origin in operations research developed in world war II, e.g., Morse and Kimball 1950 [45]. Most of the real-world optimization problems involve multiple conflicting objectives which should be considered simultaneously, so-called vector-optimization problems. The solution process for vector-optimization problems is threefold, based on decision-making methods, methods to treat nonlinear constraints and optimization algorithms to min- imize the objective function. Methods for decision-making, based on the optimality criterion by Pareto in 1896 [48], have been introduced and applied to a wide range of problems in economics by Marglin 1966 [42], Geoffrion 1968 [18] and Fandel 1972 [12]. The theory of nonlinear programming with constraints is based on the opti- mality criterion by Kuhn and Tucker, 1951 [37]. Methods for the treatment of nonlinear constraints have been developed by Zoutdendijk 1960 [70], Fiacco and McCormick 1968 [13] and Rockafellar 1973 [54] among others. Numerous optimization algorithms both using deterministic and stochastic elements have been developed in the sixties and covered in the books by Wilde 1964 [67], Rosenbrock 1966 [55], Him- melblau 1972 [25], Brent 1973 [5], and Schwefel 1977 [62]. Researchers tend to come back to genetic and evolutionary algorithms recently as they are suited for parallel processing, finding global optima, and are reported to be suitable for a large number of design variables Fogel 1994 [15], Holland 1992 [26]. Mathematical optimization techniques have been applied to computational electromagnetics al- ready for decades. Halbach 1967 [23] introduced a method for optimizing coil arrangements and pole shapes of magnets by means of finite element (FE) field calculation. Armstrong, Fan, Simkin and Trow- bridge 1982 [2] combined optimization algorithms with the volume integral method for the pole profile optimization of a H-magnet. Girdinio, Molfino, Molinari and Viviani 1983 [20] optimized a profile of an electrode. These attempts tended to be application-specific, however. Only since the late 80 th, have numerical field calculation packages for both 2d and 3d applications been placed in an optimization environment. Reasons for this delay have included constraints in computing power, problems with dis- continuities and nondifferentiabilities in the objective function arising from FE meshes, accuracy of the field solution and software implementation problems. A small selection of papers can be found in the references. The variety of methods applied shows that no general method exists to solve nonlinear optimiza- tion problems in computational electromagnetics in the same way that the simplex algorithm exists to 60 solve linear problems. There are many different applications in computational electromagnetics and each one requires its own particular procedure. Some optimization procedures are described in the following sections that have been proven efficient for problems in computational electromagnetics and are provided for general use in the ROXIE program. 2 Pareto-optimality Most of the real-world optimization problems involve multiple conflicting objectives that must be mutu- ally reconciled. Characteristic for these so-called vector-optimization problems is the appearance of an objective-conflict where the individual solutions for each single objective function differ and no solution exists where all the objectives reach their individual minimum. A vector-optimization problem in a standardized mathematical form reads: ¢¡¤£¦¥§ ©¨ ¨ ¢¡¤£¦¥§ ¨ ¨ ¨ (1) ¨ +*-,./102!3#&%4'5# "!$#&%(')# subject to , ¨ £ ¡ 76"8 "9: * (2) ¨ 7"8 =< "9:+>? /10; (3) ¥ 6 6 GH"9: @DA @DAFE @BACA (4) ¨ IJ K @ @ @ with the design variable vector % and the in general nonlinear objective functions © L¨¨ @BACE arranged in the vector . The @BACA and are the lower respectively upper bounds for the design variables. For the definition of the optimal solution of the vector-optimization problem we apply the optimality criterion by Pareto originally introduced for problems in economics Pareto [48], Stadler [65]. ¨ ¨ M A Pareto-optimal solution is given when there exists no solution in the feasible domain N O ¨ , ¨ 0 ¨ £ ¡ ¥§] QPR#&%TSU* 76"8VW/ "8V 6 6 "9 V3<Z"9 >?V[GB\9 @BA @BAFEYX @BAFA for which M K^ 2¨ 26_K^ L¨ Pba 9c(d X ` (5) M ¥ K^ 2¨ 2e_K^ L¨ HfUg&h;iDGkj;hmlUiDf j Pba 9c(d ` (6) A design where the improvement of one objective causes the degradation of at least one other objective, is an element of the Pareto-optimal solution set. It is clear that this definition yields a set of solutions rather than one unique solution. Fig. 1. shows a geometric interpretation of Pareto-optimal solutions for two conflicting objectives. 3 Methods of decision-making Applying mathematical optimization routines requires a decision-making method that guarantees a solu- tion from the Pareto-optimal solution set. Below some methods are described that have been applied to computational electromagnetics, author’s papers [56], [57]. A comprehensive overview can be found in Cohon [7]. 61 o¦p nrq ou ots oxu s|v sv u^v uv ots.ymz { v w v w ou|ymz { ouy}z { ots n ots.ymz { ~x Fig. 1: Pareto-optimal solutions. Point 1 and 2 are not pareto-optimal because one objective can always be improved without deteriorating the other 3.1 Objective weighting The objective weighting function, Kuhn and Tucker [37] is the sum of the weighted objectives and results in the minimization problem: O¢ ¨ ¨ ¡¤£¦¥ ¨ ¨ ] © iK2K^ LS P N (7) K with the weighting factors i.K representing the users preference. For convex optimization problems where ¨ ¨ ¨ ¨ ¨ ¨ Pba 9c(d ¢ ©P 86"6\9 K1 -( 9$4 -76\$K1 |( 9$4K ` for all and N , yields it can be proved indirectly, Fandel [12], that eq. (7) is a minimization problem with a unique Pareto- optimal solution. The problem is to find the appropriate weighting factors in particular when the objec- tives have different numerical values and sensitivity. Using objective weighting results therefore in an iterative solution process where a number of optimizations have to be performed with updated weighting factors. 3.2 Distance function The problem of choosing the weighting factors appropriately also occurs when the distance function M method, Charnes and Cooper [6], is applied. Most common is a least squares objective function. The K QP¨ N are the requirements for the optimumk design. The minimization problem reads for and the norm + " ;S S ©S S @ @ @ ¨ % M ¡¤£¦¥ ¨ ¨ ¨ ¨ ¡¤£t¥ ¨ © ¨ M ¡¤£¦¥ ¨ ¨ i.Km HK^ K (8) K- M For convex functions and for K taken as the minimal individual solutions it can be proved, in the same manner as for the objective weighting function, that (8) has an unique Pareto-optimal solution. The dis- advantage of least squares objective functions with the Euclidean norm is the low sensitivity for residuals smaller than one. Therefore sufficiently high weighting factors i.K have to be introduced. If the absolute value norm is applied, the disadvantage is the nondifferentiable objective function in the optimum. 62 3.3 Constraint formulation The problem with the weighting factors can be overcome by defining the problem in the constraint formulation, Marglin [42]. Only one of the objectives is minimized and the others are considered by constraints. The resulting optimization problem reads: ¡¤£¦¥ ¨ , (9) ¨ HgYK 6"8 K^ (10) £ 9cV gYK ` X[` and additional constraints, eq. (2)-(4). The represent the minimum request value specified by the user for the k-th objective. Combining (10) and (2) and, because they can be *¢¡£ P ¨ ¤ ¨ treated§Hseparately, omitting the bounds for the design variables (4) yields in a vector notation ¨ ¨ ¨ #&¥L¦ / P©# ¡ : ¡¤£¦¥ ¨ , (11) ¡ ¨ * 6 8 ¨ ¤ ¨ ¨ (12) ¡ ¨ ¨ ¨ ¨ / 8 ¨ (13) 3.4 Sensitivity analysis The constraint formulation has the advantage that a sensitivity analysis can be performed using the nec- ¨ essary optimality conditions at the optimum point M which read, see Luenberger [38]: M ¡ M ¡ M ± ¨ ¨ ¨ ¨ ¨ ª¬« "ª¬« ,+ W ª°« * ª°« / 8 ¨ ¯ ¨ ¨ ­ ­ ­ ­©® (14) ¡ M ¨ * H 8 ¨ ¤ ¨ ¨ (15) ¡ M ¨ ¨ ¨ ¨ / H 8 ¨ (16) 8 ¨ ¯² ¨ (17) ± ¨ ¯ The ¨ are the vectors of the corresponding Lagrange multipliers. Equations (14) - (17) are the Kuhn- Tucker equations. The gradient of the Lagrange function has to be zero, and the Lagrange multipliers of the active inequality constraints have to take values greater than zero, otherwise it would be possible to decrease the value of a constraint without increasing the objective function, which is of course not characteristic for an optimal point. By means of the corresponding Lagrange function L it can also be proved that (11) - (13) is a minimization problem with a unique Pareto-optimal solution if all constraints are active. A non-active constraint would be equivalent to a zero weight in the weighting function. The Lagrange-multipliers are estimated by solving the linear equation system (14) by means of the variational problem M ± ¨ ¡¤£¦¥ ¡¤£¦¥ « ª « ª°« H L¨ H·¶ ¹¸ « « ¯ ¨ µ µ ® ­ ³´ ³´ (18) 63 ¸ with the gradients of the constraints arranged in the matrices ¶ and . The Lagrange multipliers are a measure of the price which has to be paid when the constraint is decreased. Mathematically this relationship is expressed by [38] M ¨ ªº|H 7" ¯ ¨ (19) M ± ¨ ¨ 7" ª »3H (20) 3.5 Payoff table A tool which provides the decision maker with a lot of information about the hidden resources of a design is the payoff-table.
Load more

Recommended publications
  • Nonlinear Optimization Using the Generalized Reduced Gradient Method Revue Française D’Automatique, Informatique, Recherche Opé- Rationnelle
    REVUE FRANÇAISE D’AUTOMATIQUE, INFORMATIQUE, RECHERCHE OPÉRATIONNELLE.RECHERCHE OPÉRATIONNELLE LEON S. LASDON RICHARD L. FOX MARGERY W. RATNER Nonlinear optimization using the generalized reduced gradient method Revue française d’automatique, informatique, recherche opé- rationnelle. Recherche opérationnelle, tome 8, no V3 (1974), p. 73-103 <http://www.numdam.org/item?id=RO_1974__8_3_73_0> © AFCET, 1974, tous droits réservés. L’accès aux archives de la revue « Revue française d’automatique, in- formatique, recherche opérationnelle. Recherche opérationnelle » implique l’accord avec les conditions générales d’utilisation (http://www.numdam.org/ conditions). Toute utilisation commerciale ou impression systématique est constitutive d’une infraction pénale. Toute copie ou impression de ce fi- chier doit contenir la présente mention de copyright. Article numérisé dans le cadre du programme Numérisation de documents anciens mathématiques http://www.numdam.org/ R.A.LR.O. (8* année, novembre 1974, V-3, p. 73 à 104) NONLINEAR OPTIMIZATION USING THE-GENERALIZED REDUCED GRADIENT METHOD (*) by Léon S. LÀSDON, Richard L. Fox and Margery W. RATNER Abstract. — This paper describes the principles and logic o f a System of computer programs for solving nonlinear optimization problems using a Generalized Reduced Gradient Algorithm, The work is based on earlier work of Âbadie (2). Since this paper was written, many changes have been made in the logic, and significant computational expérience has been obtained* We hope to report on this in a later paper. 1. INTRODUCTION Generalized Reduced Gradient methods are algorithms for solving non- linear programs of gênerai structure. This paper discusses the basic principles of GRG, and constructs a spécifie GRG algorithm. The logic of a computer program implementing this algorithm is presented by means of flow charts and discussion.
    [Show full text]
  • 4. Convex Optimization Problems
    Convex Optimization — Boyd & Vandenberghe 4. Convex optimization problems optimization problem in standard form • convex optimization problems • quasiconvex optimization • linear optimization • quadratic optimization • geometric programming • generalized inequality constraints • semidefinite programming • vector optimization • 4–1 Optimization problem in standard form minimize f0(x) subject to f (x) 0, i =1,...,m i ≤ hi(x)=0, i =1,...,p x Rn is the optimization variable • ∈ f : Rn R is the objective or cost function • 0 → f : Rn R, i =1,...,m, are the inequality constraint functions • i → h : Rn R are the equality constraint functions • i → optimal value: p⋆ = inf f (x) f (x) 0, i =1,...,m, h (x)=0, i =1,...,p { 0 | i ≤ i } p⋆ = if problem is infeasible (no x satisfies the constraints) • ∞ p⋆ = if problem is unbounded below • −∞ Convex optimization problems 4–2 Optimal and locally optimal points x is feasible if x dom f and it satisfies the constraints ∈ 0 ⋆ a feasible x is optimal if f0(x)= p ; Xopt is the set of optimal points x is locally optimal if there is an R> 0 such that x is optimal for minimize (over z) f0(z) subject to fi(z) 0, i =1,...,m, hi(z)=0, i =1,...,p z x≤ R k − k2 ≤ examples (with n =1, m = p =0) f (x)=1/x, dom f = R : p⋆ =0, no optimal point • 0 0 ++ f (x)= log x, dom f = R : p⋆ = • 0 − 0 ++ −∞ f (x)= x log x, dom f = R : p⋆ = 1/e, x =1/e is optimal • 0 0 ++ − f (x)= x3 3x, p⋆ = , local optimum at x =1 • 0 − −∞ Convex optimization problems 4–3 Implicit constraints the standard form optimization problem has an implicit
    [Show full text]
  • Metaheuristics1
    METAHEURISTICS1 Kenneth Sörensen University of Antwerp, Belgium Fred Glover University of Colorado and OptTek Systems, Inc., USA 1 Definition A metaheuristic is a high-level problem-independent algorithmic framework that provides a set of guidelines or strategies to develop heuristic optimization algorithms (Sörensen and Glover, To appear). Notable examples of metaheuristics include genetic/evolutionary algorithms, tabu search, simulated annealing, and ant colony optimization, although many more exist. A problem-specific implementation of a heuristic optimization algorithm according to the guidelines expressed in a metaheuristic framework is also referred to as a metaheuristic. The term was coined by Glover (1986) and combines the Greek prefix meta- (metá, beyond in the sense of high-level) with heuristic (from the Greek heuriskein or euriskein, to search). Metaheuristic algorithms, i.e., optimization methods designed according to the strategies laid out in a metaheuristic framework, are — as the name suggests — always heuristic in nature. This fact distinguishes them from exact methods, that do come with a proof that the optimal solution will be found in a finite (although often prohibitively large) amount of time. Metaheuristics are therefore developed specifically to find a solution that is “good enough” in a computing time that is “small enough”. As a result, they are not subject to combinatorial explosion – the phenomenon where the computing time required to find the optimal solution of NP- hard problems increases as an exponential function of the problem size. Metaheuristics have been demonstrated by the scientific community to be a viable, and often superior, alternative to more traditional (exact) methods of mixed- integer optimization such as branch and bound and dynamic programming.
    [Show full text]
  • Combinatorial Structures in Nonlinear Programming
    Combinatorial Structures in Nonlinear Programming Stefan Scholtes¤ April 2002 Abstract Non-smoothness and non-convexity in optimization problems often arise because a combinatorial structure is imposed on smooth or convex data. The combinatorial aspect can be explicit, e.g. through the use of ”max”, ”min”, or ”if” statements in a model, or implicit as in the case of bilevel optimization where the combinatorial structure arises from the possible choices of active constraints in the lower level problem. In analyzing such problems, it is desirable to decouple the combinatorial from the nonlinear aspect and deal with them separately. This paper suggests a problem formulation which explicitly decouples the two aspects. We show that such combinatorial nonlinear programs, despite their inherent non-convexity, allow for a convex first order local optimality condition which is generic and tight. The stationarity condition can be phrased in terms of Lagrange multipliers which allows an extension of the popular sequential quadratic programming (SQP) approach to solve these problems. We show that the favorable local convergence properties of SQP are retained in this setting. The computational effectiveness of the method depends on our ability to solve the subproblems efficiently which, in turn, depends on the representation of the governing combinatorial structure. We illustrate the potential of the approach by applying it to optimization problems with max-min constraints which arise, for example, in robust optimization. 1 Introduction Nonlinear programming is nowadays regarded as a mature field. A combination of important algorithmic developments and increased computing power over the past decades have advanced the field to a stage where the majority of practical prob- lems can be solved efficiently by commercial software.
    [Show full text]
  • Nonlinear Integer Programming ∗
    Nonlinear Integer Programming ∗ Raymond Hemmecke, Matthias Koppe,¨ Jon Lee and Robert Weismantel Abstract. Research efforts of the past fifty years have led to a development of linear integer programming as a mature discipline of mathematical optimization. Such a level of maturity has not been reached when one considers nonlinear systems subject to integrality requirements for the variables. This chapter is dedicated to this topic. The primary goal is a study of a simple version of general nonlinear integer problems, where all constraints are still linear. Our focus is on the computational complexity of the problem, which varies significantly with the type of nonlinear objective function in combination with the underlying combinatorial structure. Nu- merous boundary cases of complexity emerge, which sometimes surprisingly lead even to polynomial time algorithms. We also cover recent successful approaches for more general classes of problems. Though no positive theoretical efficiency results are available, nor are they likely to ever be available, these seem to be the currently most successful and interesting approaches for solving practical problems. It is our belief that the study of algorithms motivated by theoretical considera- tions and those motivated by our desire to solve practical instances should and do inform one another. So it is with this viewpoint that we present the subject, and it is in this direction that we hope to spark further research. Raymond Hemmecke Otto-von-Guericke-Universitat¨ Magdeburg, FMA/IMO, Universitatsplatz¨ 2, 39106 Magdeburg, Germany, e-mail: [email protected] Matthias Koppe¨ University of California, Davis, Dept. of Mathematics, One Shields Avenue, Davis, CA, 95616, USA, e-mail: [email protected] Jon Lee IBM T.J.
    [Show full text]
  • 12. Coordinate Descent Methods
    EE 546, Univ of Washington, Spring 2014 12. Coordinate descent methods theoretical justifications • randomized coordinate descent method • minimizing composite objectives • accelerated coordinate descent method • Coordinate descent methods 12–1 Notations consider smooth unconstrained minimization problem: minimize f(x) x RN ∈ n coordinate blocks: x =(x ,...,x ) with x RNi and N = N • 1 n i ∈ i=1 i more generally, partition with a permutation matrix: U =[PU1 Un] • ··· n T xi = Ui x, x = Uixi i=1 X blocks of gradient: • f(x) = U T f(x) ∇i i ∇ coordinate update: • x+ = x tU f(x) − i∇i Coordinate descent methods 12–2 (Block) coordinate descent choose x(0) Rn, and iterate for k =0, 1, 2,... ∈ 1. choose coordinate i(k) 2. update x(k+1) = x(k) t U f(x(k)) − k ik∇ik among the first schemes for solving smooth unconstrained problems • cyclic or round-Robin: difficult to analyze convergence • mostly local convergence results for particular classes of problems • does it really work (better than full gradient method)? • Coordinate descent methods 12–3 Steepest coordinate descent choose x(0) Rn, and iterate for k =0, 1, 2,... ∈ (k) 1. choose i(k) = argmax if(x ) 2 i 1,...,n k∇ k ∈{ } 2. update x(k+1) = x(k) t U f(x(k)) − k i(k)∇i(k) assumptions f(x) is block-wise Lipschitz continuous • ∇ f(x + U v) f(x) L v , i =1,...,n k∇i i −∇i k2 ≤ ik k2 f has bounded sub-level set, in particular, define • ⋆ R(x) = max max y x 2 : f(y) f(x) y x⋆ X⋆ k − k ≤ ∈ Coordinate descent methods 12–4 Analysis for constant step size quadratic upper bound due to block coordinate-wise
    [Show full text]
  • Process Optimization
    Process Optimization Mathematical Programming and Optimization of Multi-Plant Operations and Process Design Ralph W. Pike Director, Minerals Processing Research Institute Horton Professor of Chemical Engineering Louisiana State University Department of Chemical Engineering, Lamar University, April, 10, 2007 Process Optimization • Typical Industrial Problems • Mathematical Programming Software • Mathematical Basis for Optimization • Lagrange Multipliers and the Simplex Algorithm • Generalized Reduced Gradient Algorithm • On-Line Optimization • Mixed Integer Programming and the Branch and Bound Algorithm • Chemical Production Complex Optimization New Results • Using one computer language to write and run a program in another language • Cumulative probability distribution instead of an optimal point using Monte Carlo simulation for a multi-criteria, mixed integer nonlinear programming problem • Global optimization Design vs. Operations • Optimal Design −Uses flowsheet simulators and SQP – Heuristics for a design, a superstructure, an optimal design • Optimal Operations – On-line optimization – Plant optimal scheduling – Corporate supply chain optimization Plant Problem Size Contact Alkylation Ethylene 3,200 TPD 15,000 BPD 200 million lb/yr Units 14 76 ~200 Streams 35 110 ~4,000 Constraints Equality 761 1,579 ~400,000 Inequality 28 50 ~10,000 Variables Measured 43 125 ~300 Unmeasured 732 1,509 ~10,000 Parameters 11 64 ~100 Optimization Programming Languages • GAMS - General Algebraic Modeling System • LINDO - Widely used in business applications
    [Show full text]
  • The Gradient Sampling Methodology
    The Gradient Sampling Methodology James V. Burke∗ Frank E. Curtisy Adrian S. Lewisz Michael L. Overtonx January 14, 2019 1 Introduction in particular, this leads to calling the negative gra- dient, namely, −∇f(x), the direction of steepest de- The principal methodology for minimizing a smooth scent for f at x. However, when f is not differentiable function is the steepest descent (gradient) method. near x, one finds that following the negative gradient One way to extend this methodology to the minimiza- direction might offer only a small amount of decrease tion of a nonsmooth function involves approximating in f; indeed, obtaining decrease from x along −∇f(x) subdifferentials through the random sampling of gra- may be possible only with a very small stepsize. The dients. This approach, known as gradient sampling GS methodology is based on the idea of stabilizing (GS), gained a solid theoretical foundation about a this definition of steepest descent by instead finding decade ago [BLO05, Kiw07], and has developed into a a direction to approximately solve comprehensive methodology for handling nonsmooth, potentially nonconvex functions in the context of op- min max gT d; (2) kdk2≤1 g2@¯ f(x) timization algorithms. In this article, we summarize the foundations of the GS methodology, provide an ¯ where @f(x) is the -subdifferential of f at x [Gol77]. overview of the enhancements and extensions to it To understand the context of this idea, recall that that have developed over the past decade, and high- the (Clarke) subdifferential of a locally Lipschitz f light some interesting open questions related to GS.
    [Show full text]
  • An Improved Sequential Quadratic Programming Method Based on Positive Constraint Sets, Chemical Engineering Transactions, 81, 157-162 DOI:10.3303/CET2081027
    157 A publication of CHEMICAL ENGINEERING TRANSACTIONS VOL. 81, 2020 The Italian Association of Chemical Engineering Online at www.cetjournal.it Guest Editors: Petar S. Varbanov, Qiuwang Wang, Min Zeng, Panos Seferlis, Ting Ma, Jiří J. Klemeš Copyright © 2020, AIDIC Servizi S.r.l. DOI: 10.3303/CET2081027 ISBN 978-88-95608-79-2; ISSN 2283-9216 An Improved Sequential Quadratic Programming Method Based on Positive Constraint Sets Li Xiaa,*, Jianyang Linga, Rongshan Bia, Wenying Zhaob, Xiaorong Caob, Shuguang Xianga a State Key Laboratory Base for Eco-Chemical Engineering, College of Chemical Engineering, Qingdao University of Science and Technology, Zhengzhou Road No. 53, Qingdao 266042, China b Chemistry and Chemistry Engineering Faculty,Qilu Normal University, jinan, 250013, Shandong, China [email protected] SQP (sequential quadratic programming) is an effective method to solve nonlinear constraint problems, widely used in chemical process simulation optimization. At present, most optimization methods in general chemical process simulation software have the problems of slow calculation and poor convergence. To solve these problems , an improved SQP optimization method based on positive constraint sets was developed. The new optimization method was used in the general chemical process simulation software named optimization engineers , adopting the traditional Armijo type step rule, L-BFGS (Limited-Memory BFGS) algorithm and rules of positive definite matrix. Compared to the traditional SQP method, the SQP optimization method based on positive constraint sets simplifies the constraint number of corresponding subproblems. L-BFGS algorithm and rules of positive definite matrix simplify the solution of the second derivative matrix, and reduce the amount of storage in the calculation process.
    [Show full text]
  • IEOR 269, Spring 2010 Integer Programming and Combinatorial Optimization
    IEOR 269, Spring 2010 Integer Programming and Combinatorial Optimization Professor Dorit S. Hochbaum Contents 1 Introduction 1 2 Formulation of some ILP 2 2.1 0-1 knapsack problem . 2 2.2 Assignment problem . 2 3 Non-linear Objective functions 4 3.1 Production problem with set-up costs . 4 3.2 Piecewise linear cost function . 5 3.3 Piecewise linear convex cost function . 6 3.4 Disjunctive constraints . 7 4 Some famous combinatorial problems 7 4.1 Max clique problem . 7 4.2 SAT (satisfiability) . 7 4.3 Vertex cover problem . 7 5 General optimization 8 6 Neighborhood 8 6.1 Exact neighborhood . 8 7 Complexity of algorithms 9 7.1 Finding the maximum element . 9 7.2 0-1 knapsack . 9 7.3 Linear systems . 10 7.4 Linear Programming . 11 8 Some interesting IP formulations 12 8.1 The fixed cost plant location problem . 12 8.2 Minimum/maximum spanning tree (MST) . 12 9 The Minimum Spanning Tree (MST) Problem 13 i IEOR269 notes, Prof. Hochbaum, 2010 ii 10 General Matching Problem 14 10.1 Maximum Matching Problem in Bipartite Graphs . 14 10.2 Maximum Matching Problem in Non-Bipartite Graphs . 15 10.3 Constraint Matrix Analysis for Matching Problems . 16 11 Traveling Salesperson Problem (TSP) 17 11.1 IP Formulation for TSP . 17 12 Discussion of LP-Formulation for MST 18 13 Branch-and-Bound 20 13.1 The Branch-and-Bound technique . 20 13.2 Other Branch-and-Bound techniques . 22 14 Basic graph definitions 23 15 Complexity analysis 24 15.1 Measuring quality of an algorithm .
    [Show full text]
  • A New Algorithm of Nonlinear Conjugate Gradient Method with Strong Convergence*
    Volume 27, N. 1, pp. 93–106, 2008 Copyright © 2008 SBMAC ISSN 0101-8205 www.scielo.br/cam A new algorithm of nonlinear conjugate gradient method with strong convergence* ZHEN-JUN SHI1,2 and JINHUA GUO2 1College of Operations Research and Management, Qufu Normal University Rizhao, Sahndong 276826, P.R. China 2Department of Computer and Information Science, University of Michigan Dearborn, Michigan 48128-1491, USA E-mails: [email protected]; [email protected] / [email protected] Abstract. The nonlinear conjugate gradient method is a very useful technique for solving large scale minimization problems and has wide applications in many fields. In this paper, we present a new algorithm of nonlinear conjugate gradient method with strong convergence for unconstrained minimization problems. The new algorithm can generate an adequate trust region radius automatically at each iteration and has global convergence and linear convergence rate under some mild conditions. Numerical results show that the new algorithm is efficient in practical computation and superior to other similar methods in many situations. Mathematical subject classification: 90C30, 65K05, 49M37. Key words: unconstrained optimization, nonlinear conjugate gradient method, global conver- gence, linear convergence rate. 1 Introduction Consider an unconstrained minimization problem min f (x), x ∈ Rn, (1) where Rn is an n-dimensional Euclidean space and f : Rn −→ R is a continu- ously differentiable function. #724/07. Received: 10/V/07. Accepted: 24/IX/07. *The work was supported in part by NSF CNS-0521142, USA. 94 A NEW ALGORITHM OF NONLINEAR CONJUGATE GRADIENT METHOD When n is very large (for example, n > 106) the related problem is called large scale minimization problem.
    [Show full text]
  • Introduction to Integer Programming – Integer Programming Models
    15.053/8 March 14, 2013 Introduction to Integer Programming – Integer programming models 1 Quotes of the Day “Somebody who thinks logically is a nice contrast to the real world.” -- The Law of Thumb “Take some more tea,” the March Hare said to Alice, very earnestly. “I’ve had nothing yet,” Alice replied in an offended tone, “so I can’t take more.” “You mean you can’t take less,” said the Hatter. “It’s very easy to take more than nothing.” -- Lewis Carroll in Alice in Wonderland 2 Combinatorial optimization problems INPUT: A description of the data for an instance of the problem FEASIBLE SOLUTIONS: there is a way of determining from the input whether a given solution x’ (assignment of values to decision variables) is feasible. Typically in combinatorial optimization problems there is a finite number of possible solutions. OBJECTIVE FUNCTION: For each feasible solution x’ there is an associated objective f(x’). Minimization problem. Find a feasible solution x* that minimizes f( ) among all feasible solutions. 3 Example 1: Traveling Salesman Problem INPUT: a set N of n points in the plane FEASIBLE SOLUTION: a tour that passes through each point exactly once. OBJECTIVE: minimize the length of the tour. 4 Example 2: Balanced Partition INPUT: A set of positive integers a1, …, an FEASIBLE SOLUTION: a partition of {1, 2, … n} into two disjoint sets S and T. – S ∩ T = ∅, S∪T = {1, … , n} OBJECTIVE : minimize | ∑i∈S ai - ∑i∈T ai | Example: 7, 10, 13, 17, 20, 22 These numbers sum to 89 The best split is {10, 13, 22} and {7, 17, 20}.
    [Show full text]