DOCSLIB.ORG

Optimization: Applications, Algorithms, and Computation

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Optimization: Applications, Algorithms, and Computation Optimization: Applications, Algorithms, and Computation 24 Lectures on Nonlinear Optimization and Beyond Sven Leyffer (with help from Pietro Belotti, Christian Kirches, Jeff Linderoth, Jim Luedtke, and Ashutosh Mahajan) August 30, 2016 To my wife Gwen my parents Inge and Werner; and my teacher and mentor Roger. This manuscript has been created by the UChicago Argonne, LLC, Operator of Argonne National Laboratory (“Argonne”) under Contract No. DE-AC02-06CH11357 with the U.S. Department of Energy. The U.S. Government retains for itself, and others acting on its behalf, a paid-up, nonexclusive, irrevocable worldwide license in said article to reproduce, prepare derivative works, distribute copies to the public, and perform publicly and display publicly, by or on behalf of the Government. This work was supported by the Office of Advanced Scientific Computing Research, Office of Science, U.S. Department of Energy, under Contract DE-AC02-06CH11357. ii Contents I Introduction, Applications, and Modeling1 1 Introduction to Optimization3 1.1 Objective Function and Constraints...............................3 1.2 Classification of Optimization Problems............................4 1.2.1 Classification by Constraint Type...........................4 1.2.2 Classification by Variable Type............................5 1.2.3 Classification by Functional Forms..........................6 1.3 A First Optimization Example.................................6 1.4 Course Outline.........................................7 1.5 Exercises............................................7 2 Applications of Optimization9 2.1 Power-System Engineering: Electrical Transmission Planning................9 2.1.1 Transmission Planning Formulation.......................... 10 2.2 Other Power Systems Applications............................... 11 2.2.1 The Power-Flow Equations and Network Expansion................. 11 2.2.2 Blackout Prevention in National Power Grid...................... 12 2.2.3 Optimal Unit Commitment for Power-Grid...................... 13 2.3 A Control Application: Optimal Transition to Clean Energy.................. 14 2.3.1 Model Description and Background.......................... 15 2.3.2 Other Control Applications............................... 17 2.4 Design of Complex Systems.................................. 17 2.5 Exercises............................................ 18 II Unconstrained and Bound-Constrained Optimization 19 3 Methods for Unconstrained Optimization 21 3.1 Optimality Conditions for Unconstrained Optimization.................... 21 3.1.1 Lines and Restrictions along Lines........................... 22 3.1.2 Local and Global Minimizers............................. 23 3.2 Iterative Methods for Unconstrained Optimization....................... 24 3.2.1 General Structure of Line-Search Methods for Unconstrained Optimization..... 24 3.2.2 Steepest Descend and Armijo Line Search....................... 25 3.3 Exercises............................................ 26 iii iv CONTENTS 4 Newton and Quasi-Newton Methods 27 4.1 Quadratic Models and Newton’s Method............................ 27 4.1.1 Modifying the Hessian to Ensure Descend....................... 30 4.2 Quasi-Newton Methods..................................... 31 4.2.1 The Rank-One Quasi-Newton Update.......................... 32 4.2.2 The BFGS Quasi-Newton Update............................ 33 4.2.3 Limited-Memory Quasi-Newton Methods....................... 33 4.3 Exercises............................................ 35 5 Conjugate Gradient Methods 37 5.1 Conjugate Direction Methods.................................. 37 5.2 Classical Conjugate Gradient Method............................. 39 5.3 The Barzilai-Borwein Method................................. 42 5.4 Exercises............................................ 42 6 Global Convergence Techniques 43 6.1 Line-Search Methods...................................... 43 6.2 Trust-Region Methods..................................... 44 6.2.1 The Cauchy Point.................................... 46 6.2.2 Outline of Convergence Proof of Trust-Region Methods............... 47 6.2.3 Solving the Trust-Region Subproblem......................... 48 6.2.4 Solving Large-Scale Trust-Region Subproblems.................... 48 6.3 Exercises............................................ 49 7 Methods for Bound Constraints 51 7.1 Optimality Conditions for Bound-Constrained Optimization................. 51 7.2 Bound-Constrained Quadratic Optimization.......................... 52 7.2.1 Projected-Gradient Step................................ 52 7.2.2 Subspace Optimization................................. 55 7.2.3 Overall Algorithm for Bound-Constrained Quadratic Optimization.......... 55 7.3 Bound-Constrained Nonlinear Optimization.......................... 56 7.4 Exercises............................................ 56 III General Constrained Optimization 57 8 Optimality Conditions 59 8.1 Preliminaries: Definitions and Notation............................ 59 8.2 First-Order Conditions..................................... 61 8.2.1 Equality Constrained Nonlinear Programs....................... 61 8.2.2 Inequality Constrained Nonlinear Programs...................... 63 8.2.3 The Karush-Kuhn-Tucker Conditions......................... 64 8.3 Second-Order Conditions.................................... 65 8.3.1 Second-Order Conditions for Equality Constraints.................. 65 8.3.2 Second-Order Conditions for Inequality Constraints................. 65 8.4 Exercises............................................ 66 CONTENTS v 9 Linear and Quadratic Programming 67 9.1 Active-Set Method for Linear Programming.......................... 67 9.1.1 Obtaining an Initial Feasible Point for LPs...................... 69 9.2 Active-Set Method for Quadratic Programming........................ 70 9.2.1 Equality-Constrained QPs............................... 70 9.2.2 General Quadratic Programs.............................. 71 9.3 Exercises............................................ 73 10 Nonlinear Programming Methods 75 10.1 Introduction........................................... 75 10.2 Convergence Test and Termination Conditions......................... 76 10.2.1 Infeasible Stationary Points............................... 76 10.3 Approximate Subproblem: Improving a Solution Estimate.................. 77 10.3.1 Sequential Quadratic Programming for Equality Constraints............. 77 10.3.2 Sequential Linear and Quadratic Programming.................... 78 10.3.3 Interior-Point Methods................................. 80 10.4 Globalization Strategy: Convergence from Remote Starting Points.............. 82 10.4.1 Penalty and Merit Function Methods.......................... 82 10.4.2 Filter and Funnel Methods............................... 83 10.4.3 Maratos Effect and Loss of Fast Convergence..................... 84 10.5 Globalization Mechanisms................................... 84 10.5.1 Line-Search Methods.................................. 84 10.5.2 Trust-Region Methods................................. 85 10.6 Nonlinear Optimization Software: Summary.......................... 86 10.7 Exercises............................................ 86 11 Augmented Lagrangian Methods 89 11.1 Augmented Lagrangian Methods................................ 89 11.1.1 Linearly Constrained Lagrangian Methods....................... 89 11.1.2 Bound-Constrained Lagrangian (BCL) Methods.................... 90 11.1.3 Theory of Augmented Lagrangian Methods....................... 90 11.2 Towards Parallel Active-Set Methods for Quadratic Programming.............. 91 11.2.1 Outline of the Algorithm................................ 92 11.2.2 An Augmented Lagrangian Filter........................... 93 11.2.3 Active-Set Prediction and Second-Order Steps.................... 94 11.2.4 Estimating the Penalty Parameter........................... 95 11.2.5 Minimizing the Augmented Lagrangian Subproblem................. 97 11.2.6 Detailed Algorithm Statement............................. 98 11.3 Exercises............................................ 99 12 Mathematical Programs with Equilibrium Constraints 101 12.1 Introduction and Applications................................. 101 12.2 Optimality Conditions and Regularization........................... 103 12.3 Convergence of Nonlinear Optimization Methods....................... 105 12.3.1 Convergence of SQP Methods............................. 106 12.3.2 Convergence of Interior-Point Methods........................ 107 12.4 A Globally Convergent Methods: A Sequential LPEC-EQP Approach............ 110 12.5 Exercises............................................ 111 vi CONTENTS IV Mixed-Integer Nonlinear Optimization 113 13 Introduction and Modeling with Integer Variables 115 13.1 Mixed-Integer Nonlinear Programming Introduction...................... 115 13.1.1 MINLP Notation and Basic Definitions........................ 116 13.1.2 Preview of Key Building Blocks of MINLP Algorithms................ 117 13.1.3 Scope and Outline................................... 121 13.2 Nonlinear Models with Integer Variables............................ 122 13.2.1 Modeling Practices for MINLP............................ 122 13.2.2 Design of Multiproduct Batch Plants.......................... 124 13.2.3 Design of Water Distribution Networks........................ 125 13.2.4 A Dynamic Subway Operation Problem........................ 126 13.2.5 Summary of MINLP Applications........................... 128 14 Branch-and-Bound Methods 131 14.1 Deterministic Methods for Convex MINLP.........................
Load more

Recommended publications
  • Advances in Interior Point Methods and Column Generation
    Advances in Interior Point Methods and Column Generation Pablo Gonz´alezBrevis Doctor of Philosophy University of Edinburgh 2013 Declaration I declare that this thesis was composed by myself and that the work contained therein is my own, except where explicitly stated otherwise in the text. (Pablo Gonz´alezBrevis) ii To my endless inspiration and angels on earth: Paulina, Crist´obal and Esteban iii Abstract In this thesis we study how to efficiently combine the column generation technique (CG) and interior point methods (IPMs) for solving the relaxation of a selection of integer programming problems. In order to obtain an efficient method a change in the column generation technique and a new reoptimization strategy for a primal-dual interior point method are proposed. It is well-known that the standard column generation technique suffers from un- stable behaviour due to the use of optimal dual solutions that are extreme points of the restricted master problem (RMP). This unstable behaviour slows down column generation so variations of the standard technique which rely on interior points of the dual feasible set of the RMP have been proposed in the literature. Among these tech- niques, there is the primal-dual column generation method (PDCGM) which relies on sub-optimal and well-centred dual solutions. This technique dynamically adjusts the column generation tolerance as the method approaches optimality. Also, it relies on the notion of the symmetric neighbourhood of the central path so sub-optimal and well-centred solutions are obtained. We provide a thorough theoretical analysis that guarantees the convergence of the primal-dual approach even though sub-optimal solu- tions are used in the course of the algorithm.
    [Show full text]
  • Numerical Methods for Large-Scale Nonlinear Optimization
    Technical Report RAL-TR-2004-032 Numerical methods for large-scale nonlinear optimization N I M Gould D Orban Ph L Toint November 18, 2004 Council for the Central Laboratory of the Research Councils c Council for the Central Laboratory of the Research Councils Enquires about copyright, reproduction and requests for additional copies of this report should be addressed to: Library and Information Services CCLRC Rutherford Appleton Laboratory Chilton Didcot Oxfordshire OX11 0QX UK Tel: +44 (0)1235 445384 Fax: +44(0)1235 446403 Email: [email protected] CCLRC reports are available online at: http://www.clrc.ac.uk/Activity/ACTIVITY=Publications;SECTION=225; ISSN 1358-6254 Neither the Council nor the Laboratory accept any responsibility for loss or damage arising from the use of information contained in any of their reports or in any communication about their tests or investigations. RAL-TR-2004-032 Numerical methods for large-scale nonlinear optimization Nicholas I. M. Gould1;2;3, Dominique Orban4;5;6 and Philippe L. Toint7;8;9 Abstract Recent developments in numerical methods for solving large differentiable nonlinear optimization problems are reviewed. State-of-the-art algorithms for solving unconstrained, bound-constrained, linearly-constrained and nonlinearly-constrained problems are discussed. As well as important conceptual advances and theoretical aspects, emphasis is also placed on more practical issues, such as software availability. 1 Computational Science and Engineering Department, Rutherford Appleton Laboratory, Chilton, Oxfordshire, OX11 0QX, England, EU. Email: [email protected] 2 Current reports available from \http://www.numerical.rl.ac.uk/reports/reports.html". 3 This work was supported in part by the EPSRC grants GR/R46641 and GR/S42170.
    [Show full text]
  • A New Trust Region Method with Simple Model for Large-Scale Optimization ∗
    A New Trust Region Method with Simple Model for Large-Scale Optimization ∗ Qunyan Zhou,y Wenyu Sunz and Hongchao Zhangx Abstract In this paper a new trust region method with simple model for solv- ing large-scale unconstrained nonlinear optimization is proposed. By employing generalized weak quasi-Newton equations, we derive several schemes to construct variants of scalar matrices as the Hessian approx- imation used in the trust region subproblem. Under some reasonable conditions, global convergence of the proposed algorithm is established in the trust region framework. The numerical experiments on solving the test problems with dimensions from 50 to 20000 in the CUTEr li- brary are reported to show efficiency of the algorithm. Key words. unconstrained optimization, Barzilai-Borwein method, weak quasi-Newton equation, trust region method, global convergence AMS(2010) 65K05, 90C30 1. Introduction In this paper, we consider the unconstrained optimization problem min f(x); (1.1) x2Rn where f : Rn ! R is continuously differentiable with dimension n relatively large. Trust region methods are a class of powerful and robust globalization ∗This work was supported by National Natural Science Foundation of China (11171159, 11401308), the Natural Science Fund for Universities of Jiangsu Province (13KJB110007), the Fund of Jiangsu University of Technology (KYY13012). ySchool of Mathematics and Physics, Jiangsu University of Technology, Changzhou 213001, Jiangsu, China. Email: [email protected] zCorresponding author. School of Mathematical Sciences, Jiangsu Key Laboratory for NSLSCS, Nanjing Normal University, Nanjing 210023, China. Email: [email protected] xDepartment of Mathematics, Louisiana State University, USA. Email: [email protected] 1 methods for solving (1.1).
    [Show full text]
  • IBM Research Report Branch and Bound, Integer and Non-Integer
    RC22890 (W0309-010) September 3, 2003 Mathematics IBM Research Report Branch and Bound, Integer and Non-Integer Programming J. J. H. Forrest IBM Research Division Thomas J. Watson Research Center P.O. Box 218 Yorktown Heights, NY 10598 J. A. Tomlin IBM Research Division Almaden Research Center 650 Harry Road San Jose, CA 95120-6099 Research Division Almaden - Austin - Beijing - Delhi - Haifa - India - T. J. Watson - Tokyo - Zurich LIMITED DISTRIBUTION NOTICE: This report has been submitted for publication outside of IBM and will probably be copyrighted if accepted for publication. It has been issued as a Research Report for early dissemination of its contents. In view of the transfer of copyright to the outside publisher, its distribution outside of IBM prior to publication should be limited to peer communications and specific requests. After outside publication, requests should be filled only by reprints or legally obtained copies of the article (e.g. , payment of royalties). Copies may be requested from IBM T. J. Watson Research Center , P. O. Box 218, Yorktown Heights, NY 10598 USA (email: [email protected]). Some reports are available on the internet at http://domino.watson.ibm.com/library/CyberDig.nsf/home . Branch and Bound, Integer, and Non-Integer Programming J.J.H. Forrest IBM Watson Research Center, Yorktown Heights, NY 10598 J. A. Tomlin IBM Almaden Research Center, San Jose, CA 95120 {[email protected],[email protected]} Abstract In this note we review the development of the first commercial branch and bound codes at CEIR/Scicon, with particular emphasis on those features not commonly emphasized today—that is the use of branch and bound to handle discrete constructs, which, while often formally representable in terms of zero-one integer variables, can be handled more elegantly and effectively by applying branch and bound to the actual dichotomies in the model.
    [Show full text]
  • Constrained Multifidelity Optimization Using Model Calibration
    Struct Multidisc Optim (2012) 46:93–109 DOI 10.1007/s00158-011-0749-1 RESEARCH PAPER Constrained multifidelity optimization using model calibration Andrew March · Karen Willcox Received: 4 April 2011 / Revised: 22 November 2011 / Accepted: 28 November 2011 / Published online: 8 January 2012 c Springer-Verlag 2012 Abstract Multifidelity optimization approaches seek to derivative-free and sequential quadratic programming meth- bring higher-fidelity analyses earlier into the design process ods. The method uses approximately the same number of by using performance estimates from lower-fidelity models high-fidelity analyses as a multifidelity trust-region algo- to accelerate convergence towards the optimum of a high- rithm that estimates the high-fidelity gradient using finite fidelity design problem. Current multifidelity optimization differences. methods generally fall into two broad categories: provably convergent methods that use either the high-fidelity gradient Keywords Multifidelity · Derivative-free · Optimization · or a high-fidelity pattern-search, and heuristic model cali- Multidisciplinary · Aerodynamic design bration approaches, such as interpolating high-fidelity data or adding a Kriging error model to a lower-fidelity function. This paper presents a multifidelity optimization method that bridges these two ideas; our method iteratively calibrates Nomenclature lower-fidelity information to the high-fidelity function in order to find an optimum of the high-fidelity design prob- A Active and violated constraint Jacobian lem. The
    [Show full text]
  • ILOG AMPL CPLEX System Version 9.0 User's Guide
    ILOG AMPL CPLEX System Version 9.0 User’s Guide Standard (Command-line) Version Including CPLEX Directives September 2003 Copyright © 1987-2003, by ILOG S.A. All rights reserved. ILOG, the ILOG design, CPLEX, and all other logos and product and service names of ILOG are registered trademarks or trademarks of ILOG in France, the U.S. and/or other countries. JavaTM and all Java-based marks are either trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries. Microsoft, Windows, and Windows NT are either trademarks or registered trademarks of Microsoft Corporation in the U.S. and other countries. All other brand, product and company names are trademarks or registered trademarks of their respective holders. AMPL is a registered trademark of AMPL Optimization LLC and is distributed under license by ILOG.. CPLEX is a registered trademark of ILOG.. Printed in France. CO N T E N T S Table of Contents Chapter 1 Introduction . 1 Welcome to AMPL . .1 Using this Guide. .1 Installing AMPL . .2 Requirements. .2 Unix Installation . .3 Windows Installation . .3 AMPL and Parallel CPLEX. 4 Licensing . .4 Usage Notes . .4 Installed Files . .5 Chapter 2 Using AMPL. 7 Running AMPL . .7 Using a Text Editor . .7 Running AMPL in Batch Mode . .8 Chapter 3 AMPL-Solver Interaction . 11 Choosing a Solver . .11 Specifying Solver Options . .12 Initial Variable Values and Solvers. .13 ILOG AMPL CPLEX SYSTEM 9.0 — USER’ S GUIDE v Problem and Solution Files. .13 Saving temporary files . .14 Creating Auxiliary Files . .15 Running Solvers Outside AMPL. .16 Using MPS File Format .
    [Show full text]
  • Computing a Trust Region Step
    Distribution Category: Mathematics and Computers ANL--81-83 (UC-32) DE82 005656 ANIr81483 ARGONNE NATIONAL LABORATORY 9700 South Cass Avenue Argonne, Illinois 60439 COMPUTING A TRUST REGION STEP Jorge J. Mord and D. C. Sorensen Applied Mathematics Division DISCLAIMER - -. ".,.,... December 1981 Computing a Trust Region Step Jorge J. Mord and D. C. Sorensen Applied Mathematics Division Argonne National Laboratory Argonne Illinois 60439 1. Introduction. In an important class of minimization algorithms called "trust region methods" (see, for example, Sorensen [1981]), the calculation of the step between iterates requires the solution of a problem of the form (1.1) minij(w): 1|w |isAl where A is a positive parameter, I- II is the Euclidean norm in R", and (1.2) #(w) = g w + Mw7Bw, with g E R", and B E R a symmetric matrix. The quadratic function ' generally represents a local model to the objective function defined by interpolatory data at an iterate and thus it is important to be able to solve (1.1) for any symmetric matrix B; in particular, for a matrix B with negative eigenvalues. In trust region methods it is sometimes helpful to include a scaling matrix for the variables. In this case, problem (1.1) is replaced by (1.3) mint%(v):IIDu Is A where D E R""" is a nonsingular matrix. The change of variables Du = w shows that problem (1.3) is equivalent to (1.4) minif(w):1|w| Is Al where f(w) _%f(D-'w), and that the solutions of problems (1.3) and (1.4) are related by Dv = w.
    [Show full text]
  • Applying Piecewise Linear Characteristic Curves in District Energy Optimisation Bryn Pickeringa#, Ruchi Choudharya^
    PROCEEDINGS OF ECOS 2017 - THE 30TH INTERNATIONAL CONFERENCE ON EFFICIENCY, COST, OPTIMIZATION, SIMULATION AND ENVIRONMENTAL IMPACT OF ENERGY SYSTEMS JULY 2-JULY 6, 2017, SAN DIEGO, CALIFORNIA, USA Applying Piecewise Linear Characteristic Curves in District Energy Optimisation Bryn Pickeringa#, Ruchi Choudharya^ a Department of Engineering, University of Cambridge Trumpington Street, Cambridge, CB2 1PZ, United Kingdom, # [email protected], ^ [email protected] Abstract: Representing nonlinear curves as piecewise elements allows complex systems to be optimised by linear programs. Piecewise linearisation has been recently introduced in the context of distributed energy system optimisation. It is an efficient technique for representing non-linear technology behaviours in linear optimisation models, which are favourable in district energy optimisation models, owing to their speed and ability to handle large numbers of design variables. This paper describes a method of automating the creation of piecewise elements of technology performance curves for minimum fit error. The results show an objective function value improvement at a relatively large penalty in solution time: from 1.6 times to 58 times longer than describing technologies as having a single value for efficiency (SVE). We show that within the context of common technology performance curves, three breakpoints yield sufficiently accurate results and any returns are diminishing beyond that. Even at three breakpoints, it is evident that the placement of breakpoints along a curve significantly influences solution time, in a way for which it is not possible to account in automation. But, large savings can be made by automation by including a constraint to ensure piecewise curves have a strictly increasing/decreasing gradient.
    [Show full text]
  • Piecewise-Linear Approximation for Feature Subset Selection in A
    PIECEWISE-LINEAR APPROXIMATION FOR FEATURE SUBSET SELECTION IN A SEQUENTIAL LOGIT MODEL Toshiki Sato Yuichi Takano Ryuhei Miyashiro University of Tsukuba Senshu University Tokyo University of Agriculture and Technology (October 7, 2015) Abstract This paper concerns a method of selecting a subset of features for a sequential logit model. Tanaka and Nakagawa (2014) proposed a mixed integer quadratic optimization formulation for solving the problem based on a quadratic approximation of the logistic loss function. However, since there is a significant gap between the logistic loss function and its quadratic approximation, their formulation may fail to find a good subset of features. To overcome this drawback, we apply a piecewise-linear approximation to the logistic loss function. Accordingly, we frame the feature subset selection problem of minimizing an information criterion as a mixed integer linear optimization problem. The computational results demonstrate that our piecewise-linear approximation approach found a better subset of features than the quadratic approximation approach. Keywords: Optimization, statistics, feature subset selection, sequential logit model, piecewise- linear approximation, information criterion 1. Introduction The analysis of ordinal categorical data [3, 23] is required in various areas including finance, econometrics, and bioinformatics. For instance, credit ratings of financial instruments are typical ordinal categorical data, and thus, many previous studies have analyzed such data by means of ordinal classification models (see, e.g., [1, 12, 28]). A sequential logit model [5, 17, 33], also known as the continuation ratio model [2, 14], is a commonly used ordinal classification model. It predicts an ordinal class label for each sample by successively applying separate logistic regression models.
    [Show full text]
  • Welcome to the 2013 MOPTA Conference!
    Welcome to the 2013 MOPTA Conference! Mission Statement The Modeling and Optimization: Theory and Applications (MOPTA) conference is an annual event aiming to bring together a diverse group of people from both discrete and continuous optimization, working on both theoretical and applied aspects. The format consists of invited talks from distinguished speakers and selected contributed talks, spread over three days. The goal is to present a diverse set of exciting new developments from different optimization areas while at the same time providing a setting that will allow increased interaction among the participants. We aim to bring together researchers from both the theoretical and applied communities who do not usually have the chance to interact in the framework of a medium- scale event. MOPTA 2013 is hosted by the Department of Industrial and Systems Engineering at Lehigh University. Organization Committee Frank E. Curtis - Chair [email protected] Tamás Terlaky [email protected] Ted Ralphs [email protected] Katya Scheinberg [email protected] Lawrence V. Snyder [email protected] Robert H. Storer [email protected] Aurélie Thiele [email protected] Luis Zuluaga [email protected] Staff Kathy Rambo We thank our sponsors! 1 Program Wednesday, August 14 – Rauch Business Center 7:30-8:10 - Registration and continental breakfast - Perella Auditorium Lobby 8:10-8:20 - Welcome: Tamás Terlaky, Department Chair, Lehigh ISE - Perella Auditorium (RBC 184) 8:20-8:30 - Opening remarks: Patrick Farrell, Provost, Lehigh University
    [Show full text]
  • Integer Optimization Methods for Machine Learning by Allison an Chang Sc.B
    Integer Optimization Methods for Machine Learning by Allison An Chang Sc.B. Applied Mathematics, Brown University (2007) Submitted to the Sloan School of Management in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Operations Research at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY June 2012 c Massachusetts Institute of Technology 2012. All rights reserved. Author............................................. ................. Sloan School of Management May 18, 2012 Certified by......................................... ................. Dimitris Bertsimas Boeing Leaders for Global Operations Professor Co-Director, Operations Research Center Thesis Supervisor Certified by......................................... ................. Cynthia Rudin Assistant Professor of Statistics Thesis Supervisor Accepted by......................................... ................ Patrick Jaillet Dugald C. Jackson Professor Department of Electrical Engineering and Computer Science Co-Director, Operations Research Center 2 Integer Optimization Methods for Machine Learning by Allison An Chang Submitted to the Sloan School of Management on May 18, 2012, in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Operations Research Abstract In this thesis, we propose new mixed integer optimization (MIO) methods to ad- dress problems in machine learning. The first part develops methods for supervised bipartite ranking, which arises in prioritization tasks in diverse domains such as infor- mation retrieval, recommender systems, natural language processing, bioinformatics, and preventative maintenance. The primary advantage of using MIO for ranking is that it allows for direct optimization of ranking quality measures, as opposed to current state-of-the-art algorithms that use heuristic loss functions. We demonstrate using a number of datasets that our approach can outperform other ranking methods. The second part of the thesis focuses on reverse-engineering ranking models.
    [Show full text]
  • Polylithic Modeling and Solution Approaches Using Algebraic Modeling Systems
    Polylithic Modeling and Solution Approaches Using Algebraic Modeling Systems Josef Kallrath Department of Astronomy University of Florida Gainesville, FL 32611, USA [email protected] Abstract. Based on the Greek term monolithos (stone consisting of one single block) Kallrath (2009) introduced the term polylithic for modeling and solution approaches in which mixed integer or non-convex nonlinear optimization problems are solved by tailor-made methods involving sev- eral models and/or algorithmic components, in which the solution of one model is input to another one. This can be exploited to initialize certain variables, or to provide bounds on them (problem-specific preprocess- ing). Mathematical examples of polylithic approaches are decomposition techniques, or hybrid methods in which constructive heuristics and local search improvement methods are coupled with exact MIP algorithms Tailor-made polylithic solution approaches with thousands or mil- lions of solve statements are challenges on algebraic modeling languages. Local objects and procedural structures are almost necessary. Warm- start and hot-start techniques can be essential. The effort of developing complex tailor-made polylithic solutions is awarded by enabling us to solve real-world problems far beyond the limits of monolithic approaches and general purpose solvers. Keywords: algebraic modeling languages, branch&price, column gen- eration, decomposition, hybrid methods, monolithic, primal methods, polylithic, problem-specific preprocessing 1 Introduction Instead of solving
    [Show full text]