DOCSLIB.ORG

Convex Relaxation for Low-Dimensional Representation: Phase Transitions and Limitations

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Convex Relaxation for Low-Dimensional Representation: Phase Transitions and Limitations Convex Relaxation for Low-Dimensional Representation: Phase Transitions and Limitations Thesis by Samet Oymak In Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy California Institute of Technology Pasadena, California 2015 (Defended June 12, 2014) c 2015 Samet Oymak All Rights Reserved ii Dedicated to my family iii Acknowledgments To begin with, it was a great pleasure to work with my advisor Babak Hassibi. Babak has always been a fatherly figure to me and my labmates. Most of our group are thousands of miles away from their home, and we are fortunate to have Babak as advisor, who always helps us with our troubles, whether they are personal or professional. As a graduate student, Babak’s intuition on identifying new problems and motivation for math was a great inspiration for me. He always encouraged me to do independent research and to be persistent with the toughest (mostly mathematical) challenges. I thank my thesis committee, Professor Maryam Fazel, Professor Joel Tropp, Professor Venkat Chan- drasekaran, and Professor PP Vaidyanathan. Maryam has literally been a second advisor to me, and I am very thankful for her professional guidance. I would like to thank Joel for the material he taught me in class as well as for answering my research questions with great patience and for his helpful feedback for this thesis. It was always great to interact with Venkat and PP. Thanks to them, Caltech has been a better and more stimulating place for me. I would like to thank fellow graduate students and lab-mates for their help and support. My labmates Amin Khajehnejad, Ramya Korlakai Vinayak, Kishore Jaganathan, and Chris Thrampoulidis were also my research collaborators with whom I spent countless hours discussing research and meeting deadlines. Kishore, Hyoung Jun Ahn, and Eyal En Gad were my partners in the fiercest racquetball games. Matthew Thill, Wei Mao, and Ravi Teja Sukhavasi were a crucial part of all the fun in the lab. During my time at Caltech, Shirley Slattery and Tanya Owen were amazingly helpful with providing information and daily issues. I cannot possibly count how many times I asked Shirley “Is Babak around?”. I can’t imagine how I could enjoy my time here without my dear friends Necmiye Ozay, Murat Acar, Aycan Yurtsever, Selim Hanay, and Jiasi Chen. I guess I could meet these great people only at Caltech. They are now literally all over the world; but I do hope our paths will cross again. Finally, I am most grateful to my family for always being there for me. Their continuous support has been a source of my ambition and motivation since primary school. It has been a challenging five years away from them, and I am very happy to know that, we are still as connected as ever. iv Abstract There is a growing interest in taking advantage of possible patterns and structures in data so as to extract the desired information and overcome the curse of dimensionality. In a wide range of applications, including computer vision, machine learning, medical imaging, and social networks, the signal that gives rise to the observations can be modeled to be approximately sparse and exploiting this fact can be very beneficial. This has led to an immense interest in the problem of efficiently reconstructing a sparse signal from limited linear observations. More recently, low-rank approximation techniques have become prominent tools to approach problems arising in machine learning, system identification and quantum tomography. In sparse and low-rank estimation problems, the challenge is the inherent intractability of the objective function, and one needs efficient methods to capture the low-dimensionality of these models. Convex op- timization is often a promising tool to attack such problems. An intractable problem with a combinatorial objective can often be “relaxed” to obtain a tractable but almost as powerful convex optimization problem. This dissertation studies convex optimization techniques that can take advantage of low-dimensional rep- resentations of the underlying high-dimensional data. We provide provable guarantees that ensure that the proposed algorithms will succeed under reasonable conditions, and answer questions of the following flavor: For a given number of measurements, can we reliably estimate the true signal? • If so, how good is the reconstruction as a function of the model parameters? • More specifically, i) Focusing on linear inverse problems, we generalize the classical error bounds known for the least-squares technique to the lasso formulation, which incorporates the signal model. ii) We show that intuitive convex approaches do not perform as well as expected when it comes to signals that have multiple low-dimensional structures simultaneously. iii) Finally, we propose convex relaxations for the graph clustering problem and give sharp performance guarantees for a family of graphs arising from the so-called stochastic block model. We pay particular attention to the following aspects. For i) and ii), we aim to provide a general geometric framework, in which the results on sparse and low-rank estimation can be obtained as special cases. For i) and iii), we investigate the precise performance characterization, which yields the right constants in our bounds and the true dependence between the problem parameters. v Contents Acknowledgments iv Abstract v 1 Introduction 1 1.1 Sparse signal estimation....................................2 1.2 Low-dimensional representation via convex optimization...................5 1.3 Phase Transitions........................................ 10 1.4 Literature Survey........................................ 11 1.5 Contributions.......................................... 18 2 Preliminaries 29 2.1 Notation............................................. 29 2.2 Projection and Distance..................................... 30 2.3 Gaussian width, Statistical dimension and Gaussian distance................. 32 2.4 Denoising via proximal operator................................ 34 2.5 Inequalities for Gaussian Processes............................... 35 3 A General Theory of Noisy Linear Inverse Problems 38 3.1 Our Approach.......................................... 43 3.2 Main Results.......................................... 50 3.3 Discussion of the Results.................................... 55 3.4 Applying Gaussian Min-Max Theorem............................. 63 3.5 After Gordon’s Theorem: Analyzing the Key Optimizations................. 68 3.6 The NSE of the C-LASSO................................... 76 vi 3.7 Constrained-LASSO Analysis for Arbitrary s ......................... 81 3.8 `2-LASSO: Regions of Operation............................... 92 3.9 The NSE of the `2-LASSO................................... 97 3.10 Nonasymptotic results on `2-LASSO.............................. 102 3.11 Proof of Theorem 3.7...................................... 104 2 3.12 `2-LASSO............................................ 108 3.13 Converse Results........................................ 115 3.14 Numerical Results........................................ 119 3.15 Future Directions........................................ 122 4 Elementary equivalences in compressed sensing 125 4.1 A comparison between the Bernoulli and Gaussian ensembles................ 125 4.2 An equivalence between the recovery conditions for sparse signals and low-rank matrices.. 133 5 Simultaneously Structured Models 147 5.1 Problem Setup.......................................... 153 5.2 Main Results: Theorem Statements............................... 156 5.3 Measurement ensembles.................................... 163 5.4 Upper bounds.......................................... 168 5.5 General Simultaneously Structured Model Recovery..................... 171 5.6 Proofs for Section 5.2.2..................................... 178 5.7 Numerical Experiments..................................... 182 5.8 Discussion............................................ 185 6 Graph Clustering via Low-Rank and Sparse Decomposition 188 6.1 Model.............................................. 191 6.2 Main Results.......................................... 192 6.3 Simulations........................................... 196 6.4 Discussion and Conclusion................................... 197 7 Conclusions 198 7.1 Generalized Lasso........................................ 198 vii 7.2 Universality of the Phase Transitions.............................. 199 7.3 Simultaneously structured signals............................... 200 7.4 Structured signal recovery beyond convexity.......................... 201 Bibliography 202 A Further Proofs for Chapter3 227 A.1 Auxiliary Results........................................ 227 A.2 Proof of Proposition 2.7..................................... 231 A.3 The Dual of the LASSO.................................... 235 A.4 Proofs for Section 3.5...................................... 236 A.5 Deviation Analysis: Key Lemma................................ 247 A.6 Proof of Lemma 3.20...................................... 251 A.7 Explicit formulas for well-known functions.......................... 256 A.8 Gaussian Width of the Widened Tangent Cone......................... 261 B Further Proofs for Chapter5 264 B.1 Properties of Cones....................................... 264 B.2 Norms in Sparse and Low-rank Model............................. 266 B.3 Results on non-convex recovery................................ 268 C Further Proofs for Chapter6 270 C.1 On the success of the simple program............................. 270
Load more

Recommended publications
  • Stochastic Combinatorial Optimization with Controllable Risk Aversion Level
    Stochastic Combinatorial Optimization with Controllable Risk Aversion Level Anthony Man–Cho So Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong, Shatin, N. T., Hong Kong email: [email protected] Jiawei Zhang Department of Information, Operations, and Management Sciences, Stern School of Business, New York University, New York, NY 10012, USA email: [email protected] Yinyu Ye Department of Management Science and Engineering and, by courtesy, Electrical Engineering, Stanford University, Stanford, CA 94305, USA email: [email protected] Most of the recent work on 2–stage stochastic combinatorial optimization problems have focused on the min- imization of the expected cost or the worst–case cost of the solution. Those two objectives can be viewed as two extreme ways of handling risk. In this paper we propose to use an one–parameter family of functionals to interpolate between them. Although such a family has been used in the mathematical finance and stochastic programming literature before, its use in the context of approximation algorithms seems new. We show that under standard assumptions, a broad class of risk–adjusted 2–stage stochastic programs can be efficiently treated by the Sample Average Approximation (SAA) method. In particular, our result shows that it is computationally feasible to incorporate some degree of robustness even when the underlying distribution can only be accessed in a black–box fashion. We also show that when combined with suitable rounding procedures, our result yields new approximation algorithms for many risk–adjusted 2–stage stochastic combinatorial optimization problems under the black–box setting.
    [Show full text]
  • Opf)P in OPERATIONS RESEARCH
    APPLICATION OF MATHEMATICAL PROGRAMMING DISSERTATION SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE AWARD OF THE DEGREE OF iWasiter of ^i)Uo«opf)p IN OPERATIONS RESEARCH By TEG ALAM Under the Supervision of DR. A. BARI DEPARTMENT OF STATISTICS & OPERATIONS-RESEARCH ALIGARH MUSLIM UNIVERSITY ALIGARH (INDIA) 2001 '^•^ P5.-32^^ ^, DS3200 DEPARTMENT OF STATISTICS & OPERATION RESEARCI ALIGARH MUSLIM UNIVERSITY (Dr. X ^ari ALIGARH - 202002 -INDIA Ph.D (AUg) ^hone: 0571-701251 (O) "^ 0571-700112 (R) Dated: OE(RTlTICA'VE Certified that the dissertation entitled "Application of Mathematical Programming" is carried out by Tcg Alaitl under my supervision. The work is sufficient for the requirement of degree of Master of Philosophy in Operations Research. ((Dr. A. (Bari) Supervisor Univ. Exchange: (0571) 700920 -23 Extns:419, 421,422, 441 Fax:+91-571-400528 This Dissertation entitled ''^Application of Mathematical programming''' is submitted to the Aligarh Muslim University, Aligarh, for the partial fiilfillment of the degree of M.Phil in OperationsResearch. Mathematical programming is concerned with the determination of a minimum or a maximum of a function of several variables, which are required to satisfy a number of constraints. Such solutions are sought in diverse fields; including engineering, Operations Research, Management Science, numerical analysis and economics etc. This mamuscript consists of five chapters. Chapter-1 deals with the brief history of Mathematical Programming. It also contains numerous applications of Mathematical Programming. Chapter-2 gives an introduction of Transportation problem and devoted to extensions and methods of solutions. A numerical example and Relation of Transportation problems with Network problems are also discussed. Chapter - 3 gives an introduction of Game theory.
    [Show full text]
  • Optima 82 Publishes the Obituary of Paul Tseng by His Friends Affected by the Proposed Change; Only Their Subtitles Would Change
    OPTIMA Mathematical Programming Society Newsletter 82 Steve Wright MPS-sponsored meetings: the International Conference on Stochas- MPS Chair’s Column tic Programming (ICSP) XII (Halifax, August 14–20), the Interna- tional Conference on Engineering Optimization (Lisbon, September 6–9), and the IMA Conference on Numerical Linear Algebra and March 16, 2010. You should recently have received a letter con- Optimization (Birmingham, September 13–15). cerning a possible change of name for MPS, to “Mathematical Op- timization Society”. This issue has been discussed in earnest since ISMP 2009, where it was raised at the Council and Business meet- ings. Some of you were kind enough to send me your views follow- Note from the Editors ing the mention in my column in Optima 80. Many (including me) believe that the term “optimization” is more widely recognized and The topic of this issue of Optima is Mechanism Design –aNobel better understood as an appellation for our field than the current prize winning theoretical field of economics. name, both among those working in the area and our colleagues We present the main article by Jay Sethuraman, which introduces in other disciplines. Others believe that the current name should the main concepts and existence results for some of the models be retained, as it has the important benefits of tradition, branding, arising in mechanism design theory. The discussion column by Garud and name recognition. To ensure archival continuity in the literature, Iyengar and Anuj Kumar address a specific example of such a model the main titles of our journals Mathematical Programming, Series A which can be solved by the means of optimization.
    [Show full text]
  • Arxiv:1411.2129V1 [Math.OC]
    INTERIOR-POINT ALGORITHMS FOR CONVEX OPTIMIZATION BASED ON PRIMAL-DUAL METRICS TOR MYKLEBUST, LEVENT TUNC¸EL Abstract. We propose and analyse primal-dual interior-point algorithms for convex opti- mization problems in conic form. The families of algorithms we analyse are so-called short- step algorithms and they match the current best iteration complexity bounds for primal-dual symmetric interior-point algorithm of Nesterov and Todd, for symmetric cone programming problems with given self-scaled barriers. Our results apply to any self-concordant barrier for any convex cone. We also prove that certain specializations of our algorithms to hyperbolic cone programming problems (which lie strictly between symmetric cone programming and gen- eral convex optimization problems in terms of generality) can take advantage of the favourable special structure of hyperbolic barriers. We make new connections to Riemannian geometry, integrals over operator spaces, Gaussian quadrature, and strengthen the connection of our algorithms to quasi-Newton updates and hence first-order methods in general. arXiv:1411.2129v1 [math.OC] 8 Nov 2014 Date: November 7, 2014. Key words and phrases. primal-dual interior-point methods, convex optimization, variable metric methods, local metric, self-concordant barriers, Hessian metric, polynomial-time complexity; AMS subject classification (MSC): 90C51, 90C22, 90C25, 90C60, 90C05, 65Y20, 52A41, 49M37, 90C30. Tor Myklebust: Department of Combinatorics and Optimization, Faculty of Mathematics, University of Wa- terloo, Waterloo, Ontario N2L 3G1, Canada (e-mail: [email protected]). Research of this author was supported in part by an NSERC Doctoral Scholarship, ONR Research Grant N00014-12-10049, and a Dis- covery Grant from NSERC.
    [Show full text]
  • A Primal-Dual Interior-Point Algorithm for Nonsymmetric Exponential-Cone Optimization
    A primal-dual interior-point algorithm for nonsymmetric exponential-cone optimization Joachim Dahl Erling D. Andersen May 27, 2019 Abstract A new primal-dual interior-point algorithm applicable to nonsymmetric conic optimization is proposed. It is a generalization of the famous algorithm suggested by Nesterov and Todd for the symmetric conic case, and uses primal-dual scalings for nonsymmetric cones proposed by Tun¸cel.We specialize Tun¸cel'sprimal-dual scalings for the important case of 3 dimensional exponential-cones, resulting in a practical algorithm with good numerical performance, on level with standard symmetric cone (e.g., quadratic cone) algorithms. A significant contribution of the paper is a novel higher-order search direction, similar in spirit to a Mehrotra corrector for symmetric cone algorithms. To a large extent, the efficiency of our proposed algorithm can be attributed to this new corrector. 1 Introduction In 1984 Karmarkar [11] presented an interior-point algorithm for linear optimization with polynomial complexity. This triggered the interior-point revolution which gave rise to a vast amount research on interior-point methods. A particularly important result was the analysis of so-called self-concordant barrier functions, which led to polynomial-time algorithms for linear optimization over a convex domain with self-concordant barrier, provided that the barrier function can be evaluated in polynomial time. This was proved by Nesterov and Nemirovski [18], and as a consequence convex optimization problems with such barriers can be solved efficently by interior-point methods, at least in theory. However, numerical studies for linear optimization quickly demonstrated that primal-dual interior- point methods were superior in practice, which led researchers to generalize the primal-dual algorithm to general smooth convex problems.
    [Show full text]
  • George B. Dantzig Papers SC0826
    http://oac.cdlib.org/findaid/ark:/13030/c8s75gwd No online items Guide to the George B. Dantzig Papers SC0826 Daniel Hartwig & Jenny Johnson Department of Special Collections and University Archives March 2012 Green Library 557 Escondido Mall Stanford 94305-6064 [email protected] URL: http://library.stanford.edu/spc Guide to the George B. Dantzig SC0826 1 Papers SC0826 Language of Material: English Contributing Institution: Department of Special Collections and University Archives Title: George B. Dantzig papers creator: Dantzig, George Bernard, 1914-2005 Identifier/Call Number: SC0826 Physical Description: 91 Linear Feet Date (inclusive): 1937-1999 Special Collections and University Archives materials are stored offsite and must be paged 36-48 hours in advance. For more information on paging collections, see the department's website: http://library.stanford.edu/depts/spc/spc.html. Information about Access The materials are open for research use. Audio-visual materials are not available in original format, and must be reformatted to a digital use copy. Ownership & Copyright All requests to reproduce, publish, quote from, or otherwise use collection materials must be submitted in writing to the Head of Special Collections and University Archives, Stanford University Libraries, Stanford, California 94305-6064. Consent is given on behalf of Special Collections as the owner of the physical items and is not intended to include or imply permission from the copyright owner. Such permission must be obtained from the copyright owner, heir(s) or assigns. See: http://library.stanford.edu/depts/spc/pubserv/permissions.html. Restrictions also apply to digital representations of the original materials. Use of digital files is restricted to research and educational purposes.
    [Show full text]
  • Semidefinite Relaxation of Quadratic Optimization Problems
    APPEARED IN IEEE SP MAGAZINE, SPECIAL ISSUE ON “CONVEX OPT. FOR SP”, MAY 2010 1 Semidefinite Relaxation of Quadratic Optimization Problems Zhi-Quan Luo, Wing-Kin Ma, Anthony Man-Cho So, Yinyu Ye, and Shuzhong Zhang 2 I. INTRODUCTION In recent years, the semidefinite relaxation (SDR) technique 1.5 has been at the center of some of the very exciting develop- 1 ments in the area of signal processing and communications, and it has shown great significance and relevance on a va- 0.5 riety of applications. Roughly speaking, SDR is a powerful, 2 0 x computationally efficient approximation technique for a host of very difficult optimization problems. In particular, it can be −0.5 applied to many nonconvex quadratically constrained quadratic −1 programs (QCQPs) in an almost mechanical fashion. These include the following problems: −1.5 T min x Cx −2 Rn −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 x∈ x1 T s.t. x Fix gi, i =1, . , p, (1) T ≥ R2 x Hix = li, i =1, . , q, Fig. 1. A nonconvex QCQP in . Colored lines: contour of the objective function; gray area: the feasible set; black lines: boundary of each constraint. where the given matrices C, F1,..., Fp, H1,..., Hq are as- sumed to be general real symmetric matrices, possibly in- definite. The class of nonconvex QCQPs (1) captures many accuracy of no worse than 0.8756 for the Maximum Cut problems that are of interest to the signal processing and com- problem (the BQP with some conditions on C). In other munications community.
    [Show full text]
  • Low-Rank Semidefinite Programming: Theory And
    Full text available at: http://dx.doi.org/10.1561/2400000009 Low-Rank Semidefinite Programming: Theory and Applications Alex Lemon Stanford University [email protected] Anthony Man-Cho The Chinese University of Hong Kong [email protected] Yinyu Ye Stanford University [email protected] Boston — Delft Full text available at: http://dx.doi.org/10.1561/2400000009 R Foundations and Trends in Optimization Published, sold and distributed by: now Publishers Inc. PO Box 1024 Hanover, MA 02339 United States Tel. +1-781-985-4510 www.nowpublishers.com [email protected] Outside North America: now Publishers Inc. PO Box 179 2600 AD Delft The Netherlands Tel. +31-6-51115274 The preferred citation for this publication is A. Lemon, A. M.-C. So, Y. Ye. Low-Rank Semidefinite Programming: Theory and Applications. Foundations and Trends R in Optimization, vol. 2, no. 1-2, pp. 1–156, 2015. R This Foundations and Trends issue was typeset in LATEX using a class file designed by Neal Parikh. Printed on acid-free paper. ISBN: 978-1-68083-137-5 c 2016 A. Lemon, A. M.-C. So, Y. Ye All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, mechanical, photocopying, recording or otherwise, without prior written permission of the publishers. Photocopying. In the USA: This journal is registered at the Copyright Clearance Cen- ter, Inc., 222 Rosewood Drive, Danvers, MA 01923. Authorization to photocopy items for internal or personal use, or the internal or personal use of specific clients, is granted by now Publishers Inc for users registered with the Copyright Clearance Center (CCC).
    [Show full text]
  • The 2010 Noticesindex
    The 2010 Notices Index January issue: 1–200 May issue: 593–696 October issue: 1073–1240 February issue: 201–328 June/July issue: 697–816 November issue: 1241–1384 March issue: 329–456 August issue: 817–936 December issue: 1385–1552 April issue: 457–592 September issue: 937–1072 2010 Award for an Exemplary Program or Achievement in Index Section Page Number a Mathematics Department, 650 2010 Conant Prize, 515 The AMS 1532 2010 E. H. Moore Prize, 524 Announcements 1533 2010 Mathematics Programs that Make a Difference, 650 Articles 1533 2010 Morgan Prize, 517 Authors 1534 2010 Robbins Prize, 526 Deaths of Members of the Society 1536 2010 Steele Prizes, 510 Grants, Fellowships, Opportunities 1538 2010 Veblen Prize, 521 Letters to the Editor 1540 2010 Wiener Prize, 519 Meetings Information 1540 2010–2011 AMS Centennial Fellowship Awarded, 758 New Publications Offered by the AMS 1540 2011 AMS Election—Nominations by Petition, 1027 Obituaries 1540 AMS Announces Congressional Fellow, 765 Opinion 1540 AMS Announces Mass Media Fellowship Award, 766 Prizes and Awards 1541 AMS-AAAS Mass Media Summer Fellowships, 1320 Prizewinners 1543 AMS Centennial Fellowships, Invitation for Applications Reference and Book List 1547 for Awards, 1320 Reviews 1547 AMS Congressional Fellowship, 1323 Surveys 1548 AMS Department Chairs Workshop, 1323 Tables of Contents 1548 AMS Email Support for Frequently Asked Questions, 268 AMS Endorses Postdoc Date Agreement, 1486 AMS Homework Software Survey, 753 About the Cover AMS Holds Workshop for Department Chairs, 765 57,
    [Show full text]
  • Prof. Yinyu Ye
    静园 号院 科研讲座 2 0 1 9 年第03号 Distributionally Robust Optimization Driven by Stochastic and Online Data Prof. Yinyu Ye 邓小铁 教授 2019年4月10日 星期三 15:10-16:10 静园五院107教室 Abstract We present decision/optimization models/problems driven by uncertain and online data, and show how analytical models and computational algorithms can be used to achieve solution efficiency and near optimality. First, we describe the so-called Distributionally or Likelihood Robust optimization (DRO) models and their algorithms in dealing stochastic decision problems when the exact uncertainty distribution is unknown but certain statistical moments and/or sample distributions can be estimated. Secondly, when decisions are made in presence of high dimensional stochastic data, handling joint distribution of correlated random variables can present a formidable task, both in terms of sampling and estimation as well as algorithmic complexity. A common heuristic is to estimate only marginal distributions and substitute joint distribution by independent (product) distribution. Here, we study possible loss incurred on ignoring correlations through the DRO approach, and quantify that loss as Price of Correlations (POC). Thirdly, we describe an online combinatorial auction problem using online linear programming technologies. We discuss near-optimal algorithms for solving this surprisingly general class of online problems under the assumption of random order of arrivals and some conditions on the data and size of the problem. Biography Yinyu Ye is currently the K.T. Li Chair Professor of Engineering at Department of Management Science and Engineering and Institute of Computational and Mathematical Engineering, Stanford University. He is also the Director of the MS&E Industrial Affiliates Program.
    [Show full text]
  • 1 Plenary Talks of Mostly OM 2019 Workshop Dimitris Bertsimas
    Plenary Talks of Mostly OM 2019 Workshop Dimitris Bertsimas Massachusetts Institute of Technology Title: The Voice of Optimization Abstract: We present a new way to see optimization problems. Using machine learning techniques we are able to predict the strategy behind the optimal solution in any continuous and mixed-integer convex optimization problem as a function of its key parameters. The benefits of our approach are interpretability and speed. We use interpretable machine learning algorithms such as optimal classification trees (OCTs) to gain insights on the relationship between the problem parameters and the optimal solution. In this way, optimization is no longer a black-box and we can understand it. In addition, once we train the predictor, we can solve optimization problems at very high speed. This aspect is also relevant for non interpretable machine learning methods such as neural networks (NNs) since they can be evaluated very efficiently after the training phase. We show on several realistic examples that the accuracy behind our approach is in the 90%-100% range, while even when the predictions are not correct, the degree of suboptimality or infeasibility is very low. We also benchmark the computation time beating state of the art solvers by multiple orders of magnitude. Therefore, our method provides on the one hand a novel insightful understanding of the optimal strategies to solve a broad class of continuous and mixed-integer optimization problems and on the other hand a powerful computational tool to solve online optimization at very high speed. (joint work with Bartolomeo Stellato, MIT). About the speaker: Dimitris Bertsimas is currently the Boeing Professor of Operations Research and the co- director of the Operations Research Center and faculty director of the Master of Business analytics at MIT.
    [Show full text]
  • Yinyu Ye Received the B.S. Degree in System Engineering from the Huazhong University of Science and Technology, Wuhan, China, and the M.S
    Yinyu Ye received the B.S. degree in System Engineering from the Huazhong University of Science and Technology, Wuhan, China, and the M.S. and Ph.D. degrees in Management Science & Engineering from Stanford University, Stanford. Currently, he is a full Professor of Management Science and Engineering and Institute of Computational and Mathematical Engineering and the Director of the MS&E Industrial Affiliates Program, Stanford University. His current research interests include Continuous and Discrete Optimization, Mathematical Programming, Algorithm Design and Analysis, Computational Game/Market Equilibrium, Metric Distance Geometry, Graph Realization, Dynamic Resource Allocation, and Stochastic and Robust Decision Making, etc. Highlights of his academic research include: J. Carlsson (supervised by Ye), the 2013 INFORMS Computing Society Prize, for the best English language papers dealing with the Operations Research/Computer Science interface I. Post, the 2013 Second Prize of INFORMS Nicholson Student Paper Competition for the joint paper with Ye “The simplex method is strongly polynomial for Markov decision processes.” 2012 ISMP Tseng Lectureship Prize (Inaugural Recipient) for outstanding contributions in the area of continuous optimization Recipient of the 2009 John von Neumann Theory Prize for fundamental sustained contributions to theory in Operations Research and the Management Sciences Recipient of the 2009 IBM Faculty Award. Elected Vice Chair of the SIAM Activity Group on Optimization (SIAG/OPT), 2008. E. Delage, First
    [Show full text]