DOCSLIB.ORG

The Ellipsoid Method

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
The Ellipsoid Method The Ellipsoid method R.M. Freund/ C. Roos (MIT/TUD) e-mail: [email protected] URL: http://www.isa.ewi.tudelft.nl/ roos ∼ WI 4218 March 21, A.D. 2007 Optimization Group 1/26 Outline ⊲ Sherman-Morrison formula ⊲ Ellipsoids ⊲ Ellipsoid method ⊲ Basic construction ⊲ Prototypical iteration ⊲ Iteration bound ⊲ Two theorems ⊲ Two more theorems ⊲ Optimization with the ellipsoid method ⊲ Convexity of the homogeneous problem ⊲ The Key Proposition Optimization Group 2/26 In memory of George Danzig and Leonid Khachyan (1) Leonid Khachiyan (1952-2005) passed away Friday April 29 at the age of 52. He died in his sleep, apparently of a heart attack, colleagues said. Khachiyan was best known for his 1979 use of the ellipsoid algorithm, originally developed for convex programming, to give the first polynomial-time algorithm to solve linear programming problems. While the simplex algorithm solved linear programs well in practice, Khachiyan gave the first formal proof of an efficient algorithm in the worst case. ”He was among the world’s most famous computer scientists,” said Haym Hirsh, chairman of the computer science department at Rutgers. George B. Dantzig, the father of Linear Programming and the creator of the Simplex Method, died at the age of 90 on May 13. In a statement INFORMS President Richard C. Larson mourned his death (see http://www.informs.org/Press/dantzigobit.htm). The tributes in- cluded obituaries in the Washington Post, the Francisco Chronicle, Mercury News and New York Times. National Public Radio commentator Keith Devlin remembered George Dantzig in a broadcast on Saturday, May 21. Optimization Group 3/26 In memory of George Danzig and Leonid Khachyan (2) Optimization Group 4/26 Leonid Khachyan in Shanghai (with Bai, Peng and Terlaky, 2002) Optimization Group 5/26 Sherman-Morrison formula Let Q, R, S, T be matrices such that Q and Q + RST are nonsingular. • R and S are n k matrices of rank k n. • × ≤ T 1 1 1 T 1 1 T 1 Then (Q + RS )− = Q− Q− R(I + S Q− R)− S Q− − Lemma 1 Let U be such that QU = R. Then I + ST U is invertible. Proof: Suppose w Rk satisfies (I + ST U)w = 0. Then, ∈ (Q + RST )Uw =(QU)w + R(ST Uw)= Rw Rw = 0. − Q + RST being nonsingular, this gives Uw = 0. Since rank (U) = k this implies w = 0. • T T T Theorem 1 If Qx0 = q and (I + S U)y = S x0 then x = x0 Uy satisfies (Q + RS )x = q. − T T T Proof: (Q + RS )(x0 Uy)= Qx0 + RS x0 QUy RS Uy − − − = q + R(I + ST U)y Ry RST Uy = q. 2 − − The solution x of (Q + RST )x = q is given by 1 1 T 1 T x = x0 Uy = Q q Q R(I + S U) S x0 − − − − − = (Q 1 Q 1R(I + ST Q 1R) 1ST Q 1)q. − − − − − − Since Theorem 1 holds for all q Rn the Sherman-Morrison formula follows. ∈ Reference: W. W. Hager. Updating the inverse of a matrix. SIAM Rev., 31(2):221–239, June 1989. Optimization Group 6/26 Ellipsoids Let M be a (symmetric) positive definite n n matrix and z Rn. Then × ∈ E := x : (x z)T M (x z) 1 M,z − − ≤ denotes an ellipsoid centered at z. Note thatn o 1 x E M 2 (x z) 1. ∈ M,z ⇔ − ≤ 1 1 Hence, putting u = M 2 (x z), we have x = z + M − 2 u and u 1. Therefore, − k k≤ 1 T E = x = z + M − 2 u : u u 1 . M,z ≤ In other words, n o 1 EM,z = z + M − 2 B(0, 1), where B(0, 1) denotes the unit sphere, centered at the origin. The volume of B(0, 1) is given by n π 2 ν(n)= n , Γ 2 + 1 and the volume of EM,z by ν(n) vol EM,z = . det(M) Hence p 1 ln vol E = ln ν(n) ln det(M). M,z − 2 Optimization Group 7/26 Ellipsoid method Given is a set convex S. We want to find s S. We assume ∈ that an ellipsoid E is given such that S E . M,z ⊆ M,z Step 1 : k = 0; M k = M; zk = z; Step 2 : if zk S: STOP; ∈ Step 3 : Find nonzero vector a such that b aT x aT zk, x S; z ≤ ∀ ∈ b Step 4 : Construct smallest volume ellipsoid that contains Rn T k EM,z x : a (x z ) 0 ; a ∩ ∈ − ≤ n o Let this ellipsoid have matrix M k+1 and center zk+1. Step 5 : k = k + 1; Step 6 : Goto Step 2. Optimization Group 8/26 Basic construction (1) Find m =(z, 0,..., 0), M = diag (a1,a2,...,an), such that the ellipsoid E = x Rn : (x m)T M(x m) 1 M,m ∈ − − ≤ contains B(0, 1) x : x1 0 and has minimal volume. Note that ∩ { ≥ } n T 2 2 (x m) M(x m)= a1(x1 z) + aix . − − − i i=2 X The unit vectors e1, e2,..., en are on the boundary of B(0, 1) x : x1 0 . We require them to lie on the boundary of the± ellipsoid.± This gives: ∩ { ≥ } 2 a1(1 z) = 1 − 2 a1z + ai = 1, 2 i n. ≤ ≤ From this we obtain 2 1 2 z 1 2z a1 = , ai = 1 a1z = 1 = − , i 2. (1 z)2 − − (1 z)2 (1 z)2 ≥ − − − Recall that vol EM,m is minimal if det M is maximal. Since n 1 (1 2z) − det M = − , (1 z)2n − = 1 one may easily verify that this occurs if z n+1. Substituting this value we obtain 1 2 1 a1 = 1+ , ai = 1 , i 2. n − n2 ≥ Optimization Group 9/26 Illustration for n = 2 1 x 0 b -1 -1 0 1 Optimization Group 10/26 Basic construction (2) (n + 1)2 n2 1 a1 = , ai = − , i 2. n2 n2 ≥ T B = y : y y 1 , H = y : y1 0 ≤ { ≥ } ¯ = n2 1 + 2 T ¯ = 1 M n−2 I n 1e 1e1 , z n+1e1 − E = EM,¯ z¯ Theorem 2 (B H) E. ∩ ⊆ Proof: One has y E if and only if ∈ T 2 T e1 n 1 2e1e e1 y − I + 1 y 1. − n + 1 n2 n 1 − n + 1 ≤ − This is equivalent to 2 n n 1 2 1 2n + 2 − y + + y1 (y1 1) 1. n2 i n2 − n2 ≤ i=1 X n 2 Now let y B H. Then 0 y1 1 implies y1 (y1 1) 0. Also y 1. Since ∈ ∩ ≤ ≤ − ≤ i=1 i ≤ n2 1 1 P − + = 1, n2 n2 we obtain y E. ∈ • Optimization Group 11/26 Basic construction (3) B = y : yT y 1 2 ≤ M¯ = nn 1 I + 2 oe eT n−2 n 1 1 1 − E = EM,¯ z¯ 1 Theorem 3 vol (E) < vol (B) e2(n−+1). Proof: One has vol(E) = √det I = 1 . Moreover, vol(B) √det M¯ √det M¯ 2 n 2 n 1 2 det M¯ = n 1 1+ 2 = n 1 − n+1 . n−2 n 1 n−2 n − Hence, using 1+ x ex we get ≤ 2 n 1 2 n 1 2 1 = n − n = 1+ 1 − 1 1 det M¯ n2 1 n+1 n2 1 − n+1 n −1 2 1 − − en2 1en−+1 = en−+1. ≤ − This implies the theorem. • Optimization Group 12/26 Prototypical iteration T 1 E := x : (x z) M (x z) 1 = z + M −2B(0, 1) M,z − − ≤ We know M, z, and an nonzero vector a. Define o 2 T 1 M¯ = n 1 M + 2 aa , z¯ = z + 1 M − a n−2 n 1 aT M 1a n+1 √ T 1 − − a M − a By the Sherman-Morrison formula, one has 2 1 T 1 1 n 1 2 M − aa M − M¯ − = M − n2 1 − n + 1 aT M 1a ! − − Theorem 4 E x : aT x aT z E . M,z ∩ ≤ ⊆ M,¯ z¯ n o Theorem 5 1 2(n−+1) vol EM,¯ z¯ < vol EM,z e . Optimization Group 13/26 Proof of Theorem 5 T 1 2(u + b)(u + b) u := M − 2 a, b := u e1, R := I k k u + b 2 − k k One has RT = R, Ru = b, R2 = I. Hence ¯ n2 1 2 aaT M = −2 M + T 1 n n 1 a M − a − 1 1 2 1 T 1 n 1 2 M − 2 aa M − 2 2 2 = −2 M I + T 1 M n n 1 a M − a − 1 1 2 1 T 1 n 1 2 RM − 2 aa M − 2 R 2 2 = −2 M R I + T 1 RM n n 1 a M − a − 1 1 2 1 T 1 n 1 2 RM − 2 aa M − 2 R 2 2 = −2 M R I + 1 1 RM n n 1 T − a M − 2 RRM − 2 a n2 1 1 2 T 1 = 2 + 2 n−2 M R I n 1e1e1 RM . − Therefore, 2 n 2 n det ¯ = n 1 det det 2 det + 2 T = n 1 1+ 2 det M n−2 M ( R) I n 1e1e1 n−2 n 1 M, − − and 2 n det M¯ n 1 2 1 = − 1+ >e n+1 . det M n2 n 1 − Hence √ 1 vol EM,¯ z¯ det M − = <e 2(n+1) vol EM,z √det M¯ This proves Theorem 5. • Optimization Group 14/26 Proof of Theorem 4 E x : aT x aT z E . M,z ∩ ≤ ⊆ M,¯ z¯ n o Proof: Suppose (x z)T M(x z) 1 and aT x aT z.
Load more

Recommended publications
  • Simplex Algorithm
    CAP 4630 Artificial Intelligence Instructor: Sam Ganzfried [email protected] 1 HW2 • Due yesterday (10/18) at 2pm on Moodle • It is ok to work with one partner for homework 2. Must include document stating whom you worked with and describe extent of collaboration. This policy may be modified for future assignments. • Remember late day policy. – 5 total late days 2 Schedule • 10/12: Wrap-up logic (logical inference), start optimization (integer, linear optimization) • 10/17: Continue optimization (integer, linear optimization) • 10/19: Wrap up optimization (nonlinear optimization), go over homework 1 (and parts of homework 2 if all students have turned it in by start of class), midterm review • 10/24: Midterm • 10/26: Planning 3 Minesweeper • Minesweeper is NP-complete • http://simon.bailey.at/random/kaye.minesweeper.pdf 4 NP-completeness • Consider Sudoku, an example of a problem that is easy to verify, but whose answer may be difficult to compute. Given a partially filled-in Sudoku grid, of any size, is there at least one legal solution? A proposed solution is easily verified, and the time to check a solution grows slowly (polynomially) as the grid gets bigger. However, all known algorithms for finding solutions take, for difficult examples, time that grows exponentially as the grid gets bigger. So Sudoku is in NP (quickly checkable) but does not seem to be in P (quickly solvable). Thousands of other problems seem similar, fast to check but slow to solve. Researchers have shown that a fast solution to any one of these problems could be used to build a quick solution to all the others, a property called NP-completeness.
    [Show full text]
  • Computational Geometry Lecture Notes1 HS 2012
    Computational Geometry Lecture Notes1 HS 2012 Bernd Gärtner <[email protected]> Michael Hoffmann <[email protected]> Tuesday 15th January, 2013 1Some parts are based on material provided by Gabriel Nivasch and Emo Welzl. We also thank Tobias Christ, Anna Gundert, and May Szedlák for pointing out errors in preceding versions. Contents 1 Fundamentals 7 1.1 ModelsofComputation ............................ 7 1.2 BasicGeometricObjects. 8 2 Polygons 11 2.1 ClassesofPolygons............................... 11 2.2 PolygonTriangulation . 15 2.3 TheArtGalleryProblem ........................... 19 3 Convex Hull 23 3.1 Convexity.................................... 24 3.2 PlanarConvexHull .............................. 27 3.3 Trivialalgorithms ............................... 29 3.4 Jarvis’Wrap .................................. 29 3.5 GrahamScan(SuccessiveLocalRepair) . .... 31 3.6 LowerBound .................................. 33 3.7 Chan’sAlgorithm................................ 33 4 Line Sweep 37 4.1 IntervalIntersections . ... 38 4.2 SegmentIntersections . 38 4.3 Improvements.................................. 42 4.4 Algebraic degree of geometric primitives . ....... 42 4.5 Red-BlueIntersections . .. 45 5 Plane Graphs and the DCEL 51 5.1 TheEulerFormula............................... 52 5.2 TheDoubly-ConnectedEdgeList. .. 53 5.2.1 ManipulatingaDCEL . 54 5.2.2 GraphswithUnboundedEdges . 57 5.2.3 Remarks................................. 58 3 Contents CG 2012 6 Delaunay Triangulations 61 6.1 TheEmptyCircleProperty . 64 6.2 TheLawsonFlipalgorithm
    [Show full text]
  • Customized Book List Computer Science
    ABCD springer.com Springer Customized Book List Computer Science FRANKFURT BOOKFAIR 2007 springer.com/booksellers Computer Science 1 K. Aberer, Korea Advanced Institute of Science and Technolo- A. Abraham, Norwegian University of Science and Technology, gy, Korea; N. Noy, Stanford University, Stanford, CA, USA; D. Alle- Trondheim, Norway; Y. Dote, Muroran Institute of Technology, Transactions on Computational mang, TopQuadrant, CA, USA; K.-I. Lee, Saltlux Inc., Korea; L. Muroran, Japan Nixon, Free University Berlin, Germany; J. Goldbeck, University of Systems Biology VIII Maryland, MD, USA; P. Mika, Yahoo! Research Barcelona, Spain; D. Maynard, University of Sheffield, United Kingdom,; R. Mizoguchi, Engineering Hybrid Soft Computing Osaka University, Japan,; G. Schreiber, Free University Amster- dam, Netherlands,; P. Cudré-Mauroux, EPFL, Switzerland, (Eds.) Systems The LNCS journal Transactions on Computation- al Systems Biology is devoted to inter- and multi- disciplinary research in the fields of computer sci- The Semantic Web This book is focused on the latest technologies re- ence and life sciences and supports a paradigmat- 6th International Semantic Web Conference, 2nd Asian Se- lated to Hybrid Soft Computing targeting academia, ic shift in the techniques from computer and infor- mantic Web Conference, ISWC 2007 + ASWC 2007, Busan, Ko- scientists, researchers, and graduate students. mation science to cope with the new challenges aris- rea, November 11-15, 2007, Proceedings ing from the systems oriented point of view of bio- Features
    [Show full text]
  • How Do Exponential Size Solutions Arise in Semidefinite Programming?
    How do exponential size solutions arise in semidefinite programming? G´abor Pataki, Aleksandr Touzov ∗ June 30, 2021 Abstract Semidefinite programs (SDPs) are some of the most popular and broadly applicable optimiza- tion problems to emerge in the last thirty years. A curious pathology of SDPs, illustrated by a classical example of Khachiyan, is that their solutions may need exponential space to even write down. Exponential size solutions are the main obstacle to solve a long standing open problem: can we decide feasibility of SDPs in polynomial time? The consensus seems to be that large size solutions in SDPs are rare. Here we prove that they are actually quite common: a linear change of variables transforms every strictly feasible SDP into a Khachiyan type SDP, in which the leading variables are large. As to \how large", that depends on the singularity degree of a dual problem. Further, we present some SDPs in which large solutions appear naturally, without any change of variables. We also partially answer the question: how do we represent such large solutions in polynomial space? Key words: semidefinite programming; exponential size solutions; Khachiyan's example; facial reduction; singularity degree MSC 2010 subject classification: Primary: 90C22, 49N15; secondary: 52A40 OR/MS subject classification: Primary: convexity; secondary: programming-nonlinear-theory 1 Introduction Contents 1 Introduction 1 1.1 Notation and preliminaries..................................5 2 Main results and proofs6 2.1 Reformulating (P) and statement of Theorem1.......................6
    [Show full text]
  • Optimizing the Imbalances in a Graph Antoine Glorieux
    Optimizing the imbalances in a graph Antoine Glorieux To cite this version: Antoine Glorieux. Optimizing the imbalances in a graph. Discrete Mathematics [cs.DM]. Institut National des Télécommunications, 2017. English. NNT : 2017TELE0011. tel-01597059 HAL Id: tel-01597059 https://tel.archives-ouvertes.fr/tel-01597059 Submitted on 28 Sep 2017 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. THESE` DE DOCTORAT DE TEL´ ECOM´ SUDPARIS Sp´ecialit´e Math´ematiques Ecole´ doctorale Informatique, T´el´ecommunications et Electronique´ (Paris) i Pr´esent´ee par Antoine Glorieux Pour obtenir le grade de DOCTEUR DE TEL´ ECOM´ SUDPARIS Optimiser les d´es´equilibresdans un graphe soutenue le 19 juin 2017 devant le jury compos´ede : Rapporteur : Antoine Deza Professeur, McMaster University, Chaire de recherche du Canada en optimisation combi- natoire Rapporteur : Mourad Ba¨ıou Directeur de recherche, CNRS, Universit´e Blaise Pascal Clermont 2 Examinateur : Fr´ed´ericMeunier Professeur, Ecole´ nationale des Ponts et Chauss´ees Examinatrice : Marie-Christine Costa
    [Show full text]
  • Final Ctw 2017
    CITY TECH WRITER Volume 12 2017 Outstanding Student Writing From All Disciplines Suzanne Miller & Megan Behrent Editors Cover by: Kacper Zawadzki Art Director: Lloyd Carr About the Cover: I've always been fascinated with shape, color and movement. In composing this piece, my goal was to experiment with these visual design elements to create a piece that captures the community of City Tech and its various writing disciplines. Our diverse community is constantly in motion and being shaped with each passing semester. These are the works of the students who shape our school. —Kacper Zawadzki New York City College of Technology City University of New York Preface Welcome to City Tech Writer, Volume 12! Once again, our students have given us a range of experiences and ideas to contemplate. Chock full of intense and passionate pieces, this volume comprises essays that peer into the past, wrestle with the present, and glimpse the future: a human face is put on the refugee, the struggle of the PTSD victim is illuminated, and the Dreamer’s journey is considered. Fake news, the rise of computerized sportswear, the struggle of the honey bee, and the accidental creation of a sci-fi kitten are just some of the ideas in store… As we go to press, there are many people to thank: the faculty throughout the College who submitted extraordinary writing from their courses; Communication Design Department Chair Mary Ann Biehl for her support; Professor Peter Fikaris, whose communication design students produced many creative cover designs; Kacper Zawadzki, who designed the intriguing cover for this volume; and Ninoshka Mason and Vladimir Pinis, who were finalists for the cover art.
    [Show full text]
  • A Calendar of Mathematical Dates January
    A CALENDAR OF MATHEMATICAL DATES V. Frederick Rickey Department of Mathematical Sciences United States Military Academy West Point, NY 10996-1786 USA Email: fred-rickey @ usma.edu JANUARY 1 January 4713 B.C. This is Julian day 1 and begins at noon Greenwich or Universal Time (U.T.). It provides a convenient way to keep track of the number of days between events. Noon, January 1, 1984, begins Julian Day 2,445,336. For the use of the Chinese remainder theorem in determining this date, see American Journal of Physics, 49(1981), 658{661. 46 B.C. The first day of the first year of the Julian calendar. It remained in effect until October 4, 1582. The previous year, \the last year of confusion," was the longest year on record|it contained 445 days. [Encyclopedia Brittanica, 13th edition, vol. 4, p. 990] 1618 La Salle's expedition reached the present site of Peoria, Illinois, birthplace of the author of this calendar. 1800 Cauchy's father was elected Secretary of the Senate in France. The young Cauchy used a corner of his father's office in Luxembourg Palace for his own desk. LaGrange and Laplace frequently stopped in on business and so took an interest in the boys mathematical talent. One day, in the presence of numerous dignitaries, Lagrange pointed to the young Cauchy and said \You see that little young man? Well! He will supplant all of us in so far as we are mathematicians." [E. T. Bell, Men of Mathematics, p. 274] 1801 Giuseppe Piazzi (1746{1826) discovered the first asteroid, Ceres, but lost it in the sun 41 days later, after only a few observations.
    [Show full text]
  • Ellipsoid Method in Encyclopedia of Optimization, Second Edition, C.A
    Steffen Rebennack Ellipsoid Method in Encyclopedia of Optimization, Second Edition, C.A. Floudas and P.M. Pardalos (Eds.), Springer, pp. 890–899, 2008 Ellipsoid Method Steffen Rebennack University of Florida Center of Applied Optimization Gainesville, FL 32611, USA e-mail: [email protected] July 2007 Key words: Ellipsoid Method, linear programming, polynomially solvable, linear inequalities, separation problem In this article we give an overview of the Ellipsoid Method. We start with a historic introduction and provide a basic algorithm in Section 2. Techniques to avoid two important assumptions required by this algorithm are considered in Section 2.2. After the discussion of some implementation aspects, we are able to show the polynomial running time of the Ellipsoid Method. The second section is closed with some modifications in order to speed up the running time of the ellipsoid algorithm. In Section 3, we discuss some theoretical implications of the Ellipsoid Method to linear programming and combinatorial optimization. 1 Introduction In 1979, the Russian mathematician Leonid G. Khachiyan published his famous paper with the title “A Polynomial Algorithm in Linear Programming”, [Kha79]. He was able to show that linear programs (LPs) can be solved efficiently; more precisely that LP belongs to the class of polynomi- ally solvable problems. Khachiyan’s approach was based on ideas similar to the Ellipsoid Method arising from convex optimization. These methods were developed by David Yudin and Arkadi Nemirovski, [YN76a, YN76b, YN77], and independently by Naum Shor, [Sho77], preceded by other methods as for instance the Relaxation Method, Subgradient Method or the Method of Cen- tral Sections, [BGT81].
    [Show full text]
  • Rich Vehicle Routing: from a Set of Tailored Methods to a Generic
    Randomized Algorithms for Rich Vehicle Routing Problems: From a Specialized Approach to a Generic Methodology Jos´ede Jes´usC´aceresCruz Department of Computer Science, Multimedia and Telecommunication, Universitat Oberta de Catalunya - Internet Interdisciplinary Institute (UOC-IN3) Directors: Dr. Angel A. Juan P. & Dr. Daniel Riera T. Commitee Member: Dr. Helena R. Louren¸co PhD. Thesis Information and Knowledge Society Doctoral Programme Network and Information Technologies July 2013 ii Abstract The Vehicle Routing Problem (VRP) is a well known domain in optimization research community. Its different basic variants have been widely explored in the literature. Some studies have considered specific combinations of real- life constraints to define the emerging Rich VRP scopes. This work deals with the integration of heuristics, biased probability, simulation, parallel & distributed computing techniques, and constraint programming. The pro- posed approaches are tested for solving some variants of VRPs, namely, first, the deterministic families: Heterogeneous VRP (HVRP), Heteroge- neous VRP with Variable cost (HVRP-V), Heterogeneous fleet VRP with Multi-trips (HVRPM), Asymmetric cost matrix VRP (AVRP), Heteroge- neous fleet with Asymmetric cost matrix VRP (HAVRP), VRP with Time Windows (VRPTW), and Distance-Constrained VRP (DCVRP); second, the stochastic nature families: VRP with Stochastic Demands (VRPSD), and Inventory Routing Problem with Stochastic Demands (IRPSD). An extensive literature review is performed for all these variants, focusing on the main contributions of each work. A first approach proposes a biased- randomization of classical heuristics for solving the deterministic problems addressed here. A second approach is centered on the combination of ran- domized heuristics with simulation (Simheuristics) to be applied on the commented stochastic problems.
    [Show full text]
  • Reviving the Simplex Algorithm
    WUCSS 2016 April 5, 2016 Page 1 Reviving the simplex algorithm Sponsored by Walailak University (WU) Interdisciplinary Center of Scientific Computing (IWR) National Electronics and Computer Technology Center (NECTEC) The Development for the Promotion of Science and Technology Talents Project (DPST) Assistant Professor Krung Sinapiromsaran, Ph. D. Department of Mathematics and Computer Science Faculty of Science, Chulalongkorn University Applied Mathematics and Computational Sciences Page 2 Outline ● Linear programming history ● Linear programming problems and models ● The standard simplex algorithm from Dantzig ● Improvement of the simplex algorithm ● Cosine angle or minimal angle concept ● Artificial-free simplex algorithm ● Simplex algorithm with an unrestricted variable problem Page 3 http://pioneer.netserv.chula.ac.th/~skrung Page 4 Linear programming history •1827: Fourier-Motzkin elimination algorithm was proposed to reduce the number of variables in a k system of linear inequalities. Complexity O ( m 2 ) Joseph Fourier where k is the number of successive step and m is the number of inequalities. •1939: Leonid Kantorovich formulated and solved a planning for expenditures and returns during World Leonid KantorovichWar II •1975: Nobel prize in economics was award to Leonid Kantorovich together with Tjalling Charles Koopmans Tjalling Koopmans Source:http://www.wikipedia.org/wiki/Linear_programming 11 March 2016 Page 5 Linear programming history •1947: George B. Dantzig published the simplex method (the simplex algorithm) to efficiently find the solution of a linear programming problem for US Air Force. •1979: Leonid Khachiyan published the ellipsoid algorithm for solving a linear programming model in polynomial time. However, his algorithm is not Leonid Khachiyan practical for the real world problems.
    [Show full text]
  • 1 the Giants
    1 ”On the Shoulders of Giants” A brief excursion into the history of mathematical programming 1 R. Tichatschke Department of Mathematics, University Trier, Germany Similar to many mathematical fields also the topic of mathematical programming has its origin in applied problems. But, in contrast to other branches of mathematics, we don’t have to dig too deeply into the past centuries to find their roots. The historical tree of mathematical programming, starting from its conceptual roots to its present shape, is remarkably short, and to quote ISAAK NEWTON, we can say: ”We are standing on the shoulders of giants”. The goal of this paper is to describe briefly the historical growth of mathematical pro- gramming from its beginnings to the seventies of the last century and to review its basic ideas for a broad audience. During this process we will demonstrate that optimization is a natural way of thinking which follows some extremal principles. 1 The Giants Let us start with LEONHARD EULER. 'Leonhard Euler (1707-1783) $ 1727: Euler has been appointed professor (by Daniel Bernoul- li) at the University of Saint Petersburg, Russia. Member of St. Petersburg’s Academy of Science and since 1741 member of the Prussian Academy of Science, Berlin. 1744: Methodus inveniendi lineas curvas maximi minimive proprietate gaudentes, sive solutio proble- matis isoperimetrici latissimo sensu accepti. (Method to find curves, possessing some property in the most or smallest degree or the resolution of the Isoperimetric problem considered in the broadest sense.) In this work he has established the variational analysis in a systematic way. &1Part of a lecture held at the University of Trier on the occasion of the Year of Mathematics 2008 in% Germany; published in: Discussiones Mathematicae, vol.
    [Show full text]
  • Arxiv:2103.00041V1 [Math.OC] 26 Feb 2021 1.1 Notation and Preliminaries
    How do exponential size solutions arise in semidefinite programming? G´abor Pataki, Aleksandr Touzov ∗ March 2, 2021 Abstract As a classic example of Khachiyan shows, some semidefinite programs (SDPs) have solutions whose size { the number of bits necessary to describe them { is exponential in the size of the input. Exponential size solutions are the main obstacle to solve a long standing open problem: can we decide feasibility of SDPs in polynomial time? We prove that large solutions are actually quite common in SDPs: a linear change of variables transforms every strictly feasible SDP into a Khachiyan type SDP, in which the leading variables are large. As to \how large", that depends on the singularity degree of a dual problem. Further, we present some SDPs in which large solutions appear naturally, without any change of variables. We also partially answer the question: how do we represent such large solutions in polynomial space? Key words: semidefinite programming; exponential size solutions; Khachiyan's example; facial reduction; singularity degree MSC 2010 subject classification: Primary: 90C22, 49N15; secondary: 52A40 OR/MS subject classification: Primary: convexity; secondary: programming-nonlinear-theory 1 Introduction Contents 1 Introduction 1 arXiv:2103.00041v1 [math.OC] 26 Feb 2021 1.1 Notation and preliminaries . .5 2 Main results and proofs6 2.1 Reformulating (P) and statement of Theorem1.......................6 2.2 Proof of Theorem1.......................................9 2.3 Computing the exponents by Fourier-Motzkin elimination . 18 ∗Department of Statistics and Operations Research, University of North Carolina at Chapel Hill 1 3 When we do not even need a change of variables 19 4 Conclusion 22 Linear programs and polynomial size solutions The classical linear programming (LP) feasi- bility problem asks whether a system of linear inequalities Ax ≥ b has a solution, where the matrix A and the vector b both have integer entries.
    [Show full text]