DOCSLIB.ORG

Divide-And-Conquer Learning by Anchoring a Conical Hull

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Divide-And-Conquer Learning by Anchoring a Conical Hull Divide-and-Conquer Learning by Anchoring a Conical Hull Tianyi Zhouy, Jeff Bilmesz, Carlos Guestriny yComputer Science & Engineering, zElectrical Engineering, University of Washington, Seattle ftianyizh, bilmes, [email protected] Abstract We reduce a broad class of fundamental machine learning problems, usually addressed by EM or sampling, to the problem of finding the k extreme rays spanning the conical hull of a1 data point set. These k “anchors” lead to a global solution and a more interpretable model that can even outperform EM and sampling on generalization error. To find the k anchors, we propose a novel divide-and- conquer learning scheme “DCA” that distributes the problem to O(k log k) same- type sub-problems on different low-D random hyperplanes, each can be solved independently by any existing solver. For the 2D sub-problem, we instead present a non-iterative solver that only needs to compute an array of cosine values and its max/min entries. DCA also provides a faster subroutine inside other algorithms to check whether a point is covered in a conical hull, and thus improves these algorithms by providing significant speedups. We apply our method to GMM, HMM, LDA, NMF and subspace clustering, then show its competitive performance and scalability over other methods on large datasets. 1 Introduction Expectation-maximization (EM) [10], sampling methods [13], and matrix factorization [20, 25] are three algorithms commonly used to produce maximum likelihood (or maximum a posteriori (MAP)) estimates of models with latent variables/factors, and thus are used in a wide range of applications such as clustering, topic modeling, collaborative filtering, structured prediction, feature engineering, and time series analysis. However, their learning procedures rely on alternating optimization/updates between parameters and latent variables, a process that suffers from local optima. Hence, their quality greatly depends on initialization and on using a large number of iterations for proper convergence [24]. The method of moments [22, 6, 17], by contrast, solves m equations by relating the first m moments of observation x 2 Rp to the m model parameters, and thus yields a consistent estimator with a global solution. In practice, however, sample moments usually suffer from unbearably large variance, which easily leads to the failure of final estimation, especially when m or p is large. Although recent spectral methods [8, 18, 15, 1] reduces m to 2 or 3 when estimating O(p) m parameters [2] by relating the eigenspace of lower-order moments to parameters in a matrix form up to column scale, the variance of sample moments is still sensitive to large p or data noise, which may result in poor estimation. Moreover, although spectral methods using SVDs or tensor decomposition evidently simplifies learning, the computation can still be expensive for big data. In addition, recovering a parameter matrix with uncertain column scale might not be feasible for some applications. In this paper, we reduce the learning in a rich class of models (e.g., matrix factorization and latent variable model) to finding the extreme rays of a conical hull from a finite set of real data points. This is obtained by applying a general separability assumption to either the data matrix in matrix factorization or the 2nd/3rd order moments in latent variable models. Separability posits that a ground set of n points, as rows of matrix X, can be represented by X = FXA, where the rows (bases) in XA are a subset A ⊂ V = [n] of rows in X, which are called “anchors” and are interesting to various 1 models when jAj = k n. This property was introduced in [11] to establish the uniqueness of non-negative matrix factorization (NMF) under simplex constraints, and was later [19, 14] extended to non-negative constraints. We generalize it further to the model X = FYA for two (possibly distinct) finite sets of points X and Y , and build a new theory for the identifiability of A. This generalization enables us to apply it to more general models (ref. Table1) besides NMF. More interestingly, it leads to a learning method with much higher tolerance to the variance of sample moments or data noise, a unique global solution, and a more interpretable model. Cone(Y Another primary contribution of this paper is a dis- A) tributed learning scheme “divide-and-conquer anchor- X = { } Y = { } ing” (DCA), for finding an anchor set A such that YA = { } X = FYA by solving same-type sub-problems on only H O(k log k) randomly drawn low-dimensional (low-D) XΦ = { } hyperplanes. Each sub-problem is of the form of cone YΦ = { } YÃ Φ = { } ( (XΦ) = F · (Y Φ)A with random projection matrix Y Ã Φ Φ, and can easily be handled by most solvers due to ) the low dimension. This is based on the observation O that the geometry of the original conical hull is partially preserved after a random projection. We analyze the Figure 1: Geometry of general minimum conical hull prob- probability of success for each sub-problem to recover lem and basic idea of divide-and-conquer anchoring (DCA). part of A, and then study the number of sub-problems for recovering the whole A with high probability (w.h.p.). In particular, we propose an very fast non-iterative solver for sub-problems on the 2D plane, which requires computing an array of cosines and its max/min values, and thus results in learning algorithms with speedups of tens to hundreds of times. DCA improves multiple aspects of algorithm design since: 1) its idea of divide-and-conquer randomization gives rise to distributed learning that can reduce the original problem to multiple extremely low-D sub-problems that are much easier and faster to solve, and 2) it provides a fast subroutine, checking if a point is covered by a conical hull, which can be embedded into other solvers. We apply both the conical hull anchoring model and DCA to five learning models: Gaussian mixture models (GMM) [27], hidden Markov models (HMM) [5], latent Dirichlet allocation (LDA) [7], NMF [20], and subspace clustering (SC) [12]. The resulting models and algorithms show significant improvement in efficiency. On generalization performance, they consistently outperform spectral methods and matrix factorization, and are comparable to or even better than EM and sampling. In the following, we will first generalize the separability assumption and minimum conical hull problem risen from NMF in x2, and then show how to reduce more general learning models to a (general) minimum conical hull problem in x3. x4 presents a divide-and-conquer learning scheme that can quickly locate the anchors of the conical hull by solving the same problem in multiple extremely low-D spaces. Comprehensive experiments and comparison can be found in x5. 2 General Separability Assumption and Minimum Conical Hull Problem The original separability property [11] is defined on the convex hull of a set of data points, namely that each point can be represented as a convex combination of certain subsets of vertices that define the convex hull. Later works on separable NMF [19, 14] extend it to the conical hull case, which replaced convex with conical combinations. Given the definition of (convex) cone and conical hull, the separability assumption can be defined both geometrically and algebraically. Definition 1 (Cone & conical hull). A (convex) cone is a non-empty convex set that is closed with respect to conical combinations of its elements. In particular, cone(R) can be defined by its k k generators (or rays) R = frigi=1 such that Xk cone(R) = αiri j ri 2 R; αi 2 + 8i : (1) i=1 R See [29] for the original separability assumption, the equivalence between separable NMF and the minimum conical hull problem, which is defined as a submodular set cover problem. 2.1 General Separability Assumption and General Minimum Conical Hull Problem By generalizing the separability assumption, we obtain a general minimum conical hull problem that can reduce more general learning models besides NMF, e.g., latent variable models and matrix factorization, to finding a set of “anchors” on the extreme rays of a conical hull. 2 Definition 2 (General separability assumption). All the n data points(rows) in X are covered in a finitely generated and pointed cone (i.e., if x 2 cone(YA) then −x 62 cone(YA)) whose generators form a subset A ⊆ [m] of data points in Y such that @i 6= j; YAi = a · YAj . Geometrically, it says 8i 2 [n];Xi 2 cone (YA) ;YA = fyigi2A: (2) 0 (n−k)×k An equivalent algebraic form is X = FYA, where jAj = k, F 2 S ⊆ R+ . (n−k)×k When X = Y and S = R+ , it degenerates to the original separability assumption given in [29]. We generalize the minimum conical hull problem from [29]. Under the general separability assumption, it aims to find the anchor set A from the points in Y rather than X. Definition 3 (General Minimum Conical Hull Problem). Given a finite set of points X and a set Y having an index set V = [m] of its rows, the general minimum conical hull problem finds the subset of rows in Y that define a super-cone for all the rows in X. That is, find A 2 2V that solves: min jAj; s.t., cone(YA) ⊇ cone(X): (3) A⊂V where cone(YA) is the cone induced by the rows A of Y . When X = Y , this also degenerates to the original minimum conical hull problem defined in [29]. A critical question is whether/when the solution A is unique. When X = Y and X = FXA, by following the analysis of the separability assumption in [29],we can prove that A is unique and identifiable given X.
Load more

Recommended publications
  • Arxiv:1908.06506V2 [Math.CO] 14 Aug 2020
    POSITIONAL VOTING AND DOUBLY STOCHASTIC MATRICES JACQUELINE ANDERSON, BRIAN CAMARA, AND JOHN PIKE Abstract. We provide elementary proofs of several results concerning the possible outcomes arising from a fixed profile within the class of positional voting systems. Our arguments enable a simple and explicit construction of paradoxical profiles, and we also demonstrate how to choose weights that realize desirable results from a given profile. The analysis ultimately boils down to thinking about positional voting systems in terms of doubly stochastic matrices. 1. Introduction Suppose that n candidates are running for a single office. There are many different social choice pro- cedures one can use to select a winner. In this article, we study a particular class called positional voting systems. A positional voting system is an electoral method in which each voter submits a ranked list of the candidates. Points are then assigned according to a fixed weighting vector w that gives wi points to a candidate every time they appear in position i on a ballot, and candidates are ranked according to the total number of points received. For example, plurality is a positional voting system with weighting vector w = [ 1 0 0 ··· 0 ]T. One point is assigned to each voter's top choice, and the candidate with the most points wins. The Borda count is another common positional voting system in which the weighting vector is given by w = [ n − 1 n − 2 ··· 1 0 ]T. Other examples include the systems used in the Eurovision Song Contest, parliamentary elections in Nauru, and the AP College Football Poll [2,8,9].
    [Show full text]
  • FUNDAMENTALS Contents 1. Mathematical Backgrounds 1 1.1
    FUNDAMENTALS HANG SI Contents 1. Mathematical Backgrounds 1 1.1. Convex sets, convex hulls 1 1.2. Simplicial complexes, triangulations 3 1.3. Planar graphs, Euler's formula 5 2. Algorithm Backgrounds 8 2.1. Algorithmic foundations 9 2.2. Incremental construction 11 2.3. Randomized algorithms 15 Exercises 18 References 19 In order to well-understood the discussion and the analysis of algorithms in this chap- ter, it is necessary to understand the fundamental geometry and combinatorics of trian- gulations. We will first present the basic and related materials in this chapter. 1. Mathematical Backgrounds 1.1. Convex sets, convex hulls. d 1.1.1. Linear, affine, and convex combinations. The space R is a vector space. A linear d subspace of R is closed under addition of vectors and under multiplication of (scalar) real numbers. For example, the linear subspaces of R2 are the origin 0, all lines passing through origin, and R2 itself. Basic geometric objects are points, lines, planes and so forth, which are affine sub- spaces, also called flats. In an affine subspaces, lines are not passing through 0. An d affine subspace of R has the form x + L where x is some vector and L is a subspace of d R . Thus non-empty affine subspaces are the translates of linear subspaces. d Let a1, a2, :::, an be points in R , and α1, α2, :::, αn be (scalar) real value in R, then a linear combination of a1, a2, :::, an is a point: d α1a1 + α2a2 + + αnan R : ··· 2 In particular, it is n an affine combination if αi = 1, or • i=1 a conical combination if αi 0; i, or • Pn≥ 8 a convex combination if i=1 αi = 1, and αi 0; i.
    [Show full text]
  • Projecting Onto the Intersection of a Cone and a Sphere
    Projecting onto the intersection of a cone and a sphere Heinz H. Bauschke,∗ Minh N. Bui,† and Xianfu Wang‡ April 12, 2018 Abstract The projection onto the intersection of sets generally does not allow for a closed form even when the individual projection operators have explicit descriptions. In this work, we systematically an- alyze the projection onto the intersection of a cone with either a ball or a sphere. Several cases are provided where the projector is available in closed form. Various examples based on finitely gen- erated cones, the Lorentz cone, and the cone of positive semidefinite matrices are presented. The usefulness of our formulae is illustrated by numerical experiments for determining copositivity of real symmetric matrices. 2010 Mathematics Subject Classification: Primary 47H09, 52A05, 90C25, 90C26; Secondary 15B48, 47H04. Keywords: ball, convex set, convex cone, copositive matrix, projection, projector, sphere. 1 Introduction Throughout this paper, we assume that is a real Hilbert space (1.1) H with inner product and induced norm . Let K and S be subsets of , with associated projec- h·|·i k · k H tion operator (or projectors) PK and PS, (1.2) respectively. Our aim is to derive a formula for the projector of the intersection PK S. (1.3) ∩ arXiv:1708.00585v2 [math.OC] 12 Apr 2018 Only in rare cases is it possible to obtain a “closed form” for PK S in terms of PK and PS: e.g., when ∩ K and S are either both half-spaces (Haugazeau; see [10] and also [3, Corollary 29.25]) or both sub- spaces (Anderson–Duffin; see [1, Theorem 8] and also [3, Corollary 25.38]).
    [Show full text]
  • Pseudometrics, the Complex of Ultrametrics, and Iterated Cycle Structures
    University of New Hampshire University of New Hampshire Scholars' Repository Doctoral Dissertations Student Scholarship Spring 2019 Pseudometrics, The Complex of Ultrametrics, and Iterated Cycle Structures Eric Robert Kehoe University of New Hampshire, Durham Follow this and additional works at: https://scholars.unh.edu/dissertation Recommended Citation Kehoe, Eric Robert, "Pseudometrics, The Complex of Ultrametrics, and Iterated Cycle Structures" (2019). Doctoral Dissertations. 2451. https://scholars.unh.edu/dissertation/2451 This Dissertation is brought to you for free and open access by the Student Scholarship at University of New Hampshire Scholars' Repository. It has been accepted for inclusion in Doctoral Dissertations by an authorized administrator of University of New Hampshire Scholars' Repository. For more information, please contact [email protected]. Pseudometrics, The Complex of Ultrametrics, and Iterated Cycle Structures BY Eric R. Kehoe DISSERTATION Submitted to the University of New Hampshire in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in Mathematics May, 2019 ALL RIGHTS RESERVED c 2019 Eric R. Kehoe ii This thesis/dissertation has been examined and approved in partial fulfillment of the re- quirements for the degree of Doctor of Philosophy in Mathematics by: Thesis/Dissertation Director, David Feldman, Professor of Mathematics and Statistics John Gibson, Professor of Mathematics and Statistics Don Hadwin, Professor of Mathematics and Statistics Maria Basterra, Professor of Mathematics and Statistics Dmitri Nikshych, Professor of Mathematics and Statistics On April 17th, 2019 Original approval signatures are on file with the University of New Hampshire Graduate School. iii DEDICATION To my love Emily. iv ACKNOWLEDGEMENTS I would like to thank my parents for always supporting my dream of becoming a mathemati- cian.
    [Show full text]
  • Control Techniques for Constrained Discrete Time Systems
    Control techniques for constrained discrete time systems Paolo d’Alessandro, Elena De Santis April 27, 2004 ii Contents Introduction vii 0.1 Notations terminology and recalls . vii iii iv CONTENTS Preface ??? v vi PREFACE Introduction Reading the book requires some mathematical background. This is proposed in form functional to the present purposes in [7] and consists in the …rst place of rudiments of topology, linear spaces, Hilbert spaces and convex analysis. The style of exposition therein is concise and therefore for most proofs and details it is useful to make reference to the classical books: [8], [11] and [12]. In addition to that, we need here some of the results contained in this latter book, especially some basic ideas underlying the conical approach and the main results of the dual conical methods for the solution of linear feasibility and optimality problems. Rather then duplicating the presentation in [7], the following section aims at establishing the notations used in the present book as well as providing a sort of check list for the reader of the concept and results that will be used in the sequel. It does not substitute reading the above references, unless one is already familiar with the required background. We do not exclude a few re…nements later on, wherever required. 0.1 Notations terminology and recalls We shall consider, unless otherwise stated, points and sets in the Euclidean real spaces Rn equipped with the usual product topology, even though many of the involved concept are valid for more general linear spaces. At times we refer to a generic linear space, especially when there is no additional e¤ort to achieve maximum generality.
    [Show full text]
  • Lecture 2: Convex Sets
    Lecture 2: Convex sets August 28, 2008 Lecture 2 Outline • Review basic topology in Rn • Open Set and Interior • Closed Set and Closure • Dual Cone • Convex set • Cones • Affine sets • Half-Spaces, Hyperplanes, Polyhedra • Ellipsoids and Norm Cones • Convex, Conical, and Affine Hulls • Simplex • Verifying Convexity Convex Optimization 1 Lecture 2 Topology Review n Let {xk} be a sequence of vectors in R n n Def. The sequence {xk} ⊆ R converges to a vector xˆ ∈ R when kxk − xˆk tends to 0 as k → ∞ • Notation: When {xk} converges to a vector xˆ, we write xk → xˆ n • The sequence {xk} converges xˆ ∈ R if and only if for each component i: the i-th components of xk converge to the i-th component of xˆ i i |xk − xˆ | tends to 0 as k → ∞ Convex Optimization 2 Lecture 2 Open Set and Interior Let X ⊆ Rn be a nonempty set Def. The set X is open if for every x ∈ X there is an open ball B(x, r) that entirely lies in the set X, i.e., for each x ∈ X there is r > 0 s.th. for all z with kz − xk < r, we have z ∈ X Def. A vector x0 is an interior point of the set X, if there is a ball B(x0, r) contained entirely in the set X Def. The interior of the set X is the set of all interior points of X, denoted by R (X) • How is R (X) related to X? 2 • Example X = {x ∈ R | x1 ≥ 0, x2 > 0} R 2 (X) = {x ∈ R | x1 > 0, x2 > 0} R n 0 (S) of a probability simplex S = {x ∈ R | x 0, e x = 1} Th.
    [Show full text]
  • Linear Algebra
    Linear algebra Integer linear programming: formulations, techniques and applications. 03 Santiago Valdés Ravelo [email protected] August, 2020 “Algebra is the offer made by the devil to the mathematician. The devil says: I will give you this powerful machine, it will answer any question you like. All you need to do is give me your soul: give up geometry and you will have this marvellous machine.”1 Michael F. Atiyah. 1Michael F. Atiyah, “Special Article: Mathematics in the 20푡ℎ Century”, Bulletin of the London Mathematical Society, 2002, p. 7. 1 Vectors 2 Matrices 3 Linear systems 4 Exercises Vectors Vectors Definition Finite sequence of real values: ⎫ value 1 } } } value 2 } } } value 3 } 푛 + ⎬ 푣 ∈ ℝ , where 푛 ∈ ℕ … } } value 푛 − 1 } } } value 푛 } } ⎭ Vectors Special vectors. Null Vector with all the elements equal to zero (null): ⎫ 0 } } 0 } } 푣 = [0, 0, … , 0]푡 } … 푣 ⎬ } 0 } } } 0 } ⎭ Vectors Special vectors. Unit Vector with one element equals to 1 and the rest equal to 0: ⎫ 0 } } } … } } 0 } } 푣 = [0, … , 0, 1, 0, … , 0]푡 } 1 ⎬ 푣 } 0 } } } … } } 0 } } * unit vectors are those whose norm is equals to 1 ⎭ Vectors Operations. Sum Sum the elements at the same position: ⎫ } 푛 푣1 + 푤1 } 푣, 푤 ∈ ℝ ∶ } } 푣2 + 푤2 } 푣 + 푤 = [푣 , 푣 , … , 푣 ]푡 + [푤 , 푤 , … , 푤 ]푡 푣 + 푤 1 2 푛 1 2 푛 … ⎬ } } 푡 푣푛 + 푤푛 } 푣 + 푤 = [푣1 + 푤1, 푣2 + 푤2, … , 푣푛 + 푤푛] } } ⎭ Vectors Operations. Scalar multiplication Multiply each element of the vector by the scalar (real number): ⎫ } 푛 휆 × 푣1 } 휆 ∈ ℝ, 푣 ∈ ℝ ∶ } } 휆 × 푣2 } 휆 × 푣 = 휆 × [푣 , 푣 , … , 푣 ]푡 휆 × 푣 1 2 푛 … ⎬ } } 푡 휆 × 푣푛 } 휆 × 푣 = [휆 × 푣1, 휆 × 푣2, … , 휆 × 푣푛] } } ⎭ Vectors Operations.
    [Show full text]
  • Mtftfl the Degree .CH?
    COMPUTATIONAL VERIFICATION OF THE CONE CONJECTURE A thesis presented to the faculty of 3 4 , San Francisco State University -Z -o \8 In partial fulfilment of The Requirements for MtfTfl The Degree .CH? Master of Arts In Mathematics by Tsz Leong Chan San Francisco, California December 2018 Copyright by Tsz Leong Chan 2018 CERTIFICATION OF APPROVAL I certify that I have read COMPUTATIONAL VERIFICATION OF THE CONE CONJECTURE by Tsz Leong Chan and that in my opinion this work meets the criteria for approving a thesis submitted m par­ tial fulfillment of the requirements for the degree: Master of Arts in Mathematics at San Francisco State University. Dr. Joseph Gubeladze Professor of Mathematics Dr. Matthias Beck Professor of Mathematics Dr. Serkan Hosten Professor of Mathematics COMPUTATIONAL VERIFICATION OF THE CONE CONJECTURE Tsz Leong Char San Francisco State University 2018 The set of polyhedral pointed rational cones form a partially ordered set with respect to elementary additive extensions of certain type. This poset captures a global picture of the interaction of all possible rational cones with the integer lattice and, also, provides an alternative approach- to another important poset, the poset of normal polytopes. One of the central conjectures in the field, is the s.c. Cone Conjecture: the order on cones is the inclusion order. The con­ jecture has been proved only in dimensions up to 3. In this work we develop an algorithmic approach to the conjecture in higher dimensions. Namely, we study how often two specific types of cone extensions generate the chains of cones in dimensions 4 and 5, whose existence follows from the Cone Conjecture.
    [Show full text]
  • Lectures on Integer Programming
    Lectures on Integer Programming L.E. Trotter, Jr. School of OR&IE Cornell University Ithaca, New York 14853 June 16, 2004 Abstract Lecture notes for the “Doctoral School in Discrete Systems Optimization,” E.P.F.- Lausanne, June 2004. This material stresses fundamental aspects of geometry and duality for Integer Programming. 1 Linear Spaces n We generally deal with IR = x = (x1,...,xn) : xj IR j , the n dimensional eu- clidean vector space over the reals{ IR. Sometimes attention∈ is restricted∀ } to−Qn, the rational n vectors. Familiarity with manipulation of vectors and matrices is assumed, but we review the− formal properties of vector spaces, stressing the manner in which the theory of linear equality systems extends naturally to linear inequality systems. Subspaces As a vector space, IRn must obey certain axioms. First, IRn is an abelian group with respect to (vector) addition: there is a zero element 0 = (0,..., 0) IRn, each element has an ∈ inverse, and vector addition is associative and commutative. 0+ a = a, a IRn ∀ ∈ n a +( a)=0, a IR (a + b)+ c = a +(− b + c), ∀ a,∈ b, c IRn a + b = b + a, ∀ a, b ∈IRn ∀ ∈ Next, IRn satisfies IR-module properties governing the action of IR on IRn via (scalar) mul- tiplication: there is a unit scalar 1 IR, scalar multiplication is associative, and scalar multiplication distributes over (both vector∈ and scalar) addition. 1a = a, a IRn ∀ ∈ n λ(µa)=(λµ)a, a IR ; λ,µ IR λ(a + b)= λa + λb, ∀ a,∈ b IRn∀; λ ∈IR ∀ ∈ n ∀ ∈ (λ + µ)a = λa + µa, a IR ; λ,µ IR ∀ ∈ ∀ ∈ Finally, IRn is closed with respect to vector addition and scalar multiplication.
    [Show full text]
  • Convex Combinations
    Convexity and Polyhedra Carlo Mannino (from Geir Dahl notes on convexity) University of Oslo, INF-MAT5360 - Autumn 2011 (Mathematical optimization) Convex Sets n Set C R is convex if (1- )x1 + x2 C whenever x1 , x2 C 0 ≤ ≤ 1 (the segment joining x1 , x2 is contained in C) x x2 1 non-convex x2 convex x1 • Show that the unit ball B = {x Rn : ||x|| ≤ 1} is convex. (Hint use the triangle inequality ||x+y|| ≤ ||x||+ ||y||) Half-spaces n T Example of convex sets: half-spaces H = {xR : a x ≤ a0} H x H aTx ≤ a (1- ) aTx ≤ (1- ) a 1 1 0 1 0 0 ≤ ≤ 1 T T x2 H a x2 ≤ a0 a x2 ≤ a0 T summing up x2 H a ((1- ) x1 + x2) ≤ a0 (1- ) x1 + x2 H Convex Cones The set of solutions to a linear system of equation is a polyhedron. n n H = {xℝ : Ax = b} H = {xℝ : Ax ≤ b , -Ax ≤ -b } n Convex Cone: C ⊆ℝ if 휆1 x1 + 휆2 x2 ∈ C whenever x1, x2 ∈ C and 휆1 , 휆2 ≥ 0. Each convex cone is a convex set. (show) m,n n Let A ∈ ℝ . Then C = {xℝ : Ax ≤ 0} is a convex cone (show). n 푡 Let x1,…,xt ∈ ℝ , and 휆1,…,휆 t ≥ 0. The vector x = 푗=1 휆푗xj is a nonnegative (or conical) combination of x1,…,xt n The set C(x1,…,xt) of all nonnegative combinations of x1,…,xt ∈ ℝ is a convex cone (show), called finitely generated cone. Linear Programming Property: C1 , C2 convex sets → C1 ∩ C2 convex (show!) Linear programming: x = (x1, …, xn) maximize c1x1 + … + cn xn Subject to 푎11x1 + … +푎 1푛 xn ≤ 푏1 ⋮ P 푎푚1x1 + +푎 푚 푛 xn ≤ 푏푚 x1 , …, xn ≥ 0 max {cTx: x ϵ P }, with P = {xRn: Ax ≤ b, x≥ 0} Find the optimum solution in P P intersection of a finite number of half-spaces: convex set (polyhedron) The set of optimal solutions to a linear program is a polyhedron (show!) Convex Combinations n 푡 Let x1,…,xt ∈ ℝ , and 휆1,…,휆 t ≥ 0, such that 푗=1 휆푗 = 1.
    [Show full text]
  • On the Polyhedrality of the Chvátal-Gomory Closure
    On the Polyhedrality of the Chv´atal-Gomory Closure Haoran Zhu ∗ Abstract In this paper, we provide an equivalent condition for the Chv´atal-Gomory (CG) closure of a closed convex set to be finitely-generated. Using this result, we are able to prove that, for any closed convex set that can be written as the Minkowski sum of a compact convex set and a closed convex cone, its CG closure is a rational polyhedron if and only if its recession cone is a rational polyhedral cone. As a consequence, this generalizes and unifies all the currently known results, for the case of rational polyhedron [24] and compact convex set [9]. Key words: Chv´atal-Gomory closure polyhedral cutting-planes · · 1 Introduction Cutting-plane method is one of the most fundamental techniques for solving (mixed) integer pro- gramming problems, and often times in practice, it is combined with the branch-and-bound method. Since the early days of Integer Programming (IP), numerous types of cutting-planes have been in- troduced and studied in the literature, several of them have also been widely implemented into commercial solvers. Among those cuts, Chv`atal-Gomory (CG) cut ([18, 5]) was the first cutting- plane that has ever been proposed, and various interesting results have been obtained from both the theoretical and practical point of view (see, e.g., [6, 3, 15]). One of the theoretical questions regarding to CG cut is, what are the structural properties of the region obtained from intersecting all of those cuts? In the terminology of cutting-plane theory, this region is referred to as Chv`atal-Gomory (CG) closure.
    [Show full text]
  • Fundamentals of Convex Analysis Theory and Decision Library
    FUNDAMENTALS OF CONVEX ANALYSIS THEORY AND DECISION LIBRARY General Editors: W. Leinfellner (Vienna) and G. Eberlein (Munich) Series A: Philosophy and Methodology of the Social Sciences Series B: Mathematical and Statistical Methods Series C: Game Theory, Mathematical Programming and Operations Research Series D: System Theory, Knowledge Engineering and Problem Solving SERIES B: MATHEMATICAL AND STATISTICAL METHODS VOLUME 24 Editor: H. J. Skala (Paderborn); Assistant Editor: M. Kraft (Paderborn); Editorial Board: 1. Aczel (Waterloo, Ont.), G. Bamberg (Augsburg), H. Drygas (Kassel), W. Eichhorn (Karlsruhe), P. Fishburn (Murray Hill, N.J.), D. Fraser (Toronto), W. Janko (Vienna), P. de Jong (Vancouver), T. Kariya (Tokyo), M. Machina (La Jolla, Calif.), A. Rapoport (Toronto), M. Richter (Kaiserslautern), B. K. Sinha (Cattonsville, Md.), D. A. Sprott (Waterloo, Ont.), P. Suppes (Stanford, Calif.), H. Theil (Gainesville, Fla.), E. Trillas (Madrid), L. A. Zadeh (Berkeley, Calif.). Scope: The series focuses on the application of methods and ideas of logic, mathematics and statistics to the social sciences. In particular, formal treatment of social phenomena, the analysis of decision making, information theory and problems of inference will be central themes of this part of the library. Besides theoretical results, empirical investigations and the testing of theoretical models of real world problems will be subjects of interest. In addition to emphasizing interdisciplinary communication, the series will seek to support the rapid dissemination of recent results. The titles published in this series are listed at the end of this volume. FUNDAMENTALS OF CONVEX ANALYSIS Duality, Separation, Representation, and Resolution by MICHAEL J. PANIK Department of Economics, University of Hartford, West Hartford, Connecticut, U.S.A.
    [Show full text]