DOCSLIB.ORG

Geometrically Enriched Latent Spaces

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Geometrically Enriched Latent Spaces Geometrically Enriched Latent Spaces Georgios Arvanitidis Søren Hauberg Bernhard Sch¨olkopf MPI for Intelligent Systems, T¨ubingen DTU Compute, Lyngby MPI for Intelligent Systems, T¨ubingen Abstract A common assumption in generative mod- els is that the generator immerses the latent space into a Euclidean ambient space. In- stead, we consider the ambient space to be a Riemannian manifold, which allows for en- coding domain knowledge through the asso- ciated Riemannian metric. Shortest paths can then be defined accordingly in the la- tent space to both follow the learned manifold and respect the ambient geometry. Through Figure 1: The proposed shortest path ( ) favors the careful design of the ambient metric we can smiling class, while the standard shortest path ( ) ensure that shortest paths are well-behaved merely minimizes the distance on the data manifold. even for deterministic generators that other- wise would exhibit a misleading bias. Ex- perimentally we show that our approach im- proves the interpretability and the function- lines in Z are not shortest paths in any meaningful ality of learned representations both using sense, and therefore do not constitute natural inter- stochastic and deterministic generators. polants. To overcome this issue, it has been proposed to endow the latent space with a Riemannian met- ric such that curve lengths are measured in the ambi- 1 Introduction ent observation space X (Tosi et al., 2014; Arvanitidis et al., 2018). In other words, this ensures that any Unsupervised representation learning has made smooth invertible transformation of Z does not change tremendous progress with generative models such the distance between a pair of points, as long as the as variational autoencoders (VAEs) (Kingma and ambient path in X remains the same. This approach Welling, 2014; Rezende et al., 2014) and generative immediately solves the identifiability problem. adversarial networks (GANs) (Goodfellow et al., While distances in X are well-defined and give rise to 2014). These, and similar, models provide a flexible an identfiable latent representation, they need not be and efficient parametrization of the density of obser- particularly useful. We take inspiration from metric vations in an ambient space X through a typically learning (Weinberger et al., 2006; Arvanitidis et al., lower dimensional latent space Z. 2016) and propose to equip the ambient observation While the latent space Z constitutes a compressed space X with a Riemannian metric and measure curve representation of the data, it is by no means unique. lengths in latent space accordingly. With this ap- Like most other latent variable models, these genera- proach it is straight-forward to steer shortest paths in tive models are subject to identifiability problems, such latent space to avoid low-density regions, but also to that different representations can give rise to identical incorporate higher level semantic information. For in- densities (Bishop, 2006). This implies that straight stance, Fig.1 shows a shortest path under an ambient metric that favors images of smiling people. In such a Proceedings of the 24th International Conference on Artifi- way, we can control, and potentially unbias, distance cial Intelligence and Statistics (AISTATS) 2021, San Diego, based methods by utilizing domain knowledge, for ex- California, USA. PMLR: Volume 130. Copyright 2021 by ample in an individual fairness scenario. Hence, we the author(s). get both identifiable and useful latent representations. Geometrically Enriched Latent Spaces D x point x 2 M. Hence, v 2 TxM is a vector v 2 R and v actually the Riemannian metric is M : M! D×D. v X R0 Thus, the simplest approach is to assume that X is equipped with the Euclidean metric MX (x) = ID and γ(t) γ(t) its restriction is utilized as the Riemannian metric on y TxM. Since the choice of MX (·) has a direct impact on M, we can utilize other metrics designed to encode x y high-level semantic information (see Sec.3). Figure 2: Examples of a tangent vector ( ) and a Another view is to consider as smooth manifold the shortest path ( ) on an embedded M ⊂ X (left) and D D whole ambient space X = R . Hence, the TxX = R on an ambient X (right). is centered at x 2 RD and again the simplest Rieman- nian metric is the Euclidean MX (x) = ID. However, we are able to use other suitable metrics that simply In summary, we consider the ambient space of a gen- change the way we measure distances in X (see Sec.3). erative model as a Riemannian manifold, where the For instance, given a set of points in X we can con- metric can be defined by the user in order to encode struct a metric with small magnitude near the data to high level information about the problem of interest. pull the shortest paths towards them (see Fig.2 right). In such a way, the resulting shortest paths in the latent For a d-dimensional embedded manifold M ⊂ X , a col- space move optimally on the data manifold, while re- lection of chart maps φ : U ⊂ M ! d is used to as- specting the geometry of the ambient space. This can i i R sign local intrinsic coordinates to neighborhoods U ⊂ be useful in scenarios where a domain expert wants i M, and for simplicity, we assume that a global chart to control the shortest paths in an interpretable way. map φ(·) exists. By definition, when M is smooth the In addition, we propose a simple method to construct φ(·) and its inverse φ−1 : φ(M) ⊂ d ! M ⊂ X exist diagonal metrics in the ambient space, as well as an R and are smooth maps. Thus, a vx 2 TxM can be ex- architecture for the generator in order to extrapolate d pressed as vx = Jφ−1 (z)vz, where z = φ(x) 2 R and meaningfully. We show how this enables us to prop- d vz 2 R are the representations in the intrinsic coor- erly capture the geometry of the data manifold in de- D×d dinates. Also, the Jacobian J −1 (z) 2 defines a terministic generators, which is otherwise infeasible. φ R basis that spans the TxM, and thus, we represent the ambient metric MX (·) in the intrinsic coordinates as 2 Applied Riemannian geometry intro | −1 hvx; vxix = hvz; Jφ−1 (z) MX (φ (z))Jφ−1 (z)vzi We are interested in Riemannian manifolds (do Carmo, = hvz; M(z)vzi = hvz; vziz; (1) 1992), which constitute well-defined metric spaces, | −1 d×d where the inner product is defined only locally and with M(z) = Jφ−1 (z) MX (φ (z))Jφ−1 (z) 2 R0 changes smoothly throughout space. In a nutshell, being smooth. As we discuss below, we should be able these are smooth spaces where we can compute short- to evaluate the intrinsic M(z) in order to find length est paths, which prefer regions where the magnitude of minimizing curves on M. However, when M is em- the inner product is small. In this work, we show how bedded the chart maps are usually unknown, as well to use these geometric structures in machine learning, as a global chart rarely exists. In contrast, for ambient where it is commonly assumed that data lie near a low like manifolds the global chart is φ(x) = x, which is dimensional manifold in an ambient observation space. convenient to use in practice. Definition 1. A Riemannian manifold is a smooth In general, one of the main utilities of a Riemannian manifold M, equipped with a positive definite Rieman- manifold M ⊆ X is to enable us compute short- nian metric M(x) 8 x 2 M, which changes smoothly p est paths therein. Intuitively, the norm hdx; dxix and defines a local inner product on the tangent space represents how the infinitesimal displacement vector TxM at each point x 2 M as hv; uix = hv; M(x)ui dx ≈ x0 − x on M is locally scaled. Thus, for a curve with v; u 2 TxM. γ : [0; 1] !M that connects two points x = γ(0) and y = γ(1), the length on M or equivalently in φ(M) A smooth manifold is a topological space, which lo- using that γ(t) = φ−1(c(t)) and Eq.1 is measured as cally is homeomorphic to a Euclidean space. An intu- itive way to think of a d-dimensional smooth manifold Z 1 q is as an embedded non-intersecting surface M in an length[γ(t)] = hγ_ (t); γ_ (t)iγ(t)dt (2) D 0 ambient space X for example R with D > d (see 1 Z p Fig.2 left). In this case, the tangent space TxM is = hc_(t); M(c(t))c _(t)idt = length[c(t)]; a d-dimensional vector space tangential to M at the 0 Georgios Arvanitidis, Søren Hauberg, Bernhard Sch¨olkopf whereγ _ (t) = @tγ(t) 2 Tγ(t)M is the velocity of the M(z) = Jg(z)|MX (g(z))Jg(z) is known as the pull- curve and accordinglyc _(t) 2 Tc(t)φ(M). The minimiz- back metric. Essentially, it captures the intrinsic ge- ers of this functional are the shortest paths, also known ometry of the immersed MZ , while taking into ac- as geodesics. We find them by solving a system of 2nd count the geometry of X . Therefore, the space Z order nonlinear ordinary differential equations (ODEs) together with M(z) constitutes a Riemannian mani- dZ defined in the intrinsic coordinates. Notably, for am- fold, but since Z = R the chart map and the TzZ bient like manifolds the trivial chart map enables us to are trivial.
Load more

Recommended publications
  • Nonlinear Elasticity in a Deforming Ambient Space Contents 1
    Nonlinear Elasticity in a Deforming Ambient Space∗ Arash Yavari† Arkadas Ozakin‡ Souhayl Sadik§ 27 May 2016 Abstract In this paper we formulate a nonlinear elasticity theory in which the ambient space is evolving. For a continuum moving in an evolving ambient space, we model time dependency of the metric by a time- dependent embedding of the ambient space in a larger manifold with a fixed background metric. We derive both the tangential and the normal governing equations. We then reduce the standard energy balance written in the larger ambient space to that in the evolving ambient space. We consider quasi-static deformations of the ambient space and show that a quasi-static deformation of the ambient space results in stresses, in general. We linearize the nonlinear theory about a reference motion and show that variation of the spatial metric corresponds to an effective field of body forces. Keywords: Geometric mechanics, nonlinear elasticity, deforming ambient space. Contents 1 Introduction 1 2 Motion of an Elastic Body in an Evolving Ambient Space3 2.1 Lagrangian Field Theory of Elasticity in an Evolving Ambient Space................3 2.2 Conservation of Mass for Motion in an Evolving Ambient Space................... 10 2.3 Energy Balance in Nonlinear Elasticity in an Evolving Ambient Space................ 11 3 Quasi-Static Deformations of the Ambient Space Metric 14 3.1 Examples of elastic bodies in evolving ambient spaces and the induced stresses.......... 14 3.2 Elastic deformations due to linear perturbations of the ambient space metric............ 20 A Geometry of Riemannian Submanifolds 24 B An Alternative Derivation of the Tangent Balance of Linear Momentum 27 1 Introduction In the geometric theory of elasticity, an elastic body is represented by a material manifold B , which defines the natural, stress-free state of the body.
    [Show full text]
  • Persistence Stability for Geometric Complexes Arxiv:1207.3885V3
    Persistence stability for geometric complexes Fr´ed´ericChazal∗, Vin de Silvay, Steve Oudotz November 11, 2013 Abstract In this paper we study the properties of the homology of different geometric filtered complexes (such as Vietoris{Rips, Cechˇ and witness complexes) built on top of totally bounded metric spaces. Using recent developments in the theory of topological persis- tence, we provide simple and natural proofs of the stability of the persistent homology of such complexes with respect to the Gromov–Hausdorff distance. We also exhibit a few noteworthy properties of the homology of the Rips and Cechˇ complexes built on top of compact spaces. 1 Introduction The inference of topological properties of metric spaces from approximations is a problem that has attracted special attention in computational topology in recent years. Given a metric space (Y; dY ) approximating an unknown metric space (X; dX ), the aim is to build a simplicial complex on the vertex set Y whose homology or homotopy type is the same as X. Note that, although Y is finite in many applications, finiteness is not a requirement a priori. Among the many geometric complexes available to us, the Vietoris{Rips complex (or simply `Rips complex') is particularly useful, being easy to compute and having good approximation arXiv:1207.3885v3 [math.AT] 15 Nov 2013 properties. We recall the definition. Let (X; dx) be a metric space and α a real parameter (the `scale'). Then Rips(X; α) is the simplical complex on X whose simplices are the finite subsets of X with diameter at most α: σ = [x0; x1; : : : ; xk] Rips(X; α) dX (xi; xj) α for all i; j 2 , ≤ ∗[email protected] [email protected] [email protected] 1 When (X; dX ) is a closed Riemannian manifold, J.-C.
    [Show full text]
  • On Orbit Configuration Spaces of Spheres
    CORE Metadata, citation and similar papers at core.ac.uk Provided by Elsevier - Publisher Connector Topology and its Applications 118 (2002) 85–102 On orbit configuration spaces of spheres Eva Maria Feichtner a,∗, Günter M. Ziegler b,∗ a Departement Mathematik, ETH Zürich, 8092 Zürich, Switzerland b Fachbereich Mathematik, MA 6-2, TU Berlin, 10623 Berlin, Germany Received 26 January 2000; received in revised form 10 July 2000 Abstract k The orbit configuration space FZ2 (S ,n) is the space of all ordered n-tuples of points on the k k-sphere such that no two of them are identical or antipodal. The cohomology algebra of FZ2 (S ,n), with integer coefficients, is here determined completely, and described in terms of generators, bases and relations. To this end, we analyze the cohomology spectral sequence of a fibration k → k FZ2 (S ,n) S , where the fiber—in contrast to the situation for the classical configuration space F(Sk,n)—is not the complement of a linear subspace arrangement. Analogies to the arrangement case, however, are crucial for getting a complete description of its cohomology. 2002 Elsevier Science B.V. All rights reserved. Keywords: Orbit configuration spaces; Integral cohomology algebra; Subspace arrangements; Leray–Serre spectral sequence 1. Introduction The ordered configuration spaces n F(M, n) := (x1,x2,...,xn) ∈ M : xi = xj for all i = j of a manifold M were first studied by Fadell and Neuwirth [15] in 1962. The cohomology of such spaces has received a lot of attention, at least since the work by Arnol’d [1] related to Hilbert’s 13th problem.
    [Show full text]
  • Geometric Meaning of Isoparametric Hypersurfaces in a Real Space Form
    Canad. Math. Bull. Vol. 43 (1), 2000 pp. 74–78 Geometric Meaning of Isoparametric Hypersurfaces in a Real Space Form Makoto Kimura and Sadahiro Maeda Abstract. We shall provide a characterization of all isoparametric hypersurfaces M’s in a real space form M˜ (c) by observing the extrinsic shape of geodesics of M in the ambient manifold M˜ (c). 0 Introduction In differential geometry it is interesting to know the shape of a Riemannian submanifold by observing the extrinsic shape of geodesics of the submanifold. For example: A hypersurface Mn isometrically immersed into a real space form Mn+1(c)ofconstantcurvaturec (that is, Mn+1(c) = Rn+1, Sn+1(c)orHn+1(c) according as the curvature c is zero, positive, or negative) is totally umbilic in Mn+1(c) if and only if every geodesic of M,consideredasa curve in the ambient space Mn+1(c), is a circle. Here we recall the definition of circles in Riemannian geometry. A smooth curve γ : R −→ M in a complete Riemannian manifold M is called a circle of curvature κ( 0) if it is parametrized by its arclength s and it satisfies the following equation: 2 ∇γ˙ ∇γ˙ γ˙(s) = −κ γ˙(s), where κ is constant and ∇γ˙ denotes the covariant differentiation along γ with respect to the Riemannian connection ∇ of M.Since∇γ˙ γ˙ = κ,thisequationisequivalenttothe equation of geodesics, when κ = 0. So we treat a geodesic as a circle of null curvature. In general, a circle in a Riemannian manifold is not closed. Of course, any circles of positive curvature in Euclidean m-space Rm are closed.
    [Show full text]
  • Arxiv:1611.04265V1 [Math.GT] 14 Nov 2016 Iiu,Seeape1 O H Qiaea Etgnllnaeten the Linkage Heptagonal Equilateral Ex the One for and 1
    ORIENTED AREA IS A PERFECT MORSE FUNCTION GAIANE PANINA Abstract. We show that an appropriate generalization of the oriented area function is a perfect Morse function on the space of three-dimensional configurations of an equilateral polygonal linkage with odd number of edges. Therefore cyclic equilateral polygons (which appear as Morse points) are interpreted as independent generators of the homology groups of the (dec- orated) configuration space. 1. Introduction A Morse function on a smooth manifold is called perfect if the number of critical points equals the sum of Betti numbers. Not every manifold possesses a perfect Morse function. Homological spheres (that are not spheres) do not possess it; manifolds with torsions in homologies do not possess it, etc. On the other hand, the celebrated Millnor-Smale cancelation technique for critical points with neighbor indices (or, equivalently, cancelation of handles) provides a series of existence-type theorems [1, 5] which are the key tool in Smale’s proof of generalized Poincare conjecture [8]. In the paper we focus on one particular example of a perfect Morse function and discuss some related problems. Namely, we bound ourselves by the space of configuration of the equilateral polygonal linkage with odd number n =2k +1 of edges. As the ambient space it makes sense to take either R2 or R3, which gives us the spaces M2(n) and M3(n). In bigger dimension the configuration space is not a manifold. The number n is chosen to be odd by the same arXiv:1611.04265v1 [math.GT] 14 Nov 2016 reason: for even n, the configuration space of the equilateral polygonal linkage has singular points.
    [Show full text]
  • Thesis Persistence Stability for Metric Thickenings
    THESIS PERSISTENCE STABILITY FOR METRIC THICKENINGS Submitted by Michael Moy Department of Mathematics In partial fulfillment of the requirements For the Degree of Master of Science Colorado State University Fort Collins, Colorado Spring 2021 Master’s Committee: Advisor: Henry Adams Emily King Asa Ben-Hur Copyright by Michael Moy 2021 All Rights Reserved ABSTRACT PERSISTENCE STABILITY FOR METRIC THICKENINGS Persistent homology often begins with a filtered simplicial complex, such as the Vietoris–Rips complex or the Cechˇ complex, the vertex set of which is a metric space. An important result, the stability of persistent homology, shows that for certain types of filtered simplicial complexes, two metric spaces that are close in the Gromov–Hausdorff distance result in persistence diagrams that are close in the bottleneck distance. The recent interest in persistent homology has motivated work to better understand the homotopy types and persistent homology of these commonly used simplicial complexes. This has led to the definition of metric thickenings, which agree with sim- plicial complexes for finite vertex sets but may have different topologies for infinite vertex sets. We prove Vietoris–Rips metric thickenings and Cechˇ metric thickenings have the same persistence diagrams as their corresponding simplicial complexes for all totally bounded metric spaces. This immediately implies the stability of persistent homology for these metric thickenings. ii ACKNOWLEDGEMENTS I would like to thank my advisor Henry Adams for his support and guidance. iii DEDICATION I dedicate this work to my family. iv TABLE OF CONTENTS ABSTRACT . ii ACKNOWLEDGEMENTS . iii DEDICATION . iv Chapter 1 Introduction . 1 Chapter 2 Background . 5 2.1 Simplicial Complexes and Filtrations .
    [Show full text]
  • 21 TOPOLOGICAL METHODS in DISCRETE GEOMETRY Rade T
    21 TOPOLOGICAL METHODS IN DISCRETE GEOMETRY Rade T. Zivaljevi´cˇ INTRODUCTION A problem is solved or some other goal achieved by “topological methods” if in our arguments we appeal to the “form,” the “shape,” the “global” rather than “local” structure of the object or configuration space associated with the phenomenon we are interested in. This configuration space is typically a manifold or a simplicial complex. The global properties of the configuration space are usually expressed in terms of its homology and homotopy groups, which capture the idea of the higher (dis)connectivity of a geometric object and to some extent provide “an analysis properly geometric or linear that expresses location directly as algebra expresses magnitude.”1 Thesis: Any global effect that depends on the object as a whole and that cannot be localized is of homological nature, and should be amenable to topological methods. HOW IS TOPOLOGY APPLIED IN DISCRETE GEOMETRIC PROBLEMS? In this chapter we put some emphasis on the role of equivariant topological methods in solving combinatorial or discrete geometric problems that have proven to be of relevance for computational geometry and topological combinatorics and with some impact on computational mathematics in general. The versatile configuration space/test map scheme (CS/TM) was developed in numerous research papers over the years and formally codified in [Ziv98].ˇ Its essential features are summarized as follows: CS/TM-1: The problem is rephrased in topological terms. The problem should give us a clue how to define a “natural” configuration space X and how to rephrase the question in terms of zeros or coincidences of the associated test maps.
    [Show full text]
  • On Embedding Polyhedra and Manifolds
    TRANSACTIONS OF THE AMERICAN MATHEMATICAL SOCIETY Volume 157, June 1971 ON EMBEDDING POLYHEDRA AND MANIFOLDS BY KRESO HORVATIC Abstract. It is well known that every «-polyhedron PL embeds in a Euclidean (2« + l)-space, and that for PL manifolds the result can be improved upon by one dimension. In the paper are given some sufficient conditions under which the dimen- sion of the ambient space can be decreased. The main theorem asserts that, for there to exist an embedding of the //-polyhedron A-into 2/z-space, it suffices that the integral cohomology group Hn(X— Int A) = 0 for some /¡-simplex A of a triangulation of X. A number of interesting corollaries follow from this theorem. Along the line of manifolds the known embedding results for PL manifolds are extended over a larger class containing various kinds of generalized manifolds, such as triangulated mani- folds, polyhedral homology manifolds, pseudomanifolds and manifolds with singular boundary. Finally, a notion of strong embeddability is introduced which allows us to prove that some class of //-manifolds can be embedded into a (2«—l)-dimensional ambient space. 1. Introduction. It was known early [14] that every «-dimensional polyhedron can be piecewise linearly embedded into a Euclidean space of dimension 2«+l, and that for piecewise linear manifolds this result can be improved upon by one dimension. On the other hand, there are counterexamples showing that these results are, in general, the best possible ([5], [6], [19]). So the natural problem arises as to characterizing those polyhedra and manifolds for which the dimension of the ambient space can be decreased.
    [Show full text]
  • Topics in Hyperplane Arrangements, by Marcelo Aguiar and Swapneel Mahajan, Mathematical Surveys and Monographs, Vol
    BULLETIN (New Series) OF THE AMERICAN MATHEMATICAL SOCIETY Volume 56, Number 2, April 2019, Pages 367–372 https://doi.org/10.1090/bull/1660 Article electronically published on December 12, 2018 Topics in hyperplane arrangements, by Marcelo Aguiar and Swapneel Mahajan, Mathematical Surveys and Monographs, Vol. 226, American Mathematical So- ciety, Providence, RI, 2017, xxiv+611 pp., US$159.00, ISBN 978-1-4704-3711-4 1. Introduction Hyperplane arrangements provide a classic example of the deep interactions be- tween space and complexity. Like knots or convex polytopes, they are combinatorial objects that arise from geometry. Also they are quite enjoyable. In just a few mo- ments I can describe to a middle-schooler some of the open questions, and challenge her to try, with pencil and paper, to brute-force an answer to easy cases. A little later, I’d recommend the recent monograph from Aguiar and Mahajan. A hyperplane arrangement is exactly what it sounds like. Take a collection of distinct codimension-1 linear subspaces, or affine subspaces, of a real, complex, or projective ambient vector space, and arrange them in that ambient space in any way you like. In any of these six options, the hyperplanes will subdivide both the ambient space and, potentially, each other. We often label all the intersections and subdivisions; these are called the faces of the arrangement. Top-dimensional faces are the chambers, the regions of the subdivided ambient space. If there is a face C that is the intersection of all the hyperplanes, we call the arrangement central ;if that center face is a point, then the central arrangement is also essential.
    [Show full text]
  • Spacelike Hypersurfaces of Constant Mean Curvature in Spacetimes with Symmetries
    Spacelike Hypersurfaces of Constant Mean Curvature in Spacetimes with Symmetries Luis J. Al¶³as1, Alfonso Romero2 and Miguel Sanchez¶ 2 1 Departamento de Matem¶aticas, Universidad de Murcia, 30100-Espinardo, Murcia, Spain 2 Departamento de Geometr¶³ay Topolog¶³a,Universidad de Granada, 18071-Granada, Spain. Abstract The role of some symmetries in General Relativity is discussed, espe- cially those involving a timelike conformal vector ¯eld. The importance of spacelike hypersurfaces of constant mean curvature in the study of the Einstein equation is explained. Uniqueness results of compact spacelike hypersurfaces of constant mean curvature in spacetimes which admits a timelike conformal vector ¯eld are given by means of several integral formulas. As an application, several uniqueness consequences for the spacelike constant mean curvature di®erential equation are obtained. 1 Introduction Our purpose is to explain some uniqueness results of spacelike constant mean curvature hypersurfaces in certain ambient spacetimes; these hypersurfaces become relevant for the study of the Einstein equation. Our ambient space- times are not so restrictive as, for example, Lorentzian space forms, which have been widely studied from a purely mathematical point of view. Nevertheless, many of them contain some general symmetries such that the hypersurfaces can be seen as solutions to a partial di®erential equation. In the remainder of this section we explain the role of symmetries in General Relativity and de¯ne our ambient spacetimes; in Section 2 we see how spacelike hypersurfaces of constant mean curvature are useful to simplify Einstein equation; in Section 3 the associated di®erential equation and variational problem is introduced; in Section 4 we write some integral formulas which are our basic tool; in Section 5 the main results, essentially contained in [2], [3], [4], are explained.
    [Show full text]
  • On the Intrinsic Dimensionality of Image Representations
    On the Intrinsic Dimensionality of Image Representations Sixue Gong Vishnu Naresh Boddeti Anil K. Jain Michigan State University, East Lansing MI 48824 fgongsixu, vishnu, [email protected] Abstract erations, such as, ease of learning the embedding function [38], constraints on system memory, etc. instead of the ef- This paper addresses the following questions pertaining fective dimensionality necessary for image representation. to the intrinsic dimensionality of any given image represen- This naturally raises the following fundamental but related tation: (i) estimate its intrinsic dimensionality, (ii) develop questions, How compact can the representation be without a deep neural network based non-linear mapping, dubbed any loss in recognition performance? In other words, what DeepMDS, that transforms the ambient representation to is the intrinsic dimensionality of the representation? And, the minimal intrinsic space, and (iii) validate the verac- how can one obtain such a compact representation? Ad- ity of the mapping through image matching in the intrinsic dressing these questions is the primary goal of this paper. space. Experiments on benchmark image datasets (LFW, The intrinsic dimensionality (ID) of a representation IJB-C and ImageNet-100) reveal that the intrinsic dimen- refers to the minimum number of parameters (or degrees of sionality of deep neural network representations is signif- freedom) necessary to capture the entire information present icantly lower than the dimensionality of the ambient fea- in the representation [4]. Equivalently, it refers to the di- tures. For instance, SphereFace’s [26] 512-dim face repre- mensionality of the m-dimensional manifold embedded sentation and ResNet’s [16] 512-dim image representation within the d-dimensional ambient (representation)M space have an intrinsic dimensionality of 16 and 19 respectively.
    [Show full text]
  • Geodesic Chord Property and Hypersurfaces of Space Forms
    S S symmetry Article Geodesic Chord Property and Hypersurfaces of Space Forms Dong-Soo Kim 1, Young Ho Kim 2,* and Dae Won Yoon 3 1 Department of Mathematics, Chonnam National University, Gwangju 61186, Korea 2 Department of Mathematics, Kyungpook National University, Daegu 41566, Korea 3 Department of Mathematics Education and RINS, Gyeongsang National University, Jinju 52828, Korea * Correspondence: [email protected]; Tel.: +82-53-950-7325 Received: 22 July 2019; Accepted: 14 August 2019; Published: 16 August 2019 Abstract: In the Euclidean space En, hyperplanes, hyperspheres and hypercylinders are the only isoparametric hypersurfaces. These hypersurfaces are also the only ones with chord property, that is, the chord connecting two points on them meets the hypersurfaces at the same angle at the two points. In this paper, we investigate hypersurfaces in nonflat space forms with the so-called geodesic chord property and classify such hypersurfaces completely. Keywords: geodesic chord property; hypersphere; hyperbolic space; isoparametric hypersurface 1. Introduction A circle in the plane E2 is characterized as a closed curve with the chord property that the chord connecting any two points on it meets the curve at the same angle at the two end points ([1], pp. 160–162). For space curves, B.-Y. Chen et al. showed that a W-curve is characterized as a curve in the Euclidean space En with the property that the chord joining any two points on the curve meets the curve at the same angle at the two points, that is, as a curve in the Euclidean space En with the chord property ([2]).
    [Show full text]