On the Rate of Convergence of Empirical Barycentres in Metric
Total Page:16
File Type:pdf, Size:1020Kb
On the rate of convergence of empirical barycentres in metric spaces: curvature, convexity and extendible geodesics Adil Ahidar-Coutrix, Thibaut Le Gouic, Quentin Paris To cite this version: Adil Ahidar-Coutrix, Thibaut Le Gouic, Quentin Paris. On the rate of convergence of empirical barycentres in metric spaces: curvature, convexity and extendible geodesics. 2019. hal-01810530v3 HAL Id: hal-01810530 https://hal.archives-ouvertes.fr/hal-01810530v3 Preprint submitted on 17 Jun 2019 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Convergence rates for empirical barycenters in metric spaces: curvature, convexity and extendable geodesics A. Ahidar-Coutrix,∗ T. Le Gouic† and Q. Paris‡ June 3, 2019 Abstract This paper provides rates of convergence for empirical (generalised) barycenters on compact geodesic metric spaces under general conditions using empirical processes techniques. Our main assumption is termed a variance inequality and provides a strong connection between usual assumptions in the field of empirical processes and central concepts of metric geometry. We study the validity of variance inequalities in spaces of non-positive and non-negative Aleksandrov curvature. In this last scenario, we show that variance inequalities hold provided geodesics, emanating from a barycenter, can be extended by a constant factor. We also relate variance inequalities to strong geodesic convexity. While not restricted to this setting, our results are largely discussed in the context of the 2-Wasserstein space. arXiv:1806.02740v3 [math.ST] 31 May 2019 ∗Aix Marseille Univ., CNRS, Centrale Marseille, I2M, Marseille, France. Email:[email protected] †Aix Marseille Univ., CNRS, Centrale Marseille, I2M, Marseille, France & National Research University Higher School of Economics, Moscow, Russia. This work has been funded by the Russian Academic Excellence Project ’5-100’. Email:[email protected] ‡National Research University Higher School of Economics, Moscow, Russia. This work has been funded by the Russian Academic Excellence Project ’5-100’. Email:[email protected] 1 1 Introduction Given a separable and complete metric space (M, d), define (M) as the set of Borel probability measures P2 P on M such that d(x,y)2 dP (y) < + , ∞ ZM for all x M. A barycenter of P (M), also called a Fréchet mean [Fré48], is any element x∗ M ∈ ∈ P2 ∈ such that x∗ arg min d(x,y)2 dP (y). (1.1) ∈ x∈M ZM When it exists, a barycenter stands as a natural analog of the mean of a (square integrable) probability measure on Rd. Alternative notions of mean value include local minimisers [Kar14], p-means [Yok17], exponential barycenters [ÉM91] or convex means [ÉM91]. Extending the notion of mean value to the case of probability measures on spaces M with no Euclidean (or Hilbert) structure has a number of applications ranging from geometry [Stu03] and optimal transport [Vil03, Vil08, San15, CP19] to statistics and data science [Pel05, BLL15, BGKL18, KSS19], and the context of abstract metric spaces provides a unifying framework encompassing many non-standard settings. Properties of barycenters, such as existence and uniqueness, happen to be closely related to geometric characteristics of the space M. These properties are addressed in the context of Riemannian manifolds in [Afs11]. Many interesting examples of metric spaces, however, cannot be described as smooth mani- folds because of their singularities or infinite dimensional nature. More general geometrical structures are geodesic metric spaces which include many more examples of interest (precise definitions and necessary background on metric geometry are reported in Appendix A). The barycenter problem has been addressed in this general setting. The scenario where M has non-positive curvature (from here on, curvature bounds are understood in the sense of Aleksandrov) is considered in [Stu03]. More generally, the case of metric spaces with upper bounded curvature is studied in [Yok16] and [Yok17]. The context of spaces M with lower bounded curvature is discussed in [Yok12] and [Oht12]. Focus on the case of metric spaces with non-negative curvature may be motivated by the increasing interest for the theory of optimal transport and its applications. Indeed, a space of central importance in this context is the Wasserstein space M = (Rd), equipped with the Wasserstein metric W , known P2 2 to be geodesic and with non-negative curvature (see Section 7.3 in [AGS08]). In this framework, the barycenter problem was first studied by [AC11] and has since gained considerable momentum. Existence and uniqueness of barycenters in (Rd) has further been studied in [LL17]. P2 A number of objects of interest, including barycenters as a special case, may be described as minimisers of the form x∗ arg min F (x,y) dP (y), (1.2) ∈ x∈M ZM for some probability measure P on metric space M and some functional F : M M R. While we × → obviously recover the definition of barycenters whenever F (x,y) = d(x,y)2, many functionals of interest are not of this specific form. With a slight abuse of language, minimisers such as x∗ will be called generalised barycenters in the sequel. A first example we have in mind, in the context where M = (Rd), P2 is the case where functional F is an f-divergence, i.e. f dµ dν if µ ν, F (µ,ν) := dν ≪ ( + otherwise, R ∞ for some convex function f : R R. Known for their importance in statistics [Le 86, Tsy09], and + → information theory [Vaj89], f-divergences have become a crucial tool in a number of other fields such as 2 geometry and optimal transport [Stu06a, Stu06b, LV09] or machine learning [GPAM+14]. Other exam- ples arise when the squared distance d(x,y)2 in (1.1) is replaced by a regularised version F (x,y) aiming at enforcing computationally friendly properties, such as convexity, while providing at the same time a sound approximation of d(x,y)2. A significant example in this spirit is the case where functional F is the entropy-regularised Wasserstein distance (also known as the Sinkhorn divergence) largely used as a proxy for W2 in applications [Cut13, CP19, AWR17, DGK18]. In the paper, our main concern is to provide rates of convergence for empirical generalised barycenters, defined as follows. Given a collection Y1,...,Yn of independent and M-valued random variables with same distribution P , we call empirical generalised barycenter any n 1 xn arg min F (x,Yi). (1.3) ∈ x∈M n Xi=1 ∗ Any such xn provides a natural empirical counterpart of a generalised barycenter x defined in (1.2). The 2 statistical properties of xn have been studied in a few specific scenarios. In the case where F (x,y)= d(x,y) and M is a Riemannian manifold, significant contributions, establishing in particular consistency and limit distribution under general conditions, are [BP03, BP05] and [KL11]. Asymptotic properties of empirical barycenters in the Wasserstein space are studied in [LL17]. We are only aware of a few contributions providing finite sample bounds on the statistical performance of xn. Paper [BGKL18] provides upper and lower bounds on convergence rates for empirical barycenters in the context of the Wasserstein space over the real line. Independently of the present contribution, [Sch18] studies a similar problem and provides results complementary to ours. In addition to more transparent conditions, our results are based on the fundamental assumption that there exists constants K > 0 and β (0, 1] such that, for all x M, ∈ ∈ β d(x,x∗)2 K (F (x,y) F (x∗,y)) dP (y) . (1.4) ≤ − ZM We show that condition (1.4) provides a connection between usual assumptions in the field of empirical processes and geometric characteristics of the metric space M. First, the reader familiar with the theory of empirical processes will identify in the proof of Theorems 2.1 and 2.5 that condition (1.4) implies a Bernstein condition on the class of functions indexing our empirical process, that is an equivalence be- tween their L2 and L1 norms. Many authors have emphasised the role of this condition for obtaining fast rates of convergence of empirical minimisers. Major contributions in that direction are for instance [MT99, Mas00, BLV03, BBM05, BJM06, Kol06, BM06] and [Men15]. In particular, this assumption may be understood in our context as an analog of the Mammen-Tsybakov low-noise assumption [MT99] used in binary classification. Second, we show that condition (1.4) carries a strong geometrical meaning. In the context where F (x,y) = d(x,y)2, [Stu03] established a tight connection between (1.4), with K = β = 1, and the fact that M has non-positive curvature. When F (x,y) = d(x,y)2, we show that (1.4) actually holds with K > 0 and β = 1 in geodesic spaces of non-negative curvature under flexible conditions related to the possibility of extending geodesics emanating from a barycenter. Finally, for a general functional F , we connect (1.4) to its strong convexity properties. Using terminology introduced in [Stu03] in a slightly more specific context, we will call by extention (1.4) a variance inequality. The paper is organised has follows. Section 2 provides convergence rates for generalised empirical barycenters under several assumptions of functional F and two possible complexity assumptions on metric space M. Section 3 investigates in details the validity of the variance inequality (1.4) in different scenarios.