An L∞ Eigenvector Perturbation Bound and Its Application to Robust
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
IMS Grace Wahba Award and Lecture
Volume 50 • Issue 4 IMS Bulletin June/July 2021 IMS Grace Wahba Award and Lecture The IMS is pleased to announce the creation of a new award and lecture, to honor CONTENTS Grace Wahba’s monumental contributions to statistics and science. These include her 1 New IMS Grace Wahba Award pioneering and influential work in mathematical statistics, machine learning and opti- and Lecture mization; broad and career-long interdisciplinary collaborations that have had a sig- 2 Members’ news : Kenneth nificant impact in epidemiology, bioinformatics and climate sciences; and outstanding Lange, Kerrie Mengersen, Nan mentoring during her 51-year career as an educator. The inaugural Wahba award and Laird, Daniel Remenik, Richard lecture is planned for the 2022 IMS annual meeting in London, then annually at JSM. Samworth Grace Wahba is one of the outstanding statisticians of our age. She has transformed 4 Grace Wahba: profile the fields of mathematical statistics and applied statistics. Wahba’s RKHS theory plays a central role in nonparametric smoothing and splines, and its importance is widely 7 Caucus for Women in recognized. In addition, Wahba’s contributions Statistics 50th anniversary straddle the boundary between statistics and 8 IMS Travel Award winners optimization, and have led to fundamental break- 9 Radu’s Rides: Notes to my Past throughs in machine learning for solving problems Self arising in prediction, classification and cross-vali- dation. She has paved a foundation for connecting 10 Previews: Nancy Zhang, Ilmun Kim theory and practice of function estimation, and has developed, along with her students, unified 11 Nominate IMS Lecturers estimation methods, scalable algorithms and 12 IMS Fellows 2021 standard software toolboxes that have made regu- 16 Recent papers: AIHP and larization approaches widely applicable to solving Observational Studies complex problems in modern science discovery Grace Wahba and technology innovation. -
Coauthorship and Citation Networks for Statisticians
Submitted to the Annals of Applied Statistics arXiv: arXiv:0000.0000 COAUTHORSHIP AND CITATION NETWORKS FOR STATISTICIANS By Pengsheng Jiy and Jiashun Jinz University of Georgiay and Carnegie Mellon Universityz We have collected and cleaned two network data sets: Coauthor- ship and Citation networks for statisticians. The data sets are based on all research papers published in four of the top journals in statis- tics from 2003 to the first half of 2012. We analyze the data sets from many different perspectives, focusing on (a) centrality, (b) commu- nity structures, and (c) productivity, patterns and trends. For (a), we have identified the most prolific/collaborative/highly cited authors. We have also identified a handful of \hot" papers, suggesting \Variable Selection" as one of the \hot" areas. For (b), we have identified about 15 meaningful communities or research groups, including large-size ones such as \Spatial Statis- tics", \Large-Scale Multiple Testing", \Variable Selection" as well as small-size ones such as \Dimensional Reduction", \Objective Bayes", \Quantile Regression", and \Theoretical Machine Learning". For (c), we find that over the 10-year period, both the average number of papers per author and the fraction of self citations have been decreasing, but the proportion of distant citations has been increasing. These suggest that the statistics community has become increasingly more collaborative, competitive, and globalized. Our findings shed light on research habits, trends, and topological patterns of statisticians. The data sets provide a fertile ground for future researches on or related to social networks of statisticians. 1. Introduction. It is frequently of interest to identify \hot" areas and key authors in a scientific community, and to understand the research habits, trends, and topological patterns of the researchers. -
Strength in Numbers: the Rising of Academic Statistics Departments In
Agresti · Meng Agresti Eds. Alan Agresti · Xiao-Li Meng Editors Strength in Numbers: The Rising of Academic Statistics DepartmentsStatistics in the U.S. Rising of Academic The in Numbers: Strength Statistics Departments in the U.S. Strength in Numbers: The Rising of Academic Statistics Departments in the U.S. Alan Agresti • Xiao-Li Meng Editors Strength in Numbers: The Rising of Academic Statistics Departments in the U.S. 123 Editors Alan Agresti Xiao-Li Meng Department of Statistics Department of Statistics University of Florida Harvard University Gainesville, FL Cambridge, MA USA USA ISBN 978-1-4614-3648-5 ISBN 978-1-4614-3649-2 (eBook) DOI 10.1007/978-1-4614-3649-2 Springer New York Heidelberg Dordrecht London Library of Congress Control Number: 2012942702 Ó Springer Science+Business Media New York 2013 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’s location, in its current version, and permission for use must always be obtained from Springer. -
A Direct Approach to Robustness Optimization
A Direct Approach to Robustness Optimization Thesis by Seungil You In Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy California Institute of Technology Pasadena, California 2016 (Defended August 12, 2015) ii c 2016 Seungil You All Rights Reserved iii To My Family iv Acknowledgments First and foremost, my gratitude goes to my advisor, John. He encouraged me to pursue my own ideas, and always gave me great feedback with a sense of humor. His unique research style made my academic journey at Caltech very special, and changed my way of looking at problems. I was simply lucky to have him as my advisor. I am also thankful to my thesis committee members: Richard Murray, Babak Hassibi, and Venkat Chandrasekaran. Richard has guided me at every milestone of my PhD studies, despite of his impossible schedule. Babak gladly agreed to be my committee member. I was in Venkat’s first class at Caltech on advanced topics in convex optimization. Of course, I was fascinated by the results he taught, and deeply influenced by his genuine point of view on solving mathematical problems. I learned convex optimization from Joel Tropp, and was immediately thrilled by the subject. His endless effort to establish a minimal, elegant theoretical framework for many challenging math problems had a profound impact on me. Lijun Chen taught me how to write a scientific paper, as well as how to formulate the problem in a concrete manner. Under his guidance, I was able to become an independent researcher. I met Ather Gattami in Florence, Italy, during CDC. -
Robust High Dimensional Factor Models with Applications to Statistical Machine Learning∗
Robust high dimensional factor models with applications to statistical machine learning∗ Jianqing Fanyz, Kaizheng Wangy, Yiqiao Zhongy, Ziwei Zhuy August 14, 2018 Abstract Factor models are a class of powerful statistical models that have been widely used to deal with dependent measurements that arise frequently from various applications from genomics and neuroscience to economics and finance. As data are collected at an ever-growing scale, statistical machine learning faces some new challenges: high dimensionality, strong dependence among observed variables, heavy-tailed variables and heterogeneity. High-dimensional robust factor analysis serves as a powerful toolkit to conquer these challenges. This paper gives a selective overview on recent advance on high-dimensional factor models and their applications to statistics including Factor-Adjusted Robust Model selection (FarmSe- lect) and Factor-Adjusted Robust Multiple testing (FarmTest). We show that classical methods, especially principal component analysis (PCA), can be tailored to many new problems and pro- vide powerful tools for statistical estimation and inference. We highlight PCA and its connec- tions to matrix perturbation theory, robust statistics, random projection, false discovery rate, etc., and illustrate through several applications how insights from these fields yield solutions to modern challenges. We also present far-reaching connections between factor models and popular statistical learning problems, including network analysis and low-rank matrix recovery. Key Words: Factor model, PCA, covariance estimation, perturbation bounds, robustness, random sketch, FarmSelect, FarmTest 1 Introduction In modern data analytics, dependence across high-dimensional outcomes or measurements is ubiq- uitous. For example, stocks within the same industry exhibit significantly correlated returns, hous- ing prices of a country depend on various economic factors, gene expressions can be stimulated arXiv:1808.03889v1 [stat.ME] 12 Aug 2018 by cytokines. -
Biographical Sketch for Jianqing Fan
Biographical Sketch for Jianqing Fan Jianqing Fan is Frederick L. Moore'18 Professor of Finance and Director of Commit- tee of Statistical Studies at Princeton University, and the past president of the Institute of Mathematical Statistics (2006-2009), and president of International Chinese Statistical Association. He is the Co-editor of Econometrical Journal published by Royal Economics Society and an associate editor of The Journal of American Statistical Association, and was the co-editor(-in-chief) of The Annals of Statistics (2004-2006) and an editor of Prob- ability Theory and Related Fields (2003-2005) and on the editorial boards of a number of other journals. After receiving his Ph.D. in Statistics from the University of California at Berkeley, he has been appointed as assistant, associate, and full professor at the Univer- sity of North Carolina at Chapel Hill (1989-2003), and as professor at the University of California at Los Angeles (1997-2000), Professor of Statistics and Chairman at the Chinese University of Hong Kong (2000-2003), and as professor at the Princeton University(2003{). He has coauthored two highly-regarded books on \Local Polynomial Modeling" (1996) and \Nonlinear time series: Parametric and Nonparametric Methods" (2003) and authored or coauthored over 150 articles on computational biology, ¯nancial econometrics, semipara- metric and non-parametric modeling, statistical learning, nonlinear time series, survival analysis, longitudinal data analysis, and other aspects of theoretical and methodological statistics. -
Norms on Complex Matrices Induced by Complete
NORMS ON COMPLEX MATRICES INDUCED BY COMPLETE HOMOGENEOUS SYMMETRIC POLYNOMIALS KONRAD AGUILAR, ÁNGEL CHÁVEZ, STEPHAN RAMON GARCIA, AND JURIJ VOLCIˇ Cˇ Abstract. We introduce a remarkable new family of norms on the space of n × n complex matrices. These norms arise from the combinatorial properties of sym- metric functions, and their construction and validation involve probability theory, partition combinatorics, and trace polynomials in noncommuting variables. Our norms enjoy many desirable analytic and algebraic properties, such as an ele- gant determinantal interpretation and the ability to distinguish certain graphs that other matrix norms cannot. Furthermore, they give rise to new dimension- independent tracial inequalities. Their potential merits further investigation. 1. Introduction In this note we introduce a family of norms on complex matrices. These are ini- tially defined in terms of certain symmetric functions of eigenvalues of complex Hermitian matrices. The fact that we deal with eigenvalues, as opposed to their absolute values, is notable. First, it prevents standard machinery, such as the the- ory of symmetric gauge functions, from applying. Second, the techniques used to establish that we indeed have norms are more complicated than one might ex- pect. For example, combinatorics, probability theory, and Lewis’ framework for group invariance in convex matrix analysis each play key roles. These norms on the Hermitian matrices are of independent interest. They can be computed recursively or directly read from the characteristic polyno- mial. Moreover, our norms distinguish certain pairs of graphs which the standard norms (operator, Frobenius, Schatten-von Neumann, Ky Fan) cannot distinguish. Our norms extend in a natural and nontrivial manner to all complex matrices. -
Embracing the Blessing of Dimensionality in Factor Models
Embracing the Blessing of Dimensionality in Factor Models Quefeng Li, Guang Cheng, Jianqing Fan and Yuyan Wang ∗ October 18, 2016 Abstract Factor modeling is an essential tool for exploring intrinsic dependence structures among high-dimensional random variables. Much progress has been made for esti- mating the covariance matrix from a high-dimensional factor model. However, the blessing of dimensionality has not yet been fully embraced in the literature: much of the available data is often ignored in constructing covariance matrix estimates. If our goal is to accurately estimate a covariance matrix of a set of targeted variables, shall we employ additional data, which are beyond the variables of interest, in the estimation? In this paper, we provide sufficient conditions for an affirmative answer, and further quantify its gain in terms of Fisher information and convergence rate. In fact, even an oracle-like result (as if all the factors were known) can be achieved when a sufficiently large number of variables is used. The idea of utilizing data as much as possible brings computational challenges. A divide-and-conquer algorithm is thus proposed to alleviate the computational burden, and also shown not to sacrifice any statistical accuracy in comparison with a pooled analysis. Simulation studies further confirm our advocacy for the use of full data, and demonstrate the effectiveness of the above algorithm. Our proposal is applied to a microarray data example that shows empirical benefits of using more data. Keywords: Asymptotic normality, auxiliary data, divide-and-conquer, factor model, Fisher information, high-dimensionality. ∗Quefeng Li is Assistant Professor, Department of Biostatistics, University of North Carolina at Chapel Hill, NC 27599 (Email:[email protected]). -
Front Matter
Cambridge University Press 0521865654 - Combinatorial Matrix Classes Richard A. Brualdi Frontmatter More information ENCYCLOPEDIA OF MATHEMATICS AND ITS APPLICATIONS FOUNDED BY G.-C. ROTA Editorial Board P. Flajolet, M. Ismail, E. Lutwak Volume 108 Combinatorial Matrix Classes © Cambridge University Press www.cambridge.org Cambridge University Press 0521865654 - Combinatorial Matrix Classes Richard A. Brualdi Frontmatter More information ENCYCLOPEDIA OF MATHEMATICS AND ITS APPLICATIONS FOUNDED EDITOR G.-C. ROTA Editorial Board P. Flajolet, M. Ismail, E. Lutwak 40 N. White (ed.) Matroid Applications 41 S. Sakai Operator Algebras in Dynamical Systems 42 W. Hodges Basic Model Theory 43 H. Stahl and V. Totik General Orthogonal Polynomials 45 G. Da Prato and J. Zabczyk Stochastic Equations in Infinite Dimensions 46 A. Bj¨orner et al. Oriented Matroids 47 G. Edgar and L. Sucheston Stopping Times and Directed Processes 48 C. Sims Computation with Finitely Presented Groups 49 T. Palmer Banach Algebras and the General Theory of *-Algebras I 50 F. Borceux Handbook of Categorical Algebra I 51 F. Borceux Handbook of Categorical Algebra II 52 F. Borceux Handbook of Categorical Algebra III 53 V. F. Kolchin Random Graphs 54 A. Katok and B. Hasselblatt Introduction to the Modern Theory of Dynamical Systems 55 V. N. Sachkov Combinatorial Methods in Discrete Mathematics 56 V. N. Sachkov Probabilistic Methods in Discrete Mathematics 57 P. M. Cohn Skew Fields 58 R. Gardner Geometric Tomography 59 G. A. Baker, Jr., and P. Graves-Morris Pad´e Approximants, 2nd edn 60 J. Krajicek Bounded Arithmetic, Propositional Logic, and Complexity Theory 61 H. Groemer Geometric Applications of Fourier Series and Spherical Harmonics 62 H. -
Gradients for the Loss!
Gradients for the Loss! Towards Provable Gradient Algorithms for Non-Convex Problems Fotis Iliopoulos, Vrettos Moulous, Vaishaal Shankar, Max Simchowitz EE227 BT Department of EECSS Contents 1 Convergence of Stochastic Gradient for PCA3 1.1 SGD for Rank 1 Stochastic PCA........................3 1.1.1 A brief overview of Shamir's result...................4 1.1.2 Rank 1 Stochastic PCA Experiments..................7 1.2 The Alecton Algorithm for Higher Rank PCA.................7 1.2.1 Algorithm Intuition...........................8 1.2.2 Precise Setup...............................9 1.3 Sketching the Proof of Theorem 1.2.1...................... 10 1.3.1 Martingales................................ 10 1.3.2 Preliminaries............................... 10 1.3.3 Main Arguments and Completing the proof.............. 11 1.3.4 Experiments for Alecton......................... 14 1.3.5 Toy Convexity in Two Dimensions................... 14 1.3.6 Convexity Results in Higher Dimensions................ 15 1.3.7 Experimental Results.......................... 17 2 Dropping Convexity for Faster Semi-definite Optimization 19 2.1 Introduction: Back to Rank r-PCA....................... 19 2.2 From PCA to General Rank-Constrained Optimization............ 20 2.2.1 Notation and Assumptions....................... 21 2.2.2 Results.................................. 23 2.2.3 A Note on Step Size Selection...................... 24 2.3 Proof Sketch of Results............................. 24 2.4 Initialization................................... 27 2.4.1 Test case: Matrix sensing problem................... 27 3 Noisy Gradients in the Presence of Saddle Points 29 3.1 The Strict Saddle Property........................... 30 3.1.1 Sketch of the Proof............................ 31 3.1.2 Proof Sketch of Lemma 3.1.4...................... 32 3.2 Gradient Descent with Manifold Constraints.................. 34 3.2.1 The Tangent and Normal Space.................... -
Proscribed Normal Decompositions of Euclidean Jordan Algebras
Proscribed normal decompositions of Euclidean Jordan algebras Michael Orlitzky June 30, 2020 Abstract Normal decomposition systems unify many results from convex ma- trix analysis regarding functions that are invariant with respect to a group of transformations—particularly those matrix functions that are unitarily-invariant and the affiliated permutation-invariant “spectral func- tions” that depend only on eigenvalues. Spectral functions extend in a nat- ural way to Euclidean Jordan algebras, and several authors have studied the problem of making a Euclidean Jordan algebra into a normal decom- position system. In particular it is known to be possible with respect to the “eigenvalues of” map when the algebra is essentially-simple. We show the converse, that essential-simplicity is essential to that process. 1 Notation All spaces we encounter will be finite-dimensional real inner-product spaces with the generic hx, yi denoting the inner product of x and y. If V is such a space, then B (V ) is the set of all linear operators on V and Aut (X) is the linear automorphism group of X ⊆ V . The inner product on V induces a norm, and Isom (V ) is the group of linear isometries on V under composition. Explicitly, −1 Aut (X) := L ∈ B (V ) L exists and L (X) = X , Isom (V ) := {L ∈ Aut (V ) | kL (x)k = kxk for all x in V } . The group Isom (V ) is endowed with the natural topology [16] so that Ln ap- proaches L in Isom (V ) if and only if Ln (x) approaches L (x) for all x ∈ V . The adjoint of any L ∈ B (V ) is denoted by L∗, and thus L∗ = L−1 when L ∈ Isom (V ). -
A Complete Bibliography of the Journal of Business & Economic
A Complete Bibliography of the Journal of Business & Economic Statistics Nelson H. F. Beebe University of Utah Department of Mathematics, 110 LCB 155 S 1400 E RM 233 Salt Lake City, UT 84112-0090 USA Tel: +1 801 581 5254 FAX: +1 801 581 4148 E-mail: [email protected], [email protected], [email protected] (Internet) WWW URL: http://www.math.utah.edu/~beebe/ 20 May 2021 Version 1.01 Title word cross-reference (0; 1; 1)12 [Bel87]. 1=2 [GI11]. 4 [Fre85]. α [McC97]. F(d) [CdM19]. L [BC11]. M [TZ19]. N [AW07, CSZZ17, GK14]. = 1 [Bel87]. P [Han97, Ray90]. R [CW96]. R2 [MRW19]. T [AW07, GK14, RSW18, FSC03, IM10, JS10, MV91, NS96, TM19]. U [TZ19]. -Estimators [TZ19]. -Processes [TZ19]. -R [Fre85]. -Squared [CW96]. -Statistic [IM10, TM19]. -Statistics [BC11]. 100 [ACW97]. 11 [McK84, dBM90]. 225 [ACW97]. 2SLS [Lee18]. 500 [ACW97]. 1 2 a.m [HRV20]. Ability [CM09, Han05, Son12]. Abnormal [MlM87]. Absorbing [KK15]. Abuse [Die15a]. Accelerated [KLS15]. Access [AKO12]. Accessibility [FM97]. Accident [GFdH87]. Accommodating [KDM20]. Accountability [BDJ12]. Accounting [HPSW88, HW84, MPTK91]. Accounts [Che12]. Accumulated [DM01]. Accumulation [Cag03, DM01]. Accuracies [Fom86]. Accuracy [DM95a, DM02, Die15a, McN86a, NP90, PG93, Wec89]. Accurate [McK18]. Acidic [PR87]. Acknowledgment [Ano93a]. Acreages [HF87]. Across [PG93, PP11]. Act [Fre85]. Action [GW85]. Activities [AC03, But85]. Activity [FS93, JLR11, Li11, RS94b]. Actually [CO19]. Adaptive [CZ14, CHL18, CL17, HHK11, HF19, Tsa93b, ZS15]. Addiction [BG01]. Additional [LZ95]. Additive [Abe99, FZ18, FH94, Lia12, OHS14]. Addressing [Mel18]. Adequacy [HM89]. Adjust [Nev03]. Adjusted [AW84, LD19, MS03, NTP94, Rad83, Rub86, Son10]. Adjusting [vdHH86]. Adjustment [BH84a, BH02, BW84, CW86b, DM84, DL87, EG98, ES01a, FMB+98a, Ghy90, GGS96a, Ghy97, HR91, HKR97, Hil85, Jai89, Jor99, Lin86, LHM20, McE17, ML90, McK84, Pat86, Pfe91a, Pie83, Pie84, PGC84, Sim85, Tay02, dBM90].