Perturbations of Operators with Application to Testing Equality of Covariance Operators. by Krishna Kaphle, M.S. A Dissertation In Mathematics and Statistics Submitted to the Graduate Faculty of Texas Tech University in Partial Fulfillment of the Requirements for the Degree of Doctor of Philoshopy Approved Frits H. Ruymgaart Linda J. Allen Petros Hadjicostas Peggy Gordon Miller Interim Dean of the Graduate School August, 2011 c 2011, Krishna Kaphle Texas Tech University, Krishna Kaphle, August-2011 ACKNOWLEDGEMENTS This dissertation would not have been possible without continuous support of my advisor Horn Professor Dr. Firts H. Ruymgaart. I am heartily thankful to Dr. Ruym- gaart for his encouragement, and supervision from the research problem identification to the present level of my dissertation research. His continuous supervision enabled me not only to bring my dissertation in this stage but also to develop an understand- ing of the subject. I am lucky to have a great person like him as my advisor. I am grateful to Horn. Professor Dr. Linda J. Allen for her help throughout my stay at Texas Tech. I would also like to thank Dr. Petros Hadjicostas for encouragement and several valuable remarks during my research. I am also heartily thankful to Dr. David Gilliam for his support and help on writing Matlab code. I am pleased to thank those who made this dissertation possible. Among those, very important people are Dr. Kent Pearce, chair, Dr. Ram Iyer, graduate advisor, and all the staffs of Department of Mathematics and Statistics, who gave me the moral, financial, and technological supports required during my study. I also would like to make a special reference to Dr. Magdalena Toda, undergraduate director, for the encouragement and support I got from her. I would like to thank all Professors of Department of Mathematics and fellow graduate students for helping me on whenever needed to finish this dissertation. ii Texas Tech University, Krishna Kaphle, August-2011 TABLE OF CONTENTS Acknowledgements . ii Abstract . v List of Tables . vi List of Figures . vii 1. Introduction . 1 2. Elements of Hilbert Space Theory . 3 2.1 Hilbert spaces . 3 2.2 Projection and Riesz representation . 5 2.2.1 Projections . 8 2.2.2 Spectral Properties of Operators . 11 3. Random Variables in A Hilbert Space . 19 3.1 Random variables . 19 3.2 Probability Measures on H .................... 22 3.2.1 Some remarks on B ...................... 22 3.2.2 Probability measures on H . 23 3.2.3 Gaussian distributions . 26 3.2.4 Karhunen - Lo`eve expansion . 28 4. Random Samples and Limit theorems . 29 4.1 Random Samples . 29 4.2 Some Central Limit Theorems . 31 5. Functions of covariance operators and Delta method . 41 5.1 Functions of bounded linear operators . 43 5.1.1 Fr´echet derivative . 45 5.1.2 Delta Method . 48 6. Perturbation of eigenvalues and eigenvectors . 51 6.1 Perturbation theory for operators . 51 6.2 Perturbation theory for matrices . 56 7. Testing equality of covariance operators . 61 7.1 Finite dimensional case . 61 7.2 Infinite dimensional case . 62 iii Texas Tech University, Krishna Kaphle, August-2011 7.2.1 Test statistic under null hypothesis . 62 7.2.2 Estimation of variance . 67 8. Generalized test . 70 8.1 The two-sample case . 73 8.2 Test statistics . 75 8.3 Estimation of the variance . 79 8.3.1 The Gaussian case . 80 9. Some Simulations . 82 9.1 Test using single eigenvalue . 83 9.2 Test using the first m largest eigenvalues . 85 9.2.1 The null Hypothesis . 85 9.2.2 The fixed alternative . 86 9.2.3 Identification of regularization parameter . 87 9.2.4 Local alternatives . 88 10. conclusion . 93 Bibliography . 94 Appendix: Matlab Code . 98 iv Texas Tech University, Krishna Kaphle, August-2011 ABSTRACT The generalization of multivariate statistical procedures to infinite dimension nat- urally requires extra theoretical work. In this dissertation, we will focus on testing the equality of covariance operators. We derive a procedure from the Union Intersec- tion principle in conjunction with a Likelihood Ratio test. This procedure leads to a statistic which is the largest eigenvalue of a product of operators. We generalize this procedure by using a test statistic that is based on the first m 2 N largest eigenvalues. Perturbation theory of operators and functional calculus of covariance operators are extensively used to derieve the required asymptotics. It is shown that the power of the test is improved with inclusion of more eigenvalues. We perform simulations to corroborate the testing procedure, using samples from two Gaussian distributions. v Texas Tech University, Krishna Kaphle, August-2011 LIST OF TABLES 9.1 Test statistic and the regularization parameter . 85 9.2 Power vs inclusion of eigenvalues . 86 9.3 Fraction of rejections under the null Hypothesis . 87 9.4 Type II error . 92 vi Texas Tech University, Krishna Kaphle, August-2011 LIST OF FIGURES 8.1 Contours used for integration . 71 8.2 Contours enclosing m eigenvalues . 72 9.1 Histogram of 1000 test statistic values under Null . 84 9.2e ^1 vs e1 .................................. 84 9.3v ^ vs v ................................... 84 9.4 Histogram of test statistic values under null when two and three eigen- value are taken. 86 9.5 Histograms of test statistics under Local alternative when two, and three eigenvalue is taken. 87 9.6 Histogram of test statistics under Null when epsilon is 0.5 two, and three eigenvalue is taken. 88 9.7 Histogram of test statistics under Null when epsilon is 1.0 when two, and three eigenvalues are taken. 89 9.8 Histogram of test statistics under Null when epsilon is 1.5 when two, and three eigenvalues are taken. 89 9.9 QQ plot of the test statistics under Null when epsilon is 0.5 when two, and three eigenvalue are taken. 90 9.10 QQ plot of the test statistics under Null when epsilon is 1.0 when two, and three eigenvalue are taken. 90 9.11 QQ plot of the test statistics under Null when epsilon is 1.5 when two, and three eigenvalue is taken. 91 9.12 Fraction of rejection vs Gamma for fixed sample size . 91 vii Texas Tech University, Krishna Kaphle, August-2011 CHAPTER 1 INTRODUCTION Univariate statistical theory is concerned with statistical inference based on the samples from a distribution on the real line. Multivariate statistics refers to the in- ference based on the samples from a distribution on Euclidean, i.e. finite dimensional spaces. Functional data analysis deals with samples in infinite dimensional spaces, typically function spaces. Indeed, the generic sample element is usually a function and an infinite dimensional object. It may also be an object of very high finite dimension. Throughout this research we will always assume the data to be infinite dimensional. More specifically, the generic sample element will be supposed to be an element of an infinite dimensional Hilbert space H. We will keep H abstract. In the theory, this has the advantage that the properties obtained for sample means, for instance, entail at once properties of the sample covariance operator, because it is also a sample mean in a Hilbert space (albeit not the same Hilbert space in which the sample elements assume their values). A good example of a Hilbert space that might be used for functional data is L2(0; 1), the space of all square integrable functions on [0,1]. Many classical statistical problems can be formulated for functional data [36]. This dissertation focuses on the testing equality of the covariance structures of two populations, based on random samples. Before going any further in the development of the theory, we will discuss some examples of functional data analysis (see [29] for detail discussion). 1. As a generic sample element we may consider the angle of the hip and knee over a child's gait cycle. The cycle begins when the heel touches the ground and ends when it touches the ground again. A natural question that arises is whether there is a relation between the two cycles (see [26])? 2. Near - infrared spectroscopy is applied to different varieties of wheat. Let Xi(t) denote the density of the reflected radiation recorded at the spectrometer when the wave length equals t and ηi represents the level of a given protein for the i-th type of wheat. Theory suggests that the relation between ηi and Xi's of the form Z b ηi = C + Xi(t)f(t)dt + error; for i =1; 2; : : : ; n a 1 Texas Tech University, Krishna Kaphle, August-2011 where C is a constant. This represents a functional regression model(see [19]). In the above examples we have seen several statistical problems for functional data that are well - known in multivariate statistics: For example, the two - sample problem, multiple and canonical correlation, and regression. Principal component analysis is another point of interest that extends to functional data. In this research we will need a version of the central limit theorem in Hilbert spaces, that is somewhat more general then the central limit theorem in its simplest form. Although very general central limit exists (see [24]), we prefer to present an independent proof tailored to the situation at hand. This yields the asymptotic distribution of both the sample mean and the sample covariance operators for certain triangular arrays of H- valued random variables. An important tool for the asymptotic distribution of the test statistic that we purpose in Chapter 7 to deal with equality of covariance structures, is a delta method for analytic functions of random operators. There are many analogies between infinite dimensional Hilbert spaces and Eu- clidean spaces.