The Problem of Moments and Exchangeability
Total Page:16
File Type:pdf, Size:1020Kb
THE PROBLEM OF MOMENTS AND EXCHANGEABILITY Chun-Kai Huang December, 2009 THE PROBLEM OF MOMENTS AND EXCHANGEABILITY by Chun-Kai Huang Thesis submitted to the University of KwaZulu-Natal in fulfilment of the re- quirements for the degree of Master of Science in the School of Statistics and Actuarial Science. Degree Assessment Board Thesis advisor Prof. A. I. Dale Thesis examiners To be announced As the candidate’s supervisor, I have/have not approved this thesis/dissertation for submission. Signed and Dated: SCHOOL OF STATISTICS AND ACTUARIAL SCIENCE WESTVILLE CAMPUS, DURBAN, SOUTH AFRICA UNIVERSITY OF KWAZULU-NATAL Declaration I, Chun-Kai Huang, declare that 1. The research reported in this thesis, except where otherwise indicated, is my original research. 2. This thesis has not been submitted for any degree or examination at any other university. 3. This thesis does not contain other persons’ data, pictures, graphs or other information, unless specifically acknowlegded as being sourced from other persons. 4. This thesis does not contain other persons’ writing, unless specifically ac- knowledged as being sourced from other researchers. Where other written sources have been quoted, then (a) their words have been re-written but the general information at- tributed to them has been referenced, or (b) where their exact words have been used, then their writing has been placed in italics and referenced. 5. This thesis does not contain text, graphics or tables copied and pasted from the internet, unless specifically acknowledged, and the source being detailed in the thesis and in the reference sections. Signed: Disclaimer This document describes work undertaken as part of a masters programme of study at the University of KwaZulu-Natal (UKZN). All views and opinions expressed therein remain the sole responsibility of the author, and do not nec- essarily represent those of the institute. Abstract In the second volume of “An introduction to Probability Theory and Its Applications”, Feller (1966) proved de Finetti’s Representation Theorem for infinite sequences of exchangeable {0, 1}-random variables via the solu- tion to the Hausdorff Moment Problem over the unit line. It can be shown that the above problems are actually equivalent, i.e. an infinite sequence of {0, 1}-random variables is exchangeable if and only if the corresponding Hausdorff Moment Problem over the unit line has a solution. The Hausdorff Moment Problem has long been studied and the theory ex- tends to multiple variables over various domains, while the concepts of Exchangeability are also vastly extended. This gave rise to the idea of the current thesis, aiming to draw some connections between the different cases of Problems of Moments and the concepts of Exchangeability. Keywords Hausdorff moment problems, moment problems, exchangeability, partial exchangeability, de Finetti’s theorem, exchangeable sequences, completely monotonic sequences. i Contents Abstract i Acknowledgements iii 1 Preliminaries 1 1.1 Introduction and Terminology . 1 1.2 A Brief History of the Moment Problems . 3 1.3 A Brief History of Exchangeability . 8 2 Hausdorff Moment Problem 11 2.1 A Basic Model . 11 2.2 HMP over the unit square . 14 2.3 HMP over a triangle . 18 2.4 Some Remarks on HMP . 24 2.5 Further Results for HMP . 27 3 Exchangeability and Partial Exchangeability 31 3.1 Introduction . 31 3.2 Exchangeable and Partially exchangeable events . 32 3.3 Exchangeable Random Variables . 33 3.4 De Finetti’s Theorems on Exchangeability . 38 3.5 De Finetti’s Theorems on Partial exchangeability . 46 4 Connections between Moment Problem and Exchangeability 51 4.1 Introduction . 51 4.2 HMP and Exchangeability on the unit square and triangle . 53 4.3 HMP and Exchangeability on several k-dim. simplexes . 56 4.4 Some final remarks . 59 Appendix 62 Bibliography 63 ii Acknowledgements First, and foremost, I wish to thank my supervisor, Professor A. I. Dale for his contin- ued interest, invaluable advice and tremendous patience throughout the preparation of this thesis. I also wish to thank Professor D. E. North, and fellow colleagues of the School of Statis- tics and Actuarial Science, for the use of facilities in the department and their contin- ued support. I am also indebted to the South African National Research Foundation and the Univer- sity of KwaZulu-Natal for the awards of postgraduate scholarships and bursaries. Finally, I would like to thank my wife, Shy-Yin, and my family for their constant sup- port and encouragement. iii Chapter 1 Preliminaries In this opening chapter, a general introduction to the motivation of this the- sis (i.e. historical links between Moment problems and Exchangeability), short summaries of the chapters and some terminology issues are discussed. Some historical reviews of both topics are also provided for interest and reference. 1.1 Introduction and Terminology It is not necessary in this thesis to discuss the pros and cons between subjective and objective views of probability. In any case, most mathematical statisticians nowadays are (or should be) equipped with enough knowledge of both to make judgements of their own. The purpose of this research is merely to discuss some relations between the concept of exchangeability and moment problems, from a view point that is unbiased and understandable by both Bayesians and non- Bayesians. The first motivation of this research was obtained from Feller (1966), on page 229 of volume II, where he proved de Finetti’s theorem for 2-valued sequences via the solution of the Hausdorff moment problem over the unit interval. He also suggested that this method can easily extend to a sequence that assumes a finite number of values. It turns out that one may also construct an exchangeable sequence of 2-valued random variables from a given sequence of Hausdorff moments over the unit interval, i.e. the two theorems are clearly equivalent. This leads to the obvi- ous questions: what happens if one move onto moment problems over other domains? And, how do other extensions of de Finetti’s theorem link up with the general moment problems? Some links between these two theories may also be found in a few other pa- 1 1.1. Introduction and Terminology pers. Dale (1983) gave an probabilistic proof of Hausdorff’s theorem for double sequences using de Finetti’s result for partially exchangeable events. Diaconis & Freedman (2004a,b) discussed the connections between the Markov moment problem and de Finetti’s theorem. Bassetti & Regazzini (2008) discussed how de Finetti proved a proposition regarding the characteristic function of an infinite exchangeable sequence of events via the solution to the Hamburger moment problem. And, most recently, Peng et al. (2009) showed that partial exchange- ability can be characterized by rectangular complete monotonicity. The current thesis will explore both topics with the aim to discuss their rela- tions. In particular, the equivalence between de Finetti’s theorems for infinite exchangeable sequences that take on a finite or infinite number of values and Hausdorff’s theorem over various domains. The content assumes some basic knowledge of probability measure theory and real analysis. Readers may also obtain detailed background information on mo- ment problems and exchangeability in Shohat & Tamarkin (1943) and Bernardo & Smith (1994), respectively. CHAPTER SUMMARIES: The remainder of this chapter will be devoted to some historical background of the two topics. Chapter 2 formally introduces the Hausdorff Moment problems. The solutions are also extended to various domains, which are useful in a later Chapter. Ex- amples and discussions are also provided for further understanding of the prob- lems. Exchangeability and Partial Exchangeability are introduced in Chapter 3. De- tailed proofs of de Finetti’s representation theorems in several cases are pro- vided. Interesting counterexamples on exchangeability are also discussed here. Chapter 4 provides some results on the links between the Hausdorff moment problems and Exchangeability. I have shown here that de Finetti’s representa- tion theorem for a partially exchangeable sequence over a finite set of values is completely characterized by the Hausdorff moment problem over several k- dimensional simplexes (various simpler cases are also given). Difficulties in expanding to infinite and continuous cases are also discussed in the final con- clusion. 2 1.2. A Brief History of the Moment Problems TERMINOLOGY AND NOTATION: The following terminology and notations will be used throughout this thesis: R = the set of real numbers N = {1, 2, 3,...} N0 = {0, 1, 2,...} [k] = {1, 2, . , k} [a, b] = the set of all real numbers between a and b P = a probability measure Ef(X) = the expected value of f(X) B(R) = the class of all Borel sets in R Cov(X, Y ) = the covariance between X and Y V ar(X) = the variance of X We shall also use “,” to mean “by definition, equals to”. Vectors will be denoted with bold fonts, e.g. x1 = (x11, x12, . , x1k) is a k-dimensional vector in some specified space. A g-IID sequence shall mean a g-fold infinite sequence of inde- pendent random variables that are identically distributed within each of the g types. Other terminology and notations in the thesis either will be commonly used or will be introduced in the context. 1.2 A Brief History of the Moment Problems In the celebrated paper Recherches sur les fractions continues (1894 and 1895), which introduced our modern concept of Stieltjes Integrals, Stieltjes also pro- posed and solved the problem which he called “The Problem of Moments”. The problem Stieltjes proposed is to determine a bounded non-decreasing func- tion F (x) in the interval [0, ∞) such that its moments have a prescribed set of values, i.e. Z ∞ n x dF (x) = µn , n = 0, 1, 2,.... (1.1) 0 The terminology “Problem of Moments” was taken by Stieltjes from Mechanics, as he often used the concepts of mass, stability, etc.