Gane Samb LO Mathematical Foundations of Probability Theory arXiv:1808.01713v1 [math.PR] 6 Aug 2018 Statistics and Probability African Society (SPAS) Books Series. Saint-Louis, Calgary, Alberta. 2018. DOI : http://dx.doi.org/10.16929/sbs/2016.0008 ISBN 978-2-9559183-8-8 i ii SPAS TEXTBOOKS SERIES GENERAL EDITOR of SPAS EDITIONS Prof Gane Samb LO [email protected], [email protected] Gaston Berger University (UGB), Saint-Louis, SENEGAL. African University of Sciences and Technology, AUST, Abuja, Nigeria. ASSOCIATED EDITORS KEhinde Dahud SHANGODOYIN [email protected] University of Botswana (Botswana) Blaise SOME [email protected] Chairman of LANIBIO, UFR/SEA Joseph Ki-Zerbo University (Ouaga I), Burkina-Faso. ADVISORS Ahmadou Bamba SOW [email protected] Gaston Berger University, Senegal. Tchilabalo Abozou KPANZOU [email protected] Kara University, Togo. iii List of published books List of published or scheduled books in English Weak Convergence (IA) - Sequences of Random Vectors. Gane Samb LO, Modou NGOM and Tchilabalo A. KPANZOU. 2016. Doi : 10.16929/sbs/2016.0001. ISBN 978-2-9559183-1-9 A Course on Elementary Probability Theory. Gane Samb LO. 2017. Doi : 10.16929/sbs/2016.0003. ISBN 978-2-9559183-3-3 Measure Theory and Integration By and For the Learner. Gane Samb LO. Doi : http://dx.doi.org/10.16929/sbs/2016.0005. ISBN 978-2-9559183- 5-7 iv Library of Congress Cataloging-in-Publication Data Gane Samb LO, 1958- Mathematical Foundations of Probability Theory. SPAS Books Series, 2018. DOI : 10.16929/sbs/2016.0008 ISBN 978-2-9559183-8-8 v Author : Gane Samb LO Emails: [email protected], [email protected]. Url’s: [email protected] www.statpas.net/[email protected]. Affiliations. Main affiliation : University Gaston Berger, UGB, SENEGAL. African University of Sciences and Technology, AUST, ABuja, Nigeria. Affiliated as a researcher to : LSTA, Pierre et Marie Curie University, Paris VI, France. Teaches or has taught at the graduate level in the following univer- sities: Saint-Louis, Senegal (UGB) Banjul, Gambia (TUG) Bamako, Mali (USTTB) Ouagadougou - Burkina Faso (UJK) African Institute of Mathematical Sciences, Mbour, SENEGAL, AIMS. Franceville, Gabon Dedicatory. To my first and tender assistants, my daughters Fatim Zahr`a Lo and Maryam Majigun Azr`aLo Acknowledgment of Funding. The author acknowledges continuous support of the World Bank Ex- cellence Center in Mathematics, Computer Sciences and Intelligence Technology, CEA-MITIC. His research projects in 2014, 2015 and 2016 are funded by the University of Gaston Berger in different forms and by CEA-MITIC. Mathematical Foundations of Probability Theory Abstract. (English) In the footsteps of the book Measure Theory and Integration By and For the Learner of our series in Probability Theory and Statistics, we intended to devote a special volume of the very probabilistic aspects of the first cited theory. The book might have assigned the title : From Measure Theory and Integration to Probability Theory. The fundamental aspects of Probability Theory, as described by the keywords and phrases below, are presented, not from ex- periences as in the book A Course on Elementary Probability Theory, but from a pure mathematical view based on Mea- sure Theory. Such an approach places Probability Theory in its natural frame of Functional Analysis and constitutes a firm preparation to the study of Random Analysis and Sto- chastic processes. At the same time, it offers a solid basis towards Mathematical Statistics Theory. The book will be continuously updated and improved on a yearly basis. (Fran¸cais) Keywords. Measure Theory and Integration; Probabilistic Terminology of Measure Theory and Applications; Probabil- ity Theory Axiomatic; Fundamental Properties of Probabil- ity Measures; Probability Laws of Random Vectors; Usual Probability Laws review; Gaussian Vectors; Probability In- equalities; Almost sure and in Probability Convergences; Weak convergences; Convergence in Lp; Kolmogorov Theory on se- quences of independent real-valued random variables; Cen- tral Limit Theorem, Laws of Large Numbers, Berry-Essen Approximation, Law of the iterated logarithm for real valued independent random variables; Existence Theorem of Kol- mogorov and Skorohod for Stochastic processes; Conditional Expectations; First examples of stochastic process : Brown- ian and Poisson Processes. AMS 2010 Classification Subjects : 60-01; 60-02; 60- 03;G0Axx; 62GXX. Contents General Preface 1 Introduction 3 Chapter 1. An update of the Terminology from Measure Theory to Probability Theory 7 1. Introduction 7 2. Probabilistic Terminology 7 3. Independence 19 4. Pointcarr´eand Bonferroni Formulas 28 Chapter 2. Random Variables in Rd, d 1 31 1. A review of Important Results for≥ Real Random variables 31 2. Moments of Real Random Variables 37 3. Cumulative distribution functions 42 d 4. Random variables on R or Random Vectors 48 5. Probability Laws and Probability Density Functions of Random vectors 64 6. Characteristic functions 77 7. Convolution, Change of variables and other properties 91 8. Copulas 98 9. Conclusion 101 Chapter 3. Usual Probability Laws 105 1. Discrete probability laws 105 2. Absolutely Continuous Probability Laws 111 Chapter 4. An Introduction to Gauss Random Measures 133 1. Gauss Probability Laws on R 133 2. Gauss Probability Law on Rd, Random Vectors 140 Chapter 5. Introduction to Convergences of Random Variables 151 1. Introduction 151 2. Almost-sure Convergence, Convergence in probability 156 3. Convergence in Lp 161 4. A simple review on weak convergence 170 i ii CONTENTS 5. Convergence in Probability and a.s. convergence on Rd 173 6. Comparison between convergence in probability and weak convergence 175 Chapter 6. Inequalities in Probability Theory 179 1. Conditional Mathematical Expectation 179 2. Recall of already known inequalities 181 3. Series of Inequalities 185 Chapter 7. Introduction to Classical Asymptotic Theorems of Independent Random variables 207 1. Easy Introduction 207 2. Tail events and Kolmogorov’s zero-one law and strong laws of Large Numbers 216 3. Convergence of Partial sums of independent Gaussian random variables 232 4. The Lindenberg-Lyapounov-Levy-Feller Central Limit Theorem 234 5. Berry-Essen approximation 254 6. Law of the Iterated Logarithm 271 Chapter 8. Conditional Expectation 279 1. Introduction and definition 279 2. The operator of the mathematical expectation 281 3. Other Important Properties 283 4. Generalization of the definition 287 5. Mathematical expectation with respect to a random variable288 6. Jensen’s Inequality for Mathematical Expectation 290 7. The Mathematical Expectation as an Orthogonal Projection in L2 293 8. Useful Techniques 294 Chapter 9. Probability Laws of family of Random Variables 297 1. Introduction 297 2. Arbitrary Product Probability Measure 298 3. Stochastic Process, Measurability for a family of Random Variables 305 4. Probability Laws of Families of random Variables 307 5. Skorohod’s Construction of real vector-valued stochastic processes 316 6. Examples 320 7. Caratheodory’s Extension and Proof of the Fundamental Theorem of Kolmogorov 325 CONTENTS iii Chapter 10. Appendix 329 1. Some Elements of Topology 329 2. Orthogonal Matrices, Diagonalization of Real Symmetrical Matrices and Quadratic forms 333 3. What should not be ignored on limits in R - Exercises with Solutions 344 4. Important Lemmas when dealing with limits on limits in R 356 5. Miscellaneous Results and facts 360 6. Quick and powerfull algorithms for Gaussian probabilities 360 Bibliography 363 General Preface This textbook is one of the elements of a series whose ambition is to cover a broad part of Probability Theory and Statistics. These text- books are intended to help learners and readers, of all levels, to train themselves. As well, they may constitute helpful documents for professors and teachers for both courses and exercises. For more ambitious people, they are only starting points towards more advanced and personalized books. So, these textbooks are kindly put at the disposal of professors and learners. Our textbooks are classified into categories. A series of introductory books for beginners. Books of this series are usually destined to students of first year in universities and to any individual wishing to have an initiation on the subject. They do not require advanced mathematics. Books on elementary probability the- ory (See Lo (2017a), for instance) and descriptive statistics are to be put in that category. Books of that kind are usually introductions to more advanced and mathematical versions of the same theory. Books of the first kind also prepare the applications of those of the second. A series of books oriented to applications. Students or researchers in very related disciplines such as Health studies, Hydrology, Finance, Economics, etc. may be in need of Probability Theory or Statistics. They are not interested in these disciplines by themselves. Rather, they need to apply their findings as tools to solve their specific prob- lems. So, adapted books on Probability Theory and Statistics may be composed to focus on the applications of such fields. A perfect example concerns the need of mathematical statistics for economists who do not necessarily have a good background in Measure Theory. A series of specialized books on Probability theory and Sta- tistics of high level. This series begins with a book on Measure 1 2 GENERAL PREFACE Theory, a book on its probability theory version, and an introductory book on topology. On that basis, we will have, as much as possible, a coherent presentation of branches of Probability theory and Statistics. We will try to have a self-contained approach, as much as possible, so that anything we need will be in the series. Finally, a series of research monographs closes this architecture. This architecture should be so diversified and deep that the readers of monograph booklets will find all needed theories and inputs in it. We conclude by saying that, with only an undergraduate level, the reader will open the door of anything in Probability theory and sta- tistics with Measure Theory and integration.