Lecture Notes for STAT 547C: Topics in Probability (Draft)
Total Page:16
File Type:pdf, Size:1020Kb
Lecture notes for STAT 547C: Topics in Probability (draft) Ben Bloem-Reddy November 26, 2019 Contents 1 Motivation 5 2 Measurable spaces7 2.1 Basic set notation and operations.............................7 2.2 σ-algebras..........................................9 2.3 Examples of σ-algebras................................... 10 2.4 Measurable spaces..................................... 13 2.4.1 Products of measurable spaces.......................... 14 3 Measurable functions 15 3.1 Measurability of functions................................. 15 3.2 Compositions of functions................................. 17 3.3 Numerical functions.................................... 17 3.4 Measurability of limits of sequences of functions..................... 18 3.5 Simple functions and approximating measurable functions............... 19 3.6 Standard and common measurable spaces........................ 21 4 Measures, probability measures, and probability spaces 23 4.1 Measures and measure spaces............................... 23 4.1.1 Probability measures and probability spaces................... 23 4.2 Examples.......................................... 23 4.3 Some properties of measures................................ 24 4.4 Negligible/null sets and completeness........................... 26 4.5 Almost everywhere/surely................................. 26 4.6 Image measures....................................... 26 5 Probability spaces 28 5.1 Probability spaces as models of reality.......................... 28 5.2 Statistical models...................................... 29 6 Random variables 31 6.1 Distribution of a random variable............................. 31 1 6.2 Examples of random variables............................... 33 6.3 Joint distributions and independence........................... 36 6.4 Information and determinability............................. 36 7 Integration 37 7.1 Definition and desiderata................................. 37 7.2 Examples.......................................... 39 7.3 Basic properties of integration............................... 39 7.4 Monotone Convergence Theorem............................. 40 7.5 Further properties of integration............................. 42 7.6 Characterization of the integral.............................. 43 7.7 More on interchanging limits and integration...................... 43 7.8 Integration and image measures.............................. 44 8 Expectation 46 8.1 Integrating with respect to a probability measure.................... 46 8.2 Changes of measure: indefinite integrals and Radon{Nikodym derivatives...... 47 8.3 Moments and inequalities................................. 50 8.4 Transforms and generating functions........................... 51 9 Independence 53 9.1 Independence of random variables............................ 54 9.2 Sums of independent random variables.......................... 55 9.3 Tail fields and Kolmogorov's 0-1 law........................... 55 10 Conditioning and disintegration 57 10.1 Conditional expectations.................................. 57 10.2 Conditional probabilities and distributions........................ 61 10.3 Interjection: kernels and product spaces......................... 62 10.4 Conditional probabilities and distributions, continued................. 67 10.5 Conditional independence................................. 70 10.6 Statistical sufficiency.................................... 72 10.7 Construction of probability spaces............................ 73 11 Convergence 74 11.1 Convergence of real sequences............................... 74 11.2 Almost sure convergence.................................. 75 11.3 Convergence in probability................................ 77 11.4 Convergence in Lp ..................................... 79 11.5 Overview of modes of convergence............................ 83 11.6 Convergence in distribution................................ 83 11.7 Central Limit Theorem.................................. 87 11.8 Laws of large numbers................................... 89 12 Martingales 92 12.1 Basic definitions...................................... 92 12.2 Martingale Convergence Theorem............................. 95 2 12.3 Martingale inequalities................................... 98 A Additional results 101 A.1 An example of the use of Radon{Nikodym derivatives................. 101 3 List of Exercises 1 Exercise (Limits of monotone decreasing sequences of sets)...............9 2 Exercise (Countable unions and limits of monotone increasing sequences of sets).. 10 3 Exercise (Countable intersections and limits of monotone decreasing sequences of sets) 10 4 Exercise (Intersection of σ-algebras)........................... 10 5 Exercise (Trace spaces, C¸inlar Ex. I.1.15)........................ 13 6 Exercise (Proof of Lemma 3.1).............................. 16 7 Exercise (Proof of Proposition 3.2)............................ 16 8 Exercise (σ-algebra generated by a function)....................... 16 9 Exercise (Measurable functions of measurable functions are measurable)....... 17 10 Exercise (Compositions under which the class of simple functions is closed)..... 19 11 Exercise (Plotting the dyadic functions)......................... 20 12 Exercise (Restrictions and traces, C¸inlar Ex. I.3.11).................. 25 13 Exercise (Extensions, C¸inlar Ex. I.3.12)......................... 25 14 Exercise (Image measure)................................. 27 15 Exercise........................................... 32 16 Exercise (Random variable from undergraduate probability).............. 35 17 Exercise (Random variable from your research interests)................ 36 18 Exercise (Insensitivity of integration)........................... 42 19 Exercise (Proof of Theorem 7.9)............................. 44 20 Exercise (Proof of Markov's inequality)......................... 50 21 Exercise (Chebyshev's inequality)............................. 50 22 Exercise (Generalized Markov's inequality)....................... 50 23 Exercise (Independence of functions of independent random variables.)........ 55 24 Exercise........................................... 56 25 Exercise........................................... 58 26 Exercise........................................... 58 27 Exercise (Independence in conditional expectation)................... 61 28 Exercise (Measurable sections).............................. 64 29 Exercise (Equivalence of conditional independence relationships)........... 71 30 Exercise (Chain rule for conditional independence)................... 72 31 Exercise (Almost sure version of the continuous mapping theorem).......... 75 32 Exercise (Applications of Borel{Cantelli Lemma).................... 76 33 Exercise (Convergence in probability and arithmetic).................. 79 34 Exercise (Implications of Lp convergence)........................ 80 35 Exercise (Convex increasing functions of submartingales)................ 94 4 1 Motivation Placeholder for these notes. 5 A few words on mathematics in these notes and generally Mathematics in these notes. I aim to be mathematically rigorous in these notes, but I will also include some (attempts at) intuitive explanations of challenging technical concepts. Appealing to intuition sometimes requires relaxing the rigor. It should be clear when I am making an intuition- based argment|if it is not, please ask. Notation. I will mostly follow the notation of C¸inlar [C¸in11], who lists some frequently used notation on page ix. If I deviate from that notation, I will say so. Lemma, Proposition, Theorem, etc. As is typically the case, only the most important results will be called theorems. Propositions are typically intermediate results that might are of interesting on their own, but whose importance|at least within this material|does not reach the lofty heights of the exalted theorems. Lemmas are intermediate or supporting results that often could be included within the body of the proof they support. Different authors have different preferences, but well- structured lemmas can enhance the conceptual clarity and readability of long or complex proofs. Writing mathematics. Something to keep in mind as you do research is that the clarity of your writing can greatly affect how your work is received|even the very best work will be hindered by poor (or even mediocre) writing. Writing mathematics can be especially challenging. I encourage you to practice in your assignment write-ups and especially in your final project. I will post some resources that I have found helpful to my writing. Disclaimer. The primary purpose of these notes is to stay organized during lecture. They are a work in progress. They do not replace the readings. Furthermore, they are not a model of good mathematical writing. If something is unclear, please ask. If you find a typo or an error, please let me know. 6 2 Measurable spaces Reading: C¸inlar, I.1. Supplemental: Overview. Recall from last time that defining a probability distribution on an uncountable space can lead to fundamental issues. Ideally, we want the elements of the space to encode all possible outcomes of an experiment, and we want to define a function for assigning probability to those outcomes in a self-consistent way (i.e., the probability of all possible outcomes is 1, and is equal to the sum of the probabilities of each possible outcome.) We ran into problems when trying to work in an uncountable space|in essence, there were too many points (uncountably many) that we required the probability to \measure" in a self-consistent way.