Modes of Convergence in Probability Theory
Total Page:16
File Type:pdf, Size:1020Kb
Modes of Convergence in Probability Theory David Mandel November 5, 2015 Below, fix a probability space (Ω; F;P ) on which all random variables fXng and X are defined. All random variables are assumed to take values in R. Propositions marked with \F" denote results that rely on our finite measure space. That is, those marked results may not hold on a non-finite measure space. Since we already know uniform convergence =) pointwise convergence this proof is omitted, but we include a proof that shows pointwise convergence =) almost sure convergence, and hence uniform convergence =) almost sure convergence. The hierarchy we will show is diagrammed in Fig. 1, where some famous theorems that demonstrate the type of convergence are in parentheses: (SLLN) = strong long of large numbers, (WLLN) = weak law of large numbers, (CLT) ^ = central limit theorem. In parameter estimation, θn is said to be a consistent ^ estimator or θ if θn ! θ in probability. For example, by the SLLN, X¯n ! µ a.s., and hence X¯n ! µ in probability. Therefore the sample mean is a consistent estimator of the population mean. Figure 1: Hierarchy of modes of convergence in probability. 1 1 Definitions of Convergence 1.1 Modes from Calculus Definition Xn ! X pointwise if 8! 2 Ω, 8 > 0, 9N 2 N such that 8n ≥ N, jXn(!) − X(!)j < . Definition Xn ! X uniformly if 8 > 0, 9N 2 N such that 8! 2 Ω and 8n ≥ N, jXn(!) − X(!)j < . 1.2 Modes Unique to Measure Theory Definition Xn ! X in probability if 8 > 0, 8δ > 0, 9N 2 N such that 8n ≥ N, P (jXn − Xj ≥ ) < δ: Or, Xn ! X in probability if 8 > 0, lim P (jXn − Xj ≥ 0) = 0: n!1 The explicit epsilon-delta definition of convergence in probability is useful for proving a.s. =) prob. p Definition Xn ! X in L , p ≥ 1, if 8 > 0, 9N 2 N such that 8n ≥ N, Z p jXn(!) − X(!)j dP (!) =: jjXn − XjjLp < . Ω p Or, Xn ! X in L if p lim jjXn − Xjj p = 0: n!1 L Definition Xn ! X almost surely (a.s.) if 9E 2 F with P (E) = 0 such that 8! 2 Ec and 8 > 0, 9N 2 N such that 8n ≥ N, jXn(!) − X(!)j < . Or, Xn ! X a.s. if 9E 2 F with P (E) = 0 such that Xn ! X pointwise on Ec. 1.3 Mode Unique to Probability Theory Definition Xn ! X in distribution if the distribution functions of the Xn converge pointwise to the distribution function of X at all points x where F (x) is continuous. That is, Xn ! X in distribution if 8x 2 R such that F (x) is continuous, 8 > 0, 9N 2 N such that 8n ≥ N, jFn(x) − F (x)j < . Or, Xn ! X in distribution if 8x 2 R such that F (x) is continuous, lim Fn(x) = F (x): n!1 2 2 Convergence Results Proposition Pointwise convergence =) almost sure convergence. Proof Let ! 2 Ω, > 0 and assume Xn ! X pointwise. Then 9N 2 N such that 8n ≥ N, jXn(!) − X(!)j < . Hence Xn ! X almost surely since this convergence takes place on all sets E 2 F. Proposition Uniform convergence =) convergence in probability. Proof Let > 0 and assume Xn ! X uniformly. Then 9N 2 N such that 8! 2 Ω and 8n ≥ N, jXn(!) − X(!)j < . Let n ≥ N. Then P (jXn − Xj ≥ ) = P (;) = 0, so of course P (jXn − Xj ≥ ) ! 0 as n ! 1. Proposition Lp convergence =) convergence in probability. p Proof Let > 0 and assume Xn ! X in L , p ≥ 1. Then p jjX − Xjj p P (jX − Xj ≥ ) ≤ n L ! 0 as n ! 1; n p where the first inequality is Chebyshev's inequality. p F Proposition Uniform convergence =) L convergence. Proof Let > 0 and assume Xn ! X uniformly. Then 9N 2 N such that 8! 2 Ω and 8n ≥ N, jXn(!) − X(!)j < . Let n ≥ N. Then Z p p jjXn − XjjLp = jXn(!) − X(!)j dP (!) < P (Ω) = . Ω F Proposition Almost sure convergence =) convergence in probability. This result relies on Egoroff's theorem, which we state now without proof. Theorem 2.1 (Egoroff) Let P (Ω) < 1 and assume Xn ! X a.s. Then 8δ > c 0, 9E 2 F with P (E) < δ such that Xn ! X uniformly on E . Proof Let > 0. Also let δ > 0 and let E 2 F such a set as in Egoroff's c c theorem. Since Xn ! X uniformly on E , 9N 2 N such that 8! 2 E and 8n ≥ N, jXn(!) − X(!)j < . Let n ≥ N. Then c P (jXn − Xj ≥ ) = P (jXn − Xj ≥ \ E) + P (jXn − Xj ≥ \ E ) < δ + P (;) = δ: 3 Proposition Convergence in probability =) convergence in distribution. Proof First, a lemma. Lemma 2.2 Let Xn and X be random variables. Then 8 > 0 and x 2 R, P (Xn ≤ x) ≤ P (X ≤ x + ) + P (jXn − Xj > ); P (X ≤ x − ) ≤ P (Xn ≤ x) + P (jXn − Xj > ); Proof (lemma) For the first line, P (Xn ≤ x) = P (Xn ≤ x; X ≤ x + ) + P (Xn ≤ x; X > x + ) ≤ P (X ≤ x + ) + P (Xn − X ≤ x − X;x − X < −) ≤ P (X ≤ x + ) + P (Xn − X < −) ≤ P (X ≤ x + ) + P (Xn − X < −) + P (Xn − X > ) = P (X ≤ x + ) + P (jXn − Xj > ): For the second line, P (X ≤ x − ) = P (X ≤ x − , Xn ≤ x) + P (X ≤ x − , Xn > x) ≤ P (Xn ≤ x) + P (x − X ≥ , Xn − X > x − X) ≤ P (Xn ≤ x) + P (Xn − X > ) ≤ P (Xn ≤ x) + P (Xn − X > ) + P (Xn − X < −) = P (Xn ≤ x) + P (jXn − Xj > ): Let x 2 R be a continuity point of F and let > 0. The lemma then tells us that Fn(x) ≤ F (x + ) + P (jXn − Xj > ); F (x − ) ≤ Fn(x) + P (jXn − Xj > ): Hence F (x − ) − P (jXn − Xj > ) ≤ Fn(x) ≤ F (x + ) + P (jXn − Xj > ): Now let n ! 1, in which case P (jXn − Xj > ) ! 0: F (x − ) ≤ lim inf Fn(x) ≤ lim sup Fn(x) ≤ F (x + ): n!1 n!1 Finally let ! 0+ and use continuity of F at x: F (x) ≤ lim inf Fn(x) ≤ lim sup Fn(x) ≤ F (x); n!1 n!1 hence limn!1 Fn(x) = F (x). 4 3 References References [1] Wikipedia Proofs of Convergence of Random Variables https://en.wikipedia.org/wiki/Proofs_of_convergence_of_ random_variables [2] University of Alabama, Birmingham Convergence in Distribution (Lecture Notes) http://www.math.uah.edu/stat/dist/Convergence.html [3] Gerald B. Folland Real Analysis: Modern Techniques and Their Applications http://www.amazon.com/Real-Analysis-Modern-Techniques-Applications/ dp/0471317160 5.