556: MATHEMATICAL STATISTICS I MEASURE, INTEGRATION AND PROBABILITY DISTRIBUTIONS In the measure-theoretic framework, random variables are merely measurable functions with respect to the probability space (­; E; P), that is, for random variable X = X(!), ! 2 ­, with domain E, the inverse image of Borel set B, X¡1(B) ´ f! 2 E : X(!) 2 Bg, is an element of σ-algebra E. The expectation of X can be written in any of the following ways Z Z Z X(!) P X(!) P(d!) X(!) dP(!) E E E Lebesgue-Stieltjes Integration: If P is a probability measure on B, then there is a unique corresponding real function F defined for x 2 R by F (x) = P((¡1; x]), termed the distribution function. Conversely, if F is a distribution function, then F defines a measure ¹F on the Borel sets of R, B: we define ¹F on B via sets (a; b]) by ¹F ((a; b]) = F (b) ¡ F (a) and then extend to B by using union operations. The probability space (R; B; ¹F ) is then completed by considering and including null sets (under ¹F ). Let LF denote the smallest σ-algebra containing B and all ¹F -null sets. Thus the triple (R; LF ; ¹F ) is the completed probability space. If g : R ¡! R is a measurable function on the probability space (R; LF ; ¹F ), then the Lebesgue- Stieltjes integral of g is Z Z Z g d¹F = g(x) dF = g(x) dF (x): The Radon-Nikodym Theorem and Change of Measure : ² σ-finite Measure : a measure ¹ defined on a σ-algebra C of subsets of a set C is called finite if ¹(C) is a finite real number. The measure ¹ is called σ-finite if C the countable union of measurable sets of finite measure. A set in a measure space has σ-finite measure, if it is a union of sets with finite measure. ² Absolute Continuity : If ¹ and º are two measures on (C; C), then º is absolutely continuous with respect to ¹, denoted º ¿ ¹, if and only if for all E 2 C, ¹(E) = 0 =) º(E) = 0. ² The RadonNikodym Theorem : For a measure space (C; C), if measure º is absolutely continuous with respect to a σ-finite measure ¹, then there exists a measurable function f defined on C and taking values in [0; 1), such that Z º(A) = f d¹ A for any measurable set A. The function f is unique almost everywhere wrt ¹; it is termed the Radon-Nikodym Derivative of density of º with respect to ¹, and is often expressed as dº f = d¹ ² Change of Measure : If ¹F is absolutely continuous wrt σ-finite measure ¹, then we can rewrite Z Z Z d¹ g d¹ = g F d¹ = gf d¹ F d¹ In practice, ¹ is either counting or Lebesgue measure. 1 Expectations : We can construct expectations for random variables using an identical method of construction as for integral with respect to measure; for a probability space (­; E; P) 1 A random variable X : ­ ¡! R is called simple if it only takes finitely many distinct values; simple random variables can be written Xn X = xiIAi i=1 for some partition A1;:::;An of ­. The expectation of X is Xn E[X] = xiP(Ai) i=1 2 Any non-negative random variable X : ­ ¡! [0; 1) is the limit of some increasing sequence of simple variables, fXng. That is, Xn(!) " X(!), for all ! 2 ­. The expectation of X is E[X] = lim E[Xn] n¡!1 and the limit may be infinite. 3 Any random variable X : ­ ¡! R can be written as X = X+ ¡ X¡, where X+(!) = maxfX(!); 0g X¡(!) = maxf¡X(!); 0g = ¡ minfX(!); 0g The expectation of X is E[X] = E[X+] ¡ E[X¡] if at least one of the two expectations on the right hand side is finite. 4 Thus the expectation of any random variable, E[X] is well-defined for every variable X such that E[jXj] = E[X+ + X¡] < 1 Expectation defined in this fashion obeys the following rules: if fXng is a sequence of rvs with Xn(!) ¡! X(!) for all ! 2 ­, then (i) Monotone Convergence: If Xn(!) ¸ 0 and Xn(!) · Xn+1(!) for all n and !, then E[Xn] ¡! E[X] (ii) Dominated Convergence: If jXn(!)j · Y (!) for all n and !, and E[jY j] < 1, then E[Xn] ¡! E[X] (iii) Bounded Convergence: If jXn(!)j · c, for some c, and for all n and !, then E[Xn] ¡! E[X] These results hold even if Xn(!) ¡! X(!) for all ! except possibly ! in sets of probability zero (termed null events), that is, if Xn(!) ¡! X(!) almost everywhere. One further result is of use in expectation calculations: Fatou’s Lemma : If fXng is a sequence of rvs with Xn(!) ¸ Y (!) almost everywhere for all n and for some Y with E[Y ] < 1, then E[ lim inf Xn ] · lim inf E[Xn] n¡!1 n¡!1 2.