MEASURE THEORY AND STOCHASTIC PROCESSES Lecture Notes José Melo, Susana Cruz Faculdade de Engenharia da Universidade do Porto Programa Doutoral em Engenharia Electrotécnica e de Computadores March 2011 Contents 1 Probability Space3 1 Sample space Ω .....................................4 2 σ-eld F .........................................4 3 Probability Measure P .................................6 3.1 Measure µ ....................................6 3.2 Probability Measure P .............................7 4 Learning Objectives..................................7 5 Appendix........................................8 2 Chapter 1 Probability Space Let's consider the experience of throwing a dart on a circular target with radius r (assuming the dart always hits the target), divided in 4 dierent areas as illustrated in Figure 1.1. 4 3 2 1 Figure 1.1: Circular Target The circles that bound the regions 1, 2, 3, and 4, have radius of, respectively, r , r , 3r , and . 4 2 4 r Therefore, the probability that a dart lands in each region is: 1 , 3 , 5 , P (1) = 16 P (2) = 16 P (3) = 16 7 . P (4) = 16 For this kind of problems, the theory of discrete probability spaces suces. However, when it comes to problems involving either an innitely repeated operation or an innitely ne op- eration, this mathematical framework does not apply. This motivates the introduction of a measure-theoretic probability approach to correctly describe those cases. We dene the proba- bility space (Ω; F;P ), where Ω is the sample space, F is the event space, and P is the probability 3 measure. Each of them will be described in the following subsections. 1 Sample space Ω The sample space Ω is the set of all the possible results or outcomes ! of an experiment or observation. For the previously described experience (hereafter referred to as E1), the sample space is Ω1 = f1; 2; 3; 4g. This is an example of a nite sample space. Considering a variant of E1, an example of an innitely repeated operation E2 would be throwing the dart until winning the game (hit the region 1). The sample space in this case would be Ω2 = f1; 21; 31; 41; 221; 231;:::g, and this is an example of an innite, yet countable, sample space. An example of an innitely ne operation E3 would be throwing the dart and viewing the point of impact as the outcome. For this experience, the sample space is Ω3 = (d; α): d 2 [0; r]; α 2 [0; 2π[, corresponding to an uncountable sample space. Thus, concerning their cardinality, we can classify the sample spaces as countable and nite, countable and innite, and uncountable. A set is countable if there exists a bijective mapping from a subset of the set of natural numbers onto that set. Therefore, all nite sets are count- able but not all countable sets are nite. The remaining countable sets are called innite and countable. For instance, the set of natural numbers itself is countable, but it is not nite. An uncountable set is an innite set which cannot be put in a one-to-one correspondence with the set of natural numbers N, i.e., one cannot order the elements into a sequence, for example the set real numbers R. 2 σ-eld F Considering a completely arbitrary non-empty space Ω, a class F of subsets of Ω is called a σ-eld 1 if it contains the empty set, is closed under the formation of complements, and is closed under the formation of countable unions: (i) ; 2 F; (ii) A 2 F ) AC 2 F; (iii) countable set S . Ai 2 F i 2 I;I = ) i2I Ai 2 F Since Ω and the empty set are complementary, by (ii) the rst condition can be replaced with Ω 2 F. Being closed under complementation also implies that (iii) is equivalent to say that F is closed under the formation of countable intersections. This results from DeMorgan's law, A \ B = (AC [ BC )C and A [ B = (AC \ BC )C . In the more general case where F is closed 1Some authors also use the terms sigma-eld, sigma-algebra or σ-algebra. 4 under nite unions/intersections F is a eld. With this result we can concluded that all σ-elds are also elds, but the opposite is not always true. The largest σ-eld containing all possible subsets of Ω, including the empty set, is called power set. It contains 2n elements, where n is the number of elements of Ω, and is usually denoted by P (Ω). Being A a collection of subsets of Ω, there is a unique σ-eld generated by A, denoted by σ(A), which is the smallest σ-eld containing A. In cases where σ(A) is more dicult to obtain than P (Ω), we can use F = P (Ω), as σ(A) ⊆ P (Ω). Example 1. Suppose Ω = fa; b; cg. Then, P (Ω) = f;; fag; fbg; fcg; fabg; fbcg; facg; Ωg. We can see that jP (Ω)j = 23 = 8 elements. Considering A = fa; bg, σ(A) = f;; fag; fbg; Ωg. Therefore, it is clear that σ(A) = P (A) ⊂ P (Ω). An interesting curiosity is the analogy between the elements of the power set and the sequence of binary numbers, where "1" means that the corresponding element is in the subset, and "0" means the opposite, as shown in Table 1.1. abc Subset 0 000 fg 1 001 fcg 2 010 fbg 3 011 fcbg 4 100 fag 5 101 facg 6 110 fabg 7 111 fabcg Table 1.1: Power set construction example The power set is useful when we are dealing with countable sample spaces. However, when this is not the case, one needs a dierent way to dene the event space. Thus, when the sample space is uncountable, the Borel σ-eld is used. The Borel σ-eld B is the smallest σ-eld in a topological space X such that every open set in X belongs to B. The members of B are called Borel sets. If X = R, B can be generated by any of the intervals (a; b); (a; b]; [a; b), or [a; b], with −∞ ≤ a ≤ b ≤ +1, i.e.: B = σ((a; b)) = σ((a; b]) = σ((a; b]) = σ([a; b]) = σ(open subsets of R): Example 2. Considering , and 1 1 , 1 1 . Being Ω = [0; 1] A = [0; 4 ); [ 4 ; 1] σ(A) = f;; [0; 4 ); [ 4 ; 1]; Ωg B([0; 1]) the collection of all the Borel sets generated by [0; 1], it is clear that σ(A) ⊂ B([0; 1]) and, therefore, both can be used as a valid event space F associated with the sample space Ω. 5 3 Probability Measure P 3.1 Measure µ A measure µ is a function dened on a σ-eld M over a sample space Ω and taking values in + such that the following properties are satised: R0 (i) The empty set has measure zero: µ(;) = 0. (ii) Countable additivity or σ-additivity: if A1;A2;::: is a countable collection of pairwise disjoint sets in F, then: 1 ! 1 [ X µ Ai = µ(Ai): i=1 i=1 A measure space (Ω; F; µ) is said to be complete if A ⊂ B; B 2 F, and µ(B) = 0, together imply that A 2 F and hence that µ(A) = 0. In appendix there can be found some interesting properties and their corresponding proofs related with measure spaces. The Borel measure, dened on the Borel σ-eld B, gives to the interval (a; b], with a < b, the measure b−a. However, the Borel measure is not complete, which is why in practice the complete Lebesgue measure is preferred. The Lebesgue measure, dened on the Lebesgue σ-eld L, is a completion of the Borel measure and L is such that B ⊂ L. Actually, L is obtained by adding to B all the subsets of sets of measure zero that belong to the power set of R. This nds explanation in the fact that a subset of a set of measure zero might not be Borel measurable. Therefore every Borel measurable set is also Lebesgue measurable, and both measures agree. Note that the elements of the Lebesgue σ-eld are called Lebesgue measurable sets. Formally the Lebesgue measure is constructed in the following way: • For any half-open set ]a; b], with a < b, dene its length as L(]a; b]) = b − a. ∗ • For any subset of R dene µ (B) as ∗ µ (B) = inf f L(E): 8 E ⊇ B where E is a countable union of disjoint intervals ]ai; bi]g : • A set A ⊆ R is Lebesgue measurable, i.e., it belongs to the Lebesgue σ-algebra, if µ∗(B) = µ∗(B \ A) + µ∗(B − A) for all B ⊆ R. • The Lebesgue measure is dened by µ(A) = µ∗(A) for all A in the Lebesgue σ-algebra L. However, there are some sets that are not measurable. One example of such sets are the Vitali sets, named after Vitali who proved their existence. For more details on this subject please refer to [6]. 6 3.2 Probability Measure P Given a σ-eld F of subsets of Ω, any measure P : F! [0; 1] such that P (Ω) = 1 is called a probability measure on F. In addition, if F is complete with respect to P , we say that the triple (Ω; F;P ) forms a probability space. Therefore we can dene a probability measure P : F! [0; 1] according to the following proper- ties: (i) P (Ω) = 1. (ii) S1 P1 with T . P ( i=1 Ai) = i=1 P (Ai) Ai 2 F;Ai Aj = ;; i 6= j Similarly to what happens with basic probabilities, probability measures also follow the properties specied below, for A; B 2 F: (i) A ⊆ B ) P (A) ≤ P (B) (ii) P (A) = 1 − P (AC ) (iii) P (A [ B) = P (A) + P (B) − P (A \ B) (iv) P , being a partition of .