STA 711: Probability & Measure Theory Robert L. Wolpert 2 Construction & Extension of Measures For any finite set Ω = {ω1, ..., ωn}, the “power set” P(Ω) is the collection of all subsets of Ω, including the empty-set ∅ and Ω itself. It has |P| =2n elements; it can also be identified with the set of all possible functions a : Ω →{0, 1} by the relation A = {ω : a(ω)=1}. Set theorists denote the power set by P(Ω) = {0, 1}Ω or more simply by 2Ω, even for infinite sets Ω. The function a := 1A equal to one if a ∈ A and otherwise zero is the “indicator” function of A. Recall that a probability measure on some σ-algebra F on a set Ω is a function P : F → R with the three properties: P1 : (∀A ∈F) P(A) ≥ 0 P2 : P(Ω) = 1 P3 : (∀Aj ∈F, i =6 j ⇒ Ai ∩ Aj = ∅), P(∪Aj)= P(Aj) We will want to assign probabilities to as many subsets of Ω as possible (so we can find P probabilities of a wide range of events) while actually specifying probabilities on as small a class of sets as possible (to minimize how much work we do). For a finite probability space Ω with n ∈ N elements, for example, we will see below that we need specify only the n probabilities {P[{ω}]: ω ∈ Ω} of the singletons (one-element sets {ω}) to determine P(A) uniquely for all 2n elements A ∈ 2Ω. Since n ≪ 2n for big n, this is a bargain. Let’s consider a number of properties that classes of sets A ⊂ 2Ω might have. A class A of subsets of Ω is called a: FIELD if F1 : Ω ∈A c F2 : E ∈A ⇒ E ∈A F3 : E1, E2 ∈A ⇒ E1 ∪ E2 ∈A. σ-FIELD if σ1 : Ω ∈A c σ2 : E ∈A ⇒ E ∈A σ3 : {Ei}⊂A ⇒ ∪ Ei ∈A. π-SYSTEM if π1 : E1, E2 ∈A ⇒ E1 ∩ E2 ∈A. λ-SYSTEM if λ1 : Ω ∈A c λ2 : E ∈A ⇒ E ∈A λ3 : {Ei}⊂A, Ei ∩ Ej = ∅ ⇒ ∪ Ei ∈A. Note that if Aα is a (F, σF, πS, resp. λS) for each α in any index set (even an uncountable one), then ∩αAα is also a (F, σF, πS, resp. λS) (Exercise: show that this is not true for even Ω finite unions). Since also 2 is a (F, σF, πS, resp. λS), it follows that for any collection A0 ⊂ Ω 2 there exists a smallest (F, σF, πS, resp. λS) that contains A0: namely, the intersection of all (F, σF, πS, resp. λS)s containing A0. We denote the smallest (F, σF, πS, resp. λS) containing A0 by F(A0), σ(A0), π(A0), and λ(A0), respectively. 1 STA 711 Week 2 R L Wolpert For example, if Ω is arbitrary and A0 = {{ω} : ω ∈ Ω}, all singletons, then F(A0) = Ω σ(A0)=2 if Ω is finite. If Ω is infinite, however, then F(A0) is the collection of finite and co-finite sets; σ(A0) and λ(A0) are both the collection of countable and co-countable sets; and π(A0) is just {A0 ∪ {∅}}. For probability and measure theory we would like for probabilities P(A) to be defined on all the sets A ⊂ Ω that we encounter. For finite or countable Ω we can usually define P(A) sensibly for all subsets A, but for uncountable Ω this typically isn’t possible (see free on-line Appendices B or C of Frank Burk’s text Lebesgue Measure and Integration: An Introduction for a nice account). If we can’t define P(A) on all of 2Ω, we still need probabilities to be defined for all sets in a sigma field F, so we can compute probabilities for countable unions and intersections. We’d like the luxury of having to specify measures on a much smaller collection, like a field F0 or a collection of sets C that generates a field F0 := F(C). That’s our goal for the next week or so. To do this we need to know that we can always extend a probability assignment µ0 defined on a field F0 to exactly one measure µ on the sigma field F = σ(F0)— i.e., that (a) there exists at least one such extension, and that (b) any two must agree on all of F. It turns out to be easier to show that µ0 extends uniquely to the λ-system λ(A0) than it is to show unique extension to the sigma field σ(A0); luckily, when A0 is a field (or even just a π-system), these are the same. This will follow from: 2.1 Dynkin’s Theorem Theorem 1 (Dynkin’s π-λ) Let P be a π-system; then λ(P)= σ(P). Proof. The proof is in two parts. First we show that λ(P) is not only a λ-system, it’s also a π-system; then, we show that any collection L ⊂ 2Ω that is both a λ-system and a π-system is also a σ-algebra. Thus σ(P) ⊆ λ(P) ⊆ σ(P), proving the theorem. I. L := λ(P) is a π-system We must show that L is closed under intersections, i.e., that A ∩ B ∈ L whenever A, B ∈L. Fix any A ∈ P and set A := {B ∈L : A ∩ B ∈ L} . As a step on the way, let’s show that: A is a λ-system containing P. There are three things to show for all B, {Bi}⊂A: λ1 : Ω ∈A : A ∩ Ω= A ∈P⊂L. c c c c c λ2 : B ∈A ⇒ B ∈A : A ∩ B = A ∩ (A ∩ B) = [A ∪ (A ∩ B)] ∈L by λ2, λ3. λ3 : Bi ∩ Bj = ∅ ⇒ ∪Bi ∈A : A ∩ (∪Bi)= ∪(A ∩ Bi) ∈A by λ3. Also P⊂A by π1, so A is a λ-system containing P and hence containing L = λ(P). Page 2 STA 711 Week 2 R L Wolpert We have just shown that A ∩ B ∈L for every A ∈ P and B ∈ L. So, for every B ∈ L, the class B = {A ∈L : A ∩ B ∈ L} contains each A ∈ P. Also Ω ∈ B (by λ1) and B is closed under complements (as before: Ac ∩ B = (A ∩ B)c ∩ B = [(A ∩ B) ∪ Bc]c ∈ L) and disjoint unions ((A ∪ A′) ∩ B = (A ∩ B) ∪ (A′ ∩ B)), so B is a λ-system containing P and hence containing L := λ(P). This completes the proof that A ∩ B ∈L for every A, B ∈L, i.e., that L is a π-system. II. If L is a π-system and a λ-system, then L is a σ-algebra. Since any λ-system satisfies conditions σ1 = λ1 and σ2 = λ2, it remains only to show σ3. Let {Ai}⊂L, and for n ∈ N let Bn be “what’s new in An,” i.e., define c c c Bn := An ∩ Ai = An ∩ Bi = An ∩ Bi . (1) i<n i<n i<n [ [ \ c The {Bn} are disjoint (since each Bn is in Bi for each i < n) and, since ∪i≤nAi = ∪i≤nBi for every n ∈ N, the {Bn} have the same union as {An}. Thus Ai = Bn ∈L i n [ [ by λ3, and L is a σ-algebra. This completes the proof of Dynkin’s π−λ theorem. How can this help us to extend uniquely a probability assignment or “pre-measure” (defined in Section (2.3)) µ0 from a π-system P (for example, a field) to the σ-field F = σ(P) it generates? First, note that λ-systems are just perfect for uniqueness: Proposition 1 Let P and Q be two probability measures on a space (Ω, F). The class L = {A ∈F : P (A)= Q(A)} is a λ-system. Can you prove that? By Dynkin’s π−λ theorem, there is at most one extension of a “pre- measure” P0 from any π-system P to the σ-algebra σ(P) = λ(P) it generates, because if P and Q were two different ones, the collection of events on which they agree would be a λ-system containing P and hence containing λ(P)= σ(P). Let’s look at examples: 1. P := {a} on Ω= {a, b, c}. To illustrate that uniqueness of extensions can fail, con- sider a probability assignment µ on the π-system P that assigns probability µ({a})= 1 1/2. For any number 0 ≤ p ≤ 2 there exists a distinct extension µp of µ to the σ- Ω 1 algebra F = 2 that assigns probabilities µp({b}) = p, µp({c})=( 2 − p). For p =6 q, the collection of events L for which µp(L)= µq(L) is L = {∅, {a}, {b, c}, Ω},a λ-system (and σ-algebra) strictly smaller than F. Page 3 STA 711 Week 2 R L Wolpert 2. P := {{ω} : ω ∈ Ω } ∪ {∅}: Given any finite or countable set Ω = {ωi} and positive numbers {pi ≥ 0} with unit sum i pi = 1, define µ0 on P by setting µ0({ωi}) = pi and µ0(∅) = 0. Then by countable additivity the only possible probability measure on Ω P Ω 2 that extends µ0 is µ(A) := [pi : ωi ∈ A]. Every probability measure on 2 for any finite or countable set Ω is of this form. P 3. P := { (−∞, b], b ∈ Q } onΩ=(−∞, ∞). The field generated by P consists of finite disjoint unions of left-open rational intervals (a, b], including semi-infinite intervals of the form (−∞, b] and (a, ∞), and Ω = (−∞, ∞). The sigma field σ(A) is not just countable unions of such sets; it is the “Borel” σ-algebra B(R) generated by the open sets in the real line and includes all open and closed sets, the Cantor set, and many others.