MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2018 Lecture 20 THE BASICS OF STOCHASTIC PROCESSES Contents 1. Stochastic processes: spaces R∞ and R[0,∞) 2. The Bernoulli process 3. The Poisson process We now turn to the study of some simple classes of stochastic processes. Exam- ples and a more leisurely discussion of this material can be found in the corre- sponding chapter of [BT]. A discrete-time stochastic is a sequence of random variables {Xn} defined on a common probability space ( , F, P). In more detail, a stochastic process is a function X of two variables n and ω. For every n, the function ω 7→ Xn(ω) is a random variable (a measurable function). An alternative perspective is provided by fixing some ω ∈ and viewing Xn(ω) as a function of n (a “time function,” or “sample path,” or “trajectory”). A continuous-time stochastic process is defined similarly, as a collection of random variables {Xt} defined on a common probability space ( , F, P), where t varies over non-negative real values R+. 1 SPACES OF TRAJECTORIES: R∞ and R[0,∞) 1.1 σ-algebras on spaces of trajectories Recall that earlier we defined the Borel σ-algebra Bn on Rn as the smallest σ algebra containing all measurable rectangles, i.e. events of the form x n B1 × · · · × Bn = { ∈ R : xj ∈ Bj ∀j ∈ [n]} where Bj are (1-dimensional) Borel subsets of R. A generalization is the fol- lowing: 1 Definition 1. Let T be an arbitrary set of indices. The product space RT is defined as T R , R = {(xt, t ∈ T )} . tY∈T T A subset JS(B) of R is called a cylinder with base B on time indices S = {s1, . , sn} if n JS(B) = {(xt):(xs1 , . , xsn ) ∈ B} ,B ⊂ R , (1) with B ∈ Bn . The product σ-algebra BT is the smallest σ-algebra containing all cylinders: T S B = σ{JS(B) : ∀S-finite and B ∈ B } . For the special case T = {1, 2,..., } the notation R∞ and B∞ will be used. The following are measurable subsets of R∞: ∞ E0 = {x ∈ R : xn–converges} The following are measurable subsets of R[0,∞): [0,∞) E1 = {x ∈ R : xt = 0 ∀t ∈ Q} (2) [0,∞) E2 = {x ∈ R : sup xt > 0} (3) t∈Q The following are not measurable subsets of R[0,∞): ′ [0,∞) E1 = {x ∈ R : xt = 0 ∀t} (4) ′ [0,∞) E2 = {x ∈ R : sup xt > 0} (5) t [0,∞) E3 = {x ∈ R : xt–continuous} (6) ′ ′ Non-measurability of E1 and E2 will follow from the next result. We mention ′ [0,+∞) that since E1 ∩ E3 = E1 ∩ E3, then by considering a trace of B on E3 ′ ′ sets E1 and E2 can be made measurable. This is a typical approach taken in the theory of continuous stochastic processes. Proposition 1. The following provides information about BT : T (i) For every measurable set E ∈ B there exists a countable set of time ∞ indices S = {s1,...} and a subset B ∈ B such that E = {(xt):(xs1 , . , xsn ,...) ∈ B} (7) 2 (ii) Every measurable set E ∈ BT can be approximated within arbitrary ǫ by a cylinder: P[E△JS(B)] ≤ ǫ , T T where P is any probability measure on (R , B ). (iii) If {Xt, t ∈ T } is a collection of random variables on ( , F), then the map T X : → R , (8) ω 7→ (Xt(ω), t ∈ T ) (9) is measurable with respect to BT . Proof: For (i) simply notice that collection of sets of the form (7) contains all cylinders and closed under countable unions/intersections. To see this simply notice that one can without loss of generality assume that every set in, for ex- ample, union F = En correspond to the same set of indices in (7) (otherwise extend the index setsS S first). S (ii) follows from the next exercise and the fact that {JS(B),B ∈ B } (under fixed finite S) form a σ-algebra. For (iii) note that it is sufficient to check that −1 T X (JS(B)) ∈ F (since cylinders generate B ). The latter follows at once from the definition of a cylinder (1) and the fact that {(Xs1 ,...,Xsn ) ∈ B} are clearly in F. Exercise 1. Let be a collection of -algebras and let be the F α , α ∈ S σ F = 2S F α smallest σ-algebra containing all of them. Call set B finitary if B ∈ F α , where W 2 S1 S1 is a finite subset of S. Prove that every E ∈ F is finitary approximable,W i.e. that for every ǫ > 0 there exists a finitary B such that P[E△B] ≤ ǫ . (Hint: Let L = {E : E–finitary approximable} and show that L contains the algebra of finitary sets and closed under monotone limits.) With these preparations we are ready to give a definition of stochastic pro- cess: Definition 2. Let ( Ω, F, P) be a probability space. A stochastic process with T T time set T is a measurable map X : ( Ω, F) → (R , B ). The pushforward −1 PX , P ◦ X is called the law of X. 3 1.2 Probability measures on spaces of trajectories According to Proposition 1 we may define probability measures on RT by sim- ply computing an induced measure along a map (9). An alternative way to define probabilities on RT is via the following construction. Theorem 1 (Kolmogorov). Suppose that for any finite S ⊂ T we have S a probability measure PS on R and that these measures are consistent. ′ Namely, if S ⊂ S then S\S ′ PS′ [B] = PS[B × R ] . T Then there exists a unique probability measure P on R such that P[JS(B)] = PS[B] for every cylinder JS(B). Proof (optional): As a simple exercise, reader is encouraged to show that it suffices to consider the case of countable T (cf. Proposition 1.(i)). We thus ∞ focus on constructing a measure on R . Let A = n≥1 Fn, where Fn is the σ-algebra of all cylinders with time indices {1, . , nS}. Clearly A is an algebra. Define a set-function on A via: , ∀E = {(x1, . , xn) ∈ B} : P[E] P{1,...,n}[B] . Consistency conditions guarantee that this assignment is well-defined and results in a finitely additive set-function. We need to verify countable additivity. Let En ց Ø (10) By repeating the sets as needed, we may assume En ∈ Fn. If we can show that P[En] ց 0 (11) then Caratheodory’s extension theorem guarantees that P extends uniquely to σ(A) = B∞ . We will use the following facts about Rn: n n 1. Every finite measure µ on (R , B ) is inner regular, namely for every n E ∈ B µ[E] = sup µ[K] , (12) K⊂E supremum over all compact subsets of E. 4 2. Every decreasing sequence of non-empty compact sets has non-empty in- tersection: Kn =6 Ø,Kn ց K ⇒ K 6= Ø (13) 3. If f : Rn → Rk is continuous, then f (K) is compact for every compact K. Then according to (12) for every En and every ǫ > 0 there exists a compact ′ n subset Kn ⊂ R such that such that ′ −n P[En \J1,...,n(Kn )] ≤ ǫ2 . Then, define by induction ′ Kn = Kn ∩ (Kn−1 × R) . n−1 (Note that Kn−1 ⊂ R and the set Kn−1 × R is simply an extension of Kn−1 n into R by allowing arbitrary last coordinates.) Since En ⊂ En−1 we have −n P[En \J1,...,n(Kn)] ≤ ǫ2 + P[En−1 \J1,...,n−1(Kn−1)] . Thus, continuing by induction we have shown that −1 −n P[En \J1,...,n(Kn)] ≤ ǫ(2 + ··· 2 ) < ǫ (14) We will show next that Kn = Ø for all n large enough. Since by construction En ⊃ J1,...,n(Kn) (15) we then have from (14) and Kn = Ø that lim sup P[En] < ǫ . n→∞ By taking ǫ to 0 we have shown (11) and the Theorem. It thus remains to show that Kn = Ø for all large enough n. Suppose otherwise, then by construction we have 2 n−1 Kn ⊂ Kn−1 × R ⊂ Kn−2 × R ⊂ ··· ⊂ K1 × R . Thus by projecting each Kn onto first coordinate we get a decreasing sequence of non-empty compacts, which by (13) has non-empty intersection. Then we can pick a point x1 ∈ R such that x1 ∈ Projn→1(Kn) ∀n . 5 Repeating the same argument but projecting onto first two coordinates, we can now pick x2 ∈ R such that (x1, x2) ∈ Projn→2(Kn) ∀n . By continuing in this fashion we will have constructed the sequence (x1, x2,...) ∈ J1,...,n(Kn) ∀n . By (15) then we have (x1, x2,...) ∈ En n\≥1 which contradicts (10). Thus, one of Kn must be empty. 1.3 Tail σ-algebra and Kolmogorov’s 0/1 law ∞ ∞ ∞ Definition 3. Consider (R , B ) and let Fn be a sub-σ-algebra generated by all cylinders Js1,...,sk (B) with sj ≥ n. Then the σ-algebra , ∞ T Fn n>\0 is called a tail σ-algebra on R∞ . If X : Ω → R∞ is a stochastic process, then σ-algebra X−1T is called a tail σ-algebra of X. Examples of tail events: E1 = {sequence Xn converges} (16) E2 = {series Xn converges} (17) X E3 = {lim sup Xn > 0} , (18) n→∞ An example of the event which is not a tail event: n E4 = {lim sup Xk > 0} n→∞ Xk=1 Theorem 2 (Kolmogorov’s 0/1 law). If Xj , j = 1,... are independent then any event in the tail σ-algebra of X has probability 0 or 1.