Gaussian, Markov and stationary processes Alejandro Ribeiro Dept. of Electrical and Systems Engineering University of Pennsylvania [email protected] http://www.seas.upenn.edu/users/~aribeiro/ November 11, 2016 Stoch. Systems Analysis Queues 1 Introduction and roadmap Introduction and roadmap Gaussian processes Brownian motion and its variants White Gaussian noise Stoch. Systems Analysis Queues 2 Stochastic processes I Assign a function X (t) to a random event I Without restrictions, there is little to say about stochastic processes I Memoryless property makes matters simpler and is not too restrictive I Have also restricted attention to discrete time and/or discrete space I Simplifies matters further but might be too restrictive I Time t and range of X (t) values continuous I Time and/or state may be discrete as particular cases I Restrict attention to (any type or a combination of types) Markov processes (memoryless) ) Gaussian processes (Gaussian probability distributions) ) Stationary processes (“limit distributions”) ) Stoch. Systems Analysis Queues 3 Markov processes I X (t) is a Markov process when the future is independent of the past I For all t > s and arbitrary values x(t), x(s)andx(u)forallu < s P X (t) x(t) X (s) > x(s), X (u) > x(u), u < s ≥ ⇥ =P X (t) x(t) ⇤X (s) > x(s) ≥ ⇥ ⇤ I Memoryless property defined in terms of cdfs not pmfs I Memoryless property useful for same reasons of discrete time/state I But not as much useful as in discrete time /state Stoch. Systems Analysis Queues 4 Gaussian processes I X (t)isaGaussianprocesswhenall prob. distributions are Gaussian I For arbitrary times t1, t2,...tn it holds Values X (t ), X (t ),...X (t ) are jointly Gaussian ) 1 2 n I Will define more precisely later on I Simplifies study because Gaussian distribution is simplest possible Suffices to know mean, variances and (cross-)covariances ) Linear transformation of independent Gaussians is Gaussian ) Linear transformation of jointly Gaussians is Gaussian ) I More details later Stoch. Systems Analysis Queues 5 Markov processes + Gaussian processes I Markov (memoryless) and Gaussian properties are di↵erent Will study cases when both hold ) I Brownian motion, also known as Wiener process I Brownian motion with drift I White noise linear evolution models ) I Geometric brownian motion pricing of stocks, arbitrages, risk neutral measures, pricing of stock) options (Black-Scholes) Stoch. Systems Analysis Queues 6 Stationary processes I Process X (t) is stationary if all probabilities are invariant to time shifts I I.e., for arbitrary times t1, t2,...,tn and arbitrary time shift s P[X (t + s) x , X (t + s) x ,...,X (t + s) x ]= 1 ≥ 1 2 ≥ 2 n ≥ n P[X (t ) x , X (t ) x ,...,X (t ) x ] 1 ≥ 1 2 ≥ 2 n ≥ n I System’s behavior is independent of time origin I Follows from our success on studying limit probabilities I Stationary process study of limit distribution ⇡ I Will study Spectral analysis of stationary stochastic processes ) Linear filtering of stationary stochastic processes ) Stoch. Systems Analysis Queues 7 Gaussian processes Introduction and roadmap Gaussian processes Brownian motion and its variants White Gaussian noise Stoch. Systems Analysis Queues 8 Jointly gaussian variables I Random variables (RV) X1, X2,...,Xn are jointly Gaussian (normal) if any linear combination of them is Gaussian T I We may also say, vector RV X =[X1,...,Xn] is Gaussian (normal) T I Formally, for any a1, a2,...,an variable (a =[a1,...,an] ) T Y = a1X1 + a2X2 + ...+ anXn = a X I is normally distributed I Consider 2 dimensions 2RVsX and X jointly normal ) 1 2 I To describe joint distribution have to specify Means: µ1 = E [X1]andµ2 = E [X2] ) 2 2 2 Variances: σ =var[X1] = E (X1 µ1) and σ =var[X2] ) 11 − 22 2 Covariance: σ = cov(X1)=E [(X1 µ1)(X2 µ2)] ) 12 ⇥ − ⇤ − Stoch. Systems Analysis Queues 9 Pdf of jointly normal RVs in 2 dimensions T I In 2 dimensions, define vector µ =[µ1,µ2] T I And covariance matrix C with elements (C is symmetric, C = C ) σ2 σ2 C = 11 12 σ2 σ2 ✓ 12 22 ◆ I Joint pdf of x is given by 1 1 T 1 f (x)= exp (x µ) C− (x µ) X 1/2 −2 − − 2⇡ det (C) ✓ ◆ I Assumed that C is invertible and as a consequence det(C) =0 6 T I Can verify that any linear combination a x is normal if the pdf of x is as given above Stoch. Systems Analysis Queues 10 Pdf of jointly normal RVs in n dimensions n I For X R (n dimensions) define µ = E [X] and covariance matrix 2 2 E (X1) E [(X1)X2] ... E [(X1)Xn] 2 E [X2X1] E X2 ... E [X2Xn] T ⇥ ⇤ C := E xx = 0 . 1 . ⇥ ⇤ .. B C ⇥ ⇤ B [X X ] [X X ] ... X 2 C B E n 1 E n 2 E n C @ A I C symmetric. Consistent with 2-dimensional def. Made⇥ µ⇤ =0 I Joint pdf of x defined as before (almost) 1 1 T 1 fX(x)= exp (x µ) C− (x µ) n/2 1/2 −2 − − (2⇡) det (C) ✓ ◆ I C invertible, therefore det(C) = 0. All linear combinations normal 6 I Expected value µ and covariance matrix C completely specify probability distribution of a Gaussian vector X Stoch. Systems Analysis Queues 11 Anotationalaside n n n n I With x R , µ R and C R ⇥ , define function (µ, C; x)as 2 2 2 N 1 1 T 1 (µ, C; x):= exp (x µ) C− (x µ) N n/2 1/2 −2 − − (2⇡) det (C) ✓ ◆ I µ and C are parameters, x is the argument of the function n I Let X R be a Gaussian vector with mean µ, and covariance C 2 I Can write the pdf of X as 1 1 T 1 fX(x) = exp (x µ) C− (x µ) := (µ, C; x) n/2 1/2 −2 − − N (2⇡) det (C) ✓ ◆ Stoch. Systems Analysis Queues 12 Gaussian processes I Gaussian processes (GP) generalize Gaussian vectors to infinite dimensions I X (t)isaGPifany linear combination of values X (t)isGaussian I I.e., for arbitrary times t1, t2,...,tn and constants a1, a2,...,an Y = a1X (t1)+a2X (t2)+...+ anX (tn) I has a normal distribution I t can be a continuous or discrete time index I More general, any linear functional of X (t)isnormallydistributed I A functional is a function of a function t2 I E.g., the (random) integral Y = X (t) dt has a normal distribution Zt1 I Integral functional is akin to a sum of X (ti ) Stoch. Systems Analysis Queues 13 Joint pdfs in a Gaussian process I Consider times t1, t2,...,tn.Themeanvalueµ(ti )atsuchtimesis µ(ti )=E [X (ti )] I The cross-covariance between values at times ti and tj is C(ti , tj )=E X (ti ) µ(ti ) X (tj ) µ(tj ) − − I Covariance matrix for values⇥ X (t1), X (t2),...,X (tn)isthen⇤ C(t1, t1) C(t1, t2) ... C(t1, tn) C(t2, t1) C(t2, t2) ... C(t2, tn) C(t1,...,tn)=0 . 1 . .. B C B C(t , t ) C(t , t ) ... C(t , t ) C B n 1 n 2 n n C @ A I Joint pdf of X (t1), X (t2),...,X (tn) then given as T T f (x1,...,xn)= [µ(t1),...,µ(tn)] , C(t1,...,tn);[x1,...,xn] X (t1),...,X (tn) N ⇣ ⌘ Stoch. Systems Analysis Queues 14 Mean value and autocorrelation functions I To specify a Gaussian process, suffices to specify: Mean value function µ(t)=E [X (t)] ;and ) ) Autocorrelation function R(t1, t2)=E X (t1)X (t2) ) ) I Autocovariance obtained as C(t , t )=R(t , t ) µ(t )µ(t ) 1 2 1 ⇥2 − 1 ⇤2 I For simplicity, most of the time will consider processes with µ(t)=0 I Can always define process Y (t)=X (t) µ (t)withµ (t)=0 − X Y I In such case C(t1, t2)=R(t1, t2) I Autocorrelation is a function of two variables t1 and t2 I Autocorrelation is a symmetric function R(t1, t2)=R(t2, t1) Stoch. Systems Analysis Queues 15 Probabilities in a Gaussian process I All probs. in a GP can be expressed on terms of µ(t)andR(t1, t2) I For example, probability distribution function of X (t)is 2 1 xt µ(t) f (x )= exp − X (t) t 2 2⇡ R(t, t) µ2(t) −2 R(t, t) µ(t) ! − − q I For a zero mean process with µ(t)=0forallt 1 x 2 f (x )= exp t X (t) t −2R(t, t) 2⇡R(t, t) ✓ ◆ p Stoch. Systems Analysis Queues 16 Joint and conditional probabilities in a GP I For a zero mean process consider two times t1 and t2 I The covariance matrix for X (t1)andX (t2)is R(t , t ) R(t , t ) C = 1 1 1 2 R(t , t ) R(t , t ) ✓ 1 2 2 2 ◆ I Joint pdf of X (t1)andX (t2) then given as 1 1 T 1 f (x , x )= exp [x , x ] C− [x , x ] X (t1),X (t2) 1 2 1/2 −2 t1 t2 t1 t2 2⇡ det (C) ✓ ◆ I Conditional pdf of X (t1) given X (t2) computed as fX (t1),X (t2)(xt1 , xt2 ) fX (t1) X (t2)(x1, x2)= | fX (t2)(x2) Stoch. Systems Analysis Queues 17 Brownian motion and its variants Introduction and roadmap Gaussian processes Brownian motion and its variants White Gaussian noise Stoch. Systems Analysis Queues 18 Brownian motion as limit of random walk I Gaussian processes are natural models due to central limit theorem I Let us reconsider a symmetric random walk in one dimension I Walker takes increasingly frequent and increasingly small steps time step = h x(t) t Stoch.