Lecture Notes 6 Random Processes • Definition and Simple Examples

Lecture Notes 6 Random Processes

• Deﬁnition and Simple Examples • Important Classes of Random Processes ◦ IID ◦ Random Walk Process ◦ Markov Processes ◦ Independent Increment Processes ◦ Counting processes and Poisson Process

• Mean and Autocorrelation Function • Gaussian Random Processes ◦ Gauss–Markov Process

EE 278: Random Processes Page 6–1 Random Process

• A random process (RP) (or stochastic process) is an inﬁnite indexed collection of random variables {X(t): t ∈ T }, deﬁned over a common probability space • The index parameter t is typically time, but can also be a spatial dimension • Random processes are used to model random experiments that evolve in time: ◦ Received sequence/waveform at the output of a communication channel ◦ Packet arrival times at a node in a communication network ◦ Thermal noise in a resistor ◦ Scores of an NBA team in consecutive games ◦ Daily price of a stock ◦ Winnings or losses of a gambler

EE 278: Random Processes Page 6–2 Questions Involving Random Processes

• Dependencies of the random variables of the process ◦ How do future received values depend on past received values? ◦ How do future prices of a stock depend on its past values? • Long term averages ◦ What is the proportion of time a queue is empty? ◦ What is the average noise power at the output of a circuit? • Extreme or boundary events ◦ What is the probability that a link in a communication network is congested? ◦ What is the probability that the maximum power in a power distribution line is exceeded? ◦ What is the probability that a gambler will lose all his captial? • Estimation/detection of a signal from a noisy waveform

EE 278: Random Processes Page 6–3 Two Ways to View a Random Process

• A random process can be viewed as a function X(t, ω) of two variables, time t ∈T and the outcome of the underlying random experiment ω ∈ Ω ◦ For ﬁxed t, X(t, ω) is a random variable over Ω ◦ For ﬁxed ω, X(t, ω) is a deterministic function of t, called a sample function

X(t, w1)

t X(t, w2)

t X(t, w3)

t t1 t2 X(t1, w) X(t2, w)

EE 278: Random Processes Page 6–4 Discrete Time Random Process

• A random process is said to be discrete time if T is a countably inﬁnite set, e.g., ◦ N = {0, 1, 2,...} ◦ Z = {..., −2, −1, 0, +1, +2,...}

• In this case the process is denoted by Xn, for n ∈ N , a countably inﬁnite set, and is simply an inﬁnite sequence of random variables • A sample function for a discrete time process is called a sample sequence or sample path • A discrete-time process can comprise discrete, continuous, or mixed r.v.s

EE 278: Random Processes Page 6–5 Example

n • Let Z ∼ U[0, 1], and deﬁne the discrete time process Xn = Z for n ≥ 1 • Sample paths: x 1 n 2 1 1 Z = 2 4 1 8 1 16 n

xn 1 1 Z = 4 4 1 16 1 64 n

xn Z = 0 00 0 ... n 1234567 ...

EE 278: Random Processes Page 6–6 n • First-order pdf of the process: For each n, Xn = Z is a r.v.; the sequence of pdfs of Xn is called the ﬁrst-order pdf of the process

z 0 1

Since Xn is a diﬀerentiable function of the continuous r.v. Z, we can ﬁnd its pdf as 1 1 1 f (x)= = xn−1 , 0 ≤ x ≤ 1 Xn nx(n−1)/n n

EE 278: Random Processes Page 6–7 Continuous Time Random Process

• A random process is continuous time if T is a continuous set • Example: Sinusoidal Signal with Random Phase X(t)= α cos(ωt + Θ) , t ≥ 0 where Θ ∼ U[0, 2π] and α and ω are constants

• Sample functions: x(t) α Θ = 0 π π 3π 2π t 2ω ω 2ω ω x(t)

Θ = π 4 t

x(t)

Θ = π 2 t

EE 278: Random Processes Page 6–8 • The ﬁrst-order pdf of the process is the pdf of X(t)= α cos(ωt + Θ). In an earlier homework exercise, we found it to be 1 fX(t)(x)= , −α

The graph of the pdf is shownp below

fX(t)(x)

1 απ

x −α 0 +α Note that the pdf is independent of t. (The process is stationary)

EE 278: Random Processes Page 6–9 Specifying a Random Process

• In the above examples we speciﬁed the random process by describing the set of sample functions (sequences, paths) and explicitly providing a probability measure over the set of events (subsets of sample functions) • This way of specifying a random process has very limited applicability, and is suited only for very simple processes • A random process is typically speciﬁed (directly or indirectly) by specifying all its n-th order cdfs (pdfs, pmfs), i.e., the joint cdf (pdf, pmf) of the samples

X(t1),X(t2),...,X(tn)

for every order n and for every set of n points t1, t2,...,tn ∈T • The following examples of important random processes will be speciﬁed (directly or indirectly) in this manner

EE 278: Random Processes Page 6–10 Important Classes of Random Processes

• IID process: {Xn : n ∈ N } is an IID process if the r.v.s Xn are i.i.d. Examples:

◦ Bernoulli process: X1,X2,...,Xn,... i.i.d. ∼ Bern(p)

◦ Discrete-time white Gaussian noise (WGN): X1,...,Xn,... i.i.d. ∼ N (0, N) • Here we specified the n-th order pmfs (pdfs) of the processes by specifying the first-order pmf (pdf) and stating that the r.v.s are independent • It would be quite difficult to provide the specifications for an IID process by specifying the probability measure over the subsets of the sample space

EE 278: Random Processes Page 6–11 The Random Walk Process

• Let Z1,Z2,...,Zn,... be i.i.d., where 1 +1 with probability 2 Zn = 1 (−1 with probability 2 • The random walk process is deﬁned by

X0 = 0 n

Xn = Zi , n ≥ 1 i=1 X • Again this process is speciﬁed by (indirectly) specifying all n-th order pmfs • Sample path: The sample path for a random walk is a sequence of integers, e.g., 0, +1, 0, −1, −2, −3, −4,... or 0, +1, +2, +3, +4, +3, +4, +3, +4,...

EE 278: Random Processes Page 6–12 Example: xn 3 2 1 n 1 2 3 4 5 6 7 8 9 10 −1 −2 −3

zn : 1 1 1 −1 1 −1 −1 −1 −1 −1

• First-order pmf: The ﬁrst-order pmf is P{Xn = k} as a function of n. Note that k ∈ {−n, −(n − 2),..., −2, 0, +2,..., +(n − 2), +n} for n even k ∈ {−n, −(n − 2),..., −1, +1, +3,..., +(n − 2), +n} for n odd

Hence, P{Xn = k} = 0 if n + k is odd, or if k < −n or k > n

EE 278: Random Processes Page 6–13 Now for n + k even, let a be the number of +1’s in n steps, then the number of −1’s is n − a, and we ﬁnd that n + k k = a − (n − a) = 2a − n ⇒ a = 2 Thus 1 P{Xn = k} = P{2(n + k) heads in n independent coin tosses}

n −n = n+k · 2 for n + k even and −n ≤ k ≤ n 2

P{Xn = k}, n even

kk ... −6 −4 −2 0 +2 +4 +6 ...

EE 278: Random Processes Page 6–14 Markov Processes

• A discrete-time random process Xn is said to be a Markov process if the process future and past are conditionally independent given its present value • Mathematically this can be rephrased in several ways. For example, if the r.v.s {Xn : n ≥ 1} are discrete, then the process is Markov iﬀ n−1 n x pXn+1|X (xn+1|xn, )= pXn+1|Xn(xn+1|xn) for every n • IID processes are Markov • The random walk process is Markov. To see this consider n n n n P{Xn+1 = xn+1 | X = x } = P{Xn + Zn+1 = xn+1 | X = x }

= P{Xn + Zn+1 = xn+1 | Xn = xn}

= P{Xn+1 = xn+1 | Xn = xn}

EE 278: Random Processes Page 6–15 Independent Increment Processes

• A discrete-time random process {Xn : n ≥ 0} is said to be independent increment if the increment random variables

Xn1,Xn2 − Xn1,...,Xnk − Xnk−1

are independent for all sequences of indices such that n1 < n2 < ··· < nk • Example: Random walk is an independent increment process because

n1 n2 nk

Xn1 = Zi ,Xn2−Xn1 = Zi ,...,Xnk−Xnk−1 = Zi i=1 i=n +1 i=n +1 X X1 Xk−1 are independent because they are functions of independent random vectors • The independent increment property makes it easy to ﬁnd the n-th order pmfs of a random walk process from knowledge only of the ﬁrst-order pmf

EE 278: Random Processes Page 6–16 • Example: Find P{X5 = 3,X10 = 6,X20 = 10} for random walk process {Xn}

Solution: We use the independent increment property as follows

P{X5 = 3,X10 = 6,X20 = 10} = P{X5 = 3,X10 − X5 = 3,X20 − X10 = 4}

= P{X5 = 3}P{X5 = 3}P{X10 = 4} 5 5 10 = 2−5 2−5 2−10 = 3000 · 2−20 4 4 7 • In general if a process is independent increment, then it is also Markov. To see this let Xn be an independent increment process and deﬁne n T ∆X =[X1,X2 − X1,...,Xn − Xn−1] Then n n n n x X x pXn+1|X (xn+1 | ) = P{Xn+1 = xn+1 | = } n n = P{Xn+1 − Xn + Xn = xn+1 | ∆X = ∆x ,Xn = xn}

= P{Xn+1 = xn+1 | Xn = xn}

• The converse is not necessarily true, e.g., IID processes are Markov but not independent increment

EE 278: Random Processes Page 6–17 • The independent increment property can be extended to continuous-time processes:

A process X(t), t ≥ 0, is said to be independent increment if X(t1), X(t2) − X(t1),..., X(tk) − X(tk−1) are independent for every 0 ≤ t1 < t2 <...

A process X(t) is said to be Markov if X(tk+1) and (X(t1),...,X(tk−1)) are conditionally independent given X(tk) for every 0 ≤ t1 < t2 <...

EE 278: Random Processes Page 6–18 Counting Processes and Poisson Process

• A continuous-time random process N(t), t ≥ 0, is said to be a counting process if N(0) = 0 and N(t)= n, n ∈ {0, 1, 2,...}, is the number of events from 0 to t (hence N(t2) ≥ N(t1) for every t2 > t1 ≥ 0) Sample path of a counting process: n(t)

5 4 3 2 1 0 t 0 t1 t2 t3 t4 t5 t1, t2,... are the arrival times or the wait times of the events

t1, t2 − t1,... are the interarrival times of the events

EE 278: Random Processes Page 6–19 The events may be: ◦ Photon arrivals at an optical detector ◦ Packet arrivals at a router ◦ Student arrivals at a class • The Poisson process is a counting process in which the events are “independent of each other” • More precisely, N(t) is a Poisson process with rate (intensity) λ> 0 if: ◦ N(0) = 0 ◦ N(t) is independent increment

◦ (N(t2) − N(t1)) ∼ Poisson(λ(t2 − t1)) for all t2 > t1 ≥ 0 • To ﬁnd the kth order pmf, we use the independent increment property

P{N(t1)= n1, N(t2)= n2,...,N(tk)= nk}

= P{N(t1)= n1, N(t2) − N(t1)= n2 − n1,...,N(tk) − N(tk−1)= nk − nk−1}

= pN(t1)(n1)pN(t2)−N(t1)(n2 − n1) ...pN(tk)−N(tk−1)(nk − nk−1)

EE 278: Random Processes Page 6–20 • Example: Packets arrive at a router according to a Poisson process N(t) with rate λ. Assume the service time for each packet T ∼ Exp(β) is independent of N(t) and of each other. What is the probability that k packets arrive during a service time? • Merging: The sum of independent Poisson process is Poisson. This is a consequence of the inﬁnite divisibility of the Poisson r.v. • Branching: Let N(t) be a Poisson process with rate λ. We split N(t) into two counting subprocesses N1(t) and N2(t) such that N(t)= N1(t)+ N2(t) as follows:

Each event is randomly and independently assigned to process N1(t) with probability p, otherwise it is assigned to N2(t)

Then N1(t) is a Poisson process with rate pλ and N2(t) is a Poisson process with rate (1 − p)λ This can be generalized to splitting a Poisson process into more than two processes

EE 278: Random Processes Page 6–21 Related Processes

• Arrival time process: Let N(t) be Poisson with rate λ. The arrival time process Tn, n ≥ 0 is a discrete time process such that:

◦ T0 = 0

◦ Tn is the arrival time of the nth event of N(t) • Interarrival time process: Let N(t) be a Poisson process with rate λ. The interarrival time process is Xn = Tn − Tn−1 for n ≥ 1

• Xn is an IID process with Xn ∼ Exp(λ) n • Tn = i=1 Xi is an independent increment process with Tn − Tn ∼ Gamma(λ, n2 − n1) for n2 > n1 ≥ 1, i.e., 2 P 1 λn2−n1tn2−n1−1 f (t)= e−λt Tn2−Tn1 (n2 − n1 − 1)!

• Example: Let N1(t) and N2(t) be two independent Poisson processes with rates λ1 and λ2, respectively. What is the probability that N1(t) = 1 before N2(t) = 1?

EE 278: Random Processes Page 6–22 • Random telegraph process: A random telegraph process Y (t), t ≥ 0, assumes values of +1 and −1 with Y (0) = +1 with probability 1/2 and −1 with probability 1/2, and Y (t) changes polarities with each event of a Poisson process with rate λ> 0 Sample path: y(t)

t 0 x1 x2 x3 x4 x5 −1

EE 278: Random Processes Page 6–23 Mean and Autocorrelation Functions

• For a random vector X the ﬁrst and second order moments are ◦ mean µ = E(X)

◦ correlation matrix RX = E(XXT ) • For a random process X(t) the ﬁrst and second order moments are

◦ mean function: µX(t)=E(X(t)) for t ∈T

◦ autocorrelation function: RX(t1, t2) = E X(t1)X(t2) for t1, t2 ∈T • The autocovariance function of a random process is deﬁned as

CX(t1, t2) = E X(t1) − E(X(t1)) X(t2) − E(X(t2)) The autocovariance function can be expressed using the mean and autocorrelation functions as

CX(t1, t2)= RX(t1, t2) − µX(t1)µX(t2)

EE 278: Random Processes Page 6–24 Examples

• IID process:

µX(n)=E(X1) 2 E(X ) n1 = n2 R (n , n )=E(X X ) = 1 X 1 2 n1 n2 2 ((E(X1)) n1 6= n2 • Random phase signal process: 2π α µX(t)=E(α cos(ωt +Θ)) = cos(ωt + θ) dθ = 0 2π Z0 RX(t1, t2) = E X(t1)X(t2) 2π α2 = cos(ωt1 + θ)cos(ωt2 + θ) dθ 2π Z0 2π α2 = cos(ω(t1 + t2) + 2θ) + cos(ω(t1 − t2)) dθ 4π Z0 α2 = cos(ω(t1 − t2)) 2

EE 278: Random Processes Page 6–25 • Random walk: n n

µX(n) = E Zi = 0 = 0 i=1 i=1 X X RX(n1, n2)=E(Xn1Xn2)

=E[Xn1(Xn2 − Xn1 + Xn1)] 2 = E(Xn1)= n1 assuming n2 ≥ n1

= min{n1, n2} in general

• Poisson process:

µN (t)= λt

RN (t1, t2)=E(N(t1)N(t2))

=E[N(t1)(N(t2) − N(t1)+ N(t1))] 2 2 2 = λt1 × λ(t2 − t1)+ λt1 + λ t1 = λt1 + λ t1t2 assuming t2 ≥ t1 2 = λ min{t1, t2} + λ t1t2

EE 278: Random Processes Page 6–26 Gaussian Random Processes

• A Gaussian random process (GRP) is a random process X(t) such that T [X(t1),X(t2),...,X(tn)]

is a GRV for all t1, t2,...,tn ∈T • Since the joint pdf for a GRV is speciﬁed by its mean and covariance matrix, a GRP is speciﬁed by its mean µX(t) and autocorrelation RX(t1, t2) functions • Example: The discrete time WGN process is a GRP

EE 278: Random Processes Page 6–27 Gauss-Markov Process

• Let Zn, n ≥ 1, be a WGN process, i.e., an IID process with Z1 ∼ N (0, N) The Gauss-Markov process is a ﬁrst-order autoregressive process deﬁned by

X1 = Z1

Xn = αXn−1 + Zn , n> 1 , where |α| < 1

• This process is a GRP, since X1 = Z1 and Xk = αXk−1 + Zk where Z1,Z2,... are i.i.d. N (0, N),

X1 1 0 ··· 0 0 Z1 X2 α 1 ··· 0 0 Z2    . . . . .    X3 = ...... Z3  .  αn−2 αn−3 ··· 1 0   .        X  αn−1 αn−2 ··· α 1  Z   n    n is a linear transformation  of a GRV and is therefore a GRV   • Clearly, the Gauss-Markov process is Markov. It is not, however, an independent increment process

EE 278: Random Processes Page 6–28 • Mean and covariance functions:

µX(n)=E(Xn) = E(αXn−1 + Zn) n−1 = α E(Xn−1)+E(Zn) = α E(Xn−1)= α E(Z1) = 0

To ﬁnd the autocorrelation function, for n2 > n1 we write

n2−n1−1 n2−n1 i Xn2 = α Xn1 + α Zn2−i i=0 X Thus n2−n1 2 RX(n1, n2) = E(Xn1Xn2) = α E(Xn1) + 0 ,

since Xn1 and Zn2−i are independent, zero mean for 0 ≤ i ≤ n2 − n1 − 1 2 Next, to ﬁnd E(Xn1), consider 2 E(X1 )= N 2 2 2 2 E(Xn1) = E (αXn1−1 + Zn1) = α E(Xn1−1)+ N Thus

1 − α2n1 E(X2 )= N n1 1 − α2

EE 278: Random Processes Page 6–29 Finally the autocorrelation function is

1 − α2 min{n1,n2} R (n , n )= α|n2−n1| N X 1 2 1 − α2 • Estimation of Gauss-Markov process: Suppose we observe a noisy version of the Gauss-Markov process, Yn = Xn + Wn,

where Wn is a WGN process independent of Zn with average power Q i We can use the Kalman ﬁlter from Lecture Notes 4 to estimate Xi+1 from Y as follows: Initialization: ˆ X1|0 = 0 2 σ1|0 = N

Substituting from the ki equation into the MSE update equation, we obtain α2Qσ2 2 i|i−1 σi+1|i = 2 + N, σi|i−1 + Q This is a Riccati recursion (a quadratic recursion in the MSE) and has a steady state solution: α2Qσ2 σ2 = + N σ2 + Q Solving this quadratic equation, we obtain N − (1 − α2)Q + 4NQ +(N − (1 − α2)Q)2 σ2 = p 2

EE 278: Random Processes Page 6–31 The Kalman gain ki converges to −N − (1 − α2)Q + 4NQ +(N − (1 − α2)Q)2 k = p2αQ and the steady-state Kalman ﬁlter is ˆ ˆ ˆ Xi+1|i = αXi|i−1 + k(Yi − Xi|i−1)

EE 278: Random Processes Page 6–32