EC505 STOCHASTIC PROCESSES Class Notes c 2011 Prof. D. Casta˜non & Prof. W. Clem Karl Dept. of Electrical and Computer Engineering Boston University College of Engineering 8 St. Mary’s Street Boston, MA 02215 Fall 2011 2 Contents 1 Introduction to Probability 11 1.1 AxiomsofProbability ............................... ......... 11 1.2 Conditional Probability and Independence of Events . 13 1.3 RandomVariables ..................................... ..... 13 1.4 Characterization of Random Variables . ........ 14 1.5 ImportantRandomVariables . ........ 19 1.5.1 Discrete-valued random variables . 19 1.5.2 Continuous-valued random variables . ..... 21 1.6 Functions of a Random Variable . ..... 25 1.6.1 Method of equivalent events . 25 1.6.2 Jacobianmethod.................................... 29 1.7 PairsofRandomVariables.............................. ........ 30 1.8 Conditional Probabilities, Densities, and Expectations . ........... 31 1.9 RandomVectors ....................................... 33 1.9.1 Functions of random vectors . 34 1.9.2 Expectations of functions of a random vector . 35 1.10 Properties of the Covariance Matrix . ........... 37 1.11 Gaussian Random Vectors . ....... 38 1.12 Inequalities for Random Variables . ........ 40 1.12.1 Markovinequality ................................ ...... 41 1.12.2 Chebyshev inequality . 41 1.12.3 Chernoff Inequality . 41 1.12.4 Jensen’s Inequality . 42 1.12.5 Moment Inequalities . ..... 42 2 Sequences of Random Variables 45 2.1 Convergence Concepts for Random Sequences . 45 2.2 The Central Limit Theorem and the Law of Large Numbers . ....... 49 2.3 Advanced Topics in Convergence . 51 2.4 Martingale Sequences . 54 2.5 Extensions of the Law of Large Numbers and the Central Limit Theorem . ........ 57 2.6 Spaces of Random Variables . 58 3 Stochastic Processes and their Characterization 61 3.1 Introduction....................................... ....... 61 3.2 Complete Characterization of Stochastic Processes . .......... 62 3.3 First and Second-Order Moments of Stochastic Processes . 62 3.4 Special Classes of Stochastic Processes . ......... 63 3.5 Properties of Stochastic Processes . ......... 66 3.6 Examples of Random Processes . ..... 67 4 CONTENTS 3.6.1 TheRandomWalk.................................... 67 3.6.2 ThePoissonProcess ................................ ..... 68 3.6.3 Digital Modulation: Phase-Shift Keying . 71 3.6.4 The Random Telegraph Process . 73 3.6.5 The Wiener Process and Brownian Motion . 74 3.7 Moment Functions of Vector Processes . ..... 75 3.8 Moments of Wide-sense Stationary Processes . ...... 76 3.9 Power Spectral Density of Wide-Sense Stationary Processes . ..... 77 4 Mean-Square Calculus for Stochastic Processes 81 4.1 Continuity of Stochastic Processes . .......... 81 4.2 Mean-Square Differentiation . ....... 83 4.3 Mean-Square Integration . ....... 85 4.4 Integration and Differentiation of Gaussian Stochastic Processes . ............... 89 4.5 Generalized Mean-Square Calculus . 89 4.6 Ergodicity of Stationary Random Processes . ........ 93 5 Linear Systems and Stochastic Processes 99 5.1 Introduction....................................... ....... 99 5.2 Review of Continuous-time Linear Systems . ....... 99 5.3 Review of Discrete-time Linear Systems . 102 5.4 Extensions to Multivariable Systems . ......... 104 5.5 Second-order Statistics for Vector-Valued Wide-Sense Stationary Processes . 104 5.6 Continuous-time Linear Systems with Random Inputs . 106 6 Sampling of Stochastic Processes 111 6.1 The Sampling Theorem . 111 7 Model Identification for Discrete-Time Processes 117 7.1 Autoregressive Models . ....... 117 7.2 MovingAverageModels ................................. 120 7.3 Autoregressive Moving Average (ARMA) Models . ......... 121 7.4 Dealing with non-zero mean processes . 122 8 Detection Theory 123 8.1 Bayesian Binary Hypothesis Testing . ......... 124 8.1.1 Bayes Risk Approach and the Likelihood Ratio Test . ....... 125 8.1.2 SpecialCases...................................... 127 8.1.3 Examples ........................................ 129 8.2 Performance and the Receiver Operating Characteristic . 131 8.2.1 PropertiesoftheROC................................ 135 8.2.2 Detection Based on Discrete-Valued Random Variables . 138 8.3 Other Threshold Strategies . 142 8.3.1 Minimax Hypothesis Testing . 142 8.3.2 Neyman-Pearson Hypothesis Testing . 144 8.4 M-ary Hypothesis Testing . ....... 145 8.4.1 SpecialCases...................................... 147 8.4.2 Examples ........................................ 148 8.4.3 M-Ary Performance Calculations . 150 8.5 GaussianExamples................................... ....... 153 CONTENTS 5 9 Series Expansions and Detection of Stochastic Processes 155 9.1 Deterministic Functions . 155 9.2 Series Expansion of Stochastic Processes . ........ 156 9.3 Detection of Known Signals in Additive White Noise . ........ 160 9.4 Detection of Unknown Signals in White Noise . ........ 162 9.5 Detection of Known Signals in Colored Noise . ........ 163 10 Estimation of Parameters 165 10.1Introduction...................................... ........ 165 10.2 General Bayesian Estimation . .......... 166 10.2.1 General Bayes Decision Rule . 166 10.2.2 General Bayes Decision Rule Performance . 167 10.3 Bayes Least Square Estimation . .......... 168 10.4 Bayes Maximum A Posteriori (MAP) Estimation . ............. 174 10.5 Bayes Linear Least Square (LLSE) Estimation . .......... 180 10.6 Nonrandom Parameter Estimation . ......... 188 10.6.1 Cramer-RaoBound .................................. 189 10.6.2 Maximum-Likelihood Estimation . ......... 193 10.6.3 Comparison to MAP estimation . ........ 195 11 LLSE Estimation of Stochastic Processes and Wiener Filtering 197 11.1Introduction...................................... ........ 197 11.2HistoricalContext ................................ .......... 198 11.3 LLSE Problem Solution: The Wiener-Hopf Equation . 199 11.4 Wiener Filtering . 201 11.4.1 Noncausal Wiener Filtering (Wiener Smoothing) . 201 11.4.2 Causal Wiener Filtering . 205 11.4.3 Summary ........................................ 219 12 Recursive LLSE: The Kalman Filter 221 12.1Introduction...................................... ........ 221 12.2HistoricalContext ................................ .......... 221 12.3 Recursive Estimation of a Random Vector . ......... 222 12.4 The Discrete-Time Kalman Filter . ........ 225 12.4.1 Initialization ................................. ........ 226 12.4.2 Measurement Update Step . 226 12.4.3 Prediction Step . 227 12.4.4 Summary ........................................ 227 12.4.5 AdditionalPoints................................ ....... 228 12.4.6 Example....................................... 229 12.4.7 Comparison of the Wiener and Kalman Filter . 232 13 Discrete State Markov Processes 233 13.1 Discrete-time, Discrete Valued Markov Processes . ......... 233 13.2 Continuous-Time, Discrete Valued Markov Processes . .......... 239 13.3 Birth-Death Processes . ....... 243 13.4 Queuing Systems . 245 13.5 Inhomogeneous Poisson Processes . ....... 247 13.6 Applications of Poisson Processes . .......... 250 A Useful Transforms 253 6 CONTENTS B Partial-Fraction Expansions 259 B.1 Continuous-Time Signals . ........ 259 B.2 Discrete-Time Signals . 260 C Summary of Linear Algebra 263 C.1 Vectors and Matrices . 263 C.2 Matrix Inverses and Determinants . ........ 266 C.3 Eigenvalues and Eigenvectors . ....... 268 C.4 Similarity Transformation . ............ 269 C.5 Positive-Definite Matrices . ....... 270 C.6 Subspaces . 271 C.7 Vector Calculus . 272 D The non-zero mean case 275 List of Figures 1.1 Illustration of method of equivalent events . ........... 27 1.2 Example of method of equivalent events . ....... 28 1.3 Example of Jacobian method . ...... 29 3.1 Interarrival Times τk. ........................................ 68 3.2 Arrival times T (n) and interarrival times τk............................. 69 3.3 The Poisson Counting.