COMPUTING ASPECTS OF PROBLEMS IN NON.LINEAR PREDICTION AND FILTERING by I. G. CUMMING A thesis submitted for the degree of Ph.D. in Engineering Centre for Computing and Automation, Imperial College, University of London, London, S.W.7. MAY 1967 _ 2 - ABSTRACT This thesis discusses some of the computational problems arising in the application of modern stochastic control theory. We deal with continuous time systems where two typical problems are the prediction of the future statistical behaviour of systems and the synthesis of systems designed from stochastic theory such as filters. In each case, computational problems occur if the system is non-linear or non-Gaussian. The non-linear prediction problem involves solving a parabolic partial differential equation, the Fokker-Planck equation, and we dis- cuss two numerical methods of solving this equation. Finding that these methods can only handle a restricted class of low-dimensional systems, we study Monte Carlo methods in the hopes of finding a more general solution procedure. We find these to be successful if we allow for accuracy limitations, and find that the theory of the Fokker- Planck equation can be extended to include the Monte Carlo solution of a wider class of parabolic equations than previous methods would accommodate. Monte Carlo methods involve the simulation of a stochastic system on a computer, and as the problem of synthesising a given system is the same as simulating it on a computer, the rest of the thesis centres around the theoretical and practical aspects of simulation techniques. We find in each case that the system we must simulate is a continuous Markov (diffusion) process, and that diffusion processes have properties which distinguish them from any process which can ,be constructed on a computer or in the physical world (we call these physical processes). Thus we discuss the statistical equivalence of physical and diffusion processes, and show how and under what conditions a physical process can be chosen to approximate a diffusion process, and vice versa. In this way, we clarify a controversy on the interpretation of limiting forms of physical processes, and give an example which confirms the accuracy of the approximation and illustrates that diffusion approxima- tions can provide a useful method of analysing the transient statistics of physical processes. On the practical side of the simulation problem, we discuss the choice of noise source and its proper characterisation on the analogue computer, and the convergence rates and efficiencies of discrete simulation formulae on the digital computer. Acknowledgements This thesis has resulted from research carried out in the Department of Electrical Engineering*, Imperial College, from October 1963 to November 1966. The author is deeply indebted to his supervisor, J. H. Westcott, Professor of Control Systems, for providing the facilities for research, and considerable personal encouragement during this period. Discussions with colleagues were helpful in clarifying the aims and certain specific points of the project, and the author would particularly like to thank J. M. C: Clark, G. S. Marliss and D. Q. Mayne in this regard. The financial support of the Ministry of Education, Province of Quebec (1963/64), and the National Research Council of Canada (NATO Scholarship 1964/65 and 1965/66) is gratefully acknowledged. Finsfly, the author is very thankful for the ever-willing help of Miss E. Stocker (typing), Miss M. Lancaster (diagrams) and hiss L. Hawkins (duplicating). * The Control Systems Group is now in the Centre for Computing and Automation - 4 - CONTENTS PAGE Abstract 2 Acknowledgements 3 Glossary of Symbols 1. INTRODUCTION 13 1.1 Scope 13 1.2 Preliminaries - the Fokker-Planck Equation 14 1.3 Outline of Thesis 19 2. THE DIRECT APPROACH TO PREDICTION PROBLEMS - SOLUTION OF THE FOKKER-PLANCK EQUATION 25 2.1 Fokker-Planck Equation for a Diffusion Process 25 2.2 Fokker-Planck Equation for a Non-Markovian Process 27 2.3 Numerical Solution by Finite Differences 38 1. Example of a Noisy Control System 39 2. Choice of Finite Difference Model 42 3. Solution Procedure and Boundary Conditions 45 4. An Application of the'Transient Solution 56 5. Solution for Higher Order Systems 65 6. Summary of Finite Difference Methods 69 2.4 Numerical Solution by Hermite Transforms 71 1. Characterization of a Random Process 71 2. Hermite Polynomial Expansions 73 3. Hermite Transformation of the FP Equation 83 4. Hermite Transformation of Normalized FP Equation 91 5. Summary of Hermite Transform Method 99 3. SIMULATION AND THE MONTE CARLO SOLUTION OF PARABOLIC EQUATIONS 103 3.1 Motivation for Simulation Techniques 103 3.2 The Monte Carlo Solution of Parabolic Equations 110 1. The FP Equation as an Equation of Conser- vation 112 2. Simulation of Parabolic Equations not of FP Form 129 3. Solution Procedure 137 4. Example: The Heat Conduction Equation 145 5. The Treatment of Spatial Discontinuities and Boundary Conditions 150 6. Numerical Results of the Heat Conduction Example 162 7. Summary of Monte Carlo Solutions 180 -5- 4. THE RELATION BETWEEN PHYSICAL AND DIFFUSION PROCESSES WITH APPLICATIONS TO SIMULATION 184 4.1 Choosing an Equivalent Diffusion Process for a Physical Process by Matching Finite Incremental Statistics 187 1. Derivation of E[oX iXtt] for a Physical Process c T i 189 2. Derivation of E[SXoX X,t] for a Physical Process 197 3. A Diffusion Model for the Physical Process 200 4. Experimental Results 203 4.2 A Limiting Form of a Physical Process 207 4.3 Applications to Linear Systems with Random Coefficients 219 4.4 The Simulation of Diffusion Processes 228 4.5 The Simulation,of Physical Processes 233 5. ANALOGUE SIMULATION 237 5.1 Some Useful Noise Sources and their Characteristic Matrices 238 1. Noises Generated by Linear Shaping Filters 238 2. Pseudo Random Sequences 244 5.2 Experimental Results Illustrating the Differing Biases of Physical Processes and Diffusion Processes 249 1. Example Illustrating the Scaling of an Independent Noise Source 250 2. Construction of a Non-Linear Filter 257 3. Example using an Asymmetrical Noise Source 263 6. DIGITAL SIMULATION 274 6.1 Digital Noise and the Form of the Ordinary Differential Equation 274 6.2 Digital Solution of the 0.D.E. and Convergence to the S.D.E. 280 1. Convergence of the Physical Process to the Diffusion Process 281 2. Discrete Approximation to the O.D.E. 286 6.3 Digital Data Smoothing by Orthogonal Functions 305 1. Orthogonal Polynomial Expansions 306 2. Data Smoothing by Finite Expansions 309 7. CONCLUSIONS 317 Appendix A The Stochastic Calculus 327 Appendix B The Normalised FP Equation 346 Appendix C Calculation of Flux Hitting Boundary in Heat Conduction Problem 353 Appendix D A White Noise Model of a Pseudo Random Binary Sequence 356 REFERENCES 396 - 6 - List of Figures 2.3.1 First Order Non-Linear Regulator with Disturbances 2.3.2 Variation of Limiting Boundary Condition Es with h 2.3.3 Variation of a Parameter of Gk Series with h 2.3.4 Transient Solution from a Delta Function 2.3.5 Time Solution of FP Equation, Noiseless Zero Measurement 2.3.6 Time Solution of FP Equation, Noisy Non-zero Measurement 2.3.7 Growth of System Uncertainty with Time (Output Variance) 2.3.8 Probability that System Exceeds Tolerance Band of t 2 2.3.9 Time Until Pr( (XI > 2) Exceeds 5% 2.4.1 Transient Solution of FP Equation by Hermite Transforms 3.2.1 Continuous Trajectories of a Markov Process 3.2.2 Solution of Heat Conduction Equation Across a Discontinuity 3.2.3 Imposed Boundary Conditions for K1 Discontinuity 3.2.4 Heat Conduction Example with Material Discontinuity 3.2.5 Behaviour of Trajectory near Material Boundary 3.2.6 Transient Solution of Heat Conduction Example 3.2.7 Average 'Temperature in Steel Region 4.1.1 Accuracy of Diffusion Model of Filtered PRBS 5.1.1 Generation of an Arbitrary Non-Stationary Noise Source 5.1.2 An Asymmetrical Noise Source 5.2.1 Analogue Simulation of Equation (5.2.2) 5.2.2 Analogue Simulation of Non-Linear Filter of Equation (5.2.23) 5.2.3 Analogue Simulation of Non-Linear Filter of Equation (5.2.24) 5.2.4 Analogue Simulation of Equation (5.2.29) 5.2.5 Experimental Determination of Characteristic Matrix 5.2.6 Normalised Correlation Function of Two Dimensional Noise Source 6.1.1 Correlation Function of Piecewise Constant Noise 6.2.1 Steps in the Convergence of a Digital Computer Simulation to a Diffusion Process 6.2.2 Absolute Sample Path Error vs. At 6.2.3 Accuracy of Estimated Distribution for Fixed Number of Samples 6.2.4 Sample Path Accuracy of Euler and Runge-Kutta Formulae Applied to Stochastic Equations 6.2.5 Distributions obtained from Correct and Incorrect Simulations of 6.3.1 RMS Value of Expansion Coefficients Eqn. 6.2.4. 6.3.2 Error in Density as a Function of Size of Expansion Coefficients 6.3.3 Monte Carlo Solution of Figure 2.3.5 Example - 7 - Glossary of Princizal Symbols A(t), A*(t) characteristic matrices of physical noise process (4.1.21, 30) a(x,t) second incremental moment of diffusion process (1.2.2) a coefficient of linear system (6.1.4) B(t) noise scaling matrix B BT = A + A* (4.1.39) ti b(x,t) first incremer_tal moment of diffusion process (1.2.2) b. coefficient of linear system (4.3.1) b coefficient of linear system (6.1.4) C noise scaling matrix (4.4.6) c(t) forcing function of linear system (4.3.1) c piecewise constant random coefficient (6.1.6), (6.2.13) 2D(t) intensity matrix of noise process = A + A* d.