Discrete Simulation of Colored Noise and Stochastic Processes and llf" Power Law Noise Generation ~ ~ N. JEREMY USDIN, MEMBER, IEEE This paper discusses techniquesfor generating digital sequences [lo], [121-[14], [271, 1341, P71, [391, [411, [441, of noise which simulate processes with certain known properties [531, [541, [571, [581, [@I, 1681, [@I, [731-[751, [771, [791. or describing equations. Part I of the paper presents a review This paper discusses the theory and mechanics of gen- of stochastic processes and spectral estimation (with some new results) and a tutorial on simulating continuous noise processes erating digital, pseudorandom sequences on a computer with a known autospectral density or autocorrelation function. In as simulations of known stochastic processes. In other defining these techniques for computer generating sequences, it words, given the autospectral density or autocorrelation of also defines the necessary accuracy criteria. These methods are a process or a differential equation describing a system, compared to some of the common techniques for noise generation methods are presented for producing digital sequences with and the problems, or advantages, of each are discussed. Finally, Part I presents results on simulating stochastic differential equa- the "correct" discrete spectrum or correlations. The precise tions. A Runge-Kutta (RK) method is presented for numerically meaning of correct is discussed but can vary depending solving these equations. upon the specific application. Though many new results are Part I1 of the paper discusses power law, or 1/ f a, noises. Such reported, particularly in the area of nonlinear differential noise processes occur frequently in nature and, in many cases, with equation integration and power law noise simulation, this nonintegral values for a. A review of 1/ f noises in devices and systems is followed by a discussion of the most common continuous paper is also intended as a tutorial on colored noise analysis 1/ f noise models. The paper then presents a new dim1 model for and generation. Thus the detailed proofs or derivations are power law noises. This model allows for very accurate and efficient omitted or referred to cited works. computer generation of l/f" noises for any a. Many of the The paper is divided into two parts. The first discusses statistical properties of this model are discussed and compared to the previous continuous models. Lastly, a number of approximate the general problem of simulating stochastic processes on techniquesfor generating power law noises are presented for rapid a computer. Some time is spent defining what is meant by or real time simulation. a stochastic simulation and criteria are presented for eval- uating the effectiveness of a generation method. Some past I. INTRODUCTION techniques, particularly those involving direct simulation in the frequency domain from given spectra, are reviewed The need for accurate simulation of stochastic processes with an eye toward pointing out some of their deficiencies. and stochastic differential equations arises across almost Part I presents two techniques for simulating noise with a all disciplines of science and engineering. Mechanical and given spectra or autocorrelation function-a time domain aerospace engineers often simulate complex, nonlinear and frequency domain method. In addition, a RK algorithm models of dynamic systems acted upon by noise. In for simulating general, nonlinear, inhomogeneous stochastic electrical engineering and physics it is common to simulate differential equations is reviewed. various types of colored noises as models for sensors The second part focuses on the problem of generating and actuators. There are numerous nonlinear systems sequences of power law or l/f" noises (also often called arising in all areas of physics that require techniques for fractal processes). This is a remarkably ubiquitous prob- simulating stochastic differential equations. And the timing lem spanning innumerable fields, from music and to community has a long history of simulation for modeling art hydrology and low temperature physics. Countless devices phase and frequency noise in oscillators [2], [4], [5], [8], and systems have been seen to produce noises with an Manuscript received August 9, 1993; revised January 30, 1995. This autospectral density proportional to f-" with f the cyclic work was supported by NASA Contract NAS8-36125. frequency and a a real number between 0 and 2. Many The author is with the W.W. Hansen Experimental Physics Laboratory, models and techniques have been proposed in the past Relativity MissiodGravity Probe B, Stanford University, Stanford, CA 94305-4085 USA. to simulate these noises. Because the spectrum is not IEEE Log Number 9410317. rational, it is impossible to use standard linear system 0018-9219/95$04.00 0 1995 IEEE 802 PROCEEDINGS OF THE IEEE, VOL. 83, NO. 5, MAY 1995 - theory and differential equation models to simulate the In most cases, and all those in this paper, only the first noise via the techniques in Part I. Thus in Part 11, after and second order properties of the process are desired rather a review of past generation techniques, a new model is than the complete joint distribution function. For Gaussian proposed. This model, first proposed by Hosking [31] and processes, these second order properties completely define later independently by the author, can be used to generate the process distribution. These properties are defined via digital sequences of l/fa noise. The properties of this the following equations: model are discussed in some detail as well as the advantages it has as a generating mechanism over previous methods. In particular, it can be used over a frequency band of arbitrary J -00 size and it is strictly scale invariant. Because noise simulation spreads over so many diverse where q is the mean value of the process and the notation disciplines, it is difficult to find a notation agreeable to all E{ } refers to the expected value operation taken over the participants. This paper, therefore, uses common engineer- ensemble of processes {z(t,C)}. ing notation and methodology for the stochastic process The two-time autocorrelation of the process is given by and system theory discussions. This notation is consistent with that used in [9], [14], [33], [52], [62]. Discussions of discrete system theory and the a-transform can be found in [231, 1511, [521. PART I: STOCHASTIC PROCESS SIMULATION The autocovariance is defined by 11. REVIEW AND DEFINITIONS (Note that all processes in this paper are real). Before embarking on the detailed discussion of noise Thus, the variance of the process, a2(t),is given by simulation, it is useful to review some fundamentals of C(t,t). For this paper, all processes will be assumed zero stochastic process theory and noise measurements. This mean so that the autocorrelation and autocovariance are section presents background information and definitions as well as some new, and important, results in stochastic identical. process analysis and spectral measurement and estimation In general, the moments of a stochastic process defined in that will form the foundation for the simulation techniques (2) and (3) vary with time. However, a stationary process discussed in Section 111. The review material in this section is defined as one whose density function is invariant to follows the presentation in [52]. Further reading can be time shifts and thus independent of the times tl,tz, . ,t,. found in [9], [26], [62], [70], [74]. Much of the information Such a process is called strict-sense stationary. A wide- on spectral density measurements and estimation can also sense stationary process is one whose first and second order be found in [9]. properties only are independent of time, that is, q(t) = q, and R(t1,tz)= R(t1 - tz). A. Stochastic Processes For stationary processes the autocorrelation function def- It is assumed that the reader is familiar with the funda- inition is often written in the asymmetric form: mental axioms of probability and basic random variables. If we are given the probability space (R,7,F), then the stochastic process z(t)is defined in [52] as follows: "z(t,C) is a stochastic process when the random variable The variance of the process is then given by R(0). z represents the value of the outcome < of an experiment Since we will be dealing with many nonstationary and 7 for every time t," where R represents the sample space, transient processes, it is more convenient to use a symmetric 7 the event space or sigma algebra, and F the probability definition of the autocorrelation function: measure [26]. Normally, the dependence upon the event space, C, is omitted and the stochastic process is written as z(t).If the probability distribution function for z is given by F(z), For stationary processes, (6) and (5) give the same result. then we can define the general nth order, time-varying, However, when we later compute spectra and other prop- erties of nonstationary processes, (6) will be necessary in joint distribution function, F(z1,.. ,z,; tl,. , tn), for the random variables z(tl),. ,z(tn). The joint probability order to achieve consistent and accurate results. density function is then given by This distinction is important as we consider noise gen- eration. In simulation, we must be concerned with the transients of processes, thus implying that all signals under consideration for simulation will be nonstationary. How- ever, we can define an asymptotically stationary process as KASDIN DISCRETE SIMULATION OF COLORED NOISE AND STOCHASTIC PROCESSES 803 ~~ ~ - __ - one whose autocorrelation function satisfies: It is straightforward to show from this that Brown- ian motion is a nonstationary process with symmetric lim R(~,T)= R(T). (7) t+m autocorrelation: Many of the process models used for simulation will be asymptotically stationary. RB(t,r)= Q(1- 7). Lastly, it is useful to have a measure of the “memory” of a process.