APPENDIX I The Fourier, Laplace and Z-transforms This appendix provides a brief introduction to the Fourier transform which is a valuable mathematical tool in time-series analysis. Further details may be found in many books such as Hsu (1967) and Sneddon (1972). The related Laplace and Z-transforms are also introduced. Given a (possibly complex-valued) function h(t) of a real variable t, the Fourier transform of h(t) is usually defined as i w H(w) = J 00 h(t)e- tdt (Al.l) - 00 provided the integral exists for every real w. Note that H(w) is in general complex.A sufficient condition for H(w) to exist is Ih(t) Idt <oo. J:oo If (Al.I) is regarded as an integral equation for h(t) given H(w), then a simple inversion formula exists of the form I 00 h(t) = - f H(w)ei w tdw (Al.2) 21T - 00 and h(t) is called the inverse Fourier transform of H(w), or sometimes just the Fourier transform of H(w). The two 233 THE ANALYSIS OF TIME SERIES functions h(t) and H(w) are commonly called a Fourier transform pair. The reader is warned that many authors use a slightly different definition of a Fourier transform to Al.l. For example some authors put a constant l/y2rr outside the integral in (AI. 1) and then the inversion formula for h(t) is symmetric. In time-series analysis many authors (e.g. Cox and Miller, 1968, p. 315) put a constant 1/2rr outside the integral in Al.l. The inversion formula then has a unity constant outside the integral. Some authors in time-series analysis (e.g. Jenkins and Watts, 1968, Blackman and Tukey , 1959) define Fourier transforms in terms of the variable f= w/(2rr) rather than w . We then find that the Fourier transform pair is G(j) = f., 00 h(t)e- 211 iftdt (Al.3) h(t)= 1=00 G(j)e211 iftdf (AlA) Note that the constant outside each integral is now unity. In time-series analysis, we will often use the discrete form of the Fourier transform when h(t) is only defined for integer values of t. Then H(w) = ~ h(t)e- iw t (Al .5) t=- 00 is the discrete Fourier transform of h(t). Note that H(w) is only defined in the interval [-rr, n ]. The inverse transform is 1 r 11 h(t) =-J H(w)eiWtdw (Al .6) 2rr - 11 Fourier transforms have many useful properties, some of which are used during the later chapters of this book . 234 THE FOURIER, LAPLACE AND Z-TRANSFORMS However we will not attempt to review them here. The reader is referred for example to Hsu (1967). One special type of Fourier transform arises when h(t) is a real-valued even function such that h(t) =h( < t ). The autocorrelation function of a stationary time series has these properties. Then using (Al.l) with a constant l/rr outside the integral, we find 1 00 • H(w)=- J h(t)e-1wtdt rr - 00 2 00 =- J h(t) cos wt dt '(Al .7) rr 0 and it is clear that H( w) is a real-valued even function. The inversion formula is then 1 00 h(t) =- J H(w)e iwtdw I(Al .8) 2 _ 00 =J00 H(w)cos wt dw o Equations (AI.7) and (Al.8) are similar to a Fourier transform pair and are useful when we only wish to define H(w) for w> O. This pair of equations appear as equations (2.73) and (2.74) in Yaglom (1962). When h(t) is only defined for integer values of t, equations (Al.7) and (Al.8) become H(w) =; {h(O) + 2 I h(t) cos wt} (Al .9) h(t) = H(w) cos wt dw (AI.10) f'o and H(w) is now only defined on [0 , rr] . The Laplace transform of a function h(r) which is defined 235 THE ANALYSIS OF TIME SERIES for t > 0 is given by H(s) =1'''' h(t)e- st dt (Al.II) o where s is a complex variable. The integral converges when the real part of s exceeds some number called the abscissa of convergence. The relationship between the Fourier and Laplace transforms is of some interest, particularly as control engineers often prefer to use the Laplace transform when investigating the properties of a linear system (e.g. Solodovnikov, 1965, Douce, 1963) as this will cope with physically realizable systems which are stable or unstable. If the function h(t) is such that h(t) = 0 (t < 0) (Al.12) and the real part of s is zero, then the Laplace and Fourier transforms of h(t) are the same. The impulse response function of a physically realizable linear system satisfies (Al.12) and so for such functions the Fourier transform is a special case of the Laplace transform. More details about the Laplace transform may be found in many books, (e.g. Sneddon, 1972). The Z-transform of a function h(t) defined on the non-negative integers is given by H(z) = L h(t)z-t (Al.13) t=O In discrete time, with a function satisfying (AI .12), some authors prefer to use the Z-transform rather than the discrete form of the Fourier transform (Le. A1.5) or the discrete form of the Laplace transform, namely 00 H(s) = L h(t)e-st (Al.I4) t = 0 236 THE FOURIER, LAPLACE AND Z-TRANSFORMS Comparing (A 1.13) with (A 1.14) we see that z = e", The reader will observe that when {h(t)} is a probability function such that h(t) is the probability of observing the value t, for t = 0, I, ... , then (AI.13) is related to the probability generating function of the distribution, while (AI.5) and (A1.14) are related to the moment generating function. Exercises I. If h(t) is real, show that the real and imaginary parts of its Fourier transform, as defined by equation (A 1.1), are even and odd functions respectively. a 2. If h(t) = e- I t I for all real t, where a is a positive real constant, show that its Fourier transform, as defined by equation (ALI), is given by (_00 < w < 00 ) 3. Show that the Laplace transform of h(t) = e- at (t > 0) , where a is a real constant, is given by His) = I/(s + a) Re(s) > a. 237 APPENDIX II The Dirac Delta Function Suppose that ¢(t) is any function which is continuous at t = O. Then the Dirac delta function, o(t), is such that o(t)¢(t)dt = ¢(O) (A2.1) J:oo Because it is defined in terms of its integral properties alone, it is sometimes called the 'spotting' function since it picks out one particular value of ¢(t). It is also sometimes called simply the delta fun ction. It is important to realize that o(t) is not a fun ct ion. Rather it is a generalized function, or distribution, which maps a function into the real line. Some authors define the delta function by t=l=O o(t) = {~ (A2.2) t = 0 such that J:00 o(t)dt = 1. But while this is often intuitively helpful, it is mathematically meaningless. The Dirac delta function can also be regarded (e.g. Schwarz and Friedland, 1965) as the limit, as e ~ 0, of a pulse of width e and height lie (i.e. unit area) defined by o<t «. e u(t) = {~/e otherwise 238 THE DIRAC DELTA FUNCTION This definition is also not mathematically rigorous, but heuristically useful. In particular, control engineers can approximate such an impulse by an impulse with unit area whose duration is short compared with the least significant time constant of the response to the linear system being studied. Even though o(t) is a generalized function, it can often be handled as if it were an ordinary function except that we will be interested in the values of integrals involving o(t) and never in the value of o(t) by itself. The derivative o'(t) of o(t) can also be defined by J=00 0' (t)¢(t)dt = - ¢' (0) ,(A2.3) where ¢'(0) is the derivative of ¢(t) evalua ted at t =O. The justification for A2.3 depends on integrating by parts as if o'(t) and oct) were ordinary functions and using (A2 .2) to give f =00 o'(t)¢(t)dt = - f=00 8(t)¢'(t)dt and then using (A2 .1). Higher derivatives of au) may be defined in a similar way. The delta function has many useful properties (see, for example, Hsu, 1967). Exercises 1. The function ¢(t) is continuous at t = to . Ifa < b, show that fb su : to)¢(t)dt= {¢(to) a <to <b a 0 to <a, to > b. 2. The function ¢(t) is continuous at t = O. Show that ¢(t)o(t) =¢(O)o(t). 239 APPENDIX III Covariance Any reader who is unfamiliar with the laws of probability, independence, probability distributions, mean, variance and expectation, and elementary statistical inference should consult an elementary book on statistics, such as Freund, J. E. (1962), Mathematical Statistics, Prentice-Hall. The idea of covariance is particularly important in the study of time series, and will now be briefly revised. Suppose two random variables, X and Y, have means J.Lx, J.Ly, respectively. Then the covariance of X and Y is defined to be Cov(X, Y) = E{ (X - J.Lx )(Y - J.Ly)} .