STATE SPACE AND UNOBSERVED COMPONENT MODELS
Theory and Applications
This volume offers a broad overview of the state-of-the-art developments in the theory and applications of state space modelling. With fourteen chap- ters from twenty three contributors, it offers a unique synthesis of state space methods and unobserved component models that are important in a wide range of subjects, including economics, finance, environmental science, medicine and engineering. The bookis divided into four sections: introduc- tory papers, testing, Bayesian inference and the bootstrap, and applications. It will give those unfamiliar with state space models a flavour of the work being carried out as well as providing experts with valuable state-of-the- art summaries of different topics. Offering a useful reference for all, this accessible volume makes a significant contribution to the advancement of time series analysis.
Andrew Harvey is Professor of Econometrics and Fellow of Corpus Christi College, University of Cambridge. He is the author of The Economet- ric Analysis of Time Series (1981), Time Series Models (1981) and Fore- casting, Structural Time Series Models and the Kalman Filter (1989).
Siem Jan Koopman is Professor of Econometrics at the Free University Amsterdam and Research Fellow of Tinbergen Institute, Amsterdam. He has published in international journals and is coauthor of Time Series Anaysis by State Space Models (with J. Durbin, 2001).
Neil Shephard is Professor of Economics and Official Fellow, Nuffield College, Oxford University.
STATE SPACE AND UNOBSERVED COMPONENT MODELS Theory and Applications
Edited by
Andrew Harvey Faculty of Economics and Politics, University of Cambridge
Siem Jan Koopman Department of Econometrics and Operations Research, Free University Amsterdam
Neil Shephard Nuffield College, University of Oxford published by the press syndicate of the university of cambridge The Pitt Building, Trumpington Street, Cambridge, United Kingdom cambridge university press The Edinburgh Building, Cambridge CB2 2RU, UK 40 West 20th Street, NewYork, NY 10011–4211, USA 477 Williamstown Road, Port Melbourne, VIC 3207, Australia Ruiz de Alarc´on13, 28014 Madrid, Spain Dock House, The Waterfront, Cape Town 8001, South Africa http://www.cambridge.org
C Cambridge University Press 2004
This book is in copyright. Subject to statutory ception and to the provisions of relevant collective licensing agreements, no reproduction of any part may take place without the written permission of Cambridge University Press.
First published 2004
Printed in the United Kingdom at the University Press, Cambridge
Typeface Computer Modern 11/14 pt. System LATEX2ε [tb]
A catalogue record for this book is available from the British Library
Library of Congress Cataloguing in Publication data State space and unobserved component models : theory and applications : proceedings of a conference in honour of James Durbin / edited by AndrewC. Harvey, Siem Jan Koopman, Neil Shephard. p. cm. Includes bibliographical references and indes. ISBN 0 521 83595 X 1. State-space methods–Congresses. 2. System analysis–Congresses. I. Durbin, J. (James), 1923– II. Harvey, A. C. (AndrewC.) III. Koopman, S. J. (Siem Jan) IV. Shephard, Neil. QA402.S835 2004 2003063508
ISBN 0 521 83595 X hardback
The publisher has used its best endeavours to ensure that the URLs for ternal websites referred to in this book are correct and active at the time of going to press. However, the publisher has no responsibility for the websites and can make no guarantee that a site will remain live or that the content is or will remain appropriate. Contents
Preface page vii Acknowledgements xiv Part I State space models 1 1 Introduction to state space time series analysis James Durbin 3 2 State structure, decision making and related issues Peter Whittle 26 3 An introduction to particle filters Simon Maskell 40 Part II Testing 73 4 Frequency domain and wavelet-based estimation for long-memory signal plus noise models Katsuto Tanaka 75 5 A goodness-of-fit test for AR(1) models and power against state space alternatives T.W. Anderson and Michael A. Stephens 92 6 Tests for cycles Andrew C. Harvey 102 Part III Bayesian inference and bootstrap 121 7 Efficient Bayesian parameter estimation Sylvia Fr¨uhwirth-Schnatter 123 8 Empirical Bayesian inference in a nonparametric regression model Gary Koop and Dale Poirier 152 9 Resampling in state space models David S. Stoffer and Kent D. Wall 171 Part IV Applications 203 10 Measuring and forecasting financial variability using realised variance Ole E. Barndorff-Nielsen, Bent Nielsen, Neil Shephard and Carla Ysusi 205 11 Practical filtering for stochastic volatility models Jonathon R. Stroud, Nicholas G. Polson and Peter M¨uller 236
v vi Contents 12 On RegComponent time series models and their applications William R. Bell 248 13 State space modelling in macroeconomics and finance using SsfPackin S+Finmetrics Eric Zivot, Jeffrey Wang and Siem Jan Koopman 284 14 Finding genes in the human genome with hidden Markov models Richard Durbin 336 References 351 Author index 373 Subject index 377 Preface
State space methods and unobserved component models are important in a wide range of subjects, including economics, finance, environmental science, medicine and engineering. The conference ‘State space and unobserved com- ponent models’, part of the Academy Colloquium programme of the Royal Netherlands Academy of Arts and Sciences, held in Amsterdam from 29 August to 3 September, 2002 brought together researchers from many different areas, but all pursuing a common statistical theme. The papers selected for this volume will give people unfamiliar with state space models a flavour of the workbeing carried out as well as providing experts with valuable state-of-the-art summaries of different topics. The conference on state space methods afforded an ideal opportunity to honour Jim Durbin. Jim has been an active researcher in statistics for over fifty years. His first paper, published in 1950, set out the theory of what was to become known as the Durbin–Watson test. He subsequently published in many other areas of statistics, including sampling theory and regression, but over the last fifteen years or so his attention has again been focussed on time series, and in particular on state space models. A steady stream of workhas appeared, beginning with the study of the British seat belt law with Andrew Harvey and culminating in the bookwith Siem Jan Koopman. It is entirely fitting that the first article in the volume should be by Jim. His clear and lucid style has been an inspiration to generations of students at the London School of Economics – including all three editors of this book– and his paper provides an ideal introduction to unobserved components models. We write some words about Jim’s career at the end of this Preface. The introductory section of the bookhas two other papers. The first, by Peter Whittle, explores various aspects of the structure of state space models, particularly as they pertain to decision making, from a control engineering perspective. The second, by Simon Maskell, is an introduction
vii viii Preface to the use of particle filtering for readers who are familiar with Kalman filtering. Particle filters have proved of enormous value in dealing with non- linear state space models. The potential for using particle filters in areas such as economics and finance is well illustrated by the contribution made by Jonathan Stroud, Nicholas Polson and Peter M¨ullerin the applications section of the book. The second section deals with testing. Katsuto Tanaka’s article is on the use of wavelet-based methods in the estimation of long memory models with additive noise. Like particle filters, wavelets have had significant impact on the engineering literature and it is interesting to see whether they are able to help in the statistical treatment of an unobserved component model. Andrew Harvey’s article brings together various tests concerning cycles. Many of the test statistics are linked by the Cram´er–von Mises distribution, a distribution that also plays an important role in the paper by Ted Anderson and Michael Stephens. They examine some goodness-of-fit statistics, a field to which Jim Durbin has made major contributions. The third section contains two papers on Bayesian inference and one on the bootstrap. Sylvia Fr¨uhwirthchnatter provides the first systematic treatment of the effect of parameterising state space models on the effective- ness of Markov chain Monte Carlo algorithms. The econometricians Gary Koop and Dale Poirier carefully study a Bayesian treatment of a spline model, focusing on the effect of priors on hyperparameters and initial values of states. David Stoffer and Kent Wall provide a survey of the recent work on the use of bootstrap to provide accurate classical inference on state space models. The last section contains applications. The papers by Ole Barndorff- Nielsen, Bent Nielsen, Neil Shephard and Carla Ysusi and by Stroud et al. both deal with stochastic volatility, a key area in financial econometrics. Eric Zivot, Jeffrey Wang and Siem Jan Koopman provide details of a port of SsfPack to S-PLUS as part of the S+FinMetrics module. SsfPack is a library of flexible functions for routine Kalman filtering, smoothing and simulation smoothing for general Gaussian, linear state space models. They are sufficiently flexible that they can also be used as components in the analysis of various non-Gaussian and nonlinear models when combined with importance sampling or Markov chain Monte Carlo techniques. The authors illustrate some of the possibilities with applications of state space meth- ods in finance and macroeconomics. William Bell’s paper is on regression component models, with particular emphasis on applications within the US Bureau of the Census. Finally, Richard Durbin, Jim’s son, shows how hidden Markov chains can be used to find genes in the human genome. Preface ix About the conference The editors of this volume, with the support of the Royal Netherlands Academy of Arts and Sciences, organised the conference ‘State space and unobserved component models’ at the end of summer 2002 (29 August–3 September). The venue was ‘Het Trippenhuis’ in the centre of Amsterdam. The conference focused on the following topics: • modern computing tools for nonlinear and non-Gaussian models; • unobserved components: estimation and testing; • signal extraction and dynamic factor models; • methods for official statistics; • applications in areas like biostatistics, economics, engineering and finance. The first part of the conference (the Academy Colloquium, 29–31 August) was designed as a workshop for about fifty invited researchers in the field. The participants and the programme of the Colloquium are listed below. The second part was the Masterclass that was attended by an international audience of more than 100 researchers and Ph.D. students. The programme of the masterclass is also presented. We were delighted that many participated in this conference by presenting a paper on a topic related to the overall theme of the conference or by discussing papers or by attending the conference. A selection of the papers
Conference participants
T.W. Anderson, Stanford University Richard T. Baillie, Michigan State University William R. Bell, Statistical Research Division, US Bureau of Census Charles S. Bos, Free University Amsterdam Fabio Busetti, Bank of Italy, Rome Richard A. Davis, Colorado State University Herman van Dijk, Erasmus University Rotterdam Arnaud Doucet, University of Cambridge Catherine Doz, University of Cergy-Pontoise, France J. Durbin, London School of Economics Richard M. Durbin, Sanger Centre, Cambridge Cristiano A.C. Fernandes, Pontificada Universidad Catolica, Rio de Janeiro David F. Findley, Statistical Research Division, US Bureau of Census Sylvia Fr¨uhwirth-Schnatter, Johannes Kepler University, Linz Andrew C. Harvey, University of Cambridge Richard H. Jones, University of Colorado Genshiro Kitagawa, Institute of Statistics and Mathematics, Tokyo Gary Koop, University of Leicester x Preface
Siem Jan Koopman, Free University, Amsterdam Hans Rudolf K¨unsch, ETH Zurich Kai Ming Lee, Tinbergen Institute Amsterdam RobE. Luginbuhl, Free University Amsterdam Jan R. Magnus, CentER for Economic Research, Tilburg University Simon Maskell, QinetiQ, Malvern Brendan McCabe, University of Liverpool Filippo Moauro, National Institute of Statistics, Italy Roderick Molenaar, ABP Investments and Research Charles Nelson, University of Washington, Seattle Jukka Nyblom, Stockholm School of Economics Marius Ooms, Free University Amsterdam Keith Ord, University of Georgetown, Washington DC Daniel Pe˜na, Carlos III University, Madrid Danny Pfeffermann, Hebrew University, Tel Aviv Mike K. Pitt, Warwick University, Coventry Tommaso Proietti, Udine University Thomas Rothenberg, University of California, Berkeley Neil Shephard, Nuffield College, University of Oxford Robert H. Shumway, University of California Pieter Jelle van der Sluis, ABP Investments and Research David Stoffer, University of Pittsburgh Jonathan R. Stroud, University of Chicago Katsuto Tanaka, Hitosubashi University A.M. Robert Taylor, University of Birmingham Richard Tiller, US Bureau of Labour Statistics Pedro Valls, IBMEC Business School, Sao Paulo Aart de Vos, Free University Amsterdam Peter Whittle, University of Cambridge Peter C. Young, University of Lancaster Kenneth F. Wallis, Warwick University, Coventry Eric Zivot, University of Washington, Seattle
Papers presented at the conference
T.W. Anderson, A goodness-of-fit test for AR(1) models and power against state space alternatives Richard T. Baillie, A high frequency perspective on the forward premium anomaly William R. Bell, On some applications of RegComponent time series models Richard A. Davis, Observation driven models for Poisson counts Arnaud Doucet, Optimisation of particle methods using stochastic approximation Preface xi
Richard M. Durbin, Finding genes in the human genome with hidden Markov models Sylvia Fr¨uhwirth-Schnatter, Efficient Bayesian parameter estimation for state space models based on reparametrisations Andrew C. Harvey, Testing for cycles Richard H. Jones, Binary longitudinal data Genshiro Kitagawa, State space modelling for signal traction problems in seismology Gary Koop, Empirical Bayesian inference in nonparametric model Siem Jan Koopman, Modelling economic convergence using unobserved converging components Hans Rudolf K¨unsch, Variations on the particle filter Charles Nelson, The structural break in the equity premium Keith Ord, The single source of error specification for state space models: an appraisal Daniel Pe˜na, Dimension reduction in multivariate time series Danny Pfeffermann, State space modelling with correlated measurement errors with application to small area estimation under benchmark constraints Neil Shephard, High frequency financial econometrics: tracting information from realised variances Robert H. Shumway, Dynamic med models for merging multiple time series fragments David Stoffer, Resampling in state space models Jonathan Stroud, Practical filtering with parameter learning Katsuto Tanaka, Wavelet methods for inference problems associated with long-memory signal plus noise models A.M.Robert Taylor, Variance shifts, structural breaks and stationarity tests Peter Whittle, State structure, decision making and related issues Peter C. Young, Data-based mechanistic modelling and state dependent parameter models
Masterclass
Richard A. Davis, The innovations algorithm and parameter driven models Jim Durbin, Introduction to state space models Simon Maskell, Monte Carlo filtering Jukka Nyblom, Testing for unobserved components models Keith Ord, A new look at models for ponential smoothing Richard Tiller, Application of state space modeling for official labor force statistics in the USA Eric Zivot, State space modelling in macroeconomics and finance using S+FinMetrics xii Preface is included in this volume. We would particularly like to thank those scholars who helped us to referee these papers.
About Professor James Durbin
James Durbin was born 1923 in England. He was educated at St John’s Col- lege, Cambridge. From 1950 he worked at the London School of Economics and Political Science (LSE) until his retirement in 1988; he remains an Emer- itus Professor of Statistics there. He visited research institutions in many corners of the world during his period at the LSE and after his retirement. His research areas include serial correlation (13 publications), time series (>30), sample survey methodology (9), goodness-of-fit tests and sample dis- tribution functions (13) and probability and statistical theory (20). He was deputy editor of Biometrika (1962–4) and associate editor of Biometrika (1960–2, 1964–7), Annals of Statistics (1973–5) and Journal of the Royal Statistical Society, Series B (1978–81). He was the President of the Royal Statistical Society for the period 1986– 7 and the President of the International Statistical Institute for the period of 1983–5. Further, he is a Fellow of the Econometric Society (since 1967), the Institute of Mathematical Statistics (since 1958) and the American Statistical Society (since 1960). In 2001 he became a Fellow of The British Academy. More details about the career of Professor Durbin can be found Econo- metric Theory (1988, volume 4, issued) in which the interview with Professor J. Durbin by Professor Peter C. B. Phillips was published. His publications between 1950 and 1988 are listed at the end of the interview. Below we report his publications from 1988 onwards.
Publications of Professor James Durbin since 1988
(i) Statistics and statistical science (presidential address). Journal of the Royal Statistical Society, Series A, 150, 177–191, 1988. (ii) Is a philosophical consensus for statistics attainable? Journal of Econ- ometrics, 37, 51–61, 1988. (iii) Maximum likelihood estimation of the parameters of a system of simultaneous regression equations. Econometric Theory, 4, 159–170, 1988. (iv) A reconciliation of two different expressions for the first passage den- sity of Brownian motion to a curved boundary. Journal of Applied Probability, 25, 829–832, 1988. Preface xiii (v) First passage densities of Gaussian and point processes to general boundaries with special reference to Kolmogorov–Smirnov tests when parameters are estimated. Proceedings of First World Conference of the Bernoulli Society, Tashkent, USSR, 1988. (vi) Extensions of Kalman modelling to non-Gaussian observations. Quadermi di Statistica e Mathematica applicata, 12, 3–12, 1990. (vii) The first passage density of the Brownian motion process to a curved boundary. Journal of Applied Probability, 29, 291–304, 1992. (viii) On a test of serial correlation for regression models with lagged depen- dent variables. The Art of Statistical Science, edited K. V. Mardia, pp. 27–32, New York: Wiley, 1992. (ix) Optimal estimating equations for state vectors in non-Gaussian and nonlinear state space time series models. Selected Proceedings of Athens, Georgia, Symposium on Estimating Functions. Edited I. V. Basawa, V. P. Godambe and R. L. Taylor, 1997. (x) (with S. J. Koopman) Monte Carlo maximum likelihood estimation for non-Gaussian state space models. Biometrika, 84, 669–684, 1997. (xi) (with B. Quenneville) Benchmarking by state space models. Interna- tional Statistical Review, 65, 23–48, 1997. (xii) (with J. R. Magnus) Estimation of regression coefficients of interest when other regression coefficients are of no interest. Econometrica, 67, 639–643, 1999. (xiii) The state space approach to time series analysis and its potential for official statistics (The Foreman Lecture). Australian and New Zealand Journal of Statistics, 42, 1–23, 2000. (xiv) (with S. J. Koopman) Time series analysis of non-Gaussian observa- tions based on state space models from both classical and Baysian perspectives (with discussion). Journal of the Royal Statistical Soci- ety, Series B, 62, 3–56, 2000. (xv) (with S. J. Koopman) Fast filtering and smoothing for multivariate state space models. Journal of Time Series Analysis, 21, 281–296, 2000. (xvi) (with S. J. Koopman) Time Series Analysis by State Space Methods. Oxford: Oxford University Press, 2001. (xvii) (with S. J. Koopman) A simple and efficient simulation smoother for state space time series analysis. Biometrika, 89, 603–616, 2002. (xviii) (with S. J. Koopman) Filtering and smoothing of state vector for diffuse state space models. Journal of Time Series Analysis, 24, 85– 98, 2003. Acknowledgements
We would like to thank the Royal Netherlands Academy of Arts and Sciences (KNAW, http://www.knaw.nl) for their financial support and for dealing with the organisation of the conference. Further financial assistance was provided by the Netherlands Organisation for Scientific Research (NWO, http://www.nwo.nl), the Tinbergen Institute for Economic Research (TI Amsterdam and Rotterdam, http://www.tinbergen.nl), and ABP Invest- ments and Research (Schiphol, Amsterdam, http://www.abp.nl). Particular thanks are due to Martine Wagenaar of KNAW for being the driving force of the organisation of the conference and for her cooperation before, during and after the conference. Further we would like to thank Charles Bos, Irma Hindrayanto and Marius Ooms of Free University, Amsterdam for their as- sistance during various stages of the organisation of the conference and the preparation of this volume. We would also like to thank all the participants, particularly those who gave papers or acted as discussants. Finally, Neil Shephard thanks the ESRC for supporting his research.
xiv Part I State space models
1 Introduction to state space time series analysis James Durbin Department of Statistics, London School of Economics and Political Science
Abstract The paper presents a broad general review of the state space approach to time series analysis. It begins with an introduction to the linear Gaussian state space model. Applications to problems in practical time series analysis are considered. The state space approach is briefly compared with the Box– Jenkins approach. The Kalman filter and smoother and the simulation smoother are described. Missing observations, forecasting and initialisation are considered. A representation of a multivariate series as a univariate series is displayed. The construction and maximisation of the likelihood function are discussed. An application to real data is presented. The treatment is extended to non-Gaussian and nonlinear state space models. A simulation technique based on importance sampling is described for analysing these models. The use of antithetic variables in the simulation is considered. Bayesian analysis of the models is developed based on an extension of the importance sampling technique. Classical and Bayesian methods are applied to a real time series.
State Space and Unobserved Component Models: Theory and Applications, eds. Andrew C. Harvey, Siem Jan Koopman and Neil Shephard. Published by Cambridge University Press. C Cambridge University Press 2004
3 4 James Durbin 1.1 Introduction to state space models 1.1.1 Basic ideas The organisers have asked me to provide a broad, general introduction to state space time series analysis. In the pursuit of this objective I will try to make the exposition understandable for those who have relatively little prior knowledge of the subject, while at the same time including some results of recent research. My starting point is the claim that state space models provide an effective basis for practical time series analysis in a wide range of fields including statistics, econometrics and engineering. I will base my exposition on the recent book by Durbin and Koopman (2001), referred to from now on as the DK book, which provides a compre- hensive treatment of the subject. Readers may wish to refer to the website http://www.ssfpack.com/dkbook/ for further information about the book. Other books that provide treatments of state space models and techniques include Harvey (1989), West and Harrison (1997), Kitagawa and Gersch (1996) and Kim and Nelson (1999). More general books on time series analysis with substantial treatments of state space methods are, for example, Brockwell and Davis (1987), Hamilton (1994) and Shumway and Stoffer (2000). I will begin with a particular example that I will use to introduce the basic ideas that underlie state space time series analysis. This refers to logged monthly numbers of car drivers who were killed or seriously injured in road accidents in Great Britain, 1969–84. These data come from a study by Andrew Harvey and me, undertaken on behalf of the British Department of Transport, regarding the effect on road casualties of the seat belt law that wasintroduced in February 1983; for details see Durbin and Harvey (1985) and Harvey and Durbin (1986). Inspection of Figure 1.1 reveals that the series is made up of a trend which initially is increasing, then decreases and subsequently flattens out, plus a seasonal effect which is high in the winter and low in the summer, together with a sudden drop in early 1983 seemingly due to the introduction of the seat belt law. Other features that could be present, though they are not apparent from visual inspection, include cycles and regression effects due to the influence of such factors as the price of petrol, weather variations and traffic density. Thus we arrive at the following model:
yt = µt + γt + ct + rt + it + εt, (1.1) where Introduction to state space time series analysis 5
yt = observation (often logged, possibly a vector) µt = trend (slow change in level) γt = seasonal (pattern can change over time) ct = cycle (of longer period than seasonal) rt = regression component (coefficients can vary over time) it =intervention effect (e.g. seat belt law) εt = random error or disturbance or noise.
7.9 drivers 7.8
7.7
7.6
7.5
7.4 number (log)
7.3
7.2
7.1
7
70 75 80 85 year
Fig. 1.1. Monthly numbers (logged) of car drivers who were killed or seriously injured in road accidents in Great Britain.
In the state space approach we construct submodels designed to model the behaviour of each component such as trend, seasonal, etc. separately and we put these submodels together to form a single matrix model called a state space model. The model used by Harvey and Durbin for the analysis of the data of Figure 1.1 included all the components of (1.1) except ct; some of the results of their analysis will be presented later.
1.1.2 Special cases of the basic model We consider the following two special cases. 6 James Durbin 1.1.2.1 The local level model This is specified by
2 yt = µt + εt,εt ∼ N(0,σε ), 2 (1.2) µt+1 = µt + ηt,ηt ∼ N(0,ση), for t =1,...,n, where the εts and ηts are all mutually independent and are also independent of µ1. The objective of this model is to represent a series with no trend or sea- sonal whose level µt is allowed to vary over time. The second equation of the model is a random walk; random walks are basic elements in many state space time series models. Although it is simple, the local level model is not an artificial model and it provides the basis for the treatment of important series in practice. It is employed to explain the ideas underlying state space time series analysis in an elementary way in Chapter 2 of the DK book. The properties of time series that are generated by a local level model are studied in detail in Harvey (1989).
1.1.2.2 The local linear trend model This is specified by
2 yt = µt + εt,εt ∼ N(0,σε ), 2 µt+1 = µt + νt + ξt,ξt ∼ N(0,σξ ), (1.3) 2 νt+1 = νt + ζt,ζt ∼ N(0,σζ ). This extends the local level model to the case where there is a trend with a slope νt where both level and slope are allowed to vary over time. It is worth noting that when both ξt and ζt are zero, the model reduces to the classical linear trend plus noise model, yt = α + βt+ error. It is sometimes useful to smooth the trend by putting ξt =0in (1.3). Details of the model and its extensions to the general class of structural time series models are given in the DK book Section 3.2 and in Harvey (1989). The matrix form of the local linear trend model is µt yt = 10 + εt, νt µt 11 µt ξt +1 = + . νt+1 01 νt ζt By considering this and other special cases in matrix form we are led to the following general model which provides the basis for much of our further treatment of state space models. Introduction to state space time series analysis 7 1.1.3 The linear Gaussian state space model This has the form
yt = Ztαt + εt,εt ∼ N(0,Ht), αt+1 = Ttαt + Rtηt,ηt ∼ N(0,Qt),t=1,...,n, (1.4) α1 ∼ N(a1,P1).
Matrices Zt, Ht, Tt, Rt and Qt are assumed known. Initially, a1 and P1 are assumed known; we will consider later what to do when some elements of them are unknown. The p × 1vector yt is the observation. The unobserved m×1vector αt is called the state. The disturbances εt and ηt are independent sequences of independent normal vectors. The matrix Rt, when it is not the identity, is usually a selection matrix, that is, a matrix whose columns are a subset of the columns of the identity matrix; it is needed when the dimensionality of αt is greater than that of the disturbance vector ηt. The first equation is called the observation equation and the second equation is called the state equation. The structure of model (1.4) is a natural one for representing the be- haviour of many time series as a first approximation. The first equation is a standard multivariate linear regression model whose coefficient vector αt varies over time; the development over time of αt is determined by the first- order vector autoregression given in the second equation. The Markovian nature of the model accounts for many of its elegant properties. In spite of the conceptual simplicity of this model it is highly flexible and has a remarkably wide range of applications to problems in practical time series analysis. I will mention just a few.
(i) Structural time series models. These are models of the basic form (1.1) where the submodels for the components are chosen to be compatible with the state space form (1.4). The local level model and the local linear trend model are simple special cases. The models are sometimes called dynamic linear models . (ii) ARMA and Box–Jenkins (BJ) ARIMA models. These can be put in state space form as described in the DK book, Section 3.3. This means that ARIMA models can be treated as special cases of state space mod- els. I will make a few remarks at this point on the relative merits of the BJ approach and the state space approach for practical time series analysis.
(a) BJ is a ‘black box’ approach in which the model is determined purely by the data without regard to the structure underlying the 8 James Durbin data, whereas state space fits the data to the structure of the system which generated the data. (b) BJ eliminates trend and seasonal by differencing. However, in many cases these components have intrinsic interest and in state space they can be estimated directly. While in BJ estimates can be ‘recovered’ from the differenced series by maximising the residual mean square, this seems an artificial procedure. (c) The BJ identification procedure need not lead to a unique model; in some cases several apparently quite different models can appear to fit the data equally well. (d) In BJ it is difficult to handle regression effects, missing observa- tions, calendar effects, multivariate observations and changes in coefficients over time; these are all straightforward in state space. A fuller discussion of the relative merits of state space and BJ is given in the DK book, Section 3.5. The comparison is strongly in favour of state space. (iii) Model (1.4) handles time-varying regression and regression with auto- correlated errors straightforwardly. (iv) State space models can deal with problems in spline smoothing in discrete and continuos time on a proper modelling basis in which pa- rameters can be estimated by standard methods, as compared with customary ad hoc methods.
1.2 Basic theory for state space analysis 1.2.1 Introduction In this section we consider the main elements of the methodology required for time series analysis based on the linear Gaussian model (1.4). Let Yt = {y1,...,yt}, t =1,...,n.Wewill focus on the following items:
• Kalman filter. This recursively computes at+1 = E(αt+1|Yt) and Pt+1 = Var(αt+1|Yt) for t =1,...,n. Since distributions are normal, these quan- tities specify the distribution of αt+1 given data up to time t. • State smoother. This estimatesα ˆt = E(αt | Yn) and Vt =Var(αt | Yn) and hence the conditional distribution of αt given all the observations for t =1,...,n. • Simulation smoother.Analgorithm for generating draws from
p(α1,...,αn|Yn). Introduction to state space time series analysis 9 This is an essential element in the analysis of non-Gaussian and nonlinear models as described in Section 1.4. • Missing observations.Weshow that the treatment of missing observations is particularly simple in the state space approach. • Forecasting is simply treated as a special case of missing observations. • Initialisation. This deals with the case where some elements of a1 = E(α1) and V1 =Var(α1) are unknown. • Univariate treatment of multivariate series. This puts a multivariate model into univariate form, which can simplify substantially the treatment of large complex models. • Parameter estimation.Weshow that the likelihood function is easily con- structed using the Kalman filter.
1.2.2 Kalman filter We calculate
at+1 = E(αt+1|Yt),Pt+1 = Var(αt+1|Yt), by the recursion
vt = yt − Ztat, Ft = ZtPtZt + Ht, −1 Kt = TtPtZ F , t t (1.5) Lt = Tt − KtZt, at+1 = Ttat + Ktvt, Pt+1 = TtPtLt + RtQtRt t =1,...,n, with a1 and P1 as the mean vector and the variance matrix of α1.
1.2.3 State smoother We calculate
αˆt = E(αt | Yn),Vt = Var(αt | Yn), by the backwards recursion