Hisashi Tanizaki NONLINEAR FILTERS Estimation and Applications Second Edition Springer-Verlag Berlin Heidelberg NewYork London Paris Tokyo Hong Kong Barcelona Budapest To My Wife Miyuki Preface Preface to Second Edition This book is a revision of Nonlinear Filters: Estimation and Applications, (Lecture Notes in Economics and Mathematical Systems, No.400), which was published from Springer-Verlag in 1993. Compared with the ¯rst edition, I have made a substantial revision in the second edition. First, titles in the following chapters, sections, terms and so on are changed as follows. The First Edition The Second Edition Chapter 3 Chapter 3 Nonlinear Filters based on Taylor = Traditional Nonlinear Filters Series Expansion ) Chapter 4 Chapter 4 Nonlinear Filters based on Density = Density-Based Nonlinear Filters Approximation ) Section 4.3 Section 4.3 Numerical Density Approximation = Numerical Integration Filter by Piecewise Linear Functions: ) Modi¯ed Kitagawa Estimator Section 4.4 Section 4.4 Simulation-based Density Estimator = Importance Sampling Filter ) Chapter 5 Chapter 5 Comparison of Nonlinear Filters: = Monte-Carlo Experiments Monte-Carlo Experiments ) Chapter 6 Chapter 6 An Application of Nonlinear = Application of Nonlinear Filters Filters: Estimation of Permanent ) Consumption Monograph = Book ) VIII Preface Thus, the most appropriate title is taken or the title is made short. Second, new contents are briefly summarized as follows. Section 2.3.3 Minimum Mean Square Linear Estimator ² As the third derivation method of the Kalman ¯lter, I put this section. This section is not utilized for the proceeding chapters but it is added as a survey of the standard Kalman ¯lter. Appendix A2.2 Conditional Normal Distribution ² The derivation method under normality assumption is discussed in Sec- tion 2.3.1. Lemmas and Proofs used in Section 2.3.1 are summarized in this appendix. Section 4.5 Density-based Monte-Carlo Filter ² This nonlinear ¯lter is one of the recent new nonlinear ¯lters. Section 4.6 Rejection Sampling Filter ² The rejection sampling ¯lter is also one of the recent topics, where a recursive algorithm of random draws from ¯ltering densities is derived. Appendix A4.5 Density-Based Monte-Carlo Filter ² For the ¯ltering estimates by the density-based Monte-Carlo ¯lter, the asymptotic properties are discussed. Appendix A4.6 Rejection Sampling ² Random number generation by rejection sampling is discussed in a gen- eral form. Appendix A5.1 On Initial Value of State-Variable ² In the Kalman ¯lter algorithm, the ¯ltering estimates are recursively obtained given the initial value. We analyze how the ¯ltering estimates are sensitive to the initial value. Appendix A5.3 On Random Draws by Importance Sampling ² Monte-Carlo integration with importance sampling is utilized to the non- linear ¯lter in Section 4.4, where random draws have to be generated from the importance density. We discuss about the random draws generated from the importance density. Appendix A5.4 Rejection Sampling ² Using rejection sampling, random draws are generated by a computer. Precision of random draws are examined. Chapter 7 Prediction and Smoothing ² The density-based ¯ltering algorithms discussed in Chapter 4 are ex- tended to prediction and smoothing. Moreover, the following chapters are substantially revised. Chapter 5 Monte-Carlo Experiments ² I changed a functional form of the nonlinear measurement and transition equations in some simulation studies. Preface to Second Edition IX Chapter 6 Application of Nonlinear Filters ² In the ¯rst edition, only the U.S. data are used. In the second edition, the same type of state-space model is estimated for Japan, U.S., U.K., France, Spain, Italy, Canada and Germany. I focus on estimation of the unknown parameters and estimation of a ratio of per capita permanent consumption relative to per capita total consumption for the above coun- tries. Thus, the second edition is substantially changed compared with the ¯rst edition. Finally, I am grateful to the editor Dr. Werner A. Mueller, who gave me a chance to revise the monograph. March, 1996 Author: Hisashi Tanizaki Associate Professor Faculty of Economics, Kobe University Rokkodai, Nadaku, Kobe 657, Japan E-mail: [email protected] X Preface Preface to First Edition Acknowledgements Originally, this monograph is a revision of my Ph.D. dissertation (\ Nonlinear Filters: Estimation and Applications, " December, 1991) at the University of Pennsylvania. In writing the dissertation, many people took care of me in di®erent ways. In particular, Professor Roberto S. Mariano was very patient as my main thesis advisor, and he gave me many useful suggestions and comments. I wish to thank him for recommending me this line of research and giving me the suggestions and comments. Also, I would like to acknowledge the support of NSF grant SES 9011917 in Professor Roberto S. Mariano's supervision of the thesis. Professor Marc Nerlove supported me ¯nancially during my staying in the U.S. in spite of my trouble with English. I would like to acknowledge the support of NSF grant SES 5-25056. Professor Francis X. Diebold has stimulated me through his studies. Although the dissertation was not directly related to their research, their work signi¯cantly a®ected me and will influence my future research. I am grateful to them for that. Moreover, I do not forget to thank the classmates in the Graduate Pro- gram in Economics who had been encouraging me since I started studying at the University. Especially, I got many \ invisible things " from Michael Blackman, Celia Chen, Ilaria Fornari and Til Schuermann (in alphabetical order) and asked them to correct my English. Also, I am grateful to Jun Sato and Shinichi Suda (the same year Japanese Ph.D. students in Economics) and Suminori Tokunaga (Japanese Ph.D. stu- dent in Regional Science) for helping me. I could not continue to study there without them. When I was depressed just after coming to the U.S., they encouraged me. Furthermore, I wish to thank Professors Mitsuo Saito, Kazuo Ogawa, Toshihisa Toyoda and Kazuhiro Ohtani. When I was in Japan before going to the U.S., I learned a lot of things under them at Kobe University. Especially, Professor Mitsuo Saito, who was my main advisor in Japan, recommended that I continue to study econometrics at the University of Pennsylvania. And I am grateful to Shinichi Kitasaka, who was one of my classmates at Kobe University, for his suggestions and comments. Moreover, I have been stimulated by Shigeyuki Hamori's passion for research. He was one of my classmates at Kobe University, and he studied economics at Duke University at the same time that I was at the University of Pennsylvania. I am grateful to him for his valuable suggestions and advice. Also, I wish to say \ Thanks " to my parents. They have not understood what I have done, but they supported me with regard to everything. Thus, I believe that I could ¯nish the dissertation thanks to all the professors and all the friends who know me. Preface to First Edition XI Finally, I am grateful to Professor Marc Nerlove, again, who suggested me to revise the dissertation for possible publication, and Professor Wilhem Krelle, who was the referee of Springer-Verlag and gave me valuable sugges- tions and comments. March, 1993 Preface The purpose of this monograph is to develop nonlinear ¯lters and demonstrate their applications. There are two approaches to nonlinear ¯lters. One is ap- proximating nonlinear measurement and transition equations by the Taylor series expansion and applying the approximated nonlinear functions directly to the standard linear recursive Kalman ¯lter algorithm. Another is approx- imating the underlying density functions of the state vector by a Gaussian sum, numerical integration or Monte-Carlo integration. For the nonlinear ¯lters based on the Taylor series expansion, ¯rst, it is shown that we need to impose some approximations on the disturbances. Next, I propose a nonlinear ¯lter which combines the extended Kalman ¯l- ter with Monte- Carlo stochastic simulations, where each expectation in the algorithm is evaluated by generating random numbers. Also, for the single- stage iteration ¯lter, a re-interpretation is given, which is di®erent from the conventional one. It is, however, known that applying the linearized nonlinear measurement and transition equations to the conventional linear algorithm leads to biased ¯ltering estimates. Therefore, it is essential to approximate the underlying conditional density functions rather than the nonlinear measurement and transition equations. A small extension is given to density approximation by numerical integration, where the nodes are taken as random numbers. Fur- thermore, the nonlinear ¯lter with importance sampling is proposed, where the importance sampling theory developed by Geweke is applied to the non- linear ¯ltering problem. Monte-Carlo experiments are performed to examine the nonlinear ¯lters. It is found that the nonlinear ¯lters based on the density approximation are better estimators than those based on the Taylor series expansion by the criteria of BIAS (bias) and RMSE (root mean squared error). Finally, as an application to the nonlinear ¯lters, I consider estimating permanent and transitory consumption separately, where the ¯ndings are as follows; (i) according to the likelihood ratio test, the hypothesis of no transitory consumption is rejected, (ii) we do not have the excess sensitivity problems on consumption, taking into account nonlinearity of the Euler equa- tion, variable gross rate of return on savings and transitory consumption, and (iii) consumption consists of three parts: permanent consumption, transitory consumption and the other factor which depends on income or trend. Table of Contents Preface ::::::::::::::::::::::::::::::::::::::::::::::::::::::: VII 1. Introduction :::::::::::::::::::::::::::::::::::::::::::::: 1 1.1 Objectives of the Book . 1 1.2 Review of Related Studies . 3 1.3 Outline of the Book. 7 2. State-Space Model in Linear Case :::::::::::::::::::::::: 15 2.1 Introduction . 15 2.2 Applications of Kalman Filter . 16 2.2.1 Time-Varying Parameter Model .