Time-Series Econometrics A Concise Course Francis X. Diebold University of Pennsylvania Edition 2019 Version 2019.01.14 Time Series Econometrics Time Series Econometrics A Concise Course Francis X. Diebold Copyright c 2013-2019, by Francis X. Diebold. All rights reserved. This work is freely available for your use, but be warned: it is highly preliminary, significantly incomplete, and rapidly evolving. It is licensed under the Creative Commons Attribution- NonCommercial-NoDerivatives 4.0 International License. (Briefly: I retain copyright, but you can use, copy and distribute non-commercially, so long as you give me attribution and do not modify. To view a copy of the license, visit http://creativecommons.org/licenses/by-nc- nd/4.0/.) In return I ask that you please cite the book whenever appropriate, as: \Diebold, F.X. (2019), Time Series Econometrics, Department of Economics, University of Pennsyl- vania, http://www.ssc.upenn.edu/ fdiebold/Textbooks.html." To Marc Nerlove, who taught me time series, and to my wonderful Ph.D. students, his \grandstudents" Brief Table of Contents About the Author xv About the Cover xvi Guide to e-Features xvii Acknowledgments xviii Preface xxii Chapter 1. The Wold Representation and its Approximation 1 Chapter 2. Spectral Analysis 23 Chapter 3. Markovian Structure, Linear Gaussian State Space, and Optimal (Kalman) Filtering 47 Chapter 4. Frequentist Time-Series Likelihood Evaluation, Optimization, and Inference 79 Chapter 5. Simulation Basics 90 Chapter 6. Bayesian Analysis by Simulation 96 Chapter 7. (Much) More Simulation 109 Chapter 8. Non-Stationarity: Integration, Cointegration and Long Memory 126 Chapter 9. Non-Linear Non-Gaussian State Space and Optimal Filtering 138 Chapter 10. Volatility Dynamics 145 Chapter 11. High Dimensionality 186 Appendices 187 Appendix A. A \Library" of Useful Books 188 Detailed Table of Contents About the Author xv About the Cover xvi Guide to e-Features xvii Acknowledgments xviii Preface xxii Chapter 1. The Wold Representation and its Approximation 1 1.1 Economic Time Series and Their Analysis1 1.2 The Environment1 1.3 White Noise 3 1.4 The Wold Decomposition and the General Linear Process4 1.5 Approximating the Wold Representation6 1.5.1 The MA(q) Process6 1.5.2 The AR(p) Process6 1.5.3 The ARMA(p; q) Process6 1.6 Wiener-Kolmogorov-Wold Extraction and Prediction6 1.6.1 Extraction6 1.6.2 Prediction 6 1.7 Multivariate 7 1.7.1 The Environment7 1.7.2 The Multivariate General Linear Process8 1.7.3 Vector Autoregressions9 1.8 Exercises, Problems and Complements 14 1.9 Notes 21 1.10 Exercises, Problems and Complements 22 Chapter 2. Spectral Analysis 23 2.1 The Many Uses of Spectral Analysis 23 2.2 The Spectrum and its Properties 23 2.3 Rational Spectra 26 2.4 Multivariate 27 2.5 Filter Analysis and Design 30 2.6 Estimating Spectra 34 2.6.1 Univariate 34 2.6.2 Multivariate 36 2.7 Approximate (asymptotic) frequency domain Gaussian likelihood 36 2.8 Exercises, Problems and Complements 37 2.9 Notes 46 xii DETAILED TABLE OF CONTENTS Chapter 3. Markovian Structure, Linear Gaussian State Space, and Optimal (Kalman) Filtering 47 3.1 Markovian Structure 47 3.1.1 The Homogeneous Discrete-State Discrete-Time Markov Process 47 3.1.2 Multi-Step Transitions: Chapman-Kolmogorov 47 3.1.3 Lots of Definitions (and a Key Theorem) 48 3.1.4 A Simple Two-State Example 49 3.1.5 Constructing Markov Processes with Useful Steady-State Distributions 50 3.1.6 Variations and Extensions: Regime-Switching and More 51 3.1.7 Continuous-State Markov Processes 52 3.2 State Space Representations 53 3.2.1 The Basic Framework 53 3.2.2 ARMA Models 55 3.2.3 Linear Regression with Time-Varying Parameters and More 60 3.2.4 Dynamic Factor Models 62 3.2.5 Unobserved-Components Models 63 3.3 The Kalman Filter and Smoother 64 3.3.1 Statement(s) of the Kalman Filter 65 3.3.2 Derivation of the Kalman Filter 66 3.3.3 Calculating P0 69 3.3.4 Predicting yt 69 3.3.5 Steady State and the Innovations Representation 70 3.3.6 Kalman Smoothing 72 3.4 Exercises, Problems and Complements 72 3.5 Notes 78 Chapter 4. Frequentist Time-Series Likelihood Evaluation, Optimization, and Inference 79 4.1 Likelihood Evaluation: Prediction-Error Decomposition and the Kalman Filter 79 4.2 Gradient-Based Likelihood Maximization: Newton and Quasi-Newton Methods 80 4.2.1 The Generic Gradient-Based Algorithm 80 4.2.2 Newton Algorithm 81 4.2.3 Quasi-Newton Algirithms 82 4.2.4 \Line-Search" vs. \Trust Region" Methods: Levenberg-Marquardt 82 4.3 Gradient-Free Likelihood Maximization: EM 83 4.3.1 \Not-Quite-Right EM" (But it Captures and Conveys the Intuition) 84 4.3.2 Precisely Right EM 84 4.4 Likelihood Inference 86 4.4.1 Under Correct Specification 86 4.4.2 Under Possible Mispecification 87 4.5 Exercises, Problems and Complements 89 4.6 Notes 89 Chapter 5. Simulation Basics 90 5.1 Generating U(0,1) Deviates 90 5.2 The Basics: c.d.f. Inversion, Box-Mueller, Simple Accept-Reject 91 5.2.1 Inverse c.d.f. 91 5.2.2 Box-Muller 92 5.2.3 Simple Accept-Reject 93 5.3 Simulating Exact and Approximate Realizations of Time Series Processes 94 5.4 more 95 5.5 Notes 95 Chapter 6. Bayesian Analysis by Simulation 96 DETAILED TABLE OF CONTENTS xiii 6.1 Bayesian Basics 96 6.2 Comparative Aspects of Bayesian and Frequentist Paradigms 96 6.3 Markov Chain Monte Carlo 98 6.3.1 Metropolis-Hastings Independence Chain 99 6.3.2 Metropolis-Hastings Random Walk Chain 99 6.3.3 More 99 6.3.4 Gibbs and Metropolis-Within-Gibbs 100 6.4 Conjugate Bayesian Analysis of Linear Regression 102 6.5 Gibbs for Sampling Marginal Posteriors 103 6.6 General State Space: Carter-Kohn Multi-Move Gibbs 104 6.7 Exercises, Problems and Complements 108 6.8 Notes 108 Chapter 7. (Much) More Simulation 109 7.1 Economic Theory by Simulation: \Calibration" 109 7.2 Econometric Theory by Simulation: Monte Carlo and Variance Reduction 109 7.2.1 Experimental Design 109 7.2.2 Simulation 110 7.2.3 Variance Reduction: Importance Sampling, Antithetics, Control Variates and Common Random Numbers 112 7.2.4 Response Surfaces 116 7.3 Estimation by Simulation: GMM, SMM and Indirect Inference 117 7.3.1 GMM 117 7.3.2 Simulated Method of Moments (SMM) 118 7.3.3 Indirect Inference 119 7.4 Inference by Simulation: Bootstrap 119 7.4.1 i.i.d. Environments 119 7.4.2 Time-Series Environments 122 7.5 Optimization by Simulation 123 7.5.1 Local 123 7.5.2 Global 124 7.5.3 Is a Local Optimum Global? 125 7.6 Interval and Density Forecasting by Simulation 125 7.7 Exercises, Problems and Complements 125 7.8 Notes 125 Chapter 8. Non-Stationarity: Integration, Cointegration and Long Memory 126 8.1 Random Walks as the I(1) Building Block: The Beveridge-Nelson Decomposition 126 8.2 Stochastic vs. Deterministic Trend 127 8.3 Unit Root Distributions 128 8.4 Univariate and Multivariate Augmented Dickey-Fuller Representations 130 8.5 Spurious Regression 131 8.6 Cointegration, Error-Correction and Granger's Representation Theorem 131 8.7 Fractional Integration and Long Memory 135 8.8 Exercises, Problems and Complements 136 8.9 Notes 137 Chapter 9. Non-Linear Non-Gaussian State Space and Optimal Filtering 138 9.1 Varieties of Non-Linear Non-Gaussian Models 138 9.2 Markov Chains to the Rescue (Again): The Particle Filter 138 9.3 Particle Filtering for Estimation: Doucet's Theorem 138 9.4 Key Application I: Stochastic Volatility (Revisited) 138 9.5 Key Application II: Credit-Risk and the Default Option 138 xiv DETAILED TABLE OF CONTENTS 9.6 Key Application III: Dynamic Stochastic General Equilibrium (DSGE) Macroe- conomic Models 138 9.7 A Partial \Solution": The Extended Kalman Filter 138 Chapter 10. Volatility Dynamics 145 10.1 Volatility and Financial Econometrics 145 10.2 GARCH 145 10.3 Stochastic Volatility 145 10.4 Observation-Driven vs. Parameter-Driven Processes 145 10.5 Exercises, Problems and Complements 185 10.6 Notes 185 Chapter 11. High Dimensionality 186 11.1 Exercises, Problems and Complements 186 11.2 Notes 186 Appendices 187 Appendix A. A \Library" of Useful Books 188 About the Author Francis X. Diebold is Paul F. and Warren S. Miller Professor of Economics, and Professor of Finance and Statistics, at the University of Pennsylvania, as well as Faculty Research As- sociate at the National Bureau of Economic Research in Cambridge, Mass. He has published widely in econometrics, forecasting, finance and macroeconomics, and he has served on the editorial boards of numerous scholarly journals. He is an elected Fellow of the Econometric Society, the American Statistical Association, and the International Institute of Forecasters; the recipient of Sloan, Guggenheim, and Humboldt fellowships; and past President of the Society for Financial Econometrics. Diebold lectures actively, worldwide, and has received several prizes for outstanding teaching. He has held visiting appointments in Economics and Finance at Princeton University, Cambridge University, the University of Chicago, the Lon- don School of Economics, Johns Hopkins University, and New York University. His research and teaching are firmly rooted in applications; he has served as an economist under Paul Volcker and Alan Greenspan at the Board of Governors of the Federal Reserve System in Washington DC, an Executive Director at Morgan Stanley Investment Management, Co- Director of the Wharton Financial Institutions Center, and Chairman of the Federal Reserve System's Model Validation Council.