Bayesian Econometrics using BayESç Grigorios Emvalomatis February 21, 2020 © 2020 2 Grigorios Emvalomatis, 2020 © 2020 by Grigorios Emvalomatis. “Bayesian Econometrics using BayES” is made available under a Creative Commons Attribution 4.0 License (international): http://creativecommons.org/licenses/by/4.0/legalcode The most up-to-date version of this document can be found at: http://www.bayeconsoft.com/pdf/BayesianEconometricsUsingBayES.pdf Contents Preface iii 1 Econometrics and Bayesian Inference 1 1.1 Overview ....................................... 1 1.2 Econometrics: Frequentist and Bayesian Approaches ............... 1 1.3 Bayes’ Theorem and Bayesian Inference ...................... 3 1.3.1 The Likelihood Function ........................... 5 1.3.2 The Prior Density .............................. 6 1.3.3 The Posterior Density ............................ 9 1.3.4 Model Comparison and the Marginal Likelihood ............. 12 1.3.5 Prediction and Forecasting ......................... 14 1.3.6 Discussion ................................... 15 1.4 Estimation by Simulation .............................. 15 1.4.1 The Strong Law of Large Numbers and a Central Limit Theorem .... 16 1.4.2 Markov-Chain Monte Carlo (MCMC) ................... 19 1.5 Synopsis ........................................ 29 2 The Linear Model 31 2.1 Overview ....................................... 31 2.2 Model Setup and Interpretation ........................... 31 2.3 Likelihood, Priors and Posterior ........................... 33 2.4 Full Conditionals and Parameter Estimation .................... 35 2.5 Other Functional Forms and Marginal Effects ................... 38 2.6 Post-Estimation Inference .............................. 42 2.6.1 Imposing Parametric Restrictions and Evaluating their Plausibility ... 42 2.6.2 Model Comparison in the Linear Regression Model ............ 45 2.6.3 Predicting the Values of the Dependent Variable ............. 47 2.7 Synopsis ........................................ 49 3 Seemingly Unrelated Regressions 51 3.1 Overview ....................................... 51 3.2 The System Approach to Linear Regression .................... 51 3.3 Likelihood, Priors and Full Conditionals ...................... 54 3.4 Cross-Equation Restrictions and the SUR Model ................. 58 3.4.1 Demand Systems ............................... 58 3.4.2 Cost Functions and Cost Share Equations ................. 59 3.4.3 Imposing Linear Restrictions ........................ 60 3.5 Synopsis ........................................ 62 4 Data Augmentation 63 4.1 Overview ....................................... 63 4.2 Data Augmentation in Latent-Data Problems ................... 63 4.3 Applications of Data Augmentation ........................ 65 4.3.1 The Linear Model with Heteroskedastic Error ............... 65 i ii CONTENTS 4.3.2 The Stochastic Frontier Model ....................... 69 4.4 Marginal Data Augmentation ............................ 74 4.5 Synopsis ........................................ 76 5 The Linear Model with Panel Data 77 5.1 Overview ....................................... 77 5.2 Panel Data and Alternative Panel-Data Models .................. 77 5.3 Estimation of the Hierarchical Panel-Data Models ................ 80 5.3.1 Estimation of the Random-Effects Model ................. 80 5.3.2 Estimation of the Random-Coefficients Model ............... 83 5.4 Extensions to Other Panel-Data Models ...................... 86 5.4.1 Correlated Random Effects ......................... 86 5.4.2 Models with Group-Specific and Common Coefficients .......... 87 5.4.3 Random-Coefficients Models with Determinants of the Means ...... 88 5.5 Synopsis ........................................ 88 6 Models for Binary Response 91 6.1 Overview ....................................... 91 6.2 The Nature of Binary-Response Models ...................... 91 6.2.1 Random Utility: An Underlying Framework for Binary Choice ..... 94 6.3 Estimation of Binary-Response Models ....................... 95 6.3.1 Estimation of the Binary Probit Model .................. 96 6.3.2 Estimation of the Binary Logit Model ................... 98 6.4 Interpretation of Parameters and Marginal Effects ................ 100 6.5 Binary-Response Models for Panel Data ...................... 102 6.6 Multivariate Binary-Response Models ....................... 105 6.7 Synopsis ........................................ 114 7 Models for Multiple Discrete Response 115 7.1 Overview ....................................... 115 7.2 The Nature of Discrete-Response Models ...................... 115 7.3 Multinomial Models ................................. 116 7.3.1 The Random-Utility Setup and the Latent-Variable Representation . 118 7.3.2 Estimation of the Multinomial Logit Model ................ 122 7.3.3 Estimation of the Multinomial Probit Model ............... 125 7.3.4 Marginal Effects in Multinomial Models .................. 128 7.4 Conditional Models .................................. 131 7.4.1 Estimation of Conditional Models for Discrete Choice .......... 133 7.4.2 Marginal Effects in Conditional Models for Discrete Choice ....... 135 7.5 Synopsis ........................................ 138 Preface This document is intended to serve as an introductory textbook for a postgraduate or advanced undergraduate course on Bayesian econometrics or as a reference for the applied econometrician who never got exposed to the Bayesian approach. Although Bayesian econometrics is increas- ingly being used in applied research, programs of study in economics usually include courses only in frequentist econometrics, even if time allows for more than a single course. Apart from tradition, this bias towards the frequentist approach to statistical inference can be attributed to the lack of specialized software for Bayesian econometrics. This is changing rapidly, with main- stream econometric software packages incorporating Bayesian techniques and the emergence of new software packages that make application of Bayesian methods considerably easier. This textbook aims at covering the basics of Bayesian econometrics, focusing on the appli- cation of the methods, rather than the techniques themselves (deriving full conditionals and coding). It does so by relying heavily on BayES for estimating the models presented in it and, as such, it can also be used as a gentle introduction to the software. BayES was chosen, apart from the obvious reason that the author is also the developer of the software, because of the nature of the software. BayES is designed from the beginning exclusively for Bayesian econometrics and it provides an intuitive graphical interface that allows first-time users to run models without having to spend hours reading the documentation. Additionally, it features a compact matrix language, which can be used by advanced users to code samplers for their own models, if these are not yet available in BayES. Equally importantly, it provides interfaces to other statistical software packages, both Bayesian and frequentist, which allow estimation of specialized models available in them. Chapter 1 starts by defining the modern meaning of the term econometrics and proceeds to present the fundamentals of Bayesian inference and techniques. This chapter is, by far, the most challenging, as it deals with the meaning and interpretation of probability, a concept that appears straightforward until one really starts thinking about it, as well as with the process of using data to update prior beliefs. The chapter has very little to do with economics and can be viewed as a crash course in Bayesian inference for readers who have never seen the concepts and methods before. Simulation methods are also covered in this chapter, as it is hard to sep- arate Bayesian estimation theory from modern estimation techniques. The extend of coverage of inference methods may seem unconventional to readers who have been exposed to frequen- tist econometrics, but one has to keep in mind that most readers of frequentist econometrics textbooks usually have already had a course in frequentist statistics and hypothesis testing, while this is rarely the case for Bayesian methods. The following two chapters cover the basic models used in econometrics and which can be estimated with the methods presented in the first chapter. These include the linear model and systems of equations and the user will be referred back to them on multiple occasions. Chapter 4 discusses data augmentation, the method that enables Bayesian inference to deal with complex models on which frequentist methods usually “choke”. Although initially discussed at a high level of abstraction, two applications of data augmentation are also presented in this chapter, as extensions to the linear model. This chapter is definitely recommended to all readers, as it forms the basis for much of the material covered in the remainder of the textbook. From this point onwards the reader could concentrate on the models of interest without any break in continuity. The textbook follows the development of BayES and, as such, it can be considered incom- plete. As BayES’ coverage extends to include more models, this document will evolve as well. Nevertheless, the material included in the current version covers the basics of Bayesian infer- ence and the most popular econometric models for cross-sectional and panel data. Therefore, it can already be used for a semester-long course on Bayesian econometrics or to provide the fundamentals for an applied econometrician. This textbook has already started being