Graduate Econometrics Review
Total Page:16
File Type:pdf, Size:1020Kb
Graduate Econometrics Review Paul J. Healy [email protected] April 13, 2005 ii Contents Contents iii Preface ix Introduction xi I Probability 1 1 Probability Theory 3 1.1 Counting ............................... 3 1.1.1 Ordering & Replacement .................. 3 1.2 The Probability Space ........................ 4 1.2.1 Set-Theoretic Tools ..................... 5 1.2.2 Sigma-algebras ........................ 6 1.2.3 Measure Spaces & Probability Spaces ........... 9 1.2.4 The Probability Measure P ................. 10 1.2.5 The Probability Space (; §; P) ............... 11 1.2.6 Random Variables & Induced Probability Measures .... 13 1.3 Conditional Probability & Independence .............. 15 1.3.1 Conditional Probability ................... 15 1.3.2 Warner's Method ....................... 16 1.3.3 Independence ......................... 17 1.3.4 Philosophical Remarks .................... 18 1.4 Important Probability Tools ..................... 19 1.4.1 Probability Distributions .................. 22 1.4.2 The Expectation Operator .................. 23 1.4.3 Variance and Coviariance .................. 25 2 Probability Distributions 27 2.1 Density & Mass Functions ...................... 27 2.1.1 Moments & MGFs ...................... 27 2.1.2 A Side Note on Di®erentiating Under and Integral .... 28 2.2 Commonly Used Distributions .................... 29 iii iv CONTENTS 2.2.1 Discrete Distributions .................... 29 2.2.2 Continuous Distributions .................. 30 2.2.3 Useful Approximations .................... 31 3 Working With Multiple Random Variables 33 3.1 Random Vectors ........................... 33 3.2 Distributions of Multiple Variables ................. 33 3.2.1 Joint and Marginal Distributions .............. 33 3.2.2 Conditional Distributions and Independence ........ 34 3.2.3 The Bivariate Normal Distribution ............. 35 3.2.4 Useful Distribution Identities ................ 38 3.3 Transformations (Functions of Random Variables) ........ 39 3.3.1 Single Variable Transformations .............. 39 3.3.2 Bivariate Transformations .................. 42 4 Famous Inequalities 43 4.1 Bonferroni's Inequality ........................ 43 4.2 A Useful Lemma ........................... 43 4.3 Holder's Inequality .......................... 43 4.4 Cauchy-Schwarz Inequality ..................... 43 4.5 Covariance Inequality ........................ 44 4.6 Markov's Inequality ......................... 44 4.7 Jensen's Inequality .......................... 44 4.8 Chebyshev's Inequality ........................ 44 II Statistics 45 5 Parameter Estimation 47 5.1 Statistics - De¯nitions and Properties ............... 47 5.2 Method of Moments ......................... 48 5.3 Maximum Likelihood Estimation (MLE) .............. 48 5.3.1 Properties of ML Estimators ................ 49 5.4 Bayes Estimation ........................... 50 5.5 The Cramer-Rao Lower Bound ................... 50 5.6 Score, Information Matrix, & Information Equality ........ 52 6 Hypothesis Testing 55 6.1 Overview ............................... 55 7 Large-Sample Results 57 7.1 Introduction .............................. 57 7.2 Notions of Convergence ....................... 57 7.3 Laws of Large Numbers ....................... 59 7.4 Central Limit Theorems ....................... 60 CONTENTS v 8 Ordinary Least Squares (OLS) 63 8.1 The Model .............................. 63 8.2 Justifying the Error Term ...................... 64 8.3 OLS Assumptions .......................... 64 8.4 The OLS Solution .......................... 65 8.4.1 Linear Algebra Interpretation of the OLS Solution .... 66 8.5 Properties of OLS .......................... 67 8.5.1 Gauss-Markov Theorem ................... 67 8.6 Various Mathematical Results and Identities ............ 69 8.7 MLE of The Linear Model ...................... 70 8.8 Model Restrictions .......................... 71 8.8.1 Omitted Variables and OLS Estimates ........... 71 8.8.2 Testing ¯ Against a Given Value .............. 72 8.8.3 General Setup for Model Restrictions Tests ........ 72 8.8.4 The F-Test .......................... 73 8.8.5 The Lagrange Multiplier Test ................ 73 8.8.6 The Likelihood Ratio Test .................. 75 8.8.7 The Wald Test ........................ 76 8.8.8 Summary of Tests ...................... 76 8.9 The ANOVA Table .......................... 78 9 Non-Spherical Disturbances 79 9.1 Introduction .............................. 79 9.2 Generalized Least Squares (GLS) .................. 80 9.3 Feasible GLS ............................. 81 9.4 Heteroscedasticity .......................... 82 9.4.1 Jackknife Estimator ..................... 82 9.4.2 Testing for Heteroscedasticity ................ 83 10 Time-Series Models 85 10.1 Introduction .............................. 85 10.2 Serial Correlation ........................... 85 10.3 Ignoring Serial Correlation ...................... 86 10.4 Various Forms of Serial Correlation ................. 86 10.4.1 AR(1) - 1st-Order Autoregressive Correlation ....... 87 10.4.2 AR(p) - pth-Order Autoregressive Correlation ....... 88 10.4.3 MA(1) - 1st-Order Moving Average Correlation ...... 88 10.4.4 MA(p) - pth-Order Moving Average Correlation ...... 88 10.5 Testing for Serial Correlation .................... 88 10.5.1 Introduction ......................... 88 10.5.2 The Durbin-Watson Test .................. 88 10.5.3 Limitations of the DW Test ................. 90 10.6 Lagged Independent Variables .................... 90 10.7 Lagged Dependent Variables ..................... 90 vi CONTENTS 11 Explanatory Variable Problems 91 11.1 Multicollinearity ........................... 91 11.2 Testing for Multicollinearity ..................... 93 11.3 Measurement Error & Random Regressors ............. 93 12 Panel Data 95 12.1 Introduction .............................. 95 12.2 The Model .............................. 95 12.3 Fixed E®ects Models ......................... 96 12.4 Random E®ects Models ....................... 96 13 Systems of Regression Equations 97 13.1 Introduction .............................. 97 13.2 The Seemingly Unrelated Regressions Model ........... 97 13.2.1 SUR Conclusions ....................... 100 14 Simultaneous Equations 101 14.1 Introduction .............................. 101 14.2 Ignoring Simultaneity ........................ 101 14.3 Recursive Systems .......................... 102 14.4 Indirect Least Squares (Reduced Form Estimation) ........ 103 14.4.1 The Identi¯cation Problem ................. 104 14.4.2 The Order Condition ..................... 105 14.4.3 The Rank Condition ..................... 106 14.4.4 Klein's Macro Model 1: An ILS Example ......... 108 14.5 Lagrange Multiplier Test for Omitted Variables .......... 109 14.6 Conclusion .............................. 110 15 Correlation Between Regressors and Error 111 15.1 Introduction .............................. 111 15.2 Instrumental Variables ........................ 111 15.3 Two-Stage Least Squares ...................... 114 16 Qualitative Response Models 115 16.1 Introduction .............................. 115 16.2 Linear Probability Model ...................... 115 16.3 The Log-Odds Ratio ......................... 116 16.4 Binary Logit & Probit Models .................... 117 16.5 Binary Model Estimation ...................... 118 16.5.1 Maximum Likelihood Estimates ............... 118 16.5.2 Minimum Â2 Estimates ................... 120 16.5.3 Method of Scoring ...................... 120 16.5.4 Non-Linear Weighted Least Squares (NLWLS) ...... 120 16.5.5 Method of Scoring and NLWLS Equivalence ........ 120 16.6 Multinomial Models ......................... 120 16.6.1 Independence of Irrelevant Alternatives .......... 121 CONTENTS vii 16.7 Ordered Models ............................ 122 16.8 Hypothesis Tests ........................... 122 III Practice Problems 123 17 Prelim Problems Solved 125 17.1 Probability Problems ......................... 125 17.1.1 Drug Testing ......................... 125 17.1.2 The Birthday Problem .................... 126 17.1.3 The Monty Hall Problem .................. 126 17.1.4 Conditional Probability 1 .................. 127 17.1.5 Conditional Probability 2 .................. 129 17.2 Transformation Problems ...................... 130 17.3 E[Y jX] Problems ........................... 130 17.4 Gauss-Markov Problems ....................... 132 17.5 Basic Binary Models ......................... 133 17.6 Information Equality - OLS ..................... 134 17.7 Information Equality - Binary Logit ................ 136 17.8 Binary Choice Regression ...................... 137 17.9 MLE, Frequency, and Consistency ................. 138 17.10Bayesian Updating .......................... 139 17.11\Research" Problems ......................... 140 IV Estimation Asymptotics 145 18 Background 147 18.1 Probability, Measures, and Integration ............... 147 18.2 Convergence .............................. 147 18.2.1 Limits and Integration .................... 149 18.3 Law of Large Numbers ........................ 151 18.4 Central Limit Theorems ....................... 151 Bibliography 153 viii CONTENTS Preface This document was created with help from fellow graduate students Ben Kle- mens, Ming Hsu, Brian Rogers, and Isa Hafalir. Nearly all of the knowledge contained within is not my own. Instead, I used a wide variety of valuable resources which I hope to completely list in the bibliography. At this point, this is not a complete document. The word \blah" is inserted as a flag that much more could be written in a particular section. Eventually,