ECON 497: Lecture Notes 13 Page 1 of 1 Metropolitan State University ECON 497: Research and Forecasting Lecture Notes 13 Dummy Dependent Variable Techniques Studenmund Chapter 13 Basically, if you have a dummy dependent variable you will be estimating a probability. Probabilities are necessarily restricted to fall in the range [0,1] and this puts special conditions on the regression. Just doing a linear regression can result in estimated probabilities that are either negative or greater than 1 and are a bit nonsensical. As a result, there are other techniques for estimating these relationships that can generate better results. The Linear Probability Model The second or third best way to estimate models with dummy dependent variables is to simply estimate the model as you normally might: Di = β0 + β1X1i + β2X2i + εi For example, if you have a sample of the U.S. adult population and you're trying to determine the probability that a person is incarcerated, you might estimate the equation: Di = β0 + β1AGEi + β2GENDERi + εi where Di is a dummy variable taking the value 1 if a person is incarcerated and 0 if not AGEi is the person's age GENDERi is a dummy variable equal to 1 if the person is male and 0 otherwise Imagine that the estimated coefficients are: ˆ Di = 0.0043 - 0.0001*AGEi + 0.0052*GENDERi Interpretation of the estimated coefficients is straightforward. If there are two women, one of whom is one year older than the other, the estimated probability that the older one will be incarcerated will be 0.0001 less than the estimated probability that the younger one will be. ECON 497: Lecture Notes 13 Page 2 of 2 If there are a man and a woman of the same age, the predicted probability that the man will be incarcerated is 0.0052 greater than the predicted probability that the woman will be incarcerated. Interestingly, a woman of age 43 will have a predicted probability of 0.0000 and women older than this will have negative predicted probabilities of incarceration. Studenmund describes issues regarding the linear probability model and you should read this discussion. One thing I will point out is that the adjusted R2 is not an accurate measure of overall fit in a linear probability model with dummies as dependent variables. The Weighted Least Squares Approach The most complicated point from Studenmund's discussion of the linear probability model is the discussion of weighted least squares. This technique is designed to get around the problem of heteroskedasticity (which we haven't really discussed yet) and can be summarized as follows: 1. Due to the structure of the linear probability model, the error terms are not identically distributed. Specifically, error terms will have greater variance when the actual probability is close to 0.5 and smaller variance when the actual probability is close to zero or one. Because all of the error terms are not identically distributed (they have different variances) there is a problem with heteroskedasticity. Coefficient estimates, however, will be unbiased as long as the other classical assumptions are satisfied. 2. To address this problem, do the standard linear regression and then use the estimated ˆ coefficients to generate the predicted probabilities ( Di ) for each observation. Excel will do this for you if you ask it nicely. ˆ 3. Use these predicted probabilities ( Di ) to generate a new value, which is equal to the ˆ ˆ square root of Di *(1- Di ). Call this ˆ ˆ 1/2 Zi = [ Di *(1- Di )] 4. Divide the dependent and explanatory variables by this new variable (Zi). 5. Now, redo the regression using the values of the dependent and explanatory variables, which have been divided by Zi. The standard errors and the t-statistics for the estimated coefficients will be different and more accurate. Briefly, the idea behind this is that observations whose error terms have greater variance should be have less influence than do those whose error terms have smaller variance. The ECON 497: Lecture Notes 13 Page 3 of 3 ˆ closer to 0.5 Di is, the larger the variance of the error term is likely to be, so the observations are weighted by ˆ ˆ 1/2 ˆ ˆ -1/2 1/[ Di *(1- Di )] = [ Di *(1- Di )] . The Binomial Logit Model The proper way to estimate these models is by using the binomial logit model. To do this, the dependent variable needs to be transformed. The equation to be estimated is: Di ln = β0 + β1X1i + β2X2i + εi 1− Di The dependent variable is the log of the odds ratio and is equal to infinity if Di=1 and is equal to negative infinity if Di=0. The predicted probability is equal to ˆ 1 Di = − βˆ +βˆ X +βˆ X 1+ e ()0 1 1i 2 2i The interpretation of the estimated coefficients is less straightforward here. Estimated coefficients show the effect of a change in an explanatory variable on the predicted log of the odds ratio, not on the probability itself. Basically, you can only tell whether an explanatory variable has a positive or negative impact on the probability, not how large that impact is. An additional complication is that logit models cannot be estimated using OLS, so they can't really be done in Excel. This is something you need a real statistical analysis package to do. Examples solicited from students. Binomial Probit Model This is a model based on some slightly different assumptions than the binomial logit model. In most cases the results from the two models are nearly identical. If you're estimating either a logit or a probit model, it's usually just one additional command to also estimate the other. You should do this, just for completeness and to check that your important results are robust to changes in the model used. ECON 497: Lecture Notes 13 Page 4 of 4 If a presenter is annoying you in some way as they discuss their binomial logit or binomial probit results, you can make yourself equally annoying by asking if they estimated the other model and if their results were robust to the change. This is kind of a cheap question and it really shouldn't disturb them too much because, if they've been even slightly responsible, they will have done both. The real different between the Logit and Probit models is that they have slightly different assumptions about the distribution of the underlying probabilities. The Probit uses the cumulative distribution function of the Normal distribution while the Logit uses a linear version of the odds ratio. Here's Studenmund's take on all this: "From a researcher's point of view, the biggest differences between the two models are that the probit is based on the cumulative normal distribution and that the probit estimation procedure uses more computer time than does the logit. As computer programs are improved, and as computer time continues to fall in price, this latter difference may eventually disappear. Since the probit is similar to the logit and is more expensive to run, why would you ever estimate one? The answer is that since the probit is based on the normal distribution, it's quite theoretically appealing (because many economic variables are normally distributed). With extremely large samples, this advantage falls away, since maximum likelihood procedures can be shown to be asymptotically normal under fairly general condition." Multinomial Logit Model If you have a qualitative dependent variable that can take multiple values, you may wish to estimate a multinomial logit model. This can be a bit tricky and uncooperative, and it can potentially require a lot of computing time, a compliant data set with lots of observations of each qualitative outcome and, most importantly, a big chunk of your life and your sanity, not necessarily in that order. Basically, the results from a multinomial logit model tell you about the effect that a change in the value of a variable has on the relative probabilities of two of the possible outcomes. Doing this with some degree of reliability apparently requires a data set in which you have a couple hundred observations of each of the qualitative outcomes. Example: Voting Choice Imagine that you have voting records showing demographic information for a lot of people and who they voted for (Democrat, Republican, Libertarian) in the last election. You might use a multinomial logit model to identify the factors that have a significant impact on making someone vote Libertarian rather than Republican or Democrat. ECON 497: Lecture Notes 13 Page 5 of 5 Example: Transportation Choice Imagine that you get a hold of a transportation survey from the Puget Sound Regional Council and you want to model transportation choice of adults based on such things as income, number of children, commute distance, etc. You might use a multinomial logit model with each possible choice as one possible value of the dependent variable. .