The Choice Between Fixed and Random Effects

Zac Townsend Mayor’s Office, City of Newark, NJ and New York University, USA Jack Buckley National Center for Educational Statistics and New York University, USA Masataka Harada New York University, USA Marc A.Scott New York University, USA

5.1 INTRODUCTION the effect estimates for each group will be different depending on the modeling choice; In prior chapters, group-specific effects were the estimates for the predictors in the model may assumed to be drawn from a distribution, typ- be different depending on the modeling choice; ically Gaussian. In applied research in the the reliability with which one can make predictions social and behavioral sciences, economics, for new groups differs; and public health, public policy, and many other there is a tradeoff between ef ciency and bias, fields, alternatives to this choice are often with random effects being more ef cient than xed made, with the most common being the “fixed effects,but potentially biased (we defer formalizing effects” approach. In its most basic formu- our de nition of bias to a later section). lation, group-specific intercepts are modeled using indicator variables, effectively making them open parameters in the model and not The remainder of this chapter will discuss assigning them a distributional form. The these options as well as a hybrid version of choice between random and fixed effects has the two. It provides guidance as to when an implications for the interpretation of param- applied researcher should use each model, and eter estimates and for estimation efficiency.1 describes situations when neither is appropri- Specifically, ate and other models should be considered.

“Handbook Sample.tex” — 2013/4/15 — 23:01 — page 73 74 THE CHOICE BETWEEN FIXED AND RANDOM EFFECTS

We proceed by describing the two models where yij is the outcome for the ith subject in Section 5.2 before discussing the differ- in the jth group. It is also represented more ent assumptions, describing estimation, and explicitly as a multilevel model as giving advice on when to use each in Sec- D C C " ; tion 5.3. Section 5.4 outlines an example of yij 0 j 1x1ij ij (5.2) using fixed effects and random effects with where data from the National Assessment of Educa- 0 j D 0 C u0 j : (5.3) tional Progress, a series of large-scale assess- Equation (5.2) is often referred to as the ments of schoolchildren in the United States. level one equation and (5.3) is the level two equation. In both formulations, it is usually 5.2 THE FIXED EFFECTS AND assumed that .0;2 / is independent u0 j N u0 RANDOM EFFECTS MODELS 2 of "ij N.0;" /. This model proposes that the values of the outcome, yij, are based on In this chapter, we outline both random and the population-level constant, 0, the single fixed effects models. We will refer to models explanatory variable, x1ij and the associated as fixed if they model unit-specific compo- parameter 1, and a group-specific components in longitudinal data or group-specific nent, u0 j . Although the random effects model components in clustered data as separate can include more complex random compo- parameters, and random effects if they are nents such as two-way (e.g., time- and unit- drawn from a (often Gaussian) probability specific) intercepts or multiple group-specific distribution. intercepts that have a nested structure (e.g., We will delve into model specifics shortly, classrooms and schools), in this chapter we but here we mention that with grouped data consider a single group-specific intercept to there are two types of relationship that are in focus on the choice between random and fixed play: between-group and within-group. Fail- effects. ure to understand this interplay can result in Researchers using random effects models an ecological fallacy, in which one makes often focus on the variance components, such inferences about the nature of units based as 2 . It is instructive, however, to examine u0 on information about the groups containing the group-specific effects, as these serve as them. In the context of multilevel models, controls for all other effects, and thus have we will show that variation naturally divides the potential to alter the other effect esti- into within- and between-group components, mates. In the random effects model, when so that potential for misinterpretation exists 2 u0 j N.0; /, given constant 0,we whenever the two types of relationship differ u0 have N. ;2 /, which are group- for a predictor—for example, when neighbor- 0 j 0 u0 specific intercepts. Gelman and Hill (2007, hood income matters for some outcome, but p. 257) call this a “soft constraint” that relative income within that group does not. is applied to the intercepts, which shrinks Diez Roux (2004) and Wakefield (2003) give the estimates of toward the common extensive overviews (see also Gelman, 2006). 0 j mean , a phenomenon they call a par- We note that the standard ecological fallacy 0 tial pooling (see Lindley and Smith, 1972; typically occurs when one does not have both Smith 1973; Efron and Morris, 1975; Mor- unit- and group-level data available to disen- ris, 1983; and Kreft and de Leeuw, 1998, tangle the two types of effects. for some background). This term derives from the “pooled” and “unpooled” model- 5.2.1 The Random Effects Model ing approaches described in Gelman and Hill Consider the multilevel model (2007) and introduced in Chapter 1. A pooled model simply applies OLS or similar estima- D C C C " ; yij 0 1x1ij u0 j ij (5.1) tion techniques to grouped data, ignoring the

“Handbook Sample.tex” — 2013/4/15 — 23:01 — page 74 5.2 THE FIXED EFFECTS AND RANDOM EFFECTS MODELS 75 grouping. An unpooled model may be defined In Equation (5.4), the group-specific resid- in at least two ways. First, it could be under- uals (yN j 1xN1 j ) employ the 1 from the no- stood as separate regressions, one for each pooling model (or fixed effects model) and group. This would entail unique regression the overall intercept ( 0) is obtained from parameters for every predictor as well as the the complete pooling model (or OLS). Like- O intercept, and small sample sizes may yield wise, the slope estimate in Equation (5.5), 1, highly imprecise estimates. A more common reduces to that of the fixed effects model when approach is to estimate group-specific inter- D 1 and that of OLS when D 0. That . = 2/=. = 2 C cepts, effectively using one parameter per is, two weight parameters, n j y n j y group, and to pool the estimates of all other 1= 2 / and , serve as a measure of the u0 predictor effects; this definition is suggested shrinkage toward the unpooled estimators. by Gelman and Hill (2007). This is equivalent When group-level effects are estimable with to estimating the so-called fixed effects model great precision, 2 becomes large relative u0 and will be what we adopt in our subsequent 2 2 to y and " , and these weight parameters discussion. approach one, very little shrinkage occurs, The intercepts in the random effects model and the random and fixed effects estimates are a weighted average of estimates from the are nearly identical. Conversely, when 2 u0 two types of pooling. In our simple model becomes relatively small, these two weight (5.1), assuming the data are balanced, the ran- parameters approach zero, and shrinkage to dom intercepts and slope coefficients are the population-level effect (ignoring group) = 2 becomes nearly complete. O n j y O 0 .yN 1xN1 / When the assumptions in the random j n = 2 C 1= 2 j j j y u0 effects model hold, these above estimates are 1= 2 more efficient than their fixed effects coun- C u0 (5.4) = 2 C = 2 0 terparts, and this is one of the key reasons for n j y 1 u 0 0 1 choosing such models.The assumptions, how- 1 X X ever, may be more or less plausible in different O @ 2A 1 D .x1ij xN1 j / (5.5) applications, such as randomized experiments 0 i j 1 versus observational studies. See Chapter 12 X X for further discussion. @ A ¡ .x1ij xN1 j /.yij yN j / ; i j 5.2.2 Fixed Effects Model where If one wants to estimate group-specific inter- s cepts without imposing a distribution or uti- 2 " D 1 ; lizing information from the other groups, one 2 C 2 " n j u0 may choose to estimate a fixed effects model. The model can be represented in a similar way n j is the number of observations within a to equation (5.1), but with a modification to N N group (group size), y j and x1 j are within- the group effects: group means of the outcome variable and the 2 2 2 explanatory variable, and y , u , and " are XJ ¨ © " 0 0 variances of y, u0, and , respectively (Gel- yij D 0 C 1x1ijC u0 j0 I j D j C"ij; man and Hill, 2007, p. 258; Afshartous and de j0D2 Leeuw, 2005, pp. 112–13; Wooldridge, 2010, ¨ © (5.6) p. 287).2 In order to utilize these formulas with the indicator function I j D j0 track- to get estimates of the coefficients, one must ing the J 1 group-level effects (group 1 replace the variances and the intercept with taken as the reference category). The impli- their estimates. cation of this reformulation is that the group

“Handbook Sample.tex” — 2013/4/15 — 23:01 — page 75 76 THE CHOICE BETWEEN FIXED AND RANDOM EFFECTS

effects u0 j may be correlated with the other effects estimators is straightforward. By set- predictor (or predictors, more generally). ting 2 D1in (5.4) and (5.5), u0 In situations in which the orthogonality of O O the random effects and other predictors is 0 j DNy j 1xN1 j ; where 0 1 implausible, fixed effects models are one 1 alternative. X X O D @ . N /2A Another way to characterize this model in 1 x1ij x1 j the context of multilevel models is to describe i0 j 1 the random effects distribution as having infi- X X nite variance, 2 D1, so the model does ¡ @ . N /. N/A : u0 x1ij x1 j yij y j not borrow any information across groups.3 i j Although, in this sense, the fixed effects model is a special case of random effects, our sim- These fixed effects estimators are often called plest fixed effects model is not really a mul- the within estimators because they do not tilevel model in the traditional use of the attempt to explain between-group differences, term, as the group level has been reduced to and are solely based on deviations of x1ij and a set of indicators and their corresponding yij from the respective estimated group means N N effects. x1¡ j and y¡ j . Fixed effect methods control for A tradeoff that may not be immediately all stable characteristics of the groups in the apparent is that one cannot include any group- study. In a longitudinal study, this would mean level explanatory variables, such as each that fixed effects control for time-invariant group’s average income, due to collinearity characteristics of each individual, such as per- with the group fixed effect. If one is interested sonal history before the observation period. in variance components and how they change While we have briefly noted some of the with the addition of predictors at group and differences between the two estimators, our subject level, then one is severely hampered next section will provide more specific details by the fixed effects approach. However, one of those estimation procedures and then dis- may not have this type of interest. cuss the relative merits or conditions favoring Fixed effects models allow group-specific each of the techniques. components u0 j to be correlated with other covariates x1ij, which is certainly a less 5.3 A COMPARISON OF THE TWO restrictive approach. This has led some ESTIMATORS researchers to interpret these components as controls that proxy for any omitted variable This section describes the assumptions behind that is “fixed” or constant for the entire group. the fixed and random effects models, outlines However, the form of “omitted variable” that the estimation techniques of both models and may be captured this way is quite restric- provides general advice regarding when to use tive (see Baltagi, 2005; Wooldridge, 2010, and not use each technique. We then describe for some discussion of omitted variable bias some hybrid models developed that satisfy and how it relates to this model; see also the “random effects assumption” while still Chapter 12). At a minimum, it is clear that the offering some of the benefits of fixed effects omitted confounder must be time- (or group-) models. constant. Of course, one may estimate (5.6) using 5.3.1 The Assumptions of Each OLS, but in practice, a demeaning approach, Model to be described in the next section, is usually employed. Given the estimators of ran- Wooldridge (2010) states that the fixed effects dom effects model, the derivation of fixed model must satisfy the strict exogeneity

“Handbook Sample.tex” — 2013/4/15 — 23:01 — page 76 5.3 A COMPARISON OF THE TWO ESTIMATORS 77

assumption, E."ijjx1i0 j ; u0 j / D 0 for all val- bias/variance trade-off. Random effects ues of i0 given any i, to obtain an unbiased models may be biased (for the within-group fixed effects estimator in the presence of any effect) when the model assumptions fail to unobserved group-specific confounders. That hold, but they are more efficient than fixed is, "ij is not only independent of its contempo- effects models when they do. The bias is often raneous covariates but also should be indepen- framed as a form of omitted variable bias dent of the covariates that belong to the same (see Bafumi and Gelman, 2006). Specifically, 0 group (x1i0 j such that i 6D i). If the condition if there is a group-constant confounder c on "ij is not met in the fixed effects model, omitted from the model, and it is correlated then potential unobserved omitted variables with another predictor, such as x1ij, (and the could bias the estimator for x1ij’s effect. outcome), then the corresponding parameter In addition to this strong exogene- 1 will be biased when c is omitted. One of ity assumption, the random effects model motivations for using grouped data is that we assumes conditional mean independence of can recover the (within-group) 1 that would group-specific effects u0 j given all covariates have been estimated had we included the x1ij (Wooldridge, 2010). Formally, confounder c. With respect to recovering this parameter, fixed effects models are unbiased E.u0 j jx1ij/ D E.u0 j / D 0: and random effects models are efficient (this is the setup for the Hausman Test). However, This assumption is also referred to as orthogo- this statement applies only to the basic forms nality or, somewhat less formally, no correla- of these models; in Section 5.3.5 we show tion between u0 j and x1ij (Cov.u0 j ; x1ij/ D that centering can be used to achieve the 0). Usually it is further assumed that u0 j same goal for random effects models. .0;2 /. This is perhaps the fundamen- N u0 tal difference between the fixed and random 5.3.2 Estimating Each Model effects approaches: the latter assumes that u0 j are orthogonal to "ij and to any predictors Fixed effects estimation can be conducted x1ij in the model, while the former implicitly either by group demeaning both the dependent allows correlation with predictors. and explanatory variables using the within This same concern of bias under violation transformation (sometimes called the fixed of assumptions exists in the random effects effects transformation), or by adding a set of model, but the additional assumption that the dummy variables for each group j and using random effects are orthogonal to explanatory OLS, as previously described. To perform the variables may be harder to justify, particu- within transformation, we can average equa- larly where causal inference is the goal. How- tion (5.6) for each group, and then subtract the ever, in practice the researcher may wish to new group means from the original, obtaining: trade a small amount of bias for efficiency, ¡ y Ny D C x C u C " and we reiterate that the hybrid models we ij j 0 1 1ij 0 j ij ¡ present in Section 5.3.5 provide a useful com- 0 C 1xN1 j C u0 j CN" j promise approach. Causal modeling for mul- (5.7) tilevel models requires a much deeper frame- D 1.x1ij Nx1 j / C ."ij N" j /: work than we present in this chapter—see Chapter 12 of this volume for further discus- Thus, the demeaning of the original equation sion of assumptions in multilevel models for ensures that demeaned covariates are orthogo- causal inference. nal to any group effects, u0 j . It also implicitly The differences in the above assumptions— controls for group effects, or partials them out and their implications—has led some of the model.The within nature of the effects is researchers to frame the choice between evident from this formulation, for now a one- random and fixed effects models as a unit change in the demeaned covariate reflects

“Handbook Sample.tex” — 2013/4/15 — 23:01 — page 77 78 THE CHOICE BETWEEN FIXED AND RANDOM EFFECTS change relative to one’s group. Chapter 6 dis- model mentioned above, while the final step cusses this concept in some detail.The estima- for this approach to estimating a random O tion of equation (5.7) with OLS yields fixed effects model is simply to regress yij yN j on 4 O effects estimators. x1ij xN1 j . To repeat, this regression is iden- Alternatively, fixed effects estimators can tical to that of the fixed effects model when be obtained by adding a set of group indicators D 1, and that of OLS when D 0. Because as explanatory variables and applying OLS to 0 1, one way to conceptualize the ran- equation (5.6). These two methods are equiv- dom effects estimator is as the regression of alent; the interested reader can consult David- partially demeaned variables. son and MacKinnon (1993) and Wooldridge Another way of conceptualizing and then (2010). estimating random effects models is through Random effects models may be estimated a matrix representation of predictors and error in a variety of ways, including feasible gen- structure and vector notation for outcomes. eralized least squares (FGLS), maximum- This requires specification of a complex likelihood estimation (MLE), or Bayesian covariance structure, in which some between- approaches with MCMC. See Chapters 3 and element correlation must be modeled. Typi- 4 of this book, Hsiao (2003), and Gelman cally, observations are assumed independent and Hill (2007) for an overview of poten- across groups, but not within groups, resulting tial approaches to estimation. Here, we intro- in a block-diagonal structure. An example of duce a step-by-step approach to calculate ran- a classic random intercepts model is given in dom effects estimators given by Johnston Chapter 2, row 4 of Table 2.2; in matrix nota- and DiNardo (1997).5 Each step illuminates tion, using the terms X, Z, , G, and R to important properties of random effects mod- capture the fixed and random effects designs, els, but the main idea is to find the weight the fixed effects parameters, the covariance parameter, . Random effects estimators are of the random effects, and the covariance of partial-pooling estimators situated between the errors, respectively, the outcome vector no pooling (or fixed effects) and complete y N.X ; ZGZ0CR/. Estimation is usually pooling (or OLS) estimators, and deter- made using the method of maximum likeli- mines the proportion of these two quantities. hood, but Bayesian approaches are also com- The first step is to find no-pooling and mon. As a by-product of this notation, we complete-pooling estimators. This can be eas- can express the best linear unbiased predictor ily done by estimating (5.6) or (5.7) and (BLUP; see Robinson, 1991) of the random by estimating (5.1) without a group-specific effect vector for group j as: component (u ) using OLS. Second, using 0 j O the residuals obtained in the first step, we esti- uO j D E.u j jY¡ j ; X¡ j ; Z¡ j ; / (5.8) 2 2 0 0 mate " and using the following formulae: D . C / 1. /;O u0 GZ¡ j Z¡ j GZ¡ j R Y¡ j X¡ j "O0 "O 2 FE FE ¡ O" D where “ j” refers to that portion of the vector n.n j k 1/ or matrix associated with group j (over all "O0 "O O 2 associated i). In the case of a random intercept O 2 D OLS OLS " ; u0 model, Z, G, and R have simpler structure, so n k n j this becomes: where "OFE is a residual vector from the O D . j ; ; ; ;O O 2; O 2/ no-pooling model, "OOLS is that from the u0 j E u0 j Y¡ j X¡ j Z¡ j u " omplete-pooling model, and k is the num- DO 2.O 2 0 CO 2 / 1. /:O u u Z¡ j Z¡ j " I Y¡ j X¡ j ber of explanatory variables. Now, we have all the information required to calculate . While most of our discussion assumes some Remember, again, that the fixed effects model interest in estimating these effects, we reit- is defined to be the same as the unpooled erate that random effects models estimate

“Handbook Sample.tex” — 2013/4/15 — 23:01 — page 78 5.3 A COMPARISON OF THE TWO ESTIMATORS 79 the parameters governing the effects, while model, this does not necessarily mean the fixed effects models incorporate them through fixed effects model is always more robust group-specific parameters. The BLUPs are against violation of its assumptions. In par- post hoc “best guesses” of the group effects, ticular, the fixed effects model is known to be which, under likelihood theory, have asymp- more susceptible to several violations of the totic distributions, and thus known precision. regression assumptions and other problems A new group’s effect along with its precision typical of regression modeling. For example, could thus be estimated using (5.8). In the the impact of measurement errors in covari- case of fixed effects, it is possible to define an ates is amplified in fixed effects models (John- analogous prediction for a group effect from ston and DiNardo, 1997). A drawback of the residualized outcomes, but the lack of model fixed effects model is its susceptibility to the assumptions offers no shrinkage of that pre- violation of strict exogeneity. As discussed in diction, nor an estimate of its precision that Section 5.3.1, under the assumption of strict utilizes information from all groups. exogeneity, the error terms must be orthogonal to any values of the covariates that belong to the same group. Typically, this assump- 5.3.3 The Debate Between Fixed tion is violated when there is an endogenous and Random Effects covariate in panel data or when a subject’sout- come may affect another subject’s outcome Whenever a textbook introduces the random within the same group. To be fair, the random and fixed effects models, it discusses which effects model must also satisfy the strict exo- method is to be used when. However, read- geneity assumption. However, because ran- ers are sometimes perplexed by the wide dom effects estimators are derived from both range of prescriptions proposed by different within-group and between-group variations, textbooks. The goal of this subsection is to the latter of which is unaffected by the strict provide some guidance on the opinions and exogeneity assumption, the violation has the prescriptions encountered throughout the lit- potential to bias fixed effects estimators more erature. Here we provide three criteria, namely (Johnston and DiNardo, 1997). satisfaction of the assumptions, compatibil- The second criterion is whether the quan- ity with the quantities of interest, and trade- tities of interest are estimable with the fixed off between bias reduction and efficiency, on effects model. Specifically, the quantities of which researchers can base their judgment in interest obtained from a fixed effects model selecting a model. are limited in two ways. First, fixed effects The first criterion is whether the selected models cannot estimate the effect of a variable method satisfies the required assumptions. that has no within-group variation because We have pointed out that the random effects fixed effects subsume all observed and unob- assumption, or conditional independence served group-specific variation. For example, between the covariates and group-specific one cannot estimate the effect of race or gen- components, may be unrealistic. An advan- der with the panel data of repeated individ- tage to the fixed effects model is that it allows ual observations because race and gender do for group-specific components to be corre- not change over time. Also, with grouped data lated with the covariates. However, when the that contain students’ academic performance, random effects assumption is satisfied, ran- one cannot use classroom fixed effects if the dom effects models are more efficient. As students are treated on a classroom basis. Fur- we see in the next section, the Hausman test ther, even if explanatory variables have some takes advantage of this property to adjudicate within-group variance, they are often impre- between the choice of fixed or random effects. cisely estimated in fixed effects models due Although the fixed effects model requires to the additional degrees of freedom used up fewer assumptions than the random effects (Ashenfelter, et al., 2003).

“Handbook Sample.tex” — 2013/4/15 — 23:01 — page 79 80 THE CHOICE BETWEEN FIXED AND RANDOM EFFECTS

As alluded to previously, fixed effects mod- have no heterogeneity within that group (e.g., els have a limited capability to make predic- no change over time in the panel data set- tions about the distribution of an outcome ting). In grouped data, e.g., observations on variable at different levels. For example, with students in classrooms, a fixed effects model the random effects model, a researcher whose would control for quality of teachers, family interest is the effect of extended class hours background, and any experiences that class- on students’ test scores can predict the distri- mates might share. It is the promise of control- bution of the test scores at classroom, school, ling for such confounders, and the expectation or population level. With the random effects that they are always lurking, that has led some model, a researcher can incorporate the infor- researchers to endorse primarily fixed effects mation from random intercepts at different modeling with multilevel data (see, for exam- levels into the predictive distribution of the ple, Allison, 2005). outcome variable and this extends to new However, the contribution to bias reduction observations and groups. Although it is pos- with the fixed effects model must also be eval- sible to estimate fixed effects for new groups uated in terms of its loss of efficiency. Fixed in an ad hoc manner in fixed effects models, effects models lose their efficiency, particu- the precision of those estimates is not read- larly in the following two instances. First, for ily available, nor does any parameter directly a large number of groups and a small num- reflect the overall between-group (and, in ber of within-group observations, the num- more nested models, between-level) variation. ber of degrees of freedom consumed by the This observation has led some researchers to fixed effects will be large. For example, if frame the tradeoff between fixed and random a researcher has panel data that consist of effects as: random effects are samples from a 100 group observations (J D 100), the intro- population while fixed effects represent the duction of a fixed effects model reduces the entire population of interest or conditional degree of freedom by 99. Or, in the panel case, inference on the available groups in the sam- if the number of time periods is two, the fixed ple (see, for example, Mundlak, 1978). The effects model halves the degrees of freedom. basis for this characterization is classical pre- The tails of the t-distribution become thicker sentations of ANOVA; we have shown that as the degrees of freedom decreases, which the implications of the choice on parameter implies that the coefficients are estimated with estimates requires the practitioner to consider greater uncertainty. Goldstein (2003) advises additional implications of the choice. using the fixed effects model when there are The final criterion is the tradeoff between a few groups and moderately large numbers how much bias can be reduced and how of units in each. Snijders and Bosker (1999) much efficiency is lost with the fixed effects also advise using a random effects model if model. Fixed effects models add group- the number of groups is more than ten. Sec- specific indicators as explanatory variables, ond, when explanatory variables have little controlling for time-constant, homogeneous within-group variation, they are highly cor- group-specific confounding, including that related with the fixed effects, and fixed effects due to unobservables, provided the underlying estimators will be inefficient (Bartels, 2008). assumptions hold.This means that, to return to In a simple example, suppose that a researcher the example of panel data on individuals, con- is interested in estimating a policy effect using founding variables such as personal or family the panel data of 50 states in the U.S. If only 10 background before the observation period and states changed the policy during the observed any personal attributes such as race, gender, period, the fixed effects estimators are calcu- or genetics are controlled for, up to the valid- lated based on these 10 states, ignoring 40 ity of the model assumptions, primary among states in the data because fixed effects sub- which is the assumption that group effects sume all time-invariant covariates.

“Handbook Sample.tex” — 2013/4/15 — 23:01 — page 80 5.3 A COMPARISON OF THE TWO ESTIMATORS 81

To make the choice between fixed and ran- biased. Moreover, when the assumption holds, dom effects, a researcher needs to consider all the fixed effects estimator is inefficient com- of what we have discussed, such as the plau- pared to the random effects model. As the sibility of the random effects assumption and violation of the assumption becomes less seri- strict exogeneity, the need to include group- ous, the difference in point estimates between level predictors, efficiency, degrees of free- two parameters decreases in the numerator, dom, the presence and degree of measurement while the difference in variances increases in error in the data, and the amount of within- the denominator. Thus, the size of H is inter- group variance. preted as the extent of the deviation from the assumption. 5.3.4 Comparing Fixed Effects It must be noted, however, that the Haus- and Random Effects: The man test does not always provide a defini- Hausman Test tive answer. One problem is that although statistical significance of the test statistic, H, Although most of the criteria for the choice is fairly reliable evidence for the bias in the between fixed and random effects models random effects estimates, statistical insignif- require subjective judgments, test statistics icance is not necessarily evidence for unbi- have been developed for some of them. The asedness. This is because the Hausman test Hausman test (Hausman, 1978) is most fre- relies on the consistency of the two estimators. quently used to check the validity of the ran- As (5.4) and (5.5) indicate, fixed and random dom effects assumption, namely the condi- effects estimators converge only if the group tional independence between group-specific size (n j ) and/or the within-group variance ( 2 ) is large. Thus, in a small finite sample intercepts (u0 j ) and covariates (xij). Briefly u0 speaking, the Hausman test is a comparison and/or with covariates that have little within- between the parameter estimates of the fixed group variance, these two estimators may not effects and random effects models, and sup- significantly differ from each other even when ports the random effects assumption if the dif- the random effects assumption is violated, and ference between two parameters is sufficiently the Hausman test cannot distinguish between small. the satisfaction of the random effects assump- To see the logic, let us show the formal def- tion and the lack of convergence.6 A number inition of the test statistic, H, below: of studies have been conducted to improve the performance of the Hausman test. Ahn and D . O O /0. O O / 1 H RE FE VFE VRE Low (1996) refined the Hausman test to bet- O O . RE FE/; ter detect the violation when the composite error term (u0 j C "ij) contains little within- where H 2 with k degrees of freedom group variance ( 2 ). Bole and Rebec (2004) u0 (the number of time-varying coefficients in show that a bootstrap version of the Haus- O both models), denotes a vector of estimated man statistics marginally improved its perfor- O coefficients, V denotes a variance-covariance mance when the violation is minimal. To date, matrix of estimators, and the subscripts RE none has been established as a definitive alter- and FE denote the random or fixed effect native to the Hausman test. estimator as the source of the estimates. We test the null hypothesis that the results of the two estimators are not different. When the 5.3.5 The Benefits of Both random effects assumption holds, both the Models: Hybrid Models fixed and random effects estimators are consistent. However, when the assumption does The classic back and forth of fixed versus not hold, only the fixed effects estimator is random effects is that a researcher should consistent, and the random effects estimator is have less concern about bias when using fixed

“Handbook Sample.tex” — 2013/4/15 — 23:01 — page 81 82 THE CHOICE BETWEEN FIXED AND RANDOM EFFECTS

effects, while random effects are more effi- included without affecting W1,as.x1ij Nx1 j / cient. However, several interesting compro- is orthogonal to any group-constant predic- mises have been proposed. The two most tor. Second, the random intercept variance u0 prominent approaches are correlated ran- measures the remaining between-group vari- dom effects (for example, Mundlak, 1978 ation in level, and inference for this param- and Chamberlain, 1982) and adaptive center- eter is obtainable. In the fixed-effects set- ing (Raudenbush, 2009). Several other recent ting, say, using group-specific indicators, one papers (Gelman, 2006; Bartels, 2008; Ley- cannot include additional group-specific pre- land, 2010) also propose various solutions to dictors, and the inferential framework for the correlation between predictors and ran- assessing between-group variance is some- dom effects that allow researchers to more what ad hoc (we describe this in the exam- comfortably estimate varying parameters. ple). In this sense, the hybrid method has all The literature has accurately redefined the of the advantages of both random- and fixed- random effects assumption outlined above effects approaches. Allison (2005), drawing as one of cluster confounding. When only in particular on Neuhaus and Kalbfleisch a single, unadjusted version of a predictor (1998), describes how this model may be is included in a model, standard random implemented in the SAS Statistical Program- effects approaches implicitly assume that the ming language, and notes that a Wald test of within- and between-group effects are equal the hypothesis W1 G1 D 0 provides a for changes in any component of xij (Skrondal Hausman-like test of whether random effects and Rabe-Hesketh, 2004; Rabe-Hesketh and models provide the same coefficients of inter- Everitt, 2006; Zorn, 2001; Bartels, 2008). We est as the fixed effects models. can decompose any particular x1ij into differ- The above approach is also quite similar ent between- and within-group effects, which to Mundlak’s (1978) formulation, and simply B W we can call x1 j and x1ij, and the result is that adds the mean for each time-varying covari- 1 is a weighted average of the two processes. ate. This identifies the endogeneity concern as Bafumi and Gelman (2006) label this a type a result of attempting to model two processes of omitted variable bias, and go on to pro- in one term (see also Snijders and Bosker, pose group mean centering as a workaround 2012, p. 56; Berlin, Kimmel, Ten Have, and solution. This is a form of the hybrid model Sammel 1999). Others similarly argue for unit described in Allison (2005). characteristics that are correlated with the In the case of one predictor, a hybrid model mean (Clark, Crawford, Steele, and Vignoles, would be 2010). Bartels (2008) and Leyland (2010) are more interpretable reformulations of Mund- D C . N / yij 0 W1 x1ij x1 j lak (1978). Proponents of fixed effects either C G1.xN1 j / C u0 j C "ij; call this method a compromise approach or a form of the Hausman test to choose between where W1 captures the within-group effect fixed and random effects (Allison, 2009, pp. of the predictor x1, G1 captures the between- 33–5; Hsiao, 2003, pp. 44–50; Wooldridge, group effect, and u0 j is a random effect. All 2010; Greene, 2011; and others). In truth, they other terms, such as the group means xN1 j , are are essentially the same model as those argued as they have previously been defined, as are for by Bafumi and Gelman (2006), Gelman the distributional assumptions on the stochas- and Hill (2007), and throughout the multi- tic terms. The within-group effect should be level models literature: random effects mod- the same as that obtained via the fixed effects els with additional time-invariant predictors, estimator.7 Two things are unique about this which address the exogeneity assumption. approach: first, we can include group-constant We also note that in fields such as psychol- predictors, such as xN1 j , in the model. In ogy and education, group mean centering has fact, any group-constant predictors may be a long history (e.g., Bryk and Raudenbush,

“Handbook Sample.tex” — 2013/4/15 — 23:01 — page 82 5.4 AN EXAMPLE: PROFICIENCY STANDARDS AND ACADEMIC ACHIEVEMENT 83

1992; Raudenbush and Bryk, 2002), but NAEP to place the states’ performance or the connections to the econometric fixed- proficiency standards on a common metric— effects approach have only been made more the NAEP scale for a given subject and grade recently (Allison, 2009). Lastly, the practical level (NCES, 2011). This enables the creation researcher should always check the plausibil- of a panel dataset at the state level that includes ity of the assumptions behind the modeling, states’ performance on NAEP, states’ profi- no matter the name, and realize that not all ran- ciency levels or the “cutscores” that they set dom effects models place the same demands on their own state assessments mapped onto on the random effects assumption. the NAEP scale, and an additional set of time- varying covariates measuring various student demographics and other factors shown to be 5.3.6 Other Extensions related to academic outcomes. These data can If a researcher is interested in the causal be used to estimate the relationship between effects of (potentially time-dependent) treat- the stringency of states’ performance stan- ments on time-varying regressors, neither dards and the average academic achievement fixed nor random effects may be appropriate of their students. (see Sobel, 2012). The methodological chal- Specifically, the models estimated in this lenge arises because the effects of later treat- section condition the state average score on ments may themselves be outcomes of earlier NAEP grade 4 mathematics (the outcome measure) on the following covariates: instances of the treatment. Structural nested models, proposed in Robins (1993, 1994, 1997), were designed to estimate the effect the state pro ciency or performance standard for of time-varying treatments, while marginal its state grade 4 mathematics test mapped to the structural models (Robins, 1998; Robins, Her- NAEP scale (the policy variable of interest); nan, and Brumback, 2000; Robins and Her- the percentage of students in the state who are nan, 2009) are a more recently developed eligible for free or reduced price school lunch as a alternative approach. This issue is discussed measure of poverty; in Chapter 12 as well. the percentage of disabled students; the percentage of English language learners; the percentage of Black students; the percentage of Hispanic students; 5.4 AN EXAMPLE: PROFICIENCY the percentage of Asian students; and STANDARDS AND ACADEMIC the percentage of NativeAmerican orAlaska Native ACHIEVEMENT students in each state.

To illustrate the fixed and random effects approaches, we consider an open question NAEP is given at the state level every other in U.S. education policy: are more rigor- year, but state test mapping data—the policy ous or stringent performance standards on variable of interest—are currently only avail- state assessments associated with improved able for academic years 2004–5, 2006–7, and academic outcomes? Although states in the 2008–9. Thus, for these examples, T D 3. U.S. develop their own assessments and set In the following models, dichotomous indica- their own proficiency standards, the Federal tors or fixed effects for time periods are also government assesses student performance for included to control for any common trends state-representative samples using a series of among the states during this time period.Also, common tests known as the National Assess- not all data are available for all states in each ment of Educational Progress or NAEP.8 time period, but these data are assumed to The National Center for Education Statis- be missing completely at random (Little and tics publishes a series of studies that use the Rubin, 2002; see Chapter 23 of this volume).

“Handbook Sample.tex” — 2013/4/15 — 23:01 — page 83 84 THE CHOICE BETWEEN FIXED AND RANDOM EFFECTS

Table 5.1 Estimation Results: Random Effects ﬁxed effects transformation (time demeaning) Model to eliminate the time-invariant unobservables: Variable Coefficient (Std Err.) NAEPit NAEPi D lagged standard 0. 058 (0.024) .xit Nxi / % free and reduced lunch 0. 086 (0.035) % disabled 0. 000 (0.001) C .Standardi;t 1 Standardi / % ELL 0. 001 (0.001) C " N" : % Black 0. 212 (0.041) it i

% Hispanic 0. 115 (0.039) Here, we see a substantively different result % Asian 0. 119 (0.051) % Indian 0. 300 (0.108) (Table 5.2). The estimated coefficient for the 2008 0. 061 (0.363) policy variable of interest, lagged state pro- 2010 1. 278 (0.451) ficiency standard, is much smaller in magni- Intercept 236. 495 (5.577) tude and no longer statistically significant at D : N 132 conventional levels (p 175, two-tailed). u0 2.743 We note that an estimate of the between-states 2 .10/ 85.127 variation captured in this fixed effects specifi- O D : cation is much larger, at u0 9 936. This standard deviation is based directly on the 50 state fixed effects implicitly estimated in Finally, because it is theoretically unlikely that this model (values not shown). Inference for a change in state standards would immediately this variance component is made indirectly translate into a change in academic achieve- through an -test on the underlying fixed ment, the mapped state proficiency standard is F effects. It is highly significant in this case, included in each specification below as lagged but implies something about the states in the one time period (two years). sample—at least one has non-zero deviation from the mean outcome level, net of all else— 5.4.1 The Random Effects Model rather than a property of all states in the population (at other historical periods, for exam- If we are willing to assume that the unobserv- ple). This value is substantially larger than ables are uncorrelated with the other covari- that estimated in the random effects model, ates, we can estimate the model: suggesting either that there is additional state- level variation that should be modeled, or that NAEP D x C Standard ; C u C " ; it it i t 1 0i it fixed effects models are overestimating the with the random effects estimator, assuming differences we would observe in this popu- . ; / lation under some sampling mechanism. that u0i Normal 0 u0 . The results are presented in Table 5.1. The key coefficient of policy interest, , is positive and statisti- 5.4.3 The Hausman Test cally significant at conventional levels, imply- ing a positive relationship between stringency So which is the correct specification? of academic performance standards and stu- Although it does not provide a definitive dent achievement. The between-states vari- answer, the Hausman test may be useful to O D the applied researcher to guide the choice ance component is estimated to be u0 2:743. between the FE and RE models, assuming unobserved covariates. For the example data,

O O 0 O O 1 5.4.2 The Fixed Effects Approach H D . RE FE/ .VFE VRE/ . O O / An alternative means of estimating the linear RE FE unobserved effects model is by means of the D 49:78:

“Handbook Sample.tex” — 2013/4/15 — 23:01 — page 84 5.5 CONCLUSION 85

Table 5.2 Estimation Results: Fixed Effects Table 5.3 Estimation Results: Hybrid Model Model Variable Coeff. (Std Err.) Variable Coeff. (Std Err.) centered lagged standard 0.032 (0.023) lagged standard 0.032 (0.023) centered % free/reduced lunch 0.058 (0.037) % free and reduced lunch 0.059 (0.037) centered % disabled 0.000 (0.001) % disabled 0.0004 (0.001) centered % ELL 0.001 (0.001) % ELL 0.001 (0.001) centered % Black 0.254 (0.305) % Black 0.257 (0.305) centered % Hispanic 0.045 (0.234) % Hispanic 0.037 (0.234) centered % Asian 0.902 (0.386) % Asian 0.896 (0.387) centered % Indian 0.091 (0.384) % Indian 0.087 (0.385) centered 2008 0.120 (0.348) 2008 0.117 (0.349) centered 2010 0.135 (0.555) 2010 0.135 (0.556) mean lagged standard 0.164 (0.041) Intercept 237.953 (7.537) mean % free/reduced lunch 0.412 (0.050) mean % disabled 0.009 (0.004) N 132 mean % ELL 0.002 (0.003) u0 9.936 mean % Black 0.017 (0.046) F .59;72/ 2.676 mean % Hispanic 0.021 (0.035) mean % Asian 0.109 (0.042) mean % Indian 0.172 (0.094) 2 Since H with k D 10 degrees of free- mean 2008 2.869 (5.474) dom (recall, the number of time-varying coef- mean 2010 2.135 (4.139) ficients in both models), the null hypothesis Intercept 222.121 (9.988) N 132 that the difference in estimated coefficients u0 2.712 between the two models is not systematic is 2 . / 216.599 rejected at p <:0005. Thus, the results of 20 the Hausman test suggest that the fixed effects model may be accounting for important unobserved heterogeneity that may otherwise be power of group-specific predictors. Group- biasing the observed relationships. level effects can be contrasted with within- group effects through the construction of a contextual effect, whose substantive interpre- 5.4.4 The Hybrid Model tation is described in Raudenbush and Bryk (2002).A Wald test for the difference between Alternatively, as noted in Section 5.3.5 above, any two such between/within effects is analo- one may instead fit a “hybrid” model of the gous to the Hausman test, and in our example, fixed and random effects approaches: for lagged standard, the difference between 0.032 and 0.164 has an associated 2 D 7:80, NAEPit D .xit Nxi / w 1 with p D 0:005, which results in the same CN C . xi g w Standardi;t 1 inferential conclusion as the prior Hausman Standardi / test at conventional significance levels. Note that the estimates from the hybrid model are C Standard C u C " : g i 0i it sensitive to the choice of predictors, just as The results of this model, presented in in any regression, be it using fixed or random Table 5.3, are informative. The hybrid model effects for groups. yields the same coefficients for within-state effects as the fixed effects model, and very 5.5 CONCLUSION similar precision. The variance component u0 has been reduced to 2.712, a small The literature is full of advice, often conflict- drop from the random effects model’s find- ing, about when to use fixed or random effects ings, reflecting the additional explanatory models. Adding to this confusion, random

“Handbook Sample.tex” — 2013/4/15 — 23:01 — page 85 86 THE CHOICE BETWEEN FIXED AND RANDOM EFFECTS

effects models have several extensions which and 0 (Lindley and Smith, 1972; Efron and Mor- relax the random effects assumption, includ- ris, 1975). In the second technique, one develops a ing the long history of hybrid models outlined weighted least squares regression (Afshartous and de Leeuw, 2005, pp. 112 13). above. Standard random effects models can 3 This is why fixed effects are sometimes called obtain estimators that are potentially biased if ‘‘unmodeled effects’’ and random effects are called the random effects assumption is violated, but ‘‘modeled effects’’ (Bafumi and Gelman, 2006). fixed effect models come with an inability to Additionally, see Gelman (2005) or Gelman and Hill account for any time-invariant variables, dif- (2007, p. 245) for a discussion of all the conflict- ficulty in making out-of-sample predictions, ing different meanings of the term ‘‘fixed effects’’. Here, we mean the fifth definition in that list. and statistical inefficiency. While the Haus- 4 The degrees of freedom associated with the man test is often noted as a tool for evaluat- demeaning of the outcomes require an adjustment ing bias in the random effects model, Clark to the standard error of the estimated effects. The and Linzer (2012) have shown through simu- reader may also be concerned that from a multilevel lations that it may not be reliable. Ultimately, perspective, these models ignore the within-group correlation between observations. However, the evaluating the balance of bias and variance in demeaning process accounts for a group-constant the two techniques is not a simple task, but effect, which captures a specific form of within- applied researchers should not immediately group correlation, analogous to first differencing shy away from using a random effects model with pre/post designs. because of the bias concerns, as the associ- 5 The following account applies to balanced data where the group size is the same for all groups. ated variance reduction may be worthwhile. Unbalanced data require some adjustments. Our example above serves a useful guide 6 Specifically, if the within variation is small, the fixed for other potential analyses in which one fits effects estimates may not be asymptotically normal fixed effects, random effects, and hybrid mod- and this may invalidate the Hausman test (Hahn, els. By exploring the relationship between all Ham, and Moon, 2011). three estimates, researchers can more com- 7 Little has been written about the relative efficiency of this compared to the fixed effects estimator, but pletely understand the associations and poten- our own simulations suggest that the precision is tial effects of some forms of confounding. comparable in singly-nested models. 8 For more information on these assessments, including subjects and grade levels tested, see http: NOTES //nces.ed.gov/nationsreportcard/.

1 Editors’ note: In this chapter, the underlying model is effectively the same throughout, but by changing REFERENCES assumptions about model parameters and, more importantly, by changing the estimator, different Afshartous D. and de Leeuw, J. (2005) ‘Prediction goals are achieved. These competing goals are in Multilevel Models’, Journal of Educational and the subject of much debate in the applied statis- Behavorial Statistics, 30: 109 39. tics literature; this chapter describes the assump- Ahn, S.C. and Low, S. (1996) ‘A Reformulation of tions underpinning various estimators and reframes the Hausman Test for Regression Models with some of the debate in terms of between- and Pooled Cross-Section-Time-Series Data’, Journal within-group predictor effects. of Econometrics, 71: 309 19. O 2 0 j is only approximately equal to the right-hand Allison, P.D. (2005) Fixed Effects Regression Methods side of equation (5.4) because the actual fitting for Longitudinal Data Using SAS. Cary, NC: SAS techniques are slightly more complex. Statistical Publishing. programs fit the models through either Bayesian Allison, P.D. (2009) Fixed Effects Regression Models. inference or an augmented least squares technique London: Sage. (see Gelman and Hill, 2007, sections 18.3 18.4 for Ashenfelter, O., Levine, P.B., and Zimmerman, D.J. detailed descriptions on both techniques and exten- (2003) Statistics and Econometrics: Methods and sions for more complex multilevel models). In the Applications. New York: J. Wiley. first technique, one uses an iterative algorithm that Bafumi, J. and Gelman, A. (2006) ‘Fitting Multilevel alternatively estimates the particular intercepts, and Models When Predictors and Group Effects Corre- the means and variances, called the hyperparam- late’, Paper presented at the annual meeting of the eters, which in our case above includes 2 , 2, Midwest Political Science Association, Chicago, IL. u0 y

“Handbook Sample.tex” — 2013/4/15 — 23:01 — page 86 5.5 CONCLUSION 87

Baltagi, B. (2005) Econometric Analysis of Panel Data, Goldstein, H. (2003) Multilevel Statistical Models,3rd 3rd edition. New York: Wiley. edition. New York: Wiley. Bartels, B.L. (2008) ‘Beyond ‘‘Fixed Versus Random Greene, W. (2011) ‘Fixed Effects Vector Decompo- Effects’’: A Framework for Improving Substan- sition: A Magical Solution to the Problem of tive and Statistical Analysis of Panel, Time-Series Time-invariant Variables in Fixed Effects Models?’, Cross-Sectional, and Multilevel data’, Political Political Analysis 19: 135 46. Methodology Conference. Ann Arbor, MI. Avail- Hahn, J., Ham, J.C., and Moon, H.R. (2011) ‘The able at http://home.gwu.edu/˜bartels/ Hausman Test and Weak instruments’, Journal of cluster.pdf [accessed 26 August 2012]. Econometrics, 160: 289 99. Berlin, J.A., Kimmel, S.E., Ten Have, T.R., and Sammel, Hausman, J.A. (1978) ‘Specification Tests in Econo- M.D. (1999) ‘An Empirical Comparison of Several metrics’, Econometrica, 46: 1251 71. Clustered Data Approaches Under Confounding Hsiao, C. (2003) Analysis of Panel Data, 2nd edition. Due to Cluster Effects in the Analysis of Compli- New York: Cambridge University Press. cations of Coronary Angioplasty’, Biometrics, 55: Johnston, J. and DiNardo, J. (1997) Econometric 470 76. Methods. New York: McGraw Hill Co. Bole, V.A. and Rebec, P. (2004) ‘Bootstrapping the Kreft, I.G. and de Leeuw, J. (1998) Introducing Multi- Hausman Test in Panel Data Models’. Avail- level Modeling. London: Sage Publications. able at SSRN: http://ssrn.com/abstract= Leyland, A.H. (2010) ‘No Quick Fix: Understanding 628321 [accessed 26 August 2012]. the Difference Between Fixed and Random Effect Bryk, A.S. and Raudenbush, S.W. (1992) Hierarchical Models’, Journal of Epidemiology and Community Linear Models in Social and Behavioral Research: Health, 64: 1027 8. Applications and Data Analysis Methods, 1st edi- Lindley D.V. and Smith, A.F.M. (1972) ‘Bayes Estimates tion. Newbury Park, CA: Sage Publications. for the Linear Model’, Journal of the Royal Statis- Chamberlain, G. (1982) ‘Multivariate Regression tical Society, Series B (Methodological), 34: 1 41. Models for Panel Data,’ Journal of Econometrics, Little, R.J.A. and Rubin, D.B. (2002) Statistical Analysis 1: 5 46. With Missing Data, 2nd edition. New York: Wiley. Clark, T.S. and Linzer, D.A. (2012) ‘Should I Morris, C.N. (1983) ‘Parametric Empirical Bayes Confi- Use Fixed Or Random Effects?’, Unpub- dence intervals’, in Scientif ic inference, Data Anal- lished working paper. Available at http:// ysis and Robustness. Box, G.E.P., Leonard, T., and userwww.service.emory.edu/˜tclark7 Wu, C.F. (eds) New York: Academic Press, 25 50. /randomeffects.pdf [accessed 26 August Mundlak, Y. (1978) ‘Pooling of Time-Series and Cross- 2012]. Section Data’, Econometrica, 46: 69 85. Clarke, P., Crawford, C., Steele F., and Vignoles, National Center for Education Statistics (2011) Map- A. (2010) ‘The Choice Between Fixed and Ran- ping State Prof iciency Standards On to the NAEP dom Effects Models: Some Considerations for Scales: Variation and Change in State Standards Educational Research’, CMPO working paper: for Reading and Mathematics, 2005 2009, (NCES University of Bristol. Available at http:// 2011 458). Institute of Education Sciences, U.S. www.bristol.ac.uk/cmpo/publicatio Department of Education, Washington, DC. ns/papers/2010/wp240.pdf [accessed 26 Neuhaus, J.M. and Kalbfleisch, J.D. (1998) ‘Between August 2012]. and Within Cluster Covariate Effects in the Analy- Davidson, R. and MacKinnon J.G. (1993) Estimation sis of Clustered Data’, Biometrics, 54: 726 31. and Inference in Econometrics. New York: Oxford Rabe-Hesketh, S. and Everitt, B.S. (2006) A Handbook University Press. of Statistical Analyses Using STATA, 4th edition. Diez Roux, A.V. (2004) ‘The Study of Group-Level Fac- Boca Raton, FL: Chapman and Hall/CRC. tors in Epidemiology: Rethinking Variables, Study Raudenbush, S.W. (2009) ‘Adaptive Centering With Designs, and Analytical Approaches’, Epidemio- Random Effects: An Alternative to the Fixed Effects logic Reviews, 26: 104 11. Model for Studying Time-Varying Treatments in Efron, B. and Morris, C. (1975) ‘Data Analysis Using School Settings’, Education Finance and Policy,4: Stein’s Estimator and its Generalizations’, Jour- 468 91. nal of the American Statistical Association, 70: Raudenbush, S.W. and Bryk, A.S. (2002) Hierarchical 311 19. Linear Models, 2nd edition. Thousand Oaks: Sage Gelman, A. (2005) ‘Analysis of Variance--Why it is Publications. More Important than Ever’, Annals of Statistics, Robins J.M. (1993) ‘Analytic Methods for HIV Treat- 33: 1 53. ment and Cofactor Effects’, in AIDS Epidemiology Gelman, A. (2006) ‘Multilevel Modeling: What It Can Methodological Issues (pp. 213 29). Ostrow, and Can’t Do’, Technometrics, 48: 241 51. D.G. and Kessler, R. (eds). New York: Plenum Gelman, A. and Hill, J. (2007) Data Analysis Using Publishing. Regression and Multilevel/Hierarchical Models. Robins, J.M. (1994) ‘Correcting for Non-Compliance New York: Cambridge. in Randomized Trials Using Structural Nested

“Handbook Sample.tex” — 2013/4/15 — 23:01 — page 87 88 THE CHOICE BETWEEN FIXED AND RANDOM EFFECTS

Mean Models’, Communications in Statistics, 23: Smith, A.F.M. (1973) ‘A General Bayesian Linear 2379 412. Model’, Journal of the Royal Statistical Society, Robins, J.M. (1997) ‘Causal inference from Complex Series B (Methodological), 35: 67 75. Longitudinal Data’, in Latent Variable Modeling Snijders, T.A.B. and Bosker, R. (1999) Multilevel Anal- and Applications to Causality. Lecture Notes in ysis: An Introduction to Basic and Advanced Mul- Statistics (120) (pp. 69 111). Berkane, M. (ed.). tilevel Modeling. London: Sage. New York: Springer Verlag. Snijders, T.A.B. and Bosker, R. (2012) Multilevel Anal- Robins, J.M. (1998) ‘Marginal Structural Models’, ysis: An Introduction to Basic and Advanced Mul- 1997 Proceedings of the American Statistical tilevel Modeling, 2nd edition. London: Sage. Association, Section on Bayesian Statistical Sci- Sobel, M.E. (2012) ‘Does Marriage Boost Men’s ence,1 10. Wages?: Identification of Treatment Effects in Robins, J.M. and Hernan, M. (2009) ‘Estimation of the Fixed Effects Regression Models for Panel Data’, Causal Effects of Time- Varying Exposures’, in Lon- Journal of the American Statistical Association, gitudinal Data Analysis, Fitzmaurice, G., Davidian, 107: 521 9. M., Verbeke, G., and Molenberghs G. (eds). Boca Wakefield, J.C. (2003) ‘Sensitivity Analyses for Ecolog- Raton, FL: Chapman and Hall, 553 97. ical Regression’, Biometrics, 59: 9 17. Robins J.M., Hernan M.A., and Brumback B. (2000) Wooldridge, J. (2010) Economic Analysis of Cross Sec- ‘Marginal Structural Models and Causal inference tion and Panel Data. Boston: MIT Press. in Epidemiology’, Epidemiology, 11: 550 60. Zorn, C. (2001) ‘Estimating Between- and Within- Robinson, GK. (1991) ‘That BLUP Is A Good Thing: The Cluster Covariate Effect, With an Application to Estimation of Random Effects’, Statistical Science, Models of international Disputes’, International 6: 15 32. Interactions, 27: 433 45. Skrondal, A. and Rabe-Hesketh, S. (2004) General- ized Latent Variable Modeling: Multilevel, Longitu- dinal and Structural Equation Models. Boca Raton: Chapman and Hall.

“Handbook Sample.tex” — 2013/4/15 — 23:01 — page 88