Organizational Research Methods 00(0) 1-39 ª The Author(s) 2012 Using the Propensity Score Reprints and permission: sagepub.com/journalsPermissions.nav Method to Estimate Causal DOI: 10.1177/1094428112447816 http://orm.sagepub.com Effects: A Review and Practical Guide Mingxiang Li1 Abstract Evidence-based management requires management scholars to draw causal inferences. Researchers generally rely on observational data sets and regression models where the independent variables have not been exogenously manipulated to estimate causal effects; however, using such models on observational data sets can produce a biased effect size of treatment intervention. This article introduces the propensity score method (PSM)—which has previously been widely employed in social science disciplines such as public health and economics—to the management field. This research reviews the PSM literature, develops a procedure for applying the PSM to estimate the cau- sal effects of intervention, elaborates on the procedure using an empirical example, and discusses the potential application of the PSM in different management fields. The implementation of the PSM in the management field will increase researchers’ ability to draw causal inferences using observational data sets. Keywords causal effect, propensity score method, matching Management scholars are interested in drawing causal inferences (Mellor & Mark, 1998). One example of a causal inference that researchers might try to determine is whether a specific manage- ment practice, such as group training or a stock option plan, increases organizational performance. Typically, management scholars rely on observational data sets to estimate causal effects of the management practice. Yet, endogeneity—which occurs when a predictor variable correlates with the error term—prevents scholars from drawing correct inferences (Antonakis, Bendahan, Jacquart, & Lalive, 2010; Wooldridge, 2002). Econometricians have proposed a number of techniques to deal 1Department of Management and Human Resources, University of Wisconsin-Madison, Madison, WI, USA Corresponding Author: Mingxiang Li, Department of Management and Human Resources, University of Wisconsin-Madison, 975 University Avenue, 5268 Grainger Hall, Madison, WI 53706, USA Email: [email protected] Downloaded from orm.sagepub.com at Vrije Universiteit 34820 on January 30, 2014 2 Organizational Research Methods 00(0) with endogeneity—including selection models, fixed effects models, and instrumental variables, all of which have been used by management scholars. In this article, I introduce the propensity score method (PSM) as another technique that can be used to calculate causal effects. In management research, many scholars are interested in evidence-based management (Rynes, Giluk, & Brown, 2007), which ‘‘derives principles from research evidence and translates them into practices that solve organizational problems’’ (Rousseau, 2006, p. 256). To contribute to evidence- based management, scholars must be able to draw correct causal inferences. Cox (1992) defined a cause as an intervention that brings about a change in the variable of interest, compared with the baseline control model. A causal effect can be simply defined as the average effect due to a certain intervention or treatment. For example, researchers might be interested in the extent to which train- ing influences future earnings. While field experiment is one approach that can be used to correctly estimate causal effects, in many situations field experiments are impractical. This has prompted scholars to rely on observational data, which makes it difficult for scholars to gauge unbiased causal effects. The PSM is a technique that, if used appropriately, can increase scholars’ ability to draw causal inferences using observational data. Though widely implemented in other social science fields, the PSM has generally been over- looked by management scholars. Since it was introduced by Rosenbaum and Rubin (1983), the PSM has been widely used by economists (Dehejia & Wahba, 1999) and medical scientists (Wolfe & Michaud, 2004) to estimate the causal effects. Recently, financial scholars (Campello, Graham, & Harvey, 2010), sociologists (Gangl, 2006; Grodsky, 2007), and political scientists (Arceneaux, Gerber, & Green, 2006) have implemented the PSM in their empirical studies. A Google Scholar search in early 2012 showed that over 7,300 publications cited Rosenbaum and Rubin’s classic 1983 article that introduced the PSM. An additional Web of Science analysis indicated that over 3,000 academic articles cited this influential article. Of these citations, 20% of the publications were in economics, 14% were in statistics, 10% were in methodological journals, and the remain- ing 56% were in health-related fields. Despite the widespread use of the PSM across a variety of disciplines, it has not been employed by management scholars, prompting Gerhart’s (2007) con- clusion that ‘‘to date, there appear to be no applications of propensity score in the management literature’’ (p. 563). This article begins with an overview of a counterfactual model, experiment, regression, and endo- geneity. This section illustrates why the counterfactual model is important for estimating causal effects and why regression models sometimes cannot successfully reconstruct counterfactuals. This is followed by a short review of the PSM and a discussion of the reasons for using the PSM. The third section employs a detailed example to illustrate how a treatment effect can be estimated using the PSM. The following section presents a short summary on the empirical studies that used the PSM in other social science fields, along with a description of potential implementation of the PSM in the management field. Finally, this article concludes with a discussion of the pros and cons of using the PSM to estimate causal effects. Estimating Causal Effects Without the Propensity Score Method Evidence-based practices use quantitative methods to find reliable effects that can be implemen- ted by practitioners and administrators to develop and adopt effective policy interventions. Because the application of specific recommendations derived from evidence-based research is not costless, it is crucial for social scientists to draw correct causal inferences. As pointed out by King, Keohane, and Verba (1994), ‘‘we should draw causal inferences where they seem appro- priate but also provide the reader with the best and most honest estimate of the uncertainty of that inference’’ (p. 76). Downloaded from orm.sagepub.com at Vrije Universiteit 34820 on January 30, 2014 Li 3 Counterfactual Model To better understand causal effect, it is important to discuss counterfactuals. In Rubin’s causal model (see Rubin, 2004, for a summary), Y1i and Y0i are potential earnings for individual i when i receives (Y1i) or does not receive training (Y0iÞ. The fundamental problem of making a causal inference is how to reconstruct the outcomes that are not observed, sometimes called counterfactuals, because they are not what happened. Conceptually, either the treatment or the nontreatment is not observed and hence is ‘‘missing’’ (Morgan & Winship, 2007). Specifically, if i received training at time t,the earnings for i at t þ 1isY1i. But if i also did not receive training at time t, the potential earnings for i at t þ 1isY0i. Then the effect of training can be simply expressed as Y1i À Y0i. Yet, because it is impossible for i to simultaneously receive (Y1i) and not receive (Y0iÞ the training, scholars need to find other ways to overcome this fundamental problem. One can also understand this fundamental issue as the ‘‘what-if’’ problem. That is, what if individual i does not receive training? Hence, recon- structing the counterfactuals is crucial to estimate unbiased causal effects. The counterfactual model shows that it is impossible to calculate individual-level treatment effects, and therefore scholars have to calculate aggregated treatment effects (Morgan & Winship, 2007). There are two major versions of aggregated treatment effects: the average treatment effect (ATE) and the average treatment effect on the treated group (ATT). A simple definition of the ATE can be written as ATE ¼ EYðÞÀ1ijTi ¼ 1; 0 EðY0ijTi ¼ 1; 0Þ; ð1:1aÞ where E(.) represents the expectation in the population. Ti denotes the treatment with the value of 1 for the treated group and the value of 0 for the control group. In other words, the ATE can be defined as the average effect that would be observed if everyone in the treated and the control groups received treatment, compared with if no one in both groups received treatment (Harder, Stuart, & Anthony, 2010). The definition of ATT can be expressed as ATT ¼ EYðÞÀ1ijTi ¼ 1 EðY0ijTi ¼ 1Þ: ð1:1bÞ In contrast to the ATE, the ATT refers to the average difference that would be found if everyone in the treated group received treatment compared with if none of these individuals in the treated group received treatment. The value for the ATE will be the same as that for the ATT when the research design is experimental.1 Experiment There are different ways to estimate treatment effects other than PSM. Of these, the experiment is the gold standard (Antonakis et al., 2010). If the participants are randomly assigned to the treated or the control group, then the treatment effect can simply be estimated by comparing the mean differ- ence between these two groups. Experimental data can generate an unbiased estimator for causal effects because the randomized design ensures