Causal Inference for Multiple Non-Randomized Treatments Using Fractional Factorial Designs ∗
Total Page:16
File Type:pdf, Size:1020Kb
Causal Inference for Multiple Non-Randomized Treatments using Fractional Factorial Designs ∗ Nicole E. Pashley Marie-Ab`eleC. Bind Rutgers University Massachusetts General Hospital March 17, 2021 Abstract We explore a framework for addressing causal questions in an observational setting with multiple treatments. This setting involves attempting to approximate an experi- ment from observational data. With multiple treatments, this experiment would be a factorial design. However, certain treatment combinations may be so rare that there are no measured outcomes in the observed data corresponding to them. We propose to conceptualize a hypothetical fractional factorial experiment instead of a full facto- rial experiment and lay out a framework for analysis in this setting. We connect our design-based methods to standard regression methods. We finish by illustrating our approach using biomedical data from the 2003-2004 cycle of the National Health and Nutrition Examination Survey to estimate the effects of four common pesticides on body mass index. Keywords: potential outcomes; interactions; joint effects; observational studies; multi- ple treatments; Neymanian inference 1 Introduction What is the effect of exposing you to pesticide A, compared to no exposure, on your health? arXiv:1905.07596v4 [stat.ME] 15 Mar 2021 What about exposing you to pesticide B? Or exposing you to both pesticides at the same ∗Email: [email protected]. The authors thank Zach Branson, Kristen Hunter, Kosuke Imai, Xinran Li, Luke Miratrix, and Alice Sommer for their comments and edits. They also thank Donald B. Rubin and Tirthankar Dasgupta for their insights and prior work that made this paper possible. Addi- tionally, they thank members of Marie-Ab`eleBind's research lab and Luke Miratrix's C.A.R.E.S. lab, as well as participants of STAT 300, the Harvard IQSS Workshop, and the Yale Quantitative Research Methods Workshop for their feedback on this project. Marie-Ab`eleBind was supported by the John Harvard Distin- guished Science Fellows Program within the FAS Division of Science of Harvard University and is supported by the Office of the Director, National Institutes of Health under Award Number DP5OD021412. Nicole Pashley was supported by the National Science Foundation Graduate Research Fellowship under Grant No. DGE1745303. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or the National Science Foundation. 1 time? To answer these questions, we need to be able to assess the impact, including inter- actions, of multiple treatments. A factorial experiment involves random assignment of all possible treatment combinations to units and can be used to help understand these different effects. There is much interest in assessing the effects of multiple treatments, as reflected by the recent growth in the literature regarding the use of factorial designs in causal in- ference (e.g., Branson et al., 2016; Dasgupta et al., 2015; Dong, 2015; Egami and Imai, 2019; Espinosa et al., 2016; Lu, 2016a,b; Lu and Deng, 2017; Mukerjee et al., 2018; Zhao et al., 2018). However, the literature primarily focuses on randomized experiments and in some cases (likely including the motivating example), a randomized experiment is infeasible and instead we must rely on observational studies. In this paper, we develop a Neymanian framework to draw causal inferences in observational studies with multiple treatments. Standard analysis methods for this setting tend to be model-based. In particular, regres- sion models with interaction terms are commonly used in observational studies to estimate the effects of multiple treatments (Bobb et al., 2015; Oulhote et al., 2017; Patel et al., 2010; Valeri et al., 2017). For instance, Bobb et al. (2015) considers a Bayesian kernel machine regression for estimating the health effects of multi-pollutant mixtures. However, the use of regression without a careful design phase, in which one tries to uncover or approximate some underlying experiment, can lead to incorrect conclusions (e.g., see Rubin, 2008). We thus view the problem of estimating causal effects with multiple treatments through an experimental design perspective. With a single binary treatment (or equivalently a single factor with two levels), conceptualizing observational studies as plausible treatment-control hypothetical randomized experiments has long been a common strategy (Rosenbaum, 2002; Rubin, 2008; Stuart, 2010; Bind and Rubin, 2019). Approximation of an experiment is achieved via a conceptualization stage and a design stage. In the design stage, treatment groups are balanced with respect to covariates, in an attempt to replicate balance achieved in a randomized experiment (Rubin, 2008; Imbens and Rubin, 2015; Bind and Rubin, 2019). Covariate balance is important not only for the unconfoundedness (Rosenbaum, 2002, Chap- ter 3), but also to increase plausibility of assumptions that allow the analysis stage to be implemented with limited model extrapolation (Imbens and Rubin, 2015, Chapter 12). For observational data with multiple contemporaneously applied treatments, a factorial 2 design is a natural choice of experimental design. There are some extensions of matching techniques for a single binary treatment to multiple treatments in the literature. For instance, Lopez and Gutman (2017) discuss matching for obtaining causal estimates in observational studies with multiple treatments, although they focus on a single factor with multiple levels, rather than multiple treatments that may be applied contemporaneously. Nilsson (2013) considers matching using the generalized propensity score (GPS) (Hirano and Imbens, 2004) in a 22 factorial setting. However, further exploration of matching and other methods, such as weighting, that attempt to approximate an experiment in observational studies with factorial structure is still needed. We discuss obtaining covariate balance further in Section 5.2. An important issue when using model-based approaches for multiple treatments in the observational setting, in addition to the usual concerns that we attempt to address in the design phase, is that we may have limited or no data available for some treatment com- binations. If a single treatment combination has no measurements, then the recreation of a full factorial design is not possible. When there are only one or two observations for a certain treatment combination, utilizing a factorial design would rely heavily on those few individuals being representative. Using design-based methods to address the issue of data limitations implies conceptualizing an experimental design that fits the observed data. We propose to embed an observational study into a hypothetical fractional factorial experiment, a design that uses only a subset of the treatment combinations in the full factorial design. We additionally discuss an alternative, more flexible design called incomplete factorial, which also uses a subset of the total treatment combinations. With missing data, linear or additive regression models, often used in practice, have implicit assumptions resulting in estimators that are not always transparent, especially with respect to the implicit imputation of the missing potential outcomes. We discuss regression estimates in such setting in Section 4 and discuss how they connect to design-based estimates throughout the paper. In summary, we discuss the estimation of the causal effects of multiple non-randomized treatments, in particular when we do not observe all treatment combinations. We make a number of contributions. Firstly, we build on the potential outcomes framework to consider causal effects with multiple treatments in observational settings. Secondly, we identify and 3 explore two designs (and their subsequent analyses) that are useful when we have lack of data issues in our observational study. Thirdly, we discuss implications of our estimation strategies in terms of what can be estimated, and compare these to what occurs with re- gression estimation. Finally, we identify challenges that still need to be addressed in this area. The paper proceeds as follows: Section 2 reviews full factorial designs within the poten- tial outcomes framework described in Dasgupta et al. (2015). Section 3 reviews extensions of this framework to fractional factorial designs and expands upon current inference results for variance and variance estimation. Section 4 explores the use of incomplete factorial designs. Section 5 examines how to embed an observational study into one of these experimental designs. Connections to regression-based methods are noted in each of the previously men- tioned sections. Section 6 illustrates our method and the challenges when working with observational data with multiple treatments with an application examining the effects of four pesticides on body mass index (BMI) using data from the National Health and Nutri- tion Examination Survey (NHANES), which is conducted through the Centers for Disease Control and Prevention (CDC). Section 7 concludes. 2 Full factorial designs 2.1 Set up We work in the Rubin Causal Model (Holland, 1986), also known as the potential outcomes framework (Splawa-Neyman, 1990; Rubin, 1974). Throughout this paper, we focus on two- level factorial and fractional factorial designs. We closely follow the potential outcomes framework for 2K factorial designs proposed by Dasgupta et al. (2015). We start by reviewing the notation and basic setup for factorial experiments.