JOSEPH PRICE Brigham Young University GORDON B. DAHL University of California, San Diego* Using Natural Experiments to Study the Impact of Media on the Family The randomized trial is the gold standard in including television, movies, print and digital scientific research and is used by several fields advertisements, and the Internet (Pergrams & to study the effects of media. Although useful Zaradic, 2006). Some of these media may have for studying the immediate response to media negative effects, whereas other types of media exposure, the experimental approach is not well may have positive effects. Social scientists from suited to studying long-term effects or behavior a variety of disciplines have used different outside the laboratory. The ‘‘natural experi- techniques to model and estimate these impacts. ment’’ approach, a quasi-experimental method In this paper, we discuss what economists commonly used by economists, exploits settings have learned about these questions through in which there is naturally occurring variation the use of ‘‘natural experiments.’’ We view that ‘‘randomly’’ influences the amount and type the economist’s approach as complementary to of media that individuals are exposed to. We dis- those in other disciplines (and also recognize cuss the methodological issues of natural exper- its occasional use by other disciplines). A iments and the ways in which findings based primary aim of this article is to encourage a on this approach can complement the findings more interdisciplinary approach to evaluating from laboratory or other experimental settings. the impact of the media on the family. The examples we discuss highlight the value of The gold standard in scientific research is interdisciplinary work in better understanding the randomized controlled trial. In a randomized media’s impact on family issues such as fertility, controlled trial, subjects are randomly assigned divorce, domestic violence, and child well-being. to a treatment or control group. By construction, a correctly implemented randomized experiment The question of how media affects families is yields a causal estimate of the effect of treatment important from both a scientific and practical on the outcome of interest because it balances all perspective, as it can help inform the decisions of of the potential confounding factors between the both practitioners and parents. Family members two groups. There are, however, limitations to are increasingly exposed to a variety of media, real randomized trials. First, many experiments would be unethical or immoral. Second, many important outcome measures are impractical to measure in a laboratory. Third, laboratory Department of Economics, Brigham Young University, and especially field experiments can be very Provo, UT 84602. ([email protected]). expensive to administer, especially for research *Department of Economics, University of California, San questions that involve large interventions or Diego, La Jolla, CA 92093 ([email protected]). require large samples. Attrition is also a major Key Words: family, media, natural experiments problem in long-term experimental studies. Family Relations 61 (July 2012): 363 – 373 363 DOI:10.1111/j.1741-3729.2012.00711.x 364 Family Relations When laboratory or field experiments are not for a better understanding of media’s effect on possible, economists have tried to take advantage families. We then provide an example of a of ‘‘natural experiments.’’ At the broadest level, natural experiment that cannot be well studied in a natural experiment is a situation that is thought the laboratory, namely, the effect of unexpected, to approximate a randomized trial even though negative emotional cues on domestic violence. there was no actual randomization involved In the second half of the paper, we turn to the (DiNardo, 2008). Economists have searched for arguably more important question, namely, the settings in which there is naturally occurring long-term effects of repeated media exposure variation in treatment, for example, a sudden on families. We explore how economists’ use of change in public policy (e.g., law changes), natural experiments adds to our understanding of random environmental shocks (e.g., weather), the long-term causal effects of media exposure, or other unexpected events (such as the birth of especially television exposure. The varied twins). The setting must be one where treatment natural experiments we discuss show that long- is arguably independent of all the confounding term exposure has both positive and negative factors that might otherwise bias an estimate. effects on families. In this section, we provide Natural experiments are a quasi-experimental several examples of how economists evaluate the methodology, and as such the key question validity of a natural experiment. Throughout, we for internal validity is whether the identifying point out the benefits and potential challenges to variation (the ‘‘treatment’’) is actually random. the use of natural experiments. Except in rare cases, there are no direct statistical tests to establish internal validity. This is because NATURAL AND LABORATORY EXPERIMENTS it is impossible to test whether treatment is IN THE SHORT RUN correlated with an unobserved variable. Instead, the researcher must provide arguments and Whereas natural experiments have a long history indirect tests in an attempt to convince the in economics (Meyer, 1995), the use of natural reader that the natural experiment can be used to experiments by economists to study the effects estimate a causal effect. To start, the researcher of media is more recent. An early pioneer lays out why the natural experiment occurred and in the use of natural experiments to study why it is likely to introduce random variation the effect of media is the sociologist David in treatment. The researcher then continues by Phillips. He initially called his settings ‘‘found empirically ruling out alternative explanations experiments,’’ although the idea is the same with a variety of indirect and placebo tests as a natural experiment. In a series of papers, as falsification exercises. Although the indirect he documented that suicide rates increase after tests will vary from application to application, mass media reporting of suicides, including some of the more common ones include an increase in teenage suicides (Phillips & showing (a) the treatment occurs before the Carstensen, 1986). In similar papers, Phillips outcome, (b) the treatment cannot be predicted found that motor vehicle accident fatalities by lagged outcomes, (c) there are no differential also increase after newspaper stories about preexisting trends in the outcome, and (d) the suicide (Phillips, 1979). Phillips has also used observed covariates are balanced in the treatment a similar methodology to study the impact of and control samples. Often, supporting evidence mass media violence on homicides, using both comes from analysis of auxiliary data. fictional television stories and news reporting as In this paper, we develop the idea of natural natural experiments (Phillips, 1983; Phillips & experiments as we discuss the contributions of Hensley, 1984). In each case, the idea behind economists to the literature on media’s causal Phillips’s natural experiments is that, once effect on families. We begin by comparing and seasonal patterns have been controlled for, media contrasting natural experiments to those done reports of suicides, murder, or other violent in the laboratory, which, by design, estimate behavior should be random events. The spikes short-term effects. In this first section, we focus in homicides and suicides following such events on two examples. We start with the effect of can be argued to be the causal result of media exposure to violent movies in the laboratory exposure. He argued that these effects operate versus the field. We outline why the two through imitation or suggestion. By their design, approaches identify different effects and why these experiments are well suited to estimate these complementary approaches are important short-run effects but not long-run impacts. Using Natural Experiments 365 One reason to conduct natural experiments the audience for blockbuster movies is high. is that they serve as useful complements to Broken down by time of day, in the evening laboratory experiments. Over the past several hours when people are in the theaters, there is a decades, laboratory experiments have greatly significant negative effect on crime for strongly added to our understanding of the causal effects and mildly violent movies; violent crime falls of media in the short run. The experimental lit- by around 1.2% per million violent movie erature using the laboratory, largely conducted goers. In contrast, weekends with nonviolent by psychologists, has focused heavily on study- blockbuster movies have small and insignificant ing the effect of media violence on aggressive effects. The authors interpreted this first effect behavior, both for violent video clips and violent as ‘‘voluntary incapacitation.’’ On evenings video games (see Anderson & Bushman, 2001; with high attendance at violent movies, many Anderson et al. 2003; Bushman & Anderson, potential violent criminals choose to be in 2002). This experimental literature has con- a movie theater and are incapacitated from vincingly demonstrated a causal link between committing a crime during the movie. violent media on aggressiveness immediately In the night-time hours after a violent movie after exposure, including a sharp increase in is