The Short-Run and Long-Run Effects of Behavioral Interventions
Total Page:16
File Type:pdf, Size:1020Kb
The Short-Run and Long-Run E¤ects of Behavioral Interventions: Experimental Evidence from Energy Conservation Hunt Allcott and Todd Rogers May 30, 2013 Abstract Interventions to a¤ect behaviors such as smoking, exercise, workplace e¤ort, or pro-social behaviors can often have large short-run impacts but uncertain or disappointing long-run e¤ects. We study the three longest-running sites of a larger program designed to induce energy conser- vation, in which home energy reports containing personalized feedback, social comparisons, and energy conservation information are being repeatedly mailed to more than six million house- holds across the United States. We show that treatment group households reduce electricity use within days of receiving each of their initial few reports, but these immediate responses decay rapidly in the months between reports. As more reports are delivered, the average treat- ment e¤ect grows but the high-frequency pattern of action and backsliding attenuates. When a randomly-selected group of households has reports discontinued after two years, the e¤ects are much more persistent than they had been between the initial reports, implying that households have formed a new "capital stock" of physical capital or consumption habits. We show how assumptions about long-run persistence can be important enough to change program adoption decisions, and we illustrate how program design that accounts for the capital stock formation process can signi…cantly improve cost e¤ectiveness. JEL Codes: D03, D11, L97, Q41. Keywords: Energy e¢ ciency, habit formation, persistence, social comparisons. ———————————————————————————— We thank Ken Agnew, David Cesarini, Gary Charness, Paul Cheshire, Xavier Gabaix, Francesca Gino, Uri Gneezy, Judd Kessler, Sendhil Mullainathan, Karen Palmer, Charlie Sprenger, sta¤ at the utilities we study, and a number of seminar participants for feedback and helpful conversations. Thanks also to Tyler Curtis, Lisa Danz, Rachel Gold, Arkadi Gerney, Marc Laitin, Laura Lewellyn, Elena Washington, and many others at Opower for sharing data and insight with us. We are grateful to the Sloan Foundation for …nancial support of our research on the economics of energy e¢ ciency. Stata code for replicating the analysis is available from Hunt Allcott’swebsite. Allcott: NYU Department of Economics. [email protected]. Rogers: Harvard Kennedy School. [email protected]. 1 1 Introduction When making choices such as whether or not to smoke, exercise, eat healthfully, work or study hard, pay bills on time, and conserve resources, our choices sometimes di¤er from those that would maximize social welfare, or perhaps even our own long-run welfare. Individuals, employers, and government agencies have therefore experimented with many di¤erent kinds of interventions to "improve" behaviors, including …nancial incentives, information provision, commitment contracts, appeals to the public good, and social comparisons. In an e¤ort to produce useful and timely insights about these interventions, evaluations often only examine short-run e¤ects. The studies that do examine long-run e¤ects often …nd that it is di¢ cult to sustainably change behaviors.1 Given this, several related questions are crucial to designing and evaluating programs aimed at changing human behaviors. First, do people continue to respond to repeated intervention, or do we eventually stop paying attention? Second, how persistent are e¤ects after the intervention ends? Third, do longer interventions cause more persistent post-intervention e¤ects? These questions often determine whether an intervention is cost-e¤ective, and understanding the answers can help optimize program design. This paper is a comprehensive study of the short-run and long-run e¤ects of an intervention that aims to induce energy conservation by sending home energy reports that feature personalized feedback, social comparisons, and energy conservation information. The reports are mailed to households monthly or every few months for an inde…nite period of time. The program, which is managed by a company called Opower, is typically implemented as a randomized control trial, giving particularly credible estimates of its e¤ects on energy use. Opower’s programs are being implemented at 85 utilities across the United States, and there now 8.9 million households in their experimental populations, including 6.2 million in treatment. Utilities hire Opower to implement the intervention primarily because the resulting energy savings help to comply with state regulations requiring energy conservation. We study three Opower programs with three important features. First, these are the three longest-running Opower programs, having begun between early 2008 and early 2009. By the end of 1 For example, see Cahill and Perera (2009) for a review of the long-run e¤ects of interventions to encourage smoking cessation, as well as a number of studies of exercise, weight loss, school performance, and other behaviors that we discuss later in the introduction. 2 our sample, some treatment group households have received home energy reports for 60 consecutive months, and one might wonder whether people attenuate to this repeated intervention over such a long time horizon. Second, a randomly-selected subset of treatment group households was dropped from treatment after about two years, allowing us to measure the persistence of e¤ects for two to three years after the intervention stops. Third, while most utilities manually record household electricity use on a monthly basis, one of our three utilities uses advanced meters that record consumption each day. Although in recent years, millions of households have been out…tted with similar "smart meters" (Joskow 2012, Joskow and Wolfram 2012), the granularity of these data has generated privacy concerns that make them especially di¢ cult to acquire for research. At this site alone, we have 225 million observations of daily electricity use at 123,000 de-identi…ed households. The data reveal a remarkable pattern of "action and backsliding": consumers reduce electricity use markedly within days of receiving each of their initial reports, but these immediate e¤orts decay at a rate that might cause the e¤ects to disappear in well under a year if the treatment were not repeated. However, this cyclical pattern of action and backsliding attenuates after the …rst few reports: the immediate consumption decreases after report arrivals become much smaller, and the decay rate between reports becomes statistically indistinguishable from zero. Although the short-run pattern of cyclical responses attenuates, the average treatment e¤ects do not: for the groups that continue to receive reports at the same frequency throughout our samples, the e¤ects continue to grow - even after 60 consecutive months. Compared to behavioral interventions in other domains, the e¤ects are remarkably persistent. For the groups whose reports are discontinued after about two years, the e¤ects begin to decay at about 10 to 20 percent per year. Furthermore, this decay rate is at least six times slower than the decay rate between the initial reports. This di¤erence implies that as the intervention is repeated, people gradually develop a new "capital stock" that makes the e¤ects persistent. This capital stock might be physical capital, such as new energy e¢ cient lightbulbs, or "consumption capital" - a stock of energy use habits in the sense of Becker and Murphy (1988). Tangibly, what are consumers doing in response to the intervention? To answer this, we analyze detailed surveys of 6000 households in treatment and control groups across …ve sites. The results suggest that the intervention makes very little di¤erence on the "extensive margin": that is, the set 3 of energy conservation actions that households report taking. Although these self-reports should be interpreted very cautiously, one interpretation is that some of the behavior changes are on the "intensive margin," by which we mean that the program motivates households to take more of the same actions that they were already taking. The surveys do suggest that the intervention increases participation in other utility-run energy e¢ ciency programs, which involve improvements to physical capital stock, such as insulation or refrigerators, that would mechanically generate persistent energy savings. To corroborate this, we analyze two utilities’administrative data on program participation by treatment and control groups. We show that the intervention does cause statistically signi…cant increases in utility pro- gram participation, but these di¤erences are not economically large: they explain only a small share of the treatment e¤ects. Our interpretation of the treatment e¤ect patterns, the surveys, and the program participation data is that instead of responding in one or two major ways, di¤erent households respond by taking a wide variety of actions, no one of which constitutes an important share of the total reductions by the treatment group. After presenting the empirical results, we turn to the economic implications. We show how long-run persistence can play an important role in whether or not it is cost-e¤ective to adopt an intervention. In these three sites, di¤erent assumptions about persistence would have changed the ex-ante predicted cost e¤ectiveness by a factor of two or more. One broader implication is that when deciding whether or not to scale up other interventions when long-run results such as these are not available, it will in some cases be optimal to …rst measure whether short-run e¤ects are persistent - even if this measurement delays the decision-making process. In our context, it turns out that analysts had systematically made assumptions that were too conservative, but in other settings this may not be the case. We also show how the dynamics of persistence can play an important role in designing behavioral interventions. In doing this, we conceptually distinguish two separate channels through which repeated intervention changes cumulative outcomes. The additionality e¤ect re‡ects the extent to which repeated intervention increases e¤ects during the treatment period, holding constant the post-intervention decay rate.