Improving Environmental Performance Assessment a Comparative Analysis of Weighting Methods Used to Evaluate Chemical Release Inventories
Total Page:16
File Type:pdf, Size:1020Kb
Join an e-mail alert list and receive the latest JIE table of contents and news, visit: http://mitpress.mit.edu/jie/e-mail To subscribe to JIE, visit: http://mitpress.mit.edu/jie_subscribe This article is provided courtesy of The MIT Press. RESEARCH AND ANALYSIS Improving Environmental Performance Assessment A Comparative Analysis of Weighting Methods Used to Evaluate Chemical Release Inventories Michael W. Toffel and Julian D. Marshall Keywords Summary emissions environmental health Managers, management scholars, regulators, nonprofit orga- environmental management nizations, and the media are increasingly using emissions in- indicators ventory data to measure organizations’ environmental perfor- metrics mance. Whereas some analysts use total mass emitted, others toxic release inventory (TRI) have applied one or more of the growing number of toxicity- weighting databases aimed at predicting the environmental and health impacts of emissions. Little research is available to guide analysts in selecting among these databases. This article compares 13 methods in terms of their sophistication, com- plexity, and comprehensiveness. Seven of these methods are then evaluated as to their usefulness in weighting emissions data from the U.S. Environmental Protection Agency’s (U.S. EPA’s) toxic release inventory, and three pair-wise compari- sons are conducted. We recommend the U.S. EPA’s Risk Screening Environmental Indicators for estimating impacts to human health. We recommend the Tool for the Reduction and Assessment of Chemical Impacts for estimating impacts to human health and the environment. Address correspondence to: Michael W. Toffel Haas School of Business University of California 545 Student Services Building #1900 Berkeley, CA 94720 USA ͗[email protected]͘ ᭧ 2004 by the Massachusetts Institute of Technology and Yale University Volume 8, Number 1–2 http://mitpress.mit.edu/jie Journal of Industrial Ecology 143 RESEARCH AND ANALYSIS Introduction ground injection when their amounts exceed a minimum reporting threshold (U.S. EPA 2002h, A growing body of research in the manage- 2003c). TRI also requires that companies report ment literature investigates the motivations and off-site transfers (e.g., to waste handlers or waste implications of organizations’ environmental processors). Compared to other environmental management practices and strategies. For exam- performance information, TRI—like many ple, researchers have recently begun examining PRTRs—offers researchers the unique combi- the diffusion of voluntary environmental stan- nation of consistently reported facility-level data dards such as the International Organization for that are required by regulations, publicly avail- Standardization (ISO) ISO 14001 standard and able, and free. TRI applies to a wide array of in- the chemical industry’s Responsible Care pro- dustries and consists of a panel of thousands of gram. Many of these researchers create facility- facilities reporting annual data since 1987. level measures of environmental performance Yet creating an environmental performance based on pollutant release and transfer registries metric from TRI data remains problematic for (PRTRs) such as the U.S. Environmental Pro- several reasons: tection Agency’s (U.S. EPA’s) toxic release in- • Data accuracy is uncertain because neither ventory (TRI) program. The TRI program is one the U.S. EPA nor state environmental pro- of many PRTRs that have emerged around tection agencies routinely validate TRI the world. Australia, Canada, Korea, the Slovak data.1 One study of 60 facilities in three Republic, and the European Union nations op- industries found errors of up to 40% in re- erate PRTRs and publicly disseminate collected ported TRI emissions (U.S. EPA 1998a). data at the facility level (OECD 2001a, 2001b; • Changes in U.S. EPA instructions contrib- European Environmental Agency 2003). uted to “a significant portion of the re- Facility-level performance metrics are often used ported reductions” in TRI’s early years to compare the facility’s performance over time (U.S. GAO 1994, 3). or relative to other facilities. Developing a mean- • Because some TRI data are estimated ingful performance metric from TRI data remains rather than measured, apparent cross- problematic, however, for reasons discussed be- sectional and longitudinal variations in re- low. The purpose of this article is to assess various leases can result from different estimation weighting methods that have been or could be methods (U.S. EPA 2002k; U.S. GAO used to aggregate emissions inventory data and 1994). to recommend which schemes are most appro- priate for use with TRI data to develop a facility- The accuracy of TRI data remains an open level environmental performance indicator. issue, and we mention it only to alert researchers contemplating using this database. The chal- lenges in estimating emissions and characterizing TRI Overview their uncertainty have been discussed by others In the TRI program, the U.S. EPA requires (Frey and Small 2003). Instead, we focus on a facilities to report releases and transfers of spe- second issue regarding the use of TRI data: ag- cific chemicals if the facility (1) is primarily en- gregation techniques. gaged in manufacturing, mining, electric utilities, hazardous waste treatment, or chemical distri- Aggregation bution; (2) has ten or more full-time employees; and (3) manufactures, imports, processes, or oth- The potential harm caused by a particular erwise uses any of the listed toxic chemicals in amount of a chemical released to the environ- amounts greater than their threshold quantities ment depends on a number of factors, including (U.S. EPA 2002g). Currently, the TRI program the properties of the chemical and the medium requires companies to report emissions of 609 to which it is released. Simply summing annual substances (579 chemicals and 30 chemical cate- emissions of all TRI substances released by a fa- gories) to air, surface water, land, and under- cility in a given year is a poor proxy for its ag- 144 Journal of Industrial Ecology RESEARCH AND ANALYSIS gregate potential harm to human health or the The Importance of Weighting Schemes environment because the toxicity of TRI chem- More rigorous approaches weight toxic emis- icals varies over more than 6 orders of magnitude sions in terms of relative harm (e.g., based on (Horvath et al. 1995). In summary, “mass is a their toxicity to humans, relative to a reference crude proxy for environmental effect” (Lifset chemical) before aggregating them. TRI emis- 2001, 1). sions data should be weighted before comparing Unfortunately, this raw summing technique the environmental performance between firms or remains a common method among mass media over time (Horvath et al. 1995). Some weighting outlets, including the Associated Press, the Wall techniques go further by incorporating the me- Street Journal, and the San Francisco Chronicle dium of release (e.g., air, water), modeling chem- (Associated Press Newswires 2002; Kay 2002; ical transport and fate, and assessing exposures. Noah 1996; Shields 1999). This technique has The most sophisticated methods estimate poten- also been widely employed in the management tial population exposure based on the physical literature (e.g., Cohen et al. 1997; Dooley and and demographic characteristics proximate to Fryxell 1999; Eskeland and Harrison 1997; Feld- the release. No single method has been estab- man et al. 1997; Khanna and Anton 2002; Konar lished as the standard because different ap- and Cohen 2001) and even in leading scientific proaches trade off particular benefits and draw- journals (e.g., Rubin 1999). It continues to be backs. Often, implementation of more complex used by several prominent nonprofit organiza- weighting methods is difficult or infeasible due tions, including the Investor Responsibility Re- to data and time constraints. search Center, Bridges to Sustainability, and En- Management scholars often use environmen- vironmental Defence Canada (Environmental tal inventory data to develop firm- or facility- Defence Canada 2002; IRRC 2002), and in gov- specific measures of environmental performance. ernment publications, including the U.S. Census The decision of which weighting scheme to use Bureau’s Statistical Abstract of the United States, is important because the various schemes differ the California Environmental Protection in their objectives, comprehensiveness, and Agency’s Strategic Vision, and the North Ameri- weighting values. The choice of scheme can lead can Commission for Cooperation’s annual Taking to different conclusions regarding which sub- stances are most damaging to human and eco- Stock reports (Cal/EPA 2000; CECS 2002; U.S. system health (Pennington and Yue 2000). This Census Bureau 2001). As one example, Lucent choice can affect the results of environmental Technology’s EcoPro, which has been used to justice investigations that explore the correlation evaluate desktop computers’ environmental per- between socioeconomic status and environmen- formance (Caudill et al. 2000), is sufficiently so- tal health burden (Cutter et al. 2002). phisticated to combine environmental impact, Selecting an appropriate weighing scheme is resource productivity, and eco-efficiency into also important in other arenas, such as creating one metric to assess performance of products and and interpreting life-cycle assessments (LCAs) facilities; however, it uses an unweighted sum of and implementing design for environment (DfE) hazardous pollutants (Mosovsky et al. 2000).