EXPERT ASSESSMENT OF UNCERTAINTIES IN DETECTION AND ATTRIBUTION OF CLIMATE CHANGE BY JAMES S. RISBEY AND MILIND KANDLIKAR The problem of detection of climate change and attribution of causes of change has been formalized as a series of discrete probability judgements by experts he study of detection and attribution of climate and further on "attributing" any detected signal to change addresses the issue of whether, and to increases in greenhouse gases or other possible causes Twhat extent, human-induced increases in green- (Hasselmann 1998; Hegerl et al. 1997; Santer et al. house gases have caused climate changes. Communi- 1996a; Zwiers 1999). Summary assessments of the cation of the science and role of uncertainties on this detection and attribution issue have tended to take a issue has been hindered by the lack of explicit formal qualitative approach to characterizing uncertainties approaches for making overall conclusions on detec- (Santer et al. 1996a; Barnett et al. 1999; Mitchell et al. tion and attribution. We have developed a protocol 2001). This study outlines some of the major uncer- to quantify uncertainties in each component of the de- tainties in detection and attribution, uses expert tection and attribution process and to provide a struc- judgements to quantify them, and gives an overall tured way to make overall conclusions (Risbey et al. quantitative assessment of detection and attribution 2000). Here we describe results from use of the pro- of climate change. tocol with a set of climate experts. In making overall assessments on detection and Studies of detection and attribution of climate attribution of climate change, a variety of scientific change have focused first on "detecting" climate judgements are called for. These relate to the quality change against the background of natural variability, of the underlying data needed to monitor climate changes, to the quality of models used to assess pos- sible causes of those changes, to the ability to moni- AFFILIATIONS: RISBEY—School of Mathematical Sciences, Monash tor the forcing of the earth system, and to model fu- University, Melbourne, Australia; KANDLIKAR—Faculty of Graduate ture consequences. The probability-based protocol Studies, University of British Columbia, Vancouver, Canada employed here makes many of these underlying CORRESPONDING AUTHOR: Dr. James S. Risbey, School of judgements explicit. The protocol breaks the detec- Mathematical Sciences, P.O. Box 28M, Monash University, Clayton, Vic 3800, Australia tion and attribution problem up into its component E-mail: [email protected] steps and allows for uncertainties on each of these judgements to be represented quantitatively via prob- In final form 31 May 2002 ©2002 American Meteorological Society ability density functions (pdfs). The protocol was completed by a set of 19 experts working in the field AMERICAN METEOROLOGICAL SOCIETY SEPTEMBER 2002 BAHfr | 1317 Unauthenticated | Downloaded 10/05/21 07:05 PM UTC tainty deriving from incomplete and TABLE 1. List of experts and their institutional affiliations. biased datasets. This is done by eliciting the pdf, /(£.)> for the observed change Expert Affiliation over a specified period for each line of evidence from the expert. The detection Myles Allen Oxford University step entails discriminating forced cli- Robert Balling Arizona State University mate changes (Shine and Forster Curt Covey Lawrence Livermore National Laboratory 1999) from internal natural variability Chris Folland Hadley Centre (Pelletier 1997). This requires an elici- Klaus Hasselmann Deutsches Klimarechenzentrum tation of the pdf,/(AT), for natural vari- ability in each line of evidence over the Gabi Hegerl Texas A&M University same time period. The distributions, Phil Jones University of East Anglia /(£.) and /(N.) can then be compared David Karoly Monash University using statistical tests to see how similar Haroon Kheshgi Exxon Corporate Research they are. Michael MacCracken U.S. Global Change Research Program Following (Bell 1986) and (Zwiers 1999), the detector, D., is defined as Nathan Mantua University of Washington D. = N{0, cfe) when S = 0 (null hypoth- Patrick Michaels University of Virginia esis); D. = N(S, eg) when S* 0 (alterna- John Mitchell Hadley Centre tive hypothesis), where D ~ &) in- Neville Nicholls Australian Bureau of Meteorology dicates that D is normally distributed Stephen Schneider Stanford University with the mean, ju, and variance, The null hypothesis of no climate change is John Wallace University of Washington rejected at the p x 100% significance Warren Washington National Center for Atmospheric Research level using a one-sided z test if the value Tom Wigley National Center for Atmospheric Research of the detector exceeds a critical value Francis Zwiers Canadian Climate Center D = Z^cfe, where Zlp is the (1 - p) quantile of the standard normal distri- bution. The signal is "detected" when of detection and attribution studies (see Table 1). Note the null hypothesis is rejected. The power of the test that the order of experts has been scrambled in pre- or probability of detecting the signal when it is present senting results to prevent identification with particu- is given by lar experts. Twenty-three experts were approached to complete the protocol and four declined to do so. The 1 foo - —(D-S)2 / (J2; experts were selected to provide depth and diversity P(D > D )= _ exp 2 dD. (1) c in a range of different areas. For example, some were V2/ro"si data specialists, some modelers, some focused on methods in detection and attribution, some were from The conventional form of detection described above the Intergovernmental Panel on Climate Change uses only information on the critical percentile of (IPCC) detection and attribution groups (Santer et al. /(N.), and may be sensitive to the choice of critical 1996b; Mitchell et al. 2001), and some were not in percentile. For that reason we perform detection tests those groups. Further, they entertained a range of for a range of values of the critical percentile (1%, 5%, views about the likely magnitude of greenhouse cli- 10%, 20%). Detection implies rejection of the null mate change. This sample of 19 experts is considered hypothesis at the critical percentile. This does not to be indicative, but not exhaustive. identify the cause of any climate change or the con- The first step in the detection and attribution pro- tribution of natural variability to observed changes. cess is the determination of a set of lines of evidence, It merely identifies the likelihood that the change ex- E. i =1,1, which serve as signifiers of climate change. y ceeds the critical threshold. For each expert the protocol elicits lines of evidence that are relatively independent in order to minimize 1 redundant information. The expert must then evalu- 1 For example, changes in global mean sea level are not consid- ate the magnitude and uncertainty in each line of evi- ered because they are a fairly direct consequence of changes in dence, since each will be associated with some uncer- global mean temperature, which is a chosen line of evidence. 1318 | BAI1S- SEPTEMBER 2002 Unauthenticated | Downloaded 10/05/21 07:05 PM UTC For the attribution step, responsibility in explain- many realizations of climate change over the recent ing observed trends (E) is partitioned out among period, the expected contribution from natural vari- competing causes. These causes can include natural ability to the observed change would be zero. But since variability as well as a range of various forcing pro- we have only one real observed period to draw from, cesses. The attribution step requires the experts to it is possible that there were long-term excursions of identify this set of forcings, and to characterize their natural variability during this particular period that patterns and magnitude. The set of forcings consid- contributed to the observed change. As for the forc- ered include changes in greenhouse gas forcing, aero- ing, natural variability can only contribute when it has sol forcing, solar forcing, volcanic forcing, and ozone the same sign as the expected change. The fractional forcing. The uncertainty in each of the j forcings rel- contribution of natural variability is calculated from evant to a line of evidence, i, can also be represented those cases where natural variability makes a contri- by a pdf,/(F ). Once the forcings have been charac- bution to the expected change and is given by the ra- terized, the next step is to elicit pdfs attributing frac- tio N./E.. Sampling from /(£.) and/(IV.) in this way tional responsibilities from 0 (no responsibility) to 1 leads to an expected contribution from natural vari- (complete responsibility) for each forcing in explain- ability, A. nv, which can range from 0 to 1. Note that ing the detected signal. Note that any given forcing when/(£.) and/(N.) are identical distributions, A.nv could explain a vanishingly small fraction of the de- = 1, and natural variability is responsible for all of the tected signal with high probability or almost all of the observed change. signal with vanishingly low probability. Calculation of The results presented here for each step in the the attribution to each forcing requires a convolu- above process were obtained in a set of detailed per- tion of the forcing pdf and attribution pdf. This step sonal interviews (lasting between 4 and 8 h each) can be simplified for the expert elicitation process by where the experts offered probabilistic judgements as discretizing the range of values that each forcing well as the underlying rationale for their judgements. can take, and by eliciting just the expected fraction, Though results included attributions to all potentially Q.., of the detected signal for line of evidence, i, ac- significant forcings, we isolate only outcomes for counted for by each forcing, j. These fractions pro- greenhouse forcing here. Greenhouse forcing is po- vide a breakdown of the contributions of only those tentially most problematic among the forcings be- forcings that could have caused a change in E.