Electronic Supplemental Material

Electronic Supplementary Material

Negative learning in context

Here we provide an example to illustrate the usefulness of the concept of functional truth as opposed to absolute truth in science, followed by a description of literature from other fields which explores expert learning and sometimes involves concepts similar to negative learning.

With respect to the existence of true belief in science, Newtonian physics is functionally true over a certain range of application; tests of the equivalence principle notwithstanding, there are very few attempts to explore its validity over that range any longer. However, Newtonian physics is certainly incomplete and therefore not absolutely true (Christie, 2001; Kitcher, 1993).

Learning from technical information about markets (Jones and Ostroy, 1984; Bernanke,

1983), foreign affairs and intelligence (Tetlock, 1991; Tetlock, 1999), technology and technological services (Arthur, 1989), and environmental amenities (Arrow and Fisher,

1974; Kolstad, 1996) has been analyzed or modeled in the context of decision making by private (March and Simon, 1958; March, 2006) and governmental organizations (Tetlock,

1999; Vaughan, 1996; Eden, 2004). Negative (Kuhn, 1962) and progressive (Lakatos, 1999) characteristics of learning have been explored in the literature on the philosophy and history of science particularly in the wake of Kuhn’s (Kuhn, 1962) categorization of “normal” and “revolutionary” science.

Pre-Copernican attempts to rectify the Ptolemeic system of planetary motion provide a classic example (Kuhn, 1959) of attempting to fix an errant model, relevant to our discussion of negative learning about climate sensitivity. Studies of decision-making by firms (March, 2006) and governments (Tetlock, 1999), particularly their treatment of accidents or catastrophic risk (Vaughan, 1996; Eden, 2004), draw on similar concepts.

Stratospheric ozone case history

Here we provide additional details pertinent the case of stratospheric ozone depletion.

Histories of the ozone depletion issue have emphasized diplomatic, technological, scientific, political, historical and philosophical frames (Christie, 2001; Molina, 1995;

Rowland, 1995; Crutzen, 1995; Dotto and Schiff, 1978; Benedick, 1998; Parson, 2003;

Anderson and Sarma, 2002; Grundmann, 2001; Nolin, 1995; Roan, 1989; Litfin, 1994;

Hammitt and Thompson, 1997). Several factors, including the variations in projected depletion shown in Figure 1 of the main text, contributed to the reluctance of governments to impose comprehensive regulation in the face of uncertainty about the cost and availability of substitute chemicals and processes (Benedick, 1998; Parson, 2003;

Hammitt and Thompson, 1997; Levy et al., 2001). Already by 1976, production of the chemicals had decreased sharply following consumer boycotts of aerosol sprays and pre- regulation substitution by some firms (Roan, 1989). The combination of lower emissions and decreased sensitivity of stratospheric ozone in models (Figure 1, main text) led to projection of total ozone depletion at only 2-4% at steady state for 1980 emissions, while it was also understood that resumption of emissions growth would lead to higher predictions of depletion (NRC, 1984). Although there were indications in observations that ozone above 35km altitude had decreased (NRC, 1984), depletion of the total ozone column had not been observed, and it was unclear how far in the future such a trend might be observable. These developments provided support for the wait-and-learn strategy adopted by many governments. Important political factors included industry opposition to regulation, easing of public concern and attention following the limited regulatory actions, and the anti-regulatory stance of several governments, particularly the

In conjunction with the resumption of emissions growth, projection of larger depletion based on such growth, and consolidation of scientific consensus (WMO, 1986; Parson,

2003), the discovery and explanation of Antarctic depletion accelerated implementation of policies to eliminate chlorofluorocarbon emissions. Some studies see the ozone hole as a key factor in developing the Montreal Protocol (Christie, 2001; Litfin, 1994;

Grundmann, 2001) while others do not (Benedick, 1998; Parson, 2003). But it certainly had a strong effect on subsequent agreements, as did the demonstration of mid-latitude and Arctic ozone depletion (WMO, 1988, 1991). In addition to the magnitude of mid- latitude depletion being larger than predicted by models, the vertical and latitudinal distributions and seasonal characteristics of depletion differed substantially from predictions, and heterogeneous reactions were implicated in this behavior as well. West Antarctic ice sheet case history

Here we provide further details pertinent to the case of the West Antarctic ice sheet. The prominence given the threat of sea level rise due to polar ice sheet melting in early assessments and policy discussions (Conservation Foundation, 1963; PSAC, 1965; NRC,

1979; Revelle, 1983; Flohn, 1982) indicate that it was an important factor leading to initial concerns about the consequences of warming. Conversely, diminished scientific estimates of the vulnerability of WAIS (Meier, 1990; Warrick and Oerlemanns, 1990;

Warrick et al., 1996; Bentley, 1997) may have contributed to a reduced sense of urgency among some policy makers as the 1990s progressed (Jaeger, 2001) although this assertion has not yet been explored in detail. Recent observations, beginning with successive, rapid collapse of sections of the Larson ice shelf, appear to be spurring another revision of opinion (Church and Gregory, 2001; Alley et al.,2005; Vaughan, in press) and are once again drawing the interest of policy makers (Schellnhuber et al., 2006; Tol, 2006) to

Population and energy projections case history

In this section we provide supporting information on how population and energy policies have been affected by projections, the types of factors that have caused such projections to err, and how negative learning in physical and socio-economic systems compare. We also provide further discussion and references on past population and energy projections, including those made following the OPEC crisis and those made of the nuclear power industry, and further details on Figure 3 in the main text illustrating errors in past population projections.

Population-related policies such as support for family planning programs (Seltzer, 2002), pro-natalist incentives (Caldwell et al., 2002; Teitelbaum, 1985), social security system design (SSA, 2005; OECD, 1998), and migration policy (NRC, 1997) have been influenced by projections of population growth, decline, and changes in age structure.

Energy projections, and the studies informing them, have had substantial consequences for energy investments (Syme, 1987) and have influenced policy by framing debates, clarifying the bases of disagreement, and in some cases influencing the formulation of energy legislation (Greenberger, 1983). Population and energy projections also play indirect roles in scientific processes informing environmental policy (Nakicenovic et al.,

2000; Carpenter et al., 2005).

Past projections characterized by progressive learning include reasonably accurate projections of global population size in 2000 made since the late 1950s (Bongaarts and

Bulatao, 2000) and decadal-scale projections of energy consumption in periods of gradual change (O'Neill and Desai, 2005; Linderoth, 2002). The list of contributing causes to projections that have gone awry (Bongaarts and Bulatao, 2000; Keilman, 1999; Smil,

2003; Koomey, 2002; Land and Schneider, 1987; Koomey, et al., 2003; Craig et al.,

2002; Draper, 1995; Bezdek and Wendling, 2003) includes incorrect base year data; inaccurate exogenous assumptions in models; assumed continuation of current structures such as market conditions; the tendency to be slow to incorporate new information into projections; over-optimistic expectations about the influence of new technologies; unforeseen events; unanticipated structural change; and unanticipated policy changes, sometimes driven partly by earlier projections themselves.

Negative learning about socio-economic and physical systems may seem qualitatively different. The former involves systems that change and models that are slow to respond to new information, while the latter involves models that change (incorrectly) while trying to simulate the unchanging laws of a physical system. However this dichotomy is misleading. Negative learning has occurred in socio-economic systems when models changed incorrectly in response to new information, in a way very similar to negative learning in physical systems. Population projections following the start of the baby boom in the U.S. are a case in point: A temporary increase in fertility led to new, incorrect beliefs about long-term trends. Likewise, negative learning in physical systems can in some cases be interpreted as a slow response of models to transitions, in a way similar to negative learning in socio-economic systems. For instance, ice-sheet models have changed slowly in response to observations of the alteration of the cryosphere caused by emissions of greenhouse gases.

Furthermore, the causes of negative learning in socio-economic and physical systems share many characteristics (see supplementary material associated with Conclusions below). Both are characterized by an imbalance between model simulations and observations, in the sense that additional observations revealed significant model errors. Many ozone depletion projections were made before much data had been gathered, particularly for polar regions and the lower stratosphere. Models used to project the future of ice sheets were developed with sparse observations and limited paleo-data available for validation. Energy and population projections were made during structural changes with which there was little historical experience. In addition, the resulting negative learning in all these cases can be understood as stemming from structural error in models, analogous to the assumed structural error in the climate sensitivity illustration.

In the case of ozone depletion, heterogeneous chemical reactions were generally absent from projection models prior to the discovery of the ozone hole; models of WAIS lack a representation of certain processes that apparently govern ice sheet dynamics; in population projections, some factors influencing fertility decline have been missing or incomplete in conceptual models of demographic transitions; in energy projections, processes that determined responses to external shocks were not included.

A number of studies have assessed factors contributing to the demographic transition

(Bongaarts and Watkins, 1996; Oppenheim Mason, 1997; Kirk, 1996; Hirschman, 1994).

Prior to its identification as a cause of inaccurate population projections, assumption drag was discussed in the context of a wider range of projections (Ascher, 1978).

The accuracy of energy projections following the OPEC crisis has been assessed by several authors (Smil, 2003; Koomey, 2002; Koomey et al., 2003; Bezdek and Wendling,

2003). Regarding inaccurate energy forecasts in the early 1980s, the original projection exercises were carried out by the Energy Modeling Forum (EMF, 1982), the International Energy Workshop (Manne, 1982), and the U.S. Energy Information Administration (U.S.

Dept. of Energy, 1982), and their performance was later assessed (Draper, 1995;

Schrattenholzer, 1999; Huntington, 1994).

Projections of nuclear power production provide a pertinent example of negative learning

(Smil, 2003). Early projections by the U.S. Atomic Energy Commission (AEC, 1974) under-predicted generating costs of future plants by a factor of five in real terms (Cohn,

1997). In the mid-1970s the AEC’s most conservative outlooks over-predicted installed nuclear generating capacity in 2000 by more than a factor of 8 for the U.S. and a factor of

4.5 for the world (AEC, 1974; IAEA, 2005). Even as late as the mid-1980s the lowest projection from the International Atomic Energy Agency over-predicted global capacity in 2000 by 40% (IAEA, 2005). This record of substantial and persistent over-optimism has been attributed to uncertainty in technological assessments that allowed the political and institutional context in which the projections were made to strongly influence their outcomes (Cohn, 1997).

Figure 3 in the main text, showing errors in projections of the total fertility rate over time, are based on UN country level projections for the world made in the years 1968, 1973,

1980, 1982, 1984, 1988, 1990, 1992, 1994, 1996, and 1998. Estimates of current and past fertility levels from the UN (U.N. Population Division, 2005) are assumed to indicate actual fertility. In order to control for variations in data quality across countries and over time, past projections are corrected for errors in estimates of fertility in the base year (i.e., the initial year of the projection) by adding a constant offset to the projection equal to the baseline error. The error in the projection is calculated as the difference between the baseline-corrected projection and the estimated actual fertility.

DICE Model Overview

We briefly overview the key elements and equations of the DICE-99 model. For details, see Nordhaus and Boyer (2000). DICE (the Dynamic Integrated Climate and Economy model), is a Ramsey optimal growth model, augmented by emissions from economic production, and a set of climate equations that relate temperature change to emissions.

Production, and therefore consumption, is reduced both by emissions abatement (t) and by damages, as a function of temperature change, from climate change. The model is solved as a nonlinear programming problem in 10-year periods from 1995 to 2385. The model runs for this paper were solved in the GAMS system (Brooke et al., 1988) using the PATH solver (Ferris and Munson, 1999).

The DICE model maximizes the discounted net present value utility function for the representative consumer, by choosing the optimal investment path and the optimal path of emissions reductions over time (eq. 1). The social rate of time preference (t) is assumed to decline over time (Weitzman, 2001), starting at 3% in 1995, and decreasing to 2.3% in

2100 and to 1.8% in 2200. The intertemporal elasticity of substitution is assumed to be unity.

max Uc(t), L(t)(1 (t))t (1) I (t),(t)  t The utility function assumed is the population times the logarithm of per capita consumption (eq. 2). The logarithm is the limit of a more general power function where the elasticity of marginal utility is one.

(2) Uc(t), L(t)  L(t)log[c(t)]

Production in the DICE model is a Cobb-Douglas function of capital K(t) and labor L(t), and a value share of capital equal to (eq. 3)  . Hicks-neutral productivity change is represented by the factor A(t). The losses to production due to abatement and climate damage are represented by the factor (t).

(3) Q(t)  (t)A(t)K(t) L(t)1

The production in each period is divided between total consumption C(t) and investment

I(t) (eq. 4).

(4) Q(t)  C(t)  I(t)

Per capita consumption c(t) is simply the total consumption divided by the population size L(t) (eq. 5).

(5) c(t)  C(t) / L(t) Investment I(t-1) augments the capital stock in the next period K(t) (eq. 6). The previous

period’s capital stock depreciates at a rate of K.

(6) K(t)  (1  K )K(t 1)  I(t 1)

Anthropogenic carbon emissions E(t) are related to production by the carbon intensity

(t), which declines over time at an exogenously determined rate (eq. 7). Abatement reduces emissions by the fraction (t).

(7) E(t)  [1 (t)] (t)Q(t)

DICE represents the carbon cycle with a three-box model, consisting of the atmosphere

MAT(t), the upper mixed layer of the ocean MUP(t), and the deep ocean MLO(t) (eqs. 8a-c).

Total emissions ET(t) is the sum of anthropogenic carbon emissions and land-use changes.

(8a) M AT (t)  10ET (t 1)  11M AT (t 1)  21M UP (t 1)

(8b) M UP (t)  22 M UP (t 1)  12 M AT (t 1)  32 M LO (t 1)

(8c) M LO (t)  23M UP (t 1)  33M LO (t 1)

Radiative forcing F(t) is given by eq. 9. Temperature change is calculated using a two- box approach, where T(t) is the temperature change of the atmosphere and mixed-layer ocean, and TLO(t) is the temperature change of the deep ocean (eqs. 10-12). (9) F(t)  4.1{log[M (t) / 590]/ log(2)} O(t)

(10) T(t)  T(t 1)  (1/ R1 ){F(t)  T(t 1)  (R2 / 12 )[T(t 1)  TLO (t 1)]}

(11) TLO (t)  TLO (t 1)  (1/ R2 ){(R2 / 12 )[T(t 1) T LO(t 1)]}

The first term in eq. 12 is the cost of emissions abatement. The second term is the damage cost from climate change.

b2 2 (12) (t)  (1 b1(t) /[11T(t)  2T(t) ]

Modification to DICE

To simulate negative learning about climate sensitivity, we assume that the perceived model is incorrect due to an additional feedback on radiative forcing that is unknown to the observer/decision-maker. We modify the original equation in DICE for radiative forcing F(t) (eq. 9 above) as follows:

(9a) If ΔT(t) > β, F *(t)  F(t)  F *(t 1) .

When temperature change exceeds some threshold β, the true radiative forcing F *(t) differs from the perceived forcing by some fraction of the previous period’s forcing. This model is meant to capture positive or negative feedback resulting from, for example, an unknown aerosol-cloud interaction (Kaufman and Koren, 2006). We explore the effects of an unknown positive feedback (α = 0.3) and also of an unknown negative feedback (α

= -0.5), which result in an equilibrium temperature change from CO2 doubling of +40% and -30%, respectively. We assume a threshold temperature of β = 1.0 degree above preindustrial, and phase in the feedback using a logistic form over 3 decades.

This is a “feedback” in the strict mathematical sense that radiative forcing affects some unknown process, which in turn affects radiative forcing further. It is distinct from climate sensitivity in that it is not a feedback directly on temperature change. One could also consider it as an additional forcing, but the critical aspect is that the functional form of the “true” model differs from the “false” model in a way that cannot be captured by linearly adjusting a parameter in the “false” model.

Bayesian Statistical Procedures

In this section, we give more details on the procedures used to calculate posterior distributions of climate sensitivity from simulated observations.

All simulated observations are of global mean surface temperature change as calculated in DICE. The observations are generated from the version with the additional unknown feedback as described in the main text. The observed temperature change over 2000- 2100 is shown in Figure 1 for no feedback, the positive feedback, and the negative feedback for all four values of climate sensitivity.

Posterior distributions for climate sensitivity are calculated using Bayes Theorem:

f x |  f ( ) (13) f  | x  f (x)

In this case, the unobservable parameter θ is the equilibrium climate sensitivity (Cubasch et al., 2001), as represented in DICE with the parameter , and the observable characteristic x is temperature change T(t). The term f(x | θ) is the likelihood function, f(θ) is the prior distribution, and f(θ | x) is the revised posterior distribution of climate sensitivity.

Calculations are performed by Monte Carlo integration on continuous probability density functions (PDF) for climate sensitivity, approximated with 100 discrete bins for each

PDF, and using a sample size of 5000. The updating calculation is performed for each decadal observation over 2000-2040, and the resulting posterior becomes the prior for the next decadal update. Temperature Change with No Additional Feedback

) 5 K (

e Climate Sensitivity = 1.5 g

n Climate Sensitivity = 3.0 a

h 4 Climate Sensitivity = 5.0 C

Climate Sensitivity = 8.0 e r u t a r

e 3 p m e T

c 2 a f r u S

l a b o l

G 0 2000 2020 2040 2060 2080 2100

Temperature Change with Temperature Change with Additional Positive Feedback ( = 0.3) Additional Negative Feedback ( = -0.5)

K 3.5 ) (

Climate Sensitivity = 1.5 g e Climate Sensitivity = 1.5

n 6 Climate Sensitivity = 3.0 g a

n 3.0 Climate Sensitivity = 3.0

h Climate Sensitivity = 5.0 a

h Climate Sensitivity = 5.0 C

Climate Sensitivity = 8.0 C e Climate Sensitivity = 8.0

u r 2.5 t u t a r a r e e p 4 p 2.0 m m e e T

3 e c 1.5 c a f a f r r u u S

2 S 1.0 n n a a e e M M

l 1 l 0.5 a a b b o o l l G 0 G 0.0 2000 2020 2040 2060 2080 2100 2000 2020 2040 2060 2080 2100

Figure 1: Global mean temperature change over 2000-2100 for climate sensitivities of

1.5, 3, 5, and 8K. a) Model with no additional feedback; b) Model with additional positive feedback; c) Model with additional negative feedback. Results of Bayesian Updating

Posteriors with an unknown positive feedback on radiative forcing

Figures 2 through 5 show the effect of updating on the median and 90% bounds of the posterior for climate sensitivity, where the true climate sensitivity generating the observations is 1.5, 3.0, 5.0, and 8.0K, respectively, and in the presence of an additional positive feedback. By performing Bayesian updating with an incorrect model (omitting the additional positive feedback that generates the observations), the “negative learning” model narrows in on a range for climate sensitivity that is higher than the true value, in all cases except where the true climate sensitivity is 8.0 degrees. For an extreme value with low probability in the prior, the anchoring effect of the prior is overcome more quickly with negative learning. If the updating were to continue, the incorrect model would likely eventually narrow on a range of sensitivity higher than 8.

If the omitted feedback had an identical functional form to the climate sensitivity, the overestimates of climate sensitivity would compensate for the omitted feedback, and temperature projections would still be unbiased. In this case, however, the functional form of the new feedback differs from sensitivity, and therefore new projections with the incorrect model, even with the overestimate of sensitivity, will underestimate future temperature change. Figure 6 shows an example in which new temperature projections are made in 2040 using the posterior PDFs of sensitivity available at that time, assuming a true climate sensitivity of 3K and a positive feedback. Note that the projected temperature from the incorrect model underestimates future temperature change, and drifts further from the correct projection as the time horizon increases.

To better explore the phenomenon of negative learning, we want to compare forecasts made with updated PDFs, like Figure 6, in different time periods in order to see whether forecasts become better or worse over time. One common measure of forecast accuracy is to compare projections of the same quantity from different times. For example, Figure

7 shows the errors in projecting the temperature change in 2150 based on three different projection exercises, one in 2000 using the priors, one in 2040 using posteriors updated on observed temperature change 2000-2040, and one in 2080 using posteriors updated on observed temperature change 2000-2080. By this measure, the trend is not negative learning, in the sense that the errors do decrease over time. However, the magnitude of the errors is large relative to the rate of improvement. This comparison may not be the most fair, since later projections are made over ever shorter time horizons, and thus can be expected to smaller errors.

An alternative, and perhaps more fair, comparison is of errors in projections over constant length time horizons. For example, Figure 8 shows the errors in projections 100 years ahead of the year of the exercise, again comparing projections made in 2000, 2040, and

2080 (projecting to 2100, 2140, and 2180, respectively) with the appropriate PDF for sensitivity at that time. In this case, the errors in the projection with the negative learning model actually grow with time, even with the additional decades of observation used for updating. Median & 90% Bounds True Climate Sensitivity = 1.5 6 Negative Learning Feedback Known 5 True Sensitivity ) K (

4 y t i v i t i s

n 3 e S

e t a m

i 2 l C

0 2000 2010 2020 2030 2040 2050

Figure 2: Evolution of perceived median and 90% bounds of climate sensitivity, where the true climate sensitivity is 1.5K and with additional positive feedback. Red dashed lines are result of updating with model without feedback, and blue dotted lines are result of updating with model with the additional feedback. Median & 90% Bounds True Climate Sensitivity = 3.0 6

5 ) K (

4 y t i v i t i s

n 3 e S

e t a m

i 2 l C Negative Learning 1 Feedback Known True Sensitivity

0 2000 2010 2020 2030 2040 2050

Figure 3: Evolution of perceived median and 90% bounds of climate sensitivity, where the true climate sensitivity is 3.0K and with additional positive feedback. Red dashed lines are result of updating with model without feedback, and blue dotted lines are result of updating with model with the additional feedback. Median & 90% Bounds True Climate Sensitivity = 5.0 10 Negative Learning Feedback Known 8 True Sensitivity ) K (

y t i v

i 6 t i s n e S

e t 4 a m i l C 2

0 2000 2010 2020 2030 2040 2050

Figure 4: Evolution of perceived median and 90% bounds of climate sensitivity, where the true climate sensitivity is 5.0K and with additional positive feedback. Red dashed lines are result of updating with model without feedback, and blue dotted lines are result of updating with model with the additional feedback. Median & 90% Bounds True Climate Sensitivity = 8.0 10 Negative Learning Feedback Known True Sensitivity 8 ) K (

v 6 i t i s n e S

t 4 a m i l C

0 2000 2010 2020 2030 2040 2050

Figure 5: Evolution of perceived median and 90% bounds of climate sensitivity, where the true climate sensitivity is 8.0K and with additional positive feedback. Red dashed lines are result of updating with model without feedback, and blue dotted lines are result of updating with model with the additional feedback. 5

) Negative Learning K

( Feedback Known

0 True Sensitivity 1 2 - 0 4 0

e g n a h C

2 e r u t a r e p 1 m e T

0 2040 2050 2060 2070 2080 2090 2100

Figure 6: Temperature Projection from 2040 to 2100 using posterior from observations where true climate sensitivity is 3.0K and with positive feedback. Errors in Forecast of Temperature in 2150

) Forecast Median and 90% Bound K (

e 4 r u t a r e

p 2 m e T

z 0 i l a e R

e v i t a l

e -4 R

a Negative Learning c

e Correct Model r -6 o No Error F

s r -8 o r r 2000 2040 2080 E

Figure 7: Comparison of errors in projected temperature change by 2150. X-axis indicates the year of the projection. Each projection uses latest posterior PDF for sensitivity, using all decades up to that year to update. Y-axis indicates the errors in the median and 90% of the projection, relative to the actual realized temperature change in

2150. Case shown has true climate sensitivity of 3.0K and a positive feedback. Errors in Forecast Temperature 100 Years Ahead Forecast Median and 90% Bound

% 0.2 (

e r u t a

r 0.0 t e s p a c m e e

r -0.2 T o

e n z i i

s -0.4 a r e o r R r

E o t -0.6 e v i t a l Negative Learning e -0.8

R Correct Model

No Error -1.0 2000 2040 2080 Starting Year for Projection

2100 2140 2180 End Year of Projection

Figure 8: Comparison of errors in projected temperature change 100 years from the projection start date. X-axis indicates the beginning and ending years of each projection.

Each projection uses latest posterior PDF for sensitivity, using all decades up to that year to update. Y-axis indicates the errors in the median and 90% of the projection, relative to the actual realized temperature change in projected year. Case shown has true climate sensitivity of 3.0K and a positive feedback.

Posteriors with an unknown negative feedback on radiative forcing

An omission or misspecification in models could result in bias in either a positive or a negative direction. For symmetry, we also present the impacts of negative learning due to an unknown negative feedback by temperature on radiative forcing. In this case, negative learning would result in an underestimate of climate sensitivity (Figure 9).

Forward projections using the mis-specified model and the underestimated climate sensitivity would cause a corresponding overestimate in future projections of temperature change (Figure 10). This in turn will result in more abatement than is optimal, as will be shown in the next section. a) True Climate Sensitivity = 1.5 b) True Climate Sensitivity = 3.0 6 6

5 5 ) ) K K ( (

t t 4 i i v v i i t t i 3 i s s

n n 3 e e S S 2 e e t t a a 2 m m i i l 1 l C C 1 0

0 2000 2010 2020 2030 2040 2050 2000 2010 2020 2030 2040 2050

c) True Climate Sensitivity = 5.0 d) True Climate Sensitivity = 8.0 7 10

6 8 ) K (

y t i v

t 6 i ) s K n ( e y S t

i 4 e v i

t 4 t i a s m n i l e

e 2 t a m i

l 2 C 0 1 2000 2010 2020 2030 2040 2050

0 Negative Learning Feedback Known 2000 2010 2020 2030 2040 2050 True Sensitivity

Figure 9: Median and 90% Bounds on climate sensitivity after each decade of observations in the presence of a negative feedback on radiative forcing ( = -0.5). Red dashed lines are result of updating with model without feedback, and blue dotted lines are result of updating with model with the additional feedback. Actual climate sensitivity that generates the observations in each case is: a) 1.5, b) 3.0, c) 5.0, and d) 8.0. 4 Negative Learning )

K Feedback Known (

0 True Sensitivity 0

2 3 2 - 0 0 0 2

n 2 a h C

e r u t a r

e 1 p m e T

0 2040 2060 2080 2100 2120 2140 2160 2180 2200

Figure 10: Temperature Projection from 2040 to 2100 using posterior from observations where true climate sensitivity is 3.0K and with negative feedback.

Stochastic Programming Experimental Design

Once the posterior distributions are calculated as described above, they are approximated with four-valued discrete probability distributions, to enable stochastic simulation in

DICE. We develop a version of DICE with 16 states of the world (SOW). Each SOW has a distinct value of climate sensitivity: 1.5, 3, 5, or 8K. In addition, each set of 4

SOWs share one of the four possible posterior distributions for sensitivity, and are constrained to have the same level of abatement during the period 2040-2100. All 16

SOWs must choose the same level of abatement during the period 2000-2040. Thus, although DICE is solved for 10-year periods, the stochastic model has essentially three periods:

1) 1995-2045: abatement decisions made under prior

2) 2055-2105: abatement decisions made under one of the four posteriors

3) 2115-2285: abatement decisions made for each SOW independently.

These timeframes are not implausibly long, both because the physical science learning may require decades of additional climate observations and controlled experiments, and because major shifts in policy regimes may only occur periodically, resulting in lock-in of capital stock, especially in the energy sector, that has a lifetime of decades.

Each SOW has a probability associated with it, calculated from the prior and the posterior distributions. The model maximizes expectation with respect to this probability distribution of net present welfare, as in equation 1, across the SOWs:

  (14) max Uc(t, s), L(t)(1 (t))t  (s) I (t,s),(t,s)    s  t  where π(s) is the probability of each state of the world s.

One additional constraint on the solution is that the change in carbon intensity from one decade to another cannot change by more or less than a rate of 2% per year (Yohe et al., 2004). This is to prevent unrealistically large shifts in emissions, implying large disruptions in the capital stock.

We do not vary the parameter R1 for different values of climate sensitivity as in Yohe et al (2004). While this additional procedure would make the joint parameters more consistent with the observed 20th century warming, it would not impact our qualitative results regarding negative learning in the presence of an unknown forcing feedback.

Results for Unknown Positive Feedback

The optimal abatement chosen under different SOWs with a positive feedback is shown in Figure 11. Consider only the solid lines. Up to 2040, one abatement level is chosen as a hedge against the uncertainty. From 2040 to 2100, there are four different abatement paths, each optimal for one of the four sets of observations from 2000-2040 and a corresponding posterior distribution. After 2100, the true climate sensitivity is revealed with certainty. ) s

n 1.0 o i s

s No Positive Feedback on RF i

m & Negative Learning E

y 0.8 Positive Feedback on RF c i l o P

f 0.6 o

n o i t c a r

F 0.4 (

t n e m e t

a 0.2 b A

l a m i t

p 0.0 O 2000 2020 2040 2060 2080 2100 2120 2140

Figure 11: Optimal abatement with reduction of uncertainty in 2040 and true state of the world revealed in 2100. Solid black lines indicate policies chosen if model omits feedback and uses posteriors updated on model with positive feedback. Red dashed lines indicate the abatement that would be undertaken if the additional feedback was known and included in model all along.

In Figure 11, the solid lines show the optimal abatement paths for the negative learning case, the world with no additional feedback. The dashed lines show the optimal abatement if the feedback is included in the model for both projections and optimization.

Because the negative learning model under-predicts temperature change, it also leads to too little abatement. In 2100, we assume that the truth about the feedback is also revealed, and so abatement thereafter adjusts to account for the higher projected temperature change. We show the corresponding impacts on optimal emissions, carbon taxes, and temperature change in figures 12-14.

Figure 15 shows the magnitude of the losses, in terms of present value of consumption.

Note that we calculate the present value of consumption using the PVCex calculation as in

Füssel (2007):

T C PVC (C ,  ,, g )  t ex 0...T 1...T 1...T  t t0 (1    g ) t'1 t' t' where the intertemporal elasticity of substitution  is unity, and the growth rate g is taken from the reference no control solution of DICE-99. The present value of lost consumption is given in order to illustrate the magnitude of the effect of negative learning; the optimal solutions for each SOW are based on maximizing discounted utility as described above.

20 No Positive Feedback on RF & Negative Learning Positive Feedback on RF )

n 15 o b

r s a n C o i

f s o s

i s m n 10 E o

m i s t n p o i O l l l i

0 2000 2020 2040 2060 2080 2100 2120 2140 Figure 12: Optimal carbon emissions with reduction of uncertainty in 2040 and true state of the world revealed in 2100. Solid black lines indicate policies chosen if model omits feedback and uses posteriors updated on model with positive feedback. Red dashed lines indicate abatement if the additional feedback was known and included in model all along.

250 No Positive Feedback on RF & Negative Learning ) Positive Feedback on RF C 200

n o t / $ (

e 150 x a T

n o b r

a 100 C

l a m i t p 50 O

0 2000 2020 2040 2060 2080 2100 2120 2140

Figure 13: Optimal carbon taxes with reduction of uncertainty in 2040 and true state of the world revealed in 2100. Solid black lines indicate policies chosen if model omits feedback and uses posteriors updated on model with positive feedback. Red dashed lines indicate abatement if the additional feedback was known and included in model all along. 10 No Positive Feedback on RF & Negative Learning 8 Positive Feedback on RF ) K (

a 6 h C

e r u t a

r 4 e p m e T 2

0 2000 2020 2040 2060 2080 2100 2120 2140

Figure 14: Resulting global mean temperature change under four different states of the world and with reduction of uncertainty in 2040 and true state of the world revealed in

2100. Solid black lines indicate policies chosen if model omits feedback and uses posteriors updated on model with positive feedback. Red dashed lines indicate abatement if the additional feedback was known and included in model all along. 3.0

s Expected Value = $1.1 Trillion e

s 2.5 s o L

n o i t

) 2.0 p n o m i l u l i s r n T

o 1.5 $

5 f 9 o

9 e 1 u ( l 1.0 a V

s 0.5 e r P

0.0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 EV State of the World

Figure 15: Net present value of consumption losses from negative learning.

Results for Unknown Negative Feedback

We perform simulations as described above for the situation in which the unknown feedback on radiative forcing is negative (cooling). Using the posteriors as calculated by the Bayesian updating (Figure 9), we again solve DICE for two cases: the negative learning case in which the feedback is unknown until 2100, and the counterfactual case in which the feedback is always known correctly. Here the negative learning results in more abatement than would have been optimal had the correct model been known (Figure 16).

The welfare losses from negative learning depend on which state of the world obtains, with the largest losses from the over-control occurring when the true climate sensitivity is 1.5K and net gains occurring when the true climate sensitivity turns out to be 8.0K

(Figure 17).

a) b) ) s

n 1.0 o i

s No Negative Feedback on RF i

m & Negative Learning No Negative Feedback on RF E

y 0.8 Negative Feedback on RF & Negative Learning c i l Negative Feedback on RF o ) n P 15

o o b r s N

a n f 0.6 o C o i

f s n o s

i o i s t m n c

E 10 o

F 0.4 o (

i s t t n n p o e i O l l l m i e B t 5 (

a 0.2 b A

l a m i t

p 0.0 0 O 2000 2020 2040 2060 2080 2100 2120 2140 2000 2020 2040 2060 2080 2100 2120 2140

250 4 No Negative Feedback on RF No Negative Feedback on RF & Negative Learning & Negative Learning ) Negative Feedback on RF Negative Feedback on RF C 200

) n 3 o K t ( /

$ e ( g s n e 150 a x h a C

e 2 n r o u t b a r r

a 100 e C p

l m a e m i T

t 1 p 50 O

0 0 2000 2020 2040 2060 2080 2100 2120 2140 2000 2020 2040 2060 2080 2100 2120 2140

Figure 16: Solution to stochastic programming problem under negative learning in the presence of a negative feedback on radiative forcing that is unknown until 2100 (solid lines), compared with the case where the additional feedback is known at all times (dashed lines): a) optimal abatement (t), b) optimal emissions E(t), c) optimal carbon taxes, d) resulting temperature change T(t).

e Expected Value = $10 Billion

s 0.10 s o

L 0.08

t 0.06 ) p n o m i l

u l 0.04 i s r n T

o 0.02 $

5 f 9 o 0.00

9 e 1 ( u l -0.02 a V

t -0.04 n e s -0.06 e r P -0.08 1 2 3 4 5 6 7 8 9 10111213141516EV State of the World

Figure 17: Net present value of consumption losses over time of negative learning in the presence of an unknown negative feedback of temperature on radiative forcing.

Computed as the difference between NPV consumption when the feedback becomes known in 2100 and the NPV consumption when the feedback is always known. Net present value is calculated using the declining discount rate as described above in section

1. Conclusions

Here we provide further details on issues raised in Section 5. With regard to the hazy line between parameter and structural uncertainty: in the ozone case, uncertainty in gas phase reaction rates was reduced while heterogeneous reactions were assumed to be so slow as to be likely irrelevant to projecting depletion. Progressive learning about parameters governing the latter processes did not occur until observations of the ozone hole forced reconsideration, and their eventual inclusion in models. Persistent errors in population and energy projections could be understood as a mismatch in the rate of learning about parameters and the rate of structural change in social and economic systems. The demographic transition was anticipated in principal, but how quickly it might be driven by other changes in societies was misjudged, and revised only slowly as the transition occurred. Similarly, energy models contained the possibility of responses of consumption and efficiency to price changes, but the magnitude of the effect was underestimated and learning about these parameters occurred slowly relative to actual changes in energy systems.

With regard to remedies for negative learning, Shlyakhter (1994) has explored the feasibility of importing experience from histories of error in the determination of parameters in physics as a means to compensate for overconfidence in the determination of probability distributions for environmental risks. Experience from outside the global change arena with accidents (Perrow, 1994), such as studies of organizational issues that inhibited learning at NASA before and after the Challenger disaster (Vaughan, 1996), is relevant in considering the difficulty of remedies. References:

Alley RB, Clark PU, Huybrechts P, Joughin I (2005) Ice-sheet and sea-level changes.

Science 310: 456-460. DOI: 10.1126/science.1114613

Anderson SO, Sarma KM (2002) Protecting the ozone layer. Earthscan Pub Ltd, London

Arrow KJ, Fisher AC (1974) Environmental protection, uncertainty, and irreversibility.

Quarterly J Econ 88: 312-319

Arthur BW (1989) Competing technologies, increasing returns and lock-in by historical

events. Economic J 99: 106-131

Ascher W (1978) Forecasting: an appraisal for policy makers and planners. Johns

Hopkins Univ Press, Baltimore, MD

Benedick R (1998) Ozone diplomacy: New directions in safeguarding the planet. Harvard

Univ Press, Cambridge, MA

Bentley CR (1997) Rapid Sea-Level Rise Soon from West Antarctic Ice Sheet Collapse?

Science 275: 1077-1078. DOI: 10.1126/science.275.5303.1077

Bernanke BS (1983) Irreversibility, uncertainty, and cyclical investment. Quart J Econ

98: 85-106

Bezdek RH, Wendling RM (2003) A half century of long-range energy forecasts: Errors

made, lessons learned, and implications for forecasting. J Fusion Energy 21: 155-

Bongaarts J, Bulatao RA (eds) (2000) Beyond six billion: Forecasting the world’s

population. National Academy Press, Washington, DC.

Bongaarts J, Watkins SC (1996) Social interactions and contemporary fertility transitions.

Pop Dev Rev 22: 639-682 Brooke A, Kendrick D, Meeraus A (1988) GAMS: a user’s guide. Scientific Press, So

San Francisco, CA

Caldwell JC, Caldwell P, MacDonald P (2002) Policy responses to low fertility and its

consequences. J Population Res 19(1): 1-24

Carpenter SR et al (ed) (2005) Ecosystems and human well being, vol 2: Scenarios,

millennium ecosystem assessment. Island Press, Washington, DC

Christie M (2001) Ozone layer: a philosophy of science perspective. Cambridge Univ

Press, Cambridge, UK, New York

Church JA, Gregory JM (2001) Changes in sea level. In: Houghton JT et al (eds) Climate

change 2001. The scientific basis, contribution of working group I to the Third

Assessment Report of the Intergovernmental Panel on Climate Change.

Cambridge Univ Press, Cambridge, UK and New York, pp 640-694

Cohn SM (1997) Too cheap to meter: an economic and philosophical analysis of the

nuclear dream. State Univ NY Press, Albany, NY

Conservation Foundation (1963) Implications of rising carbon dioxide content of the

atmosphere. Report of a conference held 12 March, New York

Craig PP, Gadgil A, Koomey JG (2002) What can history teach us? A retrospective

analysis of long-term energy forecasts for the U.S. Ann Rev Energy Environ 27:

83-118

Crutzen P (1995) My life with Oy, NOx and other YZOxs. Nobel Lecture, Dec 8

Cubasch U et al (2001) Projections of future climate change. In: Houghton JT et al (eds)

Climate change 2001: the scientific basis. Contribution of working group I to the Third Assessment Report of the Intergovernmental Panel on Climate Change.

Cambridge Univ Press, Cambridge, UK and New York

Dotto L, Schiff H (1978) The Ozone War. Doubleday, Garden City, NY

Draper D (1995) Assessment and propagation of model uncertainty. J Royal Stat Soc

Series B (Metho.) 57(1): 45-972.

Kitcher P (1993) The advancement of science: Science without legends, objectivity

without illusions. Oxford Univ Press, New York

Eden L (2004) Whole world on fire: Organizations, knowledge, and nuclear weapons

devastation. Cornell Univ Press, Ithaca, NY, London

EMF (1982) World oil: Summary report. EMF Report 6, vol 1, Energy Modeling Forum,

Stanford, CA

Ferris MC, Munson TS (1999) Interfaces to PATH 3.0: Design, implementation and

usage. Comput Opt Applications 12: 207-227

Flohn H (1982). In: Clark WC (ed) Carbon dioxide review 1982.Clarendon Press, Oxford

Univ Press, New York

Greenberger M (1983) Caught unawares: the energy decade in retrospect. Ballinger Pub

Co, Cambridge, MA

Grundmann R (2001) Transnational environmental policy: Reconstructing ozone.

Routledge, London, New York

Hammitt JK, Thompson JM (1997) Protecting the Ozone Layer. In: Graham JD, Hartwell

JK (eds) The greening of industry: a risk management approach. Harvard Univ

Press, Cambridge, MA, pp 43-92

Hirschman C (1994) Why fertility changes. Ann Rev Sociol 20: 203-233 Huntington HG (1994) Oil price forecasting in the 1980s: What went wrong? Energy J

15: 1-22

International Atomic Energy Agency (IAEA) 2005 Energy, electricity and nuclear power

estimates for the period up to 2030. Reference data series No1 (RDS-1/25)

Jaeger J (2001) In: Learning to manage global environmental risks, vol 2. The Social

Learning Group, MIT Press, Cambridge, MA, pp 9-48

Jones RA, Ostroy JM (1984) Flexibility and uncertainty. Revs Econ Studies L1: 13-32

Kaufman YJ, Koren I (2006) Smoke and pollution aerosol effect on cloud cover. Science

313: 655-658

Keilman N (1999) How accurate are United Nations population forecasts? In: Lutz W,

Vaupel JW, Ahlburg DA (eds) Issue supplement: Frontiers of population

forecasting. Pop Develop Rev 24: 15–41

Kirk D (1996) Demographic transition theory. Pop Studies 50: 361-387

Kitcher P (1993) The advancement of science: Science without legends, objectivity

without illusions. Oxford Univ Press, New York

Kolstad CD (1996). Learning and stock effects in environmental regulation: the case of

greenhouse gas emissions. Environ Econ Mgt 31: 1-18

Koomey JG (2002) Avoiding the ‘big mistake’ in forecasting technology adoption. Tech

Forecasting Soc Change 69: 511-518

Koomey JG, Craig P, Gadgil A, Lorenzetti D (2003) Improving long-range energy

modeling: a plea for historical retrospectives. Energy J 24: 75-92

Kuhn TS (1959) The Copernican revolution: Planetary astronomy in the development of

Western thought. Vintage, New York Kuhn TS (1962) The structure of scientific revolutions. Univ of Chicago Press, Chicago

Lakatos I (1999) Lectures on scientific method. In: Lakatos I, Feyerabend P, Motterlini

M (eds) For and against method. Univ of Calif, Berkeley, CA, pp 19-109

Land KC, Schneider SH (1987) Forecasting in the social and natural sciences: an

overview and analysis of isomorphisms. Clim Change 11: 7-31

Levy MA, Cavender-Bares J, Clark WC (2001). In: Learning to manage global

environmental risks, vol 2. The Social Learning Group, MIT Press, Cambridge,

MA, p 95

Litfin K (1994) Ozone discourses: Science and politics in global environmental

cooperation. Columbia Univ Press, New York

Linderoth H (2002) Forecast errors in IEA-countries’ energy consumption. Energy Policy

30: 53-61

Manne AS (1982) International Energy Workshop 1981, summary report, Stanford Univ

Inst Energy Studies, Stanford, CA

March JG, Simon HA (1958) Organizations. Wiley, New York

March JG (2006) Rationality, foolishness, and adaptive intelligence. Strat Mgmt J 27:

201-214. DOI: 10.1002/smj.515 (2006)

Meier MF (1990) The role of land ice in present and future sea-level change. In: Sea-

Level Change, National Academy Press, Washington, DC

Molina MJ (1995) Polar ozone depletion. Nobel Lecture, Dec 8

Nakicenovic N et al (2000) IPCC special report on emissions scenarios (SRES).

Cambridge Univ Press, Cambridge, UK Nolin J (1995) Ozonskiktet ich vetenskapen. Almqvist and Wiksell International,

Stockholm, pp 242-249

Nordhaus WD, Boyer J (2000) Warming the world: Economic models of global warming.

MIT Press, Cambridge, MA

NRC (1979) Carbon dioxide and cimate: a scientific assessment. National Academy

Press, Washington, DC

NRC (1984) Causes and effects of changes in stratospheric ozone: Update 1983. National

Academy Press, Washington, DC

NRC (1997) The new Americans: Economic, demographic, and fiscal effects of

immigration. National Academy Press, Washington, DC

O'Neill BC, Desai M (2005) The historical accuracy of projections of US energy

consumption. Energy Policy 33: 979-993

Oppenheim Mason K (1997) Explaining fertility transitions. Demography 34(4): 443-454

Organization for Economic Cooperation and Development (OECD) (1998) Maintaining

prosperity in an ageing society. OECD, Paris, p142

Parson E (2003) Protecting the ozone layer: Science and strategy. Oxford Univ Press,

Oxford, New York

Perrow C (1994) The limits of safety; the enhancement of a theory of accidents. J

Contingencies Crisis Mgt 2: 212-220

PSAC (1965) Report of the Environmental Pollution Panel. President’s Science Advisory

Committee, White House, November, p 123

Revelle RR (1983) In: Changing climate. Report of the Carbon Dioxide Committee.

National Academy Press, Washington, DC, pp 433-438 Roan S (1989) Ozone crisis : the 15-year evolution of a sudden global emergency. Wiley,

New York

Rowland FS (1995) Nobel Lecture in chemistry. Nobel Lecture, Dec 8

Schellnhuber HJ, Cramer W, Nakicenovic N, Wigley T, Yohe G (eds) (2006) Avoiding

dangerous climate change. Cambridge Univ Press, Cambridge, UK

Schrattenholzer L (1999) A brief history of the international energy workshop. In:

Weyant J (ed) Energy and environmental policy modeling. Kluwer Academic

Publishers, Dordrecht, The Netherlands, pp 177-185

Seltzer JR (2002) Origins and evolution of family planning programs in developing

countries. Rand Corp, Santa Monica, CA

Shlyakhter AI (1994) An improved framework for uncertainty analysis: Accounting for

unsuspected errors. Risk Anal 14: 441-447

Smil V (2003) Energy at the crossroads: Global perspectives and uncertainties. MIT

Press, Cambridge, MA

Syme J (1987) Forecast models and policy analysis: the case of oil prices.RAND, Santa

Monica, CA

Teitelbaum MS (1985) The fear of population decline. Academic Press, Orlando, FL

Tetlock PE (1991) Learning in US and Soviet foreign policy. In: Breslauer G, Tetlock PE

(eds) Search of an elusive concept. Westview, Boulder, CO

Tetlock PE (1999) Theory-driven reasoning about plausible pasts and probable futures in

world politics: are we prisoners of our own preconceptions? Am J Politcal Sci 43:

335-366

Tol RSJ (2006) Adaptation to five metres of sea level rise. J of Risk Res 9: 467-482 U.S. Atomic Energy Commission (AEC) (1974) Nuclear power growth 1974-2000.

WASH-1139(74), Office of Planning and Analysis, US Government Printing

Office, Washington, DC

UN Department of Economic and Social Affairs, Population Division (2005) World

population prospects: the 2004 revision. United Nations, New York

US Department of Energy (1982) Outlook for world oil prices. US Department of

Energy, Washington, DC

US Social Security Administration (SSA) (2005). The 2005 annual report of the board of

trustees of the federal old-age and survivors insurance and disability insurance

trust funds. http://www.ssa.gov/OACT/TR/TR05/.

Vaughan D (1996) The Challenger launch decision. Univ Chicago Press, Chicago

Vaughan DG (2006) West Antarctic ice sheet collapse − the fall and rise of a paradigm.

Clim Change, in press

Warrick RA, Oerlemanns J (1990) Sea level rise. In: Houghton JT, Jenkins GJ, Ephraums

JJ (eds) Climate change: the IPCC scientific assessment. Cambridge Univ Press,

New York, pp257-281

Warrick RA, LeProvost C, Meier MF, Oerlemanns J, Woodworth PL (1996). In:

Houghton JT et al (eds) Climate change 1995: the science of climate change.

Cambridge Univ Press, New York

Weitzman ML (2001) Gamma discounting. Amer Eco Rev 91(1): 260-271

WMO (1986) Atmospheric ozone 1985: Assessment of our understanding of the

processes controlling its present distribution and change. Rep No 16, WMO,

Geneva WMO (1988) Report of the International Ozone Trends Panel-1988, Global Ozone

Research and Monitoring Project. Rep No18, WMO, Geneva

WMO (1991) Scientific assessment of stratospheric ozone 1991. Rep No 25, WMO,

Geneva