Decadal climate forecasting

P. Geoffrey Allen, University of Massachusetts Robert Fildes, University of Lancaster Nikolaos Kourentzes, University of Lancaster Climate is what we expect, weather is what we get.

Robert A. Heinlein: Time Enough for Love Taken from A.J. Herbertson: Outlines of Physiography: An Introduction to the Study of the Earth (1901?)

By climate we mean the average weather as ascertained by many years’ observations. Climate also takes into account the extreme weather experienced during that period. Climate is what on an average we may expect, weather is what we actually get. The climate record

HadCRUT3 Annual Global Surface Temperature Anomaly Annual average global surface 0.6 temperature (T2m) expressed as 0.4 deviations from the 1961-1990

0.2 average. The red line is the

C centered 30-year moving average ° 0

-0.2 Source: UK Met Office Anomalyin http://www.metoffice.gov.uk/hadobs/hadcrut3/ -0.4 Annual average global temperature has -0.6 a standard deviation about the 30-year

-0.8 average of 0.11 °C. 1850 1870 1890 1910 1930 1950 1970 1990 2010 Year CMIP5 hindcasts of 10 years start in 1960 and are calculated at 5-year intervals. The climate record

HadCRUT3 Annual Global Surface Temperature Anomaly Annual average global surface 0.6 temperature (T2m) expressed as 0.4 deviations from the 1961-1990

0.2 average. The red line is the

C centered 30-year moving average ° 0

-0.2 Source: UK Met Office Anomalyin http://www.metoffice.gov.uk/hadobs/hadcrut3/ -0.4 Annual average global temperature has -0.6 a standard deviation about the 30-year

-0.8 average of 0.11 °C. 1850 1870 1890 1910 1930 1950 1970 1990 2010 Year CMIP5 hindcasts of 10 years start in 1960 and are calculated at 5-year intervals. The climate record

HadCRUT3 Annual Global Surface Temperature Anomaly Annual average global surface 0.6 temperature (T2m) expressed as 0.4 deviations from the 1961-1990

0.2 average. The green line is the

C trailing 30-year moving average ° 0 (climatology)

-0.2 Source: UK Met Office Anomalyin http://www.metoffice.gov.uk/hadobs/hadcrut3/ -0.4 Annual average global temperature has -0.6 a standard deviation about the 30-year

-0.8 average of 0.11 °C. 1850 1870 1890 1910 1930 1950 1970 1990 2010 Year CMIP5 hindcasts of 10 years start in 1960 and are calculated at 5-year intervals. Climate modelers do things differently Bjerknes realized that the earth’s atmosphere could Vilhelm Bjerknes be viewed as a compressible fluid that obeys the laws of physics.

On that basis he formulated the “primitive equations” - partial differential equations – that form the dynamical core of all modern numerical weather prediction (NWP) and global climate models (GCMs).

Models are deterministic, chaotic. Parameters estimated by ad-hoc, judgmental “tuning”. Simulation replaced formal tests (misspecification). Beginnings

The (US) Joint Numerical Weather Prediction Unit made its first successful operational weather forecast in early 1958, using a 53 x 57 grid (left, points roughly 400 km apart) After experiments with more complex models, this was a single- layer model.

As more powerful computers appeared models became more complex (more layers, closer grid-points, shorter time- steps) and more realistic (more variables: CO2, aerosols, sea ice, clouds, rain, etc). Also, atmospheric and ocean models were coupled. Coupled Atmosphere-Ocean General Circulation Models (AOGCMs) are complex

Atmosphere Component

100 Ocean Component 90 70 80 60 70 60 50 50 40 40 30 30 20 20 10 Number of Layers of Number 10 0 layers of Number 0 0 0.5 1 1.5 2 2.5 3 3.5 4 0 0.5 1 1.5 2 2.5 Grid spacing (degrees) Grid spacing (degrees) Dots represent the models in CMIP5. And almost all have components for sea ice, aerosols, rain, cloud cover, etc. But has complexity brought more accuracy? Chaotic systems

The system is deterministic. If started from the same initial condition, it repeats exactly. If the initial condition is perturbed slightly, a different pattern will emerge. But the patterns are stable and have similar appearance. Meteorologist view: the climate system is complex but well understood and modeled realistically. Its chaotic nature limits forecast horizon. Economist view: the economic system is complex but not well understood so is approximated by stochastic linear models. Lack of understanding limits forecast horizon. Different views on sources of error

Source of error Climate model Statistical model Significant problem; resolved by perturbation Initial conditions Not usually an issue or other sampling approach Often ignored; parameters resolved by Often ignored but statistical estimates Parameter estimates "tuning"; statistical measures unavailable available Recognized; often assessed by comparing Major concern; tests on residuals Model misspecification ensemble forecasts from different climate commonly used models, rarely different statistical models Randomness Not viewed as a source of error Explicit in the model Of concern; data reanalysis frequently Measurement error usually ignored; effect Data performed; several data sets for same variable of choice of data vintage when revisions (e.g. global surface temperature) exist occur recognized Different standard software assumed to Choice of time step, of particular calculation Algorithm give same answer (though this not always routine recognized as issue true) Ignored (though duplication of published Exist in complex models, resolved when Coding errors results often difficult, possibly the discovered consequence of tacit knowledge) The forecasting performance - I

Numerical weather forecasting skill has increased over time (in line with increasing computing power) Its accuracy can be easily checked (S1 score is a function of the average pressure difference between pairs of points) (National Centers for Environmental Prediction, formerly National Meteorological Center) What about century-scale simulations?

/8 (which use the same models, though not the

same data)

IBM701 IBM704

IBM7094

CDC660 CDC660

Cyber205

IBM360/195 CrayYMP CrayC90/16 S1 score = 푓표푟푒푐푎푠푡 − 표푏푠푒푟푣푒푑 / max(표푏푠푒푟푣푒푑, 푓표푟푒푐푎푠푡) The method of decadal forecasting

• Control run. Model run for several centuries, initiated with pre- industrial (1850) exogenous variables. Model is “tuned” until it generates stable weather-like output – the model’s climatology • Spin up. Model run from 1850 (usually) to date with actual exogenous variable values (CO2 most important) • Hindcast. (e.g., from 1960 for 10 years) Initialization a big issue • None. Spin up continues, usually with actual exogenous variables. • Full field. Actual observations, e.g., in 1960 assimilated into the model state using a form of Kalman filtering • Anomaly. Deviations from climatological mean assimilated into the model state Predicting climate or average weather?

Meteorologists seem unsure. If a simulation is allowed to run the red band indicates the distribution of outcomes (e.g., of surface temperature) that form the hindcast. If initial conditions are assimilated into the model at the start of the hindcast the model should give a better prediction of near-term annual average weather, shown by the blue band. The sharply peaked forecast distribution based on initial conditions broadens with time as the influence of the initial conditions fades until the probability distribution of the initialized prediction approaches that of an uninitialized projection. (Based on Branstator and Teng, 2010.) Source: IPCC AR5 chapter 11. Global-mean temperature (2 Bias correction is important years running mean) for (top left) HadGEM2 [Met Office (UKMO)], (top right) IFS/HOPE (ECMWF), (bottom left) ARPEGE4/OPA [Centre Europeen de Recherche et de Formation Avancee en Calcul Scientifique (CERFACS)], and (bottom right) ECHAM5 [Leibniz-Institut für Meereswissenschaften (IFM- GEOMAR)] Note that the scale on the vertical axis for the ARPEGE4/OPA model is different than for the other three models, reflecting the larger bias in this model.

Source: Suckling and Smith (2013) Comparison: with linear inverse models

• LIM: extraction of the dynamical properties of a system from its observations • Both spatial and temporal dimensions • Singular value decomposition or other principal components method

RMSE for global mean temperature, from hindcasts initialized in the years 1960–2000. LIM compared with persistence (an AR(1) model) and three AOGCMs

Source: Newman (2013)

MPI-ESM-LR is the Max Planck Institute Earth System Model, low resolution. GFDL CM2.1 is the Geophysical Fluid Dynamics Laboratory Climate Model, version 2.1. DePreSys is the UK Met Office Hadley Centre decadal forecasting variant of HadCM3 Comparison: with regression analysis

Krueger and von Storch (2011) decomposed the equation

xt = α + βCt-1 + φxt-1 + εt

Where xt is annual average surface temperature and Ct is annual CO2 concentration and t = 1883, . . . , 1999 excluding a 10-year gap (covering the hindcast), i.e., this is cross-validation. The decomposition is not needed for forecasting.

RMSE of forecasts from GCMs compared with regression Interval Year Laepple et al. (2008) made annual Model covered 1 9 predictions using results from the 3rd CMIP3 (Laepple) 1930-2006 0.106 0.14 Coupled Model Intercomparison Project (CMIP3). Smith (2007) made K & v S 1930-2006 0.100 0.12 annual predictions using DePreSys DePreSys (Smith) 1982-2004 0.07 0.13 K & v S 1982-2004 0.09 0.10

Combining and encompassing

• If one method does not forecast-encompass another, then a composite forecast will be more accurate, and the encompassing test result will give information about what one method lacks compared with the other.

• Tempt = β0 + β1 ForMeth1t-10(t) + (1 - β1)ForMeth2t-10(t) + errort • where Temp is the observed temperature anomaly, ForMeth is the forecast anomaly from the method.

• If β1 = 0, method 2 forecast encompasses (FE) method 1, if β1 > 0, method 2 does not FE method 1 • Test method1 = statistical, method2 = GCM, then reverse them Comparison with univariate and neural networks

Decadal forecasts (horizon = 10 years) of global Combination of DePreSys and NN more average surface temperature, 1992 – 2011 (20 origins) accurate than either, but DePreSys and Model MAE Holts is not. Exponential smoothing (Holt) 0.098 Why? Damped trend exponential smoothing 0.140 NN is not forecast encompassed (FE) by Neural network - univariate 0.128 DePreSys and DePreSys is not FE by NN so both contribute something and a Neural network - multivariate (CO2) 0.126 DePreSys 0.157 combination is more accurate. Holt’s DePreSys+Holt 0.104 does best alone, it does FE DePreSys. DePreSys+NN multivariate 0.121 Results of forecast encompassing tests Source: Calculations by the authors Meth1=statistical, Meth2 = DePreSys Meth1= DePreSys, Meth2 = statistical p-value of statistical p-value of DePreSys adds Model Adds to DePreSys model DePreSys information Exponential smoothing (Holt) 0.000 Yes 0.754 No Neural network - univariate 0.000 Yes 0.001 Yes

Neural network - multivariate (CO2) 0.003 Yes 0.006 Yes Lessons learned • Initial conditions and their effects the focus of much research • less concern with model specification and parameter uncertainty. • Models are tuned in an ad hoc way to deliver weather-like outputs (as often as every three hours) for hundreds of years. • their dynamics are built to deliver a steady climatic state. • The ability to forecast average weather , e.g., global surface temperature, more than a few days ahead, depends on forecasting slower evolving climate components, mainly ocean temperatures (e.g., El Nino – Southern Oscillation). • Decadal forecasts depend on so-called boundary conditions, e.g. CO2 concentrations • limited data to estimate relation between changes in CO2 concentration and global average climate. • Decadal forecasting appears to be an attempt to increase confidence in the long-term projections of global warming • however, accuracy of GCMs at decadal forecasting currently no better than simple stochastic models. • GCMs do not forecast encompass simple models, a combined forecast would be more accurate.