Nat. Hazards Earth Syst. Sci., 18, 2697–2716, 2018 https://doi.org/10.5194/nhess-18-2697-2018 © Author(s) 2018. This work is distributed under the Creative Commons Attribution 4.0 License. Spatial consistency and bias in avalanche forecasts – a case study in the European Alps Frank Techel1,2, Christoph Mitterer5, Elisabetta Ceaglio3, Cécile Coléou4, Samuel Morin6,8, Francesca Rastelli7, and Ross S. Purves2 1WSL Institute for Snow and Avalanche Research SLF, Davos, Switzerland 2Department of Geography, University of Zürich, Zurich, Switzerland 3Fondazione Montagna sicura, Ufficio neve e valanghe, Regione Autonoma Valle d’Aosta, Italy 4Météo France, Direction des Opérations pour la Prévision, Cellule Montagne Nivologie, Grenoble, France 5Lawinenwarndienst Tirol, Abteilung Zivil- und Katastrophenschutz, Innsbruck, Austria 6Météo France – CNRS, CNRM UMR 3589, Centre d’Études de la Neige, Grenoble, France 7Meteomont Carabinieri, Bormio, Italy 8Université Grenoble Alpes, Université de Toulouse, Toulouse, France Correspondence: Frank Techel ([email protected]) Received: 16 March 2018 – Discussion started: 28 March 2018 Revised: 6 September 2018 – Accepted: 5 October 2018 – Published: 23 October 2018 Abstract. In the European Alps, the public is provided with danger level 4 – high and 5 – very high. The size of the warn- regional avalanche forecasts, issued by about 30 forecast cen- ing regions, the smallest geographically clearly specified ar- ters throughout the winter, covering a spatially contiguous eas underlying the forecast products, differed considerably area. A key element in these forecasts is the communication between forecast centers. Region size also had a significant of avalanche danger according to the five-level, ordinal Eu- impact on all summary statistics and is a key parameter in- ropean Avalanche Danger Scale (EADS). Consistency in the fluencing the issued danger level, but it also limits the com- application of the avalanche danger levels by the individual munication of spatial variations in the danger level. Oper- forecast centers is essential to avoid misunderstandings or ational constraints in the production and communication of misinterpretations by users, particularly those utilizing bul- avalanche forecasts and variation in the ways the EADS is in- letins issued by different forecast centers. As the quality of terpreted locally may contribute to inconsistencies and may avalanche forecasts is difficult to verify, due to the categorical be potential sources for misinterpretation by forecast users. nature of the EADS, we investigated forecast goodness by fo- All these issues highlight the need to further harmonize the cusing on spatial consistency and bias, exploring real forecast forecast production process and the way avalanche hazard is danger levels from four winter seasons (477 forecast days). communicated to increase consistency and hence facilitate We describe the operational constraints associated with the cross-border forecast interpretation by traveling users. production and communication of the avalanche bulletins, and we propose a methodology to quantitatively explore spa- tial consistency and bias. We note that the forecast danger 1 Introduction level agreed significantly less often when compared across national and forecast center boundaries (about 60 %) than In the European Alps, public forecasts of avalanche hazard within forecast center boundaries (about 90 %). Furthermore, are provided throughout the winter. These forecasts – also several forecast centers showed significant systematic differ- called advisories, warnings, or bulletins1 – provide informa- ences in terms of more frequently using lower (or higher) tion about the current and forecast snow and avalanche con- danger levels than their neighbors. Discrepancies seemed to ditions in a specific region. In contrast to local avalanche be greatest when analyzing the proportion of forecasts with 1We use these terms synonymously. Published by Copernicus Publications on behalf of the European Geosciences Union. 2698 F. Techel et al.: Spatial consistency and bias in avalanche forecasts forecasting, e.g., for a transportation corridor or ski area, a Forecast validation and evaluation is a problem not only in regional forecast does not provide information regarding in- avalanche forecasting but also more generally in forecasting. dividual slopes or specific endangered objects. Murphy(1993), in his classic paper on the nature of a good One of the key consumer groups are those undertaking (weather) forecast, discussed three key elements which he recreational activities, such as off-piste riding and back- termed consistency, quality and value. Consistency in Mur- country touring in unsecured terrain. The importance of phy’s model essentially captures the degree of agreement be- clearly communicating to this group is underlined firstly by tween a forecaster’s understanding of a situation and the fore- avalanche accident statistics – with on average 100 fatalities cast they then communicate to the public. Quality captures each winter in the Alps (Techel et al., 2016), most of which the degree of agreement between a forecast and the events occurring during recreational activities. Secondly, very large which occur, and value the benefits or costs incurred by a numbers of individuals recreate in unsecured winter terrain, user as a result of a forecast. with for example Winkler et al.(2016) reporting that more In avalanche forecasting, two key problems come to the than 2 million winter backcountry touring days were un- fore. Firstly, the target variable is essentially categorical, dertaken in 2013 in Switzerland alone. An additional con- since, although the EADS is an ordinal scale, a real eval- sumer group is local, regional, and national risk manage- uation of a forecast would compare the forecast danger ment authorities, who base risk reduction strategies such level, qualitatively defined in the EADS, with the prevail- as avalanche control measures, road closures, and evacua- ing avalanche situation. Secondly, since the target variable tion procedures. in part on information provided in regional captures a state which may or may not lead to an (avalanche) avalanche forecasts. event, verification of forecast quality is only possible in some In all Alpine countries (Fig.1), forecasts are disseminated circumstances and for some aspects of the EADS, such as the throughout the entire winter, for individual warning regions, following: together forming a spatially contiguous area covering the en- – At higher danger levels, the occurrence of natural tire Alpine region. Furthermore, in all of these countries the avalanches can sometimes be used to verify the danger European Avalanche Danger Scale (EADS; EAWS, 2018), level (e.g., Elder and Armstrong, 1987; Giraud et al., introduced in 1993 (SLF, 1993), is used in the production 1987; Schweizer et al., 2018). and communication of forecasts (EAWS, 2017c). The EADS is an ordinal, five-level scale focusing on – At lower danger levels, the occurrence of avalanches avalanche hazard, with categorical descriptions for each dan- triggered by recreationists or the observation of signs ger level describing snowpack (in)stability, avalanche release of instability requires users being present. probability, expected size and number of avalanches, and the – Since the absence of avalanche activity is not alone an likely distribution of triggering spots (Table1). The EADS indicator of stability, verifying associated danger levels describes not only situations with spontaneous avalanches is only possible through digging multiple snow profiles but also conditions where an additional load – such as a per- and performing stability tests (Schweizer et al., 2003). son skiing a slope – can trigger an avalanche. These cate- gorical descriptions of each danger level aim to inform users Thus, avalanche danger cannot be fully measured or vali- on the nature of avalanche hazard at hand. However, individ- dated (Föhn and Schweizer, 1995). This in turn means that, ual danger levels capture a wide range of differing avalanche at least at the level of the EADS, it is conceptually difficult conditions (e.g., EAWS, 2005; Lazar et al., 2016; EAWS, to directly measure forecast quality. However, Murphy’s no- 2017a; Statham et al., 2018a) and therefore, in isolation, are tion of considering goodness of forecasts in terms of not only too basic to be used as a stand-alone decision-making tool their quality but also consistency and value suggests a possi- (e.g., Météo France, 2012). Additionally, and in order to ble way forward. describe the avalanche hazard in more detail and to provide Although Murphy defines consistency with respect to an better advice to the end users on how to manage these haz- individual forecaster, we believe that the concept can be ex- ards, the European Avalanche Warning Services (EAWS) in- tended to forecast centers, in terms of the degree to which troduced a set of five typical avalanche problems (EAWS, individual forecasters using potentially different evidence 2017d). Nonetheless, the EADS provides a consistent way reach the same judgment (LaChapelle, 1980), and across of communicating avalanche hazard. Furthermore, the EADS forecast centers, in terms of the uniformity of the forecast often serves as an important input into basic avalanche edu- issued by different forecast centers in neighboring regions. cation on planning or decision-making heuristics as practiced This reading of consistency is, we believe, true to both Mur- by many recreationists