174 Proceedings of the Estonian Academy of Sciences, 2014, 63, 2, 174–182 Proceedings of the Estonian Academy of Sciences, 2014, 63, 2, 174–182 doi: 10.3176/proc.2014.2.07 Available online at www.eap.ee/proceedings Statistics of different public forecast products of temperature and precipitation in Estonia Sirje Keevallika*, Natalja Spirinab, Eva-Maria Sulab, and Inga Vauc a Marine Systems Institute at Tallinn University of Technology, Akadeemia tee 15a, 12618 Tallinn, Estonia b Estonian Environment Agency, Mustamäe tee 33, 10616 Tallinn, Estonia c Estonian Information Technology College, Raja 4c, 12616 Tallinn, Estonia Received 30 October 2013, revised 10 March 2014, accepted 2 April 2014, available online 20 May 2014 Abstract. Day (0600–1800 UTC) maximum and night (1800–0600 UTC) minimum temperature forecasts as well as prediction of the occurrence of precipitation are evaluated for different sites in Estonia: southern coast of the Gulf of Finland (Tallinn), West- Estonian archipelago (Kuressaare), and inland Estonia (Tartu). The forecasts are collected from Estonian weather service. Several traditional verification methods are used, first of all reliability (root mean square error (RMSE)) and validity (mean error (ME)). Detailed analysis is carried out by means of the contingency tables that enable the user to calculate percent correct, percent underestimated, and percent overestimated. The contingency tables enable the user to calculate conditional probabilities of the realizations of certain forecasts. The paper is user-oriented and does not analyse the forecast technique. On the other hand, attention is drawn to the subjectivity of such evaluation, as the results may depend on the forecast presentation style and/or on the choice of the features of the meteorological parameter under consideration. For the current case study (the coldest hour during night and the warmest hour during day chosen to validate the temperature forecast, the temperature validation bin size 3 degrees, precipitation forecast validated in three categories based on the 12 h precipitation sums) one may say that the RMSE of the short- term prediction of night minimum and day maximum temperature is 1.5 °C…3.1 °C. It was also noticed that Estonian weather service predicts lower night minimum temperature than it follows in reality. The skill of the temperature forecast is estimated by comparison of its RMSE with that of the persistence forecast (next night/day will be similar to the previous one). The RMSE of the 1st day/night forecast is by 1.3 °C…1.4 °C less for Tallinn and Tartu and 0.3 °C…0.8 °C for Kuressaare than that of the persistence forecast. For the precipitation forecast, percent correct is 60…70, the probability that dry weather forecast is followed by no precipitation is 70%…80%. At the end of the paper the long-term forecasts of two international weather portals www.gismeteo.ru (Russia) and www.weather.com (USA) are briefly analysed. Key words: temperature forecast, precipitation forecast, public forecast products, forecast verification, contingency tables. 1. INTRODUCTION carried out by means of theoretical considerations or by * comparison of the given forecasts with the values of the Weather forecast is based on the predictability of atmo- meteorological parameters that were really measured. spheric processes that is tightly related to the chaos Due to the large dimensionality, the task is not trivial theory. It is widely known that there are fundamental and, as a rule, the predictability of different components limits on the deterministic predictability of the atmo- is of different quality. Even more, the results depend sphere, as the development of the processes is sensitive substantially on the details (range, bin size, intensity, to the variations in the initial conditions. Additional occurrence, etc.) of the meteorological parameters under errors to the forecasts may be introduced by the schemes consideration. of numerical weather prediction due to the approximate The simplest way to get information on the forecast simulation of atmospheric processes. Therefore, estima- quality is calculating root mean square error comparing tion of the forecast quality is necessary. This can be the predicted values with the measured ones. We speak about a good forecast when the RMSE is small and * Corresponding author, [email protected] about a bad forecast when it is large. The other simple S. Keevallik et al.: Statistics of different public forecast products of temperature and precipitation in Estonia 175 characteristic of the sample is mean error or bias. On the analysis of their specific economic or social problems. other hand, such knowledge is of low value when we This is especially important at the risk calculations in want to estimate risk that needs two components: prob- conditions of uncertain forecast. ability and cost. To evaluate the forecast of dichotomous weather events, several economically sensible charac- teristics are offered [1–3]. These estimates are based on 2. MATERIAL AND METHODS the contingency table that permits the user to calculate the accuracy in the form of percent correct (PC): the Weather forecasting at the Estonian weather service is ratio of hits plus correct negatives to the total number of based on the HIRLAM and ECMWF systems but the forecasts. A similar score can be used also when the products of the numerical weather prediction (NWP) forecast is given for a range of meteorological para- models are complemented with the analysis of surface meters, e.g., temperature, precipitation sum, wind speed, and higher atmosphere weather maps. Therefore, the etc. In this case PC is calculated as the ratio of the human experience is considered to be important [14,15]. number of correct predictions to the total number of The forecaster determines the direction of the main air predictions. In a similar way also the percentage of flow and pressure tendencies. Air moisture content is under- and overestimations can be calculated that is decisive by the forecast of cloudiness and precipitation. somewhat better indicator than bias. The isotherms on the 850 hPa level and the height of the Heideman et al. [4] confirm that although numerous air layer are important by the forecast of the winter methods to estimate skill and accuracy of forecasts are precipitation phase. The analysis of higher level maps available, there exists no single measure to evaluate the gives a possibility to estimate the development of the forecast quality, as different users have different interests. pressure systems and their motion. Additional informa- This is especially evident for precipitation forecasts as tion is drawn from the forecast charts of other countries some users need quantitative precipitation data [5] while that are available in the Internet. others are interested in the rainfall threshold and spatial The forecasters send material to the telecommunica- scale [6]. Special procedures should be used to provide tion department where forecast maps, provided with profit-oriented enterprises with the necessary information icons, are prepared for the public use. In the present to make proper decisions [7,8]. On the other hand, study, the files of the forecasters are used. economic benefits from improved forecast accuracy Estonian weather service gives official weather fore- should be estimated to justify the investments into the cast every day at 1200 UTC. Night minimum and day weather service [9]. maximum temperature is predicted and archived for Stewart et al. [10] have claimed that even the most three following nights and days. For these forecasts, accurate forecast is of little value if it is not well used. In night means 1800–0600 UTC (2000–0800 East European most cases the decisions must be made in the conditions winter time) and day means 0600–1800 UTC (0800– of imperfect weather forecasts and the user must get 2000 East European winter time). One should keep in maximal profit or minimal loss from its skilful applica- mind that in this context night minimum does not tion. One method to help the decision-makers to account necessarily mean daily minimum, as sometimes the the cost–loss balance is calculating conditional prob- lowest temperature is measured early morning after abilities from the contingency tables: what is the prob- 8 a.m. The same should be said about day maximum that ability that the predicted situation takes place? Together is not necessarily the highest daily temperature. with an economic value statistics such conditional prob- Forecasts for three Estonian sites were used in the abilities might help to make the best decision [2,11]. present study (Fig. 1): Tallinn on the northern coast of Private users are mostly interested in the precipita- Estonia, Tartu as an inland site, and Kuressaare in the tion and temperature forecasts while the most important West-Estonian archipelago. aspects are when and where precipitation will occur To estimate the quality of the local forecasts, a three- [12]. On the other hand, people do not always under- year period of 2009–2011 was used. stand the language of weather forecasts adequately and For the values of the measured night minima (day effective communication of the weather forecast un- maxima) the average temperature of the coldest certainty needs special skills [13]. (warmest) hour was taken. These data were drawn from The present paper is written from the viewpoint of the archives of Estonian weather service. Tallinn is the user of the temperature and precipitation forecasts,