TESTING A CALIBRATION APPROACH FOR A LIMITED-AREA ENSEMBLE PRECIPITATION FORECAST USING REFORECASTS Tommaso Diomede, Chiara Marsigli, Andrea Montani, Tiziana Paccagnella ARPA-SIMC, HydroMeteorological and Climate Service of the - Regional Agency for Environmental Protection, , E-mail: [email protected] Objective

To implement a calibration technique for the precipitation output provided by COSMO-LEPS, the Limited-area Ensemble Prediction System based on the model COSMO.

To investigate the potential of using reforecasts for the calibration of the ensemble precipitation forecast, especially with respect to the improvement of the forecast skill for rare events.

To test three calibration techniques: Cumulative Distribution Function based corrections, Linear Regression and Analogs. Calibration strategy methodologies study areas data collection • choice of methodologies which enable a calibration of 24h quantitative precipitation • Emilia-Romagna Region () • Observed precipitation forecasts (QPFs), not only of the probabilities of exceeding a threshold • Switzerland • Emilia-Romagna Region (1971-2007)  aim: • Germany • Switzerland (1971-2007) • Germany (1989-2007) –improvement of COSMO-LEPS QPFs especially as an input for hydrological applications • COSMO-LEPS reforecast QPFs • 30 years (1971-2000) • selected methods: • 1 member, nested on ERA40, COSMO v4.0 • 1 run every three days (forecast range 90h) - Cumulative Distribution Function based corrections (CDF) [m]

- Linear Regression (LR) • COSMO-LEPS operational QPFs • 5 years (2003-2007) similarity of precipitation (ANL) • 5-10-16 members (depending on the year), nested - Analogs on selected members of the ECMWF Ensemble similarity of geopotential at 700 hPa (anlZ) Prediction System Results • 1 run every day (forecast range 120h) Emilia-Romagna Switzerland Germany

Autumn 2003-2007 threshold : 95-th percentile fc: +20-44 h raw CDF raw Autumn 2003-2007 thresho ld: 95-th percentile fc: +18-42 h raw raw raw 1 CDF ANL 1 LR ANL verification of the calibration in terms of QPFs 1 anl Z CDF 1 LR CDF 1 CDF anl Z

Autumn 2003-2007 threshold: 95-th percentile fc: +18-42 h raw ANL ANL CDF ANL 0.1 1 0.1 ANL LR LR LR anl Z LR

(frequency of usage) of (frequency anl Z anl Z anl Z 0.01 0.1 (frequency usage) of (frequency 10 0.01 10

Log Log period: 2003-2007 0.8 0.8 Log 0.8

0.001 usage) of (frequency 0.01 0.001 10 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 forecast probability Log forecast probability season: autumn

0.001 0 0.2 0.4 0.6 0.8 1 forecast probability verification threshold: 95-th percentile of climatological observations 0.6 0.6 0.6 forecast range forecast range forecast range day 2 day 2 day 2 no skill no skill no skill

0.4 0.4 0.4  increase of forecast reliability and skill for the calibrated ensembles

observed relative frequency observedrelative frequency observedrelative frequency observedrelative over Switzerland and Germany

0.2 0.2 0.2

no resolution no resolution no resolution

0 0 0  no significant beneficial impact provided by the calibration over Emilia- 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 forecast probability forecast probability forecast probability Romagna

0.5 raw 0.5 raw 0.5 raw 0.4 cal CDF 0.4 cal CDF 0.4 cal CDF cal ANL cal ANL cal ANL 0.3 cal LR 0.3 cal LR 0.3 cal LR  overconfident forecasts for the raw and calibrated ensembles, cal anl Z cal anl Z cal anl Z 0.2 0.2 0.2 especially at the higher probabilities 0.1 0.1 0.1

BSS 0 BSS 0 BSS 0

-0.1 -0.1 -0.1  no single best calibration method for all areas -0.2 -0.2 -0.2

-0.3 -0.3 -0.3

-0.4 -0.4 -0.4 day 2 day 3 day 4 day 5 day 2 day 3 day 4 day 5 day 2 day 3 day 4 day 5  the geopotential at 700 hPa is not a good predictor for precipitation forecast range forecast range forecast range impact of using calibrated COSMO-LEPS QPFs on hydrological applications

discharge forecasts QPFs Autumn 2003-2007 raw Autumn 2003-2008 95-th percentile warning level raw Autumn 2003-2008 95-th percentile warning level raw 1 threshold: 95-th percentile study catchments 10 CDF 14 CDF cal CDF ANL ANL 13 lead time: day 2 0.9 cal ANL 9 lead time: day 2 LR LR 12 the coupling strategy observed events 0.8 cal LR 8 11 0.7 the meteorological input 7 10 the hydrological model 9 6 0.6 8 COSMO-LEPS TOPKAPI 5 7 0.5 6 raw and calibrated physically-based distributed 4 0.4 QPFs rainfall-runoff model 5 3 4 0.3 numbermisses of 3 0.2 ProbabilityDetectionOf

2 numberfalseof alarms 2 autumn 1 0.1 2003 - 2008 ensemble of TOPKAPI runs 1 0 0 0 driven by COSMO-LEPS Samoggia Reno Santerno Idice Sillaro Samoggia day 2 day 3 day 4 day 5 each day (forecast range: 5 days) catchment catchment forecast lead time Autumn 2003-2008 95-th percentile warning level raw Autumn 2003-2008 95-th percentile warning level raw Autumn 2003-2007 10 CDF 14 CDF 1 threshold: 95-th percentile study catchments lead time: day 4 ANL lead time: day 4 ANL 13 9 LR LR 0.9 observed events 12 the study catchments 8 11 0.8 7 10 0.7 9 6 8 0.6 Calcara (Idice) Mordano Casalecchio Chiusa 5 7 0.5 (Samoggia) (Santerno) (Reno) 6 raw Sesto Imolese 4 0.4 cal CDF (Sillaro) 5 3 4 0.3 FalseAlarmRate cal ANL number misses of 2 3 number falsealarms of 0.2 cal LR 2 1 0.1 Emilia- response times 1 Romagna 7-12 h 0 0 0 Reno Santerno Idice Sillaro Samoggia Reno Santerno Idice Sillaro Samoggia day 2 day 3 day 4 day 5 catchment dimensions catchment forecast lead time 200-1000 km2 missed events false alarms POD & FAR

main catchment (Reno) main catchment (Reno) all the catchments  all the events missed by the raw and LR-calibrated  no additional false alarms for the ensemble calibrated  the increase of POD and FAR for the ensembles by rainfall analogs at the shorter lead time, increase of ensembles calibrated by CDF and rainfall analogs  reduction of misses for the ensembles calibrated by false alarms only at the longer lead time with respect to the raw ensemble confirms the CDF and rainfall analogs  many false alarms for the ensemble calibrated by CDF outcomes of the meteo-hydrological coupling sub-catchments sub-catchments  the positive impact of calibration on the reduction of  similar performance in terms of false alarms for the misses is not evident as compared to Reno river basin raw and analog-calibrated ensembles; many false alarms (but few events occurred) for the CDF-calibrated ensemble at the longer lead time Conclusions & next developments  The calibration of COSMO-LEPS QPFs provides a beneficial impact over Switzerland and Germany.

 No significant improvements result over Emilia-Romagna from the statistical analysis on the calibrated QPFs, but the coupling with an hydrological model reveals a beneficial impact of calibration on the reduction of missed events for the Reno river basin.

 The lack of a strong improvement, especially for Emilia-Romagna, is due to a weak relationship between reforecast and observed data.

 As the model error is likely to have a systematic dependence on geography, orography and flow direction, the calibration strategy should be improved by dividing the training sample in order to pool data which have similar model errors with respect to a given meteorological situation ⇒ need of weather-regime dependent correction functions for the calibration performed by the CDF based corrections and Linear Regression methods.