Comparison of Methods Used to Generate Probabilistic Quantitative Precipitation Forecasts Over South America
Total Page:16
File Type:pdf, Size:1020Kb
FEBRUARY 2009 R U I Z E T A L . 319 Comparison of Methods Used to Generate Probabilistic Quantitative Precipitation Forecasts over South America JUAN RUIZ AND CELESTE SAULO Centro de Investigaciones del Mar y la Atmo´sfera–CONICET/University of Buenos Aires, and Departamento de Ciencias de la Atmo´sfera y los Oce´anos, Facultad de Ciencias Exactas y Naturales, University of Buenos Aires, Buenos Aires, Argentina EUGENIA KALNAY Department of Atmospheric and Oceanic Sciences, University of Maryland, College Park, College Park, Maryland (Manuscript received 6 December 2007, in final form 22 August 2008) ABSTRACT In this work, the quality of several probabilistic quantitative precipitation forecasts (PQPFs) is examined. The analysis is focused over South America during a 2-month period in the warm season. Several ways of generating and calibrating the PQPFs have been tested, using different ensemble systems and single-model runs. Two alternative calibration techniques (static and dynamic) have been tested. To take into account different precipitation regimes, PQPF performance has been evaluated over two regions: the northern part of South America, characterized by a tropical regime, and the southern part, where synoptic-scale forcing is stronger. The results support the adoption of such area separation, since differences in the precipitation regimes produce significant differences in PQPF performance. The more skillful PQPFs are the ones obtained after calibration. PQPFs derived from the ensemble mean also show higher skill and better reliability than those derived from the single ensemble members. The performance of the PQPFs derived from both ensemble systems is similar over the southern part of the region; however, over the northern part the superensemble approach seems to achieve better results in both reliability and skill. Finally, the impact of using Climate Prediction Center morphing technique (CMORPH) estimates to calibrate the precipitation forecast has been explored since the more extensive coverage of this dataset would allow its use over areas where the rain gauge coverage is insufficient. Results suggest that systematic biases present in the CMORPH estimates produce only a slight degradation of the resulting PQPF. 1. Introduction and even to the assimilation of single observations. Continuous efforts are devoted to improving forecast Quantitative precipitation forecasts (QPFs) are one quality, with ensemble forecasting being an example of of the most difficult and least accurate products avail- one possible strategy for dealing with errors arising able via numerical weather prediction (NWP) (Ebert from uncertainties in the initial conditions (Toth and 2001; Stensrud and Yussouf 2007). Moreover, the con- Kalnay 1993, 1997; Molteni et al. 1996, and many tinuous increase in model resolution poses an extra others). Although initially conceived for use in medium- challenge to QPFs given the highly unpredictable to long-range global forecasts, in the last few years en- character of the mesoscale precipitation features. Ac- semble forecasting has been tailored to short-range cording to Zhang et al. (2002, 2006), the forecasted weather prediction through the use of regional model details of precipitation patterns for a particular day are ensembles (Du et al. 1997; Hamill and Colucci 1997). This sensitive to the initial conditions, the model resolution, possibility makes ensemble systems more appealing to operational and/or research centers with less computa- Corresponding author address: Juan Ruiz, Centro de Inves- tional resources, as in our own case. An interesting char- tigaciones del Mar y Atmo´ sfera, Facultad de Ciencias Exactas y Naturales, Ciudad Universitaria, Pabello´ n II, 2do Piso, Buenos acteristic of ensemble systems is that probability forecasts Aires 1428, Argentina. can easily be created, leading to the generation of prob- E-mail: [email protected] abilistic QPFs (PQPFs) (Du et al. 1997, among others). DOI: 10.1175/2008WAF2007098.1 Ó 2009 American Meteorological Society Unauthenticated | Downloaded 09/29/21 10:36 PM UTC 320 WEATHER AND FORECASTING VOLUME 24 Different methodologies for obtaining PQPFs, and ensemble’’ approaches have the potential to produce corresponding measures to quantify their usefulness, reliable forecasts, since they incorporate both uncer- have been developed. Of particular interest is how to tainty in the initial conditions (since members start from obtain a reliable PQPF, that is, a system where the different times or from different analyses) and in the forecasted frequency of a particular weather phenom- model errors (particularly in the representation of enon is close to the observed probability. The impor- subgrid-scale processes) because of their multimodel tance of PQPF reliability is directly related to its effect nature, while being essentially cost free in terms of upon the economic value of the forecast. As discussed computer use. by Zhu et al. (2002) for a simple cost–loss analysis of The effects of static and dynamic calibrations upon the forecast economic value, the optimum probability PQPF quality are assessed in this study. Previous work threshold to take protection from a particular weather (e.g., HC98) used a fixed dataset to calibrate the PQPFs phenomenon can be determined theoretically, provided (i.e., static calibration), but here a dynamic calibration that the information is reliable. However, as shown by dataset consisting of data from days previous to the Hamill and Colucci (1998, hereafter HC98) the PQPFs actual forecast date is also tested. The potential benefit derived directly from the ensemble are usually not re- of the dynamic calibration is that it includes information liable since model errors and the methods selected to about the current weather regime in the calibration construct the ensemble introduce biases in the fore- process, while it is not affected by model changes, since casted probabilities. it is recalculated every day at very low computational Several techniques have been developed to generate cost. These advantages, if accompanied by a similar reliable PQPFs. For example, Hamill and Colucci level of performance as that obtained with a static cal- (1997) introduced a technique based on rank histograms ibration procedure, make this alternative very attractive constructed with previous forecasts, which are then used within an operational framework. to calibrate the PQPF. This PQPF proved to be more PQPFs obtained via the combination of the above- reliable than the uncalibrated forecasts. This technique mentioned ensemble systems/calibration techniques are was further investigated and improved by Eckel and analyzed through the computation of several scores that Walters (1998), who performed a more detailed analysis also allow for comparison with results obtained in pre- of the dependence of the rank histograms based on vious works. To the authors’ knowledge, this is the first ensemble spread. Another interesting alternative arises time that a comparison of a wide variety of PQPFs has from creating PQPFs based on the ensemble mean been carried out, especially for South America, a region (HC98), which can be as reliable as the calibrated en- that is limited by poor data coverage. semble PQPF. Gallus and Segal (2004) applied this idea The lack of enough rain gauge precipitation data to to a single-model run and showed that the PQPF de- perform the calibration process is one of the main rived from a deterministic forecast is reliable and pro- constraints in applying the previous methodologies to vides good guidance to the forecasters. These results operational PQPFs over this region. Moreover, the should be taken into account when a cost–benefit amount of rain gauge data is usually insufficient to even strategy is under consideration: Are PQPFs derived perform a proper verification of the precipitation fore- from ensemble systems reliable, skillful, and valuable? casts (Saulo and Ferreira 2003). However, in the last If so, do they outperform those generated from single- decade, precipitation datasets such as the Climate Pre- model runs, which are much cheaper and faster to obtain? diction Center morphing technique (CMORPH; Joyce To address this issue, various PQPFs generated et al. 2004), which combine microwave estimates of through two different regional ensemble systems and precipitation with high temporal resolution IR estimates through individual model runs over South America, are of cloud motion, have become available. CMORPH has evaluated in this work. One regional ensemble system a homogeneous regional coverage and high spatial as is based on the scaled lagged averaged forecasting well as temporal resolution (30-min accumulated pre- (SLAF) technique (Ebisuzaki and Kalnay 1991), which cipitation). For these reasons, we have explored the po- is one of the simplest and computationally most inex- tential of using CMORPH data for PQPF calibration, pensive methods to incorporate uncertainties in the since it could be an interesting alternative for PQPF initial and boundary conditions into a regional ensem- calibration over this and other regions where the gauge ble. The other one is based on a mixed global–regional network is too coarse. model ensemble system [Super Model Ensemble Sys- This work is organized as follows: in section 2, data- tem (SMES)] approach (Silva Dias et al. 2006). As has sets used for verification and calibration, and method- been discussed by Krishnamurti et al. (1999) and Ebert ologies used to obtain PQPFs, are presented.