FEBRUARY 2017 B U E H N E R E T A L . 617 An Ensemble Kalman Filter for Numerical Weather Prediction Based on Variational Data Assimilation: VarEnKF MARK BUEHNER Data Assimilation and Satellite Meteorology Research Section, Environment and Climate Change Canada, Dorval, Quebec, Canada RON MCTAGGART-COWAN Numerical Weather Prediction Research Section, Environment and Climate Change Canada, Dorval, Quebec, Canada SYLVAIN HEILLIETTE Data Assimilation and Satellite Meteorology Research Section, Environment and Climate Change Canada, Dorval, Quebec, Canada (Manuscript received 15 March 2016, in final form 27 September 2016) ABSTRACT Several NWP centers currently employ a variational data assimilation approach for initializing deterministic forecasts and a separate ensemble Kalman filter (EnKF) system both for initializing ensemble forecasts and for providing ensemble background error covariances for the deterministic system. This study describes a new ap- proach for performing the data assimilation step within a perturbed-observation EnKF. In this approach, called VarEnKF, the analysis increment is computed with a variational data assimilation approach both for the ensemble mean and for all of the ensemble perturbations. To obtain a computationally efficient algorithm, a much simpler configuration is used for the ensemble perturbations, whereas the configuration used for the ensemble mean is similar to that used for the deterministic system. Numerous practical benefits may be realized by using a varia- tional approach for both deterministic and ensemble prediction, including improved efficiency for the develop- ment and maintenance of the computer code. Also, the use of essentially the same data assimilation algorithm would likely reduce the amount of numerical experimentation required when making system changes, since their impacts in the two systems would be very similar. The variational approach enables the use of hybrid background error covariances and may also allow the assimilation of a larger volume of observations. Preliminary tests with the Canadian global 256-member system produced significantly improved ensemble forecasts with VarEnKF as compared with the current EnKF and at a comparable computational cost. These improvements resulted entirely from changes to the ensemble mean analysis increment calculation. Moreover, because each ensemble pertur- bation is updated independently, VarEnKF scales perfectly up to a very large number of processors. 1. Introduction Data assimilation is used to provide the analyses (i.e., initial conditions) for both deterministic and ensemble Most numerical weather prediction (NWP) centers forecasts. For deterministic forecasts, variational data operationally produce both deterministic and ensem- assimilation approaches are most often used. These in- ble forecasts. The deterministic forecast represents clude three-dimensional variational data assimilation the best single estimate of the atmospheric conditions (3DVar), four-dimensional variational data assimilation in the future, whereas the ensemble forecast provides (4DVar), and, more recently, ensemble–variational as- information on the range of possible conditions that similation (EnVar; Buehner et al. 2013; Kleist and Ide could occur given the uncertainties inherent in all as- 2015; Wang and Lei 2014). For ensemble data assimi- pects of the prediction system. lation approaches, the goal is to produce an ensemble of model states consistent with the probability density of Corresponding author e-mail: Mark Buehner, mark.buehner@ the initial condition uncertainty. Several of these en- canada.ca semble techniques are based on Monte Carlo simulation DOI: 10.1175/MWR-D-16-0106.1 For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/ PUBSReuseLicenses). Unauthenticated | Downloaded 10/07/21 08:39 PM UTC 618 MONTHLY WEATHER REVIEW VOLUME 145 in which all uncertain components of the prediction variational algorithm used for deterministic data assimi- system are randomly perturbed in a way that is consis- lation. The analysis increment is computed with a varia- tent with the presumed uncertainty of each component. tional assimilation approach separately for the ensemble This includes the observations and forecast model pa- mean and for all of the ensemble perturbations (i.e., the rameters. It is also common to directly modify the en- deviations of each member from the ensemble mean). To semble spread of the complete model state to account for obtain a computationally efficient algorithm, a much sim- multiple sources of uncertainties (e.g., Houtekamer et al. pler configuration is used for the ensemble perturbations, 2009; Whitaker and Hamill 2012). Such data assimilation whereas the configuration used for the ensemble mean is approaches include the perturbed-observation ensemble similar to that used for the deterministic system. Since the Kalman filter (EnKF; Houtekamer et al. 2005) and the new approach is essentially an EnKF implemented with a ensemble of data assimilations (EDA; Isaksen et al. variational approach, it is called VarEnKF. 2010). Other EnKF algorithms, referred to as ensemble The next section includes a description of the VarEnKF square root filters, rely on a modification to the original approach together with its expected benefits. Section 3 algorithm to avoid the need to perturb the observations provides a description of the numerical data assimilation (e.g., Whitaker and Hamill 2002; Tippett et al. 2003; Hunt experiments performed using either a standard EnKF et al. 2007). Alternatively, some centers compute per- approach, VarEnKF, or a combination of the two. The turbations with an approach not directly related to data results from these experiments are presented in section 4. assimilation (e.g., singular vectors, bred vectors; Buizza The final section provides the conclusions. et al. 2005) and these are then added to the deterministic analysis for initializing the ensemble forecast. Ensembles of short-term forecasts are also used within 2. VarEnKF approach several types of data assimilation algorithms to either a. General approach partially or fully specify the background error covariances. This includes the EnKF (both the perturbed observation The most straightforward approach for using varia- version and all of the ensemble square root filter variants), tional data assimilation within an EnKF is to simply run EnVar, and some implementations of 4DVar that use en- independent data assimilation cycles for each ensemble semble covariances to specify the background error co- member. Each assimilates independently perturbed variances at the beginning of the data assimilation time observations, while the other sources of uncertainty are window (Buehner et al. 2010; Clayton et al. 2013). simulated with appropriate random perturbations. This Several NWP centers currently employ a variational is similar to the EDA approach (Isaksen et al. 2010) and data assimilation approach for their deterministic fore- the so-called system simulation approach (Houtekamer casts and a separate EnKF system that is used for both et al. 1996), except in those approaches, unlike with initializing the ensemble forecasts and for providing EnVar and the EnKF, the assimilation does not fully use ensemble covariances for the deterministic system [e.g., the ensemble covariances. However, if the analysis step for Environment and Climate Change Canada (ECCC), the each member was performed with a variational approach National Centers for Environmental Prediction (NCEP), that uses the ensemble of background states to define the and the Met Office]. The data assimilation procedures background error covariances, such as EnVar, this would used within current EnKF systems differ fundamentally be theoretically equivalent (other than the unavoidable from the variational approach by relying on either the differences related to the use of a different solution tech- serial assimilation of individual (or small batches) of nique for obtaining each member’s analysis increment) to observations (e.g., Whitaker and Hamill 2002; Tippett the perturbed-observation EnKF (e.g., Fairbairn et al. et al. 2003; Houtekamer et al. 2005) or an algorithm that 2014).Becauseafulldataassimilationsystemisusedfor independently updates spatial subdomains by simulta- each member with the same complexity as a typical de- neously assimilating all surrounding observations (e.g., terministic system, the computational cost is comparable Hunt et al. 2007). Several practical benefits may be re- to that of the deterministic system times the number of alized by using the same data assimilation approach for members, which is much higher than the cost of current both deterministic and ensemble prediction, including a EnKF approaches. Consequently, this would typically reduction in the effort required to develop and maintain limit the number of ensemble members to O(10), too little the computer code and an improved consistency of the to be used to fully specify the background error covari- impacts from major changes made to the two systems. To ances. For example the Canadian EnKF currently uses 256 that end, the goal of the present study is to evaluate a new members and, in contrast, Météo-France and ECMWF approach for performing the data assimilation step both use an EDA consisting of only 25 members of within a perturbed-observation EnKF by adapting the 4DVar. Because of this small ensemble size, a large Unauthenticated | Downloaded 10/07/21 08:39