Synoptic Control of Mesoscale Precipitating Systems in the Pacific Northwest

SEPTEMBER 2008 ROEBBERETAL. 3465

PAUL J. ROEBBER,KYLE L. SWANSON, AND JUGAL K. GHORAI Department of Mathematical Sciences, University of Wisconsin—Milwaukee, Milwaukee, Wisconsin

(Manuscript received 22 May 2007, in final form 14 January 2008)

ABSTRACT

This research examines whether an adequate representation of flow features on the synoptic scale allows for the skillful inference of mesoscale precipitating systems. The focus is on the specific problem of landfalling systems on the west coast of the United States for a variety of synoptic types that lead to significant rainfall. The methodology emphasizes rigorous hypothesis testing within a controlled hindcast setting to quantify the significance of the results. The role of lateral boundary conditions is explicitly accounted for by the study. The hypotheses that (a) uncertainty in the large-scale analysis and (b) upstream buffer size have no impact on the skill of precipitation simulations are each rejected at a high level of confidence, with the results showing that mean precipitation skill is higher where low analysis uncertainty exists and for small nested grids. This indicates that an important connection exists between the quality of the synoptic information and predictability at the mesoscale in this environment, despite the absence of such information in the initialization or boundary conditions. Further, the flow-through of synoptic information strongly constrains the evolution of the mesoscale such that a small upstream buffer produces superior results consistent with the higher quality of the information crossing the boundary. Some preliminary evidence that synoptic type has an influence on precipitation skill is also found. The implications of these results for data assimilation, forecasting, and climate modeling are discussed.

1. Introduction however, since precipitation is a sensible weather ele- ment of primary concern to consumers of weather in- An important class of unresolved questions in me- formation. Additionally, such information is of value to teorology concerns the extent to which a given scale of studies of regional climate. atmospheric motion influences the evolution of fea- Since forecast models are largely unconstrained by tures on other scales. A subset of these questions con- observations on the meso-␤ and ␥ scales, any skill these sists of the interaction between motions on the synoptic models exhibit on these scales must result from a downscale and evolving precipitating systems on the mesoscale cascade of information. Successful inference of scale. These precipitating systems, characterized by the mesoscale thus resolves around the extent to which spatial scales of O(2–100 km) and time scales of O(1–6 synoptic scales “control” the evolution of mesoscale h), are unresolved by existing global forecast models. features, that is, the extent to which these mesoscale Further, the precipitation is poorly sampled since it falls features are slaved to features originating on the syn- outside the routine observational network, except optic scale. This control may include flow interactions where ground-truthed radars are available [but see with in situ mesoscale forcing such as topography as Westrick et al. (1999) for the limitations of radar, owing well as the evolution of mesoscale systems that develop to sensor location, terrain blockage, and shallow pre- within the constraints of the synoptic environment. Evi- cipitation effects in the U.S. Pacific Northwest]. These dence for this emergence from the synoptic background systems are of vital importance to the forecast problem, has been found for mesoscale convective systems (Roebber et al. 2002; Fowle and Roebber 2003; Done et al. 2004; Kain et al. 2006), and for landfalling systems in Corresponding author address: Paul J. Roebber, Dept. of Math- ematical Sciences, University of Wisconsin—Milwaukee, Milwau- the western United States (e.g., Mass et al. 2002). In this kee, WI 53211. work, we address in detail the extent to which an ad- E-mail: [email protected] equate representation of flow features on the synoptic

DOI: 10.1175/2008MWR2264.1

MWR2264 3466 MONTHLY WEATHER REVIEW VOLUME 136 scale implies skillful simulation of precipitation for inevitably contaminate the interior of the domain, com- landfalling systems on the U.S. west coast. This ap- promising the veracity of the mesoscale detail. Finally, proach, where the large scales are specified as accurate- it may well be that the ability of the mesoscale model to ly as possible and the mesoscale precipitating structures reproduce mesoscale features from synoptic input will that emerge from the nested model physics are exam- depend on the type of system itself. ined, is most accurately viewed as a data assimilation In this research, we control for the aforementioned rather than a forecast study. factors in a statistically rigorous manner. The work is A broad variety of landfalling systems leads to sig- carried out in a controlled hindcast setting, emphasizing nificant rainfall along the U.S. west coast (e.g., Heggli independent verification with observed precipitation and Rauber 1988; Bond et al. 1997), including (i) cy- data to quantify the significance of the results. Such an clones linked to the large-scale subtropical flow (the emphasis is crucial, as evaluating mesoscale simulations so-called Pineapple Express), (ii) mature cyclones in relatively data-rich regions is difficult in its own right. propagating from the west in association with strong Separating initial condition errors, LBC errors due to short waves and an intense polar jet over the western the placement of boundaries of the nested grid model Pacific, and (iii) postfrontal convection often associated itself, and error by type of synoptic system represents with the passage of relatively weak wave disturbances. an ambitious set of objectives but for which the payoff In each of these cases, precipitation production is is a significant advance in our understanding of the strongly influenced by topographic interaction (Parsons transfer of information from the synoptic to the meso- and Hobbs 1983; Colle and Mass 1996; Steenburgh and scale, the dynamics of error growth in landfalling sys- Mass 1996; Doyle 1997; Colle et al. 1999; Mass et al. tems, and the ability of mesoscale models to “fill in” 2002). As such, it is reasonable to suppose that the details absent from larger-scale models. As such, the mesoscale signatures of these systems are largely gov- results are of direct interest to data assimilation, where erned by the fidelity of information provided at the filling in the gaps in the synoptic-scale analysis is of synoptic scale. obvious value. But likewise, these results have some Of course, there are a number of issues that must be relevance to short-term weather forecasting, where the addressed that in principle may invalidate this idea. The ability of nested models to glean extra information on primary tool of this research, the fifth-generation Penn- the mesoscale from a global synoptic-scale forecast is sylvania State University–National Center for Atmo- desirable, and to climate modeling, where computa- spheric Research Mesoscale Model (MM5), is a nested tional limits make it advantageous to focus resources grid model, and some studies have suggested that lat- toward running high-resolution simulations only in ar- eral boundary conditions (LBCs) may have a deleteri- eas where they are truly necessary. ous effect upon the ability of mesoscale models to simu- The outline of the paper is as follows. In section 2, we late small-scale structures (e.g., Warner et al. 1997). describe the mesoscale model that will be used to simu- Further, the growth of errors from the initial state could late the landfalling systems. In section 3, statistical tech- overwhelm the ability of the nested model to accurately niques that will isolate the relative contributions of simulate mesoscale structures. Initial condition error various error sources are discussed, while in section 4 growth has been a topic of extreme interest for global the results of this analysis are presented. Section 5 pro- models for some time (e.g., Lorenz 1982; Simmons et al. vides a concluding discussion of the scientific issues. 1995; and many others), and recent studies have examined its effect in mesoscale models (e.g., Du et al. 1997; 2. Mesoscale model Hamill and Colucci 1998; Stensrud et al. 2000; Grimit and Mass 2002; Eckel and Mass 2005; Zhang et al. 2006; The fifth-generation Pennsylvania State University– Grimit and Mass 2007; Hohenegger and Schar 2007). National Center for Atmospheric Research Mesoscale When initializing with information only on synoptic Model (MM5), a nonhydrostatic, multinested primitive scales, errors in specifying the mesoscale are inevitable. equation model (Dudhia 1993; Grell et al. 1994), is used If these errors project onto structures that grow rapidly, in this study. It has been employed extensively by the the mesoscale simulation will be contaminated and its research community for case studies and real-time ability to represent precipitating structures on the me- simulations of phenomena ranging from the cyclone soscale lost. In addition, the ability to simulate the me- scale (e.g., Dudhia 1993; Kuo et al. 1995; Roebber and soscale structure will depend upon the flow of accurate Reuter 2002) to the meso-␤ and ␥ scales [2–200 km; synoptic-scale information through the boundaries into e.g., Bresch et al. (1997); Powers (1997); Colle and Mass the mesoscale domain. Large errors in the specification (2000); Roebber et al. (2002)]. of those boundary conditions on the synoptic scale will The MM5 incorporates a number of well-tested phys-

Unauthenticated | Downloaded 10/01/21 12:56 PM UTC SEPTEMBER 2008 ROEBBERETAL. 3467

FIG.1.The“front half” hemispheric domain, with 120-km grid spacing, is indicated by the outer boundaries of the map. The large and small nests, with 15-km grid spacing, are indicated by the boldface squares. The hatched region (overland sites only) indicates the target zone in which precipitation verifications are conducted for landfalling cases. ics options. The high-resolution, multilayer version of ronments rather than deep convection, case studies of the Blackadar model (Zhang and Anthes 1982), which such systems have shown that the deepening rate and has been successfully employed in many synoptic- and the detailed pressure, wind, and temperature structures mesoscale studies (e.g., Zhang and Fritsch 1986; Sea- of the simulated cyclones can be significantly affected man et al. 1989; Shafran et al. 2000; Berg and Zhong by the details of the cumulus parameterization scheme 2005), is used in this study. Radiation is handled using (Kuo and Low-Nam 1990; Kuo et al. 1996). In this a cloud radiation scheme in which diurnally varying work, the Kain and Fritsch (1990) scheme is used in short- and longwave radiative fluxes interact with ex- both the outer (120 km) and inner (15 km) domains. plicit cloud and clear air, while the surface fluxes are The Kain–Fritsch parameterization uses a sophisticated used in the ground energy budget calculations. cloud-mixing scheme to determine entrainment and de- A nested domain structure is implemented (Fig. 1). trainment and removes all available buoyant energy The outer domain (120-km grid spacing) has been es- over a relaxation time scale. This scheme was specifi- tablished to provide “front half” hemispheric coverage cally designed for the model resolution of the inner (note that the actual domain is twice the size of that domain, where evaluations by regional modelers have depicted in Fig. 1). The domain is structured with at- demonstrated that it provides excellent results (e.g., tention to the dominant categories of winter storms af- Wang and Seaman 1997). Additionally, there is some fecting the Pacific coast region and to remove the pos- evidence that it can be successfully applied to oceanic sibility of LBC error that originates along the outer cyclones at larger scales as well (Kuo et al. 1996). boundary from degrading the simulation in the target For grid-resolvable precipitation, there are a number region (see hatched area in Fig. 1). A single inner do- of microphysical options available. An explicit moisture main is specified that has the grid spacing (15 km) nec- scheme that includes snow and cloud ice below 0°C essary to resolve important mesoscale features within (Dudhia 1989), with the snowfall speed expression cor- maritime systems and first-order terrain influences on rected from Rutledge and Hobbs (1983) to that of Cox the distribution of precipitation in the target region (1988), is used. This choice is based upon Colle et al. (e.g., Colle et al. 1999; Mass et al. 2002). Further dis- (1999) and Colle and Mass (2000), who show compa- cussion of LBC errors and the transition between rable results to those using more sophisticated schemes model domains is provided in section 3a. (e.g., including supercooled water, graupel, and riming For these domain configurations, cumulus param- processes). While much further work is needed on the eterizations must be used since convective precipitation microphysics in these models, such an effort is beyond is not grid resolvable. Although the precipitation in the the scope of this study [see Stoelinga et al. (2003) and vicinity of cold-season, maritime cyclones is primarily Colle et al. (2005) for further exploration of these is- the result of synoptic-scale lift in stably stratified envi- sues].

Unauthenticated | Downloaded 10/01/21 12:56 PM UTC 3468 MONTHLY WEATHER REVIEW VOLUME 136

3. Experimental format In addition to the issues noted above, several other factors noted by Warner et al. (1997) suggest that LBCs a. Lateral boundary conditions need to be studied: strong cross-boundary flow will al- The downscale cascade of information from synoptic low noise to propagate rapidly into the domain interior, to mesoscales has long been implicit in the work of local forcing by flow over terrain may make the simu- forecasters, who use synoptic-scale models and obser- lation more sensitive to errors originating at the bound- vations to produce operational forecasts of sensible ary, and passage of transient high-amplitude meteoro- weather elements (e.g., Fowle and Roebber 2003). To logical phenomena through the boundaries (e.g., trough be consistent with this flow of information from large to and jet structures) may “excite” boundary errors. small scales, model grids are used in a one-way mode, Mitigating against these concerns is the fact that the such that information does not flow upscale to the present study design is based on a different set of as- mother domain. Model LBCs (exterior to the outer do- sumptions than were examined by Warner et al. (1997). main) are supplied by the 6-hourly National Centers for At least a portion of the LBC errors noted in that study Environmental Prediction–National Center for Atmo- stems from problems that arise from operating in fore- spheric Research (NCEP–NCAR) reanalysis fields. A cast mode rather than the assimilation mode used here. consistent simulation of the outer domain is generated Specifically, if the model providing boundary condi- by using four-dimensional data analysis (4DDA), nudg- tions to the nest differs sufficiently from the actual time ing back to the reanalysis with a time scale of 6 h. evolution of the atmosphere, it is apparent that the However, the inner domain in all cases evolves freely, nested model ultimately will fail as LBCs propagate interacting with the outer domain only where it receives into the interior. In contrast, the experimental format used here requires the outer solution providing bound- its boundary conditions. Topography and land use are ary conditions to the mesoscale grid to remain close to back-interpolated from high-resolution datasets to the the NCEP–NCAR reanalysis state as defined by model grids. 4DDA. As such, “good” information, to the extent that One might ask why a relatively large (in this case, the reanalysis adequately represents the actual atmo- 8:1) nest ratio between the grid spacings of the outer sphere on synoptic scales in data-sparse regions, is al- domain and the inner nest is used. For example, the ways entering the domain from the boundaries. As will standard nest ratio for MM5 is 3:1. This is largely a be shown in section 4, the skill in simulating the ob- consequence of the research focus, which is to evaluate served precipitation across a range of cases provides an the synoptic forcing of primarily orographic precipita- a posteriori justification for the nesting strategy em- tion in the inner nest. By slaving the outer (120 km) ployed. domain to the NCEP–NCAR reanalysis, one is assured Further support of the view that errors are not ema- that only synoptic features are preserved and that any nating from the boundary and contaminating the nest mesoscale structure must emerge on the inner nest and/ simulations is provided by Fig. 2. For this event, which or in the interactions of the flow with topography in is representative of the study results in showing im- that nest. Since the primary precipitation forcing is oro- proved precipitation skill in the small nest relative to graphic interaction, a 15-km grid spacing on the inner the large nest (see section 4), a short-wave ridge pas- domain is chosen, to ensure that this process is ad- sage is evident, but there is no evidence of noise devel- equately resolved (see Colle et al. 1999; Mass et al. 2002). oping along the upstream boundary and progressing Owing to the one-way rather than interactive grid into the nest interior, despite substantial cross- and mismatch of scales at the nest boundary, however, boundary flow in this case (500- and 250-hPa zonal it is essential to control for the potentially deleterious winds average approximately 33 and 55 m sϪ1, respec- effects of LBC error (e.g., Warner et al. 1997). To do so, tively, along the upstream boundary of the large nest two sizes of the inner domain are used in the study during the 48 h of the simulation). design. The large inner domain (large nested box in Fig. The downstream nest boundary (see Fig. 1) may also 1; hereafter large nest) is extended sufficiently far up- have an effect on the simulation performance, given stream to explicitly remove the influence of LBC error nest interpolation issues and the need to incorporate on the target region for the 48-h periods of interest to the full extent of the topographic barrier and plateau to the study, following the general guidelines of Warner et obtain the full orographic response (Braun et al. al. (1997). A second set of simulations is conducted with 1999a,b). To evaluate this issue, all observation sites an inner domain that is 25% of the area of the large were split into locations upstream and downstream of nest (small nest in Fig. 1) to directly assess the impact of 122°W and basic simulation statistics for precipitation the LBC error on the precipitation simulations. were computed across all the studied events. These

Unauthenticated | Downloaded 10/01/21 12:56 PM UTC SEPTEMBER 2008 ROEBBERETAL. 3469

certainty in the large-scale analysis, and (ii) mesoscale predictability is not strongly connected to the synoptic scale under the studied conditions. Should no rejection occur, it would be necessary to consider twin experiments with two different analyses providing the boundary–initial conditions for the nest for each case under consideration. If upscale error growth within the nest is the dominant effect, then the nest will act to strongly amplify the differences that exist in the two initial analyses. Owing to the availability of distinct global analyses from Na- tional Centers for Environmental Prediction (NCEP) and the European Centre for Medium-Range Weather Forecasts (ECMWF), such twin experiments are readily performed. In this situation, synoptic measures of the difference between the nest solutions for the two reanalyses [e.g. root-mean-square (RMS) error in 500- hPa height averaged over the nest] will amplify sharply with time, marking a strong upscale cascade of infor-

FIG. 2. Time–longitude evolution of the 500-hPa height mation from the meso- to synoptic scales. In contrast, if anomaly (m) on the small nest (relative to the average value of the the mesoscale is effectively decoupled from the synop- 500-hPa height at each grid point for the 48-h integration with an tic scale, little or no upscale cascade of information will initial time of 0000 UTC 3 Mar 2000). Col is the model x-grid occur, and the synoptic measures of the difference be- location, approximately west to east. tween the respective nest solutions will experience little or no growth over the duration of the hindcast experi- data show only small differences in simulation skill for ments. The alternative and expected result is that H1 precipitation (0.28, west; 0.33, east), as measured by the can be rejected at a high level of confidence. The only Kuiper skill score (see section 3b), while the bias was possible interpretation of this result is that an important lower to the west (0.98, west; 1.20, east). This bias pat- connection does exist between the quality of the syn- tern is consistent with the still relatively coarse resolu- optic information and the predictability at the meso- tion of the topography in upslope–downslope condi- scale for this set of landfalling system types. tions and the conclusion is that the noted performance Because of the potential importance of LBCs as a differences are connected to orographic rather than do- source of variation in the precipitation skill score, the main nesting considerations. experimental design explicitly accounts for this factor, resulting in the secondary null hypothesis to be tested b. Hypothesis testing (H2). By controlling the size of the nest, it is possible to assess the impact of LBC error vis-à-vis the beneficial We test the following (main effect) null hypotheses: effect of good synoptic-scale information flowing through the mesoscale grid boundaries. After account- H1—uncertainty in the large-scale analysis has no ing for the effects of analysis uncertainty, failure to impact on the skill of the precipitation simulations reject H2 can occur in two ways: (i) Upscale error in the target zone of the Pacific Northwest, and growth is sufficiently large as to swamp the predictabil- H2—upstream buffer size has no impact on the skill ity signal, regardless of the effect of information (good of the precipitation simulations in the target zone or bad) flowing in from the boundary, and (ii) meso- of the Pacific Northwest. scale predictability is not strongly connected to the syn- In H1, analysis uncertainty may be reflected in either optic-scale information. As above, separation as to the the initial conditions of the large-scale analysis interpo- dominant effect can be accomplished by consideration lated to the mesoscale grid or the flow of the synoptic- of twin analysis experiments. scale information entering that grid through the lateral The alternative result is that H2 can be rejected at a boundaries during the course of the simulation. Failure high level of confidence. However, this rejection may to reject H1 might occur for either of the following occur in either of the following ways: (i) LBC error reasons: (i) Upscale error growth is sufficiently large as exerts a significant influence on mesoscale predictabil- to swamp the predictability signal, regardless of the un- ity, as in Warner et al. (1997), or (ii) the flow-through of

Unauthenticated | Downloaded 10/01/21 12:56 PM UTC 3470 MONTHLY WEATHER REVIEW VOLUME 136 synoptic information strongly constrains the evolution Fig. 1) and large-scale analysis uncertainty. Consider- of the mesoscale, such that the small nest produces su- ation of a time-integrated measure of analysis un- perior results consistent with the higher quality of the certainty is vital, as the accuracy of the flow of infor- information entering the boundary. These differences mation through the nest boundary will strongly influ- will be obvious from an analysis of the data. ence the inferred mesoscale precipitating structures. To provide rigorous tests of these hypotheses, careful Such a measure is not routinely calculated for the experimental design must be employed. In that regard, NCEP–NCAR reanalysis, but a reasonable approxima- the measure of interest (dependent variable) is the 24-h tion of the uncertainty may be obtained by measuring event total precipitation skill score, constructed using the difference between two independent analyses, in all available National Weather Service (NWS) Coop- this case, the NCEP–NCAR and ECMWF reanalysis erative Observer Program (COOP), Snowpack Telem- fields. These two reanalyses are constructed using quite etry (SNOTEL), and Surface Airway Observation different underlying models and analysis techniques (SAO) sites in the Pacific coast states (Washington, [optimal interpolation versus three-dimensional varia- Oregon, and California; hatched region in Fig. 1). The tional data assimilation; Kalnay et al. (1996); Gibson et simulations are conducted for 48-h periods, and the al. (1997)]. Hence, comparison of the two provides a precipitation data are analyzed only for the last 24 h of reasonable leading-order estimate of the uncertainty in the simulation to allow for model spinup. The precipi- the fields used to provide the initial and lateral bound- tation data are not stratified into periods shorter than ary conditions to the model experiments. 24 h owing to the fact that many of the reports are from As a simple measure of the uncertainty, we use the COOP sites. Temporal resolution is thereby sacrificed RMS difference in 500-hPa height between the two re- in the interest of adequate spatial sampling for each analyses computed over a 12-yr period (1990–2001) for event. the region encompassed by the large nest. Errors in the There are many methods of measuring forecast or 500-hPa field in principle will capture uncertainties in simulation skill. It has been shown that traditional veri- the large-scale, mid- to upper-tropospheric flow that fication measures, when applied to high-resolution nu- influences the steering and development of these land- merical weather predictions, can lead to improper con- falling systems, plus uncertainties in temperature that clusions concerning performance (e.g., Baldwin et al. might influence the rate and type of precipitation for- 2001). The high degree of spatial variability in fields mation. Other measures of analysis uncertainty are pos- such as precipitation will likely lead to large errors at sible, of course, but the experimental results reported particular points relative to a forecast that has a more homogenous distribution (such as with a coarse- below (section 4) suggest that the 500-hPa error defined resolution numerical model). Despite this, the former in this manner captures the essence of the problem for forecast may provide a more realistic conceptual basis this experimental setup. for the observed evolution owing to its ability to cap- This contention is further supported by examination ture the scales and amplitudes of embedded structures. of the spatial distribution of the analysis differences The 15-km grid spacing of the nest used in this study, (Fig. 3), which range from a minimum of 8 m over the however, remains at a scale at which traditional mea- data-rich continents to a maximum of 15 m over the sures can still provide useful measures of the informa- data-sparse Pacific. Swanson et al. (2000) point out that tion content of the simulations. Colle et al. (1999, 2000) since analysis procedures weight short-term forecast showed this to be the case for topographically forced data in the absence of observations, past error growth is precipitation in the Pacific Northwest; consequently, an important determinant of analysis error. Of particu- other methods will not be pursued here. In this study, lar importance to this study is the large variation in the measure of choice will be the Kuiper skill score analysis difference over the Pacific, with nearly a factor (also known as the true skill statistic and hereafter of 2 range between the upper and lower quintiles. This KSS), calculated for precipitation greater than or equal results from the fact that in dynamically benign flows, to 25 mm. This threshold is selected based on the con- for example, error growth will be smaller and the ob- sideration that we are most interested in high-impact servational information from the well-sampled regions events, which remain relatively frequent for landfalling upstream will be transmitted more efficiently to the systems in the Pacific Northwest. This skill score mea- Pacific interior. Further examination of these issues is sures the degree that the forecast is able to separate beyond the scope of this study but these issues are ad- high-impact sites from lesser ones. dressed in detail elsewhere (Swanson and Roebber Analysis factors (independent variables) are inner 2008, hereafter SR). nest size (two levels: large and small, as depicted in Two analysis uncertainty levels are considered in this

Unauthenticated | Downloaded 10/01/21 12:56 PM UTC SEPTEMBER 2008 ROEBBERETAL. 3471

we examine whether these effects are related to synoptic type. By requiring strong evidence for rejection, the type II error will be higher. Because of the analysis design, however, this error is defined (e.g., Muller et al. 1992).

4. Results Twenty-two landfalling events were considered in the analysis (Table 1), providing a mix of synoptic types and analysis errors. As shown, the 24-h precipitation skill scores (verifying at all available COOP, SNOTEL, and SAO sites in the hatched region in Fig. 1) are quite variable, with KSS ranging from near 0 up to 0.65 (where the maximum possible skill is 1.00 for a perfect forecast at all measurement sites for a landfalling event). Mean skill (Table 2) is notably higher for low FIG. 3. The RMS difference (m) between the ECMWF and analysis uncertainty (0.416 versus 0.236) and for the NCEP–NCAR reanalysis 500-hPa geopotential height fields between 30° and 60°N for January–March for the period 1990–2001. small nest (0.309 versus 0.245). The F statistics (Table Thick, thin, and dashed lines are the average, upper quintile, and 3) show that we can reject H1 with high confidence lower quintile analysis differences, respectively. (p ϭ 0.036) and conclude that an important connection exists between the quality of the synoptic information and predictability at the mesoscale for this set of land- study: (1) low and (2) moderate to high, the former falling system types. Likewise, we can reject H2 with defined as the 25th-percentile RMS difference over the high confidence (p ϭ 0.043) and conclude that the flow- first 36 h of the simulation period. Based on experience through of the synoptic information strongly constrains in this region, sampling is conducted to ensure ample the evolution of the mesoscale. The data in Table 2 representation of the following precipitation-bearing show that the small nest produces superior results, con- synoptic systems: Pineapple Express, frontal wave, de- sistent with the higher quality of the information enter- veloped cyclone, and cutoff cyclone (Fig. 4). To make ing the upstream boundary. the experiment tractable, additional factors of potential An important point relates to sample size and statis- importance, including the atmospheric boundary layer tical significance. As concluded above, H1 and H2 are scheme, cloud microphysics, and convective parameter- rejected with a low probability of incorrectly doing so ization, are not examined in these experiments. The (approximately 4%). Since we do reject the null hy- fixed choices for these in all simulations were guided by potheses, we are at no risk of a type II error (incorrect findings in the refereed literature, as discussed in sec- acceptance of H1 or H2). Hence, the only effect of tion 2. increasing the sample size would be to further reduce The outputs from the model simulations are used to the risk of incorrect rejection, which is already small, provide a test of the null hypothesis at a specified level while having no effect on the type II error, which is of significance (i.e., the probability of incorrectly reject- already nil. Hence, further efforts to expand the dataset ing the null hypothesis, ␣, the type I error). The type II to address the fundamental hypotheses would appear error (␤, the probability of incorrectly accepting the unwarranted. null hypothesis) is specified through the power function There is some evidence that synoptic type has an (1 Ϫ ␤). For our null hypotheses, we require a low ␣ influence on precipitation skill (Table 4), and that there (0.05) to limit the likelihood of incorrect rejection; that is a dependence on the nest size (p ϭ 0.011; Table 3). is, we seek to reject the null hypothesis only if the evi- Owing to the limits of the sample size, however, the dence for rejection is strong. Hence, for strong evi- skill differences noted in Table 4 cannot be safely as- dence, the conclusion is that analysis uncertainty does cribed to other than chance variation, although the have a statistically significant impact on precipitation power calculation indicates that we may well be incor- forecast skill in the target zone. At the same time, we rectly accepting the null hypothesis that there is no dif- are able to test the impact of LBCs on precipitation ference in skill between event types. Many additional forecast skill in the target zone (e.g., strong evidence of events would need to be analyzed to resolve this ques- that main effect would allow the conclusion that nest tion with high confidence; an effort that is beyond the size does have a statistically significant effect). Finally, scope of this study.

Unauthenticated | Downloaded 10/01/21 12:56 PM UTC 3472 MONTHLY WEATHER REVIEW VOLUME 136

FIG. 4. Representative examples of (a) the Pineapple Express (5 Feb 1996), (b) a frontal wave (4 Mar 2000), (c) a developed cyclone (30 Oct 1999), and (d) a cutoff cyclone (9 Oct 1997). Shown are (top) 500-hPa and (bottom) sea level pressure analyses, obtained from the Climate Data Center (CDC) Reanalysis and the National Oceanographic and Atmospheric Agency (NOAA) Daily Weather Map series, respectively.

Unauthenticated | Downloaded 10/01/21 12:56 PM UTC SEPTEMBER 2008 ROEBBERETAL. 3473

TABLE 1. Simulated events. Listed are the event date (the start TABLE 3. Analysis of variance (ANOVA) tables for analysis time of the 48-h simulation), the event type (the subjective clas- uncertainty and nest size, and event type and nest size. Statistics sification as depicted in Fig. 4), the degree of analysis uncertainty, shown are F, p value, and power. stratified as described in the text, and the KSS computed for the two simulation domains in the verification region (see text for Effect Fpvalue Power details). Analysis uncertainty* 5.07 0.036 0.573 Nest size* 4.66 0.043 0.537 Skill score (KSS) Effect Fpvalue Power Analysis Large Small Event date Event type uncertainty inner inner Event type 1.51 0.232 0.216 Nest size* 7.75 0.011 0.754 4 Feb 1996 Pineapple High 0.23 0.35 4 Dec 1996 Cutoff High 0.19 0.29 * The effect is significant at high confidence. 8 Oct 1997 Cutoff High 0.27 0.34 22 Nov 1997 Pineapple High 0.14 0.13 22 Mar 1998 Pineapple High Ϫ0.07 0.06 28 Dec 1998 Pineapple Low 0.52 0.59 At 30 h, the relative error in the large nest arises from 27 Oct 1999 Cutoff High 0.06 0.04 a misplacement of the primary axis of the southwesterly 29 Oct 1999 Developed High 0.23 0.45 flow and an associated short wave, leading to larger 9 Nov 1999 Cutoff Low 0.12 0.27 errors in the resulting precipitation. In the March 2000 10 Nov 1999 Developed High 0.34 0.42 case, the relative error in the large nest peaks at 24 h 2 Dec 1999 Frontal High 0.47 0.59 4 Dec 1999 Frontal High 0.05 0.27 into the simulation. This error is tied to a westward 9 Dec 1999 Developed High 0.10 0.20 phase error of the 500-hPa trough, as well as a north- 14 Dec 1999 Frontal Low 0.33 0.32 ward shift of an upper-level cutoff along the West 30 Jan 2000 Developed High 0.27 0.32 Coast, leading to incorrect specification of the synoptic- 21 Feb 2000 Cutoff High 0.41 0.34 scale forcing of the precipitation. 3 Mar 2000 Frontal High 0.24 0.45 17 Mar 2000 Frontal Low 0.37 0.38 8 Oct 2000 Developed Low 0.65 0.61 5. Discussion 19 Oct 2000 Frontal High Ϫ0.09 Ϫ0.01 27 Oct 2000 Cutoff High 0.31 0.31 This research examines the extent to which an ad- 28 Nov 2000 Cutoff High 0.25 0.08 equate representation of the flow features on the synoptic scale allows for the skillful inference of mesoscale precipitating systems, emphasizing rigorous hypothesis We note that the results with respect to nest size testing within a controlled hindcast setting to quantify would not be anticipated from Warner et al. (1997). We the significance of the results. The focus of the research attribute this finding to the fact that the synoptic field in is on the specific problem of landfalling precipitating the small nest cannot deviate as far from the reality systems on the U.S. west coast for a variety of system defined by the outer domain solution, which tracks the types that lead to significant rainfall in this region. The NCEP–NCAR reanalysis. A time history of 500-hPa primary tools of this work are the MM5, the NCEP– RMS differences for two cases relative to the large- NCAR and ECMWF reanalysis datasets, and indepen- scale analysis of the outer domain shows this effect (Fig. dent precipitation measurements from all available 5). It is apparent that substantial error growth occurs in COOP, SNOTEL, and SAO sites in the Pacific coast the interior of the large nest, a result of its greater iso- states (Washington, Oregon, and California). The role lation from the good information flowing through the of LBCs is explicitly accounted for by the study. boundary. In the February 1996 case, the relative error We find that while precipitation skill scores are quite in the large nest 12 h into the simulation is associated variable from case to case, mean skill is notably higher with the phasing of a well-developed 500-hPa trough. where low analysis uncertainty exists and for a smaller

TABLE 2. Mean KSS stratified by analysis uncertainty and nest TABLE 4. Mean KSS stratified by event type and nest size. size. Event type (No. of events) Large nest Small nest Total Skill score (KSS) Analysis Cutoff (7) 0.230 0.239 0.235 uncertainty Large nest Small nest Total Pineapple (4) 0.205 0.282 0.243 Low 0.398 0.434 0.416 Developed (5) 0.318 0.400 0.359 High 0.200 0.272 0.236 Frontal (6) 0.228 0.333 0.280 Total 0.245 0.309 Total 0.245 0.309

Unauthenticated | Downloaded 10/01/21 12:56 PM UTC 3474 MONTHLY WEATHER REVIEW VOLUME 136

to Canada, the intermountain region of the west, the Great Lakes, and (perhaps) Mexico. Second, this study makes it clear that a nested grid model can be used to glean information on the mesoscale from a larger-scale forecast given topographic forcing provided that the forecast information is of good quality. In support of this finding, Colle et al. (2000) show that case screening based on the quality of the forecast 850-hPa wind at coastal sounding sites in the Pacific Northwest improves precipitation forecasts regardless of model resolution. McMurdie and Mass (2004) document the high frequency of operational numerical forecast failures in this region. Notwithstanding the positive effects of improved microphysics (e.g., Woods et al. 2007), it seems apparent that substantial increases in forecast quality in the Pacific Northwest will not be obtainable until the analysis error in the upstream Pacific data void is reduced (see Fig. 3). A forthcoming paper by this study’s authors (SR) suggests ϳ FIG. 5. Time series of the 500-hPa RMS error difference (large that a reduction of analysis error over the Pacific ( 15 minus small nest) relative to the front-half hemispheric domain, m) to levels comparable to those now over Hawaii (ϳ8 slaved to the NCEP–NCAR reanalysis for 4 Feb 1996 (solid) and m), whether through additional observations such as 3 Mar 2000 (dashed) for the 48 h of the simulation. might be obtained using advanced, aerial techniques (Bishop and Toth 1999) or through improved data as- nested grid. Indeed, from these data, we conclude that similation (Whitaker et al. 2008), would extend synop- an important connection exists between the quality of tic-scale forecast skill over the continental United the synoptic information and the predictability at the States by approximately 1 day. Thus, substantial im- mesoscale for a range of landfalling system types. That provements in precipitation forecasts in this region, is, a mesoscale model is able to fill in details absent while not achievable given the current deployment of from the initialization and/or larger-scale models for resources, are a tantalizing future possibility. It should the conditions studied. It should be noted that the low be noted, however, that such progress may not be uni- and high analysis uncertainty samples are similar in na- formly possible. Hohenegger and Schar (2007) show ture: each features a range of landfalling synoptic types that for convective precipitation, the rapid propagation and both exhibit significant regional rainfall, with mean of initial uncertainties via sound and gravity waves fol- precipitation across all measurement sites over all lowed by the amplification of perturbations in regions events of 54.6 and 48.9 mm, respectively. of convective instability are significant obstacles to Likewise, we find that the flow-through of synoptic progress on the quantitative precipitation forecast information strongly constrains the evolution of the problem. mesoscale and that a small nest produces superior re- Third, these results provide some support for down- sults consistent with the higher quality of the informa- scaling climate model simulations, where computa- tion entering the upstream boundary. We find some tional limits make it strongly desirable to focus the evidence that synoptic type has an influence on the computational resources to run high-resolution simula- precipitation skill and that there is a dependence on tions upon areas where they are truly necessary. Leung nest size, but sample size does not permit a statistically and Ghan (1998) provide an example of the type of robust result for this aspect. subgrid-scale parameterizations that are now utilized. There are several important implications of these re- To the extent that the climate model can produce reli- sults. First, these results suggest the value of mesoscale able synoptic conditions, it should be possible to pro- models in the data assimilation process, where filling in vide improved precipitation information using appro- the gaps in the synoptic-scale analysis is of obvious in- priate mesoscale models. This application takes on terest. Such an approach to data assimilation could be greater importance as scientists and policy makers at- quite effective in areas in which the synoptic pattern is tempt to assess the impacts of future climate change on well resolved but mesoscale information is lacking. In available water resources at the regional scale. North America, such areas include but are not limited Finally, it will be important to study other types of

Unauthenticated | Downloaded 10/01/21 12:56 PM UTC SEPTEMBER 2008 ROEBBERETAL. 3475 phenomena, most importantly convective systems, to forecasting of quantitative precipitation. Mon. Wea. Rev., determine if these results extend, as some studies al- 125, 2427–2459. ready suggest, to these environments. Dudhia, J., 1989: Numerical study of convection observed during the Winter Monsoon Experiment using a mesoscale two- dimensional model. J. Atmos. Sci., 46, 3077–3107. Acknowledgments. This work was supported under ——, 1993: A nonhydrostatic version of the Penn State–NCAR NSF Grant ATM-0106584. The authors thank the fol- mesoscale model: Validation tests and simulation of an At- lowing individuals for their contributions to this work: lantic cyclone and cold front. Mon. Wea. Rev., 121, 1493– Sara (Bruening) Van Klooster, Cory Behnke, and Me- 1513. Eckel, F. A., and C. F. Mass, 2005: Aspects of effective mesoscale, lissa (Butt) Schumann. short-range ensemble forecasting. Wea. Forecasting, 20, 328– 350. REFERENCES Fowle, M. A., and P. J. Roebber, 2003: Short-range (0–48 h) numerical prediction of convective occurrence, mode, and loca- Baldwin, M. E., S. Lakshmivarahan, and J. S. Kain, 2001: Verifi- tion. Wea. Forecasting, 18, 782–794. cation of mesoscale features in NWP models. Preprints, Ninth Gibson, J. K., P. Kallberg, S. Uppala, A. Hernandez, A. Normura, Conf. on Mesoscale Processes, Fort Lauderdale, FL, Amer. and E. Serrano, 1997: ERA description. ECMWF Re- Meteor. Soc., 255–258. Analysis Project Report Series, No. 1, 72 pp. [Available from Berg, L. K., and S. Zhong, 2005: Sensitivity of MM5-simulated ECMWF, Reading RG2 AX, United Kingdom.] boundary layer characteristics to turbulence parameteriza- Grell, G. A., J. Dudhia, and D. R. Stauffer, 1994: A description of tions. J. Appl. Meteor., 44, 1467–1483. the fifth-generation Penn State/NCAR Mesoscale Model Bishop, C. H., and Z. Toth, 1999: Ensemble transformation and (MM5). NCAR/TN-398ϩSTR, 1381 pp. adaptive observations. J. Atmos. Sci., 56, 1748–1765. Grimit, E. P., and C. F. Mass, 2002: Initial results of a mesoscale Bond, N. A., and Coauthors, 1997: The Coastal Observation and short-range ensemble forecasting system over the Pacific Simulation with Topography (COAST) experiment. Bull. Northwest. Wea. Forecasting, 17, 192–205. Amer. Meteor. Soc., 78, 1941–1955. ——, and ——, 2007: Measuring the ensemble spread–error rela- Braun, S. A., R. Rotunno, and J. B. Klemp, 1999a: Effects of tionship with a probabilistic approach: Stochastic ensemble coastal orography on landfalling cold fronts. Part I: Dry, in- results. Mon. Wea. Rev., 135, 203–221. viscid dynamics. J. Atmos. Sci., 56, 517–533. Hamill, T. M., and S. J. Colucci, 1998: Evaluation of Eta–RSM ——, ——, and ——, 1999b: Effects of coastal orography on land- ensemble probabilistic precipitation forecasts. Mon. Wea. falling cold fronts. Part II: Effects of surface friction. J. At- Rev., 126, 711–724. mos. Sci., 56, 3366–3384. Heggli, M. F., and R. M. Rauber, 1988: The characteristics and Bresch, J. F., R. J. Reed, and M. D. Albright, 1997: A polar-low evolution of supercooled water in wintertime storms over the development over the Bering Sea: Analysis, numerical simu- Sierra Nevada: A summary of microwave radiometric mea- lation, and sensitivity experiments. Mon. Wea. Rev., 125, surements taken during the Sierra Cooperative Pilot Project. 3109–3130. J. Appl. Meteor., 27, 989–1015. Colle, B. A., and C. F. Mass, 1996: An observational and modeling Hohenegger, C., and C. Schar, 2007: Predictability and error study of the interaction of low-level southwesterly flow with growth dynamics in cloud-resolving models. J. Atmos. Sci., the Olympic Mountains during COAST IOP 4. Mon. Wea. 64, 4467–4478. Rev., 124, 2152–2175. Kain, J. S., and J. M. Fritsch, 1990: A one-dimensional entraining/ ——, and ——, 2000: The 5–9 February 1996 flooding event over detraining plume model and its application in convective pa- the Pacific Northwest: Sensitivity studies and evaluation of rameterization. J. Atmos. Sci., 47, 2784–2802. the MM5 precipitation forecasts. Mon. Wea. Rev., 128, 593– ——, S. J. Weiss, J. J. Levit, M. E. Baldwin, and D. R. Bright, 617. 2006: Examination of convection-allowing configurations of ——, K. J. Westrick, and C. F. Mass, 1999: Evaluation of MM5 the WRF model for the prediction of severe convective and Eta-10 precipitation forecasts over the Pacific Northwest weather: The SPC/NSSL Spring Program 2004. Wea. Fore- during the cool season. Wea. Forecasting, 14, 137–154. casting, 21, 167–181. ——, C. F. Mass, and K. J. Westrick, 2000: MM5 precipitation Kalnay, E., and Coauthors, 1996: The NCEP/NCAR 40-Year Re- verification over the Pacific Northwest during the 1997–99 analysis Project. Bull. Amer. Meteor. Soc., 77, 437–471. cool seasons. Wea. Forecasting, 15, 730–744. Kuo, Y.-H., and S. Low-Nam, 1990: Prediction of nine explosive ——, M. F. Garvert, J. B. Wolfe, C. F. Mass, and C. P. Woods, cyclones over the western North Atlantic Ocean with a re- 2005: The 13–14 December 2001 IMPROVE-2 event. Part gional model. Mon. Wea. Rev., 118, 3–25. III: Simulated microphysical budgets and sensitivity studies. ——, J. R. Gyakum, and Z. Guo, 1995: A case of rapid continental J. Atmos. Sci., 62, 3535–3558. mesoscale cyclogenesis. Part I: Model sensitivity experi- Cox, G. P., 1988: Modeling precipitation in frontal rainbands. ments. Mon. Wea. Rev., 123, 970–997. Quart. J. Roy. Meteor. Soc., 114, 115–127. ——, R. J. Reed, and Y. Liu, 1996: The ERICA IOP 5 storm. Part Done, J., C. Davis, and M. Weisman, 2004: The next generation of III: Mesoscale cyclogenesis and precipitation parameteriza- NWP: Explicit forecasts of convection using Weather Re- tion. Mon. Wea. Rev., 124, 1409–1434. search and Forecast (WRF) model. Atmos. Sci. Lett., 5, 110– Leung, L. R., and S. J. Ghan, 1998: Parameterizing subgrid oro- 117, doi:10.1002/asl.72. graphic precipitation and surface cover in climate models. Doyle, J. D., 1997: The influence of mesoscale orography on a Mon. Wea. Rev., 126, 3271–3291. coastal jet and rainband. Mon. Wea. Rev., 125, 1465–1488. Lorenz, E. M., 1982: Atmospheric predictability experiments with Du, J., S. L. Mullen, and F. Sanders, 1997: Short-range ensemble a large numerical model. Tellus, 34A, 505–513.

Unauthenticated | Downloaded 10/01/21 12:56 PM UTC 3476 MONTHLY WEATHER REVIEW VOLUME 136

Mass, C. F., D. Ovens, K. Westrick, and B. A. Colle, 2002: Does condition and model physics perturbations in short-range en- increasing horizontal resolution produce more skillful fore- semble simulations of mesoscale convective systems. Mon. casts? Bull. Amer. Meteor. Soc., 83, 407–430. Wea. Rev., 128, 2077–2107. McMurdie, L., and C. Mass, 2004: Major numerical forecast fail- Stoelinga, M. T., and Coauthors, 2003: Improvement of micro- ures over the northeast Pacific. Wea. Forecasting, 19, 338– physical parameterization through observational verification 356. experiment. Bull. Amer. Meteor. Soc., 84, 1807–1826. Muller, K. E., L. M. LaVange, S. L. Ramey, and C. T. Ramey, Swanson, K. L., and P. J. Roebber, 2008: The impact of analysis 1992: Power calculations for general linear multivariate mod- error on medium-range weather forecasts. Mon. Wea. Rev., els including repeated measures applications. J. Amer. Stat. 136, 3425–3431. Assoc., 87, 1209–1226. ——, T. N. Palmer, and R. Vautard, 2000: Observational error Parsons, D. B., and P. V. Hobbs, 1983: The mesoscale and micro- structures and the value of advanced assimilation techniques. scale structure and organization of clouds and precipitation in J. Atmos. Sci., 57, 1327–1340. midlatitude cyclones. Part IX: Some effects of orography on Wang, W., and N. L. Seaman, 1997: A comparison study of con- rainbands. J. Atmos. Sci., 40, 1930–1949. vective parameterization schemes in a mesoscale model. Powers, J. G., 1997: Numerical model simulations of a mesoscale Mon. Wea. Rev., 125, 252–278. gravity wave event: Sensitivity tests and spectral analyses. Warner, T. T., R. A. Petersen, and R. E. Treadon, 1997: A tutorial Mon. Wea. Rev., 1838–1869. 125, on lateral boundary conditions as a basic and potentially se- Roebber, P. J., and G. W. Reuter, 2002: The sensitivity of precipi- rious limitation to regional numerical weather prediction. tation to circulation details. Part II: Mesoscale modeling. Bull. Amer. Meteor. Soc., 78, 2599–2617. Mon. Wea. Rev., 130, 3–23. Westrick, K. J., C. F. Mass, and B. A. Colle, 1999: The limitations ——, D. M. Schultz, and R. Romero, 2002: Synoptic regulation of of the WSR-88D radar network for quantitative precipitation the 3 May 1999 tornado outbreak. Wea. Forecasting, 17, 399– measurement over the coastal western United States. Bull. 429. Amer. Meteor. Soc., 80, 2289–2298. Rutledge, S. A., and P. V. Hobbs, 1983: The mesoscale and mi- croscale structure and organization of clouds and precipita- Whitaker, J. S., T. M. Hamill, X. Wei, Y. Song, and Z. Toth, 2008: tion in midlatitude cyclones. Part VIII: A model for the Ensemble data assimilation with the NCEP Global Forecast Mon. Wea. Rev., “seeder-feeder” process in warm-frontal rainbands. J. Atmos. System. 136, 463–482. Sci., 40, 1185–1206. Woods, C. P., M. T. Stoelinga, and J. D. Locatelli, 2007: The Seaman, N. L., F. L. Ludwig, E. G. Donall, T. T. Warner, and IMPROVE-1 storm of 1–2 February 2001. Part III: Sensitiv- C. M. Bhumralkar, 1989: Numerical studies of urban plan- ity of a mesoscale model simulation to the representation of etary boundary-layer structure under realistic synoptic con- snow particle types and testing of a bulk microphysical ditions. J. Appl. Meteor., 28, 760–781. scheme with snow habit prediction. J. Atmos. Sci., 64, 3927– Shafran, P. C., N. L. Seaman, and G. A. Gayno, 2000: Evaluation 3948. of numerical predictions of boundary layer structure during Zhang, D. L., and R. A. Anthes, 1982: A high-resolution model of the Lake Michigan Ozone Study. J. Appl. Meteor., 39, 412– the planetary boundary layer—Sensitivity tests and compari- 426. sons with SESAME-79 data. J. Appl. Meteor., 21, 1594–1609. Simmons, A. J., R. Mureau, and T. Petroliagis, 1995: Error growth ——, and J. M. Fritsch, 1986: Numerical simulation of the meso-␤ and estimates of predictability from the ECMWF forecasting scale structure and evolution of the 1977 Johnstown flood. system. Quart. J. Roy. Meteor. Soc., 121, 1739–1771. Part I: Model description and verification. J. Atmos. Sci., 43, Steenburgh, W. J., and C. F. Mass, 1996: Interaction of an intense 1913–1944. extratropical cyclone with coastal orography. Mon. Wea. Zhang, F., A. M. Odins, and J. W. Nielsen-Gammon, 2006: Me- Rev., 124, 1329–1352. soscale predictability of an extreme warm-season precipita- Stensrud, D. J., J.-W. Bao, and T. T. Warner, 2000: Using initial tion event. Wea. Forecasting, 21, 149–166.

Unauthenticated | Downloaded 10/01/21 12:56 PM UTC