Downloaded 09/23/21 10:55 PM UTC 138 WEATHER and FORECASTING VOLUME 14

VOLUME 14 WEATHER AND FORECASTING APRIL 1999

Evaluation of MM5 and Eta-10 Precipitation Forecasts over the Paci®c Northwest during the Cool Season

BRIAN A. COLLE,KENNETH J. WESTRICK, AND CLIFFORD F. M ASS Department of Atmospheric Sciences, University of Washington, Seattle, Washington

(Manuscript received 10 March 1998, in ®nal form 19 October 1998)

ABSTRACT Precipitation forecasts from the Pennsylvania State University±National Center for Atmospheric Research Mesoscale Model (MM5) and NCEP's 10-km resolution Eta Model (Eta-10) are veri®ed over the Paci®c Northwest in order to show the effects of increasing horizontal resolution, the spatial variations in model skill across the region, and the relative differences in performance between the two modeling systems. The MM5 is veri®ed at 36- and 12-km resolution for 9 December 1996 through 30 April 1997 using approximately 150 cooperative observer and National Weather Service precipitation sites across the Paci®c North- west. A noticeable improvement in bias, equitable threat, and root-mean-square (rms) error scores occurs as the horizontal resolution is increased. The spatial distribution of bias and equitable threat scores across Washington and Oregon indicate that the 12-km MM5 generates too much precipitation along the steep windward slopes and not enough precipitation in the lee of major barriers. The MM5 results were compared with Eta-10 forecasts from 7 January 1997 through 30 April 1997. The Eta- 10 overpredicts precipitation along the windward slopes even more than the 12-km MM5. As with the MM5, the Eta-10 forecasts excessive rainshadowing downwind of major barriers. Overall, the Eta-10 has lower rms errors than the 12-km MM5 at low precipitation thresholds, while the MM5 does signi®cantly better than the Eta-10 for higher thresholds (Ͼ2.54 cm in 18 h).

1. Introduction nia, Oregon, and Washington during the past decade has Precipitation over the Paci®c Northwest is greatly resulted in damage exceeding $5 billion and the loss of modulated, both in amount and type, by the complex dozens of lives (see the NOAA publication Storm Data). orography of the region (Fig. 1). Precipitation can vary For example, northern Oregon and southwest Washing- by an order of magnitude or more between the windward ton experienced the worst ¯ooding in 30 years during and leeward sides of regional mountain barriers, both the 5±9 February 1996 event. The combination of heavy in annual averages and for individual storms. For ex- precipitation (25±50 cm at many mountain sites), example, on the windward (southwestern) slopes of the tensive low-elevation snowpack, and rapid warming re- Olympic Mountains of Washington State the annual pre- sulted in ¯ooding conditions in which 30 000 residents cipitation can reach 300±500 cm, while on the lee were forced from their homes and damage exceeded $½ (northeastern) slopes the annual precipitation is gener- billion. ally less than 50 cm (Mass and Ferber 1990, their Fig. The distribution of precipitation over western North 4). These differences can be even more dramatic for America also greatly affects the regional economy individual storms, where 10±20 cm (4±8 in.) of precip- through its in¯uence on hydroelectric power generation, itation can fall along the windward side compared to a irrigation, and ®sheries. An extensive series of dams trace in the lee. over the region controls several major rivers and helps Variations in precipitation over the complex terrain mitigate ¯ooding; therefore, even modestly skillful of the region are important for several reasons. First, short-term forecasts have substantial economic value. If heavy precipitation and ¯ooding have been major heavy precipitation was accurately forecast a day or two weather problems over the mountainous coastal zone of in advance, dams could release water to provide more western North America. Flooding over coastal Califor- room to store more precipitation and meltwater from subsequent storms. Even though the accuracy of operational numerical models has increased during the past several decades Corresponding author address: Dr. B. A. Colle, Dept. of Atmo- spheric Sciences, Box 351640, University of Washington, Seattle, (Bonner 1989), precipitation forecasting along the WA 98195. mountainous West Coast has been hindered by the lack E-mail: [email protected] of horizontal grid resolution in operational models. For

᭧ 1999 American Meteorological Society 137

Unauthenticated | Downloaded 09/23/21 10:55 PM UTC 138 WEATHER AND FORECASTING VOLUME 14

FIG. 1. Topography (shaded, key at bottom) and geographical locations for the Paci®c Northwest. The ``ϫ'' locations are the synoptic sounding stations. example, numerical prediction±assimilation systems at to ameliorate some of these resolution problems, NCEP the National Centers for Environmental Prediction ran the Meso Eta (29-km resolution) between August (NCEP) such as the ``early'' Eta Model (Rogers et al. 1995 and April 1998 over most of North America, after 1995; Rogers et al. 1996) and the Nested Grid Model which the Meso Eta and 48-km ``early'' Eta were re- (NGM; Hoke et al. 1989) have used grid spacings of placed with a 32-km Eta. In addition, NCEP brie¯y 48±80 km and therefore were unable to accurately re- experimented with a 10-km resolution Eta Model (Eta- solve and forecast meteorological features forced by the 10) over the western United States. The additional hor- complex orography of western North America. In order izontal resolution used in the 29-km Meso Eta has im-

Unauthenticated | Downloaded 09/23/21 10:55 PM UTC APRIL 1999 COLLE ET AL. 139 proved precipitation forecasts over the western United States (Black 1994). For example, in contrast to the lower-resolution Eta and NGM models, the Meso Eta precipitation ®elds were in general agreement with observations for the January 1995 California ¯oods (Mar- tin 1996). Several studies have suggested that mesoscale models run at high resolution can realistically predict precipitation structures over complex terrain (Bruintjes et al. 1994; Colle and Mass 1996; Gaudet and Cotton 1998). For example, Colle and Mass (1996) ran the Pennsyl- vania State University±National Center For Atmospher- ic Research (PSU±NCAR) Mesoscale Model (MM5) at 3-km resolution around the Olympic Mountains of Washington State for a period of southwesterly ¯ow during the Coastal Observation and Simulation with To- pography ®eld experiment. The model was not only able to realistically simulate the observed precipitation structures (e.g., enhancement on the windward slopes of Van- couver Island and the Olympics, rain shadows to the lee of the Olympics and Cascades), but the model precipitation amounts were within 30% of the storm totals for FIG. 2. Model terrain contoured every 200 m for the 12-km domain. the approximately 30 observing sites within the domain. Terrain heights of 500±1300 m and greater than 1300 m are shaded Most previous studies have veri®ed precipitation fore- light and dark gray, respectively. The locations of the rain gauges casts from mesoscale models using a case study ap- used in this study are at the ``ϫ'' locations. proach. Now that mesoscale models are operational at a number of sites (Mass and Kuo 1998), model precipitation over complex terrain can be evaluated quanti- 30 April 1997. Section 5 compares the results for the tatively over longer time periods. In addition, knowl- MM5 and the Eta-10. A summary and conclusions are edge of the model precipitation biases over topography presented in the ®nal section. will hopefully identify weaknesses with model physical parameterizations, such as the bulk microphysical 2. Datasets and methods schemes. Analyzing a large number of model forecasts, this Approximately 30 National Weather Service (NWS) study addresses several important questions. stations and 120 National Climatic Data Center cooperative observer sites (COOP) in Washington and R Although mesoscale models appear to capture oro- Oregon were used to verify the real-time MM5 and Eta- graphically induced rainshadows and windward en- 10 forecasts (Fig. 2). Most of the NWS stations are hancements when run at high resolution, how does shielded tipping-bucket rain gauges, which report hourly model precipitation verify quantitatively over long at 0.254-mm (0.01 in.) resolution, while the COOP sta- time periods during the cool season? tions use primarily unshielded weighing gauges that re- R How does the ability of the MM5 to simulate meso- port hourly at 2.54-mm (0.1 in.) resolution. The ap- scale precipitation change as horizontal resolution in- proximately 100 SNOTEL (Snow Telemetry) stations, creases from 36 to 12 km? which are maintained by the U.S. Department of Ag- R How rapidly do the precipitation ®elds spin up when riculture Natural Resources Conservation Service (Mc- the model is initialized from a ``cold start'' (no initial Millan 1981), could not be used in this study since the cloud water or ice ®elds)? model precipitation was stored at 3- or 6-h time intervals R How do precipitation forecasts from the MM5 and the starting at 0000 or 1200 UTC while the SNOTEL data Eta-10 compare during the cool season over the Pa- were archived at 6- or 24-h intervals using Paci®c stan- ci®c Northwest? dard time (UTC Ϫ 8 h). R What is the spatial variability in precipitation forecast Special care should be taken when interpreting the skill? veri®cation results since rain gauge measurements can Section 2 discusses the observational datasets and have signi®cant errors. Some sources of error include analysis methods. Section 3 presents the veri®cation of excessive evaporation from heated gauges (Groisman the University of Washington real-time MM5 forecasts and Legates 1994), snow bridging over snow-pillow at 36- and 12-km resolution for the period of 9 Decem- sensors at SNOTEL sites (Ferguson et al. 1997), and ber 1996 through 30 April 1997. Section 4 describes snow capover for storage gauges. Gauges can under- the veri®cation of the Eta-10 from 7 January through estimate rainfall by 5%±15% because of wind effects

Unauthenticated | Downloaded 09/23/21 10:55 PM UTC 140 WEATHER AND FORECASTING VOLUME 14 near the gauge ori®ce and evaporation (Groisman and Legates 1994). Precipitation underestimates are particularly large in mountainous terrain where the winds are high and the precipitate is frequently frozen. For example, an unshielded gauge can fail to catch over 60% of snowfall when the winds are 10 m sϪ1 (Dingman 1994). Because of dif®culties in ascertaining individual gauge undercatchment, this study does not identify areas of possible model overprediction over mountainous terrain until the model precipitation exceeds the observed by at least 40%. Even if accurate at their location, precipitation observations are essentially point measurements and may not be representative of an areal mean. For example, FIG. 3. Layout of the contingency table, where each element of the Sinclair et al. (1997) presented an example of two rain matrix (A, B, C, and D) holds the number of occurrences in which gauges in the New Zealand Alps separated by 2 km the observations and the model reach or exceed a precipitation thresh- horizontally that measured 24-h precipitation of 315 and old amount for a given forecast period. 125 mm. Even considering the problems of gauge error and representativeness, rain gauges still provide valu- threshold amount and O is the number of occurrences able information considering that they are often the only in which the observations meet or exceed the threshold. type of precipitation measurement in complex terrain. The bias score indicates how well the model predicted In order to verify the MM5 and Eta-10 precipitation the frequency of occurrence of a given threshold, al- at the various observation locations, precipitation from though it provides no information on the accuracy of each model was interpolated to each observation site the forecasts. The bias score reveals systematic over- using an inverse distance Cressman method, prediction (bias Ͼ 1) and underprediction (bias Ͻ 1) WP by the model when averaged over many cases. The ET ͸ nn P ϭ nϭ1,4 , (1) measures the skill in predicting a given threshold at a W ͸ n location and is de®ned by nϭ1,4 H Ϫ EAϪ E where P is the model precipitation at the four model ET ϭϭ. (4) n F ϩ O Ϫ H Ϫ EAϩ B ϩ C Ϫ E grid points surrounding the observation. The weight Wn given to the surrounding gridpoint values is given by Here H is the number of forecast ``hits,'' where both Cressman (1959), the model and observation point meet or exceed a given R22Ϫ D precipitation threshold, F and O are de®ned above, and W ϭ n , (2) n 22 E is de®ned as R ϩ Dn FO (A ϩ B)(A ϩ C) where R is the model horizontal grid spacing and D is E ϭϭ , (5) the horizontal distance from the model grid point to the NN observation. where N is the total number of observations veri®ed Some of the veri®cation scores used in this study are (Mesinger 1996). The score is similar to the canonical derived using a contingency table approach (Wilks threat score, T ϭ H/(F ϩ O Ϫ H), except that the E 1995). This table represents a 2 ϫ 2 matrix (Fig. 3), term corrects for the expected number of hits by chance. where each element of the matrix holds the number of There are a few advantages in using the contingency occurrences in which the observations and the model table to verify model precipitation. For example, since did or did not reach a certain threshold amount for a every forecast event is treated equally (contingency ma- given forecast period. For example, if both the obser- trix is simply incremented by one each time a threshold vation and the model veri®ed at that point reach or criteria is met), the model bias and threat scores are not exceed the threshold criteria, the number of occurrences heavily in¯uenced by a few extremely bad forecasts. in the ``A'' box is incremented by one. Furthermore, veri®cation statistics based on the contin- Based on this contingency table one can create bias gency table do not punish the model severely because and equitable threat (ET) scores. The bias score is de- of a few bad observations. ®ned as The bias and equitable threat scores based on the FAϩ B contingency table only measure model accuracy based Bias ϭϭ , (3) on the frequency of occurrence at or above a precipi- OAϩ C tation threshold, and thus do not determine the mag- where F is the number of forecasts at the observation nitude of the precipitation errors in the model. There- stations with precipitation equal to or exceeding a given fore, it is also important to calculate model biases and

Unauthenticated | Downloaded 09/23/21 10:55 PM UTC APRIL 1999 COLLE ET AL. 141 root-mean-square errors (rmse) using the actual quantitative precipitation from the model. For example, the percentage bias score, BP, for a given threshold can be de®ned as

Pn ͸ X B ϭ nϭ1,NTOT n , (6) P NTOT where Xn is the observed precipitation, Pn is the model precipitation at an observation point, and NTOT is the total number of observations and forecasts at that location reaching a certain threshold. Therefore, when multiplied by 100 the bias gives the percentage differ- FIG. 4. Model terrain contoured every 200 m for the 36-km domain. ence between the model and the observations. Similarly, Terrain heights of 700 to 1500 m and greater than 1500 m are shaded an rms error can be de®ned by light and dark gray, respectively. The inner box shows the location of the 12-km nested domain. (P Ϫ X )2 ͸ nn RMSE ϭ nϭ1,NOBS , (7) Ί NOBS from being re¯ected off the upper boundary of the mod- where NOBS is the total number of observations at that el. location reaching a certain threshold. The rms error mea- For these real-time simulations, a stationary 12-km sures the magnitude of the difference between the fore- domain was nested within a 36-km domain using a one- cast and the observed values, and is greatly affected by way interface (Fig. 4); therefore, solutions from the 12- infrequent but large forecast errors. km domain could not feedback to the 36-km domain. Thirty-three unevenly spaced full-sigma levels were used in the vertical, with the maximum resolution in the 3. Veri®cation of real-time MM5 forecasts 1 over the Paci®c Northwest boundary layer. Five-minute-averaged terrain data were analyzed to the 36- and 12-km model grids using a The nonhydrostatic version of the PSU±NCAR MM5 Cressman analysis scheme and ®ltered by a two-pass was run twice daily (0000 and 1200 UTC) at the Uni- smoother/desmoother (Guo and Chen 1993). A 10-min versity of Washington from 9 December 1996 through land use dataset from NCAR was used to initialize 13 30 April 1997 using a SUN ES-4000 Enterprise server surface categories, and the coastline for the 12-km do- with 14 processors. A total of 228 forecasts out of the main was enhanced using a 30-s latitude±longitude to- possible 282 were veri®ed during this period. Several pography dataset by setting grid points to either conif- forecasts were missing either because of computer erous forest or water when the elevation is greater than downtime or the lack of NCEP grids for initial and or equal to zero, respectively. The sea surface temper- boundary conditions. Since early October of 1997 the atures in the model were derived using a 1Њ resolution MM5 has been run twice daily at the University of dataset available daily from NCEP. Since a high-reso- Washington with an inner nest at 4-km resolution over lution snow cover dataset was not available during this western Washington; however, a statistical comparison period, snow cover was placed at those grid points east between this domain and the others will be left for future of 120.9ЊW and north of 40ЊN where the model terrain work when more archived precipitation data become exceeded 750 m; otherwise, the elevation had to exceed available. This section describes the veri®cation of the 1200 m for snow cover to be assumed. Initial atmo- 36- and 12-km precipitation forecasts for Washington spheric conditions were obtained from NCEP, where the and Oregon. data was interpolated from the native Eta (48 km) grid to a resolution of 90 km and 25 mb in the horizontal a. Model description and vertical, respectively. These analyses were bili- nearly interpolated to the MM5 grids, while boundary All MM5 simulations used the explicit moisture conditions were obtained by linearly interpolating Eta scheme of Hsie et al. (1984), with improvements to Model forecast analyses at 6-h intervals. allow for ice-phase microphysics below 0ЊC (Dudhia 1989), and the Kain±Fritsch cumulus parameterization (Kain and Fritsch 1990) was applied in the 36- and 12- km domains. The planetary boundary layer was param- 1 The 33 full-sigma levels were ␴ ϭ 1.0, 0.99, 0.98, 0.97, 0.96, eterized using the scheme of Zhang and Anthes (1982). 0.94, 0.92, 0.90, 0.88, 0.86, 0.83, 0.80, 0.77, 0.74, 0.71, 0.68, 0.64, Klemp and Durran's (1983) upper-radiative boundary 0.60, 0.56, 0.52, 0.48, 0.44, 0.40, 0.36, 0.32, 0.28, 0.24, 0.20, 0.16, condition was applied in order to prevent gravity waves 0.12, 0.08, 0.04, 0.

Unauthenticated | Downloaded 09/23/21 10:55 PM UTC 142 WEATHER AND FORECASTING VOLUME 14

FIG. 6. Bias scores calculated every 6 h for the 36-km (light gray) FIG. 5. Bias scores calculated for the 12-km domain (24±36 h into the simulations) from 9 Dec 1996 through 30 Apr 1997 using four and 12-km (black) domains from 9 Dec 1996 through 30 Apr 1997. different methods. The three thresholds (in mm) used in the calculation are labeled in the inset box. b. Sensitivity of statistics to length of 24±36-h forecast at the COOP locations (Fig. 5). Using veri®cation period this approach the bias scores increased for all the thresh- The COOP stations use the standard Fischer±Porter olds. A 24-h carryover period was also tried by adding weighing rain gauges, which record hourly to the nearest the ``uncounted'' 24-h fractional amount (Ͻ2.54 mm) 2.54-mm increment (Fischer and Porter Co. 1967); from the sum of the 12±24-h forecast and the 12±24-h therefore, any fractional amount between 0 and 2.54 forecast from the model run completed 12 h earlier. mm that is ``left over'' contributes to the next time pe- Using this approach the model bias scores are similar riod. As a result of this precipitation carryover, the ac- to the 12-h carryover. Overall, these results suggest that curacy of the model bias and threat scores can be sen- in order to more accurately determine the model bias sitive to the length of the veri®cation period, particularly and threat scores for rain gauge locations with reporting for periods of light precipitation. For example, at the increments of 2.54 mm hϪ1, precipitation amounts less start of any veri®cation period the COOP rain gauges than 2.54 mm that would not have been ``observed'' in likely have some ®nite amount of carryover precipita- previous forecast periods should be carried over. tion (between 0 and 2.54 mm) that has not been re- An alternative to this carryover approach is to simply corded. Therefore, the model bias and threat scores may start each veri®cation period at the COOP sites with be lowered simply because at the start of a veri®cation 1.27 mm (0.05 in.) in the model ``gauge'' (Fig. 5), since period the COOP gauges began with nonzero precipi- over a long period the average amount of precipitation tation and subsequently may reach and report the next not recorded yet by the COOP rain gauges for a given 2.54-mm increment and veri®cation threshold, while the time period is 1.27 mm (0.05 in.). This method yields model does not, even if the model precipitation during nearly the same result as the 24-h spinup approach dis- the period is completely accurate. cussed above and, thus, was used in the statistical results To illustrate this point, Fig. 5 shows the 12-h bias shown below. scores [Eq. (3)] for the 12-km domain for 24±36 h into the forecasts (9 December 1996±30 April 1997). When c. The 6-h forecast statistics the model accumulation is initialized with zero precipitation at the beginning of each 12-h veri®cation period, To determine how veri®cation scores change as a the bias scores for all the thresholds are generally less function of time throughout the simulation, bias and than one but are probably not representative of the true equitable threat scores based on the contingency table model bias given the aforementioned problem. There- were computed at 6-h intervals between 0 and 36 h into fore, an attempt was made to treat the model precipi- the simulations from 9 December 1996 through 30 April tation more like a rain gauge by having a ``carryover'' 1997. Figure 6 shows the 6-h bias scores for the 36- from the preceeding period. Speci®cally, model precip- and 12-km resolutions using only observations within itation under 2.54 mm that was not counted in the pre- the 12-km domain. For both resolutions, a rapid spinup vious 12-h forecast period (12±24 h) was added to the of precipitation is evident during the ®rst 12 h since the

Unauthenticated | Downloaded 09/23/21 10:55 PM UTC APRIL 1999 COLLE ET AL. 143

FIG. 8. Bias scores vs precipitation threshold (in.) calculated for FIG. 7. Equitable threat scores calculated every 6 h for the 36-km (light gray) and 12-km (black) domains from 9 Dec 1996 through 30 the 12±36-h forecast period for the 36-km (dashed) and 12-km (solid) Apr 1997. The three thresholds (in mm) used in the calculation are domains from 9 Dec 1996 through 30 Apr 1997. labeled in the inset box. in. (2.54 and 7.62 mm) thresholds, while the 36-km bias model was initialized with a cold start (i.e., no hydro- scores are 20%±30% lower than the 12-km domain for meteors and signi®cant ageostrophic motions). This spi- the higher thresholds. The bias scores for the 12-km nup slows between hours 12 and 18 into the simulation. domain drop to around 1.07 for the 0.9-in. (22.86 mm) After 18 h, the bias scores increase only slightly. threshold, but then rise to around 1.3 for the 1.7-in. For the lightest thresholds (2.54 mm), the bias scores (43.18 mm) threshold. The bias scores for the 36-km after model spinup are fairly comparable at 36- and 12- domain also increase slightly for thresholds greater than km resolutions. Even though the bias scores (between 1.3 inch (33.02 mm). The next section (section 3e) will 1.15 and 1.20) suggest a small frequency of model over- focus on speci®c locations in the 12-km domain where prediction, the model may actually be doing quite well the high bias scores are located. considering rain gauge undercatchment. For the 7.62- The 12-km resolution grid has higher 24-h equitable and 12.70-mm thresholds, the 12-km bias scores are threat scores than 36-km resolution for all thresholds 20%±30% greater than those of the 36-km resolution, (Fig. 9). However, the difference in threat scores be- with the 12-km forecasts at the higher intensity having tween 12- and 36-km resolution does not vary with the a nearly perfect (1) bias. These results suggest the im- higher thresholds, which may be the result of the slight portance of terrain resolution for capturing higher oro- overprediction problem (bias Ͼ 1) seen in the 12-km graphic rain rates. domain for thresholds greater than 1.3 in. (33.02 mm). The 6-h equitable threat scores for the 12-km domain Section 5 will illustrate that these results are somewhat are generally greater than the 36-km domain (Fig. 7), sensitive to the months included in the veri®cation. especially for the higher (7.62 and 12.70 mm) thresholds. For the lightest threshold (2.54 mm), the highest e. Spatial statistics across Washington and Oregon threat scores occur 18 h after the initialization for both the 36- and 12-km resolutions. In contrast, the highest Even though the above statistics suggest that there threat scores for the 7.62- and 12.70-mm thresholds oc- are bene®ts in increasing resolution from 36 to 12 km, cur at hour 12, which suggests that model skill deteri- these improvements may not be distributed equally orates more rapidly for higher precipitation intensity. across the region. To determine the accuracy of the 36- and 12-km model resolutions for speci®c locations, 12-h bias and equitable threat scores were calculated for each d. The 24-h forecast statistics station and plotted spatially. This technique has been Figure 8 presents the 24-h (12±36 h) bias scores for used in the past to verify the NGM and Medium-Range the 36- and 12-km resolutions from 9 December 1996 Forecast Model across the contiguous United States through 30 April 1997 using only observations within (Junker et al. 1992). A 12-h precipitation forecast for a the 12-km domain. In agreement with the 6-h bias scores particular station was only included in the analysis when shown in Fig. 6, the bias scores are very similar (around the average wind direction at 850 mb was within a cer- 1.2) for the 12- and 36-km domains at the 0.1- and 0.3- tain range (i.e., 180Њ±270Њ or 240Њ±320Њ), since by se-

Unauthenticated | Downloaded 09/23/21 10:55 PM UTC 144 WEATHER AND FORECASTING VOLUME 14

ington and Oregon may be the result of the dif®culty in capturing the light snow events using primarily unshielded gauges. Figure 10b shows the 12-h spatial bias patterns for the 2.54-mm (0.1 in.) threshold during westerly ¯ow regimes (240Њ±320Њ),2 which is more representative of postfrontal situations and is associated with ¯ow nearly normal to the major orographic barriers of the region. The lowest bias scores (Ͻ80%) are located along the lee (eastern) side of the Cascades and on the eastern side of the Oregon coastal range. Therefore, when there is signi®cant cross-barrier ¯ow (westerly winds), the model tends to overforecast precipitation shadowing in the lee of major orographic barriers. The bias scores are greater than 140% around the Puget Sound Basin and across eastern Washington and Oregon. The best bias scores (between 90% and 120%) are located along the Paci®c coast and over the Cascades. For a higher (10.16 mm, 0.4 in.) threshold during FIG. 9. Equitable threat scores vs precipitation threshold (in.) cal- periods of southwesterly ¯ow at 850 mb (180Њ±270Њ), culated for the 12±36-h forecast period for the 36-km (dashed) and the 12-km bias scores are less than 90% for many re- 12-km (solid) domains from 9 Dec 1996 through 30 Apr 1997. gions to the lee of the coastal range and Cascades (Fig. 11). The bias scores are substantially lower to the east lecting only a limited range of wind directions at 850 of the Oregon coastal range and along the Washington mb the locations of the windward enhancement or lee coast compared to the weaker events (2.54-mm thresh- precipitation shadowing would be relatively ®xed. Since old; Fig. 10a). Bias scores are generally greater than the model winds at 850 mb were not archived, the 850- 150% along the steep windward slopes of the southern mb winds over each station were calculated by inter- Oregon Cascades and immediately upwind of Mount polating the sounding data at Port Hardy, British Co- Rainier in Washington, indicating that the model often lumbia (YZT); Quillayute, Washington (UIL); Spokane, overpredicts precipitation in these regions. Washington (OTX); Salem, Oregon (SLE); Medford, Figure 12 shows the 12-km equitable threat scores Oregon (MRF); and Boise, Idaho (BOI) to each precip- for the southwesterly (180Њ±270Њ) ¯ow regime for the itation site using the Cressman method [Eqs. (1) and combined 12±24- and 24±36-h forecasts periods. At the (2) above] with a radius of in¯uence (R) of 400 km. 2.54-mm (0.1 in.) threshold (Fig. 12a), the highest threat The winds within a given 12-h period were obtained by scores (Ͼ0.35) are located along the windward slopes time averaging between the two synoptic times (0000 of the southern Washington and Oregon Cascades. The and 1200 UTC). lowest threat scores (Ͻ0.25) are situated to the north of Figure 10a shows the 12-km domain bias scores for the Olympics, within Stampede Gap, and to the east of the 2.54-mm (0.1 in.) threshold using only those 12-h the Cascades. The spatial distributions of the equitable forecasts associated with southwesterly (180Њ±270Њ) threat scores are similar for the 10.16-mm (0.4 in.) ¯ow at 850 mb for the 12±24- and 24±36-h forecast threshold (Fig. 12b); however, some of the lowest threat periods. A station must have at least 20 occurrences for scores (Ͻ0.20) are now found over the steeper wind- a given precipitation threshold and wind regime in order ward slopes of the southern Oregon Cascades because to be included in the analysis. Because of the afore- of the overprediction problem in these areas (Fig. 11). mentioned problem of rain gauge undercatchment of Spatial bias and equitable threat scores were also cal- frozen precipitation, the model is likely not seriously culated for the 36-km domain across Washington and overpredicting precipitation until the bias score exceeds Oregon (not shown). At the lowest 12-h threshold (2.54 140%. Bias scores lower than 100% are located along mm), the differences in both bias and threat scores be- portions of the Oregon coastal range and along the im- tween the 12- and 36-km domains were not signi®cant, mediate lee (eastern side) of the northern Oregon and which agrees with the domain-averaged scores shown the central Washington Cascades. In contrast, the bias above (Figs. 6±9). Figure 13 shows the differences be- scores are between 100% and 115% for many locations tween the 36- and 12-km bias and equitable threat scores along the western slopes of the Oregon and the southern for the 10.16-mm (0.4 in.) threshold for southwesterly Washington Cascades. Possible areas of frequent overprediction, where the bias scores exceed 140%, are located over the southern Oregon Cascades, central Puget 2 The center of the westerly quadrant (240Њ±320Њ) was skewed Sound, and much of eastern Washington and Oregon. slightly to west-northwesterly (280Њ) in order to include more post- However, some of the high biases over eastern Wash- frontal cases in the analysis.

Unauthenticated | Downloaded 09/23/21 10:55 PM UTC APRIL 1999 COLLE ET AL. 145

FIG. 10. (a) Bias scores (ϫ 100) for the 12-km MM5 domain at the 2.54-mm (0.1 in.) threshold for 180Њ± 270Њ winds at 850 mb. The 90% and 140% lines are contoured with dashed and solid lines, respectively. Terrain is shaded for reference. (b) Same as (a) except for the 240Њ±320Њ 850-mb winds.

¯ow conditions (180Њ±270Њ). For these more intense pre- 50% lower in the lee of the Cascades and the Oregon cipitation events the 12-km bias scores are 20%±90% coastal range. greater over the Cascades and along the Paci®c coast Because of the excessive rain shadowing noted above than those calculated for the 36-km domain (Fig. 13a). (Fig. 11), the equitable threat scores are actually greater In contrast, the 12-km bias scores are generally 10%± for the 36-km resolution in the lee of both the Oregon coastal range and the Cascades than for the 12 km (Fig. 13b). The overprediction of precipitation over the steeper windward slopes also results in greater 36-km threat scores on the windward side of Mount Rainier and the central and southern Oregon Cascades. The 12-km domain does best near the crest of the Cascades and along moderate windward slopes. The above bias and equitable threat scores are based on frequency of occurrence and, therefore, do not indicate the total quantitative precipitation error for a given precipitation threshold. In order to quantify the percentage errors in the model, bias scores were also calculated by dividing the model and observed precipitation [Eq. (6) above] for the 12±24- and 24±36-h forecast periods (Fig. 14). In agreement with the bias scores based on frequency statistics (cf. Fig. 10b), the 12-h spatial bias patterns for the 2.54-mm (0.1 in.) threshold during westerly ¯ow regimes (240Њ±320Њ) are best along the coast and over the lowland regions of western Wash- ington and Oregon (Fig. 14a). There are areas of overprediction (Ͼ140%) along steeper windward slopes of the Cascades and throughout much of eastern Washing- ton and Oregon, while the bias scores are generally lower than 90% in the lee of the Olympics, Oregon coastal range, northern Oregon and Washington Cascades, and within Stampede Gap. For the higher threshold (10.16 FIG. 11. Bias scores (ϫ100) for the 12-km MM5 domain at the 10.16-mm (0.4 in.) threshold for 180Њ±270Њ winds at 850 mb. For mm) under southwesterly ¯ow (Fig. 14b), the model reference the 90% and 140% lines are contoured with dashed and biases range from 150% to 200% on some of the steep solid lines, respectively. Terrain is shaded for reference. windward slopes, and are generally less than 90% to the

Unauthenticated | Downloaded 09/23/21 10:55 PM UTC 146 WEATHER AND FORECASTING VOLUME 14

FIG. 12. (a) Equitable threat scores (ϫ 100) for the 12-km MM5 domain at the 2.54-mm (0.1 in.) threshold for 180Њ±270Њ winds at 850 mb. The 20 and 35 isopleths are contoured with dashed and solid lines, respectively. Terrain from the 12-km MM5 is shaded for reference. (b) Same as (a) except for the 10.16- mm (0.4 in.) threshold. lee of major barriers. Overall, these bias scores are gen- through 30 April 1997 period and dividing by the total erally 10%±20% higher in the lee of the Cascades than observed precipitation for this period (Fig. 15). This the scores based on frequency of occurrence. This sug- analysis did not add the 1.27 mm (0.05 in.) to the model gests that even though the model underforecasts pre- precipitation for each 24-h period since a longer period cipitation 20%±50% of the time in these regions (cf. was used (24 h); therefore, it provides a good check on Figs. 10b and 11), the actual 12-h model precipitation the results presented above. In agreement with the re- is only underpredicted by 10%±30%. sults from other methods shown above, the total pre- Bias scores were also calculated for each station in cipitation is generally underforecasted by 10%±30% the 12-km domain by summing up all the 24-h precip- downwind of the major orographic barriers as well as itation forecasts (12±36 h) from the 9 December 1996 within Stampede Gap of the central Washington Cas-

FIG. 13. (a) Bias scores differences (12±36-km domains) for the MM5 at the 10.16-mm (0.4 in.) threshold for 180Њ±270Њ winds at 850 mb. The 0% line is contoured (solid line). Negative values indicate higher values in the 36-km domain. Terrain from the 12-km MM5 is shaded for reference. (b) Same as (a) except for the equitable threat scores.

Unauthenticated | Downloaded 09/23/21 10:55 PM UTC APRIL 1999 COLLE ET AL. 147

FIG. 14. (a) Bias scores (ϫ100) for the 12-km MM5 domain at the 2.54-mm (0.1 in.) threshold in 12 h (12±24- and 24±36-h forecasts) for 240Њ±320Њ winds at 850 mb. The bias scores represent the model percentage of the observed precipitation produced by the model. The 90% and 140% lines are contoured with dashed and solid lines, respectively. Terrain is shaded for reference. (b) Same as (a) except for the 10.16-mm (0.4 in.) threshold and 180Њ±270Њ 850-mb winds.

cades (cf. Fig. 1), while the bias scores exceed 140% along many of the steep windward slopes. Overall, these bias statistics suggest that the model places too much precipitation along the windward slopes of barriers, re- sulting in not enough precipitation falling in the lee.

4. Veri®cation of the 10-km Eta precipitation forecasts over the Paci®c Northwest This section presents precipitation veri®cation for the 10-km Eta Model over the Paci®c Northwest during the cool season. In section 5, the Eta-10 results are compared directly with the 36- and 12-km MM5 domains using the same forecasts and observations.

a. Model setup and physics Since its introduction at NCEP in June 1993, the Eta Model has run operationally at resolutions of 80 km, 48 km (implemented September 1993), 29 km (August 1995), and 32 km (February 1998). An experimental 10-km nested version of the mesoscale Eta Model with 60 vertical levels (hereafter referred to as the Eta-10) was run daily over the western United States from 2 January 1997 through 21 August 1997 (Fig. 16a). From 2 January 1997 to 26 February 1997, the Eta-10 was nested within the 29-km/50-level Meso Eta Model initialized at 0900 UTC. After 26 February 1997 the Eta- FIG. 15. Bias scores (ϫ 100) for the 12-km MM5 domain calculated by adding all the 12±36-h model precipitation forecasts and dividing 10 initialization was switched to 0300 UTC to provide by the observed. The 90% and 140% lines are contoured with dashed more timely guidance for the ®eld forecasters. During and solid lines, respectively. Terrain is shaded for reference. the 1996±97 winter season, the 29-km Eta received its

Unauthenticated | Downloaded 09/23/21 10:55 PM UTC 148 WEATHER AND FORECASTING VOLUME 14

as the 29-km Meso Eta Model. Convective precipitation in the Eta is based on the Betts±Miller cumulus parameterization (JanjicÂ 1994). On 13 March 1997, the min- imum required depth for shallow convection was changed from 290 mb (ϳ3000 m in the lower tropo- sphere) to 2000 m to allow the model to produce more precipitation over complex terrain (McDonald and Horel 1998; Black et al. 1998). The Eta Model calculates grid- scale precipitation using a simpli®ed explicit cloud water scheme (Zhao and Carr 1997). This scheme includes supercooled water and snow processes, but does not allow for the horizontal advection of rain and snow (advection of cloud ice and cloud water are included). The radiation package used in the model was developed at the Geophysical Fluid Dynamics Laboratory (Stau- denmaier 1996). The vertical turbulence exchange in the Eta uses a modi®ed Mellor±Yamada level 2.5 scheme (JanjicÂ 1994), in which turbulent kinetic energy is a fully prognostic variable. Previous studies have suggested that the Eta-10 provides more realistic forecasts of precipitation and low- level winds than the 48- and 29-km Eta resolutions (McDonald et al. 1996; McDonald and Horel 1998; Staudenmaier and Mittelstadt 1998). This is at least par- tially attributable to the more realistic terrain used in the Eta-10 (Fig. 16b). On the other hand, some negative results have been observed. For example, McDonald and Horel (1998) veri®ed the 24-h Eta-10 precipitation forecasts over the southwestern United States and found that slight overprediction occurred for small amounts while substantial underprediction was prevalent for amounts greater than 0.75 in. In addition, during postfrontal situations too much precipitation occurs over the valleys and not enough over the mountains (Stauden- maier and Mittelstadt 1998; Baldwin and Black 1998). The next two sections of this paper will describe precipitation veri®cation for the Eta-10 across Washington and Oregon for the period 7 January 1997 through 30 April 1997.

b. The 6-h forecast statistics FIG. 16. (a) Locations for the Eta-29 and Eta-10 domains. (b) Ter- rain used in the Eta-10 around the Paci®c Northwest contoured every The time-dependent bias and equitable threat scores 300 m. The terrain contours were generated using the topography for the Eta-10 simulations based on the contingency height of the center of each grid box. table were computed at 6-h intervals between forecast hours 3 and 33 for all the available simulations between 7 January 1997 and 30 April 1997 (Fig. 17). For all ®rst guess from the Eta Data Assimilation System (Stau- thresholds (2.54, 7.62, and 12.70 mm), the 6-h bias denmaier 1996) and its boundary conditions from the scores exceed one during the 3±9-h forecast period (Fig. Aviation Model. The Eta-10 was integrated for 33 h, 17a), which suggests that the model spins up precipi- during which time the boundary conditions were inter- tation rather quickly. Between hours 9 and 21 the bias polated from the 29-km Eta output every 3 h before scores continue to increase for the 7.62- and 12.70-mm April 1997 and at 1-h intervals afterward. Unlike the thresholds followed by a rapid decline to around one 29-km Eta, which used optimal interpolation analysis between hours 27 and 33. The bias score for the 2.54- for initial conditions in early 1997, the Eta-10 used a mm threshold also slowly falls from 1.37 to 1.18 be- 3D variational analysis scheme (Staudenmaier 1996; tween hours 21 and 33. The Eta-10 equitable threat Black et al. 1998). scores are better at lighter (2.54 mm) threshold amounts The Eta-10 shares many of the same physics packages than for moderate (7.62 mm) and heavy (12.70 mm)

Unauthenticated | Downloaded 09/23/21 10:55 PM UTC APRIL 1999 COLLE ET AL. 149

FIG. 17. (a) Bias scores calculated every 6 h for the Eta-10 from 7 January 1997 through 30 April 1997. The three thresholds (in mm) used in the calculation are labeled in the inset box. (b) Same as (a) except for the equitable threat scores. rain rates (Fig. 17b). For all thresholds the threat score at the surrounding sounding stations. The number of is relatively unchanged throughout the period. occurrences at each site required to be included in the analysis was reduced from 20 (for the MM5) to 15 since the Eta-10 was run only once daily and the veri®cation c. Spatial statistics across Washington and Oregon period was approximately a month shorter than the To determine the accuracy of the Eta-10 precipitation MM5. for speci®c locations across Washington and Oregon, Figures 18a,b show the Eta-10 bias scores for the 12-h bias and equitable threat scores were calculated 2.54-mm threshold using only those observations as- for each individual station and plotted spatially. As done sociated with southwesterly (180Њ±270Њ) and westerly with the MM5 data in section 3e, these statistics were (240Њ±320Њ) ¯ow at 850 mb, respectively, for the 9±21- then separated by the 850-mb wind direction observed and 21±33-h forecasts periods. For the southwesterly

FIG. 18. (a) Bias scores (ϫ100) for the 10-km Eta at the 2.54-mm (0.1 in.) threshold in 12 h (9±21- and 21±33-h forecasts) for 180Њ±270Њ winds at 850 mb. The 90% and 140% lines are contoured with dashed and solid lines, respectively. Terrain is shaded for reference. (b) Same as (a) except for the 240Њ±320Њ 850-mb winds.

Unauthenticated | Downloaded 09/23/21 10:55 PM UTC 150 WEATHER AND FORECASTING VOLUME 14

FIG. 19. (a) Bias scores (ϫ100) for the 10-km Eta at the 7.62-mm (0.3 in.) threshold in 12 h (9±21- and 22±33-h forecasts) for 180Њ±270Њ winds at 850 mb. The 90% and 140% lines are contoured with dashed and solid lines, respectively. Terrain is shaded for reference. (b) Same as (a) except for the equitable threat score (ϫ100). The 20 and 30 isopleths are contoured with dashed and solid lines, respectively. cases (Fig. 18a), the bias scores are generally greater to high thresholds the Eta-10 often overpredicts precip- than 90% over most of the domain. The only areas of itation over the lower windward slopes and frequently modest underprediction (bias Ͻ 90%) are located over underpredicts in the lee. sections of the Oregon coastal range and in the lee of In order to quantify the percentage errors in the Eta- the Cascades to the east of Portland, Oregon. Most of 10, 12-h bias scores were also calculated by dividing eastern Washington and Oregon has bias scores greater the model by the observed precipitation [Eq. (6) above] than 150%; these high biases may simply be the result for the 9±21- and 21±33-h forecast periods. For the of using unshielded gauges to record the light snow 7.62-mm (0.3 in.) threshold under southwesterly ¯ow events. Bias scores are also greater than one along the (180Њ±270Њ) (Fig. 20), the Eta-10 overpredicts precipi- foothills of the central and northern Washington Cas- tation by 50%±100% over the windward slopes of the cades. Cascades and underforecasts by 20%±30% to the lee of For the westerly (240Њ±320Њ) regime (Fig. 18b), the major barriers. Eta bias scores are between 70% and 90% along the coastal regions of Washington and Oregon. This suggests that the Eta-10 frequently underpredicts coastal 5. Comparison between the 36- and 12-km precipitation during many of the postfrontal situations. MM5, and the Eta-10 Bias scores are also less than 70% in the immediate lee Previous sections of this paper have presented veri- of the Cascades, which indicates that model tends to ®cation results examining the MM5 and Eta-10 models overshadow precipitation when there is signi®cant separately for slightly different periods. The goal of this cross-barrier ¯ow. section is to compare the two models more directly by For the 7.62 mm in 12-h threshold under southwest- using the same forecast veri®cation times and obser- 3 bias scores are generally erly ¯ow conditions (Fig. 19a), vations. Furthermore, since the MM5 used boundary greater than 150% over the windward slopes of the Cas- and initial conditions from the Eta model, any deviations cades. This result is also true for thresholds greater than between the two models are likely related to differences 7.62 mm (not shown). In contrast, the bias scores are in model physics and resolution rather than initial or generally less than 75% in the lee of the coastal range boundary conditions. and the Cascades. The lowest equitable threat scores Figures 21a,b show the 18-h bias and equitable threat (0.15±0.25) are associated with many of the areas with scores based on the contingency table for the 12- and poor bias scores (Fig. 19b). Overall, for the moderate 36-km MM5 as well as the Eta-10 using the identical veri®cation period (7 January 1997±30 April 1997). To avoid spinup problems, the 12±30-h forecast was used 3 The value 7.62 mm (0.3 in.) was used as a higher threshold rather for the MM5, while either the 9±27- or 15±33-h forecast than 10.16 mm in order to include more data in the statistics. was used for the Eta-10 in order to match veri®cation

Unauthenticated | Downloaded 09/23/21 10:55 PM UTC APRIL 1999 COLLE ET AL. 151

then gradually decrease to around 0.85 for the 1-in. (25.4 mm) threshold. The bias score for the 36-km MM5 also starts near 1.2 for the 0.1-in. (2.54 mm) threshold, but then rapidly decreases to approximately 0.45 for the 1.1- in. threshold. Thus, there is a dramatic improvement in bias scores as the horizontal resolution in the MM5 is increased from 36 to 12 km, which is expected considering the more realistic orography in the 12-km domain. An important question is whether the veri®cation scores vary in time. At thresholds greater than 1 in. (25.4 mm), the 18-h bias scores for the 36- and 12-km MM5 domains shown in Fig. 21 for the period 7 January 1997 through 30 April 1997 are 20%±40% greater than the 24-h bias statistics from 9 December 1996 through 30 April (cf. Fig. 8). Some of these differences may be the result of using two different forecast intervals (12± 30 vs 12±36 h); however, it is also likely demonstrates that these scores can vary with the time period used in the veri®cation. For example, during the last week of December and the ®rst week of January the Paci®c Northwest experienced widespread ¯ooding and mud- slides due to several heavy precipitation events (Halpert and Bell 1997). The bias scores for both the 36- and 12-km domains are greater for the 9 December 1996 FIG. 20. Bias scores (ϫ100) for the 10-km Eta at the 7.62-mm (0.3 in.) threshold in 12 h (9±21- and 21±33-h forecasts) for 180Њ±270Њ through 7 January 1997 period than any other 4-week winds at 850 mb. The bias scores represent the model percentage of period for thresholds greater than 1 in. (25.4 mm, not the observed precipitation. For reference the 90% and 140% lines shown). are contoured with dashed and solid lines, respectively. Terrain is shaded for reference. Even though the bias scores in the Eta-10 are much greater than one at all thresholds and are worse than the MM5 for thresholds less than 0.9 in. (22.86 mm), the times with the MM5. For all thresholds the bias scores equitable threat scores are comparable with the 12-km in the Eta-10 are between 1.2 and 1.4, much higher than MM5 for those thresholds (Fig. 21b). The threat scores either MM5 domain. The bias scores for the 12-km for both models gradually drop from 0.37 to 0.25 as the MM5 are around 1.2 for the lowest two thresholds and thresholds increase from 0.1 to 0.9 in., and at thresholds

FIG. 21. (a) Bias scores calculated from 7 Jan 1997 through 30 Apr 1997 using the 12±30-h forecasts for the 36-km (long dashed) and 12-km (solid) MM5 domains and the 9±27- or 15±33-h forecasts for the Eta-10 (short dashed). (b) Same as (a) except for the equitable threat scores.

Unauthenticated | Downloaded 09/23/21 10:55 PM UTC 152 WEATHER AND FORECASTING VOLUME 14

in the 12-km domain are 10%±20% lower than the 36 km.

6. Summary and conclusions This paper veri®es the 36- and 12-km MM5 precipitation forecasts from 9 December 1996 through 30 April 1997 and NCEP's 10-km Eta (Eta-10) forecasts from 7 January 1997 through 30 April 1997 across the Paci®c Northwest using approximately 150 cooperative observer and NWS precipitation sites. This study shows some of the limitations in the current precipitation gauge network. Since many of these gauges report at resolutions of 2.54 mm (0.1 in.) per hour, the model bias and threat scores can be sensitive to the length of the veri®cation period, particularly if the precipitation is light. If zero precipitation is assumed at the beginning of a veri®cation period, the model bias and threat scores may be lowered simply because the FIG. 22. Root-mean-square error scores calculated from 7 Jan 1997 COOP stations typically begin a period with nonzero through 30 Apr 1997 using the 12±30-h forecasts for the 36-km (long dashed) and 12-km (solid) MM5 domains and the 9±27- or 15±33-h precipitation and subsequently reach a given threshold forecasts for the Eta-10 (short dashed). faster than the model even if the precipitation during the period was identical. Over a long period, adding 1.27 mm (0.05 in.) to the model provides a useful cor- greater than 0.9 in., the Eta-10 has approximately 10% rection, since the average amount of precipitation not greater threat scores than the 12-km MM5. For all recorded by the COOP rain gauges for a given time thresholds, the 36-km MM5 has substantially lower eq- period is 1.27 mm (0.05 in.). uitable threat scores than the 12-km MM5 or the 10- Because the real-time MM5 forecasts were initialized km Eta-10, especially for thresholds greater than 1 in. with a cold start (no initial hydrometeors), it takes 12± As noted by Mesinger (1996), the equitable threat 18 h on average for the model precipitation to spin up. score can be a misleading at higher thresholds because At the lightest precipitation threshold (2.54 mm in 6 h), of the in¯uence of model bias. For example, at the high- equitable threat scores suggest that the model is most er thresholds the skill of a random forecast is typically accurate at hour 18; however, for the higher thresholds negligible [E in Eq. (4) approaches zero]; therefore, a (Ͼ7.62 mm in 6 h) the model is most accurate during higher equitable threat score can result simply because the 6±12-h forecast period. an increase in the bias score translates into a greater There is a noticeable improvement in bias, equitable number of hits [H in Eq. 4]. In order to obtain a more threat, and rms error scores when the horizontal reso- reliable measure of model error at high thresholds, rms lution is decreased from 36 to 12 km. However, ex- errors were calculated for the various observed thresh- amination of the bias and equitable threat scores across olds (Fig. 22). For thresholds less than 0.5 in., the Eta- Washington and Oregon revealed that the 12-km MM5 10 has substantially lower rms errors than either MM5 tends to generate too much precipitation along the steep resolution. Therefore, even though the bias scores for windward slopes and not enough precipitation in the lee the Eta-10 are greater than the MM5 at lower thresholds of major barriers. In the latter regions, the 36-km grid and the equitable threat scores are fairly comparable, may actually be providing a more skillful forecast. the rms errors suggest that the Eta-10 has smaller quan- Eta-10 precipitation was compared directly with the titative errors than the 12- or 36-km MM5 for lighter 36- and 12-km MM5 ®elds using the same forecast pe- precipitation amounts. On the other hand, the signi®cant riod and observation locations. The Eta-10 tends to ov- overprediction problem in the Eta-10 for the higher erpredict precipitation along the lower windward slopes thresholds along the lower windward slope of the Cas- with bias scores Ͼ200% in many locations. A majority cades (cf. Fig. 20) results in large rms errors, which are of this overprediction in the Eta-10 was found to occur much worse than those of the MM5. For higher thresh- between hours 9 and 21. During this period the 6-h bias olds, equitable threat scores and rms errors are greater scores for the moderate to high thresholds increases to in the Eta-10 compared to the MM5, which illustrates over 1.4 and the equitable threat scores drop slightly; the danger in using the equitable threat score as a mea- subsequently the bias scores relax to around 1.0 by hour sure of skill for such high thresholds. 33. As in the MM5, the Eta-10 tends to overforecast The rms errors for thresholds less than 0.5 in. are rainshadowing downwind of major orographic barriers, fairly similar for the 36- and 12-km MM5 resolutions. especially for the high precipitation thresholds. Overall, For thresholds greater than 0.5 in., the rms error scores the Eta-10 has better rms errors than the 12- and 36-

Unauthenticated | Downloaded 09/23/21 10:55 PM UTC APRIL 1999 COLLE ET AL. 153 km MM5 for the lower thresholds, while the MM5 does Model. Preprints, 12th Conf. on Numerical Weather Prediction, signi®cantly better for the higher thresholds (Ͼ2.54 cm Phoenix, AZ, Amer. Meteor. Soc., 217±218. Black, T. L., 1994: The new NMC mesoscale Eta Model: Description in 18 h). and forecast examples. Wea. Forecasting, 9, 265±278. The overprediction of precipitation along the lower , M. Baldwin, G. DiMego, and E. Rogers, 1998: Results from windward slopes in the Eta-10 may be attributed to sev- daily forecasts of the NCEP Eta-10 Model over the western eral problems. First, the microphysical scheme used in United States. Preprints, 12th Conf. on Numerical Weather Pre- the Eta-10 does not include horizontal advection of rain diction, Phoenix, AZ, Amer. Meteor. Soc., 246±247. Bonner, W. B., 1989: NMC overview: Recent progress and future or snow. Under moderate to strong ¯ow conditions, plans. Wea. Forecasting, 4, 275±285. snow can advect 50±100 km downwind from its source Bruintjes, R. T., T. L. Clark, and W. D. Hall, 1994: Interactions be- region (Rauber 1992). Neglecting the advection of snow tween topographic air¯ow and cloud/precipitation development may be reasonable for coarser grid resolution (Ͼ50 km), during the passage of a winter storm in Arizona. J. Atmos. Sci., but it severely handicaps a mesoscale model such as the 51, 48±67. Colle, B. A., and C. F. Mass, 1996: An observational and modeling Eta-10, which can resolve mesoscale variations of oro- study of the interaction of low-level southwesterly ¯ow with the graphic precipitation. As a result, this scheme behaves Olympic Mountains during COAST IOP 4. Mon. Wea. Rev., 124, like a warm rain (dump bucket) scheme, where the pre- 2152±2175. cipitation falls out immediately after being generated Cressman, G., 1959: An operational objective analysis system. Mon. over the windward slopes. Second, the Eta-10 produces Wea. Rev., 87, 367±374. Dingman, S. L., 1994: Physical Hydrology. Prentice-Hall, 575 pp. excessive ¯ow blocking upwind of major barriers (Z. Dudhia, J., 1989: Numerical study of convection observed during the JanjicÂ 1998, personal communication), which can result Winter Monsoon Experiment using a mesoscale two-dimensional in too much upward motion and precipitation well up- model. J. Atmos. Sci., 46, 3077±3107. wind of the orographic crest. This excessive blocking Ferguson, S. A., S. Breyfogle, M. B. Moore, R. Marriott, and D. Judd, is related to inability of the Eta Model to properly gen- 1997: NWAC Mountain Weather. Report to the USDA Natural Resources Conservation Service, Climate Data Access Facility, erate mountain waves (McDonald et al. 1998). Finally, 40 pp. [Available from Forestry Sciences Laboratory, 4043 Roo- the cumulus parameterization used in the Eta (Betts± sevelt Way NE, Seattle, WA 98195.] Miller±Janjic scheme) tends not to be triggered over the Fischer and Porter Co., 1967: Instruction bulletin for series 35-1558 high terrain (Baldwin and Black 1998). and 35-1559 precipitation gauge recorder. Fischer and Porter Co. To a lesser degree than the Eta-10, the MM5 also had Publ. 16237, Fischer and Porter, Co., 19 pp. [Available from overprediction problems along the windward slopes and Bailey-Fischer and Porter Co., 125 East County Line Rd., War- minster, PA 18974.] underprediction in the lee of barriers. This suggests that Gaudet, B., and W. R. Cotton, 1998: Statistical characteristics of a the MM5 may be overpredicting the leeside subsidence real-time precipitation forecasting model. Wea. Forecasting, 13, and/or the microphysical scheme is not generating or 966±982. maintaining enough ice aloft as it gets advected over Groisman, P. V., and D. R. Legates, 1994: The accuracy of United the barrier. Overall, this study points toward the need States precipitation data. Bull. Amer. Meteor. Soc., 75, 215±227. Guo, Y.-R., and S. Chen, 1993: Terrain and land use for the ®fth- to verify and make improvements in model microphys- generation Penn State/NCAR Mesoscale Model (MM5): Pro- ical schemes. In a subsequent paper, MM5 precipitation gram TERRAIN. NCAR Tech. Note NCAR/TN-397ϩSTR, 113 veri®cation results will be presented down to 4-km res- pp. [Available from National Center for Atmospheric Research, olution for the 1997/98 winter season. In addition, sim- P.O. Box 3000, Boulder, CO 80307.] ulations down to 1.3-km resolution as well as micro- Halpert, M. S., and G. D. Bell, 1997: Climate assessment for 1996. physical sensitivity results will be shown for the ¯ood- Bull. Amer. Meteor. Soc., 78, S1±S49. Hoke, J. E., N. A. Phillips, G. J. DiMego, J. J. Tuccillo, and J. G. ing event of 5±9 February 1996 over the Paci®c North- Sela, 1989: The regional analysis and forecast system of the west. National Meteorological Center. Wea. Forecasting, 4, 323±334. Hsie, E.-Y., R. A. Anthes, and D. Keyser, 1984: Numerical simulation Acknowledgments. This research was supported by of frontogenesis in a moist atmosphere. J. Atmos. Sci., 41, 2581± the Of®ce of Naval Research (Grants N00014-94-1- 2594. 0098 and N00014-98-1-0193) and NSF/NOAA/ JanjicÂ, Z., 1994: The step-mountain Eta coordinate model: Further USWRP Grant ATM9612876. Use of the MM5 was developments of the convection, viscous sublayer, and turbulence closure schemes. Mon. Wea. Rev., 122, 927±945. made possible by the Microscale and Mesoscale Me- Junker, N. W., J. E., Hoke, B. E. Sullivan, K. F.Brill, and F.J. Hughes, teorological Division of NCAR, which is supported by 1992: Seasonal and geographic variations in quantitative pre- the National Science Foundation. Special thanks to Brett cipitation prediction by NMC's Nested-Grid model and medium- McDonald at the University of Utah for providing us range forecast model. Wea. Forecasting, 7, 410±429. with the Eta-10 precipitation forecasts, Dr. Brad Colman Kain, J. S., and J. M. Fritsch, 1990: A one-dimensional entraining/ at the NWSÐSeattle for obtaining the COOP precipi- detraining plume model and its application in convective parameterization. J. Atmos. Sci., 47, 2784±2802. tation data, and Dave Ovens at the University of Wash- Klemp, J. B., and D. R. Durran, 1983: An upper boundary condition ington for writing some of the Eta-10 veri®cation code. permitting internal gravity wave radiation in numerical mesoscale models. Mon. Wea. Rev., 111, 430±444. Martin, G., 1996: A dramatic example of the importance of detailed REFERENCES model terrain in producing accurate quantitative precipitation Baldwin, M. E., and T. L. Black, 1998: Postfrontal precipitation fore- forecasts for southern California. No. 96-07, Western Regional casting experiments in the western U.S. with NCEP's Eta-10 Technical Attachment, National Weather Service, 4 pp. [Avail-

Unauthenticated | Downloaded 09/23/21 10:55 PM UTC 154 WEATHER AND FORECASTING VOLUME 14

able from National Weather Service, Western Region, Salt Lake Rauber, R. M., 1992: Microphysical structure and evolution of a City, UT 84147.] central Sierra Nevada shallow orographic cloud system. J. Appl. Mass, C. F., and G. K. Ferber, 1990: Surface pressure perturbations Meteor., 31, 3±24. produced by an isolated mesoscale topographic barrier. Part I: Rogers, E., D. G. Deaven, and G. J. DiMego, 1995: The regional General characteristics and dynamics. Mon. Wea. Rev., 118, analysis system for the operational ``early'' Eta Model: Original 2579±2596. 80-km con®guration and recent changes. Wea. Forecasting, 10, , and Y.-H. Kuo, 1998: Regional real-time numerical weather 810±825. prediction: Current status and future potential. Bull. Amer. Me- , T. L. Black, D. Deaven, G. J. Dimego, Q. Zhao, M. Baldwin, teor. Soc., 79, 253±263. N. Junker, and Y. Lin, 1996: Changes to the operational ``early'' McDonald, B. E., and J. D. Horel, 1996: Precipitation forecast skill Eta analysis/forecast system at the National Centers for Envi- over California of the National Centers for Environmental Pre- ronmental Prediction. Wea. Forecasting, 11, 391±413. diction models. Preprints, 15th Conf. on Weather Analysis and Sinclair, M. R., D. S. Wratt, R. D. Henderson, and W. R. Gray, 1997: Forecasting, Norfolk, VA, Amer. Meteor. Soc., J131±J132. Factors affecting the distribution and spillover of precipitation , and , 1998: Evaluation of precipitation forecasts from the in the Southern Alps of New ZealandÐA case study. J. Appl. NCEP's 10-km nest mesoscale Eta Model. Preprints, 12th Conf. Meteor., 36, 428±442. on Numerical Weather Prediction, Phoenix, AZ, Amer. Meteor. Staudenmaier, M., 1996: The initialization procedure in the meso eta model. No. 96-30, Western Regional Technical Attachment, Na- Soc., J27±J30. tional Weather Service, 5 pp. [Available from National Weather , , C. J. Stiff, and W. J. Steenburgh, 1998: Observations Service, Western Region, Salt Lake City, UT 84147.] and simulations of three downslope wind events over the north- , and J. Mittelstadt, 1998: Results of the western region evalu- ern Wasatch Mountains. Preprints, 12th Conf. on Numerical ation of the Eta-10 Model. Preprints, 12th Conf. on Numerical Weather Prediction, Phoenix, AZ, Amer. Meteor. Soc., 62±64. Weather Prediction, Phoenix, AZ, Amer. Meteor. Soc., 131±134. McMillan, G. D., 1981: SNOTEL: A management tool for the future. Wilks, D., 1995: Statistical Methods in the Atmospheric Sciences: An Proc. Western Snow Conf., 49th Annual Meeting, St. George, Introduction. Academic Press, 467 pp. UT, Western Snow Conference, 116±119. Zhang, D., and R. A. Anthes, 1982: A high-resolution model of the Mesinger, F., 1996: Improvements in quantitative precipitation fore- planetary boundary layer-sensitivity test and comparisons with casts with the Eta regional model at the National Centers for SESAME-79 data. J. Appl. Meteor., 21, 1594±1609. Environmental Prediction: The 48-km upgrade. Bull. Amer. Me- Zhao, Q., and F. H. Carr, 1997: A prognostic cloud scheme for op- teor. Soc., 77, 2637±2649. erational NWP models. Mon. Wea. Rev., 125, 1931±1953.

Unauthenticated | Downloaded 09/23/21 10:55 PM UTC