A Case Study of 2010 Hurricane Karl to Evaluate the Precipitation Forecasts of the Post Updated 2010 Global Ensemble Forecast System Lindsey Day

Florida State University Libraries

Honors Theses The Division of Undergraduate Studies

2011 A Case Study of 2010 Hurricane Karl to Evaluate the Precipitation Forecasts of the Post Updated 2010 Global Ensemble Forecast System Lindsey Day

Follow this and additional works at the FSU Digital Library. For more information, please contact [email protected] Abstract

This research project seeks to evaluate the rainfall forecast accuracy of the GEFS

(Global Ensemble Forecast System) following the 2010 model update, analyzing the Atlantic

September 2010 Hurricane Karl as a case study. Archived forecasted accumulated precipitation amounts from the GEFS are compared with archived CMORPH (CPC Morphing Technique) precipitation estimates. A sequence of 15 six hour forecasts from 1200 UTC 14 September 2010 to 1200 UTC 18 September 2010 are graphically displayed using GrADS (Grid Analysis and

Display System). Four images were produced for each of the 15 six-hour time frames: A)

CMORPH 6-hour accumulated precipitation, B) GEFS 6-hour accumulated precipitation, C)

Difference in GEFS and CMORPH 6-hour accumulated precipitation (GEFS-CMORPH), and D)

Overlay of outlined areas of medium (≥0.25 inches) and heavy (≥1.0 inches) precipitation for the

GEFS and CMORPH. Several errors in the precipitation forecast of the GEFS model post- update were consistent with errors found in the GEFS model prior to its 2010 update from previous studies. For example, the GEFS forecasted the size of the hurricane to be too large for all evaluated times. Although the six hour GEFS track forecast was closely aligned with that of the verified track, major differences between the forecast and verification occurred as the storm approached its second landfall in Mexico. This study highlights that although the model may accurately depict the storm track, other model biases such as a lower resolution, difficulty depicting the atmosphere near land/sea boundaries and over complex terrain—may significantly impact the precipitation forecast.

Keywords: GFS, CMORPH, precipitation forecast

THE FLORIDA STATE UNIVERSITY

COLLEGE OF ARTS AND SCIENCES

A CASE STUDY OF 2010 HURRICANE KARL TO EVALUATE THE PRECIPITATION FORECASTS OF THE POST UPDATED 2010 GLOBAL ENSEMBLE FORECAST SYSTEM (GEFS)

LINDSEY R. DAY

A Thesis submitted to the

Department of Earth, Ocean, and Atmospheric Sciences

in partial fulfillment of the

requirements for graduation with

Honors in the Major

Degree Awarded:

Spring Semester, 2011

The members of the Defense Committee approve the thesis of Lindsey Day defended on 18 April 2011.

______

Dr. Jon E. Ahlquist

Thesis Director

______

Dr. Warren Nichols

Outside Committee Member

______

Dr. Henry Fuelberg

Committee Member

ACKNOWLEDGEMENTS

I would like to thank all of my committee members—Dr. Jon Ahlquist, Dr. Henry

Fuelberg, and Dr. Warren Nichols—for their support throughout this process. Under their guidance, I found this Honors in the Major Research Experiment to be a challenging, yet rewarding and extremely beneficial experience.

Dr. Fuelberg's willingness to meet with me from time to time and discuss my thesis was a huge help. His assistance in suggesting methods to go about my research project greatly helped to keep my work on track.

In particular, I would like to thank Dr. Ahlquist for the countless hours of his time he has volunteered to provide me with the programming and meteorological knowledge necessary to complete this project. His effort and encouragement to try new solutions when something did not go according to plan was very much appreciated.

Additionally, I would like to thank another professor outside of my committee, Dr. Hart, for his programming assistance with GrADS. He was always willing to answer any questions I had, and to help me to debug programming errors as they arose.

I would like to acknowledge the FSU Honors Office for granting a Bess H. Ward Honors

Thesis Award which enabled the members of Dr. Ahlquist's laboratory to purchase a new computer for research.

Last but not least, I would not have been able to juggle this project among the multiple activities I am involved in if it were not for my loving family and friends who always let me know they are proud of me, and who always encourage me to do my best.

Table of Contents

List of Tables ...... 6

List of Figures ...... 7

1. Introduction ...... 8

2. Analysis Tools ...... 9

2.1 The GEFS Numerical Forecast Model ...... 9 2.2 Evaluation of the GFS from Previous Studies ...... 10 2.3 CMORPH: A Precipitation Estimate Product ...... 12 2.4 Evaluation of CMORPH from Previous Studies ...... 12 3. Data Sets ...... 15

4. Methods ...... 16

5. Results ...... 18

5.1 Forecasted versus Analyzed Storm Center Position ...... 18

5.2 Results from Image Sets A and B ...... 20 5.3 Results from Image Set C ...... 23 5.4 Results from Image Set D ...... 26 5.5 Sources of Error ...... 27 6: Conclusion ...... 28

References ...... 30

List of Tables

Table 5.1...... Forecasted and verified storm center positions are valid for the third hour of each six hour forecast. The positive x direction is taken to be West, and the positive y direction is taken to be North.

Table 5.2...... (From set A and B Images) Maximum amounts of six hour accumulated precipitation for any point within 10 ° latitude or longitude of the storm center for the GEFS and CMORPH

Table 5.3...... (From set A and B Images) Ratio of areal coverage (GFS:CMORPH) of six-hour accumulated precipitation within 10° latitude and longitude of the verified storm center

Table 5.4...... (From set C Images) The CMORPH 6-hour precipitation accumulation fields were subtracted from the GEFS six-hour precipitation accumulation fields and the maximum value of over-predicted and under-predicted rainfall in inches was noted.

List of Figures

Figure 5.1...... (From set A and B Images) Graphic A) shows the GEFS forecasted a point maximum accumulated precipitation amount of 0.8 inches while graphic B) shows the CMORPH estimate to be 1.6 inches. This occurred when Karl was a tropical storm.

Figure 5.2...... (From set A and B Images) These images show the largest forecast error for both areal overage ad poit aiu auulated preipitatio aout. This ourred at Karl’s peak intensity, just before landfall.

Figure 5.3...... (From set A and B Images) Graph of maximum amounts of six hour accumulated precipitation for any point within 10 ° latitude or longitude of the storm center for the GEFS and CMORPH.

Figure 5.4...... (From set C Images) Graph of maximum over- and under-forecasted precipitation amounts calculated by subtracting the CMORPH rainfall estimate from the GEFS rainfall forecast field.

Figure 5.5...... (From set C Images) Difference in GEFS and CMORPH 6 hour accumulated precipitation amounts. This image shows the time at which there was the most disagreement between the forecast and estimate.

Figure 5.6...... (From set C Images) Difference in GEFS and CMORPH 6 hour accumulated precipitation amounts. This image shows a time at which there was a smaller amount of disagreement between the forecast and estimate.

Figure 5.7...... (From set D Images) Graphic A) shows the under-forecasted areal coverage of precipitation by the GEFS while graphic B) shows the over-forecasted areal coverage of precipitation by the GEFS.

1. Introduction Hurricanes are particularly important because of their capability to substantially impact crops, vegetation, property, and travel conditions. Intense tropical systems can produce rainfall totals around 15 inches; it is common for coastal locations to receive a month’s worth of rain from a single major hurricane. These facts further support the useful implications of analyzing the GEFS’s error in forecasting rainfall.

Hurricane Karl was used as a case study to evaluate the GEFS’s accuracy in forecasting precipitation post-2010 update. Karl was a major 2010 Atlantic hurricane that made landfall twice: first in at 1245 UTC September 15 on the Yucatan Peninsula as a tropical storm and second at 1645 UTC September 17 in Veracruz, Mexico as a major Hurricane. At its peak intensity at a Category 3 storm its maximum sustained winds were at 120 mph and a minimum central pressure was 967 millibars (occurring at 1500 UTC September 17). Hurricane Karl was a significant storm because it was the first time that a major Hurricane was recorded in the Bay of

Campeche. Furthermore, its heavy rainfall amounts brought floods and landslides to Mexico resulting in 22 confirmed deaths and $5.6 billion in damage—it was the most destructive

Hurricane that Veracruz is known to have witnessed (Por Esto 2011). Up to 12 inches of total accumulated rainfall amounts were recorded along the coast of Mexico (NHC 2011).

2. Analysis Tools

2.1 The GEFS Numerical Forecast Model

The Global Forecast System (GFS) is a synoptic scale, spectral numerical weather forecasting model. It predicts temperature, pressure, humidity, precipitation, and winds by using the following equations: vorticity, hydrostatic, thermodynamic, mass continuity, and conservation. The model computes forecasts four times each day every 6 hours starting with

00Z. The precipitation forecast is given as a 6-hour rainfall accumulation. It takes 2.75 hours for the weather data variables to be received and after the data is inputted it takes up to eight minutes to compute the forecast. The GFS uses a slightly different method to compute the first half of the forecast period (days 1-8) compared to the second half (days 8-16). The first forecast period computes a forecast at a higher resolution and therefore with more detail compared to the second forecast period.

As time increases, so does forecast error. Forecasts generally become unreliable past days 7-

10. The vertical resolution is given by weather variable data from 64 vertical levels ranging from

1000 hectoPascals (hPa) to a level above 100 hPa. A spectral triangular horizontal resolution divides the Earth’s surface into 574 triangular waves (NCEP 2010b).

The Global Ensemble Forecast System (GEFS) is derived from the original GFS model.

With the notion that the initial weather data that is input into the model is imperfect, slight perturbations in the initial atmospheric conditions are made to the original GFS forecast. These model runs combined with that of the original GFS gives a total of 21 different ensemble members. Each of the 21 ensemble members represents an individual forecast; they indicate the range of possibilities that can occur for a given forecast. The GEFS model is run with a slightly coarser resolution of T170.

2.2 Evaluation of the GFS from Previous Studies

Numerical forecast models, such as the GEFS, commonly produce error when forecasting for rainfall. Forecast model upgrades are implemented once or twice a year. The most recent upgrade to the GEFS numerical model was completed in July of 2010. Therefore this project has useful implications because it is among the first to assess the GEFS following its 2010 upgrade.

Several studies have been done to evaluate the accuracy of the GFS precipitation forecast prior to the most recent update. According to National Centers for Environmental Prediction

(NCEP), the GFS tended to overestimate rainfall amounts for the center of a storm, and produce error in forecasting hurricane track and intensity prior to its 2010 update (NCEP 2010d).

A study by Werth et al. (2010) found that error correlated positively in areas where there was steep topography and where land and bodies of water with contrasting characteristics were in close proximity. The actual elevation of a station located in a steep topographic area will often differ from the elevation interpolated by the GFS which results in forecast error (Werth et al.

2010).

A study by Dravitzki et al. (2010) analyzed the skill of the GFS in predicting rainfall in

New Zealand prior to the model’s 2010 upgrade. Two years of GFS rainfall forecast data were compared against surface observations. It was found that the timing and location of such heavy precipitation events showed to be less reliable with increasing time. This study notes that a common primary flaw with global models, such as the GFS, is their coarser resolution. This results in limitations in the accurate spatial depiction of rainfall (Dravitzki et al. 2010). The high bias score calculated for precipitation events between 1 and 15 mm indicates that the GFS over-

10 predicts light rainfall. The low bias score calculated for precipitation events of greater than 15 mm indicates that the GFS under-predicts heavy precipitation.

This statistical analysis showed that the GFS has limited skill in predicting heavy rainfall events (of greater than 15 mm), especially at longer lead-times. In general, the skill of the model steadily decreases as the forecast hour increases (Dravitzki et al. 2010). This error is minimized in this research project by analyzing a series of short term forecasts.

Werth et al. (2010) also confirms that an accepted source of error is the large model resolution. The GFS often cannot depict mesoscale or local atmospheric processes because of its coarser resolution. These smaller features sometimes play a role in the synoptic scale circulation, thereby implying a deficiency in the GFS’s forecast ability when this situation arises (Werth et al.

2010).

One evaluation of the GFS post 2010 update was performed by NCEP. It was found that the 2010 GEFS model upgrades have led to overall improvements in forecast accuracy.

According to three years of tests (from 2008-2010), NCEP notes improvement in quantitative precipitation forecasts (QPF); the small, localized areas of over-estimated forecasted precipitation (―bulls-eyes‖ of precipitation) were reduced. Furthermore, the upgraded GFS showed a 10% track improvement and 30% intensity improvement for Atlantic hurricane day five forecasts (NCEP 2010d). Alongside NCEP’s evaluation, this project is among the first to assess the degree of accuracy of this newly updated forecast model, in an attempt to pinpoint sources of error in GFS rainfall forecasting.

2.3 CMORPH: A Precipitation Estimate Product

CMORPH estimates previously fallen rainfall amounts based on both microwave satellite and infrared satellite data (Joyce et al. 2004, 488-491). First, a passive satellite receives microwaves to estimate the rainfall rate. The intensity and shape of the rainfall will undoubtedly change between successive passive microwave (PMW) derived precipitation estimates, and this must be accounted for. The CMORPH technique analyzes how the size and intensity of the rain evolves from successive microwave receptions. Monitoring of cloud position through an infrared (IR) satellite identifies the rainfall’s pattern of movement over time and allows for the computation of the velocity vector of rain-producing clouds. This allows for interpolation of rainfall estimates. Therefore, the microwave derived precipitation estimates can be propagated forward in time, providing temporally continuous rainfall total estimations for a particular location (Joyce et al. 2004).

2.4 Evaluation of CMORPH from Previous Studies An analysis of the accuracy of the GFS numerical model rainfall forecast can be done through comparison with the Climate Prediction Center’s (CPC’s) CMORPH (CPC morphing technique). In a study by Joyce et al. (2004), CMORPH was compared to five other indirect rainfall estimation techniques: MWCOMB, radar, and three microwave-IR blended techniques.

Rain gauges were used as a control against which these six techniques were compared against to supply the validation data. The study occurred over the course of five months from 15 June through 15 November 2003. It was found that radar and CMORPH performed better than the other four techniques overall. The CMORPH technique had higher daily correlation values that ranged from 0.65 to 0.74, while the other three microwave-IR blended techniques had correlation

12 values that ranged from 0.10-0.13. When comparing CMORPH with MWCOMB it was found that CMORPH was significantly more accurate in five out of six areas. Radar outperformed

CMORPH in skill, however CMORPH proved more capable of detecting rainfall and also had a lower bias score. CMORPH’s greater accuracy was most evident in the late summer to fall period of the study. Therefore, from this study it was concluded that CMORPH and radar are more accurate in estimating rainfall than MWCOMB, and the three other microwave-IR blended techniques (Joyce et al. 2004). Thus, for this study CMORPH was chosen as a valid product for evaluating the GEFS’s precipitation forecast.

CMORPH’s indirect measurement of the amount of rainfall does contain errors.

Therefore, the questions arise with regards to the validity of using CMORPH as a verification of accumulated rainfall. Several studies have been done that indicate that CMORPH provides one of the best estimates of the actual rainfall. Differences among CMORPH and other rainfall estimation products depend upon the product’s spatial and temporal resolution, the method in which the passive microwave and infrared satellite data are inputted into various algorithms, and the use of statistical adjustments in the final computation (such as those based on rain gauge observations).

In a study by Dinku et. al (2009), several different satellite-based rainfall estimation techniques were evaluated over complex terrain and mountainous regions in Colombia, South

America. These seven products were compared to land-based rain gauges. It was found that

CMORPH and a version of the Global Satellite Mapping of Precipitation (GSMaP) performed the best out of the seven satellite based precipitation estimate techniques. Additionally, the accuracy of the satellite based rainfall estimates were compared among different types of topography such as mountains, plains, and valleys. Colombia was divided into lowlands (areas 13 less than 750 meters) and highlands (areas greater than 750 meters). Satellite-based precipitation estimates such as CMORPH showed greater performance over lower elevations compared to higher elevations based on calculations of rainfall detection and bias. CMORPH, along with the other satellite-derived precipitation estimate products, was found to underestimate rainfall in mountainous regions. Despite these findings, it was also found that CMORPH still performed with skill over higher elevations. It was found that the highlands had larger correlation coefficients and smaller mean absolute error (Dinku et al. 2009).

The implications from Dinku et al.’s study are important for this study, as several mountains extend through Mexico where Hurricane Karl made landfall. One mountain range in particular, Pico de Orizaba, is the highest in Mexico with an elevation of over 5000 meters— comparable to the high elevations over which CMORPH was assessed in the study by Dinku et al.

(2009).

3. Data Sets

NCEP stores GEFS’S forecasts in gridded binary (GRIB) format on a computer as a compact way to store the voluminous data. Control files in WGrib(2) were utilized to read this

GRIB data into a program called GrADS (Grid Analysis and Display System). Each of the 21

GEFS ensemble members contains a series of six-hourly precipitation forecasts in inches. The model resolution is 1°x1°.

The CMORPH control file was also downloaded for analysis. This contains a series of three-hourly precipitation estimates in millimeters per hour. The model resolution is 0.25°x0.25°

Both GEFS and CMORPH control files were read by GrADS. This allowed for the production of graphical representations of the GEFS rainfall forecasts and CMORPH rainfall analyses for Hurricane Karl.

The actual storm center position at the third hour of each six-hour forecast was obtained from the Hurricane Karl Advisory Archive produced by the National Hurricane Center (NHC).

4. Methods

Archived forecasts of accumulated precipitation amounts from the GEFS are compared with archived CMORPH precipitation estimates. CMORPH is taken to be the verification in this study.

A series of 15 six-hour forecasts from 1800 UTC September 14 to 1200 UTC September

18 was analyzed. This short-range forecast implies that the observed error is a result of error in the model physics rather than a poorly evolved forecast.

The average of the 21 ensemble members’ accumulated precipitation was calculated. It should be noted that a large spread in the ensemble forecast would lead to a more even distribution of rainfall amounts when taking the ensemble average. For instance, a larger spread would produce lower calculated maxima and higher calculated minima amounts of accumulated precipitation. Additionally, a spread in the ensemble forecast would result in a spread in the areal coverage of the precipitation when mapping the GEFS precipitation field. However, the individual ensemble members were selected at random at and it was noted that there was not much variability among individual ensemble members. For example, when one member over- forecasted the areal coverage of precipitation for the storm, the others over-forecasted by approximately the same amount. In other words, the over-estimation of areal coverage that was observed was not solely a result of taking the ensemble average. Because each of the 15 time frames only looks at a six hour forecast, the forecast spread is minimal. Thus, it was decided that taking the average of all 21 ensemble members would be an acceptable means of assessing the areal coverage of the GEFS.

The ensemble mean forecasted center position of Hurricane Karl was calculated by averaging the forecasted center at the beginning and end of the six hour forecast. This was compared to the National Hurricane Center’s (NHC) verified storm center at hour three of each of the 15 six-hour time frames. The verified storm center is placed on each graphic with an ―X‖.

It was noted that the difference among each of the 20 ensembles’ forecasted storm center was minimal, therefore, it was decided that taking the average was an acceptable means of comparison.

GrADS was used to produce graphical representations of the rainfall forecasts and analyses. Two three-hourly CMORPH rainfall estimates were added together to match the six- hourly temporal resolution of the GEFS. In order to graphically display the difference between the forecast and analysis on the same grid, CMORPH was regrided to match the 1°x1° resolution of the GEFS. Each grid was centered around the current storm center position; the domain and range of each was set to 10° latitude by 10° longitude. A sequence of 15 six hour rainfall accumulations from 1200 UTC 14 September 2010 to 1200 UTC 18 September 2010 are graphically displayed using GrADS (Grid Analysis and Display System). Four images were produced for each of the 15 six-hour time frames: A) CMORPH 6-hour accumulated precipitation, B) GEFS 6-hour accumulated precipitation, C) Difference in GEFS and CMORPH

6-hour accumulated precipitation (GEFS-CMORPH), and D) Overlay of outlined areas of light/medium (≥0.25 inches) and heavy (≥1.0 inches) precipitation for GEFS and CMORPH.

From both sets of 15 images in A and B, the forecasted and verified maxima of accumulated precipitation and the areal coverage of precipitation were compared. From images in C, the amount of maximum over-forecasted and under-forecasted precipitation amounts were calculated.

Images from part D were used to qualitatively assess the differences between the GEFS and

CMORPH in areal coverage of light/medium and heavy precipitation.

5. Results 5.1 Forecasted versus Analyzed Storm Center Position

According to a study by Buckingham et al. (2010) that assessed the GFS model’s accuracy prior to its 2010 update, the GFS model’s average forecasted track tended to lag behind that of the observed track. In this case study of Hurricane Karl, it was found that there was only a slight difference between the GEFS’s six-hour forecasted center and the verified storm center

(see Table 5.1). The forecast and verification differed by 0.5° latitude/longitude or less for the observed time. Because the precipitation is mapped on a grid with a 1°x1° resolution, the error was not large enough to be represented in this research analysis. Thus, although a slight track error does exist in the GEFS forecast at hour six, it is not significant enough to allow for an analysis of the forecasted precipitation by removing track error. Despite such small differences in forecasted and observed track, there were notable differences in the precipitation distribution among the rainfall forecast and analysis.

The largest differences between forecasted and observed storm center position were found from 00Z-18Z 9/17 during the 10th, 11th, and 12th time frames where the GEFS forecasted the storm center 0.3° West, 0.3° West, and 0.5° East of the verified storm center respectively.

This occurred around the time of landfall in Veracruz, indicating that the GEFS had reduced accuracy in predicting the central storm location when the system moved in close proximity to land. In comparison with image results from parts A-D, the largest difference in forecast and

18 analysis occurred during this same time frame. Although the GEFS’s forecasted storm center position did not differ much from the verified storm center, the maximum error occurred within a few hours before and after landfall.

Time Frame: NHC NHC GEFS GEFS Difference in Difference in Verified Verified Ensemble Ensemble Storm Center Storm Center Date, Time Storm Storm Average Average Latitude (°) Longitude (°) Center Center Forecasted Forecasted (Forecast - (Forecast - Latitude Longitude Storm Storm Verification) Verification) (°N) (°W) Center Center Latitude Longitude (°N) (°W) 1) 9/14-15, 18Z-00Z 18.3 84.2 18.1 84.3 -0.2 +0.1

2) 9/15, 00Z-06Z 18.6 85.5 18.1 85.5 -0.5 0

3) 9/15, 06Z-12Z 18.5 86.7 18.3 86.5 -0.2 -0.2

4) 9/15, 12Z-18Z 18.6 88.2 18.7 88.0 +0.1 -0.2

5) 9/15-16, 18Z-00Z 19.0 89.4 19.0 89.3 0 -0.1

6) 9/16, 00Z-06Z 19.4 90.7 19.2 90.4 -0.2 -0.3

7) 9/16, 06Z-12Z 19.8 91.6 19.8 91.8 0 +0.2

8) 9/16, 12Z-18Z 19.7 92.8 19.8 92.7 +0.1 -0.1

9) 9/16-17, 18Z-00Z 19.6 93.7 19.9 93.8 +0.3 +0.1

10) 9/17, 00Z-06Z 19.7 94.5 19.7 94.2 0 -0.3

11) 9/17, 06Z-12Z 19.7 95.3 19.8 95.0 +0.1 -0.3

12) 9/17, 12Z-18Z 19.4 95.9 19.5 96.4 +0.1 +0.5

13) 9/17-18, 18Z-00Z 19.1 96.6 19.1 96.6 0 0

14) 9/18, 00Z-06Z 18.6 97.4 18.4 96.9 -0.2 -0.5

15) 9/18, 06Z-12Z 18.5 97.6 X* X* - - - -

Table 5.1. Forecasted and verified storm center positions are valid for the third hour of each six hour forecast. The positive x direction is taken to be West, and the positive y direction is taken to be North. *An X indicates no identifiable storm center.

5.2 Results from Image Sets A and B

The values of the maximum amounts of six hour accumulated precipitation for any point within the domain for the GEFS and CMORPH were compared for each of the 15 time frames.

This was used to assess the accuracy of the GEFS’s rainfall intensity forecast. The amount of under-prediction of the point maximum rainfall accumulation for each six hour GEFS forecast from 1800UTC September 14 to 0000UTC ranged from 27% to 50% (see Table 5.2). The largest under-estimation of rainfall occurred in the 5th time frame from 1800UTC September 15 to

0000UTC September 16 where the GEFS predicted the point maximum accumulated rainfall to be 0.8 inches, while the CMORPH accumulated rainfall estimate was twice that amount: 1.6 inches (see Figure 5.1). At this time, Karl was a tropical storm with maximum sustained winds of 40 knots. The GEFS tended to under-predict the point amount of accumulated precipitation in the beginning and end of Hurricane Karl’s life span. It began to over-predict the maximum precipitation amounts during the middle portion of its life span, from about 16 hours prior to

Karl’s landfall until about 8 hours after landfall. The magnitude of the percent error for over- prediction was greater than that for under-prediction, which may indicate that the GEFS had less accuracy predicting rainfall intensity on either side of landfall, or around the time that the storm reached its peak intensity at 1500 UTC September 17.

To assess the GEFS’s accuracy in predicting the spatial distribution of the rainfall, the forecasted areal coverage of precipitation was compared with that of CMORPH. The individual

CMORPH and GEFS precipitation fields were mapped on separate grids with the same latitude and longitude range so that the areal coverage of precipitation could be calculated and compared using GrADS. It was found that the GEFS consistently over-predicted the areal coverage of precipitation. The results are shown in Table 5.3. The average over-prediction of areal

20 precipitation coverage by the GEFS was 177%. The highest amount of over-prediction occurred from 1200 UTC September 17 to 1800 UTC September 17, when the GEFS had a 330% error and the ratio of rainfall areal coverage (GEFS: CMORPH) was 4.3:1 (see figure 5.2). This occurred at the same time that the GEFS had the largest percent error in forecasting the point maximum amount of accumulated precipitation. These largest forecast errors occurred at the time of the storm’s peak intensity—immediately before Karl made landfall in Mexico.

Time Frame: Date, CMORPH Maximum GEFS Maximum GEFS Percent Error Time Rainfall (inches) Rainfall (inches) 1) 9/14-15, 18Z-00Z 1.1 0.8 -27% 2) 9/15, 00Z-06Z 1.5 0.8 -46% 3) 9/15, 06Z-12Z 1.7 0.9 -47% 4) 9/15, 12Z-18Z 1.1 0.7 -36% 5) 9/15-16, 18Z-00Z 1.6 0.8 -50% 6) 9/16, 00Z-06Z 1.4 0.8 -43% 7) 9/16, 06Z-12Z 1.7 1.0 -41% 8) 9/16, 12Z-18Z 1.4 1.0 -29% 9) 9/16-17, 18Z-00Z 1.6 1.1 -31% 10) 9/17, 00Z-06Z 1.4 1.5 +7% 11) 9/17, 06Z-12Z 0.9 2.2 +144% 12) 9/17, 12Z-18Z 0.6 2.8 +367% 13) 9/17-18, 18Z-00Z 0.6 0.9 +50% 14) 9/18, 00Z-06Z 1.0 0.5 -50% 15) 9/18, 06Z-12Z 1.0 0.8 -19% Table 5.2 (From set A and B Images) Maximum amounts of six hour accumulated precipitation for any point within 10 ° latitude or longitude of the storm center for the GEFS and CMORPH. The highlighted portions show where the over-prediction occurred.

Maximum Amounts of 6-Hour Accumulated Precipitation

Precipitation (inches)

Time

Figure 5.3 Graph of maximum amounts of six hour accumulated precipitation for any point within 10 ° latitude or longitude of the storm center for the GEFS and CMORPH.

Time Frame: Date, Time Ratio of Rainfall GEFS Percent Error Areal Coverage (GEFS:CMORPH) 1) 9/14-15, 18Z-00Z 3.52:1 252% 2) 9/15, 00Z-06Z 2.70:1 170% 3) 9/15, 06Z-12Z 2.23:1 122% 4) 9/15, 12Z-18Z 2.10:1 110% 5) 9/15-16, 18Z-00Z 3.62:1 262% 6) 9/16, 00Z-06Z 2.65:1 165% 7) 9/16, 06Z-12Z 2.33:1 132% 8) 9/16, 12Z-18Z 2.14:1 113% 9) 9/16-17, 18Z-00Z 2.79:1 179% 10) 9/17, 00Z-06Z 2.41:1 141% 11) 9/17, 06Z-12Z 3.28:1 228% 12) 9/17, 12Z-18Z 4.30:1 330% 13) 9/17-18, 18Z-00Z 2.81:1 181% 14) 9/18, 00Z-06Z 2.26:1 185% 15) 9/18, 06Z-12Z 1.91:1 90% Table 5.3 (From set A and B Images) Ratio of areal coverage (GFS:CMORPH) of six-hour accumulated precipitation within 10° latitude and longitude of the verified storm center

5.3 Results from Image Set C

Images produced in set C subtract the GEFS accumulated precipitation forecast from the

CMORPH accumulated precipitation analysis. The maximum over-prediction tended to occur where the verified center of the storm is located. The graphs show a larger magnitude and greater areal coverage for over-prediction of GEFS precipitation compared to under-prediction of

GEFS precipitation (see Table 5.4). The areal coverage of over-prediction by the GEFS was greater than the under-prediction of precipitation for all 15 six-hour forecast sequences.

Additionally, the magnitude of the over-forecasted precipitation amounts was greater than the magnitude of the under-forecasted precipitation amounts for 12 out of the 15 analyzed time frames. This would indicate that the GEFS tended to over-forecast the amount of rainfall for

Karl. This provides an alternative interpretation to the analysis from image sets A and B which

23 found that the GEFS under-forecasted the point maximum precipitation accumulation amounts.

Combining the findings from image sets A and B with image set C suggests that although

CMORPH may display a higher maximum value of accumulated precipitation over water for a single point, the GEFS forecasts a broader range of higher amounts of accumulated precipitation.

It also points out how a reduction in resolution can significantly impact the representation of the precipitation field. For example, image set B displays CMORPH rainfall on a 0.25°x0.25° grid.

However, in order to produce the images displaying the subtraction of the two rainfall fields

(image set C), CMORPH had to be regrided. CMORPH was regrided from a 0.25°x0.25° grid to a 1°x1° grid to match the resolution of the GEFS. Thus, some of the higher QPE amounts shown on the individual CMORPH rainfall graph could not be captured when graphed on the coarser grid. This implies a limitation in the accuracy of precipitation forecasts for coarser resolution models such as the GEFS.

The largest difference between the forecast and estimate occurred near landfall and the storm’s peak intensity. This is shown by the greater magnitude of over- and under-prediction

(see Figure 5.4). For example, the difference in 6 hour accumulation of rain from 0600 UTC

September 17 to 1200 UTC September 17, right before Karl’s landfall at 1645 UTC September

17, was the largest compared to the other times. This maximum amount of over-forecasted rainfall was 1.2 inches and maximum amount of under-forecasted rainfall was 0.9 inches (see

Figure 5.5).

The most agreement between the GEFS and CMORPH occurred during the initial stages of the storm’s life span. For example, from 1200 UTC September 15 to 1800 UTC September 15 the magnitudes of over- and under-forecasted precipitation were 0.6 inches and 0.1 inches respectively (see figure 5.6). 24

Time Frame: Maximum Value of Maximum Value of Date, Time GEFS Over-predicted GEFS Under-predicted Rainfall (inches) Rainfall (inches) 1) 9/14-15, 18Z-00Z 0.5 0.1

2) 9/15, 00Z-06Z 0.7 0.5

3) 9/15, 06Z-12Z 0.9 0

4) 9/15, 12Z-18Z 0.6 0.1

5) 9/15-16, 18Z-00Z 0.8 0.4

6) 9/16, 00Z-06Z 0.5 0.2

7) 9/16, 06Z-12Z 0.8 0.5

8) 9/16, 12Z-18Z 0.9 0.5

9) 9/16-17, 18Z-00Z 0.9 0.1

10) 9/17, 00Z-06Z 0.8 0.9

11) 9/17, 06Z-12Z 1.2 0.9

12) 9/17, 12Z-18Z 1.2 0.3

13) 9/17-18, 18Z-00Z 0.9 0.3

14) 9/18, 00Z-06Z 0.4 0.7

15) 9/18, 06Z-12Z 0.8 0.9

Table 5.4: (From set C Images) The CMORPH 6-hour precipitation accumulation fields were subtracted from the GEFS six-hour precipitation accumulation fields and the maximum value of over-predicted and under-predicted rainfall in inches was noted.

Difference in 6-Hour Accumulated Precipitation: GEFS-CMORPH Precipitation (inches)

5.4 (From set C Images) Graph of maximum over- and under-forecasted precipitation amounts calculated by subtracting the CMORPH rainfall estimate from the GEFS rainfall forecast field.

5.4 Results from Image Set D

Images from set D that compare the areal coverage of light and heavy precipitation between CMORPH and the GEFS show that the GEFS consistently over-predicted the areal coverage of light/moderate precipitation. These findings are consistent with the study by

Dravitzki et al. (2010) which found that the pre-updated GFS tends to over-predict light rainfall events. The study by Dravitski et al. (2010) also found that the GFS tends to under-predict heavy rainfall events. In this case study of Hurricane Karl, the areal coverage of heavy precipitation was both under- and over-forecasted. During Hurricane Karl's initial stages (from the beginning of the analysis until 1200 UTC September 16) and final stages (from 1800 UTC September 17 until the end of the analysis) the areal coverage of maximum precipitation was under-forecasted.

During the middle part of Hurricane Karl's life cycle (from 1200 UTC September 16 until 1800

UTC September 17) the areal coverage of maximum precipitation was over-forecasted (see

Figure 5.7).

5.5 Sources of Error

It is important to keep in mind that errors are produced by CMORPH, as this technique is only an estimation—not a verification of the exact truth. Thus, it must be acknowledged that the method for assessing the accuracy of the updated GFS is not completely accurate within itself.

While CMORPH is more accurate than several other rainfall estimation techniques, it still has some disadvantages. CMORPH is consistent in its overestimation of rainfall estimates during the summer months (Sapiano and Arkin 2009). A study by Zeweldi et al. (2000) showed a correlation between time and error; as time increases from hours one through six, the error in recorded accumulated rain also increases. Most importantly, the study by Dinku et al. (2009) found that CMORPH tends to underestimate rainfall in mountainous regions. This was most likely the case in this study of Hurricane Karl; CMORPH’S estimation of accumulated rainfall amounts significantly decreased when the storm approached mountainous terrain, implying the verification is less accurate near Karl’s second landfall in Mexico.

Although the amount of spread among the ensemble precipitation forecasts was minimal, it should still be noted. The GEFS precipitation forecast goes six hours into the future for each of the 15 time frames. Therefore, there will be some spread among the ensemble members, with the amount of variation in the forecast increasing as time increases.

To compare the CMORPH and GEFS rainfall forecasts graphically on the same grid, the

CMORPH data had to be regrided to match the resolution of that of the GEFS data. As a result,

27 the resolution of the CMORPH precipitation was reduced from 0.25°x0.25° to 1°x1°. This led to a reduction of the fine-scale forecast of precipitation.

6: Conclusion

The six-hour forecasted hurricane center position was within 30 miles of the verified storm center position. Therefore, large track errors do not provide a main reason for differences among the forecast and verification. The forecast of six-hour accumulated precipitation was within one inch of the verification, except for immediately before and immediately after landfall.

The largest differences between forecasted and observed precipitation continued to be when the storm was at its peak intensity which was immediately before, during, and immediately after landfall. When Hurricane Karl was near landfall in Mexico, CMORPH’s rainfall amounts were lower than the forecasted rainfall amounts. According the study by Dinku et al. (2009), a possible explanation for this is that CMORPH underestimates rainfall amounts over higher altitudes. Nonetheless, because Dinku et al.’s study (2009) found that CMORPH still had skill in estimating precipitation over mountains it is likely that error within the GEFS forecast model contributed to the differences between the precipitation forecast and estimate. This is consistent with past findings that studied the GFS prior to its 2010 updates, such as the study by Werth et al.

(2010) that found the GFS to produce more error in its precipitation forecast for areas near land/sea boundaries.

GEFS under-forecasted the maximum amount of accumulated precipitation for a given point for all analyzed times except for those within ±10 hours of landfall in Mexico for which it over- forecasted. Some of this error is attributable to the lower GEFS model resolution.

The GEFS was less accurate in its spatial distribution forecast compared to its maximum precipitation amount forecast, evidenced by the larger percent errors for areal coverage. The

GEFS consistently displayed a larger areal coverage of precipitation, on average about 2.7 times greater than the verified areal coverage, with the greatest error occurring near the land/sea boundary/at the storm’s peak intensity. Some of this error is also attributed to the lower model resolution.

Results from image sets A and B suggest that the GEFS under-forecasts, while results from image set C suggest that the GEFS over-forecasts. The differences can be explained with the results from part D: although GEFS may display a lower maximum value of accumulated precipitation over water for a single point, the GEFS commonly over-forecasts the amount of light/moderate accumulations while under-forecasting amounts of heavy precipitation for the same time frame.

When the GEFS under-predicted the point maximum amount of precipitation it also tended to under-predict the areal coverage of heavy precipitation. Likewise, when the GEFS over- predicted the point maximum amount of precipitation it also tended to over-predict the areal coverage of heavy precipitation.

As past studies have indicated, the higher resolution of the GEFS significantly reduces its accuracy in forecasting for precipitation. This effect is seen in this study as the GEFS consistently forecasted the precipitation field to be too large.

This study only assessed the precipitation forecast from hours zero through six.

Further studies could be done to evaluate how the precipitation forecast evolves with time.

Another study could evaluate the precipitation forecast for all hurricanes occurring after the 29

GEFS 2010 update. To fully assess accuracy of the GEFS rainfall forecast, many more cases would need to be studied, but this provides a good starting point because it highlights areas where the forecast is least accurate.

References

Buckingham, C., T. Marchok, I. Ginis, L. Rothstein, and D. Rowe, 2010: Short and medium- range prediction of tropical and transitioning cyclone tracks within the NCEP Global Ensemble Forecasting System. Wea and Forecasting, 1, 1-52.

Chan, S.T., and M.Y. Chan, 2010: Application of satellite rain rate estimates to the prediction of tropical cyclone rainfall. ESCAP/WMO Typhoon Committee, 2-3.

Climate Prediction Center, cited 2010: NOAA CPC Morphing Technique (―CMORPH‖). [Available online at http://www.cpc.noaa.gov/products/janowiak/cmorph_description.html]

Dinku, T., F. Ruiz, S.J. Connor, P. Ceccato, 2010: Validation and intercomparison of satellite rainfall estimates over Colombia. J. Appl. Meteor. Climatol., 49, 1004–1014.

Dravitzki, S., and J. McGregor, 2010: Predictability of heavy precipitation in the Waikato River Basin of New Zealand. Mon. Wea. Rev., 0.

Fan, Y., and H. van den Dool, 2011: Bias correction and forecast skill of NCEP GFS ensemble week 1 and week 2 precipitation. Wea. Forecasting, 0.

Joyce, R.J., J. Janowiak, P.A. Arkin, and P. Xie, 2004: CMORPH: A method that produces global precipitation estimates from passive microwave and infrared data at high spatial and temporal resolution. Bull. Amer. Meteor. Soc., 5, 487-503.

Lu, C., H. Yuan, E. I. Tollerud, and N. Wang, 2010: Scale-dependent uncertainties in global QPFs and QPEs from NWP model and satellite fields. J. Hydrometeor, 11, 139–155.

Marchok, T., R. Rogers, R. Tuleya, 2007: Validation schemes for tropical cyclone quantitative precipitation forecasts: Evaluation of operational models for U.S. landfalling cases. Wea. Forecasting, 22, 726–746.

Mathur, M. B., H. S. Bedi, T. N. Krishnamurti, M. Kanamitsu, and J. S. Woollen, 1992: Use of satellite-derived rainfall for improving tropical forecasts. Mon. Wea. Rev., 120, 2540– 2560.

McBride, J. L., and E. E. Ebert, 2000: Verification of quantitative precipitation forecasts from operational numerical weather prediction models over Australia. Wea. Forecasting, 15, 103–121.

NASA, cited 2010: Tropical Rainfall Measuring Mission. [Available online at [http://trmm.gsfc.nasa.gov/overview_dir/why-univ.html]

NCEP, cited 2010a: August 2010 vs. August 2009 - A look at the first full month of new GFS. [Available online at http://www.emc.ncep.noaa.gov/GFS/perf.php]

NCEP, cited 2010b: Documentation. [Available online at http://www.emc.ncep.noaa.gov/GFS/doc.php]

NCEP, cited 2010c: GFS/GDAS Changes Since 1991. [Available online at http://www.emc.ncep.noaa.gov/GFS/doc.php]

NCEP, cited 2010d: Global Forecast System (GFS) upgrade implemented July 28, 2010 [Available online at http://www.ncep.noaa.gov/news/2010/global_forecast_system_announce.pdf]

NCEP, cited 2010e: Implementations. [Available online at http://www.emc.ncep.noaa.gov/GFS/impl.php]

NCEP, cited 2010f: Tropical storm performance monitor Page. [Available online at http://www.emc.ncep.noaa.gov/gmb/pan/track_avn.html]

NHC, cited 2011: Hurricane Karl advisory archive. [Available online at http://www.nhc.noaa.gov/archive/2010/KARL.shtml]

Por Esto, cited 2011: Yucatan. [Available online at http://www.poresto.net/ver_nota.php?zona=yucatan&idSeccion=15&idTitulo=46314]

Roeger, C., R. Stull, D. McClung, J. Hacker, X. Deng, and H. Modzelewski, 2003: Verification of mesoscale numerical weather forecasts in mountainous terrain for application to avalanche prediction. Wea. Forecasting, 18, 1140–1160.

Sapiano, M. R. P. and P. A. Arkin, 2009: An intercomparison and validation of high- resolution satellite precipitation estimates with 3-hourly gauge data. Bull. Amer. Meteor. Soc., 10, 149-166.

Spencer, R., and W. Braswell, 1997: How dry is the tropical free troposphere? Implications for

Tuleya, R. E., M. DeMaria, and R. J. Kuligowski, 2007: Evaluation of GFDL and simple statistical model rainfall forecasts for U.S. landfalling tropical storms. Wea. Forecasting, 22, 56–70.

Wang, J., and D. B. Wolff, 2009: Evaluation of TRMM ground-validation radar-rain errors using rain gauge measurements. Jour. of Applied Meteor. and Climatology, 49, 310–324.

Werth, D., and A. Garrett, 2010: Patterns of land-surface errors and biases in the global forecast system. Mo. Wea. Rev., 1, 1-44.

Wilson, J. W., W. R. Burrows, and A. Lanzinger, 1998: A strategy for verification of weather element forecasts from an ensemble prediction system. Mon. Wea. Rev., 127, 956–969.

Yang, F., H. Pan, S. K. Krueger, S. Moorthi, and S. J. Lord, 2006: Evaluation of the NCEP Global Forecast System at the ARM SGP site. Mon. Wea. Rev., 134, 3668–3690.

Zeweldi, D. A., and M. Gebremichael, 2009: Evaluation of CMORPH precipitation products at fine space–time scales, Bull. Amer Meteor. Soc., 10, 300-307.

Figure 5.1: Graphic A) shows the GEFS forecasted a point maximum accumulated precipitation amount of 0.8 inches while graphic B) shows the CMORPH estimate to be 1.6 inches. This occurred when Karl was a tropical storm.

Figure 5.2: These images show the largest forecast error for both areal coverage and point maximum auulated preipitatio aout. This ourred at Karl’s peak itesit, just efore ladfall.

Figure 5.5: Difference in GEFS and CMORPH 6 hour accumulated precipitation amounts. This image shows the time at which there was the most disagreement between the forecast and estimate.

Figure 5.6: Difference in GEFS and CMORPH 6 hour accumulated precipitation amounts. This image shows a time at which there was a smaller amount of disagreement between the forecast and estimate.

Figure 5.7: Graphic A) shows the under-forecasted areal coverage of precipitation by the GEFS while graphic B) shows the over-forecasted areal coverage of precipitation by the GEFS.