Model Output Statistics (MOS) - Objective Interpretation of NWP Model Output University of Maryland – April 4, 2012
Total Page:16
File Type:pdf, Size:1020Kb
Model Output Statistics (MOS) - Objective Interpretation of NWP Model Output University of Maryland – April 4, 2012 Mark S. Antolik Meteorological Development Laboratory Statistical Modeling Branch NOAA/National Weather Service Silver Spring, MD (301) 713-0023 ext. 110 email: [email protected] MOS Operational System “Fun Facts” With apologies to David Letterman, of course! ● 9 million regression equations ● 75 million forecasts per day ● 1200 products sent daily ● 400,000 lines of code – mostly FORTRAN ● 180 min. supercomputer time daily 8 ● All developed and maintained by ~ 12X MDL / SMB meteorologists! OUTLINE 1. Why objective statistical guidance? 2. What is MOS? Definition and characteristics The “traditional” MOS product suite (GFS, NAM) Other additions to the lineup 3. Simple regression examples / REEP 4. Development strategy - MOS in the “real world” 5. Verification 6. Dealing with NWP model changes 7. Where we’re going – GMOS and the future WHY STATISTICAL GUIDANCE? ● Add value to direct NWP model output Objectively interpret model - remove systematic biases - quantify uncertainty Predict what the model does not Produce site-specific forecasts (i.e. a “downscaling” technique) ● Assist forecasters “First Guess” for expected local conditions “Built-in” model/climo memory for new staff A SIMPLE STATISTICAL MODEL Relative Frequency of Precipitation as a Function of 12-24 Hour NGM Model-Forecast Mean RH 1.0 0.9 3-YR SAMPLE; 200 STATIONS 0.8 1987-1990 COOL SEASON 0.7 0.6 0.5 47% 0.4 0.3 0.2 0.1 OBSERVED REL. FREQUENCY REL. OBSERVED 0.0 0 10 20 30 40 50 60 70 80 90 100 NGM MEAN RELATIVE HUMIDITY (%) MOS Max Temp vs. Direct Model Output MRF MOS Max Temp 1999 Warm Season - CONUS/AK 8 7 6 CLIMO 5 MAE (deg F) DMO 4 3 NEW MOS CLIMO DMO MOS 2 24 48 72 96 120 144 168 192 Projection (hours) What is MOS? MODEL OUTPUT STATISTICS (MOS) Relates observed weather elements (PREDICTANDS) to appropriate variables (PREDICTORS) via a statistical approach. Predictors are obtained from: 1. Numerical Weather Prediction (NWP) Model Forecasts 2. Prior Surface Weather Observations 3. Geoclimatic Information Current Statistical Method: MULTIPLE LINEAR REGRESSION (Forward Selection) MODEL OUTPUT STATISTICS (MOS) Properties ● Mathematically simple, yet powerful ● Need historical record of observations at forecast points (Hopefully a long, stable one!) ● Equations are applied to future run of similar forecast model MODEL OUTPUT STATISTICS (MOS) Properties (cont.) ● Non-linearity can be modeled by using NWP variables and transformations ● Probability forecasts possible from a single run of NWP model ● Other statistical methods can be used e.g. Polynomial or logistic regression; Neural networks MODEL OUTPUT STATISTICS (MOS) ● ADVANTAGES Recognition of model predictability Removal of some systematic model bias Optimal predictor selection Reliable probabilities Specific element and site forecasts ● DISADVANTAGES Short samples Changing NWP models Availability & quality of observations MAJOR CHALLENGE TO MOS DEVELOPMENT: RAPIDLYRAPIDLY EVOLVINGEVOLVING NWPNWP MODELSMODELS ANDAND OBSERVATIONOBSERVATION PLATFORMSPLATFORMS OBS MODELS Can make for: Mesonets NCEP 1. SHORT, UNREPRESENTATIVE DATA SAMPLES 2. DIFFICULT COLLECTION OF APPROPRIATE PREDICTAND DATA New observing systems: (ASOS, WSR-88D, Satellite) (Co-Op, Mesonets) “Old” predictands: The elements don’t change! “Traditional” MOS text products GFS MOS GUIDANCE MESSAGE FOUS21-26 (MAV) KLNS GFS MOS GUIDANCE 11/29/2004 1200 UTC DT /NOV 29/NOV 30 /DEC 1 /DEC 2 HR 18 21 00 03 06 09 12 15 18 21 00 03 06 09 12 15 18 21 00 06 12 N/X 28 48 35 49 33 TMP 43 44 39 36 33 32 31 39 46 45 41 38 37 39 41 44 45 44 40 40 35 DPT 27 27 28 29 29 29 29 33 35 35 36 35 36 39 41 42 37 34 30 30 28 CLD CL BK BK BK OV OV OV OV OV OV OV OV OV OV OV OV OV BK CL CL CL WDR 34 36 00 00 00 00 00 00 00 14 12 12 10 11 12 19 28 29 29 29 28 WSP 06 02 00 00 00 00 00 00 00 01 02 04 04 06 07 08 15 17 18 09 05 P06 0 0 4 3 11 65 94 96 7 0 0 P12 6 19 94 96 0 Q06 0 0 0 0 0 3 4 4 0 0 0 Q12 0 0 4 2 0 T06 0/ 0 0/18 0/ 3 0/ 0 0/ 0 0/18 2/ 1 10/ 4 0/ 3 1/ 0 T12 0/26 0/17 0/27 10/25 1/38 POZ 2 0 0 1 2 4 4 0 1 1 2 3 3 1 1 0 2 1 2 3 1 POS 13 2 1 2 1 0 0 0 0 0 0 0 0 2 0 0 0 3 0 9 28 TYP R R R R R R R R R R R R R R R R R R R R R SNW 0 0 0 CIG 8 8 8 8 7 7 7 8 8 7 7 7 4 2 3 3 6 7 8 8 8 VIS 7 7 7 7 7 7 7 7 7 7 7 7 5 5 4 2 6 7 7 7 7 OBV N N N N N N N N N N N N BR BR BR BR N N N N N NAM MOS GUIDANCE MESSAGE FOUS44-49 (MET) KBWI NAM MOS GUIDANCE 2/27/2009 1200 UTC DT /FEB 27/FEB 28 /MAR 1 /MAR 2 HR 18 21 00 03 06 09 12 15 18 21 00 03 06 09 12 15 18 21 00 06 12 N/X 38 46 32 41 24 TMP 59 58 55 54 49 43 38 38 43 45 40 38 37 35 33 34 37 38 33 29 25 DPT 46 47 48 46 37 30 24 22 22 22 24 27 28 26 25 24 24 21 17 12 10 CLD OV OV OV OV OV SC SC SC CL BK OV OV OV OV OV OV OV OV OV OV BK WDR 21 20 22 25 31 32 34 36 01 03 05 04 01 36 35 35 35 34 35 33 34 WSP 15 09 08 06 10 11 10 12 10 09 08 10 12 13 14 16 11 13 15 16 17 P06 89 10 3 2 2 76 73 13 17 27 19 P12 10 3 81 17 30 Q06 1 0 0 0 0 4 1 0 0 0 0 Q12 0 0 4 0 0 T06 2/ 9 0/ 5 0 /0 0/ 5 3/ 1 5/ 3 0/ 0 0/ 2 2/ 5 0/ 0 T12 2/ 9 0/ 5 5/ 3 1/ 2 7/ 5 SNW 0 0 0 CIG 6 6 4 5 7 8 8 8 8 8 7 6 4 3 4 3 4 4 7 6 7 VIS 7 7 6 7 7 7 7 7 7 7 7 7 3 6 5 7 7 7 7 7 7 OBV N N N N N N N N N N N N BR N BR N N N N N N Short-range (GFS / NAM) MOS ● STATIONS: Now at approx. 1990 Forecast Sites (CONUS, AK, HI, PR, Canada) ● FORECASTS: Available at projections of 6-84 hours GFS available for 0600 and 1800 UTC cycles ● RESOLUTION: GFS predictors on 95.25 km grid; NAM on 32 km Predictor fields available at 3-h timesteps ● DEPENDENT SAMPLE NOT “IDEAL”: Fewer seasons than older MOS systems Non-static underlying NWP model Approx. 1990 sites Short-range (GFS / NAM) MOS ● STATIONS: Now at approx. 1990 Forecast Sites (CONUS, AK, HI, PR) ● FORECASTS: Available at projections of 6-84 hours GFS available for 0600 and 1800 UTC cycles ● RESOLUTION: GFS predictors on 95.25 km grid; NAM on 32 km Predictor fields available at 3-h timesteps ● DEPENDENT SAMPLE NOT “IDEAL”: Fewer seasons than older MOS systems Non-static underlying NWP model GFSX MOS GUIDANCE MESSAGE FEUS21-26 (MEX) KCXY GFSX MOS GUIDANCE 11/26/2004 0000 UTC FHR 24| 36 48| 60 72| 84 96|108 120|132 144|156 168|180 192 FRI 26| SAT 27| SUN 28| MON 29| TUE 30| WED 01| THU 02| FRI 03 CLIMO X/N 43| 29 47| 40 55| 35 51| 29 45| 32 40| 36 42| 30 45 31 46 TMP 37| 32 43| 43 46| 37 41| 32 39| 35 36| 38 37| 33 37 DPT 24| 27 37| 40 32| 28 28| 26 31| 32 30| 32 27| 24 25 CLD PC| OV OV| OV PC| CL PC| PC OV| OV OV| PC CL| CL CL WND 10| 5 11| 11 16| 10 10| 5 9| 6 10| 12 14| 12 12 P12 0| 5 13| 91 13| 3 9| 14 24| 52 54| 48 21| 12 25 20 18 P24 | 16| 100| 9| 26| 62| 72| 25 29 Q12 0| 0 0| 3 0| 0 0| 0 0| 2 2| 2 | Q24 | 0| 3| 0| 0| 4| | T12 0| 0 0| 3 0| 0 0| 0 4| 6 4| 3 1| 1 1 T24 | 0 | 3 | 0 | 0 | 6 | 4 | 1 PZP 12| 9 12| 4 3| 5 6| 10 8| 8 3| 16 10| 12 8 PSN 62| 15 3| 0 0| 10 9| 15 24| 1 0| 9 32| 27 18 PRS 26| 24 7| 0 17| 18 20| 13 15| 1 2| 18 9| 11 11 TYP S| RS R| R R| R R| R RS| R R| R RS| RS R SNW | 0| 0| 0| 0| | | MOS station-oriented products: Other additions Marine MOS 44004 GFS MOS GUIDANCE 11/22/2005 1200 UTC DT /NOV 22/NOV 23 /NOV 24 /NOV 25 HR 18 21 00 03 06 09 12 15 18 21 00 03 06 09 12 15 18 21 00 03 06 TMP 58 53 49 49 50 48 46 44 44 45 47 48 51 54 56 60 62 61 59 51 47 WD 23 25 27 28 28 29 29 28 28 27 27 25 22 22 22 23 23 23 24 27 28 WS 33 31 29 25 23 22 24 25 23 18 14 12 14 19 26 29 30 29 29 28 24 WS10 36 34 31 26 25 24 26 27 25 19 15 13 15 21 28 31 32 31 31 30 26 DT /NOV 25 / HR 09 12 15 18 21 00 TMP 45 45 45 47 47 47 WD 29 29 28 30 29 34 WS 18 15 10 10 13 12 WS10 20 16 11 11 14 13 Marine MOS sites Standard MOS sites Max/Min Guidance for Co-op Sites GFS-BASED MOS COOP MAX/MIN GUIDANCE 3/01/05 1800 UTC WED 02| THU 03| FRI 04 ANNM2 26 46| 24 45| 25 46 BERM2 28 41| 25 39| 25 43 BTVM2 23 39| 21 38| 20 43 Beltsville, MD CBLM2 20 40| 18 39| 20 46 CHEM2 25 42| 21 39| 21 44 CNWM2 21 42| 21 40| 20 45 DMAM2 20 37| 18 37| 20 42 ELCM2 25 41| 21 41| 18 45 EMMM2 23 42| 20 41| 20 43 FREM2 23 46| 21 42| 23 44 FRSM2 17 27| 13 27| 13 36 GLDM2 21 37| 18 39| 18 43 Glenn Dale, MD HAGM2 23 43| 18 43| 19 45 KAPG 27 41| 23 37| 22 43 LRLM2 23 44| 21 42| 22 46 MECM2 24 47| 20 42| 20 45 Laurel 3 W MILM2 25 48| 22 41| 20 39 MLLM2 22 39| 18 37| 18 41 OLDM2 18 31| 13 28| 12 35 OXNM2 23 42| 22 40| 23 48 PRAM2 22 49| 22 45| 18 45 Western Pacific MOS Guidance Midway, US Saipan, ROM Wake, US NSTU GFS MOS GUIDANCE 11/07/2008 1200 UTC DT /NOV 7/NOV 8 /NOV 9 /NOV 10 HR 18 21 00 03 06 09 12 15 18 21 00 03 06 09 12 15 18 21 00 06 12 TMP 84 85 85 85 82 82 81 79 80 83 84 83 81 81 80 79 81 84 86 82 80 DPT 77 77 78 77 76 77 76 75 77 78 77 77 76 77 76 75 77 78 77 77 76 WDR 08 08 08 09 08 07 05 04 06 07 08 07 05 02 35 01 02 07 07 08 10 WSP 17 17 15 13 11 08 07 07 07 08 09 08 07 05 04 04 04 08 09 06 06 P06 36 37 47 46 50 43 25 35 43 30 31 P12 60 66 60 59 47 Application of Linear Regression to MOS Development MOS LINEAR REGRESSION JANUARY 1 - JANUARY 30, 1994 0000 UTC KCMH 60 50 F) 40 ° 30 20 10 TODAY'S MAX ( TODAY'S 0 -10 1150 1200 1250 1300 1350 18-H NGM 850-1000 MB THICKNESS (M) MOS LINEAR REGRESSION JANUARY 1 - JANUARY 30, 1994 0000 UTC KCMH 60 MAX T = -352 + (0.3 x 850-1000 mb THK) 50 F) 40 ° RV=93.1% 30 20 10 TODAY'S MAX ( TODAY'S 0 -10 1150 1200 1250 1300 1350 18-H NGM 850-1000 MB THICKNESS (M) REDUCTION OF VARIANCE A measure of the “goodness” of fit and Predictor / Predictand correlation Variance - Standard Error = RV Variance MEAN RV PREDICTAND { } UNEXPLAINED VARIANCE * PREDICTOR MOS LINEAR REGRESSION JANUARY 1 - JANUARY 30, 1994 0000 UTC KUIL 60 F) ° 50 RV=26.8% 40 Same predictor, TODAY'S MAX ( MAX TODAY'S Different site, Different relationship! 30 1250 1300 1350 1400 18-H NGM 850-1000 MB THICKNESS (M) MOS LINEAR REGRESSION DECEMBER 1 1993 - MARCH 5 1994 0000 UTC KCMH 1 0 24 H PRECIPITATION .01" ≥ 24 H PRECIPITATION - 12 10 20 30 40 50 60 70 80 90 100 AVG.