Three Region Accuracy Overview 2010 Through June 2016
Total Page:16
File Type:pdf, Size:1020Kb
Three Region Accuracy Overview 2010 through June 2016 By ForecastWatch.com, a Service of Intellovations, LLC !ric Floehr Foun"er an" !# Intellovations, LLC $.#. Box 16&&&2 olum'us, #hio &(216 [email protected] htt+,--***.forecastwatch.com .61&/ &&0-01(0 Executive Summary om+anies that s+eciali1e in weather +re"iction have an investment in accuracy an" in esta'lishing their ca+a'ility to +rovi"e that accuracy. 2heir clients, in turn, re3uire access to accurate *eather forecasting *hich is critical to success in many in"ustries, 'oth +u'lic an" +rivate. 4n"erstan"ing an" evaluating the +ast is a key to assessing future risk an" o++ortunity. Forecasts from three regions an" three "ays-out grou+ings *ere analyze". Forecasts were collecte" from 2010 through June of 2016 for the 4nite" States, an" from 201( through June of 2016 for 6sia0$aci7c an" !uro+e. 6ccuracy was measure" as the com'ination of the +ercentage of high tem+erature forecasts within three "egrees, the +ercentage of low tem+erature forecasts within three "egrees, an" the +ercent correct of +reci+itation forecasts. Forecasts from eleven "ifferent +rovi"ers were collecte" for a total of over 1&2 million forecasts analyze". In the seven years, three regions, an" three forecast time +erio"s in the stu"y, only four out of eleven forecasting com+anies collecte" were foun" to 'e the most accurate, The Weather hannel, 8eteo9rou+, Weather 4n"ergroun", an" Foreca. The Weather hannel was the most accurate in 2: of the &; region-"ays0out-year grou+ings, while 2he Weather hannel tie" *ith Weather 4n"ergroun" for one +erio". Weather 4n"ergroun" was the most accurate in an a""itional 6 +erio"s, all of *hich occurre" after The Weather om+any .the +arent com+any of 2he Weather hannel/ +urchase" Weather 4n"ergroun" in 2012. Meteo9rou+ came in 7rst in seven +erio"s .1;</ while Foreca came in 7rst four times .=</. Results ForecastWatch calculate" one-to0three "ays out, three-to07ve "ays out, an" six-to-nine "ays out overall forecast accuracy for the 6sia0$aci7c region, !uro+e, an" the 4nite" States, inclu"ing 6laska an" >awaii as *ell as the territories. The +erio" for the collection was 201( through 2016 for the 6sia0$aci7c an" !uro+ean regions, an" 2010 through 2016 for the 4nite" States. 6ll "ata from 2016 was collecte" for the 7rst half of the calen"ar year only, from January to June. The accuracy ta'les *ere sorte" 'y 2016 accuracy +ercentages. 2he +rovi"er with the most accuracy for each year is highlighte". Forecasts were collecte" from eight +rovi"ers for all three regions, an" inclu"e" the BB in !uro+e, an" 'oth the ?ational Weather Service *e'site .weather.gov/ an" the ?WS 2hree @egion 6ccuracy #vervie*, 2010 through June 2016 Decem'er 2016 1 ?ational Digital Forecast Data'ase .?DFA/ in the 4nite" States. ForecastWatch "i" not 'egin collecting forecasts from some +rovi"ers until after the stu"y +erio" 'egan, an" are in"icate" 'y graye" out cells. For exam+le, ForecastWatch 'egan collecting forecasts from Worl" Weather #nline "uring 2010, Foreca an" 8eteo9rou+ in 2011, an" Dark Sky in 201(. Thus the 7rst full year of forecasts woul" 'e 2011 for Worl" Weather #nline, 2012 for Foreca an" Meteo9rou+, an" 201& for Dar5 Sky. 6""itionally Weather 4n"ergroun" only starte" +rovi"ing a 7ve0"ay out forecast in 2011, an" a ten0"ay out forecast 201&. #verall forecast accuracy was calculate" 'y averaging the num'er of high tem+erature forecasts within three "egrees, the num'er of low tem+erature forecasts within three "egrees, an" the num'er of icon an" text forecasts *hich correctly +re"icte" +reci+itation or not. 2otal forecasts collecte" 'y region were over ; million for the 6sia0$aci7c @egion, over 16 million for !uro+e, an" nearly 122 million for the 4nite" States, for a total of over 1&( million forecasts. #ne0to02hree Aays #ut 201( 201& 201; 2016 The Weather Channel :;.(2< :(.16< :6.1B< :;.;&< Weather Underground :0.B=< :(.1(< :6.10< :;.&=< AccuWeather :0.01< :0.&2< :(.=2< :&.(0< MeteoGroup :(.B:< :(.==< :&.1&< :2.&1< Foreca :0.B:< :1.B:< :2.:2< :1.=(< "ntellicast :0.:(< 6=.0&< :1.0B< :0.=0< #ar$ Sky 6=.66< 6B.1;< 6:.B6< World Weather Online 60.&=< ;B.:=< 60.;0< 60.=1< Table 1: Asia-Pacific 1-3 Day Out Overall Accuracy, 2013 through June 201 6s shown in 2a'le 1 a'ove, 2he Weather hannel was the most accurate +rovi"er of one-to0 three "ay out forecasts in the 6sia0$aci7c region for three of the four years of the stu"y. Meteo9rou+ was the most accurate in 201&, followe" 'y The Weather hannel in secon" +lace. 2hree @egion 6ccuracy #vervie*, 2010 through June 2016 Decem'er 2016 2 Weather 4n"ergroun" e%+erience" the greatest im+rovement in forecasting, increasing in accuracy 'y &.60 +ercentage +oints from 201( to 2016. 2he most accurate +rovi"er in 201&, Meteo9rou+, "ecline" in accuracy from its +eak in 201& to 2016 'y 1.;B +ercentage +oints. Dar5 Sky an" 8eteo9rou+ *ere the only +rovi"ers in this category to sho* a "ecrease in accuracy from the 7rst year of collection to the last. 201( 201& 201; 2016 Weather Underground 6B.BB< :;.20< :6.11< :;.2&< The Weather Channel :(.20< :;.2&< :6.0=< :;.1(< AccuWeather 6B.B1< :1.=2< :;.;(< :&.BB< "ntellicast :1.=;< :(.=B< :&.B&< :&.2B< Foreca :0.;1< :2.21< :&.0(< :(.&;< MeteoGroup :(.1&< :&.2=< :&.&B< :(.10< %%C 6B.(&< 6=.2;< 6=.62< 6B.6B< #ar$ Sky 6=.(B< 6B.::< 6:.1=< World Weather Online 61.B0< 62.11< 6(.6(< 6(.B;< Table 2: !uro"e 1-3 Day Out Overall Accuracy, 2013 through June 201 2a'le 2 a'ove shows 2he Weather hannel +rovi"e" the most accurate forecasts for the !uro+ean region in 201( an" 201&, while Weather 4n"ergroun" was the most accurate +rovi"er in 201; an" 2016. Worl" Weather #nline remaine" consistently the least accurate for all nine +rovi"ers of one-to-three "ay out forecasts in !uro+e 'etween 201( an" 2016. For one-to-three "ay out forecasts for the 4nite" States .'elo*, 2a'le (/, 2he Weather hannel or Weather 4n"ergroun" ha" the highest accuracy for six of the seven stu"y years. #nly in 2012 "i" Meteo9rou+ take the to+ s+ot with an overall accuracy +ercentage of :&.&&%, followe" 'y The Weather hannel at :&.2:%. 6ccuracy increase" for all +rovi"ers from the 7rst year for *hich "ata was availa'le through June of 2016. The +rovi"er with the greatest increase in accuracy was Weather 4n"ergroun", *ith an increase of 6.2; +ercentage +oints. The +rovi"er *ith the smallest increase was Worl" Weather #nline, which only im+rove" 0.06 +ercentage +oints. 2hree @egion 6ccuracy #vervie*, 2010 through June 2016 Decem'er 2016 3 2010 2011 2012 201( 201& 201; 2016 The Weather Channel :2.0B< :1.;(< :&.2:< :;.00< :&.=B< :6.62< :6.21< Weather Underground 6=.B:< 6B.(&< 6=.&&< :0.:&< :;.2&< :6.6(< :6.12< "ntellicast :1.:;< :1.1B< :(.;:< :&.1:< :&.2;< :;.=B< :;.61< AccuWeather 6=.0;< 6B.1&< :0.((< :1.26< :2.1(< :;.2:< :&.B=< Foreca :0.(=< :0.=:< :1.=1< :&.22< :&.BB< MeteoGroup :&.&&< :&.=2< :;.1:< :;.2=< :&.:6< &WS Web 6=.&(< 6B.:(< :0.06< :0.(&< :0.2:< :1.(0< :1.=2< #ar$ Sky 66.&;< 6B.(2< 6=.((< &WS &#!# 6B.;&< 6:.B6< 6=.(;< 6=.B:< 6=.&0< 6=.B&< 6=.2B< World Weather Online 6(.;0< 61.&&< 61.12< 6(.=B< 6&.;B< 6(.;6< Table 3: #$A 1-3 Days Out Overall Accuracy, 2010 through June 201 2hree0to0Five Aays #ut 201( 201& 201; 2016 The Weather Channel :1.=:< 6=.61< :2.(=< :2.11< Weather Underground 6:.(&< 6=.;6< :2.1B< :1.B0< AccuWeather 66.B;< 6:.1=< 6=.B6< :0.;=< MeteoGroup :0.2=< :0.2(< :0.;=< 6B.;;< Foreca 6:.(2< 6B.2&< 6=.00< 6B.(&< "ntellicast 6:.=;< 6;.B:< 6:.:;< 6:.61< #ar$ Sky 66.&0< 6&.2:< 6(.1&< World Weather Online ;B.61< ;:.02< ;B.;6< ;=.0&< Table %: Asia-Pacific 3-& Day Out Overall Accuracy, 2013 through June 201 2hree @egion 6ccuracy #vervie*, 2010 through June 2016 Decem'er 2016 4 2a'le & on the +revious +age shows overall accuracy for three-to07ve "ay out forecasts in the 6sia0$aci7c region. Like the one-to0three "ay out accuracy, The Weather hannel was the most accurate +rovi"er for three of the four stu"y years, with Meteo9rou+ the most accurate +rovi"er in 201&. Meteo9rou+, Intellicast, an" Dark Sky all "is+laye" a re"uction in accuracy over the four-year +erio" ca+ture" in this ta'le. 201( 201& 201; 2016 Weather Underground 62.:0< 6=.26< :0.01< 6=.10< The Weather Channel 6:.0=< 6=.22< 6=.=(< 6=.01< AccuWeather 6(.20< 66.16< 6=.11< 6B.&2< "ntellicast 66.1;< 6B.16< 6B.=;< 6B.(2< Foreca 6&.(1< 6;.;1< 6:.1B< 6:.&2< MeteoGroup 6:.&=< 6B.:2< 6B.&;< 6:.21< %%C 6&.B:< 6;.=6< 66.;:< 6;.&2< #ar$ Sky 6(.:&< 6(.&;< 61.&;< World Weather Online ;:.&(< ;B.&;< 60.06< ;=.;;< Table &: !uro"e 3-& Day Out Overall Accuracy, 2013 through June 201 2a'le ; a'ove shows three-to07ve "ay out overall accuracy for !uro+e. 8eteo9rou+ was the most accurate +rovi"er in 201(. >owever, from 201& onwar", Weather 4n"ergroun" was the most accurate +rovi"er. In 201(, the most accurate +rovi"er, Meteo9rou+, was &.:= +ercentage +oints or :.0=% more accurate than Weather 4n"ergroun", *hich was the seventh-most accurate for that year.