<<

Article A Comprehensive Forecasting System Integrating Artificial Intelligence and Numerical Prediction

Branko Kosovic 1,* , Sue Ellen Haupt 1 , Daniel Adriaansen 1, Stefano Alessandrini 1, Gerry Wiener 1, Luca Delle Monache 2, Yubao Liu 1, Seth Linden 1, Tara Jensen 1, William Cheng 1, Marcia Politovich 1 and Paul Prestopnik 1

1 National Center for Atmospheric Research, 1850 Table Mesa Dr, Boulder, CO 80305, USA; [email protected] (S.E.H.); [email protected] (D.A.); [email protected] (S.A.); [email protected] (G.W.); [email protected] (Y.L.); [email protected] (S.L.); [email protected] (T.J.); [email protected] (W.C.); [email protected] (M.P.); [email protected] (P.P.) 2 Scripps Institution of , University of California at San Diego, 9500 Gilman Dr, La Jolla, CA 92093, USA; [email protected] * Correspondence: [email protected]; Tel.: +1-303-497-2717

 Received: 5 December 2019; Accepted: 15 March 2020; Published: 16 March 2020 

Abstract: The National Center for Atmospheric Research (NCAR) recently updated the comprehensive wind power forecasting system in collaboration with Xcel addressing users’ needs and requirements by enhancing and expanding integration between numerical weather prediction and machine-learning methods. While the original system was designed with the primary focus on day-ahead power prediction in support of power trading, the enhanced system provides short-term forecasting for unit commitment and economic dispatch, uncertainty quantification in wind prediction with probabilistic forecasting, and prediction of extreme events such as icing. Furthermore, the empirical power conversion machine-learning algorithms now use a quantile approach to data quality control that has improved the accuracy of the methods. Forecast uncertainty is quantified using an analog ensemble approach. Two methods of providing short-range ramp forecasts are blended: the variational doppler analysis system and an observation-based expert system. Extreme events, specifically changes in wind power due to high winds and icing, are now forecasted by combining numerical weather prediction and a fuzzy logic artificial intelligence system. These systems and their recent advances are described and assessed.

Keywords: grid integration; machine learning; ; turbine icing; wind power forecasting; wind energy

1. Introduction The National Center for Atmospheric Research (NCAR), in collaboration with Xcel Energy addressing users’ needs and requirements, has developed a comprehensive wind power forecasting system. The original forecasting system was designed for day-ahead forecasting to support power trading. The new augmented and enhanced forecasting system provides capabilities for short-term forecasting, including wind ramp detection, prediction of extreme events such as icing conditions that can significantly impact wind power production when wind resource is abundant, empirical wind-to-power conversion techniques, and uncertainty quantification in power forecasting. This system employs artificial intelligence methods [1–3] to integrate disparate data sources with publicly available numerical weather prediction model outputs. The development of the new comprehensive forecasting

Energies 2020, 13, 1372; doi:10.3390/en13061372 www.mdpi.com/journal/energies Energies 2020, 13, 1372 2 of 16 system is motivated by risk reduction of wind power integration into a power grid and reduction of the levelized cost of wind power. The wind power forecasting system provides essential information for effective integration of variable generation into the power grid and addresses requirements for both the effective maintenance of reliable electric grids and energy trading. The grid operators and energy traders require accurate wind powerEnergies forecasts 2019, 12, xat FOR PEER scalesREVIEW spanning a range from minutes to several days ahead.2 of 17 Since no single methodology can perform optimally across all these temporal scales, we have combinedThe w numericalind power forecasting weather system predictions provides (NWPs) essential thatinformation provides for effective skillful integration predictions of at variable generation into the power grid and addresses requirements for both the effective beyond a few hours with specialized methods based on observations that can improve the very maintenance of reliable electric grids and energy trading. The grid operators and energy traders short-rangerequire forecasts. accurate wind power forecasts at time scales spanning a range from minutes to several days To developahead. aSince decision no single support weather system forecasting for the methodology effective integration can perform of optimally variable across generations, all these we have leveragedtemporal proven scales, forecasting we have methodologies combined numerical for w eacheather temporal, predictions as (NWP wells) asthat spatial, provides scale. skillful Disparate predictions at times beyond a few hours with specialized methods based on observations that can sources of data, including power generation data, as well as local and regional weather observations, improve the very short-range forecasts. are combined usingTo develop artificial a decision intelligence support system methods for the with effective the information integration ofabout variable physics generation ands, we dynamics of the atmospherehave leveraged to predict proven power forecasting output methodologies [4–7]. In addition, for each advance temporal, knowledgeas well as spatial of extreme, scale. events, such as iceDisparate , sources can greatly of data aid, including system power operations generation and data methods,, as well as and local tools and forregional generating weather warnings observations, are combined using artificial intelligence methods with the information about physics of potentialand impacts dynamics of of these the processes to on predict wind power power output generation [4–7]. In addition, are developed. advance knowledge of The schematicextreme events, diagram such as of ice the storms, comprehensive can greatly aid wind system power operation forecastings and methods system, and [1 –tools3] developed for by NCAR is showngenerating in Figure warning1.s Theof potential central impactscomponent of these of the processes system on is wind the Dynamic power generation Integrated are foreCast System (DICastdeveloped.®). DICast is an advanced machine-learning module that has been under development The schematic diagram of the comprehensive wind power forecasting system [1–3] developed at NCARby for NCAR over is twenty shown in years. Figure It1. The blends central publicly component available of the system model is the output Dynamic and Integrated high-resolution NWP modelsforeC configuredast System (DICast® for Xcel ). DICast Energy’s is anregions advanced with machine weather-learning observations module that has form been wind under farms and routine meteorologicaldevelopment at surfacesNCAR for andover uppertwenty airyears. observations. It blends publicly The available wind farmmodel data output include and high wind- speed measurementsresolution from NWP models mounted configured .for Xcel Energy’s region For ans with improved weather observations short-term form forecast wind of wind farms and routine meteorological surfaces and upper air observations. The data include ramps, wewind have speed also integrated measurements the from variational Nacelle mounted doppler anemometers. radar analysis For systeman improved (VDRAS) short-term together with an observation-basedforecast of wind expert ramps system., we have An also alternative integrated for the short-termvariational d forecastingoppler radar isanalysis the regime-switching system approach [(VDRAS)8,9]. While together the with regime-switching an observation-based approach expert system. has proven An alternative effective, for short our- focusterm forecasting was onproviding more accurateis thephysical regime-switching models approach for short-term [8,9]. While forecasting the regime-switching in the summertime approach has whenproven convectiveeffective, our focus was on providing more accurate physical models for short-term forecasting in the outflows aresummertime responsible when convective for numerous storm outflows wind rampsare responsible throughout for numerous the areaswind ramps served throughout by Xcel Energy. By assimilatingthe areas radar served observations, by Xcel Energy. VDRAS By assimilating is able to radar provide observations accurate, VDRAS short-term is able forecasts to provide not only of frontal passagesaccurate when short-term wind forecasts regimes not switchonly of frontal but also passages storm when outflows. wind regimes switch but also storm outflows.

Figure 1. FlowchartFigure 1. Flowchart of the Nationalof the National Center Center for for Atmospheric Atmospheric Research Research’s’s (NCAR’s (NCAR’s)) Xcel Energy Xcel p Energyower power predictionp system.rediction NCEP:system. U.S.NCEP: National U.S. National Centers Centers for Environmental for Environmental Prediction, Prediction NAM:, NAM: North North American American model, GFS: global forecasting system, RUC: Rapid Update Cycle, GEM Global model, GFS: global forecasting system, RUC: Rapid Update Cycle, GEM Global Environmental Multiscale model, WRF RTFDDA: Weather Research and Forecasting-based real-time four-dimensional data assimilation, and VDRAS: variational doppler radar analysis system. Energies 2020, 13, 1372 3 of 16

DICast provides deterministic point forecasts at the location of interest; however, energy traders and grid operators require power forecasts. We have developed an empirical power conversion algorithm and combined it with an analog ensemble (AnEn) approach to quantify the uncertainty of these predictions. Furthermore, we have developed an icing potential warning system. This paper describes numerical weather prediction (NWP) applications in Section2, followed by statistical processing and power conversion in Section3 and probabilistic prediction in Section4. Section5 treats the short-term problem of forecasting ramps. The AI-based extreme weather prediction system is described in Section6. Section7 summarizes and provides some thoughts for future applications.

2. Numerical Weather Prediction In addition to publicly available NWP model output in the wind forecasting system, we use a limited area, high-resolution NWP system based on the Weather Research and Forecasting (WRF) model, specifically configured for the areas of interest. The WRF model is a community mesoscale NWP model designed for both operational forecasting and atmospheric research needs for a range of applications [10]. It is maintained by NCAR and continually enhanced by university partners and a wider research community with over 20,000 users across the world. NOAA uses WRF as the basis of their operational 3-km high-resolution rapid refresh system, which runs hourly over the contiguous United States [11]. WRF includes many options for approaches to the physics, uses state-of-the-science numerical procedures, and can be configured to run over the desired domains. The team carefully configured the combination of physics schemes to optimize the prediction of boundary layer wind speed using the Yonsei University (YSU) model, Monin-Obukhov surface layer scheme, Dudhia shortwave model, rapid radiative transfer model (RRTM) for longwave radiation, Grell-Devenyi ensemble cumulus parameterization, Noah land surface model, and the Thompson microphysics parameterization. WRF is initialized with boundary conditions from the U.S. National Centers for Environmental Prediction (NCEP) global forecasting system (GFS). To optimize the model for the site, local data are assimilated into the model. NCAR applies a WRF-based real-time four-dimensional data assimilation (RTFDDA) system for wind power prediction, which is based on Newtonian relaxation [12] and adds forcing terms in the equations for momentum, , and moisture. These terms “nudge” the model solution to agree better with the local observations. This nudging approach forces the equations toward a state that represents the current state of the observed atmosphere subjects to maintaining a smooth solution to the equations of fluid motion. NCAR assimilates standard meteorological observations plus the local wind farm observations. One of the important considerations when setting up an NWP system for wind power forecasting is to collect and assimilate observations obtained at the wind farms. All wind turbines are equipped with Nacelle anemometers, which measure wind speed at a high temporal frequency. Although the wind measurements from individual turbines may present systematic errors caused by the disturbances generated by the turbine blades, aggregating clusters of observations from these Nacelle anemometers have been shown to be well-correlated with inflow wind [13,14] and, therefore, well-represent wind intensity for a mesoscale NWP model [15]. Figure2 compares the model forecast with and without assimilation of the hub-height (80 m) wind speed measurement for November 8-9, 2013. Assimilating the wind turbine hub-height wind significantly improved short-term wind predictions for this case, as well as others [15]. The performance of WRF RTFDDA was compared to that of the global forecast system (GFS) and North American model (NAM) to determine whether WRF RTFDDA significantly improved over the operational guidance provided by these National Center for Environmental Prediction (NCEP) models. Figure3 provides the mean absolute error (MAE) with an increasing forecast horizon (lead time). The scores were aggregated over a two-week period from 9–24 February, 2014. Energies 2019, 12, x FOR PEER REVIEW 4 of 17 Energies 2019, 12, x FOR PEER REVIEW 4 of 17 (NCEP)Energies 2020 models., 13, 1372 Figure 3 provides the mean absolute error (MAE) with an increasing forecast horizon4 of 16 (lead(NCEP) time). models. The scores Figure were 3 provides aggregated the m overean a absolute two-week error period (MAE) from with 9 an–24 increasing February ,forecast 2014. horizon (lead time). The scores were aggregated over a two-week period from 9–24 February, 2014.

FigureFigure 2. 2. ComparisonComparison of of hourly hourly data data assimilation assimilation and and forecasting forecasting cycles cycles without without ( (leftleft panel panel)) and and with with ((Figurerightright panel panel 2. Comparison)) assimilation assimilation of hourly of of wind wind data turbine turbine assimilation hub hub-height-height and windforecasting wind speed speed cyclesobservations observations without for for(left a a large largepanel wind wind) and farm farmwith inin(right Northern Northern panel Colorado.) Colorado. assimilation The The of black black wind lines lines turbine show show hub the the-height mean mean windwind. wind. speed Different Different observations color color lines lines for represent represent a large wind forecasts forecasts farm fromfromin Northern different different Colorado. cycles with The the black firstfirst (last)lines(last) showtwotwo digitsdig theits mean in in the the wind. legend legend Different denoting denoting color the the day linesday of of represent the the month month forecasts (hour (hour in inUTC).from UTC). different Conventional Conventional cycles obswith obs DA: the DA: conventional first conventional (last) two observation dig observationits in the data legend data assimilation assimilationdenoting and the andfarm day farm of ws the DA: ws month farmDA: (hourfarm wind windspeedin UTC). speed data Conventional assimilation.data assimilation. obs DA: conventional observation data assimilation and farm ws DA: farm wind speed data assimilation.

Figure 3. Comparison of the performance of RTFDDA with baseline numerical weather prediction FigureFigure 3. 3.Comparison Comparison ofof thethe(NWP performanceperformance) models. of MAE: RTFDDA mean with absolute baseline baseline error. numerical numerical weather weather prediction prediction (NWP) models. MAE: mean absolute(NWP) models. error. MAE: mean absolute error. The pairwise differences between WRF RTFDDA and the operational benchmarks are provided at theThe bottom pairwise of Fig didifferencesurefferences 3. The between95th percentile WRF RTFDDA confidence and intervals the operational applied benchmarks to the difference are provided curves indicateat the bottom that the of improvements FigureFigure3 3.. TheThe 95thare95th statistically percentilepercentile significant confidenceconfidence for intervalsintervals most of applied appliedthose hours. toto thethe During didifferencefference this period, curves WRFindicate RTFDDA that the (purple) improvements outperformed are statistically both GFS significantsignificant (red) and forfor NAM most of (green) those hours. out to During During 15 and this this 14 period,hours period,, respectivelyWRF RTFDDA RTFDDA, with (purple) (purple) the differences outperformed outperformed on the both order both GFS GFSof (red) 0.75 (red) and ms and-NAM1 in the NAM (green) first (green) few out hours. to out 15 and toAfter 15 14 andh,15 respectively, hours, 14 hours the , -1 1 -1 MAEwithrespectively thevalues diff erences,cluster with the onaround differences the order 1.8 ms of on 0.75 forthe ms allorder− thein of thethree 0.75 first models. ms few in hours. theThese fi Afterrst results few 15 hours. h, suggest the MAEAfter WRF values15 hours,RTFDDA cluster the 1 -1 providesaroundMAE values 1.8 value ms cluster− beyondfor allaround the threecoarser 1.8 models.ms operational for Theseall the models results three models. suggestout to 15 WRFThese hours. RTFDDA results suggest provides WRF value RTFDDA beyond theprovides coarser value operational beyond modelsthe coarser out operational to 15 hours. models out to 15 hours.

Energies 2020, 13, 1372 5 of 16

3. Statistical Postprocessing The NWP model forecasts, including both the customized WRF RTFDDA runs described above and output freely available from the national centers, can be combined and optimized to improve upon the best forecasts. NCAR uses the Dynamic Integrated Forecast System (DICast®) to seamlessly blend and optimize the wind speed forecasts [16]. DICast has been employed in numerous operational forecast systems developed by NCAR for forecasting temperature, , , irradiance, and other meteorological variables for applications, including short- and medium-range weather forecasts, surface transportation, energy applications, and other decision support applications. The AI system takes a two-step process: it first applies dynamic , which uses multilinear regression to remove bias from each model individually. It then generates a consensus forecast by using optimization methods to derive weighting coefficients to blend the models [1–3,16]. DICast thus constantly improves the forecast by learning from past errors based on comparisons of recent forecasts with observations. An advantage of DICast is that it best optimizes forecasts with a mere 90 days of data and can even produce skilled forecasts with as little as 30 days of training. The wind power forecasting system ingests real-time observations of both hub-height wind speed and power from most of the wind turbines. The system utilizes the Nacelle hub-height wind speeds and turbine power from each turbine, because it correlates best with the turbine power. Although the weather variables are what are typically produced by DICast, the utility requires an estimate of power production. Manufacturers provide power curves that relate wind speed to power; however, the actual power often deviates substantially due to site-specific influences such as elevation, terrain, and land cover. Thus, NCAR’s wind power forecasting system takes an empirical approach to power conversion. A dataset of historical fit Nacelle wind speeds (15 minute averages) and coincident power production for each type of turbine at each wind farm is data mined using the regression tree, Cubist (RuleQuest Research. https://www.rulequest.com/cubist-info.html); this allows identification of similar conditions during real-time operations in order to identify the correct portion of the empirical power curve to use for the conversion. The empirical power conversion model is based on the hierarchical regression tree model similar to the one described in the book Data Science for Wind Energy [9]. In the enhanced system, the historical data were divided into quantiles, and only the inner quantiles (such as the inner 50th percentile) were used for the training. Thus, we can avoid training to extreme events, including, curtailments and high-speed cutouts. The parameters that dictate the extreme quantiles to be removed are configurable by the user for each connection node. The system forecasts power for each wind turbine. Those forecasts are then rolled up (summed) to produce predictions for the connection node (typically, a single wind farm). If information is available regarding special conditions, such as planned outages (such as for maintenance), that is considered in finalizing the forecast. Figure4 is an example of the data-mined power curves for each decile. Thus, as seen by observing the spread of the percentile curves, the middle 50% of the power values could be considered as more representative of the power curve. This is a quality control approach that provides site-specific empirical power conversion that better represents the actual relationship between observed wind speed and produced power than the manufacturer’s power curve. Automatic verification is used to monitor the performance of the system by tracking the normalized mean absolute error (NMAE). The NMAE is obtained by normalizing the mean absolute error by the connection node maximum capacity to produce a percent error. The system runs at about 10% NMAE. Energies 2020, 13, 1372 6 of 16 Energies 2019, 12, x FOR PEER REVIEW 6 of 17

FigureFigure 4. 4. PowerPower curve curve for for a a wind wind farm farm indicating indicating percentile percentile curves. curves.

4. ProbabilisticAutomatic Predictionverification is used to monitor the performance of the system by tracking the normalizedUtilities mean such absolute as Xcel Energyerror (NMAE). require probabilisticThe NMAE is information obtained by to normalizing quantify the the uncertainty mean absolute in the eforecasts.rror by the The connection traditional node approach maximum in NWP capacity has been to produce to run a seta percent of model error. forecasts The system to form runs an ensemble at about 10%that representsNMAE. runs with different initial conditions, boundary conditions, physics packages, etc. Here, we employ a newer and more efficient AI approach of identifying analogs within a historical dataset, 4.including Probabilistic past deterministic Prediction forecasts and observations of the quantity to be predicted. Specifically, for a givenUtilities forecast such lead as time Xcel and Energy location, require we compareprobabilistic the currentinformation DICast to predictionquantify the with uncertainty past predictions in the forecasts.to determine The the traditional closest analogs. approach We in then NWP select has the been past to wind run speed a set observations of model forecasts that correspond to form an to ensemblethose analog that forecasts. represents Those runs observations with different form the initial AnEn conditions, prediction boundary for that lead conditions, time and location. physics packages,This procedure etc. Here is repeated, we employ for each a newer desired and lead more time efficient and location. AI approach Probabilistic of identifying forecasting, analogs including within aan historical example datasetof wind, speedincluding forecasting, past deterministic was reviewed forecasts by Gneiting and observations and Katzfuss of [17 the]. They quantity formalized to be predicted.the calibration Specifically, of predictive for a given distributions forecast andlead assessedtime and theirlocation, sharpness we compare by defining the current scoring DICast rules, predictionincluding thewith logarithmic past predictions score and to determine the continuous the closest ranked analogs. probability We then score. select The the AnEn past approach wind speed has observationsbeen shown totha maintaint correspond or improve to those the analog accuracy forecasts. of the deterministicThose observations prediction form that the isAnEn used prediction to build it, foras wellthat aslead providing time and an location. accurate This and procedure reliable probabilistic is repeated forecastfor each baseddesired on lead the metricstime and defined location. by ProbabilisticGneiting and forecasting, Katzfuss [18 including,19]. While an example for short-term of wind forecasting speed forecasti an effng,ective was forecasting reviewed by approach Gneiting is andbased Katzfuss on the calibrated [17]. They regime-switching formalized the methodcalibration [8,20 of], the predictive AnEn was distributions selected because and itassess combinesed their the sharpnessNWP model by output defining with scoring observations, rules, including and it istherefore the logarithmic applicable score to forecasting and the continuous at any timeframe. ranked probabilityHere, wescore. have The applied AnEn theapproa AnEnch has to generate been shown wind to forecasts maintain at or over improve 67 wind the farmsaccuracy located of the in deterministicColorado and prediction Wyoming managedthat is used by Xcel to build Energy. it, as Wind well observations as providing were an available accurate for and a 223-dayreliable probabilisticperiod (from forecast 19 October based 2013 on the to 11 metrics June 2014)defined together by Gneiting with DICastand Katzfuss daily forecasts[18,19]. While of wind for speed,short- termwind directionforecasting at the an hub-height, effective forecasting and approach startingis based at on 12 UTC the calibratedto generate regime 1–73 h predictions.-switching methodThe last [8,20], 35 days the of AnEn this period was selected were used because to test it thecombines AnEn performancethe NWP model and output to carry with out observations a comparison, andwith it DICastis therefore wind applicable speed predictions. to forecasting The at remaining any timefra portionme. of the dataset was used for training. The setHere, of optimalwe have weights applied isthe defined AnEn byto choosing,generate wind independently forecasts at for over each 67 station, wind farms the combination located in Coloradothat minimizes and Wyoming the continuous managed ranked by Xcel probability Energy. score Wind (CRPS) observations over the were training available period, for defined a 223-day as: period (from 19 October 2013 to 11 June 2014) together with DICast daily forecasts of wind speed, N P R h f i2 at the hub-height,CRPS and =sea1 level ∞pressurep (x) startingpo(x) dxat 12 UTC to generate 1–73 hour N i − i predictions. The last 35 days of this period werei=1 −∞ used( to test the AnEn performance and to carry out(1) 0 x < o a comparison with DICast wind speed predictions.po(x) = The remainingi portion of the dataset was used i 1 x o for training. The set of optimal weights is defined by choosing,≥ i independently for each station, the combination that minimizes the continuous rankedf probability score (CRPS) over the training period, where N is the total number of observations, p (x) is the cumulative distribution function (CDF) of the defined as: i o( ) forecast variable being less or equal to x at the -time location of observation i, and pi x is the

Energies 2019, 12, x FOR PEER REVIEW 7 of 17

푁 ∞ 1 2 퐶푅푃푆 = ∑ ∫ [푝푓(푥) − 푝표(푥)] 푑푥 푁 푖 푖 푖=1 −∞

표 0 푥 < 표푖 (1) 푝푖 (푥) = { 1 푥 ≥ 표푖 푓 where 푁 is the total number of observations, 푝푖 (푥) is the cumulative distribution function (CDF) of Energies 2020, 13, 1372 표 7 of 16 the forecast variable being less or equal to 푥 at the space-time location of observation 푖, and 푝푖 (푥) is the CDF of the observation (a Heaviside function). CRPS is a negatively oriented metric (i.e., lower scoresCDF of are the better) observation with a (aperfect Heaviside score function).of 0. CRPS is a negatively oriented metric (i.e., lower scores are better)We determine with a perfect optimal score weights of 0. in the analog matching metric following [21] for each of the four analogWe predictors. determine Fig optimalure 5 provides weights in an the example analog of matching the AnEn metric forecasts, following where [21 AnEn] for each means of are the comparedfour analog with predictors. wind speed Figure predictions5 provides obtained an example from of theDICast. AnEn Thus, forecasts, the AnEn where maintains AnEn means the accuracyare compared of the with DICast wind speed deterministic predictions prediction obtained fromwhile DICast. additionally Thus, the providing AnEn maintains uncertainty the quantification.accuracy of the This DICast example deterministic also shows prediction the usefulness while additionally of a probabilistic providing uncertaintyprediction; quantification.although the deterministicThis example predictions also shows underestimate the usefulness some of a ofprobabilistic the highest prediction; speeds (e.g., although lead time the hour deterministic 65), the ensemblepredictions distribution underestimate (represented some of the by highestthe shading) speeds does (e.g., provide lead time for hour the high 65), theobserved ensemble value distribution a certain level(represented of likelihood. by the shading) does provide for the high observed value a certain level of likelihood.

FigureFigure 5. 5. ExampleExample of of an an analog analog ensemble forecast probabilityprobability density density function function (PDF) (PDF over) over one one station station of ofthe the dataset. dataset. The The blue blue shadings shadings correspond correspond to the to 25–75the 25 (–darker75 (darker) and) 5–95and 5 (–lighter95 (lighter) quantiles.) quantiles. The blackThe blackand yellow and yellow dashed dashed lines represent lines represent the wind the speed wind observations speed observations and AnEn and ensemble AnEn mean, ensemble respectively. mean, respectively.The red line isThe the red DICast line is wind the DICast speed forecast.wind speed AnEn: forecast. analog AnEn: ensemble. analog ensemble. In Figure6, the AnEn mean is compared to DICast in terms of the root mean square error (RMSE) In Figure 6, the AnEn mean is compared to DICast in terms of the root mean square error (RMSE) computed from forecast hour 1 to 73 and over all the available stations. The bootstrap confidence computed from forecast hour 1 to 73 and over all the available stations. The bootstrap confidence intervals suggest that there is no statistically significant difference between the AnEn and DICast intervals suggest that there is no statistically significant difference between the AnEn and DICast performances for these forecasts. Hence, the AnEn mean can maintain the same level of accuracy as performances for these forecasts. Hence, the AnEn mean can maintain the same level of accuracy as DICast for the deterministic prediction. DICast for the deterministic prediction. Figure7 depicts the Brier skill score (BSS) [ 16] of the AnEn computed over all the stations with DICast as a reference. An assessment of important probabilistic prediction attributes such as reliability and resolution can be performed by calculating the Brier score (BS) components. The Brier score is similar to the RMSE for a deterministic forecast. It is computed as:

N 1 X 2 BS = (pn on) (2) N − n=1

where N is the total number of forecast/observation pairs, p is the forecasted probability of a categorical 1 event (e.g., in this case, wind speed greater than 10 ms− ), and on is the categorical observation (0 if the event does not occur and 1 if it does occur). A lower value of the Brier score indicates better performance, with 0 corresponding to a perfect forecast. Using the DICast forecast as a reference, the Brier skill score (BSS) is calculated as:

BS BSS = 1 (3) − BSDICast Energies 2020, 13, 1372 8 of 16

where BSDICast is the BS of DICast. In the case of DICast, pn in (2) can only take 1 or 0 values, e.g., if the 1 forecasted wind speed is greater or lower than 10 ms− , respectively. The BSS represents the measure of the probabilistic forecast performance in comparison to the reference forecast and measures the ability of the model to issue a better probabilistic forecast than the reference. Positive (negative) values ofEnergies BSS indicate 2019, 12, better x FOR PEER (worse) REVIEW performances of AnEn than DICast. 8 of 17

Figure 6. Root mean square error (RMSE) as a function of forecast lead time for DICast (red) and AnEn Figure 6. Root mean square error (RMSE) as a function of forecast lead time for DICast (red) and means (black) computed over all stations. Bootstrap 5%–95% confidence intervals are plotted for AnEn AnEn means (black) computed over all stations. Bootstrap 5%–95% confidence intervals are plotted Energiesonly. RMSE2019, 12 values, x FOR PEER listed REVIEW are computed by including all lead times. 9 of 17 for AnEn only. RMSE values listed are computed by including all lead times.

Figure 7 depicts the Brier skill score (BSS) [16] of the AnEn computed over all the stations with DICast as a reference. An assessment of important probabilistic prediction attributes such as reliability and resolution can be performed by calculating the Brier score (BS) components. The Brier score is similar to the RMSE for a deterministic forecast. It is computed as: 푁 1 퐵푆 = ∑ (푝 − 표 )2 푁 푛 푛 (2) 푛=1 where 푁 is the total number of forecast/observation pairs, 푝 is the forecasted probability of a −1 categorical event (e.g., in this case, wind speed greater than 10 ms ), and 표푛 is the categorical observation (0 if the event does not occur and 1 if it does occur). A lower value of the Brier score indicates better performance, with 0 corresponding to a perfect forecast. Using the DICast forecast as a reference, the Brier skill score (BSS) is calculated as: 퐵푆 퐵푆푆 = 1 − (3) 퐵푆퐷퐼퐶푎푠푡

where BSDICast is the BS of DICast. In the case of DICast, pn in (2) can only take 1 or 0 values, e.g., if the forecasted wind speed is greater or lower than 10 ms-1, respectively. The BSS represents the measure of the probabilistic forecast performance in comparison to the reference forecast and measures the ability of the model to issue a better probabilistic forecast than the reference. Positive (negative) values of BSS indicate better (worse) performances of AnEn than DICast. FigureFigure 7. Brier 7. Brier skill skill score score (BSS) (BSS) of of thethe analog ensemble ensemble (AnEn) (AnEn) with with DICast DICast as a asreference, a reference, as a function as a function ofFig forecastofure forecast 7 lead indicates lead time, time, computed the computed benefit over over of all allusing the stations. a probabilistic The The BSS BSS 5% forecast 5%–95%–95% bootstrap bootstrapsystem confidence such confidence as intervals AnEn, intervals rather than −1 1 a deterministicare alsoare also shown. shown. one. The The eventAlthough event considered considered the AnEn isis wind and speed speed DICast greater greater than show than 10 10ms a mssimilar. − . level of performance when compared as deterministic systems (Figure 5), a probabilistic score such as the BSS indicates that AnEn5. Short improves-range Forecasting upon DICast deterministic forecasts by about 20%–30% for most lead times. Large variations in available wind power over a short time (~30 min), or ramps, represent a challenge for grid operators. The system should ideally predict the timing, magnitude, and duration of these ramps. The latency of availability of NWP model output combined with the inherent uncertainties in the timing of wind ramps makes the use of NWP for these short-range forecasts quite challenging. The best strategies for these nowcasts (0 to about 3 h ahead) rely on observations near the wind farm. NCAR evaluated two systems: an AI observation-based expert system that utilizes upstream observations and the variational doppler radar analysis system (VDRAS) [22]. As described previously [1], VDRAS is based on a numerical scale model that produces high-resolution boundary-layer wind fields. The key to its success is in assimilating radar radial data obtained from reflectivity measurements. The VDRAS system has proven useful for many applications, including at defense ranges, for homeland security applications, and internationally in support of the Olympics [23]. VDRAS is run at about 1–4-km spatial resolution and is able to produce forecasts as frequently as every 10 minutes. It produces wind, thermodynamical, and microphysical analyses. VDRAS first assimilates data from various sources, including background weather data from the WRF RTFDDA model forecasts, surface network observations, and radar data. For efficient, operationally feasible simulation runs, the VDRAS domain must be limited in size yet big enough to include the nearby doppler radar sites. VDRAS simulations are used for short-range forecasting. These simulations enable distinguishing large-scale features such as cold fronts from gust fronts or low-level jets, as well as other weather phenomena characterized by strong wind gradients. The cloud-scale VDRAS model provides high-resolution forecasts resolving identified features and their location with up to two-hour lead times. While VDRAS-based wind nowcasts have shown some promise in case studies and real-time demonstrations, considerable effort is being undertaken to tackle the challenging problem of wind ramp prediction. NCAR’s short-range expert forecasting system leverages publicly available surface observations of 10-m wind speeds upstream of the wind farms. The expert system is designed to predict ramps

Energies 2020, 13, 1372 9 of 16

Figure7 indicates the benefit of using a probabilistic forecast system such as AnEn, rather than a deterministic one. Although the AnEn and DICast show a similar level of performance when compared as deterministic systems (Figure5), a probabilistic score such as the BSS indicates that AnEn improves upon DICast deterministic forecasts by about 20%–30% for most lead times.

5. Short-range Forecasting Large variations in available wind power over a short time (~30 min), or ramps, represent a challenge for grid operators. The system should ideally predict the timing, magnitude, and duration of these ramps. The latency of availability of NWP model output combined with the inherent uncertainties in the timing of wind ramps makes the use of NWP for these short-range forecasts quite challenging. The best strategies for these nowcasts (0 to about 3 h ahead) rely on observations near the wind farm. NCAR evaluated two systems: an AI observation-based expert system that utilizes upstream observations and the variational doppler radar analysis system (VDRAS) [22]. As described previously [1], VDRAS is based on a numerical cloud scale model that produces high-resolution boundary-layer wind fields. The key to its success is in assimilating radar radial velocity data obtained from reflectivity measurements. The VDRAS system has proven useful for many applications, including at defense ranges, for homeland security applications, and internationally in support of the Olympics [23]. VDRAS is run at about 1–4-km spatial resolution and is able to produce forecasts as frequently as every 10 minutes. It produces wind, thermodynamical, and microphysical analyses. VDRAS first assimilates data from various sources, including background weather data from the WRF RTFDDA model forecasts, surface network observations, and radar data. For efficient, operationally feasible simulation runs, the VDRAS domain must be limited in size yet big enough to include the nearby doppler radar sites. VDRAS simulations are used for short-range forecasting. These simulations enable distinguishing large-scale features such as cold fronts from thunderstorm gust fronts or low-level jets, as well as other weather phenomena characterized by strong wind gradients. The cloud-scale VDRAS model provides high-resolution forecasts resolving identified features and their location with up to two-hour lead times. While VDRAS-based wind nowcasts have shown some promise in case studies and real-time demonstrations, considerable effort is being undertaken to tackle the challenging problem of wind ramp prediction. NCAR’s short-range expert forecasting system leverages publicly available surface observations of 10-m wind speeds upstream of the wind farms. The expert system is designed to predict ramps related to changes in power at a wind farm in 15-min intervals with two hours’ lead time. Observing sites are grouped in up to twelve concentric rings approximately 12-km-wide centered on a wind farm (see Figure8). The system detects wind ramp signatures upstream from a wind farm by calculating a ramp metric based on all the observations in a group. For a given forecast lead time, an average ramp metric is computed based on all the relevant group ramp metrics. To best predict ramps, the VDRAS output is blended with the expert system results. Each of these two forecasting approaches is assigned a confidence level. The blended output describes the change in power production expected at each farm that is used to define a ramp metric. For a given lead time, the change in power production at each farm is calculated by multiplying the farm capacity by the ramp metric computed from the blended short-range system. The regional ramp forecast is created by aggregating the predicted changes in power across all the wind farms within the region, providing the utility information about the aggregate impact of the wind event. Energies 2019, 12, x FOR PEER REVIEW 10 of 17

related to changes in power at a wind farm in 15-min intervals with two hours’ lead time. Observing sites are grouped in up to twelve concentric rings approximately 12-km-wide centered on a wind farm (see Figure 8). The system detects wind ramp signatures upstream from a wind farm by Energiescalculating2020, 13a ,ramp 1372 metric based on all the observations in a group. For a given forecast lead time,10 of an 16 average ramp metric is computed based on all the relevant group ramp metrics.

Figure 8.8. SchematicSchematic of of the the observations-based observations-based expert expert system system for short-term for short-term forecasting. forecasting. Available Available wind speedwind speed observations observations at 10 mat in10 concentricm in concentric circles circles around around a wind a farmwind arefarm used are toused calculate to calculate the wind the rampwind ramp metric, metric which, which is then is used then toused compute to compute changes change in powers in power production. production. 6. Predicting Extreme Weather To best predict ramps, the VDRAS output is blended with the expert system results. Each of theseThe two e ffforecastiects of iceng accretionapproaches on windis assigned turbines a confidence present a major level. challengeThe blended to operators, output describes because the accretedchange in ice power will result production in less powerexpected output at each than farm for athat clean is used turbine to define blade. a This ramp needs metric. to be For accounted a given forlead to time, produce the change an accurate in power power production forecast. at Thus, each a farm subsystem is calculated is included by multip in thelying wind the power farm capacity forecast systemby the ramp for identifying metric computed not only icingfrom conditionsthe blended but short also- possiblerange system. extreme The regional ramp and forecast high wind is speeds.created by The aggregating latter two hazards the predicted may also changes curtail in power power production across all due the towind operational farms within decisions the region, to shut downproviding turbines. the utility This information subsystem, theabou extremet the aggregate weather impact system of or the ExWx, wind combines event. wind and weather forecasts from the DICast forecasting system with the WRF RTFDDA model output and simple rules based6. Predicting on the forecaster’sExtreme Weather experience. The result is an AI expert system that is highly configurable to suit a wide range of operator needs. The effects of ice accretion on wind turbines present a major challenge to operators, because the Development of the extreme weather system was modeled extensively on expert systems currently accreted ice will result in less power output than for a clean turbine blade. This needs to be accounted used in forecasting and diagnosing icing conditions for [24,25]. These algorithms employ for to produce an accurate power forecast. Thus, a subsystem is included in the wind power forecast adaptations of fuzzy logic theory and membership functions to combine datasets from multiple sources system for identifying not only icing conditions but also possible extreme temperatures and high and to diagnose the current state or predict the future state. In ExWx, these same concepts are applied wind speeds. The latter two hazards may also curtail power production due to operational decisions to forecasts of icing conditions, high wind speed events, and low and high temperature extremes. to shut down turbines. This subsystem, the extreme weather system or ExWx, combines wind and 6.1.weather Input forecasts Forecasts from the DICast forecasting system with the WRF RTFDDA model output and simple rules based on the forecaster’s experience. The result is an AI expert system that is highly configurableOptimal to configuration suit a wide range of the of ExWxoperator includes needs. high-resolution NWP model (WRF RTFDDA— SectionDevelopment2) model output of the with extreme output from weather the DICast system forecasting was modeled system extensively (Section3). The on expertWRF RTFDDA systems modelcurrently output used provides in forecasting the ExWx and with diagnosing forecasts icing of temperature, conditions for liquid aircraft cloud [2 droplets,4,25]. These and algorithms liquid dropsemploy at adaptationsa 3-km horizontal of fuzzy grid logic spacing theory with and variable membership vertical grid functions spacing to that combine is higher datasets in the lower from levelsmultiple of thesources atmosphere. and to diagnose Forecasts the of liquidcurrent water state (cloudor predict and the rain) future content state. from In WRFExWx, RTFDDA these sa areme generated by the Thompson microphysics parameterization [26] implemented in the WRF simulations.

Forecasts from the WRF RTFDDA system are available with lead times ranging from 0–48 h. DICast forecasts are available at select points within the operating region of a particular wind farm. These points must be chosen a priori and, in the current system, were selected at a horizontal Energies 2019, 12, x FOR PEER REVIEW 11 of 17

concepts are applied to forecasts of icing conditions, high wind speed events, and low and high temperature extremes.

6.1. Input Forecasts Optimal configuration of the ExWx includes high-resolution NWP model (WRF RTFDDA— Section 2) model output with output from the DICast forecasting system (Section 3). The WRF RTFDDA model output provides the ExWx with forecasts of temperature, liquid cloud droplets, and liquid rain drops at a 3-km horizontal grid spacing with variable vertical grid spacing that is higher in the lower levels of the atmosphere. Forecasts of liquid water (cloud and rain) content from WRF RTFDDA are generated by the Thompson microphysics parameterization [26] implemented in the WRF simulations. Forecasts from the WRF RTFDDA system are available with lead times ranging Energiesfrom 02020–48 ,h.13 , 1372 11 of 16 DICast forecasts are available at select points within the operating region of a particular wind farm. These points must be chosen a priori and, in the current system, were selected at a horizontal spacing approximately equivalent to the coarsest horizontal grid spacing of all of the models used by spacing approximately equivalent to the coarsest horizontal grid spacing of all of the models used by DICast. At each of these points, DICast provides a forecast of icing conditions at the surface through DICast. At each of these points, DICast provides a forecast of icing conditions at the surface through a variable known as the conditional probability of icing (CPOI). The calculation of CPOI in DICast a variable known as the conditional probability of icing (CPOI). The calculation of CPOI in DICast uses explicit model forecasts of surface precipitation types that could lead to turbine icing, as well as uses explicit model forecasts of surface precipitation types that could lead to turbine icing, as well as temperature and wet-bulb temperature, to determine the final likelihood of icing. The ExWx system temperature and wet-bulb temperature, to determine the final likelihood of icing. The ExWx system produces forecasts for lead times 0–72 h (Figure9). produces forecasts for lead times 0–72 h (Figure 9).

FigureFigure 9.9. SummarySummary schematicschematic visualizingvisualizing the variousvarious input datasetsdatasets toto thethe extremeextreme weatherweather systemsystem (ExWx).(ExWx). ForFor DICast,DICast, C1-C4C1-C4 indicate various configurationsconfigurations of the DICast forecast thatthat areare dependentdependent onon thethe inputinput modelmodel datadata availableavailable atat thatthat leadlead time.time.

6.2.6.2. Forecasts TheThe WRF and DICast DICast model model output outputss do do not not provide provide the the same same information information for foricing icing forecasts. forecasts. To Toutilize utilize both both forecasts, forecasts, an an icing icing potential potential is is computed. computed. For For the the WRF WRF output, output, this this is is simply simply at each grid box,box, whilewhile forfor DICast,DICast, itit isis forfor eacheach availableavailable locationlocation nearby.nearby. TheThe icingicing potentialpotential isis anan uncalibrateduncalibrated likelihoodlikelihood thatthat icingicing conditionsconditions willwill bebe possiblepossible givengiven thethe forecastforecast inputinput data.data. This This unitlessunitless valuevalue rangesranges fromfrom 0 0 to to 1, 1, with with 1 being1 being the the certainty certainty of icing of icing conditions. conditions. The ExWxThe ExWx system system is configured is configured based onbased testing on testing and evaluation and evaluation of the inputof the datasetsinput datasets currently currently in use. in use. FigureFigure 10 showsshows the the relationship relationship between between three three WRF WRF output output variables variables (height, (height, temperature, temperature, and andsupercooled supercooled liquid liquid cloud cloud droplets droplets and and rain rain drops) drops) use usedd to to compute compute the the icing icing potential potential and the the likelihoodlikelihood of of icing icing for for each each one. one. The The icing icing potential potential is computed is computed at each height at each below height a user-configured below a user- ceiling,configured above ceiling, which above icing which conditions icing areconditions not considered are not toconsidered have an e fftoect have at the an hub-height.effect at the Icinghub- potentialheight. Icing is maximized potential is when maximized each of thewhen three each WRF of the output three variables WRF output fall within variables their fall optimal within range. their Asoptimal any of range. the three As WRFany of output the three variables WRF deviateoutput fromvariables their deviate optimal from range, their the icingoptimal potential range, decreases the icing appropriatelypotential decreases based appropriately on the slope based of the on curve the slope for that of the variable. curve for For that DICast, variab valuesle. For of DICast, CPOI values above 0.4of CPOI will maximize above 0.4 the will icing maximize potential the forecast. icing potential Both the forecast. WRF and Both DICast the WRF curves and are DICast configurable curves andare customizable in order to cater to specific changes in input datasets and/or operator needs.

6.3. Wind and Temperature Forecasts Maximum and minimum temperature forecasts are also provided by the ExWx. These forecasts again leverage both DICast and the WRF RTFDDA model output. Unlike for icing, both DICast and the WRF RTFDDA produce temperature forecasts natively at each grid box in the WRF and at each DICast site. This makes the combination of DICast and WRF temperature forecasts simpler than the icing forecasts. The other difference is that there is no derived variable provided to the operator; the temperature forecast is simply provided verbatim, and decisions about impacts are made at the display level. The same is true for high wind speed forecasts, which could be enhanced by adding a wind gust forecast. Energies 2019, 12, x FOR PEER REVIEW 12 of 17 Energies 2020, 13, 1372 12 of 16 configurable and customizable in order to cater to specific changes in input datasets and/or operator needs.

FigureFigure 10. 10.Curves Curves representing representing thethe interestinterest in icing conditions conditions (0 (0–1)–1) for for three three variables variables from from the the WRF WRF RTFDDARTFDDA model model data. data. TopTop panelpanel isis heightheight of the model level, level, middle middle panel panel represents represents supercooled supercooled liquidliquid water, water, (i.e., (i.e. liquid, liquid rain orrain cloud or cloud drops belowdrops freezing), below freezing), and the andbottom the panel bottom shows pan temperature.el shows Thetemperature. intersection The of theintersection three curves of the where three theycurves return where a they value return of 1 will a value maximize of 1 will the maximize icing potential the fromicing the potential WRF RTFDDA from the modelWRF RTFDDA data. SLW: model supercooled data. SLW: liquid supercooled water. liquid water.

6.4.6.3. Turbine Wind and Forecasts Temperature Forecasts ToMaximum collect and and present minimum the data temperature from the icingforecasts and temperatureare also provided forecasts, by the a ExWx. configurable These systemforecasts was developedagain leverage to produce both DICast a value and of the the icing WRF potential RTFDDA and model temperatures output. Unlike (both for maximum icing, both and DICast minimum) and at eachthe individual WRF RTFDDA wind produce turbine. temperature The region offorecasts influence natively for both at DICasteach grid forecast box in sitesthe WRF and WRFand at RTFDDA each modelDICast grid site. boxes This is makes configurable. the combination Thus, if theof DICast underlying and WRF NWP temperature model grid forecasts within the simpler operating than regionthe changesicing forecasts. the horizontal The other grid difference spacing oris thethat numberthere is no of DICastderived sitesvariable in an provided operating to the region operator; increases the or decreases,temperature the user forecast can adjust is simply the sizeprovided of these verbatim regions, and that decisions influence abanout individual impacts turbine are made accordingly. at the displayOnce level. the population The same is of true influencing for high pointswind speed has been forecasts, identified, which the could icing be potential enhanced and by temperatureadding a wind gust forecast. forecasts are combined to produce a single value for each wind turbine in the operating region. For icing potential forecasts, a percentile threshold is applied to all WRF RTFDDA model points and DICast points. The maximum forecast from either the DICast or the WRF is then chosen as the final icing potential forecast for that turbine, unless only a single input is available; in which case, the icing Energies 2020, 13, 1372 13 of 16

potential forecast for that turbine will simply be the icing potential forecast from whichever dataset is available. The percentile used to threshold the influencing data is also configurable. This allows for a dynamic system which enables wind farm operators to tailor the guidance to their preferences and experiences. For temperature forecasts, the average temperature of all DICast points in the immediate region and for WRF RTFDDA model points are computed separately. If both forecasts are available, then the final temperature forecast is the average of those two values; otherwise it is set to either the DICast or WRF forecast only, whichever is available. Additional processing can combine turbine output at the farm, node, or region level.

6.5. Case Study Since the extreme weather system runs hourly, it generates a new 0 to 72-h prediction every hour. ExWx performs optimally between hours 0 and 48, when both the DICast forecast system and WRF model output are available. As a case study, we analyze an icing event that occurred on 26 December 2014 at a wind farm located in Minnesota. The model output shown for this case combined forecasts at each of the turbines in the farm. Based on wind power and forecast outputs for the farm, the event window was chosen to be nine hours, centered on 0000 UTC on 26 December. Selecting a threshold of 0.5 for the icing potential highlights the capability of the system to predict the icing conditions at the

Energieswind farm 2019, up12, x to FOR 48 PEER h prior REVIEW to the event (Figure 11). 14 of 17

Figure 11. 11. ForecastsForecasts of of icing icing potential, potential, combined combined from fromall turbines all turbines on a single on farm, a single from farm, 57 consecutive from 57 consecutiveruns of the extremeruns of the weather extreme system weather (y-axis). system The (y- firstaxis). run The shown first run is theshown 2000 is UTCthe 2000 forecast UTC fromforecast the fromextreme the weatherextreme systemweather on system 24 December, on 24 December, 2014. Black 2014. shading Black indicatesshading indicates valid times valid within times the within event thewindow event(2000 window UTC (2000 on 25 UTC December–0400 on 25 December UTC–0400 on 26 UTC December) on 26 December) with an icing with potential an icing forecast potential of 0.5 forecast within ofthe >event 0.5 within window, theorange event window,shading indicatesorange shading an icing indicates potential an forecasticing potential of > 0.5 forecast outside of the > 0.5 event outside window, the event and window, gray shading and grayindicates shading an icingindicates potential an icing forecast potential of < fo0.5recast outside of < the 0.5 eventoutside window. the event Vertical window. lines Vertical indicate lines the indicatedivision the between division C1, between C2, and C1, C3 DICastC2, and forecast C3 DICast system forecast data system (see Figure data9 ).(see Figure 9).

To provide the best configuration, a thorough database of known icing events provided via operator information and/or an icing sensor located at a wind farm is preferred. However, initial examination of several known icing cases shows promising performances of the system at alerting an operator of potential icing conditions with one to two days of lead time.

Energies 2020, 13, 1372 14 of 16

With limited truth data, robust statistical verification of the icing potential forecasts from the extreme weather system was challenging. Much of the initial configuration was based on nontraditional data sources, such as operator’s logs. In order to infer icing conditions, a metric was developed to identify them based on the wind turbine power output. This metric is simply a wind speed weighted ratio of the observed power to the expected power based on the turbine manufacturer’s power curve. This methodology must be used in conjunction with other data, such as icing sensors and/or operator-provided confirmation of icing conditions, in order to exclude those time periods when curtailment of the wind turbines may have occurred. Several possible icing periods are shown in Figure 12. For the two days in this particular event, there appear to have been three distinct periods of possible icing conditions (Figure 11). The ExWx forecasts of icing potential > 0.5 (Figure 11) are roughly coincident with the decreased power production shown in Figure 12. Additional information Energiesobtained 2019 from, 12, x theFORoperator PEER REVIEW confirmed that icing occurred during these time periods. 15 of 17

FigureFigure 12. 12. WindWind speed speed weighted weighted metric metric to to identify identify periods periods when when power power production production was was less less than than expectedexpected and and wind wind speeds speeds were were high high enough enough to to have have confidence confidence in in the the manufacturer manufacturer’s’s power power curve curve (i.e.(i.e.,, not not in in the the lower lower tail tail of of the the curve). curve). The The red red line line shows shows t thehe farm farm average, average, while while the the black black dashed dashed lineslines show show +/+/- -the the standard standard deviation deviation for for the the farm. farm. Top Top panel panel is is for for 25 25 December December 2014 2014,, and and bottom bottom panelpanel is is for for 26 26 December December 2014. 2014. Higher Higher values values indicate indicate a a higher higher likelihood likelihood of of icing icing (or (or curtailment). curtailment).

7. ConclusionsTo provide the best configuration, a thorough database of known icing events provided via operator information and/or an icing sensor located at a wind farm is preferred. However, initial A comprehensive wind power forecasting system developed in collaboration with Xcel Energy examination of several known icing cases shows promising performances of the system at alerting an integrates forecasting at a range of time scales utilizing an observation-based expert system and the operator of potential icing conditions with one to two days of lead time. radar assimilation model, VDRAS, for short-term forecasting and global model output with limited area high-resolution WRF RTFDDA modeling with several artificial intelligence approaches. The statistical, machine-learning system, DICast® , has been enhanced, as have the power conversion algorithms, through use of a regression tree algorithm and by deploying a quantile data quality control scheme. Finally, an AI analog ensemble approach provides both probabilistic information and improves upon the deterministic forecast. Additional modules provide estimates of extreme events, including icing, high winds, and extreme temperatures. Xcel Energy is employing wind power forecasting for economic reasons, as it allows them to effectively integrate wind power into their operations. Since forecast errors are real costs in the energy market, more accurate forecasts can save utilities and their ratepayers substantial amounts of money. Xcel Energy estimates a savings of over $60M in the past six years due to lowered day-ahead forecast errors. They depend on accurate forecasts more as their wind power capacity increases in order to integrate the variable wind resources into the grid more efficiently, economically, and reliably. Thus, highly accurate wind power forecasts enable renewable energy to effectively increase its fraction in the energy portfolio. These technologies have been reported at workshops and conferences [27], where they generate interest and are picked up by commercial forecast providers.

Energies 2020, 13, 1372 15 of 16

7. Conclusions A comprehensive wind power forecasting system developed in collaboration with Xcel Energy integrates forecasting at a range of time scales utilizing an observation-based expert system and the radar assimilation model, VDRAS, for short-term forecasting and global model output with limited area high-resolution WRF RTFDDA modeling with several artificial intelligence approaches. The statistical, machine-learning system, DICast®, has been enhanced, as have the power conversion algorithms, through use of a regression tree algorithm and by deploying a quantile data quality control scheme. Finally, an AI analog ensemble approach provides both probabilistic information and improves upon the deterministic forecast. Additional modules provide estimates of extreme events, including icing, high winds, and extreme temperatures. Xcel Energy is employing wind power forecasting for economic reasons, as it allows them to effectively integrate wind power into their operations. Since forecast errors are real costs in the energy market, more accurate forecasts can save utilities and their ratepayers substantial amounts of money. Xcel Energy estimates a savings of over $60M in the past six years due to lowered day-ahead forecast errors. They depend on accurate forecasts more as their wind power capacity increases in order to integrate the variable wind resources into the grid more efficiently, economically, and reliably. Thus, highly accurate wind power forecasts enable renewable energy to effectively increase its fraction in the energy portfolio. These technologies have been reported at workshops and conferences [27], where they generate interest and are picked up by commercial forecast providers.

Author Contributions: Conceptualization, B.K. and S.E.H.; methodology, D.A., G.W., L.D.M., Y.L., S.L., and M.P.; software, P.P.; validation, T.J.; formal analysis, D.A., S.A., W.C., and S.L.; investigation, S.A. and G.W.; resources, S.E.H. and G.W.; data curation, G.W., S.L., and P.P.; writing—original draft preparation, B.K. and S.E.H.; writing—review and editing, B.K. and S.E.H.; visualization, B.K., D.A., S.A., G.W., L.D.M., S.L., and W.C.; supervision, S.E.H.; project administration, S.E.H.; and funding acquisition, S.E.H. All authors have read and agreed to the published version of the manuscript. Funding: This research was funded by Xcel Energy. Acknowledgments: The authors gratefully acknowledge the contributions of the rest of the project team and of the Xcel Energy oversight team. Conflicts of Interest: The authors declare no conflicts of interest.

References

1. Mahoney, W.P.; Parks, K.; Wiener, G.; Liu, Y.; Myers, W.L.; , J.; Monache, L.D.; Hopson, T.; Johnson, D.; Haupt, S.E. A Wind Power Forecasting System to Optimize Grid Integration. IEEE Trans. Sustain. Energy 2012, 3, 670–682. [CrossRef] 2. Haupt, S.E.; Mahoney, W.P.; Parks, K. Wind Power Forecasting; Springer Science and Business Media LLC: New York, NY, USA, 2014; pp. 295–318. 3. Haupt, S.E.; Mahoney, W.P.; Parks, K. Wind Power Forecasting. Weather Energy 2014, 295–318. 4. Giebel, G.; Kariniotakis, G. Best Practice in Short-term Forecasting—A Users Guide. In Proceedings of the European Wind Energy Conference and Exhibition, Milan, Italy, 7–10 May 2007. 5. Monteiro, C.; Bessa, R.; Miranda, V.; Botterud, A.; Wang, J.; Conzelmann, G.; Porto, I. Wind power forecasting: State-of-the-art 2009. In Proceedings of the Argonne National Laboratory, Argonne, IL, USA, 20 November 2009. 6. Ahlstrom Jone, L.; Zavadil, R.; Grant, W. The future of wind forecasting and utility operations. IEEE Power Energy Mag. 2005, 3, 57–64. [CrossRef] 7. Wilczak, J.; Finley, C.; Freedman, J.; Cline, J.; Bianco, L.; Olson, J.; Djalalova, I.; Sheridan, L.; Ahlstrom, M.; Manobianco, J.; et al. The Wind Forecast Improvement Project (WFIP): A Public-Private Partnership Addressing Wind Energy Forecast Needs. Bull. Am. Meteorol. Soc. 2015, 96, 1699–1718. [CrossRef] 8. Gneiting, T.; Larson, K.; Westrick, K.; Genton, M.G.; Aldrich, E. Calibrated Probabilistic Forecasting at the Stateline Wind Energy Center. J. Am. Stat. Assoc. 2006, 101, 968–979. [CrossRef] 9. Ding, Y. Data Science for Wind Energy, 1st ed.; CRC: Boca Raton, FL, USA, 2019; p. 424. Energies 2020, 13, 1372 16 of 16

10. Skamarock, W.C.; Klemp, J.B.; Dudhia, J.; Gill, D.O.; Barker, D.M.; Duda, M.G.; Huang, X.-Y.; Wang, W.; Powers, J.G. A Description of the Advanced Research WRF Version 3; NCAR Tech Note NCAR/TN 475 STR; National Center for Atmospheric Research: Boulder, CO, USA, 2008; p. 125. 11. Benjamin, S.G.; Weygandt, S.S.; Brown, J.M.; Hu, M.; Alexander, C.; Smirnova, T.G.; Olson, J.B.; James, E.; Dowell, D.C.; Grell, G.A.; et al. A North American Hourly Assimilation and Model Forecast Cycle: The Rapid Refresh. Mon. Weather. Rev. 2016, 144, 1669–1694. [CrossRef] 12. Liu, Y.; Warner, T.; Swerdlin, S.; Betancourt, T.; Knievel, J.; Mahoney, B.; Pace, J.; Rostkier-Edelstein, D.; Jacobs, N.A.; Childs, P.; et al. NCAR ensemble RTFDDA: Real-time operational forecasting applications and new data assimilation developments. In Proceedings of the 24th Conference on Weather Analysis and Forecasting and the 20th Conference on Numerical Weather Prediction, Seattle, WA, USA, 11–15 January 2011. 13. Antoniou, I.; Pedersen, T.F. Nacelle Anemometry on a 1 MW Wind Turbine: Comparing the Power Performance Results by Use of the Nacelle or ; Risø Report Risø-R-941(EN); Forskningscenter Risø: Roskilde, Denmark, 1997; p. 34. 14. Smith, B.; Link, H.; Randall, G.; McCoy, T. Applicability of Nacelle Anemometer Measurements for Use in Turbine Power Performance Tests; National Renewable Energy Laboratory preprint NREL/CP-500-32494; National Renewable Energy Laboratory: Golden, CO, USA, 2002. 15. Cheng, W.Y.; Liu, Y.; Bourgeois, A.J.; Wu, Y.; Haupt, S.E. Short-term wind forecast of a data assimilation/weather forecasting system with wind turbine anemometer measurement assimilation. Renew. Energy 2017, 107, 340–351. [CrossRef] 16. Myers, W.; Wiener, G.; Linden, S.; Haupt, S.E. A consensus forecasting approach for improved turbine hub height wind speed predictions. In Proceedings of the WindPower 2011, Anaheim, CA, USA, 24 May 2011. 17. Gneiting, T.; Katzfuss, M. Probabilistic Forecasting. Annu. Rev. Stat. Its Appl. 2014, 1, 125–151. [CrossRef] 18. Monache, L.D.; Eckel, F.A.; Rife, D.L.; Nagarajan, B.; Searight, K. Probabilistic Weather Prediction with an Analog Ensemble. Mon. Weather Rev. 2013, 141, 3498–3516. [CrossRef] 19. Alessandrini, S.; Monache, L.D.; Sperati, S.; Nissen, J. A novel application of an analog ensemble for short-term wind power forecasting. Renew. Energy 2015, 76, 768–781. [CrossRef] 20. Ezzat, A.A.; Jun, M.; Ding, Y. Spatio-temporal short-term wind forecast: A calibrated regime-switching method. Ann. Appl. Stat. 2019, 13, 1484–1510. [CrossRef] 21. Junk, C.; Delle Monache, L.; Alessandrini, S.; Cervone, G.; von Bremen, L. Predictor-weighting strategies for probabilistic wind power forecasting with an analog ensemble. Meteorol. Z. 2015, 24, 361–379. [CrossRef] 22. Sun, J.; Crook, N.A. Dynamical and Microphysical Retrieval from Doppler Radar Observations Using a Cloud Model and Its Adjoint. Part I: Model Development and Simulated Data Experiments. J. Atmos. Sci. 1997, 54, 1642–1661. [CrossRef] 23. Sun, J.; Chen, M.; Wang, Y. A Frequent-Updating Analysis System Based on Radar, Surface, and Mesoscale Model Data for the Beijing 2008 Forecast Demonstration Project. Weather. Forecast. 2010, 25, 1715–1735. [CrossRef] 24. Bernstein, B.C.; McDonough, F.; Politovich, M.K.; Brown, B.G.; Ratvasky, T.P.; Miller, D.R.; Wolff, C.A.; Cunning, G. Current Icing Potential: Algorithm Description and Comparison with Aircraft Observations. J. Appl. Meteorol. 2005, 44, 969–986. [CrossRef] 25. McDonough, F.; Bernstein, B.C.; Politovich, M.K.; Wolff, C.A. The forecast icing potential (FIP) algorithm, Preprints. In Proceedings of the 20th International Conference on Interactive Information and Processing Systems (IIPS) for , Oceanography, and Hydrology, Seattle, WA, USA, 14 January 2004. 26. Thompson, G.; , P.; Rasmussen, R.M.; Hall, W.D. Explicit Forecasts of Winter Precipitation Using an Improved Bulk Microphysics Scheme. Part II: Implementation of a New Parameterization. Mon. Weather. Rev. 2008, 136, 5095–5115. [CrossRef] 27. Kosovic, B.; Haupt, S.E.; Wiener, G.; Delle Monache, L.; Liu, Y.; Linden, S.; Politovich, M.; Sun, J. Scientific Advances in Wind Power Forecasting. In Proceedings of the American Wind Energy Association, Orlando, FL, USA, 21 May 2015.

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).