sustainability

Article Development of a Hydrological Ensemble Prediction System to Assist with Decision-Making for Floods during

Sheng-Chi Yang 1, Tsun-Hua Yang 2,* , Ya-Chi Chang 3,*, Cheng-Hsin Chen 4 , Mei-Ying Lin 5, Jui-Yi Ho 5 and Kwan Tun Lee 6,7

1 National Applied Research Laboratories, 106, ; [email protected] 2 Department of Civil Engineering, National Chiao Tung University, Hsinchu 300, Taiwan 3 Green Energy and Environment Research Laboratories, Industrial Technology Research Institute, Hsinchu 310, Taiwan 4 Department of Civil Engineering, National Chung Hsing University, 402, Taiwan; [email protected] 5 National Science and Technology Center for Disaster Reduction, Taipei 231, Taiwan; [email protected] (M.-Y.L.); [email protected] (J.-Y.H.) 6 Department of River and Harbor Engineering, National Taiwan Ocean University, Keelung 202, Taiwan; [email protected] 7 Center of Excellence for Ocean Engineering, National Taiwan Ocean University, Keelung 202, Taiwan * Correspondence: [email protected] (T.-H.Y.); [email protected] (Y.-C.C.)

 Received: 18 March 2020; Accepted: 19 May 2020; Published: 22 May 2020 

Abstract: Hydrological ensemble prediction systems (HEPSs) can provide decision makers with early warning information, such as peak stage and peak time, with enough lead time to take the necessary measures to mitigate disasters. This study develops a HEPS that integrates meteorological, hydrological, storm surge, and global tidal models. It is established to understand information about the uncertainty of numerical weather predictions and then to provide probabilistic flood forecasts instead of commonly adopted deterministic forecasts. The accuracy of flood forecasting is increased. However, the spatiotemporal uncertainty associated with these numerical models in the HEPS and the difficulty in interpreting the model results hinder effective decision-making during emergency response situations. As a result, the efficiency of decision-making is not always increased. Thus, this study also presents a visualization method to interpret the ensemble results to enhance the understanding of probabilistic runoff forecasts for operational purposes. A small watershed with area of 100 km2 and four historical events were selected to evaluate the performance of the method. The results showed that the proposed HEPS along with the visualization approach improved the intelligibility of forecasts of the peak stages and peak times compared to that of approaches previously described in the literature. The capture rate is greater than 50%, which is considered practical for decision makers. The proposed HEPS with the visualization method has potential for both decreasing the uncertainty of numerical rainfall forecasts and improving the efficiency of decision-making for flood forecasts.

Keywords: hydrological ensemble prediction system; numerical weather model; flood forecast; peak flow; visualization

1. Introduction Ensemble prediction systems (EPSs), which consist of an adequate number of equiprobable numerical weather prediction (NWP) models, have been established to provide probabilistic

Sustainability 2020, 12, 4258; doi:10.3390/su12104258 www.mdpi.com/journal/sustainability Sustainability 2020, 12, 4258 2 of 20 precipitation forecasts instead of a single deterministic forecast [1]. An EPS provides predictions with greater skill than those obtained from individual runs of NWP models or deterministic model runs, especially for longer lead times [2]. A hydrological ensemble prediction system (HEPS) is an integrated system that couples an EPS with catchment-scale hydrological models to provide flood forecasts. Many studies have used HEPS to conduct flood forecasting, with each study applying a different meteorological EPS input coupled with a different hydrological model based on the properties of the catchment of interest [3–5]. Balint et al. [4] analyzed more than 5000 model runs and tested the feasibility of HEPS for flood forecasting of the Danube. Based on the European Center for Medium-Range Weather Forecasts (ECMWF) ensemble predictions, the HEPS is implemented by the National Hydrological Forecasting Service (NHFS) modeling system of VITUKI (the Hungarian hydrological service). Davolio et al. [5] proposed a HEPS based on multi-model ensemble QPF (quantitative precipitation forecast) along with the rainfall-runoff model TOPKAPI (TOPographic Kinematic APproximation and Integration) [6] for the Reno River basin in northern Italy. Researchers are in general agreement that HEPSs for flood warning purposes have added forecast value compared to the previously adopted deterministic forecasts [1,7]. Uncertainties stemming from factors including boundary conditions, initial conditions, and model parameter values or different models applied in HEPSs for flood forecasting affect the forecast results. Abebe et al. [8] suggested that the consideration of the degree of uncertainty under which the forecasts are produced is essential for emergency response. Effective communication of ensemble forecasts means that clearly express the uncertainties associated with HEPSs is important so that end-users can easily respond to the provided information [9–11]. Pappenberger et al. [9] argued that the uncertainty information provided by HEPSs sometimes results in resistance on the part of the public if experts or nonexperts cannot easily understand the provided information. Pagano et al. [11] noted that defining effective methods for the communication of ensemble forecasts is a challenge for operational river forecasting and represents a research challenge. Uncertainties or differences within HEPSs still rely on conventional visualization techniques, such as “spaghetti diagrams” or box plots, to display the distributions of forecast results. Pappenberger et al. [9] focused on expert users of HEPSs and the communication among these experts and identified key information for the public, such as discharge, lead time, warning levels, return periods, and worst/best scenario. Zappa et al. [10] proposed a visualization approach to support the interpretation and verification of HEPS results. Giordani et al. [7] modified the approach from Zappa et al. [10] and developed another method for multiple peak flow detections. Olsson et al. [12] used this approach to process the ensemble runoff forecasts for joint probabilistic visualization of peak flow runoff and timing. This approach has been used to obtain quantitative and qualitative insights, such as the timing, water level, and discharge associated with peak flow. An average of 3.7 typhoons hit Taiwan annually, resulting in enormous loss of life and properties [13]. Hsiao et al. [14] used an ensemble meteorological modeling system to predict typhoon rainfall and coupled it with a hydrological model that considers only rainfall runoff and river routing simulations to predict flood in the Lanyang watershed in Taiwan. Instead of providing a single-valued forecast from a deterministic system, they deliver a probabilistic forecast to the users. It is well known in the scientific community around the world that ensemble or probabilistic forecasts contain more information than a single-valued forecast. This study therefore integrates the EPS from the Taiwan Cooperative Precipitation Ensemble Forecast Experiment (TAPEX) with 5 km spatial resolution and hydrological models to establish HEPSs. Since Taiwan is an island, the water level downstream are related to storm surge and tides during a typhoon. The hydrological models in the proposed HEPSs therefore include not only rainfall-runoff and river-routing models but also a storm-surge model, which have rarely been used. In practice, however, the users might prefer deterministic hydro-meteorological forecasts because they commonly misused the information of probabilistic forecasts. To solve the conflicts between probabilistic and deterministic forecasts for end-users, this study also proposes a visualization approach to simplify the HEPS forecast information Sustainability 2020, 12, 4258 3 of 20 Sustainability 2020, 12, x FOR PEER REVIEW 3 of 20 efficiencyand therefore will decrease be improved. the complexity The same of theidea decision-making can be applied process to areas for or common countries users. that Together are usually with influencedthe presented by visualizationtyphoon events. approach, The remainder the forecast of this accuracy paper and is organized emergency as response follows. eSectionfficiency 2 briefly will be describesimproved. the The study same area idea and can typhoon be applied events to areas analyzed or countries in the thatstudy. are Section usually 3 influenceddetails the bydeveloped typhoon HEPSevents. and The the remainder proposed of thisvisualization paper is organized approach. as follows.Finally, SectionSections2 briefly 4 and describes 5 present the the study results, area discussion,and typhoon and events conclusions. analyzed in the study. Section3 details the developed HEPS and the proposed visualization approach. Finally, Sections4 and5 present the results, discussion, and conclusions. 2. Study Area and Typhoon Events 2. Study Area and Typhoon Events 2.1. Study Area 2.1. Study Area The Yilan River in northeastern Taiwan was selected as the study area (Figure 1). The river flows throughThe the River of Yilan, in northeastern the main river Taiwan length was is selected approximately as the study 24.4 areakm, (Figureand the1 ).catchment The river area flows is 149.06through km the2. It city has of four Yilan, main the tributaries: main river the length Wushi is approximately River, the Dahu 24.4 River, km, andthe Dajiao the catchment River, and area the is Xiaojiao149.06 km River.2. It hasThere four are main 11 tributaries:rainfall gauging the Wushi stations, River, 16 thewater-stage Dahu River, gauging the Dajiao stations, River, five and water the surface-velocityXiaojiao River. Theregauging are stations, 11 rainfall and gauging 36 inundati stations,on-depth 16 water-stage gauging stations gauging installed stations, in fivethe waterYilan Riversurface-velocity Basin. Figure gauging 1 also stations,shows the and locations 36 inundation-depth of the water-stage gauging and stationsrainfall gauging installed stations in the Yilan that collectedRiver Basin. the Figuredata used1 also in shows this thestudy. locations The moni of thetoring water-stage data have and rainfallbeen carefully gauging collected stations thatand processed.collected the In data addition, used inthe this flow study. discharge The monitoring data used data in this have study been were carefully measured collected by and moving-boat processed. AcousticIn addition, Doppler the flow Current discharge Profiler data (ADCP) used in and this velocity-extrapolation study were measured methods by moving-boat during Acoustictyphoon events.Doppler Current Profiler (ADCP) and velocity-extrapolation methods during typhoon events.

Figure 1.1. StudyStudy area area and and locations locations of streamflowof streamflow gauges. gaug Blackes. Black dots and dots triangles and triangles indicate indicate the locations the locationsof water-stage of water-stage gauging stationsgauging and stations rain gaugeand rain stations, gauge respectively.stations, respectively.

2.2. Typhoon Events 2.2. Typhoon Events Typhoon tracks are usually determined by the center of a typhoon. The Central Weather Bureau Typhoon tracks are usually determined by the center of a typhoon. The Central Weather Bureau (CWB) categorized the typhoon tracks that invaded Taiwan into nine specific categories (Category (CWB) categorized the typhoon tracks that invaded Taiwan into nine specific categories (Category 1 1 through 9) and one special category using historical typhoon data from 1911 to 2018 as shown in through 9) and one special category using historical typhoon data from 1911 to 2018 as shown in Figure2[ 15]. Of the 10 categories, Track-2 and Track-3 typhoons account for approximately 28% of all Figure 2 [15]. Of the 10 categories, Track-2 and Track-3 typhoons account for approximately 28% of typhoons and bring heavy rainfall to the Yilan River Basin. For instance, a rainfall of 158 mm in 4 h was all typhoons and bring heavy rainfall to the Yilan River Basin. For instance, a rainfall of 158 mm in 4 observed at the rainfall gauging station C1U610 (shown in Figure1) during Typhoon Soulik. Table1 h was observed at the rainfall gauging station C1U610 (shown in Figure 1) during Typhoon Soulik. shows all the typhoons with a warning issued by CWB from 2012 through 2015. Five of these events Table 1 shows all the typhoons with a warning issued by CWB from 2012 through 2015. Five of these are Track-2 and Track-3 typhoons, which have the biggest impact on the Yilan River Basin. Therefore, events are Track-2 and Track-3 typhoons, which have the biggest impact on the Yilan River Basin. this study selected the typhoons Saola (2012), Soulik (2013), Soudelor (2015), and Dujuan (2015) to Therefore, this study selected the typhoons Saola (2012), Soulik (2013), Soudelor (2015), and Dujuan (2015) to calibrate and validate the Yilan River HEPS. , a Track-3 typhoon that occurred in 2014, was not included due to its weak intensity. Sustainability 2020, 12, 4258 4 of 20 calibrate and validate the Yilan River HEPS. Typhoon Matmo, a Track-3 typhoon that occurred in 2014, wasSustainability not included 2020, 12,due x FOR to PEER its weak REVIEW intensity. 4 of 20

FigureFigure 2.2.Schematic Schematic diagram diagram showing showing the the tracks tracks of typhoons of typhoons invading invading Taiwan. Taiw Thean. percentages The percentages shown inshown the figure in the are figure the statisticalare the statistical results from result 1911s from through 1911 2018through obtained 2018 obtained from the from Central the Weather Central BureauWeather (CWB) Bureau in (CWB) Taiwan. in The Taiwan. dark grayThe polygondark gray located polygon in northernlocated in Taiwan northern indicates Taiwan the indicates Yilan River the catchment.Yilan River Track-2 catchment. and Track-3 Track-2 typhoons and Track-3 bring heavytyphoons rainfall bring to theheavy Yilan rainfall River catchment.to the Yilan River catchment. Table 1. All typhoons with warning issued by CWB in Taiwan during 2012 through 2015. A total of four typhoons of Track-2 and Track-3, namely, Saola in 2012, Soulik in 2013, Soudelor in 2015, and Dujuan Table 1. All typhoons with warning issued by CWB in Taiwan during 2012 through 2015. A total of in 2015, were selected to calibrate the system and test the performance. Typhoon Matmo, a Track-3 four typhoons of Track-2 and Track-3, namely, Saola in 2012, Soulik in 2013, Soudelor in 2015, and typhoon that occurred in 2014, was not selected due to its weak intensity. Dujuan in 2015, were selected to calibrate the system and test the performance. Typhoon Matmo, a Track-3 typhoon thatTyphoon occurred in 2014, Track was not Intensity selected due to its Warning weak intensity. Period DUJUANTyphoon Track 2 Intensity 3 27–29Warning September Period 2015 DUJUANGONI - 2 - 3 27–29 20–23 September August 2015 2015 SOUDELORGONI 3 - 3 - 20–23 6-9 August August 2015 2015 SOUDELORLINFA - 3 - 3 6-9 6–9 August July 2015 2015 CHAN-HOM - 2 9–11 July 2015 LINFA - - 6–9 July 2015 NOUL - - 10–11 May 2015 CHAN-HOM - 2 9–11 July 2015 FUNG-WONG Special - 19–22 September 2014 MATMONOUL 3 - - - 10–1121–23 May July 20142015 FUNG-WONGHAGIBIS Special - 3 - 19–22 14–15 September June 2014 2014 FITOWMATMO 1 3 - - 4–7 21–23 October July 2014 2014 USAGIHAGIBIS 5 - 3 3 19–22 14–15 September June 2014 2013 KONG-REYFITOW 6 1 - - 27–29 4–7 October August 2014 2013 TRAMIUSAGI 1 5 - 3 19–22 20–22 September August 2013 2013 CIMARONKONG-REY - 6 - - 27–29 17–18 August July 2013 2013 SOULIKTRAMI 2 1 1 - 20–22 11–13 August July 2013 2013 JELAWATCIMARON - - - 27–28 17–18 September July 2013 2012 - 21–25 August 2012 TEMBINSOULIK Special 2 1 11–13 July 2013 - 26–28 August 2012 JELAWAT - 27–28 September 2012 KAI-TAK - 1 14–15 August 2012 - 21–25 August 2012 HAIKUITEMBIN Special - - 6–7 August 2012 SAOLA 2 4- 30 26–28 July–3 August August 2012 2012 DOKSURIKAI-TAK - - - 1 14–15 28–29 August June 2012 2012 TALIMHAIKUI 9 - - - 6–7 19–21 August June 20122012 SAOLA 2 4 30 July–3 August 2012 DOKSURI - - 28–29 June 2012 TALIM 9 - 19–21 June 2012

3. Hydrological Ensemble Prediction System Sustainability 2020, 12, 4258 5 of 20

3. Hydrological Ensemble Prediction System Figure3 shows the flowchart of data processing in the HEPS. It integrates NWP models that Sustainability 2020, 12, x FOR PEER REVIEW 5 of 20 provide 72-h ensemble precipitation forecasts, a rainfall-runoff model that calculates the surface runoff and estimates flowFigure discharge 3 shows the at flowchart Hsincheng of data and processing Yuanshan in Bridgesthe HEPS. as Itthe integrates upstream NWP boundary models that conditions, a storm-surgeprovide model 72-h coupledensemble withprecipitation a global forecasts, tide model a rainfall-runoff that generate model the that water calculates stage the at Kemalansurface Bridge as the downstreamrunoff and estimates boundary flow conditions, discharge at andHsincheng a flood-routing and Yuanshan model Bridges that as the simulates upstream boundary Yilan River flows. conditions, a storm-surge model coupled with a global tide model that generate the water stage at The abovementionedKemalan Bridge geographical as the downstream locations boundary can beconditions, found inand Figure a flood-routing1. The Yilan model River that simulates HEPS produces ensemble floodYilan River forecasts flows. with The abovementioned a 72-h lead time geographical four times locations a day. can The be details found in of Figure the models 1. The Yilan are described in the followingRiver HEPS subsections. produces ensemble flood forecasts with a 72-h lead time four times a day. The details of the models are described in the following subsections.

Figure 3. Flowchart describing the flow of data processing within the Yilan River hydrological Figure 3. Flowchart describing the flow of data processing within the Yilan River hydrological ensemble ensemble prediction system (HEPS). prediction system (HEPS). 3.1. Ensemble Precipitation Forecasts 3.1. Ensemble Precipitation Forecasts TAPEX began in 2010. It is a collective effort among academic institutes and government TAPEXagencies, began such in 2010. as the It National is a collective Taiwan eUniversityffort among, the academicNational Central institutes University, andgovernment the National agencies, such as theTaiwan National Normal Taiwan University, University, the Chinese the Culture National Univ Centralersity, the University, CWB, the National the National Center for Taiwan High- Normal Performance Computing, the Taiwan Typhoon and Flood Research Institute (TTFRI), and the University,National the Chinese Science and Culture Technology University, Center for theDisaster CWB, Reduction. the National TAPEX is Centerthe first attempt for High-Performance to design Computing,a high-spatial-resolution the Taiwan Typhoon (5 km) and numerical Flood Research ensemble Institutemodel in Taiwan. (TTFRI), This and effort the applies National various Science and TechnologyNWP Center models, for Disastersuch as the Reduction. Weather Research TAPEX and is theForecasting first attempt Model to(WRF), design the aFifth-Generation high-spatial-resolution (5 km) numericalPenn State/NCAR ensemble Mesoscale model Model in Taiwan. (MM5), the This Clou effd-Resolvingort applies Storm various Simulator NWP (CReSS), models, and the such as the Hurricane Weather Research and Forecasting Model (HWRF). It also considers different setups in Weather Researchterms of the and model Forecasting initial conditions, Model (WRF),data assimilation the Fifth-Generation processes and model Penn Statephysics./NCAR TAPEX Mesoscale Model (MM5),generates the Cloud-Resolvingfour runs a day and Stormprovides Simulator ensemble predictions (CReSS), andof the the wind Hurricane and pressure Weather fields Researchand and Forecastingquantitative Model (HWRF). estimates of It precipitation also considers with dia leadfferent time setupsof 72 h. inFurther terms information of the model can be initialfound in conditions, data assimilationreference [13]. processes This study and focuses model on a physics.one-way coupling TAPEX in which generates TAPEX four provides runs rainfall a day forecast and provides to the rainfall-runoff model; feedbacks from the rainfall-runoff model to TAPEX are not considered. ensemble predictions of the wind and pressure fields and quantitative estimates of precipitation with a lead time of3.2. 72 Hydrological h. Further Methods information can be found in reference [13]. This study focuses on a one-way coupling in which TAPEX provides rainfall forecast to the rainfall-runoff model; feedbacks from the 3.2.1. Rainfall-Runoff Model rainfall-runoff model to TAPEX are not considered. The Yilan River HEPS uses the surface runoff forecast generated by a kinematic-wave-based 3.2. Hydrologicalgeomorphologic Methods instantaneous unit hydrograph model (the KW-GIUH model) as the upstream boundary condition of the river-routing model. The KW-GIUH model, which was developed by Lee and Yen [16], can reflect the effects of watershed geomorphology, land cover conditions, soil 3.2.1. Rainfall-Runoff Model characteristics and rainfall intensity on runoff. It has been successfully applied to many Taiwanese The Yilancatchments River [17–19]. HEPS uses the surface runoff forecast generated by a kinematic-wave-based geomorphologic instantaneous unit hydrograph model (the KW-GIUH model) as the upstream boundary condition of the river-routing model. The KW-GIUH model, which was developed by Lee and Yen [16], can reflect the effects of watershed geomorphology, land cover conditions, soil characteristics and rainfall intensity on runoff. It has been successfully applied to many Taiwanese catchments [17–19]. Sustainability 2020, 12, 4258 6 of 20

Two parameters in the proposed HEPS KW-GIUH model have been calibrated using in situ observationsSustainability made 2020 during, 12, x FOR typhoon PEER REVIEW events. These parameters are the roughness coefficient6 for of 20 overland

flow (n0) andTwo the parameters roughness in coethe ffiproposedcient for HEPS channel KW-GIUH flow (modelnc) and have depend been calibrated on the type using of in land situ use and bed materialobservations of the made river induring the analyzedtyphoon events. basin. Thes Fivee rainfallparameters gauges, are the including roughness LTGX, coefficient YSGZ, for C1U610, C0U520,overland and C1U630 flow (n0 (see) and Figurethe roughness2 for locations), coefficient for were channel used, flow and (nc) the and Thiessen depend on polygon the type of method land [ 20] was useduse to and estimate bed material the hourlyof the river spatial-average in the analyzed rainfallbasin. Five intensities rainfall gauges, used including as rainfall LTGX, input YSGZ, data to the KW-GIUHC1U610, model. C0U520, The observedand C1U630 flow (see data Figure at the2 for Hsincheng locations), were and Yuanshanused, and the Bridges Thiessen during polygon Typhoons method [20] was used to estimate the hourly spatial-average rainfall intensities used as rainfall input Saola, Soulik, and Soudelor were used to calibrate and validate the parameters. As shown in Figure4, data to the KW-GIUH model. The observed flow data at the Hsincheng and Yuanshan Bridges during the percentTyphoons errors Saola, in the Soulik, peak and discharges Soudelor between were used observed to calibrate and and simulated validate the flow parameters. of the selected As shown typhoons were 4.59%in Figure (Saola), 4, the 2.07% percent (Soulik), errors in and the peak5.89% disc (Soudelor)harges between at the observed Hsincheng and simulated Bridge and flow 14.88% of the (Saola), − 5.28% (Soulik),selected typhoons and 3.05% were(Soudelor) 4.59% (Saola), at 2.07% the Yuanshan (Soulik), and Bridge. −5.89% All (Soudelor) of the errorsat the Hsincheng in the peak Bridge times were − less thanand one 14.88% hour. (Saola), The results 5.28% (Soulik), show that and the −3.05% proposed (Soudelor) HEPS at the KW-GIUH Yuanshan model Bridge. is All capable of the errors of providing accuratein simulations the peak times for were peak less time than as one well hour. as peakThe results discharge. show that the proposed HEPS KW-GIUH model is capable of providing accurate simulations for peak time as well as peak discharge.

Figure 4. Comparison of simulated discharges (red circles) and recorded discharges (solid lines) for model calibration (Typhoons Saola and Soulik) and validation () experiments at Hsinsheng (left) and Yuanshen (right). The blue bars are the hourly spatial-average rainfall intensities measured in the watershed upstream of Hsinsheng and Yuanshen. Sustainability 2020, 12, x FOR PEER REVIEW 7 of 20

Figure 4. Comparison of simulated discharges (red circles) and recorded discharges (solid lines) for model calibration (Typhoons Saola and Soulik) and validation (Typhoon Soudelor) experiments at SustainabilityHsinsheng2020, 12 (left), 4258 and Yuanshen (right). The blue bars are the hourly spatial-average rainfall intensities 7 of 20 measured in the watershed upstream of Hsinsheng and Yuanshen.

3.2.2.3.2.2. Storm-Surge Storm-Surge Model Model StormStorm surges surges are are abnormal abnormal increasesincreases inin waterwater le levelsvels above above those those expected expected from from astronomical astronomical tides.tides. They They are are generated generated by by strong strong windswinds and atmo atmosphericspheric pressure pressure changes changes and and affect affect water water levels levels downstreamdownstream (near (near estuaries) estuaries) during during typhoons.typhoons. The The Yilan Yilan River River HEPS HEPS uses uses the the storm storm surge surge and and tide tide forecastsforecasts generated generated by by the the Princeton Princeton OceanOcean ModelModel (POM) and and the the TOPEX-POSEIDON TOPEX-POSEIDON global global tidal tidal modelmodel (TPXO6.2) (TPXO6.2) as as downstream downstream boundaryboundary conditions.conditions. The The POM POM model, model, which which was was developed developed by by BlumbergBlumberg and and Mellor Mellor [21 [21],], is ais three-dimensional, a three-dimensional, nonlinear, nonlinear, primitive primitive equation equation finite finite diff erencedifference ocean model.ocean model. It has beenIt has appliedbeen applied to simulate to simulate a wide a wide range range of of ocean ocean scenarios,scenarios, including including coastal coastal storm storm surgesurge in in Taiwan Taiwan [22 [22,23].,23]. In In this this study,study, thethe TAPEXTAPEX model provides provides ensemble ensemble pressure pressure field field and and wind wind fieldfield forecasts forecasts to to POM POM and and the the TPXO6.2 TPXO6.2 modelmodel for ob obtainingtaining tidal tidal level level predictions. predictions. As As with with TAPEX, TAPEX, it generatesit generates four four runs runs a day,a day, and and each each run run has has a 72-h a 72-h lead lead time. time. Two Two typhoons typhoons (Saola (Saola and and Soulik) Soulik) were usedwere to evaluateused to evaluate the storm-surge the storm-surge model due model to the due severe to the lack severe of observations lack of observations for Typhoon for Typhoon Soudelor at SuaoSoudelor tide station. at Suao Astide shown station. in As Figure shown5, thein Figure percent 5, the errors percent in the errors peak in storm the peak surge storm were surge9.4% were for −9.4% for Saola and 19.4% for Soulik. − Saola and 19.4% for Soulik.

FigureFigure 5. 5.Comparison Comparison ofof simulatedsimulated (red(red circles) and observed observed (solid (solid lines) lines) storm storm surges surges for for model model calibrationcalibration (Typhoon (Typhoon Saola) Saola) and and validation validation (Typhoon(Typhoon Soulik)Soulik) experiments at at Suao Suao tide tide station. station.

3.2.3.3.2.3. River-Routing River-Routing Model Model TheThe Numerical Numerical Model Model SimulatingSimulating WaterWater Flow and Contaminant Contaminant and and Sediment Sediment Transport Transport in in WAterSHedWAterSHed Systems Systems of of 1D 1D Stream Stream/River/River Networks,Networks, 2D Overland Regimes, Regimes, and and 3D 3D Subsurface Subsurface Media Media (WASH123D)(WASH123D) developed developed by by Yeh Yeh et et al. al. [[24]24] waswas usedused to simulate simulate one-dimensional one-dimensional channel channel networks, networks, two-dimensionaltwo-dimensional overland overland flow, flow, and and three-dimensionalthree-dimensional variably variably saturated saturated subsurface subsurface flow. flow. The The flow flow inin river river/stream/stream networks networks in in WASH123D WASH123Dis is basedbased on the one-dimensional one-dimensional St. St. Venant Venant equation equation and and the the simulationsimulation in in this this model model is conductedis conducted with with finite-element finite-element approaches approaches of variousof various spatial-temporal spatial-temporal scale computationsscale computations [25]. It has[25]. been It has applied been applied successfully successfully in Taiwan in Taiwan and around and around the world, the world, and it and was it chosenwas chosen by the US Army Corps of Engineers as the core computational code used in modeling the by the US Army Corps of Engineers as the core computational code used in modeling the Lower East Lower East Coast (LEC) Wetland Watershed [26–28]. The Yilan River HEPS uses the one-dimensional Coast (LEC) Wetland Watershed [26–28]. The Yilan River HEPS uses the one-dimensional channel channel model of WASH123D as its flood-routing model to simulate water stages in rivers. model of WASH123D as its flood-routing model to simulate water stages in rivers. The Yilan River HEPS adopted the most recent available cross-sectional bathymetry of the Yilan The Yilan River HEPS adopted the most recent available cross-sectional bathymetry of the Yilan River, which was measured in 2010, as its input topography data. The upstream boundary of the River, which was measured in 2010, as its input topography data. The upstream boundary of the model model is set at the Hsincheng and Yuanshan Bridges, and the downstream boundary of the model is is setset atat thethe HsinchengKemalan Bridge. and Yuanshan Figure 6 Bridges,shows that and the the percent downstream errors in boundary comparison of the with model the observed is set at the Kemalanpeak stages Bridge. for FigureTyphoons6 shows Saola, that Soulik, the percent and Soudelor errors inwere comparison 2.1%, 5.7%, with and the 10.6% observed at Zhongshan, peak stages for12.9% Typhoons and 2.2% Saola, at Soulik, Leawood, and and Soudelor 7.4%, were6.0%, 2.1%,and 2.1% 5.7%, at and Jhuangwei, 10.6% at Zhongshan,respectively. 12.9%There andwas 2.2%one at Leawood,data gap and at 7.4%,Leawood 6.0%, due and to 2.1% incomplete at Jhuangwei, observ respectively.ed data collection There wasduring one Typhoon data gap atSoudelor. Leawood due to incomplete observed data collection during Typhoon Soudelor. Nevertheless, all of the errors in the peak times were less than one hour. The results show that the Yilan River HEPS is capable of providing confident predictions of peak times, as well as peak stages. Sustainability 2020, 12, x FOR PEER REVIEW 8 of 20

SustainabilityNevertheless,2020, 12 all, 4258 of the errors in the peak times were less than one hour. The results show that the8 of 20 Yilan River HEPS is capable of providing confident predictions of peak times, as well as peak stages.

FigureFigure 6. 6.Comparison Comparison of of simulated simulated (red(red circles)circles) and observed (solid (solid lines) lines) water water levels levels for for model model calibrationcalibration (Typhoons (Typhoons Saola Saola and Soulik)and Soulik) and validation and validation (Typhoon (Typhoon Soudelor) Soudelor) experiments experiments at Zhongshan at (left),Zhongshan Leawood (left), (central) Leawood and Jhungwei(central) and (right). Jhungwei (right).

3.3.3.3. A VisualizationA Visualization Approach Approach for for Supporting Supporting thethe InterpretationInterpretation of Operational Operational Ensemble Ensemble Peak Peak Flow Flow Forecasts duringForecasts Typhoon during Events Typhoon Events ForecastsForecasts of of HEPSs HEPSs are usually usually characterized characterized by bya large a large ensemble ensemble spread. spread. Consequently, Consequently, their theirevaluation evaluation may may be becomplex complex for for decision-making decision-making by bycommon common users, users, especially especially during during emergency emergency response.response. Therefore, Therefore, ZappaZappa et et al. al. [10] [10 first] first proposed proposed a visualization a visualization approach, approach, called called“Peak-Box”, “Peak-Box”, with withthe the purpose purpose of ofhelping helping decision-making decision-making when when foreca forecastingsting flood flood events. events. Giordani Giordani et al. et al.[7] [ 7modified] modified it toit to detect detect multiple multiple peak peak flows.flows. Figure Figure 77 Left Left shows shows the the “Peak-Box” “Peak-Box” visualization visualization approach approach and the and theconcept concept is isdescribed described as as follows. follows. First, First, a arectangle rectangle (the (the “Peak-Box”), “Peak-Box”), which which is isenveloped enveloped by bythe the coordinates of four values, i.e., the earliest time of peak flow (t0), the latest time of peak flow (t100), the coordinates of four values, i.e., the earliest time of peak flow (t0), the latest time of peak flow (t100), lowest peak discharge (p0) and the highest peak discharge (p100) of all ensemble members. Second, an the lowest peak discharge (p0) and the highest peak discharge (p100) of all ensemble members. Second, aninner inner rectangle rectangle (the (the “IQR-Box”), “IQR-Box”), which which is is enveloped enveloped by by the the coordinates coordinates of offour four values, values, i.e., i.e., the the 25% 25% quartile of the peak timing (t25), the 75% quartile of the peak timing (t75), the 25% quartile of the peak quartile of the peak timing (t25), the 75% quartile of the peak timing (t75), the 25% quartile of the peak discharge (p25) and the 75% quartile of the peak discharge (p75) of all ensemble members. A horizontal discharge (p25) and the 75% quartile of the peak discharge (p75) of all ensemble members. A horizontal line is drawn from t0 to t100, which represents the median of the peak discharge (p50) of all ensemble line is drawn from t0 to t100, which represents the median of the peak discharge (p ) of all ensemble forecasts. Finally, a vertical line is plotted from p0 to p100, which represents the median50 of the peak forecasts. Finally, a vertical line is plotted from p0 to p100, which represents the median of the peak timing (t50) of all ensemble forecasts. Based on the above-mentioned approach, this study proposed a timing (t ) of all ensemble forecasts. Based on the above-mentioned approach, this study proposed a modified50 visualization method specifically for the proposed HEPS and for typhoon events. Figure 7 modifiedRight shows visualization the proposed method visua specificallylization approach for the and proposed the modifi HEPScations and forare typhoondescribed events. below. Figure7 Right shows the proposed visualization approach and the modifications are described below. a. Simplify and avoid misused information. Pappenberger et al. [9] noted a considerable desire on a. Simplifythe part of and end-users avoid misused to reduce information. probabilisticPappenberger forecasts to deterministic et al. [9] noted actions. a considerable The proposed desire onvisualization the part of approach end-users removes to reduce the probabilistic horizontal and forecasts vertical to lines deterministic because end-users actions. The commonly proposed visualizationmisused them approach for decision-making. removes the horizontalThe two li andnes verticalmay lead lines end-users because to end-users believe commonlythat the misusedinformation them provided for decision-making. represents a single The determin two linesistic forecast, may lead rather end-users than a toprobabilistic believe that one. the informationThe outer rectangle provided “Peak-Box” represents is also a single removed deterministic when the forecast,availability rather of too than much a probabilistic data may one.obscure The critical outer rectangleinformation “Peak-Box” during typhoons. is also removed Therefore, when only the one availability rectangle is of shown too much in the data mayproposed obscure approach. critical informationThis rectangle during only typhoons.indicates where Therefore, the observed only one rectanglepeak stage is and shown its in theoccurring proposed time approach. are likely to This occur. rectangle only indicates where the observed peak stage and its occurring time are likely to occur. b. Rescale the rectangle. This study defines an “SD-Box” that uses the mean (µ) and the standard deviation (σ), instead of the 25% and 75% quartiles, to define the enveloping rectangle. As shown Sustainability 2020, 12, x FOR PEER REVIEW 9 of 20 b.Sustainability Rescale 2020the ,rectangle.12, 4258 This study defines an “SD-Box” that uses the mean (μ) and the standard9 of 20 deviation (σ), instead of the 25% and 75% quartiles, to define the enveloping rectangle. As shown in the right panel of Figure 7, the lower left coordinate of the “SD-Box” is defined as the mean in the right panel of Figure7, the lower left coordinate of the “SD-Box” is defined as the mean peak time minus one standard deviation (μt − σt), and the mean peak stage minus one standard peak time minus one standard deviation (µt σt), and the mean peak stage minus one standard deviation (μh − σh) produced by all of the ensemble− members. The upper right coordinate is deviation (µh σh) produced by all of the ensemble members. The upper right coordinate defined as the mean− peak time plus one standard deviation (μt + σt) and the mean peak stage plus is defined as the mean peak time plus one standard deviation (µt + σt) and the mean peak one standard deviation (μh + σh) of all of the ensemble members. The standard deviation takes stage plus one standard deviation ( + ) of all of the ensemble members. The standard into account the spread of ensemble forecasts.µh σh In addition, using the mean and the standard deviation takes into account the spread of ensemble forecasts. In addition, using the mean and deviation instead of the quartile deviation to determine the second rectangle allows the inclusion of a thehigher standard number deviation of ensemble instead forecasts of the quartile and ha deviationve more opportunities to determine theto cover second observed rectangle peaks. allows In principle,the inclusion the “IQR-Box” of a higher should number contain of ensemble 25% (50% forecasts of the peak and have discharge more multiplied opportunities by 50% to cover of the observedpeak times) peaks. of all In forecasts. principle, Zappa the “IQR-Box” et al. [10] shouldshowedcontain that this 25% box (50% only of contained the peak between discharge 12.5%multiplied and 37.5% by due 50% to of the the distri peakbution times) of of ensemble all forecasts. members. Zappa etThe al. “SD-Box” [10] showed includes that this the box mean only andcontained the standard between deviation 12.5% andand 37.5%results due in toa thelarg distributioner area. It ofincludes ensemble 46.60% members. of the The ensemble “SD-Box” forecastsincludes (68.27% the mean of peak and water the standardlevel multiplied deviation by 68.27% and results of the in peak a larger times); area. therefore, It includes it should 46.60% haveof a the greater ensemble chance forecasts of including (68.27% ofthe peak observed water levelpeaks. multiplied A greater by rectangle 68.27% of may the peak generate times); overestimation.therefore, it shouldBecause have there a greateris only chanceone rectan of includinggle remaining, the observed the simplified peaks. A information greater rectangle can makemay for generateefficient decision-making. overestimation. Because there is only one rectangle remaining, the simplified c. Includeinformation all forecasts can make with for different efficient lead decision-making. times in the rectangle. Descriptive statistics, such as c. the Includequartile alldeviation forecasts and with the di stffandarderent lead deviation, times in are the susceptible rectangle. toDescriptive outliers when statistics, calculated such as usingthe insufficient quartile deviation sample sizes. and the Since standard adding deviation, extra ensemble are susceptible members to produce outliers whenmore forecasts calculated andusing thus increase insufficient sample sample size sizes. consumes Since addingcomput extraer resources, ensemble this members study includes to produce present more forecasts(t) and previousand thus forecasts increase (t − sample1, t − 2, t size − 3… consumes t − n, where computer n is the resources,number of thisavailable study forecasts includes when present the ( t) and previous forecasts (t 1, t 2, t 3 ... t n, where n is the number of available forecasts system is initiated) to expand− the sample− − size and− provide a better interpretation of results. As shownwhen in the the right system panel is initiated) of Figure to7, expandthe green the area sample illustrates size andthe “SD-Box”. provide a The better black interpretation and gray solidof dots results. represent As shown the current in the and right previous panel of peak Figure flow7, forecasts, the green respectively. area illustrates The the “SD-Box” “SD-Box”. is designedThe black exclusively and gray for solid typhoons dots represent for operational the current purposes, and previous and peakit is initiated flow forecasts, when respectively.the CWB issuesThe a “SD-Box”sea warning is designedand ends exclusivelywhen the next for ensemb typhoonsle forecast for operational is six hours purposes, less than and the it isleft initiated edge of thewhen “SD-Box”. the CWB issues a sea warning and ends when the next ensemble forecast is six hours less than the left edge of the “SD-Box”.

FigureFigure 7. 7.TheThe left left panel panel shows shows a a graphical graphical explanation ofofthe the “Peak-Box” “Peak-Box” approach approach from fromZappa Zappa et et al. al.[10 ]. [10].The The outer outer rectangle rectangle is the is “Peak-Box”,the “Peak-Box”, and theand internal the internal rectangle rectangle (the yellow (the yellow area) is area) the inner is the rectangle inner rectanglebox (IQR-Box). box (IQR-Box). The solid The dots solid represent dots represent all of the ensembleall of the ensemble forecasts, andforecasts, the red and cross the is red the cross observed is thepeak observed flow. The peak right flow. panel Theshows right panel a graphic shows comparison a graphic ofcomparison the “IQR-Box” of the and “IQR-Box” “SD-Box” and in the “SD-Box” proposed inmodification the proposed of modification the “Peak-Box” of the approach. “Peak-Box” The a envelopingpproach. The rectangle enveloping is the rectangle “SD-Box” is (thethe “SD-Box” green area), and the yellow rectangle is “IQR-Box”. The solid black and gray dots represent current and previous (the green area), and the yellow rectangle is “IQR-Box”. The solid black and gray dots represent peak flow forecasts, respectively. current and previous peak flow forecasts, respectively. Sustainability 2020, 12, 4258 10 of 20

4. Results and Discussion

4.1. Performance Evaluation of the Yilan River HEPS This study focused exclusively on peak flow in terms of its magnitude during the peak stage and peak timing. Therefore, this study applied two widely used performance measures to evaluate the performance of the HEPS: the root-mean-square error (RMSE) and the spread-skill score. The RMSE and the spread-skill score are defined as follows: r  2 RMSE = O µ (1) peak − RMSE Spread skill score = (2) − σ 1 Xm µ = Ppeak i (3) m i=1 , r 1 Xm  2 σ = Ppeak i µ (4) m i=1 , − where µ is the mean of the peak flow forecast ensemble; σ is the standard deviation of peak flow forecast ensemble; Opeak is the observed peak flow; Ppeak,i is the peak flow prediction of the ith member; and m is the number of ensemble members. The RMSE, which is commonly referred to as skill, measures the difference between the observations and the ensemble mean without considering the direction. The closer the RMSE is to zero, the better the ensemble mean is as a forecast. The spread-skill score is one of many metrics used to evaluate an ensemble system, which ranges from zero to infinity, is the ratio of the standard deviation of the ensemble peak flow forecasts (spread) to the RMSE [29]. Ensemble prediction should provide spread between the members that is large enough to cover the uncertainty in the prediction. For a well-calibrated ensemble forecast system, the spread should be of the same magnitude as the RMSE [10]. In that regard, the spread-skill score might be appropriate to evaluate the ensemble system proposed in this study and should be close to one to meet the requirements of a qualified ensemble system. The forecasts used in this study were initiated when the CWB issues a sea warning and end when the next ensemble forecast is six hours less than the left edge of the “SD-Box”. In that regard, 93 forecasts are available for the four selected typhoons. Table2 shows the spread-skill score in peak stages and peak times considering only the current forecasts for the Dujuan, Soudelor, Soulik and Saola events at the Zhongshan, Leawood, and Zhuangwei Bridges. We evaluated 7, 9, 8 and 10 forecasts from 27 September 08:00 to 28 September 28 20:00 with 6-h intervals in Dujuan, from 6 August 08:00 to 8 August 08:00 with 6-h intervals in Soudelor, from 11 July 08:00 to 13 July 02:00 with 6-h intervals in Soulik, and 30 July 20:00 to 2 August 02:00 with 6-h intervals in Saola, respectively. The spread-skill scores were not calculated for the Leawood Bridge during Typhoon Soudelor due to the lack of complete observations. The spread-skill scores that are less than one (indicating that the spread of the forecasts exceeds the prediction uncertainty) in the table are highlighted. Most of the spread-skill scores of Dujuan were greater than 2. The system performed the worst for this event due to the large forecast track error from the NWP model used in the Yilan River HEPS, resulting in poor performance in rainfall forecasts [30]. Because most of the uncertainty comes from the NWP model, the accuracy of rainfall forecasts exhibited a significant impact on the performance of the HEPS. This also explained why Saola and Soulik had the best performance among all events. Overall, the “SD-Box” method yielded average spread-skill scores of 1.13 for the peak stages and 1.00 for the peak times. Sustainability 2020, 12, 4258 11 of 20

Table 2. Comparisons of the spread-skill scores in the peak stage and peak time for the Yilan River HEPS. Scores less than one (highlighted) indicate that the enveloping box did contain the observed peak. (a) The spread-skill scores in peak stage forecasts. (b) The spread-skill scores in peak timing forecasts.

(a) Forecast Location/Typhoon 1 2 3 4 5 6 7 8 9 10 Zhongshan Bridge Dujuan (2015) 2.54 2.59 2.64 2.09 4.79 2.62 0.57 --- Soudelor (2015) 0.41 0.60 1.88 0.93 2.76 2.82 2.27 4.59 1.78 - Soulik (2013) 1.07 1.27 1.39 0.76 0.64 0.38 0.15 0.40 -- Saola (2012) 0.20 0.07 0.71 0.56 0.55 0.55 1.36 1.23 2.18 0.54 Leawood Bridge Dujuan (2015) 1.21 1.27 1.75 1.24 3.48 1.48 1.67 - - - Soudelor (2015) ------Soulik (2013) 0.79 0.95 1.06 0.36 0.20 0.10 0.27 0.54 -- Saola (2012) 0.93 1.25 1.66 1.32 1.41 0.16 0.29 0.22 0.04 1.36 Zhuangwei Bridge Dujuan (2015) 1.97 2.13 0.60 0.21 0.46 1.51 2.94 - - - Soudelor (2015) 1.19 0.17 0.45 0.10 1.01 1.24 0.55 1.81 2.64 - Soulik (2013) 0.62 0.71 0.79 0.17 0.03 0.32 0.47 0.90 -- Saola (2012) 0.82 1.08 1.40 1.14 1.26 0.09 0.70 0.09 0.22 1.29 (b) Forecast Location/Typhoon 1 2 3 4 5 6 7 8 9 10 Zhongshan Bridge Dujuan (2015) 1.34 1.38 4.33 1.83 2.83 1.86 0.68 --- Soudelor (2015) 0.68 0.70 1.74 0.97 3.49 1.75 1.08 1.08 0.66 - Soulik (2013) 1.48 1.60 2.64 0.59 1.37 0.23 0.36 1.29 - - Saola (2012) 0.07 0.26 0.28 0.02 0.37 0.58 0.30 0.01 0.79 0.48 Leawood Bridge Dujuan (2015) 0.46 0.11 1.69 0.32 2.24 0.58 0.71 --- Soudelor (2015) ------Soulik (2013) 0.40 1.17 1.96 0.39 0.71 0.11 0.09 0.96 -- Saola (2012) 0.04 0.09 0.34 0.17 0.04 0.11 0.46 0.07 0.67 0.53 Zhuangwei Bridge Dujuan (2015) 2.90 3.54 3.06 4.17 2.57 3.91 0.86 --- Soudelor (2015) 0.40 0.48 1.32 0.72 3.20 1.42 1.04 1.08 0.28 - Soulik (2013) 0.42 0.59 1.08 0.81 0.16 0.68 0.08 0.70 -- Saola (2012) 0.25 0.07 0.17 0.28 0.54 1.05 0.79 0.33 0.09 0.68

4.2. Comparison of Enveloping Rectangles Defined Using the “SD-Box” and the “IQR-Box” Methods for Supporting the Interpretation of Ensemble Peak Flow Results Both “SD-Box” and “IQR-Box” were applied to provide peak flow forecasts. There is only one box left for both approaches because “Peak-Box” was removed to avoid any confusion. Figures8 and9 show all forecast results. The horizontal axis is the number of forecasts. The vertical axes represent the length in water depth in Figure8 and amount of time in Figure9 to address the size of the rectangle, which is the size of the “Box” in Figure7. In this analysis, only one forecast ensemble cycle at a time was compared, as indicated by the solid line shown as “xxx-Box single”. If the observed peak fell inside the box, it was labeled as a “hit” (shown as solid circles and squares in Figures8 and9) at that Sustainability 2020, 12, 4258 12 of 20 numberSustainability of forecasts. 2020, 12, x TheFOR PEER average REVIEW hit rates using the “IQR-Box” method were 33.3% (31/93) and2 of 52.6% 20 (49/93) for peak flow stage and timing, respectively. Using the “SD-Box” method improved the average rateand to 51.6%52.6% (49/93) (48/93) for and peak 64.5% flow (60 stage/93) forand stage timing, and respectively. timing, respectively. Using the “SD-Box” It is noted method that the improved individual the average rate to 51.6% (48/93) and 64.5% (60/93) for stage and timing, respectively. It is noted that performance of Typhoons Saola and Soulik are better than other events. One of the reasons is that the individual performance of Typhoons Saola and Soulik are better than other events. One of the the parameters in the hydrological models were calibrated using observations from Typhoons Saola reasons is that the parameters in the hydrological models were calibrated using observations from and Soulik. The purpose of the “Box” is not to generate a deterministic forecast but to simplify the Typhoons Saola and Soulik. The purpose of the “Box” is not to generate a deterministic forecast but information out of the ensemble forecasts. The sizes of the box help users to understand the event to simplify the information out of the ensemble forecasts. The sizes of the box help users to rangeunderstand in terms the of peakevent stagerange andin terms peak of timing. peak st Theage decisionand peak maker timing. can The act decision accordingly maker based can act on a rangeaccordingly as long asbased the on observations a range as long are as contained the observations in the box. are contained The results in the showed box. The that results the enveloping showed rectanglesthat the definedenveloping using rectangles the “SD-Box” defined method using werethe “S moreD-Box” reliable method for were decision-making more reliable during for decision- typhoon eventsmaking (i.e., during both have typhoon average events hit (i.e., rates both exceeding have average 50%). hit rates exceeding 50%). FiguresFigures8 and 8 and9 shows 9 shows that that “SD-Box” “SD-Box” has has comparatively comparatively larger larger values values of lengthof length in waterin water stage stage and in timeand in than time “IQR-Box”. than “IQR-Box”. The results The results show thatshow the th lengthat the length in water in stagewater of stage the downstreamof the downstream location (i.e.,location Zhuanwei (i.e., Zhuanwei Bridge) is Bridge) smaller is than smaller the onesthan the of upstream ones of upstream (i.e., Zhongshan (i.e., Zhongshan and Leawood and Leawood Bridge). However,Bridge). theHowever, results the of theresults length of the in timelength are in consistent time are consistent along the stream.along the Because stream. theBecause method the of definingmethod the of sizedefining of the the box size di ffofered, the box the “SD-Box”differed, the method “SD-Box” resulted method in aresulted comparatively in a comparatively larger region thanlarger the “IQR-Box”region than methodthe “IQR-Box” for most method cases for in themost figures. cases in It the is why figures. blue It makersis why areblue usually makers largerare thanusually black larger ones inthan the black figures. ones It in is the arguable figures. that It is a arguable larger box that should a larger have box a should better hithave rate. a better However, hit therate. performance However, (hits the whichperformance are solid (hits ones which in figures) are solid still ones highly in dependsfigures) onstill the highly performance depends of on rainfall the forecasts.performance For example,of rainfall the forecasts. length For in waterexample, stage the for length “SD-Box in water All” stage is larger for “SD-Box than that All” for is“SD-Box larger Single”than that during for “SD-Box of Dujuan Single” Typhoon, during however, of Dujuan it shows Typhoon, no hitshowever, for “SD-Box it shows All”. no hits A largerfor “SD-Box box does All”. not alwaysA larger mean box a higher does hitnot ratealways if none mean of spreada higher results hit canrate catchif none the of observations. spread results The can rainfall catch forecast the observations. The rainfall forecast capability of the system is important and reliable so the box can be capability of the system is important and reliable so the box can be described based on the forecast described based on the forecast results. As a comparison with previous study, the “SD-Box” method results. As a comparison with previous study, the “SD-Box” method produced a smaller box than the produced a smaller box than the “Peak-Box” method. For decision-making during typhoon events, it “Peak-Box” method. For decision-making during typhoon events, it is considered more efficient to is considered more efficient to apply the “SD-Box” method instead of a combination of the “Peak- apply the “SD-Box” method instead of a combination of the “Peak-Box” and “IQR-Box” methods. Box” and “IQR-Box” methods.

FigureFigure 8. Changes8. Changes in thein the magnitude magnitude of theof peakthe peak stage stag usinge using the “SD-Box”the “SD-Box” (blue) (blue) and “IQR-Box”and “IQR-Box” (black) methods(black) consideringmethods considering only the currentonly the forecast current (solidforecast lines) (solid and lines) inclusion and inclusion of all the ofavailable all the available forecasts at theforecasts present at timethe present (dotted lines).time (dotted The solid lines). circles The andsolid squares circles represent and squares that therepresent observations that the are containedobservations in the are “Box”, contained which in isthe a hit,“Box”, at that whic numberh is a hit, of at forecasts. that number of forecasts. Sustainability 2020, 12, 4258 13 of 20 Sustainability 2020, 12, x FOR PEER REVIEW 3 of 20

FigureFigure 9. 9.Changes Changes in in the the duration duration ofof thethe peak stage us usinging the the “SD-Box” “SD-Box” (blue) (blue) and and “IQR-Box” “IQR-Box” (black) (black) methodsmethods considering considering only only the the current current forecastforecast (solid lines) and and inclusion inclusion of of all all the the available available forecasts forecasts atat the the present present time time (dotted (dotted lines). lines). TheThe solidsolid circle circless and and squares squares represent represent that that the the observations observations are are containedcontained in in the the “Box”, “Box”, which which is is a a hit, hit, atat thatthat numbernumber of forecasts. forecasts.

4.3.4.3. Inclusion Inclusion of of All All Forecasts Forecasts with with DiDifferentfferent LeadLead Times during an an Event Event to to Expand Expand the the Sample Sample Size Size TheThe sample sample size size has has aa strongstrong eeffectffect inin termsterms of determining whether whether a aresult result is isstatistically statistically significant.significant. In In otherother words, the the number number of ofavaila availableble ensemble ensemble memb membersers is important is important for both for the both “SD- the “SD-Box”Box” and and “IQR-Box” “IQR-Box” methods. methods. For example, For example, the nu thember number of available of available ensemble ensemble members members for each for eachforecast forecast ranged ranged from from 11 to 11 14 to during 14 during operations. operations. Thus, the Thus, descriptive the descriptive statistics statistics were calculated were calculated using insufficient sample sizes (less than 30). The same issue exists in other studies that employ HEPSs using insufficient sample sizes (less than 30). The same issue exists in other studies that employ [10,31]. It is difficult to increase the number of ensemble members used in HEPSs, due to the limited HEPSs [10,31]. It is difficult to increase the number of ensemble members used in HEPSs, due to the computational resources that are available. limited computational resources that are available. As mentioned in Yang et al. [32], there is no evidence to show the forecast performance will be As mentioned in Yang et al. [32], there is no evidence to show the forecast performance will be significantly improved with lead time during a typhoon event. In order to discuss the relationship significantlybetween accuracy improved of forecast with lead and time lead during time, this a typhoon study calculated event. In orderthe error to discussin the maximum the relationship 4-h betweenrainfall accuracybetween the of average forecast forecasts and lead and time, the average this study observations calculated at the watershed error in the upstream maximum of the 4-h rainfallZhongshan between Bridge the average(4-h used forecasts herein indicates and the averagethe time observationsof concentration at the of watershedthe peak flow upstream at the of theZhongshan Zhongshan Bridge Bridge is approximately (4-h used herein 4 h). indicates Figure 10 theshows time that of there concentration is no obvious of thetrend peak in the flow errors at the Zhongshanin stage and Bridge timing, isapproximately regardless of the 4 h).length Figure of the 10 leadshows time. that The there correlation is no obvious coefficients trend were in the −0.09 errors inand stage 0.11, and respectively, timing, regardless and these of values the length indicate of the that lead no time.significant The correlationcorrelations coe existffi cientsbetween were errors0.09 − andin 0.11,stage respectively, or timing on and the theseone ha valuesnd and indicate lead time that on no the significant other. For correlations example, the exist best between and worst errors inforecasts stage or timingduring onTyphoon the one Dujuan hand andin terms lead timeof st onage the error other. were For the example, first and the fifth best forecasts, and worst forecastsrespectively. during However, Typhoon the Dujuan 6th forecast in terms was of better stage errorthan the were 5th, the which first andimplies fifth that forecasts, there is respectively. no trend However,in the cascading the 6th forecast forecasting was process. better than the 5th, which implies that there is no trend in the cascading forecastingIt is worth process. noting that the conclusion that there is no significant trend in forecast accuracy and leanIt istime worth is made noting based that theon limited conclusion ensemble that there members. is no significant Bulk ensemble trend members in forecast of accuracy forecast andmight lean timeimprove is made the based accuracy on limited of metrological ensemble variables members. when Bulk the ensemble forecast membersis close to ofthe forecast event. mightHowever, improve as thementioned accuracy ofabove, metrological it is difficult variables to increase when ensemb the forecastle members is close for to each the run event. or the However, number as of mentioned events analyzed. As a trade-off, this study proposes a method for including present and above, it is difficult to increase ensemble members for each run or the number of tropical cyclone previous forecasts in order to expand the sample size during the estimation process. events analyzed. As a trade-off, this study proposes a method for including present and previous forecasts in order to expand the sample size during the estimation process. Sustainability 2020, 12, 4258 14 of 20 Sustainability 2020, 12, x FOR PEER REVIEW 4 of 20

(a)

(b)

FigureFigure 10. 10.The The box-and-whisker box-and-whisker plot plot of of (a ()a the) the magnitude magnitude error error in in the the maximum maximum 4-h 4-h rainfall rainfall and and ( b(b)) the timingthe timing error error of maximum of maximum 4-h 4-h rainfall rainfall at at the the watershed watershed upstream upstream ofof the the Zhongshan Zhongshan Bridge Bridge during during thethe four four selected selected typhoon typhoon events. events. TheThe blueblue dotsdots indicateindicate the the ensemble ensemble means. means. The The inverted inverted triangles triangles indicateindicate the the time time of of occurrence occurrence of of the the maximummaximum 4-h rainfall. The The results results show show that that there there is isno no obvious obvious trendtrend in in lead lead time time for for the the errors errors in in either either thethe rainfallrainfall or timing. timing. The The error error is is defined defined as as the the predicted predicted rainfallrainfall (timing) (timing) minus minus observed observed rainfall rainfall (timing).(timing). Sustainability 2020, 12, 4258 15 of 20

Figure 11 illustrates comparisons between using one forecast and using all available forecasts with the “SD-Box” method (hereinafter indicated as “SD-Box Single” and “SD-Box All”) at Zhongshan Bridge. The performance of “SD-Box All” was more consistent than that of “SD-Box Single” in terms of both stage and timing. For example, the spread-skill scores for stage during Typhoon Soudelor ranged from 0 to 5 when the “SD-Box Single” method was used, but they were more consistent and closer to 1 with “SD-Box All”. The results showed that the inclusion of all available forecasts (“SD-Box All”) in the calculation process could expand the sample size and obtain more robust descriptive statistics than “SD-Box Single” does. The spread-skill score of “SD-Box Single” ranges between 2 and 5 implies that the spread between the members is too concentrated to cover the observed peaks. In addition, Figures8 and9 show the features in terms of the size of the box using “SD-Box All” and “SD-Box Single” approaches. The performance of “SD-Box Single” is comparatively similar to the one of “SD-Box All”. The biggest differences appear especially in the performance of upstream location. For example, the “SD-Box Single” cannot catch the peak stage at Zhongshan Bridge during Soudelor Typhoon but “SD-Box All” can. Although the box sizes of “SD-Box All” is larger than that of “SD-Box Single” (in most of forecast runs), the variations in the water depth and length of time were consistent in each update for “SD-Box All”. In other words, the variation in box size of “SD-Box All” is getting stable from update to update and the stable variation of the box might be useful for decision-making. This result implies that the forecasts including present and previous forecasts improve the performance of the HEPS and help decision-making because the results of “SD-Box All” is more consistent and stable. Finally, Figure 12 illustrates the scores of all of the forecasts for the different typhoon events. The “SD-Box Single” contained 51.7% of the observed peaks in terms of stage (40.9% + 10.8%), whereas “SD-Box All” contained 69.9% (62.4% + 7.5%) of the observed peaks. Furthermore, the “SD-Box Single” contained 64.6% (40.9% + 23.7%) of the observed peaks in terms of timing, whereas “SD-Box All” contained 77.5% (62.4% + 15.1%). Since the scores of “SD-Box Single” are widespread, it implies that the performance may experience wild swings in each run. In terms of stage and timing 62.4% of the scores of “SD-Box All” in comparison with 40.9% of “SD-Box Single” are less than one. It indicates that due to larger sample size the spread of “SD-Box All” is more appropriate to cover the uncertainty in the prediction. This just confirms the stability of the performance of “SD-Box All” and steady performance provides decision makers a basis to respond to a potential disaster more consistently. Sustainability 2020, 12, 4258 16 of 20

(a)

(b)

Figure 11. TheThe spread-skillspread-skill scores scores for for (a )( peaka) peak stage stage and and (b) peak (b) peak timing timing of the singleof the (“SD-Boxsingle (“SD-Box Single”) Single”)and accumulating and accumulating (“SD-Box (“SD-Box All”) methods All”) methods at the Zhongshan at the Zhongshan Bridge during Bridge the during four selectedthe four typhoonselected typhoonevents. The events. inverted The trianglesinverted indicatetriangles the indicate time of the occurrence time of occurrence of the observed of the peak observed stage. peak The resultsstage. Theshow results that the show performance that the performance of the “SD-Box of All”the “SD-Box method (solidAll” method circles) was(solid more circles) stable was than more that stable of the “SD-Boxthan that Single”of the “SD-Box method Single” (open circles) method in (open terms circles) of both in stage terms and of timing. both stage and timing.

Sustainability 2020, 12, x; doi: FOR PEER REVIEW www.mdpi.com/journal/sustainability Sustainability 2020, 12, 4258 17 of 20

Sustainability 2020, 12, x FOR PEER REVIEW 2 of 20

Figure 12. Comparison of the spread-skill scores obtained for “SD-Box Single” and “SD-Box All”. The Figure 12. Comparison of the spread-skill scores obtained for “SD-Box Single” and “SD-Box All”. results show that the “SD-Box All” approach provides comparatively consistent forecasts in terms of The results show that the “SD-Box All” approach provides comparatively consistent forecasts in terms variation of the score and it serves as a good basis for decision makers to respond the disasters. of variation of the score and it serves as a good basis for decision makers to respond the disasters.

5. Conclusions Conclusions This studystudy proposedproposed a HEPSa HEPS that that employs employs NWP NW modelsP models to perform to perform rainfall rainfall forecasts forecasts and employs and employshydrologic hydrologic models, including models, rainfall-runoincluding ffrainfall-runoff,, river-routing, andriver-routing, combined stormand combined surge/global storm tide surge/globalmodels to produce tide ensemblemodels to flood produce forecasts ensemble during typhoonflood forecasts events. Theduring communication typhoon events. of ensemble The communicationforecasts is critical of forensemble helping end-usersforecasts is respond. critical Afor visualization helping end-users approach respond. called “SD-Box” A visualization was also approachdeveloped called to support “SD-Box” the interpretation was also developed of ensemble to su forecastpport the results interpretation for operational of ensemble purposes. forecast A total resultsof 93 forecast for operational runs from purposes. four typhoon A total events of 93 during forecast the periodruns from 2012–2015 four typhoon and observations events during collected the periodin the Yilan2012–2015 Experimental and observations Watershed collected were used in the to evaluateYilan Experimental the performance Watershed of these were techniques. used to evaluateThe results the showedperformance that theof these proposed techniques. HEPS isThe able results to provide showed flood that forecaststhe proposed during HEPS the is selected able to providetyphoon flood events. forecasts The peak during flow the information, selected typhoon such ev asents. peak The stage peak and flow timing, information, are usually such the as mostpeak stageimportant and timing, for decision-making. are usually the For most the important purposes of fo simplicityr decision-making. and efficiency For the of operation,purposes of the simplicity “SD-Box” andvisualization efficiency approach of operation, considers the the “SD-Box” mean and visualization the standard deviationapproach toconsiders define the the size mean of the and box andthe standardfocuses on deviation the peak to flow, define capturing the size more of the of box the an observedd focuses peaks on the during peak typhoon flow, capturing events. more of the observedThis peaks study usedduring a spread-skill typhoon events. score based on peak stage and peak timing accuracy to evaluate the similarityThis study of the used ensemble a spread-skill averages score and observationsbased on peak and stage to verify and peak the peak timing stage accuracy forecasts. to evaluate Further theevaluation similarity measures of the ensemble related to averages flow volumes and observ can beations found and in to the verify literature the peak (e.g., stage series-distance forecasts. Furthermethods evaluation [33,34]). The measures average related spread-skill to flow scores volumes of the proposedcan be found HEPS in forthe the literature selected (e.g., events series- were distance1.13 and 1.00methods in terms [33,34]). of peak The stage average and peak spread-skill timing, respectively.scores of the Scores proposed of close HEPS to one for indicated the selected that eventsthe spread were of 1.13 the and ensemble 1.00 in forecaststerms of peak was largestage enoughand peak to timing, contain respectively. the prediction Scores uncertainty. of close to As one an indicated that the spread of the ensemble forecasts was large enough to contain the prediction uncertainty. As an input to the HEPS, the rainfall forecasts from NWP models exhibited a significant impact on the performance. The worst performance was found during because of Sustainability 2020, 12, 4258 18 of 20 input to the HEPS, the rainfall forecasts from NWP models exhibited a significant impact on the performance. The worst performance was found during Typhoon Dujuan because of its large track forecast errors. As a result, the spread-skill scores were above 2. However, details of the performance of the NWP models are beyond the scope of this study. Overall, the average score was close to one and satisfied the statement “One of the main objectives of ensemble flood forecasts is the representation of the full spectrum of forecast uncertainty or predictability in form of different hydrological responses to the input of the various members obtained from an atmospheric EPS” made by Zappa et al. [10]. The supporting visualization approach can therefore decrease the complexity of the HEPS forecast information and improve the efficiency of end-user’s decision-making. This study also proposed a method that enables increasing the sample size, leading to statistically significant results. This method involves including current and previously available forecasts in the calculation process. For example, the proposed HEPS generated 11 available ensemble members at each forecast during Typhoon Dujuan. By including all of the present and previously available forecasts (the “SD-Box All” method), the sample size increased to 22 for the second forecast, 33 for the third forecast, and so on. The results showed that the “SD-Box All” made more consistent predictions by including all available forecasts in the calculation process. The descriptive statistics, such as the quartile deviation and the standard deviation, can be more meaningful after expanding sample size. As a result, the rectangles defined by the “SD-Box All” method contained 57.8% of the observed peaks in stage and timing. Coughlan de Perez et al. [35] suggested that a HEPS that produces a false alarm rate below 50% is tolerable for decision makers in terms of the economic and practical consequences of taking action. However, this study assumed that the forecast performance of the proposed HEPS is independent of the length of the lead time and conducted an experiment to prove it. Other studies, such as that of Zappa et al. [10], have claimed that the most accurate forecasts were obtained for lead times of two or more days. Such statements imply that the performance of HEPSs do not improve with shorter lead times or are independent of lead time, and Yang et al. [32] found that the best performance is obtained before a typhoon makes landfall. This assumption is still susceptible to the topography of the applied area and the type of extreme event being considered. Further investigation of various conditions must be performed before firm conclusions can be drawn. Regardless, the proposed HEPS and the modified visualization approach have been shown to produce convincing peak stage and peak timing forecasts for operational purposes during a typhoon. In addition, the modified visualization approach is based on the well calibrated HEPS and is applicable only for typhoon events. Therefore, the users should properly apply it under these prerequisites.

Author Contributions: Data curation, C.-H.C.; formal analysis, S.-C.Y. and Y.-C.C.; funding acquisition, T.-H.Y.; methodology, Y.-C.C., M.-Y.L., J.-Y.H., and K.T.L.; validation, Y.-C.C.; visualization, S.-C.Y.; writing—original draft, S.-C.Y. and Y.-C.C.; writing—review and editing, T.-H.Y. All authors have read and agreed to the published version of the manuscript. Funding: This work was funded by the Ministry of Science and Technology, R.O.C., under grant MOST 105-3011-F-492-009. Acknowledgments: The authors thank the Water Resources Agency of Taiwan for providing the hydrological observations from the rainfall gauges and water-level stations in the Yilan River Basin. Thanks are also due to the Taiwan Typhoon and Flood Research Institute and the National Applied Research Laboratories for providing the results from the Taiwan Cooperative Precipitation Ensemble Forecast Experiment and historical records from the Yilan Experimental Watershed. Conflicts of Interest: The authors declare no conflict of interest.

References

1. Cloke, H.L.; Pappenberger, F. Ensemble flood forecasting: A review. J. Hydrol. 2009, 375, 613–626. [CrossRef] 2. Cuo, L.; Pagano, T.; Wang, Q. A Review of Quantitative Precipitation Forecasts and Their Use in Short- to Medium-Range Streamflow Forecasting. J. Hydrometeorol. 2011, 12, 713–728. [CrossRef] Sustainability 2020, 12, 4258 19 of 20

3. Pappenberger, F.; Beven, K.J.; Hunter, N.M.; Bates, P.D.; Gouweleeuw, B.T.; Thielen, J.; De Roo, A.P.J. Cascading model uncertainty from medium range weather forecasts (10 days) through a rainfall-runoff model to flood inundation predictions within the European Flood Forecasting System (EFFS). Hydrol. Earth Syst. Sci. 2005, 9, 381–393. [CrossRef] 4. Balint, G.; Csik, A.; Bartha, P.; Gauzer, B.; Bonta, I. Application of Meterological Ensembles for Danube Flood Forecasting and Warning. In Gheorghe Stancalie and Gabor Balint, Transboundary Floods: Reducing Risks through Flood Management; Marsalek, J., Ed.; Springer: Berlin/Heidelberg, Germany, 2006. 5. Davolio, S.; Miglietta, M.M.; Diomede, T.; Marsigli, C.; Morgillo, A.; Moscatello, A. A meteo-hydrological prediction system based on a multi-model approach for precipitation forecasting. Nat. Hazards Earth Syst. Sci. 2008, 8, 143–159. [CrossRef] 6. Todini, E. Predictive Uncertainty Assessment in Real Time Flood Forecasting. In Uncertainties in Environmental Modelling and Consequences for Policy Making; Baveye, P.C., Laba, M., Mysiak, J., Eds.; Springer: Berlin/ Heidelberg, Germany, 2009; pp. 205–228. 7. Giordani, A.; Zappa, M.; Rotach, M. Estimating Ensemble Flood Forecasts’ Uncertainty: A Novel “Peak-Box” Approach for Detecting Multiple Peak-Flow Events. Atmosphere 2019, 11, 2. [CrossRef] 8. Abebe, A.J.; Price, R.K. Decision support system for urban flood management. J. Hydroinformatics 2005, 7, 3–15. [CrossRef] 9. Pappenberger, F.; Stephens, L.; Thielen, J.; Salamon, P.; DeMeritt, D.; Van Andel, S.J.; Wetterhall, F.; Alfieri, L. Visualizing probabilistic flood forecast information: Expert preferences and perceptions of best practice in uncertainty communication. Hydrol. Process. 2012, 27, 132–146. [CrossRef] 10. Zappa, M.; Fundel, F.; Jaun, S. A ‘Peak-Box’ approach for supporting interpretation and verification of operational ensemble peak-flow forecasts. Hydrol. Process. 2012, 27, 117–131. [CrossRef] 11. Pagano, T.C.; Wood, A.W.; Ramos, M.-H.; Cloke, H.L.; Pappenberger, F.; Clark, M.; Cranston, M.; Kavetski, D.; Mathevet, T.; Sorooshian, S.; et al. Challenges of Operational River Forecasting. J. Hydrometeorol. 2014, 15, 1692–1707. [CrossRef] 12. Olsson, J.; Pers, C.; Bengtsson, L.; Pechlivanidis, I.; Berg, P.; Körnich, H. Distance-dependent depth-duration analysis in high-resolution hydro-meteorological ensemble forecasting: A case study in Malmö City, Sweden. Environ. Model. Softw. 2017, 93, 381–397. [CrossRef] 13. Wu, C.-C.; Kuo, Y.-H. Typhoons Affecting Taiwan: Current Understanding and Future Challenges. Bull. Am. Meteorol. Soc. 1999, 80, 67–80. [CrossRef] 14. Hsiao, L.-F.; Yang, M.-J.; Lee, C.-S.; Kuo, H.-C.; Shih, D.-S.; Tsai, C.-C.; Wang, C.-J.; Chang, L.-Y.; Chen, D.Y.-C.; Feng, L.; et al. Ensemble forecasting of typhoon rainfall and floods over a mountainous watershed in Taiwan. J. Hydrol. 2013, 506, 55–68. [CrossRef] 15. Huang, J.-C.; Yu, C.-K.; Lee, J.-Y.; Cheng, L.-W.; Lee, T.-Y.; Kao, S.-J. Linking typhoon tracks and spatial rainfall patterns for improving flood lead time predictions over a mesoscale mountainous watershed. Water Resour. Res. 2012, 48.[CrossRef] 16. Lee, K.T.; Yen, B.C. Geomorphology and Kinematic-Wave–Based Hydrograph Derivation. J. Hydraul. Eng. 1997, 123, 73–80. [CrossRef] 17. Lee, T.K.; Chang, C.H.; Yang, M.S.; Yu, W.S. Reservoir Attenuation of Floods from Ungauged Watersheds. Hydrol. Sci. J. 2001, 46, 349–362. [CrossRef] 18. Lee, T.K.; Chung, Y.R.; Lau, C.C.; Meng, C.C.; Chiang, S. A Windows-Based Inquiry System for Design Discharge Based on Geomorphic Runoff Modeling. Comput. Geosci. 2006, 32, 203–211. [CrossRef] 19. Lee, K.T.; Chen, N.-C.; Gartsman, B.I. Impact of stream network structure on the transition break of peak flows. J. Hydrol. 2009, 367, 283–292. [CrossRef] 20. Thiessen, A.H. Precipitation averages for large areas. Mon. Weather Rev. 1911, 39, 1082–1084. [CrossRef] 21. Blumberg, F.A.; Mellor, G.L. A Description of a Three-Dimensional Coastal Ocean Circulation Model, in Three-Dimensional Coastal Ocean Models; American Geophysical Union: Washington, DC, USA, 1987. 22. Ou, S.; Liu, J.; Tsai, C.; Hsu, T. Numerical Studies of Typhoon-Induced Storm Surge Using Pom and Finite Element Depth-Averaged Model in Taiwan. In Proceedings of the Chinese-German Joint Symposium on Hydraulic and Ocean Engineering, Darmstadt, Germany, 24–30 August 2008. 23. Chiou, M.D. Characteristic and Numerical Simulation of Astronomic Tide and Storm Surge in Taiwan Water. Ph.D. Thesis, National Cheng Kung University, Tainan, Taiwan, 2010. Sustainability 2020, 12, 4258 20 of 20

24. Yeh, T.G.; Cheng, H.P.; Cheng, J.R.; Lin, J.H. A Numerical Model to Simulate Flow and Contaminant and Sediment 1 Transport in Watershed Systems (Wash12d). In Technical Rep. Chl-98-15, Waterways Experiment Station; U.S. Army Corps of Engineers: Vicksburg, MS, USA, 1998. 25. Shih, D.-S.; Liau, J.-M.; Yeh, G.-T. (George) Model Assessments of Precipitation with a Unified Regional Circulation Rainfall and Hydrological Watershed Model. J. Hydrol. Eng. 2012, 17, 43–54. [CrossRef] 26. Yeh, T.G.; Huang, G.B.; Zhang, F.; Cheng, H.P.; Lin, H.C. Wash123d: A Numerical Model of Flow, Thermal Transport, and Salinity, Sediment, and Water Quality Transport in Watershed Systems of 1-D Stream-River Network, 2-D Overland Regime, and 3-D Subsurface Media; Technical Report Submitted to US EPA; Department of Civil and Environmental Engineering, University of Central Florida: Orlando, FL, USA, 2005. 27. Yeh, G.-T.; Shih, D.-S.; Cheng, J.-R.C. An integrated media, integrated processes watershed model. Comput. Fluids 2011, 45, 2–13. [CrossRef] 28. Shih, D.-S.; Chen, C.-H.; Yeh, G.-T. Improving our understanding of flood forecasting using earlier hydro-meteorological intelligence. J. Hydrol. 2014, 512, 470–481. [CrossRef] 29. Levine, R.A.; Wilks, D.S. Statistical Methods in the Atmospheric Sciences. J. Am. Stat. Assoc. 2000, 95, 344. [CrossRef] 30. Yang, T.-H.; Tsai, C.-C. Using numerical weather model outputs to forecast wind gusts during typhoons. J. Wind. Eng. Ind. Aerodyn. 2019, 188, 247–259. [CrossRef] 31. Yang, S.-C.; Yang, T.-H. Uncertainty Assessment: Reservoir Inflow Forecasting with Ensemble Precipitation Forecasts and HEC-HMS. Adv. Meteorol. 2014, 2014, 1–11. [CrossRef] 32. Yang, T.-H.; Hwang, G.-D.; Tsai, C.-C.; Ho, J.-Y. Using rainfall thresholds and ensemble precipitation forecasts to issue and improve urban inundation alerts. Hydrol. Earth Syst. Sci. 2016, 20, 4731–4745. [CrossRef] 33. Ehret, U.; Zehe, E. Series distance–an intuitive metric to quantify hydrograph similarity in terms of occurrence, amplitude and timing of hydrological events. Hydrol. Earth Syst. Sci. 2011, 15, 877–896. [CrossRef] 34. Seibert, S.P.; Ehret, U.; Zehe, E. Disentangling timing and amplitude errors in streamflow simulations. Hydrol. Earth Syst. Sci. 2016, 20, 3745–3763. [CrossRef] 35. De Perez, E.C.; Hurk, B.V.D.; Van Aalst, M.K.; Amuron, I.; Bamanya, D.; Hauser, T.; Jongma, B.; Lopez, A.; Mason, S.; De Suarez, J.M.; et al. Action-based flood forecasting for triggering humanitarian action. Hydrol. Earth Syst. Sci. 2016, 20, 3549–3560. [CrossRef]

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).