Abistado, K. G., Arellano, C. N., and Maravillas, E. A.

Paper: Using Artificial Neural Network and Bayesian Network

Klent Gomez Abistado∗, Catherine N. Arellano∗∗, and Elmer A. Maravillas∗∗

∗Advanced World Systems Inc. Cebu City, Philippines ∗∗Department of Computer Science, Cebu Institute of Technology University Cebu City, Philippines E-mail: aklent [email protected], [email protected], [email protected] [Received February 9, 2014; accepted May 25, 2014]

This paper presents a scheme of weather forecasting existing methods of weather forecasting give only short using artificial neural network (ANN) and Bayesian time span for which a weather forecast applies [1]. network. The study focuses on the data represent- Researchers mainly in the field of atmospheric science ing central Cebu weather conditions. The parame- attempted to predict the state of the atmosphere in a given ters used in this study are as follows: mean , location for a future time. A lot of ways were utilized minimum , maximum temperature, mean to do the weather forecasting based on the data gath- temperature, mean relative , rainfall, aver- ered from weather instruments, such as the , age wind speed, prevailing , and mean hygrometer, , and radar. Collection of cloudiness. The weather data were collected from the data about the current state of the atmosphere is the ba- PAG-ASA Mactan-Cebu Station located at latitude: sic step. Particularly, data on the (temperature, humidity 10◦19, longitude: 123◦59 starting from January 2011 and wind) and understanding the atmospheric processes to December 2011 and the values available represent (through ) is important in determining how daily averages. These data were used for training the the atmospheric conditions evolve [2]. multi-layered backpropagation ANN in predicting the Information obtained from the instruments is effec- weather conditions of the succeeding days. Some out- tively analyzed using numerical weather prediction. This puts from the ANN, such as the humidity, tempera- method was first proposed by Lewis Fry Richardson in ture, and amount of rainfall, are fed to the Bayesian 1922 [3], though computers with the capability to do such network for statistical analysis to forecast the prob- computations do not yet exist at that time. ability of rain. Experiments show that the system As the computers grow into a more powerful instru- achieved 93%–100% accuracy in forecasting weather ment, using numerical data weather prediction becomes conditions. more eminent. This leads to an essential definition that a model is a computer program that produces meteoro- logical information for future times at given locations and Keywords: artificial neural networks, backpropagation, altitudes [4]. Modern models are contained by the set of bayesian network, weather forecast, PAG-ASA equations, known as the primitive equations, used to pre- dict the future state of the atmosphere [5]. These existing methods basically use a sample state of 1. Introduction the fluid at a given time and use the equations of fluid dynamics and thermodynamics to estimate the state of the Weather condition is very important for everybody. fluid at some time in the future [4]. Though this method Government authorities, businessmen, farmers, security provides us a clear definition, still it does not contain the planners, and a host of interested entities would want to ability to adapt changes and learn from experiences. With know what lies ahead as far as weather is concerned. Cor- these abovementioned factors, the application of artificial rect understanding of weather behavior is a must to be intelligence for weather forecasting becomes an interest able to make plans to prevent damages of catastrophic for the modern technology. proportions and to be able to focus meager resources of Neural network is an effective tool for capturing com- government to areas that will be unavoidably affected. plex relationship between input and output [6]. The con- Inaccurate forecasts bring a lot of problems. Many lives cept of neural network technology is motivated for the de- will be unnecessarily sacrificed and endangered, farm sire that it could perform intelligent tasks similar to those products lost, damage to properties amounting to mil- performed by the human brain. Neural network [7] pic- lions of dollars, can be caused by wrong weather fore- tures a human brain in two behaviors: casts. It cannot be denied that weather forecasts nowa- days have been tainted with inaccuracies. This is because 1) A neural network acquires knowledge through learn-

812 Journal of Advanced Computational Intelligence Vol.18 No.5, 2014 and Intelligent Informatics Weather Forecasting Using ANN and Bayesian Network

represent the weather conditions of the following day. The number of hidden neurons is defined as twice the sum of input and output neurons. There are 14 input parameters and 12 output parameters for the network. Each param- eter is presented to the network as a 16-bit binary value. Therefore, the number of neurons on the input layer is Fig. 1. Weather forecasting systems using artificial neural 224, while on the output layer is 192. Therefore, there are network and bayesian network. (224+192)(2) or 832 hidden neurons in the network.

3. Weather Data Preprocessing

The inputs to the neural network are weather data ob- tained from the history of weather conditions of Cebu province as gathered by PAG-ASA which situated in Mac- tan, Cebu from January 2011 to December 2011. The val- ues in the data are the daily means of the extreme weather conditions. The output of the network is the weather con- dition of the day following from when the input weather condition was taken. Neurons like to see data in a particular input range to be most effective. If you present data that varies from 100 to 500 will not be useful, since the middle layer of neurons have a Sigmoid Activation function that squashes large in- putstoeither0or+1. In other words, one should choose Fig. 2. Weather forecasting neural network architecture us- data that fit a range that does not saturate, or overwhelm ing backpropagation training algorithm. the network neurons. Choosing inputs from −1to1or0 to 1 is a good idea. By the same token, one should nor- malize the expected values for the outputs to the 0 to 1 sigmoidal range [8]. ing. Cognizant of the abovementioned guidelines, it is de- 2) A neural network’s knowledge is stored within in- cided that each input parameter shall be represented by terneuron connection strengths known as synaptic a bit-string of length 16. Since each of the weather con- weights. dition parameter has maximum and minimum values, an input parameter value shall be assigned a value within the In broad representation, neural networks have the abil- range from 0 to 100000. The actual input value will be ity to represent both linear and nonlinear relationships di- computed using Eq. (1) below. rectly from the data being modeled. Actual input value =((P − P )/(P − P )) This proposed Weather Prediction system using BPN min max min ∗ , Neural Network and Bayesian Network, as shown in 100000 ..... (1) Fig. 1 , was developed and tested using the weather data where P is any parameter value, Pmin is the day’s mini- from PAG-ASA Mactan-Cebu, Cebu, Philippines. mum value of P, and Pmax is the day’s maximum value of P. The resulting actual input value will be converted into a 16 bit binary number. The same will be done to the other 2. Weather Forecasting Artificial Neural Net- input parameters’ values. work (WFANN) Given the values of each input parameter, maxima and minima values of each is also identified. Example for ev- The artificial neural network used in this study was ery month of April, statistics show that the maximum tem- trained using the backpropagation training algorithm, as perature is 33 degrees Celsius with a probability of 0.84 shown in Fig. 2. The inputs to the network are as fol- and minimum temperature is 23 degrees Celsius with a lows: month of the year, day of the month, mean sea probability of 0.7. To get the partial value of the input level pressure, dry-bulb temperature, wet-bulb tempera- parameter for temperature, percentage value is computed. ture, mean dew point, minimum temperature, maximum Using the previous example, maxima is 33 with 0.84 and temperature, mean temperature, mean relative humidity, minima is 23 with 0.7. Get the range with the formula rainfall, average wind speed, prevailing wind direction, maxima – minima. Therefore 33 − 23 is 10. Initial in- and mean cloudiness. The output parameters of the net- put value for an example is 28 degrees Celsius. 28 sub- work are similar to the input parameters except for the tracted by minima which results to 5. 5 divided by the month of the year and the day of the month which are no rangewhichis10thenresultedto0.5.0.5 × 100 is 50. longer necessary since the output values of the network

Vol.18 No.5, 2014 Journal of Advanced Computational Intelligence 813 and Intelligent Informatics Abistado, K. G., Arellano, C. N., and Maravillas, E. A.

and there is no rain, respectively. The values for humid- ity, temperature, and rainfall are classified into three (3) classes, namely, Low, Medium, and High. The full joint probability distribution of a Bayesian net- work [9] is given by the formula n P(x1,...,xn)=∏P(xi|parents(Xi)) .... (2) i=1

where parents (xi) denotes the specific values of the vari- ables that directly influence xi. Thus, for the Bayesian network shown in Fig. 3, the joint probability distribution Fig. 3. Bayesian network for predicting the chance of is given by rain. There are three independent variables, namely, Humid- ( , , , )= ( | ) ( | ) ( | ) ( ), ity, Temperature, and Rainfall that are directly influenced P R H T RF P H R P T R P RF R P R (3) by Rain. The corresponding conditional probability tables where R is rain, H is humidity, T is temperature, and RF (CPTs) for each variable are shown. is rainfall. Input values for the instance of Humidity and Temper- ature ranges from 0 to 100. The probability value for the 4 instances depends on the time and place of the forecast The value 50 is converted to a 16 bit binary number which to be made and to be analyzed. Humidity value and tem- is 0000000000110010. For more accuracy you can range perature value are categorized as low, medium, and high. the values from 0 to 1000. In this paper 0 to 100000 is be- The percentage value of humidity is from the output data ing used to include decimal values. Each bit value holds of the neural network. The percentage value for the tem- 1 input neuron. All the bit values of each parameter are perature is computed using the ratio of wet-bulb to dry- stored in a data set for neural networks calculation. bulb . (Wet bulb/Dry bulb) × 100 is value Inputs from the training data set are fed to neural net- for Temperature in the Bayesian network instance. works through the input layer units. These somewhat nor- To illustrate this, we can calculate the probability of malized values that are represented in binary are directed rain given that humidity is Low, temperature is High, and into a series of calculations within the hidden layer. After rainfall is Low. The computation is given below: the specified number of iterations, weights will have been ( = , = , = , = ) adjusted and the given training set is learned. P R Yes H Low T High RF Low Using the new adjusted weights within layers, the out- = P(H = Low|R = Yes)P(T = High|R = Yes) put layer will produce values in percentages. These val- P(RF = Low|R = Yes)P(R = Yes) ues determine the probability of change from the given P(R = Yes, H = Low, T = High, RF = Low) inputs. These values will be converted into real number =( . )( . )( . )( . )= . , data using the maxima and minima value of the specific 0 1 0 7 0 1 0 45 0 00315 ...(4) parameter before running it to the next iteration. thus, the probability that there is rain given that humid- ity is Low, temperature is High, and rainfall is Low is 0.315%. 4. Bayesian Network for Probability of Rain Exact inference can be done when not all variable data are available. In this case we have three types of variables, Since the output data of the artificial neural are fore- namely, the query variables, evidence variables, and the cast data, it is necessary to state the confidence level at hidden variables. If we want to infer the probability of which these data would actually occur. To do this task, rain given that humidity is medium, and temperature is the Bayesian network is employed. Bayesian network is high, then the query variable is rain, the evidence vari- a data structure that represents the dependencies among ables are humidity and temperature, while the hidden vari- variables with a concise specification of any full joint able is rainfall. The formula that we will use to compute probability distribution [9]. this probability is as follows: The topology of the Bayesian network for computing ( = | = , = ) the chance of rain is shown in Fig. 3. The three variables P R Yes H Medium T High humidity, temperature, and rainfall are conditionally inde- = αP(R = Yes)ΣRF P(H = Medium|R = Yes) pendent from each other given that it rained. The changes P(T = High|R = Yes)P(RF|R = Yes), .(5) in the values of humidity, temperature, and rainfall are α ( ( = | = directly influenced by rain, whereas no direct causal rela- where is equal to the reciprocal of P R Yes H , = )+ ( = | = , = tionship exists between them. Medium T High P R No H Medium T )) Σ ( = | = ) ( = | = Figure 3 also shows the conditional probability tables High . RF P H Medium R Yes P T High R ) ( | = ) (CPTs) for the four nodes. The CPTs provide a decom- Yes P RF R Yes means that we’re going to get the sum posed representation of the probability distribution. The of this product when RF is Low, RF is Medium, and RF symbols YES, Yes, NO, and No stand for there is rain

814 Journal of Advanced Computational Intelligence Vol.18 No.5, 2014 and Intelligent Informatics Weather Forecasting Using ANN and Bayesian Network

Table 1. Sample data from PAG-ASA. PRESSURE TEMPERATURE (◦C) WIND (hPa) Month Day Mean Dry Wet Max. Min. Mean Mean Mean Rain- Average Prevail- Mean Iden- of Sea Level Bulb Bulb Dew Relative fall Speed ing Cloudi- tifica- the Pressure Point Humid- (mm) (mps) Direction ness tion Month ity (degrees) (Qkt) 1 1 1013.2 27.2 24.9 31.0 23.8 27.4 24.1 83 10.4 3 20 7 1 2 1012.6 26.8 25.1 31.0 24.0 27.5 24.5 87 1.2 3 20 5 1 3 1012.6 27.2 25.2 30.5 24.4 27.5 24.5 85 0.0 2 320 5 1 4 1013.0 27.5 25.6 30.4 24.6 27.5 25.0 86 6.6 3 360 6 1 5 1013.6 25.8 24.9 26.8 24.8 25.8 24.6 93 4.6 3 360 8 1 6 1014.0 26.8 25.4 30.0 24.2 27.1 24.9 89 24.0 3 40 6 1 7 1013.9 26.5 25.2 29.5 24.0 26.8 24.8 90 17.5 3 360 7 1 8 1013.9 26.8 25.4 29.0 24.0 26.5 24.9 89 1.4 3 360 7 1 9 1012.6 26.9 25.5 30.0 25.0 27.5 25.0 89 1.1 3 360 7 1 10 1012.9 26.4 25.2 30.0 24.0 27.0 24.8 91 20.8 4 360 8 1 11 1013.9 25.6 24.5 29.0 24.0 26.5 24.1 91 19.4 4 20 6

Table 2. Experimental results for predicting next day weather conditions using seen day data. Experiment No. Date Day Parameter Actual Target Error Seen Mean Sea Level Pressure 1011.4 1011.4 0.0000 Seen Mean Temp 24.8 24.8 0.0000 Seen Max Temp 26.2 26.2 0.0000 1 2-Jan-11 Seen Min Temp 23.4 23.4 0.0000 Seen Dry Bulb 24.4 24.4 0.0000 Seen Wet Bulb 23.7 23.7 0.0000 Seen Humidity 94 94 0.0000 Seen Dew Point 23.5 23.5 0.0000

is High. Therefore, ing day. This WFANN was able to do so with 99-100% accuracy. Table 2 shows the result of the weather con- P(R = Yes|H = Medium, T = High) ditions for January 2, 2011 when WFANN was fed with = α(0.45)(0.3)(0.7)(0.1 + 0.2 + 0.7)=0.0945α weather conditions for January 1, 2011. In the second method, prediction is done by using data P(R = No|H = Medium, T = High) of a particular day that was not used in training (unseen = α(0.55)(0.7)(0.3)(0.9 + 0.8 + 0.3)=0.231α day) the network to predict the weather conditions of the following day. This WFANN was able to do so with an where α = 1/(0.0945 + 0.231)=3.072197. average of 93% accuracy. Table 3 shows the result of the Thus, weather conditions for January 2, 2012 when WFANN P(R = Yes|H = Medium, T = High)= was fed with weather conditions for January 1, 2012. (0.0945)(3.072197)=0.290323 or 29% chance that The WFANN was also tested to predict the weather there is rain. conditions multiple days ahead from the day where the P(R = No|H = Medium, T = High)= input data were gathered. In this testing, the input data (0.231)(3.072197)=0.709078 or 71% chance that used were that of December 31, 2011. These data were there is no rain. also used when the WFANN was trained. The objec- tive of the testing is to predict weather conditions two (2) days after December 31, three (3) days after, and so 5. Experimental Results on. Fig. 4 shows the performance of WFANN in predict- ing the weather conditions2, 3, ..., 7 daysafter when The WFANN was trained using 365 weather reports December 31, 2011 data were used as inputs. The per- coming from PAGASA-Cebu for the period from January formance does not indicate any constant decay for longer 2011 to December 2011, a portion of it is shown in Ta- prediction days. But apparently the performance becomes ble 1. Then it was tested using data coming from weather weaker for longer periods. reports for the period from January 2012 to December Figure 5 shows a snapshot of the graph of atmospheric 2012. The WFANN was tested in two ways. temperatures for future weather conditions. It is evident In the first method, prediction is done by using data of that the predicted temperature values are consistently po- a particular day that was used in training (seen day) the sitioned, that is, the mean dew point is always below the network to predict the weather conditions of the follow- wet-bulb, and the wet-bulb is always below the dry-bulb,

Vol.18 No.5, 2014 Journal of Advanced Computational Intelligence 815 and Intelligent Informatics Abistado, K. G., Arellano, C. N., and Maravillas, E. A.

Table 3. Experimental results for predicting next day weather conditions using unseen day data. Experiment No. Date Day Parameter Actual Target Error Unseen Mean Sea Level Pressure 1011.6 1012.6 −0.0010 Unseen Mean Temp 27.1 27.5 −0.0145 Unseen Max Temp 29.4 31 −0.0516 3 2-Jan-12 Unseen Min Temp 24.9 24 0.0375 Unseen Dry Bulb 28.1 26.8 0.0485 Unseen Wet Bulb 25.7 25.1 0.0239 Unseen Humidity 86 87 −0.0115 Unseen Dew Point 24.1 24.5 −0.0163 Unseen Mean Sea Level Pressure 1011.2 1012.6 -0.0014 Unseen Mean Temp 27.9 27.5 0.0145 Unseen Max Temp 31.3 30.5 0.0262 4 3-Jan-12 Unseen Min Temp 24.5 24.4 0.0041 Unseen Dry Bulb 27.6 27.2 0.0147 Unseen Wet Bulb 24.6 25.2 -0.0238 Unseen Humidity 85 85 0.0000 Unseen Dew Point 24.7 24.5 0.0082

Fig. 4. Performance of WFANN in predicting weather con- ditions2,3,..., 7daysafterwhenDecember31, 2011 data were used as inputs.

Fig. 5. A snapshot of the system’s graph for future temper- ature behavior. andsoon. Fig. 6 shows a snapshot of the graph of fu- ture wind speeds 2,3,..., 8 days in the future. The graph indicates that from 3 miles per second it increased to 4 miles per second for two (2) before dropping eventually to 2 miles per second. This means that the turbulence will be gaining speed in the vicinity for two (2) days before it will finally leave four (4) days after. This is consistent with normal wind behavior. Precipitation percentages are shown in Fig. 7. From a high of 60% it went down to Fig. 6. A snapshot of the system’s graph for future wind behavior. 5% for two days then it went high again at 60% before it slowly gone to 5% again on the 8th days after. More or less this is indicative of a normal situation wherein if it’s raining today chances are it will not tomorrow. to actually occur. For the system to maintain accuracy in predicting weather conditions in the future, synaptic weights of the 6. Conclusion and Recommendations artificial neural networks must be up to date. The system needs to be retrained in an acceptable time frame for it to The Back propagation neural network and Bayesian function accordingly. network are undoubtedly useful in weather forecasting. The WFANN as results would show is able to recall and predict weather conditions even for several days ahead Acknowledgements with acceptable accuracy. The Bayesian network helps in This research has been made possible thru the generous support affirming the chances of prospective weather conditions of PAGASA Mactan-Cebu office and the CIT-University.

816 Journal of Advanced Computational Intelligence Vol.18 No.5, 2014 and Intelligent Informatics Weather Forecasting Using ANN and Bayesian Network

Name: Catherine N. Arellano

Affiliation: Instructor, Computer Science Department, Col- lege of Computer Studies, Cebu Institute of Technology University

Address: Katipunan, Labangon, Cebu City, Philippines Brief Biographical History: 2002-2004 Management Information Systems Staff at Super Value Fig. 7. A snapshot of the system’s graph for future precipitation. Incorporated 2004-2007 Faculty at Northern Negros State College of Science and Technology 2007- Faculty at Cebu Institute of Technology University References: Main Works: • [1] S. Santhosh Baboo and I. Kadar Shereef, “An Efficient Weather Philnits Passer Forecasting System using Artificial Neural Network,” Int. J. of En- • Data Communication and Networking vironmental Science and Development, Vol.1, No.4, pp. 321-326, • Neural Networks 2010. Membership in Academic Societies: [2] Science Daily, Weather forecasting, • Philippine Society of IT Educators (PSITE) http://www.sciencedaily.com/articles/w/weather forecasting.htm. [3] P. Lynch, “The Emergence of Numerical Weather Prediction,” Cam- bridge University Press, 32, 2006. [4] Wikipedia.com., Weather forecasting, Numerical weather predic- tion. [5] R. A. Pieke, “Mesoscale Meteorological Modeling,” Academic Press, pp. 48-49, ISBN 0-12-54766-8, 2002. Name: Elmer A. Maravillas [6] M. Hayati and Z. Mohebi, “Application of Artificial Neural Net- works for Temperature Forecasting,” Proc. of World Academy of Science, Engineering and Technology, Vol.22, p.275, 2007. Affiliation: [7] S. Chattopadhyay, “Multilayered feed forward Artificial Neural Full Professor, College of Computer Studies, Network model to predict the average summer-monsoon rainfall in Cebu Institute of Technology (CIT) University India,” 2006. [8] V. Rao and H. Rao, “C++ Neural Networks and Fuzzy Logic,” 2nd Ed., MIS:Press, New York, 1995. [9] S. Russell and P. Norvig, “Artificial Intelligence: A Modern Ap- proach,” 2nd Ed., PEARSON-Prentice Hall, 2003. Address: Villa St. Joseph, Zacate, Basak, Pardo, Cebu City, Philippines Brief Biographical History: 1981- Received B.S. degree in Mechanical Engineering from Western Name: Mindanao State Klent Gomez Abistado 1982-1984 Mechanical Project Engineer and Power Plant Engineer, M&S Company Inc. Affiliation: 1984-2003 Department Head, Computer Engineering Department, Western Research and Development Engineer, Advanced Mindanao State University World Systems Inc. 1998- Received M.S. degree in Computer Science from De La Salle University-Manila 2005-2013 Chair, Computer Science, College of Computer Studies, CIT-University 2006- Received Ph.D. degree in Mechanical Engineering from De La Salle University-Manila Address: 2006-2007 Department Head, Research and Development Coordinating 5F PDI Condominium, Archbishop Reyes Ave. cor. Panis St., Banilad, Office (RDCO), CIT-University Cebu City 6000, Philippines 2008-2014 Consultant, Research and Development Coordinating Office Brief Biographical History: (RDCO), CIT-University 2012 Passed Philippine Information Technology General Certification 2014- Chair, Graduate Program, College of Computer Studies, Examination (Phil-IT-GCE) CIT-University 2013 Finished B.S. degree course in Computer Science from the Cebu Main Works: Institute of Technology-University • Modeling and Simulations 2014- Passed PhilNITS Certification Exam • Fuzzy Logic Systems Main Works: • Neural Networks • Java and C# Applications, Image Processing, Fuzzy Logic Systems, and • Intelligent Systems, and Expert Systems Artificial Neural Networks Membership in Academic Societies: • Business Applications and Systems Development • Philippine Society of IT Educators (PSITE) Membership in Academic Societies: • Philippine National I.T. Standards (PhilNITS) Society • Computer Student Society : Cebu Institute of Technology-University

Vol.18 No.5, 2014 Journal of Advanced Computational Intelligence 817 and Intelligent Informatics