PDF, Analysis of Backpropagation Method with Sigmoid Bipolar And

Home , Activation function, Sigmoid function

Journal of Physics: Conference Series

PAPER • OPEN ACCESS Recent citations Analysis of Backpropagation Method with Sigmoid - Performance One-step secant Training Method for Forecasting Cases Bipolar and Linear Function in Prediction of N L W S R Ginantra et al Population Growth - Application of ELECTRE Algorithm in Skincare Product Selection Nurliana Nasution et al To cite this article: Eben Siregar et al 2019 J. Phys.: Conf. Ser. 1255 012023

View the article online for updates and enhancements.

This content was downloaded from IP address 170.106.35.229 on 24/09/2021 at 16:34

The International Conference on Computer Science and Applied Mathematic IOP Publishing IOP Conf. Series: Journal of Physics: Conf. Series 1255 (2019) 012023 doi:10.1088/1742-6596/1255/1/012023

Analysis of Backpropagation Method with Sigmoid Bipolar and Linear Function in Prediction of Population Growth

Eben Siregar1, Herman Mawengkang1, Erna Budhiarti Nababan1, Anjar Wanto2 1 Universitas Sumatera Utara, Medan - Indonesia 2 STIKOM Tunas Bangsa Pematangsiantar, Medan - Indonesia

*[email protected]

Abstract. Backpropagation method is an artificial neural network method that is often used for prediction. However, the use of activation functions and training functions greatly affects the accuracy of a prediction. In this study will discuss the backpropagation method by applying the activation function of Sigmoid bipolar and linear to predict population growth in Simalungun regency, Indonesia. The purpose of this paper is to look at the level of population growth in the district so that the government has a benchmark in determining policies so that a surge in population growth can be minimized and that the government pays more attention to the level of welfare of its population. As for academics, this research can be used as input if you want to do a prediction or forecasting with different cases. The data used in this paper is population density data in Indonesia's Simalungun district, which is sourced from the Simalungun regency statistics center of Indonesia. This study uses 5 architectural models, namely 3-5-1, 3-10-1, 3- 5-10-1, 3-5-15-1 and 3-10-15-1. Of these 5 models, the best architectural model is 3-5-10-1 with an accuracy of 97% and an MSE value of 0.00034833. Minimum Error 0,001-0,01 and learning rate 0,01.

1. Introducing Figures on human intelligence have been applied to computers called artificial intelligence. The use of computers to replace humans is not without cause, because based on the results of research by experts, some work done by computers results in more efficient, measurable and effective. So it's no wonder now that the role of networks has become increasingly important. Backpropagation is a learning algorithm to reduce the level of error by adjusting its weight based on the difference in output and the desired target. Backpropagation is also a systematic method for multilayer training of Artificial Neural Networks [1][2]. Backpropagation is said to be a multilayer training algorithm because Backpropagation has three layers in the training process, namely input layer, the hidden layer and output layer, where backpropagation is a development of single layer network which has two layers, namely input layer and output layer [3][4]. With the hidden layer in backpropagation can cause the level of error in backpropagation is smaller than the level of failure in the single layer network. This is because the hidden layer in backpropagation serves as a place to update and adjust the weight so that we get a new weight value that can be directed towards the desired output target [5][6]. Backpropagation method is often used in the forecasting process; it is because the backpropagation method can optimize the weights and biases that will be used in forecasting. By optimizing the weight, the error level (mean square error) is getting smaller, saying that the forecasting value is better representing the actual value. One indicator that can influence the results of the backpropagation algorithm is the use of an activation function. The activation function is used to determine the output of a neuron. In backpropagation the activation function used must meet several conditions, namely:

Content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI. Published under licence by IOP Publishing Ltd 1

The International Conference on Computer Science and Applied Mathematic IOP Publishing IOP Conf. Series: Journal of Physics: Conf. Series 1255 (2019) 012023 doi:10.1088/1742-6596/1255/1/012023 continuous, differentiable easily and is a function that does not go down [7]–[9]. One function that fulfills these three conditions so that they are often used is the sigmoid binary function (Logsig) which has a range (0, 1). Another function that is often used is the sigmoid bipolar function (Tansig) whose function is similar to the sigmoid binary function, but with a range (-1, 1). While these two functions are often combined with linear functions (purelin) which have the same output value as the input value. Therefore the author will analyze the sigmoid bipolar function (tansig) and linear function (purelin), with the aim of finding the best combination of activation functions that can be used as a reference in the forecasting process. Especially because the sigmoid binary function (logsig) and linear functions (purelin) have been used by researchers before, which the author has stated in previous research in this paper. In previous research, Research has been conducted to predict population density by using the binary sigmoid activation function and linear functions. Forecasting accuracy utilizing this combination of services is 94%. The drawback of this research is that this study only uses binary sigmoid functions and linear functions without even a single discussion of bipolar sigmoid functions [10]. Next, research was conducted using the Tansig Activation Function (MLP Network) to detect Abnormal Hearts using the Tansig activation function (Bipolar). The results of this study indicate that the Multilayer Perceptron (MLP) network with the Modified Recursive Prediction Error (MRPE) training algorithm provides the lowest Mean Square Error (MSE) performance and the highest regression value. The lack of this research is the activation function which only uses the Tansig activation function (bipolar), does not explain other activation functions that are likely to produce better results [11]. From this background, the author feels interested in raising the title of the research “Analysis of Backpropagation Method with Sigmoid Bipolar and Linear Function in Prediction of Population Growth”.

2. Methodology

2.1. Data Used The data used is population data taken from the Simalungun Statistics Indonesia Central Board. (https://simalungunkab.bps.go.id).

Table 1. Simalungun Regency Population Data in Indonesia

Population Density (Soul/Km2) No Districts 2010 2011 2012 2013 2014 2015 1 Bandar 582,00 598,00 651,00 654,41 669,14 677,20 2 Bandar Huluan 251,00 254,00 242,00 242,65 244,80 245,75 3 Bandar Masilam 249,00 250,00 269,00 270,00 271,08 271,91 4 Bosar Maligas 132,00 134,00 138,00 138,72 140,39 141,22 5 Dolok Batu Nanggar 312,00 315,00 373,00 373,29 377,01 378,73 ...... 28 Silimakuta 176,00 184,00 194,00 196,00 203,80 208,36 29 Silou Kahean 77,00 78,00 75,00 75,33 76,08 76,43 30 Tanah Jawa 218,00 219,00 269,00 269,65 271,68 272,52 31 Tapian Dolok 325,00 334,00 327,00 328,43 335,62 339,54 32 Ujung Padang 181,00 182,00 178,00 178,69 179,84 180,29

2.2. Stages of Research Broadly speaking, the stages of research in this study can be described as follows:

The International Conference on Computer Science and Applied Mathematic IOP Publishing IOP Conf. Series: Journal of Physics: Conf. Series 1255 (2019) 012023 doi:10.1088/1742-6596/1255/1/012023

Figure 1. Stages of Research

Explanation in Figure 1, Literature study means collecting data or taking from sources related to the topic of discussion. Literature study is obtained from various sources such as journals, documentation books, and the internet. Then sampling the data from the Simalungun Regency Statistics Indonesia. Data will be processed using the Backpropagation method, activation function of sigmoid bipolar (tansig) and linear function (purelin). System Design means designing input, file structure, programs, procedures needed to support information systems. Implementation in the form of actions or plans that are prepared based on system design. System testing is an evaluation stage of the system architecture that has been built. System Evaluation includes a review of the results of system performance.

2.3. Data Normalization The data in table 1 will be normalized using the normalization formula [12][13]: 0.8( x a ) x '  0.1 (1) b a

Explanation : x' : Data transformation x : Data to be normalized a : The lowest value data b : The Highest value data Simalungun district population data is divided into 2 parts, the first data for 2010-2012 is used as training data, while 2013 data is used as a training target. The second data for 2012-2014 is used as testing data, while 2015 data is used as a target for testing data.

3. Results and Discussion This study uses 5 architectural models, including: 3-5-1 (3 are input layers, 5 are hidden layer neurons, and 1 is the output layer), 3-10-1 (3 are input layers, 10 are hidden layer neurons, and 1 is the output layer), 3-5-10-1 (3 is the input layer, 5 and 10 are hidden layer neurons, and 1 is the output layer), 3-5-15-1 (3 is the input layer, 5 and 15 are hidden layer neurons, and 1 are output layers) and 3- 10-15-1 (3 are input layers, 10 and 15 are hidden layer neurons, and 1 is the output layer). Parameters used are bipolar sigmoid activation function and linear function (tansig and purelin), gradient descent training function, epochs 150000, learning rate 0.01, and minimum error 0.001-0.01. Of these 5 architectural models, each learning will be carried out using the Matlab application. The results obtained are the best architecture with the 3-5-10-1 model. The results of the training can be seen in Figure 2.

The International Conference on Computer Science and Applied Mathematic IOP Publishing IOP Conf. Series: Journal of Physics: Conf. Series 1255 (2019) 012023 doi:10.1088/1742-6596/1255/1/012023

Figure 2. Data Training with Architectural models 3-5-10-1

From Figure 2 it can be explained that training with the 3-5-10-1 architectural model will produce an Epoch 5049 iterations with a duration of 11 seconds.

Based on Table 2 it is explained that the best architectural model of the 5 architectural models used is 3-5-10-1 with 97% accuracy and MSE Testing 0,00034833. The model is best because of the smaller MSE level and better efficiency compared to the other 4 models.

Table 2. Result of Backpropagation Method with Sigmoid Bipolar and Linear Function Training Testing No Architecture Epoch Time MSE Function MSE Accuracy 1 3-5-1 3216 00:22 0,00099992 Tansig, Purelin 0,00062093 78% 2 3-10-1 9640 01:04 0,00099929 Tansig, Purelin 0,00206334 56% 3 3-5-10-1 5049 00:11 0,00099975 Tansig, Purelin, Tansig 0,00034833 97% 4 3-5-15-1 1545 00:12 0,00100026 Tansig, Purelin, Tansig 0,00132363 75% 5 3-10-15-1 1085 00:09 0,00099877 Tansig, Purelin, Tansig 0,00097048 78%

Figure 3. Graph of Results of Backpropagation Method with Sigmoid Bipolar and Linear Functions

The International Conference on Computer Science and Applied Mathematic IOP Publishing IOP Conf. Series: Journal of Physics: Conf. Series 1255 (2019) 012023 doi:10.1088/1742-6596/1255/1/012023

Figure 3 is a graph of the results of using the backpropagation method with the activation function of sigmoid bipolar (tansig) and linear function (purelin). From the Figure, it can be seen that the highest Accuracy is in the 3-5-10-1 architectural model, the smallest epoch is in the 3-10-15-1 model and the fastest time is in the 3-5-10-1 model. Thus 3-5-10-1 architectural models proved the best.

Table 3. Comparison of Initial Data and Prediction Result Data Population Density (Soul/Km2) Prediction (Tansig+Purelin) No Districts 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 1 Bandar 582,00 598,00 651,00 654,41 669,14 677,20 693,47 634,38 596,02 448,56 295,68 2 Bandar Huluan 251,00 254,00 242,00 242,65 244,80 245,75 248,75 215,72 221,60 134,07 124,00 3 Bandar Masilam 249,00 250,00 269,00 270,00 271,08 271,91 281,93 238,65 251,79 168,60 139,16 4 Bosar Maligas 132,00 134,00 138,00 138,72 140,39 141,22 151,34 158,93 158,45 135,91 126,86 5 Dolok Batu Nanggar 312,00 315,00 373,00 373,29 377,01 378,73 425,99 372,98 408,81 372,94 254,34 6 Dolok Panribuan 116,00 117,00 122,00 121,89 122,82 123,20 133,97 145,33 142,40 136,71 127,74 7 Dolok Pardamean 161,00 161,00 156,00 156,06 156,80 157,03 165,46 166,18 167,85 128,55 124,33 8 Dolok Silou 48,00 48,00 46,00 46,30 46,96 47,30 44,56 65,01 19,03 74,43 85,28 9 Girsang Sipangan Bolon 116,00 118,00 112,00 112,11 113,40 114,02 124,70 138,88 133,34 137,59 127,98 10 Gunung Malela 300,00 306,00 346,00 347,03 353,13 356,35 393,87 346,07 375,51 339,04 235,49 11 Gunung Maligas 447,00 456,00 522,00 523,90 533,47 538,54 558,57 511,87 501,64 431,77 283,62 12 Haranggaol Horison 145,00 145,00 23,00 122,72 123,46 123,75 134,39 145,33 142,66 136,07 127,60 13 Hatonduhan 77,00 77,00 63,00 63,11 63,39 63,47 66,78 85,17 55,52 100,49 106,00 14 Hutabayu Raja 187,00 188,00 153,00 153,63 154,78 155,26 164,09 165,98 167,33 130,23 124,80 15 Jawa Maraja Bah Jambi 271,00 279,00 531,00 534,77 549,22 557,33 571,20 528,84 508,80 434,73 284,41 16 Jorlang Hataran 166,00 167,00 165,00 164,98 166,21 166,73 173,67 172,04 173,20 126,31 123,16 17 Kabupaten Simalungun 186,00 189,00 190,00 190,56 193,03 194,26 197,26 186,63 187,18 120,32 120,31 18 Pamatang Bandar 330,00 331,00 357,00 356,78 358,42 358,89 404,61 348,35 386,12 346,48 239,89 19 Pamatang Sidamanik 130,00 131,00 119,00 119,27 120,17 120,54 131,65 143,24 140,33 136,79 127,84 20 Pamatang Silimahuta 152,00 154,00 132,00 132,37 134,19 135,10 145,24 155,26 153,27 137,51 127,46 21 Panei 296,00 300,00 280,00 279,00 281,99 283,42 295,30 252,15 265,43 189,39 149,63 22 Panombeian Panei 233,00 235,00 262,00 262,19 263,85 264,49 272,13 231,60 242,22 156,93 133,59 23 Purba 127,00 131,00 131,00 131,86 135,33 137,27 146,82 159,73 155,69 141,26 128,07 24 Raya 92,00 93,00 95,00 94,82 96,04 96,65 105,85 123,39 111,43 132,71 125,69 25 Raya Kahean 77,00 77,00 86,00 85,78 86,46 86,75 94,90 112,47 96,93 125,59 122,23 26 Siantar 795,00 808,00 867,00 869,89 883,02 889,76 838,79 755,20 658,57 386,77 287,78 27 Sidamanik 324,00 326,00 337,00 337,62 340,16 341,21 379,12 325,72 358,34 314,50 221,51 28 Silimakuta 176,00 184,00 194,00 196,00 203,80 208,36 206,63 200,83 193,91 126,95 121,85 29 Silou Kahean 77,00 78,00 75,00 75,33 76,08 76,43 83,10 101,45 80,72 117,52 117,22 30 Tanah Jawa 218,00 219,00 269,00 269,65 271,68 272,52 282,03 239,84 251,79 169,80 139,67 31 Tapian Dolok 325,00 334,00 327,00 328,43 335,62 339,54 368,49 324,33 347,56 306,42 216,74 32 Ujung Padang 181,00 182,00 178,00 178,69 179,84 180,29 185,99 178,19 180,79 121,60 121,34

4. Conclusion The conclusion that can be drawn from this study is that the 3-5-10-1 architectural model is the best architectural model of the 5 models used to predict the population density in the Simalungun district of Indonesia using the Backpropagation method with the bipolar sigmoid activation function (tansig) and

The International Conference on Computer Science and Applied Mathematic IOP Publishing IOP Conf. Series: Journal of Physics: Conf. Series 1255 (2019) 012023 doi:10.1088/1742-6596/1255/1/012023 linear function (purelin). Accuracy obtained reached 97% or was the highest, and MSE was the lowest compared to other models, which was 0,00034833.

References [1] B. Febriadi, Z. Zamzami, Y. Yunefri, and A. Wanto, “Bipolar function in backpropagation algorithm in predicting Indonesia’s coal exports by major destination countries,” IOP Conference Series: Materials Science and Engineering, vol. 420, no. 12089, pp. 1–9, 2018. [2] N. Nasution, A. Zamsuri, L. Lisnawita, and A. Wanto, “Polak-Ribiere updates analysis with binary and linear function in determining coffee exports in Indonesia,” IOP Conference Series: Materials Science and Engineering, vol. 420, no. 12089, pp. 1–9, 2018. [3] A. Wanto, M. Zarlis, Sawaluddin, and D. Hartama, “Analysis of Artificial Neural Network Backpropagation Using Conjugate Gradient Fletcher Reeves in the Predicting Process,” Journal of Physics: Conference Series, vol. 930, no. 1, pp. 1–7, 2017. [4] Sumijan, A. P. Windarto, A. Muhammad, and Budiharjo, “Implementation of Neural Networks in Predicting the Understanding Level of Students Subject,” International Journal of Software Engineering and Its Applications, vol. 10, no. 10, pp. 189–204, 2016. [5] S. P. Siregar and A. Wanto, “Analysis of Artificial Neural Network Accuracy Using Backpropagation Algorithm In Predicting Process (Forecasting),” International Journal Of Information System & Technology, vol. 1, no. 1, pp. 34–42, 2017. [6] A. P. Windarto, L. S. Dewi, and D. Hartama, “Implementation of Artificial Intelligence in Predicting the Value of Indonesian Oil and Gas Exports With BP Algorithm,” International Journal of Recent Trends in Engineering & Research (IJRTER), vol. 3, no. 10, pp. 1–12, 2017. [7] D. Huang and Z. Wu, “Forecasting outpatient visits using empirical mode decomposition coupled with backpropagation artificial neural networks optimized by particle swarm optimization,” PLoS ONE, vol. 12, no. 2, pp. 1–17, 2017. [8] R. Hrasko, A. G. C. Pacheco, and R. A. Krohling, “Time Series Prediction Using Restricted Boltzmann Machines and Backpropagation,” Procedia Computer Science, vol. 55, pp. 990–999, 2015. [9] A. Wanto et al., “Levenberg-Marquardt Algorithm Combined with Bipolar Sigmoid Function to Measure Open Unemployment Rate in Indonesia,” 2018, pp. 1–7. [10] A. Wanto, A. P. Windarto, D. Hartama, and I. Parlina, “Use of Binary Sigmoid Function And Linear Identity In Artificial Neural Networks For Forecasting Population Density,” International Journal Of Information System & Technology, vol. 1, no. 1, pp. 43–54, 2017. [11] J. Adnan, N. Ghazali, N. Daud, M. T. Ishak, Z. Ismael, and M. I. A. R. Rahman, “Tansig Activation Function (of MLP Network) for Cardiac Abnormality Detection,” International Conference on Engineering and Technology (IntCET 2017), vol. 1930, no. 1, pp. 1–6, 2018. [12] M. Fauzan et al., “Epoch Analysis and Accuracy 3 ANN Algorithm Using Consumer Price Index Data in Indonesia,” 2018, pp. 1–7. [13] A. Wanto et al., “Analysis of Standard Gradient Descent with GD Momentum And Adaptive LR for SPR Prediction,” 2018, pp. 1–9.