2016 Joint International Conference on Artificial Intelligence and Computer Engineering (AICE 2016) and International Conference on Network and Communication Security (NCS 2016) ISBN: 978-1-60595-362-5
AdaBoost Artificial Neural Network for Stock Market Predicting Xiao-Ming BAI1,a, Cheng-Zhang WANG2,b,* 1Information School, Capital University of Economics and Business, Beijing, China 2School of Statistics and Mathematics, Central University of Finance and Economics, Beijing, China [email protected], [email protected] *Corresponding author
Keywords: AdaBoost, Artificial Neural Network, Stock Index Movement.
Abstract. In this work, we propose a new direction of stock index movement prediction algorithm, coined the Ada-ANN forecasting model, which exploits AdaBoost theory and ANN to fulfill the predicting task. ANNs are employed as the weak forecasting machines to construct one strong forecaster. Technical indicators from Chinese stock market and international stock markets such as S&P 500, NSADAQ, and DJIA are selected as the predicting independent variables for the period under investigation. Numerical results are compared and analyzed between strong forecasting machine and the weak one. Experimental results show that the Ada-ANN model works better than its rival for predicting direction of stock index movement.
Introduction Stock price index movement is a primary factor that investors have to consider during the process of financial decision making. Core of stock index movement prediction is to forecast the close price on the end point of the time period. Research scheme based on technical indicators analysis assumes that behavior of stock has the property of predictability on the basis of its performance in the past and all effective factors are reflected by the stock price. By analyzing technical indicators of previous stock price, one could obtain important information which could be explored to forecast the following stock price [1]. Stock market prediction was done on ISE by ANN in [2]. Prediction results of different classic approaches and ANN were examined and they found ANN was superior to other methods. ANN models were examined in prediction of the following day index direction in [3]. ANN model and linear regression model were explored to forecast emerging stock market in [4]. Financial indicators including the close price, the highest price, the lowest price, were selected as independent variables for ANN input to forecast the direction of ISE National-100 in [5]. Predicting performance of ANN and SVM were studied for predicting direction of stock price index movement on ISE in [6]. Ten technical indicators were selected and utilized for prediction. Study results showed that ANN was significantly better than SVM. ANN was utilized to forecast stock market index movement on ISE in [7]. Forecasting performance was assessed in different period of time. They confirmed that ANN got high percentage of correctly forecasted signs. ANN was integrated with metaheuristics to forecast stock price on Turkish Stock market in [8]. In the study, 45 technical indicators, such as the close price, MACD, the highest price, were selected as the input to ANN. It asserted that HS based ANN model did better than other models to be compared. ANN was adopted to forecast stock price index on Taiwan Stock Exchange in [9]. It was said that volatilities among the data was consistent with ARCH patterns. ANN mixed with ARCH could enhance price prediction. ANN and principal component analysis methods were examined in [10] for predicting the stock price on Tehran Stock Exchange. It claimed that principal component analysis could forecast the stock price accurately according to twenty accounting variables. The study showed that ANN model was superior to other methods. On the basis of accounting ratios, [11] compared neural network for forecasting stock returns on Canadian stock market with other two methods,
ordinary least squares and logistic regression techniques. Results indicated that neural networks outperformed the other ones. Volatility of stock price on Korea Stock Exchange was studied in [12]. ANN was combined with time series patterns to forecast the volatility of stock price. Results asserted the superiority of ANN over its rivals. The forecasting performance of ARIMA and ANN model were examined based on the stock data from New York Stock Exchange in [13]. Results revealed the superiority of ANN model over ARIMA model. In order to predict the maximum and minimum day stock prices of Brazilian power distribution companies, ANN was employed to do that in [14]. It revealed that ANN with one hidden layer and five hidden neurons achieved the best results. During the process of learning, the values of initial weights of an ANN model are usually set at random. Results of one ANN model for classifying or predicting may exhibit high volatility. Additionally, ANN is prone to local optimum during the training process which may result in bad or unsatisfactory classification or prediction results. While according to the theory of boosting [15], rules of thumb, or weak classifiers, can be combined to form highly accurate combined classifiers. And weak-learners can be employed to discover these simple rules which make the combined algorithm widely applicable. Inspired by the idea, we attempt to construct a boosting predictor in this paper for stock price forecasting. ANN models are employed as the weak predictors to form the combined strong one. We elaborately design the boosting rules and propose a boosting prediction algorithm based on ANN for stock price forecasting. Our work mainly focuses on constructing a strong predictor using multiple ANN models, which is different from most of previous works for stock market forecasting concentrating on optimizing parameters and architecture of one single ANN.
Research Methodology According to the theory of economics, the direction of the stock index movement is defined by the following formula:
Direction sign(PPEnd Point Start Point ) (1)
Where PEnd Point refers to the stock index price of the end point over the time period, PStart Point represents the stock index price of the start point over the time period. For daily direction prediction,
PEndPoint is today's stock index close price and PStart Point point is previous day's stock index close price. Target population of this research contains stock index prices information on Shanghai Stock Exchange over a period from Jan. 2011 to Mar. 2016. Stock prices information from international stock markets, such as S&P 500, NSADAQ, and DJIA, and foreign exchange rates to US dollar over the same period of time are also considered.
Technical Indicators 38 technical indicators of stock market are selected as the independent variables, see in Table 1, and are utilized as the input to our Ada-ANN model to forecast the direction of stock index movement.
Table 1. Technical indicators of stock market. 1: Today's open price 20: Slow stochastic %D 2: Previous open price 21: William's %R 3: Previous highest price 22: Bollinger middle band 4: Previous lowest price 23: Bollinger higher band 5: Previous close price 24: Bollinger lower band 6: Today's open price of S&P 500 25: 5-day simple moving average of close price 7: Previous open price of S&P 500 26: 5-day exponential moving average of close price 8: Previous close price of S&P 500 27: 5-day triangular moving average of close price 9: Exchange rates to US dollar 28: 6-day simple moving average of close price 10: Today's open price of NSADAQ 29: 6-day exponential moving average of close price 11: Previous open price of NSADAQ 30: 6-day triangular moving average of close price 12: Previous close price of NSADAQ 31: 10-day simple moving average of close price 13: Today's open price of DJIA 32: 10-day exponential moving average of close price
Table 1. Technical indicators of stock market (Cont.). 14: Previous open price of DJIA 33: 10-day triangular moving average of close price 15: Previous close price of DJIA 34: 20-day simple moving average of close price 16: Momentum close price 35: 20-day exponential moving average of close price 17: Fast stochastic %K 36: 20-day triangular moving average of close price 18: Fast stochastic %D 37: Close price moving average convergence/divergenc 19: Slow stochastic %K 38: Accumulation/distribution oscillator AdaBoost Theoretical framework of boosting for machine learning is PAC (probably approximately correct) learning model. In order to solve lots of practical difficulties encountered by the earlier boosting algorithms, AdaBoost algorithm was proposed on the basis of boosting in [16].
Suppose training set is Sxyxytrain (, 1 1 ),,(, mm ). Where (,xiiyi )(1,,) m represent training samples. xi X is the independent variable and X denotes the domain space. yYi is the response variable and Y represents the value domain. For binary classification problem, it is assumed that Y 1, 1 . Weak learning algorithm is called iteratively by AdaBoost for round T. While a distribution or set of weights over Strain should be maintained during the process. Let Dit ( ) denote the weight of the distribution on sample i at round tt(1,,) T . In the process of iteration, Dit ( ) will be increased at the following round for misclassified samples which forced the following weak learner to pay more attention to the hard samples.
During each iteration, the optimal "weak hypothesis" represented by hXYt : is acquired with respect to the distribution Dt . Error of the weak learning algorithm is formulated as:
Pr [hx ( ) y ] Di ( ) (2) tiDtii~ t ih:() x y t ti i
Having got the weak hypothesis, the final hypothesis H(x) can be obtained using the weighted majority vote strategies:
T Hx() sign( hx ()) t1 tt (3)
ANN Generally, ANN is a nonlinear system which is constructed by a set of interconnected neurons[17]. A multilayer perception (MLP) is a feedforward ANN which is formed by multiple layers of nodes in
a directed manner. Nodes of each layer in MLP are fully connected to those of the next following layer. Every node except for the ones of input layer has a nonlinear activation function. Error backpropagation strategy which belongs to supervised learning technique is utilized by MLP for learning process. Architecture of a MLP with one hidden layer is illustrated in Fig. 1.
Figure 1. Three-layer MLP architecture.
For each node except for the ones of input layer, let ui denotes the summation input of node i, then:
m uwxb iijiji j1 (4)
Where bi is the bias of the node i, wij is the weight of ANN. Let f represents the activation function of ANN which is usually a nonlinear function such as:
eex x fx() tanh() x (5) eex x
By the activation operation, the output of the node i is:
eeuuii ofuii( ) tanh( u i ) (6) eeuuii
Given training pattern (,,;)xiipi1 xy, suppose the output of ANN is yi . The error backpropagation technique updates the weights wij of ANN by optimizing the following error-energy function[34]:
1 :()yy2 (7) 2 i ii
Ada-ANN Forecasting Model In this study, we represent the increase and decrease of the stock index movement as 1 and 1 respectively. As ANN algorithm exhibits high performance on the task of stock market forecasting, it is employed as the weak learning algorithm in our AdaBoot forecasting model. The pseudocode of our Ada-ANN forecasting model is illustrated in Table 2.
Table 2. Ada-ANN algorithm.
Input: Training sample set Sxyxytrain (, 1 1 ),,(, mm ); Maximum iteration number: T; Hidden layer size of ANN: N; Train epoches of ANN: M T Output: The final hypothesis: Hx() sign( netx ()) t1 tt 1 1: Initialize weight of sample: Di() (1,,)im 1 m 2: For tT 1, , 3: For mM 1, ,
4: Initialize weights of ANN randomly: wij
5: Compute the error-energy function with respect to distribution Dt : 1 :()yy2 2 i ii 6: Do while ( )
7: Compute gradient of with respect to wij : : wij 8: Update weights of ANN: wwij ij wij 9: Compute the error-engergy function 10: EndDo 11: EndFor
12: Obtain the optimal weak hypothesis hxtt(): netx () and its error: Pr [net ( x ) y ] tiDtii~ t
11 t 13: Compute t ln( ) 2 t 14: Update weight of sample:
t Dit () enetxy,if()ti i ; Dit1() ( Zt is a normalization factor) t Zt e ,else. 15: EndFor
Results and discussions
Prices information related to the direction of stock index movement forecasting on Shanghai Stock Exchange and international stock markets is utilized to carry out experiments. We extract the data over the period of time from 01/10/2011 to 03/02/2016. Data on stock-market-close date is excluded. Data extracted from 01/10/2011 to 08/04/2015 is employed as the training set. The rest is used as the testing set. Sample size of the training set is 1100, and that of the testing set is 147. Descriptive statistics of the training and test datasets are illustrated in Table 3.
Table 3. Descriptive statistics of the training and test datasets. Dataset Increase Decrease Total Training 521 579 1100 Test 72 75 147 Sum 593 654 1247 The Ada-ANN forecasting model is proposed to predict the stock index movement. To each MLP, three groups of experiments are conducted with respect to the hidden layer size is set to 20, 25 and 30 respectively. During the process of learning, every MLP model will be trained for many times to find
the optimal weak hypothesis. The training epoches is set to 20. In AdaBoost algorithm, there are several weak learners. In this work, the number of weak learners is set to 10. To verify the performance of our forecasting model, prediction accuracy on three groups of hidden layer size are calculated and reported. At the same time, predicting results of single ANN model on the same test set for the three groups of experiments are also recorded. The single ANN forecasting model used in our work has the same architecture with that used as the weak learner in the Ada-ANN forecasting model. For each group of experiments, the single ANN forecasting model is tested and the results are utilized as the benchmark. Accuracy comparisons under the same conditions are summarized in Table 4.
Table 4. Accuracy comparison. Prediction Accuracy(%) Model Group 1 Group 2 Group 3 Ada-ANN 76.87 77.55 68.71 ANN 72.11 74.15 63.27 As the experimental results shown, our Ada-ANN forecasting model outperforms the single ANN predicting model in terms of the statistical accuracy criteria. In addition, these two forecasting models are all verified on the same test data set. Statistical results can be considered as the true indictors of the predicting performance. From the results of the three groups of experiments, one can see that the predicting performance of our Ada-ANN model is superior to that of the single ANN model for the direction of stock index movement forecasting task. Another point deserved to be noticed is that the architecture of ANN has impact on the performance of the predicting model constructed by it. Statistical results of the three groups of experiments show that both our Ada-ANN forecasting model and the single ANN one has got the best performance in group 2. That is to say, for three-layer ANN with 38 inputs and one output, the best result is obtained with the hidden layer size been set to 25. The best result of our forecasting model is 77.55%, while that of single ANN is 74.15%. In similar work, the best results are reported as 60.81% in [3], 57.80% in [4], 74.51% in [5] and 76.70% in [7]. Our forecasting model has got the best result.
Conclusions To assess the predictability of Chinese stock market, 38 technical indicators are extracted from Shanghai Stock Exchange and other international stock markets. A new forecasting model is proposed which employs MLP as the weak learners along with the idea of AdaBoost to form the strong predictor. To verify the effectiveness and performance of the proposed model on the task of predicting direction of stock index movement, experiments are conducted in this work. Statistical analyses indicate that the direction of stock index movement on Shanghai Stock Exchange can be predicted precisely. Comparative analyses of statistical experimental results show that the Ada-ANN forecasting model has superiority over its rivals. It can improve the accuracy effectively.
References [1] H. Mihaninia, Stock price prediction by using unobserved components model and random volatilities (M.S. Thesis), Facaulty of Engineering, University of Science and Culture, 2012. [2] B. Egeli, M. Ozturan, B. Badur, Stock market prediction using artificial neural networks, Proceedings of the 3rd Hawaii International Conference on Business, (2003) 1-8. [3] A.I. Diler, Predicting direction of ise national-100 index with back propagation trained neural network, Journal of Istanbul Stock Exchange, 7 (25-26)(2003): 65-81. [4] E. Altay, M.H. Satman, Stock market forecasting: Artificial neural networks and linear regression comparison in an emerging market, Journal of Financial Management and Analysis, 18 (2) (2005): 18-33. [5] B. Yildiz, A. Yalama, M. Coskun, Forecasting the istanbul stock exchange national 100 index using an artificial neural network, World Academy of Science, Engineering and Technology, 22 (2008): 36-39. [6] Y. Kara, M. Boyacioglu, O. Baykan, Predicting direction of stock price index movement using artificial neural networks and support vector machines: the sample of the istanbul stock exchange, Expert Systems with Applications, 38 (5) (2011): 5311-5319. [7] K. Karymshakov, Y. Abdykaparov, Forecasting stock index movement with artificial neural networks: the case of istanbul stock exchange, Trakya University Journal of Social Science, 14 (2) (2012): 231-24. [8] M. Gocken, M. Ozcalici, A. Ayse Tugba Dosdogruc, Integrating metaheuristics and artificial neural networks for improved stock price prediction, Expert Systems with Applications, 44 (2016): 320-331. [9] Y.-H. Wang, Nonlinear neural network forecasting model for stock index option price: Hybrid gjr-garch approach, Expert Systems with Applications, 36 (2009): 564-570. [10] J. Zahedi, M. Rounaghi, Application of artificial neural network models and principal component analysis method in predicting stock prices on Tehran stock exchange, Physica A, 438 (2015): 178-187. [11] D. Olson, C. Mossman, Neural network of canadian stock returns using accounting ratios, International Journal of Forecasting, 19 (2003): 453-65. [12] T. Roh, Forecasting the volatility of stock price index, Expert Systems with Applications, 33 (4) (2007): 916-922. [13] A. Adebiyi, A. Adewumi, C. Ayo, Comparison of arima and artificial neural networks models for stock price prediction, Journal of Applied Mathematics, 375 (2014): 1-7. [14] L. Laboissiere, R. Fernandes, G. Lage, Maximum and minimum stock price forecasting of brazilian power distribution companies based on artificial neural networks, Applied Soft Computing, 35 (2015): 66-74. [15] I. Mukherjee, R. Schapire, A theory of multiclass boosting, The Journal of Machine Learning Research, 14 (1) (2013): 437-497. [16] Y. Freund, R.E. Schapire, A decision-theoretic generalization of on-line learning and an application to boostings, Journal of Computer and System Sciences, 55 (1) (1997): 119-139. [17]D. Graupe, Principles of artificial neural networks, World Scientific, 2013.