Stock Market Prediction Software using Recurrent Neural Networks.
Senior Project Session 2004-2005
S
t
o
c
k
R
M
e
a
c
r
u
k
r
e
r
t
e
n P
t
r
e
N
d
e
i
c
u
t
r
i
a
o
l
n
N
S Submitted By
e
o
t
f
w
t
w
o
r
a
k
r
s e Asim Maqbool (2005-02-0028)
.
u
s Muhammad Umer Malik (2005-02-0108)
i
n
g
Mohsin Fazal (2005-02-0096) Muttee Ahmed (2005-02-0114)
May 12, 2005 Department of Computer Science Lahore University of Management Sciences CS-405 CS Senior Project
Stock Market Prediction Software - SMPS
Group 0514 Documentation A report submitted to the Department of Computer Science in
partial fulfillment of the requirements for the degree
BSc (Honors) in Computer Science to
Dr M.M. Awais (Primary Faculty Advisor)
Dr Asim Karim
Mr Tauqeer Hussain
Lahore University of Management Sciences
13th May, 2005 Acknowledgements
We truly acknowledge the cooperation and help provided by Dr. Asim Karim, Dr. M.M.
Awais and Mr. Tauqeer Hussain. They have been a constant source of guidance throughout the course of this project. We would also like to thank Bassam Ahmed Fawad and Ali Farid for their help. We are also thankful to our friends and families whose silent support led us to complete our project.
(Signed)
Asim Maqbool ( 2005-02-0028 ) ______
Muhammad Umer Malik ( 2005-02-0108 ) ______
Mohsin Fazal ( 2005-02-0096 ) ______
Muttee Ahmed ( 2005-02-0114 ) ______
Date
May 13th, 2005
iv Stock Price Predictions Using Recurrent Neural Network–- SMPS Table of Contents
ABSTRACT...... VI CHAPTER 1...... 1 INTRODUCTION...... 1
1.1 MOTIVATION...... 1 1.1.1 Failure of the ‘Efficient Market Hypothesis’...... 1 1.1.2 Time Series Analysis...... 3 1.1.3 Neural Networks...... 4 1.2 PROBLEM STATEMENT...... 7 1.2.1 Neural Network Model...... 8 1.2.2 De-noising technique...... 8 1.2.3 Financial Analysis...... 9 CHAPTER 2...... 10 BACKGROUND RESEARCH...... 10
1.0 WAAVELETS...... 10 2.1 SIGNAL ESTIMATION...... 10 2.1.1 Signal Estimation and denoising...... 10 2.1.2 Hard Thresholding...... 10 2.1.3 Diffrencing...... 11 2.1.4 Scaling Function...... 11 2.2 ARTIFICIAL NEURAL NETWORKS...... 11 2.2.1Neural Networks...... 11 2.2.2 Feed Forward Neural Networksr...... 11 2.2.3 Recurrent Neural Networks...... 12 2.3 WAVELET TRANSFORM...... 13 2.3.2 The Redundant Haar Wavelet Transform...... 14 2.4 DENOISING...... 15 2.4.1 Wden Fuction...... 16
CHAPTER 3...... 17 REQUIREMENTS...... 17
3.1 DATA REQUIREMENTS...... 17 3.1.1Product Percpective...... 17 3.1.1.1 System Interface...... 18 3.1.1.2 User Interface...... 18 3.1.1.3 Hardware Interface...... 18
3.2 BASIC INPUT DATA/SOLUTION METHODS...... 19 3.3 AUTHENTICATION...... 20 3.4 ACCURACY...... 20 3.5 TEST RUNS...... 20 v Stock Price Predictions Using Recurrent Neural Network–- SMPS CHAPTER 4...... 21 ANALYSIS TECHNIQUES...... 21
4.1 NORMALIZED MEAN SQUARE ERROR...... 26 4.2 TREND HIT RATE...... 27 CHAPTER 5...... 28 PROTOTYPE...... 28
5.1 ANALYSIS OF PROTOTYPE...... 28 5.1.1 Training Functions...... 28 5.1.2 Data Points Selection...... 29 5.1.3Network Parameters...... 30 5.1.4 Wavelet Transfrom...... 38 5.2 IMPLEMENTATION...... 38 5.2.1 System Interface...... 39 5.2.2 User Interface...... 39 5.2.3 Hardware Interface...... 41 5.2.4 Software Interface...... 41 5.3 FUNCTIONAL REQUIREMENTS...... 41 5.3.1 Data Flow Diagrams...... 41 5.3.2 Structure Chats...... 47
CHAPTER 6...... 50 RFIR NEURAL NETWORK RESULTS...... 50
6.1 OPTIMIZING THE NEURAL NETWORK...... 50 6.1.1 Optimizing the Time Steps...... 51 6.2 SIMULATION RESULTS...... 52 6.2.1 Pakistan State Oil (PSO)...... 52 6.2.2 KESC...... 53 6.2.3 ICI Pakistan...... 53 6.2.4 Glaxo Smithkline...... 53 6.2.5 Lakson Tobacco...... 54 6.2.6 Colgate...... 54 6.2.7 Engro Chemicals and Fertilizers...... 54 6.2.8 Shell...... 54 CHAPTER 7...... 56 CONCLUSION OF RESEARCH...... 56 CHAPTER 8...... 57 FUTURE WORK...... 57
8.1 PREDICTIONS USING THE LIVE DATA...... 57 8.2 REPORT GENERATION AND PORTFOLIO ANALYSIS...... 57 8.3 CURRENCY EXCHANGE RATE PREDICTION...... 58 REFERENCES...... 59
vi Stock Price Predictions Using Recurrent Neural Network–- SMPS Abstract
Efficient Market Hypothesis is the popular theory about time series prediction, with its failure much research has been carried out in the area of neural networks being used to predict future time series data. As part of our project we have implemented using recurrent neural network. The network was tested using data of companies being traded on the Lahore Stock Exchange, over the year 2003. The Stock Market Prediction Software (SMPS) intends to predict stock market by using various Artificial Intelligence techniques. It does it by processing the past stock data and hence anticipating the future market curve and share prices respectively.
Various techniques were considered for this purpose like neural networks and statistical learning. We have used Multi Layer Perceptron (MLP) is used with back-propagation algorithm and sigmoid activation function at each layer to train and predict stock market.
This software is also unique in the sense that according to our knowledge Lahore Stock Exchange (LSE) as well as Karachi Stock Exchange (KSE) both do not use this kind of software commercially. Thus SMPS stands a good chance of getting into the potential stock markets which can immensely change the way investors invest in the stocks. We have obtained the past year data for stock market on the stock market websites www.brain.lse.com.pk This data is stored in excel files and is the imported in SMPS to be de-noised using MATLAB and then used to train MLP. Thus SMPS is commercially viable, and challenging to implement.
vii Stock Price Predictions Using Recurrent Neural Network–- SMPS List Of Figures
FIGURE 4.1 ...... 24 4.2 ...... 25
FIGURE ...... 30 5.1 ...... 31 5.2 ...... 32 5.3 I ...... 33 5.4 ...... 34 5.5 ...... 35 5.6 ...... 36 5.7 ...... 36 5.8 ...... 40 5.9 ...... 41 5.10 ...... 42 5.11 ...... 43 5.12 ...... 44 5.13 I ...... 45 5.14 ...... 46 5.15 ...... 47 5.16 ...... 47 5.17 ...... 48 5.18 ...... 48 5.19 ...... 49 5.20 ...... 49
viii Stock Price Predictions Using Recurrent Neural Network–- SMPS Chapter 1 Introduction
1.1 Motivation
1.1.1 Failure of the ‘Efficient Market Hypothesis’
The movement of prices in the stock market are among a few phenomena that have cut
across the boundaries of academic disciplines and have a cumulative research evidence
spanning almost a century. Today the field of financial market research seems to be at the
exciting stage of "crisis" (to use Kuhn's analogy) - past results are being questioned, and
new solutions are being proposed. There seems to be growing dissatisfaction among
academic researchers with the body of literature developed on the assumption of market
efficiency. Keynesian ideologies on speculative market phenomena (hitherto ignored) are
being resurrected to explain the volatile nature of the stock market. While only time will tell
whether or not the present crisis will lead to a revolution in thought and development of a
coherent theory of stock market behavior, it seems appropriate at this point to take stock of
the current literature in the area of market efficiency and to identify the emerging lines of
thought. Much of the financial prediction that has taken place in the past was based on The
Efficient Market Hypothesis (EMH). EMH states that the current price of a stock reflects
assimilation of all the information available. Thus, if the information is given no future
changes in the price of the stock can be predicted. However, in recent years this idea has
been widely disproved and analysts have suggested that the Time Series models better
predict the price of the stock as compared to the Random Walk model. The forecasting of
1 Stock Price Predictions Using Recurrent Neural Network–- SMPS financial time series relies on the discovery of strong empirical regularities in observations
of the system. Because these regularities are often masked by noise and the time series often
have nonlinear and non-stationary behavior it has been suggested that some financial time
series are not predictable. To determine whether a time series is predictable we must
examine the random walk hypothesis.
The domain of financial time series prediction is a highly complicated task due to following
reasons:
1. Financial time series often behave nearly like a random-walk process, rendering
(from a theoretical point of view) the prediction impossible. The predictability of most
common financial time series (stock prices, levels of indices) is a controversial issue and has
been questioned in scope of the efficient market hypothesis (EMH).
2. Financial time series are subject to regime shifting, i.e. statistical properties of the
time series are different at different points in time (the process is time-varying).
3. Financial time series are usually very noisy, i.e. there is a large amount of random
(unpredictable) day-to-day variations.
4. In the long run, a new prediction technique becomes a part of the process to be
predicted, i.e. it influences the process to be predicted.
The estimated time series model, both linear & non-linear, have smaller out-of-sample
forecast errors than the Random Walk Model. Applied to daily rates on the financial market
the random walk hypothesis (RWH) states that a financial time series is defined by constant
expected price changes between successive trading days and zero correlation between the
price changes for any pair of different trading days. The RWH means that the time series is
2 Stock Price Predictions Using Recurrent Neural Network–- SMPS Markov and the best forecast of tomorrow’s price requires today’s price but not previous
prices. If this is a stochastic process i.e. time series is called Markov if the probability of a
transition from one state to another depends only on the current state is actually true, it is not
necessary to spend more resources to develop a good forecasting system. All prediction
systems based on the previous behavior of the time series are then useless because the best
prediction of tomorrow’s price is today’s price plus some constant price change. However
the RWH does not appear valid for many financial time series. Using different trend
statistics and tests, experts have concluded that several financial time series have non-
random behavior. The financial market is not completely efficient and correlation and trends
can sometimes be found within the data.
1.1.2 Time Series Analysis
An ordered sequence of values of a variable at equally spaced time intervals. Time series
analysis accounts for the fact that data points taken over time may have an internal structure
(such as autocorrelation, trend or seasonal variation) that should be accounted for. A time
series is stationary if it has constant mean and variance; otherwise, it is non-stationary. Non-
stationarity is hard to model, as its future behavior may be unpredictable. Stock prices are
characterized as a non-stationary ‘noisy’ time series. In order to reduce this noise, various
techniques can be employed which filter the time series.
Thus, all contemporary research being carried out in stock predictions uses the Time series
model. Time series predictions have been used in many areas of science, industry and
3 Stock Price Predictions Using Recurrent Neural Network–- SMPS commercial and financial activities, such as financial forecasts on stock prices and currency
exchange rates, product sale and demand forecasting, population growth, and earthquake
activities. Time series forecasting, or time series prediction, takes an existing series of data
x ,, x , x , x x , x , tn t2 t1 t and forecasts the t1 t2 data values. The goal is to observe or
model the existing data series to enable future unknown data values to be forecasted
accurately. Examples of data series include financial data series (stocks, indices, rates, etc.),
physically observed data series (sunspots, weather, etc.), and mathematical data series
(Fibonacci sequence, integrals of differential equations, etc.). The phrase “time series”
generically refers to any data series, whether or not the data are dependent on a certain time
increment. Neural networks are an artificial intelligence method for modeling complex
target functions. During the last decade they have been widely applied to the domain of
financial time series prediction and their importance in this field is growing. The present
work aims at serving as an introduction to the domain of financial time series prediction,
emphasizing the issues particularly important with respect to the neural network approach to
this task.
1.1.3 Neural Networks
Neural networks have a large appeal to many researchers due to their great closeness to the structure of the brain, a characteristic not shared by more traditional systems.
In an analogy to the brain, an entity made up of interconnected neurons, neural networks are made up of interconnected processing elements called units, which respond in parallel to a set of input signals given to each. The unit is the equivalent of its brain counterpart, the neuron.
A neural network consists of four main parts: 4 Stock Price Predictions Using Recurrent Neural Network–- SMPS 1. Processing units{uj}, where each uj has a certain activation level aj(t) at any point in time.
2. Weighted interconnections between the various processing units which determine how the activation of one unit leads to input for another unit.
3. An activation rule which acts on the set of input signals at a unit to produce a new output signal, or activation.
4. Optionally, a learning rule that specifies how to adjust the weights for a given input/output pair.
A processing unit uj takes a number of input signals, say a1j, a2j,...,anj with corresponding weights w1j, w2j,...,wnj, respectively. The net input to uj given by: netj = SUM (wij * aij)
The new state of activation of uj given by: aj(t+1) = F(aj(t),netj), where F is the activation rule and aj(t) is the activation of uj at time t. The output signal oj of unit uj is a function of the new state of activation of uj: oj(t+1) = fj(aj(t+1)).
One of the most important features of a neural network is its ability to adapt to new environments.
Therefore, learning algorithms are critical to the study of neural networks.1
The human brain, for example, is estimated to contain a densely interconnected network of approximately 1011 neurons, each connected, on average, to 104 others. Neuron activity is typically excited or inhibited through connections to other neurons. The fastest neuron switching times are known to be on the order of 10¡3 seconds, quite slow compared to computer switching speeds of
10¡10 seconds. Yet humans are able to make surprisingly complex decisions, surprisingly quickly.
For example, it requires approximately 10¡1 seconds to visually recognize one’s mother. Notice the
1 Ingrid Russell of the University of Hartford , Neural Networks Module 5 Stock Price Predictions Using Recurrent Neural Network–- SMPS sequence of neuron firings that can take place during this 10¡1-second interval cannot possibly be longer than a few hundred steps, given that the information-processing abilities of biological neural systems must follow from highly parallel processes operating on representations that are distributed over many neurons. One motivation for ANN systems is to capture this kind of highly parallel computation based on distributed representations.
Most ANN software runs on sequential machines emulating distributed processes, although faster versions of the algorithms have also been implemented on highly parallel machines and on specialized hardware designed specifically for ANN applications.
The most frequent application domains of ANN are productions/operations (53.5%) and finance
(25.4 %). Approximately 95 % of reported neural network business application studies utilize multilayered feed-forward neural networks with the back-propagation learning rule. In most analyzed applications, the NN results outperform statistical methods, such as multiple linear regression analysis, discriminative analysis and others.
The business world is becoming increasingly dependent upon intelligent techniques like neural networks to solve a variety of problems, new research problems are emerging.
Researchers are now devising techniques for extracting rules from neural networks, and combining neural networks with other intelligent techniques like genetic algorithms, fuzzy logic and expert systems. As more complex business problems are tackled, more research challenges are created.2
2 Neural Networks For Financial Time Series Prediction: Overview Over Recent Research” by Dimitri Pisarrenko. 6 Stock Price Predictions Using Recurrent Neural Network–- SMPS 1.2 Problem Statement
The primary purpose of this document is to accurately state and describe the requirements of software that predicts the fiscal curve of the stock market. Previously it was thought that it was not possible to predict the market trends, this was based on the ‘Efficient Market Hypothesis (EMH)’.
According to (EMH), it is not possible to predict the changes in the price curves for the stock market based on the available data. The primary purpose of this project would be to test the hypothesis of EMH and construct a model to predict the market curves for financial time series data like the stock prices, in Pakistan Stock Market using Statistical Analysis with the help of available data. Artificial Neural Networks are used for predicting this change, a special type of neural net called Recurrent Neural Networks. The stock market predictor’s primary aim would be to help investors to get an idea of whether a certain share would be going up (+ve) of down (-ve). By developing this software the following objectives are hoped to be achieved:
Avoiding unnecessary paperwork since everything would be catered by the computer.
Saving time and effort since the computer would be able to predict the increase or decrease in the market price using available data.
Achieve efficiency by making it easier for the user to be able to see the stock trend in order to be able to make an educated decision while investing.
Compare different models for noise removing from the time series and choose the optimum one.
Use different strategies for implementing Recurrent Neural Network such as Elman, Jordan or self looping at the hidden layer.
Implement the RNN’s our self to analyze it against the MATLAB neural network implementation.
7 Stock Price Predictions Using Recurrent Neural Network–- SMPS This document high-light all the requirements of the users as understood by the developers and is to be treated as the starting point of the design phase. It would be hard to change the system later on so we would recommend that before signing this document it should be carefully read and it must be ensured that these requirements were correct. In case of a conflict we should be immediately informed of the conflict so that we can alter our design accordingly. The efficient market hypothesis, Malkiel explained, refers to the notion that the prices of stocks rapidly adjust to any new information that relates to the stocks’ prospective returns. If indeed stock prices reflect all currently available information, then future price changes will reflect only future news, which is by definition unpredictable. The consequence, then, is that future stock prices should be unpredictable.
In recent years, however, a growing number of critics have argued that the efficient market hypothesis does not reflect the reality of stock price behavior. Many of these theories are based on the notion that investors do not react rationally to information, but are instead influenced by fads and other psychological phenemona. All of the theories, however, draw their motivation from apparent “anomalies” in stock price behavior that imply predictability.
1.2.1 Neural Network Model
To analyze the effectiveness of using appropriate neural network model (RFIR) and to
optimize it in terms of the various factors associated with it (training function, learning rate,
window size etc.).
1.2.2 De-noising technique
8 Stock Price Predictions Using Recurrent Neural Network–- SMPS Use an appropriate wavelet transform and measure its efficacy as opposed to the
‘differencing’ technique as it had been implemented by Ali Farid, in his paper.3 In the past
differencing was used as the technique for de-noising in a lot of the literature, but due to
problems such as accumulation of error associated with it, the shift has been towards using
‘wavelet transforms’, and we have used the built in function in MATLAB ( wden ) do the
same.
1.2.3 Financial Analysis
One of the primary objectives of this project is to also provide financial analysts with a tool
with which they can predict movements of stocks in a certain direction. The ability of the
network to keep updating it with historical information will allow it to predict the data even
more accurately and thus aim to increase the profitability for the investors and the users of
this application. Some aspects that we aim at achieving is this regard are to:
To aid the users of the software to make valuable decisions.
Help the investors to generate and maximize their profits out of the investments they
make in the stock market.
Reduce the risk involved when investing in stocks.
Support the users in an efficient portfolio management.
3 Farid, Ali. 2003, “Modeling and Forecasting Financial Time Series, a comparison of forecasting ability of neural networks, genetic programming and econometric methods”
9 Stock Price Predictions Using Recurrent Neural Network–- SMPS Chapter 2
Background Research
During the course of exploring this broad area of research, we came across certain areas in which we had to examine in detail to gain a detailed understanding of the project. The areas that we focused on are explained below:
1.0 Wavelets:
A function that results in a set of high frequency differences, or wavelet coefficients, is called a wavelet. Wavelets are used to decompose financial time series into bands in such away that each band contains a set of near stationary signals and that the low frequency band is smooth enough to be converted to stationary series.
2.0 Signal Estimation:
2.1.1 Signal Estimation and de-noising:
This is also referred to as "de-noising". Signal estimation algorithms attempt to characterize portions of the time series and remove those that fall into a particular model of noise.
2.1.2 Hard-Thresholding :
We assess each wavelet coefficient, at each resolution level, to see if it is "signal" or "noise". If the latter, we set it to zero. Then we apply the inverse wavelet transform, recreating our noise-filtered result4
4 http://strule.cs.qub.ac.uk/~gzheng/financial-engineering/ 10 Stock Price Predictions Using Recurrent Neural Network–- SMPS 2.1.3 Differencing:
Differencing substitutes each value by the differences between that value and the preceding value in the time series.5
2.1.4 Scaling Function:
The scaling function produces a smoother version of the data set, which is half the size of the input data set. Wavelet algorithms are recursive and the smoothed data becomes the input for the next step of the wavelet transform.
2.2 Artificial Neural Networks
It is well-known that ANNs are universal function approximators and do not require knowledge on the process under consideration. ANNs for modeling time series generally have special structures that store temporal information either explicitly using time-delayed structures r implicitly using feedback structures. Examples of the first class include time-delayed neural networks (TDNN) and FIR neural networks (FIR-NN), whereas examples of the latter include recurrent neural networks (RNN). Other architectures, such as radial-basis function network (RBF) and supporting vector machines, store approximate history information in either radial-basis functions or the so-called supporting vectors.
2.2.1 Neural Networks :
A Neural Network can be viewed as consisting of four components:
5 Data Mining in Finance. Numerical Data Mining Models and Financial Applications. Pg23. 11 Stock Price Predictions Using Recurrent Neural Network–- SMPS 2.2.2 Feed Forward Neural Networks
This is a network that has one or more inputs that are propagated through a variable number of hidden layers, each layer containing a variable number of nodes, which final reaches the output layer, containing one or more output nodes. Feed-forward neural networks can be used for many tasks, including classification and prediction.
One standard implementation for feed-forward networks is that the input values are taken, and each input value is fed into every node in the following layer. Within each node, the individual input values are multiplied by a specific weight (which depends upon which node the value came from, and which node it is in). These modified inputs are summed, and an optional bias value is added to the sum. This is then the output value from that specific node. Then, the output value from each node in that layer becomes the inputs for the next layer. This procedure is repeated until the output(s) from the output layer is obtained. Since the values are fed forward through each layer, the term feed-forward network was coined. The value of any one node can be expressed as:
t(x)=s( w0+Swihi(x) ) where s( ) represents a sigmoid (activation) function, wi represents the weight of the neuron from the ith input where w0 is the bias term and the hi(x) are the inputs from each of the preceding neurons. 6
2.2.3 Recurrent Neural Networks
Recurrent neural networks are available in two categories, Elman and Hopfield. The Hopfield network is used to store one or more stable target vectors. These stable vectors can be viewed as memories that the network recalls when provided with similar vectors that act as a cue to the network memory.
6 http://www.physics.carleton.ca/courses/75.502/slides/projects/1999/bbates/ 12 Stock Price Predictions Using Recurrent Neural Network–- SMPS Hopfield networks are interesting from a theoretical standpoint, but are seldom used in practice. Even the best Hopfield designs may have spurious stable points that lead to incorrect answers. For the purposes of our project we decided to use the Elman network to implement the recurrent structure of the network.
The Elman network commonly is a two-layer network with feedback from the first-layer output to the first layer input. This recurrent connection allows the Elman network to both detect and generate time-varying patterns. The Elman network has ‘tansig’ neurons in its hidden (recurrent) layer, and ‘purelin’ neurons in its output layer. This combination is special in that two-layer networks with these transfer functions can approximate any function (with a finite number of discontinuities) with arbitrary accuracy. The only requirement is that the hidden layer must have enough neurons. More hidden neurons are needed as the function being fit increases in complexity. The Elman network differs from conventional two-layer networks in that the first layer has a recurrent connection. The delay in this connection stores values from the previous time step, which can be used in the current time step. Thus, even if two Elman networks, with the same weights and biases, are given identical inputs at a given time step, their outputs can be different due to different feedback states. Because the network can store information for future reference, it is able to learn temporal patterns as well as spatial patterns. The Elman network can be trained to respond to, and to generate, both kinds of patterns. 7
2.3 Wavelet Transform
According to the definition, wavelet is a waveform that is bounded in both frequency and duration. Wavelet transform is a process that converts a signal into a series of wavelets.
It can be looked upon as an alternative to the more traditional way of analyzing waveforms called the Fourier Transforms.8 However, the biggest disadvantage of Fourier Transform is that it has only
7 http://www.mathworks.com/access/helpdesk/help/toolbox/nnet/nnet.shtml 8 http://www.realdictionary.com/computer/Computer/wavelet.asp
13 Stock Price Predictions Using Recurrent Neural Network–- SMPS frequency resolution and no time resolution. This means that although we might be able to determine all the frequencies present in a signal, we do not know when they are present. To overcome this problem, wavelet transform has been developed which is able to represent a signal in the time and frequency domain at the same time. The idea behind this time-frequency joint representation is to cut the signal of interest into several parts and then analyze the parts separately. The wavelet transforms uses a fully scalable modulated window which is shifted along the signal and for every position the spectrum is calculated. Then this process is repeated many times with a slightly shorter (or longer) window for every new cycle. In the end the result will be a collection of time-frequency representations of the signal, all with different resolutions. Because of this collection of representations we can speak of a multi-resolution analysis.9 While doing the research, we came across two wavelet transforms which we feel are worth mentioning.
2.3.1 The Redundant Haar Wavelet Transform
It is the same as the àtrous but with a different pair of scaling and wavelet functions. The wavelet function used is asymmetric which makes it a good choice for edge detection, i.e. localized jumps. It uses a simple low pass filter h, (1/2,1/2) as opposed to the low-pass filter h, (1/16,1/4,3/8,1/4,1/16) used in àtrous algorithm. Consider the creation of the first wavelet resolution level. We have created from by convolving the latter with h. Then:
and
More generally,
9 http://perso.wanadoo.fr/polyvalens/clemens/wavelets/wavelets.html 14 Stock Price Predictions Using Recurrent Neural Network–- SMPS At any time point, k, we never use information after k in calculating the wavelet coefficient.
Haar discovered the wavelets which are called after him around 1909, before wavelets where discovered. He was studying unconditional bases for classical function spaces such as the Lebesgue spaces; his construction actually gives birth to a complete (orthonormal, where this has sense) system.
The simplicity of this construction makes this one of the favourite examples: it is relatively easy to explain and easy to understand. The completeness of this system in some spaces is also "naked-eye visible".
Recently Haar wavelets have been used to explore the possibility of constructing wavelets on general domains (manifolds) such as, to give an example, on spheres. This is because they can be manipulated in order to satisfy particular geometric conditions. In the same way, Haar wavelets have been constructed on spaces with measures which are not translation-invariant
2.4 De-Noising:
Use an appropriate wavelet transform and measure its efficacy as opposed to the ‘differencing’ technique as it had been implemented by Ali Farid, in his paper.10 In the past differencing was used as the technique for de-noising in a lot of the literature, but due to problems such as accumulation of
10 Farid, Ali. 2003, “Modeling and Forecasting Financial Time Series, a comparison of forecasting ability of neural networks, genetic programming and econometric methods”
15 Stock Price Predictions Using Recurrent Neural Network–- SMPS error associated with it, the shift has been towards using ‘wavelet transforms’, and we have used the built in function in MATLAB ( wden ) do the same.
2.4.1 WAVE-Den (Wden) Function:
WDEN performs an automatic de-noising process of a 1-D signal using wavelets.
[XD,CXD,LXD] = WDEN(X,TPTR,SORH,SCAL,N,'wname') returns a de-noised version XD of input signal X obtained by thresholding the wavelet coefficients. Additional output arguments [CXD,LXD] are the wavelet decomposition structure of de-noised signal XD.
TPTR string contains threshold selection rule: 'rigrsure' use principle of Stein's Unbiased Risk. 'heursure' is an heuristic variant of the first option. 'sqtwolog' for universal threshold sqrt(2*log(.)). 'minimaxi' for minimax thresholding. SORH ('s' or 'h') is for soft or hard thresholding SCAL defines multiplicative threshold rescaling: 'one' for no rescaling. 'sln' for rescaling using a single estimation of level noise based on first level coefficients. 'mln' for rescaling done using level dependent estimation of level noise.
Wavelet decomposition is performed at level N and 'wname' is a string containing the name of desired orthogonal wavelet.
[XD,CXD,LXD] = WDEN(C,L,TPTR,SORH,SCAL,N,'wname') returns same output arguments, using the same options as above, but obtained directly from the input wavelet decomposition structure [C,L] of the signal to be de-noised, at level N and using 'wname' orthogonal wavelet.
16 Stock Price Predictions Using Recurrent Neural Network–- SMPS Chapter 3 Requirements
This section gives an overview of the rest of his document. Section 2 of this document would provide the user with the general overview of the system. It tries to capture all the requirements of the user. The requirements of the user have been identified by drawing Data Flow Diagrams. We have tried our best to develop exactly what the user wants. Moreover, for the ease of the user a data dictionary has been included. The data dictionary includes items such as the data elements and data structures which will be used in the developing of the system. The user can refer to the dictionary if he finds any difficulty in understanding the use of any data which we have used.
To detect the changes in the stock prices and to make the network learn the trends in order to predict the future prices with as much accuracy as possible, long periods (about one year) of reliable stock price data have to be available. A relatively long period of impact monitoring is also required if accurate deductions, on the impacts, are to be made. Research is needed in order to analyze the records and to make projections into the future and on the details of the response mechanisms which will be required in order to adapt to and mitigate price change.
3.1 Data Requirements
3.1.1 Product perspective This is a specialized product which predicts the stock prices using a special kind of Artificial Neural Network (ANN), which at the moment is a major challenge for the stock investors. This is a different kind of product from those present in the market because it uses “Recurrent Neural Network” for predicting the future trends of the stock market and mainly “Haar” wavelet
17 Stock Price Predictions Using Recurrent Neural Network–- SMPS transformation to make the data stationary and some more filtering techniques for removing the noise from the signals. This software is independent and totally self-contained. However a completely functional operating system is required. More details on the software Interfaces have been provided in section 2.1.3.
3.1.1.1 System interfaces This system will take the input data from the stock exchange website and apply different de- noising techniques on the data to make it a stationary one. After this step the de noised time series is feed to the Recurrent Neural Networks, which is used to predict tend of the selected company.
3.1.1.2 User interfaces The user will be able to choose the company for which he wants to predict the trend also he will be able to choose which kind of technique (Elman or Jordan) for the Recurrent Neural Networks he wants to choose. The system will be able to generate the graphs using the predicted trends.
3.1.1.3 Hardware Interfaces The system will be able to work with the minimum requirements of the system, but for better performance following things can be used:
According to the requirement of the software, the data first had to be assembled and carefully checked before it could be used as an input. In many cases, some of the desired data will not be available and we have had to make certain assumptions regarding the data.
During the data requirement phase, we went through three types of things; model input data, calibration data, and verification data. Input data simply consists of the required parameters to run the model, and it typically includes information regarding the closing stock prices of everyday, for each company. Calibration is the process of parameter adjustments to obtain a match between predicted and the actual output. Where as, in verification, we hold the parameters constant and test
18 Stock Price Predictions Using Recurrent Neural Network–- SMPS the calibration on an independent data set. One can say that, calibration is used to estimate the value of these parameters, and verification is used to test the validity of the estimate. While going through
2nd and 3rd data phase, we used certain wavelet transformations on the original data set, and in order to do this, we used an Add-In for Microsoft Excel, which carries out all the transformation calculations.
Data sets of such types are usually not available, and in our case, we had to manipulate some data sets in order to bring them in the required format, as to calibrate and verify.
3.2 Basic Input Data/Solution Methods
The software requires an input of some form of fundamental data, which comprises of the closing prices (of each day) of all the listed companies, with absolutely no fluctuations. The data which we used was from the Lahore Stock Exchange, of the Year 2002, ranging from 1st of January to the 31st of December. Regarding the data with fluctuations, we have followed some techniques which give us an optimal solution dealing with the unsteady and varied flow.
The data that we had was in the format of a text file (.txt). It was like the raw data. It had a vast variety of information, only some of which was required by us. What we basically required was the everyday closing prices of all the companies, i.e. the listed companies. In order to bring the data in the required format, we made a small program in Microsoft Access, which incorporated all the data in a database, and then certain queries were run on the data, which added, deleted, and appended certain information and finally gave us our required output in the form an Excel file (.xls).
19 Stock Price Predictions Using Recurrent Neural Network–- SMPS Up to this point, one target was achieved, the target being that all the required data that was needed should be in one place, in one same format. But still, after running many queries on the raw data, in order to filter it accordingly, we were still left with an enormous amount of data. So in order to tackle this problem, the data was divided between different excel worksheets, and data filters were applied in order to extract the data format required as input by the Wavelet Add-In.
3.3 Authentication
We varied certain parameters of the data, and also the network, in order to check the difference in the output data. Many simulations were run in order to test the accuracy of the system, and thus many conclusions were reached, as to what values to assign to each of the parameters. Verification of the model involved further confirmation, after the process of calibration had been completed, of the model’s ability to reproduce accurate data, or one may say the ‘Trend’ in the data.
3.4 Accuracy
We have used various methods which test the accuracy of the results. While testing the accuracy, our primary concern is the MSE (Mean Square Error). On the assumption that a given computation scheme is stable, consideration of the accuracy of the scheme involves assessment of the
“correctness” of the results yielded by the scheme – that is, of the extent to which the calculated values are in agreement with the “true” physical values.
3.5 Test Runs
We went through this phase in order to check the reliability of our system as far as the predictions were concerned. In this part, after applying the calibration and verification processes, we looked at 20 Stock Price Predictions Using Recurrent Neural Network–- SMPS the outputs from the test runs. During this phase, just as earlier, all model parameters and results were double-checked for reasonableness.
Chapter 4 4.1 Analysis Techniques
The basic principle of our project is the decomposition of the financial time series into scale related components. The forecast at each such scale is then combined to get the desired prediction result. In order to do that, first we will make the time series stationary. Wavelets allow a time series to be viewed in multiple resolutions. Each resolution reflects a different frequency. The wavelet technique takes averages and differences of a signal, breaking the signal down into spectrum11. There are a wide variety of popular wavelet algorithms, including Daubechies wavelets, Mexican Hat wavelets and Morlet wavelets. These wavelet algorithms have the advantage of better resolution for smoothly changing time series. But they have the disadvantage of being more expensive to calculate than the Haar wavelets.
The Haar wavelet transform has a number of advantages:
It is conceptually simple. It is fast. It is memory efficient, since it can be calculated in place without a temporary array.
11 http://www.bearcave.com/misl/misl_tech/wavelets/haar.html#Introduction 21 Stock Price Predictions Using Recurrent Neural Network–- SMPS It is exactly reversible without the edge effects that are a problem with other wavelet transforms.
The higher resolution provided by these wavelets is not worth the cost for financial time series, which are characterized by jagged transitions. We will be using the ‘Non-Decimated Haar Wavelet Transform’ which decomposes the original time series into several components in multiple scales. Since wavelets are able to represent a finite time sequence in multiple scales and wavelet transforms is a non-parametric method wavelet analysis of time series has proven to be very useful.
If there is noise in our data, then it is reasonable to suppose that removing this will lead to better quality predictions. We will use soft-thresholding in comparison to hard-thresholding for noise removal. WDEN function in MATLAB which does automatic 1-D de-noising using wavelets is used for denoising. WDEN performs an automatic de-noising process of a 1-D signal using wavelets .Auto Correlation function of MATLAB , ‘autocorr’, to check for the stationarity and filtering of data by checking if the values obtained henceforth converge to zero or not Once this preprocessing is done then we input this data to our prediction model. For this several models can be analyzed. Auto regressive Integrated moving average model (ARIMA) which includes auto- regression, differencing to eliminate the integration of series, and moving averages. Since the model-building procedure is not a formal computer algorithm as it requires user’s decision at various level this is not preferred here. Instance based learning (IBL) and Case based learning (CBL) is the other two methods. While IBL uses set of numeric attributes of a fixed size the latter uses general objects. We have used ‘Recurrent Neural Network ‘ a form of Artificial Neural Networks (ANN) as our predictive model.
Neural network which allows components to move from input to output have hidden layers/nodes in between. If a node is interpreted as a neuron the variable t, which is a biologically motivated threshold represents the minimum total weighted input needed to fire neuron. Such a function, called activation function, is represented as
St (x) = { 1 if, x>=t; 0, if x 22 Stock Price Predictions Using Recurrent Neural Network–- SMPS Thus a neural network can be seen as tuple of the set of input links, output links and current activation level. A RNN can be seen as a feed-forward network with following units added: new unit b to the hidden layer new input c(t) The value of c(t) is defined as the value of unit b at time t-1 i.e., c(t) = b(t-1). Thus it is possible for b to summarize information from earlier values of time series that are arbitrarily distinct in time. So, let T(t+1) be the stock price on date t+1. This model then predicts the stock price by using some economic indicator x(t) on the date t and values of x on previous days arbitrarily distinct in time. Now having seen the model we next need to select the number of data points needed to be used as the previous values in our RNN model. This is normally done through experimenting and be taking several set of these data points this is eventually achieved. 23 Stock Price Predictions Using Recurrent Neural Network–- SMPS Input Layer { 1,2,3 } 1 3 2 4 5 6 Hidden Layer { 4,5,6,7,8 } 7 8 10 9 Output Layer { 9,10 } Figure 4.1: Basic Recurrent Neural Network Model with three input neurons, five fully connected neuron and three output neurons. As shown in figure 4.1 above the use of RNN consist of: - Three input neurons. First one to enter time series input and the other two are used for the delayed inputs. - One hidden layer with five fully connected neurons. - Two output neurons. First one is trained to predict the probability of positive change while the other one is used to predict the negative change. 24 Stock Price Predictions Using Recurrent Neural Network–- SMPS Figure 4.2 represents the steps after initial preprocessing till time series prediction 25 Stock Price Predictions Using Recurrent Neural Network–- SMPS Figure 4.2 specifies the work flow of the processed time series. It uses Elman Network which takes feedback from the hidden layer. This model is completed by the use of ‘Rule Extraction from Neural Networks’. A structure can be extracted from RNN which is defined by set of ordered triple {state, input, next state}. These triples are called Markov chain or discrete Markov process which is equivalent to deterministic finite automata. This can be done by clustering the activation values of the recurrent state neurons. The automata extracted with this process can only recognize regular grammars12. Sample deterministic finite state automata (DFA) extracted from trained financial prediction networks using a quantization level of 5 can be seen in the figures in the appendix. Thus RNN with DFA with discrete Markov process form a hybrid approach which will be used to predict the time series. The analysis mechanisms that we have used are explained in detail below: 4.2 Normalized Mean Square Error Mean Square Error (MSE) is an accepted measure of control and quality but its practical use as a measure of quality has been limited. In application, MSE suffers from three problems, all of which can be avoided by using nMSE, a normalized form of the MSE. Just briefly describing the problems faced by MSE would be: It depends upon a unit of measure and cannot be averaged meaningfully over a mixture or processes, The utility of MSE is reduced due to the fact that there might be awkward numbers and have no apparent significance, except just comparing with the previous values, and It is difficult to use MSE to compare the quality of a process. 12 5A regular grammar G is a 4-tuple G = {S,N,T,P} _ where S is the start symbol, N and T _ are non-terminal and terminal symbols, respectively, and _ represents productions of the form A a or AaB where A, B belong to N and a belongs to T. 26 Stock Price Predictions Using Recurrent Neural Network–- SMPS Normalized Mean Square Error circumvents these three problems. This measuring metric is a very wide used method. Defining it mathematically, it looks something like: Where σ2 is the variance of the true time series in period [t0,t1], N is the number of patterns tested, and o(t) and d(t) are, respectively, the network and desired outputs at the time t. 4.3 Trend Hit Rate Coming down to the second method used, the hit rate is very useful when a trading decision is based solely on whether the predicted future price goes up or down as compared to the current price(Saadetal.1998). The hit rate of a predictor indicates how often the sign of the return, i.e. the daily change is correctly predicted. It is defined as follows: Let D ( t + h ) = sin ( S ( t + h ) - S( t ) ) be the actual direction of change for S(t), and D^(t + h) = sin ( S^(t + h) – S^(t) ) be the predicted direction change. We call a prediction for horizon h a hit if and only if D^(t + h) * D(t + h) > 0. The hit rate H(h) is defined as: 27 Stock Price Predictions Using Recurrent Neural Network–- SMPS where |E |represents the number of elements in set E. Chapter 5 Prototype 5.1 Analysis of Prototype 5.1.1 Training Functions The optimal settings of the prototype are achieved by running simulations on different training functions using different parameters of each training function every time. The criteria on which the simulations were assessed was minimizing MSE. The training functions used are given below. 1. Least Mean Squared (LMS) algorithm which involves Gradient descent with momentum and batch learning rate back-propagation 2. Least Mean Squared (LMS) algorithm which involves Gradient descent without momentum and batch learning rate back-propagation 28 Stock Price Predictions Using Recurrent Neural Network–- SMPS 1. Least Mean Squared (LMS) algorithm which involves Gradient descent with momentum and batch learning rate back-propagation It is a network training function that updates weight and bias values according to gradient descent momentum and batch mode learning rate and alpha rate. It is implemented in C sharp as comprehensive training and predicting algorithm. The parameters studied were the number of epochs, alpha (momentum) rate, number of hidden nodes and learning rate. Several experiments were done in the process to testify our learning rate parameter of 0.16 and 350 epochs with 5 hidden layers and alpha rate of 0.05. For these set of parameters we get the least mean squared error value. 2. Least Mean Squared (LMS) algorithm which involves Gradient descent without momentum and batch learning rate back-propagation It is a network training function that updates weight and bias values according to gradient descent momentum and batch mode learning rate with alpha parameter value to be equal to zero. It is implemented in C sharp as comprehensive training and predicting algorithm. The parameters studied were the number of epochs, number of hidden nodes and learning rate. Several experiments were done in the process to testify our learning rate parameter of 0.2 and 1500 epochs with 3 hidden layers. For this set of parameters we get the least mean squared error value. 5.1.2 Data Points selection: 29 Stock Price Predictions Using Recurrent Neural Network–- SMPS To decide upon the number of input data points to our neural network we have implemented a algorithm in MATLAB which is given four, five, six and seven as data points and after running it the results are shown in figure below and six comes out to be the right number for out stock data as the number of input to the neural networks. Figure 5.1 5.1.3 Network Parameters: Note: This mean squared error is calculated for each epoch and first 100 values are shown in the screen shot graphs Stock Name KESC stock Dataset Size 110 values epochs 200 Inputs 9 neurons hidden 8 neurons ourputs 3 neurons –2 for recurrence 1 for prediction Learning rate 0.01 Graph Scale 1 * 10-4 Results: Figure 5.2 30 Stock Price Predictions Using Recurrent Neural Network–- SMPS Figure 5.3 Stock Name PICIC 31 Stock Price Predictions Using Recurrent Neural Network–- SMPS Dataset Size 110 values epochs 500 Inputs 9 neurons hidden 8 neurons ourputs 3 neurons –2 for recurrence 1 for prediction Learning rate 0.1 Graph Scale 1 * 10-4 32 Stock Price Predictions Using Recurrent Neural Network–- SMPS Figure 5.4 Stock Name PICIC Dataset Size 110 values epochs 500 Inputs 9 neurons hidden 8 neurons ourputs 3 neurons –2 for recurrence 1 for prediction Learning rate 0.1 Graph Scale 1 * 10-4 33 Stock Price Predictions Using Recurrent Neural Network–- SMPS Figure 5.5 34 Stock Price Predictions Using Recurrent Neural Network–- SMPS Figure 5.6 Stock Name PICIC Dataset Size 110 values epochs 20 Inputs 9 neurons hidden 3neurons ourputs 3 neurons –2 for recurrence 1 for prediction Learning rate 0.01 Graph Scale 1 * 10-4 35 Stock Price Predictions Using Recurrent Neural Network–- SMPS Figure 5.7 Stock Name PICIC Dataset Size 110 values epochs 20 Inputs 9 neurons 36 Stock Price Predictions Using Recurrent Neural Network–- SMPS hidden 3neurons outputs 3 neurons –2 for recurrence 1 for prediction Learning rate 0.01 Graph Scale 1 * 10-4 37 Stock Price Predictions Using Recurrent Neural Network–- SMPS 5.1.4 Wavelet Transform The wavelet transform we have used is redundant or non-decimated Haar transform. By redundancy we mean that all the scales have the same size as the original signal. The need to have a non-decimated transform is that in a decimated one it is very difficult to visually or graphically relate information at a given time point at the different scales. More importantly, it is not possible to have shift invariance in a decimated transform. This means that if we had deleted the first few values of our input time series, then the output wavelet transformed, decimated, data would not be the same as heretofore. We can get around this problem at the expense of a greater storage requirement, by means of a redundant or non-decimated wavelet transform.13 13 The Wavelet Transform for Filtering Financial Data Streams 38 Stock Price Predictions Using Recurrent Neural Network–- SMPS 5.2 Implementation As this was more of a research project, we focused more on research, and implemented just enough in order to put all the research work into play. 5.2.1 System Interfaces The interfaces will require: Past Stock data from an excel file 5.2.2 User Interfaces A list of all the User Interfaces the system requires are given below. After making manipulations and carrying out calculations on the data, we then come towards the front end. There is a direct jump to the ‘front end’ because there are certain small calculations, which you will come across shortly, that cannot be put under a separate heading. After the file is loaded through the excel reader into our program then the number of instruments, that is the total amount of past data, is presented to our software. User Interfaces used by the potential investor: 39 Stock Price Predictions Using Recurrent Neural Network–- SMPS Figure 5.8 Once the system is given file name for the stock data then it up to the user to select the mode in which the user needs to get this data. There are several ways to do it which are listed as follows: Use noisy data by selecting “Use Noised data”. Use De-noise data by selecting “Automatic de-noising using wavelets”. Once data is imported into the program, then user needs to decide on the several learning rate parameters, number of epochs and number of hidden nodes used. There is a further classification into the type of training: With momentum Without momentum 40 Stock Price Predictions Using Recurrent Neural Network–- SMPS Using momentum make learning faster which is at cost of the depth of learning. Its is upto user to decide that with what network parameter does he want to train data. Once training is done then “predict stock” predict the stock for the next day. Again there are two modes: Price Prediction Trend prediction As the name specifies that the prediction is done either in a price mode or in trend mode. 5.2.3Hardware Interfaces No Hardware Interface Required. 5.2.4Software Interfaces Microsoft Excel Reader MICROSOFT Excel Excel reader Files DOT NET ARCHITECT URE Data Stock SMPS returned Data as data - Extracted source 5.3Functional requirements 5.3.1 Data Flow Diagrams 41 Stock Price Predictions Using Recurrent Neural Network–- SMPS Stock Trend LSE Website Daily Stock Prices Predicted Stock Trend External User Predictor Figure 5.9: Level 0 DFD 42 Stock Price Predictions Using Recurrent Neural Network–- SMPS LSE Website External User Daily Stock Prices In Reliability Ratios Text Files 1 4 Data Modeling Reliability Tests Previous Stock Data Formatted Data Predicted Stock In Excel Prices 3 2 Gausian Filtered Data Recurrent Preprocessing Neural Networks Figure 5.10: Level 1 DFD 43 Stock Price Predictions Using Recurrent Neural Network–- SMPS Formatted Data In Excel 2.1 Stationary Scale Related Components Non Decimated Haar Stationary Time Series 2.2 Auto Correlation Test 2.3 Gausian Noise Filter Gausian Filtered Data Figure 3.11: Level 2 DFD 44 Stock Price Predictions Using Recurrent Neural Network–- SMPS Preprocessed Time Series 3.1 Recurrent Neural Networks 3.2 Trained Output Marcov Discrete Predicted Stock Trend Process & Finite State Automation Figure 5.12: Level 2 DFD 45 Stock Price Predictions Using Recurrent Neural Network–- SMPS Previous Stock Prices 4.1 Calculated Hit Ratio Calculate Previous Hit Ratio Predicted Stock Prices 4.2 Trend Precision Calculate Reliabitliy Figure 5.13: Level 2 DFD 46 Stock Price Predictions Using Recurrent Neural Network–- SMPS 5.3.2 Structure Charts: 2.0 Preprocessor Controller e a m i t a T s D y e i r r d a e e n r S o e i t t l i a t F S 2.1 2.2 Preprocessor Gausian Filter Input Controller s t n s e t l e n a n e l e c o a n a S p t c o e y a m S p r l o D m a e y m i r C c n o d T a x s o e d i C n t y e t E t e i r o a t i d a r t t a a e n e l t I S a m n t e r S a l o S o i R e t F a R t S 2.3 2.4 Non-Decimated Auto correlation Harr Figure 5.14: Structure Chart 47 Stock Price Predictions Using Recurrent Neural Network–- SMPS 3.0 Recurrent Network Controller s d e i n r e e r S T t t e u u k p m p c i t t o u u T t O d O S e d d d s e e e s n t e i n i c c a i r a o d r r T e p T r e r P P 3.2 3.1 Marcov Discrete Recurrent Neural Process & Finite State Network Automation Figure 5.19: Structure Chart 48 Stock Price Predictions Using Recurrent Neural Network–- SMPS 4.0 Recurrent Network Controller s e c s i o r i e t o c P i i n a t r o k i a R P c s t R i i o k t c t c i H e S o r H d t d P e S d e t t e d a s t l c n i u a u l e d o r c i u l e r v T c a l e P r a C . . P C 1 2 4.1 4.2 Calculate Previous Hit Calculate Reliability Ratio Figure 5.20: Structure Chart 49 Stock Price Predictions Using Recurrent Neural Network–- SMPS Chapter 6 Recurrent Neural Network Results 6.1 Optimizing the Neural network The Recurrent Neural Network will be optimized in terms of reducing the MSE during the training session by manipulating the following variables: 1. Learning Rate. For 0 < lr < 1; 2. alpha, the cutoff frequency used in momentum . For 0 ≤ ∂ ≤ 0.1 WINDOW SIZE =6 (as determined via Matlab Dataselection.m file via simulation). We ran simulations on the Recurrent neural network varying the values of window sizes. The results were obtained via the following simple algorithm which uses the Matlab “newlin” function to adapt it self to the training model and then do prediction: interval = 1:length(data) delays = [3 4 5 6 7]; for i = 1:4 net = newlin(minmax(cat(2,P{:})),1,delays(:,1:i),learning_rate); [net,a,e,Pf] = adapt(net,P,T); 50 Stock Price Predictions Using Recurrent Neural Network–- SMPS net.trainParam.epochs = 5; net = train(net,P,T); Y = sim(net,[P]); meanForTrain = mse(T-Y) mmssee = [mmssee pref]; 6.1.1 Optimizing the Time Steps In order to optimize the Recurrent neural network, the window size was taken as 6 (obtained via Matlab algorithm. However, to still justify the Window Size, we varied the values over a certain range and found that 6 indeed gave us an optimal answer with varying Time Steps. Keeping a consistent value for the Window Size as 6, the time steps were varied over the following range, and the MSE’s obtained are listed as follows: into account a larger number of values in determining the ‘MSE’, therefore we decided to use Window Size as ‘7’, which gave us an equally smaller ‘MSE’ 51 Stock Price Predictions Using Recurrent Neural Network–- SMPS 6.2 Simulation Results The Simulation studies carried out were aimed at measuring the ‘ Trend Hit Rate’ of the neural networks with regards to a certain company. The companies that we chose to analyze and experiment with in this respect were the following; 1. Pakistan State Oil (PSO) 2. KESC 3. ICI Pakistan. 4. Glaxo Smithkline 5. Lakson Tobacco 6. Colgate 7. Engro Chemicals and Fertilizers 8. Shell 6.2.1 Pakistan State Oil (PSO) The results obtained from the noisy time series were fairly straightforward and expected. There was a 60% chance of a hit rate. After further processing by implementing “wden” function in MATLAB, , the results tuned out to be highly accurate; the Trend Hit rate jumped up to 90%. 52 Stock Price Predictions Using Recurrent Neural Network–- SMPS 6.2.2 KESC The time series data for this company was fairly stable over the entire time period. The minimum and maximum ranges were from Rs. 7 to 9 It is perhaps for this reason that the results obtained via noisy series and the de-noised series were fairly close, i.e. approximating around 80%. After a first few simulation which wielded incorrect results the results were almost all positive ‘hits’ for the rest of the data. 6.2.3 ICI Pakistan The results obtained for this company were fairly straightforward. Even the noisy time series was easily adapted to by the neural network. In fact, the network gave a few incorrect results when the simulations were run for the de-noised time series. Other than this the noisy time series gave almost a 100% accurate result for the hit rate. 6.2.4 Glaxo Smithkline The results obtained for the noisy time series and the de-noised time series were almost of the same character. After a few initial incorrect results the network was fairly able to predict the movement in the stock prices with a 70& accuracy. 53 Stock Price Predictions Using Recurrent Neural Network–- SMPS 6.2.5 Lakson Tobacco The results obtained for the noisy time series were fairly inaccurate yielding almot 55-60% accuracy. However, after applying ‘wden’ for de-noising it, the results jumped up to almost 70%. This once again was achieved after a few incorrect results after which the results showed steady positive results. 6.2.6 Colgate The results obtained for this data, showed that after a phase of de-noising the data, the results showed a positive improvement. The accuracy in terms of the hit rate reached 85%. 6.2.7 Engro Chemicals and Fertilizers Like ICI, Engro also showed almost 100% accurate hit rate for the noisy time series. Furhter de-noising only led to misses being included in the measure of hit rates. 6.2.8 Shell Just the straightforward wavelet-transforms on the data were enough to yield accurate results for this company. However, when de-noising was applied the results obtained were almost the same with very little change in the ‘hit-rate’ 54 Stock Price Predictions Using Recurrent Neural Network–- SMPS accuracy. The accuracy without de-noising was around 75-80%, while after de- noising it the accuracy bordered close to 80%. 55 Stock Price Predictions Using Recurrent Neural Network–- SMPS Chapter 7 Conclusion of Research Based on biological inspiration the Recurrent neural network architecture was proposed which provides the feed back to the system. The neural network incorporates sigmoid activation function, the nonlinearity that is a prominent characteristic of a human brain. The RNN has a simple yet effective learning rule that can incorporate changes in to the learning rule it self to become even more effective. This give us at the same time the ability to learn in depth as well as learn quickly. Multi Layered Perceptron with back propagation learning is by far the most efficient way to train neural networks (Simon Haykin, page 161). The effectiveness of this algorithm comes form the LMS gradient decent rule that is used to implement . One of the most important facts established during the course of the project is that stock prices of different companies exhibit unrelated behavior. Meaning that some companies’ data perform well without applying denoising function and some perform well after denoising. And for this reason we have in our software provided several options to choose from in regards to de-noising of data and applying modifications to the momentum rule itself. 56 Stock Price Predictions Using Recurrent Neural Network–- SMPS Chapter 8 Future Work Since we have implemented our system from scratch thus there are many areas in our system which can be extended and modified. Following are possible areas for future work; 8.1 Predictions using the live data Although the input data given to the system was as representative of real data as possible but the data used was historic and the prices predicted were known in advance. One exciting and useful extension of the project is connecting the system with Karachi stock exchange. System can be connected to Karachi stock exchange web site through Remote method invocation (RMI) using java or any other environment. This will feed the live data into the system and system will be able to predict in real time. This may prove extremely useful for short term investment and portfolio decision, serving the very purpose for which the system was designed. 8.2 Report generation and portfolio analysis Another possible and logical extension of our project is report generation. Currently the system only takes input based on historic prices and gives the output, the future price. More comprehensive output could be given to the user. The system could use the predicted data for decision making purpose and hence can give portfolio advices to the 57 Stock Price Predictions Using Recurrent Neural Network–- SMPS user. These detailed reports will make use of the predicted prices and company information, taking in consideration the direction of change and also the credibility of the company. The output will be a detailed report advising the user about the sale and purchase of the shares in the market. It can even predict the profit margin that user could obtain by this arbitrage. But the magnitude of the profit margin may not be correct. 8.3 Currency exchange rate prediction Since the stock prices and the currency exchange rates are essentially the time series prediction problem, our project can easily be modified to be used for currency exchange rate predictions. In effect it the prediction problems can be incorporated into a single system, making it a complete portfolio adviser. This will not only improve the market of our product but will also improve the report generation option. Since it will now have more data to use for decision making and analysis. 58 Stock Price Predictions Using Recurrent Neural Network–- SMPS References 1. “IEEE Recommended Practice for Software Requirements Specifications”, IEEE Computer Society. 2. Applying the Haar Wavelet Transform to Time Series Information 3. The Wavelet Transform for Filtering Financial Data Streams. 4. Data Mining in Finance. Numerical Data Mining Models and Financial Applications.Network and Grammatical Inference 1. C. Lee Giles___, Steve Lawrence_ , Ah Chung Tsoi_.. Noisy Time Series Prediction using a Recurrent Neural 2. 3. Wan, Eric. , FINITE IMPULSE RESPONSE NEURAL NETWORKS WITH APPLICATIONS IN TIME SERIES PREDICTION 4. 5. Haykin Simon, Neural Networks A Comprehensive foundation, S Second Edition. 59 Stock Price Predictions Using Recurrent Neural Network–- SMPS