Master of Science Thesis in Industrial Engineering and Management Department of Management and Engineering, Linköping University June 2017 Artificial neural networks for financial time series prediction and portfolio optimization Samuel Björklund Tobias Uhlin Credits: 30HP Level: A Advisor: Jörgen Blomvall Department of Management and Engineering, Linköping University Examiner: Ou Tang Department of Management and Engineering, Linköping University ISRN: LIU-IEI-TEK-A--17/02920—SE Abstract Predicting the return of financial times series using traditional technical analysis and widely used economic models such as the capital asset pricing model, has proven to be difficult. Machine learning is a subfield of computer science, a concept that is frequently being used within different domains and recurrently delivers successful results. Artificial neural network is a product from the field of machine learning, a black box model that if properly designed processes data, learns its dynamic and subsequently provides an informative output. The objective of this thesis is to develop artificial neural networks that predict time varying expected return of financial time series with purpose of optimizing portfolio weights. This thesis begins with a review of prevailing characteristics of financial times series, and existing methods for estimating statistical properties. Background on artificial neural network along with empirical case studies of their suitability within the fi- nancial domain is put into a brief discussion vis-a-vis the efficient market hypothesis where potential anomalies is highlighted. Based on the theoretical review, an inter- disciplinary approach between machine learning and finance culminated into a model that estimates the expected return of time series on a quarterly basis. To evaluate the use of predictions of future returns in a relevant context a portfolio optimization model, utilizing stochastic programming was built. The financial time series include FX-rates and indices from different asset classes such as equities and commodities. The results do not show any statistically significant improvement of the expected return estimation compared to CAPM and random walk models. However, our model outperforms a linear regression model for 12 of the 15 time series, which is in line with the performance of previous studies. Conclusively, no evidence is provided that the proposed model could predict accurately and regardless of the accuracy, the vast portfolio turnover of the model is not congruent with the context for which the model hypothetically would be used. i Acknowledgements We would like to show our appreciation to our supervisor, Jörgen Blomvall - Thank you for your support during the course of this thesis. Your engagement, comments and ideas have contributed with inspiration and invaluable insights in the fulfillment of this project. We are grateful for your expertise in the numerous courses that you have taught us that are applicable to this thesis. Also, we would like to express our gratitude to Söderberg & Partners for the opportu- nity to perform this study. We truly appreciate your friendly and welcoming attitude, a special thanks goes to Meng Chen for supportive advice and the assistance throughout this thesis. At last, we are grateful for the support from our family and friends. You have all been a source of energy through good and hard times of our studies. ii Nomenclature Abbreviations Table 1 contains the most commonly used abbreviations in this thesis. Table 1: Commonly used abbreviations in the thesis and the corresponding phrase Abbreviation Phrase AMH Adaptive Market Hypothesis ANN/NN Artificial Neural Network/Neural Network ARMA Autoregressive Moving Average CAPM Capital Asset Pricing Model EMH Efficient Market Hypothesis EWMA Exponentially Weighted Moving Average FFNN Feedforward neural network FX Foreign Exchange GARCH Generalized Autoregressive Conditional Heteroscedasticity MPT Modern Portfolio Theory MSA Model Selection Algorithm MSS Model Selection data Set OLS Ordinary least squares linear regression PCA Principal Component Analysis Q-Q Quantile to Quantile RW Random Walk SP Stochastic Programming Machine learning terminology Table 2 contains the most common machine learning specific terminology used in this thesis. The statistics equivalent terms are provided when possible. Table 2: Glossary of Machine Learning terminology Machine learning Statistics Backpropagation Repetead chain rule of partial derivatives Error function Objective function Input Explanatory variables Model selection data set In-sample data set Output Response variable Test set Out-of-sample data set Training Optimization Weights Regression coefficients iii Contents Abstract i Acknowledgements ii Nomenclature iii 1 Introduction 1 1.1 Company background . .5 1.2 Objective . .6 1.3 Limitations . .6 1.4 Disposition . .6 2 Scientific method 7 2.1 Literature review . .7 2.2 Phase 1: Forecasting using neural networks . .8 2.3 Phase 2: Portfolio optimization . 10 2.4 Analysis and evaluation . 10 3 Theoretical framework 12 3.1 Economic theory . 12 3.1.1 Efficient Market Hypothesis (EMH) . 12 3.1.2 Empirical properties of financial times series . 13 3.2 Point estimation . 13 3.2.1 Properties of a point estimator . 14 3.2.2 Point estimators . 15 3.3 Point estimation in finance . 17 3.3.1 Expected return . 18 3.4 Artificial neural networks paradigm . 21 3.4.1 Machine Learning . 21 3.4.2 Neurodynamics . 23 3.4.3 Architectures . 29 3.4.4 Classes of network . 36 3.5 Data preprocessing . 42 3.6 Optimization of neural network parameters . 43 3.6.1 Purpose of network training . 43 3.6.2 Training, validation and testing data sets . 43 3.6.3 Testing . 44 iv CONTENTS 3.6.4 Error functions . 45 3.6.5 Problem structure . 46 3.6.6 Optimization of neural network weights . 47 3.6.7 Generalization error . 53 3.6.8 Hyper-parameter optimization . 56 3.7 Portfolio optimization . 58 3.7.1 Utility functions . 58 3.7.2 Mean Variance . 59 3.7.3 Stochastic Programming . 60 3.7.4 Scenario generation . 61 3.8 Evaluation . 64 3.8.1 Prediction of expected return . 64 3.8.2 Portfolio optimization . 68 3.9 Empirical tests . 69 3.9.1 The use of data mining and neural networks for forecasting stock market returns . 69 3.9.2 An investigation of model selection criteria for neural network time series forecasting . 71 3.9.3 Much ado about nothing? Exchange rate forecasting: Neural networks vs. linear models using monthly and weekly data . 73 4 Method 75 4.1 Data . 75 4.1.1 Software . 75 4.1.2 Processing . 75 4.2 Phase 1: Neural network . 76 4.2.1 Model selection algorithm . 77 4.2.2 Class . 78 4.2.3 Architecture . 79 4.2.4 Preprocessing . 83 4.2.5 Postprocessing . 84 4.2.6 Neurodynamics . 84 4.2.7 Error function . 85 4.2.8 Training . 86 4.2.9 Overfitting prevention . 87 4.3 Phase 2: Portfolio optimization . 88 4.3.1 Foreign exchange . 88 4.3.2 External specifications . 88 4.3.3 Portfolio optimization model . 88 4.3.4 Scenario generation . 90 4.3.5 Estimating the portfolio optimization parameters . 92 4.3.6 Solving the portfolio optimization problem . 93 4.4 Evaluation . 93 4.4.1 Phase 1 . 93 4.4.2 Phase 2 . 95 4.4.3 Summary of evaluation . 96 v CONTENTS 5 Results & Analyses 97 5.1 Phase 1: Neural Network . 97 5.1.1 Error function . 97 5.1.2 Hyper-parameters . 100 5.1.3 Results from the optimized neural networks . 102 5.1.4 Evaluation . 104 5.1.5 Sensitivity analysis . 106 5.1.6 Test for model convergence . 108 5.2 Phase 2: Portfolio optimization . 110 5.2.1 Results from estimated stochastic processes . 110 5.2.2 Results from portfolio optimization . 111 5.2.3 Evaluation . 114 5.3 Summary of results and analyses . 115 6 Conclusions & Discussion 116 6.1 Conclusions . 116 6.2 Discussion . 117 6.3 Ethical aspects . 120 Bibliography 120 A Variable declaration 128 B Data description 132 C Estimations of volatility and correlation 135 D Univariate distributions 140 E Plots for different error functions 143 F Resulting networks 150 G Out-of-sample performance 155 vi Chapter 1 Introduction It is widely accepted that predicting the future with 100 % accuracy is impossible. However, the famous French mathematician Henri Poincaré once said "It is far better to foresee even without certainty than not to foresee at all". What if an investor could foresee the statistical properties of financial times series? This thesis will try to improve estimation of expected return versus traditional models by using machine learning with the purpose of generating higher portfolio returns and in the end earn more money. Modern Portfolio Theory (MPT) is a commonly used investment model that assumes that an investor wants to maximize the expected return given an expected risk level, commonly measured by the standard deviation of the return (Markowitz, 1952). Hence, the model requires not only to estimate the future return, but also to estimate the risk of the portfolio. In MPT an optimal portfolio is defined as a portfolio where the investor is required to raise the risk level in order to achieve higher expected return. (Markowitz, 1952) As such, the investor needs to ately be able to predict both the return of a set of assets and the covariance matrix for the constituents in order to maintain optimal allocation of the assets within the portfolio. Black and Litterman (1992) argue that two fundamental problems with most quantitative portfolio models are to come up with reasonable forecasts of the return, and that small differences in the predicted expected return drastically changes the asset allocations of the portfolio. The Efficient Market Hypothesis (EMH) suggests that the asset prices on capital mar- kets fully reflect all the available information, and that new information instantly will be incorporated in the price. This implies that an investor cannot exploit any available information to forecast returns.