DOCSLIB.ORG

Artificial Intelligence in Finance

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Artificial Intelligence in Finance Artificial Intelligence in Finance: Forecasting Stock Market Returns Using Artificial Neural Networks Alexandra Zavadskaya Department of Finance Hanken School of Economics Helsinki 2017 HANKEN SCHOOL OF ECONOMICS Department of: Finance Type of work: Thesis Author: Alexandra Zavadskaya Date: 29.09.2017 Title of thesis: Artificial Intelligence in Finance: Forecasting Stock Market Returns Using Artificial Neural Networks Abstract: This study explored various Artificial Intelligence (AI) applications in a finance field. It identified and discussed the main areas for AI in the finance: portfolio management, bankruptcy prediction, credit rating, exchange rate prediction and trading, and provided numerous examples of the companies that have invested in AI and the type of tasks they use it for. This paper focuses on a stock market prediction, and whether Artificial Neural Networks (ANN), being proxies for Artificial Intelligence, could offer an investor more accurate forecasting results. This study used two datasets: monthly returns of S&P500 index returns over the period 1968-2016, and daily S&P 500 returns over the period of 2007-2017. Both datasets were used to test for univariate and multivariate (with 12 explanatory variables) forecasting. This research used recurrent dynamic artificial neural networks and compared their performance with ARIMA and VAR models, using both statistical measures of a forecast accuracy (MSPE and MAPE) and economic (Success Ratio and Direction prediction) measures. The forecasting was performed for both in-sample and out-of-sample. Furthermore, given that ANN may produce different results during each iteration, this study has performed a sensitivity analysis, checking for the robustness of the results given different network configuration, such as training algorithms and number of lags. Even though some networks have outperformed certain linear models, the overall result is mixed. ARMA models were the best in minimizing forecast errors, while networks often had a better accuracy in a sign or direction prediction. Artificial neural networks outperformed respective VAR models in many parameters, but the difference of this outperformance was not significant, measured by S-test. Moreover, all model produced significant results in accurate stock index direction predictions, as 75-80% of the time the models predicted correct direction change in S&P500 return. Furthermore, combining Artificial Intelligence with the concept of Big Data, Google Trends search terms were used as a measure of a market sentiment. This study finds the use of Google trends search terms valuable in the modeling of S&P 500 returns. Keywords: forecasting, stock returns, S&P500, market sentiment, ANN, artificial neural networks, artificial intelligence, forecasting accuracy, forecast evaluation CONTENTS 1 INTRODUCTION ................................................................................................ 8 1.1 Purpose of the study ................................................................................................ 9 1.2 Study limit ................................................................................................................ 9 1.3 Contribution........................................................................................................... 10 1.4 Thesis organization ................................................................................................. 11 1.5 List of abbreviations .............................................................................................. 12 2 BACKGROUND INFORMATION ...................................................................... 13 2.1 What is AI .............................................................................................................. 13 2.2 Applications of AI in finance ................................................................................. 14 2.2.1 Anomaly detection .......................................................................................... 14 2.2.2 Portfolio management and robo-advisory ..................................................... 14 2.2.3 Algorithmic trading .........................................................................................15 2.2.4 Text mining ..................................................................................................... 16 2.2.5 Market, sentiment or news analysis ................................................................ 17 2.2.6 Credit evaluation and loan/insurance underwriting ....................................... 17 2.3 Types of Artificial Intelligence (AI) ....................................................................... 18 2.4 Artificial Neural Network (ANN) ........................................................................... 18 2.4.1 Types of ANN .................................................................................................. 18 3 THEORY REVIEW ............................................................................................ 22 3.1 Traditional Investment Theories ........................................................................... 22 3.2 Behavioral finance ................................................................................................. 22 3.3 Adaptive Market Hypothesis (AMH) ..................................................................... 24 3.4 Chaos Theory ......................................................................................................... 24 3.5 Time-series momentum theory. ............................................................................ 25 3.6 Forecasting methods .............................................................................................. 25 3.6.1 Linear vs. nonlinear methods ......................................................................... 26 3.6.2 Benefits of ANN .............................................................................................. 26 3.6.3 ANN drawbacks .............................................................................................. 27 3.6.4 Forecast Combinations ................................................................................... 27 1 4 LITERATURE REVIEW .................................................................................... 29 4.1 Bahrammirzaee’s (2010) review of previous studies of ANN applications ........... 29 4.2 Individual papers review ....................................................................................... 30 5 DATA .................................................................................................................. 38 5.1 Dataset 1 description .............................................................................................. 38 5.1.1 Variables ......................................................................................................... 39 5.1.2 Descriptive statistics ....................................................................................... 44 5.1.3 Correlations .....................................................................................................51 5.2 Dataset 2 description ............................................................................................. 53 5.2.1 Variables ......................................................................................................... 53 5.2.2 Descriptive statistics ....................................................................................... 57 5.2.3 Correlation analysis ........................................................................................ 59 6 RESEARCH METHOD ....................................................................................... 61 6.1 Research Question and Research Hypothesis: ...................................................... 61 6.2 Traditional long-term forecasting methods........................................................... 61 6.2.1 ARIMA ............................................................................................................ 61 6.2.2 VAR ................................................................................................................. 63 6.2.3 In sample forecasting versus out-of-sample .................................................. 64 6.3 Research method for ANN ..................................................................................... 64 6.3.1 Process of constructing and utilizing ANN .................................................... 64 6.3.2 NAR network .................................................................................................. 67 6.3.3 NARX network................................................................................................ 69 6.4 Method for comparing the model’s forecast accuracy ........................................... 70 6.4.1 Statistical loss functions ................................................................................. 70 6.4.2 Financial/economic loss function .................................................................. 72 6.5 Statistical hypotheses............................................................................................. 75 7 EMPIRICAL RESEARCH ................................................................................... 77 7.1 ARIMA modeling for monthly dataset .................................................................. 77 7.1.1 ARMA (5,2) model ............................................................................................. 79 7.2 ARIMA
Load more

Recommended publications
  • Decision Intelligence in Public Health – DIONE Stolk, J. and Nyon, S
    22nd International Congress on Modelling and Simulation, Hobart, Tasmania, Australia, 3 to 8 December 2017 mssanz.org.au/modsim2017 Decision Intelligence in Public Health – DIONE Jacob Stolk1 and Simone Nyon1 1 Dione Complex Systems, Gold Coast, Queensland, Australia Email: [email protected] Abstract: Public health surveillance is the ongoing systematic collection, analysis, interpretation, and dissemination of health data for the planning, implementation, and evaluation of public health action. To achieve effective public health interventions, it is pivotal to analyse and interpret the vast amounts of data collected by surveillance systems to enable good understanding of all factors having an impact on health. For example, we can consider child protection, which is an important public health issue. Often relatively extensive data exist on families in official statistics, research reports, social services reports, school and medical records, etc. However, these data are dispersed and hard if not impossible to relate and compare. This leads to numerous interventions that are conducted without adequate knowledge of target families that are to benefit from these interventions. Inadequate knowledge also leads to lack of intervention where it is needed, such as undetected cases of child abuse. In many cases the basic data that are needed for intervention decisions exist, but are not available to decision makers due to inadequate communication and lack of data integration, analysis and interpretation. Chronic condition management is another area where extensive disparate data exist from statistics and various health services and intervention agencies. In this area too there are numerous organizations offering services and a great need to better coordinate these services to achieve better outcomes for patients and also to reduce soaring costs of the healthcare system.
    [Show full text]
  • Introduction to Decision Intelligence - Towards Data Science
    10/26/2019 Introduction to Decision Intelligence - Towards Data Science Listen to this story --:-- 18:26 Introduction to Decision Intelligence A new discipline for leadership in the AI era Cassie Kozyrkov Follow Aug 3 · 13 min read Curious to know what the psychology of avoiding lions on the savannah has in common with responsible AI leadership and the challenges of designing data warehouses? Welcome to decision intelligence! Source: xijian/Getty Decision intelligence is a new academic discipline concerned with all aspects of selecting between options. It brings together the best of applied data science, social science, and managerial science into a unified field that helps people use data to improve their lives, their businesses, and the world around them. It’s a vital science for the AI era, covering https://towardsdatascience.com/introduction-to-decision-intelligence-5d147ddab767 1/12 10/26/2019 Introduction to Decision Intelligence - Towards Data Science the skills needed to lead AI projects responsibly and design objectives, metrics, and safety-nets for automation at scale. Decision intelligence is the discipline of turning information into better actions at any scale. Let’s take a tour of its basic terminology and concepts. The sections are designed to be friendly to skim-reading (and skip-reading too, that’s where you skip the boring bits… and sometimes skip the act of reading entirely). What’s a decision? Data are beautiful, but it’s decisions that are important. It’s through our decisions — our actions — that we affect the world around us. We define the word “decision” to mean any selection between options by any entity, so the conversation is broader than MBA-style dilemmas (like whether to open a branch of your business in London).
    [Show full text]
  • Demystifying AI in Cybersecurity
    WHITE PAPER | Demystifying AI in Cybersecurity Demystifying AI in Cybersecurity A look at the technology, myth vs. reality, and how it benefits the cybersecurity industry Table of Contents Introduction ...................................................................................................................3 AI: What it is and What it isn’t .......................................................................................3 Machine Learning: the Next Step ...................................................................................3 Deep Learning: Putting a Finer Point on it ......................................................................3 Plenty of Promise, for Better or Worse ..........................................................................3 The Difference AI Makes in Cybersecurity .....................................................................4 Malware Detection ....................................................................................................4 Classification and Scoring .........................................................................................4 Phishing Detection ....................................................................................................4 Key Requirements for AI/ML-based Security ..................................................................4 Data ..........................................................................................................................4 Speed ........................................................................................................................5
    [Show full text]
  • The Dynamics Duo: Microsoft Dynamics GP and AP Automation
    Summer 2015 Self-service Business Intelligence for GP: What does that mean for me? The Dynamics Duo: Microsoft Dynamics GP and AP Automation Managing Long-Term Investment Assets in Dynamics GP GP OPTIMIZER I On the Cover GP Optimizer Magazine 5 Published by Rockton Software Self-service Business PO Box 921 Lafayette, CO 80026 Intelligence for GP: Editor: Mark Rockwell Welcome to the Summer 2015 Edition of the GP Optimizer Magazine. Our original goal [email protected] What does that mean for me? of The GP Optimizer Magazine was to reach 15,000 Microsoft Dynamics GP Users. Last Design: Lori Hartmann, Feline Graphics publication, we exceeded this goal by reaching over 50,000 users. [email protected] Advertising Inquiries: Steven Solomon The GP Optimizer Magazine contains articles written by Microsoft Dynamics GP Add-On [email protected] Partners; these articles are focused on making your investment in Microsoft Dynamics Editorial Inquiries: Steven Solomon GP more worthwhile by solving an issue that you might be experiencing. [email protected] Rockton Software has been in the Microsoft Dynamics GP Channel for over 15 years. You may have seen us at Convergence dressed up as bartenders, pirates, Vikings, or The GP Optimizer Magazine is published by cavemen. Regardless of our crazy costumes, we’ve established ourselves as fervent Rockton Software, with principal offices in Lafayette, CO. If you wish to receive this publication, please go to supporters of the greater GP Channel, and we want you to excel in business by www.rocktonsoftware.com and click on the GP leveraging other tools and knowledge from our friends in the community.
    [Show full text]
  • Lorien Pratt, Ph.D. Curriculum Vita
    Lorien Pratt, Ph.D. Curriculum Vita Sunnyvale, CA USA +1.303.589.7476 [email protected] Blog: www.lorienpratt.com SUMMARY Professional machine learning and decision intelligence consultant, developer, entrepreneur, writer, evangelist. Author of Link: How Decision Intelligence Connects Data, Actions, and Outcomes for a Better World. Co-organizer of the annual Responsible AI/DI Summit. Two TEDx talks. Inventor of machine learning inductive transfer. Co-Founder and Chief Scientist of Quantellia, LLC. Co-inventor and evangelist for Decision Intelligence. Over three decades delivering applied machine learning solutions, including for the Human Genome Project, the US Department of Energy, budget analysis for a $100 million annual spend for the Administrative Office (AO) of the US Courts, award-winning forensic analysis for the Colorado Bureau of Investigation, RBS, DARPA, and many more. EDUCATION Ph.D., Computer Science, Rutgers University, 5/93. Thesis: Transferring previously learned back-propagation neural networks to new learning tasks. M.S., Computer Science, Rutgers University, 5/88. A.B., Computer Science, Dartmouth College, 5/83. CONSULTING A variety of engagements including the following: Regular invited speaker to user groups, executive roundtables, keynotes, webinars, and workshops for organizations and companies like SAP, HP, Unisys, the TM Forum, Comptel, SAS. Machine learning consulting on a wide variety of projects for cyber security, web advertising, financial prediction, medical decision making, cable company capital expenditure, LIDAR image analysis, and more. Five-star rated consultant on the Experfy platform (www.experfy.com), from the Harvard Innovation Laboratory. Strategic consulting in decision intelligence, analytics, and telecom OSS/BSS to companies and organizations like the US Department of Defense, Solvina, ConceptWave, the Colorado Bureau of Investigation (CBI), Sun, IBM, Comptel, SAP, HP, Amdocs, Telcordia (now Ericsson).
    [Show full text]
  • Decision Intelligence: a Two-Part Course
    Decision Intelligence: A two-part course Every decade or so, a new business discipline is born. Did you know that there was a time before project planning? Before Business Intelligence? Before Business Process Management? Today, Decision Intelligence is the next such breakthrough: it is a proven approach that has saved hundreds of millions of dollars for organizations worldwide. It brings the best of Big Data, Predictive Analytics, Deep Learning, and Systems Modeling to every role in the organization, including executive management, program managers, and quantitative modeling experts. Join us for workshop Getting Started with Decision Intelligence plus, optionally, Decision Intelligence Hands-On. You’ll learn: - How to answer the question: “If I make this decision today, what will be the outcome tomorrow?” - How to create a decision collaboration team to design, test, and update your organization’s most important decisions, creating continuous improvement and organizational learning - How to combine intangible factors , like employee engagement and customer experience, with tangible ones, like cost of goods and closed sales - Two secrets that, together, will increase your team’s decision-making intelligence tenfold - How to make decisions that have multiple outcomes - How to avoid the most common trap that causes Big Data projects to fail, and how to radically accelerate the value that you receive from all data, big and small - The three secrets of small data, and how you can use it to supercharge your decision intelligence - How to ensure
    [Show full text]
  • A Global Civic Debate FUTURE SOCIETY on Governing the Rise of Artificial Intelligence
    THE THE A Global Civic Debate FUTURE SOCIETY on Governing the Rise of Artificial Intelligence Report 2018 O1O1●●●●●●●●●O1OOO11O1O●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●. OOO 11OO1O1OO1OOOO● 1●●●1 ●O111O1O1 O1●●●●●●●●●●●●11O1OOO111O1O1O11 1OO●●●11OO1O1OO1OO●●●●●●●●●●●●●● ●●●●●●●●●OOOO1O1OO11O11O1111O1● ●●O11O11O1OO1O11OO1O1O111O1OOO1 111O●●●O1O11OOOO1O●●●●OOOOO1OOO OO1OO1OOOOOO1●●●111O1●●●●1O11OO. O11O1111O11OOO1OO11OO.•●●●•●●● OO 1●●●●1O11OOOO1OOOOO● OOOO11O11O 1OO1O111●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●O11OO11O1OO1O11OOO●●● ●●●●●●●●●●●●11OO1OOOOOO1OOO1OOO. ●●●●●●●●●●●●●●●.. 11OO1O1O11●●●●●● ●●●●●● ● ●●●●●●●●●●●●●●●OOO1OO. 11OOOO1O111O1OO. ●●●●●●O11OO1 ●●● 1OO ●OO11O1111●●●●●●1OOO1OOOOOO 1O●●●11O11O1111O11●●●●●●1O11OO 11OO1O1O111OO1●●●•••●1OO11O1OO1 O11O111OO11OO111OO1●●●OO●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●111O1OOO11O1OOOO11OO1O1OO1 OOOOO●●●1OO1OO11O1OO1O111OO11O1 “The Future Society, a Global Civic Debate” translated by the binary table of the American THE AI INITIATIVE Standard Code for Information Interchange. Contents : Executive Sumary p. 4 Our Promise p. 15 Key insights p. 17 Top recommandations p. 51 Tales from Imaginaries p. 65 Our Methodology p. 76 Debate Health p. 78 Beyond the Global Civic Debate p. 80 Authors : Nicolas Miailhe Arohi Jain Caroline Jeanmaire Yolanda Lannquist Josephine Png The Future Society © 2018 www.thefuturesociety.org The Future Society 2 A Global Civic Debate on Governing the rise of Artificial Intelligence 5 languages Engagement & Participation : English, French, Chinese, 70 390 Page views Japanese,
    [Show full text]
  • Marketing Optimization
    Marketing Optimization An Experian white paper Table of contents Executive summary .....................................................................................................................................2 Background - the marketing challenge ...................................................................................................4 The traditional approach - data driven marketing .................................................................................5 The next step - marketing optimization .................................................................................................10 What is marketing optimization? ............................................................................................................11 Marketing optimization applications .....................................................................................................15 How optimization improves traditional marketing approachs ..........................................................21 Optimization solutions from Experian ...................................................................................................22 About Experian Decision Analytics ........................................................................................................24 An Experian white paper | Page 1 Executive summary For your organization to thrive it is important to make the most of each customer interaction and maximize customer value. In a world where there are thousands or millions of customers, multiple touch
    [Show full text]
  • Sample Chapter [PDF]
    LINK Link is an exercise in abstraction, causality, and modeling. It is about discovering and making visible interdependencies in complex systems. The author distills what she has learned in pithy insights. It takes discipline. You won’t regret reading this book. À Vint Cerf, Internet Pioneer Link is the missing link to our understanding of unintended consequences of many of our decisions and actions. This is a book for the ages, moving both technology (like AI) and human decision making to the next level. Must read. À V R Ferose, SVP and Head of SAP Academy for Engineering There is an explosion today in the impacts, risks, and opportunities of many decisions by private or governmental entities intended to impact future events. Link is about understanding these decisions and causal relationships. Societal transactions are accelerating: many more than ever are intangible, and there are substantial complexities created by newly discovered information as well as the resulting increase of global interdependencies. Surveillance capitalism, especially as enhanced by AI, is also a substantial risk today. Link is part of the solution: a crucial resource to understand causal chains, especially with the goal of avoiding unintended consequences of decisions involving data and technology. À Bill Fenwick, Partner Emeritus, Fenwick & West LLP LINK How Decision Intelligence Connects Data, Actions, and Outcomes for a Better World BY LORIEN PRATT United Kingdom À North America À Japan À India À Malaysia À China Emerald Publishing Limited Howard House, Wagon Lane, Bingley BD16 1WA, UK First edition 2019 Copyright r Lorien Pratt, 2019. Published under exclusive licence.
    [Show full text]
  • Lorien Pratt, Ph.D. Curriculum Vita Sunnyvale, CA
    Lorien Pratt, Ph.D. Curriculum Vita Sunnyvale, CA . Denver, CO USA +1.303.589.7476 [email protected], @lorienpratt, www.lorienpratt.com SUMMARY Professional machine learning and decision intelligence consultant, developer, entrepreneur, writer, evangelist. Author of Link: How Decision Intelligence Connects Data, Actions, and Outcomes for a Better World. Co- organizer of the annual Responsible AI/DI Summit. Two TEDx talks. Inventor of transfer learning (TL). Co- Founder and Chief Scientist of Quantellia, LLC. Co-inventor and evangelist for Decision Intelligence (DI). Over three decades delivering applied machine learning solutions, including for the Human Genome Project, the US Department of Energy, budget analysis for a $100 million annual spend for the Administrative Office (AO) of the US Courts, award-winning forensic analysis for the Colorado Bureau of Investigation, RBS, DARPA, and many more EDUCATION • Ph.D., Computer Science, Rutgers University, 5/93. Thesis: Transferring previously learned back-propagation neural networks to new learning tasks. • M.S., Computer Science, Rutgers University, 5/88. • A.B., Computer Science, Dartmouth College, 5/83. CONSULTING A variety of engagements including: • Regular invited speaker to user groups, executive roundtables, keynotes, webinars, and workshops for organizations and companies like Global Knowledge, MCubed, MLConf, SAP, HP, Unisys, the TM Forum. • Machine learning consulting on a wide variety of projects for wellness, call centers, cyber security, web advertising, financial prediction, medical decision making, cable company capital expenditure, LIDAR image analysis, and more. • Five-star rated consultant on the Experfy platform (www.experfy.com), from the Harvard Innovation Laboratory. • Strategic consulting in decision intelligence, analytics, and telecom OSS/BSS to companies and organizations like the US Department of Defense, Solvina, ConceptWave, the Colorado Bureau of Investigation (CBI), Sun, IBM, Comptel, SAP, HP, Amdocs, Telcordia (now Ericsson).
    [Show full text]
  • Decision Intelligence for Wet Weather Management Xylem Digital Solutions
    ™ Intelligent Urban Watersheds Decision Intelligence for Wet Weather Management Xylem Digital Solutions Who we are A new Xylem platform of disruptive technologies to help water utilities substantially reduce the capital and operating costs of operating a water network What we do We use data gathering, artificial intelligence and machine learning tools to help utilities monitor, optimize and control the condition and performance of their networks Our impact We create outsize economic benefits (10x), substantially reducing non-revenue water and capital and operating costs through predictive analytics and design optimization Xylem Digital Solutions XDS major site Other major MCS site Nanjing Edmonton Manila Calgary Middletown Toronto South Bend Singapore Yellow Springs San Francisco Columbia Raleigh San Diego Atlanta Dallas Miami Sydney Problem: Increasing intensity and frequency of extreme wet weather Largest ever infrastructure spend for many cities Poorest citizens face largest economic burden Industry Status Quo Industry Status Quo . Intelligent Urban Watersheds Blue Infrastructure: Utilize data analytics to solve water challenges economically and efficiently. What is the Problem? Infrastructure Investment Affordability Status Quo Computing Capacity 1BillionX Era when many CSO interceptors and treatment plants were designed and built Intelligent Urban Watersheds The Internet of Things Big Data Analytics Machine Learning Digital Copy of Urban Watersheds Observation/ Digital Copy Tuning Urban Watershed Implementation Intelligent Urban
    [Show full text]
  • Cassie Kozyrkov, Chief Decision Scientist, Google
    Cassie Kozyrkov, Chief Decision Scientist, Google @quaesita @quaesita @quaesita @quaesita @quaesita @quaesita What is machine learning? @quaesita @quaesita @quaesita @quaesita @quaesita @quaesita @quaesita @quaesita Why do teams fail at machine learning? @quaesita Machine Learning Data Algorithms Models Predictions Ingredients Appliances Recipes Dishes @quaesita How does it work? @quaesita @quaesita @quaesita @quaesita Support Vector Classifier Decision Tree Neural Network @quaesita Label: Y or N @quaesita Label: Y or N @quaesita Label: Y or N Applied machine learning is a team sport @quaesita Meet the team Decision-maker @quaesita Data Decisions Is it tasty? Y or N ? Inputs Outputs @quaesita Meet the team Decision-maker Software engineer @quaesita Programmer Model Decisions Y or N ? Ada, Countess of Lovelace, 1815-1852 Expert Recipe Outputs @quaesita Meet the team Decision-maker Software engineer Data engineer @quaesita Data Programmer Models Decisions Is it tasty? Y or N ? Ingredients Expert Recipes Outputs @quaesita Meet the team Decision-maker Software engineer Data engineer Descriptive analyst @quaesita Data Analyst Models Decisions Y or N ? Is it tasty? Karl Friedrich Gauss 1777-1855 Y or N ? Florence Nightingale 1820-1910 Ingredients Expert Recipes Outputs @quaesita Meet the team Decision-maker Software engineer Data engineer Descriptive analyst Machine learning engineer @quaesita Data Algorithms Models Decisions Y or N ? Is it tasty? Y or N ? Y or N ? Ingredients Tools Recipes Outputs @quaesita Data Algorithms Models Decisions
    [Show full text]