RECENT DEVELOPMENT IN DEEP LEARNING IN FINANCE Harsh Prasad DISCLAIMER: 1. Any opinion in this presentation is in my personal capacity 2. Material presented here is in the nature of literature review of work done by others Adoption of Deep Learning in Finance 1. Deep Learning for Financial Applications : A Survey Ahmet Murat Ozbayoglua , Mehmet Ugur Gudeleka , Omer Berat Sezera 2. Deep learning in finance and banking: A literature review and classification Jian Huang , Junyi Chai and Stella Cho 3. Artificial Intelligence in Finance (Bonnie G Buchanan, Alan Turing Institute) 1. Financial text mining, Algo-trading, risk assessments, sentiment analysis, portfolio management and fraud detection are among the most studied areas of finance research. 2. Even though DL models already had better achievements compared to traditional counterparts in almost all areas, the overall interest is still on the rise in all research areas. 3. Cryptocurrencies, blockchain, behavioral finance, HFT and derivatives market have promising potentials for research 4. RNN based models (in particular LSTM), CNN and DMLP have been used extensively in implementations 5. In most of the studies, DL models performed better than their ML counterparts. 6. Hybrid models based on Spatio-temporal data representations, NLP, semantics and text mining-based models might become more important in the near future. Purpose of Finance 1. Why Finance Matters: Building an industry that serves its customers and society Pitt-watsonand Mann 2. On the theory and measurement of financial intermediation Phillipon 2015, 2016 1. The academic literature identifies the four principle purposes of the finance industry, against which its output can be measured. These are: • The safe-keeping of assets; • Providing an effective payment system; • Pooling risk; • Intermediation –matching the users and suppliers of money. 2. Other consideration which are in the nature of enabling function or externalities include providing information in a decentralized system and manage asymmetric information (Merton and Bodie) and providing liquidity, or developing new processes (Epstein). 3. “enabling functions” such as successful innovation, or the management of asymmetric information andeExternalities of intermediation leading to price discovery and separation of ownership and management. 4. It is difficult not to see finance as an industry with excessive rents and poor overall efficiency. The puzzle is why this has persisted for so long. There are several plausible explanations for this: zero-sum games in trading activities, inefficient regulations, barriers to entry, increasing returns to size, etc Is Big data enabling finance? 1. Market efficiency in the age of big data Ian Martin, Stefan Nagel, 2020 2. Artificial Intelligence and Asymmetric Information Theory Tshilidzi Marwala and Evan Hurwitz 1. Does big data give more predictability? 2. Does it help make information symmetry? 3. Efficient market hypothesis states that the market incorporates all the information such that it is impossible to beat the market (Fama, 1965). It thus follows that the only way to beat the market is to engage in high risk transactions. Implicit in the efficient market hypothesis is the fact that the agents that participate in the market are rational. Of course we now know that human agents are not rational and therefore the markets cannot be rational. Theories such as prospect theory and bounded rationality have proven that at best human agents are not fully rational but almost always are not rational (Simon, 1974; Kahneman and Tversky, 1979). Marwala (2015) surmised that artificial intelligent agents make markets more rational than human agents. 4. Five specific economic patterns influenced by AI are discussed: (1) following in the footsteps of ‘homo economicus’ a new type of agent, ‘machina economica’, enters the stage of the global economy. (2) The pattern of division of labor and specialization is further accelerated by AI-induced micro-division of labor. (3) The introduction of AI leads to triangular agency relationships and next level information asymmetries. (4) Data and AI-based machine laborhave to be understood as new factors of production. (5) The economics of AI networks can lead to market dominance and unwanted external effects. NEURAL NETWORK ퟎ 풊풇 σ풋 풘풋풙풋 ≤ 풕풉풓풆풔풉풐풍풅 Output = ൝ ퟏ 풊풇 σ풋 풘풋풙풋 > 풕풉풓풆풔풉풐풍풅 ퟎ 풊풇 풘. 풙 + 풃 ≤ ퟎ Output = ቊ ퟏ 풊풇 풘. 풙 + 풃 > ퟎ ACTIVATION FUNCTIONS ACTIVATION FUNCTIONS NETWORK AND ACTIVATION FEEDFORWARD VS RECURRENT USE CASE: HANDWRITING RECOGNITION SOLVING THE PROBLEM – FINAL DECISION BREAKING THE PROBLEM STATEMENT LAYER 2 DECISION LAYER 1 DECISION FEEDFORWARD DECISION FLOW COST FUNCTION COST MINIMIZATION GRADIENT DESCENT BACKPROPAGATION Deep Learning for Fraud Detection and Risk Assessment 1. Deep learning detecting fraud in credit card transactions Abhimanyu Roy, Jingyi Sun, Robert Mahoney, Loreto Alonzi, Stephen Adams, and Peter Beling 2. Towards automated feature engineering for credit card fraud detection using multi-perspective HMMs Y Lucas et al 2020 1. Roy et al (2017) while studying fraud detection in credit card transaction indicated that he LSTM and GRU model significantly outperformed the baseline ANN which indicates that order of transactions for an account contains useful information in differentiating between fraud and non-fraudulent transactions. 2. Lucas et al presented a feature engineering framework to model a sequence of credit card transactions from three different perspectives, namely (i) The sequence contains or doesn’t contain a fraud (ii) The sequence is obtained by fixing the card-holder or the payment terminal (iii) It is a sequence of spent amount or of elapsed time between the current and previous transactions. Combinations of the three binary perspectives give eight sets of sequences from the (training) set of transactions. Each one of these sequences is modelled with a Hidden Markov Model (HMM). Each HMM associates a likelihood to a transaction given its sequence of previous transactions. These likelihoods are used as additional features in a Random Forest classifier for fraud detection. 3. The feature engineering strategy is shown to perform well for e-commerce and face-to-face credit card fraud detection: the results show an increase in the precision–recall AUC of 18.1% for the face-to-face transactions and 9.3% for the e-commerce ones. The feature engineering strategy is shown to be rele- vant for various types of classifiers (random forest, logistic regression and Adaboost) and robust to hyperparameters choices made for constructing the features. Deep Learning for Fraud Detection and Risk Assessment 1. Dual Sequential Variational Autoencoders for Fraud Detection Ayman Alaziz, Amaury Habrard, Francois Jacquenet, LiyunHe-Guelton, and Frederic Oble, 2020 1. An autoencoder is a special type of feedforward neural network where the input is same as the output. It compresses the input into a lower dimension and then reconstructs the output from this representation. It is an unsupervised algorithm that applies backpropagation to set the target value to be equal to the input. It is made up of two parts linked together: an encoder E(x) and a decoder D(z). Given an input sample x, the encoder generates z, a condensed representation of x. The decoder is then tuned to reconstruct the original input x from the encoded representation z. The objective function used during the training of the AE is given by: LAE (x) = ∥x − D(E(x))∥ where ∥ · ∥ denotes an arbitrary distance function. The l2 norm is typically applied here. The AE can be optimized for example using stochastic gradient descent. 2. A Variational autoencoder (VAE) is an attractive probabilistic generative version of the standard autoencoder. It can learn a complex distribution and then use it as a generative model defined by a prior p(z) and conditional distribution pθ(x|z). Due to the fact that the true likelihood of the data is generally intractable, a VAE is trained through maximizing the evidence lower bound (ELBO): 3. Negative learning is a technique used for regularizing the training of the AE in the presence of labelled data by limiting reconstruction capability (LRC) . The basic idea is to maximize the reconstruction error for abnormal instances, while minimizing the reconstruction error for normal ones in order to improve the discriminative ability of the AE. Given an input instance x ∈ Rn and y ∈ {0, 1} denotes its associated label where y = 1 stands for a fraudulent instance and y = 0 for a genuine one. The objective function of LRC to be minimized is: (1 − y)LAE (x) − (y)LAE (x) Deep Learning for Fraud Detection and Risk Assessment 1. Dual Sequential Variational Autoencoders for Fraud Detection Ayman Alaziz, Amaury Habrard, Francois Jacquenet, LiyunHe-Guelton, and Frederic Oble, 2020 1. The DuSVAE model consists of a generative model that takes into account the sequential nature of the data. It combines two variational autoencoders that can generate a condensed representation of the input sequential data that can then be processed by a classifier to label each new sequence as fraudulent or genuine. 2. One of the main contribution of this paper is to propose a method to identify fraudulent sequences of credit transactions in the context of highly imbalanced data. For this purpose, the Dual Sequential Variational Autoencoders is used, that consists of a combination of two variational autoencoders. The first one is trained from fraudulent sequences of transactions in order to be able to project the input data into another feature space and to assign a fraud score to each sequence thanks to the reconstruction error information. Once this model is trained, a second VAE is plugged at the output of the first one. This second VAE is then trained with a negative learning approach with the objective to maximize the reconstruction error of the fraudulent sequences and minimize the reconstruction error of the genuine ones. Deep Learning for Text Mining/Sentiment Analysis 1. Decision support from financial disclosures with deep neural networks and transfer learning (Mathias Kraus, Stefan Feuerriegel 2017) 2.