DOCSLIB.ORG

Arxiv:2006.14698V1 [Math.NA] 10 Jun 2020

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Arxiv:2006.14698V1 [Math.NA] 10 Jun 2020 Entanglement-Embedded Recurrent Network Architecture Entanglement-Embedded Recurrent Network Architecture: Tensorized Latent State Propagation and Chaos Forecasting Xiangyi Meng1, a) and Tong Yang2, b) 1)Center for Polymer Studies and Department of Physics, Boston University, USA 2)Department of Physics, Boston College, USA (Dated: 29 June 2020) Chaotic time series forecasting has been far less understood despite its tremendous potential in theory and real-world applications. Traditional statistical/ML methods are inefficient to capture chaos in nonlinear dynamical systems, espe- cially when the time difference Dt between consecutive steps is so large that a trivial, ergodic local minimum would most likely be reached instead. Here, we introduce a new long-short-term-memory (LSTM)-based recurrent architec- ture by tensorizing the cell-state-to-state propagation therein, keeping the long-term memory feature of LSTM while simultaneously enhancing the learning of short-term nonlinear complexity. We stress that the global minima of chaos can be most efficiently reached by tensorization where all nonlinear terms, up to some polynomial order, are treated ex- plicitly and weighted equally. The efficiency and generality of our architecture are systematically tested and confirmed by theoretical analysis and experimental results. In our design, we have explicitly used two different many-body entan- glement structures—matrix product states (MPS) and the multiscale entanglement renormalization ansatz (MERA)—as physics-inspired tensor decomposition techniques, from which we find that MERA generally performs better than MPS, hence conjecturing that the learnability of chaos is determined not only by the number of free parameters but also the tensor complexity—recognized as how entanglement entropy scales with varying matricization of the tensor. I. INTRODUCTION ing methods. The notorious indication of chaos, jdx j ≈ elt jdx j; (1) Time series forecasting1, despite its undoubtedly tremen- t 0 dous potential in both theoretical issues (e.g., mechanical (where l denotes the spectrum of Lyapunov exponents) sug- analysis, ergodicity) and real-world applications (e.g., traffic, gests that the difficulty of chaos forecasting is two-fold: first, weather), has long been known as an intricate field. From any small error will propagate exponentially, and thus multi- classical work of statistics such as auto-regressive moving step-ahead predictions will be exponentially worse than one- 2 average (ARMA) families and basic hidden Markov mod- step-ahead ones; second, and more subtly, when the actual 3 els (HMM) to contemporary machine-learning (ML) meth- time difference Dt between consecutive steps increases, the 4 ods such as gradient boosted trees (GBT) and neural net- minimum redundancy needed for smoothly descending to the works (NN), the essential complexity inhabiting in time series global minima during NN training also increases exponen- has been more and more recognized, the forecasting models tially. Most studies on chaos forecasting only address the first having extended themselves from linear, Markovian assump- difficulty by improving prediction accuracy at the global min- tions to nonlinear, non-Markovian, and even more general sit- ima, yet the latter is indeed more crucial, especially when Dt 5 uations . Among all known methods, recurrent NN archi- is so large that a trivial, ergodic local minimum would most 6 7 tectures , including plain recurrent neural networks (RNN) likely be reached instead. Recently, tensorization has been 8 and long short-term memory (LSTM) , are the most capa- introduced in recurrent NN architectures16,17. A tensorized ble of capturing the complexity, as they admit the funda- version of HO-RNN/LSTM, namely HOT-RNN/LSTM18, has mental recurrent behavior of time series data. LSTM has claimed its advantage in learning long-term nonlinearity on 9 proved useful on speech recognition and video analysis tasks Lorenz systems of small Dt. On the one hand, we believe where maintaining long-term memory is the essential com- that the global minima of chaos (where dominance of linear plexity. To this objective, novel architectures such as higher- dependence is absent) can be most efficiently reached by ten- 10 arXiv:2006.14698v1 [math.NA] 10 Jun 2020 order RNN/LSTM (HO-RNN/LSTM) have been introduced sorization approaches, where all nonlinear terms, up to some to capture long-term non-Markovianity explicitly, further im- polynomial order, are treated explicitly and weighted equally. proving performance and leading to more accurate theoretical On the other hand, for simple chaotic dynamical systems, non- analysis. linear complexity is only encoded in the short term, not long Still, another dominance of complexity—chaos—has been term, which HO/HOT models will not be very useful in cap- far less understood. Even though enormous theory/data- turing when Dt is large. Hence, a new tensorization-based driven studies on forecasting chaotic time series by recurrent recurrent NN architecture is desired so as to push our under- NN have been conducted11–15, there is still no clear guidance standing of chaos in time series and meet practical needs, e.g., of which features play the most general role in chaos forecast- modeling laminar flame fronts and chemical oscillations19–22. In this paper, we introduce a new LSTM-based architec- ture by tensorizing the cell-state-to-state propagation therein, keeping the long-term memory feature of LSTM while si- a)Electronic mail: [email protected]. multaneously enhancing learning of short-term nonlinear b)Electronic mail: [email protected]. complexity. Compared with traditional LSTM architec- Entanglement-Embedded Recurrent Network Architecture 2 23 tures including stacked LSTM and other aforementioned Next, a realization of an LSTM is given by letting state st statsitics/ML-based forecasting methods, our model is shown and cell state ct be h-dimensional covectors and input xt to be a general and the best approach for capturing chaos in al- be a d-dimensional covector (Fig. 1). Therefore Wi (also most every typical chaotic continuous-time dynamical system Wm, W f , and Wo) has a direct-sum contravariant realization and discrete-time map with controlled comparable NN train- Wi 2 M(h;1) ⊕ M(h;d) ⊕ M(h;h) and contains h(1 + d + h) ing conditions, justified by both our theoretical analysis and free real parameters at maximum. During NN training, only experimental results. Our model has also been tested on real- the free parameters of each linear operator W are learnable, world time series datasets where the improvements range up while all s (i.e., activation functions) are fixed to be tanh, to 6:3%. sigmoid or other nonlinear functions. ct is known to suffer During the tensorization, we have explicitly embedded less from the vanishing gradient problem and thus captures many-body quantum state structures—a way of reducing the long-term memory better, while st tends to capture short-term exponentially large degree of freedom of a tensor (a.k.a. ten- dependence. sor decomposition)—from condensed matter physics, which Our tensorized LSTM architecture (Fig. 1) is exactly based is not uncommon for NN design24. Since a many-body on Eq. (2), from which the only change is entangled state living in a tensor-product Hilbert space is st = g(1;xt− ;st− ;Wo) ◦ g(T (s(ct ));W ): (3) hardly separable, this similarity motivates us to adopt a spe- 1 1 T cial measure of tensor complexity, namely, the entanglement g(T (s(ct ));WT ) is called a tensorized state propagation 25 entropy (EE) , which is commonly used in quantum physics function as WT : T ! G acts on a covariant tensor, and quantum information24. For one-dimensional many-body O O T (s(ct )) = 1 ⊕ q = (1 ⊕ W (s(ct ))): (4) states, two thoroughly studied, popular but different struc- t;l l l l tures exist: multiscale entanglement renormalization ansatz (MERA)26 or matrix product states (MPS)27, of which the EE Each Wl in Eq. (4) maps from the cell state ct to a new cov- scales with the subsystem size or not at all, respectively25. For ector qt;l 2 Q. Here, Q is named a local q-space, as by way most pertinent studies, MPS has been proved efficient enough of analogy it is considered encoding the local degree of free- to be applicable to a variety of tasks28–31. However, our exper- dom in quantum mechanics. Q can be extended to the com- iments show that, regarding our entanglement-embedded de- plex number field if necessary. Mathematically, Eq. (4) offers sign of the new tensorized LSTM architecture, LSTM-MERA the possibility of directly constructing orthogonal polynomi- als up to order L from s(ct ) to build up nonlinear complex- performs even better than LSTM-MPS in general without in- ⊗L ity. In fact, when L goes to infinity, T = lim (1 ⊕ Q) = creasing the number of parameters. Our finding leads to an- L!¥ other interesting result: we conjecture that not only should 1 ⊕ Q ⊕ Q ⊗ Q ⊕ ··· becomes a tensor algebra (up to a mul- tensorization be introduced, but the tensor’s EE has to scale tiplicative coefficient), and T (s(ct )) admits any nonlinear with the system size as well—and hence MERA is more effi- smooth function of ct . cient than MPS at learning chaos. Following the same procedure above, Eq. (4) is realized by choosing L independent realizations, Wl 2 M(P − 1;h), l = 1;2;··· ;L, which in total contain L(P − 1)h learnable pa- II. TENSORIZED STATE PROPAGATION rameters at maximum, 00 1 1 0 1 1 ··· 0 1 11 0[tanhc ] 1 A. Formalism t 1 linear + [q ] [q ] ··· [q ] [tanhc ] BB t 21C B t 22C B t 2LCC B t 2C padding BB[q ] C B[q ] C ··· B[q ] CC B . C −−−−−−! BB t 31C B t 32C B t 3LCC; The formalism starts from an operator-theoretical perspec- @ .
Load more

Recommended publications
  • Neural Networks (AI) (WBAI028-05) Lecture Notes
    Herbert Jaeger Neural Networks (AI) (WBAI028-05) Lecture Notes V 1.5, May 16, 2021 (revision of Section 8) BSc program in Artificial Intelligence Rijksuniversiteit Groningen, Bernoulli Institute Contents 1 A very fast rehearsal of machine learning basics 7 1.1 Training data. .............................. 8 1.2 Training objectives. ........................... 9 1.3 The overfitting problem. ........................ 11 1.4 How to tune model flexibility ..................... 17 1.5 How to estimate the risk of a model .................. 21 2 Feedforward networks in machine learning 24 2.1 The Perceptron ............................. 24 2.2 Multi-layer perceptrons ......................... 29 2.3 A glimpse at deep learning ....................... 52 3 A short visit in the wonderland of dynamical systems 56 3.1 What is a “dynamical system”? .................... 58 3.2 The zoo of standard finite-state discrete-time dynamical systems .. 64 3.3 Attractors, Bifurcation, Chaos ..................... 78 3.4 So far, so good ... ............................ 96 4 Recurrent neural networks in deep learning 98 4.1 Supervised training of RNNs in temporal tasks ............ 99 4.2 Backpropagation through time ..................... 107 4.3 LSTM networks ............................. 112 5 Hopfield networks 118 5.1 An energy-based associative memory ................. 121 5.2 HN: formal model ............................ 124 5.3 Geometry of the HN state space .................... 126 5.4 Training a HN .............................. 127 5.5 Limitations ..............................
    [Show full text]
  • Predrnn: Recurrent Neural Networks for Predictive Learning Using Spatiotemporal Lstms
    PredRNN: Recurrent Neural Networks for Predictive Learning using Spatiotemporal LSTMs Yunbo Wang Mingsheng Long∗ School of Software School of Software Tsinghua University Tsinghua University [email protected] [email protected] Jianmin Wang Zhifeng Gao Philip S. Yu School of Software School of Software School of Software Tsinghua University Tsinghua University Tsinghua University [email protected] [email protected] [email protected] Abstract The predictive learning of spatiotemporal sequences aims to generate future images by learning from the historical frames, where spatial appearances and temporal vari- ations are two crucial structures. This paper models these structures by presenting a predictive recurrent neural network (PredRNN). This architecture is enlightened by the idea that spatiotemporal predictive learning should memorize both spatial ap- pearances and temporal variations in a unified memory pool. Concretely, memory states are no longer constrained inside each LSTM unit. Instead, they are allowed to zigzag in two directions: across stacked RNN layers vertically and through all RNN states horizontally. The core of this network is a new Spatiotemporal LSTM (ST-LSTM) unit that extracts and memorizes spatial and temporal representations simultaneously. PredRNN achieves the state-of-the-art prediction performance on three video prediction datasets and is a more general framework, that can be easily extended to other predictive learning tasks by integrating with other architectures. 1 Introduction
    [Show full text]
  • Mathematics Calendar
    Mathematics Calendar The most comprehensive and up-to-date Mathematics Calendar information is available on e-MATH at http://www.ams.org/mathcal/. August 2006 wide to discuss the most recent developments in anomalous energy (heat) transport in low dimensional systems, synchronization of 1–5 Ninth Meeting of New Researchers in Statistics and Proba- chaotic systems and applications to communication of information. bility,University of Washington, Seattle, Washington. (Mar. 2006, It also serves as a forum to promote regional as well as international p. 379) scientific exchange and collaboration. Description:TheIMSCommittee on New Researchers is organizing ameeting of recent Ph.D. recipients in Statistics and Probability. Information and registration: http://www.ims.nus.edu.sg/ The purpose of the conference is to promote interaction among new Programs/chaos/;email:[email protected] on researchers primarily by introducing them to each other’s research scientific aspects of the program, please email Baowen Li at in an informal setting. As part of the conference, participants will [email protected]. present talks and posters on their research and discuss interests and 2–4 31st Sapporo Symposium on Partial Differential Equations, professional experiences over meals and social activities organized Department of Mathematics, Hokkaido University, Sapporo, Japan. through the meeting as well as by the participants themselves. The (Jan. 2006, p. 70) relationships established in this informal collegiate setting among Description:TheSapporo Symposium on Partial Differential Equa- junior researchers are ones that maylastacareer (lifetime?!) The tions has been held annually to present the latest developments on meeting is to be held prior to the 2006 Joint Statistical Meetings in PDE with a broad spectrum of interests not limited to the methods Seattle, WA.
    [Show full text]
  • Mxnet-R-Reference-Manual.Pdf
    Package ‘mxnet’ June 24, 2020 Type Package Title MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Sys- tems Version 2.0.0 Date 2017-06-27 Author Tianqi Chen, Qiang Kou, Tong He, Anirudh Acharya <https://github.com/anirudhacharya> Maintainer Qiang Kou <[email protected]> Repository apache/incubator-mxnet Description MXNet is a deep learning framework designed for both efficiency and flexibility. It allows you to mix the flavours of deep learning programs together to maximize the efficiency and your productivity. License Apache License (== 2.0) URL https://github.com/apache/incubator-mxnet/tree/master/R-package BugReports https://github.com/apache/incubator-mxnet/issues Imports methods, Rcpp (>= 0.12.1), DiagrammeR (>= 0.9.0), visNetwork (>= 1.0.3), data.table, jsonlite, magrittr, stringr Suggests testthat, mlbench, knitr, rmarkdown, imager, covr Depends R (>= 3.4.4) LinkingTo Rcpp VignetteBuilder knitr 1 2 R topics documented: RoxygenNote 7.1.0 Encoding UTF-8 R topics documented: arguments . 15 as.array.MXNDArray . 15 as.matrix.MXNDArray . 16 children . 16 ctx.............................................. 16 dim.MXNDArray . 17 graph.viz . 17 im2rec . 18 internals . 19 is.mx.context . 19 is.mx.dataiter . 20 is.mx.ndarray . 20 is.mx.symbol . 21 is.serialized . 21 length.MXNDArray . 21 mx.apply . 22 mx.callback.early.stop . 22 mx.callback.log.speedometer . 23 mx.callback.log.train.metric . 23 mx.callback.save.checkpoint . 24 mx.cpu . 24 mx.ctx.default . 24 mx.exec.backward . 25 mx.exec.forward . 25 mx.exec.update.arg.arrays . 26 mx.exec.update.aux.arrays . 26 mx.exec.update.grad.arrays .
    [Show full text]
  • Jssjournal of Statistical Software MMMMMM YYYY, Volume VV, Issue II
    JSSJournal of Statistical Software MMMMMM YYYY, Volume VV, Issue II. http://www.jstatsoft.org/ OSTSC: Over Sampling for Time Series Classification in R Matthew Dixon Diego Klabjan Lan Wei Stuart School of Business, Department of Industrial Department of Computer Science, Illinois Institute of Technology Engineering and Illinois Institute of Technology Management Sciences, Northwestern University Abstract The OSTSC package is a powerful oversampling approach for classifying univariant, but multinomial time series data in R. This article provides a brief overview of the over- sampling methodology implemented by the package. A tutorial of the OSTSC package is provided. We begin by providing three test cases for the user to quickly validate the functionality in the package. To demonstrate the performance impact of OSTSC,we then provide two medium size imbalanced time series datasets. Each example applies a TensorFlow implementation of a Long Short-Term Memory (LSTM) classifier - a type of a Recurrent Neural Network (RNN) classifier - to imbalanced time series. The classifier performance is compared with and without oversampling. Finally, larger versions of these two datasets are evaluated to demonstrate the scalability of the package. The examples demonstrate that the OSTSC package improves the performance of RNN classifiers applied to highly imbalanced time series data. In particular, OSTSC is observed to increase the AUC of LSTM from 0.543 to 0.784 on a high frequency trading dataset consisting of 30,000 time series observations. arXiv:1711.09545v1 [stat.CO] 27 Nov 2017 Keywords: Time Series Oversampling, Classification, Outlier Detection, R. 1. Introduction A significant number of learning problems involve the accurate classification of rare events or outliers from time series data.
    [Show full text]
  • Verification of RNN-Based Neural Agent-Environment Systems
    The Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19) Verification of RNN-Based Neural Agent-Environment Systems Michael E. Akintunde, Andreea Kevorchian, Alessio Lomuscio, Edoardo Pirovano Department of Computing, Imperial College London, UK Abstract and realised via a previously trained recurrent neural net- work (Hausknecht and Stone 2015). As we discuss below, We introduce agent-environment systems where the agent we are not aware of other methods in the literature address- is stateful and executing a ReLU recurrent neural network. ing the formal verification of systems based on recurrent We define and study their verification problem by providing networks. The present contribution focuses on reachability equivalences of recurrent and feed-forward neural networks on bounded execution traces. We give a sound and complete analysis, that is we intend to give formal guarantees as to procedure for their verification against properties specified whether a state, or a set of states are reached in the system. in a simplified version of LTL on bounded executions. We Typically this is an unwanted state, or a bug, that we intend present an implementation and discuss the experimental re- to ensure is not reached in any possible execution. Similarly, sults obtained. to provide safety guarantees, this analysis can be used to show the system does not enter an unsafe region of the state space. 1 Introduction The rest of the paper is organised as follows. After the A key obstacle in the deployment of autonomous systems is discussion of related work below, in Section 2, we introduce the inherent difficulty of their verification. This is because the basic concepts related to neural networks that are used the behaviour of autonomous systems is hard to predict; in throughout the paper.
    [Show full text]
  • Anurag Soni's Blog
    ANURAG SONI 447 Sharon Rd Pittsburgh, PA, USA |[email protected] | 765-775-0116 | GitHub | Blog | LinkedIn PROFILE • MS Business Analytics student seeking position in data science and decision-making business units • 5 years of experience as Analyst/Product Mgr. for a technology consulting firm with Investment Banking clients • Skills in figuring out impactful stories with data EDUCATION Purdue University, Krannert School of Management West Lafayette, IN Master of Science in Business Analytics and Information Management May 2018 Indian Institute of Technology Guwahati Guwahati, INDIA Bachelor of Technology, Chemical Engineering May 2011 HIGHLIGHTS • Tools: PL/SQL, Python, R, Excel(VBA), SAS, MATLAB, Tableau, Gurobi • Environment: TensorFlow, Keras, Hadoop, Hive, Docker, Google Cloud, UNIX • Process: Data Mining, SDLC, Process Simulation, Optimization, Data Modeling • Skills: Client-Facing, Strong Communication, Change Management, Team Management, Mentoring EXPERIENCE Purdue BIZ Analytics LAB West Lafayette, IN Associate Analyst(Co-Op) July 2017 – Mar 2018 • Demand Forecasting of supermarket sales: Built a production ready machine-learning solution using - Python deep learning libraries with results expecting to save significant inventory costs • Delivery Rebate Forecast for retailers: Built a forecasting model that supports reporting of weekly expected rebate numbers in an annual tiered delivery contract. • Decision Support System for Music Producer: Implemented dashboard application that uses Spotify datasets to partially cluster songs by their popularity and revenue capability • Production implementation of ML solution: Created an automated machine learning pipeline of an object- detection solution built using TensorFlow on Google Cloud using Docker and Kubernetes technology • Hate Speech Recognition using NLP: Implemented a text mining model that identifies hate speech characteristics in the text.
    [Show full text]
  • Deep Learning Architectures for Sequence Processing
    Speech and Language Processing. Daniel Jurafsky & James H. Martin. Copyright © 2021. All rights reserved. Draft of September 21, 2021. CHAPTER Deep Learning Architectures 9 for Sequence Processing Time will explain. Jane Austen, Persuasion Language is an inherently temporal phenomenon. Spoken language is a sequence of acoustic events over time, and we comprehend and produce both spoken and written language as a continuous input stream. The temporal nature of language is reflected in the metaphors we use; we talk of the flow of conversations, news feeds, and twitter streams, all of which emphasize that language is a sequence that unfolds in time. This temporal nature is reflected in some of the algorithms we use to process lan- guage. For example, the Viterbi algorithm applied to HMM part-of-speech tagging, proceeds through the input a word at a time, carrying forward information gleaned along the way. Yet other machine learning approaches, like those we’ve studied for sentiment analysis or other text classification tasks don’t have this temporal nature – they assume simultaneous access to all aspects of their input. The feedforward networks of Chapter 7 also assumed simultaneous access, al- though they also had a simple model for time. Recall that we applied feedforward networks to language modeling by having them look only at a fixed-size window of words, and then sliding this window over the input, making independent predictions along the way. Fig. 9.1, reproduced from Chapter 7, shows a neural language model with window size 3 predicting what word follows the input for all the. Subsequent words are predicted by sliding the window forward a word at a time.
    [Show full text]
  • Deep Learning for Source Code Modeling and Generation: Models, Applications and Challenges
    Deep Learning for Source Code Modeling and Generation: Models, Applications and Challenges TRIET H. M. LE, The University of Adelaide HAO CHEN, The University of Adelaide MUHAMMAD ALI BABAR, The University of Adelaide Deep Learning (DL) techniques for Natural Language Processing have been evolving remarkably fast. Recently, the DL advances in language modeling, machine translation and paragraph understanding are so prominent that the potential of DL in Software Engineering cannot be overlooked, especially in the field of program learning. To facilitate further research and applications of DL in this field, we provide a comprehensive review to categorize and investigate existing DL methods for source code modeling and generation. To address the limitations of the traditional source code models, we formulate common program learning tasks under an encoder-decoder framework. After that, we introduce recent DL mechanisms suitable to solve such problems. Then, we present the state-of-the-art practices and discuss their challenges with some recommendations for practitioners and researchers as well. CCS Concepts: • General and reference → Surveys and overviews; • Computing methodologies → Neural networks; Natural language processing; • Software and its engineering → Software notations and tools; Source code generation; Additional Key Words and Phrases: Deep learning, Big Code, Source code modeling, Source code generation 1 INTRODUCTION Deep Learning (DL) has recently emerged as an important branch of Machine Learning (ML) because of its incredible performance in Computer Vision and Natural Language Processing (NLP) [75]. In the field of NLP, it has been shown that DL models can greatly improve the performance of many classic NLP tasks such as semantic role labeling [96], named entity recognition [209], machine translation [256], and question answering [174].
    [Show full text]
  • Julia: Next Generation Language Scipy 2019, Universidad De Los Andes
    Julia: Next Generation Language SciPy 2019, Universidad de los Andes Let us understand the necessity: Scientific AI Machine Learning + Domain Power modeling AI & Robotics Team Kiwibot Do I have to learn another language? Current situation: I'm here talking to experts in the scientific python community At the moment we have the two language problem in Scientific AI When I was TA in tools for computing: ● NumPy for numerical linear algebra. ● R for statistical analysis. ● C for the fast stuff ● Bash to tie all up A.k.a "Ousterholt's Dichotomy" "System languages" "Scripting languages" ● Static ● Dynamic ● Compiled ● Interpreted ● User types ● Standard types AI & Robotics Team Kiwibot The Two Language Problem ? Because of this dichotomy, a two-tier compromise is standard: ● For convenience, use a a scripting language (Matlab, R, Python) ● But do all the hard stuff in a system language (C, C++, Fortran) Pragmatic for many applications, but has drawbacks ● Aren't the hard parts exactly where you need an easier language? ● Forces vectorization everywhere, even when awkward or wasterful ● Creates a social barrier - a wall between users and developers @ucb_notanartist Enter The Julian Unification ● Dynamic ● Compiled ● User Types and standard types ● Standalone or glue AI & Robotics Team Kiwibot Julia is Fast!! Let us understand the necessity: Scientific AI Machine Learning + Domain Power modeling AI + Robotics Team Kiwibot Mechanistics vs Non-Mechanistics Models Let's say we want to define a nonlinear model for the population of rabbits. There are three basic ways to do this: Rabbits tomorrow = Model(Rabbits today) ● Analytical solutions require that someone explicitly define a nonlinear model.
    [Show full text]
  • Comparative Analysis of Recurrent Neural Network Architectures for Reservoir Inflow Forecasting
    water Article Comparative Analysis of Recurrent Neural Network Architectures for Reservoir Inflow Forecasting Halit Apaydin 1 , Hajar Feizi 2 , Mohammad Taghi Sattari 1,2,* , Muslume Sevba Colak 1 , Shahaboddin Shamshirband 3,4,* and Kwok-Wing Chau 5 1 Department of Agricultural Engineering, Faculty of Agriculture, Ankara University, Ankara 06110, Turkey; [email protected] (H.A.); [email protected] (M.S.C.) 2 Department of Water Engineering, Agriculture Faculty, University of Tabriz, Tabriz 51666, Iran; [email protected] 3 Department for Management of Science and Technology Development, Ton Duc Thang University, Ho Chi Minh City, Vietnam 4 Faculty of Information Technology, Ton Duc Thang University, Ho Chi Minh City, Vietnam 5 Department of Civil and Environmental Engineering, Hong Kong Polytechnic University, Hong Kong, China; [email protected] * Correspondence: [email protected] or [email protected] (M.T.S.); [email protected] (S.S.) Received: 1 April 2020; Accepted: 21 May 2020; Published: 24 May 2020 Abstract: Due to the stochastic nature and complexity of flow, as well as the existence of hydrological uncertainties, predicting streamflow in dam reservoirs, especially in semi-arid and arid areas, is essential for the optimal and timely use of surface water resources. In this research, daily streamflow to the Ermenek hydroelectric dam reservoir located in Turkey is simulated using deep recurrent neural network (RNN) architectures, including bidirectional long short-term memory (Bi-LSTM), gated recurrent unit (GRU), long short-term memory (LSTM), and simple recurrent neural networks (simple RNN). For this purpose, daily observational flow data are used during the period 2012–2018, and all models are coded in Python software programming language.
    [Show full text]
  • Transformations)
    TRANSFORMACJE (TRANSFORMATIONS) Transformacje (Transformations) is an interdisciplinary refereed, reviewed journal, published since 1992. The journal is devoted to i.a.: civilizational and cultural transformations, information (knowledge) societies, global problematique, sustainable development, political philosophy and values, future studies. The journal's quasi-paradigm is TRANSFORMATION - as a present stage and form of development of technology, society, culture, civilization, values, mindsets etc. Impacts and potentialities of change and transition need new methodological tools, new visions and innovation for theoretical and practical capacity-building. The journal aims to promote inter-, multi- and transdisci- plinary approach, future orientation and strategic and global thinking. Transformacje (Transformations) are internationally available – since 2012 we have a licence agrement with the global database: EBSCO Publishing (Ipswich, MA, USA) We are listed by INDEX COPERNICUS since 2013 I TRANSFORMACJE(TRANSFORMATIONS) 3-4 (78-79) 2013 ISSN 1230-0292 Reviewed journal Published twice a year (double issues) in Polish and English (separate papers) Editorial Staff: Prof. Lech W. ZACHER, Center of Impact Assessment Studies and Forecasting, Kozminski University, Warsaw, Poland ([email protected]) – Editor-in-Chief Prof. Dora MARINOVA, Sustainability Policy Institute, Curtin University, Perth, Australia ([email protected]) – Deputy Editor-in-Chief Prof. Tadeusz MICZKA, Institute of Cultural and Interdisciplinary Studies, University of Silesia, Katowice, Poland ([email protected]) – Deputy Editor-in-Chief Dr Małgorzata SKÓRZEWSKA-AMBERG, School of Law, Kozminski University, Warsaw, Poland ([email protected]) – Coordinator Dr Alina BETLEJ, Institute of Sociology, John Paul II Catholic University of Lublin, Poland Dr Mirosław GEISE, Institute of Political Sciences, Kazimierz Wielki University, Bydgoszcz, Poland (also statistical editor) Prof.
    [Show full text]