DOCSLIB.ORG

Transfer-Entropy-Regularized Markov Decision Processes Takashi Tanaka1 Henrik Sandberg2 Mikael Skoglund3

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Transfer-Entropy-Regularized Markov Decision Processes Takashi Tanaka1 Henrik Sandberg2 Mikael Skoglund3 1 Transfer-Entropy-Regularized Markov Decision Processes Takashi Tanaka1 Henrik Sandberg2 Mikael Skoglund3 Abstract—We consider the framework of transfer-entropy- can be used as a proxy for the data rate on communication regularized Markov Decision Process (TERMDP) in which the channels, and thus solving TERMDP provides a fundamental weighted sum of the classical state-dependent cost and the performance limitation of such systems. The second applica- transfer entropy from the state random process to the control random process is minimized. Although TERMDPs are generally tion of TERMDP is non-equilibrium thermodynamics. There formulated as nonconvex optimization problems, we derive an has been renewed interests in the generalized second law of analytical necessary optimality condition expressed as a finite set thermodynamics, in which transfer entropy arises as a key of nonlinear equations, based on which an iterative forward- concept [10]. TERMDP in this context can be interpreted as backward computational procedure similar to the Arimoto- the problem of operating thermal engines at a nonzero work Blahut algorithm is proposed. It is shown that every limit point of the sequence generated by the proposed algorithm is a stationary rate near the fundamental limitation of the second law of point of the TERMDP. Applications of TERMDPs are discussed thermodynamics. in the context of networked control systems theory and non- In contrast to the standard MDP [11], TERMDP penalizes equilibrium thermodynamics. The proposed algorithm is applied the information flow from the underlying state random pro- to an information-constrained maze navigation problem, whereby cess to the control random process. Consequently, TERMDP we study how the price of information qualitatively alters the optimal decision polices. promotes “information-frugal” decision policies, under which control actions tend to be statistically less dependent on the underlying Markovian state dynamics. This is often a favorable I. INTRODUCTION property in various real-time decision-making scenarios (for both humans and robots) in which information acquisition, Transfer entropy [1] is a quantity that can be understood as processing, and transmission are costly operations. Therefore, a measure of information flow between random processes. It is it is expected that TERMDP plays major roles in broader a generalization of directed information, a concept proposed in contexts beyond the aforementioned applications, although the the information theory literature for the analysis of communi- interpretations of transfer entropy in each application must be cation systems with feedback [2]–[4]. Closely related concepts carefully discussed. include the KL-causality measure [5], which was originally In the literature, a few alternative approaches have been introduced in the economic statistics literature for the causality 1 suggested to apply information-theoretic cost functions to analysis. Recently, these concepts have been applied in a capture decision-making costs in MDPs. Similarities and dif- broad range of academic disciplines, including neuroscience ferences between TERMDP and the existing problem for- [7], finance [8], and social science [9]. mulations are noteworthy. The rationally inattentive control In this paper, we formulate the problem of transfer-entropy- problem [12], [13] has been motivated in a macroeconomic regularized Markov Decision Process (TERMDP), and de- context, where Shannon’s mutual information (a special case velop a numerical solution algorithm. TERMDP is an optimal of transfer entropy) is adopted as an attention cost for decision- control problem in which we seek a causal decision-making makers. The authors of [14]–[16] present a class of optimal policy that minimizes the weighted sum of the classical state- control problems in which control costs are modeled as the arXiv:1708.09096v3 [math.OC] 27 May 2020 dependent cost and transfer entropy from the state random Kullback-Leibler (KL) divergence from the “uncontrolled” process to the control actions. As we will discuss in the se- state trajectories to the “control” state trajectories. Alternative quel, TERMDP predicts a fundamental performance limitation information-theoretic decision costs in dynamic environments of feedback control systems from an information-theoretic include predictive information [17], past-future information- perspective. The first context in which TERMDP naturally bottleneck [18], and information-to-go [19], [20]. Information- arises is networked control systems theory, where the trade-off theoretic bounded rationality and its analogy to thermody- between the best achievable control performance and the data namics are discussed in [21]. While intuitively plausible, rate at which sensor information is fed back to the controller is some of these problem formulations lack physical (or coding- a central question. Prior work has shown that transfer entropy theoretic) justifications, unlike TERMDP, whose operational interpretation can be found in the aforementioned contexts. 1University of Texas at Austin, USA, [email protected]; 2KTH Royal Institute of Technology, Sweden, [email protected]; 3KTH Royal Insti- An equivalent problem formulation to TERMDP first ap- tute of Technology, Sweden, [email protected]. peared in [22] and [23], where the problem was formulated in a 1Sometimes (e.g., in statistical physics [6]), transfer entropy is used as a general (Polish state space) setup. Linear-Quadratic-Gaussian synonym for directed information. It appears that the concepts of transfer entropy [1], directed information [2], [3], and Kullback causality measure [5] (LQG) control with minimum directed information [24] is a were introduced independently. version of TERMDP specialized to the LQG regime. While 2 the problem in the LQG setup was shown to be tractable distribution: t t−1 by semidefinite programming [24], algorithmic aspects of qt(utjx ; u ): (1) TERMDP beyond the LQG regime have not been thoroughly studied. Therefore, the primary goal of this paper is to provide The joint distribution of the state and control trajectories is de- µ (xt+1; ut) an efficient computational algorithm to find a stationary point noted by t+1 , which is uniquely determined by the µ (x ) (an optimal solution candidate) of the given TERMDP. The initial state distribution 1 1 , the state transition probability p (x jx ; u ) q (u jxt; ut−1) contributions of this paper are as follows: t+1 t+1 t t and the decision policy t t by a recursive formula • We derive a necessary optimality condition expressed as t+1 t a set of nonlinear equations involving a finite number µt+1(x ; u ) of variables. This result recovers, and partly strengthens, t t−1 t t−1 = pt+1(xt+1jxt; ut)qt(utjx ; u )µt(x ; u ): (2) results obtained in prior work [22]. • We propose a forward-backward iterative algorithm that A stage-additive cost functional can be viewed as a generalization of the Arimoto-Blahut T T +1 T X algorithm [25], [26] to solve the optimality condition J(X ;U ) , Ect(Xt;Ut) + EcT +1(XT +1) (3) numerically. t=1 The proposed algorithm is the first application of the the is a function of random variables XT +1 and U T with a Arimoto-Blahut algorithm for transfer entropy minimization. T +1 T joint distribution µT +1(x ; u ). Transfer entropy is an Our algorithm should be compared with the generalized information-theoretic quantity defined as follows: Arimoto-Blahut algorithm for transfer entropy maximization Definition 1: For nonnegative integers m and n, the transfer proposed in [27]. The algorithm in [27] can be viewed as entropy of degree (m; n) is defined by a generalization of the Arimoto-Blahut “capacity algorithm” T t t−1 in [25], while our proposed algorithm can be viewed as a X µt+1(UtjX ;U ) I (XT ! U T ) log t−m t−n (4) generalization of the Arimoto-Blahut “rate-distortion algo- m;n , E t−1 µt+1(UtjUt−n) rithm” in [25]. Unfortunately, we discover that the proposed t=1 T t t−1 algorithm may not converge to the global minimum due to X X t t µt+1(utjxt−m; ut−n) = µt+1(x ; u ) log : the non-convex nature of TERMDP. This result is somewhat µ (u jut−1 ) t=1 xt2X t;ut2U t t+1 t t−n surprising as the global convergence of the original Arimoto- Blahut rate-distortion algorithm, which is a special case of Using conditional mutual information [28], transfer entropy our algorithm, is well-known. Nevertheless, observing that the can also be written as proposed algorithm belongs to the class of block coordinate T descent (BCD) algorithms, we show that every limit point T T X t t−1 Im;n(X ! U ) , I(Xt−m; UtjUt−n): (5) generated by the algorithm is guaranteed to be a stationary t=1 point of the given TERMDP. When m = 1 and n = 1,(4) coincides with directed Organization of the paper: The problem formulation of information [3]: TERMDP is formally introduced in SectionII. Mathemati- cal preliminaries are summarized in Section III. SectionIV T T T X t t−1 presents the main results. Derivation of the main results are I(X ! U ) , I(X ; UtjU ): (6) summarized in SectionV. SectionVI discusses applications of t=1 the TERMDP framework. A numerical demonstration of the The main problem studied in this paper is now formulated as proposed algorithm is presented in Section VII. We conclude follows. with a list of future work in Section VIII. Problem 1: (TERMDP) Let the initial state distribution Notation: Upper case symbols such as X are used to µ1(x1) and the state transition probability pt+1(xt+1jxt; ut) t+1 t represent random variables, while lower case symbols such be given, and assume that the joint distribution µt+1(x ; u ) as x are used to represent a specific realization. Notation is recursively given by (2). For a fixed constant β ≥ 0, the l t xk , (xk; xk+1; :::; xl) and x , (x1; x2; :::; xt) will be Transfer-Entropy-Regularized Markov Decision Processes is used to specify subsequences.
Load more

Recommended publications
  • JIDT: an Information-Theoretic Toolkit for Studying the Dynamics of Complex Systems
    JIDT: An information-theoretic toolkit for studying the dynamics of complex systems Joseph T. Lizier1, 2, ∗ 1Max Planck Institute for Mathematics in the Sciences, Inselstraße 22, 04103 Leipzig, Germany 2CSIRO Digital Productivity Flagship, Marsfield, NSW 2122, Australia (Dated: December 4, 2014) Complex systems are increasingly being viewed as distributed information processing systems, particularly in the domains of computational neuroscience, bioinformatics and Artificial Life. This trend has resulted in a strong uptake in the use of (Shannon) information-theoretic measures to analyse the dynamics of complex systems in these fields. We introduce the Java Information Dy- namics Toolkit (JIDT): a Google code project which provides a standalone, (GNU GPL v3 licensed) open-source code implementation for empirical estimation of information-theoretic measures from time-series data. While the toolkit provides classic information-theoretic measures (e.g. entropy, mutual information, conditional mutual information), it ultimately focusses on implementing higher- level measures for information dynamics. That is, JIDT focusses on quantifying information storage, transfer and modification, and the dynamics of these operations in space and time. For this purpose, it includes implementations of the transfer entropy and active information storage, their multivariate extensions and local or pointwise variants. JIDT provides implementations for both discrete and continuous-valued data for each measure, including various types of estimator for continuous data (e.g. Gaussian, box-kernel and Kraskov-St¨ogbauer-Grassberger) which can be swapped at run-time due to Java's object-oriented polymorphism. Furthermore, while written in Java, the toolkit can be used directly in MATLAB, GNU Octave, Python and other environments. We present the principles behind the code design, and provide several examples to guide users.
    [Show full text]
  • Exploratory Causal Analysis in Bivariate Time Series Data
    Exploratory Causal Analysis in Bivariate Time Series Data Abstract Many scientific disciplines rely on observational data of systems for which it is difficult (or impossible) to implement controlled experiments and data analysis techniques are required for identifying causal information and relationships directly from observational data. This need has lead to the development of many different time series causality approaches and tools including transfer entropy, convergent cross-mapping (CCM), and Granger causality statistics. A practicing analyst can explore the literature to find many proposals for identifying drivers and causal connections in times series data sets, but little research exists of how these tools compare to each other in practice. This work introduces and defines exploratory causal analysis (ECA) to address this issue. The motivation is to provide a framework for exploring potential causal structures in time series data sets. J. M. McCracken Defense talk for PhD in Physics, Department of Physics and Astronomy 10:00 AM November 20, 2015; Exploratory Hall, 3301 Advisor: Dr. Robert Weigel; Committee: Dr. Paul So, Dr. Tim Sauer J. M. McCracken (GMU) ECA w/ time series causality November 20, 2015 1 / 50 Exploratory Causal Analysis in Bivariate Time Series Data J. M. McCracken Department of Physics and Astronomy George Mason University, Fairfax, VA November 20, 2015 J. M. McCracken (GMU) ECA w/ time series causality November 20, 2015 2 / 50 Outline 1. Motivation 2. Causality studies 3. Data causality 4. Exploratory causal analysis 5. Making an ECA summary Transfer entropy difference Granger causality statistic Pairwise asymmetric inference Weighed mean observed leaning Lagged cross-correlation difference 6.
    [Show full text]
  • Empirical Analysis Using Symbolic Transfer Entropy
    University of South Florida Scholar Commons Graduate Theses and Dissertations Graduate School June 2019 Measuring Influence Across Social Media Platforms: Empirical Analysis Using Symbolic Transfer Entropy Abhishek Bhattacharjee University of South Florida, [email protected] Follow this and additional works at: https://scholarcommons.usf.edu/etd Part of the Computer Sciences Commons Scholar Commons Citation Bhattacharjee, Abhishek, "Measuring Influence Across Social Media Platforms: Empirical Analysis Using Symbolic Transfer Entropy" (2019). Graduate Theses and Dissertations. https://scholarcommons.usf.edu/etd/7745 This Thesis is brought to you for free and open access by the Graduate School at Scholar Commons. It has been accepted for inclusion in Graduate Theses and Dissertations by an authorized administrator of Scholar Commons. For more information, please contact [email protected]. Measuring Influence Across Social Media Platforms: Empirical Analysis Using Symbolic Transfer Entropy by Abhishek Bhattacharjee A thesis submitted in partial fulfillment of the requirements for the degree of Master of Science Department of Computer Science and Engineering College of Engineering University of South Florida Major Professor: Adriana Iamnitchi, Ph.D. John Skvoretz, Ph.D. Giovanni Luca Ciampaglia, Ph.D. Date of Approval: April 29, 2019 Keywords: Social Networks, Cross-platform influence Copyright © 2019, Abhishek Bhattacharjee DEDICATION Dedicated to my parents, brother, girlfriend and those five humans. ACKNOWLEDGMENTS I would like to pay my thanks to my advisor Dr Adriana Iamnitchi for her constant support and feedback throughout this project. The work wouldn’t have been possible without Leidos who provided us the data, and DARPA for funding this research work. I would also like to acknowledge the contributions made by Dr.
    [Show full text]
  • Entropic Measures of Connectivity with an Application to Intracerebral Epileptic Signals Jie Zhu
    Entropic measures of connectivity with an application to intracerebral epileptic signals Jie Zhu To cite this version: Jie Zhu. Entropic measures of connectivity with an application to intracerebral epileptic signals. Signal and Image processing. Université Rennes 1, 2016. English. NNT : 2016REN1S006. tel-01359072 HAL Id: tel-01359072 https://tel.archives-ouvertes.fr/tel-01359072 Submitted on 1 Sep 2016 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. ANNÉE 2016 THÈSE / UNIVERSITÉ DE RENNES 1 sous le sceau de l’Université Bretagne Loire pour le grade de DOCTEUR DE L’UNIVERSITÉ DE RENNES 1 Mention : Traitement du Signal et Télécommunications Ecole doctorale MATISSE Ecole Doctorale Mathématiques, Télécommunications, Informatique, Signal, Systèmes, Electronique Jie ZHU Préparée à l’unité de recherche LTSI - INSERM UMR 1099 Laboratoire Traitement du Signal et de l’Image ISTIC UFR Informatique et Électronique Thèse soutenue à Rennes Entropic Measures of le 22 juin 2016 Connectivity with an devant le jury composé de : MICHEL Olivier Application to Professeur
    [Show full text]
  • An Information-Theoretic Perspective on Credit Assignment in Reinforcement Learning
    An Information-Theoretic Perspective on Credit Assignment in Reinforcement Learning Dilip Arumugam Peter Henderson Department of Computer Science Department of Computer Science Stanford University Stanford University [email protected] [email protected] Pierre-Luc Bacon Mila - University of Montreal [email protected] Abstract How do we formalize the challenge of credit assignment in reinforcement learn- ing? Common intuition would draw attention to reward sparsity as a key contribu- tor to difficult credit assignment and traditional heuristics would look to temporal recency for the solution, calling upon the classic eligibility trace. We posit that it is not the sparsity of the reward itself that causes difficulty in credit assignment, but rather the information sparsity. We propose to use information theory to define this notion, which we then use to characterize when credit assignment is an ob- stacle to efficient learning. With this perspective, we outline several information- theoretic mechanisms for measuring credit under a fixed behavior policy, high- lighting the potential of information theory as a key tool towards provably-efficient credit assignment. 1 Introduction The credit assignment problem in reinforcement learning [Minsky, 1961, Sutton, 1985, 1988] is concerned with identifying the contribution of past actions on observed future outcomes. Of par- ticular interest to the reinforcement-learning (RL) problem [Sutton and Barto, 1998] are observed future returns and the value function, which quantitatively answers “how does choosing an action a in state s affect future return?” Indeed, given the challenge of sample-efficient RL in long-horizon, sparse-reward tasks, many approaches have been developed to help alleviate the burdens of credit assignment [Sutton, 1985, 1988, Sutton and Barto, 1998, Singh and Sutton, 1996, Precup et al., 2000, Riedmiller et al., 2018, Harutyunyan et al., 2019, Hung et al., 2019, Arjona-Medina et al., 2019, Ferret et al., 2019, Trott et al., 2019, van Hasselt et al., 2020].
    [Show full text]
  • Fast and Effective Pseudo Transfer Entropy for Bivariate Data-Driven
    www.nature.com/scientificreports OPEN Fast and efective pseudo transfer entropy for bivariate data‑driven causal inference Riccardo Silini* & Cristina Masoller Identifying, from time series analysis, reliable indicators of causal relationships is essential for many disciplines. Main challenges are distinguishing correlation from causality and discriminating between direct and indirect interactions. Over the years many methods for data‑driven causal inference have been proposed; however, their success largely depends on the characteristics of the system under investigation. Often, their data requirements, computational cost or number of parameters limit their applicability. Here we propose a computationally efcient measure for causality testing, which we refer to as pseudo transfer entropy (pTE), that we derive from the standard defnition of transfer entropy (TE) by using a Gaussian approximation. We demonstrate the power of the pTE measure on simulated and on real‑world data. In all cases we fnd that pTE returns results that are very similar to those returned by Granger causality (GC). Importantly, for short time series, pTE combined with time‑shifted (T‑S) surrogates for signifcance testing strongly reduces the computational cost with respect to the widely used iterative amplitude adjusted Fourier transform (IAAFT) surrogate testing. For example, for time series of 100 data points, pTE and T‑S reduce the computational time by 82% with respect to GC and IAAFT. We also show that pTE is robust against observational noise. Therefore, we argue that the causal inference approach proposed here will be extremely valuable when causality networks need to be inferred from the analysis of a large number of short time series.
    [Show full text]
  • Task-Driven Estimation and Control Via Information Bottlenecks
    Task-Driven Estimation and Control via Information Bottlenecks Vincent Pacelli and Anirudha Majumdar Abstract— Our goal is to develop a principled and general algorithmic framework for task-driven estimation and control for robotic systems. State-of-the-art approaches for controlling robotic systems typically rely heavily on accurately estimating the full state of the robot (e.g., a running robot might estimate joint angles and velocities, torso state, and position relative to a goal). However, full state representations are often excessively rich for the specific task at hand and can lead to significant Fig. 1. A schematic of our technical approach. We seek to synthesize computational inefficiency and brittleness to errors in state (offline) a minimalistic set of task-relevant variables (TRVs) x~t that create x u estimation. In contrast, we present an approach that eschews a bottleneck between the full state t and the control input t. These TRVs π such rich representations and seeks to create task-driven repre- are estimated online in order to apply the policy t. We demonstrate our approach on a spring-loaded inverted pendulum model whose goal is to sentations. The key technical insight is to leverage the theory of run to a target location. Our approach automatically synthesizes a one- information bottlenecks to formalize the notion of a “task-driven dimensional TRV x~ sufficient for achieving this task. representation” in terms of information theoretic quantities that t measure the minimality of a representation. We propose novel estimated. Second, since only a few prominent variables need iterative algorithms for automatically synthesizing (offline) a to be estimated, fewer sources of measurement uncertainty task-driven representation (given in terms of a set of task- relevant variables (TRVs)) and a performant control policy that result in a more robust policy.
    [Show full text]
  • TRENTOOL 3.3.1 Beta – User Documentation
    TRENTOOL 3.3.1 beta – User Documentation Patricia Wollstadt∗ Michael Lindner Raul Vicente Michael Wibral Nicu Pampu Mario Martinez-Zarzuela Version 0.93 http://www.trentool.de/ Contents 1 Introduction 4 2 Installation and configuration6 3 Background 7 3.1 Transfer entropy....................................7 3.2 Notation and preliminaries..............................7 3.3 Practical TE estimation in TRENTOOL.......................8 3.4 Additional functionalities implemented in TRENTOOL.............. 11 3.4.1 Reconstruction of interaction delays..................... 11 3.4.2 Estimating TE from an ensemble of time series............... 11 3.4.3 Group analysis................................. 12 3.4.4 Graph correction for multivariate effects................... 12 4 Using TRENTOOL 14 4.1 Overview........................................ 14 4.2 TRENTOOL core functions.............................. 16 4.2.1 TEprepare.m .................................. 18 4.2.2 TEsurrogatestats.m ............................. 18 4.3 Reconstruction of interaction delays......................... 19 4.3.1 InteractionDelayReconstruction_calculate.m ............. 21 4.3.2 InteractionDelayReconstruction_plotting.m .............. 22 4.4 Estimating time-resolved transfer entropy from an ensemble of time series.... 22 4.4.1 TEsurrogatestats_ensemble.m ....................... 24 4.4.2 Combining time-resolved transfer entropy estimation with interaction delay reconstruction................................. 25 4.5 Binomial Test for Single Subject Results......................
    [Show full text]
  • Direction of Information Flow in Large-Scale Resting-State
    Direction of information flow in large-scale resting- state networks is frequency-dependent Arjan Hillebranda,1, Prejaas Tewariea, Edwin van Dellena,b, Meichen Yua, Ellen W. S. Carboc, Linda Douwc,d, Alida A. Gouwa,e, Elisabeth C. W. van Straatena,f, and Cornelis J. Stama aDepartment of Clinical Neurophysiology and Magnetoencephalography Center, VU University Medical Center, 1081 HV Amsterdam, The Netherlands; bDepartment of Psychiatry, Brain Center Rudolf Magnus, University Medical Center Utrecht, 3508 GA Utrecht, The Netherlands; cDepartment of Anatomy and Neurosciences, VU University Medical Center, 1081 HV Amsterdam, The Netherlands; dDepartment of Radiology, Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, MA 02129; eAlzheimer Center and Department of Neurology, VU University Medical Center, 1081 HV Amsterdam, The Netherlands; and fNutricia Advanced Medical Nutrition, Nutricia Research, 3584 CT Utrecht, The Netherlands Edited by Marcus E. Raichle, Washington University in St. Louis, St. Louis, MO, and approved February 24, 2016 (received for review August 8, 2015) Normal brain function requires interactions between spatially sepa- front pattern has also been observed in EEG (26–28). An important rated, and functionally specialized, macroscopic regions, yet the advantage of MEG over EEG in this context is that it is reference- directionality of these interactions in large-scale functional networks free. Moreover, the large number of sensors (several hundreds) in is unknown. Magnetoencephalography was used to determine the modern whole-head MEG systems allow for sophisticated spatial directionality of these interactions, where directionality was inferred filtering approaches to accurately reconstruct time series of neuronal from time series of beamformer-reconstructed estimates of neuronal activation across the cortex (29, 30).
    [Show full text]
  • Interpretations of Directed Information in Portfolio Theory, Data
    1 Interpretations of Directed Information in Portfolio Theory, Data Compression, and Hypothesis Testing Haim H. Permuter, Young-Han Kim, and Tsachy Weissman Abstract We investigate the role of Massey’s directed information in portfolio theory, data compression, and statistics with causality constraints. In particular, we show that directed information is an upper bound on the increment in growth rates of optimal portfolios in a stock market due to causal side information. This upper bound is tight for gambling in a horse race, which is an extreme case of stock markets. Directed information also characterizes the value of causal side information in instantaneous compression and quantifies the benefit of causal inference in joint compression of two stochastic processes. In hypothesis testing, directed information evaluates the best error exponent for testing whether a random process Y causally influences another process X or not. These results give a natural interpretation of directed n n n information I(Y → X ) as the amount of information that a random sequence Y = (Y1,Y2,...,Yn) causally n provides about another random sequence X =(X1, X2,...,Xn). A new measure, directed lautum information, is also introduced and interpreted in portfolio theory, data compression, and hypothesis testing. Index Terms Causal conditioning, causal side information, directed information, hypothesis testing, instantaneous compression, Kelly gambling, Lautum information, portfolio theory. NTRODUCTION arXiv:0912.4872v1 [cs.IT] 24 Dec 2009 I. I Mutual information I(X; Y ) between two random variables X and Y arises as the canonical answer to a variety of questions in science and engineering. Most notably, Shannon [1] showed that the capacity C, the maximal data rate for reliable communication, of a discrete memoryless channel p(y|x) with input X and output Y is given by C = max I(X; Y ).
    [Show full text]
  • Causal Coupling Inference from Multivariate Time Seriesbasedonordinalpartitiontransitionnetworks
    CAUSAL COUPLING INFERENCE FROM MULTIVARIATE TIME SERIES BASED ON ORDINAL PARTITION TRANSITION NETWORKS Narayan Puthanmadam Subramaniyam Faculty of Medicine and Health Technology Tampere University [email protected] Reik V. Donner Magdeburg–Stendal University of Applied Sciences and Potsdam Institute for Climate Impact Research (PIK) – Member of the Leibniz Association Davide Caron Gabriella Pannucio Enhanced Regenerative Medicine Enhanced Regenerative Medicine Istituto Italiano di Tecnologia Istituto Italiano di Tecnologia Jari Hyttinen Faculty of Medicine and Health Technology Tampere University June 3, 2021 ABSTRACT Identifying causal relationships is a challenging yet crucial problem in many fields of science like epidemiology, climatology, ecology, genomics, economics and neuroscience, to mention only a few. Recent studies have demonstrated that ordinal partition transition networks (OPTNs) allow inferring the coupling direction between two dynamical systems. In this work, we generalize this concept to the study of the interactions among multiple dynamical systems and we propose a new method to de- tect causality in multivariate observational data. By applying this method to numerical simulations of coupled linear stochastic processes as well as two examples of interacting nonlinear dynamical systems (coupled Lorenz systems and a network of neural mass models), we demonstrate that our approach can reliably identify the direction of interactions and the associated coupling delays. Fi- arXiv:2010.00948v4 [stat.ME] 1 Jun 2021 nally, we study real-world observational microelectrode array electrophysiology data from rodent brain slices to identify the causal coupling structures underlying epileptiform activity. Our results, both from simulations and real-world data, suggest that OPTNs can provide a complementary and robust approach to infer causal effect networks from multivariate observational data.
    [Show full text]
  • Disentangling Causal Webs in the Brain Using Functional Magnetic Resonance Imaging: a Review of Current Approaches
    REVIEW Disentangling causal webs in the brain using functional magnetic resonance imaging: A review of current approaches Natalia Z. Bielczyk 1,2, Sebo Uithol 1,3, Tim van Mourik 1,2,PaulAnderson 1,4, Jeffrey C. Glennon1,2, and Jan K. Buitelaar 1,2 1Donders Institute for Brain, Cognition and Behavior, Nijmegen, the Netherlands 2Department of Cognitive Neuroscience, Radboud University Nijmegen Medical Centre, Nijmegen, the Netherlands 3Bernstein Centre for Computational Neuroscience, Charité Universitätsmedizin, Berlin, Germany 4Faculty of Science, Radboud University Nijmegen, Nijmegen, the Netherlands an open access journal Keywords: Causal inference, Effective connectivity, Functional Magnetic Resonance Imaging, Dynamic Causal Modeling, Granger Causality, Structural Equation Modeling, Bayesian Nets, Directed Acyclic Graphs, Pairwise inference, Large-scale brain networks ABSTRACT In the past two decades, functional Magnetic Resonance Imaging (fMRI) has been used to relate neuronal network activity to cognitive processing and behavior. Recently this approach has been augmented by algorithms that allow us to infer causal links between component populations of neuronal networks. Multiple inference procedures have been proposed to approach this research question but so far, each method has limitations when it comes to establishing whole-brain connectivity patterns. In this paper, we discuss eight ways to infer causality in fMRI research: Bayesian Nets, Dynamical Causal Modelling, Granger Citation: Bielczyk, N. Z., Uithol, S., Causality, Likelihood Ratios, Linear Non-Gaussian Acyclic Models, Patel’s Tau, Structural van Mourik, T., Anderson, P., Glennon, J. C., & Buitelaar, J. K. (2019). Equation Modelling, and Transfer Entropy. We finish with formulating some recommendations Disentangling causal webs in the brain using functional magnetic resonance for the future directions in this area.
    [Show full text]