Arxiv:1905.06216V2 [Cond-Mat.Stat-Mech] 19 Sep 2019 B

Information thermodynamics for interacting stochastic systems without bipartite structure

R. Chétrite,1 M.L. Rosinberg,2 T. Sagawa,3 and G. Tarjus2 1Laboratoire J.A. Dieudonné,UMR CNRS 7351, Universitéde Nice Sophia-Antipolis, Parc Valrose, 06108 Nice Cedex 02, France 2Sorbonne Université,CNRS, Laboratoire de Physique Théoriquede la Matière Condensée,LPTMC, F-75005 Paris, France 3Department of Applied Physics, School of Engineering, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656, Japan Fluctuations in biochemical networks, e.g., in a living cell, have a complex origin that precludes a description of such systems in terms of bipartite or multipartite processes, as is usually done in the framework of stochastic and/or information thermodynamics. This means that fluctuations in each subsystem are not independent: subsystems jump simultaneously if the dynamics is modeled as a Markov jump process, or noises are correlated for diffusion processes. In this paper, we consider information and thermodynamic exchanges between a pair of coupled systems that do not satisfy the bipartite property. The generalization of information-theoretic measures, such as learning rates and transfer entropy rates, to this situation is non-trivial and also involves introducing several additional rates. We describe how this can be achieved in the framework of general continuous- time Markov processes, without restricting the study to the steady-state regime. We illustrate our general formalism on the case of diffusion processes and derive an extension of the second law of information thermodynamics in which the difference of transfer entropy rates in the forward and backward time directions replaces the learning rate. As a side result, we also generalize an important relation linking information theory and estimation theory. To further obtain analytical expressions we treat in detail the case of Ornstein-Uhlenbeck processes, and discuss the ability of the various information measures to detect a directional coupling in the presence of correlated noises. Finally, we apply our formalism to the analysis of the directional influence between cellular processes in a concrete example, which also requires considering the case of a non-bipartite and non-Markovian process.

Contents

I. Introduction 2

II. Setup and brief reminder of the bipartite case 4 A. Setup 4 B. Deﬁnition of information measures 5 C. Inequalities and suﬃcient statistic 7 D. Entropy production and second law 7

III. Information measures for non-bipartite processes 8 A. Learning rates 8 B. Transfer entropy rates 10 C. Backward transfer entropy rates 11 D. Inequalities 12

IV. Markov diﬀusion processes and second law of information thermodynamics 14 A. Learning rates 14 arXiv:1905.06216v2 [cond-mat.stat-mech] 19 Sep 2019 B. Single-time-step transfer entropy rates 15 C. Multi-time-step transfer entropy rates 17 D. Second law for non-bipartite processes 17

V. Stationary bi-dimensional Ornstein-Uhlenbeck process 19 A. Expression of the multi-time-step TE rates 20 B. Numerical illustration 21 2

VI. Generalization to a non-Markovian process and application to the study of directional inﬂuence between cellular processes 23 A. Extension to a bivariate non-Markovian process 23 B. Application to the study of directional inﬂuence between cellular processes 24

VII. Summary 27

Acknowledgements 27

A. Expression of the learning rates for a pure jump Markov process in discrete space 27

B. Proof of various relations of the main text 27

C. Suﬃcient statistic for a non-bipartite dynamics 30

D. Analytical expressions for a stationary bi-dimensional Ornstein-Uhlenbeck process 30

E. Multi-time-step TE rates for the three-component process of Sec. VI B 35

References 37

I. INTRODUCTION

One recent and important field of application of information theory is biological systems, in particular gene regu- lation and signal transduction systems. Cells must sense, process and adapt to their environment or their own phys- iological state, which are noisy processes subjected to fluctuations (see e.g. [1]). In addition, information transfers consume energy, so that there is a competition between the information gain and the energy cost. This fundamental issue is the realm of information thermodynamics, a recent and active field of research, as reviewed in [2]. In this framework, the present work is motivated by the observation that there exist, broadly speaking, two different sources of fluctuations contributing to the stochasticity of biochemical processes, for instance in cell metabolic networks. The first one – commonly called “intrinsic”– is due the small numbers of biomolecules involved in a given reaction. The second one – the “extrinsic” source– arises from the heterogeneity in the physical environment of the cell and the occurrence of (many) other biochemical reactions (see, e.g., [3–11]). This implies that the noise in the input biochemical signal – to be detected– and the noise of the reactions that form the network are correlated. In short, stochastic noises have a nontrivial structure and fluctuations in each subsystem are not independent. In contrast, in the context of stochastic and information thermodynamics, the so-called bipartite assumption is usually made (e.g., for modeling Maxwell’s demons): one assumes that subsystems cannot jump simultaneously if the dynamics is modeled as a Markov jump process or that noises are uncorrelated if the dynamics is modeled as diffusion processes.This simplifies the theoretical analysis and allows the contribution of each components of the system to the entropy production to be clearly identified [12–28]. Although the abandon of the bipartite (or multipartite [22]) structure seriously complicates the interpretation of information and thermodynamics exchanges, our objective in the present work is to show that a detailed description is still available. The price to pay is that several information- theoretic measures must be added to those already introduced in the literature (information flow, aka learning rate, and transfer entropy), which characterize how information is exchanged between two interacting systems in the course of their dynamical evolution. In this paper, we will mostly consider non-equilibrium systems that can be modeled by continuous-time Markov processes (diffusions, jump processes, or both). It turns out however that many definitions or relations are also valid beyond the Markovian description and we will therefore provide a general framework. Moreover, in order to offer a sufficiently general perspective, we assume the presence of multiplicative noises (additive noises being regarded as a only special case) and we do not restrict the study to steady-state situations, as is often done. On the other hand, as far as stochastic thermodynamics is concerned, we only consider averaged quantities and do not derive fluctuation relations. We leave this important issue to future investigations. The main results of the paper can be summarized as follows: 1) We introduce a set of information-theoretic measures to consistently characterize information exchanges in a generic non-bipartite stochastic system composed of two interacting subsystems. This includes learning rates and transfer entropy rates. In particular, we define a learning rate that contains the footprints of time-irreversibility. To make the reading of the following sections easier, the definitions of all these information measures are listed in Table I. 3

2) We derive a set of inequalities among these quantities and show under which circumstances some inequalities become equalities, which may signal the presence of a so-called “sufficient statistic” [27, 29]. For Markov processes, we then propose a generalization of the notion of sensory capacity introduced in [23]. 3) We give explicit expressions of the information measures for Markov diffusion processes in terms of probability currents, diffusion tensor, and propagators. In passing, we obtain a generalization to non-bipartite systems of a classical result linking information theory and estimation theory [30, 31]. 5) We derive a generalized version of the so-called “second law of information thermodynamics” [2] that applies to non-bipartite diffusion processes, showing that the entropy production rate in one of the subsystems is lower bounded by the difference in transfer entropy rates in the forward and backward time directions. 6) We illustrate the behavior of the various information-theoretic measures in the case of a stationary bidimensional Ornstein-Uhlenbeck process and show their evolution as one systematically varies the parameter quantifying the correlation between the noises. 7) We apply our formalism to the study of the directional influence between cellular processes in the metabolism of E. coli [8, 11] and exhibit an intriguing feature that may indicate some optimality in the transmission of information.

Information measures Name Deﬁnition

+ 1 D P (Yt|Xt+h) E l (t) Forward learning rate (a.k.a. information ﬂow) ln | + X h P (Yt|Xt) 0

− 1 D P (Yt+h|Xt+h) E l (t) Backward learning rate ln | + X h P (Yt+h|Xt) 0

s s 1 + − lX (t) Symmetric learning rate lX (t) = 2 [lX (t) + lX (t)]

t t 1 D P (Yt+h|X0,Y0 ) E TX→Y (t) Transfer entropy (TE) rate h ln t |0+ P (Yt+h|Y0 )

1 D P (Yt+h|Xt,Yt) E T X→Y (t) Single-time-step TE rate ln | + h P (Yt+h|Yt) 0

T T † 1 D P (Yt|Xt+h,Yt+h) E TX→Y (t) Backward TE rate h ln T |0+ P (Yt|Yt+h)

† 1 D P (Yt|Xt+h,Yt+h) E T X→Y (t) Single-time-step backward TE rate ln | + h P (Yt|Yt+h) 0

t 1 D P (Xt+h,Yt+h|Y0 ) E TbX→Y (t) Filtered TE rate h ln t t |0+ P (Xt+h|Y0 )P (Yt+h|Y0 )

1 D P (Xt+h,Yt+h|Yt) E Tb X→Y (t) Single-time-step ﬁltered TE rate ln | + h P (Xt+h|Yt)P (Yt+h|Yt) 0

TABLE I: Deﬁnitions of information measures for a non-bipartite process Zt = (Xt,Yt). Similar quantities are deﬁned by exchanging X and Y . In the third column, it is implicit that the limit h → 0+ is taken.

More specifically, the paper is organized as follows. In Sec. II, we describe our general setup and briefly review the existing results for bipartite processes. To this aim, we first present the formal tools that will be used throughout the paper, in particular those related to continuous-time Markov processes. We then define the information-theoretic measures that are commonly considered in the framework of information thermodynamics and that satisfy some useful inequalities. We then recall the corresponding formulations of the second law. In Sec. III, the bipartite assumption is lifted and we introduce the relevant information measures. The usual inequalities are then generalized. In Sec. IV, to make all of the introduced definitions and relations more explicit, we focus on Markov diffusion processes. In Sec. V as a special case, we consider a stationary bidimensional Ornstein-Uhlenbeck process with additive noises for which a full analytical study can be carried out. This allows us to illustrate on a simple example the ability of the various information measures to infer a directional coupling in the underlying dynamics. Finally, in Sec. VI, we generalize the formalism to a class of non-Markovian processes and apply it to cellular processes in the metabolism of E. coli. This complements the previous experimental and theoretical investigations of Refs. [8, 11]. A brief summary is given in Sec. VII and some demonstrations and technical details are presented in Appendices. 4

II. SETUP AND BRIEF REMINDER OF THE BIPARTITE CASE

A. Setup

We are interested in the information and thermodynamic exchanges between two subsystems, denoted by X and Y , of a stochastic system Z whose microscopic states at time t are denoted by Zt = (Xt,Yt). The random variables X and Y may be multivariate (X and Y are then vectors), continuous or discrete, and live in arbitrary, and not necessarily identical, spaces. In the following, the full process Zt can be Markovian or non-Markovian, but even in the former case the individual dynamics of Xt and Yt, viewed as coarse-grained descriptions of Zt, are in general non-Markovian. When Zt is a continuous-time Markovian process [32–36], which may involve a combination of drift, diﬀusion, and 0 0 0 jump, the building block of its description is the transition probability P (Zt = z|Zt0 = z ), for t ≤ t and all z, z . Such 0 an object is generated by a kernel Lt(z, z ), called the Markovian generator, according to the forward Kolmogorov equation, Z d 0 00 00 0 00 P (Z = z|Z 0 = z ) = dz P (Z = z |Z 0 = z )L (z , z) , (1) dt t t t t t where dz00 is the appropriate measure for either continuous or discrete space. For pure jump processes in a discrete 0 space the Markovian generator is a matrix involving the transition rates Wt (z, z ) (where by convention the transition is from z to z0)

0 0 X 00 Lt (z, z ) = δz6=z0 Wt (z, z ) − δz=z0 Wt (z, z ) . (2) z006=z

On the other hand, pure diffusion processes in continuous space are usually described by the stochastic differential equations X dXt = FX,t (Zt) dt + σX,j,t (Zt) dWj,t j X dYt = FY,t (Zt) dt + σY,j,t (Zt) dWj,t , (3) j where FX,t, FY,t, σX,j,t, and σY,j,t are time-dependent vector fields, and the Wj,t’s are independent Brownian motions. The non-negative covariance (diffusion) matrix Dt (z) is the 2 × 2 block matrix with components

1 X 1 X D (z) ≡ σ ⊗ σ (z) ,D (z) ≡ σ ⊗ σ (z) , XX,t 2 X,j,t X,j,t Y Y,t 2 Y,j,t Y,j,t j j 1 X D (z) = D (z)T ≡ σ ⊗ σ (z) , (4) XY,t Y X,t 2 X,j,t Y,j,t j where the symbol ⊗ applied to two vectors U and V means the matrix construction (U ⊗V )ij ≡ U iV j. The associated Markovian generator is then obtained as

0 FP 0 0 Lt(z, z ) = Lt (z )[δ(z − z )] , (5)

FP where Lt is the (Fokker-Planck) second-order diﬀerential operator

FP Lt (z) = −∇x ◦ FX,t (z) − ∇y ◦ FY,t (z) + ∇x ◦ ∇x ◦ DXX,t (z) + ∇y ◦ ∇y ◦ DY Y,t (z) + 2∇x ◦ ∇y ◦ DXY,t (z) (6) obtained by interpreting Eqs. (3) with Ito convention. (In the above expression the last term is a priori ambiguous because DXY,t is not necessarily symmetric. The notation ∇x ◦ ∇y ◦ DXY,t should thus be interpreted as ∇xi ◦ ∇yj ◦ i,j (DXY,t) , using Einstein summation convention for repeated indices.) As usual, one can introduce the probability currents

JX,t(z) ≡ FX,t(z)Pt(z) − ∇x[DXX,t(z)Pt(z)] − ∇y[DXY,t(z)Pt(z)]

JY,t(z) ≡ FY,t(z)Pt(z) − ∇x[DY X,t(z)Pt(z)] − ∇y[DY Y,t(z)Pt(z)] , (7) 5 and recast the Fokker-Planck (FP) equation as the continuity equation

∂tPt(z) + ∇xJX,t(z) + ∇yJY,t(z) = 0 . (8) Note that the currents are defined up to a divergence-free vector. We will use the definitions in Eq. (7) in the following, which means that correction terms must be added in all expressions involving the currents if another decomposition is adopted. In continuous time, the Markov process Z is called bipartite if the transition probability P (Zt+h|Zt) satisfies the property

2 P (Zt+h|Zt) = P (Xt+h|Zt)P (Yt+h|Zt) + O(h ), (9) when h → 0+. From the forward Kolmogorov equation (1), the above condition is equivalent to assuming that the 0 Markovian generator Lt(z, z ) can be written as 0 0 0 0 0 Lt(z, z ) = δ(y − y )Lt,y(x, x ) + δ(x − x )Lt,x(y, y ) , (10)

0 0 where Lt,y(x, x ) and Lt,x(y, y ) are called partial generators. The delta functions become Kronecker matrices in the case of discrete space. The partial generators must individually satisfy conservation of probability, i.e., R 0 0 R 0 0 dx Lt,y(x, x ) = dy Lt,x(y, y ) = 0. In particular, a pure jump process is bipartite if the transition rates have 0 0 0 the additive form Wt[z, z ] = δy=y0 Wt,y(x, x ) + δx=x0 Wt,x(y, y ), which implies Eq. (10), as can be readily checked. On the other hand, a pure diffusion process is bipartite if DXY,t = 0, i.e., if the diffusion matrix is block diagonal. From Eqs. (4), a sufficient condition is that σX,j,t ⊗ σY,j,t = 0 for all j, which means that the overall noises affecting Xt and Yt are independent.

B. Deﬁnition of information measures

We start our reminder of information thermodynamics by recalling the definitions of several information-theoretic measures that are usually introduced in this framework. As already stressed, a consequence of the abandon of the bipartite assumption will be a proliferation of information measures. It is thus desirable to use transparent notations as much as possible. (Already in the bipartite case, a given quantity may have different names or be defined with different signs, which is a source of confusion.) It is also important to clearly state under which condition a relation is valid: in the following, the capital letter M on the left of an equation indicates that the joint process is Markovian, the capital letter B indicates that the process is Markovian and bipartite, and the capital letter S indicates that the process is stationary.

1. Information ﬂows, aka learning rates

Information flows quantify how the dynamical evolution of Xt or Yt contributes to the change in the mutual information, I(Xt : Yt) ≡ hln(P (Zt)/[P (Xt)P (Yt)]i, where P (Zt) is the joint probability distribution and P (Xt), P (Yt) its marginals. The latter characterizes the instantaneous correlation between X and Y at time t. These information- theoretic measures were first considered in the context of interacting diffusion processes [12] and subsequently introduced in the analysis of the thermodynamics of continuously-coupled, discrete-space stochastic systems [15, 18, 21]. Consider for instance the dynamical evolution of Xt. Introducing the time-shifted mutual information I(Xt+h : Yt) (with h > 0) and taking the limit h → 0+, one then defines [12]

1 1 D P (Yt|Xt+h)E lX (t) ≡ lim [I(Xt+h : Yt) − I(Xt : Yt)] = lim ln , (11) h→0+ h h→0+ h P (Yt|Xt) where here and in the following we use the bracket symbol for an expectation. Similarly, lY (t) is defined from I(Yt+h : Xt). (For brevity, it will be implicit in the following that similar quantities can be defined by exchanging X and Y .) One could also introduce Shannon entropies instead of mutual informations, by using I(Xt : Yt) = H(Xt) − H(Xt|Yt) = H(Yt) − H(Yt|Xt), with H(Xt) ≡ −hln P (Xt)i and H(Xt|Yt) ≡ −hln P (Xt|Yt)i [29]; however, we will try to avoid too many equivalent formulations throughout the paper. Note that the definition (11) is not restricted to a steady state. In a steady state the information flow identifies with the so-called learning rate lX defined in [17, 23]. Hereafter, we will use the denomination learning rate for lX (t) whether or not the system is in a steady state [37]. As discussed in [12, 15], learning rates have a clear meaning: For instance, lX (t) > 0 reveals that the dynamical evolution of X increases the mutual information I(Xt : Yt). In other words, the future of X is more predictable than its present from the viewpoint of Y [12], or X is “learning about” Y through its dynamics [15]. 6

For a bipartite Markov process, one has the natural decomposition of the time derivative of I(Xt : Yt) [12, 15, 27]

(B) dtI(Xt : Yt) = lX (t) + lY (t) , (12) as will be explicitly illustrated below for Markov processes.

2. Transfer entropy

Transfer entropy (TE) is an information-theoretic measure that is used to assess directional dependencies between time series and possibly infer causal interactions [38, 39]. Instead of I(Xt : Yt), one considers the change in the mutual information between stochastic trajectories observed during some time interval, say from 0 to t, and which t t are denoted by X0 and Y0 hereafter. Speciﬁcally, we deﬁne the TE rate from X to Y in continuous time as

1 t t+h t t 1 t t TX→Y (t) ≡ lim [I(X0 : Y0 ) − I(X0 : Y0 )] = lim I(X0 : Yt+h|Y0 ) h→0+ h h→0+ h 1 D P (Y |Xt,Y t)E = lim ln t+h 0 0 . (13) + t h→0 h P (Yt+h|Y0 ) t+h t where we have assumed that Y0 ∼ (Y0 ,Yt+h) for h infinitesimal and used the chain rule for mutual information, I(A : {B,C}) = I(A : C) + I(A : B|C), where I(A : B|C) is a conditional mutual information [29]. Like the learning rate, TX→Y (t) has a clear interpretation in terms of information transfer: It quantifies how much the knowledge of the t t trajectory X0 reduces the uncertainty about Yt+h (for h infinitesimal) when the trajectory Y0 is already known. As a conditional mutual information, TX→Y (t) is a non-negative quantity, whereas lX (t) has no definite sign. Note that the present definition is more general than the one adopted in [23] or [27] since we do not assume at this stage that t t t the joint process is Markovian. When the joint process is Markovian, one has P (Yt+h|X0,Y0 ) = P (Yt+h|X0,Yt) = t P (Yt+h|Xt,Y0 ) = P (Yt+h|Xt,Yt), and, after some manipulations, Eq. (13) can be rewritten as

1 t 1 t+h t (M) TX→Y (t) = lim I(Xt : Yt+h|Y0 ) = lim [I(Xt : Y0 ) − I(Xt : Y0 )] , (14) h→0+ h h→0+ h which clearly shows the difference with the learning rate lY (t) = limh→0+ (1/h)[I(Xt : Yt+h) − I(Xt : Yt)]. Note that the original definition of transfer entropy in discrete time is even more general since the number of time bins in the past of Xt and Yt may be different [38]. This definition can also be extended to continuous time [24, 40, 41]. Finally, see Ref. [42] for a rigorous definition via a partition of the time interval. For a Markov bipartite process, in full analogy with Eq. (12), one has the decomposition

t t 1 t+h t+h t t dtI(X0 : Y0 ) ≡ lim [I(X0 : Y0 ) − I(X0 : Y0 )] h→0+ h 1 D P (X ,Y |X ,Y ) E (M) = lim ln t+h t+h t t + t t h→0 h P (Xt+h|X0)P (Yt+h|Y0 ) (B) = TX→Y (t) + TY →X (t) , (15)

t+h t where we have used Eq. (9) and assumed Z0 ∼ (Z0, Zt+h) for h infinitesimal. Let us stress that the trajectory t t t t mutual information I(X0 : Y0 ) is a time-extensive quantity, in contrast with I(Xt : Yt). As a consequence, dtI(X0 : Y0 ) does not vanish in a steady state and, then, TX→Y 6= −TY →X , while lX = −lY . Here and throughout the paper, quantities without explicit time-dependence refer to a steady state. Although this is rarely evoked in the stochastic thermodynamic literature, we recall that transfer entropy is es- sentially a non-linear extension of Granger causality (GC) [43], which is a concept widely used in econometrics and neuroscience for analyzing the relationships between time series and inferring causal interactions (see e.g. [44] of a review). The general issue is whether the knowledge of one of the variables can help forecast another one. In contrast with transfer entropy, GC is usually identified with a model-based viewpoint, for instance a vector autoregressive modeling of the time series data [45]. It turns out that linear GC and transfer entropy rate are fully equivalent when the variables are Gaussian distributed, with a simple factor of 2 relating the two quantities [46]. This equivalence, first shown in the case of discrete-time random processes [47], can be extended to the continuous-time version [48]. Since the TE rates are conditioned on whole trajectories, they are very hard to compute numerically [49] and one t t often replaces X0 and Y0 by the states Xt and Yt at the latest time t. One then defines [12, 18, 23, 27]

1 1 D P (Yt+h|Xt,Yt)E T X→Y (t) ≡ lim I(Xt : Yt+h|Yt) = lim ln , (16) h→0+ h h→0+ h P (Yt+h|Yt) which is called a “single-time-step” TE rate in [27] to contrast with the “multi-time-step” TE rate TX→Y (t). We will adopt this terminology hereafter. 7

C. Inequalities and suﬃcient statistic

For a Markov bipartite process in a steady state, one has the two inequalities [18, 23]

T ≤ T (17) (B + S) X→Y X→Y lY ≤ TX→Y , (18) and we do not report the demonstration here since more general inequalities will be derived in the next section. The second inequality expresses the intuitive idea that the instantaneous value of Y is less informative about the instantaneous value of X that the whole past trajectory of Y . Within the context of a sensory system, where Xt and Yt denote the states of the signal and the sensor, respectively, this prompted the authors of [23] to introduce a so-called “sensory capacity” CY = lY /TX→Y as a tool to quantify the performance of the sensor (assuming that lY ≥ 0). In particular, CY reaches its maximal value 1 when inequality (18) is saturated. As discussed in [27], inequalities (17) and (18) are both saturated when the following condition is satisﬁed:

t P (Xt|Y0 ) = P (Xt|Yt) , (19) which means that “Yt is a suﬃcient statistic of Xt” [29] and no more information about Xt is contained in the t trajectory Y0 than in Yt alone. Interestingly, such an optimization of information transfer may occur in actual biological signaling circuits [27, 51]. By construction, this condition is realized by the Kalman-Bucy ﬁlter [52–54], as will be illustrated later on. As can be expected, things become more complicated when the bipartite assumption is dropped, and we show in the next section that this requires introducing additional information-theoretic measures.

D. Entropy production and second law

While the conventional second law of thermodynamics deals with the irreversibility of the whole process Zt, information measures can be used to formulate modified versions of the second law (which may then be called “second laws of information thermodynamics ”) that assess the irreversibility of one subsystem alone, say Xt, in the presence of the coupling with the other subsystem. The key quantity is the (fixed-time) entropy production rate σX (t) which is defined by considering X as an open system and Y as just a fictitious external protocol (or idealized work source) [55]. On general grounds (see, e.g., [56]), σX (t) can be decomposed as

B σX (t) ≡ dtSX (t) + σX (t) , (20) R where dtSX (t), the time derivative of the marginal Shannon entropy SX (t) = −kB dx Pt(x) ln Pt(x), is the rate of B change of the entropy of X, and σX (t) is the rate of change of the entropy of the environment or the bath. (From now on the Boltzmann constant kB is set equal to 1, so that we may use S instead of H as Shannon entropy.) As is now standard in the framework of stochastic thermodynamics (see, e.g., [57]), the cumulative entropy change B R t B ΣX = 0 ds σX (s) can be expressed as the mean of the logratio of the probabilities to observe a trajectory in forward and backward “experiments”. As Y is treated as an external protocol, one has

t D PbY t (X |X0) E (B)ΣB = ln 0 0 , (21) X 0 P 0 (X |X ) bYt t t

t 0 where P t (X |X ) is the probability of the trajectory of X for a ﬁxed trajectory of Y and P 0 (X |X ) is the bY0 0 0 bYt t t 0 0 corresponding probability of the time-reversed trajectory Xt for the ﬁxed time-reversed trajectory Yt [58]. For a B bipartite pure jump process, σX (t) is then given by

0 B X 0 Wt,y(x, x ) σX (t) ≡ Pt (z) Wt,y(x, x ) ln 0 , (22) Wt,y(x , x) z,x0 whereas for a bipartite diffusion process it is equal to Z B −1 σX (t) ≡ dz DXX,t(z) JX,t(z)FbX,t(z) , (23) 8 where the diffusion matrix DXX,t > 0 and the probability current JX,t have been defined above, and FbX,t(z) is the modified drift defined by FbX,t (z) ≡ FX,t (z) − ∇x.DXX,t(z) [62]. In cases where the thermodynamics of subsystem B X can be defined and the environment is a single thermal bath at a given inverse temperature β, σX identifies with the heat flow βQ˙ from X to the bath. Since the two subsystems are coupled, σX (t) may become negative, but a lower bound is provided by including the information shared with Y . For a bipartite Markov process, the various second-law-like inequalities proven in the literature [12, 14, 15, 18, 20] can be summarized by the following hierarchy of bounds,

t σX (t) ≥ lX (t) ≥ dtI(Xt : Y0 ) − TX→Y (t), (24) or, for the time-integrated quantities, Z t Z t Z t Z t Z t ΣX ≡ ds σX (s) ≥ ds lX (s) ≥ ∆I − ds TX→Y (s) ≥ ∆I − ds TX→Y (s) ≥ ∆I − ds T X→Y (s) , (25) 0 0 0 0 0

t where ∆I = I(Xt : Y0 ) − I(X0,Y0) and ∆I = I(Xt : Yt) − I(X0,Y0), with ∆I ≥ ∆I by marginalization. The key fact is that the tightest bound is provided by the learning rate.

III. INFORMATION MEASURES FOR NON-BIPARTITE PROCESSES

A. Learning rates

We ﬁrst search for a generalization of Eq. (12). The decomposition of dtI(Xt : Yt) introduced above in the bipartite case suggests to deﬁne the new rate

− 1 lX (t) ≡ lim [I(Xt+h : Yt+h) − I(Xt : Yt+h)] (26) h→0+ h

+ in addition to lX (t). Accordingly, lX (t) will be denoted lX (t) hereafter to make the notations more consistent. − − + Indeed, lX (t) can be also written as lX (t) = −dI(Xt−h : Yt)/dh|h=0+ whereas lX (t) = dI(Xt+h : Yt)/dh|h=0+ . This − + also suggests to call lX (t) a backward learning rate, in contrast with the forward rate lX (t). We stress that the − deﬁnition of lX (t) as a derivative has nothing to do with stationarity but simply results from a Taylor expansion in h: 2 Indeed, for any continuously diﬀerentiable function f(s, t), one has f(t+h, t+h)−f(t, t+h) = h(∂1f)(t, t)+O(h ) = 2 − f(t, t) − f(t − h, t) + O(h ). Note also that lX (t) 6= dI(Xt : Yt+h)/dh|h=0+ . − − Thanks to the introduction of lX (t), and of the corresponding lY (t), Eq. (12) is now replaced by the two relations

+ − + − dtI(Xt : Yt) = lX (t) + lY (t) = lY (t) + lX (t). (27) The distinction between the forward and backward learning rates in the general (i.e., non-bipartite) case suggests to introduce the symmetric quantities 1 1 lS (t) ≡ l+ (t) + l− (t) , lS (t) ≡ l+(t) + l−(t) , (28) X 2 X X Y 2 Y Y such that Eqs. (27) now yield

S S dtI(Xt,Yt) = lX (t) + lY (t) . (29) As will be seen below, these symmetric learning rates play a natural role in the thermodynamics since they vanish when the joint system is at equilibrium, i.e., when the condition of detailed balance is satisﬁed. On the other hand, the non-symmetric rates vanish when the processes Xt and Yt are independent. Note that we use the adjective S “symmetric” to qualify lX (t) because it is the half-sum of the forward and backward learning rates, i.e., of right and left derivatives. There must be no confusion with the time-symmetric, kinetic (path-) observables such a “frenesy” which also play a signiﬁcant role in the study of non-equilibrium phenomena [63]. In the general case, but in a steady state, there are only two independent learning rates since dtI(Xt : Yt) = 0 and Eqs. (27) yield the “conservation” relations

+ − + (S) lX = −lY (6= −lY ) + − + lY = −lX (6= −lX ) , (30) 9 and thus S S (S) lY = −lX . (31) A basic feature of the learning rates is that they can be expressed in terms of the two-point probability distribution 0 0 0 P (z, t; z , t ) ≡ hδ(Zt − z)δ(Zt0 − z )i, with z ≡ (x, y), and the corresponding marginal distributions. (Hereafter, variables with a prime symbol such as z0, x0, y0 will always refer to a time t0 ≤ t in the two-point probability distribution functions.) Starting from the deﬁnitions (11) and (26), and using the normalization condition R dxdy0 P (x, t; y0, t0) = 1, we get Z 0 + 0 d 0 Pt(x, y ) lX (t) = dx dy P (x, t + h; y , t)|h=0+ ln 0 (32a) dh Pt(x)Pt(y ) Z 0 − 0 d 0 Pt(x , y) lX (t) = − dx dy P (y, t; x , t − h)|h=0+ ln 0 , (32b) dh Pt(x )Pt(y) where we have used the notation Pt(x, y) ≡ P (x, t; y, t) for the joint probability distribution at the same time t (and + − Pt(x), Pt(y) for the marginal distributions); similar expressions are obtained for lY (t) and lY (t). (We recall that we use the same notations for continuous and discrete spaces. In the latter case, integrals must be replaced by sums.) From these equations, we readily see that the learning rates vanish when Pt(z) = Pt(x)Pt(y), which means that the processes X and Y are independent. We stress that these formulas are fully general and do not require the joint process Z to be Markovian. However, further simpliﬁcations occur in the Markovian case, as one can replace the 0 derivative with respect to h by using the Kolmogorov equation and introducing the Markovian generator Lt(z , z), which leads to Z 0 + 0 0 0 Pt(x, y ) (M) lX (t) = dz dz Pt(z )Lt (z , z) ln 0 . (33) Pt(x)Pt(y ) Furthermore, by using the decomposition in Eq. (27) and the expression of the time derivative of the mutual information, Z Pt(z) dtIt(Xt,Yt) = dz ∂tPt(z) ln , (34) Pt(x)Pt(y) − lX (t) is obtained as Z 0 − 0 0 0 Pt(x, y)Pt(x ) (M) lX (t) = dz dz Pt(z )Lt (z , z) ln 0 . (35) Pt(x , y)Pt(x) R 0 Finally, after using the conservation of probability dzLt(z , z) = 0, we obtain the symmetric learning rates as

Z 0 0 2! S 1 0 0 0 0 Pt(x, y )Pt(z) Pt(x ) (M) lX (t) = dz dz [(Pt(z )Lt (z , z) − Pt(z)Lt (z, z )] ln 0 0 . (36) 4 Pt(x , y)Pt(z ) Pt(x)

From the above expression, one can immediately see that if Zt is an equilibrium process, such that the probability 0 0 0 current Pt(z )Lt (z , z) − Pt(z)Lt (z, z ) vanishes, the symmetric learning rates both vanish. In Sec. IV A, we will provide more explicit expressions for these learning rates in the case of a Markovian diffusion process. The case of a Markovian pure jump process in a discrete space is treated in Appendix A. If we now come back to the special situation of a bipartite process, due to the additive form of the Markovian generator [Eq. (10)] and the conservation of probability, the formulas given in Eqs. (33) and (35) coincide and Z 0 + − 0 0 0 Pt(x, y ) 0 (B) lX (t) = lX (t) = dz dxPt(z )Lt,y (x , x) ln 0 . (37) Pt(x)Pt(y ) + − A similar relation holds for lY (t) and lY (t). There are thus only two independent learning rates instead of four, and Eq. (27) then gives back Eq. (12). Note that the present equalities between learning rates differ from those expressed in Eq. (30). More generally, one must carefully distinguish relations valid for a bipartite process from those valid for a non-bipartite process in a steady state [64]. Of course, if the joint process is both bipartite and stationary, Eqs. (30) S S + − + − and (37) imply that only one independent learning rate subsists, for instance lY = −lX = lY = lY = −lX = −lX . We conclude this part on the learning rates by briefly discussing their content in terms of information. The + − S various quantities lX (t), lX (t), lX (t), and their counterparts for the Y subsystem, all measure the change of mutual information between X and Y due to different aspects of an infinitesimal dynamical evolution of one subsystem or + − S the other. When both lX (t) and lX (t) are strictly positive, and as a direct consequence lX (t) > 0, one can plausibly conclude that X is “learning about” Y through its dynamics. However, the non-bipartite structure of the process + − allows cases with lX (t) > 0 and lX (t) < 0, which have no manifest interpretation in the context of learning. 10

B. Transfer entropy rates

Transfer entropy rates TX→Y (t) and TY →X (t) can be defined by the same formulas as in the bipartite case: see Eq. (13). They keep the same property of being non-negative and the same meaning as information-theoretic measures. However, whereas the generalization of the decomposition of the variation of mutual information [Eq. (12)] to a non-bipartite process was straightforward, a similar operation for the pathwise mutual information [Eq. (15)] turns out to be problematic. Indeed, using the second line of Eq. (15), one can write t t dtI(X0,Y0 ) = TX→Y (t) + TY →X (t) + TX.Y (t) , (38) −1 t t where TX.Y (t) ≡ limh→0+ h I(Xt+h : Yt+h|X0,Y0 ) is a symmetric quantity measuring the “instantaneous” dependence of the two processes (see, e.g., [65] for the discrete-time version). However, there is a serious obstruction, at least for diffusion processes: TX.Y (t) is either zero if Z is bipartite [cf. Eq. (15) above] or infinite otherwise! In other t t t t words, dtI(X0 : Y0 ) and thus I(X0 : Y0 ) itself are infinite for non-bipartite diffusions. Indeed, as noticed in [66], Xt and Yt have a non-zero quadratic variation if the noises are correlated, which results in the singularity of their joint distribution with respect to the product of the corresponding marginals. Such a difficulty does not occur in discrete time, as briefly discussed in note [67], nor in discrete space. We thus turn our attention to another class of rates which are well-defined in the non-bipartite case and will allow t us to generalize the important inequality (18). These rates are associated with the mutual informations I(Xt : Y0 ) t and I(Yt : X0) that appear in filtering theory [69, 70]. We introduce the TE rate, called “filtered transfer entropy rate”,

1 t+h t 1 t TbX→Y (t) ≡ lim [I(Xt+h : Y0 ) − I(Xt+h : Y0 )] = lim I(Xt+h : Yt+h|Y0 ) h→0+ h h→0+ h 1 D P (X ,Y |Y t) E = lim ln t+h t+h 0 , (39) + t t h→0 h P (Xt+h|Y0 )P (Yt+h|Y0 ) which quantiﬁes how much the prediction of Xt+h, for h inﬁnitesimal, is improved by knowing Yt+h in addition to t the trajectory Y0 . In general there is no simple relation between TbX→Y (t) and TX→Y (t), except when the process is Markov bipartite, where

(B) TbX→Y (t) = TX→Y (t) . (40) Indeed, from the definitions (13) and (39), we have the general equation t t 1 D P (Yt+h|X0,Y0 ) E TX→Y (t) − TX→Y (t) = lim ln , (41) b + t h→0 h P (Yt+h|Y0 ,Xt+h) and it can be proven that the right-hand side of this equation is equal to 0 when the process is bipartite. The demonstration for jump processes is in Appendix C of [18] and for diffusion processes it is in given in Appendix B of the present paper. There is of course a single-time-step TE rate corresponding to TbX→Y (t), which is defined as 1 Tb X→Y (t) ≡ lim I(Xt+h : Yt+h|Yt) . (42) h→0+ h Then,

1 P (Yt+h|Xt,Yt) T X→Y (t) − Tb X→Y (t) = lim hln i , (43) h→0+ h P (Yt+h|Xt+h,Yt) and

(B) Tb X→Y (t) = T X→Y (t) (44) in the bipartite case, as will be illustrated below for diﬀusion processes [see Eq. (80)]. As for the learning rates, the single-time-step TE rates can be expressed in terms of the two-point probability 0 0 0 distribution P (z, t; z , t ) ≡ hδ(Zt − z)δ(Zt0 − z )i, with z ≡ (x, y), and the corresponding marginal distributions: Z 0 1 0 0 P (y, t + h|z , t) T X→Y (t) = lim dy dz P (y, t + h; z , t) ln (45a) h→0+ h P (y, t + h|y0, t) Z 0 1 0 0 P (z, t + h|y , t) Tb X→Y (t) = lim dz dy P (z, t + h; y , t) ln . (45b) h→0+ h P (x, t + h|y0, t)P (y, t + h|y0, t) 11

In contrast with the learning rates [Eqs. (32)], one cannot generally introduce derivatives with respect to h in these expressions. On the other hand, we shall derive explicit expressions for Markov diﬀusion processes: see Sec. IV B below.

C. Backward transfer entropy rates

Finally, we add to our list of information-theoretic measures another TE rate which can be used to assess the directionality of information transfer (see Sec. VI B) and which will play an important role in the generalization of the second law (see Sec. IV D). To this aim, we slightly change our notations by assuming that the trajectories of X and Y are now observed in the time interval [0,T ]. We then deﬁne

† 1 T T T T 1 T T TX→Y (t) ≡ lim [I(Xt+h : Yt ) − I(Xt+h : Yt+h)] = lim I(Xt+h : Yt|Yt+h) h→0+ h h→0+ h T T 1 D P (Yt|X ,Y )E = lim ln t+h t+h , (46) + T h→0 h P (Yt|Yt+h)

T T † where Xt+h and Yt+h denote the trajectories of X and Y in the time interval [t + h, T ]. TX→Y (t) has clearly the meaning and the properties of a TE rate, but it involves the future trajectories of X and Y instead of their past. It may thus be called a backward TE (BTE) rate and regarded as the continuous-time version of the BTE introduced in [25] in the discrete-time framework (see also [71–73] for the introduction of time-reversed Granger causality). It is actually much simpler to consider continuous time from the outset as this makes the generalization of the relations derived in [25] to the non-bipartite case a straightforward operation. We draw attention to the fact that the BTE deﬁned in this way has no relation with the time-reversed transfer entropy considered in [24, 74, 75]. When Z is a Markov process, the deﬁnition (46) can be also rewritten as

1 D P (Y |X ,Y )E (M) T † (t) ≡ lim ln t t+h t+h . (47) X→Y + T h→0 h P (Yt|Yt+h) As before, we also deﬁne a single-time-step BTE rate as

† 1 1 D P (Yt|Xt+h,Yt+h)E T X→Y (t) ≡ lim I(Xt+h : Yt|Yt+h) = lim ln , (48) h→0 h h→0 h P (Yt|Yt+h) which will play a useful role in Sec. IV D [76]. Note however that this does not add a new independent measure of − information to our list since it can be easily seen from the deﬁnitions (26 ) of lY (t) and (42) of Tb X→Y (t) that

† − T X→Y (t) = Tb X→Y (t) − lY (t) . (49) Combining Eq. (49) with Eq. (43) also yields

† − 1 D P (Yt+h|Xt,Yt) E T X→Y (t) − T X→Y (t) = lY (t) + lim ln , (50) h→0+ h P (Yt+h|Xt+h,Yt) which, in the bipartite case, implies that

† (B) T X→Y (t) − T X→Y (t) = lY (t). (51) This is in agreement with Eq. (18) in [25, 78]. Furthermore, when the joint process Zt is Markovian, simple manipulations yield

T † † 1 D P (Yt|Yt+h) P (Yt+h)E (M)[TX→Y (t) − T (t)] − [T X→Y (t) − T (t)] = lim ln X→Y X→Y + t h→0 h P (Yt+h|Y0 ) P (Yt) d = [S(Y t) + S(Y T ) − S(Y )] , (52) dt 0 t t

t+h t T T where we have used Y0 ∼ (Y0 ,Yt+h) and Yt ∼ (Yt,Yt+h) for h inﬁnitesimal to obtain the second equality. By t T T T integrating from 0 to T , the left-hand side of this equation vanishes since [S(Y0 )+S(Yt )−S(Yt)]0 = S(Y0 )−S(Y0)+ 12

T S(YT ) − S(Y0 ) − S(YT ) + S(Y0) = 0.We ﬁnally obtain a simple relation (not evident from the outset) involving the forward and backward time-integrated TEs,

Z T Z T † † (M) dt [TX→Y (t) − TX→Y (t)] = dt [T X→Y (t) − T X→Y (t)] . (53) 0 0 Although this may be regarded as the continuous-time analog of Eq. (17) in [25], we stress that the derivation of this relation does not require the joint process to be bipartite. Eq. (53) also leads to the interesting steady-state relation [79]

† † (M + S) TX→Y − TX→Y = T X→Y − T X→Y . (54)

D. Inequalities

We are now in position to generalize the standard inequalities (17) and (18) obtained in the bipartite case. 1) First, one easily obtains from the deﬁnitions that the single-time-step TE rate is an upper bound on the forward learning rate,

+ lY (t) ≤ T X→Y (t) , (55) and that the single-time-step ﬁltered TE rate is an upper bound on the backward learning rate,

− lY (t) ≤ TbX→Y (t) . (56)

+ −1 − Indeed, one has T X→Y (t) − lY (t) = limh→0+ h [S(Xt|Yt+h) − S(Xt|Yt,Yt+h)] and Tb X→Y (t) − lY (t) = −1 limh→0+ h [S(Xt+h|Yt+h) − S(Xt+h|Yt,Yt+h)], and Shannon entropy never increases by conditioning. These inequalities hold for a general (non-stationary and possibly non-Markovian) process. 2) If the joint process is Markovian, the single-time-step TE rate is an upper bound on the multi-time-step TE rate,

(M) TX→Y (t) ≤ T X→Y (t) , (57)

−1 t since T X→Y (t) − TX→Y (t) = limh→0+ h [S(Yt+h|Yt) − S(Yt+h|Y0 )] ≥ 0. On the other hand, note that Tb X→Y (t) is not an upper bound on TbX→Y (t). 3) In a steady state, the second inequality (18) is replaced by

− (M + S) lY ≤ TbX→Y , (58) which is a special case of the more general inequality

+ d t (M) l (t) ≥ I(Xt : Y ) − TbX→Y (t), (59) X dt 0

+ − t −1 t t t since lX = −lY and I(Xt : Y0 ) is not time extensive, i.e. limt→∞ t I(Xt : Y0 ) = 0 [in contrast with I(X0 : Y0 )]. Inequality (18) is then recovered for a bipartite process since TbX→Y = TX→Y in this case, as we have seen before [Eq. (40)]. We prove the inequality in Eq. (59) in Appendix B. To summarize all these inequalities and be more concrete, let us give a numerical illustration. Anticipating the calculations performed in Sec. V for a bi-dimensional Ornstein-Uhlenbeck process, we show in Fig. 1 the behavior of the various information measures as a function of the parameter −1 ≤ ρ ≤ 1 that quantifies the correlations between the noises affecting the two Langevin subprocesses. The numerical values of the other parameters of the model are chosen in such a way that X may be considered as a source signal measured by Y (see the discussion in Sec. V). The small arrows in the figure indicate values of ρ for which the system has a non-generic but remarkable behavior. S S S i) The first one on the left side (ρ = −0.6) indicates that the symmetric learning rate lY (and thus also lX = −lY ) vanishes. As seen from Eq. (36), this occurs when the joint system is at equilibrium (in the sense of satisfied detailed balance and zero probability currents). + + S ii) The next arrow (ρ = −0.35) indicates that lY = lX = lY = 0, which occurs when the two subprocesses X and Y become independent: see Eqs. (33) and (35). − iii) The third arrow (ρ ≈ −0.185) indicates that TX→Y = T X→Y and lY = TbX→Y = Tb X→Y . Extending the analysis performed in [27] and taking the continuous-time limit from the outset, we show in Appendix C that this occurs when 13

0.06

0.04

0.02

Learning rates and TE 0

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 ρ

FIG. 1: (Color on line) Steady-state transfer entropy and learning rates for a stationary bi-dimensional Ornstein-Uhlenbeck process as a function of the parameter −1 ≤ ρ ≤ 1 that quantiﬁes the correlations between the noises: TX→Y (solid black line), + − + T X→Y (long-dashed black line), TbX→Y (solid red line), Tb X→Y (long-dashed red line), lY (dashed-dotted brown line), lY = −lX S + − + − S (black dotted line), and lY = (lY + lY )/2 (short dashed green line). One has lY = lY = lY , TbX→Y = TX→Y , Tb X→Y = T X→Y + − − in the bipartite case (ρ = 0) and lY ≤ T X→Y , lY ≤ Tb X→Y , TX→Y ≤ T X→Y , lY ≤ TbX→Y more generally, as predicted by inequalities (55-58) (but Tb X→Y is not an upper bound on TbX→Y ). Special values of ρ are indicated by little arrows: a) ρ = −0.6: S + + S the joint system is at equilibrium, so that lY = 0; b) ρ = −0.35: the subprocesses are independent, so that lY = lX = lY = 0; c) ρ ≈ −0.185: Yt is a suﬃcient statistic of Xt [Eq. (19)], so that inequalities (57)) and (58) are saturated and TbX→Y = Tb X→Y ; d) ρ = 0.4: inequality (55) is saturated. The model parameters are a11 = 1, a22 = 0.25, a12 = −0.05, a21 = −0.5 and D11 = D22 = 2.

Yt is a sufficient statistic of Xt, as expressed by Eq. (19). Thanks to Eqs. (49) and (54), this also implies that † † TX→Y = T X→Y = 0. + iv) Finally, the last arrow on the right side (ρ = 0.4) indicates that inequality (55) is saturated and lY (t) = T X→Y (t). + This generally does not coincide with the saturation of inequalities (57) and (58). Indeed, since T X→Y (t) − lY (t) = −1 ∂hI(Xt : Yt|Yt+h)|h=0+ , equality is obtained when limh→0+ h [P (Xt|Yt,Yt+h) − P (Xt|Yt+h)] = 0, which differs from condition (19). The case of a bipartite process considered in [27] is an exception, as inequalities (55) and (58) then − + coincide (T X→Y = Tb X→Y and lY = lY ). Although inequality (58), which appears as the generalization of inequality (18), has no intuitive interpretation [in contrast with (18)], the fact that it becomes an equality if Yt is a sufficient statistic of Xt may suggest to generalize the concept of a sensory capacity as

− lY (M + S) CY = . (60) TbX→Y 14

Likewise, we may deﬁne a “single-time-step” capacity,

− − lY lY (S) CY = = , (61) − † Tb X→Y lY + T X→Y which is also bounded by 1 thanks to inequality (56), and equal to 1 if Yt is a sufficient statistic of Xt (see Appendix C). Note that the joint dynamics needs not be Markovian. Moreover, since CY involves the single-time-step TE’s † T X→Y or T X→Y instead of TbX→Y , it is much simpler to obtain this quantity from experimental time series than CY . Of course, it remains to be seen on specific examples of sensory systems if these quantities are helpful to estimate the performance of the sensor in the presence of correlations between the observation and signal noises, a situation classically treated in the framework of filtering theory [69, 70].

IV. MARKOV DIFFUSION PROCESSES AND SECOND LAW OF INFORMATION THERMODYNAMICS

To make all the above definitions and relations more explicit and to derive a second law, we now focus on Markov diffusion processes as defined in Eq. (3). To reduce the amount of notation, we consider the case where Xt and Yt are unidimensional processes. The vector fields FX,t, FY,t and the matrix fields DXX,t, DXY,t, and DY Y,t are now all scalar fields. The general case can be easily extrapolated from this one.

A. Learning rates

As we have already pointed out, the expressions (32) of the learning rates can be simpliﬁed when the joint process Zt is Markovian. From the forward Kolmogorov equation (1) and the Markovian generator, we readily obtain

d 0 FP 0 P (z, t + h|z , t)| + = L (z)δ(z − z ) (62) dh h=0 t for t0 ≤ t. Integrating over y and using Eq. (6) then yields

2 d 0 ∂ 0 0 ∂ 0 P (x, t + h|z , t)| + = − [F (x, y )δ(x − x )] + [D (z)δ(x − x )] , (63) dh h=0 ∂x X,t ∂x2 XX,t as all terms involving derivatives with respect to y vanish at the boundaries (assuming natural boundary conditions). As a result, Z d 0 0 d 0 0 P (x, t + h; y , t)| + = dx P (x, t + h|z , t)| P (z ) dh h=0 dh h=0 t ∂ ∂2 = − [F (x, y0)P (x, y0)] + [D (z)P (x, y0)] , (64) ∂x X,t t ∂x2 XX,t t and, after integration by parts, we transform Eq. (32a) into Z + lX (t) = dz (FX,t(z)Pt(z) − ∂x[DXX,t(z)Pt(z)]) ∂x ln Pt(y|x) (65) where Pt(y|x) ≡ Pt(z)/Pt(x) is the conditional probability distribution function. Since we only consider Markov processes in this section, we no longer add the bold letter M on the left of the equations. − + − The rate lX (t) is obtained by using the relation dtIt(Xt,Yt) = lY (t) + lX (t) [Eq. (27)], the expression of the time derivative of the mutual information [Eq. (34)], and the FP equation. It reads Z − lX (t) = dz (FX,t(z)Pt(z) − ∂x[DXX,t(z)Pt(z)] − 2∂y[DXY,t(z)Pt(z)]) ∂x ln Pt(y|x) . (66)

+ − + − We immediately see that lX (t) = lX (t) and lY (t) = lY (t) when the process is bipartite (i.e., DXY,t(z) = 0) in agreement with Eq. (37). 15

One may also rewrite these expressions in terms of the probability currents deﬁned by Eq. (7). This yields Z ± lX (t) = dz (JX,t(z) ± ∂y[DXY,t(z)Pt(z)]) ∂x ln Pt(y|x) , (67)

S + − and the symmetric learning rate lX (t) = (1/2)[lX (t) + lX (t)] is then simply given by Z S lX (t) = dz JX,t(z)∂x ln Pt(y|x) . (68)

S S As announced before, we observe that the two symmetric rates lX (t) and lY (t) vanish in a steady state when the two probability currents are zero, which corresponds to equilibrium. On the other hand, the non-symmetric rates remain finite. All the learning rates vanish when the two subprocesses X and Y are independent, i.e., when Pt(z) = Pt(x)Pt(y). The above equations generalize the expressions in the current literature obtained for bipartite processes and additive noises [12, 19, 23] (note that the sign convention may differ). These expressions are immediately recovered by setting DXY,t = 0 and taking DXX,t and DY Y,t independent of x and y. Finally, we note that the learning rates obtained above are finite, at least if the integrals in the right-hand sides of (67) and (68) are finite, for all diffusion processes described by Eq. (3). This will not necessarily be the case of the single-time-step transfer entropy rates that we consider in the following.

B. Single-time-step transfer entropy rates

We now derive the expressions of the various single-time-step TE rates. It turns out that it suffices to compute the rate T X→Y (t) given by Eq. (45a) (a similar calculation was presented in [11], but it is worth repeating it for † completeness). The backward rate T X→Y (t) is then deducible from T X→Y (t), and Tb X→Y (t) is finally obtained from Eq. (49). We start from the expression of the infinitesimal transition probability (or propagator),

0 0 FP 0 2 P (z, t + h|z , t) = δ(z − z ) + hLt (z)[δ(z − z )] + O(h ) , (69)

FP with Lt the Focker-Planck operator appearing in Eq. (62) and deﬁned by Eq. (6). Integrating the propagator over x and then integrating P (y, t + h; z0, t) over x0, we obtain

∂ ∂2 P (y, t + h|z0, t) = δ(y − y0) − hF (z0) − D (z0) δ(y − y0) + O(h2) Y,t ∂y Y Y,t ∂y2 ∂ ∂2 P (y, t + h|y0, t) = δ(y − y0) − hF (y0) − D (y0) δ(y − y0) + O(h2) , (70) Y,t ∂y Y Y,t ∂y2 where Z F Y,t(y) ≡ dx Pt(x|y)FY,t(z) (71) and Z DY Y,t(y) ≡ dx Pt(x|y)DY Y,t(z) . (72)

To compute the logarithms of the transition probabilities, we need to replace Eqs (70) by their Gaussian small-time expressions, [32]

− 1 [y−y0−hF (z0)]2 0 1 4hD (z0) Y,t P (y, t + h|z , t) = e Y Y,t p 0 4πhDY Y,t(z ) − 1 [y−y0−hF (y0)]2 0 1 4hD (y0) Y,t P (y, t + h|y , t) = q e Y Y,t , (73) 0 4πhDY Y,t(y )

+ 0 when h → 0 , using the convention that the argument of the functions DY Y,t and FY,t is z . Inserting Eq. (73) into Eq. (45a), we readily see that T X→Y (t) diverges if DY Y,t(y) 6= DY Y,t(z), that is if DY Y,t is also a function of 16

t t x. We thus recover the conditions specified in [31] for the mutual information I(X0,Y0 ) to be finite in the case of multiplicative independent noises. To proceed, we thus assume that DY Y,t only depends on y. Then, 0 P (y, t + h|z , t) 1 n 0 0 h 2 0 2 0 o 2 ln 0 = 0 (y − y )[FY,t(z ) − F Y,t(y)] − [FY,t(z ) − F Y,t(y )] + O(h ) , (74) P (y, t + h|y , t) 2DY Y,t(y ) 2 and using Eq. (45a), we finally obtain after some manipulations

( 1 R Pt(z) 2 dz [FY,t(z) − F Y,t(y)] if DY Y,t(z) = DY Y,t(y) 4 DY Y,t(y) T X→Y (t) = (75) ∞ otherwise. This generalizes the expression given in [23] for a bipartite system with additive noises (an explicit calculation is also performed in [20] for a bi-dimensional Ornstein-Uhlenbeck model). Note that, contrary to the learning rates, the single-time-step TE rate is inﬁnite when DY Y,t = 0 (or in the multi-dimensional case when the matrix DY Y,t is not invertible), which implies for instance that it is not well-suited for underdamped processes. The same is true for the other TE rates considered below. † To obtain the expression of the BTE rate T X→Y (t) deﬁned by Eq. (48), a possible method is to use Bayes theorem to modify the argument of the logarithm and recast Eq. (48) as

† 1 D P (Xt+h,Yt+h|Yt) E T X→Y (t) = lim ln . (76) h→0 h P (Xt+h|Yt+h)P (Yt+h|Yt) The calculation would then follow the same lines as above. However, it is more instructive to use the fact that † T X→Y (t) is the TE rate at time T − t corresponding to the process ZT −t, which is the time reversal of the process Zt (as the state at time t along a forward trajectory is now conditioned on the state at time t + h). As is well known, ZT −t is also a diffusion process, under some mild conditions (see, e.g., [34, 80]). The covariance (diffusion) matrix ∗ and drift coefficients of the time-reversed process are given respectively by Dt (z) = DT −t(z) and

∗ 2 JX,T −t(z) FX,t(z) = −FX,T −t(z) + ∂x[DXX,T −t(z)PT −t(z)] + ∂y[DXY,T −t(z)PT −t(z)] = FX,T −t(z) − 2 PT −t(z) PT −t(z)

∗ 2 JY,T −t(z) FY,t(z) = −FY,T −t(z) + ∂x[DY X,T −t(z)PT −t(z)] + ∂y[DY Y,T −t(z)PT −t(z)] = FY,T −t(z) − 2 . PT −t(z) PT −t(z) (77)

∗ ∗ ∗ ∗ Denoting the single-time-step TE associated to Dt and FX,t,FY,t by T X→Y , we have by deﬁnition of the single-time- step backward TE rate that † ∗ T X→Y (t) = T X→Y (T − t) . (78)

† ∗ Of course, the same is true for the multi-time-step TE rate TY →X (t) which identiﬁes with TY →X (T − t). After some algebra described in Appendix B, we obtain

† Z 1 T X→Y (t) = T X→Y (t) − dz JY,t(z) ∂y ln Pt(x|y) + ∂x[DXY,t(z)Pt(z)] if DY Y,t(z) = DY Y,t(y) DY Y,t(y)Pt(z) (79)

† and T X→Y (t) = ∞ otherwise. Using Eq. (67), we see that relation in Eq. (51) is recovered in the bipartite case (DXY,t = 0), as expected. Moreover, if the joint system Z is at equilibrium (i.e., if the probability currents vanish), † we also have T X→Y (t) = T X→Y (t), which is not obvious from Eq. (50). From Eq. (54), this also implies that † TX→Y = TX→Y . In other words, the TE rate is time-symmetric at equilibrium, as it should be. − Finally, after using Eq. (49) and the expression of lY , i.e., Eq. (66) with X and Y interchanged, we find Z 1 h i Tb X→Y (t) = T X→Y (t) − dz JY,t(z) + ∂y[DY Y,t(y)Pt(z)] ∂x[DXY,t(z)Pt(z)] if DY Y,t(z) = DY Y,t(y) DY Y,t(y)Pt(z) (80) and Tb X→Y (t) = ∞ otherwise. This also immediately shows that Tb X→Y (t) = T X→Y (t) in the bipartite case [Eq. (44)]. We reiterate that for diffusion processes with multiplicative noises, one must have that DY Y,t(z) = DY Y,t(y), and similarly DXX,t(z) = DXX,t(x), otherwise the various TE rates are infinite, even in the bipartite case: see also Appendix B. 17

C. Multi-time-step transfer entropy rates

It turns out that an expression somewhat similar to Eq. (75) can also be obtained for the multi-time-step TE rate TX→Y (t). To do this one has to generalize the preceding derivation for T X→Y (t). A new ingredient is the presence 0t 0t of an inﬁnitesimal propagator P (y, t + h|y 0) that is conditioned by a whole path y 0 from 0 to t instead of a single value at time t. By using Bayes theorem and the Markovian property, we rewrite it as Z 0t 0 0 0 0t P (y, t + h|y 0) = dx P (y, t + h|z , t)P (x , t|y 0), (81)

0t 0 where by convention y 0 includes the endpoint y at time t. From Eq. (70) we then obtain 2 0t 0 0t ∂ 0t ∂ 0 2 P (y, t + h|y ) = δ(y − y ) − h FeY,t(y ) − DeY Y,t(y ) δ(y − y ) + O(h ), (82) 0 0 ∂y 0 ∂y2 with Z t t FeY,t(y0) ≡ dx P (x, t|y0)FY,t(z) Z t t DeY Y,t(y0) ≡ dx P (x, t|y0)DY Y,t(z) . (83)

The above infinitesimal propagator can also be cast in the form of a Gaussian small-time expression as in Eq. (73). The derivation of the expression of TX→Y (t) from its definition (13) then directly follows that of the single-time-step TE rate T X→Y (t) with the final result Z t 1 t P (x, t; y0) t 2 TX→Y (t) = dx D[y0] [FY,t(z) − FeY,t(y0)] if DY Y,t(z) = DY Y,t(y) (84) 4 DY Y,t(y) and TX→Y (t) = ∞ otherwise. As found for the single-time-step TE rate, TX→Y (t) is only finite if the diffusion coefficient DY Y,t(z) only depends on the variable y. As it should be, Eq. (75) is recovered and TX→Y (t) = T X→Y (t) t t when Yt is a sufficient statistic of Xt [Eq. (19)], as FeY,t(y0) = F Y,t(y) and DeY Y,t(y0) = DY Y,t(y) in this case. Note also that the above formula has been derived for simplicity for unidimensional processes Xt and Yt but mutatis mutandis it is easily extended to multidimensional processes. For instance, the result in Eq. (84) can be generally rewritten as

1D t −1 t E TX→Y (t) = [FY,t(Xt,Yt) − FeY,t(Y )].DY Y,t(Yt) .[FY,t(Xt,Yt) − FeY,t(Y )] , (85) 4 0 0 which expresses the transfer entropy rate in terms of the minimum mean-square error (MMSE) of the causal estimation. This generalizes the relation obtained in [42] for a bipartite diﬀusion process with additive noise and extends the classical and beautiful result of Duncan [30, 31] linking information theory and estimation theory: see [81, 82] for more on this theme.

D. Second law for non-bipartite processes

So far we have discussed learning rates and transfer entropy rates from the strict viewpoint of information exchange between two interacting systems. We now wish to use these concepts to discuss non-equilibrium thermodynamics. In particular, we want to investigate how the second-law inequality involving the learning rate, which provides the tightest lower bound for bipartite processes (see Sec. II D) is modiﬁed when the bipartite assumption is dropped. We stress that we are interested in the average entropy production (EP) during a ﬁnite time interval [0, t] and not only in the stationary state or in the limit t → ∞. Let us again focus on subsystem X. At the ensemble level, an entropy balance equation can be obtained as usual by decomposing the time-derivative of the marginal Shannon entropy SX (t), Z dtSX (t) = − dz JX,t(z)∂x ln Pt(x) , (86) which can be rewritten as Z S dtSX (t) = lX (t) − dzJX,t(z)∂x ln Pt(z) , (87) 18

S after using Eq. (68) to introduce the symmetric learning rate lX (t). Inserting the Fokker-Planck equation and performing a few manipulations, we then obtain Z S irr JX,t(z) dtSX (t) = lX (t) + σX (t) − dz [FbX,t(z) − DXY,t(z)∂y ln Pt(z)] , (88) DXX,t(z) where we have deﬁned the non-negative quantity

Z 2 irr (JX,t(z)) σX (t) ≡ dz , (89) DXX,t(z)Pt(z)

(traditionally referred to as the “irreversible” EP), and the modiﬁed drift FbX,t(z) is deﬁned by [62]

FbX,t(z) ≡ FX,t(z) − ∂xDXX,t(z) − ∂yDXY,t(z) . (90) Eq. (88) can be further transformed as Z irr S DXY,t(z) σX (t) = σX (t) − lX (t) − dz JX,t(z)∂y ln Pt(z) , (91) DXX,t(z)

B B where σX (t) is deﬁned as in the bipartite case by Eq. (20), i.e., σX (t) ≡ dtSX (t) + σX (t), with σX (t) formally deﬁned by [83] Z B JX,t(z) σX (t) ≡ dz FbX,t(z) . (92) DXX,t(z)

Finally, exchanging X and Y in Eq. (79) and using again Eq. (68), we obtain [provided that DXX,t(z) = DXX,t(x)] Z irr † JX,t(z) σX (t) = σX (t) − [T Y →X (t) − T Y →X (t)] + dz ∂yDXY,t(z) . (93) DXX,t(x) So far, we have been only performing mathematical substitutions. However, as already mentioned in Sec. II D, B connection to physics is possible if the thermodynamics of subsystem X can be defined and σX (t) identified as the heat flow from X to the environment, e.g., a thermal bath at a given temperature. In this regard, the existence of correlations between the noises is not an obstacle (see also the related discussion in [75]). For an external observer monitoring only X, the quantity σX (t) defined by Eq. (20) would then be interpreted as the EP of the thermodynamic system X, ignoring that X also influences the dynamics of Y . Note that Eq. (91) implies that σX = 0 at equilibrium, irr S since σX = 0 from Eq. (89) (as JX = 0) and lX = 0, as pointed out in the preceding section. Since the two subsystems are coupled, σX (t) may be negative, but thanks to Eqs. (91) and (93) there is a lower bound, Z S DXY,t(z) σX (t) ≥ lX (t) + dz JX,t(z)∂y ln Pt(z) , (94) DXX,t(z) which is conveniently rewritten as Z † JX,t(z) σX (t) ≥ T Y →X (t) − T Y →X (t) − dz ∂yDXY,t(z) . (95) DXX,t(x)

This inequality takes a remarkably simple form in the case where DXY,t(z) = DXY,t(x), which includes the important case of additive noises, as it reduces to

† σX (t) ≥ T Y →X (t) − T Y →X (t), (96) or, after integration from 0 to T ,

Z T Z T Z T † † ΣX ≡ dt σX (t) ≥ dt [T Y →X (t) − T Y →X (t)] = dt [TY →X (t) − TY →X (t)] , (97) 0 0 0 where we have used Eq. (53) to write the second equality. In practice, the ﬁrst equality may be more useful since the single-time-step TE rates can be extracted more easily from time series than the multi-time-step TE’s. 19

† Since T Y →X (t) − T Y →X (t) = lX (t) in the bipartite case [Eq. (51)], inequality (96) [or its time-integrated version (97)] may be considered as the natural generalization of the second law of information thermodynamics involving the learning rate. As far as we know, it is a new result, which represents a significant outcome of the present work [84]. It can be easily extended to non-bipartite Markov diffusion processes in which the two subprocesses are multidimensional. Anticipating again the calculations performed in Sec. V for a bi-dimensional Ornstein-Uhlenbeck process, we † illustrate in Fig. 2 the generalized second law and show the variations of σX and of the difference TT →X − TY →X = † T Y →X − T Y →X with the parameter ρ quantifying the correlations between the noises affecting X and Y . The ˙int figure also displays the lower bound −[TX→Y + ∆IX→Y ] recently obtained in Ref. [75] by interpreting the various ˙int contributions to the entropy production in terms of computational irreversibilities. The additional term ∆IX→Y vanishes for a bipartite dynamics and the bound is then known to be weaker than the one with the learning rate [see Eq. (24)]. For the specific case shown in Fig. 2, we see that this remains true for ρ 6= 0 (but this is not a ˙int general feature). Note also that TX→Y + ∆IX→Y 6= 0 when the joint system is at equilibrium, in contrast with σX . † Incidentally, we see that one may have TY →X = TY →X even in a non-equilibrium steady state, which occurs when the second term of Eq. (79) is zero.

0.02

-0.02

-0.04

-0.06 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 ρ

FIG. 2: (Color on line) Numerical illustration of the second-law inequality (96) for a stationary bi-dimensional Ornstein- † † Uhlenbeck process: σX (black solid line), TY →X − TY →X = T Y →X − T Y →X (red dashed line). The blue dashed-dotted line ˙int represents −[TX→Y + ∆IX→Y ] (see Eq. (87) in [75]). Note that σX varies linearly with ρ (the explicit expression is given by † Eq. (D34). One has σX = 0 and TY →X = TY →X for ρ = −0.6, which corresponds to equilibrium, as indicated by the small arrow. The model parameters are the same as in Fig. 1.

V. STATIONARY BI-DIMENSIONAL ORNSTEIN-UHLENBECK PROCESS

To make one further step to derive explicit analytical expressions and make the discussion of the consequences of dropping the bipartite property even more concrete, we restrict ourselves to a Gaussian Markov process, more speciﬁcally a bi-dimensional stationary Ornstein-Uhlenbeck (OU) process with additive noises. This allows us to obtain simple analytical expressions for all the information measures, including the multi-time-step TE rates. In the following, we only explain how these quantities can be computed, the details being given in Appendix D. In this Appendix, we also list the expressions of the learning rates and the single-time-step TE rates, which are easily obtained from the general formulas derived previously. To alleviate the notations we now denote the two (one-dimensional) subprocesses by X1 and X2 in place of X and Y and the combined process by X instead of Z: lX will for instance be replaced by l1 and TX→Y by T1→2. The stochastic dynamics is governed by the coupled Langevin equations ˙ Xt = −AXt + ξt , (98) 20 where A = [aij] is a 2 by 2 matrix and ξ = {ξi} is a vector formed by two Gaussian noises with zero mean and 0 0 covariances hξi(t)ξj(t )i = 2Dijδ(t − t ). We assume that all eigenvalues of A have a positive real part so that a stable steady-state solution exists [32, 33]. Since we only focus on this regime hereafter, we may assume that the process has started at t = −∞ and forget about the initial condition [accordingly, the past history of Xi(t) up to time t is − t now denoted by Xi (t) ≡ {Xi(s): s ≤ t} instead of (Xi)0]. The solution of Eq. (98) then reads Z t X(t) = ds H(t − s)ξ(s) , (99) −∞

−At where H(t) = [e ]ij is the response (or Green’s or transfer) functions matrix, and the power-spectrum matrix whose 0 0 elements are the Fourier transform of the stationary correlation functions φij(t) = hXi(t )Xj(t + t)i is given by

S(ω) = H(ω)(2D)HT (−ω) , (100)

R ∞ iωt −1 where H(ω) = −∞ dt e H(t) = [A(ω) − iωI] and 2D is the diﬀusion matrix with elements 2Dij.

A. Expression of the multi-time-step TE rates

An expression of Ti→j for a non-bipartite stationary OU process has already been given in [75], but it turns out that it contains an error, as explained below. Moreover, the derivation is convoluted. We thus believe that it is worth presenting an alternative and much simpler route which has also been recently used for computing the TE rate in the presence of time delay [85]. This is actually a mere application of the formalism presented in [48] for computing Granger causality for discrete and continuous-time autoregressive processes. As has already been mentioned, Granger causality and transfer entropy are identical (up to a factor 1/2) when the random variables are Gaussian distributed [47], which is the case here. By definition, Ti→j is the slope at h = 0 of the finite-horizon TE defined by [86]

− D P (Xj(t + h)|X (t)) E Ti→j(h) ≡ ln − . (101) P (Xj(t + h)|Xj (t)) From the expression of the entropy of Gaussian distributions in terms of their covariance matrix, we then readily obtain

1 σjj,j(h) Ti→j(h) = ln , (102) 2 σjj(h) where

D − 2E σjj(h) ≡ [Xj(t + h) − hXj(t + h)|X (t)i] (103) and

D − 2E σjj,j(h) ≡ [Xj(t + h) − hXj(t + h)|Xj (t)i] (104)

− − are the mean of the variances of the conditional probabilities P (Xj(t+h)|X (t)) and P (Xj(t+h)|Xj (t)), respectively. − − In the language of estimation theory, the conditional expectations hXj(t + h)|X (t)i and hXj(t + h)|Xj (t)i represent the minimum mean-square error (MMSE) estimates of Xj(t + h) when the trajectory up to time t of either the full − − process X or of only Xj is known: They are the orthogonal projections onto X (t) and Xj (t), respectively. As detailed in Appendix D, the calculation of Ti→j amounts to computing first the conditional expectations, then the mean-square errors σjj(h) and σjj,j(h), and finally expanding around h = 0. In particular, this requires to determine the causal factor of the function Sjj(ω), which is a simple task since it is a rational function [54]. The final expression of T1→2 is

1 D12 T1→2 = [r2 − a11 + a21] , (105) 2 D22

2 2 1/2 where r2 = [a11 + (D11/D22)a21 − 2(D12/D22)a11a21] ; T2→1 is of course obtained by interchanging the roles of 1 and 2. As it should be, one can verify that the same result is obtained from Eq. (84), which is actually no more − straightforward because the calculation of the eﬀective drift Fe2 requires to compute the MMSE estimate hX1(t)|X2 (t)i. 21

As noticed above, Eq. (105) differs from the expression of T1→2 obtained in [75] after a lengthy calculation (see also [41]). In this expression, a11 is replaced by |a11| (cf. Eq. (88) in [75]), which is erroneous and may even lead to negative values of T1→2 for a11 < 0. (Having a11 < 0 does not preclude the existence of a stable steady state so long as a11 + a22 > 0 and a11a22 − a12a21 > 0.) More generally, we stress that one must be careful in using a spectral representation of the TE rate, as done for instance in [19] in the bipartite case. Indeed, as discussed in [85], the spectral expression has a limited range of validity (specifically, one must have a11 − (D12/D22)a21 > 0). Otherwise, it underestimates the actual value of the TE rate, as was already pointed out in [89] in the case of discrete-time Granger causality. † From Eq. (105), one obtains the expression of the BTE rate T1→2 by modifying the drifts coefficients according to Eq. (77). For linear Gaussian processes, this simply amounts to changing the matrix A into A∗ = −A + 2DΣ−1 = ΣAT Σ−1 [90, 91], where Σ is the stationary covariance matrix, solution of the Lyapunov equation A.Σ + Σ.AT = ∗ 2D [32, 33]. The quantity r2 is invariant under the transformation A → A , and we obtain

† 1 ∗ D12 ∗ T1→2 = [r2 − a11 + a21] , (106) 2 D22

∗ ∗ with a11 and a21 given by Eqs. (D7). It can be checked that this is in agreement with the expression obtained from † Eq. (54), with T 1→2 and T 1→2 given by Eqs. (D6) and (D8), respectively. On the other hand, the calculation of the ﬁltered TE rate Tb1→2 is more involved. For Gaussian processes, Eq. (39) yields

1 σ11,2(h)σ22,2(h) Tb1→2 = lim ln 2 , (107) h→0 2h σ11,2(h)σ22,2(h) − [σ12,2(h)]

− 2 where σ11,2(h) ≡ h X1(t + h) − hX1(t + h)|X2 (t)i i, σ22,2(h) is deﬁned above by Eq. (104), and σ12,2(h) ≡ h X1(t + − − h) − hX1(t + h)|X2 (t)i X2(t + h) − hX2(t + h)|X2 (t)i i. The calculation, which uses the fact that the conditional − − expectation hX1(t + h)|X2 (t)i is the orthogonal projection of X1(t + h) onto the trajectory X2 (t), is detailed in Appendix D. After some algebra this leads to the expression

2 a11(r2 − a11) Tb1→2 = D22 2 2 . (108) a21D11 − D22(r2 − a11) Finally, as also shown in Appendix D, one may recast the non-Markovian Langevin equation for the marginal process X2 as Z t ˙ −r2(t−s) 0 X2(t) = −(ω+ + ω− − r2)X2(t) − (r2 − ω+)(r2 − ω−) e X2(s)ds + ξ2(t) , (109) −∞ where ω± are the eigenvalues of the matrix A whose expressions are given after Eq. (D16). This formulation is instructive because it shows that a remarkable simpliﬁcation occurs when r2 = ω±, that is when

(r2 − a11)(r2 − a22) − a12a21 = 0 . (110) The second term in the right-hand side of Eq. (109) then vanishes and the equation describes a Markovian dynamics. − Accordingly, one has P (X2(t + h)|X2 (t)) = P (X2(t + h)|X2(t)) and T1→2 becomes equal to its upper bound T 1→2, as discussed in Appendix C. We show in Appendix D that from the perspective of the Kalman-Bucy filter [52–54], where X1(t) is a state variable that is not fully observed and the equation for X2(t) describes the dynamics of the filter state, this means that the observer gain is optimal. Eq. (110) generalizes the conditions for sufficient statistic discussed in Ref. [27] to the non-bipartite case. This may be viewed either as an optimal condition for the set of parameters aij’s for given noise intensities Dij’s or as an optimal condition for the noises for a given set of the aij’s. Note that there are in general two distinct solutions when the eigenvalues ω± of the matrix A are real. Otherwise, there is no real value of r2 that satisfies Eq. (110) and the statistic is never sufficient.

B. Numerical illustration

We now give a numerical illustration which will allow us to discuss the behavior of the various information measures in the presence of correlated noises. To this aim, we consider a situation with a quasi-unidirectional coupling between the two random variables X1 and X2. Indeed, the presence of both bidirectional interactions and correlated noises 22 would make the interpretation of the information exchanges almost impossible. Actually, as far as we know, this case is seldom considered in the literature on Granger causality. Speciﬁcally, we choose the parameters of the model (see the caption of Fig. 1) such that the coupling in the direction 1 → 2 is signiﬁcantly larger than that in the opposite direction, so that one may consider X1 as the source signal (the driver) and X2 as the receiver.

0.3 0.08 (a) (b) 0.06

0.2 0.04

0.02 0.1 0

-0.02 0

-0.04

-0.1 -0.06 -1 -0.5 0 0.5 1 -1 -0.5 0 0.5 1 ρ ρ

FIG. 3: (Color on line) Forward and backward TE rates for a stationary bi-dimensional Ornstein-Uhlenbeck process: (a): (net) †(net) † † † † T1→2 ≡ T1→2 −T2→1 (black solid line), T1→2 ≡ T1→2 −T2→1(red dashed line). (b): T1→2 −T1→2 (black solid line), T2→1 −T2→1 S S (blue dashed line), l2 = −l1 (red dashed-dotted line). The joint system is at equilibrium for ρ = −0.6, as indicated by the small arrow. The model parameters are given in Fig. 1.

On general ground, one is more interested in the net information flow in the system than in the magnitude of the flows in each direction. Inspired by the recent literature on Granger causality [73], and replacing Granger causality (net) (net) by transfer entropy, we then consider various differences of the TE rates, such as T1→2 = −T2→1 ≡ T1→2 − T2→1, †(net) †(net) † † † † T1→2 = −T2→1 ≡ T1→2 − T2→1, and the individual differences T1→2 − T2→1, T2→1 − T2→1. We also compare with S S the behavior of the symmetric learning rate l2 = −l1 . The results are shown in Fig. 3. In the bipartite case (ρ = 0), all quantities behave as expected and indicate that the information mainly flows from 1 to 2, due to the fact that the interaction 1 → 2 predominates (a21/a12 = 10). More precisely, we observe in Fig. (net) †(net) 3a that T1→2 > 0 and T1→2 < 0. The rationale given in the literature (see, e.g., [71–73]) for the second inequality is that the directed information should be reduced (if not reversed) when the temporal order is reversed. We also † † S verify in Fig. 3b that T1→2 − T1→2 = −(T2→1 − T2→1) = l2 > 0. As expected, X2 is “learning about” X1 through its dynamics. One observes that introducing correlations between the noises has a strong effect on most quantities, even if the (net) †(net) properties found for ρ = 0 remain valid in some interval of ρ around 0 (for instance, T1→2 remains larger than T1→2 in the whole interval −0.6 . ρ . 0.3). What appears robust whatever the correlation of the noises is that the net (net) information flow measured by T1→2 always detects the correct dominant interaction. All of the other differences, as well as the learning rates, wildly vary and change sign as ρ varies. There is clearly a complex competition between the feedback (delayed) effects and the instantaneous influence generated by the correlation of the noises. The downside is that this competition sensitively depends on the quantity under study and on other details of the dynamics (for −1 −1 instance on the relative magnitudes of the intrinsic time scales of each subprocesses, i.e., a11 and a22 in the present model). The upside is that one can infer the presence of a strong correlation between the noises if one of these (net) quantities has not the expected sign when T1→2 is positive, which may be a useful piece of information. It is interesting to draw a comparison with the information about the dynamics of the system that could be extracted from the cross-correlation function φ12(t) = hX1(0)X2(t)i. This is indeed a widely used method to infer the directional influence between biological processes [5], as will be evoked in the next section. This function is shown in Fig. 4 for several values of the parameter ρ. The two random variables are generally positively correlated, except at short times when the noises are strongly 23

3.5

2.5

1.5

0.5

-0.5

-1

-1.5 -20 -10 0 10 20 t

FIG. 4: (Color on line) Cross-correlation function φ12(t) = hX1(0)X2(t)i for a stationary bi-dimensional Ornstein-Uhlenbeck process for diﬀerent values of ρ (from top to bottom: ρ = 0.8, 0.6, 0.4, 0.2, 0, −0.2, −0.4, −0.6, −0.8). The red dashed curve corresponds to ρ = 0. The model parameters are the same as in Fig. 1.

anti-correlated (and for ρ & −0.8, φ12(t) < 0 for all t > 0). For ρ & −0.3, the peak at t > 0 indicates that X2(t) correlates more strongly with the value of X1 at an earlier time, suggesting that X1 drives X2. On the other hand, the presence of a maximum for both t > 0 and t < 0 in the interval −0.3 & ρ & −0.7 is hard to interpret and could perhaps erroneously suggest the presence of bidirectional coupling. This again illustrates the subtle competition between the correlation in the noises and the actual feedback.

VI. GENERALIZATION TO A NON-MARKOVIAN PROCESS AND APPLICATION TO THE STUDY OF DIRECTIONAL INFLUENCE BETWEEN CELLULAR PROCESSES

We now consider the application of the general formalism for non-bipartite processes presented in Sec. III to a class of non-Markovian processes. We have in mind a situation that is commonly encountered in biological networks in which one is interested in the information exchanges between two random variables, say X1 and X2 (typically one- dimensional), but other random variables - intrinsic or extrinsic to the system under study - come into play (see, e.g., Fig. 1 in [92]). Within the linear-noise approximation, the network dynamics can be described by a set of chemical Langevin equations where Xi(t) is the deviation of the concentration of species i from its mean value (see, e.g., [4, 93– 95]). The network dynamics corresponds to a multi-dimensional Markov process, but the (coarse-grained) dynamics of the combined process (X1,X2) (and not only the dynamics of the individual processes) is non-Markovian. From this coarse-grained perspective, the system therefore corresponds to a process that is non-bipartite and furthermore non-Markovian in general. This is the case on which we focus below.

A. Extension to a bivariate non-Markovian process

For a bivariate non-Markovian process (X1,X2) obtained as discussed above by coarse-graining a multi-dimensional Markov process over all the extraneous random variables, all the general definitions and relations of information measures introduced in Sec. III directly apply. Furthermore, the Markov character of the underlying network brings a drastic simplification for the derivation of explicit expressions for the information measures, and the latter can be obtained by some form of averaging over the extraneous dynamical variables. Skipping details, and considering only the case of additive noises for simplicity, we find Z ± ¯ l1 (t) = dx1dx2[J1,t(x1, x2) ± D12∂x2 Pt(x1, x2)]∂x1 ln Pt(x2|x1) , (111) 24

Z 1 Pt(x1, x2) 2 2 T 1→2(t) = dx1dx2 [F 2,t(x1, x2) − F 2,t(x2)] , (112) 4 D22

Z † ¯ D12 T 1→2(t) = T 1→2(t) − dx1dx2 J2,t(x1, x2)[∂x2 ln Pt(x1|x2) + ∂x1 ln Pt(x1, x2)] , (113) D22 ¯ R Q R Q with Ji,t(x1, x2) = ( j6=1,2 dxj) Ji,t(x)(i = 1, 2), F 2,t(x1, x2) = ( i6=1,2 dxi) F2,t(x)Pt(x|x1, x2), and F 2,t(x2) = R dx1 F 2,t(x1, x2)Pt(x1|x2), where x denotes all the variables of the network, and Fi,t(x) and Ji,t(x) denote the drift and the probability current for species i in the full multi-dimensional Markov process. The calculation of the multi-time-step TE rates is more involved and is detailed in Appendix E in the special case of the three-dimensional model considered in Sec. VI B. We also do not discuss here the extension of the second law inequalities (95) or (96) as this requires a more extensive and delicate analysis which we defer to future investigations. Indeed, one must ﬁrst decide which components or processes must be taken into account in the theoretical description, how information is transmitted throughout the network, and identify the sources of stochasticity [92]. This is a nontrivial task which is better done on a case by case basis. In particular, the non-bipartite character of the dynamics, i.e., the existence of transitions aﬀecting simultaneously the states of the subsystems, may strongly depend on the level of coarse-graining of the description (see, e.g., [10] for a recent and detailed experimental and theoretical study of a chemical nanomachine).

B. Application to the study of directional inﬂuence between cellular processes

In this final section, we apply the previous framework to revisit and complement the study performed in Ref. [11] about the information transmission between single cell growth rate and gene expression in the metabolism of E. coli. Understanding how fluctuations in gene expression can affect the growth stability of a cell and, in turn, how the growth noise affects gene expression is an important issue that was initially investigated in Ref. [8]. The purpose of Ref. [11] – which actually prompted our concern for the problem of correlated noises – was to show that transfer entropy is a versatile and model-free tool that can be used to infer directional interactions in biochemical networks, in addition to (or possibly as a substitute for) the standard method based on time-delayed cross-correlation functions [5]. A characteristic feature of the biochemical processes studied in [8, 11] is the presence of a common extrinsic noise source affecting both the lac enzyme concentration E(t) and the growth rate µ(t), which makes the stochastic system under study non-bipartite. Specifically, the stochastic model proposed in [8] to account for the experimental data leads to the following equations within the linear response approximation:

X˙ 1 = µ0[(TEE − 1)X1 + TEµX2 + TEGNG + NE]

X2 = TµEX1 + TµGNG + Nµ , (114) where X1(t) ≡ (E(t) − E0)/E0 and X2(t) ≡ (µ(t) − µ0)/µ0 quantify the deviations of E(t) and µ(t) from their mean E and µ , N (t)(l = E, µ, G) are three independent Ornstein-Uhlenbeck (OU) noises that are generated by the 0 0 l √ auxiliary equations N˙ l = −βlNl +ξl where the ξl’s are zero-mean Gaussian white noises with amplitudes θl = ηl 2βl) 0 and Tll0 are logarithmic gains representing how a variable l responds to the fluctuations of a source l (with TEµ = −1 and TµG = 1. In fact, as shown in [11], the intrinsic noise NE(t) affecting the enzyme concentration may be replaced 2 by a delta-correlated noise with the same intensity DE = ηE/βE without deteriorating the quality of the fit to the experimental time-dependent correlation functions for E(t) and µ(t). Then, after some simple manipulations and eliminating the variable NG, Eqs. (114) become equivalent to three coupled linear Langevin equations [11],

X˙ 1 = −[µE + µ0TµE(TEG − 1)]X1 + µ0(TEG − 1)X2 − µ0TEGX3 + ξ1 ˙ X2 = TµE[βG − µE − µ0TµE(TEG − 1)]X1 + [µ0TµE (TEG − 1) − βG]X2 + [βG − βµ − µ0TµETEG]X3 + ξ2

X˙ 3 = −βµX3 + ξ3 (115) where the third equation describes the dynamics of the OU noise X3(t) ≡ Nµ(t) affecting the growth rate [Eqs (115) correspond to Eqs. (59) in [11] where X1,X2 and X3 are denoted x, y and v, respectively]. The rate µE = µ0(1+TµE −TEE) sets the time scale of E fluctuations and the three Gaussian white noises ξ1 ≡ µ0NE, ξ2 ≡ ξµ +ξG + 2 2 2 2 2 2 2 µ0TµENE, ξ3 ≡ ξµ have covariances D11 = DEµ0, D22 = βµηµ + βGηG + DEµ0TµE, D33 = βµηµ, D12 = DEµ0TµE, 2 D13 = 0, D23 = βµηµ. The numerical values of all these parameters are given in Table S1 of [8]. Eqs. (115) define a Markov process for a set of 3 interacting random variables, but we are interested in analyzing the information transfer between X1 and X2 which together form a joint non-Markovian process, as discussed just above. 25

We can thus study the information-theoretic quantities previously introduced. This task was partially accomplished + + in [11] where the single-time-step TE rates T 1→2, T 2→1 and the learning rates l1 , l2 – dubbed information flows – were computed (see note [64] that explains a regrettable confusion in the definition of the learning rates). To † † complement this analysis, we consider the multi-time-step TE rates T1→2, T2→1 and the backward rates T1→2, T2→1 † † and T 1→2, T 2→1. To the best of our knowledge, this is the first time that the backward TE rates are used to infer † † the direction of information exchanges in a real biochemical network. Whereas T 1→2 and T 2→1 are obtained from Eq. (113), the calculation of the multi-time-step TE rates is more involved and is detailed in Appendix E.

Conc. of IPTG Low Intermediate High

ρ12 0.408 0.362 0

ρ23 0.537 0.498 0.135

T1→2 (T 1→2) 0.053 (0.053) 0.020 (0.020) 0 (0)

T2→1 (T 2→1) 0.008 (0.008) 0.001 (0.001) 0.036 (0.036) † T1→2 0.020 0.011 0.026 † −4 T2→1 0.001 < 10 0.010 † T 1→2 0.021 0.011 0.026 † −4 T 2→1 0.002 < 10 0.010 + − l1 = −l2 −0.327 −0.219 0.026 + − l2 = −l1 −0.198 −0.177 −0.026

TABLE II: Theoretical values of the various TE rates for three IPTG concentrations (given in Table S1 of [8]). The time + unit is the inverse of the average growth√ rate µ0. The values of T√i→j (numbers in brackets) and li computed in [11] are also reported. The parameters ρ12 = D12/ D11D22 and ρ23 = D23/ D22D33 quantify the correlations between the white noises ξ1, ξ2 , and ξ2, ξ3 in Eqs. (115).

The numerical results are presented in Tables II and III where the inverse of the average growth rate µ0 is taken + as the time unit. For comparison, the values of T i→j and li computed in [11] are also reported in Table II. As in [8, 11], we consider three different concentrations (low, intermediate, and high) of the inducer IPTG, which allows one to explore different regimes of noise transmission. Comparing the results for T1→2 and T2→1 in Table II to the corresponding results for T 1→2 and T 1→2 obtained in [11] (shown in brackets in the Table), we observe that the numerical values are almost identical. The same is true for the backward rates. In fact, we find numerically that Tij(h) ≈ T ij(h) for all values of the prediction horizon h (recall that the TE rates are given by the slopes of the finite-horizon curves at the origin). More precisely, − − − hXi(t + h)|X1 (t),X2 (t)i ≈ hXi(t + h)|X1(t),X2(t)i and hXi(t + h)|Xi (t)i ≈ hXi(t + h)|Xi(t)i separately, which means that the joint process (X1,X2) and also the marginal processes X1 and X2 can be reformulated (in the stationary regime) as Markov processes to a very good approximation. More details are given in Appendix E. This is an intriguing result that suggests some kind of optimization in the transmission of information. Another indication is the behavior of the single-time-step sensory capacity C1 defined by Eq. (61) as one varies the transmission coefficient TEG describing the response of lac expression to the common noise NG(t) (see Eqs. (114)). As shown in Fig. 5, C1 at low and intermediate IPTG concentrations reaches the maximal value 1 for TEG ≈ 1.5, which is close to the value TEG = 1.3 used in [8] to fit the experimental cross-correlation functions (the fit is actually satisfactory for TEG in the range 1.3 − 1.5). On the other hand, the maximum of C2 occurs for a smaller value of TEG, significantly below the acceptable range [96]. Is this a real feature of the metabolic network? Giving a definite answer to this question is difficult because there are many ingredients in the stochastic description and it is not easy to identify those which are responsible for such a behavior. We leave the discussion of this interesting issue to a future investigation. Finally, we complement the analysis performed in [11] by exploiting our determination of the backward TE rates. As suggested in Refs. [71–73] and tested on time series generated by multivariate autoregressive processes, time-reversed Granger causality, or backward TE in the present framework, may lead to a better estimate of the directionally of (net) †(net) † † information flows. As in Section V B, we thus consider the quantities T1→2 = T1→2 − T2→1, T1→2 = T1→2 − T2→1, † † and the individual differences T1→2 − T1→2 and T2→1 − T2→1. (We could also consider the corresponding quantities built from the single-time-step TE rates but they are almost identical.) The results are presented in Table III where S S we also indicate the values of the symmetric learning rates l1 = −l2 . 26

1 1

0.8 0.8

0.6 0.6

0.4 0.4

0.2 0.2

0 0 1 1.5 2 2.5 1 1.5 2 2.5 T TEG EG

FIG. 5: (Color on line) Single-time-step sensory capacities C1 (left) and C2 (right) as a function of the noise transmission coeﬃcient TEG. Circles, squares, and triangles refer to the low, intermediate, and high IPTG concentrations, respectively. − − C2 → −∞ in the latter case because l2 < 0 and Tb 1→2 = l2 + T 1→2 = 0 as TµE = 0 in Eqs. (114) and X1 no longer inﬂuences X2.

Conc. of IPTG Low Intermediate High (net) T1→2 = T1→2 − T2→1 0.045 0.019 −0.036 †(net) † † T1→2 = T1→2 − T2→1 0.019 0.011 0.016 † T1→2 − T1→2 0.033 0.009 −0.026 † T2→1 − T2→1 0.007 0.001 0.026 S S l1 = −l2 −0.064 −0.021 0.026

−1 TABLE III: Diﬀerences in the TE rates and symmetric learning rates (the time unit is µ0 ).

(net) †(net) † S We first observe that T1→2 > T1→2 , T1→2 − T1→2 > 0 and l2 > 0 at low and intermediate ITPG concentrations, (net) which strengthens the conclusion of [11] (based solely on the positivity of T 1→2 ) that information flows from X1 to X2 in these two cases. That lac fluctuations propagate through the metabolic network and perturb growth was also † the conclusion of [8] based on the corresponding time-correlation functions. Note however that T2→1 − T2→1 is also slightly positive. As discussed in Section V B, this may result from the competition between the correlation in the noises and the direct interactions between the variables X1 and X2. We must also take into account that the process described by Eqs. (115) is significanltly more complicated to than the one discussed in Section V B because of the presence of bidirectional interactions. At high IPTG concentration, all scores consistently indicate that there is a backward transmission from X2 to X1, i.e., from growth to expression, which is again in line with the conclusions of [8] and [11]: see Table 1 in [11] where (net) (net) T 1→2 is computed directly from the experimental time series collected in [8]. Of course, one has T1→2 < 0 and † T1→2 − T1→2 < 0 in this case simply because X1 no longer influences X2 and thus T1→2 = 0. Indeed, the transmission coefficient TµE describing the response of µ to E fluctuations is taken equal to 0 in the stochastic model (see Table † S1 in [8]). On the other hand, a less trivial observation is that the difference T2→1 − T2→1 is significantly more positive than at low and intermediate ITPG concentrations. It would certainly be useful in future investigations to also estimate the backward TE rates directly from the experimental time series. † † S S In passing, note that the numerical results for T1→2 − T1→2, −(T2→1 − T2→1) and l2 = −l1 are equal in the last column of Table III. One could think that this directly results from the fact that (i) the white noises ξ1 and ξ2 are mutually independent in this case (D12 = 0) and (ii) the noise Nµ ≡ X3 no longer affects X2 (as βµ = βG in the 27

eff eff model of Ref. [8]). The reason turns out to be less straightforward. Indeed, the colored noises ξ1 and ξ2 aﬀecting X1 and X2 are still correlated because X3 aﬀects X1 and D23 6= 0 [see Eqs. (E1) and (E2)]. But in the end, when the 3 coupled Langevin equations (115) are replaced by the equivalent Langevin representation for X1 and X2 given by Eqs. (E15), one discovers that in this case the joint process (X1,X2) is both bipartite and quasi-Markov. Accordingly, † S † S S one has T1→2 − T1→2 ≈ l2 and T2→1 − T2→1 ≈ l1 = −l2 .

VII. SUMMARY

In this paper, we have considered information and thermodynamic exchanges between two coupled stochastic systems that do not satisfy the bipartite property. This should correspond to the generic situation in biochemical networks, in which noises can be correlated (for diffusion processes) or transitions of the two systems can simultaneously take place (for jump processes). The generalization of information quantities, such as learning rates and transfer entropy rates, to the non-bipartite situation is non-trivial and involves introducing several additional rates. We have described how this can be achieved in the framework of continuous-time Markov processes. We have also derived several inequalities that are valid beyond the bipartite assumption and generalized a classical relationship between mutual information and causal estimation error. We have illustrated our general formalism on the case of Markov diffusion processes and obtained a new formulation of the second law of information thermodynamics in which the learning rate is replaced by a difference of transfer entropy rates in the forward and backward time directions. Explicit analytical expressions of all information measures have been derived for the special case of a bivariate Ornstein-Uhlenbeck process, allowing a discussion of the influence of the correlation between the noises on the sign and/or the relative magnitude of the various information measures. Finally, we have applied our formalism to the analysis of the directional influence between cellular processes in a concrete example, which required considering the case of a non-bipartite non-Markovian process. An intriguing “optimal” transmission of information has been observed, which calls for future investigations. More generally, it remains as an interesting future work to extend the present study to describe the fluctuations of pathwise thermodynamic and information-theoretic quantities.

Acknowledgements

T.S. is supported by JSPS KAKENHI Grant Number JP16H02211.

Appendix A: Expression of the learning rates for a pure jump Markov process in discrete space

In this Appendix, we give the expression of the learning rate for non-bipartite Markov pure jump processes. In this 0 case, the Markovian generator takes the form (2) in terms of the transition rates Wt (z, z ). After some algebra, the general Markovian expressions (33) and (33) of the learning rates can be cast in the form

0 0 ( + P 0 0 Pt(x )Pt(x,y ) l (t) = 0 Pt(z )Wt (z , z) ln 0 0 X z,z Pt(x)Pt(x ,y ) 0 . (A1) − P 0 0 Pt(x,y)Pt(x ) l (t) = 0 Pt(z )Wt (z , z) ln 0 X z,z Pt(x ,y)Pt(x)

For a bipartite pure jump process, the bipartite transition-rate relation [below Eq. (10)] allows one to show that the two formulas in Eq. A1 coincide, (see also [18, 21])

0 0 0 + − X 0 0 Pt(x )Pt(x, y ) X 0 0 Pt(y |x) 0 0 lX (t) = lX (t) = Pt(z )Wt,y (x , x) ln 0 0 = Pt(z )Wt,y (x , x) ln 0 0 . (A2) Pt(x)Pt(x , y ) Pt(y |x ) x,z0 x,z0

Appendix B: Proof of various relations of the main text

1. Proof of Eq. (40) for a bipartite Markov diﬀusion process 28

t t t To prove Eq. (40) we start with the first line of Eq. (41) and consider < ln[P (Yt+h|X0,Y0 )/P (Yt+h|Y0 ,Xt+h)] > which we rewrite as t t Z 0 0t D P (Yt+h|X0,Y0 ) E 0t 0t 0t 0t P (y, t + h|z , t)P (x, t + h|y 0) ln t = dx dy D[x 0] D[y 0] P (z, t + h; x 0, y 0) ln 0t P (Yt+h|Y0 ,Xt+h) P (z, t + h|y 0) Z 0 R 00 00 0 00 0t 0t 0t 0 0t 0t P (y, t + h|z , t) dx P (x, t + h|x , y , t)P (x , t|y 0) = dx dy D[x 0] D[y 0] P (z, t + h|z , t)P (x 0, y 0) ln R 00 00 0 00 0t (B1) dx P (z, t + h|x , y , t)P (x , t|y 0) 0t 0t where x 0 (resp. y 0) is a short-hand notation to indicate paths of X (resp. Y ) between 0 and t and we have used 0t the fact that the joint process is Markovian. For convenience of notation, integration with the measure D[x 0] (resp. 0t 0 0 D[y 0]) also includes the integration over the initial and final states, this latter being noted x (resp. y ). We can now use the properties of a bipartite process, i.e., the factorization of the infinitesimal Markovian propagator (or transition probability) in Eq. (9) and the decomposition of the Markovian generator in Eq. (10), to write P (z, t + h|z0, t) = P (x, t + h|z0, t)P (y, t + h|z0, t) 0 0 0 0 0 0 2 = δ(x − x )δ(y − y ) + h[δ(y − y )Lt,y0 (x , x) + δ(x − x )Lt,x0 (y , x)] + O(h ). (B2) The specific problem to be handled in the case of diffusion processes in the appearance of singular delta functions in the numerator and denominator of the argument of the logarithm. This can be conveniently bypassed by considering the Gaussian expressions of the infinitesimal propagators, as done in Eq. (73). So, for instance, up to a O(h2), Z 00 00 0 00 0t dx P (z, t + h|x , y , t)P (x , t|y 0)

[y−y0−hF (x00,y0)]2 Z − Y,t 00 00 0t 00 0 1 4hD (x00,y0) = dx P (x , t|y )[δ(x − x ) + hLt,x00 (y , x)] e Y Y,t 0 p 00 0 4πhDY Y,t(x , y ) 0 0 2 0 00 0 2 0t [y−y −hFY,t(x,y )] 00 0t 0 [y−y −hFY,t(x ,y )] − Z 00 − P (x, t|y 0) 4hD (x,y0) 00 P (x , t|y 0)Lt,x (y , x) 4hD (x00,y0) = e Y Y,t + h dx e Y Y,t (B3) p 0 p 00 0 4πhDY Y,t(x, y ) 4πhDY Y,t(x , y ) where there was no need to consider the Gaussian expression for P (x, t + h|x00, y0, t) because the associated delta function is integrated over. In the following we consider the case where DY Y,t(z) is independent of x. Otherwise, the TE rates from X → Y are infinite, as shown in Sec. IV C. One therefore has 0 0 0 0 2 0 0 2 P (y, t + h|z , t) 0t [y − y − hFY,t(x , y )] − [y − y − hFY,t(x, y )] ln R 00 00 0 00 0t = − ln P (x, t|y 0) − 0 dx P (z, t + h|x , y , t)P (x , t|y 0) 4hDY Y,t(y ) [y−y0−hF (x00,y0)]2−[y−y0−hF (x,y0)]2 Z − Y,t Y,t 1 00 00 0t 0 4hD (y0) 2 00 Y Y,t − h 0t dx P (x , t|y 0)Lt,x (y , x)e + O(h ) , (B4) P (x, t|y 0) and Z Z 00 00 0 00 0t 00 00 0 00 0t ln dx P (x, t + h|x , y , t)P (x , t|y 0) = ln dx [δ(x − x ) + hLt,x00 (y , x)]P (x , t|y 0) Z 0t 1 00 00 0t 0 2 00 = ln P (x, t|y 0) + h 0t dx P (x , t|y 0)Lt,x (y , x) + O(h ) , (B5) P (x, t|y 0) so that 0 R 00 00 0 00 0t 0 0 0 P (y, t + h|z , t) dx P (x, t + h|x , y , t)P (x , t|y 0) 0 [FY,t(x , y ) − FY,t(x, y )] ln R 00 00 0 00 0t = (y − y ) 0 dx P (z, t + h|x , y , t)P (x , t|y 0) 2DY Y,t(y ) 0 0 00 0 ! 0 2 0 2 Z 00 0t 0 0 [FY,t(x ,y )−FY,t(x ,y )] [FY,t(z ) − FY,t(x, y ) ] P (x , t|y )Lt,x00 (y , x) −(y−y ) 0 00 0 2DY Y,t(y ) 2 − h 0 + h dx 0t 1 − e + O(h ) . (B6) 4DY Y,t(y ) P (x, t|y 0) One can check that the above expression reduces to a O(h2) when x = x0 and y = y0. As a consequence when combining Eqs. (B1), (B2) and (B6), it only remains, up to a O(h2), t t D P (Yt+h|X0,Y0 ) E ln t P (Yt+h|Y0 ,Xt+h) Z Z 0 0 0t 0t 0t 0t 0 0 0 0 0 [FY,t(z ) − FY,t(x, y )] 0 0 = h D[x 0] D[y 0] P (x 0, y 0) dx dy [δ(y − y )Lt,y (x , x) + δ(x − x )Lt,x (y , x)](y − y ) 0 , 2DY Y,t(y ) (B7) 29 and it is straightforward to see that the above term linear in h exactly vanishes. One can therefore conclude from Eq. (41) that the equality in Eq. (40) is satisfied. Note that we have considered unidimensional processes Xt et Yt for simplicity but the demonstration is easily generalized to multidimensional diffusion processes.

2. Proof of inequality (59)

To prove inequality (59) we consider the diﬀerence

+ d t 1 h t t i lX (t) − [ I(Xt : Y0 ) − TbX→Y (t)] = lim I(Xt+h : Yt) − I(Xt : Yt) − I(Xt+h : Y0 ) + I(Xt : Y0 ) dt h→0 h t 1 D P (Xt|Y0 ) E D P (Xt|Yt) E = lim [ ln t − ln ] , (B8) h→0 h P (Xt+h|Y0 ) P (Xt+h|Yt)

t−0+ Furthermore, the integral analogue of the log-sum inequality applied to the trajectory Y0 yields t D P (Xt+h,Xt,Y0 )E D P (Xt+h,Xt,Yt)E ln t ≥ ln . (B10) P (Xt+h,Y0 ) P (Xt+h,Yt) As a result,

3. Proof of the expression (79) for the single-time-step BTE rates for a Markov diﬀusion process

† For the same reason as T X→Y (t), the BTE rate T X→Y (t) is inﬁnite when DY Y,t(z) 6= DY Y,t(y). We thus consider in the following the case where DY Y,t(z) = DY Y,t(y) = DY Y,t(y). ∗ Inserting the expression of FY,t(z) given in Eq. (77) into Eq. (75), we obtain

∗ 2 ∗ 2 2 2 FY,T −t(z) − [F Y,T −t(y)] = FY,t(z) − [F Y,t(y)] ¯ JY,t(z) JY,t(y) − 4 2 ∂x[DXY,t(z)Pt(z)] + ∂y[DY Y,t(y)Pt(z)] + 4 2 ∂y[DY Y,t(y)Pt(y)] , (B12) Pt(z) Pt(y) where J¯Y,t(y) = F Y,t(y)Pt(y) − ∂y[DY Y,t(y)Pt(y)]. After combining Eqs. (75), (78) and (B12), we arrive at

† T X→Y (t) − T X→Y (t) Z Z JY,t(z) JY,t(z) = − dz ∂x[DXY,t(z)Pt(z)] − dz [DY Y,t(y)∂yPt(z) + Pt(z)∂yDY Y,t(y)] DY Y,t(y)Pt(z) DY Y,t(y)Pt(z) Z J¯Y,t(y) + dy [DY Y,t(y)∂yPt(y) + Pt(y)∂yDY Y,t(y)] , (B13) DY Y,t(y)Pt(y) ∗ ∗ where we have used the fact that DY Y,T −t(y) = DY Y,t(y) and PT −t(z) = Pt(z). With a few manipulations and the R help of the relation dx JY,t(z) = J¯Y,t(y), the above equation can be rewritten as Z Z Z † JY,t(z) ¯ T X→Y (t) − T X→Y (t) = − dz ∂x[DXY,t(z)Pt(z)] − dz JY,t(z)∂y ln Pt(z) + dy JY,t(y)∂y ln Pt(y) DY Y,t(y)Pt(z) Z Z JY,t(z) = − dz ∂x[DXY,t(z)Pt(z)] − dz JY,t(z)∂y ln Pt(x|y) , (B14) DY Y,t(y)Pt(z) which corresponds to Eq. (79) of the main text. 30

Appendix C: Suﬃcient statistic for a non-bipartite dynamics

(M) = P (Xt+h|Yt) , (C2) which yields from Eq. (B8)

t + d t 1 D P (Xt|Y0 )P (Xt+h|Yt)E (M) lX (t) − [ I(Xt : Y0 ) − TbX→Y (t)] = lim ln t = 0 . (C3) dt h→0 h P (Xt+h|Y0 )p(Xt|Yt) Therefore, the steady-state inequality (58) is also saturated. t Finally, since P (Xt+h,Yt+h|Y0 ) = P (Xt+h,Yt+h|Yt), one also has TbX→Y = Tb X→Y and inequality (56) is saturated (note that the joint process does not need to be Markovian in this case).

Appendix D: Analytical expressions for a stationary bi-dimensional Ornstein-Uhlenbeck process

In this appendix, we give the analytical expressions of the learning rates and various TE rates for a stationary bi-dimensional OU process.

1. Learning rates and single-time-step TE rates

These rates are obtained from the general formulas for Markov diﬀusion processes derived in Sec. IV. To this aim, we use the expression of the stationary probability distribution function

1 − 1 xT Σ−1x P (x) = e 2 , (D1) st 2π|Σ|1/2 where Σ is the covariance matrix whose elements σij ≡ hXi(0)Xj(0)i are obtained by solving the Lyapunov equation. This gives

2 (a22 + a11a22 − a12a21)D11 + a12(a12D22 − 2a22D12) σ11 = (a11a22 − a12a21)(a11 + a22) 2 (a11 + a11a22 − a12a21)D22 + a21(a21D11 − 2a11D12) σ22 = (a11a22 − a12a21)(a11 + a22) 2a11a22D12 − a22a21D11 − a11a12D22 σ12 = σ21 = . (D2) (a11a22 − a12a21)(a11 + a22)

2 One can check that σ11 and σ22 are positive when the two conditions a11a22 − a12a21 > 0 and D11D22 − D12 ≥ 0 are satisﬁed. Some useful relations among these quantities are obtained by multiplying the stationary Fokker-Planck 31

2 2 equation by x1, x2 and x1x2, respectively, and integrating over x. This gives

a11σ11 + a12σ12 = D11 (D3a)

a22σ22 + a21σ21 = D22 (D3b)

(a11 + a22)σ12 + a21σ11 + a12σ22 = 2D12 . (D3c) From Eq. (67), we then ﬁnd

+ − σ12 σ12 l1 = −l2 = − (a12 + D11) σ11 |Σ| σ = a − 22 D , (D4) 11 |Σ| 11 where we have used relation (D3a) to go from the ﬁrst to the second line. Likewise, the symmetric learning rate S S l1 = −l2 given by Eq. (68) reads σ D − σ D lS = a − 22 11 12 12 . (D5) 1 11 |Σ| R The TE rate T 1→2 is obtained from Eq. (75), using F 2(x2) = −a21 dx1x1p(x1|x2)−a22x2 = −(a21σ12/σ22+a22)x2, and thus F2(x) − F 2(x2) = −a21x1 + a21(σ12/σ22)x2. This yields

2 1 a21|Σ| T 1→2 = , (D6) 4D22 σ22 and for D12 = 0 one recovers the expressions already given in the literature [23, 27]. † The BTE rate T 1→2 is obtained by modifying the drifts according to Eq. (77), which amounts to replacing A by A∗ = ΣAΣ−1 [91]. The modiﬁed drifts are (a σ + a σ )σ − (a σ + a σ )σ a∗ = 11 11 12 12 22 21 11 22 12 12 11 |Σ| a σ2 − a σ2 + (a − a )σ σ a∗ = 12 22 21 12 11 22 22 12 (D7) 21 |Σ|

∗ ∗ with a22 and a12 obtained by interchanging 1 and 2. Inserting these expressions into Eq. (D6) and noting that the determinant |Σ| of the covariance matrix is invariant under the transformation A → A∗, we obtain after some algebra

2 † (a21|Σ| + 2D22σ12 − 2D12σ22) T 1→2 = . (D8) 4D22|Σ|σ22 Finally, Eq. (49) yields 2 2D12σ22 − a21|Σ| Tb 1→2 = . (D9) 4D22σ22|Σ|

2. Multi-time-step TE rates

We now detail the calculation of the multi-time-step TE rates T1→2 and Tb1→2. As explained in the main text, this − − − requires to compute the conditional expectations hX2(t + h)|X (t)i, hX2(t + h)|X2 (t)i, hX1(t + h)|X2 (t)i, and the corresponding mean-square errors σ22(h), σ22,2(h), and σ11,2(h), σ12,2(h). − a) We ﬁrst compute hX2(t + h)|X (t)i and σ22(h), which is is straightforward. Starting from Z t+h X2(t + h) = ds [H21(t + h − s)ξ1(s) + H22(t + h − s)ξ2(s)] , (D10) −∞ we readily get Z t D − E X2(t + h)|X (t) = ds [H21(t + h − s)ξ1(s) + H22(t + h − s)ξ2(s)] , (D11) −∞ 32 since the noises are ﬁxed for s ≤ t by Eq. (98) and average to zero in the time interval [t, t + h]. Eq. (103) then yields

Z h 2 2 σ22(h) = 2 dt [D11H21(t) + D22H22(t) + 2D12H21(t)H22(t)] . (D12) 0 Of course, since the joint process is Markovian, X−(t) can be replaced by X(t) in Eqs. (101) and (103), and the same result could be obtained from the well-known expression of the transition probability of the OU process [32, 33]. − b) The calculation of hX2(t + h)|X2 (t)i and thus σ22,2(h) is less straightforward because fixing only the past of the process X2 up to time t is not sufficient to fix the past of the noises ξ1 and ξ2. However, this difficulty is bypassed by 0 determining a causal function H22(t) such that Z t 0 0 X2(t) = ds H22(t − s)ξ2(s) , (D13) −∞

0 where ξ2(t) is another Gaussian white noise with variance 2D22. Then, following the same reasoning as above, we get Z t − 0 0 hX2(t + h)|X2 (t)i = ds H22(t + h − s)ξ2(s) , (D14) −∞ and in turn Z h 02 σ22,2(h) = 2D22 dt H22(t) . (D15) 0 0 0 0 Since Eq. (D13) implies that S22(ω) ≡ hX2(ω)X2(−ω)i = 2D22H22(ω)H22(−ω), the Fourier transform of H22(t) is simply obtained by identifying the component of S22(ω) that is analytic in the upper-half plane Im(ω) > 0. Speciﬁcally,

2 S22(ω) = h|H21(ω)ξ1(ω) + H22(ω)ξ2(ω)| i , (D16) √ 1 where H21(ω) = −a21/[(ω+ −iω)(ω− −iω)] and H22(ω) = (a11 −iω)/[(ω+ −iω)(ω− −iω)] with ω± = 2 (a11 +a22 ± ∆) 2 and ∆ = (a11 − a22) + 4a12a21 (ω± are the eigenvalues of A). This yields

2 2 r2 + ω S22(ω) = 2D22 2 2 2 2 , (D17) (ω+ + ω )(ω− + ω ) where r 2 D11 2 D12 r2 = a11 + a21 − 2 a11a21 . (D18) D22 D22 It then comes that

0 r2 − iω H22(ω) = . (D19) (ω+ − iω)(ω− − iω)

0 0 By construction, H22(ω) has no poles in the upper-half plane Im(ω) > 0 and therefore H22(t) vanishes for t√ < 0 0 −ω+t −ω−t [specifically, H22(t) = (u+e − u−e )Θ(t) where Θ(t) is the Heaviside step function and u± = (ω± − r2)/ ∆]. p 2 0 Moreover, we have chosen r2 = r2 > 0 so that H22(ω) is also zero-free in this region. As stressed in [85], this condition (which corresponds to the so-called “minimum-phase” condition in the language of control theory [97, 98]) ensures that there is a one-to-one correspondence between the process and the corresponding forcing white noise: 0 Fixing the history of the process X2 up to time t is equivalent to fixing the history of the noise ξ2(t) and vice versa. + The TE rate T1→2 is finally obtained by expanding σ22(h) and σ22,2(h) in powers of h. Using H21(t = 0 ) = 0 and + 0 + H22(t = 0 ) = H22(t = 0 ) = 1, we get ˙ 0 + 2 3 1 2D22[h + H22(t = 0 )h + O(h )] T1→2(h) = ln , (D20) + + 2 3 2 2D22h + 2[D22H˙ 22(t = 0 ) + D12H˙ 21(t = 0 )]h + O(h ) and then

1 ˙ 0 + ˙ + D12 ˙ + T1→2 = [H22(t = 0 ) − H22(t = 0 ) − H21(t = 0 )] . (D21) 2 D22 33

˙ + ˙ + ˙ 0 + Since H22(t = 0 ) = −a22, H21(t = 0 ) = −a21, and H22(t = 0 ) = r2 − (ω+ + ω−) = r2 − a11 − a22, we ﬁnally 0 arrive at Eq. (105) in the main text. Moreover, inserting the expression of H22(t) into Eq. (D13) and performing a few manipulations, we can transform this equation into the (non-Markovian) Langevin equation (109). − c) Finally, we consider the ﬁltered TE rate Tb1→2. This requires to calculate the conditional mean hX1(t+h)|X2 (t)i. − To this aim, we use the fact that hX1(t + h)|X2 (t)i is the orthogonal projection of X1(t + h) onto the trajectory − − X2 (t). It is a linear functional of X2 (t), Z t − hX1(t + h)|X2 (t)i = f(h)X2(t) + ds g(t − s, h)X2(s) , (D22) −∞ where f(h) and g(t, h) are unknown functions to be determined by satisfying the orthogonality condition,

− [X1(t + h) − hX1(t + h)|X2 (t)i]X2(s) = 0 ∀s ≤ t . (D23)

Note that we have singled out the dependence on the value of X2 at time t from the dependence on the trajectory up to to time t. Plugging Eq. (D22) into Eq. (D23), we then obtain the integral equation

Z t φ21(t + h) = f(h)φ22(t) + ds g(t − s, h)φ22(s) ∀ t ≥ 0 , (D24) −∞

0 0 where φij(t) ≡ hXi(t )Xj(t + t)i. This equation is easily solved by using the fact that the stationary correlation functions φij(t) of an OU process are sums of exponentials [33]. Speciﬁcally, for t ≥ 0,

−ω+t −ω−t D22 2 2 e 2 2 e φ21(t) = 2 2 [(r2 − ω+)(a22 − ω+) − (r2 − ω−)(a22 − ω−) ] a21(ω+ − ω−) ω+ ω− −ω+t −ω−t D22 2 2 e 2 2 e φ22(t) = 2 2 [(ω+ − r2) − (ω− − r2) ] , (D25) ω+ − ω− ω+ ω− where ω± are the eigenvalues of the matrix A defined after Eq. (D16). This suggests to introduce the exponential ansatz g(t, h) = g(h)e−λt. Then, by comparing the left-hand and right-hand sides of Eq. (D24), we readily find that λ = r2 (defined by Eq. (D18)), and the remaining unknown functions f(h) and g(h) are obtained by identifying the coefficients of e−ω+t and e−ω−t. After some algebra, we get

(r − ω )(a − ω )e−ω+h − (r − ω )(a − ω )e−ω−h f(h) = 2 + 22 + 2 − 22 − a21(ω+ − ω−) (r − ω )(r − ω )[(ω − a )e−ω+h − (ω − a )e−ω−h] g(h) = 2 + 2 − + 22 − 22 . (D26) a21(ω+ − ω−) In particular, when h = 0, a − r f(0) = 11 2 a21 (r − ω )(r − ω ) g(0) = 2 + 2 − . (D27) a21

− From the expression of hX1(t + h)|X2 (t)i, we can easily compute the mean-square errors σ11,2(h) and σ12,2(h), and expanding in powers of h finally leads to Eq. (108) in the main text. Interestingly, the above calculation has a connection to classical linear filtering and stochastic control theory [54, 97, 98]. To cast the model described by the coupled Langevin equations (98) as a linear control problem of the state-space form, we introduce two yet unspecified (but non-zero) parameters G and K such that

GK = a12

G − a21K = a22 − a11 (D28) and the variable

Xb1(t) = KX2(t) . (D29) 34

The Langevin equation for X1(t) and X2(t) can then be rewritten as ˙ X1(t) = −a11X1(t) − GXb1(t) + ξ1(t) (D30a) ˙ Xb 1(t) = −a11Xb1(t) − GXb1(t) + K[−a21X1(t) + ξ2(t) + a21Xb1(t)] . (D30b)

One realizes that, provided one defines an “observation variable” Y (t) ≡ −a21X1(t)+ξ2(t), the above set of equations can be interpreted as describing the dynamics of a state variable X1(t) (the “signal process”) and of its linear estimator − Xb1(t) given the trajectory Y (t) ≡ {Y (s)}s∈[−∞,t] (see also [99]). In this representation, G and K denote the control gain and the observer (or Kalman) gain, respectively [100]. The first two terms in the r.h.s. of Eq. (D30b) copy the model of the signal process [including the linear control feedback −GXb1(t)] whereas the third term, which can also be written as Y (t) + a21Xb1(t), corrects the model by a linear feedback that drives Xb1 closer to X1 at time t + dt. The process e(t) ≡ Y (t) + a21Xb1(t) is the so-called “innovation”, which is the part of the measurement that provides new information about the state of the system [53]. ∗ In the standard filtering problem one looks for the linear estimate Xb1 (t) that minimizes the mean-square error 2 ∗ − P (t) = h[X1(t) − Xb1(t)] i at every time t. This optimal estimate is known to be Xb1 (t) = hX1(t)|Y (t)i, the − orthogonal projection of X1(t) onto the trajectory Y (t). The main insight of the Kalman-Bucy theory is to convert the associated Wiener-Hopf integral equation into a nonlinear differential equation by exploiting the state-space formulation [53]. The specification of the optimal Kalman-Bucy filter thus amounts to computing the gain K∗ that minimizes the estimation error variance P (t). In the present stationary case, one simply has (cf. Eqs. (III) and (IV) 0 in section 7 in [52] with F (t) = −a11,H (t) = −a21,G(t) = 1,Q(t) = 2D11,R(t) = 2D22)

∗ ∗ P K = −a21 , (D31) 2D22 and P ∗ is solution of the algebraic Ricatti equation

∗ 2 ∗ 2 (P ) −2a11P − a21 + 2D11 = 0 . (D32) 2D22 The physical solution is a − r K∗ = 11 2 , (D33) a21 which can be shown to also be the solution of the Wiener-Kolmogorov causal filter [101–103] (as the two filters are 2 equivalent in the stationary case). Plugging this expression into the quadratic equation a21K +(a22 −a11)K −a12 = 0 obtained by eliminating G from Eqs. (D28), one then finds that the optimal filter is obtained when r2 satisfies Eq. ∗ (110) of the main text, i.e., r2 = r2 = ω±. This condition of course requires specific relations between the parameters of the original bi-dimensional Ornstein-Uhlenbeck model. When this occurs, the individual dynamics of X2(t) [or, − equivalently, Xb1(t|Y (t))] is then Markovian, which is no coincidence. Indeed, as is well known, the innovation e(t) is a white noise process with the same variance R as the measurement noise when the filter is optimal [53]. This fact underscores the property that the Kalman-Bucy filter is so efficient at extracting information from the measurements ∗ − that one is just left with a white noise afterwards. The filter state Xb1 (t|Y (t)) is then a sufficient statistic for the ∗ ∗ conditional distribution of X1(t). In passing, note that r2 and in turn K do not depend on the parameter a12 that defines the feedback from 2 to 1. This illustrates the separation principle in optimal control theory that states that the optimal observer gain obtained by minimizing the estimation error is independent of the cost of the control and therefore is independent of the feedback gain G. 3. Entropy production rate

In the steady state, the entropy production rate in system 1 is equal to the rate of entropy change in the environment, and is computed from Eq (92) where FbX,t, DXX,t, and JXX,t are replaced by F1(x) = −a11x1 − a12x2, D11, and

J1(x) = F1(x)P (x) − D11∂x1 P (x) − D12∂x2 P (x), respectively. This yields

B a12 σ1 = σ1 = [a12D22 − a21D11 + (a11 − a22)D12] . (D34) (a11 + a22)D11 35

Appendix E: Multi-time-step TE rates for the three-component process of Sec. VI B

In this Appendix, we generalize the calculations of Sec. V to the case of a multi-dimensional OU process. Since the complexity of the calculation rapidly increases with the number of components, we only consider the stochastic model studied in section VI B which remains simple because the third Langevin equation describing the dynamics of the OU colored noise is decoupled from the two other equations [see Eqs. (115)]. The matrix A in Eq. (98) is now a 3 × 3 matrix with a31 = a32 = 0, and a33 > 0. The main difference with the bi-dimensional Markov case is that the (coarse-grained) dynamics of the joint process of interest, X1,2 ≡ (X1,X2), is no longer Markovian because the effective noises affecting the two sub-processes are colored, Z t eff −a33(t−s) ξi (t) = ξi(t) − ai3X3(t) = ξi(t) − ai3 ds e ξ3(s), i = 1, 2 , (E1) −∞ with covariances

a 0 eff eff 0 0 i3 −a33|t−t | hξi (t)ξi (t )i = 2Diiδ(t − t ) − ai3(2Di3 − D33)e a33 0 0 a a 0 eff eff 0 0 −a33(t−t ) 0 −a33(t −t) 0 13 23 −a33|t−t | hξ1 (t)ξ2 (t )i = 2D12δ(t − t ) − 2a13D23e Θ(t − t ) − 2a23D13e Θ(t − t) + D33e . a33 (E2)

− Therefore, X1,2(t) cannot be replaced by X1,2(t) in Eqs. (101) and (103), and Eq. (D11) (or the corresponding − equation for hX1(t + h)|X1,2(t)i) is no longer valid because the noises in the time interval [t, t + h] do not average to zero (as they are correlated with the noises for s ≤ t, which are ﬁxed). To circumvent the problem, we need again to whiten the noises, which amounts to determining four causal functions Heij(t) such that Z t Xi(t) = ds [Hei1(t − s)ξe1(s) + Hei2(t − s)ξe2(s)], i = 1, 2 , (E3) −∞

0 0 where ξe1(t) and ξe2(t) are Gaussian noises with zero-mean and covariances hξei(t)ξej(t )i = 2Dijδ(t − t ). Then, Eq. (D12) is just replaced by

Z h 2 2 σ22(h) = 2 dt [D11He21(t) + D22He22(t) + 2D12He21(t)He22(t)] , (E4) 0 and σ11(h) is given by the symmetric equation. By construction, the matrix He (ω) = [Heij(ω)] must satisfy the equation

S(2)(ω) = He (ω)(2D)He T (−ω) , (E5)

(2) where S (ω) is the 2 × 2 sub-matrix of S(ω) with elements Sij(ω) = hXi(ω)Xj(−ω)i for i, j = 1, 2. Speciﬁcally,

2 ω + fijω + gij Sij(ω) = 2Dij 2 2 2 2 , (E6) (ω + ω+)(ω + ω−) where ω± are defined in Appendix D after Eq. (D16), f11 = f22 = 0, and f12, g11, g12, g22 are complicated functions of the model parameters. Note that the numerator and the denominator of Sij(ω) are only quadratic polynomials despite the fact that the OU process defined by Eqs. (115) is trivariate. This simplification occurs because a31 = a32 = 0 and we also set βG = βµ (as the two timescales 1/βG and 1/βµ are taken equal to the measured autocorrelation time of µ when fitting the model parameters to experiments [8]). In order to solve Eq. (E5), we seek a matrix He (ω) = [Heij(ω)] in the form ! 1 ω11 − iω ω12 He (ω) = , (E7) (ω+ − iω)(ω− − iω) ω21 ω22 − iω

+ imposing limω→∞ −iωHe (ω) = 1 so that the response functions satisfy He (t = 0 ) = 1. By matching the coeﬃcients n of ω in the numerators of Sij(ω), we can then determine the four unknown quantities ωij. The corresponding nonlinear equations are solved numerically and the correct solution is the one that ensures that the elements of the matrix 36

He −1(ω) have no poles in the upper-half plane Im(ω) > 0 (this is the generalization of the minimum-phase condition introduced in Appendix D). 0 It remains to calculate σ22,2(h) from Eq. (D15), where H22(t) is the response function deﬁned by Eq. (D13) and obtained from the Wiener-Hopf factorization of S22(ω). This yields 0 0 ω − iω H22(ω) = , (E8) (ω+ − iω)(ω− − iω)

0 2 where ω is the root of the equation ω + g22 with a positive real part. 0 + + ˙ 0 + Finally, expanding σ22(h) and σ22,2(h) in powers of h, using H22(t = 0 ) = He22(t = 0 ) = 1, H22(t = 0 ) = 0 ˙ + ˙ + ω − (a11 + a22), He 22(t = 0 ) = ω22 − (a11 + a22), and He 21(t = 0 ) = ω21, we obtain ˙ 0 + 2 3 1 2D22h + 2D22H22(t = 0 )h + O(h ) T1→2(h) = ln 2 ˙ + ˙ + 2 3 2D22h + [2D22He 22(t = 0 ) + 2D12He 21(t = 0 )]h + O(h ) 0 2 3 1 D22h + D22(ω − a11 − a22)h + O(h )] = ln 2 3 , (E9) 2 D22h − [D22(ω22 − a11 − a22) + D12ω21]h + O(h ) and thus

1 0 D12 T1→2 = [ω − ω22 − ω21] . (E10) 2 D22

T2→1 is given by the symmetric formula. In order to compute the backward TE rates, we again have to replace the aij’s by the elements of the matrix A∗ = ΣAT Σ−1. However, this simply amounts to repeating the calculation of the forward TE rates after interchanging † the power-spectrum functions S12(ω) and S21(ω) [as S12(ω) = S21(ω)]. It is useful to illustrate the above equations numerically to see why the marginal processes X1 and X2 have a quasi-Markovian behavior. Let us for instance consider the case of the low-IPTG experiment. The three coupled Eqs. (115) are then equivalent to the two non-Markovian equations ˙ eff X1 = −0.209X1 + 0.069X2 + ξ1 ˙ eff X2 = 0.084X1 − 0.282X2 + ξ2 , (E11) with

eff eff 0 0 −0.33|t−t0| hξ1 (t)ξ1 (t )i = 2D11 δ(t − t ) − 0.003 e eff eff 0 0 −0.33|t−t0| hξ2 (t)ξ2 (t )i = 2D22 δ(t − t ) + 0.001 e eff eff 0 0 −0.33(t−t0) 0 −0.33|t−t0| hξ1 (t)ξ2 (t )i = 2D12 δ(t − t ) − 0.005 e Θ(t − t ) + 0.002 e , (E12) and 2D11 = 0.020, 2D22 = 0.059, and 2D12 = 0.014. Although the exponential terms seem to only give small corrections, these noises cannot be approximated by their white components. Indeed, this would signiﬁcantly change −3 the values of the TE rates and even reverse the directionality of the information ﬂow (T1→2 ≈ 2.2 × 10 , T2→1 ≈ 8.6×10−3, to be compared with the values given in Table II). On the other hand, by taking the inverse of the response functions Heij(ω), we obtain from Eqs. (E3) the equivalent representation Z t ˙ −0.398(t−s) X1 = −0.069X1 − 0.034X2 − ds e [0.018X1(s) − 0.002X2(s)] + ξe1 −∞ Z t ˙ −0.398(t−s) X2 = 0.183X1 − 0.354X2 − ds e [0.012X1(s) − 0.002X2(s)] + ξe2 , (E13) −∞

0 0 with hξei(t)ξej(t )i = 2Dijδ(t − t ). Moreover, by using Eq. (D13) and the similar equation for X1, and taking the 0 inverse of the response functions Hii(ω), the individual processes X1 and X2 can be represented by Z t ˙ −0.402(t−s) 0 X1 = −0.089X1 − 0.017 ds e X1(s) + ξ1 −∞ Z t ˙ −0.205(t−s) 0 X2 = −0.286X2 + 0.005 ds e X2(s) + ξ2 , (E14) −∞ 37

0 0 0 0 with hξi(t)ξj(t )i = 2Dijδ(t − t ). Let us recall that, by construction, Eqs. (E13) and (E14) yield the same statistical properties of the stationary processes X1 and X2 as the original Eqs. (115). Although these are still non-Markovian equations, neglecting the contributions of the memory kernels is now harmless and leads to exactly the same values of the TE rates as the full equations. Likewise, for the high-IPTG experiment, the three coupled Eqs. (115) are equivalent to the two equations (similar to Eqs. (E13))

Z t ˙ −3.254(t−s) X1 = −1.009X1 + 0.232X2 − ds e [0.053X1(s) + 0.006X2(s)] + ξe1 −∞ ˙ X2 = −3.230X2 + ξe2 . (E15)

In this case the white noises ξe1 and ξe2 are independent, as D12 = 0.

[1] C. G. Bowsher and P. S. Swain, Curr. Opin. Biotechnol. 28, 149 (2014). [2] J. M. Parrondo, J. M. Horowitz, and T. Sagawa, Nature Physics 11, 131 (2015). [3] M. B. Elowitz et al., Science 297,118 (2002). [4] S. Tˇanase-Nicola,P. B. Warren, and P. R. ten Wolde, Phys. Rev. Lett. 97, 068102 (2006); q-bio.MN/0508027 (2006). [5] M. J. Dunlop, R. S. Cox, J. H. Levine, R. M. Murray, and M. B. Elowitz, Nat. Genet., 40,1493 (2008). [6] M. Ullah and O. Wolkenhauer, Stochastic Approaches for Systems Biology (Springer, 2011). [7] C.C. Govern and P. R. ten Wolde, Phys. Rev. Lett. 113, 258102 (2014). [8] D. J. Kiviet et al., Nature 514, 376 (2014). [9] M. Hinczewski and D. Thirumalai, J. Phys. Chem. B, 120, 6166 (2016). [10] D. Loutchko, M. Eisbach, and A. S. Mikhailov, J. Chem. Phys. 146, 025101 (2017). [11] S. Lahiri, P. Nghe, S. J. Tans, M. L. Rosinberg, and David Lacoste, PLoS ONE 12(11): e0187431 (2017). [12] A. E. Allahverdyan, D. Janzing, and G. Mahler, J. Stat. mech. P09011 (2009). [13] A.C. Barato, D. Hartich, and U. Seifert, Phys. Rev. E 87, 042104 (2013); J. Stat. Phys. 153, 460 (2013). [14] S. Ito and T. Sagawa, Phys. Rev. Lett. 111, 180603 (2013). [15] J. M. Horowitz and M. Esposito, Phys. Rev. X, 4, 031015 (2014). [16] G. Diana and M. Esposito, J. Stat. mech. P04010 (2014). [17] A.C. Barato, D. Hartich, and U. Seifert, New. J. Phys. 16, 103024 (2014). [18] D. Hartich, A. C. Barato, and U. Seifert, J. Stat. mech. P02016 (2014). [19] J. M. Horowitz and H. Sandberg, New J. Phys. 6, 125007 (2014). [20] S. Ito and T. Sagawa, Nat. Commun. 6, 7498 (2015). [21] N. Shiraishi and T. Sagawa, Phys. Rev. E 91, 012130 (2015). [22] J. M. Horowitz, J. Stat. Mech. P03006 (2015). [23] D. Hartich, A. C. Barato, and U. Seifert, Phys. Rev. E 93, 022116 (2016). [24] R. E. Spinney, J. T. Lizier, and M. Prokopenko, Phys. Rev. E 94, 022135 (2016). [25] S. Ito, Scientific Reports 6, 36831 (2016). [26] M. L. Rosinberg and J. M. Horowitz, Euro. Phys. Lett. 116,10007 (2016). [27] T. Matsumoto and T. Sagawa, Phys. Rev. E 97, 042103 (2018). [28] S. Ito, arXiv: 1810.09545 (2018). [29] T. M. Cover and J. A. Thomas, Elements of information theory, 2nd Ed., Wiley, New York (2006). [30] T. E. Duncan, SIAM J. Appl. Math. 19, 215 (1970). [31] T. E. Duncan, Inform. Control. 19, 265 (1971). [32] H. Risken, The Fokker-Planck Equation - Methods of Solution and Applications (Springer, Berlin, 1989). [33] W. C. Gardiner, Handbook of stochastic Methods, 3rd edition. (Springer, Berlin, 2004). [34] K.L. Chung and J. B. Walsh, Markov Processes, Brownian Motion, and Time Symmetry, Springer Science, 2nd Ed. (2005). [35] S. N. Ethier and T. G. Kurtz, Markov Processes : Characterization and Convergence, Wiley Series in Probability and Mathematical Statistics, (New York 2005). [36] L. C. G. Rogers and D. Williams, Diffusions, Markov Processes and Martingales, Cambridge University Press (2000). 1 R t [37] More generally, one can define the asymptotic flow lX ≡ limt→∞ t 0 ds lX (s). Of course, this quantity identifies with the learning rate of [17, 23] in the stationary regime. Asymptotic quantities can be defined in a similar way for the other information measures introduced in the paper. [38] T. Schreiber, Phys. Rev. Lett. 85, 461 (2000). [39] M. Paluˇs,V. Komárek,Z. Hrnˇc´ıˇr,and K. Stˇerbová,Phys.ˇ Rev. E 63, 046211 (2001). [40] T. Bossomaier, L. Barnett, M. Harré,and J.T. Lizier, An Introduction to Transfer Entropy: Information Flow in Complex Systems (Springer, 2016). [41] R. E. Spinney and J. T. Lizier, Phys. Rev. E 98, 012314 (2018). 38

[42] T. Weissman, Y-H Kim, and H. H. Permuter, IEEE Trans. Inform. Theory 59, 1271 (2013). [43] C. W. J. Granger, Econometrica, 37, 424 (1969). [44] P.-O. Amblard and O. J. J. Michel, Entropy 15, 113 (2013). [45] H. Lütkepohl, New introduction to multiple time series analysis (Springer Berlin 2005). [46] The equivalence stems from the fact that the entropy of a Gaussian distribution is directly proportional to the logarithm of the determinant of its covariance matrix. [47] L. Barnett, A. B. Barrett, and A. K. Seth, Phys. Rev. Lett. 103, 238701 (2009). [48] L. Barnett and A. K. Seth, J. of Neuroscience methods 275, 93 (2017). [49] Whereas numerical techniques have been proposed in the stochastic thermodynamics litterature to estimate mutual information and Kullback-Leibler divergence between time series generated by stochastic dynamics [13, 50], the problem of estimating numerically transfer entropy rates is even more serious. This is still an active research area (see [40] for a recent discussion about transfer entropy estimators). [50] E. Roldan, and J. M. R. Parrondo, Phys. Rev. E 85, 031129 (2012). [51] D. Hathcock, J. Sheehy, C. Weisenberger, E. Ilker, and M. Hinczewski, IEEE Trans. Mol. Biol. Multi-Scale Commun. 2, 16 (2016). [52] R. E. Kalman and R. S. Bucy, Trans. ASME J. Basic. Eng. Series 83/D, 95 (1961). [53] T. Kailath, A. H. Sayed, and B. Hassibi, Linear estimation, (Upper Saddle River, NJ: Prentice-Hall, 2000). [54] K. J. Aström,˚ Introduction to Stochastic Control Theory (Dover Publications, 2006). [55] T. Sagawa and M. Ueda, Phys. Rev. E 85, 021104 (2012). [56] C. Van den Broeck and M. Esposito, Physica A 418, 6 (2015). [57] C. Maes, Séminaire Poincaré 2, 29 (2003). t t t [58] We stress that P t (X0|X0) differs from the conditional probability P (X0|X0,Y0 ) which takes into account the coevolution bY0 t t −S[X0] of Xt and Yt. For instance, in the case of the diffusion process defined by Eqs. (3), one has P t (X0|X0) ∝ e , where bY0 t t 1 R t ˙ −1 ˙ S[X0] is the so-called Onsager-Machlup action S[X0] ≡ 2 0 ds [Xs − FX,s(Xs)]DXX,s(Zs) [Xs − FX,s(Xs)], using Itôconvention for the arguments of the functions FX,t and DXX,t (see, e.g., [59–61]). One must then use the anti-Itô discretization for defining the conditional probability of the time-reversed path, which explains the occurrence of the modified drift Fˆ in Eq. (23) when the noises are multiplicative. [59] F. Langouche, D. Roekaertst, and E. Tirapegui, Functional integration and semi-classical expansions, Springer (1982). [60] A. W. C. Lau and T. C. Lubensky, Phys. Rev. E 76, 011123 (2007). [61] M. Itami and S-i Sasa, J. Stat. Phys. 167, 46 (2017). [62] R. Chétriteand K. Gawedzki, Commun. Math. Phys. 282, 469518 (2008). [63] C. Maes, arXiv: 1904.10485 (2019). [64] For instance, there is a confusion in the definition of the steady-state information flows (or learning rates) in Ref. [11]: flow + Eq. (4) in this reference does not define IX→Y (i.e., lY with the present notations), contrary to what is written, but flow + −IY →X (i.e., −lX ). Whereas these two quantities are equal when the process is bipartite, this is no longer true when the + + noises are correlated, as specified by Eq. (30) in the present work. Accordingly, one must compare lX (resp. lY ) to T Y →X flow flow flow flow (resp. T X→Y ). When the error is corrected (which amounts to changing IE→µ to −Iµ→E and Iµ→E to −IE→µ in Table + + 5 of [11]), one observes that the inequalities lX ≤ T Y →X and lY ≤ T X→Y are always satisfied. This is indeed the correct result, even for a non-bipartite process. Therefore, contrary to the statement made in [11], such inequalities cannot be used for inferring that the noises are correlated. [65] D. Chicharro, Biol. Cybern. 105, 331 (2011). [66] N. J. Newton, arXiv:1604.01969 (2016). N N [67] For instance, in the context of communication theory where X0 ≡ (X0, ...XN ) and Y0 ≡ (Y0, ...YN ) are two time series corresponding respectively to the input and output of a communication channel, one typically considers the sum M PN n+1 n DX→Y (N) = n=0 I(X0 : Yn+1|Y0 ), an information-theoretic measure called directed information [68]. This quantity is well-defined in discrete time, despite the fact that it involves the instantaneous dependence of Yn+1 on Xn+1, in PN n n contrast with the sum DX→Y (N) = n=0 I(X0 : Yn+1|Y0 ) [44]. The continuous-time versions of the two sums are R t R t 0 ds [TX→Y (s) + TX.Y (s)] and 0 ds TX→Y (s), respectively [66], and the former quantity is carefully defined in [42]. However, TX.Y (s) diverges when the joint process is non-bipartite. [68] J. Massey, in Proceedings of Int. Symp. Inform. Theory and Appl., p. 303 (1990). [69] H. J. Kushner, J. SIAM control, Ser. A 2, 106 (1962). [70] R. Lipster and A. Shiryaev, Statistics of Random Processes. II. Applications, Stochastic Modelling and Applied Probability (Springer, Berlin, 2001). [71] S. Haufe, V. Nikulin, K.-R. Müller,and G. Nolte, NeuroImage, 64, 120 (2013). [72] M. Vinck et al. NeuroImage 108, 301 (2015). [73] I. Winkler, D. Panknin, D. Bartz, K.-R. Müller,and S. Haufe, IEEE Trans. Signal Processing 64, 2746 (2016). [74] G. E. Crooks and S. Still, arXiv:1611.04628 (2016). [75] R. E. Spinney, J. T. Lizier, and M. Prokopenko, Phys. Rev. E 98, 032141 (2018). −1 [76] The finite-horizon backward TE hln[P (Yt|Xt+τ ,Yt+τ )/P (Yt|Yt+τ )]i is also considered in Ref. [77] where τ is the sampling frequency of an observed process. This makes the resulting time-series non-bipartite even when the underlying dynamics considered in [77] are bipartite. [77] A. Auconi, A. Giansanti, and E. Klipp, Entropy 21, 177 (2019). 39

† [78] More precisely, the continuous-time version of Eq. (18) in [25] is T X→Y (t) − T X→Y (t) − dtI(Xt : Yt) = −lX (t), but this coincides with Eq. (51). [79] When considering the BTE rate in a steady state, it is implicit that the limit of an infinite trajectory is taken, so that † TX→Y no longer depends on T . Eqs. (52) and (53) then imply Eq. (54). [80] U. G. Haussmann and E. Pardoux, The Annals of Probability 14, 1188 (1986). [81] H. Asnani, K. Venkat, and T. Weissman, in Information and Control in Networks, Lecture Notes in Control and Infor- mation Sciences, 450, 157, Springer (2014). [82] E. Mayer-Wolf and M. Zakai, in Lecture Notes in Control and Information Sciences, 61, 164, Springer-Verlag, (1983). T [83] One can show that this definition also satisfies Eq. (21), where the path probability Pb T (X0 |X0) has the same definition Y0 T as in the bipartite case. Since the noises are correlated, it may be surprising that Pb T (X0 |X0) is expressed in terms of Y0 the same Onsager-Machlup action functional as in the bipartite case which only involves the element DXX of the diffusion T T T matrix (see [58]). However, we recall that Pb T (X0 |X0) is the probability of observing the trajectory X0 when Y0 is Y0 QN−1 considered as an external protocol. It may be viewed as the continuous-time limit of the probability k=0 P (Xk+1|Zk) where the evolution over the time interval [0,T ] is discretized into steps of width ∆t = T/N with tk = k∆t for k = 0, ...N, Xk ≡ X(tk) and Yk ≡ Y (tk) (see, e.g., [19, 55]). As can be seen from Eq. (73), P (Xk+1|Zk) only depends on DXX . [84] In particular, we stress that inequality (97) is not the continuous-time version of inequality (16) in [25] (see also [28]). This latter inequality, which is obtained under the bipartite assumption, involves the difference between the forward and backward TEs in the direction X → Y instead of Y → X, and it simply reduces to the standard inequality R T † σX (t) ≥ lX (t) [12, 15, 18] in the continuous-time limit. We also recall that the time-integrated BTE 0 dt TY →X (t) differs from the time reversed transfer entropy considered in [26, 46, 48]. [85] M. L. Rosinberg, G. Tarjus, and T. Munakata, Phys. Rev. E 98, 032130 (2018). [86] This is the generalization of the finite-horizon Granger-causality considered in [45, 48, 87]. Ti→j (h) may be a useful tool to analyze the predictive behavior of cellular signaling networks, like the so-called “predictive mutual information” used in Ref. [88] which corresponds to I(Xt+h : Yt). The finite-horizon TE was also used recently to infer the presence of a time delay in the coupling between two stochastic systems [85]. [87] J.-M. Dufour and E. Renault, Econometrica 66, 1099 (1998). [88] N. B. Becker, A. Mugler, and P. Rein ten Wolde, Phys. Rev. Lett. 115 258103 (2015). [89] J. Geweke, J. Am. Stat. Assoc. 77, 304 (1982); ibid 79, 907 (1984). [90] L. Ljung and T. Kailath, IEEE Trans. Inform. Theory IT-22 488 (1976). [91] B. D. O. Anderson, Stoch. Process. Appl. 12, 313 (1982). [92] C. G. Bowsher and P. S. Swain, Proc. Natl. Acad. Sci. U. S. A., 109, E1320 (2012). [93] T. D. Gillespie, J. Chem. Phys. 113, 297 (2000). [94] M. Hinczewski and D. Thirumalai, Phys. Rev. X 4, 041017 (2014). [95] D. Hathcock et al., IEEE Trans. Mol. Biol. Multi-Scale Commun. 2, 16 (2016). [96] Note that we only consider the single-time-step sensory capacities Ci because the joint dynamics is non-Markovian and therefore inequality (58) is not necessarily satisfied. This implies that the sensory capacities Ci defined by Eq. (60) may be larger than 1. [97] K. J. Aströmand˚ R. M. Murray, Feedback Systems : An Introduction for Scientists and Engineers (Princeton University Press, Princeton, NJ, 2008). [98] J. Bechhoefer, Rev. Mod. Phys. 77, 783 (2005). [99] H. Sandberg, J. C. Delvenne, N. Newton, and S. K. Mitter, Phys. Rev. E 90, 042119 (2014). [100] The transformation from Eqs. (98) to Eqs. (D30) is physically meaningful and Eqs. (98) describes the dynamics of a pair 2 signal-observer at the condition that K and G are real, which requires that ∆ = (a11 − a22) + 4a12a21 ≥ 0. [101] N. Wiener, Extrapolation, Interpolation and Smoothing of Stationary Times Series (Wiley, New York, 1949). [102] A. N. Kolmogorov, Selected Works of A. N. Kolmogorov: Probability theory and mathematical statistics (Springer Science and Business Media, 1992). [103] H. W. Bode and C. E. Shannon, Proc. Inst. Radio. Eng. 38, 417 (1950).