Thermodynamics of Markov Processes with Non-Extensive Entropy and Free Energy

Thermodynamics of Markov Processes with Non-extensive Entropy and Free Energy

Liangrong Peng1,2, Hong Qian3, Liu Hong2,3 1College of Mathematics and Data Science, Minjiang University, Fuzhou, 350108, P.R.C. 2Zhou Pei-Yuan Center for Applied Mathematics, Tsinghua University, Beijing, 100084, P.R.C. 3Department of Applied Mathematics, University of Washington, Seattle, WA 98195-3925, U.S.A.a)

(Dated: 1 April 2020)

Statistical thermodynamics of small systems shows dramatic differences from nor- mal systems. Parallel to the recently presented steady-state thermodynamic formalism for master equation and Fokker-Planck equation, we show that a “thermodynamic” theory can also be developed based on Tsallis’ generalized entropy (q) PN q (q) S = i=1(pi − pi )/[q(q − 1)] and Shiino’s generalized free energy F = PN q−1 (q) [ i=1 pi(pi/πi) − 1]/[q(q − 1)], where πi is the stationary distribution. dF /dt = (q) (q) (q) (q) −fd ≤ 0 and it is zero iff the system is in its stationary state. dS /dt − Qex = fd (q) where Qex characterizes the heat exchange. For systems approaching equilibrium (q) with detailed balance, fd is the product of Onsager’s thermodynamic flux and force. However, it is discovered that the Onsager’s force is non-local. This is a consequence of the particular transformation invariance for zero energy of Tsallis’ statistics.

Keywords: Tsallis Thermodynamics, Non-equilibrium Steady-State, Master Equa- tion, Fokker-Planck Equation arXiv:1005.1251v3 [math-ph] 31 Mar 2020

a)Author to whom correspondence should be addressed. Electronic mail: [email protected],

[email protected]

1 1. INTRODUCTION

The Boltzmann-Gibbs (B-G) statistical thermodynamics is one of the fundamental theo- ries of modern physics with wide applications in chemistry, biology, and even social sciences. Recently, a complete statistical thermodynamic formalism based on both the B-G entropy and B-G free energy (relative entropy) has been presented for stochastic dynamics based on master equations1,2 and Fokker-Planck equations3,4. The theory introduces several key quantities: entropy production rate, free energy dissipation rate, excess and house-keeping heat, etc. Statistical thermodynamics of small systems requires the traditional thermodynamic quantities such as energy and entropy to be non-extensive5. This extension of the classical B-G entropy provides a framework to study the systems with long-range interactions, multi-fractal structures, long-term memories, or systems whose interaction with macroscopic parts cannot be ignored6. It is therefore natural to ask whether the thermodynamic formalism can be developed in parallel for non-extensive entropy and free energy, such as the Tsallis entropy7, Renyi entropy8,9 and many others6,10. Here we focus on the Tsallis entropy, which enjoys the properties of concavity, experimen- tal robustness (Lesche stability), non-extensivity (as a generalization of the B-G entropy), and the ﬁniteness of the rate of entropy production11,12. This work shows that a similar structure does exist for Tsallis thermodynamics, which therefore would be of interest to a wide range applications, like defect turbulence13, optical lattices14, mathematical ﬁnance15, and so on. PN Generalized Entropy. The notions of Shannon’s entropy in the form − i=1 fi ln fi PN and relative entropy in the form i=1 fi ln(fi/pi) arise from the asymptotic expressions for

the probability of frequency {fi} within a large set of independent and identically distributed 1 (i.i.d.) random samples, each with an equal probability of N and {pi} respectively. The

−NI(f1,··· ,fN ) exponent I(f1, ··· , fN ) in the asymptotic expression e , known as large deviations rate function, provides the divergence function for an information geometry16. When a set of samples are not independent, the probability for the frequency is no longer a simple multi-nomial distribution; Tsallis entropy is believed to arise in this context in connection with q-binomial distribution17. The Shannon entropy is intimately related to the logarithmic function, y = ln(x), which is the inverse of exponential function x = exp(y). The exponential

2 function is deﬁned through the limit:

1 exp(y) = lim (1 + ry) r . (1) r→0

The inverse function of the right-hand side of Eq. (1) is xr − 1 y = . (2) r If one takes this as the analogue of the y = ln(x), then corresponding to Gibbs’ entropy P − i pi ln pi, one can have a generalization of the entropy:

N X pr − 1 − p i . (3) i r i=1 If we denote r = q − 1, then Eq. (3) becomes

N q−1 X 1 − p S(q) = p i , (4) i q(q − 1) i=1

as q → 1. This is the non-extensive entropy ﬁrst proposed by Tsallis7, and q ∈ R (q 6= 0, 1) is known as the non-extensive parameter. Here we added an additional factor q to the denominator to maintain the deﬁnite sign of free energy and its dissipation rate, which will be shown later. Because of relations in Eqs. (1) and (2), the Tsallis entropy enjoys many of the properties of the Gibbs’ entropy. Relative entropy. Recognizing the correspondence between ln(x) and (xq−1 −1)/[q(q − 18,19 1)] as q → 1, a generalization of the relative entropy with respect to a distribution gi can be similarly introduced20:

" N q−1 # 1 X pi H(q)({p }||{g }) = p − 1 . (5) i i q(q − 1) i g i=1 i

q Statistical internal energy and H as the Gibbs free energy. Assuming {πi} is a stationary distribution to the master equation in Eq. (12) below, one can deﬁne a statistical internal energy by applying the Boltzmann’s law in reverse1:

π1−q − 1 (q) = i , (6) i q(q − 1)

in analogous to B-G energy i = ln(1/πi). Then the mean statistical internal energy is

N 1−q X (q) X π − 1 U (q) = pq = pq i . (7) i i i q(q − 1) i=1 i=1

3 q (q) The mean internal energy being the sum of pi i rather than the conventional pii is a peculiar feature of the Tsallis statistics. Then we have generalized free energy:

" N q−1 # 1 X pi F (q) = U (q) − S(q) = p − 1 = H(q)({p }||{π }). (8) q(q − 1) i π i i i=1 i Conditional entropy. Tsallis entropy does not enjoy the standard equation concerning conditional entropy of a pair of correlated random variables A and B19,21:

S(AB) = S(A) + SA(B), (9a)

where X S(AB) = − piqij log (piqij) , (9b) i,j X S(A) = − pi log pi, (9c) i ! X X SA(B) = − pi qij log qij . (9d) i j For Tsallis’s entropy, one has ! " # 1 X 1 X X X S(q)(AB) = 1 − (p q )q = 1 − pq + pq − (p q )q q(q − 1) i ij q(q − 1) i i i ij i,j i i i,j P q X 1 − j qij X = S(q)(A) + pq = S(q)(A) + pq S(q)(B|A = i) i q(q − 1) i i i (q) X (q) (q) (q) ≤ S (A) + pi S (B|A = i) = S (A) + SA (B), i when q > 1. And the inequality sign changes when q < 1. Eq. (9), together with entropy reaches its maximum with uniform distribution 1 1 1 1 S ({p , p , p , ··· , p }) ≤ S , , , ··· , , (10) 1 2 3 N N N N N and entropy is invariant with adding an impossible event

S ({p1, p2, p3, ··· , pN , 0}) = S ({p1, p2, p3, ··· , pN }) , (11)

providing Khinchin’s uniqueness theorem for Gibbs entropy21. It is noted that Tsallis entropy does satisfy both Eqs. (10) and (11). By replacing Eq. (9) with a more general non-extensive one, Abe has shown that the Tsallis entropy follows uniquely from the generalized version of Khinchin’s axioms22.

4 This paper is organised as follows. In Sec. 2 and 3, the Tsallis thermodynamics of master equations and F-P equations are constructed separately, with emphasis on the correct non- negative form of house-keeping heat. In Sec. 4, the underlying mathematical connection and correspondence between the two formalisms is explored and justified for both the discretized F-P equations and tridiagonal master equations, in the limit of infinitesimal jump step in space. Sec. 5 provides two simple but insightful examples to illustrate the application of Tsallis thermodynamics and its difference from B-G thermodynamics. Finally, we summarize our results in Sec. 6. Proofs on theorems in Sec. 4 can be found in appendix for completeness.

2. TSALLIS THERMODYNAMICS OF MASTER EQUATIONS

We now consider a Markov process with master equation N dpi(t) X = (p w − p w ) , i = 1, 2, ··· ,N, (12) dt j ji i ij j=1 which has been widely applied for modeling various irreversible processes, including quantum physics, chemical reactions and open biological systems23–25.

Relative entropy and positive free energy dissipation. We assume both {pi(t)}

and {gi(t)} satisfy the same master equation, with diﬀerent initial distributions. Then the time derivative of the relative entropy N " q−1 q # d 1 X pi(t) dpi pi(t) dgi H(q)(t) = q − (q − 1) dt q(q − 1) g (t) dt g (t) dt i=1 i i N ( " q−1 q−1# q q 1 X pi pj pi pj = w qp − −(q − 1)g − q(q − 1) ji j g g j g g i,j=1 i j i j N q ( q−1 q ) 1 X wjipj pigj pigj = q − (q − 1) − 1 ≤ 0. (13) q(q − 1) q−1 g p g p i,j=1 gj i j i j

The last step is because wij ≥ 0 when i 6= j, the summand is zero when i = j, and the term in the {· · · }: qzq−1 − (q − 1)zq − 1 (1 − z)2 d 1 − zq = − ≤ 0, (z > 0). (14) q(q − 1) q(q − 1) dz 1 − z

Now if we take a stationary distribution {πi} as the {gi}, then we have the time derivative of generalized free energy F (q): (q) N " q−1 q−1# N dF 1 X pi pj 1 X (q) = w p − = − φ Φ , (15) dt q − 1 ji j π π 2 ij ij i,j=1 i j i,j=1

5 where (p /π )q−1 − (p /π )q−1 φ (t) = p w − p w , Φ(q)(t) = i i j j . (16) ij i ij j ji ij q − 1 Markovian dynamics with detailed balance. Master equation with detailed balance is a special class of stochastic systems which represents closed systems approaching to

equilibrium. For master equations with detailed balance, πiwij = πjwji is equivalent to the Kolmogorov cycle condition: w w ··· w w i0i1 i1i2 in−1in ini0 = 1, (17) wi1i0 wi2i1 ··· winin−1 wi0in

26 for any cycle path i0, i1, ··· , in−1, in, i0 . Then we have q−1 q−1 (q) 2 (piwij) − (pjwji) Φij (t) = q−1 q−1 . (18) (πjwji) + (πiwij) q − 1

(q) In this case, the φij and Φij can be identiﬁed as Onsager’s thermodynamic ﬂux and force, respectively. Therefore, we shall call their product the free energy dissipation rate

(q) N dF (q) 1 X (q) − = f = φ Φ ≥ 0. (19) dt d 2 ij ij i,j=1

For standard extensive thermodynamics, the Onsager’s thermodynamic force Φij is “lo-

cally determined”. It depends only on pi, pj, wij, wji and the local energy diﬀerence (q) i − j = ln(wij/wji). However, Eq. (18) shows that the Φij here depends also on stationary πi and πj, not just their ratio which can be determined locally (Or equivalently, it (q) (q) depends on the absolute value of i and j , not just their diﬀerence). This is a consequence of the particular transformation invariance for zero energy of Tsallis’ statistics. Entropy production rate. With or without detailed balance, we have

(q) N N q−1 q−1 ! dS (t) 1 X 1 X pi − pj = − pq−1 (p w − p w ) = φ dt q − 1 i j ji i ij 2 ij q − 1 i,j=1 i,j=1 N (q) q X (q) (q) = f − φ pq−1 − pq−1 , (20) d 2 ij i i j j i,j=1

(q) where statistical internal energy i is deﬁned in Eq. (6) from the stationary distribution 27 πi. The last term in Eq. (20) can be identiﬁed as excess heat exchange rate :

N q X (q) (q) Q(q) = − φ pq−1 − pq−1 . (21) ex 2 ij i i j j i,j=1

6 (q) The Qex = 0 in the steady state. However, for equilibrium steady state with detailed balance, each term in the summand is zero. In contrast, for nonequilibrium steady state, the summand is not zero. But the sum is zero due to Tellegen theorem: The “heat, or kinetic energy” continuously circulates in the system in nonequilibrium steady state. Similar as in the B-G thermodynamics1,2,27, the time derivative of Tsallis entropy can also be separated into two distinct parts

(q) N q−1 q−1 ! dS (t) 1 X pi − pj (q) = φ ≡ e(q) − h , dt 2 ij q − 1 p d i,j=1 where the heat dissipation rate

N (q) 1 X h = φ pq−1 wq−1 − 1 − pq−1 wq−1 − 1 , (22) d 2(q − 1) ij i ij j ji i,j=1 and the entropy production rate

1 X e(q) = p w (p w )q−1 − (p w )q−1 (23) p q − 1 i ij i ij j ji i,j 1 X = φ (p w )q−1 − (p w )q−1 ≥ 0. 2(q − 1) ij i ij j ji i,j

(q) q−1 q−1 The ep is greater than zero since if piwij ≥ pjwji, then ((piwij) − (pjwji) ) /(q−1) ≥ 0. Correct form of house-keeping heat. In B-G thermodynamics, the diﬀerence of (q) (q) entropy production rate and free energy dissipation rate ep − fd is also known as the house-keeping heat, which is always greater than zero. However, in Tsallis thermodynamics, the sign of

(q) (q) 1 X pi pj Qg = e(q) − f = p w (p w )q−1 − (p w )q−1 − ( )q−1 + ( )q−1, (24) hk p d q − 1 i ij i ij j ji π π i,j i j

is undeﬁned. Consider a three-state Markov jump process and choose the transition rate matrix as   −1 1/2 1/2     W = 1/2 −1 1/2 ,   1/2 1/2 −1

whose equilibrium distribution is π = (1/3, 1/3, 1/3). Given the initial distribution p(0) = g(q) (1, 0, 0) and q = 2, it is direct to see Qhk (t = 0) = −2.5 < 0.

7 g(q) Therefore, it is natural to ask whether a part of Qhk can play the same role of house- keeping heat while still being non-negative? Before introducing the correct formula, we (q) 1 P q−1 show that, a direct generalization of house-keeping heat Qhk = q−1 i,j piwij (πiwij) − q−1 (πjwji) from B-G to Tsallis thermodynamics fails to do this. In fact, even though (q) P limq→1 Qhk = i,j piwij[ln(πiwij) − ln(πjwji)] recovers the house-keeping heat in B-G thermodynamics, it is straightforward to see that by choosing q = 2, the transition rate matrix   −3 1 2     W =  2 −3 1  ,   2 1 −3

(q) and the initial distribution p(0) = (1, 0, 0), we have Qhk (t = 0) < 0. (q) To be correct, the diﬀerence between entropy production rate ep and free energy dissipa- (q) (q) tion rate fd is decomposed into two parts: the house-keeping heat Qhk and non-extensive (q) heat Qne , i.e.

(q) 1 X 1 X pi Q = p w (π w )q−1 − (π w )q−1 − (π w )q − (π w )q, (25) hk q − 1 i ij i ij j ji q π i ij j ji i,j i,j i 1 X pi pj Q(q) = p w (p w )q−1 − (p w )q−1 − ( )q−1 + ( )q−1 ne q − 1 i ij i ij j ji π π i,j i j q−1 q−1 q − 1 1 q q − (πiwij) − (πjwji) + (πiwij) − (πjwji) , (26) q πiwij

(q) (q) where Qhk represents the distance from detailed balance condition, and Qne arises due to the non-extensibility of Tsallis thermodynamics, which vanishes in the limit of q → 1. Based (q) on the polynomial inequality, the sign of Qhk is readily determined as follows.

N Proposition 2.1. Let {πi > 0}i=1 be the steady state of master equations in (12), then the house-keeping heat

(q) 1 X 1 X pi Q = p w (π w )q−1 − (π w )q−1 − (π w )q − (π w )q ≥ 0, hk q − 1 i ij i ij j ji q π i ij j ji i,j i,j i is non-negative deﬁnite, which becomes zero if and only if at the equilibrium state.

Proof. The house-keeping heat for master equations can be reformulated as (q) 1 X πjwji πjwji Q = p w (π w )q−1 1 − q( )q−1 + (q − 1)( )q ≥ 0, hk q(q − 1) i ij i ij π w π w i,j i ij i ij

8 q−1 q where in the last step we have used the inequality 1−qz +(q−1)z ≥ 0 for z = πj wji > 0 as q(q−1) πiwij proved in (14). The Q(q) = 0 if and only if z = πj wji = 1 for each pair of (i, j), that is, the hk πiwij system is at equilibrium.

Laws of Tsallis thermodynamics. With above thermodynamic quantities in hand, laws of Tsallis thermodynamics for master equations are reformulated as dS(q) + h(q) = e(q) ≥ 0, (27a) dt d p dS(q) − Q(q) = f (q) ≥ 0, (27b) dt ex d (q) (q) (q) (q) Qex + hd − Qne = Qhk ≥ 0. (27c)

(q) The ﬁrst relation shows that the entropy production rate ep is always non-negative, as a manifestation of the second law of thermodynamics, while the other two relations go beyond the classical ones. Connections with B-G thermodynamics. By recalling the correspondence between q-logarithm (xq−1 − 1)/(q − 1) and natural logarithm ln(x), as q → 1, we have the entropy, internal energy and free energy in Tsallis thermodynamics become the corresponding items in B-G thermodynamics respectively1,2, i.e.

N (q) X lim S = − pi ln pi, (28a) q→1 i=1 N (q) X lim U = − pi ln πi, (28b) q→1 i=1 N (q) X lim F = pi ln(pi/πi). (28c) q→1 i=1 Similar conclusions also hold for the entropy production rate, free energy dissipation rate and house-keeping heat:

N (q) 1 X piwij lim ep = (piwij − pjwji) ln ≥ 0, (29) q→1 2 p w i,j=1 j ji N (q) 1 X piπj lim f = (piwij − pjwji) ln ≥ 0, (30) q→1 d 2 p π i,j=1 j i N (q) 1 X πiwij lim Q = (piwij − pjwji) ln ≥ 0, (31) q→1 hk 2 π w i,j=1 j ji

9 where the right-hand side of Eqs. (29)-(31) are identiﬁed as the corresponding entropy production rate, free energy dissipation rate and house-keeping heat in B-G thermodynamics of master equations1,2. Furthermore, limits of heat dissipation rate and change rate of excess heat

N (q) 1 X wij lim h = (piwij − pjwji) ln , (32) q→1 d 2 w i,j=1 ji N (q) 1 X πi lim Qex = (piwij − pjwji) ln , (33) q→1 2 π i,j=1 j agree exactly with the deﬁnitions in B-G thermodynamics. It is noticeable that, the non-extensive heat reduces to zero as q approaches to 1, that (q) is, limq→1 Qne = 0. Thus the house-keeping heat equals to the diﬀerence between entropy (q) production and free energy dissipation rates in B-G thermodynamics. This implies that Qne can serve as a characteristic quantity to describe the non-extensivity of the system.

3. TSALLIS THERMODYNAMICS OF FOKKER-PLANCK EQUATIONS

In this Section, we turn to the Tsallis thermodynamics of Fokker-Planck (F-P) equations, which read ∂ ∂ ∂p p(x, t) = − K(x, t),K(x, t) = u(x)p − D(x) , (34) ∂t ∂x ∂x

where u(x) and D(x) > 0 denote the drift and diﬀusion coeﬃcients respectively. In particular, the steady-state probability distribution is given by a time-independent solution of ∂ ∂t p(x, t) = 0, that is,

∂ ∂π Ks(x) = 0,Ks(x) = u(x)π − D(x) , (35) ∂x ∂x

where π(x) ≥ 0 for x ∈ R. Compared to master equations, the information of transition rate matrix is largely compressed to the draft and diffusion coefficients, which as a consequence will simplify the modeling procedure and numerical computations a lot. Tsallis thermodynamics of F-P equations. By extending the state variables into continuous cases, we can directly borrow the definitions of the generalized entropy, internal

10 energy and free energy from master equations and apply to F-P equations20,

Z pq−1 − 1 S(q)(t) = − dxp , (36a) q(q − 1) Z π1−q − 1 U (q)(t) = dxpq , (36b) q(q − 1) 1 Z p q−1 F (q)(t) = dx p − p , (36c) q(q − 1) π

(q) (q) where (q 6= 0, 1). The free energy dissipation rate fd = −dF /dt associated with F-P equations is given by

1 Z ∂ p 1 Z p ∂K p ∂Ks f (q) = − dx K ( )q−1 = dx q( )q−1 − (q − 1)( )q d q − 1 ∂x π q(q − 1) π ∂x π ∂x Z 1 ∂ p 1 ∂ p Z π2 p ∂ p = − dx K ( )q−1 − Ks ( )q = dx D ( )q−1( ( ))2 (37) q − 1 ∂x π q ∂x π p π ∂x π Z p p K Ks = dx ( )q−1( − )2 ≥ 0, D π p π

where in the second line we have used the integration by parts, and in the last line we have K Ks ∂ p used the relation ( p − π ) = −D ∂x ln( π ). As a result, the free energy dissipation rate 20 (q) is always non-negative , and vanishes if and only if in the steady state. Notice that fd p q−1 possesses a similar form as in classical B-G thermodynamics except for the factor ( π ) . Now we proceed to derive the balance equation of Tsallis entropy.

dS(q)(t) Z Z p K u = − dxKpq−2∂ p = dx pq−1( )2 − pq−1K dt x D p D Z p K Z p K u = dx (Dp)q−1( )2 − dx pq−1(Dq−1 − 1)( )2 + pq−1K D p D p D

(q) (q) ≡ ep − hd ,

where the ﬁrst contribution

Z p K e(q) = dx (Dp)q−1( )2 ≥ 0, (38) p D p

denotes the entropy production rate, which is always non-negative as a manifestation of the second law of thermodynamics; and the second contribution denotes the heat dissipation rate, or the entropy ﬂux from environment:

Z p K u h(q) = dx pq−1(Dq−1 − 1)( )2 + pq−1K . (39) d D p D

11 House-keeping and non-extensive heats. Similarly, the house-keeping heat together with non-extensive heat is identiﬁed as the diﬀerence between entropy production and free energy dissipation rate, that is,

Z p K p K Ks e(q) − f (q) = dx (Dp)q−1( )2 − ( )q−1( − )2 ≡ Q(q) + Q(q), (40) p d D p π p π hk ne Z p Ks Q(q) = dx (Dπ)q−1( )2 ≥ 0, (41) hk D π Z p K Ks p K Ks Q(q) = dx (Dp)q−1( )2 − (Dπ)q−1( )2 − ( )q−1( − )2 . (42) ne D p π π p π

(q) (q) Here Qhk represents the distance from detailed balance condition, and Qne arises due to the non-extensibility of Tsallis thermodynamics, which vanishes in the limit of q → 1,

Z s s (q) p K 2 K 2 K K 2 lim Qne = dx ( ) − ( ) − ( − ) = 0. q→1 D p π p π

During above calculations, we use an important relation for F-P equations which is stated through the following lemma.

Lemma 3.1. For q, n ∈ R, such that, q 6= 0, 1 and n ≥ 1, the following integration

Z p p q−1 K Ks (Ks)n dx − = 0, (43) D π p π π as long as the probability distribution vanished at the boundary.

K Ks ∂ p Proof. By using p − π = −D ∂x ln( π ), we have

Z p p K Ks (Ks)n Z ∂ p p dx ( )q−1( − ) = − dx ( )( )q−1(Ks)n D π p π π ∂x π π 1 Z ∂ p 1 Z p ∂ = − dx ( )q(Ks)n = dx( )q(Ks)n−1 Ks = 0, q ∂x π q π ∂x

∂ s where in the last step we have utilized the divergence-free relation as ∂x K = 0 and assumed the probability distribution p(x, t) vanished at the boundary.

Laws of Tsallis thermodynamics. If we further introduce the change rate of excess heat as dU (q) Z ∂ p ∂p Q(q) = = dxpq−2K π2−q ( ) − . (44) ex dt ∂x π ∂x

12 Then, similar as master equations, laws of Tsallis thermodynamics for F-P equations are readily derived as dS(q) + h(q) = e(q) ≥ 0, (45a) dt d p dS(q) − Q(q) = f (q) ≥ 0, (45b) dt ex d (q) (q) (q) (q) Qex + hd − Qne = Qhk ≥ 0. (45c) which are also known as three diﬀerent faces of the second law of thermodynamics according 28 (q) (q) to Ref. . Again, it should be noted that the non-negativity of ep − fd is not guaranteed for F-P equations in the framework of Tsallis thermodynamics. Connections with B-G thermodynamics. As q → 1, the entropy production rate, free energy dissipation rate and house-keeping heat of F-P equations become Z (q) p K 2 lim ep = dx ( ) ≥ 0, (46) q→1 D p Z p K Ks lim f (q) = dx ( − )2 ≥ 0, (47) q→1 d D p π Z p K K Ks Z p Ks lim Q(q) = dx ( )2 − ( − )2 = dx ( )2 ≥ 0, (48) q→1 hk D p p π D π

by using Lemma 3.1. The right-hand side of Eqs. (46)-(48) are identiﬁed as the corresponding entropy production rate, free energy dissipation rate and house-keeping heat of F-P equations in B-G thermodynamics1,4. Furthermore, the limit of heat dissipation rate and change rate of excess heat become, respectively, Z u lim h(q) = dx K , (49) q→1 d D Z (q) lim Qex = − dx(K∂xln π), (50) q→1

which agree with the classical deﬁnitions in B-G thermodynamics as expected.

4. EMERGENCY OF A UNIFIED FORMULATION IN THE MATHEMATICAL LIMIT

In Ref.29, a uniﬁed B-G thermodynamic formalism has been established for both master equations and F-P equations. Particularly, when a Markov process is restrained to one-step

13 jump, the B-G thermodynamics for the master equations and F-P equations has a one-to- one correspondence. However, in Tsallis thermodynamics, due to the non-extensivity and nonlinearity of Tsallis entropy and free energy, above conclusions may not hold any more. In this part, we are going to show that, within the framework of Tsallis thermodynamics (q) discussed in the last two sections, the free energy dissipation rate fd , entropy production (q) (q) rate ep , and house keeping heat Qhk of master equations with a tridiagonal transition rate matrix can be derived exactly from that of F-P equations; and vice versa. Thus a one-to-one correspondence between the Tsallis thermodynamics of master equations and F-P equations can be rigorously established in mathematics.

A. From F-P equations to master equations by direct discretization

First, we start with F-P equations with continuous time and state variables, and derive discrete master equations as its approximation. For notational simplicity, F-P equations in (34) is rewritten into a convenient form

∂ ∂ ∂ p(x, t) = − K(x, t),K(x, t) =u ¯(x)p(x, t) − [D(x)p(x, t)] , (51) ∂t ∂x ∂x

whereu ¯(x) ≡ u(x) + ∂D(x)/∂x. Introduce uniform grids along the x-axis,

x = xi = i∆x = i, i = 0, ±1, ±2, ··· . (52)

∂ By using the forward difference to the drift term − ∂x [¯u(x)p(x, t)], and central difference to ∂2 the diffusion term ∂x2 [D(x)p(x, t)], we discretize the right-hand side of (51) at coordinate

(xi = i, t):

dp (t) 1 1 i = − [¯u p (t) − u¯ p (t)] + [D p (t) − 2D p (t) + D p (t)] , dt i+1 i+1 i i 2 i+1 i+1 i i i−1 i−1 whereu ¯i ≡ u¯(xi),Di ≡ D(xi), and pi(t) denotes the probability distribution at (xi, t). The above equation can be reformulated into the standard form as

dp (t) i = [p (t)w − p (t)w ] + [p (t)w − p (t)w ], i = 0, ±1, ±2, ··· , (53) dt i−1 i−1,i i i,i−1 i+1 i+1,i i i,i+1 by deﬁning the forward and backward transition rates as

1 1 1 w = D , w = D − u¯ , i−1,i 2 i−1 i,i−1 2 i i

14 respectively. Thus we deduce master equations with a tridiagonal transition rate matrix from F-P equations. Now we are at a position to derive the free energy dissipation rate of master equations from F-P equations. Based on results for Tsallis thermodynamics of F-P equations in Sec. 3, the following theorems can be established. In what follows, for notational simplicity, we (q) M F will drop the superscript (q) from fd , and use fd and fd to distinguish the free energy dissipation rates of master equations and F-P equations respectively. Similar notations also apply to other quantities without further mention.

Theorem 4.1. In limit of inﬁnitesimal discretization step → 0, the density of free energy dissipation rate (37) for F-P equations

−1 F 1 X M lim fd = φi,j(t)Φi,j(t) ≡ fd ≥ 0, (54) →0 2 i,j

becomes the corresponding result in (19) for master equations. Here the thermodynamic ﬂux q−1 q−1 (pi/πi) −(pj /πj ) φi,j(t) = piwi,j − pjwj,i and force Φi,j(t) = q−1 , with j = i ± 1.

Theorem 4.2. In the limit of → 0, the densities of entropy production rate (38) and house-keeping heat (41) for F-P equations

1−2q F 1 X q−1 q−1 M lim ep = (piwi,j − pjwj,i)[(piwi,j) − (pjwj,i) ] ≡ ep ≥ 0, (55) →0 2(q − 1) i,j

1−2q F 1 X q−1 q−1 lim Qhk = (piwi,j − pjwj,i) (πiwij) − (πjwji) (56) →0 2(q − 1) i,j 1 X pi pj − ( − )(π w )q − (π w )q ≡ QM ≥ 0, 2q π π i ij j ji hk i,j i j

lead to, respectively, the corresponding results in Eqs. (23) and (25) for master equations.

B. From tridiagonal master equations to F-P equations

Traditionally, F-P equations can be deduced from master equations by Kramers-Moyal expansion, i.e., by expanding the transition rates and neglecting jump moments of third order and above. Although Kramers-Moyal expansion oﬀers a straightforward routine from master equations to F-P equations, direct calculations show that thermodynamic elements

like fd are not preserved during this procedure.

15 To construct a correct limit process, we consider master equations with a tridiagonal 4 transition matrix . The elements of wi,j in Eq. (12) are nonzero only if j = i ± 1 or j = i with wi,i = −wi,i−1 − wi,i+1. Then master equation (12) is written as follows dp i = (p w − p w ) − (p w − p w ). (57) dt i+1 i+1,i i i,i−1 i i,i+1 i−1 i−1,i To make the state variables to be continuous, let us introduce

2 x = i, p(x, t) = pi(t), 2D(x) = (wi−1,i + wi,i−1) , u(x) = (wi−1,i − wi,i−1), (58)

with a small parameter (|| 1) to measure the relative importance of ﬂuctuations, as 1/ represents the system volume or number of particles. Taking the limit → 0, Eq. (57) becomes ∂p 1 1 2 1 1 = lim ∂x p( D − u) − [p − ∂xp + O( )]( D + u) + O() ∂t →0 2 2 2 2

= −∂x(up − D∂xp) ≡ −∂xK(x, t), (59)

which is the classical F-P equation with the probability ﬂux K = up − D∂xp. We have shown that F-P equations can be derived rigorously from the tridiagonal master equations. Starting from this point, we can further prove that the free energy dissipation rate M F fd for master equations converges automatically to fd for the corresponding F-P equations when → 0.

Theorem 4.3. In the limit → 0, the density of free energy dissipation rate (19) for master equations Z M K ∂ p q−1 F lim fd = − dx ( ) ≡ fd ≥ 0, (60) →0 q − 1 ∂x π becomes the free energy dissipation rate (37) for F-P equations.

Theorem 4.4. In the limit → 0, the densities of entropy production rate (23) and house- keeping heat (25) for master equations Z 2q−1 M p q−1 K 2 F lim ep = dx (Dp) ( ) ≡ ep ≥ 0, (61) →0 D p Z s 2q−1 M p q−1 K 2 F lim Qhk = dx (Dπ) ( ) ≡ Qhk ≥ 0, (62) →0 D π become, respectively, the entropy production rate (38) and house-keeping heat (41) for F-P equations.

16 Remark 4.5. Generally speaking, there are many diﬀerent coarse-graining ways from a description of lower level to that of higher level, for instance the Kramers-Moyal expansion and canonical form expansion of master equations in our case. We notice that the canonical form expansion29? well preserves the free energy dissipation rate during the coarse-graining procedure, while the Kramers-Moyal expansion fails. Together with the results in Ref.29, we may conclude that canonical form expansion is more appropriate than Kramers-Moyal expansion from the viewpoint of nonequilibrium thermodynamics.

5. APPLICATIONS

A. Two-state chemical reactions

We ﬁrstly consider N particles occupying two giving states 1 and 230, like protein folding and unfolding, DNA phosphorylation and dephosphorylation, gene on and oﬀ. With constant jumping rates µ and λ, the probability of the system with i particles at state 1 and (N − i) particles at state 2 evolves according to the following master equations,

N dp(i, N − i) X = [µp(i + 1,N − i − 1) − (µ + λ)p(i, N − i) + λp(i − 1,N − i + 1)] , (63) dt i=0 where i = 0, 1, ··· ,N. Here we particularly focus on the dependence of Tsallis thermodynamics and B-G thermodynamics with respect to the system size. As illustrated through Fig. 1, Tsallis thermodynamics with q > 1 generally shows a more apparent dependence on particle numbers N

than the B-G thermodynamics. For example, in B-G thermodynamics both fd and ep scale independent of the system size, but in Tsallis thermodynamics fd shows a size dependence while ep does not. This fact highlights the distinction of Tsallis thermodynamics on its non-extensive nature from B-G thermodynamics.

B. Optical lattices

In the second example, we look into optical lattices described by a Fokker-Planck equation (34) for the Wigner function, where

αx D1 u(x) = − 2 ,D(x) = D0 + 2 . (64) 1 + (x/xc) 1 + (x/xc)

17 FIG. 1. Comparison of Tsallis thermodynamics (q = 2) and B-G thermodynamics (q = 1) on their system size dependence. The initial state is prepared at p(N, 0) = 1, which ﬁnally reaches a uniform distribution.

Note the Wigner function is actually in the momentum space, though we keep using variable x in accordance with previous notations in this paper. Lutz ﬁrstly pointed out the above F-P equation admits Tsallis statistics as a stationary solution14,

D x 2 1/(1−q) π(x) = C 1 + 0 , (65) D0 + D1 xc

2 where q = 1 + 2D0/(αxc ). So the optical lattice eﬀectively maximizes the Tsallis entropy in its steady state.

Typical temporal and spatial behaviors of the Tsallis thermodynamics for optical lattices with q = 2 are explored in Fig. 2. Starting from a Gaussian distribution with center a little shifted from zero, solutions of F-P equations in (34) and (64) gradually approach to the stationary Tsallis statistics as time evolves. During this procedure, the free energy (q) (q) (q) F , free energy dissipation rate fd and entropy production rate ep all converge to zero from the positive side as our results have pointed out. A spatial plot on these quantities before integration further shows that their non-negativeness also holds from point to point, (q) expect for the free energy. The non-extensive heat Qne in this case is almost as large as the free energy dissipation rate but with a minus sign, which indicates the importance of this quantity in maintaining a correct form of Tsallis thermodynamics.

18 FIG. 2. The solution and Tsallis thermodynamics for optical lattices modeling by F-P equations with α = D1 = xc = 1 and D0 = 1/2. Spatial distributions of each quantity (before integration) are illustrated at four representative time points, while their overall time evolutions (after integration) are drawn in the insets correspondingly.

6. CONCLUSION

It is well-known that the B-G thermodynamics is inadequate for small systems, in which the non-extensivity plays a fundamental role. Parallel to the B-G statistical thermodynamics, this paper aims to establish a rigorous and self-consistent thermodynamic formalism of master and Fokker-Planck equations with non-extensive Tsallis entropy and free energy. Interestingly, the conventional thermodynamics with B-G entropy turns to be a special case of our study in the limit of q → 1. To be concrete, we ﬁrst derived the Tsallis thermodynamics for Markov processes de- (q) scribed by master equations, including the entropy production rate ep , free energy dissi- (q) (q) (q) pation rate fd , house-keeping heat Qhk , heat dissipation rate hd , excess heat exchange (q) (q) rate Qex and the non-extensive heat Qne , as well as three balance equations involving above quantities as a manifestation of “three faces of the second law of thermodynamics” according to Esposito et al.2. Then, we follow the same routine and build the Tsallis thermodynamics for F-P equations. Finally, to justify above formulations, we deduced the Tsallis thermodynamics of master equations from the discretized F-P equations when the discretization step

19 size goes to zero. A similar one-to-one correspondence emerges from master equations to F-P equations too, when the Markov process is restrained to one-step jump.

Compared to results for B-G thermodynamics, a direct consequence of the non-extensivity of Tsallis free energy is the emergency of a non-local Onsager’s force (see Eq. (18)). Besides, the diﬀerence between the entropy production rate and free energy dissipation rate is not positive any more. Actually, as we have shown, only a part of it gives the correct house- keeping heat, which is always non-negative and vanishes if and only if at the steady state. The reminding part characterizes the non-extensivity of the system and is thus named as (q) the non-extensive heat Qne , since limq→1 Qne = 0. Furthermore, we have also shown the power of coarse-graining procedure or correct mathematical limit process in discovering new thermodynamic expressions and relations, exploring underlying novel connections and correspondence between diﬀerent levels of descriptions of the same system, which turns out to both mathematically critical and physically insightful in the study of non-equilibrium thermodynamics.

ACKNOWLEDGMENT

L.H. acknowledged the ﬁnancial supports from the National Natural Science Foundation of China (Grants 21877070) and Tsinghua University Initiative Scientiﬁc Research Program (Grants 20151080424).

APPENDIX: RIGOROUS PROOFS ON TSALLIS THERMODYNAMICS OF MASTER EQUATIONS AND F-P EQUATIONS

(1) Proof of Theorem 4.1: The free energy dissipation rate for F-P equations is given by

Z 1 ∂ p f F = − dxK(x, t) ( )q−1. d q − 1 ∂x π

20 By Taylor series expansion, we discretize the integral function with respect to x = i to obtain X 1 1 pi+1 pi f F () = − u¯ p − (D p − D p ) + O() ( )q−1 − ( )q−1 + O() d q − 1 i i i+1 i+1 i i π π i i+1 i X Di+1pi+1 − Dipi − u¯i+1pi+1 1 pi+1 pi = + O(1) ( )q−1 − ( )q−1 + O(2) 2 q − 1 π π i i+1 i X ≡ I1(i, t)I2(i, t), i where the summand is denoted as I1(i, t)I2(i, t). The ﬁrst part and second part read separately as 1 I (i, t) ≡ D p − D p − u¯ p + O(1) 1 2 i+1 i+1 i i i+1 i+1

= −(piwi,i+1 − pi+1wi+1,i) + O(1), 1 pi q−1 pi+1 q−1 2 I2(i, t) ≡ − ( ) + ( ) + O( ). (q − 1) πi πi+1

We notice that the leading terms of I1(i, t) and I2(i, t) are of orders O(1/) and O(), respectively. Direct calculations indicate that,

−1 F lim fd →0 X 1 pi q−1 pi+1 q−1 2 = lim (piwi,i+1 − pi+1wi+1,i) + O(1) · ( ) − ( ) + O( ) →0 (q − 1) π π i i i+1 X 1 pi pi+1 = (p w − p w )( )q−1 − ( )q−1 q − 1 i i,i+1 i+1 i+1,i π π i i i+1 1 X pi pj = (p w − p w )( )q−1 − ( )q−1, 2(q − 1) i i,j j j,i π π i,j=i±1 i j where in the last step we have translated the summation index from i to (i − 1). This completes the proof.

F (2) Proof of Theorem 4.2: Discretizing the entropy production rate ep for F-P equations, one has 2 X 1 eF () = (D p )q−2 u¯ p − (D p − D p ) + O() p i i i i i+1 i+1 i i i 1 X Di+1pi+1 − Dipi − u¯i+1pi+1 = D p − D p − u¯ p + O(2) + O(2)(D p )q−1 i+1 i+1 i i i+1 i+1 D p i i i i i X ≡ I1(i, t)I3(i, t). i

21 Here

1 I (i, t) ≡ D p − D p − u¯ p + O(2) = −(p w − p w ) + O(1), 1 2 i+1 i+1 i i i+1 i+1 i i,i+1 i+1 i+1,i Di+1pi+1 − Dipi − u¯i+1pi+1 2 q−1 I3(i, t) ≡ + O( ) (Dipi) Dipi 1 Di+1pi+1 − u¯i+1pi+1 q−1 2 q−1 = ( ) − 1 + O( ) (Dipi) q − 1 Dipi 2(q−1) = (p w )q−1 − (p w )q−1 + O(2), q − 1 i+1 i+1,i i i,i+1

1 q−1 2 where we have used the relation q−1 [(1 + ζ) − 1] = ζ + O(ζ ) for |ζ| 1 in the second

step during the derivation of I3. Therefore, the limit of entropy production rate becomes

1−2q F X 1 q−1 q−1 lim ep = (piwi,i+1 − pi+1wi+1,i) (piwi,i+1) − (pi+1wi+1,i) →0 q − 1 i 1 X = (p w − p w )(p w )q−1 − (p w )q−1, 2(q − 1) i i,j i+1 j,i i i,j j j,i i,j

M which is exactly the expression of ep in Eq. (23) for master equations with j = i ± 1.

F As to the house-keeping heat, we can rewrite Qhk in Eq. (41) into an alternative form:

Z p Ks Z p QF = dx (Dπ)q−1( )2 = dx [K − (K − Ks)]Ks(Dπ)q−2 hk D π π Z ∂ p = dx [K + Dπ ( )]Ks(Dπ)q−2 . ∂x π

F Discretizing Qhk by x = i, one has

X 1 1 pi+1 pi QF () = u¯ p − (D p − D p ) + D π ( − ) + O() hk i i i+1 i+1 i i i i π π i i+1 i 1 · u¯ π − (D π − D π ) + O() (D π )q−2 i i i+1 i+1 i i i i X Di+1pi+1 − Dipi − u¯i+1pi+1 = + O(1) D π − D π − u¯ π + O(2)(D π )q−2 2 i+1 i+1 i i i+1 i+1 i i i X Diπi pi+1 pi − ( − ) + O(1) D π − D π − u¯ π + O(2)(D π )q−2 2 π π i+1 i+1 i i i+1 i+1 i i i i+1 i X ≡ [I1(i, t)I4(i, t) − I5(i, t)]. i

22 Similarly,

Di+1πi+1 − Diπi − u¯i+1πi+1 2 q−1 I4 ≡ + O( ) (Diπi) Diπi 2(q−1) = (π w )q−1 − (π w )q−1 + O(2), q − 1 i+1 i+1,i i i,i+1

1 pi+1 pi 2 Di+1πi+1 − Diπi − u¯i+1πi+1 2 q I5 ≡ 2 ( − ) + O( ) + O( ) (Diπi) πi+1 πi Diπi 2(q−1) pi+1 pi 2 q q 2 = ( − ) + O( ) (πi+1wi+1,i) − (πiwi,i+1) + O( ) . q πi+1 πi Finally, we arrive at the house-keeping heat of master equations with a tridiagonal transition rate matrix as

1−2q F 1 X q−1 q−1 lim Qhk = (piwi,j − pjwj,i) (πiwi,j) − (πjwj,i) →0 2(q − 1) i,j 1 X pi pj − ( − )(π w )q − (π w )q, 2q π π i i,j j j,i i,j i j based on the symmetry of summands with respect to the indices i and j = i ± 1.

(3) Proof of Theorem 4.3: Recalling the free energy dissipation rate for master equations 1 X f M (t) = φ (t)Φ (t) ≥ 0, d 2 i,j i,j i,j with j = i ± 1. We take j = i − 1 as an example. The case j = i + 1 is the same, P P P since i φi,i+1Φi,i+1 = i φi−1,iΦi−1,i = i φi,i−1Φi,i−1 due to the symmetry of indices and reﬂecting boundary conditions. First, we substitute the relations (58) into the ﬂux

φi,i−1 =piwi,i−1 − pi−1wi−1,i 1 1 1 1 =p( D − u) − [p − ∂ p + O(2)]( D + u) 2 2 x 2 2 1 1 = − (up − D∂ p) + O(1) ≡ − K + O(1). x Second, the force term is treated as

1 pi q−1 pi−1 q−1 ∂ p q−1 Φi,i−1 = ( ) − ( ) = ( ) + O() . q − 1 πi πi−1 q − 1 ∂x π P 1 R Recalling the relation i = dx, one has the density of free energy dissipation rate for master equations Z M K ∂ p q−1 F lim fd = − dx ( ) ≡ fd ≥ 0. →0 q − 1 ∂x π

23 (4) Proof of Theorem 4.4: Denote the entropy production rate for master equations as

1 X 1 X eM = (p w − p w )[(p w )q−1 − (p w )q−1] ≡ φ Ψ , p 2(q − 1) i i,j j j,i i i,j j j,i 2 i,j i,j i,j i,j

with

1 Ψ = [(p w )q−1 − (p w )q−1]. i,j q − 1 i i,j j j,i Choosing j = i − 1 and substituting the relations (58) into the above equation, one deduces that

1 Ψ = [(p w )q−1 − (p w )q−1] i,i−1 q − 1 i i,i−1 i−1 i−1,i 1 1 1 1 1 = p( D − u)q−1 − p − ∂ p + O(2)q−1( D + u)q−1 q − 1 2 2 x 2 2 1 (Dp)q−1 u u = 1 − q−1 − 1 + − ∂ (ln p) + O(2)q−1 2(q−1) q − 1 2D 2D x 1 = − (Dp)q−2[K + O()]. 2q−3

Combining the ﬂux φi,i−1 and the associated force Ψi,i−1, we have Z 2q−1 M q−2 2 lim ep = dx(Dp) K ≥ 0, (66) →0

F which is exactly the deﬁnition of ep in (38) for F-P equations. M The house-keeping heat could be treat analogously. Based on Eq. (25), Qhk is rewritten as

X 1 QM = (p w − p w )[(π w )q−1 − (π w )q−1] hk q − 1 i i,i−1 i−1 i−1,i i i,i−1 i−1 i−1,i i X 1 pi pi−1 − ( − )(π w )q − (π w )q q π π i i,i−1 i−1 i−1,i i i i−1 X ¯ X ≡ φi,i−1Ψi,i−1 − ∆i,i−1. i i

M We notice that the ﬁrst term of Qhk is the product of the ﬂux φi,i−1 and corresponding force 1 1 Ψ¯ ≡ (π w )q−1 − (π w )q−1 = − (Dπ)q−2[Ks + O()], i,i−1 q − 1 i i,i−1 i−1 i−1,i 2q−3

24 while the second terms is

1 pi pi−1 q q ∆i,i−1 ≡ ( − ) (πiwi,i−1) − (πi−1wi−1,i) q πi πi−1 1 ∂ p 1 1 1 1 = ( ) + O(2) π( D − u)q − π − ∂ π + O(2)q( D + u)q q ∂x π 2 2 x 2 2 1 ∂ p = − (Dπ)q−1 ( ) + O()[Ks + O()]. 2q−2 ∂x π

As a result, in the limit of → 0, the density of house-keeping heat of master equations becomes Z 2q−1 M q−2 q−1 ∂ p s lim Qhk = dx (Dπ) K + (Dπ) ( ) K →0 ∂x π Z p Ks = dx (Dπ)q−1( )2 ≥ 0, D π

F which emerges as Qhk for F-P equations. This completes our proof.

REFERENCES

1Hao Ge and Hong Qian. Physical origins of entropy production, free energy dissipation, and their mathematical representations. Physical Review E, 81(5):051133, 2010. 2Massimiliano Esposito and Christian Van den Broeck. Three faces of the second law. i. master equation formulation. Physical Review E, 82(1):011143, 2010. 3Hao Ge. Extended forms of the second law for general time-dependent stochastic processes. Physical Review E, 80(2):021137, 2009. 4Christian Van den Broeck and Massimiliano Esposito. Three faces of the second law. ii. Fokker-Planck formulation. Physical Review E, 82(1):011144, 2010. 5Terrell L. Hill. Thermodynamics of small systems. Dover, New York, 1963. 6Sumiyoshi Abe and Yuko Okamoto. Nonextensive statistical mechanics and its applications, volume 560. Springer Science & Business Media, 2001. 7Constantino Tsallis. Possible generalization of Boltzmann-Gibbs statistics. Journal of Statistical Physics, 52(1):479–487, 1988. 8A. R´enyi. On the dimension and entropy of probability distributions. Acta Mathematica Academiae Scientiarum Hungarica, 10(1):193–215, Mar 1959. 9A. R´enyi. Probability Theory. North Holland, Amsterdam, 1970.

25 10G. Kaniadakis. Non-linear kinetics underlying generalized statistics. Physica A: Statistical Mechanics and its Applications, 296(3-4):405–425, 2001. 11Jean Pierre Boon and Constantino Tsallis. Special issue overview nonextensive statistical mechanics: new trends, new perspectives. Europhysics News, 36(6):185–186, 2005. 12Christian Beck. Generalised information and entropy measures in physics. Contemporary Physics, 50(4):495–510, 2009. 13Karen E Daniels, Christian Beck, and Eberhard Bodenschatz. Defect turbulence and generalized statistical mechanics. Physica D: Nonlinear Phenomena, 193(1-4):208–217, 2004. 14Eric Lutz. Anomalous diﬀusion and tsallis statistics in an optical lattice. Physical Review A, 67(5):051402, 2003. 15Lisa Borland. Option pricing formulas based on a non-gaussian stock price model. Physical Review Letters, 89(9):098701, 2002. 16Shun-ichi Amari and Hiroshi Nagaoka. Methods of information geometry, volume 191. American Mathematical Soc., 2007. 17Victor Kac and Pokman Cheung. Quantum calculus. Springer Science & Business Media, 2001. 18A. Hobson. A new theorem of information theory. Journal of Statistical Physics, 1(3):383– 391, 1969. 19John E Shore and Rodney W Johnson. Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy. Information Theory IEEE Transactions on, 26(1):26–37, 1980. 20Masatoshi Shiino. H-theorem with generalized relative entropies and the Tsallis statistics. Journal of the Physical Society of Japan, 67(11):3658–3660, 1998. 21A.I. Khinchin. The entropy concept in probability theory. English translation in Khinchin, A.I. (1957) Mathematical Foundations of Information Theory. Translated by R.A. Silver- man and M.D. Friedman, Dover, New York, pp. 1-28. Uspekhi Matematicheskikj Nauk., 8:3–20, 1953. 22Sumiyoshi Abe. Axioms and uniqueness theorem for tsallis entropy. Physics Letters A, 271(1-2):74–79, 2000. 23Nicolaas Godfried Van Kampen. Stochastic processes in physics and chemistry, volume 1. Elsevier, 1992.

26 24Yoshitaka Tanimura. Stochastic Liouville, Langevin, Fokker-Planck, and master equation approaches to quantum dissipative systems. Journal of the Physical Society of Japan, 75(8):082001, 2006. 25Hao Ge and Hong Qian. Mesoscopic kinetic basis of macroscopic chemical thermodynamics: A mathematical theory. Physical Review E, 94(5-1):052150, 2016. 26Da Quan Jiang, Min Qian, and Min Ping Qian. Mathematical Theory of Nonequilibrium Steady-state, volume 1833. Springer, New York, 2005. 27Yoshitsugu Oono and Marco Paniconi. Steady state thermodynamics. Progress of Theo- retical Physics Supplement, 130:29–44, 1998. 28Massimiliano Esposito and Christian Van den Broeck. Three detailed ﬂuctuation theorems. Physical Review Letters, 104(9):090601, 2010. 29Liangrong Peng, Yi Zhu, and Liu Hong. The Markov process admits a consistent steady- state thermodynamic formalism. Journal of Mathematical Physics, 59(1):013302, 2018. 30Hong Qian. Phosphorylation energy hypothesis: open chemical systems and their biological functions. Annu. Rev. Phys. Chem., 58:113–142, 2007.