Nonlinear Stochastic Dynamics of Complex Systems, III

Home , Detailed balance

arXiv:1606.02391v1 [cond-mat.stat-mech] 8 Jun 2016 investigated. fmteaial mret“ispto” hc sntrel not is which “dissipation”, emergent mathematically of S notemdt ehv riuae n[]tento that notion the [2] in articulated have wrench we monkey minus-log- midst, a a the Throwing sharing into principle”. unifyi except single a entropy unrelated of parts maximum are completely “they to ‘entropy’,” are term common “they r from discussions endless ing generated has [3] theory information space. probability underlying hroyaiswt neatcas-riig n .G. J. and coarse-graining, chemic exact of statistical this concept Kirkwood’s an equilibrium of origin to with The back thermodynamics dynamics. traced complex was a practice in force entropic an n .Qa eal [email protected]). (email: Qian H. and hrfr,teter fifraini hoyconcerning theory a is information communication. of in theory messages mathematical a the certain the or Therefore, represent is dynamics, contex information that its specific Shannon, symbols engineering of by the character In discussed [5]? a first [4], or st “energy” a of system, it form Is complex information? a is What of remains: still question naive oeulbimsed tt probability state steady nonequilibrium riigi mhszd nakntcsse hr et fab a of death where system kinetic a In emphasized. is graining oue naprmti aiyo rbblt distribution probability of family theor parametric information p a b and motivated on mechanics II, focuses Part statistical overview. Gibbsian an both provides I bee Part have space, presented. phase e.g in non (stoc-NET) approach, stochastic thermodynamics librium a kinetic as nonlinear well as stochastic Delbrück-Gillespie processes, mesoscopic a of I ytmsre sa rhtp o iceia self-replica biochemical for archetype an i as process, Delbrück-Gillespie respectivel serves a energy, system by free represented and kinetics, entropy thermodynamic become ytmi iutdi neulbimevrnet tagrees it environment, equilibrium an in situated is system rbblt distribution probability eocpckntcsses ohteisatnosShanno instantaneous the Both systems. kinetic mesoscopic ( orsodn uhr:D .Sain(mi:saakian@phys (email: Saakian B. D. authors: Corresponding ,β x, ne Terms Index h eainbtenGbsa ttsia ehnc and mechanics statistical Gibbsian between relation The ebifl eiwtercnl eeoe,Mro rcs bas process Markov developed, recently the review briefly We atIadPr Io hssre 1,[] chemical a [2], [1], series terms in this systems of complex II on Part perspective kinetic and reaction I Part N sadnmrbesto vns and events, of set denumerable a is ) oeiiru hroyaiso efRpiainKinetics Self-Replication of Thermodynamics Noneqilibrium sanovel a as p olna tcatcDnmc fCmlxSses III: Systems, Complex of Dynamics Stochastic Nonlinear ss Bohsc,ceia atreuin hmclpotential chemical eqution, master chemical —Biophysics, 3 a eavr eiiaeptnilfnto of function potential legitimate very a be can eateto ple ahmtc,Uiest fWashingt of University Mathematics, Applied of Department 2 hoeia device theoretical .I lkaynNtoa cec aoaoy(eea Phys (Yerevan Laboratory Science National Alikhanyan I. A. .I I. oeta fma force mean of potential { p NTRODUCTION α 1 nttt fPyis cdmaSnc,Nnag api115 Taipei Nankang, Sinica, Academia Physics, of Institute ( t ) } lypoietrls h hoyi eea;ada specia a as and general; is theory The roles. prominent play , lkainBohr tet eea 706 Armenia 375036, Yerevan Street, Brothers Alikhanian 2 o xlrn single a exploring for P ( ai .Saakian B. David x ( ) S , h seemingly The . x , F ∈ .sinica.edu.tw) P , S ) where , hmsai oeulbimevrnet h pn driv open, The environment. nonequilibrium chemostatic a n sthe is equi- ang- in h infiac of significance The tion. entropy n tdt h etmaue ntrsof terms in measured heat the to ated efcl ihGbsa hmclthermodynamics: chemical Gibbsian with perfectly ate ng .W pl hster oaflyrvril uoaayi r autocatalytic reversible fully a to theory this apply We y. al y, ooia raimi rae stervra fisbrh t birth, its of reversal the as treated is organism iological y n ., s t 1,2 ebeter feulbimmtes[0,teprobability the [10], state matters a equilibrium of theory semble formula. Gibbs-Shannon gives value expected whose is blt distribution ability eypriua ucino admvariable random simpl a is of information function which particular very in a probability of theory iomatic o notion the [7]. on an [6], emphasis entropy as singular biology, Shannon essentially a to with used theory probability was information applied theory applying of information particula days mathematical sense, early broader uncer- much the a under in in hand, other communications the On and tainty. representations, coding, h motn equation important the aldfe nry n nrp is entropy and energy, free called sBlzansconstant, Boltzmann’s is tr nKelvin, in ature nsaitclthermodynamics, statistical In htteetoyo h atri then is st matter mechanics the statistical of of entropy theory the The that matter. the of states nwhich in diohra hmcltemdnmc o olna,driv nonlinear, for thermodynamics chemical isothermal ed n ogQian Hong and ti motn ormme hti ib’saitclen- statistical Gibbs’ in that remember to important is It ietpigteeise,w olwKlooo’ ax- Kolmogorov’s follow we issues, these Sidestepping p S S x ealdblne nrp,Glepeagrtm informa algorithm, Gillespie entropy, balance, detailed , ( [ p T = x α = ) ( e famte hti ncnatwt eprtr bath temperature a with contact in is that matter a of E t − )] x Ξ( E − n etl,W 89-95 USA 98195-3925, WA Seattle, on, steitra nryo h state the of energy internal the is x n eaieentropy relative and k T /k B ) B k x c nttt)Foundation Institute) ics T B X 3 hroyaial consistent thermodynamically ∈ p , S 1 = x p where 8,[9]: [8], 9 Taiwan 29, x F . 38064852 ln S = p E stestcnann l possible all containing set the is x aewe hmclreaction chemical a when case l Ξ( = − H F T k x ∂T S, T F ( = ) ∂ B ( × T x [ T − p = ) ) 10 ean ob further be to remains , ∂F/∂T α x k X ∈ ( − ≡ − B t S 23 − T )] endbsdon based defined , e nΞ( ln ln m k − k B n lohas also One . E x p 2 B iei coarse- kinetic x S x , T gs kg x /k T ( T x nchemical en and emeaning he ) nΞ( ln B ihprob- with ) stemper- is , T . − x , 2 k eaction T ∈ B K tion. (1b) ) (1c) (1a) ates F T S rly − en of is y 1 1 f , 2 in which E is the mean internal energy thermal physics has already been lost.1 This difficulty has been hinted in [21]: “If we assume that the process I → II is ‘irreversible’ this implies that the time-reversed dynamics is E(T )= Expx. (1d) very unstable so that Pr{II → I} is hard to estimate precisely. x∈S X One uses instead a bigger probability Pr{II → I} based on observable processes.” Unfortunately, any remotely reasonable Viewed from these four widely known mathematical equations, substitution for an authentic de-birth is currently out of reach; the theoretical device discussed in [2] simply provides an death is not an acceptable alternative even for a lower bound extended exploration of the information content of P (x) and estimation. its underlying (S , F, P ). The paper is structured as follows: In Sec. II we give a brief review of the modern theory of stochastic nonequilibrium There is a growing interest in quantitative relations between the theory of thermodynamics that rules many aspects of the thermodynamics of Markov systems [22], [23], [24], [25], as presented in Part I [1]. While many of the terminologies and biochemical world and the biological dynamics of organisms, their ecology and evolution. “Every species of living thing can mathematical relations can be found in the review articles [26], [27], [28] and classic text [29], we like to point out that the make a copy of itself by exchanging energy and matter with its coherent narrative constructed based on the mathematics, e.g., surroundings.” [11] A living organism has two distinct aspects: One with “genomic information” which is carried in the their interpretations, is unique. We shall show how mathematical logic dictates that entropy in systems with uniform sequence of polynucleic acids [7], possibly including methy- stationary probability should be replaced by a relative entropy, lation modifications of base-pairs; and the other is a space- time biochemical dynamics in terms of molecules, which can also known as free energy, in systems with non-uniform stationary probability, and how entropy production rate exhibit chemical, mechanical, and electrical characteristics. ep arises as a distinctly different concept as entropy change The time scales of these two aspects are vastly different: dS . Then in Sec. III, we first give the standard chemical The former cumulates changes generation after generation, dt thermodynamic treatment of the simple kinetic system with which constitutes an evolutionary process. The latter defines “what is living” for each and every individual organism. The two reversible reactions relationship between these two has been the subject of many k+1 k+2 FGGGGB GGGG FGGGGB GGGG scholarly studies in the field of biology [12], [13], [14], [15], A + X 2X, X B. (2) k−1 k−2 [16]. To a classic physicist or chemist, the cellular biochemical Note, if one neglects the two “backward” reactions, e.g., letting processes are various transformations of energy and matters; k−1 = k−2 = 0, then the chemical kinetics can be described to a modern cell biologist, the same processes are usually by the differential equation described as regulation and processing of informations. The dx = gx − k x, g ≡ k a. (3) physical and chemical processes are interpreted with biological dt +2 +1 functions [17]. The present paper is an attempt to provide a treatment of this latter aspect in terms of a stochastic Eq. 3 describes the “birth and death” of individual X kinetic theory. In particular, we analyze rigorously a chemical molecules. model that makes the notion of “making a copy of itself by The simple chemical model in (2) has a clear self-replication exchanging energy and matter with its surroundings” precise characteristics, it allows a rigorous thermodynamics analysis and quantitative. To this end, we revisit a kinetic model for of the replication/synthesis aspect of a biological organism. a fully reversible chemical reaction system that consists of Certainly the individual X lacks other fundamentals of a an autocatalytic step A + X −→ 2X [18], [19]. This model living being: An individual X itself is a dead molecule, not a first appeared in the wonderful book by the late Professor living organism. Therefore, while it is meaningful to ask the Keizer [20]. This extremely simple model, however, offers us heat dissipation in a self-replication from a complex chemical an insight on why a truly meaningful chemical thermodynamic synthesis standpoint [30], it might not be a sufficient model for study of biological replication is premature: This is because the heat dissipation in the self-replication of a living organism neither the current cosmological time scale of our universe since even just being alive, an organism has a continuous heat nor the size of entire biological population on our planet are dissipation as an individual entity, e.g., active metabolism. sufficiently large to provide a meaningful estimation of the Simply put: a living organism, which has both metabolism and probability of “two [daughter cells] somehow spontaneously self-replication, already has a basal level of heat dissipation revert back into one” [11]. Note the “revert back” should not due to the former even in the absence of the latter. The present be a fusion of two cells, rather it has to be a process that analysis, however, makes a conceptual separation between the reverses the entire event-by-event of a cell replication. two processes in an organism. We note that the open chemical systems theory of motor protein chemomechanics serves as a It is the non-zero probability of such completely incon- concrete model for the former [31]. ceivable processes, in a physicist’s world, that provides an estimation of thermodynamic heat production. When a biolo- 1On the other hand, the upper bound of such an extremely small probability gist considers only feasible events to a living organism being can be estimated from knowing the minimal amount of calories required to birth and death, the quantitative connection to the world of reproduce an organism using the very equation (2) in [11]. 3

In Sec. IV, a mesoscopic stochastic thermodynamic treat- Inspired by the similarity between the expression in Eq. 5 ment of the replication kinetics is carried out. Entropy produc- and Boltzmann’s law, let us deﬁne “the internal energy density tion rate for the kinetic process is studied. Then in Sec. V, we of the Markov state α”: articulate a distinction between thermodynamically consistent ln π E = − α . (6) and inconsistent kinetic approximations. The paper concludes α ω with a discussion in Sec. VI. Then the mean energy density at time t:

−1 II. AMODERN STOCHASTIC NONEQUILIBRIUM E(t)= pα(t)Eα = −ω pα(t) ln πα, (7) THERMODYNAMICS OF MARKOV SYSTEMS α∈S α∈S X X The very brief summary of Gibbs’ equilibrium statistical where pα(t) is the probability of the system in state α at time thermodynamics, given in Eq. 1a-d, illustrates the universality t. Then the free energy, also known as Massieu potential, of of the entropy function, be it in information theory or in the Markov system at time t thermal physics. Indeed there is now a more unifying theory of pα(t) entropy and relative entropy, also known as Kullback-Leibler F pα(t) = ωE − S(t)= pα(t) ln . (8) πα (KL) divergence in information theory and free energy in α∈S X statistical thermodynamics, based on the probability theory of Then, concerning these quantities, one has a series of mathe- Markov processes that describe complex system’s dynamics in matical results: 2 phase space [22], [23], [25]. A free energy balance equation. First, an equation that is The ﬁrst basic assumption of this stochastic nonequilibrium valid for systems with and without detailed balance, and for thermodynamic theory is that a complex mesoscopic dynami- processes in stationary and in transient: cal system can be represented by an irreducible continuous time Markov process with appropriate state space S and d F pα(t) = ωEin pα(t) − ep pα(t) , (9a) transition probability rates qαβ, α, β ∈ S , where qαβ = 0 dt if and only if qβα = 0. Under these assumptions, it can be in which shown that there exists a unique positive stationary probability 1 π q E p := p q − p q ln α αβ ≥ 0, πα for the Markov system in stationarity: in α α αβ β βα ω πβ qβα α,β∈S X παqαβ =0. (4) (9b) α∈S pαqαβ X ep pα := pαqαβ −pβqβα ln ≥ 0. (9c) pβqβα For a stationarity process deﬁned by {π } and q , a further α,β∈S α αβ X distinction between an equilibrium steady state and a nonequi- Eq. 9 is interpreted as a free energy balance equation for librium steady state (NESS) can be made (see below): The a Markov system with Ein[pα] being the instantaneous rate former has zero entropy production rate, and the latter has of input energy, a source term, and ep[pα], a sink, as the a strictly positive entropy production rate. This distinction is instantaneous rate of energy lost, or entropy production rate. determined by the set of transition probability rates {qαβ}, Entropy change, free energy change, and entropy produc- which necessarily satisfy detailed balance, παqαβ = πβqβα tion. Second, an inequality that is valid for systems with and S ∀α, β ∈ , if and only if the steady state is an equilibrium. without detailed balance, and for processes in stationary and The theory presented below is applicable to both equilibrium in transient: and nonequilibrium steady state, as well as a time-dependent dF (t) ≤ 0. (10) non-stationary process. dt For a mesoscopic system such as the discrete chemical Combined with (9), this implies ω−1e ≥ E . S p in reactions in (2) with state space being non-negative in- One notices that for very particular Markov systems with tegers, a second assumption is that it has a macroscopic q = q ∀α, β ∈ S , they have a uniform π ≡ constant. n αβ βα α corresponding continuous dynamics x(t), x ∈ R . Let ω be dS Then ep(t)= dt . Such systems are analogous to microcanon- the “size parameter” that connects the mesoscopic system and ical ensembles, where the entropy production is the same as its macroscopic limit when , with α being number ω → ∞ x ≡ ω entropy increase. density. Then, for a large class of such Markov systems [24], For Markov systems with detailed balance, Ein(t) ≡ 0 ∀t, [33], π x x and ω ≃ e−ωϕ( ) → δ x − z , (5) ω d ep pα = S pα(t) − ωEex pα(t) ≥ 0, (11a) in which function ϕ(x) ≥ 0 and its global minimum is at z. dt In applied mathematical theory of singular perturbation, Eq. 5 in which is known as WKB ansatz [34]; in the theory of probability, it (11b) is called large deviations principle [32]. Eex pα := pαqαβ − pβqβα Eβ − Eα , α,β∈S X 2Another unifying approach to entropy is the theory of large deviations is interpreted as the instantaneous rate of energy exchange. [32]. A deep relation between the entropy function in large deviations theory and the entropy function in complex dynamics is Sanov’s theorem. See below The inequality in Eq. 11a is analogous to Clausius inequility as well as [24], [33]. for spontaneous thermodynamic processes, which gives rise 4

to the notion of entropy production as a distinctly different notices that in a semi-reversible case when k−2 = 0 in the dS concept as dt . system (2), extinction of X is the only long time fate of the dx 2 Macroscopic limit. For a macroscopic system in the limit kinetics. The differential equation dt = g − k+2 x − k−1x , ss of ω → ∞, the probability distribution p (t) becomes a however, predicts a stable population x = g − k /k− . α +2 1 deterministic dynamics x(t), and Eq. 9 becomes a novel This disagreement is known as Keizer’s paradox [18], [36]. macroscopic equation [24], [33] We shall use this model as a concrete case of the chemical nature of exchanging of energy and matter in self-replication. dϕ x(t) = cmf x(t) − σ x(t) , (12a) dt in which A. NESS of an open chemical system According to (2), an X molecular has transformed the raw cmf[x] = J +(x) − J −(x) ℓ ℓ material in the form of A into a copy of itself, a second X. all reactions ℓ: X The canonical mathematical description of this autocatalytic J +(x) ℓ −νℓ·∇xϕ(x) dx × ln e , (12b) reaction is dt = gx where the constant g = k+1a. We J −(x) ℓ shall use a,x,b to denote the concentrations of A,X,B, J +(x) respectively. Clearly, g can be identified as per capita birth σ[x] = J +(x) − J −(x) ln ℓ , ℓ ℓ J −(x) rate if one is interested in the population dynamics of X. all reactions ℓ ℓ: X Combining with the second reaction, one has dx = gx−k x, (12c) dt +2 where k+2 can be identified as a per capita death rate. When + − th Jℓ and Jℓ are the forward and backward fluxes of the ℓ the X is dead, the material in terms of atoms are in the form of reversible reaction, integer vector νℓ is its stoichiometry coeffi- B. Therefore, the “birth” and “death” of X, or more precisely cients. cmf is the chemical motive force that sustains a reaction the synthesis and degradation of X, involve an exchange of system out of its equilibrium, and σ is the macroscopic rate materials, as source and waste, with its surroundings. One can of entropy production. in fact assume that the concentrations of A and B, as the − ν x + x x ℓ·∇xϕ( ) x If Jℓ ( )= Jℓ ( )e ∀ℓ, , it is known as detailed environment of the chemical reaction system in (2), are kept balance in chemistry, or G. N. Lewis’ law of entire equi- at constant, as a chemostat. This is precisely why living cells librium [35]. In such systems, cmf = 0, ϕ(x) is the Gibbs have to be “cultured”. function, ∂ϕ(x)/∂xk is the chemical potential of species k, The calorie count one reads from a food label in a su- and νℓ · ∇xϕ(x) is the chemical potential difference of the permarket gives the chemical potential difference between A ℓth reaction. and B, introduced below. This is not different from one reads Macroscopic systems with fluctuations. For a system with the electrical potential difference on a battery to be used for very large but finite ω, Eq. 5 provides a “universal”, asymptotic keeping a radio “alive”. expression for stationary, fluctuating x [32]: One might wonder why we assume the reactions in (2) reversible? It turns out, as anyone familiar with chemical e−ωϕ(x)+ψ(x) fx(x)= , (13) thermodynamics knows, one can not discuss energetics in an Ξ(ω) irreversible reaction system. If the k−2 =0, then the chemical where energy difference between X and B are infinite; which is x x Ξ(ω)= e−ωϕ( )+ψ( )dx, (14) clearly unrealistic. In fact, the chemical potential difference Rn Z between A and B, in kB T unit, is in which ϕ(x) is uniquely defined with minx ϕ(x)=0. µA − µB = µA − µX + µX − µB k a k x III. AN OPEN CHEMICAL SYSTEM AS A = ln +1 + ln +2 SELF-REPLICATING ENTITY k−1x k−2b a We now apply the general theory in Sec. II to reaction sys- = − ln Keq + , (16) AB b tem (2). There are two reversible reactions with stoichiometric coefficients and , respectively. Let be in which the overall equilibrium constant between A and B, ν1 = +1 ν2 = −1 x(t) eq the concentration of X at time t, the kinetics of the reaction KAB = k−1k−2/(k+1k+2). In a chemical or biochemical system in (2) can be described by laboratory, the equilibrium constant is usually determined in an experiment according to dx(t) + − = νℓ Jℓ (x) − Jℓ (x) equi. conc. of A dt Keq = . ℓX=1,2 AB equi. conc. of B 2 = k+1ax − k−1x − k+2x + k−2b, (15) If µA >µB, then there is a continuous material flow from A to + − 2 + with J1 (x) = k+1ax, J1 (x) = k−1x , J2 = k+2x, and B, even when the concentration of X is in a steady state: There − J2 = k−2b. The first of these two reversible reactions in (2) is a continuous birth and death, synthesis and degradation: is known as autocatalytic. One can find many many examples metabolism in an “living system”. There is an amount of of this abstract system of reactions in biochemical literature; entropy being produced in the surroundings. An agent has to see [19] for more extensive biochemical motivations. One also constantly generating A with high chemical potential from B 5 with low chemical potential. This entropy production in fact in which (see Eq. 37 below) is precisely the µA − µB [37]. x 2 k− z + k z The kinetics of (2), described by Eq. 15, eventually reach a ϕss x = ln 1 +2 dz, (23) ss k az + k− b steady state, with the concentration of X, x as the positive Z0 +1 2 root of the polynomial on the right-hand-side of (15): and the instantaneous chemical motive force

ss 1 2 + − x = k+1a − k+2 + k+1a − k+2 +4k−1k−2b . cmf x = Jℓ (x) − Jℓ (x) 2k−1 q ℓ=1,2 (17) X + J (x) − ss The net flux from A to B in the steady state, × ln ℓ e νℓ∂ϕ (x)/∂x (24) J −(x) ss + ss − ss + ss − ℓ J net A→B = J (x ) − J (x ) = J (x ) − J ss 1 1 2 2 2 k+1ax ∂ϕ (x)/∂x = k+1ax − k−1x ln e 1 2 k− x2 = 2λ + k+2 k+1a + k+2 − 4λ 1 − 2k 1 ss q k+2x −∂ϕ (x)/∂x 2 + k+2x − k−2b ln e ,(25) − ak k + k , (18) k− b +1 +2 +2 2 i with where ss 2 ∂ϕ (x) k− x + k x − 1 +2 µA µB = ln . (26) λ = ak k − bk− k− = bk− k− e kB T − 1 . ∂x k+1ax + k−2b +1 +2 1 2 1 2 And the steady-state entropy production rate according to Eq. That is, 12c is 2 cmf x = k+1ax − k−1x ss ss σ x = J net A→B × µA − µB . (19) −1 k+2x k−2b × ln 1+ 1+ 2 We see that the steady state is actually an equilibrium if k−1x k+1ax ! ss and only if when µA = µB. In this case, σ[x ] is zero, ss + k+2x − k−2b (27) λ = 0, and one can check that J net A→B = 0: There 2 −1 is no net transformation of A to B via X. Otherwise, the k+1ax k−1x ss × ln +1 +1 . J net A→B 6=0, and it has the same sign as λ and (µA − µB). k−2b k+2x ! Therefore, the macroscopic entropy production rate defined in (19), which is never negative, mathematically quantifies the This is an example of Eq. 12a. Finally, statistical irreversibility in the self-replication of X. 2 d dx(t) k− x + k x Generalizing this example is straightforward; hence with ϕ x(t) = ln 1 +2 dt dt k ax + k− b this in mind one can claim that [11] “[e]very species of living +1 2 2 thing can make a copy of itself by exchanging energy and = k+1ax − k−1x − k+2x + k−2b matter with its surroundings”, which can be exactly computed 2 k− x + k x if all the reversible biochemical reactions involved are known. × ln 1 +2 ≤ 0. (28) k ax + k− b +1 2 This inequality implies even for a driven chemical reaction B. Time-dependent entropy production rate system that approaches to a NESS, there exists a meaningful We now turn our attention to the non-stationary transient “potential function” ϕ[x] which never increases. thermodynamics of system (2). First, the time-dependent con- To connect to the known Gibbsian equilibrium chemical centration of X, x(t) is readily solved from (15): thermodynamics, we notice that chemical detailed balance − cmf[x]=0 ∀x implies βCe βt (20) x(t)= −βt , + − 1 − Ck−1e J1 (x) ∂ϕ(x)/∂x J2 (x) − = e = + , (29) in which J1 (x) J2 (x)

2 x(0) that is k+1k+2a/k−1k−2b =1. Under this condition, ϕ[x] in β = k+1a − k+2 +4k−1k−2b, C = . k−1x(0) + β (23) becomes q Then, the time-dependent entropy production rate , in k−1x σ[x(t)] ϕ x = x ln = x µ − µ , (30) k T unit, k a X A B +1 2 k+1a which is the Gibbs function, in unit of kB T , with state A as σ x(t) = k+1ax(t) − k−1x (t) ln k−1x(t) reference. k+2x(t) The function ϕ[x] in (23) is a nonequilibrium generalization + k+2x(t) − k−2b ln (21) of the Gibbs free energy for open chemical systems that k− b 2 d approach to NESS [24], [33], and Eq. 28 is an open-system = cmf x + ϕss x(t) . (22) generalization of the 2nd Law. dt 6

IV. STOCHASTIC KINETIC DESCRIPTIONBYTHE B. Macroscopic limit DELBRUCK¨ -GILLESPIE PROCESS With increasing ω, the behavior of the Markov process Chemical reactions at the individual molecule level are described by Eq. 31 becomes very close to that described by stochastic [38], which can be described by the theory of the mass-action kinetics. In fact, if we let x = n/ω as the Chemical Master Equation (CME) [39] first appeared in the concentration of the species X, then work of Leontovich [40] and Delbrück [41], whose fluctuating p (t) ωx ∼ e−ωϕ(x,t) → δ x − z(t) , (33) trajectories can be exactly computed using the stochastic ω simulation method widely known as Gillespie algorithm [42]. where These two descriptions are not two different theories, rather dz(t) − = J +(z) − J (z), (34) they are the two aspects of a same Markov process, just as the dt Fokker-Planck equation and the It¯ointegral descriptions of a uωz − wωz J +(z) = lim , J (z) = lim , (35) same Langevin dynamics. More importantly, this probabilistic ω→∞ ω ω→∞ ω description and the deterministic mass-action kinetics in Eq. and [24], [44], [45] 15 are just two parts of a same dynamic theory: The latter is 2 ∂ϕ(x, t) ′ the limit of the former if fluctuations are sufficiently small, + νℓϕx(x,t) = Jℓ (x) 1 − e when the volume of the reaction vessel, ω, is large [43]. ∂t Xℓ=1 h i We now show this theory provides a more complete kinetic − − ′ + J (x) 1 − e νℓϕx(x,t)] . (36) characterization of (2) and it is in perfect agreement with ℓ the classical chemical kinetics as well as Gibbs’ chemical It turns out that the macroscopic,h nonequilibriumi chemical thermodynamics. Since chemical species in (2) are discrete energy function appeared in Eq. 28 is the stationary solution entities, and the chemical reactions at the level of single- to Eq. 36: molecules are stochastic, let pn(t) be the probability of having x + − ss J1 (z)+ J2 (z) n number of X in the reaction system at time t. Then, pn(t) ϕ (x) = − ln dz, (37) J −(z)+ J +(z) follows the CME: Z0 1 2 and d ss pn(t)= un−1pn−1 − un + wn pn + wn+1pn+1, ∂ϕ dt (31a) ∂x (1) (2) (1) (2) (1) − un = u + u , wn = w + w , u = ak+1n, + n n n n n J1 (x)+ J2 (x) (31b) = − ln − J (x)+ J +(x) 1 2 (2) (1) k−1n(n − 1) (2) u = k− ωb, w = , w = k n. ak+1x + k−2b n 2 n ω n +2 = − ln (38) (31c) k− x2 + k x 1 +2 (µ −µ )/k T (µ −µ )/k T k e B X B + k− x e A X B = − ln +2 1 , k + k− x A. Stationary distribution +2 1 in which µ − µ = k T ln ak+1k+2 . We observe that The steady state of such an open chemical reaction system A B B bk−1k−2 has a probability distribution πn that is no longer changing when the reaction system is not driven chemically, µB = µA. − with time, after a certain period of relaxation kinetics. By open Then (38) is µX µA , the chemical potential of X in unit of kB T system, we mean it has a constant chemical flux between A kBT , with state A as reference. and B, with its direction being determined by which of the µA and µB being greater. The stationary probability distribution C. The stochastic NESS entropy production rate for πn is According to stochastic thermodynamics, the NESS entropy n production rate for the stochastic dynamics in (31), in kBT ak (ℓ − 1) + k− ωb +1 2 (32) unit, is given in Eq. 9c: πn = C −1 k−1ω (ℓ − 1)ℓ + k+2ℓ ℓ=1 ∞ (1) Y − µA µB (1) (1) πnun k T ωk+2 e π = π u − π w ln n n (ℓ − 1)e B + p n n n n+1 n+1 (1) C k− ωb k−1 2 n=0 πn+1w ! =   , X n+1 n! k+2 ωk+2 ∞ (2) ℓ=1 (ℓ − 1)+ −  π u Y k 1  (2) (2) n n + πnun − πn+1wn+1 ln (2) πn+1w ! in which C is a normalization constant. We observe that when n=0 n+1   ∞X ss (2) (1) µ = µ , the p is a Poisson distribution, as predicted by un w A B n = π u(2) − π w(2) ln n+1 Gibbs’ equilibrium theory of grand canonical ensemble, with n n n+1 n+1 (2) (1) n=0 wn+1un ! the mean number of X being (k−2ωb/k+2), or equivalently X∞ (2) bk−1k−2 the mean concentration (k−2b/k+2). = π u(2) − π w ln n n n+1 n+1 ak k The chemical thermodynamics presented above does not n=0 +1 +2 X ss reference to anything with probability. But we know that the ak+1k+2 hnX i = ω ln k+2 − k−2b . (39) very notion of Gibbs’ chemical potential has a deep root in it. bk− k− ω 1 2 7

This agrees exactly with Eq. 19, which is the entropy produc- B. Birth-and-death approximation of replication kinetics tion rate per unit volume. For a macroscopic sized system (2) with externally sustained a and b, the equation for its chemical kinetics is V. COARSE GRAINING:KINETICS AND THERMODYNAMICS dx The entropy production given in (19) and (39) can be = k+1a + k−2b x − k−1x + k+2 x. (43) dt decomposed into two terms: Let us now consider a particular case in which ss ss ss σ[x ]= Jnet A→X × µA−µX + J net X→B× µX −µB , k+1a ≫ k−2b and x ≪ k+2/k−1. (44) n o n (40)o ss ss ss 2 in which Jnet A→X = k+1ax − k−1(x ) , µA − µX = Then it is completely legitimate to approximate the kinetic ss ss ss kBT ln k+1a/(k−1x ) , J net X→B = k+2x − k−2b, and equation in (43) by an approximated ss µX − µB = kB T ln k+2x /(k−2b) . Both terms in {···} dx are positive. More importantly, if k+1 and k−1 are very large = g − k x, g ≡ k a. (45) ss +2 +1 while Jnet A→X is kept constant, then µA − µX will be very dt small; they are nearly at equilibrium. In this case, we can In fact, the approximation produces accurate kinetic result if lump the A and X as a single chemical species with rapid the two inequalities in (44) are strong enough. internal equilibrium. Such coarse-graining always leads to If system (2) is mesoscopic sized, then the deterministic under-estimating the entropy production [46], [47]. dynamics in (45) also has a stochastic, Markov counterpart, as a birth-and-death process with master equation for the A. Thermodynamically consistent kinetic approximation probability distribution pn(t) ≡ Pr nX (t) = n , where n (t) is the number of X in the system at time t: We now carry out a more in depth analysis on kinetic coarse- X graining. In particular, we try to show the following: There d p (t) = g (n − 1)p − − np are at least two types of kinetic approximations: those are dt n n 1 n thermodynamically meaningful and those are not. In mathe- − k+2 npn(t) − (n + 1)pn+1(t) . (46) matical terms: No matter how inaccurate, the former gives a ﬁnite approximation of the “true entropy production of the For very large n, one can approximate n +1 ≈ n ≈ n − 1, system” while the latter yields a numerical inﬁnity, e.g., the then this is the equation [9] in ref. [11], if we identify k+2 as thermodynamics is lost. δ. According to T. L. Hill [48], the expression in (18) However, as discussed in Sec. V-A, while coarse-graining is has a unique, thermodynamically meaningful representation a type of mathematical approximation, not all mathematically ss ss ss ss Jnet A→B = JA→B − JB→A, in which JA→B = ak+1k+2/Σ, legitimate kinetic approximations are valid coarse-graining for ss JB→A = bk−1k−2/Σ, and thermodynamics. The approximated (45) now yields meaning-

2 less chemical thermodynamics. To see this, we consider the 1 k k+1a + k+2 − 4λ − k+1a + k+2 exact result in Eq. 16 Σ= + +2 . k−1 2k−1 q λ (41) ak+1k+2 µA − µB = ln , (47) bk− k− One can show that Σ is strictly positive if all the parameters 1 2 having finite values. Then, the entropy production rate in (19) and compare it with the mathematical entropy production in ss the Markov transition of the birth-and-death process (46), from ss ss ss JA→B σ x = J → − J → ln . (42) A B B A ss nX = n to nX = (n + 1): JB→A With respect to the expression in Eq. 42, we can reach the entropy production of birth-and-death following conclusions: ss ∗ ss Pr{a birth in ∆t time} (a) If an approximation leads to JA→B → J+, JB→A → = ln ∗ ∗ ∗ Pr{a death in ∆t time} J− but J+/J− finite, then there is a meaningful, finite approximated σ. This is a sufficient condition but not necessary. ss ss ss ss ank+1pn (b) If both J → , J → → ∞, but (J → − J → ) is = ln . (48) A B B A A B B A k (n + 1)p finite, then there is a finite σ. +2 n+1 ss ss (c) It is possible that (JA→B − JB→A) → ∞, Comparing Eq. 48 and Eq. 47, we see there is no definitive ss ss ss (JA→B /JB→A) → 1, and σ is finite. For example, JA→B = relation, nor definitive inequality between the two quantities: 2 ss 2 x + x , JB→A = x . Then when x → ∞ we have σ =1. Both k+1 and k+2, which are on the numerator and denomi- ss ss (d) However, if one of the JA→B and JB→A is finite and nator in (48), are on the numerator in (47). another one is not, then σ = ∞. One also notices that when k−1 = 0, the Markov process We see that if k−1, k−2 → 0 while all other parameters are described by Eq. 31 is possible to continously increase without finite, then it is the scenario (d). In this case, a meaningful reaching stationarity: thermodynamics no longer exists for the kinetic approximation. ak+1n + k−2bω > k+2n, ∀n. (49) 8

However, for k−1 6=0, the inequality in (49) can not be valid Also, with a thermodynamically valid coarse-graining, the for all n: There must be an n∗, when n>n∗: computation of entropy production on a coarse level can provide a lower bound for the mechanics based free energy k−1(n − 1) ak+1n + k−2bω < + k+2 n. (50) dissipation, as stated in [11]. This result is not new; it has ω been discussed already in [46] and [47]. In a nutshell, coarse- This shows that a chemical kinetics with meaningful thermo- graining involves “rapid equilibrium” assumption which hides dynamics is intrinsically stable. dissipation in the fast modes of motions. In fact, with a given − As we have stated earlier, considering microscopic re- difference J + − J which fixes the rate of an irreversible − versibility is a fundamental tenet of stochastic thermodynam- process, the entropy production ln J +/J decrease with − ics. Any coarse-graining that is thermodynamically meaning- increasing one-way fluxes J + and J . ful has to respect the nature of microscopic reversibility. On The physics of living systems is outside classical me- the other hand, the death step represented by δ = k+2 simply is chanics. While no living phenomenon and process disobey not the reverse step of the birth step represented by g = k+1a; classical mechanics, the former are phenomena with such a it cannot provide any meaningful estimation. The reversed large degree of freedom, heterogeneity, and complexity, their step of birth is actually an infant going back to the mother’s understanding has to be founded on laws and descriptions womb! In a population dynamics model like (45), while it outside classical mechanics. By classical mechanics, we mean could be completely accurate in kinetic modeling, nevertheless the view of the world in terms of point masses and their is not thermodynamically meaningful on the level of physical movement based on the Newtonian system of equations of chemistry. There are irreversible approximations involved. motion. In particular, we try to show that the understanding As pointed out in [11], “it is much more likely that one of “heat”, which is so fundamental in the irreversibility of bacterium should turn into two than that two should somehow macroscopic mechanical systems, actually has very little to spontaneously revert back into one.” Nevertheless, it is the do with irreversibility in living systems! In contrast, chemical non-zero probability of such completely inconceivable pro- thermodynamics and its irreversibility, a` la Gibbs, Lewis and cesses, in a physicist’s world, that provides an estimation of Randall, naturally arises from the theory of probability, the thermodynamic heat production. When a biologist considers mathematics J. W. Gibbs employed in terms of the notion of only feasible events to a living organism being birth and death, ensemble, for developing his brand of statistical mechanics. It the connection to the physical world has already lost in the did not escape our notice that Gibbs was also the originator of mathematical thinking. the chemical thermodynamics, and the inventor of the notion of chemical potential. Understanding thermodynamics from Newtonian mechanics VI. CONCLUSION AND DISCUSSION was the central thesis of L. Boltzmann’s life work. Indeed, Another 2nd Law for systems with irreversible processes? one of the key results of Boltzmann, together with H. von First, we should emphasize that while classical mechanics Helmholtz, was to cast the phenomenological First Law of represents a system in terms of point masses each with a thermodynamics into a mathematical corollary of the Newto- distinct label, reasonable “variables” in biology, physiology, nian equation of motion: One knows a Hamiltonian dynamics and biochemistry, more often are counting numbers of various is restricted on a level set of H({xi}, {vi}) = E, where “species”:3 number of molecules in biochemistry, number of the E is determined by the initial condition. Recognizing cells in a tissue, and number of individuals in a population. that a thermodynamic state is actually a state of perpetual In these latter cases, the Crooks’ equality, e.g., Eq. (3) in motion, e.g., an entire level set4, there is a mathematical [11] is mathematically hold, but its interpretation as “heat”, function between the phase-space volume contained by the based on Gibbs’ chemical thermodynamics, is subtle [37]: It level-set Ω, E, and other parameters in the Hamiltonian depends on how the environment of an open system is set function H({xi}, {vi},V,N): Ω(E,V,N). Since the phase- up. The discussion in Sec. V serves as a warning. Note when space volume monotonically increases with E, one can define counting the number of individuals in a population, say nX , E = E(S,V,N), where S = ln Ω(E,V,N). (51) the increase of nX from n to n +1, and its decrease from n +1 to n, even they are through reversible reaction steps, it It is then a matter of simple calculus to obtain cannot be identified whether they occur by a same individual ∂E ∂E ∂E or different individuals: This information has already lost in dE = dS + dV + dN, ∂S V,N ∂V S,N ∂N S,V the number counting representation of nameless individuals. (52) However, since the mathematics is valid, it is legitimate, and define the emergent thermodynamic quantities T = even scientifically desirable, to propose a different type of as temperature, as pres- ∂E/∂S V,N p = − ∂E/∂V S,N “heat” in Markov systems that is not necessarily connected sure, and µ = ∂E/∂N as chemical potential. Note, from S,V to mechanical energy via kBT . the mechanical standpoint, T,p,µ are emergent quantities; they characterize how one invariant torus is related to another 3Note that when dealing with many-body systems, fluid mechanics and quantum mechanics changed their representations from tracking the state of invariant torus, i.e., one thermodynamic equilibrium state to individually labeled particles to counting the numbers: the switching from another thermodynamic equilibrium state. Lagrangian to Eulerian descriptions in the former and second quantization in the latter. 4Here, the importance of ergodicity arises. 9

It is important to emphasize that, no matter how imperfect, subsystem due to spontaneous processes that cause entropy of both T and p have well-accepted and widely understood the total closed system to increase. This view is justiﬁed in dF mechanical interpretation, as mean kinetic energy and mean terms of dt ≤ 0 for closed systems. We refer the readers momentum transfer to the wall of a container, respectively. to T. L. Hill’s notion of “cycle completion” in stochastic However, µ has no interpretation in terms of classical mo- thermodynamics [57] for a much more complete view of the tion, whatsoever; rather, it has an interpretation in terms of matter. This concept nicely echoes R. Landauer and C. H. probability, and in terms of Brownian motion: Bennett’s principle of computational irreversibility being associated only with memory erasing [58], [59]. It also provides a ∂ρ(x, t) ∂2ρ(x, t) 1 ∂(Fρˆ ) = D = − , (53) powerful conceptual framework for further investigating other ∂t ∂x2 η ∂x nagging concepts such as heat dissipation associated with a where subsystem in a nonequilibrium steady state (NESS) [37] and ∂µ endo-reversibility in ﬁnite time thermodynamics (FTT) [55], Fˆ = − , and µ = Dη ln ρ(x, t)= k T ln ρ(x, t). (54) [60]. ∂x B

Fˆ is known as entropic force in chemistry, and µ is its potential ACKNOWLEDGMENT function. D. B. S. and H. Q. acknowledge the MOST grant 104-2811- A living system is sustained neither by a temperature M-001-102 and the NIH grant R01-GM109964, respectively, difference, nor by a pressure difference. It is a phenomenon for support. H. Q. thanks Ken Dill, Tolya Kolomeisky, and driven by chemical potential difference. Therefore, we believe Chao Tang for pointing him to the problem of self-replication, any discussion of irreversibility in living systems based on the that initiated the present work. notion of heat is misguided. Nevertheless, as the physicist in th 18 century had recognized in connection to the notion of REFERENCES “heat death of the universe”, a sustained chemical potential [1] H. Qian, “Nonlinear stochastic dynamics of complex systems, I: A difference has to have a consequence in generating heat in chemical reaction kinetic perspective with mesoscopic nonequilibrium a closed, cyclic universe, if one indeed can treat the entire thermodynamics,” arXiv:1605.08070, 2016. universe as an isolated mechanical system. However, it is [2] L. F. Thompson and H. Qian, “Nonlinear stochastic dynamics of complex systems, II: Potential of entropic force in Markov systems with equally likely, according to our current, limited understanding nonequilibrium steady state, generalized Gibbs function and criticality,” of the cosmology and planet formation, that what is being arXiv:1605.08071, 2016. dissipated is simply local inhomogeneity, e.g., low entropy [3] C. E. Shannon and W. Weaver, The Mathematical Theory of Communi- cation. Chicago: Univ. Illinois Press, 1963. initial condition, in our world, which was formed 13 some [4] S. Toyabe, T. Sagawa, M. Ueda, E. Muneyuki and M. Sano, “Experimen- billion years ago. tal demonstration of information-to-energy conversion and validation of Information and entropy. There is a growing interest in the generalized Jarzynski equality,” Nat. Phys. vol. 6 988–992, 2010. [5] D. Mandal and C. Jarzynski, “Work and information processing in a the phenomenon of “information to energy conversion” in solvable model of Maxwells demon,” Proc. Natl Acad. Sci. U.S.A. vol. Maxwell-demon like devices [49]. To quantify information, 109, 11641–11645, 2012. entropy change has to be distinguished from work. An un- [6] H. A. Johnson, “Information theory in biology after 18 years,” Science, vol. 168, pp. 1545–1550, 1970. ambiguous distinction between work and heat is fundamental [7] T. D. Schneider, “Some lessons for molecular biology from information to thermodynamics: “The difference between heat and work theory,” In Entropy Measures, Maximum Entropy Principle and Emerg- cannot always be uniquely specified. It is assumed that there ing Applications, E. Vargas, M. A. Jiménez-Montaño and J. L. Rius, eds., New York: Springer-Verlag, pp. 229–237, 2003. are cases, involving ideal processes, in which the two can be [8] A. N. Kolmogorov, “Three approaches to the quantitative definition of strictly distinguished from one another” [50]. Unfortunately information,” Int. J. Comput. Math. vol. 2, 157–168, 1968. outside classical mechanics, at least in biochemistry where [9] H. Qian, “Mesoscopic nonequilibrium thermodynamics of single macro- molecules and dynamic entropy-energy compensation,” Phys. Rev. E vol. temperature-dependent heat capacity is a commonplace [51], 65, art. 016102, 2002. distinguishing enthalpy change from entropy change becomes [10] R. H. Fowler, Statistical Mechanics: The Theory of the Properties of increasingly challenging [52], [53], [37]. In a more modern Matter in Equilibrium. U.K.: Camberage Univ. Press, 1929. [11] J. L. England, “Statistical physics of self-replication,” J. Chem. Phys., physics setting, heat, probability, and time are intimately vol. 139, art. 121923, 2013. related [54]. The success of our present theory on chemical [12] J. Roughgarden, Theory of Population Genetics and Evolutionary Ecol- thermodynamics owes to a large extent to a sidestepping the ogy: An Introduction. New York: Macmillan Pub., 1979. [13] B. M. R. Stadler and P. F. Stadler, “The topology of evolutionary notion of heat, and thus temperature that follows. biology,” In Modelling in Molecular Biology. G. Ciobanu and G. The physical locus of entropy production. Whether the Rozenberg eds., pp. 267–286, New York: Springer, 2004. entropy is produced inside a subsystem, or in the environment [14] M. W. Kirschner and J. C. Gerhart, The Plausibility of Life: Resolving Darwins Dilemma. New Haven: Yale Univ. Press, 2005. outside the system, or just at the boundary [55], is one [15] D. Noble, The Music of Life: Biology Beyond Genes. U.K.: Oxford Univ. of the deeper issues in nonequilibrium thermodynamics: To Press, 2008. L. Onsager, a transport process driven by a thermodynamic [16] R. A. Weinberg, “Coming full circle — From endless complexity to simplicity and back again,” Cell, vol. 157, pp. 267–271, 2014. force constitutes dissipation [56]. This is a NESS view of [17] L. H. Hartwell, J. J. Hopfield, S. Leibler and A. W. Murray, “From an “insider” of an open subsystem; it can be mathematically molecular to modular cell biology,” Nature, vol. 402, pp. C47–C52, justified in terms of the positive chemical motive force (cmf) 1999. [18] M. Vellela and H. Qian, “A quasistationary analysis of a stochastic introduced in Eq. 12b. To classical physicists, however, the chemical reaction: Keizers paradox,” Bullet. Math. Biol., vol. 69, pp. mechanistic origin of the “thermodynamic force” is outside the 1727–1746, 2007. 10

[19] L. M. Bishop and H. Qian, “Stochastic bistability and bifurcation in a [50] W. Pauli, Pauli Lectures on Physics: Thermodynamics and the Kinetic mesoscopic signaling system with autocatalytic kinase,” Biophys. J. , Theory of Gas, S. Margulies and H. R. Lewis, transl., Cambridge, MA: vol. 98, pp. 1–11, 2010. The MIT Press, 1973. [20] J. Keizer, Statistical Thermodynamics of Nonequilibrium Processes. New [51] T. H. Benzinger, “Thermodynamics, chemical reactions and molecular York: Springer-Verlag, 1987. biology,” Nature, vol. 229, pp. 100–102, 1971. [21] D. Ruelle, “Biology and nonequilibrium: Remarks on a paper by J. L. [52] H. Qian and J. J. Hopfield, “Entropy-enthalpy compensation: Perturba- England,” Eur. Phys. J. Sp. Top., vol. 224, pp. 935–945, 2015. tion and relaxation in thermodynamic systems,” J. Chem. Phys., vol. [22] H. Ge and H. Qian, “The physical origins of entropy production, free 105, pp. 9292–9296, 1996. energy dissipation and their mathematical representations,” Phys. Rev. [53] H. Qian, “Entropy-enthalpy compensation: Conformational fluctuation E, vol. 81, art. 051133, 2010. and induced-fit,” J. Chem. Phys., vol. 109, pp. 10015–10017, 1998. [23] H. Qian, S. Kjelstrup, A. B. Kolomeisky and D. Bedeaux, “Entropy [54] C. Rovelli, Seven Brief Lessons on Physics. S. Carnell and E. Segre production in mesoscopic stochastic thermodynamics: Nonequilibrium transl., New York: Riverhead Books, 2016. kinetic cycles driven by chemical potentials, temperatures, and mechan- [55] D. P. Sekulic, “A fallacious argument in the finite time thermodynamics ical forces,” J. Phys. Cond. Matt., vol. 28, art. 153004, 2016. concept of endoreversibility,” J. Appl. Phys., vol. 90, pp. 4560–4565, [24] H. Ge and H. Qian, “Mesoscopic kinetic basis of macroscopic chemical 2001. thermodynamics: A mathematical theory,” arXiv:1601.03159, 2016. [56] L. Onsager, “Reciprocal relations in irreversible processes. I,” Phys. Rev., [25] H. Qian, “Thermodynamics of the general diffusion process: Equilibrium vol. 37, pp. 405–426, 1931. supercurrent and nonequilibrium driven circulation with dissipation,” [57] T. L. Hill, “Some general principles in free energy transduction,” Proc. Eur. Phys. J. Sp. Top., vol. 224, pp. 781–796, 2015. Natl. Acad. Sci. USA, vol. 80, pp. 2922–2925, 1983. [58] R. Landauer, “Irreversibility and heat generation in the computing [26] C. Jarzynski, “Equalities and inequalities: Irreversibility and the second process,” IBM J. Res., vol. 5, pp. 183–191, 1961. law of thermodynamics at the nanoscale,” Ann. Rev. Cond. Matt. Phys. [59] C. H. Bennett, “Notes on Landauer’s principle, reversible computation vol. 2, 329–351, 2011. and Maxwell’s demon,” Stud. Hist. Phil. Mod. Phys., vol. 34, pp. 501– [27] U. Seifert, “Stochastic thermodynamics, fluctuation theorems, and 510, 2003. molecular machines,” Rep. Prog. Phys. vol. 75, art. 126001, 2012. [60] H. Ouerdane and H. Qian “Debate: Thermodynamics of the general [28] C. Van den Broeck and M. Esposito, “Ensemble and trajectory thermo- diffusion process; Equilibrium supercurrent and nonequilibrium driven dynamics: A brief introduction,” Physica A vol. 418, 6–16, 2015. circulation with dissipation,” Eur. Phys. J. Sp. Top., vol. 224, pp. 797– [29] S. R. de Groot and P. Mazur, Non-Equilibrium Thermodynamics, North- 799, 2015. Holland Pub., Amsterdam, 1962. [61] H. Ge and H. Qian, “Nonequilibrium thermodynamic formalism of [30] E. Johannessen and S. Kjelstrup, “Minimum entropy production rate nonlinear chemical reaction systems with Waage-Guldberg’s law of mass in plug flow reactors: An optimal control problem solved for SO2 action,” Chem. Phys., vol. 472, pp. 241–248, 2016. oxidation,” Energy, vol. 29, pp. 2403–2423, 2004. [31] A. B. Kolomeisky, Motor Proteins and Molecular Motors, New York: CRC Press, 2015. [32] H. Touchette, “The large deviation approach to statistical mechanics,” Phys. Rep., vol. 478, pp. 1–69, 2009. [33] H. Ge and H. Qian, “Mathematical formalism of nonequilibrium thermodynamics for nonlinear chemical reaction systems with general rate law,” arXiv:1604.07115, 2016. David B. Saakian is a lead researcher of A. I. Alikhanyan National Science [34] R. E. O’Malley, Jr., Historical Developments in Singular Perturbations, Laboratory (Yerevan Physics Institute) Foundation and a visiting scholar New York: Springer, 2014. at the Institute of Physics, Academia Sinica. He graduated from Moscow [35] G. N. Lewis, “A new principle of equilibrium,” Proc. Natl. Acad. Sci. Engineering Physics Institute in 1977 and obtained his Ph.D. in physics from U.S.A., vol. 11, pp. 179–183, 1925. Moscow Lebedev Institute of Physics of Russian Academy of Science in [36] P. Childs and J. P. Keener, “Slow manifold reduction of a stochastic 1980. He had worked in the fields of optics and high energy physics before chemical reaction: Exploring Keizer’s paradox,” Discr. Cont. Dyn. Sys. moved to Yerevan Physics Institute and started research in statistical physics B, vol. 17, pp. 1775–1794, 2012. of disordered systems. In 1990s, the group he led developed an optimal coding [37] H. Ge and H. Qian, “Dissipation, generalized free energy, and a self- theory using the Random Energy Model of Derrida. His current research is consistent nonequilibrium thermodynamics of chemically driven open focused on the solutions of evolution models, for which he had developed subsystems,” Phys. Rev. E, vol. 87, art. 062125, 2013. methods for obtaining exact results. These include the dynamics of Crow- Kimura and Eigen models. Very recently he derived exact results for solutions [38] W. E. Moerner, “Single-molecule spectroscopy, imaging, and photo- of chemical master equations (CMEs) by organizing them as a strip of 1- control: Foundations for super-resolution microscopy (Nobel lecture),” dimensional chain of equations. He has served as referees for numerous top Angew. Chem. Int. Ed., vol. 54, pp. 8067–8093, 2015. journals in the field of evolution models and CMEs. [39] P. Erdi´ and G. Lente, Stochastic Chemical Kinetics: Theory and (Mostly) Systems Biological Applications. New York: Springer-Verlag, 2014. [40] M. A. Leontovich, “Basic equations of kinetic gas theory from the viewpoint of the theory of random processes,” J. Exp. Theoret. Phys., vol. 5, pp. 211–231, 1935. [41] M. Delbrück, “Statistical fluctuations in autocatalytic reactions,” J. Chem. Phys. vol. 8, pp. 120–124, 1940. [42] D. T. Gillespie, “Stochastic simulation of chemical kinetics,” Annu. Rev. Hong Qian received his B.A. in astrophysics from Peking University and Phys. Chem. vol. 58, pp. 35–55, 2007. obtained his Ph.D. in molecular biology from Washington University (St. [43] D. A. Beard and H. Qian, Chemical Biophysics: Quantitative Analysis Louis). His research interests turned to theoretical biophysical chemistry and of Cellular Systems, U.K.: Cambridge Univ. Press, 2008. mathematical biology when he was a postdoctoral fellow at the University [44] G. Hu, “Lyapounov function and stationary probability distributions,” of Oregon and at the California Institute of Technology, where he studied Zeit. Phys. B Cond. Matter, vol. 65, pp. 103–106, 1986. protein folding. He was with the Department of Biomathematics at the UCLA [45] A. Shwartz and A. Weiss, Large Deviation for Performance Analysis, School of Medicine between 1994 and 1997, when he worked on the theory New York: Chapman & Hall, 1995. of motor proteins and single-molecule biophysics. He joined the University [46] M. Santillán and H. Qian, “Irreversible thermodynamics in multiscale of Washington (Seattle) in 1997 and is now professor of applied mathematics. stochastic dynamical systems,” Phys. Rev. E, vol. 83, art. 041130, 2011. He is a fellow of the American Physical Society, and served and serves on the editorial board of journals on computational and systems biology. His current [47] M. Esposito, “Stochastic thermodynamics under coarse graining,” Phys. research interest is in stochastic analysis and statistical physics of cellular Rev. E, vol. 85, art. 041125, 2012. systems. He has coauthored a book “Chemical Biophysics: Quantitative [48] T. L. Hill, Free Energy Transduction and Biochemical Cycle Kinetics. Analysis of Cellular Systems” (Cambridge University Press, 2008) with Daniel New York: Dover, 2004. A. Beard. [49] S. Toyabe, T. Sagawa, M. Ueda, E. Muneyuki and M. Sano, “Demonic device converts information to energy,” Nat. Phys., vol. 6, pp. 988–992, 2010.