PHYSICAL REVIEW X 6, 041064 (2016)

Nonequilibrium Thermodynamics of Chemical Reaction Networks: Wisdom from Stochastic Thermodynamics

Riccardo Rao and Massimiliano Esposito Complex Systems and Statistical Mechanics, Physics and Materials Science Research Unit, University of Luxembourg, L-1511 Luxembourg, Luxembourg (Received 23 February 2016; revised manuscript received 26 October 2016; published 22 December 2016) We build a rigorous nonequilibrium thermodynamic description for open chemical reaction networks of elementary reactions. Their dynamics is described by deterministic rate equations with mass action kinetics. Our most general framework considers open networks driven by time-dependent chemostats. The energy and entropy balances are established and a nonequilibrium Gibbs free energy is introduced. The difference between this latter and its equilibrium form represents the minimal work done by the chemostats to bring the network to its nonequilibrium state. It is minimized in nondriven detailed-balanced networks (i.e., networks that relax to equilibrium states) and has an interesting information-theoretic interpretation. We further show that the entropy production of complex-balanced networks (i.e., networks that relax to special kinds of nonequilibrium steady states) splits into two non-negative contributions: one characterizing the dissipation of the nonequilibrium steady state and the other the transients due to relaxation and driving. Our theory lays the path to study time-dependent energy and information transduction in biochemical networks.

DOI: 10.1103/PhysRevX.6.041064 Subject Areas: Chemical Physics, Nonlinear Dynamics, Statistical Physics

I. INTRODUCTION chemical systems [6–11] using a stochastic description based on the (chemical) master equation [12,13]. These Thermodynamics of chemical reactions has a long studies played an important role during the first decade of history. The second half of the 19th century witnessed the 21st century for the development of stochastic thermo- the dawn of the modern studies on thermodynamics of dynamics, a theory that systematically establishes a non- chemical mixtures. It is indeed at that time that Gibbs equilibrium thermodynamic description for systems introduced the concept of chemical potential and used it to obeying stochastic dynamics [14–17], including chemical define the thermodynamic potentials of noninteracting reaction networks (CRNs) [18–22]. mixtures [1]. Several decades later, this enabled de Another significant part of the attention moved to the Donder to approach the study of chemical reacting mixtures thermodynamic description of biochemical reactions in from a thermodynamic standpoint. He proposed the terms of deterministic rate equations [23,24]. This is not concept of affinity to characterize the chemical force so surprising since living systems are the paramount irreversibly driving chemical reactions and related it to example of nonequilibrium processes and they are powered the thermodynamic properties of mixtures established by by chemical reactions. The fact that metabolic processes Gibbs [2]. Prigogine, who perpetuated the Brussels School involve thousands of coupled reactions also emphasized founded by de Donder, introduced the assumption of local the importance of a network description [25–27]. While equilibrium to describe irreversible processes in terms of complex dynamical behaviors such as oscillations were equilibrium quantities [3,4]. In doing so, he pioneered the analyzed in small CRNs [28,29], most studies on large connections between thermodynamics and kinetics of biochemical networks focused on the steady-state dynamics. chemical reacting mixtures [5]. Very few studies considered the thermodynamic properties During the second half of the 20th century, part of the of CRNs [30–33]. One of the first nonequilibrium thermo- attention moved to systems with small particle numbers dynamic descriptions of open biochemical networks was “ ” which are ill described by deterministic rate equations. proposed in Ref. [34]. However, it did not take advantage of The Brussels School, as well as other groups, produced chemical reaction network theory, which connects the net- various studies on the nonequilibrium thermodynamics of work topology to its dynamical behavior and which was extensively studied by mathematicians during the 1970s [35–37] (this theory was also later extended to stochastic dynamics [38–41]). As far as we know, the first and single Published by the American Physical Society under the terms of the Creative Commons Attribution 3.0 License. Further distri- study that related the nonequilibrium thermodynamics of bution of this work must maintain attribution to the author(s) and CRNs to their topology is Ref. [22], still restricting itself to the published article’s title, journal citation, and DOI. steady states.

2160-3308=16=6(4)=041064(23) 041064-1 Published by the American Physical Society RICCARDO RAO and MASSIMILIANO ESPOSITO PHYS. REV. X 6, 041064 (2016)

In this paper, we consider the most general setting for the hidden), and all reactions must be reversible, i.e., each study of CRNs, namely, open networks driven by chemo- forward reaction þρ has a corresponding backward statted concentrations that may change over time. To the reaction −ρ. Each pair of forward-backward reactions best of our knowledge, this was never considered before. In is a reaction pathway denoted by ρ ∈ R. The orientation this way, steady-state properties as well as transient ones of the set of reaction pathways R is arbitrary. Hence, a are captured. Hence, in the same way that stochastic generic CRN is represented as thermodynamics is built on top of stochastic dynamics, X kþρ X we systematically build a nonequilibrium thermodynamic σ σ ∇ ρXσ ⇌ ∇−ρXσ: 1 þ −ρ ð Þ description of CRNs on top of deterministic chemical rate σ k σ equations. In doing so, we establish the energy and entropy balance and introduce the nonequilibrium entropy of the The constants kþρ (k−ρ) are the rate constants of the CRN as well as its nonequilibrium Gibbs free energy. We forward (backward) reactions. The stoichiometric coef- σ σ show the latter to bear an information-theoretical interpre- ficients −∇ ρ and ∇−ρ identify the number of molecules – þ tation similar to that of stochastic thermodynamics [42 45] of Xσ involved in each forward reaction þρ (the and to be related to the dynamical potentials derived by stoichiometric coefficients of the backward reactions have mathematicians. We also show the relation between the opposite signs). Once stacked into two non-negative minimal chemical work necessary to manipulate the CRNs ∇ ∇σ ∇ ∇σ matrices, þ ¼f þρg and − ¼f −ρg, they define far from equilibrium and the nonequilibrium Gibbs free the integer-valued stoichiometric matrix energy. Our theory embeds both the Prigoginian approach to thermodynamics of irreversible processes [5] and the ∇≡∇− − ∇ : ð2Þ thermodynamics of biochemical reactions [23]. Making full þ use of the mathematical chemical reaction network theory, The reason for the choice of the symbol “∇” will become we further analyze the thermodynamic behavior of two clear later. important classes of CRNs: detailed-balanced networks Example 1.—The stoichiometric matrix of the CRN and complex-balanced networks. In the absence of time- depicted in Fig. 1 is dependent driving, the former converges to thermodynamic 0 1 equilibrium by minimizing their nonequilibrium Gibbs free −10 energy. In contrast, the latter converges to a specific class of B C B 20C nonequilibrium steady states and always allows for an B C B C adiabatic–nonadiabatic separation of their entropy produc- ∇ ¼ B 1 −1 C: ð3Þ B C tion, which is analogous to that found in stochastic @ 0 −1 A thermodynamics [46–50]. Recently, a result similar to the latter was independently found in Ref. [51]. 01 □ A. Outline and notation The paper is organized as follows. After introducing the necessary concepts in chemical kinetics and chemical reaction network theory, Sec. II, the nonequilibrium thermodynamic description is established in Sec. III. As in stochastic thermodynamics, we build it on top of the dynamics and formulate the entropy and energy balance, Secs. III D and III E. Chemical work and non- equilibrium Gibbs free energy are also defined, and the information-theoretic content of the latter is discussed. The special properties of detailed-balanced and of com- plex-balanced networks are considered in Secs. V and IV, respectively. Conclusions and perspectives are drawn in Sec. VI, while some technical derivations are detailed in the appendixes. FIG. 1. Representation of a closed CRN. The chemical species X ; …;X We now proceed by fixing the notation. We consider a are f a eg. The two reaction pathways are labeled by 1 and X −∇a −1 system composed of reacting chemical species σ, each 2. The nonzero stoichiometric coefficients are þ1 ¼ , σ ∈ S S b c of which is identified by an index , where is the ∇−1 ¼ 2, and ∇−1 ¼ 1 for the first forward reaction and −∇c −1 −∇d −1 ∇e 1 set of all indices or species. The species populations þ2 ¼ , þ2 ¼ , and −2 ¼ for the second one. change due to elementary reactions, i.e., all reacting Since the network is closed, no chemical species is exchanged species and reactions must be resolved (none can be with the environment.

041064-2 NONEQUILIBRIUM THERMODYNAMICS OF CHEMICAL … PHYS. REV. X 6, 041064 (2016)

Physical quantities associated with species and reactions Example 2.—For the CRN in Fig. 1 the currents are are represented in upper-lower indices vectorial notation. Upper and lower indexed quantities have the same physical J1 ¼ kþ1Za − k−1ðZbÞ2Zc; i values, e.g., Z Zi, ∀i. We use the Einstein summation 2 2 −2 ¼ J ¼ kþ ZcZd − k Ze: ð6Þ notation: repeated upper-lower indices implies the summa- — tion over all the allowed values for those indices e.g., □ σ ∈ S for species and ρ ∈ R for reactions. Given two arbitrary vectorial quantities a ¼faig and b ¼fbig, the following notation is used: A. Closed CRNs Y A closed CRN does not exchange any chemical species ib ib a i ≡ a i : with the environment. Hence, the species concentrations i vary solely due to chemical reactions and satisfy the rate i equations Finally, given the matrix C, whose elements are fCjg, the j CT C σ σ ρ elements of the transposed matrix are f i g. dtZ ¼ ∇ρJ ; ∀σ ∈ S: ð7Þ The time derivative of a physical quantity A is denoted by dtA, its steady state value by an overbar A, and its Since rate equations are nonlinear, complex dynamical A Aeq A_ equilibrium value by eq or . We reserve the overdot behaviors may emerge [29]. The fact that the rate equations to denote the rate of change of quantities that are not exact [Eq. (7)] can be thought of as a continuity equation for the time derivatives. concentration, where the stoichiometric matrix ∇ [Eq. (2)] acts as a discrete differential operator, explains the choice of the symbol “∇” for the stoichiometric matrix [56]. II. DYNAMICS OF CRNS

In this section, we formulate the mathematical descrip- B. Driven CRNs tion of CRNs [52,53] in a suitable way for a thermo- dynamic analysis. We introduce closed and open CRNs In open CRNs, matter is exchanged with the environ- and show how to drive these latter in a time-dependent ment via reservoirs that control the concentrations of some way. We then define conservation laws and cycles and specific species, Fig. 2. These externally controlled species review the dynamical properties of two important classes are said to be chemostatted, while the reservoirs controlling of CRNs: detailed-balanced networks and complex- them are called chemostats. The chemostatting procedure balanced networks. may mimic various types of controls by the environment. We consider a chemical system in which the reacting For instance, a direct control could be implemented via species fXσg are part of a homogeneous and ideal dilute external reactions (not belonging to the CRN) or via solution: the reactions proceed slowly compared to abundant species whose concentrations are negligibly diffusion and the solvent is much more abundant than affected by the CRN reactions within relevant time scales. the reacting species. Temperature T and pressure p are An indirect control may be achieved via semipermeable kept constant. Since the volume of the solution V is membranes or by controlled injection of chemicals in overwhelmingly dominated by the solvent, it is assumed continuous stirred-tank reactors. constant. The species abundances are large enough so that the molecule’s discreteness can be neglected. Thus, at any time t, the system state is well described by the molar concentration distribution fZσ ≡ Nσ=Vg, where Nσ is the molarity of the species Xσ. The reaction kinetics is controlled by the reaction rate functions JρðfZσgÞ, which measure the rate of occurrence of reactions and satisfy the mass action kinetics [52,54,55]:

ρ Jρ ≡ JρðfZσgÞ ¼ kρZσ∇σ : ð4Þ

The net concentration current along a reaction pathway ρ is thus given by FIG. 2. Representation of an open CRN. With respect to the CRN in Fig. 1, the species Xa and Xe are chemostatted, hence, Y Y þρ −ρ represented as a and e. The green boxes on the sides represent Jρ ≡ Jþρ − J−ρ kþρZσ∇σ − k−ρZσ∇σ : ¼ ð5Þ the reservoirs of chemostatted species.

041064-3 RICCARDO RAO and MASSIMILIANO ESPOSITO PHYS. REV. X 6, 041064 (2016)

σ Among the chemical species, the chemostatted ones are Conservation laws identify conserved quantities L ≡ lσZ , σ ∈ S denoted by the indices y y, and the internal ones by called components [23,25], which satisfy σ ∈ S S ≡ S ∪ S x x ( x y). Also, the part of the stoichiometric σ matrix related to the internal (chemostatted) species is dtL ¼ lσdtZ ¼ 0: ð13Þ σ σy denoted by ∇X ¼f∇ρx g (∇Y ¼f∇ρ g). — Example 3. When chemostatting the CRN in Fig. 1 as We denote a set of independent conservation laws of the in Fig. 2 the stoichiometric matrix Eq. (3) splits into closed network by flλg and the corresponding components λ λ σ 0 1 by L ≡ lσZ . The choice of this set is not unique, and 20 f g B C −10 different choices have different physical meanings. This set ∇X ¼ @ 1 −1 A; ∇Y ¼ : ð8Þ 01 is never empty since the total mass is always conserved. 0 −1 Physically, conservation laws are often related to parts of molecules, called moieties [58], which are exchanged □ between different species and/or subject to isomerization In nondriven open CRNs, the chemostatted species (see Example 4). σ σ have constant concentrations, i.e., fdtZ y ¼ 0g. In driven In an open CRN, since only fZ x g are dynamical open CRNs, the chemostatted concentrations change over variables, the conservation laws become the left null time according to some time-dependent protocol πðtÞ: eigenvectors of the stoichiometric matrix of the internal σ σ fZ y ≡ Z y (πðtÞ)g. The changes of the internal species species ∇X. Stated differently, when starting from the are solely due to reactions and satisfy the rate equations closed CRN, the chemostatting procedure may break a subset of the conservation laws of the closed network flλg σ σ ρ d Z x ∇ x J ; ∀σ ∈ S : t ¼ ρ x x ð9Þ [56]. For example, when the first chemostat is introduced the total mass conservation law is always broken. Within σ Instead, the changes of chemostatted species fdtZ y g are lλ λ the set f g, we label the broken ones by b and the σy ρ not only given by the species formation rates f∇ρ J g but unbroken ones by λ . The broken conservation laws are σ u must in addition contain the external currents fI y g, characterized by which quantify the rate at which chemostatted species σ enter into the CRN (negative if chemostatted species λ σ λ y lσb ∇ρx þ lσb ∇ρ ¼ 0; ∀ρ ∈ R; ð14Þ leave the CRN), |fflfflffl{zfflfflffl}x y ≠0 σ σ ρ σ d Z y ∇ y J I y ; ∀σ ∈ S : t ¼ ρ þ y y ð10Þ where the first term is nonvanishing for at least one ρ ∈ R. λ λ σ This latter equation is not a differential equation since the The broken components fL b ≡ lσb Z g are no longer σ chemostatted concentrations fZ y g are not dynamical constant over time. On the other hand, the unbroken variables. It shows that the external control of the conservation laws are characterized by chemostatted concentration is not necessarily direct, via σ the chemostatted concentrations, but can also be indi- λu σx λu y lσ ∇ρ þ lσy ∇ρ ¼ 0; ∀ρ ∈ R; ð15Þ rectly controlled via the external currents. We note that |fflfflffl{zfflfflffl}x Eq. (10) is the dynamical expression of the decomposi- ¼0 tion of changes of species populations in internal-external ρ ∈ R introduced by de Donder (see Secs. 4.1 and 15.2 where the first term vanishes for all . Therefore, the λ λ σ of Ref. [57]). unbroken components fL u ≡ lσu Z g remain constant over σ λ A steady-state distribution fZ x g, if it exists, must satisfy time. Without loss of generality, we choose the set fl g such that the entries related to the chemostatted species σ ∇ x Jρ 0; ∀σ ∈ S ; λ ρ ¼ x x ð11aÞ l u 0 ∀ λ σ vanish, σy ¼ , u, y. — σ ρ σ Example 4. For the CRN in Fig. 1, an independent set ∇ y J I y 0; ∀σ ∈ S ; ρ þ ¼ y y ð11bÞ of conservation laws is

σ for given chemostatted concentrations fZ y g. l1 ¼ð21000Þ; 2 C. Conservation laws l ¼ð00011Þ; l l 3 1 In a closed CRN, a conservation law ¼f σg is a left l ¼ð0 2 −110Þ: ð16Þ null eigenvector of the stoichiometric matrix ∇ [23,25]: When chemostatting as in Fig. 2, the first two conservation l ∇σ 0; ∀ρ ∈ R: σ ρ ¼ ð12Þ laws break while the last one remains unbroken. We also

041064-4 NONEQUILIBRIUM THERMODYNAMICS OF CHEMICAL … PHYS. REV. X 6, 041064 (2016)

the rate constants must satisfy and call it a local detailed- balance property. It is a universal property of elementary reactions that holds regardless of the network state. Indeed, while the equilibrium distribution depends on the compo- nents, the rhs of Eq. (18) does not. This point will become explicit after introducing the thermodynamic structure, Eq. (88) in Sec. V. The local detailed-balance property will be rewritten in a thermodynamic form in Sec. III B, Eq. (50). In open nondriven CRNs, the chemostatting procedure may prevent the system from reaching an equilibrium state. To express this scenario algebraically, we now introduce FIG. 3. Specific implementation of the CRN in Fig. 2. the concepts of emergent cycle and cycle affinity. A cycle c~ ¼fc~ρg is a right null eigenvector of the note that this set is chosen so that the unbroken conserva- stoichiometric matrix [56], namely, tion law satisfies l3 ¼ l3 ¼ 0. When considering the a e ∇σc~ρ 0; ∀σ ∈ S: specific implementation in Fig. 3 of the CRN in Fig. 2, ρ ¼ ð19Þ we see that the first two conservation laws in Eq. (16) represent the conservation of the concentrations of the Since ∇ is integer valued, c~ can always be rescaled to moiety H and C, respectively. Instead, the third conserva- only contain integer coefficients. In this representation, tion law in Eq. (16) does not have a straightforward its entries denote the number of times each reaction interpretation. It is related to the fact that when the species occurs (negative signs identify reactions occurring in backward direction) along a transformation that overall H or C are produced, also O must be produced and vice σ versa. □ leaves the concentration distributions fZ g unchanged; see Example 5. We denote by fc~ αg a set of linearly independent cycles. An emergent cycle c ¼fcρg is defined algebrai- D. Detailed-balanced networks cally as [56] A steady state [Eq. (11)] is said to be an equilibrium state σ σ Z ∇ x cρ 0; ∀σ ∈ S ; f eqg if it satisfies the detailed-balance property [[57], ρ ¼ x x § 9.4], i.e., all concentration currents Eq. (5) vanish: σ ∇ y cρ ≠ 0; σ ∈ S : ρ for at least one y y ð20Þ ρ ρ σ Jeq ≡ J ðfZ gÞ ¼ 0; ∀ρ ∈ R: ð17Þ eq In its integer-valued representation, the entries of c denote For open networks, this means that the external currents, the number of times each reaction occurs along a trans- σ y formation that overall leaves the concentrations of the Eq. (11b), must also vanish, fIeq ¼ 0g. By virtue of mass σ internal species fZ x g unchanged while changing the action kinetics, Eq. (4), the detailed-balance property concentrations of the chemostatted species by an amount Eq. (17) can be rewritten as σy ρ ∇ρ c . These latter are, however, immediately restored to σ ρ −∇ y cρ kþ ρ their prior values due to the injection of ρ molecules Zσ ∇σ ; ∀ρ ∈ R: −ρ ¼ eq ð18Þ Xσ k of y performed by the chemostats. Emergent cycles are, thus, pathways transferring chemicals across chemostats A CRN is said to be detailed balanced if, for given while leaving the internal state of the CRN unchanged. We ρ σ kinetics fk g and chemostatting fZ y g, its dynamics denote by fcεg a set of linearly independent emergent exhibits an equilibrium steady state, Eq. (17). For each cycles. λ set of unbroken components fL u g—which are given by When chemostatting an initially closed CRN, for each the initial condition and constrain the space where the species that is chemostatted, either a conservation law dynamics dwells—the equilibrium distribution is globally breaks—as mentioned in Sec. II C—or an independent stable [59]. Equivalently, detailed-balanced networks emergent cycle arises [56]. This follows from the rank always relax to an equilibrium state, which for a given nullity theorem for the stoichiometric matrices ∇ and ∇X, S kinetics and chemostatting is unique and depends on the which ensures that the number of chemostatted species j yj λ unbroken components only; see also Sec. V. equals the number of broken conservation laws j bj plus ε S Closed CRNs must be detailed balanced. This statement the number of independent emergent cycles j j: j yj¼ λ ε can be seen as the zeroth law for CRNs. Consequently, j bjþj j. Importantly, the rise of emergent cycles is a rather than considering Eq. (18) as a property of the topological feature: it depends on the species that are equilibrium distribution, we impose it as a property that chemostatted, but not on the chemostatted concentrations.

041064-5 RICCARDO RAO and MASSIMILIANO ESPOSITO PHYS. REV. X 6, 041064 (2016)

We also note that emergent cycles are modeled as “flux Any cycle c~ α and emergent cycle cε bears a cycle modes” in the context of metabolic networks [60–62]. affinity [56], Example 5.—To illustrate the concepts of cycles and J emergent cycles, we use the following CRN [56]: ~ ρ þρ Aα ¼ c~αRT ln ; ð26Þ J−ρ

J ρ þρ Aε ¼ cεRT ln : ð27Þ ð21Þ J−ρ

From the definition of cycle, Eq. (19), and current, Eq. (5), and the local detailed balance, Eq. (18), it follows that the ~ cycle affinities along the cycles Eq. (19) vanish, fAα ¼ 0g, whose Y1 and Y2 species are chemostatted. The stoichio- metric matrix decomposes as and that the cycle affinities along the emergent cycles depend on only the chemostatted concentrations 0 1 −11−11 k σy ρ þρ −∇ρ B C A cεRT Zσ : ∇X 1 −10 0; ε ¼ ln y ð28Þ ¼ @ A k−ρ 001−1 −10 0 1 Since emergent cycles are pathways connecting different ∇Y ¼ : ð22Þ chemostats, the emergent affinities quantify the chemical 01−10 forces acting along the cycles. This point will become clearer later, when the thermodynamic expressions of the The set of linearly independent cycles, Eq. (19), consists of emergent cycle affinities fAεg is given, Eq. (49). only one cycle, which can be written as A CRN is detailed balanced if and only if all the emergent cycle affinities fAεg vanish. This condition is c~ ¼ð1111ÞT: ð23Þ equivalent to the Wegscheider condition [59]. This happens when the chemostatted concentrations fit an equilibrium As the CRN is chemostatted, one linearly independent distribution. As a special case, unconditionally detailed- emergent cycle Eq. (20) arises: balanced networks are open CRNs with no emergent cycle. Therefore, they are detailed balanced for any choice of the c 11−1 −1 T: chemostatted concentrations. Consequently, even when a ¼ð Þ ð24Þ time-dependent driving acts on such a CRN and prevents it from reaching an equilibrium state, a well-defined equi- We now see that if each reaction occurs a number of times librium state exists at any time: the equilibrium state to given by the entry of the cycle Eq. (23), the CRN goes back which the CRN would relax if the time-dependent driving to the initial state, no matter which one it is. On the other were stopped. hand, when the emergent cycle Eq. (24) is performed, the Example 6.—Any CRN with one chemostatted species state of the internal species does not change, while two S 1 only (j yj¼ ) is unconditionally detailed balanced. molecules of Y1 are annihilated and two of Y2 are created. Indeed, as mentioned in Sec. II C, the first chemostatted However, since the chemostats restore their initial values, species always breaks the mass conservation law jλ j¼1, c Y b the overall result of is to transfer two 1, transformed in and, thus, no emergent cycle arises, jεj¼jS j − jλ j¼0. Y y b 2, from the first to the second chemostat. The open CRN in Fig. 2 is an example of an The closed version of this CRN has two independent unconditionally detailed-balanced network with two che- conservation laws, mostatted species, since the chemostatting breaks two conservation laws; see Example 4. Indeed, a nonequili- l1 ¼ ð 01111Þ; brium steady state would require a continuous injection Y Y l2 11100; of a and ejection of e (or vice versa). But this would ¼ð Þ ð25Þ X necessarily result in a continuous production of b and consumption of X , which is in contradiction with the 1 d the first of which, l , is broken following the chemostatting steady-state assumption. □ of any of the two species Y1 or Y2. The other chemostatted Finally, a tacit assumption in the above discussion is that species, instead, gives rise to the emergent cycle the network involves a finite number of species and S λ ε Eq. (24), so that the relationship j yj¼j bjþj j is reactions, i.e., the CRN is finite dimensional. Infinite- satisfied. □ dimensional CRNs can exhibit long-time behaviors

041064-6 NONEQUILIBRIUM THERMODYNAMICS OF CHEMICAL … PHYS. REV. X 6, 041064 (2016) 0 1 different from equilibrium even in the absence of emergent 0 −10 0 1 B C cycles [63]. 10100 B 100C B B C B C B C Γ ¼ @ 01120A; ∂ ¼ B 0 −10C; E. Complex-balanced networks B C 00001 @ 01−1 A To discuss complex-balanced networks and complex- 001 balanced distributions, we first introduce the notion of complex in open CRNs. ð32Þ A complex is a group of species that combines in a reaction as products or as reactants. Each side of Eq. (1) defines a complex, but different reactions might involve the where the complexes are ordered as in Example 7. same complex. We label complexes by γ ∈ C, where C is the The corresponding reaction hypergraph is set of complexes. Example 7.—Let us consider the following CRN [64]

kþ1 ð33Þ Xa ⇌ Xb; k−1

kþ2 kþ3 Xa þ Xb ⇌ 2Xb ⇌ Xc: ð29Þ k−2 k−3 where only the forward reactions are depicted. □ C X ;X ;X X ; 2X ;X γ ∈ C The set of complexes is ¼f a b a þ b b cg, In an open CRN, we regroup all complexes of the 2X and the complex b is involved in both the second and closed CRN that have the same stoichiometry for the internal third reaction. □ species (i.e., all complexes with the same internal part of the X The notion of complex allows us to decompose the composition matrix Γγ regardless of the chemostatted part ∇ Y stoichiometric matrix as Γγ )insetsdenotedbyCj,forj ¼ 1; 2; …. Complexes of the closed network made solely of chemostatted species in the σ σ γ ∇ρ ¼ Γγ ∂ρ: ð30Þ open CRN are all regrouped in the same complex C0.This allows one to decompose the internal species stoichiometric σ We call Γ ¼fΓγ g the composition matrix [35,37]. Its matrix as σ entries Γγ are the stoichiometric number of species Xσ σ σ j γ x x in the complex . The composition matrix encodes the ∇ρ ¼ Γj ∂ρ: ð34Þ structure of each complex in terms of species; see Example γ 8. The matrix ∂ ¼f∂ρg denotes the incidence matrix of the σ σ Γ x ≡ Γ x ; γ ∈ C CRN, whose entries are given by where f j γ for jg are the entries of the com- positionP matrix corresponding to the internal species, and 8 j γ 1 γ ρ ∂ρ ≡ ∂ρ are the entries of the incidence matrix < if is the product complex of þ f γ∈Cj g γ ∂ρ ¼ −1 if γ is the reactant complex of þ ρ ð31Þ describing the network of regrouped complexes. This : regrouping corresponds to the—equivalent—CRN made 0 : otherwise of only internal species with the effective rate constant ρ σ ∇ρ k Z y σy The incidence matrix encodes the structure of the network f g ruling each reaction. Example 9.—Let us consider the CRN Eq. (29) where the at the level of complexes, i.e., how complexes are con- X X nected by reactions. If we think of complexes as network species a and c are chemostatted. The five complexes of the closed network, see Example 7, are regrouped as nodes, the incidence matrix associates an edge to each C X ;X C X ;X X C 2X reaction pathway and the resulting topological structure is a 0 ¼f a cg, 1 ¼f b b þ ag, and 2 ¼f bg.In reaction graph, see, e.g., Fig. 1 and Eqs. (21) and (29). The terms of these groups of complexes, the composition matrix stoichiometric matrix instead encodes the structure of and incidence matrix are the network at the level of species. If we think of species 0 1 as the network nodes, the stoichiometric matrix does not −10 1 B C define a graph, since reaction connects more than a pair of ΓX ¼ð012Þ; ∂C ¼ @ 1 −10A; ð35Þ species, in general. The structure originating is rather a 01−1 hypergraph [56,65] or, equivalently, a Petri net [66,67]. Example 8.—The composition matrix and the incidence matrix of the CRN in Eq. (29) are which corresponds to the effective representation

041064-7 RICCARDO RAO and MASSIMILIANO ESPOSITO PHYS. REV. X 6, 041064 (2016)

complex balanced. Throughout this paper, we refer to these CRNs as unconditionally complex balanced, as in the seminal work [35]. ð36Þ Example 10.—The open CRN, Eq. (36), has a single steady state Zb for any given set of rate constants and chemostatted concentrations Za and Zc [64], defined by Eq. (11a): □ σ d Zb J1 J2 − 2J3 A steady-state distribution fZ x g (11) is said to be t ¼ þ complex balanced if the net current flowing in each group ¼ kþ1Za − k−1Zb þ kþ2ZaZb − k−2ðZbÞ2 ρ of complexes Cj vanishes, i.e., if the currents fJ g satisfy þ 2k−3Zc − 2kþ3ðZbÞ2 ¼ 0: ð39Þ X ∂j Jρ ≡ ∂γ Jρ 0; ∀j: ρ ρ ¼ ð37Þ If the stronger condition Eq. (37) holds, γ∈Cj 3 1 J − J ¼ 0 ðgroup C0Þ; Complex-balanced steady states are, therefore, a subclass 1 2 of steady states Eq. (11a) that include equilibrium ones, J − J ¼ 0 ðgroup C1Þ; Eq. (17), as a special case: 2 3 J − J ¼ 0 ðgroup C2Þ; ð40Þ σ j Γ x ∂ Jρ : j ρ |{z} ð38Þ which is equivalent to |fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}¼0 iff Detailed-Balanced Steady State kþ1Za − k−1Zb ¼ kþ2ZaZb − k−2ðZbÞ2 |fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}¼0 iff Complex-Balanced Steady State kþ3 Zb 2 − k−3Zc; ¼0 for generic steady states ¼ ð Þ ð41Þ

While for generic steady states only the internal species the steady state is complex balanced. Yet, if the steady-state formation rates vanish, for complex-balanced ones the currents are all independently vanishing, complex formation rates also vanish. kρ S J1 J2 J3 0; For a fixed kinetics (f g) and chemostatting ( y and ¼ ¼ ¼ ð42Þ σ fZ y g), a CRN is complex balanced if its dynamics exhibits a complex-balanced steady state, Eq. (37) [35,36]. The i.e., Eq. (41) is equal to zero, then the steady state is complex-balanced distribution Eq. (37) depends on the detailed balanced. λ unbroken components fL u g, which can be inferred from When, for simplicity, all rate constants are taken as 1, the the initial conditions, and is always globally stable [68]. complex-balanced set of quadratic equations, Eq. (41), admits a positive solution Zb only if Za 2 − Zc Hence, complex-balanced networks always relax to a— pffiffiffiffiffi ¼ complex-balanced—steady state. Detailed-balanced net- (0

041064-8 NONEQUILIBRIUM THERMODYNAMICS OF CHEMICAL … PHYS. REV. X 6, 041064 (2016)

Z Pconcentration of the solution is denoted by tot ¼ σ 0 0 σZ þ Z , where Z is the concentration of the solvent. ð44Þ We assume for simplicity that the solvent does not react with the solutes. In case it does, our results still hold provided one treats the solvent as a nondriven chemostatted species, as discussed in Appendix A. Since the solvent is much more abundant than the solutes, the total concen- This network is deficiency zero and, hence, unconditionally tration is almost equal to that of the solvent which is a Z ≃ Z complex balanced [22]. Therefore, given any set of rate constant, tot 0. Without loss of generality, the constant 1 2 3 −RT Z ≃ −RT Z0 constants k , k , and k , and the chemostatted con- term ln tot ln in Eq. (45) is absorbed in the centrations Za and Ze, the steady state of this CRN is standard-state chemical potentials. Consequently, many complex balanced, i.e., the steady state always satisfies a equations appear with nonmatching dimensions. We also set of condition like those in Eq. (40). Indeed, contrary to emphasize that standard-state quantities, denoted with “∘”, Example 10, steady-state currents fJ1; J2; J3g different are defined as those measured in ideal conditions, at ∘ from each other cannot exist since they would induce a standard pressure (p ¼ 100 kPa) and molar concentration ∘ 3 growth or decrease of some concentrations. □ (Zσ ¼ 1 mol=dm ), but not at a standard temperature (Ref. [71], p. 61). III. THERMODYNAMICS OF Because of the assumption of local equilibrium and CHEMICAL NETWORKS homogeneous reaction mixture, the densities of all extensive thermodynamic quantities are well defined Using local equilibrium, here we build the connection and equal everywhere in the system. With a slight abuse between the dynamics and the nonequilibrium thermody- of notation, we use the same symbol and name for namics for arbitrary CRNs. In the spirit of stochastic densities as for their corresponding extensive quantity. thermodynamics, we derive an energy and entropy balance, For example, S is the molar entropy divided by the and express the dissipation of the CRN as the difference volume of the solution, but we denote it as entropy.We between the chemical work done by the reservoirs on the apply the same logic to rates of change. For example, we CRN and its change in nonequilibrium free energy. We, call entropy production rate the molar entropy production finally, discuss the information-theoretical content of the density rate. nonequilibrium free energy and its relation to the dynami- cal potentials used in chemical reaction network theory. B. Affinities, emergent affinities, and local detailed balance A. Local equilibrium The thermodynamic forces driving reactions are given Since we consider homogeneous reaction mixtures in by differences of chemical potential [Eq. (45)], ideal dilute solutions, the assumption of local equilibrium (Ref. [57], Sec. 15.1, and Ref. [70]) means that the σ Δ Gρ ≡∇ρμσ; ð46Þ equilibration following any reaction event is much faster r than any reaction time scale. Thus, what is assumed is that also called Gibbs free energies of reaction (Ref. [23], the nonequilibrium nature of the thermodynamic descrip- Sec. III. 2, and Ref. [57], Sec. IX.3). Since these must all tion is solely due to the reaction mechanisms. If all σ eq vanish at equilibrium, ∇ρμσ ¼ 0, ∀ρ,wehave reactions could be instantaneously shut down, the state of the whole CRN would immediately become an equili- Z Δ G −RT∇σ σ : brated ideal mixture of species. As a result, all the intensive r ρ ¼ ρ ln Zeq ð47Þ thermodynamic variables are well defined and equal every- σ T where in the system. The temperature is set by the The local detailed balance, Eq. (18)] allows us to express p solvent, which acts as a thermal bath, while the pressure these thermodynamic forces in terms of reaction affinities, is set by the environment the solution is exposed to. As a result, each chemical species is characterized by a chemical J A ≡ RT þρ −Δ G ; potential (Ref. [23], Sec. III. 1), ρ ln ¼ r ρ ð48Þ J−ρ Z μ μ∘ RT σ ; ∀σ ∈ S; σ ¼ σ þ ln Z ð45Þ which quantify the kinetic force acting along each reaction tot pathway (Ref. [57], Sec. IV.1.3). ∘ ∘ The change of Gibbs free energy along emergent cycles, where R denotes the gas constant and fμσ ≡ μσðTÞg are the standard-state chemical potentials, which depend on the ρ ρ σy Aε ¼ −cεΔ Gρ ¼ −cε ∇ρ μσ ; ð49Þ temperature and on the nature of the solvent. The total r y

041064-9 RICCARDO RAO and MASSIMILIANO ESPOSITO PHYS. REV. X 6, 041064 (2016)

Δ S ∇σs ; gives the external thermodynamic forces the network is r ρ ¼ ρ σ ð54Þ coupled to, as we see in Eq. (61), and thus provides a thermodynamic meaning to the cycle affinities Eq. (28). called entropies of reaction [[23] § 3.2]. Combining the detailed-balance property Eq. (18) eq and the equilibrium condition on the affinities Aρ ¼ 0 D. Entropy balance [Eq. (46)], we can relate the Gibbs free energies of reaction to the rate constants 1. Entropy production rate   The entropy production rate is a non-negative measure of kþρ Δ G∘ r ρ the break of detailed balance in each chemical reaction. Its −ρ ¼ exp − ; ð50Þ k RT typical form is given by (Ref. [8] and Ref. [57], Sec. IX.5)

∘ σ ∘ where ΔrGρ ≡∇ρμσ. This relation is the thermodynamic J TS_ ≡ RT J − J þρ ≥ 0; i ð þρ −ρÞ ln ð55Þ counterpart of the local detailed balance Eq. (18). It plays J−ρ the same role as in stochastic thermodynamics, namely, connecting the thermodynamic description to the stochastic because (1) it is non-negative and vanishes only at dynamics. We emphasize that the local detailed-balance equilibrium, i.e., when the detailed-balance property property as well as the local equilibrium assumption by Eq. (17) is satisfied, and (2) it vanishes to first order no mean imply that the CRN operates close to equili- around equilibrium, thus allowing for quasistatic reversible brium. Their importance is to assign well-defined equilib- transformations. Indeed, defining rium potentials to the states of the CRN, which are then connected by the nonequilibrium mechanisms, i.e., Zσ − Zσ eq ϵσ; ϵσ ≪ 1; ∀σ ∈ S; reactions. Zσ ¼ j j ð56Þ eq C. Enthalpies and entropies of reaction we find that To identify the heat produced by the CRN, we need to distinguish the enthalpic change produced by each reaction S_ Eσ ϵσ0 ϵ ϵ3 ; i ¼ σ0 σ þ Oð Þ ð57Þ from the entropic one. We consider the decomposition of the standard-state chemical potentials (Ref. [23], E ≡ Eσ where f σ0 g is a positive semidefinite symmetric Sec. III. 2): matrix. μ∘ h∘ − Ts∘ : Furthermore, it can be rewritten in a thermodynamically σ ¼ σ σ ð51Þ appealing way using [Eq. (48)] h∘ The standard enthalpies of formation f σg take into TS_ −JρΔ G : account the enthalpic contributions carried by each species i ¼ r ρ ð58Þ (Ref. [23], Sec. III. 2, and Ref. [72], Sec. X.4.2). Enthalpy changes caused by reactions give the enthalpies of reaction It can be further expressed as the sum of two distinct (Ref. [23], Sec. III. 2, and Ref. [57], Sec. II. 4), contributions [56]: _ σ σ σ Δ H ∇σh∘ ; TS ¼ −μσ dtZ x −μσ ðdtZ y − I y Þ : ð59Þ r ρ ¼ ρ σ ð52Þ i |fflfflfflfflfflffl{zfflfflfflfflfflffl}x |fflfflfflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflfflfflffl}y ≡TS_ ≡TS_ which at constant pressure measure the heat of reaction. x y This is the content of the Hess law (see, e.g., Ref. [72], ∘ The first term is due to changes in the internal species and Sec. X.4.1). The standard entropies of formation fsσg take thus vanishes at steady state. The second term is due to the into account the internal entropic contribution carried by each species under standard-state conditions (Ref. [23], chemostats. It takes into account both the exchange of Sec. III. 2). Using Eq. (51), the chemical potentials Eq. (45) chemostatted species and the time-dependent driving of their concentration. If the system reaches a nonequilibrium can be rewritten as σ steady state, the external currents fI y g do not vanish and μ h∘ − T s∘ − R Z : the entropy production reads σ ¼ σ |fflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflffl}ð σ ln σÞ ð53Þ ≡sσ σ TS_ I y μ : i ¼ σy ð60Þ ∘ The entropies of formation fsσ ≡ sσ − R ln Zσg account for the entropic contribution of each species in the CRN This expression can be rewritten as a bilinear form of (Ref. [23], Sec. III. 2). Entropy changes along reactions emergent cycle affinities fAεg Eq. (49) and currents along ε ε ρ are given by the emergent cycle fJ ≡ cρJ g [56]

041064-10 NONEQUILIBRIUM THERMODYNAMICS OF CHEMICAL … PHYS. REV. X 6, 041064 (2016)

TS_ J εA ; the entropy changes in the system plus those in the i ¼ ε ð61Þ reservoirs (thermal and chemostats) [57]. which clearly emphasizes the crucial role of emergent cycles in steady-state dissipation. E. Energy balance 1. First law of thermodynamics 2. Entropy flow rate Since the CRN is kept at constant pressure p, its enthalpy The entropy flow rate measures the reversible entropy σ ∘ changes in the environment due to exchange processes with H ¼ Z hσ þ H0 ð66Þ the system [57]. Using the expressions for the enthalpy of reaction Eq. (52) and entropy of formation Eq. (53),we is equal to the CRN internal energy, up to a constant. express the entropy flow rate as Indeed, the enthalpy H is a density which, when written in terms of the internal energy (density) U, reads H ¼ U þ p. _ ρ σ TS ≡ J Δ Hρ þ I y Tsσ : ð62Þ Using the rate equations (9) and (10), the enthalpy rate of e |fflfflfflffl{zfflfflfflffl}r y change can be expressed as the sum of the heat flow rate, _ ≡Q defined in Eq. (62), and the enthalpy of formation exchange rate: The first contribution is the heat flow rate (positive if heat is absorbed by the system). When divided by temperature, it ∘ σ _ σ ∘ dtH ¼ hσdtZ ¼ Q þ I y hσ : ð67Þ measures minus the entropy changes in the thermal bath. y The second contribution accounts for minus the entropy Equivalently, it can be rewritten in terms of the entropy change in the chemostats. flow rate Eq. (62) as (Ref. [57], Sec. IV.1.2)

3. System entropy σ d H TS_ I y μ : t ¼ e þ σ ð68Þ The entropy of the ideal dilute solution constituting the y CRN is given by (see Appendix A) The last term on the rhs of Eq. (68) is the free energy exchanged with the chemostats. It represents the chemical σ S S ¼ Z sσ þ RZ þ S0: ð63Þ work rate performed by the chemostats on the CRN [21,23]: The total concentration term, σ W_ ≡ I y μ : X c σy ð69Þ ZS ≡ Zσ; ð64Þ σ∈S Either Eq. (67) or (68) may be considered as the non- equilibrium formulation of the first law of thermodynamics S and the constant 0 together represent the entropic con- for CRNs. The former has the advantage to solely focus on tribution of the solvent. S0 may also account for the entropy energy exchanges. The latter contains entropic contribu- of chemical species not involved in the reactions. We also tions but is appealing because it involves the chemical prove in Appedix B that the entropy [Eq. (63)] can be work Eq. (69). obtained as a large particle limit of the stochastic entropy of CRNs. 2. Nonequilibrium Gibbs free energy S would be an equilibrium entropy if the reactions could all be shut down. But in the presence of reactions, it We are now in the position to introduce the thermody- becomes the nonequilibrium entropy of the CRN. Indeed, namic potential regulating CRNs. The Gibbs free energy of Using eqs. (53), (58), and (62), we find that its change can ideal dilute solutions reads be expressed as σ S G ≡ H − TS ¼ Z μσ − RTZ þ G0: ð70Þ σ σ S dtS ¼ sσdtZ þ Z dtsσ þ RdtZ As for entropy, the total concentration term −RTZS and the σ ¼ sσdtZ constant G0 represent the contribution of the solvent (see ρ σ Appendix A). Furthermore, in the presence of reactions, G ¼ J Δ Sρ þ I y sσ r y becomes the nonequilibrium Gibbs free energy of CRNs. S_ S_ : ¼ i þ e ð65Þ We now show that the nonequilibrium Gibbs free energy of a closed CRN is always greater than or equal This relation is the nonequilibrium formulation of the to its corresponding equilibrium form. A generic non- second law of thermodynamics for CRNs. It demonstrates equilibrium concentration distribution fZσg is character- Lλ lλ Zσ Zσ that the non-negative entropy production Eq. (55) measures ized by the set of components f ¼ σ g. Let f eqg be

041064-11 RICCARDO RAO and MASSIMILIANO ESPOSITO PHYS. REV. X 6, 041064 (2016)

σ σ S the corresponding equilibrium distribution defined by the dtG ¼ μσdtZ þ Z dtμσ þ RdtZ detailed-balance property Eq. (18) and characterized by the σ ¼ μσdtZ : ð76Þ same set of components fLλg [a formal expression for the equilibrium distribution is given in Eq. (88)]. At When using the rate equation for closed CRNs Eq. (7),we equilibrium, the Gibbs free energy Eq. (70) reads ρ σ find that dtG ¼ −J ∇ρμσ. Using Eq. (74) together with G Zσ μeq − RTZS G : Eqs. (46) and (58), we get eq ¼ eq σ eq þ 0 ð71Þ d G RTd L Zσ Z σ −TS_ ≤ 0; As we discuss in Sec. III B, the equilibrium chemical t ¼ t ðf gjf eq gÞ ¼ i ð77Þ σ eq eq potentials must satisfy ∇ρμσ ¼ 0. We deduce that μσ must be a linear combination of the closed system conservation which proves the aforementioned result. laws Eq. (12), 3. Chemical work eq λ μσ ¼ fλlσ; ð72Þ In arbitrary CRNs, the rate of change of nonequilibrium Gibbs free energy, Eq. (76), can be related to the entropy where ffλg are real coefficients. Thus, we can write the production rate, Eq. (59), using the rate equations of open equilibrium Gibbs free energy as CRN, Eqs. (9) and (10), and the chemical work rate, Eq. (69): λ S G ¼ fλL − RTZ þ G0: ð73Þ eq eq TS_ W_ − d G ≥ 0: i ¼ c t ð78Þ In this form, the first term of the Gibbs free energy appears This important result shows that the positivity of the as a bilinear form of components fLλg and conjugated entropy production sets an intrinsic limit on the chemical generalized forces ffλg (Ref. [23], Sec. III. 3), which can work that the chemostats must perform on the CRN to be thought of as chemical potentials of the components. change its concentration distribution. The equality sign is From Eq. (72) and the properties of components Eq. (13), S_ ≃ 0 σ eq σ eq achieved for quasistatic transformations ( i ). the equality Z μσ Z μσ follows. Hence, using the eq ¼ If we now integrate Eq. (78) along a transformation definition of chemical potential Eq. (45), the nonequili- generated by an arbitrary time-dependent protocol π t , G Zσ ð Þ brium Gibbs free energy of the generic distribution f g which drives the CRN from an initial concentration dis- G defined above is related to eq [Eq. (73)]by Zσ Zσ tribution f i g to a final one f f g, we find G G RTL Zσ Zσ ; ¼ eq þ ðf gjf eqgÞ ð74Þ TΔ S W − ΔG ≥ 0; i ¼ c ð79Þ where we introduce the relative entropy for non-normalized ΔG G − G where ¼ f i is the difference of nonequilibrium concentration distributions, also called the Shear Lyapunov Gibbs free energies between the final and the initial state. function or the pseudo-Helmholtz function [35,73,74]: Zσ Zσ Let us also consider the equilibrium state f eqig (f eqfg) obtained from fZσg (fZσg) if one closes the network (i.e., Zσ i f L Zσ Z0σ ≡ Zσ ln − ZS − Z0S ≥ 0: 75 interrupts the chemostatting procedure) and lets it relax to ðf gjf gÞ Z0 ð Þ ð Þ σ equilibrium, as illustrated in Fig. 4. The Gibbs free energy difference between these two equilibrium distributions, This quantity is a natural generalization of the relative ΔG ¼ G − G , is related to ΔG via the difference entropy, or Kullback-Leibler divergence, used to compare eq eqf eqi of relative entropies, Eq. (74): two normalized probability distributions [75]. For sim- plicity, we still refer to it as relative entropy. It quantifies ΔG ¼ ΔG þ RTΔL; ð80Þ the distance between two distributions: it is always eq positive and vanishes only if the two distributions are where identical: fZσg¼fZ0σg. Hence, Eq. (74) proves that the nonequilibrium Gibbs free energy of a closed CRN is ΔL ≡ L Zσ Zσ − L Zσ Zσ : ðf gjf eq gÞ ðf gjf eq gÞ ð81Þ always greater than or equal to its corresponding equi- f f i i G ≥ G librium form, eq. Thus, the chemical work Eq. (79) can be rewritten as We now proceed to show that the nonequilibrium Gibbs W − ΔG RTΔL TΔ S; free energy is minimized by the dynamics in closed CRNs; c eq ¼ þ i ð82Þ G— L Zσ Zσ — viz., or, equivalently, ðf gjf eqgÞ [59,76] acts as ΔG a Lyapunov function in closed CRNs. Indeed, the time which is a key result of our paper. eq represents the derivative of G Eq. (70) always reads reversible work needed to reversibly transform the CRN

041064-12 NONEQUILIBRIUM THERMODYNAMICS OF CHEMICAL … PHYS. REV. X 6, 041064 (2016)

IV. THERMODYNAMICS OF COMPLEX- BALANCED NETWORKS In this section, we focus on unconditionally complex- balanced networks. We see that the thermodynamics of these networks bears remarkable similarities to stochastic thermodynamics. Let us first observe that whenever a CRN displays a σ well-defined steady-state distribution fZ x g, the entropy production rate Eq. (55) can be formally decomposed as the sum of an adiabatic and a nonadiabatic contribution,

J Z ρ þρ σ σ TS_ J RT −d Z x RT x ; i ¼ ln t ln ð84Þ J−ρ Zσ |fflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflffl} |fflfflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflfflffl}x _ _ ≡TSa ≡TSna FIG. 4. Pictorial representation of the transformation between two nonequilibrium concentration distributions. The nonequili- in analogy with what was done in stochastic thermody- brium transformation (blue line) is compared with the equilib- namics [46–50]. As we discuss in Sec. II E, unconditionally rium one (green line). The equilibrium transformation depends on complex-balanced networks have a unique steady-state the equilibrium states corresponding to the initial and final σ σ distribution, fZ x ≡ Z x (πðtÞ)g, Eq. (37), for any value concentration distributions. In Sec. III E 3, for an arbitrary σ σ of the chemostatted concentrations, fZ y ≡ Z y ðπðtÞÞg, and CRN, these equilibrium states are obtained by first closing the λ network and then letting it relax to equilibrium. Instead, in Sec. V, of the fixed unbroken components fL u g. The decompo- for a detailed balance CRN, the equilibrium states are obtained by sition Eq. (84) is thus well defined at any time, for any simply stopping the time-dependent driving and letting the protocol πðtÞ. As a central result, we prove in Appendix C system spontaneously relax to equilibrium. that the adiabatic and nonadiabatic contribution are non- negative for unconditionally complex-balanced networks σ σ as well as for complex-balanced networks without time- from fZ g to fZ g. Implementing such a reversible eqi eqf dependent driving. transformation may be difficult to achieve in practice. The adiabatic entropy production rate encodes the However, it allows us to interpret the difference Wirr ≡ σ c dissipation of the steady state fZ x g. It can be rewritten W − ΔG c eq in Eq. (82) as the chemical work dissipated in terms of the steady-state Gibbs free energy of reaction during the nonequilibrium transformation, i.e., the irre- fΔrGρg Eq. (48) as versible chemical work. The positivity of the entropy production implies that TS_ −JρΔ G ≥ 0: a ¼ r ρ ð85Þ Wirr ≥ RTΔL: c ð83Þ This inequality highlights the fact that the transient dynamics—generating the currents fJρg—is constrained This relation sets limits on the irreversible chemical work by the thermodynamics of the complex-balanced steady Δ G involved in arbitrary far-from-equilibrium transformations. state, i.e., by f r ρg. For transformations connecting two equilibrium distribu- The nonadiabatic entropy production rate characterizes Wirr ≥ 0 tions, we get the expected inequality c . More the dissipation of the transient dynamics. It can be interestingly, Eq. (83) tells us how much chemical work decomposed as the chemostat needs to provide to create a nonequilibrium σ σ TS_ −RTd L Z x Z x distribution from an equilibrium one. It also tells us how na ¼ t ðf gjf gÞ much chemical work can be extracted from a CRN relaxing S σ þ RTdtZ x − Z x dtμσ ≥ 0; ð86Þ to equilibrium. |fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}x The conceptual analogue of Eq. (82) in stochastic _ ≡TSd thermodynamics (where probability distributions replace P S σ non-normalized concentration distributions) is called the where Z x Z x (see Refs. [46,48] for the analo- ¼ σx∈Sx nonequilibrium Landauer principle [42,43] (see also gous decomposition in the stochastic context). The first Refs. [77–79]). It has been shown to play a crucial role term is proportional to the time derivative of the relative in analyzing the thermodynamic cost of information entropy Eq. (75) between the nonequilibrium concentration processing (e.g., for Maxwell demons, feedback control, distribution at time t and the corresponding complex- or proofreading). The inequality Eq. (83) is, therefore, a balanced steady-state distribution. Hence, it describes the nonequilibrium Landauer principle for CRN. dissipation of the relaxation towards the steady state. The

041064-13 RICCARDO RAO and MASSIMILIANO ESPOSITO PHYS. REV. X 6, 041064 (2016)

TS_ forces fλ can be determined by inverting the nonlinear set second term, d, is related to the time-dependent driving u λ λ σ performed via the chemostatted species and thus denoted L u l u Z x of equations ¼ σx eq. They, therefore, depend on both σ λ the driving entropy production rate [46]. It vanishes in fZ y g and fL u g. nondriven networks where we obtain One can easily recover the local detailed-balanced property [Eqs. (50) and (18)] using Eq. (88). σ σ S_ −Rd L Z x Z x ≥ 0: na ¼ t ðf gjf gÞ ð87Þ B. Open nondriven networks This result shows the role of the relative entropy σ σ LðfZ x gjfZ x gÞ as a Lyapunov function in nondriven As a consequence of the break of conservation laws, the complex-balanced networks with mass action kinetics. It nonequilibrium Gibbs free energy G Eq. (70) is no longer was known in the mathematical literature [35,80],butwe minimized at equilibrium in open detailed-balanced provide a clear thermodynamic interpretation to this result networks. In analogy to equilibrium thermodynamics by demonstrating that it derives from the nonadiabatic [23], the proper thermodynamic potential is obtained from entropy production rate. G by subtracting the energetic contribution of the broken We mention that an alternative derivation of the conservation laws. This transformed nonequilibrium Gibbs adiabatic–nonadiabatic decomposition for nondriven com- free energy reads plex-balanced networks with mass action kinetics was very λ G ≡ G − f L b recently found in Ref. [51]. λb σ λ S ¼ Z ðμσ − fλ lσb Þ − RTZ þ G0: ð90Þ V. THERMODYNAMICS OF OPEN b DETAILED-BALANCED NETWORKS We proceed to show that G is minimized by the dynamics σ We finish our study by considering detailed-balanced in nondriven open detailed-balanced networks. Let fZ x g networks. We discuss the equilibrium distribution, intro- be a generic concentration distribution in a detailed- λ σ duce a new class of nonequilibrium potentials, and derive a balanced network characterized by fL u g and fZ y g, and σx new work inequality. let fZeqg be its corresponding equilibrium. Using the Let us also emphasize that open detailed-balanced relation between equilibrium chemical potentials and con- CRNs are a special class of open complex-balanced servation laws Eq. (72), the transformed Gibbs free energy CRNs for which the adiabatic entropy production rate Eq. (90) at equilibrium reads vanishes (since the steady state is detailed balanced) and λ S thus the nonadiabatic entropy production characterizes the G f L u − RTZ G : eq ¼ λu eq þ 0 ð91Þ entire dissipation. Yet, combining Eq. (72) and the properties of unbroken σ eq λ A. Equilibrium distribution Z μσ − f lσb components, one can readily show that eqð λb Þ¼ ρ λ k σ eq b As we discuss in Sec. II D, for given kinetics f g, Z ðμσ − fλ lσ Þ. The relation between the nonequilibrium σ λ b chemostatting fZ y g and unbroken components fL u g, G and the corresponding equilibrium value thus follows detailed-balanced networks always relax to a unique σ σ G G RTL Z x Z x equilibrium distribution. Since the equilibrium chemical ¼ eq þ ðf gjf eqgÞ ð92Þ potentials can be expressed as a linear combination of conservation laws, Eq. (72), we can express the equilibrium (we show in Appendix A the derivation of the latter distribution as in the presence of a reacting solvent). The non-negativity   of the relative entropy for concentration distributions μ∘ − f lλ σ σ eq σ λ σ LðfZ x gjfZ x gÞ ensures that the nonequilibrium trans- Zσ ¼ exp − ; ð88Þ eq RT formed Gibbs free energy is always greater than or equal to G ≥ G its equilibrium value, eq. Since entropy production inverting the expression for the chemical potentials and nonadiabatic entropy production coincide, using Eq. (45). Since the independent set of unbroken conserva- Eqs. (87) and (92), we obtain λ λ l u l u 0 ∀λ σ tion laws f g is such that σy ¼ , u, y, see Sec. II C, σ σ d G RTd L Z x Z x −TS_ ≤ 0; we have that t ¼ t ðf gjf eq gÞ ¼ i ð93Þ

λ μeq f l b ; ∀σ ∈ S : G σy ¼ λ σy y y ð89Þ which demonstrates the role of as a Lyapunov function. b σ σ The relative entropy LðfZ x gjfZ0 x gÞ was known to be a λ We thus conclude that the j bj broken generalized forces Lyapunov function for detailed-balanced networks [76,81], ffλ g depend on only the chemostatted concentrations but we provide its clear connection to the transformed σb Z y λ f g. Instead, the remaining j uj unbroken generalized nonequilibrium Gibbs free energy. To summarize, instead

041064-14 NONEQUILIBRIUM THERMODYNAMICS OF CHEMICAL … PHYS. REV. X 6, 041064 (2016) of minimizing the nonequilibrium Gibbs free energy G 1 M1 ¼ Za þ Zb; Eq. (70) as in closed CRNs, the dynamics minimizes the 2 G 2 transformed nonequilibrium Gibbs free energy in open M ¼ Zd þ Ze: ð97Þ nondriven detailed-balanced CRNs. For the specific implementation of that CRN, Fig. 3, C. Open driven networks the first term (second term) is the total number of We now consider unconditionally detailed-balanced moiety 2H (C) in the system, which can be exchanged CRNs. As we discuss in Sec. II D, they are characterized with the environment only via the chemostatted species σx σx H2O (CO). □ by a unique equilibrium distribution fZeq ≡ ZeqðπðtÞÞg, defined by Eq. (18), for any value of the chemostatted We now turn to the new work relation. From the general σ σ work relation Eq. (78), using Eqs. (90) and (95), we find concentrations fZ y ¼ Z y (πðtÞ)g. σ We start by showing that the external fluxes fI y g can be TS_ W_ − d G ≥ 0; expressed as the influx rate of moieties. Since the CRN is i ¼ d t ð98Þ open and unconditionally detailed balanced, each chemo- statted species breaks a conservation law (no emergent where the driving work due to the time-dependent driving cycle is created, Sec. II D). Therefore, the matrix whose of the chemostatted species is obtained using the chemical λ l b work rate Eq. (69) together with Eqs. (95) and (96): entries are f σy g in Eq. (89) is square and also nonsingular [82]. We can thus invert Eq. (89) to get λ W_ ≡ W_ − d f L b d c tð λb Þ σ eq σ eq σ eq ˆ y μσ d M y − d μσ M y fλ ¼ μσ l ; ð94Þ ¼ y t tð y Þ b y λb eq σ −dtμσ M y : 99 σ ¼ y ð Þ where flˆ y g denote the entries of the inverse matrix of that λb λ l b Equivalently, the driving work rate Eq. (99) can be defined with entries f σy g. Hence, using the definition of a broken λ λ σ as the rate of change of the transformed Gibbs free energy component, L b ≡ lσb Z , we obtain that f g Eq. (90) due to the time-dependent driving only; i.e., σ λ eq ˆ y λb σ fλ L b ¼ μσ lλ lσ Z : ð95Þ ∂G ∂G b y |fflfflfflfflffl{zfflfflfflfflffl}b W_ ≡ ≡ d μeq : d t σy eq ð100Þ σ ∂t ∂μ ≡M y σy

σ From the rate equations for the chemostatted concentrations To relate this alternative definition to Eq. (99), all fZ y g μeq Eq. (10), we find that must be expressed in terms of f σy g using the definition of chemical potential Eq. (45). σ σ _ d M y I y ; ∀σ ∈ S : W t ¼ y y ð96Þ The driving work rate d vanishes in nondriven CRNs, where Eq. (98) reduces to Eq. (93). After demonstrating σ We can thus interpret M y as the concentration of a moiety that the entropy production rate is always proportional to that is exchanged with the environment only through the the difference between the chemical work rate and the chemostatted species Xσ . Equation (95) shows that the change of nonequilibrium Gibbs free energy in Eq. (79),we y show that, for unconditionally detailed-balanced CRNs, it energetic contribution of the broken components can be is also proportional to the difference between the driving expressed as the Gibbs free energy carried by these specific work rate and the change in transformed nonequilibrium moieties. Gibbs free energy, Eq. (98). Example 12.—A simple implementation of this scenario We end by formulating a nonequilibrium Landauer is the thermodynamic description of CRNs at constant pH principle for the driving work instead of the chemical (Ref. [23], Chap. 4), where the chemostatted species þ work done in Sec. III E 3. We consider a time-dependent becomes the ion Hþ and MH is the total amount of þ transformation driving the unconditionally detailed- H ions in the system. The transformed Gibbs potential Zσ Zσ þ balanced CRN from f i g to f f g. The distribution G0 G − μ MH σ σ thus becomes ¼ H and the transformed Z Z þ f eqig (f eqf g) denotes the equilibrium distribution chemical potentials can be written in our formalism as Zσ Zσ obtained from f i g (f f g) by stopping the time-dependent 0 ˆ Hþ b b μσ μσ − μ l lσ lσ x ¼ x Hþ b , where is the conservation law driving and letting the system relax towards the equilib- broken by chemostatting Hþ. □ rium, Fig. 4. Note that this reference equilibrium state is Example 13.—For the CRN in Fig. 2, whose conserva- different from the one obtained by closing the network in tion laws are given in example 4, the concentrations of the Sec. III E 3. Integrating Eq. (98) over time and using exchanged moieties are Eq. (92), we get

041064-15 RICCARDO RAO and MASSIMILIANO ESPOSITO PHYS. REV. X 6, 041064 (2016)

W − ΔG RTΔL TΔ S; d eq ¼ þ i ð101Þ thermodynamics. Closed CRNs (nondriven open detailed- balanced networks) always relax to a unique equilibrium where by minimizing their nonequilibrium Gibbs free energy (transformed nonequilibrium Gibbs free energy). This σ σ σ σ ΔL ≡ L Z x Z x − L Z x Z x : ðf f gjf eqf gÞ ðf i gjf eqigÞ ð102Þ latter is given, up to a constant, by the relative entropy between the nonequilibrium and equilibrium concentra- ΔG eq represents the reversible driving work, and the tion distribution. Nondriven complex-balanced networks irreversible driving work satisfies the inequality relax to complex-balanced nonequilibrium steady states by minimizing the relative entropy between the non- Wirr ≡ W − ΔG ≥ RTΔL: d d eq ð103Þ equilibrium and steady-state concentration distribution. In all these cases, even in the presence of driving, we show This central relation sets limits on the irreversible work how the rate of change of the relative entropy relates to spent to manipulate nonequilibrium distributions. It is a the CRN dissipation. For complex-balanced networks, we nonequilibrium Landauer principle for the driving work by also demonstrate that the entropy production rate can be the same reasons that inequality Eq. (83) is a nonequili- decomposed, as in stochastic thermodynamics, in its brium Landauer principle for the chemical work. The key adiabatic and nonadiabatic contributions quantifying, difference is that the choice of the reference equilibrium respectively, the dissipation of the steady state and of state is different in the two cases. The above discussed the transient dynamics. inequality Eq. (103) holds only for unconditionally Our framework could be used to shed new light on a detailed-balanced CRNs, while Eq. (83) is valid for any broad range of problems. We mention only a few. CRNs. Stochastic thermodynamics has been successfully used to study the thermodynamics cost of information process- VI. CONCLUSIONS AND PERSPECTIVES ing in various synthetic and biological systems [44,83–87]. Following a strategy reminiscent of stochastic thermo- However, most of these are modeled as few state systems or — dynamics, we systematically build a nonequilibrium linear networks [8,9] e.g., quantum dots [88], molecular — thermodynamic description for open driven CRNs made motors [89,90], and single enzyme mechanisms [91,92] of elementary reactions in homogeneous ideal dilute while biochemical networks involve more-complex solutions. The dynamics is described by deterministic rate descriptions. The present work overcomes this limitation. equations whose kinetics satisfies mass action law. Our It could be used to study biological information-handling framework is not restricted to steady states and allows processes, such as kinetic proofreading [93–99] or enzyme- us to treat transients as well as time-dependent drivings assisted copolymerization [92,100–105], which have cur- performed by externally controlled chemostatted concen- rently only been studied as single enzyme mechanisms. trations. Our theory embeds the nonequilibrium thermo- Our theory could also be used to study metabolic dynamic framework of irreversible processes established by networks. However, these require some care, since complex the Brussels School of Thermodynamics. enzymatic reaction mechanisms are involved [106]. We now summarize our results. Starting from the Nevertheless, our framework provides a basis to build expression for the entropy production rate, we establish effective coarse-graining procedures for enzymatic reac- a nonequilibrium formulation of the first and second law of tions [107]. For instance, proofreading mechanisms oper- thermodynamics for CRNs. The resulting expression for ating in metabolic processes could be considered [108].We the system entropy is that of an ideal dilute solution. The foresee an increasing use of thermodynamics to improve clear separation between chemostatted and internal species the modeling of metabolic networks, as recently shown in allows us to identify the chemical work done by the Refs. [30,32,33]. chemostats on the CRN and to relate it to the nonequili- Since our framework accounts for time-dependent brium Gibbs potential. We are also able to express the drivings and transient dynamics, it could be used to minimal chemical work necessary to change the non- represent the transmission of signals through CRNs or equilibrium distribution of species in the CRN as a differ- their response to external modulations in the environment. ence of relative entropies for non-normalized distributions. These features become crucial when considering problems The latter measure the distance of the initial and final such as signal transduction and biochemical switches concentration distributions from their corresponding equi- [24,109,110], biochemical oscillations [28,111], growth librium ones, obtained by closing the network. This result is and self-organization in evolving biosystems [112,113], reminiscent of the nonequilibrium Landauer principle or sensory mechanisms [85,87,114–117]. Also, since derived in stochastic thermodynamics [43] and which prove transient signals in CRNs can be used for computation very useful to study the energetic cost of information [118] and have been shown to display Turing universality processing [45]. We also highlight the deep relationship [119–122], one should be able to study the thermodynamic between the topology of CRNs, their dynamics, and their cost of chemical computing [123].

041064-16 NONEQUILIBRIUM THERMODYNAMICS OF CHEMICAL … PHYS. REV. X 6, 041064 (2016)

Finally, one could use our framework to study any while that of the solvent reads process that can be described as nucleation or reversible – ZS polymerization [124 129] (see also Ref. [130], Chaps. 5 ad ° μ0 ≃ μ0 − RT ; ðA4Þ 6) since these processes can be described as CRNs [63]. Z0 As closing words, we believe that our results constitute P S σ an important contribution to the theoretical study of CRNs. where Z ≡ σ∈SZ . Therefore, the Gibbs free energy It does for nonlinear chemical kinetics what stochastic Eq. (A1) reads thermodynamics has done for stochastic dynamics, namely, build a systematic nonequilibrium thermodynamics on top σ 0 ° S G ≃ Z μσ þ Z μ − RTZ ; ðA5Þ of the dynamics. It also opens many new perspectives and 0 builds bridges between approaches from different com- 0 ° which is Eq. (70), where G0 is equal to Z μ plus possibly munities studying CRNs: mathematicians who study CRNs 0 the Gibbs free energy of solutes that do not react. as dynamical systems, physicists who study them as We now consider the case where the solvent reacts with nonequilibrium complex systems, and biochemists as well as bioengineers who aim for accurate models of metabolic the solutes. We assume that both the solutes and the solvent networks. react according to the stoichiometric matrix 0 1 ∇0 ACKNOWLEDGMENTS B C ∇ ¼ @ ∇X A; ðA6Þ The present project was supported by the Luxembourg ∇Y National Research Fund (Project No. FNR/A11/02 and AFR Ph.D. Grant 2014-2, No. 9114110) as well as by the European Research Council (Project No. 681456). where the first row refers to the solvent, the second block of rows to the internal species, and the last one to the chemostatted species. The solvent is treated as a chemo- APPENDIX A: THERMODYNAMICS OF IDEAL statted species such that dtZ0 0. DILUTE SOLUTIONS ¼ In order to recover the expression for the transformed We show that the nonequilibrium Gibbs free energy Gibbs free energy Eq. (92) in unconditionally detailed- Eq. (70) is the Gibbs free energy of an ideal dilute solution balanced networks, we observe that, at equilibrium, (Ref. [131], Chap. 7) (see also Ref. [51]). We also show that σ eq 0 eq in open detailed-balanced networks in which the solvent ∇ρμσ þ ∇ρμ0 ¼ 0: ðA7Þ reacts with the solutes, the expression of the transformed Gibbs free energy Eq. (92) is recovered by treating the Therefore, the equilibrium chemical potentials are a linear solvent as a special chemostatted species. combination of the conservation laws of ∇ [Eq. (A6)], The Gibbs free energy (density) of an ideal dilute mixture of chemical compounds kept at constant temper- eq λ μσ ¼ fλlσ; ature and pressure reads eq λ μ0 ¼ fλl0: ðA8Þ σ G ¼ Z μσ þ Z0μ0 ðA1Þ As mentioned Sec. II C, the chemostatting procedure λ where the labels σ ∈ S refer to the solutes and 0 to the breaks some conservation laws, which are labeled by b. λ solvent. The chemical potentials of each species Eq. (45) The unbroken ones are labeled by u. read The transformed Gibbs free energy is defined as in Eq. (90), reported here for convenience, ∘ Zσ μσ ¼ μσ þ RT ln ; ∀σ ∈ S; λ Z G ≡ G − f L b ; tot λb ðA9Þ Z0 μ μ∘ RT : λ 0 ¼ 0 þ ln Z ðA2Þ where G reads as in Eq. (A1), fL b g are the broken tot components, and ffλ g are here interpreted as the con- P b Z Zσ Z ≃ Z Since the solution is dilute, tot ¼ σ∈S þ 0 0 and jugated generalized forces. Adding and subtracting the term ∘ σ eq eq the standard-state chemical potentials fμσg depend on the Z μσ þ Z0μ0 from the last equation and using Eq. (A8), nature of the solvent. Hence, the chemical potentials of the we obtain solutes read G G Zσ μ − μeq Z0 μ − μeq ; ¼ eq þ ð σ σ Þþ ð 0 0 Þ ðA10Þ ∘ Zσ μσ ≃ μσ þ RT ln ; ∀σ ∈ S; ðA3Þ Z0 where

041064-17 RICCARDO RAO and MASSIMILIANO ESPOSITO PHYS. REV. X 6, 041064 (2016)

λ nˆ σ X G ¼ fλ L u : ðA11Þ σ ∘ σ σ eq u sðnˆ Þ ≃ nˆ s~σ − nˆ kB ln þ kB nˆ n0 σ σ σ y n From Eqs. (A3) and (A4) and the fact that Z y ¼ Z and σ ∘ 0 eq nˆ s~σ kB ln eq ¼ þ VN Z0 ¼ Z0 , we obtain A σ X σ nˆ σ þ −nˆ kB ln þ kB nˆ σ eq S S VN G ≃ G Z x μ − μσ − RT Z x − Z x A σ eq þ ð σx x Þ ð eq Þ σ σ ∘ Z x ≡ nˆ ðs~σ þ kB ln Z0Þ σ S Sx ¼ G þ Z x RT ln σ − RTðZ x − Zeq Þ X eq Z x σ σ σ eq þ −nˆ kB ln Z þ kB nˆ : ðB4Þ σ σ σ ≡ G RTL Z x Z x ; eq þ ðf gjf eqgÞ ðA12Þ V R N k Dividing by and using the relation ¼ A B, we finally in agreement with the expression derived in the main get the macroscopic entropy density Eq. (63) text, Eq. (92). σ ∘ σ S hSi=V ≃ Z sσ − Z R ln Zσ þ RZ ; ðB5Þ

∘ APPENDIX B: ENTROPY OF CRNs where the (molar) standard entropies of formation sσ reads We show how the nonequilibrium entropy Eq. (63) can s∘ N s~∘ k Z : σ ¼ Að σ þ B ln 0Þ ðB6Þ be obtained as a large particle limit of the stochastic entropy. We point out that very recently similar derivations Mindful of the information-theoretical interpretation of for other thermodynamic quantities have been obtained in the entropy [133], we note that the uncertainty due to the Refs. [51,132]. stochasticity of the state disappears [the first term on In the stochastic description of CRNs, the state is σ the rhs of Eq. (B1)]. However, the uncertainty due to the characterized by the population vector n ¼fn g. The indistinguishability of the molecules of the same species— probability to find the network is in state n at time t is quantified by the configurational entropy Eq. (B2)— denoted ptðnÞ. The stochastic entropy of that state reads remains and contributes to the whole deterministic entropy [21,107], up to constants, function Eq. (63).

SðnÞ¼−kB ln ptðnÞþsðnÞ: ðB1Þ APPENDIX C: ADIABATIC-NONADIABATIC DECOMPOSITION The first term is a Shannon-like contribution, while the We prove the positivity of the adiabatic and nonadiabatic second term is the configurational entropy, entropy production rates Eq. (84) using the theory of complex-balanced networks; see Sec. II E. σ X We first rewrite the mass action kinetics0 currents n ! ρ 0 γ s n ≡ nσs~∘ − k : Jρ K ψ γ ψ γ ≡ ZσΓσ K ð Þ σ B ln nσ ðB2Þ Eq. (5) as [53,81] ¼ γ0 , where and ¼ n0 ρ þρ −ρ σ fKγ ≡ Kγ − Kγ g is the rate constants matrix whose entries are defined by ∘ s~σ is the standard entropy of one single Xσ molecule, and n0 8 kþρ γ ρ is the very large number of solvent molecules. < if is the reactant complex of þ ρ −ρ We now assume that the probability becomes very Kγ ¼ −k if γ is the product complex of þ ρ nσ ≫ 1 : narrow in the large particle limit and behaves as 0 otherwise: a discrete delta function ptðnÞ ≃ δ½n − nˆ ðtÞ. The vector nˆ ðtÞ ≡ fnˆ σg denotes the most probable and macroscopic ðC1Þ Zσ nˆ σ= VN amount of chemical species, such that ¼ ð AÞ. Hence, the average entropy becomes Hence, the definition of a complex-balanced network Eq. (37) reads X X S p n S n ≃ s nˆ : Wγ ψ γ0 0; ∀j; h i¼ tð Þ ð Þ ð Þ ðB3Þ γ0 ¼ ðC2Þ n γ∈Cj

W ≡∂K ∂γ Kρ ≡ Wγ m! ≃ m m − where ¼f ρ γ0 g f γ0 g is the so-called When using the Stirling approximation (ln ln γ m for m ≫ 1), we obtain kinetic matrix [35], and ψ γ ≡ ZσΓσ .

041064-18 NONEQUILIBRIUM THERMODYNAMICS OF CHEMICAL … PHYS. REV. X 6, 041064 (2016)

The kinetic matrix W is a Laplacian matrix [76,81]: Concerning the nonadiabatic term Eq. (84), using the Z Z any off-diagonal term is equal to the rate constant of rate equations (9) and the fact that f σy ¼ σy g, we can the reaction having γ0 as a reactant and γ as a product rewrite it as if the reaction exists, and it is zero otherwise. Also, it Z ψ satisfies S_ ≡ −d Zσ σ −Wγ ψ γ0 γ : na t ln ¼ γ0 ln ðC9Þ X Zσ ψ γ Wγ 0; γ0 ¼ ðC3Þ γ∈C Because of Eq. (C3), we further get that

γ0 which is a consequence of the fact that the diagonal ψ γψ S_ −Wγ ψ γ0 : na ¼ γ0 ln γ0 ðC10Þ terms are equal to minus the sum of the off-diagonal ψ γψ terms along the columns. The detailed balanced prop- erty Eq. (18) implies that From the log inequality − ln x ≥ 1 − x and from Eq. (C4),

γ γ0 eq eq 0 γ0 W 0 ψ 0 ¼ Wγ ψ γ ; ∀γ; γ ; ðC4Þ ψ ψ ψ γ γ _ γ γ0 γ γ γ0 γ S ≥ W 0 ψ 1 − −W 0 ψ 0: C11 na γ γ0 ¼ γ ψ ¼ ð Þ eq eqΓσ ψ γψ γ where ψ γ ≡ Zσ γ . In order to prove the non-negativity of the adiabatic term The last equality again follows from the assumption of a Eq. (84), we rewrite it as complex-balance steady state Eq. (C2) as in Eq. (C8).

σ J Z −∇ρ S_ ≡ Jρ þρ Kρ ψ γ0 σ a ln ¼ γ0 ln eq J−ρ Zσ APPENDIX D: DEFICIENCY OF CRNs ψ −Wγ ψ γ0 γ : The deficiency of an open CRN is defined as [22] ¼ γ0 ln eq ðC5Þ ψ γ δ ¼ dim ker ∇X − dim ker ∂C ≥ 0; ðD1Þ The detailed balance property is used in the first P C j γ ∂ ∂ρ ≡ ∂ρ equality, and the decomposition of the stoichiometric where ¼f γ∈Cj g. Other equivalent definitions matrix Eq. (30) in the second one. Also, the constant RT can be found in Refs. [52,53]. The kernel of ∇X identifies is taken equal to 1. Using Eq. (C3),Eq.(C5) can be the set of cycles, Eqs. (19) and (20), while the kernel of the rewritten as incidence matrix ∂ˆ identifies the set of cycles of the reaction graph. Hence, the deficiency measures the differ- γ0 ψ γψ ence between the number of cyclic transformations on S_ −Wγ ψ γ0 eq : a ¼ γ0 ln eq γ0 ðC6Þ ψ γ ψ chemical species and how many of them can be represented as cycles on the reaction graph. Deficiency-zero networks From the log inequality − ln x ≥ 1 − x and the detailed are defined by δ ¼ 0; i.e., they exhibit a one-to-one balance property Eq. (C4),weobtain correspondence between the two. This topological property has many dynamical consequences, the most important of ψ ψ γ0 which is that deficiency-zero networks are unconditionally S_ ≥ Wγ ψ γ0 1 − γ eq a γ0 eq γ0 complex balanced [36,37]. As shown in Ref. [22], defi- ψ γ ψ ciency also has implications on the stochastic thermody- ψ ψ γ0 ψ −Wγ ψ γ0 γ −Wγ0 ψ γ γ0 0: namic description of networks: the stochastic entropy ¼ γ0 eq eq γ0 ¼ γ ¼ ðC7Þ ψ γ ψ ψ γ0 production of a deficiency-zero network converges to the deterministic entropy production in the long-time limit. The last equality follows from the assumption of a Linear networks are the simplest class of deficiency-zero complex-balanced steady state Eq. (C2), the properties networks. Since only one internal species appears in each of the groups of complexes fCjg (Sec. II E), and the fact complex with a stoichiometric coefficient equal to 1, Z Z ∇X ≡∂C, and thus δ ¼ 0. that f σy ¼ σy g. Indeed, X X Γσx γ0 ψ γ0 γ0 Zσ γ0 W ψ γ W ψ γ x γ ψ ¼ γ Z γ0 0 σ j r ∈Cj x X Γσx X [1] J. W. Gibbs, The Scientific Papers of J. Willard Gibbs, Zσ j 0 x γ γ Vol. 1: Thermodynamics (Dover, New York, 1961). ¼ Wγ ψ ¼ 0: ðC8Þ Z [2] T. de Donder, L’affinité, Mémoires de la Classe des j σx r0∈C j Sciences (Gauthier-Villars, Paris, 1927), Vol. 1.

041064-19 RICCARDO RAO and MASSIMILIANO ESPOSITO PHYS. REV. X 6, 041064 (2016)

[3] I. Prigogine, Etude Thermodynamique des Phénomènes [26] B. Ø. Palsson, Systems Biology: Simulation of Dynamic Irréversibles (Desoer, Liège, 1947). Network States (Cambridge University Press, Cambridge, [4] I. Prigogine, Introduction to Thermodynamics of Irrevers- England, 2011). ible Processes (John Wiley & Sons, New York, 1967). [27] B. Ø. Palsson, Systems Biology: Constraint-Based Net- [5] I. Prigogine and R. Defay, Chemical Thermodynamics work Reconstruction and Analysis (Cambridge University (Longmans, Green & Co., New York, 1954). Press, Cambridge, England, 2015). [6] G. F. Oster, A. S. Perelson, and A. Katchalsky, Network [28] A. Goldbeter, Biochemical Oscillations and Cellular Thermodynamics: Dynamic Modelling of Biophysical Rhythms (Cambridge University Press, Cambridge, Systems, Q. Rev. Biophys. 6, 1 (1973). England, 1996). [7] M. Malek-Mansour and G. Nicolis, A Master Equation [29] I. R. Epstein and J. A. Pojman, in An Introduction to Description of Local Fluctuations, J. Stat. Phys. 13, 197 Nonlinear Chemical Dynamics: Oscillations, Waves, (1975). Patterns, and Chaos, Topics in Physical Chemistry [8] J. Schnakenberg, Network Theory of Microscopic and (Oxford University Press, New York, 1998). Macroscopic Behavior of Master Equation Systems, Rev. [30] D. A. Beard, S.-d. Liang, and H. Qian, Energy Balance for Mod. Phys. 48, 571 (1976). Analysis of Complex Metabolic Networks, Biophys. J. 83, [9] T. L. Hill, Free Energy Transduction in Biology 79 (2002). (Academic Press, New York, 1977). [31] H. Qian, D. A. Beard, and S.-d. Liang, Stoichiometric [10] C. Y. Mou, J. Luo, and G. Nicolis, Stochastic Thermo- Network Theory for Nonequilibrium Biochemical Systems, dynamics of Nonequilibrium Steady States in Chemical Eur. J. Biochem. 270, 415 (2003). Reaction Systems, J. Chem. Phys. 84, 7011 (1986). [32] D. A. Beard, E. Babson, E. Curtis, and H. Qian, Thermo- [11] J. Ross, Thermodynamics and Fluctuations Far from dynamic Constraints for Biochemical Networks, J. Theor. Equilibrium (Springer, New York, 2008). Biol. 228, 327 (2004). [12] D. A. McQuarrie, Stochastic Approach to Chemical [33] A. Chakrabarti, L. Miskovic, K. C. Soh, and V. Kinetics, J. Appl. Probab. 4, 413 (1967). Hatzimanikatis, Towards Kinetic Modeling of Genome- [13] D. T. Gillespie, A Rigorous Derivation of the Chemical Scale Metabolic Networks without Sacrificing Stoichio- Master Equation, Physica A 188, 404 (1992). metric, Thermodynamic and Physiological Constraints, [14] K. Sekimoto, in Stochastic Energetics, 1st ed., Lecture Biotechnol. J. 8, 1043 (2013). Notes in Physics (Springer-Verlag, Berlin, 2010), Vol. 799. [34] H. Qian and D. A. Beard, Thermodynamics of Stoichio- [15] C. Jarzynski, Equalities and Inequalities: Irreversibility metric Biochemical Networks in Living Systems Far from and the Second Law of Thermodynamics at the Equilibrium, Biophys. Chem. 114, 213 (2005). Nanoscale, Annu. Rev. Condens. Matter Phys. 2,329 [35] F. Horn and R. Jackson, General Mass Action Kinetics, (2011). Arch. Ration. Mech. Anal. 47, 81 (1972). [16] U. Seifert, Stochastic Thermodynamics, Fluctuation The- [36] M. Feinberg, Complex Balancing in General orems and Molecular Machines, Rep. Prog. Phys. 75, Kinetic Systems, Arch. Ration. Mech. Anal. 49, 187 126001 (2012). (1972). [17] C. Van den Broeck and M. Esposito, Ensemble and [37] F. Horn, Necessary and Sufficient Conditions for Complex Trajectory Thermodynamics: A Brief Introduction, Balancing in Chemical Kinetics, Arch. Ration. Mech. Physica A 418, 6 (2015). Anal. 49, 172 (1972). [18] P. Gaspard, Fluctuation Theorem for Nonequilibrium [38] T. G. Kurtz, The Relationship between Stochastic and Reactions, J. Chem. Phys. 120, 8898 (2004). Deterministic Models for Chemical Reactions, J. Chem. [19] D. Andrieux and P. Gaspard, Fluctuation Theorem and Phys. 57, 2976 (1972). Onsager Reciprocity Relations, J. Chem. Phys. 121, 6167 [39] D. F. Anderson, G. Craciun, and T. G. Kurtz, Product- (2004). Form Stationary Distributions for Deficiency Zero [20] T. Schmiedl, T. Speck, and U. Seifert, Entropy Production Chemical Reaction Networks, Bull. Math. Biol. 72, for Mechanically or Chemically Driven Biomolecules, 1947 (2010). J. Stat. Phys. 128, 77 (2007). [40] D. F. Anderson, G. Craciun, M. Gopalkrishnan, and C. [21] T. Schmiedl and U. Seifert, Stochastic Thermodynamics of Wiuf, Lyapunov Functions, Stationary Distributions, and Chemical Reaction Networks, J. Chem. Phys. 126, 044101 Non-Equilibrium Potential for Reaction Networks, Bull. (2007). Math. Biol. 77, 1744 (2015). [22] M. Polettini, A. Wachtel, and M. Esposito, Dissipation in [41] D. Cappelletti and C. Wiuf, Product-Form Poisson-like Noisy Chemical Networks: The Role of Deficiency, J. Distributions and Complex Balanced Reaction Systems, Chem. Phys. 143, 184103 (2015). SIAM J. Appl. Math. 76, 411 (2016). [23] R. A. Alberty, Thermodynamics of Biochemical Reactions [42] K. Takara, H.-H. Hasegawa, and D. J. Driebe, Generali- (Wiley-Interscience, New York, 2003). zation of the Second Law for a Transition between Non- [24] D. A. Beard and H. Qian, Chemical Biophysics. Quanti- equilibrium States, Phys. Lett. A 375, 88 (2010). tative Analysis of Cellular Systems (Cambridge University [43] M. Esposito and C. Van den Broeck, Second Law and Press, Cambridge, England, 2008). Landauer Principle Far from Equilibrium, Europhys. Lett. [25] B. Ø. Palsson, Systems Biology: Properties of 95, 40004 (2011). Reconstructed Networks (Cambridge University Press, [44] T. Sagawa, Thermodynamics of Information Processing in Cambridge, England, 2006). Small Systems (Springer, Japan, 2013).

041064-20 NONEQUILIBRIUM THERMODYNAMICS OF CHEMICAL … PHYS. REV. X 6, 041064 (2016)

[45] J. M. R. Parrondo, J. M. Horowitz, and T. Sagawa, [66] C. A. Petri and W. Reisig, Petri Net, Scholarpedia 3, 6477 Thermodynamics of Information, Nat. Phys. 11, 131 (2008), revision 91646. (2015). [67] J. C. Baez and B. Fong, Quantum Techniques for Studying [46] M. Esposito, U. Harbola, and S. Mukamel, Entropy Equilibrium in Reaction Networks, J. Complex Netw. 3,22 Fluctuation Theorems in Driven Open Systems: Applica- (2015). tion to Electron Counting Statistics, Phys. Rev. E 76, [68] G. Craciun, Toric Differential Inclusions and a Proof of 031132 (2007). the Global Attractor Conjecture, arXiv:1501.02860. [47] R. J. Harris and G. M. Schütz, Fluctuation Theorems for [69] A. Dickenstein and M. P. Millán, How Far is Complex Stochastic Dynamics, J. Stat. Mech. (2007) P07020. Balancing from Detailed Balancing?, Bull. Math. Biol. 73, [48] M. Esposito and C. Van den Broeck, Three Faces of the 811 (2011). Second Law. I. Master Equation Formulation, Phys. Rev. [70] I. Prigogine, Le Domaine de Validité de la Thermodyna- E 82, 011143 (2010). mique des Phénomènes Irréversibles,” Physica A 15, 272 [49] C. Van den Broeck and M. Esposito, Three Faces of the (1949). Second Law. II. Fokker-Planck Formulation, Phys. Rev. E [71] E. R. Cohen, T. Cvitas, J. G. Frey, B. Holmström, K. 82, 011144 (2010). Kuchitsu, R. Marquardt, I. Mills, F. Pavese, M. Quack, J. [50] H. Ge and H. Qian, Physical Origins of Entropy Produc- Stohner, H. L. Strauss, M. Takami, and A. J. Thor, tion, Free Energy Dissipation, and Their Mathematical Quantities, Units and Symbols in Physical Chemistry, Representations, Phys. Rev. E 81, 051133 (2010). 3rd ed. (IUPAC and RSC Publishing, 2008). [51] H. Ge and H. Qian, Nonequilibrium Thermodynamic [72] R. Holyst and A. Poniewierski, Thermodynamics for Formalism of Nonlinear Chemical Reaction Systems with Chemists, Physicists and Engineers (Springer, New York, Waage-Guldberg’s Law of Mass Action, Chem. Phys. 472, 2012). 241 (2016). [73] D. Shear, An Analog of the Boltzmann H-Theorem [52] M. Feinberg, “Lecture on Chemical Reaction Networks” (a Liapunov Function) for Systems of Coupled Chemical (unpublished). Reactions, J. Theor. Biol. 16, 212 (1967). [53] J. Gunawardena, “Chemical Reaction Network Theory for [74] J. Higgins, Some Remarks on Shear’s Liapunov Function In Silico Biologists Contents” (unpublished). for Systems of Chemical Reactions, J. Theor. Biol. 21, 293 [54] S. R. de Groot and P. Mazur, Non-Equilibrium Thermo- (1968). dynamics (Dover, New York, 1984). [75] T. M. Cover and J. A. Thomas, Elements of Information [55] M. Pekar,ˇ Thermodynamics and Foundations of Mass- Theory, 2nd ed. ( Wiley-Interscience, New York, 2006). Action Kinetics, Prog. React. Kinet. Mech. 30, 3 (2005). [76] A. van der Schaft, S. Rao, and B. Jayawardhana, On the [56] M. Polettini and M. Esposito, Irreversible Thermodynam- Mathematical Structure of Balanced Chemical Reaction ics of Open Chemical Networks. I. Emergent Cycles and Networks Governed by Mass Action Kinetics, SIAM J. Broken Conservation Laws, J. Chem. Phys. 141, 024117 Appl. Math. 73, 953 (2013). (2014). [77] I. Procaccia and R. D. Levine, Potential Work: A [57] D. Kondepudi and I. Prigogine, Modern Thermodynamics: Statistical-Mechanical Approach for Systems in Disequi- From Heat Engines to Dissipative Structures, 2nd ed. librium, J. Chem. Phys. 65, 3357 (1976). (Wiley, New York, 2014). [78] S. Vaikuntanathan and C. Jarzynski, Dissipation and Lag [58] A. D. McNaught and A. Wilkinson, Compendium of in Irreversible Processes, Europhys. Lett. 87, 60005 Chemical Terminology, 2nd ed. (Blackwell Science, 1997). (2009). [59] S. Schuster and R. Schuster, A Generalization of [79] H.-H. Hasegawa, J. Ishikawa, K. Takara, and D. J. Driebe, Wegscheider’s Condition. Implications for Properties of Generalization of the Second Law for a Nonequilibrium Steady States and for Quasi-Steady-State Approximation, Initial State, Phys. Lett. A 374, 1001 (2010). J. Math. Chem. 3, 25 (1989). [80] M. Gopalkrishnan, On the Lyapunov Function for [60] J. Leiser and J. J. Blum, On the Analysis of Substrate Complex-Balanced Mass-Action Systems, arXiv: Cycles in Large Metabolic Systems, Cell Biophysics 11, 1312.3043. 123 (1987). [81] A. van der Schaft, S. Rao, and B. Jayawardhana, Complex [61] S. Schuster and C. Hilgetag, On Elementary Flux Modes in and Detailed Balancing of Chemical Reaction Networks Biochemical Reaction Systems at Steady State, J. Biol. Revisited, J. Math. Chem. 53, 1445 (2015). lλb Syst. 02, 165 (1994). [82] The fact that the matrix whose entries are f σy g is λ [62] S. Klamt and J. Stelling, Two Approaches for Metabolic nonsingular follows from l u ∈ ker ∇, ∀λu, and from Pathway Analysis?, Trends Biotechnol. 21, 64 (2003). the linear independence of flλg. [63] R. Rao, D. Lacoste, and M. Esposito, Glucans Monomer- [83] M. Esposito and G. Schaller, Stochastic Thermodynamics Exchange Dynamics as an Open Chemical Network, J. for “Maxwell Demon” Feedbacks, Europhys. Lett. 99, Chem. Phys. 143, 244903 (2015). 30003 (2012). [64] F. Horn, Stability and Complex Balancing in Mass-Action [84] G. Diana, G. B. Bagci, and M. Esposito, Finite-Time Systems with Three Short Complexes, Proc. R. Soc. A 334, Erasing of Information Stored in Fermionic Bits, Phys. 331 (1973). Rev. E 87, 012111 (2013). [65] S. Klamt, U.-U. Haus, and F. Theis, Hypergraphs and [85] J. M. Horowitz and M. Esposito, Thermodynamics with Cellular Networks, PLoS Comput. Biol. 5, e1000385 Continuous Information Flow, Phys. Rev. X 4, 031015 (2009). (2014).

041064-21 RICCARDO RAO and MASSIMILIANO ESPOSITO PHYS. REV. X 6, 041064 (2016)

[86] A. C. Barato and U. Seifert, Unifying Three Perspectives [106] A. Cornish-Bowden, Fundamentals of Enzyme Kinetics, on Information Processing in Stochastic Thermodynamics, 4th ed. (Wiley-Blackwell, 2012). Phys. Rev. Lett. 112, 090601 (2014). [107] M. Esposito, Stochastic Thermodynamics under Coarse [87] S. Bo, M. Del Giudice, and A. Celani, Thermodynamic Graining, Phys. Rev. E 85, 041125 (2012). Limits to Information Harvesting by Sensory Systems, J. [108] C. L. Linster, E. van Schaftingen, and A. D. Hanson, Stat. Mech. (2015) P01014. Metabolite Damage and Its Repair or Pre-emption, Nat. [88] P. Strasberg, G. Schaller, T. Brandes, and M. Esposito, Chem. Biol. 9, 72 (2013). Thermodynamics of a Physical Model Implementing a [109] H. Qian and T. C. Reluga, Nonequilibrium Thermodynam- Maxwell Demon, Phys. Rev. Lett. 110, 040601 (2013). ics and Nonlinear Kinetics in a Cellular Signaling Switch, [89] N. Golubeva and A. Imparato, Efficiency at Maximum Phys. Rev. Lett. 94, 028101 (2005). Power of Interacting Molecular Machines, Phys. Rev. Lett. [110] H. Qian and S. Roy, An Information Theoretical Analysis 109, 190602 (2012). of Kinase Activated Phosphorylation Dephosphorylation [90] B. Altaner, A. Wachtel, and J. Vollmer, Fluctuating Cycle, IEEE Trans. Nanobiosci. 11, 289 (2012). Currents in Stochastic Thermodynamics. II. Energy Con- [111] J. M. G. Vilar, H. Y. Kueh, N. Barkai, and S. Leibler, version and Nonequilibrium Response in Kinesin Models, Mechanisms of Noise-Resistance in Genetic Oscillators, Phys. Rev. E 92, 042133 (2015). Proc. Natl. Acad. Sci. U.S.A. 99, 5988 (2002). [91] U. Seifert, Stochastic Thermodynamics of Single Enzymes [112] G. Nicolis and I. Prigogine, Self-Organization in Non- and Molecular Motors, Eur. Phys. J. E 34, 26 (2011). equilibrium Systems: From Dissipative Structures to [92] R. Rao and L. Peliti, Thermodynamics of Accuracy in Order Through Fluctuations (Wiley-Blackwell, 1977). Kinetic Proofreading: Dissipation and Efficiency Trade- [113] R. Feistel and W. Ebeling, Physics of Self-Organization Offs, J. Stat. Mech. (2015) P06001. and Evolution (Wiley VCH, 2011). [93] J. J. Hopfield, Kinetic Proofreading: A New Mechanism [114] P. Mehta and D. J. Schwab, Energetic Costs of Cellular for Reducing Errors in Biosynthetic Processes Requiring Computation, Proc. Natl. Acad. Sci. U.S.A. 109, 17978 High Specificity, Proc. Natl. Acad. Sci. U.S.A. 71, 4135 (2012). (1974). [115] P. Sartori, L. Granger, C. F. Lee, and J. M. Horowitz, [94] J. Ninio, Kinetic Amplification of Enzyme Discrimination, Thermodynamic Costs of Information Processing in Biochimie 57, 587 (1975). Sensory Adaptation, PLoS Comput. Biol. 10, e1003974 [95] J. J. Hopfield, The Energy Relay: A Proofreading Scheme (2014). Based on Dynamic Cooperativity and Lacking All Char- [116] D. Hartich, A. C. Barato, and U. Seifert, Nonequilibrium acteristic Symptoms of Kinetic Proofreading in DNA Sensing and Its Analogy to Kinetic Proofreading, Replication and Protein Synthesis, Proc. Natl. Acad. New J. Phys. 17, 055026 (2015). Sci. U.S.A. 77, 5248 (1980). [117] S.-W. Wang, Y. Lan, and L.-H. Tang, Energy Dissipation [96] E. D. Sontag, Structure and Stability of Certain Chemical in an Adaptive Molecular Circuit, J. Stat. Mech. (2015) Networks and Applications to the Kinetic Proofreading P07025. Model of T-Cell Receptor Signal Transduction, IEEE [118] DNA Computing and Molecular Programming, edited by Trans. Autom. Control 46, 1028 (2001). Y. Sakakibara and Y. Y. Mi (Springer, Berlin, 2011); DNA [97] H. Ge, M. Qian, and H. Qian, Stochastic Theory of Computing and Molecular Programming, edited by Luca Nonequilibrium Steady States. Part II: Applications in Cardelli and W. Shih (Springer, Berlin, 2011); DNA Chemical Biophysics, Phys. Rep. 510, 87 (2012). Computing and Molecular Programming, edited by D. [98] A. Murugan, D. H. Huse, and S. Leibler, Speed, Dissipa- Stefanovic and A. Turberfields (Springer, Berlin, 2012); tion, and Error in Kinetic Proofreading, Proc. Natl. Acad. DNA Computing and Molecular Programming, edited by Sci. U.S.A. 109, 12034 (2012). D. Soloveichik and B. Yurke (Springer International [99] A. Murugan, D. H. Huse, and S. Leibler, Discriminatory Publishing, 2013); DNA Computing and Molecular Pro- Regimes in Non-Equilibrium Systems, Phys. Rev. X 4, gramming, edited by S. Murata and S. Kobayashi (Springer 021016 (2014). International Publishing, 2014); DNA Computing and [100] C. H. Bennett, Dissipation-Error Tradeoff in Proofread- Molecular Programming, edited by A. Phillips and P. ing, BioSystems 11, 85 (1979). Yin (Springer International Publishing, 2015). [101] D. Andrieux and P. Gaspard, Nonequilibrium Generation [119] A. Hjelmfelt, E. D. Weinberger, and J. Ross, Chemical of Information in Copolymerization Processes, Proc. Natl. Implementation of Neural Networks and Turing Machines, Acad. Sci. U.S.A. 105, 9516 (2008). Proc. Natl. Acad. Sci. U.S.A. 88, 10983 (1991). [102] D. Andrieux and P. Gaspard, Molecular Information [120] A. Hjelmfelt, E. D. Weinberger, and J. Ross, Chemical Processing in Nonequilibrium Copolymerizations, J. Implementation of Finite-State Machines, Proc. Natl. Chem. Phys. 130, 014901 (2009). Acad. Sci. U.S.A. 89, 383 (1992). [103] J. R. Arias-Gonzalez, Entropy Involved in Fidelity of DNA [121] M. O. Magnasco, Chemical Kinetics is Turing Universal, Replication, PLoS One 7, e42272 (2012). Phys. Rev. Lett. 78, 1190 (1997). [104] P. Sartori and S. Pigolotti, Kinetic versus Energetic [122] D. Soloveichik, M. Cook, E. Winfree, and J. Bruck, Discrimination in Biological Copying, Phys. Rev. Lett. Computation with Finite Stochastic Chemical Reaction 110, 188101 (2013). Networks, Nat. Comput. 7, 615 (2008). [105] P. Sartori and S. Pigolotti, Thermodynamics of Error [123] C. H. Bennett, The Thermodynamics of Computation—A Correction, Phys. Rev. X 5, 041039 (2015). Review, Int. J. Theor. Phys. 21, 905 (1982).

041064-22 NONEQUILIBRIUM THERMODYNAMICS OF CHEMICAL … PHYS. REV. X 6, 041064 (2016)

[124] T. P. J. Knowles, W. Shu, G. L. Devlin, S. Meehan, S. Auer, Reticulum as a Non-Equilibrium Phase Transition, Nat. C. M. Dobson, and M. E. Welland, Kinetics and Thermo- Commun. 5, 3620 (2014). dynamics of Amyloid Formation from Direct Measure- [128] Ö. Kartal, S. Mahlow, A. Skupin, and O. Ebenhöh, ments of Fluctuations in Fibril Mass, Proc. Natl. Acad. Sci. Carbohydrate-Active Enzymes Exemplify Entropic Princi- U.S.A. 104, 10016 (2007). ples in Metabolism, Mol. Syst. Biol. 7, 542 (2011). [125] T. P. J. Knowles, C. A. Waudby, G. L. Devlin, S. I. A. [129] S. Lahiri, Y. Wang, M. Esposito, and D. Lacoste, Kinetics Cohen, A. Aguzzi, M. Vendruscolo, E. M. Terentjev, and Thermodynamics of Reversible Polymerization in M. E. Welland, and C. M. Dobson, An Analytical Solution Closed Systems, New J. Phys. 17, 085008 (2015). to the Kinetics of Breakable Filament Assembly, Science [130] P. Krapivsky, S. Redner, and E. Ben-Naim, A Kinetic 326, 1533 (2009). View of Statistical Physics (Cambridge University Press, [126] S. I. A. Cohen, S. Linse, L. M. Luheshi, E. Hellstrand, Cambridge, England, 2010). D. A. White, L. Rajah, D. E. Otzen, M. Vendruscolo, C. M. [131] Enrico Fermi, Thermodynamics (Dover, New York, Dobson, and T. P. J. Knowles, Proliferation of Amyloid- 1956). β42 Aggregates Occurs through a Secondary Nucleation [132] H. Ge and H. Qian, Mesoscopic Kinetic Basis of Macro- Mechanism, Proc. Natl. Acad. Sci. U.S.A. 110, 9758 scopic Chemical Thermodynamics: A Mathematical (2013). Theory, Phys. Rev. E 94, 052150 (2016). [127] Z. Budrikis, G. Costantini, C. A. M. La Porta, and [133] E. T. Jaynes, Information Theory and Statistical Mechan- S. Zapperi, Protein Accumulation in the Endoplasmic ics, Phys. Rev. 106, 620 (1957).

041064-23