Approximate probability distributions of the

Philipp Thomas∗ School of Mathematics and School of Biological Sciences, University of Edinburgh

Ramon Grima† School of Biological Sciences, University of Edinburgh

Master equations are common descriptions of mesoscopic systems. Analytical solutions to these equations can rarely be obtained. We here derive an analytical approximation of the time-dependent of the master equation using orthogonal polynomials. The solution is given in two alternative formulations: a series with continuous and a series with discrete support both of which can be systematically truncated. While both approximations satisfy the system size expansion of the master equation, the continuous distribution approximations become increasingly negative and tend to oscillations with increasing truncation order. In contrast, the discrete approximations rapidly converge to the underlying non-Gaussian distributions. The theory is shown to lead to particularly simple analytical expressions for the probability distributions of molecule numbers in metabolic reactions and gene expression systems.

I. INTRODUCTION first few moments [12–14]; alternative methods are based on moment closure [15]. It is however the case that the Master equations are commonly used to describe fluc- knowledge of a limited number of moments does not al- tuations of particulate systems. In most instances, how- low to uniquely determine the underlying distribution ever, the number of reachable states is so large that their functions. Reconstruction of the probability distribution combinatorial complexity prevents one from obtaining therefore requires additional approximations such as the analytical solutions to these equations. Explicit solutions maximum entropy principle [16] or the truncated moment are known only for certain classes of linear birth-death generating function [17] which generally yield different processes [1], under detailed balance conditions [2], or results. While the accuracy of these repeated approxi- for particularly simple examples in stationary conditions mations remains unknown, analytical expressions for the [3]. Considerable effort has been undertaken to approx- probability density can rarely be obtained, or might not imate the solution of the master equation under more even exist [18]. A systematic investigation of the distri- general conditions including time-dependence and condi- butions implied by the higher order terms in the SSE, tions lacking detailed balance [4, 5]. without resorting to moments, is therefore still missing. A common technique addressing this issue was given We here analytically derive, for the first time, a closed- by van Kampen in terms of the system size expansion form series expansion of the probability distribution un- (SSE) [6]. The method assumes the existence of a spe- derlying the master equation. We proceed by outlining cific parameter, termed the system size, for which the the expansion of the master equation in Section I and master equation approaches a deterministic limit as its briefly review the solution of the leading order terms value is taken to infinity. The leading order term of this given by the LNA in Section II. While commonly the expansion describes small fluctuations about this limit SSE is truncated at this point, we show that the higher in terms of a Gaussian probability density, called the order terms can be obtained using an asymptotic expan- linear noise approximation (LNA). This approximation sion of the continuous probability density. The resulting has been widely applied in biochemical kinetics [7], but series is given in terms orthogonal polynomials and can also in the theory of polymer assembly [8], epidemics [9], be truncated systematically to any desired order in the economics [10], and machine learning [11]. The benefit inverse system size. Analytical expressions are given for of the LNA is that it yields generic expressions for the the expansion coefficients. probability density. Its deficiency lies in the fact that, Thereby we establish two alternative formulations of strictly speaking, it is valid only in the limit of infinite this expansion: a continuous and a discrete one both sat- system size. Hence one generally suspects that its pre- isfying the expansion of the master equation. We show arXiv:1411.3551v2 [cond-mat.stat-mech] 2 Oct 2015 dictions become inaccurate when one studies fluctuations that for linear birth-death processes, the continuous ap- that are not too small compared to the and there- proximation often fails to converge reasonably fast. In fore implying non-Gaussian statistics. contrast, the discrete approximation introduced in Sec- Higher order terms in the SSE have been employed to tion III accurately converges to the true distribution with calculate non-Gaussian corrections to the LNA for the increasing truncation order. In Section IV, we show that for nonlinear birth-death processes, renormalization is required for achieving rapid convergence of the series. Our analysis is motivated by the use of simple examples ∗ [email protected] throughout. Using a common model of gene expression, † [email protected] we conclude in Section VI that the new method allows to 2 predict the full time-dependence of the molecule number explicitly on Ω. We assume that the propensity possesses distribution. a power series in the inverse volume ∞ X γ (Ω[X], Ω) = Ω Ω−sf (s) ([X]) . (6) II. SYSTEM SIZE EXPANSION r r s=0

As a starting point, we focus on the master equation For mass-action kinetics, for instance, the propensity is 1−`r n  given by γr(n, Ω) = Ω kr`r! , where `r is the re- formulation of biochemical kinetics. We therefore con- `r sider a set of R chemical reactions involving a single action order of the rth reaction. Using the Taylor ex- (0) species confined in a well-mixed volume Ω. Note that pansion of the binomial coefficient, we have fr ([X]) = (s) (s) for chemical systems the system size coincides with the `r `r −s kr[X] , fr ([X]) = kr[X] S`r ,`r −s, and fr = 0 for reaction volume. We denote by Sr the net-change in the s ≥ `r, where S denotes the Stirling numbers of the first th molecule numbers in the r reaction and by γr(n, Ω) the kind. Note also that effective propensities being deduced probability per unit time for this reaction to occur. The from mass action kinetics have an expansion similar to probability of finding n molecules in the volume Ω at time Eq. (6). The Michaelis-Menten propensity γr(n, Ω) = t, denoted by P (n, t), then obeys the master equation n (0) [X] Ωkr n+KΩ [19], for instance, has fr ([X]) = kr [X]+K (s) and fr ([X]) = 0 for s > 0. R dP (n, t) X  Substituting now Eqs. (3-6) into Eq. (1) and rearrang- = E−Sr − 1 γ (n, Ω) P (n, t), (1) dt r ing the result in powers of Ω−1/2, we find r=1 where E−Sr is the step operator defined as E−Sr g(n) =  ∂ d[X] ∂  − Ω1/2 Π(, t) g(n − Sr) for any function g(n) of the molecule numbers ∂t dt ∂ [6]. Note that throughout the article, deterministic initial R N conditions are assumed. The system size expansion now  X ∂ X  = −Ω1/2 S f (0)([X]) + Ω−k/2L Π(, t) proceeds by separating the instantaneous concentration r r ∂ k into a deterministic part, given by the solution of the rate r=1 k=0 −(N+1)/2 equations [X], and a fluctuating part , + O(Ω ). (7) Equating terms to order Ω1/2 yields the deterministic n = [X] + Ω−1/2, (2) rate equation Ω which is van Kampen’s ansatz. The expansion of the R d[X] X master equation can be summarized in three steps: = S f (0)([X]). (8) dt r r (i) Using Eq. (2) one expands the step operator r=1

−Sr The higher order terms in the expansion of the master E γr (n, Ω) P (n, t) = γr (n − Sr, Ω) P (n − Sr, t) −1/2 equation can be written out explicitly −Ω Sr ∂ 1/2 1/2 = e γr(Ω[X] + Ω , Ω)P (Ω[X] + Ω , t), dk/2e k−2(s−1) k−p−2(s−1) (3) X X Dp,s L = (−∂ )pk−p−2(s−1), k p!(k − p − 2(s − 1))!  ∂ s=0 p=1 where ∂ denotes ∂ . (ii) Next, the probability for the molecule numbers is (9) cast into a probability density Π(, t) for the fluctuations where d·e denotes the ceiling value and the coefficients using van Kampen’s ansatz, are given by 1/2 1/2 Π(, t) = Ω P (Ω[X] + Ω , t), (4) R X Dq = (S )p∂q f (s)([X]), (10) which is essentially a change of variables. Note that p,s r [X] r this step implicitly assumes a continuous approximation r=1 Π(, t) of the probability distribution as thought in the which depend explicitly on the solution of the rate equa- original derivation of van Kampen [6]. tion (8). Note that in the following the abbreviation q q (iii) It remains to expand the propensity about the Dp = Dp,0 is used. deterministic limit

∞ k k X  ∂ γr(Ω[X], Ω) III. EXPANSION OF THE CONTINUOUS γ (Ω[X] + Ω1/2, Ω) = Ω−k/2 . (5) r k! ∂[X]k PROBABILITY DENSITY k=0

Note that γr(Ω[X], Ω) is just the propensity evaluated at We here study the time-dependent solution of the par- the macroscopic concentration and hence it must depend tial differential equation approximation of the master 3 equation, Eq. (7). We therefore expand the probabil- ity density of Eq. (4),  ∂  − L Ψ = λ Ψ , (16) ∂t 0 m m m N X −j/2 −(N+1)/2 Π(, t) = Ω πj(, t) + O(Ω ), (11) which is solved by λm = −mJ and Ψm = ψm(, t)π0(, t) j=0 with which also allows the expansion of the time-dependent moments to be deduced in closed-form. Finally we re- −1 m 1    ψm(, t) = π0 (−∂) π0 = Hm . (17) cover the stationary solution as a particular case. σm σ

The functions Hm denote the Hermite orthogonal poly- nomials which are given explicitly in Appendix A. To A. Linear Noise Approximation verify the solution of the eigenvalue problem, we set Ψm+1 = (−∂)Ψm and observe that (∂t − L0)Ψm+1 = Substituting Eq. (11) into Eq. (7) and equating terms −J Ψm+1 − ∂(∂t − L0)Ψm. Using this in Eq. (16), we 0 to order O(Ω ) we find obtain λm+1 = (−J + λm) from which the result follows because λ0 = 0 and Ψ0 = π0.   Using the completeness of the eigenfunctions, we can ∂ P∞ (j) − L0 π0 = 0, (12) write πj(, t) = m=0 am (t)ψm(, t)π0(, t). We verify ∂t in Appendix B that the jth order term in the expansion

1 2 0 involves only the first Nj = 3j eigenfunctions. The con- where L0 = −∂J  + 2 ∂ D2 is a Fokker-Planck operator tinuous SSE approximation is consequently given by the 1 with linear coefficients, and J = D1 is the Jacobian of asymptotic expansion the rate equation. The probability density of fluctuations about the macroscopic concentration, described by , is given by a centered Gaussian  N Nj  X −j/2 X (j) Π(, t) =π0(, t) 1 + Ω am (t)ψm (, t) j=1 m=1  2  1  −(N+1)/2 π0(, t) = exp − , (13) + O(Ω ), (18) p2πσ2(t) 2σ2(t) for which the coefficients can be determined us- 2 which acquires time-dependence via its σ (t). ing the orthogonality of the functions ψm, i.e., The latter satisfies σ2n R n! d ψn(, t)ψm(, t)π0(, t) = δm,n

∂σ2 = 2J (t)σ2 + D0(t), (14) C. The equation for the expansion coefficients ∂t 2 (j) which is the familiar LNA result [6]. In the following The coefficients an are now determined by inserting we will drop the time-dependence of the coefficients for the expansion of πj into Eq. (15), multiplying the re- convenience of notation. σ2n R sult by n! d ψn(, t), and performing the integration. Using Eq. (16), the left hand side of Eq. (15) becomes

B. Higher order terms σ2n X Z  ∂  d ψ (, t) − L a(j)ψ (, t) π (, t) n! n ∂t 0 m m 0 Substituting Eq. (11) into Eq. (7), rearranging the m remaining terms, and equating terms to order Ω−j/2, we  ∂  = − nJ a(j). (19) find ∂t n

The calculation of terms in the summation on the right  ∂  − L0 πj(, t) = L1πj−1 + ... + Ljπ0 hand side of Eq. (15) is greatly simplified by defining the ∂t integral j X = Lkπj−k(, t). (15) σ2n Z k=1 Iαβ = d ψ (, t)(−∂ )αβψ (, t)π (, t), mn n!α!β! n  m 0 This system of partial differential equations can be solved (20) using the eigenfunction approach. We consider which yields 4

(j) where a0 = δ0,j and b·c denotes the floor value. In particular, it follows that mean and variance are given by 2n Z σ PN −j/2 (j) −(N+1)/2 2 2 dψ (, t)L ψ (, t)π (, t) hi = Ω a + O(Ω ) and h i = σ + n! n k m 0 j=1 1 2 PN Ω−j/2a(j) + O(Ω−(N+1)/2). dk/2e k−2(s−1) j=1 2 X X k−p−2(s−1) p,k−p−2(s−1) It is now evident that the coefficients of the expansion = Dp,s Imn . (21) s=0 p=1 are intricately related to the system size expansion of the distribution moments. Naturally, one may seek to invert Using Eqs. (19) and (21) in Eq. (15), we find that the co- this relation. Indeed, as we show in Appendix C, given efficients satisfy the following set of ordinary differential the expansion for a finite set moments, the coefficients equations in Eq. (18) can be uniquely determined. In particular, to construct the probability density to order Ω−j/2 one requires the expansion of the first 3j moments to the  ∂  same order. Thus the problem of moments provides an − nJ a(j) = ∂t n equivalent route of systematically constructing solutions to the master equation. j Nj−k dk/2e k−2(s−1) X X (j−k) X X k−p−2(s−1) p,k−p−2(s−1) am Dp,s Imn , k=1 m=0 s=0 p=1 (22) E. Solution in stationary conditions (j) where we have assumed an = 0 for n > Nj. Explicitly, the non-zero integrals are given by Of particular interest is the expansion of the proba- bility density under stationary conditions. Implicitly, we min(n−α,m) assume here that the rate equation, Eq. (8), has a sin- σβ−α+n−m X m Iαβ = × gle asymptotically stable fixed point, and hence the LNA mn α! s variance is given by σ2 = D0/(−2J ). Setting the time- s=0 2 derivative on the left hand side of Eq. (22) to zero, we (β + α + 2s − (m + n) − 1)!! , (23) find that the coefficients of Eq. (18) can be expressed in (β + α + 2s − (m + n))!(n − α − s)! terms of lower order ones

(2k)! and zero for odd (α+β)−(m+n). Here (2k−1)!! = 2kk! is the double factorial. Along with Eq. (23), in Appendix j dk/2e k−2(s−1) B we verify the following two important properties of 1 X X X a(j) = − × the asymptotic series solution given deterministic initial n nJ conditions: (i) We have Nj = 3j and hence Eq. (22) k=1 s=0 p=1 (j) 3(j−k) indeed yields a finite number of equations, and (ii) an k−p−2(s−1) X (j−k) p,k−p−2(s−1) vanishes for all times when (n + j) is odd. Dp,s am Imn . (25) q pq m=0 Finally, we note that Dp,s and Imn are generally time- dependent because they are functions of the solution of the rate equation and the LNA variance. Explicit expres- For example, truncating after terms of order Ω−1, we sions for the approximate probability density can now be obtain evaluated to any desired order.

−1/2  (1) (1)  D. Moments of the distribution Π() = π0() + Ω a1 ψ1 () + a3 ψ3 () π0()

−1  (2) (2) (2)  The solution for the probability density enables one to + Ω a2 ψ2 () + a4 ψ4 () + a6 ψ6 () π0() derive closed-form expressions for the moments. These + O(Ω−3/2). (26) are obtained by multiplying Eq. (18) by R d β and per- forming the integration using Eq. (A4) of Appendix B. We find The non-zero coefficients to order Ω−1/2 are given by

2 2 0 bβ/2c (1) σ D D1,1 N β! a = − 1 − , β X −j/2 X 2k (j) −(N+1)/2 1 2J J h i = Ω k σ aβ−2k + O(Ω ), 2 k! 4 2 2 1 0 j=0 k=0 σ D σ D D a(1) = − 1 − 2 − 3 , (27) (24) 3 6J 6J 18J 5

F. The continuous approximation fails under low molecule number conditions

We now study the SSE solution for a linear birth-death process, i.e., its propensities depend at most linearly on the molecular populations. Specifically, we consider the synthesis and decay of a molecular species X,

k0 ∅ )−*− X. (29) k1

The master equation is constructed using S1 = +1, γ1 = Ωk0, S2 = −1, γ2 = k1n, and R = 2 in Eq. (1). The exact stationary solution of the master equation is a Poisson distribution with mean Ω[X] where [X] = k0/k1. The coefficients in Eq. (10) are then given by

m n Dn = δm,0k0 + (−1) k1 (δm,0[X] + δm,1) , (30)

m and Dn,s = 0 for s > 0. The leading order corrections to the LNA given by Eqs. (26-28) lead to very compact expressions for the expansion coefficients and are given by FIG. 1. (Color online) Linear birth-death process. We consider the reaction system (29) in stationary conditions. 2 (1) [X] (2) [X] (2) [X] (A) We compare the exact Poisson distribution (gray) to a3 = , a4 = , a6 = (31) the continuous SSE approximation [Eq. (18) together with 6 24 72 0 −1 Eqs. (25) and (30)] truncated after Ω (LNA, blue line), Ω (1) (2) −3 and a = a = 0. (green), and Ω -terms (red) for parameter values k0 = 0.5, 1 2 k1 = 1 and Ω = 1 giving half a molecule on average. We ob- Though the continuous approximation is expected to serve that the continuous approximation becomes increasingly perform well at large values of Ω, we are particularly negative and tends to oscillations with increasing truncation interested in its performance when the value of Ω is de- order. (B) In contrast the discrete approximation shows no creased. Since the expansion is carried out at constant oscillations, and the overall agreement with the exact Poisson average concentration, low values of Ω typically imply distribution (gray bars) improves with increasing truncation low numbers of molecules and non-Gaussian distribu- order. tions. In Fig. 1A we show that for parameters yielding half a molecule on average, the continuous approxima- tion obtained in this section, given by Eq. (18) together with Eqs. (25) and (30), is unsatisfactory since as higher while those to order Ω−1 are orders are taken into account, one observes large oscil- lations in the tails of the distribution. In the following section we show that the disagreement arises due to the 0 1 2 2 ! 2 assumption that the support of the distribution is con- (2) (1) D1,1 D2 3σ D1 (1) 3D1 a2 = − a1 + + − a3 tinuous rather than discrete as implied by the master 2J 4J 4J 2J equation. D0 σ2D1 σ2D2 σ4D3 − 2,1 − 1,1 − 2 − 1 , 4J 2J 8J 4J IV. DISCRETE APPROXIMATION OF THE  D0 σ2D1 σ4D2  D0 σ2D1 a(2) = − a(1) 3 + 2 + 1 − 4 − 3 PROBABILITY DISTRIBUTION 4 1 24J 8J 8J 96J 24J ! σ4D2 σ6D3 D0 3D1 7σ2D2 The aim of this paragraph is to establish a discrete for- − 2 − 1 − a(1) 1,1 + 2 + 1 , 16J 24J 3 4J 8J 8J mulation of the distribution approximations.. To clarify this issue, we note that the exact characteristic function ∞ (2) 1 (1) P ikn a = (a )2. (28) G(k, t) = n=0 e P (n, t) is a 2π-periodic function, and 6 2 3 hence can be inverted as follows

Z π dk The accuracy of this distribution approximation is stud- P (n, t) = e−iknG(k, t). (32) ied through an example in the following. −π 2π 6

We now associate our continuous approximation, Eq. The above follows from the definition of the eigenfunc- (18), with this characteristic function, i.e., G(k, t) = tions, Eq. (17), and using the derivative property of the R ∞ ikΩ([X]+Ω−1/2) convolution given after Eq. (34). Note that the coeffi- −∞ d e Π(, t). Substituting this together with Eq. (11) into Eq. (32) one establishes a connection cients in this equation are exactly the same as given in formula between these discrete and continuous approxi- Eq. (18) and hence are determined by Eq. (22). One mations via the convolution can verify two limiting cases: (i) as Σ → 0 and Ω[X] be- ing integer-valued, then P0(n) = K(n − Ω[X]) = δn,Ω[X] is just the Kronecker delta as required for determinis- N Z ∞ tic initial conditions; (ii) as Ω → ∞ with y/Σ constant, X −j/2 1/2 P (n, t) = Ω d K(n − Ω[X] − Ω )πj(, t) the probability distribution P0 reduces to the density π0 j=0 −∞ given by Eq. (13) and hence it follows that in this limit + O(Ω−(N+1)/2), (33) the continuous and discrete series give the same results. with kernel A. The discrete approximation performs well for Z π dk sin(πs) linear birth-death processes K(s) = e−iks = . −π 2π πs For the linear birth-death process in the previous sec- The convolution can be used to define the derivatives of tion, in Fig. 1B we show that the discrete approximation the discrete probability via given by Eq. (36) with Eq. (31) is in good agreement with the true distribution when truncated after terms of −1 Z ∞ order Ω and shows no oscillations. This agreement is 1/2 −1/2 ∂nP (n, t) = d K(n − Ω[X] − Ω )(Ω ∂)Π(, t), remarkable given the compact form of the solution given −∞ by Eq. (31) and (36). The approximation is almost in- (34) distinguishable from the exact result when the series is −3 ∞ truncated after Ω -terms using Eqs. (25) and (30) in −Sj R and hence it satisfies E P (n, t) = −∞ d K(n−Ω[X]− Eq. (36). We hence conclude that the discrete series −1/2 1/2 −Ω ∂Sj Ω )e Π(, t), as well as γj(n, Ω)P (n, t) = approximates better the underlying distribution of the R ∞ 1/2 1/2 master equation than the continuous approximation. −∞ d K(n−Ω[X]−Ω )γj(Ω[X]+Ω , Ω)Π(, t) for analytic γj. It then follows from the fact that P (n, t) and Ω1/2Π(Ω−1/2(n − Ω[X]), t) have the same characteristic function expansion, that (i) both approximations possess B. The discrete approximation fails for non-linear birth processes the same asymptotic expansion of their moments, and that (ii) they satisfy the same expansion of the master equation. Next, we turn our attention to the analysis of nonlin- For example, to leading order Ω0, Eq. (33) replaces ear birth-death processes, i.e., a process whose propensi- ties depend nonlinearly on the number of molecules. A the conventional continuous LNA estimate, π0 given by Eq. (13), with a discrete approximation particular feature of such processes is that the LNA es- timates for mean and are generally no longer y2 exact, but agree with those of the true distribution only − 2   2   2  1 e 2Σ iy + πΣ iy − πΣ in the limit of large system size [13]. P0(n, t) = √ erf √ − erf √ , 2 2πΣ 2Σ 2Σ Exemplary, we here consider a simple metabolic re- (35) action confined in a small subcellular compartment of volume Ω with substrate input, where y = n − Ω[X], Σ2 = Ωσ2 is the LNA’s estimate for the variance of molecule numbers, and erf is the error x 2 h √2 R −t 0 function defined by erf(x) = π 0 e dt. ∅ −→ S, (37a) −j/2 Associating the Ω -term of Eq. (18) with π in Eq. h1 h3 j S + E )−*− C −→ E. (37b) (33), higher order approximations can now be obtained h2 from The reactions describe the input of substrate molecules S and their catalytic conversion by enzyme species E P (n, t) = P0(n, t) via the enzyme-substrate complex C. The SSE of the 3j average concentrations correcting the macroscopic rate N m X −j/2 X (j)  1/2  equations have been extensively studied [12]. Since our + Ω am −Ω ∂n P0(n, t) j=1 m=1 theory applies to a single species only, we here consider a reduced model in which reaction (37b) is modelled via + O(Ω−(N+1)/2). (36) an effective propensity: this gives S1 = +1, γ1 = Ωk0, 7

FIG. 2. (Color online) Nonlinear birth-death process. A metabolic reaction with Michaelis-Menten kinetics, scheme (37), is studied using the reduced model described in Sec. IV B. The exact stationary distribution is a negative binomial (shown in gray). (A) The discrete SSE approximation given by Eq. (36) with Eq. (25) and (30) is shown in the low molecule number 0 −3/2 −4 regime (k0/k1 = 0.25, 1 molecule on average) when truncated after Ω (blue), Ω (green) and Ω -terms (red dots). We observe that the expansion tends to oscillations and negative values of probability as the truncation order is increased. (B) Similar oscillations are observed for moderate molecule numbers (k0/k1 = 0.9, 27 molecules on average) for the discrete series truncated after Ω0 (blue), Ω−3/2 (green) and Ω−3-terms (red lines). In (C) and (D) we show the approximations corresponding to the same parameters used in (A) and (B), respectively, but obtained using the renormalization procedure given by Eq. (41) with Eq. (42) as described in the main text. The renormalized approximations avoid oscillations and are in excellent agreement with the true probability distributions (gray bars). We note that for the cases (B) and (D) the continuous and discrete approximations give essentially the same result. The remaining parameters are given by Ω = 10 and K = 0.1.

n and S2 = −1, γ2 = Ωk1 n+ΩK . This simplification is V. RENORMALIZATION OF NONLINEAR valid when the enzyme-substrate association is in rapid BIRTH-DEATH PROCESSES equilibrium, which holds when [ET ]  K and h3  h2 where [ET ] is the total enzyme concentration [19]. The Van Kampen’s ansatz, Eq. (2), bears the particularly parameters in the reduced model are related to those in simple interpretation that for linear birth-death processes the developed model by k0 = h1, k1 = h3[ET ], and K =  denotes the fluctuations about the average given by the h2/h1. This reduced master equation is solved exactly solution of the rate equation [X]. As noted in the pre- by a negative binomial distribution [20]. vious example, for nonlinear birth-death processes these estimates are only approximate. Their asymptotic series The system size coefficients are obtained from Eq. expansions will therefore require additional terms that (10), and are given by compensate for the deviations of the LNA from the true concentration mean and variance. It would therefore be m desirable to find an approximation for nonlinear processes m n ∂ [X] Dn = δm,0k0 + (−1) k1 m , (38) that yields more accurate mean and variance than the ∂[X] K + [X] LNA. For instance by rewriting van Kampen’s ansatz as n = [X] + Ω−1/2hi + Ω−1/2¯ . (39) m Ω and Dn,s = 0 for s > 0. In Fig. 2A and 2B, we con- | {z } | {z } sider two parameter sets corresponding to low and mod- mean fluctuations erate numbers of substrate molecules, respectively. We Here, ¯ =  − hi denotes a centered variable that quan- observe that in contrast to the linear case, the discrete tifies the fluctuations about the true average which is a approximation of the nonlinear birth-death process tends priori unknown. These estimates can however be approx- to oscillate with increasing truncation order. This issue imated using the SSE beforehand, and the asymptotic ex- is addressed in the following section. pansion of the distributions can then be performed about 8

2 2 −1 (2) (1) 2 −3/2 these new estimates. This idea is called renormalization σ¯ = σ +Ω (2a2 −(a1 ) )+O(Ω ). Using Eq. (42) and makes use of the fact that the terms correcting mean the renormalized coefficients can be expressed in terms and variances can be summed exactly. As we show in the of the bare ones following the resummation allows to better control the convergence by effectively reducing the number of terms (1) (1) (1) in the summation while at the same time it retains the a¯1 = 0, a¯3 = a3 , (44a) accuracy of the expansion. a¯(2) = 0, a¯(2) = a(2) − a(1)a(1), a¯(2) = a(2). (44b) The system size expansion of the moments, Eq. (24), 2 4 4 1 3 6 6 yields the following estimates for mean and variance of This result can for instance be used to renormalize the the fluctuations stationary solution using the bare coefficients given in Sec. III E, Eqs. (27-28). The non-zero renormalized

N coefficients evaluate to (j) X −j/2 −(N+1)/2 4 2 2 1 0 hi = Ω a1 + O(Ω ), (40a) σ D σ D D a¯(1) = − 1 + 2 + 3 , (45a) j=0 3 6J 6J 18J N 0 2 1 4 2 6 3 X (2) D4 σ D3 σ D2 σ D1 σ¯2 = σ2 + Ω−j/2σ2 + O(Ω−(N+1)/2), (40b) a¯ = − − − − (j) 4 96J 24J 16J 24J j=1 3D1 3σ2D2  − a¯(1) 2 + 1 , 2 (j) (χ) j−1 3 8J 4J respectively, whereσ ¯(j) = 2(a2 − Bj,2({χ!a1 }χ=1)/j!) 1 and Bj,n are the partial Bell polynomials [21]. a¯(2) = (¯a(1))2. (45b) The renormalization procedure amounts to replacing 6 2 3 1/2 2 2 2 y byy ¯ = (n − Ω[X] − Ω hi), Σ by Σ¯ = Ω¯σ in Eq. m ¯ Note that for linear birth-death processes Dn,s = 0 for (35) and associating a new Gaussian P0(n) with these s > 0 and m > 1, and hence the above equations reduce estimates. The renormalized expansion is then given by to Eqs. (27-28).

P (n, t) = P¯ (n, t) 0 A. The renormalized approximation performs well 3j N m for nonlinear birth-death processes X −j/2 X (j)  1/2  ¯ + Ω a¯m −Ω ∂n P0(n, t) j=1 m=1 For the metabolic reaction (37), mean and variance + O(Ω−(N+1)/2), (41) can be obtained to be hi = Ω−1/2ς + O(Ω−2),σ ¯2 = σ2 + Ω−1ς(ς + 1) + O(Ω−2), where ς = [X]/K is the where the renormalized coefficients can be calculated reduced substrate concentration and σ2 = Kς(ς + 1). from the bare ones using Substituting now Eq. (38) into Eqs. (45), we obtain the expansion coefficients

j 3k 2 X X (j−k) (1) σ a¯(j) = a(k)κ , (42) a¯ = (2ς + 1), (46a) m n m−n 3 6 n=0 k=0 σ2 1 a¯(2) = (6ς(ς + 1) + 1) , a¯(2) = (¯a(1))2, (46b) and 4 24 6 2 3 which determine the renormalized series expansion to or- −1 bj/2c n−m   der Ω . Using Eq. (25), (38) and (42) we can give the (n) 1 X X n κ = (−1)(j+m) × next order terms to order Ω−3/2 analytically j n! k m=0 k=j−2m (1) (1) n o χ!  (3) a¯3 (3) a¯3 B χ!a(χ) B σ¯2 , (43) a¯3 = , a¯5 = (12ς(ς + 1) + 1), k,j−2m 1 n−k,m 2 (χ) K 20 (3) (1) (2) (3) 1 (1) 3 a¯7 =a ¯3 a¯4 , a¯9 = (¯a3 ) . (46c) where again Bk,n({xχ}) denote the partial Bell polyno- 6 mials [21]. The result is verified at the end of this section. In Fig. 2C and 2D we compare the renormalized approx- Note that the renormalized series has generally less non- imation given by Eq. (41) with the respective bare ap- (j) (j) zero coefficients since by constructiona ¯1 =a ¯2 = 0. proximations in Fig. 2A and 2B. We observe that the Note that for linear birth-processes, mean and variance renormalization technique avoids oscillations and even are exact to order Ω0 (LNA), and hence for this case the simple analytical approximation given by Eqs. (46) is expansion (36) coincides with Eq. (41). in reasonable agreement with the exact result. We note For example, truncating after Ω−1-terms, from Eq. that the asymptotic approximations shown in C and D −1/2 (1) −3/2 (40) it follows that hi = Ω a1 + O(Ω ) and are almost indistinguishable for higher truncation orders. 9

B. Proof of the renormalization formula and similarly for the second term

The renormalized coefficients can in principle be ob- ∞ !m 1 1 X tained by matching the expansions given by Eq. (36) and Ω−n/2σ¯2 m! 2 (n) (41) via their characteristic functions. For convenience n=1 we consider the characteristic function of the series (18) ∞ n X Ω−n/2 X χ!  = δ B σ¯2 . (53) n! m,k n,k 2 (χ)  ∞ 3j  n=0 k=0 X −j/2 X (j) n G(k) = G0(k) 1 + Ω an (ik)  , (47) Using the above expansions in Eq. (51) and rearranging j=1 n=1 −1/2 (n) in powers of Ω , Eq. (43) for the coefficients κj −(kσ)2/2 with G0(k) = e being the characteristic function follows. solution of the LNA π0() and we have omitted the ex- Finally, one associates with the centered variable ¯ = 2 plicit time-dependence to ease the notation. We are now  − hi, a Gaussianπ ¯0(¯) with varianceσ ¯ . It then looking for a different expansion with corrected estimates follows from inverting Eq. (48) that Π(¯) =π ¯0(¯) + PN −j/2 P3j (j) −(N+1)/2 for the mean and variance. j=1 Ω n=1 a¯n ψn (¯)π ¯0(¯) + O(Ω ). As- −j/2 sociating now the Ω -term of this equation with πj in  ∞ 3j  Eq. (33), the discrete series for P (n, t) given by Eq. (41) ¯ ¯ X −j/2 X (j) n G(k) = G0(k) 1 + Ω a¯n (ik)  , (48) follows. j=1 n=1

ikhi −(kσ¯)2/2 Note that G¯0(k) = e e is the characteristic VI. APPLICATION function for a Gaussian with mean hi 2 and varianceσ ¯ given by Eqs. (40). The models studied so far have been useful to develop Equating now Eq. (47) and (48), we find the method. It remains however to be demonstrated that it remains accurate in cases where analytical solution is ∞ 3j not feasible, as for instance, for out-of-steady-state and X −j/2 X (j) n 1+ Ω a¯n (ik) non-detailed balance systems. We here consider the syn- j=1 n=1 thesis of a protein P which is degraded through an en-  ∞ 3j  zyme G0(k) X −j/2 X (j) n = 1 + Ω an (ik)  . (49) G¯0(k) j=1 n=1 h0 h1 h2 ∅ −→ M −→ ∅,M −→ M + P, (54a) Expanding the prefactor in the above equation in powers h3 h5 of k and then in Ω, we have P + E )−*− C −→ E, (54b) h4

∞ ∞ 2n where M denotes the transcript, E the enzyme and C G0(k) X X X (n) = (ik)jκ = Ω−n/2 (ik)jκ , (50) complex species as has been studied in Ref. [22]. Since G¯ (k) j j 0 j=0 n=0 j=0 our theory applies only to a single species, we consider the limiting case in which the protein dynamics represents from which Eq. (42) follows, which expresses the new co- (j) (j) the slowest timescale of the system. It has be shown [23] efficientsa ¯n in terms of the bare ones an . It remains to (n) that when species M is degraded much faster than the derive an explicit expression for the κj . The expansion protein P , the protein synthesis (54a) reduces to the tran- in powers of (ik) yields sition S1 = +z, γ1 = Ωk0 in which z is a random variable z 1  b  following the geometric distribution ϕ(z) = 1+b 1+b  2 2 m bj/2c j−2m σ¯ −σ X hi 2 with average b, which is called the burst approximation. κ = (−1)(j+m) . (51) j (j − 2m)! m! Similarly to the metabolic reaction studied in Sec. IV, m=0 the enzymatic degradation process (54b) can be reduced n We now expand the first term in inverse powers of Ω to S2 = −1, γ2 = Ωk1 ΩK+n with a nonlinear depen- using the partial Bell polynomials dence on the protein number n. The master equation describing the protein number is then given by ∞ !j−2m 1 X −n/2 (n) ∞ Ω a1 (j − 2m)! d X −z n=1 P (n) =Ω (E − 1)k ϕ(z)P (n) dt 0 ∞ Ω−n/2 n n o z=0 X X (χ) n = δj−2m,kBn,k χ!a1 , (52) +1 n! + Ω(E − 1)k1 P (n). (55) n=1 k=0 ΩK + n 10

and are given by

∂m [X] Dm = δ k hzni + (−1)nk , (56) n m,0 0 ϕ 1 ∂[X]m K + [X]

m n P∞ n and Dn,s = 0 for s > 0, where hz iϕ = z=0 z ϕ(z) = 1 b 1+b Li−n( 1+b ) denotes the average over the geometric distribution in terms of the polylogarithm function [24]. The deterministic equation is given by

d[X] k [X] = k b − 1 , (57) dt 0 K + [X]

0 which follows from the expression for D1. Using the Ja- 1 0 cobian J = D1 and D2 in Eq. (14), we find that the LNA variance obeys

∂σ2 2k K k [X] = − 1 σ2 + k b(1 + 2b) + 1 . (58) ∂t ([X] + K)2 0 K + [X]

The ODEs given by Eq. (57) and (58) are integrated numerically and the solution is used in Eq. (35) from which the leading order approximation follows. Higher FIG. 3. (Color online) Predicting transient distributions order approximations are now be obtained by using Eq. of gene expression. The dynamics of protein synthesis (56) in (22) which govern the time-evolution of the coef- with enzymatic degradation, scheme (54), is studied using (j) the burst approximation (55). (A) We compare the time- ficients am (t) and using the result in Eq. (41) and (42). dependence of the renormalized discrete approximations to We assume deterministic initial conditions with zero pro- (j) exact stochastic simulations at times 1, 2, and 14min. The teins meaning am (0) = δm,0δj,0. In Fig. 3A we compare overall shape (mode, skewness, distribution tails) of the sim- the time-evolution obtained by the leading order approx- ulated distributions (bars) is in excellent agreement with the −3 imation P0 and Eq. (41) truncated after the Ω -term. series approximation when truncated after Ω−3-terms (solid 0 The latter distributions are in excellent agreement with lines) but not when only Ω are taken into account (dashed the distributions sampled using the stochastic simulation lines). This agreement is also observed for the first two mo- ments shown in the inset: while the Ω−3-approximation (blue algorithm [25]. In particular, unlike the leading order solid line) agrees with the moment dynamics of the simu- approximation, these describe well mode, skewness, and lated distributions (dots) of the reduced model (55), the Ω0- tails of the distribution. We note that also the mean approximation underestimates the mean (gray solid line) and and variance of these distribution approximations are in variance by 25%. The area within one standard deviation of excellent agreement as verified in inset of Fig. 3A. the mean obtained from simulations is shown in blue, the Despite the overall good agreement, in Fig. 3B we boundary obtained from the approximations are shown as show that there are discrepancies at very short times dashed lines (Ω0 grey, Ω−3 blue). (B) Despite the good agree- where and, again, the distribution approximations tend ment shown in (A) we found that at very short times (12s – −3 to oscillations. Motivated by this numerical observation, blue solid line) the series truncated after Ω -terms tends we speculate that this behavior of the expansion is due a to oscillations which quickly disappear for later times (24s – temporal boundary layer as commonly observed in singu- green, 48s – red solid line). See main text for discussion. Pa- −1 −1 lar perturbation expansions [26]. Theoretically, the layer rameters are k0Ω = 15min , k1Ω = 100min , KΩ = 20, Ω = 100, and b = 5. Histograms were obtained from 10, 000 must be located at times of the same order as the expan- −1/2 stochastic simulations. sion parameter, i.e., t = (ΩK) min ≈ 13s, coinciding with the simulation in Fig. 3B. This suggests that our approach does only describe the outer solution. Further The relation between the parameters in the reduced and analysis would be required to investigate also the inner the developed model are given by k0 = h0h2/h1, b = solution which is beyond the scope of this article. h2/h1, k1 = h5[ET ], K = h5/h3, where [ET ] denotes the total enzyme concentration. This description involves countably many reactions: one for the degradation of VII. DISCUSSION the protein, and one for each value of z. Therefore, the reactions cannot obey detailed balance in steady state. We have here presented an approximate solution The system size coefficients now follow from Eq. (10), method for the probability distribution of the master 11 equation. The solution is given in terms of an asymp- in terms of the derivatives of a centered Gaussian π0 with totic series expansion that can be truncated systemati- variance σ2, cally to any desired order in the inverse system size. For biochemical systems with large numbers of molecules, we    H = π−1()(−σ∂ )nπ (). (A1) have derived a continuous series approximation that ex- n σ 0  0 tends van Kampen’s LNA to higher orders in the SSE. In low molecule number conditions, we have found that this An explicit formula is continuous approximation becomes inaccurate. Instead, in most practical situations the prescribed discrete distri- bn/2c   n−2k bution approximations incorporating higher order terms    X n k    Hn = (−1) (2k − 1)!! . (A2) in the SSE better capture the underlying solution of the σ 2k σ k=0 master equation. While the terms to order Ω−1 have been given explicitly, we found that for the examples studied These functions are orthogonal −3 −4 1 R ∞     here up to Ω or Ω -terms had to be taken into account n! −∞ d Hm σ Hn σ π0() = δnm, with respect to accurately characterize these non-Gaussian distribu- to the Gaussian measure π0. The derivative satisfies tions. We note, however, that the asymptotic expansion cannot generally guarantee the positivity of the probabil- m    n!    ity law. These undulations are particularly pronounced (σ∂) Hn = Hn−m . (A3) σ (n − m)! σ in the short-time behavior of the expansion studied in Sec. VI, which our theory does not describe. Since these polynomials are complete, every function f() Previous of solving the master equation have in L2(R, π0) (not necessarily positive) can be expanded either been numerical in nature [27] or have focused on P∞   as f() = n=0 bnHn σ π0(), where the coefficients the inverse problem, i.e., reconstruction of the proba- 1 R   are given by bn = n! d Hn σ f(). We note because bility density from the moments. While a numerical H   = 1 and π is normalized, we must have b = 1 if solution for the master equation of a single species is 0 σ 0 0 R d f() = 1. rather straightforward, we expect our procedure to be- come computationally advantageous when generalized to the multivariate case where numerical solution is usually Appendix B: Explicit derivation of Eq. (23) and the prohibitive because of combinatorial explosion. properties of the expansion coefficients Methods based on moments typically require approx- imations such as moment closure [16] and also require Changing variables  = xσ and letting I˜αβ = the prior assumption of the first few moments contain- mn σα−β+m−nIαβ , the integral (20) can be written ing all information on the probability distribution. Con- mn versely, using the system size expansion, we have here 1 Z I˜αβ = dxH (x)(−∂ )αxβH (x) π (x), obtained the probability distribution directly from the mn n!α!β! n x m 0 master equation without the need to resort to moments. (A1) This method enjoys the particular advantage over previ- where π0(x) is a centered Gaussian with unit variance. ous ones that the first few terms of this expansion can Using partial integration, property (A3), and the relation be written down explicitly as a function of the rate con- stants and for any number of reactions. For small models min(α,β) we have demonstrated that the procedure leads to par- X Hα+β−2s(x) H (x)H (x) = α!β! , (A2) ticularly simple expressions for the non-Gaussian distri- α β s!(α − s)!(β − s)! butions. This development could prove particularly valu- s=0 able for parameter estimation of biochemical reactions in given in Ref. [28], one obtains living cells.

min(n−α,m)  R β 1 X m dx x Hm+n−α−2s(x)π0(x) I˜αβ = . ACKNOWLEDGMENTS mn α!β! s (n − α − s)! s=0 (A3) It is a pleasure to thank Claudia Cianci and David Schnoerr for careful reading of the manuscript. The remaining integral can now be evaluated as the mo- ments of the unit Gaussian which yields

Appendix A: Useful properties of the Hermite Z b! Z polynomials dx xbH (x)π (x) = dx xb−aπ (x) a 0 (b − a)! 0 b! We here briefly review some properties of the Hermite = (b − a − 1)!! . (A4) orthogonal polynomials. The polynomials can be defined (b − a)! 12 for even (b − a) ≥ 0 and zero otherwise. Explicitly, the matrix elements are given by bn/2c 1 X  n  hn−2ki b = (−1)k (2k − 1)!!. (A1) n n! 2k σn−2k min(n−α,m) k=0 1 X m I˜αβ = × mn α! s Assuming now that the moments can be expanded in a s=0 series in powers of Ω, i.e., (β + α + 2s − (m + n) − 1)!! , (A5) (β + α + 2s − (m + n))!(n − α − s)! N X hβi = Ω−j/2[β] + O(Ω−(N+1)/2), (A2) for even (α + β) − (m + n) and zero otherwise. Note that j j=0 the above quantity is strictly positive. Note also that the argument of the double factorial is taken to be positive the bn can be matched to the coefficients an in Eq. and hence the summation is non-zero only if α + β + n PN −j/2 (j) −(N+1)/2 (18) using σ bn = j=0 Ω an +O(Ω ), from 2 min(n − α, m) ≥ m + n and hence for even β = 2k we which one obtains have n = m + α ± 2l, while for odd β = (2k + 1) we have n = m + α ± (2l + 1), with l = 0, . . . , k. bn/2c The integral formula can be used to verify two impor- 1 X  n  a(j) = (−σ2)k(2k − 1)!![n−2k] , (A3) tant properties of the solution of Eq. (22) given determin- n n! 2k j istic initial conditions: (i) We have Nj = 3j and hence k=0 Eq. (22) indeed yields a finite number of equations. (ii) 0 (j) with [ ]j = δj,0. The above formula relates the expan- The coefficients an for which (n + j) is odd vanish at all sion of the moments to the expansion of distribution func- times. tions. It is now evident that the system size expansion of To verify property (i), let Nj be the index of the highest the distribution can be constructed from the system size eigenfunction required to order Ω−j/2. Using Eq. (22) expansion for a finite set of moments. one can show that a(j) ∼ a(j−1)Ip,3−p for p ∈ {1, 2, 3}. Specifically, to order Ω−1/2 the non-zero coefficients Nj Nj−1 Nj−1,Nj By virtue of the properties given after Eq. (23), we find evaluate to Nj = Nj−1 + 3. Since for deterministic initial conditions (1) (1) 1 3 3 2  we have N = 0, it follows that N = 3j. a = []1, a = [ ] − 3σ []1 (A4) 0 j 1 3 3! 1 Finally, we verify property (ii). To the summation in Eq. (22) there contribute only terms for which while the coefficients to order Ω−1 are given by p,k−p−2(s−1) Imn is non-zero. Hence, by the condition given (2) 1 2 (2) 1 4 2 2  after Eq. (23), k − (m + n) is an even number. Consider- a2 = [ ]2, a4 = [ ]2 − 6σ [ ]2 , (j) 2 4! ing the equation for an for which n+j is even, it follows 1 that in the summation on the right hand side of Eq. (22) a(2) = 45σ4[2] − 15σ2[4] + [6]  . (A5) 6 6! 2 2 2 there appear only coefficients for which m + (j − k) is even. Conversely, for n + j being odd then same holds A different series is obtained using the Edgeworth expan- (j) for m + (j − k). Hence the pairs of equations for an sion which instead of using the system size expansion of for which (j + n) is even or odd are mutually uncoupled. the moments, Eq. (A2), proceeds by scaling the cumu- For deterministic initial conditions, only terms with j +n lants by a size parameter. even differ from zero initially from which the result fol- lows.

Appendix C: Solution to the problem of moments using the system size expansion

Having obtained the moment expansion in terms of the (j) coefficients an , it would be desirable to invert this rela- tion and the coefficients in terms of the expansion of the moments. This can be derived using the completeness of the Hermite polynomials, and writing the probabil- P∞   ity density as Π() = n=0 bnHn σ π0(), where the 1 R   bn = n! dHn σ Π() can be expressed in terms of the moments using Eq. (A2), as follows 13

[1] T. Jahnke and W. Huisinga, J Math Biol 54, 1 (2007). 25, 12 (2015). [2] H. Haken, Phys Lett A 46, 443 (1974); N. G. Van Kam- [17] C. Cianci, F. Di Patti, and D. Fanelli, Europhys Lett pen, ibid. 59, 333 (1976). 96, 50011 (2011). [3] R. M. Mazo, J Chem Phys 62, 4244 (1975); J. Peccoud [18] T. M. Cover and J. A. Thomas, Elements of informa- and B. Ycart, Theor Popul Biol 48, 222 (1995); P. Bokes, tion theory, 2nd ed. (John Wiley & Sons, Hoboken, New J. R. King, A. T. Wood, and M. Loose, J Math Biol 64, Yersey, 2012). 829 (2012). [19] P. Thomas, A. V. Straube, and R. Grima, J Chem Phys [4] R. G¨ortzand D. Walls, Z Phys B Con Mat 25, 423 (1976); 135, 181103 (2011); K. R. Sanft, D. T. Gillespie, and G. Haag and P. H¨anggi, ibid. 34, 411 (1979). L. R. Petzold, IET Syst Biol 5, 58 (2011). [5] D. Schnoerr, G. Sanguinetti, and R. Grima, J Chem [20] J. Paulsson, O. G. Berg, and M. Ehrenberg, Proc Natl Phys 141, 024103 (2014). Acad Sci 97, 7148 (2000). [6] N. G. Van Kampen, Adv Chem Phys 34, 245 (1976); [21] L. Comtet, Advanced Combinatorics (Springer, Nether- Stochastic Processes in Physics and Chemistry., 3rd ed. lands, 1974); The partial Bell polynomials are defined as  jn−k+1 (Elsevier, Amsterdam, 1997). P0 n! x1 j1 xn−k+1 Bn,k({xχ}) = .. , [7] J. Elf and M. Ehrenberg, Genome Res 13, 2475 (2003). j1!..jn−k+1! 1! (n−k+1)! P0 [8] A. Melbinger, L. Reese, and E. Frey, Phys Rev Lett 108, where the summation is such that j1 +...+jn−k+1 = 258104 (2012); J. Szavits-Nossan, K. Eden, R. J. Morris, k and j1 + 2j2 + ... + (n − k + 1)jn−k+1 = n and {xχ} C. E. MacPhee, M. R. Evans, and R. J. Allen, 113, denotes the sequence (x1, x2, x3,...). These are available 098101 (2014). in Mathematica via the function BellY. [9] G. Rozhnova and A. Nunes, Phys Rev E 79, 041922 [22] P. Thomas, H. Matuschek, and R. Grima, Computation (2009). of biochemical pathway fluctuations beyond the linear [10] M. Aoki, Modeling aggregate behavior and fluctuations noise approximation using iNA. in Bioinformatics and in economics (Cambridge University Press, Cambridge, Biomedicine (BIBM), 2012 IEEE International Confer- 2001). ence on (IEEE, 2012) pp. 1–5. [11] T. Heskes, J Phys A: Math Gen 27, 5145 (1994). [23] V. Shahrezaei and P. S. Swain, Proc Natl Acad Sci 105, [12] R. Grima, Phys Rev Lett 102, 218103 (2009); 17256 (2008). P∞ xk P. Thomas, A. V. Straube, and R. Grima, J Chem Phys [24] The polylogarithm is defined by Lis(x) = k=1 ks and 133, 195101 (2010). implemeted in Mathematica by PolyLog[x, s]. [13] R. Grima, P. Thomas, and A. V. Straube, J Chem Phys [25] D. T. Gillespie, J Phys Chem 81, 2340 (1977); Ex- 135, 084103 (2011). act realizations of the process described by Eq. (55) are [14] C. Cianci, F. Di Patti, D. Fanelli, and L. Barletti, obtained using the Gillespie’s algorithm with the incre- Eur Phys J Special Topics 212, 5 (2012); P. Thomas, ments of the synthesis reaction being replaced by inde- C. Fleck, R. Grima, and N. Popovic, J Phys A: Math pendently and identically geometrically distributed inte- Theor 47, 455007 (2014). gers of mean b. [15] S. Engblom, Appl Math Comput 180, 498 (2006); [26] F. Verhulst, Methods and applications of singular pertur- R. Grima, J Chem Phys 136, 154105 (2012); A. Ale, bations (Springer, New York, 2006). P. Kirk, and M. P. Stumpf, 138, 174101 (2013). [27] B. Munsky and M. Khammash, J Chem Phys 124, [16] V. Sotiropoulos and Y. N. Kaznessis, Chem Eng Sci 66, 044104 (2006). 268 (2011); P. Smadbeck and Y. N. Kaznessis, Proc Natl [28] N. N. Lebedev, Special functions and their applications Acad Sci 110, 14261 (2013); A. Andreychenko, L. Mi- (Dover Publications, New York, 1972). keev, and V. Wolf, ACM Trans Model Comput Simul