bioRxiv preprint doi: https://doi.org/10.1101/2020.01.05.895359; this version posted January 6, 2020. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under aCC-BY-NC-ND 4.0 International license.
Exactly solvable models of stochastic gene expression Lucy Ham,1, a) David Schnoerr,2, b) Rowan D. Brackston,2, c) and Michael P. H. Stumpf1, 2, d) 1)School of BioSciences and School of Mathematics and Statistics, University of Melbourne, Parkville VIC 3010, Australia 2)Dept. Life Sciences, Imperial College London, SW7 2AZ, UK (Dated: 6 January 2020) Stochastic models are key to understanding the intricate dynamics of gene expression. But the simplest models which only account for e.g. active and inactive states of a gene fail to capture common observations in both prokaryotic and eukaryotic organisms. Here we consider multistate models of gene expression which generalise the canonical Telegraph process, and are capable of capturing the joint effects of e.g. transcription factors, heterochromatin state and DNA accessibility (or, in prokaryotes, Sigma-factor activity) on transcript abundance. We propose two approaches for solving classes of these generalised systems. The first approach offers a fresh perspective on a general class of multistate models, and allows us to “decompose" more complicated systems into simpler processes, each of which can be solved analytically. This enables us to obtain a solution of any model from this class. We further show that these models cannot have a heavy-tailed distribution in the absence of extrinsic noise. Next, we develop an approximation method based on a power series expansion of the stationary distribution for an even broader class of multistate models of gene transcription. The combination of analytical and computational solutions for these realistic gene expression models also holds the potential to design synthetic systems, and control the behaviour of naturally evolved gene expression systems, e.g. in guiding cell-fate decisions.
I. INTRODUCTION ing feedback. Recent work has shown that, while the Tele- graph model is able to capture some degree of the variability The need for models in cellular biology is ever-increasing. in gene expression levels, it fails to explain the large vari- Technological and experimental advances have led not only to ability and over-dispersion in the tails seen in many experi- 11 progress in our understanding of fundamental processes, but mental datasets . Furthermore, it has become evident that also to the provision of a great wealth of data. This presents gene promoters can be in multiple states and may involve in- enormous opportunities, but also requires further development teractions between a number of regulatory factors, leading to of the mathematical and computational tools for the extraction a different mRNA synthesis rate for each state; such states and analysis of the underlying biological information. may be associated with the presence of multiple copies of a The development of mathematical models for the complex gene in a genome, or with the key steps involved in chromatin 12–15 16 17 dynamics of gene expression is of particular interest. The remodelling , DNA looping or supercoiling . Mathe- stochastic nature of the transcriptional process generates in- matical models—and accompanying analytic or approximate tracellular noise, and results in significant heterogeneity be- solutions—that capture the kind of data arising from these tween cells; a phenomenon that has been widely observed in more complex multistate dynamics are now essential. mRNA copy number distributions, even for otherwise identi- Despite its deficiencies, the Telegraph model remains a use- cal cells subject to homogeneous conditions1–6. Modelling is ful framework around which more complicated models can be key to understanding and predicting the dynamics of the pro- constructed and, as we will show here, provides a foundation cess, and subsequently, to quantifying this observed variabil- from which to develop further analytical results. A number ity. The Telegraph model, as introduced by Ko et al.7 in 1991, of extensions and modifications of the Telegraph model have is the canonical model for gene expression and explains some emerged, the simplest of which is perhaps the extension to al- of the variability observed in mRNA copy number distribu- low for a second basal rate of transcription—the “leaky" Tele- tions. This mathematical model treats gene promoter activity graph model11,18,19. A further extension incorporating the ad- as a two-state system and has the advantage of a tractable sta- ditional effects of extrinsic noise is given in Refs. 11, 20, and tionary mRNA distribution4,8,9, enabling insights into limiting 21. Several recent studies of gene expression have shown that cases of the system10 and the behaviour of the distribution in the waiting time distribution in the inactive state for the gene general. promoter is non-exponential22–24, leading to the proposal of a With the advancement of single-cell experiments, the de- number of “refractory” models25–28. A distinguishing feature ficiencies of the Telegraph model are increasingly apparent. of these models is that the gene promoter can transcribe only The model does not account for more complex control mech- in the active state and, after each active period, has to progress anisms, nor more complex gene regulatory networks involv- through a number of inactive states before returning to the active state. An exact solution for the stationary distribution of a general refractory model is derived in Ref. 26, building upon the existing work of Refs. 25 and 27. In addition to a)Electronic mail: [email protected] direct extensions of the Telegraph model, many variants of b)Electronic mail: [email protected] the model have been proposed, some of which have been ad- c)Electronic mail: [email protected] ditionally solved analytically. Examples include a two-state d)Electronic mail: [email protected] model incorporating a gene regulatory feedback loop29,30; a bioRxiv preprint doi: https://doi.org/10.1101/2020.01.05.895359; this version posted January 6, 2020. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under aCC-BY-NC-ND 4.0 International license. 2 three-stage model accounting for the discrepancy in abun- by Peccoud and Ycart8 in 1995, and provides an alternative dances at the mRNA and protein level31; and more recently, a approximative approach to solving master equations. The multi-scale Telegraph model incorporating the details of poly- method is reminiscent of Taylor’s solution to differential equa- merase dynamics is solved in Ref. 32. Some approximation tions39, using the differential equations to construct a recur- methods have been developed to solve more general multi- rence relation for derivatives of the solution. We show how state models19,33. our approach can be used to recover the full solution to the Obtaining analytical solutions to models of gene transcrip- previously mentioned multistate model to arbitrary numerical tion is a challenging problem, and there currently exist only precision. However, the method is also applicable to more a few classes of systems and a handful of special cases for complex systems for which no analytic solutions exist in the which an analytic solution is known. A number of these have literature. When an analytical solution is known, we are able just been listed; see Ref. 34 for more information. to benchmark the computational speed of the two approaches. Existing methods for solving such models adopt a chemi- For some systems, the approximation method is in fact faster cal master equation approach and typically employ a gener- than the analytical solution, as the latter typically requires ating function method, in which the original master equation multiple numerical calculations of the confluent hypergeomet- is transformed into a finite system of differential equations35. ric function. When an exact solution to the differential equations defining This article is structured as follows. We begin in Section II the generating function can be found, the analytical solution by presenting a general class of multistate models that cover to the model can in principle be recovered by way of succes- many examples of gene transcription models used in the anal- sive derivatives. A major complication in finding analytical ysis of experimental data. We explain how these multistate solutions in this way is that typically even slight changes to processes can be decomposed into simple processes acting in- the original model lead to considerably more complicated sys- dependently, and how these can be solved analytically. We tems of differential equations. An additional challenge is that complete the section by giving some examples of previously- analytic solutions to models usually involve hypergeometric unsolved models. In Section III, we consider the effects of ex- functions, which can lead to numerical difficulties in further trinsic noise, and show that the main results of Ref. 11 can be computational and statistical analysis. Due to the lack of suc- extended to these models. Next, in Section IV, we outline the cess in the search for analytic solutions, a significant amount strategy for turning differential equations into recurrence rela- of effort has been on stochastic simulation36 and approxima- tions that can be computationally iterated to produce solutions tion methods34. However, some of these approaches can be of arbitrary accuracy. We then utilise the approach to solve computationally expensive, making them unviable for larger some examples of gene transcription models for which no an- systems, while most existing approximation methods are ad- alytic solutions are currently known. Finally, we assess the hoc approximations that do not allow to control the approxi- accuracy and computational efficiency of our approach. Com- mation error. To address these obstacles, we present two alter- parisons are made between our approximative approach, ana- native methods for solving certain classes of models of gene lytical solutions, and the stochastic simulation algorithm40. transcription. The first approach offers a novel conceptualisation of a class of multistate models for gene transcription and allows us to extend a number of known solutions for relatively basic II. DECOMPOSING MULTISTATE PROCESSES processes to more complicated systems. For a certain class of models, we are able to abstractly decompose the more com- The standard method of solving models of gene transcrip- plicated system as the independent sum of simpler processes, tion is to transform the chemical master equation into a sys- each of which is amenable to analytic solution. The distribu- tem of ordinary differential equations by way of generating tion of the overall system can then be captured by way of a functions. We will observe that there are certain situations convolution of the simpler distributions. in which the need to solve these differential equations can be The method applies to a wide range of models of stochas- avoided, by decomposing into sums of independent simpler tic gene expression. Examples include systems with multiple processes, each of which can be treated analytically. These copies of a single gene in a genome, or systems with multiple can then be recombined to solve the original system. promoters or enhancers for a single gene, each independent, We now briefly outline the content of this section. After first and contributing additively to the transcription rate. Here we defining the leaky Telegraph model, we present a generalisa- will provide analytical expressions for a multistate process tion of this model that allows for a finite number of discrete consisting of a finite, exponential number of activity states, activity states. As motivation, we will apply our decomposi- each with a particular mRNA transcription rate. Similar theo- tion method to the leaky Telegraph model, which has a known retical models have been employed to predictively model mul- exact solution for the steady-state mRNA copy number distri- tistate processes37,38, however, the analyses in these are done bution. We will demonstrate how efficiently and easily this so- numerically. Thus, our results enable exact solutions to some lution can be derived using the new approach. A more formal natural examples of multistate gene action, with an unlimited exposition of the method will then be given. Novel results are (though exponential) number of discrete states. presented in the remaining subsections, where we describe in The second method draws as inspiration from the origi- detail how this approach can be used to solve more complex, nal derivation of the solution to the Telegraph model, given previously unsolved systems. bioRxiv preprint doi: https://doi.org/10.1101/2020.01.05.895359; this version posted January 6, 2020. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under aCC-BY-NC-ND 4.0 International license. 3
µ Trans. rate KI KI I A K + mRNA A ≡ K −K KA − KI A I λ KI KA δ time
∅ FIG. 2. Decomposing the leaky Telegraph process (left) into the in- dependent sum of a constitutive and Telegraph process (right). FIG. 1. A schematic of the leaky Telegraph model. The nodes rep- resent the state of the gene, labelled I and A for inactive and active, respectively. The mRNA are represented as blue wiggly lines. The control features of the other so that it transcribes constitu- parameters λ and µ are the rates at which the gene transitions be- tively at a constant rate of K0 := KI. The other copy of the tween the two states I and A; the parameters KI and KA are the tran- gene is governed by a Telegraph process with transcription scription rate parameters for I and A, respectively; the degradation rate K := K − K when active (and 0 when inactive). Note rate is denoted by δ. 1 A I that when the second gene is active, the combined transcrip- tion rate is K0 + K1 = KA, so this model is indistinguishable from the leaky Telegraph model; see Fig. 2 for a pictorial rep- A. The leaky Telegraph model resentation of this situation. The new interpretation is more easily modelled, as it is simply two independent systems, both The leaky Telegraph model has been considered in a num- of which are simpler than the original. ber of previous studies such as Refs. 11, 18, 19, and 25, and We are able to make this observation more precise by way incorporates the well known phenomenon of promotor leak- of analysis of the corresponding probability generating func- age and basal gene expression41–43. In this model, the gene ∞ n tions, defined as g(z) = ∑n=0 z p(n), where p(n) is the sta- is assumed to transition between two states, active (A) and in- tionary probability distribution. Let P be a random variable active (I), with rates λ and µ, respectively. Transcription of that counts the copy number of a constitutive process with mRNA is modelled as a Poisson process, occurring at a basal constant rate K0, which is well-known to be Pois(K0) dis- background rate of KI when inactive, and at a rate KA > KI tributed at stationarity45–47, and let T be an independent ran- when active. Degradation occurs as a first-order Poisson pro- dom variable distributed by a Telegraph distribution with rates cess with rate δ, and is assumed to be independent of the gene λ, µ,K1 = KA − KI. Then the total copy number X for the bi- promoter state. A schematic of the system is given in Fig. 1. nary system is simply the independent sum of P and T.A Observe that the standard Telegraph model corresponds to the standard result in probability theory says that the probabil- special case for KI = 0. ity generating function of a sum of two independent random We recall here that the mRNA copy number from the leaky variables is the product of the two component probability gen- Telegraph process, as shown in Ref. 11, has the following erating functions48 (Section 3.6(5)). The probability generat- steady-state distribution: ing function for the random variable P is eK0(z−1) while the probability generating function for the Telegraph distributed n (r) 8 1 n µ T is 1F1 (λ, µ + λ,K1(z − 1)) . So the probability generating p˜ (n) = Kn−re−KA (K − K )r L ∑ A I A (r) function for the overall system is: n! r=0 r (µ + λ) eK0(z−1) F ( , + ,K (z − 1)). (3) 1F1(µ + r, µ + λ + r,−(KI − KA)) , (1) 1 1 λ µ λ 1 This solution, (3), should match the probability generating wherep ˜L(n) is the probability of observing n mRNA function yielding (1). From Ref. 11, the probability gener- molecules in the system, 1F1 is the confluent hypergeomet- ating function for the leaky Telegraph model is given by: ric function44, and, for real number x and positive integer n, (n) KA(z−1) the notation x abbreviates the rising factorial of x. Note g(z) = e 1F1 (µ, µ + λ,(KI − KA)(z − 1)). (4) also that the rates are scaled so that δ = 1. When K0 = 0, we can recover the following well-known expression for the Multiplying (4) though by eKI −KI = 1, and applying Kum- Telegraph process in the steady-state: mer’s transformation for the confluent hypergeometric func- tion, we obtain (3), noting that K0 = KI and K1 = KA − KI. n (n) KAλ p˜T (n) = 1F1(λ + n, µ + λ + n,−KA). (2) n!(µ + λ)(n) B. The 2m-multistate model Throughout this work, we will refer to the probability mass functionsp ˜L(n) andp ˜T (n) as the “leaky Telegraph distribu- We now extend the leaky Telegraph process of the previ- tion” and “Telegraph distribution”, respectively. ous section to allow for a finite number of discrete activity Another way to arrive at the same overall behaviour as the states. For a non-negative integer m, consider a gene with leaky Telegraph process is to consider a system consisting of m distinct activating enhancers a0,...,am−1, where for each two independent copies of the same gene, one lacking the i ∈ {0,...,m − 1}, the enhancer ai, independently, is either bioRxiv preprint doi: https://doi.org/10.1101/2020.01.05.895359; this version posted January 6, 2020. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under aCC-BY-NC-ND 4.0 International license. 4
m enhancers am−1 ... a1 a0
Each ai ∈ {0,1} 0 1 ... 0 1 0 1
m 2 states s2m−1 = (1,1,...,1) ... s1 = (0,0,...,1) s0 = (0,0,...,0)
Ki = KB + k · si K2m−1 K0 KB
δ ∅
m FIG. 3. A diagrammatic representation of the 2 -multistate model, for m ∈ N∪{0}. There are m enhancers, a0,...,am−1, where each enhancer m is either unbound (0) or bound (1): so ai ∈ {0,1}. This leads to 2 possible activity states (am−1,...,a0) for the gene; the state s0 = (0,0,...,0) corresponds to when all of the ai are unbound, the state s1 = (0,0,...,1) corresponds to when a0 is bound, and all other ai are unbound, while m the state s2m−1 = (1,1,...,1) corresponds to when all of the ai are bound. The gene transitions between these 2 states, where switching events between adjacent states si and s j occur at rate νi j (this is, however, not depicted in the above). Each state si has an associated synthesis rate, Ki, equal to KB + k · si, where k = (km−1,km−2,...,k0) is the vector of mRNA transcription rates for the enhancers. Degradation occurs at rate δ, independently of the activity state.
m bound or unbound to activators with rates λi and µi (resp.) and defining the 2 -multistate model can be summarised as: contributes, additively, with rate k to the overall transcription i rate of the gene. So each a is either bound or unbound, lead- νi j m i si −→ s j, for i, j ∈ {0,...,2 − 1} and di j = 1, m ing to 2 possible states for the activity of the gene, which we ki m Mm := si −→ si + M, for i ∈ {0,...,2 − 1}, denote by s0 = (0,0,...,0),s1 = (0,0,...,0,1),...,s2m−1 = δ (1,1,...,1), where for notational convenience si is the m- M −→ ∅, bit binary representation of the number i as a tuple. Tuples (5) differing in just one position (that is, of Hamming distance where di j is the Hamming distance between si and s j. Note d(si,s j) = 1) will be said to be adjacent. Changes in state re- also that νi j,ki ≥ 0, and δ > 0. The transition digraphs arising sult from a binding or unbinding of an enhancer, from which it from the reactions defined in (5) can be visualised as hyper- follows that state transitions will occur only between adjacent cubes (m-dimensional cubes); see Fig. 4. states. Observe that the leaky Telegraph model coincides with the 1 2 -multistate model, M1, by setting KB = KI and k0 = KA −KI. We allow for a basal (leaky) rate of transcription KB, which As a second example, the 22-multistate model, M , consists of is independent of the state of the enhancers. Then in each state 2 two distinct activating enhancers a0 and a1, each with associ- si, the mRNA are transcribed according to a Poisson process m ated transcription rates k0 and k1, respectively. This leads to with a constant rate Ki = KB + k · si, for i ∈ {0,...,2 − 1}, four possible activity states s0 = (0,0),s1 = (0,1),s2 = (1,0) where k = (km−1,km−2,...,k0) is the vector of mRNA tran- and s = (1,1). The state s = (0,0) corresponds to when scription rates for the enhancers and · is the usual dot product. 3 0 both a0 and a1 are unbound. The state s1 = (0,1) corre- Note that for i = 0 (all enhancers unbound) this gives K0 = KB, sponds to when a0 is bound and a1 is unbound. Similarly, while for j ∈ {0,...,m − 1}, we have K j = KB + k j. 2 the state s2 = (1,0) corresponds to when a0 is unbound and a = ( , ) The enhancers switch between bound and unbound with 1 is bound, and state s3 1 1 corresponds to when both a and a are bound. The enhancers contribute additively to rates λ and µ , respectively. This means that the system tran- 0 1 i i , , , sitions between adjacent states s and s only: states differ- the overall transcription rate, so the rates for s0 s1 s2 s3 are j1 j2 K = K K = K + k K = K + k K = K + k + k ing in at most one bit, i say. This means that transition from 0 B, 1 0 0, 2 0 1, 3 0 0 1, re- spectively. Refer to Fig. 5(a) for a representation of the model s j1 to s j2 occurs stochastically at rate ν j1 j2 equal to either λi th M2. (if j1 < j2, meaning the i bit of j1 is 0) or µi (if j2 < j1). Degradation of mRNA occurs as a first-order Poisson process with rate δ, and is assumed to be independent of the promoter C. Equivalent systems for M state. By a 2m-multistate model, we will mean a multistate m model with 2m states arising from the independent and ad- m ditive transcription activity of m enhancers. We denote this As with our motivating example, the 2 -multistate model m model by Mm. Figure 3 gives a depiction of the 2 -multistate has a courser-grained formulation obtained by disregarding process. the finer level information about the states of the activating enhancers. In this view, the 2m-multistate model can be equiv- If we let M represent the mRNA molecules, the reactions alently thought of as a model for the transcriptional activity of bioRxiv preprint doi: https://doi.org/10.1101/2020.01.05.895359; this version posted January 6, 2020. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under aCC-BY-NC-ND 4.0 International license. 5
(a) T1 (b) T2 (c) T3 (d) T4
FIG. 4. Transition digraphs arising from the 2m-multistate process. For m = 1, the transition digraph is a line segment, for m = 2 the transition digraph is a square, for m = 3 the digraph is a cube, and m = 4 corresponds to when the graph is a tesseract, and so on. Note that here we use Tm to denote the underlying transition digraph of the model Mm.
a. multiple enhancers of a single gene, where each en- i ∈ {0,...,m − 1}, let Ti be a random variable distributed by hancer has a particular additive effect on the mRNA a Telegraph distribution,p ˜T (n), with rates λi, µi,ki. We have transcription rate; already seen that the leaky Telegraph system (which can be thought of as arising from a 21-multistate process) has the as- b. multiple independent copies of a single gene, where sociated random variable decomposition X = P + T, where each copy of the gene is governed by a (possibly leaky) T is Telegraph distributed with rate k0. It follows from the two-state system, and with possibly different transcrip- equivalence of (a) and (b) above that this can be generalised tion and switching rates; to:
c. a gene promoter with a finite, exponential number of Y = P + ∑ Ti (6) states, where each promoter has an associated mRNA 0≤i 2 D.Exact solution for the 2m-multistate model As an example of (8), the 2 -multistate model has probabil- ity mass function: Here we provide analytical solutions for any model of the 1 n rc class M = {Mm | m ∈ N0}. Let m ∈ N0 and let Y be a ran- p˜2(n) = ∑ K0 exp(−K0) m n! rc,r0,r1 dom variable that counts the mRNA copy number of the 2 - rc+r0+r1=n multistate system in the steady-state. Recall that in this sys- kr0 λ (r0) m a ,...,a 0 0 F ( + r , + + r ,−k ) tem the gene has distinct activating enhancers 0 m−1, (r ) 1 1 λ0 0 µ0 λ0 0 0 each switching between the states bound or unbound, inde- (λ0 + µ0) 0 pendently, with rates λ and µ , and contributing additively r1 (r1) i i k1 λ1 with rate k , for i ∈ {0,...,m − 1}, to the overall transcrip- 1F1(λ1 + r1, µ1 + λ1 + r1,−k1) . (9) i (λ + µ )(r1) tion rate of the gene. There is also a background rate of tran- 1 1 scription K0 = KB. As before, we let P be a random variable Figure 5(b) gives a comparison ofp ˜2(n) and the probability for a constitutive process with constant rate K0 and, for each mass function from simulated data. A second example of (8) bioRxiv preprint doi: https://doi.org/10.1101/2020.01.05.895359; this version posted January 6, 2020. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under aCC-BY-NC-ND 4.0 International license. 6 0.04 Simulation Analytic 0.02 Probability, p ( n ) 0.00 0 25 50 75 100 Copy number, n FIG. 5. (a): A schematic of the 22-multistate model (b): A comparison of the analytical solution for the steady-state mRNA distribution for the 2 2 -multistate model and the probability mass function from simulated data. The parameter values for both curves are K0 = 5, k0 = 21, k1 = 33 2 and λi = µi = 0.01, for i ∈ {0,1}. The degradation rate δ is set to 1. (c): Decomposing the 2 -multistate model (left) into constitutive and Telegraph parts (right). is a solution to the 23-multistate model, depicted in Fig. 6. tain properties. In both cases we observe excellent agreement of our method Our argument relies on moment generating functions, which with stochastic simulations. 1 for a random variable X with distribution f , is defined as While the 2 -multistate model (corresponding to the leaky tx M f (t) := E(e ) for t ∈ R. We here take heavy-tailed to mean Telegraph model) has been solved before, to the best of our n that the moment generating function is undefined for positive knowledge this is not the case for 2 -multistate models with t. As in Ref. 11, the effects of extrinsic variability on the mul- n ≥ 2. tistate systems is captured via a compound distribution; see Eq. 9 there. Items (a), (b), (c) follow directly from the fol- lowing inequality for the moment generating function of the III. EFFECTS OF EXTRINSIC NOISE Telegraph distribution,p ˜T (n): (Ref. 11, Eq. 16) for all posi- tive t, In a similar way to Ref. 11, we now jointly consider the M (t) ≤ M (t) ≤ M (t), (10) effects of intrinsic and extrinsic noise on the probability dis- λ K p˜T Pois(KA) Pois µ+λ A tributions arising from the class of multistate models M = where M denotes the moment generating function for distri- {Mm | m ∈ N0}. The results given in Ref. 11 are in the con- g text of the leaky and standard Telegraph processes. We will bution g. We establish an analogous result for our multistate show that the decomposition established in (6) allows us to systems. First let m ∈ N0. Now since the moment generat- straightforwardly extend the main analysis and results derived ing function for Mm is simply the product of a Poisson mo- in Ref. 11 to M . More specifically, we show that ment generating function and m Telegraph moment generating functions it follows immediately from (10) that (a) intrinsic noise alone, as arising from the inherent stochas- m ticity of the 2 -multistate process, never results in a MPois(KB)(t) M λi (t) ≤ Mp˜m (t) ∏ Pois ki heavy-tailed mRNA copy number distribution; 0≤i cific distribution, but to any distribution that satisfies cer- MPois(η1)(t) ≤ Mp˜m (t) ≤ MPois(η2)(t), (12) bioRxiv preprint doi: https://doi.org/10.1101/2020.01.05.895359; this version posted January 6, 2020. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under aCC-BY-NC-ND 4.0 International license. 7 (a) (b) 0.02 0.02 Simulation Analytic 0.01 0.01 p ( n ) Probability, Probability, p ( n ) 0.00 0.00 0 100 200 300 0 100 200 300 Copy number, n Copy number, n FIG. 6. A comparison of the analytic solution to the 23-multistate model and the probability mass function from simulated data. The parameter values for (a) are KB = 5, k0 = 30, k1 = 50, k2 = 100 and λi = µi = 0.01, for i ∈ {0,1,2}. For (b) the parameters are KB = 5, k0 = 20, k1 = 65, k2 = 115 and λi = µi = 0.01, for i ∈ {0,1,2}. λi where η1 = KB + ∑ ki and η2 = KB + ∑ ki. Now other systems of gene transcription, including those involving µi+λi 0≤i We now introduce a recurrence method that can be used for νi j si −→ s j, for i, j ∈ {1,...,`}, approximating solutions to master equation models of gene Ki transcription. We show that the method applies to a class of M` := si −→ si + M, for i ∈ {1,...,`}, (13) δ general multistate models in which any finite number of dis- M −→ ∅ crete states, along with arbitrary transitions between states, m is allowed. In comparison to the 2 -multistate model, the where again M represents the mRNA molecules, and νi j,ki ≥ number of states is no longer restricted to powers of two, and 0, and δ > 0. We let M` denote the model defined by (13), transitions between states are no longer restricted to adjacent and refer to M` as the `-switch model. states. In fact, the 2m-multistate model is a special case of Comparing with (5) we find that the 2m-multistate model this more general model. In addition, we show that the recur- agrees with the `-switch model by setting ` = 2m, and setting rence method is applicable to systems that are non-linear. We νi j = 0 whenever d(si,s j) > 1 and provided that the transcrip- illustrate this by applying the method to the gene regulatory tions rates adhere to the necessary additivity condition. feedback model given in Ref 29. We are interested in analysing stationary distributions of the We begin this section by introducing the master equation systems described in (13). Let pi(n) denote the probability at for the class of more general multistate models. Following stationarity that the gene is in state i with n mRNA molecules, this, we present a step-by-step breakdown of our recurrence for i ∈ {1,...,`}. The generalised chemical master equation method that can be used to provide a solution of desired ac- for the probability mass function, pi(n), is given by: curacy to any multistate model from this class. We then il- ! lustrate the method through an example; see Subsection IV C. ` When an analytical solution can be obtained using the results (∀n ≥ 1) ∂t pi(n,t) = − ∑ νi j + Ki + δn pi(n,t) j= of the previous section, we assess the computational efficiency 1 and accuracy of the two solutions. We also make a compari- ` + ν p (n,t) + δ(n + 1)p (n + 1,t) son of our approach with the stochastic simulation algorithm. ∑ ji j i j=1 These results are given in Subsection IV F. Finally, we discuss the applicability and usefulness of our recurrence approach to + Ki pi(n − 1,t), (14) bioRxiv preprint doi: https://doi.org/10.1101/2020.01.05.895359; this version posted January 6, 2020. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under aCC-BY-NC-ND 4.0 International license. 8 for i ∈ {1,...,`}. i ∈ {1,...,`}: ` (n+1) (n) δ(z − 1)gi (z)= − δn + ∑ vi j + Ki(1 − z) gi (z) j=1 B. The recurrence method ` (n−1) (n) + Kingi (z) + ∑ v jig j (z). (17) In the following, we give a step-by-step breakdown of the j=1 recurrence method applied to the system described in (14). After transforming the master equation (Step 1), the idea is Evaluating each of the equations in (17) at z = 1, we imme- to transform the resulting coupled system of differential equa- diately obtain the system of recurrence relations: (n) ` ` tions into a closed system of recurrence relations in gi (1) (n) (n−1) (n) (Step 2), where gi(z) is the generating function of pi(n). Once δn + ∑ vi j gi (1)= Kingi (1) + ∑ v jig j (1). (18) j=1 j=1 the initial conditions gi(1) have been found, we can iteratively (n) generate the gi (1) from the obtained recurrence relations, Rearranging each of these so that the LHS is equal to 0, the re- which in turn can be used to solve for g(z). This is detailed sulting system of equations can be represented in matrix form in Step 3. Finally, in Step 4, we can recover the solution for as: p˜(n) by way ofp ˜(n) = 1 g(n)(0). n! RX = 0, (19) A similar approach for transforming differential equa- tions into recurrence relations is employed in Ref. 33 to where R is a ` × 2` matrix defined as: find approximate solutions to l-switch models. It can be demonstrated that the final recurrence relations obtained from [AB], (20) the transformation given here agree with those obtained in where A is an ` × ` matrix defined by: Ref. 33. While our method presents an alternative pathway to obtaining recurrence relations, the primary focus here will ` be on efficiency comparisons between our method, the previ- δn + ∑ vik, for i = j and k 6= i Ai j := k=1 (21) ously obtained analytical results, and stochastic simulations, −v ji, for i 6= j as well as applicability to non-linear models that do not be- long to the l-switch class. and B is a ` × ` matrix defined by: ( Kin, for i = j Bi j := (22) 0 for i 6= j. Step 1. Transform the master equation We also have As usual we can follow the generating function X = [g(n)(1),...,g(n)(1),g(n−1)(1),...,g(n−1)(1)]T . approach35,49 by defining, for each i ∈ {1,...,`}, a sta- 1 ` 1 ` tionary generating function: Eq. (18) constitutes a system of first-order linear recurrence relations and can be decoupled by applying Gaussian elimina- ∞ tion to (19). This enables us to write each of the g(n)(1) in g (z) = zn p (n). i i ∑ ˜i (15) (n−1) n=0 terms of the g j . We let R denote the recurrence relations obtained from the resulting matrix; as usual with Gaussian We transform the master equation (14) by multiplying through elimination, it is not practical to present a general form for the by zn and summing over n from 0 to ∞, obtaining a system of final simplified matrix. ` coupled differential equations: ` ! ` Step 3. Iteratively solve the recurrence relations 0 δ(z − 1)gi(z)= − ∑ νi j + Ki(1 − z) gi(z)+ ∑ ν jig j(z). j=1 j=1 We require first the initial conditions gi(1), for i ∈ (16) {1,...,`}. These can be found by evaluating equation (16) at z = 1, and solving the resulting system of linear equations. This system can be represented in matrix form as: Step 2. Transform the differential equations into a system of CX = 0, (23) first-order recurrence relations where the matrix C is defined as: th , i 6= j Noting that in general the k derivative of a function of ν ji for (k) (k−1) ` the form (z − 1)h(z) is (z − 1)h (z) + kh (z), by differ- Ci j := (24) − ∑ νik, for i = j and k 6= i, entiating (n times) the equations in (16), we obtain for each k=1 bioRxiv preprint doi: https://doi.org/10.1101/2020.01.05.895359; this version posted January 6, 2020. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under aCC-BY-NC-ND 4.0 International license. 9 and we have T X = g1(1),...,g`(1) . (25) K1n 0 0 B = 0 K2n 0 Now, as ∑ gi(1) = 1, the gi(1) are found as a normalised 0 0 K3n 1≤i≤` version of Null(A), which is necessarily one dimensional for We also have that these multistate systems. (n) (n) (n) (n−1) (n−1) (n−1) Now, if gi(z) is considered as a power series in (z−1), then T X = [g1 (1),g2 (1),g3 (1),g1 (1),g2 (1),g3 (1)] . n 1 (n) the coefficient of (z − 1) is n! gi (1). Thus, the coefficients (n) After applying Gaussian elimination to R, we obtain the fol- h (n) := 1 g (1) may be generated by iteration using the i n! i lowing system of first-order linear recurrence relations, for all first-order recurrence relations, R, obtained from (20). One i ∈ {1,2,3} and j < k from {1,2,3}\{i}: way to do this would be to simply solve the recurrence rela- tions for g(n)(1) and to subsequently divide by n!. However, i (n) 1 h 2 2 (n) gi (1) = Ki δ n + δn(v ji + v jk + vki + vk j) this involves computing a large number (gi (1)) and dividing D(n) it by another large number (n!), which can lead to numerical + v v + v v + v v g(n−1)(1) problems. Instead, we transform the recurrence relations for k j ji jk ki ji ki i (n) (n−1) gi (1) into recurrence relations for the hi(n). + Kj δnv ji + v jkvki + v ji(vk j + vki) g j (1) Computationally, we may store the values for hi(n) in a (n−1) i system of ` lists from which we can compute the coefficients + Kk δnvki + v jkvki + v ji(vk j + vki) gk (1) , n h(n) = ∑i hi(n) of (z−1) in the expansion of g(z) as a power (28) series in (z − 1). So, where ∞ n g(z)= h(i)(z − 1) . (26) 2 2 ∑ D(n) =δ δ n + δn(v12 + v13 + v21 + v23 + v31 + v32) i=0 + v12(v23 + v31 + v32) + v13(v21 + v23 + v32) + v (v + v ) + v v . (29) Step 4. Recover the stationary probability distribution. 21 31 32 23 31 Step 3. We first find the initial conditions. From (24), the Asp ˜(n) is the coefficient of zn in the expansion of g(z) as matrix, C, is given by: a power series in z, we are able to recoverp ˜(n) by way of 1 (n) p˜(n) = n! g (0). It then follows from (26) that −(v12 + v13) v21 v31 C = v12 −(v21 + v23) v32 ∞ (n) (i + 1) v13 v23 −(v31 + v32) p˜(n) = h(n + i)(−1)i, (27) ∑ n! i=0 It can be shown that the null space, Null(C), is given by: where again x(n) denotes the rising factorial. For practical v v + v v + v v implementations, we truncate the sum in (27) after a certain 21 31 23 31 21 32 Null(A)= v v + v v + v v number of terms. The distributionp ˜(n) typically becomes 12 31 12 32 13 32 v v + v v + v v negligibly small for n larger than a certain value, say 100. 13 21 12 23 13 23 Computing h(n) up to n = 500 for example would thus leave Thus, the initial conditions are: at least 400 terms for approximatingp ˜(n) for n ≤ 100. For the studied example systems we found that 100 − 350 terms were 1 g (1) = (v v + v v + v v ), (30) sufficient to achieve accurate results. 1 N 21 31 23 31 21 32 1 g (1)= (v v + v v + v v ), (31) 2 N 12 31 12 32 13 32 C. Example 1: the 3-switch model 1 g (1)= (v v + v v + v v ). (32) 3 N 13 21 12 23 13 23 We illustrate the recurrence method for the case l = 3. Steps 1 and 2. From (20), the matrix, R := [AB] is com- 3 prised of: Here N is the normalisation constant and is equal to ∑ ar,1, r=1 where each ar,1 is an element of the matrix Null(A). δn + (v12 + v13) −v21 −v31 Next, we need to compute the coefficients hi(n) := A = −v12 δn + +(v21 + v23) −v32 1 g(n)(1) of g (z) considered as a power series in (z − 1). As −v13 −v23 δn + (v31 + v32) n! i i mentioned in Step 3 above, we transform the recurrence rela- (n) and tions for gi (1) into recurrence relations for the hi(n). These bioRxiv preprint doi: https://doi.org/10.1101/2020.01.05.895359; this version posted January 6, 2020. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under aCC-BY-NC-ND 4.0 International license. 10 can be obtained from (28) by dividing the RHS of each equa- the process can be written as: tion by n: for each {i, j,k} = {1,2,3}, r D −→u D + P, u u (n) rb (n) gi (1) Db −→ Db + P, hi (1) = (33) D(n)n k f P −→ ∅, (34) kb Step 4. Finally we recover the stationary distribution,p ˜(n). Db −→ Du, s For a given set of parameter values, we generated a list of b Du + P Db values for h(n) of length 500, and used this to approximate the su probability mass function (27) from approximately 400 terms. We refer the reader to Ref. 29 for further details of the model The results for two different sets of parameter combinations including the associated master equation (Ref. 29, Eq. 8). Ap- are given in Fig. 7. plying the recurrence method to the feedback model (34), we obtain an approximation to the probability mass function. For two sets of parameter values, we compare this with the known D. Discussion of wider applicability analytic solution and present the results in Fig. 8. For details of the application of the method, including the derivation of So far, we have seen how the recurrence method can be ap- the recurrence relations used for the approximation see Sec- plied to linear l-switch models. In this section, we discuss the tion A 2 of the appendix. The analytic solution for this model wider applicability of the method, and demonstrate that it can is given as equation (A23) in the appendix. in principle also be applied to certain non-linear systems. As To this point, our discussion of the applicability of the re- an example, we show that the method can be used to approxi- currence method has concerned only chemical master equa- mate the solution of the gene regulatory feedback model given tion models with a two-dimensional state space (tuples con- in Ref. 29. sisting of the state of the gene and the mRNA copy number We would like to point out that the applicability to this sys- or protein). Preliminary investigations suggest the method tem is limited, however. Application of the recurrence method can be extended to models of gene transcription giving rise to higher dimensional state spaces. The three-stage model to the feedback model relies on the initial conditions being 31 supplied, and these are currently derived from the known an- of Shahrezaei and Swain is an example. In this model, the alytic solution. This apparent circularity does not undermine production of mRNA and protein are treated as separate pro- the overall approach. The problem of solving chemical mas- cesses, resulting in a three-dimensional state space. In such ter equation systems can essentially be broken into two sep- cases, the recurrence method gives rise to equations contain- arate tasks: solving for the initial conditions for the gener- ing partial derivatives. This makes the task of finding the ini- ating function and solving for the probability mass function. tial conditions for the recurrence approach even more chal- Indeed, there are situations where the initial conditions of a lenging. A characterisation of when the recurrence method given system can be easily found, but the probability mass is applicable to chemical master equation models will be the function remains unknown; the `-model is an example. On focus of future work. the other hand, there are situations where a general form for the probability mass function can be stated for a given set of initial conditions, but the initial conditions for the system are E. Notes on numerical implementation not yet known; this is the situation presented in Ref. 29. For the feedback model considered here, we are able to illustrate We found that the recurrence method can become numeri- applicability to one half of the problem: for a given set of ini- cally unstable when solved with standard numerical precision tial conditions for the generating function we can provide an for some of the studied example systems and parameter val- approximation to the probability mass function. ues. However, for sufficiently larger numerical precision we We now apply the recurrence approximation to the system are able to obtain numerically stable and accurate results for of differential equations arising from the feedback loop in all systems. We used Mathematica for the numerically dif- Ref. 29. In this model of a single gene, the protein produced ficult cases and provide example code in the supplementary can bind to the promoter of this same gene, regulating its own information. expression. The process is modelled by recording only the number of proteins and the state of the gene promoter, which can be either bound or unbound. Here the rate of production F. Assessment of computational efficiency of proteins depends on the state of the promoter region of the gene. Following Ref. 29, we let ru and rb denote the protein We now assess the computational efficiency of the re- production rates for the bound and unbound states, respec- currence method by examining how the computational time tively. We let k f be the degradation rate of proteins, and let kb scales with system size. We do this for the leaky Telegraph be the degradation rate of the bound protein. Also letting Du, model, using the analytical solution to verify the accuracy of Db denote the unbound and bound states of the gene promoter, the result. As displayed by the example in Fig. 9(a), we find respectively, and P denote the protein, the reaction scheme for the two methods to be in complete agreement provided that bioRxiv preprint doi: https://doi.org/10.1101/2020.01.05.895359; this version posted January 6, 2020. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under aCC-BY-NC-ND 4.0 International license. 11 (a) (b) 0.03 Simulation 0.06 Recurrence 0.04 0.02 0.02 0.01 p ( n ) Probability, Probability, p ( n ) 0.00 0.00 0 10 20 30 40 0 20 40 60 80 Copy number, n Copy number, n FIG. 7. Stationary distributions of the 3-switch model in Section IV C. The figure shows the results obtained from stochastic simulations and from the recurrence method describes in Section IV. Figure (a) has parameters v12 = v21 = v31 = 0.1, v13 = 0.4, v32 = 0.9, and K1 = 2, K2 = 16, K3 = 5. Figure (b) displays a trimodal distribution with parameters v12 = v13 = 0.045, v21 = v32 = 0.035, v31 = v23 = 0.015, K1 = 5, K2 = 30, K3 = 60. We used 300 (a) and 305 (b) terms respectively for the recurrence method to approximate the sum in (27). (a) (b) 0.06 0.03 Analytic Recurrence 0.04 0.02 0.02 0.01 p ( n ) Probability, Probability, p ( n ) 0.00 0.00 0 10 20 30 0 20 40 60 80 Copy number, n Copy number, n FIG. 8. Stationary distributions of the feedback model in Section IV D. We compare the analytic solution with the results obtained from the recurrence approximation. Figure (a) has parameters ρu = 0.125, ρb = 20, θ = 0.5, σu = 0.2, σb = 0.35, and (b) has parameters ρu = 60, ρb = 6, θ = 0, σu = 0.5, σb = 0.004. We used 110 (a) and 300 (b) terms, respectively, for the recurrence method to approximate the probability mass function. a sufficient number of terms are considered in the expansion. For the results shown in Fig. 7 for the 3-switch model, for ex- For all of the models considered in this paper, the maximum ample, we found that simulating 300,000 samples using our copy number that must be considered nmax, is directly related implementation of the SSA was about two orders of magni- to the largest rate, in this case K1. We therefore apply the tudes slower than the recurrence method (∼ 57 (a) and ∼ 155 method for a number of K1, some examples of which are dis- (b) seconds using the SSA, respectively, versus ∼ 0.1 (a) and played in Fig. 9(b). We find that for each case a different ∼ 2 (b) seconds using the recurrence method.). number of terms in the expansion, and consequently a differ- ent computational cost, are required (Figs. 9(c), (d)). Scaling of the number of terms is observed to be approximately linear 3 while scaling of the run-time is polynomial of O(nmax). Interestingly, we observe in the numerical examples that the main computational cost of the recurrence method does not It is worth noting that even for systems for which analytic occur from solving the recurrence relation itself, but instead solutions exist, the solutions are typically expressed in terms from the subsequent reconstruction of the probability mass of so-called “special functions” such as Bessel or Hypergeo- function according to (27). This suggests that the computa- metric functions, which still need to be evaluated numerically tional cost of our method should be similar to or less than the and often by recurrence relations or series expansions Ref. 50. computational cost when the analytic solution to the generat- As an example see the probability mass function of the feed- ing functions is known. For the feedback model studied above back model (A23) which involves a sum over confluent Hy- we observe that this is indeed the case. pergeometric functions. Such solutions can hence still be nu- For systems where no analytic solution is known one typi- merically expensive if one needs to evaluate over many points cally utilises the SSA to simulate sample trajectories from the in state space. Indeed, we find that our recurrence method stochastic process. Use of the SSA to obtain distributions is is considerably faster for approximating the probability mass notoriously expensive Ref.34, and requires a sufficiently large function of the feedback model than evaluating the analytic number of statistically independent samples to be obtained. expression. bioRxiv preprint doi: https://doi.org/10.1101/2020.01.05.895359; this version posted January 6, 2020. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under aCC-BY-NC-ND 4.0 International license. 12 (a) Comparison for K1=50 (b) Distribution for varying K1 0.04 Analytic 0.10 K1=10 Recurrence K1=30 K1=50 K1=70 K1=90 0.02 0.05 Probability, p ( n ) 0.00 0.00 0 20 40 60 80 0 25 50 75 100 n n (c) Required number of terms in expansion (d) Runtime 500 Results 0 O(n3) 400 -1 300 -2 -3 200 log(runtime), [s] p ( n ) Probability, terms of Number -4 100 -5 2525 50 75 100 125 3.5 4.0 4.5 5.0 nmax log(nmax) FIG. 9. Scaling of the computational cost with increasing system size for the two-state model. In Fig. (a), we compare the analytic probability mass function for the leaky Telegraph model with the approximation obtained by recurrence method. The parameters are λ = 0.1, µ = 0.1, K0 = 5 and K1 = 50 and we used 270 terms to approximate the sum in (27) for the recurrence method. In (b), we plot the different leaky Telegraph distributions obtained by varying the parameter K1. Figure (c) shows the number of terms required in the recurrence expansion as a function of the maximum copy number with non-negligible probability in the system. We observe a linear relationship. Figure (d) shows that the runtime of the recurrence method scales roughly as the cube of the maximum copy number. V. CONCLUSION rence method applies is more general than (and contains) the class of 2m-multistate models to which the analytic method Despite the importance of stochastic gene expression mod- applies. The recurrence method is not limited to linear `- els in the analysis of intracellular behaviour, their mathemat- switch models, however, but can also be applied to certain ical analysis still poses considerable challenges. Analytic so- non-linear systems. Specifically, we have shown that given lutions are only known for special cases, while few system- some known initial conditions, the method can be used to ap- atic approximation methods exist and stochastic simulations proximate the solution to a gene regulatory network with a are computationally expensive. In this work, we provide two feedback loop. For all studied systems, we found an excellent partial solutions to these challenges. agreement of the recurrence method and analytic solutions or stochastic simulations. In this first part, we developed an analytic solution method for chemical master equations of a certain class of multistate In cases where no analytic solution is known, the recurrence gene expression models, which we call 2m-multistate models. method was found to significantly outperform stochastic sim- The method is based on a decomposition of a process into in- ulations in terms of computational efficiency. Even in cases dependent sub-processes. Convolution of the solutions of the where an analytic solution exists, we found the method to be latter, which are easily obtained, gives rise to the solution of computationally more efficient than the evaluation of the an- the full model. The solutions can be derived straightforwardly alytic solution. While a characterisation of precisely which and are computationally efficient to evaluate. While most ex- models the method is applicable to is not yet known, the re- isting solutions only apply to specific models, our decomposi- sults presented here suggest the recurrence method is an accu- tion approach applies to the entire class of 2m-multistate mod- rate and flexible tool for analysing stochastic models of gene els. This class covers a wide range of natural examples of expression. We believe that together with the presented ana- multistate gene systems, including the leaky Telegraph model. lytical method it will contribute to a deeper understanding of the underlying genetic processes in biological systems. In the second part of this work, we derived a recurrence method for directly approximating steady-state solutions to chemical master equation models. The method was employed to approximate solutions to a broad class of linear multi- SUPPLEMENTARY MATERIAL state gene expression models, which we term `-switch mod- els, most of which currently have no known analytic solution. See the supplementary material for sample Mathematica In particular, the class of `-switch models to which the recur- code for each application of the recurrence method considered bioRxiv preprint doi: https://doi.org/10.1101/2020.01.05.895359; this version posted January 6, 2020. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under aCC-BY-NC-ND 4.0 International license. 13 in the present paper. Step 2. Differentiating (n times) the equations (A6) and (A7) gives: ACKNOWLEDGMENTS (n+1) (n) (n) ((z−1)+σbz)g0 (z)+(n+σbn)g0 (z)=(θ +σuz)g1 (z) (n−1) (n) (n−1) LH and MPHS gratefully acknowledge support from the + σung1 (z) + ρu(z − 1)g0 (z) + ρung0 (z) (A8) University of Melbourne DVCR fund; RDB and MPHS were funded by the BBSRC through Grant BB/N003608/1; DS was and funded by the BBSRC through Grant BB/P028306/1. (n+1) (n) (n) (z − 1)g1 (z)+ ng1 (z)=(ρb(z − 1) − (θ + σu))g1 (z) (n−1) (n+1) + nρbg (z) + σbg (z). (A9) Appendix A 1 0 Evaluated at z = 1, we immediately obtain the recurrence re- 1. Recurrence relations for the leaky Telegraph model lations (n+1) (n) (n−1) The following recurrence relations and initial conditions, σbg0 (1)=(θ + σu)g1 (1)+ σung1 (1) obtained by applying the recurrence method to the leaky Tele- (n) (n− ) − (n + σ n)g (1) + ρ ng 1 (1) (A10) graph model, can be used to give an approximation to the b 0 u 0 probability mass function (1). (n) (n−1) (n+1) (n + θ + σu)g (1)= ρbng (1) + σbg (1). (A11) h0(1) = µ/(λ + µ) and h1(1)= λ/(λ + µ); (A1) 1 1 0 Substituting (A10) into (A11) leads to the following system of second-order linear recurrence relations: for n ≥ 1, µ hnδ + µ h (n) = K h (n − 1) 0 (n + )(n + ) − I 0 δ µ δ λ λ µ µ (n+1) 1 h (n) (n−1) i g0 (1) = (θ + σu)g1 (1)+ σung1 (1) + KAh1(n − 1) (A2) σb (n) (n−1) i − (σb + 1)ng0 (1)+ ρung0 (1) (A12) λ hnδ + λ h1(n) = KAh1(n − 1)(1) (nδ + µ)(nδ + λ) − λ µ λ g(n)(1)=(ρ + σ )g(n−1)(1) − (σ + 1)g(n)(1) i 1 b u 1 b 0 + KIh0(n − 1) . (A3) (n−1) + ρug0 (1). (A13) Step 3. Provided that the initial conditions g0(1), g1(1) 0 0 θ+σu 2. Recurrence relations for the feedback model and g (1) are known (here g (1)= g1(1)), the values of 0 0 σb (n) (n) g0 (1) and g1 (1) can be obtained by iteration. In this exam- We provide details of the recurrence approximation applied ple, we use the initial conditions calculated from the known to the system of differential equations arising from the feed- analytic solution (see Section A 3 for details of the initial con- back loop in Ref. 29. For convenience we list here the dimen- ditions and the analytic solution). sionless variables that appear there: Step 4. As before, we use a simplification that avoids the computational issue of calculating one huge number g(n)(1), ρu = ru/k f , ρb = rb/k f , θ = kb/k f , σu = su/k f , σb = sb/k f i (A4) and then dividing by another huge number n!. The actual coef- 1 (n) and the parameters: ficients hi(n) := n! gi (1) may be generated by iteration using the following recurrence relations obtained from (A12) and Σb = 1 + σb, R = ρu − ρbΣb. (A5) (A13): Step 1. The conveniently transformed equations from (n+1) 1 h (n) (n−1) Ref. 29 (Equations (12) and (13)) are: h0 (1) = (θ + σu)h1 (1) + σuh1 (1) σb(n + 1) (n) (n−1) i 0 − (σb + 1)nh (1)] + ρuh (1) (A14) ρu(z − 1)g0(z) − (z − 1)g0(z) + (θ + σuz)g1(z) 0 0 0 − σbzg0(z)= 0 (A6) (n) h(ρb + σu) (n−1) (n) h (1)= h (1) − (σb + 1)h (1) 0 1 n 1 0 ρb(z − 1)g1(z) − (z − 1)g1(z) − (θ + σu)g1(z) ρu (n−1) i 0 + h (1) . (A15) + σbg0(z)= 0 (A7) n 0 bioRxiv preprint doi: https://doi.org/10.1101/2020.01.05.895359; this version posted January 6, 2020. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under aCC-BY-NC-ND 4.0 International license. 14 Note that here Equation (A14) is obtained from (A12) by di- Again from (A22), the initial conditions g0(1) and g1(1) are: viding each term g (n + 1 − j) by (n + 1) , and similarly, i ( j+1) Equation (A15) is obtained from (A13) by dividing each term Σb α θ − α g0(1)= A 1F1(α + 1,β,w1) + 1F1(α,β,w1) gi(n − j) by n( j), where x(i) denotes the falling factorial of x. σb ρu ρu − ρb Step 5. We are now able to recover the stationary proba- (A24) bility mass function. Again, we generate a list of values for and h(n) and use this to approximate the probability mass func- tion (27). A comparison of the approximate solution and the g1(1) = A 1F1(α,β,w1). (A25) known analytic solution is given in Figure 8. 1M. B. Elowitz and S. Leibler, “A synthetic oscillatory network of transcrip- 3. Explicit solution to the feedback model tional regulators,” Nature 403, 335–339 (2000). 2M. B. Elowitz, A. J. Levine, E. D. Siggia, and P. S. Swain, “Stochastic gene expression in a single cell,” Science 297, 1183–1186 (2002). Here we provide an explicit solution for the stationary prob- 3A. P. Gasch, F. B. Yu, J. Hose, L. E. Escalante, M. Place, R. Bacher, J. Kan- ability mass function of the regulatory feedback loop consid- bar, D. Ciobanu, L. Sandor, I. V. Grigoriev, C. Kendziorski, S. R. Quake, and M. N. McClean, “Single-cell rna sequencing reveals intrinsic and ex- ered in Section IV D. From Refs. 29 and 30, the exact solu- trinsic regulatory heterogeneity in yeast responding to stress,” PLOS Biol. tions for the generating functions are given by: 15, e2004050 (2017). 4A. Raj, C. S. Peskin, D. Tranchina, D. Y. Vargas, and S. Tyagi, “Stochastic ρ (z−1)hΣb α mRNA synthesis in mammalian cells,” PLoS Biol. 4, e309 (2006). g (z) = Ae b F (α + 1,β,w) 5 0 σ ρ 1 1 J. M. Raser and E. K. O’Shea, “Noise in gene expression: origins, conse- b u quences, and control,” Science 304, 1811–1814 (2004). θ − α i 6P. S. Swain, M. B. Elowitz, and E. D. Siggia, “Intrinsic and extrinsic con- + 1F1(α,β,w) , (A16) tributions to stochasticity in gene expression,” Proc. Natl. Acad. Sci. 99, ρu − ρb 12795–12800 (2002). and 7M. S. H. Ko, “A stochastic model for gene induction,” J. Theor. Biol. 153, 181–194 (1991). 8 ρb(z−1) J. Peccoud and B. Ycart, “Markovian modeling of gene-product synthesis,” g1(z)= Ae 1F1(α,β,w), (A17) Theor. Popul. Biol. 48, 222–234 (1995). 9 where I.-B. Srividya, F. Hayot, and C. Jayaprakash, “Transcriptional pulsing and consequent stochasticity in gene expression,” Phys. Rev. E 79, 031911 σu(ρu − ρp) (2009). α = θ + , (A18) 10J. Paulsson and M. Ehrenberg, “Random signal fluctuations can reduce R random fluctuations in regulated components of chemical regulatory net- 1 Σb − 1 works,” Phys. Rev. Lett. 84, 5447–5450 (2000). β = 1 + θ + σu + ρu , (A19) 11L. Ham, R. D. Brackston, and M. P. Stumpf, “Ex- Σb Σb trinsic noise and heavy-tailed laws in gene expres- Σ z − 1 w = R b , (A20) sion,” bioRxiv:10.1101/623371v1 (2019), 10.1101/623371, 2 https://www.biorxiv.org/content/early/2019/04/30/623371.full.pdf. Σb 12A. Coulon, O. Gandrillon, and G. Beslon, “On the spontaneous stochastic and dynamics of a single gene: complexity of the molecular interplay at the promoter,” BMC Syst. Biol. 4 (2010), 10.1186/1752-0509-4-2. Σ α θ − α 13A. Sánchez and J. Kondev, “Transcriptional control of A−1 = b F (α +1,β,w )+ 1 + F (α,β,w ). σ ρ 1 1 1 ρ − ρ 1 1 1 noise in gene expression,” PNAS 105, 5081–5086 (2008), b u u b https://www.pnas.org/content/105/13/5081.full.pdf. (A21) 14T. C. Voss and G. L. Hager, “Dynamic regulation of transcriptional states Here w1 denotes w evaluated at z = 1. The generating function by chromatin and transcription factors,” Nat. Rev. Genet. 15, 69–81 (2014). 15 is then obtained by g(z)= g0(z) + g1(z). So, T. C. Voss, R. L. Schiltz, M.-H. Sung, T. A. Johnson, S. John, and G. L. Hager, “Combinatorial probabilistic chromatin interactions pro- hΣ α duce transcriptional heterogeneity,” J. Cell Sci. 122, 345–356 (2009), ρb(z−1) b g(z)= Ae 1F1(α + 1,β,w) https://jcs.biologists.org/content/122/3/345.full.pdf. σb ρu 16A. Sánchez, H. G. Garcia, J. D., R. Phillips, and J. Kondev, “Effect of pro- θ − α i moter architecture on the cell-to-cell variability in gene expression,” PLoS + 1 + F (α,β,w) (A22) ρ − ρ 1 1 Comput. Biol. 7 (2011), 10.1371/journal.pcbi.1001100. u b 17S. A. Sevier, D. A. Kessler, and H. Levine, “Mechanical bounds to tran- It follows that the stationary probability mass function is: scriptional noise,” Proc. Natl. Acad. Sci. 113, 13983–13988 (2016). 18T. B. Kepler and T. C. Elston, “Stochasticity in transcriptional regulation: origins, consequences, and mathematical representations,” Biophys. J. 81, A n n Rmα(m) n−m −ρb 3116–3136 (2001). p˜(n) = ρ e 19 n! ∑ m b Σmβ (m) P. Thomas, N. Popovic,´ and R. Grima, “Phenotypic switching in gene reg- m=0 b ulatory networks,” Proceedings of the National Academy of Sciences of the " United States of America 111, 6994–6999 (2014). Σb(α + m) 20 1F1(α + m + 1,β + m,w0) M. S. Sherman, K. Lorenz, M. H. Lanier, and B. A. Cohen, “Cell-to-cell σbρu variability in the propensity to transcribe explains correlated fluctuations in # gene expression,” Cell Systems 1, 315–325 (2015). θ − α 21O. Lenive, P. D. W. Kirk, and M. P. H. Stumpf, “Inferring extrinsic noise + 1 + 1F1(α + m,β + m,w0) . (A23) from single-cell gene expression data using approximate bayesian compu- ρu − ρ b tation,” BMC Syst. Biol. 10, 81 (2016). bioRxiv preprint doi: https://doi.org/10.1101/2020.01.05.895359; this version posted January 6, 2020. The copyright holder for this preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under aCC-BY-NC-ND 4.0 International license. 15 22C. V. Harper, B. Finkenstadt, D. J. Woodcock, S. Friedrichsen, S. Sem- 48D. A. Berry and B. W. Lindgren, Statistics: Theory and Methods prini, L. Ashall, D. G. Spiller, J. J. Mullins, D. A. Rand, J. R. E. Davis, (Brooks/Cole Publishing Company, 1990). and M. R. H. White, “Dynamic analysis of stochastic transcription cycles,” 49M. F. Weber and E. Frey, Reports on progress in physics. Physical Society PLoS Biology 9 (2011), 10.1371/journal.pbio.1000607. (Great Britain) 80, 046601 (2017). 23D. M. Suter, N. Molina, D. Gatfield, K. Schneider, U. Schi- 50J. W. Pearson, S. Olver, and M. A. Porter, “Numerical methods for the bler, and F. Naef, “Mammalian genes are transcribed with computation of the confluent and Gauss hypergeometric functions,” Numer. widely different bursting kinetics,” Science 332, 472–474 (2011), Algorithms 74, 821–866 (2017). https://science.sciencemag.org/content/332/6028/472.full.pdf. 24B. Zoller, D. Nicolas, N. Molina, and F. Naef, “Structure of silent transcrip- tion intervals and noise characteristics of mammalian genes,” Mol. Syst. Biol. 11 (2015). 25J. Dattani, Exact solutions of master equations for the analysis of gene tran- scription models, Ph.D. thesis, Imperial College London (2016). 26U. Herbach, “Stochastic gene expression with a multistate promoter: Breaking down exact distributions,” SIAM J. Appl. Math 79, 1007–1029 (2019). 27T. Zhou and J. Zhang, “Analytical results for a multistate gene model,” SIAM J. Appl. Math. 72, 789–818. (2012). 28J. Zhang, L. Chen, and T. Zhou, “Analytical distribution and tunability of noise in a model of promoter progress,” Biophys J. 102, 1247–1257 (2012). 29R. Grima, D. R. Schmidt, and T. J. Newman, “Steady-state fluctuations of a genetic feedback loop: an exact solution,” J. Chem. Phys. 137, 035104 (2012). 30G. C. P. Innocentini, A. F. Ramos, and J. E. M. Hornos, “A comment on “steady-state fluc- tuations of a genetic feedback loop: an exact solution”,” J. Chem. Phys. 142 (2015). 31V. Shahrezaei and P. S. Swain, “Analytical distributions for stochastic gene expression,” Proc. Natl. Acad. Sci. 105, 17256–17261 (2008). 32C. Zhixing, T. Filatova, D. A. Oyarzún, and R. Grima, “Multi-scale bursting in stochastic gene expression,” bioRxiv:10.1101/717199v1 (2019), 10.1101/717199, https://www.biorxiv.org/content/early/2019/07/28/717199.full.pdf. 33G. C. P. Innocentini, M. Forger, A. F. Ramos, O. Radulescu, and J. E. M. Hornos, “Multimodality and flexibility of stochastic gene expression,” Bul- letin of Mathematical Biology 75, 2600–2630 (2013). 34D. Schnoerr, G. Sanguinetti, and R. Grima, “Approximation and infer- ence methods for stochastic biochemical kinetics–a tutorial review,” J. Phys. Math. Theor. 50, 093001 (2017). 35C. Gardiner, Stochastic Methods: A Handbook For The Natural And Social Sciences (Springer, 2009). 36D. T. Gillespie, A. Hellander, and L. R. Petzold, “Perspective: Stochastic algorithms for chemical kinetics,” J. Chem. Phys. 138, 05B201_1 (2013). 37G. Neuert, B. Munsky, R. Z. Tan, L. Teytelman, M. Khammash, and A. van Oudenaarden, “Systematic Identification of Signal-Activated Stochastic Gene Regulation,” Science 339, 584–587 (2013). 38L. A. Sepúlveda, H. Xu, J. Zhang, M. Wang, and I. Golding, “Measure- ment of gene regulation in individual cells reveals rapid switching between promoter states,” Science 351, 1218–1222 (2016). 39G. Corliss and Y. F. Chang, “Solving Ordinary Differential Equations Using Taylor Series,” ACM Trans. Math. Softw. 8, 114–144 (1982). 40D. T. Gillespie, “Exact stochastic simulation of coupled chemical reac- tions,” J. Phys. Chem. 81, 2340–2361 (1977). 41L. Huang, Z. Yuan, P. Liu, and T. Zhou, “Effects of promoter leakage on dynamics of gene expression,” BMC Syst Biol 9, 16 (2015). 42A. Sanchez and I. Golding, “Genetic Determinants and Cellular Constraints in Noisy Gene Expression,” Science 342, 1188–1193 (2013). 43C. R. Clapier and B. R. Cairns, “The Biology of Chromatin Remodeling Complexes,” Annu. Rev. Biochem. 78, 273–304 (2009). 44M. Abramowitz and I. A. Stegun, Handbook Of Mathematical Functions With Formulas, Graphs, And Mathematical Tables (U.S. Govt. Print. Off., 1965). 45M. Thattai and A. van Oudenaarden, “Intrinsic noise in gene regulatory networks,” Proceedings of the National Academy of Sciences 98, 8614– 8619 (2001). 46B. B. Kaufmann and A. van Oudenaarden, “Stochastic gene expression: From single molecules to the proteome,” Current Opinion in Genetics & Development 17, 107–112 (2007). 47B. Munsky, G. Neuert, and A. van Oudenaarden, “Using gene expression noise to understand gene regulation,” Science 336, 183–187 (2012).