Intrinsic Noise in Systems with Switching Environments

PHYSICAL REVIEW E 93, 052119 (2016)

Intrinsic noise in systems with switching environments

Peter G. Hufton,* YenTingLin,† Tobias Galla,‡ and Alan J. McKane§ Theoretical Physics, School of Physics and Astronomy, The University of Manchester, Manchester M13 9PL, United Kingdom (Received 2 December 2015; revised manuscript received 17 March 2016; published 10 May 2016) We study individual-based dynamics in ﬁnite populations, subject to randomly switching environmental conditions. These are inspired by models in which genes transition between on and off states, regulating underlying protein dynamics. Similarly, switches between environmental states are relevant in bacterial populations and in models of epidemic spread. Existing piecewise-deterministic Markov process approaches focus on the deterministic limit of the population dynamics while retaining the randomness of the switching. Here we go beyond this approximation and explicitly include effects of intrinsic stochasticity at the level of the linear-noise approximation. Speciﬁcally, we derive the stationary distributions of a number of model systems, in good agreement with simulations. This improves existing approaches which are limited to the regimes of fast and slow switching.

DOI: 10.1103/PhysRevE.93.052119

I. INTRODUCTION Kramers-Moyal and van Kampen expansions. The latter is also known as the linear-noise approximation (LNA) [1,2]. There is now a broad consensus that noise plays a crucial These methods are now used routinely for the analysis role in most dynamical systems in biology, chemistry, and in of individual-based models with intrinsic noise in the weak- the social sciences. The theory with which to describe these noise limit. Most applications so far focus on systems in stochastic processes is well established and has its roots in which the reaction rates are set by constant model param- statistical physics. Modelling tools such as master equations, eters, and the only time dependence is in the evolution Fokker-Planck equations, and Langevin dynamics are standard of the population of individuals itself. Recently, exceptions and can be found in a number of textbooks [1–3]. Much of have gained attention [6,22–26]. In these models, reaction this work focuses on processes between discrete interacting rates vary continuously and deterministically in time, for individuals, which can be members of a population in example, to capture periodically varying infection rates to epidemiology [4–6], atoms or molecules in chemical reaction model seasonal variation in epidemic spreading. Crucially, systems [7,8], or proteins in the context of gene regulatory no additional stochasticity is introduced in these dynamics by networks [9,10]. Many of these models are Markovian and the environment, and the only discreteness in the dynamics their natural description is in terms of a master equation which is in the evolution of a finite population of individuals. Other describes the time evolution of the underlying probability authors have considered models with an environment which distribution of microstates. varies stochastically and continuously [27–29]. Solving the master equation analytically is often a difficult For many model systems it is more realistic to assume that task: Only very simple linear model dynamics allow further model parameters switch between different discrete states. treatment [1–3,11,12]. It is therefore common to employ This includes phases of antibiotic treatment in the context approximation techniques, most notably ones built around of bacteria [30,31], genetic switches [9,32–38], evolutionary the assumption that the size of the population is large but game theory [39], and predator-prey models in switching finite. The inverse system size (or its square root) then environmental conditions [40,41]. Such models describe two serves as a small parameter in which an expansion can be types of discreteness: that of the state of the environment and performed. The lowest order in this expansion corresponds that of the population of interacting individuals. In principle, to the limit of infinite systems and provides a deterministic the environmental switching can occur stochastically or follow description devoid of stochasticity. The second-order term in a deterministic pattern (e.g., prescribed periods of antibiotic the expansion introduces some stochastic effects of noise in the treatment, regularly interspersed with periods of no antibiosis). population but approximates the individual-level dynamics by In the present work we focus on stochastically switching a simpler Gaussian process on a continuum domain [13]. These environments. Assuming again a large but finite population, techniques have been very successful in capturing elements of the demographic noise in the population can be approximated noise-induced phenomena, for example, the weak selection using the above expansion techniques. The noise relating to of competing species [14–18], cyclic behavior, patterns, and switches in the environmental state, however, cannot be dealt waves [19–21]. The main techniques used to characterize these with in this way: There is no large parameter to expand in effects are system-size expansion methods, most notably the when the number of environmental states is small. In models with switching environments the lowest-order expansion in the strength of the intrinsic noise leads to a “piecewise-deterministic Markov process” (PDMP) [42,43]. *[email protected] In this approximation the dynamics of the population of †[email protected] individuals is described by deterministic rate equations be- ‡[email protected] tween stochastic switches of the environment. This approach §[email protected] neglects all intrinsic stochasticity from the reaction dynamics

2470-0045/2016/93(5)/052119(13) 052119-1 ©2016 American Physical Society HUFTON, LIN, GALLA, AND MCKANE PHYSICAL REVIEW E 93, 052119 (2016) within the population—the population scale is taken to be environment, often labeled as G0 and G1, correspond to infinite. The only type of randomness retained is that of regimes in which a certain promoter—a region on DNA which the switching of the environmental states. The application initiates transcription—is either inactive (G0) or activated of PDMPs has recently gained attention in the description (G1). A protein (A) is produced with rates b0 and b1 in the of genetic networks [32,44,45]. corresponding state. Independently of the state of the gene, The PDMP approximation has been surprisingly effective proteins degrade at a constant rate d. This simple model has in modeling systems with very large populations [32,45,46]; been studied, for example, in Refs. [9,37,38,45,47], and it can however, it does not produce accurate results outside this be written in the form limit. Recently, an alternative approach has incorporated some b0 b1 effects of demographic noise, but it is only valid if there is a G0 −−→ G0 + A, G1 −−→ G1 + A, very large separation between the time scales of environmental (2) G −λ→+ G G −λ→− G A −→d ∅ switching and that of the population dynamics [37,38]. Here, 0 1, 1 0, . we develop the theory further and construct a systematic The reaction rates of this model are linear in n. It is possible expansion in the noise strength about the PDMP. to develop exact solutions to linear models of this type using a The remainder of this paper is organized as follows: In generating-function approach [11,12,47–50]. However, such Sec. II we give a more formal introduction to the problem using solutions are often limited to simple model systems and a relatively simple linear model. We show how the system-size frequently they only provide limited insight into the actual expansion can be applied in the presence of switching envi- physical dynamics. We use this linear model to introduce our ronments and we analytically derive the resulting stationary approximation method. In later sections we will then apply distribution for the linear model. In Sec. III we construct a more this approach to cases with nonlinear reaction rates, where an general theory and describe how the method can be applied to a exact solution is no longer feasible. larger set of model dynamics. Section IV contains applications to a number of model systems with nonlinear reaction rates and/or other additional features. In Sec. V we summarize our B. Simulation of results and general behavior findings and give an outlook on future work. The Appendix To illustrate the general behavior of the model, we first contains further details of the relevant calculations. show the outcome of a set of characteristic simulations in Fig. 1. Figures 1(a), 1(c), and 1(e) depict individual simulation II. INTRODUCTORY EXAMPLE runs. In each of these panels a trajectory of the population density x(t) = n(t)/ is shown as a solid line, and the A. Model definition switching of the environmental states is indicated by the We first focus on a simple example with linear reaction shading of the background. Figure 1(a) shows an example rates. We consider a population of individuals of type A, and of relatively slow switching. In each environmental state, we write n for the number of individuals in the population at ∗ = the population tends to a fixed point, φσ bσ /d, specific to any given time. Individuals can be created and they can decay, the environment. It then fluctuates about this fixed point. The so the model describes a birth-death process. The death rate stationary distribution [Fig. 1(b)] is bimodal. As the switching per individual is assumed to be a constant d. The creation rate rate is increased, Figs. 1(c) and 1(d), the stochastic dynamics is taken to depend on the state of the external environment; spends more time in between the two fixed points and the individuals are born with rate bσ , where σ represents the bimodality of the stationary distribution is lost; we observe a state of the environment. This can be summarized as follows: nearly flat distribution between the two fixed points. At very

bσ fast switching, Figs. 1(e) and 1(f), the stationary distribution ∅ −−→ A, becomes unimodal, peaked at a value between the two fixed d A −→ ∅, (1) points. The system spends most of its time fluctuating about a point in the interior of phase space, away from either of the for times at which the environment is in state σ. The parameter two fixed points. has been introduced as per normal convention to set a typical We set out to characterize this behavior analytically, and our scale of the population size [1,2]. If the environment were to aim is to approximate the stationary outcome of the dynamics. be fixed at σ , then the number of individuals in the system We will develop a general approximation technique, which would fluctuate around the value bσ /d in the long run. These is applicable to a wider class of models, in the next section. 1/2 fluctuations can be expected to be of order and reflect the Before we do this, it is useful to outline our approach and to demographic stochasticity. describe the main steps of the analysis in the simpler model At any given time, the state of the full system is completely defined in Eqs. (2). described by the state σ of the environment and the number of individuals in the population n. We restrict ourselves to cases in which the environment has two states, σ ∈{0,1}.The C. Master equation switching between these states is assumed to be independent We write n(t) for the number of individuals at time t of the state of the population n and it occurs with constant and σ (t) for the state of the environment. These follow rates. We write λ+ for the rate of switching from state 0 to random-jump Markov processes [2]. We write P (n,σ,t)for state 1 and λ− for the rate of switching from state 1 to state 0. the probability to find the system and environment in state This stylized model can be interpreted in the context (n,σ ) at time t. We will frequently suppress the explicit time of genetic networks as follows [9]. The two states of the dependence to keep the notation compact.

052119-2 INTRINSIC NOISE IN SYSTEMS WITH SWITCHING . . . PHYSICAL REVIEW E 93, 052119 (2016)

2 3.5 (a) (b) 3 1.5 2.5

) 2 t

( 1

x 1.5

0.5 1 probability density 0.5 0 0 0 10203040500 0.5 1 1.5 2 t x 2 3.5 (c) (d) 3 1.5 2.5

) 2 t

( 1

x 1.5

0.5 1 probability density 0.5 0 0 0 10203040500 0.5 1 1.5 2 t x 2 3.5 (e) (f) 3 1.5 2.5

) 2 t

( 1

x 1.5

0.5 1 probability density 0.5 0 0 0 10203040500 0.5 1 1.5 2 t x

FIG. 1. Sample paths of the dynamics and the stationary distribution of the linear birth-death process described by Eq. (2). Panels (a) and (b) show the regime of slow environmental switching (λ+ = λ− = 0.1), panels (c) and (d) show intermediate environmental switching (λ+ = λ− = 1), and panels (e) and (f) show fast environmental switching (λ+ = λ− = 10). Panels (a), (c), and (e) show individual trajectories (solid line) and the state of the environment (shaded background). Panels (b), (d), and (f) on the right show the stationary distribution from simulations, averaged over multiple runs (histogram). We also plot the theoretical prediction for the stationary distribution ∗(x) obtained fromEqs.(16), shown as solid lines for the three scenarios. The trajectories and stationary distributions have been obtained by application of the Gillespie algorithm [51,52]. Model parameters are b0 = 1/3, b1 = 5/3, d = 1, and = 150.

The master equation governing the time evolution of this joint stationary distribution of the number of individuals n and distribution then can be written as environmental state σ , P ∗(n,σ ). d P (n,0) = L0P (n,0) − λ+P (n,0) + λ−P (n,1), dt (3) D. Approximation of the master equation d P (n,1) = L P (n,1) + λ+P (n,0) − λ−P (n,1). We now proceed to approximate the above master equation. dt 1 To this end, it is useful to define the population density x(t) = These equations consist of two components. The operators L0 n(t)/. Assuming that the typical system size is large but and L1 characterize the creation and removal of individuals finite, the operators, L0 and L1, can be approximated by a assuming a fixed state of the environment. They are given by Taylor expansion with respect to −1, L = b (E−1 − 1) + d(E − 1)n, (4) σ σ 1 ≈ L =− − + 2 + = + Lσ σ ∂x (bσ xd) ∂x (bσ xd). (5) where E is the shift operator [1]: Ef (n) f (n 1). It is 2 important to note that operators, such as E or Lσ , act on everything that follows to their right throughout our paper, This is the Kramers-Moyal expansion [1,2] in terms of the e.g., Enf(n) = (n + 1)f (n + 1). The latter two terms in the variable x = n/, while keeping the state of the environment, master equation describe the switching between the two σ , discrete. We write Lσ for the operators obtained from this environmental states. expansion, retaining only leading and subleading orders in The master equation (3) describes the time evolution of −1. The state of the system is now expressed in terms of x the full process, and in our analysis, we wish to calculate the and σ , and we will write (x,σ) for the resulting probability

052119-3 HUFTON, LIN, GALLA, AND MCKANE PHYSICAL REVIEW E 93, 052119 (2016) density. The explicit time dependence is again suppressed in this notation. Substituting this into the master equation (3) allows us to approximate the process by

∂t (x,0) = L0(x,0) − λ+(x,0) + λ−(x,1), (6) ∂t (x,1) = L1(x,1) + λ+(x,0) − λ−(x,1). This expansion differs from the standard Kramers-Moyal expansion in that the environmental states are not included FIG. 2. Illustration of the Liouville flow of the model described in the expansion. Equation (6) accordingly is not a standard by Eq. (7). Arrows indicate the Liouville flow in each of the two Fokker-Planck equation; it retains the discrete switching environments, and stable fixed points are shown as filled circles. terms, akin to terms in the master equation of a conventional The PDMP converges into the interval between the two fixed points telegraph process. Equations (6) describe a diffusion process (shaded region) at long times. with a Markovian switching in between two sets of drift and diffusion [53,54]. section. For the linear model defined in Eqs. (1) we find Since Eqs. (6) contain multiplicative noise, it is difficult to solve these equations directly. In the following sections, we ∗ λ+ ∗ λ− (φ − φ ) d (φ − φ) d propose an approximation method to analyze this dynamics. ∗(φ,0) = N 0 1 , φ − φ∗ Our approach is similar to the conventional linear-noise 0 (8) ∗ λ+ ∗ λ− approximation, and describes the effects of intrinsic noise to (φ − φ ) d (φ − φ) d ∗ φ, = N 0 1 , subleading order. ( 1) ∗ − φ1 φ ∈ ∗ ∗ N E. Leading-order approximation: Piecewise-deterministic for φ (φ0 ,φ1 ). The prefactor is a normalization constant, Markov process determined by the condition In the above expansion we have retained leading and φ∗ −1 1 subleading terms in . To proceed, it is useful to first [∗(φ,0) + ∗(φ,1)] dφ = 1. (9) ∗ consider the leading-order terms only, i.e., to study the limit φ0 →∞. We obtain This result is consistent with those reported by other au- =− − ∂t (φ,0) ∂φ(b0 dφ)(φ,0) thors [45,55,56]. − λ+(φ,0) + λ−(φ,1), (7) F. Comparison against simulations ∂t (φ,1) =−∂φ(b1 − dφ)(φ,1) In Fig. 3 we illustrate the behavior of the PDMP. + λ+(φ,0) − λ−(φ,1), Figures 3(a), 3(c), and 3(e) show individual time series where we have written φ = lim→∞ n/ to indicate that we for slow, medium, and fast switching of the environment have taken the limit of infinite populations. In this limit the (parameters are as in Fig. 1). Figures 3(b), 3(d), and 3(f) Lσ are Liouville operators describing deterministic flow of depict the corresponding stationary distributions of the PDMP, φ. The functional form of this flow at any one time is entirely as obtained from Eqs. (8). For slow switching rates, the determined by the state of the environment, σ. The process φ(t) marginal stationary distribution ∗(φ) = ∗(φ,0) + ∗(φ,1) is a piecewise deterministic Markov process [42,43,53,55], a is bimodal, with singularities at the end points. In this regime, random process composed of deterministic motion in between the PDMP typically spends enough time in each environment the discrete environmental transitions. The PDMP is a descrip- between switches to come close to the corresponding fixed tion of the system which accounts for the stochasticity of the point. For fast switching ∗(φ) is unimodal. In this situation, switching only; the demographic noise, on the other hand, is the system typically does not have sufficient time to reach the neglected in the above truncation after the leading-order term. vicinity of the fixed points. An intermediate case is shown in The long-term behavior of the system can be determined Figs. 3(c) and 3(d). For this particular choice of parameters, from inspection of the Liouville operators. In each state σ the the resulting stationary distribution is seen to be flat between ∗ = ∗ ∗ process tends towards a stable fixed point, φσ bσ /d.Inthe the two fixed points, φ and φ . ∗ ∗ 0 1 following, we assume that b0

052119-4 INTRINSIC NOISE IN SYSTEMS WITH SWITCHING . . . PHYSICAL REVIEW E 93, 052119 (2016)

2 3 (a) (b) 2.5 1.5 2 ) t

( 1 1.5 φ 1 0.5 probability density 0.5

0 0 0 10203040500 0.5 1 1.5 2 t φ 2 3 (c) (d) 2.5 1.5 2 ) t

( 1 1.5 φ 1 0.5 probability density 0.5

0 0 0 10203040500 0.5 1 1.5 2 t φ 2 3 (e) (f) 2.5 1.5 2 ) t

( 1 1.5 φ 1 0.5 probability density 0.5

0 0 0 10203040500 0.5 1 1.5 2 t φ

FIG. 3. Sample paths and stationary distribution of the PDMP for the linear model in infinite populations. Panels (a) and (b) show the regime of slow environmental switching (λ+ = λ− = 0.1), panels (c) and (d) show intermediate environmental switching (λ+ = λ− = 1), and panels (e) and (f) show fast environmental switching (λ+ = λ− = 10). Panels (a), (c), and (e) show individual trajectories (solid line) and the state of the environment (shaded background). Panels (b), (d), and (f) show the stationary distribution of the PDMP, as obtained from from Eq. (8). Comparison with Fig. 1 shows that this stationary distribution does not adequately reflect the stationary distribution of the full noisy process in finite populations. The sample paths have been obtained by Runge-Kutta integration of Eq. (11) between randomly generated exponential switching times. Model parameters are b0 = 1/3, b1 = 5/3, d = 1, and = 150. of the stationary distribution of the model with intrinsic noise process, σ (t), we decompose the dynamics of the population ∗ ∗ includes concentrations below φ0 and above φ1 . as follows: These results stress the significance of intrinsic noise for the dynamics of the system. In order to approximate the stationary n 1 = φ(t) + √ ξ(t). (10) behavior to a better accuracy, it is necessary to include higher-order terms in the above Kramers-Moyal expansion. The corresponding formalism is well established for systems This is along the lines of the system-size expansion in systems without random switches of the environment, and it takes with time-dependent rates [26,57]. For a given path of the the form of an expansion about the deterministic path of the environment, σ (t), the trajectory φ(t) of the resulting PDMP infinite system [26]. In our case, the leading-order behavior is given by the solution of (the PDMP) is a stochastic process itself due to the randomness of the environmental switching. The subleading description φ˙(t) = b − dφ(t), (11) we discuss below is hence an expansion about this random σ (t) process. where the birth rate bσ (t) at time t is determined by the state of the environment at that time, σ (t). The dynamics of Eq. (11)is G. Subleading order: Linear-noise approximation the equivalent of the usual rate equations for systems without Before we present a more detailed account of the expansion environmental switching. to subleading order (Sec. III), we briefly outline the general To subleading order, systems of this form, but without en- idea. For a fixed realization of the environmental switching vironmental switching, are described by stochastic differential

052119-5 HUFTON, LIN, GALLA, AND MCKANE PHYSICAL REVIEW E 93, 052119 (2016) equations of the form 2 w(x) x˙(t) = v(x) + η(t), (12) 1.5 ) t

( 1 where we have suppressed the obvious time dependence of x x on the right-hand side. The dependence of the amplitude w(x) 0.5 on x indicates multiplicative noise, and η(t)isGaussianwhite noise of unit amplitude, i.e., η(t)η(t ) =δ(t − t ). The term 0 0 20 40 60 80 100 v(x) represents deterministic drift. In a birth-death process t with fixed birth rate b and death rate d one would have v(x) = b − dx, for example. FIG. 4. Sample path of the PDMP of the linear model [φ(t), The equivalent of Eq. (12) for the model with environmental solid line] and environmental state σ (t) (background shading). switching is given by The dynamics in the finite system for the same realisation of the environmental process will deviate from the PDMP. The standard w (x) deviation of the deviation is approximated by Eq. (36) as discussed x˙(t) = v (x) + σ (t) η(t). (13) σ (t) below and shown here as shading around the PDMP trajectory. Parameters are b0 = 1/3, b1 = 5/3, d = 1, λ+ = λ− = 1, and = Equation (13) is similar to the outcome of the procedure 150. described in Refs. [26,57], where the system-size expansion was carried out for systems with periodically driven rates. From these approximations the stationary distribution of The main difference is the fact that σ(t) is now a stochastic x = φ + −1/2ξ then can be estimated as process itself, whereas the external driving was deterministic in Ref. [26]. In the limit →∞ the noise η(t) does not ∗(x) = dφ dξ[∗(ξ|φ)∗(φ,σ) contribute, and we recover the PDMP dynamics of Eq. (11). For the case of the linear model described above we have σ × − − −1/2 vσ (x) = bσ − dx and wσ (x) = bσ + dx. These are the drift δ(x φ ξ)]. (16) term and noise amplitude one would obtain from a standard Returning to Fig. 1, we compare the stationary distribution Kramers-Moyal expansion at fixed environmental state σ . of the linear model as obtained from Eq. (16) against the results In the spirit of the usual linear-noise√ approximation (LNA) of numerical simulations. The results from the theory are we next write x(t) = φ(t) + ξ(t)/ and obtain shown as solid lines, and the simulation data as histograms. We find that the approximation reproduces the numerical results φ˙(t) = vσ (t)(φ), (14) to a good accuracy. ˙ = + ξ(t) vσ (t)(φ)ξ(t) wσ (t)(φ)η(t). = III. GENERAL FORMALISM We have written vσ dvσ (φ)/dφ, and we have suppressed the time dependence of φ on the right-hand side. A. Definition and master equation From the LNA it is possible to proceed and to approximate the stationary distribution of the process, ∗(ξ,φ,σ) = The linear model discussed so far was deliberately simple ∗(ξ|φ,σ)∗(φ,σ). The distribution ∗(φ,σ) is the station- and the main purpose of studying it was to develop a general ary outcome of the PDMP, and it can be computed exactly, see intuition. In order to extend the method beyond the linear case, Eqs. (8) for the linear birth-death model. The general case is we now consider a more general model. This will introduce discussed in the following section. several new aspects to the problem. In the stationary regime the distribution of ξ will, in As before, we restrict our discussion to the case of a single species and two environmental states. We write n for the principle, depend on the state σ of the environment and on ∈{ } the variable φ. In order to proceed we now assume that the number of individuals in the population, and σ 0,1 for dependence on σ can be neglected, so we write ∗(ξ|φ,σ) ≈ the state of the environment. We use the notation λ+(n)forthe ∗(ξ|φ). This is an approximation, but it turns out to work rate with which the environment switches from state 0 to state well for all models we have tested. Making this assumption 1, and λ−(n) for the rate of switches in the opposite direction. we write Unlike in the previous sections, we now allow for an explicit dependence of these rates on the state of the population, ∗(ξ,φ,σ) ≈ ∗(ξ|φ) ∗(φ,σ). (15) λ± = λ±(n). We assume that there are M possible reactions in the The distribution ∗(ξ|φ) is Gaussian with mean zero, and with population, labeled m = 1,...,M. Each reaction m occurs a variance which depends on φ and which can be obtained with rate am,σ (n) dependent on the current state of the analytically as described in the next section. The resulting environment and population. Any occurrence of a reaction picture is illustrated in Fig. 4. A given realization of the of type m is taken to change the number of individuals in environmental process generates a realization of the PDMP. the population by Sm. These are the underlying stoichiometric The state of a finite population, subject to the same path of coefficients. The set of propensity functions, am,σ (n), together environmental states, will fluctuate about the PDMP trajectory, with the stoichiometric coefficients completely define the as indicated by the shading in Fig. 4. dynamics of the population.

052119-6 INTRINSIC NOISE IN SYSTEMS WITH SWITCHING . . . PHYSICAL REVIEW E 93, 052119 (2016)

The master equation describing the time evolution of P (n,σ ) then reads d P (n,0) = L P (n,0) − λ+(n)P (n,0) + λ−(n)P (n,1), dt 0 d P (n,1) = L P (n,1) + λ+(n)P (n,0) − λ−(n)P (n,1). dt 1 (17)

The operators L0 and L1 are given by

M −Sm Lσ = (E − 1)am,σ (n). (18) FIG. 5. Illustration of the physical interpretation of the currents m=1 Jσ (φ) (see text). Anticipating the system-size expansion, we write the tran- φ x sition rates in the form a (n) = r (x), where x = n/ As before we write instead of in the limit of an infinite m,σ m,σ system, and we find the PDMP dynamics and where rm,σ carries no explicit dependence. In slight abuse of notation we will write λ±(x) instead of λ±(x). The ∂t (φ,0) =−∂φ[v0(φ)(φ,0)] master equation can be expressed in terms of (x,σ,t), the − + probability density of finding the random processes at x,σ at λ+(φ)(φ,0) λ−(φ)(φ,1), time t. The dynamics of the joint probability distribution can ∂t (φ,1) =−∂φ[v1(φ)(φ,1)] be approximated by + λ+(φ)(φ,0) − λ−(φ)(φ,1). (23) = L − + ∂t (x,0) 0(x,0) λ+(x)(x,0) λ−(x)(x,1), This indicates a flow of the form

∂t (x,1) = L1(x,1) + λ+(x)(x,0) − λ−(x)(x,1), φ˙ = vσ (φ) (24) (19) in between switches of the environment. These switches in turn occur with rates λ±(φ). where the operators L0 and L1 are given by We now proceed by assuming that the deterministic flow in M S 2 each of the environments is towards a unique fixed point, i.e., L = −S ∂ + m ∂2 r (x). (20) that there are points φ∗ for σ ∈{0,1} such that σ m x 2 x m,σ σ m=1 ∗ = vσ (φσ ) 0. (25) As before, we have truncated the expansion of the operators in − For the time being we assume that there is only one such fixed powers of 1 after the subleading terms. In explicit form we point per state; for a more general case see Sec. IV C.We have ∗ ∗ assume φ0 <φ1 without loss of generality. After a potential 1 transient the PDMP will eventually be confined to the interval L =− + 2 σ ∂x vσ (x) ∂x wσ (x), (21) ∗ ∗ 2 (φ0 ,φ1 ). Our assumption above (only one fixed point in each environmental state) implies that the vσ do not change sign where we have introduced ∈ ∗ ∗ on this interval, v0(φ) < 0 and v1(φ) > 0forφ (φ0 ,φ1 ): ∗ = M the flow is always towards fixed point φ0 in state σ 0, and towards φ∗ in state σ = 1, as also illustrated in Fig. 2. vσ (x) = Smrm,σ (x), 1 m=1 To analyze the PDMP further, it is useful to introduce (22) currents M = 2 wσ (x) Smrm,σ (x). J0(φ) = v0(φ)(φ,0) = m 1 φ − [−λ+(u)(u,0) + λ−(u)(u,1)]du, Again, the process described by Eqs. (19) contains multiplica- φ∗ tive noise and it is difficult to solve these equations in general. 0 J1(φ) = v1(φ)(φ,1) In the next section, we begin by analyzing the dynamics in the limit of an infinite population. φ − [λ+(u)(u,0) − λ−(u)(u,1)]du. (26) ∗ φ0 B. Leading-order approximation: Piecewise-deterministic The physical interpretation of these currents is illustrated in Markov process ∗ ∗ ∗ Fig. 5. We focus on a domain (φ0 ,φ), where φ0 φ φ1 . We first analyze the process in the limit of infinite system The first term of the right-hand side of Eqs. (26) accounts for size. Here we outline the main steps of the analysis and give the probability flowing out of such a domain at location φ the central results. The details of the calculation can be found due to the Liouville flow in environmental state σ .Theterm in Appendix A. containing the integral describes the net flow of probability

052119-7 HUFTON, LIN, GALLA, AND MCKANE PHYSICAL REVIEW E 93, 052119 (2016) out of the domain due to switching between the environmental These relations describe the evolution of the population (within states. Thus, the quantity Jσ (φ + φ) − Jσ (φ) represents the the LNA) between switches of the environment. When a total amount of probability leaving the interval (φ,φ + φ) change of the environment occurs the variable σ (t) changes at per unit time. a discrete point in time, and the next such interval of constant This leads to equations of continuity σ begins. Within the LNA the evolution of the probability to observe ∂ φ,σ =−∂ J φ . t ( ) φ σ ( ) (27) state φ,ξ,σ is described by the following set of equations: In the stationary state the currents are divergence free =− − ∗ = ∂t (φ,ξ,0) ∂φ[v0(φ)(φ,ξ,0)] v0(φ) ∂ξ [ξ(φ,ξ,0)] (∂φJσ 0). Using the zero-current boundary conditions at ∗ ∗ ∗ ≡ w (φ) φ0 and φ1 , we find Jσ (φ) 0 throughout. Summing the + 0 2 − ∗ + ∗ = ∂ξ (φ,ξ,0) λ+(φ)(φ,ξ,0) two stationary currents, J0 (φ) J1 (φ) 0, we immediately 2 ∗ =− ∗ find (φ,1) [v0(φ)/v1(φ)] (φ,0), which allows one to + λ−(φ)(φ,ξ,1), replace ∗(φ,1) in favor of ∗(φ,0) (or vice versa) in the =− − zero-current conditions. This results in two closed equations ∂t (φ,ξ,1) ∂φ[v1(φ)(φ,ξ,1)] v1(φ) ∂ξ [ξ(φ,ξ,1)] ∗ ∗ for (φ,0) and (φ,1), respectively. These can then be w1(φ) 2 + ∂ (φ,ξ,1) + λ+(φ)(φ,ξ,0) integrated directly, and we find 2 ξ N − λ−(φ)(φ,ξ,1). (33) ∗(φ,0) = h(φ), −v0(φ) (28) While the expressions on the right-hand side look complicated, N ∗ = the different terms have a clear physical meaning. The (φ,1) h(φ), − v1(φ) Liouvillian terms ∂φ[vσ (φ)(φ,ξ,σ)] describe the deter- ˙ = N ministic flow [φ vσ (φ)] between switches of the environ- where is a normalization constant. The function h(φ)is − + ment. The Fokker-Planck-like terms vσ (φ)∂ξ [ξ(φ,ξ,σ)] given by 2 wσ (φ)∂ξ (φ,ξ,σ)/2 capture the evolution of ξ within the φ λ+(u) λ−(u) LNA of Eq. (31). Finally, the terms proportional to λ±(φ) h(φ) ≡ exp − + du . (29) describe switching of the environment. Consistent with the v0(u) v1(u) expansion in the system-size we have replaced λ±(x)by Further details of the calculation are given in Appendix A. λ±(φ), i.e., any dependence of the switching rates on the state Later we will need the stationary conditional probability of the population is taken to be on the state of the PDMP φ. ∗(σ |φ)—the stationary probability of having environmental state σ given the population has state φ. As discussed above we ∗ ∗ ∗ D. Stationary state within the linear-noise approximation have v0(φ) (φ,0) + v1(φ) (φ,1) = 0. Writing (φ,σ) = ∗(φ)∗(σ |φ) and using ∗(0|φ) + ∗(1|φ) = 1inEq.(28) At stationarity the time derivatives on the left-hand side of leads to Eqs. (33) vanish. Using asterisks as before to denote stationary distributions, and writing ∗(φ,ξ,σ) = ∗(φ,σ)∗(ξ|φ,σ), ∗ v1(φ) (0|φ) = , we find v1(φ) − v0(φ) (30) =− ∗ | −v (φ) 0 (φ,0) v0(φ) ∂ξ [ξ(ξ φ,0)] ∗(1|φ) = 0 . 1 ∗ 2 ∗ v1(φ) − v0(φ) + | 2 (φ,0) w0(φ) ∂ξ (ξ φ,0) ∗ ∗ It is perhaps surprising that these conditional probabilities are − ∂φ[v0(φ) (φ,0) (ξ|φ,0)] independent of the switching rates λ+ and λ−. − ∗ | (φ,1) v1(φ) ∂ξ [ξ(ξ φ,1)] + 1 ∗ 2 ∗ | C. Subleading order: Linear-noise approximation 2 (φ,1) w1(φ) ∂ξ (ξ φ,1) ∗ ∗ We now proceed by including contributions of intrinsic − ∂φ[v1(φ) (φ,1) (ξ|φ,1)] (34) noise to subleading order. We focus on a time interval between switches of the environment, i.e., we assume σ is constant. from summing the two equations (33) at stationarity. During such time intervals the environmental noise has no At this point we introduce a further approximation. We ∗ | ≈ ∗ | ∗ | effect, and the problem reduces to that of a conventional assume (ξ φ,0) (ξ φ,1) and simply write (ξ φ) individual-based system with a fixed environment. Following for either of these. This will be justified below. Making this established procedures we write n/ = φ(t) + −1/2ξ(t). assumption leads to The deterministic dynamics is given by φ˙ = v (φ). The σ ∗ ∗ ∂ ∗ outcome of a standard LNA can be expressed as a linear 0 =−( (0|φ)v (φ) + (1|φ)v (φ)) ξ (ξ|φ) 0 1 ∂ξ Langevin equation, 2 1 ∗ ∗ ∂ ∗ ξ˙(t) = v (φ)ξ + w (φ)η(t), (31) + ( (0|φ)w0(φ) + (1|φ)w1(φ)) (ξ|φ), (35) σ σ 2 ∂ξ2 for fluctuations about φ(t), where ∗ where we have used the relation v0(φ) (φ,0) + ∗ = 2 v1(φ) (φ,1) = 0, valid at stationarity, to show that the terms wσ (φ) Smrm,σ (φ). (32) m containing derivatives with respect to φ cancel out.

052119-8 INTRINSIC NOISE IN SYSTEMS WITH SWITCHING . . . PHYSICAL REVIEW E 93, 052119 (2016)

∗ ∗ 1.5 in the region away from the two ﬁxed points φ0 and φ1 .Forthe 2 s0(φ) linear model, numerical results and the theoretical prediction 2 s1(φ) are virtually indistinguishable. 1 s2(φ)

IV. FURTHER EXAMPLES 0.5 A. Nonlinear reactions rates In order to demonstrate the generality of our approach, we 0 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 now proceed to a model system with nonlinear reaction rates. φ The system is identical to the one described in Eq. (2), except FIG. 6. The variance of ξ, conditioned on the state φ of the PDMP that the last reaction (removal of individuals) is replaced by and the state of the environment (see text for details). Results are d/ 2 2A −−→ ∅. (38) shown for the dimerization process of Sec. IV A. Solid lines are s0 (φ) and s2(φ) obtained from simulations (see text for definitions). The 1 The above notation indicates that this last reaction occurs with dashed line shows the approximation of Eq. (37). Parameters used rate dn(n − 1)/(2), where n is the number of individuals in are b = 0.667, b = 4.048, d = 2, λ+ = λ− = 1, and = 150. 0 1 the system. The reaction can be interpreted as a dimerization process, in which two particles of type A form a complex Equation (35) resembles a stationary Fokker-Planck equa- which is chemically inert and hence effectively removed. In tion. The drift and diffusion coefficients are obtained from the addition to the switches between G0 and G1, this system is vσ and wσ , weighted by the likelihood to find the environment described by two reactions, with stoichiometric coefficients in each of its states when the PDMP is at φ. and reaction rates Solving the stationary equation (35) is standard [3]; we ∗ =+ = find a Gaussian distribution, (ξ|φ), with zero mean and S1 1,r1,σ bσ , with variance d S =−2,r = x2. (39) 1 ∗(0|φ)w (φ) + ∗(1|φ)w (φ) 2 2,σ 2 s2 φ =− 0 1 . ( ) ∗ | + ∗ | (36) 2 (0 φ)v0(φ) (1 φ)v1(φ) We have written x = n/ as before. Using the notation of Using Eq. (30), this can be simplified to give the preceding sections, we have the following drift and the diffusion terms 1 w (φ)v (φ) − w (φ)v (φ) s2(φ) = 0 1 1 0 . (37) = − 2 = + 2 − vσ (x) bσ x d, and wσ (x) bσ 2x d. (40) 2 v0(φ)v1(φ) v1(φ)v0(φ) In the√ limit →∞, we obtain a PDMP with fixed points We conclude this section with a brief comment on the ap- ∗ ∗ ∗ = = proximation (ξ|φ,σ) ≈ (ξ|φ). In the fast-switching limit φσ bσ /d for σ 0,1. As before we assume b0

052119-9 HUFTON, LIN, GALLA, AND MCKANE PHYSICAL REVIEW E 93, 052119 (2016)

2 2 (a)

1.5 1.5 ) t

1 ( 1 x

0.5 0.5 probability density

0 0 0 0.5 1 1.5 2 0 20 40 60 80 100 x t 2.5 FIG. 7. Stationary distribution for the nonlinear model of (b) Sec. IV A. The solid line is the theoretical prediction, and the 2 histogram is data obtained from simulations using the Gillespie 1.5 algorithm. Parameters are b0 = 0.667, b1 = 4.048, d = 2, λ+ = λ− = 1, and = 150. 1

probability density 0.5 case in which the rates with which the environment switches 0 between states depends on the state of the population, i.e., 0 0.5 1 1.5 2 x λ+(n) G0 −−→ G1, (43) FIG. 8. Model in which rates of environmental switching depends λ−(n) on the state of the population. Panel (a) shows a typical time course, as G1 −−→ G0, generated by the Gillespie algorithm. Panel (b) depicts the stationary where we choose the linear form λ±(n) = α± + β±(n/). The distribution; the histogram is from simulations and the solid line coefficients α± and β± are constants, chosen such that λ± represents the approximation from our theory. Model parameters are ======remain non-negative. α± 0,β± 1,b0 1/3,b1 5/3,d 1, and 150. The drift and diffusion terms of the dynamics within the population are fixed points of the flows vσ (x). The switching between the two environmental states is taken to occur with constant rates λ±. vσ (x) = bσ − xd, and wσ (x) = bσ + xd, (44) Specifically, the population dynamics is now modelled by the as before. The stationary distribution of the PDMP in the limit following reactions: of infinite populations is obtained from Eqs. (28)as c1,σ ∗ ∗ G −−→ G + A + − κ+ − κ− σ σ , ∗ β+ β− φ (φ φ0 ) (φ1 φ) (φ,0) = N e d ∗ , − c2,σ φ φ0 G + A −→ G , (45) σ σ ∗ + ∗ − (47) β++β− − κ − κ ∗ φ (φ φ0 ) (φ1 φ) c3,σ / (φ,1) = N e d , G + 2A −−−→ G + 3A, φ∗ − φ σ σ 1 2 G + A −c−−−4,σ /→ G + A where the exponents κ± are given by σ 3 σ 2 ,

α± β±b0 with constant parameters ci,σ > 0. The corresponding stoi- κ± = + . (46) d d2 chiometric coefﬁcients for the four reactions are

The dynamics within the population is the same as in the linear S1 = S3 =+1,S2 = S4 =−1, (48) model above, so evaluating Eq. (37) again leads to s2(φ) = φ. and the propensities r (x) read The theoretical estimate of the stationary distribution ∗(x), m,σ finally, is again found from numerical integration of Eq. (16). r1,σ (x) = c1,σ ,r2,σ (x) = c2,σ x, Figure 8 shows a sample path of the dynamics and the c c = = r (x) = 3,σ x2,r(x) = 4,σ x3. (49) stationary distribution for the choice λ+(x) λ−(x) x (i.e., 3,σ 2 4,σ 6 α± = 0 and β± = 1). In this case, transitions between states are more likely at higher concentrations of the system. Again, Additional details of the model and the numerical values of we find good agreement between the predicted distribution the parameters we used for our analysis can be found in and the simulation results. It is important to note that the Appendix B. →∞ parameters we used in the figure are not special in any way: Again, we first consider the PDMP, i.e., the limit . We have tested other choices of the parameter, and we find an For the parameters chosen for our analysis one finds three ˙ = ∗ ≈ agreement between simulations and theory of a similar quality. fixed points of the dynamics φ v0(φ), at φ 0.20,0.90, and 1.6. We label these φ(1)∗,φ(3)∗, and φ(5)∗. These fixed points are linearly stable, unstable, and stable, respectively. In the C. Multiple fixed points environmental state σ = 1 we have fixed points φ∗ ≈ 0.4,1.1, In the final example, we consider more complicated and 1.8, labeled φ(2)∗,φ(4)∗,and φ(6)∗(again stable, unstable, dynamics within the population such that there are multiple and stable, respectively). This arrangement of fixed points is

052119-10 INTRINSIC NOISE IN SYSTEMS WITH SWITCHING . . . PHYSICAL REVIEW E 93, 052119 (2016)

remains close to the interval between φ(1)∗ and φ(2)∗ or near the interval between φ(5)∗ and φ(6)∗. Intrinsic noise will allow for excursions outside these intervals, similar to what was observed in Sec. IV A. The finite system can traverse the region between φ(2)∗ and φ(5)∗, at least in principle, and move from one of the dynamic modes to the other. This is similar to escaping from a basin of attraction in systems with constant environment. We generally expect that the rate with which this happens is exponentially suppressed in the noise strength. We will assume that such events are sufficiently rare so they can FIG. 9. Illustration of the Liouville flow in the two environmental be ignored for the purposes of our analysis. states σ = 0,1 for the model in Sec. IV C. Stable fixed points are The general approach we have developed can then be shown as filled circles, and unstable fixed points as open circles. The applied to the system in either of the two dynamic modes. shaded areas represent the two stable dynamic modes described in Due to the complexity of the flow fields vσ (x), the relevant the text. equations have to be evaluated numerically. In Fig. 10 we show the resulting predictions for the stationary distribution in either mode. As in the previous examples, comparison with illustrated in Fig. 9. The dynamics of the PDMP depends on data from numerical simulations shows very good agreement. the initial condition. If started between the fixed points φ(1)∗ and φ(2)∗, then the system will be confined between these two values and follow a dynamics similar to that of the system in V. CONCLUSION Sec. IV A. Similarly, if the initial condition is between the two (5)∗ (6)∗ In summary, we have constructed a systematic approach fixed points φ and φ , then the PDMP will operate in the with which to investigate the effects of demographic noise in interval between these two points. We will refer to these as the systems subject to sudden random switches of reaction rates. two “stable modes” of the PDMP dynamics. For other initial We have focused on relatively simple birth-death processes of conditions, the PDMP will eventually reach one of these two one single species and in which birth and death rates depend stable modes as well, and which one this is will depend on the on both the state of the population and on the state of an starting point and on the exact path the environment takes. external environment. The states of the environment follow a If the system is finite, then the dynamics are subject telegraph process, with transition rates which may depend on to intrinsic noise. Two representative trajectories are shown the state of the population. Previously existing approaches Fig. 10. Depending on initial conditions, the system either either disregard intrinsic fluctuations and only account for environmental noise or they focus on cases in which there 2 6 (a) (b) is a clear separation of time scales between the population 5 dynamics and the dynamics of the environment. The former 1.5 4 approach in particular leads to the well-established picture of ) t so-called piecewise deterministic Markov processes. Our work ( 1 3 x systematically improves on this view; we take into account 2 0.5 demographic fluctuations to leading order and carry out a probability density 1 system-size expansion and linear-noise approximation about 0 0 0 50 100 150 200 0 0.5 1 1.5 2 the PDMP dynamics. t x Using the linear-noise approximation, and retaining the 2 4 (c) (d) discreteness of the environmental process, we then approximate the resulting stationary distribution of the population 3 1.5 dynamics. We have tested the resulting theory on a number ) t of model systems, both with linear and nonlinear reaction ( 1 2 x rates, situations in which environmental switching depends 0.5 1 on the state of the population, and covering systems with probability density single dynamic modes and cases where there are multiple 0 0 0 50 100 150 200 0 0.5 1 1.5 2 attractors. In all cases we have tested, the approximation is in t x good agreement with results from simulations. In particular, FIG. 10. Representative realizations and stationary distributions no separation of time scales is required. for the model with described in Sec. IV C. Panels (a) and (b) represent The technique we provide makes our understanding of one dynamic mode and panels (c) and (d) represent the other. More processes involving both intrinsic and extrinsic noise more precisely, panels (a) and (b) show the outcome when the initial complete. While the existing PDMP description has been condition is to the left of φ(3)∗; panels (c) and (d) show a situation shown to be successful in many instances, it disregards in which the dynamics is started to the right of φ(4)∗. The histograms intrinsic noise entirely. We are now in a position to describe in panels (b) and (d) are from Gillespie simulations, and the solid the effects of demographic noise in systems with switching lines are from the analytical approximation. Model parameters are environments in the spirit of the van Kampen expansion. This λ+ = λ− = 0.2and = 1000, remaining parameters as described in allows us to investigate models subject to combinations of Appendix B. intrinsic and extrinsic noise and, in particular, systems in which

052119-11 HUFTON, LIN, GALLA, AND MCKANE PHYSICAL REVIEW E 93, 052119 (2016) some degrees of freedom can be treated within a linear-noise Equation (A3) and its analog for ρ1(φ) can directly be approximation, while other variables remain fundamentally integrated and we find discrete. Such systems can be of relevance, for example, in the N N context of genetic switches, bacterial populations subject to ∗(φ,0) = 0 h(φ),∗(φ,1) = 1 h(φ), (A4) varying external conditions, or to predator-prey dynamics. In v0(φ) v1(φ) order to make the method more applicable, several extensions where can be considered in future work. This includes the case of φ λ+(u) λ−(u) multiple environmental states, systems with more than one h(φ) = exp − du + , (A5) species, or, indeed, environmental dynamics beyond simple v0(u) v1(u) telegraph processes. Work along these lines is currently in and where N0 and N1 are normalization constants. Using ∗ ∗ progress. again the relation v0(φ) (φ,0) + v1(φ) (φ,1) = 0, derived above, we conclude N0 =−N1 ≡−N , and so we have ACKNOWLEDGMENTS N N ∗ φ, = h φ ,∗ φ, = h φ . ( 0) − ( ) ( 1) ( ) (A6) P.G.H., Y.T.L., and T.G. thank the Engineering and Physical v0(φ) v1(φ) Sciences Research Council (EPSRC) for funding (Grant Recalling that v0 < 0 and v1 > 0 throughout the domain of No. EP/K037145/1). the PDMP, N is a positive constant, to be determined from the normalization condition ∗ φ1 APPENDIX A: THE STATIONARY DISTRIBUTION [∗(φ,0) + ∗(φ,1)]dφ = 1. (A7) OF THE PDMP ∗ φ0 In this section we briefly discuss the calculation of the stationary distribution of the PDMP. Following from Eq. (26), APPENDIX B: FURTHER DETAILS OF THE MODEL ∗ ≡ WITH MULTIPLE FIXED POINTS and writing ∂φJσ (φ) 0 in the stationary state, we have J ∗(φ) ≡ 0 throughout, using zero-current conditions at the σ For the model described in Sec. IV C the functions vσ (x) boundaries. This leads to and wσ (x) as defined in Eq. (22)aregivenby ∗ = ∗ v (x) = c − c x + c x2 − c x4, J0 (φ) v0(φ) (φ,0) σ 1,σ 2,σ 3,σ 3,σ (B1) φ = + + 2 + 4 ∗ ∗ wσ (x) c1,σ c2,σ x c3,σ x c3,σ x . − [−λ+(u) (u,0) + λ−(u) (u,1)] du = 0, ∗ φ0 The parameters chosen for the analysis in Sec. IV C are ∗ = ∗ J1 (φ) v1(φ) (φ,1) c = 0.11,c = 0.31, 1,0 1,1 φ ∗ ∗ c2,0 = 0.76,c2,1 = 1.24, − [λ+(u) (u,0) − λ−(u) (u,1)] du = 0. φ∗ 0 c3,0 = 2.14,c3,1 = 2.60, (A1) c4,0 = 2.40,c4,1 = 2.40. (B2) Summing the two stationary currents, J ∗(φ) + J ∗(φ) = 0, we ∗ 0 ∗ 1 immediately find v0(φ) (φ,0) + v1(φ) (φ,1) = 0 for all APPENDIX C: ACCURACY OF OUR APPROACH ∗ =− ∗ φ, i.e., (φ,1) [v0(φ)/v1(φ)] (φ,0). The accuracy of our approach can be characterized by Inserting this into the first equation of (A1)gives computing the Kullback-Leibler divergence [58] between the ∗ true stationary distribution and our approximation. Figure 11 v0(φ) (φ,0) φ ∗ v0(φ) ∗ + λ+(u) (u,0) + λ−(u) (u,0) du = 0. ∗ v (φ) 10-2 φ0 1 (A2) 10-4 Differentiating with respect to φ one obtains -6 10 slow switching + λ+(φ) + λ−(φ) = intermediate switching ρ0(φ) ρ0(φ) 0, (A3) fast switching v0(φ) v1(φ) 10-8 8 16 32 64 128 256 512 1024 ∗ system size Ω where we have introduced ρ0(φ) ≡ v0(φ) (φ,0), and where

ρ indicates a derivative with respect to φ. An analogous FIG. 11. The Kullback-Leibler divergence between the true sta- 0 ∗ derivation shows that ρ1(φ) ≡ v1(φ) (φ,1) fulﬁlls the same tionary distribution of the system and our approximation for the relation [we note that the expression in the square brackets in linear model. Each line shows a different parameter regime from Eq. (A3) is symmetric with respect to simultaneous exchanges Fig. 1. The true stationary distribution is determined by fourth-order 0 ↔ 1 and λ+ ↔ λ−]. Runge-Kutta integration of the master equation.

052119-12 INTRINSIC NOISE IN SYSTEMS WITH SWITCHING . . . PHYSICAL REVIEW E 93, 052119 (2016) shows how the Kullback-Leibler divergence changes with average population of the system. The divergence decreases system size for the linear model. Each line represents one as the system size is increased, with relatively small system of the parameter regimes displayed in Fig. 1. Birth rates, death sizes having appreciably small divergences. Similar results are rates, and switching rates have been chosen such that is the observed for the nonlinear and feedback models.

[1] N. G. van Kampen, Stochastic Processes in Physics and [29] M. Assaf, M. Mobilia, and E. Roberts, Phys. Rev. Lett. 111, Chemistry (North-Holland, Amsterdam, 2007). 238101 (2013). [2] C. W. Gardiner, Handbook of Stochastic Methods (Springer- [30] E. Kussell, R. Kishony, N. Q. Balaban, and S. Leibler, Genetics Verlag, Berlin, 2004). 169, 1807 (2005). [3] H. Risken, The Fokker–Planck Equation: Methods of Solution [31] E. Kussell and S. Leibler, Science 309, 2075 (2005). and Applications (Springer-Verlag, Berlin, 1989). [32] M. Thattai and A. Van Oudenaarden, Proc. Natl. Acad. Sci. USA [4] M. J. Keeling and P. Rohani, Modeling Infectious Diseases in 98, 8614 (2001). Humans and Animals (Princeton University Press, Princeton, [33] P. S. Swain, M. B. Elowitz, and E. D. Siggia, Proc. Natl. Acad. NJ, 2008). Sci. USA 99, 12795 (2002). [5] C. J. Mode and C. K. Sleeman, Stochastic Processes in Epidemi- [34] F. Hayot and C. Jayaprakash, Phys. Biol. 1, 205 (2004). ology: HIV/AIDS, Other Infectious Diseases, and Computers [35] A. Bobrowski, T. Lipniacki, K. Pichor,´ and R. Rudnicki, J. Math. (World Scientiﬁc, Singapore, 2000). Anal. Appl. 333, 753 (2007). [6] A. J. Black and A. J. McKane, J. Theor. Biol. 267, 85 (2010). [36] M. S. Sherman and B. A. Cohen, PLoS Comput. Biol. 10, [7] D. J. Wilkinson, Stochastic Modelling for Systems Biology (CRC e1003596 (2014). Press, Boca Raton, FL, 2012). [37] P. Thomas, N. Popovic,´ and R. Grima, Proc. Natl. Acad. Sci. [8] D. T. Gillespie, Annu. Rev. Phys. Chem. 58, 35 (2007). USA 111, 6994 (2014). [9] T. B. Kepler and T. C. Elston, Biophys. J. 81, 3116 (2001). [38] A. Duncan, S. Liao, T. Vejchodsky,´ R. Erban, and R. Grima, [10] A. M. Walczak, A. Mugler, and C. H. Wiggins, in Computational Phys. Rev. E 91, 042111 (2015). Modeling of Signaling Networks, Vol. 880, Methods in Molec- [39] P. Ashcroft, P. M. Altrock, and T. Galla, J. R. Soc. Interface 11, ular Biology (Humana Press, New York, 2012), pp. 273–322. 20140663 (2014). [11] V. Shahrezaei and P. S. Swain, Proc. Natl. Acad. Sci. USA 105, [40] Q. Luo and X. Mao, J. Math. Anal. Appl. 334, 69 17256 (2008). (2007). [12] N. Kumar, T. Platini, and R. V. Kulkarni, Phys.Rev.Lett.113, [41] C. Zhu and G. Yin, J. Math. Anal. Appl. 357, 154 (2009). 268105 (2014). [42] M. H. A. Davis, J. R. Stat. Soc. B 46, 353 (1984). [13] N. G. van Kampen, Can. J. Phys. 39, 551 (1961). [43] A. Faggionato, D. Gabrielli, and M. R. Crivellari, J. Stat. Phys. [14] Y. T. Lin, H. Kim, and C. R. Doering, J. Stat. Phys. 148, 647 137, 259 (2009). (2012). [44] S. Zeiser, U. Franz, O. Wittich, and V.Liebscher, IET Syst. Biol. [15] Y. T. Lin, H. Kim, and C. R. Doering, J. Math. Biol. 70, 647 2, 113 (2008). (2015). [45] S. Zeiser, U. Franz, and V. Liebscher, J. Math. Biol. 60, 207 [16] Y. T. Lin, H. Kim, and C. R. Doering, J. Math. Biol. 70, 679 (2010). (2015). [46] J. Realpe-Gomez, T. Galla, and A. J. McKane, Phys.Rev.E86, [17] G. W. A. Constable and A. J. McKane, Phys. Rev. Lett. 114, 011137 (2012). 038101 (2015). [47] R. Grima, D. Schmidt, and T. Newman, J. Chem. Phys. 137, [18] O. Kogan, M. Khasin, B. Meerson, D. Schneider, and C. R. 035104 (2012). Myers, Phys. Rev. E 90, 042149 (2014). [48] J. Peccoud and B. Ycart, Theor. Popul Biol. 48, 222 [19] T. Biancalani, D. Fanelli, and F. Di Patti, Phys.Rev.E81, 046215 (1995). (2010). [49] J. E. M. Hornos, D. Schultz, G. C. P. Innocentini, J. Wang, A. M. [20] T. Biancalani, T. Galla, and A. J. McKane, Phys. Rev. E 84, Walczak, J. N. Onuchic, and P. G. Wolynes, Phys. Rev. E 72, 026201 (2011). 051907 (2005). [21] T. Butler and N. Goldenfeld, Phys.Rev.E84, 011112 (2011). [50] A. Raj, C. S. Peskin, D. Tranchina, D. Y. Vargas, and S. Tyagi, [22] S. Leibler and E. Kussell, Proc. Natl. Acad. Sci. USA 107, 13183 PLoS Biol. 4, e309 (2006). (2010). [51] D. T. Gillespie, J. Comput. Phys. 22, 403 (1976). [23] C. Escudero and J. A.´ Rodr´ıguez, Phys. Rev. E 77, 011130 [52] D. T. Gillespie, J. Phys. Chem. 81, 2340 (1977). (2008). [53] X. Mao and C. Yuan, Stochastic Differential Equations with [24] M. Assaf, A. Kamenev, and B. Meerson, Phys.Rev.E78, 041123 Markovian Switching (Imperial College Press, London, 2006). (2008). [54] D. A. Potoyan and P. G. Wolynes, J. Chem. Phys. 143, 195101 [25] S. Schuler, T. Speck, C. Tietz, J. Wrachtrup, and U. Seifert, (2015). Phys.Rev.Lett.94, 180602 (2005). [55] I. Bena, Int. J. Mod. Phys. B 20, 2825 (2006). [26] R. P. Boland, T. Galla, and A. J. McKane, Phys. Rev. E 79, [56] K. Kitahara, W. Horsthemke, and R. Lefever, Phys. Lett. A 70, 051131 (2009). 377 (1979). [27] A. Kamenev, B. Meerson, and B. Shklovskii, Phys. Rev. Lett. [57] R. P. Boland, T. Galla, and A. J. McKane, J. Stat. Mech: Theory 101, 268103 (2008). Exp. (2008) P09001. [28] M. Assaf, E. Roberts, Z. Luthey-Schulten, and N. Goldenfeld, [58] S. Kullback, Information Theory and Statistics (Dover, Mineola, Phys.Rev.Lett.111, 058102 (2013). NY, 1968).

052119-13