Lecture 2 { Master equation and system size expansion February 7, 2020 Lecture 2 { Master equation and system size expansion February 7, 2020 1 / 47 Outline 1 The master equation for time continuous processes. 2 Birth and death processes - Intro. 3 Simulating stochastic orbits: the Gillespie algorithm. 4 Stochastic predictions: the van Kampen system size expansion. 5 Theory vs. simulations. Lecture 2 { Master equation and system size expansion February 7, 2020 2 / 47 The Master Equation This is basically the continuous time version of Markov chains. Derivation from the Chapman-Kolmogorov (CK) equation The CK equation is X 0 0 p(n; t + ∆tjn0; t0) = p(n; t + ∆tjn ; t)p(n ; tjn0; t0): n0 Now, we assume that 2 0 0 1 − κn(t)∆t + O(∆t) ; if n = n ; p(n; t + ∆tjn ; t) = 2 0 (1) wnn0 (t)∆t + O(∆t) ; if n 6= n : 0 wnn0 (t) is the transition rate and is only defined for n 6= n Now, by normalisation, X 1 = p(n; t + ∆tjn0; t) n 2 X 2 = 1 − κn0 (t)∆t + O(∆t) + wnn0 (t)∆t + O(∆t) n6=n0 Lecture 2 { Master equation and system size expansion February 7, 2020 3 / 47 X ) κn0 (t) = wnn0 (t): n6=n0 Alternatively, switching the indices, X κn(t) = wn0n(t): (2) n06=n Therefore, using (1), we see that the CK equation reads p(n; t + ∆tjn0; t0) = (1 − κn(t)∆t + :::)p(n; tjn0; t0) X 0 2 + wnn0 (t)p(n ; tjn0; t0)∆t + O(∆t) ; n06=n p(n; t + ∆tjn ; t ) − p(n; tjn ; t ) ) 0 0 0 0 ∆t X 0 = −κn(t)p(n; tjn0; t0) + wnn0 (t)p(n ; tjn0; t0) + O(∆t): n06=n Lecture 2 { Master equation and system size expansion February 7, 2020 4 / 47 Now let ∆t ! 0: dp(n; tjn0; t0) X 0 = −κ (t)p(n; tjn ; t ) + w 0 (t)p(n ; tjn ; t ): dt n 0 0 nn 0 0 n06=n Using (2), the middle term may be rewritten to find dp(n; tjn0; t0) X X 0 = − w 0 (t)p(n; tjn ; t ) + w 0 (t)p(n ; tjn ; t ): dt n n 0 0 nn 0 0 n06=n n06=n If we had started, not from the CK equation, but from the first relation which defines Markov processes: X p(n; t + ∆t) = p(n; t + ∆tjn0; t)p(n0; t) n0 then exactly the same sequence of steps would lead to dp(n; t) X X 0 = − w 0 (t)p(n; t) + w 0 (t)p(n ; t): dt n n nn n06=n n06=n Lecture 2 { Master equation and system size expansion February 7, 2020 5 / 47 This equation could also have been obtained by multiplying the equation for p(n; tjn0; t0) by p(n0; t0) and summing over all initial states, since X p(n; t) = p(n; tjn0; t0)p(n0; t0): n0 Therefore both p(n; tjn0; t0) and p(n; t) satisfy the same equation. We will frequently write it for p(n; t), with the understanding that this could also be thought of as the conditional probability. The equation is the desired master equation, dp(n; t) X 0 X = w 0 (t)p(n ; t) − w 0 (t)p(n; t): (3) dt nn n n n06=n n06=n The interpretation of the master equation is straightforward. The first term is just the probability of going from n0 ! n, and the second the probability of going from n ! n0. Lecture 2 { Master equation and system size expansion February 7, 2020 6 / 47 In words, the rate of change of being in a state n is equal to the probability of making a transition into n, minus the probability of transitioning out of n. In applications the wnn0 are assumed to be given (this specifies the model) and we wish to determine the p(n; t). Comments: • Notice that because of the Markov property, wnn0 only depends on the current state of the system (n0), and does not depend on previous states (i.e. how the system got to n0). • In the derivation we have assumed that wnn0 depends on time (which it does in general, just as for Markov chains), but usually we are only interested in situations where it is time-independent. Lecture 2 { Master equation and system size expansion February 7, 2020 7 / 47 More informal derivations of the master equation The master equation is essentially a balance equation; the rate of moving into the state n minus the rate of moving out of the same state is the rate of change of p(n; t). That is, Rate of change of p(n; t) = [ Rate due to transitions into the state n from all the other states n0]- [ Rate due to transitions out of the state n into all other states n0] When expressed this way, and assuming the process is Markov, the master equation dp(n; t) X 0 X = w 0 (t)p(n ; t) − w 0 (t)p(n; t); dt nn n n n06=n n06=n appears very reasonable. Lecture 2 { Master equation and system size expansion February 7, 2020 8 / 47 Another derivation, which is also less rigorous than the original, starts from the system described as a Markov chain, and moves to continuous time by taking the duration of the time-step to zero: So, let us start from X Pn(t + 1) = Qnn0 (t)Pn0 (t); n0 where the columns of the transition matrix add to unity: X Qn0n(t) = 1: n0 So, if we write X X Pn(t + 1) − Pn(t) = Qnn0 (t)Pn0 (t) − Qn0n(t)Pn(t); n0 n0 then the terms in the sums with n0 = n can be cancelled to give X X Pn(t + 1) − Pn(t) = Qnn0 (t)Pn0 (t) − Qn0n(t)Pn(t); n06=n n06=n Lecture 2 { Master equation and system size expansion February 7, 2020 9 / 47 Now, we take the time step to be ∆t, rather than unity, and divide through by the time step; Pn(t + ∆t) − Pn(t) X Qnn0 X Qn0n = P 0 (t) − P (t): ∆t ∆t n ∆t n n06=n n06=n Then, it is clear that taking the time step to zero, reduces the left-hand side to a differential of the probability, with respect to time. Furthermore instead of assuming that exactly one sampling event happens per time step, we assume that on average one event happens in the time step. To achieve this set 2 0 Qnn0 (t) = wnn0 (t)∆t + O(∆t) ; n 6= n : So letting ∆t ! 0 we obtain the master equation: dPn X X = w 0 (t)P 0 (t) − w 0 (t)P (t): dt nn n n n n n06=n n06=n Lecture 2 { Master equation and system size expansion February 7, 2020 10 / 47 The birth and death stochastic model Bacteria dynamics. Demography. Queueing theory. 1 Simple model of population growth. 2 Individuals enter/exit the community. Lecture 2 { Master equation and system size expansion February 7, 2020 11 / 47 Exact numerical scheme: the Gillespie algorithm Consider a given chemical equation, R1. Assume Si , with i = 1; 2, to label the involved reactants (called Substrates in the original paper by Gillespie): c S1 + S2 −! 2S1 In words: the (individual!) molecule of type S1 can combine with an (individual!) molecule of type S2 to result into two molecules of type S1. The probability that such a reaction will take place in the forthcoming time interval dt is controlled by: the number of molecules of type S1 and S2 and the number of possible combinations that yield to an encounter between a molecule of type S1 and another of type S2. the average probability that given a pair of molecules S1 ed S2, the reaction R1 takes over. Lecture 2 { Master equation and system size expansion February 7, 2020 12 / 47 Assume n1 e n2 to label the number of molecules of type 1 e 2 respectively. Then, h = n1n2 is the number of independent combinations that result in a pair S1 - S2. On the other hand c measures the probability per unit of time of reacting. Hence: P1 = chdt = cn1n2dt is the probability that the reaction R1 takes over in a given time interval dt. So far so good! What is going to happen if we have instead a system of reactions? How are we going to sort out which reaction is going to happen first? Lecture 2 { Master equation and system size expansion February 7, 2020 13 / 47 Let us consider first the case where two reactions are at play, namely R1 and R2, specified as follows: c1 S1 + S2 −! 2S1 c2 2S1 −! S3 Answering to two questions is mandatory at this point: When is the next reaction going to occur? Which reaction is going to happen? Focus on the general framework. Imagine to have k type of molecules partitioned in the following families (n1, n2, n3...) and assume that those molecules can react according to M distinct reaction channels, labelled with Ri con i = 1; ::; M. Lecture 2 { Master equation and system size expansion February 7, 2020 14 / 47 We need to calculate the quantity: P(τ; i)dτ i.e. the probability that given the system in the state (n1, n2, n3...) at time t: the next reaction occurs in the time interval from t + τ to t + τ + dτ. it is the reaction Ri . The core of the algorithm is to evaluate the, presently unknown, quantity P(τ; i)dτ.