Appendix a Review of Probability Distribution
Total Page:16
File Type:pdf, Size:1020Kb
Appendix A Review of Probability Distribution In this appendix we review some fundamental concepts of probability theory. After deriving in Sect. A.1 the binomial probability distribution, in Sect. A.2 we show that for systems described by a large number of random variables this distribution tends to a Gaussian function. Finally, in Sect. A.3 we define moments and cumulants of a generic probability distribution, showing how the central limit theorem can be easily derived. A.1 Binomial Distribution Consider an ideal gas at equilibrium, composed of N particles, contained in a box of volume V . Isolating within V a subsystem of volume v, as the gas is macroscop- ically homogeneous, the probability to find a particle within v equals p = v/V , while the probability to find it in a volume V − v is q = 1 − p. Accordingly, the probability to find any one given configuration (i.e. assuming that particles could all be distinguished from each other), with n molecules in v and the remaining (N − n) within (V − v), would be equal to pnqN−n, i.e. the product of the respective prob- abilities.1 However, as the particles are identical to each other, we have to multiply this probability by the Tartaglia pre-factor, that us the number of possibilities of 1Here, we assume that finding a particle within v is not correlated with finding another, which is true in ideal gases, that are composed of point-like molecules. In real fluids, however, since molecules do occupy an effective volume, we should consider an excluded volume effect. R. Mauri, Non-Equilibrium Thermodynamics in Multiphase Flows, 181 Soft and Biological Matter, DOI 10.1007/978-94-007-5461-4, © Springer Science+Business Media Dordrecht 2013 182 A Review of Probability Distribution choosing n particles within a total of N identical particles, obtaining:2 n N−n N − N! v V − v Π (n) = pnqN n = . (A.1) N n n!(N − n)! V V This is the binomial distribution, expressing the probability to find n particles in the volume v and the remaining (N − n) within (V − v). Normalization of the Binomial Distribution Recall the binomial theorem: N N! − (p + q)N = pnqN n. (A.2) n!(N − n)! n=0 This shows that, as p + q = 1, the binomial distribution is normalized, N ΠN (n) = 1. (A.3) n=0 Calculation of the Mean Value From N N N! − n = nΠ (n) = pnqN nn, (A.4) N n!(N − n)! n=0 n=0 considering that ∂ np n = p pn , (A.5) ∂p we obtain: N ∂ N! − n = p pnqN n . (A.6) ∂p n!(N − n)! n=0 Then considering (A.2), we obtain: ∂ − n = p (p + q)N = pN(p + q)N 1, (A.7) ∂p 2The number of ways with which N distinguishable objects can fill N slots is equal to N!.In fact, if we imagine to fill the slots one by one, the first particle will have N free slots to choose from, the second particle will have (N − 1),thethird(N − 2), etc., until the last particle will find only one free slot. Now, suppose that of the N objects, n are, say, white and indistinguishable (this is equivalent, in our case, to having them located within v), while the remaining (N − n) are black, i.e. they are located within (V − v). That means that we must divide N! by the number of permutations among the n white particles and by the number of permutations among the (N − n) black particles, to account for the fact that if any white (or black) particles exchange their places we find a configuration that is indistinguishable from the previous one. So, at the end, we find the factor N|/n!(N − n)!, which correspond to the Tartaglia prefactors, appearing in the Tartaglia triangle. A.1 Binomial Distribution 183 and finally, as p + q = 1, we conclude: n = Np. (A.8) Calculation of the Variance Considering that, (n)2 = (n − n)2 = n2 − n2, (A.9) ¯ we see that we need to compute n2, defined as N N N! − n2 = n2Π (n) = pnqN nn2. (A.10) N n!(N − n)! n=0 n=0 Considering that ∂ ∂ 2 n2pn = n p pn = p pn, (A.11) ∂p ∂p we obtain: 2 N 2 ∂ N! − ∂ n2 = p pnqN n = p (p + q)N , (A.12) ∂p n!(N − n)! ∂p n=0 where we have considered (A.2) and (A.7). Then, we obtain: ∂ − − − n2 = p pN(p + q)N 1 = pN (p + q)N 1 + p(N − 1)(p + q)N 2 , (A.13) ∂p and finally, as p + q = 1, we conclude: n2 = Np(1 + Np − p) = (Np)2 + Npq, (A.14) so that: (n)2 = Npq. (A.15) As expected, defining the mean root square dispersion, we find as expected: δn 1 q = √ , where δn ≡ (n)2. (A.16) n N p √ In particular, when p = q = 1/2, we have: δn/n = 1/ n. 184 A Review of Probability Distribution A.2 Poisson and Gaussian Distribution Let us consider the case when v V ,i.e.n N. Then N! =∼ (N − n)!N n and the binomial distribution becomes: nn n N Π (n) = 1 − , (A.17) N n! N where n = Np. Now, letting N →∞and considering that x a − lim 1 − = e a, (A.18) x→∞ x we obtain the so-called Poisson distribution,3 nne−n Π(n)= . (A.19) n! It is easy to see that, predictably, the Poisson distribution gives the same results as the binomial distribution in terms of normalization condition, mean value and dispersion, i.e. ∞ ∞ ∞ Π(n)= 1; nΠ(n)= n; (n − n)2Π(n)= n. (A.20) n=0 n=0 n=0 When fluctuations are small and√n 1, the Poisson distribution can be simplified applying Stirling’s formula, n!= 2πnnne−n, valid when n 1, obtaining: 1 n n Π(n)= √ en, (A.21) 2πn n where n = n − n. Now, considering that n n n n 1 n 2 = exp −n ln 1 + = exp −n + +··· , (A.22) n n n 2 n we see that at leading order it reduces to the following Gaussian distribution: 1 (n − n)2 Π(n)= √ exp − . (A.23) 2πn 2n Often, instead of a discrete probability distribution Π(n), it is convenient to de- fine a continuous distribution, Π(x), defined so that Π(x)dx is the probability that 3Consider that in this case we keep Np = n fixed as we let N →∞,sothatq = 1. A.3 Moments and Cumulants 185 x assumes values between x and x + dx. In our case, this can be done considering the limit of x = n/N as N →∞, dn Π(x)dx = lim Π(n) , (A.24) N→∞ N where dn = 1 and dx = limN→∞(1/N). Predictably, Π(x) is again a Gaussian distribution, 1 (x − x)2 Π(x)= √ exp − , (A.25) 2πσ 2σ 2 where σ = δx. A.3 Moments and Cumulants The rth central moment, or moment about the mean, of a random variable X with probability density Π(x) is defined as the mean value of (x − μ)r , i.e., μ(r) = (x − μ)r = (x − μ)r Π(x)dx, r = 0, 1,..., (A.26) where μ = xΠ(x)dx is the mean value, provided that the integrals converge.4 Note that μ(0) = 1 because of the normalization condition, while μ(1) = 0 by def- inition. An easy way to combine all of the moments into a single expression is to consider the following Fourier transform of the probability density, − − Π(k) = eik(x μ)Π(x)dx = eik(x μ) , (A.27) which is generally referred to as the characteristic (or moment generating) function. Now, since Π coincides with the mean value of eik(x−μ), expanding it in a Taylor series about the origin, we easily find: ∞ 1 1 Π(k) = 1 + (x − μ) (ik) + (x − μ)2 (ik)2 +···= μ(r)(ik)r . (A.28) 2 r! r=0 Therefore, we see that the r-th moment is the r-th derivative of the characteristic function with respect to (ik) at k = 0. Sometimes it is more convenient to consider the logarithm of the probability density. In that case, proceeding as before, we can define the cumulant generating function, ∞ 1 Φ(k) = ln Π(k) = κ(r)(ik)r , (A.29) r! r=1 4In general, the rth moment of a random variable is defined as the mean values of xr . 186 A Review of Probability Distribution where κ(r) are called cumulants, with obviously κ(0) = κ(1) = 0. The first three cumulants are equal to the respective central moments, that is κ(1) = μ(1) is the mean value,5 κ(2) = μ(2) is the variance, and κ(3) = μ(3) is the skewness, measuring the lopsidedness of the distribution. Instead, higher order moments and cumulants are different from each other. For example, κ(4) = μ(4) − 3(μ2)2, where the forth central moment is a measure of whether the distribution is tall and skinny or short and squat, compared to the normal distribution of the same variance.6 When the random variable x is the sum of many mutually independent random variables xi , we know that the global probability is the product of the probabilities of the individual variables, and therefore the cumulant generating function (A.29)is = the sum of the individual functions, i.e. Φ(k) i Φi(k). Therefore, from (A.29) we see that, (r) = (r) κ κi .