POISSON PROCESSES 1. THE LAW OF SMALL NUMBERS 1.1. The Rutherford-Chadwick-Ellis Experiment. About 90 years ago Ernest Rutherford and his collaborators at the Cavendish Laboratory in Cambridge conducted a series of pathbreaking experiments on radioactive decay. In one of these, a radioactive substance was observed in N = 2608 time intervals of 7.5 seconds each, and the number of decay particles reaching a counter during each period was recorded. The table below shows the number Nk of these time periods in which exactly k decays were observed for k = 0,1,2,...,9. Also shown is N pk where k pk = (3.87) exp 3.87 =k! {− g The parameter value 3.87 was chosen because it is the mean number of decays/period for Rutherford’s data. k Nk N pk k Nk N pk 0 57 54.4 6 273 253.8 1 203 210.5 7 139 140.3 2 383 407.4 8 45 67.9 3 525 525.5 9 27 29.2 4 532 508.4 10 16 17.1 5 408 393.5 ≥ This is typical of what happens in many situations where counts of occurences of some sort are recorded: the Poisson distribution often provides an accurate – sometimes remarkably ac- curate – fit. Why? 1.2. Poisson Approximation to the Binomial Distribution. The ubiquity of the Poisson distri- bution in nature stems in large part from its connection to the Binomial and Hypergeometric distributions. The Binomial-(N ,p) distribution is the distribution of the number of successes in N independent Bernoulli trials, each with success probability p. If p is small, then successes are rare events; but if N is correspondingly large, so that λ = N p is of moderate size, then there are just enough trials so that a few successes are likely. Theorem 1. (Law of Small Numbers) If N and p 0 in such a way that N p λ, then the Binomial-(N ,p) distribution converges! to 1 the Poisson-!λ distribution, that is, for! each k = 0,1,2,..., k e λ N k N k λ − (1) p (1 p) − k − −! k! What does this have to do with the Rutherford-Chadwick-Ellis experiment? The radioactive substances that Rutherford was studying are composed of large numbers of individuals atoms 24 – typically on the order of N = 10 . The chance that an individual atom will decay in a short 1 2 POISSON PROCESSES time interval, and that the resulting decay particle will emerge in just the right direction so as to colide with the counter, is very small. Finally, the different atoms are at least approximately independent.1 Theorem 1 can be proved easily by showing that the probability generating functions of the binomial distributions converge to the generating function of the Poisson distribution. Alter- natively, it is possible (and not very difficult) to show directly that the densities converge. For either proof, the following important analytic lemma is needed. Lemma 2. If λn is a sequence of real numbers such that limn λn = λ exists and is finite, then !1 n λn λ (2) lim 1 = e . n n − !1 − Proof. Write the product on the left side as the exponential of a sum: n (1 λn =n) = exp n log(1 λn =n) . − f − g This makes it apparent that what we must really show is that n log(1 λn =n) converges to λ. Now λn =n converges to zero, so the log is being computed at an argument− very near 1, where− log1 = 0. But near x = 1, the log function is very well approximated by its tangent line, which has slope 1 (because the derivative of log x is 1=x ). Hence, n log(1 λn =n) n( λn =n) λ. − ≈ − ≈ − To turn this into a rigorous proof, use the second term in the Taylor series for log to estimate the error in the approximation. Exercise 1. Prove Theorem 1 either by showing that the generating functions converge or by showing that the densities converge. The Law of Small Numbers is closely related to the next proposition, which shows that the exponential distribution is a limit of geometric distributions. Proposition 3. Let Tn be a sequence of geometric random variables with parameters pn , that is, k (3) P Tn > k = (1 pn ) for k = 0,1,2,... f g − If npn λ > 0 as n then Tn =n converges in distribution to the exponential distribution with parameter! λ. ! 1 Proof. Set λn = npn ; then λn λ as n . Substitute λn =n for pn in (3) and use Lemma 2 to get ! ! 1 [nt ] lim P Tn > nt 1 λn =n exp λt . n = ( ) !1 f g − −! {− g 1This isn’t completely true, though, and so observable deviations from the Poisson distribution will occur in larger experiments. POISSON PROCESSES 3 2. THINNING AND SUPERPOSITION PROPERTIES Superposition Theorem . If Y1, Y2,..., Yn are independent Poisson random variables with means EYi = λi then n n X X (4) Yi Poisson λi i =1 ∼ − i =1 There are various ways to prove this, none of them especially hard. For instance, you can use probability generating functions (Exercise.) Alternatively, you can do a direct calculation of the probability density when n = 2, and then induct on n. (Exercise.) But the clearest way to see that theorem theorem must be true is to use the Law of Small Numbers. Consider, for definiteness, the case n = 2. Consider independent Bernoulli trials Xi , with small successs parameter p. Let N1 = [λ1=p] and N2 = [λ2=p] (here [x ] means the integer part of x ), and set N = N1 + N2. Clearly, N X1 Xi Binomial (N1,p), i =1 ∼ − N N X2+ 1 Xi Binomial (N2,p),and i =N1+1 ∼ − N X Xi Binomial (N ,p). i =1 ∼ − The Law of Small Numbers implies that when p is small and N1,N2, and N are correspondingly large, the three sums above have distributions which are close to Poisson, with means λ1, λ2, and λ, respectively. Consequently, (4) has to be true when n = 2. It then follows by induction that it must be true for all n. Thinning Theorem . Suppose that N Poisson(λ), and that X1, X2,... are independent, iden- tically distributed Bernoulli-p random∼ variables independent of N . Let S Pn X . Then S n = i =1 i N has the Poisson distribution with mean λp. This is called the “Thinning Property” because, in effect, it says that if for each occurence counted in N you toss a p coin, and then record only those occurences for which the coin toss is a Head, then you still end− up with a Poisson random variable. Proof. You can prove this directly, by evaluating P SN = k (exercise), or by using generating functions, or by using the Law of Small Numbers (exercise).f g The best proof is by the Law of Small Numbers, in my view, and I think it worth one’s while to try to understand the result from this perspective. But for variety, here is a generating function proof: First, recall that the S n generating function of the binomial distribution is E t n = (q + p t ) , where q = 1 p. The generating function of the Poisson distribution with parameter λ is − k N X1 (t λ) E t e λ exp λt λ . = k! − = k=0 f − g 4 POISSON PROCESSES Therefore, SN X1 Sn n λ E t = E t λ e − =n! n=0 X1 n n λ = (q + p t ) λ e − =n! n=0 = exp (λq + λp t ) λ f − g = exp λp t λp , f − g and so it follows that SN has the Poisson distribution with parameter λp. A similar argument can be used to prove the following generalizaton: Generalized Thinning Theorem . Suppose that N Poisson(λ), and that X1, X2,... are inde- pendent, identically distributed multinomial random∼ variables with distribution Multinomial (p1,p2,...,pm ), that is, − P Xi = k = pk for each k = 1,2,...,m f g Then the random variables N1,N2,...,Nm defined by N X Nk = 1 Xi = k i =1 f g are independent Poisson random variables with parameters E Nk = λpk . Notation: The notation 1A (or 1 ... ) denotes the indicator function of the event A (or the event ... ), that is, the random variablef thatg takes the value 1 when A occurs and 0 otherwise. Thus, inf theg statement of the theorem, Nk is the number of multinomial trials Xi among the first N that result in outcome k. 3. POISSON PROCESSES Definition 1. A point process on the timeline [0, ) is a mapping J NJ = N (J ) that assigns to 2 1 7! each subset J [0, ) a nonnegative, integer-valued random variable NJ in such a way that if J1, J2,.. are pairwise⊂ 1 disjoint then X (5) N ( i Ji ) = N (Ji ) [ i The counting process associated with the point process N (J ) is the family of random variables Nt = N (t ) t 0 defined by f g ≥ (6) N (t ) = N ((0, t ]). NOTE: The sample paths of N (t ) are, by convention, always right-continuous, that is, N (t ) = lim" 0+ N (t + "). ! There is a slight risk of confusion in using the same letter N for both the point process and the counting process, but it isn’t worth using a different letter for the two. The counting process N (t ) has sample paths that are step functions, with jumps of integer sizes. The discontinuities represent occurences in the point process. 2actually, to each Borel measurable subset POISSON PROCESSES 5 Definition 2.