Geometric, Negative Binomial Distributions
Total Page:16
File Type:pdf, Size:1020Kb
Geometric Distribution We will discuss two distributions in this section, both which have two parameterizations, so we must be careful. Suppose you roll a single die until you roll a 6. Let us consider \rolling a 6" a \success" and \not rolling a 6" a \failure". Our goal is to determine a distribution for the total number of rolls necessary to roll a 6. However, we can also write a distribution for the number of failures that occur, rather than the number of total rolls. Counting the number of failures results in a simpler probability formula, so this is what is typically used. The situation above, regardless of which term you count (number of failures or number of rolls) is a geo- metric distribution. We will derive the probability formula for this distribution, because it will help us get a feel for deriving the formula for the next distribution as well. Let X = the number of failures before the first success - in this case, the number of non-six rolls prior to 1 1 5 rolling a 6. The probability of rolling a 6 is 6 and the probability of a non-6 is 1 − 6 = 6 . Then, 1 50 1 P (X = 0) = = · Since there are no failures, we immediately succeed. 6 6 6 5 1 51 1 P (X = 1) = · = · Here we fail once, and then succeed. 6 6 6 6 5 5 1 52 1 P (X = 2) = · · = · Here we fail twice, and then succeed. 6 6 6 6 6 5 5 5 1 53 1 P (X = 3) = · · · = · Here we fail three times in a row, and then succeed. 6 6 6 6 6 6 In this case, we see the distribution described as follows: Geometric Distribution: X = number of failures: 8 > k <> q · p k = 0; 1; 2; :::; where q = 1 − p • P (X = k) = > :> 0 otherwise 1 − p 1 − p p • E[X] = • Var(X) = • MX (t) = * p p2 1 − qet Additionally, P (X ≥ n) = (1 − p)n = qn 1 If we instead choose to parameterize the distribution by counting the number of total trials, or total rolls in our scenario, each line of the earlier derivation is the same. Instead of starting with P (X = 0) and increasing X by 1 each time, we will start with P (N = 1) and increase N by one each time. The ending criteria for out experiment is the same in both cases - roll a 6. However in doing so we can fail zero times, but we must roll at least once. Thus, the alternative parameterization is as follows. Let N = the number of trials in order to achieve success - in this case, the number of rolls needed to roll a 6. Then, 1 50 1 P (N = 1) = = · Here we have a single trial where we immediately succeed. 6 6 6 5 1 51 1 P (N = 2) = · = · Here we have two trials; fail once and then succeed. 6 6 6 6 5 5 1 52 1 P (N = 3) = · · = · Three trials; fail twice, and then succeed. 6 6 6 6 6 5 5 5 1 53 1 P (N = 4) = · · · = · Four trials; fail three times in a row, and then succeed. 6 6 6 6 6 6 Geometric Distribution: N = number of trials: 8 > n−1 <> q · p n = 1; 2; 3; :::; where q = 1 − p • P (N = n) = > :> 0 otherwise 1 1 − p pet • E[N] = • Var(X) = • M (t) = * p p2 N 1 − qet where t < − ln q Additionally, P (N ≥ n) = (1 − p)n−1 = qn−1 Note on Mean: The probability formula is slightly more intricate here, since we must remember to include a minus 1 in the exponent. However, I think the expected value here makes a little more sense. What is 1 1 the expected number of rolls need to roll a 6? 6! Which is p , where p = 6 . The easiest way (for me) to count the number of failures is to compare it to the total number of trails and subtract one. However, we can show this is equivalent to the earlier formula. 1 1 p 1 − p E[X = number of failures] = E[N] − 1 = − 1 = − = p p p p Note on Variance: The variance is the same in each formula. Since the distributions are the same other than their starting positions, the spread of each distribtion is equal. 2 Negative Binomial Distribution I do not have the Negative Binomial probability formula memorized. In my opinion, it is easier to reason out the formula again each time I see a question about it. We do need to familiarize ourselves with that process though, which is the reason I went into detail to derive the probability formula for the Geometric Distribu- tion. The Geometric Distribution is a special, simple case of the Negative Binomial Distribution. Similar to some previous distributions, the probability formula is confusing, but it will hopefully make more sense if we examine a concrete example. Again, similar to other complex distributions, I have never seen a question ex- pect you to know the mean or variance of a hypergeometric, but I will provide all of that information anyway. Suppose we are again rolling a die, and we wish to count the number of trials necessary to roll four 6s. There are again two ways to parameterize this. We can count the total number of trials - or rolls, N, or we can count the total number of failures - or non-6 rolls, X. Either way we choose to parameterize this, the stopping criteria for the experiment is the same - we roll the fourth 6. The derivation here is a bit more complicated, because we no longer stop on the first success. In a geometric distribution, we put all the failed attempts first, followed by the lone success. This time our failures and successes are intermingled, with the exception of the very last roll, which must be a success. Again, let X = the number of failures before the fourth success. We will generalize this to r successes soon. The only outcome we know for certain is the last roll must be a success, thus we separate that term out. Then depending on the number of rolls, we \shuffle" all other outcomes by choosing exactly three of them to be successes. Thus, 13 1 3 50 14 P (X = 0) = · = Here we simply succeed four times in a row. 6 6 3 6 6 Our final roll is a 6, but the preceeding four rolls 4 5 13 1 4 51 14 P (X = 1) = · = can happen in any order, so we must shuffle them 3 6 6 6 3 6 6 by choosing three of them to be successes. 5 52 13 1 5 52 14 Our final roll is still a 6, but now there are five P (X = 2) = · = preceeding rolls. These are shuffled by choosing 3 6 6 6 3 6 6 three of them to be successes. 6 53 13 1 6 53 14 Notice the total number of successes never changes, P (X = 2) = · = 3 6 6 6 3 6 6 and we always choose exactly three of the early rolls to be successes. The top term in our binomial coefficient is always the total number of rolls minus one. This includes all successes and failures, except the very last roll. The last roll's outcome is guaranteed, so it does not need to be shuffled in with the other terms. We have shown a distribution for the number of failures before the fourth success, but in general, we could look at the number of failures before the rth success. 3 Negative Binomial Distribution: X = number of failures before the rth success: 8 > k + r − 1 k r <> · q · p k = 0; 1; 2; :::; • P (X = k) = r − 1 > :> 0 otherwise 1 − p 1 − p • E[X] = r · • Var(X) = r · p p2 The expected value and variance are very similar to that of a geometric distribution, but multiplied by r. The distribution can be reparamaterized in terms of the total number of trials as well: Negative Binomial Distribution: N = number of trials to achieve the rth success: 8 > n − 1 n−r r <> · q · p n = r; r + 1; r + 2; :::; • P (N = n) = r − 1 > :> 0 otherwise 1 1 − p • E[X] = r · • Var(X) = r · p p2 4 Note on Expected Value of Geometric Distribution: When calculating the E[X] early, we used logic to say it made sense to take 6 attempts to roll a 6. However, the actual formula for expected value looks like this: 1 1 X X E[X] = k · qk · p = p · k · qk k=0 k=0 This formula occasionally appears in situations where it is not so intuitive what the expected value is, so it is useful to recognize. Since p is a constant, I will pull it out of the formula for the proof. This is a useful proof for your FM exam as well. It shows us the sum of an increasing geometric series. a First, recall the sum of a geometric series with initial term, a, and ratio, r, is S = . 1 − r I will drop the k = 0 term, since it is zero and I need the space in the next table. 1 1 X X k · qk = k · qk = k=0 k=1 1 · q + 2 · q2 + 3 · q3 + 4 · q4 + 5 · q5 + ··· = q = q + q2 + q3 + q4 + q5 + ··· = 1 − q q2 + q2 + q3 + q4 + q5 + ··· = 1 − q q3 + q3 + q4 + q5 + ··· = 1 − q q4 + q4 + q5 + ··· = 1 − q q5 + q5 + ··· = 1 − q Each of these lines is a geometric series with common ratio, q.