Math 141 Lecture 3: the Binomial Distribution

Coin Tossing The General Case

Math 141 Lecture 3: The Binomial Distribution

Albyn Jones1

1Library 304 [email protected] www.people.reed.edu/∼jones/courses/141

Albyn Jones Math 141 Coin Tossing The General Case Outline

Coin Tosses The General Case

Albyn Jones Math 141 Coin Tossing The General Case Independent Coin Tosses Crucial Features

Dichotomous Trials: Each toss results in either Heads, H, or Tails T . Independence: Successive tosses are independent; knowing we got H on the ﬁrst toss does not help us predict the outcome of the second toss. Constant probability: Each trial has the same probability P(H) = 1/2 = P(T ) (a ‘fair coin’).

Albyn Jones Math 141 Coin Tossing The General Case Examples of Different Experiments

Binomial: Count the number of Heads in a ﬁxed number of tosses. Geometric: Count the number of Tails before the ﬁrst Head. Negative Binomial: Count the number of Tails before before the k-th Head.

Albyn Jones Math 141 Coin Tossing The General Case Random Variables

Random Variable: A random variable is a function mapping points in the sample space to R, the real numbers.

Example: Let X be the number of Heads in 3 independent tosses of a fair coin.

Albyn Jones Math 141 Coin Tossing The General Case A Binomial Random Variable

Toss a fair coin 3 times, count the number of Heads. Let X be the number of Heads. What are the possible outcomes?

{HHH} 7→ X = 3

{HHT } ∪ {HTH} ∪ {THH} 7→ X = 2 {HTT } ∪ {THT } ∪ {TTH} 7→ X = 1 {TTT } 7→ X = 0

Albyn Jones Math 141 Coin Tossing The General Case Probabilities

Thus we can compute probabilities for the random variable (RV) X is we can compute probabilities of events in the original sample space.

P(X = 3) = P({HHH})

P(X = 2) = P({HHT } ∪ {HTH} ∪ {THH}) P(X = 1) = P({HTT } ∪ {THT } ∪ {TTH}) P(X = 0) = P({TTT })

Albyn Jones Math 141 Coin Tossing The General Case Probabilities

Successive tosses are independent, so

13 (X = 3) = ({HHH}) = (H) (H) (H) = P P P P P 2

and

13 (X = 0) = ({TTT }) = (T ) (T ) (T ) = P P P P P 2

In fact, the probability of any sequence of 3 tosses is the same, for example

13 ({HHT }) = (H) (H) (T ) = P P P P 2

Albyn Jones Math 141 Coin Tossing The General Case More Probabilities

Since probabilities of unions of disjoint events add, we just have to count the number of sequences of three tosses with 2 Heads to get P(X = 2): P(X = 2) = P({HHT } ∪ {HTH} ∪ {THH})

= P({HHT }) + P({HTH}) + P({THH}) Again, due to independence, the order of getting the two Heads and one Tail doesn’t matter, so 13 ({HHT }) = ({HTH}) = ({THH}) = P P P 2 Thus 13 3 (X = 2) = 3 = P 2 8

Albyn Jones Math 141 Coin Tossing The General Case Finally: Probabilities for 3 tosses

13 1 (X = 3) = 1 = P 2 8

13 3 (X = 2) = 3 = P 2 8 13 3 (X = 1) = 3 = P 2 8 13 1 (X = 0) = 1 = P 2 8 Note:

P(X = 0) + P(X = 1) + P(X = 2) + P(X = 3) = 1

Albyn Jones Math 141 Coin Tossing The General Case X ∼ Binomial(n, 1/2) Count Heads in n independent tosses of a fair coin

As with 3 independent tosses, any sequence of n independent tosses of a fair coin has the same probability. Example: for 8 tosses,

18 (HHHHTTTT ) = (HTHTHTHT ) = (TTTTTTHH) = P P P 2

The point: to compute P(X = k) we have to count the number of sequences of n symbols {H, T } with k H’s, and thus (n − k) T ’s, then multiply by the probability of any sequence of n tosses.

Albyn Jones Math 141 Coin Tossing The General Case Counting Things: Permutations

How many ways are there to rearrange the numbers 1, 2, 3, 4? We could list them all, with some effort:

{1, 2, 3, 4}, {2, 1, 3, 4}, {3, 1, 2, 4}, {4, 1, 2, 3},...

We could be clever: There are 4 choices for the first position. After choosing the first, there remain 3 choices for the second position. After choosing the first and second positions, there are 2 possibilities for the third, and only 1 for the fourth.

4 · 3 · 2 · 1 = 4! = 24

Albyn Jones Math 141 Coin Tossing The General Case Permutations of n objects

How many ways are there to rearrange the numbers 1, 2, 3,..., n? There are n choices for the first position. After choosing the first, there remain (n − 1) choices for the second position. After choosing the first and second positions, there are (n − 2) possibilities for the third, and so forth.

n · (n − 1) · (n − 2) ... · 1 = n! n! gets big quickly: 10! = 3628800.

Albyn Jones Math 141 Coin Tossing The General Case Example: Permutations of 4 objects

Each row lists permuations beginning with the same object, grouped by the choices for the second object:

1234 1243 1324 1342 1423 1432

2134 2134 2314 2341 2413 2431

3124 3142 3214 3241 3412 3421

4123 4123 4213 4231 4312 4321

Albyn Jones Math 141 Coin Tossing The General Case Practice with Factorials

What is 1! ? What is 2! ? What is 3! ? What is 0! ?

Albyn Jones Math 141 Coin Tossing The General Case A Formal deﬁnition for Factorials

The standard recursive deﬁnition is

0! = 1

For n > 0 ∈ Z, n! = n · (n − 1)! It is useful to remember this recursive deﬁnition! For example it makes obvious the relation n! = (n − 1)! n

Albyn Jones Math 141 Coin Tossing The General Case Combinations

How many ways are there to rearrange the 5 symbols H, H, H, T , T , that is three Heads and two Tails?

Label them uniquely: H1, H2, H3, T4, T5 We have 5 objects, there are 5! orders.

Some of those orders are redundant: {H1, H2, H3, T4, T5} is indistinguishable from {H3, H2, H1, T5, T4} or {H3, H1, H2, T4, T5}. Divide out the number of indistinguishable (or equivalent) patterns of 3 Heads, i.e. 3!, and 2 Tails, i.e. 2!. Answer: 5! 5 · 4 · 3! 5 · 4 = = = 10 3! · 2! 3! · 2! 2 · 1

Albyn Jones Math 141 Coin Tossing The General Case Example: 4 Coin Tosses

4 4 HHHH 4

4 3 HHHT , HHTH, HTHH, THHH 3

4 2 HHTT , HTHT , THHT , TTHH, THTH, HTTH 2

4 1 TTTH, TTHT , THTT , HTTT 1

4 0 TTTT 0

Albyn Jones Math 141 Coin Tossing The General Case Sequences of k Heads in n tosses

For 3 H and 2 T the number of possible orders was 5! 3! · 2! For k H and (n − k) T in n tosses the number of possible orders will be‘ n choose k’: the number of ways to select k objects out of a set of size n.

n n! = k k! · (n − k)!

Note: n n! n = = n − k (n − k)! · k! k

Albyn Jones Math 141 Coin Tossing The General Case Practice with Binomial Coefﬁcients

What is each of the following?

n 0

n n

n 1

n n − 1

n 2

Albyn Jones Math 141 Coin Tossing The General Case The General Case

In general, we might be working with dichotomous trials where the event of interest has probability p, possibly not 1/2. Most of what we have learned about coin-tossing carries over to the general case.

Albyn Jones Math 141 Coin Tossing The General Case The Binomial Distribution X ∼ Binomial(n, p)

Dichotomous Trials: Each trial results in either a ‘Success’, S, or a ‘Failure’ F. Independence: Trials are mutually independent; knowing we got S on one trial does not help us predict the outcome of any other trial. Constant probability: Each trial has the same probability P(S) = p, and P(F) = 1 − p. X counts the number of S’s. n the number of trials is ﬁxed.

Albyn Jones Math 141 Coin Tossing The General Case Binomial(n, p) Probabilities

Independence: Every sequence of n trials with k successes has the same probability!

P(SSF) = P(S)P(S)P(F) = P(F)P(S)P(S) = P(FSS)

Let p = P(S) and q = 1 − p = P(F), then for a RV X ∼ Binomial(n, p)

n (X = k) = pk qn−k P k

n Again, k counts the number of sequences of length n with k successes.

Albyn Jones Math 141 Coin Tossing The General Case Example

Suppose we roll a fair die 5 times. What is the probability we get no ones? What Binomial distribution should we use? Let S be the event we roll a 1. If the die is fair, p = P(S) = 1/6. n = 5, as there are 5 trials. Let X be the number of 1’s we get in the 5 rolls. Then

1 X ∼ Binomial(5, ) 6 Thus

5 10 55 3125 (X = 0) = = 1 · 1 · ∼ 0.402 P 0 6 6 7776

Albyn Jones Math 141 Coin Tossing The General Case Binomial(10,1/2) 0.20 0.15 0.10 0.05 0.00 0 1 2 3 4 5 6 7 8 9 10

Albyn Jones Math 141 Coin Tossing The General Case Binomial(10,1/5) 0.30 0.25 0.20 0.15 0.10 0.05 0.00 0 1 2 3 4 5 6 7 8 9 10

Albyn Jones Math 141 Coin Tossing The General Case Sums of Binomial RV’s

Suppose that X ∼ Binomial(n, p) and Y ∼ Binomial(m, p) are independent Binomial RV’s with the same probability p. What is the distribution of X + Y ? X + Y ∼ Binomial(n + m, p)!

Albyn Jones Math 141 Coin Tossing The General Case R functions

Density Function: P(X = k) = dbinom(k, n, p) (Cumulative) Distribution Function: P(X ≤ k) = pbinom(k, n, p) Random numbers: rbinom(N, n, p) 1 Last Example: X ∼ Binomial(5, 6 ), P(X = 0) = dbinom(0, 5, 1/6) = 0.4018776

Albyn Jones Math 141 Coin Tossing The General Case Binomial Random Walk Random Walk functions on 141 website

RW = function(n) { x = sample(c(-1,1),size=n,replace=T) rw = cumsum(x) plot(1:n,rw,xlab="N",ylim=c(-3.1*sqrt(n),3.1*sqrt(n)),type="l") abline(h=0,lty=2) }

RW1= function(n,color="blue") { x = sample(c(-1,1),size=n,replace=T) rw = cumsum(x) lines(1:n,rw,col=color) }

Albyn Jones Math 141 Coin Tossing The General Case Summary

There are n! permutations of n objects. Binomial coefﬁcients, the number of subsets of size k from a set of n objects:

n n! = k k!(n − k)!

The Binomial Distribution: n (X = k) = pk qn−k P k

Albyn Jones Math 141