DEPARTMENT OF MATHEMATICS UNIVERSITY OF CALIFORNIA, BERKELEY C Probability theory H.W. Lenstra, Jr. These notes contain material on probability for Math 55, Discrete mathematics. They were written to supplement sections 3.6 and 3.7 of C. L. Liu, Elements of discrete math- ematics McGraw-Hill, second edition, 1985, but they can b e used in conjunction with other textb o oks as well. Novemb er 1988. 1. Sample spaces, events and conditional probabilities. A sample space is a nite or countable set S together with a function P : S ! [0; 1] = fy 2 R :0 y 1g; such that the following condition is satis ed: X P x=1: x2S One thinks of S as the set of all p ossible outcomes of an exp eriment, with P x equal to the probability that the outcome is x.We call P the probability function. In manyinteresting cases S is nite and P x=1=S for all x 2 S . Let S b e a sample space, with probability function P .Anevent is a subset A of S , and the probabilityofanevent A is given by X P A= P x: x2A If A, B are twoevents, and P B 6= 0, then the conditional probability of A given B is de ned by P A \ B : P AjB = P B Examples of sample spaces, events and conditional probabilities, and exercises on them, can b e found in many textb o oks on discrete mathematics. 1 2. Indep endence. Twoevents A, B are called indep endent if P A \ B =P A P B . This means the same as P AjB =P A if P B 6= 0, so twoevents are indep endent if the o ccurrence of one of them do es not make the other more likely or less likely to o ccur. Example 1. Let S = f1; 2; 3; 4; 5; 6gf1; 2; 3; 4; 5; 6g, the set of outcomes of rolling 1 two dice, with each outcome having probability . Consider the events 36 A = the rst die equals 3, B = the second die equals 4, C = the total equals 7, D = the total equals 6, more precisely A = f3gf1; 2; 3; 4; 5; 6g; B = f1; 2; 3; 4; 5; 6g f4g; C = f1; 6; 2; 5; 3; 4; 4; 3; 5; 2; 6; 1g; D = f1; 5; 2; 4; 3; 3; 4; 2; 5; 1g: 1 = P A P B . Also A and C are Here A and B are indep endent, since P A \ B = 36 indep endent, and B and C are indep endent. However, A and D are not indep endent, since 1 1 5 1 P A \ D = , whereas P A P D = , which is smaller than . Likewise B and 36 6 36 36 D are not indep endent. Are C and D indep endent? Warning.Anytwo of the three events A, B , C are indep endent, but nevertheless the three events A, B , C cannot b e considered indep endent, since the o ccurrence of anytwoof them implies the o ccurrence of the third one. For three events A, B , C to b e indep endent one requires not only that anytwo are indep endent, but also that P A \ B \ C =P A P B P C . For more than three events one has to lo ok at even more combinations, see for example Exercise 25 a. 2 3. Random variables. It frequently happ ens that one is not interested in the actual outcome of the exp eriment, but in a certain function of the outcome. For example, if one throws two dice one is often just interested in the total score; or if one tosses a coin n times, it may b e that one is just interested in the numb er of heads app earing, or in the longest consecutive sequence of tails. These are examples of random variables. Let, generally, S b e a sample space, with probability function P .Arandom variable is a function f : S ! R.Ifr is a real numb er in the image of f , then the probability that f assumes the value r is de ned by X P f = r = P x: x2S; f x=r This is the same as the probability of the event f x=r . The exp ectation or exp ected value,ormean of a random variable f is de ned by X f xP x: E f = x2S This may b e thought of as the \average" value of f if one rep eats the exp eriment a great numb er of times. An alternative formula for E f is X E f = r P f = r : r 2f S To prove this, notice that for each r 2 f S wehave X f xP x=r P f = r ; x2S; f x=r by the de nition of P f = r . Now sum this over r 2 f S . 1 , and let f : S ! R b e the inclusion Example 2. Let S = f0; 1g, with P 0 = P 1 = 2 1 map so f 0 = 0, f 1 = 1. Then E f = . 2 1 Example 3. Let S = f1; 2; 3; 4; 5; 6g, with P x= for each x 2 S throwing one die, 6 1 and let f : S ! R again b e the inclusion map. Then E f = 1+2+3+4+5+6=6= 3 . 2 If f , g are two random variables, then f + g is also a random variable; namely,f + g x is de ned to b e f x+g x, for all x 2 S . One has E f + g =E f +E g : 3 This follows directly from the de nition. Example 4. Let S , P b e as in Example 1 rolling two dice. If f is the value of the rst die so f a; b = a, and g is the value of the second die g a; b = b, then 1 . The total is f + g , so the exp ectation of the total score equals E f =E g =3 2 E f + g = E f +E g =7. 4. Events as random variables. Let S b e a sample space, with probability function P . Let A b e an event; so this is a subset of S . It gives rise to a random variable f that is de ned by n 1 if x 2 A, f x= 0 if x=2 A. This random variable assumes only the values 0 and 1; and any random variable that assumes only the values 0 and 1 arises in this way from some event. The probabilityof the event A is the same as the exp ectation E f of the corresp onding random variable f . Example 5. Let p 2 R,0<p<1, and write q =1 p. Let S b e the sample space f0; 1g, with probability function given by P 1 = p, P 0 = q . This sample space can b e thought 1 of as the set of outcomes of tossing a coin that is not fair unless p = . The probability 2 of the event A = f1g is the same as the exp ectation of the inclusion map see Example 2, namely p. Example 6. Rep eat the exp eriment from the previous example n times; so S must then n k nk b e replaced by f0; 1g , with x 2 S having probability p q if x contains exactly k 1's any x 2 S is a sequence of n 0's and 1's. For each i =1,2, :::, n, let A b e the event i that the i-th toss of the coin gives a 1, and let f b e the corresp onding random variable; i so if x =x ;x ;:::; x , then f x=x . Each A has probability p, so each f has 1 2 n i i i i exp ectation p. Now let f = f + f + + f . Then f x is the total numb er of 1's in x, and 1 2 n E f =E f + f + + f =E f +E f + + E f =np: 1 2 n 1 2 n So the exp ected numb er of 1's is np.We can also verify this by a direct calculation. First one notes that n k nk p q ; P f = k = k n b ecause there are elements x 2 S that have exactly k 1's, and each such x has proba- k k nk bility p q .Now n n X X n 1 n k 1 nk n1 k nk p q = npp + q = np: p q = np k E f = k 1 k k =1 k =0 4 A random variable f : S ! R is said to b e a binomial random variable,ortohavea binomial distribution, if there exists an integer n 0 and a real number p,0<p <1, such that 1 , then we see not surprisingly that the holds for all k ,0 k n.Ifwe take p = 2 1 average numb er of elements of a subset of a set of n elements equals n. 2 Example 7. Ten men went to a party and checked their hats when they arrived. The hats were randomly returned to them when they departed. Wewant to know the exp ected numb er of men that get their own hats back. In this example, S is the set of bijective functions of a set M of 10 men to itself, all these functions having the same probability, : whichis1=S =1=10! = 1=3628800 =0:0000002755732.