1 Probability 1.1 Introduction The next topic we want to take up is probability. Probability is a mathematical area that studies randomness. The question about what exactly "random" means is something for philosophers to ponder. Basically we call a process a random experiment if its exact outcome is unpredictable, haphazard, and without pattern. We usually can just recognize randomness when we see it. Rolling a pair of dice is a random experiment, as is tossing a coin and drawing a card from a well shu• ed deck. In mathematics something is said to be random when it can reasonably be as- sumed that individual results are unpredictable. Probability Theory, however, provides a mathematical way to make predictions about the results anyway. Probability can make rather strong predictive statements about repeated ran- dom events. The results of an individual event remain a surprise, but after a large enough number of repetitions, the overall results can form very strong patterns. Probability theory is the mathematical framework for describing those patterns. Probability cannot predict exact results, but can make very strong statements about general results. Probability theory is the mathematical background of the normal intuition we all have about random things. Consider the following example. Suppose you go for co¤ee each morning with a co-worker. Rather than argue over it, you strike a deal to determine who should pay. The co-worker ‡ips a quarter; if it comes up heads, he pays; if it comes up tails, you pay. All this seems fair until you realize that, after 10 days of this, you have paid every time. On day eleven, you ask to examine the coin. Your intuition about how a fair coin toss works causes you to question the fairness of your co-worker’s coin toss. It is not that it is impossible for a coin to come up tails 10 consecutive times, but it seems highly unlikely that it would happen the …rst ten times you use it in this game. The result of one coin ‡ip should be totally unpredictable. This means you can assume that the coin ‡ip will produce random results. But randomness is predictable over the long run. In making the deal, you and the co-worker expect that, over the long term, each should pay a roughly equal amount. In 10 tries, it is too hopeful to expect exactly 5 heads and 5 tails. A split of 4-6 or 6-4 should not be that suspicious. Even 3-7 or 7-3 might be chalked up to an odd string of ‡ips. But 0-10 is enough to get even the most trusting person wondering if more than luck is involved. A coin ‡ip is a random event; 10 coin ‡ips are just as random. However, taken collectively 10 coin ‡ips are de…nitely more predictable. 1.2 Terminology Before we get started we set out our vocabulary. Probability theory can get complicated, but using the right words can help avoid a lot of confusion. 1 Experiment: an experiment is any process that produces unpredictable outcomes. Sample Space: a list of every possible individual outcome of an experi- ment. Sample Points: these are individual outcomes in the sample space. Event: an event is any outcome or collection of outcomes of an experiment. Now the same physical experiment can have several di¤erent sample spaces, and in fact, one important technique in probability is changing the sample space. A sample space may be selected for several reasons. If we are interested in collecting data on the occurrence of speci…c results, we may choose a sample space that makes collecting that data easy. If we intend to do additional theoretical probability or statistics work with our experiment, we may choose a sample space just for its theoretical properties. If we have an experiment with a natural sample space that happens to be large or cumbersome, we may choose to combine parts to form a more convenient sample space. For an example consider the experiment produced by rolling two dice on a Monopoly board. What is a good sample space? One possibility is to list all the possible totals on the two dice. This would give 2; 3; 4; 5; 6; 7; 8; 9; 10; 11; 12 . f g But we all know that occasionally one die gets hung up on the Chance Deck. Should that be listed as a possible outcome. If we are actually interested in this case, then "yes" it should be in the sample space. However, in the normal play of the game, a "cocked die" leads to a "do-over." Basically that possible outcome is ignored, and it does not count. If we follow this approach, the sample space above is su¢ cient. Suppose we …nd ourselves in Monopoly jail. To get out, we need to roll doubles; that is we need to roll two 1’s, two 2’s,two 3’s,two 4’s,two 5’s or two 6’s. Anything else keeps us in jail. A reasonable sample space for this version of the experiment is Doubles, Not Doubles : f g Once we realize that it matters in the game if we roll doubles or not, we may need a sample space that keeps track of totals and doubles. Now the collec- tion 2; 3; 4; 5; 6; 7; 8; 9; 10; 11; 12; Doubles, Not Doubles is NOT a sample space. Thef outcomes in the list are not distinct possible results.g If both dice come up 6’s, then this outcome appears in the list as both a total of 12 and as doubles. Each possible outcome of the experiment should have a unique identi…er in the sample space. To keep track of both the total and doubles, we need to expand our sample space and to introduce events into the mix. A better sample space for the rolled dice might be 1; 1 ; 1; 2 ; 1; 3 ; 1; 4 ; 1; 5 ; 1; 6 ; 2; 2 ; ff g f g f g f g f g f g f g 2; 3 ; 2; 4 ; 2; 5 ; 2; 6 ; 3; 3 ; 3; 4 ; 3; 5 ; f g f g f g f g f g f g f g 3; 6 ; 4; 4 ; 4; 5 ; 5; 5 ; 5; 6 ; 6; 6 f g f g f g f g f g f gg 2 We use brackets here to signify that the order of the numbers inside does not matter. We are still interested in the dice totals and whether the roll produced doubles or not. Now we are interested in events in this sample more than the sample points. The events we are interested in are Total 2 = 1; 1 ff gg Total 3 = 1; 2 ff gg Total 4 = 1; 3 ; 2; 2 ff g f gg Total 5 = 1; 4 ; 2; 3 ff g f gg Total 6 = 1; 5 ; 2; 4 ; 3; 3 ff g f g f gg Total 7 = 1; 6 ; 2; 5 ; 3; 4 ff g f g f gg Total 8 = 2; 6 ; 3; 5 ; 4; 4 ff g f g f gg Total 9 = 3; 6 ; 4; 5 ff g f gg Total 10 = 4; 6 ; 5; 5 ff g f gg Total 11 = 5; 6 ff gg Total 10 = 6; 6 ff gg Doubles = 1; 1 ; 2; 2 ; 3; 3 ; 4; 4 ; 5; 5 ; 6; 6 ff g f g f g f g f g f gg Not Doubles = 1; 2 ; 1; 3 ; 1; 4 ; 1; 5 ; 1; 6 ; 2; 3 ff g f g f g f g f g f g 2; 4 ; 2; 5 ; 2; 6 ; 3; 4 ; 3; 5 ; 3; 6 ; f g f g f g f g f g f g 4; 5 ; 4; 6 ; 5; 6 f g f g f gg There is another good choice for a sample space in this experiment. Its advantages are theoretical, and we will see later what those advantages are. In this sample space possibility, we will keep track of which dice comes up with which number. In practice, we might need to use two di¤erent color dice to do this, but theoretically all we need do is assume we can tell the dice apart. We list all possible outcomes keeping track of each die. To signify that the order of the numbers matters, we use parentheses to group the pairs. (1; 1); (1; 2); (1; 3); (1; 4); (1; 5); (1; 6); f (2; 1); (2; 2); (2; 3); (2; 4); (2; 5); (2; 6); (3; 1); (3; 2); (3; 3); (3; 4); (3; 5); (3; 6); (4; 1); (4; 2); (4; 3); (4; 4); (4; 5); (4; 6); (5; 1); (5; 2); (5; 3); (5; 4); (5; 5); (5; 6); (6; 1); (6; 2); (6; 3); (6; 4); (6; 5); (6; 6) g 3 Notice that the sample points in all the previous sample spaces are identi…ed as events in this sample space: 3; 5 (3; 5); (5; 3) f g f g 2; 2 (2; 2) f g f g Total 4 = (1; 3); (2; 2); (3; 1) f g Total 5 = (1; 4); (2; 3); (3; 2); (4; 1) f g Doubles = (1; 1); (2; 2); (3; 3); (4; 4); (5; 5); (6; 6) f g We have the following terms Experiment: an experiment is any process that produces unpredictable outcomes. Sample Space: a list of every possible individual outcome of an experi- ment. Sample Points: these are individual outcomes in the sample space.