I Was Wondering How Exactly Do You Decide Which Approach to Use When Figuring out The

Frequently Asked Questions: Basic Probability

1. What does “correlation” mean? If two events are correlated, then they tend to change together. Knowing something about the one event tells you something about the other. If two events are independent, then the occurrence or nonoccurrence of the one tells you nothing about the other. Consider this example. I look out the window and tell you “I see a person coming out of the Wright building. What is your subjective probability that that person is female? Write it down on paper.” Then I tell you that the person’s last name is “Jones” and I invite you to revise your subjective probability. Since in your experience there has been no relationship between having a last name of “Jones” and being of the one or the other sex, your new subjective probability is unchanged. You consider the two events (being of the one sex or the other and having a last name of “Jones”) to be independent, that is, not correlated. Then I add, “and Jones is wearing a skirt, you may revise your subjective probability again, if you wish.” Now you probably would revise your subjective probability, upwards, since in your experience people who wear skirts are usually female. You think of being female and wearing a skirt as positively correlated (they tend to go together), not independent. Likewise, you think of wearing a skirt and being male as negatively correlated – knowing that the one event has happened makes the other event less likely.

2. How does one decide which approach to use when estimating a probability – empirical, rational, or subjective? It is both a matter of personal preference and the data that are available. Suppose I told you that I have a box of poker chips. Some are red, some are white, some are blue. I ask you to estimate the probability of drawing, on a single trial, a blue chip. You ask me if you can first experiment with the box of chips. I say “sorry, no.” You think about it (the rational approach). You make the assumption that the three types of chips are all equally likely to be drawn (there are equal numbers of them in the box, they are all the same size and shape, and they are randomly distributed within the box). You know (or assume) that drawing a blue chip and drawing a not blue chip are mutually exclusive events – a chip cannot be both blue and not blue, and the same is true of red versus not red and white versus not white. You also assume that the drawing blue, drawing red, and drawing white are mutually exhaustive events – there are no other events that can happen. Accordingly, you reason that P(blue) + P(red) + P (white) = 1 or 100%. Since you have assumed that the three events are equally likely, that is, P(blue) = P(red) = P(white), each of the probabilities must be 1/3.

Now I offer you 3:1 odds against the drawn chip being blue (this is just another way of saying 1:3 odds in favor of the drawn chip being blue). You are thinking that is a good bet (better than fair) if betting on blue. If you are right, and the P(blue) is 1/3, then 2:1 odds would be fair. You take the bet. You put in $1, I put in $2. You draw a chip from the box. It is white. You lose a dollar. Hey, what do you expect, you probably would not hit blue on the first draw, bet again. We go through ten bets, and you get only one blue chip. You won $2 on that one bet but lost $9 on the other nine. That does not sound fair. I want you to keep on betting. You say “only if I can experiment with the box first.” Reluctantly, I say “OK.” You do a 1,000 trial experiment. On each trial you draw one chip, note its color, and then put it back in the box and shake the box well. In 1,000 trials you draw blue chips only 102 times. From that result, you estimate the probability of drawing a blue chip to be 102/1,000 = about 10%. This is the empirical approach. After you finish, I say “OK, lets bet.” You say “OK, but the bet won’t be fair unless the odds are 9:1 against getting a blue chip.” The subjective approach is whatever it is that humans do when making decisions that involve the likelihood of something happening or not. Although there are strong common threads here, there are also individual differences. Generally speaking, humans make a lot of errors in this domain. There is a large and interesting research literature on this topic, and I would love to talk about it, but, frankly, we just do not have the time to talk about that right now.

3. What is the relationship between the addition rule and mutual exclusion?

Suppose that I ask you to bet on whether the next chip drawn from the box will be blue or red. From your experiment you know that the P(blue) = 10% and the P(red) = 30%. The addition rule is P(A  B) = P(A) + P(B) if A and B are mutually exclusive. You did not see any chips that were both red and blue, so you estimate the P(blue or red) to be 10% + 30% = 40%. A fair bet would be 60:40 or 3:2 odds against the chip being blue or red. Now I bring out a new box of chips. I promise you that for this box P(blue) = 10% and P(red) = 30%. What is the probability that a randomly drawn chip will be blue or red? What I do not tell you is that 5% of the chips are red with blue polka dots, that is they are both blue and red. Now being blue and being red are not mutually exclusive. In this case the simplified addition rule (above) will not work, we need the more general addition rule, P(A  B) = P(A) + P(B) - P(A  B). P(blue or red) = 10% + 30% -5% = 35%, Look back at my handout “Basic Probability, Including Contingency Tables” for another example of the addition rule when events are not mutually exclusive, where we would have decided that 127% of female students are going to hell if we did not take into account that smoking and being sexually active are not mutually exclusive. Consider the diagram to the right. The area of the red circle on the left represents P(A). The area of the blue circle on the right represents P(B). The overlap between the two circles, with the purple dots, represents P(AB), the joint occurrence of A and B – events that are both A and B. A and B are not mutually exclusive. If we were to calculate P(A  B) = P(A) + P(B) we would count the overlap area twice. To correct for this, we need to subtract out the overlap area once, that is, P(A  B) = P(A) + P(B)  P(A  B).