SI 486H, Unit 2: Data Structures Daniel S. Roche ([email protected]) Spring 2016 In this unit, we are going to look at some randomized data structures. We’ll go into more detail on this later, but it is important to recognize up front what is random and what is not. In a randomized data structure, like any other data structure, the contents are not chosen randomly; they are picked by whoever is using the data structure. What is going to be random is the way those items are organized internally, the links and ordering of the user’s input to facilitate efficient retrieval. We will see how choosing this structure randomly gives the same expected behavior as the more complicated data structures you’ve already learned about. 1 Fun with probability distributions This unit is going to do a lot of big-O running time analysis with randomized data structures. Let’s start by reviewing some probability tools that will come into play. Actually, you’ve already been using some of these ideas in your problems - now let’s write them down and give them some names. Disjunction:A disjunction of two events A and B, written A ∪ B, means the event where either A or B (or both) happens. It’s an or operation with probabilities, and you can write down the probability using the inclusion-exclusion principle as follows: Pr[A ∪ B] = Pr[A] + Pr[B] − Pr[A ∩ B] This corresponds to what you see if you look at a Venn diagram. Frequently, we would rather not try and compute the intersection probability Pr[A ∩ B]. In that case, we can get an upper-bound estimate by just ignoring that part of the equation. And then it extends trivially to any number of events as well: Union Bound Pr[A1 ∪ A2 ∪ · · · ∪ An] ≤ Pr[A1] + Pr[A2] + ··· Pr[An] I know, it seems pretty obvious at first glance, but it will be useful especially when the events A1,...,An are all pretty rare and we just need to say “none of these things will (probably) happen.” Conjunction: This is the and operation with probabilities, written Pr[A ∩ B]. There’s not always a simple formula for this, but there is when A and B are independent events. In probability, “independence” just means that one event has no influence on the other one. For example, if a couple has two children, then (as far as I know) the gender of each is an independent random event. If one sibling is a girl, it doesn’t make the other sibling any more or less likely to be a girl. But other genetic traits - maybe hair color for example - are not independent. You are more likely to have brown hair if you have a sibling that also has brown hair. OK, so here’s the formula for conjunction of independent events. Again, it’s pretty simple; the main thing is to remember not to apply it if the events are not actually independent! 1 Conjunction of independent events Pr[A ∩ B] = Pr[A] Pr[B] if A and B are independent events. PDF, CDF, CCDF. We’ve talked a bit so far about probability distributions, which is the way the total realm of possibility is divided up among the possible values of a random variable. There are three functions that can be used to describe a probability distribution. They are, the probability distribution function (PDF), the cumulative distribution function (CDF), and the complementary cumulative distribution function (CCDF). Formal definitions are as follows: Distribution functions If X is a random variable, its distribution is defined by any one of the three functions: • PDF: f(k) = Pr[X = k] • CDF: f(k) = Pr[X ≤ k] • CCDF: f(k) = Pr[X > k] The PDF is the probability that X is exactly equal to some value. This is the most obvious and basic of the three. The CDF and CCDF are obviously related to each other (each equals 1 minus the other), and the crucial difference is the inequality (≤ or >) in their definitions. For some complicated probability distributions that come up with randomized data structures, it’s easier to get an inequality like this than it is to get the exact PDF. To illustrate what this means, here is a table showing the PDF, CDF, and CCDF for the random variable X that represents the sum of 2 fair 6-sided dice rolls: k PDF(k) CDF(k) CCDF(k) 1 0 0 1 2 1/36 1/36 35/36 3 2/36 3/36 33/36 4 3/36 6/36 30/36 5 4/36 10/36 26/36 6 5/36 15/36 21/36 7 6/36 21/36 15/36 8 5/36 26/36 10/36 9 4/36 30/36 6/36 10 3/36 33/36 3/36 11 2/36 35/36 1/36 12 1/36 1 0 2 Expected Value and Probability (Wikipedia) We saw expected value defined in Unit 1, and it’s going to become more and more important as we go along. If you read in the textbook or on most online sources, you’ll see a bunch of discussion about random variables. We’ll try to avoid the formalisms and complictions to the extent possible here though. Remember that expected value is, intuitively, just the average of some quantity weighted according to the probabilities of each quantity value occurring. Take lottery winnings as a simple example: Consider a very simple lottery, where each ticket costs 1 dollar, there is one winner who gets J dollars, and 2 the chances of winning are one in a million. We want to decide how large J needs to be before it’s worth it to play. This will correspond to the expected value of the payoff. Here there are two possibilities: with 999999 1 probability 1000000 we will lose 1 dollar, and with probability 1000000 we will win J − 1 dollars. Therefore the expected payoff is 999999 1 −6 E[winnings] = (−1) + (J − 1) = J · 10 − 1 1000000 1000000 You could solve this equation for an expected value of 0 to discover that, if the jackpot is worth at least a million dollars, we expect to gain money on average by playing. OK, so in this simple example that’s not so surprising. But the same idea holds more generally. For example, there might be some intermediate wins in the lottery with lower payoffs but higher probability, or we might account for the probability of a split jackpot, etc. The point is that expected value is a useful tool to make decisions in the face of uncertainty, when we know the probabilities of each thing occurring. The definition of expected value connects directly with the definition of a PDF: PDF to Expected Value If X is a nonnegative integer-valued random variable, then ∞ X E[X] = Pr[X = i] · i. i=1 Sometimes those sums can be a little annoying to compute because of the multiplication by i, and it’s easier to use the CCDF instead. (For fun, see if you can work out the proof!) CCDF to Expected Value If X is a nonnegative integer-valued random variable, then ∞ X E[X] = Pr[X > i]. i=0 Sometimes we have to deal with multiple random variables - corresponding to multiple events that we’re interested in. When those random variables get combined in complicated ways, it can be difficult to work out the resulting expectation. But if they’re just added up, it’s easy! (Can you prove this one?) Linearity of expectation For any random variables X1,X2,...,Xn, E[X1 + X2 + ... + Xn] = E[X1] + E[X2] + ··· + E[Xn] In other words, the expected value of the sum of a bunch of things, is exactly the same as the sum of the expected values. This means that, if the expected value of my winnings after playing the lottery once is even as small as $0.01, then if I play 1000 times the expected winnings go up to $10. If I could afford to play a million times, well then the expected winnings go up substantially. If this all seems way to obvious, consider (as we will) the expected value of the maximum. You might be tempted to say that E[max(X1,X2)] = max(E[X1], E[X2]). Not so! Consider the height of U.S. males. The average height of adult males is 5’10“, or 70 inches. That means that the expected height of a single randomly-chosen adult male is exactly 70 inches. 3 But what if I choose 100 U.S. males randomly, and consider the maximum height among those 100? Since they all have the same expected height, the maximum expected height among them is exactly 70 inches. But would you actually expect the tallest among those 100 to be 70 inches? I think not. We would need more information to determine what the expected value of the maximum is precisely, but certainly it is greater than the maximum of the expected values. More on this later. One last thing on expected value: sometimes you need to connect expectation back to probability. For example, consider two gambling games. In the first, I bet you a dollar that a coin flip comes up heads. It costs me a dollar to play, and if it’s heads, I get two dollars back. So the expected value here would be 1 dollar. The second gambling game is a lottery where you spend one dollar to play, and there is a million-dollar jackpot which you have a one-in-a-million chance of winning.