Dicing with the Unknown
Total Page:16
File Type:pdf, Size:1020Kb
focus DDicingicing wwithith tthehe uunknownnknown There are many things that I am uncertain about, says Tony O’Hagan. Some are merely unknown to me, while others are unknowable. This article is about different kinds of uncertainty, and how the distinction between them impinges on the foundations of Probability and Statistics. Two kinds of uncertainty certainties are potentially reducible by further investigation. But it is easy to see how much There are things that I am uncertain about sim- more fundamental it should be for statisticians, ply because I lack knowledge, and in principle for whom randomness and uncertainty are their my uncertainty might be reduced by gathering very raison d’être. more information. Others are subject to random The theory of Statistics rests on describing variability, which is unpredictable no matter how uncertainties by using probability. A probability much information I might get; these are the un- near 1 represents an event that is almost certain knowables. The two kinds of uncertainty have to occur, while a probability near 0 represents been debated by philosophers, who have given one that is almost certain not to occur. As we them the names epistemic uncertainty (due to move away from these extremes towards the lack of knowledge) and aleatory uncertainty (due probability of ½, there is increasing uncertainty. to randomness). Here, however, is where we meet the fundamental Examples of aleatory uncertainty are famil- dichotomy between the two principal theories of iar to students of probability theory, and include Statistics: the frequentist and Bayesian theories. the outcomes of tossing dice and drawing cards One characterisation of the difference between from a shuffl ed pack. In statistics, aleatory un- these two schools of statistical theory is that fre- certainty is present in almost all data that we Charles Trevelyan quentists do not accept that aleatory uncertainty obtain, due to random variability between the can be described or measured by probabilities, members of a population that we sample from, while Bayesians are happy to use probabilities to or to random measurement errors. quantify any kind of uncertainty. Examples of epistemic uncertainty are all around us. I am uncertain about the atomic weight of zinc, about the population of the city Two kinds of probability of Paris, and about whether the river Thames froze over in London during the winter of Delving more deeply, the root of this disagree- 1600–1601. At least for the fi rst two of these, ment lies in what probability means. Almost my uncertainty could be resolved by looking in everyone who encounters probability for the fi rst a suitable reference book. It may be that there time in their education will be taught it using is no such source of information about the river aleatory uncertainties, like the familiar games of Thames in 1600–1601, but in principle this is tossing dice or coins. And the way probability will a question that might be resolved by histori- be taught is that it represents the long-run fre- cal investigation. Epistemic uncertainty about quency with which the event in question occurs any given question varies from one person to if it is repeated an indefi nite number of times, another. For instance, I have negligible uncer- and this is accordingly known as the frequency tainty about my height. Someone who has seen defi nition of probability. The nature of random me might be able to guess reasonably accurately events is that they are, at least conceptually, re- but could not be certain. Someone who knows peatable in this way. Epistemic uncertainties are only that I am a British male would have more generally not. uncertainty. If probability is to encompass epistemic un- The distinction between aleatory and epis- certainty it needs another defi nition. temic uncertainties is valuable in many areas In Bayesian statistics the probability of a where it is important to appreciate which un- proposition simply represents a degree of belief 132 september2004 112-focus.indd2-focus.indd 113232 112/08/20042/08/2004 115:30:505:30:50 PProcessrocess CCyanyan PProcessrocess MMagentaagenta PProcessrocess YYellowellow PProcessrocess BBlacklack PPANTONEANTONE 11807807 C in the truth of that proposition. Notice in pass- interval were computed from each of the same ing that we tend to use the word “proposition” infi nite sequence of data sets, then 95% of those Six of one and half a (a statement that is either true or false) rather intervals will contain the true value of the pa- than “event” (something which may or may not rameter. dozen of the other occur) when discussing this kind of probability, Neither of them says anything about the If I toss an ordinary coin, my probabil- since “event” has connotations of randomness chance that the null hypothesis is true, or that ity that it will land Heads is 0.5. Suppose and repeatability. A proposition might simply as- the parameter lies in the interval, for these data. now that I have a bag of poker chips, and sert that an event will occur, but it may also refer If we condition on the single set of data in front I know only that some are red and some to a statement with epistemic uncertainty. of us, there is no randomness in the problem, and are green. I have no reason to suppose that The degree-of-belief interpretation of prob- so no frequentist probabilities can be stated. there are equal numbers of red and green ability is sometimes referred to as personal prob- In contrast, Bayesian inference does make chips. Indeed, almost certainly one colour ability or subjective probability because, as noted probability statements about parameters. It can will be more abundant than the other, but already, different people may have different de- do so because the epistemic uncertainty in pa- I have no idea which colour that will be, grees of uncertainty about a proposition. It is rameters can be quantifi ed using the Bayesian’s or how much more abundant it will be than the “subjectivity” of this approach to probabil- personal probability. Indeed, Bayesian inference the other colour. If one chip is to be pulled ity that is most objected to by followers of the describes how the acquisition of data modifi es out of the bag my probability that it will be frequentist theory. Bayesians steadfastly defend (and usually reduces) the uncertainty about red is 0.5. this defi nition, and question the extent to which a parameter, from “prior” uncertainty to “pos- Now surely my uncertainties in the coin their methods are any more subjective than fre- terior” uncertainty. The Bayesian equivalent of toss and the poker chip draw are different— quentist practice, or indeed scientifi c practice a signifi cance test asserts the probability that the coin toss being very familiar and the generally. However, that debate is beyond the the null hypothesis is true. In the same way, the bag of poker chips full of uncertainty—so scope of this article! Bayesian analogue of the confi dence interval why do I give both events the same prob- (usually called a credible interval) has exactly the ability? It is true that the uncertainties interpretation that is so often erroneously attrib- Two kinds of statistics are different. The uncertainty about the uted to the frequentist confi dence interval. coin toss is purely aleatory, whereas there To see the implications of the frequentist po- is clearly epistemic uncertainty about the sition on probability, it is enough to note that And two kinds of statistician make-up of the bag of chips. Nevertheless, uncertainty about parameters in statistical mod- for a single coin toss and a single poker chip els is almost invariably epistemic. If, for instance, On a personal note, I have been both kinds of all the uncertainty is quantifi ed in a single I was conducting experiments to measure empiri- statistician in my career. It was my experience, probability, that of Heads or red. cally the atomic weight of zinc, the unknown pa- as a young statistician, of analysing data and The difference emerges when I consider rameter is that atomic weight. I cannot consider producing frequentist tests and confi dence inter- a sequence of tosses of that coin, and a zinc as a randomly chosen element. Indeed, it vals for other scientists that convinced me that sequence of chips drawn from the bag. My is particularly zinc that I wish to know about. the Bayesian approach is the right one for sta- uncertainty about the coin tosses is still In a similar way, nearly all statistical analysis tistical analysis. I had great diffi culty persuading purely aleatory. No matter how many times is to learn about parameters, and so to reduce the scientists not to misinterpret the frequentist I toss the coin, my uncertainty about get- our epistemic uncertainty about them. Since fre- inferences I was giving them. And it was clear to ting Heads on the next toss is the same, and quentist statistics does not and cannot quantify me that this was because the correct interpreta- is expressed by a probability of 0.5. On the that uncertainty with probabilities, conventional tion was of no use to them. Frequentist infer- other hand, as I draw chips from the bag statistical inferences (such as signifi cance tests ences make only indirect statements about pa- my epistemic uncertainty about its compo- and confi dence intervals) never make probability rameters, and can only be interpreted in terms of sition reduces, and my probability for the statements about parameters. repeated sampling. Bayesian inferences directly next chip being red changes according to Yet the recipients of those conventional in- answered the scientists’ questions, making state- the chips I have now seen.