This page intentionally left blank Symmetry and Its Discontents This volume brings together a collection of essays on the history and philosophy of probability and statistics by one of the eminent scholars in these subjects. Written over the last fifteen years, they fall into three broad categories. The first deals with the use of symmetry arguments in inductive probability, in particular, their use in deriving rules of succession (Carnap’s “continuum of inductive methods”). The second group deals with four outstanding individuals who made lasting contributions to probability and statistics in very different ways: Frank Ramsey, R. A. Fisher, Alan Turing, and Abraham de Moivre. The last group of essays deals with the problem of “predicting the unpredictable” – making predictions when the range of possible outcomes is unknown in ad- vance. The essays weave together the history and philosophy of these subjects and document the fascination that they have exercised for more than three centuries. S. L. Zabell is professor of mathematics and statistics at Northwestern Uni- versity. A Fellow of the Institute of Mathematical Statistics and the American Statistical Association, he serves on the editorial boards of Cambridge Studies in Probability, Induction, and Decision Theory, and The Collected Works of Rudolph Carnap. He received the Distinguished Teaching Award from North- western University in 1992. Cambridge Studies in Probability, Induction, and Decision Theory General editor: Brian Skyrms Advisory editors: Ernest W. Adams, Ken Binmore, Jeremy Butterfield, Persi Diaconis, William L. Harper, John Harsanyi, Richard C. Jeffrey, James M. Joyce, Wlodek Rabinowicz, Wolfgang Spohn, Patrick Suppes, Sandy Zabell Ellery Eells, Probabilistic Causality Richard Jeffrey, Probability and the Art of Judgment Robert C. Koons, Paradoxes of Belief and Strategic Rationality Cristina Bicchieri and Maria Luisa Dalla Chiara (eds.), Knowledge, Belief, and Strategic Interactions Patrick Maher, Betting on Theories Cristina Bicchieri, Rationality and Coordination J. Howard Sobel, Taking Chances Jan von Plato, Creating Modern Probability: Its Mathematics, Physics, and Philosophy in Historical Perspective Ellery Eells and Brian Skyrms (eds.), Probability and Conditionals Cristina Bicchieri, Richard Jeffrey, and Brian Skyrms (eds.), The Dynamics of Norms Patrick Suppes and Mario Zanotti, Foundations of Probability with Applications Paul Weirich, Equilibrium and Rationality Daniel Hausman, Causal Asymmetries William A. Dembski, The Design Inference James M. Joyce, The Foundations of Causal Decision Theory Yair Guttmann, The Concept of Probability in Statistical Physics Joseph B. Kadane, Mark B. Schervish, and Teddy Seidenfeld (eds.), Rethinking the Foundations of Statistics Phil Dowe, Physical Causation Sven Ove Haussan, The Structure of Values and Norms Paul Weirich, Decision Space Symmetry and Its Discontents Essays on the History of Inductive Probability S. L. ZABELL Northwestern University Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, São Paulo Cambridge University Press The Edinburgh Building, Cambridge ,UK Published in the United States of America by Cambridge University Press, New York www.cambridge.org Information on this title: www.cambridg e.org /9780521444705 © S. L. Zabell 2005 This publication is in copyright. Subject to statutory exception and to the provision of relevant collective licensing agreements, no reproduction of any part may take place without the written permission of Cambridge University Press. First published in print format 2005 - ---- eBook (NetLibrary) - --- eBook (NetLibrary) - ---- hardback - --- hardback - ---- paperback - --- paperback Cambridge University Press has no responsibility for the persistence or accuracy of s for external or third-party internet websites referred to in this publication, and does not guarantee that any content on such websites is, or will remain, accurate or appropriate. For Dick Jeffrey, mentor and friend. Contents Preface page ix part one. probability 1. Symmetry and Its Discontents3 2. The Rule of Succession 38 3. Buffon, Price, and Laplace: Scientific Attribution in the 18th Century 74 4. W. E. Johnson’s “Sufficientness” Postulate 84 part two. personalities 5. The Birth of the Central Limit Theorem [with Persi Diaconis] 99 6. Ramsey, Truth, and Probability 119 7. R. A. Fisher on the History of Inverse Probability 142 8. R. A. Fisher and the Fiducial Argument 161 9. Alan Turing and the Central Limit Theorem 199 part three. prediction 10. Predicting the Unpredictable 217 11. The Continuum of Inductive Methods Revisited 243 Index 275 ix Preface These essays tell the story of inductive probability, from its inception in the work of Thomas Bayes to some surprising current developments. Hume and Bayes initiated a dialogue between inductive skepticism and probability the- ory that persists in various forms throughout the history of the subject. The skeptic insists that we start in a state of ignorance. How does one quan- tify ignorance? If knowledge gives rise to asymmetric probabilities, perhaps ignorance is properly characterized by symmetry. And non-trivial prior sym- metries generate non-trivial inductive inference. Then perhaps symmetries are not quite such innocent representations of ignorance as one might have thought. That is a sketch of the theme that is developed in the title essay, “Symmetry and its Discontents”, and that runs throughout the book. In the second section of this book, we meet Sir Alexander Cuming, who instigated important investigations by De Moivre and Stirling, before be- ing sent to prison for fraud. We view Ramsey’s famous essay “Truth and Probability” against the Cambridge background of Robert Leslie Ellis, John Venn and John Maynard Keynes. Fisher’s discussion of inverse probabili- ties is set in the context of Boole, Venn, Edgeworth and Pearson and his various versions of the fiducial argument are examined. We learn of Alan Turing’s undergraduate rediscovery of Lindeberg’s central limit theorem, and of his later use of Bayesian methods in breaking the German naval code in World War II. The last section deals with developments in inductive probability, which are still not generally well-known, and that some philosophers have thought impossible. The question is how a Bayesian theory can deal in a principled way with the possibility of new categories that have not been foreseen. On the face of it the problem appears to be intractable, but a deeper analysis shows that something sensible can be done. The development of the appropriate mathematics is a story that stretches from the beginnings of the subject to the end of the twentieth century. xi These essays have appeared, over almost twenty years, in a variety of disparate and sometimes obscure places. I remember eagerly waiting for the next installment. Each essay is like a specially cut gem, and it gives me great satisfaction that they can be brought together and presented in this volume. Brian Skyrms xii PART ONE Probability 1 Symmetry and Its Discontents The following paper consists of two parts. In the first it is argued that Bruno de Finetti’s theory of subjective probability provides a partial resolution of Hume’s problem of induction, if that problem is cast in a certain way. De Finetti’s solution depends in a crucial way, however, on a symmetry assumption – exchangeability – and in the second half of the paper the broader question of the use of symmetry arguments in probability is analyzed. The problems and difficulties that can arise are explicated through historical ex- amples which illustrate how symmetry arguments have played an important role in probability theory throughout its development. In a concluding section the proper role of such arguments is discussed. 1. the de finetti representation theorem Let X1, X2, X3,...be an infinite sequence of 0,1-valued random variables, which may be thought of as recording when an event occurs in a sequence of repeated trials (e.g., tossing a coin, with 1 if heads, 0 if tails). The sequence is said to be exchangeable if all finite sequences of the same length with the same number of ones have the same probability, i.e., if for all positive integers n and permutations σ of {1, 2, 3,...,n}, P[X1 = e1, X2 = e2,...,Xn = en] = P[X1 = eσ(1), X2 = eσ(2),...,Xn = eσ(n)], where ei denotes either a 0 or a 1. For example, when n = 3, this means that P[1, 0, 0] = P[0, 1, 0] = P[0, 0, 1] and P[1, 1, 0] = P[1, 0, 1] = P[0, 1, 1]. (Note, however, that P[1, 0, 0] is not assumed to equal P[1, 1, 0]; in general, these probabilities may be quite different.) Reprinted with permission from Brian Skyrms and William L. Harper (eds.), Causation, Chance, and Credence 1 (1988): 155–190, c 1988 by Kluwer Academic Publishers. 3 In 1931 the Italian probabilist Bruno de Finetti proved his famous de Finetti Representation Theorem. Let X1, X2, X3,... be an infinite ex- changeable sequence of 0,1-valued random variables, and let Sn = X1 + X2 +···+Xn denote the number of ones in a sequence of length n. Then it follows that: 1. the limiting frequency Z =: limn→∞(Sn/n) exists with probability 1. 2. if µ(A) =: P[Z ∈ A] is the probability distribution of Z, then 1 n k n−k P[Sn = k] = p (1 − p) dµ(p) 0 k for all n and k.1 This remarkable result has several important implications. First, contrary to popular belief, subjectivists clearly believe in the existence of infinite limiting relative frequencies – at least to the extent that they are willing to talk about an (admittedly hypothetical) infinite sequence of trials.2 The existence of such limiting frequencies follows as a purely mathematical consequence of the assumption of exchangeability.3 When an extreme subjectivist such as de Finetti denies the existence of objective chance or physical probability, what is really being disputed is whether limiting frequencies are objective or physical properties. There are several grounds for such a position, but all center around the question of what “object” an objective probability is a property of.