Bayes Or Bust? a Critical Examination of Bayesian Confirmation Theory
Total Page:16
File Type:pdf, Size:1020Kb
Bayesor Bust? A Critical Examination of Bayesian Confirmation Theory John Earman A Bradford Book The MIT Press Cambridge, Massachusetts London, England Second printing. 1996 © 1992 Massachusetts Institute of Technology For Peter Hempel, teacher extraordinaire, All rights reserved. No part of this book may be reproduced in any form by any electronic And Grover Maxwell, a gentleman who left his mark on an or mechanical means (including photocopying, recording, or information storage and ungentlemanly world retrieval) without permission in writing from the publisher. This book was set in Times Roman by Asco Trade Typesetting Ltd., Hong Kong, and was printed and bound in the United States of America. Library of Congress Cataloging-in-Publication Data Earman, John. Bayes or bust? : a critical examination of Bayesian confirmation theory I John Earman. p. em. "A Bradford book." Includes bibliographical references and index. ISBN 0-262-05046-3 1.Science-Philosophy. 2. Science-Methodology. 3. Bayesian statistical decision theory. I. Title. Q175.E23 1992 502.8-dc20 91-35623 CIP Contents Preface xi Notation xiii Introduction 1 1 Bayes's Bayesianism 7 1 Bayes's Problem 7 2 Bayes's Two Concepts of Probability 8 3 Bayes's Attempt to Demonstrate the Principles of Probability 9 4 Bayes's Princip1e-of-Insufficient-Reason Argument 14 5 Choosing a Prior Distribution 16 6 Proxy Events and the Billiard-Table Model 20 7 Applications of Bayes's Results 24 8 Conclusion 26 Appendix: Bayes's Definitions and Propositions 27 2 TheMachinery of Modern Bayesianism 33 1 The Elements of Modern Bayesianism 33 2 The Probability Axioms 35 3 Dutch Book and the Axioms of Probability 38 4 Difficulties with the Dutch Book Argument 40 5 Non-Dutch Book Justifications of the Probability Axioms 44 6 Justifications for Conditionalization 46 7 Lewis's Principal Principle 51 8 Descriptive versus Normative Interpretations of Bayesianism 56 9 Prior Probabilities 57 10 Conclusion 59 Appendix 1: Conditional Probability 59 Appendix 2: Laws of Large Numbers 61 3 Success Stories 63 1 Qualitative Confirmation: The Hypotheticodeductive Method 63 2 Hempel's Instance Confirmation 65 Contents ix viii Contents 3 The Washing Out of Priors: Some Bayesian Folklore 141 3 The Ravens Paradox 69 4 Convergence to Certainty and Merger of Opinion as a 4 Bootstrapping and Relevance Relations 73 Consequence of Martingale Convergence 144 5 Variety of Evidence and the Limited Variety of Nature 77 5 The Results of Gaifman and Snir 145 6 Putnam and Hempel on theIndispensability of 6 Evaluation ofthe Convergence-to-Certainty and Theories 79 Merger-of-Opinion Results 147 7 The Quine and Duhem Problem 83 7 Underdetermination and Antirealism 149 8 Conclusion 86 8 Confirmability and Cognitive Meaningfulness 153 9 Alternative Explanations of Objectivity 154 4 Challenges Met 87 10 The Evolutionary Solution 155 1 The Problem of Zero Priors: Carnap's Version 87 11 Modest but Realistic Solutions 156 2 The Problem ofZero Priors: Jeffrey's Version 90 12 Non-Bayesian Solutions 158 3 The Problem of Zero Priors: Popper's Versions 92 13 Retrenchment 158 4 The Popper and Miller Challenge 95 14 Conclusion 160 5 Richard Miller and the Return to Adhocness 98 6 Griinbaum's Worries 102 7 A Plea for Eliminative Induction 163 7 Goodman's New Problem of Induction 104 1 Teaching Dr. Watson to Do Induction 163 8 Novelty of Prediction and Severity ofTest 113 2 The Necessity ofthe Eliminative Element in Induction 163 9 Conclusion 117 3 Salmon's Retreat from Bayesian Inductivism 171 4 Twentieth-Century Gravitational Theories: A Case The Problem of Old Evidence 119 S Study 173 1 Old Evidence as a Challenge to Bayesian Confirmation 5 Conclusion 180 Theory 119 2 Preliminary Attempts to Solve or Dissolve the Old- 8 Normal Science, Scientific Revolntions, and All That: Evidence Problem 120 Thomas Bayes versus Thomas Kuhn 187 3 Garber's Approach 123 1 Kuhn's Structure of Scientific Revolutions 187 4 Jeffrey's Demonstration 126 2 Theory-Ladenness of Observation and the Incommen- 5 The Inadequacy ofthe Garber, Jeffrey, and Niiniluoto surability ofTheories 189 Solution 130 3 Tom Bayes and Tom Kuhn: Incommensurability? 191 6 New Theories and Doubly Counting Evidence 132 4 Revolutions and New Theories 195 7 Conclusion: A Pessimistic Resolution of the Old- 5 Objectivity, Rationality, and the Problem of Consensus 198 133 Evidence Problem 6 A Partial Resolution of the Problem of Consensus 201 7 Conclusion 203 6 The Rationality and Objectivity of Scientific Inference 137 1 Introduction 137 2 Constraining Priors 139 x Contents Preface 9 Bayesianism versns Formal-Learning Theory 207 Philosophers of science can be justifiably proud of the progress achieved 1 Putnam's Diagonalization Argument 207 in their discipline since the early days of logical positivism. A glaring 2 Taking Stock 209 exception concerns the analysis of the testing of scientific hypotheses and 3 Formal-Learning Theory 210 theories, an exception that threatens to block further progress toward a 4 Bayesian Learning Theory versus Formal-Learning central goal not only of the logical positivists but also of their predecessors Theory 211 and heirs. That goal is the understanding of "the scientific method." What 5 Does Formal Learning Have an Edge over Bayesian ever else this method involves, its principal concern is with the issue of how Learning? 212 the results of observation and experiment serve to support or undermine scientific conjectures. Not only does contemporary philosophy of science 6 The Dogmatism of Bayesianism 215 fail to provide a persuasive analysis of this issue, there are even rumblings 7 In What Sense Does a Formal Learner Learn? 218 to the effect that a search for an "inductive logic" or "theory of confirma 8 Putnam's Argument Revisited 220 tion" is a fruitless quest for a nonexistent philosopher's stone. The princi 9 Conclusion 222 pal difficulty here, it should be emphasized, does not derive from Kuhnian incommensurability and its fellow travelers, for the issue has resisted reso 10 A Dialogue 225 lution even for cases that do not stray near the frontiers of scientific revolutions, Notes 237 This work explores one dimension of this impasse by providing a critical References 253 evaluation of the approach I take to provide the best good hope for a Index 265 comprehensive and unified treatment of induction, confirmation, and sci entific inference: Bayesianism. It is intended for students of the philosophy of scientific methodology, where 'student' is used in the broad sense to include advanced undergraduates, graduate students, and professional phi losophers. It is also intended to annoy both the pro- and contra-Bayesians, To fling down the gauntlet, the critics of Bayesianism have generally failed to get the proper measure of the doctrine, while the Bayesians themselves have failed to appreciate the pitfalls and limitations of their approach. And to add insult to insult, neither side has appreciated the source of the doctrine-the Reverend Bayes's essay. Nor is there much appreciation of what the new discipline of formal learning theory has to tell us about the latent assumptions responsible for the apparent reliability of Bayesian methods. Ifthe annoyance serves as a spur to further progress, I will count this book a success. I want to emphasize as strongly as I can that this work is not intended as a comprehensive review of the pros and cons of Bayesianism. Issues and points of views on issues have been selected with an eye to giving the reader a sense of the strengths and weaknesses of the Bayesian approach to confirmation, and my selections should be judged on that basis. xii Preface Notation The reader who penetrates very far into this work may begin to experi Logic ence a topsy-turvy feeling. Those who complain will in: effect be crying, Stop the world, I want to get off! For the world of confirmation is a (Vx) universal quantifier topsy-turvy world. (3x) existential quantifier My intellectual debts are too numerous to recount here. But I would be & conjunction, A & B (A and B) remiss ifI did not acknowledge the many helpful suggestions I received on earlier drafts of this book. Thank you Jeremy Butterfield, Charles Chihara, A l &A2 & ... &A. Alan Franklin, Donald Gillies, Clark Glymour, David Hillman, Colin disjunction, A v B (A or B or both) Howson, Richard Jeffrey, Cory Juhl, Kevin Kelly, Philip Kitcher, Tim Al v A z v ... v An Maudlin, John Norton, Teddy Seiden:feld, Elliott Sober, Paul Teller, Jan material implication, A -->B (if A then B) von Plato, Bas van Fraassen, and Sandy Zabell. With all of this help, the biconditional, A ....B (A if and only ifB) book should be better than it is. The material in chapter 1 appeared as "Bayes' Bayesianism," Studies in negation, I A (not A) the History and Philosophy of Science 21 (1990): 351-370. A version of definition chapter 5 appeared as "Old Evidence, New Theories: Two Unsolved Prob I=A A is valid (i.e., A is true in every possible world or model) lems in Bayesian Con:firmation Theory," Pacific Philosophical Quarterly70 AI=B A semantically implies B (i.e., B is true in every possible (1989): 323-340. I am grateful to the editors of these journals for their world or model in which A is true) permission to reprint this material. The excerpts from Bayes's essay found in the appendix to chapter 1 are reproduced with the kind-permission of the editors of Biometrika. Mathematics XEX x is an element of X XsY X is a subset of Y XnY the intersection of X and Y XuY the union of X and Y L' summation over the index i J integral N natural numbers D1I real line f: X --> Y fmaps x E X tof(x) E Y supremum the limit as n approaches in:finity xiv Notation Bayes or Bust? Probability andStatistics Pr(A) the (unconditional) prohability of A Pr(A/B) the conditional probability of A on B E(X) the expectation value of the random variable X (:) n!/[m!(n - m)!] Introduction The Reverend Thomas Bayes died in 1761, leaving a not inconsiderable estate.