DOCSLIB.ORG

Bayesian Philosophy of Science

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Bayesian Philosophy of Science Jan Sprenger Stephan Hartmann Bayesian Philosophy of Science Variations on a Theme by the Reverend Thomas Bayes p(EjH) p(HjE) = p(H) p(E) Ich bin ein Esel, und will getreu, wie meine Väter, die Alten, an der alten, lieben Eselei, am Eseltume halten. Und weil ich ein Esel, so rat ich euch, den Esel zum König zu wählen; wir stiften das große Eselreich, wo nur die Esel befehlen. Wir sind alle Esel! I-A! I-A! Wir sind keine Pferdeknechte. Fort mit den Rossen! Es lebe—Hurra!— Der König vom Eselsgeschlechte! Heinrich Heine Contents Theme: Bayesian Philosophy of Science 1 Variation 1: Learning Conditional Information 31 1.1 The Kullback-Leibler Divergence and Probabilistic Updating 33 1.2 Three Challenges for Minimizing Divergence . 38 1.3 Meeting the Challenges . 40 1.4 Discussion . 48 1.5 Proofs of the Theorems . 51 Variation 2: Confirmation 57 2.1 Motivating Bayesian Confirmation Theory . 58 2.2 Confirmation as (Increase in) Firmness . 59 2.3 The Plurality of Bayesian Confirmation Measures . 69 2.4 Discussion . 73 Variation 3: The Problem of Old Evidence 77 3.1 The Dynamic POE: The GJN Approach . 79 3.2 Solving the Dynamic POE: Alternative Explanations . 83 3.3 The Static POE: A Counterfactual Perspective . 86 3.4 Solving the Hybrid POE: Learning Explanatory Relationships 88 3.5 Discussion . 92 3.6 Proofs of the Theorems . 96 Variation 4: The No Alternatives Argument 101 4.1 Modeling the No Alternatives Argument . 102 4.2 Results and Discussion . 107 4.3 Proof of the Theorems . 111 v vi Contents Variation 5: Scientific Realism and the No Miracles Argument 115 5.1 The Probabilistic No Miracles Argument . 116 5.2 Extending the No Miracles Argument to Stable Scientific Theories . 122 5.3 The Frequency-Based No Miracles Argument . 129 5.4 Discussion . 133 5.5 Proofs of the Theorems . 136 Variation 6: Causal Effect 141 6.1 The Framework: Causal Bayes Nets . 143 6.2 General Adequacy Conditions . 147 6.3 Causal Production and the Suppes-Pearl Measure . 150 6.4 The Multiplicativity Principle and the Difference Measure . 152 6.5 The No Dilution for Irrelevant Effects Principle and Proba- bility Ratio Measures . 154 6.6 Conjunctive Closure and the Logarithmic Ratio Measure . 157 6.7 Application: Quantifying Causal Effect in Medicine . 158 6.8 Discussion . 159 6.9 Proofs of the Theorems . 163 Variation 7: Explanatory Power 171 7.1 Toward a Statistical Relevance Account of Explanatory Power 173 7.2 Explicating Explanatory Power . 177 7.3 Discussion . 184 Variation 8: Intertheoretic Reduction 187 8.1 The Generalized Nagel-Schaffner Model . 188 8.2 Reduction and Confirmation . 191 8.3 Why Accept a Purported Reduction? . 197 8.4 Discussion . 199 8.5 Proofs of the Theorems . 201 Variation 9: Hypothesis Testing and Corroboration 205 9.1 Confirmation versus Corroboration . 209 9.2 Popper’s Measure of Degree of Corroboration . 212 9.3 The Impossibility Results . 216 9.4 Toward a New Explication of Corroboration . 223 9.5 Discussion . 227 Contents vii 9.6 Proofs of the Theorems . 230 Variation 10: Simplicity and Model Selection 237 10.1 Simplicity in Model Selection . 238 10.2 Curve Fitting and Estimation Error . 240 10.3 The Akaike Information Criterion (AIC) . 244 10.4 The Bayesian Information Criterion (BIC) . 247 10.5 Discussion . 251 10.6 Sketch of the Derivation of Akaike’s Information Criterion . 254 Variation 11: Scientific Objectivity 257 11.1 Forms of Scientific Objectivity . 258 11.2 Challenge 1: The Choice of the Prior Distribution . 260 11.3 Challenge 2: Belief vs. Evidence . 263 11.4 Challenge 3: Neglect of Experimental Design . 266 11.5 Discussion: A Digression on Scientific Objectivity . 270 The Theme Revisited 275 Bibliography 287 List of Figures 321 List of Tables 323 viii Theme: Bayesian Philosophy of Science Among the greatest achievements of science are the laws, models and the- ories it has came up with. Newton’s laws of mechanics, Bohr’s model of the atom and Einstein’s Theory of Relativity gave us unprecedented insights into the nature of physical reality. Similarly, Mendel’s laws of inheritance, Darwin’s theory of natural selection, and Crick and Watson’s innovations in molecular biology elucidated how species develop and how traits and properties are passed on from one generation to the next. Pi- oneers of rational choice theory like Von Neumann, Savage and Arrow explained behavior in terms of beliefs and preferences and came up with powerful axioms for rational decision-making under uncertainty. This enumeration does not intend to depreciate the value of experi- mental work in science. Rather, we would like to motivate why theories and models are central for understanding phenomena, predicting future events and transferring scientific knowledge to other domains. Therefore the assessment of scientific theories and models is a central part of sci- entific reasoning. This book investigates how Bayesian inference can contribute to this goal. While we do not want to claim that scientific reasoning is essentially Bayesian, we claim that Bayesian models can elucidate diverse aspects of scientific reasoning, increasing our understanding of how science works and why it is so successful. The book is written a cycle of variations on this theme; it applies Bayesian inference to eleven different aspects of scientific reasoning. In this introduction, we explain the constitutive principles and philo- sophical foundations of Bayesian inference, as well as some particular rea- 1 2 Contents soning techniques (e.g., Bayesian networks) that we need in the remainder of the book. Then we describe our methodological approach—Bayesian philosophy of science—in slightly more detail. The level is introductory; no knowledge of calculus or higher mathematics is required. The Static Dimension of Bayesian Inference: Probability and Degrees of Belief In science as well as in ordinary life, we routinely make a distinction be- tween more and less credible hypotheses. Consider, for example, the ques- tion which nation will win the 2016 European Football Cup. We may con- sider Albania a very unlikely candidate, England not completely implausi- ble, and we may find it likely that the winner will be either France or Ger- many. This example illustrates that the epistemic standing of a hypothesis is no all-or-nothing affair, but a matter of gradation. Here the Bayesians step in: they use the concept of degree of belief to describe epistemic at- titudes about uncertain propositions, and they represent these degrees of belief by a particular mathematical structure: probability functions. These two modeling assumptions are the central elements of Bayesian inference. In other words, Bayesians regard probabilities as expressions of subjective uncertainty—an interpretation that goes back to the English mathemati- cian, reverend and philosopher Thomas Bayes (1701–1761) (Bayes, 1763). The probability calculus has a long history as a tool for handling sub- jective uncertainty. In particular, it is one of the dominant paradigms in the psychology of human reasoning (e.g., Oaksford and Chater, 2000). Other scientific applications abound, such as Bayesian inference in phylogenet- ics, Bayesian interpretations of quantum mechanics, statistical inference and causal induction (e.g., Bernardo and Smith, 1994; Spirtes et al., 2000). Bayesian reasoning is also widely applied in philosophy: it is a standard tool in various branches of epistemology (e.g., Bovens and Hartmann, 2003; Pettigrew, 2015) and in the foundations of decision theory and ratio- nal choice (e.g., Jeffrey, 1971; Savage, 1972). The prominence of Bayesian inference in established scientific and philosophical theories recommends it as the default model for modeling uncertain reasoning. We do not claim that it is on foundational grounds superior to alternatives such as rank- ing functions (Spohn, 2012) or Dempster-Shafer theory (Shafer, 1976): we only claim that the past successes and the wide scope of Bayesian mod- Contents 3 els recommend them as an excellent tool for studying scientific reasoning. Practical considerations, such as the simplicity of the probability axioms and the existence of a well-developed mathematical theory underneath them, support the case for Bayesian philosophy of science, too. The distinctive feature of Bayesian inference is the central role of degree of belief. More traditional descriptions of epistemic attitudes, e.g., theories that just distinguish between belief, disbelief and suspension of judgment, struggle to adequately describe graded epistemic attitudes. This fails to account for many cases of scientific reasoning where we do hold graded beliefs. But psychological realism is not the only force that pulls into the direction of a graded theory of epistemic attitudes. An all-or-nothing account of epistemic attitudes also leads into philosophical trouble, such as in the famous lottery paradox (Kyburg, 1961). For instance, if just one out of 1.000.000 tickets in a lottery is winning, then for each single ticket #i (i 2 f1, 2, . , 1.000.000g), we seem to be entitled to believe that it is not the winning one. However, we also believe that there is a winning ticket in the lottery. Hence, the set of the propositions that we believe (“There is a winning ticket”, “ticket #1 does not win”, “ticket #2 does not win”, etc.) is inconsistent, which is at least prima facie an undesirable consequence. For theories of graded belief, no such inconsistency arises (for discussion of the full vs. graded belief relationship, see Leitgeb, 2014; Fitelson et al., 2016). If we want to use the concept of degree of belief to describe which scientific theories are more credible than others, we have to say something about the rules that rational degrees of belief have to satisfy. After all, the objects of degrees of belief—propositions—are related to each other in manifold ways, and these relations constrain the set of degrees of belief that we can rationally entertain.
Load more

Recommended publications
  • The Interpretation of Probability: Still an Open Issue? 1
    philosophies Article The Interpretation of Probability: Still an Open Issue? 1 Maria Carla Galavotti Department of Philosophy and Communication, University of Bologna, Via Zamboni 38, 40126 Bologna, Italy; [email protected] Received: 19 July 2017; Accepted: 19 August 2017; Published: 29 August 2017 Abstract: Probability as understood today, namely as a quantitative notion expressible by means of a function ranging in the interval between 0–1, took shape in the mid-17th century, and presents both a mathematical and a philosophical aspect. Of these two sides, the second is by far the most controversial, and fuels a heated debate, still ongoing. After a short historical sketch of the birth and developments of probability, its major interpretations are outlined, by referring to the work of their most prominent representatives. The final section addresses the question of whether any of such interpretations can presently be considered predominant, which is answered in the negative. Keywords: probability; classical theory; frequentism; logicism; subjectivism; propensity 1. A Long Story Made Short Probability, taken as a quantitative notion whose value ranges in the interval between 0 and 1, emerged around the middle of the 17th century thanks to the work of two leading French mathematicians: Blaise Pascal and Pierre Fermat. According to a well-known anecdote: “a problem about games of chance proposed to an austere Jansenist by a man of the world was the origin of the calculus of probabilities”2. The ‘man of the world’ was the French gentleman Chevalier de Méré, a conspicuous figure at the court of Louis XIV, who asked Pascal—the ‘austere Jansenist’—the solution to some questions regarding gambling, such as how many dice tosses are needed to have a fair chance to obtain a double-six, or how the players should divide the stakes if a game is interrupted.
    [Show full text]
  • The Open Handbook of Formal Epistemology
    THEOPENHANDBOOKOFFORMALEPISTEMOLOGY Richard Pettigrew &Jonathan Weisberg,Eds. THEOPENHANDBOOKOFFORMAL EPISTEMOLOGY Richard Pettigrew &Jonathan Weisberg,Eds. Published open access by PhilPapers, 2019 All entries copyright © their respective authors and licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. LISTOFCONTRIBUTORS R. A. Briggs Stanford University Michael Caie University of Toronto Kenny Easwaran Texas A&M University Konstantin Genin University of Toronto Franz Huber University of Toronto Jason Konek University of Bristol Hanti Lin University of California, Davis Anna Mahtani London School of Economics Johanna Thoma London School of Economics Michael G. Titelbaum University of Wisconsin, Madison Sylvia Wenmackers Katholieke Universiteit Leuven iii For our teachers Overall, and ultimately, mathematical methods are necessary for philosophical progress. — Hannes Leitgeb There is no mathematical substitute for philosophy. — Saul Kripke PREFACE In formal epistemology, we use mathematical methods to explore the questions of epistemology and rational choice. What can we know? What should we believe and how strongly? How should we act based on our beliefs and values? We begin by modelling phenomena like knowledge, belief, and desire using mathematical machinery, just as a biologist might model the fluc- tuations of a pair of competing populations, or a physicist might model the turbulence of a fluid passing through a small aperture. Then, we ex- plore, discover, and justify the laws governing those phenomena, using the precision that mathematical machinery affords. For example, we might represent a person by the strengths of their beliefs, and we might measure these using real numbers, which we call credences. Having done this, we might ask what the norms are that govern that person when we represent them in that way.
    [Show full text]
  • The Bayesian Approach to the Philosophy of Science
    The Bayesian Approach to the Philosophy of Science Michael Strevens For the Macmillan Encyclopedia of Philosophy, second edition The posthumous publication, in 1763, of Thomas Bayes’ “Essay Towards Solving a Problem in the Doctrine of Chances” inaugurated a revolution in the understanding of the confirmation of scientific hypotheses—two hun- dred years later. Such a long period of neglect, followed by such a sweeping revival, ensured that it was the inhabitants of the latter half of the twentieth century above all who determined what it was to take a “Bayesian approach” to scientific reasoning. Like most confirmation theorists, Bayesians alternate between a descrip- tive and a prescriptive tone in their teachings: they aim both to describe how scientific evidence is assessed, and to prescribe how it ought to be assessed. This double message will be made explicit at some points, but passed over quietly elsewhere. Subjective Probability The first of the three fundamental tenets of Bayesianism is that the scientist’s epistemic attitude to any scientifically significant proposition is, or ought to be, exhausted by the subjective probability the scientist assigns to the propo- sition. A subjective probability is a number between zero and one that re- 1 flects in some sense the scientist’s confidence that the proposition is true. (Subjective probabilities are sometimes called degrees of belief or credences.) A scientist’s subjective probability for a proposition is, then, more a psy- chological fact about the scientist than an observer-independent fact about the proposition. Very roughly, it is not a matter of how likely the truth of the proposition actually is, but about how likely the scientist thinks it to be.
    [Show full text]
  • University of Oklahoma Graduate College The
    UNIVERSITY OF OKLAHOMA GRADUATE COLLEGE THE VIRTUES OF BAYESIAN EPISTEMOLOGY A DISSERTATION SUBMITTED TO THE GRADUATE FACULTY in partial fulfillment of the requirements for the Degree of DOCTOR OF PHILOSOPHY By MARY FRANCES GWIN Norman, Oklahoma 2011 THE VIRTUES OF BAYESIAN EPISTEMOLOGY A DISSERTATION APPROVED FOR THE DEPARTMENT OF PHILOSOPHY BY ______________________________ Dr. James Hawthorne, Chair ______________________________ Dr. Chris Swoyer ______________________________ Dr. Wayne Riggs ______________________________ Dr. Steve Ellis ______________________________ Dr. Peter Barker © Copyright by MARY GWIN 2011 All Rights Reserved. For Ted How I wish you were here. Acknowledgements I would like to thank Dr. James Hawthorne for his help in seeing me through the process of writing my dissertation. Without his help and encouragement, I am not sure that I would have been able to complete my dissertation. Jim is an excellent logician, scholar, and dissertation director. He is also a kind person. I am very lucky to have had the opportunity to work with him. I cannot thank him enough for all that he has given me. I would also like to thank my friends Patty, Liz, Doren, Murry, Becky, Jim, Rebecca, Mike, Barb, Susanne, Blythe, Eric, Ty, Rosie, Bob, Judy, Lorraine, Paulo, Lawrence, Scott, Kyle, Pat, Carole, Joseph, Ken, Karen, Jerry, Ray, and Daniel. Without their encouragement, none of this would have been possible. iv Table of Contents Acknowledgments iv Abstract vi Chapter 1: Introduction 1 Chapter 2: Bayesian Confirmation Theory 9 Chapter 3: Rational Analysis: Bayesian Logic and the Human Agent 40 Chapter 4: Why Virtue Epistemology? 82 Chapter 5: Reliability, Credit, Virtue 122 Chapter 6: Epistemic Virtues, Prior Probabilities, and Norms for Human Performance 145 Bibliography 171 v Abstract The aim of this dissertation is to address the intersection of two normative epistemologies, Bayesian confirmation theory (BCT) and virtue epistemology (VE).
    [Show full text]
  • Most Honourable Remembrance. the Life and Work of Thomas Bayes
    MOST HONOURABLE REMEMBRANCE. THE LIFE AND WORK OF THOMAS BAYES Andrew I. Dale Springer-Verlag, New York, 2003 668 pages with 29 illustrations The author of this book is professor of the Department of Mathematical Statistics at the University of Natal in South Africa. Andrew I. Dale is a world known expert in History of Mathematics, and he is also the author of the book “A History of Inverse Probability: From Thomas Bayes to Karl Pearson” (Springer-Verlag, 2nd. Ed., 1999). The book is very erudite and reflects the wide experience and knowledge of the author not only concerning the history of science but the social history of Europe during XVIII century. The book is appropriate for statisticians and mathematicians, and also for students with interest in the history of science. Chapters 4 to 8 contain the texts of the main works of Thomas Bayes. Each of these chapters has an introduction, the corresponding tract and commentaries. The main works of Thomas Bayes are the following: Chapter 4: Divine Benevolence, or an attempt to prove that the principal end of the Divine Providence and Government is the happiness of his creatures. Being an answer to a pamphlet, entitled, “Divine Rectitude; or, An Inquiry concerning the Moral Perfections of the Deity”. With a refutation of the notions therein advanced concerning Beauty and Order, the reason of punishment, and the necessity of a state of trial antecedent to perfect Happiness. This is a work of a theological kind published in 1731. The approaches developed in it are not far away from the rationalist thinking.
    [Show full text]
  • Maty's Biography of Abraham De Moivre, Translated
    Statistical Science 2007, Vol. 22, No. 1, 109–136 DOI: 10.1214/088342306000000268 c Institute of Mathematical Statistics, 2007 Maty’s Biography of Abraham De Moivre, Translated, Annotated and Augmented David R. Bellhouse and Christian Genest Abstract. November 27, 2004, marked the 250th anniversary of the death of Abraham De Moivre, best known in statistical circles for his famous large-sample approximation to the binomial distribution, whose generalization is now referred to as the Central Limit Theorem. De Moivre was one of the great pioneers of classical probability the- ory. He also made seminal contributions in analytic geometry, complex analysis and the theory of annuities. The first biography of De Moivre, on which almost all subsequent ones have since relied, was written in French by Matthew Maty. It was published in 1755 in the Journal britannique. The authors provide here, for the first time, a complete translation into English of Maty’s biography of De Moivre. New mate- rial, much of it taken from modern sources, is given in footnotes, along with numerous annotations designed to provide additional clarity to Maty’s biography for contemporary readers. INTRODUCTION ´emigr´es that both of them are known to have fre- Matthew Maty (1718–1776) was born of Huguenot quented. In the weeks prior to De Moivre’s death, parentage in the city of Utrecht, in Holland. He stud- Maty began to interview him in order to write his ied medicine and philosophy at the University of biography. De Moivre died shortly after giving his Leiden before immigrating to England in 1740. Af- reminiscences up to the late 1680s and Maty com- ter a decade in London, he edited for six years the pleted the task using only his own knowledge of the Journal britannique, a French-language publication man and De Moivre’s published work.
    [Show full text]
  • Statistical Science Meets Philosophy of Science Part 2: Shallow Versus Deep Explorations
    RMM Vol. 3, 2012, 71–107 Special Topic: Statistical Science and Philosophy of Science Edited by Deborah G. Mayo, Aris Spanos and Kent W. Staley http://www.rmm-journal.de/ Deborah Mayo Statistical Science Meets Philosophy of Science Part 2: Shallow versus Deep Explorations Abstract: Inability to clearly defend against the criticisms of frequentist methods has turned many a frequentist away from venturing into foundational battlegrounds. Conceding the dis- torted perspectives drawn from overly literal and radical expositions of what Fisher, Ney- man, and Pearson ‘really thought’, some deny they matter to current practice. The goal of this paper is not merely to call attention to the howlers that pass as legitimate crit- icisms of frequentist error statistics, but also to sketch the main lines of an alternative statistical philosophy within which to better articulate the roles and value of frequentist tools. 1. Comedy Hour at the Bayesian Retreat Overheard at the comedy hour at the Bayesian retreat: Did you hear the one about the frequentist. “who defended the reliability of his radiation reading, despite using a broken radiometer, on the grounds that most of the time he uses one that works, so on average he’s pretty reliable?” or “who claimed that observing ‘heads’ on a biased coin that lands heads with probability .05 is evidence of a statistically significant improve- ment over the standard treatment of diabetes, on the grounds that such an event occurs with low probability (.05)?” Such jests may work for an after-dinner laugh, but if it turns out that, despite being retreads of ‘straw-men’ fallacies, they form the basis of why some statis- ticians and philosophers reject frequentist methods, then they are not such a laughing matter.
    [Show full text]
  • The P–T Probability Framework for Semantic Communication, Falsification, Confirmation, and Bayesian Reasoning
    philosophies Article The P–T Probability Framework for Semantic Communication, Falsification, Confirmation, and Bayesian Reasoning Chenguang Lu School of Computer Engineering and Applied Mathematics, Changsha University, Changsha 410003, China; [email protected] Received: 8 July 2020; Accepted: 16 September 2020; Published: 2 October 2020 Abstract: Many researchers want to unify probability and logic by defining logical probability or probabilistic logic reasonably. This paper tries to unify statistics and logic so that we can use both statistical probability and logical probability at the same time. For this purpose, this paper proposes the P–T probability framework, which is assembled with Shannon’s statistical probability framework for communication, Kolmogorov’s probability axioms for logical probability, and Zadeh’s membership functions used as truth functions. Two kinds of probabilities are connected by an extended Bayes’ theorem, with which we can convert a likelihood function and a truth function from one to another. Hence, we can train truth functions (in logic) by sampling distributions (in statistics). This probability framework was developed in the author’s long-term studies on semantic information, statistical learning, and color vision. This paper first proposes the P–T probability framework and explains different probabilities in it by its applications to semantic information theory. Then, this framework and the semantic information methods are applied to statistical learning, statistical mechanics, hypothesis evaluation (including falsification), confirmation, and Bayesian reasoning. Theoretical applications illustrate the reasonability and practicability of this framework. This framework is helpful for interpretable AI. To interpret neural networks, we need further study. Keywords: statistical probability; logical probability; semantic information; rate-distortion; Boltzmann distribution; falsification; verisimilitude; confirmation measure; Raven Paradox; Bayesian reasoning 1.
    [Show full text]
  • Paradoxes Situations That Seems to Defy Intuition
    Paradoxes Situations that seems to defy intuition PDF generated using the open source mwlib toolkit. See http://code.pediapress.com/ for more information. PDF generated at: Tue, 08 Jul 2014 07:26:17 UTC Contents Articles Introduction 1 Paradox 1 List of paradoxes 4 Paradoxical laughter 16 Decision theory 17 Abilene paradox 17 Chainstore paradox 19 Exchange paradox 22 Kavka's toxin puzzle 34 Necktie paradox 36 Economy 38 Allais paradox 38 Arrow's impossibility theorem 41 Bertrand paradox 52 Demographic-economic paradox 53 Dollar auction 56 Downs–Thomson paradox 57 Easterlin paradox 58 Ellsberg paradox 59 Green paradox 62 Icarus paradox 65 Jevons paradox 65 Leontief paradox 70 Lucas paradox 71 Metzler paradox 72 Paradox of thrift 73 Paradox of value 77 Productivity paradox 80 St. Petersburg paradox 85 Logic 92 All horses are the same color 92 Barbershop paradox 93 Carroll's paradox 96 Crocodile Dilemma 97 Drinker paradox 98 Infinite regress 101 Lottery paradox 102 Paradoxes of material implication 104 Raven paradox 107 Unexpected hanging paradox 119 What the Tortoise Said to Achilles 123 Mathematics 127 Accuracy paradox 127 Apportionment paradox 129 Banach–Tarski paradox 131 Berkson's paradox 139 Bertrand's box paradox 141 Bertrand paradox 146 Birthday problem 149 Borel–Kolmogorov paradox 163 Boy or Girl paradox 166 Burali-Forti paradox 172 Cantor's paradox 173 Coastline paradox 174 Cramer's paradox 178 Elevator paradox 179 False positive paradox 181 Gabriel's Horn 184 Galileo's paradox 187 Gambler's fallacy 188 Gödel's incompleteness theorems
    [Show full text]
  • A Philosophical Treatise of Universal Induction
    Entropy 2011, 13, 1076-1136; doi:10.3390/e13061076 OPEN ACCESS entropy ISSN 1099-4300 www.mdpi.com/journal/entropy Article A Philosophical Treatise of Universal Induction Samuel Rathmanner and Marcus Hutter ? Research School of Computer Science, Australian National University, Corner of North and Daley Road, Canberra ACT 0200, Australia ? Author to whom correspondence should be addressed; E-Mail: [email protected]. Received: 20 April 2011; in revised form: 24 May 2011 / Accepted: 27 May 2011 / Published: 3 June 2011 Abstract: Understanding inductive reasoning is a problem that has engaged mankind for thousands of years. This problem is relevant to a wide range of fields and is integral to the philosophy of science. It has been tackled by many great minds ranging from philosophers to scientists to mathematicians, and more recently computer scientists. In this article we argue the case for Solomonoff Induction, a formal inductive framework which combines algorithmic information theory with the Bayesian framework. Although it achieves excellent theoretical results and is based on solid philosophical foundations, the requisite technical knowledge necessary for understanding this framework has caused it to remain largely unknown and unappreciated in the wider scientific community. The main contribution of this article is to convey Solomonoff induction and its related concepts in a generally accessible form with the aim of bridging this current technical gap. In the process we examine the major historical contributions that have led to the formulation of Solomonoff Induction as well as criticisms of Solomonoff and induction in general. In particular we examine how Solomonoff induction addresses many issues that have plagued other inductive systems, such as the black ravens paradox and the confirmation problem, and compare this approach with other recent approaches.
    [Show full text]
  • Inference to the Best Explanation, Bayesianism, and Knowledge
    Inference to the Best Explanation, Bayesianism, and Knowledge Alexander Bird Abstract How do we reconcile the claim of Bayesianism to be a correct normative the- ory of scientific reasoning with the explanationists’ claim that Inference to the Best Explanation provides a correct description of our inferential practices? I first develop Peter Lipton’s approach to compatibilism and then argue that three challenges remain, focusing in particular on the idea that IBE can lead to knowl- edge. Answering those challenges requires renouncing standard Bayesianism’s commitment to personalism, while also going beyond objective Bayesianism regarding the constraints on good priors. The result is a non-standard, super- objective Bayesianism that identifies probabilities with evaluations of plausibil- ity in the light of the evidence, conforming to Williamson’s account of evidential probability. Keywords explanationism, IBE, Bayesianism, evidential probability, knowledge, plausibility, credences. 1 Introduction How do scientists reason? How ought they reason? These two questions, the first descriptive and the second normative, are very different. However, if one thinks, as a scientific realist might, that scientists on the whole reason as they ought to then one will expect the two questions to have very similar answers. That fact bears on debates between Bayesians and explanationists. When we evaluate a scientific hypothesis, we may find that the evidence supports the hypoth- esis to some degree, but not to such a degree that we would be willing to assert out-
    [Show full text]
  • The Theory That Would Not Die Reviewed by Andrew I
    Book Review The Theory That Would Not Die Reviewed by Andrew I. Dale The solution, given more geometrico as Proposi- The Theory That Would Not Die: How Bayes’ tion 10 in [2], can be written today in a somewhat Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant compressed form as from Two Centuries of Controversy posterior probability ∝ likelihood × prior probability. Sharon Bertsch McGrayne Yale University Press, April 2011 Price himself added an appendix in which he used US$27.50, 336 pages the proposition in a prospective sense to find the ISBN-13: 978-03001-696-90 probability of the sun’s rising tomorrow given that it has arisen daily a million times. Later use In the early 1730sThomas Bayes (1701?–1761)was was made by Laplace, to whom one perhaps really appointed minister at the Presbyterian Meeting owes modern Bayesian methods. House on Mount Sion, Tunbridge Wells, a town that The degree to which one uses, or even supports, had developed around the restorative chalybeate Bayes’s Theorem (in some form or other) depends spring discovered there by Dudley, Lord North, in to a large extent on one’s views on the nature 1606. Apparently not one who was a particularly of probability. Setting this point aside, one finds popular preacher, Bayes would be recalled today, that the Theorem is generally used to update if at all, merely as one of the minor clergy of (to justify the updating of?) information in the eighteenth-century England, who also dabbled in light of new evidence as the latter is received, mathematics.
    [Show full text]