DOCSLIB.ORG

Causation, Prediction, and Search

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Causation, Prediction, and Search Causation, Prediction, and Search Peter Spirtes, Clark Glymour, and Richard Scheines Department of Philosophy Carnegie Mellon University Copyright © by Peter Spirtes, Clark Glymour, and Richard Scheines All rights reserved. No portion of this book may be reproduced for commercial use, by any process or tchnique, without the express consent of the authors. To my parents, Morris and Cecile Spirtes - P.S. In memory of Lucille Lynch Schwartz Watkins Speede Tindall Preston - C. G. To Martha, for her support and love - R.S. It is with data affected by numerous causes that Statistics is mainly concerned. Experiment seeks to disentangle a complex of causes by removing all but one of them, or rather by concentrating on the study of one and reducing the others, as far as circumstances permit, to comparatively small residium. Statistics, denied this resource, must accept for analysis data subject to the influence of a host of causes, and must try to discover from the data themselves which causes are the important ones and how much of the observed effect is due to the operation of each. --G. U. Yule and M. G. Kendall 1950 The Theory of Estimation discusses the principles upon which observational data may be used to estimate, or to throw light upon the values of theoretical quantities, not known numerically, which enter into our specification of the causal system operating. -- Sir Ronald Fisher, 1956 George Box has [almost] said "The only way to find out what will happen when a complex system is disturbed is to disturb the system, not merely to observe it passively." These words of caution about "natural experiments" are uncomfortably strong. Yet in today's world we see no alternative to accepting them as, if anything, too weak. --G. Mosteller and J. Tukey, 1977 Causal inference is one of the most important, most subtle, and most neglected of all the problems of Statistics. -- P. Dawid, 1979 Table of Contents Preface ................................................................................................................................vii Acknowledgments ..............................................................................................................xiii Notational Conventions .....................................................................................................xxi 1. Introduction and Advertisement ..................................................................................1 1.1 The Issue............................................................................................................1 1.2 Advertisements ..................................................................................................10 1.2.1 Bayes Networks from the Data...........................................................11 1.2.2 Structural Equation Models from the Data.........................................13 1.2.3 Selection of Regressors.......................................................................14 1.2.4 Causal Inference without Experiment ................................................17 1.2.5 The Structure of the Unobserved........................................................19 1.3 Themes...............................................................................................................21 2. Formal Preliminaries.....................................................................................................25 2.1 Graphs................................................................................................................25 2.2 Probability..........................................................................................................31 2.3 Graphs and Probability Distributions ................................................................32 2.3.1 Directed Acyclic Graphs.....................................................................32 2.3.2 Directed Independence Graphs...........................................................34 2.3.3 Faithfulness.........................................................................................35 2.3.4 d-separation.........................................................................................36 2.3.5 Linear Structures.................................................................................36 2.4 Undirected Independence Graphs......................................................................37 2.5 Deterministic and Pseudo-Indeterministic Systems ..........................................38 2.6 Background Notes .............................................................................................39 3. Causation and Prediction: Axioms and Explications .................................................41 3.1 Conditionals.......................................................................................................41 3.2 Causation ...........................................................................................................42 3.2.1 Direct vs. Indirect Causation ..............................................................42 3.2.2 Events and Variables ..........................................................................43 3.2.3 Examples.............................................................................................45 3.2.4 Representing Causal Relations with Directed Graphs........................47 3.3 Causality and Probability...................................................................................49 3.3.1 Deterministic Causal Structures .........................................................49 3.3.2 Pseudo-Indeterministic and Indeterministic Causal Structures ..........51 3.4 The Axioms .......................................................................................................53 3.4.1 The Causal Markov Condition............................................................53 3.4.2 The Causal Minimality Condition ......................................................55 3.4.3 The Faithfulness Condition.................................................................56 3.5 Discussion of the Conditions .............................................................................57 3.5.1 The Causal Markov and Minimality Conditions ................................57 3.5.2 Faithfulness and Simpson's Paradox...................................................64 3.6 Bayesian Interpretations ....................................................................................70 3.7 Consequences of The Axioms ...........................................................................71 3.7.1 d-Separation........................................................................................71 3.7.2 The Manipulation Theorem ................................................................75 3.8 Determinism ......................................................................................................81 3.9 Background Notes .............................................................................................86 4. Statistical Indistinguishability ......................................................................................87 4.1 Strong Statistical Indistinguishability................................................................88 4.2 Faithful Indistinguishability...............................................................................89 4.3 Weak Statistical Indistinguishability .................................................................90 4.4 Rigid Indistinguishability ..................................................................................93 4.5 The Linear Case.................................................................................................94 4.6 Redefining Variables .........................................................................................99 4.7 Background Notes .............................................................................................101 5. Discovery Algorithms for Causally Sufficient Structures..........................................103 5.1 Discovery Problems...........................................................................................103 5.2 Search Strategies in Statistics ............................................................................104 5.2.1 The Wrong Hypothesis Space ............................................................105 5.2.2 Computational and Statistical Limitations..........................................107 5.2.3 Generating a Single Hypothesis..........................................................108 5.2.4 Other Approaches ...............................................................................109 5.2.5 Bayesian Methods...............................................................................109 5.3 The Wermuth-Lauritzen Algorithm...................................................................111 5.4 New Algorithms.................................................................................................112 5.4.1 The SGS Algorithm ............................................................................114 5.4.2 The PC Algorithm...............................................................................116 5.4.3 The IG (Independence Graph) Algorithm ..........................................124 5.4.4 Variable Selection...............................................................................125 5.4.5 Incorporating Background Knowledge...............................................127 5.5 Statistical Decisions...........................................................................................128
Load more

Recommended publications
  • Rothamsted in the Making of Sir Ronald Fisher Scd FRS
    Rothamsted in the Making of Sir Ronald Fisher ScD FRS John Aldrich University of Southampton RSS September 2019 1 Finish 1962 “the most famous statistician and mathematical biologist in the world” dies Start 1919 29 year-old Cambridge BA in maths with no prospects takes temporary job at Rothamsted Experimental Station In between 1919-33 at Rothamsted he makes a career for himself by creating a career 2 Rothamsted helped make Fisher by establishing and elevating the office of agricultural statistician—a position in which Fisher was unsurpassed by letting him do other things—including mathematical statistics, genetics and eugenics 3 Before Fisher … (9 slides) The problems were already there viz. o the relationship between harvest and weather o design of experimental trials and the analysis of their results Leading figures in agricultural science believed the problems could be treated statistically 4 Established in 1843 by Lawes and Gilbert to investigate the effective- ness of fertilisers Sir John Bennet Lawes Sir Joseph Henry Gilbert (1814-1900) land-owner, fertiliser (1817-1901) professional magnate and amateur scientist scientist (chemist) In 1902 tired Rothamsted gets make-over when Daniel Hall becomes Director— public money feeds growth 5 Crops and weather and experimental trials Crops & the weather o examined by agriculturalists—e.g. Lawes & Gilbert 1880 o subsequently treated more by meteorologists Experimental trials a hotter topic o treated in Journal of Agricultural Science o by leading figures Wood of Cambridge and Hall
    [Show full text]
  • Cos513 Lecture 2 Conditional Independence and Factorization
    COS513 LECTURE 2 CONDITIONAL INDEPENDENCE AND FACTORIZATION HAIPENG ZHENG, JIEQI YU Let {X1,X2, ··· ,Xn} be a set of random variables. Now we have a series of questions about them: • What are the conditional probabilities (answered by normalization / marginalization)? • What are the independencies (answered by factorization of the joint distribution)? For now, we assume that all random variables are discrete. Then the joint distribution is a table: (0.1) p(x1, x2, ··· , xn). Therefore, Graphic Model (GM) is a economic representation of joint dis- tribution taking advantage of the local relationships between the random variables. 1. DIRECTED GRAPHICAL MODELS (DGM) A directed GM is a Directed Acyclic Graph (DAG) G(V, E), where •V, the set of vertices, represents the random variables; •E, the edges, denotes the “parent of” relationships. We also define (1.1) Πi = set of parents of Xi. For example, in the graph shown in Figure 1.1, The random variables are X4 X4 X2 X2 X 6 X X1 6 X1 X3 X5 X3 X5 XZY XZY FIGURE 1.1. Example of Graphical Model 1 Y X Y Y X Y XZZ XZZ Y XYY XY XZZ XZZ XZY XZY 2 HAIPENG ZHENG, JIEQI YU {X1,X2 ··· ,X6}, and (1.2) Π6 = {X2,X3}. This DAG represents the following joint distribution: (1.3) p(x1:6) = p(x1)p(x2|x1)p(x3|x1)p(x4|x2)p(x5|x3)p(x6|x2, x5). In general, n Y (1.4) p(x1:n) = p(xi|xπi ) i=1 specifies a particular joint distribution. Note here πi stands for the set of indices of the parents of i.
    [Show full text]
  • Data-Mining Probabilists Or Experimental Determinists?
    Gopnik-CH_13.qxd 10/10/2006 1:39 PM Page 208 13 Data-Mining Probabilists or Experimental Determinists? A Dialogue on the Principles Underlying Causal Learning in Children Thomas Richardson, Laura Schulz, & Alison Gopnik This dialogue is a distillation of a real series of conver- Meno: Leaving aside the question of how children sations that took place at that most platonic of acade- learn for a while, can we agree on some basic prin- mies, the Center for Advanced Studies in the ciples for causal inference by anyone—child, Behavioral Sciences, between the first and third adult, or scientist? \edq1\ authors.\edq1\ The second author had determinist leanings to begin with and acted as an intermediary. Laplace: I think so. I think we can both generally agree that, subject to various caveats, two variables will not be dependent in probability unless there is Determinism, Faithfulness, and Causal some kind of causal connection present. Inference Meno: Yes—that’s what I’d call the weak causal Narrator: [Meno and Laplace stand in the corridor Markov condition. I assume that the kinds of of a daycare, observing toddlers at play through a caveats you have in mind are to restrict this princi- window.] ple to situations in which a discussion of cause and effect might be reasonable in the first place? Meno: Do you ever wonder how it is that children manage to learn so many causal relations so suc- Laplace: Absolutely. I don’t want to get involved in cessfully, so quickly? They make it all seem effort- discussions of whether X causes 2X or whether the less.
    [Show full text]
  • Bayesian Networks
    Bayesian Networks Read R&N Ch. 14.1-14.2 Next lecture: Read R&N 18.1-18.4 You will be expected to know • Basic concepts and vocabulary of Bayesian networks. – Nodes represent random variables. – Directed arcs represent (informally) direct influences. – Conditional probability tables, P( Xi | Parents(Xi) ). • Given a Bayesian network: – Write down the full joint distribution it represents. • Given a full joint distribution in factored form: – Draw the Bayesian network that represents it. • Given a variable ordering and some background assertions of conditional independence among the variables: – Write down the factored form of the full joint distribution, as simplified by the conditional independence assertions. Computing with Probabilities: Law of Total Probability Law of Total Probability (aka “summing out” or marginalization) P(a) = Σb P(a, b) = Σb P(a | b) P(b) where B is any random variable Why is this useful? given a joint distribution (e.g., P(a,b,c,d)) we can obtain any “marginal” probability (e.g., P(b)) by summing out the other variables, e.g., P(b) = Σa Σc Σd P(a, b, c, d) Less obvious: we can also compute any conditional probability of interest given a joint distribution, e.g., P(c | b) = Σa Σd P(a, c, d | b) = (1 / P(b)) Σa Σd P(a, c, d, b) where (1 / P(b)) is just a normalization constant Thus, the joint distribution contains the information we need to compute any probability of interest. Computing with Probabilities: The Chain Rule or Factoring We can always write P(a, b, c, … z) = P(a | b, c, ….
    [Show full text]
  • (Ω, F,P) Be a Probability Spac
    Lecture 9: Independence, conditional independence, conditional distribution Definition 1.7. Let (Ω, F,P ) be a probability space. (i) Let C be a collection of subsets in F. Events in C are said to be independent if and only if for any positive integer n and distinct events A1,...,An in C, P (A1 ∩ A2 ∩···∩ An)= P (A1)P (A2) ··· P (An). (ii) Collections Ci ⊂ F, i ∈ I (an index set that can be uncountable), are said to be independent if and only if events in any collection of the form {Ai ∈ Ci : i ∈ I} are independent. (iii) Random elements Xi, i ∈ I, are said to be independent if and only if σ(Xi), i ∈ I, are independent. A useful result for checking the independence of σ-fields. Lemma 1.3. Let Ci, i ∈ I, be independent collections of events. Suppose that each Ci has the property that if A ∈ Ci and B ∈ Ci, then A∩B ∈ Ci. Then σ(Ci), i ∈ I, are independent. Random variables Xi, i =1, ..., k, are independent according to Definition 1.7 if and only if k F(X1,...,Xk)(x1, ..., xk)= FX1 (x1) ··· FXk (xk), (x1, ..., xk) ∈ R Take Ci = {(a, b] : a ∈ R, b ∈ R}, i =1, ..., k If X and Y are independent random vectors, then so are g(X) and h(Y ) for Borel functions g and h. Two events A and B are independent if and only if P (B|A) = P (B), which means that A provides no information about the probability of the occurrence of B. Proposition 1.11.
    [Show full text]
  • A Bunched Logic for Conditional Independence
    A Bunched Logic for Conditional Independence Jialu Bao Simon Docherty University of Wisconsin–Madison University College London Justin Hsu Alexandra Silva University of Wisconsin–Madison University College London Abstract—Independence and conditional independence are fun- For instance, criteria ensuring that algorithms do not discrim- damental concepts for reasoning about groups of random vari- inate based on sensitive characteristics (e.g., gender or race) ables in probabilistic programs. Verification methods for inde- can be formulated using conditional independence [8]. pendence are still nascent, and existing methods cannot handle conditional independence. We extend the logic of bunched impli- Aiming to prove independence in probabilistic programs, cations (BI) with a non-commutative conjunction and provide a Barthe et al. [9] recently introduced Probabilistic Separation model based on Markov kernels; conditional independence can be Logic (PSL) and applied it to formalize security for several directly captured as a logical formula in this model. Noting that well-known constructions from cryptography. The key ingre- Markov kernels are Kleisli arrows for the distribution monad, dient of PSL is a new model of the logic of bunched implica- we then introduce a second model based on the powerset monad and show how it can capture join dependency, a non-probabilistic tions (BI), in which separation is interpreted as probabilistic analogue of conditional independence from database theory. independence. While PSL enables formal reasoning about Finally, we develop a program logic for verifying conditional independence, it does not support conditional independence. independence in probabilistic programs. The core issue is that the model of BI underlying PSL provides no means to describe the distribution of one set of variables I.
    [Show full text]
  • Week 2: Causal Inference
    Week 2: Causal Inference Marcelo Coca Perraillon University of Colorado Anschutz Medical Campus Health Services Research Methods I HSMP 7607 2020 These slides are part of a forthcoming book to be published by Cambridge University Press. For more information, go to perraillon.com/PLH. This material is copyrighted. Please see the entire copyright notice on the book's website. 1 Outline Correlation and causation Potential outcomes and counterfactuals Defining causal effects The fundamental problem of causal inference Solving the fundamental problem of causal inference: a) Randomization b) Statistical adjustment c) Other methods The ignorable treatment assignment assumption Stable Unit Treatment Value Assumption (SUTVA) Assignment mechanism 2 Big picture We are going to review the basic framework for understanding causal inference This is a fairly new area of research, although some of the statistical methods have been used for over a century The \new" part is the development of a mathematical notation and framework to understand and define causal effects This new framework has many advantages over the traditional way of understanding causality. For me, the biggest advantage is that we can talk about causal effects without having to use a specific statistical model (design versus estimation) In econometrics, the usual way of understanding causal inference is linked to the linear model (OLS, \general" linear model) { the zero conditional mean assumption in Wooldridge: whether the additive error term in the linear model, i , is correlated with the
    [Show full text]
  • Basic Statistics
    Statistics Kristof Reid Assistant Professor Medical University of South Carolina Core Curriculum V5 Financial Disclosures • None Core Curriculum V5 Further Disclosures • I am not a statistician • I do like making clinical decisions based on appropriately interpreted data Core Curriculum V5 Learning Objectives • Understand why knowing statistics is important • Understand the basic principles and statistical test • Understand common statistical errors in the medical literature Core Curriculum V5 Why should I care? Indications for statistics Core Curriculum V5 47% unable to determine study design 31% did not understand p values 38% did not understand sensitivity and specificity 83% could not use odds ratios Araoye et al, JBJS 2020;102(5):e19 Core Curriculum V5 50% (or more!) of clinical research publications have at least one statistical error Thiese et al, Journal of thoracic disease, 2016-08, Vol.8 (8), p.E726-E730 Core Curriculum V5 17% of conclusions not justified by results 39% of studies used the wrong analysis Parsons et al, J Bone Joint Surg Br. 2011;93-B(9):1154-1159. Core Curriculum V5 Are these two columns different? Column A Column B You have been asked by your insurance carrier 19 10 12 11 to prove that your total hip patient outcomes 10 9 20 19 are not statistically different than your competitor 10 17 14 10 next door. 21 19 24 13 Column A is the patient reported score for you 12 18 and column B for your competitor 17 8 12 17 17 10 9 9 How would you prove you are 24 10 21 9 18 18 different or better? 13 4 18 8 12 14 15 15 14 17 Core Curriculum V5 Are they still the same? Column A: Mean=15 Column B: Mean=12 SD=4 Core Curriculum V5 We want to know if we're making a difference Core Curriculum V5 What is statistics? a.
    [Show full text]
  • Principles of the Design and Analysis of Experiments
    A FRESH LOOK AT THE BASIC PRINCIPLES OF THE DESIGN AND ANALYSIS OF EXPERIMENTS F. YATES ROTHAMSTED EXPERIMENTAL STATION, HARPENDEN 1. Introduction When Professor Neyman invited me to attend the Fifth Berkeley Symposium, and give a paper on the basic principles of the design and analysis of experiments, I was a little hesitant. I felt certain that all those here must be thoroughly conversant with these basic principles, and that to mull over them again would be of little interest. This, however, is the first symposium to be held since Sir Ronald Fisher's death, and it does therefore seem apposite that a paper discussing some aspect of his work should be given. If so, what could be better than the design and analysis of experiments, which in its modern form he created? I do not propose today to give a history of the development of the subject. This I did in a paper presented in 1963 to the Seventh International Biometrics Congress [14]. Instead I want to take a fresh look at the logical principles Fisher laid down, and the action that flows from them; also briefly to consider certain modern trends, and see how far they are really of value. 2. General principles Fisher, in his first formal exposition of experimental design [4] laid down three basic principles: replication; randomization; local control. Replication and local control (for example, arrangement in blocks or rows and columns of a square) were not new, but the idea of assigning the treatments at random (subject to the restrictions imposed by the local control) was novel, and proved to be a most fruitful contribution.
    [Show full text]
  • Conditional Independence and Its Representations*
    Kybernetika, Vol. 25:2, 33-44, 1989. TECHNICAL REPORT R-114-S CONDITIONAL INDEPENDENCE AND ITS REPRESENTATIONS* JUDEA PEARL, DAN GEIGER, THOMAS VERMA This paper summarizes recent investigations into the nature of informational dependencies and their representations. Axiomatic and graphical representations are presented which are both sound and complete for specialized types of independence statements. 1. INTRODUCTION A central requirement for managing reasoning systems is to articulate the condi­ tions under which one item of information is considered relevant to another, given what we already know, and to encode knowledge in structures that display these conditions vividly as the knowledge undergoes changes. Different formalisms give rise to different definitions of relevance. However, the essence of relevance can be captured by a structure common to all formalisms, which can be represented axio- matically or graphically. A powerful formalism for informational relevance is provided by probability theory, where the notion of relevance is identified with dependence or, more specifically, conditional independence. Definition. If X, Y, and Z are three disjoint subsets of variables in a distribution P, then X and Yare said to be conditionally independent given Z, denotedl(X, Z, Y)P, iff P(x, y j z) = P(x | z) P(y \ z) for all possible assignments X = x, Y = y and Z = z for which P(Z — z) > 0.1(X, Z, Y)P is called a (conditional independence) statement. A conditional independence statement a logically follows from a set E of such statements if a holds in every distribution that obeys I. In such case we also say that or is a valid consequence of I.
    [Show full text]
  • Causality, Conditional Independence, and Graphical Separation in Settable Systems
    ARTICLE Communicated by Terrence Sejnowski Causality, Conditional Independence, and Graphical Separation in Settable Systems Karim Chalak Downloaded from http://direct.mit.edu/neco/article-pdf/24/7/1611/1065694/neco_a_00295.pdf by guest on 26 September 2021 [email protected] Department of Economics, Boston College, Chestnut Hill, MA 02467, U.S.A. Halbert White [email protected] Department of Economics, University of California, San Diego, La Jolla, CA 92093, U.S.A. We study the connections between causal relations and conditional in- dependence within the settable systems extension of the Pearl causal model (PCM). Our analysis clearly distinguishes between causal notions and probabilistic notions, and it does not formally rely on graphical representations. As a foundation, we provide definitions in terms of suit- able functional dependence for direct causality and for indirect and total causality via and exclusive of a set of variables. Based on these founda- tions, we provide causal and stochastic conditions formally characterizing conditional dependence among random vectors of interest in structural systems by stating and proving the conditional Reichenbach principle of common cause, obtaining the classical Reichenbach principle as a corol- lary. We apply the conditional Reichenbach principle to show that the useful tools of d-separation and D-separation can be employed to estab- lish conditional independence within suitably restricted settable systems analogous to Markovian PCMs. 1 Introduction This article studies the connections between
    [Show full text]
  • Notes: Hypothesis Testing, Fisher's Exact Test
    Notes: Hypothesis Testing, Fisher’s Exact Test CS 3130 / ECE 3530: Probability and Statistics for Engineers Novermber 25, 2014 The Lady Tasting Tea Many of the modern principles used today for designing experiments and testing hypotheses were intro- duced by Ronald A. Fisher in his 1935 book The Design of Experiments. As the story goes, he came up with these ideas at a party where a woman claimed to be able to tell if a tea was prepared with milk added to the cup first or with milk added after the tea was poured. Fisher designed an experiment where the lady was presented with 8 cups of tea, 4 with milk first, 4 with tea first, in random order. She then tasted each cup and reported which four she thought had milk added first. Now the question Fisher asked is, “how do we test whether she really is skilled at this or if she’s just guessing?” To do this, Fisher introduced the idea of a null hypothesis, which can be thought of as a “default position” or “the status quo” where nothing very interesting is happening. In the lady tasting tea experiment, the null hypothesis was that the lady could not really tell the difference between teas, and she is just guessing. Now, the idea of hypothesis testing is to attempt to disprove or reject the null hypothesis, or more accurately, to see how much the data collected in the experiment provides evidence that the null hypothesis is false. The idea is to assume the null hypothesis is true, i.e., that the lady is just guessing.
    [Show full text]