29 Empirical Evidence: Its Nature and Sources Julian Reiss INTRODUCTION structural model, which is based on eco- nomic theory, then to operationalise its terms With the rise of evidence-based movements and test the fit of the model against data. in medicine and social policy, the topic This structural equation modelling approach of evidence has come to the forefront of was developed by members of the Cowles research in the philosophy and methodology Commission in the 1930s and 1940s and of science. But the issue is far from new. has since been supplemented by a variety Observation, experiment, induction and con- of other techniques in econometrics such firmation – all practises very closely related as the analysis of natural experiments, but to evidence – have been central concerns of also laboratory experiments, simulation philosophers ever since the birth of Western and various forms of conceptual or thought philosophy. The primary aim of this article experimentation. is to provide an introduction to and illumi- This article has two parts. In the first part, I nation of these topics in so far as they are will survey philosophical theories of evidence relevant to the social sciences. and in so doing attempt to answer questions Empirical evidence in the social sciences regarding the nature of evidence and the is extraordinarily varied. It is produced by nature of the inference from evidence to methods including the collection of physical hypotheses. The second part will give a nec- artefacts in archaeology, conducting cen- essarily incomplete overview of the different suses in demography, mathematical model- sources of evidence in the social sciences and ling in economics, thought experimentation asks how to combine their products. in history, expert judgement in political Before diving into the various philosophi- science, laboratory experimentation in psy- cal proposals for theories of evidence and chology and causal modelling in sociology, inference, a few terminological clarifications among many others. Even within one and the are in order. First, the notion of scientific same science, evidence can have a variety evidence has at least three connotations that of sources. To take economic policy as an are relevant here. According to Webster’s example, the traditional way to substantiate New World Dictionary (Second College edi- economic policy claims is to first build a tion), ‘evidence’ refers to: (1) the condition 55579-Jarvie-Chap29.indd579-Jarvie-Chap29.indd 555151 111/10/20101/10/2010 33:26:05:26:05 PPMM 552 THE SAGE HANDBOOK OF THE PHILOSOPHY OF SOCIAL SCIENCES of being evident; (2) something that makes ‘data’ or ‘test result’ or something similar another thing evident; indication; sign; and for it. (3) something that tends to prove; ground Second, as the passage from Salmon illus- for belief. Accordingly, scientific evidence trates, the notion of evidence is closely means, first, the more or less observable related to those of confirmation and induc- outcomes of scientific tests such as experi- tion. Hypotheses are confirmed by evidence, ments, statistical analyses and surveys. Used and most theories of evidence that will be in this way, the notion is more or less syn- discussed below have been introduced as onymous with ‘data’ or ‘scientific result’. theories of confirmation. Induction refers According to the second entry, scientific to the mode of reasoning or inference from evidence means hint, sign, indication of or evidence to hypothesis. As the hypothesis a reason to believe (the negation of) a sci- contains more, or at least different, infor- entific hypothesis. According to the third, mation than the evidence, this mode of the word means (something that furnishes) reasoning is ampliative – enlarging what is proof of or good or cogent reason to believe already known. It contrasts with deductive (the negation of) a hypothesis. The ambiguity reasoning, which proceeds in reverse order between the latter two meanings is illustrated from a more general claim to a more specific by a passage from Wesley Salmon (Salmon, claim. Unlike deductive reasoning, which is 1975: 6): truth preserving, reasoning from evidence is fallible. As Carnap pointed out in Logical Foundations of Probability, the concept of confirmation is radically ambiguous. If we say, for example, that the special theory of relativity has been confirmed by experi- mental evidence, we may have either of two quite TAKING THE CON OUT OF distinct meanings in mind. On the one hand, we may intend to say that the special theory has CONFIRMATION become an accepted part of scientific knowledge and that it is very nearly certain in light of its sup- Some Preliminary Remarks porting evidence. If we admit that scientific hypotheses can have numerical degrees of Philosophers of science often treat theories of confirmation, the sentence, on this construal, says evidence and of induction (or confirmation) that the degree of confirmation of the special as if they were of the same kind. But this theory of relativity on the available evidence is high. On the other hand, the same sentence might would be a mistake. Theories of evidence be used to make a very different statement. It are supposed to answer questions regard- might be taken to mean that some particular evi- ing the nature of evidence and the kinds of dence – for example, observations on the lifetimes observations or tests a researcher needs to of mesons – renders the special theory more make in order to have evidence in favour of acceptable or better founded than it was in the absence of this evidence. If numerical degrees of the hypothesis at stake. Theories of induc- confirmation are again admitted, this latter con- tion, by contrast, begin with an antecedently strual of the sentence amounts to the claim that understood notion of evidence and ask what the special theory has a higher degree of the kinds of inferences are that one is justi- confirmation on the basis of the new evidence fied to make given one has evidence of the than it had on the basis of the previous evidence alone. required type at hand. Bayesianism, for example, is often In what follows I will use ‘evidence’ almost described as a theory of evidence (as in always in sense 2, indication or sign, the only Achinstein, 2001) but it is completely silent exception being Peter Achinstein’s theory on the issue of the nature of evidence. Rather, of evidence, which is a theory of a good or it tells us what a rational agent should do cogent reason to believe. Evidence in sense in a situation where she comes to believe 1 will play an important role but I will say an evidential statement (viz., to update her 55579-Jarvie-Chap29.indd579-Jarvie-Chap29.indd 555252 111/10/20101/10/2010 33:26:06:26:06 PPMM EMPIRICAL EVIDENCE: ITS NATURE AND SOURCES 553 degree of belief in the hypothesis according theory of induction that tells us what infer- to a specific rule). Colin Howson and Peter ences are warranted after the evidence has Urbach describe the matter as follows: come to be believed. This theory is silent about what evidence is. A full account of The Bayesian theory we are proposing is a theory learning from evidence requires both. Here I of inference from data; we say nothing about will look at theories of evidence first, then at whether it is correct to accept the data … The Bayesian theory of support is a theory of how the theories of induction and finally at a number acceptance as true of some evidential statement of hybrid theories. affects your belief in some hypothesis. How you came to accept the truth of the evidence, and whether you are correct in accepting it as true, are matter which, from the point of view of the Theories of Evidence theory, are simply irrelevant. (Howson and Urbach, 1993: 272) In this section I will look at two families of theories of evidence, instance theories and By contrast, Mill’s methods, though often hypothetico-deductive theories. The former described as a theory of induction (as in regards an instance of a hypothesis evidence Norton, 2003) are in fact at best informative for it, the latter, its entailments. about the types of observations one must make in order to support a (in this case, Instance Theories causal) hypothesis. For instance, the method According to the first family of theories of of difference tells us evidence in favour of a evidence, a state of affairs provides evidence causal hypothesis can be provided by two sit- for a general hypothesis if and only if it is an uations that are exactly identical except with instance of the hypothesis. Here I will look at respect to the phenomenon of interest. The theories of evidence regarding two kinds of method, by itself, is not informative about general hypotheses: simple subject-predicate the kinds of inferences warranted by the hypotheses and causal hypotheses. observation of two such situations. It does not say, for example, whether, after having made Simple natural laws the observation, we should accept the causal By ‘simple natural laws’ I refer to universally hypothesis as true or raise our degree of con- quantified statements that ascribe a property fidence in the hypothesis or rather assess the to a kind or substance such as ‘All ravens probability of accepting a false hypothesis are black’ (Hempel, 1945) or ‘All samples of if the test were run again and again.1 the element bismuth melt at 271°C’ (Norton, That inference to the best explanation 2003). According to this first theory, evi- and Bayesianism are compatible has been dence for such a generalisation is constituted noticed before (see for instance Lipton, 2004: by the instances of the generalisation.