Testing Hypotheses: Prediction and Prejudice Peter Lipton, et al. Science 307, 219 (2005); DOI: 10.1126/science.1103024 The following resources related to this article are available online at www.sciencemag.org (this information is current as of May 31, 2007 ): Updated information and services, including high-resolution figures, can be found in the online version of this article at: http://www.sciencemag.org/cgi/content/full/307/5707/219 A list of selected additional articles on the Science Web sites related to this article can be found at: http://www.sciencemag.org/cgi/content/full/307/5707/219#related-content This article cites 5 articles, 2 of which can be accessed for free: http://www.sciencemag.org/cgi/content/full/307/5707/219#otherarticles This article has been cited by 10 article(s) on the ISI Web of Science. This article has been cited by 4 articles hosted by HighWire Press; see: on May 31, 2007 http://www.sciencemag.org/cgi/content/full/307/5707/219#otherarticles This article appears in the following subject collections: Physics http://www.sciencemag.org/cgi/collection/physics Information about obtaining reprints of this article or about obtaining permission to reproduce this article in whole or in part can be found at: www.sciencemag.org http://www.sciencemag.org/about/permissions.dtl Downloaded from Science (print ISSN 0036-8075; online ISSN 1095-9203) is published weekly, except the last week in December, by the American Association for the Advancement of Science, 1200 New York Avenue NW, Washington, DC 20005. Copyright c 2005 by the American Association for the Advancement of Science; all rights reserved. The title SCIENCE is a registered trademark of AAAS. REVIEW Testing Hypotheses: Prediction and Prejudice Peter Lipton hoc’’ literally means ‘‘purpose-built.’’ To say Observations that fit a hypothesis may be made before or after the hypothesis is formulated. that an accommodation is ad hoc in this sense is Can that difference be relevant to the amount of support that the observations provide for just to repeat that it is accommodation; it is not the hypothesis? Philosophers of science and statisticians are both divided on this question, to say or to show that the hypothesis is poorly but there is an argument that predictions ought to count more than accommodations, supported or otherwise deficient. On the other because of the risk of ‘‘fudging’’ that accommodations run and predictions avoid. hand, the expression ‘‘ad hoc hypothesis’’ is often used in a derogatory sense that implies that n the case of Baccommodation,[ a hypoth- that the other predicts. If there really were an the hypothesis is poorly supported. But on that esis is constructed to fit an observation that advantage to prediction, we ought to say that reading, the argument becomes circular: Ac- I has already been made. In the case of the predictor has more reason to believe the commodating hypotheses are poorly supported, Bprediction,[ the hypothesis, though it may hypothesis than the accommodator, though therefore they are poorly supported. On either already be partially based on an existing data they share hypothesis, data, and background reading, the ad hoc argument fails. set, is formulated before the empirical claim in beliefs. This is counterintuitive, but things A second argument for the advantage question is deduced and verified by observa- get worse. Suppose that the twins meet and thesis is the argument from testing. According tion. Well-supported hypotheses often have discover their situation. It seems clear that to this argument, predictions are worth more both accommodations and successful predic- they should leave the meeting with a com- than accommodations because it is only tions to their credit. Most people, however, mon degree of confidence in the hypothesis through its predictions that a hypothesis gets appear to be more impressed by predictions they share. If they came to the meeting with properly tested, and it is only by passing a on May 31, 2007 than by accommodations. Edmond Halley was different degrees of rational confidence in test that a hypothesis gains genuine support. able to account for the observed comets of their hypothesis, at least one of them ought The idea is that a test is something that could 1531, 1607, and 1682 as a single object with a to leave with a revised view. But what level be failed, and it is only a prediction that a perturbed elliptical orbit. Natural philosophers should they settle on: the higher one of the hypothesis can fail. Here the hypothesis is took some notice when he published his views predictor, the lower one of the accommoda- made to commit in advance and say how things in the Philosophical Transactions in 1705, but tor, or somewhere in between? will be, so that we may go on to discover that it was only when his prediction of the return of There seems no way to answer the things actually are not that way. So if the the comet in 1758 was confirmed that the question. Moreover, if there is a relevant hypothesis passes the test of prediction, the entire European intellectual world embraced difference between prediction and accom- hypothesis has earned some credit. This thought www.sciencemag.org Halley_s Comet. The single prediction appears modation, then the twin who should revise is closely related to Karl Popper’s emphasis on to have been far more impressive than the her view after she meets her sibling must not the importance of getting hypotheses to stick three accommodations (1). revise simply because she knew all along their necks out, though Popper himself took the Was this reaction rational? It is surprisingly that someone like her twin might have existed. radical view that there is no such thing as difficult to establish an advantage thesis: to If revision were in order merely because of positive evidence (7). In accommodation, by show that predictions tend to provide stronger this possibility, then the difference between contrast, the hypothesis does not stick its neck support than accommodations. The content of accommodation and prediction would vanish. out: It cannot be shown to be wrong, because the hypothesis, of the statements needed to link Whenever data are accommodated, we know the hypothesis is constructed after the data are Downloaded from the hypothesis to the observation, of back- that there might have been someone who known and compatibility is thus guaranteed. ground beliefs, and of the observation itself may produced the hypothesis earlier and predicted An analogy helps to bring out the intuitive all be unaffected by the question of whether the the data instead. But how can the question of strength of the argument from testing (8). observation was accommodated or predicted, whether there actually is such a person make Suppose that Jacob, my elder son, takes his and these seem to be the only factors that can any difference to our justified confidence in trusty bow and arrow and shoots at a target on affect the degree to which a hypothesis is sup- the hypothesis? Any adequate defense of the the side of a barn, hitting the bull’s-eye. We are ported by evidence. The difference between ac- putative difference between prediction and duly impressed. Now Jonah, my younger son, commodation and prediction seems merely one accommodation will have to explain how an steps up to a different barn, pulls back his of timing. Some observations are more reliable actual meeting could be different from a hypo- bow, and shoots his arrow at the barn. Then and more telling than others, but the date when thetical meeting. Those who reject the distinc- he walks up to the side of his barn and paints they were made seems to be irrelevant (2–6). tionseemtobeonfirmgroundwhenthey a bull’s-eye around his arrow. We would give To make the case against the advantage maintain that no such explanation is possible. him rather less credit, for archery anyway. thesis more vivid, consider a fictitious case Accommodation is like drawing the bull’s-eye of twin scientists. These twins independently Ad Hoc Hypotheses, Real Tests, and afterwards, whereas in prediction the target is and coincidentally generate the same hypoth- Best Explanations there in advance. The argument from testing esis. The only difference between them is Here are three popular defenses of the seems clearly to show why successful predic- that one twin accommodates an observation advantage thesis. One is that accommodation tion should count more than accommodation. allows a hypothesis to be built around the Nevertheless, as it stands, this too is a data, but such a hypothesis would be ad hoc poor argument. It confuses the scientific hy- Department of History and Philosophy of Science, Free School Lane, Cambridge CB2 3RH, UK. E-mail: and hence only poorly supported. As it stands, pothesis with the scientist who formulates it. [email protected] this is not a good argument. The expression ‘‘ad What is true is that only in the case of pre- www.sciencemag.org SCIENCE VOL 307 14 JANUARY 2005 219 R EVIEW diction does the scientist run the risk of having who just repeats the same experiment over and about what predictions to check: Halley is a been mistaken, because it is only in the case of over eventually reaches a point of diminishing case in point. Nevertheless, the argument from prediction that the scientist may have to admit returns, whereas a hypothesis supported by a choice does show why successful predictions to having made a false prediction. But we ought variety of experiments rightly inspires greater tend sometimes to be more powerful than ac- to concern ourselves here with the science, not confidence.