Students often receive job offers from the sponsor the HMC Mathematics Clinic has had 130 projects at the project’s conclusion. from 58 sponsors involving 553 students. A large The academic year ends with all the forty or so number of sponsors return year after year, and the Mathematics and Engineering Clinics projects pre- Math Clinic has inspired other institutions to cre- senting their results at a professional-level meeting ate similar programs, among them San Jose State called Projects Day. Projects Day attracts some 400 University, the University of South , and people and presents a great opportunity to recruit the Institute for Pure and Applied Mathematics projects for the following year. (IPAM) at UCLA. To create a viable Clinic program, these ques- For detailed project abstracts and other infor- tions must be answered: mation about the HMC Math Clinic program, visit http://www.math.hmc.edu/clinic. For further • What makes a good Clinic project? information contact the HMC Math Clinic director, • How well can undergrads handle Susan Martonosi, at [email protected] open-ended projects? (or at CGU contact Ellis Cumberbatch at ellis. • What exactly is the role of the fac- [email protected]). Institutions wanting to ulty advisor? start a Clinic program are invited to contact either • How are faculty advisors recruited? Math Clinic director for advice. Of course the best • How are sponsors recruited? introduction is for an interested faculty member Several indicators show that HMC has successfully to go through a complete Clinic cycle while on sab- answered these questions: To date (1973 –2010), batical leave for a year!

Book Review The Cult of Reviewed by Olle Häggström

The Cult of Statistical Significance: How the about 40 percent of the time about conclusions we Standard Error Costs Us Jobs, Justice, and Lives claim to be 98 percent sure about [AR]. S.T. Ziliak and D. McCloskey The scientific method can be seen as an orga- University of Michigan Press, 2008 nized attempt to overcome such “bugs” in our US$26.95, 352 pages search for accurate knowledge about the world ISBN-13: 978-0472050079 around us. One ingredient, which during the course of the twentieth century has permeated all of sci- There are excellent evolutionary reasons why we ence to the extent that it is nowadays recognized humans have far-reaching abilities to observe the as indispensable, is mathematical , which world around us and to draw sensible conclusions helps researchers distinguish between pattern and about it. But evolution is very far from a perfect noise and to quantify how much confidence in our optimization algorithm, so it should come as no conclusions the warrant. surprise that our cognitive capacities, too, are far There can hardly be any doubt that this devel- from perfect. One example is our strong tendency opment has been of immense benefit to . to deduce patterns from meaningless noise. An- All the more interesting, then, that two prominent other is our inclination toward overconfidence in economists, Stephen Ziliak and Deirdre McCloskey, our conclusions, as evidenced by studies showing claim in their recent book The Cult of Statistical how in certain kinds of situations we are wrong Significance [ZM] that the reliance on statistical methods has gone too far and turned into a ritual and an obstacle to scientific progress. Olle Häggström is professor of mathematical statistics at A typical situation is the following. A scientist Chalmers University of Technology in Gothenburg, Swe- formulates a null hypothesis. By means of a signifi- den. His email address is [email protected]. cance test, she tries to falsify it. The analysis leads

OCTOBER 2010 NOTICES OF THE AMS 1129 to a p-value, which indicates how The Cult of Statistical Significance likely it would have been, if the null is written in an entertaining and po- hypothesis were true, to obtain data lemical style. Sometimes the authors at least as extreme as those she actu- push their position a bit far, such as ally got. If the p-value is below a cer- when they ask themselves: “If null- tain prespecified threshold (typically hypothesis significance testing is as 0.01 or 0.05), the result is deemed idiotic as we and its other critics have statistically significant, which, al- so long believed, how on earth has though far from constituting a defi- it survived?” (p. 240). Granted, the nite disproof of the null hypothesis, single-minded focus on statistical counts as evidence against it. significance that they label sizeless Imagine now that a new drug for science is bad practice. Still, to throw reducing blood pressure is being out the use of significance tests tested and that the fact of the matter would be a mistake, considering how is that the drug does have a positive often it is a crucial tool for conclud- effect (as compared with a placebo) ing with confidence that what we but that the effect is so small that it see really is a pattern, as opposed is of no practical relevance to the patient’s health to just noise. For a data set to provide reasonable or well-being. If the study involves sufficiently evidence of an important deviation from the null many patients, the effect will nevertheless with hypothesis, we typically need both statistical and high probability be detected, and the study will subject-matter significance. yield statistical significance. The lesson to learn The book also offers a short history of the sig- from this is that in a medical study, statistical nificance test. Here Ziliak and McCloskey take their significance is not enough—the detected effect polemical style to even further heights in their also needs to be large enough to be medically sig- portrayal of William Gossett (inventor of Student’s nificant. Likewise, empirical studies in t-test, the most widely used of all significance (or , geology, etc.) need to consider tests) as a hero and an angel and of not only statistical significance but also economic (the father of modern mathematical statistics, who (psychological, geological, etc.) significance. arguably did more than anyone else to give signifi- A major point in The Cult of Statistical Signifi- cance testing the central role it has today) as pretty cance is the observation that many researchers are much the devil himself. For instance, they make no so obsessed with statistical significance that they attempt at concealing their schadenfreude when neglect to ask themselves whether the detected quoting what Robert Oppenheimer (allegedly) had discrepancies are large enough to be of any said upon Fisher’s arrival in Berkeley in 1936: “I subject-matter significance. Ziliak and McCloskey took one look at him and decided I did not want call this neglect sizeless science. They exemplify to meet him” (p. 222). and discuss instances of sizeless science in, among To sum up, if statistical practice in the em- other disciplines, medicine and psychology, but for pirical is as bad as the authors say, what obvious reasons they focus most of their attention should be done? No easy fix is offered, but they do on economics. In one study, they have gone over all advocate a larger degree of pluralism among sta- of the 369 papers published in the prestigious jour- tistical methods. Here, one would have liked to see nal American Economic Review during the 1980s them address the danger that this might lead to an and 1990s that involve regression analysis. In the increase in a particular kind of misuse of statistics: 1980s, 70 percent of the studied papers committed to tune the choice of statistical approach to the sizeless science, and in the 1990s this alarming particular data that were obtained. Many of the au- figure had increased to a stunning 79 percent. A thors’ comments seem to imply a commitment to number of other kinds of misuse of statistics are the Bayesian paradigm, but it is not clear whether considered in the same study, with mostly equally they are really aware of this. In any case they never depressing results. explicitly step out of the Bayesian closet. One particular error, which every teacher of References mathematical statistics is painfully familiar with, [AR] M. Alpert and H. Raiffa, A progress report on the is to conflate the probability of the observed data training of probability assessors, Judgment under given the null hypothesis with the probability of Uncertainty: Heuristics and Biases (D. Kahneman, the null hypothesis given the data (the latter can- P. Slovic, and A. Tversky, eds.), Cambridge University not, of course, be obtained unless we resort to Press, 1982. Bayesian statistics, a framework that is still rare in [ZM] S. T. Ziliak and D. McCloskey, The Cult of Statistical the fields under study). This error, known as the Significance: How the Standard Error Costs Us Jobs, of the transposed conditional, is discussed Justice, and Lives, The University of Michigan Press, in the book but does not appear as a separate item Ann Arbor, MI, 2008. in the American Economic Review literature study.

1130 NOTICES OF THE AMS VOLUME 57, NUMBER 9