The Principles of Uncertainty Statistics and Real Life? It’S Messy, Writes Andrew Gelman on Ian Stewart’S Analysis

COMMENT BOOKS & ARTS

The book is decorated, in the manner of bear (Ursus arctos); the spontaneous actions simply not yet known. But she nevertheless our best philosophers, with pithy illustrative of farmers who milk the cows of a neighbour makes a mighty effort. Conscience is examples. Many are drawn from Church- stricken by influenza; a sign in a farm kitchen illuminating, entertaining and wise. ■ land’s upbringing on a farm in the wilderness proclaiming, “Them that works, eats.” of the Pacific Northwest. (She calls herself a The limitations in Churchland’s account Nicholas A. Christakis is the Sterling “country bumpkin”.) They are wonderful: are mostly limitations in the state of the field. Professor of Social and Natural Science at rafting teams circumventing rapids in Cana- As she repeatedly notes, many aspects of Yale University and the author of Blueprint: da’s Yukon Territory; ways to chop firewood; how conscience comes to be embodied in the The Evolutionary Origins of a Good Society. the strategic hunting behaviour of the grizzly brain, and shaped by natural selection, are e-mail: [email protected]

MATHEMATICS The principles of uncertainty Statistics and real life? It’s messy, writes Andrew Gelman on Ian Stewart’s analysis.

ncertainty “isn’t always bad”, begins the familiar P < 0.05 Stewart makes the common mathematical Do Dice Play God?, the latest book (in which P signifies error of transposing the probabilities. He from celebrated mathematics writer probability) so often interprets 0.05 as the probability that the UIan Stewart. It ends: “The future is uncertain, used in this and other hypothesis is true; it is actually a statement but the science of uncertainty is the science journals to indicate about how probable it would be to see the of the future.” In between, Stewart discusses that a certain empirical results or something more extreme if the null topics from mathematics to meteorology, in result has a statistical hypothesis were true. (It isn’t, in this case.) which accepting uncertainty is necessary to seal of approval. Later, he erroneously states that a understand how the world works. He touches Here’s how Stewart confidence interval indicates “the level of on probability theory and chaos (the subject puts it in the context Do Dice Play God? confidence in the results”; in fact, it is a sta- The Mathematics of his 1989 book Does God Play Dice?). And of an otherwise char- of Uncertainty tistical procedure for expressing uncertainty, he probes the connection between quantum acteristically clearly IAN STEWART or a range of values consistent with the data. entanglement and communication, with described example of Profile (2019) Stewart does, however, discuss a mistake interesting excursions into the history of counts of births of boys all too common among researchers and mathematics, gambling and science. and girls: “The upshot here is that p = 0.05, so students: using the statistical rejection of a My favourite aspect of the book is the there’s only a 5% probability that such extreme straw-man null hypothesis to validate a sci- connections it makes in a sweeping voyage values arise by chance”; thus, “we’re 95% con- entific claim about the real world. In simple from familiar (to me) paradoxes, through fident that the null hypothesis is wrong, and cases, this might not be an issue. In reject- modelling in human affairs, up to modern we accept the alternative hypothesis”. (In gen- ing the model that births of boys and of girls ideas in coding and much more. We get a eral, the null hypothesis is a comparison point are equally likely, we at the same time learn sense of the different “ages of uncertainty”, in a statistical analysis. Here, it is the supposi- the general fact of likelier boy births. But as Stewart puts it. tion that births of boys and girls occur with this kind of learning-by-rejection can fail in But not all the examples work so well. The equal probabilities; in fact, the birth of a boy more complicated settings. A null hypoth- book’s main weakness, from my perspective, is slightly more likely.) esis is extremely specific, and the alternative is its assumption that mathematical models apply directly to real life, without recognition of how messy real data are. That is something I’m particularly aware of, because it is the business of my field — applied statistics. For example, after a discussion of uncertainty, surveys and random sampling, Stewart writes, “Exit polls, where people are asked who they voted for soon after they cast their vote, are often very accurate, giving the cor-

rect result long before the official vote count VIA GETTY KRISZTIAN BOCSI/BLOOMBERG reveals it.” This is incorrect. Raw exit polls are not directly useful. Before they are shared with the public, the data need to be adjusted for non-response, to match voter demograph- ics and election outcomes. The raw results are never even reported. The true value of the exit poll is not that it can provide an accurate early vote tally, but that it gives a sense of who voted for which parties once the election is over. It is also disappointing to see Stewart trot- ting out familiar misconceptions of hypothesis testing, the statistical theory underlying Drug testing is reliant on statistical models.

BOOKS & ARTS COMMENT

includes not just one correct answer, but all other possibilities. In a medical experiment, the null Books in brief hypothesis might be that a new drug has no effect. But the hypothesis will come pack- The Age of Addiction aged in a statistical model that assumes that David T. Courtwright Belknap (2019) there is zero systematic error. This is not Opioids, processed foods, social-media apps: we navigate an necessarily true: errors can arise even in a addictive environment rife with products that target neural pathways randomized, blinded study, for example if involved in emotion and appetite. In this incisive medical history, some participants work out which treatment David Courtwright traces the evolution of “limbic capitalism” from group they have been assigned to. This can prehistory. Meshing psychology, culture, socio-economics and lead to rejection of the null hypothesis even urbanization, it’s a story deeply entangled in slavery, corruption when the new drug has no effect — as can and profiteering. Although reform has proved complex, Courtwright other complexities, such as unmodelled posits a solution: an alliance of progressives and traditionalists aimed measurement error. at combating excess through policy, taxation and public education. To say that P = 0.05 should lead to acceptance of the alternative hypothesis is tempting — a few million scientists do it Cosmological Koans every year. But it is wrong, and has led to Anthony Aguirre W. W. Norton (2019) replication crises in many areas of the social, Cosmologist Anthony Aguirre explores the nature of the physical behavioural and biological sciences. Universe through an intriguing medium — the koan, that paradoxical Statistics — to paraphrase Homer riddle of Zen Buddhist teaching. Aguirre uses the approach playfully, Simpson’s thoughts on alcohol — is the cause to explore the “strange hinterland” between the realities of cosmic of, and solution to, all of science’s problems. structure and our individual perception of them. But whereas his Many difficulties have been associated with discussions of time, space, motion, forces and the quantum are the misuse of statistics to make inappropri- eloquent, the addition of a second framing device — a fictional ately strong claims from noisy data, but I journey from Enlightenment Italy to China — often obscures rather don’t think that the solution is to abandon than clarifies these chewy cosmological concepts and theories. formal statistics. Variation and uncertainty are inherent in modern science. Rather, we need to go deeper in our statistical model- Vanishing Fish ling. For example, in polling, we accept that Daniel Pauly Greystone (2019) we cannot get clean randomized or repre- In 1995, marine biologist Daniel Pauly coined the term ‘shifting sentative sampling, so we gather the data baselines’ to describe perceptions of environmental degradation: necessary to adjust our sample to match the what is viewed as pristine today would strike our ancestors as population. damaged. In these trenchant essays, Pauly trains that lens on fisheries, As I recall the baseball analyst Bill James revealing a global ‘aquacalypse’. A “toxic triad” of under-reported writing somewhere, the alternative to good catches, overfishing and deflected blame drives the crisis, he argues, statistics is not no statistics: it’s bad statistics. complicated by issues such as the fishmeal industry, which absorbs We must design our surveys, our clinical tri- a quarter of the global catch. If current subsidies were redirected to als and our meteorological studies with an sustainable ends, he avers, the worst outcomes might be avoided. eye to eliminating potential biases, and we must adjust the resulting data to make up the biases that remain. If we do not, people In the Shadow of Vesuvius can take the numbers that are available and Daisy Dunn William Collins (2019) draw all sorts of misleading conclusions. On 24 August in ad 79, the Roman aristocrat Pliny the Younger One thing I like about Stewart’s book is that witnessed a “cloud, both strange and enormous in appearance” he faces some of these challenges directly. above the Bay of Naples. The eruption of Vesuvius engulfed cities, In a sense, the answer to Stewart’s including Pompeii, and killed his uncle, the renowned admiral and question, “Do dice play god?”, is yes. Proba- natural historian Pliny the Elder. Daisy Dunn’s nuanced biography bility is an unreasonably effective mathemat- breathes new life into the younger Pliny, revealing his uncle’s ical model for uncertainty in so many areas scientific and philosophical influence on his own evolution into poet, of life. I believe that a key future develop magistrate, senator, curator of drains and prolific letter writer. An ment in the science of uncertainty will be evocative portrait of Renaissance men before the Renaissance. tools to ensure that the adjustments we need to make to data are more transparent and easily understood. And we will develop Math Art this understanding, in part, through math- Stephen Ornes Sterling (2019) ematical and historical examples of the sort The elusive elegance of mathematics might seem inaccessible to those discussed in this stimulating book. ■ who don’t speak the language — but some artists brilliantly bridge the gap. And in this visually sumptuous, intellectually compelling study, Andrew Gelman is a professor of statistics science writer Stephen Ornes has tapped that community for a virtual and political science at Columbia University maths-art gallery. Here are Self-Similar Surface by Robert Fathauer, a in New York City, and has written books on ruffled homage to rotational and mirror symmetry; Anita Chowdry’s applied regression analysis, Bayesian statistics, sculptural ‘steampunk’ harmonograph, Iron Genie; the glowing statistics education and American politics. fractal ‘portrait’ Buddhabrot by Melinda Green — and much more. e-mail: [email protected] Mathematics made manifest, and beautifully. Barbara Kiser