Apophenia and Anesthesia: How We Sometimes Change Our Practice
Total Page：16
File Type：pdf, Size：1020Kb
Can J Anesth/J Can Anesth (2021) 68:1185–1196 https://doi.org/10.1007/s12630-021-02005-2 SPECIAL ARTICLE Apophenia and anesthesia: how we sometimes change our practice prematurely Apophe´nie et anesthe´sie : comment il nous arrive de modifier pre´mature´ment notre pratique Neil A. Hanson, MD • Matthew B. Lavallee, MD • Robert H. Thiele, MD Received: 9 November 2020 / Revised: 8 February 2021 / Accepted: 16 February 2021 / Published online: 7 May 2021 Ó Canadian Anesthesiologists’ Society 2021 Abstract Human beings are predisposed to identifying les manuscrits prometteurs ou notables, les incitatifs tant false patterns in statistical noise, a likely survival dans les milieux acade´miques qu’e´ditoriaux, qui pre´fe`rent advantage during our evolutionary development. le changement a` la ve´rite´ et a` la rigueur scientifique, et une Moreover, humans seem to prefer ‘‘positive’’ results over de´pendance croissante a` l’e´gard de techniques statistiques ‘‘negative’’ ones. These two cognitive features lay a complexes que certains re´viseurs ne comprennent pas. framework for premature adoption of falsely positive L’objectif de cet article est de de´crire les causes sous- studies. Added to this predisposition is the tendency of jacentes d’adoption pre´mature´e de nouveaute´setde journals to ‘‘overbid’’ for exciting or newsworthy proposer des recommandations afin d’ame´liorer la manuscripts, incentives in both the academic and qualite´ de la science publie´e. publishing industries that value change over truth and scientific rigour, and a growing dependence on complex Keywords apophenia Á anesthesia Á bias Á statistical techniques that some reviewers do not premature adoption Á incentives understand. The purpose of this article is to describe the underlying causes of premature adoption and provide recommendations that may improve the quality of Science is in the midst of a crisis. A string of high-profile published science. retractions and clear evidence of outright fraud, most recently related to the COVID-19 pandemic, have captured ´ ´ ˆ ` Resume Les etres humains ont tendance a identifier de the world’s attention and shaken the public’s belief in ´ fausses correlations dans le bruit de fond statistique, ce qui scientific integrity.1,2 Criticism of the peer review process ´ ´ ` nous a probablement confere un avantage en matiere de has reached the mainstream,3 and major news outlets now ´ ´ survie au cours de notre developpement evolutionnaire. De routinely report on the results of scientific trials in the pre- ˆ ´ ´ ´ plus, l’etre humain semble preferer les resultats « positifs » review stage of publication (e.g., www.medrxiv.org). This ´ ´ ´ aux resultats « negatifs ». Ces deux caracteristiques removes an important check on data integrity, through the ˆ cognitives posent un cadre expliquant l’adoption hative peer review process, and allows the general public to ´ ` ´ d’etudes faussement positives. A cette predisposition consume ‘‘news’’ that has not been properly verified by ` ´ s’ajoutent la tendance des revues a « surencherir » pour subject matter experts. While intentional fabrication of data is heinous and newsworthy, it is a relatively infrequent occurrence.4,5 A much larger, more complex, and more This article is accompanied by an editorial. Please see Can J Anesth sinister threat to scientific data integrity is the premature 2021; this issue. acceptance of non-fraudulent data that, while scientifically N. A. Hanson, MD (&) Á M. B. Lavallee, MD Á valid and ‘‘statistically significant,’’ for reasons we will R. H. Thiele, MD describe below does not warrant wholesale adoption. It is Department of Anesthesiology, University of Virginia Health in this space—premature adoption—that the specialty of System, PO Box 800710, ville, VA 22908-0710, USA anesthesiology (which includes critical care, perioperative e-mail: [email protected] 123 1186 N. A. Hanson et al. medicine, and pain management) has, like many spe- because for most of human history it was far safer to err on cialties, been damaged. the side of overidentification than underidentification of The purpose of this manuscript is to describe the important patterns and relationships around us. underlying economic, mathematical, social, and scientific While the Atropa belladonna example is an extreme causes of premature adoption. We will provide the reader example, it illustrates two key points in how humans make with a chronological list of high-profile examples of decisions. The first is referred to as cost asymmetry and it premature adoption in three domains of anesthesiology has numerous implications in religion, sociology, and (critical care, perioperative, and pain), and, based on both economics—but also science and statistics—the sum of the underlying causes and notable examples in our which are included in a relatively new field of study called specialty, make recommendations that may improve the error management theory.9 Simply put, humans are biased quality of our own literature as well as the ability of our towards making less costly errors even if it means readership to effectively integrate anesthesiology science increasing overall error rates. The second key point, that into their practice. the human mind tends to identify patterns that do not really exist, is well documented in the psychology literature and deserves greater discussion, as it is particularly relevant to Underlying psychology the problem of premature adoption. Most individuals seem to develop erroneous perceptions In two contemporary works, Fooled by Randomness: The about the meaning of random, even binary, data. Robert Hidden Role of Chance in Life and in the Markets6 and Ladouceur measured this directly in a series of experiments Blink: The Power of Thinking Without Thinking,7 authors which showed that the subjects believed that objectively Nassim Taleb and Malcom Gladwell make an evolutionary independent events were causally linked in some fashion.10 argument for the premature acceptance of data. Both Taleb This is perhaps best described in the gambler’s fallacy— and Gladwell argue that millennia of external threats and the belief, for instance, that after receiving three red selection pressures have created a species that very quickly numbers in a row while playing roulette, that a black draws conclusions and acts based on very small data sets. number is ‘‘due’’.11 The underlying thinking behind this Let us use an example to illustrate their theory. Say, for perception has to do with how humans conceive of the instance, that you and your family were foraging for food concept of chance itself. ‘‘Chance is commonly viewed as in eastern Europe thousands of years ago. You stumble a self-correcting process in which a deviation in one across Atropa belladonna, a tropane alkaloid-producing direction induces a deviation in the opposite direction to plant that is one of the most lethal in the Western restore the equilibrium.’’12 In fact, the term ‘‘corrected’’ Hemisphere. But belladonna also makes berries that appear should not be used to describe this process, as it is not enticing. Perhaps your uncle and his son both died after deterministic in any fashion. Rather, we need to understand eating berries from belladonna many years ago, but you are that the ‘‘deviations’’ are merely diluted over time as the currently starving for nourishment. What do you do? You number of events increases. This is the essence of what know of only two people who have ever consumed Amos Tversky and Daniel Kahneman referred to as the law belladonna, both of whom have died. of large numbers. Unfortunately, humans predominantly As you might imagine, humans who concluded that ascribe to the law of small numbers, wherein they Atropa belladonna was lethal, even after only one or two incorrectly perceive that small samples are representative observations, might be more likely to survive (and of the greater population from which they are drawn.13 procreate). Avoiding Atropa belladonna is, in the Psychosis is simply an extreme manifestation of our circumstances described, the correct decision. Those propensity to create connections that do not actually exist. individuals who required ‘‘more data’’ to be certain of This understanding has led to the development of the the validity of their hypothesis, more than likely, perished. concept of apophenia—the ‘‘tendency to perceive meaning Clearly, this ‘‘study’’ is currently underpowered, and in noise’’—and ‘‘magical thinking,’’ both of which are submitting a scientific study with an n of 2 would be associated with anomalous perceptual experiences and considered malpractice in 2021. But, in this specific frank psychosis.14-17 This is of concern because, when circumstance, does it truly matter if the study is individuals are presented with graphical depictions of underpowered?8 While no one would ever categorize synthetic data, over 60% of user-generated insights are these observations as an application of the scientific patently false (Figure).18 Substantial efforts have been method, the experience provided ‘‘data’’ that changed made to identify the neuroanatomical foundation of ‘‘practice’’. Thousands and thousands of years of this apophenia. For instance, the use of transcranial magnetic selection process has ultimately led to a species that is stimulation to inhibit activity in the left lateral temporal primed to identify patterns that do not necessarily exist, area significantly reduces the tendency of healthy 123 Apophenia and anesthesiology 1187 volunteers to report meaningful information when 1,163