The Interface Theory of Perception

!1 The Interface Theory of Perception (To appear in The Stevens' Handbook of Experimental Psychology and Cognitive Neuroscience) DONALD D. HOFFMAN ABSTRACT Our perceptual capacities are products of evolution and have been shaped by natural selection. It is often assumed that natural selection favors veridical perceptions, namely, perceptions that accurately describe those aspects of the environment that are crucial to survival and reproductive fitness. However, analysis of perceptual evolution using evolutionary game theory reveals that veridical perceptions are generically driven to extinction by equally complex non-veridical perceptions that are tuned to the relevant fitness functions. Veridical perceptions are not, in general, favored by natural selection. This result requires a comprehensive reframing of perceptual theory, including new accounts of illusions and hallucinations. This is the intent of the interface theory of perception, which proposes that our perceptions have been shaped by natural selection to hide objective reality and instead to give us species-specific symbols that guide adaptive behavior in our niche. INTRODUCTION Our biological organs — such as our hearts, livers, and bones — are products of evolution. So too are our perceptual capacities — our ability to see an apple, smell an orange, touch a grapefruit, taste a carrot and hear it crunch when we take a bite. Perceptual scientists take this for granted. But it turns out to be a nontrivial fact with surprising consequences for our understanding of perception. The evolution of perception may be taken for granted, but what this evolution entails is not yet well understood. The evolution of perception profoundly affects the answers to questions that are fundamental to the science of perception: What is the relationship between one’s perceptions and objective reality, i.e., reality as it is !2 when one does not observe? Do our perceptions accurately estimate objective reality? Why do our actions transform our perceptions in systematic and predictable ways? What do such transformations entail about the structure of objective reality? What are psychophysical laws? Why do they have the form that they do? What are illusions? What are hallucinations? How precisely do they differ from normal perceptions? What is the content of a perceptual experience? The evolution of perception also profoundly affects the answers to questions that are fundamental to cognitive neuroscience more generally: What is the relationship between mental states and neural activity? Do neural states and processes have causal powers? Do they cause conscious experiences and other mental states? It is widely assumed by vision scientists that evolution shapes our perceptions to accurately estimate true properties of reality. For instance, Palmer (1999) says, “Evolutionarily speaking, visual perception is useful only if it is reasonably accurate ... Indeed, vision is useful precisely because it is so accurate. By and large, what you see is what you get. When this is true, we have what is called veridical perception ... perception that is consistent with the actual state of affairs in the environment. This is almost always the case with vision...” (emphasis his). Knill et al (1996, p. 6) say, “Visual perception ... involves the evolution of an organism’s visual system to match the structure of the world and the coding scheme provided by nature.” Marr (1982, p. 340) says, “We ... very definitely do compute explicit properties of the real visible surfaces out there, and one interesting aspect of the evolution of visual systems is the gradual movement toward the difficult task of representing progressively more objective aspects of the visual world.” The intuition is that those of our ancestors who saw more accurately enjoyed a competitive advantage over those who saw less accurately, and were therefore more likely to pass on their genes that coded for more accurate perceptions. We are the result of thousands of generations of this process, and thus we can be !3 confident that, in the normal case, our perceptions accurately estimate those properties of reality that are critical for our survival and reproduction. Geisler and Diehl (2003) say this succinctly: “In general, (perceptual) estimates that are nearer the truth have greater utility than those that are wide of the mark.” Trivers (2011) spells it out a bit more: “...our sense organs have evolved to give us a marvelously detailed and accurate view of the outside world—we see the world in color and 3-D, in motion, texture, nonrandomness, embedded patterns, and a great variety of other features. Likewise for hearing and smell. Together our sensory systems are organized to give us a detailed and accurate view of reality, exactly as we would expect if truth about the outside world helps us to navigate it more effectively.” This intuition is compelling but, as we shall see, false. Monte Carlo simulations of evolutionary games demonstrate that perceptions which accurately estimate reality never outcompete perceptions of equal complexity which do not estimate reality but are instead tuned to the relevant fitness functions (Mark et al., 2010; Hoffman et al., 2013; Marion, 2013; Mark, 2013). The key idea here is the fitness function. What is the fitness conveyed by, say, a piece of raw beef? The answer depends on the organism, its state, and its action. For a hungry cheetah looking to eat, the beef enhances fitness. For a sated cheetah looking to mate, it does not. And for a cow looking to do anything, it does not. Thus a fitness function depends not just on the state of objective reality, but also, and crucially, on the organism, its state and action. Fitness functions, not objective reality, are the coin of the realm in evolutionary competition. The results of Monte Carlo simulations are now buttressed by the Fitness-Beats-Truth (FBT) Theorem: For an infinitely large class of generically chosen worlds, for generically chosen probabilities of states on the worlds, and for generically chosen fitness functions, an organism that accurately estimates reality is never, in an infinite class of evolutionary games, more fit than an organism of equal complexity that does not estimate objective reality but is instead tuned to the relevant fitness functions. !4 The FBT Theorem says the probability is low, approaching zero, that any of our perceptions estimate true properties of objective reality. More deeply, it says the very predicates of our perceptions — predicates such as space, time, physical objects, position, momentum, and 3D shape — are the wrong language to describe reality. The problem is not that our perceptual estimates are a tad off here or there and need minor corrections. The problem is that no correction is possible because the language of physical objects in spacetime cannot possibly describe reality as it is. This point is fundamental. Current models of perception — such as Bayesian, inverse optics, ecological optics, and enactive models — disagree on much, but they all agree that perceptual predicates such as space, time and shape are appropriate to describe objective reality. The FBT Theorem says that they are wrong. But how could perception be useful if it does not, and could not, describe objective reality? How could failing to see objective reality grant a competitive advantage? The interface theory of perception (ITP) answers this question; its answer entails radical, and empirically testable, answers to the panoply of questions that opened this section (Fields 2014; Hoffman 1998; 2009; 2011; 2012; 2013; Hoffman and Prakash 2014; Hoffman and Singh 2012; Hoffman et al. 2013; Hoffman et al. 2015a, 2015b; Koenderink 2011; 2013; Mark et al. 2009; Mausfeld 2002; Singh and Hoffman 2013; see also von Uexküll (1909; 1926; 1934) for his related idea of an Umwelt). ITP: AN INFORMAL INTRODUCTION This section describes ITP informally, the next mathematically. ITP says that our perceptions are not a window onto objective reality, but instead they are more like the windows interface of a computer. Suppose you are editing a text file, and the icon for that file is green, rectangular and in the center of the desktop. Does this mean that the text file itself is green, rectangular and in the center of the computer? Of course not. Anyone who thinks so completely misunderstands the purpose !5 of the interface. The shapes, colors and positions of its icons are not meant to depict the real shapes, colors and positions of files in the computer. Indeed, these are the wrong predicates; files have no colors or well- defined shapes. Instead the purpose of the interface and its icons is to hide the real nature and complexity of the computer, and to provide simple tools that allow the user to edit files and photos without the burden of dealing directly with transistors, voltages, magnetic fields and megabytes of software. Good luck trying to craft an email by directly manipulating voltages in a computer. Fortunately, the interface lets you manipulate them without knowing anything about them, so you can easily write and send that email. According to ITP, space-time as we perceive it is our species-specific desktop, and physical objects as we perceive them are species-specific icons in that desktop. Our perceptions of space-time and physical objects are not an insight into objective reality. Instead, they are a species-specific interface that hides objective reality and guides adaptive behaviors. Perception is not about seeing truth, it’s about having kids. The stakes are high. If ITP is right, then space-time is doomed. Not only are our perceptions in terms of space and time not an insight into objective reality, but even more importantly the very predicates of space and time are almost surely the wrong predicates to describe objective reality. Thus ITP makes a prediction that goes to the heart not just of theories of perception, but also of theoretical physics. ITP predicts that physicists will find that space-time is doomed, that the language of space-time is the wrong language in which to formulate the deepest theories of physics.

Load more