CHAPTER 2 Audition JOSH H. MCDERMOTT INTRODUCTION challenges of everyday audition are reflected in the gap between biological and machine Sound is created when matter in the world hearing systems—machine systems for vibrates and takes the form of pressure waves interpreting sound currently fall short of that propagate through the air, containing human abilities. At present, smart phones clues about the environment around us. and other machine systems recognize speech Audition is the process by which organisms reasonably well in quiet conditions, but in utilize these clues to derive information about a noisy restaurant they are all but useless. the world. Understanding the basis of our success in Audition is a crucial sense for most ani- perceiving sound will hopefully help us to mals. Humans use sound to infer a vast replicate it in machine systems and restore number of important things—what some- it in biological auditory systems when their one said, their emotional state when they function becomes impaired. said it, and the whereabouts and nature of The goal of this chapter is to provide a objects we cannot see, to name but a few. bird’s-eye view of contemporary hearing When hearing is impaired (via congenital research. The chapter is an updated version conditions, noise exposure, or aging), the of one I wrote a few years ago (McDermott, consequences can be devastating, such that 2013). I provide brief overviews of classic a large industry is devoted to the design of areas of research as well as some central prosthetic hearing devices. themes and advances from the past 10 years. As listeners, we are largely unaware of the The first section describes the sensory trans- computations underlying our auditory sys- duction of the cochlea. The second section tem’s success, but they represent an impres- discusses modulation and its measurement by sive feat of engineering. The computational subcortical and cortical regions of the audi- tory system, a key research focus of the last This chapter is an updated version of a chapter written for The Oxford Handbook of Cognitive Neuroscience. few decades. The third and fourth sections I thank Dana Boebinger, Alex Kell, Wiktor Mlynarski, describe some of what is known about the and Kevin Woods for helpful comments on earlier drafts primary and nonprimary auditory cortex, of this chapter. Supported by a McDonnell Scholar Award and a National Science Foundation CAREER respectively. The fifth section discusses Award. the perception of sound source properties, Stevens’ Handbook of Experimental Psychology and Cognitive Neuroscience, Fourth Edition, edited by John T. Wixted. Copyright © 2018 John Wiley & Sons, Inc. DOI: 10.1002/9781119170174.epcn202 1 2 Audition focusing on location, loudness, and pitch. The “auditory scene analysis” (Bregman, 1990), sixth section presents an overview of auditory this problem has analogs in other sensory scene analysis. I conclude with a discussion modalities, but the nature of sound presents of where hearing research is headed. the auditory system with unique challenges. THE PROBLEM SOUND MEASUREMENT—THE PERIPHERAL AUDITORY SYSTEM Just by listening, we can routinely apprehend many aspects of the world around us: the The transformation of the raw acoustic input size of a room in which we are talking, into representations that are useful for behav- whether it is windy or raining outside, the ior is apparently instantiated over many brain speed of an approaching car, or whether the areas and stages of neural processing, span- surface someone is walking on is gravel or ning the cochlea, midbrain, thalamus, and marble. This ability is nontrivial because the cortex (Figure 2.1). The early stages of this properties of the world that are of interest cascade are particularly intricate in the audi- to a listener are generally not explicit in tory system relative to other sensory systems, the acoustic input—they cannot be easily with many processing stations occurring recognized or discriminated from the sound prior to the cortex. The sensory organ of the waveform itself. The brain must process the cochlea is itself a complex multicomponent sound entering the ear to generate represen- system, whose investigation remains a con- tations in which the properties of interest are siderable challenge—the mechanical nature more evident. One of the main objectives of of the cochlea renders it much more difficult hearing science is to understand the nature to probe (e.g., with electrodes) than the retina of this transformation and its instantiation in or olfactory epithelium, for instance. Periph- the brain. eral coding of sound is also unusual relative Like other senses, audition is further to that of other senses in its degree of clinical complicated by a second challenge—that of relevance. Unlike vision, for which the most scene analysis. Although listeners are gener- common forms of dysfunction are optical in ally interested in the properties of individual nature, and can be fixed with glasses, hearing objects or events, the ears are rarely pre- impairment typically involves altered periph- sented with the sounds from isolated sources. eral neural processing, and its treatment has Instead, the sound signal that reaches the ear benefited from a detailed understanding of is typically a mixture of sounds from differ- the processes that are altered. Much of hear- ent sources. Such mixtures of sound sources ing research has accordingly been devoted occur frequently in natural auditory environ- to understanding the nature of the measure- ments, for example in social settings, where ments made by the auditory periphery, and a single speaker of interest may be talking they provide a natural starting point for any among many others, and in music. From the discussion of how we hear. mixture it receives as input, the brain must derive representations of the individual sound Frequency Selectivity and the Cochlea sources of interest, as are needed to under- stand someone’s speech, recognize a melody, Hearing begins with the ear, where sound or otherwise guide behavior. Known as the is transduced into action potentials that are “cocktail party problem” (Cherry, 1953), or sent to the brain via the auditory nerve. Sound Measurement—The Peripheral Auditory System 3 Auditory Medial cortex geniculate nucleus Inferior colliculus Left cochlea Superior olivary nucleus Auditory nerve Cochlear nucleus Figure 2.1 The auditory system. Sound is transduced by the cochlea, processed by an interconnected set of subcortical areas, and then fed into the core regions of auditory cortex. Source: From Goldstein (2007). © 2007 South-Western, a part of Cengage, Inc. Reproduced with permission. www.cengage/com/permissions The transduction process is marked by named for the stereocilia that protrude from several distinctive signal transformations, them. The inner hair cells are responsible the most obvious of which is produced by for sound transduction. When the section of frequency tuning. membrane on which they lie vibrates, stere- The key components of sound transduc- ocilia shear against the membrane above, tion are depicted in Figure 2.2. Sound induces opening mechanically gated ion channels and vibrations of the eardrum, which are then inducing a voltage change within the body of transmitted via the bones of the middle ear to the cell. Neurotransmitter release is triggered the cochlea, the sensory organ of the auditory by the change in membrane potential, gener- system. The cochlea is a coiled, fluid-filled ating action potentials in the auditory nerve tube. Several membranes extend through fibers that the hair cell synapses with. This the tube and vibrate in response to sound. electrical signal is carried by the auditory Transduction of this mechanical vibration nerve fibers to the brain. into an electrical signal occurs in the organ of The frequency tuning of the transduction Corti, a mass of cells attached to the basilar process occurs because different parts of membrane. The organ of Corti in particu- the basilar membrane vibrate maximally in lar contains what are known as hair cells, response to different frequencies. This is 4 Audition Cross section of cochlea Cochlea Auditory Reissner’s membrane nerve Vestibular Tectorial membrane nerve Auditory Middle nerve canal Vestibular canal Inner Oval hair cells window Spiral Round ganglion window Cochlea Tympanic canal Outer hair cells Tectorial Basilar Stereocilia Organ membrane membrane of Corti Outer Basilar Tunnel Efferent hair cells membrane Afferent Inner of Corti fibers Auditory nerve fibers hair cells Figure 2.2 Structure of the peripheral auditory system. (Top left) Diagram of ear. The eardrum trans- mits sound to the cochlea via the middle ear bones (ossicles). (Top middle) Inner ear. The semicircular canals abut the cochlea. Sound enters the cochlea via the oval window and causes vibrations along the basilar membrane, which runs through the middle of the cochlea. (Top right) Cross section of cochlea. The organ of Corti, containing the hair cells that transduce sound into electrical potentials, sits on top of the basilar membrane. (Bottom) Schematic of section of the organ of Corti. The shearing that occurs between the basilar and tectorial membranes when they vibrate (in response to sound) causes the hair cell stereocilia to deform. The deformation causes a change in the membrane potential of the inner hair cells, transmitted to the brain via afferent auditory nerve fibers. The outer hair cells, which are 3 times more numerous than the inner hair cells, serve as a feedback system to alter the basilar membrane motion, tightening its tuning and amplifying the response to low-amplitude sounds. Source: From Wolfe (2006, Chapter 9). Reproduced with permission of Oxford University Press. partly due to mechanical resonances—the rather than transduce it. They expand and thickness and stiffness of the membrane contract in response to sound (Ashmore, vary along its length, producing a differ- 2008; Dallos, 2008; Hudspeth, 2008). Their ent resonant frequency at each point.