Perceptual Learning of Degraded Speech by Minimizing Prediction Error
Total Page:16
File Type:pdf, Size:1020Kb
Perceptual learning of degraded speech by minimizing PNAS PLUS prediction error Ediz Sohoglua,1,2 and Matthew H. Davisa,2 aMedical Research Council Cognition and Brain Sciences Unit, Cambridge CB2 7EF, United Kingdom Edited by Barbara Anne Dosher, University of California, Irvine, CA, and approved January 29, 2016 (received for review November 26, 2015) Human perception is shaped by past experience on multiple time- distinct time courses of their effects suggest that they originate scales. Sudden and dramatic changes in perception occur when prior in different brain mechanisms. Dual-mechanism accounts there- knowledge or expectations match stimulus content. These immediate fore propose that the influence of prior knowledge resides at a effects contrast with the longer-term, more gradual improvements hierarchically late (e.g., decision) stage of processing and attribute that are characteristic of perceptual learning. Despite extensive the effect of learning to offline synaptic changes in earlier-level investigation of these two experience-dependent phenomena, there sensory cortex that take place after sensory stimulation (14, 15). is considerable debate about whether they result from common or However, other work has shown that perceptual learning of de- dissociable neural mechanisms. Here we test single- and dual- graded stimuli is enhanced if accurate prior knowledge is provided before the presentation of degraded or otherwise ambiguous mechanism accounts of experience-dependent changes in perception – using concurrent magnetoencephalographic and EEG recordings of sensory input (6, 14, 16 18). Consistent with this behavioral as- sociation, alternative single-mechanism accounts have proposed neural responses evoked by degraded speech. When speech clarity that a single system containing multiple interacting levels of rep- was enhanced by prior knowledge obtained from matching text, resentation supports the effects of both prior knowledge and we observed reduced neural activity in a peri-auditory region of the perceptual learning (15, 19, 20). According to these single-mech- superior temporal gyrus (STG). Critically, longer-term improvements anism accounts, abstract higher-level representations derived from in the accuracy of speech recognition following perceptual prior knowledge are used to inform and guide earlier, lower-level learning resulted in reduced activity in a nearly identical STG region. sensory processes. These interactions not only modulate immedi- Moreover, short-term neural changes caused by prior knowledge ate perceptual outcomes but also lead to subsequent learning: and longer-term neural changes arising from perceptual learning Early sensory processing is modified to ensure that presentations were correlated across subjects with the magnitude of learning- of similar stimuli are more processed effectively in the future. induced changes in recognition accuracy. These experience-dependent Thus, this account makes two key experimental predictions, effects on neural processing could be dissociated from the neural that (i) prior knowledge and perceptual learning should affect effect of hearing physically clearer speech, which similarly enhanced neural responses in the same brain network and (ii) the effect of perception but increased rather than decreased STG responses. prior knowledge observed online during perception should predict Hence, the observed neural effects of prior knowledge and percep- the magnitude of subsequent perceptual learning. However, be- tual learning cannot be attributed to epiphenomenal changes in cause the brain systems supporting the influences of prior listening effort that accompany enhanced perception. Instead, our knowledge and perceptual learning typically have been observed COGNITIVE SCIENCES PSYCHOLOGICAL AND results support a predictive coding account of speech perception; separately(3,4,21–26), neither of these predictions has been computational simulations show how a single mechanism, mini- tested successfully before. mization of prediction error, can drive immediate perceptual In this study, we obtained concurrent high-density EEG and effects of prior knowledge and longer-term perceptual learning of magnetoencephalographic (MEG) recordings to compare the degraded speech. Significance perceptual learning | predictive coding | speech perception | magnetoencephalography | vocoded speech Experience-dependent changes in sensory processing are critical for successful perception in dynamic and noisy environments. How- uccessful perception in a dynamic and noisy environment ever, the neural and computational mechanisms supporting such Scritically depends on the brain’s capacity to change how sen- changes have remained elusive. Using electrical and magnetic re- sory input is processed based on past experience. Consider the way cordings of human brain activity, we demonstrate that two dif- in which perception is enhanced by accurate prior knowledge or ferent sources of experience-dependent improvement in perception expectations. Sudden and dramatic changes in subjective experi- of degraded speech—the immediate effects of prior knowledge — ence can occur when a distorted and otherwise unrecognizable and longer-term changes resulting from perceptual learning arise perceptual object is seen or heard after the object’s identity is from common neural machinery in the sensory cortex although revealed (1–4). Such effects occur almost immediately; striking they operate over dramatically different behavioral timescales. Our changes in perceptual outcomes occur over a timescale of seconds findings support a specific neuro-computational account of per- or less. However, not all effects of past experience emerge as ception in which a single mechanism, minimization of prediction rapidly as these effects of prior knowledge. With perceptual error, drives both the immediate perceptual effects of prior learning, practice in perceiving certain types of stimuli results in knowledge and longer-term perceptual learning. gradual and incremental improvements in perception that develop A Author contributions: E.S. and M.H.D. designed research; E.S. and M.H.D. performed re- over a timescale of minutes or longer (Fig. 1 ) (5, 6, 7). Critically, search; E.S. analyzed data; and E.S. and M.H.D. wrote the paper. perceptual learning can generalize beyond the stimuli experienced The authors declare no conflict of interest. during training, e.g., to visual forms presented in different retinal This article is a PNAS Direct Submission. positions or orientations or spoken words that have not been heard 1 – Present address: University College London Ear Institute, London WC1X 8EE, United before (6, 8 10). Thus, perceptual learning may have great potential Kingdom. – in ameliorating sensory deficits (11 13), and understanding the 2To whom correspondence may be addressed. Email: [email protected] or matt.davis@ neural and computational mechanisms supporting learning is critical. mrc-cbu.cam.ac.uk. Although prior knowledge and perceptual learning are both This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10. experience-dependent forms of perceptual improvement, the 1073/pnas.1523266113/-/DCSupplemental. www.pnas.org/cgi/doi/10.1073/pnas.1523266113 PNAS | Published online March 8, 2016 | E1747–E1756 Downloaded by guest on September 30, 2021 A Prior knowledge vs. Perceptual learning Prior knowledge Perceptual learning song moon clay clay haze bead cheese High Low 12ch Perceptual clarity Short timescale (Seconds) Long timescale (Minutes) B Design and stimuli Time (mins) 0 15 25 35 45 60 Train 1 Train 2 Train 3 Pre-test Training Post-test (speech recognition) (clarity rating) (speech recognition) 1050 (±50) ms 1050 (±50) ms (Mismatching) song Say word or 1 2 3 4 Fig. 1. Overview of study design. (A) Illustration of the haze (response cue) (Matching) clay clay (response cue) distinction between the immediate influence of prior (1/6/24 channels) 1050 (±50) ms (3/6/12 channels) knowledge and the more gradual influence of per- Time (ms) Time (ms) ceptual learning. (Left) For mismatching prior knowl- 0 400 800 1200 −1200 −800 −400 0 400 800 1200 edge (provided by written presentation of the word “song” before the presentation of the degraded spo- C Summary of behavioral outcomes ken word “moon”), perceptual clarity is low. However, seconds later, perceptual clarity is enhanced dramati- Sensory detail Prior knowledge Perceptual learning cally if prior knowledge matches speech content. (Right) Perceptual learning results from practice at perceptual tasks (e.g., recognition of degraded speech) leading to gradual improvements in perceptual clarity over a timescale of minutes, hours, or days. (B)Time- Decreased line of the experiment including a timeline of example listening effort Increased clarityIncreased trials for the pretest, training, and posttest phases. or recognition accuracy or recognition Training trials were divided into three blocks (Train Pre 1/Train 2/Train 3). (C) Summary of changes in behavioral Post outcomes resulting from experimental manipulations of Matching sensory detail, prior knowledge, and perceptual learning. Mismatching All three manipulations enhance perceptual clarity or More channels Fewer channels accuracy and decrease listening effort. impact of prior knowledge and perceptual learning on neural re- both when sensory detail increased [F (2, 40) = 295, P < 0.001] sponses to degraded speech (Fig. 1B). Speech is an ideal stimulus and when listeners had prior knowledge