DOCSLIB.ORG

Toward Collaboration Between the Fields of Signal and Sentence Processing

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Toward Collaboration Between the Fields of Signal and Sentence Processing Cortical Tracking of Speech: Toward Collaboration between the Fields of Signal and Sentence Processing Eleonora J. Beier1, Suphasiree Chantavarin1,2, Gwendolyn Rehrig1, Fernanda Ferreira1, and Lee M. Miller1 Abstract ■ In recent years, a growing number of studies have used corti- between the two fields could be mutually beneficial. Specifically, cal tracking methods to investigate auditory language processing. we argue that the use of cortical tracking methods may help re- Although most studies that employ cortical tracking stem from solve long-standing debates in the field of sentence processing that the field of auditory signal processing, this approach should also commonly used behavioral and neural measures (e.g., ERPs) have be of interest to psycholinguistics—particularly the subfield of failed to adjudicate. Similarly, signal processing researchers who sentence processing—given its potential to provide insight into usecorticaltrackingmaybeabletoreducenoiseintheneuraldata dynamic language comprehension processes. However, there and broaden the impact of their results by controlling for linguis- has been limited collaboration between these fields, which we tic features of their stimuli and by using simple comprehension suggest is partly because of differences in theoretical background tasks. Overall, we argue that a balance between the methodolog- and methodological constraints, some mutually exclusive. In this ical constraints of the two fields will lead to an overall improved paper, we first review the theories and methodological con- understanding of language processing as well as greater clarity on straints that have historically been prioritized in each field and what mechanisms cortical tracking of speech reflects. Increased provide concrete examples of how some of these constraints collaboration will help resolve debates in both fields and will lead may be reconciled. We then elaborate on how further collaboration to new and exciting avenues for research. ■ INTRODUCTION 2019; see later sections for further discussion of terminol- Recent years have seen a growing interest in the cortical ogy and debates in this field). It has been proposed that tracking of speech as a potential measure of acoustic, lin- entrainment may contribute to improved speech pro- guistic, and cognitive processing (Meyer, Sun, & Martin, cessing and language comprehension—for instance, by 2020; Obleser & Kayser, 2019; Kösem & van Wassenhove, instantiating temporal predictions that enable segmenta- 2017; Meyer, 2018; see Tyler, 2020). The terms “cortical tion of the continuous speech signal into units at several tracking” or “speech tracking” loosely refer to continuous timescales (Keitel, Gross, & Kayser, 2018; Kösem et al., neural activity that is somehow time-locked to ongoing 2018; Meyer & Gumbert, 2018; Meyer et al., 2017; Ding events in the speech signal. According to one common et al., 2016; Zoefel & VanRullen, 2015; Doelling et al., interpretation, cortical tracking reflects the tendency for 2014; Peelle et al., 2013; Giraud & Poeppel, 2012; neural oscillations to align, or phase-lock, with quasiperi- Peelle & Davis, 2012; Ahissar et al., 2001). odic features in the speech signal. These quasiperiodic Cortical tracking methods should therefore be of great elements of speech can be acoustic, such as the fluctua- interest to researchers studying sentence processing tions in the amplitude envelope associated with syllables from a psycholinguistic perspective. Sentence processing (Doelling, Arnal, Ghitza, & Poeppel, 2014; Peelle, Gross, research makes frequent use of ERPs to draw inferences & Davis, 2013) or linguistic representations generated in about neural responses to isolated, discrete events such the mind of the listener, such as syntactic phrase bound- as word onsets or sentence boundaries (Swaab, Ledoux, aries (Meyer, Henry, Gaston, Schmuck, & Friederici, Camblin, & Boudewyn, 2012; Kutas & Federmeier, 2011; 2017; Ding, Melloni, Zhang, Tian, & Poeppel, 2016). Kutas, Van Petten, & Kluender, 2006) as well as time– Researchers adopting this approach often refer to cortical frequency analyses of EEG oscillatory power at specific tracking as “neural entrainment” (Obleser & Kayser, bands (Prystauka & Lewis, 2019). However, the field of sentence processing has yet to fully incorporate cortical 1University of California, Davis, 2Chulalongkorn University, tracking as a tool to investigate language processing Bangkok, Thailand mechanisms continuously, rather than at discrete epochs. © 2021 Massachusetts Institute of Technology Journal of Cognitive Neuroscience 33:4, pp. 574–593 https://doi.org/10.1162/jocn_a_01676 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/jocn_a_01676 by guest on 26 September 2021 As we will argue, combining measures of cortical tracking about linguistic processing by using more of the contin- with typical psycholinguistic paradigms may help resolve uous EEG data through cortical tracking methods, signal long-standing debates and distinguish between compet- processing research would gain a better understanding of ing theories of language processing, while also making the role of cortical tracking in the processing of speech use of continuous EEG data that are typically treated as by controlling and manipulating the linguistic features of noise in ERP paradigms (a point we discuss further in the stimulus, which are often underspecified in current the section titled Contributions to Psycholinguistics). studies. In general, there has been limited collaboration be- In this paper, we first review the theoretical back- tween signal processing neuroscientists who use cortical ground and the methodological constraints that have his- tracking paradigms and psycholinguists, despite the fact torically been prioritized in the study of sentence that both fields share the common goal of elucidating processing and the neuroscience of auditory signal pro- how listeners process spoken language. To be more spe- cessing. We then explore in more detail what further col- cific, the research areas that are most relevant for our laboration between these two fields could bring, purposes are the study of human sentence processing, highlighting how cortical tracking methods could be which is a subfield of psycholinguistics, and the study used to improve our understanding of continuous sen- of auditory signal processing, which is often carried out tence comprehension and how paradigms from psycho- by perceptual neurophysiologists and engineers who are linguistics may in turn improve our understanding of the interested in how the brain transforms auditory signals transformations listeners apply to speech as an input sig- and who may implement cortical tracking in their re- nal. Finally, we will provide concrete ideas for how to rec- search methods. These research topics intersect when oncile each field’s constraints to reach these goals. We the auditory signal comprises spoken sentences. For con- conclude by arguing that, although no perfect experi- venience, we will refer to these research areas as “sen- ment can be constructed that will satisfy all constraints, tence processing” and “signal processing” throughout, a better mutual understanding of each field’sapproach with the caveat that the fields are not mutually exclusive; will greatly improve experimental designs in both fields there are psycholinguists who employ cortical tracking and open up new exciting avenues for research. methods to study sentence processing (e.g., Song & Iverson, 2018; Martin & Doumas, 2017; Meyer et al., 2017), signal processing researchers who are interested OVERVIEW OF RESEARCH ON SIGNAL AND in the linguistic properties of the speech signal (e.g., SENTENCE PROCESSING Ding et al., 2016), and investigators with clear interest in both fields who already conduct studies that incorpo- Cortical Tracking in Signal Processing Research rate the methodological compromises we suggest later Cortical tracking refers to the observed alignment of on (e.g., Giraud & Poeppel, 2012; Peelle & Davis, 2012; rhythmic neural activity with an external periodic or qua- Obleser & Kotz, 2011). siperiodic stimulus. It has been observed in response to Despite these notable examples of interaction, the two both visual and auditory stimuli (Besle et al., 2011; fields remain largely independent. Part of the reason for Gomez-Ramirez et al., 2011; Luo, Liu, & Poeppel, 2010; this limited collaboration stems from the different ways Lakatos, Karmos, Mehta, Ulbert, & Schroeder, 2008; these two fields conceptualize and define language pro- Lakatos et al., 2005). Although this phenomenon has re- cessing; whereas sentence processing research focuses ceived growing interest in the past decade, there is still on the cognitive and linguistic representations formed considerable debate as to the neural causes of cortical during comprehension, signal processing research treats tracking and the role it may play in various aspects of cog- speech as an example of a complex auditory signal and nition (e.g., attention, temporal prediction, speech pro- seeks to characterize the system that transforms that in- cessing). Specifically, debate has centered around put signal into an output signal or response (e.g., whether cortical tracking results from the phase-locking Rimmele, Morillon, Poeppel, & Arnal, 2018; Morillon & of ongoing, endogenous oscillations (Calderone, Lakatos, Schroeder, 2015). An additional hurdle to collaboration Butler, & Castellanos, 2014; Doelling et al., 2014; Giraud stems
Load more

Recommended publications
  • The ERP Technique in Children's Sentence Processing: a Review a Técnica De ERP No Processamento De Sentenças De Crianças: U
    Revista de Estudos da Linguagem, Belo Horizonte, v.25, n.3, p.1537-1566, 2017 The ERP technique in children’s sentence processing: a review A técnica de ERP no processamento de sentenças de crianças: uma revisão Marília Uchôa Cavalcanti Lott de Moraes Costa Universidade Federal do Rio de Janeiro, Rio de Janeiro, Rio de Janeiro / Brasil [email protected] Resumo: A medição da ativação cerebral por meio da técnica de potenciais relacionados a eventos (ERP) tem sido valiosa para lançar luz sobre diversas cognições humanas. A linguagem é uma das cognições que têm sido estudadas com essa técnica de grande resolução temporal entre o estímulo apresentado e a ativação observada decorrente desse estímulo. A área de aquisição da linguagem tem se beneficiado especialmente dessa técnica, dado que é possível investigar relações entre dados linguísticos e a ativação cerebral sem a necessidade de uma resposta explícita, como apertar um botão ou apontar para uma imagem. O objetivo deste artigo é apresentar o estado da arte sobre o processamento de frases em crianças utilizando a técnica de ERP. Palavras-chave: ERP/EEG; processamento de frases; aquisição da linguagem; N400; P600. Abstract: The measurement of brain activation through the technique of event related potentials (ERP) has been valuable in shedding light on various human cognitions. Language is one of those cognitions that has been studied using this technique, which allows for a more accurate temporal resolution between the stimulus presented and the time in which we observe an activation resulting from this stimulus. The area of eISSN: 2237-2083 DOI: 10.17851/2237-2083.25.3.1537-1566 1538 Revista de Estudos da Linguagem, Belo Horizonte, v.25, n.3, p.1537-1566, 2017 language acquisition has especially benefited from this technique since it is possible to investigate relationships between linguistic data and brain activation without the need for an explicit response.
    [Show full text]
  • The Potentials for Basic Sentence Processing: Differentiating
    The Potentials for Basic Sentence 20 Processing: Differentiating Integrative Processes Marta Kutas and Jonathan W. King ABSTRACT We show that analyzing voltage fluctuations known as "event-related brain potentials," or ERPs, recorded from the human scalp can be an effective way of tracking integrative processes in language on-line. This is essential if we are to choose among alternative psychological accounts of language comprehension. We briefly review the data implicating the N400 as an index of semantic integration and describe its use in psycholinguistic research. We then introduce a cognitive neuroscience approach to normal sentence processing, which capitalizes on the ERP's fine temporal resolution as well as its potential linkage to both psychological constructs and activated brain areas. We conclude by describing several reliable ERP effects with different temporal courses, spatial extents, and hypothesized relations to comprehension skill during the reading of simple transitive sentences; these include (1) occipital potentials related to fairly low-level, early visual processing, (2) very slow frontal positive shifts related to high-level integration during construction of a mental model, and (3) various frontotemporal potentials associated with thematic role assignment, clause endings, and manipulating items that are in working memories. In it comes out it goes and in between nobody knows how flashes of vision and snippets of sound get bound to meaning. From percepts to concepts seemingly effortless integration of highly segregated
    [Show full text]
  • Psych150/ Ling155 Spring 2015 Review Questions: Speech Perception, Word Recognition, Sentence Processing, and Speech Production
    Psych150/ Ling155 Spring 2015 Review Questions: Speech perception, word recognition, sentence processing, and speech production Terms/concepts to know: mondegreen, lack of invariance problem, Motor Theory of Speech Perception, McGurk effect, Ganong effect, priming, lexical decision, facilitation, inhibition, neighborhood density, homophone, homograph, Cohort model, TRACE model, Dual Route model, graphemes, developmental dyslexia, acquired dyslexia, parsing, incrementality, garden path sentence, global ambiguity, garden path theory, constraint-based approach, thematic relation, intransitive, transitive, ditransitive, sentential complement, slip of the tongue, tip of the tongue state, serial model, cascaded model, bidirectional cascaded model 1. The lack of invariance problem led to the development of the Motor Theory of Speech Perception (MToSP). What is the lack of invariance problem? How did the MToSP propose to solve that problem? 2. How does the McGurk effect differ from the Ganong effect? 3. Can priming occur without conscious awareness of the prime? What paradigm do researchers use to assess this possibility? 4. How does a lexical decision task work? Can lexical decision be used for both visual (written) and auditory (spoken) experiments? What is the dependent measure in lexical decision tasks? 5. Priming can cause facilitation or inhibition. How would these processes be reflected in reaction times? 6. How do homophones and homographs differ? 7. According to the Cohort model of auditory word recognition, what happens as the beginning of a word is detected? 8. What does the TRACE model predict about possibility of “speaker” and “beaker” being activated at the same time during word recognition? Why? 9. Writing systems differ in terms of the primary level of linguistic representations that they tap into, i.e., that symbols refer to (phonemes, syllables, morphemes).
    [Show full text]
  • A Neurolinguistic Model of Grammatical Construction Processing
    A Neurolinguistic Model of Grammatical Construction Processing Peter Ford Dominey1, Michel Hoen1, and Toshio Inui2 Downloaded from http://mitprc.silverchair.com/jocn/article-pdf/18/12/2088/1756017/jocn.2006.18.12.2088.pdf by guest on 18 May 2021 Abstract & One of the functions of everyday human language is to com- construction processing based on principles of psycholinguis- municate meaning. Thus, when one hears or reads the sen- tics, (2) develop a model of how these functions can be imple- tence, ‘‘John gave a book to Mary,’’ some aspect of an event mented in human neurophysiology, and then (3) demonstrate concerning the transfer of possession of a book from John to the feasibility of the resulting model in processing languages of Mary is (hopefully) transmitted. One theoretical approach to typologically diverse natures, that is, English, French, and Japa- language referred to as construction grammar emphasizes this nese. In this context, particular interest will be directed toward link between sentence structure and meaning in the form of the processing of novel compositional structure of relative grammatical constructions. The objective of the current re- phrases. The simulation results are discussed in the context of search is to (1) outline a functional description of grammatical recent neurophysiological studies of language processing. & INTRODUCTION ditransitive (e.g., Mary gave my mom a new recipe) One of the long-term quests of cognitive neuroscience constructions (Goldberg, 1995). has been to link functional aspects of language process- In this context, the ‘‘usage-based’’ perspective holds ing to its underlying neurophysiology, that is, to under- that the infant begins language acquisition by learning stand how neural mechanisms allow the mapping of the very simple constructions in a progressive development surface structure of a sentence onto a conceptual repre- of processing complexity, with a substantial amount of sentation of its meaning.
    [Show full text]
  • EEG Theta and Gamma Responses to Semantic Violations in Online Sentence Processing
    Brain and Language 96 (2006) 90–105 www.elsevier.com/locate/b&l EEG theta and gamma responses to semantic violations in online sentence processing Lea A. Hald a,c, Marcel C.M. Bastiaansen a,b,¤, Peter Hagoort a,b a Max Planck Institute for Psycholinguistics, P.O. Box 310, 6500 AH Nijmegen, The Netherlands b F. C. Donders Centre for Cognitive Neuroimaging, Radbout Universiteit Nijmegen, P.O. Box 9101, 6500 HB Nijmegen, The Netherlands c Center for Research in Language, University of California, San Diego, 9500 Gilman Dr., Dept. 0526, La Jolla, CA 92093-0526, USA Accepted 18 June 2005 Available online 3 August 2005 Abstract We explore the nature of the oscillatory dynamics in the EEG of subjects reading sentences that contain a semantic violation. More speciWcally, we examine whether increases in theta (t3–7 Hz) and gamma (around 40 Hz) band power occur in response to sen- tences that were either semantically correct or contained a semantically incongruent word (semantic violation). ERP results indicated a classical N400 eVect. A wavelet-based time-frequency analysis revealed a theta band power increase during an interval of 300– 800 ms after critical word onset, at temporal electrodes bilaterally for both sentence conditions, and over midfrontal areas for the semantic violations only. In the gamma frequency band, a predominantly frontal power increase was observed during the processing of correct sentences. This eVect was absent following semantic violations. These results provide a characterization of the oscillatory brain dynamics, and notably of both theta and gamma oscillations, that occur during language comprehension. 2005 Elsevier Inc.
    [Show full text]
  • Sentence Processing What Makes up Psycholinguistics?
    Introduction to Psycholinguistics Lecture 3: Sentence Processing Matthew W Crocker Computerlinguistik Universität des Saarlandes What makes up Psycholinguistics? Computational models of the representations, architectures and mechanisms that underlie human language processing. Linguistics: Wow people represent linguistic knowledge Psycholinguistics: How people use this knowledge to produce and understand language Computational Models: Implementations of psycholinguistic theory, using linguistic representations. More complete theory: models and predicts human behaviour Experiments & Data: Descriptive studies of human language processing Off-line: grammaticality judgement, completions, global reading time (?) On-line: “word-by-word”, self-paced reading, eye-tracking, ERP Test prediction of theories and computer models Corpus data: word-frequency, category and sense bias, subcategorization © Matthew W. Crocker Introduction to Psycholinguistics 2 1 Sentence Processing Sentence processing is the means by which the words of an utterance are combined to yield and interpretation All people do it well It is a difficult task: complexity and ambiguity Not simple ‘retrieval’, like lexical access Compositional: interpretation must be built, rapidly, even for novel word/structure input What are the architectures, mechanisms and representations underlying this process? Architectures: modularity vs. interaction Mechanisms: how is input mapped to interpretations using knowledge Representations: How is knowledge encoded © Matthew W. Crocker Introduction to Psycholinguistics 3 A Simple Theory of Grammar The Grammar The Lexicon S NP VP Det = {the, a, every} NP PN N = {man, woman, book, NP Det N hill, telescope} NP NP PP PN = {John, Mary} PP P NP P = {on, with} VP V V = {saw, put, open, VP V NP read, reads} VP V NP PP © Matthew W. Crocker Introduction to Psycholinguistics 4 2 A Generated Sentence the man read every book S ei S NP VP NP VP ty ru NP Det N VP V NP Det N V NP g g g tu NP Det N the man read Det N g g every book © Matthew W.
    [Show full text]
  • Recovery from Misinterpretations During Online Sentence Processing
    RECOVERY FROM MISINTERPRETATIONS 1 Recovery from misinterpretations during online sentence processing Lena M Blott1a, Jennifer M Rodd1, Fernanda Ferreira2, and Jane E Warren1 1 Division of Psychology and Language Sciences, University College London, UK 2 Department of Psychology, University of California, Davis, CA, USA a Corresponding author: Lena M Blott Chandler House 2 Wakefield Street London WC1N 1PF United Kingdom Email: [email protected] RECOVERY FROM MISINTERPRETATIONS 2 Abstract Misinterpretations during language comprehension are common. The ability to recover from such processing difficulties is therefore crucial for successful day-to-day communication. The present study investigated the outcome of comprehension processes and on-line reading behaviour when misinterpretations occurred. Although group-level effects of reinterpretation on sentence comprehension and on-line processing are of great theoretical interest, individual differences in the recovery from processing difficulty are of particular practical relevance. Even adult readers vary considerably in their “lexical expertise”, their knowledge of word forms and meanings and their experience with written material. We therefore also investigated the effect of individual differences in lexical expertise on processes related to the recovery from misinterpretations. Ninety-six adult participants read “garden-path” sentences in which an ambiguous word was disambiguated towards an unexpected meaning (e.g. The ball was crowded), while their eye movements were monitored. A Meaning Coherence Judgement task additionally required them to decide whether or not each sentence made sense. Results suggested that readers did not always engage in reinterpretation processes but instead followed a “good enough” processing strategy. Successful detection of a violation to sentence coherence and associated reinterpretation processes also required additional processing time compared to sentences that did not induce a misinterpretation.
    [Show full text]
  • Van Assche, Drieghe, Duyck, Welvaert, & Hartsuiker
    Bilingual lexical access in sentence context 1 Running head: BILINGUAL LEXICAL ACCESS IN SENTENCE CONTEXT The influence of semantic constraints on bilingual word recognition during sentence reading Eva Van Assche1, Denis Drieghe1,2, Wouter Duyck1, Marijke Welvaert1 and Robert J. Hartsuiker1 1Ghent University, Belgium 2University of Southampton, UK Address for correspondence: Eva Van Assche Department of Experimental Psychology Ghent University Henri Dunantlaan 2 B-9000 Ghent (Belgium) E-mail: [email protected] Tel: 32-9-264-64-05 - Fax: 32-9-264-64-96 Bilingual lexical access in sentence context 2 Abstract The present study investigates how semantic constraint of a sentence context modulates language-nonselective activation in bilingual visual word recognition. We recorded Dutch-English bilinguals’ eye movements while they read cognates and controls in low and high semantically constraining sentences in their second language. Early and late eye movement measures yielded cognate facilitation, both for low- and high-constraint sentences. Facilitation increased gradually as a function of cross-lingual overlap between translation equivalents. A control experiment showed that the same stimuli did not yield cognate effects in English monolingual controls, ensuring that these effects were not due to any uncontrolled stimulus characteristics. The present study supports models of bilingual word recognition with a limited role for top-down influences of semantic constraints on lexical access in both early and later stages of bilingual word recognition. Keywords: bilingualism, visual word recognition, sentence processing, semantic constraint, reading, cognate effect Bilingual lexical access in sentence context 3 Ever since it was estimated that about half of the world’s population is bilingual (Grosjean, 1982), research on bilingualism has attracted the attention of a growing community of researchers.
    [Show full text]
  • Sentence Processing in a Second Language: Ambiguity Resolution in German Learners of English
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Kölner UniversitätsPublikationsServer Sentence Processing in a Second Language: Ambiguity Resolution in German Learners of English Inauguraldissertation zur Erlangung des Doktorgrades an der Philosophischen Fakultät der Universität zu Köln vorgelegt von Anne Rah aus Aurich am 06.04.2009 Gutachter: Prof. Dr. Dany Adone Prof. Dr. Christiane M. Bongartz Disputation: 01.07.2009 Table of Contents 1 INTRODUCTION ................................................................................................................. 1 2 SENTENCE PROCESSING MODELS ............................................................................... 5 3 SENTENCE PROCESSING IN THE SECOND LANGUAGE ....................................... 21 4 METHODOLOGICAL ISSUES ........................................................................................ 41 4.1 Participants .......................................................................................................................... 41 4.2 Choice of experimental procedures .................................................................................... 44 4.3 Development of materials ................................................................................................... 49 4.4 Evaluation of materials ....................................................................................................... 51 4.5 Data analysis .......................................................................................................................
    [Show full text]
  • Syntactic Ambiguity Resolution in Sentence Processing: New Evidence from a Morphologically Rich Language
    Syntactic Ambiguity Resolution in Sentence Processing: New Evidence from a Morphologically Rich Language Daria Chernova ([email protected]) Laboratory for Conitive Studies, St. Petersburg State University, 190000, 58-60 Galernaya St., St.Petersburg, Russia Tatiana Chernigovskaya ([email protected]) Laboratory for Conitive Studies, St. Petersburg State University, 190000, 58-60 Galernaya St., St.Petersburg, Russia Abstract (1) I met the servant of the lady that was on the balcony. In (1), the relative clause can be attached either high (HA) An experimental study dedicated to structurally ambiguous sentences processing was carried out. We analyzed the case of or low (LA), i.e. to the first (head) noun or to the second participial construction attachment to a complex noun phrase. In (dependent) noun. Based on the first experiments on English Experiment 1, we used self-paced reading technique which indicating LA preference, Frazier and Fodor (1978) enables to measure reading times of each word in a sentence and suggested that this kind of ambiguity is resolved according error rates in the interpretation of the sentences. Error rates in to the Late Closure Principle. This principle states that locally ambiguous sentences reveal high attachment preference incoming lexical items tend to be associated with the phrase – for sentences with low attachment error rates are higher. However, high attached modifiers are processed slower than low or clause currently being processed. However, LA attached ones. In experiment 2, we use eye-tracking technique. preference was not confirmed cross-linguistically, which Early effects (first-pass time) show that high attachment requires prima facie contradicted the very idea of universality of more time to process than low or ambiguous attachment (as Late parsing principles.
    [Show full text]
  • Computational Models of Retrieval Processes in Sentence Processing
    In Press: Trends in Cognitive Sciences Computational models of retrieval processes in sentence processing Shravan Vasishth Department of Linguistics, University of Potsdam, Potsdam, Germany Bruno Nicenboim Department of Linguistics, University of Potsdam, Potsdam, Germany Felix Engelmann Manchester University, UK Frank Burchert Department of Linguistics, University of Potsdam, Potsdam, Germany September 3, 2019 Abstract Sentence comprehension requires that the comprehender work out who did what to whom. This process has been characterized as retrieval from memory. This review summarizes the quantitative predictions and empirical coverage of the two existing computational models of retrieval, and shows how the predictive performance of these two competing models can be tested against a benchmark data-set. We also show how computational modeling can help us better understand sources of variability in both unimpaired and impaired sentence comprehension. Keywords: cue-based retrieval; retrieval interference; comprehension impair- ments in aphasia; individual differences Cue-based retrieval and interference in sentence processing Comprehending a sentence involves an array of cognitive processes, including lexical access of words from memory, incremental structure building, and computing the meaning of the sentence. One important aspect of sentence comprehension is working out who did what to whom. As an example, consider the following sentence: “The worker was surprised that the resident who was living near the dangerous neighbor was complaining about the investigation.” Correspondence: [email protected] RETRIEVAL IN SENTENCE PROCESSING 2 To fully understand this sentence, the reader must work out who was surprised, what they were surprised about, who was doing the complaining, and what they were complaining about. In sentence processing research, working out these connections between the words is often termed dependency completion.
    [Show full text]
  • Neural Activity During Sentence Processing As Reflected in Theta
    YNIMG-13010; No. of pages: 12; 4C: 5, 7 NeuroImage xxx (2016) xxx–xxx Contents lists available at ScienceDirect NeuroImage journal homepage: www.elsevier.com/locate/ynimg Neural activity during sentence processing as reflected in theta, alpha, beta, and gamma oscillations Nietzsche H.L. Lam, Jan-Mathijs Schoffelen ⁎, Julia Uddén, Annika Hultén 1, Peter Hagoort ⁎ Donders Institute for Brain, Cognition and Behaviour, Centre for Cognitive Neuroimaging, Radboud University Nijmegen, Nijmegen, Netherlands Max Planck Institute for Psycholinguistics, Nijmegen, Netherlands article info abstract Article history: We used magnetoencephalography (MEG) to explore the spatiotemporal dynamics of neural oscillations associ- Received 29 July 2015 ated with sentence processing in 102 participants. We quantified changes in oscillatory power as the sentence Accepted 3 March 2016 unfolded, and in response to individual words in the sentence. For words early in a sentence compared to Available online xxxx those late in the same sentence, we observed differences in left temporal and frontal areas, and bilateral frontal and right parietal regions for the theta, alpha, and beta frequency bands. The neural response to words in a sen- Keywords: Oscillations tence differed from the response to words in scrambled sentences in left-lateralized theta, alpha, beta, and MEG gamma. The theta band effects suggest that a sentential context facilitates lexical retrieval, and that this facilita- Sentence processing tion is stronger for words late in the sentence. Effects in the alpha and beta bands may reflect the unification of Lexical retrieval semantic and syntactic information, and are suggestive of easier unification late in a sentence. The gamma oscil- Unification lations are indicative of predicting the upcoming word during sentence processing.
    [Show full text]