DOCSLIB.ORG

A Model of Discourse Predictions in Human Sentence Processing

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
A Model of Discourse Predictions in Human Sentence Processing A Model of Discourse Predictions in Human Sentence Processing Amit Dubey and Frank Keller and Patrick Sturt Human Communication Research Centre, University of Edinburgh 10 Crichton Street, Edinburgh EH8 9AB, UK {amit.dubey,frank.keller,patrick.sturt}@ed.ac.uk Abstract prediction has been demonstrated by studies that show anticipation based on selectional restrictions: This paper introduces a psycholinguistic listeners are able to launch eye-movements to the model of sentence processing which combines predicted argument of a verb before having encoun- a Hidden Markov Model noun phrase chun- tered it, e.g., they will fixate an edible object as soon ker with a co-reference classifier. Both mod- els are fully incremental and generative, giv- as they hear the word eat (Altmann and Kamide, ing probabilities of lexical elements condi- 1999). Semantic prediction has also been shown in tional upon linguistic structure. This allows the context of semantic priming: a word that is pre- us to compute the information theoretic mea- ceded by a semantically related prime or by a seman- sure of surprisal, which is known to correlate tically congruous sentence fragment is processed with human processing effort. We evaluate faster (Stanovich and West, 1981; Clifton et al., our surprisal predictions on the Dundee corpus 2007). An example for syntactic prediction can be of eye-movement data show that our model achieve a better fit with human reading times found in coordinate structures: readers predict that than a syntax-only model which does not have the second conjunct in a coordination will have the access to co-reference information. same syntactic structure as the first conjunct (Fra- zier et al., 2000). In a similar vein, having encoun- tered the word either, readers predict that or and a 1 Introduction conjunct will follow it (Staub and Clifton, 2006). Recent research in psycholinguistics has seen a Again, priming studies corroborate this: Compre- growing interest in the role of prediction in sentence henders are faster at naming words that are syntacti- processing. Prediction refers to the fact that the hu- cally compatible with prior context, even when they man sentence processor is able to anticipate upcom- bear no semantic relationship to it (Wright and Gar- ing material, and that processing is facilitated when rett, 1984). predictions turn out to be correct (evidenced, e.g., Predictive processing is not confined to the sen- by shorter reading times on the predicted word or tence level. Recent experimental results also provide phrase). Prediction is presumably one of the factors evidence for discourse prediction. An example is the that contribute to the efficiency of human language study by van Berkum et al. (2005), who used a con- understanding. Sentence processing is incremental text that made a target noun highly predictable, and (i.e., it proceeds on a word-by-word basis); there- found a mismatch effect in the ERP (event-related fore, it is beneficial if unseen input can be antici- brain potential) when an adjective appeared that was pated and relevant syntactic and semantic structure inconsistent with the target noun. An example is (we constructed in advance. This allows the processor to give translations of their Dutch materials): save time and makes it easier to cope with the con- stant stream of new input. (1) The burglar had no trouble locating the secret Evidence for prediction has been found in a range family safe. of psycholinguistic processing domains. Semantic a. Of course, it was situated behind a 304 Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 304–312, Edinburgh, Scotland, UK, July 27–31, 2011. c 2011 Association for Computational Linguistics bigneu but unobtrusive paintingneu. of prediction, and again only proposes a loose in- b. Of course, it was situated behind a tegration of co-reference and syntax. Furthermore, bigcom but unobtrusive bookcasecom. Dubey’s (2010) model has only been tested on two experimental data sets (pertaining to the interaction Here, the adjective big, which can have neutral or of ambiguity resolution with context), no broad cov- common gender in Dutch, is consistent with the pre- erage evaluation is available. dicted noun painting in (1-a), but inconsistent with it The aim of the present paper is to overcome these in (1-b), leading to a mismatch ERP on big in (1-b) limitations. We propose a computational model that but not in (1-a). captures discourse effects on syntax in terms of pre- Previous results on discourse effects in sentence diction. The model comprises a co-reference com- processing can also be interpreted in terms of pre- ponent which explicitly stores discourse mentions diction. In a classical paper, Altmann and Steed- of NPs, and a syntactic component which adjust man (1988) demonstrated that PP-attachment pref- the probabilities of NPs in the syntactic structure erences can change through discourse context: if the based on the mentions tracked by the discourse com- context contains two potential referents for the tar- ponent. Our model is HMM-based, which makes get NP, then NP-attachment of a subsequent PP is it possible to efficiently process large amounts of preferred (to disambiguate between the two refer- data, allowing an evaluation on eye-tracking cor- ents), while if the context only contains one target pora, which has recently become the gold-standard NP, VP-attachment is preferred (as there is no need in computational psycholinguistics (e.g., Demberg to disambiguate). This result (and a large body of and Keller 2008; Frank 2009; Boston et al. 2008; related findings) is compatible with an interpretation Mitchell et al. 2010). in which the processor predicts upcoming syntactic The paper is structured as follows: In Section 2, attachment based on the presence of referents in the we describe the co-reference and the syntactic mod- preceding discourse. els and evaluate their performance on standard data Most attempts to model prediction in human lan- sets. Section 3 presents an evaluation of the overall guage processing have focused on syntactic pre- model on the Dundee eye-tracking corpus. The pa- diction. Examples include Hale’s (2001) surprisal per closes with a comparison with related work and model, which relates processing effort to the con- a general discussion in Sections 4 and 5. ditional probability of the current word given the previous words in the sentence. This approach has 2 Model been elaborated by Demberg and Keller (2009) in a model that explicitly constructs predicted structure, This model utilises an NP chunker based upon a hid- and includes a verification process that incurs ad- den Markov model (HMM) as an approximation to ditional processing cost if predictions are not met. syntax. Using a simple model such as an HMM fa- Recent work has attempted to integrate semantic cilitates the integration of a co-reference component, and discourse prediction with models of syntactic and the fact that the model is generative is a prereq- processing. This includes Mitchell et al.’s (2010) uisite to using surprisal as our metric of interest (as approach, which combines an incremental parser surprisal require the computation of prefix probabil- with a vector-space model of semantics. However, ities). The key insight in our model is that human this approach only provides a loose integration of sentence processing is, on average, facilitated when the two components (through simple addition of a previously-mentioned discourse entity is repeated. their probabilities), and the notion of semantics used This facilitation depends upon keeping track of a list is restricted to lexical meaning approximated by of previously-mentioned entities, which requires (at word co-occurrences. At the discourse level, Dubey the least) shallow syntactic information, yet the fa- (2010) has proposed a model that combines an incre- cilitation itself is modeled primarily as a lexical phe- mental parser with a probabilistic logic-based model nomenon. This allows a straightforward separation of co-reference resolution. However, this model of concerns: shallow syntax is captured using the does not explicitly model discourse effects in terms HMM’s hidden states, whereas the co-reference fa- 305 cilitation is modeled using the HMM’s emissions. However, using Bayes’ theorem we can compute: The vocabulary of hidden states is described in Sec- P(tag word)P(word) tion 2.1 and the emission distribution in Section 2.2 P(word tag) = | (1) | P(tag) 2.1 Syntactic Model which is what we need for surprisal. The pri- A key feature of the co-reference component of our mary information from this probability comes from model (described below) is that syntactic analysis P(tag word), however, reasonable estimates of | and co-reference resolution happen simultaneously. P(tag) and P(word) are required to ensure the prob- This could potentially slow down the syntactic anal- ability distribution is proper. P(tag) may be esti- ysis, which tends to already be quite slow for ex- mated on a parsed treebank. P(word), the probabil- haustive surprisal-based incremental parsers. There- ity of a particular unseen word, is difficult to esti- fore, rather than using full parsing, we use an HMM- mate directly. Given that our training data contains based NP chunker which allows for a fast analysis. approximately 106 words, we assume that this prob- 6 NP chunking is sufficient to extract NP discourse ability must be bounded above by 10− . As an ap- mentions and, as we show below, surprisal values proximation, we use this upper bound as the proba- computed using HMM chunks provide a useful fit bility of P(word). on the Dundee eye-movement data. Training The chunker is trained on sections 2– To allow the HMM to handle possessive construc- 22 of the Wall Street Journal section of the Penn tions as well as NP with simple modifiers and com- Treebank.
Load more

Recommended publications
  • Eye-Movement Research: an Overview of Current and Past Developments
    Elsevier AMS Ch01-I044980 Job code: EMAW 14-2-2007 1:04p.m. Page:1 Trimsize:165×240MM Chapter 1 EYE-MOVEMENT RESEARCH: AN OVERVIEW OF CURRENT AND PAST DEVELOPMENTS ROGER P. G. VAN GOMPEL, MARTIN H. FISCHER AND WAYNE S. MURRAY University of Dundee, UK ROBIN L. HILL University of Edinburgh, UK Eye Movements: A Window on Mind and Brain Edited by R. P. G. van Gompel, M. H. Fischer, W. S. Murray and R. L. Hill Copyright © 2007 by Elsevier Ltd. All rights reserved. Basal Fonts:Times Margins:Top:4.6pc Gutter:4.6pc Font Size:10/12 Text Width:30pc Depth:43 Lines Elsevier AMS Ch01-I044980 Job code: EMAW 14-2-2007 1:04p.m. Page:2 Trimsize:165×240MM 2 R. P. G. van Gompel et al. Abstract This opening chapter begins with an overview of the issues and questions addressed in the remainder of this book, the contents of which reflect the wide diversity of eye-movement research. We then provide the reader with an up-to-date impression of the most significant developments in the area, based on findings from a survey of eye-movement researchers and database searches. We find that the most heavily cited publications are not necessarily those rated as most influential. It is clear that eye-movement research is published across a wide variety of journals and the number of articles is increasing, but the relative proportion of eye-movement articles has remained almost constant. The United States produces the largest number of these articles, but other countries actually produce more per capita.
    [Show full text]
  • A Method to Analyze Similarities of Eye Movements: a Giscience Based Spatiotemporal Approach
    University of South Florida Scholar Commons Graduate Theses and Dissertations Graduate School November 2019 A Method to Analyze Similarities of Eye Movements: a GIScience Based Spatiotemporal Approach Mehrdad Vaziri University of South Florida Follow this and additional works at: https://scholarcommons.usf.edu/etd Part of the Geographic Information Sciences Commons Scholar Commons Citation Vaziri, Mehrdad, "A Method to Analyze Similarities of Eye Movements: a GIScience Based Spatiotemporal Approach" (2019). Graduate Theses and Dissertations. https://scholarcommons.usf.edu/etd/8691 This Dissertation is brought to you for free and open access by the Graduate School at Scholar Commons. It has been accepted for inclusion in Graduate Theses and Dissertations by an authorized administrator of Scholar Commons. For more information, please contact [email protected]. A Method to Analyze Similarities of Eye Movements: a GIScience Based Spatiotemporal Approach by Mehrdad Vaziri A dissertation submitted in partial fulfillment of the requirement for the degree of Doctor of Philosophy in Geography and Environmental Science and Policy School of Geosciences College of Arts and Sciences University of South Florida Major Professor: Joni Firat, Ph.D. Steven Reader, Ph.D. Philip Van Beynen, Ph.D. Elizabeth Schotter, Ph.D. Date of Approval: November 19, 2019 Keywords: Similarity Analysis, GIScience, Eye Tracking, Spatiotemporal Copyright © 2019, Mehrdad Vaziri DEDICATION Dedicated to my wife Tina and my Uncle Reza, my Mother and Father, My Brothers and Sisters and all their beautiful children, Mahshid, Sina, Kian, Parnyan ACKNOWLEDGMENT I would like to thank my major professor Dr. Joni Firat for her neverending support and mentorship throughout my time at USF.
    [Show full text]
  • The Effects of Word Length and Suffixation on Eye Movement Control in Turkish Reading
    THE EFFECTS OF WORD LENGTH AND SUFFIXATION ON EYE MOVEMENT CONTROL IN TURKISH READING A THESIS SUBMITTED TO THE GRADUATE SCHOOL OF INFORMATICS OF THE MIDDLE EAST TECHNICAL UNIVERSITY BY TUĞÇE NUR BOZKURT IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE IN THE DEPARTMENT OF COGNITIVE SCIENCE SEPTEMBER 2017 THE EFFECTS OF WORD LENGTH AND SUFFIXATION ON EYE MOVEMENT CONTROL IN TURKISH READING Submitted by TUĞÇE NUR BOZKURT in partial fulfillment of the requirements for the degree of Master of Science in Cognitive Science Department, Middle East Technical University by, Prof. Dr. Deniz Zeyrek Bozşahin Dean, Graduate School of Informatics Prof. Dr. Cem Bozşahin Head of Department, Cognitive Science Asst. Prof. Dr. Cengiz Acartürk Supervisor, Cognitive Science Dept., METU Examining Committee Members: Prof. Dr. Cem Bozşahin Cognitive Science Dept., METU Asst. Prof. Dr. Cengiz Acartürk Cognitive Science Dept., METU Prof. Dr. Deniz Zeyrek Bozşahin Cognitive Science Dept., METU Asst. Prof. Dr. Burcu Can Buğlalılar Computer Engineering Dept., Hacettepe University Assoc. Prof. Dr. Bilal Kırkıcı Foreign Language Education Dept., METU Date: _September 7, 2017 I hereby declare that all information in this document has been obtained and presented in accordance with academic rules and ethical conduct. I also declare that, as required by these rules and conduct, I have fully cited and referenced all material and results that are not original to this work. Name, Last name : Tuğçe Nur Bozkurt Signature : iii ABSTRACT THE EFFECTS OF WORD LENGTH AND SUFFIXATION ON EYE MOVEMENT CONTROL IN TURKISH READING Bozkurt, Tuğçe Nur MSc., Department of Cognitive Science Supervisor: Asst.
    [Show full text]
  • I COGNITIVE CONTROL of EYE MOVEMENTS in READING and VISUAL
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by D-Scholarship@Pitt COGNITIVE CONTROL OF EYE MOVEMENTS IN READING AND VISUAL SEARCH: EVIDENCE FROM FREQUENCY-BASED EFFECTS by Polina M. Vanyukov B.S., Carnegie Mellon University, 2001 Submitted to the Graduate Faculty of the Kenneth P. Dietrich School of Arts and Sciences in partial fulfillment of the requirements for the degree of Doctor of Philosophy University of Pittsburgh 2013 i UNIVERSITY OF PITTSBURGH THE KENNETH P. DIETRICH SCHOOL OF ARTS AND SCIENCES This dissertation was presented by Polina M. Vanyukov It was defended on March 20th, 2013 and approved by Charles A. Perfetti, Professor, Department of Psychology David C. Plaut, Professor, Department of Psychology, Carnegie Mellon University Natasha Tokowicz, Associate Professor, Department of Psychology Dissertation Advisor: Tessa Warren, Associate Professor, Department of Psychology ii Copyright © by Polina M. Vanyukov 2013 iii COGNITIVE CONTROL OF EYE MOVEMENTS IN READING AND VISUAL SEARCH: EVIDENCE FROM FREQUENCY-BASED EFFECTS Polina M. Vanyukov, PhD University of Pittsburgh, 2013 Experiments in this dissertation investigate the role of cognition in eye-movement behavior during scanning and reading. Shorter and fewer fixations on the more frequent and predictable words have been observed during reading, but not when scanning text for a target word, e.g., zebra (e.g., Rayner & Raney, 1996). Past research has employed these effects of word frequency and predictability to argue that cognition drives eye movements during reading, but not during scanning. Similarly, the present studies use effects of stimulus frequency and predictability to index cognitive control of eye-movement behavior.
    [Show full text]
  • The Imagined Voice 2017 JULY
    Cover Page The handle http://hdl.handle.net/1887/58691 holds various files of this Leiden University dissertation. Author: Kyriakides, Y. Title: Imagined Voices : a poetics of Music-Text-Film Issue Date: 2017-12-21 PART I: Three Voices 16 Chapter 1: The Mimetic Voice One of the bedrocks of the history of aesthetics in western culture is Plato's formulation of 'mimesis' and 'diegesis'. In a dialogue between Socrates and Adeimantus in Book 3 of The Republic, Plato sets out to differentiate all forms of poetry in 'mimesis' and 'diegesis', which roughly translate respectively into 'showing' and 'telling': One kind of poetry and story-telling employs only imitation - tragedy and comedy, as you say. Another kind employs only narration by the poet himself - you find this most of all in Dithyrambs. A third kind uses both - as in epic poetry... (Plato, The Republic, 394c) Here Plato sets out the basic duality between narration and imitation. In diegesis the poet or narrator is speaking in their own voice, never leading the audience into thinking they are anyone other than that. In mimesis, the poet utilises imitation, and takes on the persona of another, by voice or gesture to show, to act out, as is the convention of much staged drama. In later chapters of The Republic, Plato expresses his prejudice against mimetic art, which he considers inferior, because in his view it simply copies the appearance of the real, reproducing shadows rather than shedding light on truths. This is somewhat ironic, because Plato himself utilises the form of the dramatic dialogue in much of his writing, using the voice of Socrates as a medium to channel his ideas.
    [Show full text]
  • Eye Movement, Perception, and Legibility in Reading by Miles A
    EYE MOVEMENT, PERCEPTION, AND LEGIBILITY IN READING BY MILES A. TINKER University of Minnesota This review is a continuation of the series previously published in this journal (69, 70, 71, 73). A few early papers which were missed in the previous reviews are included in the list of references (31,32, 33,34,51,55,57,79,86). No publications appearing later than October, 1935, are included. Bibliographies, popular discussions, summaries of part of the field,, and textbook surveys will be found in references 12, 28, 35, 36, 37, 45, 46, 49, 50a, 66. New methods and new devices, or modifications of previous apparatus constructed for use in reading investigations are described in references 2, 6, 7, 16, 17, 18, 60, 68, 72, 78, 83. EYE MOVEMENTS Recent investigations of eye movements in reading have involved for the most part the measurement of oculomotor behavior in special reading and learning situations, determination of validity and relia- bility of the measures, and evaluation of the importance of eye move- ments in diagnostic and remedial reading. A few experiments have been concerned with establishing norms, the use of eye movements as measures of legibility, and the relation of eye movements to other measures of performance. Studies of eye movements in the reading of oriental and foreign languages continue to appear. Futch (27) reports on a comparison of the various measures of eye move- ments in reading Latin and English by beginning and experienced students. Both groups of subjects exhibited less efficiency in reading Latin than in reading English. Although the beginning and the advanced Latin students were affected similarly in going from English to Latin, the performance of the beginners was most affected.
    [Show full text]
  • Eye-Movement Analysis in Reading Content Words and Function Words
    Eye-movement analysis in reading content words and function words Sophie Roussel1, Anne Rohr1, Eric Raufaste 2, Jean-Luc Nespoulous1 1Laboratoire Lordat – CEA.1941- IFR N°96 - Université Toulouse-II, France 2 Laboratoire Travail et Cognition – UMR5551 du CNRS - Université Toulouse-II, France Address for correspondence: Sophie Roussel - Laboratoire Lordat Université Toulouse-II Maison de la Recherche 5, Allées A. Machado, 31058 Toulouse Cedex France Email: [email protected] Tel: (+33) 561-503-594 Fax: (+33) 561-504-918 Running Head: EYE-MOVEMENTS in READING ROUSSEL EYE-MOVEMENTS in READING 1 ABSTRACT Detection task studies and studies of eye-movement in reading all pointed out a differential processing of content and function words. Several determinants were proposed: grammatical class, word configuration, word frequency, etc. Although function words are both “short” and “highly frequent”, interactions between those factors were left widely underdetermined. Hence, the present eye-movement study aimed at isolate the specific effect of grammatical class, controlling other factors, and specifically frequency and form properties (e.g. word length). Results confirm that grammatical class induces by itself different attentional patterns with more fixations on content words than on function words. When words were gazed, there were longer gaze durations and smaller pupil diameter on content words than on function words. Keywords: Reading - Eye-movements - content word – function words ROUSSEL EYE-MOVEMENTS in READING 2 When reading a text, all words are not read in a similar way. The hypothesis of a differential processing of content and function words in non-pathological persons comes from the observation of aphasic patients with agrammatism.
    [Show full text]
  • Eye-Based Interaction in Graphical Systems: Theory & Practice
    Eye-Based Interaction in Graphical Systems: Theory & Practice Andrew Duchowski Clemson University Part I Introduction to the Human Visual System (HVS) Siggraph 2000 Course Notes I–1 Eye-Based Interaction in Graphical Systems A Visual Attention “Everyone knows what attention is. It is the taking possession by the mind, in clear and vivid form, of one out of what seem several simultaneously possible objects or trains of thought. Focalization, concentration, of consciousness are of its essence. It implies withdrawal from some things in order to deal effectively with others...” “When the things are apprehended by the senses, the number of them that can be attended to at once is small, ‘Pluribus intentus, minor est ad singula sensus.’” –William James [Jam81, pp.381-382,p.384] The Latin phrase used above by James roughly translates to “Many filtered into few for perception”. The faculty implied as the filter is attention. Humans are finite beings that cannot attend to all things at once. Attention is used to focus our mental capacities on selections of the sensory input so that the mind can successfully process the stimulus of interest. Our capacity for information processing is roughly bounded by the “magic” number 7 2 [Mil56]. While listening to an orchestra, it is possible to concentrate on specific instruments, e.g., the flute or the oboe. The brain processes sensory input by concentrating on specific components of the entire sensory realm so that interesting sights, sounds, smells, etc., may be examined with greater attention to detail than peripheral stimuli. This is particularly true of vision.
    [Show full text]
  • The Impact of Left to Right Eye Movement on Arab EFL/ESP Learners’ Output∗
    Sino-US English Teaching, January 2017, Vol. 14, No. 1, 16-24 doi:10.17265/1539-8072/2017.01.003 D DAVID PUBLISHING The Impact of Left to Right Eye Movement on Arab EFL/ESP Learners’ Output∗ Dr. Abdellah Benahnia Casablanca: Morocco English Language Center, King Fahad Medical City, Riyadh, Kingdom of Saudi Arabia Though a lot has been said about L2 acquisition and learners’ output, there is still much to be done in terms of research pertaining to the cognitive part of eye movement based on the nature of the target language (TL) within the process of learning. Thus, eye movement according to the nature of L2 can be a challenge that learners might face. Arab EFL, or ESP learners might find it more difficult, taking into consideration the nature of the script of their Arabic language mother tongue (MT) going from right to left, and being a total opposite of all Anglo Saxon scripts (left to right). The following article tries to focus on the input process of Arab learners while trying to adapt to, or learn English as a foreign language, and find out if the paradox of eye movement which is left to right vs. right to left, (in the case of EFL/ESP learning), has any impact on these particular learners’ actual output and performance. Keywords: eye movement, reading span, learning, EFL Arab learners Introduction Though it has been claimed that lexical access cannot be the “engine” driving eye movements during reading, and that when and where the eyes move are instead primarily determined by visual and oculomotor factors (Reilly & O’Regan, 1998), there are yet other claims that eye movement in reading scripts can be subject to habit formation.
    [Show full text]
  • Exploring Translation and Interpreting Hybrids. the Case of Sight Translation Barbara Dragsted and Inge Gorm Hansen
    Document generated on 09/28/2021 11:24 p.m. Meta Journal des traducteurs Translators' Journal Exploring Translation and Interpreting Hybrids. The Case of Sight Translation Barbara Dragsted and Inge Gorm Hansen Volume 54, Number 3, septembre 2009 Article abstract This article reports on a comparative study of written translation and sight URI: https://id.erudit.org/iderudit/038317ar translation, drawing on experimental data combining keystroke logging, DOI: https://doi.org/10.7202/038317ar eye-tracking and quality ratings of spoken and written output produced by professional translators and interpreters. Major differences in output rate See table of contents were observed when comparing oral and written modalities. Evaluation of the translation products showed that the lower output rate in the written condition was not justified by significantly higher quality in the written products. Publisher(s) Observations from the combination of data sources point to fundamental behavioural differences between interpreters and translators. Overall, Les Presses de l'Université de Montréal working in the oral modality seems to have a lot to offer in terms of saving time and effort without compromising the output quality, and there seems to ISSN be a case for increasing the role of oral translation in translator training, incorporating it as a deliberate practice activity. 0026-0452 (print) 1492-1421 (digital) Explore this journal Cite this article Dragsted, B. & Hansen, I. G. (2009). Exploring Translation and Interpreting Hybrids. The Case of Sight Translation. Meta, 54(3), 588–604. https://doi.org/10.7202/038317ar Tous droits réservés © Les Presses de l’Université de Montréal, 2009 This document is protected by copyright law.
    [Show full text]
  • An Integrated Model of Eye Movements and Visual Encoding Dario D
    Journal of Cognitive Systems Research 1 (2001) 201±220 www.elsevier.com/locate/cogsys An integrated model of eye movements and visual encoding Dario D. Salvucci* Cambridge Basic Research, Four Cambridge Center, Cambridge, MA 02142, USA Received 6 June 2000; accepted 2 October 2000 Abstract Recent computational models of cognition have made good progress in accounting for the visual processes needed to encode external stimuli. However, these models typically incorporate simpli®ed models of visual processing that assume a constant encoding time for all visual objects and do not distinguish between eye movements and shifts of attention. This paper presents a domain-independent computational model, EMMA, that provides a more rigorous account of eye movements and visual encoding and their interaction with a cognitive processor. The visual-encoding component of the model describes the effects of frequency and foveal eccentricity when encoding visual objects as internal representations. The eye-movement component describes the temporal and spatial characteristics of eye movements as they arise from shifts of visual attention. When integrated with a cognitive model, EMMA generates quantitative predictions concerning when and where the eyes move, thus serving to relate higher-level cognitive processes and attention shifts with lower-level eye-movement behavior. The paper evaluates EMMA in three illustrative domains Ð equation solving, reading, and visual search Ð and demonstrates how the model accounts for aspects of behavior that simpler models of cognitive and visual processing fail to explain. 2001 Elsevier Science B.V. All rights reserved. Keywords: Eye movements; Cognitive models; Reading; Visual search; Computational models 1. Introduction researchers have developed such models in domains including reading (e.g., Just & Carpenter, 1980), Models of cognition have recently taken great arithmetic (e.g., Suppes, 1990), analogy (e.g., Sal- strides in interacting with the visual world.
    [Show full text]
  • Eye Movements While Reading and Searching Spatially Transformed Text: a Developmental Examination
    Memory & Cognition 1976, Vol. 4 (1),3642 Eye movements while reading and searching spatially transformed text: A developmental examination ANNE B. SPRAGINS and LESTER A. LEFTON University ofSouth Carolina, Columbia, South Carolina 29208 and DENNIS F. FISHER HumanEngineering Laboratory, AberdeenProving Ground, Maryland The effects of spatial manipulation on eye movement when subjects either read or searched through paragraphs were examined. Adults, third, and fifth graders were presented with paragraphs which were typed normally or in alternating case (upper and lower). The spacing between the words was either normal, filled, or absent. The results show that approximately twice as many character spaces are processed during search as during reading. Subjects made more fixations of shorter duration during search than reading. The data indicated that the ability to vary the size of the perceptual unit develops with experience. When spatial cues were unavailable, all subjects resorted to a letter-by-letter-like processing strategy in reading but not in search. Although reading and search were found to be sensitive to the same types of spatial manipulations, discrepancies of span and speed suggest qualitative differences; comprehension demands during reading can account for these differences. The data were interpreted to provide support for the two-stage model of Hochberg (1970). Reading can be defined as a translation of spatially stimulus materials. Reliance upon the periphery was presented lines and spaces into meaningful information. hypothesized to be developmentally related. According A number of experiments have recently been performed to this view, adults are reliant upon the periphery for examining the spacial manipulations of printed text in physical featural cues, whereas children rely upon foveal reading and visual search.
    [Show full text]