DOCSLIB.ORG

On Mental Imagery and Its Relevance for AI

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
On Mental Imagery and Its Relevance for AI On Mental Imagery and its Relevance for AI Benjamin F. Maier∗ Final paper in the course “Philosophy of AI”, Department of Philosophy, Utrecht University (Dated: June 29, 2012) During the last fifty years there has been a lively debate on the topic of mental imagery, where this term relates colloquially to “thinking in mind’s pictures”. In this work, the positions of the major parties in this debate are briefly presented, an existing model of mental imagery for symbol manipulators is recapitulated and a slightly new model for mental imagery processing is proposed. Working out the meaning of mental imagery for de- velopments in artificial intelligence (AI) based on the existing positions, we will come to the conclusion that a theory of visual perception may propose advantages for weak AI, while mental imagery itself might be a trick of nature helping humans to solve problems and hence would be unnecessary for computers. I. INTRODUCTION view on the whole subject, mainly referring to mental im- agery as a method to solve problems and give ideas on fur- Before any statements on mental imagery can be made, we ther experiments to analyze the potential for such. have to present a first indication of the definition of the Afterwards, in section III, the whole topic will be con- term. Roughly spoken, the term “mental imagery” how it nected to actual and hypothetical developments in weak will mostly be used in this article relates to an eyes-closed and strong AI and a conclusion for the value of the mental picture one has in mind when thinking of a certain object or imagery debate for AI will be drawn. scenery, without the actual object or scenery being present for the senses to perceive [1, 2]. It is therefore often de- scribed as “quasi-perceptual” [3]. For example, given the II. DIFFERENT VIEWS ON MENTAL IMAGERY question “How many windows does your house have?”, a subject may first picture her house in the mind and then proceed to count the windows in order to answer the ques- A. The Structural Descriptive View tion, which would be some kind of a perceptual experience without the house being present. Although the literature Defenders of the structural descriptive view claim that im- mostly describes images rather vague as sensual experi- ages are of an abstract nature. The main defender of ences without being specific of the kind of interpretation, this view is Z. Pylyshyn. According to them, mental im- images will mainly be referred to pictures in this article. ages have to be interpreted as complex representations of The question whether or not humans think in images arose smaller parts, called “propositions”, where propositions in ancient philosophy already, being discussed until and are informations with truth value, object information and throughout early modern philosophy. Using the method of spatial relation and carry semantic as well syntactic infor- introspection, philosophers tended to declare images as lit- mation (i.e. a set of propositions connected in two different eral pictures in the head, sometimes even more substantial ways delivers two different representations) [5]. It is un- than thinking in language. Later, in the early 20th century, necessary for the mind to process visual information, since philosophers could not agree on certain positions, until the all the information is present in form of propositions. Any debate had generally been declared meaningless since the occurrence of picturesque perception in the head is epiphe- method of introspection did not provide enough empirical nomenal. Pictorial data is not as fundamental as proposi- insight [4]. tional. However, during the last 50 years, cognitive scientists/psy- To dispute that mental images are induced from something chologists started the debate again, forming mainly two like a visual memory (even a propositional), Pylyshyn per- views, 1) the pictorial and 2) the structural descriptive, formed experiments, such as the “inclined beaker” experi- where the first view is mainly supported by S. Kosslyn and ment [6]. He showed an inclined fluid-filled beaker to chil- the second by Z. Pylyshyn. They worked out their views in dren of the age of 4, where naturally the fluid’s surface great detail and supported and disputed them by empirical was not perpendicular to the beaker’s sides. Afterwards he experiments. asked the children to draw a sketch of the beaker, which In section II, their views will be summarized briefly, while resulted in drawings where the fluid’s surface was actually indications of experiments and explanations of some of drawn perpendicular to the sides. He concludes that the the implications will be given. Afterwards a third model children were just not familiar enough with the proposi- of mental imagery is introduced, developed bye M. Tye, tional concept of a constant angle of fluid’s surfaces, yield- which is also a model of pictorial kind, but concentrates ing the next close concept of “perpendicular to the side”. more on a connection between pictures and symbol repre- On the other hand, assuming there would be something sentations. Based on these models I will present my own like a visual memory, the children would have been able to memorize the actual picture of the inclined beaker and would have drawn it right. Mental image indeterminacy is also explained, using the model of “lack of concepts”. ∗Electronic address: [email protected] A similar view by G.E. Hinton and L.M. Parsons pro- 2 Image generation THE VISUAL LONG-TERM BUFFER Information arriving from VISUAL Quasi-pictorial the visual system (after MEMORY patterns of acti- preprocessing, e.g., vation detection of edges) Image inspection Image transformation FIG. 1: Chart including the basic components of Kosslyn’s theory, recreated from [9] poses a hierarchical structure for propositional memory (iii) for any three represented O parts, X, Y and Z, if X [7]. Given a question for a detail of a whole, the mind has appears at a greater surface distance from Y than from to start with a mind-map-like representation of the whole, Z, then this fact is represented in R if and only if there working its way through the branches to find detailed infor- are more R parts representing apparently adjacent O mation, connected to the epiphenomenal activation of pic- parts that are connected by the shortest apparent path turesque perception. E.g., asked “Do horses have toes?”, on the surface of O between X and Y and that are the mind would start with the whole representational net- each of the same length L as measured along that path work of a horse, go to the branch “leg” and would eventu- than there are R parts representing the corresponding ally find the information “hoof” enabling it to answer the O parts of apparent length L between X and Z.” [11] question with “no”. Using this view he explains the pic- turesque perception one encounters through introspection when asked a question of this kind, stating that proposi- tional content is directly used, rather than visual. Using this view, a quasi-pictorial is attached with spatial The propositional or descriptive view of mental imagery is properties, which is a main difference to the descriptive described very well by J.-P. Sartre, who states “I can keep view. To support this view, various experiments have been looking at an image for as long as I wish: I will never find performed of which I want to introduce two. The first is anything but what I put there” [8]. the rotation experiment [12]. Subjects were asked to look at two pictures of three-dimensional objects, which looked similar, most of them equal but rotated. Then, they were B. The Pictorial View asked whether one of the objects can be transformed into the other. They discovered that the relation between the The pictorialists claim that mental images do have a pic- angle of a rotation and the time it took the subjects to give torial representation in the head which is as fundamental an answer was linear, indicating that a processing of the as propositions. While some even propose a strict distinc- image had occurred in order to give an answer. tion between verbal and picture-like processing, S. Koss- Some other experiments investigated the scanning of men- lyn, as the main defender of this view, sees mental imagery tal images [13]. E.g., subjects were asked to build a mental as both, processing propositions and picture-like informa- image of a map with certain objects on it they have been tion. He introduces so called “surface representations” of shown before. Then, they had to put a mental marker on information using visual propositional long-term memory one of the map’s objects and to push a button. Afterwards, with information about spatial relations, which is not avail- they scanned the shortest path between two objects with able for the mind per se. These representations are con- the inner marker and pushed the button again, the button structed in a visual buffer and can be seen and interpreted responsible for a time measurement. The experimenters by a functional mind’s eye (see figure 1) [10]. Furthermore observed that the time needed to scan the mental image they may be transformed inside the visual buffer. Asked a was linear with respect to the seperation of the objects. question which is connected to a picture, the mind uses that Again, they conclude that mental images apparently have procedure to actually work with the surface representation spatial properties and are processed by a visual perception in order to obtain answers for the problem. The surface system. representations are described as quasi-pictorial, where M.
Load more

Recommended publications
  • Alister Mcgrath's Anti-Mind-Body Dualism: Neuroscientific and Philosophical Quandaries for Christian Physicalism Brandon Rickabaugh* I
    TRINJ40NS (2019) 215-240 ALISTER MCGRATH'S ANTI-MIND-BODY DUALISM: NEUROSCIENTIFIC AND PHILOSOPHICAL QUANDARIES FOR CHRISTIAN PHYSICALISM BRANDON RICKABAUGH* I. INTRODUCTION Here is a staggering truth: the ontology of the human person currently embraced by the most vocal Christian scholars working on this issue is a view that almost no Christians thought plausible only 100 years ago. Until recently, the dominant view among Christian thinkers has been various forms of mind-body dualism (hereafter, dualism), according to which the human person comprises body and soul.1 In stark disagreement, many contemporary Christian scholars vigorously advance antidualism and defend physicalism (reductive or nonreductive), understanding the human person as fundamentally physical.2 These Christian physicalists proffer the strong impression of a uniform rejection of dualism across the neuroscientific, theological, and philosophical communities, as if dualism has been defeated, just as phlogiston was in in the 1770s. Here is another staggering truth: this certain-defeat-of-dualism narrative is demonstrably false. There is, in fact, a growing resurgence of dualism in philosophy. The recent Blackwell Companion Brandon Rickabaugh is a PhD candidate in the Department of Philosophy at Baylor University. This paper won the 2018-2019 Harold O. J. Brown Award for Student Scholarship. aSee Paul Gavarilyuk, "The Incorporeality of the Soul in Patristic Thought," in Christian Physicalism? Philosophical Theological Criticisms, ed. Keith Loftin and Joshua Farris (Lanham, MD: Lexington Books, 2017), 1-26; and Thomas Atkinson, "Christian Physicalism: Against the Medieval Divines," in Loftin and Farris, Christian Physicalism?, 27-42. This isn't to say that dualism was the only view, as there is a tiny minority of Christian physicalists in the history of the church.
    [Show full text]
  • Vasubandhu's) Commentary on His "Twenty Stanzas" with Appended Glossary of Technical Terms
    AN INTRODUCTION AND TRANSLATION OF VINITADEVA'S EXPLANATION OF THE FIRST TEN VERSES OF (VASUBANDHU'S) COMMENTARY ON HIS "TWENTY STANZAS" WITH APPENDED GLOSSARY OF TECHNICAL TERMS Gregory Alexander Hillis Palo Alto, California B.A., University of California, Santa Cruz, 1979 A Thesis Presented to the Graduate Faculty of the University of Virginia in Candidacy for the Degree of Master of Arts Department of Religious Studies University of Virginia May, 1993 ABSTRACT In this thesis I argue that Vasubandhu categorically rejects the position that objects exist external to the mind. To support this interpretation, I engage in a close reading of Vasubandhu's Twenty Stanzas (Vif!lsatika, nyi shu pa), his autocommentary (vif!lsatika- vrtti, nyi shu pa'i 'grel pa), and Vinrtadeva's sub-commentary (prakaraiJa-vif!liaka-f'ika, rab tu byed pa nyi shu pa' i 'grel bshad). I endeavor to show how unambiguous statements in Vasubandhu's root text and autocommentary refuting the existence of external objects are further supported by Vinitadeva's explanantion. I examine two major streams of recent non-traditional scholarship on this topic, one that interprets Vasubandhu to be a realist, and one that interprets him to be an idealist. I argue strenuously against the former position, citing what I consider to be the questionable methodology of reading the thought of later thinkers such as Dignaga and Dharmak:Irti into the works of Vasubandhu, and argue in favor of the latter position with the stipulation that Vasubandhu does accept a plurality of separate minds, and he does not assert the existence of an Absolute Mind.
    [Show full text]
  • The Manifest Image and the Scientific Image(1)
    (1) The Manifest Image and the Scientific Image Bas C. van Fraassen Princeton University (published: pp. 29-52 in D. Aerts (ed.). Einstein Meets Magritte: The White Book -- An Interdisciplinary Reflection. Dordrecht: Kluwer, 1999.) PART ONE. WORLDVIEWS IN COLLISION (?) .................................... 2 1. The Clash ........................................................... 2 2. The three main differences between the Images ....................... 3 PART TWO. THE PLAGUE OF IRREMEDIABLE VAGUENESS .......................... 4 3. Deconstructing the Manifest Image ................................... 4 4. Deconstructing the Scientific Image ................................. 5 5. Philosophical choices in response ................................... 7 PART THREE. AN INCOHERENT FICTION ....................................... 8 6. The Images as philosophical miscreants .............................. 8 6.1 What is this thing called the Manifest Image? ................... 8 6.2 And what of that thing called the Scientific Image? ............. 9 6.3 The dialectic that engenders the dichotomy ...................... 9 7. The very idea of images .......................................... 10 PART FOUR. REAL LIFE WITH SCIENCE ..................................... 11 8. A new beginning .................................................... 12 9. The continuity of common sense and science in method ............... 13 10. Perspectival discourse and relativity ............................ 13 11. Value- and function-laden discourse .............................
    [Show full text]
  • Reflexive Monism
    Reflexive Monism Max Velmans, Goldsmiths, University of London; email [email protected]; http://www.goldsmiths.ac.uk/psychology/staff/velmans.php Journal of Consciousness Studies (2008), 15(2), 5-50. Abstract. Reflexive monism is, in essence, an ancient view of how consciousness relates to the material world that has, in recent decades, been resurrected in modern form. In this paper I discuss how some of its basic features differ from both dualism and variants of physicalist and functionalist reductionism, focusing on those aspects of the theory that challenge deeply rooted presuppositions in current Western thought. I pay particular attention to the ontological status and seeming “out- thereness” of the phenomenal world and to how the “phenomenal world” relates to the “physical world”, the “world itself”, and processing in the brain. In order to place the theory within the context of current thought and debate, I address questions that have been raised about reflexive monism in recent commentaries and also evaluate competing accounts of the same issues offered by “transparency theory” and by “biological naturalism”. I argue that, of the competing views on offer, reflexive monism most closely follows the contours of ordinary experience, the findings of science, and common sense. Key words: Consciousness, reflexive, monism, dualism, reductionism, physicalism, functionalism, transparency, biological naturalism, phenomenal world, physical world, world itself, universe itself, brain, perceptual projection, phenomenal space, measured space, physical space, space perception, information, virtual reality, hologram, phenomenological internalism, phenomenological externalism, first person, third person, complementary What is Reflexive Monism? Monism is the view that the universe, at the deepest level of analysis, is one thing, or composed of one fundamental kind of stuff.
    [Show full text]
  • The Refutation of Idealism Author(S): G
    Mind Association The Refutation of Idealism Author(s): G. E. Moore Source: Mind, New Series, Vol. 12, No. 48 (Oct., 1903), pp. 433-453 Published by: Oxford University Press on behalf of the Mind Association Stable URL: http://www.jstor.org/stable/2248251 . Accessed: 30/03/2011 03:28 Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at . http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you may use content in the JSTOR archive only for your personal, non-commercial use. Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at . http://www.jstor.org/action/showPublisher?publisherCode=oup. Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed page of such transmission. JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected]. Oxford University Press and Mind Association are collaborating with JSTOR to digitize, preserve and extend access to Mind. http://www.jstor.org NEW SERIES. NO. 48.] [OCTOBER, 1903. MIND A QUARTERLY REVIEW OF PSYCHOLOGYAND PHILOSOPHY I.-THE REFUTATION OF IDEALISM.
    [Show full text]
  • Consciousness Revisited: Materialism Without Phenomenal Concepts
    Consciousness Revisited Materialism without Phenomenal Concepts Michael Tye A Bradford Book The MIT Press Cambridge, Massachusetts London, England 6 2009 Massachusetts Institute of Technology All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or informa- tion storage and retrieval) without permission in writing from the publisher. For information on quantity discounts, email [email protected]. Set in Times New Roman and Syntax on 3B2 by Asco Typesetters, Hong Kong. Printed and bound in the United States of America. Library of Congress Cataloging-in-Publication Data Tye, Michael. Consciousness revisited : materialism without phenomenal concepts / Michael Tye. p. cm. — (Representation and mind series) ‘‘A Bradford book.’’ Includes bibliographical references and index. ISBN 978-0-262-01273-7 (hard cover : alk. paper) 1. Consciousness. 2. Phenomenology. 3. Materialism. I. Title. B808.9.T943 2009 126—dc22 2008030920 10987654321 1 Phenomenal Consciousness At the very heart of the mind-body problem is the question of the nature of consciousness. It is consciousness, and in particular phe- nomenal consciousness, that makes the mind-body relation so deeply perplexing. Many philosophers agree that phenomenal consciousness (P- consciousness, for short) cannot be reductively defined. For example, Ned Block writes: Let me acknowledge at the outset that I cannot define P-consciousness in any re- motely non-circular way. I don’t consider this an embarrassment. The history of reductive definitions in philosophy should lead one not to expect a reductive defi- nition of anything. But the best one can do for P-consciousness is in some respects worse than for many other things because really all one can do is point to the phenomenon.
    [Show full text]
  • New Essays on Human Understanding Book II: Ideas
    New Essays on Human Understanding Book II: Ideas G. W. Leibniz Copyright © Jonathan Bennett 2017. All rights reserved [Brackets] enclose editorial explanations. Small ·dots· enclose material that has been added, but can be read as though it were part of the original text. Occasional •bullets, and also indenting of passages that are not quotations, are meant as aids to grasping the structure of a sentence or a thought. Every four-point ellipsis . indicates the omission of a brief passage that seems to present more difficulty than it is worth.—Longer omissions are [explained] as they occur. Very small bold unbracketed numerals indicate the corresponding section number in Locke’s Essay; most of these are provided by Leibniz. This version does not follow Leibniz’s practice of always avoiding Locke’s name in favour of ‘this author’, ‘our gifted author’, etc. First launched: February 2005 Last amended: April 2008 Contents Chapter i: Ideas in general, and the question ‘Does the soul of man always think?’ 35 Chapter ii: Simple ideas 41 Chapter iii: Ideas of one sense 41 Chapter iv: Solidity 42 Chapter v: Simple ideas of more than one sense 45 Chapter vi: Simple ideas of reflection 46 New Essays II G. W. Leibniz Chapter vii: Ideas of both sensation and reflection 46 Chapter viii: More considerations about simple ideas 46 Chapter ix: Perception 49 Chapter x: Retention 53 Chapter xi: Discerning, or the ability to distinguish ideas 53 Chapter xii: Complex ideas 55 Chapter xiii: Simple modes, starting with the simple modes of space 56 Chapter xiv: Duration and its simple modes 59 Chapter xv: Duration and expansion considered together 61 Chapter xvi: Number 62 Chapter xvii: Infinity 63 Chapter xviii: Other simple modes 65 Chapter xix: The modes of thinking 65 New Essays II G.
    [Show full text]
  • 6. Seeing with the Mind's Eye 1: the Puzzle of Mental Imagery
    S EEING AND V ISUALIZING: I T ’ S N OT W HAT Y OU T HINK * An Essay On Vision and Visual Imagination {CHAPTERS 6, 7, 8 FOR CLASS USE} ZENON PYLYSHYN, RUTGERS CENTER FOR COGNITIVE SCIENCE Table of Contents 6. Seeing With the Mind’s Eye 1: The Puzzle of Mental Imagery .................................................6-2 6.1 What is the puzzle about mental imagery?..............................................................................6-2 6.2 Content, form and substance of representations ......................................................................6-7 6.3 What is responsible for the pattern of results obtained in imagery studies?.................................6-9 6.3.1 Cognitive architecture or tacit knowledge .........................................................................................................6-9 6.3.2 Problem-solving by “mental simu lation”: Some additional examples ..........................................................6-13 6.3.2.1 Scanning mental images ............................................................................................................................6-14 6.3.2.2 The “size” of mental images......................................................................................................................6-18 6.3.2.3 Mental “paper folding”..............................................................................................................................6-19 6.3.2.4 Mental Rotation..........................................................................................................................................6-21
    [Show full text]
  • What Else Remains in Śūnyatā? an Investigation of Terms for Mental Imagery in the Madhyāntavibhāga-Corpus
    J ournal of the international Association of Buddhist Studies Volume 17 • Number 1 • Summer 1994 HUGH B. URBAN and PAUL J. GRIFFITHS What Else Remains in Sunyata? An Investigation of Terms for Mental Imagery in the Madhyantavibhaga-Corpus 1 BROOK ZIPORYN Anti-Chan Polemics in Post Tang Tiantai 26 DING-HWA EVELYN HSIEH Yuan-wu K'o-ch'in's (1063-1135) Teaching of Ch'an Kung-an Practice: A Transition from the Literary Study of Ch'an Kung-an to the Practical JCan-hua Ch'an 66 ALLAN A. ANDREWS Honen and Popular Pure Land Piety: Assimilation and Transformation 96 ROGER JACKSON Guenther's Saraha: A Detailed Review of Ecstatic Spontaneity 111 HUGH B. URBAN and PAUL J. GRIFFITHS What Else Remains In Sunyata? An Investigation of Terms for Mental Imagery in the Madhyantavibhaga-Corpus PROLEGOMENA In 1978 Gadjin Nagao published a short paper called "'What Remains' in Sunyata: A Yogacara Interpretation of Emptiness." There he argued that, according to the views expressed in the texts of the classical Indian Yogacara, "emptiness" (tunyata) does not denote simple absence or nonexistence (abhava)\ rather, there is always something left over or remaining (avafista) in emptiness, something that is identified with the basis for or locus of all human activity, and that is otherwise called the "dependent" (paratantra) aspect of experience. This remains even for Buddha: the realization of emptiness, claims Nagao, does not entail the end of the flow of experience, of what the Yogacara calls abhuta- parikalpa, the comprehensive construction of what is unreal. Rather, this constructive activity continues, though it is now radically different, and is called "perfected" (parinispanna).
    [Show full text]
  • Why Leibniz Should Have Agreed with Berkeley About Abstract Ideas
    This is the penultimate (preprint/AOM) version; it does not reflect changes made in response to the comments of referees. The Version of Record of this manuscript has been published and is available in British Journal for the History of Philosophy, https://www.tandfonline.com/doi/full/10.1080/09608788.2021.1895066. Why Leibniz Should Have Agreed with Berkeley about Abstract Ideas Stephen Puryear North Carolina State University Abstract Leibniz claims that Berkeley “wrongly or at least pointlessly rejects abstract ideas.” What he fails to realize, however, is that some of his own core views commit him to essentially the same stance. His belief that this is the best (and thus most harmonious) possible world, which itself stems from his Principle of Sufficient Reason, leads him to infer that mind and body must perfectly represent or “express” one another. In the case of abstract thoughts he admits that this can happen only in virtue of thinking some image that, being essentially a mental copy of a brain state, expresses (and is expressed by) that state. But here he faces a problem. In order for a thought to be genuinely abstract, its representational content must differ from that of any mental image, since the latter can represent only something particular. In that case, however, an exact correspondence between the accompanying mental image and the brain state would not suffice to establish a perfect harmony between mind and body. Even on Leibniz’s own principles, then, it appears that Berkeley was right to dismiss abstract ideas. Keywords: Leibniz, Berkeley, Abstraction, Abstract Ideas, Abstract Thoughts 1.
    [Show full text]
  • Spinoza, Benedict De (1632-77) Henry E
    From the Routledge Encyclopedia of Philosophy Spinoza, Benedict de (1632-77) Henry E. Allison Biography A Dutch philosopher of Jewish origin, Spinoza was born Baruch de Spinoza in Amsterdam. Initially given a traditional Talmudic education, he was encouraged by some of his teachers to study secular subjects as well, including Latin and modern philosophy. Perhaps as a result of this study, he abandoned Jewish practices and beliefs and, after receiving stern warnings, he was excommunicated from the synagogue in 1656. Alone and without means of support, he Latinized his name and took up the trade of lens grinder with the intention of devoting his life to philosophy. He remained in Amsterdam until 1660, lived for the next decade in nearby villages, and in The Hague from 1670 until his death from consumption in 1677. During these years he worked continuously on his philosophy and discussed it with a small circle of friends and correspondents. His masterpiece, Ethica Ordine Geometrico Demonstrata (Ethics Demonstrated in a Geometrical Manner), was completed in 1675; but because of its radical doctrines, it was only published after his death. The full scope of Spinoza’s Ethics is not indicated by its title. It begins with a highly abstract account of the nature of substance, which is identified with God, and culminates in an analysis of human beings, their nature and place in the universe, and the conditions of their true happiness. Written in a geometrical form modelled after Euclid, each of its five parts contains a set of definitions, axioms and propositions which are followed by their demonstrations and frequently by explanatory scholia.
    [Show full text]
  • The Mental Image Revealed by Gaze Tracking
    The Mental Image Revealed by Gaze Tracking Xi Wang Andreas Ley Sebastian Koch David Lindlbauer TU Berlin TU Berlin TU Berlin TU Berlin / ETH Zurich James Hays Kenneth Holmqvist Marc Alexa Georgia Institute of Universität Regensburg TU Berlin Technology correlate stimuli with gaze stimuli database ... user equipped with eye-tracker record fixation data Figure 1: We use eye movements of a person recalling an image while looking at nothing (a white wall in this case) to retrieve a set of potentially matching images from a database. ABSTRACT CCS CONCEPTS Humans involuntarily move their eyes when retrieving an • Human-centered computing → Interactive systems image from memory. This motion is often similar to actually and tools; Interaction techniques; Interaction paradigms; observing the image. We suggest to exploit this behavior as a new modality in human computer interaction, using the KEYWORDS motion of the eyes as a descriptor of the image. Interaction gaze pattern, mental imagery, eye tracking requires the user’s eyes to be tracked, but no voluntary physi- ACM Reference Format: cal activity. We perform a controlled experiment and develop Xi Wang, Andreas Ley, Sebastian Koch, David Lindlbauer, James matching techniques using machine learning to investigate Hays, Kenneth Holmqvist, and Marc Alexa. 2019. The Mental Image if images can be discriminated based on the gaze patterns Revealed by Gaze Tracking. In CHI Conference on Human Factors in recorded while users merely recall an image. Our results Computing Systems Proceedings (CHI 2019), May 4–9, 2019, Glasgow, indicate that image retrieval is possible with an accuracy Scotland UK. ACM, New York, NY, USA, 12 pages.
    [Show full text]