Visual perception of spatial relations in depth Michelle Doumen Universiteit Utrecht Copyright © 2006 by M.J.A. Doumen All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, without written permission of the author. Cover design: Mirjam van Tilburg ISBN-10: 90-393-4298-9 ISBN-13: 978-90-393-4298-5 Visual perception of spatial relations in depth Visuele waarneming van ruimtelijke relaties in de diepte (met een samenvatting in het Nederlands) Proefschrift ter verkrijging van de graad van doctor aan de Universiteit Utrecht op gezag van de rector magnificus, prof. dr. W.H. Gispen ingevolge het besluit van het college voor promoties in het openbaar te verdedigen op woensdag 6 september 2006 des middags te 12.45 uur door Michelle Jacoba Adriana Doumen geboren op 10 mei 1977, te Roosendaal en Nispen Promotoren: prof dr. J.J. Koenderink prof. dr. A.M.L. Kappers The investigations described in this thesis were financially supported by the Netherlands Organization for Scientific Research (NWO, ALW). Contents 1 Introduction 7 1.1 The visual system 8 1.2 Reference frames 10 1.3 Depth cues 10 1.4 Visual space research 12 1.5 This thesis 15 2 Visual space under free viewing conditions 17 2.1 Introduction 18 2.2 General methods 21 2.3 Experiment 1: Exocentric pointing 22 2.4 Experiment 2: Parallelity task 25 2.5 Experiment 3: collinearity task 29 2.6 General discussion and conclusions 34 3 Horizontal-vertical anisotropy in visual space 37 3.1 Introduction 38 3.2 General methods 41 3.3 Experiment A: Horizontal separation angle 43 3.4 Experiment B: Relative distance 47 3.5 Experiment C: Vertical separation angle 1 50 3.6 Experiment D: Vertical separation angle 2 53 3.7 General discussion and conclusions 55 6 Contents 4 Effects of context on a 3D pointing task 59 4.1 Introduction 60 4.2 General methods 62 4.3 Experiment 1: Allocentric references 64 4.4 Experiment 2: Egocentric and allocentric references 69 4.5 General discussion and conclusions 75 5 Forward-backward asymmetry: the effect of body position and pointer shapes 77 5.1 Introduction 78 5.2 General methods 80 5.3 Experiment 1: The effect of body position 81 5.4 Experiment 2: The effect of pointer shapes 84 5.5 General discussion and conclusions 87 6 Visual perception of tilted planes 89 6.1 Introduction 90 6.2 Methods 91 6.3 Results 94 6.4 Discussion and conclusions 99 Summary 101 Nederlandse samenvatting 105 Reference list 109 Publication list 113 Dankwoord 115 Curriculum vitae 119 Chapter 1 Introduction Having thought about visual perception for five years, I have become more and more impressed by the power and ability of the visual system. It is amazing that this system is able to abstract so much information that is useful for the observer from an enormous amount of available data. In some respects the visual process is similar to the memory process: the basis of remembering is forgetting everything that is not useful and the basis of visual processing is perceiving important aspects of the environment by ignoring the rest. But how does the system work? How does it combine different features into an object? How does it integrate changes over time? And an important question for this thesis: how does the brain derive depth from two-dimensional projections on the retinae? My introduction is twofold: first of all I would like the reader to share my admiration for the processes involved in visual perception, and secondly I would like to explain why the work I have been doing is important. I will begin by giving a short introduction to the human visual system. After looking at reference frames and depth cues, I will look at the traditional visual space research and go on to discuss the newer paradigms that have been proposed. Finally I will introduce my own work that is described in detail in the following chapters of this thesis. 8 Chapter 1 1.1 The Visual System The visual system consists of sensory organs (the eyes) and brain-structures that process the signals that have entered the eyes. However, visual processing already begins in the retinae. I have no intention of discussing every aspect of the visual system here, since the purpose of this introduction is merely to introduce the reader to the basics of the system. Light entering the eyes Light enters the eye through the lens. The shape of the lens causes the beam of light to bend. This bending of the light rays enables the eye to focus on objects at different distances from the observer. If an object is close to the observer, the ciliary muscle contracts to make the lens spherical. The light-rays are therefore bent to a considerable degree. If an object is far away, the lens has to be flat in order to have the object in focus. This happens when the ciliary muscles relax, leading to less bending of the light-rays. The light hits the retina after entering the eye. The outer layer of the retina contains the photoreceptors, the rods and cones, which contain pigments that absorb photons. Bipolar cells transmit the signal from the photoreceptors to the ganglion cells. These ganglion cells have long axons that leave the eye at the blind spot. The blind spot got its name because this area of the retina has no photoreceptors and thus gives no output to the visual system. In the bipolar layer, there are neurons that contribute to the communication between adjacent bipolar cells. The initial processing of visual information (lateral inhibition) takes place in these cells. The light has to enter through this layer of cells to reach the photoreceptors. The fovea is the spot on the retina on to which the part of the visual field that is in focus is projected. This area of the retina contains a large amount of cones. The cells that are used for further processing are not in front of the photoreceptors here. This is why we have a particularly sharp image of the part of the visual field that is projected on to the fovea. Figure 1.1 gives a schematic picture of the eye. Figure 1.1 Horizontal cross-section of the eye Introduction 9 Visual processing We can distinguish multiple types of ganglion cells. The most widely known ones are the M and P cells. The M cells have a larger cell-body and also have a larger receptive field. These cells are distributed fairly evenly throughout the retina. The P cells, on the other hand, are smaller and have small receptive fields. They are located in and close to the fovea. The axons of the ganglion cells together form the optic nerve as they leave the eye through the blind spot. The optic nerves of both eyes cross in the optic chiasm. Here the ganglion cells from the nasal areas of the retinae cross to the other side of the brain (see Figure 1.2). This causes the information from the left visual field to be projected on to the right side of the brain and vice versa. From the optic chiasm the optic tract passes through the lateral geniculate nucleus. From this point we can speak of two separate visual pathways: the magnocellular pathway and the parvocellular pathway; these are projected mainly by the M and P ganglion cells, respectively. The parvocellular visual pathway is responsible for color and form perception and the recognition of objects. It transmits detailed information that can only be derived from objects that are in focus, due to the presence of P ganglion cells primarily near the fovea. The magnocellular pathway is concerned with the perception of depth and movement, and derives information mainly from the M ganglion cells. These two pathways are not as strictly separated as is often assumed, as they interact at many stages of processing. Figure 1.2 Horizontal cross-section of the brain showing the visual pathway from eyes to visual cortex The visual pathways connect the geniculate nucleus to the primary visual cortex (V1) of the occipital lobe and then lead on to higher visual areas like the secondary visual cortex (V2). Some processing of depth-information takes place in V1, however spatial processing takes place primarily in the parietal lobe of the cortex, which is part of the magnocellular or dorsal visual pathway. There has been much discussion about whether this dorsal pathway is concerned mainly with visual control of action (Goodale, Milner, Jakobson, & Catey, 1991) or with a more general spatial representation of the world. However, according to Creem and Proffitt (2001), the discussion could be redundant if one thinks in terms of reference frames. 10 Chapter 1 They suggest that in the parietal lobe different areas are involved in different reference frames. Since visual control of action and spatial representation of the world are based on different reference frames, this might explain the two theories of parietal lobe function (Creem, & Proffitt, 2001). However, others have suggested that the dorsal pathway is mainly concerned with egocentric representations (Neggers, van der Lubbe, Ramsey, & Postma, In press). Several authors suggest that it is the ventral pathway (the parvocellular pathway) rather than part of the dorsal pathway that is involved in the formation of allocentric representations. 1.2 Reference frames The concept of reference frames is important in visual space research, and indeed for perception in general. By ‘reference frame’ we mean the locus (or set of loci) with respect to which the spatial position of an object is defined (Pick, & Lockman, 1981).