('~mtput & (;raphit~ Vol. 17, No. 6. pp. 643 653. 1993 0097-8493/93 $6.00 + .DO Printed in Great Britain. (c 1993 Pergamon Press Ltd. Virtual Reality SIMULATING PERIPHERAL VISION IN IMMERSIVE VIRTUAL ENVIRONMENTS MEL SLATER and MARTIN USOH Department of Computer Science and London Parallel Applications Centre (LPAC), QMW University of London. Mile End Road, London El 4NS UK Abstract--The standard graphics viewing pipeline is designed for images that are to be viewed externally, as on a screen or photograph, it may not be suitable for Immersive Virtual Environments (Virtual Reality) for many different reasons, one being the lack of peripheral vision. Peripheral vision offers important cues for direction of gaze and movement in the environment. This paper presents an alternative (but simple) viewing pipeline, that includes visual cues that could stimulate peripheral vision. After a brief discussion of lmmersive Virtual Environments, there is a description of the functioning of peripheral vision. This is followed by an analysis of the standard viewing model of computer graphics and the presentation of an alternative. Implementation details and a simple experiment on peripheral vision are described. I. INTRODUCI'ION displays, in the visual, auditory, and tactual modalities. A virtual environment (VE) is a dynamic environment The participant operates in an extended virtual space that is created by presenting a human observer with a created by the interaction between the human percep- world displayed by computer. The displays provide tual system and the computer generated displays. This signals in the visual, auditory, and tactual (tactile, kin- affords the possibility of participants maintaining a esthetic, force-feedback) modalities. In immersive VEs sense of presence in the VE, that is the (suspension of (IVEs) sensory input to the human from the external dis-) belief that they are in a world other than where world is supplemented, or wholly taken over, by at their real bodies are located. This is the unique pos- least one computer generated display. We include the sibility that IVEs offer: Just as computers are general term "supplemented," since we would also include as purpose machines, IVEs may be considered general IVEs the "see through" type, which merge inputs to purpose presence-transforming machines. the senses from the real world with those that are com- In this paper we examine peripheral vision--a par- puter generated[l]. To properly qualify, the merging ticular aspect of the interaction between the human should be seamless. Thus provision of a stereo display perceptual system and the visual display. In Section 2 on a 2D screen is not an immersive virtual environ- we outline the importance of peripheral vision in hu- ment, since the screen itself can be seen. man visual processing. In Section 3 we describe a sim- Following Ellis[2] an environment consists of con- ple experiment with users in order to assess whether it tent. geometry, and dynamics. The content consists of is possible to stimulate the effects of peripheral vision objects, a subset of which are actors. Objects have spe- at all in IVEs. In Section 4 we show how the standard cific properties such as position, orientation, velocity computer graphics viewing pipeline does not support and acceleration in space, as well as physical properties peripheral vision, and we present a modified viewing such as shape, colour, mass, and so on. Actors are dis- pipeline that does incorporate this. In Section 5 we tinguished from objects in that they are capable of ini- discuss the implementation of the modified pipeline, tiating an interaction with other actors or objects. The with results in Section 6. The conclusions are presented geometry of an environment defines the space in which in Section 7. the objects exist. It consists of a metric, a dimensional- ity, and an extent. Finally, the dynamic specifies rules of interaction between objects. 2. PERIPHERAL VISION In an IVE there is at least one actor that provides Current theories of vision postulate a two phase the egocentric point from which the environment can process--the primary visual system, based on opera- be described. This point determines a visual point of tions mainly in the eye and midbrain regions, and ma- view, an auditory location, and a tactual frame of ref- jor processing in the visual cortex. Images are focussed erence, from which the environment can be displayed onto the retina, which is a very thin tissue ( 150-300 by the computer. We use the term display here to in- #m) that lies at the back of the eye. The focussing is corporate outputs that provide consistent inputs (ide- achieved through the refractive power of the cornea ally) to all of the human senses. This actor embodies and lens and the fluids that fill the eye (acqueous and the human participant, who constructs the world vitreous humor). The retina is transparent, so light through perception of this display. passes completely through it before stimulating the Such IVEs, popularly called Virtual Reality (VR), photo-receptors at the back of the retina. These photo- therefore provide a tightly coupled human-computer receptors are of two types, called rods and cones, there interface: input to the sensory organs of the human being 120 × 10 6 rods and 5 × 10 6 cones. The rods are participant are directly generated through computer responsible for vision in poorly illuminated environ- 643 644 M. SLATER and M. USOH ments, whereas the cones operate in bright light and liculus" [ 5, p. 165 ]. Bruce and Green further point out provide visual acuity and colour. (p. 51 ) that such automatic turning to fixate a moving The distribution of rods and cones is not uniform object in peripheral vision is often an unconscious be- over the retina. The fovea at the centre of the retina haviour. contains the vast majority of cones, and a smaller cen- The human visual system has field of view (FOV) tral area within this contains only cones. It is in this approximately 180 ° (binocular) horizontally and 120 ° region that visual acuity is maximum, and images vertically. Figure 1 shows typical visual fields of each formed outside of this region decrease in acuity the eye, mapped out on a perimeter[6, p. 451]. Commer- further away they are from the centre. From the fovea cially available head-mounted displays (HMDs) do not outwards, the rod density at first increases from zero achieve anything like this. For example, the authors to 150,000 per mm 2 at 18 ° from the fovea, and then measured the Virtual Research Flight Helmet TM to decreases to less than 40,000 per per mm 2 at the edge. have a FOV of approximately 75 ° in the horizontal Images that focus on the periphery of the retina are and 50 ° in the vertical, which is superior to many other received by the rods alone, which are therefore re- HMDs. A wider FOV can be achieved by reducing the sponsible also for peripheral vision. It should be noted extent of binocular overlap of the images presented to that images in the periphery are not clear[ 3 ]. If acuity each eye. However, experimental evidence suggests[7] in the fovea centralis is 20-20, then in the peripheral that making the extent of overlap less than 100% de- regions it is of the order 20-400. creases the level of performance, for example, in visual Movement detection rather than clarity is an im- search tasks. Therefore, for the foreseeable future portant function of the peripheral regions; events de- HMDs are likely to impose a great loss in peripheral tected in the periphery trigger a change in direction of vision for participants in IVEs. The effects of this on gaze, causing the eye and/or head to turn towards the performance, and sense of presence are unknown, al- direction of the event. A standard textbook on physi- though [ 8 ] reports that immersion requires at least a ology states, "Even after the visual cortex has been 60 ° FOV. destroyed, a sudden visual disturbance in a lateral area The work of the present authors to date has con- of the visual field will cause immediate turning of the centrated on factors influencing the sense of presence eyes in that direction.... In addition to causing the in architectural walkthrough applications[9-11]. In eyes to turn towards the visual disturbance, signals are particular, we have concentrated on the influence of also relayed from the superior colliculi through the bestowing a virtual body (VB) on participants, which medial longitudinal fasciculus to other levels of the is of particular importance in the architectural walk- brain stem to cause turning of the whole head and through context. Our pilot experiments have suggested even of the whole body toward the direction of the strongly that the VB does play a significant role in en- disturbance" [4, p. 567 ]. Cotter writes: "The superior hancing participants' sense of presence. This is slightly colliculus is involved in integration of a variety of sen- surprising since the relatively small FOV implies that sory stimuli and mechanisms of attention that are im- most of the time subjects in the experiments were not portant in eye movements.., it has been hypothe- visually aware of their VB, although the VB would sized that the accessory optic system coordinates head come into view at crucial moments--for example, and eye movements so that visual images do not blur while they were reaching or bending to select an object as a result of head movement"[3, p. 14]. Bruce and (Figs. 2 and 3). In order to overcome this problem, Green write: "In the periphery of the visual field motion the experimenters had to supply verbal instructions to serves an orienting function.