Hajime Nagahara* Super Wide Field of View Graduate School of Engineering Science Head Mounted Display Osaka University Using Catadioptrical Optics Osaka, Japan Yasushi Yagi The Institute of Scientific and Industrial Research Osaka University Abstract Osaka, Japan Many virtual reality, mixed reality, and telepresence applications use head mounted Masahiko Yachida displays (HMD). HMD systems are portable and can display stereoscopic images. Graduate School of Engineering However, the field of view (FOV) of commercial HMD systems is too narrow for Science conveying the feeling of immersion. The horizontal FOV is typically around 60°, Osaka University significantly narrower than that of the human eye. In this paper, we propose new Osaka, Japan display optics for a super wide FOV head mounted display. The proposed optics consists of an ellipsoidal and a hyperboloidal mirror that will display distortionless images by using the characteristics of the mirrors, even if the image has a large FOV. We constructed a prototype HMD system with a 180° horizontal ϫ 60° ver- tical FOV that includes the peripheral vision of the human eye. The FOV has a 60° ϫ 60° overlap area that can display stereoscopic images. We estimated the resolution, focus, and aberration of the prototype in an optical simulation and ex- perimentally confirmed that the prototype displays distortionless wide FOV images. 1 Introduction Many virtual reality, mixed reality, and telepresence applications use head mounted displays (HMD). HMD systems are portable and can display stereo- scopic images. However, the field of view (FOV) of commercial HMD systems is too narrow for conveying a feeling of immersion. The horizontal FOV of many such systems is around 60°, significantly narrower than that of the hu- man eye. On the other hand, immersive projection displays (IPD) such as CAVE (Neira, Sandin, & DeFanti, 1993) usually have been used for applica- tions that display large FOV images surrounding users. Moreover, the IPD is not easy to use, because of its high cost and its huge size. Hence, an inexpen- sive and compact HMD with a wide FOV is being sought for such applica- tions. In this paper, we describe a super wide field of view head mounted dis- play consisting of an ellipsoidal and a hyperboloidal curved mirror. The horizontal FOV of the prototype HMD we constructed is 180° and so it in- cludes human peripheral vision. It is well known that the lack of peripheral vision seriously influences pos- tural control in humans (Dickinson & Leonard, 1967). Furthermore, Furness (1990) reported that a FOV over 80° was required for a feeling of immersion. Presence, Vol. 15, No. 5, October 2006, 588–598 © 2006 by the Massachusetts Institute of Technology *Correspondence to [email protected] 588 PRESENCE: VOLUME 15, NUMBER 5 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/pres.15.5.588 by guest on 02 October 2021 Nagahara et al. 589 Takahashi et al. (Takahashi, Arai, & Yamamoto, 1998) 140° FOV prototype HMD system using four liquid also measured the influence of a wide FOV on human crystal display (LCD) panels and fresnel lenses. The attitude control. They concluded that an HMD with a FOV was stitched over the panels and the displayed im- wide FOV, such as 140°, was better than a typical ages had seams. Inami et al. (Inami, Kawakami, Maeda, HMD. Caldwell et al. (Caldwell, Reddy, Kocak, & Yanagida, & Tachi, 1997) developed a prototype HMD Wardle, 1996) reported that a narrow FOV hindered that extended the horizontal FOV to 110° monocular task efficiency and some of the feeling of reality in tele- by using Maxwellian optics. The prototype realized a operations, even though they compared only 30° and large FOV with simple optics, but the size of the ob- 60° FOVs. These studies indicate that a wide FOV served pupil was extremely small. These examples indi- along with peripheral vision are important factors for cate that while there are a number of wide FOV HMDs, increasing the feeling of immersion. The sense of reality their FOVs are still smaller than that of the human a user gets from viewing a display is affected by factors vision system. such as image resolution, dynamic range, field of view, In this paper, we propose new HMD optics to realize and whether the view is stereoscopic (Bolas, 1994). We a super wide FOV. The optics use catadioptrical optics focus on the sense of immersion and vection (visually consisting of a hyperboloidal convex and an ellipsoidal induced perception of self motion) given by human pe- concave mirror. They achieve a distortionless wide FOV ripheral vision, and define these qualities as the sense of by using simple optics. Several previous HMD have also reality in this paper. used catadioptrical optics that have reflective devices In previous studies, various manufacturers and re- and refractive devices. However, none has exploited hy- searchers have developed HMDs (Robinett & Rolland, perboloidal and ellipsoidal characteristics. We con- 1992). Some of them have wide FOVs, such as for a structed a prototype HMD unit that for each eye covers flight simulator. Eyephone02 (VPL Inc.) has an 80° a 120° horizontal view and a 60° vertical view. Thus, a horizontal FOV. Gemini-Eye 3 (CAE Inc.) increased 180° horizontal FOV is achieved for both eyes and a the horizontal FOV to 100°. Sim Eye XL100A (Kaiser stereopsis view can be achieved within a 60° horizontal Electro-Optics Inc.) also has a horizontal FOV of 100°. and 60° vertical FOV. The prototype HMD covers the Datavisor 80 (n-Vision Inc.) has a 120° FOV. Most whole human FOV including peripheral vision. In the commercial HMD systems treat their optics simply as a next sections, we describe the concept of our HMD and magnifier and use eyepiece-based optics such as LEEP the omnidirectional video-based virtual reality system of optics (Howlett, 1983, 1992). As the FOV increases in our prototype HMD. We also describe simulation and such systems, the barrel-like distortion introduced by experimental results confirming the characteristics of the the optics is also enlarged. This distortion has usually prototype HMD. been ignored. Moreover, eyepiece-based optics have a trade-off between the FOV and eye relief (clearance be- tween eye and lens). Hence, the eyepiece lenses have 2 Proposed HMD Optics restricted the FOV. Some HMDs were made with unique, possibly better, designs. Fiber-Optic HMD Figure 1 shows the proposed HMD optics consist- (CAE Inc.; Welch & LaRussa, 1998) has a 120° FOV. ing of planar, hyperboloidal, and ellipsoidal mirrors, a HMD 120/40 (SEOS Inc.; Coates & Huxford, 2004) lens, and an LCD. The lens is aligned on the focus of that uses mirrors has a 120° FOV. However, these de- the hyperboloidal mirror. The planar mirror between vices are used for shrinking the size of optics and not for the lens and the hyperboloidal mirror inclines the rays focusing or magnifying rays. Therefore, the problem of to avoid interference with the observer’s face. The axis conventional eyepiece-based magnifiers has remained an of the ellipsoidal mirror is inclined to avoid the hyper- issue with these optics. boloidal mirror obstructing the FOV of the observer’s Takahashi, Arai, and Yamamoto (1998) constructed a eye. The proposed optics does not require an eyepiece Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/pres.15.5.588 by guest on 02 October 2021 590 PRESENCE: VOLUME 15, NUMBER 5 Figure 1. Optics of the proposed catadioptrical unit. lens system, so the FOV is not limited by the lens size of the eyepiece lens system. Using this property, the proto- type HMD can display an FOV virtual image of about 120°h ϫ 60°v to the observer. Figure 2 shows the hyperboloidal, ellipsoidal mirrors and their geometric relation. In this figure, bold lines indicate the hyperboloidal and ellipsoidal parts of the HMD optics. The hyperboloid and the ellipsoid are de- fined by Equations (1–2), and each has two focal points, Ϫ Fhϩ(0, 0, ch), FhϪ(0, 0, ch) and Feϩ(0, 0, ce), Ϫ e e e T FeϪ(0, 0, ce). The ellipsoidal coordinate [ x, y, z] is rotated and shifted from the hyperboloidal coordinate [x, y, z]T as described in Equation (3). M is the rotation and translation matrix. The focal point Fhϩ of the hy- perboloidal mirror is aligned with the focal point Feϩ of Figure 2. Characteristics of (a) hyperboloid and (b) ellipsoid. the ellipsoidal mirror. x2 ϩ y2 z2 Ϫ ϭϪ 2 2 1 (1) ah bh shown in Figure 2b. This relation is the same as Snell’s e ˜ e ˜ reflective law. Here, V1 and V2 are defined as ray vec- e 2 e 2 e 2 z e ˜ e ˜ x ϩ y tors along these lines, and V1 is reflected to V2 as de- ϩ b2 ϭ 2 e 1 (2) scribed in Equations 5 and 6, where ␭ in Equation (5) ae 1 is derived from Equations (2) and (5). where e ϭ V˜ 1 MV˜ 1 (4) 2 ϩ 2 ϭ 2 2 Ϫ 2 ϭ 2 ah bh ch, be ce ae e ϭ ␭ e ϩ e Pe 1 V˜ a FeϪ (5) ͓ex,ey,ez,1͔T ϭ M ͓x,y,z,1͔T (3) e ˜ ϭ e Ϫ e e V2 Feϩ Pe (6) The normal vector at an arbitrary point Pe on the ellipsoid bisects the angle between lines passing through Because the focal point Fhϩ of the hyperboloidal mir- e e e Pe and Feϩ or Fe Ϫ in the ellipsoidal coordinates as ror is set on the focal point Feϩ of the ellipsoidal mirror, Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/pres.15.5.588 by guest on 02 October 2021 Nagahara et al.