Development of a Real 3D Display System

2020 IEEE 20th International Conference on Software Quality, Reliability and Security Companion (QRS-C)

Development of A Real 3D Display System

Chong Zeng Weihua Li Hualong Guo College of Mathematics and College of Mathematics and College of Mathematics and information Engineering information Engineering information Engineering Longyan University Longyan University Longyan University Longyan, China Longyan, China Longyan, China

Tung-lung Wu School of Mechanical and Automotive Engineering Dennis Bumsoo Kim Zhaoqing University College of Mathematics and information Engineering Zhaoqing, China Longyan University Longyan, China

Abstract—This paper introduces a three-dimensional light achieve high fidelity of the reproduced objects by recording field display system, which is composed of a high-speed total object information on the phase, the amplitude, and the projector, a directional scattering mirror, a circular stainless- intensity at each point in the light wave [4]. steel bearing plate, a rotating shaft and a high-speed micro motor. The system reduces information redundancy and Recently, Cambridge Enterprise invented a novel 360e computational complexity by reconstructing the light intensity 3D light-field capturing and display system which can have distribution of the observed object, thus generating a real three- applications in gaming experiences, an enhanced reality and dimensional suspended image. The experimental results show autonomous vehicle infotainment and so on [5]. Motohiro et that the suspension three-dimensional image can be generated al. presented an interactive 360-degree tabletop display system by properly adjusting the projection rate of the image and the for collaborative works around a round table so that users rotation speed of the rotating mirror (i.e. the motor speed). The could see 3D objects on the tabletop display anywhere around clarity and accuracy of a three-dimension display depending on the table without 3D glasses [6]. Nonetheless, there are some the number of slices selected, meaning that the more slices can disadvantages which are huge amount of the information, be projected per minute, the finer the three-dimension display. redundancy of holographic images, and a limit of visibility at Finally, this study provides the basic parameter matching, certain angles. Thus, it is needed to develop a new real 3D which shows the feasibility of developing a real light field 3D display which overcomes such problems and reconstructs 3D display system with stable performance, strong portability, easy implementation, and low cost. In short, naked-eye 3D allows the images with the appropriate light filed distribution to show the observer to view objects from any angle and direction, without object images in every direction. This study purposes to the need for any auxiliary tools (such as glasses, helmets, etc.). suggest a prototype real 3D display based on principles of the persistence of vision and a light field with CMOS Keywords— real 3D, DLP, DMD, high-speed projector, light (complementary metal̢oxide̢semiconductor) image field display sensors, Infrared sensors and DLP (digital light processing). The system expects to project images through a single high-

I. INTRODUCTION speed projector, with the light filed distribution in every In stereoscopic vision, the eye can capture two different direction of the object. The new real 3D display is expected to views of a 3D object, and the retinal image is merged in the apply for many fields of industries for education, medical, observer's brain to produce a three-dimensional image of the manufacturing and so forth. object, by converting parallax to depth perception. The

distance determination is possible through the stereopsis II. RELATED WORK usually with binocular depth perception owing to the Since Levoy et al. created light fields through large-scale horizontal separation and disparity of two eyes [1]. The 3D rendering and digital image arrays to generate views that can display is visually persistent, in which light stays in the eye for be viewed from any angle within a certain range, light fields a short time after it stops entering, leading to optical illusions. have become one of the important tools in computational Geng defined a perfect 3D display to be a “window to the photography [7]–[9]. Zwicker et al. proposed a method to world” through which viewers can perceive the same 3D scene record the light fields include camera aperture, the number of as the viewers see the real object. Although many of enhanced views rendered around the circle, the amount of horizontal 3D images have depth or volume, a flat screen can only show diffusion [10]. There are many displays based on light fields, still only 2D images due to the nature of the 2D display [2]. such as near field display [11], compressed light field display Beigzadeh noted that the primary real (perfect) 3D display [12]–[14], and glasses-free light field displays [15]. In technology involved interference and diffraction to record and particular, the emergence of autostereoscopic display reconstruct a real 3D object like holography which can achieve technology based on light field display makes 3D display high fidelity of the reproduced object by recording total object develop significantly. The earliest autostereoscopic display information of the phase, amplitude, and intensity at each adopting three high-speed DLP projectors as display point in the light wave [3]. The primary real 3D display components were with an observable 3D field of view of only technology considers interference and a diffraction to record 180 degrees [8]. Then Andrew et al. introduced a set of and reconstruct 3D objects like a holography which can rendering technology for interactive horizontal parallax light

978-1-7281-8915-4/20/$31.00 ©2020 IEEE 644 DOI 10.1109/QRS-C51114.2020.00109 field display systems, in which only a high-speed DLP degrees, and the scattering rate in the Y direction was projection, a rotating mirror, and FPGA circuit and so on to more than 100 degrees. present 360-degree three-dimensional surround display [4]. Since then, various autostereoscopic display technology based The carrying plate is made of solid stainless steel for on light field display has been emerging ongoing. For fixing the diffuser and the rotating shaft is connected example, a digital projector array and a micro-lens enhanced to the micromotor. Considering the huge eccentricity projection screen are used to simulate the light field of a given generated by high-speed rotation, it is necessary to 3D scene to form an automatic stereo display [16]; Douglas et accurately calculate the load bearing and center of al. illustrates a dynamic light field display adopting multi- gravity. layered LCDs [17]; Xia et al. created the light field of real 3D The system requires a proportional reducer to reduce scene floating in the air, using high-frame-rate color projector eccentricity, improve power control, and improve and flat light field scanning screen [18]; A 360-degree light equipment stability. The speed of the micromotor field 3D display system is proposed, which consists of a liquid needs to be adjusted between 1000-1500 rpm to reach crystal display, a new triplex lens array, and a holographic the rotating frequency of the diffuse reflector. functional screen .

III. SURROUND LIGHT FIELD DISPLAY SYSTEM 濼瀀濴濺濸澳濛濼濺濻激瀆瀃濸濸濷澳 A. Principles and Main Components of the System 瀃瀅瀂濽濸濶瀇瀂瀅澳 1) Persistence of vision: This makes a reference to the optical illusion that occurs when visual proceeding from it has ceased to enter the eye [19].

2) Light field: The surround 3D display reconstructs the 濗濼瀅濸濶瀇濼瀂瀁濴濿澳 light field of all light rays in 3D objects from and through 濦濶濴瀇瀇濸瀅濼瀁濺澳 every direction and point. The light rays are captured using 濠濼瀅瀅瀂瀅澳 the 5D (five dimensional) plenoptic function in three 澳 coordinate axis and two angles to specify the directions of 3D objects. Removing any redundant information form the five- dimensional plenoptic function makes it easy to record the 濖濴瀅瀅瀌濼瀁濺澳 light field. A traditional digital camera only captures 2D 濗濼瀆濶澳 images and its image sensor records colors and brightness of 濥瀂瀇濴瀇濼瀁濺澳 all light rays in every individual pixel. However, a light field 濦濻濴濹瀇澳 camera records not only the brightness and colors, but also 濠濼濶瀅瀂澳濠瀂瀇瀂瀅澳 directions of all the light rays reaching to its sensor. Then, the 瀅 camera refocuses the images [20].

3) Digital Light Processing (DLP): The high-speed projector (DLP) is used to project the preprocessed image (a Fig. 1. The model of Surround 3D Display light field image) onto the high-speed rotating scattering mirror. A DLP is a set of chipsets that use a DMD (digital micromirror device) [4]. B. Eq The Model of the Surround 3D Display System Figure 1 shows the imaging structure of the 3D display system, which is consisted of a high-speed projector, a directional scattering mirror, a circular stainless steel carrying 澳 disk, a rotating shaft, and a micro electric motor. 3D Max was used to model the observed object and 瀂澳 sliced the image. High - speed projector adopts DMD - based DLP with 瀌澳 circuit module. It has high - speed graphic sequence storage and display function, and is equipped with high - performance, high - resolution spatial light modulator. The projected graphic sequence ranges from 150hz /s to 200hz /s to match the rotation speed of the scattering mirror. The DSM (directional scattering mirror) is placed on the bearing plate at a 45-degree angle to the horizontal plane. As shown in Fig. 2, its center of gravity should Fig. 2. The Diagram of DSM be the center of gravity of the disk to ensure stability. Meantime, the mirror surface is optically coated by using the Light Shaping Diffusers (LSD), so that the scattering rate in the X direction was less than 2

645 C. The principle of the real 3D Display B. Prevention of Vibration As shown in Fig. 2, DLP is projected onto the DSM surface It is very important to stabilize the equipment without and light is scattered to the surrounding environment at a fixed vibration for better images. When the motor rotates at 1800 angle. When the system is operating, the slice image is rpm, the rotating mirror structure shakes and swings projected by DLP to the high-speed rotation DSM, then the rigorously. TABLE2 shows the specification of the number of its Angle of view µ, namely the scattering Angle various sizes and materials for the equipment. Turntable ¥ of the image per rotation, is: 3 and Mirror 3 chose to prevent vibrations.

360 . (1) The excessive vibration may come from: 1) when the centers of gravity of the turntable components The position of each image projected by DLP when the are not the same. The machining accuracy should be scattering mirror rotates at high speed is LLL guaranteed, and the shaft holes must fit with the shafts exactly. 2,1 n 2) when the system design does not coordinate turntables The principle of the 3D light field display system is that with the entire system. the scattered light at each position is perpendicular to the human eye, and the observed view consists of light at each Therefore, we consider the following measures to prevent position. Based on the persistence of vision and the parallax the vibration: principle of the human eye, the observer can observe the three- dimensional suspension display. 1) The centers of gravity must be in the same position. This needs the precision machining of the parts. IV. THE REAL 3D DISPLAY SYSTEM 2) The total weight of the system must have proper weight A. The system configuration to balance itself and to absorb the vibration caused by high- The model structure of the system is simulated by speed rotation of the turntable assembly. Thicker steel and SolidWorks®, a 3D software. The detailed information of its aluminum plates and goblets with rubber are used for shock structure is shown in Fig. 1, while the actual system is shown absorbing. in Fig. 3. 3) We optimize the size and the quality of turntables and 1) The T-bracket and L-shaped slide rail (made of brackets to eliminate the wind resistance problem during aluminum) assembly form an upright bracket that is rotation. embedded in the high-speed projector and move freely to 4) Through SolidWorks® three-dimensional modeling, adjust the image size. the quality attribute of the bracket and the lens assembly is 2) The stand for the system is made of 12 different kinds analyzed first. The centers of mass must be aligned to of aluminum materials, and acrylic plates are added around it assemble each part. Finally, the center of mass of the overall for protection. assembly is obtained through the quality attributes.

3) A 10 mm thick stainless-steel plate is installed in the 5) Simulations with SolidWorks® are carried out for the middle to fix the motor and to support the balancing weight dynamic balance optimization. High-speed rotation of plate. irregular objects produces a greater eccentricity, which causes 4) The rotating mirror with steel plate which surface is severe vibration of the overall equipment. Therefore, we covered by LSD optical scattering film. Specification of the establish a motion example, adding components and sensors LSD is described in TABLE1. virtually by the simulation function of SolidWorks®. Then, 5) The holographic technology is used to form a micro- we obtain the best parameters of turntable’s diameter, triangle lens structure on the film surface. The light passes through the bracket’s thickness and turntable’s thickness by using film with the preset scattering angle to be diffused evenly, automatic optimization calculation. Thus, turntable 4 has which achieves a clear visible 3D imaging effect. 197.22 mm of diameter, 15 mm of triangle bracket’s thickness 6) A high-speed micro motor is located under the and 17 mm of thickness. stainless-steel plate and connected to the rotating mirror TABLE1 LSD SPECIFICATION through the rotating shaft.

Property Specification

Diffuser type Elliptical diffuser sheet

Transmission 88%±1% (Spectrophotometer)

LSD Angle 60 o (FWHM)

Angle Tolerance 6.0 x ± 0.5 degree

400 to 850 nm Fig. 3. The equipment of the real 3D display system Transmission spectral range (Spectrophotometer)

646 TABLE2 EQUIPMENT SPECIFICATION and the micro motor speed of 700 rpm. Fig. 6 (b) is an image with the micro motor speed of 1600 rpm.

Length Width Height Items Materials (mmͤ ͣmmͤ ͣmmͤ Turntable 45steel 190 190 14 1 Turntable 303stainless 180 180 17 2 steel

Turntable 303stainless 190 197.22 17 3 steel stainless Mirror 1 148 100 1 steel stainless Mirror 2 195 140 1 澻濴澼澳 steel stainless Mirror 3 180 130 1 steel

V. RESULTS AND DISCUSSIONS A. The real 3D display quality by various rotational speed and projection rate The performance of the system was assessed by testing and comparing its graphical display performance under different motor speed and different output power of the DLP. We used an original 3D image of a teapot from 3 DS Max image processing of Autodesk®. Fig. 4 shows the teapot image to be projected onto a stationary directional scattering mirror after the processing. 澻濵澼澳

Fig. 5. (a) The projection rate is 120 frames/s (b) The projection rate is 200 frames/s

The best match with the projection rate of 200 frames/s was the micro motor speed of 1420 rpm. In this match, the stereoscopic imaging of the teapot was displayed steadily on the disc. When we slowed down the rotational speed, the outline of the image was blurred. However, higher than 1420 rpm also created blurred images as shown in Fig. 6 (c).

Fig. 4. The projected teapot models

The number of views for one revolution of the scattering mirror is 36 with the rotation angle up to 10 according to Eq. (1), a total of 36 slice views were projected onto the directional scattering mirror in sequence. Fig. 5 (a) and (b) shows the symmetric stereoscopic image on the disc, with 120 frames/s of DLP projection rates and 200 frames/s when the micro motor speed is 700 rpm (revolutions per minute). The higher projection rate gives the finer outline of the displayed image, though a complete stereoscopic image is not presented yet. 澻濴澼澳 Figure 6 (a) shows an image in red which is the basic outline of the teapot with DLP projection rate of 200 frames/s

647

澻濵澼澳 Fig. 8. Different display effects in different background light

The results implied that matches of a projection rate with a rotation speed of the motor can create clear and stable stereoscopic 3D images. In this study, rotation angle of 1 (360 views for one revolution) with a projection rate of 120 frames/s and a motor speed of 1420 rpm produced the best images. More slice views with an appropriate projection rate and a motor speeds seem to create more stable and clearer 3D images. However, more trials for the best matches between the parameters are needed with the same system to establish a real 澻濶澼澳 3D display system.

VI. CONCLUSION Fig. 6. (a) Image at the motor speed of 1000 rpm This study demonstrates that a real 3D display system can (b) Image displayed at a motor speed of 1420 rpm be achieved by reconstructing the dedicated light field. The experiment results manifested that the system presents clear (c) Image displayed at a motor speed of 1420 rpm and stable stereoscopic images without auxiliary tools. In addition, the design for the system in this study provides a safe, When we set the rotation angle as 1 degree, Eq. (1) gave 360 durable and lightweight device with improved portability and views for one revolution of the scattering mirror. That is, a practicality. Increased sensitivity of sensors of the system is total of 360 slice views are projected onto the disc in sequence. expected to enhance the quality of real 3D images The DLP projection rate of 120 frames/s with the micro motor considerably. This device has a limited observable range for rate of 1420 rpm brought out clear 3D image of the teapot the observer, with a limited image display size. The displayed steadily on the disc. Fig. 7 and Fig. 8 shows the performance of tabletop light field 3D displays is limited by images which look slightly different in different light low resolution and narrow viewing angle. Du et al. proposed conditions. Since the displayed images are all semi- a floating 3D light field display based on spatial data transparent images, the darker background light produced reconstruction with a large horizontal perspective [21]. clearer views from viewers’ standpoint in every direction Recently, Cambridge Enterprise [5] and Hirayama [22] have integrated true 3D display with multiple senses such as visual, auditory, haptic, etc., greatly expanding the practicality and usability of products. Finally, we expect an infrared sensing technology to increase interactivity and a deep learning algorithm to be added to enable the system to achieve real- time dynamic transmission and display. There are wide ranges of commercial applications for the real 3D display such as education, health care, and public services, etc. Specifically, it can be adopted as teaching aids to improve children's cognitive abilities, such as real 3D globe display, space exploration simulation, planetary knowledge popularization and so on; Or, as a medical auxiliary product, to promote health and medical advice for elderly patients with weak professional medical knowledge, such as the demonstration of the principle of rheumatic joint diseases, and the popularization of common chronic diseases such as Fig. 7. Stereoscopic 360 ° suspension display, a projection rate of 120 frames/s respiratory system and circulatory system; Or as a public and a motor speed of 1420 rpm. service product, increasing and protecting the rights and interests of vulnerable groups in specific environments, such as providing real 3D simulation tests for humans with physical disabilities etc..

648 ACKNOWLEDGMENT Dual-Layer 3D Displays using Low-Rank Light Field This work was supported by “2019 Fujian Province Young Factorization,” ACM Trans. Graph., vol. 29, no. 6, and Middle-aged Teacher Education Research Project pp. 1–10, 2010, doi: 10.1145/1882261.1866164. (JAT190740)”, China. [13] G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar, “Tensor displays: Compressive light field synthesis REFERENCES using multilayer displays with directional [1] H. Wu et al., “Evaluating stereoacuity with 3D backlighting,” ACM Trans. Graph., vol. 31, no. 4, shutter glasses technology,” BMC Ophthalmol., vol. 2012, doi: 10.1145/2185520.2185576. 16, no. 1, 2016, doi: 10.1186/s12886-016-0223-3. [14] A. Maimone, G. Wetzstein, M. Hirsch, D. Lanman, [2] J. Geng, “Three-dimensional display technologies,” R. Raskar, and H. Fuchs, “Focus 3D: Compressive Adv. Opt. Photonics, vol. 5, no. 4, p. 456, 2013, doi: Accommodation Display,” ACM Trans. Graph., vol. 10.1364/aop.5.000456. 32, no. 5, 2013, doi: 10.1145/2503144. [3] A. M. Beigzadeh, M. R. R. Vaziri, and F. Ziaie, [15] F. C. Huang, G. Wetzstein, B. A. Barsky, and R. “Modelling of a holographic interferometry based Raskar, “Eyeglasses-free display: Towards calorimeter for radiation dosimetry,” Nucl. correcting visual aberrations with computational light Instruments Methods Phys. Res. Sect. A Accel. field displays,” ACM Trans. Graph., vol. 33, no. 4, Spectrometers, Detect. Assoc. Equip., vol. 864, pp. 2014, doi: 10.1145/2601097.2601122. 40–49, 2017, doi: 10.1016/j.nima.2017.05.019. [16] Y. Zhang, “I-TASSER server for protein 3D structure [4] A. Jones, I. McDowall, H. Yamada, M. Bolas, and P. prediction,” BMC Bioinformatics, vol. 9, 2008, doi: Debevec, “Rendering for an interactive 360° light 10.1186/1471-2105-9-40. field display,” ACM Trans. Graph., vol. 26, no. 3, [17] P. S. Douglas et al., 2007, doi: 10.1145/1276377.1276427. “ACCF/ASE/AHA/ASNC/HFSA/HRS/SCAI/SCC [5] University of Cambridge Enterprise, “360° 3D Light- M/SCCT/SCMR 2011 Appropriate Use Criteria for field Display,” 2019. . Echocardiography,” J. Am. Coll. Cardiol., vol. 57, [6] M. Makiguchi, D. Sakamoto, H. Takada, K. Honda, no. 9, pp. 1126–1166, 2011, doi: and T. Ono, “Interactive 360-Degree Glasses-Free 10.1016/j.jacc.2010.11.002. Tabletop 3D Display,” pp. 625–637, 2019, doi: [18] X. Xia et al., “A 360-degree floating 3D display 10.1145/3332165.3347948. based on light field regeneration,” Opt. Express, vol. [7] M. Levo and P. Hanrahan, “Intramolecular vs . 21, no. 9, p. 11237, May 2013, doi: Intermolecular Forces Intramolecular Forces,” pp. 1– 10.1364/oe.21.011237. 8, 1996, doi: 10.1145/237170.237199. [19] T. Yamaguchi, “Volume hologram printer to record [8] S. J. Gortler, R. Grzeszczuk, R. Szeliski, and M. F. the wavefront of three-dimensional objects,” Opt. Cohen, “Lumigraph,” ACM SIGGRAPH Conference Eng., vol. 51, no. 7, p. 075802, 2012, doi: on Computer Graphics, New York, 1996, pp. 43–54, 10.1117/1.oe.51.7.075802. doi: 10.1007/978-3-030-03243-2_8-1. [20] R. Ng et al., “Light Field Photography with a Hand- [9] A. Isaksen, L. McMillan, and S. J. Gortler, held Plenoptic Camera,” Main, pp. 1–11, 2005. “Dynamically reparameterized light fields,” ACM [21] J. Du et al., “Large viewing angle floating three- SIGGRAPH Conf. Comput. Graph., New York, pp. dimensional light field display based on the spatial 297–306, 2000, doi: 10.1145/344779.344929. data reconstruction (SDR) algorithm,” Opt. [10] M. Zwicker, W. Matusik, F. Durand, H. Pfister, and Commun., vol. 475, 2020, doi: C. Forlines, “Antialiasing for automultiscopic 3D 10.1016/j.optcom.2020.126229. displays,” ACM SIGGRAPH 2006 Sketches, [22] R. Hirayama, D. Martinez Plasencia, N. Masuda, and SIGGRAPH ’06, p. 2005, 2006, doi: S. Subramanian, “A volumetric display for visual, 10.1145/1179849.1179983. tactile and audio presentation using acoustic [11] D. Lanman and D. Luebke, “Near-eye light field trapping,” Nature, vol. 575, no. 7782, pp. 320–323, displays,” ACM Trans. Graph., vol. 32, no. 6, 2013, 2019, doi: 10.1038/s41586-019-1739-5. doi: 10.1145/2508363.2508366. [12] D. Lanman, M. Hirsch, Y. Kim, and R. Raskar, “Content-Adaptive Parallax Barriers: Optimizing

澳

649