Electronic Tabletop Holographic Display: Design, Implementation, and Evaluation

applied sciences

Article Electronic Tabletop Holographic Display: Design, Implementation, and Evaluation

Jinwoong Kim 1,* , Yongjun Lim 1, Keehoon Hong 1, Hayan Kim 1, Hyun-Eui Kim 1, Jeho Nam 1, Joongki Park 1, Joonku Hahn 2 and Young-ju Kim 3

1 Broadcasting and Media Research Laboratory, Electronics and Telecommunications Research Institute, 218 Gajeong-ro, Yuseong-gu, Daejeon 34129, Korea; [email protected] (Y.L.); [email protected] (K.H.); [email protected] (H.K.); [email protected] (H.-E.K.); [email protected] (J.N.); [email protected] (J.P.) 2 School of Electronics Engineering, Kyungpook National University, 80 Daehakro, Bukgu, Daegu 41566, Korea; [email protected] 3 Yunam Optics Inc., 33-7 Eongmalli-ro, Majang-myeon, Icheon-si, Gyeonggi-do 17389, Korea; [email protected] * Correspondence: [email protected]

Received: 19 January 2019; Accepted: 15 February 2019; Published: 18 February 2019

Abstract: Most of the previously-tried prototype systems of digital holographic display are of front viewing ﬂat panel-type systems having narrow viewing angle, which do not meet expectations towards holographic displays having more volumetric and realistic 3-dimensional image rendering capability. We have developed a tabletop holographic display system which is capable of 360◦ rendering of volumetric color hologram moving image, looking much like a real object. Multiple viewers around the display can see the image and perceive very natural binocular as well as motion parallax. We have previously published implementation details of a mono color version of the system, which was the ﬁrst prototype. In this work, we present requirements, design methods, and the implementation result of a full parallax color tabletop holographic display system, with some recapitulation of motivation and a high-level design concept. We also address the important issue of performance measure and evaluation of a holographic display system and image, with initial results of experiments on our system.

Keywords: digital hologram; holographic display; tabletop display; hologram measurement

1. Introduction Digital electroholographic display technology has been believed to be an ultimate solution for 3-dimensional displays due to its theoretically perfect 3-dimensional visual cue support without any conflict among them. There has been active research and development of many prototype systems, though most suffer from critical performance limitations such as small image size or narrow viewing angle. We can categorize the realization of electronic dynamic holographic displays into two groups: one is a flat panel-based direct front viewing type and the other is tabletop type which renders volumetric 3D images floating on or above the display. The former type can render extruding 3-dimensional images in front of or behind the display, which is not much different from conventional stereoscopic or multiview 3D displays. Almost all displays, including TVs, PC monitors, information kiosk, and smartphones, are of the former type and it is natural trying to make a 3-dimensonal display of the same type. However, holograms and holographic displays are expected to have more powerful 3D rendering capability with volumetric 3D images looking much like a real object. This expectation is universally depicted in many SF movies including old classic Star Wars or the very recent Avatar. Moreover, current prototype systems of the former type reported so far have a very narrow viewing

Appl. Sci. 2019, 9, 705; doi:10.3390/app9040705 www.mdpi.com/journal/applsci Appl. Sci. 2019, 9, 705 2 of 16 angle, so that even the basic requirements, such as supporting motion parallax or binocular parallax, are hardly satisfied. In the authors’ opinion, a tabletop display is the more desirable type of electronic holographic display realization. Tabletop display can show full perspective views of an object or scene from 360 degree around, so that it gives viewers much better perception of the whole shape, volume, and relation among parts of the scene or objects. Miniaturized scene or moving objects on a tabletop display, rendered by holographic principle, will surely give excellent realism that cannot be achieved by front-viewing displays. In this regard, we believe that the tabletop holographic display has much wider application areas. We have successfully implemented several versions of tabletop holographic display system with commercially available DMD (Digital Micromirror Device) for SLM (Spatial Light Modulator) device. We have previously reported on the first version of the system, which is monocolor one [1]. We have upgraded the system to a full color version, and the system can, for the first time, render color holographic video, which can be watched by multiple viewers at the same time, freely from 360 degrees around the display. In this article, we report on the enhanced version of the system, which is full-color, full parallax with extended vertical viewing range, and address initial experimental result of performance measurement and evaluation of the system. In Section2, the prior art of the holographic display implementation is briefly reviewed. In Section3, the overall design concept and methodologies of the target system are capitulated, followed by implementation details of the enhanced parts and experimental results including the system configuration and rendered images are given in Section4. In Section5, we address on the performance evaluation issues including definition of metrics, measurements, and evaluation regarding each metric, followed by conclusions and discussions on further issues.

2. Prior Art of Holographic Display Implementation Since Prof. Benton of MIT Spatial Media Group introduced the ﬁrst prototype system of electronic holographic display [2], called Mark-1, there have been several different approaches with different SLM device [3–11]. The basic challenge is overcoming the Space Bandwidth Product (SBP) limitation of the SLM device used, by using spatiotemporal multiplexing techniques for wider viewing angle and bigger image size. Table1 shows the most prominent systems reported so far with their characteristics and speciﬁcations.

Table 1. Electronic holographic display systems.

Systems SLM Architecture Characteristics References MIT Holo-video P: HPO, Multichannel [2] display AOM C: MC (1992), FC mechanical scanning [3] (Mark-I, II, III, IV) (2012) P: HPO, LCD (pixel pitch: Viewing window ISD: 20 inch SeeReal VISIO 20 [4] ~100 um) tracking VA: NA (eye tracking) SNU Curved P:FP Curved spatial tiling of holographic LCD ISD: 0.5 inch [5] multiple LCDs display VA: 22.8◦ ISD: 85 mm NICT Full color [6] 4K LCoS 4 × 4 spatial tiling VA: 5.6◦, display [7] FR: 20 fps Appl. Sci. 2019, 9, 705 3 of 16

Table 1. Cont.

Systems SLM Architecture Characteristics References TUAT Horizontal scanning P: HPO Horizontal [8] DMD with anamorphic ISD: 6.2 inch scanning [9] system VA: 10.2◦ holography Univ. of Arizona Rewritable 6-ns pulsed laser at a P: FP Rewritable printing photorefractive rate of 50 Hz write, ISD: 12 inch [10] display polymer material LEDs for readout VA: NAFR: 0.5 fps P: FP LCD (pixel pitch: Viewing window ISD: 16 inch ETRI [11] ~120 um) tracking VA: NA (eye tracking) P: Parallax (HPO or Full Parallax (FP)), C: Color (Monocolor(MC) or Full color (FC)), ISD: Image Size Diagonal, VA: Viewing Angle, FR: Frame Rate, DT: Data Throughput, NA: Not Available, MIT: Massachusetts Institute of Technology, SNU: Seoul National University, NICT: National Institute of Information and Communications Technology, TUAT: Tokyo University of Agriculture and Technology.

They used almost all different types of SLMs, including acousto-optic modulator (AOM), liquid crystal display (LCD), digital micromirror device (DMD), liquid crystal on silicon (LCoS), and rewritable photorefractive photopolymer, which reflects that none of them has satisfactory capability to build a high quality holographic display system; for an extensive review and analysis of these systems and similar approaches, please refer to [12]. All of these holographic display systems are of the ‘front-viewing’ type implementations. Holographic 3-dimensional images are seen in front or behind the display depending on the content to be rendered. Since the viewing angle is small, it can hardly give any volumetric feeling or perception different from conventional multiview displays, while image size and quality is much inferior to them. So, even though the theory tells strength of the holographic display, practical implementations do not yet realize desired superiority of the technology. This situation is depicted in Figure1. Recently, there have been several other approaches for different type of holographic displays trying to render more volumetric holographic images, either floating in the air [13] or seen from 360 degrees around [14,15]. In reference [13], only one viewer can see the image, and in reference [15], one can see only side views, while the display in reference [14] can show top side views, thus neither one is fulfilling the capability of rendering true volumetric image. On the other hand, there have also been several tabletop display systems demonstrated their quite impressive capability of volumetric 3-dimensional image rendering in the free air, which easily enables some manipulations and interactions with the content [16–18]. These displays shows the usefulness of the tabletop displays, but crucial weakness remains regarding human factor issue, since it cannot support visual accommodation cue of the human 3D vision. This can only be achieved by holographic technology, though super-multiview or integral imaging has also been claimed to have a very similar property [19–21]. In summary, the tabletop holographic display system with rendering capability of more realistic volumetric 3D images is yet to be developed, which is far better than typical ‘front-viewing’ type ones. Appl.Appl. Sci. Sci.2019 2019, ,9 9,, 705 x FOR PEER REVIEW 4 ofof 1616

Figure 1. Current status of state-of-the-art holographic displays. Figure 1. Current status of state-of-the-art holographic displays. 3. Design Concept and Methodologies 3. Design Concept and Methodologies In the following are our design goals of tabletop holographic display implementation. What we had inIn mind the following from the start are our is that design the system goals of should tablet beop designed holographic to evolve display eventually implementation. to a commercially What we viablehad in system. mind from the start is that the system should be designed to evolve eventually to a commercially viable system. (1) The1) size The ofsize the of hologram the hologram image shouldimage should be reasonably be reasonably big enough big soenough that viewers so thatcan viewers perceive can perceivethe 3-dimentional the 3-dimentional volume volume and someand some details details of the of image. the image. (2) The2) hologram The hologram image image should should be seenbe seen by multipleby multiple viewers viewers at at the the same same time, time, and and freely freely from from differentdifferent positions positions and and perspectives perspectives so sothat that each each viewer viewer can can easily easily perceive perceive the whole shapeshape ofof renderedrendered objects objects and andall viewers all viewers can canshare share the thewhole whole scene scene information information in order in order to todiscuss discuss about, about, or collaborativelyor collaboratively manipulate, manipulate, on it. on it. (3) The3) image The image should should be rendered be rendered in such in such a way a way that that binocular binocular disparity disparity and and motion motion parallax parallax is is perceivedperceived to to a asatisfactory satisfactory level. level. In In other other word words,s, the the image image should deliver thethe perceptionperception ofof realreal objectsobjects being being there, there, showing showing all around all around perspect perspectiveive views views of the of object the object with with full fullparallax. parallax. (4) No4) physical No physical apparatus apparatus of the of displaythe display should should occupy occupy the the image image space, space, so so that that the the hologram hologram imageimage is shown is shown in free in freeair, enabling air, enabling manipulation manipulation or interaction or interaction with with the theimage image as if as we if wedo with do with real objects.real objects. (5) High5) High quality quality color color hologram hologram image image with with dynamic dynamic content, content, i.e., i.e., the the hologram hologram video video should should be presented.be presented. Also note that vergence-accommodation conflict-free property should basically be supported sinceAlso the notesystem that is vergence-accommodationreal holographic display. Perf conflict-freeormance propertyspecification should of the basically display be system supported to be sincerealized the is system drawn is from real holographicthese requiremen display.ts, which Performance is summarized specification in Table of 2. the display system to be realized is drawn from these requirements, which is summarized in Table2. Table 2. Performance specification of the system. Table 2. Performance specification of the system. Requirement  Specification 1. Size of the hologramRequirement image  → 7.63Specification cm or bigger 2. 360 degree1. Sizeviewable of the hologramfreely by image → Horizontal viewing7.63 cm orangle bigger 360 degrees  multiple users Vertical viewing angle 20 degrees 2. 360 degree viewable freely by → Horizontal viewing angle 360 degrees 3. Volumetric imagemultiple with users full Vertical viewing angle 20 degrees  parallax Floating image in the center of the display with 45 3. Volumetric image with full → Floating image in the center of the 4. Image should notparallax occupy degrees of downward viewing direction  display with 45 degrees of downward physical space viewing direction 4. Image should not occupy → 5. Color hologramphysical video space  R/G/B full color video rendering 5. Color hologram video → R/G/B full color video rendering A drawing of the system design goal and schematic diagram of the main system specifications are shown in Figure 2. Appl. Sci. 2019, 9, 705 5 of 16

A drawing of the system design goal and schematic diagram of the main system speciﬁcations Appl. Sci. 2019, 9, x FOR PEER REVIEW 5 of 16 areAppl. shown Sci. 2019 in, 9 Figure, x FOR PEER2. REVIEW 5 of 16

FigureFigureFigure 2. 2.2. ( (a(aa)) )Conceptual ConceptualConceptual drawing drawingdrawing of ofof the thethe design designdesign goal. goal.goal. ( (b(bb)) )Schematics SchematicsSchematics of ofof system systemsystem specifications. speciﬁcations.specifications. It is quite challenging to realize these requirements into working display system with currently ItIt is is quite quite challenging challenging to to realize realize these these requirem requirementsents into into working working display display system system with with currently currently available commercial SLM devices, since most of them have a physical size of modulation area less availableavailable commercial commercial SLM SLM devices, devices, since since most most of of them them have have a a physical physical size size of of modulation modulation area area less less than one inch and diffraction angle less than 2 degrees. Figure3 shows this issue in a schematic thanthan oneone inchinch andand diffractiondiffraction angleangle lessless thanthan 22 degrees.degrees. FigureFigure 33 showsshows thisthis issueissue inin aa schematicschematic diagram. We need to enlarge virtual modulation device to bigger than three inches and also enlarge diagram.diagram. We We need need to to enlarge enlarge virtua virtual lmodulation modulation device device to to bigger bigger than than three three inches inches and and also also enlarge enlarge the ﬁeld of view to cover 360 degrees. thethe field field of of view view to to cover cover 360 360 degrees. degrees.

Figure 3. Basic challenge of the system design. FigureFigure 3. 3. Basic Basic challenge challenge of of the the system system design. design. Viewing angle of 360 degrees can only be achieved by either temporally scanning or spatially ViewingViewing angle angle ofof 360 360 degrees degrees cancan onlyonly be be achievachieveded by by eithereither temporallytemporally scanning scanning oror spatially spatially tilingtiling a a small small viewing viewing angle angle of of native native SLM SLM device. device. The The former former method method needs needs temporal temporal multiplexing multiplexing withtiling fast a small SLM viewing and the latterangle oneof native needs SLM spatial device. tiling The with former multiple method SLMs. needs The verytemporal ﬁrst thing multiplexing to do in withwith fast fast SLM SLM and and the the latter latter one one n needseeds spatial spatial tiling tiling with with multiple multiple SLMs SLMs. .The The very very first first thing thing to to do do in in thethe system system design design is is to to decide decide on on the the basic basic architec architectureture with with the the selection selection of of appropriate appropriate SLM SLM device. device. Wethe decidedsystem design to adopt is to the decide temporal on the multiplexing basic architec schemeture with and the fast selection SLM, based of appropriate on the comparison SLM device. of WeWe decided decided to to adopt adopt the the temporal temporal multiplexing multiplexing scheme scheme and and fast fast SLM, SLM, based based on on the the comparison comparison of of expectedexpected system system complexity complexity and and maximum maximum data data thro throughputughput of SLMs. SLMs. Usually Usually the the performance performance of of an an SLMexpected device system is denoted complexity by SBP, and which maximum is cited data most thro oftenughput to compare of SLMs. different Usually SLM the devices. performance SBP simply of an SLMSLM devicedevice isis denoteddenoted byby SBP,SBP, whichwhich isis citedcited momostst oftenoften toto comparecompare differentdifferent SLMSLM devices.devices. SBPSBP simplyrepresents represents the total the pixel total numbers pixel numbers of an SLM of an device, SLM device, while totalwhile data total throughput data throughput of a device of a device comes fromsimply the represents combination the oftotal pixel pixel count numbers and frame of an rate. SLM Total device, data while throughput total data actually throughput determines of a device whole comescomes from from the the combination combination of of pixel pixel count count and and frame frame rate. rate. Total Total data data throughput throughput actually actually determines determines system performance in terms of hologram image rendering, thus is more appropriate to judge the wholewhole system system performance performance in in terms terms of of hologram hologram imag imagee rendering, rendering, thus thus is is more more appropriate appropriate to to judge judge performance of an SLM. This quantity is named extended SBP(eSBP), which is deﬁned in Equation (1): thethe performanceperformance ofof anan SLM.SLM. ThisThis quantityquantity isis namednamed extendedextended SBP(eSBP),SBP(eSBP), whichwhich isis defineddefined inin Equation (1): Equation (1): eSBP = SBP ∗ frame rate (or refresh rate) = (pixel resolution) ∗ (frame rate) (1) eSBPeSBP = = SBP SBP * * frame frame rate(or rate(or refresh refresh rate) rate) = = (pixel (pixel resolution) resolution) * * (frame (frame rate) rate) (1)(1)

SomeSome of of typical typical SLM SLM devices devices are are compared compared in in terms terms of of SBP SBP and and eSBP eSBP in in Table Table 3. 3. From From the the table, table, wewe see see that that although although the the SBP SBP of of DMD DMD is is smaller smaller th thanan LCD LCD or or LCoS, LCoS, the the eSBP eSBP value value is is much much larger. larger. Appl. Sci. 2019, 9, 705 6 of 16

Some of typical SLM devices are compared in terms of SBP and eSBP in Table3. From the table, we see that although the SBP of DMD is smaller than LCD or LCoS, the eSBP value is much larger.

Table 3. Comparison of SBP and eSBP for different SLM devices.

Appl. Sci. 2019, 9, x FOR PEER REVIEW Refresh eSBP 6 of 16 Pixel Pixel SBP (Total eSBP SLM Type Rate/bit (Total System Pitch (µm) Resolution Pixels) Ratio Table 3. Comparison of SBP and eSBP for differentDepth SLMData devices. Rate) LCD 156 × 52 2560 × 2048 5,242,880 60/8 2,516,582,400 0.63 SeeReal Pixel Refresh eSBP SLM ×Pixel ×SBP (Total eSBP LCoSPitch 4.8 4.8 3840 2160 8,294,400Rate/bit 60/8 (Total3,981,312,000 Data 1System NICT Type 13.68Resolution× Pixels) Ratio DMD(μm) 1024 × 768 786,432depth 32,000/1 Rate)25,165,824,000 6.32 ETRI 13.68 LCD 156 × 52 2,560 × 2,048 5,242,880 60/8 2,516,582,400 0.63 SeeReal LCoS 4.8 × 4.8 3,840 × 2,160 8,294,400 60/8 3,981,312,000 1 NICT We13.68 utilize × the large eSBP value of a DMD device to spatially distribute fast refreshing hologram DMD 1,024 × 768 786,432 32,000/1 25,165,824,000 6.32 ETRI images by13.68 temporal multiplexing, thus achieving very wide viewing angle. The following details the basic methods that we applied to the display system design for the required specifications. We utilize the large eSBP value of a DMD device to spatially distribute fast refreshing hologram images(1) byImage temporal size multiplexing, is first enlarged thus by achieving simple optical very wide magnification viewing angle. using The4-f followingoptics, which details results the in basic methodsenlarged that virtual we applied SLM (fromto the 1.27display cm system to 7.62 cm).design The for price the required of this image specifications. enlargement is reduced (1) diffraction Image size angle is first of enlarged the virtual by enlargedsimple optical SLM (frommagnification 2◦ to 0.3◦ using). 4-f optics, which results in enlarged(2) We virtual make a SLM viewing (from window 1.27cm at to viewing 7.62cm). distance The price of 900 of mmthis byimage adding enlargement a field lens is inreduced the position diffractionof angle virtual of SLM the virtual plane, en forminglarged theSLM size (from of 4.7 2° mmto 0.3°).× 4.7 mm rectangular viewing window. (3)(2) Seamless We make horizontala viewing window viewing at zone viewing is made distan byce circularly of 900 mm tiling by adding the viewing a field windowslens in the of the position of virtual SLM plane, forming the size of 4.7mm × 4.7mm rectangular viewing window. appropriate number of viewpoint-dependent hologram images. In our system, the viewing zone (3) Seamless horizontal viewing zone is made by circularly tiling the viewing windows of the has a circumferential length of ~4000 mm, which can be covered by tiling more than 851 viewing appropriate number of viewpoint-dependent hologram images. In our system, the viewing zone has windows. In actual implementation, 1024 viewing windows are tiled with some overlap between a circumferential length of ~4000 mm, which can be covered by tiling more than 851 viewing adjacent ones. This is achieved by temporal multiplexing with mechanical scanning optics. windows. In actual implementation, 1024 viewing windows are tiled with some overlap between adjacentSteps ones. (1)–(3)This is areachieved shown by in temporal Figure4. multiplexing with mechanical scanning optics. Steps (1)–(3) are shown in Figure 4.

Figure 4. Schematic diagram of the design (1), (2), and (3) denotes the corresponding step in the text. Figure 4. Schematic diagram of the design (1), (2), and (3) denotes the corresponding step in the text. (4) The viewpoint-dependent hologram images are designed to be seen at the center of the tabletop (4) The viewpoint-dependent hologram images are designed to be seen at the center of the display, so that coalition of them makes a volumetric 3-dimentional image ﬂoating in the air. This tabletop display, so that coalition of them makes a volumetric 3-dimentional image floating in the air. is achieved by two stages of optics. In the ﬁrst stage, the light path of the hologram image is This is achieved by two stages of optics. In the first stage, the light path of the hologram image is guided by a combination of flat mirrors into 45-degree slanted direction and in a way to overlap at the axial area of the system. In the second stage, these virtual SLM planes are transported to the top center area of the tabletop display (final hologram image plane in the air above the tabletop of the display) by image transportation function of double parabolic mirrors from one focal area to another. This is shown in Figure 5. Appl. Sci. 2019, 9, 705 7 of 16

guided by a combination of ﬂat mirrors into 45-degree slanted direction and in a way to overlap at the axial area of the system. In the second stage, these virtual SLM planes are transported to the top center area of the tabletop display (ﬁnal hologram image plane in the air above the tabletop of the display) by image transportation function of double parabolic mirrors from one focal area to another. This is shown in Figure5. Appl. Sci. 2019, 9, x FOR PEER REVIEW 7 of 16

FigureFigure 5.5. 360-degree360-degree volumetricvolumetric imageimage ﬂoatingfloating onon tabletoptabletop display.display.

BasedBased onon thisthis principalprincipal designdesign method,method, wewe havehave successfullysuccessfully implementedimplemented thethe firstfirst monocolormonocolor versionversion of of tabletop tabletop holographic holographic display display systems, systems, which which has been has reported been reported in the previous in the publication previous publication[22]. In the [first22]. prototype In the first system, prototype four system, DMDs four are DMDsspatially are tiled spatially to make tiled a to2 × make 2 multivision a 2 × 2 multivision style SLM stylemodule SLM for module enhanced for enhancedresolution resolution (from 768 (from × 768 768to 1,536× 768 × to1,536) 1,536 and× 1536) size (1.27 and sizeto 2.54 (1.27 cm). to 2.54This cm).still Thissmall still size small of SLM size is of enlarged SLM is enlargedby a cascade by aof cascade two 4-f ofoptics, two 4-fwithoptics, the magnification with the magnification factor of 1.67 factor and of1.65 1.67 each, and overall 1.65 each, resulting overall in resulting 2.76 times in of 2.76 magnification times of magnification effect. For rejection effect. For of rejection DC and ofhigh DC order and highnoise, order a single-sideband noise, a single-sideband (SSB) filter (SSB) is inserted filter is insertedin the Fourier in the Fourierplane of plane the offirst the 4-f first optics.4-f optics. It is Itmonocolor is monocolor system system with with a green a green (wavelength (wavelength of of632 632 nm) nm) laser laser as as a a light light source. For thethe detaildetail specificationsspecifications ofof opticaloptical componentscomponents usedused inin thethe displaydisplay system,system, pleasepleaserefer referto to[ [1].1]. InIn thethe secondsecond andand laterlater implementationsimplementations ofof ourour system,system, thethe following following are are added added as as well well as as the the enhanced enhanced functions. functions. (1) The SLM module for color hologram generation is designed using three DMDs for each (1) The SLM module for color hologram generation is designed using three DMDs for each monocolor monocolor (R, G, and B) hologram, combining optics consisting of two prisms (one is trichroic and (R, G, and B) hologram, combining optics consisting of two prisms (one is trichroic and the other the other is TIR (total internal reflection) type), and three laser light sources with collimated beam is TIR (total internal reflection) type), and three laser light sources with collimated beam output. output. In this way we preserve the image resolution of the monocolor display in color holographic In this way we preserve the image resolution of the monocolor display in color holographic display. Refer to [22] for some implementation details. display. Refer to [22] for some implementation details. (2) We have thus far a vertically narrow (4.7 mm) viewing stripe around and over the display. (2) We have thus far a vertically narrow (4.7 mm) viewing stripe around and over the display. A vertical viewing angle of 20° is also required in order to permit the viewers’ free position or posture A vertical viewing angle of 20◦ is also required in order to permit the viewers’ free position or (sitting on a chair, standing, or somewhere in between, as shown in Figure 2b) to view the hologram posture (sitting on a chair, standing, or somewhere in between, as shown in Figure2b) to view the image, and this is achieved by adding vertical diffuser optics (a lenticular sheet, which is hologram image, and this is achieved by adding vertical diffuser optics (a lenticular sheet, which schematically shown in Figure 5) at the virtual SLM plane. Since diffusing optics degrades the spatial is schematically shown in Figure5) at the virtual SLM plane. Since diffusing optics degrades the focusing capability of hologram somewhat, there is a tradeoff in this regard. spatial focusing capability of hologram somewhat, there is a tradeoff in this regard. (3) Vertical parallax is added inside this 20° of vertical viewing range (between 35° and 55° (3) Vertical parallax is added inside this 20◦ of vertical viewing range (between 35◦ and 55◦ from from the tabletop). For this, precise and fast pupil tracking of viewers with a dynamic update of the the tabletop). For this, precise and fast pupil tracking of viewers with a dynamic update of hologram images is implemented for a limited number of viewers. Pupil tracking module consists of the hologram images is implemented for a limited number of viewers. Pupil tracking module multiple IR illuminator and cameras. Two cameras working as a stereo camera covers a viewer in 90° consists of multiple IR illuminator and cameras. Two cameras working as a stereo camera covers range, and four camera sets are used for 360° coverage, while supporting four viewers a viewer in 90◦ range, and four camera sets are used for 360◦ coverage, while supporting four simultaneously. Figure 6 shows the working concept of the module and experimental setup for 180° viewers simultaneously. Figure6 shows the working concept of the module and experimental coverage with reconstructed image at four different vertical positions. Currently, update of pupil position is limited to 10 times per second, which is not sufficient for smooth update of hologram images following viewer’s motion. (4) Capture of 3D information from moving real objects and the 360-degree perspective hologram CGH is implemented, and thus the signal chain from capture-CGH to display is demonstrated.

Appl. Sci. 2019, 9, 705 8 of 16

setup for 180◦ coverage with reconstructed image at four different vertical positions. Currently, update of pupil position is limited to 10 times per second, which is not sufﬁcient for smooth update of hologram images following viewer’s motion.

Appl.(4) Sci. 2019Capture, 9, x FOR of PEER 3D information REVIEW from moving real objects and the 360-degree perspective8 of hologram 16 CGH is implemented, and thus the signal chain from capture-CGH to display is demonstrated. Appl. Sci. 2019, 9, x FOR PEER REVIEW 8 of 16

Figure 6. Vertical parallax support by using pupil tracking and dynamic hologram update. (a) ConceptualFigure 6.modelVertical and (b parallax) 180° pupil support tracking by module using pupiland holo trackinggram images and dynamic seen at four hologram different update. Figurevertical(a) Conceptual6. viewing Vertical position. modelparallax and support (b) 180 ◦bypupil using tracking pupil moduletracking andand hologram dynamic imageshologram seen update. at four different(a) Conceptualvertical viewingmodel and position. (b) 180° pupil tracking module and hologram images seen at four different 4. Implementationvertical viewing andposition. Experimental Result 4. Implementation and Experimental Result The basic system we implemented is a vertically diffused horizontal parallax only (VD-HPO) 4. Implementation and Experimental Result hologramThe system. basic systemThis system we implemented is the one withou is a verticallyt pupil tracking diffused module, horizontal having parallax a 20° only vertical (VD-HPO) viewinghologramThe anglebasic system. systemand a Thisnonsupported we systemimplemented is thevertical oneis a without parallax,vertically pupil which diffused tracking is similar horizontal module, to rainbow parallax having hologram. aonly 20◦ vertical(VD-HPO) With viewing hologrampupilangle tracking and system. a module nonsupported This added, system verticalthe is system the parallax, one be comeswithou which at full pupil is similarparallax tracking to hologram rainbow module, system. hologram. having Witha 20° pupil vertical tracking viewingmoduleSchematic angle added, anddiagram thea nonsupported system of the becomesbasic displayvertical a full systemparallax, parallax and hologramwhich photo is of similar system.the final to systemrainbow shape hologram. are shown With pupilin Figure trackingSchematic 7. The module system diagram added, consists of the the of systeman basic SLM display be modulecomes system aconsisting full parallax and photoof laser hologram of light the source final system. system and three shape DMDs, are shown 4-f inrelaySchematic Figure optics7. Thefordiagram image system ofmagnification consiststhe basic of display an and SLM noise system module filtering, and consisting photo temporal of the of multiplexing laser final lightsystem sourcemodule shape and consisting are three shown DMDs, inof Figurea4-f scanningrelay 7. The optics motor system for and image consistslight magnificationpath of redirecting an SLM module and optics, noise consisting and filtering, volumetric of temporal laser image light multiplexing floating source opticsand modulethree consisting DMDs, consisting 4-fof doublerelayof a scanning optics parabolic for motor image mirror and magnifications or light equivalent path redirecting and optics. noise filt optics,ering, and temporal volumetric multiplexing image floating module optics consisting consisting of aof scanning double parabolicmotor and mirrors light path or equivalentredirecting optics.optics, and volumetric image floating optics consisting of double parabolic mirrors or equivalent optics.

Figure 7. (a) Schematic configuration. (b) Photo of the implemented system. (c) Demonstration. Figure 7. (a) Schematic configuration. (b) Photo of the implemented system. (c) Demonstration. Three-hundred-and-sixty-degree viewable volumetric color hologram image is rendered in the Three-hundred-and-sixty-degree viewable volumetric color hologram image is rendered in the displayFigure by 7. coalition (a) Schematic of 1024 configuration. view-dependent (b) Photo holograms, of the implemented which are system. generated (c) Demonstration. from 3D data consisting display by coalition of 1024 view-dependent holograms, which are generated from 3D data consisting of color and depth image for each viewpoint from a CG model or scanning real objects. Figure 8 Three-hundred-and-sixty-degree viewable volumetric color hologram image is rendered in the shows the whole signal chain from 360° 3D data capture from real moving or static objects, possibly display by coalition of 1024 view-dependent holograms, which are generated from 3D data consisting including humans, CGH computation, and real-time hologram data transmission via data server with of color and depth image for each viewpoint from a CG model or scanning real objects. Figure 8 high speed interface up to display system. There are several highly complicated signal processing shows the whole signal chain from 360° 3D data capture from real moving or static objects, possibly including humans, CGH computation, and real-time hologram data transmission via data server with high speed interface up to display system. There are several highly complicated signal processing Appl. Sci. 2019, 9, 705 9 of 16 of color and depth image for each viewpoint from a CG model or scanning real objects. Figure8 shows the whole signal chain from 360◦ 3D data capture from real moving or static objects, possibly including humans, CGH computation, and real-time hologram data transmission via data server with high Appl. Sci. 2019, 9, x FOR PEER REVIEW 9 of 16 speed interface up to display system. There are several highly complicated signal processing tasks ◦ involvedtasks involved in the in chain. the chain. The 360 Thedata 360° capture data capture part has part two has alternative two alternative setups: setups: one consists one consists of a single of a depthsingleAppl. sensingdepth Sci. 2019 sensing camera, 9, x FOR camera PEER with REVIEW a turntablewith a turntable for a static for a object static and object the and other, the for other, moving for objects,moving9 of consistsobjects, 16 ofconsists multiple of synchronizedmultiple synchronized depth sensing depth cameras sensing arranged cameras 120 degreesarranged apart. 120 Imagedegrees registration apart. Image and tasks involved in the chain. The 360° data capture part has two alternative setups: one consists of a stitching,registration smoothing, and stitching, and hole-filling smoothing, algorithms and hole-filling are applied algorithms to get pointare applied cloud data to get which point is cloud optionally data convertedsingle todepth 3D sensing mesh model. camera Finallywith a turntable RGB and for depth a static image object data and arethe extractedother, for moving via virtual objects, camera whichconsists is optionally of multiple converted synchronized to 3D meshdepth model sensing. Finally cameras RGB arranged and depth 120 degreesimage dataapart. are Image extracted setting for required number of viewpoints of the tabletop display. This data is input to the CGH via virtualregistration camera and settingstitching, for smoothing, required andnumber hole-filling of vi ewpointsalgorithms ofare the applied tabletop to get display. point cloud This data data is computation module, algorithm of which is calculating wave propagation model of the display optics, inputwhich to the is CGH optionally computation converted module, to 3D mesh algorithm model. Finallyof which RGB is andcalculating depth image wave data propagation are extracted model outputtingof thevia display virtual hologram cameraoptics, fringe settingoutputting patterns. for required hologram Details number of fringe this of CGH vipatterns.ewpoints are presented Detailsof the tabletop inof referencethis display.CGH [23 are This]. Thepresented data hologram is in datareference isinput uploaded [23].to the The CGH to datahologram computation server data and module, finallyis uploaded algorithm transmitted to ofdata which to server the isSLM calculating and module finally wave of transmitted thepropagation display to inmodel realthe SLM time viamodule highof the of speed thedisplay display interface optics, in from realoutputting time data via server hologram high to speed the fringe display interface patterns. system. from Details data of server this CGH to the are display presented system. in reference [23]. The hologram data is uploaded to data server and finally transmitted to the SLM module of the display in real time via high speed interface from data server to the display system.

Figure 8. Signal processing chain of the tabletop holographic display system. Figure 8. Signal processing chain of the tabletop holographic display system. Images of optically reconstructed hologram images from the implemented display system are ImagesImages of optically of optically reconstructed reconstructed hologram hologram imagesages from from the the implemen implementedted display display system system are are shown in Figure9. shownshown in Figure in Figure 9. 9.

FigureFigure 9. Optically-reconstructed 9. Optically-reconstructed hologramhologram image image captur captureded from from different different horizontal horizontal viewpoints: viewpoints: (a) (a) CGCG model model and and ((bb)) realreal human human object. object. Figure 9. Optically-reconstructed hologram image captured from different horizontal viewpoints: (a) 5. Measurements and Evaluation CG model and (b) real human object.

5. Measurements and Evaluation Appl. Sci. 2019, 9, x FOR PEER REVIEW 10 of 16

It is not less important to measure and evaluate the performance and image qualities of the final holographic displays than system implementation itself. A study of how to define display Appl. Sci. 2019, 9, 705 10 of 16 performance measure or image quality has been carried out by only a few researchers so far [24,25]. In order to evaluate performance of a holographic display and compare with different systems, 5.we Measurements need to define and appropriate Evaluation performance metrics. Since holographic displays have different characteristics and behavior, existing performance metrics for conventional 2D displays are not It is not less important to measure and evaluate the performance and image qualities of the final sufficient. In Table 4, some essential performance metrics we defined for holographic 3D displays are holographic displays than system implementation itself. A study of how to define display performance listed. For the implemented display system, we measured and evaluated several important properties measure or image quality has been carried out by only a few researchers so far [24,25]. based on this performance metric. In order to evaluate performance of a holographic display and compare with different systems, we need to define appropriate performance metrics. Since holographic displays have different Table 4. Metrics and their definitions for holographic performance measure. characteristics and behavior, existing performance metrics for conventional 2D displays are not sufficient.Category In Table 4, some essentialMetrics performance metrics we defined Definition/Description for holographic 3D displays are listed. For the implemented display system, we measured andMaximally evaluated representable several important image properties size 3-dimensional volume basedImage on thisvolume performance size metric. measured in lateral and depth direction, or (width × height × depth) equivalent Viewing angleTable 4. Metrics and their definitions for holographicLateral or performance circular viewing measure. range measured Horizontal/vertical VA Category(VA) Metrics Definition/Descriptionin angle 3-dimensional volume MTFMaximally measured representable through image depth size measured range of in Image volume size 3D-MTF(image(width × height resolution)× depth) imagelateral volume, and depth which direction, represents or equivalent image Viewing angle (VA) Horizontal/vertical VA Lateral or circularresolution viewing range measured in angle Image quality ColorMTF distortion measured measured through depth for range standard of image color 3D-MTFColor (imagefidelity resolution) volume, which representschart image resolution Image quality Color fidelity Color distortion measured for standard color chart Speckle noise Speckle contrast measured Speckle noise Speckle contrast measured Brightness Perceptive brightness level of the image Brightness Perceptive brightness level of the image Support of Degree of Depth of The smaller the DOF, the better the The smaller the DOF, the better the capability of Support of accommodation Degree of Depth of Focus (DOF) accommodation Focus(DOF) capability supportingof supporting accommodation accommodation

5.1.5.1. Size Size of of the the Image Image Volume Volume Space Space InIn our our tabletop tabletop holographic holographic display, display, a 3-dimensional a 3-dimensional image image is rendered is rendered in free space in free and space virtually and occupiesvirtually a occupies volume space,a volu asme represented space, as represented in Figure 10 .in This Figure image 10. space This isimage made space from is intersections made from ofintersections adjacent viewing of adjacent cones, viewing each of cones, which each corresponds of which corresponds to a viewpoint to a andviewpoint hologram and pair.hologram It can pair. be dividedIt can be into divided real and into virtual real imageand virtual areas, image which areareas, seamlessly which are composing seamlessly the composing whole volume the spacewhole forvolume the hologram space for image. the hologram If we render image. hologram If we ren imageder hologram inside this image image inside volume, this thenimage it isvolume, seen much then likeit is a seen real much object like supporting a real object binocular supporting as well binocular as motion as parallax. well as motion parallax.

FigureFigure 10. 10.Image Image volume volume space. space.

5.2.5.2. Hologram Hologram Image Image Resolution Resolution Since the hologram image is rendered in free space as a volumetric image, simple pixel resolution does not properly represent the resolution or quality of optically reconstructed hologram image. Appl. Sci. 2019, 9, x FOR PEER REVIEW 11 of 16

Appl. Sci.Since2019, 9,the 705 hologram image is rendered in free space as a volumetric image, simple11 ofpixel 16 resolution does not properly represent the resolution or quality of optically reconstructed hologram image. We defined a new metric for representing the image resolution, which we named 3D-MTF. WeThe defined modulation a new transfer metric forfunction representing (MTF) is the usually image used resolution, to measure which the we performance named 3D-MTF. of an optical The modulationdevice like transfer lens in function terms of (MTF) how iswell usually it conveys used to th measuree contrast the of performance an object to of the an opticalimage. device MTF is likemodulation lens in terms depth of howdenoted well as it a conveys function the of contrastfrequency of of an the object test topattern the image. as shown MTF in is Equation modulation (2). A depth3D-MTF denoted is defined as a function as an extension of frequency of conventional of the test pattern MTF asby shown adding in depth Equation as another (2). A 3D-MTF parameter, is definedwhich as is andefined extension as in of Equation conventional (3): MTF by adding depth as another parameter, which is defined as in Equation (3): 𝐼 𝐼 " 𝑀𝜉# 𝑀Imax − Imin , 𝑀𝑇𝐹M image(ξ) (2) M = 𝐼 𝐼, MTF = 𝑀𝜉 (2) Imax + Imin Mobject(ξ)

k=N∆D 3𝐷 𝑀𝑇𝐹 𝑀𝑇𝐹 (3) 3DMTF = {MTFk}k=1 (3) where, M is modulation depth, Imax and Imin are the maximum and minimum signal intensity of signal, where, M is modulation depth, Imax and Imin are the maximum and minimum signal intensity of respectively, Ma(b) is the modulation depth of a as a function of b, ΔD is depth resolution, and N is signal, respectively, Ma(b) is the modulation depth of a as a function of b, ∆D is depth resolution, the number of depth layers used in the measurement. MTFk is the MTF value measured at the k-th and N is the number of depth layers used in the measurement. MTFk is the MTF value measured atdepth the k-th plane. depth Defined plane. as Deﬁned such, 3D-MTF as such, is 3D-MTFa collection is a of collection conventional of conventional MTF measured MTF for measured images at fordifferent images depth. at different We measured depth. We 3D-MTF measured value 3D-MTF of the implemented value of the implementeddisplay system display as shown system in Figure as shown11. For in the Figure purpose 11. For of the accurate purpose measurement, of accurate measurement,hologram image hologram is captured image directly is captured by CCD directly (Sony byICX834) CCD (Sony placed ICX834) at different placed depth at different plane for depth a fixed plane viewpoint for a ﬁxed (Figure viewpoint 11a). A (Figurelinear stage 11a). and A lineara green stagelaser and with a green a 532 laser nm withwavelength a 532 nm are wavelength used in the are experiment. used in the experiment.An example An black example and white black andstripe whitepattern stripe used pattern in the usedmeasurement in the measurement is shown in isFigure shown 11c, in in Figure the order 11c, of in original the order shape, of original its CGH shape, fringe itspattern, CGH fringe and reconstructed pattern, and reconstructedhologram image. hologram For each image. stripe pair For eachof the stripe test pattern, pair of modulation the test pattern, depth modulationvalue is measured depth value and is average measured value and of average whole valuestripe of pairs whole is stripeplotted pairs for iseach plotted depth for in each Figure depth 11d. inThe Figure ranges 11d. of The measured ranges values of measured for whole values stripe for pairs whole at each stripe depth pairs are at denoted each depth by vertical are denoted bars. MTF by verticalchanges bars. with MTF image changes depth, with and image the average depth, andvalue the of average MTF is value35.9%@1.2cycles/mm of MTF is 35.9%@1.2cycles/mm test pattern. More testdetails pattern. of measurement More details of and measurement evaluation andare evaluationgiven in Appendix are given A. in In Appendix referenceA. [26], In reference a holographic [ 26], a holographicstereogram stereogramis analyzed is analyzedfor MTF for modeling, MTF modeling, showing showing the theeffect effect of ofphase phase error arisingarising from from approximatingapproximating the the wavefront wavefront of a of 3D a image3D image with with a wavefront a wavefront of a projected of a projected image atimage stereogram at stereogram plane. Whileplane. a holographicWhile a holographic stereogram stereogram can be analyzed can be analyzed as a diffraction-limited as a diffraction-limited imaging imaging system withsystem some with aberrations,some aberrations, our display our systemdisplay relies system on therelies diffraction on the di toffraction make a to real make or virtual a real image,or virtual which image, requires which differentrequires analysis. different Furthermore, analysis. Furthermore, binary encoding binary of computedencoding fringeof computed pattern resultsfringe inpattern distortions results of in frequencydistortions spectrum, of frequency which spectrum, should be which taken should care of. be This taken is remained care of. This for furtheris remained study. for further study.

FigureFigure 11. 11.3D-MTF 3D-MTF measurement: measurement: (a )(a design) design of of test test method, method, (b ()b environment) environment setup setup for for measurement, measurement, (c)( testc) test pattern, pattern, and and (d )(d result) result of theof the measurement. measurement. Appl. Sci. 2019, 9, 705 12 of 16 Appl. Sci. 2019, 9, x FOR PEER REVIEW 12 of 16

5.3. Color Gamut and Color Reproduction Fidelity InIn aa fullfull colorcolor holographicholographic display,display, colorcolor reproductionreproduction fidelityfidelity isis notnot actuallyactually guaranteedguaranteed eveneven thoughthough colorcolor gamutgamut ofof thethe displaydisplay systemsystem isis widerwider thanthan conventionalconventional ones.ones. Figure 1111aa shows the measured color gamut of our display, whichwhich is ~160%~160% wider than CIE 1976 NTSC standard (1953). On the contrary, FigureFigure 1212dd showsshows thethe measuredmeasured colorcolor fidelityfidelity forfor thethe 2424 colors in a standard Macbeth color chart. Blue Blue dots dots represent represent desired desired values values and and red reddots dots are measured are measured ones. ones.As shown As shown in Figure in Figure12c, optically 12c, optically reconstructed reconstructed hologram hologram image imageis corrupt is corrupteded with speckle with speckle and other and noise, other so noise, that so colors that colorscorresponding corresponding to two to modulation two modulation levels levels(125 and (125 255) and are 255) not are clearly not clearly distinguished. distinguished. As a result, As a result, color colorseparation separation is very is bad. very Quantificati bad. Quantificationon of this of result this result is under is under study. study.

Figure 12. ColorColor gamut gamut and and color color fidelity fidelity of implemented of implemented system, system, (a) color (a) gamut, color gamut, (b) 24 color (b) 24 Gretag- color Gretag-MacbethMacbeth ColorChecker, ColorChecker, (c) hologram (c) hologram image, image,and (d) and measured (d) measured color fidelity. color fidelity. 5.4. Speckle Noise 5.4. Speckle Noise Speckle noise in holographic displays mainly comes from the use of coherent light sources like Speckle noise in holographic displays mainly comes from the use of coherent light sources like laser, which is one of major sources of quality degradation of hologram image. Measurement and laser, which is one of major sources of quality degradation of hologram image. Measurement and analysis of speckle noise is well-studied, and speckle contrast is the most used metric to judge the analysis of speckle noise is well-studied, and speckle contrast is the most used metric to judge the severity of the speckle [27]. There has been quite a lot of researches for speckle noise reduction severity of the speckle [27]. There has been quite a lot of researches for speckle noise reduction methods [28,29], and most of them is based on the concept of temporal multiplexing and averaging methods [28,29], and most of them is based on the concept of temporal multiplexing and averaging down overall speckle contrast. We developed an angular spectrum interleaving method, which is down overall speckle contrast. We developed an angular spectrum interleaving method, which is verified being able to reduce—by 60%—the level of speckle noise of the system down to 10% [30]. verified being able to reduce—by 60%—the level of speckle noise of the system down to 10% [30]. We We are currently working on the physical realization of this method for our display system. are currently working on the physical realization of this method for our display system. 5.5. Degree of Depth of Focus (DoF) 5.5. Degree of Depth of Focus (DoF) As is always mentioned, the holographic display is most distinguished from other conventional 3D displaysAs is always in the mentioned, sense that the it has holographic the capability display of is supporting most distinguished accommodation from other of human conventional visual system.3D displays This in means the sense that holographic that it has the display capabi canlity render of supporting an image accommo at a specifieddation depth of human plane, whichvisual cansystem. be focused This means as real that objects. holographic In previous display studies, can thisrender has an been image simply at a demonstrated specified depth by plane, showing which two orcan more be focused objects as rendered real objects. at different In previous distance, studies, one of this which has isbeen focused simply the demonstrated others become by defocused showing andtwo blurredor more when objects captured rendered by a at shallow different DoF distan camera.ce, Weone can of evaluatewhich is this focused capability the others of a display become by usingdefocused a very and similar blurred method when to captured measuring by 3D-MTF.a shallow For DoF an camera. MTF test We pattern can evaluate imaged atthis a certaincapability depth of plane,a display measuring by using MTF a very around similar the method plane in to viewing measuring direction 3D-MTF. can tell For how an MTF well thetest image pattern is focusedimaged toat thea certain plane depth and defocused plane, measuring elsewhere. MTF Figure around 13a–c the shows plane modulation in viewing depth direction values can measured tell how forwell a testthe patternimage is imaged focused at to distances the plane of and 57 mm,defocused 158 mm, elsewh andere. 213 Figure mm from 13a–c the shows virtual modulation SLM plane, depth respectively. values Theremeasured are discrepanciesfor a test pattern between imaged the at measured distances imageof 57 mm, depth 158 and mm, the and intended 213 mm depth from renderedthe virtual in SLM the CGHplane, algorithm, respectively. which There are are 40 discrepancies mm, 120 mm, betw andeen 160 the mm. measured This is believedimage depth to be and caused the fromintended the depth rendered in the CGH algorithm, which are 40 mm, 120 mm, and 160 mm. This is believed to be caused from the slight difference between the dimensions of designed and actual optical signal Appl. Sci. 2019, 9, 705 13 of 16

Appl. Sci. 2019, 9, x FOR PEER REVIEW 13 of 16 slight difference between the dimensions of designed and actual optical signal path of the system. Itpath represents of the system. varying It depthrepresents representation varying depth capability representation of the display capability through of the the display image through rendering the volume.image rendering These modulation volume. depth These curves modulation are obtained depth using curves the exactare sameobtained measurement using the method exact same as is usedmeasurement for obtaining method MTF curves.as is used Since for for obtaining a real hologram MTF curves. image theSince spherically for a real converging hologram wavefrontimage the isspherically supposed converging to focus at the wavefront image plane, is supposed defocusing to focu ors blur at the in image front or plane, behind defocusing this focal or plane blur shouldin front beor symmetric.behind this So, focal deviations plane should more thanbe symmetric. measurement So, deviations error from more this expected than measurement symmetry mayerror come from fromthis expected aberrations symmetry of the optics may ofcome the system.from aberrations of the optics of the system.

FigureFigure 13. 13.Degree Degree of of DoF DoF at at different different depths, depths, measured measured with with MTF MTF test test patterns. patterns.

6.6. Conclusions Conclusions AsAs visualvisual mediamedia becomebecome pervasivepervasive inin ourour dailydaily lifelife andand moremore visualvisual informationinformation isis usedused forfor effectiveeffective communicationcommunication amongamong people,people, newnew toolstools forfor moremore realistic realistic visual visual content content representation representation andand exchangeexchange areare required. required. DigitalDigital holographicholographic technologytechnology isis thethe mostmost wantedwanted oneone inin thisthis regard,regard, sincesince it it can can perfectly perfectly reproduce reproduce the the visual visual experience experience we we do do meet meet every every day. day. Due Due to to the the rapid rapid progress progress inin every every field field of of technology, technology, commercialization commercialization of of digital digital holographic holographic displays displays and and imaging imaging systems systems becomesbecomes visible. visible. Since Since the the working working principle principle and and requirements requirements of holographic of holographic display display systems systems are quite are differentquite different from those from of conventionalthose of conventional displays, wedisplays, need to we explore need diverse to explore ways diverse to realize ways commercially to realize viablecommercially systems. viable In this systems. paper, we In presentedthis paper, design we presented concepts, design implementation concepts, implementation methods of a new methods type ofof tabletopa new type holographic of tabletop display holographic system, and displa addressedy system, the and important addressed issue the of important measurement issue and of quantitativemeasurement evaluation and quantitative for objective evaluation comparison for object of differentive comparison systems. Especiallyof different important systems. isEspecially that the qualityimportant and is resolution that the quality of the reconstructedand resolution hologram of the reconstructed image is measured hologram and image analyzed is measured in terms and of 3D-MTF,analyzed color in terms reproduction of 3D-MTF, fidelity, color and reproduction degree of DoF. fidelity, Though and wedegree designed of DoF. each Though optical we component designed foreach minimum optical component aberration [ 20for], minimu the systemm aberration we developed [20], hasthe severalsystem elementswe developed which has degrade several the elements quality ofwhich final image,degrade including the quality binary of final amplitude image, encoding including of binary the fringe amplitude and mechanical encoding movement of the fringe during and imagemechanical rendering movement time. Rigorous during image analysis rendering of the system time. Rigorous regarding analysis this issue of remainsthe system as aregarding future work. this Weissue hope remains this work as a gives future some work. hint We for futurehope this development work gives of commerciallysome hint for acceptablefuture development holographic of displaycommercially systems. acceptable holographic display systems.

AuthorAuthor Contributions: Contributions:Writing—Original Writing—Original Draft Draft Preparation, Preparation, J.K.; J.K.; Writing—Review Writing—Review and Editing,and Editing, Y.L., K.H.,Y.L., H.K.,K.H., H.-E.K.,H.K., H.-E.K., J.N., J.P., J.N., J.H., J.P., and J.H., Y.K. and Y.K.

Funding:Funding: ThisThis workwork waswas supportedsupported byby thethe crosscross ministry ministry GigaKorea GigaKorea project project (GK18D0100, (GK18D0100,Development Development ofof Telecommunications Terminal with Digital Holographic Table-top Display) grant funded by the Korea government (MSIT).Telecommunications Terminal with Digital Holographic Table-top Display) grant funded by the Korea government (MSIT). Acknowledgments: This work is a part of the result of collaboration of ETRI and participating universities, companiesAcknowledgments: and research This institutes. work is a Participatedpart of the result organizations: of collaboration LG display, of ETRI ASTEL, and participating Aoptics, Yunam universities, Optics, Funzin,companies SiliconWorks, and research MVTech, institutes. KETI, Participated KIST, Konyang organi University,zations: KoreaLG display, University, ASTEL, Inha Aoptics, University, Yunam Kyungpook Optics, University,Funzin, SiliconWorks, Sejong University, MVTech, Kwangwoon KETI, University,KIST, Konyang and Warsaw University, University Korea of Technology.University, Inha University, ConﬂictsKyungpook of Interest: University,The Sejong authors Univer declaresity, no Kwangwoon conﬂict of interest. University, and Warsaw University of Technology.

Conflicts of Interest: The authors declare no conflict of interest.

Appendix A Appl. Sci. 2019, 9, 705 14 of 16

Appendix A In this appendix, more details of 3D-MTF measurements and result of evaluation are presented. Figure A1 shows an optically reconstructed hologram image for a test pattern. Each stripe pair consists of a black and a white vertical bar of width 416.67 µm (1.2 cycles/mm). The CCD sensor (Sony ICX834) we used in the experiment has pixel size of 3.1 µm, so a black (or white) stripe is captured by about 134 pixels in horizontal direction. For each of horizontal pixel positions of the CCD sensor, pixel values are accumulated in vertical direction, which are the raw data we used for MTF calculation. Minmax-based MTF is calculated using maximum value (among 134 values) of white stripe and minimum value of black stripe. Average-based MTF is calculated using the average value of each black and white stripe. Depending on the depth we measure, twelve to fourteen stripe pairs were used, and the average value (yellow box in Table A1) is represented as the MTF value of corresponding depth in Figure 11d, with the range of values denoted by vertical bars in the graph. Standard deviation is also calculated and shown in the last column with heading of ‘stdev’ in Table A1. The measurement is carried out for a system without the lenticular sheet, which is used for extending vertical viewing angle. Measurements for Figure 13 are done with the same method.

Table A1. Measured values for MTF calculations. a. Minmax_based MTF. depth 20 40 60 80 100 120 max 0.519 0.576 0.572 0.551 0.551 0.581 mean 0.467 0.533 0.525 0.521 0.527 0.529 min 0.433 0.482 0.494 0.468 0.490 0.511

b. SP# 1 2 3 4 5 6 7 8 9 10 11 12 13 14 average Stdev 20 0.459 0.458 0.447 0.461 0.454 0.469 0.475 0.498 0.484 0.519 0.433 0.453 0.449 0.474 0.467 0.022 40 0.527 0.545 0.550 0.543 0.576 0.523 0.542 0.521 0.482 0.519 0.529 0.537 0.530 0.533 0.022 60 0.518 0.507 0.532 0.526 0.529 0.504 0.572 0.528 0.494 0.528 0.531 0.505 0.561 0.509 0.525 0.022 80 0.497 0.551 0.523 0.533 0.516 0.536 0.541 0.546 0.484 0.518 0.468 0.542 0.521 0.026 100 0.545 0.536 0.517 0.547 0.526 0.551 0.515 0.499 0.519 0.529 0.536 0.544 0.490 0.527 0.019 120 0.520 0.521 0.524 0.541 0.517 0.524 0.527 0.523 0.581 0.511 0.532 0.526 0.529 0.018

c. Average_based MTF.

depth 20 40 60 80 100 120 max 0.286 0.418 0.400 0.396 0.402 0.393 mean 0.259 0.393 0.370 0.377 0.374 0.381 min 0.217 0.372 0.336 0.357 0.336 0.369

d. SP# 1 2 3 4 5 6 7 8 9 10 11 12 13 14 average Stdev 20 0.286 0.260 0.276 0.247 0.284 0.271 0.243 0.262 0.279 0.252 0.251 0.260 0.217 0.231 0.259 0.020 40 0.405 0.418 0.400 0.413 0.400 0.393 0.394 0.380 0.378 0.382 0.372 0.383 0.391 0.393 0.014 60 0.375 0.369 0.400 0.363 0.383 0.371 0.388 0.370 0.359 0.373 0.366 0.336 0.365 0.360 0.370 0.015 80 0.359 0.396 0.376 0.387 0.381 0.382 0.386 0.382 0.365 0.363 0.357 0.385 0.377 0.013 100 0.388 0.370 0.378 0.402 0.380 0.368 0.392 0.361 0.383 0.368 0.363 0.368 0.336 0.374 0.017 120 0.369 0.370 0.382 0.384 0.387 0.393 0.383 0.382 0.387 0.370 0.383 0.382 0.381 0.008 SP#: stripe pair number. Appl. Sci. 2019, 9, x FOR PEER REVIEW 15 of 16

Appl. Sci. 2019, 9, 705 15 of 16 SP#: stripe pair number

FigureFigure 1. A1. ReconstructedReconstructed hologram hologram image image at 80 atmm 80 depth mm depth (brightness (brightness level is level enhanced is enhanced for clear for presentation).clear presentation).

ReferencesReferences 1. Lim, Y.; Hong, K.; Kim, H.; Kim, H.-E.; Chang, E.-Y.; Lee, S.; Kim, T.; Nam, J.; Choo, H.-G.; Kim, J.; et al. 1. Lim, Y.; Hong, K.; Kim, H.; Kim, H.-E.; Chang, E.-Y.; Lee, S.; Kim, T.; Nam, J.; Choo, H.-G.; Kim, J.; et al. 360-degree tabletop electronic holographic display. Opt. Express 2016, 24, 24999–25009. [CrossRef][PubMed] 360-degree tabletop electronic holographic display. Opt. Express 2016, 24, 24999–25009. 2. St-Hilaire, P.; Benton, S.A.; Lucente, M.E.; Jepsen, M.L.; Kollin, J.; Yoshikawa, H.; Underkofﬂer, J.S. 2. St-Hilaire, P.; Benton, S.A.; Lucente, M.E.; Jepsen, M.L.; Kollin, J.; Yoshikawa, H.; Underkoffler, J.S. Electronic display system for computational holography. In Proceedings of the SPIE, Los Angeles, CA, USA, Electronic display system for computational holography. In Proceedings of SPIE, Los Angeles, CA, USA, 18–19 January 1990; pp. 174–182. 18–19 January 1990; pp. 174–182. 3. Smalley, D.E.; Smithwick, Q.Y.J.; Bove, V.M.; Barabas, J.; Jolly, S. Anisotropic leaky-mode modulator for 3. Smalley, D.E.; Smithwick, Q.Y.J.; Bove, V.M.; Barabas, J.; Jolly, S. Anisotropic leaky-mode modulator for holographic video displays. Nature 2013, 498, 313–317. [CrossRef][PubMed] holographic video displays. Nature 2013, 498, 313–317. 4. Häussler, R.; Reichelt, S.; Leister, N.; Zschau, E.; Missbach, R.; Schwerdtner, A. Large real-time holographic 4. Häussler, R.; Reichelt, S.; Leister, N.; Zschau, E.; Missbach, R.; Schwerdtner, A. Large real-time holographic displays: From prototypes to a consumer product. IS&T/SPIE Electron. Imaging 2009, 7237, 72370. displays: From prototypes to a consumer product. IS&T/SPIE Electron. Imaging 2009, 7237, 72370. 5. Hahn, J.; Kim, H.; Lim, Y.; Park, G.; Lee, B. Wide viewing angle dynamic holographic stereogram with a 5. Hahn, J.; Kim, H.; Lim, Y.; Park, G.; Lee, B. Wide viewing angle dynamic holographic stereogram with a curved array of spatial light modulators. Opt. Express 2008, 16, 12372–12386. [CrossRef][PubMed] curved array of spatial light modulators. Opt. Express 2008, 16, 12372–12386. 6. Senoh, T.; Mishina, T.; Yamamoto, K.; Oi, R.; Kurita, T. Viewing-Zone-Angle-Expanded Color Electronic 6. Senoh, T.; Mishina, T.; Yamamoto, K.; Oi, R.; Kurita, T. Viewing-Zone-Angle-Expanded Color Electronic Holography System Using Ultra-High-Deﬁnition Liquid Crystal Displays with Undesirable Light Holography System Using Ultra-High-Definition Liquid Crystal Displays with Undesirable Light Elimination. J. Display Technol. 2011, 7, 382–390. [CrossRef] Elimination. J. Display Technol. 2011, 7, 382–390. 7. Oi, R.; Sasaki, H.; Wakunami, K.; Ichihashi, Y.; Senoh, T.; Yamamoto, K. Large size three-dimensional video 7. Oi, R.; Sasaki, H.; Wakunami, K.; Ichihashi, Y.; Senoh, T.; Yamamoto, K. Large size three-dimensional video by electronic holography using multiple spatial light modulators. Sci. Rep. 2014, 4, 6177. by electronic holography using multiple spatial light modulators. Sci Rep 2014, 4, 6177. 8. Takaki, Y.; Okada, N. Hologram generation by horizontal scanning of a high-speed spatial light modulator. 8. Takaki, Y.; Okada, N. Hologram generation by horizontal scanning of a high-speed spatial light modulator. Appl. Opt. 2009, 48, 3255–3260. [CrossRef][PubMed] Appl. Opt. 2009, 48, 3255–3260. 9. Takaki, Y.; Matsumoto, Y.; Nakajima, T. Color image generation for screen-scanning holographic display. 9. Takaki, Y.; Matsumoto, Y.; Nakajima, T. Color image generation for screen-scanning holographic display. Opt. Express 2015, 23, 26986–26998. [CrossRef][PubMed] Opt. Express 2015, 23, 26986–26998. 10. Blanche, P.-A.; Bablumian, A.; Voorakaranam, R.; Christenson, C.; Lin, W.; Gu, T.; Flores, D.; Wang, P.; 10. Blanche, P.-A.; Bablumian, A.; Voorakaranam, R.; Christenson, C.; Lin, W.; Gu, T.; Flores, D.; Wang, P.; Hsieh, W.-Y.; Kathaperumal, M.; et al. Holographic three-dimensional telepresence using large-area Hsieh, W.-Y.; Kathaperumal, M.; et al. Holographic three-dimensional telepresence using large-area photorefractive polymer. Nature 2010, 468, 80–83. [CrossRef][PubMed] photorefractive polymer. Nature 2010, 468, 80–83. 11. Park, M.; Chae, B.G.; Kim, H.; Hahn, J.; Moon, K.; Moon, K.; Kim, J. Digital Holographic Display System 11. Park, M.; Chae, B.G.; Kim, H.; Hahn, J.; Moon, K.; et al. Digital Holographic Display System with Large with Large Screen Based on Viewing Window Movement for 3D Video Service. ETRI J. 2014, 36, 232–241. Screen Based on Viewing Window Movement for 3D Video Service. ETRI J. 2014, 36, 232–241. [CrossRef] 12. Blinder, D.; Ahar, A.; Bettens, S.; Birnbaum, T.; Symeonidou, A.; Ottevaere, H.; Schretter, C.; Schelkens, P. 12. Blinder, D.; Ahar, A.; Bettens, S.; Birnbaum, T.; Symeonidou, A.; Ottevaere, H.; Schretter, C.; Schelkens, P. Signal processing challenges for digital holographic video display systems. Signal Process. Image Commun. Signal processing challenges for digital holographic video display systems. Signal Process. Image Commun. 2019, 70, 114–130. 2019, 70, 114–130. [CrossRef] 13. Kawashima, T.; Kakue, T.; Nishitsuji, T.; Suzuki, K.; Shimobaba, T.; Ito, T. Aerial projection of three- 13. Kawashima, T.; Kakue, T.; Nishitsuji, T.; Suzuki, K.; Shimobaba, T.; Ito, T. Aerial projection of dimensional motion pictures by electro-holography and parabolic mirrors. Sci. Rep. 2015, 5, 11750. three-dimensional motion pictures by electro-holography and parabolic mirrors. Sci. Rep. 2015, 5, 11750. 14. Inoue, T.; Takaki, Y. Table screen 360-degree holographic display using circular viewing-zone scanning. 14. Inoue, T.; Takaki, Y. Table screen 360-degree holographic display using circular viewing-zone scanning. Opt. Express 2015, 23, 6533–6542. Opt. Express 2015, 23, 6533–6542. [CrossRef][PubMed] 15. Sando, Y.; Barada, D.; Yatagai, T. Holographic 3D display observable for multiple simultaneous viewers 15. Sando, Y.; Barada, D.; Yatagai, T. Holographic 3D display observable for multiple simultaneous viewers from from all horizontal directions by using a time division method. Opt. Lett. 2014, 39, 5555. all horizontal directions by using a time division method. Opt. Lett. 2014, 39, 5555. [CrossRef][PubMed] 16. Jones, A.; McDowall, I.; Yamada, H.; Bolas, M.; Debevec, P. Rendering for an Interactive 360° Light Field 16. Jones, A.; McDowall, I.; Yamada, H.; Bolas, M.; Debevec, P. Rendering for an Interactive 360◦ Light Field Display. In Proceedings of the ACM SIGGRAPH, San Diego, CA, USA, 2007. Display. ACM Trans. Graph. 2007, 26, 40. Appl. Sci. 2019, 9, 705 16 of 16

17. Butler, A.; Hilliges, O.; Izadi, S.; Hodges, S.; Molyneaux, D.; Kim, D.; Kong, D. Vermeer: Direct Interaction with a 360◦ Viewable 3D Display. In Proceedings of the ACM Symposium on User Interface Software and Technology, Santa Barbara, CA, USA, 16–19 October 2011. 18. Yoshida, S. fVisiOn: 360-degree viewable glasses-free tabletop 3D display composed of conical screen and modular project arrays. Opt. Express 2016, 24, 13194–13203. [CrossRef][PubMed] 19. Takaki, Y. High-Density Directional Display for Generating Natural Three-Dimensional Images. Proc. IEEE 2006, 94, 654–663. [CrossRef] 20. Takaki, Y.; Nago, N. Multi-projection of lenticular displays to construct a 256-view super multi-view display. Opt. Express 2010, 18, 8824–8835. [CrossRef] 21. Hua, H.; Javidi, B. A 3D integral imaging optical see-through head-mounted display. Opt. Express 2014, 22, 13484. [CrossRef] 22. Kim, J.; Hong, K.; Lim, Y.; Kim, J.-H.; Park, M. Design options for 360 degree viewable table-top digital color holographic displays. In Proceedings of the SPIE Commercial + Scientiﬁc Sensing and Imaging, Orlando, FL, USA, 15–19 April 2018. 23. Chang, E.-Y.; Choi, J.; Lee, S.; Kwon, S.; Yoo, J.; Choo, H.-G.; Kim, J. 360-degree Color Hologram Generation for Real 3-D Object. Dig. Holography Three-Dimens. Imaging 2018, 57, A91–A100. [CrossRef] 24. Seo, W.; Song, H.; An, J.; Seo, J.; Sung, G.; Kim, Y.-T.; Choi, C.-S.; Kim, S.; Kim, H.; Kim, Y.; et al. Image Quality Assessment for Holographic Display. In Proceedings of the IS&T International Symposium on Electronic Imaging 2017, Burlingame, CA, USA, 29–31 January 2017. 25. Yoshikawa, H.; Yamaguchi, T. Image Quality Evaluation of a Computer-Generated Hologram. In Proceedings of the Digital Holography & 3-D Imaging Meeting, Orlando, FL, USA, 25–28 June 2018. 26. Hilaire, P.S. Modulation transfer function and optimum sampling of holographic stereograms. Appl. Opt. 1994, 33, 768–774. [CrossRef] 27. Ong, D.C.; Solanki, S.; Liang, X.; Xu, X. Analysis of laser speckle severity, granularity, and anisotropy using the power spectral density in polar-coordinate representation. Opt. Eng. 2012, 51, 054301. [CrossRef] 28. Takaki, Y.; Yokouchi, M. Speckle-free and grayscale hologram reconstruction using time-multiplexing technique. Opt. Express 2011, 19, 7567–7579. [CrossRef][PubMed] 29. Makowski, M. Minimized speckle noise in lens-less holographic projection by pixel separation. Opt. Express 2013, 21, 29205–29216. [CrossRef][PubMed] 30. Lim, Y.; Park, J.; Hahn, J.; Kim, H.; Hong, K.; Kim, J. Reducing speckle artifacts in digital holography by the use of programmable ﬁltration. ETRI J. 2019.[CrossRef]

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).