New Realities New Realities, Synthetic Media, 00 01 02 03 04 05 06 07 08 09 10 11 12 News & Information

Keeping Track of New Realities

As a constellation of technologies AR vs. VR MR vs. XR DR is AR including smart eyewear, game Though both can be experienced via Mixed reality (MR) anchors virtual ele- (but not the other way around) engines, natural language pro- smart glasses or head-mounted displays, ments to corresponding physical elements Diminished reality (DR) is not fully cessing and generation, volumet- augmented reality (AR) and virtual reality in your environment—you can still phys- immersive, leaving you anchored in your ric video, and haptics continues to (VR) are fundamentally different, and ically interact with objects and surfaces, physical environment but with certain evolve, it is critical that we draw should be treated as such. AR makes but their appearance and reactivity may visuals, sounds, or other sensory elements distinctions between the various digital alterations or additions to your be virtually altered or enhanced. MR suppressed. Thus, all DR is typically digitally mediated realities that existing environment, but you generally experiences do not take place fully in considered a subset of AR—that does not span the reality-virtuality spec- remain oriented to your physical sur- the physical nor the virtual world, but mean, however, that all AR is DR. trum. With AR, VR, MR, XR, and roundings. VR immerses you fully in a in a hybrid of the two. Extended reality virtual environment, one that is either ar- (XR) is more of an umbrella term that DR—and more R’s inevitably on tificially generated or emulates real-world spans the reality-virtuality continuum, the way—reviewing their defini- surroundings other than your own. including AR, MR, and VR. XR some- tions and differences is the first While they may share hardware, AR and times overlaps with varying definitions step toward better understanding VR each have unique applications, with of the virtuality-reality spectrum, spatial the future of the field. enterprise and consumer use cases for AR computing, or Web 3.0, and the so-called eclipsing those for VR at present. metaverse.

Technology now blends the physical and virtual worlds.

1ST YEAR ON THE LIST Diminished Reality

KEY INSIGHT EXAMPLES DISRUPTIVE IMPACT EMERGING PLAYERS DR has existed in some form for over a Diminished reality forces us to rethink • Amazon Sumerian Diminished reality (DR) decade, with one of the most ubiquitous how digital mediation can shape our • Apple’s ARKit is the field of AR that examples being noise-canceling head- surroundings by subtractive rather than • Google’s ARCore focuses on virtually phones. But as the technology matures additive means, improving quality of and is developed for the audiovisual ap- life for a broad range of users, but not • Vuforia Engine masking, reducing, or plications of smart eyewear, it will allow without risks. suppressing features of users to target specific stimuli to suppress, isolating a specific speaker’s words and one’s environment. As appearance in a crowded room, or re-

Diminished reality has the power to remove ob- smart glasses prolifer- moving all advertising from view during jects from view, seamlessly filling in backgrounds ate and AR becomes a walk through the city center. But there to mask the objects’ absence. are also therapeutic applications for those commonplace over the with unique sensitivities, such as for next decade, DR pres- those suffering from PTSD. One study ents an opportunity to published by the Institute of Electrical and Electronics Engineers outlines a series of virtually shape our re- experimental workshops that used DR to ality rather than simply assist individuals on the autism spectrum “who are adversely affected by continu- build on top of it. ously changing surroundings or distract- ing visual incidents.”

4TH YEAR ON THE LIST Smart Eyewear and Head-Mounted Displays

KEY INSIGHT EXAMPLES DISRUPTIVE IMPACT EMERGING PLAYERS The transition from hands-on to heads- Smart eyewear is expected to upend • Amazon’s Echo Frames Smart eyewear and up mobile computing has begun, with industries and interfaces designed for the • Apple’s forthcoming smart eyewear head-mounted displays everyday smart eyewear trickling into smartphone by offering a more versa- product (HMDs) are already the consumer market. Amazon’s Echo tile hands-free alternative to the trusted Frames, which were quietly made avail- mobile device. HMDs, on the other hand, • Epson Moverio on the market; as the able to the public at the end of 2020, have already begun to reshape enterprise • Facebook’s Oculus Quest 2 devices spread, gener- prove that smart glasses can have “dumb” solutions, from the field to the factory • Microsoft’s HoloLens 2 lenses—the tech in these frames is purely floor. Current smart eyewear, often ating more real-world auditory (they have an on-board Alexa designed to resemble recognizable tradi- • Solos data with which to up- voice assistant), using bone conduction tional eyeglasses with simple but useful • Snap Spectacles 3 date and fine-tune their technology rather than in-ear compo- connected features, is meant to seamlessly • Vue nents. The audio-first approach is a clever blend into our everyday lives and facili- hardware and operat- way to ease smart eyewear into the mar- tate daily activities. HMDs, which offer • Vuzix Blade ing systems, they are ket, avoiding the privacy concerns sur- more robust functionality but are too strongly positioned to rounding video-enabled products like the cumbersome and restrictive for casual use conspicuous—and widely criticized—orig- on the go, are more commonly used in replace smartphones inal Google Glass. It also gives manufac- controlled workplace environments, or as the primary personal turers more time to engineer the complex for mostly stationary entertainment and smart lens systems that will eventually gaming. As smart eyewear adds more ad- device. support XR applications. Watch for sleek- vanced features, and HMDs become more er but simpler smart eyewear devices and lightweight and comfortable, the two The first generation of smart eyewear resembles bulkier but more powerful HMDs like closely related technologies may begin to traditional glasses, but future iterations will incor- porate the technology into something akin to a Facebook’s Oculus Quest and Microsoft’s converge. contact lens. HoloLens to inch closer to one another in terms of capabilities and form factor.

4TH YEAR ON THE LIST AR for the Enterprise

KEY INSIGHT EXAMPLES DISRUPTIVE IMPACT EMERGING PLAYERS Microsoft’s HoloLens 2 head-mounted Companies of all sizes and industries • CAE From factory floors to display was designed specifically with should be asking themselves where • GIGXR virtual meeting rooms, business solutions in mind, incorporat- AR can be implemented to streamline • Hevolus AR has a broad range of ing cloud and AI functionality, interop- and enhance functions and processes erability with industry partners, and a throughout the organization. AR devices • Kognitiv Spark business applications, suite of developer tools. The device has are now being offered as part of a larger • Medivis fueling accelerated in- already been adapted for the U.S. Army, ecosystem, compatible with third-party with thermal imaging and night vision operating systems and developer kits that • Nreal vestment and growth in among its many advanced capabilities. allow smaller companies to plug in to • PTC the field, while VR takes Smart eyewear and HMD maker Nreal existing systems, while bigger companies • Spatial longer to mature. has announced an Android-compatible can design and customize their own. “all in one” enterprise headset that looks • Trimble more like a helmet than a pair of glasses, with on-board edge computing capabilities. The uses of these business-focused headsets are wide-ranging, encompass- ing everything from monitoring supply Microsoft’s HoloLens 2 was designed specifically chains and complex equipment via digital with business solutions in mind. twins, to hosting remote meetings in 3D, to providing guided AR tutorials as part of workplace training.

7TH YEAR ON THE LIST Holograms

KEY INSIGHT EXAMPLES DISRUPTIVE IMPACT EMERGING PLAYERS In December 2020, Netflix launched an The accurate digital reproduction of faces, • Blank XR Holograms are light bodies, and other complex structures in AR hologram tool ahead of releasing the • Netflix field recordings that, George Clooney movie “The Midnight dynamic 3D form is critical to the evolu- tion of augmented and virtual reality. As • Portl when reproduced, can Sky.” It let people record and upload a short video message and see it trans- smart eyewear edges out the smartphone • Kaleida as our primary personal device, holo- appear as static or formed into AR holograms. Blank XR is grams, variously combined with deepfake • Eyellusion developing a concert platform that would dynamic three-dimen- technology and synthetic media, may • Base Hologram Productions sional visuals. The term allow fans to engage with musicians’ soon inhabit our everyday environments. Prominent attorney Robert Kardashian (left), who holograms via a mixed-reality applica- They’ll represent a range of artificially • Hologram USA passed away in 2003, was re-created in holo- is also more generally tion. Los Angeles startup Portl launched graphic form (right) by production house Kaleida generated characters, celebrity stand-ins, • EchoPixel phone booth-sized boxes that can project in 2020 as a birthday gift for his daughter Kim. applied to any image brand spokespeople, historical figures, a real-time, full-size holographic likeness and lost loved ones. In the medical field, • OpenSight Image Credit: GETTY IMAGES/@KIMKARDASHIAN that is rendered to ap- of a person or character. Holograms have holographic mapping can provide doc- • Crypton Future Media pear in 3D. been key to successful concert tours fea- tors with a 360-degree view of a patient’s turing bygone stars like Roy Orbison and internal organs, vessels, bones, and tissue, Frank Zappa, and will soon allow produc- which can assist with diagnostics and tion companies to draw popular synthetic surgeries, with multiple apps already approved by the U.S. Food and Drug Ad- media influencers like Lil Miquela and ministration. As it evolves, this technol- artists like Japanese act Hatsune Miku ogy could be used in remote workplace out of our screens and into our spatial training and collaboration. Expect to hear environment. more about holograms as resolution, vol- umetrics, and depth of field improve, and as 5G fuels the level of high-bandwidth instantaneous data transfer needed for lifelike holographic streaming.