Virtual legality Virtual Reality and Augmented Reality – Legal Issues 2 • dentons.com With predictions that the impact of Virtual Reality (VR) and Augmented Reality (AR) is comparable to the impact of the internet1 Dentons, as part of its on-going TechTalk series, thought it was timely to examine the legal issues associated with this relatively new technology. The hype is real at least. There are predictions of the AR and VR sectors generating $90 billion by 2023.2 This article aims to get you Are VR and AR the AR comprehensively up to speed and up to date. It will be of particular same thing? By contrast, Augmented Reality interest to anyone in sectors such (AR) supplements the real world. First – some definitions. as media, retail, hotels, education, The user sees the real world but manufacturing, finance and VR with the addition of computer healthcare. generated images which are Virtual Reality (VR) uses software overlaid onto various real objects. Read on if you need a “VR and and devices to show (usually Augmentations can be sound, AR 101” session which gives you realistic) images, sounds and video, graphics or data. an overview and covers the legal other sensations (such as scent)3 issues. This article provides you AR achieves this by adding to replicate real or imaginary with what you need to know computer vision and object environments and to map the right now about: recognition to data about the real user’s physical presence in them. environment surrounding the user. VR’s aim is to be realistic and 1. What are VR and AR? AR presents that content overlaid immersive, and to simulate a onto the user’s real world view. 2. How does the VR/AR three dimensional environment The AR information itself can be technology work? and experience. VR does this either virtual or real data. 3. Who is who in the world by interactive software and of VR and AR? hardware which is controlled and AR devices range from experienced by movement of the smartphones and tablets (say, 4. Key legal issues associated human body, head or eyes. A VR for playing games like Pokémon with VR and AR. user can move or look around Go) to windscreens in vehicles 5. he future of VR and AR. and interact with the technology (for example, to display satnav via a screen or headset/goggles information) or HMDs worn (called a head-mounted display by the user (such as a harness, or HMD). helmet or eyeglasses). 1. See “Digital Trends: Why 2017 Will Be Shaped By VR, AR, AI and Personalized Digital Assistants” published by Newsweek.com 3 January 2017 at http://europe.newsweek.com/virtual-reality-virtual-reality-sets-phones-technology-537969?rm=eu 2. According to “Augmented And Virtual Reality To Hit $150 Billion, Disrupting Mobile By 2020” dated 6 April 2015 by Tim Merel from @DigiCapitalist at https://techcrunch.com/2015/04/06/augmented-and-virtual-reality-to-hit-150-billion-by-2020/ 3. Both scent and taste are current developments. See, for example, the VR experiment to taste virtual sushi and donuts which is described in the article “We Sampled The Gastronomic Frontier Of Virtual Reality” dated 29 January 2016 at http://www.npr.org/ sections/thesalt/2016/01/29/464885833/we-sampled-the-gastronomic-frontier-of-virtual-reality dentons.com • 3 The AR device displays the a three dimensional space). AR content by projecting through 6DoF systems plot the user’s or by reflecting off the surfaces movements along x, y and z axes of the device. to measure in every direction. In short, the main differences VR systems use a few mechanisms are that AR enhances reality to plot movement. These range whereas VR replaces reality with from light emitting diodes (LEDs) a simulated one. In doing so, Image of Sony PlayStation and cameras, to infrared sensors, there is a difference in the level of VR © 2016 Sony Interactive magnetometers, gyroscopes immersion of the user experience. Entertainment LLC or accelerometers. The aim is to minimise lag between How do AR and VR The experience of immersion user physical movements and actually work? depends on how wide the display image changes. The process of view is. A 360 degree display converting a user’s motion into VR and AR technology works in would be more immersive than commands in the VR computer slightly different ways. What they the 100 or 110 degree displays software is vital to the sense of generally have in common is the which are currently most common immersion. A low latency in terms need for a user to buy a device in VR HMDs. of lag is crucial to the quality and to download an app to it of the VR experience. Latency The headsets refresh the views as to get started. is usually measured usually in the user moves around. Users will milliseconds. feel disoriented and sick however VR if the field of view does not refresh VR systems mainly use one of two In VR, getting the maximum sense often enough. Companies such methods to track body motion. of immersion is key to a good as Sony impose standards on These are either optical tracking experience. its game developers to meet (using imaging devices to track certain frame rate per second body motion) or non-optical VR works by feeding video from (fps) requirements or they will not tracking (using other sensors a computer to the HMD or be certified on the PlayStation attached to the body such as smartphone/tablet —either by system.4 At least 60 frames per magnetic fields or sound waves). cable or wirelessly. The headsets second is needed to avoid user take the video feeds and send disorientation. Optical: them to a single display or to two A user will wear optical markers displays (one for each eye). How is motion tracked? (such as dots of highly reflective Sometimes the lenses in an HMD material) on parts of their body, are adjustable to match the VR devices track the user’s head the HMD or handheld controller distance between a user’s eyes. or arm/hand/body movements and a camera maps their This helps the image to be as and through the software shift movement to three dimensional focused and shaped as possible the picture in the headset display space. and facilitates a “stereoscopic” as the user moves around. VR three dimensional image (created devices constantly trace and Non-optical: by angling the two separate two retrace the real world movements dimensional images to simulate through a process called spatial A user will wear equipment how human eyes have small mapping. Tracking is measured (such as an exoskeleton motion differences in the way each one according to a “six degrees capture suit system or HMD) or views images). of freedom” (6DoF) system (a use a handheld controller which reference to the freedom of contains micro-electromechanical movement of a rigid body in sensors (such as accelerometers, 4. See Charlie Hall’s article for Polygon, “Sony to devs: If you drop below 60 fps in VR we will not certify your game. You cannot drop below 60 fps. Period. Ever.” dated 17 March 2016 at http://www.polygon.com/2016/3/17/11256142/sony-framer- ate-60fps-vr-certification 4 • dentons.com gyroscopes and magnetometers). through a camera, webcam These sensors convert movement or phone. into electrical signals and so track motion. Some newer technology There are two main types of AR: interpret electrical impulses AR that is marker-based or AR that directly from muscles to enable is marker-less or location-based. gesture-based control of the Marker-based AR: VR software. • This type uses a camera and a How are eyes tracked? visual marker to determine the centre, orientation and range of Eye tracking is a newer VR frontier its spherical coordinate system. and involves use of an infrared The AR device is activated when sensor to monitor eyes and where an item (the marker) is placed the user is looking inside the in front of a webcam (which is HMD. This allows more precise connected to or embedded in reactions in VR imagery and a the AR device). The webcam depth of field that is more realistic. captures a video feed (by For example, the VR display might taking lots of photos in quick blur the foreground slightly if the succession) and processing or user is looking into the distance. recognising the information and – which is what naturally would patterns in the photos (such occur when eyes are looking into as a barcode or symbol). The the distance in the real world. This AR computer then recognises Image of Spyglass Compass and feels more natural to the user as that information and overlays GPS navigation app for iPhone and it replicates the way the human the marker with an image or iPad © 2016 Happy Magenta UAB brain works and so a user is less animation. A classic example likely to feel disoriented or dizzy. is the world record line which Who is in on the act? is dynamically added across Other senses: the lanes to show the current There are a number of key players VR devices are now aiming to world record in a broadcast of in the VR and AR sectors – either increase the sense of immersion a sports race. When a camera exploiting it, producing content by adding aural/video technology points at the marker, the digital or producing the hardware and and apps that give the user the image is overlaid at that point software.