Augment Hololens' Body Recognition and Tracking

Total Page:16

File Type:pdf, Size:1020Kb

Augment Hololens' Body Recognition and Tracking AUGMENT HOLOLENS’ BODY RECOGNITION AND TRACKING CAPABILITIES USING KINECT A thesis submitted in partial fulfillment of the requirement for the degree of Master of Science By KRISHNA CHAITHANYA MATHI B.Tech, Jawaharlal Nehru Technological University,2012 2016 Wright State University WRIGHT STATE UNIVERSITY GRADUATE SCHOOL th January 12 , 2017 I HEREBY RECOMMEND THAT THE THESIS PREPARED UNDER MY SUPERVISION BY Krishna Chaithanya Mathi ENTITLED Augment HoloLens‟ Body Recognition and Tracking Capabilities Using Kinect BE ACCEPTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF Master of Science. _____________________________________ Yong Pei, Ph.D. Thesis Director _____________________________________ Mateen M. Rizki, Ph.D. Chair, Department of Computer Science and Engineering Committee on Final Examination _____________________________________ Yong Pei, Ph.D. _____________________________________ Mateen M. Rizki, Ph.D. _____________________________________ Paul Bender, Ph.D. _____________________________________ Robert E.W. Fyffe, Ph.D. Vice President for Research and Dean of the Graduate School Abstract Mathi, Krishna Chaithanya. M.S. Department of Computer Science and Engineering, Wright State University, 2016. AUGMENT HOLOLENS‟ BODY RECOGNITION AND TRACKING CAPABILITIES USING KINECT. In this thesis, we are primarily interested in exploring the HoloLens technologies for medical practices. Particularly, we will address the limitation of HoloLens‟ capability in human body sensing, recognition and tracking. We will then introduce and demonstrate the use of Kinect to augment HoloLens‟ sensory and processing capabilities in order to produce time and space-synchronized immersive environment with both virtual body and real body of the same patient for supporting distributed medical collaborations. Specifically, we are looking at a distributed solution in which we are collecting the patient body data from Kinect, followed by body recognition and position/motion tracking processing at a server; Then we will display and align the virtual body object, composed based on the Kinect-obtained patient body or body part position information, with the real patient body in the doctor‟s augmented reality environment using HoloLens. We have implemented a prototype system to illustrate the distributed solution. Our experimental results using the prototype system demonstrate successful data flow and workflows. This results in effective time and space-synchronizations between the virtual and real bodies in the augmented reality environment. III Through the prototype system, we can track movement, e.g., an arm, in all directions by the Kinect and the coordinates will be sent to HoloLens through which we will augment the real body with a virtual meshed body composed from Kinect-collected data. We have demonstrated that the solution and prototype can effectively address related limitations of HoloLens, and successfully present in real-time the movement of a recognized subject in the form of both virtual and real world human body, and aligned with each other. We believe this work has helped lay a solid foundation for future works to build complete virtual human atlas into the AR views, which may potentially assist surgeons with context-aware guidance by displaying him/her the instructions and feeding him with only the closely associated information through the course of operation. This distributed collaborative solution is of particular advantage for doctors to master faster and more safely advanced and complicated surgical techniques, like using advanced radiation therapy system for cancer treatments by eliminating the physical barriers such as distance, space, resource. IV CONTENTS Chapter 1 INTRODUCTION ......................................................................................................................................... 1 1.1 OVERVIEW .......................................................................................................................................................... 1 1.2 History of Augmented Reality................................................................................................................................ 3 1.3 Difference and Similarities between AR and VR ................................................................................................... 5 1.4 AR Application Domains ....................................................................................................................................... 8 1.4.1 Medical ........................................................................................................................................................... 8 1.4.2 Engineering Design ........................................................................................................................................ 9 1.4.3 Sports and Entertainment ............................................................................................................................... 9 1.4.4 Military Training .......................................................................................................................................... 10 1.4.5 Augmented Mapping .................................................................................................................................... 11 Chapter 2 OVERVIEW OF HOLOLENS .................................................................................................................. 12 2.1 Basic Knowledge about HoloLens ....................................................................................................................... 12 2.2 HoloLens Specifics .............................................................................................................................................. 14 2.3 Shell and Mixed Reality ....................................................................................................................................... 14 2.4 Configuring HoloLens .......................................................................................................................................... 16 2.5 Developing Application for HoloLens ................................................................................................................. 17 2.6 Understanding of HoloLens Application Blocks .................................................................................................. 20 2.6.1 HoloLens Spatial Coordinates ...................................................................................................................... 20 2.6.2 Interact with Holograms in HoloLens .......................................................................................................... 21 2.6.3 Spatial Mapping and Environment Understanding ....................................................................................... 24 Chapter 3 KINECT-BASED HUMAN BODY RECOGNITION AND POSITION/MOTION TRACKING ....... 27 3.1 Capabilities to Meet Our Need ............................................................................................................................. 28 3.2 Body Tracking Using Kinect ................................................................................................................................ 29 Chapter 4 DEVELOMENT OF THE PROTOTYPE SYSTEM ............................................................................... 31 4.1 Design of a Basic HoloLens Project ..................................................................................................................... 31 4.2 Our Prototype System Implementation ................................................................................................................ 34 4.2.1 Client Server Architecture ............................................................................................................................ 35 4.2.2 Deploying process of the application............................................................................................................ 39 4.3 Implementation Results ........................................................................................................................................ 40 Chapter 5 CONCLUSION AND FUTURE WORK ................................................................................................... 45 5.1 Conclusion ........................................................................................................................................................... 45 5.2 Future Work ......................................................................................................................................................... 46 REFERENCES .............................................................................................................................................................. 47 V LIST OF FIGURES Figure 1 Example of Augmented Reality on a Globe Image [2] ..................................................... 2 Figure 2 augmented reality and virtual reality [10] ......................................................................... 5 Figure 3 Virtual reality using Oculus Rift device [10] .................................................................... 6 Figure 4 Augmented reality in medical field [9] .............................................................................. 8 Figure 5 Augmented reality in sports [8] ......................................................................................... 9 Figure 6 HoloLens
Recommended publications
  • Samsung Announces New Windows-Based Virtual-Reality Headset at Microsoft Event 4 October 2017, by Matt Day, the Seattle Times
    Samsung announces new Windows-based virtual-reality headset at Microsoft event 4 October 2017, by Matt Day, The Seattle Times Samsung is joining Microsoft's virtual reality push, Microsoft also said that it had acquired AltspaceVR, announcing an immersive headset that pairs with a California virtual reality software startup that was Windows computers. building social and communications tools until it ran into funding problems earlier this year. The Korean electronics giant unveiled its Samsung HMD Odyssey at a Microsoft event in San ©2017 The Seattle Times Francisco recently. It will sell for $499. Distributed by Tribune Content Agency, LLC. The device joins Windows-based immersive headsets built by Lenovo, HP, Acer and Dell, and aimed for release later this year. Microsoft is among the companies seeking a slice of the emerging market for modern head-mounted devices. High-end headsets, like Facebook-owned Oculus's Rift and the HTC Vive, require powerful Windows PCs to run. Others, including the Samsung Gear VR and Google's Daydream, are aimed at the wider audience of people who use smartphones. Microsoft's vision, for now, is tied to the PC, and specifically new features in the Windows operating system designed to make it easier to build and display immersive environments. The company also has its own hardware, but that hasn't been on display recently. Microsoft's HoloLens was a trailblazer when it was unveiled in 2015. The headset, whose visor shows computer-generated images projected onto objects in the wearer's environment without obscuring the view of the real world completely, was subsequently offered for sale to developers and businesses.
    [Show full text]
  • Holographic Computer
    MECHANICAL ENGINEERING | DECEMBER 2018 | P.33 bioengineering WEARABLE Holographic Computer Microsoft’s HoloLens mixed reality headset is a breakthrough in human-computer interaction. STORY BY AGAM SHAH • ILLUSTRATION BY ZINA SAUNDERS he Kinect camera, which headset, Thyssenkrupp is equipping Microsoft released in 2010, elevator service technicians with brought unprecedented HoloLens to visualize and identify T levels of gesture and voice problems ahead of a job, and have interaction to video games. “When remote, hands-free access to technical we created Kinect for Xbox 360, and expert information when onsite. we aimed to build a device capable The company says that has helped of recognizing and understanding improve service times. Ford is also people so that computers could using HoloLens, enabling their design operate in ways that are more and engineering teams to visualize human,” said Alex Kipman, technical that projects the graphics onto a full-scale models in 3-D. They’ve fellow at Microsoft, who helped lens. As a user walks around a room converted processes that used to invent the camera. or turns his head, the position or take weeks down to days, and more Microsoft wanted to build on orientation of the graphical image is easily and securely share ideas across the success of the Kinect’s sensor altered so that it appears to the user the company and consider more technology by parlaying it into a as occupying a consistent location concepts than before. And NASA has radically new device—a mixed reality in space. provided HoloLens to astronauts on headset that would project computer- HoloLens has emerged as the go- the International Space Station as a generated images seemingly into real to device for engineers to integrate holographic instruction manual and a space.
    [Show full text]
  • New Realities Risks in the Virtual World 2
    Emerging Risk Report 2018 Technology New realities Risks in the virtual world 2 Lloyd’s disclaimer About the author This report has been co-produced by Lloyd's and Amelia Kallman is a leading London futurist, speaker, Amelia Kallman for general information purposes only. and author. As an innovation and technology While care has been taken in gathering the data and communicator, Amelia regularly writes, consults, and preparing the report Lloyd's does not make any speaks on the impact of new technologies on the future representations or warranties as to its accuracy or of business and our lives. She is an expert on the completeness and expressly excludes to the maximum emerging risks of The New Realities (VR-AR-MR), and extent permitted by law all those that might otherwise also specialises in the future of retail. be implied. Coming from a theatrical background, Amelia started Lloyd's accepts no responsibility or liability for any loss her tech career by chance in 2013 at a creative or damage of any nature occasioned to any person as a technology agency where she worked her way up to result of acting or refraining from acting as a result of, or become their Global Head of Innovation. She opened, in reliance on, any statement, fact, figure or expression operated and curated innovation lounges in both of opinion or belief contained in this report. This report London and Dubai, working with start-ups and corporate does not constitute advice of any kind. clients to develop connections and future-proof strategies. Today she continues to discover and bring © Lloyd’s 2018 attention to cutting-edge start-ups, regularly curating All rights reserved events for WIRED UK.
    [Show full text]
  • Entity Framework with Microsoft Azure
    Microsoft Azure Step by Step Guide This free book is provided by courtesy of C# Corner and Mindcracker Network and its authors. Feel free to share this book with your friends and co-workers. Please do not reproduce, republish, edit or copy this book. Nitin Pandit Software Engineer, Microsoft.Net Consultant Noida C# corner Delhi Chapter Lead C# corner MVP Amit Mishra (Co-Author, Format Editor) Index Page No Chapter 1 - Getting Started With Microsoft Azure 1-17 Chapter 2 - How To Host Your Web Apps On Azure Via FileZilla 18-31 Chapter 3 - Configuring A Custom Domain Name For An Azure 32-40 Website Chapter 4 - Virtual Machine In Microsoft Azure Step by Step - Part 41-50 One Chapter 5 - Creating A Virtual Disk Virtual Machine In Microsoft 51-66 Azure Step by Step: Part Two Chapter 6 - Use of SQL Azure In Visual Studio 2015 Step By Step 67-98 Guide Chapter 7 - Entity Framework With Microsoft Azure SQL 99-113 Chapter 8 - Introducing U-SQL- Make Big Data Processing Easy 114-117 Chapter 9 - Installation Of Azure SDK And Blob Storage In Azure: 118-139 Part 1 Chapter 10 - Upload Image To Azure Blob Storage In ASP.NET: Part 140-163 2 Chapter 11 - Demo Project Azure Blob Storage With ASP.NET MVC 164-204 5.0: Part 3 Chapter 12 - Getting Started With Azure DocumentDB: Day 1 205-224 Chapter 13 - Querying In The Azure DocumentDB: Day Two 225-242 Chapter 14 - Manage Azure Mobile Services: Part One 243-259 Chapter 15 - How To Create A Mobile App In Microsoft Azure 260-277 Chapter 16 - Manage Database Settings Of Azure Mobile Services 278-281 Chapter 17
    [Show full text]
  • Windows Protocols and Test Suites Overview
    Windows Protocols and Test Suites Overview Althea Champagnie, Senior Program Manager Min Li, Principal Software Engineering Manager Agenda • Overview of Windows Protocols • What do the Windows protocols cover • What resources are available • Introduction to Protocols Test Suites • What test suites are available and how they work • What test cases are covered by the RDP test suites • Where to find the test suites and how to use them Overview of Windows Protocols Althea Champagnie, Senior Program Manager Microsoft Interoperability Guidelines Our goal is to ensure that third-party developers have access to the interoperability information, tools, and resources they need to successfully develop the services, products, and platforms that apply to Microsoft We love our developers! Technical Events Tools Documents Support Events • Co-sponsored industry events • SNIA Storage Development Conference • SambaXP • RDP-focused events • Presentations, I/O Labs, Meetups, and more. • A virtual RDP event is being planned for later in the year • Find more info at InteropEvents.com. What Are The • Detailed technical specifications for Microsoft protocols Protocols that are implemented and used by Windows to interoperate or communicate with other Microsoft Technical products Specs? • Descriptions of extensions to industry-standard and other published protocols that are used by Windows • Essential resource for developers doing protocol implementation work • Includes use cases • Find them at https://aka.ms/techspecs What Are The • “… illustrate how Windows
    [Show full text]
  • Microsoft Aims at Apple with High-End Pcs, 3D Software 26 October 2016
    Microsoft aims at Apple with high-end PCs, 3D software 26 October 2016 unlocking a more natural and immersive way to create on the thinnest display ever built," said Microsoft vice president Terry Myerson, The new PC has an ultra-HD 4.5K display, offering more pixels than most new high-definition televisions. It will be available in "limited quantities" for the upcoming holiday season, with more units available in 2017. The new device drew a mixed response from analysts. "Yes, this is very cool. But what's the market for a $3,000 computer you can draw on? How many people have that job?" asked Benedict Evans of the Microsoft chief executive officer Satya Nadella talks at a venture firm Andreessen Horowitz in a tweet. Microsoft news conference October 26, 2016 in New York But Avi Greengart at the research firm Current Analysis said the new Microsoft products "are Microsoft launched a new consumer offensive aimed at setting the high mark for the Windows Wednesday, unveiling a high-end computer that ecosystem." challenges the Apple iMac along with an updated Windows operating system that showcases three- dimensional content and "mixed reality." The US tech giant announced its first desktop computer, called Surface Studio, a $3,000 high- end "all-in-one" device that aims at creative professionals, a segment dominated by Apple. "We're creating a new category that transforms your desk into a creative studio," Microsoft chief executive Satya Nadella said at the unveiling in New York. With a large, 28-inch (71-centimeter) hinged touchscreen display touted as "the thinnest desktop monitor ever created," Surface Studio Microsoft Corporate VP of Devices, Panos Panay adds to the Microsoft lineup of tablet and laptop introduces Microsoft Surface Studio on October 26, 2016 devices for the premium segment.
    [Show full text]
  • Evaluating the Microsoft Hololens Through an Augmented Reality Assembly Application" (2017)
    Mechanical Engineering Conference Presentations, Mechanical Engineering Papers, and Proceedings 5-5-2017 Evaluating the Microsoft oloLeH ns through an augmented reality assembly application Gabriel Evans Iowa State University, [email protected] Jack Miller Iowa State University, [email protected] Mariangely Iglesias Pena Iowa State University, [email protected] Anastacia MacAllister Iowa State University, [email protected] Eliot H. Winer Iowa State University, [email protected] Follow this and additional works at: https://lib.dr.iastate.edu/me_conf Part of the Applied Mechanics Commons, and the Manufacturing Commons Recommended Citation Evans, Gabriel; Miller, Jack; Pena, Mariangely Iglesias; MacAllister, Anastacia; and Winer, Eliot H., "Evaluating the Microsoft HoloLens through an augmented reality assembly application" (2017). Mechanical Engineering Conference Presentations, Papers, and Proceedings. 179. https://lib.dr.iastate.edu/me_conf/179 This Article is brought to you for free and open access by the Mechanical Engineering at Iowa State University Digital Repository. It has been accepted for inclusion in Mechanical Engineering Conference Presentations, Papers, and Proceedings by an authorized administrator of Iowa State University Digital Repository. For more information, please contact [email protected]. Evaluating the Microsoft oloLeH ns through an augmented reality assembly application Abstract Industry and academia have repeatedly demonstrated the transformative potential of Augmented Reality (AR) guided assembly instructions. In the past, however, computational and hardware limitations often dictated that these systems were deployed on tablets or other cumbersome devices. Often, tablets impede worker progress by diverting a user's hands and attention, forcing them to alternate between the instructions and the assembly process. Head Mounted Displays (HMDs) overcome those diversions by allowing users to view the instructions in a hands-free manner while simultaneously performing an assembly operation.
    [Show full text]
  • Virtual & Augmented Reality
    EXCERPTED FROM THE ORIGINAL: See inside cover for details. FINANCIAL BACKING GLOBAL INTEREST SHIPPING OUT SELLING OUT INTEREST IN THE PAST READY TO BUILD A WAVE OF CONTENT ON THE WAY RETAIL VALUE HOME REDESIGN, REIMAGINED EASIER TO IMAGINE YOURSELF AT HOME Video • Jaunt The Ecosystem • NextVR Virtual Reality / Augmented Reality • VRSE • Oculus Story Studio • GoPro • IG Port Processors Games • TI Applications • Sony • Qualcomm • Ubisoft 3D Audio • STMicro Graphics • CCP Games • Nvidia • TI • Realtek • Oculus Story Studio • AMD • Wolfson • Himax • Tammeka Games • Qualcomm • Realtek • MediaTek • Pixel Titans • Intel • Capcom Augmented Reality Virtual Reality Engineering • Microsoft HoloLens • Facebook Oculus Head–mounted devices • Autodesk • Google Glass • Samsung Gear VR • Dassault Systèmes • Magic Leap • Google Cardboard • IrisVR • Atheer • HTC Vive • Visidraft • Osterhout Design • Sony PSVR • MakeVR Memory Group • Vuzix iWear (DRAM/SSD) • VR Union Claire • Micron Healthcare • Samsung • Psious • SK Hynix • zSpace • Toshiba • Conquer Mobile • 3D Systems Social • Altspace VR • High Fidelity • Podrift Commerce • Sixense {shopping} • Matterport {real estate} Display • Samsung • JDI Cameras • Himax • 360Heros • Crystal • GoPro Odyssey 3D Lenses • Nokia OZO • Wearality • Jaunt NEO • Zeiss • Matterport Pro 3D Components • Canon • Nikon Haptics • Largan • Alps Position/ Room Tracker • AAC • Hon Hai • Nidec • Pegatron • Flex • Jabil • HTC Motion Sensors • Leap Motion • InvenSense • TI • STMicro • Honeywell The Ecosystem Virtual Reality / Augmented
    [Show full text]
  • [MS-RDPRFX]: Remote Desktop Protocol: Remotefx Codec Extension
    [MS-RDPRFX]: Remote Desktop Protocol: RemoteFX Codec Extension Intellectual Property Rights Notice for Open Specifications Documentation . Technical Documentation. Microsoft publishes Open Specifications documentation (“this documentation”) for protocols, file formats, data portability, computer languages, and standards support. Additionally, overview documents cover inter-protocol relationships and interactions. Copyrights. This documentation is covered by Microsoft copyrights. Regardless of any other terms that are contained in the terms of use for the Microsoft website that hosts this documentation, you can make copies of it in order to develop implementations of the technologies that are described in this documentation and can distribute portions of it in your implementations that use these technologies or in your documentation as necessary to properly document the implementation. You can also distribute in your implementation, with or without modification, any schemas, IDLs, or code samples that are included in the documentation. This permission also applies to any documents that are referenced in the Open Specifications documentation. No Trade Secrets. Microsoft does not claim any trade secret rights in this documentation. Patents. Microsoft has patents that might cover your implementations of the technologies described in the Open Specifications documentation. Neither this notice nor Microsoft's delivery of this documentation grants any licenses under those patents or any other Microsoft patents. However, a given Open Specifications document might be covered by the Microsoft Open Specifications Promise or the Microsoft Community Promise. If you would prefer a written license, or if the technologies described in this documentation are not covered by the Open Specifications Promise or Community Promise, as applicable, patent licenses are available by contacting [email protected].
    [Show full text]
  • Mixed Reality
    Mixed Reality Matt Fedorovich Authorized Reseller Lead, Immersive Technology Agenda Part One 01 Remote Assist Overview Part Two 02 Device and Infrastructure Readiness Part Three 03 End User Readiness Part Four 04 Recent Stories Remote Work for the Future Remote Assistance Remote Collaboration • “See what I see” • Everyone in virtual space – AR, VR or MR • Leverage physical world for interaction • Leverage digital world for interaction • Cross-Platform • Cross-Platform • Integration into existing systems is key • Integration into existing systems is key • Solution : D365 Remote Assist • Solutions : Spatial.io & AltSpace VR Remote Assist Overview Work together from anywhere. Microsoft Dynamics 365 Dynamics 365 Remote Assist Empower technicians to collaborate more efficiently by working together from different locations with Dynamics 365 Remote Assist on HoloLens, Android, or iOS devices. Enabled by Microsoft Teams, OneDrive and Dynamics 365. Dynamics 365 Guides Operators learn during the flow of work by providing holographic instructions when and where they are needed. These instruction cards are visually tethered to the place where the work must be done, and can include images, videos, and 3D holographic models How It Works iOS & Android Device Microsoft HoloLens 2 Front Line Technician Remote Expert Technician needs help. Login to Remote Assist Receives call on their Teams Mobile or PC Application and calls expert Device Requires Device Requires Remote Assist App Remote Assist Microsoft Teams Teams License On iOS / Android License On PC or Mobile (w/ Video) Per user Remote Assist App OR On Microsoft HoloLens 2 Per Device Key Journey Points Envisioning & Planning Readiness & Support Innovation & Integration Accelerate Infrastructure Readiness Identify key components that are needed before you think about devices and end users.
    [Show full text]
  • Remote Desktop Protocol: Graphics Pipeline Extension
    [MS-RDPEGFX-Diff]: Remote Desktop Protocol: Graphics Pipeline Extension Intellectual Property Rights Notice for Open Specifications Documentation . Technical Documentation. Microsoft publishes Open Specifications documentation for protocols, file formats, languages, standards as well as overviews of the interaction among each of these technologies. Copyrights. This documentation is covered by Microsoft copyrights. Regardless of any other terms that are contained in the terms of use for the Microsoft website that hosts this documentation, you may make copies of it in order to develop implementations of the technologies described in the Open Specifications and may distribute portions of it in your implementations using these technologies or your documentation as necessary to properly document the implementation. You may also distribute in your implementation, with or without modification, any schema, IDL's, or code samples that are included in the documentation. This permission also applies to any documents that are referenced in the Open Specifications. No Trade Secrets. Microsoft does not claim any trade secret rights in this documentation. Patents. Microsoft has patents that may cover your implementations of the technologies described in the Open Specifications. Neither this notice nor Microsoft's delivery of the documentation grants any licenses under those or any other Microsoft patents. However, a given Open Specification may be covered by Microsoft Open Specification Promise or the Community Promise. If you would prefer a written license, or if the technologies described in the Open Specifications are not covered by the Open Specifications Promise or Community Promise, as applicable, patent licenses are available by contacting [email protected]. Trademarks. The names of companies and products contained in this documentation may be covered by trademarks or similar intellectual property rights.
    [Show full text]
  • Mixed Reality & Microsoft Hololens
    Mixed Reality & Microsoft HoloLens Laboratorio di Realtà Virtuale Manuel Pezzera – [email protected] Mixed Reality • “Mixed reality (MR) is a spectrum of immersive experiences, connecting and blending physical and digital worlds together in augmented-reality and virtual-reality applications” [cit. Microsoft] Mixed Reality Augmented and Virtual Reality • Virtual Reality: it tricks your eyes into thinking that you are somewhere else. • Augmented reality: it is an extra dimension over VR since it can show the real world around wearer with graphic overlayed. • AR can also totally immerse user just like VR by simply showing pixels everywhere with no transparency to the real world. • In virtual world, users are best advised to stay seated to avoid collision with physical objects which they cannot see in the real world. What is a hologram? • A hologram is an object like any other object in the real world, with only one difference: instead of being made of physical matter, a hologram is made entirely of light. • Holographic objects can be viewed from different angles and distances, just like physical objects. They can also interact with the environment (e.g., bouncing ball on a table) • Hologram do not offer any physical resistance when touched or pushed because they don’t have a mass. Mixed reality vs Augmented Reality • MR can be mentioned as an enhanced form of Augmented Reality (AR). In AR the visible natural environment is overlaid with a layer of digital content. • But the case is slightly different with MR, as virtual objects are integrated into and responsive to the real world. For instance, a virtual ball under your desk would be blocked from a user’s view unless he bends.
    [Show full text]