Design and Development of a Mixed Reality Application in the Automotive Eld
Total Page:16
File Type:pdf, Size:1020Kb
Politecnico di Torino Department of Control and Computer Engineering (DAUIN) Master Degree in Computer Engineering Design and development of a Mixed Reality application in the automotive eld Supervisors: Author: Prof. Maurizio Morisio Giovanna Galeano December 2017 Contents 1 Mixed Reality1 1.1 Real Environment . .2 1.2 Augmented Reality . .2 1.2.1 Augmented Reality (AR) Categories . .2 1.2.2 Key Components to Augmented Reality Devices . .3 1.2.3 AR headsets categories . .4 1.3 Augmented Virtuality . .5 1.4 Virtual Reality . .5 1.4.1 Virtual Reality (VR) Categories . .5 1.4.2 Key Components in a Virtual Reality System . .6 1.4.3 Key Components Inside of a Virtual Reality Headset . .7 1.4.4 Performance parameters . .8 2 Context 11 2.1 Technological scouting . 12 2.1.1 Automotive eld . 13 2.1.2 Other examples . 17 3 System design 23 3.1 Helmet-mounted display design features . 25 3.2 Devices Comparison . 28 3.2.0.1 Final considerations . 29 3.3 Architecture design . 32 4 Development environment 35 4.1 Game engine components . 36 4.2 Game engines for Mixed Reality . 37 4.2.1 Unreal Engine 4 . 37 4.2.2 CryEngine . 38 4.2.3 Unity . 39 4.2.4 Unity as nal choice . 41 4.2.4.1 ARToolkit SDK . 42 4.2.4.2 Vuforia SDK . 43 4.2.4.3 Wikitude . 44 1 5 Prototype design and development 46 5.1 HoloLens hardware review . 46 5.2 Hololens inputs . 48 5.3 Hololens emulator . 49 5.4 Development basics . 50 5.5 First phase . 53 5.6 Second phase . 56 5.7 Third phase . 59 6 Conclusions and future work 64 6.1 Conclusions . 64 6.2 Future work . 66 Ringraziamenti 68 2 List of Figures 1.1 Reality-Virtuality Continuum . .1 1.2 Mixed Reality Venn Diagram . .2 1.3 Ikea Augmented Reality App . .3 1.4 Intel proposal for Augmented Virtuality . .5 1.5 Example of immersive Virtual Reality . .6 2.1 BMW F1 HMD . 13 2.2 BMW glasses . 14 2.3 Toyota AR Windshield . 15 2.4 Jaguar Land Rover Virtual Windscreen . 15 2.5 iSCOUT HUD . 16 2.6 Navdy HUD . 17 2.7 DAQRI Smart Helmet . 17 2.8 F-35 HMD . 18 2.9 Ukranian HMD . 18 2.10 BMW Helmet . 19 2.11 RideOn Ski Goggles . 19 2.12 HTC Vive . 20 2.13 Oculus Rift . 20 2.14 Epson Moverio BT-2000 . 21 2.15 Microsoft Hololens . 21 2.16 Meta 2 . 22 2.17 ODG R7 . 22 3.1 AR System . 24 3.2 FOV- resolution relatioship . 26 3.3 High-level Architecture . 32 3.4 Headquarters internal architecture . 32 3.5 Garage internal architecture . 33 3.6 Car internal architecture . 33 3.7 ECU possible conguration . 34 3.8 AR System Architecture . 34 4.1 Game Engines . 37 4.2 Augmented Reality SDKs for Unity . 42 5.1 Devices on the mixed reality spectrum . 46 3 5.2 The display . 47 5.3 The sensor bar . 47 5.4 The motherboard . 47 5.5 The Hololens gesture frame . 49 5.6 The Hololens Emulator . 50 5.7 Eye Tracking on an F1 car . 52 5.8 Test drive with Hololens . 53 5.9 A primitive object . 53 5.10 Car model implementation . 55 5.11 Simple UI . 55 5.12 Sport car 3D model . 56 5.13 Phase 2 interface . 57 5.14 Phase 2 Scene View . 57 5.15 Race track around the lake . 59 5.16 NavMesh baking . 60 5.17 Curve zones and waypoints . 61 5.18 Curve example . 61 5.19 Ghost car example . 62 5.20 Alternative interface . 63 4 Abstract Advancements in computing have continued to enhance user eectiveness and ef- ciency in all aspects of home and work life. Mixed Reality technology is rapidly becoming one of those major technological progressions. Mixed reality is the result of blending the physical world with the digital world. It is the next evolution in human, computer, and environment interaction and unlocks possibilities that before now were restricted to our imaginations. The term mixed reality was originally introduced in a 1994 paper by Paul Mil- gram and Fumio Kishino, "A Taxonomy of Mixed Reality Visual Displays." Their paper introduced the concept of the virtuality continuum and focused on how the categorization of taxonomy applied to displays. Since then, the application of mixed reality goes beyond displays but also includes environmental input, spatial sound, and location. The enterprise market for these technologies is growing rapidly. In particular, the market value for Augmented Reality applications is expected to signicantly surpass that of Virtual Reality. Experts are predicting that VR and AR will be a $150 billion market by 2020, with $120 billion of that dedicated to AR. The reason for this is because Virtual Reality, although ashier, takes the user out of the real world. Augmented Reality instead is more applicable to interaction with the physical world around us. Developers and engineers have been progressively releasing more business productivity and enterprise applications with the goal of enhancing business value with this technologies. This thesis work is indeed an example of this trend. It is a proposal of a headset for sport drivers that exploits the potential of this emerging technology. The main goal was the denition and development of an advanced head-mounted device that could allow the driver to see on the display of his helmet useful information. This thesis is focused mainly on the rst phases of the software/hardware development. After an initial analysis of the main requirements of this particular context and the state of art, a rst possible architecture for the head-mounted device has been proposed. The next step was the choice of the best platform to be used for the development of this kind of application and the best engine that could support the design and coding of it. The comparison made showed that the best option was the Microsoft Hololens System, that is a holographic devices with a see-through display that allows the user to see the physical environment while wearing the headset. This technology should be adapted and incorporated into a driver headset. As far as concerns the computer engine, the comparison I made among the main engines on the market showed that Unity3D was the best option. The last part of the work done has been the development of a prototype that could give an idea to the user of the nal result. II Chapter 1 Mixed Reality The word mixed reality comes from the research paper by Paul Milgram and Fumio Kishino entitled A Taxonomy of Mixed Reality Visual Displays published in 1994. In that paper, the term mixed reality appeared in reference to a part of the virtuality continuum (later also called the Reality-Virtuality (RV) continuum). Figure 1.1: Reality-Virtuality Continuum This spectrum basically shows a range of environments, from the real world with no virtual elements to a totally virtual environment with none of our reality visible. The space in between when real world elements start to mix with virtual ones to some degree or another is mixed reality in their original denition. The most straightforward way to view a Mixed Reality environment, therefore, is one in.