VTI notat 25A-2016 Published 2016 www.vti.se/en/publications Tracking techniques for automotive virtual reality Björn Blissing VTI notat 25A-2016 | Tracking techniques for automotive virtual reality techniques for automotive Tracking VTI notat 25A-2016 Tracking techniques for automotive virtual reality Björn Blissing Diarienr: 2012/0511-25 Omslagsbild: Karin Linhardt Tryck: LiU-tryck, Linköping 2016 Preface This work has been carried out at VTI and has been financed by internal funding. Linköping, september 2016 Björn Blissing VTI notat 25A-2016 Quality review Internal peer review was performed on 7 September 2016 by Camilla Ekström. Björn Blissing has made alterations to the final manuscript of the report. The research director Arne Nåbo examined and approved the report for publication on 25 October 2016. The conclusions and recommendations expressed are the author’s and do not necessarily reflect VTI’s opinion as an authority. Kvalitetsgranskning Intern peer review har genomförts 7 september 2016 av Camilla Ekström. Björn Blissing har genomfört justeringar av slutligt rapportmanus. Forskningschef Arne Nåbo har därefter granskat och godkänt publikationen för publicering 25 oktober 2016. De slutsatser och rekommendationer som uttrycks är författarens egna och speglar inte nödvändigtvis myndigheten VTI:s uppfattning. VTI notat 25A-2016 Contents Summary ............................................................ 9 Sammanfattning ....................................................... 11 1. Introduction........................................................ 13 1.1. Previous surveys................................................... 13 2. Tracking metrics .................................................... 14 2.1. Degrees of freedom................................................. 14 2.2. Input delay....................................................... 15 2.2.1. Example of scene motion.......................................... 15 2.3. Tracking artifacts .................................................. 16 2.3.1. Static errors ................................................... 16 2.3.2. Dynamic errors................................................. 16 3. General principles of tracking........................................... 17 3.1. Dead reckoning ................................................... 17 3.2. Trilateration...................................................... 17 3.3. Triangulation..................................................... 20 4. Tracking Techniques.................................................. 22 4.1. Mechanical trackers................................................. 22 4.2. Acoustical trackers ................................................. 22 4.3. Electromagnetic trackers ............................................. 22 4.4. Inertial trackers.................................................... 23 4.5. Optical trackers ................................................... 23 4.6. Video trackers .................................................... 24 4.7. Hybrid trackers.................................................... 24 4.8. Full body tracking.................................................. 25 5. Tracking for Automotive Virtual Reality Applications.......................... 26 5.1. Tracking the vehicle ................................................ 26 5.2. Tracking inside the vehicle............................................ 27 6. Conclusions........................................................ 28 References............................................................ 30 VTI notat 25A-2016 VTI notat 25A-2016 Summary Tracking techniques for automotive virtual reality – A review by Björn Blissing (VTI) This publication is a review of available technologies for tracking the user in virtual reality systems. Tracking the user location is important for generating views that adapts to the user’s movements. This review begins with the basic terms used in virtual reality in general. Followed by the important characteristics for tracking equipment. This is followed by a chapter on the fundamental algorithms used for position calculations. Then the most common technologies with their advantages and disadvantages are presented. The text conclude with how these technologies are used in automotive virtual reality. VTI notat 25A-2016 9 10 VTI notat 25A-2016 Sammanfattning Spårningstekniker för fordonsbaserad virtuell verklighet – En kunskapsöversikt av Björn Blissing (VTI) Denna publikation är en sammanställning av den teknologi som används i virtuell verklighet för att spåra användaren. Att spåra var användaren befinner sig är viktigt för att alstra vyer som anpassar sig till användarens rörelser. Publikationen inleds med de grundläggande termer som används inom virtuell verklighet i allmänhet. Därefter presenteras viktiga egenskaper för spårningsutrustning. Detta följs av ett kapitel om de grundläggande algoritmer som används för att beräkna positioner. Sedan presenteras de vanligaste teknologierna för spårning med deras för- och nackdelar. Texten avslutas med hur dessa teknologier används inom fordonsbaserad virtuell verklighet. VTI notat 25A-2016 11 12 VTI notat 25A-2016 1. Introduction Virtual reality (VR) is meant to immerse the user in a computer simulation, generating an oriented view in respect to the user (Bishop and Fuchs, 1992). There is also steps between the real world and being in a totally virtual world, which has been described by Milgram et al.(1994) as the Reality-Virtuality Continuum (See figure1). Everything between the two extremes in this continuum is known as Mixed Reality (MR). Augmented Reality (AR) is when the virtual objects or annotations have been added to the view of the real world, while Augmented Virtuality (AV) is when real world objects are brought into an otherwise virtual world. To be able to achieve any of these types of VR experiences some form of tracking is needed. The requirements for the tracking system depends on the selected display technology. The three most common display technology categories are: Fixed Displays — Displays fixed to a static position relative to the user of the system. Starting with a computer monitor with an oriented view, so called fish tank VR (Ware et al., 1993) — up to large room sized six-sided back projected spaces, known as CAVEs (Cruz-Neira et al., 1993). Handheld Displays — Mobile phones or tablets can be used for virtual reality experiences. Most modern devices are already equipped with sensors which can be used for tracking. Especially AR solutions have been common for these types of devices. Head Mounted Displays — Wearable displays which enables the users to be completely immersed in to the virtual world. These usually feature two displays, one for each eye, providing stereoscopic views. Head Mounted Displays (HMD) comes in three main categories; Opaque HMDs for pure virtual reality, Optical see-through HMDs and Video see-through HMDs for mixed reality. The tracking requirements for VR-solution based on fixed screens the requirements are less strict than for handheld or head mounted displays used for MR. This is due to the fact that in MR the user can use the real world as reference, which in turn makes tracking artifacts easier to detect. The user of the virtual world may not be the only thing that is desired to be tracked. Tracking technologies can also be used to position and orient tools used in the virtual worlds; such as wands, styluses and 3D mice. Mixed Reality (MR) Actual Augmented Augmented Virtual Reality Reality (AR) Virtuality (AV) Reality (VR) Figure 1. The Reality-Virtuality Continuum as suggested by Milgram et al.(1994). 1.1. Previous surveys There is a lack of modern surveys regarding tracking technologies. The survey by Ferrin(1991) is focused on helmet tracking technologies mainly for military use. Then there are surveys on tracking technologies for virtual reality, which focuses on the working principles and the different performance of individual systems such as the survey by Bhatnagar(1993) as well as a similar survey by Rolland et al.(2001). There is also a book chapter by Foxlin(2002) as well as a Siggraph course by Allen et al.(2001). VTI notat 25A-2016 13 2. Tracking metrics When comparing tracking systems there are some different metrics that can be important to consider depending on the desired application. Update rate How often measurements are performed and reported. Input delay The time from change of sensor position until a new measurement is reported. This is sometimes called Lag or Latency. (See section 2.2 for a detailed description.) Precision How spread repeated measurements of a stationary target are. Accuracy The difference between the true value and the measured value. Resolution The smallest change in position or orientation that can be measured. Absolute/Relative If the tracker reports measurements in absolute coordinates or as relative changes Working Volume The volume within the tracker can report data Degrees of freedom How many degrees of freedom the tracker is able to measure. (See section 2.1 for a detailed description.) Environmental robustness How robust the tracker is for the environment it is supposed to work in, i.e tolerance for temperature, humidity, noise, lighting conditions etc. Ergonomics The weight and physical dimensions of the sensors. Do the tracker restrict movement in any way, for example due to wires, mechanical limits or gimbal lock situations(See section 4.1). It can also be important to know if the tracker can handle occlusion problems (line of sight)