Web AR Solution for UAV Pilot Training and Usability Testing

Total Page:16

File Type:pdf, Size:1020Kb

Web AR Solution for UAV Pilot Training and Usability Testing sensors Article Web AR Solution for UAV Pilot Training and Usability Testing Roberto Ribeiro 1 , João Ramos 1 , David Safadinho 1 , Arsénio Reis 2 , Carlos Rabadão 1 , João Barroso 2 and António Pereira 1,3,* 1 Computer Science and Communication Research Centre, School of Technology and Management, Polytechnic Institute of Leiria, Campus 2, Morro do Lena-Alto do Vieiro, Apartado 4163, 2411-901 Leiria, Portugal; [email protected] (R.R.); [email protected] (J.R.); [email protected] (D.S.); [email protected] (C.R.) 2 INESC TEC, University of Trás-os-Montes e Alto Douro, Quinta de Prados, 5001-801 Vila Real, Portugal; [email protected] (A.R.); [email protected] (J.B.) 3 INOV INESC Inovação, Institute of New Technologies, Leiria Office, Campus 2, Morro do Lena-Alto do Vieiro, Apartado 4163, 2411-901 Leiria, Portugal * Correspondence: [email protected] Abstract: Data and services are available anywhere at any time thanks to the Internet and mobile devices. Nowadays, there are new ways of representing data through trendy technologies such as augmented reality (AR), which extends our perception of reality through the addition of a virtual layer on top of real-time images. The great potential of unmanned aerial vehicles (UAVs) for carrying out routine and professional tasks has encouraged their use in the creation of several services, such as package delivery or industrial maintenance. Unfortunately, drone piloting is difficult to learn and requires specific training. Since regular training is performed with virtual simulations, we decided to propose a multiplatform cloud-hosted solution based in Web AR for drone training and usability testing. This solution defines a configurable trajectory through virtual elements represented over barcode markers placed on a real environment. The main goal is to provide an inclusive and Citation: Ribeiro, R.; Ramos, J.; accessible training solution which could be used by anyone who wants to learn how to pilot or Safadinho, D.; Reis, A.; Rabadão, C.; Barroso, J.; Pereira, A. Web AR test research related to UAV control. For this paper, we reviewed drones, AR, and human–drone Solution for UAV Pilot Training and interaction (HDI) to propose an architecture and implement a prototype, which was built using a Usability Testing. Sensors 2021, 21, Raspberry Pi 3, a camera, and barcode markers. The validation was conducted using several test 1456. https://doi.org/10.3390/ scenarios. The results show that a real-time AR experience for drone pilot training and usability s21041456 testing is achievable through web technologies. Some of the advantages of this approach, compared to traditional methods, are its high availability by using the web and other ubiquitous devices; the Academic Editor: Enrico Natalizio minimization of technophobia related to crashes; and the development of cost-effective alternatives to train pilots and make the testing phase easier for drone researchers and developers through Received: 10 January 2021 trendy technologies. Accepted: 11 February 2021 Published: 19 February 2021 Keywords: augmented reality; AR obstacle courses; human–drone interaction; marker-based AR; unmanned aerial vehicles; UAV control interfaces; UAV pilot training; Web AR Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affil- iations. 1. Introduction The great advantage of unmanned aerial vehicle (UAV) usage includes the exploration of locations that are difficult to access and data gathering from an aerial perspective, which makes it useful in different areas and scenarios. The countless benefits and potential Copyright: © 2021 by the authors. Licensee MDPI, Basel, Switzerland. of drones are strongly related to their capacity to perform tasks that are difficult, dull, This article is an open access article and dangerous [1], which is being intensively explored either for professional or routine distributed under the terms and activities [2]. Nonetheless, as stated in Cisco’s 2020 Global Networking Trends Report, conditions of the Creative Commons immersive experiences powered with virtual reality (VR) and augmented reality (AR) are Attribution (CC BY) license (https:// trending and emergent technologies [3]. Whilst VR allows the user to interact with an creativecommons.org/licenses/by/ immersive and entirely virtual application, AR can present a layer of virtual content over a 4.0/). layer of real-world images captured in real time. The latter can help users improve their Sensors 2021, 21, 1456. https://doi.org/10.3390/s21041456 https://www.mdpi.com/journal/sensors Sensors 2021, 21, 1456 2 of 32 Sensors 2021, 21, 1456 2 of 30 a layer of real-world images captured in real time. The latter can help users improve their performanceperformance inin many many tasks tasks by by providing providing pertinent pertinent information information about about the picturedthe pictured scenario. sce- Mixednario. Mixed reality reality (MR) extends(MR) extends the concept the concept of AR of with AR featureswith features to detect to detect changes changes in the in real the world,real world, allowing allowing the virtual the virtual elements elements to react to react and and adapt, adapt, e.g., e.g., by simulating by simulating the collisionthe colli- ofsion a virtualof a virtual ball ball with with a real a real wall wall [4]. [4]. VR, VR, AR, AR, and and MR MR are are represented represented by by Milgram Milgram andand KishinoKishino through a virtuality continuum continuum diagram diagram [5], [5], presented presented in in Figure Figure 1.1 .Whilst Whilst VR, VR, at atthe the rightmost rightmost end, end, is represented is represented by bya fully a fully virtual virtual environment, environment, AR AR remains remains closer closer to the to theexperiences experiences comprising comprising a real a real environment. environment. Additionally, Additionally, MR MR brings brings the thetwo two ends ends of the of thecontinuum continuum together, together, i.e., i.e.,reality reality and andvirtuality. virtuality. The Thethree three concepts concepts are grouped are grouped in the in gen- the generaleral concept concept of extended of extended reality reality (XR), (XR), which which brings brings them them all alltogether together [6]. [ 6]. FigureFigure 1.1. AdaptationAdaptation ofof thethe MilgramMilgram andand Kishino’sKishino’s virtualityvirtuality continuumcontinuum [[5].5]. ThisThis studystudy isis motivatedmotivated byby thethe needneed toto simplifysimplify andand improveimprove dronedrone pilotpilot trainingtraining andand usabilityusability testing,testing, thusthus decreasingdecreasing technophobiatechnophobia andand thethe exclusionexclusion thatthat stillstill existexist regardingregarding trendytrendy technologiestechnologies like like UAVs UAVs and and AR. AR. The The great grea majorityt majority of training of training alternatives alternatives for drone for pilotsdrone are pilots performed are performed in entirely in entirely virtual simulators,virtual simulators, which detaches which detaches people frompeople the from feeling the offeeling piloting of piloting a real drone a real withdrone a with real controllera real controller in a real in a environment. real environment. This This motivates motivates the combinationthe combination of UAVs of UAVs and ARand to AR create to create training training experiences experiences that enhance that enhance the real the world real withworld virtual with virtual elements. elements. WeWe identifiedidentified andand addressedaddressed thethefollowing followingissues: issues: • • TheThe controlcontrol of of UAVs UAVs and and their their flight flight dynamics dynamics tend tend to beto difficultbe difficult for newfor new users users to un- to derstand,understand, especially especially for thosefor those who who are not are used not used to traditional to traditional remote remote controllers controllers (RCs). • Traditional AR experiences based on surface detection are limited to enthusiasts, since (RCs). high-end hardware, such as HoloLens or Magic Leap 1, is required to run them. In the • Traditional AR experiences based on surface detection are limited to enthusiasts, case of mobile devices, the oftware Development Kits (SDKs) for AR development are since high-end hardware, such as HoloLens or Magic Leap 1, is required to run them. compatible with a specific list of devices which tend to be expensive, consequently In the case of mobile devices, the oftware Development Kits (SDKs) for AR develop- reducing the number of users that can benefit from the technology in their daily lives. ment are compatible with a specific list of devices which tend to be expensive, con- • For UAV usability testing, developers usually set up simple scenarios and trajectories sequently reducing the number of users that can benefit from the technology in their made of real obstacles. Every time a new trajectory is needed, the real obstacles (e.g., daily lives. cones, poles, loops) must be moved. Though higher obstacles bring added value • For UAV usability testing, developers usually set up simple scenarios and trajectories to tests of aerial maneuvers, it can be difficult and dangerous to change an obstacle made of real obstacles. Every time a new trajectory is needed, the real obstacles (e.g., every time a new trajectory is required, for instance, when it is placed on the top of a cones, poles, loops) must be moved. Though higher obstacles bring added value to high wall. • Newtests usersof aerial are maneuvers,
Recommended publications
  • View the Manual
    Base game Manual V0.4 Environment Remove any objects in the playing area that you might touch or hit while playing with your body. Please also make sure that lamps or fans are not in the playing area. If you are at the edge of the playing area, do not make any big movements, otherwise you could touch the walls with your body or hands/controllers. It is best to stand. Step movements are not necessary during the game. You can move and rotate completely with the controllers. Technical requirements To play the game, you need Virtual Reality headset (VR headset). Without VR glasses, the game will not run. A (free) Steam account is also required. Also installed must be the VR software "SteamVR" (when using HTC VIVE or the Valve Index) or the additional software "Windows Mixed Reality for SteamVR" for the use of Windows Mixed Reality glasses on Steam with "SteamVR". Before the first launch, a room measuring should also have been done in SteamVR (when using the HTC VIVE or the Valve Index). When using Windows Mixed Reality glasses, there is an option to change the position of the floor from the height. The controllers of the VR headset should also be connected to the VR system and be charged. The game Container You start the game in the container and go to the selected construction site through the door. Later, you can start different locations here depending on the existing DLC. You can now set options to one of the boards by moving the switch accordingly.
    [Show full text]
  • VR Headset Comparison
    VR Headset Comparison All data correct as of 1st May 2019 Enterprise Resolution per Tethered or Rendering Special Name Cost ($)* Available DOF Refresh Rate FOV Position Tracking Support Eye Wireless Resource Features Announced Works with Google Subject to Mobile phone 5.00 Yes 3 60 90 None Wireless any mobile No Cardboard mobile device required phone HP Reverb 599.00 Yes 6 2160x2160 90 114 Inside-out camera Tethered PC WMR support Yes Tethered Additional (*wireless HTC VIVE 499.00 Yes 6 1080x1200 90 110 Lighthouse V1 PC tracker No adapter support available) HTC VIVE PC or mobile ? No 6 ? ? ? Inside-out camera Wireless - No Cosmos phone HTC VIVE Mobile phone 799.00 Yes 6 1440x1600 75 110 Inside-out camera Wireless - Yes Focus Plus chipset Tethered Additional HTC VIVE (*wireless tracker 1,099.00 Yes 6 1440x1600 90 110 Lighthouse V1 and V2 PC Yes Pro adapter support, dual available) cameras Tethered All features HTC VIVE (*wireless of VIVE Pro ? No 6 1440x1600 90 110 Lighthouse V1 and V2 PC Yes Pro Eye adapter plus eye available) tracking Lenovo Mirage Mobile phone 399.00 Yes 3 1280x1440 75 110 Inside-out camera Wireless - No Solo chipset Mobile phone Oculus Go 199.00 Yes 3 1280x1440 72 110 None Wireless - Yes chipset Mobile phone Oculus Quest 399.00 No 6 1440x1600 72 110 Inside-out camera Wireless - Yes chipset Oculus Rift 399.00 Yes 6 1080x1200 90 110 Outside-in cameras Tethered PC - Yes Oculus Rift S 399.00 No 6 1280x1440 90 110 Inside-out cameras Tethered PC - No Pimax 4K 699.00 Yes 6 1920x2160 60 110 Lighthouse Tethered PC - No Upscaled
    [Show full text]
  • Hp Mixed Reality Headset System Requirements
    Hp Mixed Reality Headset System Requirements Pachydermatous Meir ionizes enlargedly. Wandering and iterative Jakob tithes hereof and enamelled his aeons tranquilly and primordially. Gaga and unruffled Claudio shortens her mom Gaea librate and gunfighting slam-bang. Vr is mixed reality headset toward your preference on the It requires a good to your preferences and accessories, and the prices for too many users assume that showed that you are not these devices. Best vr headset toward your mixed reality headsets operate with a better with an experience by far the requirements are also requires are much that it? Its strengths include its high image clarity as well as the resulting the great level of detail. CPU, GPU, and memory are the most critical components. How tart the tech compares? Dive into place the company offers and reality system. Oculus Go and PSVR. The bag on the MR Portal also makes it marry very productivity focused, not gaming focused. Use voice commands to laugh stuff easier in mixed reality. Acer mixed reality system requirements may require separate windows mixed reality. Get fast access to breaking news, the hottest reviews, great deals and helpful tips. The compatible virtual reality headsets that run the Windows Mixed Reality system are manufactured by various Microsoft hardware partners. VR headsets contain combat or one controls for browsing virtual environments. Hp is designed for steam app to manage your reality headset is better job of the entire kit, but it weighs surprisingly, analysis and online stores beginning in. Some AR headsets are available on the market today, with more rumored to be coming in the future.
    [Show full text]
  • PROGRAMS for LIBRARIES Alastore.Ala.Org
    32 VIRTUAL, AUGMENTED, & MIXED REALITY PROGRAMS FOR LIBRARIES edited by ELLYSSA KROSKI CHICAGO | 2021 alastore.ala.org ELLYSSA KROSKI is the director of Information Technology and Marketing at the New York Law Institute as well as an award-winning editor and author of sixty books including Law Librarianship in the Age of AI for which she won AALL’s 2020 Joseph L. Andrews Legal Literature Award. She is a librarian, an adjunct faculty member at Drexel University and San Jose State University, and an international conference speaker. She received the 2017 Library Hi Tech Award from the ALA/LITA for her long-term contributions in the area of Library and Information Science technology and its application. She can be found at www.amazon.com/author/ellyssa. © 2021 by the American Library Association Extensive effort has gone into ensuring the reliability of the information in this book; however, the publisher makes no warranty, express or implied, with respect to the material contained herein. ISBNs 978-0-8389-4948-1 (paper) Library of Congress Cataloging-in-Publication Data Names: Kroski, Ellyssa, editor. Title: 32 virtual, augmented, and mixed reality programs for libraries / edited by Ellyssa Kroski. Other titles: Thirty-two virtual, augmented, and mixed reality programs for libraries Description: Chicago : ALA Editions, 2021. | Includes bibliographical references and index. | Summary: “Ranging from gaming activities utilizing VR headsets to augmented reality tours, exhibits, immersive experiences, and STEM educational programs, the program ideas in this guide include events for every size and type of academic, public, and school library” —Provided by publisher. Identifiers: LCCN 2021004662 | ISBN 9780838949481 (paperback) Subjects: LCSH: Virtual reality—Library applications—United States.
    [Show full text]
  • LPVR-CAD Manual
    LPVR-CAD Manual Version 2.5 1 CONTENTS Version History ........................................................................................................................................................ 3 Overview ................................................................................................................................................................. 4 System Versions .................................................................................................................................................. 5 LPVR Middleware DETAILS .................................................................................................................................. 5 Driver Implementation ............................................................................................................................................ 6 SteamVR Driver Wrapper .................................................................................................................................... 6 Tracking And Sensor Fusion ................................................................................................................................ 7 System Setup ........................................................................................................................................................... 8 Hardware Installation ......................................................................................................................................... 8 Driver Setup .......................................................................................................................................................
    [Show full text]
  • Hp Reverb Pre Order
    Hp Reverb Pre Order Is Norton always unconfederated and gradational when coughs some trials very heliotropically and lustily? Abdul usually redefined communally or mutch lustfully when reflexive Casey interspaces poco and miserably. Unprovident and luetic Alf contemporizes her insolvencies tranquilized while Mikey spoilt some guesswork sturdily. Eye relief is probably competitive in close second time at hp reverb virtual reality and each other airbus and higher resolution, for media features an amazon Would be more! Sadly there policy to be problems with fulfilling pre-orders by HP and even communicating delivery dates Reddit thread FAQ and troubleshooting. But edit the US can already pre-order the device HP Reverb G2 worldwide pre-orders aren't available in We experience when they accept be. HP's Reverb G2 virtual reality headset is the culmination of a. Did not see any submissions nor want better in many great headset and great gaming community and try with valve which is bobby carlton. HP has confirmed those who pre-ordered will grow their headset arrived between early and mid-November if you don't know cause the Reverb. HP Press Release September 24th 2020 Customers who she already pre-ordered the HP Reverb can expect they receive their shipments this fall. Ask a refund. Hp reverb g2 fov mod. Review soon after unboxing their own on topic is enough gamers do. New Shipment Date for HP Reverb G2 For those what you who pre-ordered the HP Reverb G2 HP originally announced that all pre-orders for the G2 will only be. It to be present at every way more natural and series editor of red light can handle my use cookies are and collaboration with.
    [Show full text]
  • Flightsim Community Survey 2019
    FlightSim Community Survey 2019 Final Report 1 Copyright Notice © 2020 Navigraph By licensing our work with the CC BY-SA 4.0 license it means that you are more than welcome to copy, remix, transform and build upon the results of this survey and then redistribute it to whomever you want in any way. You only have to give credit back to Navigraph and keep the same license. https://creativecommons.org/licenses/by-sa/4.0/ 2 Preamble This is the annual flightsim community survey, a collaborative effort between partners – developers, companies and organizations in the flightsim domain – coordinated and compiled by Navigraph. This survey is freely distributed for the common good of the flightsim community to guide future projects and attract new pilots. This flightsim community survey is the largest and most comprehensive of its kind. This year 17,800 respondents participated in the survey. This is an 18.6% increase from last year when 15,000 participated. This year’s survey consisted of 93 questions, compared to last year’s 77 questions. However, this year many more of the questions were conditional, allowing us to add new sections and ask in-depth questions only to those respondents for whom it was relevant. New sections this year contained questions specifically aimed at pilots using flight simulator software on mobile devices and helicopter flight simulators. We also added questions on combat simulators, air traffic control and flight planning. Finally, because of the upcoming release of the new Microsoft Flight Simulator 2020, we added questions on this topic as well. Our main objective this year was to recruit more and diverse partners to broaden the reach of the survey to get an even more representative sample of the community.
    [Show full text]
  • Équipement Léger De Simulation À Domicile
    Équipement léger de simulation à domicile Le confinement vous a incité à trouver des solutions pour voler par-delà les interdictions administratives ou une météo capricieuse ? Alors, un ordinateur, un manche, CONDOR 2 et vous voilà prêt (à minima) à décoller *! *Les matériels que vous trouverez ci-dessous ont été testées par les membres du groupe « lab Planeur FFVP ». Ces préconisations ne sont pas exhaustives mais représentent le meilleur rapport qualité/prix du moment sur le matériel testé. Les liens vers les commerces en ligne sont proposés à titre indicatif et la FFVP n’a contracté aucun partenariat avec le distributeur ni le fabricant. Les matériels sont susceptibles d’être trouvés dans tout commerce dédié à un prix inférieur. Les prix peuvent variés d’un jour à l’autre suivant les promotions. Matériel requis : 1) Ordinateur : • Avec ou sans Track IR : processeur I3 minimum avec 4 Go de mémoire, carte graphique GTX 1050 TI • Avec un casque de réalité virtuelle : processeur I7 avec 8Go de mémoire carte graphique GTX 1080 2) Condor 2 et accès réseau internet En plus d’acquérir Condor 2 et de disposer d’un réseau internet de qualité, il vous faudra un disque dur de 500 Go minimum (recommandé 1 ou 2 To) pour stocker les scènes Condor... 3) Le matériel de vol Un joystick avec au minimum 1 manette de gaz et 8 boutons. Si vous voulez allez plus loin, nous conseillons l’acquisition d’un palonnier (ou la fabrication maison avec les nombreux tutos que vous trouverez via internet). a) manche à moins de 75€ Manche thrusmaster T 16000 FCS PC b) palonnier à moins de 150 € Thrustmaster TFRP Rudder c) les combinés manche/palonnier (150 à 250€) ▪ T.16000M FCS FLIGHT PACK (palonnier + manche avec trim 16 boutons + manette des gaz pour volet ou aérofreins) ▪ Thrusmaster T flight à 150 € environ Pour aller plus loin pour votre confort de pilotage Vous pouvez acquérir un Track Ir ou un masque de réalité virtuelle.
    [Show full text]
  • E-Book Your Guide to Buying a Successful VR Attraction for Your Entertainment Venue
    e-book Your Guide to Buying a Successful VR Attraction for Your Entertainment Venue expert opinions of tech innovators EBOOK : EDUCATIONAL 2021 introduction There is a lot of talk regarding Location-Based Virtual Reality whether it’s a solid attraction or a passing trend. People wonder if it’s profitable, easy to operate, and so on. In this e-book, you will learn what to look for in a VR System for guaranteed success! Check out the most important things to consider if you want to add a VR Attraction to your entertainment venue! This subject is tackled based on 3 perspectives, as they are all linked together, and they contribute to a successful or a failed investment. The Technology Part - the hardware and everything it entails, the Content Part – namely VR games, and the Management Part - the dos and don’ts of opera- ting a VR System at your location - these are all important elements that will contribute to the success of your investment! WE ARE #innovators INOVATE the footprint 19-20 CONTROLLERS AND 21 GADGETS in vr content is vr games 23 king VR CONTENT CONTENT e-book introduction 02-03 inowize.com Chapter 1 05-21 27 making VR A PROFITABLE INVESTMENT vr technology-Choosing the Right VR Attraction for Your Center chapter 2 22-25 Virtual Reality Games 32-33 Quality Content is Vital let’s talk chapter 3 26-29 about vr VR Management Making VR a Profitable Investment vr technology Types of VR Technologies There are a lot of different types of VR attractions out there that are used in entertainment centers.
    [Show full text]
  • Viral SKILLS 2018-1-AT02-KA204-039300 COMPENDIUM
    Fostering Virtual Reality applications within Adult Learning to improve low skills and qualifications Project No. 2018-1-AT02-KA204-039300 The European Commission support for the production of this publication does not constitute an endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein. ViRAL SKILLS 2018-1-AT02-KA204-039300 COMPENDIUM Project Information Project Acronym: ViRAL Skills Project Title: Fostering Virtual Reality applications within Adult Learning to improve low skills and qualifications Project No.: 2018-1-AT02-KA204-039300 Funding Programme: Erasmus+ Key Action 2: Strategic Partnerships More Information: www.viralskills.eu www.facebook.com/viralskillsEU [email protected] With the support of the Erasmus+ Programme of the European Union. Disclaimer: The European Commission support for the production of this publication does not constitute an endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein. ViRAL SKILLS 2018-1-AT02-KA204-039300 COMPENDIUM Table of contents Introduction ......................................................................................................................... 1 1 Technical Introduction to VR ....................................................................................... 3 1.1 Virtual Reality in the educational sector ............................................................
    [Show full text]
  • Windows Mixed Reality Recommended Specs
    Windows Mixed Reality Recommended Specs Sometimes vulned Fitz improvised her grabbler swaggeringly, but Pecksniffian Bryon shunned wealthily or titivate perspectively. Dreich and prokaryotic Bing agnize generically and crave his tyne listlessly and presently. Asunder and raiding Saundra always aliment impersonally and parboil his penises. In small white leds, and registered trademarks of the performance despite the oculus rift, it back up to leave vr home windows mixed reality has since before putting the Mark at the windows creators, but none of recommended for users to earn compensation on our data set up of games and more intuitive and. EVERYDAY GOLF VR is a realistic golf VR game. The world within that dtna asked themselves. Vr players inside the image has a timeline as their next iteration of recommended specs. Windows mixed reality, or other programs you tried restarting your back down with tracked controllers with integrated into many of recommended specs required, but some features or clicked back up. Windows Mixed Reality headsets. Find answers to common Windows Mixed Reality questions in our FAQs. Pc vendors and unofficial vr requires a windows mixed reality recommended specs, footer and adobe premiere pro is better screen or registered trademarks and. Apple would be changed server and windows mixed reality recommended specs actually interact with. How to windows mixed reality is possible by releasing a window to release of recommended specs that work properly. Steam VR theater and working. Adjust on windows mixed reality recommended specs and those who consider the specs and. Am did that makes me and windows mixed reality recommended specs.
    [Show full text]
  • Platform Overview. December 2020 Contents
    Platform Overview. December 2020 Contents. Contents. .................................................................................................................................... 2 Introduction. ............................................................................................................................... 3 Glue overview. ............................................................................................................................ 4 Glue Teams .............................................................................................................................................................. 4 Authentication .................................................................................................................................................... 4 Glue Team Spaces .................................................................................................................................................. 5 Spatial audio ....................................................................................................................................................... 5 3D avatars ............................................................................................................................................................ 5 Objects ................................................................................................................................................................. 5 Persistence .........................................................................................................................................................
    [Show full text]