DOCSLIB.ORG

Standard Operating Procedure for Augmented Reality Operations Using Microsoft Hololens Prepared For: Ohio UAS Center

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Standard Operating Procedure for Augmented Reality Operations Using Microsoft Hololens Prepared For: Ohio UAS Center Standard Operating Procedure for Augmented Reality Operations Using Microsoft HoloLens Prepared for: Ohio UAS Center Prepared by: University of Cincinnati Date: March 24, 2020 Version Number: 2.0 Table of Contents List of Figures ..................................................................................................................................3 1.0 Introduction ................................................................................................................................4 2.0 What is Microsoft HoloLens? ....................................................................................................5 3.0 Input Methods on HoloLens ......................................................................................................6 3.1 Gaze .......................................................................................................................................6 3.2 Gesture ...................................................................................................................................7 3.3 Voice ......................................................................................................................................8 4.0 Prerequisites ...............................................................................................................................8 5.0 How to change the 3D model in the scene? .............................................................................10 6.0 Common Operating Platform ...................................................................................................14 7.0 How does HoloLens work with Common Operating Platform? ..............................................15 8.0 How to deploy your application from Unity to HoloLens? .....................................................19 9.0 How to create Shared Session between two or more HoloLens? ............................................23 References ......................................................................................................................................26 2 | Page List of Figures Figure 1 – A three-step process for 3D model generation .............................................................. 4 Figure 2 – Exploded view of HoloLens[1] ..................................................................................... 5 Figure 3 – Example showing cursor hugging a 3D model .............................................................. 6 Figure 4 – Air tap gesture [2] .......................................................................................................... 7 Figure 5 – Bloom Gesture[3] .......................................................................................................... 7 Figure 6 – Bloom gesture opens the system menu as shown in the figure below. [4] .................... 8 Figure 7 – Adding 3D model to Unity Scene ............................................................................... 10 Figure 8 – Newly added 3D model preview in the Unity scene ................................................... 11 Figure 9 – Model without mesh colliders ..................................................................................... 12 Figure 10 – 3D model with mesh colliders ................................................................................... 12 Figure 11 – Adding the name of the model in the scripts for markers to work and for HoloLens connection on server. .................................................................................................................... 13 Figure 12 – Placeholder to add a model in the script for UI buttons and slider to work. ............. 13 Figure 13 – Common Operating Platform .................................................................................... 14 Figure 14 – HoloLens connectivity with Common Operating Platform....................................... 15 Figure 15 – 3D model request from HoloLens to Common Operating Platform ......................... 16 Figure 16 – 3D model request and response between HoloLens and Common Operating Platform......................................................................................................................................... 16 Figure 17 – Representation of shared session connectivity between HoloLens ........................... 17 Figure 18 – User interface for interacting with the 3D model ...................................................... 18 Figure 19 – User’s avatar in the virtual space of a shared session ............................................... 18 Figure 20 – Setting IP address of Server ...................................................................................... 19 Figure 21 – Steps for HoloLens application deployment ............................................................. 19 Figure 22 – Build application in Unity ......................................................................................... 20 Figure 23 – Application build status in Unity ............................................................................... 21 Figure 24 – Deployment settings in visual studio ......................................................................... 22 Figure 25 – Wi-Fi connectivity check on HoloLens ..................................................................... 23 Figure 26 – Sharing service execution – new session initiation ................................................... 24 Figure 27 – Sharing service – connected users preview ............................................................... 24 3 | Page 1.0 Introduction Since the beginning of content creation, the visualization platform has evolved. Artists created 3D data but was viewed on a 2D paper and then on a 2D screen. In the last two decades, virtual reality has gained huge attention from major IT giants like Microsoft, Facebook, Apple and has become the dominant tool for exploring and visualizing 3D content. After a thorough literature review on existing AR/VR technologies, it was decided to use HoloLens for the project as it is far ahead of the competition and unlike HoloLens, others are not completely wireless, do not run autonomously on an operating system, need continuous HDMI connection, remote controllers for inputs, have poor battery backup. We primarily use HoloLens as a visualization tool for the 3D model. And the way these models are created is a 3-step process. Figure 1 – A three-step process for 3D model generation Step 1: A UAV flies a planned mission over a site and captures overlapping images from multiple angles. These images are uploaded to the UCII common operating platform (web server) Step 2: These images are then stitched together to generate the 3D model using professional photogrammetry software. Step 3: The 3D model is then available on web server. A 3D model is downloaded from webserver and then deployed on HoloLens at compile-time. The users could create the shared user experience, where the user may or may not be in same physical location but will always be in same virtual space and be able to interact with the model. During the shared session, users will be able to talk to each other over voice chat using VoIP technology, look at each other’s avatar while being able to manipulate the 3D model with respect to scale and position, and the synchronize these changes among everyone in the 3D conference with a single button. Users can interact by dropping notes, markers on the models which will be visible to other users at runtime. Users also have provision to mute their mic and toggle on and off for dropping markers. 4 | Page 2.0 What is Microsoft HoloLens? HoloLens is the flagship augmented reality headset from Microsoft. It was launched in early 2016 and has a heads-up display. This headset is way ahead of other competitors in this field. Unlike few of the other headsets which need their sensors mounted in the corners of the room, HoloLens has all the sensors mounted on the device as seen in the Figure 2 below. Figure 2 – Exploded view of HoloLens[1] • Four Microphones are used for giving voice commands to Cortana and for dictation. It can also be used to create custom voice commands in applications for any type of action. • Ambient light sensor helps in auto adjusting the brightness of the display. When the room is brightly lit, the brightness of holograms will be high, so the user can see it clearly and when the room is dim (poorly lit), the brightness of holograms will be low to reduce stress on eyes. • IMU (Inertia Measurement Unit) helps in tracking the user’s head movement. This helps in rendering user’s avatar in shared session which will rotate and reposition with user’s movement in the virtual space. • The environment understanding cameras and the depth camera work together to understand user’s surrounding and generate a 3D map of it, so it can simultaneously locate and map user’s location. • The front camera is a simple 2-megapixel camera capable of capturing mixed reality, so it sees what a user is seeing. We can take pictures, videos and even live stream what the user is looking at. • HoloLens runs on the Windows 10 operating system and does not need HDMI connection a computer system for functioning. This means that the HoloLens can run almost all the Universal Windows Platform (UWP) apps in the Store as 2D apps. UWP is a developer’s 5 | Page platform introduced with Windows 10 that runs applications on all devices like Desktop, Mobile, Xbox, and HoloLens as well as future Windows devices Moreover, HoloLens has
Load more

Recommended publications
  • Walk-Centric User Interfaces for Mixed Reality
    Walk-Centric User Interfaces for Mixed Reality Wallace Santos Lages Dissertation submitted to the faculty of the Virginia Polytechnic Institute and State University in partial fulfillment of the requirements for the degree of Doctor of Philosophy In Computer Science and Applications Doug A. Bowman, Committee Chair Joe Gabbard Tobias Höllerer Chris L. North Nicholas F. Polys June 22, 2018 Blacksburg, Virginia Keywords: Augmented Reality, Virtual Reality, 3D User Interfaces Copyright © 2018 by Wallace Santos Lages Walk-Centric User Interfaces for Mixed Reality Wallace Santos Lages ABSTRACT Walking is a natural part of our lives and is also becoming increasingly common in mixed reality. Wireless headsets and improved tracking systems allow us to easily navigate real and virtual environments by walking. In spite of the benefits, walking brings challenges to the design of new systems. In particular, designers must be aware of cognitive and motor requirements so that walking does not negatively impact the main task. Unfortunately, those demands are not yet fully understood. In this dissertation, we present new scientific evidence, interaction designs, and analysis of the role of walking in different mixed reality applications. We evaluated the difference in performance of users walking vs. manipulating a dataset during visual analysis. This is an important task, since virtual reality is increasingly being used as a way to make sense of progressively complex datasets. Our findings indicate that neither option is absolutely better: the optimal design choice should consider both user’s experience with controllers and user’s inherent spatial ability. Participants with reasonable game experience and low spatial ability performed better using the manipulation technique.
    [Show full text]
  • Latest Features Available from the Windows 10 Updates That Could Be Beneficial for Students & Businesses Based Within the Milwaukee Area
    Latest features available from the windows 10 updates that could be beneficial for students & businesses based within the Milwaukee area By: Jeremy Konetz | November 20, 2018 | Informative Article What are some of the benefits that the latest windows 10 updates provide to students and Milwaukee area-based businesses? Note the last three updates made available through windows 10 updates are windows 10 version 1709 (Released: January 23, 2018), windows 10 version 1803 (Released: July 6, 2018), and windows 10 version 1809 (Released: October 1, 2018). What are some of the beneficial updates provided in the Windows 10 update version 1709 released on January 23, 2018, one of the first updates Windows 10 released this year. What are the areas that this update has improved on? 1) Deployment a. Launching the autopilot application. i. Accomplished through a zero-touch experience. Example shown in figure 1. Figure 1 Resource link: https://docs.microsoft.com/en-us/windows/whats-new/whats-new-windows-10- version-1709 ii. Client or organization profile configuration can be accomplished at the vendor with the devices sent directly to them upon completion. Example shown in figure 2. Figure 2 1 Resource link: https://docs.microsoft.com/en-us/windows/whats-new/whats-new-windows-10- version-1709 b. Activation on subscription to windows 10. i. Feature allows for Windows 10 enterprise to be deployed within an organizational networks structure without applying any keys or rebooting of devices or components within an organizations operational system. See figure 3. Figure 3 1 Resource link: https://docs.microsoft.com/en-us/windows/whats-new/whats-new-windows-10- version-1709 ii.
    [Show full text]
  • Samsung Announces New Windows-Based Virtual-Reality Headset at Microsoft Event 4 October 2017, by Matt Day, the Seattle Times
    Samsung announces new Windows-based virtual-reality headset at Microsoft event 4 October 2017, by Matt Day, The Seattle Times Samsung is joining Microsoft's virtual reality push, Microsoft also said that it had acquired AltspaceVR, announcing an immersive headset that pairs with a California virtual reality software startup that was Windows computers. building social and communications tools until it ran into funding problems earlier this year. The Korean electronics giant unveiled its Samsung HMD Odyssey at a Microsoft event in San ©2017 The Seattle Times Francisco recently. It will sell for $499. Distributed by Tribune Content Agency, LLC. The device joins Windows-based immersive headsets built by Lenovo, HP, Acer and Dell, and aimed for release later this year. Microsoft is among the companies seeking a slice of the emerging market for modern head-mounted devices. High-end headsets, like Facebook-owned Oculus's Rift and the HTC Vive, require powerful Windows PCs to run. Others, including the Samsung Gear VR and Google's Daydream, are aimed at the wider audience of people who use smartphones. Microsoft's vision, for now, is tied to the PC, and specifically new features in the Windows operating system designed to make it easier to build and display immersive environments. The company also has its own hardware, but that hasn't been on display recently. Microsoft's HoloLens was a trailblazer when it was unveiled in 2015. The headset, whose visor shows computer-generated images projected onto objects in the wearer's environment without obscuring the view of the real world completely, was subsequently offered for sale to developers and businesses.
    [Show full text]
  • Holographic Computer
    MECHANICAL ENGINEERING | DECEMBER 2018 | P.33 bioengineering WEARABLE Holographic Computer Microsoft’s HoloLens mixed reality headset is a breakthrough in human-computer interaction. STORY BY AGAM SHAH • ILLUSTRATION BY ZINA SAUNDERS he Kinect camera, which headset, Thyssenkrupp is equipping Microsoft released in 2010, elevator service technicians with brought unprecedented HoloLens to visualize and identify T levels of gesture and voice problems ahead of a job, and have interaction to video games. “When remote, hands-free access to technical we created Kinect for Xbox 360, and expert information when onsite. we aimed to build a device capable The company says that has helped of recognizing and understanding improve service times. Ford is also people so that computers could using HoloLens, enabling their design operate in ways that are more and engineering teams to visualize human,” said Alex Kipman, technical that projects the graphics onto a full-scale models in 3-D. They’ve fellow at Microsoft, who helped lens. As a user walks around a room converted processes that used to invent the camera. or turns his head, the position or take weeks down to days, and more Microsoft wanted to build on orientation of the graphical image is easily and securely share ideas across the success of the Kinect’s sensor altered so that it appears to the user the company and consider more technology by parlaying it into a as occupying a consistent location concepts than before. And NASA has radically new device—a mixed reality in space. provided HoloLens to astronauts on headset that would project computer- HoloLens has emerged as the go- the International Space Station as a generated images seemingly into real to device for engineers to integrate holographic instruction manual and a space.
    [Show full text]
  • Hp Mixed Reality Headset System Requirements
    Hp Mixed Reality Headset System Requirements Pachydermatous Meir ionizes enlargedly. Wandering and iterative Jakob tithes hereof and enamelled his aeons tranquilly and primordially. Gaga and unruffled Claudio shortens her mom Gaea librate and gunfighting slam-bang. Vr is mixed reality headset toward your preference on the It requires a good to your preferences and accessories, and the prices for too many users assume that showed that you are not these devices. Best vr headset toward your mixed reality headsets operate with a better with an experience by far the requirements are also requires are much that it? Its strengths include its high image clarity as well as the resulting the great level of detail. CPU, GPU, and memory are the most critical components. How tart the tech compares? Dive into place the company offers and reality system. Oculus Go and PSVR. The bag on the MR Portal also makes it marry very productivity focused, not gaming focused. Use voice commands to laugh stuff easier in mixed reality. Acer mixed reality system requirements may require separate windows mixed reality. Get fast access to breaking news, the hottest reviews, great deals and helpful tips. The compatible virtual reality headsets that run the Windows Mixed Reality system are manufactured by various Microsoft hardware partners. VR headsets contain combat or one controls for browsing virtual environments. Hp is designed for steam app to manage your reality headset is better job of the entire kit, but it weighs surprisingly, analysis and online stores beginning in. Some AR headsets are available on the market today, with more rumored to be coming in the future.
    [Show full text]
  • New Realities Risks in the Virtual World 2
    Emerging Risk Report 2018 Technology New realities Risks in the virtual world 2 Lloyd’s disclaimer About the author This report has been co-produced by Lloyd's and Amelia Kallman is a leading London futurist, speaker, Amelia Kallman for general information purposes only. and author. As an innovation and technology While care has been taken in gathering the data and communicator, Amelia regularly writes, consults, and preparing the report Lloyd's does not make any speaks on the impact of new technologies on the future representations or warranties as to its accuracy or of business and our lives. She is an expert on the completeness and expressly excludes to the maximum emerging risks of The New Realities (VR-AR-MR), and extent permitted by law all those that might otherwise also specialises in the future of retail. be implied. Coming from a theatrical background, Amelia started Lloyd's accepts no responsibility or liability for any loss her tech career by chance in 2013 at a creative or damage of any nature occasioned to any person as a technology agency where she worked her way up to result of acting or refraining from acting as a result of, or become their Global Head of Innovation. She opened, in reliance on, any statement, fact, figure or expression operated and curated innovation lounges in both of opinion or belief contained in this report. This report London and Dubai, working with start-ups and corporate does not constitute advice of any kind. clients to develop connections and future-proof strategies. Today she continues to discover and bring © Lloyd’s 2018 attention to cutting-edge start-ups, regularly curating All rights reserved events for WIRED UK.
    [Show full text]
  • Windows 10-New Features & Apps
    Windows 10-New Features & Apps By Tom Krauser This article discusses some of the new features and apps that come packaged in Windows 10. It is only a brief summary of these features. For more information you can search the internet or check YouTube for instructional videos on your topic of interest. The following links provide some good basic information on Windows 10 and should be helpful to you. https://support.microsoft.com/en-us/products/windows?os=windows-10 https://support.microsoft.com/en-us/help/4043948/windows-10-whats-new-in-fall-creators-update-1709 The following article from PC World Magazine provides articles on a lot of new features in Windows 10. https://www.pcworld.com/tag/windows10/ The following article by CNET discusses some of new features in the latest update to Windows 10. https://www.cnet.com/how-to/windows-10-tips-best-features/ Alarms & Clocks: A combination of alarm clock, world clock, timer, and stopwatch. Set alarms and reminders, check times around the world, and time your activities, including laps and splits. The following link discusses how to set timers, alarms, and stopwatches: https://www.howtogeek.com/225211/how-to-set-timers-alarms-and-stopwatches-in-windows-10/ Camera: Many modern devices with Windows include a webcam and, to use it, you need an app that helps you take pictures, record videos or stream video while video chatting. For this purpose, Microsoft has built an app called Camera, which is available by default in Windows 10. Connect: Use Connect App to Cast Your Smartphone Screen to Your PC.
    [Show full text]
  • Hp Reverb Pre Order
    Hp Reverb Pre Order Is Norton always unconfederated and gradational when coughs some trials very heliotropically and lustily? Abdul usually redefined communally or mutch lustfully when reflexive Casey interspaces poco and miserably. Unprovident and luetic Alf contemporizes her insolvencies tranquilized while Mikey spoilt some guesswork sturdily. Eye relief is probably competitive in close second time at hp reverb virtual reality and each other airbus and higher resolution, for media features an amazon Would be more! Sadly there policy to be problems with fulfilling pre-orders by HP and even communicating delivery dates Reddit thread FAQ and troubleshooting. But edit the US can already pre-order the device HP Reverb G2 worldwide pre-orders aren't available in We experience when they accept be. HP's Reverb G2 virtual reality headset is the culmination of a. Did not see any submissions nor want better in many great headset and great gaming community and try with valve which is bobby carlton. HP has confirmed those who pre-ordered will grow their headset arrived between early and mid-November if you don't know cause the Reverb. HP Press Release September 24th 2020 Customers who she already pre-ordered the HP Reverb can expect they receive their shipments this fall. Ask a refund. Hp reverb g2 fov mod. Review soon after unboxing their own on topic is enough gamers do. New Shipment Date for HP Reverb G2 For those what you who pre-ordered the HP Reverb G2 HP originally announced that all pre-orders for the G2 will only be. It to be present at every way more natural and series editor of red light can handle my use cookies are and collaboration with.
    [Show full text]
  • Appendix a and Appendix B
    This PDF contains 2 Appendices: Appendix A and Appendix B. Appendix A Answers to the Test Your Knowledge Questions This appendix has the answers to the questions in the Test Your Knowledge section at the end of each chapter. Chapter 1 – Hello, C#! Welcome, .NET! 1. Why can a programmer use different languages, for example, C# and F#, to write applications that run on .NET? Answer: Multiple languages are supported on .NET because each one has a compiler that translates the source code into intermediate language (IL) code. This IL code is then compiled to native CPU instructions at runtime by the CLR. 2. What do you type at the prompt to create a console app? Answer: You enter dotnet new console. 3. What do you type at the prompt to build and execute C# source code? Answer: In a folder with a ProjectName.csproj file, you enter dotnet run. 4. What is the Visual Studio Code keyboard shortcut to view Terminal? Answer: Ctrl + ` (back tick). Answers to the Test Your Knowledge Questions 5. Is Visual Studio 2019 better than Visual Studio Code? Answer: No. Each is optimized for different tasks. Visual Studio 2019 is large, heavy- weight, and can create applications with graphical user interfaces, for example, Windows Forms, WPF, UWP, and Xamarin.Forms mobile apps, but it is only available on Windows. Visual Studio Code is smaller, lighter-weight, code-focused, supports many more languages, and is available cross-platform. In 2021, with the release of .NET 6 and .NET Multi-platform App User Interface (MAUI), Visual Studio Code will get an extension that enables building user interfaces for desktop and mobile apps.
    [Show full text]
  • HP VR 1000 - Setting up the HP Windows Mixed Reality Headset (Windows 10)
    HP VR 1000 - Setting Up the HP Windows Mixed Reality Headset (Windows 10) This document is for head mounted display (HMD) devices and PCs with Windows 10. With the HP Windows Mixed Reality VR 1000 head mounted display (HMD), you can experience a combination of augmented reality and virtual reality. You see and experience virtual objects mixed with actual, real surroundings as seen by the front-facing cameras. Or you see the virtual surroundings mixed with real objects. Use this document to set up and begin using the HMD. To troubleshoot issues with Windows Mixed Reality setup, go to Microsoft webpage Troubleshooting Windows Mixed Reality (in English). Step 1: Prepare to set up Windows Mixed Reality Before connecting the HMD and installing Windows Mixed Reality, gather the required supplies and make sure your PC is compatible and up to date. 1. Sign in to an administrator user account, and then make sure your PC is connected to the Internet. 2. Click Start , click Settings , click System, and then click About to make sure your PC has Windows 10 Fall Creators Update (Windows 10, version 1709) or later for Windows Mixed Reality support. 3. Check Windows Update and HP Support Assistant and install any available driver updates. 4. Make sure your PC is compatible with Windows Mixed Reality. Go to Microsoft webpage Mixed Reality minimum PC hardware guidelines (in English) for a list of requirements, or download and run the Windows Mixed Reality PC Check app from the Windows Store. 5. Gather the supplies you need. o Adapters: You might need adapters for Bluetooth or USB 3.0 and full-sized HDMI ports.
    [Show full text]
  • Security and Privacy Approaches in Mixed Reality:A Literature Survey
    0 Security and Privacy Approaches in Mixed Reality: A Literature Survey JAYBIE A. DE GUZMAN, University of New South Wales and Data 61, CSIRO KANCHANA THILAKARATHNA, University of Sydney and Data 61, CSIRO ARUNA SENEVIRATNE, University of New South Wales and Data 61, CSIRO Mixed reality (MR) technology development is now gaining momentum due to advances in computer vision, sensor fusion, and realistic display technologies. With most of the research and development focused on delivering the promise of MR, there is only barely a few working on the privacy and security implications of this technology. is survey paper aims to put in to light these risks, and to look into the latest security and privacy work on MR. Specically, we list and review the dierent protection approaches that have been proposed to ensure user and data security and privacy in MR. We extend the scope to include work on related technologies such as augmented reality (AR), virtual reality (VR), and human-computer interaction (HCI) as crucial components, if not the origins, of MR, as well as numerous related work from the larger area of mobile devices, wearables, and Internet-of-ings (IoT). We highlight the lack of investigation, implementation, and evaluation of data protection approaches in MR. Further challenges and directions on MR security and privacy are also discussed. CCS Concepts: •Human-centered computing ! Mixed / augmented reality; •Security and privacy ! Privacy protections; Usability in security and privacy; •General and reference ! Surveys and overviews; Additional Key Words and Phrases: Mixed Reality, Augmented Reality, Privacy, Security 1 INTRODUCTION Mixed reality (MR) was used to pertain to the various devices – specically, displays – that encompass the reality-virtuality continuum as seen in Figure1(Milgram et al .
    [Show full text]
  • Business Finland Neogames Fivr Mixed Reality Report 2017
    MIXED REALITY REPORT 2017 BUSINESS FINLAND NEOGAMES FIVR 2 3 BUSINESS FINLAND | NEOGAMES BUSINESS FINLAND | NEOGAMES MIXED REALITY 2017 MIXED REALITY 2017 Content 1. Introduction 1. Introduction 3 lready from the 1990’s there has been a strong will and hope towards a virtual- and augmented reality based gaming experience. For a couple of 2. Terminology of VR, AR, MR and XR 3 decades, the development of technology was quite slow, but after HTC 3. Current Status of the VR/AR field 5 AVive, and the first Oculus consumer version release in March 2016 it seemed that 3.1 Available VR & AR devices and platforms for consumers 6 the technology is finally advanced enough, and the market for B2C VR applications, 3.1.1 High-end tethered VR headsets 6 including games, is ready to open. 3.1.2 Smartphone-based mobile VR headsets 7 The Oculus and Vive releases together with available VC funding and the 3.2 Technological demands in general 7 saturation of the mobile market (resulting in some mobile developers fleeing to 3.3 User expectations 8 VR/AR development) created high hopes towards VR. Basically everything required 4. Future - Towards casual VR 8 was coming together, funding, technology, skills and companies. However, after a 4.1 Four tiers of future VR devices 9 good start and excessive hype the VR games’ B2C market didn’t develop as expected. 4.2 High-end consumer VR devices 10 One clear indicator of that was that some existing VR studios have closed and even 5. AR Devices 10 Icelandic CCP, a big advocate of VR games since 2013, announced in the end of 6.
    [Show full text]