Microsoft Hololens Microsoft Mixed Reality Strategy & Vision The opportunity to enhance human cognition cognition [kɒɡˈnɪʃ(ə)n] the mental action or process of acquiring knowledge and understanding through thought, experience, and the senses. synonyms: perception · discernment · awareness · apprehension · learning · understanding · comprehension · enlightenment · insight · intelligence · reason · reasoning · thinking · (conscious) thought The opportunity to enhance human cognition
Humans forget 80% of what they learn after 30 days
MIXED REALITY – BLENDING THE PHYSICAL AND DIGITAL WORLD
A blending of the physical and digital worlds, in which users may interact with digital and real-world objects, whilst maintaining presence in the physical world
PHYSICAL WORLD DIGITAL WORLD
AUGMENTED REALITY VIRTUAL REALITY
MIXED REALITY SPECTRUM THIRD WAVE OF COMPUTING
Personal Computers Smartphones Mixed Reality
ComputingComputing powerpower forfor manymany Computing power for all with Computing power at the on-the-go access and some butbut immobileimmobile andand nono on-the-go access and some edge that blends of the contextual awareness physical and digital worlds contextualcontextual awarenessawareness Lockheed Martin creates magic with HoloLens 2 - YouTube Proven Enterprise Use Cases As we have worked with thousands of enterprise customers the past several years, we have seen a handful of use cases that can be applied across many industries which have a fast time to value (TTV) and a proven return on investment (ROI):
Training & Guides & Task Remote Design & Sales Contextual Simulation Management Assistance Prototyping Assistance Data Overlays
Train and onboard new Enable your employees employees faster than Walk employees step- to remote in an expert Carry less inventory and Make pertinent and ever using a “learning by-step through guided anytime they need it. Convert your CAD and close large ticket sales real-time data available by doing” approach. experiences to help BIM files to 3D digital faster by using mixed when and where it is them complete tasks Experts can see and twins to quickly iterate reality to showcase any needed, enabling your Simulate scenarios so faster and more interact with the same and collaborate on new configuration or employees to make that your employees accurately then ever environment as the product designs. customization to your better, faster, more can practice how to before. employee as if they customer. informed decisions. respond in certain were there. situations White Light Gray Gray Mid Gray Dark Gray Rich Black R255 G255 B255 R230 G230 B230 R210 G210 B210 R115 G115 B115 R80 G80 B80 R0 G0 B0
Light Yellow Light Orange Light Magenta Light Purple Light Blue Light Teal Light Green R255 G241 B0 R255 G140 B0 R227 G0 B140 R180 G160 B255 R0 G188 B242 R0 G178 B148 R186 G216 B10 Yellow Orange Red Magenta Purple Blue Teal Green R255 G185 B0 R216 G59 B1 R232 G17 B35 R180 G0 B158 R92 G45 B145 R0 G120 B215 R0 G130 B114 R16 G124 B16 Mid Blue R0 G24 B143
Dark Red Dark Magenta Dark Purple Dark Blue Dark Teal Dark Green R168 G0 B0 R92 G0 B92 R50 G20 B90 R0 G32 B80 R0 G75 B80 R0 G75 B28 & governance & tools tools processes & Outdated Outdated obsolete & Uneven security practices security Uneven Firstline Workers New training, skills & skills & training, New team & company team & Disconnected from Disconnectedfrom professional development professional 2B Workers 480M Information rce rce o generational & - Slow to receive Slow to receive diverse workf diverse Multi information & resources resources information & THE MIXED REALITY JOURNEY
2010 2016 2017
Kinect for Xbox HoloLens HoloLens Development Edition Commercial Suite Introducing HoloLens
What is HoloLens?
• HoloLens is the first self-contained holographic computer, enabling you to interact with high-definition holograms in your world.
• A fully untethered experience, HoloLens needs no external wires, markers, or cameras, nor a connection to a phone or PC, so you can move freely.
• HoloLens runs Windows 10, the premier platform to connect, work, and play in mixed reality.
• Microsoft HoloLens has advanced sensors to capture information about what you're doing and the environment you’re in.
• See-through holographic lenses use an advanced optical projection system, so you can see holograms in your world. THE MIXED REALITY JOURNEY
2010 2016 2017 2018 2019
Kinect for Xbox HoloLens HoloLens Dynamics 365 Mixed Reality Development Edition Commercial Suite Applications Portfolio Microsoft Mixed Reality Platform The most comprehensive Mixed Reality Stack
Most comfortable and Largest ecosystem The reliability, security, and immersive mixed reality of 1st & 3rd Party scalability of cloud and AI experience available mixed reality solutions services from Microsoft The most comprehensive Mixed Reality Stack
Intelligent Devices Support for virtual reality head mounted Market leading mixed reality Support for Mixed Reality capable displays with market leading services mounted display for commercial customers mobile phones with market leading services
Mixed reality
applications Remote Assist and Guides An ecosystem of ISV partners with Enterprise ready apps for all frontline workers enterprise ready commercial solutions across industries
Azure mixed reality services Azure Spatial Anchors Azure Remote Rendering Azure Object Anchors Map and anchor 3D Render high quality, interactive Align and anchor content to physical spaces 3D content and stream it to your 3D content to physical objects devices Overview of HoloLens 2 What can HoloLens 2 do? - YouTube IMMERSION COMFORT TIME TO VALUE
Field of View More Comfortable Open Development
IMMERSION
FULLY ARTICULATED WINDOWS HELLO AND LARGER FIELD OF VIEW HAND TRACKING EYE-TRACKING Voice
ERGONOMICS
ALL MORPHOLOGY TYPES UNIVERSAL FIT SYSTEM BALANCED WEIGHT DISTRIBUTION
A comprehensive HoloLens 2 portfolio to Meet the needs of your enterprise
HoloLens 2 Development Edition
HoloLens 2 HoloLens 2 Trimble XR10 HoloLens 2 Industrial Edition with HoloLens 2 Development Edition 28 Meeting customer needs on their terms with The HoloLens 2 Industrial Edition
Designed, built and tested to support highly-regulated environments.
Device features:
• Meets particle emissions standards for ISO 14644-1 Class 5 clean rooms
• Suitable for use in Class I, Division 2-Groups A, B, C, and D HAZLOC environments
Added benefits:
• Extended 2-year warranty
• Rapid replacement program
MIXED REALITY ISV INDUSTRY SOLUTIONS
Easily leverage existing 3D assets and incorporate IoT data to create scalable, hands-free mixed reality experiences.
Vuforia Studio Microsoft HoloLens 2: Partner Spotlight with PTC - YouTube MIXED REALITY ISV INDUSTRY SOLUTIONS
Easily leverage Augmented Surgeon existing 3D assets and Solution: Plan, incorporate IoT data Educate, Collaborate to create scalable, & Navigate hands-free mixed reality experiences.
Vuforia Studio VSI Holomedicine A hospital revolution: VSI Holomedicine - YouTube MIXED REALITY PARTNER SOLUTIONS
Easily leverage A 4D construction .rooms solution provides existing 3D assets and visualization app for a holographic incorporate IoT data viewing digital experience, to to create scalable, collaborate more models in mixed effectively with their hands-free mixed reality. reality experiences. customers on how they design and build interior spaces
Vuforia Studio VSI Telemedicine Island Labs Mixed-reality solution creates revolutionary new vision for interior design - YouTube
Dynamics 365 Remote Assist & Guides Overview Introducing Dynamics 365 Remote Assist Dynamics 365 Remote Assist
Collaboration
Solve problems in real-time
• Share real-time view from anywhere on HoloLens, iOS or Android devices with remote collaborators
• Combine video, screenshots, and annotations in the call to improve speed and accuracy of collaboration
• Communicate with individuals or groups using Microsoft Teams so important participants are involved
• Call the Dynamics field service expert listed in the Support Contact field of a booking to quickly connect with the right person to solve the issue Dynamics 365 Remote Assist
Collaboration
Bring critical information into view
• Combine video, screenshots, and augmented reality annotations with the devices you already use for enhanced collaboration
• Place virtual documents such as PDF or JPG directly into the real-world with HoloLens or share in chat on Mobile
• Collaborators can also share their screen to display any relevant information such as schematics and diagrams from familiar applications
• Access and interact with work order information in-view with Dynamics 365 Field Service Dynamics 365 Remote Assist
Collaboration
Walk the site without being on location
• Eliminate travel costs and reduce environmental impact with remote inspections, real-time auditing, and live training from anywhere with individuals or groups
• Record the entire call and call log information for reference - participant(s), day and time, and call duration
• Take screenshots of issues to capture pertinent detail for future reference and faster resolution
• Save call log information and call artifacts to a booking with Dynamics 365 Field Service Dynamics 365 Remote Assist How it leverages Microsoft Teams
With Remote Assist, With Teams: you can do more:
Connect and collaborate Resolve issues quickly with mixed reality in real-time annotations and remote access to documentation and work orders Chat with participants and share files Improve inspections and remote audits with markups in real-time view and support for low bandwidth situations Record one-to-one, group calls, and meetings Connect with the right person to resolve by calling an expert directly from the Dynamics 365 Field Service booking
Add additional context to your work order by posting call logs, files, and snapshots to Dynamics 365 Field Service Dynamics 365 Remote Assist
Mobile & HoloLens
Collaboration with the mobile app
• Access Remote Assist on iOS or Android devices*
• Annotate in 3D for best experience, or annotate in 2D using high resolution still image capture
• Collaborate with audio and video calling and in-call text chat with users on Microsoft Teams
• Low bandwidth mode Dynamics 365 Remote Assist
Mobile & HoloLens
Enhanced experience with HoloLens 2
• Use HoloLens for a hands-free experience. Read more about HoloLens 2 usage environments
• Collaborate comfortably with new ergonomic design and battery life for longer wear times
• Increase effectiveness of calls with natural gestures and wider field of view
• Stay current of new enhancements with continued HoloLens operating system updates Introducing Dynamics 365 Guides Dynamics 365 Guides
Reduce errors and help increase safety Provide employees with step by step guided instruction to reduce errors and showcase warnings and safety hazards in place so they know what to avoid while they work.
Standardize skills and reinforce compliance Capture best practices in standardized instructions that move with employees as they work, so no step is missed. Equip managers to create guides, with no coding or mixed reality experience required.
Increase retention and close knowledge gaps Quickly deploy skill-building initiatives that target a specific gap within a team and create consistency across locations enabling employees to have confidence with new tasks and processes.
Improve training and processes Onboard employees and aggregate their task performance data into real-time Microsoft Power BI dashboards, making it easier to identify where process improvements are needed. Dynamics 365 Guides
Author Operate Analyze Easily create content Instructions that follow you Workforce insights 3D assets built-in Holograms to show vs tell Improve processes WYSIWYG Heads up hands free Integrate operations Dynamics 365 Guides
Open a Power App from any step (with a Common Data Service upgrade)
• Utilize existing PowerApps that are already deployed in your organization from HoloLens
• Create new PowerApps to enhance your Guides functionality
(See instructions here for how to retrieve the link to a PowerApp) Dynamics 365 Guides
Empowered workforce
Analyze training effectiveness (Requires Power BI Service)
• Gain data-driven insights to improve the effectiveness of your training through Guides
• Use Guides data to support continuous improvements for processes and workflows
• Share your Guides Analytics Power BI reports within your organization using Power BI Service Dynamics 365 Guides
Assembly Service Operations Certification Safety Show employees the tools Equip technicians with Rollout new processes with Ensure every employee Experience dangerous and parts they need and how guided instructions at the increased consistency by meets high standards by procedures virtually before to use them in real work job site, eliminating the need turning employee knowledge quickly identifying who attempting in the physical situations. to schedule additional visits. into a repeatable tool. needs help where. environment.
Standardization and compliance
Unfamiliar equipment or process
Ongoing training Dynamics 365 Guides works across your organization Dynamics 365 Guides What’s New
Spatial triggers (with a CDS upgrade)
• Use hand tracking to place ‘trigger zones’ in world and detect when the operator’s hand enters the zone, automatically advancing to the next step.
• Enable operators to train the correct movements and stay in flow so that they can improve their speed of execution and prepare to be on the line. Dynamics 365 Guides What’s New
Branching guides (with a CDS upgrade)
• Create guides with multiple choices for more complex scenarios.
• Add up to 3 choices on a step and branch to any other step in a guide.
• Apply a ‘branch’ to a spatial trigger for truly adaptable workflows. Azure Mixed Reality Services
Mixed Reality Microsoft Azure Mixed Reality Services
Remote Rendering Spatial Anchors Object Anchors
Render high-quality, interactive Map and anchor 3D content to Align and anchor 3D content to 3D content, and stream it to physical spaces physical objects your device
PREVIEW AVAILABLE PREVIEW Here can be anywhere
Feel presence Experience together Connect from anywhere Here can be anywhere Customer Stories & Scenario Inspiration Proven Enterprise Use Cases As we have worked with thousands of enterprise customers the past several years, we have seen a handful of use cases that can be applied across many industries which have a fast time to value (TTV) and a proven return on investment (ROI):
Training & Guides & Task Remote Design & Sales Contextual Simulation Management Assistance Prototyping Assistance Data Overlays
Train and onboard new Enable your employees employees faster than Walk employees step- to remote in an expert Carry less inventory and Make pertinent and ever using a “learning by-step through guided anytime they need it. Convert your CAD and close large ticket sales real-time data available by doing” approach. experiences to help BIM files to 3D digital faster by using mixed when and where it is them complete tasks Experts can see and twins to quickly iterate reality to showcase any needed, enabling your Simulate scenarios so faster and more interact with the same and collaborate on new configuration or employees to make that your employees accurately then ever environment as the product designs. customization to your better, faster, more can practice how to before. employee as if they customer. informed decisions. respond in certain were there. situations Training & Simulation University of Canberra uses HoloHuman & HoloPatient to transform how future Doctors and Nurses are trained Challenge Traditional teaching has long helped prepare healthcare professionals to provide critical care to their future patients. Challenges remain in simulating these experiences, whilst boosting knowledge of the anatomy and physical assessments.
Solution HoloPatient is a collaborative platform of Standardized holographic simulations for safe and practical instructor-led development of clinical skills. Users can see, hear and assess holographic patients with new pathologies that exhibit varied symptoms and presentations of different illnesses or injuries.
Results “We can discuss the signs and symptoms of anaphylaxis, but that’s not like actually seeing a patient go into shock. HoloPatient allows students to see and really explore a patient’s deterioration in a safe environment without the need to act.” – Dr Frost
Impact on COVID Assessment Free Simulation App Prepares Healthcare Workers to Treat COVID-19 Patients
www.gigxr.com Guides & Task Management Lockheed Martin Drive Improvement on Assembly Processes for NASA’s Orion Program
Challenge The Orion assembly is an incredibly complex program of work, with each day bringing forth a range of tasks and processes that users have never before worked through. Until now, technicians used paper instructions or 3-D models on a computer in certain manufacturing processes of Orion.
Solution After assessing a range of options, Lockheed Martin deployed guided holographic instructions for training and assembly, enabling their teams to accelerate training, and rapidly complete tasks laid out in their environment.
Results • Productivity gains: • 8 hours of work completed in 45 minutes • 8 shift actvitiy completed in 6 hours • Quality: • Zero errors in 2 years of the program to date. • Improved Simulation/Training processes Remote Assistance Mercedes-Benz Transforms Technician Support
Challenge Modern cars have become computerized machines packed with AI, augmented reality and other fancy tech, the work required to keep them on the road has become more complex – and technicians’ dreams aren’t yielding as many. easy answers. COVID shutdowns have created a lot of pressure on dealerships, and efficiency is critical.
Solution Remote Assist on HoloLens 2 connect technicians with expertise across the organisation, spanning many countries. Whilst working hands-free, teams can rapidly resolve issues and get cars back with their owners in less time, and through the challenges brought on through COVID.
Results • Efficiency: “We can now resolve issues often in minutes and hours as compared to days, allowing us to serve our customers more quickly with faster turnaround times,” • Health & Safety: “engines now have high-pressure, direct fuel injection. The fuel line is at almost 400 bars of pressure. If you crack open the wrong line, fuel will enter your skin, whether you like it or not.” Design & Prototyping Ford Re-imagines Vehicle Design
Problem Traditionally, a lot of this work is done using clay models, which are expensive and time-consuming to develop, and costly to change. While these models will continue to be an extremely important part of the design process, the team needs to feel confident in their design decisions early on before they are built at scale in clay.
Solution With HoloLens, Ford designers are blending 3D holograms digitally with both clay models and physical production vehicles. This allows designers to experiment much more quickly without having to physically build every design prototype in clay, which can limit creativity with longer steps.
Results What used to take days with multiple physical models could take only hours using a HoloLens and one physical mode
It’s amazing that we can combine the old and new – clay models and holograms in a way that both saves time and allows designers to experiment and iterate quickly, to dream up even more stylish and clever vehicles. Microsoft HoloLens is a powerful tool for designers as we continue to reimagine vehicles and mobility experiences in fast changing times.” Moray Callum, Vice President, Design Sales Assistance Hevolus and Natuzzi create the Augmented Store
Problem As Retail spaces continuously evolve, many retailers will struggle to create urban stores with footprint they need to portray products in their best light, and demonstrate their full range.
Solution With HoloLens, Natuzzi introduces a new retail space in which Mixed Reality coexists with the physical store to offer personalized services and improve the relationship one-to-one with the consumer. Consumers put on a headset and interact with spaces and furniture—shifting products around, changing patterns and colours, and moving through the space to gain a full-range view of what a room could look like with their favourite Natuzzi furniture.
Results • Retail Week 2020: Hermes Customer Experience Game Changer Award • Reduce inventory required at each location • Lower the space and architectural investment required for each store • Reduce Contextual Data Overlay Onsite maintenance of Nuclear Reactors
Problem Nuclear sites are very demanding, with increasingly difficult work conditions and rising demands in more stringent health and safety processes.
Solution Improve training with step-by-step instructions Onsite maintenance and inspections at nuclear reactor
Results Boost operational efficiency and improve quality Increase safety and security
This project will enable our operators to be more autonomous in their interventions even in very demanding situations. With HoloLens, our technicians will have direct access to onsite documentation or will be able to call for expert help thanks to an internal audio and visual communication solution, while allowing freedom of movements. It will reduce the time spent in the heart of the reactors, as well as boost operational performances and security at work. Yves André, Strategy Director, Framatome Training & Guides & Task Remote Design & Sales Contextual Simulation Management Assistance Prototyping Assistance Data Overlays Qualifying Opportunities Mixed Reality HoloLens 2 Customer Qualification Customer Considerations Eligible Country? Environmental factors? Existing AR/MR/VR customer? ▪ North America: US and Canada ➢ Where will device be used? ▪ Europe: Austria, Belgium, Denmark, ➢ Factory floor/protected NO Finland, France, Germany, Italy, environments ? ➢ Position out-of-the-box scenarios Ireland, Netherlands, Norway, ➢ Extreme external with D365 MR apps Portugal, Spain, Sweden, Switzerland, conditions? ➢ Remote Assist (for HL2 and and UK ➢ Review Guidance mobile), • APAC: Australia, China, Hong Kong, ➢ Guides (HL2 only) Japan, Singapore, South Korea, ➢ Is their connectivity available YES Taiwan, and New Zealand (WiFi, 5G, etc.) ? ➢ Discover their existing scenarios ➢ Identify specific needs? Focus Verticals? ➢ Does the customer require ➢ Identify any gaps with current ✓ Manufacturing (& mftg processes in Certifications solution? other verticals) and Utilities ➢ ATEX/EX ✓ Health/ Health Education ➢ Clean Room MR Strategy ✓ Retail (high value/luxury goods) ➢ Looking to drive new business ✓ Defence ➢ Does the customer use Intune enablement vs. efficiency (cost ✓ Transport for MDM? savings), or both? ✓ AEC Thank You