OpenXR: On an HMD near you now!
Brent Insko
® This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2021 - Page 1 Unifying Reality
OpenXR: On an HMD near you now! Brent E. Insko, PhD Lead XR Architect at Intel & OpenXR Working Group Chair © Khronos® Group Inc. 2020 - Page 2 What is OpenXR?
OpenXR is a royalty-free, open standard that provides high-performance access to Augmented Reality (AR) and Virtual Reality (VR)—collectively known as XR—platforms and devices.
This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2021 - Page 3 OpenXR handles communication to and from an application and an XR platform
Application XR device
This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2021 - Page 4 OpenXR handles communication to and from an application and an XR platform
• Image(s) to display • Audio • Haptic responses
Application XR device
• Head & hand pose information • Controller/input state • Display configuration / form factor
This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2021 - Page 5 Consider the variety of XR Platforms available…
This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2021 - Page 6 Cross Platform XR Application Development
Before OpenXR, program to each platform’s proprietary API
With OpenXR, program to a single, common, high-performance API
This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2021 - Page 7 OpenXR is used with a 3D API Application or Engine
High-performance, low-latency Display, Cross-platform access to XR composition and 3D rendering and composition* optical correction HMDs and sensors Multiview parameters XR application lifecycle Context priority Input device discovery and events Front buffer rendering Sensor tracking and pose calculation Tiled rendering (beam racing) Frame timing and display composition Variable rate rendering Haptics Control
OpenXR is strongly influenced by Vulkan with a * OpenXR can be used with other 3D APIs shared spec toolchain and support for API layers. OpenXR is a “lower-frequency” API than Vulkan such as Direct3D, OpenGL and OpenGL ES and is a much smaller spec This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2021 - Page 8 OpenXR Flexibility
VR
Oculus Quest / Quest 2 AR VR
Valve Index and other SteamVR compatibile HMDs Oculus Rift S
Microsoft HoloLens 2 HTC Vive Cosmos Windows Mixed Reality All In One Tethered
This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2021 - Page 9 Applications, Game Engines, Rendering Engines, Development Browsers using OpenXR! - Fall 2020 - 2021
Conformant AR and VR platforms released - Summer & Fall 2020
Conformance suite developed and Adopters program released - July 2020
OpenXR 1.0 Specification Released - SIGGRAPH, July 2019
0.90 Provisional Specification Released - GDC, March 2019
This workOpenXR is licensed underWorking a Creative Group Commons Formed Attribution - 20174.0 International License © The Khronos® Group Inc. 2021 - Page 10 Latest Developments with OpenXR
This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2021 - Page 11 OpenXR Widespread Industry Adoption Conformant Implementations
Microsoft HoloLens 2 Oculus Rift S HTC Vive Cosmos Windows Mixed Reality Headsets Quest and Quest 2
Valve SteamVR Varjo Collabora Transitioning from OpenVR to OpenXR Preliminary support shipping now Open-source Implementation Engine Support
Unreal Engine Unity Engine Chromium 81 for Chrome and Edge Browsers Support in 4.24. Optimizations in 4.25 OpenXR Plugin Preview in 2020.2+ OpenXR default backend for WebXR
This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2021 - Page 12 All the Major Platforms Transitioning to OpenXR for Future Development
This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2021 - Page 13 OpenXR Applications
Blender 2.83 integrates OpenXR to deliver native VR scene inspection capabilities
Minecraft’s RenderDragon rendering engine uses OpenXR for desktop VR support
Google Chrome
Google Chromium Microsoft Edge uses OpenXR as its default backend for WebXR, Microsoft Flight Simulator Sim Update 2 enabling Google Chrome and Microsoft Edge uses OpenXR for VR functionality browsers to use any OpenXR-compatible hardware
This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2021 - Page 14 Latest OpenXR Game
• Single Executable
• Multiple Platforms
• Built in Unity
Nominated for 'VR Game of the Year' at the 2021 VR Awards, ‘Zombieland VR: Headshot Fever’ XR Games, Sony Pictures Virtual Reality This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2021 - Page 15 What’s Next for OpenXR?
This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2021 - Page 16 OpenXR Overlays Extension • Simultaneous in-world applications in OpenXR • Can overlay one app’s 2D content “on top” of an existing application’s content • Overlay applications add a rich content into other XR Applications: e.g. Desktop OS windows in-world, In-game HUD, Virtual keyboard, Chat (e.g. Pluto VR's Pluto)
This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2021 - Page 17 Advanced UI Cross-Vendor OpenXR Extensions
Developers can build cross-platform applications that use advanced UI solutions from different technology vendors OpenXR API layers can be used implement extensions
Hand Tracking Eye Tracking
26 unique joints per hand for fully Eye gaze interaction for intuitive interfaces articulated hands visible to the user 2-Step Interaction Shipping on HoloLens 2 Hand-eye coordination and Microsoft Hand Mesh Extension Natural aiming for HoloLens 2 layers over it Shipping on HoloLens 2 Ultraleap developer preview available
This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2021 - Page 18 OpenXR and Edge Server Applications with 5G
OpenXR will enable AR Significant industry applications to run portably effort in developing
on edge server architectures Low latency this use case Sensor Data
Runtime using 5G implemented across device and server and accessed through OpenXR API
Sensor handling
MEC (Multi-access Edge Display composition Computing) Server 1. Processes sensor data, can include Wireless mobile device machine learning for environmental with display and sensors lighting, occlusion, scene semantics, object reconstruction and UI Generated 2. Generates imagery from 3D Augmentations & models, can include stereo, foveal rendering, ray-tracing, optics pre- Scenes distortion, varifocal processing
This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2021 - Page 19 What is in the future for of OpenXR?
•Clarifying and fixing a few things from OpenXR 1.0
•Expanding support for AR and MR - Scene understanding
•Additional User input/output capabilities - Expanded haptics support - Expanded body tracking support - Expanded controller and device support
This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2021 - Page 20 With OpenXR on an HMD near you, now is the time to get involved!
Officially conformant runtimes from: Microsoft, Oculus, HTC, Valve, Varjo and Collabora
OpenXR being used by real software: Unreal! Blender! Chrome! Edge! Unity! Flight Simulator! Minecraft! Games!
Virtual Reality Now is the time for application V developers to leverage OpenXR for widespread deployment! A Augmented Reality
This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2021 - Page 21 How to get started… https://openxr.org
• OpenXR Forum and Slack Channel - Forum: https://khr.io/openxrfeedback - Discussion: https://khr.io/slack
This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2021 - Page 22 Join Us… BOF
August 10th, 1PM Pacific
This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2021 - Page 23