OpenXR Ecosystem Update Bringing to Life the Dream of Portable Native XR July 2020

This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2020 - Page 1 Khronos Standards for XR

Create and deploy 3D assets and scenes

Vision and sensor processing, inferencing acceleration

High-performance, low-latency 3D Graphics

Portable access to native XR runtimes

This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2020 - Page 2 OpenXR Ecosystem Updates

Khronos open sources conformance tests and launches Adopters Program First officially conformant runtimes shipping from and Preview implementations from Valve, Varjo and Hand and eye tracking cross-vendor extensions for advanced UI OpenXR being used by key games and open source Minecraft! ! WebXR!

Virtual Reality Now is the time for application V developers to leverage OpenXR for widespread deployment! A

This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2020 - Page 3 XR Portability

XR App/ XR App/ XR App/ Experience Experience Experience

Game WebXR Engine

Application Interface

And More!

OpenXR provides cross-platform, high-performance access directly into XR device runtimes across multiple platforms

This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2020 - Page 4 Widespread Industry Support

Companies publicly supporting OpenXR

OpenXR is a collaborative design that integrates many lessons from proprietary ‘first-generation’ XR to create a new generation API with cutting-edge capabilities and a flexible, extensible, future-proof architecture

This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2020 - Page 5 Broadening OpenXR 1.0 Availability

OpenXR 1.0 Adopter Program Released Enable Officially Conformant Implementations Significant Community Conformant OpenXR 1.0 for Windows Feedback Improved Input subsystem headsets and HoloLens 2 Game engine editor support PLUS extensions to support HoloLens 2 hand tracking, Loader and Layers… eye tracking, spatial mapping and spatial anchors Provisional Conformant OpenXR 1.0 for and Quest Specification SDK Oculus PC SDK & Android SDK V19 include OpenXR native C/C++ development GDC, March 2019 Developers can now submit their OpenXR apps to the Oculus Store Valve OpenXR 1.0 Developer Preview New SteamVR features will ship through OpenXR, rather than OpenVR

Ratify and Release ‘Monado’ OpenXR 1.0 open source implementation OpenXR 1.0 Supports variety of HMDs, including Project Northstar AR HMD SIGGRAPH, July 2019 Varjo OpenXR 1.0 Developer Preview For Varjo headsets

OpenXR 1.0 plugin for v4.2.5 Enhanced support for late stage reprojection, mixed reality capture from a 3rd person camera view, optimized rendering by up to 2ms/frame

Hand and eye tracking cross-vendor extensions for advanced UI Shipping on HoloLens 2. Ultraleap hand tracking developer preview

This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2020 - Page 6 First Conformant OpenXR Devices

OpenXR Use tests to aid Implementer development

Prototype OpenXR OpenXR Open source Implementation Conformance Tests

Contribute test fixes and enhancements

Test Submit Test Results Results Review A conformant OpenXR can use the Production OpenXR OpenXR OpenXR logo and has patent protection under the Khronos IP OpenXR Adopters Working Framework Implementation Website Group

Test Results Approved. Khronos grants Trademark and Patent License Conformant Devices

Windows Mixed Oculus HoloLens 2 Reality Headsets Quest

This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2020 - Page 7 OpenXR Architecture Flexibility

VR

Oculus Quest AR VR

Windows Mixed Reality

Microsoft Oculus Rift S HoloLens 2

All In One Tethered

This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2020 - Page 8 Advanced UI Cross-Vendor OpenXR Extensions

Developers can build cross-platform applications that use advanced UI solutions from different technology vendors OpenXR API layers can be used implement extensions

Hand Tracking Eye Tracking

26 unique joints per hand for fully Eye gaze interaction for intuitive interfaces articulated hands visible to the user 2-Step Interaction Shipping on HoloLens 2 Hand-eye coordination and Microsoft Hand Mesh Extension Natural aiming for HoloLens 2 layers over it Shipping on HoloLens 2 Ultraleap developer preview available

This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2020 - Page 9 OpenXR and Minecraft

Microsoft is excited to announce that Minecraft’s new RenderDragon rendering engine is building its desktop VR support using OpenXR!

This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2020 - Page 10 OpenXR and Open Source

Microsoft open source OpenXR Samples for Mixed Reality Developers, shows how to use OpenXR to access the full capabilities of HoloLens 2

Google 81 uses OpenXR as its default backend for WebXR, enabling Chrome and browsers to use any OpenXR-compatible hardware Blender 2.83 integrates OpenXR to deliver native VR scene inspection capabilities

This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2020 - Page 11 OpenXR is used with a 3D API Application or Engine

High-performance, low-latency Display, Cross-platform access to XR composition and 3D rendering and composition* optical correction HMDs and sensors Multiview parameters XR application lifecycle Context priority Input device discovery and events Front buffer rendering Sensor tracking and pose calculation Tiled rendering (beam racing) Frame timing and display composition Variable rate rendering Haptics Control

OpenXR is strongly influenced by Vulkan with a * OpenXR can be used with other 3D APIs shared spec toolchain and support for API layers. OpenXR is a “lower-frequency” API than Vulkan such as Direct3D, OpenGL and OpenGL ES and is a much smaller spec This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2020 - Page 12 Bringing XR to the Web

Native XR Apps Web XR Apps

Lifting OpenXR functionality into the Web stack

Native 3D Web 3D Engines Close cooperation Engines between WebXR and OpenXR

Khronos provides the foundation for native and Web-based 3D/XR

This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2020 - Page 13 Structure of an OpenXR App • Get started - Create an Instance - Choose extensions, layers, bind to graphics API • Find out where/how to run - Get HMD characteristics - mono/stereo, form factor etc.. /user/hand/left/input/a/click • Set up interaction/input handles (/interaction_profile/ControllerCorp/fancy_controller/input/a/click) - Bind physical inputs to actions OpenXR Runtime grab_object, teleport etc. .../input/trigger/click Explode .../input/a/click Teleport • Prepare your immersive experience - Create Session

- Create Swapchain to drive the display XrAction: “Teleport” • Participate in the frame loop - Handle input and haptics - Manage swapchain to drive imagery to the display - Poll for events

This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2020 - Page 14 Cloud XR with 5G and OpenXR

OpenXR will enable AR Significant industry applications to run portably effort in developing

on edge server architectures Low latency this use case Sensor Data

Runtime using 5G implemented across device and server and accessed through OpenXR API

Sensor handling

MEC (Multi-access Edge Display composition Computing) Server 1. Processes sensor data, can include Wireless mobile device machine learning for environmental with display and sensors lighting, occlusion, scene semantics, object reconstruction and UI Generated 2. Generates imagery from 3D Augmentations & models, can include stereo, foveal rendering, ray-tracing, optics pre- Scenes distortion, varifocal processing

This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2020 - Page 15 OpenXR Win-Win-Win XR End-Users Can run the apps they want on their system – reducing market confusion and increasing consumer confidence

XR Vendors Can bring more applications onto their platform by leveraging the OpenXR content ecosystem Now is the time for application developers to leverage OpenXR for widespread application deployment! www.khronos.org/openxr/

XR ISVs Can easily ship on more platforms for increased market reach

This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2020 - Page 16