Khronos Cross-Platform Standards Update Vulkan, Anari, Openxr, Gltf and Opencl

KHRONOS CROSS-PLATFORM STANDARDS UPDATE VULKAN, ANARI, OPENXR, GLTF AND OPENCL

Neil Trevett, GTC, March 2020 VP NVIDIA, Khronos President AGENDA

Latest updates on key Khronos open standards That are of most interest to the GTC audience Vulkan, ANARI, SPIR-V, OpenXR, glTF and OpenCL On Khronos format slides

NVIDIA and Khronos Standards How NVIDIA is supporting and deploying Khronos Open Standards On NVIDIA format slides

2 Khronos Connects Software to Silicon Open interoperability standards to enable software to effectively harness the power of multiprocessors and accelerator silicon

3D graphics, XR, parallel programming, vision acceleration and machine learning

Non-profit, member-driven standards-defining industry consortium

Open to any interested company

All Khronos standards are royalty-free

Well-defined IP Framework protects participant’s intellectual property >150 Members ~ 40% US, 30% Europe, 30% Asia

3D Graphics 3D Assets Portable XR Parallel Computation Desktop, Mobile, Web Authoring Augmented and Vision, Inferencing, Machine Embedded and Safety Critical and Delivery Virtual Reality Learning

Guidelines for creating APIs to streamline system safety certification

NVIDIA is fully committed to continuing to support OpenGL Exploring expanded interop with Vulkan New functionality is primarily Vulkan-focused

http://vulkan.gpuinfo.org/ Platforms

Apple Desktop Android (via porting Media Players Consoles (Android 7.0+) layers) (Vulkan 1.1 required on Android Q)

Virtual Reality Cloud Services Game Streaming Embedded Engines

Croteam Serious Engine

Note: The version of Vulkan available will depend on platform and vendor © The Khronos® Group Inc. 2020 - Page 5 Vulkan AAA Content

Vulkan 1.1 Extensions Maintenance updates plus additional functionality Roadmap Discussions Timeline semaphores Ray Tracing DX/HLSL compatibility Variable Rate Shading Bindless resources Reduced precision arithmetic Accelerated Video Encode/Decode Formal memory model Machine Learning Primitives Buffer references Mesh Shaders SPIR-V 1.5

January 2020

DLSS 2.0 smart scaling can be used with Vulkan applications Does not need per application training

Set of Extensions to Vulkan, GLSL and SPIR-V Seamlessly integrates ray tracing into Vulkan 1.X Familiar Ray Tracing Pipeline Architecture The industry’s first open, cross- Straightforward porting between Vulkan Ray Tracing and DXR vendor, cross-platform standard for - including re-use of ray tracing shaders written in HLSL ray tracing acceleration Can be accelerated on existing Vulkan Ray GPU compute and dedicated ray DX12 / DXR Tracing tracing cores Extensions are Provisional Ray Tracing Pipelines Yes Yes Launched 17th March 2020 DXR 1.1 Open to developer feedback Ray Queries Optional Inline raytracing before finalization https://khr.io/vkrayprovfeedback Language for GLSL or HLSL HLSL Ray Tracing Shaders

Pipeline Libraries Yes Yes

Build Acceleration Structure Optional No on Host Straightforward port from NVIDIA VKRay Deferred Host Operations Optional No vendor extension to KHR extensions Shipping beta drivers today Capture/Replay Support for Example code coming soon Optional No Tools (e.g. RenderDoc) https://www.khronos.org/registry/vulkan/

HLSL

HLSL and Vulkan with DXC DXC Microsoft’s DXC HLSL compiler was open sourced in Jan 2017 Front-end Parsing Google and others have added SPIR-V code generation to DXC and Validation with Microsoft’s knowledge and approval Vulkan developers can now choose between GLSL and HLSL! AST Abstract Syntax Tree Vulkan Ray Tracing includes GLSL and SPIR-V Extensions Enabling compiled GLSL/SPIR-V shaders to operate in a Ray Tracing Pipeline – similar to HLSL features used in Direct3D’s DXR SPIR-V DXIL Backend Backend HLSL for Vulkan Ray Tracing NVIDIA added code generation to DXC to generate SPIR-V for the NVIDIA VKRay ray tracing vendor extension from HLSL SPIR-V DXIL Vulkan Ray Tracing Extensions supported in HLSL soon

Developers can port HLSL shaders SPIR-V Tools DXIL Tools with minimal changes between Vulkan Ray Tracing and DXR

© The Khronos® Group Inc. 2020 - Page 9 NVIDIA AND VULKAN NVIDIA deeply engaged in Vulkan and driving extend/consolidate cycle NVIDIA shipped Vulkan 1.0, 1.1 and 1.2 on day of spec releases Shipped beta Vulkan Ray Tracing extensions on day of spec release Increased Vulkan support in NSIGHT 2020.2 development tool – close parity now with DX12 NVIDIA chairing multiple Vulkan initiatives at Khronos Ray Tracing: contributed NVIDIA VKRay to catalyze Vulkan Ray Tracing KHR extensions Vulkan Portability: bringing layered Vulkan to Apple, WebGPU, silicon without native drivers etc. Machine Learning: low-level inferencing primitives

Cloud – Windows and Linux Desktop – Kepler and later Nintendo Switch Shield Android TV Jetson

10 Open Source Layering Projects Fighting Platform Fragmentation

Layers Vulkan OpenGL OpenCL OpenGL ES DX12 DX9-11 Over Vulkan is effective clspv GLOVE DXVK porting layer for API Vulkan Zink clvk Angle vkd3d WineD3D portability and stack simplification gfx-rs OpenGL Ashes Angle WineD3D

Microsoft Microsoft Microsoft DX12 gfx-rs ‘GLOn12’ ‘CLOn12’ D3D11On12

gfx-rs DX9-11 Ashes Angle

MoltenVK MoltenGL Metal gfx-rs Angle

‘Vulkan everywhere’! Working towards Even if no native ‘OpenCL drivers on platform Everywhere’!

Applications

Vulkan Portability Extension provides query for available functionality

Vulkan Open source SDK on macOS All present functionality must be macOS SDK Build, run, and debug apps conformant to use logo with validation layer support https://vulkan.lunarg.com/

SPIRV-Cross macOS / iOS Open source Convert SPIR-V shaders to Run-time For MacOS/iOS Metal Shading Language Maps Vulkan to Metal Beta open source For MacOS/iOS Vulkan applications shipping today on macOS or iOS using MoltenVK

Scientific Visualization Portability APIs to describe objects in a scene - the renderer does the rest Ray tracing was key motivation – but can drive any renderer

SciViz Apps Scene Graphs and Engines

Renderers: Intel OSPRAY, AMD ProRender, NVIDIA VisRTX etc.

Acceleration APIs: Embree, Optix, Radeon Rays, CUDA, OpenCL, Vulkan etc.

Hardware: CPUs, GPUs etc. Industry Support

Working Group Participants

Games Engines

Desktop/Console VR

Mobile VR

GPUs

OpenXR is a collaborative design UI Devices Integrating many lessons from proprietary ‘first-generation’ XR API designs

* OpenXR 1.0 is focused on enabling cross-platform applications. Optional device plugin interface will be supported post V1.0 © Copyright Khronos® Group Inc. 2020 Page 14 OpenXR 1.0 Availability

Significant Community Feedback Finalize Conformance Test Suite Enable Officially Conformant Implementations Improved Input subsystem, game engine editor support, loader … Provisional OpenXR for Windows Mixed Reality headsets Specification and HoloLens 2 GDC, March 2019 PLUS extensions to support HoloLens 2 hand tracking, eye tracking, spatial mapping and spatial anchors Many more coming! Ratify and Release OpenXR support for Oculus Rift and Oculus Quest OpenXR 1.0 Oculus PC (for Rift) and Android SDK (for Quest) SIGGRAPH, July 2019 include OpenXR for native C/C++ development

‘Monado’ OpenXR open source implementation Support OpenHMD hardware and Nova Northstar AR HMD

OpenXR 1.0 plugin for Unreal Engine v4.2.4

Application or Engine

Display, High-performance, low-latency composition and Cross-platform access to XR 3D rendering and composition* optical correction HMDs and sensors Multiview parameters XR application lifecycle Context priority Input device discovery and events Front buffer rendering Sensor tracking and pose calculation Tiled rendering (beam racing) Frame timing and display composition Variable rate rendering Haptics Control

* OpenXR can be used with other 3D APIs such as Direct3D, OpenGL and OpenGL ES

Low latency Sensor Data

Location–aware Content Requests OpenXR APIs hide the 5G round trip from applications MEC (Multi-access Edge Sensor handling Computing) Server 1. Processes sensor data, including machine learning for environmental lighting, occlusion, Needed assets Display composition scene semantics, object loaded to reconstruction and UI edge server 2. Generates imagery from 3D Wireless mobile device with models, including stereo, foveal rendering, ray-tracing, optics pre- display and sensors distortion, varifocal processing Apps and 3D Assets NVIDIA EGX

Generated Augmentations & Scenes

17 Modo Paint 3D

Titania Discover Repositories Game Engines

3D Authoring Tools Web Engines

3D Builder Oculus Create Experience 3D printing Tools Apps / Engines 3D Asset Format for Efficient VR / AR Authoring Tools Transmission and Processing

Sony 3D Creator Drive Demand Apps and Engines Users 3D Live Object 3D Scanning Tools gltfpack Mixed Reality Viewer

Collada2gltf OBJ2GLTF FBX2glTF Windows Mixed Convertors and Optimizers Reality Home

glTF Reference Viewer VR / AR Apps and Engines gltf-vscode glTF-asset-generator glTF-validator glTF-Toolkit Validation and Reference Tools Productivity and Social Apps

© The Khronos® Group Inc. 2020 - Page 18 glTF Roadmap Second Generation Physically-Based Rendering (PBR) JPEG must be fully Set of coherent extensions decompressed into 14,000,000Clear coat (imminent) GPU memory Absorption/attenuation 12,582,912 File Size GPU Size Subsurface scattering Anisotropy

Inspiration from Dassault 10,500,000 Systèmes Enterprise PBR Shading Model (DSPBR) and MDL

Wide industry cooperation

7,000,000

Universal textures can beSeeking Requirements directly transcoded to Subdivision3,500,000 surfaces FlightHelmet_baseColor 2,778,518 compressed GPU textures 2048 x 2048, RGB Advanced Animation 2,097,152 LOD and Streaming Compressed Point Clouds Cross-asset linking 315,619 232,104 0Enhanced Metadata Composability PNG JPEG

glTF Universal Textures ETC1S Instancing (imminent) Working Group is constantly balancing CAD/BIM model support Basis Universal encoding/transcoding feature requests against the ‘glTF Encryption and security Uncompressed Basis Universal KTX2 Container Prime Directive’ - remain a universal 3D Printing and easy to process delivery format

• Heterogeneous • CUDA-X libraries and tools • Widely available across compute devices • Integrated HW/SW roadmap multiple platforms • Cross-vendor for rapid innovation • Integrated rendering, data open standard • BUT GPU Only movement and compute • Simpler to program • Explicit hardware control than rendering APIs • BUT GPU Only

Increasing layering, tooling and interop functionality enabling enhanced development and deployment flexibility

20 OpenCL is Widely Deployed and Used

Desktop Creative Apps Parallel Machine Learning Languages Libraries and Frameworks Intel Math and Physics Vision and Imaging Machine Learning Compilers Libraries Xiaomi Libraries

Modo Intel clDNN NNAPI SYCL-DNN Linear Algebra Libraries VeriSilicon

SYCL-BLAS Synopsis MetaWare EV TI DL Library (TIDL) CLBlast Arm Compute Library

Hardware Implementations

This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2020 - Page 21 OpenCL Applications over Vulkan • Clspv – Google’s open source OpenCL kernel to Vulkan SPIR-V compiler - Tracks top-of-tree LLVM and clang, not a fork • Clvk – prototype open source OpenCL to Vulkan run-time API translator • Used for shipping production apps and engines on Android - Adobe Premiere Rush video editor – 200K lines of OpenCL C kernel code - Butterfly Network iQ Ultrasound on Android - Xiaomi MACE inferencing engine

OpenCL C or OpenCL C++ for OpenCL Application Kernel Sources Host Code

Clspv Clvk run-time Compiler API Translator

Runtime

https://github.com/kpet/clvk https://github.com/google/clspv

Production-class OpenCL 1.2 on Linux and Windows NVIDIA Chairs the Active, ongoing improvements in performance and power efficiency for new architectures Khronos OpenCL Working Group Multi-GPU optimizations, multi-command-queue use case tuning Optimized data transfers and GPU memory allocation New Functionality Half & Half2, arithmetic and conversions - in development Expanding OpenCL Interop options OpenGL and D3D9/10/11 – today. DX12 being planned OpenCL/Vulkan interop – aiming for 2H20 - using standard Vulkan external memory interop hooks NVIDIA driving Vulkan/OpenCL interop extension at Khronos Providing guidance for deploying OpenCL apps over Vulkan Leveraging the open source SPIR-V compiler

23 NVIDIA AND KHRONOS API STANDARDS

Cloud – Windows and Linux Desktop – Kepler and later Nintendo Switch Shield Android TV Jetson NVIDIA has shipped Vulkan 1.2 and Ray Tracing drivers on day of specification release NVIDIA chairing multiple Vulkan initiatives at Khronos: Ray Tracing, Machine-Learning, Vulkan Portability etc. Increased Vulkan support in NSIGHT graphics development tools https://developer.nvidia.com/Vulkan

Actively working to help evolve Production-class OpenCL 1.2 on Linux and Windows NVIDIA initiated industry cooperation OpenXR to complement VRWorks Active improvements in efficiency and performance Portable Analytic Rendering API and enable EGX Edge Server AR Vulkan Interop in development NVIDIA chairing the working group https://developer.nvidia.com/vrworks https://developer.nvidia.com/opencl https://www.khronos.org/anari

24 How To Get Involved! • More information at other GTC recorded sessions - [S21770] NVIDIA Vulkan Features Update – including Vulkan 1.2 and Ray Tracing - [S22694] Ray Traced Reflections in Wolfenstein Youngblood • Any company or organization is welcome to join Khronos! - For a voice and a vote in any of these standards – membership starts at $3,500 • OR request an invite to Vulkan, OpenCL, OpenXR Advisory Panels - No fee, execute Khronos NDA and IP Framework, provide requirements and spec feedback • We welcome your feedback at NVIDIA or Khronos - Khronos Forums: https://forums.khronos.org/ - Khronos Slack Channels: https://khronosdevs.slack.com/messages - Khronos open source GitHub repositories: https://github.khronos.org/ • Contact Neil Trevett - [email protected] | @neilt3d | www.khronos.org