Khronos Template 2015

KHRONOS STANDARDS UPDATE Neil Trevett, GTC, 26th March 2018 Khronos Mission

Software

Silicon

Khronos is an International Industry Consortium of over 100 companies creating royalty-free, open standards to enable software to access hardware acceleration for 3D Graphics, Virtual and Augmented Reality, Parallel Computing, Neural Networks and Vision Processing © Copyright Khronos™ Group 2018 - Page 2 Some Popular Khronos Standards

Run-time APIs File Formats

GPU 3D and Compute

Heterogenous Compute (Parallel Processing)

Vision and Inferencing

VR and AR

Non-proprietary, royalty-free open standard ‘By the industry for the industry’ Portable across multiple platforms - desktop and mobile Modern architecture | Low overhead| Multi-thread friendly EXPLICIT GPU access for EFFICIENT, LOW-LATENCY, PREDICTABLE performance

Vulkan Porting Tools

Vulkan is the primary platform 3D API on Android 7.0+

http://vulkan.gpuinfo.org/ Platforms

Desktop Mobile Consoles (Android 7.0+) Media Players

Virtual Reality Cloud Services Embedded

Game Engines

Croteam Serious Engine

Publicly announced games as of March 2018 #Vulkan = 34 #DX12 = 27 https://en.wikipedia.org/wiki/List_of_games_with_Vulkan_support

Vulkan-only AAA Titles on PC

Dota 2 on PC and macOS AAA on Linux

Strengthening the Ecosystem Improved developer tools (SDK, validation/debug layers) More rigorous conformance testing Shader toolchain improvements (size, speed, robustness) Shading language flexibility – HLSL and OpenCL C support Vulkan Public Ecosystem Forum

Vulkan 1.0 Extensions Maintenance updates plus additional functionality Building Vulkan’s Future Explicit Building Blocks for VR Deliver complete ecosystem – not just specs Explicit Building Blocks for Homogeneous Multi-GPU Listen and prioritize developer needs Enhanced Windows System Integration Increased Shader Language Flexibility - including Ray Tracing Enhanced Cross-Process and Cross-API Sharing Drive GPU technology February 2016 Vulkan 1.0 March 2018 Vulkan 1.1 Integration of Proven and New Technology into Core

Widening Platform Support Detailed specification overview and GDC session videos: Pervasive GPU vendor driver availability https://www.khronos.org/developers/library/2018-gdc Port Vulkan apps to macOS/iOS and DX12 Open source drivers

© Copyright Khronos™ Group 2018 - Page 7 New Functionality in Vulkan 1.1 • Protected Content - Restrict access or copying from resources used for rendering and display - Secure playback and display of protected multimedia content • Subgroup Operations - Efficient mechanisms that enable parallel shader invocations to communicate - Wide variety of parallel computation models supported

Example Subgroup Operations A subgroup is a set of invocations (tasks) running on a GPU Compute Unit (Note many GPUs typically support subgroup sizes of 32/64 invocations)

5 2 1 6 7 9 3 4 3 1 3 2 2 3 Active Invocations

+ X Inactive Invocations

5 5 5 5 3 9 9 7 9 9 9 9 6 6 Broadcast Shuffle Add Clustered Multiply

©© Copyright Copyright Khronos™ Khronos Group 20182017 - Page 8 Bringing Vulkan 1.0 Apps to Apple Platforms

Dota 2 running on Mac up to Applications 50% faster than native OpenGL

Very little functionality not supported. Open source SDK to build, run, and debug applications on macOS Worst Case missing functionality: Vulkan including validation layer support Triangle fans, Separate stencil reference masks Vulkan Events, Allocation callbacks macOS SDK Some texture-specific swizzles

SPIRV-Cross macOS / iOS Convert SPIR-V shaders to Run-time platform source formats Maps Vulkan to Metal Previously a paid product MoltenVK for macOS and iOS Now released into OPEN SOURCE For macOS 10.11, iOS 9.0 and up Completely free to use - no fees or royalties - including for commercial applications

Widened Platform Support Diverse open source run-times over additional backends E.g. DX12 backends useful E.g. Mozilla helping to drive gfx-rs for Vulkan for porting Vulkan to Xbox over DX12, OpenGL and Metal and UWP Platforms https://github.com/gfx-rs/gfx https://github.com/gfx-rs/portability

Enhanced SDK Layers and Tools E.g. Simulation layer to flag use of features not present in a selected target system

TODAY Supported subset features will Free, open source tools, SDKs and vary per platform and typically libraries to bring Vulkan 1.0 be increased over time with applications to Apple using Metal underlying platform capabilities Coordinated Subset Management Cross-platform standardized subsetted functionality queries Conformance tests tailored per target – what is present must work!

© Copyright Khronos™ Group 2018 - Page 10 SPIR-V Ecosystem GLSL HLSL SPIR-V • Khronos defined cross-API IR Open source Third party kernel and • Native graphics and parallel compute tools and translators glslang • Easily parsed/extended 32-bit stream shader languages DXC • Data object/control flow retained for https://github.com/KhronosGroup/SPIRV-Tools effective code generation/translation

MSL OpenCL C OpenCL C++ SYCL HLSL Front-end Front-end Front-end SPIRV-Cross GLSL

SPIR-V (Dis)Assembler

SPIR-V Validator LLVM LLVM to SPIR-V Bi-directional Khronos liaising with SPIRV-opt | SPIRV-remap Translator Clang/LLVM Community E.g. discussing SPIR-V as supported Clang target SPIR-V Optimizations Additional • Inlining (exhaustive) Intermediate Forms • Store/Load Elimination SPIR-V 1.3 released • Dead Code Elimination • Dead Branch Elimination with Vulkan 1.1 • Common Uniform Elimination • Loop Unrolling and Constant Folding IHV Driver • Common Subexpression Elimination Runtimes

11 © Copyright Khronos™ Group 2018 - Page 11 Two HLSL Front-ends For SPIR-V • Glslang - Khronos’ reference parser for GLSL, ESSL - Used by Vulkan conformance test suite and features land here first e.g. Subgroups • DXC - Microsoft/DirectXShaderCompiler for HLSL - source released Spring 2017 - “Spiregg” backend add SPIR-V code generation - Use Microsoft’s parser – tracks undocumented HLSL, including C++ features

Glslang DXC Overview First with Vulkan features Uses Microsoft’s parser Shader Model SM 4, 5.1 SM 5.1, 6ish * Platforms Windows, Linux, OSX, Android Windows Code Size Big (much of Clang + LLVM) Smallish / embeddable

© Copyright Khronos Group 2018 - Page 12 Clspv OpenCL C to Vulkan Compiler • Experimental collaboration between Google, Codeplay, and Adobe - Successfully tested on over 200K lines of Adobe OpenCL C production code - Open source - tracks top-of-tree LLVM and clang, not a fork • Compiles OpenCL C to Vulkan’s SPIR-V execution environment - Proof-of-concept that OpenCL compute can be brought seamlessly to Vulkan - Most of OpenCL 1.2 so far – shaped by submitted workloads

Gathering sample kernels OpenCL C OpenCL from the industry to drive Source Host Code direction and development

Existing open Clspv Run-time Possible future Khronos source project Compiler API Translator open source project https://github.com/google/clspv

Runtime Increasing deployment options for OpenCL developers https://github.com/google/clspv

© Copyright Khronos Group 2018 - Page 13 Public Vulkan Ecosystem Forum A new public forum to share ecosystem issues and opportunities - and coordinate solutions! Vulkan Vulkan-related Ecosystem Vulkan open source Forum Developer (open to both Khronos projects members and non- Community members)

Vulkan Working Group and Advisory Panel

Being chaired by NVIDIA. Join the Conversation at https://github.com/KhronosGroup/Vulkan-Ecosystem

VR VR VR VR VR VR VR VR App App App App App App App App 1 2 3 4 1 2 3 4

Proprietary Proprietary Engine Engine

Application Interface

Device Layer

VR VR VR VR VR VR VR VR VR VR Device Device Device Device Device Device Device Device Device Device 1 2 3 4 5 1 2 3 4 5 Before OpenXR After OpenXR VR Market Wide interoperabilityof Fragmentation VR apps and devices

OpenXR is a collaborative design 1) for cross-platform AR/PR portability 2) Integrating many lessons from proprietary ‘first-generation’ API designs

- E.g. “Move,” “Jump,” “Teleport” /user/hand/left/input/trigger/click (/devices/ControllerCorp/fancy_controller/ • Many advantages input/trigger/click) - Existing content can easily use new devices OpenXR Runtime - Mix-and-match multiple input .../input/button_a/click Explode sources to create a unified UI .../input/trigger/click Teleport .../input/grip/value SpawnKittens - Easy optional feature support … (e.g. body tracking) - Future-proofing for innovation in input devices and form factors XrAction: “Teleport”

© Copyright Khronos Group 2018 - Page 17 OpenXR Viewport Configurations • Applications can: - Query for runtime supported Viewport Configurations - Applications can then set the Viewport Configurations that they plan to use - Select and change their active configuration over the lifetime of the session

Camera Passthrough AR Stereoscopic VR / AR Projection CAVE

Photo Credit: Dave Pape One Viewport Two Viewports (one per eye) Twelve Viewports (six per eye)

/viewport_configuration/ar_mono/magic_window /viewport_configuration/vr/hmd /viewport_configuration/vr_cube/cave_vr

Call for Participation / Exploratory Group Formation Fall F2F, October 2016: Korea

Statement of Work / Working Group Formation Winter F2F, January 2017: Vancouver

Specification Work Spring F2F, April 2017: Amsterdam

Specification Work Much more detailed specification overview Interim F2F, July 2017: Washington and GDC session videos: https://www.khronos.org/developers/library/2018-gdc Defining the MVP Fall F2F, September 2017: Chicago

Resolving Implementation Blockers Winterim F2F, November 2017: Washington

Raising Implementation Issues Winter F2F, January 2018: Taipei Implementations Underway! Present Day First Public Information! GDC, March 2018 Spec Finalization – incorporating Coming Soon implementation experience Provisional Release

Conformance Testing and Implementation

Ratification and Release

All glTF spec development on open GitHub: https://github.com/KhronosGroup/glTF

OpenGL Transmission Format glTF 1.0 – Primarily for WebGL glTF 2.0 - Physically Based Rendering! Efficient transmission of 3D Uses GLSL for materials Metallic-Roughness and Specular-Glossiness Materials scenes and assets Released December 2015 Rendering API independence Released June 2017

Discover

Sony 3D Creator Create Experience Oculus

Modo Paint 3D

Drive Demand gltf-vscode Users Collada2gltf FBX2glTF Windows Mixed Reality Home glTF-validator glTF-asset-generator

© Copyright Khronos™ Group 2018 - Page 21 glTF Recent Highlights • Facebook now supports glTF – drag and drop models to your feed - Driving exporter/tool demand: FBX2glTF, Modo, … • Sketchfab has over 150K glTF 2.0 models - Under Creative Commons license GDC session videos: https://www.khronos.org/developers/library/2018-gdc • Adobe Dimension is adopting glTF - glTF publishing service for interactive marketing materials • Not just for the web – native engines adopting! - Unity helping open source glTF importer/exporter - Unreal 4.19 has experimental glTF import • Focus on ecosystem robustness – test, test, test! - gltf-vscode & glTF-Validator - glTF-Asset-Generator - glTF-Sample-Models • Vibrant open-source community

© Copyright Khronos Group 2018 - Page 22 Google Draco Geometry Compression Extension • Draco glTF Extension released in February 2018 - Designed and built for compression efficiency and speed - great fit with glTF! - Up to 12X Model Compression - With no loss of visual fidelity • Google Draco encoders and decoders in open source - C++ source code encoders | C++ and JavaScript decoders • Draco compression already in use - Producing: AMD Compressonator, FBX2glTF, glTF pipeline, Insimo - Ingesting: three.js, Babylon.js, PlayCanvas, Cesium, UX3D Engine, Hilo3D

© Copyright Khronos™ Group 2018 - Page 23 glTF KHR_texture_transmission Extension • Universal Texture File Format - Extension in design - looking for industry feedback - Compact to transmit – efficient to unpack to variety to hardware texture formats - Built upon the Basis file format and transcoders that Binomial are contributing • glTF extension will enable import/export of scenes with universal format textures - Optional LZ and rANS lossless encoders to compress texture data down to JPG-level sizes - Open-source transcoders and lossless decoder implementations

Hardware Texture Original TextureOriginal Universal Universal TextureOriginal Universal Hardware Assets Format Texture Format AssetsTexture Texture Assets Encoder Assets Decoder

Compassed for Platform can select efficient transmission Hardware a natively accelerated format Texture

Desktop – Kepler and later Nintendo Switch Shield Android TV Embedded Tegra / Jetson

NVIDIA shipped Vulkan 1.1 across our product range on day of specification release Driving experimental extensions, and adding Vulkan support to tools https://developer.nvidia.com/Vulkan

Actively working to define and Production-class OpenCL 1.2 on Linux and First to ship production OpenVX 1.0. drive OpenXR to enable and Windows. Active ongoing investment to further Extended OpenVX used in VisionWorks complement VRWorks increase efficiency and performance for Tegra/Jetson https://developer.nvidia.com/vrworks https://developer.nvidia.com/opencl https://developer.nvidia.com/embedded/visionworks

25 How to get Involved! • Any company or organization is welcome to join Khronos! - For a voice and a vote in any of these standards – membership starts at $3,500 • OR request an invite to Vulkan, OpenCL, OpenXR Advisory Panels - No fee, execute Khronos NDA and IP Framework, provide requirements and spec feedback • Bring your feedback to the Vulkan Ecosystem Forum - Or any of the Khronos open source GitHub repositories • Contact Neil Trevett - [email protected] | @neilt3d | www.khronos.org