Khronos SIGGRAPH Asia Macau Dec16

NNEF New! Neural Net Exchange Format Khronos VR Initiative!

Graphics, Compute and Vision APIs Including Vulkan Next Generation GPU Acceleration SIGGRAPH Asia 2016, Macau

Neil Trevett Vice President Developer Ecosystem, NVIDIA | President, Khronos [email protected] | @neilt3d November 2016

Software

Silicon

Khronos is an International Industry Consortium of over 100 companies creating royalty-free, open standard APIs to enable software to access hardware acceleration for 3D graphics, parallel computing and vision processing

3D for the Web - JavaScript API for 3D Graphics - Transmission of 3D runtime assets New! Khronos VR Initiative!

NNEF

Vision / Camera - Tracking and positioning - Scene Analysis 3D Graphics - Games and Virtual Reality - Augmented Reality - CG Effects Parallel Computation - Product Design - Machine Learning - Safety Critical - Scene Comprehension - 3D scene model construction - Video Transcoding

© Copyright Khronos Group 2016 - Page 3 OpenCL – Low-level Parallel Programing • Low level programming of heterogeneous parallel compute resources - One code tree can be executed on CPUs, GPUs, DSPs and FPGA • OpenCL C language to write kernel programs to execute on any compute device - Platform Layer API - to query, select and initialize compute devices - Runtime API - to build and execute kernels programs on multiple devices • New in OpenCL 2.2 - OpenCL C++ kernel language - a static subset of C++14 - Adaptable and elegant sharable code – great for building libraries - Templates enable meta-programming for highly adaptive software - Lambdas used to implement nested/dynamic parallelism

GPU OpenCL CPU KernelOpenCL Kernel code DSP Runtime API CodeKernelOpenCL compiled for CPU loads and executes CPU CodeKernelOpenCL FPGA CodeKernel devices kernels across devices Devices Code Host

Single Source C++ Programming Full support for features in C++14-based Kernel Language

API and Language Specs Brings C++14-based Kernel Language into core specification OpenCL C++ Kernel Language SPIR-V 1.1 with C++ support Portable Kernel Intermediate Language SYCL 2.2 for single source C++ Support for C++14-based kernel language e.g. constructors/destructors SPIR-V in Core Subgroups into core Subgroup query operations Shared Virtual Memory clCloneKernel On-device dispatch Low-latency device Generic Address Space timer queries 3-component vectors Enhanced Image Support Device partitioning Additional image formats C11 Atomics Separate compilation and linking Multiple hosts and devices Pipes Enhanced image support Buffer region operations Android ICD Built-in kernels / custom devices Enhanced event-driven execution Enhanced DX and OpenGL Interop Additional OpenCL C built-ins Improved OpenGL data/event interop

Dec08 18 months Jun10 18 months Nov11 24 months Nov13 24 months Nov15 7months May16 OpenCL 1.0 OpenCL 1.1 OpenCL 1.2 OpenCL 2.0 OpenCL 2.1 OpenCL 2.2 Specification Specification Specification Specification Specification PROVISIONAL

© Copyright Khronos Group 2016 - Page 5 SPIR-V Transforms the Language Ecosystem • First multi-API, intermediate language for parallel compute and graphics - Natively represents structures in shader and kernel languages - https://www.khronos.org/registry/spir-v/papers/WhitePaper.pdf

Diverse Languages and Frameworks Multiple Developer Advantages Use same front-end compiler for all platforms Standard Ship SPIR-V – not shader source code Tools for Portable Simpler and more reliable drivers analysis and optimization Reduces runtime kernel compilation time Intermediate Representation Hardware runtimes on multiple architectures

Khronos has open sourced ‘glslang’ GLSL to these tools and translators HLSL GLSL SPIR-V compiler

Khronos plans to open source these tools soon OpenCL C OpenCL C++ https://github.com/KhronosGroup/SPIRV-Tools

SPIR-V (Dis)Assembler LLVM to SPIR-V Bi-directional Translator SPIR-V Validator LLVM

Other SPIR-V Intermediate • Khronos defined and controlled Forms cross-API intermediate language • Native support for graphics and parallel constructs • 32-bit Word Stream • Extensible and easily parsed • Retains data object and control IHV Driver ARB_gl_spirv flow information for effective Runtimes extension code generation and translation © Copyright Khronos Group 2016 - Page 7 SYCL for OpenCL • Single-source heterogeneous programming using STANDARD C++ - Use C++ templates and lambda functions for host & device code • Aligns the hardware acceleration of OpenCL with direction of the C++ standard - C++14 with open source C++17 Parallel STL hosted by Khronos

Developer Choice The development of the two specifications are aligned so code can be easily shared between the two approaches

C++ Kernel Language Single-source C++ Low Level Control Programmer Familiarity ‘GPGPU’-style separation of Approach also taken by device-side kernel source C++ AMP and OpenMP code and host code

© Copyright Khronos Group 2016 - Page 8 OpenCV OpenCV Transparent API for OpenCL Kernel Offload • One OpenCL queue per CPU thread • Extensive and widely used open source • CPU threads can share a device vision library - written in optimized C/C++ • OpenCL kernels are executed asynchronously - Free-use BSD license • C++, C, Python and Java interfaces OpenCV Application - Windows, Linux, Mac OS, iOS and Android • Increasingly taking advantage of heterogeneous processing using OpenCL CPU CPU … CPU - OpenCV 3.X Transparent API; Thread Thread Thread single API entry for each function/algorithm - Dynamically loads OpenCL runtime if available; otherwise falls back to CPU code ocl::Queue ocl::Queue … ocl::Queue - Runtime Dispatching; no recompilation! ocl::Device … ocl::Device

ocl::Context

1.0 | May09 1.1 | Jul11 1.2 | Jun12 1.0 | Aug09 1.1 | Aug10 1.2 | May12 2.0 | Dec14 1.0 | May10 1.1 | Feb11 Desktop 1.1 |Mar11 1.2 | Dec12 2.0 | Jul14 2.1 | Jun15

1.0 | May09 1.1 | Jun10 1.2 | May15

1.1 | Aug12 1.2 | Mar16

1.0 | Feb11 1.2 | Sep13 Mobile

1.1 | Nov12 1.2 | Apr14 2.0 | Nov15

1.1 | Apr12 1.2 | Dec14

1.2 | Sep14

1.1 | May13 1.0 | Jan10 Embedded 1.2 | May15 1.2 | Aug15 1.0 | Jul13 FPGA 1.0 | Dec14

Vendor timelines are Dec08 Jun10 Nov11 Nov13 Nov15 first implementation of OpenCL 1.0 OpenCL 1.1 OpenCL 1.2 OpenCL 2.0 OpenCL 2.1 each spec generation Specification Specification Specification Specification Specification © Copyright Khronos Group 2016 - Page 10 OpenCL Roadmap DISCUSSION • Thin, powerful, explicit run-time for control and predictability • Feature sets and dial-able precision for target market agility • Installable tools and three layer ecosystem for flexibility

Applications

Vendor-supplied and Math Language Tool open source middleware Libraries Front-ends Layers

API Definitions

Dial-able types Thin, explicit run-time with rigorous Real-time Pre- Explicit Self-synchronized, Stream and precision memory/execution model. emption and Asynch self-scheduled Processing … QoS scheduling DMA graphs Installable tool & Low-latency, fine-grain pre-emption validation layers and synchronization Features that can be enabled for particular target markets

3D for the Web - JavaScript API for 3D Graphics - Transmission of 3D runtime assets New! Khronos VR Initiative!

NNEF

© Copyright Khronos Group 2016 - Page 12 The Need for a New Generation GPU API • Explicit - Direct GPU control, predictable - no driver surprises • Faster - Reduce CPU overhead and latency, multi-thread scaling • Portable - Efficient on cloud, desktop, console, mobile and embedded

OpenGL has evolved over 25 years and GPUs are increasingly programmable and GPUs will accelerate graphics, compute, vision continues to meet industry needs – but there is compute capable + platforms are becoming and deep learning across diverse platforms: a need for a complementary API approach mobile, memory-unified and multi-core FLEXIBILITY and PORTABILITY are key

Only Windows 10 Only Apple

Cross Platform N

Vulkan is extensible – just like OpenGL - and so is the new only generation API where hardware vendors can deliver innovations to market whenever they need

• Application tells the driver what it is going to do You have to supply a LOT of information! - In enough detail that driver doesn’t have to guess - When the driver needs to know it And, you have to supply it early

• In return, driver promises to do - What the application asks for - When it asks for it - Very quickly Efficient, predictable performance

• No driver magic – no surprises

Potential Performance Gain

‘Half Way New Gen’ Retains Traditional Binding Model Mixes OpenGL ES 3.1/OpenCL 1.2 C++11-based kernel language Bindings to Objective-C and Swift Amount of work to port (missing functionality such as Geometry Shaders) from traditional OpenGL or OpenGL ES

Vulkan Benefits

Resource management in app code: Less driver hitches and surprises Application Single thread per context Simpler drivers: Application Improved efficiency/performance Memory allocation Reduced CPU bottlenecks Thread management Language Front-end Lower latency Synchronization Compilers Increased portability High-level Driver Multi-threaded generation Initially GLSL Abstraction of command buffers Multi-threaded Command Buffers: Context management Command creation can be multi-threaded Memory allocation Multiple CPU cores increase performance Full GLSL compiler SPIR-V pre-compiled shaders Error detection Graphics, compute and DMA queues: Layered GPU Control Thin Driver Work dispatch flexibility Explicit GPU Control Loadable debug and SPIR-V Pre-compiled Shaders: validation layers No front-end compiler in driver GPU GPU Future shading language flexibility Loadable Layers No error handling overhead in Vulkan 1.0 provides access to production code OpenGL ES 3.1 / OpenGL 4.X-class GPU functionality but with increased performance and flexibility

© Copyright Khronos Group 2016 - Page 17 Vulkan Multi-threading Efficiency 1. Multiple threads can construct Command Buffers in parallel Application is responsible for thread management and synch CPU Thread Command Buffer

CPU Thread Command CPU Buffer Thread

CPU Command Thread Command Buffer Queue GPU

CPU Command Thread 2. Command Buffers placed in Command Buffer Queue by separate submission thread

Applications can create graphics, compute CPU and DMA command buffers with a general Thread Command queue model that can be extended to more Buffer heterogeneous processing in the future

Khronos members from all segments of the graphics industry Significant proposals, IP contributions Khronos’ first API agree the need for new and engineering effort from many ‘hard launch’ generation cross-platform GPU API working group members February 2016

Unprecedented level of 18 months Specifications, Conformance Tests, SDKs, Reference Materials, participation from game A high-energy Compiler front-ends, Samples - all open source… engine developers working group effort Multiple Conformant Drivers on multiple OS

Vulkan Working Group Participants

© Copyright Khronos Group 2016 - Page 19 Which Developers Should Use Vulkan? • Vulkan puts more work and responsibility into the application - Not every developer will need or want to make that extra investment • Vulkan provides a choice for developers - For many developers OpenGL and OpenGL ES will remain the most effective API - Vulkan can be used to create new classes of end-user experience

Can your graphics Is your app Yes work creation be CPU bound? parallelized?

You’ll do whatever Yes Yes it takes to squeeze Maximum perf Yes from a system No You put You can a premium on manage your avoiding Yes graphics resource hitches allocations

Applications using game engines Application will automatically benefit from Application uses Vulkan’s enhanced performance Applications utility libraries to can use Vulkan speed directly for development maximum Game Engines flexibility and fully optimized control Utility libraries over Vulkan and layers

Rich Area for Innovation • Many utilities and layers will be in open source • Layers to ease transition from OpenGL • Layers to address intersection of Vulkan, DX12, Metal • Domain specific flexibility • Performance across diverse hardware

Similar ecosystem dynamic as WebGL A widely pervasive, powerful, flexible foundation layer enables diverse middleware tools and libraries

‘WebGL Next’ - Lifts ‘Vulkan Higher-Level’ API to JS - Mandates Portable Shading Language - Language extensions to JavaScript compiled to SPIR-V for Web-friendly compute (aka Intel River Trail)

‘Portability API’ - Defines native API for portable code with simple mapping to underlying APIs - Portable Shading Language - compiled to underlying native IR through SPIR-V

Overlap Analysis

• Desktop Hardware - AMD GCN (production) - Intel Skylake and Broadwell (advanced beta, production coming soon) - NVIDIA GeForce and Quadro boards on Windows and Linux (production) - Kepler, Maxwell, Pascal GPUs • Mobile and Embedded Hardware - Samsung Galaxy S7 - NVIDIA Shield Tablet, Shield Android TV, Jetson TX1 (Linux) - Google Nexus 5X, 6P, Player, Pixel C with Android 7.x (“Nougat”) - Lots more on the way!

Dota 2 on Vulkan port of Source 2 ‘ProtoStar’ demo on Vulkan port of Unreal Engine 4 Talos Principle on Vulkan port of Serious Engine

Vulkan support coming

Vulkan support in V1.8

Developer Preview DOOM on Vulkan port of id Tech 6 © Copyright Khronos Group 2016 - Page 24 Android Daydream VR

Daydream uses Unity and Unreal - accelerated over Vulkan

© Copyright Khronos Group 2016 - Page 25 MoltenVK • MoltenVK is an implementation of Vulkan on iOS & MacOS - Made by Brenwill Workshop - Built over Metal • Vulkan & Metal are both static- state, command-buffer APIs - Small overhead - so MoltenVK has good performance • MoltenVK feature set dependent on Metal - Metal’s focus is on providing a convenient API - MoltenVK is a subset of Vulkan - helps define cross-platform compatibility

https://moltengl.com/moltenvk/ Vulkan application running on Apple iPad

www.khronos.org/vulkan/ Canonical Resources Specifications, Header Files Feature Set Definitions (Windows and Linux - post developer feedback) Quick Reference and Reference Pages Conformance Test Source and Test Process Materials to Build SDKs and Tools Compiler toolchain sources Validation Layer Source Loader Source Layers and Loader documentation (open source resources in github.com/KhronosGroup)

Everything needed to create SDKs for any platform or market

LunarG Windows and Linux Installable SDKs Loader and Validation Layer binaries Tools Layers - source and binaries Samples - source and binaries Windows get started guide

IHV Websites Drivers and Loader Vendor tools and layers

Third Party Websites https://lunarg.com/vulkan-sdk/ Layers, Samples etc.

© Copyright Khronos Group 2016 - Page 27 Resources • Drivers - AMD: http://support.amd.com and search for driver downloads - Intel: (search) - NVIDIA: https://developer.nvidia.com/Vulkan (also other resources)

• Khronos resources under https://github.com/KhronosGroup/ - Vulkan-LoaderAndValidationLayers - Glslang (GLSL to SPIR-V compiler) - SPIRV-Cross and SPIRV-Tools - Vulkan-Hpp Get these!!!

• Other SDKs and Tools - RenderDoc: https://github.com/baldurk/renderdoc - LunarG: https://vulkan.lunarg.com/ - AMD wrapper library: https://github.com/GPUOpen-LibrariesAndSDKs/Anvil

© Copyright Khronos Group 2016 - Page 28 Vulkan Status: Community and Ecosystem • A huge amount of Vulkan activity on GitHub! - https://github.com/KhronosGroup • Please get involved! - All projects under Apache 2.0

Ports Tools Tutorials © Copyright Khronos Group 2016 - Page 29 Vulkan Community • The Vulkan ecosystem is as open as we can make it - Welcome community engagment • NVIDIA Vulkan developer hub - www.khronos.org/vulkan/ - We need examples, tutorials, demos, tools… • Let us help you promote your use of Vulkan! - Got a cool Vulkan-generated demo, app, engine, video? Let us help promote it! - Send mail to [email protected]

https://www.khronos.org/files/vulkan10-reference-guide.pdf

Vulkan Working Group January 2016

DO-178B/C

ISO 26262

Experience and Guidelines New Generation APIs for safety certifiable vision, graphics and compute OpenGL SC 1.0 - 2005 OpenGL SC 2.0 - April 2016 Fixed function graphics subset Shader programmable pipeline subset

OpenGL ES 1.0 - 2003 OpenGL ES 2.0 - 2007 Fixed function graphics Shader programmable pipeline

Safety Critical Safety Critical vision heterogeneous Small driver size processing compute Advanced functionality Graphics and compute

https://www.khronos.org/news/press/advisory-panel-to-create-design-guidelines-for-safety-critical-apis © Copyright Khronos Group 2016 - Page 31 Khronos Roadmap Possibilities in Discussion

SPIR-V Ingestion in OpenVX and OpenGL for programmable node and Vulkan Roadmap Priorities shading language flexibility - Multi-GPU - Virtual Reality (asynch compositing, efficient multi-view rendering, direct screen access) - Cross-API/process resource/event interop - Subgroup instructions (e.g. shader ballot) - Rigorous memory model

Thin and predictable graphics and compute for 1. C++ Shading Language safety critical 2. Single source C++ systems Programming from SYCL 3. OpenCL-class Heterogeneous Compute to Vulkan runtime Khronos members decide how to evolve and mix and match a rich set of APIs and technologies to meet market needs

3D for the Web - JavaScript API for 3D Graphics - Transmission of 3D runtime assets New! Khronos VR Initiative!

NNEF

Vision Frameworks

Neural Net Libraries OpenVX Neural Net Extension

High-level Language-based Acceleration Frameworks

3D Graphics Explicit Kernels

DSP Dedicated Hardware GPU FPGA

© Copyright Khronos Group 2016 - Page 34 OpenVX – Low Power Vision Acceleration • Targeted at vision acceleration in real-time, mobile and embedded platforms - Precisely defined API for production deployment • Higher abstraction than OpenCL for performance portability across diverse architectures - Multi-core CPUs, GPUs, DSPs and DSP arrays, ISPs, Dedicated hardware… • Extends portable vision acceleration to very low power domains - Doesn’t require high-power CPU/GPU Complex or OpenCL precision

Vision Engine Vision Processing X100 Dedicated Efficiency Middleware Hardware Application Vision DSPs X10 GPU Compute Multi-core CPU Power Efficiency Power X1 GPU DSP Computation Flexibility Hardware

© Copyright Khronos Group 2016 - Page 35 OpenVX Graphs • OpenVX developers express a graph of image operations (‘Nodes’) - Nodes can be on any hardware or processor coded in any language • Graphs can execute almost autonomously - Possible to Minimize host interaction during frame-rate graph execution • Graphs are the key to run-time optimization opportunities…

Pyr Camera OpenVX Nodes t Rendering Input Output

RGB YUV Gray Array of Frame Color Frame Channel Frame Image Optical Keypoints Harris Conversion Extract Pyramid Flow Track

Array of Features OpenVX Graph Ftrt-1

Feature Extraction Example Graph

Graph Memory Kernel Data Scheduling Management Merge Tiling

Execute a sub- Split the graph Reuse Replace a sub- graph at tile execution across pre-allocated graph with a granularity instead the whole system: memory for single faster node of image CPU / GPU / multiple granularity dedicated HW intermediate data

Faster execution Less allocation overhead, Better memory Better use of or lower power more memory for locality, less kernel data cache and consumption other applications launch overhead local memory

10 Relative Performance 9 8.7

8 OpenCV (GPU accelerated) 7 OpenVX (GPU accelerated) NVIDIA 6 implementation experience. Geometric mean of 5 >2200 primitives, grouped into each 4 categories, 2.9 running at different 3 image sizes and 2.5 parameter settings 2 1.5 1.1 1

0 Arithmetic Analysis Filter Geometric Overall

Implementers may use OpenCL or Compute Shaders to implement OpenVX nodes on programmable processors And then developers can use OpenVX to enable a developer to easily connect those nodes into a graph AMD OpenVX - Open source, highly optimized Application for x86 CPU and OpenCL for GPU - “Graph Optimizer” looks at entire processing pipeline and removes/replaces/merges functions to improve C/C++ performance and bandwidth - Scripting for rapid prototyping, without re-compiling, at Programmable Vision Dedicated Vision production performance levels Processors Hardware http://gpuopen.com/compute-product/amd-openvx/

OpenVX enables the graph to be extended to include hardware architectures that don’t support programmable APIs

The OpenVX graph enables implementers to optimize execution across diverse hardware architectures an drive to lower power implementations

© Copyright Khronos Group 2016 - Page 39 OpenVX 1.0 Shipping, OpenVX 1.1 Released! • Multiple OpenVX 1.0 Implementations shipping – spec in October 2014 - Open source sample implementation and conformance tests available • OpenVX 1.1 Specification released 2nd May 2016 - Expands node functionality AND enhances graph framework • OpenVX is EXTENSIBLE - Implementers can add their own nodes at any time to meet customer and market needs

= shipping implementations

© Copyright Khronos Group 2016 - Page 40 OpenVX Neural Net Extension • Convolution Neural Network topologies can be represented as OpenVX graphs - Layers are represented as OpenVX nodes - Layers connected by multi-dimensional tensors objects - Layer types include convolution, activation, pooling, fully-connected, soft-max - CNN nodes can be mixed with traditional vision nodes • Import/Export Extension - Efficient handling of network Weights/Biases or complete networks • The specification is provisional - Welcome feedback from the deep learning community

Vision Node Native Vision Vision Downstream Camera Application Node Node Control CNN Nodes Processing

An OpenVX graph mixing CNN nodes with traditional vision nodes

NN Authoring Framework 1 Inference Engine 1

NN Authoring Framework 2 Inference Engine 2

NN Authoring Framework 3 Inference Engine 3

NN Authoring Framework 1 Inference Engine 1 Neural Net NN Authoring Framework 2 Exchange Inference Engine 2 Format NN Authoring Framework 3 Inference Engine 3

NNEF encapsulates neural network structure, data formats, commonly used operations (such as convolution, pooling, normalization, etc.) and formal network semantics

NNEF 1.0 is currently being defined OpenVX will import NNEF files

Growing Camera Diversity Diverse Vision Processors Sensor Proliferation

Flexible sensor and camera Use efficient acceleration to Combine vision output control to GENERATE PROCESS with other sensor data an image stream the image stream on device

3D for the Web - JavaScript API for 3D Graphics - Transmission of 3D runtime assets New! Khronos VR Initiative!

Content downloaded Content from the Web JavaScript, HTML, CSS, ... Low-level WebGL API provide a Middleware provides accessibility powerful foundation for a rich for non-expert programmers JavaScript middleware JavaScript Middleware E.g. three.js library ecosystem

CSS Reliable WebGL Browser provides WebGL relies on work by 3D engine alongside other HTML5 both GPU and technologies - no plug-in required JavaScript HTML5 Browser Vendors -> Khronos has the right membership OS Provided Drivers to enable that WebGL uses OpenGL ES 2.0 or cooperation Angle for OpenGL ES 2.0 over DX9

© Copyright Khronos Group 2016 - Page 45 WebGL Timeline Tessellation and geometry shaders 32-bit integers and floats ASTC Texture Compression Programmable NPOT, 3D/depth textures Floating point render targets Vertex and Texture arrays Compute Shaders Debug and robustness for security Fixed function fragment shaders Multiple Render Targets Pipeline

Epic’s Rivalry demo using full Unreal Engine 4on mobile https://www.youtube.com/watch?v=jRr-G95GdaM Driver Silicon Silicon Driver Silicon Update Update Update Update Update 2003 2004 2007 2012 2014 2015 1.0 1.1 2.0 3.0 3.1 3.2

http://caniuse.com/#feat=webgl

© Copyright Khronos Group 2016 - Page 48 WebGL Applications Becoming Mainstream • Artists showcase their work on Sketchfab, which just passed the 1 million model mark • The New York Times publishes scientific visualizations using Three.js • Create amazing, fast, and interactive mathematical visualizations using MathBox

Epic Citadel - WebGL HTML 5 Benchmark (Firefox 22) https://www.youtube.com/watch?v=l9KRBuVBjVo © Copyright Khronos Group 2016 - Page 49 Google Maps • All rendering (2D and 3D) in Google Maps uses WebGL

Audio Video Images 3D

MP3 H.264 JPEG

?! ...until now.

.gltf JSON describes node hierarchy, materials, cameras

.bin .glsl .png Geometry: vertices and Shaders .jpg indices … Animation: key-frames Textures Skins: inverse-bind matrices

NORAD’s Santa Tracker

2012 – 2014 Dec 2015 2016 Development Standardization Deployment

Design iteration V 1.0 Spec V 1.1 Spec Update Multiple implementations Ratified and Released Syntax Validator

Original motivation: glTF 1.1 makes the specificaton more testable standardized way to deliver and includes some simplifications e.g. animation 3D into WebGL applications Will finalize/ratify V 1.1 when specification is proven by loader implementations

Publicly Stated Support for glTF

Tools model/gltf+json MIME type Drag and Drop FBX -> glTF Blender DIRECT export Export (coming soon) Approved by IANA! http://gltf.autodesk.io/

Autodesk FBX -> glTF AssImp Translators Validator OBJ2GLTF Convert | Optimize Validate glTF Pipeline COLLADA2GLTF Cesium converter glTF 1.1 Spec in Review and glTF Validator in open source! http://github.khronos.org/glTF-Validator/

Drag and drop COLLADA -> glTF http://cesiumjs.org/convertmodel.html

SDO Liaisons Contributions can be licensed under Liaison Agreements

Royalty-free specifications Vendor Extensions Anyone can create extensions for their own use Khronos glTF Working Group - Corporate, Academic, Individual Members - Detailed contributions and spec drafting - Decision making quorum - Members grant protection for Ratified specs through the Khronos IP framework Anyone can participate in the GitHub discussions around Open Meetings glTF design Detailed discussions and requirements

glTF GitHub Community IP Firewall https://github.com/KhronosGroup/glTF Must not adopt ‘high-risk’ Feedback on specifications IP Contributions into the Khronos Open source tools – open community specifications without license coverage

© Copyright Khronos Group 2016 - Page 57 ‘glTF Next Generation’ NVIDIA MDL Physically Based Rendering • Prioritized Features - Targeting Draft Specification by Spring 2017 • Remove API dependencies from core for portability - https://github.com/KhronosGroup/glTF/issues/733 - Extensions for API specifics: WebGL 2.0, Vulkan, - DX12, Metal, GLSL, HLSL, SPIR-V, Metal C++ - Compressed texture formats, compute? • Portable Physically Based Rendering (PBR) - https://github.com/KhronosGroup/glTF/issues/696 - Must be scalable from simple/fast to high-end/photorealistic renderers - Fraunhofer, NVIDIA MDL? • Improved Efficiency - https://github.com/KhronosGroup/glTF/issues/507 - BSPs, arrays, bounding boxes, spatial constructs, texture channels/usage, syntax cleanup • Animation System Enhancements - Morph targets, keyframes and interpolation types, compression • Mesh Streaming and Compression - Progressive mesh, quantization, LOD, TFAN

Web VR Applications

Increasing VR capability being built into GPU driver stacks – if/how Web apps access general GPU expose through WebGL? acceleration? Physics, Direct display access, audio, vision.. Javascript multi-projection, time- Language extensions? ‘WebKCam’? warp?, space-warp etc. VR Displays Developer APIs System APIs ? Native GPU APIs are GPGPU and Portability API Compute Shaders Camera fragmenting – a new over DX12, Metal Control generation WebGL will and Vulkan API? Cameras be the intersection of the three main Camera diversity about to ‘explicit’ industry APIs? Low power vision explode – multiple sensors, acceleration – first stereo, depth sensors – how appearing on avoid app fragmentation? GPU embedded systems

VR Application Game Engine VR Application Portable VR Applications and Engines

Apps and engines use standardized APIs Apps and engines Khronos VR Standard APIs must port VR Platform Runtime to multiple VR APIs Device Vendor Driver

Devices can integrate Device Vendors can to a single provide custom standardized driver runtimes and API interface extensions

Device vendors VR VR only have access Device Device to some VR VR VR applications Device Device Device Diverse Virtual Reality Devices

Industry growth limited by costs of Standardized hardware/software interfaces application porting and device integration encourage industry diversity and innovation

© Copyright Khronos Group 2016 - Page 61 Valve has Contributed OpenVR • One of multiple expected design contributions Application Distribution • Application Interface Application - Query tracking data and other hardware state - Receive input events Loader library - Render to HMD - Access other features: Standard Application Interface - Application list, Game-independent overlays - Front-facing camera access, Chaperone/guardian access Runtime Distribution - Drive output events to hardware (haptics) • Driver Interface Display Output Tracking - Enumerate devices and provide data for each: - Tracking state, Display output details Input Other - Input state, Camera presence and state • Runtime Distribution Standard Driver Interface - Manage multiple drivers Driver - Transfer state from driver to application - Compositor functions: Hardware-Specific Driver - Distortion correction, Display correction (Mura, Ghosting) - Adding chaperone/overlays over apps, Handling missed frames - Provide cross-application UX © Copyright Khronos Group 2016 - Page 62 Call for Participation! • Why at Khronos? - All VR Platform vendors are already members - Concentration of GPU and vision expertise - Proven process • Now is the perfect time to get involved! - Fast-paced work on detailed proposals and designs will start after the current exploratory phase to define the standard’s scope and key objectives - The work has just started – this is your chance to influence its direction! • Who is involved in this initiative? - Here are just some of the companies helping to get this initiative under way - Many more companies are already getting involved. Your company can be one of them!

WebVR - experimental JavaScript API to access Virtual Reality devices, such as the Oculus Rift, HTC Vive, Samsung Gear VR, or Google Cardboard, in your browser https://w3c.github.io/webvr/ Web Technologies WebVR drives all rendering through WebGL

VR Initiative Silicon Acceleration

The Khronos VR Initiative can help boost the development of WebVR by solving native level VR platforms fragmentation

Chapters Chair: Tomasz Bednarz Administrative Support: Angela Cheng

Email: [email protected] Chapter page link: https://chapters.khronos.org/ Chapter meetup.com link: http://www.meetup.com/pro/KhronosChapters/

Tomasz – Chapter Chair + Angela – Admin Support + Chapter Leaders Developer and Education Community

Khronos Chapters are vital to enable grassroots engagement with the developer and Education communities about cool Khronos technologies! And they are fun! 

• Khronos Chapter is: - Local groups of Khronos API users who meet virtually or in person - Expand understanding of Khronos APIs and technical achievements - Inspire others through use of Khronos technologies - A Khronos Chapter is a community, - A platform for local developers to share and communicate - Organized around a geographical region, such as city or state supporting the local developers community - An opportunity to meet Khronos member companies and get the latest updates - A chance to socialize with other developers - See: https://chapters.khronos.org/

• Khronos Chapter is: - Organized and run by one or more Chapters Leaders - Chapter events are held under the name of Khronos and get support from us - Chapters have meetups, demos, talks, competitions, workshops, cross-cities events - Khronos has an elected Chapter Chair, who is a Khronos member and liaison between Chapter Leaders and Khronos membership and management - Contact Chapter Chair under [email protected]

© Copyright Khronos Group 2016 - Page 68 Benefits • Khronos provides Chapters with the following benefits: - Budget - Web presence - Meetup Account - Upcoming Chapter meeting list - Social media promotion - Promotion - Event management assistance - Speakers program - Khronos trademarks - Event invitations - Chapter Leader email list - Materials - Preferred access

• Visit the Khronos Chapter site: http://chapters.khronos.org/locations • If there’s a Chapter in your area: - Contact the Chapter Leader to get involved - Join the Chapter group on Meetup.com - Find out and stop by the next local Chapter event! • If there’s no Chapter in your area: - It’s time to set up a local chapter! - Visit the Chapter site to learn more about becoming a Chapter Leader - Complete a Khronos Chapter Application - We’ll review your information and help you set up a local chapter - Provide a Review and Forecast report (RAF) - You will be contacted by Chapter Chair if your Chapter is approved • For any questions about Chapter, please contact Chapter Chair: [email protected]

• Chapter Leader: Tomasz Bednarz • Contact: [email protected] • Signed in members: 61 • Recent meetups: - High-Performance and Immersive Visualisation - OpenGL for Big Data Analytics - High-Performance Visualisation using OpenGL/WebGL - Pub meetings • Visualisation Matters 2016 http://visualisation.matters.today • Collocating with other meetups http://www.meetup.com/Khronos-Brisbane-Chapter/ © Copyright Khronos Group 2016 - Page 71 Milano Khronos Chapter, Italy

• Chapter Leader: Giorgio Mazzucchelli • Contact: [email protected] • Started: June 2015 • Members: +240

• Recent Meetup - Physics simulation libraries for WebGL - AngularJS for 3D app development - X3DOM Instant 3D, the HTML way - Firefox DevTools + Mozilla - Augmented Reality for art in public spaces

Understand early Ship products that conform to industry requirements! international standards! Influence how Access draft specs to standards evolve! build products faster!

Khronos is proven to RAPIDLY generate hardware API standards that create significant market opportunities

Any company or organization is welcome to join Khronos for a voice and a vote in any of its standards www.khronos.org