Introduction to Khronos Group and OpenXR June 2, 2020 Agenda Item Overview about the AREA and 5 mins the Interoperability and Standards Program 10 min Paul Davies, The Boeing Company 10 min Neil Trevett, President Khronos Group 25 mins Brent Insko, Khronos Group OpenXR and Intel 10 mins Questions from Participants SPEAKERS Paul Davies Neil Trevett, NVIDIA Brent Insko, Intel The Boeing Company Khronos Group Khronos Open XR WG 6/5/2020 3 The Comprehensive Ecosystem Enterprises Becoming aware of the benefits of AR through research, piloting or implementing projects Providers Non-commercial Developing services, Universities, public software and hardware research institutes and products/technology for agencies who deliver the AR ecosystem research and push the boundaries of AR technology 4 Interoperability and Standards Program Automotive Aerospace Industrial equipment Industries Smart cities AREA Medical Engineering Development Schema of AR Factory Field Operations Settings User Operation Customer Needs Sales & marketing Field service operation Statements of Engineering review Scenarios Interactive user manuals Pain Points/ Manufacturing assembly (business processes) Inspection Problems that Remote assistance AR can Address Step by step instructions Visual compare (inspection) Use cases Multi-participant collaboration Product status documentation Build, operate/manage, use Requirements/ Integration with existing multi-vendor solutions needs IT for enterprise without special engineering and operational management Solutions (hardware and software) Security & Connectivity Performance Experience Practicality Safety Authoring Apps Toolkits Data Integration Privacy Bluetooth Graphics Field of view Battery App creation Interaction API needs Formats Business systems Sensors CPU Mono/stereo Ergonomics Scripting UX Languages Content PLM Audio Tracking Resolution Wearability UI creation Tracking Compression ERP Video Audio Durability Sensor usage Encryption SLM RFID Touch Environmental Analytics Conversion IoT Program Goals . To increase knowledge about benefits and approaches to achieving interoperability . To advance the development of standards or other approaches to interoperability through a set of AREA policies, procedures and programs 6/5/2020 7 Tactics … to target specific barriers to interoperability and/or promote increased awareness and use of existing (and/or development of new) standards for open, interoperable and safe AR solutions by 1. Develop/share thought leadership content 2. Support standards development 3. Review draft specifications 4. Co-organize/conduct events focusing on interoperability 5. Test/build reference implementations 6/5/2020 8 Role of Standards in AR at Boeing . Future-proofing . Standardization across internal Boeing teams . Supportability . Interoperability . Avoiding reliance on one particular vendor 6/5/2020 9 Open Standards for Augmented and Virtual Reality Neil Trevett Khronos President NVIDIA VP Developer Ecosystems [email protected]| @neilt3d © Khronos® Group 2020 This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2020 - Page 10 What is An Open Standard? An INTEROPERABILITY STANDARD enables two entities to COMMUNICATE E.g. Software <-> Hardware Standards Grow Markets Standards Reduce Costs By reducing consumer confusion and increasing By sharing development between many companies capabilities and usability and driving volume Standards Speed Time to Market Standards Do Not Stifle Innovation With well-proven testing Companies can compete on implementation and interoperability quality, performance, power etc. etc. True OPEN Standards Are not controlled by a single company – but HMD Connectivity Wireless and 5G Verticals and Security by the industry – typically through an SDO Well defined participation, governance and intellectual property frameworks VR Video Formats 3D and XR Hardware XR in the Browser Widespread XR will take a constellation of industry standards This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2020 - Page 11 Khronos Connects Software to Silicon Open interoperability standards to enable software to effectively harness the power of multiprocessors and accelerator silicon 3D graphics, XR, parallel programming, vision acceleration and machine learning Non-profit, member-driven standards-defining industry consortium Open to any interested company All Khronos standards are royalty-free Well-defined IP Framework protects participant’s intellectual property >150 Members ~ 40% US, 30% Europe, 30% Asia This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2020 - Page 12 Khronos Standards for XR Create 3D Assets Download 3D object Vision and sensor processing - and scene data including neural network inferencing for machine learning High-performance, low-latency 3D Graphics Portable interaction with VR/AR sensor, haptic and display devices This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2020 - Page 13 OpenXR is used with a 3D API Application or Engine Display, High-performance, low-latency composition and Cross-platform access to XR 3D rendering and composition* optical correction HMDs and sensors Multiview parameters XR application lifecycle Context priority Input device discovery and events Front buffer rendering Sensor tracking and pose calculation Tiled rendering (beam racing) Frame timing and display composition Variable rate rendering Haptics Control * OpenXR can be used with other 3D APIs such as Direct3D, OpenGL and OpenGL ES This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2020 - Page 14 The Metaverse will be the Web! https://xkcd.com/1367/ This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2020 - Page 15 Bringing XR to the Web The Web will Evolve Native XR Apps into the Metaverse Web XR Apps Lifting OpenXR functionality into the Web stack Native 3D Web 3D Engines Close cooperation Engines between WebXR and OpenXR Khronos provides the foundation for native and Web-based 3D/XR This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2020 - Page 16 OpenXR and Edge Server Applications with 5G Low latency Sensor Data Location–aware Content Requests OpenXR APIs hide the 5G round trip from applications MEC (Multi-access Edge Sensor handling Computing) Server 1. Processes sensor data, including machine learning for environmental lighting, occlusion, scene semantics, object Needed assets Display composition reconstruction and UI loaded to 2. Generates imagery from 3D edge server models, including stereo, foveal Wireless mobile device with rendering, ray-tracing, optics pre- distortion, varifocal processing display and sensors Apps and 3D Assets Generated Augmentations & Scenes This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2020 - Page 17 glTF 2.0 Scene Description Structure .gltf (JSON) Node hierarchy, PBR material textures, cameras .bin .png Geometry: vertices and indices .jpg Animation: key-frames .ktx2 Skins: inverse-bind matrices Textures Mandatory Metallic-Roughness Materials Optional Specular-Glossiness Materials Geometry Texture based PBR materials This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2020 - Page 18 Modo Paint 3D Titania Discover Game Engines VoxEdit Repositories Ashes Creature Web Engines 3D Authoring Tools 3D Builder Oculus Create Experience 3D printing Tools Apps / Engines 3D Asset Format for Efficient VR / AR Authoring Tools Transmission and Processing http://github.khronos.org/glTF-Project-Explorer/ Sony 3D Creator Drive Demand Apps and Engines Users 3D Live Object 3D Scanning Tools Mixed gltfpack Reality Viewer Collada2gltf OBJ2GLTF FBX2glTF Convertors, Optimizers and Loaders Windows Mixed Reality Home glTF Sample Viewer glTF-validator VR / AR Apps and Engines gltf-vscode glTF-asset-generator glTF-Toolkit Validation and Reference Tools Productivity and Social Apps This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2020 - Page 19 3D Commerce - Four Initial Areas of Focus Asset Creation Guidelines Product Configuration For tools and product designers to Universal product configurability create assets with consistent data to be data and guidelines on how to drive used through the 3D Commerce pipeline consistent product display First Goals Helping evolve glTF to meet the needs of 3D Commerce - Next generation PBR - Advanced metadata Industry cooperation to urgently develop guidelines and tools to address priority problem areas Metadata Viewer Validation and Certification Structured metadata definitions and Test models, reference viewer, display analysis examples to consistently carry product tools and capability specifications to guarantee information through the retail pipeline a consistent and accurate end user experience This work is licensed under a Creative Commons Attribution 4.0 International License © The Khronos® Group Inc. 2020 - Page 20 How May Standards Help Your Business? • Seek friction points in your industry caused by lack of agreed communication protocols - Between people, companies, products, subsystems, components … • Find or create a safe space for