DOCSLIB.ORG

Khronos China Update Oct18

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Khronos China Update Oct18 Connecting Software to Silicon China Outreach, October 2018 Neil Trevett | Khronos President NVIDIA | VP Developer Ecosystems Copyright © Khronos® Group Inc. 2018 - Page 1 Khronos Mission Asian Members Software Silicon Khronos is an International Industry Consortium creating royalty-free, open standards to enable software to access hardware acceleration for 3D Graphics, Virtual and Augmented Reality, Parallel Computing, Neural Networks and Vision Processing Copyright © Khronos® Group Inc. 2018 - Page 2 What is An Open Standard? An INTEROPERABILITY STANDARD enables two entities to COMMUNICATE. E.g. Software <-> Hardware Bad Standards Good Standards - Overprescribes implementation details - Prescribes ONLY interoperability - Forces lowest common denominator - Enables implementation diversity - Stifles innovation – Encourages innovation -> Commoditization -> Differentiation A Truly OPEN standard Is not controlled by a single company – but by the whole industry Is freely available to use by any company without royalty payments Has a well defined IP Framework to protect standard AND member’s IP Standards Grow Markets… … Reduce Costs … … Accelerate Time to Market By reducing consumer confusion and By sharing development between With well-proven functionality, increasing capabilities and usability many companies and driving volume testing and interoperability Copyright © Khronos® Group Inc. 2018 - Page 3 Khronos Standards Download 3D object and scene data Vision and sensor processing - including neural network inferencing for machine learning High-performance, low-latency 3D Graphics Portable interaction with VR/AR sensor, haptic and display devices Copyright © Khronos® Group Inc. 2018 - Page 4 Vulkan and New Generation GPU APIs Modern architecture | Low overhead| Multi-thread friendly EXPLICIT GPU access for EFFICIENT, LOW-LATENCY, PREDICTABLE performance Vulkan Porting Tools Non-proprietary, royalty-free open standard ‘By the industry for the industry’ Portable across multiple platforms - desktop and mobile Copyright © Khronos® Group Inc. 2018 - Page 5 What is an Explicit API? Explicit API • Provides information at the right time Timeline(s) state definitions - To prepare for efficient execution API Object • Better scheduling control over CPU and GPU generation - Split multi-threaded work creation from work submission api object - Explicit synchronization primitives usage • Predictable performance costs - Creating pipelines, allocating memory … Legacy API state context • No driver magic on-the-clock manipulation - Removes guesswork and late decision-making • Simpler drivers Implicit Internal - The application has the most complete Validation & knowledge for holistic optimization Objects Copyright © Khronos® Group Inc. 2018 - Page 6 Vulkan Explicit GPU Control Vulkan = high performance and low latency 3D and GPU Compute Ideal for VR/AR applications Resource management offloaded to app: low-overhead, low-latency Complex drivers Application driver cause overhead Single thread per context and inconsistent Application Consistent behavior: behavior across Memory allocation Multiple Front-end no ‘fighting with driver vendors Thread management heuristics’ Synchronization Compilers High-level Driver GLSL, HLSL etc. Always active Multi-threaded generation Validation and debug layers of command buffers error handling Abstraction loaded only when needed Layered GPU Control Full GLSL Context management SPIR-V SPIR-V intermediate preprocessor and Memory allocation pre-compiled shaders language: shading language compiler in Full GLSL compiler flexibility Error detection driver Thin Driver Loadable debug and Explicit GPU Control validation layers Multi-threaded command OpenGL vs. creation. Multiple graphics, OpenGL ES command and DMA queues GPU GPU Unified API across all platforms with feature set flexibility Vulkan 1.0 provides access to OpenGL ES 3.1 / OpenGL 4.X-class GPU functionality but with increased performance and flexibility Copyright © Khronos® Group Inc. 2018 - Page 7 Vulkan Multi-threading Efficiency 1. Multiple threads can construct Command Buffers in parallel Application is responsible for thread management and synch CPU Thread Command Buffer CPU Thread Command CPU Buffer Thread CPU Command Thread Command Buffer Queue GPU CPU Command Thread 2. Command Buffers placed in Command Buffer Queue by separate submission thread CPU Thread Command Buffer Copyright © Khronos® Group Inc. 2018 - Page 8 Vulkan Tools Architecture • Layered design for cross-vendor tools innovation and flexibility - IHVs plug into a common, extensible architecture for code validation, debugging and profiling during development without impacting production performance • Khronos Open Source Loader enables use of tools layers during debug - Finds and loads drivers, dispatches API calls to correct driver and layers Production Path Interactive (Performance) Debug Layers can be Debugger Vulkan-based Title installed during Development Validation and Sim Layers Vulkan’s Common Loader Debug Layers IHV’s Installable Client Debug information via Driver standardized API calls Copyright © Khronos® Group Inc. 2018 - Page 9 Vulkan – Now Available Everywhere VR-Related Features NOW Multi-GPU support Multiview Rendering Context priority Front buffer rendering IN DISCUSSION Variable Rate Rendering Tiled rendering (beam racing) Copyright © Khronos® Group Inc. 2018 - Page 10 Vulkan 1.1 Launch and Ongoing Momentum Strengthening the Ecosystem Improved developer tools (SDK, validation/debug layers) More rigorous conformance testing Building Vulkan’s Future Shader toolchain improvements (size, speed, robustness) Deliver complete ecosystem – not just specs Shading language flexibility – HLSL and OpenCL C support Listen and prioritize developer needs Vulkan Public Ecosystem Forum Drive GPU technology Released Vulkan 1.1 Extensions KHR_draw_indirect_count Vulkan 1.0 Extensions Source draw count parameter from a buffer in GPU-writable Maintenance updates plus additional functionality memory for greater flexibility for GPU-generated work Explicit Building Blocks for VR: KHR_8bit_storage e.g. multiview 8-bit types in uniform and storage buffers for improved compute Explicit Building Blocks for support in apps such as inferencing and vision Homogeneous Multi-GPU EXT_descriptor_indexing Enhanced Windows System Integration Dynamically non-uniform (aka bindless) resource access Increased Shader Flexibility: Required by some modern game engine architectures February 2016 16 bit storage, Variable Pointers Enhanced Cross-Process and Discussions in Flight Vulkan 1.0 March 2018 Cross-API Sharing Vulkan 1.1 Reduced precision arithmetic types in shaders Detailed driver property queries Integration of 1.0 Extensions. Variable-resolution rendering New Technology into Core e.g. Cross-vendor performance counter queries Subgroup Operations Memory residency management Depth/stencil resolve Ray tracing Widening Platform Support Video Pervasive GPU vendor native driver availability New synchronization primitives Open source drivers – ANV (Intel), AMDVLK/RADV (AMD) Port Vulkan apps to macOS/iOS and DX12 Copyright © Khronos® Group Inc. 2018 - Page 11 New Functionality in Vulkan 1.1 • Protected Content - Restrict access or copying from resources used for rendering and display - Secure playback and display of protected multimedia content • Subgroup Operations - Efficient mechanisms that enable parallel shader invocations to communicate - Wide variety of parallel computation models supported Example Subgroup Operations A subgroup is a set of invocations (tasks) running on a GPU Compute Unit (Note many GPUs typically support subgroup sizes of 32/64 invocations) 5 2 1 6 7 9 3 4 3 1 3 2 2 3 Active Invocations + X Inactive Invocations 5 5 5 5 3 9 9 7 9 9 9 9 6 6 Broadcast Shuffle Add Clustered Multiply Copyright © Khronos® Group Inc. 2018 - Page 12 Content is shipping on desktop… Vulkan-only AAA Titles on PC Dota 2 on PC and macOS AAA titles on Linux Copyright © Khronos® Group Inc. 2018 - Page 13 …and Mobile Plus…. Lineage 2 Revolution Heroes of Incredible Tales Dream League Soccer… Copyright © Khronos® Group Inc. 2018 - Page 14 Fortnite on Android! Fortnite on Android uses Vulkan on select phones for optimal performance, including the best-performing Samsung – the Galaxy Note9 Copyright © Khronos® Group Inc. 2018 - Page 15 Vulkan Developer Activity – SDK and GitHub LunarG Vulkan SDK Download rate increases every year since launch http://vulkan.lunarg.com SIGGRAPH 2016 Vulkan Related GitHub Repos SIGGRAPH 2018 2500 2000 1500 SIGGRAPH 2017 1000 500 0 SIGGRAPH GDC SIGGRAPH GDC SIGGRAPH 2016 2017 2017 2018 2018 Copyright © Khronos® Group Inc. 2018 - Page 16 NVIDIA Vulkan Raytracing Vendor Extension • Vulkan Raytracing Extension to access RTX Raytracing - Running on Turing-based boards • Nsight Graphics with increased Vulkan support - Including Raytracing extension Copyright © Khronos® Group Inc. 2018 - Page 17 OpenGL and OpenGL ES January 2018 June 2018 OpenGL 4.6 conformance test suite OpenGL ES CTS 3.2.5.0 released in open source released in open source Intel and NVIDIA released conformant Raises the quality bar for OpenGL OpenGL 4.6 drivers 3.2 implementations OpenGL ES still the most prevalent April 2018 3D API (billions of units!) More conformant products added OpenGL 4.6.0.1 CTS bugfix update OpenGL ES 3.2 adoption increasing released in April Working Group Meetings Merged under one Chairperson for Improved Efficiency 13.9% 30.6% GLSL and ESSL specs merged and migrated from LibreOffice to AsciiDoctor to improve
Load more

Recommended publications
  • Progressive Rendering of Massive Point Clouds in Webgl 2.0 Compute
    Progressive Rendering of Massive Point Clouds in WebGL 2.0 Compute BACHELORARBEIT zur Erlangung des akademischen Grades Bachelor of Science im Rahmen des Studiums Software & Information Engineering eingereicht von Wolfgang Rumpler Matrikelnummer 01526299 an der Fakultät für Informatik der Technischen Universität Wien Betreuung: Associate Prof. Dipl.-Ing. Dipl.-Ing. Dr.techn. Michael Wimmer Mitwirkung: Dipl.-Ing. Markus Schütz Wien, 10. Oktober 2019 Wolfgang Rumpler Michael Wimmer Technische Universität Wien A-1040 Wien Karlsplatz 13 Tel. +43-1-58801-0 www.tuwien.ac.at Progressive Rendering of Massive Point Clouds in WebGL 2.0 Compute BACHELOR’S THESIS submitted in partial fulfillment of the requirements for the degree of Bachelor of Science in Software & Information Engineering by Wolfgang Rumpler Registration Number 01526299 to the Faculty of Informatics at the TU Wien Advisor: Associate Prof. Dipl.-Ing. Dipl.-Ing. Dr.techn. Michael Wimmer Assistance: Dipl.-Ing. Markus Schütz Vienna, 10th October, 2019 Wolfgang Rumpler Michael Wimmer Technische Universität Wien A-1040 Wien Karlsplatz 13 Tel. +43-1-58801-0 www.tuwien.ac.at Erklärung zur Verfassung der Arbeit Wolfgang Rumpler Hiermit erkläre ich, dass ich diese Arbeit selbständig verfasst habe, dass ich die verwen- deten Quellen und Hilfsmittel vollständig angegeben habe und dass ich die Stellen der Arbeit – einschließlich Tabellen, Karten und Abbildungen –, die anderen Werken oder dem Internet im Wortlaut oder dem Sinn nach entnommen sind, auf jeden Fall unter Angabe der Quelle als Entlehnung kenntlich gemacht habe. Wien, 10. Oktober 2019 Wolfgang Rumpler v Danksagung An dieser Stelle möchte ich allen Personen danken die mich bei dem Abschluss dieses Studienabschnitts unterstützt haben.
    [Show full text]
  • Inclusive XR Roadmap Possible Next Steps
    Inclusive XR Roadmap Possible next steps ● Research ○ Where? Industry? Acadameia? EU project? RQTF? ● Prototyping & experimentation ○ Collaboration / open source opportunities ● (Pre-) standardization PRIORITIES? Immersive Web Architecture Hooks Immersive Web Application (Best Practices) Engine / Framework (Best Practices, prototypes, patches) Web browser (APIs, Best Practices) XR Platform Operating System Towards standardizing solutions in W3C ● W3C standardizes technologies through its Recommendation process ● Working Groups are responsible for standardization - for W3C Member organizations and Invited Experts ● Before starting standardization efforts, the preference is to incubate the ideas ● W3C has an incubation program open for free to any one - Community Groups Existing Standardization Landscape (W3C) ● Immersive Web Working Group ○ WebXR: Core, Gamepad, Augmented Reality modules ● ARIA Working Group ○ ARIA ● Accessible Platforms Architecture Working Group ○ RQTF ○ XR Accessibility User Requirements (Note) ○ Personalization Task Force ● Timed Text Working Group ○ TTML, IMSC ○ WebVTT ● Audio Working Group ○ Web Audio API ● Accessibility Guidelines Working Group ○ WCAG 2.x, Silver Existing Incubation Landscape (W3C) ● Immersive Web Community Group ○ Potential host for any new proposals related to Immersive Web ○ Ongoing discussion on Define and query properties of "things" #54 ● Immersive Captions Community Group ○ best practices for captions in Immersive Media ● Web Platform Incubator Community Group (WICG) ○ Accessibility Object
    [Show full text]
  • Rethinking the Usage and Experience of Clustering Markers in Web Mapping
    Rethinking the Usage and Experience of Clustering Markers in Web Mapping Lo¨ıcF ¨urhoff1 1University of Applied Sciences Western Switzerland (HES-SO), School of Management and Engineering Vaud (HEIG-VD), Media Engineering Institute (MEI), Yverdon-les-Bains, Switzerland Corresponding author: Lo¨ıc Furhoff¨ 1 Email address: [email protected] ABSTRACT Although the notion of ‘too many markers’ have been mentioned in several research, in practice, displaying hundreds of Points of Interests (POI) on a web map in two dimensions with an acceptable usability remains a real challenge. Web practitioners often make an excessive use of clustering aggregation to overcome performance bottlenecks without successfully resolving issues of perceived performance. This paper tries to bring a broad awareness by identifying sample issues which describe a general reality of clustering, and provide a pragmatic survey of potential technologies optimisations. At the end, we discuss the usage of technologies and the lack of documented client-server workflows, along with the need to enlarge our vision of the various clutter reduction methods. INTRODUCTION Assuredly, Big Data is the key word for the 21st century. Few people realise that a substantial part of these massive quantities of data are categorised as geospatial (Lee & Kang, 2015). Nowadays, satellite- based radionavigation systems (GPS, GLONASS, Galileo, etc.) are omnipresent on all types of new digital devices. Trends like GPS Drawing, location-based game (e.g. Pokemon´ Go and Harry Potter Wizard Unite), Recreational Drone, or Snapchat ‘GeoFilter’ Feature have appeared in the last five years – highlighting each time a new way to consume location. Moreover, scientists and companies have built outstanding web frameworks and tools like KeplerGL to exploit geospatial data.
    [Show full text]
  • Tensorflow.Js: Machine Learning for the Web and Beyond Via Frameworks Like Electron
    TENSORFLOW.JS:MACHINE LEARNING FOR THE WEB AND BEYOND Daniel Smilkov * 1 Nikhil Thorat * 1 Yannick Assogba 1 Ann Yuan 1 Nick Kreeger 1 Ping Yu 1 Kangyi Zhang 1 Shanqing Cai 1 Eric Nielsen 1 David Soergel 1 Stan Bileschi 1 Michael Terry 1 Charles Nicholson 1 Sandeep N. Gupta 1 Sarah Sirajuddin 1 D. Sculley 1 Rajat Monga 1 Greg Corrado 1 Fernanda B. Viegas´ 1 Martin Wattenberg 1 ABSTRACT TensorFlow.js is a library for building and executing machine learning algorithms in JavaScript. TensorFlow.js models run in a web browser and in the Node.js environment. The library is part of the TensorFlow ecosystem, providing a set of APIs that are compatible with those in Python, allowing models to be ported between the Python and JavaScript ecosystems. TensorFlow.js has empowered a new set of developers from the extensive JavaScript community to build and deploy machine learning models and enabled new classes of on-device computation. This paper describes the design, API, and implementation of TensorFlow.js, and highlights some of the impactful use cases. 1 INTRODUCTION numeric computation capabilities to JS. While several open source JS platforms for ML have appeared, to our knowl- Machine learning (ML) has become an important tool in edge TensorFlow.js is the first to enable integrated training software systems, enhancing existing applications and en- and inference on the GPU from the browser, and offer full abling entirely new ones. However, the available software Node.js integration for server-side deployment. We have at- platforms for ML reflect the academic and industrial roots tempted to ensure that TensorFlow.js meets a high standard of the technology.
    [Show full text]
  • Performance and Ease of Use in 3D on the Web
    "$ # % ! #"% #" !""#"% ! $ This bachelor thesis is submitted to the Faculty of Computing at Blekinge Institute of Technology in partial fulfilment of the requirements for a bachelor’s degree in Software Engineering. The thesis is equivalent to 10 weeks of full-time studies. Contact Information Author: Julia Johansson E-mail: [email protected] University advisor: Emil Folino Department of Computer Science Faculty of Computing Internet : www.bth.se Blekinge Institute of Technology Phone : +46 455 38 50 00 SE-371 79 Karlskrona Sweden Fax : +46 455 38 50 57 2 Abstract 3D JavaScript frameworks are used for creating interactive 3D in web applications. There are two prominent frameworks: Babylon.js and Three.js. They are often compared to one another where performance and ease of use are mentioned as important factors. A web developer could face choosing a 3D framework. This thesis aims to aid in that process. It investigates how Babylon.js compares to Three.js when it comes to performance such as memory use and frames per second for the end-user consuming a web app. It also investigates how they compare when it comes to ease of use for the developer in terms of initial learnability. For the performance part, a web app with a minimalistic one box animation was created with each framework and deployed on Netlify. The web app was accessed with a browser’s developer tools to record memory and frames per second. For the ease of use part, observation sessions were made with nine developers who were tasked with getting a basic 3D model up and running locally with each framework with the official documentation as a starting point, one hour limit respectively.
    [Show full text]
  • Interactive Computer Vision Through the Web
    En vue de l'obtention du DOCTORAT DE L'UNIVERSITÉ DE TOULOUSE Délivré par : Institut National Polytechnique de Toulouse (Toulouse INP) Discipline ou spécialité : Informatique et Télécommunication Présentée et soutenue par : M. MATTHIEU PIZENBERG le vendredi 28 février 2020 Titre : Interactive Computer Vision through the Web Ecole doctorale : Mathématiques, Informatique, Télécommunications de Toulouse (MITT) Unité de recherche : Institut de Recherche en Informatique de Toulouse ( IRIT) Directeur(s) de Thèse : M. VINCENT CHARVILLAT M. AXEL CARLIER Rapporteurs : M. MATHIAS LUX, ALPEN ADRIA UNIVERSITAT Mme VERONIQUE EGLIN, INSA LYON Membre(s) du jury : Mme GÉRALDINE MORIN, TOULOUSE INP, Président M. AXEL CARLIER, TOULOUSE INP, Membre M. CHRISTOPHE DEHAIS, ENTREPRISE FITTINGBOX, Membre M. OGE MARQUES, FLORIDA ATLANTIC UNIVERSITY, Membre M. VINCENT CHARVILLAT, TOULOUSE INP, Membre ii Acknowledgments First I’d like to thank my advisors Vincent and Axel without whom that PhD would not have been possible. I would also like to thank Véronique and Mathias for reviewing this manuscript, as well as the other members of the jury, Oge, Géraldine and Christophe for your attention, remarks and interesting discussions during the defense. Again, a special thank you Axel for all that you’ve done, throughout this long period and even before it started. I haven’t been on the easiest path toward completion of this PhD but you’ve always been there to help me continue being motivated and that’s what mattered most! Wanting to begin a PhD certainly isn’t a one-time moment, but for me, the feeling probably started during my M1 internship. I was working in the VORTEX research team (now REVA) on a project with Yvain and Jean-Denis and it was great! Yet “I don’t think so” was more or less what I kept answering to my teachers when they would ask if I wished to start a PhD at that time.
    [Show full text]
  • Webgpu: a Scalable Online Development Platform for GPU Programming Courses
    WebGPU: A Scalable Online Development Platform for GPU Programming Courses Abdul Dakkak Carl Pearson and Wen-mei Hwu Department of Computer Science Department of Electrical and Computer Engineering University of Illinois at Urbana-Champaign University of Illinois at Urbana-Champaign Urbana, USA Urbana, USA [email protected] fpearson, [email protected] Abstract—The popularity of computer science classes offered material that explains the concepts and programming tech- through Massive Open On-line Courses (MOOCs) creates both niques for mapping algorithms into GPU programs that opportunities and challenges. Programming-based classes need architecture and tools available. Similar materials was de- to provide consistent development infrastructures that are both scalable and user friendly to students. The “Heterogeneous veloped as single-core, multi-core, and distributed program- Parallel Programming” class offered through Coursera teaches ming matured. GPU programming and encountered these problems. We de- All GPU curriculum requires students to access systems veloped WebGPU – an online GPU development platform – with GPUs for programming assignments. There are three providing students with a user friendly scalable GPU comput- key challenges to overcome when providing such systems ing platform throughout the course. It has been used as the CUDA, OpenACC, and OpenCL programming environment for a MOOC: for large Coursera courses, short-running summer schools, and • Number of students and student-educator ratio. The traditional semester-long
    [Show full text]
  • Moving Deep Learning Into Web Browser: How Far Can We Go?
    Moving Deep Learning into Web Browser: How Far Can We Go? Yun Ma1;2, Dongwei Xiang1, Shuyu Zheng1, Deyu Tian1, Xuanzhe Liu1 1Key Lab of High-Confidence Software Technology, MoE (Peking University), Beijing, China 2Tsinghua University, Beijing, China {mayun,xdw,zhengshuyu,tiandeyu,xzl}@pku.edu.cn ABSTRACT Compared to the native applications, Web applications can in- Recently, several JavaScript-based deep learning frameworks have deed make the cross-platform portability issues much simpler. The emerged, making it possible to perform deep learning tasks directly same implementation of a DL-powered Web application can be de- in browsers. However, little is known on what and how well we can ployed in the browser on all platforms regardless of the underlying do with these frameworks for deep learning in browsers. To bridge hardware device types (PC, smartphones, and wearable devices) the knowledge gap, in this paper, we conduct the first empirical and the operating systems (Windows, Mac, iOS, and Android). Ad- study of deep learning in browsers. We survey 7 most popular vancements in HTML5, CSS3, and especially JavaScript language, JavaScript-based deep learning frameworks, investigating to what started to enable the creation of DL-powered Web applications that extent deep learning tasks have been supported in browsers so offer a comparable experience to native applications, especially for far. Then we measure the performance of different frameworks the popular Web game applications [38][34]. In particular, bene- when running different deep learning tasks. Finally, we dig out fited from the development of WebGL [32][19][20], current major the performance gap between deep learning in browsers and on browsers such as Google Chrome, Mozilla FireFox, and Apple Sa- native platforms by comparing the performance of TensorFlow.js fari, can better utilize the integrated graphics card to accelerate DL and TensorFlow in Python.
    [Show full text]
  • Languages Education Experience Projects David Peicho
    DAVID PEICHO EDUCATION Software Engineer with experience in MASTER IN COMPUTER SCIENCE AND MACHINE LEARNING computer graphics. EPITA, Paris Sep 2013 - Sept 2018 EPITA is a 5-year engineering school focused on C/C++ programming and Unix systems. EXPERIENCE ABOUT ME RESEARCH SCIENTIST AT SIEMENS HEALTHINEERS Princeton, NJ, USA - London, GB - Paris, France Feb 2018 - Current Degree Lead developer of a real-time medical visualization library, Master based on Three.js / WebGL. Working on: - Volume pathtracing with real-time refinements City of Residence - Volumes and surfaces merging Paris, France - Multiplanar reconstruction Phone SOFTWARE ENGINEER INTERN AT SKETCHFAB +33 6 27 55 98 25 Paris, France sketchfab.com Sept 2016 - Jan 2017 Email Developed: [email protected] - The sketchfab.com SSAO post-process - The sketchfab.com ASTC texture decompressor Website - A glTF loader for the OSG.js open-source framework https://davidpeicho.github.io Languages SOFTWARE ENGINEER INTERN AT AERYS French, English Paris, France aerys.in May 2015 - Sept 2015 Worked on some small JS games, as well as a C++ Wii sport- like game LANGUAGES TEACHING JS Rust C++ C Rendering Theory & Physically-Based Rendering EPITA, Paris 2021 - Current Introduction to Three.js EPITA, Paris 2021 - Current GRAPHICS C / C++ / C# / Unix EPITA, Paris 2016 - 2018 OpenGL, WebGL WebGPU PROJECTS WASM Three.js ArtFlow JS, Three.js, GLSL https://github.com/artflow-vr Unity 3D web browser application to draw in VR (Tilt Brush-like). Blender PatternTrackingController C++, SFML https://github.com/DavidPeicho/pattern-tracking-controller LINKS Real-time arrow recognition library used to control a player in a game.
    [Show full text]
  • Building Webgpu with Rust
    Building WebGPU with Rust Fosdem, 2th Feb 2020 Dzmitry Malyshau @kvark (Mozilla / Graphics Engineer) Agenda 1. WebGPU: Why and What? 2. Example in Rust 3. Architecture 4. Rust features used 5. Wrap-up 6. (bonus level) Browsers Can we make this simpler? Screenshot from RDR2 trailer, PS4 Situation ● Developers want to have rich content running portably on the Web and Native ● Each native platform has a preferred API ● Some of them are best fit for engines, not applications ● The only path to reach most platforms is OpenGL/WebGL ○ Applications quickly become CPU-limited ○ No multi-threading is possible ○ Getting access to modern GPU features portably is hard, e.g. compute shaders are not always supported OpenGL Render like it’s 1992 Future of OpenGL? ● Apple -> deprecates OpenGL in 2018, there is no WebGL 2.0 support yet ● Microsoft -> not supporting OpenGL (or Vulkan) in UWP ● IHVs focus on Vulkan and DX12 drivers ● WebGL ends up translating to Dx11 (via Angle) on Windows by major browsers OptionGL: technical issues ● Changing a state can cause the driver to recompile the shader, internally ○ Causes 100ms freezes during the experience... ○ Missing concept of pipelines ● Challenging to optimize for mobile ○ Rendering tile management is critical for power-efficiency but handled implicitly ○ Missing concept of render passes ● Challenging to take advantage of more threads ○ Purely single-threaded, becomes a CPU bottleneck ○ Missing concept of command buffers ● Tricky data transfers ○ Dx11 doesn’t have buffer to texture copies ● Given that WebGL2
    [Show full text]
  • A Platform for Virtual Reality Applications
    Bachelor Thesis Czech Technical University in Prague Faculty of Electrical Engineering F3 Department of Cybernetics A Platform for Virtual Reality Applications Jakub Hlusička Supervisor: Ing. David Sedláček, Ph.D. January 2020 ii Acknowledgements Declaration I would like to express my gratitude to I declare that the presented work was de- my supervisor Ing. David Sedláček, Ph.D. veloped independently and that I have for the opportunity to work on a topic I listed all sources of information used find genuine interest in, and for his useful within it in accordance with the method- advice. ical instructions for observing the ethical I would also like to thank my family principles in the preparation of university for the support they provided me with theses. during my studies and the completion of this thesis. Prohlašuji, že jsem předloženou práci I am very thankful for being in a privi- vypracoval samostatně a že jsem uvedl leged enough position to be able to work veškeré použité informační zdroje v on personally motivated projects, such as souladu s Metodickým pokynem o do- the one covered in this thesis. One of the držování etických principů při přípravě reasons why I chose this topic for my the- vysokoškolských závěrečných prací. sis, is that I believe that free communi- cation is the basis of a free society, and Prague, 13. 5. 2020 __________ I hope it will benefit those who were not as fortunate as me. iii Abstract Abstrakt The World Wide Web has revolutionized Vynález webu (World Wide Web) vyvolal the way humanity shares and accesses in- převrat ve způsobu, jakým lidstvo sdílí in- formation.
    [Show full text]
  • Towards Engineering Computer Vision Systems: from the Web to Fpgas
    UNIVERSITY OF CALIFORNIA, IRVINE Towards Engineering Computer Vision Systems: From the Web to FPGAs DISSERTATION submitted in partial satisfaction of the requirements for the degree of DOCTOR OF PHILOSOPHY in Computer Science by Sajjad Taheri Dissertation Committee: Professor Alexandru Nicolau, Chair Professor Alexander Veidenbaum, Co-chair Professor Nikil Dutt 2019 © 2019 Sajjad Taheri DEDICATION To my mother, Nahid. ii TABLE OF CONTENTS Page LIST OF FIGURES v LIST OF TABLES vii ACKNOWLEDGMENTS viii CURRICULUM VITAE ix ABSTRACT OF THE DISSERTATION xi 1 Introduction 1 1.1 Challenges of Portable and Efficient Computer Vision . 3 1.2 Thesis Contributions . 4 1.2.1 Regarding Software Quality . 7 2 OpenCV.js: Computer Vision Processing for the Open Web platform 8 2.1 Introduction . 8 2.2 Compiling to The Web . 10 2.2.1 Portable Low-level Formats for Web Compilation . 12 2.3 Generating OpenCV.js . 16 2.4 Using OpenCV.js . 20 2.5 Performance Evaluation . 23 2.6 Availability . 27 3 Parallel Processing in OpenCV.js 28 3.1 Parallel Processing using ECMAScript SIMD . 29 3.2 Thread-level Parallelism Using Web Workers . 31 4 A Library for FPGA Implementation of Vision Algorithms 34 4.1 Introduction to OpenVX . 35 4.1.1 Examples . 40 4.2 A Computer Vision Library for FPGAs . 44 4.2.1 FPGA Technology and High-level Syntheis . 44 4.2.2 Library Design and Implementaion . 47 4.2.3 Programming Interface . 54 iii 5 A Framework for FPGA Acceleration of Computer Vision Algorithms 56 5.1 High-Level Optimizing Transformations . 58 5.2 Pipelining using Graph Partitioning .
    [Show full text]