DOCSLIB.ORG

Vulkan, Opengl And/Or Zerocopy

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Vulkan, Opengl And/Or Zerocopy Vulkan, OpenGL and/or Zerocopy Matthew Waters (ystreet00) GStreamer conference 20"# "0 October 2016 Who Am & ● %ustralian ● Work ( )entricular ● Graph'cs * OpenGL, Vulkan ● Mult'med'a +u'ck &ntroduct'on * OpenGL ● OpenGL $orn rom SG& in ",,0-s ● )ross(plat orm .D %0& – 1"", wayland, OS 1, 'OS, %ndro'd, Windows, 2m$edded Linu3 +u'ck &ntroduct'on * GStreamer OpenGL/2S ● Minimum tar4et OpenGL 2S !50 – 2ssentially the $e4innin4 o GLSL support ● Versions supported * OpenGL 2S !50/.53 /esktop !53/.53/653 ● 0lat orms supported * L'nu3 (1"" + Wayland), OS 1, W'ndows, 'OS, %ndroid, 2m$edded Linu3 ● Various elements a8aila$le * 4lima4es'nk, 4lcolorcon8ert, 4l8ideom'3er, 4ltrans ormat'on, 4lo8erlay, 4le9ects:;, etc What-s <ew= - l'$4st4l ● 4l8iewcon8ert now supported on OpenGL 2S !50 plat orms (stereo elements as well) ● 4lcolorcon8ert now allows con8ert'n4 to mult'(planar colorspaces 'n OpenGL 2S .53 ● 4ldownload %0& remo8ed rom the library ● dma($u GL uploader ● GL >ueries ● /elayed GStreamer de$u44in4 What-s <ew= * l'$4st4l GL memory ● <ew GL $u9er $ased GstMemory ● GstGLMemory0?O * GL te3tures w'th 0'3el ?u9er O$@ects ● GstGLMemory2GL * GL te3tures w'th 2GL&ma4eAs ● <ew GL render$u9er $ased GstMemory ● GstGLBrame$u9er +u'ck &ntroduct'on * Vulkan ● Vulkan released Be$ruary !0"# ● )ross(plat orm .D %0& – 1"", wayland, %ndro'd, Windows, 2m$edded Linu3 ● %ims to $e a $etter Ct or modern G0Us ● More control o8er synchron'Eation Vulkan 'n GStreamer ● 8ulkansink and vulkanupload elements ● Only $asics 'mplemented – Modelled on l'$4st4l %0& ● Much more work needed to $e on par with OpenGL support Vulkan 'n GStreamer ● Somewhat sim'lar 'n rastructure to li$4st4l – Gst)onte3t – /'splay – Window – &nstance – /e8'ce ● Some thin4s are 8ery d'9erent – Synchron'Eat'on * semaphores, e8ents, ences – More applicat'on state Zerocopy - &ntroduct'on ● /efinit'on ran4es romF – <o copies at all – <o )0U per ormed copies ● Most common $etween decoder and renderer ($ut also occur $etween capturer/encoder) Zerocopy * Where= ● V%(%0& ● OpenM%1 &L ● %ndro'dAs Media)odec ● iOS/macOS V'deoGool$o3 ● V/0%D ● 18MC/18?% ● /1V% Zerocopy * How= ● ?ind – VM MMD – Synchron'Eat'on ● Dse ● Dn$'nd – Synchron'Eat'on Zerocopy * dma($u ● Dsed $y 86l!, (V%(%0& on 2GL) ● dma($u is con8erted to an 2GL&ma4e ● 2GL&ma4e is $ound to a GL te3ture with 4l2GL&ma4eGar4etGe3ture!D() Zerocopy * Andro'd Med'a)odec ● Hold the consumer end o a >ueue ● )an only pop o9 the end o the >ueue $ut donAt ha8e un'>ue rame handles ● Synchron'Eat'on 's all internal ● )an only $'nd to one GL conte3t at a time Zerocopy * OpenM%1 &L ● Dses 2GL&ma4eAs ● I0' has a separate om3 component * e4l_render ● 2GL&ma4e $acked $y a GL te3ture passed into OM1 ● )ustom GstGLMemory Zerocopy * V'deoGool$o3 - macOS ● Dses &OSur ace ● )GLGe3&ma4e&OSur ace!/() ● )ustom GstGLMemory su$class Zerocopy * V'deoGool$o3 - 'OS ● &OSur ace is a8a'la$le $ut not pu$lic ● Dses )VOpenGL2SGe3ture)ache instead – )VOpenGL2SGe3ture)ache)reateGe3tureBrom&ma4e() ● )ustom GstGLMemory su$class Zerocopy * 4streamer(8aap' ● GstVideoGLGe3tureUploadMeta – 1"" uses GL:21G:texture: rom:p'3map – Wayland/2GL uses dma($u What-s Happen'n4= ● OpenGL helper li$rary * mo8e to 4st(plu4'ns($ase ● OpenGL model viewer * st'll F() GhanksJ ystreet00 in #4streamer on freenode .
Load more

Recommended publications
  • Introduction to the Vulkan Computer Graphics API
    1 Introduction to the Vulkan Computer Graphics API Mike Bailey mjb – July 24, 2020 2 Computer Graphics Introduction to the Vulkan Computer Graphics API Mike Bailey [email protected] SIGGRAPH 2020 Abridged Version This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License http://cs.oregonstate.edu/~mjb/vulkan ABRIDGED.pptx mjb – July 24, 2020 3 Course Goals • Give a sense of how Vulkan is different from OpenGL • Show how to do basic drawing in Vulkan • Leave you with working, documented, understandable sample code http://cs.oregonstate.edu/~mjb/vulkan mjb – July 24, 2020 4 Mike Bailey • Professor of Computer Science, Oregon State University • Has been in computer graphics for over 30 years • Has had over 8,000 students in his university classes • [email protected] Welcome! I’m happy to be here. I hope you are too ! http://cs.oregonstate.edu/~mjb/vulkan mjb – July 24, 2020 5 Sections 13.Swap Chain 1. Introduction 14.Push Constants 2. Sample Code 15.Physical Devices 3. Drawing 16.Logical Devices 4. Shaders and SPIR-V 17.Dynamic State Variables 5. Data Buffers 18.Getting Information Back 6. GLFW 19.Compute Shaders 7. GLM 20.Specialization Constants 8. Instancing 21.Synchronization 9. Graphics Pipeline Data Structure 22.Pipeline Barriers 10.Descriptor Sets 23.Multisampling 11.Textures 24.Multipass 12.Queues and Command Buffers 25.Ray Tracing Section titles that have been greyed-out have not been included in the ABRIDGED noteset, i.e., the one that has been made to fit in SIGGRAPH’s reduced time slot.
    [Show full text]
  • GLSL 4.50 Spec
    The OpenGL® Shading Language Language Version: 4.50 Document Revision: 7 09-May-2017 Editor: John Kessenich, Google Version 1.1 Authors: John Kessenich, Dave Baldwin, Randi Rost Copyright (c) 2008-2017 The Khronos Group Inc. All Rights Reserved. This specification is protected by copyright laws and contains material proprietary to the Khronos Group, Inc. It or any components may not be reproduced, republished, distributed, transmitted, displayed, broadcast, or otherwise exploited in any manner without the express prior written permission of Khronos Group. You may use this specification for implementing the functionality therein, without altering or removing any trademark, copyright or other notice from the specification, but the receipt or possession of this specification does not convey any rights to reproduce, disclose, or distribute its contents, or to manufacture, use, or sell anything that it may describe, in whole or in part. Khronos Group grants express permission to any current Promoter, Contributor or Adopter member of Khronos to copy and redistribute UNMODIFIED versions of this specification in any fashion, provided that NO CHARGE is made for the specification and the latest available update of the specification for any version of the API is used whenever possible. Such distributed specification may be reformatted AS LONG AS the contents of the specification are not changed in any way. The specification may be incorporated into a product that is sold as long as such product includes significant independent work developed by the seller. A link to the current version of this specification on the Khronos Group website should be included whenever possible with specification distributions.
    [Show full text]
  • Computer Graphics on Mobile Devices
    Computer Graphics on Mobile Devices Bruno Tunjic∗ Vienna University of Technology Figure 1: Different mobile devices available on the market today. Image courtesy of ASU [ASU 2011]. Abstract 1 Introduction Computer graphics hardware acceleration and rendering techniques Under the term mobile device we understand any device designed have improved significantly in recent years. These improvements for use in mobile context [Marcial 2010]. In other words this term are particularly noticeable in mobile devices that are produced in is used for devices that are battery-powered and therefore physi- great amounts and developed by different manufacturers. New tech- cally movable. This group of devices includes mobile (cellular) nologies are constantly developed and this extends the capabilities phones, personal media players (PMP), personal navigation devices of such devices correspondingly. (PND), personal digital assistants (PDA), smartphones, tablet per- sonal computers, notebooks, digital cameras, hand-held game con- soles and mobile internet devices (MID). Figure 1 shows different In this paper, a review about the existing and new hardware and mobile devices available on the market today. Traditional mobile software, as well as a closer look into some of the most important phones are aimed at making and receiving telephone calls over a revolutionary technologies, is given. Special emphasis is given on radio link. PDAs are personal organizers that later evolved into de- new Application Programming Interfaces (API) and rendering tech- vices with advanced units communication, entertainment and wire- niques that were developed in recent years. A review of limitations less capabilities [Wiggins 2004]. Smartphones can be seen as a that developers have to overcome when bringing graphics to mobile next generation of PDAs since they incorporate all its features but devices is also provided.
    [Show full text]
  • Report of Contributions
    X.Org Developers Conference 2020 Report of Contributions https://xdc2020.x.org/e/XDC2020 X.Org Developer … / Report of Contributions State of text input on Wayland Contribution ID: 1 Type: not specified State of text input on Wayland Wednesday, 16 September 2020 20:15 (5 minutes) Between the last impromptu talk at GUADEC 2018, text input on Wayland has become more organized and more widely adopted. As before, the three-pronged approach of text_input, in- put_method, and virtual keyboard still causes confusion, but increased interest in implementing it helps find problems and come closer to something that really works for many usecases. The talk will mention how a broken assumption causes a broken protocol, and why we’re notdone with Wayland input methods yet. It’s recommended to people who want to know more about the current state of input methods on Wayland. Recommended background: aforementioned GUADEC talk, wayland-protocols reposi- tory, my blog: https://dcz_self.gitlab.io/ Code of Conduct Yes GSoC, EVoC or Outreachy No Primary author: DCZ, Dorota Session Classification: Demos / Lightning talks I Track Classification: Lightning Talk September 30, 2021 Page 1 X.Org Developer … / Report of Contributions IGT GPU Tools 2020 Update Contribution ID: 2 Type: not specified IGT GPU Tools 2020 Update Wednesday, 16 September 2020 20:00 (5 minutes) Short update on IGT - what has changed in the last year, where are we right now and what we have planned for the near future. IGT GPU Tools is a collection of tools and tests aiding development of DRM drivers. It’s widely used by Intel in its public CI system.
    [Show full text]
  • Opencl on the GPU San Jose, CA | September 30, 2009
    OpenCL on the GPU San Jose, CA | September 30, 2009 Neil Trevett and Cyril Zeller, NVIDIA Welcome to the OpenCL Tutorial! • Khronos and industry perspective on OpenCL – Neil Trevett Khronos Group President OpenCL Working Group Chair NVIDIA Vice President Mobile Content • NVIDIA and OpenCL – Cyril Zeller NVIDIA Manager of Compute Developer Technology Khronos and the OpenCL Standard Neil Trevett OpenCL Working Group Chair, Khronos President NVIDIA Vice President Mobile Content Copyright Khronos 2009 Who is the Khronos Group? • Consortium creating open API standards ‘by the industry, for the industry’ – Non-profit founded nine years ago – over 100 members - any company welcome • Enabling software to leverage silicon acceleration – Low-level graphics, media and compute acceleration APIs • Strong commercial focus – Enabling members and the wider industry to grow markets • Commitment to royalty-free standards – Industry makes money through enabled products – not from standards themselves Silicon Community Software Community Copyright Khronos 2009 Apple Over 100 companies creating authoring and acceleration standards Board of Promoters Processor Parallelism CPUs GPUs Multiple cores driving Emerging Increasingly general purpose performance increases Intersection data-parallel computing Improving numerical precision Multi-processor Graphics APIs programming – Heterogeneous and Shading e.g. OpenMP Computing Languages Copyright Khronos 2009 OpenCL Commercial Objectives • Grow the market for parallel computing • Create a foundation layer for a parallel
    [Show full text]
  • Blackberry QNX Multimedia Suite
    PRODUCT BRIEF QNX Multimedia Suite The QNX Multimedia Suite is a comprehensive collection of media technology that has evolved over the years to keep pace with the latest media requirements of current-day embedded systems. Proven in tens of millions of automotive infotainment head units, the suite enables media-rich, high-quality playback, encoding and streaming of audio and video content. The multimedia suite comprises a modular, highly-scalable architecture that enables building high value, customized solutions that range from simple media players to networked systems in the car. The suite is optimized to leverage system-on-chip (SoC) video acceleration, in addition to supporting OpenMAX AL, an industry open standard API for application-level access to a device’s audio, video and imaging capabilities. Overview Consumer’s demand for multimedia has fueled an anywhere- o QNX SDK for Smartphone Connectivity (with support for Apple anytime paradigm, making multimedia ubiquitous in embedded CarPlay and Android Auto) systems. More and more embedded applications have require- o Qt distributions for QNX SDP 7 ments for audio, video and communication processing capabilities. For example, an infotainment system’s media player enables o QNX CAR Platform for Infotainment playback of content, stored either on-board or accessed from an • Support for a variety of external media stores external drive, mobile device or streamed over IP via a browser. Increasingly, these systems also have streaming requirements for Features at a Glance distributing content across a network, for instance from a head Multimedia Playback unit to the digital instrument cluster or rear seat entertainment units. Multimedia is also becoming pervasive in other markets, • Software-based audio CODECs such as medical, industrial, and whitegoods where user interfaces • Hardware accelerated video CODECs are increasingly providing users with a rich media experience.
    [Show full text]
  • History and Evolution of the Android OS
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Springer - Publisher Connector CHAPTER 1 History and Evolution of the Android OS I’m going to destroy Android, because it’s a stolen product. I’m willing to go thermonuclear war on this. —Steve Jobs, Apple Inc. Android, Inc. started with a clear mission by its creators. According to Andy Rubin, one of Android’s founders, Android Inc. was to develop “smarter mobile devices that are more aware of its owner’s location and preferences.” Rubin further stated, “If people are smart, that information starts getting aggregated into consumer products.” The year was 2003 and the location was Palo Alto, California. This was the year Android was born. While Android, Inc. started operations secretly, today the entire world knows about Android. It is no secret that Android is an operating system (OS) for modern day smartphones, tablets, and soon-to-be laptops, but what exactly does that mean? What did Android used to look like? How has it gotten where it is today? All of these questions and more will be answered in this brief chapter. Origins Android first appeared on the technology radar in 2005 when Google, the multibillion- dollar technology company, purchased Android, Inc. At the time, not much was known about Android and what Google intended on doing with it. Information was sparse until 2007, when Google announced the world’s first truly open platform for mobile devices. The First Distribution of Android On November 5, 2007, a press release from the Open Handset Alliance set the stage for the future of the Android platform.
    [Show full text]
  • The Opengl Graphics System
    OpenGL R ES Native Platform Graphics Interface (Version 1.0) Editor: Jon Leech Copyright c 2002-2003 Promoters of the Khronos Group (3Dlabs, ARM Ltd., ATI Technologies, Inc., Discreet, Ericsson Mobile, Imagination Technologies Group plc, Motorola, Inc., Nokia, Silicon Graphics, Inc., SK Telecom, and Sun Microsystems). This document is protected by copyright, and contains information proprietary to The Khronos Group. Any copying, adaptation, distribution, public performance, or public display of this document without the express written consent of the copy- right holders is strictly prohibited. The receipt or possession of this document does not convey any rights to reproduce, disclose, or distribute its contents, or to manu- facture, use, or sell anything that it may describe, in whole or in part. R This document is a derivative work of ”OpenGL Graphics with the X Window System (Version 1.4)”. Silicon Graphics, Inc. owns, and reserves all rights in, the latter document. OpenGL is a registered trademark, and OpenGL ES is a trademark, of Silicon Graphics, Inc. Contents 1 Overview 1 2 EGL Operation 2 2.1 Native Window System and Rendering APIs . 2 2.1.1 Scalar Types . 2 2.1.2 Displays . 3 2.2 Rendering Contexts and Drawing Surfaces . 3 2.2.1 Using Rendering Contexts . 4 2.2.2 Rendering Models . 4 2.2.3 Interaction With Native Rendering . 4 2.3 Direct Rendering and Address Spaces . 5 2.4 Shared State . 5 2.4.1 Texture Objects . 6 2.5 Multiple Threads . 6 2.6 Power Management . 7 3 EGL Functions and Errors 8 3.1 Errors .
    [Show full text]
  • The Opencl Specification
    The OpenCL Specification Version: 2.0 Document Revision: 22 Khronos OpenCL Working Group Editor: Aaftab Munshi Last Revision Date: 3/18/14 Page 1 1. INTRODUCTION ............................................................................................................... 10 2. GLOSSARY ......................................................................................................................... 12 3. THE OPENCL ARCHITECTURE .................................................................................... 23 3.1 Platform Model ................................................................................................................................ 23 3.2 Execution Model .............................................................................................................................. 25 3.2.1 Execution Model: Mapping work-items onto an NDRange ........................................................................28 3.2.2 Execution Model: Execution of kernel-instances ........................................................................................30 3.2.3 Execution Model: Device-side enqueue ......................................................................................................31 3.2.4 Execution Model: Synchronization .............................................................................................................32 3.2.5 Execution Model: Categories of Kernels ....................................................................................................33 3.3 Memory
    [Show full text]
  • 414 Advances in Opengl ES for Ios 5 V3 DD F
    Advances in OpenGL ES for iOS 5 Session 414 Gokhan Avkarogullari Eric Sunalp iPhone GPU Software These are confidential sessions—please refrain from streaming, blogging, or taking pictures 1 iPad 2 2 3 Per Pixel Lighting/ Normal Maps 4 LightMaps GlossMaps SpecularMaps 5 Dynamic Shadows 6 MSAA 7 Higher Poly Models 8 GLKit New Features 9 GLKit New Features 10 OpenGL ES 2.0 Molecules.app OpenGL ES 1.1 OpenGL ES 2.0 11 GLKit Goals • Making life easier for the developers ■ Find common problems ■ Make solutions available • Encourage unique look ■ Fixed-function pipeline games look similar ■ Shaders to rescue ■ How about porting 12 GLKit GLKTextureLoader GLKTextureLoader • Give a reference—get an OpenGL Texture Object • No need to deal ImageIO, CGImage, libjpg, libpng… 13 GLKit GLKView and GLKViewController GLKTextureLoader GLKView/ViewController • First-class citizen of UIKit hierarchy • Encapsulates FBOs, display links, MSAA management… 14 GLKit GLKMath GLKTextureLoader GLKView/ViewController GLKMath • 3D Graphics math library • Matrix stack, transforms, quaternions… 15 GLKit GLKEffects GLKTextureLoader GLKView/ViewController GLKMath GLKEffects • Fixed-function pipeline features implemented in ES 2.0 context 16 GLKTextureLoader 17 GLKTextureLoader Overview • Makes texture loading simple • Supports common image formats ■ PNG, JPEG, TIFF, etc. • Non-premultiplied data stays non-premultiplied • Cubemap texture support • Convenient loading options ■ Force premultiplication ■ Y-flip ■ Mipmap generation 18 GLKTextureLoader Basic usage • Make an EAGLContext
    [Show full text]
  • Standards for Vision Processing and Neural Networks
    Standards for Vision Processing and Neural Networks Radhakrishna Giduthuri, AMD [email protected] © Copyright Khronos Group 2017 - Page 1 Agenda • Why we need a standard? • Khronos NNEF • Khronos OpenVX dog Network Architecture Pre-trained Network Model (weights, …) © Copyright Khronos Group 2017 - Page 2 Neural Network End-to-End Workflow Neural Network Third Vision/AI Party Applications Training Frameworks Tools Datasets Trained Vision and Neural Network Network Inferencing Runtime Network Model Architecture Desktop and Cloud Hardware Embedded/Mobile Embedded/MobileEmbedded/Mobile Embedded/Mobile/Desktop/CloudVision/InferencingVision/Inferencing Hardware Hardware cuDNN MIOpen MKL-DNN Vision/InferencingVision/Inferencing Hardware Hardware GPU DSP CPU Custom FPGA © Copyright Khronos Group 2017 - Page 3 Problem: Neural Network Fragmentation Neural Network Training and Inferencing Fragmentation NN Authoring Framework 1 Inference Engine 1 NN Authoring Framework 2 Inference Engine 2 NN Authoring Framework 3 Inference Engine 3 Every Tool Needs an Exporter to Every Accelerator Neural Network Inferencing Fragmentation toll on Applications Inference Engine 1 Hardware 1 Vision/AI Inference Engine 2 Hardware 2 Application Inference Engine 3 Hardware 3 Every Application Needs know about Every Accelerator API © Copyright Khronos Group 2017 - Page 4 Khronos APIs Connect Software to Silicon Software Silicon Khronos is an International Industry Consortium of over 100 companies creating royalty-free, open standard APIs to enable software to access
    [Show full text]
  • Openbricks Embedded Linux Framework - User Manual I
    OpenBricks Embedded Linux Framework - User Manual i OpenBricks Embedded Linux Framework - User Manual OpenBricks Embedded Linux Framework - User Manual ii Contents 1 OpenBricks Introduction 1 1.1 What is it ?......................................................1 1.2 Who is it for ?.....................................................1 1.3 Which hardware is supported ?............................................1 1.4 What does the software offer ?............................................1 1.5 Who’s using it ?....................................................1 2 List of supported features 2 2.1 Key Features.....................................................2 2.2 Applicative Toolkits..................................................2 2.3 Graphic Extensions..................................................2 2.4 Video Extensions...................................................3 2.5 Audio Extensions...................................................3 2.6 Media Players.....................................................3 2.7 Key Audio/Video Profiles...............................................3 2.8 Networking Features.................................................3 2.9 Supported Filesystems................................................4 2.10 Toolchain Features..................................................4 3 OpenBricks Supported Platforms 5 3.1 Supported Hardware Architectures..........................................5 3.2 Available Platforms..................................................5 3.3 Certified Platforms..................................................7
    [Show full text]