DOCSLIB.ORG

Standards for Vision Processing and Neural Networks

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Standards for Vision Processing and Neural Networks Standards for Vision Processing and Neural Networks Radhakrishna Giduthuri, AMD [email protected] © Copyright Khronos Group 2017 - Page 1 Agenda • Why we need a standard? • Khronos NNEF • Khronos OpenVX dog Network Architecture Pre-trained Network Model (weights, …) © Copyright Khronos Group 2017 - Page 2 Neural Network End-to-End Workflow Neural Network Third Vision/AI Party Applications Training Frameworks Tools Datasets Trained Vision and Neural Network Network Inferencing Runtime Network Model Architecture Desktop and Cloud Hardware Embedded/Mobile Embedded/MobileEmbedded/Mobile Embedded/Mobile/Desktop/CloudVision/InferencingVision/Inferencing Hardware Hardware cuDNN MIOpen MKL-DNN Vision/InferencingVision/Inferencing Hardware Hardware GPU DSP CPU Custom FPGA © Copyright Khronos Group 2017 - Page 3 Problem: Neural Network Fragmentation Neural Network Training and Inferencing Fragmentation NN Authoring Framework 1 Inference Engine 1 NN Authoring Framework 2 Inference Engine 2 NN Authoring Framework 3 Inference Engine 3 Every Tool Needs an Exporter to Every Accelerator Neural Network Inferencing Fragmentation toll on Applications Inference Engine 1 Hardware 1 Vision/AI Inference Engine 2 Hardware 2 Application Inference Engine 3 Hardware 3 Every Application Needs know about Every Accelerator API © Copyright Khronos Group 2017 - Page 4 Khronos APIs Connect Software to Silicon Software Silicon Khronos is an International Industry Consortium of over 100 companies creating royalty-free, open standard APIs to enable software to access hardware acceleration for 3D graphics, Virtual and Augmented Reality, Parallel Computing, Vision Processing and Neural Networks © Copyright Khronos Group 2017 - Page 5 NNEF - Solving Neural Network Fragmentation Before NNEF – NN Training and Inferencing Fragmentation NN Authoring Framework 1 Inference Engine 1 NN Authoring Framework 2 Inference Engine 2 NN Authoring Framework 3 Inference Engine 3 Every Tool Needs an Exporter to Every Accelerator With NNEF- NN Training and Inferencing Interoperability NN Authoring Framework 1 Inference Engine 1 NN Authoring Framework 2 Inference Engine 2 NN Authoring Framework 3 Inference Engine 3 Optimization and processing tools NNEF is a Cross-vendor Neural Net file format Encapsulates network formal semantics, structure, data formats, commonly-used operations (such as convolution, pooling, normalization, etc.) © Copyright Khronos Group 2017 - Page 6 OpenVX - Solving Inferencing Fragmentation Before OpenVX – Vision and NN Inferencing Fragmentation Inference Engine 1 Hardware 1 Vision/AI Inference Engine 2 Hardware 2 Application Inference Engine 3 Hardware 3 Every Application Needs know about Every Accelerator API With OpenVX – Vision and NN Inferencing Interoperability Inference Engine 1 Hardware 1 Vision/AI Inference Engine 2 Hardware 2 Application Inference Engine 3 Hardware 3 OpenVX is an open, royalty-free standard for cross platform acceleration of computer vision and neural network applications. © Copyright Khronos Group 2017 - Page 7 Neural Net Workflow with Khronos Standards Neural Network Third Vision/AI Party Applications Training Frameworks Tools Datasets Trained Vision and Neural Network Network Inferencing Runtime Network Model Architecture Desktop and Cloud Hardware Embedded/Mobile Embedded/MobileEmbedded/Mobile Embedded/Mobile/Desktop/CloudVision/InferencingVision/Inferencing Hardware Hardware cuDNN MIOpen MKL-DNN Vision/InferencingVision/Inferencing Hardware Hardware GPU DSP CPU Custom FPGA © Copyright Khronos Group 2017 - Page 8 Application Development Workflow Vision/AI Applications Structure plus weights (quantized or not). Trained Can be augmented with private data Format Graph Inference Compiler Engine Training Framework Vendor Structure only Format OR Third Structure plus weights Party (quantized or not) Tools © Copyright Khronos Group 2017 - Page 9 NNEF Status and Roadmap • V1.0 is under development, industry comments are being sought now - NNEF has formed an advisory panel, you are invited today to participate • First version will focus on interface between framework and embedded inference engines - But will allow training as secondary goal • Support ‘First cut’ range of network types - Field is moving very fast but we aim to keep up with developments • NNEF Roadmap - Track development of new network types - Allow authoring and retraining (3rd party tools) - Address a wider range of applications (outside vision apps) - Increase the expressive power of the format © Copyright Khronos Group 2017 - Page 10 Khronos Advisory Panels Specification drafts and Requirements and invitations for feedback on requirements and feedback specification drafts Working Shared Email Advisory list and Group Repository Panel Hosted by Khronos. Khronos Members Under Khronos NDA Khronos Advisors Any company can join Invited industry experts Membership Fee $0 Cost Sign NDA and IP Framework Sign NDA and IP Framework Advisory Panels Active for NNEF, Vulkan, and OpenCL/SYCL © Copyright Khronos Group 2017 - Page 11 An open, royalty-free standard for cross platform acceleration of computer vision and neural network applications. © Copyright Khronos Group 2017 - Page 12 OpenVX Vision Engines Wide range of vision hardware architectures Middleware OpenVX provides a high-level Graph-based abstraction -> Applications Enables Graph-level optimizations! Can be implemented on almost any hardware or processor! -> Portable, Efficient Vision Processing! Software GPU Portability DSP Hardware X100 Dedicated Hardware Vision DSPs X10 Vision GPU Node Compute Vision Vision Node Node Multi-core CNN CPU Nodes Power Efficiency Power X1 Computation Flexibility Vision Processing Graph © Copyright Khronos Group 2017 - Page 13 OpenVX - Graph-Level Abstraction • OpenVX developers express a graph of image operations (‘Nodes’) - Using a C API • Nodes can be executed on any hardware or processor coded in any language - Implementers can optimize under the high-level graph abstraction • Graphs are the key to run-time power and performance optimizations - E.g. Node fusion, tiled graph processing for cache efficiency etc. Pyr Camera OpenVX Nodes t Rendering Input Output RGB YUV Gray Array of Frame Color Frame Channel Frame Image Optical Keypoints Harris Conversion Extract Pyramid Flow Track Array of Features Ftr OpenVX Graph t-1 Feature Extraction Example Graph © Copyright Khronos Group 2017 - Page 14 OpenVX Efficiency through Graphs.. Graph Memory Kernel Data Scheduling Management Fusion Tiling Split the graph Execute a sub- Reuse execution across Replace a sub- graph at tile pre-allocated the whole graph with a granularity memory for system: CPU / single faster node instead of image multiple GPU / dedicated granularity intermediate data HW Faster execution Less allocation overhead, Better memory Better use of or lower power more memory for locality, less kernel data cache and consumption other applications launch overhead local memory © Copyright Khronos Group 2017 - Page 15 Simple Edge Detector in OpenVX vx_graph g = vxCreateGraph(); vx_image input = vxCreateImage(1920, 1080); Declare Input and Output Images vx_image output = vxCreateImage(1920, 1080); vx_image horiz = vxCreateVirtualImage(g); Declare Intermediate Images vx_image vert = vxCreateVirtualImage(g); vx_image mag = vxCreateVirtualImage(g); vxSobel3x3Node(g, input, horiz, vert); Construct the Graph topology vxMagnitudeNode(g, horiz, vert, mag); vxThresholdNode(g, mag, THRESH, output); Compile the Graph Execute the Graph status = vxVerifyGraph(g); h status = vxProcessGraph(g); i S M m T o v © Copyright Khronos Group 2017 - Page 16 OpenVX Evolution Conformant Conformant Implementations Implementations New Functionality Conditional node execution Feature detection Classification operators Expanded imaging operations New Functionality Extensions Under Discussion Neural Network Acceleration New Functionality NNEF Import Expanded Nodes Functionality Graph Save and Restore 16-bit image operation Enhanced Graph Framework Programmable user AMD OpenVX Tools kernels with accelerator offload - Open source, highly optimized Safety Critical for x86 CPU and OpenCL for GPU OpenVX 1.1 SC for - “Graph Optimizer” looks at Streaming/pipelining safety-certifiable systems entire processing pipeline and removes, replaces, merges functions to improve performance OpenVX and bandwidth Roadmap - Scripting for rapid prototyping, OpenVX 1.2 without re-compiling, at production performance levels Spec released May 2017 http://gpuopen.com/compute-product/amd-openvx/ OpenVX 1.0 OpenVX 1.1 Spec released October 2014 Spec released May 2016 © Copyright Khronos Group 2017 - Page 17 AMD’s open-source implementation • Highly-optimized for x86 CPU and OpenCL GPU • Available on github.com http://github.com/GPUOpen-ProfessionalCompute-Libraries/amdovx-modules • Additional modules • LOOM: highly optimized library for real-time 360 degree video stitching • NN: neural network module built on top MIOpen (OpenCL-based machine learning primitives) Includes a tool to import pre-trained Caffe models into NN (develop branch) … © Copyright Khronos Group 2017 - Page 18 New OpenVX 1.2 Functions • Feature detection: find features useful for object detection and recognition - Histogram of gradients – HOG • Template matching - Local binary patterns – LBP • Line finding • Classification: detect and recognize objects in an image based on a set of features - Import a classifier model trained offline - Classify objects based on a set of input features • Image Processing: transform an
Load more

Recommended publications
  • Introduction to the Vulkan Computer Graphics API
    1 Introduction to the Vulkan Computer Graphics API Mike Bailey mjb – July 24, 2020 2 Computer Graphics Introduction to the Vulkan Computer Graphics API Mike Bailey [email protected] SIGGRAPH 2020 Abridged Version This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License http://cs.oregonstate.edu/~mjb/vulkan ABRIDGED.pptx mjb – July 24, 2020 3 Course Goals • Give a sense of how Vulkan is different from OpenGL • Show how to do basic drawing in Vulkan • Leave you with working, documented, understandable sample code http://cs.oregonstate.edu/~mjb/vulkan mjb – July 24, 2020 4 Mike Bailey • Professor of Computer Science, Oregon State University • Has been in computer graphics for over 30 years • Has had over 8,000 students in his university classes • [email protected] Welcome! I’m happy to be here. I hope you are too ! http://cs.oregonstate.edu/~mjb/vulkan mjb – July 24, 2020 5 Sections 13.Swap Chain 1. Introduction 14.Push Constants 2. Sample Code 15.Physical Devices 3. Drawing 16.Logical Devices 4. Shaders and SPIR-V 17.Dynamic State Variables 5. Data Buffers 18.Getting Information Back 6. GLFW 19.Compute Shaders 7. GLM 20.Specialization Constants 8. Instancing 21.Synchronization 9. Graphics Pipeline Data Structure 22.Pipeline Barriers 10.Descriptor Sets 23.Multisampling 11.Textures 24.Multipass 12.Queues and Command Buffers 25.Ray Tracing Section titles that have been greyed-out have not been included in the ABRIDGED noteset, i.e., the one that has been made to fit in SIGGRAPH’s reduced time slot.
    [Show full text]
  • Visual Development Environment for Openvx
    ______________________________________________________PROCEEDING OF THE 20TH CONFERENCE OF FRUCT ASSOCIATION Visual Development Environment for OpenVX Alexey Syschikov, Boris Sedov, Konstantin Nedovodeev, Sergey Pakharev Saint Petersburg State University of Aerospace Instrumentation Saint Petersburg, Russia {alexey.syschikov, boris.sedov, konstantin.nedovodeev, sergey.pakharev}@guap.ru Abstract—OpenVX standard has appeared as an answer II. STATE OF THE ART from the computer vision community to the challenge of accelerating vision applications on embedded heterogeneous OpenVX is intended to increase performance and reduce platforms. It is designed as a low-level programming framework power consumption of machine vision applications. It is that enables software developers to leverage the computer vision focused on embedded systems with real-time use cases such as hardware potential with functional and performance portability. face, body and gesture tracking, video surveillance, advanced In this paper, we present the visual environment for OpenVX driver assistance systems (ADAS), object and scene programs development. To the best of our knowledge, this is the reconstruction, augmented reality, visual inspection etc. first time the graphical notation is used for OpenVX programming. Our environment addresses the need to design The using of OpenVX standard functions is a way to ensure OpenVX graphs in a natural visual form with automatic functional portability of the developed software to all hardware generation of a full-fledged program, saving the programmer platforms that support OpenVX. from writing a bunch of a boilerplate code. Using the VIPE visual IDE to develop OpenVX programs also makes it possible to work Since the OpenVX API is based on opaque data types, with our performance analysis tools.
    [Show full text]
  • GLSL 4.50 Spec
    The OpenGL® Shading Language Language Version: 4.50 Document Revision: 7 09-May-2017 Editor: John Kessenich, Google Version 1.1 Authors: John Kessenich, Dave Baldwin, Randi Rost Copyright (c) 2008-2017 The Khronos Group Inc. All Rights Reserved. This specification is protected by copyright laws and contains material proprietary to the Khronos Group, Inc. It or any components may not be reproduced, republished, distributed, transmitted, displayed, broadcast, or otherwise exploited in any manner without the express prior written permission of Khronos Group. You may use this specification for implementing the functionality therein, without altering or removing any trademark, copyright or other notice from the specification, but the receipt or possession of this specification does not convey any rights to reproduce, disclose, or distribute its contents, or to manufacture, use, or sell anything that it may describe, in whole or in part. Khronos Group grants express permission to any current Promoter, Contributor or Adopter member of Khronos to copy and redistribute UNMODIFIED versions of this specification in any fashion, provided that NO CHARGE is made for the specification and the latest available update of the specification for any version of the API is used whenever possible. Such distributed specification may be reformatted AS LONG AS the contents of the specification are not changed in any way. The specification may be incorporated into a product that is sold as long as such product includes significant independent work developed by the seller. A link to the current version of this specification on the Khronos Group website should be included whenever possible with specification distributions.
    [Show full text]
  • The Importance of Data
    The landscape of Parallel Programing Models Part 2: The importance of Data Michael Wong and Rod Burns Codeplay Software Ltd. Distiguished Engineer, Vice President of Ecosystem IXPUG 2020 2 © 2020 Codeplay Software Ltd. Distinguished Engineer Michael Wong ● Chair of SYCL Heterogeneous Programming Language ● C++ Directions Group ● ISOCPP.org Director, VP http://isocpp.org/wiki/faq/wg21#michael-wong ● [email protected] ● [email protected] Ported ● Head of Delegation for C++ Standard for Canada Build LLVM- TensorFlow to based ● Chair of Programming Languages for Standards open compilers for Council of Canada standards accelerators Chair of WG21 SG19 Machine Learning using SYCL Chair of WG21 SG14 Games Dev/Low Latency/Financial Trading/Embedded Implement Releasing open- ● Editor: C++ SG5 Transactional Memory Technical source, open- OpenCL and Specification standards based AI SYCL for acceleration tools: ● Editor: C++ SG1 Concurrency Technical Specification SYCL-BLAS, SYCL-ML, accelerator ● MISRA C++ and AUTOSAR VisionCpp processors ● Chair of Standards Council Canada TC22/SC32 Electrical and electronic components (SOTIF) ● Chair of UL4600 Object Tracking ● http://wongmichael.com/about We build GPU compilers for semiconductor companies ● C++11 book in Chinese: Now working to make AI/ML heterogeneous acceleration safe for https://www.amazon.cn/dp/B00ETOV2OQ autonomous vehicle 3 © 2020 Codeplay Software Ltd. Acknowledgement and Disclaimer Numerous people internal and external to the original C++/Khronos group, in industry and academia, have made contributions, influenced ideas, written part of this presentations, and offered feedbacks to form part of this talk. But I claim all credit for errors, and stupid mistakes. These are mine, all mine! You can’t have them.
    [Show full text]
  • GLSL Specification
    The OpenGL® Shading Language Language Version: 4.60 Document Revision: 3 23-Jul-2017 Editor: John Kessenich, Google Version 1.1 Authors: John Kessenich, Dave Baldwin, Randi Rost Copyright (c) 2008-2017 The Khronos Group Inc. All Rights Reserved. This specification is protected by copyright laws and contains material proprietary to the Khronos Group, Inc. It or any components may not be reproduced, republished, distributed, transmitted, displayed, broadcast, or otherwise exploited in any manner without the express prior written permission of Khronos Group. You may use this specification for implementing the functionality therein, without altering or removing any trademark, copyright or other notice from the specification, but the receipt or possession of this specification does not convey any rights to reproduce, disclose, or distribute its contents, or to manufacture, use, or sell anything that it may describe, in whole or in part. Khronos Group grants express permission to any current Promoter, Contributor or Adopter member of Khronos to copy and redistribute UNMODIFIED versions of this specification in any fashion, provided that NO CHARGE is made for the specification and the latest available update of the specification for any version of the API is used whenever possible. Such distributed specification may be reformatted AS LONG AS the contents of the specification are not changed in any way. The specification may be incorporated into a product that is sold as long as such product includes significant independent work developed by the seller. A link to the current version of this specification on the Khronos Group website should be included whenever possible with specification distributions.
    [Show full text]
  • Marc Benito Bermúdez, Matina Maria Trompouki, Leonidas Kosmidis
    Evaluation of Graphics-based General Purpose Computation Solutions for Safety Critical Systems: An Avionics Case Study Marc Benito Bermúdez, Matina Maria Trompouki, Leonidas Kosmidis Juan David Garcia, Sergio Carretero, Ken Wenger {marc.benito, leonidas.Kosmidis}@bsc.es Motivation • Massively parallel architectures, high computational power and high energy efficiency, in thermally limited systems • OpenCL and CUDA dominate the market of GPGPU Embedded GPUs can programming in HPC provide the required • Easily programmable APIs performance • Cannot be used in safety critical systems because of pointers and dynamic memory allocation • Safety Critical Systems require higher performance to support new advanced functionalities • In this Bachelor’s thesis [1] awarded with a Technology Transfer Award we: Characteristics of Safety Critical Systems: • Analyze their differences compared to desktop graphics APIs • Certification: Need to comply with safety standards: ISO26262 / DO178 • Demonstrate how a safety-critical application written in a non-certifiable • Very conservative in terms of hardware and software: simple programming model can be converted to use safety-critical APIs. processors, mainly single core • Evaluate performance and programmability trade-offs OpenGL and Vulkan versions diagram Visual output of the Avionics Application Initial prototype GPU-based avionics application, written in Vulkan and ported to OpenGL SC 2 following the guidelines of [2][3]. Basic Compute Shader Shader 3D Image Processed Image Brook SC Porting and Comparison
    [Show full text]
  • SID Khronos Open Standards for AR May17
    Open Standards for AR Neil Trevett | Khronos President NVIDIA VP Developer Ecosystem [email protected] | @neilt3d LA, May 2017 © Copyright Khronos Group 2017 - Page 1 Khronos Mission Software Silicon Khronos is an International Industry Consortium of over 100 companies creating royalty-free, open standard APIs to enable software to access hardware acceleration for 3D graphics, Virtual and Augmented Reality, Parallel Computing, Neural Networks and Vision Processing © Copyright Khronos Group 2017 - Page 2 Khronos Standards Ecosystem 3D for the Web Real-time 2D/3D - Real-time apps and games in-browser - Cross-platform gaming and UI - Efficiently delivering runtime 3D assets - VR and AR Displays - CAD and Product Design - Safety-critical displays VR, Vision, Neural Networks Parallel Computation - VR/AR system portability - Tracking and odometry - Machine Learning acceleration - Embedded vision processing - Scene analysis/understanding - High Performance Computing (HPC) - Neural Network inferencing © Copyright Khronos Group 2017 - Page 3 Why AR Needs Standard Acceleration APIs Without API Standards With API Standards Platform Application Fragmentation Portability Everything Silicon runs on CPU Acceleration Standard Acceleration APIs provide PERFORMANCE, POWER AND PORTABILITY © Copyright Khronos Group 2017 - Page 4 AR Processing Flow Download 3D augmentation object and scene data Tracking and Positioning Generate Low Latency Vision Geometric scene 3D Augmentations for sensor(s) reconstruction display by optical system Semantic scene understanding
    [Show full text]
  • Khronos Template 2015
    Ecosystem Overview Neil Trevett | Khronos President NVIDIA Vice President Developer Ecosystem [email protected] | @neilt3d © Copyright Khronos Group 2016 - Page 1 Khronos Mission Software Silicon Khronos is an Industry Consortium of over 100 companies creating royalty-free, open standard APIs to enable software to access hardware acceleration for graphics, parallel compute and vision © Copyright Khronos Group 2016 - Page 2 http://accelerateyourworld.org/ © Copyright Khronos Group 2016 - Page 3 Vision Pipeline Challenges and Opportunities Growing Camera Diversity Diverse Vision Processors Sensor Proliferation 22 Flexible sensor and camera Use efficient acceleration to Combine vision output control to GENERATE PROCESS with other sensor data an image stream the image stream on device © Copyright Khronos Group 2016 - Page 4 OpenVX – Low Power Vision Acceleration • Higher level abstraction API - Targeted at real-time mobile and embedded platforms • Performance portability across diverse architectures - Multi-core CPUs, GPUs, DSPs and DSP arrays, ISPs, Dedicated hardware… • Extends portable vision acceleration to very low power domains - Doesn’t require high-power CPU/GPU Complex - Lower precision requirements than OpenCL - Low-power host can setup and manage frame-rate graph Vision Engine Middleware Application X100 Dedicated Vision Processing Hardware Efficiency Vision DSPs X10 GPU Compute Accelerator Multi-core Accelerator Power Efficiency Power X1 CPU Accelerator Computation Flexibility © Copyright Khronos Group 2016 - Page 5 OpenVX Graphs
    [Show full text]
  • Migrating from Opengl to Vulkan Mark Kilgard, January 19, 2016 About the Speaker Who Is This Guy?
    Migrating from OpenGL to Vulkan Mark Kilgard, January 19, 2016 About the Speaker Who is this guy? Mark Kilgard Principal Graphics Software Engineer in Austin, Texas Long-time OpenGL driver developer at NVIDIA Author and implementer of many OpenGL extensions Collaborated on the development of Cg First commercial GPU shading language Recently working on GPU-accelerated vector graphics (Yes, and wrote GLUT in ages past) 2 Motivation for Talk Coming from OpenGL, Preparing for Vulkan What kinds of apps benefit from Vulkan? How to prepare your OpenGL code base to transition to Vulkan How various common OpenGL usage scenarios are re-thought in Vulkan Re-thinking your application structure for Vulkan 3 Analogy Different Valid Approaches 4 Analogy Fixed-function OpenGL Pre-assembled toy car fun out of the box, not much room for customization 5 AZDO = Approaching Zero Driver Overhead Analogy Modern AZDO OpenGL with Programmable Shaders LEGO Kit you build it yourself, comes with plenty of useful, pre-shaped pieces 6 Analogy Vulkan Pine Wood Derby Kit you build it yourself to race from raw materials power tools used to assemble, adult supervision highly recommended 7 Analogy Different Valid Approaches Fixed-function OpenGL Modern AZDO OpenGL with Vulkan Programmable Shaders 8 Beneficial Vulkan Scenarios Has Parallelizable CPU-bound Graphics Work yes Can your graphics Is your graphics work start work creation be CPU bound? parallelized? yes Vulkan friendly 9 Beneficial Vulkan Scenarios Maximizing a Graphics Platform Budget You’ll yes do whatever Your graphics start it takes to squeeze platform is fixed out max perf. yes Vulkan friendly 10 Beneficial Vulkan Scenarios Managing Predictable Performance, Free of Hitching You put yes You can a premium on manage your start avoiding graphics resource hitches allocations yes Vulkan friendly 11 Unlikely to Benefit Scenarios to Reconsider Coding to Vulkan 1.
    [Show full text]
  • Real-Time Computer Vision with Opencv Khanh Vo Duc, Mobile Vision Team, NVIDIA
    Real-time Computer Vision with OpenCV Khanh Vo Duc, Mobile Vision Team, NVIDIA Outline . What is OpenCV? . OpenCV Example – CPU vs. GPU with CUDA . OpenCV CUDA functions . Future of OpenCV . Summary OpenCV Introduction . Open source library for computer vision, image processing and machine learning . Permissible BSD license . Freely available (www.opencv.org) Portability . Real-time computer vision (x86 MMX/SSE, ARM NEON, CUDA) . C (11 years), now C++ (3 years since v2.0), Python and Java . Windows, OS X, Linux, Android and iOS 3 Functionality Desktop . x86 single-core (Intel started, now Itseez.com) - v2.4.5 >2500 functions (multiple algorithm options, data types) . CUDA GPU (Nvidia) - 250 functions (5x – 100x speed-up) http://docs.opencv.org/modules/gpu/doc/gpu.html . OpenCL GPU (3rd parties) - 100 functions (launch times ~7x slower than CUDA*) Mobile (Nvidia): . Android (not optimized) . Tegra – 50 functions NEON, GLSL, multi-core (1.6–32x speed-up) 4 Functionality Image/video I/O, processing, display (core, imgproc, highgui) Object/feature detection (objdetect, features2d, nonfree) Geometry-based monocular or stereo computer vision (calib3d, stitching, videostab) Computational photography (photo, video, superres) Machine learning & clustering (ml, flann) CUDA and OpenCL GPU acceleration (gpu, ocl) 5 Outline . What is OpenCV? . OpenCV Example – CPU vs. GPU with CUDA . OpenCV CUDA functions . Future of OpenCV . Summary OpenCV CPU example #include <opencv2/opencv.hpp> OpenCV header files using namespace cv; OpenCV C++ namespace int
    [Show full text]
  • The Openvx™ Specification
    The OpenVX™ Specification Editor: Radhakrishna Giduthuri, Intel, The Khronos® OpenVX Working Group Version 1.3, Thu, 10 Sep 2020 07:04:36 +0000: Git branch information not available Table of Contents 1. Introduction . 2 1.1. Abstract . 2 1.2. Purpose . 2 1.3. Scope of Specification . 2 1.4. Normative References . 2 1.5. Version/Change History . 3 1.6. Deprecation . 3 1.7. Normative Requirements . 3 1.8. Typographical Conventions . 3 1.8.1. Naming Conventions . 4 1.8.2. Vendor Naming Conventions . 4 1.9. Glossary and Acronyms . 5 1.10. Acknowledgements . 5 2. Design Overview . 8 2.1. Software Landscape . 8 2.2. Design Objectives . 8 2.2.1. Hardware Optimizations . 8 2.2.2. Hardware Limitations . 9 2.3. Assumptions . 9 2.3.1. Portability. 9 2.3.2. Opaqueness . 9 2.4. Object-Oriented Behaviors . 9 2.5. OpenVX Framework Objects . 9 2.6. OpenVX Data Objects . 10 2.7. Error Objects . 11 2.8. Graphs Concepts . 11 2.8.1. Linking Nodes . 11 2.8.2. Virtual Data Objects . 11 2.8.3. Node Parameters . 14 2.8.4. Graph Parameters . 15 2.8.5. Execution Model . 15 Asynchronous Mode . 15 2.8.6. Graph Formalisms . 15 Contained & Overlapping Data Objects . 16 2.8.7. Node Execution Independence . 18 2.8.8. Verification. 20 2.9. Callbacks . 21 2.10. User Kernels. 21 2.10.1. Parameter Validation. 23 The Meta Format Object. 23 2.10.2. User Kernels Naming Conventions . 23 2.11. Immediate Mode Functions . 24 2.12. Targets . ..
    [Show full text]
  • Opengl SC 2.0?
    White Paper: Should You Be Using OpenGL SC 2.0? Introduction Ten years after the release of the first OpenGL® safety certifiable API specification the Khronos Group, the industry group responsible for graphics and video open standards, has released a new OpenGL safety certifiable specification, OpenGL SC 2.0. While both OpenGL SC 2.0 and the earlier safety critical OpenGL specification, OpenGL SC 1.0.1, provide high performance graphics interfaces for creating visualization and user interfaces, OpenGL SC 2.0 enables a higher degree of application capability with new levels of performance and control. This white paper will introduce OpenGL SC 2.0 by providing context in relationship to existing embedded OpenGL specifications, the differences compared to the earlier safety certifiable OpenGL specification, OpenGL SC 1.0.1 and an overview of new capabilities. Overview of OpenGL Specifications OpenGL Graphics Language (the GL in OpenGL) Application Programming Interfaces (API) is an open specification for developing 2D and 3D graphics software applications. There are three main profiles of the OpenGL specifications. OpenGL – desktop OpenGL ES – Embedded Systems OpenGL SC – Safety Critical From time to time the desktop OpenGL specification was updated to incorporate new extensions which are widely supported by Graphics Processing Unit (GPU) vendors reflecting new GPU capabilities. The embedded systems OpenGL specification branches from the desktop OpenGL specification less frequently and is tailored to embedded system needs. The safety critical OpenGL specification branches from the embedded systems OpenGL specifications and is further tailored for safety critical embedded systems. Figure 1 shows the high level relationship between the OpenGL specifications providing cross-platform APIs for full-function 2D and 3D graphics on embedded systems.
    [Show full text]