Towards Engineering Computer Vision Systems: from the Web to Fpgas
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Differential Fuzzing the Webassembly
Master’s Programme in Security and Cloud Computing Differential Fuzzing the WebAssembly Master’s Thesis Gilang Mentari Hamidy MASTER’S THESIS Aalto University - EURECOM MASTER’STHESIS 2020 Differential Fuzzing the WebAssembly Fuzzing Différentiel le WebAssembly Gilang Mentari Hamidy This thesis is a public document and does not contain any confidential information. Cette thèse est un document public et ne contient aucun information confidentielle. Thesis submitted in partial fulfillment of the requirements for the degree of Master of Science in Technology. Antibes, 27 July 2020 Supervisor: Prof. Davide Balzarotti, EURECOM Co-Supervisor: Prof. Jan-Erik Ekberg, Aalto University Copyright © 2020 Gilang Mentari Hamidy Aalto University - School of Science EURECOM Master’s Programme in Security and Cloud Computing Abstract Author Gilang Mentari Hamidy Title Differential Fuzzing the WebAssembly School School of Science Degree programme Master of Science Major Security and Cloud Computing (SECCLO) Code SCI3084 Supervisor Prof. Davide Balzarotti, EURECOM Prof. Jan-Erik Ekberg, Aalto University Level Master’s thesis Date 27 July 2020 Pages 133 Language English Abstract WebAssembly, colloquially known as Wasm, is a specification for an intermediate representation that is suitable for the web environment, particularly in the client-side. It provides a machine abstraction and hardware-agnostic instruction sets, where a high-level programming language can target the compilation to the Wasm instead of specific hardware architecture. The JavaScript engine implements the Wasm specification and recompiles the Wasm instruction to the target machine instruction where the program is executed. Technically, Wasm is similar to a popular virtual machine bytecode, such as Java Virtual Machine (JVM) or Microsoft Intermediate Language (MSIL). -
Visual Development Environment for Openvx
______________________________________________________PROCEEDING OF THE 20TH CONFERENCE OF FRUCT ASSOCIATION Visual Development Environment for OpenVX Alexey Syschikov, Boris Sedov, Konstantin Nedovodeev, Sergey Pakharev Saint Petersburg State University of Aerospace Instrumentation Saint Petersburg, Russia {alexey.syschikov, boris.sedov, konstantin.nedovodeev, sergey.pakharev}@guap.ru Abstract—OpenVX standard has appeared as an answer II. STATE OF THE ART from the computer vision community to the challenge of accelerating vision applications on embedded heterogeneous OpenVX is intended to increase performance and reduce platforms. It is designed as a low-level programming framework power consumption of machine vision applications. It is that enables software developers to leverage the computer vision focused on embedded systems with real-time use cases such as hardware potential with functional and performance portability. face, body and gesture tracking, video surveillance, advanced In this paper, we present the visual environment for OpenVX driver assistance systems (ADAS), object and scene programs development. To the best of our knowledge, this is the reconstruction, augmented reality, visual inspection etc. first time the graphical notation is used for OpenVX programming. Our environment addresses the need to design The using of OpenVX standard functions is a way to ensure OpenVX graphs in a natural visual form with automatic functional portability of the developed software to all hardware generation of a full-fledged program, saving the programmer platforms that support OpenVX. from writing a bunch of a boilerplate code. Using the VIPE visual IDE to develop OpenVX programs also makes it possible to work Since the OpenVX API is based on opaque data types, with our performance analysis tools. -
GLSL 4.50 Spec
The OpenGL® Shading Language Language Version: 4.50 Document Revision: 7 09-May-2017 Editor: John Kessenich, Google Version 1.1 Authors: John Kessenich, Dave Baldwin, Randi Rost Copyright (c) 2008-2017 The Khronos Group Inc. All Rights Reserved. This specification is protected by copyright laws and contains material proprietary to the Khronos Group, Inc. It or any components may not be reproduced, republished, distributed, transmitted, displayed, broadcast, or otherwise exploited in any manner without the express prior written permission of Khronos Group. You may use this specification for implementing the functionality therein, without altering or removing any trademark, copyright or other notice from the specification, but the receipt or possession of this specification does not convey any rights to reproduce, disclose, or distribute its contents, or to manufacture, use, or sell anything that it may describe, in whole or in part. Khronos Group grants express permission to any current Promoter, Contributor or Adopter member of Khronos to copy and redistribute UNMODIFIED versions of this specification in any fashion, provided that NO CHARGE is made for the specification and the latest available update of the specification for any version of the API is used whenever possible. Such distributed specification may be reformatted AS LONG AS the contents of the specification are not changed in any way. The specification may be incorporated into a product that is sold as long as such product includes significant independent work developed by the seller. A link to the current version of this specification on the Khronos Group website should be included whenever possible with specification distributions. -
Interaction Between Web Browsers and Script Engines
IT 12 058 Examensarbete 45 hp November 2012 Interaction between web browsers and script engines Xiaoyu Zhuang Institutionen för informationsteknologi Department of Information Technology Abstract Interaction between web browser and the script engine Xiaoyu Zhuang Teknisk- naturvetenskaplig fakultet UTH-enheten Web browser plays an important part of internet experience and JavaScript is the most popular programming language as a client side script to build an active and Besöksadress: advance end user experience. The script engine which executes JavaScript needs to Ångströmlaboratoriet Lägerhyddsvägen 1 interact with web browser to get access to its DOM elements and other host objects. Hus 4, Plan 0 Browser from host side needs to initialize the script engine and dispatch script source code to the engine side. Postadress: This thesis studies the interaction between the script engine and its host browser. Box 536 751 21 Uppsala The shell where the engine address to make calls towards outside is called hosting layer. This report mainly discussed what operations could appear in this layer and Telefon: designed testing cases to validate if the browser is robust and reliable regarding 018 – 471 30 03 hosting operations. Telefax: 018 – 471 30 00 Hemsida: http://www.teknat.uu.se/student Handledare: Elena Boris Ämnesgranskare: Justin Pearson Examinator: Lisa Kaati IT 12 058 Tryckt av: Reprocentralen ITC Contents 1. Introduction................................................................................................................................ -
Machine Learning in the Browser
Machine Learning in the Browser The Harvard community has made this article openly available. Please share how this access benefits you. Your story matters Citable link http://nrs.harvard.edu/urn-3:HUL.InstRepos:38811507 Terms of Use This article was downloaded from Harvard University’s DASH repository, and is made available under the terms and conditions applicable to Other Posted Material, as set forth at http:// nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of- use#LAA Machine Learning in the Browser a thesis presented by Tomas Reimers to The Department of Computer Science in partial fulfillment of the requirements for the degree of Bachelor of Arts in the subject of Computer Science Harvard University Cambridge, Massachusetts March 2017 Contents 1 Introduction 3 1.1 Background . .3 1.2 Motivation . .4 1.2.1 Privacy . .4 1.2.2 Unavailable Server . .4 1.2.3 Simple, Self-Contained Demos . .5 1.3 Challenges . .5 1.3.1 Performance . .5 1.3.2 Poor Generality . .7 1.3.3 Manual Implementation in JavaScript . .7 2 The TensorFlow Architecture 7 2.1 TensorFlow's API . .7 2.2 TensorFlow's Implementation . .9 2.3 Portability . .9 3 Compiling TensorFlow into JavaScript 10 3.1 Motivation to Compile . 10 3.2 Background on Emscripten . 10 3.2.1 Build Process . 12 3.2.2 Dependencies . 12 3.2.3 Bitness Assumptions . 13 3.2.4 Concurrency Model . 13 3.3 Experiences . 14 4 Results 15 4.1 Benchmarks . 15 4.2 Library Size . 16 4.3 WebAssembly . 17 5 Developer Experience 17 5.1 Universal Graph Runner . -
The Importance of Data
The landscape of Parallel Programing Models Part 2: The importance of Data Michael Wong and Rod Burns Codeplay Software Ltd. Distiguished Engineer, Vice President of Ecosystem IXPUG 2020 2 © 2020 Codeplay Software Ltd. Distinguished Engineer Michael Wong ● Chair of SYCL Heterogeneous Programming Language ● C++ Directions Group ● ISOCPP.org Director, VP http://isocpp.org/wiki/faq/wg21#michael-wong ● [email protected] ● [email protected] Ported ● Head of Delegation for C++ Standard for Canada Build LLVM- TensorFlow to based ● Chair of Programming Languages for Standards open compilers for Council of Canada standards accelerators Chair of WG21 SG19 Machine Learning using SYCL Chair of WG21 SG14 Games Dev/Low Latency/Financial Trading/Embedded Implement Releasing open- ● Editor: C++ SG5 Transactional Memory Technical source, open- OpenCL and Specification standards based AI SYCL for acceleration tools: ● Editor: C++ SG1 Concurrency Technical Specification SYCL-BLAS, SYCL-ML, accelerator ● MISRA C++ and AUTOSAR VisionCpp processors ● Chair of Standards Council Canada TC22/SC32 Electrical and electronic components (SOTIF) ● Chair of UL4600 Object Tracking ● http://wongmichael.com/about We build GPU compilers for semiconductor companies ● C++11 book in Chinese: Now working to make AI/ML heterogeneous acceleration safe for https://www.amazon.cn/dp/B00ETOV2OQ autonomous vehicle 3 © 2020 Codeplay Software Ltd. Acknowledgement and Disclaimer Numerous people internal and external to the original C++/Khronos group, in industry and academia, have made contributions, influenced ideas, written part of this presentations, and offered feedbacks to form part of this talk. But I claim all credit for errors, and stupid mistakes. These are mine, all mine! You can’t have them. -
Javascript API Deprecation in the Wild: a First Assessment
JavaScript API Deprecation in the Wild: A First Assessment Romulo Nascimento, Aline Brito, Andre Hora, Eduardo Figueiredo Department of Computer Science Federal University of Minas Gerais, Brazil romulonascimento, alinebrito, andrehora,figueiredo @dcc.ufmg.br { } Abstract—Building an application using third-party libraries of our knowledge, there are no detailed studies regarding API is a common practice in software development. As any other deprecation in the JavaScript ecosystem. software system, code libraries and their APIs evolve over JavaScript has become extremely popular over the last years. time. In order to help version migration and ensure backward According to the Stack Overflow 2019 Developer Survey1, compatibility, a recommended practice during development is to deprecate API. Although studies have been conducted to JavaScript is the most popular programming language in this investigate deprecation in some programming languages, such as platform for the seventh consecutive year. GitHub also reports Java and C#, there are no detailed studies on API deprecation that JavaScript is the most popular language in terms of unique in the JavaScript ecosystem. This paper provides an initial contributors to both public and private repositories2. The npm assessment of API deprecation in JavaScript by analyzing 50 platform, the largest JavaScript package manager, states on popular software projects. Initial results suggest that the use of 3 deprecation mechanisms in JavaScript packages is low. However, their latest survey that 99% of JavaScript developers rely on wefindfive different ways that developers use to deprecate API npm to ease the management of their project dependencies. in the studied projects. Among these solutions, deprecation utility This survey also points out the massive growth in npm usage (i.e., any sort of function specially written to aid deprecation) and that started about 5 years ago. -
SID Khronos Open Standards for AR May17
Open Standards for AR Neil Trevett | Khronos President NVIDIA VP Developer Ecosystem [email protected] | @neilt3d LA, May 2017 © Copyright Khronos Group 2017 - Page 1 Khronos Mission Software Silicon Khronos is an International Industry Consortium of over 100 companies creating royalty-free, open standard APIs to enable software to access hardware acceleration for 3D graphics, Virtual and Augmented Reality, Parallel Computing, Neural Networks and Vision Processing © Copyright Khronos Group 2017 - Page 2 Khronos Standards Ecosystem 3D for the Web Real-time 2D/3D - Real-time apps and games in-browser - Cross-platform gaming and UI - Efficiently delivering runtime 3D assets - VR and AR Displays - CAD and Product Design - Safety-critical displays VR, Vision, Neural Networks Parallel Computation - VR/AR system portability - Tracking and odometry - Machine Learning acceleration - Embedded vision processing - Scene analysis/understanding - High Performance Computing (HPC) - Neural Network inferencing © Copyright Khronos Group 2017 - Page 3 Why AR Needs Standard Acceleration APIs Without API Standards With API Standards Platform Application Fragmentation Portability Everything Silicon runs on CPU Acceleration Standard Acceleration APIs provide PERFORMANCE, POWER AND PORTABILITY © Copyright Khronos Group 2017 - Page 4 AR Processing Flow Download 3D augmentation object and scene data Tracking and Positioning Generate Low Latency Vision Geometric scene 3D Augmentations for sensor(s) reconstruction display by optical system Semantic scene understanding -
Standards for Vision Processing and Neural Networks
Standards for Vision Processing and Neural Networks Radhakrishna Giduthuri, AMD [email protected] © Copyright Khronos Group 2017 - Page 1 Agenda • Why we need a standard? • Khronos NNEF • Khronos OpenVX dog Network Architecture Pre-trained Network Model (weights, …) © Copyright Khronos Group 2017 - Page 2 Neural Network End-to-End Workflow Neural Network Third Vision/AI Party Applications Training Frameworks Tools Datasets Trained Vision and Neural Network Network Inferencing Runtime Network Model Architecture Desktop and Cloud Hardware Embedded/Mobile Embedded/MobileEmbedded/Mobile Embedded/Mobile/Desktop/CloudVision/InferencingVision/Inferencing Hardware Hardware cuDNN MIOpen MKL-DNN Vision/InferencingVision/Inferencing Hardware Hardware GPU DSP CPU Custom FPGA © Copyright Khronos Group 2017 - Page 3 Problem: Neural Network Fragmentation Neural Network Training and Inferencing Fragmentation NN Authoring Framework 1 Inference Engine 1 NN Authoring Framework 2 Inference Engine 2 NN Authoring Framework 3 Inference Engine 3 Every Tool Needs an Exporter to Every Accelerator Neural Network Inferencing Fragmentation toll on Applications Inference Engine 1 Hardware 1 Vision/AI Inference Engine 2 Hardware 2 Application Inference Engine 3 Hardware 3 Every Application Needs know about Every Accelerator API © Copyright Khronos Group 2017 - Page 4 Khronos APIs Connect Software to Silicon Software Silicon Khronos is an International Industry Consortium of over 100 companies creating royalty-free, open standard APIs to enable software to access -
Learning Javascript Design Patterns
Learning JavaScript Design Patterns Addy Osmani Beijing • Cambridge • Farnham • Köln • Sebastopol • Tokyo Learning JavaScript Design Patterns by Addy Osmani Copyright © 2012 Addy Osmani. All rights reserved. Revision History for the : 2012-05-01 Early release revision 1 See http://oreilly.com/catalog/errata.csp?isbn=9781449331818 for release details. ISBN: 978-1-449-33181-8 1335906805 Table of Contents Preface ..................................................................... ix 1. Introduction ........................................................... 1 2. What is a Pattern? ...................................................... 3 We already use patterns everyday 4 3. 'Pattern'-ity Testing, Proto-Patterns & The Rule Of Three ...................... 7 4. The Structure Of A Design Pattern ......................................... 9 5. Writing Design Patterns ................................................. 11 6. Anti-Patterns ......................................................... 13 7. Categories Of Design Pattern ............................................ 15 Creational Design Patterns 15 Structural Design Patterns 16 Behavioral Design Patterns 16 8. Design Pattern Categorization ........................................... 17 A brief note on classes 17 9. JavaScript Design Patterns .............................................. 21 The Creational Pattern 22 The Constructor Pattern 23 Basic Constructors 23 Constructors With Prototypes 24 The Singleton Pattern 24 The Module Pattern 27 iii Modules 27 Object Literals 27 The Module Pattern -
Ecmascript (Or ES)
Lesson: Web Programming(1) Omid Jafarinezhad Sharif University of Technology Objective Covers languages, tools, and techniques for developing interactive and dynamic web pages. Topics include page styling, design, and layout; client and server side scripting; web security; and interacting with data sources such as databases Web development can range from developing the simplest static single page of plain text to the most complex web apps (such as electronic businesses, and social network services) ● HTTP, JavaScript, CSS, HTML5, ReactJs, Flow, Progressive Web App ● Golang, NodeJs, MongoDB, PostgreSQL, Redis ● Docker, Git, YUIDoc, Jest, Materials WebPack, Gulp, Browserify, Locust ● (Optional/Research) Kubernetes, InfluxDB, RabbitMQ, gRPC, Ansible Grading Big Picture Internal or external Content Delivery Email/SMS/... services; may be Network (CDN) Service developed in different language Win HTTP, gRPC HTTP Linux WebSocket front-end back-end Data storage Mac JavaScript, Html, NodeJs, mongoDB, CSS, Ajax, GoLang, cache postgreSQL, WebRTC, ReactJs, C#, Java, InfluxDB, ... Mobile AngularJs,... Dart, ... Redis, AMQP, ... Memcached, ... logs queue Logstash, RabitMQ, Fluentd, ... ZeroMQ, ... back-end 1 Load front-end back-end 2 balancing kubernetes cluster, HAProxy, Docker Swarm, ... back-end 3 Git repository Test, Continuous deployment, Code coverage, Merge, Review Build automation, Deployment automation Development Staging Production Bug User feedback, Crash report,... Continuous ... Continuous Integration basically just means that the developer's -
Khronos Template 2015
Ecosystem Overview Neil Trevett | Khronos President NVIDIA Vice President Developer Ecosystem [email protected] | @neilt3d © Copyright Khronos Group 2016 - Page 1 Khronos Mission Software Silicon Khronos is an Industry Consortium of over 100 companies creating royalty-free, open standard APIs to enable software to access hardware acceleration for graphics, parallel compute and vision © Copyright Khronos Group 2016 - Page 2 http://accelerateyourworld.org/ © Copyright Khronos Group 2016 - Page 3 Vision Pipeline Challenges and Opportunities Growing Camera Diversity Diverse Vision Processors Sensor Proliferation 22 Flexible sensor and camera Use efficient acceleration to Combine vision output control to GENERATE PROCESS with other sensor data an image stream the image stream on device © Copyright Khronos Group 2016 - Page 4 OpenVX – Low Power Vision Acceleration • Higher level abstraction API - Targeted at real-time mobile and embedded platforms • Performance portability across diverse architectures - Multi-core CPUs, GPUs, DSPs and DSP arrays, ISPs, Dedicated hardware… • Extends portable vision acceleration to very low power domains - Doesn’t require high-power CPU/GPU Complex - Lower precision requirements than OpenCL - Low-power host can setup and manage frame-rate graph Vision Engine Middleware Application X100 Dedicated Vision Processing Hardware Efficiency Vision DSPs X10 GPU Compute Accelerator Multi-core Accelerator Power Efficiency Power X1 CPU Accelerator Computation Flexibility © Copyright Khronos Group 2016 - Page 5 OpenVX Graphs