DOCSLIB.ORG

Neuromorphic Vision: Sensors to Event-Based Algorithms Q2 7

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Neuromorphic Vision: Sensors to Event-Based Algorithms Q2 7 Journal Code Article ID Dispatch: 15-FEB-19 CE: D, Subramani WIDM 1310 No. of Pages: 34 ME: Received: 6 September 2018 Revised: 23 January 2019 Accepted: 24 January 2019 DOI: 10.1002/widm.1310 1 2 3 ADVANCED REVIEW 4 5 6 Neuromorphic vision: Sensors to event-based algorithms Q2 7 8 1,2 3 2 9 A. Lakshmi | Anirban Chakraborty | Chetan S. Thakur Q1 Q3 10 11 1Centre for Artificial Intelligence and Robotics, Regardless of the marvels brought by the conventional frame-based cameras, they 12 Defence Research and Development Organization, Bangalore, India have significant drawbacks due to their redundancy in data and temporal latency. 13 2Department of Electronic Systems Engineering, This causes problem in applications where low-latency transmission and high- 14 Indian Institute of Science, Bangalore, India speed processing are mandatory. Proceeding on this line of thought, the neurobio- 15 3 Department of Computational and Data Sciences, logical principles of the biological retina have been adapted to accomplish data 16 Indian Institute of Science, Bangalore, India Q4 sparsity and high dynamic range at the pixel level. These bio-inspired neuro- 17 Correspondence morphic vision sensors alleviate the more serious bottleneck of data redundancy by 18 Chetan Singh Thakur, Department of Electronic Systems Engineering, Indian Institute of Science, responding to changes in illumination rather than to illumination itself. This paper 19 Bangalore 560012, India. reviews in brief one such representative of neuromorphic sensors, the activity- 20 Email: [email protected] driven event-based vision sensor, which mimics human eyes. Spatio-temporal 21 encoding of event data permits incorporation of time correlation in addition to spa- 22 tial correlation in vision processing, which enables more robustness. Henceforth, 23 the conventional vision algorithms have to be reformulated to adapt to this new 24 generation vision sensor data. It involves design of algorithms for sparse, asynchro- 25 nous, and accurately timed information. Theories and new researches have begun 26 emerging recently in the domain of event-based vision. The necessity to compile 27 the vision research carried out in this sensor domain has turned out to be consider- 28 ably more essential. Towards this, this paper reviews the state-of-the-art event- 29 based vision algorithms by categorizing them into three major vision applications, 30 object detection/recognition, object tracking, localization and mapping. 31 32 This article is categorized under: 33 Technologies > Machine Learning Q5 34 35 36 37 38 1 | INTRODUCTION 39 40 In this era of rapid innovations and technological marvels, much effort is being put into developing visual sensors that aim to 41 impersonate the working principles of the human retina. These efforts are motivated by and driven towards the ultimate objec- 42 tive of catering for applications such as high-speed robotics and parsing/analyzing high dynamic range scenes. There are situa- 43 tions where the conventional video cameras fall short. The colossal amount of data from a conventional camera overwhelms 44 the processing of embedded systems and henceforth unavoidably turns into a bottleneck. This necessitates that some funda- 45 mental level of preprocessing has to be done at the acquisition level, as opposed to loading processing systems with redundant 46 data. In any case, developing sensors for this novel modality requires information-processing technologies built from the 47 domain of neuromorphic engineering. One such research effort prompted the development of neuromorphic vision sensors, 48 49 also known as event-based vision sensors, which have shown incredible accomplishment in furnishing solutions to the issues 50 encountered in conventional cameras. These sensors have led to a revolutionary way of recording information by asynchro- 51 nously registering only changes in illumination with microsecond accuracy. 52 These sensors find utility extensively in high-speed vision-based applications, which are indispensable in various fields 53 such as micro-robotics. The frame-based vision algorithms would not be effective in analyzing these event data-streams. As 54 55 WIREs Data Mining Knowl Discov. 2019;e1310. wires.wiley.com/dmkd © 2019 Wiley Periodicals, Inc. 1of34 https://doi.org/10.1002/widm.1310 2of34 LAKSHMI ET AL. 1 they expect synchronous data, these would not enable us to leverage upon the aforementioned benefits obtainable from neuro- 2 morphic vision sensor data. Consequently, a shift in the paradigm of algorithms from conventional frame-based to event-based 3 sensors is essential. The development of event-based vision algorithms is yet to reach the scale of development of their frame- 4 based counterparts as well as the level of accuracy these algorithms have been able to achieve in the conventional video data 5 domain. The main reason is the very limited commercial availability of these sensors and that these are accessible only as pro- 6 totypes or proof-of-concept. Nevertheless, quite a number of interesting computer vision algorithms are recently being devel- 7 oped for these type of sensors, which are optimally designed to exploit the higher temporal resolution and data sparsity of 8 these sensors. 9 With the development of present-day technologies, these sensors are starting to show up in the commercial field. This also 10 necessitates a structured and detailed study of important vision algorithms already conceived or needed to be developed for 11 12 analyzing the event data streams. However, there is hardly any such survey in the event domain unlike the plethora of similar 13 works existing for its frame-based counterparts. Hence, we present a survey of vision algorithms available for event data. 14 Despite the fact that our focus is to review event-based vision algorithms, we have additionally intended to provide a concise 15 overview of silicon retinae, with more emphasis on a specific type of silicon retina known as the temporal difference silicon 16 retina. Furthermore, we also briefly review the event datasets available in the public domain to evaluate the aforementioned 17 event-based vision algorithms. 18 The structure of the paper is as follows. The paper introduces silicon retina and temporal difference silicon retina in a nut- 19 shell, followed by the details of the state-of-the-art of three important event-based vision applications, viz., object detection 20 and recognition, object tracking as well as localization and mapping. As a closing remark, the paper gives a gist of open- 21 source event datasets along with the available codes1 and implementations of event-based vision sensor algorithms. 22 23 24 2 SILICON RETINA 25 | 26 Information processing of neurobiological systems is very exceptionally one-of-a-kind in relation to present-day image- 27 capturing devices. Biological vision systems dependably beat human-invented vision sensors. Conventional image sensors are 28 29 operated by a synchronous clock signal, which has resulted in an enormous amount of monotonous and redundant data accu- 30 mulation. This aspect has motivated the research in neuromorphic engineering to emulate vision sensors that mimic human 31 eyes, which has led to the development of silicon retinae. Silicon retinae are event-based, unlike conventional sensors, which 32 are frame-based. This makes it more similar to the biological model of the human eye, which is driven by events occurring in 33 the real world. The first electronic model of the retina came into existence in the early 70's (Fukushima, Yamaguchi, 34 Yasuda, & Nagata, 1970; Mahowald, 1994), following which numerous forms of silicon retinae widely known as event-based 35 vision sensors came into existence, including the spatial contrast retina that encodes relative light intensity change with respect 36 to the background (Barbaro, Burgi, Mortara, Nussbaum, & Heitger, 2002; Bardallo, Gotarredona, & Barranco, 2009; Ruedi 37 et al., 2009), gradient-based sensors that encode static edges, temporal intensity retina (Mallik, Clapp, Choi, Cauwenberghs, & 38 Cummings, 2005), and temporal difference retina (Kramer, 2002; Lichtsteiner, Posch, & Delbruck, 2006). This study reviews 39 the rationale of temporal difference silicon retinae, which are commonly known as activity-driven event-based vision sensors. 40 Temporal difference silicon retina generates events in response to changes in illumination in the scene in temporal space. 41 The following section discusses, in brief, the details of their architecture, types, history, advantages, companies involved, and 42 software available. 43 44 45 2.1 | Event generation mechanism 46 I (t), whose value is sampled at fixed time intervals Δt. This results in a 47 In a conventional sensor, an image consists of pixels x,y 48 representation that is incompatible with compact representation. In temporal difference silicon retina, the pixels with signifi- 0 49 cant luminance changes across time, Ix, yðÞt , are retained. An event at spatial location (x, y) and time t can be represented as 50 8 no < 0 0 △ 51 sign Ix, yðÞt if j Ix, yðÞjt > et = , ð1Þ 52 : 0 △ 0 if j Ix, yðÞjt < 53 54 where △ is the threshold. These events are spatio-temporal in nature. 55 LAKSHMI ET AL. 3of34 1 2.2 | Event transmission mechanism 2 As temporal difference silicon retina is spike based, it utilizes a spike-based interfacing procedure known as address event rep- 3 resentation (AER: Figure 1) to transmit asynchronous information over a narrow channel (Gotarredona, Andreou, & Barranco, F1 4 1999). In AER, each neuron is assigned a unique address by its encoder/arbiter. When an event is generated, the address of 5 the neuron and relevant information are sent to the destination over the bus. The decoder in the receiver interprets the received 6 information and assigns the received information to the particular destination. When the number of event-generating neurons 7 surpasses the transmission capacity of the bus, multiplexing is employed. 8 9 10 11 2.3 | Event processing mechanism 12 This section describes the way in which spatio-temporal data of silicon retina could be interpreted.
Load more

Recommended publications
  • A Morphic Based Live Programming Graphical User Interface Implemented in Python
    Final Thesis PyMorphic - a Morphic based Live Programming Graphical User Interface implemented in Python by Anders Osterholm¨ LIU-KOGVET-D--06/12--SE 23th August 2006 PyMorphic - a Morphic based Live Programming Graphical User Interface implemented in Python Final Thesis performed at Human Centered Systems division in the Department of Computer and Information Science at Link¨oping University by Anders Osterholm¨ LIU-KOGVET-D--06/12--SE 23th August 2006 Examiner: Dr. Mikael Kindborg Department of Computer and Information Science at Link¨oping University Abstract Programming is a very complex activity that has many simultaneous learn- ing elements. The area of Live-programming offers possibilities for enhanc- ing programming work by speeding up the feedback loop and providing means for reducing the cognitive load on the working memory during the task. This could allow for better education for novice programmers. In this work a number of systems with a shared aim of providing educational tools for scholars from compulsory level to undergraduate college were studied. The common approach in the majority of the tools was to use program ab- stractions like tangible morphs, playing cards, capsules for code segments, and visual stories. For the user these abstractions and tools offer better focus on the constructive and creative side of programming because they relieve the user from the cumbersome work of writing program code, but they also sacrifice some of the expressiveness of a low-level language. A Live programming system, called PyMorphic, based on the Morphic model was built in the Python programming language. Two different so- lutions, based on the Wx toolkit for Python, were constructed and evalu- ated.
    [Show full text]
  • Neural Network Transformation and Co-Design Under Neuromorphic Hardware Constraints
    NEUTRAMS: Neural Network Transformation and Co-design under Neuromorphic Hardware Constraints Yu Ji ∗, YouHui Zhang∗‡, ShuangChen Li†,PingChi†, CiHang Jiang∗,PengQu∗,YuanXie†, WenGuang Chen∗‡ Email: [email protected], [email protected] ∗Department of Computer Science and Technology, Tsinghua University, PR.China † Department of Electrical and Computer Engineering, University of California at Santa Barbara, USA ‡ Center for Brain-Inspired Computing Research, Tsinghua University, PR.China Abstract—With the recent reincarnations of neuromorphic (CMP)1 with a Network-on-Chip (NoC) has emerged as a computing comes the promise of a new computing paradigm, promising platform for neural network simulation, motivating with a focus on the design and fabrication of neuromorphic chips. many recent studies of neuromorphic chips [4]–[10]. The A key challenge in design, however, is that programming such chips is difficult. This paper proposes a systematic methodology design of neuromorphic chips is promising to enable a new with a set of tools to address this challenge. The proposed computing paradigm. toolset is called NEUTRAMS (Neural network Transformation, One major issue is that neuromorphic hardware usually Mapping and Simulation), and includes three key components: places constraints on NN models (such as the maximum fan- a neural network (NN) transformation algorithm, a configurable in/fan-out of a single neuron, the limited range of synaptic clock-driven simulator of neuromorphic chips and an optimized runtime tool that maps NNs onto the target hardware for weights) that it can support, and the hardware types of neurons better resource utilization. To address the challenges of hardware or activation functions are usually simpler than the software constraints on implementing NN models (such as the maximum counterparts.
    [Show full text]
  • Feihong Hsu 217 S
    Feihong Hsu 217 S. Leavitt St., #1S Chicago, IL 60612 Cell: 847.219.6000, Home: 312.738.3179 [email protected] www.cs.uic.edu/~fhsu EDUCATION: University of Illinois at Chicago Expected graduation in May 2004 Master of Science in Computer Science GPA: 4.80/5.00 University of Illinois at Urbana-Champaign Graduated December 2000 Bachelor of Science in Mathematics & Computer Science GPA: 3.78/4.00 Courses : Software Engineering, Database Systems, User Interface Design, Computer Networks, Object-Oriented Languages & Environments, Computer Architecture, Numerical Methods, Combinatorial Algorithms, Programming Languages & Compilers, Artificial Intelligence EXPERIENCE: Teaching Assistant , Introduction to Programming , August 2002-December 2003 University of Illinois at Chicago, Computer Science Department http://logos.cs.uic.edu/102 • Teach up to 30 students in weekly lab sections (C/C++/Java). • Develop online material, quizzes, lab assignments. Assist in writing exam problems. • Proposed and implemented new hands-on approach to labs, causing students to become visibly more engaged in the material. • Designed assignment in which students programmed an image manipulation application which included convolution and distortion filters. Teaching Assistant , Computer Literacy , January 2004-Present University of Illinois at Chicago, Computer Science Department http://wiggins.cs.uic.edu/cs100 • Most duties are similar to above. • Propose course topics and determine direction of the course Research Assistant , Team Engineering Collaboratory (TEC) , May 2000-December 2001 University of Illinois at Urbana-Champaign, Speech Communications Department http://www.spcomm.uiuc.edu/Projects/TECLAB/ • Continued development of Blanche, a modeling and simulation environment for the study of social networks. • Managed two undergrad programmers. Maintained project web page and distributions.
    [Show full text]
  • Deep Learning with Tensorflow 2 and Keras Second Edition
    Deep Learning with TensorFlow 2 and Keras Second Edition Regression, ConvNets, GANs, RNNs, NLP, and more with TensorFlow 2 and the Keras API Antonio Gulli Amita Kapoor Sujit Pal BIRMINGHAM - MUMBAI Deep Learning with TensorFlow 2 and Keras Second Edition Copyright © 2019 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. Commissioning Editor: Amey Varangaonkar Acquisition Editors: Yogesh Deokar, Ben Renow-Clarke Acquisition Editor – Peer Reviews: Suresh Jain Content Development Editor: Ian Hough Technical Editor: Gaurav Gavas Project Editor: Janice Gonsalves Proofreader: Safs Editing Indexer: Rekha Nair Presentation Designer: Sandip Tadge First published: April 2017 Second edition: December 2019 Production reference: 2130320 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN 978-1-83882-341-2 www.packt.com packt.com Subscribe to our online digital library for full access to over 7,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career.
    [Show full text]
  • Nordic ID Morphic User Manual
    Nordic ID Morphic User Manual Nordic ID International Headquarters Phone: +358 2 727 7700 Myllyojankatu 2A Fax: +358 2 727 7720 FI-24100 SALO Email: [email protected] FINLAND www.nordicid.com Table of Contents 3.3 Stylus and Touch Screen............................ 20 3.3.1 Input Panel: Soft Keyboard ................ 21 (i) A few words of caution..................................... 4 3.4 Rebooting your Nordic ID Morphic ............ 21 3.4.1 Soft-resetting your device................... 21 (ii) Trademarks......................................................... 4 3.4.2 Hard-resetting your device ................. 22 (iii) Safety precautions .......................................... 5 3.5 Registry ......................................................... 22 Laser beam ........................................................... 5 3.6 Non-volatile Storage.................................... 23 Note of caution about the laser beam in 3.7 Control Panel................................................ 23 European languages............................................ 6 3.7.1 Accessing Control Panel..................... 24 3.7.2 Opening Applet .................................... 24 Battery.................................................................... 7 3.7.3 Closing Applet ...................................... 24 Wall charger.......................................................... 8 3.7.4 Closing Control Panel ......................... 24 Medical devices.................................................... 8 3.8 Control Panel Applets
    [Show full text]
  • A Sparsity-Driven Backpropagation-Less Learning Framework Using Populations of Spiking Growth Transform Neurons
    ORIGINAL RESEARCH published: 28 July 2021 doi: 10.3389/fnins.2021.715451 A Sparsity-Driven Backpropagation-Less Learning Framework Using Populations of Spiking Growth Transform Neurons Ahana Gangopadhyay and Shantanu Chakrabartty* Department of Electrical and Systems Engineering, Washington University in St. Louis, St. Louis, MO, United States Growth-transform (GT) neurons and their population models allow for independent control over the spiking statistics and the transient population dynamics while optimizing a physically plausible distributed energy functional involving continuous-valued neural variables. In this paper we describe a backpropagation-less learning approach to train a Edited by: network of spiking GT neurons by enforcing sparsity constraints on the overall network Siddharth Joshi, spiking activity. The key features of the model and the proposed learning framework University of Notre Dame, United States are: (a) spike responses are generated as a result of constraint violation and hence Reviewed by: can be viewed as Lagrangian parameters; (b) the optimal parameters for a given task Wenrui Zhang, can be learned using neurally relevant local learning rules and in an online manner; University of California, Santa Barbara, United States (c) the network optimizes itself to encode the solution with as few spikes as possible Youhui Zhang, (sparsity); (d) the network optimizes itself to operate at a solution with the maximum Tsinghua University, China dynamic range and away from saturation; and (e) the framework is flexible enough to *Correspondence: incorporate additional structural and connectivity constraints on the network. As a result, Shantanu Chakrabartty [email protected] the proposed formulation is attractive for designing neuromorphic tinyML systems that are constrained in energy, resources, and network structure.
    [Show full text]
  • Multilayer Spiking Neural Network for Audio Samples Classification Using
    Multilayer Spiking Neural Network for Audio Samples Classification Using SpiNNaker Juan Pedro Dominguez-Morales, Angel Jimenez-Fernandez, Antonio Rios-Navarro, Elena Cerezuela-Escudero, Daniel Gutierrez-Galan, Manuel J. Dominguez-Morales, and Gabriel Jimenez-Moreno Robotic and Technology of Computers Lab, Department of Architecture and Technology of Computers, University of Seville, Seville, Spain {jpdominguez,ajimenez,arios,ecerezuela, dgutierrez,mdominguez,gaji}@atc.us.es http://www.atc.us.es Abstract. Audio classification has always been an interesting subject of research inside the neuromorphic engineering field. Tools like Nengo or Brian, and hard‐ ware platforms like the SpiNNaker board are rapidly increasing in popularity in the neuromorphic community due to the ease of modelling spiking neural networks with them. In this manuscript a multilayer spiking neural network for audio samples classification using SpiNNaker is presented. The network consists of different leaky integrate-and-fire neuron layers. The connections between them are trained using novel firing rate based algorithms and tested using sets of pure tones with frequencies that range from 130.813 to 1396.91 Hz. The hit rate percentage values are obtained after adding a random noise signal to the original pure tone signal. The results show very good classification results (above 85 % hit rate) for each class when the Signal-to-noise ratio is above 3 decibels, vali‐ dating the robustness of the network configuration and the training step. Keywords: SpiNNaker · Spiking neural network · Audio samples classification · Spikes · Neuromorphic auditory sensor · Address-Event Representation 1 Introduction Neuromorphic engineering is a discipline that studies, designs and implements hardware and software with the aim of mimicking the way in which nervous systems work, focusing its main inspiration on how the brain solves complex problems easily.
    [Show full text]
  • Pharo Morphic Application Framework
    Pharo Morphic Application Framework [email protected] Abstract The purpose of this framework is to have a comprehensive Pharo Application hierarchy to easily construct an enterprise ready application, that is scalable and built out as modules that link together. Morphic View Framework: Simplify creating Morphic UI with creating views from Components/ Parts / Modular UI construction: Create multiple pieces of independent views and compose it into one composite application view easily with: • Specs based View Construction • Simple Morphic View • Panels Composition: Complex GUI • API for: View attributes initializations intervention in intermediate steps • SupervisoryController aka MVP Presenter • Locale based translation integrated • LoggerManager to easily plugin Toothpick logger • Configuration based constants from external properties file / defaults code based.. • Named Prototype to optimize space/ time complexities as well as extension to dependency injection • Keyboard Event Handler at view level • Mouse Listener at a view level • Flexible Explicit View Construction Additionally incorporate existing/create standard frameworks, possibly as plugins or independent package, for: • Evolve a UI Builder that is a simple drag n drop capability of widgets • Complete RPC capabilites over XMLRPC for various purposes vis: • a JDBC connector along with an example complimentary Groovy Server backend / Groovy Client code References: 1. Various Smalltalk implementations: MVC architecture leading to the SupervisoryController architecture of MVP. 2. Spring Framework 3. General Enterprise Application Architecture patterns Motivation: Smalltalk is a beautiful and enjoyable language. Morphic is a wonderful but most under exploited UI framework that has immense potential to exponentially grow with contribution from the world of developers. Enterprise developers today are not experts, but casual programmers who depend on existing frameworks, patterns to adapt quickly.
    [Show full text]
  • Neuromorphic Computing Using Emerging Synaptic Devices: a Retrospective Summary and an Outlook
    electronics Review Neuromorphic Computing Using Emerging Synaptic Devices: A Retrospective Summary and an Outlook Jaeyoung Park 1,2 1 School of Computer Science and Electronic Engineering, Handong Global University, Pohang 37554, Korea; [email protected]; Tel.: +82-54-260-1394 2 School of Electronic Engineering, Soongsil University, Seoul 06978, Korea Received: 29 June 2020; Accepted: 5 August 2020; Published: 1 September 2020 Abstract: In this paper, emerging memory devices are investigated for a promising synaptic device of neuromorphic computing. Because the neuromorphic computing hardware requires high memory density, fast speed, and low power as well as a unique characteristic that simulates the function of learning by imitating the process of the human brain, memristor devices are considered as a promising candidate because of their desirable characteristic. Among them, Phase-change RAM (PRAM) Resistive RAM (ReRAM), Magnetic RAM (MRAM), and Atomic Switch Network (ASN) are selected to review. Even if the memristor devices show such characteristics, the inherent error by their physical properties needs to be resolved. This paper suggests adopting an approximate computing approach to deal with the error without degrading the advantages of emerging memory devices. Keywords: neuromorphic computing; memristor; PRAM; ReRAM; MRAM; ASN; synaptic device 1. Introduction Artificial Intelligence (AI), called machine intelligence, has been researched over the decades, and the AI now becomes an essential part of the industry [1–3]. Artificial intelligence means machines that imitate the cognitive functions of humans such as learning and problem solving [1,3]. Similarly, neuromorphic computing has been researched that imitates the process of the human brain compute and store [4–6].
    [Show full text]
  • Static and Runtime API Usage Analysis on .NET Diplomarbeit
    Fachbereich 4: Informatik Static and Runtime API Usage Analysis on .NET Diplomarbeit zur Erlangung des Grades eines Diplom-Informatikers im Studiengang Informatik vorgelegt von Rufus Linke Erstgutachter: Prof. Dr. Ralf Lämmel Institut für Informatik, AG Softwaresprachen Zweitgutachter: Dipl. Math. Ekaterina Pek Institut für Informatik, AG Softwaresprachen Koblenz, im März 2011 IchERKLÄRUNG versichere, dass ich die vorliegende Arbeit selbständig verfasst und keine anderen als die angegebenen Quellen und Hilfsmittel benutzt habe. Ja Nein Mit der Einstellung der Arbeit in die Bibliothek bin ich einver- standen. Der Veröffentlichung dieser Arbeit im Internet stimme ich zu. ........................................................................................ (Ort, Datum) (Unterschrift) TheABSTRACT reuse of existing software libraries and frameworks, commonly referenced by their API, has generally been established in modern software development. But as software evolves, so do APIs and thus various reasons for an existing project to migrate from one API to another can occur. To support research on such API migration, this work gathers fundamental API usage data from real world projects. Such data can be helpful to identify the most significant aspects of an API and those that need to be treated cautiously when performing transformations. The present work uses a corpus of 18 manually selected and mature open-source soft- ware projects for this purpose. The analysis methods are driven by both static and dy- namic approaches. A bytecode-based process statically analyzes the structure of a soft- ware to discover its external API feature usages. Then, a dynamic analysis takes advan- tage of instrumentation to collect data about the execution of the static feature references as well as additional information that is only available at runtime like the type of the receiver object of a virtual method call.
    [Show full text]
  • Neuromorphic Processing and Sensing
    1 Neuromorphic Processing & Sensing: Evolutionary Progression of AI to Spiking Philippe Reiter, Geet Rose Jose, Spyridon Bizmpikis, Ionela-Ancuța Cîrjilă machine learning and deep learning (MLDL) developments. Abstract— The increasing rise in machine learning and deep This accelerated pace of innovation was spurred on by the learning applications is requiring ever more computational seminal works of LeCun, Hinton and others in the 1990s on resources to successfully meet the growing demands of an always- convolutional neural networks, or CNNs. Since then, numerous, connected, automated world. Neuromorphic technologies based on more advanced learning models have been developed, and Spiking Neural Network algorithms hold the promise to implement advanced artificial intelligence using a fraction of the neural networks have become integral to industry, medicine, computations and power requirements by modeling the academia and commercial devices. From fully autonomous functioning, and spiking, of the human brain. With the vehicles to rapid facial recognition entering popular culture to proliferation of tools and platforms aiding data scientists and enumerable innovations across almost all domains, it is not an machine learning engineers to develop the latest innovations in exaggeration to claim that CNNs and their progeny have artificial and deep neural networks, a transition to a new paradigm will require building from the current well-established changed both the technological and physical worlds. foundations. This paper explains the theoretical workings of CNNs can be exceedingly computationally intensive, neuromorphic technologies based on spikes, and overviews the however, as will be explained, often limiting their use to high- state-of-art in hardware processors, software platforms and performance computers and large data centres.
    [Show full text]
  • Neuromorphic Computing in Ginzburg-Landau Lattice Systems
    Neuromorphic computing in Ginzburg-Landau polariton lattice systems Andrzej Opala and Michał Matuszewski Institute of Physics, Polish Academy of Sciences, Warsaw Sanjib Ghosh and Timothy C. H. Liew Division of Physics and Applied Physics, Nanyang Technological University, Singapore < Andrzej Opala and Michał Matuszewski Institute of Physics, Polish Academy of Sciences, Warsaw Sanjib Ghosh and Timothy C. H. Liew Division of Physics and Applied Physics, Nanyang Technological University, Singapore < Machine learning with neural networks What is machine learning? Modern computers are extremely efficient in solving many problems, but some tasks are very hard to implement. ● It is difficult to write an algorithm that would recognize an object from different viewpoints or in di erent sceneries, lighting conditions etc. ● It is difficult to detect a fradulent credit card transaction. !e need to combine a large number of weak rules with complex dependencies rather than follow simple and reliable rules. "hese tasks are often relatively easy for humans. Machine learning with neural networks Machine learning algorithm is not written to solve a speci#c task, but to learn how to classify / detect / predict on its own. ● A large number of examples is collected and used as an input for the algorithm during the teaching phase. ● "he algorithm produces a program &eg. a neural network with #ne tuned weights), which is able to do the job not only on the provided samples, but also on data that it has never seen before. It usually contains a great number of parameters. Machine learning with neural networks "hree main types of machine learning: 1.Supervised learning – direct feedback, input data is provided with clearly de#ned labels.
    [Show full text]