DOCSLIB.ORG

Machine Learning for Numerical Stabilization of Advection-Diffusion

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Machine Learning for Numerical Stabilization of Advection-Diffusion Machine learning for numerical stabilization of advection-diffusion PDEs Courses: Numerical Analysis for Partial Differential Equations - Advanced Programming for Scientific Computing Margherita Guido, Michele Vidulis Suprevisor: prof. Luca Ded`e 09/09/2019 Contents 1 Introduction5 2 Problem: SUPG stabilization and Isogeometric analysis7 2.1 Advection-diffusion problem.......................7 2.1.1 Weak formulation and numerical discretization using FE...8 2.2 SUPG stabilization............................9 2.2.1 A Variational Multiscale (VMS) approach...........9 2.3 Isogeometric analysis........................... 11 2.3.1 B-Spline basis functions..................... 11 2.3.2 B-Spline geometries....................... 11 2.3.3 NURBS basis functions..................... 12 2.3.4 NURBS as trial space for the solution of the Advection-Diffusion problem.............................. 12 2.3.5 Mesh refinement and convergence results............ 12 3 Artificial Neural Networks 15 3.1 Structure of an Artificial Neural Network............... 15 3.1.1 Notation.............................. 15 3.1.2 Design of an ANN........................ 16 3.2 Universal approximation property.................... 17 3.3 Backpropagation and training...................... 17 3.3.1 Some terminology........................ 17 3.3.2 The learning algorithm..................... 18 4 A Neural Network to learn the stabilization parameter 21 4.1 Our Neural Network scheme....................... 21 4.2 Mathematical formulation of the problem............... 22 4.3 Expected results............................. 24 4.4 Implementation aspects......................... 24 4.4.1 Keras............................... 24 4.4.2 C++ libraries........................... 25 5 Isoglib and OpenNN 27 5.1 Structure of IsoGlib........................... 27 5.1.1 Definition of the problem.................... 27 5.1.2 Main steps in solving process.................. 28 5.1.3 Export and visualization of the results............. 28 5.2 Structure of OpenNN.......................... 28 5.2.1 Uses of the library........................ 28 5.2.2 Vectors and matrices....................... 29 5.2.3 DataSet class........................... 29 5.2.4 NeuralNetwork class....................... 30 5.2.5 TrainingStrategy class..................... 30 5.2.6 LossIndex class......................... 30 5.2.7 OptimizationAlgorithm class................. 31 5.2.8 Backpropagation......................... 31 3 4 CONTENTS 6 Implementation: interface of OpenNN and IsoGlib 35 6.1 SUPG solver in IsoGlib......................... 35 6.1.1 Our test class: SUPGdata .................... 35 6.1.2 SUPGLocalMatrix class..................... 36 6.2 New OpenNN classes........................... 36 6.2.1 IsoglibInterface class..................... 36 6.2.2 Customized loss function: OutputFunction class....... 38 6.3 SUPG example in OpenNN....................... 42 6.3.1 Data of the problem....................... 42 6.3.2 Training.............................. 42 6.3.3 Testing.............................. 43 7 Numerical Tests 45 7.1 Training of the network......................... 46 7.2 Prediction of the SUPG parameter................... 47 7.3 Analysis of the results.......................... 48 7.4 L2 error trend in a different PDE.................... 50 8 Conclusions 53 Bibliography 55 Chapter 1 Introduction High-order numerical methods for the approximation of advection–diffusion PDEs based on Galerkin finite element may be affected from numerical instabilities, which occur as spurious oscillations and compromise the solution accuracy. A stabilized and strongly consistent method named Streamline Upwind Petrov- Galerkin Method (SUPG) can be obtained by adding a further term to the Galerkin approximation (see (HB82) and (Q17, chap.13)). This term includes an elementwise stabilization parameter that should be carefully determined, since an exact formula defining its optimal value is still lacking. Through the years, several approximations of this coefficient have been proposed: we present the simplest and most used, trying to frame the ideas that lead to their definition (see (HSF18) and (C97)). The aim of our work is to exploit the universal approximation power of the Arti- ficial Neural Networks to reconstruct a suitable approximation of such stabilization parameter. We train an Artificial Neural Network, implemented inside the C++ li- brary OpenNN (OpenNN), so that, once the parameters characterizing the problem and its discretization are known, we predict an optimal value of the stabilization parameter. To train our network we developed a customized loss function that re- quires several numerical resolution of the stabilized PDE, for which we employ a C++ library (IsoGlib), based on Isogeometric analysis (CHB09). We start presenting the problem in its theoretical framework, describing the family of PDE equations we want to solve, some instability issues and how they can be confined, constructing a theoretical background for this problem. In Chapter2 we will also introduce Isogeometric Analysis, the technique that will be used to solve numerically the PDEs problems. Chapter3 is devoted to the description of Artifi- cial Neural Networks and the algorithm involved in their training, with particular attention to the description of how we designed our customized Neural Network. In Chapter4 our original problem is formalized, with the purpose of putting in com- munication the world of EDPs with the one of ANNs, and describing the effort we made to realize our project. Chapter5 contains a description of the libraries used for the implementation (OpenNN and IsoGlib), while in Chapter6 we will present the classes we developed to interface them. Numerical results are presented in Chapter 7 to demonstrate the performance of this network-based technique and results are critically discussed comparing them with the known formulas. At the end (Chapter 8) , possible further developments of this technique are presented. Indeed we start from the implementation of a neural network that works on a simplified version of the PDE problem but, once this network is tested, we already have in mind how to customize it for more complicated problems. 5 Chapter 2 Problem: SUPG stabilization and Isogeometric analysis 2.1 Advection-diffusion problem Advection–diffusion PDEs (see (Q17, chap.13)) are used to model a wide range of phenomena such as semiconductor devices modeling, magnetostatics and electro- static flows, heat and mass-transfer and flows in porous media related to oil and groundwater applications. The linear advection–diffusion equation on a domain d Ω Ď R “ 2; 3 is a boundary value problem of the form: Lu “ ´divpµpxqruq ` bpxq ¨ ru “ f in Ω (2.1) #u “ 0 on BΩ where µpxq and bpxq are respectively the diffusion and advection coefficent. In many practical applications, the diffusion term ´divpµpxqruq is dominated by the transport term bpxq ¨ ru. In such cases the solution can give rise to boundary layers, namely regions, generally close to the boundary of Ω, where the solution is characterized by strong gradients. From a numerical point of view, solving these types of advection-dominated problems using Galerkin Finite Element method leads to nonphysical oscillation near the boundary layers. An example of this phe- nomenon can be seen in 2.1, obtained using the following data µ “ 10´4; β “ r1; 1s in Ω “ p0; 1q2 and a forcing term f such that the exact solution is given by: px´0:5q2`py´0:5q2´ 1 ? 16 uex “ ´atan µ A challenging problem is to find numerical scheme that works´ well in all types¯ of regime, without increasing the computational effort to solve the PDE problem. The goal of this project is to find an innovative way to code the SUPG stabi- lization method, exploiting Artificial Neural Networks to select the best value of the stabilization parameter. 7 8CHAPTER 2. PROBLEM: SUPG STABILIZATION AND ISOGEOMETRIC ANALYSIS Figure 2.1: Numerical oscillations in 2D 2.1.1 Weak formulation and numerical discretization using FE 1 Let V “ H0 pΩq be the reference Sobolev space, let a : V ˆ V Ñ R be the bilinear form apu; vq “ µru ¨ rv dΩ ` b ¨ ruv dΩ »Ω »Ω and let F : V Ñ R be the linear functional F pvq “ fv dΩ »Ω The weak formulation of problem (2.1) reads Find u P V : apu; vq “ F pvq @v P V (2.2) The existence of a unique solution follows from the Lax-Milgram Theorem. By choosing a suitable family tVh; h ¡ 0u of finite dimensional subspaces of V we can state the Galerkin formulation of the problem: Find uh P Vh : apuh; vhq “ F pvhq @vh P Vh (2.3) that is well posed thanks to the previous analysis. Moreover the Galerkin error inequality gives: M ku ´ u k ¤ inf kv ´ v k (2.4) h V α vhPVh h V µ0 where α “ 2 and M “ kµkL8pΩq ` kbkL8pΩq are the coercivity and continuity 1`CΩ constant of ap¨; ¨q, given µ0 the lower bound for the diffusive coefficient µ. M By the definitions of α and M, the upper-bounding constant α of the error becomes larger (and, correspondingly, the estimate 2.4 meaningless) as the ratio kbk L8pΩq grows and tends to infinity. This happens in the advection dominated kµkL8pΩq regime: in such cases the Galerkin method can give inaccurate solutions, presenting numerical oscillations. The dimensionless coefficient that measures the relation between transport and diffusion is called Peclet number and is defined as |b|h e “ (2.5) P 2µ being h the dimension of the mesh. To understand the significance of the Peclet number it can be useful to look at the 1D problem: ´µu2 ` bu1 “ 0; 0 ă x ă 1 #up0q “ 0; up1q “ 1 2.2. SUPG STABILIZATION 9 exppbx{µq´1 The exact solution of this problem is given by upxq “ exppb{µq´1 . When b " µ the solution is close to zero on almost all of the interval, except for the boundary layer that generates in a neighborhood of x “ 1. It can be shown that the Finite Element solution is monotone if and only if the condition Pe ¤ 1 is satisfied: in practice, in order to avoid oscillations h has to be chosen small enough to solve the boundary layer, which, in turn, becomes thinner as the ratio b{µ tends to infinity. In figure 2.2 different trends of the solution to vary of the magnitude of the Peclet number.
Load more

Recommended publications
  • March 2012 Version 2
    WWRF VIP WG CONNECTED VEHICLES White Paper Connected Vehicles: The Role of Emerging Standards, Security and Privacy, and Machine Learning Editor: SESHADRI MOHAN, CHAIR CONNECTED VEHICLES WORKING GROUP PROFESSOR, SYSTEMS ENGINEERING DEPARTMENT UA LITTLE ROCK, AR 72204, USA Project website address: www.wwrf.ch This publication is partly based on work performed in the framework of the WWRF. It represents the views of the authors(s) and not necessarily those of the WWRF. EXECUTIVE SUMMARY The Internet of Vehicles (IoV) is an emerging technology that provides secure vehicle-to-vehicle (V2V) communication and safety for drivers and their passengers. It stands at the confluence of many evolving disciplines, including: evolving wireless technologies, V2X standards, the Internet of Things (IoT) consisting of a multitude of sensors that are housed in a vehicle, on the roadside, and in the devices worn by pedestrians, the radio technology along with the protocols that can establish an ad-hoc vehicular network, the cloud technology, the field of Big Data, and machine intelligence tools. WWRF is presenting this white paper inspired by the developments that have taken place in recent years in standards organizations such as IEEE and 3GPP and industry consortia efforts as well as current research in academia. This white paper provides insights into the state-of-the-art regarding 3GPP C-V2X as well as security and privacy of ETSI ITS, IEEE DSRC WAVE, 3GPP C-V2X. The White Paper further discusses spectrum allocation worldwide for ITS applications and connected vehicles. A section is devoted to a discussion on providing connected vehicles communication over a heterogonous set of wireless access technologies as it is imperative that the connectivity of vehicles be maintained even when the vehicles are out of coverage and/or the need to maintain vehicular connectivity as a vehicle traverses multiple wireless access technology for access to V2X applications.
    [Show full text]
  • Giant List of AI/Machine Learning Tools & Datasets
    Giant List of AI/Machine Learning Tools & Datasets AI/machine learning technology is growing at a rapid pace. There is a great deal of active research & big tech is leading the way. Luckily there are also a lot of resources out there for the technologist to utilize. So many we had to cherry pick what look like the most legit & useful tools. 1. Accord Framework http://accord-framework.net 2. Aligned Face Dataset from Pinterest (CCO) https://www.kaggle.com/frules11/pins-face-recognition 3. Amazon Reviews Dataset https://snap.stanford.edu/data/web-Amazon.html 4. Apache SystemML https://systemml.apache.org 5. AWS Open Data https://registry.opendata.aws 6. Baidu Apolloscapes http://apolloscape.auto 7. Beijing Laboratory of Intelligent Information Technology Vehicle Dataset http://iitlab.bit.edu.cn/mcislab/vehicledb 8. Berkley Caffe http://caffe.berkeleyvision.org 9. Berkley DeepDrive https://bdd-data.berkeley.edu 10. Caltech Dataset http://www.vision.caltech.edu/html-files/archive.html 11. Cats in Movies Dataset https://public.opendatasoft.com/explore/dataset/cats-in-movies/information 12. Chinese Character Dataset http://www.iapr- tc11.org/mediawiki/index.php?title=Harbin_Institute_of_Technology_Opening_Recognition_Corpus_for_Chinese_Characters_(HIT- OR3C) 13. Chinese Text in the Wild Dataset (CC4.0) https://ctwdataset.github.io 14. CelebA Dataset (research only) http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html 15. Cityscapes Dataset https://www.cityscapes-dataset.com | License 16. Clash of Clans User Comments Dataset (GPL 2) https://www.kaggle.com/moradnejad/clash-of-clans-50000-user-comments 17. Core ML https://developer.apple.com/machine-learning 18. Cornell Movie Dialogs Corpus http://www.cs.cornell.edu/~cristian/Cornell_Movie-Dialogs_Corpus.html 19.
    [Show full text]
  • Deep Neural Networks Practical Examples of Deep Neural Networks
    Introduction Why to use (deep) neural networks? Types of deep neural networks Practical examples of deep neural networks Deep Neural Networks Convolutional Neural Networks René Pihlak [email protected] Department of Software Sciences School of Information Technologies Tallinn University of Technology April 29, 2019 . René Pihlak CNN Introduction Why to use (deep) neural networks? Self-introduction Types of deep neural networks Topics to cover Practical examples of deep neural networks Table of Contents 1 Introduction Types of training Self-introduction Types of structures Topics to cover Convolutional neural network 2 Why to use (deep) neural networks? 4 Practical examples of deep neural Description networks Comparision Road defect detection Popular frameworks YOLO3: darknet 3 Types of deep neural networks Estonian sign language . René Pihlak CNN 2nd year Master’s student Current: Department of Software Sciences Past: Member of Board, Tiigrihüppe SA (now HITSA) STUDIES: WORK: Introduction Why to use (deep) neural networks? Self-introduction Types of deep neural networks Topics to cover Practical examples of deep neural networks Background . René Pihlak CNN Current: Department of Software Sciences Past: Member of Board, Tiigrihüppe SA (now HITSA) 2nd year Master’s student WORK: Introduction Why to use (deep) neural networks? Self-introduction Types of deep neural networks Topics to cover Practical examples of deep neural networks Background STUDIES: . René Pihlak CNN Current: Department of Software Sciences Past: Member of Board, Tiigrihüppe SA (now HITSA) WORK: Introduction Why to use (deep) neural networks? Self-introduction Types of deep neural networks Topics to cover Practical examples of deep neural networks Background STUDIES: 2nd year Master’s student .
    [Show full text]
  • Big Data Management of Hospital Data Using Deep Learning and Block-Chain Technology: a Systematic Review
    EAI Endorsed Transactions on Scalable Information Systems Research Article Big Data Management of Hospital Data using Deep Learning and Block-chain Technology: A Systematic Review Nawaz Ejaz1, Raza Ramzan1, Tooba Maryam1,*, Shazia Saqib1 1 Department of Computer Science, Lahore Garrison University, Lahore, Pakistan Abstract The main recompenses of remote and healthcare are sensor-based medical information gathering and remote access to medical data for real-time advice. The large volume of data coming from sensors requires to be handled by implementing deep learning and machine learning algorithms to improve an intelligent knowledge base for providing suitable solutions as and when needed. Electronic medical records (EMR) are mostly stored in a client-server database and are supported by enabling technologies like Internet of Things (IoT), Sensors, cloud, big data, Deep Learning, etc. It is accessed by several users involved like doctors, hospitals, labs, insurance providers, patients, etc. Therefore, data security from illegal access is crucial especially to manage the integrity of data . In this paper, we describe all the basic concepts involved in management and security of such data and proposed a novel system to securely manage the hospital’s big data using Deep Learning and Block-Chain technology. Keywords: Electronic medical records, big data, Security, Block-chain, Deep learning. Received on 23 December 2020, accepted on 16 March 2021, published on 23 March 2021 Copyright © 2021 Nawaz Ejaz et al., licensed to EAI. This is an open access article distributed under the terms of the Creative Commons Attribution license, which permits unlimited use, distribution and reproduction in any medium so long as the original work is properly cited.
    [Show full text]
  • Machine Learning Technique Used in Enhancing Teaching Learning Process in Education Sector and Review of Machine Learning Tools Diptiverma #1, Dr
    International Journal of Scientific & Engineering Research Volume 9, Issue 6, June-2018 ISSN 2229-5518 741 Machine Learning Technique used in enhancing Teaching Learning Process in education Sector and review of Machine Learning Tools DiptiVerma #1, Dr. K. Nirmala#2 MCA Department, SavitribaiPhule Pune University DYPIMCA, Akurdi, Pune , INDIA [email protected], [email protected] Abstract: Machine learning is one of the widely rigorous study that teaching learning process has used research area in today’s world is artificial been changing over the time. The traditional intelligence and one of the scope full area is protocol of classroom teaching has been taken to Machine Learning (ML). This is study and out of box level, where e-learning, m-learning, analysis of some machine learning techniques Flipped Classroom, Design Thinking (Case used in education for particularly increasing the Method),Gamification, Online Learning, Audio & performance in teaching learning process, along Video learning, “Real-World” Learning, with review of some machine learning tools. Brainstorming, Classes Outside the Classroom learning, Role Play technique, TED talk sessions Keyword:Machine learning, Teaching Learning, are conducted to provide education beyond text Machine Learning Tools, NLP book and syllabus. I. INTRODUCTION II. MACHINE LEARNING PROCESS: Machine Learning is a science making system Machine Learning is a science of making system (computers) to understand from the past behavior smart. Machine learning we can say has improved of data or from historic data to behave smartly in the lives of human being in term of their successful every situation like human beings do. Without implementation and accuracy in predicting having the same type of situation a human can results.[11] behave or can react to the condition smartly.
    [Show full text]
  • Essential Tools for Scientific Machine Learning and Scientific AI
    Essential Tools for Scientific Machine Learning and Scientific AI Comparison of tools readily usable with differentiable programming (automatic differentiation) frameworks Subject | AD Frameworks ADIFOR or TAF ADOL-C Stan Julia (Zygote.jl, Tracker.jl, ForwardDiff.jl, etc.) TensorFlow PyTorch Misc. other good packages Language Fortran C++ Misc. Julia Python, Swift, Julia, etc. Python Neural Networks neural-fortran OpenNN None Flux.jl Built-in Built-in ADIFOR DifferentialEquations.jl / DiffEqFlux.jl (ODE, DifferentialEquations.jl Neural Differential Equations Sundials (ODE+DAE) Sundials (ODE+DAE) Sundials (ODE+DAE) torchdiffeq (non-stiff ODEs) PyMC3 (Python) SDE, DDE, DAE, hybrid, (S)PDE) (through Tensorflow.jl) FATODE PETSc TS Built-in (non-stiff ODE) Sundials.jl (ODE through DiffEqFlux.jl) diffeqpy SMT (Python) Probabilistic Programming None CPProb Built-In Gen.jl Edward Pyro sensitivity (R) Turing.jl PyMC4 pyprob ColPACK (Fortran) Sparsity Detection Built-in (TAF) Built-in None SparsityDetection.jl None None Dakota Sparse Differentiation Built-in (TAF) Built-in None SparseDiffTools.jl None None PSUDAE GPU Support CUDA CUDA OpenCL CUDANative.jl + CuArrays.jl Built-in Built-in Mondrian torch.distributed (no Distributed Dense Linear Algebra ScaLAPACK Elemental None Elemental.jl Built-in SimLab (MATLAB) factorizations) DistributedArrays.jl Elemental Halide Distributed Sparse Linear Algebra ScaLAPACK PETSc None Elemental.jl Built-in (no factorizations) Elemental PARASOL Trilinos PETSc.jl None petsc4py Elemental None Structured Linear Algebra
    [Show full text]
  • Design and Implementation of a Domain Specific Language for Deep Learning Xiao Bing Huang University of Wisconsin-Milwaukee
    University of Wisconsin Milwaukee UWM Digital Commons Theses and Dissertations May 2018 Design and Implementation of a Domain Specific Language for Deep Learning Xiao Bing Huang University of Wisconsin-Milwaukee Follow this and additional works at: https://dc.uwm.edu/etd Part of the Artificial Intelligence and Robotics Commons, and the Electrical and Computer Engineering Commons Recommended Citation Huang, Xiao Bing, "Design and Implementation of a Domain Specific Language for Deep Learning" (2018). Theses and Dissertations. 1829. https://dc.uwm.edu/etd/1829 This Dissertation is brought to you for free and open access by UWM Digital Commons. It has been accepted for inclusion in Theses and Dissertations by an authorized administrator of UWM Digital Commons. For more information, please contact [email protected]. DESIGN AND IMPLEMENTATION OF A DOMAIN SPECIFIC LANGUAGE FOR DEEP LEARNING by Xiao Bing Huang A Dissertation Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in Engineering at The University of Wisconsin-Milwaukee May 2018 ABSTRACT DESIGN AND IMPLEMENTATION OF A DOMAIN SPECIFIC LANGUAGE FOR DEEP LEARNING by Xiao Bing Huang The University of Wisconsin-Milwaukee, 2018 Under the Supervision of Professor Tian Zhao Deep Learning (DL) has found great success in well-diversified areas such as machine vision, speech recognition, big data analysis, and multimedia understanding recently. However, the existing state-of-the-art DL frameworks, e.g. Caffe2, Theano, TensorFlow, MxNet, Torch7, and CNTK, are programming libraries with fixed user interfaces, internal represen- tations, and execution environments. Modifying the code of DL layers or data structure is very challenging without in-depth understanding of the underlying implementation.
    [Show full text]
  • Deep Learning for Image Processing in WEKA Environment
    Int. J. Advance Soft Compu. Appl, Vol. 11, No. 1, March 2019 ISSN 2074-2827 Deep Learning for Image Processing in WEKA Environment Zanariah Zainudin1, Siti Mariyam Shamsuddin2 and Shafaatunnur Hasan3 1,2,3 School of Computing, Faculty of Engineering, Universiti Teknologi Malaysia, 81310 Skudai, Johor e-mail: [email protected], [email protected], and [email protected] Abstract Deep learning is a new term that is recently popular among researchers when dealing with big data such as images, texts, voices and other types of data. Deep learning has become a popular algorithm for image processing since the last few years due to its better performance in visualizing and classifying images. Nowadays, most of the image datasets are becoming larger in terms of size and variety of the images that can lead to misclassification due to human eyes. This problem can be handled by using deep learning compared to other machine learning algorithms. There are many open sources of deep learning tools available and Waikato Environment for Knowledge Analysis (WEKA) is one of the sources which has deep learning package to conduct image classification, which is known as WEKA DeepLearning4j. In this paper, we demonstrate the systematic methodology of using WEKA DeepLearning4j for image classification on larger datasets. We hope this paper could provide better guidance in exploring WEKA deep learning for image classification. Keywords: Image Classification, WEKA DeepLearning4j, WEKA Image, Deep Learning, Convolutional Neural Network. Zainudin. Z et. al. 2 1 Introduction Nowadays, image processing is becoming a popular approach in many areas such as engineering, medical and computer science area.
    [Show full text]
  • Review on Blue Brain for Novice Kowshalya .G
    ISSN XXXX XXXX © 2017 IJESC Research Article Volume 7 Issue No.8 Review on Blue Brain for Novice Kowshalya .G. MCA Assistant Professor Department of Computer Science Sri GVG Visalakshi College for Women, Udumalpet, India Abstract: Uploading the human brain into the supercomputer is called Blue Brain. So the machine can function as human and it can take decisions. The Cognitive learning method is used for simulation here. Based on the working principle of human brain, virtual brain is modeled. Blue brain will not come under the category of Artificial intelligence (AI),it comes under the subset of AI that is deep machine learning. The more processing power is needed to here to simulate the whole brain. With advantages there also demerits associated with this technology. The major objective of the Blue Brain Project is to crack open secrets of how the brain rewires itself every moment of its existence. The resultant knowledge would lead to a new breed of super computers. Keywords: Cognitive Learning, Machine Learning, Nano robots, Pattern Recognition, Liquid Computing. I. INTRODUCTION 2.2 Functioning of human brain: Human brain is the most valuable creation of God. Because of 2.2.1 Sensory Input: death, our body and brain get destroyed. It is possible to transplant our body organs and make them alive even after our The getting of information from our surroundings is called death. It is also possible to make our brain (i.e. intelligence) to sensory input. For example, if we smell a rose or our eyes see alive even after death. Human brain gets copied into computer, something, suppose our hands touch a hot water, then these so the intelligence of anyone can be loaded into the computer.
    [Show full text]
  • High Performance Optimization Algorithms for Neural Networks
    University of Salamanca Master's Degree Physics and Mathematics End of Master's Project High performance optimization algorithms for neural networks Author: Tutors: Carlos Barranquero D´ıez Vidal Moreno Rodilla Francisco Javier Villarroel Rodr´ıguez September 3, 2019 Vidal Moreno Rodilla and Javier Villarroel Rodr´ıguez, professors of the \Dept. de Inform´aticay Autom´atica" and, respectively, \Estad´ıstica”,of the \Universidad de Salamanca", certify: That they have acted as tutors of Mr. Carlos Barranquero Diez during the studies for the Master's degree in Physics and Mathematics at the University of Salamanca, and that the mem- ory entitled \High performance optimization algorithms for neural networks" that is presented here as a final Project has been satisfactorily carried out by Mr. Carlos Barranquero Diez. Salamanca, September 3, 2019. Acknowledgement I would like to convey my warm acknowledgement towards Mr. Roberto L´opez Gonz´alez,chief executive officer in Artificial Intelligence Techniques S.L.(Artelnics), for all his support and help during the undertaking of this project and the consideration shown during my master's degree studies. Contents 1 Introduction2 1.1 State of the art..........................2 1.2 Motivation.............................4 2 Machine learning techniques6 2.1 Foundations of statistics and probability............7 2.1.1 Random variables and distributions...........7 2.1.2 Expected value and moments..............9 2.1.3 Common distributions.................. 10 2.1.4 Estimators......................... 11 2.1.5 Conditional probability distributions.......... 12 2.1.6 Machine learning approach................ 13 2.2 Optimization methods...................... 15 2.2.1 Ordinary least squares.................. 18 2.2.2 Gradient decent.....................
    [Show full text]
  • A Proposal for Enhancing Training Speed in Deep Learning Models Based on Memory Activity Survey
    This article has been accepted and published on J-STAGE in advance of copyediting. Content is final as presented. IEICE Electronics Express, Vol.VV, No.NN, 1–6 LETTER A Proposal for Enhancing Training Speed in Deep Learning Models Based on Memory Activity Survey Dang Tuan Kiet1a), Binh Kieu-Do-Nguyen1b), Trong-Thuc Hoang1c), Khai-Duy Nguyen1d), Xuan-Tu Tran2e), and Cong-Kha Pham1f) Abstract forms such as mobile devices or embedded systems [1]. Deep Learning (DL) training process involves intensive computations that However, it is not the case for the training process. The require a large number of memory accesses. There are many surveys on training process is still a data-intensive challenge for many memory behaviors with the DL training. They use well-known profiling tools or improving the existing tools to monitor the training processes. computing systems. It is characterized by millions of para- This paper presents a new approach to profile using a co-operate solution meters with billions of data transactions to compute. There- from software and hardware. The idea is to use Field-Programmable-Gate- fore, besides improving the training algorithms or models, Array memory as the main memory for the DL training processes on a memory access enhancement is another approach to speed computer. Then, the memory behaviors from both software and hardware up such a process. point-of-views can be monitored and evaluated. The most common DL models are selected for the tests, including ResNet, VGG, AlexNet, and DL training is a data-intensive process that requires millions GoogLeNet. The CIFAR-10 dataset is chosen for the training database.
    [Show full text]
  • Tensors All Around Us by Branimir K
    DATA ANALYSIS IN MEDICAL RESEARCH: FROM FOE TO FRIEND 369 Croat Med J. 2019;60:369-74 https://doi.org/10.3325/cmj.2019.60.369 Tensors all around us by Branimir K. Hackenberger Department of Biology, Josip Juraj Strossmayer University of Osijek, Osijek, Croatia [email protected] Advanced artificial intelligence (AI) technologies have be- experienced a surge in popularity and are significantly come so integrated in our daily lives that they no longer represented in all areas that use data processing. Howev- fascinate us. Online database search engines, image search er, this phenomenon may be a topic of another column. engines with embedded recognition systems, device man- The openness of Python and R enabled the rapid imple- agement by voice, have all become normal and common. mentation of AI technology, as well as its utilization in a However, every time I use Shazam, I am amazed at its speed range of applications. A crucial step that allowed this to in identifying music, its version and author. Sometimes in happen occurred in 2015, when Google Brain, Google’s AI less than a second! research team, released a free and open-source software library named TensorFlow. In 2017, its first stable version It seems impossible to keep up with all the trends when it became available. The most important parts of the library, comes to the technology of data processing. But maybe we computational ones, are written in C ++, while Python is should not bother. The computers will follow the trends for used as an excellent higher-level language management us, using AI algorithms.
    [Show full text]