Explainable Deep Learning Models in Medical Image Analysis
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Data Science Initiative 1
Data Science Initiative 1 3 credits in data and computational science, 1 credit in societal implications and opportunities, Data Science Initiative 1 elective credit to be drawn from a wide range of focused applications or deeper theoretical exploration, and 1 credit capstone experience. Director We also offer an option as a 5-th Year Master's Program if you are an Sohini Ramachandran undergraduate at Brown. This allows you to substitute maximally 2 credits Direfctor of Graduate Studies with courses you have already taken. Samuel S. Watson Master of Science in Data Science Brown University's Data Science Initiative serves as a campus hub Semester I for research and education in data science. Engaging partners across DATA 1010 Probability, Statistics, and Machine 2 campus and beyond, the DSI 's mission is to facilitate and conduct both Learning domain-driven and fundamental research in data science, increase data DATA 1030 Hands-on Data Science 1 fluency and educate the next generation of data scientists, and ultimately DATA 1050 Data Engineering 1 explore the impact of the data revolution on culture, society, and social justice. We envision our role in the university and beyond as something to Semester II build over time, with the flexibility to meet the changing needs of Brown’s DATA 2020 Statistical Learning 1 students and research community. DATA 2040 Deep Learning and Special Topics in Data 1 The Master’s Program in Data Science (Master of Science, Science ScM) prepares students from a wide range of disciplinary backgrounds DATA 2080 Data and Society 1 for distinctive careers in data science. -
Disentangled Variational Auto-Encoder for Semi-Supervised Learning
Information Sciences 482 (2019) 73–85 Contents lists available at ScienceDirect Information Sciences journal homepage: www.elsevier.com/locate/ins Disentangled Variational Auto-Encoder for semi-supervised learning ∗ Yang Li a, Quan Pan a, Suhang Wang c, Haiyun Peng b, Tao Yang a, Erik Cambria b, a School of Automation, Northwestern Polytechnical University, China b School of Computer Science and Engineering, Nanyang Technological University, Singapore c College of Information Sciences and Technology, Pennsylvania State University, USA a r t i c l e i n f o a b s t r a c t Article history: Semi-supervised learning is attracting increasing attention due to the fact that datasets Received 5 February 2018 of many domains lack enough labeled data. Variational Auto-Encoder (VAE), in particu- Revised 23 December 2018 lar, has demonstrated the benefits of semi-supervised learning. The majority of existing Accepted 24 December 2018 semi-supervised VAEs utilize a classifier to exploit label information, where the param- Available online 3 January 2019 eters of the classifier are introduced to the VAE. Given the limited labeled data, learn- Keywords: ing the parameters for the classifiers may not be an optimal solution for exploiting label Semi-supervised learning information. Therefore, in this paper, we develop a novel approach for semi-supervised Variational Auto-encoder VAE without classifier. Specifically, we propose a new model called Semi-supervised Dis- Disentangled representation entangled VAE (SDVAE), which encodes the input data into disentangled representation Neural networks and non-interpretable representation, then the category information is directly utilized to regularize the disentangled representation via the equality constraint. -
Data Science (ML-DL-Ai)
Data science (ML-DL-ai) Statistics Multiple Regression Model Building and Evaluation Introduction to Statistics Model post fitting for Inference Types of Statistics Examining Residuals Analytics Methodology and Problem- Regression Assumptions Solving Framework Identifying Influential Observations Populations and samples Detecting Collinearity Parameter and Statistics Uses of variable: Dependent and Categorical Data Analysis Independent variable Describing categorical Data Types of Variable: Continuous and One-way frequency tables categorical variable Association Cross Tabulation Tables Descriptive Statistics Test of Association Probability Theory and Distributions Logistic Regression Model Building Picturing your Data Multiple Logistic Regression and Histogram Interpretation Normal Distribution Skewness, Kurtosis Model Building and scoring for Outlier detection Prediction Introduction to predictive modelling Inferential Statistics Building predictive model Scoring Predictive Model Hypothesis Testing Introduction to Machine Learning and Analytics Analysis of variance (ANOVA) Two sample t-Test Introduction to Machine Learning F-test What is Machine Learning? One-way ANOVA Fundamental of Machine Learning ANOVA hypothesis Key Concepts and an example of ML ANOVA Model Supervised Learning Two-way ANOVA Unsupervised Learning Regression Linear Regression with one variable Exploratory data analysis Model Representation Hypothesis testing for correlation Cost Function Outliers, Types of Relationship, -
A Deep Hierarchical Variational Autoencoder
NVAE: A Deep Hierarchical Variational Autoencoder Arash Vahdat, Jan Kautz NVIDIA {avahdat, jkautz}@nvidia.com Abstract Normalizing flows, autoregressive models, variational autoencoders (VAEs), and deep energy-based models are among competing likelihood-based frameworks for deep generative learning. Among them, VAEs have the advantage of fast and tractable sampling and easy-to-access encoding networks. However, they are cur- rently outperformed by other models such as normalizing flows and autoregressive models. While the majority of the research in VAEs is focused on the statistical challenges, we explore the orthogonal direction of carefully designing neural archi- tectures for hierarchical VAEs. We propose Nouveau VAE (NVAE), a deep hierar- chical VAE built for image generation using depth-wise separable convolutions and batch normalization. NVAE is equipped with a residual parameterization of Normal distributions and its training is stabilized by spectral regularization. We show that NVAE achieves state-of-the-art results among non-autoregressive likelihood-based models on the MNIST, CIFAR-10, CelebA 64, and CelebA HQ datasets and it provides a strong baseline on FFHQ. For example, on CIFAR-10, NVAE pushes the state-of-the-art from 2.98 to 2.91 bits per dimension, and it produces high-quality images on CelebA HQ as shown in Fig. 1. To the best of our knowledge, NVAE is the first successful VAE applied to natural images as large as 256×256 pixels. The source code is available at https://github.com/NVlabs/NVAE. 1 Introduction The majority of the research efforts on improving VAEs [1, 2] is dedicated to the statistical challenges, such as reducing the gap between approximate and true posterior distributions [3, 4, 5, 6, 7, 8, 9, 10], formulating tighter bounds [11, 12, 13, 14], reducing the gradient noise [15, 16], extending VAEs to discrete variables [17, 18, 19, 20, 21, 22, 23], or tackling posterior collapse [24, 25, 26, 27]. -
Data Science: the Big Picture Data Science with R Exploratory Data Analysis with R Data Visualization with R (3-Part)
Deep Learning: The Future of AI @MatthewRenze #DevoxxUK Human Cat Dog Car Job Postings for Machine Learning Source: Indeed.com Average Salary by Job Type (USA) $108,000 $101,000 $100,000 Source: Stack Overflow 2017 What is deep learning? What can it do for me? How do I get started? What is deep learning? Deep Learning Deep Learning Artificial intelligence Machine learning Neural network Multiple hidden layers Hierarchical representations Makes predictions with data Deep Learning Artificial intelligence Machine learning Neural network Multiple hidden layers Hierarchical representations Makes predictions with data Artificial Machine Deep Intelligence Learning Learning Artificial Intelligence Explicit programming Explicit programming Encoding domain knowledge Explicit programming Encoding domain knowledge Statistical patterns detection Machine Learning Machine Learning Artificial Machine Statistics Intelligence Learning 푓 푥 푓 푥 Prediction Data Function 푓 푥 Prediction Data Function Cat Dog 푓 푥 Prediction Data Function Cat Dog Is cat? 푓 푥 Prediction Data Function Cat Dog Is cat? Yes Artificial Neuron 푥1 푥2 푦 푥3 inputs neuron outputs Artificial Neuron Σ Artificial Neuron Σ Artificial Neuron 휔1 휔2 휔3 Artificial Neuron 휔0 Artificial Neuron 휔0 휔1 휔2 휔3 Artificial Neuron 푥 1 휔0 휔1 푥2 휑 휔2 Σ 푦 푚 휔3 푥3 푦푘 = 휑 푤푘푗푥푗 푗=0 Artificial Neural Network Artificial Neural Network input hidden output Artificial Neural Network Forward propagation Artificial Neural Network Forward propagation Backward propagation Artificial Neural Network Artificial Neural Network -
Double Backpropagation for Training Autoencoders Against Adversarial Attack
1 Double Backpropagation for Training Autoencoders against Adversarial Attack Chengjin Sun, Sizhe Chen, and Xiaolin Huang, Senior Member, IEEE Abstract—Deep learning, as widely known, is vulnerable to adversarial samples. This paper focuses on the adversarial attack on autoencoders. Safety of the autoencoders (AEs) is important because they are widely used as a compression scheme for data storage and transmission, however, the current autoencoders are easily attacked, i.e., one can slightly modify an input but has totally different codes. The vulnerability is rooted the sensitivity of the autoencoders and to enhance the robustness, we propose to adopt double backpropagation (DBP) to secure autoencoder such as VAE and DRAW. We restrict the gradient from the reconstruction image to the original one so that the autoencoder is not sensitive to trivial perturbation produced by the adversarial attack. After smoothing the gradient by DBP, we further smooth the label by Gaussian Mixture Model (GMM), aiming for accurate and robust classification. We demonstrate in MNIST, CelebA, SVHN that our method leads to a robust autoencoder resistant to attack and a robust classifier able for image transition and immune to adversarial attack if combined with GMM. Index Terms—double backpropagation, autoencoder, network robustness, GMM. F 1 INTRODUCTION N the past few years, deep neural networks have been feature [9], [10], [11], [12], [13], or network structure [3], [14], I greatly developed and successfully used in a vast of fields, [15]. such as pattern recognition, intelligent robots, automatic Adversarial attack and its defense are revolving around a control, medicine [1]. Despite the great success, researchers small ∆x and a big resulting difference between f(x + ∆x) have found the vulnerability of deep neural networks to and f(x). -
Generative Models
Lecture 11: Generative Models Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 11 - 1 May 9, 2019 Administrative ● A3 is out. Due May 22. ● Milestone is due next Wednesday. ○ Read Piazza post for milestone requirements. ○ Need to Finish data preprocessing and initial results by then. ● Don't discuss exam yet since people are still taking it. Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 11 -2 May 9, 2019 Overview ● Unsupervised Learning ● Generative Models ○ PixelRNN and PixelCNN ○ Variational Autoencoders (VAE) ○ Generative Adversarial Networks (GAN) Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 11 - 3 May 9, 2019 Supervised vs Unsupervised Learning Supervised Learning Data: (x, y) x is data, y is label Goal: Learn a function to map x -> y Examples: Classification, regression, object detection, semantic segmentation, image captioning, etc. Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 11 - 4 May 9, 2019 Supervised vs Unsupervised Learning Supervised Learning Data: (x, y) x is data, y is label Cat Goal: Learn a function to map x -> y Examples: Classification, regression, object detection, Classification semantic segmentation, image captioning, etc. This image is CC0 public domain Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 11 - 5 May 9, 2019 Supervised vs Unsupervised Learning Supervised Learning Data: (x, y) x is data, y is label Goal: Learn a function to map x -> y Examples: Classification, DOG, DOG, CAT regression, object detection, semantic segmentation, image Object Detection captioning, etc. This image is CC0 public domain Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 11 - 6 May 9, 2019 Supervised vs Unsupervised Learning Supervised Learning Data: (x, y) x is data, y is label Goal: Learn a function to map x -> y Examples: Classification, GRASS, CAT, TREE, SKY regression, object detection, semantic segmentation, image Semantic Segmentation captioning, etc. -
Reinforcement Learning Data Science Africa 2018 Abuja, Nigeria (12 Nov - 16 Nov 2018)
Reinforcement Learning Data Science Africa 2018 Abuja, Nigeria (12 Nov - 16 Nov 2018) Chika Yinka-Banjo, PhD Ayorkor Korsah, PhD University of Lagos Ashesi University Nigeria Ghana Outline • Introduction to Machine learning • Reinforcement learning definitions • Example reinforcement learning problems • The Markov decision process • The optimal policy • Value function & Q-value function • Bellman Equation • Q-learning • Building a simple Q-learning agent (coding) • Recap • Where to go from here? Introduction to Machine learning • Artificial Intelligence (AI) is the study and design of Intelligent agents. • An Intelligent agent can perceive its environment through sensors and it can act on its environment through actuators. • E.g. Agent: Humanoid robot • Environment: Earth? • Sensors: Camera, tactile sensor etc. • Actuators: Motors, grippers etc. • Machine learning is a subfield of Artificial Intelligence Branches of AI Introduction to Machine learning • Machine learning techniques learn from data without being explicitly programmed to do so. • Machine learning models enable the agent to learn from its own experience by extracting useful information from feedback from its environment. • Three types of learning feedback: • Supervised learning • Unsupervised learning • Reinforcement learning Branches of Machine learning Supervised learning • Supervised learning: the machine learning model is trained on many labelled examples of input-output pairs. • Such that when presented with a novel input, the model can estimate accurately what the correct output should be. • Data(x, y): x is input data, y is label Supervised learning task in the form of classification • Goal: learn a function to map x -> y • Examples include; Classification, regression object detection, image captioning etc. Unsupervised learning • Unsupervised learning: here the model extract useful information from unlabeled and unstructured data. -
Infinite Variational Autoencoder for Semi-Supervised Learning
Infinite Variational Autoencoder for Semi-Supervised Learning M. Ehsan Abbasnejad Anthony Dick Anton van den Hengel The University of Adelaide {ehsan.abbasnejad, anthony.dick, anton.vandenhengel}@adelaide.edu.au Abstract with whatever labelled data is available to train a discrimi- native model for classification. This paper presents an infinite variational autoencoder We demonstrate that our infinite VAE outperforms both (VAE) whose capacity adapts to suit the input data. This the classical VAE and standard classification methods, par- is achieved using a mixture model where the mixing coef- ticularly when the number of available labelled samples is ficients are modeled by a Dirichlet process, allowing us to small. This is because the infinite VAE is able to more ac- integrate over the coefficients when performing inference. curately capture the distribution of the unlabelled data. It Critically, this then allows us to automatically vary the therefore provides a generative model that allows the dis- number of autoencoders in the mixture based on the data. criminative model, which is trained based on its output, to Experiments show the flexibility of our method, particularly be more effectively learnt using a small number of samples. for semi-supervised learning, where only a small number of The main contribution of this paper is twofold: (1) we training samples are available. provide a Bayesian non-parametric model for combining autoencoders, in particular variational autoencoders. This bridges the gap between non-parametric Bayesian meth- 1. Introduction ods and the deep neural networks; (2) we provide a semi- supervised learning approach that utilizes the infinite mix- The Variational Autoencoder (VAE) [18] is a newly in- ture of autoencoders learned by our model for prediction troduced tool for unsupervised learning of a distribution x x with from a small number of labeled examples. -
An Artificial Neural Networks Primer with Financial Applications Examples in Financial Distress Predictions and Foreign Exchange Hybrid Trading System ’
‘An Artificial Neural Networks Primer with Financial Applications Examples in Financial Distress Predictions and Foreign Exchange Hybrid Trading System ’ by Dr Clarence N W Tan, PhD Bachelor of Science in Electrical Engineering Computers (1986), University of Southern California, Los Angeles, California, USA Master of Science in Industrial and Systems Engineering (1989), University of Southern California Los Angeles, California, USA Masters of Business Administration (1989) University of Southern California Los Angeles, California, USA Graduate Diploma in Applied Finance and Investment (1996) Securities Institute of Australia Diploma in Technical Analysis (1996) Australian Technical Analysts Association Doctor of Philosophy Bond University (1997) URL: http://w3.to/ctan/ E-mail: [email protected] School of Information Technology, Bond University, Gold Coast, QLD 4229, Australia Table of Contents Table of Contents 1. INTRODUCTION TO ARTIFICIAL INTELLIGENCE AND ARTIFICIAL NEURAL NETWORKS ..........................................................................................................................................2 1.1 INTRODUCTION ...........................................................................................................................2 1.2 ARTIFICIAL INTELLIGENCE ..........................................................................................................2 1.3 ARTIFICIAL INTELLIGENCE IN FINANCE .......................................................................................4 1.3.1 Expert System -
Lecture 23: Recurrent Neural Network [SCS4049-02] Machine Learning and Data Science
Lecture 23: Recurrent Neural Network [SCS4049-02] Machine Learning and Data Science Seongsik Park ([email protected]) AI Department, Dongguk University Tentative schedule week topic date (수 / 월) 1 Machine Learning Introduction & Basic Mathematics 09.02 / 09.07 2 Python Practice I & Regression 09.09 / 09.14 3 AI Department Seminar I & Clustering I 09.16 / 09.21 4 Clustering II & Classification I 09.23 / 09.28 5 Classification II (추석) / 10.05 6 Python Practice II & Support Vector Machine I 10.07 / 10.12 7 Support Vector Machine II & Decision Tree and Ensemble Learning 10.14 / 10.19 8 Mid-Term Practice & Mid-Term Exam 10.21 / 10.26 9 휴강 & Dimensional Reduction I 10.28 / 11.02 10 Dimensional Reduction II & Neural networks and Back Propagation I 11.04 / 11.09 11 Neural networks and Back Propagation II & III 11.11 / 11.16 12 AI Department Seminar II & Convolutional Neural Network 11.18 / 11.23 13 Python Practice III & Recurrent Neural network 11.25 / 11.30 14 Model Optimization & Autoencoders 12.02 / 12.07 15 Final exam (휴강)/ 12.14 1/22 Introduction of recurrent neural network (RNN) • Rumelhart, et. al., Learning Internal Representations by Error Propagation (1986) • fx(1); :::; x(t−1); x(t); x(t+1); :::; x(N)g와 같은 sequence를 처리하는데 특화 • RNN 은 긴 시퀀스를 처리할 수 있으며, 길이가 변동적인 시퀀스도 처리 가능 • Parameter sharing: 전체 sequence에서 파라미터를 공유함 • CNN은 시퀀스를 다룰 수 없나? • 시간 축으로 1-D convolution ) 시간 지연 신경망도 가능하지만 깊이가 얕음 (shallow) • RNN은 깊은 구조(deep)가 가능함 • Data type • 일반적으로 xt 는 시계열 데이터 (time series, temporal data) • 여기서 t = 1; 2; :::; N은 time step 또는 sequence 내에서의 순서 • 전체 길이 N은 변동적 2/22 Recurrent Neurons Up to now we have mostly looked at feedforward neural networks, where the activa‐ tions flow only in one direction, from the input layer to the output layer (except for a few networks in Appendix E). -
Variational Autoencoder for End-To-End Control of Autonomous Driving with Novelty Detection and Training De-Biasing
Variational Autoencoder for End-to-End Control of Autonomous Driving with Novelty Detection and Training De-biasing Alexander Amini1, Wilko Schwarting1, Guy Rosman2, Brandon Araki1, Sertac Karaman3, Daniela Rus1 Abstract— This paper introduces a new method for end-to- Uncertainty Estimation end training of deep neural networks (DNNs) and evaluates & Novelty Detection it in the context of autonomous driving. DNN training has Uncertainty been shown to result in high accuracy for perception to action Propagate learning given sufficient training data. However, the trained models may fail without warning in situations with insufficient or biased training data. In this paper, we propose and evaluate Encoder a novel architecture for self-supervised learning of latent variables to detect the insufficiently trained situations. Our method also addresses training data imbalance, by learning a set of underlying latent variables that characterize the training Dataset Debiasing data and evaluate potential biases. We show how these latent Weather Adjacent Road Steering distributions can be leveraged to adapt and accelerate the (Snow) Vehicles Surface Control training pipeline by training on only a fraction of the total Command Resampled data dataset. We evaluate our approach on a challenging dataset distribution for driving. The data is collected from a full-scale autonomous Unsupervised Latent vehicle. Our method provides qualitative explanation for the Variables Sample Efficient latent variables learned in the model. Finally, we show how Accelerated Training our model can be additionally trained as an end-to-end con- troller, directly outputting a steering control command for an Fig. 1: Semi-supervised end-to-end control. An encoder autonomous vehicle.