DOCSLIB.ORG

NIPS 2017 Workshop Book

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

NIPS 2017 Workshop book Schedule Highlights Generated Fri Dec 18, 2020 Workshop organizers make last-minute changes to their schedule. Download this document again to get the lastest changes, or use the NIPS mobile application. Dec. 8, 2017 101 A Learning in the Presence of Strategic Behavior Haghtalab, Mansour, Roughgarden, Syrgkanis, Wortman Vaughan 101 B Visually grounded interaction and language Strub, de Vries, Das, Kottur, Lee, Malinowski, Pietquin, Parikh, Batra, Courville, Mary 102 A+B Advances in Modeling and Learning Interactions from Complex Data Dasarathy, Kolar, Baraniuk 102 C 6th Workshop on Automated Knowledge Base Construction (AKBC) Pujara, Arad, Dalvi Mishra, Rocktäschel 102 C Synergies in Geometric Data Analysis (TWO DAYS) Meila, Chazal, Chen 103 A+B Competition track Escalera, Weimer 104 A Machine Learning for Health (ML4H) - What Parts of Healthcare are Ripe for Disruption by Machine Learning Right Now? Beam, Beam, Fiterau, Fiterau, Schulam, Schulam, Fries, Fries, Hughes, Hughes, Wiltschko, Wiltschko, Snoek, Snoek, Antropova, Antropova, Ranganath, Ranganath, Jedynak, Jedynak, Naumann, Naumann, Dalca, Dalca, Dalca, Dalca, Althoff, Althoff, ASTHANA, ASTHANA, Tandon, Tandon, Kandola, Kandola, Ratner, Ratner, Kale, Kale, Shalit, Shalit, Ghassemi, Ghassemi, Kohane, Kohane 104 B Acting and Interacting in the Real World: Challenges in Robot Learning Posner, Hadsell, Riedmiller, Wulfmeier, Paul 104 C Deep Learning for Physical Sciences Baydin, Prabhat, Cranmer, Wood 201 A Machine Learning for Audio Signal Processing (ML4Audio) Purwins, Sturm, Plumbley 201 B Nearest Neighbors for Modern Applications with Massive Data: An Age-old Solution with New Challenges Chen, Shah, Lee 202 Machine Deception Goodfellow, Hwang, Goodman, Rodriguez 203 Discrete Structures in Machine Learning Singer, Bilmes, Krause, Jegelka, Karbasi 204 Transparent and interpretable Machine Learning in Safety Critical Environments Tosi, Vellido, Álvarez Grand NIPS 2017 Time Series Workshop Ballroom A Kuznetsov, Anava, Yang, Khaleghi Grand Conversational AI - today's practice and tomorrow's potential Ballroom B Geramifard, Williams, Heck, Glass, Bordes, Young, Tesauro Hall A OPT 2017: Optimization for Machine Learning Sra, J. Reddi, Agarwal, Recht Hall C From 'What If?' To 'What Next?' : Causal Inference and Machine Learning for Intelligent Decision Making 2 Volfovsky, Swaminathan, Toulis, Kallus, Silva, Shawe-Taylor, Joachims, Li Hyatt Extreme Classification: Multi-class & Multi-label Learning in Extremely Large Hotel, Label Spaces Regency Varma, Kloft, Dembczynski Ballroom A+B+C Hyatt Learning on Distributions, Functions, Graphs and Groups Hotel, d'Alché-Buc, Muandet, Sriperumbudur, Szabó Regency Ballroom D+E+F+H Hyatt Machine Learning for Creativity and Design Hotel, Eck, Ha, Eslami, Dieleman, Fiebrink, Elliott Seaview Ballroom Hyatt Machine Learning and Computer Security Hotel, Steinhardt, Papernot, Li, Liu, Liang, Song Shoreline S1 ML Systems Workshop @ NIPS 2017 Lakshmiratan, Bird, Sen, Ré, Li, Gonzalez, Crankshaw S4 Machine Learning for Molecules and Materials Chmiela, Hernández-Lobato, Schütt, Aspuru-Guzik, Tkatchenko, Ramsundar, von Lilienfeld, Kusner, Tsuda, Paige, Müller S5 Workshop on Worm's Neural Information Processing (WNIP) Hasani, Zimmer, Larson, Grosu S7 Machine Learning for the Developing World De-Arteaga, Herlands Seaside Advances in Approximate Bayesian Inference Ballroom Ruiz, Mandt, Zhang, McInerney, Tran, Broderick, Titsias, Blei, Welling Dec. 9, 2017 101 A (Almost) 50 shades of Bayesian Learning: PAC-Bayesian trends and insights Guedj, Germain, Bach 101 B Deep Learning at Supercomputer Scale Elsen, Hafner, Stone, Saeta 102 A+B Machine Learning on the Phone and other Consumer Devices Aradhye, Quinonero Candela, Prasad 102 C Synergies in Geometric Data Analysis (2nd day) Meila, Chazal, Chen 103 A+B Medical Imaging meets NIPS Glocker, Konukoglu, Lombaert, Bhatia 103 C Workshop on Prioritising Online Content Shawe-Taylor, Pontil, Cesa-Bianchi, Yilmaz, Watkins, Riedel, Grobelnik 104 A Cognitively Informed Artificial Intelligence: Insights From Natural Intelligence Mozer, Lake, Yu 104 B Machine Learning in Computational Biology Zou, Kundaje, Quon, Fusi, Mostafavi 3 104 C The future of gradient-based machine learning software & techniques Wiltschko, van Merriënboer, Lamblin 201 A 2017 NIPS Workshop on Machine Learning for Intelligent Transportation Systems Li, Dragan, Niebles, Savarese 201 B Aligned Artificial Intelligence Hadfield-Menell, Steinhardt, Duvenaud, Krueger, Dragan 202 NIPS Highlights (MLTrain), Learn How to code a paper with state of the art frameworks Dimakis, Vasiloglou, Van den Broeck, Ihler, Araki 203 Learning Disentangled Features: from Perception to Control Denton, Narayanaswamy, Kulkarni, Lee, Bouchacourt, Tenenbaum, Pfau 204 BigNeuro 2017: Analyzing brain data from nano to macroscale Dyer, Kiar, Gray Roncal, , Koerding, Vogelstein Grand Hierarchical Reinforcement Learning Ballroom A Barto, Precup, Mannor, Schaul, Fox, Florensa Grand Learning with Limited Labeled Data: Weak Supervision and Beyond Ballroom B Augenstein, Bach, Belilovsky, Blaschko, Lampert, Oyallon, Platanios, Ratner, Ré Hall A Deep Learning: Bridging Theory and Practice Arora, Raghu, Salakhutdinov, Schmidt, Vinyals Hall C Bayesian Deep Learning Gal, Hernández-Lobato, Louizos, Wilson, Kingma, Ghahramani, Murphy, Welling Hyatt Workshop on Meta-Learning Beacon Calandra, Hutter, Larochelle, Levine Ballroom D+E+F+H Hyatt Interpreting, Explaining and Visualizing Deep Learning - Now what ? Hotel, Müller, Vedaldi, Hansen, Samek, Montavon Regency Ballroom A+B+C Hyatt Optimal Transport and Machine Learning Hotel, Bousquet, Cuturi, Peyré, Sha, Solomon Seaview Ballroom Hyatt Collaborate & Communicate: An exploration and practical skills workshop that Hotel, builds on the experience of AIML experts who are both successful collaborators Shoreline and great communicators. Gorman S1 Machine Learning Challenges as a Research Tool Guyon, Viegas, Escalera, Abernethy S4 Emergent Communication Workshop Foerster, Mordatch, Lazaridou, Cho, Kiela, Abbeel S7 Bayesian optimization for science and engineering Martinez-Cantin, Hernández-Lobato, Gonzalez Seaside Teaching Machines, Robots, and Humans Ballroom Cakmak, Rafferty, Singla, Zhu, Zilles 4 Dec. 8, 2017 Dec. 8, 2017 these problems include but are not limited to 1. Learning from data that is produced by agents who have vested interest in the outcome or the Learning in the Presence of Strategic learning process. For example, learning a Behavior measure of quality of universities by surveying members of the academia who stand to gain or Nika Haghtalab, Yishay Mansour, Tim Roughgarden, lose from the outcome, or when a GPS routing Vasilis Syrgkanis, Jenn Wortman Vaughan app has to learn patterns of traffic delay by routing individuals who have no interest in taking 101 A, Fri Dec 08, 08:00 AM slower routes. Machine learning is primarily concerned with the 2. Learning a model for the strategic behavior of design and analysis of algorithms that learn one or more agents by observing their about an entity. Increasingly more, machine interactions, for example, learning economical learning is being used to design policies that demands of buyers by observing their bidding affect the entity it once learned about. This can patterns when competing with other buyers. cause the entity to react and present a different behavior. Ignoring such interactions could lead to 3. Learning as a model of interactions between solutions that are ultimately ineffective in agents. Examples of this include applications to practice. For example, to design an effective ad swarm robotics, where individual agents have to display one has to take into account how a viewer learn to interact in a multi-agent setting in order would react to the displayed advertisements, for to achieve individual or collective goals. example by choosing to scroll through or click on them. Additionally, in many environments, 4. Interactions between multiple learners. In multiple learners learn concurrently about one or many settings, two or more learners learn about more related entities. This can bring about a the same or multiple related concepts. How do range of interactions between individual learners. these learners interact? What are the scenarios For example, multiple firms may compete or under which they would share knowledge, collaborate on performing market research. How information, or data. What are the desirable do the learners and entities interact? How do interactions between learners? As an example, these interactions change the task at hand? What consider multiple competing pharmaceutical are some desirable interactions in a learning firms that are learning about the effectiveness of environment? And what are the mechanisms for a certain treatment. In this case, while competing bringing about such desirable interactions? These firms would prefer not to share their findings, it is are some of the questions we would like to beneficial to the society when such findings are explore more in this workshop. shared. How can we incentivize these learners to perform such desirable interactions? Traditionally, learning theory has adopted two extreme views in this respect: First, when The main goal of this workshop is to address learning occurs in isolation from strategic current challenges and opportunities that arise behavior, such as in the classical PAC setting from the presence of strategic behavior in where the data is drawn from a fixed distribution; learning theory. This workshop
Recommended publications
  • Kitsune: an Ensemble of Autoencoders for Online Network Intrusion Detection

    Kitsune: an Ensemble of Autoencoders for Online Network Intrusion Detection

    Kitsune: An Ensemble of Autoencoders for Online Network Intrusion Detection Yisroel Mirsky, Tomer Doitshman, Yuval Elovici and Asaf Shabtai Ben-Gurion University of the Negev yisroel, tomerdoi @post.bgu.ac.il, elovici, shabtaia @bgu.ac.il { } { } Abstract—Neural networks have become an increasingly popu- Over the last decade many machine learning techniques lar solution for network intrusion detection systems (NIDS). Their have been proposed to improve detection performance [2], [3], capability of learning complex patterns and behaviors make them [4]. One popular approach is to use an artificial neural network a suitable solution for differentiating between normal traffic and (ANN) to perform the network traffic inspection. The benefit network attacks. However, a drawback of neural networks is of using an ANN is that ANNs are good at learning complex the amount of resources needed to train them. Many network non-linear concepts in the input data. This gives ANNs a gateways and routers devices, which could potentially host an NIDS, simply do not have the memory or processing power to great advantage in detection performance with respect to other train and sometimes even execute such models. More importantly, machine learning algorithms [5], [2]. the existing neural network solutions are trained in a supervised The prevalent approach to using an ANN as an NIDS is manner. Meaning that an expert must label the network traffic to train it to classify network traffic as being either normal and update the model manually from time to time. or some class of attack [6], [7], [8]. The following shows the In this paper, we present Kitsune: a plug and play NIDS typical approach to using an ANN-based classifier in a point which can learn to detect attacks on the local network, without deployment strategy: supervision, and in an efficient online manner.
  • An Introduction to Incremental Learning by Qiang Wu and Dave Snell

    An Introduction to Incremental Learning by Qiang Wu and Dave Snell

    Article from Predictive Analytics and Futurism July 2016 Issue 13 An Introduction to Incremental Learning By Qiang Wu and Dave Snell achine learning provides useful tools for predictive an- alytics. The typical machine learning problem can be described as follows: A system produces a specific out- Mput for each given input. The mechanism underlying the system can be described by a function that maps the input to the output. Human beings do not know the mechanism but can observe the inputs and outputs. The goal of a machine learning algorithm is regression, perceptron for classification and incremental princi- to infer the mechanism by a set of observations collected for the pal component analysis. input and output. Mathematically, we use (xi,yi ) to denote the i-th pair of observation of input and output. If the real mech- STOCHASTIC GRADIENT DESCENT anism of the system to produce data is described by a function In linear regression, f*(x) = wTx is a linear function of the input f*, then the true output is supposed to be f*(x ). However, due i vector. The usual choice of the loss function is the squared loss to systematic noise or measurement error, the observed output L(y,wTx) = (y-wTx)2. The gradient of L with respect to the weight y satisfies y = f*(x )+ϵ where ϵ is an unavoidable but hopefully i i i i i vector w is given by small error term. The goal then, is to learn the function f* from the n pairs of observations {(x ,y ),(x ,y ),…,(x ,y )}.
  • Arxiv:1811.04918V6 [Cs.LG] 1 Jun 2020

    Arxiv:1811.04918V6 [Cs.LG] 1 Jun 2020

    Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers Zeyuan Allen-Zhu Yuanzhi Li Yingyu Liang [email protected] [email protected] [email protected] Microsoft Research AI Stanford University University of Wisconsin-Madison November 12, 2018 (version 6)∗ Abstract The fundamental learning theory behind neural networks remains largely open. What classes of functions can neural networks actually learn? Why doesn't the trained network overfit when it is overparameterized? In this work, we prove that overparameterized neural networks can learn some notable con- cept classes, including two and three-layer networks with fewer parameters and smooth activa- tions. Moreover, the learning can be simply done by SGD (stochastic gradient descent) or its variants in polynomial time using polynomially many samples. The sample complexity can also be almost independent of the number of parameters in the network. On the technique side, our analysis goes beyond the so-called NTK (neural tangent kernel) linearization of neural networks in prior works. We establish a new notion of quadratic ap- proximation of the neural network (that can be viewed as a second-order variant of NTK), and connect it to the SGD theory of escaping saddle points. arXiv:1811.04918v6 [cs.LG] 1 Jun 2020 ∗V1 appears on this date, V2/V3/V4 polish writing and parameters, V5 adds experiments, and V6 reflects our conference camera ready version. Authors sorted in alphabetical order. We would like to thank Greg Yang and Sebastien Bubeck for many enlightening conversations. Y. Liang was supported in part by FA9550-18-1-0166, and would also like to acknowledge that support for this research was provided by the Office of the Vice Chancellor for Research and Graduate Education at the University of Wisconsin-Madison with funding from the Wisconsin Alumni Research Foundation.
  • Analysis of Perceptron-Based Active Learning

    Analysis of Perceptron-Based Active Learning

    JournalofMachineLearningResearch10(2009)281-299 Submitted 12/07; 12/08; Published 2/09 Analysis of Perceptron-Based Active Learning Sanjoy Dasgupta [email protected] Department of Computer Science and Engineering University of California, San Diego La Jolla, CA 92093-0404, USA Adam Tauman Kalai [email protected] Microsoft Research Office 14063 One Memorial Drive Cambridge, MA 02142, USA Claire Monteleoni [email protected] Center for Computational Learning Systems Columbia University Suite 850, 475 Riverside Drive, MC 7717 New York, NY 10115, USA Editor: Manfred Warmuth Abstract Ω 1 We start by showing that in an active learning setting, the Perceptron algorithm needs ( ε2 ) labels to learn linear separators within generalization error ε. We then present a simple active learning algorithm for this problem, which combines a modification of the Perceptron update with an adap- tive filtering rule for deciding which points to query. For data distributed uniformly over the unit 1 sphere, we show that our algorithm reaches generalization error ε after asking for just O˜(d log ε ) labels. This exponential improvement over the usual sample complexity of supervised learning had previously been demonstrated only for the computationally more complex query-by-committee al- gorithm. Keywords: active learning, perceptron, label complexity bounds, online learning 1. Introduction In many machine learning applications, unlabeled data is abundant but labeling is expensive. This distinction is not captured in standard models of supervised learning, and has motivated the field of active learning, in which the labels of data points are initially hidden, and the learner must pay for each label it wishes revealed.
  • The Sample Complexity of Agnostic Learning Under Deterministic Labels

    The Sample Complexity of Agnostic Learning Under Deterministic Labels

    JMLR: Workshop and Conference Proceedings vol 35:1–16, 2014 The sample complexity of agnostic learning under deterministic labels Shai Ben-David [email protected] Cheriton School of Computer Science, University of Waterloo, Waterloo, Ontario N2L 3G1, CANADA Ruth Urner [email protected] School of Computer Science, Georgia Institute of Technology, Atlanta, Georgia 30332, USA Abstract With the emergence of Machine Learning tools that allow handling data with a huge number of features, it becomes reasonable to assume that, over the full set of features, the true labeling is (almost) fully determined. That is, the labeling function is deterministic, but not necessarily a member of some known hypothesis class. However, agnostic learning of deterministic labels has so far received little research attention. We investigate this setting and show that it displays a behavior that is quite different from that of the fundamental results of the common (PAC) learning setups. First, we show that the sample complexity of learning a binary hypothesis class (with respect to deterministic labeling functions) is not fully determined by the VC-dimension of the class. For any d, we present classes of VC-dimension d that are learnable from O~(d/)- many samples and classes that require samples of size Ω(d/2). Furthermore, we show that in this setup, there are classes for which any proper learner has suboptimal sample complexity. While the class can be learned with sample complexity O~(d/), any proper (and therefore, any ERM) algorithm requires Ω(d/2) samples. We provide combinatorial characterizations of both phenomena, and further analyze the utility of unlabeled samples in this setting.
  • Sinkhorn Autoencoders

    Sinkhorn Autoencoders

    Sinkhorn AutoEncoders Giorgio Patrini⇤• Rianne van den Berg⇤ Patrick Forr´e Marcello Carioni UvA Bosch Delta Lab University of Amsterdam† University of Amsterdam KFU Graz Samarth Bhargav Max Welling Tim Genewein Frank Nielsen ‡ University of Amsterdam University of Amsterdam Bosch Center for Ecole´ Polytechnique CIFAR Artificial Intelligence Sony CSL Abstract 1INTRODUCTION Optimal transport o↵ers an alternative to Unsupervised learning aims at finding the underlying maximum likelihood for learning generative rules that govern a given data distribution PX . It can autoencoding models. We show that mini- be approached by learning to mimic the data genera- mizing the p-Wasserstein distance between tion process, or by finding an adequate representation the generator and the true data distribution of the data. Generative Adversarial Networks (GAN) is equivalent to the unconstrained min-min (Goodfellow et al., 2014) belong to the former class, optimization of the p-Wasserstein distance by learning to transform noise into an implicit dis- between the encoder aggregated posterior tribution that matches the given one. AutoEncoders and the prior in latent space, plus a recon- (AE) (Hinton and Salakhutdinov, 2006) are of the struction error. We also identify the role of latter type, by learning a representation that maxi- its trade-o↵hyperparameter as the capac- mizes the mutual information between the data and ity of the generator: its Lipschitz constant. its reconstruction, subject to an information bottle- Moreover, we prove that optimizing the en- neck. Variational AutoEncoders (VAE) (Kingma and coder over any class of universal approxima- Welling, 2013; Rezende et al., 2014), provide both a tors, such as deterministic neural networks, generative model — i.e.
  • Data-Dependent Sample Complexity of Deep Neural Networks Via Lipschitz Augmentation

    Data-Dependent Sample Complexity of Deep Neural Networks Via Lipschitz Augmentation

    Data-dependent Sample Complexity of Deep Neural Networks via Lipschitz Augmentation Colin Wei Tengyu Ma Computer Science Department Computer Science Department Stanford University Stanford University [email protected] [email protected] Abstract Existing Rademacher complexity bounds for neural networks rely only on norm control of the weight matrices and depend exponentially on depth via a product of the matrix norms. Lower bounds show that this exponential dependence on depth is unavoidable when no additional properties of the training data are considered. We suspect that this conundrum comes from the fact that these bounds depend on the training data only through the margin. In practice, many data-dependent techniques such as Batchnorm improve the generalization performance. For feedforward neural nets as well as RNNs, we obtain tighter Rademacher complexity bounds by considering additional data-dependent properties of the network: the norms of the hidden layers of the network, and the norms of the Jacobians of each layer with respect to all previous layers. Our bounds scale polynomially in depth when these empirical quantities are small, as is usually the case in practice. To obtain these bounds, we develop general tools for augmenting a sequence of functions to make their composition Lipschitz and then covering the augmented functions. Inspired by our theory, we directly regularize the network’s Jacobians during training and empirically demonstrate that this improves test performance. 1 Introduction Deep networks trained in
  • Adversarial Examples: Attacks and Defenses for Deep Learning

    Adversarial Examples: Attacks and Defenses for Deep Learning

    1 Adversarial Examples: Attacks and Defenses for Deep Learning Xiaoyong Yuan, Pan He, Qile Zhu, Xiaolin Li∗ National Science Foundation Center for Big Learning, University of Florida {chbrian, pan.he, valder}@ufl.edu, [email protected]fl.edu Abstract—With rapid progress and significant successes in a based malware detection and anomaly detection solutions are wide spectrum of applications, deep learning is being applied built upon deep learning to find semantic features [14]–[17]. in many safety-critical environments. However, deep neural net- Despite great successes in numerous applications, many of works have been recently found vulnerable to well-designed input samples, called adversarial examples. Adversarial examples are deep learning empowered applications are life crucial, raising imperceptible to human but can easily fool deep neural networks great concerns in the field of safety and security. “With great in the testing/deploying stage. The vulnerability to adversarial power comes great responsibility” [18]. Recent studies find examples becomes one of the major risks for applying deep neural that deep learning is vulnerable against well-designed input networks in safety-critical environments. Therefore, attacks and samples. These samples can easily fool a well-performed defenses on adversarial examples draw great attention. In this paper, we review recent findings on adversarial examples for deep learning model with little perturbations imperceptible to deep neural networks, summarize the methods for generating humans. adversarial examples, and propose a taxonomy of these methods. Szegedy et al. first generated small perturbations on the im- Under the taxonomy, applications for adversarial examples are ages for the image classification problem and fooled state-of- investigated.
  • Nearly Tight Sample Complexity Bounds for Learning Mixtures of Gaussians Via Sample Compression Schemes∗

    Nearly Tight Sample Complexity Bounds for Learning Mixtures of Gaussians Via Sample Compression Schemes∗

    Nearly tight sample complexity bounds for learning mixtures of Gaussians via sample compression schemes∗ Hassan Ashtiani Shai Ben-David Department of Computing and Software School of Computer Science McMaster University, and University of Waterloo, Vector Institute, ON, Canada Waterloo, ON, Canada [email protected] [email protected] Nicholas J. A. Harvey Christopher Liaw Department of Computer Science Department of Computer Science University of British Columbia University of British Columbia Vancouver, BC, Canada Vancouver, BC, Canada [email protected] [email protected] Abbas Mehrabian Yaniv Plan School of Computer Science Department of Mathematics McGill University University of British Columbia Montréal, QC, Canada Vancouver, BC, Canada [email protected] [email protected] Abstract We prove that Θ(e kd2="2) samples are necessary and sufficient for learning a mixture of k Gaussians in Rd, up to error " in total variation distance. This improves both the known upper bounds and lower bounds for this problem. For mixtures of axis-aligned Gaussians, we show that Oe(kd="2) samples suffice, matching a known lower bound. The upper bound is based on a novel technique for distribution learning based on a notion of sample compression. Any class of distributions that allows such a sample compression scheme can also be learned with few samples. Moreover, if a class of distributions has such a compression scheme, then so do the classes of products and mixtures of those distributions. The core of our main result is showing that the class of Gaussians in Rd has a small-sized sample compression. 1 Introduction Estimating distributions from observed data is a fundamental task in statistics that has been studied for over a century.
  • Sample Complexity Results

    Sample Complexity Results

    10-601 Machine Learning Maria-Florina Balcan Spring 2015 Generalization Abilities: Sample Complexity Results. The ability to generalize beyond what we have seen in the training phase is the essence of machine learning, essentially what makes machine learning, machine learning. In these notes we describe some basic concepts and the classic formalization that allows us to talk about these important concepts in a precise way. Distributional Learning The basic idea of the distributional learning setting is to assume that examples are being provided from a fixed (but perhaps unknown) distribution over the instance space. The assumption of a fixed distribution gives us hope that what we learn based on some training data will carry over to new test data we haven't seen yet. A nice feature of this assumption is that it provides us a well-defined notion of the error of a hypothesis with respect to target concept. Specifically, in the distributional learning setting (captured by the PAC model of Valiant and Sta- tistical Learning Theory framework of Vapnik) we assume that the input to the learning algorithm is a set of labeled examples S :(x1; y1);:::; (xm; ym) where xi are drawn i.i.d. from some fixed but unknown distribution D over the the instance space ∗ ∗ X and that they are labeled by some target concept c . So yi = c (xi). Here the goal is to do optimization over the given sample S in order to find a hypothesis h : X ! f0; 1g, that has small error over whole distribution D. The true error of h with respect to a target concept c∗ and the underlying distribution D is defined as err(h) = Pr (h(x) 6= c∗(x)): x∼D (Prx∼D(A) means the probability of event A given that x is selected according to distribution D.) We denote by m ∗ 1 X ∗ errS(h) = Pr (h(x) 6= c (x)) = I[h(xi) 6= c (xi)] x∼S m i=1 the empirical error of h over the sample S (that is the fraction of examples in S misclassified by h).
  • Deep Learning and Neural Networks Module 4

    Deep Learning and Neural Networks Module 4

    Deep Learning and Neural Networks Module 4 Table of Contents Learning Outcomes ......................................................................................................................... 5 Review of AI Concepts ................................................................................................................... 6 Artificial Intelligence ............................................................................................................................ 6 Supervised and Unsupervised Learning ................................................................................................ 6 Artificial Neural Networks .................................................................................................................... 8 The Human Brain and the Neural Network ........................................................................................... 9 Machine Learning Vs. Neural Network ............................................................................................... 11 Machine Learning vs. Neural Network Comparison Table (Educba, 2019) .............................................. 12 Conclusion – Machine Learning vs. Neural Network ........................................................................... 13 Real-time Speech Translation ............................................................................................................. 14 Uses of a Translator App ...................................................................................................................
  • Benchmarking Model-Based Reinforcement Learning

    Benchmarking Model-Based Reinforcement Learning

    Benchmarking Model-Based Reinforcement Learning Tingwu Wang1;2, Xuchan Bao1;2, Ignasi Clavera3, Jerrick Hoang1;2, Yeming Wen1;2, Eric Langlois1;2, Shunshi Zhang1;2, Guodong Zhang1;2, Pieter Abbeel3& Jimmy Ba1;2 1University of Toronto 2 Vector Institute 3 UC Berkeley {tingwuwang,jhoang,ywen,edl,gdzhang}@cs.toronto.edu {xuchan.bao,matthew.zhang}@mail.utoronto.ca, [email protected], [email protected], [email protected] Abstract Model-based reinforcement learning (MBRL) is widely seen as having the potential to be significantly more sample efficient than model-free RL. However, research in model-based RL has not been very standardized. It is fairly common for authors to experiment with self-designed environments, and there are several separate lines of research, which are sometimes closed-sourced or not reproducible. Accordingly, it is an open question how these various existing MBRL algorithms perform relative to each other. To facilitate research in MBRL, in this paper we gather a wide collection of MBRL algorithms and propose over 18 benchmarking environments specially designed for MBRL. We benchmark these algorithms with unified problem settings, including noisy environments. Beyond cataloguing performance, we explore and unify the underlying algorithmic differences across MBRL algorithms. We characterize three key research challenges for future MBRL research: the dynamics bottleneck, the planning horizon dilemma, and the early-termination dilemma. Finally, to maximally facilitate future research on MBRL, we open-source our benchmark in http://www.cs.toronto.edu/~tingwuwang/mbrl.html. 1 Introduction Reinforcement learning (RL) algorithms are most commonly classified in two categories: model-free RL (MFRL), which directly learns a value function or a policy by interacting with the environment, and model-based RL (MBRL), which uses interactions with the environment to learn a model of it.