DOCSLIB.ORG

Probabilistic Circuits: Representations, Inference, Learning and Theory

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Probabilistic Circuits: Representations, Inference, Learning and Theory Inference Probabilistic Representations Learning Circuits Theory Antonio Vergari YooJung Choi University of California, Los Angeles University of California, Los Angeles Robert Peharz Guy Van den Broeck TU Eindhoven University of California, Los Angeles January 7th, 2021 - IJCAI-PRICAI 2020 Fully factorized NaiveBayes AndOrGraphs PDGs Trees PSDDs CNets LTMs SPNs NADEs Thin Junction Trees ACs MADEs MAFs VAEs DPPs FVSBNs TACs IAFs NAFs RAEs Mixtures BNs NICE FGs GANs RealNVP MNs The Alphabet Soup of probabilistic models 2/153 Fully factorized NaiveBayes AndOrGraphs PDGs Trees PSDDs CNets LTMs SPNs NADEs Thin Junction Trees ACs MADEs MAFs VAEs DPPs FVSBNs TACs IAFs NAFs RAEs Mixtures BNs NICE FGs GANs RealNVP MNs Intractable and tractable models 3/153 Fully factorized NaiveBayes AndOrGraphs PDGs Trees PSDDs CNets LTMs SPNs NADEs Thin Junction Trees ACs MADEs MAFs VAEs DPPs FVSBNs TACs IAFs NAFs RAEs Mixtures BNs NICE FGs GANs RealNVP MNs tractability is a spectrum 4/153 Fully factorized NaiveBayes AndOrGraphs PDGs Trees PSDDs CNets LTMs SPNs NADEs Thin Junction Trees ACs MADEs MAFs VAEs DPPs FVSBNs TACs IAFs NAFs RAEs Mixtures BNs NICE FGs GANs RealNVP MNs Expressive models without compromises 5/153 Fully factorized NaiveBayes AndOrGraphs PDGs Trees PSDDs CNets LTMs SPNs NADEs Thin Junction Trees ACs MADEs MAFs VAEs DPPs FVSBNs TACs IAFs NAFs RAEs Mixtures BNs NICE FGs GANs RealNVP MNs a unifying framework for tractable models 6/153 Why tractable inference? or expressiveness vs tractability 7/153 Why tractable inference? or expressiveness vs tractability Probabilistic circuits a unified framework for tractable probabilistic modeling 7/153 Why tractable inference? or expressiveness vs tractability Probabilistic circuits a unified framework for tractable probabilistic modeling Learning circuits learning their structure and parameters from data 7/153 Why tractable inference? or expressiveness vs tractability Probabilistic circuits a unified framework for tractable probabilistic modeling Learning circuits learning their structure and parameters from data Advanced representations tracing the boundaries of tractability and connections to other formalisms 7/153 Why tractable inference? or the inherent trade-off of tractability vs. expressiveness Why probabilistic inference? q1: What is the probability that today is a Monday and there is a traffic jam on Westwood Blvd.? © fineartamerica.com 9/153 Why probabilistic inference? q1: What is the probability that today is a Monday and there is a traffic jam on Westwood Blvd.? q2: Which day is most likely to have a traffic jam on my route to campus? © fineartamerica.com 9/153 Why probabilistic inference? q1: What is the probability that today is a Monday and there is a traffic jam on Westwood Blvd.? q2: Which day is most likely to have a traffic jam on my route to campus? How to answer several of these probabilistic queries? © fineartamerica.com 9/153 “What is the most likely street to have a traffic jam at 12.00?” q1? answering queries… 10/153 “What is the most likely street to have a traffic jam at 12.00?” X1 X2 X3 X4 X5 x8 q1? x7 x6 x5 x4 x3 x2 x1 answering queries… 10/153 “What is the most likely street to have a traffic jam at 12.00?” X1 X2 X3 X4 X5 x8 q1(m1)? ≈ x7 x6 pm1 (Y j X) x5 x4 x3 x2 x1 …by fitting predictive models! 10/153 “What is the most likely street to have a traffic jam at 12.00?” X1 X2 X3 X4 X5 x8 q1(m1)? ≈ x7 x6 pm1 (Y j X) x5 x4 x3 x2 x1 …by fitting predictive models! 10/153 “What is the most likely time to see a traffic jam at Sunset Blvd.?” X1 X2 X3 X4 X5 x8 q1(m1)? x7 x6 x5 x4 x3 x2 q2(m2)? ≈ x1 pm2 (Y j X) …by fitting predictive models! 10/153 “What is the probability of a traffic jam on Westwood Blvd. on Monday?” X1 X2 X3 X4 X5 x8 q1(m1)? x7 x6 x5 q3(m?)? pm? (Y) ? x4 x3 x2 q2(m2)? x1 …by fitting predictive models! 10/153 q1(m)? X1 X2 X3 X4 X5 q2(m)? x8 ≈ x7 ... x6 x5 qk(m)? pm(X) x4 x3 x2 x1 …by fitting generative models! 10/153 X1 X2 X3 X4 X5 x8 q1(m)? x7 x6 q2(m)? x5 ≈ x4 ... x3 x2 qk(m)? pm(X) x1 …e.g. exploratory data analysis 10/153 Why probabilistic inference? q1: What is the probability that today is a Monday and there is a traffic jam on Westwood Blvd.? © fineartamerica.com 11/153 Why probabilistic inference? q1: What is the probability that today is a Monday and there is a traffic jam on Westwood Blvd.? X = fDay; Time; JamStr1; JamStr2;:::; JamStrNg q1(m) = pm(Day = Mon; JamWwood = 1) © fineartamerica.com 11/153 Why probabilistic inference? q1: What is the probability that today is a Monday and there is a traffic jam on Westwood Blvd.? X = fDay; Time; JamStr1; JamStr2;:::; JamStrNg q1(m) = pm(Day = Mon; JamWwood = 1) © fineartamerica.com ) marginals 11/153 Why probabilistic inference? q2: Which day is most likely to have a traffic jam on my route to campus? X = fDay; Time; JamStr1; JamStr2;:::; JamStrNg W ^ q2(m) = argmaxd pm(Day = d i2route JamStri) © fineartamerica.com 11/153 Why probabilistic inference? q2: Which day is most likely to have a traffic jam on my route to campus? X = fDay; Time; JamStr1; JamStr2;:::; JamStrNg W ^ q2(m) = argmaxd pm(Day = d i2route JamStri) © fineartamerica.com ) marginals + MAP + logical events 11/153 Tractable Probabilistic Inference A class of queries Q is tractable on a family of probabilistic models M iff for any query q 2 Q and model m 2 M exactly computing q(m) runs in time O(poly(jmj)). 12/153 Tractable Probabilistic Inference A class of queries Q is tractable on a family of probabilistic models M iff for any query q 2 Q and model m 2 M exactly computing q(m) runs in time O(poly(jmj)). ) often poly will in fact be linear! 12/153 Tractable Probabilistic Inference A class of queries Q is tractable on a family of probabilistic models M iff for any query q 2 Q and model m 2 M exactly computing q(m) runs in time O(poly(jmj)). ) often poly will in fact be linear! ) Note: if M is compact in the number of random variables X, that is, jmj 2 O(poly(jXj)), then query time is O(poly(jXj)). 12/153 Tractable Probabilistic Inference A class of queries Q is tractable on a family of probabilistic models M iff for any query q 2 Q and model m 2 M exactly computing q(m) runs in time O(poly(jmj)). ) often poly will in fact be linear! ) Note: if M is compact in the number of random variables X, that is, jmj 2 O(poly(jXj)), then query time is O(poly(jXj)). ) Why exactness? Highest guarantee possible! 12/153 Stay tuned for... Next: 1. What are classes of queries? 2. Are my favorite models tractable? 3. Are tractable models expressive? After: We introduce probabilistic circuits as a unified framework for tractable probabilistic modeling 13/153 Q M : tractable bands 14/153 Complete evidence (EVI) q3: What is the probability that today is a Monday at 12.00 and there is a traffic jam only on Westwood Blvd.? © fineartamerica.com 15/153 Complete evidence (EVI) q3: What is the probability that today is a Monday at 12.00 and there is a traffic jam only on Westwood Blvd.? X = fDay; Time; JamWwood ; JamStr2;:::; JamStrNg q3(m) = pm(X = fMon; 12:00; 1; 0;:::; 0g) © fineartamerica.com 15/153 Complete evidence (EVI) q3: What is the probability that today is a Monday at 12.00 and there is a traffic jam only on Westwood Blvd.? X = fDay; Time; JamWwood ; JamStr2;:::; JamStrNg q3(m) = pm(X = fMon; 12:00; 1; 0;:::; 0g) …fundamental in maximum likelihood learning © fineartamerica.com Q MLE θm = argmaxθ x2D pm(x; θ) 15/153 Generative Adversarial Networks E E − minθ maxϕ x∼pdata(x) log Dϕ(x) + z∼p(z) log(1 Dϕ(Gθ(z))) Goodfellow et al., “Generative adversarial nets”, 2014 16/153 Generative Adversarial Networks E E − minθ maxϕ x∼pdata(x) log Dϕ(x) + z∼p(z) log(1 Dϕ(Gθ(z))) no explicit likelihood! ) adversarial training instead of MLE ) no tractable EVI good sample quality ) but lots of samples needed for MC unstable training ) mode collapse Goodfellow et al., “Generative adversarial nets”, 2014 17/153 Q EVI M : GANs tractable bands 18/153 Variational Autoencoders R pθ(x) = pθ(x j z)p(z)dz an explicit likelihood model! Rezende et al., “Stochastic backprop. and approximate inference in deep generative models”, 2014 Kingma and Welling, “Auto-Encoding Variational Bayes”, 2014 19/153 Variational Autoencoders ≥ E j − KL j jj log pθ(x) z∼qϕ(zjx) log pθ(x z) (qϕ(z x) p(z)) an explicit likelihood model! ... but computing log pθ(x) is intractable ) an infinite and uncountable mixture ) no tractable EVI we need to optimize the ELBO… ) which is “tricky” [Alemi et al. 2017; Dai et al. 2019; Ghosh et al. 2019] 20/153 Q EVI M : GANs VAEs tractable bands 21/153 Normalizing flows −1 −1 δf pX(x) = pZ(f (x)) det δx Z an explicit likelihood! …plus structured Jacobians f f −1 ) tractable EVI queries! many neural variants X RealNVP [Dinh et al. 2016], MAF [Papamakarios et al. 2017] MADE [Germain et al. 2015], PixelRNN [Oord et al. 2016] 22/153 Normalizing flows −1 −1 δf pX(x) = pZ(f (x)) det δx Z an explicit likelihood! …plus structured Jacobians f f −1 ) tractable EVI queries! many neural variants X RealNVP [Dinh et al.
Load more

Recommended publications
  • Risk Bounds for Classification and Regression Rules That Interpolate
    Overfitting or perfect fitting? Risk bounds for classification and regression rules that interpolate Mikhail Belkin Daniel Hsu Partha P. Mitra The Ohio State University Columbia University Cold Spring Harbor Laboratory Abstract Many modern machine learning models are trained to achieve zero or near-zero training error in order to obtain near-optimal (but non-zero) test error. This phe- nomenon of strong generalization performance for “overfitted” / interpolated clas- sifiers appears to be ubiquitous in high-dimensional data, having been observed in deep networks, kernel machines, boosting and random forests. Their performance is consistently robust even when the data contain large amounts of label noise. Very little theory is available to explain these observations. The vast majority of theoretical analyses of generalization allows for interpolation only when there is little or no label noise. This paper takes a step toward a theoretical foundation for interpolated classifiers by analyzing local interpolating schemes, including geometric simplicial interpolation algorithm and singularly weighted k-nearest neighbor schemes. Consistency or near-consistency is proved for these schemes in classification and regression problems. Moreover, the nearest neighbor schemes exhibit optimal rates under some standard statistical assumptions. Finally, this paper suggests a way to explain the phenomenon of adversarial ex- amples, which are seemingly ubiquitous in modern machine learning, and also discusses some connections to kernel machines and random forests in the interpo- lated regime. 1 Introduction The central problem of supervised inference is to predict labels of unseen data points from a set of labeled training data. The literature on this subject is vast, ranging from classical parametric and non-parametric statistics [48, 49] to more recent machine learning methods, such as kernel machines [39], boosting [36], random forests [15], and deep neural networks [25].
    [Show full text]
  • Risk Bounds for Classification and Regression Rules That Interpolate
    Overfitting or perfect fitting? Risk bounds for classification and regression rules that interpolate Mikhail Belkin1, Daniel Hsu2, and Partha P. Mitra3 1The Ohio State University, Columbus, OH 2Columbia University, New York, NY 3Cold Spring Harbor Laboratory, Cold Spring Harbor, NY October 29, 2018 Abstract Many modern machine learning models are trained to achieve zero or near-zero training error in order to obtain near-optimal (but non-zero) test error. This phenomenon of strong generalization performance for “overfitted” / interpolated classifiers appears to be ubiquitous in high-dimensional data, having been observed in deep networks, kernel machines, boosting and random forests. Their performance is consistently robust even when the data contain large amounts of label noise. Very little theory is available to explain these observations. The vast majority of theoretical analyses of generalization allows for interpolation only when there is little or no label noise. This paper takes a step toward a theoretical foundation for interpolated classifiers by analyzing local interpolating schemes, including geometric simplicial interpolation algorithm and singularly weighted k-nearest neighbor schemes. Consistency or near-consistency is proved for these schemes in classification and regression problems. Moreover, the nearest neighbor schemes exhibit optimal rates under some standard statistical assumptions. Finally, this paper suggests a way to explain the phenomenon of adversarial examples, which are seemingly ubiquitous in modern machine learning, and also discusses some connections to kernel machines and random forests in the interpolated regime. 1 Introduction arXiv:1806.05161v3 [stat.ML] 26 Oct 2018 The central problem of supervised inference is to predict labels of unseen data points from a set of labeled training data.
    [Show full text]
  • Part I Officers in Institutions Placed Under the Supervision of the General Board
    2 OFFICERS NUMBER–MICHAELMAS TERM 2009 [SPECIAL NO.7 PART I Chancellor: H.R.H. The Prince PHILIP, Duke of Edinburgh, T Vice-Chancellor: 2003, Prof. ALISON FETTES RICHARD, N, 2010 Deputy Vice-Chancellors for 2009–2010: Dame SANDRA DAWSON, SID,ATHENE DONALD, R,GORDON JOHNSON, W,STUART LAING, CC,DAVID DUNCAN ROBINSON, M,JEREMY KEITH MORRIS SANDERS, SE, SARAH LAETITIA SQUIRE, HH, the Pro-Vice-Chancellors Pro-Vice-Chancellors: 2004, ANDREW DAVID CLIFF, CHR, 31 Dec. 2009 2004, IAN MALCOLM LESLIE, CHR, 31 Dec. 2009 2008, JOHN MARTIN RALLISON, T, 30 Sept. 2011 2004, KATHARINE BRIDGET PRETTY, HO, 31 Dec. 2009 2009, STEPHEN JOHN YOUNG, EM, 31 July 2012 High Steward: 2001, Dame BRIDGET OGILVIE, G Deputy High Steward: 2009, ANNE MARY LONSDALE, NH Commissary: 2002, The Rt Hon. Lord MACKAY OF CLASHFERN, T Proctors for 2009–2010: JEREMY LLOYD CADDICK, EM LINDSAY ANNE YATES, JN Deputy Proctors for MARGARET ANN GUITE, G 2009–2010: PAUL DUNCAN BEATTIE, CC Orator: 2008, RUPERT THOMPSON, SE Registrary: 2007, JONATHAN WILLIAM NICHOLLS, EM Librarian: 2009, ANNE JARVIS, W Acting Deputy Librarian: 2009, SUSANNE MEHRER Director of the Fitzwilliam Museum and Marlay Curator: 2008, TIMOTHY FAULKNER POTTS, CL Director of Development and Alumni Relations: 2002, PETER LAWSON AGAR, SE Esquire Bedells: 2003, NICOLA HARDY, JE 2009, ROGER DERRICK GREEVES, CL University Advocate: 2004, PHILIPPA JANE ROGERSON, CAI, 2010 Deputy University Advocates: 2007, ROSAMUND ELLEN THORNTON, EM, 2010 2006, CHRISTOPHER FORBES FORSYTH, R, 2010 OFFICERS IN INSTITUTIONS PLACED UNDER THE SUPERVISION OF THE GENERAL BOARD PROFESSORS Accounting 2003 GEOFFREY MEEKS, DAR Active Tectonics 2002 JAMES ANTHONY JACKSON, Q Aeronautical Engineering, Francis Mond 1996 WILLIAM NICHOLAS DAWES, CHU Aerothermal Technology 2000 HOWARD PETER HODSON, G Algebra 2003 JAN SAXL, CAI Algebraic Geometry (2000) 2000 NICHOLAS IAN SHEPHERD-BARRON, T Algebraic Geometry (2001) 2001 PELHAM MARK HEDLEY WILSON, T American History, Paul Mellon 1992 ANTHONY JOHN BADGER, CL American History and Institutions, Pitt 2009 NANCY A.
    [Show full text]
  • Machine Learning Conference Report
    Part of the conference series Breakthrough science and technologies Transforming our future Machine learning Conference report Machine learning report – Conference report 1 Introduction On 22 May 2015, the Royal Society hosted a unique, high level conference on the subject of machine learning. The conference brought together scientists, technologists and experts from across academia, industry and government to learn about the state-of-the-art in machine learning from world and UK experts. The presentations covered four main areas of machine learning: cutting-edge developments in software, the underpinning hardware, current applications and future social and economic impacts. This conference is the first in a new series organised This report is not a verbatim record, but summarises by the Royal Society, entitled Breakthrough Science the discussions that took place during the day and the and Technologies: Transforming our Future, which will key points raised. Comments and recommendations address the major scientific and technical challenges reflect the views and opinions of the speakers and of the next decade. Each conference will focus on one not necessarily that of the Royal Society. Full versions technology and cover key issues including the current of the presentations can be found on our website at: state of the UK industry sector, future direction of royalsociety.org/events/2015/05/breakthrough-science- research and the wider social and economic implications. technologies-machine-learning The conference series is being organised through the Royal Society’s Science and Industry programme, which demonstrates our commitment to reintegrate science and industry at the Society and to promote science and its value, build relationships and foster translation.
    [Show full text]
  • AAAI News AAAI News
    AAAI News AAAI News Winter News from the Association for the Advancement of Artificial Intelligence AAAI-18 Registration Student Activities looking for internships or jobs to meet with representatives from over 30 com - As part of its outreach to students, Is Open! panies and academia in an informal AAAI-18 will continue several special "meet-and-greet" atmosphere. If you AAAI-18 registration information is programs specifically for students, are representing a company, research now available at aaai.org/aaai18, and including the Doctoral Consortium, organization or university and would online registration can be completed at the Student Abstract Program, Lunch like to participate in the job fair, please regonline.com/aaai18. The deadline with a Fellow, and the Volunteer Pro - send an email with your contact infor - for late registration rates is January 5, gram, in addition to the following: 2018. Complete tutorial and workshop mation to [email protected] no later than January 5. The organizers of information, as well as other special Student Reception the AAAI/ACM SIGAI Job Fair are John program information is available at AAAI will welcome all students to Dickerson (University of Maryland, these sites. AAAI-18 by hosting an evening stu - USA) and Nicholas Mattei (IBM, USA). dent reception on Friday, February 2. Make Your Hotel Reservation Although the reception is especially The Winograd Now! beneficial to new students at the con - Schema Challenge AAAI has reserved a block of rooms at ference, all are welcome! Please join us and make all the newcomers welcome! Nuance Communications, Inc. is spon - the Hilton New Orleans Riverside at soring a competition to encourage reduced conference rates.
    [Show full text]
  • Probabilistic Machine Learning: Foundations and Frontiers
    Probabilistic Machine Learning: Foundations and Frontiers Zoubin Ghahramani1,2,3,4 1 University of Cambridge 2 Alan Turing Institute 3 Leverhulme Centre for the Future of Intelligence 4 Uber AI Labs [email protected] http://mlg.eng.cam.ac.uk/zoubin/ Sackler Forum, National Academy of Sciences Washington DC, 2017 Machine Learning Zoubin Ghahramani 2 / 53 Many Related Terms Statistical Modelling Artificial Intelligence Neural Networks Machine Learning Data Mining Data Analytics Deep Learning Pattern Recognition Data Science Zoubin Ghahramani 3 / 53 Many Related Fields Engineering Computer Science Statistics Machine Learning Computational Applied Mathematics Neuroscience Economics Cognitive Science Physics Zoubin Ghahramani 4 / 53 Many Many Applications Bioinformatics Computer Vision Robotics Scientific Data Analysis Natural Language Processing Machine Learning Information Retrieval Speech Recognition Recommender Systems Signal Processing Machine Translation Medical Informatics Targeted Advertising Finance Data Compression Zoubin Ghahramani 5 / 53 Machine Learning • Machine learning is an interdisciplinary field that develops both the mathematical foundations and practical applications of systems that learn from data. Main conferences and journals: NIPS, ICML, AISTATS, UAI, KDD, JMLR, IEEE TPAMI Zoubin Ghahramani 6 / 53 Canonical problems in machine learning Zoubin Ghahramani 7 / 53 Classification x x x o x xxx o x o o o x x o o x o • Task: predict discrete class label from input data • Applications: face recognition, image recognition,
    [Show full text]
  • Bayesian Activity Recognition Using Variational Inference
    BAR: Bayesian Activity Recognition using variational inference Ranganath Krishnan Mahesh Subedar Omesh Tickoo [email protected] [email protected] [email protected] Intel Labs Hillsboro, OR (USA) Abstract Uncertainty estimation in deep neural networks is essential for designing reliable and robust AI systems. Applications such as video surveillance for identifying suspicious activities are designed with deep neural networks (DNNs), but DNNs do not provide uncertainty estimates. Capturing reliable uncertainty estimates in safety and security critical applications will help to establish trust in the AI system. Our contribution is to apply Bayesian deep learning framework to visual activity recognition application and quantify model uncertainty along with principled confidence. We utilize the stochastic variational inference technique while training the Bayesian DNNs to infer the approximate posterior distribution around model parameters and perform Monte Carlo sampling on the posterior of model parameters to obtain the predictive distribution. We show that the Bayesian inference applied to DNNs provide reliable confidence measures for visual activity recognition task as compared to conventional DNNs. We also show that our method improves the visual activity recognition precision-recall AUC by 6.2% compared to non-Bayesian baseline. We evaluate our models on Moments-In-Time (MiT) activity recognition dataset by selecting a subset of in- and out-of-distribution video samples. 1 Introduction Activity recognition is an active area of research with multiple approaches depending on the applica- tion domain and the types of sensors [1, 2, 3]. In recent years, the DNN models applied to the visual activity recognition task outperform the earlier methods based on Gaussian Mixture Models and Hid- den Markov Models [4, 5] using handcrafted features.
    [Show full text]
  • Automatic Opioid User Detection from Twitter
    Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18) Automatic Opioid User Detection From Twitter: Transductive Ensemble Built On Different Meta-graph Based Similarities Over Heterogeneous Information Network Yujie Fan, Yiming Zhang, Yanfang Ye ∗, Xin Li Department of Computer Science and Electrical Engineering, West Virginia University, WV, USA fyf0004,[email protected], fyanfang.ye,[email protected] Abstract 13,000 people died from heroin overdose, both reflecting sig- nificant increase from 2002 [NIDA, 2017]. Opioid addiction Opioid (e.g., heroin and morphine) addiction has is a chronic mental illness that requires long-term treatment become one of the largest and deadliest epidemics and care [McLellan et al., 2000]. Although Medication As- in the United States. To combat such deadly epi- sisted Treatment (MAT) using methadone or buprenorphine demic, in this paper, we propose a novel frame- has been proven to provide best outcomes for opioid addiction work named HinOPU to automatically detect opi- recovery, stigma (i.e., bias) associated with MAT has limited oid users from Twitter, which will assist in sharp- its utilization [Saloner and Karthikeyan, 2015]. Therefore, ening our understanding toward the behavioral pro- there is an urgent need for novel tools and methodologies to cess of opioid addiction and treatment. In HinOPU, gain new insights into the behavioral processes of opioid ad- to model the users and the posted tweets as well diction and treatment. as their rich relationships, we introduce structured In recent years, the role of social media in biomedical heterogeneous information network (HIN) for rep- knowledge mining has become more and more important resentation.
    [Show full text]
  • Deep Learning
    ADVANCEMENTS AND TRENDS IN MEDICAL IMAGE ANALYSIS USING DEEP LEARNING A presentation by Shekoofeh Azizi University of British Columbia, Canada, Ph.D. 2014-2018 Electrical and Computer Engineering Isfahan University of Technology, Iran, M.Sc. 2011-2013 Computer Engineering / Hardware Design Isfahan University of Technology, Iran, B.Sc. 2007-2011 Computer Engineering / Hardware Engineering Philips Research North America, 2015-now National Institutes of Health (NIH), 2015-now MICCAI Student Board Officer, 2016-now Women in MICCAI, 2017-now 2 UBC Queen’s University SFU Univ. of Western Ontario VGH Robarts Research Technical Univ. of Munich ETH Zurich NVidia Sejong Univ., Korea Philips IBM Univ. of Colorado NIH Size is related to the number of collaborators Academic Collaborators Industrial/Clinical Collaborators 3 OUTLINE Deep Learning Medical Imaging Challenges and Vision Opportunities 4 DEEP LEARNING 5 ARTIFICIAL INTELLIGENCE VS. DATA SCIENCE AI is a technique which enables machines to mimic the “human behaviour” It’s 17°C Hey Google, and sunny in Weather in Victoria Victoria! Voice Services Voice to Command Brain/Model Voice Output User Google Home Google Home 6 ARTIFICIAL INTELLIGENCE VS. DATA SCIENCE Data Science is about processes and systems to extract knowledge or insights from data in Artificial Intelligence various forms. (AI) Machine Learning is the connection Machine Learning between data science and artificial (ML) intelligence since machine learning is the process of learning from data over time. Data Science Deep Learning (DL) 7 MACHINE LEARNING (ML) Machine learning is the process of learning from data over time. Reinforcement Learning Andrew Ng, Machine Learning Course, Coursera. 9 DEEP LEARNING (DL) Inspired by the functionality of our brains Number of sides: 4? Closed form shape? Perpendicular sides? Square ? Equal sides? 10 DEEP LEARNING (DL) Cat Feature Learning Dog 11 DEEP LEARNING (DL) 12 WHY DEEP LEARNING? Consistent improvement over the state-of-the-art across a large variety of domains.
    [Show full text]
  • Uncertainty in Deep Learning
    References Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jefrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geofrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dan Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. URL http://tensorĆow.org/. Software available from tensorĆow.org. Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mane. Concrete problems in ai safety. arXiv preprint arXiv:1606.06565, 2016. C Angermueller and O Stegle. Multi-task deep neural network to predict CpG methylation proĄles from low-coverage sequencing data. In NIPS MLCB workshop, 2015. O Anjos, C Iglesias, F Peres, J Martínez, Á García, and J Taboada. Neural networks applied to discriminate botanical origin of honeys. Food chemistry, 175:128Ű136, 2015. Christopher Atkeson and Juan Santamaria. A comparison of direct and model-based reinforcement learning. In In International Conference on Robotics and Automation. Citeseer, 1997. Jimmy Ba and Brendan Frey. Adaptive dropout for training deep neural networks. In Advances in Neural Information Processing Systems, pages 3084Ű3092, 2013. P Baldi, P Sadowski, and D Whiteson. Searching for exotic particles in high-energy physics with deep learning. Nature communications, 5, 2014. Pierre Baldi and Peter J Sadowski. Understanding dropout. In Advances in Neural Information Processing Systems, pages 2814Ű2822, 2013.
    [Show full text]
  • Arxiv:2012.12294V2 [Stat.ML] 16 Apr 2021 1 Introduction
    Evolutionary Variational Optimization of Generative Models Jakob Drefs1, Enrico Guiraud1;2, Jörg Lücke1 [email protected], [email protected], [email protected] 1 Machine Learning Lab, University of Oldenburg, Germany 2 CERN, Switzerland Abstract We combine two popular optimization approaches to derive learning algorithms for generative models: variational optimization and evolutionary algorithms. The combination is realized for generative models with discrete latents by using truncated posteriors as the family of variational distributions. The variational parameters of truncated posteriors are sets of latent states. By interpreting these states as genomes of individuals and by using the variational lower bound to define a fitness, we can apply evolutionary algorithms to realize the variational loop. The used variational distributions are very flexible and we show that evolutionary algorithms can effectively and efficiently optimize the variational bound. Furthermore, the variational loop is generally applicable (“black box”) with no analytical derivations required. To show general applicability, we apply the approach to three generative models (we use noisy-OR Bayes Nets, Binary Sparse Coding, and Spike-and-Slab Sparse Coding). To demonstrate effectiveness and efficiency of the novel variational approach, we use the standard competitive benchmarks of image denoising and inpainting. The benchmarks allow quantitative comparisons to a wide range of methods including probabilistic approaches, deep deterministic and generative networks, and non-local image processing methods. In the category of “zero-shot” learning (when only the corrupted image is used for training), we observed the evolutionary variational algorithm to significantly improve the state-of-the-art in many benchmark settings. For one well-known inpainting benchmark, we also observed state-of-the-art performance across all categories of algorithms although we only train on the corrupted image.
    [Show full text]
  • Lms Society and Annual General Meeting 2017 Agenda
    LMS SOCIETY AND ANNUAL GENERAL MEETING 2017 Friday, 10 November 2017 at 3.00 pm at BMA House, Tavistock Square, London AGENDA 1. Elections to Council and Nominating Committee Members wishing to vote in person at the AGM will be invited to hand their ballot paper to a Scrutineer. 2. Minutes of General Meeting, 30 June 2017 To agree the minutes of the General Meeting held on 30 June 2017 (minutes attached). 3. Review of Society Activities 2016-17 A report will be given by Vice-President Professor Ken Brown on the Society’s activities over the last year. 4. Report of the Treasurer The Society’s Treasurer, Professor Rob Curtis, will present his report on the past year and invite questions. 5. Resolutions: 5.1 Adoption of the Trustees’ Report including the year-end accounts for 2016-17 (Trustees’ Report attached) 5.2 Appointment of Auditors The Society’s President, Professor Simon Tavaré, will invite questions on the resolutions and recommend the adoption of the Trustees’ Report and the appointment of the Auditors. 6. Presentation of Certificates to the 2017 LMS Prize Winners Certificates will be presented to the winners of the Pólya Prize, Senior Whitehead Prize, Naylor Prize and Lectureship, Senior Anne Bennett Prize, Whitehead Prizes and Berwick Prize. 7. Zoubin Ghahramani (Professor of Information Engineering, University of Cambridge; Chief Scientist, Uber) Bayesian statistics, non-parametrics, neural networks, and artificial intelligence Tea break 8. Election Results The election results will be declared by the Scrutineers. 9. Handover of Presidential Office 10. Simon Tavaré (Department of Applied Mathematics and Theoretical Physics & Cancer Research UK Cambridge Institute, University of Cambridge) – Presidential Address The magical Ewens Sampling Formula Fiona Nixon, Executive Secretary 20 October 2017 Please note there will be a photographer at the meeting.
    [Show full text]