Generative Adversarial Networks (GANs) The coolest idea in Machine Learning in the last twenty years - Yann Lecun Generative Adversarial Networks (GANs) 1 / 28 Introduction Generative Adversarial Networks (GANs) 2 / 28 Supervised Learning Find deterministic function f: y = f(x), x:data, y:label Generative Adversarial Networks (GANs) 3 / 28 Unsupervised Learning More challenging than supervised learning. No label or curriculum. Some NN solutions: Boltzmann machine AutoEncoder Generative Adversarial Networks Generative Adversarial Networks (GANs) 4 / 28 Unsupervised Learning "Most of human and animal learning is unsupervised learning. If intelligence was a cake, unsupervised learning would be the cake, supervised learning would be the icing on the cake, and reinforcement learning would be the cherry on the cake. We know how to make the icing and the cherry, but we do not know how to make the cake. We need to solve the unsupervised learning problem before we can even think of getting to true AI." - Yann Lecun "What I cannot create, I do not understand" - Richard P. Feynman Generative Adversarial Networks (GANs) 5 / 28 Generative Models Introduction What does "generative" mean? You can sample from the model and the distribution of samples approximates the distribution of true data points To build a sampler, do we need to optimize the log-likelihood? Does this sampler work? Generative Adversarial Networks (GANs) 6 / 28 Generative Adversarial Networks Desired Properties of the Sampler What is wrong with the sampler? Why is not desired? Desired Properties We don’t want to sample the same existing data points Build a generative model that understands structure and underlying distribution of data points Output similar but not the same points in the training dataset Example: Faces with different poses, Different handwriting etc Generative Adversarial Networks (GANs) 7 / 28 Generative Adversarial Networks Implicit Models Given samples from data distribution pdata : x1, x2,..., xn Given a sampler qφ(z)= DNN(z; φ) where z ∼ p(z) Let pmodel be the density function of x = qφ(z) We do not have an explicit form for pdata or pmodel Objective: Make pmodel as close as possible to pdata by learning φ Generative Adversarial Networks (GANs) 8 / 28 Generative Adversarial Networks Moment Matching Given samples X = x1, x2,..., xn from PX and samples Y = y1, y2,..., yn from PY , how do we compare PX and PY ? Match moments of both PX and PY to bring them closer Not feasible to compute higher order moments in higher dimensions Generative Adversarial Networks (GANs) 9 / 28 Generative Adversarial Networks Moment Matching Generative Adversarial Networks (GANs) 10 / 28 Generative Adversarial Networks Generative Adversarial Networks (GANs) 11 / 28 Generative Adversarial Networks Generative Adversarial Networks (GANs) 12 / 28 Generative Adversarial Networks Generative Adversarial Networks (GANs) 13 / 28 Generative Adversarial Networks What is going on? What is the optimal discriminator given generated and true distributions? Generative Adversarial Networks (GANs) 14 / 28 Generative Adversarial Networks What is going on? Generative Adversarial Networks (GANs) 15 / 28 Generative Adversarial Networks What is going on? What is the objective of the generator given the optimal discriminator? Generative Adversarial Networks (GANs) 16 / 28 Generative Adversarial Networks Discriminator Saturation Generative Adversarial Networks (GANs) 17 / 28 Generative Adversarial Networks Optimization Generative Adversarial Networks (GANs) 18 / 28 Generative Adversarial Networks What is going on? Generative Adversarial Networks (GANs) 19 / 28 Generative Adversarial Networks Live Action GAN LAB Generative Adversarial Networks (GANs) 20 / 28 Generative Adversarial Networks Result Generative Adversarial Networks (GANs) 21 / 28 Generative Adversarial Networks DCGAN Results Generative Adversarial Networks (GANs) 22 / 28 Evaluation of GANs Still an open research problem Some Measures: Inception Score Frechet Inception Distance Generative Adversarial Networks (GANs) 23 / 28 Inception Score How do we say generated images are good? Good generators generate semantically diverse samples Use a trained model (Inception v3) to model p(y | x) Each image should have a recognizable object from a class → p(y | x) should have low entropy There should be many classes generated → p(y) should have high entropy Generative Adversarial Networks (GANs) 24 / 28 Inception Score Generative Adversarial Networks (GANs) 25 / 28 What GAN you do? Generative Adversarial Networks (GANs) 26 / 28 What GAN you do? Generative Adversarial Networks (GANs) 27 / 28 References CS294-158 Deep Unsupervised Learning Spring 2019 - UC Berkeley CS231n Deep Learning for Computer Vision - Stanford University Tutorial on GANs, Ian Goodfellow Generative Adversarial Networks (GANs) 28 / 28.