Generative Adversarial Networks (GANs)

The coolest idea in in the last twenty years - Yann Lecun

Generative Adversarial Networks (GANs) 1 / 28

Introduction

Generative Adversarial Networks (GANs) 2 / 28

Find deterministic function f: y = f(x), x:data, y:label

Generative Adversarial Networks (GANs) 3 / 28

Unsupervised Learning

More challenging than supervised learning. No label or curriculum. Some NN solutions: Generative Adversarial Networks

Generative Adversarial Networks (GANs) 4 / 28

"Most of human and animal learning is unsupervised learning. If intelligence was a cake, unsupervised learning would be the cake, supervised learning would be the icing on the cake, and would be the cherry on the cake. We know how to make the icing and the cherry, but we do not know how to make the cake. We need to solve the unsupervised learning problem before we can even think of getting to true AI." - Yann Lecun "What I cannot create, I do not understand" - Richard P. Feynman

Generative Adversarial Networks (GANs) 5 / 28

Generative Models Introduction

What does "generative" mean? You can sample from the model and the distribution of samples approximates the distribution of true data points To build a sampler, do we need to optimize the log-likelihood? Does this sampler work?

Generative Adversarial Networks (GANs) 6 / 28 Generative Adversarial Networks Desired Properties of the Sampler

What is wrong with the sampler? Why is not desired? Desired Properties We don’t want to sample the same existing data points Build a generative model that understands structure and underlying distribution of data points Output similar but not the same points in the training dataset Example: Faces with different poses, Different handwriting etc

Generative Adversarial Networks (GANs) 7 / 28

Generative Adversarial Networks Implicit Models

Given samples from data distribution pdata : x1, x2,..., xn Given a sampler qφ(z)= DNN(z; φ) where z ∼ p(z) Let pmodel be the density function of x = qφ(z) We do not have an explicit form for pdata or pmodel Objective: Make pmodel as close as possible to pdata by learning φ

Generative Adversarial Networks (GANs) 8 / 28 Generative Adversarial Networks Moment Matching

Given samples X = x1, x2,..., xn from PX and samples Y = y1, y2,..., yn from PY , how do we compare PX and PY ? Match moments of both PX and PY to bring them closer Not feasible to compute higher order moments in higher dimensions

Generative Adversarial Networks (GANs) 9 / 28

Generative Adversarial Networks Moment Matching

Generative Adversarial Networks (GANs) 10 / 28 Generative Adversarial Networks

Generative Adversarial Networks (GANs) 11 / 28

Generative Adversarial Networks

Generative Adversarial Networks (GANs) 12 / 28 Generative Adversarial Networks

Generative Adversarial Networks (GANs) 13 / 28

Generative Adversarial Networks What is going on?

What is the optimal discriminator given generated and true distributions?

Generative Adversarial Networks (GANs) 14 / 28 Generative Adversarial Networks What is going on?

Generative Adversarial Networks (GANs) 15 / 28

Generative Adversarial Networks What is going on?

What is the objective of the generator given the optimal discriminator?

Generative Adversarial Networks (GANs) 16 / 28 Generative Adversarial Networks Discriminator Saturation

Generative Adversarial Networks (GANs) 17 / 28

Generative Adversarial Networks Optimization

Generative Adversarial Networks (GANs) 18 / 28 Generative Adversarial Networks What is going on?

Generative Adversarial Networks (GANs) 19 / 28

Generative Adversarial Networks Live Action

GAN LAB

Generative Adversarial Networks (GANs) 20 / 28 Generative Adversarial Networks Result

Generative Adversarial Networks (GANs) 21 / 28

Generative Adversarial Networks DCGAN Results

Generative Adversarial Networks (GANs) 22 / 28 Evaluation of GANs

Still an open research problem Some Measures: Inception Score Frechet Inception Distance

Generative Adversarial Networks (GANs) 23 / 28

Inception Score

How do we say generated images are good? Good generators generate semantically diverse samples Use a trained model (Inception v3) to model p(y | x) Each image should have a recognizable object from a class → p(y | x) should have low entropy There should be many classes generated → p(y) should have high entropy

Generative Adversarial Networks (GANs) 24 / 28 Inception Score

Generative Adversarial Networks (GANs) 25 / 28

What GAN you do?

Generative Adversarial Networks (GANs) 26 / 28 What GAN you do?

Generative Adversarial Networks (GANs) 27 / 28

References

CS294-158 Deep Unsupervised Learning Spring 2019 - UC Berkeley CS231n for - Stanford University Tutorial on GANs,

Generative Adversarial Networks (GANs) 28 / 28