> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE
Analysis by synthesis
Marcel Lüthi, 03. Juni 2019
Graphics and Vision Research Group Department of Mathematics and Computer Science University of Basel University of Basel
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Analysis by synthesis
Comparison
Parameters 휃 Update 휃 Synthesis 휑(휃)
Being able to synthesize data means we can understand how it was formed. − Allows reasoning about unseen parts. University of Basel
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Analysis by synthesis
Comparison
Parameters 휃 Update 휃 Synthesis 휑(휃) University of Basel
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Analysis by synthesis – Computer vision
Comparison
Parameters 휃 Update 휃 Synthesis 휑(휃)
Computer graphics University of Basel
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Mathematical Framework: Bayesian inference
Comparison: 푝 data 휃)
Prior 푝(휃)
Parameters 휃 Update using 푝(휃|data) Synthesis 휑(휃)
• Principled way of dealing with uncertainty. University of Basel
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Algorithmic implementation: MCMC
Comparison: 푝 data 휃)
Prior 푝(휃)
Parameters 휃 Sample from 푝(휃|data) Synthesis 휑(휃)
Posterior distribution over parameters University of Basel
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE The course in context
Pattern Theory Text Music Ulf Grenander
Computational Research at anatomy Gravis Medical Images Fotos
Speech Natural language This course University of Basel
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Pattern theory – The mathematics University of Basel
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Pattern theory vs PMM
• Pattern theory is about developing a theory for understanding real- world signals • Probabilistic Morphable Models are about using theoretical well founded concepts to analyse images. • GPs for modelling • MCMC for model fitting • Working software University of Basel
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE
Analysis by Synthesis in 5 (simple) steps University of Basel
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Analysis by synthesis in 5 simple steps
1. Define a parametric model • a representation of the world 휃푛 • State of the world is determined by parameters 휃 = (휃1, … , 휃푛)
…
휃1
11 University of Basel
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Analysis by synthesis in 5 simple steps
2. Define a synthesis function 휑 휃1, … , 휃푛 • generates/synthesize the data given the “state of the world” • 휑 can be deterministic or stochastic
휃푛
휑(휃1, … , 휃푛)
…
휃1
12 University of Basel
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Analysis-by-Synthesis in 5 simple steps
3. Define likelihood function:
• Define a probabilistic model 푝 data 휃1, … , 휃푛 that models how the synthesized data compares to the real data • Includes stochastic factors on the data, such as noise
Comparison 푝(data|휃1, … , 휃푛)
휃푛
휑(휃1, … , 휃푛)
…
휃1 University of Basel
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Bayesian inference
We have: 푃 푑푎푡푎|휃1, … , 휃푛 We want: 푃 휃1, … , 휃푛|푑푎푡푎
Bayes rule:
푃 퐷|휃 푃 휃 푃 휃|퐷 = 푃 퐷
Lets us compute from 푝 퐷 휃 its “inverse” 푝(휃|퐷) University of Basel
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Analysis by synthesis in 5 simple steps
4. Define prior distribution: 푝 휃 = 푝(휃1, … , 휃푛) • Our believe about the “state of the world”
• Makes it possible to invert mapping 푝(data|휃1, … , 휃푛)
Comparison 푝(image|휃1, … , 휃푛)
휃푛
푝 휃 휑(휃1, … , 휃푛)
…
휃1 University of Basel
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Analysis by synthesis in 5 simple steps
5. Do inference 푝 휃 , … , 휃 푝 data 휃 , … , 휃 푝 휃 , … , 휃 data = 1 푛 1 푛 1 푛 푝 data
Comparison 푝(image|휑(휃1, … , 휃푛)
Purely conceptual formulation: 휃푛 • Independent of algorithmic 휑(휃 , … , 휃 ) implementation 1 푛 • … But usually done iteratively 휃1 Parameters 휃 Update using 푝(휃|Image) Synthesis 휑(휃)
16 University of Basel
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Analysis by synthesis in 5 simple steps
5. Possibility 1: Find best (most likely) solution:
푝 휃1, … , 휃푛 푝 data 휃1, … , 휃푛 arg max 푝 휃1, … , 휃푛 data = arg max 휃1,…,휃푛 휃1,…,휃푛 푝 data
MAP Solution
Local Most popular approach Maxima • Usually based on gradient- descent • May miss good solutions
17 University of Basel
> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Analysis by synthesis in 5 simple steps
5. Possibility 2: Find posterior distribution:
푝 휃 , … , 휃 푝 data 휃 , … , 휃 푝 휃 , … , 휃 data = 1 푛 1 푛 1 푛 푝 data
Core of this course • Obtain samples from the distribution • Based on Markov Chain Monte Carlo methods
18