LEARNINGAND INFERENCEIN GRAPHICAL MODELS

Chapter 10: Random Fields

Dr. Martin Lauer

University of Freiburg Machine Learning Lab

Karlsruhe Institute of Technology Institute of Measurement and Control Systems

Learning and Inference in Graphical Models. Chapter 10 – p. 1/38 References for this chapter

◮ Christopher M. Bishop, Pattern Recognition and Machine Learning, ch. 8, Springer, 2006 ◮ Michael Ying Yang and Wolfgang Forstner,¨ A hierarchical conditional random field model for labeling and classifying images of man-made scenes. In: IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 196-203, 2011

Learning and Inference in Graphical Models. Chapter 10 – p. 2/38 Motivation

Bayesian networks model clear dependencies, often causal dependencies. Bayesian networks are acyclic. How can we model mutual and cyclic dependencies?

Example (economy): ◮ demand and supply determine the price ◮ high price fosters supply ◮ low price fosters demand

Learning and Inference in Graphical Models. Chapter 10 – p. 3/38 Motivation

Example (physics): modeling ferromagnetism in statistical mechanics

◮ a grid of magnetic dipoles in a volume ◮ every dipole causes a force on its neighbors ◮ every dipole is forced by its neighbors

The dipoles might change their orientation. Every configuration of the magnetic dipole field can be characterized by its energy. The probability of a certain configuration depends on its energy: high energy configurations are less probable, low energy configurations are more probable. → Ising-model (Ernst Ising, 1924)

Learning and Inference in Graphical Models. Chapter 10 – p. 4/38 Markov random fields

◮ a Markov random field (MRF) is a undirected, connected graph A ◮ each node represents a random variable • open circles indicate non-observed random variables C • filled circles indicate observed random variables B • dots indicate given constants ◮ links indicate an explicitly modeled stochastic D dependence

Learning and Inference in Graphical Models. Chapter 10 – p. 5/38 Markov random fields

Joint probability distribution of a MRF is defined over cliques in the graph X1 Definition: A clique of size k is a subset C of k nodes of the MRF so that for each pair X,Y ∈ C with X =6 Y holds that X and Y are connected by an edge. X4

Example: X2 The MRF on the right has ◮ one clique of size 3: X3 {X2,X3,X4} ◮ four cliques of size 2: {X1,X2}, {X2,X3}, {X2,X4}, {X3,X4} ◮ four cliques of size 1: {X1}, {X2}, {X3}, {X4}

Learning and Inference in Graphical Models. Chapter 10 – p. 6/38 Markov random fields

For every clique C in the MRF we specify a potential function

ψC : C → R>0

◮ large values of ψC indicate that a certain configuration of the random variables in the clique is more probable

◮ small values of ψC indicate that a certain configuration of the random variables in the clique is less probable

The joint distribution of the MRF is defined as the product of the potential functions for all cliques 1 p(X ,...,X )= ψ (C) 1 n Z C C Cliques ∈Y with Z = C∈Cliques ψC (C)d(X1,...,Xn) the partition function Remark: calculatingR Q Z might be very hard in practice

Learning and Inference in Graphical Models. Chapter 10 – p. 7/38 Markov random fields

Potential functions are usually given in terms of Gibbs/Boltzmann distributions

−EC (C) ψC (C)= e with EC : C → R an “energy function” ◮ large energy means low probability ◮ small energy means large probability Hence, the overall probability distribution of an MRF is 1 E C p(X ,...,X )= e− PC∈Cliques C ( ) 1 n Z

Learning and Inference in Graphical Models. Chapter 10 – p. 8/38 Markov random fields

Example: let us model the food preferences of a group of four persons: Antonia, Ben, Charles, and Damaris. They might choose between pasta, fish, and meat ◮ Ben likes meat and pasta but hates fish ◮ Antonia, Ben, and Charles prefer to choose the same ◮ Charles is vegetarian ◮ Damaris prefers to choose something else than all the other → create an MRF on the blackboard that models the food preferences of the four persons and assign potential functions to the cliques.

Learning and Inference in Graphical Models. Chapter 10 – p. 9/38 Markov random fields

One way to model the food preference task A C Random variables A, B, C, D model Antonias, Bens, Charles, and Damaris’ choice. Discrete variables with values 1=pasta, 2=fish, 3=meat Energy functions which are relevant B D (all others are constant): 0 if b ∈{1, 3} 0 if a =6 d E{B}(b)= E{A,D}(a, d)= (100 if b =2 (10 if a = d 0 if a = b = c 0 if b =6 d E{A,B,C}(a,b,c)= E{B,D}(b,d)= (30 otherwise (10 if b = d 0 if c =1 0 if c =6 d E{C,D}(c,d)= E{C}(c)= 50 if c =2 (10 if c = d 200 if c =3

Learning and Inference in Graphical Models. Chapter 10 – p. 10/38  Factor graphs

Like for Bayesian networks we can define factor graphs over MRFs. A factor graph is a bipartite graph with two kind of nodes: ◮ variable nodes that model random variables ◮ factor nodes that model a probabilistic relationship between variable nodes. Each factor node is assigned with a potential function Variable nodes and factor nodes are connected by undirected links. For each MRF we can create a factor graph as follows: ◮ the set of variable nodes is taken from the nodes of the MRF

◮ for each non-constant potential function ψC • we create a new factor node f • we connect f with all variable nodes in clique C

• we assign the potential function ψC to f Hence, the joint probability of the MRF is equal to the Gibbs distribution over the sum of all factor potentials

Learning and Inference in Graphical Models. Chapter 10 – p. 11/38 Factor graphs

The factor graph of the food preference task looks likes

E{C}

A C

E{A,B,V } E{C,D} E{A,D}

B D

E{B,D} E{B}

Learning and Inference in Graphical Models. Chapter 10 – p. 12/38 Stochastic inference in Markov random fields

How can we calculate p(U = u|O = o) and arg maxu p(U = u|O = u)? ◮ if the factor graph related to a MRF is a tree, we can use the sum-product and max-sum algorithm introduced in chapter 4. ◮ in the general case there are no efficient exact algorithms ◮ we can build variational approximations (chapter 6) for approximate inference ◮ we can use MCMC samplers (chapter 7) for numerical inference ◮ we can use local optimization (chapter 8)

Example: in the food preference task, ◮ what is the overall best choice of food? ◮ what is the best choice of food if Antonia eats fish?

Learning and Inference in Graphical Models. Chapter 10 – p. 13/38 Special types of MRFs

MRFs are very general and can be used for many purposes. Some models have been shown to be very useful. In this lecture, we introduce ◮ the Potts model. Useful for image segmentation and noise removal ◮ Conditional random fields. Usefule for image segmentation ◮ the Boltzmann machine. Useful for unsupervised and supervised learning ◮ Markov logic networks. Useful for logic inference on noisy data (chapter 11)

Learning and Inference in Graphical Models. Chapter 10 – p. 14/38 Potts Model

Learning and Inference in Graphical Models. Chapter 10 – p. 15/38 Potts model

The Potts model can be used for segmentation and noise removal in images and other sensor data. We discuss it in the image segmentation case

Assume, ◮ an image is composed out of several areas (e.g. foreground/background, object A/object B/background) ◮ each area has a characteristic color or gray value ◮ pixels in the image are corrupted by noise ◮ neighboring pixels are very likely to belong to the same area How can we model these assumptions with a MRF?

Learning and Inference in Graphical Models. Chapter 10 – p. 16/38 Potts model

◮ every pixel belongs to a certain area. We Yi−1,j−1 Yi−1,j Yi−1,j+1 model it with a discrete random variable Xi−1,j−1 Xi−1,j Xi−1,j+1 Xi,j. The true class label is unobserved.

◮ the color/gray value of each pixel is Yi,j−1 Yi,j Yi,j+1 described by a random variable Y . i,j Xi,j−1 Xi,j Xi,j+1 The color value is observed.

Yi+1,j−1 Yi+1,j Yi+1,j+1 ◮ Xi,j and Yi,j are stochastically

dependent. This dependency can be Xi+1,j−1 Xi+1,j Xi+1,j+1 described by an energy function ◮ the class labels of neighboring pixels are stochastically dependent. This can be described by an energy functions. ◮ we can provide priors for the class label as energy function on individual Xi,j

Learning and Inference in Graphical Models. Chapter 10 – p. 17/38 Potts model

energy functions on cliques: ◮ similarity of neighboring nodes

0 if xi,j = xi+1,j E{Xi,j ,Xi+1,j }(xi,j,xi+1,j)= (1 if xi,j =6 xi+1,j

0 if xi,j = xi,j+1 E{Xi,j ,Xi,j+1}(xi,j,xi,j+1)= (1 if xi,j =6 xi,j+1 ◮ dependecy between observed color/gray value and class label. Assume each class k can be characterized by a typical color/gray value ck

E{Xi,j ,Yi,j }(xi,j,yi,j)= ||Yi,j − cXi,j || ◮ overall preference for certain classes. Assume a prior distribution p over the

classes E{Xi,j }(xi,j)= − log p(Xi,j)

Learning and Inference in Graphical Models. Chapter 10 – p. 18/38 Potts model

energy function for the whole Potts model:

E = κ E{Xi,j ,Yi,j }(xi,j,yi,j) i,j X +λ E{Xi,j ,Xi+1,j } i,j X +λ E{Xi,j ,Xi,j+1} i,j X +µ E{Xi,j }(xi,j) i,j X with weighting factors κ,λ,µ ≥ 0

Learning and Inference in Graphical Models. Chapter 10 – p. 19/38 Potts model for image segmentation

Let us apply the Potts model to image segmentation as described before Determining a segmentation is done by maximizing the conditional probability p(...,Xi,j,... | ...,Yi,j,... ) where Yi,j are the color/gray values of a given picture. This is equal to minimizing the overall energy keeping the Yi,j values fixed. Solution techniques: ◮ finding an exact solution is NP-hard in general, in the two-class-case O(n3) if n is the number of pixels (solution using graph cuts) ◮ local optimization ◮ MCMC sampling → Matlab-demo Think about extensions of the Potts model that can cope with cases in which the reference colors of the segments are a priori vague or unknown → homework

Learning and Inference in Graphical Models. Chapter 10 – p. 20/38 Conditional Random Fields

Learning and Inference in Graphical Models. Chapter 10 – p. 21/38 Segmentation with Potts model revisited

Using a Potts model for segmentation requires adequate energy functions

E{Xi,j ,Yi,j } ◮ easy for a color segmentation task with pre-specified segment colors ◮ possible for a color segmentation task with roughly pre-specified segment colors Task: segment picture into areas ◮ almost impossible for texture-based of road, buildings, vegetation, sky, segmentation cars.

Idea: combine random field based segmentation with traditional classifiers (e.g. neural networks, support vector machines, decision trees, etc.) ◮ apply classifier on small patches of the image ◮ use a random field to integrate neighborhood relationships

Learning and Inference in Graphical Models. Chapter 10 – p. 22/38 Combination of random fields and classifiers

A classifier is ◮ a mapping from a vector of observations (features) to class labels ◮ a mapping from a vector of observations (features) to class probabilities With the second definition, the classifier provides a distribution p(X|Y ) with X the class label and Y the observation vector. A classifier does not provide a distribution on Y nor on X.

Learning and Inference in Graphical Models. Chapter 10 – p. 23/38 Combination of random fields and classifiers

Let us try to build a Potts model integrating the classifiers to model p(X|Y ) Yi−1,j−1 Yi−1,j Yi−1,j+1

Xi−1,j−1 Xi−1,j Xi−1,j+1 ◮ we can model the prior on the class labels as before using a potential Yi,j−1 Yi,j Yi,j+1

function Xi,j−1 Xi,j Xi,j+1 ◮ we can model the relationship between Yi+1,j−1 Yi+1,j Yi+1,j+1 neighboring X nodes by a potential function as before Xi+1,j−1 Xi+1,j Xi+1,j+1

◮ we can model p(Xi,j|Yi,j) with the classifier

How does the joint distribution p({Xi,j,Yi,j}) over all (i, j) look like?

The joint distribution is not fully specified since we do not know p({Yi,j})

Learning and Inference in Graphical Models. Chapter 10 – p. 24/38 Conditional random fields

Conditional random fields (CRF) overcome the problem of missing p({Yi,j}) by modeling only A E p({Xi,j}|{Yi,j}). This is sufficient if we do not want to make inference on {Yi,j} but only on {Xi,j} A conditional random field consists of C ◮ a set of observed nodes O B ◮ a set of unobserved random variables U ◮ edges between pairs of unobserved nodes D ◮ edges between observed and unobserved nodes Note that cliques in a conditional random field contain at most one observed node.

Learning and Inference in Graphical Models. Chapter 10 – p. 25/38 Conditional random fields

For every clique that contains at least one unobserved node we specifiy a potential function A E ψC : C → R>0 A CRF specifies the conditional distribution p(U|O) as 1 p(U|O)= ψC (C) C Z C Cliques ∈Y B

D

Learning and Inference in Graphical Models. Chapter 10 – p. 26/38 Example: facade segmentation

Segmentation of pictures into categories building/car/door/pavement/road/ sky/vegetation/window. Work of Michael Ying Yang

Approach: Hierarchical CRF combined with random decision forest. Result: c.f. Yang and Forstner,¨ 2011

Learning and Inference in Graphical Models. Chapter 10 – p. 27/38 Boltzmann Machines

Learning and Inference in Graphical Models. Chapter 10 – p. 28/38 Boltzmann machines

Definition: A Boltzmann machine is a fully connected MRF with binary random variables. Its energy function is defined over 1-cliques and 2-cliques by:

EX (x)= −θX · x

EX,Y (x,y)= −wX,Y · x · y

with θX , wX,Y non-negative real weight factors.

Hence, if we enumerate all random variables with X1,...,Xn − 1 Pn Pi 1 (w ·x ·x )+Pn (θ ·x ) p(x ,...,x )= e i=1 j=1 Xi,Xj i j i=1 Xi i 1 n Z Note, that wX,X =0 and wX,Y = wY,X .

Learning and Inference in Graphical Models. Chapter 10 – p. 29/38 Boltzmann machines

What is a Boltzmann machine good for?

Two tasks: ◮ pattern classification ◮ denoising of patterns

Learning and Inference in Graphical Models. Chapter 10 – p. 30/38 Boltzmann machines for pattern classification

Goal: we assume some patterns (data) which belong to different categories. Applying a pattern to the Boltzmann machine we want the Boltzman machine to return the appropriate class label. Structure of a Boltzmann machine for classification There are three different types of nodes: ◮ observed nodes O. We apply a pattern to the observed nodes by setting their value to the respective value of the pattern and never change it afterwards ◮ label nodes L. These serve as output of the Boltzmann machine. We have one label node for each class. Finally, the label nodes indicate the class probabilities for each class ◮ hidden nodes H. These nodes are unobserved and used for stochastic inference on the pattern

Learning and Inference in Graphical Models. Chapter 10 – p. 31/38 Boltzmann machines for pattern classification

Process of class predicition: 1. we apply a pattern to the observed nodes, i.e. the value of i-th observed node is set to the i-th value of the pattern. Afterwards, we do not change the observed nodes any more 2. we use Gibbs sampling to update the values of all hidden nodes H and label nodes L, i.e. we try to determine most probable configurations of p(L,H|O). If we are only interested in the most probable configuraton we might also use simulated annealing to find it. 3. after a while we interpret the label nodes. We might assume that the value of the i-th label node is proportional to the posterior probability of the i-th class

Learning and Inference in Graphical Models. Chapter 10 – p. 32/38 Gibbs sampling for Boltzmann machines

To implement Gibbs sampling we need to know p(Xi|X1,...,Xi−1,Xi+1,...,Xn) W.l.o.g. we get

p(Xn|X1,...,Xn−1) ∝ p(Xn,X1,...,Xn−1) − Pn Pi 1 (w ·x ·x )+Pn (θ ·x ) ∝ e i=1 j=1 Xi,Xj i j i=1 Xi i − − − − x ·Pn 1(w ·x )+θ ·x +Pn 1 Pi 1 (w ·x ·x )+Pn 1(θ ·x ) = e n j=1 Xn,Xj j Xn n i=1 j=1 Xi,Xj i j i=1 Xi i − − − − x ·Pn 1(w ·x )+θ ·x Pn 1 Pi 1 (w ·x ·x )+Pn 1(θ ·x ) = e n j=1 Xn,Xj j Xn n · e i=1 j=1 Xi,Xj i j i=1 Xi i − x ·Pn 1(w ·x )+θ ·x ∝ e n j=1 Xn,Xj j Xn n Hence, 1 p(X =0|X ,...,X )= · e0 n 1 n−1 Z − 1 Pn 1(w ·x )+θ p(X =1|X ,...,X )= · e j=1 Xn,Xj j Xn n 1 n−1 Z From p(Xn =0|X1,...,Xn−1)+ p(Xn =1|X1,...,Xn−1)=1 follows − Pn 1(w ·x )+θ Z =1+ e j=1 Xn,Xj j Xn

Learning and Inference in Graphical Models. Chapter 10 – p. 33/38 Boltzmann machines for denoising

Goal: we assume that all patterns have a typical structure. Applying a pattern we want the Boltzmann machine to return a typical pattern that is most similar to the pattern applied. Structure of a Boltzmann machine for denoising There are two different types of nodes: ◮ observed nodes O. We apply a pattern to the observed nodes by setting their value to the respective value of the pattern and never change it afterwards ◮ hidden nodes H. These nodes are unobserved and used for stochastic inference on the pattern

Learning and Inference in Graphical Models. Chapter 10 – p. 34/38 Boltzmann machines for denoising

Process of denoising: 1. we apply a pattern to the observed nodes, i.e. the value of i-th observed node is set to the i-th value of the pattern. 2. we use Gibbs sampling (or simulated annealing) to update the values of all hidden nodes H and observed nodes O, i.e. we try to determine most probable configurations of p(H,O). 3. after a while we consider the values of the observed nodes as pattern after denoising

Learning and Inference in Graphical Models. Chapter 10 – p. 35/38 Training of Boltzmann machines

For both tasks, we need to train a Boltzmann machine before we can use it, i.e. determine appropriate parameters wX,Y and θX Assume we are given T training examples (patterns and labels for the classification task, only patterns for the denoising task). Now, we want to maximize the likelihood w.r.t. wX,Y and θX T (t) (t) p(O ,L |{wX,Y |X,Y ∈ O ∪ H ∪ L}, {θX |X ∈ O ∪ H ∪ L}) t=1 →Ygradient ascent (calculating the gradient is not trivial)

Learning and Inference in Graphical Models. Chapter 10 – p. 36/38 Boltzmann machines

Some remarks on Boltzmann machines: ◮ training Boltzmann machines is very time-consuming ◮ however, there are more efficient variants (restricted Boltzmann machines, deep belief networks) which are subject to recent research and which are better suitable for pattern recognition and machine learning ◮ we do not want to discuss Boltzmann machines in depth in this lecture since they have been discussed in Prof. Sperschneider’s machine learning lecture already

Learning and Inference in Graphical Models. Chapter 10 – p. 37/38 Summary

◮ definition of Markov random fields • joint probability distribution • factor graph ◮ Potts model • image segmentation example ◮ Conditional random fields • image segmentation example of Michael Ying Yang ◮ Boltzmann machines

Learning and Inference in Graphical Models. Chapter 10 – p. 38/38