CSC 578 Neural Networks and

9. Hopfield Networks, Boltzmann Machines

1 Noriko Tomuro Unsupervised Neural Networks

1. Hopfield Networks 1. Concepts 2. Boltzmann Machines 1. Concepts 2. Restricted Boltzmann Machines 3. Deep Boltzmann Machines

2 Noriko Tomuro 1 • A Hopfield network is a form of recurrent artificial neural network. It is also one of the oldest neural networks. • There are many variations. The one presented here is a discrete network which takes bipolar inputs (1 or -1). • Hopfield network stores patterns -- then recovers stored patterns from partial or corrupted patterns. -- Associative Memory. • Hopfield networks also have been applied to combinatorial optimization problems, e.g. Traveling Salesman Problem

3 Noriko Tomuro 4 Overview of Hopfield Network x1 • Weights between units are bi- directional (thus "Feedback" or

"Recurrent" network) => A network W21= W12 W31= W13 is a fully-connected network (but no x3 self-loop weights, i.e., wii = 0). x2 • Each unit/node represents a W23= W32 neuron. And the activation of a   1, if ∑ w ij ⋅ x j ≥ 0 neuron is (in the case of binary x = j i  network; similar to thresholded −1, otherwise )

• If xi is 1, a unit is called "active", and if -1, it is called "inactive". • Every neuron functions as both input and output unit.

5 • A state of a network is defined by the activation of nodes -- . • Given a set of (current) weights, values of nodes are updated asynchronously (parallel relaxation). 1. Pick a node randomly, and compute the new activation for that node -- a node fires if it becomes 1. 2. Repeat the procedure until no node changes value. • Then, the network settles in to one of the stable state.

6 (0) Pattern <-1, -1, -1> presented (1) After activating x2 x1 x1

-1 -1 -2 1 -2 1

x2 -1 -1 x3 x2 1 -1 x3 1 1

(2) After activating x1 (3) After activating x3 x1 x1

-1 -1 -2 1 -2 1

1 1 x3 x2 1 -1 x3 x2 1 1

(4), (5), (6) After activating x1 through x3 again, no more state change occurs.

Noriko Tomuro 7 • It is proven that Hopfield network with asynchronous update will always converge to a stable state. • Depending on the weights and input values, there may be several states to which the network converges. • The change of the network state is essentially a search through the possible state space. • Closeness to a stable state is measured by the notion of

energy = +

𝐸𝐸 − � 𝑤𝑤𝑖𝑖𝑖𝑖𝑠𝑠𝑖𝑖𝑠𝑠𝑗𝑗 � 𝜃𝜃𝑖𝑖 𝑠𝑠𝑖𝑖 – j is the connection𝑖𝑖<𝑗𝑗 weight between𝑖𝑖 unit j and unit i. – is the state, s ∈ {0,1}. 𝑤𝑤𝑖𝑖𝑖𝑖 i – is the bias of unit i. (- is the activation threshold for the unit). 𝑠𝑠𝑖𝑖 𝜃𝜃𝑖𝑖 𝜃𝜃𝑖𝑖 When a node is activated, the change in the energy is always <= 0.

8 • So, searching for a stable state is a minimization problem, and the same can be used to find the minimum. • However, there is the danger of getting stuck in a local minima -- the convergence goes to the closest local minima.

9 • Capacity limitation of Hopfield networks: It has been shown that Hopfield networks can only memorize limited numbers of patterns.

A Hopfield network with N nodes can store M patterns where • M = 0.15N (for binary network), or • M = N / 2 log2N (for bipolar network)

• A newer result showed the recall accuracy between vectors and nodes was 0.138 (approximately 138 vectors can be recalled from storage for every 1000 nodes) (Hertz et al., 1991).

10 • Training for Hopfield networks: Values in the input patterns are bipolar: e.g. <1, -1, 1> Weights are updated incrementally. We basically want the stored patterns to be the stables states.

0. Initialize network weights. 1. Do until no change in the weights occur 2. Initialize delta_w's to be 0.0. 3. For each pattern d in the training set, do 4. Present the pattern to the network. 5. For each node xi, 6. If xi's activation is different from input xi, 7. update the weights connected to xi.

And the weight update for node xi is

∆wij ←η ⋅ xi ⋅ x j where eta is the learning rate, and xi, xj are the values in the input pattern d.

11 Modern Hopfield networks: • Instead of using the net to store memories, we use it to construct interpretations of sensory input. The input is represented by the visible units, the interpretation is represented by the states of the hidden units, and the badness of the interpretation is represented by the energy. [Video by Geoff Hinton, 2012]

12 Summary: Hopfield networks • suffer from spurious local minima that form on the energy hypersurface. • require the input patterns to be uncorrelated. • are limited in capacity of patterns that can be stored. • are usually fully connected and not stacked.

13 2 Boltzmann Machines • A (also called stochastic Hopfield network with hidden units) is a type of stochastic (and Markov ). • Its units produce binary results. Unlike Hopfield nets, Boltzmann machine units are stochastic.[Wikipedia]

A graphical representation of an example Boltzmann machine. Each undirected edge represents dependency. In this example there are 3 hidden units and 4 visible units.

14 https://keras.io/getting-started/sequential-model-guide/ • The global energy in a Boltzmann machine is identical to that of a Hopfield network: = + where 𝐸𝐸 − � 𝑤𝑤𝑖𝑖𝑖𝑖𝑠𝑠𝑖𝑖𝑠𝑠𝑗𝑗 � 𝜃𝜃𝑖𝑖 𝑠𝑠𝑖𝑖 𝑖𝑖<𝑗𝑗 𝑖𝑖 – j is the connection weight between unit j and unit i. – is the state, s ∈ {0,1}. 𝑤𝑤𝑖𝑖𝑖𝑖 i – is the bias of unit i. (- is the activation threshold for the unit). 𝑠𝑠𝑖𝑖 𝜃𝜃𝑖𝑖 𝜃𝜃𝑖𝑖 • The goal of learning for Boltzmann algorithm is to maximize the product of the probabilities that the Boltzmann machine assigns to the binary vectors in the training set. • For the connection between the assigned probabilities and the energy (and temperature) consult this [Wikipedia] page.

15 Noriko Tomuro 16 • Training of Boltzmann Machines usually use KL-divergence, or log likelihood. The loss function G (for binary vectors) is

( ) = ( ) ln ( ) + where + 𝑃𝑃− 𝑣𝑣 – ( ) is the𝐺𝐺 distribution� 𝑃𝑃 𝑣𝑣 � over𝑃𝑃 the𝑣𝑣 training set V 𝑣𝑣 – +( ) is the distribution over the visible (i.e., not hidden) units in the 𝑃𝑃 𝑣𝑣 network− 𝑃𝑃 𝑣𝑣 And we want to maximize G since the probabilities are not negated.

17 Restricted Boltzmann Machines • It was discovered that original Boltzmann Machines stop learning correctly when the machine is scaled up to anything larger than a trivial machine. • Then in 2014, an architecture called the "restricted Boltzmann machine" or "RBM“ was invented. • RBM does not allow intralayer connections between hidden units. This type of architecture was shown to make inference and learning easier.

18 19 • RBM Learning:

20 Deep Restricted Boltzmann Machines

21

RBM vs. Autoencoder • Autoencoder is a simple 3-layer neural network where output units are directly connected back to input units. The task of training is to minimize an error or reconstruction, i.e. find the most efficient compact representation (encoding) for input data. • RBM shares similar idea, but uses stochastic stochastic units with particular (usually binary of Gaussian) distribution. The task of training is to find out how visible random variables are actually connected/related to hidden random variables.

23 https://www.quora.com/What-is-the-difference-between-autoencoders-and-a-restricted-Boltzmann-machine RBM vs. GAN • Generative adversarial networks (GANs) are a class of algorithms used in unsupervised machine learning, implemented by a system of two neural networks contesting with each other in a zero-sum game framework.

24 https://en.wikipedia.org/wiki/Generative_adversarial_network 25 https://stats.stackexchange.com/questions/338328/restricted-boltzmann-machines-vs-gan