Jeroen Soeters NEUROEVOLUTION The Hitchhiker’s Guide to Neuroevolution in Erlang. JEROEN… HOW DO YOU PRONOUNCE THAT? Well, it goes like this: YUH-ROON 2 ARTIFICIAL NEURAL NETWORKS 3 BIOLOGICAL NEURAL NETWORKS Dendrites Synapse Soma Axon 4 A COMPUTER MODEL FOR A NEURON Dendrites Synapses Axon x1 w1 x2 w2 Soma Y wn xn 5 A COMPUTER MODEL FOR A NEURON Input signals Weights Output signal x1 w1 x2 w2 Neuron Y wn xn 5 HOW DOES THE NEURON DETERMINE IT’S OUTPUT? n Y =sign ∑ xiwi - ⍬ n=1 6 ACTIVATION FUNCTION Y X 7 MEET FRANK ROSENBLATT 8 PERCEPTRON LEARNING RULE ℮(p) = Yd(p) - Y(p) wi(p + 1) = wi(p) + α • wi(p) • ℮(p) = 9 PERCEPTRON TRAINING ALGORITHM set weights and threshold to start random values [-0.5, 0.5] activate the perceptron no weights converged? yes weight training stop 10 LOGIC GATES input values x1 AND x2 x1 OR x2 x1 XOR x2 x1 x2 0 0 0 0 0 0 1 0 1 1 1 0 0 1 1 1 1 1 1 0 11 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2 0 0 0 0.3 -0.1 0 0 0.3 -0.1 0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1 1 1 1 0.2 -0.1 0 1 0.3 0.0 Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 = 12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2 0 0 0 0.3 -0.1 0 0 0.3 -0.1 0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1 1 1 1 0.2 -0.1 0 1 0.3 0.0 Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 = 12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2 0 0 0 0.3 -0.1 0 0 0.3 -0.1 0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1 1 1 1 0.2 -0.1 0 1 0.3 0.0 Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 = 12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2 0 0 0 0.3 -0.1 0 0 0.3 -0.1 0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1 1 1 1 0.2 -0.1 0 1 0.3 0.0 Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 = 12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2 0 0 0 0.30.3 -0.1 0 0 0.3 -0.1 0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1 1 1 1 0.2 -0.1 0 1 0.3 0.0 Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 = 12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2 0 0 0 0.3 -0.1 0 0 0.3 -0.1 0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1 1 1 1 0.2 -0.1 0 1 0.3 0.0 Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 = 12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2 0 0 0 0.3 -0.1 0 0 0.3 -0.1 0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1 1 1 1 0.2 -0.1 0 1 0.3 0.0 Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 = 12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2 0 0 0 0.3 -0.1 0 0 0.3 -0.1 0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1 1 1 1 0.2 -0.1 0 1 0.3 0.0 Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 = 12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2 0 0 0 0.3 -0.1 0 0 0.3 -0.1 0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1 1 1 1 0.2 -0.1 0 1 0.3 0.0 Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 = 12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2 0 0 0 0.3 -0.1 0 0 0.3 -0.1 0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1 1 1 1 0.2 -0.1 0 1 0.3 0.0 Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 = 12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2 0 0 0 0.3 -0.1 0 0 0.3 -0.1 0 1 0 0.30.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1 1 1 1 0.2 -0.1 0 1 0.3 0.0 Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 = 12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2 0 0 0 0.3 -0.1 0 0 0.3 -0.1 0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1 1 1 1 0.2 -0.1 0 1 0.3 0.0 Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 = 12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2 0 0 0 0.3 -0.1 0 0 0.3 -0.1 0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1 1 1 1 0.2 -0.1 0 1 0.3 0.0 Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 = 12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2 0 0 0 0.3 -0.1 0 0 0.3 -0.1 0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1 1 1 1 0.2 -0.1 0 1 0.3 0.0 Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 = 12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2 0 0 0 0.3 -0.1 0 0 0.3 -0.1 0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1 1 1 1 0.2 -0.1 0 1 0.3 0.0 Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 = 12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2 0 0 0 0.3 -0.1 0 0 0.3 -0.1 0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1 1 1 1 0.2 -0.1 0 1 0.3 0.0 Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 = 12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2 0 0 0 0.3 -0.1 0 0 0.3 -0.1 0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.30.3 -0.1 1 -1 0.2 -0.1 1 1 1 0.2 -0.1 0 1 0.3 0.0 Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 = 12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2 0 0 0 0.3 -0.1 0 0 0.3 -0.1 0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1 1 1 1 0.2 -0.1 0 1 0.3 0.0 Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 = 12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2 0 0 0 0.3 -0.1 0 0 0.3 -0.1 0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1 1 1 1 0.2 -0.1 0 1 0.3 0.0 Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 = 12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2 0 0 0 0.3 -0.1 0 0 0.3 -0.1 0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1 1 1 1 0.2 -0.1 0 1 0.3 0.0 Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 = 12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2 0 0 0 0.3 -0.1 0 0 0.3 -0.1 0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1 1 1 1 0.2 -0.1 0 1 0.3 0.0 Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 = 12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2 0 0 0 0.3 -0.1 0 0 0.3 -0.1 0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1 1 1 1 0.2 -0.1 0 1 0.3 0.0 Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 = 12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2 0 0 0 0.3 -0.1 0 0 0.3 -0.1 0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1 1 1 1 0.2 -0.1 0 1 0.3 0.0 Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 = 12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2 0 0 0 0.3 -0.1 0 0 0.3 -0.1 0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1 1 1 1 0.2 -0.1 0 1 0.3 0.0 Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 = 12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2 0 0 0 0.3 -0.1 0 0 0.3 -0.1 0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1 1 1 1 0.2 -0.1 0 1 0.3 0.0 Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 = 12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2 0 0 0 0.3 -0.1 0 0 0.3 -0.1 0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1 1 1 1 0.2 -0.1 0 1 0.3 0.0 Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 = 12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2 0 0 0 0.3 -0.1 0 0 0.3 -0.1 0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1 1 1 1 0.2 -0.1 0 1 0.3 0.0 Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 = 12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2 0 0 0 0.3 -0.1 0