Jeroen Soeters
NEUROEVOLUTION
The Hitchhiker’s Guide to Neuroevolution in Erlang. JEROEN… HOW DO YOU PRONOUNCE THAT?
Well, it goes like this: YUH-ROON
2 ARTIFICIAL NEURAL NETWORKS
3 BIOLOGICAL NEURAL NETWORKS
Dendrites Synapse
Soma Axon
4 A COMPUTER MODEL FOR A NEURON
Dendrites Synapses Axon
x1 w1
x2 w2 Soma Y
wn xn
5 A COMPUTER MODEL FOR A NEURON
Input signals Weights Output signal
x1 w1
x2 w2 Neuron Y
wn xn
5 HOW DOES THE NEURON DETERMINE IT’S OUTPUT?
n
Y =sign ∑ xiwi - ⍬ n=1
6 ACTIVATION FUNCTION
Y
X
7 MEET FRANK ROSENBLATT
8 PERCEPTRON LEARNING RULE
℮(p) = Yd(p) - Y(p)
wi(p + 1) = wi(p) + α • wi(p) • ℮(p)
=
9 PERCEPTRON TRAINING ALGORITHM
set weights and threshold to start random values [-0.5, 0.5]
activate the perceptron no
weights converged? yes weight training stop
10 LOGIC GATES
input values x1 AND x2 x1 OR x2 x1 XOR x2 x1 x2
0 0 0 0 0
0 1 0 1 1
1 0 0 1 1
1 1 1 1 0
11 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 -0.1 0 0 0.3 -0.1
0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 -0.1 0 0 0.3 -0.1
0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 -0.1 0 0 0.3 -0.1
0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 -0.1 0 0 0.3 -0.1
0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.30.3 -0.1 0 0 0.3 -0.1
0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 -0.1 0 0 0.3 -0.1
0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 -0.1 0 0 0.3 -0.1
0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 -0.1 0 0 0.3 -0.1
0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 -0.1 0 0 0.3 -0.1
0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 -0.1 0 0 0.3 -0.1
0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 -0.1 0 0 0.3 -0.1
0 1 0 0.30.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 -0.1 0 0 0.3 -0.1
0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 -0.1 0 0 0.3 -0.1
0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 -0.1 0 0 0.3 -0.1
0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 -0.1 0 0 0.3 -0.1
0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 -0.1 0 0 0.3 -0.1
0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 -0.1 0 0 0.3 -0.1
0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.30.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 -0.1 0 0 0.3 -0.1
0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 -0.1 0 0 0.3 -0.1
0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 -0.1 0 0 0.3 -0.1
0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 -0.1 0 0 0.3 -0.1
0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 -0.1 0 0 0.3 -0.1
0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 -0.1 0 0 0.3 -0.1
0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 -0.1 0 0 0.3 -0.1
0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 -0.1 0 0 0.3 -0.1
0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 -0.1 0 0 0.3 -0.1
0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 -0.1 0 0 0.3 -0.1
0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 -0.1 0 0 0.3 -0.1
0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 0.0 0 0 0.3 0.0
0 1 0 0.3 0.0 0 0 0.3 0.0 2 1 0 0 0.3 0.0 1 -1 0.2 0.0
1 1 1 0.2 0.0 1 0 0.2 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
13 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 0.0 0 0 0.3 0.0
0 1 0 0.3 0.0 0 0 0.3 0.0 2 1 0 0 0.3 0.0 1 -1 0.2 0.0
1 1 1 0.2 0.0 1 0 0.2 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
13 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 0.0 0 0 0.3 0.0
0 1 0 0.3 0.0 0 0 0.3 0.0 2 1 0 0 0.3 0.0 1 -1 0.2 0.0
1 1 1 0.2 0.0 1 0 0.2 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
13 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.3 0.0 0 0 0.3 0.0
0 1 0 0.3 0.0 0 0 0.3 0.0 2 1 0 0 0.3 0.0 1 -1 0.2 0.0
1 1 1 0.2 0.0 1 0 0.2 0.0
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
13 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.2 0.0 0 0 0.2 0.0
0 1 0 0.2 0.0 0 0 0.2 0.0 3 1 0 0 0.2 0.0 1 -1 0.1 0.0
1 1 1 0.1 0.0 0 1 0.2 0.1
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
14 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.2 0.0 0 0 0.2 0.0
0 1 0 0.2 0.0 0 0 0.2 0.0 3 1 0 0 0.2 0.0 1 -1 0.1 0.0
1 1 1 0.1 0.0 0 1 0.2 0.1
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
14 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.2 0.0 0 0 0.2 0.0
0 1 0 0.2 0.0 0 0 0.2 0.0 3 1 0 0 0.2 0.0 1 -1 0.1 0.0
1 1 1 0.1 0.0 0 1 0.2 0.1
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
14 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.2 0.0 0 0 0.2 0.0
0 1 0 0.2 0.0 0 0 0.2 0.0 3 1 0 0 0.2 0.0 1 -1 0.1 0.0
1 1 1 0.1 0.0 0 1 0.2 0.1
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
14 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.2 0.1 0 0 0.2 0.1
0 1 0 0.2 0.1 0 0 0.2 0.1 4 1 0 0 0.2 0.1 1 -1 0.1 0.1
1 1 1 0.1 0.1 1 0 0.1 0.1
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
15 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.2 0.1 0 0 0.2 0.1
0 1 0 0.2 0.1 0 0 0.2 0.1 4 1 0 0 0.2 0.1 1 -1 0.1 0.1
1 1 1 0.1 0.1 1 0 0.1 0.1
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
15 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.2 0.1 0 0 0.2 0.1
0 1 0 0.2 0.1 0 0 0.2 0.1 4 1 0 0 0.2 0.1 1 -1 0.1 0.1
1 1 1 0.1 0.1 1 0 0.1 0.1
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
15 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.2 0.1 0 0 0.2 0.1
0 1 0 0.2 0.1 0 0 0.2 0.1 4 1 0 0 0.2 0.1 1 -1 0.1 0.1
1 1 1 0.1 0.1 1 0 0.1 0.1
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
15 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.1 0.1 0 0 0.1 0.1
0 1 0 0.1 0.1 0 0 0.1 0.1 5 1 0 0 0.1 0.1 0 0 0.1 0.1
1 1 1 0.1 0.1 1 0 0.1 0.1
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
16 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.1 0.1 0 0 0.1 0.1
0 1 0 0.1 0.1 0 0 0.1 0.1 5 1 0 0 0.1 0.1 0 0 0.1 0.1
1 1 1 0.1 0.1 1 0 0.1 0.1
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
16 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.1 0.1 0 0 0.1 0.1
0 1 0 0.1 0.1 0 0 0.1 0.1 5 1 0 0 0.1 0.1 0 0 0.1 0.1
1 1 1 0.1 0.1 1 0 0.1 0.1
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
16 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION
inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2
0 0 0 0.1 0.1 0 0 0.1 0.1
0 1 0 0.1 0.1 0 0 0.1 0.1 5 1 0 0 0.1 0.1 0 0 0.1 0.1
1 1 1 0.1 0.1 1 0 0.1 0.1
Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =
16 A LITTLE GEOMETRY…
x2 x2 x2
1 1 1
0 1 x1 0 1 x1 0 1 x1
x1 AND x2 x1 OR x2 x1 XOR x2
17 WE NEED MORE LAYERS
x1 1 3
Y 5
2 4 x2
Input layer Hidden layer Output layer
18 LIMITATIONS OF BACK PROPAGATION
•a training set of sufficient size is required •topology of the network needs to be known in advance •no recurrent connections are allowed •activation function must be differentiable
Only works well for supervised learning!
20 MEET JOHN HOLLAND
21 THE CHROMOSOME
1 0 0 1 1 1 0 1
22 CROSSOVER parents
1 1 0 1 0 1 0 1 0 0 1 1 1 0 0 1
23 CROSSOVER parents
1 1 0 1 0 1 0 1 0 0 1 1 1 0 0 1 ✂ ✂
23 CROSSOVER parents
1 1 0 1 0 1 0 1 0 0 1 1 1 0 0 1 ✂ ✂
offspring
1 1 0 1 1 0 0 1
23 MUTATION
1 0 0 1 1 A1 D0 1
24 MUTATION
1 0 0 1 1 A1 D0 1
24 MUTATION
1 0 0 1 1 A10 D01 1
•represent the candidate solution as a chromosome
•define the genetic operators for the chromosome •define a fitness function to measure the performance of the chromosome
•chose the initial population size N, crossover probability (Pc) and mutation
probability (Pm)
25 EVOLUTIONARY ALGORITHM
start generate a population replace population
yes
no calculate fitness
termination criteria satisfied? new population size = N?
select pair for mating add to new population no
yes
stop crossover and mutation
26 TRAVELING SALESMAN
C
A D
B
E
F
G H
27 EVOLUTIONARY ALGORITHM
•represent the candidate solution as a chromosome •define the genetic operators for the chromosome •define a fitness function to measure the performance of the chromosome
•chose the initial population size N, crossover probability Pc and mutation
probability Pm
28 THE CHROMOSOME
C B G E H A D F
29 EVOLUTIONARY ALGORITHM
•represent the candidate solution as a chromosome
•define the genetic operators for the chromosome •define a fitness function to measure the performance of the chromosome
•chose the initial population size N, crossover probability Pc and mutation
probability Pm
30 CROSSOVER parents
C B G E H A D F B E A F H C G D
31 CROSSOVER parents
C B G E H A D F B E A F H C G D ✂ ✂
31 CROSSOVER parents
C B G E H A D F B E A F H C G D ✂ ✂
offspring
H A D
31 CROSSOVER parents
C B G E H A D F B E A F H C G D ✂ ✂
offspring
B H A D
31 CROSSOVER parents
C B G E H A D F B E A F H C G D ✂ ✂
offspring
B E H A D
31 CROSSOVER parents
C B G E H A D F B E A F H C G D ✂ ✂
offspring
B E F H A D
31 CROSSOVER parents
C B G E H A D F B E A F H C G D ✂ ✂
offspring
B E F C H A D
31 CROSSOVER parents
C B G E H A D F B E A F H C G D ✂ ✂
offspring
B E F C H A D G
31 MUTATION
C B G E H A D F
32 MUTATION
C B G E H A D F
32 MUTATION
C B G E H DA DA F
32 EVOLUTIONARY ALGORITHM
•represent the candidate solution as a chromosome
•define the genetic operators for the chromosome •define a fitness function to measure the performance of the chromosome
•chose the initial population size N, crossover probability Pc and mutation
probability Pm
Fitness = 1 / total distance
34 EVOLUTIONARY ALGORITHM
•represent the candidate solution as a chromosome •define a fitness function to measure the performance of the chromosome •define the genetic operators for the chromosome
•chose the initial population size N, crossover probability Pc and
mutation probability Pm
35 DEMO
36 DEMO
36 NEUROEVOLUTION
37 MEET GENE SHER
38 SIMULATION
•sensors (provide input for the network) •actuators (act upon output from the network) •fitness function
39 CLEANING ROBOT
40 FOREX TRADING
41 AND LOTS MORE…
•data compression •training NPCs in a video game •cyber warfare
42 EVOLUTIONARY ALGORITHM
•represent the candidate solution as a chromosome
•define the genetic operators for the chromosome •define a fitness function to measure the performance of the chromosome
•chose the initial population size N, crossover probability (Pc) and mutation
probability (Pm)
43 EVOLUTIONARY ALGORITHM
•represent the candidate solution as a chromosome •define the genetic operators for the chromosome •define a fitness function to measure the performance of the chromosome
•chose the initial population size N, crossover probability Pc and mutation
probability Pm
44 THE CHROMOSOME
# neuron # neuron { { id: 4, id: 2, af: cosine, af: gaussian, input_ids_and_weights: input_ids_and_weights: [ [{sensor, 1, 0.64}] {actuator, 6, 0.77} output_ids: ], } output_ids: [6] }
# sensor # actuator { { id: 1, id: 6, fanout_ids: [2,3] fanin_ids: [4,5] } }
# neuron # neuron { { id: 3, id: 5, af: gaussian, af: gaussian, input_ids_and_weights: input_ids_and_weights: [{sensor, 1, 0.10}] [ output_ids: {neuron, 2, 0.64}, } {neuron, 3, -0.55} ], output_ids: [6] } 45 EVOLUTIONARY ALGORITHM
•represent the candidate solution as a chromosome
•define the genetic operators for the chromosome •define a fitness function to measure the performance of the chromosome
•chose the initial population size N, crossover probability Pc and mutation
probability Pm
46 CROSSOVER
Crossover doesn’t work for large neural nets!
47 MUTATE ACTIVATION FUNCTION
48 MUTATE ACTIVATION FUNCTION
48 ADD CONNECTION
49 ADD CONNECTION
49 ADD NEURON
50 ADD NEURON
50 OUTSPLICE
51 OUTSPLICE
51 MUTATION OPERATORS
and lots more…
52 EVOLUTIONARY ALGORITHM
•represent the candidate solution as a chromosome
•define the genetic operators for the chromosome •define a fitness function to measure the performance of the chromosome
•chose the initial population size N, crossover probability Pc and mutation
probability Pm
53 FITNESS FUNCTION
Fitness = performance of network on an actual problem
54 EVOLUTIONARY ALGORITHM
•represent the candidate solution as a chromosome
•define the genetic operators for the chromosome •define a fitness function to measure the performance of the chromosome
•chose the initial population size N, crossover probability Pc and
mutation probability Pm
55 RANDOM INTENSITY MUTATION
Number of mutations = random(1, network size)
start
generate a population
apply to a problem
local search:
create offspring Hill Climber calculate effective fitness
select fit organisms
57 STOCHASTIC HILL CLIMBER (LOCAL SEARCH)
start
apply NN to a problem
new fitness > best fitness so far?
perturb weights back up weights restore weights yes no
no stopping condition reached? yes
stop
58 MEET JOE ARMSTRONG
59 1:1 MAPPING
neuron neuron
genotype
neuron neuron
process process
phenotype / process process erlang
process process
60 THE NEURAL NETWORK
cortex
neuron neuron
sensor actuator
neuron neuron
scape
61 THE NEURAL NETWORK
cortex
sync
neuron neuron
sensor actuator
neuron neuron
scape
61 THE NEURAL NETWORK
cortex
neuron neuron
sensor actuator
neuron neuron
scape sense
61 THE NEURAL NETWORK
cortex
neuron neuron
sensor actuator
percept neuron neuron
scape
61 THE NEURAL NETWORK
cortex
forward neuron neuron
sensor actuator
neuron neuron forward
scape
61 THE NEURAL NETWORK
cortex
forward neuron neuron
sensor actuator
forward neuron neuron
scape
61 THE NEURAL NETWORK
cortex
neuron neuron forward
sensor actuator
neuron neuron forward
scape
61 THE NEURAL NETWORK
cortex
neuron neuron
sensor actuator
neuron neuron
scape action
61 THE NEURAL NETWORK
cortex
neuron neuron
sensor actuator {fitness, halt_flag}
neuron neuron
scape
61 THE NEURAL NETWORK
cortex sync
neuron neuron
sensor actuator
neuron neuron
scape
61 THE NEURAL NETWORK
cortex
neuron neuron
sensor actuator
neuron neuron
scape
61 THE EXOSELF
exoself
cortex
neuron neuron
sensor actuator
neuron neuron
scape
62 THE EXOSELF
exoself
{evaluation_completed, fitness}
cortex
neuron neuron
sensor actuator
neuron neuron
scape
62 THE EXOSELF
exoself fitness > best fitness
cortex
neuron neuron
sensor actuator
neuron neuron
scape
62 THE EXOSELF
exoself
cortex backup_weights backup_weights
neuron neuronneuron
backup_weightssensor backup_weightsactuator
neuron neuron
scape
62 THE EXOSELF
exoself
cortex perturb_weights
neuron neuronneuron
perturb_weights
sensor actuator
neuron neuron
scape
62 THE EXOSELF
exoself
cortex
neuron neuron
sensor actuator
neuron neuron
scape
62 THE EXOSELF
exoself
reactivate
cortex
neuron neuron
sensor actuator
neuron neuron
scape
62 THE POPULATION MONITOR
population monitor database
63 THE POPULATION MONITOR
population monitor database
private private private private private
63 THE POPULATION MONITOR
population monitor database
start start start start start
private private private private private
63 THE POPULATION MONITOR
population monitor database
private private private private private
63 THE POPULATION MONITOR
population monitor database
terminated terminated
terminated
terminated terminated
private private private private private
63 THE POPULATION MONITOR
population monitor database
private private private private private
63 THE POPULATION MONITOR
population monitor database
private private private
63 THE POPULATION MONITOR
population monitor database
private private private private private
63 THE POPULATION MONITOR
population monitor database
start start start start start
private private private private private
63 THE DEVIL IS IN THE DETAILS
•recurrent connections •newer generations get a higher chance for mutation •neural plasticity •public scapes and steady state evolution
64 POLE BALANCING
agent percepts
actions cart
65 DEMO
66 DEMO
66 BENCHMARK RESULTS
Method Double-Pole/Partial Information W/O Double-Pole W/Damping Damping
RWG 415209 1232296
SANE 262700 451612
CNE 76906 87623
ESP 7374 26342
NEAT - 6929
CMA-ES* 3521* 6061*
CoSyNE* 1249* 3416*
DXNN v1 2359 2313
DXNN v2 1289 1830
67 THE HANDBOOK
68 THANK YOU
Feel free to reach out at:
e. [email protected] t. @JeroenSoeters