Jeroen Soeters

NEUROEVOLUTION

The Hitchhiker’s Guide to in Erlang. JEROEN… HOW DO YOU PRONOUNCE THAT?

Well, it goes like this: YUH-ROON

2 ARTIFICIAL NEURAL NETWORKS

3 BIOLOGICAL NEURAL NETWORKS

Dendrites Synapse

Soma Axon

4 A COMPUTER MODEL FOR A NEURON

Dendrites Synapses Axon

x1 w1

x2 w2 Soma Y

wn xn

5 A COMPUTER MODEL FOR A NEURON

Input signals Weights Output signal

x1 w1

x2 w2 Neuron Y

wn xn

5 HOW DOES THE NEURON DETERMINE IT’S OUTPUT?

n

Y =sign ∑ xiwi - ⍬ n=1

6 ACTIVATION FUNCTION

Y

X

7 MEET FRANK ROSENBLATT

8 PERCEPTRON LEARNING RULE

℮(p) = Yd(p) - Y(p)

wi(p + 1) = wi(p) + α • wi(p) • ℮(p)

=

9 PERCEPTRON TRAINING ALGORITHM

set weights and threshold to start random values [-0.5, 0.5]

activate the perceptron no

weights converged? yes weight training stop

10 LOGIC GATES

input values x1 AND x2 x1 OR x2 x1 XOR x2 x1 x2

0 0 0 0 0

0 1 0 1 1

1 0 0 1 1

1 1 1 1 0

11 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 -0.1 0 0 0.3 -0.1

0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1

1 1 1 0.2 -0.1 0 1 0.3 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 -0.1 0 0 0.3 -0.1

0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1

1 1 1 0.2 -0.1 0 1 0.3 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 -0.1 0 0 0.3 -0.1

0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1

1 1 1 0.2 -0.1 0 1 0.3 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 -0.1 0 0 0.3 -0.1

0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1

1 1 1 0.2 -0.1 0 1 0.3 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.30.3 -0.1 0 0 0.3 -0.1

0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1

1 1 1 0.2 -0.1 0 1 0.3 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 -0.1 0 0 0.3 -0.1

0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1

1 1 1 0.2 -0.1 0 1 0.3 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 -0.1 0 0 0.3 -0.1

0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1

1 1 1 0.2 -0.1 0 1 0.3 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 -0.1 0 0 0.3 -0.1

0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1

1 1 1 0.2 -0.1 0 1 0.3 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 -0.1 0 0 0.3 -0.1

0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1

1 1 1 0.2 -0.1 0 1 0.3 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 -0.1 0 0 0.3 -0.1

0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1

1 1 1 0.2 -0.1 0 1 0.3 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 -0.1 0 0 0.3 -0.1

0 1 0 0.30.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1

1 1 1 0.2 -0.1 0 1 0.3 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 -0.1 0 0 0.3 -0.1

0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1

1 1 1 0.2 -0.1 0 1 0.3 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 -0.1 0 0 0.3 -0.1

0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1

1 1 1 0.2 -0.1 0 1 0.3 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 -0.1 0 0 0.3 -0.1

0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1

1 1 1 0.2 -0.1 0 1 0.3 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 -0.1 0 0 0.3 -0.1

0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1

1 1 1 0.2 -0.1 0 1 0.3 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 -0.1 0 0 0.3 -0.1

0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1

1 1 1 0.2 -0.1 0 1 0.3 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 -0.1 0 0 0.3 -0.1

0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.30.3 -0.1 1 -1 0.2 -0.1

1 1 1 0.2 -0.1 0 1 0.3 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 -0.1 0 0 0.3 -0.1

0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1

1 1 1 0.2 -0.1 0 1 0.3 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 -0.1 0 0 0.3 -0.1

0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1

1 1 1 0.2 -0.1 0 1 0.3 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 -0.1 0 0 0.3 -0.1

0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1

1 1 1 0.2 -0.1 0 1 0.3 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 -0.1 0 0 0.3 -0.1

0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1

1 1 1 0.2 -0.1 0 1 0.3 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 -0.1 0 0 0.3 -0.1

0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1

1 1 1 0.2 -0.1 0 1 0.3 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 -0.1 0 0 0.3 -0.1

0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1

1 1 1 0.2 -0.1 0 1 0.3 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 -0.1 0 0 0.3 -0.1

0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1

1 1 1 0.2 -0.1 0 1 0.3 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 -0.1 0 0 0.3 -0.1

0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1

1 1 1 0.2 -0.1 0 1 0.3 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 -0.1 0 0 0.3 -0.1

0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1

1 1 1 0.2 -0.1 0 1 0.3 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 -0.1 0 0 0.3 -0.1

0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1

1 1 1 0.2 -0.1 0 1 0.3 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 -0.1 0 0 0.3 -0.1

0 1 0 0.3 -0.1 0 0 0.3 -0.1 1 1 0 0 0.3 -0.1 1 -1 0.2 -0.1

1 1 1 0.2 -0.1 0 1 0.3 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

12 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 0.0 0 0 0.3 0.0

0 1 0 0.3 0.0 0 0 0.3 0.0 2 1 0 0 0.3 0.0 1 -1 0.2 0.0

1 1 1 0.2 0.0 1 0 0.2 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

13 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 0.0 0 0 0.3 0.0

0 1 0 0.3 0.0 0 0 0.3 0.0 2 1 0 0 0.3 0.0 1 -1 0.2 0.0

1 1 1 0.2 0.0 1 0 0.2 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

13 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 0.0 0 0 0.3 0.0

0 1 0 0.3 0.0 0 0 0.3 0.0 2 1 0 0 0.3 0.0 1 -1 0.2 0.0

1 1 1 0.2 0.0 1 0 0.2 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

13 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.3 0.0 0 0 0.3 0.0

0 1 0 0.3 0.0 0 0 0.3 0.0 2 1 0 0 0.3 0.0 1 -1 0.2 0.0

1 1 1 0.2 0.0 1 0 0.2 0.0

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

13 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.2 0.0 0 0 0.2 0.0

0 1 0 0.2 0.0 0 0 0.2 0.0 3 1 0 0 0.2 0.0 1 -1 0.1 0.0

1 1 1 0.1 0.0 0 1 0.2 0.1

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

14 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.2 0.0 0 0 0.2 0.0

0 1 0 0.2 0.0 0 0 0.2 0.0 3 1 0 0 0.2 0.0 1 -1 0.1 0.0

1 1 1 0.1 0.0 0 1 0.2 0.1

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

14 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.2 0.0 0 0 0.2 0.0

0 1 0 0.2 0.0 0 0 0.2 0.0 3 1 0 0 0.2 0.0 1 -1 0.1 0.0

1 1 1 0.1 0.0 0 1 0.2 0.1

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

14 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.2 0.0 0 0 0.2 0.0

0 1 0 0.2 0.0 0 0 0.2 0.0 3 1 0 0 0.2 0.0 1 -1 0.1 0.0

1 1 1 0.1 0.0 0 1 0.2 0.1

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

14 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.2 0.1 0 0 0.2 0.1

0 1 0 0.2 0.1 0 0 0.2 0.1 4 1 0 0 0.2 0.1 1 -1 0.1 0.1

1 1 1 0.1 0.1 1 0 0.1 0.1

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

15 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.2 0.1 0 0 0.2 0.1

0 1 0 0.2 0.1 0 0 0.2 0.1 4 1 0 0 0.2 0.1 1 -1 0.1 0.1

1 1 1 0.1 0.1 1 0 0.1 0.1

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

15 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.2 0.1 0 0 0.2 0.1

0 1 0 0.2 0.1 0 0 0.2 0.1 4 1 0 0 0.2 0.1 1 -1 0.1 0.1

1 1 1 0.1 0.1 1 0 0.1 0.1

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

15 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.2 0.1 0 0 0.2 0.1

0 1 0 0.2 0.1 0 0 0.2 0.1 4 1 0 0 0.2 0.1 1 -1 0.1 0.1

1 1 1 0.1 0.1 1 0 0.1 0.1

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

15 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.1 0.1 0 0 0.1 0.1

0 1 0 0.1 0.1 0 0 0.1 0.1 5 1 0 0 0.1 0.1 0 0 0.1 0.1

1 1 1 0.1 0.1 1 0 0.1 0.1

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

16 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.1 0.1 0 0 0.1 0.1

0 1 0 0.1 0.1 0 0 0.1 0.1 5 1 0 0 0.1 0.1 0 0 0.1 0.1

1 1 1 0.1 0.1 1 0 0.1 0.1

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

16 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.1 0.1 0 0 0.1 0.1

0 1 0 0.1 0.1 0 0 0.1 0.1 5 1 0 0 0.1 0.1 0 0 0.1 0.1

1 1 1 0.1 0.1 1 0 0.1 0.1

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

16 TRAINING A PERCEPTRON TO PERFORM THE AND OPERATION

inputs desired initial weights actual final weights epoch output output error x1 x2 Yd w1 w2 Y ℮ w1 w2

0 0 0 0.1 0.1 0 0 0.1 0.1

0 1 0 0.1 0.1 0 0 0.1 0.1 5 1 0 0 0.1 0.1 0 0 0.1 0.1

1 1 1 0.1 0.1 1 0 0.1 0.1

Threshold: ⍬ = 0.2 ; learning rate: α = 0.1 =

16 A LITTLE GEOMETRY…

x2 x2 x2

1 1 1

0 1 x1 0 1 x1 0 1 x1

x1 AND x2 x1 OR x2 x1 XOR x2

17 WE NEED MORE LAYERS

x1 1 3

Y 5

2 4 x2

Input layer Hidden layer Output layer

18 LIMITATIONS OF BACK PROPAGATION

•a training set of sufficient size is required •topology of the network needs to be known in advance •no recurrent connections are allowed •activation function must be differentiable

Only works well for supervised learning!

19

20 MEET JOHN HOLLAND

21 THE CHROMOSOME

1 0 0 1 1 1 0 1

22 CROSSOVER parents

1 1 0 1 0 1 0 1 0 0 1 1 1 0 0 1

23 CROSSOVER parents

1 1 0 1 0 1 0 1 0 0 1 1 1 0 0 1 ✂ ✂

23 CROSSOVER parents

1 1 0 1 0 1 0 1 0 0 1 1 1 0 0 1 ✂ ✂

offspring

1 1 0 1 1 0 0 1

23 MUTATION

1 0 0 1 1 A1 D0 1

24 MUTATION

1 0 0 1 1 A1 D0 1

24 MUTATION

1 0 0 1 1 A10 D01 1

24

•represent the candidate solution as a chromosome

•define the genetic operators for the chromosome •define a fitness function to measure the performance of the chromosome

•chose the initial population size N, crossover probability (Pc) and mutation

probability (Pm)

25 EVOLUTIONARY ALGORITHM

start generate a population replace population

yes

no calculate fitness

termination criteria satisfied? new population size = N?

select pair for mating add to new population no

yes

stop crossover and mutation

26 TRAVELING SALESMAN

C

A D

B

E

F

G H

27 EVOLUTIONARY ALGORITHM

•represent the candidate solution as a chromosome •define the genetic operators for the chromosome •define a fitness function to measure the performance of the chromosome

•chose the initial population size N, crossover probability Pc and mutation

probability Pm

28 THE CHROMOSOME

C B G E H A D F

29 EVOLUTIONARY ALGORITHM

•represent the candidate solution as a chromosome

•define the genetic operators for the chromosome •define a fitness function to measure the performance of the chromosome

•chose the initial population size N, crossover probability Pc and mutation

probability Pm

30 CROSSOVER parents

C B G E H A D F B E A F H C G D

31 CROSSOVER parents

C B G E H A D F B E A F H C G D ✂ ✂

31 CROSSOVER parents

C B G E H A D F B E A F H C G D ✂ ✂

offspring

H A D

31 CROSSOVER parents

C B G E H A D F B E A F H C G D ✂ ✂

offspring

B H A D

31 CROSSOVER parents

C B G E H A D F B E A F H C G D ✂ ✂

offspring

B E H A D

31 CROSSOVER parents

C B G E H A D F B E A F H C G D ✂ ✂

offspring

B E F H A D

31 CROSSOVER parents

C B G E H A D F B E A F H C G D ✂ ✂

offspring

B E F C H A D

31 CROSSOVER parents

C B G E H A D F B E A F H C G D ✂ ✂

offspring

B E F C H A D G

31 MUTATION

C B G E H A D F

32 MUTATION

C B G E H A D F

32 MUTATION

C B G E H DA DA F

32 EVOLUTIONARY ALGORITHM

•represent the candidate solution as a chromosome

•define the genetic operators for the chromosome •define a fitness function to measure the performance of the chromosome

•chose the initial population size N, crossover probability Pc and mutation

probability Pm

33

Fitness = 1 / total distance

34 EVOLUTIONARY ALGORITHM

•represent the candidate solution as a chromosome •define a fitness function to measure the performance of the chromosome •define the genetic operators for the chromosome

•chose the initial population size N, crossover probability Pc and

mutation probability Pm

35 DEMO

36 DEMO

36 NEUROEVOLUTION

37 MEET GENE SHER

38 SIMULATION

•sensors (provide input for the network) •actuators (act upon output from the network) •fitness function

39 CLEANING ROBOT

40 FOREX TRADING

41 AND LOTS MORE…

•data compression •training NPCs in a video game •cyber warfare

42 EVOLUTIONARY ALGORITHM

•represent the candidate solution as a chromosome

•define the genetic operators for the chromosome •define a fitness function to measure the performance of the chromosome

•chose the initial population size N, crossover probability (Pc) and mutation

probability (Pm)

43 EVOLUTIONARY ALGORITHM

•represent the candidate solution as a chromosome •define the genetic operators for the chromosome •define a fitness function to measure the performance of the chromosome

•chose the initial population size N, crossover probability Pc and mutation

probability Pm

44 THE CHROMOSOME

# neuron # neuron { { id: 4, id: 2, af: cosine, af: gaussian, input_ids_and_weights: input_ids_and_weights: [ [{sensor, 1, 0.64}] {actuator, 6, 0.77} output_ids: ], } output_ids: [6] }

# sensor # actuator { { id: 1, id: 6, fanout_ids: [2,3] fanin_ids: [4,5] } }

# neuron # neuron { { id: 3, id: 5, af: gaussian, af: gaussian, input_ids_and_weights: input_ids_and_weights: [{sensor, 1, 0.10}] [ output_ids: {neuron, 2, 0.64}, } {neuron, 3, -0.55} ], output_ids: [6] } 45 EVOLUTIONARY ALGORITHM

•represent the candidate solution as a chromosome

•define the genetic operators for the chromosome •define a fitness function to measure the performance of the chromosome

•chose the initial population size N, crossover probability Pc and mutation

probability Pm

46 CROSSOVER

Crossover doesn’t work for large neural nets!

47 MUTATE ACTIVATION FUNCTION

48 MUTATE ACTIVATION FUNCTION

48 ADD CONNECTION

49 ADD CONNECTION

49 ADD NEURON

50 ADD NEURON

50 OUTSPLICE

51 OUTSPLICE

51 MUTATION OPERATORS

and lots more…

52 EVOLUTIONARY ALGORITHM

•represent the candidate solution as a chromosome

•define the genetic operators for the chromosome •define a fitness function to measure the performance of the chromosome

•chose the initial population size N, crossover probability Pc and mutation

probability Pm

53 FITNESS FUNCTION

Fitness = performance of network on an actual problem

54 EVOLUTIONARY ALGORITHM

•represent the candidate solution as a chromosome

•define the genetic operators for the chromosome •define a fitness function to measure the performance of the chromosome

•chose the initial population size N, crossover probability Pc and

mutation probability Pm

55 RANDOM INTENSITY MUTATION

Number of mutations = random(1, network size)

56

start

generate a population

apply to a problem

local search:

create offspring Hill Climber calculate effective fitness

select fit organisms

57 STOCHASTIC HILL CLIMBER (LOCAL SEARCH)

start

apply NN to a problem

new fitness > best fitness so far?

perturb weights back up weights restore weights yes no

no stopping condition reached? yes

stop

58 MEET JOE ARMSTRONG

59 1:1 MAPPING

neuron neuron

genotype

neuron neuron

process process

phenotype / process process erlang

process process

60 THE NEURAL NETWORK

cortex

neuron neuron

sensor actuator

neuron neuron

scape

61 THE NEURAL NETWORK

cortex

sync

neuron neuron

sensor actuator

neuron neuron

scape

61 THE NEURAL NETWORK

cortex

neuron neuron

sensor actuator

neuron neuron

scape sense

61 THE NEURAL NETWORK

cortex

neuron neuron

sensor actuator

percept neuron neuron

scape

61 THE NEURAL NETWORK

cortex

forward neuron neuron

sensor actuator

neuron neuron forward

scape

61 THE NEURAL NETWORK

cortex

forward neuron neuron

sensor actuator

forward neuron neuron

scape

61 THE NEURAL NETWORK

cortex

neuron neuron forward

sensor actuator

neuron neuron forward

scape

61 THE NEURAL NETWORK

cortex

neuron neuron

sensor actuator

neuron neuron

scape action

61 THE NEURAL NETWORK

cortex

neuron neuron

sensor actuator {fitness, halt_flag}

neuron neuron

scape

61 THE NEURAL NETWORK

cortex sync

neuron neuron

sensor actuator

neuron neuron

scape

61 THE NEURAL NETWORK

cortex

neuron neuron

sensor actuator

neuron neuron

scape

61 THE EXOSELF

exoself

cortex

neuron neuron

sensor actuator

neuron neuron

scape

62 THE EXOSELF

exoself

{evaluation_completed, fitness}

cortex

neuron neuron

sensor actuator

neuron neuron

scape

62 THE EXOSELF

exoself fitness > best fitness

cortex

neuron neuron

sensor actuator

neuron neuron

scape

62 THE EXOSELF

exoself

cortex backup_weights backup_weights

neuron neuronneuron

backup_weightssensor backup_weightsactuator

neuron neuron

scape

62 THE EXOSELF

exoself

cortex perturb_weights

neuron neuronneuron

perturb_weights

sensor actuator

neuron neuron

scape

62 THE EXOSELF

exoself

cortex

neuron neuron

sensor actuator

neuron neuron

scape

62 THE EXOSELF

exoself

reactivate

cortex

neuron neuron

sensor actuator

neuron neuron

scape

62 THE POPULATION MONITOR

population monitor database

63 THE POPULATION MONITOR

population monitor database

private private private private private

63 THE POPULATION MONITOR

population monitor database

start start start start start

private private private private private

63 THE POPULATION MONITOR

population monitor database

private private private private private

63 THE POPULATION MONITOR

population monitor database

terminated terminated

terminated

terminated terminated

private private private private private

63 THE POPULATION MONITOR

population monitor database

private private private private private

63 THE POPULATION MONITOR

population monitor database

private private private

63 THE POPULATION MONITOR

population monitor database

private private private private private

63 THE POPULATION MONITOR

population monitor database

start start start start start

private private private private private

63 THE DEVIL IS IN THE DETAILS

•recurrent connections •newer generations get a higher chance for mutation •neural plasticity •public scapes and steady state

64 POLE BALANCING

agent percepts

actions cart

65 DEMO

66 DEMO

66 BENCHMARK RESULTS

Method Double-Pole/Partial Information W/O Double-Pole W/Damping Damping

RWG 415209 1232296

SANE 262700 451612

CNE 76906 87623

ESP 7374 26342

NEAT - 6929

CMA-ES* 3521* 6061*

CoSyNE* 1249* 3416*

DXNN v1 2359 2313

DXNN v2 1289 1830

67 THE HANDBOOK

68 THANK YOU

Feel free to reach out at:

e. [email protected] t. @JeroenSoeters