Memristor-based Neuromorphic Computing Design and simulation of a Neural Network based on Memristor technology María García Fernández Master of Science Thesis Delft Center for Systems and Control Memristor-based Neuromorphic Computing Design and simulation of a Neural Network based on Memristor technology Master of Science Thesis For the degree of Master of Science in Embedded Systems at Delft University of Technology María García Fernández August 31, 2020 Faculty of Electrical Engineering, Mathematics and Computer Science (EEMCS) · Delft University of Technology Copyright c Delft Center for Systems and Control (DCSC) All rights reserved. Table of Contents Acknowledgements vii 1 Background 4 1-1 Machine Learning . 4 1-1-1 Neural Networks . 5 1-2 Research question . 11 1-3 History and Evolution of Neuromorphic Computing . 12 1-3-1 Hardware Implementations of Neural Networks . 14 1-3-2 Brief overview . 15 2 Introduction to memristors 17 2-1 Memristors in Neural Networks . 20 3 Memristor Characterization 22 3-1 Experiments and Goal . 22 3-1-1 Experiment Description . 22 3-1-2 Goal . 23 3-2 Brief Explanation of the Set-up . 24 3-2-1 Memristor’s resistance calculation . 26 3-2-2 The memristor endurance problem . 29 3-3 First experiment . 29 3-3-1 Second experiment: From High Resistance State (HRS) to Low Resistance State (LRS) with trains of identical pulses . 31 3-3-2 The Mean Metastable Switch Memristor Model . 34 3-3-3 Model exploration via optimization runs . 38 3-3-4 Analysis of the data and the results . 41 3-4 Simulation Work . 44 3-5 Conclusion . 47 Master of Science Thesis María García Fernández Table of Contents ii 4 Memristors in Neural Networks: Introduction 48 4-1 Unsupervised Learning . 48 4-1-1 Unsupervised implementation of the Simplified Spiking-Time Dependant Plasticity Learning Rule . 49 4-1-2 Accuracy challenge . 52 4-2 Supervised Learning . 52 4-2-1 Supervised implementation of the Simplified Spiking-Time Dependant Plas- ticity Learning Rule . 52 4-2-2 Online Back-propagation Rule with Stochastic Gradient Descent . 53 4-2-3 Manhattan Rule-based Back-Propagation . 55 4-3 Summary and Design Choices . 55 5 Memristor Circuit Design 57 5-1 Main consideration when designing a memristor-based chip for Neural Networks . 57 5-2 Development . 58 6 Neural Network Simulation 63 6-1 Spiking Timing-Dependant Plasticity (STDP): Hebbian and anti-Hebbian learning 64 6-1-1 Supervised STDP . 64 6-1-2 Unsupervised STDP . 65 6-2 Classic Artificial Neural Network (ANN) Implementations . 66 6-2-1 Supervised ANN without hidden layers (1 input layer, 1 output layer) . 67 6-2-2 Supervised with one hidden layer . 69 6-3 Conclusions . 72 7 Conclusions 73 7-1 Summary . 73 7-2 Main Contributions . 73 7-3 Future Work . 75 7-3-1 The dR/dE ratio . 76 7-3-2 Time series of the experiments . 79 7-3-3 Parameter Tuning - Matlab Code . 84 7-3-4 Memristor’s stochasticity simulation . 86 7-3-5 Matrix Circuit Design . 87 Bibliography 88 Glossary 94 List of Acronyms . 94 List of Symbols . 95 Master of Science Thesis María García Fernández List of Figures 1 A typical task that the human brain performs constantly. 2 1-1 Representation of a neuron [1] . 6 1-2 Some examples of activation functions [2] . 7 1-3 Representation of a neural network [3] . 7 1-4 Spiking neuron network implementations [4] . 9 1-5 Simplified version of STDP proposed in [5] . 11 1-6 Simplified overview of the von Neumann Architecture . 13 1-7 Percentage of papers in each field within neuromorphic computing [6] . 13 2-1 The four fundamental two-terminal circuit elements[7] . 17 2-2 Ideal behavior of a memristor’s current and conductance with respect to voltage. 18 2-3 A series resistor can be used to measure the memristor’s resistance. 19 2-4 I(V) curve. 19 2-5 Example of a crossbar architecture. 21 3-1 The experiment was added to the GUI provided by Knowm with the setup, which was open-source. 23 3-2 The experiments performed aim to help the design of a controller that works with the Self-Directed Channel (SDC) memristors given. 24 3-3 Initial Experiment Set-Up. Images provided with permission by Knowm Inc . 24 3-4 Circuit within the Knowm set-up. 25 3-5 How the voltage VREAD is selected. 27 3-6 Circuit simulated in LTSpice, as defined by Knowm. 27 3-7 Voltage and current across the memristor for a 10 µs pulse width. 28 3-8 Algorithm proposed in [8] . 30 Master of Science Thesis María García Fernández List of Figures iv 3-9 Circuit simulated in LTSpice, as defined by Knowm. 31 3-10 Histogram of the voltage applied directly on the memristor (measured via a scope) 32 3-11 Experiment 2 on one of the memristors. 33 3-12 ............................................ 34 3-13 Graphical explanation of the metastable switch memristor model . 35 3-14 Mean Metastable Switch Memristor Model as explained in [9] and [10] . 36 3-15 The predicted change is negligible for most points. 40 3-16 dX vs VMEM for all pulses in which 0.5 V were applied to the circuit. 42 3-17 Real Data vs Optimization Runs Model Data . 43 3-18 Model built in Simulink . 44 3-19 Simulated X vs Voltage(V) and dX vs Voltage(V). 45 3-20 VMEM over time. Resistance setpoints (blue) and actual read resistance (yellow) over time. 46 4-1 STDP as explained in vs the implemented version in [5]. 49 4-2 Visual explanation of the Leaky Integrate-And-Fire (LIF) neuron. 50 4-3 Behavior of the system in [5]. An output spike will only increase a synaptic weight its input is also active. Otherwise it will decrease. The input spikes are longer than the output spikes. Leakage is not explained by this plot, it is added to the simulation via external equations. 51 4-4 Scintillating Grid Illusion [11] . 52 4-5 Architecture proposed in [12] . 53 5-1 Inputs of the circuit designed. 58 5-2 Multiple inputs stacked together and connected an output neuron (the amplifying stage) . 59 5-3 Transfer function of the amplifying stage clipped at 0.5 and -0.5 V, compared to scaled tanh and sigmoid with offset. 61 5-4 ............................................ 61 5-5 VOUT during reading operation with 8 active inputs. 62 5-6 Voltage drop on several memristors while writing one of them. 62 6-1 Some examples of handwritten characters from the MNIST database [13] . 64 6-2 Supervised Implementation of STDP. 64 6-3 Accuracy achieved by using STDP on 600 samples. 65 6-4 Unsupervised Implementation of STDP. 66 6-5 Supervised Implementation of an ANN without hidden layers. 67 6-6 Success of the experiment . 68 6-7 Success of the experiment. 69 6-8 Effects of letting the network learn for too long (no early stopping) . 69 6-9 Supervised implementation of an ANN with one hidden layer with 300 neurons. 70 6-10 Success of the experiment . 70 Master of Science Thesis María García Fernández List of Figures v 6-11 Success of the experiment . 71 6-12 Weight Distribution. 72 7-1 ............................................ 76 7-2 ............................................ 76 7-3 ............................................ 77 7-4 ............................................ 77 7-5 ............................................ 77 7-6 ............................................ 78 7-7 ............................................ 78 7-8 ............................................ 78 7-9 ............................................ 79 7-10 ............................................ 79 7-11 ............................................ 79 7-12 ............................................ 80 7-13 ............................................ 80 7-14 ............................................ 81 7-15 ............................................ 81 7-16 ............................................ 82 7-17 ............................................ 82 7-18 ............................................ 82 7-19 ............................................ 82 7-20 ............................................ 83 7-21 ............................................ 83 7-22 ............................................ 83 7-23 ............................................ 83 Master of Science Thesis María García Fernández List of Tables 1-1 Comparison of the usage of some neuromorphic devices . 16 3-1 Numerical values thrown by both simulations . 29 3-2 Memristor estimation error for Experiment 1. 38 3-3 Memristor estimation error for Experiment 2. 38 3-4 Lower and Upper bounds given to the parameters to be tuned . 39 3-5 Comparison between three different optimization algorithms. They seem to give similar results. 40 3-6 Comparison between three different optimization algorithms. They give similar results. 41 Master.