Neuromorphic computing in Ginzburg-Landau polariton lattice systems
Andrzej Opala and Michał Matuszewski Institute of Physics, Polish Academy of Sciences, Warsaw
Sanjib Ghosh and Timothy C. H. Liew Division of Physics and Applied Physics, Nanyang Technological University, Singapore
< Andrzej Opala and Michał Matuszewski Institute of Physics, Polish Academy of Sciences, Warsaw
Sanjib Ghosh and Timothy C. H. Liew Division of Physics and Applied Physics, Nanyang Technological University, Singapore
< Machine learning with neural networks
What is machine learning?
Modern computers are extremely efficient in solving many problems, but some tasks are very hard to implement.
● It is difficult to write an algorithm that would recognize an object from different viewpoints or in different sceneries, lighting conditions etc.
● It is difficult to detect a fradulent credit card transaction. We need to combine a large number of weak rules with complex dependencies rather than follow simple and reliable rules.
These tasks are often relatively easy for humans.
Machine learning with neural networks
Machine learning algorithm is not written to solve a specific task, but to learn how to classify / detect / predict on its own.
● A large number of examples is collected and used as an input for the algorithm during the teaching phase.
● The algorithm produces a program (eg. a neural network with fine tuned weights), which is able to do the job not only on the provided samples, but also on data that it has never seen before. It usually contains a great number of parameters.
Machine learning with neural networks
Three main types of machine learning:
1.Supervised learning – direct feedback, input data is provided with clearly defined labels.
2.Unsupervised learning – no labels or feedback, machine tries to understand / classify data on its own.
3.Reinforcement learning – machine learns how to behave in an environment to maximize rewards, which may be delayed in time.
Machine learning with neural networks
Y. LeCun, Y. Bengio, and G. Hinton, Nature 521, 436 (2015). Machine learning with neural networks
Modified National Institute of Standards andTechnology (MNIST) dataset: A fruit fly of machine learning
Best networks achieve <0.3% error rate http://yann.lecun.com/exdb/mnist/ Machine learning with neural networks
Nonlinear transformation
Dimensional expansion
Y. LeCun, Y. Bengio, and G. Hinton, Nature 521, 436 (2015); David Verstraeten, PhD thesis Machine learning with neural networks
Nonlinear transformation Dimensional expansion
Machine learning with neural networks
Backpropagation algorithm – an efficient method to teach deep (multilayer) neural networks
Recurrent neural networks
Recurrent networks can handle time sequences, while feed-forward networks are static
https://towardsdatascience.com/recurrent-neural-networks-and-lstm-4b601dd822a5 Recurrent neural networks
DeepLearning.TV
Recurrent neural networks
DeepLearning.TV
Recurrent neural networks
Kelvin Xu et al., PMLR 37:2048-2057, 2015. http://arxiv.org/abs/1502.03044 (2015) Recurrent neural networks
Transformation of a recurrent network into a feedforward network
After ufolding, recurrent network can be taught using the backpropagation algorithm.
Neuromorphic computing
Neuromorphic computing is the attempt to adjust the architecture of the physical system to the architecture of the neural network
Implementation Model von Neumann architecture P. A. Merolla et al., Science 345, 668 (2014) Neuromorphic computing
● Neuromorphic engineering will be probably necessary to effectively transfer ML from data centers to end user devices (eg. smartphones)
● It can mimic neurobiological systems and contributes to their understanding
IBM
Heidelberg Manchester Stanford ● The Human Brain Project (EU) ● BRAIN initiative (US) Reservoir computing
Reservoir computing
Tunable ● A simple way to quickly teach a network is weights to tune only the output weights, while keeping the other connections random and fixed
● Teaching is reduced to a simple linear regression.
● Random expansion of the input in the early layers helps significantly
Fixed weights Reservoir computing
● The idea works well in recurrent networks
● The random fixed weights in the “reservoir” must be chosen carefully so that the activation does not die out or get amplified exponentially (echo state property)
Reservoir computing
● Reservoir networks work very well with time-series, especially one-dimensional
● They can be trained very quickly, but require more nodes than a fully tunable recurrent neural network for the same task
Reservoir computing
As the weights in the reservoir are not tuned, this scheme is well suited for hardware implementations in many systems
Brunner et al, Nat. Commun. 4, 1364 (2013); C. Du et al., Nat. Commun. 8, 2204 (2017) L. Appeltant et al., Nat. Commun. 2, 468 (2011); K. Vandoorne et al., Nat. Commun. 5, 3541 (2014) K. Nakajima et al., Sci. Rep. 5, 10487 (2015) Reservoir computing with polariton lattices
Properties of exciton-polariton condensates
X X γ
Excellent transport properties and extremely low effective mass thanks to the photonic component
Very strong interparticle interactions thanks to the exciton component – world record of (ultrafast) optical nonlinearity
Short lifetime (1-200 ps) is an issue, but also interesting for fundamental research of nonequilibrium systems
Polariton lattices
Lattices of microcavity polariton micropillars can be fabricated with extreme precision
C2N Polariton Quantum Fluids group C. E. Whittaker et al, S. Klembt et al., Nature 562, 552 (2018) PRL 120, 097401 (2018)
Polariton lattices
M Milicevic et al., 2D Mater. 2 (2015) 034012 Reservoir computing with polaritons
?
Reservoir computing with polaritons
Discrete Complex Ginzburg-Landau equation
Resonant injection NN Coupling
Balance between nonresonant Nonlin. losses Interactions pumping and linear losses
Reservoir computing with polaritons
Random matrix
● System is excited with resonant lasers (input) and nonresonant background pump P
● At the end of evolution, density in each node is recorded and used for prediction Reservoir computing with polaritons
Random matrix
● Fixed couplings within the reservoir are chosen as random, realistic values
● Supervised teaching consists of tuning the output weights to minimize the cost (error) function Reservoir computing with polaritons
First test: Mackey-Glass equation – nonlinear, time-dependent process
The system is taught to predict future state of a nonlinear process w/memory
Reservoir computing with polaritons
First test: Mackey-Glass equation – nonlinear, time-dependent process
The system is taught to predict future state of a nonlinear process w/memory
Target vs. prediction of a polariton reservoir
Reservoir computing with polaritons
A more ambitious task: MNIST dataset
Reservoir computing with polaritons
A more ambitious task: MNIST dataset
Reservoir computing with polaritons
Polariton reservoir works optimally at the threshold of condensation (polariton lasing)
Reservoir computing with polaritons
Performance versus size of the polariton lattice
Optimal error rate is comparable to a fully tunable network with a single layer of 300 hidden nodes Speech recognition, TI 46 set
Estimate of efficiency
Conclusions
● Reservoir computing is a neural network architecture which can be implemented in various physical systems
● We demonstrated that systems described by the complex Ginzburg–Landau equation can be used for machine learning applications
● Exciton-polariton lattices may benefit from extremely fast timescales of dynamics and the precision of lattice fabrication
arXiv:1808.05135 To be published in Phys. Rev. Applied
<