Complex Neural Networks – A Useful Model of Human Learning? Mark Hardman Canterbury Christ Church University [email protected] Presented at: British Educational Research Association Annual conference University of Warwick 2nd September, 2010 Page | 1 Artificial neural networks are computational systems which are increasingly common and sophisticated. Computational scientists who work with such systems however, often assume that they are simplistic versions of the neural systems within our brains and Cilliers (1998) has gone further in proposing that human learning takes place through the self-organisation of such systems according to the stimuli they receive. This continues a long tradition of using metaphors to machinery in order to describe the human brain and its processes. Unlike other metaphors however, artificial neural networks were originally conceived of in relation to how biological neurons function. As such, does this metaphor allow us to accurately understand brain function? In this paper I wish to consider the level of support this model has from contemporary neuroscience and in doing so evaluate the possible implications for educationalists. A complex neural network may be considered as a series of nodes connected by neurons such that they are highly Figure 1 – A simple neural network interconnected. The use of the term complex here denotes that each node is only responding to signals from the neurons it is directly connected to, yet the system as a whole is able to respond to its environment. Such artificial neural networks are able to respond because they adapt according to Hebb’s law. This states that the more a neural path between two nodes is used, the more efficient that path becomes. As such, constant input to a neural system results in what Edelman (1987) referred to as ‘neural Darwinism’: a neural system adapts according to input stimulus making the used pathways more effective within it. Artificial neural networks differ from the majority of modern day computers which utilise what is denoted as Von Neumann architecture. Computers, such as the one I am using to present today, step through rules and processes in sequence. Even though contemporary quad core computer processors divide up tasks so rules can be stepped through by four processors at once, the system is still stepping through a set of rules to respond to input. Neural networks do not have rules or predetermined programs, they simply respond according to the internal structure of the system. This makes neural networks much better than traditional computers at recognising and responding to patterns and as such they have applications in number plate recognition (Draghici, 1997), sales and marketing forecasts (Kuo & Xue, 1999) and image processing. Johnson & Hogg (1995) show how a system can be trained to predict where a person will walk after observing people walking through a scene a number of times1. Cilliers (1998) discusses an example by which a neural network is trained to predict past tense verbs given the present tense verb. In order to train such a system, it is presented with an input, and the system is manually adjusted by changing the path strengths until the desired output is given. This is repeated with a variety of inputs, in this case verbs. After this training phase the system is able to deal with unfamiliar input and continues to adapt to the input it is presented with. 1 I shall be presenting videos of their system in operation. Page | 2 His account goes on to discuss the requirement that the training input is sufficiently varied, otherwise the neural system will be able to recognise only very specific patterns, for example a specific tree, rather than all trees. For face recognition for example, the system must be trained using pictures of the face in varied light and from various angles in order to effectively ‘learn’ to recognise it. Of course, we need to establish whether human brains actually work in this way before we can ascertain the usefulness of complex neural network models to the field of education. Whilst artificial neural networks were originally developed in line with an understanding of how neurons in the brain work, there have, since that time, been advances in both neuroscience and artificial networks. However, I think it would be fruitful to first hypothesise with respect to what neural network models might tell us about human learning before we see to what level our assertions are supported. This will provide an appreciation of the possibilities of neural network models within education, which we can then rationalise according to the strength of evidence within the scientific literature. The first thing to note is the very different way learning is conceived of within educational theory and within neural modelling (both computational and biological). Traditional notions within education of learning as the acquisition of facts or skills are at odds with the interpretation of learning as a process within a complex system. Both Kelso (1995) and Cilliers (1998) conceive of learning as the reorganisation of the whole system. There are no structures or representations within the network which correspond to specific knowledge or abilities. Cilliers (1998) denotes this as “distributed representation”: the whole system is involved in the response to stimuli. If we accept this then we also need to recognise that it is not just the information or task that a neural system is responding to, but the stimuli of the entire environment. Furthermore as well as the external environment, the internal structure at any given time is important because this determines the starting point for any reorganisation. As educationalists then this model suggests that we need to conceptualise learning as the interaction of the complex neural network within the mind and the entire classroom environment. What is of great interest to me is the suggestion from complex neural models that the brain learns to respond to familiar contexts. Elsewhere2 I argue that we can consider a classroom itself as a complex system and if this is the case then pupils are learning to respond to classroom systems. As such they will learn behavioural norms within specific classrooms and the learning within each classroom will be determined by the social dynamics of that classroom. Beyond the reconceptualisation of learning, Cilliers’ account of the need for varied training examples suggests that repetition is important, but also that presented contexts must be varied. This begs the question of whether our curricula or the contexts we use in the classroom allow for learning in this way. It would also be interesting to see what light this neurological model sheds on many constructivist and behaviourist learning theories as it seems to suggest a need for repetition but with variety of context and consideration of the history of individuals, although this is beyond the scope of this paper. 2 “Learning to Teach in ‘Urban Complex Schools’” to be presented Saturday 4th September in Session 7. Page | 3 I hope you agree that there are a range of very interesting hypotheses that can be generated by considering learning in complex neural networks. We now need to consider current neuroscientific understanding in order to ascertain whether any of these hypotheses may be supported. Firstly let’s consider Hebb’s law which, although being a tenet of neuroscience since the 1950s, still has not accumulated a large amount of scientific evidence to support it. This is due to the experimental difficulties of seeing neural development within a living creature. Work by Antonov et al (2003) with Aplysia (a type of sea slug) does however support the development of specific pathways according to stimulus. Sylwester (1995) suggests that the mechanism for this is the development of dendrites: projections from the neural cell that allow more neurotransmitter to cross synapses, thus making the pathway more efficient. There is less certainty about what reduces path strengths if neurons are not fired together but it is a modelling assumption of neural networks that this must also happen. Further support comes from work on Lampreys (primitive jawless fish), which have long and simple neural networks which are conducive to investigation (Kelso, 1995). The networks can be observed and replicated with artificial networks in order to study their adaptation. Support for the need of repeated yet varied learning contexts comes primarily from the cognitive sciences and fits with contemporary view of memory development. This suggests that initially we have episodal memories of specific circumstances; then further experience leads to semantic memory which can be applied in new situations; then eventually procedural memory develops in which we are able to perform tasks subconsciously. Sylwester (1995) describes learning to type as initially being about the specific typewriter he used, then his ability to type on any keyboard and finally his ability to type without looking at a keyboard at all. Despite the above tentative support however, there are a number of important limitations to the complex neural model which are highlighted by reference to neuroscience. The most apparent obstacle to either supporting or denying this model of brain development is the fact that the majority of contemporary neuroscience does not deal with individual neurons, or even neural systems, but instead focuses on the specific areas or modules within the brain. This is not because neural networks do not exist but because we currently lack the technology to track neural activity to that resolution. What can be drawn from the focus on brain areas however is that if the same brain areas correspond to certain behaviours in all studied humans, then the structure of the brain must be determined by biological processes emanating from our genetics and the brain cannot be considered as a system that is fully plastic with respect to stimuli.