The Many Facets of Natural Computing Lila Kari0 Grzegorz Rozenberg Department of Computer Science Leiden Inst. of Advanced Computer Science University of Western Ontario Leiden University, Niels Bohrweg 1 London, ON, N6A 5B7, Canada 2333 CA Leiden, The Netherlands [email protected] Department of Computer Science University of Colorado at Boulder Boulder, CO 80309, USA [email protected] “Biology and computer science - life and computation - are various natural phenomena. Thus, rather than being over- related. I am confident that at their interface great discoveries whelmed by particulars, it is our hope that the reader will await those who seek them.” (L.Adleman, [3]) see this article as simply a window onto the profound rela- tionship that exists between nature and computation. There is information processing in nature, and the natu- 1. FOREWORD ral sciences are already adapting by incorporating tools and Natural computing is the field of research that investi- concepts from computer science at a rapid pace. Conversely, gates models and computational techniques inspired by na- a closer look at nature from the point of view of information ture and, dually, attempts to understand the world around processing can and will change what we mean by computa- us in terms of information processing. It is a highly in- tion. Our invitation to you, fellow computer scientists, is to terdisciplinary field that connects the natural sciences with take part in the uncovering of this wondrous connection.1 computing science, both at the level of information tech- nology and at the level of fundamental research, [98]. As a matter of fact, natural computing areas and topics come in many flavours, including pure theoretical research, algo- 2. NATURE AS INSPIRATION rithms and software applications, as well as biology, chem- Among the oldest examples of nature-inspired models of istry and physics experimental laboratory research. computation are the cellular automata conceived by Ulam In this review we describe computing paradigms abstracted and von Neumann in the 1940s. John von Neumann, who from natural phenomena as diverse as self-reproduction, the was trained in both mathematics and chemistry, investigated functioning of the brain, Darwinian evolution, group be- cellular automata as a framework for the understanding of haviour, the immune system, the characteristics of life, cell the behaviour of complex systems. In particular, he believed membranes, and morphogenesis. These paradigms can be that self-reproduction was a feature essential to both biolog- implemented either on traditional electronic hardware or ical organisms and computers, [120]. on alternative physical media such as biomolecular (DNA, A cellular automaton is a dynamical system consisting RNA) computing, or trapped-ion quantum computing de- of a regular grid of cells, in which space and time are dis- vices. Dually, we describe several natural processes that can crete. Each of the cells can be in one of a finite number be viewed as information processing, such as gene regula- of states. Each cell changes its state according to a list of tory networks, protein-protein interaction networks, biolog- given transition rules that determine its future state, based ical transport networks, and gene assembly in unicellular on its current state and the current states of some of its organisms. In the same vein, we list efforts to understand neighbors. The entire grid of cells updates its configura- biological systems by engineering semi-synthetic organisms, 1 and to understand the universe from the point of view of A few words are in order about the organization of this ar- ticle. The classifications and labels we use for various fields information processing. of research are purely for the purpose of organizing the dis- This review was written with the expectation that the course. In reality, far from being clear-cut, many of the reader is a computer scientist with limited knowledge of nat- fields of research mentioned here overlap, or fit under more ural sciences, and it avoids dwelling on the minute details of than one category. The very general audience for whom this paper is intended, our respective fields of expertise, and es- 0Corresponding author. pecially the limited space available for this review affected both the depth and breadth of our exposition. In partic- ular, we did not discuss some fields of research that have large overlaps with natural computing, such as bioinformat- ics, computational molecular biology, and their roles in, for Permission to make digital or hard copies of all or part of this work for example, genomics and proteomics. In addition, our expla- personal or classroom use is granted without fee provided that copies are nations of various aspects, themes, and paradigms had to not made or distributed for profit or commercial advantage and that copies be necessarily oversimplified. As well, the space we devoted bear this notice and the full citation on the first page. To copy otherwise, to to various fields and topics was influenced by several factors republish, to post on servers or to redistribute to lists, requires prior specific and, as such, has no relation to the respective importance of permission and/or a fee. the field or the relative size of the body of research in that Copyright 200X ACM X-XXXXX-XX-X/XX/XX ...$5.00. field. tion synchronously according to the a priori given transition features of a single biological organism, evolutionary compu- rules. tation draws its inspiration from the dynamics of an entire Cellular automata have been applied to the study of phe- species of organisms. An artificial evolutionary system is nomena as diverse as communication, computation, con- a computational system based on the notion of simulated struction, growth, reproduction, competition, and evolution. evolution. It features a constant- or variable-size popula- One of the best known examples of cellular automata, the tion of individuals, a fitness criterion according to which “game of life”, invented by Conway, [45], was shown to be the individuals of the population are being evaluated, and computationally universal. Cellular automata have been ex- genetically-inspired operators that produce the next gener- tensively studied as an alternative explanation to the phe- ation from the current one. In an evolutionary system, the nomenon of emergence of complexity in the natural world, initial population of individuals is generated at random or [125], and used, among others, for modeling in physics, [16], heuristically. At each evolutionary step, the individuals are [118], and biology [36]. evaluated according to a given fitness function. To form In parallel to early comparisons, [119], between comput- the next generation, firstly offspring are generated from se- ing machines and the human nervous system, McCulloch lected individuals by using operators such as mutation of a and Pitts, [73], proposed the first model of artificial neu- parent, or recombination of pairs or larger subsets of par- rons. This research eventually gave rise to the field of neu- ents. The choice of parents for recombination can be guided ral computation, and it also had a profound influence on by a fitness-based selection operator, thus reflecting the bi- the foundations of automata theory, [59]. The goal of neu- ological principle of mate selection. Secondly, individuals ral computation was two-fold. On one hand, it was hoped of the next generation are selected from the set of newly that it would help unravel the structure of computation in created offspring, sometimes also including the old parents, nervous systems of living organisms (How does the brain according to their fitness - a process reflecting the biological work?). On the other hand, it was predicted that, by using concept of environmental selection. the principles of how the human brain processes information, Evolutionary systems have first been viewed as optimiza- neural computation would yield significant computational tion processes in the 1930s, [126]. The basic idea of viewing advances (How can we build an intelligent computer?). The evolution as a computational process gained momentum in first goal has been pursued mainly within the neurosciences the 1960s, and evolved along three main branches, [29]. Evo- under the name of brain theory or computational neuro- lution strategies, [88], [103], [89], use evolutionary processes science, while the quest for the second goal has become to solve parameter optimization problems, and are today mainly a computer science discipline known as artificial neu- used for real-valued as well as discrete and mixed types of pa- ral networks or simply neural networks, [6]. rameters. Evolutionary programming, [40], originally aimed An artificial neural network consists of interconnected ar- at achieving the goals of artificial intelligence via evolution- tificial neurons, [94]. Modeled after the natural neurons, ary techniques, namely by evolving populations of intelli- each artificial neuron A has n real-valued inputs, x1, x2, . , xn, gent agents modeled, for example, as finite-state machines. and it computes its own primitive function fA as follows. Today, these algorithms are also often used for real-valued Usually, the inputs have associated weights, w1, w2, . , wn. parameter optimization problems. Genetic algorithms, [53], Upon receiving the n inputs, the artificial neuron A produces [54], originally featured a population of individuals encoded the output fA(w1x1 + w2x2 + ··· + wnxn). An artificial neu- as fixed-length bit strings, wherein mutations consisted of ral network is a network of such neurons, and thus a network bit-flips according to a typically small, uniform mutation of their respective primitive functions. Some neurons are se- rate, the recombination of two parents consisted of a cut- lected to be the output neurons, and the network function and-paste of a prefix of one parent with a suffix of the other, is a vectorial function that, for n input values, associates and the fitness function was problem-dependent. If the ini- the outputs of the m output neurons.