Challenges and Opportunities of Evolutionary Robotics
Total Page:16
File Type:pdf, Size:1020Kb
In Proceedings of the Second International Conference on Computational Intelligence, Robotics and Autonomous Systems (CIRAS), Singapore, December 2003. Challenges and Opportunities of Evolutionary Robotics D. A. Sofge, M. A. Potter, M. D. Bugajska, A. C. Schultz Navy Center for Applied Research in Artificial Intelligence Naval Research Laboratory Washington, D.C. 20375, USA {sofge, mpotter, magda, schultz}@aic.nrl.navy.mil Abstract The need for ER arises from the fact that as robotic sys- tems and the environments into which they are placed Robotic hardware designs are becoming more complex as increase in complexity, the difficulty of programming the variety and number of on-board sensors increase and their control systems to respond appropriately increases to as greater computational power is provided in ever- the point where it becomes impracticable to foresee every smaller packages on-board robots. These advances in possible state of the robot in its environment. Evolution- hardware, however, do not automatically translate into ary algorithms are used to generate control algorithms better software for controlling complex robots. Evolu- using the Darwinian principle of survival-of-the-fittest. A tionary techniques hold the potential to solve many diffi- population of controllers is maintained and evolved by cult problems in robotics which defy simple conventional evaluating individual control systems based on a measure approaches, but present many challenges as well. Nu- of how well they achieve desired characteristics such as merous disciplines including artificial life, cognitive sci- executing appropriate behaviors at appropriate times. ence and neural networks, rule-based systems, behavior- Only the fitter members of the population survive and based control, genetic algorithms and other forms of evo- pass the characteristics that made them successful on to lutionary computation have contributed to shaping the future generations. The ultimate goal is to produce the current state of evolutionary robotics. This paper pro- best possible controller given some design criteria. As vides an overview of developments in the emerging field of discussed in the following sections, this approach has evolutionary robotics, and discusses some of the opportu- proven very successful in a wide variety of challenging nities and challenges which currently face practitioners in robotics domains. the field. The remainder of this paper will provide a sampling of recent development efforts in evolutionary robotics re- 1. Introduction search with the goal of showing both the opportunities presented by the use of evolutionary techniques for solv- The field of evolutionary robotics has emerged in recent ing difficult problems in robotics, but also in showing years as the application of artificial evolution to the de- some of the challenges of applying these techniques. velopment of robotic systems. While most of the work in evolutionary robotics has focused on the development of control systems for autonomous mobile robots, some re- 2. Evolved Controllers for Autonomous searchers have used the techniques to evolve robotic Mobile Robots hardware configurations and even robot body parts along with the controllers. A key objective in evolutionary robotics is to evolve be- havior-based controllers for autonomous mobile robots Evolutionary robotics (ER) has its origins in several dis- [8,9]. Autonomous mobile robots often incorporate both ciplines including artificial life [1], cognitive science and reactive and longer-term planning components in order to neural networks [2,3,4], behavior-based control [5], ge- accommodate goal-driven behaviors. The reactive por- netic programming [6] and genetic algorithms [7]. Cur- tion of the controller may be encoded in a variety of rent practitioners of ER incorporate techniques from a forms. Common choices include stimulus-response rules, variety of disciplines to achieve the desired result, often a neural networks, and state-machines. The planning stage robotic control system for an autonomous mobile robot may be represented as a series of goal states. Behavior- (or other autonomous system) which exhibits a set of de- based controllers are thus driven by a combination of cur- sired behaviors, and for which those behaviors were ac- rent state, as determined by current sensor readings and quired “automatically” (i.e. without custom programming possibly short-term memory, and goal(s). The controller of each individual behavior). attempts to match its current state readings and goal, which together comprise the stimulus part, and then to produce the appropriate output, or response. 2.1. Evolved Rule-Based Control techniques [14]. Evolutionary algorithms have proven quite successful in training recurrent neural network- Work by Grefenstette and Schultz [10] resulted in the based controllers for autonomous mobile robots [9]. application of the SAMUEL learning system to evolve stimulus-response rules to produce a reactive control sys- Potter et al. [15] demonstrated the evolution of neural tem for autonomous mobile robots. Behaviors achieved network controllers for multiple robots engaged in shep- using this system include tracking, navigation, and obsta- herding a sheep robot. The goal of this work was to ex- cle avoidance. SAMUEL maintains a population of can- amine the effects of evolving a single homogeneous con- didate behaviors which are evaluated in a simulated ro- troller for a group of autonomous mobile robots perform- botic environment. The population is scored, mutation ing a collective herding task, versus coevolving separate and crossover operators applied, and the population is heterogeneous controllers, and to determine if the com- adjusted to remove the lesser scoring rule sets. The proc- plexity of the task favored homogeneous or heterogene- ess is run for a number of generations until a stopping ous control. The heterogeneous controllers were produced criteria is met, at which point the best evolved controller using a cooperative coevolutionary architecture [16] in is uploaded to the robot hardware for validation and test- which the controller for each robot is evolved in a sepa- ing. This technique proved successful for evolving reac- rate genetically-isolated species. It was found that het- tive controllers for autonomous mobile robots. Schultz et erogeneous controllers are indeed advantageous when the al. [12] demonstrated the evolution of rule-based control- task can be decomposed into subtasks that can be solved lers for learning complex robotic behaviors by evolving by robots specializing in substantially different skill sets. the behavior for a shepherd robot to coerce a sheep robot Otherwise homogeneous controllers have an advantage into a corral. This technique and was extended to evolv- due to their generality. ing controllers for simulated autonomous aircraft and autonomous underwater vehicles [7,11]. Quinn et al. [17] described another experiment in which neural network controllers were evolved for a team of real Challenges with evolving rule-based controllers include robots. The objective was to study the capability of the determining how to map continuous inputs and outputs to robots to learn formation forming behaviors starting from discrete state variables, establishing appropriate interme- random positions and using a minimal sensor set consist- diate and goal states, determining how many production ing of 4 IR sensors on each robot. rules are required, and performing conflict resolution. 3. Hyper-Redundant Robot Control 2.2. Evolved Neural Network Based Control One of the greatest challenges in robotics is the design of An alternative representation used by many researchers in control systems for robots with high degrees of freedom, evolutionary robotics is artificial neural networks, which particularly if many of the degrees of freedom are cou- have a number of characteristics that are desirable from pled. A key example of this is a highly segmented ser- an evolutionary robotics perspective. Neural networks are pentine or snake-like robotic arm (Fig. 1). This is an ex- relatively insensitive to noise in the environment since the ample of a hyper-redundant robot, where there are many output of each node is typically a function of the input possible kinematic solutions to achieve the same end- from a variety of sources. This characteristic also pro- effector position or trajectory. duces a smooth search space with a well-behaved map- ping between changes to the network and changes in the resultant network behavior. Neural networks also natu- rally handle continuous input and can produce either con- tinuous or discrete output as desired [9]. A key advantage of using evolutionary algorithms for producing robotic neural network based controllers is that the evolutionary techniques may be used equally well Figure 1. Hyper-redundant robotic manipulator with feed-forward or recurrent networks [13]. Recurrent networks offer many advantages for dynamic control sys- Sofge [18] used evolutionary algorithms to generate the tems because they allow recent state information to be inverse kinematics for the hyper-redundant robotic ma- combined with current state information in the decision- nipulator. The continuous work space of the robot was making process. In effect, the recurrent connections pro- discretized into a grid of regularly spaced points. A popu- vide a kind of short-term memory capability for the neural lation of genomes was created such that each genome network so that decisions