220 シ ス テ ム 制 御 情 報 学 会 論 文 誌,Vol. 19, No. 6, pp. 220-232, 2006

論 文

Exploring the Analogy in the Emergent Proper- ties of Locomotion of Adapted to Challenging Terrain and Partial Damage *

Ivan Tanev †§ ・Thomas Ray ‡ ・Katsunori Shimohara §

Exploring the Analogy in the Emergent Properties of Locomotion Gaits of Snakebot Adapted to Challenging Terrain and Partial Damage *

Ivan TANEv †§, Thomas RAY ‡ and Katsunori SHIMoHARA §

The objective of this work is to investigate the emergent properties of the gaits of the simulated -like robot, Snakebot. The gaits are automatically designed through Genetic Programming (GP) to be robust, general, adaptive, and the fastest possible sidewinding, locomotion. Considering the notion of emergent intelligence as the ability of Snakebot to achieve its goals (of moving fast) without the need to be explicitly taught how to do so, we present empirical results demonstrating the emergence of sidewinding locomotion from relatively simple motion patterns of morphologi- cal segments of Snakebot. We discuss the emergent properties of the evolved robust high velocity sidewinding locomotion gaits of Snakebot when situated in challenging environments. Then we elab- orate on the ability of Snakebot to adapt to partial damage by gradually improving its velocity characteristics, and the emergent properties of obtained adaptive gaits . Verifying the practical im- plications of the analogy between the emergent properties of the robust and the adaptive sidewinding gaits, this work could be viewed as a step towards building real , which are able to perform robustly in challenging environments.

1. Introduction nisms [4,21]. Robots with these properties open up Wheelless, limbless snake-like robots (Snakebots) several critical applications in exploration, reconnais- feature potential robustness characteristics beyond the sance, medicine and inspection. However, compared capabilities of most wheeled and legged vehicles - abil- to the wheeled and legged vehicles, Snakebots feature ity to traverse terrain that would pose problems for (i) smaller payload, (ii) more difficult thermal control, traditional wheeled or legged robots, and insignificant (iii) more difficult control of locomotiongaits and (iv) inferior speed characteristics. performance degradation when partial damage is in- flicted. Moreover, due to their modular design, Snake- Consideringthe first two drawbacks as beyond the bots may be cheaper to build, maintain and repair. scope of our work, and focusing on the issues of con- Some useful features of Snakebots include smaller size trol and speed, we intend to address the following of the cross-sectional areas, stability, ability to op- challenge: how to develop control sequencesof Snake- erate in difficult terrain, good traction, high redun- bot's actuators, which allow for the fastest possible dancy, and complete sealing of the internal mecha- speed of locomotion achievable with Snakebot mor- phology. * 原 稿 受 付2005年3月14日 For many tasks and robot morphologies,it might

† 同 志 社 大 学Department of Information Systems De - be seen as a natural approach to handcraft the lo- comotion control code by applying various theoreti- sign, Doshisha University; 1-3 Miyakodani, Tatara, Ky- cal approaches [3,5,22]. However,handcrafting might otanabe, Kyoto 610-0321, JAPAN not be feasible for developing the control code of a ‡ Department of Zoology, 730 Van Vleet Oval , Room real Snakebot due to its morphological complexity 314, University of Oklahoma Norman, Oklahoma 73019, and the anticipated need of prompt adaptation under USA § ATRネ ッ ト ワ ー ク 情 報 学 研 究 所ATR Network In degraded mechanical abilities and/or unanticipated - environmental conditions. Moreover, while fast lo- formatics Laboratories; 2-2-2 Hikaridai, "Keihanna Sci- comotion might emerge from relatively simply ence City", Kyoto 619-0288, JAPAN defined motion patterns of morphologicalsegments of Key Words : emergence, Snakebot, evolution, adaptation .

10 TANEV•ERAY•ESHImoHARA:Exploring the Analogy in the Emergent Properties of Locomotion Gaits of Snakebot•@ 221

Snakebot, the natural implication of the phenomenon bility of success for various runtime intervals is of emergence in complex systems is that neither the applied as a characteristic of computational ef- degree of optimality of the developed code nor the ficiency of GP. The non-determinism of GP is way to incrementally improve the code is evident to viewed as a natural consequence of the stochas- the human designer [13]. Thus, an automated, holis- tic nature of both the way of creating the initial tic approach for evaluation and incremental optimiza- population of potential solutions to the problem tion of the intermediate solution(s) are needed (e.g. and the genetic operations applied to evolve this based on various models of learning or evolution in population (crossover,mutation, and selection), Nature) [6,12,19].The proposed approach of employ- and ing Genetic Programming (GP) implies that the code, (3) Often the solution, automatically obtained via which governs the locomotionof Snakebot is automat- GP is quite complex and difficult to be com- ically designed by a computer system via simulated prehended by a human designer. Consequently, evolution through selection and survival of the fittest even simple man-made modification to such a in a way similar to the evolution of species in the na- solution is not a straightforward task. ture [8,9]. The use of an automated process to design As an instance of evolutionary algorithms, Genetic the control code opens the possibility of creating a Algorithms (GA) differ from GP mainly in the geno- solution that would be better than one designed by a typic representation (i.e. chromosome) of potential human [10]. solutions. Instead of representing the solution as a In principle, the task of designingthe code of Snake- computer program (usually-a parsing tree) featur- bot could be formalized and the formal mathematical ing arbitrary structure and complexity as in GP, GA models incorporated into direct programmable con- employs a fixed-length linear chromosome. This dif- trol strategies. However,the eventual models would ference implies a favorable computational advantage feature enormous complexity and such models are not of GA over GP for simple problems, because the linear recognizedto have a known, analytically obtained ex- chromosomes are computationally efficiently manip- act optimal solution. The complexity of the model ulated by genetic operations and interpreted by fit- stems from the considerableamount of degrees of free- ness evaluation routines. For complex tasks however dom of the Snakebot, which cannot be treated inde- (such as evolution of locomotion gaits of Snakebot) pendently of each other. The dynamic patterns of the the runtime overhead associated with the manipula- position, orientation, velocity vectors, and moreover, tion of genotype is negligible compared to the much the points and times of contact with the surface (and more significant overhead of the fitness evaluation of consequently-the vectors of resulting traction forces, the evolved (simulated or real) artifact in the (simu- which propel the Snakebot) of each of the morpholog- lated or real) environment. Moreover, an efficient GA ical segments of Snakebot has to be considered within (in terms of computational effort, or number of fitness the context of other segments. Furthermore, often the evaluations) often requires incorporation of extremely dynamic patterns of these parameters cannot be de- computationally heavy probabilistic learning models terministically inferred from the desired velocity char- aimed at maintaining the complex inter-relations be- acteristics of the locomotionof Snakebot. Instead, the tween the variables in the chromosome. In addition, locomotion of the Snakebot is viewed as an emergent the fixed-length chromosome usually implies that the property at higher level of consideration of a com- latter comprises various, carefully encoded problem- plex hierarchical system, comprising many relatively domain-dependent parameters of the solution with an simply defined entities (morphological segments). In a priori known structure and complexity. This might such systems the higher-levelproperties of the system be a concern when no such knowledge is available in and the lower-levelproperties of comprising entities advance, but rather needs to be automatically and au- cannot be induced from each other. GP (and evo- tonomously discovered by the evolving artifact. The lutionary algorithms in general) is considered as an latter is especially true when the artifact has to per- efficient way to tackle such ill-posed problems due to form in unanticipated, uncertain environmental con- the ability of GP to find a near-optimal solution in ditions or under its own (possibly degraded) mechan- a reasonable runtime. This ability often overcompen- ical abilities. sates the drawbacks of GP, which can be summarized Evolving a Snakebot's locomotion (and in general, as follows: behavior of any robot) by applying GP could be per- (1) Relatively long runtime stemming from the sig- formed as a first step in the sequence of simulated nificant computational effort (many potential off-line evolution (phylogeneticlearning) on the soft- solutions need to be evaluated before the suf- ware model, followedby on-line adaptation (ontoge- ficiently good solution is discovered) and poor netic learning) of evolved code on a physical robot computational performance (often the fitness eval- situated in a real environment [14]. Off-line soft- uation is a time-consuming routine) of GP, ware simulation facilitates the process of Snakebot's (2) Non-determinism - the exact runtime needed to controller design because the verification of behavior obtain the solution cannot be estimated in ad- on physical Snakebot is extremely time consuming, vance. Instead, a statistically obtained proba- costly and often dangerous for the Snakebot and sur-

11 222 システム制御情報学会論文誌 第19巻 第6号(2006) rounding environment. Moreover, in some cases it vertical planes respectively (Fig. 1). The task of de- is appropriate to initially model not only the loco- signing the Snakebot locomotion can be rephrased as motion, but also to co-evolve the most appropriate developing temporal patterns of desired turning an- morphology of the artifact (i.e. number of pheno- gles of horizontal and vertical actuators of each seg- typic segments; types and parameters of joints which ment, that result in the fastest overall locomotion. link segments; actuators' power; type, amount and location of sensors; etc.) [16,17] and only then (if appropriate) to physically implement it as hardware. The software model used to simulate Snakebot should fulfill the basic requirements of being quickly devel- oped, adequate, and fast [7]. The typically slow development time of GP stems from the highly specific semantics of the main attributes of GP (e.g. representation, genetic operations, fitness evaluation) Fig. 1 Morphological segments of Snakebot linked via and can be significantly reduced through incorporat- universal joint. Horizontal and vertical actuators ing off-the-shelfsoftware components and open stan- attached to the joint perform rotation of the seg- dards in software engineering. To address this issue, ment #i+1 in vertical and horizontal planes re- we developed a GP framework based on open XML spectively standard. And to ensure adequacy and runtime ef- The proposed representation of Snakebot as a ho- ficiency of the Snakebot simulation we applied the mogeneous system comprising identical morphologi- Open Dynamic Engine (ODE) freeware software li- cal segments is intended to significantly reduce the brary for simulation of rigid body dynamics. size of the search space of the GP. Moreover, because The objectives of our work are (i) to explore the the size of the search space does not necessarily in- feasibility of applying GP for automatic design of the crease with the increase of the number of morpho- fastest possible, robust, general and adaptive locomo- logical segments of Snakebot, the proposed approach tion gaits of realistically simulated Snakebots and (ii) allows achievement of favorable scalability character- to investigate the emergent properties of these loco- istics of the GP. motion gaits. Inspired by the fast sidewinding loco- An alternative approach of employing phase au- motion of the rattlesnake , this work tomata has been recently proposed for representing is motivated by our desires (i) to model the sidewind- and programming the functionality of segments in ing locomotion of natural , (ii) to explore the modular chain-type artifacts [22]. The approach is phenomenon of emergence of locomotion of complex based on an event-driven input/output state automa- bodies from simply defined morphological segments ton with an initial phase delay, and promises great comprising these bodies. versatility, robustness and scalability. However, the The remainder of this document is organized as eventual automatic programming of these locomotion follows. Section 2 emphasizes the main features of gaits (rather than handcrafting them) is still an open the GP proposed for evolution of locomotion gaits of issue in this approach. the simulated Snakebot. Section 3 presents empirical In the proposed representation of Snakebot, no results of emergent properties of evolving locomotion anisotropic (directional) friction between the morpho- gaits of Snakebot and discusses robustness, generality logical segments and the surface is considered. De- and adaptation of sidewinding in various fitness land- spite the anticipated ease of simulation and design of scapes caused by various, unanticipated environmen- eventual morphological segments featuring anisotropic tal conditions and partial damage of Snakebot. The friction with the surface (using simple attached wheels same section explores the relevant practical implica- [5] or belly scales), such an approach would have the tions of the observed analogy between the emergent following drawbacks: properties of the robust and adaptive sidewinding lo- (1) Wheels, attached to the morphologicalsegments comotion gaits of Snakebot. Finally, Section 4 draws of Snakebot are mainly effectivein two-dimensional conclusions. locomotion gaits. However, neither the fastest 2. The Approach gaits in unconstrained environments nor the adap- tive gaits in challenging environments (narrow 2.1 Representation of Snakebot tunnels, obstacles etc.) are necessarilytwo-dimen- Snakebot is simulated as a set of identical spheri- sional. In three-dimensional locomotion gaits cal morphological segments (vertebrae, linked together the orientation (the pitch, roll and yaw angles) via universal joints. All joints feature identical finite of morphologicalsegments at the instant of con- angle limits and each joint has two attached actuators tact with the surface are arbitrary, which ren- (muscles. In the initial standstill position, the rota- ders the design of effectivewheels for such loco- tion axes of the actuators are oriented vertically (ver- motion gaits a non-trivial engineering task. tical actuator) and horizontally (horizontal actuator) (2) Wheels may compromise the potential robust- and perform rotation of the joint in the horizontal and ness characteristics of Snakebot because they

12 TANEV•ERAY•ESHimoHARA:Exploring the Analogy in the Emergent Properties of Locomotion Gaits of Snakebot 223

can be trapped easily in the challenging envi- typically comprises coupled neural controllers, which ronments (rugged terrain, obstacles, etc.). generate (without the need of external feedback) the (3) Wheels potentially reduce the application areas motion pattern of actuators in the respective mor- of the Snakebot because their engineering de- phological segments of the artifact. The approach of sign implies lack of complete sealing of all mech- employing CPG for developing the locomotion gaits of anisms of Snakebot. the Snakebot would be based on an iterative process (4) Belly scales (if implemented) would not promote (e.g. employing the machine learning and/or evolu- any anisotropic friction when Snakebot operates tionary computations paradigms) of tuning the main on smooth, flat, clean and/or too loose surfaces parameters of CPG including, for example, the com- which compromises the generality of derived lo- mon single frequency across the coupled oscillators, comotion gaits and their robustness to various the fixed phase-relationship between the oscillators, environmental conditions. and the amplitude of each of oscillations. The pro- Belly scales are efficiently utilized as a source of posed approach of applying GP for evolution of lo- anisotropic friction in some locomotion gaits of natu- comotion gaits of Snakebot shares some of the fea- ral snakes. However, these gaits usually require an in- tures of CPG-based approaches such as the open-loop volvement of large amount of complex muscles located control scheme and the incorporation of coupled os- immediately under the skin of the snake. These mus- cillators. Conversely to the CPG-based approaches cles lift the scales off the ground, angle them forward, however, the proposed method incorporates too little and then push them back against the surface. In the domain-specific knowledge about the task. The com- Snakebot, implementing actuators, which mimic such parative flexibility of GP, resulting from not consider- muscles in the natural snakes, would be expensive and ing all the domain-specific constrains, can potentially thus infeasible from an engineering point of view. yield an optimal solution with the following, typically uncommon for CPG properties: 2.2 Algorithmic Paradigm (1) The optimal oscillations of segments might be 2.2.1 GP an arbitrary superposition of several oscillations GP [8,9]is a domain-independent problem-solving featuring different frequencies. Moreover, the approach in which a population of computer programs proposed method of using GP does not neces- (individuals' genotypes) is evolved to solve problems. sarily imply that the frequency across the oscil- The simulated evolution in GP is based on the Dar- lators is common, winian principle of reproduction and survival of the (2) The relationship between the oscillators in the fittest. The fitness of each individual is based on the morphological segments of Snakebot is not nec- quality with which the phenotype of the simulated in- essarily a simple phase relationship. Arbitrary dividual is performing in a given environment. The relationships involving amplitude, phase, and major attributes of GP-function set, terminal set, frequency are allowed to be developed by the fitness evaluation, genetic representation, and genetic simillntpd evolution via GP. and operations are elaborated in the remainder of this Sec- (3) The evolved optimal phase relationship between tion. the oscillators in the morphological segments 2.2.2 Function Set and Terminal Set might vary along the body of Snakebot, rather In applying GP to the evolution of Snakebot, the than being fixed. genotype is associated with two algebraic expressions, The above-mentioned features are achieved via the which represent the temporal patterns of desired turn- incorporation of the terminal symbol segment_ID (an ing angles of both the horizontal and vertical actua- unique index of morphological segments of Snakebot), tors of each morphological segment. Since locomo- which allows GP to discover how to specialize (by tion gaits are periodical, we include the trigonomet- ric functions sin and cos in the GP function set in phase, amplitude, frequency etc.) the temporal mo- tion patterns (i.e. the turning angles) of actuators of addition to the basic algebraic functions. The choice each of the identical morphological segments of the of these trigonometric functions reflects our intention Snakebot. In addition, the terminal symbols of GP to verify the hypothesis (first expressed by Petr Mi- include the variables time and two constants: Pi, turich in 1920's [1]) that undulative motion mecha- and random constant within the range [0,2]. The nisms could yield efficient gaits of snake-likeartifacts introduction of variable time reflects our objective to operating in air, land, or water. develop temporal patterns of turning angles of actua- From another perspective, the introduction of sin tors. The main parameters of the GP are summarized and cos in the function set of GP reflects our inten- in Table 1. tion to mimic (at functional, rather than neurological level) to some extend the Central Pattern Generator 2.2.3 Fitness Evaluation The fitness function is based on the velocity of (CPG) in the central nervous system (usually located in the ganglia or spinal cord) of animals, believed Snakebot, estimated from the distance which the cen- to be necessary and sufficient for the generation of ter of the mass of Snakebot travels during the trial. rhythmic patterns of activities. CPG for robot control The energy consumed by the Snakebot during the

13 224 システム制御情報学会論文誌 第19巻 第6号(2006)

Table 1 Main parameters of GP

trial is not considered in our work. The real values of the raw fitness, which are usually within the range (0, 2) are multiplied by a normalizing coefficientin order to deal with integer fitness values within the range (0,200). A normalized fitness of 100 (one of the ter- mination criteria shown in Table 1) is equivalent to a velocity, which displaced Snakebot a distance equal to twice its length. The fitness evaluation routine is Fig. 2 Fitness evaluation routine shown in Fig. 2. 2.2.4 Representation of Genotype ing results indicate that the fitness evaluation routine Inspired by its flexibility,and the recent widespread consumes more than 99% of the GP runtime. How- adoption of the Document Object Model (DOM) and ever, even for relatively complex genetic programs fea- EXtensible Markup Language (XML), we represent turing a few hundred tree nodes, most of the fitness evolved genotypes of simulated Snakebot as DOM- evaluation runtime at each time step is associated parse trees featuring equivalent flat XML-text, as first with the relatively large computational cost of the discussed in [20]. Our approach implies that both physics simulation (actuators, joint limits, friction, (i) the calculation of the desired turning angles dur- gravity, collisions,etc.) of phenotypic segments of the ing fitness evaluation (function EvalDesiredAngle, simulated Snakebot (routine dWorldStep in Fig. 2, shown in Fig. 2, line 15) and (ii) the genetic opera- line 20), rather than computing the desired turning tions are performed on DOM-parse trees using off-the angles of actuators. shelf, platform and language neutral DOM-parsers. 2.2.5 Genetic Operations The corresponding XML-text representation (rather We employ binary tournament selection - a ro- than S-expression)is used as a fiat file format, feasible bust, commonlyused selection mechanism, which has for migration of genetic programs among the compu- proved to be efficientand simple to code. The crossover tational nodes in an eventual distributed implemen- operation is defined in a strongly typed way in that tation of the GP. The benefits of using DOM/XML- only the DOM-nodes (and corresponding DOM- based representations of genetic programs are (i) fast subtrees) of the same data type (i.e. labeled with the prototyping of GP by using standard built-in API same tag) from parents can be swapped. The sub-tree of DOM-parsers for traversing and manipulating ge- mutation is allowed in a strongly typed way in that a netic programs, (ii) generic support for the represen- random node in the genetic program is replaced by a tation of grammar of strongly-typed GP using W3C- syntactically correct sub-tree. The mutation routine standardized XML-schema; and (iii) inherent Web- refers to the data type of the currently altered node compliance of eventual parallel distributed implemen- and applies a randomly chosen rule from the set of tation of GP. applicable rewriting rules as defined in the grammar The slight performance degradation in computing of the GP. the desired turning angles of actuators by traversing 2.2.6 ODE the DOM/XML-based representation of genetic pro- We have chosen Open Dynamics Engine (ODE) grams during fitness evaluation is not relevant for the [18]to provide a realistic simulation of physics in ap- overall performance of GP. The performance profil- plying forces to phenotypic segments of Snakebot, for

14 TANEV•ERAY•ESHimoHARA:Exploring the Analogy in the Emergent Properties of Locomotion Gaits of Snakebot 225

of the problem solver and a fitness function [2]. Table 2 ODE-related parameters of simulated Snakebot 3.1 Emergent Properties of Evolved Fastest Locomotion Gaits Fig. 3 shows the fitness convergencecharacteristics of 10 independent runs of GP (Fig. 3(a)) and sam- ple snapshots of evolved best-of-run locomotion gaits (Fig. 3(b) and Fig. 3(c)) when fitness is measured regardless of direction in an unconstrained environ- ment. Despite the fact that fitness is unconstrained and measured as velocity in any direction, sidewind- ing locomotion (defined as locomotionpredominantly perpendicular to the long axis of Snakebot) emerged in all 10 independent runs of GP, suggesting that it provides superior speed characteristics for Snakebot morphology. The evolved locomotion gait is much similar to the locomotion of the natural snake Cro- talus cerastes, or sidewinder. In the proposed repre- sentation of Snakebot, similarly to the natural snake, no anisotropic (directional) friction between the mor- phological segments and the surface is considered, consequently, an efficient forward locomotion, which requires an extensive utilization of anisotropic friction could not emerge. simulation of Snakebot locomotion. ODE is a free, The genotype of a sample best-of-run genetic pro- industrial quality software library for simulating ar- gram is shown in Fig. 4. The dynamics of evolved ticulated rigid body dynamics. It is fast, flexible and turning angles of actuators in sidewindinglocomotion robust, and it has built-in collision detection. The result in characteristic circular motion pattern of seg- ODE-related parameters of the simulated Snakebot ments around the center of mass as shown in Fig. 5(a). are summarized in Table 2. The circular motion pattern of segments and the char- acteristic track on the ground as a series of diagonal 3. Empirical Results lines (as illustrated in Fig. 5(b)) suggest that during This section presents the experimental results ver- sidewinding the shape of Snakebot takes the form of ifying the feasibility of applying GP for evolution of a rolling helix (Fig. 5(c)). Fig. 5 demonstrates that the fast locomotion gaits of Snakebot for various fit- the simulated evolution of locomotion via GP is able ness and environmental conditions. In addition, it to invent the improvised cylinder of the sidewinding investigates the emergent properties of (i) the fastest Snakebot to achieve fast locomotion. locomotion gaits, evolved in unconstrained environ- By modulating the oscillations of the actuators mental conditions and (ii) the robust locomotiongaits along the snake's body, the diameter of the cross- evolved in challengingenvironments. The section also section of the sylinder, can be tapered towards ei- discusses the gradual adaptation of the locomotion ther the tail or head of the snake, providing an effi- gaits to degraded mechanical abilities of Snakebot. cient way of steering the Snakebot (Fig. 6(a), 6(b)). These challenges are considered as relevant for suc- We consider the benefits of modulating the oscilla- cessfulaccomplishment of various practical tasks dur- tions of actuators along the body of Snakebot as a ing anticipated exploration, reconnaissance, medicine straightforward implication of our understanding of and inspection missions. In all of the cases considered, the emergent form of a rolling helical Snakebot. Such the fitness of Snakebot reflects the low-levelobjective modulation is implemented as a handcrafted (rather (i.e. what is required to be achieved) of Snakebot than evolved) feature of evolved best-of-run sidewind- in these missions, namely, to be able to move fast re- ing Snakebots. Snapshots, shown in Fig. 6(c) and 6(d) gardless of environmental challengesor degraded abil- illustrate the ability of Snakebot to perform sharp ities. The experiments discussed illustrate the ability turns with a radius similar to its length in both clock- of the evolving Snakebot to learn how (e.g. by dis- wise and counterclockwisedirections. covering beneficial locomotion traits) to accomplish In order to verify the superiority of velocity char- the required objective without being explicitly taught acteristics of sidewindinglocomotion for Snakebot mor- about the means to do so. Such know-how acquired phology we compared the fitness convergencecharac- by Snakebot automatically and autonomously can be teristics of evolution in an unconstrained environment viewed as a demonstration of emergent intelligencein for the followingtwo cases: (i) unconstrained fitness that the task-specificknowledge of how to accomplish measured as velocity in any direction (as discussed the task emerges in the Snakebot from the interaction above), and (ii) fitness, measured as velocity in the

15 226 システム制御情報学会論文誌 第19巻 第6号(2006)

(a)

(a)

(b)

(b)

(c)

Fig. 5 Trajectory of the central segment (cs) around (c) the center of mass (cm) of Snakebot for a sam- ple evolved best-of-run sidewinding locomotion Fig. 3 Evolution of locomotion gaits for cases where fit- (a), traces of ground contacts (b), and Snakebot, ness is measured as velocity in any direction. Fit- wrapped around an imagined cylinder taking the ness convergencecharacteristics of 10 independent form of a rolling helix runs (a), probability of success (b), and snapshots of sample evolved best-of-run sidewinding loco- of the simulated Snakebot confined in a narrow tun- motion gaits of simulated Snakebot (c). The dark nel are shown in Fig. 8. The width of the tunnel is trailing circles in (c) depict the trajectory of the three times the diameter of the cross-section of the center of the mass of Snakebot. Timestamp inter- segment of Snakebot. Compared to forward locomo- val between each of these circles is fixed and it is tion in an unconstrained environment (Fig. 7), the the same (10 time steps) for both snapshots. velocity in this experiment is superior, and compa- rable to the velocity of sidewinding (Fig. 3). This, GenH = (sin(((sin(-8))* (segmentid-time))+(3 * seemingly anomalous emergent phenomenon demon- time)))/(sin(-8)); strates the ability of simulated evolution to discovera GenV = sin(ADF) way to utilize the walls of the tunnel as a source of (i) extra grip and (ii) locomotion gaits (e.g., vertical un- Fig. 4 Normalized algebraic expressions of the genotype dulations) which are fast yet unbalanced in an uncon- of a sample best-of-run genetic program: dynam- strained environment. Indeed, as soon as the Snake- ics of turning angle of horizontal (GenH) and ver- tical (GenV) actuators. The value of the auto- bot clears the tunnel, the gait flattens (Fig. 8(c)) and matically defined function ADF, in GenV, is eval- velocity (visually estimated as a distance between the uated by equalizing it to the value of the currently traces of the center of gravity of Snakebot) drops dra- evaluated GenH matically. forward direction only. The results of evolution of 3.2 Robustness via Adaptation to Chal- forward locomotion, shown in Fig. 7, indicate that lenging Environment. Generality non-sidewinding motion features much inferior veloc- of the Evolved Robust Gaits ity characteristics, compared to sidewinding. Adaptation in Nature is viewed as an ability of The results of evolution of species to discover the best phenotypic (i.e. pertain-

16 TANEV•ERAY•ESmmoHARA:Exploring the Analogy in the Emergent Properties of Locomotion Gaits of Snakebot 227

(a) (b)

(a)

(c) (d)

Fig. 6 Steering the Snakebot. The Snakebot moving straight is wrapped around an imagined cylinder taking the form of a rolling helix (a). By modu- lating the oscillations of the actuators along the snake's body, the diameter of the cross-section of the "cylinder" can be tapered towards either the tail or head of the snake, providing an efficient way of "steering" the Snakebot: (b) illustrates the Snakebot turning counterclockwise. The im- ages in (a) and (b) are idealized: in the simulated (b) (c) Snakebot (and in snakes in Nature too) the cross- sectional areas of the imagined "cylinder" (a) and Fig. 8 Evolution of locomotion gaits of Snakebot when "cone" (b) are much more similar to ellipses (as confined in a narrow tunnel: fitness convergence shown in Fig. 5(a)) rather than to perfect circles characteristics of 10 independent runs of GP (a) as depicted here. The snapshots shown in (c) and and snapshots of sample evolved best-of-run gaits (d) illustrate the Snakebot performing sharp turns at the intermediate (b) and final stages of the trial in both clockwise and counterclockwisedirections, (c). respectively. morphism, polyphenism, immune response, adaptive metabolism, etc.) occurring during the lifetime of in- dividuals. In our approach we employ GP for adap- tation of Snakebot to changes in the fitness landscape caused by (i) a challenging environment and (ii) par- tial damage to 1, 2, 4 and 8 (out of 15) morphological segments. The former case is discussed in this subsec- tion, while the latter case is elaborated in the follow- ing subsection. In both cases of adaptation, GP is ini- (a) (b) tialized with a population comprising 20 best-of-run genetic programs, obtained from 10 independent evo- Fig. 7 Evolution of locomotion gaits for cases where fit- lutionary runs in unconstrained environments, plus ness is measured as velocity in the forward direc- an additional 180 randomly created individuals. tion only: fitness convergence characteristics of 10 The challenging environment is modeled by the in- independent runs of GP (a) and snapshots of sam- troduction of immobile obstacles comprising 40 small, ple best-of-run forward locomotion (b). Times- randomly scattered boxes, a wall with height equal to tamp interval between the traces of the center of 0.5 diameters of the cross-section of Snakebot, and a the mass is the same as for sidewinding locomo- flight of 3 stairs, each with height equal to 0.33 di- tion gaits, shown in Fig. 3(c). ameters of the cross-section of Snakebot. Both the wall and the flat have a finite length. However, be- ing to biochemistry, morphology, physiology, and be- havior) traits for survival in a continuously chang- cause no feedback from the environment to steer the ing fitness landscape. Adaptive phenotypic traits are snakebot is employed in our experiment, any attempt the result of beneficial genetic changes which occurred of the Snakebot to bypass the wall would lead to a sort of sustained cycling trajectories of the snakebot. during the course of evolution (phylogenesis) and/or However, these trajectories would be discouraged by phenotypic plasticity (ontogenesis-learning, poly- the simulated evolution because they feature inferior

17 228 システム制御情報学会論文誌 第19巻 第6号(2006)

a) b) c) d)

e) f) g)

Fig. 9 Snapshots illustrating the sidewinding Snakebot, initially evolved in unconstrained environment, before the adap- tation - initial (a), intermediate (b and c) and final stages of the trial (d), and after the adaptation to challenging environment via GP-initial (e), intermediate (f) and final stages of the trial (g). The challenging environment is modeled by the introduction of immobile obstacles comprising 40 small, randomlyscattered boxes, a wall with height equal to 0.5 diameters of the cross-section of Snakebot, and a flight of 3 stairs, each with height equal to 0.33 diameters of the cross-section of Snakebot.

a) c)

b) d)

e)

Fig. 10 Snapshots of frontal view (a, c) and view from the above (b, d) of sample sidewinding Snakebots before and after the adaptation, respectively. The frontal views (a and c) comparatively illustrates the additional elevation of the body of the adapted Snakebot. The trajectory of the central segment (cs) around the center of mass (cm) of Snakebot for sample best-of-run sidewinding locomotion after the adaptation (e) to challenging environment indicates that the elevation of the central segment after the adaptation (a) is twice as high as before the adaptation (as illustrated in Fig. 5(a)). distance between the position of the snakebot at the The emergent properties of the robust sidewind- start and the finish of the trial, and consequently, in- ing gaits are shown in Fig. 10. As depicted in the ferior fitness values. The fitness of adapting Snakebot Figure, the additional elevation of the body, required is measured in any direction. to negotiate the obstacles faster represents the emer- The empirical results of adaptation of the sidewind- gent know-how in the adapting Snakebot. As shown ing Snakebot, obtained over 10 independent runs re- in Fig. 10(e), the trajectory of the central segment veal the poor performance of the best-of-run Snake- around the center of the mass of sample adapted Snake- bots initially evolved in unconstrained environments. bot is almost twice as high as before the adaptation The fitness of the best-of-run Snakebots immediately (Fig. 5(a)). Moreover, as the snapshots of the adapted drops from initial value of 100 in the unconstrained gaits of Snakebot viewed from the above (Fig. 10(b) environment to only 65 when Snakebot is first tested and 10(d)) reveal, the robust locomotion gaits are as- (at Generation #0) on the challenging terrain, which sociated with much higher winding angle of locomo- indicates the poor initial robustness of these locomo- tion (about 120) yielding longitudinally more com- tion gaits. However, adapting to the new environ- pact sidewinding Snakebots. ment, the evolving Snakebots are able to discover lo- The generality of the evolved robust sidewinding comotion gaits which are robust enough to allow the gaits is demonstrated by the ease with which Snake- Snakebots to overcome the various kinds of obstacles bot, evolved in known challenging terrain overcomes introduced in the environment. About 20 generations various types of unanticipated obstacles such as a pile of computational effort is required to reach fitness val- of or burial under boxes, and small walls, as illus- ues of 100 in the challenging environment with prob- trated in Figs. 11, 12, and 13. ability of success 0.9. Snapshots illustrating the per- formance of a sample best-of-run Snakebot initially 3.3 Adaptation to Partial Damage evolved in unconstrained environment, before and af- The adaptation of sidewinding Snakebot to par- ter the adaptation to the challenging environment are tial damage to 1, 2, 4 and 8 (out of 15) segments by shown in Fig. 9. gradually improving its velocity is shown in Fig. 14.

18 TANEV•ERAY•ESHimoHARA:Exploring the Analogy in the Emergent Properties of Locomotion Gaits of Snakebot 229

a) b) c)

d) e) f)

Fig. 11 Snapshots illustrating the generality of sidewinding Snakebot adapted to the challenging environment depicted in Fig. 9. Before the adaptation to the challenging environment the Snakebot overcomes an unanticipated pile of boxes slower (a, b and c) than after the adaptation (d, e, and f).

a) b) c)

d) e) f)

Fig. 12 Snapshots illustrating the generality of sidewinding Snakebot adapted to the challenging environment depicted in Fig. 9. Before the adaptation to the challenging environment the Snakebot emerges from an unanticipated burial under a pile of boxes slower (a, b and c) than after the adaptation (d, e, and f).

a) b) c) d)

e) f) g)

Fig. 13 Snapshots illustrating the generality of sidewinding Snakebot adapted to the challenging environment depicted in Fig. 9. Before the adaptation to the challenging environment the Snakebot clears unanticipated walls forming a pen slower (a, b, c and d) than after the adaptation (e, f and g). The walls are twice as high as in the challenging terrain of Fig. 9, and their height is equal to the diameter of the cross-section of Snakebot.

Demonstrated results are averaged over 10 indepen- segments, Snakebot recovers to an average of 100% of dent runs for each case of partial damage to 1, 2, 4 its previous velocity. With 4 and 8 damaged segments and 8 segments. The damaged segments are evenly the degree of recovery is 92% and 72% respectively. distributed along the body of Snakebot. Damage in- The emergent properties of adapted sidewinding lo- flicted to a particular segment implies a complete loss comotion gaits are shown in Fig. 15. of functionality of both horizontal and vertical actu- ators of the corresponding joint. As Fig. 14 depicts, Snakebot quickly and completely recovers from dam- age to a single segment, attaining its previous velocity only in 7 generations. Also in the case of 2 damaged

19 230 システム制御情報学会論文誌 第19巻 第6号(2006)

or/and degraded Snakebots' abilities). Although the alternative approaches of online adaptation to the considered changes in the fitness landscape of the evolved Snakebot do really provide a feasible solution, they are usually obtained through a series of trials in which the adapting artifact interacts with the sur- rounding environment. These trials, conducted on- a) b) c) d) line, might be costly, time and energy consuming, and even dangerous for the artifact itself. Fig. 14 Adaptation of Snakebot to damage of 1 (a), 2 From another perspective, the similarity of the (b), 4 (c) and 8 (d) segments, respectively. The emergent properties of the robust and the adapted results are averaged over 10 independent runs of GP for the cases when GP is initialized with the gaits might imply even faster adaptation to unantic- best-of-run Snakebots evolved in unconstrained ipated partial damage when the initial population in and challenging terrain respectively. GP is fed by the best-of-run robust Snakebots (i.e. Snakebots, evolvedin advanceon challengingterrain), rather than with the Snakebots evolved on smooth terrain as considered in this work. In order to verify the latter hypothesis, we conducted an experiment similar to the one elaborated in Section 3.3, initializ- ing the population with 20 best-of-run robust Snake- bots evolved in challenging terrain rather than with the best-of-run Snakebots evolved in unconstrained a) b) environment. As shown in Fig. 14, the population, initialized with robust Snakebots, adapts to unan- ticipated partial damage faster and to higher veloc- ity than populations of Snakebots evolved in uncon- c) d) e) f) strained environment. 4. Conclusion We investigated the emergent properties of sim- ulated Snakebots which were automatically designed g) h) i) j) through Genetic Programming (GP) to achieve the fastest possible robust, general, and adaptive sidewind- Fig. 15 The emergent properties of adapted sidewinding ing locomotion gaits. Considering the notion of emer- locomotion gaits: frontal view of the Snakebot gent intelligence as the ability of Snakebot to achieve before (a) and after the adaptation (b) to the its goals (of moving fast) without the need to be ex- damage of a single segment demonstrates the ad- plicitly taught how to do so, we present empirical ditional elevation of the adapted Snakebot. View results demonstrating the emergence of sidewinding of the shape of the sidewinding Snakebot from above reveals the emergent tendency of increas- (as a fastest possible locomotion) from relatively sim- ing the winding angle of locomotion in a way ple morphological segments of Snakebot. The emer- similar to adaptation to the challenging environ- gent properties of both (i) the evolved robust and ment (as shown in Fig. 9): Snakebot with 1 (c, (ii) the adapted sidewinding locomotion gaits, fea- d), 2 (e, f), 4 (g, h) and 8 (i, j) damaged segments ture analogous elevation of the Snakebot above the before and after the adaptation, respectively. ground. The additional elevation of the robust gaits of Snakebot can be explained by the emergent knowl- 3.4 Implications of the Analogy between edge of the need to overcome obstacles by the Emergent Properties of Sidewind- over them similar to way legged artifacts negotiate ing Gaits obstacles in challenging environments. The rationale The anticipated practical implications of the anal- of the emerged additional elevation of adapted gaits ogy between the emergent properties of the robust however could be seen in the know-how, discovered sidewinding gait adapted to a challenging environ- by the simulated evolution, that the friction between ment and the gait adapted to unanticipated partial the damaged (i.e. inactive) segments and the surface damage are associated with the possibility to develop is no longer a source of traction forces as it is the case a general locomotiongait which could be autonomously of active segments. Moreover, this friction results in activated in case of any degradation of Snakebot's drag forces, which are detrimental to the velocity of performance (e.g. the velocity), without the need the locomotion, and consequently, such forces should for the Snakebot to diagnose the underlying reason be minimized. for such degradation (e.g., a challenging environment We verified that the similarity of the emergent properties of the robust and the adapted gaits im-

20 TANEV•ERAY•ESHIMOHARA : Exploring the Analogy in the Emergent Properties of Locomotion Gaits of Snakebot 231 ply even faster adaptation to unanticipated partial genetic programming; Genetic Programming and damage when the initial population in GP is fed by Evolvable Machines, Vol. 1 No. 1-2, pp. 121-164 the best-of-run robust Snakebots. Inheriting and fur- (2000) ther evolving the relevant emergent properties of the [11] I.B. Levitan and L.K. Kaczmarek: The Neuron: robust gaits, the population, initialized with robust Cell and Molecular Biology, Oxford University Press Snakebots adapts to unanticipated partial damage (2002) [12] S. Mandavi and P. Bentley: Evolving motion of faster and to higher velocity than population of Snake- robots with muscles; Proc. of EvoROB2003, the bots evolved in unconstrained environment. 2nd European Workshop on Evolutionary Robotics, Considering the situational awareness (or situat- EuroGP-2003, pp. 655-664 (2003) edness [15]) as a necessary condition for any intelli- [13] H.J. Morowitz: The Emergence of Everything: How gent autonomous artifact, in our future work we are the World Became Complex,Oxford University Press planning to investigate the feasibility of incorporating (2002) sensors to allow the Snakebot to explicitly perceive [14] L. Meeden and D. Kumar: Trends in evolutionary the surrounding environment. We are especially in- robotics; Soft Computingfor Intelligent Robotic Sys- terested in sensors which do not compromise the ro- tems (edited by L.C. Jain and T. Fukuda), Physica- bustness characteristics of the Snakebot - such as, for Verlag, pp. 215-233 (1998) example Golgi's tendon receptors, incorporated inside [15] R. Pfeifer and C. Scheier: Understanding Intelli- the potentially completely sealed Snakebot. gence, MIT Press (1999) [16] K. Sims: Evolving 3D morphology and behavior by competition; Artificial Life IV Proceedings, MIT Acknowledgements Press, pp. 28-39 (1994) [17] T. Ray: Aesthetically evolved virtual pets; Leonardo, The research was supported in part by the Na- Vol. 34, No. 4, pp. 313-316 (2001) tional Institute of Information and Communications [18] R. Smith: Open dynamics engine, 2001, URL: Technology of Japan. http ://q 12.org / ode/ [19] S. Takamura, G.S. Hornby, T. Yamamoto, J. Yokono References and M. Fujita: Evolution of dynamic gaits for a robot; IEEE International Conference on Consumer [1] Y. Andrusenko: Russian Culture Navigator: Electronics, pp. 192-193 (2000) Miturich- Khlebnikovs: Art Trade Runs In The [20] I.T. Taney: DOM/XML-Based portable genetic rep- Family resentation of morphology, behavior and communica- URL : http ://www. vor. ru / culture / cultarchl91_eng.html tion abilities of evolvable agents; Artificial Life and [2] P.J. Angeline: Genetic programming and emer- Robotics, an International Journal, Vol. 8, No. 1, gent intelligence; Advances in Genetic Programming pp. 52-56, Springer-Verlag (2004) (K.E. Kinnear, Jr., Ed.), MIT Press, pp. 75-98 [21] R. Worst: Robotic snakes; Proceedings of the Third (1994) German Workshop on Artificial Life, Verlag Harri [3] J. W. Burdick, J. Radford and G.S. Chirikjian: A 'Sidewinding' locomotion gait for hyper-redundant Deutsch, pp. 113-126 (1998) [22] Y. Zhang, M.H. Yim, C. Eldershaw, D.G. Duff robots; Proceedings of the IEEE int. conf. on Robotics and K.D. Roufas: Phase automata: a programming 〓 Automation, PP. 101-106 (1993) model of locomotion gaits for scalable chain-type [4] K. Dowling: Limbless locomotion: learning to modular robots; IEEE/RSJ International Confer- crawl with a snake robot, doctoral dissertation, ence on Intelligent Robots and Systems (IROS 2003) Tech. report CMU-RI-TR-97-48, Robotics Institute, Carnegie Mellon University (1997) (2003) [5] S. Hirose: Biologically Inspired Robots: Snake-like Locomotors and Manipulators, Oxford University Press (1993) [6] S. Kamio, H. Mitsuhashi and H. Iba: Integration of genetic programming and reinforcement learning for real robots; Proceedings of the Genetic and Evo- lutionary Computations Conference (GECCO 2003), pp. 470-482 (2003) [7] N. Jacobi: Minimal Simulations for Evolutionary Robotics, Ph.D. thesis, School of Cognitive and Com- puting Sciences, Sussex University (1998) [8] J.R. Koza: Genetic Programming: On the Program- ming of Computers by Means of Natural Selection, MIT Press (1992) [9] J.R. Koza: Genetic Programming 2: Automatic Dis- covery of Reusable Programs, MIT Press (1994) [10] J.R. Koza, M.A. Keane, J.Yu, F.H. Bennett III and W. Mydlowec: Automatic creation of human- competitive programs and controllers by means of

21 232 システム制御情報学会論文誌 第19巻 第6号(2006)

22