Evolutionary Pressure in Lexicase Selection Jared M
Total Page:16
File Type:pdf, Size:1020Kb
When Specialists Transition to Generalists: Evolutionary Pressure in Lexicase Selection Jared M. Moore1 and Adam Stanton2 1School of Computing and Information Systems, Grand Valley State University, Allendale, MI, USA 2School of Computing and Mathematics, Keele University, Keele, ST5 5BG, UK [email protected], [email protected] Abstract Downloaded from http://direct.mit.edu/isal/proceedings-pdf/isal2020/32/719/1908636/isal_a_00254.pdf by guest on 02 October 2021 Generalized behavior is a long standing goal for evolution- ary robotics. Behaviors for a given task should be robust to perturbation and capable of operating across a variety of envi- ronments. We have previously shown that Lexicase selection evolves high-performing individuals in a semi-generalized wall crossing task–i.e., where the task is broadly the same, but there is variation between individual instances. Further Figure 1: The wall crossing task examined in this study orig- work has identified effective parameter values for Lexicase inally introduced in Stanton and Channon (2013). The ani- selection in this domain but other factors affecting and ex- plaining performance remain to be identified. In this paper, mat must cross a wall of varying height to reach the target we expand our prior investigations, examining populations cube. over evolutionary time exploring other factors that might lead wall-crossing task (Moore and Stanton, 2018). Two key to generalized behavior. Results show that genomic clusters factors contributing to the success of Lexicase were iden- do not correspond to performance, indicating that clusters of specialists do not form within the population. While early tified. First, effective Lexicase parameter configurations ex- individuals gain a foothold in the selection process by spe- perience an increasing frequency of tiebreaks. That is, if two cializing on a few wall heights, successful populations are or more individuals are tied after going through all objec- ultimately pressured towards generalized behavior. Finally, tives considered during the selection event, a random selec- we find that this transition from specialists to generalists also tion from the tied set of individuals is performed. Tiebreak leads to an increase in tiebreaks, a mechanism in Lexicase, during selection providing a metric to assess the performance events encourage an increase in population diversity be- of individual replicates. cause the choice is not based on performance. With a suf- ficient number of environments (>= 5) considered during Introduction a selection event, the frequency of tiebreaks increases over Robotic systems should exhibit robust behaviors that are re- time in effective replicates. The second factor is that high- silient to perturbation and operate effectively across a wide performing individuals evolve in populations that settle into variety of conditions. However, realizing these goals is a a specific range of population diversity. Still, as we demon- continuing challenge from an algorithmic standpoint. In this strated in Moore and Stanton (2019), tiebreaks appear to be paper, we expand our earlier investigation into Lexicase Se- one of the key factors of Lexicase selection. These exper- lection in the context of evolutionary robotics (ER). Lexi- iments pushed the limits of Lexicase to understand when case is a many-objective selection operator (Spector, 2012) it breaks down. Lexicase exhibits a surprising resilience, which we applied as part of a generational genetic algo- maintaining high performance even when the range of wall rithm (GA) to an established wall-crossing task (Moore and heights considered in a selection event is heavily restricted Stanton, 2017). Figure 1 illustrates this generalized con- to only a narrow band of adjacent wall heights. Under these trol problem where individuals are challenged to express a constraints, tiebreaks continue to increase over time, but gait capable of reaching a target position by crossing a wall their correlation to performance breaks down as the search whose height varies on a per-trial basis. Our initial investi- is biased towards small areas of the objective space. gation demonstrated that a GA with Lexicase selection reli- The two prior investigations into Lexicase’s effectiveness ably discovered generalized controllers capable of crossing explored performance metrics aggregated across many repli- the majority of wall heights, significantly exceeding the per- cates and their effect on population diversity. Another direc- formance of custom algorithms designed for the task (ibid.) tion is to examine population genetics and the performance We then attempted to identify the underlying mechanisms of ancestral populations across the 100 wall heights over responsible for the high performance of Lexicase in this the course of evolution identifying trends that might indi- 719 cate successful versus unsuccessful conditions. Here, we using distance traveled as the primary metric, but secondary look for possible speciation in genome space. We hypothe- objectives such as efficiency can also be considered if mul- size that since Lexicase evolves specialists (Pantridge et al., tiple objectives are employed (Moore and McKinley, 2016). 2018), individuals that are high performing in one or a lim- Other robust controllers consider evolving multiple behav- ited set of objectives, clusters will form in genome space iors for a single robotic platform (Doncieux and Mouret, exhibiting similar performance characteristics to each other 2013). but different from those of other clusters. Second, we eval- These pursuits of robustness (Lehman et al., 2013), uate performance of populations at different points in evo- resilience (Kriegman et al., 2019), behavioral diversity lution to see how individuals, and the populations at large, and generalized behavior thus involve multi-objective (Deb transition from specialists to generalists. et al., 2002) or many-objective algorithms like Lexicase se- In this paper, we expand our investigation further on data lection (Spector, 2012) that evaluate performance of individ- originally reported in Moore and Stanton (2018). We focus uals on more than one metric. Generalized behavior involves on exploring the assumption that initial populations contain the expression of multiple robust behaviors, for example specialists, individuals whose performance is poor on all forward locomotion, turning, and reversing (Cully et al., but a few task configurations. These individuals may then 2015). In this paper, we continue to investigate the semi- Downloaded from http://direct.mit.edu/isal/proceedings-pdf/isal2020/32/719/1908636/isal_a_00254.pdf by guest on 02 October 2021 begin to produce offspring that carry this genetic informa- generalized, many-objective task explored in Stanton and tion forward, adding additional competencies resulting in a Channon (2013) and Moore and Stanton (2017, 2018, 2019) more general capability that comes to dominate the popu- wherein individuals evolve to express a robust behavior ca- lation. Alternatively, specialist genetic information may be pable of crossing a wall of varying heights with the same ephemeral and persist in the population only until generalists controller. We consider the behavior to be semi-generalized evolve that can out-compete specialists. We examine ances- as individuals are evolved on different wall heights, each tral populations over evolutionary time to determine what height constituting a unique objective. However, the over- features arise that could indicate the procession of evolving all goal for all objectives is similar, each being a variant of species from sparse specialists to converged generalists. We a general wall-crossing task, thus the composite problem re- look specifically at two areas. First, whether clusters of indi- quires generalization. viduals (putative sub-species) in genotype space correspond In our earliest work to address the generalized wall- with clusters in performance space (indicating specialists) crossing task, individuals were exposed to a variety of objec- and whether changes in these clusters over evolutionary time tives over evolutionary time, according to a number of dif- are indicative of the emergence of generalists in the popula- ferent presentation strategies (Stanton and Channon, 2013). tion. Second, we consider whether the number of tiebreaks It was found that commonly used strategies such as random provide insight into the transition of species from being pri- presentation of the individual objectives are inferior to cus- marily specialists to primarily generalists. tom strategies that take into account linkage between wall Results of this paper are as follows. First, we find that in- heights. In these strategies, evolutionary pressure to adapt dividuals clustered by genomes do not exhibit similar perfor- is thus maintained, whilst ensuring that species do not for- mance, in contradiction of our initial hypothesis. There are get previously learned behaviors that address specific wall also no apparent performance differences between clusters. heights. However, these strategies all used single-objective Instead, populations transition from individual specialists to- selection mechanisms, albeit with the full spectrum of task wards generalized wall crossing behavior over the course of parameterisations explored through environmental change 5,000 generations. We also find additional evidence sup- over evolutionary time. In contrast, Lexicase accommodates porting earlier observations that a high number