From: Proceedings of the Third International Conference on Multistrategy Learning. Copyright © 1996, AAAI (www.aaai.org). All rights reserved. CoevolutionLearning: Synergistic Evolution of LearningAgents and ProblemRepresentations LawrenceHunter National Library of Medicine 8600 RockvillePike Bethesda, MD20894 hunter@,nlm.nih.gov Abstract representationsand parametersettings, usingas inspiration This paper describes exploratory workinspired by a a recent mathematicalmodel of the coevolutionof genetics recent mathematicalmodel of genetic and cultural and culture from anthropology (Durham, 1991). coevohition.In this work,a simulator implementstwo importantadditional feature of this workis that is provides independent evolutionary competitions which act a simple, seamlessand effective wayof synergistically simultaneouslyon a diverse population of learning combiningmultiple inference methodsin an integrated and agents: one competition searches the space of free extensible framework. Whenonly a single inference parameters of the learning agents, and the other methodis used, this frameworkreduces to a variant on searches the space of input representations used to constructive induction. However,with multiple and characterize the training data. The simulations diverse learningagents, the systemis able to generateand interact with eachother indirectly, both effecting the exploit synergies betweenthe methodsand achieve results fitness (and hencereproductive success) of agents that canbe superiorto anyof the individualmethods acting the population. This frameworksimultaneously alone. addressesseveral openproblems in machinelearning: The workpresented here fits into a growingbody of selection of representation, integration of multiple research on these issues. The importance of bringing heterogeneouslearning methodsinto a single system, learningengineering tasks withinthe purviewof the theory and the automated selection of learning bias and practice of automated systems themselves was appropriatefor a particular problem. described at length in (Schank, et al., 1986). Some effective computationalmethods for aspects of this task have been reported recently. (Kohavi & John, 1995,) Introduction describes an automatedmethod for searching through the space of possible parameter values for C4.5. Genetic Oneclear lesson of machinelearning research is that algorithms have also been used to search the space of problemrepresentation is crucial to the success of all parameter values for artificial neural networks (Yao, inference methods(see, e.g. (Dietterich, 1989;Rendell 1993). There has also been recent work on selecting Cho, 1990; Rendell &Ragavan, 1993) ). However,it appropriate("relevant") subsets of features froma superset generallythe case that the choiceof problemrepresentation of possible features for input representation in machine is a task doneby a humanexperimenter, rather than by an learning ((Langley, 1994) is a review of 14 such automatedsystem. Also significant in the generalization approaches), as well as a long history of work on performanceof machinelearning systemsis the selection constructive induction ((Wnek & Michalski, 1994) of the inference method’sfree parametervalues (e.g. the includes a review, but see also (Wisniewski& Medin, numberof hiddennodes in an artificial neural network,or 1994) for critical analysis of this work. This paper the pruningseverity in a decision tree inductionsystem), describes an exploratory approachthat appears to have which is also a task generally accomplishedby human promisein addressingthese issues in an integratedway. "learning engineers"rather than by the automatedsystems themselves. The effectiveness of input representations and free parametervalues are mutuallydependent. For example,the Background appropriatenumber of hiddennodes for an artificial neural networkdepends crucially on the numberand semanticsof Theidea of coevolutionlearning is to use a kindof genetic the input nodes. This paper describes exploratorywork on algorithm to search the space of values for the free a method for simultaneously searching the spaces of parametersfor a set of learningagents, and to use a system Hunter 81 From: Proceedings of the Third International Conference on Multistrategy Learning. Copyright © 1996, AAAI (www.aaai.org). All rights reserved. metaphorically based on recent anthropological theories of synthesis as the source of novelty. By providing a cultural evolution to search through the space of possible computationalmodel in whichindividuals are able to learn input representations for the learning agents. This section from their experiences and share what they have learned, it provides somebrief backgroundon these ideas. becomespossible to simulate a cultural kind of evolution. Evolutionary algorithms are a weak search method A key aspect of any model of cultural evolution is the related to, although clearly distinguishable from, other specification of the smallest unit of information that is weak methods (e.g. beam search). They are based on transmitted from one agent to another during cultural metaphor with naturally occurring genetic evolution. In transmission, christened the "meme"by Richard Dawkins. general, evolution has three components:inheritance, or the Although there is a great deal of argument over what passage of traits from one individual to another: variation, memosare (ideas. symbols, thoughts, rules patterns, values, of the generation of novel traits or combinationsof traits" principles, postulates, concepts, essences and premises and selection, a competition betweenindividuals based on have all been suggested), and a great deal of theoretical their traits that effects the probability that an individualwill analysis describing howmemes compete, are transformed, have descendants that inherit from it. There are manyways interact with genes, etc., I am aware of no attempts to of building computational models that evolve, including operationalize the term so that it wouldbe possible to build genetic algorithms, genetic programmingand evolutionary computational simulations of populations of memes. strategies; see (Angeline, 1993) for an excellent survey A memomust play several roles in a simulation of and analysis of the approach. cultural inheritance. First, it mustbe able to have an effect Anthropological theories of culture have been on the behavior of the individuals in simulation. Second, converging for some time on what are now known as individuals must be able to create newmemos as a result of "’ideational" theories. Theyhold that culture "consists of their experiences (e.g. by innovation, discovery or shared ideational phenomena(values, beliefs, ideas, and the synthesis). Third, it must be possible for other individuals like) in the minds of humanbeings. [They refer] to a body in the simulation to evaluate memesto determine whether or ’pool’ of informationthat is both public (socially shared) or not they will adopt a particular memefor its ownuse. and prescriptive (in the sense of actually or potentially In the sections below, I will show how the input guiding behavior)." (Durham,1991), p. 3. Note that this representation used by a machine learning system can be view is different from one that says culture is some used to meet these requirements. particular set of concrete behavior patterns; it instead suggests that culture is just one of the factors that shapes A Formal Definition of Coevolution Learning behavior. An individual’s phenotype(here, its behavio0 is influenced by its genotype, its individual psychologyand A coevolution learning system functions by evolving a its culture. Durhamsummarizes "*the new consensus in population of learning agents. The population is defmedby anthropology regards culture as systems of symbolically a classification task T, a set of classified examplesof that encoded conceptual phenomena that are socially and task expressed in a primitive representation Ep, a fitness historically transmitted within and between populations" function for agents fA, and a set of learning agents A: (p. 8-9). This historically rooted, social transmission ideational elements can be analyzed as an evolutionary ecoevffi-{Y, Ep, fA, A} process. The characterization of that evolutionary process A fitness function for agents mapsa memberof the set A to (and its relationship to genetic evolution) is the subject a real number between 0 and 1. For convenience, it is Durham’sbook, and also the subject of a great deal other useful to define PL to be the subset of Pcoevwhere all the research dating back at least one hundred years (muchof agents use learning methodL (see below). which is surveyed by Durham). Each learning agent A is defined by a learning methodL, a Thereare several significant differences betweencultural vector of parameter values v, a fitness function for memes and genetic evolution. Cultural traits are transmitted fro, and an ordered set of problem representation differently than genetic ones in various ways.It is possible transformations R: to transfer cultural traits to other membersof your current generation, or even to your cultural "parents." It is also Ai-{L, V, fm, R} possible for one individual to pass cultural waits to very The vector of parameter values mayhave different length manymore others than he or she could genetically. The for different learning methods. Each memberof the set of selection pressure in the competition among cultural problem representation transformations Ri is a mapping entities is not based on their reproductive fitness