LectureLecture 1010 EvolutionaryEvolutionary Computation:Computation: EvolutionEvolution strategiesstrategies andand geneticgenetic programmingprogramming EvolutionEvolution strategiesstrategies GeneticGenetic programmingprogramming SummarySummary
Negnevitsky, Pearson Education, 2002 1 EvolutionEvolution StrategiesStrategies AnotherAnother approachapproach toto simulatingsimulating naturalnatural evolutionevolution waswas proposedproposed inin GermanyGermany inin thethe earlyearly 1960s.1960s. UnlikeUnlike geneticgenetic algorithms,algorithms, thisthis approachapproach −− calledcalled anan evolutionevolution strategystrategy −− waswas designeddesigned toto solvesolve technicaltechnical optimisationoptimisation problems.problems.
Negnevitsky, Pearson Education, 2002 2 InIn 19631963 twotwo studentsstudents ofof thethe TechnicalTechnical UniversityUniversity ofof Berlin,Berlin, IngoIngo RechenbergRechenberg andand Hans-PaulHans-Paul SchwefelSchwefel,, werewere workingworking onon thethe searchsearch forfor thethe optimaloptimal shapesshapes ofof bodiesbodies inin aa flow.flow. TheyThey decideddecided toto trytry randomrandom changeschanges inin thethe parametersparameters definingdefining thethe shapeshape followingfollowing thethe exampleexample ofof naturalnatural mutation.mutation. AsAs aa result,result, thethe evolutionevolution strategystrategy waswas born.born. EvolutionEvolution strategiesstrategies werewere developeddeveloped asas anan alternativealternative toto thethe engineer’sengineer’s intuition.intuition. UnlikeUnlike GAsGAs,, evolutionevolution strategiesstrategies useuse onlyonly aa mutationmutation operator.operator.
Negnevitsky, Pearson Education, 2002 3 BasicBasic evolutionevolution strategiesstrategies InIn itsits simplestsimplest form,form, termedtermed asas aa (1(1++1)-evolution1)-evolution strategy,strategy, oneone parentparent generatesgenerates oneone offspringoffspring perper generationgeneration byby applyingapplying normallynormally distributeddistributed mutation.mutation. TheThe (1(1++1)-evolution1)-evolution strategystrategy cancan bebe implementedimplemented asas follows:follows: StepStep 11:: ChooseChoose thethe numbernumber ofof parametersparameters NN toto representrepresent thethe problem,problem, andand thenthen determinedetermine aa feasiblefeasible rangerange forfor eacheach parameter:parameter: {}{}{} x1min , x1max , x2min, x2max , . . . , xNmin , xNmax DefineDefine aa standardstandard deviationdeviation forfor eacheach parameterparameter andand thethe functionfunction toto bebe optimisedoptimised..
Negnevitsky, Pearson Education, 2002 4 StepStep 22:: RandomlyRandomly selectselect anan initialinitial valuevalue forfor eacheach parameterparameter fromfrom thethe respectiverespective feasiblefeasible range.range. TheThe setset ofof thesethese parametersparameters willwill constituteconstitute thethe initialinitial populationpopulation ofof parentparent parameters:parameters:
xx1,, xx2,, ...... ,, xxN StepStep 33:: CalculateCalculate thethe solutionsolution associatedassociated withwith thethe parentparent parameters:parameters:
XX == ff ((xx1,, xx2,, ...... ,, xxN))
Negnevitsky, Pearson Education, 2002 5 StepStep 44:: CreateCreate aa newnew (offspring)(offspring) parameterparameter byby addingadding aa normallynormally distributeddistributed randomrandom variablevariable aa withwith meanmean zerozero andand pre-selectedpre-selected deviationdeviation δδ toto eacheach parentparent parameterparameter:: ′ = + ()δ xi xi a 0, ii == 1,1, 2,2, ...... ,, NN NormallyNormally distributeddistributed mutationsmutations withwith meanmean zerozero reflectreflect thethe naturalnatural processprocess ofof evolutionevolution (smaller(smaller changeschanges occuroccur moremore frequentlyfrequently thanthan largerlarger ones).ones).
StepStep 55:: CalculateCalculate thethe solutionsolution associatedassociated withwith thethe offspringoffspring parameters:parameters: ′ = ()′ ′ ′ X f x1, x2, . . . , xN
Negnevitsky, Pearson Education, 2002 6 StepStep 66:: CompareCompare thethe solutionsolution associatedassociated withwith thethe offspringoffspring parametersparameters withwith thethe oneone associatedassociated withwith thethe parentparent parameters.parameters. IfIf thethe solutionsolution forfor thethe offspringoffspring isis betterbetter thanthan thatthat forfor thethe parents,parents, replacereplace thethe parentparent populationpopulation withwith thethe offspringoffspring population.population. Otherwise,Otherwise, keepkeep thethe parentparent parameters.parameters.
StepStep 77:: GoGo toto StepStep 4,4, andand repeatrepeat thethe processprocess untiluntil aa satisfactorysatisfactory solutionsolution isis reached,reached, oror aa specifiedspecified numbernumber ofof generationsgenerations isis consideredconsidered..
Negnevitsky, Pearson Education, 2002 7 AnAn evolutionevolution strategystrategy reflectsreflects thethe naturenature ofof aa chromosome.chromosome. AA singlesingle genegene maymay simultaneouslysimultaneously affectaffect severalseveral characteristicscharacteristics ofof thethe livingliving organism.organism. OnOn thethe otherother hand,hand, aa singlesingle characteristiccharacteristic ofof anan individualindividual maymay bebe determineddetermined byby thethe simultaneoussimultaneous interactionsinteractions ofof severalseveral genes.genes. TheThe naturalnatural selectionselection actsacts onon aa collectioncollection ofof genes,genes, notnot onon aa singlesingle genegene inin isolation.isolation.
Negnevitsky, Pearson Education, 2002 8 GeneticGenetic programmingprogramming OneOne ofof thethe centralcentral problemsproblems inin computercomputer sciencescience isis howhow toto makemake computerscomputers solvesolve problemsproblems withoutwithout beingbeing explicitlyexplicitly programmedprogrammed toto dodo so.so. GeneticGenetic programmingprogramming offersoffers aa solutionsolution throughthrough thethe evolutionevolution ofof computercomputer programsprograms byby methodsmethods ofof naturalnatural selection.selection. InIn fact,fact, geneticgenetic programmingprogramming isis anan extensionextension ofof thethe conventionalconventional geneticgenetic algorithm,algorithm, butbut thethe goalgoal ofof geneticgenetic programmingprogramming isis notnot justjust toto evolveevolve aa bit-bit- stringstring representationrepresentation ofof somesome problemproblem butbut thethe computercomputer codecode thatthat solvessolves thethe problem.problem.
Negnevitsky, Pearson Education, 2002 9 GeneticGenetic programmingprogramming isis aa recentrecent developmentdevelopment inin thethe areaarea ofof evolutionaryevolutionary computation.computation. ItIt waswas greatlygreatly stimulatedstimulated inin thethe 1990s1990s byby JohnJohn KozaKoza.. AccordingAccording toto Koza,Koza, geneticgenetic programmingprogramming searchessearches thethe spacespace ofof possiblepossible computercomputer programsprograms forfor aa programprogram thatthat isis highlyhighly fitfit forfor solvingsolving thethe problemproblem atat hand.hand. AnyAny computercomputer programprogram isis aa sequencesequence ofof operationsoperations (functions)(functions) appliedapplied toto valuesvalues (arguments),(arguments), butbut differentdifferent programmingprogramming languageslanguages maymay includeinclude differentdifferent typestypes ofof statementsstatements andand operations,operations, andand havehave differentdifferent syntacticsyntactic restrictions.restrictions.
Negnevitsky, Pearson Education, 2002 10 SinceSince geneticgenetic programmingprogramming manipulatesmanipulates programsprograms byby applyingapplying geneticgenetic operators,operators, aa programmingprogramming languagelanguage shouldshould permitpermit aa computercomputer programprogram toto bebe manipulatedmanipulated asas datadata andand thethe newlynewly createdcreated datadata toto bebe executedexecuted asas aa program.program. ForFor thesethese reasons,reasons, LISPLISP waswas chosenchosen asas thethe mainmain languagelanguage forfor geneticgenetic programming.programming.
Negnevitsky, Pearson Education, 2002 11 LISPLISP structurestructure LISPLISP hashas aa highlyhighly symbol-orientedsymbol-oriented structure.structure. ItsIts basicbasic datadata structuresstructures areare atomsatoms andand listslists.. AnAn atomatom isis thethe smallestsmallest indivisibleindivisible elementelement ofof thethe LISPLISP syntax.syntax. TheThe numbernumber 2121,, thethe symbolsymbol XX andand thethe stringstring “This“This isis aa string”string” areare examplesexamples ofof LISPLISP atoms.atoms. AA listlist isis anan objectobject composedcomposed ofof atomsatoms and/orand/or otherother lists.lists. LISPLISP listslists areare writtenwritten asas anan orderedordered collectioncollection ofof itemsitems insideinside aa pairpair ofof parentheses.parentheses.
Negnevitsky, Pearson Education, 2002 12 LISPLISP structurestructure ForFor example,example, thethe listlist −− (( ((* AA B)B) C)C) callscalls forfor thethe applicationapplication ofof thethe subtractionsubtraction functionfunction ((−−)) toto twotwo arguments,arguments, namelynamely thethe listlist (*A(*A B)B) andand thethe atomatom C.C. First,First, LISPLISP appliesapplies thethe multiplicationmultiplication functionfunction (*)(*) toto thethe atomsatoms AA andand B.B. OnceOnce thethe listlist (*A(*A B)B) isis evaluated,evaluated, LISPLISP appliesapplies thethe subtractionsubtraction functionfunction ((−−)) toto thethe twotwo arguments,arguments, andand thusthus evaluatesevaluates thethe entireentire listlist −− (( ((* AA B)B) C).C).
Negnevitsky, Pearson Education, 2002 13 GraphicalGraphical representationrepresentation ofof LISPLISP S-expressionsS-expressions BothBoth atomsatoms andand listslists areare calledcalled symbolicsymbolic expressionsexpressions oror S-expressionsS-expressions.. InIn LISP,LISP, allall datadata andand allall programsprograms areare S-expressions.S-expressions. ThisThis givesgives LISPLISP thethe abilityability toto operateoperate onon programsprograms asas ifif theythey werewere data.data. InIn otherother words,words, LISPLISP programsprograms cancan modifymodify themselvesthemselves oror eveneven writewrite otherother LISPLISP programs.programs. ThisThis remarkableremarkable propertyproperty ofof LISPLISP makesmakes itit veryvery attractiveattractive forfor geneticgenetic programmingprogramming.. AnyAny LISPLISP S-expressionS-expression cancan bebe depicteddepicted asas aa rootedrooted point-labelledpoint-labelled treetree withwith orderedordered branches.branches.
Negnevitsky, Pearson Education, 2002 14 LISPLISP S-expressionS-expression ((−− (*A(*A B)B) C)C)
−
* C
A B
Negnevitsky, Pearson Education, 2002 15 HowHow dodo wewe applyapply geneticgenetic programmingprogramming toto aa problem?problem? BeforeBefore applyingapplying geneticgenetic programmingprogramming toto aa problem,problem, wewe mustmust accomplishaccomplish fivefive preparatorypreparatory stepssteps:: 1.1. DetermineDetermine thethe setset ofof terminals.terminals. 2.2. SelectSelect thethe setset ofof primitiveprimitive functions.functions. 3.3. DefineDefine thethe fitnessfitness function.function. 4.4. DecideDecide onon thethe parametersparameters forfor controllingcontrolling thethe run.run. 5.5. ChooseChoose thethe methodmethod forfor designatingdesignating aa resultresult ofof thethe runrun..
Negnevitsky, Pearson Education, 2002 16 TheThe PythagoreanPythagorean TheoremTheorem helpshelps usus toto illustrateillustrate thesethese preparatorypreparatory stepssteps andand demonstratedemonstrate thethe potentialpotential ofof geneticgenetic programming.programming. TheThe theoremtheorem sayssays thatthat thethe hypotenuse,hypotenuse, cc,, ofof aa rightright triangletriangle withwith shortshort sidessides aa andand bb isis givengiven byby
c = a2 + b2
TheThe aimaim ofof geneticgenetic programmingprogramming isis toto discoverdiscover aa programprogram thatthat matchesmatches thisthis function.function.
Negnevitsky, Pearson Education, 2002 17 ToTo measuremeasure thethe performanceperformance ofof thethe as-yet-as-yet- undiscoveredundiscovered computercomputer program,program, wewe willwill useuse aa numbernumber ofof differentdifferent fitnessfitness casescases.. TheThe fitnessfitness casescases forfor thethe PythagoreanPythagorean TheoremTheorem areare representedrepresented byby thethe samplessamples ofof rightright trianglestriangles inin Table.Table. TheseThese fitnessfitness casescases areare chosenchosen atat randomrandom overover aa rangerange ofof valuesvalues ofof variablesvariables aa andand bb..
Side a Side b Hypotenuse c Side a Side b Hypotenuse c 3 5 5.830952 12 10 15.620499 8 14 16.124515 21 6 21.840330 18 2 18.110770 7 4 8.062258 32 11 33.837849 16 24 28.844410 4 3 5.000000 2 9 9.219545
Negnevitsky, Pearson Education, 2002 18 StepStep 11:: DetermineDetermine thethe setset ofof terminals.terminals. TheThe terminalsterminals correspondcorrespond toto thethe inputsinputs ofof thethe computercomputer programprogram toto bebe discovered.discovered. OurOur programprogram takestakes twotwo inputs,inputs, aa andand bb.. StepStep 22:: SelectSelect thethe setset ofof primitiveprimitive functions.functions. TheThe functionsfunctions cancan bebe presentedpresented byby standardstandard arithmeticarithmetic operations,operations, standardstandard programmingprogramming operations,operations, standardstandard mathematicalmathematical functions,functions, logicallogical functionsfunctions oror domain-specificdomain-specific functions.functions. OurOur programprogram willwill useuse fourfour standardstandard arithmeticarithmetic operationsoperations +,+, −−,, ** andand //,, andand oneone mathematicalmathematical functionfunction sqrtsqrt..
Negnevitsky, Pearson Education, 2002 19 StepStep 33:: DefineDefine thethe fitnessfitness function.function. AA fitnessfitness functionfunction evaluatesevaluates howhow wellwell aa particularparticular computercomputer programprogram cancan solvesolve thethe problem.problem. ForFor ourour problem,problem, thethe fitnessfitness ofof thethe computercomputer programprogram cancan bebe measuredmeasured byby thethe errorerror betweenbetween thethe actualactual resultresult producedproduced byby thethe programprogram andand thethe correctcorrect resultresult givengiven byby thethe fitnessfitness case.case. Typically,Typically, thethe errorerror isis notnot measuredmeasured overover justjust oneone fitnessfitness case,case, butbut insteadinstead calculatedcalculated asas aa sumsum ofof thethe absoluteabsolute errorserrors overover aa numbernumber ofof fitnessfitness cases.cases. TheThe closercloser thisthis sumsum isis toto zero,zero, thethe betterbetter thethe computercomputer program.program.
Negnevitsky, Pearson Education, 2002 20 StepStep 44:: DecideDecide onon thethe parametersparameters forfor controllingcontrolling thethe run.run. ForFor controllingcontrolling aa run,run, geneticgenetic programmingprogramming usesuses thethe samesame primaryprimary parametersparameters asas thosethose usedused forfor GAs.GAs. TheyThey includeinclude thethe populationpopulation sizesize andand thethe maximummaximum numbernumber ofof generationsgenerations toto bebe run.run.
StepStep 55:: ChooseChoose thethe methodmethod forfor designatingdesignating aa resultresult ofof thethe run.run. ItIt isis commoncommon practicepractice inin geneticgenetic programmingprogramming toto designatedesignate thethe best-so-farbest-so-far generatedgenerated programprogram asas thethe resultresult ofof aa run.run.
Negnevitsky, Pearson Education, 2002 21 OnceOnce thesethese fivefive stepssteps areare complete,complete, aa runrun cancan bebe made.made. TheThe runrun ofof geneticgenetic programmingprogramming startsstarts withwith aa randomrandom generationgeneration ofof anan initialinitial populationpopulation ofof computercomputer programs.programs. EachEach programprogram isis composedcomposed ofof −− // functionsfunctions +,+, ,, *,, andand sqrtsqrt,, andand terminalsterminals aa andand bb..
InIn thethe initialinitial population,population, allall computercomputer programsprograms usuallyusually havehave poorpoor fitness,fitness, butbut somesome individualsindividuals areare moremore fitfit thanthan others.others. JustJust asas aa fitterfitter chromosomechromosome isis moremore likelylikely toto bebe selectedselected forfor reproduction,reproduction, soso aa fitterfitter computercomputer programprogram isis moremore likelylikely toto survivesurvive byby copyingcopying itselfitself intointo thethe nextnext generation.generation.
Negnevitsky, Pearson Education, 2002 22 CrossoverCrossover inin geneticgenetic programming:programming: TwoTwo parentalparental S-expressionsS-expressions
/ +
− * − sqrt
sqrt a a b sqrt b /
+ − a b
− a * *
a a a b b b
(/ (− (sqrt (+ (* a a) (− a b))) a) (* a b)) (+ (− (sqrt (− (* b b) a)) b) (sqrt (/ a b)))
Negnevitsky, Pearson Education, 2002 23 CrossoverCrossover inin geneticgenetic programming:programming: TwoTwo offspringoffspring S-expressionsS-expressions / +
− sqrt − sqrt
sqrt a − * b /
+ a a * b a b
− * b b
a a a b
(/ (− (sqrt (+ (* a a) (− a b))) a) (sqrt (− (* b b) a))) (+ (−(* a b) b) (sqrt (/ a b)))
Negnevitsky, Pearson Education, 2002 24 MutationMutation inin geneticgenetic programmingprogramming AA mutationmutation operatoroperator cancan randomlyrandomly changechange anyany functionfunction oror anyany terminalterminal inin thethe LISPLISP S-expression.S-expression. UnderUnder mutation,mutation, aa functionfunction cancan onlyonly bebe replacedreplaced byby aa functionfunction andand aa terminalterminal cancan onlyonly bebe replacedreplaced byby aa terminal.terminal.
Negnevitsky, Pearson Education, 2002 25 MutationMutation inin geneticgenetic programming:programming: OriginalOriginal S-expressionsS-expressions
/ +
✖ − * − sqrt
sqrt a a b sqrt b ✖ /
+ − a b
− * * a
a a a b b b
(/ (− (sqrt (+ (* a a) (− a b))) a) (* a b)) (+ (− (sqrt (− (* b b) a)) b) (sqrt (/ a b)))
Negnevitsky, Pearson Education, 2002 26 MutationMutation inin geneticgenetic programming:programming: MutatedMutated S-expressionsS-expressions
/ +
+ * − sqrt
sqrt a a b sqrt a /
+ − a b
− * * a
a a a b b b
(/ (+ (sqrt (+ (* a a) (− a b))) a) (* a b)) (+ (− (sqrt (− (* b b) a)) a) (sqrt (/ a b)))
Negnevitsky, Pearson Education, 2002 27 InIn summary,summary, geneticgenetic programmingprogramming createscreates computercomputer programsprograms byby executingexecuting thethe followingfollowing steps:steps: StepStep 11:: AssignAssign thethe maximummaximum numbernumber ofof generationsgenerations toto bebe runrun andand probabilitiesprobabilities forfor cloning,cloning, crossovercrossover andand mutation.mutation. NoteNote thatthat thethe sumsum ofof thethe probabilityprobability ofof cloning,cloning, thethe probabilityprobability ofof crossovercrossover andand thethe probabilityprobability ofof mutationmutation mustmust bebe equalequal toto one.one. StepStep 22:: GGenerateenerate anan initialinitial populationpopulation ofof computercomputer programsprograms ofof sizesize NN byby combiningcombining randomlyrandomly selectedselected functionsfunctions andand terminals.terminals.
Negnevitsky, Pearson Education, 2002 28 StepStep 33:: ExecuteExecute eacheach computercomputer programprogram inin thethe populationpopulation andand calculatecalculate itsits fitnessfitness withwith anan appropriateappropriate fitnessfitness function.function. DesignateDesignate thethe best-best- so-farso-far individualindividual asas thethe resultresult ofof thethe run.run. StepStep 44:: WithWith thethe assignedassigned probabilities,probabilities, selectselect aa geneticgenetic operatoroperator toto performperform cloning,cloning, crossovercrossover oror mutation.mutation.
Negnevitsky, Pearson Education, 2002 29 StepStep 55:: IfIf thethe cloningcloning operatoroperator isis chosen,chosen, selectselect oneone computercomputer programprogram fromfrom thethe currentcurrent populationpopulation ofof programsprograms andand copycopy itit intointo aa newnew population.population. •• IfIf thethe crossovercrossover operatoroperator isis chosen,chosen, selectselect aa pairpair ofof computercomputer programsprograms fromfrom thethe currentcurrent population,population, createcreate aa pairpair ofof offspringoffspring programsprograms andand placeplace themthem intointo thethe newnew population.population. •• IfIf thethe mutationmutation operatoroperator isis chosen,chosen, selectselect oneone computercomputer programprogram fromfrom thethe currentcurrent population,population, performperform mutationmutation andand placeplace thethe mutantmutant intointo thethe newnew population.population.
Negnevitsky, Pearson Education, 2002 30 StepStep 66:: RepeatRepeat StepStep 44 untiluntil thethe sizesize ofof thethe newnew populationpopulation ofof computercomputer programsprograms becomesbecomes equalequal toto thethe sizesize ofof thethe initialinitial population,population, NN.. StepStep 77:: ReplaceReplace thethe currentcurrent (parent)(parent) populationpopulation withwith thethe newnew (offspring)(offspring) population.population.
StepStep 88:: GoGo toto StepStep 33 andand repeatrepeat thethe processprocess untiluntil thethe terminationtermination criterioncriterion isis satisfied.satisfied.
Negnevitsky, Pearson Education, 2002 31 FitnessFitness historyhistory ofof thethe bestbest S-expressionS-expression
100 sqrt 80
60 +
40
F i t n e s s, % s i t n e F s, * * 20 a a bb 0 0 1 234 G e n e r a t i o n s B e s t o f g e n e r a t i o n
Negnevitsky, Pearson Education, 2002 32 WhatWhat areare thethe mainmain advantagesadvantages ofof geneticgenetic programmingprogramming comparedcompared toto geneticgenetic algorithms?algorithms? GeneticGenetic programmingprogramming appliesapplies thethe samesame evolutionaryevolutionary approach.approach. However,However, geneticgenetic programmingprogramming isis nono longerlonger breedingbreeding bitbit stringsstrings thatthat representrepresent codedcoded solutionssolutions butbut completecomplete computercomputer programsprograms thatthat solvesolve aa particularparticular problem.problem. TheThe fundamentalfundamental difficultydifficulty ofof GAsGAs lieslies inin thethe problemproblem representation,representation, thatthat is,is, inin thethe fixed-lengthfixed-length coding.coding. AA poorpoor representationrepresentation limitslimits thethe powerpower ofof aa GA,GA, andand eveneven worse,worse, maymay leadlead toto aa falsefalse solution.solution.
Negnevitsky, Pearson Education, 2002 33 AA fixed-lengthfixed-length codingcoding isis ratherrather artificial.artificial. AsAs itit cannotcannot provideprovide aa dynamicdynamic variabilityvariability inin length,length, suchsuch aa codingcoding oftenoften causescauses considerableconsiderable redundancyredundancy andand reducesreduces thethe efficiencyefficiency ofof geneticgenetic search.search. InIn contrast,contrast, geneticgenetic programmingprogramming usesuses high-levelhigh-level buildingbuilding blocksblocks ofof variablevariable length.length. TheirTheir sizesize andand complexitycomplexity cancan changechange duringduring breeding.breeding. GeneticGenetic programmingprogramming worksworks wellwell inin aa largelarge numbernumber ofof differentdifferent casescases andand hashas manymany potentialpotential applicationsapplications..
Negnevitsky, Pearson Education, 2002 34