Hižak, J.; Logožar, R. Prikaz genetičkog algoritma i njegova uporaba za nalaženje ekstrema — s implementacijama u MATLABU

ISSN 1846-6168 (Print), ISSN 1848-5588 (Online) ID: TG-20160212192336

AN OVERVIEW OF THE GENETIC AND ITS USE FOR FINDING EXTREMA ─ WITH IMPLEMENTATIONS IN MATLAB

PRIKAZ GENETIČKOG ALGORITMA I NJEGOVA UPORABA ZA NALAŽENJE EKSTREMA — S IMPLEMENTACIJAMA U MATLABU

Jurica Hižak, Robert Logožar

Original scientific paper Abstract: The paper outlines the main concepts of the genetic algorithm (GA) in a combined, educational-scientific style. Every step of the GA is first motivated by its biological paragon, then mathematically formalized and explained on simple examples, and finally supported by implementations in MATLAB. Two programs that use GA for the illustra- tive cases of finding functions’ extrema are shown. The authors conclude the paper by presenting the original use of GA in the Iterated Prisoner Dilemma, which gave a new insight into this problem.

Keywords: genetic algorithm, function, function extrema, stochastic iterative prisoner dilemma.

Izvoran znanstveni rad Sažetak: Ovaj rad predstavlja glavne koncepte genetičkog algoritma (GA) u kombiniranom, edukativno-znanstvenom stilu. Svaki je korak GA isprva motiviran svojim biološkim uzorom, potom matematički formaliziran i objašnjen na jednostavnim primjerima te konačno potkrijepljen primjerima u MATLABU. Predstavljena su dva programa koja koris- te GA za ilustrativne slučajeve nalaženja ekstrema funkcija. Autori zaključuju članak prikazom izvorne uporabe GA u stohastičkoj iterativnoj dilemi zatvorenika, što je dalo novi pogled na ovaj problem.

Ključne riječi: genetički algoritam, funkcija podobnosti, ekstremi funkcija, stohastička iterativna dilema zatvorenika.

1. INTRODUCTION of all of them is the same: given a population of solu- tions, high-quality solutions are more likely to be select-

The concept of is omnipresent not only in na- ed during the process of selection and reproduction. [2] ture and natural sciences, but can be also found in all Thus, the basic idea and even the terminology of evo- other human activities. A good general definition of this lutionary follow the actions of the natural notion, with explication in biology is found in [1]: “Evo- evolution. Of course, one might expect that there are also lution is, in effect, a method of searching among an several technical differences between the two, which may enormous number of possibilities for ‘solutions’. In biol- depend on the particular field of the evolutionary compu- ogy, an enormous set of possibilities is the set of possible ting. Nevertheless, the fundamental functioning of all genetic sequences, and the desired ‘solutions’ are highly evolutionary algorithms can be grasped through the fol- fit organisms—organisms which are capable of surviving lowing brief insight into the genetic algorithm. and reproducing in their environments.” For a given problem, the reduced set of potential so- lutions is generated in a random fashion. Every solution, 1.1. Genetics in computation or individual—also called (artificial) chromosome— contains artificial . Unlike the real chromosomes, In computing (), an evolutionary algo- which consists of the DNAs built of series of pairs of the rithm investigates the space of feasible solutions of some four nucleotide bases (A-T, C-G, T-A, G-C), the artificial quantitative problem and—according to some prescribed chromosomes are simple sequences of zeros and ones.1 criteria—finds the best among the selected ones. This process is then iterated until one of the obtained solutions 1 Since the nature is the inspiration for this whole area of is “good enough.” Therefore, a computational procedure computing, here we give a short explication of how the coding in an is a process analogous to is done in the DNA. It should be stressed that this coding is not what is happening to live organisms in the natural evolu- nearly as straightforward as for the artificial chromosomes. tion. The evolutionary algorithms are studied within the Furthermore, it is highly adjusted to the concrete function it has field of evolutionary computing. ─ to store the information on how ribosomes will form proteins Nowadays the evolutionary computing comprehends from amino-acids. That information is translated by the mes- three big scientific fields: genetic algorithms, evolution- senger RNA. The RNA is a copy of the DNA structure, with the ary strategies, and . There are many only difference that it has the T (Timine) base replaced with the U (Uracil) base. Of the two helixes, only one—the coding different evolutionary algorithms, as well as several strand—is enough to define the information held in the RNA variations of each of them. However, the underlying idea (the other is determined by the pairwise complements, as in the

Tehnički glasnik 10, 3-4(2016), 55-70 55 An overview of the genetic algorithm and its use for finding extrema ─ with implementations in MATLAB Hižak, J.; Logožar, R.

From every generation of chromosomes, the best ones theoretical presentation is made in an educational, easy- are chosen. And while the best individuals in biological to-follow approach, but mathematically corroborated populations are selected by complex biological and envi- where needed. For the readers who are already familiar ronmental mechanisms, the selection process in genetic with the field of on a general algorithms is governed by the evaluation of the observed level, a few illustrative examples written in MATLAB population by a . It assigns a fitness value can make this article interesting as a step further toward to every individual in the population. Then, the selection the practical use of genetic algorithms — possibly also for of the best individuals (chromosomes) is done in a sto- their own problems. On the other hand, for the huge chastic process, in which the probability of selection of group of MATLAB users who are novices to the field, an individual is proportional to its fitness value. So, alt- this could be a motivation to learn the topic that was hough better individuals have a greater chance of being mostly being connected to computer science. The final selected, their selection is not guaranteed. Even the worst aim of this endeavor was to show that the genetic algo- individuals have a slight chance for survival. [2] rithm could be effectively programmed in MATLAB, and The stochasticity of the procedure is further enhanced to describe how this was done in another authors’ work, by the processes of recombination and mutation. Their within the field of the . action ensures the variability of solutions, and thus con- After the broader outline of the concepts used in evo- tributes to the exploration of the whole search or state lutionary computation and the genetic algorithm was space (the set of all possible solutions). This is essential already given in this introduction, in section 2 we give for avoiding the traps of local extrema, in which tradi- the basic ideas that underlie the GA more precisely. tional algorithms get stuck quite often. In section 3 we deal with the creation of individuals Altogether, the combined action of the variation and for the initial population. We spend some time to formal- selection operators results in a very efficient method for ize the problem mathematically in a precise and complete improving every next generation of solutions and rela- manner. However, to keep the reading simple and inter- tively fast convergence to the desired optimal solution. esting for the broader audience, we provide several sim- That is why the evolutionary algorithms are very popular ple examples. And along with them, the formalized steps in solving a great variety of problems in both applied and are immediately implemented in MATLAB. fundamental sciences, such as finding extrema of com- Section 4 formalizes and then exemplifies the process plex functions, discovering the best game strategies, of selection of individuals for the new population (gener- solving transport problems and modeling complex shape ation) and discusses the fitness function. The roulette 2 objects. Among those are also the hardest computational wheel parent selection was illustrated on a simple exam- problems, which belong to the so-called NP ple and implemented in MATLAB. In section 5 we ex- 3 class. Despite some limitations and deficiencies, the plain the recombination and mutation, and, as always, evolutionary algorithms tackle all these tough tasks very cover the theoretical part with examples and the successfully, thanks to their changed paradigm of opera- MATLAB code. Then, in section 6 we discuss the behav- tion and the loosened aim. That is, if the accurate solu- ior of the GA in finding extrema for two illustrative func- tion is so hard-to-find, why not search for a solution that tions. The full MATLAB codes for these programs are is just “good enough.” given in Appendix A. In section 7, we discuss the use of GA in the iterated 1.2. The purpose and the plan of the paper prisoner dilemma problem and show how it can discover unexpected aspects of such nontrivial systems. It should In this paper, we shall focus our attention to the genetic be—as we see it—the main scientific novelty of this algorithm (GA). We shall describe the GA in comparison work. Finally, section 8 concludes this paper. to the corresponding biological processes and give the formalization of its basic steps. Then we shall implement these steps in the widely used mathematical and engi- ® 4 2. OUTLINE OF THE GENETIC ALGORITHM neering programming environment MATLAB . The

The genetic algorithm was invented by John Holland in DNA). In the coding strand, the minimal complete biochemical 1975. [3] In order to find the function extrema, Holland information is stored in the three-letter words called codons, presented each potential solution (extremum) as a se- formed by the letters A, U, C, G. There are 4 = 64 such quence of bits, called the artificial chromosome. The bits, words. The sequences of these words convey the3 information containing zeros and ones, or groups of several bits, can necessary for the formation of proteins, which is the basis for be considered as artificial, binary genes (Figure 1, confer the existence and reproduction of all living organisms. [11] also footnote number 1). 2 The shape of NASA ST5 spacecraft antenna was modeled by using GA to create the best radiation pattern. [12] 3 NP stands for nondeterministic, polynomial time, meaning that the solution of the NP problem class can be found by the nondeterministic Turing machine (NTM) in the polynomial (favorable) time. The NTM is a theoretical machine that can guess the answers of the hard problems in the polynomial time, even though they require exponential time on normal, determin- istic machines (algorithms). A simple comment and a connec- Figure 1. The population of artificial chromosomes, with tion of the NP problems to the GA can be found in [9]. genes presented by bits. 4 MATLAB® is a trademark of the MathWorks Inc.

56 Technical Journal 10, 3-4(2016), 55-70 Hižak, J.; Logožar, R. Prikaz genetičkog algoritma i njegova uporaba za nalaženje ekstrema — s implementacijama u MATLABU

Obviously, an artificial chromosome can be any form Algorithm 1. A version of the genetic algorithm (GA). of data that properly describes the features of individuals within some population. Some of the individuals, i.e. Input: original function, search space (the set of all indi- chromosomes—as we shall simply call them in the fur- vidual solutions, or chromosomes), fitness function. ther text—would eventually appear as the solution of the Optional input: good enough solution, from which the given problem. good enough fitness value is calculated. 1. Create initial population by randomly selecting a 2.1. Genetic algorithm certain number of chromosomes from the search space; 2. Evaluate the fitness of the current population by calcu- A pseudocode of a common version of the GA is given in lating every chromosome’s fitness value. Algorithm 1. The input is clear from the previous text. The optional input enables the calculation of the satisfy- 3. While the termination criteria are not met, do: [Terminat. criteria: the best fitness value is good enough ing fitness value that can serve as one of the termination or the specified number of iterations is done, or …] criteria (discussed more in the next subsection). In step 1, the initial population of solutions (or indi- 3.1 While the new population is not completed, do: viduals, or chromosomes) is created by random selection 3.1.1 Select two parent chromosomes by random from the whole search space. This is elaborated in more selection from the previous population; details in section 3. In step 2, the initial population is 3.1.2 Recombine the parent chromosomes to ob- evaluated by the fitness function, i.e. each chromosome tain two children; of the population is assigned its fitness value (more on 3.1.3 Enlist the two children in the new population; the fitness function is given in §4.1 and §4.2). 3.2 Mutate the chromosomes in the new population; The iterative process that follows is presented by the two nested while loops. The condition of the outer loop is 3.3 Replace the previous population with the new one; the negation of the termination criteria, which are ex- 3.4 Evaluate the fitness of the current population (0.2). plained in §2.2. For now, we assume that the criteria are Output: the best chromosome found (the one with the not met so that the algorithm enters the outer loop. highest fitness value). Within the outer loop, in step 3.1, the GA enters an- other while-loop, which governs the selection of individ- uals for the next population. Within the nested loop, in 2.2. Termination criteria step 3.1.1, a pair of chromosomes is selected randomly from the previous population (generation). The probabil- If the fitness value of at least one chromosome satisfies ity for this reselection is proportional to the relative fit- some “prescribed criterion,” the algorithm is finished. If ness values of the chromosomes (see §4.3), i.e. the more not, the algorithm enters the next iteration. probable chromosomes will be reselected more likely. In A careful reader will probably note that finding and general, some of them can be selected more than once, specifying such a criterion pose a fundamental problem and some of them will not survive the selection process not only for GA but also for all other evolutionary algo- and will not appear in the next generation. In step 3.1.2, rithms. Namely, GA does not investigate the search space the chromosomes from the selected parent pair are sub- thoroughly, methodologically, or in any “systematic” jected to the recombination operator, which first splits way. So, there are no sure indicators of when the compu- them, and then mix the split parts in a specified way. tation is over. For instance, if the GA is searching for the Normally, every pair of parents will produce two de- unknown global extrema of some function, it cannot scendants, so that every new generation of offspring possibly know how far it is from them. Also, if we didn’t contains the same number of individuals as the previous search the whole search space, we cannot be sure that generation. In step 3.1.3, the two new children are enlist- somewhere there isn’t a more extremal value. ed as the new chromosomes of the next population. The Obviously, in the spirit of our unsystematic and sto- process is iterated until the new population is filled with chastically governed algorithm, we need a similar kind of the desired number of chromosomes. When this is done, termination criteria, i.e. (i) something loose but practical, the while-loop 3.1 is completed. or (ii) something heuristically based. In step 3.2, the descendant chromosomes are exposed As for (i), sometimes we might be given a hint of to mutation. Depending on the chosen mutation rate, the what would be a “desired” or “good enough” solution mutation changes a certain number of genes in the popu- (e.g. from the knowledge about the observed problem lation’s pool (for further explanations see §5.4). gathered by other means, or by the specifications given The processes of mutation and recombination are by a client or by a boss). By knowing it, the good enough usually described by the action of the recombination and fitness value could be calculated (opt. input) and this mutation operator, respectively, commonly known as the would be our first criterion. However, although our client variation operators (more about them in section 5). or boss could be satisfied, we as scientists may still won- In step 3.3, the old population is replaced with the der how good our solution is and if it can be made better. new one and the list of children is deleted. In step 3.4, the This will be discussed a bit more in a moment. new population is evaluated by the fitness function, And if the good enough solution is not known, then which is the same action as in step 2 (the duplication of the “ criterion” (ii) boils down to: “stop the itera- these steps is a sacrifice to achieve the structured, while tion after a certain number of iterations,” or “check the loop). Now the first iteration of the outer loop is finished, trends of certain indicators and decide about the termina- and the termination criteria are checked again. tion.” The indicator most used is the population average

Tehnički glasnik 10, 3-4(2016), 55-70 57 An overview of the genetic algorithm and its use for finding extrema ─ with implementations in MATLAB Hižak, J.; Logožar, R. fitness. By tracking it in each iteration, one can draw where is the set’s low and its high limit, and is some conclusions on how the GA is progressing. If the the measure of the set’s resolution, , , . Ob- 푙 ℎ indicator is stalling in its “extremal plateau” for several viously,푥 is the set of all possible푥 solutions, i.e.∆ 푥the 푙 ℎ iterations, the algorithm can be halted. However, this is search space of the problem. The extrema푥 푥 of∆ 푥(∈)ℝ on this by no means a sure decision. Namely, although one can discrete set푋 must be achieved for one of its elements, expect that with every new iteration of the GA the popu- = , where: 푓 푥 lations “evolve,” i.e. that their average fitness values 푁푋 푒푒푒푒 = card( ) = + 1 . (2) improve and get closer to a desired extremum, this can- 푥 �ℎ−�푙 not be taken for granted. The GA’s method of searching Effectively, we have푁푋 provided푋 a discrete∆� representation of is stochastic and it often results in unpredictable behavior some closed real interval [ , ], as is always the case of the chosen indicators. For instance, the average fitness when numerical methods are used on digital computers. 푙 ℎ value of the subsequent populations may stagnate for This is valid regardless of푥 the푥 details of the number many iterations, making us believe that it is stable and presentation, i.e. it does not matter if they are coded as without a chance for improvements. But after such inter- integers or floating point numbers.5 So, taking into ac- vals of stability, GAs can suddenly show the solutions count the eq. (2), we can choose the -bit unsigned inte- that are worse than previous, or they can surprise us with ger arithmetic without essential loss of generality. For it, sudden improvements. This will be illustrated in §6. the (whole) numbers span from 0 to푏 2 1. Thus, the Further discussion of the possible indicators for the achieved resolution is 푏 − termination of the GA and their behavior are beyond the = , (3a) scope of this paper. We can simply say that the GA can ∆푥 2 1 푥ℎ − 푥푙 be run for several times with a different number of itera- and an arbitrary from∆푥 the search푏 space can be pre- tions or, alternatively, started with different initial popu- sented as the -bit number : − lations. Then the obtained results can be compared. 푥 푋 = = ( ) = ( ). (3b) 푏 푛 푏 −1 2 −1 2.3. Use of genetic algorithms Vice-versa,푛 푛the〈푏 〉 -bit∆ 푥number푥 − 푥푙 =�ℎ− represent�푙 푥 − 푥s푙

= + × = + As was already hinted at the end of §1.1, the power of 푏 푛〈푏〉 푛 . (3c) �ℎ−�푙 GA is in the great diversification of the intermediary 푏 The binary푥 푥presentation푙 ∆푥 푛 푥푙 can∆푥 2be− 1regarded푛 as a solutions, which ensure that the most of the search space vector,6 which comes as a nice formal point in the further will be properly investigated. This is essential for the treatment of chromosomes as푛 individual〈푏〉 solutions, and in functions with several extrema, the so-called multi-modal performing operations on them. functions, whose search space is multi-modal search Now, having established the general transformation space. For such functions GA does excel. On the other between the individual solution ( ) and its presentation hand, the classical methods—based on the analytical in some -bit arithmetic, the elaboration can be approach—will very often fail on them. They will tend to simplified even further. By taking푥 = 1, the search go straight toward the nearest local extremum. Another space푛〈푏〉 becomes푏 a subset of integers. Furthermore, by advantage is that the GA does not impose any restrictions taking = 0, the lowest individual ∆coincides푥 with the on the functions that they investigate. The functions need low limit of the unsigned arithmetic and the highest indi- not be continuous or analytic (derivable). 푙 vidual 푥coincides with the maximal number of the un-

signed arithmetic, = 2 1 , so that eq. (3c) gives 푏 = = . (3d) 3. SEARCH SPACE AND INITIAL POPULATION 푥ℎ − In an undersized example (for the sake of keeping the 〈 〉 In the outline of the genetic algorithm in §2, we have things simple), let us푥 choose푛 the푛 푏 8-bit integer arithmetic. already discussed the notion of the artificial chromo- It has the search space of 2 = 256 individuals , in the some. From there, it is clear that the chromosomes are a range 0 255. These8 solutions can be indexed in 푖 suitable representation of individual solutions within the an appropriate manner. Instead of the programming푥 -style 푖 search space of some problem. indexing,≤ starting푥 ≤ from 0, here we use the mathematical indexation to comply with the notation in the rest of the 3.1. Search space and individual solutions — paper. So the indices are for 1 greater than values, and their presentations and values the search space is = { , , … , }. Further indexa- tions of the individuals푖 within the populations derived 1 2 256 In order to illustrate the functioning of GA and the crea- from will be changed푋 according푥 푥 푥to the total number of tion of its initial population, we define a common task of the individuals in those populations (see §3.2). this field: 푋 Find the global extremum (minimum or maxi- 5 For instance, let us suppose that the interval of positive mum) of the function ( ) of discrete equidistant numbers [0, 1] is covered by the single precision floating point values . numbers. The maximal resolution achieved by this arithmetic is To formalize the problem푓 푥 a bit more, we specify that given by the minimal accurate across the whole interval. For 푥 = 1, it is the relative precision of this data type, belongs to the discrete set , 10 . For the double precision∆푥 arithmetic, 10 . ℎ6 { } − The7 chromosomes = are vectors in the −vector14 ∆푥 ≈ 푥 = , + 푋 , + 2 , … , , (1) 푥 space above the Galois filed GF(2 ). ∆푥 ≈ 푙 푙 푙 ℎ 푖 푖 푥 ∈ 푋 푥 푥 ∆푥 푥 ∆푥 푥 푥 푛 〈푏〉 푏 58 Technical Journal 10, 3-4(2016), 55-70 Hižak, J.; Logožar, R. Prikaz genetičkog algoritma i njegova uporaba za nalaženje ekstrema — s implementacijama u MATLABU

It turns out that the presentation of individuals in such that the following obvious restrictions would imply: the binary system is both illustrative and practical for (i) there are no repetitions of the same points in the se- 푖 their further evaluation and manipulation in the form푥 of quence, (ii) the total number of points in the sequence is artificial chromosomes in GAs. In our example, the bina- . ry presentation of the -th individual is given as: We end this formal part with a few notes on the actu- ≤ 푘 = 8 = … , , (4a) al sizes of the search spaces in the practical use of GAs. 푖 E.g., if the space of 32-bit integers is used as the search where are푖 binary푖 digits7 6, 1=00,푖1푏,�푏… , 1 = 7, 푥 푛 〈 〉 〈푑 푑 푑 푑 〉 space, the total of 4Gi 4 × 10 individual solutions {0,1}. The case of = (where we use the 푑푗 푗 푏 − (chromosomes) exist. For modern 9computers, searching above, “shifted” indexation), over such search space by≈ the “brute force” would be no 푑푗 ∈ 푥 푥27 = 8 = 00011010 , problem at all. With the computer performance of the is shown in Figure 2 (the subscript will be omitted order of 100 GIPS, and supposing that the function can 27 27〈 〉 푏�푏 further on where푥 the use푛 of binary numerals is obvious). be calculated with about 10 instructions, it would be 푏𝑏 done in the order of a second2 (here we assume that푓 the Since our simplified choice implies that = [confer eq. (3c)], we do not need to differentiate between program code is written and compiled optimally, and is an individual solution (chromosome) and its푥 binary푛〈푏〉 running much faster than the MATLAB code). However, presentation. And of course, if the value is attributed to for a two-variable function with the squared search space ( ) every such binary presentation (vector), our simplified [amounting to 2 = 16 Ei 2 × 10 ], the situation case leads back to the initial value of the푉 solution in the is drastically changed,32 2 and computing time19 would rise to ≈ search space: the order of 10 s. This and similar examples clearly show why the computer9 science ought to rely on efficient algorithms and not on the sheer power of computers. ( ) = 푏−1 × 2 = . (5a) 푗 In practical applications, the GA can be used for the 푉 푛푖〈푏〉 � 푑푗 푥푖 very large, exponentially growing search spaces, for This is so because the mapping푗=0 between and which the exhaustive searches are inefficient. If the fit- ( ) is a bijective function. ness function is not too complex for evaluation, it is now 푛푖〈푏〉 The (mathematical) indexation of individuals in the commonly accepted that the GA will produce satisfactory 푉 푛푖〈푏〉 8-bit search space chosen above, gives ( ) = 1. results in acceptable time for most of such problems. 푖〈 〉 푉 푛 푏 푖 − 3.2. Creation of the initial population From the defined search space , the initial population = is created by choosing a relatively small portion of the possible solutions from the푋 search space, 푋푙 푋0 = , , , , … , , . (8a)

As we have concluded in the previous0 subsection, the 푋0 �푥0 1 푥0 2 푥0 푁 � ∈ 푋 GA is in practice normally used on very large search Figure 2. Summary of the notation via an example: indi- spaces. Then the cardinality of the initial population is vidual solution, or chromosome = , its binary reduced from the cardinality of the whole search space presentation 8 in 8 bits, and the value, ( 8 ). for many orders of magnitude, which we write as 푥푖 푥 card( ) = card( ) = . (8b) In the case of푛〈 finding〉 extrema of a function,푉 푛〈 the〉 ap- It turns out that the efficiency of GA and the quality propriate choice of the fitness function would be either 푋0 푁0 ≪ 푋 푁푋 the absolute value as defined above or some simple, of its results do not improve much with enlargement of linear function of it (see §4). its population. A typical for 1-D problems is of the order of magnitude 10 to 10 . For the -bit chromo- To generalize the above deliberation, for the function 0 푁 of two variables, ( , ), the chromosome is defined by some presentations it is1 often suggested2 to take = (for more details see [9]). Once chosen—푏as it was al- the ordered pair , . In the binary presentation, it 0 ready stated in §2—the number = of individuals푁 is푏 would be written as푓: 푥 푦 푖 푗 normally kept throughout the iterations of the GA. �푥 푦 � 푙 0 , = … , … . (6) The selected number of 푁individuals푁 of the initial population will be chosen randomly from the total of If the 푖problem푗 is 푏of−1 different1 0 푖nature,푏−1 the 1search0 푗 space 0 �푥 푦 � �〈푑 푑 푑 〉 〈푑 푑 푑 〉 � possible solutions of the search푁 space. One way how to and its presentation should be changed appropriately. 푋 do it is described in the next subsection. 푁 E.g., if the task is to find the shortest path between the given points in a plane or some higher order space, then the chromosomes should be presented as sequences of 3.3. Definition of the search space and creation points in that space. For points , = 1,2, … , , and of the initial population in MATLAB the initial (final) point ( ), the search space X 푘 푇푙 푙 푘 Example 1. Let the first, simple task be to find the max- would be the set of all sequences푖푖푖푖 푓 𝑓 , ( ) = 36 푇푙 푇푙 imum of the function . For the sake of making the example as simple as possible,2 we observe = , , 푥, 푖… , , (7) the interval of integers푓 from푥 0 up푥 − to푥 and including 31, 푥푖 〈푇푙푖푖푖푖 푇푙1 푇푙2 푇푙푓𝑓〉 Tehnički glasnik 10, 3-4(2016), 55-70 59 An overview of the genetic algorithm and its use for finding extrema ─ with implementations in MATLAB Hižak, J.; Logožar, R.

Figure 3. Binary presentation of the set of integers from Figure 4. Another example of the 5-bit search space, 0 to 31 — which form the search space of Example 1. organized in the real interval [0,1], with = 1 31.

Code Listing 1. Creation of the population of eight (8) Code Listing 2. Conversion of the binary∆푥 presented⁄ 5-bit chromosomes in the MATLAB command prompt. chromosomes to their decimal values in MATLAB. The default orientation of binary numbers in MATLAB is reversed, with the most significant bit to the right.

The MATLAB works with matrices, so that it can find the decimal value of the whole set of chromosomes by a single command: which implies that = 0, = 31, = 1. The mini- popul_dec = xl + (M3) mal number of bits that will allow the coding of the cor- 푙 ℎ + bi2de(popul)*(xh-xl)/(2^nbts - 1) responding chromosomes푥 is푥 = 5 (see∆푥 Figure 3), leading to the search space 푏 = {00000, 00001, … ,11110,11111} . 4. SELECTION = 8 If we푋 want to create the initial population of After the creation of the initial population of solutions, chromosomes from , this can be done very easily in 0 the individual solutions in it must be evaluated to get the 푁 MATLAB. The simplest way is to create the 8 × 5 ma- estimation of how fit they are for reproduction. As was 푋 trix of random numbers in the range [0, 1) by using the already mentioned in §1 and §2, the evaluation is done function rand, and then to round them to 0s and 1s with by the fitness function, in literature also known as quali- the round function (Code Listing 1). ty, capability, goodness, or objective function. Getting back—from the binary numbers to their dec- imal values—is done by the MATLAB function bi2de. 4.1. Choice of the fitness function Thus, the binary presentation of the chromosome (= n) is translated to its decimal value (= x) by the follow- The fitness function maps the search space to the set of 〈 〉 ing statement (see also Code Listing 2): 푛 푏 real (rational) or integer nonnegative values, i.e. it as- 푥 signs the nonnegative퐹 goodness or fitness value ( ) to x = bi2de(n) (M1) each individual solution, or chromosome ,7 As a more general case, let us suppose that a search 퐹 푥푖 푖 space in the interval [0, 1] must be covered with a resolu- : ( ), : ( 푥) 0 . (10) tion of at least 0.05. From eq. (3a) it follows that + + 0 0 푖 푖 The bigger퐹 푋 →is ℝthe valueℤ 퐹(푥 )↦ the퐹 better푥 ≥ will be the

log + 1 , (9) ’s chance for survival and crossbreeding. �ℎ−�푙 퐹 푥푖 2 In some cases, the fitness function can be the same as leading to 4푏.392≥ , and� the∆� minimal� = 5. With 5 푥푖 bits, a slightly better is achieved, = 1 31 = the function whose extrema are being found. However, this is so only if the fitness values are of the same, posi- 0.3226 . This푏 is≥ shown in Figure 4. 푏 tive sign or zero for all the individual solutions. That is For this case, the scaling∆푥 from eq. (3c) ∆gives푥 the⁄ fol- the case of our first example, in §3.3, where the maxi- lowing MATLAB statement for conversion of the binary mum of ( ) = 36 is being sought. There, the presentation of the chromosome ( ) to its decimal = n greater fitness value belongs2 to the chromosome that is value (= x), 푓 푥 푥 − 푥 푛〈푏〉 closer to the parabola maximum, so that one can put

( ) = ( ) x =푥xl + bi2de(n)*(xh-xl)/(2^nbts - 1) (M2) simply . Here the rest of the variables are: xl = , xh = , 푥푖 푓 푥푖 nbts = . 7 푥푙 푥ℎ The reason for non-negativity will become clear soon. 푏 60 Technical Journal 10, 3-4(2016), 55-70 Hižak, J.; Logožar, R. Prikaz genetičkog algoritma i njegova uporaba za nalaženje ekstrema — s implementacijama u MATLABU

Otherwise, if the negative or both signs of the fitness global shift up can make the chromosomes’ fitness val- values appear when finding some function’s maximum, a ues very large positive numbers, and cause a compres- problem arises when the probabilities proportional to the sion of their relative differences. This will in turn make fitness values are to be assigned to the chromosomes. To the assigned probabilities less distinct. shift the fitness values to the positive side, a positive The solution is to stick to the first idea but to reduce constant should be added, so that in general ( ) = the pool from which the minimal or maximal value of ( ) + , 0 . ( ) is being found. In the -th iteration of the GA, the 푖 The similar problem arises when the minimum퐹 푥is be- natural choice is to find these values only among the 푖 푖 ing푓 푥 sought푐 푐for≥ predominantly positive values of the orig- chromosomes푓 푥 of the previous푙 population, which will be inal function ( ). Now the chromosomes closer to the shortly designated as , = 1, for = 1, 2, … minimal value of ( ) should be favored, meaning that … , , where is the total′ number of iterations. 푖 ( ) must be푓 proportional푥 to ( ) . Again, if ( ) Now, the eqs. (13) translate푙′ 푙 in푙to− the following푙 practi- 푖 푖𝑖 푖𝑖� produces some negative푓 푥 values, a constant must be add- cal 푁formulas for푁 : 푖 푖 푖 ed,퐹 푥 so that ( ) = ( ) + −, 푓 푥 0 . −푓 푥 Fnd. max. of ( ): = min , , , < 0 ; (14a) Altogether, this leads to the following criterion for 푐 푖 푖 Fnd. min. of ( 푖): = max 푙′ 푖 , 푙′ 푖 > 0 . (14b) the fitness function:퐹 푥 −푓 푥 푐 푐 ≥ 푓 푥 푐 � �푓�푥 , ��� 푓�푥 , � The fitness function must be such that its higher After the 푓above푥푖 푐deliberation,� �푓�푥 푙a′ 푖careful��� 푓�푥 푙reader′ 푖� proba- values—to which the higher probabilities are to be bly noted that the constant could be applied even if for assigned—are given to the chromosomes closer to finding maxima (minima) there are no negative (positive) the extremum that is being sought. values of ( ). In that case푐, would be negative, to shift the fitness values closer to zero and thus expand the relative The general form of the fitness functions for finding 푖 푓 푥 푐 extrema can now be summarized as follows: differences among the chromosomes. To achieve that, the absolute values around the minimum and maximum func- Finding max. of ( ): ( ) = ( ) + . (11a) tions and the following range limits should be simply re-

( ) ( ) ( ) moved. With that, the formulas for ( ), now without Finding min. of 푖 : 푖 = 푖 + . (11b) 푓 푥 퐹 푥 푓 푥 푐 restrictions, are: The choice of the constant will be discussed in the 푓 푥푖 퐹 푥푖 −푓 푥푖 푐 푐 ∈ ℝ ℤ next subsection. Fnd. max. of ( ): = min , ; (15a) 푐 For the travelling salesman problem, mentioned at the Fnd. min. of ( 푖): = max 푙′ ,푖 . (15b) end of the §3.1, the appropriate value attributed to the 푓 푥 푐 �푓�푥 �� Of course, if the achieved푖 expanding of the푙′ 푖 fitness values and its presentation , , , … , would be the 푓 푥 푐 �푓�푥 �� 푖 is not desirable, one should stick to the formulas (12). total length ( ) of the path defined by that presentation.푥 푙퐼𝐼� 푙1 푙2 푙퐼𝐼� Finally, the extra control over the expansion (com- Then one seeks the 〈solution푇 푇 푇 , for푇 which〉 is mini- 휆 pression) of the fitness function can be achieved by in- mal: 푚𝑚 , 푥푖 휆 troduction of the parameter that would multiply Finding min. of ( ): ( ) = ( ) + . (12) the ( ) and shrink it for 0 < <+ 1 and expand it for 훾 ∈ ℝ0 Further comments and푖 an example푖 of the푖 application > 1. With that factor, the formulas (11) become: 휆 푥 퐹 푥 −휆 푥 푐 푓 푥푖 훾 of GA to this problem can be found e.g. in [9, 10]. Finding max. of ( ): ( ) = ( ) + . (16a) 훾 Generally, the fitness function should be picked ap- Finding min. of ( ): ( ) = ( ) + . (16b) propriately to the problem. Furthermore, if the system’s 푓 푥푖 퐹 푥푖 훾� 푥푖 푐 푖 푖 푖 quality is described by a very complex function, the use 4.3. Population, average푓 푥 퐹 and푥 relative−훾� 푥 fitness푐 of the GA could be questionable. In that case, the corre- sponding fitness function could impose too high a burden values on the algorithm performance. In the -th iteration of GA, we calculate the fitness values

, of the individuals from the ( 1) = -th popu- 푙 8 4.2. The choice of (the problem of negative lation, , with generally individuals. From′ these, 푙′ 푖 fitness values) the퐹�푥 total� fitness of the population is푙 defined,− 푙 푙′ 푙′ 풄 푋 푁 ′ The role of the constant is to shift the original func- 푙 tion ( ) to the nonnegative values. In the case of find- ( ) = 푁푙′ , , (17) ing extrema of ( ) that has푐 negative values for some 푖 퐹 푋푙′ � 퐹�푥푙′ 푖� , the푓 푥ideal choice for would be: as well as the average fitness푖= 1of a chromosome within the 푖 푓 푥 population : Fnd. max. of ( ): = |min{ ( )}|, ( ) < 0 ; (13a) 푥푖 푐 ( ) 푙′ ( ) | { ( )}| ( ) Fnd. min. of 푖 : = max 푖 , 푖 > 0. (13b) 푋 , = = . (18) 푓 푥 푐 푓 푥 푓 푥 푙′ However, the min푖 { ( )} and max푖 { ( 푖)} are gener- ������푙′��횤� 푙′ 퐹 푋 푓 푥 푐 푓 푥 푓 푥 The ratio of the퐹� 푥 -th� chromosome퐹��� 푙 ′fitness and the total ally not known. They pose the optimization problem that 푁푋 푖 푖 is just the opposite of the푓 푥given one. 푓 푥 population fitness gives the relative fitness (goodness) of the chromosome. This푘 quantity will be chosen as the Another possibility to dissolve the problem of nega- probability for the crossbreeding of that chromosome, tive fitness values would be to make so big that it sure- ly lifts the ( ) into the first quadrant for all . 8 The numbers of chromosomes, , are kept population- 푐 However, quite often this will cause another problem — dependent for the sake of generality (see also the discussion at 푓 푥푖 푥푖 ∈ 푋 the loss of details in the original function. That is, a great the end of the subsection). 푁푙

Tehnički glasnik 10, 3-4(2016), 55-70 61 An overview of the genetic algorithm and its use for finding extrema ─ with implementations in MATLAB Hižak, J.; Logožar, R. and denoted the same way: 9 Table 1. Calculations prior the process of selection. For the chromosomes from the previous population, , = , 1 Pr , = = , . (19) , = , the table shows: 5 = 5-bit binary presenta- ( ) 푥푙−1 푖 퐹�푥푙′ 푘� , tion, ( ) = value, ( ) = fitness value, Pr( ) = 푙′ 푘 푁푙′ 푙′ 푘 0 푖 푖 푖〈 〉 �푥 � 푙′ 퐹�푥 � 푥 probability푥 of selection푛 in the next population, Having defined퐹 the푋 probabilities푖=1 of푙′ 푖the chromosomes 푖 푖 푖 ∑ 퐹�푥 � 푉 푛Pr( ) 퐹 푥 푥 in the previous population, the set of chromosomes for = cumulative probability. ( ) ( ) ( ) the next, ( + 1) = -th population is randomly selected �5 푖 Pr Pr ( ) 푁푙 푥 as described in the step 3.1 of Algorithm 1. It should be 푖 01000푖 푖8 224푖 0,128푖 12.8%푖 푙′ 푙 푥 푛 〈 〉 푉 푛 퐹 푥 푥 �′ 푥 noted that this could produce quite a different set of new 11110 30 180 0,103 23.1% individuals. Some of the previous chromosomes (pre- 푥1 01010 10 260 0,148 37.9% sumably the very probably ones) can appear more than 푥2 00001 1 35 0,020 39.9% once, and some of them can disappear (presumably those 푥3 11010 26 260 0,148 54.7% with small probabilities). 푥4 00101 5 155 0,088 63.5% We end this subsection with a comment why it could 푥5 be useful to keep the number of chromosomes gen- 10001 17 323 0,185 82.0% 푥6 eral, despite it is standardly fixed to = in all itera- 10101 21 315 0,180 100.0% 푙 푥7 tions (as stated in §3.2). Namely,푁 although it is known : 푙 0 8 1752 1.000 that the GA’s efficiency and quality of푁 results푁 is not very 푥 dependent푙 on the population cardinality, one might resort 훴 to changing it in some occasions. For instance, if the indirect termination indicators stall for several iterations, one may consider increasing to (re)check the stability of obtained solutions. This idea could be implemented in 푙 some enhanced versions of GA.푁

4.4. The example and the implementation of the

selection in MATLAB Figure 5. The roulette wheel parent selection for We continue our example from §3.3, for finding the Example 1. To each of the 8 chromosomes of the initial maximum of ( ) = 36 . The initial population population, a surface proportional to the chromosome probability is attributed. ( ) created there consists of 2 = 8 chromosomes. Now we proceed푓 푥 to the푥 − next푥 (first) iteration of the 푋0 푁0 In our example, the probability distribution Pr is de- GA. We calculate the fitness values of the previous popu- fined in Table 1, lation by applying the fitness function , = , , Pr = [0.128, 0.103, 0.148, 0.020, (M3) because , 0 for each , , and also for all . 0 푖 0 푖 0.148, 0.088, 0.185, 0.180] The results are shown in Table 1, with 퐹somewhat�푥 � 푓simpl�푥 �i- 푓�푥0 푖� ≥ 푥0 푖 푥푖 from which the cumulative probabilities are obtained by fied notation: , , , = ( ). E.g., for = 01000, ( ) = [ (01000)] = (8) = 224. CumPr = cumsum(Pr) (M4) 푥0 푖 ↦ 푥푖 퐹�푥0 푖� 퐹 푥푖 The total fitness of the initial population is ( ) = 푥1 퐹 푥1 푓 푉 푓 That gives the cumulative probability distribution: 1752 , so that the relative fitness, and in the same time 0 the probability for selection of the chromosome 퐹 , 푋is CumPr = [0.128, 0.231, 0.379, 0.399, (M5) 0.547, 0.635, 0.820, 1.000] , , 1 Pr( ) = == = 0.128푥 ( ) . Now, a random number from the interval [0, 1] is 퐹��0 푘� 퐹��0 푘�, 224 푁 generated; in MATLAB, by function . The value of 1 0 0 rand On 푥the basis퐹 푋 of the∑푖= 1 퐹chromosome��0 푖� 1752 ≅probabilities the random number determines the position and the sec- ( ), = 1,2, … ,8 , listed in Table 1, two new chromo- tor on the imaginary roulette, and thus also the belonging somes will be selected to form a pair of parents (step 푖 chromosome (the random number is compared with the 3.1.1푃푃 푥 in 푖Algorithm 1), and this will be—together with intervals defined by the CumPr, and translated accord- other actions—iterated four times (within the while-loop ingly into the index of the corresponding chromosome). 3.1 in Algorithm 1). The MATLAB function that does that (taken from The standard procedure for random choice is roulette [10]) is shown in Code Listing 3. Given the probability wheel parent selection. It is illustrated in Figure 5 as a distribution ), the function roulette on which each chromosome has its own sector, P (= Pr RandChoseN(P,N) returns the indices of N selected chromosomes. At the with the area percentage proportional to the chromo- beginning of the function, the bin edges are defined from some’s probability. For the right simulation of the chro- by the function, and the vector is mosome sectors, the cumulative probabilities must be P cumsum roulette zeroed. In the -loop, the random number is creat- calculated from the chromosomes’ probability distribu- for x ed. Then the histogram of its incidences (only one!) in tion. In MATLAB, this is done by the cumsum function. the bins is made by function histc and stored in . The index of with the one (and only) 9 Of course, the relative fitness qualifies for a probability Counts Counts hit is stored in roulette. The loop is repeated N times. function because 0 Pr , 1 and Pr , = 1. 푁푙 푙 푘 푖=1 푙 푖 62 ≤ �푥 � ≤ ∑ �푥 � Technical Journal 10, 3-4(2016), 55-70 Hižak, J.; Logožar, R. Prikaz genetičkog algoritma i njegova uporaba za nalaženje ekstrema — s implementacijama u MATLABU

Code Listing 3. The MATLAB function for the roulette 5.1. Recombination in genetic algorithms wheel parent selection. RandChooseN(P,N) gives the As it was already mentioned in §2.1, the process of re- indices of N randomly selected chromosomes. combination in evolutionary algorithms is mathematical- ly abstracted to the action of the crossover operator. This binary operator—in the sense that it acts on two oper- ands—exchanges the genes (bits) of two artificial chro- mosomes. As conceived by Holland in 1975, the cross- breeding performed by the crossover operator is a three-

step process. [4] First, two individuals are taken from the parent (previous) population. Second, a point or several points of the chromosome splitting are chosen — the so- called crossover points. These points define the segments of genes (bits) between them. The contents of some of those segments are being exchanged, resulting in the 4.5. A few comments on the selection process generation of two new descendants with generally differ- ent chromosome structure (Figure 7). The above depicted selection—implemented by the rou- The simplest way of choosing the crossover points is lette method—is called simple selection. Another type of by random selection. Then some of the formed sections 10 selection is elimination selection. While the simple selec- of bits are exchanged and some are not. This is the case tion deletes the old generation only after it is used for the of the ordinary crossbreeding, which is governed by the selection process of all pairs of children for the new corresponding ordinary crossover operator. generation (before the step 3.3 in Algorithm 1), the elim- In the simple example of Figure 7, the ordinary ination selection first detects bad chromosomes and de- crossbreeding with single crossover point is applied. It letes them prior the crossover and mutation operations. defines the left and the right sections of the two chromo- The shortage of chromosomes is then compensated by somes, of which the right sections are exchanged. Thus, the additional offspring from the good chromosomes. To the probability that the descendant has its corresponding decrease or avoid the accumulation of duplicate chromo- (upper or lower) parent’s genes in the left section is one, somes, some variants of this method check if the de- and in the right section is zero, meaning that the right scendants are identical to their parents. If they are, the section has the genes from the other, “crossed” parent. process of selection is repeated. This sort of selection is called the elimination selection without duplicates.

5. RECOMBINATION AND MUTATION

The recombination (crossover) and mutation in biological systems are complex processes of exchange of genetic Figure 7. The recombination of chromosomes with one material that are happening between the pairs of chromo- breaking point. From the previous generation ( = somes (Figure 6). After the interphase, in which the 1), two parent chromosomes were selected. The ′crosso- chromosomes are duplicated, they align to each other, ver operator then splits them between the second푙 and푙 the− break in one or more places, and swap their fragments. third bit and exchanges their right parts. Thus, a pair of This mixing results in the enhanced variability of the new chromosomes of the -the generation is formed. population. The above and all the other푙 probability schemes can

be generalized by the crossover probability mask ,

= ( ), ( ), … , ( ) . (20a퐶 ) 퐦 The mask퐦퐶 consists�푃푃 푑of0 푃푃-plets푑1 of probabilities푃푃 푑푏−1 � , = 0,1, … , 1, each of which presents the probability 푏 푃푃�푑푗� that the descendant has the gene—here the binary digit —푗 inherited 푏from− its corresponding predecessor. If the 푗 mask for the first chromosome in a pair (the upper one 푑in

Figure 7) is , the mask applied to its conjugal partner

(the lower one in Figure 7) must be , with the com- 퐦퐶 plementary probabilities, = 1 퐶 : Figure 6. Meiosis, or reduction division. It starts from a 퐦���� somatic cell with a diploid number of chromosomes and = ( ),�푃푃����(��푑�횥��), … , −(푃푃�푑푗)� . (20b) ends with four sex cells that have the haploid number of With this퐶 notation,0 the probability1 masks푏−1 for the ex- 퐦���� �푃푃�����푑���� 푃푃�����푑���� 푃푃�����푑�������� chromosomes. The most important phase is prophase (P), ample of ordinary crossover in Figure 7 are: in which the recombination and generation of gametes

with completely different genetic material occurs. 10 By exchanging all of them, the net result would be the two starting chromosomes with swapped indices.

Tehnički glasnik 10, 3-4(2016), 55-70 63 An overview of the genetic algorithm and its use for finding extrema ─ with implementations in MATLAB Hižak, J.; Logožar, R.

= (1, 1, 0,0,0), = (0, 0, 1,1,1) . Code Listing 4. Implementation of the recombination with one randomly chosen crossover point in MATLAB. As we퐶퐶 have�퐶 already pointed 퐶퐶out�퐶, there can be more 퐦 �퐦������� Prior to this code, the vectors of indices for dad and mom than one breaking point. For bits there are maximally 1 breaking points. The introduced notation enables are to be defined. They denote the rows of the population the specification of such and other푏 cases, by forming the matrix that present the corresponding parents. adequate푏 − recombination probability masks. The uniform crossover operator allows equal-chance swapping of all bits, meaning that all the probabilities in the mask are equal, = = 1 2, leading to

= (0.5, 0.5, … , 0.5) . (21) 푃푃�푑푗� �푃푃������푑�횥�� ⁄ In other words, this mask states that all genes from the 퐦퐶퐶𝐶퐶 first parent have the opportunity of 1 2 to end up in the corresponding (first) descendant chromosome, and that the same is valid for the genes of the⁄ second parent and the corresponding (second) descendant chromosome. Finally, the fully general form of the mask in eq. (20a) allows the fully variant recombination (sometimes misleadingly called the -uniform crossover). We shall call it the variant crossover and denote it by CVar index. To affirm the meaning of푝 their masks, we give the fol- influenced or stimulated by outer factors, like radiation, lowing example: and sudden environmental changes. In natural conditions, it is very rare. If it happens to a somatic cell, it can cause = (0.2, 0.3, 1.0, 1.0, 0.0) . its damage or death. It can occasionally trigger uncon- Here, the first child will inherit the first gene from her/his 퐶퐶퐶� trolled division and malign diseases. However, the con- first parent with퐦 probability 0.2, and the second gene sequences of such mutations are not inherited by the with probability 0.3 (meaning that the probabilities that future generations. So, from the evolutionary point of these genes will be inherited from the second parent are view, much more significant are mutations that happen in 0.8, and 0.7, respectively). The third and fourth genes the sex cells, because they are transferred to the next will be surely inherited from the parent one (i.e. they stay generations. They are manifested in the changes of the as they were in the first parent), and the last gene will be organisms’ phenotypes, i.e. as new inherent characteris- surely inherited from the second parent (i.e. it is surely tics of the descendants. This makes the mutation the exchanged from what it was in the first parent). second important cause of the variability of biological The recombination does not have to be done on all populations. individuals within a population. The relative portion of the chromosomes that participate in the recombination— which can be also formulated as the rate of the activation 5.4. Mutation in genetic algorithms of the crossover operator—is called the crossover rate or crossover probability. We shall denote is as , In a genetic algorithm, the mutation will be performed by a mutation operator. It acts on a single chromosome and [0, 1]. If in population (iteration) the chromosomes 퐶 퐶 is therefore a unary operator. It changes one or more were selected, only , of them will be selected훾 훾 for∈ 푙 crossbreeding, 푙 푁 randomly chosen genes (bits) of the artificial chromo- 푁푙 퐶 some. For instance, the mutation operator can change a , = × . (22) chromosome = 11111 to its mutant = 11101. The values of that푙 퐶 are 퐶reported푙 to give the best be- We formalize the action of the mutation′ operator in a 푁 훾 푁 푛 푖 푖 havior of GA are between 0.5 and 1.0. [5] way similar to푥 the one used for the crossover푥 operator. 훾퐶 First, we introduce the mutation rate [0,1], which 5.2. Implementation of recombination in MATLAB decides what percentage of the whole gene pool will be 푀 influenced by mutation. That is, with 훾 chromosomes∈ in A simple implementation of recombination in MATLAB a population, of which each has genes, the gene pool is shown in Code Listing 4. It selects one breaking point 푙 contains a total of genes. Then the푁 number of muta- by using the function randi(nbits-1), which ran- 푛 1 1 tions performed in each iteration of the GA is domly picks an integer between and . Then the 푁 ∙ 푛 part of the chromosome to the right of the breaking point , = × . (23) is exchanged. The exchange of these chromosome푏 − parts Following the example푙 푀 of푀 the natural푙 mutation, in is performed with the last two statements of the code. 푁 훾 푁 푛 GA is normally made quite small, typically between 푀 0.001 and 0.01. E.g., with = 0.01 and a gene훾 pool 5.3. Mutation in biology with 20 5-bit chromosomes, only one gene (bit) and one 푀 As a motivation for the implementation of mutation chromosome are expected to undergo훾 the mutation. in GA, here we give a short reminder of the influence of To further generalize the action of the mutation oper- mutation on living organisms. Unlike the recombination, ator, once can introduce the mutation probability mask, the mutation in biology is a random change of genetic , which is fully analogous to the crossover probabil- material in a somatic or a sex cell. That change can be ity mask, defined by eq. (20a). 퐦푀 64 Technical Journal 10, 3-4(2016), 55-70 Hižak, J.; Logožar, R. Prikaz genetičkog algoritma i njegova uporaba za nalaženje ekstrema — s implementacijama u MATLABU

mutation mask. The parameters of the mutation mask can also be generated randomly. In the inverting mutation,

the bits specified by the mutation mask are inverted. [6] Both mutations are sketched in Figure 8.

6. EXAMPLES IN MATLAB

Figure 8. The action of inverting and mixing mutations Two complete MATLAB programs for finding extrema 1101101 illustrated on the chromosome . The inverting of the polynomial functions are given in Appendix A. (flip) mutation inverts the zeros into ones and vice-versa. Relying on the provided comments, the readers acquaint- The mixing mutation permutes (scrambles) the bits while ed with MATLAB basics should follow their code easily. conserving their parity (the numbers of both zeros and The Appendix A.1 contains the code for our simple ones stay the same). The partial versions of the mutations Example 1, introduced in §3.3. The performance of GA operate only on the selected fields of bits (shaded). for this case is shown by two diagrams of Figure 9. The upper diagram shows that the average fitness value rises In a simple mutation, randomly chosen genes , to its upper plateau in only about 20 iterations, but keeps are mutated, meaning that its mask is the same as the 푝 푀 oscillating. In fact, it looks like the GA is quite uniform crossover mask in푁 eq. (21). Besides that, the nd struggling to get to the maximum of this simple 2 order mixing and inverting mutations are used. The mixing function. A few iterations prior 30 and 40, it finds that mutation permutes the genes within the , randomly chosen chromosomes or within their parts specified by a 푁푝 푀

Figure 9. Diagrams of the average fitness (up) and the th Figure 10. Diagrams of the 4 -degree polynomial (up) fittest GA solution (down), for square function of Exmpl. and the fittest GA solution. As can be seen by inspecting 1. The population average fitness increases quickly, but both diagrams, the algorithm successfully finds the glob- there is no sure indicator of the optimal solution. During al minimum and gives its accurate coordinates without the generations 55-75, the fittest chromosome is , = being trapped in the local minimum (around = 6.75). 20. After 75 generations, GA finally finds the exact 푥푖 max solution, , = 18. 푥 푥푖 max

Tehnički glasnik 10, 3-4(2016), 55-70 65 An overview of the genetic algorithm and its use for finding extrema ─ with implementations in MATLAB Hižak, J.; Logožar, R.

We shall expose the problem briefly at the expense of the maximum is at = 20, but does not converge imme- diately to it. Then, from 55th to 75th iteration it stabilizes accuracy (and hopefully not of the clarity). In the classic at that value. However,푥 the average fitness value keeps form of the IPD, the population usually consists of play- ers represented by deterministic strategies that always on oscillating and shows a few dips. The right answer, follow their algorithms. For example, the ALLC strategy = 18 , is finally found after 75 iterations. plays cooperatively always, and the strategy ALT plays The Appendix A.2 gives the code for GA that finds nd 11 푥the global minimum of the 4th-degree polynomial. The 5- cooperatively in every 2 round. The TFT strategy bit chromosome representation is adjusted to the interval simply copies the moves of the opponent player from the [1, 8] by formula (3b). As can be seen in the upper dia- previous round. However, in the real-life situations, both gram of Figure 10, the polynomial has two minima (of animals and humans make mistakes. This invites for the course, with a local maximum between them). The right implementation of the uncertainty into the players’ minimum is only local. On the lower diagram, one can moves. The IPD that includes that is the above men- see how the GA hits the global minimum in less than 30 tioned Stochastic IPD. In SIPD, each player is represent- ( , ) iterations. It is found for the chromosome 5 = 00100, ed by a pair of probability values , whereby stands for the probability of cooperation after the oppo- with the value ( 5 ) = = 4, for which eq. (3c) nent’s cooperation, and stands for the푝 푞 probability of푝 gives = = 1.9032. This abscise value푛〈 〉 is marked cooperation after the opponent’s defection.13 Now sup- on the polynomial푉 curve푛〈 〉 as a푛 small circle. It can be seen min 11 pose that we want to find푞 the best strategy of the SIPD in that the푥 result푥 is very accurate. All in all, it looks like which the total payoff for a particular player equals to the the previous simple function wasn’t enough of a chal- sum of payoffs gained in the duels with all other players. lenge for this mighty algorithm! The payoff as a function of ( , ) depends on various However, a careful reader will notice that not every- parameters, such as the total number of players, the strat- thing is in a steady state. Slightly below 65 iterations egies of the players and the number푝 푞 of the game rounds. there is an instability, showing slightly higher value for Interpreted in another way, the payoff is a multivariable the . Such behavior is quite common for genetic function, whose maximum cannot be found by the classi- algorithms, due to the action of the variation operators. 푥min cal optimization algorithms. One possibility to find the extrema of such payoff 7. USE OF THE GENETIC ALGORITHM IN THE function is to simulate the . One can generate a set of strategies and then “leave it to the natu- STOCHASTIC ITERATED PRISONER ral selection.” The best strategy will be the one that be- DILEMMA comes dominant in a population. Most of the studies in this field have been done under After showing the efficiency of the genetic algorithm for the assumption that the individuals reproduce asexually; finding global extrema of the single-variable functions, the strategies fight among each other, the generation after here we elaborate on how it was used for a function that generation, with relative frequencies proportional to their depicts cooperation in the heterogeneous population of payoff in the previous generation. It means that particular selfish individuals in the problem known as Stochastic strategy could be, and usually is used by more than just Iterated Prisoner’s Dilemma (SIPD), which is a version one player. The players with copied strategies might be of the Iterated Prisoner’s Dilemma (IPD).12 considered as the clones of the corresponding original strategies. It should be stressed that in this approach there

11 is no crossbreeding between different strategies. The accuracy would be significantly diminished if the in- terval [0, 8] was chosen. For it, = 0.25806, and the indi- On the other hand, the GA could be applied to the vidual closest to the minimum is 00111, with value 7, giving SIPD if the strategies are suitably encoded in the form of

= 1.8065 (0.0967 lower thanΔ 푥 ). The second best is artificial chromosomes. An example of such approach is 01000, with value 8, giving = 2.0645 (0,1613 greater than our model, in which each reactive strategy ( , ) is rep- 푥 푥min ). The high dependence of the result on the choice of inter- resented by a 10-bit chromosome. [8] The first five bits val is an obvious consequence푥 of the low resolution of the in the chromosome encode the probability 푝 푞, and the 푥searchmin space, i.e. of the low number of the chromosome bits. 12 remaining five bits encode the probability (Figure 11). The Prisoner’s Dilemma is a well-known game in which 푝 two players can cooperate with or betray each other. The out- 푞 come for each player varies according to the combined outcome for them both. It is rational that each player defect (D), even though they would both gain more if they cooperated (C). If there is another encounter of the players, in another round of the game, the cooperation may be really profitable. This kind of game is called Iterated Prisoner's Dilemma (IPD). It is consid- ered as the paradigm for the evolution of cooperation among

selfish individuals. [7] In the context of biology, the game Figure 11. Double-point crossover operator applied to payoff (food, water, etc.) may be considered as fitness. To find the parent chromosomes which represent a ( , ) SIPD out which IPD strategy is the most effective, the mathemati- strategy. cian, political scientist and game theorist Robert Axelrod orga- 푝 푞 nized a tournament in 1979, and invited the scientists to submit their solutions. The simplest strategy, known as Tit for Tat round (ALLD) is the only strategy that is generally evolutionar- (TFT), won the tournament. This was a big surprise because it ily stable (in more details commented in [8]). was well known by then that the strategy of defecting on every 13 This kind of strategy is known as reactive strategy.

66 Technical Journal 10, 3-4(2016), 55-70 Hižak, J.; Logožar, R. Prikaz genetičkog algoritma i njegova uporaba za nalaženje ekstrema — s implementacijama u MATLABU

Figure 12. The cooperation coefficients over time in (a) populations that reproduce asexually, (b) populations with sexual reproduction. For the asexual reproduction, the cooperative strategies win after around 1000 generations. The sexual reproduction does not allow the extinction of different strategies and results in their erratic and cyclic exchanges.

The payoff gained in one round serves as a fitness operators and the exploiters can rule the world by them- function for the evaluation of strategies — the most suc- selves just temporarily. cessful strategies will have the highest chances for mat- ing. After being selected, every chromosome is subjected to double point crossover operator. One crossover point 8. CONCLUSION is randomly selected on the and another on the chromosome segment (confer Figure 11). When some optimization problem is being solved using a At the beginning of each iteration푝 (each round of the푞 genetic algorithm (GA), it is not guaranteed that it will game), the cooperation coefficients are calculated as the give a perfect or even a very good solution. But quite averages of the strategy -probabilities, often it will deliver a “good enough solution”, and almost always it will provide some kind of useful information 푝 about the problem. = 푁푠�푠 . (24) The examples presented in this paper have shown that 푖 푖 the global extremum of the single-variable functions was 푝̅ � 휔 푝 th Here stands for the 푖relative=1 frequency of the -th found very effectively, especially in the case of the 4 - strategy with the cooperation probability , and the degree polynomial. Its derivative gives a third order 푖 summation휔 is over all strategies. 푖 equation that cannot be solved easily. Obviously, the 푝푖 This, sexually reproduced SIPD model have shown strength and the justification for the use of GA increase 푁푠𝑠 that the subsequent populations are going through the with the number of function’s local extrema and general- erratic and cyclic exchanges of cooperation and exploita- ly with the degree of the function complexity. tion. Such result is just the opposite from what the previ- On the other hand, in the case of the Stochastic Iterat- ous models—with asexual reproduction of individuals— ed Prisoner's Dilemma, the GA did not perform that are giving. In them, once some strategy is extinct, its efficiently. The populations of strategies do not evolve restoration is impossible. Figure 12 shows the coopera- toward any dominant type of strategy. Instead, they di- tion coefficients for both of these cases. verge and jump from one near-optimal solution to anoth- The instabilities in our model are the consequence of er, showing that there is no clear winning strategy. How- the GA variation operators. They preserve the genetic ever, although “the best” strategy wasn’t found, the GA variability and diversity of the dominant strategies. In revealed erratic and circular patterns in the level of coop- this case, the genetic algorithm does not deliver an opti- eration over time. mal solution (the best strategy). On the contrary, it re- Henceforth, we conclude that—besides the determi- stores the defective strategies and leads to the instability nation of the function extrema—the genetic algorithm and divergence. There is no dominant strategy that would can occasionally give us something quite different — a eliminate the opponent players for good. Both, the co- new insight into the subject of study.

Tehnički glasnik 10, 3-4(2016), 55-70 67 An overview of the genetic algorithm and its use for finding extrema ─ with implementations in MATLAB Hižak, J.; Logožar, R.

9. REFERENCES APPENDIX A. FINDING EXTREMA OF TWO POLYNOMIAL FUNCTIONS BY THE GENETIC [1] Mitchell, M.: An Introduction to Genetic Algorithms. ALGORITHM ─ MATLAB CODE Bradford Book MIT Press, Cambridge, Massachu- setts, 1996. A.1. Genetic algorithm for finding the maximum [2] Gondra, I.: Parallelizing Genetic Algorithms, in for Advanced Problem Solving of a square function Techniques, Vrakas, D. & Vlahavas, I. eds., Hershey, Information Science Reference, 2008. %%MAXIMUM %This program finds the x-value for which [3] Negnevitsky, M.: Artificial Intelligence – A Guide to %the function f(x)=36x-x^2 reaches the Intelligent Systems, Addison-Wesley, Harlow, 2005. %maximum value.

[4] Booker, L. B.: Recombination, in Handbook of Evo- %Program written by Jurica Hižak lutionary Computation, Kenneth de Jong ed., Oxford %University North, Varaždin,2016 University Press, 1997. %FITNESS FUNCTION: [5] Lin, W. Y.; Lee W. Y., Hong, T. P.: Adapting ff=inline('36.*x-x.^2') Crossover and Mutation Rates, Journal of Infor- %Parameters: mation Science and Engineering, No. 19, 2003, pp. n=100 %number of generations 889-903. popsize=40 %population size mutrate=0.001 %mutation rate [6] Soni, N.; Kumar, T.: Study of Various Mutation Operators in Genetic Algorithms, International Jour- nbits=5 %number of bits in a %chromosome nal of Computer Science and Information Technolo- gies, Vol. 5, No 3, 2014, pp. 4519-4521. %CREATE an initial population:

[7] Nowak, S.: Tit for tat in heterogeneous populations, popul=round(rand(popsize,nbits)) Nature, Vol 355, 1992, pp. 250-253. %Create a vector that will record near- %optimal solutions across the generations: [8] Hižak, J.: Genetic Algorithm Applied to the Sto- best=zeros(1,n) chastic Iterated Prisoner’s Dilemma, conference pa- %Create a vector that will track the per, Basoti 2016, Tallinn, Estonia (to be published) %average the fitness of the population: https://www.researchgate.net/publication/305768564 avg_fitness=zeros(1,n) _Genetic_Algorithm_Applied_to_the_Stochastic_ Iterated_Prisoner_Dilemma. %EVOLUTION [9] Obitko, M.: Introduction to Genetic Algorithms, Web iter=0 while iter

68 Technical Journal 10, 3-4(2016), 55-70 Hižak, J.; Logožar, R. Prikaz genetičkog algoritma i njegova uporaba za nalaženje ekstrema — s implementacijama u MATLABU

%Create a vector that will track %MUTATION %the average fitness of the population: %Number of mutations in the population: avg_fitness=zeros(1,n); nmut=ceil(popsize*nbits*mutrate) for i=1:nmut col=randi(nbits) %EVOLUTION row=randi(popsize) iter=0 if popul(row,col)==1 while iter

%Choose the crossover point: %FITNESS FUNCTION: xp=randi(nbits-1) clear all %Take every 2nd dad's chromosomes f=inline('x.^4-18*x.^3+110*x.^2-... %and exchange the genes with mom’s: 255*x+300') popul(1:2:popsize,:)=[popul(dad,1:xp)... %Plot the function and hold the figure: popul(mom,xp+1:nbits)]; x=0:0.1:8; popul(2:2:popsize,:)=[popul(mom,1:xp)... figure popul(dad,xp+1:nbits)]; ymin=feval(f,x); plot(x,ymin) axis([0 8 50 200 ]) %MUTATION xlabel('x') %Number of mutations in the population: ylabel('f(x)') nmut=ceil(popsize*nbits*mutrate); hold on for i=1:nmut; n=100 %number of generations col=randi(nbits); popsize=40 %population size row=randi(popsize); mutrate=0.001 %mutation rate if popul(row,col)==1; nbits=5 %number of bits in a popul(row,col)=0; %chromosome else popul(row,col)=1; %CREATE an initial population: end popul=round(rand(popsize,nbits)) end %Create a vector that will record end %near-optimal solutions across %the generations: best=zeros(1,n);

Tehnički glasnik 10, 3-4(2016), 55-70 69 An overview of the genetic algorithm and its use for finding extrema ─ with implementations in MATLAB Hižak, J.; Logožar, R.

%MINIMUM: xmin=best(n) ymin=feval(f,xmin) %Plot (xmin,ymin) as a red ring on the %current figure: plot(xmin,ymin, 'ro')

%PLOT the x-value of the best chromosome %over time: figure %Generate an array of integers from the %first to the last generation: t=1:n plot(t,best) axis([1 n -1 10 ]) xlabel('number of generations') ylabel('x-min') %Show textbox with the output value %(xmin,ymin) dim = [.26, .2, .2, .6]; str = ['xmin=',num2str(xmin),... 'ymin=',num2str(ymin)]; annotation('textbox',dim,'String',... str,'FitBoxToText','on')

70 Technical Journal 10, 3-4(2016), 55-70