Evolutionary Manytasking Optimization Based on Symbiosis in Biocoenosis

The Thirty-Third AAAI Conference on Artiﬁcial Intelligence (AAAI-19)

Rung-Tzuo Liaw, Chuan-Kang Ting Department of Power Mechanical Engineering National Tsing Hua University Hsinchu 30013, Taiwan [email protected], [email protected]

Abstract Evolutionary multitasking introduces the new concept of simultaneously solving multiple problems through a single Evolutionary multitasking is a significant emerging search run of EA (Gupta, Ong, and Feng 2016). The multi-factorial paradigm that utilizes evolutionary algorithms to concurrently optimize multiple tasks. The multi-factorial evolution- evolutionary algorithm (MFEA) has proved effective at real- ary algorithm renders an effectual realization of evolution- izing evolutionary multitasking by leveraging the synergy of ary multitasking on two or three tasks. However, there re- fitness landscapes among different problems. By regarding mains room for improvement on the performance and ca- each problem as a task, MFEA seeks the optima for all tasks pability of evolutionary multitasking. Beyond three tasks, in a unified decision space, where the search space of mul- this paper proposes a novel framework, called the symbio- tiple problems are unified by a transformation function. The sis in biocoenosis optimization (SBO), to address evolution- solution information of different tasks is exchanged through ary many-tasking optimization. The SBO leverages the notion skill factor and mating. MFEA has achieved several promis- of symbiosis in biocoenosis for transferring information and ing results in dealing with two or three tasks. However, there knowledge among different tasks through three major com- exists room for improving EAs in multitasking. ponents: 1) transferring information through inter-task individual replacement, 2) measuring symbiosis through inter- This paper aims to address three issues at evolutionary task paired evaluations, and 3) coordinating the frequency multitasking. First, information transfer plays a crucial role and quantity of transfer based on symbiosis in biocoenosis. in solving multiple tasks concurrently. In MFEA, informa- The inter-task individual replacement with paired evaluations tion is transferred through recombination of individuals that caters for estimation of symbiosis, while the symbiosis in bio- are good at some specific tasks under the user-defined ran- coenosis provides a good estimator of transfer. This study ex- dom mating probability (rmp). Nevertheless, MFEA does amines the effectiveness and efficiency of the SBO on a suite not control the rmp during evolution; that is, it lacks a mech- of many-tasking benchmark problems, designed to deal with anism for controlling the quantity and frequency of informa- 30 tasks simultaneously. The experimental results show that SBO leads to better solutions and faster convergence than the tion transfer. Second, MFEA is inapplicable to the model- state-of-the-art evolutionary multitasking algorithms. More- based EAs, such as estimation of distribution algorithm over, the results indicate that SBO is highly capable of identi- (Hauschild and Pelikan 2011) and ant colony optimization fying the similarity between problems and transferring infor- (Dorigo, Maniezzo, and Colorni 1996), in that these EAs mation appropriately. seldom adopt recombination of individuals. The information transfer among tasks is therefore disabled. Third, evolutionary multitasking is focused on two or three tasks, whereas 1 Introduction concurrently solving more tasks is highly desirable but has Evolutionary algorithms (EAs) (Holland 1975; Goldberg not been investigated yet. In particular, appropriate transfer 1989; Schwefel 1995) have shown their great capability of becomes even harder as the number of tasks increases due to tackling search and optimization problems. Inspired from m the squared number 2 of possible transfers. Therefore, the Darwinian evolution theory (Darwin 1859), EAs mimic nat- increase of tasks intensifies the importance of balancing the ural evolution to search for the optimal solutions by ma- exploitation within a task and the exploration among multi- nipulating a population of candidate solutions. There have ple tasks. been various EAs proposed for different problems (Eiben This paper proposes a novel framework for evolutionary and Smith 2003; Gen and Cheng 1997). In EA, a population multitasking, called the symbiosis in biocoenosis optimiza- of individuals evolves for searching the optimal solution, tion (SBO), which manipulates multiple EAs and each one is where a solution is encoded as chromosome, and the fitness responsible for a task. In SBO, the collection of EA popula- implies the quality of a solution. The principle of “survival tions constitutes the biocoenosis, while the transfer of infor- of the fittest” drives the population towards better and opti- mation between populations caters to the symbiosis. Specifi- mal solutions. cally, the inter-task individual replacement is proposed for Copyright c 2019, Association for the Advancement of Artificial information transfer; hence, SBO enables the use of EAs Intelligence (www.aaai.org). All rights reserved. without recombination for evolutionary multitasking. For

4295 measuring the symbiosis, we combine the inter-task individual replacement with paired evaluations. Furthermore, SBO Species A Species F controls the quantity and frequency of information transfer Biocoenosis through symbiosis to balance exploitation within single task and exploration between different tasks. The effectiveness Symbiosis of SBO is verified on the suite of many-tasking problems Species B Species E (MaTPs), in which each problem comprises 30 test functions of CEC 2017 competition. A series of experiments is conducted on the MaTPs to investigate the effects and advantages of SBO in evolutionary many-tasking. Species C Species D The main contributions of this study are summarized as follows: • A novel framework SBO for evolutionary multitasking. Figure 1: An illustration of SBO framework • Three features of SBO: transferring information through inter-task individual replacement, measuring symbiosis et al. (2015) incorporated a nested bi-level EA into MFEA through inter-task paired evaluations, and coordinating for tackling bi-level optimization problems. Sagarna and the frequency and quantity of transfer based on symbiosis Ong (2016) used MFEA to solve the software testing prob- in biocoenosis. lems. Chandra et al. (2016) utilized MFEA to optimize the • Empirical study on the effectiveness and efficiency of architecture and parameters of feed forward neural network. SBO, in comparison with single-task optimization and Zhou et al. (2016) adopted MFEA on a combinatorial opti- multitask optimization methods. mization problem, i.e., the capacitated vehicle routing prob- • Comprehensive analysis of transfer behavior for SBO and lem. In (Gupta et al. 2016b), MFEA tackled multi-objective MFEA. optimization problem by treating two performance metrics for multi-objective optimization problem, i.e., the nondomi- The remainder of this paper is organized as follows. Sec- nated front and crowding distance, as different tasks. tion 2 reviews the related work about evolutionary multi- Some studies focus on improving or analyzing the ef- tasking. Section 3 elucidates the proposed SBO framework fect of transfer. The synergy of fitness landscapes affects and its components. Section 4 presents the experimental re- the effectiveness and efficiency of MFEA. That is, a bet- sults. Finally, we draw conclusions in Section 5. ter movement in decision space for one task can be good for the other task. Gupta et al. (2016a) analyzed the syn- 2 Related Work ergy of fitness landscapes on some test functions. Wen and Evolutionary multitasking establishes a new class of EAs ca- Ting (2017) designed a parting ways strategy based on the pable of solving multiple problems simultaneously. MFEA survival rate of transferred individuals; such strategy aims is a renowned EA for evolutionary multitasking (Gupta et at terminating information transfer between tasks if transfer al. 2017; Gupta, Ong, and Feng 2018; Ong and Gupta 2016; is useless. Li et al. (2018) enabled multiple rmp to deter- Strasser et al. 2016). MFEA utilizes a single population for mine the frequency of transfer, where each rmp is adapted optimizing multiple tasks. The main ideas behind MFEA are according to the survival rate after genetic transfer. Cheng et the designs of scalar fitness serving as a unified fitness func- al. (2017) applied the scheme of co-evolution to evolution- tion for survival selection and the assortative mating opera- ary multitasking, yet the performance is similar to MFEA tor for information transfer. The skill factor of an individual in bi-tasking test problems. Ding et al. (2017) improved represents the task index with the best rank over all tasks, the transfer mechanism in MFEA by learning the decision while the scalar fitness is the reciprocal of the rank of the space transformation, including the location and permuta- most talented task. The assortative mating operator performs tion of decision vector. Feng et al. (2018) proposed transfer- crossover in two cases: the two parents have the same skill ring knowledge through task mapping, where the mapping is factor or the predefined random mating probability (rmp) learnt by a denoising autoencoder. Some studies utilize the is met. The offspring generated by crossover operator ran- transfer from past experience or knowledge. In (Feng et al. domly imitates the skill factor from either parent, whereas 2015), a memetic search paradigm was proposed by incor- the offspring perturbed only by mutation operator inherits porating EA with transfer learning. Specifically, the knowl- the skill factor. A newly generated offspring is evaluated edge is learnt as memes from past solved problems and then only on the task of the skill factor. It is worth noting that used to guide the search of EA. Feng et al. (2017) incor- assortative mating supports transfer of information between porated EA with knowledge from past experiences, namely, different tasks. When the number of tasks m increases, the the solutions from the other tasks for improving the search frequency of information transfer will gradually change to efficacy. the predefined rmp owing to the decreasing probability 1/m Recent studies suggest using island model or multiple of selecting individuals with the same skill factor. populations for evolutionary multitasking. Hashimoto et MFEA has been applied to a variety of applications. Wen al. (2018) presented an island model with two populations and Ting (2016) adopted the concept of MFEA on genetic for evolutionary multitasking. Liaw and Ting (2017) pro- programming for building ensemble of decision trees. Gupta posed a multi-population based method, called the evolution

4296 Algorithm 1 Symbiosis in biocoenosis optimization Algorithm 2 Update of symbiosis th EAi: EA for i task m: number of tasks th τi: i task 1: for i ← 1 to m do th λi: Offspring size of i task 2: for each c ∈ Ofspi do m: number of tasks 3: j ← task(c) . Belonging task 1: for i ← 1 to m do 4: if i 6= j then . Transfer occurs 2: Initialize EAi for τi 5: Update M, N , C, O, P, and A by (10). 3: end for 6: end if 4: {M, N , C, O, P, A} ← 1 . Initialize symbiosis 7: end for 5: Update transfer rates Ri,j . Algorithm 3 8: end for 6: while Not terminated do 7: for i ← 1 to m do Algorithm 3 Update of transfer rates 8: Ofsp ←Variation (EA ) . Generate offspring i i m: number of tasks 9: end for R : transfer rate from task τ to task τ 10: for i ← 1 to m do i,j j i 1: for i ← 1 to m do 11: j ← arg max Ri,k . Transferred task k∈{1,...,m|k6=i} 2: for j ← 1 to m and j 6= i do pos 12: Ri ← Ri,j . Transfer rate 3: T i,j ← Mi,j + Oi,j + Pi,j neg 13: if Rand (0,1) < Ri then 4: T i,j ← Ai,j + Ci,j 14: S ← bR · λ c . Transfer quantity neu i i i 5: T i,j ← Ni,j 15: for k ← 1 to Si do pos pos neg neu 6: Ri,j ← T / (T + T + T ) 16: Ofsp ← Ofsp i,j i,j i,j i,j i,λi−Si+k j,k 7: end for 17: end for 8: end for 18: Evaluate (Ofspi,τi) 19: Survive (EAi) 20: end if 21: end for (Alg. 3) are alternately performed as per transferred individ- 22: Update symbiosis . Algorithm 2 uals. SBO can be viewed as an island-model EA because it 23: Update transfer rates . Algorithm 3 manipulates several EAs in the course of evolution. In par- 24: end while ticular, SBO features adaptive control over the transfer (mi- gration) rate, transfer frequency, and selection of transferees. Moreover, SBO does not require to predetermine the migra- tion topology (ring, grid, etc.)—the topology is formed au- of biocoenosis through symbiosis (EBS), in which the trans- tomatically and adaptively in the SBO. ferees are randomly selected from all offspring. 3.1 Analogues of Symbiosis and Biocoenosis 3 Symbiosis in Biocoenosis Optimization In a biocoenosis, the symbiosis defines the effects of inter- This study proposes a novel framework SBO for evolution- action between two species. Table 1 lists the six main types ary many-tasking. Inspired from the symbiosis in the bio- of symbiosis in a biocoenosis. These six types of symbiosis coenosis, SBO enables multiple populations, each of which are determined according to two unidirectional influences is associated with an EA (see Fig. 1). The transfer of in- of two species with three cases, i.e., beneficial, neutral, and formation among all tasks constitutes symbiosis. Restated, harmful. A species is defined to be beneficial / neutral / SBO considers the pairwise correlation between tasks when harmful to the other species if such species has positive / selecting transferees, and further enables adaptive control no / negative effect to the other species. Two species are in over the transfer frequency and the number of transferees. mutualism (M), neutralism (N ), or competition (C) if they The SBO holds three main features: 1) information trans- are beneficial, neutral, or harmful to each other, respectively. fer through inter-task individual replacement, 2) symbiosis On the other hand, if a species A is beneficial to the other estimation via paired evaluations, and 3) adaptive control species B, the facts that the species B is neutral or harmful of transfer frequency and quantity based on the measure of to the species A result in different symbiosis. The relation symbiosis. of species A to species B forms commensalism (O) for the The proposed SBO framework is presented in Alg. 1. former case, and parasitism (P) for the latter case. In the last Given m tasks to be solved concurrently, SBO manipulates type of symbiosis, the amensalism (A) comes from the situ- m EAs for the m tasks. In each iteration, the m EAs trans- ation that a species A is neutral to the other species B, while fer information according to the transfer rates, determined the species B is harmful to the species A. by the degree of symbiosis among them. In SBO, the unit The symbiosis in biocoenosis can be a good analogue of of information to transfer is individual, more precisely, the information transfer in evolutionary multitasking. As afore- offspring. The information transfer in SBO relies on off- mentioned, SBO comprises multiple EAs, each of which is spring replacement and paired evaluations. After survival se- responsible for a task. The populations of different tasks lection, the update of symbiosis (Alg. 2) and transfer rates form species, and the information transfers among EAs re-

4297 Table 1: Six types of symbiosis in biocoenosis, including Definition 7 (Commensalism). A task τi to the other task mutualism (M), commensalism (O), parasitism (P), neu- τj forms commensalism if there is an individual c which is tralism (N ), amensalism (A), and competition (C) beneficial to τi, and neutral to τj, c τ and c ≈ τ . (7) Interaction Species A i j Benefit Neutral Harm Definition 8 (Parasitism). A task τi to the other task τj forms parasitism if there is an individual c which is bene- Benefit MOP ficial to τ , but harmful to τ , Species B Neutral ONA i j Harm PAC c τ i and c ≺ τ j. (8)

Definition 9 (Amensalism). A task τi to the other task τj forms amensalism if there is an individual c which is harmful flect symbiosis. In addition, this study uses implicit measure to τi, but neutral to τj, of symbiosis through individuals because explicit measure requires landscapes analysis, which is computationally ex- c ≺ τ i and c ≈ τ j. (9) pensive. Hence, this study defines the meaning of beneficial, As per the above six types of symbiosis, this study ap- neutral, and harmful from an individual to a task. proximates the degree of symbiosis (Alg. 2) by counting the Definition 1 (Beneficial). An individual c is said to be ben- number of times these conditions are satisfied: eficial to a task τ if c’s fitness is at task τ’s top BN ranking,  Mi,j ← Mi,j + 1 c τi, c τj, denoted by  N ← N + 1 c ≈ τ , c ≈ τ , c τ, (1)  i,j i,j i j C ← C + 1 c ≺ τ , c ≺ τ , where N represents the population size, and B is the benefi- i,j i,j i j (10) O ← O + 1 c τ , c ≈ τ , cial factor.  i,j i,j i j P ← P + 1 c τ , c ≺ τ , Definition 2 (Harmful). An individual c is defined to be  i,j i,j i j  harmful to a task τ if c’s fitness is at task τ’s bottom HN Ai,j ← Ai,j + 1 c ≈ τi, c ≺ τj, ranking, expressed by where M, N , C, O, P, and A maintain the degrees of mutu- c ≺ τ, (2) alism, neutralism, competition, commensalism, parasitism, where H stands for the harmful factor. and amensalism, respectively. Through the degree of symbiosis, transfers between different tasks can be measured as Definition 3 (Neutral). An individual c is defined to be neu- having positive, neutral, or negative effect. tral to a task τ if c is neither beneficial nor harmful to task τ, denoted as: 3.2 Transfer Strategy c ≈ τ. (3) Rather than directly measuring the symbiosis between two According to the above three relations between a solution tasks, this study leverages individuals to estimate the degree to a task, we can define the six types of symbiosis, i.e., mutu- of symbiosis. The unit of transfer in SBO framework is an alism, neutralism, competition, commensalism, parasitism, individual. Given two tasks τi , and τj, and assume that τi and amensalism. accepts the transfer from τj. When the inter-task individ- Definition 4 (Mutualism). Any two tasks τi 6= τj are re- ual replacement occurs, a portion Si of offspring of τi is garded as having mutualism with respect to an individual c replaced by the offspring of τj: if such individual is beneficial to both of the tasks, Ofspi,λi−Si+k ← Ofspj,k k ∈ {1, ..., Si}, (11) c τ and c τ . (4) i j where λi denotes the offspring size of task τi. Paired eval- Similar concept can be extended to symbiosis of neutral- uations of the two tasks are applied on these transferees, ism and competition. and the paired evaluations will serve as the basis for measuring the degree of symbiosis (10). Note that offspring are Definition 5 (Neutralism). Any two tasks τi 6= τj are re- transferred before they are evaluated; in addition, the trans- garded as having neutralism with respect to an individual c ferred offspring will replace the same amount of offspring in if such individual is neutral to both of the tasks, the target population. Therefore, the total number of fitness evaluations remains the same, and SBO does not require ad- c ≈ τ i and c ≈ τ j. (5) ditional evaluations. Based on the degree of symbiosis, SBO Definition 6 (Competition). Any two tasks τi 6= τj are re- controls the information transfer adaptively. garded as having competition with respect to an individual c if such individual is harmful to both of the tasks, 3.3 Coordinating Information Transfer Algorithm 3 is the procedure of updating the transfer rates. c ≺ τ i and c ≺ τ j. (6) th pos Given the i task τi, transfers with positive effect (T i,j ) Aside from the above symmetric relations, the other three from the other tasks τj include mutualism Mi,j, commen- types of symbiosis are asymmetric; that is, an individual has salism Oi,j and parasitism Pi,j, while transfers with neg- neg different effects to two tasks. ative effect (T i,j ) are composed of amensalism Ai,j and

4298 Table 2: Parameter setting the positions of optimal solutions of all tasks, there are four MaTPs labeled as MaTPZ for zero shift, MaTPS for small Parameter Value shift, MaTPM for medium shift, and MaTPL for large shift of the positions of optimal solutions, and the four MaTPs Problem size (d) 100 correspond to four shift ranges: 1) zero (no) shift, 2) small #Evaluations 104d shift U(−1, 1), 3) medium shift U(−5, 5), and 4) large shift Pop. and ofsp. sizes (GA) (50m, 50m) U(−10, 10) at each dimension between any two test func- Pop. and ofsp. sizes (CMAES) (17m, 34m) tions, where U denotes uniform distribution. The similarity Beneficial factor (B) 0.25 among tasks decreases from MaTP to MaTP . Harmful factor (H) 0.50 Z L 4.2 Solution Quality and Convergence neu This study examines the significance of difference in so- competition Ci,j. The transfers with neutral effect (T i,j ) are from neutralism Ni,j. The transfer rate ought to be pro- lution quality through non-parametric significance analysis. portional to the transfers with positive effect; therefore, the Table 3 lists results of Wilcoxon-Mann-Whitney U-test with transfer rate Ri,j from τj to τi is defined by level of significance α = 0.05 for GA, CMAES, MFEA, EBSGA, EBSCMAES, SBGA, and SBCMAES on the four pos pos neg neu MaTP benchmarks. The SBGA outperforms GA on most Ri,j ← T / (T + T + T i,j ). (12) i,j i,j i,j test functions, validating that the SBO framework can im- The transfer rate plays an important role in the control of prove the solution quality of GA. As for CMAES, SBC- transferring information. It determines 1) the task of individ- MAES acquires significantly better results than CMAES uals to be transferred, 2) the transfer frequency, and 3) the on 27, 24, 19, and 12 test functions in MaTPZ, MaTPS, transfer quantity. First, each task τi selects transferees from MaTPM, and MaTPL, respectively. Note that the decrease the task that has the highest transfer rate, denoted by Ri. The in the number of better results respond to the increase in transfer frequency for task τi is determined by Ri. If trans- the shift of optimal solutions. Comparing the performance fer occurs, the transfer quantity Si for τi is then calculated of MFEA and SBGA, SBGA achieves better solution qual- by ity than MFEA does on 29, 21, 15, and 11 test functions in Si ← bRi · λic . (13) MaTPZ, MaTPS, MaTPM, and MaTPL, respectively. Ad- In this way, SBO is capable of adaptively controlling the ditionally, SBCMAES excels MFEA on all test functions in transfer frequency and quantity. all four MaTPs. In comparison of SBO and EBS, SBGA performs better than EBSGA does on MaTPs, MaTPM, and 4 Experimental Results MaTPL; moreover, SBCMAES outperforms EBSCMAES This study investigates the performance of the proposed on the four MaTPs except two functions in MaTPL. These SBO using GA (SBGA) and CMAES (Hansen 2006) (SBC- results indicate that the proposed SBO framework can effec- MAES), in comparison with MFEA (Gupta, Ong, and Feng tively improve single-task methods (GA and CMAES) and 2016) and EBS (Liaw and Ting 2017) on four MaTPs (cf. prevail multitask optimization methods (MFEA and EBS) Sec. 4.1). The experiments examine the solution quality and on many-tasking problems with varied similarities among convergence speed of the proposed method. This study fur- tasks. ther looks into the behaviors of MFEA and SBO methods Figure 2 compares the variation of fitness values during in the course of evolution through the analysis of transfer evolution for GA, CMAES, MFEA, EBSGA, EBSCMAES, among tasks. SBGA and SBCMAES on the four MaTP benchmarks. Due Table 2 lists the parameter setting used in the following to the space limitation we illustrate only one function for experiments. The beneficial and harmful factors are set to each type. The SBCMAES achieves fastest convergence on 0.25 and 0.50, respectively. The setting of MFEA follows all four MaTPs, and the SBGA gains faster convergence the use of simulated binary crossover, polynomial mutation, speed than GA and MFEA on MaTPZ, MaTPS, MaTPM, and rmp set to 0.3 in (Gupta, Ong, and Feng 2016). The and composite function in MaTPL. Likewise, SBGA con- population size is set to 50m to better handling the many- verges faster than EBSGA does on unimodal, multi-modal, tasking benchmarks. All experiments run over 30 trials due and hybrid functions in MaTPS, MaTPM, and MaTPL. to the stochastic nature of EAs. Significant analysis is done These results validate the nice convergence of the proposed by using Wilcoxon-Mann-Whitney U-test with level of sig- SBO framework. nificance α = 0.05. 4.3 Effect on Transfers 4.1 Many-tasking Benchmark Problems In this study, we use survival rates to examine the utility This study presents a test suite based on the benchmark of information transfer. Figure 3 compares the variation of functions of CEC 2017 competition (Awad et al. 2016; survival rates of transferred and non-transferred individuals Liaw and Ting 2017). A many-tasking problem (MaTP) is during evolution in MFEA, SBGA, and SBCMAES on the composed of 30 benchmark functions, including 3 unimodal, four MaTP benchmarks. The SBGA and SBCMAES both 7 simple multi-modal, 10 hybrid, and 10 composite func- have higher survival rates of transferred individuals than tions, and each function is regarded as a task. By adjusting MFEA does. The survival rate of transferred individuals in

4299 Table 3: The numbers of functions that the former is superior / equal / inferior to the latter with signiﬁcance

SBGA SBGA SBGA SBCMAES SBCMAES SBCMAES GA MFEA EBSGA CMAES MFEA EBSCMAES

MaTPZ 30 / 0 / 0 29 / 1 / 0 0 / 1 / 29 27 / 3 / 0 30 / 0 / 0 11 / 19 / 0 MaTPS 22 / 8 / 0 21 / 7 / 2 15 / 12 / 3 24 / 0 / 6 30 / 0 / 0 25 / 5 / 0 MaTPM 17 / 10 / 3 15 / 14 / 1 16 / 7 / 7 19 / 10 / 1 30 / 0 / 0 20 / 10 / 0 MaTPL 8 / 21 / 1 11 / 16 / 3 16 / 11 / 3 12 / 16 / 2 30 / 0 / 0 17 / 11 / 2

1010 104 108 106 8 6 10 2 10 4 2 7 10 6 GA 10 GA 15 22 10 4 CMAES CMAES 10 2 4 MFEA 0 MFEA 10 10 EBSGA 10 EBSGA 2 10 0 102 EBSCMAES EBSCMAES 10 SBGA -2 SBGA 0 GA 10 10 CMAES 100 SBCMAES SBCMAES -2 -2 MFEA 10 GA -2 -4 10 EBSGA 10 10 -4 CMAES -4 EBSCMAES 10 MFEA -4 10 SBGA EBSGA 10 -6 SBCMAES Mean Best Fitness on F Mean Best Fitness on F -6 -6 10 Mean Best Fitness on F -6 Mean Best Fitness on F 10 EBSCMAES 10 10 SBGA SBCMAES 10-8 10-8 10-8 10-8 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 #Evals (106) #Evals (106) #Evals (106) #Evals (106)

(a) F2 in MaTPZ (b) F7 in MaTPZ (c) F15 in MaTPZ (d) F22 in MaTPZ

1010 104 108 106 GA GA 8 10 CMAES 107 CMAES 4

2 7 10 MFEA 15 MFEA 22 6 10 EBSGA 6 EBSGA EBSCMAES 10 EBSCMAES 2 4 10 10 GA SBGA 5 SBGA CMAES 3 SBCMAES 10 SBCMAES 0 102 MFEA 10 10 EBSGA 104 100 EBSCMAES -2 3 10 GA -2 SBGA 10 10 SBCMAES -4 CMAES 2 10 MFEA 10-4 10 EBSGA

Mean Best Fitness on F Mean Best Fitness on F -6 -6 2 Mean Best Fitness on F 1 Mean Best Fitness on F 10 EBSCMAES 10 10 10 SBGA SBCMAES 10-8 100 10-8 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 #Evals (106) #Evals (106) #Evals (106) #Evals (106)

(e) F2 in MaTPS (f) F7 in MaTPS (g) F15 in MaTPS (h) F22 in MaTPS

1010 104 108 106 GA GA GA 8 10 CMAES 7 CMAES CMAES

2 7 10 MFEA 15 MFEA 22 MFEA 6 EBSGA EBSGA EBSGA 10 105 EBSCMAES 106 EBSCMAES EBSCMAES 104 SBGA SBGA SBGA SBCMAES 5 SBCMAES SBCMAES 102 10 103 104 100 4 GA 10 -2 10 CMAES 3 MFEA 10 3 10-4 EBSGA 10

Mean Best Fitness on F Mean Best Fitness on F 2 -6 EBSCMAES Mean Best Fitness on F 10 Mean Best Fitness on F 10 SBGA SBCMAES 10-8 102 101 102 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 #Evals (106) #Evals (106) #Evals (106) #Evals (106)

(i) F2 in MaTPM (j) F7 in MaTPM (k) F15 in MaTPM (l) F22 in MaTPM

1010 104 106 106 GA GA GA 108 CMAES CMAES CMAES 2 7

MFEA 15 MFEA 22 MFEA 6 10 EBSGA 5 EBSGA 5 EBSGA EBSCMAES 10 EBSCMAES 10 EBSCMAES 104 SBGA SBGA SBGA SBCMAES SBCMAES SBCMAES 102 103 104 104 100 GA 10-2 CMAES MFEA 3 3 10-4 EBSGA 10 10 Mean Best Fitness on F Mean Best Fitness on F

EBSCMAES Mean Best Fitness on F Mean Best Fitness on F 10-6 SBGA SBCMAES 10-8 102 102 102 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10 #Evals (106) #Evals (106) #Evals (106) #Evals (106)

(m) F2 in MaTPL (n) F7 in MaTPL (o) F15 in MaTPL (p) F22 in MaTPL

Figure 2: Variation of ﬁtness values in the course of evolution for GA, CMAES, MFEA, SBGA, and SBCMAES on the four MaTP benchmarks

SBCMAES is higher than that of non-transferred individu- mechanism of transfer rates (cf. Sec. 4.4). By contrast, the als during most of the evolution on MaTPZ, MaTPS, and survival rate of transferred individuals in MFEA is smaller MaTPM. On MaTPL, the survival rate of transferred indi- than that of non-transferred individuals on the four MaTP viduals in SBCMAES is lower than that of non-transferred benchmarks. individuals. Such phenomenon is caused by the adaptation Furthermore, Figs. 4 and 5 plot the variation of positive

4300 1.0 0.40 100 100 MFEA (Non-Transfer) MFEA (Non-Transfer) Z MFEA (Transfer) S MFEA (Transfer) SBGA (Non-Transfer) 0.35 SBGA (Non-Transfer) 0.8 SBGA (Transfer) SBGA (Transfer) -1 -1 SBCMAES (Non-Transfer) 0.30 SBCMAES (Non-Transfer) 10 10 SBCMAES (Transfer) SBCMAES (Transfer) 0.6 0.25 0.20 10-2 10-2 0.4 0.15

0.10 10-3 10-3 0.2 MFEA MFEA 0.05 Mean Postive Transfer Rate Mean Postive Transfer Rate Mean Survival Rate on MaTP Mean Survival Rate on MaTP SBGA SBGA SBCMAES SBCMAES 0.0 0.00 10-4 10-4 0 5000 10000 15000 0 5000 10000 15000 0 5000 10000 15000 0 5000 10000 15000 Generations Generations Generations Generations

(a) MaTPZ (b) MaTPS (a) MaTPZ (b) MaTPS

0.40 0.40 100 100 MFEA (Non-Transfer) MFEA (Non-Transfer) L

M MFEA (Transfer) MFEA (Transfer) 0.35 SBGA (Non-Transfer) 0.35 SBGA (Non-Transfer) SBGA (Transfer) SBGA (Transfer) -1 -1 0.30 SBCMAES (Non-Transfer) 0.30 SBCMAES (Non-Transfer) 10 10 SBCMAES (Transfer) SBCMAES (Transfer) 0.25 0.25

0.20 0.20 10-2 10-2 0.15 0.15

0.10 0.10 10-3 10-3 MFEA MFEA 0.05 0.05 Mean Postive Transfer Rate Mean Postive Transfer Rate Mean Survival Rate on MaTP Mean Survival Rate on MaTP SBGA SBGA SBCMAES SBCMAES 0.00 0.00 10-4 10-4 0 5000 10000 15000 0 5000 10000 15000 0 5000 10000 15000 0 5000 10000 15000 Generations Generations Generations Generations

Figure 3: Variation of survival rates of transferred and non- Figure 4: Variation of positive transfer rates for transferees transferred individuals during evolution during evolution

and negative transfer rates for transferees in MFEA, SBGA, 1.0 1.00 and SBCMAES, respectively. SBCMAES achieves highest 0.9 MFEA SBGA positive transfer rate and lowest negative transfer rate. SBC- 0.8 SBCMAES 0.95 0.7 MAES and SBGA both gain higher positive transfer rates 0.6 0.90 and lower negative transfer rates than MFEA does on the 0.5 0.4 0.85 MFEA four MaTPs. These results validate that SBO framework has Mean Negative Transfer Rate 0.3 Mean Negative Transfer Rate SBGA SBCMAES 0.2 0.80 better mechanism for transfer than MFEA. 0 5000 10000 15000 0 5000 10000 15000 Generations Generations

4.4 Adaptation of Transfers (a) MaTPZ (b) MaTPS Figure 6 further shows the variation of total transfer rate, i.e., the sum of positive, negative, and neutral transfer rates, 1.00 1.00 in the course of evolution. The total transfer rate in SBGA 0.95 0.95 decreases during evolution owing to the decrease of positive transfer. Similar trend can be found in SBCMAES; more- 0.90 0.90 over, SBCMAES endures the total transfer rate in a high 0.85 0.85 MFEA MFEA Mean Negative Transfer Rate SBGA Mean Negative Transfer Rate SBGA level in the MaTPZ due to its high rate of positive trans- SBCMAES SBCMAES 0.80 0.80 fer in high similarity many-tasking problem. On the other 0 5000 10000 15000 0 5000 10000 15000 Generations Generations hand, the total transfer rate in MFEA stays at relatively high (c) MaTP (d) MaTP level in the course of evolution on the four MaTPs, reflect- M L ing that MFEA fails to respond to environmental changes for Figure 5: Variation of negative transfer rates for transferees different problems. during evolution 5 Conclusions Evolutionary multitasking is an emerging topic. Previous task individual replacement, measuring symbiosis through research concentrated on resolving problems with a small inter-task paired evaluations, and coordinating the transfer number of tasks concurrently. Enlarging the number of tasks frequency and quantity based on symbiosis in biocoenosis. intensifies the importance and need for appropriately trans- The efficacy of the proposed SBO is validated on a set of ferring information among tasks. This study proposes a many-tasking problems (MaTPs) with four different shifts of novel framework SBO for evolutionary many-tasking. From optima, i.e., the MaTPZ, MaTPS, MaTPM, and MaTPL. the inspiration of symbiosis in the biocoenosis, SBO con- The results have shown that the proposed SBO achieves the siders the interaction of multiple populations. In SBO, each best solution quality and convergence speed, in compari- EA is responsible for a task, and the transfer of informa- son with conventional single-task optimization methods and tion among tasks brings about symbiosis. SBO has three state-of-the-art multitask optimization methods. Analysis on main components: transferring information through inter- the effect of transfer demonstrates the advantages of SBO

4301 Transactions on Systems, Man, and Cybernetics, Part B: Cy- bernetics 26(1):29–41. -1 Eiben, A. E., and Smith, J. E. 2003. Introduction to Evolu- 10 MFEA SBGA tionary Computing. Natural Computing. Springer-Verlag. SBCMAES Feng, L.; Ong, Y.-S.; Tan, A.-H.; and Tsang, I. W. 2015. Memes as building blocks: A case study on evolutionary op-

-2 timization + transfer learning for routing problems. Memetic 10 Computing 7(3):159–180.

Mean Trasfer Rate Feng, L.; Ong, Y.-S.; Jiang, S.-W.; and Gupta, A. 2017. Autoencoding evolutionary search with learning across het- erogeneous problems. IEEE Transactions on Evolutionary 10-3 Computation 21(5):760–772. 0 5000 10000 15000 Generations Feng, L.; Zhou, L.; Zhong, J.-H.; Gupta, A.; Ong, Y.-S.; Tan, K.-C.; and Qin, A.-K. 2018. Evolutionary multitasking via Figure 6: Variation of total transfer rate during evolution. explicit autoencoding. IEEE Transactions on Cybernetics The square, circle, triangle, and diamond symbols represent PP(99):1–14. the rates on MaTPZ, MaTPS, MaTPM, and MaTPL, re- Gen, M., and Cheng, R. 1997. Genetic Algorithms and En- spectively. gineering Optimization. Wiley Series in Engineering Design and Automation. John Wiley & Sons. Goldberg, D. E. 1989. Genetic Algorithms in Search, Opti- framework over MFEA. More specifically, the SBO can ap- mization and Machine Learning. Addison Wesley. propriately manipulate the transfer frequency and quantity. Gupta, A.; Ong, Y.-S.; Da, B.; Feng, L.; and Handoko, S. D. Some directions remain for future work. For example, 2016a. Landscape synergy in evolutionary multitasking. In concurrent optimization of homogeneous and even hetero- Proceedings of IEEE Congress on Evolutionary Computa- geneous problems can be considered. Incorporating different tion, 3076–3083. EAs into SBO is also promising. Gupta, A.; Ong, Y.-S.; Feng, L.; and Tan, K. C. 2016b. Mul- tiobjective multifactorial optimization in evolutionary mul- 6 Acknowledgment titasking. IEEE Transactions on Cybernetics. The authors would like to thank the reviewers for their valu- Gupta, A.; Da, B.; Yuan, Y.; and Ong, Y.-S. 2017. On the able comments and suggestions. This work was supported emerging notion of evolutionary multitasking: A computa- by the Ministry of Science and Technology of Taiwan, un- tional analog of cognitive multitasking. Recent Advances in der contract MOST 107-2634-F-007-010. Evolutionary Multi-objective Optimization 20:139–157. Gupta, A.; Mandziuk,´ J.; and Ong, Y.-S. 2015. Evolutionary References multitasking in bi-level optimization. Complex & Intelligent Awad, N. H.; Ali, M. Z.; Suganthan, P. N.; Liang, J. J.; and Systems 1(1):83–95. Qu, B.-Y. 2016. Problem definitions and evaluation criteria Gupta, A.; Ong, Y.-S.; and Feng, L. 2016. Multifactorial for the CEC 2017 special session and competition on single evolution: Toward evolutionary multitasking. IEEE Trans- objective real-parameter numerical optimization. Technical actions on Evolutionary Computation 20(3):343–357. report, Nanyang Technological University. Gupta, A.; Ong, Y.-S.; and Feng, L. 2018. Insights on trans- Chandra, R.; Gupta, A.; Ong, Y.-S.; and Goh, C.-K. 2016. fer optimization: Because experience is the best teacher. Evolutionary multi-task learning for modular training of IEEE Transactions on Emerging Topics in Computational feedforward neural networks. In Proceedings of Interna- Intelligence 2(1):51–64. tional Conference on Neural Information Processing. Hansen, N. 2006. The CMA evolution strategy: A com- Cheng, M.-Y.; Gupta, A.; Ong, Y.-S.; and Ni, Z.-W. 2017. paring review. In Towards a new evolutionary computation. Coevolutionary multitasking for concurrent global optimiza- Advances in estimation of distribution algorithms. Springer. tion: With case studies in complex engineering design. En- 75–102. gineering Applications of Artificial Intelligence 64:13–24. Hashimoto, R.; Ishibuchi, H.; Masuyama, N.; and Nojima, Darwin, C. 1859. The Origin of Species. John Murray, Y. 2018. Analysis of evolutionary multi-tasking as an is- London. land model. In Proceedings of the Genetic and Evolutionary Ding, J.-L.; Yang, C.; Jin, Y.; and Chai, T.-Y. 2017. Gener- Computation Conference Companion, 1894–1897. alized multi-tasking for evolutionary optimization of expen- Hauschild, M., and Pelikan, M. 2011. An introduction and sive problems. IEEE Transactions on Evolutionary Compu- survey of estimation of distribution algorithms. Swarm and tation. Evolutionary Computation 1(3):111–128. Dorigo, M.; Maniezzo, V.; and Colorni, A. 1996. Ant sys- Holland, J. 1975. Adaptation in Natural and Artificial Sys- tem: Optimization by a colony of cooperating agents. IEEE tems. University of Michigan Press.

4302 Li, G.-H.; Zhang, Q.; and Gao, W.-F. 2018. Multipopula- tion evolution framework for multifactorial optimization. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, 215–216. Liaw, R.-T., and Ting, C.-K. 2017. Evolutionary many- tasking based on biocoenosis through symbiosis: A framework and benchmark problems. In Proceedings of IEEE Congress on Evolutionary Computation, 2266–2273. Ong, Y.-S., and Gupta, A. 2016. Evolutionary multitasking: A computer science view of cognitive multitasking. Cogni- tive Computation 8(2):125–142. Sagarna, R., and Ong, Y.-S. 2016. Concurrently searching branches in software tests generation through multitask evolution. In Proceedings of IEEE Symposium Series on Com- putational Intelligence. Schwefel, H.-P. 1995. Evolution and Optimum Seeking. John Wiley & Sons, Inc. Strasser, S.; Sheppard, J.; Fortier, N.; and Goodman, R. 2016. Factored evolutionary algorithms. IEEE Transactions on Evolutionary Computation. Wen, Y.-W., and Ting, C.-K. 2016. Learning ensemble of decision trees through multifactorial genetic programming. In Proceedings of IEEE Congress on Evolutionary Compu- tation, 5293–5300. Wen, Y.-W., and Ting, C.-K. 2017. Parting ways and re- allocating resources in evolutionary multitasking. In Pro- ceedings of IEEE Congress on Evolutionary Computation, 2404–2411. Zhou, L.; Feng, L.; Zhong, J.; Ong, Y.-S.; Zhu, Z.; and Sha, E. 2016. Evolutionary multitasking in combinatorial search spaces: A case study in capacitated vehicle routing problem. In Proceedings of IEEE Symposium Series on Com- putational Intelligence.

4303