Improved Step Size Adaptation for the MO-CMA-ES Thomas Voß, Nikolaus Hansen, Christian Igel To cite this version: Thomas Voß, Nikolaus Hansen, Christian Igel. Improved Step Size Adaptation for the MO-CMA-ES. Genetic And Evolutionary Computation Conference, Jul 2010, Portland, United States. pp.487-494, 10.1145/1830483.1830573. hal-00503251 HAL Id: hal-00503251 https://hal.archives-ouvertes.fr/hal-00503251 Submitted on 18 Jul 2010 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Improved Step Size Adaptation for the MO-CMA-ES Thomas Voß Nikolaus Hansen Christian Igel Institut für Neuroinformatik Université de Paris-Sud Institut für Neuroinformatik Ruhr-Universität Bochum Centre de recherche INRIA Ruhr-Universität Bochum 44780 Bochum, Germany Saclay – Íle-de-France 44780 Bochum, Germany [email protected] F-91405 Orsay Cedex, France [email protected] [email protected] ABSTRACT 1. INTRODUCTION The multi-objective covariance matrix adaptation evolution The multi-objective covariance matrix adaptation evolu- strategy (MO-CMA-ES) is an evolutionary algorithm for tion strategy (MO-CMA-ES, [14, 16, 19]) is an extension continuous vector-valued optimization. It combines indica- of the CMA-ES [12, 11] for real-valued multi-objective opti- tor-based selection based on the contributing hypervolume mization. It combines the mutation and strategy adaptation with the efficient strategy parameter adaptation of the elitist of the (1+1)-CMA-ES [14, 15, 19] with a multi-objective se- covariance matrix adaptation evolution strategy (CMA-ES). lection procedure based on non-dominated sorting [6] and Step sizes (i.e., mutation strengths) are adapted on indivi- the contributing hypervolume [2] acting on a population of dual-level using an improved implementation of the 1/5-th individuals. success rule. In the original MO-CMA-ES, a mutation is In the MO-CMA-ES, step sizes (i.e., mutation strengths) regarded as successful if the offspring ranks better than its are adapted on individual-level. The step size update pro- parent in the elitist, rank-based selection procedure. In con- cedure originates in the well-known 1/5-th rule originally trast, we propose to regard a mutation as successful if the presented by [18] and extended by [17]. If the success rate, offspring is selected into the next parental population. This that is, the fraction of successful mutations, is high, the step criterion is easier to implement and reduces the computa- size is increased, otherwise it is decreased. In the original tional complexity of the MO-CMA-ES, in particular of its MO-CMA-ES, a mutation is regarded as successful if the re- steady-state variant. The new step size adaptation improves sulting offspring is better than its parent. In this study, we the performance of the MO-CMA-ES as shown empirically propose to replace this criterion and to consider a mutation using a large set of benchmark functions. The new update as being successful if the offspring becomes a member of the scheme in general leads to larger step sizes and thereby coun- next parent population. We argue that this notion of success teracts premature convergence. The experiments comprise is easier to implement, computationally less demanding, and the first evaluation of the MO-CMA-ES for problems with improves the performance of the MO-CMA-ES. more than two objectives. In the next section, we briefly review the MO-CMA-ES. In Sec. 3, we discuss our new notion of success for the step Categories and Subject Descriptors size adaptation. Then, we empirically evaluate the resulting algorithms. In this evaluation, the MO-CMA-ES is for the G.1.6 [Optimization]: Global Optimization; I.2.8 [Problem first time benchmarked on functions with more than two Solving, Control Methods, and Search]: Heuristic meth- objectives. As a baseline, we consider a new variant of the ods NSGA-II, in which the crowding distance is replaced by the contributing hypervolume for sorting individuals at the same General Terms level of non-dominance. Algorithms, Performance 2. THE MO-CMA-ES Keywords In the following, we briefly outline the MO-CMA-ES ac- multi-objective optimization, step size adaptation, covari- cording to [14, 16, 19], see Algorithm 1. For a detailed ance matrix adaptation, evolution strategy, MO-CMA-ES description and a performance evaluation on bi-objective benchmark functions we refer to [14, 21]. We consider ob- n m T jective functions f : R R , x (f1(x), . , fm(x)) . → 7→ (g) In the MO-CMA-ES, a candidate solution ai in generation g is a tuple x(g), p¯(g) , σ(g), p(g), C(g) , where x(g) Rn is i succ,i i i,c i i ∈ Permission to make digital or hard copies of all or part of this work for (g) the currenth search point,p ¯ ,i [0, 1] isi the smoothed suc- personal or classroom use is granted without fee provided that copies are succ ∈ cess probability, σ(g) R+ is the global step size, p(g) Rn not made or distributed for profit or commercial advantage and that copies i ∈ 0 i,c ∈ bear this notice and the full citation on the first page. To copy otherwise, to (g) n×n is the cumulative evolution path, Ci R is the covari- republish, to post on servers or to redistribute to lists, requires prior specific ance matrix of the search distribution.∈ For an individual permission and/or a fee. GECCO’10, July 7–11, 2010, Portland, Oregon, USA. a encoding search point x, we write f(a) for f(x) with a Copyright 2010 ACM 978-1-4503-0072-8/10/07 ...$10.00. slight abuse of notation. We first describe the general ranking procedure and sum- Algorithm 1: (µ +λ)-MO-CMA-ES marize the other parts of the MO-CMA-ES. The MO-CMA- ES relies on the non-dominated sorting selection scheme [6]. 1 g 0, initialize parent population Q(0); As in the SMS-EMOA [2], the hypervolume-indicator serves 2 repeat← as second-level sorting criterion to rank individuals at the 3 for k = 1,...,λ do (g) same level of non-dominance. Let A be a population, and let 4a ik 1, ndom Q ; a, a′ be two individuals in A. Let the non-dominated solu- ←U | | ′ ′ 4b ik k;“ “ ” ” tions in A be denoted by ndom(A) = a A ∄a A : a ←g g { ∈ ∈ ≺ 5 a′( +1) a( ) ; a , where denotes the Pareto-dominance relation. The k ← ik } ≺ ˛ ′(g+1) (g) (g) (g) elements in ndom(A) are assigned a level of non-dominance˛ 6 x x + σ 0, C ; k ik ik ik of 1. The other ranks of non-dominance are defined recur- ∼ N 7 Q(g) Q(g) a′(g+1) “; ” sively by considering the set A without the solutions with ← ∪ k lower ranks [6]. Formally, let dom0(A) = A, doml(A) = 8 for k = 1,...,λ don o doml−1(A) ndoml(A), and ndoml(A) = ndom(doml−1(A)) (g+1) \ 9 p¯′ for l 1. For a A we define the level of non-dominance succ,k ← ≥ ∈ ′(g+1) (g) ′(g+1) rank(a, A) to be i iff a ndomi(A). (1 cp)p¯ + cp succ (g) a , a ; ∈ − succ,k Q ik k The hypervolume measure or -metric was introduced p¯′(g+1)“ −ptarget ” S 10 ′(g+1) ′(g+1) 1 succ,k succ in the domain of evolutionary multi-objective optimization σ k σ k exp d target ; ← 1−psucc (MOO) in [26]. It is defined as „ « 11 ¯′(g+1) if p succ,k < pthresh then 12 ′(g+1) ref ref p c,k f ref (A)=Λ f1(a), f1 fm(a), fm , ← ′ S ×···× x (g+1)−x(g) a∈A ! ′(g+1) k ik h i h i (1 cc) p + cc(2 cc) ; [ c,k σ(g) (1) − − ik ref m with f R referring to an appropriately chosen refer- 13 ′(g+1) p ∈ C k ence point and Λ( ) being the Lebesgue measure. The con- ← ′(g+1) ′(g+1) ′(g+1)T · ′ C p p tributing hypervolume of a point a A = ndom(A) is given (1 ccov) k + ccov c,k c,k ; 14 − by ∈ else ′ ′ ′ ′ g ′ g ∆S (a, A ) = f ref (A ) f ref (A a ) . (2) 15 ( +1) ( +1) S − S \{ } p c,k (1 cc) p c,k ; ′ ← − Now we define the contribution rank cont(a, A ) of a. This is 16 ′(g+1) ′(g+1) C k (1 ccov) C k + again done recursively. The element, say a, with the smallest ′(←g+1) −′(g+1)T ′(g+1) ccov p p + cc (2 cc) C ; contributing hypervolume is assigned contribution rank 1. c,k c,k − k ′ The next rank is assigned by considering A a etc. More (g) “ (g) (g) ′(g+1)” 17 p¯ (1 cp)¯p + cp succ (g) a , a ; ′ ′ \{ } ik ik Q ik k precisely, let c0(A ) = argmina∈A′ ∆S (a, A ) and ← − p(g) −ptarget (g) (g) ¯succ,i succ “ ” − 18 1 k i 1 σi σi exp d target ; ′ ′ ′ k ← k 1−psucc ci(A ) = c0 A cj (A ) (3) „ « \ 19 g g + 1; j=0 ! ← [ ˘ ¯ (g) (g−1) 20 Q Q≺ 1 i µ ; for i > 0.