Mixed Integer Evolution Strategies for Parameter Optimization Rui Li [email protected] Natural Computing Group, Leiden Institute of Advanced Computer Science, Leiden University, Leiden, 2333 CA, The Netherlands Michael T.M. Emmerich [email protected] Natural Computing Group, Leiden Institute of Advanced Computer Science, Leiden University, Leiden, 2333 CA, The Netherlands Jeroen Eggermont [email protected] Division of Image Processing, Department of Radiology C2S, Leiden University Medical Center, Leiden, 2300 RC, The Netherlands Thomas Back¨ [email protected] Natural Computing Group, Leiden Institute of Advanced Computer Science, Leiden University, Leiden, 2333 CA, The Netherlands M. Schutz¨ [email protected] Blumenthalstraße 14, 69120 Heidelberg, Germany J. Dijkstra [email protected] Division of Image Processing, Department of Radiology C2S, Leiden University Medical Center, Leiden, 2300 RC, The Netherlands J.H.C. Reiber [email protected] Division of Image Processing, Department of Radiology C2S, Leiden University Medical Center, Leiden, 2300 RC, The Netherlands Abstract Evolution strategies (ESs) are powerful probabilistic search and optimization algo- rithms gleaned from biological evolution theory. They have been successfully applied to a wide range of real world applications. The modern ESs are mainly designed for solving continuous parameter optimization problems. Their ability to adapt the param- eters of the multivariate normal distribution used for mutation during the optimization run makes them well suited for this domain. In this article we describe and study mixed integer evolution strategies (MIES), which are natural extensions of ES for mixed inte- ger optimization problems. MIES can deal with parameter vectors consisting not only of continuous variables but also with nominal discrete and integer variables. Following the design principles of the canonical evolution strategies, they use specialized muta- tion operators tailored for the aforementioned mixed parameter classes. For each type of variable, the choice of mutation operators is governed by a natural metric for this variable type, maximal entropy, and symmetry considerations. All distributions used for mutation can be controlled in their shape by means of scaling parameters, allowing self-adaptation to be implemented. After introducing and motivating the conceptual design of the MIES, we study the optimality of the self-adaptation of step sizes and mutation rates on a generalized (weighted) sphere model. Moreover, we prove global C 2013 by the Massachusetts Institute of Technology Evolutionary Computation 21(1): 29–64 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/EVCO_a_00059 by guest on 26 September 2021 R. Li et al. convergence of the MIES on a very general class of problems. The remainder of the article is devoted to performance studies on artificial landscapes (barrier functions and mixed integer NK landscapes), and a case study in the optimization of medical image analysis systems. In addition, we show that with proper constraint handling techniques, MIES can also be applied to classical mixed integer nonlinear programming problems. Keywords Evolution strategies, mixed integer evolution strategies, NK landscapes. 1 Introduction The classical problem formulation of continuous parameter optimization reads: min{f (x)|x ∈ M ⊆ Rn}. Aside from this, in integer programming and combinatorial optimization discrete search spaces are considered, such as Zn or Bn.Mostofthere- search in optimization theory is either focused on discrete search spaces or focused on continuous search spaces. However, there are many practical optimization problems from industry in which the set of decision variables involves continuous as well as dis- crete variables. These problems are classically referred to as mixed integer optimization problems (Dakin, 1965; Wolsey and Nemhauser, 1999). Classical techniques from mathematical programming have focused mainly on problems of which the structure is clearly defined by a set of mathematical expressions. For instance, mixed integer nonlinear programming (MINLP; Bussieck and Pruessner, 2003) is a natural approach of formulating problems where continuous and discrete parameters need to be optimized simultaneously. It is based on the assumption that the search space can efficiently be explored using a divide and conquer scheme. As opposed to these white box optimization problems, less attention has been paid to black box optimization problems where the structure of the objective function is not fully known. This type of problem became very apparent in applications where objective functions are based on large-scale simulation models. In this article, we discuss a promising heuristic approach for solving such problems, the so-called mixed integer evolution strategies (MIES). Evolution strategies (ESs; Back,¨ 1996; Rechenberg, 1994; Schwefel, 1995) are robust global optimization strategies that are frequently used for continuous optimization. They belong to the class of randomized search heuristics and are based on principles of biological evolution theory, such as selection, recombination, and mutation. As flexible and robust search techniques, they have been successfully applied to various real- world problems. The dynamic behavior of ESs was subject to thorough theoretical and empirical studies. For instance, Schwefel compared it to traditional direct optimization algorithms and Back¨ to other evolutionary algorithms. Theoretical studies of the conver- gence behavior of the ES were carried out for instance by Beyer (2001), Oyman (Oyman et al., 2000), and Rudolph (1997). The results indicate that the ES is a robust optimization tool that can deal with a large number of practically relevant function classes, including discontinuous and multimodal functions with many variables. This article discusses the MIES, an extension of the ES for the simultaneous opti- mization of continuous, integer, and nominal discrete parameters. It combines muta- tion operators of ESs in the continuous domain (Schwefel, 1995), for integer program- ming (Rudolph, 1994), and for binary search spaces (Back,¨ 1996). These operators have in common that they have certain desirable properties, such as symmetry, scalability, and maximal entropy, the details of which will be discussed later. The MIES was originally developed for optical filter optimization (Back;¨ Schutz¨ and Sprave, 1996), and chem- ical engineering plant optimization (Emmerich et al., 2000; Groß, 1999). Recently, as 30 Evolutionary Computation Volume 21, Number 1 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/EVCO_a_00059 by guest on 26 September 2021 Mixed Integer Evolution Strategies for Parameter Optimization discussed in this contribution, it has been used in the context of medical image analysis (Li, Emmerich, Bovenkamp et al., 2006; Bovenkamp et al., 2006). In the latter work, the MIES convergence behavior on various artificial landscapes was studied empirically, including a collection of single-peak landscapes (Emmerich et al., 2000) and landscapes with multiple peaks (Li, Emmerich, Eggermont, and Bovenkamp, 2006; Li, Emmerich, Eggermont, Bovenkamp, et al., 2006). This article summarizes and extends recent studies of MIES and their applications in various fields of science and technology. It extends the existing work by providing a self-contained detailed description of the algorithm and its motivation. Moreover, we provide theoretical studies on the optimality of the step size adaptation. A theorem on the convergence with probability one to the global optimum is derived for a very general class of functions. In addition, the article surveys existing applications of the MIES in systems design and extends empirical results by assessing the performance of the MIES on a set of classical MINLP benchmark problems. For the latter study, constraint handling methods are introduced into the framework. In Section 2, we provide a mathematical definition of mixed integer parameter op- timization and discuss methodologies applied for solving such problems. In Section 3, the MIES is introduced and we discuss its conceptual design. We study the behav- ior of the algorithm in artificial landscapes in Section 4. In addition, we introduce extensions of landscape models that are used typically either in binary optimization (NK-landscapes) or continuous optimization (barrier model). Furthermore, we apply MIES to the selected MINLP test problems by using constraint handling techniques, more specifically, penalty function methods. In Section 5, we report on the application of MIES to real-world problems. The article concludes with a summary discussion of the obtained results and discusses promising paths for future work. 2 Problem Definition and Related Work Many application problems from industry involve the simultaneous use of continuous, integer, and nominal discrete objective variables. The problem of mixed integer param- eter optimization can be formalized as follows: let r1,...,rnr denote a set of real-valued decision variables, z1,...,znz denote a set of integer decision variables, and d1,...,dnd denote a set of nominal discrete decision variables, each of which is taken from a finite domain. The finite domains for the nominal discrete variables will be denoted with D(1), ...,D(nd ). We do not encode nominal discrete variables as integers, in order to exploit the fact that there is no meaningful a priori ordering given for their domain. Fur- thermore, let f : Rnr × Znz × D(1) ×···×D(nd