DOCSLIB.ORG

CMA-ES and Advanced Adaptation Mechanisms Youhei Akimoto, Nikolaus Hansen

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
CMA-ES and Advanced Adaptation Mechanisms Youhei Akimoto, Nikolaus Hansen CMA-ES and Advanced Adaptation Mechanisms Youhei Akimoto, Nikolaus Hansen To cite this version: Youhei Akimoto, Nikolaus Hansen. CMA-ES and Advanced Adaptation Mechanisms. GECCO ’18 Companion: Proceedings of the Genetic and Evolutionary Computation Conference Companion, Jul 2018, Kyoto, Japan. hal-01959479 HAL Id: hal-01959479 https://hal.inria.fr/hal-01959479 Submitted on 18 Dec 2018 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. CMA-ES and Advanced Adaptation Mechanisms Youhei Akimoto1 & Nikolaus Hansen2 1. University of Tsukuba, Japan 2. Inria, Research Centre Saclay, France [email protected] [email protected] Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author. GECCO '18 Companion, July 15–19, 2018, Kyoto, Japan © 2018 Copyright is held by the owner/author(s). ACM ISBN 978-1-4503-5764-7/18/07. https://doi.org/10.1145/3205651.3207854 1 Tutorial: Evolution Strategies and CMA-ES (Covariance Matrix Adaptation) Anne Auger & Nikolaus Hansen Inria We are happy to answerProject team questions TAO at any time. Research Centre Saclay – ˆIle-de-France University Paris-Sud, LRI (UMR 8623), Bat. 660 91405 ORSAY Cedex, France https://www.lri.fr/~hansen/gecco2014-CMA-ES-tutorial.pdf Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author. Copyright is held by the owner/author(s). GECCO ’14, Jul 12-16 2014, Vancouver, BC, Canada ACM 978-1-4503-2881-4/14/07. http://dx.doi.org/10.1145/2598394.2605347 2 Anne Auger & Nikolaus Hansen CMA-ES July, 2014 1 / 81 Topics 1. What makes the problem difficult to solve? 2. How does the CMA-ES work? • Normal Distribution, Rank-Based Recombination • Step-Size Adaptation • Covariance Matrix Adaptation 3. What can/should the users do for the CMA-ES to work effectively on their problem? • Choice of problem formulation and encoding (not covered) • Choice of initial solution and initial step-size • Restarts, Increasing Population Size • Restricted Covariance Matrix 3 Topics 1. What makes the problem difficult to solve? 2. How does the CMA-ES work? • Normal Distribution, Rank-Based Recombination • Step-Size Adaptation • Covariance Matrix Adaptation 3. What can/should the users do for the CMA-ES to work effectively on their problem? • Choice of problem formulation and encoding (not covered) • Choice of initial solution and initial step-size • Restarts, Increasing Population Size • Restricted Covariance Matrix 4 Problem Statement Black Box Optimization and Its Difficulties Problem Statement Continuous Domain Search/Optimization Task: minimize an objective function (fitness function, loss function) in continuous domain n f : R R, x f (x) X ✓ ! 7! Black Box scenario (direct search scenario) x f(x) I gradients are not available or not useful I problem domain specific knowledge is used only within the black box, e.g. within an appropriate encoding Search costs: number of function evaluations 5 Anne Auger & Nikolaus Hansen CMA-ES July, 2014 3 / 81 Problem Statement Black Box Optimization and Its Difficulties Problem Statement Continuous Domain Search/Optimization Goal I fast convergence to the global optimum . or to a robust solution x I solution x with small function value f (x) with least search cost there are two conflicting objectives Typical Examples I shape optimization (e.g. using CFD) curve fitting, airfoils I model calibration biological, physical I parameter calibration controller, plants, images Problems I exhaustive search is infeasible I naive random search takes too long I deterministic search is not successful / takes too long Approach: stochastic search, Evolutionary Algorithms 6 Anne Auger & Nikolaus Hansen CMA-ES July, 2014 4 / 81 Problem Statement Black Box Optimization and Its Difficulties What Makes a Function Difficult to Solve? Why stochastic search? non-linear, non-quadratic, non-convex on linear and quadratic functions much better search policies are available 100 ruggedness 90 80 70 non-smooth, discontinuous, multimodal, and/or 60 50 noisy function 40 30 20 10 0 −4 −3 −2 −1 0 1 2 3 4 dimensionality (size of search space) 3 (considerably) larger than three 2 1 non-separability 0 −1 dependencies between the objective variables −2 −3 ill-conditioning −3 −2 −1 0 1 2 3 non-smooth level sets gradient direction Newton direction 7 Anne Auger & Nikolaus Hansen CMA-ES July, 2014 6 / 81 Problem Statement Black Box Optimization and Its Difficulties Ruggedness non-smooth, discontinuous, multimodal, and/or noisy 100 90 80 70 60 50 Fitness 40 30 20 10 0 −4 −3 −2 −1 0 1 2 3 4 cut from a 5-D example, (easily) solvable with evolution strategies 8 Anne Auger & Nikolaus Hansen CMA-ES July, 2014 7 / 81 Problem Statement Non-Separable Problems Separable Problems Definition (Separable Problem) A function f is separable if arg min f (x1,...,xn)= arg min f (x1,...),...,arg min f (...,xn) (x ,...,x ) x1 xn 1 n ✓ ◆ it follows that f can be optimized in a sequence of n independent ) 1-D optimization processes Example: Additively 3 decomposable functions 2 1 n 0 f (x1,...,xn)= fi(xi) −1 Xi=1 Rastrigin function −2 −3 −3 −2 −1 0 1 2 3 9 Anne Auger & Nikolaus Hansen CMA-ES July, 2014 9 / 81 Problem Statement Non-Separable Problems Non-Separable Problems Building a non-separable problem from a separable one (1,2) Rotating the coordinate system f : x f (x) separable 7! f : x f (Rx) non-separable 7! R rotation matrix 3 3 2 2 1 R 1 0 0 ! −1 −1 −2 −2 −3 −3 −3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3 1 Hansen, Ostermeier, Gawelczyk (1995). On the adaptation of arbitrary normal mutation distributions in evolution strategies: The generating set adaptation. Sixth ICGA, pp. 57-64, Morgan Kaufmann 2 Salomon (1996). ”Reevaluating Genetic Algorithm Performance under Coordinate Rotation of Benchmark Functions; A survey of some theoretical and practical aspects of genetic algorithms.” BioSystems, 39(3):263-278 10 Anne Auger & Nikolaus Hansen CMA-ES July, 2014 10 / 81 Problem Statement Ill-Conditioned Problems Ill-Conditioned Problems Curvature of level sets Consider the convex-quadratic function 1 T 1 2 1 f (x)= 2 (x x⇤) H(x x⇤)= 2 i hi,i (xi xi⇤) + 2 i=j hi,j (xi xi⇤)(xj xj⇤) − − − 6 − − H is Hessian matrix of f and symmetric positive definite P P gradient direction f (x)T − 0 Newton direction H 1f (x)T − − 0 Ill-conditioning means squeezed level sets (high curvature). Condition number equals nine here. Condition numbers up to 1010 are not unusual in real world problems. If H I (small condition number of H) first order information (e.g. the ⇡ gradient) is sufficient. Otherwise second order information (estimation 1 of H− ) is necessary. 11 Anne Auger & Nikolaus Hansen CMA-ES July, 2014 11 / 81 Section Subsection Non-smooth level sets (sharp ridges) Similar difficulty but worse than ill-conditioning 1-norm scaled 1-norm 1/2-norm 12 Problem Statement Ill-Conditioned Problems What Makes a Function Difficult to Solve? . and what can be done The Problem Possible Approaches Dimensionality exploiting the problem structure separability, locality/neighborhood, encoding Ill-conditioning second order approach changes the neighborhood metric Ruggedness non-local policy, large sampling width (step-size) as large as possible while preserving a reasonable convergence speed population-based method, stochastic, non-elitistic recombination operator serves as repair mechanism restarts ...metaphors 13 Anne Auger & Nikolaus Hansen CMA-ES July, 2014 12 / 81 Topics 1. What makes the problem difficult to solve? 2. How does the CMA-ES work? • Normal Distribution, Rank-Based Recombination • Step-Size Adaptation • Covariance Matrix Adaptation 3. What can/should the users do for the CMA-ES to work effectively on their problem? • Choice of problem formulation and encoding (not covered) • Choice of initial solution and initial step-size • Restarts, Increasing Population Size • Restricted Covariance Matrix 14 Evolution Strategies (ES) A Search Template Stochastic Search A black box search template to minimize f : Rn R ! Initialize distribution parameters ✓, set population size λ N 2 While not terminate n 1 Sample distribution P (x ✓) x1,...,xλ R | ! 2 2 Evaluate x1,...,xλ on f 3 Update parameters ✓ F✓(✓, x1,...,xλ, f (x1),...,f (xλ)) Everything depends on the definition of P and F✓ deterministic algorithms are covered as well In many Evolutionary Algorithms the distribution P is implicitly defined via operators on a population, in particular, selection, recombination and mutation Natural template for (incremental) Estimation
Load more

Recommended publications
  • Stochastic Optimization from Distributed, Streaming Data in Rate-Limited Networks Matthew Nokleby, Member, IEEE, and Waheed U
    1 Stochastic Optimization from Distributed, Streaming Data in Rate-limited Networks Matthew Nokleby, Member, IEEE, and Waheed U. Bajwa, Senior Member, IEEE T Abstract—Motivated by machine learning applications in T training samples fξ(t) 2 Υgt=1 drawn independently from networks of sensors, internet-of-things (IoT) devices, and au- the distribution P (ξ), which is supported on a subset of Υ. tonomous agents, we propose techniques for distributed stochastic There are, in particular, two main categorizations of training convex learning from high-rate data streams. The setup involves a network of nodes—each one of which has a stream of data data that, in turn, determine the types of methods that can be arriving at a constant rate—that solve a stochastic convex used to find approximate solutions to the SO problem. These optimization problem by collaborating with each other over rate- are (i) batch training data and (ii) streaming training data. limited communication links. To this end, we present and analyze In the case of batch training data, where all T samples two algorithms—termed distributed stochastic approximation fξ(t)g are pre-stored and simultaneously available, a common mirror descent (D-SAMD) and accelerated distributed stochastic approximation mirror descent (AD-SAMD)—that are based on strategy is sample average approximation (SAA) (also referred two stochastic variants of mirror descent and in which nodes to as empirical risk minimization (ERM)), in which one collaborate via approximate averaging of the local, noisy subgra- minimizes the empirical average of the “risk” function φ(·; ·) dients using distributed consensus. Our main contributions are in lieu of the true expectation.
    [Show full text]
  • Metaheuristics1
    METAHEURISTICS1 Kenneth Sörensen University of Antwerp, Belgium Fred Glover University of Colorado and OptTek Systems, Inc., USA 1 Definition A metaheuristic is a high-level problem-independent algorithmic framework that provides a set of guidelines or strategies to develop heuristic optimization algorithms (Sörensen and Glover, To appear). Notable examples of metaheuristics include genetic/evolutionary algorithms, tabu search, simulated annealing, and ant colony optimization, although many more exist. A problem-specific implementation of a heuristic optimization algorithm according to the guidelines expressed in a metaheuristic framework is also referred to as a metaheuristic. The term was coined by Glover (1986) and combines the Greek prefix meta- (metá, beyond in the sense of high-level) with heuristic (from the Greek heuriskein or euriskein, to search). Metaheuristic algorithms, i.e., optimization methods designed according to the strategies laid out in a metaheuristic framework, are — as the name suggests — always heuristic in nature. This fact distinguishes them from exact methods, that do come with a proof that the optimal solution will be found in a finite (although often prohibitively large) amount of time. Metaheuristics are therefore developed specifically to find a solution that is “good enough” in a computing time that is “small enough”. As a result, they are not subject to combinatorial explosion – the phenomenon where the computing time required to find the optimal solution of NP- hard problems increases as an exponential function of the problem size. Metaheuristics have been demonstrated by the scientific community to be a viable, and often superior, alternative to more traditional (exact) methods of mixed- integer optimization such as branch and bound and dynamic programming.
    [Show full text]
  • Removing the Genetics from the Standard Genetic Algorithm
    Removing the Genetics from the Standard Genetic Algorithm Shumeet Baluja Rich Caruana School of Computer Science School of Computer Science Carnegie Mellon University Carnegie Mellon University Pittsburgh, PA 15213 Pittsburgh, PA 15213 [email protected] [email protected] Abstract from both parents into their respective positions in a mem- ber of the subsequent generation. Due to random factors involved in producing “children” chromosomes, the chil- We present an abstraction of the genetic dren may, or may not, have higher fitness values than their algorithm (GA), termed population-based parents. Nevertheless, because of the selective pressure incremental learning (PBIL), that explicitly applied through a number of generations, the overall trend maintains the statistics contained in a GA’s is towards higher fitness chromosomes. Mutations are population, but which abstracts away the used to help preserve diversity in the population. Muta- crossover operator and redefines the role of tions introduce random changes into the chromosomes. A the population. This results in PBIL being good overview of GAs can be found in [Goldberg, 1989] simpler, both computationally and theoreti- [De Jong, 1975]. cally, than the GA. Empirical results reported elsewhere show that PBIL is faster Although there has recently been some controversy in the and more effective than the GA on a large set GA community as to whether GAs should be used for of commonly used benchmark problems. static function optimization, a large amount of research Here we present results on a problem custom has been, and continues to be, conducted in this direction. designed to benefit both from the GA’s cross- [De Jong, 1992] claims that the GA is not a function opti- over operator and from its use of a popula- mizer, and that typical GAs which are used for function tion.
    [Show full text]
  • Genetic Algorithm: Reviews, Implementations, and Applications
    Paper— Genetic Algorithm: Reviews, Implementation and Applications Genetic Algorithm: Reviews, Implementations, and Applications Tanweer Alam() Faculty of Computer and Information Systems, Islamic University of Madinah, Saudi Arabia [email protected] Shamimul Qamar Computer Engineering Department, King Khalid University, Abha, Saudi Arabia Amit Dixit Department of ECE, Quantum School of Technology, Roorkee, India Mohamed Benaida Faculty of Computer and Information Systems, Islamic University of Madinah, Saudi Arabia How to cite this article? Tanweer Alam. Shamimul Qamar. Amit Dixit. Mohamed Benaida. " Genetic Al- gorithm: Reviews, Implementations, and Applications.", International Journal of Engineering Pedagogy (iJEP). 2020. Abstract—Nowadays genetic algorithm (GA) is greatly used in engineering ped- agogy as an adaptive technique to learn and solve complex problems and issues. It is a meta-heuristic approach that is used to solve hybrid computation chal- lenges. GA utilizes selection, crossover, and mutation operators to effectively manage the searching system strategy. This algorithm is derived from natural se- lection and genetics concepts. GA is an intelligent use of random search sup- ported with historical data to contribute the search in an area of the improved outcome within a coverage framework. Such algorithms are widely used for maintaining high-quality reactions to optimize issues and problems investigation. These techniques are recognized to be somewhat of a statistical investigation pro- cess to search for a suitable solution or prevent an accurate strategy for challenges in optimization or searches. These techniques have been produced from natu- ral selection or genetics principles. For random testing, historical information is provided with intelligent enslavement to continue moving the search out from the area of improved features for processing of the outcomes.
    [Show full text]
  • A Sequential Hybridization of Genetic Algorithm and Particle Swarm Optimization for the Optimal Reactive Power Flow
    sustainability Article A Sequential Hybridization of Genetic Algorithm and Particle Swarm Optimization for the Optimal Reactive Power Flow Imene Cherki 1,* , Abdelkader Chaker 1, Zohra Djidar 1, Naima Khalfallah 1 and Fadela Benzergua 2 1 SCAMRE Laboratory, ENPO-MA National Polytechnic School of Oran Maurice Audin, Oran 31000, Algeria 2 Departments of Electrical Engineering, University of Science and Technology of Oran Mohamed Bodiaf, Oran 31000, Algeria * Correspondence: [email protected] Received: 21 June 2019; Accepted: 12 July 2019; Published: 16 July 2019 Abstract: In this paper, the problem of the Optimal Reactive Power Flow (ORPF) in the Algerian Western Network with 102 nodes is solved by the sequential hybridization of metaheuristics methods, which consists of the combination of both the Genetic Algorithm (GA) and the Particle Swarm Optimization (PSO). The aim of this optimization appears in the minimization of the power losses while keeping the voltage, the generated power, and the transformation ratio of the transformers within their real limits. The results obtained from this method are compared to those obtained from the two methods on populations used separately. It seems that the hybridization method gives good minimizations of the power losses in comparison to those obtained from GA and PSO, individually, considered. However, the hybrid method seems to be faster than the PSO but slower than GA. Keywords: reactive power flow; metaheuristic methods; metaheuristic hybridization; genetic algorithm; particles swarms; electrical network 1. Introduction The objective of any company producing and distributing electrical energy is to ensure that the required power is available at all points and at all times.
    [Show full text]
  • Stochastic Optimization
    Stochastic Optimization Marc Schoenauer DataScience @ LHC Workshop 2015 Content 1 Setting the scene 2 Continuous Optimization 3 Discrete/Combinatorial Optimization 4 Non-Parametric Representations 5 Conclusion Content 1 Setting the scene 2 Continuous Optimization 3 Discrete/Combinatorial Optimization 4 Non-Parametric Representations 5 Conclusion Stochastic Optimization Hypotheses Search Space Ω with some topological structure Objective function F assume some weak regularity Hill-Climbing Randomly draw x0 2 Ω and compute F(x0) Initialisation Until(happy) y = Best neighbor(xt) neighbor structure on Ω Compute F(y) If F(y) F (xt) then xt+1 = y accept if improvement else xt+1 = xt Comments Find closest local optimum defined by neighborhood structure Iterate from different x0's Until(very happy) Stochastic Optimization Hypotheses Search Space Ω with some topological structure Objective function F assume some weak regularity Stochastic (Local) Search Randomly draw x0 2 Ω and compute F(x0) Initialisation Until(happy) y = Random neighbor(xt) neighbor structure on Ω Compute F(y) If F(y) F (xt) then xt+1 = y accept if improvement else xt+1 = xt Comments Find one close local optimum defined by neighborhood structure Iterate, leaving current optimum IteratedLocalSearch Stochastic Optimization Hypotheses Search Space Ω with some topological structure Objective function F assume some weak regularity Stochastic (Local) Search { alternative viewpoint Randomly draw x0 2 Ω and compute F(x0) Initialisation Until(happy) y = Move(xt) stochastic variation on
    [Show full text]
  • Adaptive Stochastic Optimization
    Industrial and Systems Engineering Adaptive Stochastic Optimization Frank E. Curtis1 and Katya Scheinberg2 1Department of Industrial and Systems Engineering, Lehigh University 2Department of Operations Research and Information Engineering, Cornell University ISE Technical Report 20T-001 Adaptive Stochastic Optimization The purpose of this article is to summarize re- Frank E. Curtis Katya Scheinberg cent work and motivate continued research on ◦ the design and analysis of adaptive stochastic op- January 18, 2020 timization methods. In particular, we present an analytical framework—new for the context Abstract. Optimization lies at the heart of of adaptive deterministic optimization—that sets machine learning and signal processing. Contem- the stage for establishing convergence rate guar- porary approaches based on the stochastic gradi- antees for adaptive stochastic optimization tech- ent method are non-adaptive in the sense that niques. With this framework in hand, we remark their implementation employs prescribed param- on important open questions related to how it eter values that need to be tuned for each appli- can be extended further for the design of new cation. This article summarizes recent research methods. We also discuss challenges and oppor- and motivates future work on adaptive stochastic tunities for their use in real-world systems. optimization methods, which have the potential to offer significant computational savings when 2 Background training large-scale systems. Many problems in ML and SP are formulated as optimization problems. For example, given a 1 Introduction data vector y Rm from an unknown distribu- tion, one often∈ desires to have a vector of model The successes of stochastic optimization algo- parameters x Rn such that a composite objec- ∈ rithms for solving problems arising machine tive function f : Rn R is minimized, as in learning (ML) and signal processing (SP) are → now widely recognized.
    [Show full text]
  • An Improved Adaptive Genetic Algorithm for Two-Dimensional Rectangular Packing Problem
    applied sciences Article An Improved Adaptive Genetic Algorithm for Two-Dimensional Rectangular Packing Problem Yi-Bo Li 1, Hong-Bao Sang 1,* , Xiang Xiong 1 and Yu-Rou Li 2 1 State Key Laboratory of Precision Measuring Technology and Instruments, Tianjin University, Tianjin 300072, China; [email protected] (Y.-B.L.); [email protected] (X.X.) 2 Canterbury School, 101 Aspetuck Ave, New Milford, CT 06776, USA; [email protected] * Correspondence: [email protected] Abstract: This paper proposes the hybrid adaptive genetic algorithm (HAGA) as an improved method for solving the NP-hard two-dimensional rectangular packing problem to maximize the filling rate of a rectangular sheet. The packing sequence and rotation state are encoded in a two- stage approach, and the initial population is constructed from random generation by a combination of sorting rules. After using the sort-based method as an improved selection operator for the hybrid adaptive genetic algorithm, the crossover probability and mutation probability are adjusted adaptively according to the joint action of individual fitness from the local perspective and the global perspective of population evolution. The approach not only can obtain differential performance for individuals but also deals with the impact of dynamic changes on population evolution to quickly find a further improved solution. The heuristic placement algorithm decodes the rectangular packing sequence and addresses the two-dimensional rectangular packing problem through continuous iterative optimization. The computational results of a wide range of benchmark instances from zero- waste to non-zero-waste problems show that the HAGA outperforms those of two adaptive genetic algorithms from the related literature.
    [Show full text]
  • Conservative Stochastic Optimization with Expectation Constraints
    1 Conservative Stochastic Optimization with Expectation Constraints Zeeshan Akhtar,? Amrit Singh Bedi,y and Ketan Rajawat? Abstract This paper considers stochastic convex optimization problems where the objective and constraint functions involve expectations with respect to the data indices or environmental variables, in addition to deterministic convex constraints on the domain of the variables. Although the setting is generic and arises in different machine learning applications, online and efficient approaches for solving such problems have not been widely studied. Since the underlying data distribution is unknown a priori, a closed-form solution is generally not available, and classical deterministic optimization paradigms are not applicable. Existing approaches towards solving these problems make use of stochastic gradients of the objective and constraints that arrive sequentially over iterations. State-of-the-art approaches, such as those using the saddle point framework, are able to ensure that the optimality gap as well as the constraint violation − 1 decay as O T 2 where T is the number of stochastic gradients. The domain constraints are assumed simple and handled via projection at every iteration. In this work, we propose a novel conservative − 1 stochastic optimization algorithm (CSOA) that achieves zero average constraint violation and O T 2 optimality gap. Further, we also consider the scenario where carrying out a projection step onto the convex domain constraints at every iteration is not viable. Traditionally, the projection operation can be avoided by considering the conditional gradient or Frank-Wolfe (FW) variant of the algorithm. The state-of-the-art − 1 stochastic FW variants achieve an optimality gap of O T 3 after T iterations, though these algorithms have not been applied to problems with functional expectation constraints.
    [Show full text]
  • Monte Carlo Methods for Sampling-Based Stochastic Optimization
    Monte Carlo methods for sampling-based Stochastic Optimization Monte Carlo methods for sampling-based Stochastic Optimization Gersende FORT LTCI CNRS & Telecom ParisTech Paris, France Joint works with B. Jourdain, T. Leli`evre,G. Stoltz from ENPC and E. Kuhn from INRA. A. Schreck and E. Moulines from Telecom ParisTech P. Priouret from Paris VI Monte Carlo methods for sampling-based Stochastic Optimization Introduction Simulated Annealing Simulated Annealing (1/2) Let U denote the objective function one wants to minimize. U(x) min U(x) () max exp(−U(x)) () max exp − 8T > 0 x2X x2X x2X T In order to sample from πT? where U(x) π (x) = exp − T T sample successively from a sequence of tempered distributions πT1 , πT2 , ··· with T1 > T2 > ··· > T?. Monte Carlo methods for sampling-based Stochastic Optimization Introduction Simulated Annealing Simulated Annealing (1/2) Let U denote the objective function one wants to minimize. U(x) min U(x) () max exp(−U(x)) () max exp − 8T > 0 x2X x2X x2X T In order to sample from πT? where U(x) π (x) = exp − T T sample successively from a sequence of tempered distributions πT1 , πT2 , ··· with T1 > T2 > ··· > T?. or sample successively from a sequence of (nt-iterated) kernels (PTt (x; ·))t such that πT PT = πT . Monte Carlo methods for sampling-based Stochastic Optimization Introduction Simulated Annealing Simulated Annealing (2/2) Under conditions on X, on the cooling schedule (Tt)t, on the kernels (Pt)t, on the dominating measure and the set of minima, ··· Kirkpatrick, Gelatt and Vecchi. Optimization via Simulated Annealing. Science (1983) Geman and Geman.
    [Show full text]
  • Stochastic Optimization,” in Handbook of Computational Statistics: Concepts and Methods (2Nd Ed.) (J
    Citation: Spall, J. C. (2012), “Stochastic Optimization,” in Handbook of Computational Statistics: Concepts and Methods (2nd ed.) (J. Gentle, W. Härdle, and Y. Mori, eds.), Springer−Verlag, Heidelberg, Chapter 7, pp. 173–201. dx.doi.org/10.1007/978-3-642-21551-3_7 STOCHASTIC OPTIMIZATION James C. Spall The Johns Hopkins University Applied Physics Laboratory 11100 Johns Hopkins Road Laurel, Maryland 20723-6099 U.S.A. [email protected] Stochastic optimization algorithms have been growing rapidly in popularity over the last decade or two, with a number of methods now becoming “industry standard” approaches for solving challenging optimization problems. This chapter provides a synopsis of some of the critical issues associated with stochastic optimization and a gives a summary of several popular algorithms. Much more complete discussions are available in the indicated references. To help constrain the scope of this article, we restrict our attention to methods using only measurements of the criterion (loss function). Hence, we do not cover the many stochastic methods using information such as gradients of the loss function. Section 1 discusses some general issues in stochastic optimization. Section 2 discusses random search methods, which are simple and surprisingly powerful in many applications. Section 3 discusses stochastic approximation, which is a foundational approach in stochastic optimization. Section 4 discusses a popular method that is based on connections to natural evolution—genetic algorithms. Finally, Section 5 offers some concluding remarks. 1 Introduction 1.1 General Background Stochastic optimization plays a significant role in the analysis, design, and operation of modern systems. Methods for stochastic optimization provide a means of coping with inherent system noise and coping with models or systems that are highly nonlinear, high dimensional, or otherwise inappropriate for classical deterministic methods of optimization.
    [Show full text]
  • Contextual Covariance Matrix Adaptation Evolutionary Strategies
    Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI-17) Contextual Covariance Matrix Adaptation Evolutionary Strategies Abbas Abdolmaleki1;2;3, Bob Price5, Nuno Lau1, Luis Paulo Reis2;3, Gerhard Neumann4∗ 1: IEETA, DETI, University of Aveiro 2: DSI, University of Minho 3: LIACC, University of Porto, Portugal 4: CLAS, TU Darmstadt, Germany 5: PARC, A Xerox Company, USA Abstract the ball to its team mates which are positioned on differ- ent locations on the field. Here, the soccer robot should Many stochastic search algorithms are designed to learn to kick the ball to any given target location, which is optimize a fixed objective function to learn a task, specified by the context vector, on the field. In such cases, i.e., if the objective function changes slightly, for learning for every possible context is clearly inefficient or example, due to a change in the situation or con- even infeasible. Therefore our goal is to generalize learned text of the task, relearning is required to adapt to tasks from similar contexts to a new context. To do so, we the new context. For instance, if we want to learn learn a context-dependent policy for a continuous range of a kicking movement for a soccer robot, we have to contexts without restarting the learning process. In this pa- relearn the movement for different ball locations. per, we consider stochastic search algorithms for contextual Such relearning is undesired as it is highly inef- learning. Stochastic search algorithms [Hansen et al., 2003; ficient and many applications require a fast adap- Sun et al., 2009; Ruckstieߨ et al., 2008] optimize an objec- tation to a new context/situation.
    [Show full text]