DOCSLIB.ORG

Covariance Matrix Adapted Evolution Strategy (CMAES)

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Covariance Matrix Adapted Evolution Strategy (CMAES) View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Directory of Open Access Journals ISSN: 2229-6956(ONLINE) ICTACT JOURNAL ON SOFT COMPUTING, OCTOBER 2012, VOLUME: 03, ISSUE: 01 APPLICATION OF RESTART COVARIANCE MATRIX ADAPTATION EVOLUTION STRATEGY (RCMA-ES) TO GENERATION EXPANSION PLANNING PROBLEM K. Karthikeyan1, S. Kannan2, S. Baskar3, and C. Thangaraj4 1,2Department of Electrical and Electronics Engineering, Kalasalingam University, India E-mail: [email protected] and [email protected] 3Department of Electrical and Electronics Engineering, Thiagarajar College of Engineering, India E-mail: [email protected] 4Anna University of Technology Chennai, India E-mail: [email protected] Abstract The Covariance Matrix Adaptation Evolution Strategy This paper describes the application of an evolutionary algorithm, (CMA-ES) is a robust stochastic evolutionary algorithm for Restart Covariance Matrix Adaptation Evolution Strategy (RCMA- difficult non-linear non-convex optimization problems [15]. The ES) to the Generation Expansion Planning (GEP) problem. RCMA- CMA-ES is typically applied to unconstrained or bounded ES is a class of continuous Evolutionary Algorithm (EA) derived from constraint optimization problems, and it efficiently minimizes the concept of self-adaptation in evolution strategies, which adapts the unimodal objective functions and is in particularly superior for covariance matrix of a multivariate normal search distribution. The ill-conditioned and non-separable problems [15], [16]. Anne original GEP problem is modified by incorporating Virtual Mapping Procedure (VMP). The GEP problem of a synthetic test systems for 6- Auger and Nikolaus Hansen showed that increasing the year, 14-year and 24-year planning horizons having five types of population size improves the performance of the CMA-ES on candidate units is considered. Two different constraint-handling multi-modal functions [17]. They suggest a CMA-ES restart methods are incorporated and impact of each method has been strategy with successively increasing population size. In this compared. In addition, comparison and validation has also made with paper, CMA-ES with restart strategy is applied for solving GEP dynamic programming method. Problem with two different constraint handling schemes namely Penalty factor less constraint handling scheme (PFL) and Self- Keywords: Adaptive Penalty scheme (SAP). The paper is organized as Constraint Handling, Dynamic Programming, Generation Expansion follows: Section 2 describes the GEP problem formulation. Planning, Restart Covariance Matrix Adaptation Evolution Strategy Section 3 describes RCMA-ES implementation to the GEP and Virtual Mapping Procedure problem. Section 4 explains two different constraint-handling methods; Section 5 provides test results and section 6 concludes. 1. INTRODUCTION 2. GENERATION EXPANSION PLANNING The Generation Expansion Planning (GEP) problem seeks to PROBLEM FORMULATION identify which generating units should be commissioned and when they should become available over the long-term planning The GEP problem is equivalent to finding a set of best horizon [1], [2]. This GEP model is applicable for developing decision vectors over a planning horizon that minimizes the countries like India, where planning is coordinated by central or investment and operating costs under relevant constraints. state government owned utilities for capacity addition. The major objectives of GEP are to minimize the total investment 2.1 COST OBJECTIVE and the operating cost of the generating units, and to meet the The cost objective is represented by the following demand criteria, fuel-mix ratio, and the reliability criteria. GEP expression: is a constrained, nonlinear, discrete optimization problem. The solution for the GEP problem can be obtained, through complete T min Cost IUt M X t OX t SUt (1) enumeration in the entire planning horizon. Linear programming t1 and Dynamic Programming (DP) were applied to solve the GEP problem [3], [4]. The emerging techniques applied to solve GEP where, were reviewed in [5]. The improved Genetic Algorithm (GA) Xt Xt1 Ut , t 1,2,...T (2) was used to solve the GEP problem in [6]. The Evolutionary N 2t Programming (EP) technique with Gaussian mutation and IUt 1 d CIi Ut,i (3) quadratic approximation technique was applied to solve the GEP i1 problem [7]. The comprehensive survey of the GEP models can N T be found in [8]. Nine different meta-heuristic techniques were SUt 1 d CIi i Ut,i (4) applied to solve the GEP problem [9]. The Elitist Non- i1 1 dominated Sorting Genetic Algorithm version II (NSGA-II) is 1.5t's' applied to multi-objective GEP problem [10], [11]. The M X t 1 d X t FC MC (5) ' environmental constraints and impact of various incentive s 0 mechanisms are considered in [12], [13] and [14]. 401 K KARTHIKEYAN et. al.: APPLICATION OF RESTART COVARIANCE MATRIX ADAPTATION EVOLUTION STRATEGY (RCMA-ES) TO GENERATION EXPANSION PLANNING PROBLEM 1 ' ' 1.5t s Rmin minimum reserve margin OX t EENSOC 1 d (6) ' s 0 Rmax maximum reserve margin The outage cost calculation of Eq.(6), used in Eq.(1), Dt demand at the t-th stage in megawatts (MW) depends on Expected Energy Not Served (EENS). The equivalent energy function method [2] is used to calculate EENS Xt,i cumulative capacity of i-th unit at t-th stage (and also to calculate loss of load probability, LOLP, used in the 3) Fuel Mix Ratio: The GEP has different types of constraint objective). generating units such as coal, liquefied natural gas t' = 2(t – 1) and T' = 2 T – t' (7) (LNG), oil, and nuclear. The selected units along with the and existing units of each type must satisfy the fuel mix ratio, N Cost total cost, $ j j FMmin X t, j X t,i FMmax j 1,2,...., N (10) i1 Ut N-dimensional vector of introduced units in t-th stage (1 stage = 2 years) where, j minimum fuel mix ratio of j-th type Ut,i the number of introduced units of i-th type in t-th stage FMmin X cumulative capacity vector of existing units in t-th t FM j maximum fuel mix ratio of j-th type stage, (MW) max j type of the unit (e.g., oil, LNG, coal, nuclear) I(Ut) is the investment cost of the introduced unit at the t-th stage, $ 4) Reliability Criterion: The introduced units along with the existing units must satisfy a reliability criterion on loss of M(Xt) total operation and maintenance cost of existing and the newly introduced units, $ load probability (LOLP), LOLP(X ) (11) s' variable used to indicate that the maintenance cost is t calculated at the middle of each year where, is the reliability criterion, a fraction, for maximum allowable LOLP. Minimum reserve margin O(Xt) outage cost of the existing and the introduced units, $ constraint avoids the need for a separate demand S(Ut) salvage value of the introduced unit at t-th interval, $ constraint. d discount rate 3. IMPLEMENTATION TO THE GEP CIi capital investment cost of i-th unit, $ PROBLEM δi salvage factor of i-th unit 3.1 VIRTUAL MAPPING PROCEDURE (VMP) [9] T length of the planning horizon (in stages) Convergence problems were encountered when N total number of different types of units implementing the standard RCMA-ES algorithm on the GEP FC fixed operation and maintenance cost of the units, problem. Investigation revealed the reason pertained to the $/MW sensitivity of capacity to changes in the decision vector. To illustrate, consider the range of the decision vector lies between MC variable operation and maintenance cost of the units, $ [0] and Umax,t as given in [6]. The capacity vector is [200 450 EENS Expected energy not served, MWhrs 500 1000 700]. Let a candidate solution be U1= [5 0 1 1 0]; then its corresponding capacity will be 2500 MW (5200 + 0450 + OC value of outage cost constant, $/ MWhrs 1500 + 11000 + 0700). If U1 changes to [5 0 1 2 0] (via mutation operators), then the capacity is increased from 2500 2.2 CONSTRAINTS MW to 3500 MW, a difference of 1000 MW, which is a large deviation. 1) Construction limit: Let Ut represent the units to be committed in the expansion plan at t-th stage that must So VMP is introduced to improve the effectiveness of the satisfy, RCMA-ES in solving the GEP problem. VMP is concerned with the solution representation; it transforms each combination of 0 Ut Umax,t candidate units for every year into a dummy decision variable, where, U is the maximum construction capacity of the max,t referred to as the VMP variable. The value of the VMP variable units at t-th stage. for a specific candidate solution is the rank of that solution when 2) Reserve Margin: The selected units must satisfy the all solutions are sorted in ascending order of capacity. For minimum and maximum reserve margin. example, the solution [5 0 1 1 0] with capacity of 2500 has VMP N value of 190. A change (by mutation) to VMP value of 198 1 Rmin Dt X t,i 1 Rmax Dt (9) would correspond to a solution of [2 1 2 0 1] with capacity of i1 2550, a relatively small change in capacity. Without VMP, to get where, a solution with capacity of 2550 MW, all 5 variable values must 402 ISSN: 2229-6956(ONLINE) ICTACT JOURNAL ON SOFT COMPUTING, OCTOBER 2012, VOLUME: 03, ISSUE: 01 be changed. This may take a very large number of iterations, and Stop if the range of the best objective function values of in general, moving from sub-optimal to optimal may take much the last 10 + [30n / λ] generations is zero. iteration. Stop if the standard deviation of the normal distribution Since five different types of units are assumed to be available is smaller than 10-12 δ(0) in all coordinates.
Load more

Recommended publications
  • Metaheuristics1
    METAHEURISTICS1 Kenneth Sörensen University of Antwerp, Belgium Fred Glover University of Colorado and OptTek Systems, Inc., USA 1 Definition A metaheuristic is a high-level problem-independent algorithmic framework that provides a set of guidelines or strategies to develop heuristic optimization algorithms (Sörensen and Glover, To appear). Notable examples of metaheuristics include genetic/evolutionary algorithms, tabu search, simulated annealing, and ant colony optimization, although many more exist. A problem-specific implementation of a heuristic optimization algorithm according to the guidelines expressed in a metaheuristic framework is also referred to as a metaheuristic. The term was coined by Glover (1986) and combines the Greek prefix meta- (metá, beyond in the sense of high-level) with heuristic (from the Greek heuriskein or euriskein, to search). Metaheuristic algorithms, i.e., optimization methods designed according to the strategies laid out in a metaheuristic framework, are — as the name suggests — always heuristic in nature. This fact distinguishes them from exact methods, that do come with a proof that the optimal solution will be found in a finite (although often prohibitively large) amount of time. Metaheuristics are therefore developed specifically to find a solution that is “good enough” in a computing time that is “small enough”. As a result, they are not subject to combinatorial explosion – the phenomenon where the computing time required to find the optimal solution of NP- hard problems increases as an exponential function of the problem size. Metaheuristics have been demonstrated by the scientific community to be a viable, and often superior, alternative to more traditional (exact) methods of mixed- integer optimization such as branch and bound and dynamic programming.
    [Show full text]
  • Covariance Matrix Adaptation for the Rapid Illumination of Behavior Space
    Covariance Matrix Adaptation for the Rapid Illumination of Behavior Space Matthew C. Fontaine Julian Togelius Viterbi School of Engineering Tandon School of Engineering University of Southern California New York University Los Angeles, CA New York City, NY [email protected] [email protected] Stefanos Nikolaidis Amy K. Hoover Viterbi School of Engineering Ying Wu College of Computing University of Southern California New Jersey Institute of Technology Los Angeles, CA Newark, NJ [email protected] [email protected] ABSTRACT We focus on the challenge of finding a diverse collection of quality solutions on complex continuous domains. While quality diver- sity (QD) algorithms like Novelty Search with Local Competition (NSLC) and MAP-Elites are designed to generate a diverse range of solutions, these algorithms require a large number of evaluations for exploration of continuous spaces. Meanwhile, variants of the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) are among the best-performing derivative-free optimizers in single- objective continuous domains. This paper proposes a new QD algo- rithm called Covariance Matrix Adaptation MAP-Elites (CMA-ME). Figure 1: Comparing Hearthstone Archives. Sample archives Our new algorithm combines the self-adaptation techniques of for both MAP-Elites and CMA-ME from the Hearthstone ex- CMA-ES with archiving and mapping techniques for maintaining periment. Our new method, CMA-ME, both fills more cells diversity in QD. Results from experiments based on standard con- in behavior space and finds higher quality policies to play tinuous optimization benchmarks show that CMA-ME finds better- Hearthstone than MAP-Elites. Each grid cell is an elite (high quality solutions than MAP-Elites; similarly, results on the strategic performing policy) and the intensity value represent the game Hearthstone show that CMA-ME finds both a higher overall win rate across 200 games against difficult opponents.
    [Show full text]
  • Long Term Memory Assistance for Evolutionary Algorithms
    mathematics Article Long Term Memory Assistance for Evolutionary Algorithms Matej Crepinšekˇ 1,* , Shih-Hsi Liu 2 , Marjan Mernik 1 and Miha Ravber 1 1 Faculty of Electrical Engineering and Computer Science, University of Maribor, 2000 Maribor, Slovenia; [email protected] (M.M.); [email protected] (M.R.) 2 Department of Computer Science, California State University Fresno, Fresno, CA 93740, USA; [email protected] * Correspondence: [email protected] Received: 7 September 2019; Accepted: 12 November 2019; Published: 18 November 2019 Abstract: Short term memory that records the current population has been an inherent component of Evolutionary Algorithms (EAs). As hardware technologies advance currently, inexpensive memory with massive capacities could become a performance boost to EAs. This paper introduces a Long Term Memory Assistance (LTMA) that records the entire search history of an evolutionary process. With LTMA, individuals already visited (i.e., duplicate solutions) do not need to be re-evaluated, and thus, resources originally designated to fitness evaluations could be reallocated to continue search space exploration or exploitation. Three sets of experiments were conducted to prove the superiority of LTMA. In the first experiment, it was shown that LTMA recorded at least 50% more duplicate individuals than a short term memory. In the second experiment, ABC and jDElscop were applied to the CEC-2015 benchmark functions. By avoiding fitness re-evaluation, LTMA improved execution time of the most time consuming problems F03 and F05 between 7% and 28% and 7% and 16%, respectively. In the third experiment, a hard real-world problem for determining soil models’ parameters, LTMA improved execution time between 26% and 69%.
    [Show full text]
  • A Discipline of Evolutionary Programming 1
    A Discipline of Evolutionary Programming 1 Paul Vit¶anyi 2 CWI, Kruislaan 413, 1098 SJ Amsterdam, The Netherlands. Email: [email protected]; WWW: http://www.cwi.nl/ paulv/ » Genetic ¯tness optimization using small populations or small pop- ulation updates across generations generally su®ers from randomly diverging evolutions. We propose a notion of highly probable ¯t- ness optimization through feasible evolutionary computing runs on small size populations. Based on rapidly mixing Markov chains, the approach pertains to most types of evolutionary genetic algorithms, genetic programming and the like. We establish that for systems having associated rapidly mixing Markov chains and appropriate stationary distributions the new method ¯nds optimal programs (individuals) with probability almost 1. To make the method useful would require a structured design methodology where the develop- ment of the program and the guarantee of the rapidly mixing prop- erty go hand in hand. We analyze a simple example to show that the method is implementable. More signi¯cant examples require theoretical advances, for example with respect to the Metropolis ¯lter. Note: This preliminary version may deviate from the corrected ¯nal published version Theoret. Comp. Sci., 241:1-2 (2000), 3{23. 1 Introduction Performance analysis of genetic computing using unbounded or exponential population sizes or population updates across generations [27,21,25,28,29,22,8] may not be directly applicable to real practical problems where we always have to deal with a bounded (small) population size [9,23,26]. 1 Preliminary version published in: Proc. 7th Int'nl Workshop on Algorithmic Learning Theory, Lecture Notes in Arti¯cial Intelligence, Vol.
    [Show full text]
  • Automated Antenna Design with Evolutionary Algorithms
    Automated Antenna Design with Evolutionary Algorithms Gregory S. Hornby∗ and Al Globus University of California Santa Cruz, Mailtop 269-3, NASA Ames Research Center, Moffett Field, CA Derek S. Linden JEM Engineering, 8683 Cherry Lane, Laurel, Maryland 20707 Jason D. Lohn NASA Ames Research Center, Mail Stop 269-1, Moffett Field, CA 94035 Whereas the current practice of designing antennas by hand is severely limited because it is both time and labor intensive and requires a significant amount of domain knowledge, evolutionary algorithms can be used to search the design space and automatically find novel antenna designs that are more effective than would otherwise be developed. Here we present automated antenna design and optimization methods based on evolutionary algorithms. We have evolved efficient antennas for a variety of aerospace applications and here we describe one proof-of-concept study and one project that produced fight antennas that flew on NASA's Space Technology 5 (ST5) mission. I. Introduction The current practice of designing and optimizing antennas by hand is limited in its ability to develop new and better antenna designs because it requires significant domain expertise and is both time and labor intensive. As an alternative, researchers have been investigating evolutionary antenna design and optimiza- tion since the early 1990s,1{3 and the field has grown in recent years as computer speed has increased and electromagnetics simulators have improved. This techniques is based on evolutionary algorithms (EAs), a family stochastic search methods, inspired by natural biological evolution, that operate on a population of potential solutions using the principle of survival of the fittest to produce better and better approximations to a solution.
    [Show full text]
  • Evolutionary Algorithms
    Evolutionary Algorithms Dr. Sascha Lange AG Maschinelles Lernen und Naturlichsprachliche¨ Systeme Albert-Ludwigs-Universit¨at Freiburg [email protected] Dr. Sascha Lange Machine Learning Lab, University of Freiburg Evolutionary Algorithms (1) Acknowlegements and Further Reading These slides are mainly based on the following three sources: I A. E. Eiben, J. E. Smith, Introduction to Evolutionary Computing, corrected reprint, Springer, 2007 — recommendable, easy to read but somewhat lengthy I B. Hammer, Softcomputing,LectureNotes,UniversityofOsnabruck,¨ 2003 — shorter, more research oriented overview I T. Mitchell, Machine Learning, McGraw Hill, 1997 — very condensed introduction with only a few selected topics Further sources include several research papers (a few important and / or interesting are explicitly cited in the slides) and own experiences with the methods described in these slides. Dr. Sascha Lange Machine Learning Lab, University of Freiburg Evolutionary Algorithms (2) ‘Evolutionary Algorithms’ (EA) constitute a collection of methods that originally have been developed to solve combinatorial optimization problems. They adapt Darwinian principles to automated problem solving. Nowadays, Evolutionary Algorithms is a subset of Evolutionary Computation that itself is a subfield of Artificial Intelligence / Computational Intelligence. Evolutionary Algorithms are those metaheuristic optimization algorithms from Evolutionary Computation that are population-based and are inspired by natural evolution.Typicalingredientsare: I A population (set) of individuals (the candidate solutions) I Aproblem-specificfitness (objective function to be optimized) I Mechanisms for selection, recombination and mutation (search strategy) There is an ongoing controversy whether or not EA can be considered a machine learning technique. They have been deemed as ‘uninformed search’ and failing in the sense of learning from experience (‘never make an error twice’).
    [Show full text]
  • Evolutionary Programming Made Faster
    82 IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION, VOL. 3, NO. 2, JULY 1999 Evolutionary Programming Made Faster Xin Yao, Senior Member, IEEE, Yong Liu, Student Member, IEEE, and Guangming Lin Abstract— Evolutionary programming (EP) has been applied used to generate new solutions (offspring) and selection is used with success to many numerical and combinatorial optimization to test which of the newly generated solutions should survive problems in recent years. EP has rather slow convergence rates, to the next generation. Formulating EP as a special case of the however, on some function optimization problems. In this paper, a “fast EP” (FEP) is proposed which uses a Cauchy instead generate-and-test method establishes a bridge between EP and of Gaussian mutation as the primary search operator. The re- other search algorithms, such as evolution strategies, genetic lationship between FEP and classical EP (CEP) is similar to algorithms, simulated annealing (SA), tabu search (TS), and that between fast simulated annealing and the classical version. others, and thus facilitates cross-fertilization among different Both analytical and empirical studies have been carried out to research areas. evaluate the performance of FEP and CEP for different function optimization problems. This paper shows that FEP is very good One disadvantage of EP in solving some of the multimodal at search in a large neighborhood while CEP is better at search optimization problems is its slow convergence to a good in a small local neighborhood. For a suite of 23 benchmark near-optimum (e.g., to studied in this paper). The problems, FEP performs much better than CEP for multimodal generate-and-test formulation of EP indicates that mutation functions with many local minima while being comparable to is a key search operator which generates new solutions from CEP in performance for unimodal and multimodal functions with only a few local minima.
    [Show full text]
  • Evolutionary Algorithms in Intelligent Systems
    mathematics Editorial Evolutionary Algorithms in Intelligent Systems Alfredo Milani Department of Mathematics and Computer Science, University of Perugia, 06123 Perugia, Italy; [email protected] Received: 17 August 2020; Accepted: 29 August 2020; Published: 10 October 2020 Evolutionary algorithms and metaheuristics are widely used to provide efficient and effective approximate solutions to computationally difficult optimization problems. Successful early applications of the evolutionary computational approach can be found in the field of numerical optimization, while they have now become pervasive in applications for planning, scheduling, transportation and logistics, vehicle routing, packing problems, etc. With the widespread use of intelligent systems in recent years, evolutionary algorithms have been applied, beyond classical optimization problems, as components of intelligent systems for supporting tasks and decisions in the fields of machine vision, natural language processing, parameter optimization for neural networks, and feature selection in machine learning systems. Moreover, they are also applied in areas like complex network dynamics, evolution and trend detection in social networks, emergent behavior in multi-agent systems, and adaptive evolutionary user interfaces to mention a few. In these systems, the evolutionary components are integrated into the overall architecture and they provide services to the specific algorithmic solutions. This paper selection aims to provide a broad view of the role of evolutionary algorithms and metaheuristics in artificial intelligent systems. A first relevant issue discussed in the volume is the role of multi-objective meta-optimization of evolutionary algorithms (EA) in continuous domains. The challenging tasks of EA parameter tuning are the many different details that affect EA performance, such as the properties of the fitness function as well as time and computational constraints.
    [Show full text]
  • A New Evolutionary System for Evolving Artificial Neural Networks
    694 IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 8, NO. 3, MAY 1997 A New Evolutionary System for Evolving Artificial Neural Networks Xin Yao, Senior Member, IEEE, and Yong Liu Abstract—This paper presents a new evolutionary system, i.e., constructive and pruning algorithms [5]–[9]. Roughly speak- EPNet, for evolving artificial neural networks (ANN’s). The evo- ing, a constructive algorithm starts with a minimal network lutionary algorithm used in EPNet is based on Fogel’s evolution- (i.e., a network with a minimal number of hidden layers, nodes, ary programming (EP). Unlike most previous studies on evolving ANN’s, this paper puts its emphasis on evolving ANN’s behaviors. and connections) and adds new layers, nodes, and connections This is one of the primary reasons why EP is adopted. Five if necessary during training, while a pruning algorithm does mutation operators proposed in EPNet reflect such an emphasis the opposite, i.e., deletes unnecessary layers, nodes, and con- on evolving behaviors. Close behavioral links between parents nections during training. However, as indicated by Angeline et and their offspring are maintained by various mutations, such as partial training and node splitting. EPNet evolves ANN’s archi- al. [10], “Such structural hill climbing methods are susceptible tectures and connection weights (including biases) simultaneously to becoming trapped at structural local optima.” In addition, in order to reduce the noise in fitness evaluation. The parsimony they “only investigate restricted topological subsets rather than of evolved ANN’s is encouraged by preferring node/connection the complete class of network architectures.” deletion to addition. EPNet has been tested on a number of Design of a near optimal ANN architecture can be for- benchmark problems in machine learning and ANN’s, such as the parity problem, the medical diagnosis problems (breast mulated as a search problem in the architecture space where cancer, diabetes, heart disease, and thyroid), the Australian credit each point represents an architecture.
    [Show full text]
  • Neuroevolution Strategies for Episodic Reinforcement Learning ∗ Verena Heidrich-Meisner , Christian Igel
    J. Algorithms 64 (2009) 152–168 Contents lists available at ScienceDirect Journal of Algorithms Cognition, Informatics and Logic www.elsevier.com/locate/jalgor Neuroevolution strategies for episodic reinforcement learning ∗ Verena Heidrich-Meisner , Christian Igel Institut für Neuroinformatik, Ruhr-Universität Bochum, 44780 Bochum, Germany article info abstract Article history: Because of their convincing performance, there is a growing interest in using evolutionary Received 30 April 2009 algorithms for reinforcement learning. We propose learning of neural network policies Available online 8 May 2009 by the covariance matrix adaptation evolution strategy (CMA-ES), a randomized variable- metric search algorithm for continuous optimization. We argue that this approach, Keywords: which we refer to as CMA Neuroevolution Strategy (CMA-NeuroES), is ideally suited for Reinforcement learning Evolution strategy reinforcement learning, in particular because it is based on ranking policies (and therefore Covariance matrix adaptation robust against noise), efficiently detects correlations between parameters, and infers a Partially observable Markov decision process search direction from scalar reinforcement signals. We evaluate the CMA-NeuroES on Direct policy search five different (Markovian and non-Markovian) variants of the common pole balancing problem. The results are compared to those described in a recent study covering several RL algorithms, and the CMA-NeuroES shows the overall best performance. © 2009 Elsevier Inc. All rights reserved. 1. Introduction Neuroevolution denotes the application of evolutionary algorithms to the design and learning of neural networks [54,11]. Accordingly, the term Neuroevolution Strategies (NeuroESs) refers to evolution strategies applied to neural networks. Evolution strategies are a major branch of evolutionary algorithms [35,40,8,2,7].
    [Show full text]
  • A Hybrid Neural Network and Genetic Programming Approach to the Automatic Construction of Computer Vision Systems
    AHYBRID NEURAL NETWORK AND GENETIC PROGRAMMING APPROACH TO THE AUTOMATIC CONSTRUCTION OF COMPUTER VISION SYSTEMS Cameron P.Kyle-Davidson A thesis submitted for the degree of Master of Science by Dissertation School of Computer Science and Electronic Engineering University of Essex October 2018 ii Abstract Both genetic programming and neural networks are machine learning techniques that have had a wide range of success in the world of computer vision. Recently, neural networks have been able to achieve excellent results on problems that even just ten years ago would have been considered intractable, especially in the area of image classification. Additionally, genetic programming has been shown capable of evolving computer vision programs that are capable of classifying objects in images using conventional computer vision operators. While genetic algorithms have been used to evolve neural network structures and tune the hyperparameters of said networks, this thesis explores an alternative combination of these two techniques. The author asks if integrating trained neural networks with genetic pro- gramming, by framing said networks as components for a computer vision system evolver, would increase the final classification accuracy of the evolved classifier. The author also asks that if so, can such a system learn to assemble multiple simple neural networks to solve a complex problem. No claims are made to having discovered a new state of the art method for classification. Instead, the main focus of this research was to learn if it is possible to combine these two techniques in this manner. The results presented from this research in- dicate that such a combination does improve accuracy compared to a vision system evolved without the use of these networks.
    [Show full text]
  • Generative Genetic Algorithm for Music Generation
    GGA-MG: Generative Genetic Algorithm for Music Generation Majid Farzaneh1 . Rahil Mahdian Toroghi1 1Media Engineering Faculty, Iran Broadcasting University, Tehran, Iran Abstract Music Generation (MG) is an interesting research topic today which connects art and Artificial Intelligence (AI). The purpose is training an artificial composer to generate infinite, fresh, and pleasurable music. Music has different parts such as melody, harmony, and rhythm. In this work, we propose a Generative Genetic Algorithm (GGA) to generate melody automatically. The main GGA uses a Long Short-Term Memory (LSTM) recurrent neural network as its objective function, and the LSTM network has been trained by a bad-to-good spectrum of melodies which has been provided by another GGA with a different objective function. Good melodies have been provided by CAMPIN’s collection. We also considered the rhythm in this work and experimental results show that proposed GGA can generate good melodies with natural transition and without any rhythm error. Keywords: Music Generation, Generative Genetic Algorithm (GGA), Long Short-Term Memory (LSTM), Melody, Rhythm 1 Introduction By growing the size of digital media, video games, TV shows, Internet channels, and home-made video clips, using the music has been grown as well to make them more effective. It is not cost-effective financially and also in time to ask musicians and music composers to produce music for each of the mentioned purposes. So, engineers and researchers are trying to find an efficient way to generate music automatically by machines. Lots of works have proposed good algorithms to generate melody, harmony, and rhythms that most of them are based on Artificial Intelligence (AI) and deep learning.
    [Show full text]