DOCSLIB.ORG

Evolving Test Environments to Identify Faults in Swarm Robotics Algorithms

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Evolving Test Environments to Identify Faults in Swarm Robotics Algorithms Evolving Test Environments to Identify Faults in Swarm Robotics Algorithms HAO WEI PhD University of York Computer Science January 2018 iii Abstract Swarm robotic systems are often considered to be dependable. However, there is little empirical evidence or theoretical analysis showing that de- pendability is an inherent property of all swarm robotic systems. Re- cent literature has identified potential issues with respect to dependabil- ity within certain types of swarm robotic control algorithms. However, there is little research on the testing of swarm robotic systems; this pro- vides the motivation for developing a novel testing method for swarm robotic systems. An evolutionary testing method is proposed in this thesis to identify unintended behaviours during the execution of swarm robotic systems autonomously. Three case studies are carried out on flocking con- trol algorithm, foraging algorithm, and task partitioning algorithm. These case studies not only show that the evolutionary testing method has the ability to identify faults in swarm robotic system, but also show that this evolutionary testing method is able to reveal failures in various swarm control algorithms. The experimental results show that the evolutionary testing method can lead to worse swarm performance and reveal more failures than the random testing method within the same number of com- puting evaluations. Moreover, the case study of flocking control algorithm also shows that the evolutionary testing method covers more failure types than the random testing method. In all three case studies, the depend- ability of each swarm robotic system has been improved by tackling the faults identified during the testing phase. Consequently, the evolution- ary testing method has the potential to be used to help the developers of swarm robotic systems to design and calibrate the swarm control algo- rithms thereby assuring the dependability of swarm robotic systems. v Contents Abstract iii List of Tables viii List of Figures viii Acknowledgements xv Declaration xvii 1 Introduction 1 1.1 The Differences between Multi-Robot Systems and Swarm Robotic Systems . .2 1.2 Dependability of Swarm Robotic Systems . .3 1.3 Evolutionary Testing Method . .3 1.4 Research Hypothesis . .4 1.4.1 Research Questions . .4 1.5 Thesis Contributions . .5 1.6 Thesis Structure . .6 2 Background and Related Work 9 2.1 Swarm Robotics . .9 2.1.1 What is Swarm Robotics? . .9 2.1.2 Motivations for Swarm Robotics Research . 11 2.1.3 Examples of Inspiration to Swarm Intelligence . 11 2.1.4 Potential Applications . 13 2.1.5 The Design Problem of Swarm Robotics . 14 2.1.6 Summary . 21 2.2 Dependability of Swarm Robotics . 22 2.2.1 Dependability Concepts . 22 2.2.2 Dependability of Current Swarm Robotics . 25 2.2.3 Fault detection in swarm robotics . 28 2.2.4 Summary . 29 2.3 Analysis and Testing . 30 2.3.1 Analysis . 30 vi 2.3.2 Testing . 33 2.3.3 Summary . 40 2.4 Swarm Behaviours . 40 2.4.1 Flocking Behaviour . 40 2.4.2 Ant Foraging Behaviour . 43 2.4.3 Task Partitioning Behaviour . 46 2.5 Chapter Summary . 48 3 Evolutionary Testing Method for Swarm Robotic System 51 3.1 Problem Analysis . 51 3.2 Proposed solution - Evolutionary Testing Method . 54 3.2.1 Chromosome . 55 3.2.2 Fitness Function . 57 3.2.3 Genetic Algorithm Parameters . 58 3.2.4 Parameter Analysis . 60 3.2.5 Evolutionary Process of Test Cases . 61 3.3 Experimental Infrastructure . 62 3.3.1 ARGoS Simulator . 63 3.3.2 Foot-bot robot model . 65 3.3.3 External Assistance Tools . 66 3.3.4 Test Case Generator . 67 3.3.5 Cluster Computing . 68 3.4 Chapter Summary . 69 4 Testing Method for Flocking Behaviour 71 4.1 Experimental Scenario for Testing Flocking Behaviour . 71 4.1.1 Total Failures . 72 4.2 Metrics for flocking behaviour . 72 4.2.1 Angular Order . 74 4.2.2 The Proportion of the Swarm Left . 75 4.3 Flocking Control Algorithm . 75 4.3.1 The Control Algorithm for The Prey . 76 4.4 Failure Classification . 77 4.5 Evolutionary Testing method . 78 4.5.1 Chromosomes . 78 4.5.2 Fitness Function . 78 4.5.3 Parameter Analysis . 80 4.6 Experimental Results . 80 4.6.1 The Selection of Statistical Tests . 81 4.6.2 Comparison of Severity of Failures . 83 vii 4.6.3 Comparison of Diversity of Failure Type . 85 4.6.4 Discussion . 87 4.7 Improving Control Algorithm by Fault Removal . 87 4.7.1 Total Failure Analysis . 88 4.7.2 Algorithm Improvement . 89 4.7.3 Experimental Results of the Improved Flocking Con- trol Algorithm . 89 4.8 Chapter Summary . 91 5 Testing Method for Foraging Behaviour 93 5.1 Metrics for Ant Foraging Behaviour . 93 5.1.1 Total Failure in Ant Foraging Algorithm . 94 5.2 Control algorithm for Ant Foraging Behaviour . 95 5.2.1 Virtual Pheromone Algorithm . 95 5.3 Evolutionary Testing Method . 98 5.3.1 Chromosome . 98 5.3.2 Fitness Function . 99 5.3.3 Parameters Analysis of GA . 99 5.4 Experiments . 100 5.4.1 Experimental Setup . 100 5.4.2 Experimental Results of VP Algorithm . 101 5.4.3 Discussion . 103 5.5 Improving Control Algorithm by Fault Removal . 104 5.5.1 Total Failure Analysis . 104 5.5.2 Algorithm Improvement . 105 5.5.3 Experimental Results using the Improved VP Algo- rithm . 107 5.6 Chapter Summary . 109 6 Testing Method for Task Partitioning Behaviour 111 6.1 Metrics for Task Partitioning Behaviour . 111 6.1.1 Total Failure in Task Partitioning Behaviour . 113 6.2 Control Algorithm for Task Partitioning . 113 6.2.1 Path Planning Method . 115 6.3 Evolutionary Testing Method . 116 6.3.1 Chromosome . 116 6.3.2 Fitness Function and Parameter Analysis . 117 6.3.3 Parameter Analysis of Genetic Algorithm . 117 6.4 Experimental Results . 117 6.4.1 Experimental Results for the Static Partitioning Strat- egy............................. 118 viii 6.4.2 Convergence Time of Task Partitioning . 120 6.5 Improving the Control Algorithm by Fault Removal . 122 6.5.1 Total Failure Analysis . 123 6.5.2 Algorithm Improvement . 124 6.5.3 Experimental Results for the Improved Static Parti- tioning Strategy . 126 6.6 Chapter Summary . 129 7 Evaluation and Conclusions 131 7.1 Evaluation of Research Hypothesis . 131 7.2 Thesis Contributions . 132 7.3 Limitations . 133 7.4 Future Work . 134 7.5 Conclusion . 136 Bibliography 137 ix List of Tables 1.1 Differences between Swarm Robotics and Other Multi-Robotics2 2.1 INTERNAL HAZARDS FOR A SINGLE ROBOT (Winfield and Nembrini, 2006) . 26 2.2 SUMMARY OF FAILURE MODES AND EFFECTS (Win- field and Nembrini, 2006) . 27 4.1 PARAMETER VALUES OF GENETIC ALGORITHM (Wei, Timmis, and Alexander, 2017) . 80 4.2 DETAILS OF THE EXPERIMENTS OF FAILURE SEVERITY 83 4.3 DETAILED RESULTS OF MANN-WHITNEY U-TEST . 84 4.4 DETAILS OF THE EXPERIMENTS OF FAILURE TYPE . 86 4.5 FAILURE TYPES DISCOVERED. Grey stands for failure type discovered by the random testing method and Cyan stands for failure type discovered by the evolutionary testing method. 87 4.6 NUMBER OF TOTAL FAILURES DISCOVERED BY EVOLVED TEST CASES FOR EACH FAILURE TYPE. 88 4.7 THREE PAIRS OF RESULT DETAILS FOR THE MANN- WHITNEY U-TEST . 90 5.1 PARAMETER VALUES OF GENETIC ALGORITHM . 99 5.2 DETAILS OF THE EXPERIMENTS OF VP ALGORITHM . 102 5.3 DETAILED RESULTS OF MANN-WHITNEY U-TEST OF VP ALGORITHM . 102 5.4 TOTAL FAILURES OF VP ALGORITHM FOUND DURING THE EXPERIMENTS . 104 5.5 THREE PAIRS OF DETAILED RESULTS OF MANN-WHITNEY U-TEST . 108 6.1 DETAILS OF THE EXPERIMENTS OF TASK PARTITIONING119 6.2 RESULT DETAILS OF THE MANN-WHITNEY U-TEST . 119 6.3 TOTAL FAILURES OF STATIC PARTITIONING STRATEGY FOUND DURING THE EXPERIMENTS . 123 x 6.4 THREE PAIRS OF RESULT DETAILS OF THE MANN-WHITNEY U-TEST . 127 xi List of Figures 2.1 Laprie’s dependability tree (Laprie, 1995) . 22 2.2 Reliability for a swarm with partially failed robots (Bjerknes and Winfield, 2013) . 28 2.3 Maximizing the minimum number of turns for 100 genera- tions (Ashlock, Manikas, and Ashenayi, 2006) . 36 2.4 Steering rules of Boids (Reynolds, 1987). 41 2.5 Snapshots of steered flocking with 7 robots. At the time of the second snapshot, 4 of the robots are commanded to turn 90 degrees to the right of their current direction. White lines indicate the heading directions of the robots (Çelikkanat and ¸Sahin,2010). 42 2.6 The finite state machine of the basic foraging algorithm . 44 3.1 model of system development life cycle . 52 3.2 A bird’s-eye view of a chromosome containing only one gene (obstacle). In the simulator, the height of each obstacle is set to be 1 meter. 56 3.3 A model of how Headquarters and Observer operate. 67 3.4 A model of the working procedure of the test case genera- tor. GA step2 to step 5 is listed in section 3.2.5. 69 4.1 Angular Order . 75 4.2 A comparison between the path found by A* algorithm (red) and the path found by modified version of A* algorithm (blue). 76 4.3 An example of how a swarm is split into two clusters. 77 4.4 Fitness values for four different orientations of a swarm. The white.
Load more

Recommended publications
  • Evolving Symbolic Controllers
    Evolving Symbolic Controllers Nicolas Godzik1, Marc Schoenauer1, and Mich`ele Sebag2 1 Projet Fractales, INRIA Rocquencourt, France 2 LRI, Universit´eParis-Sud, France Published in G. Raidl et al., eds, Applications of Evolutionary Computing, pp 638-650, LNCS 2611, Springer Verlag, 2003. Abstract. The idea of symbolic controllers tries to bridge the gap be- tween the top-down manual design of the controller architecture, as advo- cated in Brooks’ subsumption architecture, and the bottom-up designer- free approach that is now standard within the Evolutionary Robotics community. The designer provides a set of elementary behavior, and evolution is given the goal of assembling them to solve complex tasks. Two experiments are presented, demonstrating the efficiency and show- ing the recursiveness of this approach. In particular, the sensitivity with respect to the proposed elementary behaviors, and the robustness w.r.t. generalization of the resulting controllers are studied in detail. 1 Introduction There are two main trends in autonomous robotics. There are two main trends in autonomous robotics. The first one, advocated by R. Brooks [2], is a human- specified deterministic approach: the tasks of the robot are manually decom- posed into a hierarchy of independent sub-tasks, resulting in the the so-called subsumption architecture. On the other hand, evolutionary robotics (see e.g. [13]), is generally viewed as a pure black-box approach: some controllers, mapping the sensors to the actua- tors, are optimized using the Darwinian paradigm of Evolutionary Computation; the programmer only designs the fitness function. However, the scaling issue remains critical for both approaches, though for arXiv:0705.1244v1 [cs.AI] 9 May 2007 different reasons.
    [Show full text]
  • A Model for Virtual Reconfigurable Modular Robots Thomas Breton, Yves Duthen
    A Model for Virtual Reconfigurable Modular Robots Thomas Breton, Yves Duthen To cite this version: Thomas Breton, Yves Duthen. A Model for Virtual Reconfigurable Modular Robots. 2011. hal- 01298411 HAL Id: hal-01298411 https://hal.archives-ouvertes.fr/hal-01298411 Preprint submitted on 5 Apr 2016 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. A Model for Virtual Reconfigurable Modular Robots Thomas Breton1 and Yves Duthen1 1VORTEX Research Team, IRIT UMR 5505 University of Toulouse, France [email protected], [email protected] Abstract and communication methods. We propose here a model to accomplish different tasks such as motion planning, This paper presents a model for virtual reconfigurable modu- object displacement, and structure reconfiguration; evolving lar robots in order to evolve artificial creatures, able of self- adaptation to the environment as well as good adjustment to a controller (a neural network) by the mean of a genetic various given tasks. For this purpose, a simulator has been algorithm. entirely developed with the assistance of a physics engine to represent force activities. One of the most crucial points It is true that any consideration taken in modular robotic in modular robot construction is the choice of module type, complexity and diversity.
    [Show full text]
  • Evolutionary Robotics in Two Decades: a Review
    Sadhan¯ a¯ Vol. 40, Part 4, June 2015, pp. 1169–1184. c Indian Academy of Sciences Evolutionary robotics in two decades: A review SAMEER GUPTA and EKTA SINGLA∗ School of Mechanical, Materials and Energy Engineering, IIT Ropar, Rupnagar 140001, India e-mail: [email protected]; [email protected] MS received 29 August 2014; revised 1 January 2015; accepted 11 February 2015 Abstract. Evolutionary robotics (ER) has emerged as a fast growing field in the last two decades and has earned the attention of a number of researchers. Principles of biological evolution are applied in the form of evolutionary techniques for solv- ing the complicated problems in the areas of robotic design and control. The diversity and the intensity of this growing field is presented in this paper through the contribu- tions made by several researchers in the categories of robot controller design, robot body design, co-evolution of body and brain and in transforming the evolved robots in physical reality. The paper discusses some of the recent achievements in each of these fields along with some expected applications which are likely to motivate the future research. For the quick reference of the readers, a digest of all the works is presented in the paper, spanning the years and the areas of the research contributions. Keywords. Evolutionary robotics; evolutionary control; robot morphology; body– brain design. 1. Introduction With the advent of bio-inspired computational techniques (Brooks 1986; Beer & Gallagher 1992; Haykin 1994; Ram et al 1994), the interests of many roboticists bend towards the utility of these techniques in the complicated designs and control of robots.
    [Show full text]
  • Evolving Robot Empathy Through the Generation of Artificial Pain in An
    Evolving Robot Empathy through the Generation of Artificial Pain in an Adaptive Self-Awareness Framework for Human-Robot Collaborative Tasks Muh Anshar Faculty of Engineering and Information Technology University of Technology Sydney This dissertation is submitted for the degree of Doctor of Philosophy March 2017 Bismillahirrahmanirrahim All Praise and Gratitude to the Almighty God, Allah SWT, for His Mercy and Guidance which have given me strength and tremendous support to maintain my motivation from the very beginning of my life journey and into the far future. I would like to dedicate this thesis to my love ones, my wife and my son, Nor Faizah & Abdurrahman Khalid Hafidz for always being beside me which has been a great and undeniable support throughout my study. CERTIFICATE OF ORIGINAL AUTHORSHIP This thesis is the result of a research candidature conducted jointly with another University as part of a collaborative Doctoral degree. I certify that the work in this thesis has not previously been submitted for a degree nor has it been submitted as part of requirements for a degree except as part of the collaborative doctoral degree and/or fully acknowledged within the text. I also certify that the thesis has been written by me. Any help that I have received in my research work and the preparation of the thesis itself has been acknowledged. In addition, I certify that all information sources and literature used are indicated in the thesis. Signature of Student: Date: 13 March 2017 Muh Anshar March 2017 Acknowledgements I would like to acknowledge and thank my Principal Supervisor, Professor Mary-Anne Williams for her great dedication, support and supervision throughout my PhD journey.
    [Show full text]
  • Swarm Robotics Distributed Embodied Evolutionary Robotics
    Swarm Robotics Distributed Embodied Evolutionary Robotics (Evolutionary) Swarm Robotics: a gentle introduction Inaki˜ Fernandez´ Perez´ [email protected] www.loria.fr/˜fernandi ISAL Student Group ECAL 2017 September 8th 2017 Universite´ de Lorraine, LARSEN Team, Inria Nancy, France 1 / 10 Swarm Robotics Distributed Embodied Evolutionary Robotics So::: what is a robot swarm? Large/huge set of Focus in collective Real robots are cool::: simple agents dynamics but simulation works too! 2 / 10 Swarm Robotics Distributed Embodied Evolutionary Robotics So::: what is a robot swarm? Large/huge set of Focus in collective Real robots are cool::: simple agents dynamics but simulation works too! 2 / 10 Swarm Robotics Distributed Embodied Evolutionary Robotics Where to start::: many approaches By hand: ∼ engineering approach [Brambilla et al., 2013] Clones: classical EA with copies on each robot (homogeneous) [Tuci et al., 2008] Coevolution: either cooperative or competitive (heterogeneous) [Gomes et al., 2016] (distributed) Embodied Evolution runs onboard on each robot (heterogeneous) [Watson et al., 2002, Fernandez´ Perez´ et al., 2017] 3 / 10 Swarm Robotics Distributed Embodied Evolutionary Robotics What goals? ALife models to understand biology/test biological hypothesis ALife tools to build systems that can solve a problem for us 4 / 10 Swarm Robotics Distributed Embodied Evolutionary Robotics What robotic tasks? Navigation Flocking Item collection Foraging Shepherding ::: 5 / 10 Swarm Robotics Distributed Embodied Evolutionary Robotics
    [Show full text]
  • Simulating the Evolution of Soft and Rigid-Body Robots
    Simulating the Evolution of So and Rigid-Body Robots Sam Kriegman Collin Cappelle∗ Francesco Corucci MEC Labx MEC Lab e BioRobotics Institute University of Vermont University of Vermont Scuola Superiore Sant’Anna Burlington, VT, USA Burlington, VT, USA Pisa, Italy [email protected] [email protected] Anton Bernatskiy Nick Cheney Josh C. Bongard MEC Lab Creative Machines Lab MEC Lab University of Vermont Cornell University University of Vermont Burlington, VT, USA Ithaca, NY, USA Burlington, VT, USA ABSTRACT here two high-level Python wrappers around two dierent simula- In evolutionary robotics, evolutionary methods are used to optimize tors: Open Dynamics Engine (ODE) and Voxelyze. Our goal in this robots to dierent tasks. Because using physical robots is costly paper is to convey from experience what kinds of things are di- in terms of both time and money, simulated robots are generally cult/easy to instantiate in physics engines in general and how our used instead. Most physics engines are wrien in C++ which can user-friendly modules at least partially alleviate this, and how they be a barrier for new programmers. In this paper we present two may be extended in the future through open-source collaborations. Python wrappers, Pyrosim and Evosoro, around two well used simulators, Open Dynamics Engine (ODE) and Voxelyze/VoxCAD, 2 RIGID BODY ROBOTS which respectively handle rigid and so bodied simulation. Python Rigid body dynamics engines are generally what is most thought is an easier language to understand so more time can be spent of when one thinks of a simulator. ey are 3D engines where on developing the actual experiment instead of programming the every body in the simulation is ‘rigid’, meaning the body cannot simulator.
    [Show full text]
  • Evolutionary Robotics
    Evolutionary Robotics IAR Lecture 13 Barbara Webb Basic process Population of genomes, e.g. Decode each into robot binary strings, tree structures controller and/or morphology, e.g. weights in neural net, position of Produce new set of sensors genomes, e.g. breed, crossover, mutate Place in environment and run Use fitness to select for Evaluate behaviour reproduction, e.g. only if using a fitness function achieved task, or best e.g. achieve task, speed, individuals, or proportional time survived, find mate to fitness score Motivation • Lack of design methods that will ensure the right dynamics emerge from the environment-robot-task interaction • Automate the trial-and-error approach • Avoid preconceptions in design • Allow self-organising processes to discover novel and efficient solutions • Good enough for biology (and might help us understand biology) ‘Typical’ example Floreano & Mondada (1996): evolving Braitenberg-type control for a Khepera robot to move around maze • Eight IR sensor input units, feed-forward to two motor output units with recurrent connections • Standard sigmoidal ANN n 1 y f w x , where f (x) i ij j kx j 1 e • Genome – bit string encoding weight values • Fitness function: V(1 v)(1i) where i is highest IR value, V vleft vright v vleft vright • Population of 80, each tested for approx 30s • Copied proportional to fitness, then random paired single point crossover and mutation (prob.=0.2) • 100 generations, get smooth travel round maze Similar approach has been used to evolve controllers for more complex
    [Show full text]
  • Evolutionary Developmental Soft Robotics Towards Adaptive and Intelligent Machines Following Nature’S Approach to Design
    Evolutionary Developmental Soft Robotics Towards adaptive and intelligent machines following Nature’s approach to design Francesco Corucci, PhD November 16th, 2017 - ShanghAI Lectures Motivations: diversity, complexity, sophistication F. Corucci Evolutionary Developmental Soft Robotics 2 Motivations: intelligent and adaptive behavior Camouflage Creativity Skills Reasoning, cognition F. Corucci Evolutionary Developmental Soft Robotics 3 Motivations Can we automatically design a wealth of artificial systems that are as sophisticated, adaptive, robust, intelligent, for a wide variety of tasks and environments? F. Corucci Evolutionary Developmental Soft Robotics 4 Adaptivity, robustness, intelligence State of the art robots still lack many of these features Keep failing outside controlled environments (where they are most needed) DARPA Robotics Challenge Finals, 2015 F. Corucci Evolutionary Developmental Soft Robotics 5 Biologically inspired robotics (biorobotics) Cheetah robot, MIT Bat robot, Brown Soft fish, MIT OCTOPUS, SSSA RoboBees, Harvard Lampetra, SSSA Plantoid robot, IIT ECCE robot F. Corucci Evolutionary Developmental Soft Robotics 6 Biologically inspired robotics: Soft Robotics F. Corucci Evolutionary Developmental Soft Robotics 7 Biologically inspired robotics: pros and cons Pros: New technologies and design principles New knowledge related to the biological model (sometimes) Insights related to the intelligence of particular species (sometimes) Cons: Requires a lot of human knowledge and careful engineering Focuses on very specific
    [Show full text]
  • DART: Diversity-Enhanced Autonomy in Robot Teams
    DART: Diversity-enhanced Autonomy in Robot Teams Nora Ayanian Abstract This paper defines the research area of Diversity-enhanced Autonomy in Robot Teams (DART), a novel paradigm for the creation and design of policies for multi-robot coordination. While current approaches to multi-robot coordination have been successful in structured, well understood environments, they have not been successful in unstructured, uncertain environments, such as disaster response. The reason for this is not due to limitations in robot hardware, which has advanced significantly in the past decade, but in how multi-robot problems are solved. Even with significant advances in the field of multi-robot systems, the same problem- solving paradigm has remained: assumptions are made to simplify the problem, and a solution is optimized for those assumptions and deployed to the entire team. This results in brittle solutions that prove incapable if the original assumptions are inval- idated. This paper introduces a new multi-robot problem-solving paradigm which relies on a diverse set of control policies that work together synergistically to make multi-robot systems more resilient in unstructured and uncertain environments. 1 Introduction The field of multi-robot systems (MRS) is growing at a rapid pace. Research in MRS spans many different areas, including automated delivery [1–3], surveillance [4], and disaster response [5, 6]. There have also been many successful demonstrations of increasing numbers of robots [7–9, 11–13]. MRS have also been successfully deployed in the field including in warehousing [15], manufacturing [16], and enter- tainment [17]. While these outcomes show the promise of MRS, the environments in which MRS have been successful are highly controlled, and some are highly in- strumented, enabling precise tuning of controllers and nearly perfect knowledge of environmental conditions.
    [Show full text]
  • Bootstrapping Q-Learning for Robotics from Neuro-Evolution Results Matthieu Zimmer, Stephane Doncieux
    Bootstrapping Q-Learning for Robotics from Neuro-Evolution Results Matthieu Zimmer, Stephane Doncieux To cite this version: Matthieu Zimmer, Stephane Doncieux. Bootstrapping Q-Learning for Robotics from Neuro-Evolution Results. IEEE Transactions on Cognitive and Developmental Systems, Institute of Electrical and Electronics Engineers, Inc, 2017, 10.1109/TCDS.2016.2628817. hal-01494744 HAL Id: hal-01494744 https://hal.archives-ouvertes.fr/hal-01494744 Submitted on 23 Mar 2017 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. IEEE TRANSACTIONS ON COGNITIVE AND DEVELOPMENTAL SYSTEMS, VOL. X, NO. Y, JANUARY 2016 1 Bootstrapping Q-Learning for Robotics from Neuro-Evolution Results Matthieu Zimmer, Stephane Doncieux, Member, IEEE Abstract—Reinforcement learning problems are hard to solve require exploring large and continuous search spaces and in a robotics context as classical algorithms rely on discrete typically rely on demonstrations by an expert to bootstrap, representations of actions and states, but in robotics both are or on non-random initial policies, or else on dedicated policy continuous. A discrete set of actions and states can be defined, but it requires an expertise that may not be available, in particular representations [2,3,4].
    [Show full text]
  • Evolutionary Robotics
    Introduction to Evolutionary Robotics Robotics Class, 2017, Prof. Laschi Teaching Assistant: Francesco Corucci f.corucci santannapisa.it http://sssa.bioroboticsinstitute.it/user/1507 @f_corucci Who am I • 4th year PhD Student in BioRobotics at The BioRobotics Institute, Scuola Superiore Sant’Anna, Pisa (IT) (Advisor: Prof. Laschi) • Visiting researcher, Morphology, Evolution and Cognition Lab, Vermont Complex Systems Center, University of Vermont (USA) (Advisor: Prof. Josh Bongard) TA for this course, 4h on Evolutionary Robotics + projects on this topic More info: http://sssa.bioroboticsinstitute.it/user/1507 F. Corucci - Introduction to Evolutionary Robotics Part of this material is inspired by or adapted from: • Floreano, Matiussi, “Bio-Inspired Artificial Intelligence - Theories, Methods, and Technologies”, MIT press (2008). • Pfeifer, Bongard, “How the body shapes the way we think: a new view of intelligence”. MIT press (2006). • “Evolutionary Robotics” class (CS206) at University of Vermont (USA), Prof. J. Bongard • LudoBots, a Reddit-based MOOC on Evolutionary Robotics https://www.reddit.com/r/ludobots/wiki/index http://www.reddit.com/r/ludobots http://www.meclab.org/ F. Corucci - Introduction to Evolutionary Robotics Outline • Introduction and motivation • Evolutionary Algorithms • Human-competitive design and “perverse instantiation” • Developmental encodings and co-evolution of artificial brains and bodies • The “reality gap” problem Appendix: • VoxCad, multi-material physics engine for soft-robot analysis, design and evolution F. Corucci - Introduction to Evolutionary Robotics Introduction and motivation F. Corucci - Introduction to Evolutionary Robotics Biorobotics: sometimes successful… Passive dynamic walker, Mc Geer 1990 F. Corucci - Introduction to Evolutionary Robotics Biorobotics: sometimes successful… Koh et al. 2015, Science F. Corucci - Introduction to Evolutionary Robotics …sometimes not so much Robots failing at the Darpa Robotic Challenge 2015 (IEEE Spectrum) F.
    [Show full text]
  • The AAAI 1999 Mobile Robot Competitions and Exhibitions
    AI Magazine Volume 21 Number 3 (2000) (© AAAI) Articles The AAAI 1999 Mobile Robot Competitions and Exhibition Lisa Meeden, Alan Schultz, Tucker Balch, Rahul Bhargava, Karen Zita Haigh, Marc Böhlen, Cathryne Stein, and David Miller I The Eighth Annual Mobile Robot Competition place was captured by University of Arkansas’s and Exhibition was held as part of the Sixteenth ELECTRO, with Douglas Blank as the adviser and National Conference on Artificial Intelligence in students Jared Hudson, Brian Mashburn, and Orlando, Florida, 18 to 22 July. The goals of these Eric Roberts. robot events are to foster the sharing of research In Hors d’oeuvres, Anyone? the entrants and technology, allow research groups to showcase served finger-food items at the large reception their achievements, encourage students to enter robotics and AI fields at both the undergraduate for conference attendees at the American Asso- and graduate level, and increase awareness of the ciation for Artificial Intelligence festival. The field. The 1999 events included two robot contests; objective was to unambiguously demonstrate a new, long-term robot challenge; an exhibition; interaction with the spectators. The teams’ and a National Botball Championship for high robots were encouraged to display distinctive school teams sponsored by the KISS Institute. Each personalities; allow two-way communication of these events is described in detail in this article. with the spectators; and, if necessary, physical- The contests require entrants to tackle a set of ly nudge spectators out of the way to better established tasks in a competitive environ- serve a wide area of the room. The Hors d’oeu- ment.
    [Show full text]