Deep Reinforcement Learning for Soft, Flexible Robots: Brief Review with Impending Challenges
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
An Overview of Novel Actuators for Soft Robotics
actuators Review An Overview of Novel Actuators for Soft Robotics Pinar Boyraz 1,2,* ID , Gundula Runge 3 and Annika Raatz 3 1 Mechanics and Maritime Sciences Department, Chalmers University of Technology, 41296 Gothenburg, Sweden 2 Mechanical Engineering Department, Istanbul Technical University, Istanbul 34437, Turkey 3 Institut fur Montagetechnik (match), Leibniz Universität Hannover, 30823 Garbsen, Hannover, Germany; [email protected] (G.R.); [email protected] (A.R.) * Correspondence: [email protected]; Tel.: +46-730-49-8780 Received: 10 June 2018; Accepted: 9 August 2018; Published: 16 August 2018 Abstract: In this systematic survey, an overview of non-conventional actuators particularly used in soft-robotics is presented. The review is performed by using well-defined performance criteria with a direction to identify the exemplary and potential applications. In addition to this, initial guidelines to compare the performance and applicability of these novel actuators are provided. The meta-analysis is restricted to five main types of actuators: shape memory alloys (SMAs), fluidic elastomer actuators (FEAs), shape morphing polymers (SMPs), dielectric electro-activated polymers (DEAPs), and magnetic/electro-magnetic actuators (E/MAs). In exploring and comparing the capabilities of these actuators, the focus was on eight different aspects: compliance, topology-geometry, scalability-complexity, energy efficiency, operation range, modality, controllability, and technological readiness level (TRL). The overview presented here provides a state-of-the-art summary of the advancements and can help researchers to select the most convenient soft actuators using the comprehensive comparison of the suggested quantitative and qualitative criteria. Keywords: soft-robotics; actuator performance; SMA; SMP; FEA; DEAP; E/MA 1. -
Learning Maps for Indoor Mobile Robot Navigation
Learning Maps for Indoor Mobile Robot Navigation Sebastian Thrun Arno BÈucken April 1996 CMU-CS-96-121 School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 The ®rst author is partly and the second author exclusively af®liated with the Computer Science Department III of the University of Bonn, Germany, where part of this research was carried out. This research is sponsored in part by the National Science Foundation under award IRI-9313367, and by the Wright Laboratory, Aeronautical Systems Center, Air Force Materiel Command, USAF, and the Advanced Research Projects Agency (ARPA) under grant number F33615-93-1-1330. The views and conclusionscontained in this documentare those of the author and should not be interpreted as necessarily representing of®cial policies or endorsements, either expressed or implied, of NSF, Wright Laboratory or the United States Government. Keywords: autonomous robots, exploration, mobile robots, neural networks, occupancy grids, path planning, planning, robot mapping, topological maps Abstract Autonomous robots must be able to learn and maintain models of their environ- ments. Research on mobile robot navigation has produced two major paradigms for mapping indoor environments: grid-based and topological. While grid-based methods produce accurate metric maps, their complexity often prohibits ef®cient planning and problem solving in large-scale indoor environments. Topological maps, on the other hand, can be used much more ef®ciently, yet accurate and consistent topological maps are considerably dif®cult to learn in large-scale envi- ronments. This paper describes an approach that integrates both paradigms: grid-based and topological. Grid-based maps are learned using arti®cial neural networks and Bayesian integration. -
Capability by Stacking: the Current Design Heuristic for Soft Robots
biomimetics Review Capability by Stacking: The Current Design Heuristic for Soft Robots Stephen T. Mahon 1 ID , Jamie O. Roberts 1,2, Mohammed E. Sayed 1, Derek Ho-Tak Chun 1,2 ID , Simona Aracri 1, Ross M. McKenzie 1,2, Markus P. Nemitz 1,3 and Adam A. Stokes 1,* 1 School of Engineering, The Institute for Integrated Micro and Nano Systems, The University of Edinburgh, The King’s Buildings, Edinburgh EH9 3LJ, UK; [email protected] (S.T.M.); [email protected] (J.O.R.); [email protected] (M.E.S.); [email protected] (D.H.-T.C.); [email protected] (S.A.); [email protected] (R.M.M.); [email protected] (M.P.N.) 2 Engineering and Physical Sciences Research Council (EPSRC) Centre for Doctoral Training (CDT) in Robotics and Autonomous Systems, School of Informatics, The University of Edinburgh, Edinburgh EH9 3LJ, UK 3 Department of Computer Science and Engineering, University of Michigan, 2260 Hayward St. BBB3737, Ann Arbor, MI 48109, USA * Correspondence: [email protected]; Tel.: +44-131-650-5611 Received: 1 June 2018; Accepted: 10 July 2018; Published: 13 July 2018 Abstract: Soft robots are a new class of systems being developed and studied by robotics scientists. These systems have a diverse range of applications including sub-sea manipulation and rehabilitative robotics. In their current state of development, the prevalent paradigm for the control architecture in these systems is a one-to-one mapping of controller outputs to actuators. -
Design and Evaluation of Modular Robots for Maintenance in Large Scientific Facilities
UNIVERSIDAD POLITÉCNICA DE MADRID ESCUELA TÉCNICA SUPERIOR DE INGENIEROS INDUSTRIALES DESIGN AND EVALUATION OF MODULAR ROBOTS FOR MAINTENANCE IN LARGE SCIENTIFIC FACILITIES PRITHVI SEKHAR PAGALA, MSC 2014 DEPARTAMENTO DE AUTOMÁTICA, INGENIERÍA ELECTRÓNICA E INFORMÁTICA INDUSTRIAL ESCUELA TÉCNICA SUPERIOR DE INGENIEROS INDUSTRIALES DESIGN AND EVALUATION OF MODULAR ROBOTS FOR MAINTENANCE IN LARGE SCIENTIFIC FACILITIES PhD Thesis Author: Prithvi Sekhar Pagala, MSC Advisors: Manuel Ferre Perez, PhD Manuel Armada, PhD 2014 DESIGN AND EVALUATION OF MODULAR ROBOTS FOR MAINTENANCE IN LARGE SCIENTIFIC FACILITIES Author: Prithvi Sekhar Pagala, MSC Tribunal: Presidente: Dr. Fernando Matía Espada Secretario: Dr. Claudio Rossi Vocal A: Dr. Antonio Giménez Fernández Vocal B: Dr. Juan Antonio Escalera Piña Vocal C: Dr. Concepción Alicia Monje Micharet Suplente A: Dr. Mohamed Abderrahim Fichouche Suplente B: Dr. José Maráa Azorín Proveda Acuerdan otorgar la calificación de: Madrid, de de 2014 Acknowledgements The duration of the thesis development lead to inspiring conversations, exchange of ideas and expanding knowledge with amazing individuals. I would like to thank my advisers Manuel Ferre and Manuel Armada for their constant men- torship, support and motivation to pursue different research ideas and collaborations during the course of entire thesis. The team at the lab has not only enriched my professionally life but also in Spanish ways. Thank you all the members of the ROMIN, Francisco, Alex, Jose, Jordi, Ignacio, Javi, Chema, Luis .... This research project has been supported by a Marie Curie Early Stage Initial Training Network Fellowship of the European Community’s Seventh Framework Program "PURESAFE". I wish to thank the supervisors and fellow research members of the project for the amazing support, fruitful interactions and training events. -
Learning for Microrobot Exploration: Model-Based Locomotion, Sparse-Robust Navigation, and Low-Power Deep Classification
Learning for Microrobot Exploration: Model-based Locomotion, Sparse-robust Navigation, and Low-power Deep Classification Nathan O. Lambert1, Farhan Toddywala1, Brian Liao1, Eric Zhu1, Lydia Lee1, and Kristofer S. J. Pister1 Abstract— Building intelligent autonomous systems at any Classification Intelligent, mm scale Fast Downsampling scale is challenging. The sensing and computation constraints Microrobot of a microrobot platform make the problems harder. We present improvements to learning-based methods for on-board learning of locomotion, classification, and navigation of microrobots. We show how simulated locomotion can be achieved with model- Squeeze-and-Excite Hard Activation System-on-chip: based reinforcement learning via on-board sensor data distilled Camera, Radio, Battery into control. Next, we introduce a sparse, linear detector and a Dynamic Thresholding method to FAST Visual Odometry for improved navigation in the noisy regime of mm scale imagery. Locomotion Navigation We end with a new image classifier capable of classification Controller Parameter Unknown Dynamics Original Training Image with fewer than one million multiply-and-accumulate (MAC) Optimization Modeling & Simulator Resulting Map operations by combining fast downsampling, efficient layer Reward structures and hard activation functions. These are promising … … SLIPD steps toward using state-of-the-art algorithms in the power- Ground Truth Estimated Pos. State Space limited world of edge-intelligence and microrobots. Sparse I. INTRODUCTION Local Control PID: K K K Microrobots have been touted as a coming revolution p d i for many tasks, such as search and rescue, agriculture, or Fig. 1: Our vision for microrobot exploration based on distributed sensing [1], [2]. Microrobotics is a synthesis of three contributions: 1) improving data-efficiency of learning Microelectromechanical systems (MEMs), actuators, power control, 2) a more noise-robust and novel approach to visual electronics, and computation. -
Robot Learning
Robot Learning 15-494 Cognitive Robotics David S. Touretzky & Ethan Tira-Thompson Carnegie Mellon Spring 2009 04/06/09 15-494 Cognitive Robotics 1 What Can Robots Learn? ● Parameter tuning, e.g., for a faster walk ● Perceptual learning: ALVINN driving the Navlab ● Map learning, e.g., SLAM algorithms ● Behavior learning; plans and macro-operators – Shakey the Robot (SRI) – Robo-Soar ● Learning from human teachers – Operant conditioning: Skinnerbots – Imitation learning 04/06/09 15-494 Cognitive Robotics 2 Lots of Work on Robot Learning ● IEEE Robotics and Automation Society – Technical Committee on Robot Learning – http://www.learning-robots.de ● Robot Learning Summer School – Lisbon, Portugal; July 20-24, 2009 ● Workshops at major robotics conferences – ICRA 2009 workshop: Approachss to Sensorimotor Learning on Humanoid Robots – Kobe, Japan; May 17, 2009 04/06/09 15-494 Cognitive Robotics 3 Parameter Optimization ● How fast can an AIBO walk? Figures from Kohl & Stone, ICRA 2004, for the ERS-210 model: – CMU (2002) 200 mm/s – German Team 230 mm/s Hand-tuned gaits – UT Austin Villa 245 mm/s – UNSW 254 mm/s – Hornsby (1999) 170 mm/s – UNSW 270 mm/s Learned gaits – UT Austin Villa 291 mm/s 04/06/09 15-494 Cognitive Robotics 4 Walk Parameters 12 parameters to optimize: ● Front locus (height, x pos, ypos) ● Rear locus (height, x pos, y pos) ● Locus length ● Locus skew multiplier (in the x-y plane, for turning) ● Height of front of body ● Height of rear of body From Kohl & Stone (ICRA 2004) ● Foot travel time ● Fraction of time foot is on ground 04/06/09 15-494 Cognitive Robotics 5 Optimization Strategy ● “Policy gradient reinforcement learning”: – Walk parameter assignment = “policy” – Estimate the gradient along each dimension by trying combinations of slight perturbations in all parameters – Measure walking speed on the actual robot – Optimize all 12 parameters simultaneously – Adjust parameters according to the estimated gradient. -
Customizing a Self-Healing Soft Pump for Robot
ARTICLE https://doi.org/10.1038/s41467-021-22391-x OPEN Customizing a self-healing soft pump for robot ✉ Wei Tang 1, Chao Zhang 1 , Yiding Zhong1, Pingan Zhu1,YuHu1, Zhongdong Jiao 1, Xiaofeng Wei1, ✉ Gang Lu1, Jinrong Wang 1, Yuwen Liang1, Yangqiao Lin 1, Wei Wang1, Huayong Yang1 & Jun Zou 1 Recent advances in soft materials enable robots to possess safer human-machine interaction ways and adaptive motions, yet there remain substantial challenges to develop universal driving power sources that can achieve performance trade-offs between actuation, speed, portability, and reliability in untethered applications. Here, we introduce a class of fully soft 1234567890():,; electronic pumps that utilize electrical energy to pump liquid through electrons and ions migration mechanism. Soft pumps combine good portability with excellent actuation per- formances. We develop special functional liquids that merge unique properties of electrically actuation and self-healing function, providing a direction for self-healing fluid power systems. Appearances and pumpabilities of soft pumps could be customized to meet personalized needs of diverse robots. Combined with a homemade miniature high-voltage power con- verter, two different soft pumps are implanted into robotic fish and vehicle to achieve their untethered motions, illustrating broad potential of soft pumps as universal power sources in untethered soft robotics. ✉ 1 State Key Laboratory of Fluid Power and Mechatronic Systems, Zhejiang University, Hangzhou, China. email: [email protected]; [email protected] NATURE COMMUNICATIONS | (2021) 12:2247 | https://doi.org/10.1038/s41467-021-22391-x | www.nature.com/naturecommunications 1 ARTICLE NATURE COMMUNICATIONS | https://doi.org/10.1038/s41467-021-22391-x nspired by biological systems, scientists and engineers are a robotic vehicle to achieve untethered and versatile motions Iincreasingly interested in developing soft robots1–4 capable of when the customized soft pumps are implanted into them. -
Towards Autonomous Localization of an Underwater Drone
TOWARDS AUTONOMOUS LOCALIZATION OF AN UNDERWATER DRONE A Thesis presented to the Faculty of California Polytechnic State University, San Luis Obispo In Partial Fulfillment of the Requirements for the Degree Master of Science in Computer Science by Nathan Sfard June 2018 c 2018 Nathan Sfard ALL RIGHTS RESERVED ii COMMITTEE MEMBERSHIP TITLE: Towards Autonomous Localization of an Underwater Drone AUTHOR: Nathan Sfard DATE SUBMITTED: June 2018 COMMITTEE CHAIR: Lynne Slivovsky, Ph.D. Professor of Computer Engineering COMMITTEE MEMBER: John Seng, Ph.D. Professor of Computer Science COMMITTEE MEMBER: Xiao-Hua Yu, Ph.D. Professor of Electrical Engineering iii ABSTRACT Towards Autonomous Localization of an Underwater Drone Nathan Sfard Autonomous vehicle navigation is a complex and challenging task. Land and aerial vehicles often use highly accurate GPS sensors to localize themselves in their envi- ronments. These sensors are ineffective in underwater environments due to signal attenuation. Autonomous underwater vehicles utilize one or more of the following approaches for successful localization and navigation: inertial/dead-reckoning, acous- tic signals, and geophysical data. This thesis examines autonomous localization in a simulated environment for an OpenROV Underwater Drone using a Kalman Fil- ter. This filter performs state estimation for a dead reckoning system exhibiting an additive error in location measurements. We evaluate the accuracy of this Kalman Filter by analyzing the effect each parameter has on accuracy, then choosing the best combination of parameter values to assess the overall accuracy of the Kalman Filter. We find that the two parameters with the greatest effects on the system are the con- stant acceleration and the measurement uncertainty of the system. -
Position Control for Soft Actuators, Next Steps Toward Inherently Safe Interaction
electronics Article Position Control for Soft Actuators, Next Steps toward Inherently Safe Interaction Dongshuo Li, Vaishnavi Dornadula, Kengyu Lin and Michael Wehner * Department of Electrical and Computer Engineering, University of California, Santa Cruz, CA 95064, USA; [email protected] (D.L.); [email protected] (V.D.); [email protected] (K.L.) * Correspondence: [email protected] Abstract: Soft robots present an avenue toward unprecedented societal acceptance, utility in popu- lated environments, and direct interaction with humans. However, the compliance that makes them attractive also makes soft robots difficult to control. We present two low-cost approaches to control the motion of soft actuators in applications common in human-interaction tasks. First, we present a passive impedance approach, which employs restriction to pneumatic channels to regulate the inflation/deflation rate of a pneumatic actuator and eliminate the overshoot/oscillation seen in many underdamped silicone-based soft actuators. Second, we present a visual servoing feedback control approach. We present an elastomeric pneumatic finger as an example system on which both methods are evaluated and compared to an uncontrolled underdamped actuator. We perturb the actuator and demonstrate its ability to increase distal curvature around the obstacle and maintain the desired end position. In this approach, we use the continuum deformation characteristic of soft actuators as an advantage for control rather than a problem to be minimized. With their low cost and complexity, these techniques present great opportunity for soft robots to improve human–robot interaction. Citation: Li, D.; Dornadula, V.; Lin, Keywords: soft robot; human–robot interaction; control K.; Wehner, M. Position Control for Soft Actuators, Next Steps toward Inherently Safe Interaction. -
Lab Notebook – Soft Robotics
Technology Lab: Lab Notebook – Soft Robotics ã2018 Boy Scouts of America. All rights reserved. No portion of this publication may be reproduced, stored in a retrieval system, or transmitted in any form by any means without the proper written permission of the Boy Scouts of America or as expressly permitted by law. The design of a circle divided into four quadrants with wavy lines along the perimeter (the “BSA STEM logo”) is a trademark of the Boy Scouts of America. Lab Notebook Soft Robotics OVERVIEW ........................................................................................................................... 3 MEETING 1: SOFT ROBOT STRUCTURES ............................................................................. 5 MEETING 2: POWERING SOFT ROBOTS ............................................................................. 16 MEETING 3: BUILD A SOFT ROBOTIC GRASPER ............................................................... 24 MEETING 4: BUILD A SOFT TENTACLE-ARM ROBOT – PART 1 ......................................... 30 MEETING 5: BUILD A SOFT TENTACLE-ARM ROBOT – PART 2 ........................................ 36 MEETING 6: PROGRAMMING MOTIONS ............................................................................41 2 Lab Notebook Soft Robotics Overview This module will delve into an exciting and newly developing branch of robotics—soft or flexible robotic structures. When you look at an elephant’s trunk or an octopus’s arm, you can see natural examples of flexible structures used to grasp and manipulate objects. There are many tasks in our world that a rigid structure just does not perform well. Soft Robotics explores the development of flexible structures to work in these areas. In this module, you and your team will explore the concept of soft robots and learn how flexible structures move and how they can be programmed to perform useful tasks. This module was developed by the REACH Lab at the University of Tennessee, Knoxville. The group focuses on medical applications of robots and soft robots, and the mathematics behind them. -
Systematic Review of Research Trends in Robotics Education for Young Children
sustainability Review Systematic Review of Research Trends in Robotics Education for Young Children Sung Eun Jung 1,* and Eun-sok Won 2 1 Department of Educational Theory and Practice, University of Georgia, Athens, GA 30602, USA 2 Department of Liberal Arts Education, Mokwon University, Daejeon 35349, Korea; [email protected] * Correspondence: [email protected]; Tel.: +1-706-296-3001 Received: 31 January 2018; Accepted: 13 March 2018; Published: 21 March 2018 Abstract: This study conducted a systematic and thematic review on existing literature in robotics education using robotics kits (not social robots) for young children (Pre-K and kindergarten through 5th grade). This study investigated: (1) the definition of robotics education; (2) thematic patterns of key findings; and (3) theoretical and methodological traits. The results of the review present a limitation of previous research in that it has focused on robotics education only as an instrumental means to support other subjects or STEM education. This study identifies that the findings of the existing research are weighted toward outcome-focused research. Lastly, this study addresses the fact that most of the existing studies used constructivist and constructionist frameworks not only to design and implement robotics curricula but also to analyze young children’s engagement in robotics education. Relying on the findings of the review, this study suggests clarifying and specifying robotics-intensified knowledge, skills, and attitudes in defining robotics education in connection to computer science education. In addition, this study concludes that research agendas need to be diversified and the diversity of research participants needs to be broadened. To do this, this study suggests employing social and cultural theoretical frameworks and critical analytical lenses by considering children’s historical, cultural, social, and institutional contexts in understanding young children’s engagement in robotics education. -
Concevoir Et Animer Pour L'acceptation De Robots
Concevoir et animer pour l’acceptation de robots zoomorphiques Adrien Gomez To cite this version: Adrien Gomez. Concevoir et animer pour l’acceptation de robots zoomorphiques. Art et histoire de l’art. Université Toulouse le Mirail - Toulouse II, 2018. Français. NNT : 2018TOU20048. tel- 02466318 HAL Id: tel-02466318 https://tel.archives-ouvertes.fr/tel-02466318 Submitted on 4 Feb 2020 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Délivré par Université Toulouse – Jean Jaurès Adrien Gomez Le 30 Août 2018 Concevoir et Animer pour l’Acceptation de Robots Zoomorphiques École doctorale et discipline ou spécialité ED ALLPH@: Études audiovisuelles Unité de recherche LARA (Université Toulouse Jean Jaurès) – LIRMM (Université Montpellier) Directeur(s) de Thèse M. Gilles METHEL M. René Zapata Co-encadrant de Thèse M. Sébastien Druon Jury Chu-Yin Chen, Professeure, université Paris 8 Sylvie Lelandais, Professeure, Université Evry-Val d’Essonnes Gilles Methel, Professeur, Université Toulouse2-Jean Jaurès René Zapata, Professeur, LIRMM Montpellier Sébastien