University of Southern Denmark Dynamic Simulation of Manipulation & Assembly Actions Thulesen, Thomas Nicky Publication date: 2016 Document version: Accepted manuscript Document license: Unspecified Citation for pulished version (APA): Thulesen, T. N. (2016). Dynamic Simulation of Manipulation & Assembly Actions. Syddansk Universitet. Det Tekniske Fakultet. Go to publication entry in University of Southern Denmark's Research Portal Terms of use This work is brought to you by the University of Southern Denmark. Unless otherwise specified it has been shared according to the terms for self-archiving. If no other license is stated, these terms apply: • You may download this work for personal use only. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying this open access version If you believe that this document breaches copyright please contact us providing details and we will investigate your claim. Please direct all enquiries to [email protected] Download date: 05. Oct. 2021 Dynamic Simulation of Manipulation & Assembly Actions Thomas Nicky Thulesen The Maersk Mc-Kinney Moller Institute Faculty of Engineering University of Southern Denmark PhD Dissertation Odense, November 2015 c Copyright 2015 by Thomas Nicky Thulesen All rights reserved. The Maersk Mc-Kinney Moller Institute Faculty of Engineering University of Southern Denmark Campusvej 55 5230 Odense M, Denmark Phone +45 6550 3541 www.mmmi.sdu.dk Abstract To grasp and assemble objects is something that is known as a difficult task to do reliably in a robot system. Yet humans are able to quickly learn how to manipulate and assemble objects based on previous experience. Most learn to do these tasks as infants and the knowledge from previous experience allows solution of new similar, but different, tasks intuitively. In this dissertation dynamic simulation is seen as a necessary tool to allow robots to get the same intuition for solving manipulation and assembly tasks. By virtual modelling of the task, simulation will allow robots to reason about the optimum strategies without complex programming by highly skilled specialists. In this dissertation, a different paradigm is proposed for performing dynamic simulation in the context of tight-fitting assembly. A new physics engine is developed for this purpose. It focuses very much on the geometric aspects of contact generation and temporal coherence between contacts. This allows enhanced possibilities regarding modelling of the interaction between bodies in contact. Regarding development of robust control strategies for assembly, it is important to have a simulated environment that is realistic compared to the real environ- ment. The problem with simulation of tactile sensors is that they lead to redundancy in the mathematical formulation of the dynamical system. In this dissertation, we present a method that includes an objective for distributed force measurements in the case of redundancy, and we show that this objective causes simulated force measurements that are more realistic and more ideal for control strategies. The main purpose of the work performed in this dissertation is to make an engine which is easily extendable, well-documented and also has interfaces that makes it transparent what happens step-by-step during execution. An assembly simulation framework is presented with the purpose to create a unified way of defining assembly actions and results. This also forces the user to write control strategies that are reusable in both simulation and the real world. Different comparisons are performed in order to qualitatively illustrate the benefits from using the newly developed engine for manipulation and assembly. This shows that our ap- proach to dynamic simulation is viable and that, even though it adds some complexity to contact management, it is still very efficient in the tight-fitting assembly use case, while at the same time allowing detailed modelling of the interaction. I II In Danish Resumé Et kendt problem inden for robotteknologi er at gribe og samle objekter på en pålidelig måde. For mennesker er håndtering og samling af objekter en evne, som hurtigt kan tilegnes baseret på forudgående erfaringer. Disse evner tilegnes typisk i børneårene, og på baggrund af vores erfaringer bliver løsning af nye, lignende opgaver intuitiv. I denne afhandling ses dynamisk simulering som et nødvendigt værktøj for at give robotter en lignende intuition for at gribe, håndtere og samle objekter. Ved at opstille en virtuel model vil simulering gøre det muligt for robotten selv at ræsonnere sig frem til den optimale strategi, fremfor at denne skal programmeres eksplicit af højtuddannede specialister. I denne afhandling foreslås et nyt paradigme til dynamisk simulering af samling af objek- ter med lav tolerance. Vi har udviklet en ny fysikmotor med særlig fokus på modellering af kontinuerlig kontakt mellem legemer. Dette giver nye muligheder for at anvende detaljerede modeller for interaktion mellem legemer, der er i kontakt. Udvikling af robuste kontakt- strategier til at samle objekter kræver et simuleringsmiljø, som er realistisk i sammenligning med et fysisk miljø. Simulering af taktile sensorer kan føre til redundans i den matematiske formulering af det fysiske system. Her præsenteres en simuleringsmetode, som involverer dis- tribuerede kræfter som et objektiv i tilfælde af redundans. Det illustreres, hvordan et sådan objektiv giver simulerede kraftmålinger, som er mere realistiske og mere ideelle for udvikling af kontrolstrategier. Hovedformålet med det udførte arbejde er at udvikle en fysikmotor, der kan udvides og er veldokumenteret samt har grænseflader, der gør det transparent, hvad der sker skridt for skridt under simulering. Et simuleringsframework for samling af objekter præsenteres med det formål at udvikle et generelt format for definitionen af samleopgaver og resultater. Målet er, at brugeren tvinges til at skrive kontrolstrategier, som er generiske og kan genbruges både i simulering og i en fysisk opstilling. Forskellige sammenligninger er udført for kvalitativt at illustrere fordelene ved at bruge den nyudviklede fysikmotor til håndtering og samling af objekter. Derved redegøres for, at metoden er en lovende tilgang til dynamisk simulering, samt at fysikmotoren på trods af øget kompleksitet ved detektering af kontakter stadig er effektiv til samling af objekter. Samtidig er det muligt at definere detaljerede modeller for interaktionen mellem objekter. III IV Preface & Acknowledgements For the past four years, I have worked exclusively with dynamic simulation in robotics, focusing on some of the hard fundamental issues in the field. The work performed in my PhD is a continuation of my master’s thesis [1], in which I presented some basic theory about dynamic grasp simulation in the context of grasping of objects. For readers not familiar with advanced rigid body dynamics, the thesis provides a good starting point. It outlines many of the complex issues that are addressed in this dissertation. It is assumed that the reader is familiar with physics, mathematics and robotics at a graduate level. Otherwise, the dissertation will be difficult to follow. The project has been funded by the national projects and Center for Ad- vanced Robotics Manufacturing Engineering (CARMEN). In the former project the context is using service robots in healthcare by assisting patients in their homes after being discharged from hospital. In CARMEN, the task is to perform manipulation actions in an industrial context. Manipulating objects using robots is a fundamental challenge regardless of whether manipulation is being used in service robotics or in industrial applications. I want to thank Professor Henrik Gordon Petersen for supervision and guidance during the years. Much of the work presented in this dissertation are based on a common software framework used in the group, and I want to thank Jimmy Alison Rytz and Lars-Peter Ellekilde for their previous work on the framework. I would like to thank former and present members of the Cognitive and Applied Robotics group at the University of Southern Denmark. People that I have worked with and come to know during the years. This includes Lilita Kiforenko, Leon Bodenhagen, Nadezda Rukavishnikova, Preben Hagh Strunge Holm, Professor Norbert Krüger, Thiusius Rajeeth Savarimuthu, Dirk Kraft, Anders Glent Buch, Jacob Pørksen Buch, Thomas Fridolin Iversen, Thorbjørn Mosekjær Iversen, Troels Bo Jørgensen, Johan Sund Laursen, Simon Mathiesen, Ole Wennerberg Nielsen, Stefan-Daniel Suvei, Lars Carøe Sørensen, Mikkel Tang Thomsen, Zhiyuan Xie, Michael Bejer Andersen, Dorthe Sølvason, Adam Wolniakowski and Severin Fichtl. Finally, I would like to thank my parents, Thule and Jonna, my sister, Michaela Louise Thulesen, and my friends for their patience, understanding and support during the years. Also thanks to Michaela for proofreading. V VI Contents Abstract I Resumé III In Danish Preface & Acknowledgements V Contents VII Figures XI Tables XII Algorithms XIII Listings XIII Notation & Terminology 1 1 Introduction 3 1.1 Kinematic Simulation . 3 1.2 Dynamic Simulation . 4 1.3 Simulation of Manipulation & Assembly . 4 1.4 Physics Engines . 5 1.5 The RobWork Framework . 6 1.6 Contributions . 7 1.7 Overview . 8 2 State of the Art 9 2.1 Overview of High-level Software for Robot Simulation . 9 2.1.1 Abstraction Layers & Data Interchange . 12 2.2 Motion of Rigid Multi-body Systems . 13 2.2.1 Unconstrained Motion of a Rigid Body . 13 2.2.2 Constrained Motion of a Rigid Body . 14 2.2.3 Contacts and Non-penetration Constraints . 15 2.2.4 Friction in a Constraint-based Formulation . 17 2.2.5 Error Correction and Stabilisation . 18 VII VIII Contents 2.3 Rigid Multi-Body Physics Engines in 3D . 18 2.4 Previous Physics Engine Comparisons . 23 2.5 Modelling of Rigid Bodies & Materials . 25 2.5.1 Geometry . 25 2.5.2 Inertial Parameters . 26 2.6 Contact Detection . 26 2.7 Summary . 28 3 A New Engine for Manipulation 29 3.1 The Body-Constraint Graph . 29 3.1.1 Notation . 30 3.1.2 Graph Dynamics . 31 3.1.3 Operations on the Graph .