Teleoperation with Significant Dynamics
Total Page：16
File Type：pdf, Size：1020Kb
Teleoperation with significant dynamics MATTIAS BRATT Licentiate Thesis Stockholm, Sweden 2009 TRITA-CSC-A 2009:16 ISSN-1653-5723 KTH School of Computer Science and Communication ISRN-KTH/CSC/A–09/16-SE SE-100 44 Stockholm ISBN 978-91-7415-486-3 SWEDEN Akademisk avhandling som med tillstånd av Kungl Tekniska högskolan framlägges till offentlig granskning för avläggande av teknologie licentiatexamen i datalogi fredagen den 27 november 2009 klockan 13.00 i hörsal D3, Lindstedtsvägen 5, Kungl Tekniska högskolan, Stockholm. © Mattias Bratt, november 2009 Tryck: Universitetsservice US AB iii Abstract The subject of this thesis is teleoperation, and especially teleoperation with demanding time constraints due to significant dynamics inherent in the task. A comprehensive background is given, describing many aspects of tele- operation, from history and applications to operator interface hardware and relevant control theory concepts. Then follows a presentation of the research done by the author. Two prototypical highly dynamic teleoperation tasks have been attempted: high speed driving, and ball catching. Systems have been developed for both, employing operator interfaces tailored to facilitate perception of the remote scene and including assistive features to promote successful task completion within the required time frame. Prediction of the state at the remote site as well as of operator action has been applied to address the problem of delays arising when using the Internet as the communication channel. v Sammanfattning Detta arbete handlar om teleoperation, som skulle kunna översättas med fjärrstyrning, och speciellt sådan som ställer stränga tidskrav på grund av att uppgiftens natur inbegriper en avsevärd dynamik. Först ges en bred bakgrund där många aspekter av teleoperation belyses, från dess historia och använd- ningsområden till hårdvara för användargränssnitt och relevant reglerteori. Sedan följer en presentation av författarens forskning på området. Två prototypuppgifter har använts, som båda involverar snabba dynamis- ka förlopp: styrning av en mobil robot i hög hastighet och fångst av kastade bollar i luften. Teleoperationssystem har utvecklats för båda uppgifterna. An- vändargränssnit har skräddarsytts för att göra det lättare för operatören att uppfatta vad som händer med och omkring den styrda roboten, och aktiva hjälpmedel har byggts in för att ge större möjligheter att fullgöra uppgiften på tillgänglig tid. Modellering och prediktion av roboten och dess omgivning, men också av operatörens kommandon, har använts för att lösa de fördröjnings- problem som uppstår när internet används som kommunikationsmedium. Acknowledgements First of all I would like to thank my supervisors at CVAP and CAS: Jan-Olof Eklundh introduced me to the world of robotics research during my first years as a PhD student. Later, Henrik I. Christensen helped me restart my studies within the Neurobotics EU project and co-authored the papers in which my work has been published. After Henrik had left KTH for Georgia Tech, Danica Kragic guided my final steps up to the finishing of this thesis. My closest colleagues, Kristina Winbladh, Simone Frintrop, and Christian Smith, deserve special thanks for our many fruitful discussions not only related to work, and for making the traveling to Neurobotics meetings and robotics conferences so much more enjoyable. Parts of the work presented here was done in cooperation with Christian; the teleoperated catching experiments would not have been possible without his participation. I thank all the people at CVAP/CAS for the friendly and creative atmosphere in which I have had the pleasure to work. Patric Jensfelt helped me out many times, e.g. with miscellaneous hardware and software problems. Elena Pacchierotti introduced me to Pepe, Luca, and Dulce, and we spent many nice lunch hours together, during which I learned how to enjoy a good espresso. Thank you Lisa and Peter for your constant support. And finally, Fialotta. I would not have made it without you! This work was funded by the European Union sixth framework program Neu- robotics project (FP6-IST-001917). vii Contents Contents ix 1 Introduction 1 1.1Outlineofthethesis........................... 3 1.2Listofpublications............................ 5 2 Background 7 2.1Teleoperationhistory........................... 10 2.2Userinterfaces.............................. 13 2.2.1 Haptics.............................. 16 2.2.2 Video/3D Graphics . 18 2.3Teleoperationcontroltheory....................... 24 2.3.1 Two-port model of haptic interfaces . 24 2.3.2 Stability and passivity . 26 2.3.3 Wavevariables.......................... 28 2.3.4 Smithprediction......................... 32 2.4Catching/interceptingflyingobjects.................. 33 2.4.1 Human catching and interception . 33 2.4.2 Automated catching and interception . 35 2.5Spacetelerobotics............................ 36 3 Remote robot driving system 41 3.1Controlmodel............................... 42 3.1.1 Basiccontrollerstructure.................... 43 3.1.2 Multiplemeasurementchannelscontroller........... 44 3.2Operatorinterface............................ 46 3.3Obstacleforces.............................. 48 3.3.1 Relatedwork........................... 49 3.3.2 Mapping obstacles to the (v, κ)commandspace........ 50 3.3.3 Obstaclelocations........................ 54 3.4Softwareimplementation......................... 54 3.4.1 Operatorinterfacecomputer.................. 54 3.4.2 Networkcommunication..................... 59 ix x CONTENTS 3.4.3 Robotcomputer......................... 60 3.5Earlyevaluation............................. 61 3.6Openissusesforfuturework....................... 62 4 Teleoperated catching system 65 4.0.1 Robotic hardware and ball sensing . 65 4.1Controlmodel............................... 67 4.1.1 Operator input modeling . 68 4.2Operatorinterface............................ 71 4.2.1 HMD/magnetic tracker interface . 72 4.2.2 Feedbackforces.......................... 73 4.3Softwareimplementation......................... 74 4.3.1 Additional visualization features . 77 4.3.2 Interfacing the Nest of Birds magnetic tracker . 77 4.4 Experiment 1: Catching simulation with one subject . 78 4.4.1 ResultsfromExperiment1 ................... 79 4.4.2 Prediction............................. 79 4.4.3 Trajectoryclassification..................... 82 4.5 Experiment 2: Catching simulation with ten subjects and two interfaces 85 4.5.1 ResultsfromExperiment2 ................... 85 4.6 Experiment 3: Teleoperated catching with user input prediction and semi-autonomy.............................. 89 4.6.1 ResultsfromExperiment3 ................... 91 4.7 Experiment 4: Teleoperated catching with user instruction and semi- autonomy................................. 93 4.7.1 ResultsfromExperiment4 ................... 93 4.8 Conclusions from teleoperated catching . 95 5 Summary and conclusions 99 5.1Teleoperateddriving...........................100 5.2Teleoperatedcatching..........................102 5.3Futurework................................103 Bibliography 105 Chapter 1 Introduction Teleoperation, i.e. using a machine to physically perform actions controlled by a human operator at a different location (Figure 1.1), has been studied for a long time, at least since World War II when it was used for handling nuclear material (Goertz, 1954). Other applications, apart from the handling of dangerous materials, include space telerobotics, as popularly exemplified by the NASA Mars Rover program, and telesurgery where the surgeon and patient might be on separate continents. Traditionally, delays in command and control systems have been handled by predictive methods (Åström and Wittenmark, 1995) or through increased autonomy — i.e. higher level control. Large communication delays are unavoidable when the robot is located on a remote planet, but delays can also be caused by imperfections in the communication channel. This is the case when using the Internet as the teleoperation medium like the teleoperation systems presented in this thesis do. The main question addressed in research presented here is how teleoperation can be performed in the presence of significant dynamics as related to the communica- tion delay in the system. I.e., how can teleoperation tasks be handled, for which the operator has to react to changes in the feedback from the remote end of the system on the same timescale as that of the communication delay? Put in another Figure 1.1: Teleoperation. 1 2 CHAPTER 1. INTRODUCTION delay prediction delay 2 x delay comm. link Figure 1.2: Simplified representation of teleoperation for a highly dynamic task (ball catching) using prediction to compensate for communication delay. way, the situation dealt with is when the time delay Td is significant compared to a time Tp characteristic to the dynamics of the remote robot process. In fact, delays may not only exist in the teleoperation system. There is also the related case of delays being biological, as human reaction times can be significant for some tasks. Since teleoperation systems include a control loop closed over the human op- erator and the communication delay, any such delay may cause instability and/or decreased performance. However, the requirements of a highly dynamic task limit the tolerable amount of performance degradation. The challenge, therefore, is one of achieving adequate performance while still maintaining system stability. Few studies have considered teleoperation in the presence of significant dynamics. Two significantly dynamic remote robot processes