DOCSLIB.ORG

Teleoperation of Autonomous Vehicle with 360° Camera Feedback

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Teleoperation of Autonomous Vehicle with 360° Camera Feedback Teleoperation of Autonomous Vehicle With 360° Camera Feedback Master’s thesis in Systems, Control and Mechatronics OSCAR BODELL ERIK GULLIKSSON Department of Signals and Systems CHALMERS UNIVERSITY OF TECHNOLOGY Gothenburg, Sweden 2016 Master’s thesis in systems, control and mechatronics Teleoperation of Autonomous Vehicle With 360° Camera Feedback OSCAR BODELL ERIK GULLIKSSON Department of Signals and Systems Division of Control, Automation and Mechatronics Chalmers University of Technology Gothenburg, Sweden 2016 Teleoperation of Autonomous Vehicle With 360° Camera Feedback OSCAR BODELL ERIK GULLIKSSON © OSCAR BODELL, ERIK GULLIKSSON 2016. Master’s Thesis 2016:43 Department of Signals and Systems Division of Control, Automation and Mechatronics Chalmers University of Technology SE-412 96 Gothenburg Sweden Telephone +46 (0)31 772 1000 Cover: Teleoperation setup with stitched camera view & graphical user interface. Chalmers Reproservice Gothenburg, Sweden 2016 Teleoperation of Autonomous Vehicle With 360° Camera Feedback Master’s thesis in Systems, Control and Mechatronics OSCAR BODELL ERIK GULLIKSSON Department of Signals and Systems Division of Control, Automation and Mechatronics Chalmers University of Technology Abstract Teleoperation is using remote control from outside line of sight. The operator is often assisted by cameras to emulate sitting in the vehicle. In this report a system for tele- operation of an autonomous Volvo FMX truck is specified, designed, implemented and evaluated. First a survey of existing solutions on the market is conducted find- ing that there are a few autonomous and teleoperation solutions available, but they are still new and information is sparse. To identify what types of requirements are needed for such a system and how to design it a literature study is performed. The system is then designed from the set requirements in a modular fashion using the Robot Operating System as the underlying framework. Four cameras are mounted on the cab and in software the images are stitched together into one 360◦ image that the operator can pan around in. The system is designed so that the operator at any time can pause the autonomous navigation and manually control the vehicle via teleoperation. When the operators intervention is completed the truck can resume autonomous navigation. A solution for synchronization between manual and autonomous mode is specified. The truck is implemented in a simulation where the functionality and requirements of the system is evaluated by a group of test subjects driving a test track. Results from simulation show that latencies higher than 300 ms lead to difficulties when driving, but having a high frame rate is not as critical. The benefit of a full 360◦ camera view compared to a number of fixed cameras in strategic places is not obvious. The use of a head mounted display together with the 360◦ video would be of interest to investigate further. Keywords: Teleoperation, autonomous vehicle, surround view, remote steering. i Acknowledgements First of all we would like to thank CPAC Systems for letting us become part of the family and our supervisors at CPAC Annegret Render and Michael Pettersson. Thank you also Knut Åkesson for being our supervisor and examiner at Chalmers University of Technology and helping us with the report. A final big thank you to Josef Kindberg for your help and expertise in ROS. Oscar Bodell, Erik Gulliksson, Gothenburg, June 2016 iii Contents Abstracti Acknowledgements iii Contentsv Nomenclature ix List of Figures xi List of Tables xiii 1 Introduction1 1.1 Purpose & Objective........................... 1 1.2 Main Research Questions......................... 2 1.3 Boundaries ................................ 2 1.4 Method .................................. 2 1.5 Thesis Contribution............................ 3 1.6 Thesis Outline............................... 4 2 Application Overview5 2.1 Level of Autonomy............................ 5 2.2 Evaluation vehicle............................. 6 2.3 Cameras and stitching .......................... 6 2.4 Visualization and Operator Support................... 6 2.5 User Controls............................... 7 2.6 Autonomous Functions.......................... 7 2.7 Implementation.............................. 8 3 Existing Solutions and Standards9 3.1 Existing standards for remote control.................. 9 3.2 Original Equipment Manufacturer Solutions .............. 9 3.3 Retrofit Solutions............................. 11 3.4 Outcome of Market Survey........................ 12 4 System Requirements 13 4.1 Identification of Requirements...................... 13 4.2 Video Feedback.............................. 14 4.3 Latency Effects.............................. 15 v Contents 4.4 Sensor Data Presentation......................... 16 4.4.1 Map ................................ 16 4.5 Driver Control Inputs........................... 16 4.6 Autonomous Synchronization ...................... 17 4.7 Communication Interface......................... 17 4.8 Communication Link........................... 17 4.8.1 Free-space Path Loss....................... 18 4.8.2 Alternatives to Wireless ..................... 18 4.8.3 5th Generation Wireless Systems ................ 19 4.8.4 Link Requirement......................... 19 4.9 Safety Requirements ........................... 20 5 System Design 21 5.1 System Architecture ........................... 21 5.1.1 Vehicle............................... 21 5.1.2 User Interface........................... 22 5.2 Framework - Robot Operating System ................. 23 5.3 Subsystems ................................ 23 5.3.1 Cameras.............................. 24 5.3.2 Image Processing......................... 24 5.3.2.1 Image Stitching..................... 25 5.3.2.2 Information Overlay .................. 27 5.3.3 Maps ............................... 28 5.3.4 Control Inputs .......................... 29 5.3.4.1 Haptic Feedback .................... 30 5.3.5 Additional Sensors ........................ 30 5.3.5.1 Light Detection And Ranging............. 30 5.3.5.2 Global Navigation Satellite System.......... 30 5.3.5.3 Inertial Measurement Unit .............. 31 5.4 System Coordinator............................ 31 5.4.1 Graphical User Interface..................... 32 5.4.2 Autonomous Functions...................... 32 5.4.2.1 Navigation ....................... 33 5.4.2.2 Synchronization..................... 33 5.4.2.3 Paths .......................... 33 5.5 Gazebo Simulation ............................ 34 5.5.1 Visual ............................... 34 5.5.2 Mass and Inertia ......................... 35 5.5.3 Collision.............................. 36 5.5.4 Propulsion............................. 36 5.5.5 Steering.............................. 36 5.5.6 Sensors............................... 37 5.5.7 World ............................... 38 5.5.8 Performance............................ 38 6 Results 39 6.1 Standards and System Layout...................... 39 vi Contents 6.2 Autonomous synchronization ...................... 40 6.3 Evaluation................................. 41 6.3.1 Driving Experience and Support Functions........... 41 6.3.2 Impact of Latency ........................ 43 6.4 Results Summary............................. 45 7 Conclusion & Future Work 47 Bibliography 49 vii Contents viii Nomenclature ∗B Nr. of Bytes ∗b Nr. of bits 5G 5th Generation Wireless Systems AHS Autonomous Haulage System CAN Controller Area Network CPU Central Processing Unit FOV Field Of View FPS Frames Per Second FSPL Free-Space Path Loss GNSS Global Navigation Satellite System GPS Global Positioning System (see GNSS) GPU Graphics Processing Unit GUI Graphical User Interface HMD Head Mounted Display HSV Hue Saturation Value IEEE Institute of Electrical and Electronics Engineers IMU Inertial Measurement Unit IP Internet Protocol LAN Local Area Network LiDAR Light Detection And Ranging LOS Line Of Sight OEM Original Equipment Manufacturer OpenCV Open source Computer Vision library PoE Power over Ethernet RGB Red Green Blue ROI Region Of Interest ROS Robot Operating System RTK Real Time Kinematic RTSP Real Time Streaming Protocol RTT Round Trip Time SLAM Simultaneous Localization And Mapping Volvo FMX Volvo Forward control Medium Xtreme VR Virtual Reality Wi-Fi IEEE 802.11xx wireless communication WLAN Wireless Local Area Network ix x List of Figures 2.1 Screenshot of the evaluation vehicle in the Gazebo simulation..... 6 2.2 Screenshot from the stitched video feed overlaid with map, speedome- ter and align assistance for autonomous paths.............. 7 2.3 Screenshot from the operator interface controlling autonomous func- tions and maps............................... 8 5.1 System overview of control center and vehicle with communication nodes.................................... 22 5.2 Four cameras with a 120◦ FOV. Mounted to capture a complete 360◦ view. ................................... 24 5.3 IP camera used for surround view of the vehicle............ 25 5.4 Illustration of the effects of the homography matrix........... 26 5.5 110◦ Stitched camera view from vehicle in simulation with map and offset to chosen path............................ 28 5.6 Overview maps with surroundings and recorded paths. 29 5.7 LiDAR sensor data presentation. .................... 29 5.8 Screenshot of the GUI .......................... 32 5.9 The model of the Volvo FMX truck in Gazebo simulation. 35 5.10 The inertial model of the truck...................... 35 5.11 The collision model of the truck. .................... 36 5.12
Load more

Recommended publications
  • Review of Advanced Medical Telerobots
    applied sciences Review Review of Advanced Medical Telerobots Sarmad Mehrdad 1,†, Fei Liu 2,† , Minh Tu Pham 3 , Arnaud Lelevé 3,* and S. Farokh Atashzar 1,4,5 1 Department of Electrical and Computer Engineering, New York University (NYU), Brooklyn, NY 11201, USA; [email protected] (S.M.); [email protected] (S.F.A.) 2 Advanced Robotics and Controls Lab, University of San Diego, San Diego, CA 92110, USA; [email protected] 3 Ampère, INSA Lyon, CNRS (UMR5005), F69621 Villeurbanne, France; [email protected] 4 Department of Mechanical and Aerospace Engineering, New York University (NYU), Brooklyn, NY 11201, USA 5 NYU WIRELESS, Brooklyn, NY 11201, USA * Correspondence: [email protected]; Tel.: +33-0472-436035 † Mehrdad and Liu contributed equally to this work and share the first authorship. Abstract: The advent of telerobotic systems has revolutionized various aspects of the industry and human life. This technology is designed to augment human sensorimotor capabilities to extend them beyond natural competence. Classic examples are space and underwater applications when distance and access are the two major physical barriers to be combated with this technology. In modern examples, telerobotic systems have been used in several clinical applications, including teleoperated surgery and telerehabilitation. In this regard, there has been a significant amount of research and development due to the major benefits in terms of medical outcomes. Recently telerobotic systems are combined with advanced artificial intelligence modules to better share the agency with the operator and open new doors of medical automation. In this review paper, we have provided a comprehensive analysis of the literature considering various topologies of telerobotic systems in the medical domain while shedding light on different levels of autonomy for this technology, starting from direct control, going up to command-tracking autonomous telerobots.
    [Show full text]
  • Panospheric Video for Robotic Telexploration
    Panospheric Video for Robotic Telexploration John R. Murphy CMU-RI-TR-98-10 Submined in partial fulfiient of the requirements for the degree of Doctor of Philosophy in Robotics The Robotics Institute Cknegie Mellon University Pittsburgh, Pennsylvania 15213 May 1998 0 1998 by John R. Murphy. All rights reserved. This research was supported in part by NASA grant NAGW-117.5. The views and conclusions contained in this document are those of the author and should not be interpreted as representing the official policies, either expressed or implied, of NASA or the U.S. Government. Robotics Thesis Panospheric Video for Robotic Telexploration John Murphy Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in the field of Robotics ACC-. William L. Whittaka TheEis Cormittee Chair Date Matthew T. Mason- - Program Chir Y Date APPROVED: fL-4 36 JiiIff+? Pa Christian0 Rowst Date - Abstract Teleoperation using cameras and monitors fails to achieve the rich and natural perceptions available during direct hands-on operation. Optimally. a reniote telepresence reproduces visual sensations that enable equivalent understanding and interaction as direct operation. Through irnrnersive video acquisition and display. the situational awareness ola remote operator is brought closer lo that of direct operation. Early teleoperation research considered the value of wide tields of view. Howe~er. utilization of wide fields of view meant sacrificing resolution and presenting distorted imagery to the viewer. With recent advances in technology. these problems are no longer barriers to hyper-wide field of view video. Acquisition and display of video conveying the entire surroundings of a tclcoperated mobile robot requires innolwtion of hardware and software.
    [Show full text]
  • Teleoperation of a 7 DOF Humanoid Robot Arm Using Human Arm Accelerations and EMG Signals
    Teleoperation of a 7 DOF Humanoid Robot Arm Using Human Arm Accelerations and EMG Signals J. Leitner, M. Luciw, A. Forster,¨ J. Schmidhuber Dalle Molle Institute for Artificial Intelligence (IDSIA) & Scuola universitaria professionale della Svizzera Italiana (SUPSI) & Universita` della Svizzera italiana (USI), Switzerland e-mail: [email protected] Abstract 2 Background and Related Work We present our work on tele-operating a complex hu- The presented research combines a variety of subsys- manoid robot with the help of bio-signals collected from tems together to allow the safe and robust teleoperation of the operator. The frameworks (for robot vision, collision a complex humanoid robot. For the experiments herein, avoidance and machine learning), developed in our lab, we are using computer vision, together with accelerom- allow for a safe interaction with the environment, when eters and EMG signals collected on the operator’s arm combined. This even works with noisy control signals, to remotely control the 7 degree-of-freedom (DOF) robot such as, the operator’s hand acceleration and their elec- arm – and the 5 fingered end-effector – during reaching tromyography (EMG) signals. These bio-signals are used and grasping tasks. to execute equivalent actions (such as, reaching and grasp- ing of objects) on the 7 DOF arm. 2.1 Robotic Teleoperation Telerobotics usually refers to the remote control of 1 Introduction robotic systems over a (wireless) link. First use of tele- robotics dates back to the first robotic space probes used More and more robots are sent to distant or dangerous during the 20-th century. Remote manipulators are nowa- environments to perform a variety of tasks.
    [Show full text]
  • Autonomous Unmanned Ground Vehicle and Indirect Driving
    Autonomous Unmanned Ground Vehicle and Indirect Driving ABSTRACT This paper describes the design and challenges faced in the development of an Autonomous Unmanned Ground Vehicle (AUGV) demonstrator and an Indirect Driving Testbed (IDT). The AUGV has been modified from an M113 Armoured Personnel Carrier and has capabilities such as remote control, autonomous waypoint seeking, obstacle avoidance, road following and vehicle following. The IDT also enables the driver of an armoured vehicle to drive by looking at live video feed from cameras mounted on the exterior of the vehicle. These two technologies are complementary, and integration of the two technology enablers could pave the way for the deployment of semi-autonomous unmanned (and even manned) vehicles in battles. Ng Keok Boon Tey Hwee Choo Chan Chun Wah Autonomous Unmanned Ground Vehicle and Indirect Driving ABSTRACT This paper describes the design and challenges faced in the development of an Autonomous Unmanned Ground Vehicle (AUGV) demonstrator and an Indirect Driving Testbed (IDT). The AUGV has been modified from an M113 Armoured Personnel Carrier and has capabilities such as remote control, autonomous waypoint seeking, obstacle avoidance, road following and vehicle following. The IDT also enables the driver of an armoured vehicle to drive by looking at live video feed from cameras mounted on the exterior of the vehicle. These two technologies are complementary, and integration of the two technology enablers could pave the way for the deployment of semi-autonomous unmanned (and even manned) vehicles in battles. Ng Keok Boon Tey Hwee Choo Chan Chun Wah Autonomous Unmanned Ground Vehicle and Indirect Driving 54 AUGV has been tested.
    [Show full text]
  • Advanced C2 Vehicle Crewstation for Control of Unmanned Ground Vehicles
    UNCLASSIFIED/UNLIMITED Advanced C2 Vehicle Crewstation for Control of Unmanned Ground Vehicles David Dahn Bruce Brendle Marc Gacy, PhD U.S. Army Tank-automotive & Micro Analysis & Design Inc. Armaments Command 4949 Pearl East Circle, Ste 300 AMSTA-TR-R (MS 264: Brendle) Boulder, CO 80301 Warren, MI 48397-5000 UNITED STATES UNITED STATES Tele: (303) 442-6947 Phone: 586 574-5798 Fax: (303) 442-8274 Fax: 586 574-8684 Email: [email protected] / [email protected] [email protected] 1.0 INTRODUCTION 1.1 Rationale Future military operational requirements may utilize unmanned or unattended assets for certain military missions that currently place military personnel in harms way for performing jobs and tasks that can be accomplished with robotic assets. To answer these needs, the US Army established two advanced technology demonstration (ATD) programs, the Crew integration & Automation Testbed (CAT) and the Robotic Follower (RF), for the development, integration and demonstration of technologies for reduced crew operations and unmanned vehicle control. The Tank Automotive Research, Development and Engineering Center (TARDEC) of the U.S. Army Tank-automotive and Armaments Command (TACOM) is executing these programs together in an effort entitled Vetronics Technology Integration (VTI). VTI comprises a number of different commercial, and government participants working research issues for near and far term vehicular electronics and robotic challenges. 1.2 Context The use of robotics in the near term will be through soldier-robot teams. A great deal of research is being conducted into developing semi-autonomous robotic systems and our research and development efforts are focused on defining new interfaces to empower the warfighter to employ these autonomous systems while conducting their existing mission (Dahn and Gacy, 2002).
    [Show full text]
  • Teleoperation with Significant Dynamics
    Teleoperation with significant dynamics MATTIAS BRATT Licentiate Thesis Stockholm, Sweden 2009 TRITA-CSC-A 2009:16 ISSN-1653-5723 KTH School of Computer Science and Communication ISRN-KTH/CSC/A–09/16-SE SE-100 44 Stockholm ISBN 978-91-7415-486-3 SWEDEN Akademisk avhandling som med tillstånd av Kungl Tekniska högskolan framlägges till offentlig granskning för avläggande av teknologie licentiatexamen i datalogi fredagen den 27 november 2009 klockan 13.00 i hörsal D3, Lindstedtsvägen 5, Kungl Tekniska högskolan, Stockholm. © Mattias Bratt, november 2009 Tryck: Universitetsservice US AB iii Abstract The subject of this thesis is teleoperation, and especially teleoperation with demanding time constraints due to significant dynamics inherent in the task. A comprehensive background is given, describing many aspects of tele- operation, from history and applications to operator interface hardware and relevant control theory concepts. Then follows a presentation of the research done by the author. Two prototypical highly dynamic teleoperation tasks have been attempted: high speed driving, and ball catching. Systems have been developed for both, employing operator interfaces tailored to facilitate perception of the remote scene and including assistive features to promote successful task completion within the required time frame. Prediction of the state at the remote site as well as of operator action has been applied to address the problem of delays arising when using the Internet as the communication channel. v Sammanfattning Detta arbete handlar om teleoperation, som skulle kunna översättas med fjärrstyrning, och speciellt sådan som ställer stränga tidskrav på grund av att uppgiftens natur inbegriper en avsevärd dynamik. Först ges en bred bakgrund där många aspekter av teleoperation belyses, från dess historia och använd- ningsområden till hårdvara för användargränssnitt och relevant reglerteori.
    [Show full text]
  • Vehicle Teleoperation Interfaces
    Autonomous Robots 11, 9–18, 2001 c 2001 Kluwer Academic Publishers. Manufactured in The Netherlands. Vehicle Teleoperation Interfaces TERRENCE FONG The Robotics Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213, USA; Institut de Systemes` Robotiques, Ecole Polytechnique Fed´ erale´ de Lausanne, CH-1015 Lausanne, Switzerland CHARLES THORPE The Robotics Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213, USA Abstract. Despite advances in autonomy, there will always be a need for human involvement in vehicle tele- operation. In particular, tasks such as exploration, reconnaissance and surveillance will continue to require human supervision, if not guidance and direct control. Thus, it is critical that the operator interface be as efficient and as capable as possible. In this paper, we provide an overview of vehicle teleoperation and present a summary of interfaces currently in use. Keywords: human robot interaction, mobile robots, remote driving, vehicle teleoperation, ROV, RPV, UAV, UGV 1. Introduction sensor data. Because vehicles may cover large dis- tances, map building and registration are significant 1.1. Vehicle Teleoperation issues. Vehicle teleoperation means simply: operating a vehi- cle at a distance (Fig. 1). It is used for difficult to reach 1.2. Interfaces environments, to reduce mission cost, and to avoid loss of life. Although some restrict the term teleoperation In order for vehicle teleoperation to perform well, the to denote only direct control (i.e., no autonomy), we human-robot interface must be as efficient and as capa- consider teleoperation to encompass the broader spec- ble as possible. All interfaces provide tools to perceive trum from manual to supervisory control. Furthermore, the remote environment, to make decisions, and to gen- the type of control can vary and may be shared/traded erate commands.
    [Show full text]
  • Teleoperation Interfaces in Human-Robot Teams Band 1
    Institut für Informatik Lehrstuhl für Robotik und Telematik Prof. Dr. K. Schilling Würzburger Forschungsberichte in Robotik und Telematik Uni Wuerzburg Research Notes in Robotics and Telematics Frauke Driewer Teleoperation Interfaces in Human-Robot Teams Band 1 Teleoperation Interfaces in Human-Robot Teams Dissertation zur Erlangung des naturwissenschaftlichen Doktorgrades der Julius-Maximilians-Universit¨atW¨urzburg vorgelegt von Frauke Driewer aus Ettlingen W¨urzburg2008 Eingereicht am: 01.10.2008 bei der Fakult¨atf¨urMathematik und Informatik 1. Gutachter: Prof. Dr. Klaus Schilling 2. Gutachter: Prof. Dr. Aarne Halme Tag der m¨undlichen Pr¨ufung: 08.05.2009 Danksagung An erster Stelle gilt mein allerherzlichster Dank meinem Doktorvater Prof. Dr. Klaus Schilling f¨urdie M¨oglichkeit an diesem spannenden Thema zu arbeiten und f¨ur seine Unterst¨utzung bei der Bearbeitung. In meiner Zeit an seinem Lehrstuhl habe ich viel von ihm lernen k¨onnen und ich bin besonders dankbar f¨ur das Ver- trauen, das er in den zur¨uckliegenden Jahren immer in mich gesetzt hat. Weiterhin m¨ochte ich Prof. Dr. Aarne Halme herzlich f¨urdie Erstellung des Zweitgutachtens danken, sowie den Mitgliedern der Pr¨ufungskommission Prof. Dr. Rainer Kolla und Prof. Dr. J¨urgen Albert. Am Lehrstuhl f¨ur Robotik und Telematik des Instituts f¨ur Informatik an der Universit¨atW¨urzburg hatte ich die Chance in einem aussergew¨ohnlichen Team mitzuarbeiten, in dem die Arbeit jeden Tag Freude gemacht hat. Danke daf¨urbesonders an Markus Sauer, Florian Zeiger, Dr. Lei Ma, Rajesh Shankar Priya, Marco Schmidt, Daniel Eck, Christian Herrmann, Martin Hess, Dr. Peter Hokayem, Dieter Ziegler, Annette Zillenbiller, Edith Reuter und Heidi Schaber.
    [Show full text]
  • From Teleoperation to Autonomy: “Autonomizing” Non-Autonomous Robots
    From Teleoperation to Autonomy: “Autonomizing” Non-Autonomous Robots B. Kievit-Kylar1, P. Schermerhorn1, and M. Scheutz3 1Cognitive Science Program, Indiana University, Bloomington, IN 47404, USA 2Department of Computer Science, Tufts University, Medford, MA 02155, USA Abstract— Many complex tasks (search and rescue, explo- close to the robotic platform may put the operator in danger, sive ordinance disposal, and more) that eventually will be eliminating the primary advantage of using a robot. Even in performed by autonomous robots are still performed by the absence of such obstacles, the need for constant human human operators via teleoperation. Since the requirements attention adds expense and limits the overall effectiveness of of teleoperation are different from those of autonomous oper- the robot. The ability to operate these robots autonomously ation, design decisions of teleoperation platforms can make would avoid many of these limitations, but researchers need it difficult to convert such platforms for autonomous use. access to the robots to continue to improve the performance In this paper, we discuss the differences and the potential of autonomous robot architectures to the point where the difficulties involved in this conversion, as well as strategies human operator is not needed. for overcoming them. And we demonstrate these strategies by The common response to the above issues has been “autonomizing” a fully teleoperated robot designed for tasks the design of “hybrid autonomous/teleoperated” systems such as bomb disposal, using an autonomous architecture that combine the strengths of both approaches. Mixed- that has the requisite capabilities. initiative architectures are suitable as extraplanetary rover control systems, where prohibitive communication delays Keywords: tele-operation, autonomy, conversion necessitate some degree of autonomy, while the higher- level decision-making remains in the hands of a team of 1.
    [Show full text]
  • Shared Control for Vehicle Teleoperation with a Virtual Environment Interface
    DEGREE PROJECT IN THE FIELD OF TECHNOLOGY ENGINEERING PHYSICS AND THE MAIN FIELD OF STUDY ELECTRICAL ENGINEERING, SECOND CYCLE, 30 CREDITS STOCKHOLM, SWEDEN 2020 Shared Control for Vehicle Teleoperation with a Virtual Environment Interface ANTON BJÖRNBERG KTH ROYAL INSTITUTE OF TECHNOLOGY SCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE Shared Control for Vehicle Teleoperation with a Virtual Environment Interface Anton Bj¨ornberg Master in Systems, Control and Robotics June 2020 Supervisor: Frank Jiang Examiner: Prof. Jonas M˚artensson KTH Royal Institute of Technology School of Electrical Engineering and Computer Science Abstract Teleoperation has been suggested as a means of overcoming the high uncertainty in auto- mated driving by letting a remote human driver assume control of an automated vehicle when it is unable to proceed on its own. Remote driving, however, suffers from restricted situational awareness and network latency, which can increase the risk for accidents. In most cases though, the automation system will still be functional, which opens the possibility for shared control where the human and machine collaborates to accomplish the task at hand. This thesis develops one such method in which an operator demonstrates their intentions by maneuvering a virtual vehicle in a virtual environment. The real vehicle observes the virtual counterpart and acts autonomously to mimic it. For this, the vehicle uses a genetic algorithm to find a path that is similar to that of the virtual vehicle, while avoiding poten- tial obstacles. Longitudinal control is performed with a method inspired by Adaptive Cruise Control (ACC) and lateral control is achieved with a model predictive controller (MPC).
    [Show full text]
  • Collaborative Teleoperation Via the Internet *
    Collaborative Teleoperation via the Internet Ken Goldberg, Billy Chen, Rory Solomon, Steve Bui, Bobak Farzin, Jacob Heitler, Derek Poon, Gordon Smith Alpha Lab, IEOR Department, UC Berkeley We describe a system that allows a distributed group is achieved using two-sided color-coded paddles and a com- of users to simultaneously teleoperate an industrial robot puter vision system that aggregates the color field resulting arm via the Internet. A java applet at each client streams from all paddle positions. The developers report that groups mouse motion vectors from up to 30 users; a server aggre- of untrained users are able to coordinate aggregate motion, gates these inputs to produce a single control stream for the for example to move a cursor through a complex maze on robot. Users receive visual feedback from a digital camera the screen. mounted above the robot arm. To our knowledge, this is the first collaboratively controlled robot on the Internet. To test it please visit: http://ouija.berkeley.edu/ 1 Introduction In most teleoperation systems, a single user input controls a single robot. Several researchers have considered the prob- lem of one user controlling multiple robots; for a review of research in cooperative teleoperation see [1]. Here we con- sider the inverse problem: combining multiple user inputs to control a single industrial robot arm. We proposed the term “collaborative teleoperation” to describe such a many- one control architecture. Consider the following scenario in space or undersea: a group of users are working together to control a telerobot. Each user monitors a different sensor and submits control inputs appropriate to that sensor information.
    [Show full text]
  • Medical Telerobotic Systems: Current Status and Future Trends
    Avgousti et al. BioMed Eng OnLine (2016) 15:96 DOI 10.1186/s12938-016-0217-7 BioMedical Engineering OnLine REVIEW Open Access Medical telerobotic systems: current status and future trends Sotiris Avgousti1* , Eftychios G. Christoforou2, Andreas S. Panayides3,6, Sotos Voskarides4, Cyril Novales5, Laurence Nouaille5, Constantinos S. Pattichis6 and Pierre Vieyres5 *Correspondence: [email protected] Abstract 1 Nursing Department, Teleoperated medical robotic systems allow procedures such as surgeries, treatments, School of Health and Science, Cyprus and diagnoses to be conducted across short or long distances while utilizing wired University of Technology, 30 and/or wireless communication networks. This study presents a systematic review of Archbishop Kyprianou Street, the relevant literature between the years 2004 and 2015, focusing on medical teleop- 3036 Limassol, Cyprus Full list of author information erated robotic systems which have witnessed tremendous growth over the exam- is available at the end of the ined period. A thorough insight of telerobotics systems discussing design concepts, article enabling technologies (namely robotic manipulation, telecommunications, and vision systems), and potential applications in clinical practice is provided, while existing limi- tations and future trends are also highlighted. A representative paradigm of the short- distance case is the da Vinci Surgical System which is described in order to highlight relevant issues. The long-distance telerobotics concept is exemplified through a case study on diagnostic ultrasound scanning. Moreover, the present review provides a clas- sification into short- and long-distance telerobotic systems, depending on the distance from which they are operated. Telerobotic systems are further categorized with respect to their application field.
    [Show full text]