VR-Based Teleoperation of a Mobile Robotic Assistant: Progress Report
Total Page:16
File Type:pdf, Size:1020Kb
DEMO 2000/13 VR-based Teleoperation of a Mobile Robotic Assistant: Progress Report Costas S. Tzafestas, Dimitris Valatsos ¤ Technical Report DEMO 2000/13 Institute of Informatics and Telecommunications National Center for Scienti¯c Research \Demokritos", 15310 Aghia Paraskevi Athens, Greece E-mail: [email protected] Tel: +301-6503153, Fax: +301-6532175 November 2000 Abstract This report presents work in progress within the framework of a research project aim- ing at the development of a mobile robotic system to perform assistive tasks in a hospital environment. This robotic assistant will consist of a mobile robot platform equipped with a variety of on-board sensing and processing equipment as well as a small manipulator for performing simple fetch-and-carry operations. In this report, we focus on the design of the teleoperation system integrating virtual reality techniques and Web-based capabilities in the human operator interface. Relative work found in the literature in the ¯eld of intervention and service telerobotics is reviewed, and an overview of the methodologies that will be fol- lowed is presented. Some speci¯c issues requiring particular attention for the design of a teleoperation system for the mobile robotic assistant are investigated and include: (a) the speci¯cation of the teleoperation modes supported by the system, integrating various au- tomatic computer assistance and shared-autonomy behaviour-based control modes, (b) the design of the user interface, built on Java technology to enable web-operation and support various multimodal VR-based functionalities, and (c) the integration with the other sub- systems and control modules of the mobile robotic assistant, in the framework of a general teleoperation/telemanipulation control architecture. Keywords: Telerobotics, virtual reality, human/machine interfaces, mobile service robots. ¤This research work was supported by the General Secretariat for Research and Technology (Greek Ministry of Development) and the European Community, under grant PENE¢-99-E¢623 C. S. Tzafestas & D. Valatsos - DEMO 2000/13 2 1 Introduction Virtual Reality (VR) and its applications in the general ¯eld of Telerobotics (VR and Teler- obotics - VRT) are research areas that have known a great development during the last decade. Virtual Reality [5] constitutes a multidisciplinary scienti¯c/technological ¯eld, which comprises research and development on areas such as computer graphics and animation, physical mod- eling, mechatronic design and control, as well as human sensori-motor modeling, perception and psychophysics. VR technology, in fact, aims at enabling a more natural and intuitive human/computer interaction, based on the use of multimodal/multi-sensory interfaces. This human/machine interface technology implicating various perceptuo-motor modalities of the hu- man being (not only vision but also direct hand action and manipulation, haptic perception as well as auditive feedback) is recognized as the most promising solution for a number of problems related to telerobot control. All these approaches involving the integration of VR techniques in telerobotics constitute in fact: (a) a generalization of the concept of predictive displays, cop- ing with the problem of time delay and stability in teleoperation systems and (b) an attempt to provide human operator assistance and achieve better transparency characteristics for the teleoperation system. Application of VR techniques in Telerobotics and the related concept of telepresence (or tele- symbiosis [44]) are ideas which have been around for more than twenty years, and have been used mainly for the telemanipulation/teleoperation of robotic mechanisms in hostile environ- ments (such as in the nuclear industry, space, underwater, or for other di±cult or dangerous service/intervention tasks, like bomb disposal, civil works etc.). Nowadays, on the other hand, the rapid development of new broadly expanded networking technologies, such as those related to the Internet, and the numerous relevant applications, described by the general term of e- commerce/e-business, can give new potential to the use of VRT in novel application domains. In fact VRT and Internet technologies can mutually bene¯t from ideas developed in the respec- tive ¯elds. This merging of technological potential can lead to a generalization of the concept of telework, where remote control through the network of actual physical processes will be possible. One can even think, for instance, of supervising and actively controlling a whole manufacturing process without having to move from his home. A major research objective must be of course to enable and promote new potential applications that can derive from the merging of such tech- nologies, so that wider categories of the population can ¯nally take bene¯t of these technological advances. This report presents work in progress, which focuses on the application of these technologies for the teleoperation control of a mobile robot. The work is part of a research project which aims at developing a mobile robotic system for performing assistive tasks in a hospital environment. This robotic assistant will consist of a mobile platform equipped with a variety of on-board sensing and processing equipment as well as a small manipulator for performing simple fetch-and- carry operations. In this report we focus on the design of the teleoperation system integrating virtual reality techniques and Web-based capabilities in the human operator interface. This report begins with a literature survey presenting the state-of-the-art in the general ¯eld of telerobotics. Section 2 briefly traces the history of teleoperation systems, the integration of VR technology and the development of a new ¯eld called intervention and service robotics. Section 3 presents the general architecture of the mobile robotic assistant that is under development. It briefly describes the main system components and control modules and illustrates some basic interoperability issues. The rest of the report focuses on the design of the teleoperation system for the mobile robotic assistant. Issues related to the modes of teleoperation, as well as to the design and implementation of the human operator interface, are discussed. An initial version of the developed user interface for remote monitoring and control of the mobile robotic assistant, C. S. Tzafestas & D. Valatsos - DEMO 2000/13 3 is presented in section 4.1. This human/machine interface is implemented using Java/Java3D, and is designed to support various modes of operation, integrating VR techniques and enabling web-based operation. Section 4.2 discusses some future work directions and research objectives. The hardware con¯guration of the mobile robotic assistant is then desribed in section 5. The mobile robotic platform will be equipped with a vision system, a ring of ultrasonic sensors for local navigation, a wireless Ethernet link for communication and a small integrated manipulator to enable the execution of simple fetch-and-carry tasks. Some concluding remarks for the work exposed in this report, are ¯nally presented in section 6. 2 Literature Survey In this section we present a brief survey and a current state-of-the-art of research and develop- ment carried out in the general ¯eld of telerobotics. 2.1 Teleoperation and Virtual Reality Telemanipulation as a scienti¯c term describes all the methodologies and techniques enabling a human operator to perform from a distance a manipulative task, using his own hand through the use of an intermediate mechatronic system. Telemanipulation control of a remote manipulative task, besides its fascinating character related to the notion of extending human capabilities by some tool beyond usual space or time limits, it can prove extremely bene¯cial in cases where human intervention is indispensable to perform a task taking place in an unstructured \hostile" environment, due to the increased uncertainty and non-repetitiveness characteristics of such tasks, and the complex task/path planning required for timely and correct execution. Original master-slave telemanipulation systems consisted of a couple of mechanical or electromechani- cal arms (one called the master, controlled by the human operator, and the other, called the slave, performing the remote manipulation task). Bilateral exchange of energy (position and force signals) was initially ensured through a mechanical linkage and, later-on, through the use of electrical links and servo-control loops. In its infancy, telemanipulation technology found outstanding applications in the nuclear industry for the remote manipulation of radioactive ma- terials in environments where human presence was hazardous. Typical example is the work accomplished by Raymond Goertz at Argonne National Laboratories, USA, or by Jean Vertut and the French group at the CEA [44]. Bilateral servo-controlled telemanipulation and industrial computer-controlled robotics were two technological ¯elds developed originally in parallel and, in some extent, independently. The awareness that both these ¯elds can bene¯t from development accomplished in each other has led to the fusion of these technologies and the creation of what is generally described under the term of telerobotics. Robotics was initially concerned with the development of industrial man- ufacturing systems performing programmable, repetitive operations in an autonomous sensor- based manner, while telemanipulation was focusing on a di®erent class of tasks, which should clearly rely on