Toward High-Fidelity Telepresence in Space and Surgery Robotics

Detlef Reintsema Toward High-Fidelity [email protected] Telepresence in Space and Carsten Preusche Surgery Robotics Tobias Ortmaier Gerd Hirzinger German Aerospace Center (DLR) Institute of Robotics and Mechatronics 82230 Wessling, Germany Abstract High-fidelity telepresence is considered to be a key subject for the development of advanced space and surgery robotic systems. The emphasis of this paper are the key challenges like multimodal data servicing, bilateral and shared control concepts, and kinesthetic feedback devices. These technologies are the basic principles in the development of advanced space and surgery applications. Beside these technologies, advanced mechatronic systems are required as shown within this paper. The appli- cability of the high-fidelity telepresence concept is explored by selected space and surgery scenarios. 1 Introduction The realm of telerobotics is the manipulation of a remote physical environment. Today’s telerobotic systems are used in many situations to overcome several kinds of barriers that block the human operator from task fulfillment, such as distance, scale, material, or hazardous matter. For space missions, distance and inhuman and hazardous areas are perceived as the most characteris- tic barriers; whereas, for minimally invasive surgery (MIS), cramped spaces and restricted freedom of movement are more likely to be encountered. Therefore, the mechatronic manipulator used within a telerobotic system differs between the realm of space and MIS because of the constraints of the specific environment. But, as we will show, the basic telepresence-enabling system technology is quite the same, and independent of the realm. The telerobotic paradigm is typified by sensing the physical environment, measuring positions, forces, and accelerations, and responding with move- ments and forces to directly manipulate the physical environment (Conway, Volz, & Walker, 1987). Telepresence can be characterized as an advanced concept of telerobotics: the remote robot is directly operated by a human within a closed-loop control mode using a telerobotic system (the so-called teleoperator) to perform remote manipulations. Three kinds of telepresence definitions are commonly used (Draper, 1995): ● Simple telepresence summarizes the ability to function in a remote environment, such as controlling a remote machine. ● Cybernetic telepresence is defined by an index of the quality of a man- machine interface, such as examining the performance capabilities and lim- Presence, Vol. 13, No. 1, February 2004, 77–98 itations of the human operator. © 2004 by the Massachusetts Institute of Technology ● The experiential definition is defined by the degree of perception in which Reintsema et al. 77 Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/105474604774048243 by guest on 26 September 2021 78 PRESENCE: VOLUME 13, NUMBER 1 a user feels physically present within the remote the-art robotic systems are still not able to cope environment. For example, Sheridan (1992) pro- autonomously with these complex requirements. vides an ideal measure of telepresence quality in Telepresence as an extension to the supervisory or terms of immersiveness perceived by the operator: teleoperated control in the way of direct manipula- “With sufficiently good technology a person would tion of the robot’s behavior by the human operator not be able to discriminate among actual presence, is the only way to overcome the lack of sophisti- telepresence and virtual presence” (p. 6). cated autonomy. Thus, experiential telepresence systems enable a hu- A high-fidelity telepresence concept may overcome man operator to manipulate tasks in an inaccessible en- these drawbacks and barriers within the field of space vironment while feeling present at the event while phys- and surgery robotics. But telepresence requests high ically being at some other place (space shifting) or time demands to the technologies lying beneath the applica- (time shifting). For high-fidelity telepresence, the hu- tion. In particular the robot needs sensors compared man operator must feel as if he/she is present at a dis- with the human senses to gather the remote environ- tant location and interpret the mechatronic manipulator ment, which has to be displayed to the human operator. as a natural extension of his/her own body. This sug- The exploration and manipulation capabilities need to gests that the human operator receives input to (almost) be similar to the human capabilities, and the communi- all the human senses (vision, hearing, haptic, sense of cation has to be a broadband communication with low smell, and degustation) and commands the teleoperator delay to transport the sensorial input and the operator’s in a nearly natural way by demonstration. The last two reactions (commands) almost instantaneously, otherwise senses (smell and degustation) have no practical evi- the human’s feeling of being present at the distant loca- dence at present because of the lack of sensors and ac- tion is disturbed. tuators to measure and display them. This article focuses on the current telepresence activi- The fact that there are at present few robots in space ties of our institute in the field of space and surgical ro- and surgery is explained mainly by the lack of broadly botics. As we will show, both applications are essentially available sophisticated autonomy and tactile feedback driven by the same technology to overcome a combina- within current systems. tion of the abovementioned barriers. ● The technique of MIS was established in the 1980s. Surgeons use long instruments through small inci- 2 General Telepresence System sions. The advantages of MIS compared to open surgery are, among others, reduced pain and Within the scope of designing a general telepres- trauma, shorter hospital stays, and cosmetic advan- ence system, several classifications may be considered; tages. But the surgeon now loses direct access to each one examines a different aspect. We are interested the operation field, which in essence yields to re- in two major points of view. duced sight and tactile feedback due to the long instruments. Robotic surgery in combination with ● The basic human supervisory control paradigm as high-fidelity telepresence technologies like sen- defined by Sheridan (1992) separates computers by sorized scalpels and forceps may overcome the a barrier of distance, inconvenience, or time—one drawbacks and physical barriers, such as the pa- that loops directly to the human and one that loops tient’s chest. remotely to the human. ● The absence of robots in space is explained by the ● The service layer architecture is a set of service com- lack of sophisticated autonomy. Tasks in inhuman, ponents for setting up connections, connection unknown environments require a high level of plan- management and multisensory information, con- ning and reaction to unforeseen events. State-of- trol, and transmission between separate computers. Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/105474604774048243 by guest on 26 September 2021 Reintsema et al. 79 Figure 1. A Multimodal Telepresence and Teleaction System (TPTA) by Buss and Schmidt (1999). 2.1 Design by Control (OSI) reference model—is to enable partitioning of the problem domain into different areas of concern. Figure Based on the human supervisory control concept, 2 associates a layered abstraction for a TPTA system. Buss and Schmidt (1999) generalize a system structure Each layer offers services to the level above and provides for multimodal telepresence and teleaction (TPTA), its own services on the services of the layer below. A which is split up into the human operator and the tele- single layer may contain several entities providing the operator environment (see Figure 1). Toward telepres- same service. ence, multimodal control commands (messages) of the As long as the services are defined in terms of their human operator (e.g., voice or motion) have to be as- similated by adequate man-machine interfaces and functionality and not as a specific implementation (e.g., transmitted to the teleoperator, which interprets and program code), each layer’s services can be extended performs the command input. Thus, to provide human without destroying the overall structure. The layered senses with proprietary information, sophisticated MMIs approach also simplifies the protocol specification. In- are required, like kinesthetic displays in the case of the stead of a single all-embracing protocol, specific inde- human haptic sense. pendent protocols for each layer can be specified that One important aspect of multimodal supervisory con- are less complex than a single protocol. trol is the kinesthetic coupling of the human operator The network service layer provides transport network and the teleoperator based on the concept of bilateral communication and offers interfaces to common stan- control. Hence, sensory feedback of the teleoperator dardized protocol stacks (e.g., Internet Protocol (IP) raised during the interaction between the teleoperator Suite, Integrated Services Digital Network (ISDN), and its environment is needed as an input to the bilat- Asynchronous Transfer Mode (ATM), or Space Com- eral closed-loop control process and to the remote sen- munications Protocol Standards (SCPS)). Direct service sory feedback loop in the case of shared or autonomous access to the transport network requires the assembling control. (marshaling) and reassembling (unmarshaling) of appli- cation data into a form suitable for transmission

Load more