forum Spotlight On: The Concept of Telepresence Telepresence

Although the term "telepresence" is often used in discus- tory, and tactile in the remote environ- sions of teleoperation, it never has been adequately de- ment and display the non-harmful, task-relevant com- fined. According to Akin, Minsky, Thiel, and Kurtzman ponents of this information to an operator in a way (1983), telepresence occurs when the following condi- that very closely replicates the pattern of stimulation tions are satisfied: available to an on-site observer. Such a system would permit the operator to extend his sensory-motor func- At the worksite, the manipulators have the dexterity tions and problem solving skills to remote or hazard- to allow the operator to perform normal human func- ous sites as if he were actually there. tions. At the control station, the operator receives suf- ficient quantity and quality of sensory to pro- In addition to its value in a general purpose teleopera- vide a feeling of actual presence at the worksite. tor system, telepresence is likely to be useful in a variety of other applications. More specifically, it should en- A major limitation of this definition is that it is not hance performance in applications (i.e., virtual environ- or It does not sufficiently operational quantitative. spec- ments) where the operator interacts with synthetic how to measure the of Also as ify degree telepresence. worlds created by . The most obvi- indicated the normal human by phrase "perform ous cases in this category are those associated with train- functions" in the first it fails to address the is- sentence, ing people to perform certain motor functions (e.g., fly- sue of for that are to trans- telepresence systems designed ing an airplane) or with entertaining people (i.e., form as well as transport and to perform abnormal hu- providing imaginary worlds for fun). Less obvious, but man functions. equally important, are cases in which the system is used Independent of the precise definition of telepresence, as a research tool to study human sensorimotor perfor- why should one care about telepresence? What good is mance and cases in which it is used as an interactive dis- it? Certainly, there is no theorem that states that an in- play for data presentation (e.g., Fisher, 1987; Bolt, crease in telepresence necessarily leads to improved per- 1984). formance. In our opinion, a high degree of telepresence An important obstacle at present to scientific use of is desirable in a teleoperator system primarily in situa- the telepresence concept is the lack of a well-defined tions in which the tasks are wide ranging, complex, and means for measuring telepresence. It should not only be uncertain (i.e., when the system must function as agen- possible to develop subjective scales of telepresence (us- eral-purpose system). In such situations, a high degree of ing standardized scale-construction techniques), but also telepresence is desirable because the best general purpose to create tests, both psychological and physiological, to system known to us (as engineers) is us (as operators). measure telepresence objectively. For example, some test In a passage that is relevant both to this issue and to the based on the "startle response" might prove useful. (Will definition of telepresence, Pepper and Hightower the human operator flinch when a swinging baseball bat (1984) state: is aimed at the eyes of the telerobot?) Certainly, such a We feel that anthropomorphically-designed teleopera- tors offer the best means of transmitting man's re- markably adaptive problem solving and manipulative Richard M. Held and skills into the ocean's and other Nathaniel I. Durlach depths inhospitable of Brain and Science and environments. The Department Cognitive anthropomorphic motorapproach Department of Electrical and Computer Science calls for development of teleoperator subsystems Massachusetts Institute of which sense highly detailed patterns of visual, audi- Cambridge, Massachusetts 02139

Held and Durlach 109

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/pres.1992.1.1.109 by guest on 03 October 2021 MO PRESENCE: VOLUME I , NUMBER I

test could distinguish reliably between different degrees way that the operator is aware of the mounting via the of realism in the area of cinematic projection. Also, of sense of touch. Clearly, attempting to satisfy both of course, it would be important to study the relations be- these constraints simultaneously is a very challenging tween the subjective and objective measures. task. Beyond questions related to the definition and mea- Motor factors necessary for high telepresence involve surement of telepresence, the core issue is how one similar issues. Perhaps the most crucial requirement is to achieves telepresence. In other words, what are the fac- provide for a wide range of sensorimotor interactions. tors that contribute to a sense of telepresence? In fact, One important category of such interactions concerns what are the essential elements ofjust plain "presence"? movements of the sensory organs. It must be possible Or, how can the ordinary sense of presence be de- for the operator to sweep the direction of gaze by rotat- stroyed? ing the head and/or eyeballs and to have the visual input Given the vague and qualitative character of defini- to the retinas change appropriately. This requires using a tions and estimates of telepresence, it is not surprising with a rotating head, the position of which is that there is no scientific body of data and/or theory de- slaved to the operator's head. The desired result can then lineating the factors that underlie the phenomenon. Our be achieved in two ways, depending on whether the sys- remarks on this topic thus make substantial use of intu- tem is designed to have the position of the robot's eye- ition and speculation, as well as extrapolation from re- balls (1) fixed relative to the robot's head (e.g., pointing sults in other areas. straight ahead) or (2) slaved to the position of the opera- Sensory factors that must certainly contribute to tele- tor's eyeballs in the operator's head. In the first case, ap- presence include high resolution and large field of view. propriate results can be obtained using binocular images Obviously, reduction of input information either by de- that remain fixed relative to the operator's head position graded resolution or restricted field of view will interfere during eyeball scanning. In the second case, the posi- with the extent to which the display system is transpar- tions of the projected images must be slaved to the posi- ent to the operator. Perhaps these two variables are trad- tion of the operator's eyeballs. If they were instead held able in the sense that the effective parameter in deter- fixed, then whenever the operator's eyeballs were ro- mining the degree of telepresence is the number of tated, the projected images would rotate. For example, if resolvable elements in the field, or, equivalently, for the operator's eyeballs were rotated to look at an object fields with uniform resolution over the field, Area of whose images were on the right side of the projection Field/Area of Resolvable Element. Also important, of screens, the slave robot's eyeballs would rotate to the course, is the consistency of information across modali- right, the images of the object in question would move ties: the information received through all channels to the centers of the two screens, and these images should describe the same objective world (i.e., should be would then be sensed to the left of the foveal region. To consistent with what has been learned through these eliminate this problem, the projected images would also channels about the normal world during the normal de- have to be rotated to the right. In other words, if the velopment process). In addition, the devices used for position of the robot eyeballs is slaved, the position of displaying the information to the operator's senses in the the projected images must also be slaved. To the best of teleoperator station should, to the extent possible, be our knowledge, no such system has yet been developed free from the production of artifactual stimuli that signal (although monitoring of operator eyeball position is the existence of the display. Thus, for example, the visual being used to capitalize on reduced resolution require- display should be sufficiently large and close to the eyes ments in the peripheral field in the pursuit of reduced to prevent the operator from seeing the edges of the dis- ). play (or anything else in the teleoperator station, includ- Another category of sensorimotor interactions that is ing the operator's own hands and body). At the same essential for high telepresence concerns movements of time, the display should not be head mounted in such a viewed effectors. It must be possible for the operator to

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/pres.1992.1.1.109 by guest on 03 October 2021 Held and Durlach 111

move his/her hands (receiving the internal kinesthetic sensations associated with these movements) and simul- taneously see the slave robot hands move accordingly. Also, as with the sensory display, the devices used in the teleoperator station to detect and monitor the operator's movements should, to the extent possible, be undetect- able to the operator. The more the operator is aware of these devices, the harder it will be to achieve a high de- gree of telepresence. An amusing picture that is ad- dressed to the issue of viewing one's own effectors, or more generally, one's own body parts, and that is of his- torical interest, is shown in Figure 1 (Mach, 1914). The most crucial factor in creating high telepresence is, perhaps, high correlation between (1) the movements of the operator sensed directly via the internal kines- thetic senses of the operator and (2) the actions of the slave robot sensed via the sensors on the slave robot and the displays in the teleoperator station. Clearly, the de- struction of such correlation in the normal human situa- tion (in which the slave robot is identified with the oper- ator's own body) would destroy the sense of presence. In general, correlation will be reduced by time delays, noises, or noninvertable distortions internally generated Figure I. Mach observing visible parts of his own body and the that occur between the actions of the and the operator surroundings (Mach, 1914). sensed actions of the slave robot. How these variables interact, combine, and trade in limiting telepresence and teleoperator performance is a crucial topic for research. tent to which such limitations can be overcome by ap- Note also that telepresence will generally tend to in- propriate exposure to the system and development of crease with an increase in the extent to which the opera- appropriate models of the transformed world, task, self, tor can identify his or her own body with the slave ro- etc. (though adaptation, training, learning, etc.). Figure bot. Many of the factors mentioned above (in particular, 2 illustrates schematically how the internal dynamics of the correlation between movements of the body and the operator are originally established and may be al- movements of the robot) obviously play a major role in tered over time when interaction with the world is trans- such identification. Additional factors, however, may formed. The cognitive representation of the operator's also be important. For example, it seems plausible that interaction with the world is an important factor in the identification, and therefore telepresence, would be in- sense of presence. The operator identifies his or her own creased by a similarity in the visual appearance of the actions as such in accord with the concomitant sensory operator and the slave robot. changes. Loss of such concomitance may reduce the Finally, it is important to consider the extent to which sense of presence. But an updating of the internal model telepresence can increase with operator familiarization. may promote the recovery of a lost sense of presence Even if the system is designed merely to transport rather within that world. The figure shows how the motor than to transform, it will necessarily involve a variety of command originating in the central nervous system transformations that initially limit the sense of telepres- (CNS) activates the musculature, which in turn causes ence. A fundamental topic for research concerns the ex- sensory changes that feed back to the CNS. The compar-

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/pres.1992.1.1.109 by guest on 03 October 2021 112 PRESENCE: VOLUME I, NUMBER I

Although we recognize the phenomenon of telepres- ence, we neither understand it nor are clear about its COMPARATOR usefulness in teleoperation. A number of related issues and possible variables have been mentioned in the above text. What is now needed is a systematic research effort designed to gain an understanding of the sensorimotor MODEL OF WORLD, SELF. TASK, and factors that determine the sense of INTERACTIONS. ETC cognitive pres- ence.

MOTOR COMMAND References AND CONTROL

Akin, D. L., Minsky, M. L., Thiel, E. D., & Kurtzman, C. R. (1983). Space applications of automation, robotics, and ma- in Figure 2. Information flow and feedback loops involved actions of chine intelligence systems, (ARAMIS) phase II, Vol. 3: Ex- the operator in the environment. For simplicity, only the visual and ecutive Summary, M.I.T., Contract NASA 8-34381, NASA manual sensorimotor channels are Included. Environment is sensed Marshall Space Flight Center. visual sensors and manual sensors through (tactile, proprioceptive, Bolt, R. A. (1984). The human interface: Where people and com- ¡anesthetic). These inputs are influenced by motor actions that may puters meet. Belmont CA: Lifetime Learning Publications (a alter the external environment as well as control the sensors, that is, division of Wadsworth, London). the direction ofgaze and the position and posture of the hand. They Fisher, S. (1987). Telepresence in dataspace. Paper presented at are used to the internal model the world, task, etc. update of self, by Symposium and Workshop on Spatial Displays and Spatial means of a comparator and the updated model then directs Instruments, Asilomar, CA. subsequent motor actions and, perhaps, modifies the comparator. Mach, Ernst (1914). The analysis ofsensations. Chicago: Open Court. Pepper, R. L., & Hightower, J. D. (1984). Research issues in ator is designed to receive a feedforward signal from the teleoperator systems. Paper presented at the 28th Annual Hu- internal model, which derives from past experience and man Factors Society Meeting, San Antonio, TX. anticipates the consequences of activity based on that previous experience. That signal is then compared with the contemporary consequences of action. Any trans- Note form in the feedback loop will alter the expected feed- back and be discrepant with the feedforward signal. In This article was reproduced (in modified form) with per- that event, the discrepant signal may be used to update mission from Pictorial Communication in Real and Vir- the world model and lead to more accurate anticipations tual Environments, edited by S. R. Ellis et al., 1991, Tay- of action and improved sense of presence. lor & Francis Ltd., London.

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/pres.1992.1.1.109 by guest on 03 October 2021