Collaborative Control: a Robot-Centric Model for Vehicle Teleoperation
Total Page:16
File Type:pdf, Size:1020Kb
Collaborative Control: A Robot-Centric Model for Vehicle Teleoperation Terrence Fong CMU-RI-TR-01-34 The Robotics Institute Carnegie Mellon University 5000 Forbes Avenue Pittsburgh, Pennsylvania 15213 November 2001 Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy Thesis Committee Charles Thorpe (chair) Charles Baur Eric Krotkov Chris Atkeson © 2001 by Terrence Fong. All rights reserved. This research was partially funded by grants from the DARPA ITO “Mobile Autonomous Robot Software” and TTO “Tactical Mobile Robots” (NASA Jet Propulsion Laboratory 1200008) programs, the National Science Foundation (Dissertation Enhancement Award 1120030), and SAIC. The Institut de Systèmes Robotiques (DMT-ISR) of the Ecole Polytechnique Fédérale de Lausanne (EPFL) served as host institution (institution d’accueil) for this thesis and provided research facilities and infrastructure. Abstract Telerobotic systems have traditionally been designed and solely operated from a human point of view. Though this approach suffices for some domains, it is sub-optimal for tasks such as operating multiple vehicles or controlling planetary rovers. Thus, I believe it is worthwhile to examine a new system model for teleoperation: collaborative control. In collaborative control, a human and a robot collaborate to perform tasks and to achieve common goals. Instead of a supervisor dictating to a subordinate, the human and the robot engage in dialogue to exchange information, to ask questions, and to resolve differences. Instead of serving the human as a mere tool, the robot can operate more like a partner. With this approach, the robot has more freedom in execution and is more likely to find good solutions when there are problems. With collaborative control, the human is able to function as a resource for the robot, providing information and processing just like other system modules. In particular, the robot can ask the human questions as it works, to obtain assistance with cognition and perception during task execution. This enables the human to compensate for inadequacies of autonomy, but does not require time-critical nor situation-critical response. Collaborative control is both a novel and a useful paradigm for teleoperation. Collaborative control is novel because it uses dialogue as a framework for coordination, to direct joint task performance and to focus attention where it is needed. Collaborative control is useful because it provides an efficient mechanism for adaptation, to adjust autonomy and human-robot interaction to fit situational needs and user capabilities. This thesis investigates the numerous human-robot interaction and system design issues raised by collaborative control, describes the implementation of a dialogue-based user interface and control architecture, and examines the use of this new approach in several vehicle teleoperation applications and a user study. Keywords: collaborative control, human-robot collaboration, human-robot communication, human-robot interaction, man-machine sysems, mobile robots, PDA interface, remote driving, robot control architecture, safeguarded teleoperation, vehicle teleoperation. iii For Mom and Dad v Acknowledgements When I started the Robotics Ph.D. program in August 1994, I had no idea the road to this thesis would be so long and winding. In fact, when Jessica and I arrived in Pittsburgh, we both thought, “We’ll be back in California before the year 2000.” Even two years later, I was sure that I would finish my research before Navlab II was retired. But then one day, Jessica said, “I think I want to join the High-energy (Physics) group. Do you want to move to Switzerland?” So, we packed our bags and moved across the Atlantic. A few months later, after having settled in at EPFL, I caught myself saying, “It’s really not so hard to understand. I’m a CMU student, but I’m doing my research in Switzerland. I have two advisors named Charles and I’m being paid by a start-up company in California.” At that point, I realized that this was probably not the usual (or easy) route to a Ph.D. But having now reached the end, I must say that the journey has been memorable and rich, and I am truly glad for hav- ing traveled it. There are many people to whom I would like to express my appreciation and gratitude. First and foremost, I would like to thank Dr. Charles (Chuck) Thorpe and Dr. Charles Baur for their guidance, encouragement, patience, and wisdom during the past few years. It has been a great privilege to work with such far thinking and inspirational individuals. I am particularly indebted to Chuck for allowing me to follow such an unconventional path and to Charles for his unwavering support along the way. All that I have learned and all that I have accomplished during the course of this thesis would not have been possible without their dedication. To the community that is the Robotics Institute, my thanks go to: Marie Elm, Jim “Fraz” Frazier, Ruth Gaus, Martial Hebert, Suzanne Lyons Muth, Dorothy “Dot” Marsh, Matt Mason, Stephanie Riso, Sandy Rocco, Red Whittaker, and Marce Zaragoza. To the Navlab Group (Al, Barry, Dirk, Jennie, Julio, Kate, Jim), thanks for making the Slag Heap “enjoy- able”. To Robograds ‘94 (Andy, Daniel, Dong Mei, Henry, Lisa, Murali, Patrick, Sundar, Zack), thanks for making Pittsburgh fun. To my fellow Planeteers (Butler, Erik, Laurent, Mike, and Phil), thanks for being so flexi- ble and supportive even when I was on the other side of the world. Fourth Planet was a great experience, a joy, a family. I will forever remember our telecons, meetings, and lunches. I hope that we will soon have the opportunity to all work together again (and that I will once more have the pleasure of amusing you all with my choice of “breakfast”). vii Un grand merci à tous ceux à l’EPFL qui m’ont aidé. Je suis particulièrement reconnai- sant: au Docteur Reymond Clavel qui m’a accueilli au sein de l’ISR; à Mme Heidi Banz, à Mlle Karine Genoud, à Mme Marie-Jose Pellaud et à Mme Katrin Schleiss pour leur assis- tance avec tous les taches administratives grandes et petites; et à Mme Nadia Nibbio (CFRC) qui a gracieusement financé quelques voyages aux Etats-Unis. Je remercie tous les étudiants qui ont travaillé avec moi (Gilbert Bouzeid, Mathias Deschler, Roger Meier, et Grégoire Terrien). Bon travail! Je remercie chaleureusement le Groupe VRAI (Didier, François, Gaëtan, Nicolas, Olivier, Patrice, Philippe et Sébastien) grâce à qui ce doctorat a put être réalisé dans une ambiance amicale et motivante. A Lorenzo et à Cécile, merci pour votre amitié, votre generosité, et pour tout le reste. I wish to thank my family for their love and for encouraging me to follow my dreams. To Tammy and Tim, thanks for always being there. To Mom and Dad, thanks for caring, for believing, and for supporting me all these years, no matter how far away I have been. To Karina, un très gros bisou de ton papa. Finally, to Jessica, thanks for your hugs, your kisses, your laughter, your smile and your love. You make it all worthwhile. Lausanne, Switzerland November 2001 viii Contents 1 Introduction 1 1.1 Robot as Tool. 1 1.2 Human as Controller . 2 1.3 Collaborative Control . 2 1.4 Why Collaborative Control? . 5 1.5 Related Research. .7 1.5.1 Supervisory Control . 7 1.5.2 Cooperative Teleoperation . 8 1.5.3 Human Robot Control Architectures . 9 1.5.4 Adjustable Autonomy and Mixed Initiative. 9 1.5.5 KTH Collaborative Human-Robot Systems. 10 1.6 Thesis Organization. 11 2 Developing Collaborative Control 13 2.1 Vehicle Teleoperation . 13 2.2 Conventional System Models . 14 2.2.1 Direct Control. 15 2.2.2 Supervisory Control . 16 2.2.3 Fully Autonomous Control . 17 2.2.4 Limitations . 18 2.3 Collaborative Control System Model . 19 2.4 Key Design Issues . 20 2.4.1 Dialogue . 20 2.4.2 Awareness. 20 2.4.3 Self-reliance . 21 2.4.4 Adaptiveness . 21 2.4.5 Human-Robot Interaction. 22 2.4.6 User Interface Design. 22 2.4.7 Control and Data Flow . 23 ix Contents 2.5 Evaluation. .23 2.6 Summary . .24 3 System Design 25 3.1 Design. .25 3.1.1 Principles. .25 3.1.2 Approach . .26 3.1.3 Architecture. .26 3.2 UserModeller . .29 3.2.1 Attributes. .29 3.2.2 Definition. .29 3.2.3 Use. .30 3.3 QueryManager . .31 3.4 EventLogger . .32 3.5 Human-Robot Dialogue . .33 3.6 Background . .36 3.6.1 Human-Robot Interaction . .36 3.6.2 Collaboration Systems. .37 3.6.3 Human-Computer Dialogue . .38 3.7 Summary . .40 4 PdaDriver: A User Interface for Collaborative Control 41 4.1 Requirements . .41 4.1.1 Collaborative Control Needs . .42 4.1.2 Vehicle Teleoperation Needs. ..