Vector Averaging for Collaborative Control of an Internet Robot ∗

Vector Averaging for Collaborative Control of an Internet Robot ∗ Ken Goldberg, Billy Chen, Rory Solomon IEOR and EECS Departments, UC Berkeley This paper describes a collaboratively controlled Internet robot, where a distributed group of users shares control of an industrial robot arm. A Java applet at each browser collects motion vectors from up to 30 simultaneous users. A central server averages these vectors to produce a single control stream for the robot. Users receive visual feedback from a digital camera mounted above the robot arm. The system was online for 18 months of continuous operation and used by over 25,000 remote users. We conclude with two experiments that quantify how vector averaging can improve performance in the presence of noise as suggested by the Central Limit Theorem. 1 Introduction In most teleoperation systems, a single user input controls a single robot. Several re- searchers have considered the problem of one user controlling multiple robots. We consider the inverse problem: combining multiple user inputs to control a single industrial robot arm. Consider the following scenario in space or undersea: a group of users are work- ing together to control a telerobot. Each user monitors a different sensor and submits control inputs appropriate to that sensor’s information. All of these inputs must be combined to produce a single control input for the telerobot. If the inputs can be put into vector form, one solution to this “control fusion” problem is to average all inputs. Since each scientist has access to a different noisy sensor, the Central Limit Theorem suggests that the mean may yield a more effective control signal than that from any individual input. We note that the CLT assumes independence and zero-mean noise, which may not be satisfied in practice. In the taxonomy proposed by Tanie et al. [9], a system with one user controlling one robot is a Single Operator Single Robot (SOSR) system. In a Multiple Operator Single Robot (MOSR) system, a group of users shares simultaneous access to a single robot. Inputs from all users must be combined to generate a single control stream for the robot. We refer to these as ”collaborative control” systems. There are benefits to ∗Contact: [email protected]. This work was supported in part by the National Science Foun- dation Award IIS-0113147. 1 collaboration: the group of users may be more reliable than a single user. In this paper we describe a MOSR teleoperation system that allows a group of online users to control the movements of a single industrial robot arm. We describe a client-server system on the Internet that facilitates many simultaneous users cooperating to share a single robot resource. As illustrated in Figure 1, each user/client downloads two Java applets by visiting the system website. These applets interact with two PC’s in our lab to control the Robot Arm and display video images. Client Applet V Applet C Internet Webserver V Video Card Webserver C Robot Control CCD Camera Robot Arm Figure 1: System Architecture for Collaborative Internet Robot: each user client runs two applets, Applet V for video feedback and Applet C for control. Since 1994, our lab has implemented five systems that allow users to interact remotely via the Internet to control a physical robot. In the Mercury Project [18], users queued up for 5 minute individual control sessions. In the 1995 Telegarden project [19], user motion requests were interleaved so that each user seemed to have simultaneous control of the robot. These two control models are analogous to the batch and multi- tasking approaches to mainframe computing. They suffer from the standard latency and throughput problems associated with time sharing. In this paper we report on experiments with a control model where motion commands from multiple simultaneous users are aggregated into a single stream of motion commands. 2 2 Related Work Goertz demonstrated one of the first “master-slave” teleoperators 50 years ago at the Argonne National Laboratory[15]. Remotely operated mechanisms have long been desired for use in inhospitable environments such as radiation sites, undersea [4] and space exploration [5]. See Sheridan [40] for an excellent review of the extensive liter- ature on teleoperation and telerobotics. Internet interfaces to soda and candy machines were demonstrated in the early 1980s, well before the introduction of the WWW in 1992. One of the first web cameras was the Trojan coffee pot at Cambridge. The Mercury Project’s Internet robot [18] went online in August 1994. In Sept 1994, Ken Taylor, at the University of Western Australia, demonstrated a remotely controlled six-axis telerobot with a fixed observing camera [10]. Although Taylor’s initial system required users to type in spatial coor- dinates to specify relative arm movements, he and his colleagues have subsequently explored a variety of user interfaces [23]. Also in October 1994, Wallace demonstrated a robotic camera [22] and Cox put up a system that allows WWW users to remotely schedule photos from a robotic telescope [24]. Paulos and Canny have implemented several Internet robots with elegant user interfaces [32, 33]. Bekey and Steve Goldberg used a six-axis robot, rotating platform, and stereo cameras to allow remote scholars to closely inspect a rare sculpture [41]. Internet robots are an active research area. In addition to the challenges associated with time delay, supervisory control, and stability, Internet robots must be designed to be operated by non-specialists through intuitive user interfaces and to be accessible 24 hours a day. See [25, 39] for recent papers and [20] for a collection of recent projects. In a ”collaborative” or Multiple Operator Single Robot (MOSR) system, a group of users shares simultaneous access to a single robot. Inputs from all users are combined to generate a single control stream for the robot as illustrated in Figure 2. User n . EnvironmentΣ Aggregate Control User 1 Figure 2: In a collaborative control system, an ensemble of user inputs are aggregated into a single control signal. Pirjanian studies how reliable robot behavior can be produced from an ensemble of sources [34]. Drawing on research in fault-tolerant software [29], Pirjanian considers systems with a number of homogenous sources (sharing a common objective), and considers a variety of voting schemes. He shows that fault-tolerant behavior fusion can be optimized using plurality voting [6] but does not study the motion paths generated by specific malfunctioning modes. McDonald, Small, Graves, and Cannon [31] describe an Internet-based collabora- 3 tive control system that allows several users to assist in waste cleanup using Point-and- Direct (PAD) commands [8]. In their system, collaboration is pipelined, with overlap- ping plan and execution phases. They demonstrate that collaboration improves overall execution time but they do not address conflict resolution between users. Chong et al [9] study a multi-operator-multi-robot (MOMR) networked system and propose several control methods to cope with collision arising from network time delays. Adapting their terminology, ours would be a multi-operator-single-robot (MOSR) system. Cinematrix is a system similar in spirit to MOSR. Cinematrix is a commercial software system that allows a roomful of participants to collaboratively control events projected on a screen [36]. Collaborative control is achieved using two-sided color- coded paddles and a computer vision system that aggregates the color field resulting from all paddle positions. The developers report that groups of untrained users are able to coordinate aggregate motion, for example to move a cursor through a complex maze on the screen. Fong et al use the term “collaborative control” to describe systems where collaboration occurs between a single mobile robot and a single human operator who is treated as a peer to the robot and modeled as a noisy information source [12]. This would be a SOSR system in Tanie’s terminology. Related models of SOSR collaboration such as Cobots are analyzed in [14, 2, 30]. Collaborative control is related to the very active research topic of Cooperative (behavior-based) robots, where groups of autonomous robots interact to solve an objective [3]. These might be described as No Operator Multiple Robot (NOMR) systems. Recent results are reported in [28, 11, 38, 35, 7]. Collaborative Control is also related to MOMR online collaborative games such as Quake (Capture the Flag), where users remotely control individual avatars. Outside of robotics, the notion of collaborative control is relevant to a very broad range of collaborative human activities including economic markets, pricing behavior, voting, traffic flows, etc. Excellent overviews of the broader context can be found in [27, 37]. There is also a substantial body of research on Distributed Artificial Intelligence (DAI) and Multi-Agent Systems (MAS), emphasizing multiple actors, multiple contexts, multiple representations, resource limitations, compatibility-based problems and robustness [13]. In our MOSR model of collaborative control, the focus is on group control of a single shared resource in contrast to groups of resources. A preliminary version of this paper appeared in [16]. For recent research in MOSR collaborative control, see [17, 21]. 3 User Interface To attract online users to use our system, we selected the popular Ouija [26] board game. Several users play the game together by placing their hands together on a sliding plastic disk known as a “planchette”. The planchette is free to slide over a board marked with letters and messages such as Yes and No. The group of users poses a question. 4 As each user concentrates, the planchette slides across the board to indicate an answer. Although we do not address the claim that the planchette is influenced by supernatural powers, it is in many cases influenced by the conscious and unconscious movements of all participants.

Load more