Techniques for Web Telerobotics Barnaby Dalton This thesis is presented for the degree of Doctor of Philosophy Department of Mechanical and Materials Engineering University of Western Australia Perth, Australia 2001 Abstract Web Telerobotics is a new and rapidly growing field of telerobotic research. Initially it was con- sidered a novelty, but developments in the field combined with rapidly improving Internet tech- nologies have opened up new application areas. The most promising area is the use of web devices as teaching aids in school and university course work. Given that web telerobots are appearing in increasingly large numbers with many varying appli- cations, the motivation for this research was to identify, implement, and evaluate techniques for controlling robots (and other devices) over the web. Of particular importance was the identifi- cation of a set of requirements that all web telerobot applications require. A solution to these requirements could then be encapsulated in a form that is easily reusable in other web control projects. In order to understand what was required in a telerobot system, two systems were built. The first used the common gateway interface and enabled operators to control the robot using a standard web browser. During the course of operation and development of this system it became apparent that some functionality was impossible to implement due to the underlying techniques that had been used. This led to the identification of a set of functional requirements that should be provided by any web telerobotics framework. A Java based framework was developed to implement these requirements. The framework con- sisted of a central router (MOM) that clients connected into. Users and applications connected via the MOM were able to collaborate together to achieve tasks using shared objects known as contexts. There are three types of contexts: domains – used to define shared areas, channels – used to exchange messages of a particular type, and tokens – used to control access to resources. The framework was designed to be generic and reusable in other web applications. The framework not only enabled device control over the Internet, but also opened up new possibil- ities for distributed control. Multiple operators can cooperate together, while software agents con- tribute to analysis and planning. Multiple robots can be controlled by one operator, or a user could use multiple interface screens to control a single robot. In fact, applications of this framework are iv Abstract not limited to control of robots. It is suitable for any application that requires collaboration and control over the Internet. The framework was tested by applying it using the original telerobot hardware. The final system consisted of a number of servers that controlled hardware (such as cameras, and the robot), helper applications, and any number of Java Applets operated by users. The framework enabled all these different applications to communicate and collaborate together, and to negotiate for control of the robot. The system has run for the last year and has had thousands of users. Analysis of usage statistics and user feedback has shown that it is an improvement on the original common gateway interface system, and has shown users collaborating together in ways that were not previously possible. Contents Abstract iii Contents v Figures and Tables ix Acknowledgements xiii 1 Introduction 1 1.1 Experimental Setup ................................. 4 1.2 Scope of this thesis ................................. 5 1.3 Development Timeline ............................... 5 2 Background and Related Work 7 2.1 Telerobotics ..................................... 7 2.2 Web Telerobots ................................... 8 2.2.1 Web Interface for Telescience (WITS) ................... 9 2.2.2 Berkeley .................................. 11 2.2.3 Xavier .................................... 12 2.2.4 Pumapaint ................................. 14 2.2.5 Swiss Federal Institute of Technology ................... 16 2.2.6 TeleZone .................................. 18 2.2.7 Other Web Robots ............................. 20 2.3 Other telerobots ................................... 21 2.3.1 Distributed Telerobotics .......................... 23 2.4 Summary ...................................... 25 3 The CGI System 27 3.1 Extending HTTP and HTML ............................ 27 3.2 CGI ......................................... 28 3.2.1 WIN CGI .................................. 30 3.2.2 Generating Data for CGI .......................... 30 3.2.3 State Management ............................. 32 3.2.4 Other Techniques for producing dynamic content ............. 33 3.2.5 Frames ................................... 34 3.3 Client Side Extensions ............................... 35 3.3.1 Scripting languages ............................. 35 3.3.2 Plugins ................................... 35 3.3.3 Executables ................................. 36 3.4 Using CGI to control a Robot ............................ 36 3.5 The UWA System .................................. 37 3.5.1 Architectural Changes ........................... 38 3.5.2 Functional Changes ............................. 40 3.6 CGI Implementation ................................ 44 vi CONTENTS 3.7 User State ...................................... 45 3.7.1 Users .................................... 47 3.8 Move Commands .................................. 47 3.8.1 Single Moves ................................ 49 3.8.2 Image Defined Moves ........................... 49 3.8.3 Multiple Moves ............................... 50 3.9 Images ........................................ 51 3.10 HTML page ..................................... 52 3.11 Robot Server .................................... 55 3.11.1 Control ................................... 55 3.11.2 Communication Protocol .......................... 59 3.12 Image Server .................................... 60 3.12.1 Communication Protocol .......................... 61 3.13 Live Video ...................................... 62 4 CGI Results and Discussion 63 4.1 Telerobot Usage ................................... 63 4.2 Reliability and Errors ................................ 68 4.2.1 Errors .................................... 70 4.2.2 Closure ................................... 71 4.3 Strengths ...................................... 72 4.3.1 Flexibility .................................. 72 4.4 Limitations ..................................... 73 4.5 Conclusions ..................................... 75 5 Network Protocols and Architectures 77 5.1 Protocol Design ................................... 78 5.2 Internet Protocols .................................. 78 5.2.1 Network Models .............................. 80 5.2.2 TCP Operation ............................... 82 5.2.3 What can be controlled by an Applet? ................... 84 5.3 Testing network behaviour ............................. 84 5.3.1 Round Trip Times - TCP .......................... 85 5.3.2 Round Trip Times - UDP .......................... 87 5.3.3 Conclusions from Network Tests ...................... 95 5.4 Web Telerobot Requirements ............................ 97 5.5 Distributed Architectures .............................. 99 5.5.1 CORBA ................................... 100 5.5.2 Remote Method Invocation ......................... 100 5.5.3 Message Oriented Middleware ....................... 101 5.5.4 Java based frameworks ........................... 101 5.5.5 Connection Architectures .......................... 102 5.6 Conclusions ..................................... 103 6 A First Step Away from CGI 105 6.1 Introduction ..................................... 105 6.2 Internet Robotics .................................. 106 6.3 Middleware Technologies .............................. 107 6.3.1 CORBA ................................... 107 6.3.2 Remote Method Invocation ......................... 108 CONTENTS vii 6.3.3 Message Oriented Middleware ....................... 108 6.3.4 Comparison of Middleware ........................ 108 6.4 The Framework ................................... 109 6.4.1 The Router ................................. 109 6.4.2 Peers .................................... 111 6.5 The Framework in Practice ............................. 112 6.6 Future Work and Conclusion ............................ 115 7 Distributed Architecture 117 7.1 Revised Framework ................................. 119 7.1.1 Contexts .................................. 119 7.1.2 Transceivers ................................ 121 7.1.3 Parcels ................................... 122 7.2 The MOM Architecture ............................... 122 7.2.1 Communication Layer ........................... 122 7.2.2 Context Layer ................................ 124 7.2.3 Parcels ................................... 125 7.2.4 Queues ................................... 127 7.3 Peer Architecture .................................. 128 7.3.1 Communication Layer ........................... 129 7.3.2 Context Layer ................................ 130 7.3.3 Client Layer ................................ 132 7.3.4 Presentation Layer ............................. 132 7.4 Communication ................................... 132 7.4.1 Socket Transceiver ............................. 133 7.4.2 RMI Transceiver .............................. 139 7.4.3 Other Transceivers ............................. 142 7.4.4 Comparison of Transceivers ........................ 143 7.4.5 Adaptive Bandwidth and Stream Separation ................ 144 7.5