Project Deliverable D4

PROJECT DELIVERABLE D4 TOURBOT DESIGN

Shared-cost RTD

Project acronym: TOURBOT Project full title: Interactive Museum Tele-presence Through Robotic Avatars

Contract Number: IST-1999-12643 Key Action: 3 Action Line: 3-2-3 TOURBOT Design Project Deliverable

TOURBOT: Interactive Museum Tele-presence Through Robotic Avatars

Project Deliverable D4: Application Design Date Produced: June 27, 2000 Authors: Wolfram Burgard and Dirk Hähnel

Contents

1 Introduction...... 3 2 Hardware Design Specification...... 3 2.1 Hardware Layers of the Platform...... 4 2.1.1 Base: Drive and Shape...... 4 2.1.2 Enclosure: Computing Resources and Interfaces...... 5 2.1.3 Top: User Interaction...... 5 2.2 Sensor Systems of the Platform...... 5 2.3 Interfaces...... 6 2.4 Hardware of the On-site PC’s...... 6 3 Software Design Specification...... 7 3.1 Localization...... 9 3.2 Path Planning and Navigation Behaviors...... 10 3.3 Collision Avoidance...... 11 3.4 User Interaction...... 12 3.4.1 On-board Interface...... 13 3.4.2 Web Interface...... 13 3.5 Map Updating...... 13 4 System Integration and Validation Specification...... 14 4.1 System Integration...... 14 4.2 System Validation...... 15

TOURBOT Design Project Deliverable

1 Introduction

TOURBOT presents a complex system which, in order to effectively fulfill its goals, should simultaneously perform various tasks. It has to navigate reliably through the exhibition and simultaneously interact with visitors of the museum and users connected to the robot via the Web. This requires a flexible mobile system that is also backed by sufficient computing resources at the base station. Figure 1 presents the overall system setup. It consists of a mobile platform connected via a wireless link to a base station, which is directly connected to the Internet. According to the scenario of the application, the system has to control the on-board motors, update displays and furthermore has to monitor its environment to safely navigate the robot and to appropriately interact with Web and on-site users. The current document constitutes a design specification of the TOURBOT-System in order to fulfill the operational requirements set. The proposed design, besides serving the needs of the TOURBOT project, it also offers a high degree of modularity and flexibility for necessary adaptations, possible improvements and extensions of the system in future applications.

The next section will give a specification of the robot’s hardware and it’s interfaces in order to be controllable by the software-system design described in Section 3. Section 4 elaborates on system integration and validation issues.

2 Hardware Design Specification

According to the operational requirements, the TOURBOT mobile avatar should navigate safely through the museum and additionally should be able to interact with people in the museum. The overall hardware design covers three aspects. First it specifies the drive and shape of the robot, which will be installed in the base of the TOURBOT system. The second part, denoted as enclosure, contains the on-board computers, device-interfaces and networking components. The upper part (top) contains all necessary devices to interact with people in the museum and controlling or observing the robot over the Web. To acquire information about the environment, a variety of sensors is assigned, which are installed at different parts of the platform.

INTERNET

High-Speed Internet Connection

Wireless Connection

On-Site PC Figure 1. Hardware components of the TOURBOT-System

2.1 Hardware Layers of the Platform 2.1.1 Base: Drive and Shape To efficiently navigate in a populated environment and provide meaningful tours, the system must be able to move at speeds of at least 60 cm/sec and for long periods of time. Consequently, the system requires appropriate drives and battery power. To reduce the complexity of the navigation tasks, the system is designed to have a circular or an almost circular shape, with the drive being either synchro-drive or differential drive. Both kinds of drives allow the robot to turn in place, facilitating thus the use of path-planning techniques in which the robot is regarded as a point. Appropriate brakes allow the robot to stop quickly in order to avoid collisions with objects or people in the exhibition.

2.1.2 Enclosure: Computing Resources and Interfaces TOURBOT robotic avatar requires on-board computing power for various reactive processes designed to control the robot over its device-drivers and to quickly process and react to sensory input. Since these processes do not perform complex algorithms, a standard workstation, comparable to a 500 MHz Intel Platform, is deemed sufficient. This computer must have various interfaces to connect it to the robot’s hardware interfaces.

2.1.3 Top: User Interaction The devices installed on top of the enclosure are used for interaction with the system users. In order to provide information regarding its activities, the system is designed to carry an LCD-display, which is connected to the computer system in the enclosure. Additionally, the user interface will provide buttons to allow users to enter possible requests. Furthermore, the robot will have the option to install a motorized face, which is able to express different moods and to support interaction with the users. A pan-tilt device carrying a camera system is also included in the design. These cameras will yield as a pointing device showing the directions to which the robot will move and to point to the exhibits the robot is approaching in order to deliver images to the Web or to explain them to the users in the museum. Finally, emergency stop buttons will be provided in the top part, allowing untrained users to stop the robot in case of emergency.

2.2 Sensor Systems of the Platform To gather all relevant information about the environment, the robot needs sensory apparatuses. Based on sensory information the robot can update its knowledge about the environment and can adapt its behavior. To overcome the disadvantages of certain types of sensors our design includes different sensors. More specifically, we employ ultrasound sensors, laser-range finders, tactile sensors and cameras. The most important sensor for navigation purposes is the laser-range finder system of the type SICK LMS or SICK PLS. Such a sensor covers 180 degrees of the robot’s surrounding at a angular resolution of 0.25 to 0.5 degrees and at a distance resolution of 2.5 to 5cm. This sensor serves many different navigation purposes, such as localization, collision avoidance and map updating.

Since a laser-range finder is a light-based sensor, the system additionally will use ultrasound sensors to deal with transparent materials, such as glass or reflecting surfaces (i.e. mirrors). The ultrasound sensors will be mounted so that they cover the complete surrounding of the robot (ring of sensors). The information provided by these sensors will

be integrated with the data coming from the laser-range finder in order to support localization, collision avoidance and map building.

On top of the robot resides the stereo camera system, installed on a pan/tilt-unit. These cameras will be used to grab images and video streams that are broadcasted over the Web. The video streams of the viewed scene in the museum will also serve the purpose of remote control of the robot by the Web-users. Finally, the system will have a number of tactile sensors that are used to detect collisions of the system.

2.3 Interfaces To communicate with the different sensor systems and actuators, the computer system in the enclosure needs different interface devices:

 Frame-grabber or TV-cards for grabbing images with the on-board camera system

 Serial interfaces for the connection to the drives and the odometry, the sensor systems (ultra-sound, laser, tactile), the pan/tilt unit carrying the camera system, the motorized face and the buttons or the LCD/touch-screen.

 Wireless communication interface to connect the robot to the base station.

 Sound-device including amplifier for spoken texts.

 Graphics device to display multimedia information on the robot’s screen.

2.4 Hardware of the On-site PC’s For the overall control of the system and to communicate with users connected over the Internet, the TOURBOT system contains one or several on-site PC’s. These workstations are connected to the robot via wireless LAN and provide a fast communication link. Some of the high-level processes such as the Web server as well as some of the computationally intensive tasks are executed on the on-site PC’s. This ensures that the processor installed on the platform can serve all interfaces it is connected to.

User Interface

Navigation System

Map Updating

Path Planning and Navigation Behaviors

Collision Localization Avoidance

Figure 2. Software modules of the TOURBOT system.

3 Software Design Specification

The overall software architecture consists of a variety of modules (processes), which are executed in parallel on the robot’s on-board computers as well as on off-board servers. All processes are connected via Ethernet and communicate using TCX, a decentralized communication protocol for point-to-point socket communication. Figure 2 shows the major software modules of the overall software architecture along with the flow of information between different modules. Similar to other robot control architectures, the TOURBOT system is also organized in a hierarchical manner, with the device drivers at the lowest level and the user interfaces at the highest. The hierarchy, however, is not strict, in the sense that modules would pass information only within the same and not across adjacent layers.

With respect to environment sensing and related navigational competences, the system will be endowed with appropriate techniques to overcome various problems, a mobile robot operating in a populated environment is faced with. First, our design employs probabilistic representations, reasoning, and learning. Since the sensors used by mobile

robots are inherently inaccurate and noisy, robots generally cannot uniquely determine the state of the world. Probabilistic approaches, in contrast, are able to integrate sensory information over time and this way allow a mobile robot to robustly deal with the different sources of uncertainty. Instead of extracting just a single interpretation from the sensors, the TOURBOT system will consider multiple possible interpretations weighted by a numeric plausibility factor that is expressed as a conditional probability. By considering multiple hypotheses, the robot can deal in a mathematically rigorous way with ambiguities and uncertainty. This in turn facilitates recovery from false beliefs, a prerequisite for the robot to exhibit robust behavior. In addition, the probabilistic representation allows the robotic avatar to make optimal decisions under uncertainty.

To cope with the limited resources of the system, the computationally intensive algorithms used will have on-line characteristics and exhibit resource flexibility. Modules like the path planning component or the localization system will adapt to the available computational resources. Regardless of the available computation time, these modules will be configured as “any-time” algorithms, producing at any time the necessary output for the control of the platform. The more processing cycles are available, the more accurate, or optimal will be the result.

Finally, the processes on the TOURBOT system are configured to run in a modularized, distributed and asynchronous fashion and apply decentralized decision making. Thus, the TOURBOT software does not require a centralized clock or a centralized communication module. Synchronization of different modules is strictly de-central. Time-critical software (e.g., device drivers), and software that is important for the safety of the robot, such as the collision avoidance, run on the robot's on-board computers. Other higher- level software, such as the task control module or the localization system may run on the on-board or off-board computer, depending on the system’s load. Possible communication break-downs will be handled by the on-board control systems. The modular, decentralized software organization eases the task of software configuration and extension. Each module provides certain functionality, but not all modules are required to run the robot. For example, certain device drivers can be omitted if the corresponding device is not needed.

3.1 Localization Localization is the task of estimating the position of the robot in its environment. Without knowing its position, the TOURBOT system cannot efficiently perform typical tasks such as navigating to a certain exhibit and taking high-resolution images from user-

selected viewpoints. The TOURBOT system will employ a recently developed technique, denoted as Markov localization, to estimate the position of the robot. This technique offers the advantage of being able to globally estimate the position of the robot and to detect and recover from localization failures. To actually estimate the position of the robot in the workspace, a previously acquired map of the environment is used. The input of the robot’s sensors is then compared with the model of the environment (see Figure 3). To cope with uncertainties, the system maintains a probability distribution about the possible positions of the robot. To determine the actions to be taken it averages over the current belief, i.e. it considers, if useful, all possible states of the robot to determine the next action of the robot.

MAP SENSORY INPUT

POSITION

Figure 3. The localization module uses a map of the environment and sensory input to estimate the position of the robot.

In order to achieve the desired efficiency, the TOURBOT system uses a sample-based representation of the underlying position probability density. The advantage of this approach is that the system can efficiently update the density based on sensory input.

3.2 Path Planning and Navigation Behaviors To efficiently plan the trajectories of the robot from its current position to the next object, our design employs a variant of dynamic programming. The given map of the environment is transferred into a grid-based approximation denoted as occupancy grid map. Based on this representation, the system applies value iteration to determine the

cost-optimal path from the current position of the robot to the target location (see Figure 4). This approach operates in an any-time fashion, i.e., whenever a new target point is given, the system can at any time compute an intermediate target point which then is transferred to the collision avoidance module which itself controls the platform in order to reach this target point. This module will permanently monitor the current position of the robot and compute the cost-optimal path in reverse order, i.e. from the target point to all possible states of the robot. This way the system can immediately react to changes of the robot’s states caused by the reactive behaviors in the collision avoidance module.

MAP GOAL-POINT

NAVIGATION PLAN

Figure 4. The path-planning component uses a map of the environment and a target location entered by a user to compute a navigation plan.

Additionally, this module provides certain navigational behaviors, such as turning to a given target location, moving around a given exhibit in order to provide images of the specified object from different viewpoints.

3.3 Collision Avoidance The collision avoidance system (Figure 5) will use a recently developed technique denoted as the “dynamic window approach”. The advantage of this approach over other approaches developed so far is that it incorporates the dynamics of the robot to quickly react unforeseen obstacles. The key idea of the dynamic window approach is to consider all admissible velocities that can be reached within 0.25 seconds. Among these velocities, the collision avoidance module chooses that velocity which maximizes the expected future reward of the system given by an evaluation function. This evaluation function simultaneously maximizes the progress of the system measured by the distance to the

(intermediate) target location, the heading to the target point as well as the velocity of the system. Thereby it considers given constraints such as the minimum distance to obstacles and people in the vicinity of the robot. This is achieved by introducing a safety margin, effectively keeping the robot away from objects.

ODOMETRY INFORMATION SENSORY INPUT NAVIGATION PLAN

STEERING COMMAND

Figure 5. The collision avoidance system uses the information provided by the odometry, the sensory input, and the current navigation plan to compute the next steering command issued to the platform.

3.4 User Interaction The user interfaces constitute an important part of the overall system, ensuring that the robot can interact with people in the museum and over the Web. To satisfy the needs of different users (users tele-operating the system and on-site users), the TOURBOT system is equipped with two interface modules.

ROBOT CAMERA

MOBILE PLATFORM

WIRELESS CONNECTION

CONTROL SOFTWARE A/V STREAMING PRODUCER

WWW SERVER A/V STREAMING SERVER

ON-SITE PC

INFORMATION A/V STREAM WEB REQUEST

WWW BROWSER

USER PC

Figure 6. User interface architecture. In addition to the standard Web server, the system will include an A/V stream server to provide live images from the museum.

3.4.1 On-board Interface The task of the on-board interface is to perform interaction with the users in the museum. This will be achieved by displaying necessary information on the robot’s screen installed on top of the robot. This screen will be a graphics display to allow the visualization of high-resolution graphics and images. It, furthermore, outputs directives on how to interact with the robot.

3.4.2 Web Interface The Web interface allows users all over the world to access the TOURBOT system. The TOURBOT will be controlled from a Web page providing all necessary information for giving directives to the robot and to acquire video of the viewed scene and high- resolution images from arbitrary places of the environment. This interface will be provided by a dedicated Web server that forwards user requests to the robot control system. Users have the ability to choose two different modes, denoted as the control and observation modes. In the control mode a user has individual access to the robot and can control its motion in the robot’s operational space. This is achieved by appropriate selections on a representation of the environment on the users Web page. Additionally the users can direct the robot to grab images using the robot’s on-board cameras. In the observation mode, the users cannot control the motion of the robot but can get various information also provided to the control pages. Whereas the control page offers individual access to the TOURBOT system, the observation pages offer simultaneous access by large groups of Web visitors.

In addition to the Web server providing the Web pages for controlling the robot and for the on-site information, the TOURBOT system will provide a streaming server, which broadcasts video streams over the Internet (see Figure 6).

3.5 Map Updating The map-updating component allows the robot to maintain its knowledge about the state of the environment. Using this component, the robot can quickly learn a map about the environment from scratch (see Figure 7). Thus, whenever the robot is deployed in a new environment or whenever the environment changes, the robot uses this component to update the world model. Additionally, the TOURBOT-system will include a component to track people in the surrounding of the robot. This information will be used by the navigation systems to improve the behavior of the system. For example, the robot will avoid trajectories leading it through a crowd of visitors and in such situations choose a detour around people in the museum. On the other hand, it may use the information about the environment to check whether people are intentionally blocking the path of the robot. In such cases the robot may kindly ask for free space in the direction indicated by its pointing device.

POSITION SENSORY INPUT

Figure 7. The map updating system computes a map of the environment based on the current position of the robot and the sensory input.

4 System Integration and Validation Specification

The system will be integrated and validated in a series of steps with increasing complexity. In all steps the new software modules will be evaluated under usual and extreme conditions in order to achieve a high level of robustness.

4.1 System Integration 1. Device drivers. In the first step, the necessary device drives which connect the software to the hardware of the TOURBOT system will be developed. This includes extensive evaluations of the drivers with respect to the availability of necessary commands and their correct execution.

2. Collision avoidance and map-updating. The next step is the integration of the collision avoidance module into the TOURBOT system. This will also include the evaluation of the interfaces to the device drivers of the motors and the odometry. Additionally, this step will validate whether the system provides the required safety margins to the objects in the exhibition.

3. Localization. In the third step the localization module will be connected to the navigation system. Based on the odometry information provided by the robot’s drive and the sensory input, this module will be evaluated according to accuracy and robustness of the position estimate.

4. Path planning and navigation behaviors. The goal of the next integration step addresses the trajectory planning system. This module communicates with the localization component and the collision avoidance module in order to perform complex navigation actions. The major task in this integration activity therefore is to ensure that the corresponding actions are carried out by the mobile platform.

5. On-board user interface. After finishing the integration of the basic navigation modules, we will connect the user interface to the navigation system. This includes assessment whether the interface provides the correct output in typical interaction scenarios. This phase will furthermore include the integration of the motorized face and the pointing device.

6. Internet access. The final step in the development process is designed to connect the TOURBOT system to the Internet. Here we will evaluate whether the navigation system provides the required safety when the platform is controlled by users over the Web. Additionally we will validate whether the update rates of the displays are high enough in order to give Web users a “live-impression” about what is going on at the remote site.

4.2 System Validation For the purposes of final assessment and validation, the project’s immediate application sites will be involved. The developed system will be installed in the sites for a short period and will navigate and interact autonomously both with members of the museums’ staff and with museum visitors on-site or through the Web.

Such trials are considered of utmost importance for TOURBOT validation, since they involve both professional users (museum staff) and ordinary users (visitors) and are conducted under the every-day working conditions of an application site. TOURBOT assessment will be based on (a) the stability and effectiveness of the developed technology in complex environments, and (b) successful performance of Web and on-site tours. The criteria that will be used are specified in Annex I; they provide effective measures for system evaluation in a quantitative way:

1. frequency of failures: this measure applies to each individual module of the system (robotic platform, navigation module, interfaces, communication module) and quantitatively evaluates their operation in realistic conditions.

2. quality of service in delivered visual information: this measure applies to the system’s web interface, evaluating the achieved ratio ImageQuality/ResponseTime for a (prespecified) system set-up.

3. Frequency of uncompleted tours: this measure applies to the system as a whole and quantifies the system’s effectiveness in carrying out web and on-site tours.

4. Frequency of operator’s interventions: this measure applies to the system as a whole and quantifies its degree of autonomy; it differs from the previous measure in that a tour may be completed even after a temporary intervention of the operator.