AI Magazine Volume 16 Number 2 (1995) (© AAAI) Articles The Mobile Robot RHINO
Joachim Buhmann, Wolfram Burgard, Armin B. Cremers, Dieter Fox, Thomas Hofmann, Frank E. Schneider, Jiannis Strikos, and Sebastian Thrun
■ RHINO was the University of Bonn’s entry in the learning is employed to interpret sonar mea- 1994 AAAI Robot Competition and Exhibition. surements. RHINO is a mobile robot designed for indoor navi- Real-time operation: To act continuously gation and manipulation tasks. The general scien- in real time, any-time solutions (Dean and tific goal of the RHINO project is the development Boddy 1988) are employed wherever possible. and the analysis of autonomous and complex Any-time algorithms are able to make deci- learning systems. This article briefly describes the major components of the RHINO control software sions regardless of the time spent for compu- as they were exhibited at the competition. It also tation. The more time that is available, how- sketches the basic philosophy of the RHINO archi- ever, the better the results are. tecture and discusses some of the lessons that we Reactive control and deliberation: RHINO’s learned during the competition. navigation system integrates a fast, reactive on-board obstacle-avoidance routine with knowledge- and computation-intense map HINO, shown in figure 1, is a B21 mobile building and planning algorithms. robot platform manufactured by Real- RHINO’s software consists of a dozen differ- RWorld Interface. It is equipped with 24 ent modules. The interface modules (a base- sonar proximity sensors, a dual-color camera sonar sensor interface, a camera interface, and system mounted on a pan-tilt unit, and 2 on- a speech interface) control the basic commu- board I486 computers. Sonar information is nication to and from the hardware compo- obtained at a rate of 1.3 hertz (Hz), and cam- nents of the robot. On top of these, a fast era images are processed at a rate of 0.7 Hz. obstacle-avoidance routine analyzes sonar RHINO communicates with external computers measurements to avoid collisions with obsta- (two Sun SPARCSTATIONs) by a tetherless Ether- cles and walls at a speed as high as 90 cen- net link. timeters a second. Global metric and topolog- The RHINO project is generally concerned ical maps are constructed on the fly using a with the design of autonomous and complex neural network–based approach combined learning systems (Thrun 1994). The 1994 with a database of maps showing typical AAAI Robot Competition and Exhibition, rooms, doors, and hallways. RHINO employs a sponsored by the American Association for dynamic programming planner to explore Artificial Intelligence (AAAI), ended an initial unknown terrain and navigate to arbitrary six-month period of software design. Key fea- target locations. It locates itself by continu- tures of RHINO’s control software, as exhibited ously analyzing sonar information. In addi- at the competition, are as follows: tion, a fast vision module segments images Autonomy: RHINO operates completely from two color cameras to find target objects autonomously. It has been operated repeated- and obstacles that block the path of the ly for durations as long as one hour in popu- robot. RHINO’s control flow is monitored by an lated office environments without human integrated task planner and a central user intervention. interface. Learning: To increase the flexibility of the The integration of a dozen different soft- software, learning mechanisms support the ware modules, which all exhibit different tim- adaptation of the robot to its sensors and the ing and response characteristics, requires a environment. For example, neural network flexible scheme for the flow and synchroniza-
Copyright © 1995, AAAI. All rights reserved. 0738-4602-1994 / $2.00 SPRING 1994 31 Articles
ule, (2) the modules concerned with sensor interpretation and map building, (3) the planner and explorer, and (4) the visual rou- tines. The article concludes with a discussion that highlights some of the lessons that were learned during the AAAI competition.
Fast Obstacle Avoidance The obstacle avoidance runs on board, inde- pendent of other software components such as the planner. Every 0.25 seconds, a new velocity and motion direction are chosen according to the most recent sonar measure- ments. To rapidly adapt to new situations, only the last three sonar sweeps are consid- ered. RHINO can react immediately to changes in the environment and hard-to-see and moving obstacles such as humans. The obstacle-avoidance module controls both the velocity and the motion direction of the robot. At every instant in time, the veloci- ty is determined such that no collision will occur within the next two seconds (two-sec- ond rule). The motion direction is deter- mined based on target points, which are gen- erated by the planner (see discussion later). To reach a given target, the robot can choose among different trajectories on which it will travel with different velocities. RHINO selects Figure 1. The RHINO Robot from the its motion direction by maximizing its trans- University of Bonn. lational velocity, denoted by v, while mini- mizing the angle to the target point, denoted by θ. tion of information. The key principles for To determine v, a simplified model of the the design of RHINO’s software are as follows: robot’s environment is constructed. Proximi- Distributed control and communication: ty information, obtained from RHINO’s sonar Each module communicates with several oth- sensors, is used to construct a two-dimension- er modules through Ethernet (Fedor 1993). al obstacle line field. Every sonar reading is There is no single control unit, and commu- converted to a line in this field, as depicted in nication is not centralized. figure 2. To avoid collisions with obstacles, Asynchronous communication: RHINO’s the obstacle-avoidance routine considers a software lacks a central clock. Each of the variety of circular trajectories, one of which is modules runs independently of the other shown in figure 2. For each trajectory, the dis- modules. To resolve conflicts, certain mod- tance between the robot and the closest ules (such as the on-board obstacle-avoidance obstacle line along the projected trajectory is module) can take priority over other modules computed. This distance determines the (such as the planner) in determining the translational velocity v, according to the two- robot’s motion direction. second rule. The projected angle to the target Software fault tolerance: RHINO’s software point, θ, is calculated for the estimated robot is designed to accommodate sudden failures position and orientation after 0.25 seconds. of most of its software components. Almost For both values v and θ, a smoothed his- all modules can be stopped and restarted at togram is constructed. Because of the dynam- any time. Effective mechanisms ensure that ic constraints, only a small number of trajec- restarted modules will immediately obtain tories are reachable within the next 0.25 the currently available global information. seconds and are consequently considered in The following sections present some of the the histogram. Finally, the trajectory that key components of the RHINO approach in maximizes a weighted difference of v and θ is more detail: (1) the obstacle-avoidance mod- chosen. To increase the safety of the robot, a
32 AI MAGAZINE Articles security distance of 10 centimeters is kept to surrounding objects. This security distance is increased to as much as 30 centimeters as the robot’s velocity increases. RHINO’s obstacle-avoidance approach is easi- ly extendible to other sensors. For example, prior to the competition, we successfully employed camera information to identify small obstacles on the floor that block the path of the robot, as described later. Each visually detected obstacle is mapped into a few lines in the obstacle field, much like the sonar information described earlier. However, visual information was not used by the obsta- cle-avoidance routine during the AAAI com- petition, basically because sonar information was fast and accurate enough in the competi- tion ring.
Map Building and Position Control
RHINO’s global-navigation system builds and uses occupancy maps of the robot’s environ- Figure 2. Obstacle Line Field. ment. More specifically, when traveling through possibly unknown terrain, RHINO Each sonar reading is indicated by a line centered around the robot. interprets its sonar readings to generate a The trajectory, which is finally chosen by RHINO, is also shown. two-dimensional, discrete probabilistic occu- pancy map. Sonar sensors are interpreted using an artificial neural network, which esti- been found to be effective for the detection mates the likelihood of occupancy of any of rotational errors at the competition as well point in a three-meter circle around the robot as in various office environments. If RHINO (Thrun 1993). Multiple measurements are operates some 30 minutes with velocities as integrated using Bayesian inference (Moravec high as 90 centimeters a second in unknown 1993). Figure 3a shows a map that was con- terrain, the total error is usually smaller than structed while we manually steered the robot 30 centimeters. Without correcting the dead through the competition arena. This map reckoning, this error often accumulates as describes an area of approximately 30 x 20 much as 30 meters. meters. The hallways, rooms, large obstacles, To obtain topological information concern- and doors can clearly be recognized. ing the location of rooms, doors, and hall- To navigate based on global metric infor- ways, RHINO analyzes its metric occupancy mation, it is imperative that the robot be able map continuously. Walls are identified by to locate itself accurately in its map. RHINO is thresholding. A large database of examples of equipped with fairly accurate wheel encoders. door regions, hallways, and rooms (and parts However, even small angular errors in dead thereof) is continuously matched to assign reckoning can have devastating effects on the topological labels to the unoccupied areas in internal position estimation. To compensate the occupancy map. By analyzing the con- for such error, the robot continuously match- nectivity of the labeled map, RHINO is able to es its current sonar readings with its global identify doors, hallways, and rooms. An occupancy map. If a mismatch is found example of a topologically labeled map is between the occupancy map and the obsta- shown in figure 3b. This map, which is based cles predicted based on the most recent sonar on the metric map shown in figure 3a, subdi- sweep, the internal position is corrected vides the terrain into seven rooms or hall- accordingly. In addition, RHINO registers the ways (gray) and nine door regions (white). As angular orientation of walls with respect to is easy to see, most of the rooms and hallways its current location to correct more accurately have been identified correctly. In the bottom for rotational errors. This mechanism, which left corner of the figure, however, a small rests on the assumption that walls are typical- room has not been identified: Because of sen- ly perpendicular or parallel to each other, has sor noise, the occupancy map failed to cap-
SUMMER 1995 33 Articles
(a) (b)
unidenti®ed room
Figure 3. Two Views of the Competition Ring. A. This map of the competition ring was constructed from sonar measurements. Bright regions indicate free space, and dark regions indi- cate walls and obstacles. Walls and obstacles are enlarged by a robot diameter. B. Shown here is a topological analysis of the map. Obsta- cles are shown in black. As indicated by the different shading, the free space is divided into seven room-hallways (gray) and 9 door regions (white). The arrow points to an unidentified room in the competition ring.
ture a small wall—a problem that might par- paths to arbitrary goal locations or, as ticularly occur with thin walls, such as those described later, to unexplored regions. These found at the competition. paths are constantly refined and communi- The topological map analyzer works con- cated to the obstacle-avoidance routine, tinuously. At any point in time, it can be which then determines the final motion queried to output a topological map. Howev- direction and velocity of the robot. er, the quality of the topological maps RHINO’s main planning engine consists of a increases with time. The generation of the dynamic programming routine, which computes labels shown in figure 3b requires approxi- trajectories with minimum cost to a goal mately 15 minutes of processing time on a location (Howard 1960). The occupancy map Sun SPARCSTATION 10. Note that the underlying is translated into a cost function, such that database of topological examples consists of occupied territory results in a high traversal preselected prototypes based on occupancy cost and free territory in a low traversal cost. maps that were constructed at the University Dynamic programming propagates path of Bonn prior to the competition. information from the goal(s) to arbitrary loca- tions on the map. Consequently, steepest Planning and Exploration descent results in a minimum-cost path to the “cost-nearest” goal. Control can be gener- In this section, we describe how occupancy ated at any time without any significant com- maps are used when controlling the robot. putation. However, deliberation time is trad- RHINO’s planner generates minimum-cost ed for the quality of the resulting path.
34 AI MAGAZINE Articles
Because occupancy maps are often too inac- curate to generate collision-free motion con- trol, in dynamic environments, RHINO’s plan- ner commands only the rough motion start direction, which is then finalized by the colli- sion-avoidance routine. Consequently, if unmodeled obstacles block the robot’s path, the planner is faced with unexpected robot actions. Dynamic programming preplans for arbitrary robot locations. Goal information is propagated for every location in the map, not just the current location of the robot. Conse- quently, RHINO can quickly react if it finds itself to be in an unexpected location and generate appropriate motion directions with- out any additional computational effort. This rapid exception-handling ability provides the necessary freedom for the collision-avoidance routine to modify actions commanded by the robot planner at its own will. In both stages of the competition, RHINO explored and mapped unknown terrain. RHI- NO’s planning mechanism can easily be applied to generate explorative paths, lacking a specific goal point. If the set of goal posi- tions is defined as the set of positions for which no map information is available, RHINO moves straight to the unexplored. Figure 4 illustrates the path of some 15 minutes of autonomous robot exploration in the compe- tition ring. In this prototypical example, the Figure 4. Occupancy Map Constructed from Scratch during main hallways have already been traversed, 15 Minutes of Autonomous Robot Exploration. and RHINO continues to explore the unex- The robot’s path, which starts at the upper left corner, is also plored rooms. RHINO’s speed on the straight- shown. line segments of the exploration path was generally between 50 and 90 centimeters a second. Further details on planning and to reduce the data transfer by the radio link exploration can be found in Thrun (1993). and preprocess the image for the next stage. This process is performed on one of the on- Vision and Object Recognition board I486 computers. Sampling in space The images from the color camera system are (image size) and in time (frame rate) is done the input to a four-stage vision system, which dynamically, dependent on the actual veloci- solves two different tasks: First, it has to rec- ty of the robot. Thereby, the transmission- ognize important objects typically found in channel capacity is allocated in a task-driven the environment (for example, objects in an way. office environment). Second, it supplies valu- The second processing stage is done by an able information for the robot-navigation image-segmentation algorithm that partitions task by providing local occupancy maps to the transmitted image into homogeneous, the map builder and obstacle locations rela- connected regions (figure 5). Homogeneity is tive to the robot’s position to the collision- measured by a dissimilarity measure between avoidance module. This second task, howev- neighboring image sites (pixels or blocks of er, was not performed during the final runs at pixels). For reasons of efficiency, the dissimi- the competition. Here, map building and larity measure is restricted to a weighted obstacle avoidance relied solely on sonar squared sum of color and luminance differ- information, which turned out to be suffi- ences between sites (with an additional ciently reliable in the competition ring. threshold). Formally, the segmentation task In the first stage of low-level processing, can be described as a minimization problem images are low-pass filtered and subsampled of a cost function, which sums up the local
SUMMER 1995 35 Articles
Figure 5. In the Second Processing Stage, the Transmitted Image Is Partitioned. A. The raw image. B. The segmented image in coarse resolution (9 segments).
inhomogeneities of all regions for a given estimated based on the object location calcu- partition. To achieve real-time performance lated in the third stage of the vision system. without the need for special hardware, the At the competition, we used between 30 and segmentation is implemented by a fast 50 training examples to model each of the 7 region-merging scheme. The decision about object classes. These example images showed whether two neighboring regions should be different objects varying in object distance, merged depends on a comparison between lighting conditions, and the choice of the the current costs and the costs after merging. class representative. The assignment to a class The third stage takes the segmented image is done by minimizing the Mahalanobis dis- as an input and seeks to identify and label tance to the class mean (Duda and Hart certain elements of a typical indoor scene, for 1973). If the likelihood is below a certain example, the floor, walls, and doors. Impor- threshold, the candidate object is not accept- tant additional information for both naviga- ed as a member of a known class and is con- tion and object recognition can be derived: sidered unclassified. The distances and sizes of all objects or Once a target object is found, its location is regions located on the floor are calculated communicated to the planner and other based on knowledge of the position, viewing modules concerned with task and motion angle, and so on, of the cameras. This dis- control. tance and size information for walls and objects is incorporated into the occupancy Results and Discussion map. At the top of RHINO’s vision-processing This article surveys the software architecture architecture, a feature-based object recognizer of the fully autonomous RHINO robot, as it was detects objects of interest in the environ- exhibited at the 1994 AAAI Robot Competi- ment. The recognition module is able to learn tion and Exhibition. RHINO is controlled by a from labeled examples of feature vectors dozen software modules that work and com- extracted from example images. For every municate asynchronously. Special emphasis is type object, a Gaussian model (mean and put on real-time operation, learning, and the covariance matrix) is estimated according to integration of reactivity and global map the maximum likelihood principle. Typical knowledge. RHINO does not require prior features are the normalized mean and the knowledge on the locations of walls or obsta- variance of object luminance; the mean and cles or the topology of its environment. the variance of object color (hue and satura- In the first stage of the competition (Office tion); and geometric features such as the Delivery), RHINO had to move to a designated absolute size, width, and height of the object, target location. This stage consisted of three
36 AI MAGAZINE Articles trials, two of which counted for the final acquire new skills necessary for achieving a score. Because we are specifically interested in broad variety of tasks. navigation without prior information, we attempted to use the first trial for exploring Acknowledgments and mapping the competition ring and the Some of the low-level software (TCX [Fedor remaining two trials for the delivery task. 1993], device drivers for the speech board However, although RHINO traveled fast, we and the cameras) was provided by Carnegie learned that the arena could not be explored Mellon University, which is gratefully completely without prior information in the acknowledged. We also acknowledge the allotted time. Consequently, we had to “buy” steady and helpful support of Real-World a metric map for subsequent trials. Interface. Travel to the competition would One of the major problems we encoun- not have been possible without a generous tered at the competition was RHINO’s unreli- travel grant from AAAI and the invaluable able radio link. Unpredictable radio commu- assistance of Peter Lachart, Wolli Steiner, and nication, possibly based on interference with Peter Wallossek. The work of Hofmann was other radio links, caused RHINO’s on-board supported in part by the Ministry of Science operating system (LINUX, a PC-version of UNIX) and Research of the state of North Rhine- to suspend obstacle avoidance for periods of Westphalia. 10 seconds or more. In the second and third References trial of stage one, RHINO suffered from severe communication failures and collided repeat- Dean, T. L., and Boddy, M. 1988. An Analysis of Time-Dependent Planning. In Proceedings of the edly with walls. Thus, the first stage of the Seventh National Conference on Artificial Intelli- competition could not be completed, and RHI- gence, 49–54. Menlo Park, Calif.: American Associa- NO was excluded from the finals of this stage. tion for Artificial Intelligence. The communication problem was fixed in Duda, R. O., and Hart, P. E. 1973. Pattern Classifica- the second stage of the competition by mov- tion and Scene Analysis. New York: Wiley. ing the radio link closer to the competition Fedor, C. 1993. TCX: An Interprocess Communica- ring. In this stage (Office Cleanup), RHINO was tion System for Building Robotic Architectures. Pro- required to find and fetch objects such as grammer’s Guide to Version 10.xx. Pittsburgh, soda cans and paper wads, pick them up, and Penn.: Carnegie Mellon University. drop them in groups of three into a nearby Howard, R. A. 1960. Dynamic Programming and trash bin. Because RHINO is currently not sup- Markov Processes. Cambridge, Mass.: MIT Press. plied with a manipulator, it indicated its Moravec, H. P. 1988. Sensor Fusion in Certainty intention to pick up and drop objects by Grids for Mobile Robots. AI Magazine 9(2): 61–74. voice. RHINO used essentially the same explo- Thrun, S. B. 1994. A Lifelong Learning Perspective ration routines as in the first stage but at a for Mobile Robot Control. In Proceedings of the reduced speed. In addition, the visual rou- IEEE/RSJ/GI International Conference on Intelli- tines described earlier were employed for the gent Robots and Systems, 23–30. Washington, D.C.: IEEE Computer Society. identification of obstacles. At the competi- Thrun, S. B. 1993. Exploration and Model Building tion, RHINO found most of the objects in the in Mobile Robot Domains. In Proceedings of the starting room and then continued to clean up ICNN-93, 175–180. Washington, D.C.: IEEE Com- the hallway. Here, RHINO scored second, puter Society. defeated only by a collaborating team of three robots (see article by Tucker Balch and Joachim Buhmann is an associ- his colleagues, also in this issue). ate professor of computer science The AAAI competition ended an initial six- at the University of Bonn. He month period of software engineering. RHI- received a Ph.D. in theoretical NO’s software is generally applicable to physics from the Technical Uni- autonomous navigation in indoor environ- versity of Munich in 1988. He has ments. In the future, RHINO will operate 24 held postdoctoral positions at the hours a day, interrupted only by battery University of Southern California and the Lawrence Livermore National Laboratory. charging. Our main scientific interest is the His current research interest focuses on the theory study and the design of autonomous, com- of neural networks and their applications to image plex learning systems, which, in the domain understanding and signal processing. of robotics, includes adaptive approaches to sensory processing and lifelong robot learn- Wolfram Burgard is a research associate at the ing (Thrun 1994). We are currently imple- University of Bonn. He received his Ph.D. in com- menting various learning techniques that puter science from the University of Bonn in 1991 allow RHINO to adapt to new situations and
SUMMER 1995 37 Articles
and a Master’s degree from the University of Dort- Thrun’s main scientific interests mund. Burgard’s research focuses on expert sys- are AI, machine learning, and tems, fuzzy logic, logic programming, and robotics. computers on wheels. Thrun led the RHINO team at the 1994 AAAI Armin B. Cremers is a full profes- Robot Competition and Exhibi- sor of computer science and chair tion. of the Computer Science Institute at the University of Bonn. Prior to this position, he was a full profes- sor at the University of Dortmund and an assistant professor at the University of Southern California. Florida AI Graduate (MS) His scientific research is in the areas of database Student Fellowship and information systems and AI.
Dieter Fox received a Bachelor’s The Institute for the Interdisciplinary Study degree in 1990 and a Master’s of Human & Machine Cognition (IHMC) at degree in 1993, both from the the University of West Florida invites your University of Bonn. Currently, he application for the Fall of 1995. This is a is enrolled in the Ph.D. program special research fellowship restricted to in computer science at the Uni- Florida residents. versity of Bonn. His main scientif- ic interests are in robotics, map The IHMC was founded with legislative sup- building, and collision avoidance. port in 1990 as an interdisciplinary research unit. Additionally, IHMC has succeeded in Thomas Hofmann received a securing substantial extramural support and Bachelor’s degree in computer sci- has established an enviable research and ence in 1989 and a Master’s publication record. Interdisciplinary degree from the University of Bonn in 1993. Currently, he is a research is underway in the computational Ph.D. student in the Computer and philosophical foundations of AI, com- Science Department at the Univer- puters in education, knowledge-based sys- sity of Bonn. His main scientific tems, neural networks, internet-based interests are in computer vision, learning theory, autonomous agents, cognitive psychology, and statistics. and reasoning under uncertainty.
Frank E. Schneider received a The University is situated in a 1000-acre Master’s degree from the Universi- protected nature preserve bordering the ty of Bonn in 1994. Since 1989, Escambia River, and is approximately 14 he has worked for the Research miles north of the country's finest white- Institute of Applied Science in sand beaches. New Orleans is only 3 hours Wachtberg, Germany. His scientif- away by car. ic interests are robotics; virtual reality; and vision and simulation, in particular, the visualization of sensory data using The successful applicant(s) will have a BS in virtual reality techniques and video compression. Computer Science with outstanding aca- demic performance, a GRE score of at least 1400, and a strong interests in one or more Jiannis Strikos received a Bache- lor’s degree in computer science of the following areas: AI, cognitive science, from the University of Bonn in human/machine interaction, or any of sev- 1993. Since this time, he has been eral other areas. an active member of the Comput- The Florida AI Graduate Student Fellowship er Vision Group. His current includes a tuition-waiver and a half-time research is in the area of active graduate research assistantship at the IHMC vision, visual search, selective while pursuing a MS in Computer Science. attention, and real-time face recognition. Applicants should email
38 AI MAGAZINE