Journal of Automation, Mobile Robotics & Intelligent Systems VOLUME 9, N° 4 2015 Improving Self-localization Efficiency In a Small Mobile Robot by Using a Hybrid Field of View Vision System Submitted: 27th August 2015; accepted 18th September 2015 Marta Rostkowska, Piotr Skrzypczyński DOI: 10.14313/JAMRIS_4-2015/30 gorithms require data from highly precise sensors, such as laser scanners [28], or have high computing Abstract: power demands, if less precise data (e.g. from pas- In this article a self-localization system for small mobile sive cameras) are used [8]. Thus, the SLAM approach robots based on inexpensive cameras and unobtrusive, is rather unsuitable for small mobile robots, such like passive landmarks is presented and evaluated. The main our SanBot [19], which have quite limited resources contribution is the experimental evaluation of the hybrid with respect to on-board sensing, computing power, field of view vision system for self-localization with arti- and communication bandwidth. Thus, for such a robot ficial landmarks. The hybrid vision system consists of an an approach to self-localization that does not need to omnidirectional, upward-looking camera with a mirror, construct a map of the environment, or uses a simple and a typical, front-view camera. This configuration is and easy to survey representation of the known area inspired by the peripheral and foveal vision co-operation is required. Moreover, the self-localization system in animals. We demonstrate that the omnidirectional should use data from compact and low-cost sensors. camera enables the robot to detect quickly landmark In the context of navigation CCD/CMOS cameras candidates and to track the already known landmarks are the most compact and low-cost sensors for mobile in the environment. The front-view camera guided by robots [6]. However, most of the passive vision-based the omnidirectional information enables precise mea- localization methods fail under natural environmen- surements of the landmark position over extended dis- tal conditions, due to occlusions, shadows, changing tances. The passive landmarks are based on QR codes, illumination, etc. Therefore, in practical applications which makes possible to easily include in the landmark of mobile robots artificial landmarks are commonly pattern additional information relevant for navigation. employed. They are objects purposefully placed in We present evaluation of the positioning accuracy of the the environment, such as visual patterns or reflect- system mounted on a SanBot Mk II mobile robot. The ex- ing tapes. Landmarks enhance the efficiency and ro- perimental results demonstrate that the hybrid field of bustness of vision-based self-localization [29]. It was view vision system and the QR code landmarks enable also demonstrated that simple artificial landmarks the small mobile robot to navigate safely along extended are a valuable extension to visual SLAM [3]. An obvi- paths in a typical home environment. ous disadvantage is that the environment has to be engineered. This problem can be alleviated by using Keywords: self-localization, artificial landmark, omnidi- simple, cheap, expendable and unobtrusive markers, rectional camera which can be easily attached to walls and various ob- jects. In this research we employ simple landmarks printed in black and white that are based on the ma- 1. Introduction trix QR (Quick Response) codes commonly used to An important requirement for any mobile robot is recognize packages and other goods. to figure out where it is within its environment. The In our recent work [21] we evaluated the QR code pose of a wheeled robot (position and orientation landmarks as self-localization aids in two very differ- T xR R yR R] ) can be estimated by means of odom- ent configurations of the camera-based perception etry, but this method alone is insufficient [27], and the system: an overhead camera that observed a land- pose = [x has to θ be corrected using measurements from mark attached on top of a mobile robot, and a front- external sensors. Although there are many approach- view camera attached to a robot, which observed es to self-localization known from the literature, now- landmarks freely placed in the environment. Both so- adays the Simultaneous Localization and Mapping lutions enable to localize the robot in real-time with (SLAM) is considered the state-of-the-art approach to a sufficient accuracy, but both have important practi- obtain information about the robot pose [7]. cal drawbacks. The overhead camera provides inex- The SLAM algorithms estimate from the sensory pensive means to localize a group of few small mobile measurements both the robot pose and the environ- robots in a desktop application, but cannot be easily ment map, thus they do not need a predefined map scaled up for larger mobile robots operating in a real of the workspace. This is an important advantage, environment. The front-view camera with on-board because obtaining a map of the environment that is image processing is a self-contained solution for self- suitable for self-localization is often a tedious and localization, which enables the robot to work autono- time-consuming task. However, the known SLAM al- mously, making it independent from possible com- 28 Journal of Automation, Mobile Robotics & Intelligent Systems VOLUME 9, N° 4 2015 munication problems. However, the landmarks are tained hybrid field of view vision system called HOPS detectable and decodable only over a limited range (Hybrid Omnidirectional Pin-hole Sensor), which is of viewing configurations. Thus, the robot has to turn quite similar in concept to our design is presented in the front-mounted camera towards the area of land- [5], where the calibration procedure is described that mark location before it starts to acquire an image. In enables to use this sensor for 3D measurements of a complicated environment, with possible occlusions the scene. Unfortunately, [5] gives no real application this approach may lead to a lot of unnecessary mo- examples. Also Adorni et al. [1] describe the use of tion. Eventually, the robot can get lost if it cannot find a combined peripheral/foveal vision system including a landmark before the odometry drifts too much. an omnidirectional camera in the context of mobile In this paper we propose an approach that com- robot navigation. Their system uses both cameras in bines to some extent the advantages of the overhead a stereo vision setup and implements obstacle detec- camera and the front-view camera for self-localiza- tion and avoidance, but not self-localization. tion with passive landmarks, avoiding the aforemen- Although the bioinspired vision solutions in mo- tioned problems. We designed an affordable hybrid bile robot navigation mostly extract natural salient features, in many practical applications artificial from nature, and resembles the peripheral and foveal landmarks are employed in order to simplify and visionfield of in view animals. vision The system, system which consists takes of inspirationa low-cost speed-up the image processing and to make the de- omnidirectional camera and a typical, front-view tection and recognition of features more reliable [15]. camera. The omnidirectional component, employing Visual self-localization algorithms are susceptible to - errors due to unpredictable changes in the environ- vides to the robot an analogy of the peripheral vision ment [11], and require much computing power to inan animals.upward-looking It gives thecamera robot and the a ability profiled to mirrorquickly pro de- process natural features, e.g. by employing local vi- sual descriptors [24]. The need to circumvent these contrast, the front-view camera provides an analog of problems in a small mobile robot that is used for edu- fovealtect interesting vision. The objects robot canover focus a large on details field of of view. already In cation and requires reliable self-localization, offering only limited computing resources motivated us to The cooperation of these two subsystems enables to enhance the scene by artificial landmarks. Although trackdetected in real-time objects inmany a much landmarks narrower located field in of the view. en- active beacons can be employed, such like infra-red vironment, without the need to move the robot plat- LEDs [27], most of the artificial visual landmarks are form, whereas it is still possible to precisely measure passive. This greatly simplifies deployment of the the distances and viewing angle to the already found markers and makes them independent of any power landmarks. source. Depending on the robot application and the The reminder of this paper is organized as follows: characteristics of the operational environment very In the next Section we analyze the most relevant re- different designs of passive landmarks have been pro- lated work. Section 3 introduces the concept and de- posed [9, 22]. In general, simple geometric shapes can sign of the hybrid vision system, whereas the land- be quickly extracted from the images, particularly if marks based on QR codes and the image processing they are enhanced by color [3]. A disadvantage of such algorithms used in self-localization are described in simple landmarks is that only very limited informa- Section 4. The experimental results are presented in tion (usually only the landmark ID) can be embedded Section 5. Section 6 concludes the paper and presents in the pattern. In contrast, employing in landmark de- an outlook of further research. sign the idea of barcode, either one-dimensional [4] or two-dimensional [12] makes it possible to easily 2. Related Work encode additional information. In particular, matrix The advantages of biologically-inspired vision for codes, that proliferated recently due to their use in robot self-localization have been demonstrated in smartphone-based applications enable to fabricate few papers – for instance Siagnian and Itti [25] have much more information-rich landmarks. Moreover, shown that extracting the “gist” of a scene to produce landmarks based on matrix codes are robust to partial a coarse localization hypothesis, and then refining occlusion or damage of the content.