Mobile Robot Navigation Using Visual Servoing T. TEPE DC 2010.018 1 DYNAMICS & CONTROL TECHNOLOGY GROUP MOBILE ROBOT NAVIGATION USING VISUAL SERVOING M.Sc. INTERNSHIP Supervisor : Prof. Dr. Henk NIJMEIJER Coach : Dr. Dragan KOSTIĆ Student : Tufan TEPE Student ID: 0666323 2 ABSTRACT Equipping robots with vision systems increases the versatility of the robots but also complexity of their control. Despite the increasing complexity, vision remains an attractive sensory modality for mobile robot navigation since it provides rich information about the robot's environment. In this work, a problem of visual servoing based on a fixed monocular camera mounted on a mobile robot is investigated. A homography based control method is used for autonomous navigation of a mobile robot with nonholonomic motion constraints. The visual control task uses the idea of homing. With this approach, an image is taken previously at the desired position. Then, the robot is driven from an initial position towards the desired position by using the information extracted from the target image and the images taken during movement of the robot. 3 Table of Contents 1 INTRODUCTION .............................................................................................................................. 5 2 DESIGN ISSUES ............................................................................................................................... 5 2.1 Camera configuration .............................................................................................................. 5 2.1 Servoing architectures ............................................................................................................. 6 3 AN INSIGHT INTO VISUAL SERVOING METHODS ............................................................................ 7 3.1 The geometry of image formation ............................................................................................ 7 3.1 Analysis of visual servoing methods ......................................................................................... 9 4 PROJECT DESCRIPTION ................................................................................................................. 13 5 HOMOGRAPHY BASED VISUAL SERVOING of a NONHOLONOMIC MOBILE ROBOT...................... 13 5.1 Homography and its estimation ............................................................................................. 13 5.1.1 Geometric transformations ......................................................................................... 14 5.1.2 Situations in which solving a homography arises .......................................................... 18 5.1.3 How to find the homography? ..................................................................................... 19 5.2 Motion model of the mobile robot ......................................................................................... 29 5.3 Input-output linearization and control law ............................................................................. 33 5.3.1 Input-output linearization ............................................................................................ 34 5.3.2 Control law .................................................................................................................. 35 5.3.3 Desired trajectories of the homography elements ....................................................... 36 5.4 Stability analysis ..................................................................................................................... 38 6 SIMULATIONS .............................................................................................................................. 39 7 EXPERIMENTAL ARRANGEMENTS ................................................................................................ 58 8 CONCLUSIONS .............................................................................................................................. 59 APPENDIX A .................................................................................................................................... 60 APPENDIX B .................................................................................................................................... 62 4 1-INTRODUCTION Robots are electro-mechanical machines which are designed in such a way that they interact with their environment. In order to realize that interaction in a desired manner, they must be equipped with appropriate sensory modalities. In today's world so far, most of robotic applications take place in known environments or in the environments which are arranged to be suitable for robots. Robots have been rarely used until lately in the work environments which can not be controlled fully or about which not much information is available. The main reason of this limitation lies under the insufficient sensory capabilities of the robots. In order to compensate for the lack of information obtained from the surroundings, integration of different sensors to the robots is made to be one of the crucial steps in the design of the robots and vision is recognized to be very important to increase the versatility of robots. In the last couple of decades, a lot of work and investigation have been carried out successfully in the area of robotic vision [1], [2], [3]. Increased computing power and developed pixel processing hardware enable analysis of images at a sufficient rate to guide the robotic manipulators without touching the objects [1]. With the use of vision devices and the information obtained from them in robotic applications, the term "visual servoing or visual servo control" is started to be used. "Visual Servo Control" refers to closed loop control of the pose of a robot by utilizing the information extracted from vision sensors and it relies on the offerings and techniques from many elemental areas such as image processing, computer vision, kinematics, dynamics and control theory. 2-DESIGN ISSUES While designing a vision-based control system, one can raise many questions ranging from the type of the camera to be used to the type of the lens, from the number of cameras to where to place the cameras, from which kind of image features to utilize to whether to derive three dimensional description of the scene or to use two dimensional image data or combination of both etc. Since vision has a broad application area and new techniques and solutions are being developed day by day, the number of this type of questions can be increased easily. However, two very crucial issues in the design step of vision based control systems are explained to stay in the bounds of this project and people can consult with numerous academic sources easily to obtain detailed information for other aspects. 2.1. Camera Configuration One main issue when constructing a vision based control system is the determination of the place where the camera is positioned. There are two main options: the camera can be placed at a fixed location and it does not possess any motion or it can be mounted to the robot. These configurations are named as "fixed camera" and "eye-in-hand" configurations respectively. If a fixed camera configuration is used, the camera is placed at a location that it is allowed to observe the task space and the robot/manipulator. Since the camera is not exposed 5 to any motion, the geometric link between the task space and the camera does not change. However, the clear view of the task space of the camera can be hampered by the manipulator motion and this kind of occlusions can create severe degradation of the performance or even some instability issues. With an eye-in-hand system, the camera is mounted on the robot/manipulator. This configuration enables the camera to see the task space without any occlusions while the robot travels around the work space. As opposed to the fixed camera configuration, the geometric relationship between the task space and the camera alters when the robot moves in this configuration. On the other hand, the scene that the camera sees can change very drastically when the position of the camera attachment point is exposed to large and fast movements. This drawback may be encountered especially with multiple link robotic manipulators and could have undesired performance consequences. 2.2. Servoing Architectures Different servoing architecture classifications are offered by different people in the literature but the mostly used one is based upon the question: "Is the error signal or the task function defined in three dimensional work space coordinates or directly in terms of the image features?" and the answer to this question resulted in such a taxonomy that the error signal can be defined in 3D workspace coordinates or directly in terms of image features or combination of them. 2.2.1. Image Based Visual Servoing This approach uses the image data directly to control the robot motion and the task function is defined in the image such that there is no need to estimate the pose error in Cartesian space explicitly. The image measurements that are used to determine the task/error function are the pixel coordinates of a set of image features such as interest points and the task function is isomorphic to the camera pose. A control law is constructed to map the image error to robot motion directly. A system can either use a fixed camera or eye-in-hand configuration. In either case, the motion of the robot results in changes of the image provided by the vision system. Hence, determination