Corke Thesis
Total Page：16
File Type：pdf, Size：1020Kb
High-Performance Visual Closed-Loop Robot Control By Peter Ian Corke A thesis submitted in total fulfillment of the requirements for the degree of Doctor of Philosophy July 1994 Department of Mechanical and Manufacturing Engineering University of Melbourne Abstract This thesis addresses the use of monocular eye-in-hand machine vision to control the position of a robot manipulator for dynamically challenging tasks. Such tasks are defined as those where the robot motion required approaches or exceeds the performance limits stated by the manufacturer. Computer vision systems have been used for robot control for over two decades now, but have rarely been used for high-performance visual closed-loop control. This has largely been due to technological limitations in image processing, but since the mid 1980s advances have made it feasible to apply computer vision techniques at a sufficiently high rate to guide a robot or close a feedback control loop. Visual servoing is the use of com- puter vision for closed-loop control of a robot manipulator, and has the potential to solve a number of problems that currently limit the potential of robots in industry and advanced applications. This thesis introduces a distinction between visual kinematic and visual dynamic con- trol. The former is well addressed in the literature and is concerned with how the ma- nipulator should move in response to perceived visual features. The latter is concerned with dynamic effects due to the manipulator and machine vision sensor which limit per- formance and must be explicitly addressed in order to achieve high-performance control. This is the principle focus of the thesis. In order to achieve high-performance it is necessary to have accurate models of the system to be controlled (the robot) and the sensor (the camera and vision system). Despite the long history of research in these areas individually, and combined in visual servoing, it is apparent that many issues have not been addressed in sufficient depth, and that much of the relevant information is spread through a very diverse literature. Another contribution of this thesis is to draw together this disparate information and present it in a systematic and consistent manner. This thesis also has a strong theme of experimentation. Exper- iments are used to develop realistic models which are used for controller synthesis, and these controllers are then verified experimentally. iii iv Acknowledgments This work has been inspired by the earlier work of Lee Weiss, who formalized the so- called `image-based visual servo' structure, and Russell Andersson who built `King Pong', the amazing real-time system that played ping-pong. Initial work on the topic of this thesis was conducted during a CSIRO Fellowship visit to the GRASP Laboratory at the University of Pennsylvania in 1988/9. I am grateful to Dr. Bob Brown, then Chief of the Division of Manufacturing Technology, for supporting me with the Fellowship, and Professor Richard (Lou) Paul of Penn for the invitation to work at the GRASP laboratory. I am extremely grateful to CSIRO which has financially supported my part-time study and made available unique laboratory facilities for this investigation. Additional support has come from a University of Melbourne/ARC small grant. My supervisors, Professor Malcolm Good of the University of Melbourne, and Dr. Paul Dunn at CSIRO have provided valuable discussion and guidance in the course of this research. In particular I would like to thank Malcolm Good for his very thorough reading of draft material and his questions which have forced me to clarify my thinking and hopefully also clarify the text. My colleagues at the Division have helped, via numerous discussions, to answer many questions that I have raised in the course of this work. In particular I would like to thank Paul Dunn, Patrick Kearney, Robin Kirkham, Dennis Mills and Vaughan Roberts. Paul Dunn helped me come to grips with many issues related to cameras and lighting, and also developed the FPGA based hardware board and hardware programming language which is used for timing purposes in the robot controller. Robin Kirkham built the Mark III Puma/VME interface and low level library, both of which have given sterling service. Vaughan Roberts helped with his experience in modeling dynamic systems, particularly resonant mechanical structures, and was always willing to discuss issues of dynamics, modeling and control. He also built some essential hardware including the analog filter board and quadrature signal decoder, and wrote a number of useful software tools for fit- v ting transfer functions and uploading data from the FFT analyzer. Patrick Kearney helped to educate a poor engineer in some of the complex issues manifest in lens systems and CCD sensors. Murray Jensen and Geoff Lamb manage and maintain the excellent com- puting facilities of the laboratory, without which this work could not have been achieved. The Division's librarians, Jannis Young and Karyn Gee, have been a great help in tracking down and obtaining copies of hundreds of references for me, some quite obscure. Addi- tional help has come from Les Ewbank for mechanical design and drafting, Ian Brittle's Research Support Group for mechanical construction, and Terry Harvey and Steve Hogan for electronic construction. Kim Ng of Monash University and Rick Alexander, previously of Monash, helped in discussions on camera calibration and lens distortion, and also loaned me the SHAPE system calibration target used in Chapter 3. Vision Systems Ltd. of Adelaide, through their then US distributor Tom Seitzler of Vision International, loaned me an APA-512 video-rate feature extractor unit for use while I was at the GRASP Laboratory. My father David Corke, a wild-life photographer by inclination, was cameraman for a number of late night video recording sessions and later assisted with editing. Malcolm Paterson from CSIRO Communication Services Group helped with video editing, production and video format conversions. Special thanks to David Hoadley who (volunteered) to proof read much of this thesis at very short notice. I am also very grateful to my next door neighbour, Jack Davies, who fixes things around my house that I don't seem to get around to doing. My family, Phillipa, Lucy and Madeline have been enormously supportive for rather a long period of time. They have endured my long working hours at home and in the laboratory, worse than usual absent mindedness, and my out of control horizontal filing system. I dedicate this thesis to them. vi To Phillipa, Lucy and Madeline. vii viii Contents 1 Introduction 1 1.1 Limitations of conventional robots . 1 1.2 Visual servoing . 2 1.3 Research questions and methodology . 7 1.4 Structure of the thesis . 10 2 Robot manipulators 13 2.1 Manipulator kinematics . 15 2.1.1 Forward and inverse kinematics . 18 2.1.2 Accuracy and repeatability . 20 2.1.3 Manipulator kinematic parameters . 21 2.2 Manipulator rigid-body dynamics . 22 2.2.1 Recursive Newton-Euler formulation . 26 2.2.2 Symbolic manipulation . 29 2.2.3 Direct dynamics . 31 2.2.4 Rigid-body inertial parameters . 32 2.2.5 Transmission and gearing . 38 2.2.6 Quantifying rigid body effects . 40 2.3 Electro-mechanical dynamics . 41 2.3.1 Friction . 44 2.3.2 Motor . 47 ix x CONTENTS 2.3.3 Current loop . 57 2.3.4 Combined motor and current-loop dynamics . 59 2.3.5 Velocity loop . 65 2.3.6 Position loop . 68 2.3.7 Conclusion . 74 2.4 Significance of dynamic effects . 75 2.5 Manipulator control . 77 2.5.1 Rigid-body dynamics compensation . 77 2.5.2 Electro-mechanical dynamics compensation . 82 2.6 Computational issues . 83 2.6.1 Parallel computation . 84 2.6.2 Symbolic simplification of run-time equations . 86 2.6.3 Significance-based simplification . 87 2.6.4 Comparison . 88 2.7 Conclusion . 90 3 Computer vision 93 3.1 Light . 95 3.1.1 Illumination . 95 3.1.2 Surface reflectance . 97 3.1.3 Spectral characteristics and color temperature . 98 3.2 Image formation . 101 3.2.1 Light gathering and metering . 103 3.2.2 Focus and depth of field . 105 3.2.3 Image quality . 106 3.2.4 Perspective transform . 109 3.3 Camera and sensor technologies . 110 3.3.1 Sensors . 111 CONTENTS xi 3.3.2 Spatial sampling . 115 3.3.3 CCD exposure control and motion blur . 118 3.3.4 Linearity . 120 3.3.5 Sensitivity . 120 3.3.6 Dark current . 125 3.3.7 Noise . 125 3.4 Video standards . 127 3.4.1 Interlacing and machine vision . 131 3.5 Image digitization . 133 3.5.1 Offset and DC restoration . 133 3.5.2 Signal conditioning . 134 3.5.3 Sampling and aspect ratio . 134 3.5.4 Quantization . 139 3.5.5 Overall MTF . 140 3.6 Camera and lighting constraints . 142 3.6.1 Illumination . 144 3.7 Image interpretation . 146 3.7.1 Segmentation . 147 3.7.2 Binary image features . 155 3.7.3 Visual temporal sampling . 162 3.8 Perspective and photogrammetry . 165 3.8.1 Close-range photogrammetry . 167 3.8.2 Camera calibration techniques . 168 3.8.3 Eye-hand calibration . 177 3.9 The human eye . 179 3.10 Conclusion . 180 xii CONTENTS 4 Kinematics of visual control 185 4.1 Fundamentals . 186 4.2 Prior work . 190 4.3 Position-based visual servoing . 195 4.3.1 Photogrammetric techniques . 196 4.3.2 Stereo vision . 197 4.3.3 Depth from motion . 197 4.4 Image based servoing . 198 4.4.1 Approaches to image-based visual servoing . 200 4.5 Implementation details . 204 4.5.1 Cameras .