A Minimum Variance Calibration Algorithm for Pan-Tilt Robotic Cameras in Natural Environments
Total Page：16
File Type：pdf, Size：1020Kb
1 A Minimum Variance Calibration Algorithm for Pan-Tilt Robotic Cameras in Natural Environments Dezhen Song1,NiQin1, and Ken Goldberg2 1: CS Department, Texas A&M University, College Station, TX 77843 2: IEOR and EECS Departments, University of California, Berkeley, CA 94720 Abstract— A new generation of inexpensive robotic pan-tilt cameras can maintain high-resolution panoramic displays of natural environments. However, the pan-tilt mechanisms are imprecise: small errors can produce large errors in the panoramic display. It is thus important to accurately estimate pan-tilt values. We present a new calibration algorithm that does not rely on calibration markers or fixed orthogonal edges which are rarely available in natural scenes. Our calibration algorithm uses image variance density to optimally estimate camera pan and tilt values by incrementally refining image registration using overlapping images from prior frames. Experiments suggest that the new calibration algorithm can reduce calibration error by 81%.In a companion paper [19], we present a new image registration Nominal/desired Actual algorithm based on spherical projection that optimally aligns camera coverage camera coverage the resulting frames. Fig. 1. Operators’ desired camera coverage and the actual camera coverage cannot perfectly match with each other when there is a calibration error. The I. INTRODUCTION calibration error causes difficulties when the tele-operated pan-tilt robotic Scientific study of wild animals requires continuous ob- camera is used to track moving animals over a distance. servation over a distance. Recent developments in wireless telecommunications facilitate low-bandwidth connectivity to remote sites. A new class of low-cost tele-operated pan-tilt- pan and tilt potentiometer readings are approximate or have zoom robotic video cameras allows fast deployment of systems a limited accuracy. We propose a new calibration algorithm that can provide high-resolution images from a wide field of that optimally estimates the pan and tilt positions for a new view in a remote environment. One example is the Panasonic camera frame by incrementally refining the image registration HCM 280 camera with built-in streaming server, 22x zoom solution for prior overlapping frames in increasing order of k motorized optical lens, 350◦ pan range, and 90◦ tilt range. location variance density. For images, our algorithm runs O k k However, minor errors in the camera pan and tilt mechanism in time ( log ). Experiments show that our algorithm can can produce large errors between nominal camera coverage reduce calibration error by 81% if compare with a method that and actual coverage as illustrated in Figure 1. For example, simply selects frames with large overlapping regions. an error of 0.5◦ in camera tilt position can cause a 41.67% error in coverage when a Panasonic HCM 280 camera operates II. RELATED WORK at its highest zoom. Camera calibration is used to determine accurate intrinsic Recent progress in camera calibration provides the capa- parameters (CCD sensor size, skew factor, focus length, and bility to calibrate an outdoor camera under natural lighting lens distortion) and/or extrinsic parameters (position and ori- without using predefined calibration objects [7]. Those meth- entation of a camera). Our calibration problem focuses on the ods utilize linear edges in a rigid scene such as contours of camera pan and tilt mechanism, which is part of a camera’s buildings as a calibration reference. Unfortunately, it cannot extrinsic parameters. be directly applied to online calibration problems in a natural The fundamental work on camera calibration is to calibrate a environment where linear edges are often not available. still camera using still calibration objects, which is credited as We notice that intrinsic camera parameters such as CCD photogrammetric calibration. This is based on a parameterized sensor size, focus length, and skew factor do not change over camera imaging model such as the pinhole perspective pro- time. Hence, we assume these parameters are known and focus jection model [26]. The unique characteristics of the imaging on calibrating the pan and tilt mechanism. We assume that model distinguish camera calibration from robot calibration. This work was supported in part by the National Science Foundation Imprecision in camera parameters causes discrepancy between under IIS-0534848 and IIS-0535218, by Intel Corporation, by Panasonic, by image coordinate systems and the world coordinate system. TAMU startup fund, and by UC Berkeley’s Center for Information Technology Research in the Interest of Society (CITRIS). For more information please Sometimes, the discrepancy is caused by the fact that the contact [email protected] or [email protected]. pinhole model, which is an approximate model itself, cannot accurately model the imaging process [11]. The calibration an online pan-tilt calibration problem. We begin with assump- process can be viewed as model-fitting or parameter identifi- tions. cation. Photogrammetric calibration observes a 3D calibration object with a known geometry, usually consisting of several mutually orthogonal planes [3], [8], [9], [15]. This approach A. Assumptions is very efficient, but requires carefully designed calibration We assume that all images are taken from a fixed camera, objects located at 2 or 3 orthogonal planes [26]. which only performs pan and tilt movements. Due to cost and Since it is not convenient to accurately set up calibration space limitations, the camera’s angular potentiometer usually objects involving 2 or 3 orthogonal planes, a different ap- has limited accuracy and may deteriorate over time. Hence proach, known as self-calibration, does not use any calibration these extrinsic parameter readings are inherently approximate object, but relies on the rigidity of a scene to calibrate calibrate and need to be calibrated. We assume that the rate of the error intrinsic camera parameters. It takes a series of images while change is slow and hence periodic calibration can compensate moving a camera in a static scene. The assumed rigidity of for it. We assume that the intrinsic parameters including image a scene is used to compute camera parameters [5], [6], [13], resolution, camera focus length, and CCD sensor size are pre- [14], [16]–[18], [21], [23], [24]. Self-calibration reduces the calibrated and known. complexity of setting up the calibration process by assuming the perfect motion accuracy, which can be viewed as the inverse of the problem that we are facing. Because we assume B. Nomenclature and Feature-based Calibration known intrinsic parameters, only a camera’s pan and tilt Salient and fixed points in the environment are identified mechanism needs to be calibrated. as calibration feature points. Figure 2 shows some sample Realizing that it is not convenient to use a predefined feature points including the center of a clock, a corner of the calibration pattern to calibrate an outdoor camera, Basu and bookshelf, a power outlet on the wall, and a nail in the wall. active calibration Ravi [2] propose the notion of , which For the jth feature point, we can pan and tilt the camera to calibrates the image center and focus of a camera using a set of center a frame j on it. Hence, we also reference it as the jth linear edges in a scene. Collins and Tsin [7] further developed frame. this idea, which can estimate both intrinsic and extrinsic Let us define the following variables, camera parameters for pan-tilt-zoom cameras based on how X∗ p∗,t∗ j the linear edges change while rotation and zoom operations • j =( j j ): true camera position of the th frame. It are performed. Since linear edges are difficult to find in a remains unknown during the entire calibration process. natural environment, we expand the approach by using the Sometimes, it is also referred to as an optimal camera offset generated by imaging alignment techniques to replace position because the calibration problem is an error- linear edges. We address the error propagation problems in minimization problem and the true camera position is the multiple imaging alignments by choosing an optimal set of optimal position. Xˆ p , t j images to calibrate the pan and tilt positions for a newly- • j =(ˆj ˆj): the nominal camera position of the th captured image. frame. It is the reading from the camera potentiometer. It Sinha and Pollefeys [20] develop automatic calibration al- is referred to as the nominal camera position because of gorithms for a pan-tilt-zoom camera with a focus on automatic the errors in the camera potentiometer. X p ,t j zoom calibration. Similar to our approach, their method does • j =( j j): the measured camera position of the th not require a structured scene or calibration object. They first frame. It is the corresponding measured value of the true determine intrinsic camera parameters at the lowest zoom and camera position. The measurement process can be done then increase camera zoom settings to obtain radial distortion using pre-defined calibration objects. In our paper, we use parameters. During the calibration, they focus on intrinsic image alignment. e X − X∗ j parameter calibration and assume camera pan-tilt is accurate • j = j j : the measurement error of the th and repeatable. They obtain extrinsic parameters by matching frame. We assume that the measurement is non-biased, mean e mean X X∗ images captured to a pre-constructed panorama. The accuracy ( j )=0. Therefore, ( j)= j and σ2 p of extrinsic parameters depends on the panorama quality, Var e Var X ( j)0 ( j )= ( j) = σ2 t because which is sensitive to the number of frames and lighting 0 ( j) pan and tilt are independent variables. variations across frames. This paper complements their work 2 2 2 • σ {σ p ,σ t } by concentrating on extrinsic parameter calibration without j = max ( j) ( j) , the maximum variance of X e assuming an existing panorama.