Calibration of Ultra-Wide Fisheye Lens Cameras by Eigenvalue Minimization
Total Page：16
File Type：pdf, Size：1020Kb
Memoirs of the Faculty of Engineering, Okayama University, Vol. 46, pp. 10-20, January 2012 Calibration of Ultra-Wide Fisheye Lens Cameras by Eigenvalue Minimization Kenichi KANATANI¤ Department of Computer Science, Okayama University Okayama 700-8530 Japan (Received November 21, 2011) We present a new technique for calibrating ultra-wide ¯sheye lens cameras by imposing the con- straint that collinear points be recti¯ed to be collinear, parallel lines to be parallel, and orthogonal lines to be orthogonal. Exploiting the fact that line ¯tting reduces to an eigenvalue problem, we do a rigorous perturbation analysis to obtain a Levenberg-Marquardt procedure for the optimization. Doing experiments, we point out that spurious solutions exist if collinearity and parallelism alone are imposed. Our technique has many desirable properties. For example, no metric information is required about the reference pattern or the camera position, and separate stripe patterns can be displayed on a video screen to generate a virtual grid, eliminating the grid point extraction processing. 1. Introduction et al. [8], and Okutsu et al. [13]. Komagata et al. [9] further introduced the parallelism constraint and the Fisheye lens cameras are widely used for surveil- orthogonality constraint, requiring that parallel lines lance purposes because of their wide angles of view. be recti¯ed to be parallel and orthogonal lines to be They are also mounted on vehicles for various pur- orthogonal. However, the cost function has been di- poses including obstacle detection, self-localization, rectly minimized by brute force means such as the and bird's eye view generation [10, 13]. However, Brent method and the Powell method [16]. ¯sheye lens images have a large distortion, so that In this paper, we adopt the collinearity- in order to apply the computer vision techniques ac- parallelism-orthogonality constraint of Komagata et cumulated for the past decades, one ¯rst needs to al. [9] and optimize it by eigenvalue minimization. rectify the image into a perspective view. Already, The fact that imposing collinearity implies eigenvalue there is a lot of literature for this [3, 5, 7, 8, 11, 12, minimization and that the optimization can be done 13, 14, 17, 18]. by invoking the perturbation theorem was pointed The standard approach is to place a reference out by Onodera and Kanatani [15]. Using this princi- grid plane and match the image with the reference, ple, they recti¯ed perspective images by gradient de- whose precise geometry is assumed to be known cent. Here, we consider ultra-wide ¯sheye lenses and [3, 4, 5, 7, 18]. However, this approach is not very do a more detailed perturbation analysis to derive the practical for recently popularized ultra-wide ¯sheye ± Levenberg-Marquardt procedure, currently regarded lenses, because they cover more than 180 angles of as the standard tool for e±cient and accurate opti- view and hence any (even in¯nite) reference plane mization. cannot cover the entire ¯eld of view. This di±culty The eigenvalue minimization principle has not can be circumvented by using the collinearity con- been known in the past collinearity-based work [1, straint pointed out repeatedly, ¯rst by Onodera and 8, 12, 13, 17]. Demonstrating its usefulness for ultra- Kanatani [15] in 1992, later by Swaminathan and Na- wide ¯sheye lens calibration is the ¯rst contribution yar [17] in 2000, and by Devernay and Faugeras [1] in of this paper. We also show by experiments that 2001. They pointed out that camera calibration can the orthogonality constraint plays an essential role, be done by imposing the constraint that straight lines pointing out that a spurious solution exists if only should be recti¯ed to be straight. This principle was collinearity and parallelism are imposed. This fact applied to ¯sheye lenses by Nakano, et al. [12], Kase has not been known in the past collinearity-based ¤E-mail [email protected] work [1, 8, 12, 13, 15, 17]. Pointing this out is the This work is subjected to copyright. All rights are reserved by this author/authors. 10 Kenichi KANATANI MEM.FAC.ENG.OKA.UNI. Vol. 46 second contribution of this paper. For data acquisition, we take images of stripes of di®erent orientations on a large-screen display by placing the camera in various positions. Like the past θ m collinearity-based methods [1, 8, 12, 13, 15, 17], our method is non-metric in the sense that no metric in- formation is required about the camera position or the reference pattern. Yet, many researchers pointed f out the necessity of some auxiliary information. For example, Nakano et al. [11, 12] proposed vanishing r point estimation using conic ¯tting to straight line images (recently, Hughes et al. [5] proposed this same Figure 1: The imaging geometry of a ¯sheye lens and the technique again). Okutsu et al. [14] picked out the incident ray vector m. images of antipodal points by hand. Such auxiliary information may be useful to suppress spurious so- and determine the values of f, a1, a2, ... along with lutions. However, we show that accurate calibra- the principal point position. Here, f0 is a scale con- tion is possible without any auxiliary information stant to keep the powers rk within a reasonable nu- by our eigenvalue minimization for the collinearity- merical range (in our experiment, we used the value parallelism-orthogonality constraint. f0 = 150 pixels). The linear term r=f0 has no coe±- This paper is organized as follows. In Sec. 2, we cient because f on the right-hand side is an unknown describe our imaging geometry model. Section 3 gives parameter; a1 = a2 = ¢ ¢ ¢ = 0 corresponds to the derivative expressions of the fundamental quantities, stereographic projection. Since a su±cient number of followed by a detailed perturbation analysis of the correction terms could approximate any function, the collinearity constraint in Sec. 4, of the parallelism right-hand side of (2) could be any function of θ, e.g., constraint in Sec. 5, and of the orthogonality con- the perspective projection model (f=f0) tan θ or the straint in Sec. 6. Section 7 describes our Levenberg- equidistance projection model (f=f0)θ. We adopt the Marquardt procedure for eigenvalue minimization. In stereographic projection model merely for the ease of Sec. 8, we experiment our non-metric technique, using initialization. stripe images on a video display. We point out that In (2), even power terms do not exist, because the a spurious solution arises if only collinearity and par- lens has circular symmetry; r is an odd function of allelism are imposed and that it can be eliminated θ. We assume that the azimuthal angle of the projec- without using any auxiliary information if orthogo- tion is equal to that of the incident ray. In the past, nality is introduced. We also show some examples of these two were often assumed to be slightly di®erent, real scene applications. In Sec. 9, we conclude. and geometric correction of the resulting \tangential distortion" was studied. Currently, the lens manu- 2. Geometry of Fisheye Lens Imaging facturing technology is highly advanced so that the tangential distortion can be safely ignored. If not, We consider recently popularized ultra-wide ¯sh- we can simply include the tangential distortion terms eye lenses with the imaging geometry modeled by the in (2), and the subsequent calibration procedure re- stereographic projection mains unchanged. In the literature, the model of the form r = c1θ + θ 3 5 r = 2f tan ; (1) c2θ + c3θ + ¢ ¢ ¢ is frequently assumed [7, 8, 13, 12]. 2 As we see shortly, however, the value of θ for a spec- where θ is the incidence angle (the angle of the inci- i¯ed r is necessary in each step of the optimization dent ray of light from the optical axis) and r (in pix- iterations. So, many authors computed θ by solv- els) is the distance of the corresponding image point ing a polynomial equation using a numerical means from the principal point (Fig. 1). The constant f [7, 8, 13, 12], but this causes loss of accuracy and ef- is called the focal length. We consider (1) merely be- ¯ciency. It is more convenient to express θ in terms cause our camera is as such, but the following calibra- of r from the beginning. From (2), the expression of tion procedure is identical whatever model is used. θ is given by For a manufactured lens, the value of f is unknown ³ ³ ³ ´3 ³ ´5 ´´ ¡1 f0 r r r or may not be exact if provided by the manufacturer. θ = 2 tan + a1 + a2 + ¢ ¢ ¢ : Also, the principal point may not be at the center 2f f0 f0 f0 (3) of the image frame. Moreover, (1) is an idealization, and a real lens may not exactly satisfy it. So, we generalize (1) into the form 3. Incident Ray Vector ³ ´ ³ ´ 3 5 Let m be the unit vector in the direction of the r r r ¢ ¢ ¢ 2f θ + a1 + a2 + = tan ; (2) incident ray of light (Fig. 1); we call m the incident f0 f0 f0 f0 2 11 January 2012 Calibration of Ultra-Wide Fisheye Lens Cameras by Eigenvalue Minimization ³ 1 ³ ´ ´ X 2k ¡ ray vector. In polar coordinates, it has the expression @θ ¡ 1 2 θ ¡ r y v0 0 1 = cos 1 + (2k 1)ak : @v0 f 2 f0 r sin θ cos Á k=1 m = @ sin θ sin Á A ; (4) (10) cos θ Substituting these into (9), we can evaluate @m=@u0 where θ is the incidence angle from the Z-axis and Á and @m=@v0. is the azimuthal angle from the X-axis. Since Á, by Next, we consider derivation with respect to f.