A Generic Non-Linear Method for Fisheye Correction
Total Page:16
File Type:pdf, Size:1020Kb
International Journal of Computer Applications (0975 – 8887) Volume 51– No.10, August 2012 A Generic Non-Linear Method for Fisheye Correction Pranali Dhane Krishnan Kutty Sachin Bangadkar Research Associate Associate Technical Fellow Research Associate KPIT Cummins Infosystems KPIT Cummins Infosystems KPIT Cummins Infosystems Ltd., Ltd., Ltd., Pune, India Pune, India Pune, India ABSTRACT These distortions continue to increase as we move away from With rapid advances in the field of vision based systems and the centre of the fisheye image. Perceiving these distorted ever increasing applications that they foster to; it brings along views can be both unusual and confusing to the viewer. It is many challenges both in terms of algorithm design and therefore desirable to correct fisheye images into their associated hardware. One such widely used gadget is the approximately rectilinear versions before presenting to the fisheye camera that is used in myriad of applications in end viewer [8]. various fields owing to their wide Field of View. However, Michal Kedzierski et. al. presented a fisheye correction these lenses introduce distortions when any real-world object method based on differential geometry and arc curvature in gets mapped on to the image plane. The amount of distortion every segment of input image [4]. Brauer-Burchardt, C. et. al. in fish eye images increases while moving radially outward proposed the fisheye correction algorithm based on circle from the image centre. Therefore, the amount of correction to fitting. This method only requires extraction of distorted be applied should also vary accordingly based on the pixel image points from straight lines in the 3D scene [5]. location. This paper presents a fast and real-time implementable technique for fisheye correction. The proposed Different models for fisheye distortion correction viz. method uses non-linear radial stretching and scaling down of polynomial and non-polynomial models have been studied in pixels thereafter; both in X and Y dimensions for correction. detail in literature. The problem with polynomial models is For real-time implementation of the proposed algorithm, we that there is no analytical method to invert them, i.e. there is propose the use of inverse mapping matrix. This enables no generic method to invert a forward model to its inverse for transformation of computationally extensive equations used use in correction of radial distortion. This leads to the vacant for correction into a simple look-up table that can easily be pixels in the corrected image. On the other hand, the non- implemented on a FPGA platform. polynomial models such as perspective model is based on the calculation of apparent focal length of fisheye camera which General Terms does not necessarily equate with the actual focal length of the Image Processing, Fisheye Images fish-eye camera[1][6]. Keywords In this paper, we propose a method for correcting fish eye Fisheye Distortion, Field of View (FOV), radial correction, distortion using a non-linear radial stretching technique. This mapping matrix is followed by correction in both X and Y direction for removing distortions. In order to implement this proposed 1. INTRODUCTION technique in real-time, we also introduce the concept of Fisheye cameras are finding increasing number of inverse mapping matrix. The final output is a simple look-up applications in surveillance, robotic vision, automotive rear- table that maps every pixel in the output fish eye corrected view imaging systems etc. because of their wide angle image to a corresponding pixel in the input uncorrected properties. There are broadly two types of fisheye lens; one is image. a circular fisheye lens whose vertical angle of view is 180 The rest of the paper is organized as follows. We explain our degree while the other one is full frame fisheye lens with 180 approach for fisheye correction in Section 2. In Section 3, we degree diagonal angle of view [9]. However, while fisheye describe calculation of mapping matrix. In Section 4 we lenses provide very large wide-angle views (theoretically the characterize the performance of our proposed approach by entire frontal hemispheric view of 180°) the images produced comparing the curvature of known landmark regions in the suffer from severe distortion since the hemispherical scene original and corrected images. This is followed by conclusion gets projected onto a flat surface [7]. and discussions in Section 5. There are two types of these distortions, radial distortion and tangential distortion. Tangential distortion is usually 2. FISHEYE IMAGE CORRECTION insignificant and is not considered while correcting the Typical fish eye lenses have a large field of view (FoV) that distortions. The effect of radial distortion is that straight falls in the range of 120 to 180 degrees. A sample fisheye lines/objects in the real world are distorted into curves and image is as shown in the Fig. 1. It is very obvious from this points are moved in radial direction from their correct position image that objects that are away from the center of the image [2][3]. tend to distort more. The formation of a fisheye image is explained in detail in the following subsection. 58 International Journal of Computer Applications (0975 – 8887) Volume 51– No.10, August 2012 domain. Based on these parameters, the proposed algorithm corrects for the distortion viz. estimates point ‘Zout’ for every point ‘Zin’ in the input image. The algorithm for correcting the distortion is detailed in the following section. 2.2 Distortion Correction A high level flowchart illustrating the steps involved in the proposed approach is as shown in Fig. 3. Input Fisheye Image Fig 1: Sample Fisheye Image A Calculation of image L specific parameters M G A 2.1 Formation of a Fisheye Image Alternate Process P Fig. 2 depicts the path that is traced by a ray of light in case of O Non-linear Radial P a pin-hole camera and a fish-eye lens. In the case of a pin-hole I R Stretching of Input camera, the reflected light ray coming from any object in the N G real world does not bend towards the normal and continues on I Image the same straight path until the film plane. It is for this reason T Vertical Stretching/ Mapping Matrix M that the objects in the images that are captured by a pin-hole H A camera do not suffer any appreciable distortion [8]. Vertical Correction Calculation T M R However, when the same reflected light ray falls on a fisheye I X lens, it bends more towards the normal. Because of this, the F Horizontal Stretching/ image captured by fisheye lens suffers from distortion. With Horizontal Correction F increase in the bending of the refracted ray, the distortion in L L the captured image also increases. This explains why the O O W distortion in fisheye images is more as we move away from W Rescaling Image to the center of the image [8]. input resolution Picture Plane Output Fisheye Corrected Zout Ray passing through Image normal lens Lens Position Zin R Fig 3: Sample Fisheye Image The description of each step as shown in the flowchart (Fig. 3) Ray refracted due Ɵ 1 is as given below: to fisheye lens r Ɵ2 2.2.1 Calculate Image specific parameters Ø Optical Axis f The area captured by the lens depends on its FoV. FoV of the fisheye lens is given as [9]: ( ) Q Ray1 From (1), Ray2 ( ) Fig 2: Fisheye Image Formation The FoV of typical fisheye lenses falls in the range of 120 to As shown in Fig. 2, the ray shown in blue color (Ray1) 180 degrees. Therefore denominator of (2) varies from 2 to 3. depicts the path of a reflected ray of light as it passes through For our experiments, with a FoV of 180 degrees, the focal a lens of a pinhole camera. The ray shown in red (Ray2), length is now deduced as: however, shows the path of the same ray when it travels through the fisheye lens. The amount of bending observed, in case of a fish eye lens, depends upon the characteristics of the lens. The proposed algorithm estimates the parameters required for estimating the bend (refraction) in the pixel 59 International Journal of Computer Applications (0975 – 8887) Volume 51– No.10, August 2012 It is worth noticing that the ‘f’ calculated is used to normalize For Ray 1 the image; thus making the algorithm agnostic of image resolution. (5) The center of the input fisheye image of width For Ray 2 and height is calculated as: (6) (4) Now from (5) & (6) For our proposed approach, all the calculations are done in the polar coordinate system. Therefore, the Cartesian coordinate (7) of each pixel is first converted into its polar coordinate where is the radius and ‘ ’ is the angle calculated with Where, is a constant dependent on characteristics of the positive X-axis as shown in Fig. 4. fisheye lens. Varying would vary the amount of correction in the 2.2.2 Radial Correction fisheye image. The constant for a particular fisheye lens can be estimated using heuristic approach. For our (X, Y) Y-axis experiments, we have fixed the value of to 1.15. (X’, Y’) From Fig.2, R r Ɵ Center X-axis (Cx, Cy) As shown in Fig. 4, the scene in real world gets mapped in the lesser number of pixels using fisheye lens as compared to the same area captured by pinhole camera. Because of this there will be certain pixel positions in the corrected image where pixel values will be of vacant intensities, which lead to interpolation of the vacant pixel values. In order to avoid interpolation, we proposed an output to input mapping technique viz.