Fast Vignetting Correction and Color Matching for Panoramic Image Stitching

FAST VIGNETTING CORRECTION AND COLOR MATCHING FOR PANORAMIC IMAGE STITCHING Colin Doutre and Panos Nasiopoulos Department of Electrical and Computer Engineering The University of British Columbia, Vancouver, Canada ABSTRACT When images are stitched together to form a panorama there is often color mismatch between the source images due to vignetting and differences in exposure and white balance between images. In this paper a low complexity method is proposed to correct vignetting and differences in color Fig. 1. Two images showing severe color mismatch aligned with no between images, producing panoramas that look consistent blending (left) and multi-band blending [1] (right) across all source images. Unlike most previous methods To compensate for brightness/color differences between which require complex non-linear optimization to solve for images in panoramas, several techniques have been correction parameters, our method requires only linear proposed [1],[6]-[9]. The simplest of these is to multiply regressions with a low number of parameters, resulting in a each image by a scaling factor [1]. A simple scaling can fast, computationally efficient method. Experimental results somewhat correct exposure differences, but cannot correct show the proposed method effectively removes vignetting for vignetting, so a number of more sophisticated techniques effects and produces images that are highly visually have been developed. consistent in color and brightness. A common camera model is used in most previous work on vignetting and exposure correction for panoramas [6]-[9]. Index Terms— color correction, vignetting, panorama, A scene radiance value L is mapped to an image pixel value image stitching I, through: 1. INTRODUCTION I = f ()eV ()x L (1) A popular application of image registration techniques is to In equation (1), e is the exposure with which the image stitch together multiple photos into a panorama [1][2]. This was captured, and f(E) is the cameras response function, allows wide angle images (up to 360°) to be created from which in general can be a non-linear function. The pixel multiple photos taken with a normal camera. Panoramic I V x image stitching has been extensively studied in the literature value represents a red, green or blue sample. ( ) is the [2] and there are also several commercial software programs vignetting function which is dependent on the position of the for creating panoramas. pixel in the image (x). V(x) is one at the image’s optical When two images are aligned, there are almost always center and it decreases towards the edges. The irradiance visible seams due to mismatches in the brightness and color observed by the image sensor at location x is E = eV(x)L. between images (Fig 1). There are several causes of these With an estimate of the exposure, and models for the camera seams, such as registration errors, exposure differences, response f(E) and the vignetting V(x), the scene radiance can vignetting [3] (radial falloff towards the edges of an image) be recovered from the measured pixel value as: and variable white balance. g()I L = To prevent visible edges in the final panorama, blending eV (x) (2) techniques are used to make transitions between images smooth. Popular methods include multi-band blending where g(I) is the inverse of the camera response function. [4][1] and gradient domain blending [5]. While these Once the radiance values are found, each image can be techniques do remove visible seams, they do nothing to rendered with a common exposure and no vignetting (i.e. remove global color differences. Therefore using blending V(x)=1) with equation (1), which should correct color techniques alone can result in unnatural looking panoramas, mismatches between images. as seen in Figure 1 (right). In [6], the camera response function is modeled as a gamma curve and vignetting is modeled as cos4(d/f), where d 978-1-4244-5654-3/09/$26.00 ©2009 IEEE 709 ICIP 2009 is the radial distance between the pixel and the center of the the image used as the color reference). We will first image and f is the focal length. In the [7] authors generalize describe the method for correcting two images, and then the camera response model to be a polynomial. describe how it is extended to a greater number of images. An influential method for removing vignetting and Vignetting correction is usually done by dividing the exposure differences is proposed by Goldman and Chen in irradiance by the estimated vignetting attenuation, as in [8], which uses an empirical camera response model equation (2). In order to use a simple linear regression we together with a polynomial model for vignetting. would like to compensate for vignetting with an additive th Specifically a 6 order even polynomial is used: term. Define E0 to be the irradiance that would be measured by the sensor if there were no vignetting (E =eL). The drop = + α 2 + α 4 + α 6 0 V (d ) 1 1 d 2 d 3 d (3) in irradiance due to vignetting using the polynomial model (3) is: In (3), the Į values are weights adjusted to make the polynomial fit the observed vignetting. ΔE = E − E vig 0 (4) Another method based on Goldman and Chen work is = − ()+ α 2 + α 4 + α 6 E0 E0 1 1d 2 d 3 d proposed in [9]. A white balance factor is introduced, so = − ()α 2 + α 4 + α 6 E0 1d 2 d 3 d that I=f(eV(x)Lwc). The optimization problem is reformulated so that a single non-linear optimization is done Since an image pixel value is proportional to the with the Levenberg-Marquardt method rather than the irradiance, we can say that the corresponding drop in the iterative optimization method used in [8]. image sample will be approximately: While methods for removing vignetting and color ΔI = −I(α d 2 +α d 4 +α d 6 ) (5) differences between images do greatly reduce seams, they do vig 1 2 3 not completely remove them, so blending techniques are still Note that the vignetting model of equation (5) would be required [6]-[9]. Even if the color of images could be equivalent to the model in equation (4) if the cameras perfectly matched, blending would still be necessary to response were linear. In image 2, we will only correct disguise small alignment errors and scene movement [2]. vignetting, so applying equation (5) the corrected image is In previous work that corrects for both vignetting and calculated with: exposure differences, non-linear optimization is used [6]-[9]. = + (α 2 + α 4 + α 6 ) Non-linear optimization methods have considerably higher I 2 ,cor I 2 I 2 1 d 2 2 d 2 3 d 2 (6) complexity than linear ones, and therefore may take a long time to run [8], and may have problems with convergence In image 1, we have to correct both for vignetting and [7]. for exposure and white balance differences between images. nd In this paper we proposed a vignetting correction and To do this we propose to use a 2 order polynomial to color matching method that uses only linear least-squares model a transfer function to render a pixel in image 1 with regression. Linear regression has far lower complexity than the exposure and white balance of image 2. Hence the non-linear optimization and has no problems with function we use for correcting vignetting in image 1 and convergence since the global optimum can be determined in making its color match image 2 is: closed form. In order to use simple linear regression, we = + 2 + + (α 2 +α 4 +α 6 ) (7) correct vignetting with an additive term, and match the color I1,cor a1I1 a2I1 a3 I1 1d1 2d1 3d2 between two images with a 2nd order polynomial. The optimal weights ai and Įi that will make the images Experiments show the proposed method effectively corrects consistent will be estimated from matching points in the for vignetting, exposure and white balance differences overlapping region. Vignetting affects all color channels between images. When combined with blending methods equally, so the weights for vignetting should be common to the proposed method gives panoramas with no visible all three channels (R, G, and B), but the other weights are mismatch between images. different for each channel. The rest of this paper is organized as follows. The The corrected values of image 1 calculated with (7) proposed method is described in section 2. Experimental should equal the corrected values of image 2 calculated with results are presented in section 3 and conclusions are given (6). Applying this to the R, G and B channels gives the set in section 4. of equations: 2. PROPOSED METHOD +α 2 +α 4 +α 6 = + 2 + +α 2 +α 4 +α 6 R2 1d2 R2 2d2 R2 3d2 R2 aR1R1 aR2 R1 aR3 1d1 R1 2d1 R1 3d1 R1 G +α d 2G +α d 4G +α d 6G = a G + a G2 + a +α d 2G +α d 4G +α d 6G We assume that we have two images, I1(x,y) and I2(x,y) that 2 1 2 2 2 2 2 3 2 2 G1 1 G 2 1 G3 1 1 1 2 1 1 3 1 1 B +α d 2 B +α d 4B +α d 6 B = a B + a B2 + a +α d 2 B +α d 4 B +α d 6 B have been registered so that an overlapping area has been 2 1 2 2 2 2 2 3 2 2 B1 1 B2 1 B3 1 1 2 1 3 1 1 identified (in our experiments we align images using (8) autostitch [1]).

Load more