Single-Image Vignetting Correction Yuanjie Zheng∗ Stephen Lin Sing Bing Kang Shanghai Jiaotong University Microsoft Research Asia Microsoft Research Abstract To determine the vignetting effects in an image, the most straightforward approach involves capturing an image In this paper, we propose a method for determining the completely spanned by a uniform scene region, such that vignetting function given only a single image. Our method brightness variations can solely be attributed to vignetting is designed to handle both textured and untextured regions [12, 1, 6, 14]. In such a calibration image, ratios of inten- in order to maximize the use of available information. To sity with respect to the pixel on the optical axis describe the extract vignetting information from an image, we present vignetting function. Suitable imaging conditions for this ap- adaptations of segmentation techniques that locate image proach, however, can be challenging to produce due to un- regions with reliable data for vignetting estimation. Within even illumination and camera tilt, and the vignetting mea- each image region, our method capitalizes on frequency surements are valid only for images captured by the camera characteristics and physical properties of vignetting to dis- under the same camera settings. Moreover, a calibration tinguish it from other sources of intensity variation. The image can be recorded only if the camera is at hand; con- vignetting data acquired from regions are weighted accord- sequently, this approach cannot be used to correct images ing to a presented reliability measure to promote robustness captured by unknown cameras, such as images downloaded in estimation. Comprehensive experiments demonstrate the from the web. effectiveness of this technique on a broad range of images. A vignetting function can alternatively be computed from image sequences with overlapping views of an arbi- trary static scene [5, 9, 4]. In this approach, point corre- spondences are first determined in the overlapping image 1. Introduction regions. Since a given scene point has a different position in each image, its brightness may be differently attenuated Vignetting refers to the phenomenon of brightness atten- by vignetting. From the aggregate attenuation information uation away from the image center, and is an artifact that from all correspondences, the vignetting function can be ac- is prevalent in photography. Although not objectionable to curately recovered without assumptions on the scene. the average viewer at low levels, it can significantly impair These previous approaches require either a collection computer vision algorithms that rely on precise intensity of overlapping images or an image of a calibration scene. data to analyze a scene. Applications in which vignetting However, often in practice only a single image of an arbi- distortions can be particularly damaging include photomet- trary scene is available. The previous techniques gain in- ric methods such as shape from shading, appearance-based formation for vignetting correction from pixels with equal techniques such as object recognition, and image mosaic- scene radiance but differing attenuations of brightness. For ing. a single arbitrary input image, this information becomes Several mechanisms may be responsible for vignetting challenging to obtain, since it is difficult to identify pixels effects. Some arise from the optical properties of camera having the same scene radiance while differing appreciably lenses, the most prominent of which is off-axis illumina- in vignetting attenuation. tion falloff or the cos4 law. This contribution to vignetting results from foreshortening of the lens when viewed from In this paper, we show that it is possible to correct or increasing angles from the optical axis [7]. Other sources reduce vignetting given just a single image. To maximize of vignetting are geometric in nature. For example, light ar- the use of available information in the image, our tech- riving at oblique angles to the optical axis may be partially nique extracts vignetting information from both textured obstructed by the field stop or lens rim. and untextured regions. Large image regions appropriate for vignetting function estimation are identified by pro- ∗This work was done while Yuanjie Zheng was a visiting student at posed adaptations to segmentation methods. To counter the Microsoft Research Asia. adverse effects of vignetting on segmentation, our method 2.1. Kang-Weiss model We consider an image with zero skew, an aspect ratio of 1, and principal point at the image center with image coor- dinates (u, v)=(0, 0). In the Kang-Weiss vignetting model [6], brightness ratios are described in terms of an off-axis illumination factor A, a geometric factor G, and a tilt factor T . For a pixel i at (ui,vi) with distance ri from the image center, the vignetting function ϕ is expressed as τ χ ϕ = A G T = ϑ T i =1···N, Figure 1. Tilt angles and in the Kang-Weiss vignetting model. i i i i ri i for (1) where 1 Ai = , 2 2 iteratively re-segments the image with respect to progres- (1 + (ri/f) ) sively refined estimates of the vignetting function. Addi- Gi =(1− α1ri), tionally, spatial variations in segmentation scale are used ϑ = A G , in a manner that enhances collection of reliable vignetting ri i i 3 data. tan τ Ti =cosτ 1+ (ui sin χ − vi cos χ) . (2) In extracting vignetting information from a given region, f we take advantage of physical vignetting characteristics to N is the number of pixels in the image, f is the effective diminish the influence of textures and other sources of in- focal length of the camera, and α1 represents a coefficient tensity variation. With the joint information of disparate in the geometric vignetting factor. The tilt parameters χ, τ image regions, we describe a method for computing the vi- respectively describe the rotation angle of a planar scene gnetting function. The effectiveness of this vignetting cor- surface around an axis parallel to the optical axis, and the rection method is supported by experiments on a wide vari- rotation angle around the x-axis of this rotated plane, as ety of images. illustrated in Fig. 1. The model ϕ in Eq. (1) can be decomposed into the global vignetting function ϑ of the camera and the natural 2. Vignetting model attenuation caused by local tilt effects T in the scene. Note that ϑ is rotationally symmetric; thus, it can be specified as a Most methods for vignetting correction use a paramet- 1D function of the radial distance ri from the image center. ric vignetting model to simplify estimation and minimize the influence of image noise. Typically used are empirical 2.2. Extended vignetting model models such as polynomial functions [4, 12] and hyperbolic In an arbitrary input image, numerous regions with dif- cosine functions [14]. Models based on physical consider- ferent local tilt factors may exist. To account for multiple ations include that of Asada et al. [1], which accounts for surfaces in an image, we present an extension of the Kang- off-axis illumination and light path obstruction, and that of Weiss model in which different image regions can have dif- Kang and Weiss [6] which additionally incorporates scene- ferent tilt angles. The tilt factor of Eq. (2) is modified to based tilt effects. Tilt describes intensity variations within a tan τ 3 scene region that are caused by differences in distance from si Ti =cosτs 1+ (ui sin χs − vi cos χs ) , (3) the camera, i.e., closer points appear brighter due to the in- i f i i verse square law of illumination. Although not intrinsic to where si indexes the region containing pixel i. the imaging system, the intensity attenuation effects caused We also extend the linear geometric factor to a more gen- by tilt must be accounted for in single-image vignetting es- eral polynomial form: timation. p Gi =(1− α1ri −···−αpr ), (4) Besides having physically meaningful parameters, an i important property of physical models is that their highly where p represents a polynomial order that can be arbitrarily structured and constrained form facilitates estimation in set according to a desired precision. This generalized rep- cases where data is sparse and/or noisy. In this work, we use resentation provides a closer fit to the geometric vignetting an extension of the Kang-Weiss model, originally designed effects that we have observed in practice. In contrast to us- for a single planar surface of constant albedo, to multiple ing a polynomial as the overall vignetting model, represent- surfaces of possibly different color. Additionally, we gen- ing only the geometric component by a polynomial allows eralize its linear model of geometric vignetting to a polyno- the overall model to explicitly account for local tilt effects mial form. and global off-axis illumination. Figure 3. Overview of vignetting function estimation. optimizing the underlying global vignetting parameters f, α1, ···,αp. With the estimated parameters, the vignetting corrected z /ϑ image is then given by i ri . We note that the estimated local tilt factors may contain other effects that can appear similar to tilt, such as non-uniform illumination or shad- ing. In the vignetting corrected image, these tilt and tilt-like factors are all retained so as not to produce an unnatural- looking result. Only the attenuation attributed to the imag- ing system itself (off-axis illumination and geometric fac- tors) is corrected. In this formulation, the scene is assumed to contain some Figure 2. Vignetting over multiple regions. Top row: without and piecewise planar Lambertian surfaces that are uniformly il- with vignetting for a single uniform region. Bottom row: without luminated and occupy significant portions of the image. Al- and with vignetting for multiple regions. though typical scenes are considerably more complex than uniform planar surfaces, we will later describe how vi- gnetting data in an image can be separated from other in- 2.3.