Color Balancing of Digital Photos Using Simple Image Statistics
Total Page:16
File Type:pdf, Size:1020Kb
Pattern Recognition 37 (2004) 1201–1217 www.elsevier.com/locate/patcog Color balancing of digital photos using simple image statistics Francesca Gasparini , Raimondo Schettini∗ DISCO (Dipartimento di Informatica, Sistemistica e Comunicazione), Universita degli Studi di Milano-Bicocca, Ediÿcio 7, Via Bicocca degli Arcimboldi 8, Milano 20126, Italy Received16 May 2003; accepted19 December 2003 Abstract The great di.usion of digital cameras and the widespread use of the internet have produced a mass of digital images depicting a huge variety of subjects, generally acquired by unknown imaging systems under unknown lighting conditions. This makes color balancing, recovery of the color characteristics of the original scene, increasingly di2cult. In this paper, we describe a methodfor detectingandremoving a color cast (i.e. a superimposedcolor dueto lighting conditions,or to the characteristics of the capturing device), from a digital photo without any a priori knowledge of its semantic content. First a cast detector, using simple image statistics, classiÿes the input images as presenting no cast, evident cast, ambiguous cast, a predominant color that must be preserved(such as in underwaterimages or single color close-ups) or as unclassiÿable. A cast remover, a modiÿed version of the white balance algorithm, is then applied in cases of evident or ambiguous cast. The method we propose has been testedwith positive results on a dataset of some 750 photos. ? 2003 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved. Keywords: Color constancy; Cast detection; Cast removal; Von Kries; White balance; Gray world algorithm 1. Introduction of designing reliable automatic tools that can improve the overall quality of digital photographs pragmatically by de- The great di.usion of digital cameras and the widespread signing a modular enhancing procedure (Fig. 1). Each mod- use of the internet have produced a mass of digital im- ule can be considered as an autonomous element, that can ages depicting a huge variety of subjects, generally acquired be combinedin more or less complex algorithms. by non-professional photographers using unknown imaging This paper focuses on unsupervisedcolor correction. The systems under unknown lighting conditions. The quality of problem is that of automatically andreliably removing a these real-worldphotographs can often be considerablyim- color cast (a superimposed color due to lighting conditions, provedby digitalimage processing. At present color and or to the characteristics of the capturing device). The solu- contrast corrections are usually manually performedwithin tion we have designed exploits only simple image statistics the framework of speciÿc software packages. Since these to drive the color procedure, preventing artifacts. interactive processes may prove di2cult and tedious, espe- Section 2 illustrates the problem addressed, and the related cially for amateur users, an automatic image enhancement methods available in the literature. In Section 3, we describe tool wouldbe most desirable. the automatic procedure for color balancing, which we have Typical image properties requiring correction are color, structuredin two main parts: a cast detectoranda cast re- contrast andsharpness. We have approachedthe open issues mover. The detector, described in Section 3.1, classiÿes the input images with respect to their chromaticity as: (i) no-cast images, (ii) evident cast images, (iii) ambiguous cast images ∗ Corresponding author. Tel.: +39-02-6448-7840; fax: +39-02- (images with a weak cast, or for which whether or not a cast 6448-7839. exists is a subjective opinion), (iv) images with a predomi- E-mail addresses: [email protected] (F. Gasparini), nant color which must be preserved(such as in underwater [email protected] (R. Schettini). images, or single color close-ups), (v) unclassiÿable. 0031-3203/$30.00 ? 2003 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.patcog.2003.12.007 1202 F. Gasparini, R. Schettini / Pattern Recognition 37 (2004) 1201–1217 Input Color Contrast Edge Correction Output image Enhancement Sharpening image Fig. 1. Our enhancement chain constituted by independent enhancement modules. The remover, a modiÿed version of the white balance that require a stable perception of an object’s color, as in algorithm [1,2], pilotedby the class of the cast, is applied object recognition, image retrieval, or image reproduc- to images labeledas having evidentor ambiguous cast. This tion, e.g. Ref. [4]. step is described in Section 3.2. Our experimental results are reportedandcommentedin Section 4, while our conclusions Color constancy is an under-determined problem and thus are presentedin Section 5. impossible to solve in the most general case [1]. Several strategies are proposedin the literature. These, in general, require some information about the camera being used, and are basedon assumptions about the statistical properties of 2. Related work the expectedilluminants andsurface reJectances. From a computational perspective, color constancy is a two-stage Automatic color correction is a challenging issue as the process: the illuminant is estimated, and the image colors RGB images recorded by an imaging device depend on the are then correctedon the basis of this estimate. The correc- following elements: tion generates a new image of the scene as if taken under a known standard illuminant. The color correction step is 1. The response of each color channel as a function of usually based on a diagonal model of illumination change, intensity, usually modeled as a gamma correction. For deriving from the Von Kries hypothesis that color constancy generic images the channel responses of the imaging de- is an independent gain regulation of the three cone signals, vice are usually unknown. Cardei et al. [2,3] have shown through three di.erent gain coe2cients [5]. This diagonal that gamma-on images (i.e. images for which gamma model is generally a good approximation of change in illu- does not equal unity) can be color corrected without ÿrst mination, as demonstrated by Finlayson et al. in Ref. [6]. linearizing them. Consequently, it is common practice to Shouldthe modelleadto large errors, its performance can address color and gamma corrections independently. still be improvedwith sensor sharpening [ 7,8]. In formula 2. The device white balancing, usually set by the device the Von Kries hypothesis can be written as manufacturer so that it produces equal RGB values for a white patch under some chosen illuminant. Images of un- L kL 00 L known color balance can still be corrected, but this poses an additional challenge to the color correction algorithm. M = 0 kM 0 M ; (1) 3. The relative spectral sensitivities of the capturing de- S 00 kS S vice as a function of wavelength. This information is of- ten not available (e.g. for images downloaded from the where L, M, S and L, M , S are the initial and Internet), and sensitivities can di.er signiÿcantly for dif- post-adaptation cone signals and kL;M;S are the scaling co- ferent cameras. As Cardei et al. [2,3] have pointedout, e2cients [5]. The scaling coe2cients can be expressedas while two di.erent camera models balanced for the same the ratio between the cone responses to a white under the illuminant will have by deÿnition the same response to reference illuminant andthose of the current one. A typical white, they may have di.erent responses to other colors. reference illuminant, which is also the one we have used 4. The surface properties of the objects present in the scene here, is the D65 CIE standard illuminant [9]. In practical depicted and the lighting conditions (lighting geometry situations the L, M, S retinal wavebands are transformed andilluminant color). Unlike human vision, imaging de- into CIE XYZ tristimulus values by a linear transformation, vices (such as digital cameras) can not adapt their spec- or approximatedby image RGB values [ 12]. tral responses to cope with di.erent lighting conditions. Whatever the features usedto describethe colors, we must As a result, the acquiredimage may have a cast, i.e. an have some criteria for estimating the illuminant andthus undesirable shift in the entire color range. The ability of the scaling coe2cients in Eq. (1). The gray worldalgorithm the human visual system to discount the illuminant, ren- assumes that, given an image of su2ciently variedcolors, dering the perceived colors of objects almost independent the average surface color in a scene is gray [10]. This means of illumination is called color constancy. This wouldbe that the shift from gray of the measuredaverages on the a useful property in any vision system performing tasks three channels corresponds to the color of the illuminant. F. Gasparini, R. Schettini / Pattern Recognition 37 (2004) 1201–1217 1203 The three scaling coe2cients in Eq. (1) are therefore set to in order to produce an image where the white objects ap- compensate this shift. pear white. Moreover, an adhoc adjustmentis commonly The white balance algorithm, instead, looks for a white made to force an image’s maximum luminance value to patch in the image, the chromaticity of which will then be pure white. Some cameras also adjust the minimum lumi- the chromaticity of the illuminant. The white patch is eval- nance to set the black point at a certain energy level. The uatedas the maximum foundin each of the three image movement of the white andblack point regions of the im- bands