Gamut Constrained Illuminant Estimation

Total Page:16

File Type:pdf, Size:1020Kb

Gamut Constrained Illuminant Estimation Gamut Constrained Illuminant Estimation G. D. Finlayson and S. D. Hordley School of Computing Sciences, University of East Anglia, UK I. Tastl Hewlett Packard Incorporated, Palo Alto, USA Abstract. This paper presents a novel solution to the illuminant estimation problem: the problem of how, given an image of a scene taken under an unknown illuminant, we can recover an estimate of that light. The work is founded on previous gamut mapping solutions to the problem which solve for a scene illuminant by determining the set of diagonal mappings which take image data captured under an unknown light to a gamut of reference colours taken under a known light. Unfortunately, a diagonal model is not always a valid model of illumination change and so previous approaches sometimes return a null solution. In addition, previous methods are difficult to implement. We address these problems by recasting the problem as one of illuminant classification: we define a priori a set of plausible lights thus ensuring that a scene illuminant estimate will always be found. A plausible light is represented by the gamut of colours observable under it and the illuminant in an image is classified by determining the plausible light whose gamut is most consistent with the image data. We show that this step (the main computational burden of the algorithm) can be performed simply and efficiently by means of a non-negative least-squares optimisation. We report results on a large set of real images which show that it provides excellent illuminant estimation, outperforming previous algorithms. Keywords: Colour Constancy, Illuminant Estimation, Gamut Mapping 1. Introduction It is well established that colour is a useful cue which helps to solve a number of classical computer vision problems such as object recog- nition [29], object tracking [27] and image segmentation [8]. However, implicit in this use of colour is the assumption that colour is an inherent property of an object. In reality an object’s colour depends in equal measure on the object’s physical properties and the characteristics of the light by which it is illuminated. That is, the image colours recorded by a camera change when the colour of the light illuminating the scene is changed. One way to deal with this problem is to derive so called colour invariant features from the image data which are invariant to the prevailing illumination (for some examples of such an approach see [16, 15, 17]). An alternative approach, which is the focus of this paper, is to correct images to account for the effect of the scene il- luminant on recorded colours. This correction procedure is typically c 2006 Kluwer Academic Publishers. Printed in the Netherlands. FinHorTastl_IJCV05.tex; 27/02/2006; 20:35; p.1 2 achieved in a two-stage process. In the first stage the colour of the scene illumination is estimated from the image data. Then, the estimate of the scene illuminant is used to correct the recorded image such that the corrected image is independent of the scene illuminant. The first stage of the process – estimating the scene illuminant – is the more difficult of the two and it is this problem which is the subject of this paper. Despite much research [22, 6, 24, 14, 9, 7, 10, 2, 3] there does not exist a satisfactory solution to the illuminant estimation problem. Early attempts at a solution sought to simplify the problem by making certain assumptions about the composition of a scene. For example it was assumed that a scene will contain a “white” (maximally reflective) surface [22] or that the average of all surfaces in the scene is neutral [6]. Other authors modelled lights and surfaces by low-dimensional linear models and derived algebraic solutions to the problem [24]. It is easy to understand why such approaches do not work in practice: the con- straints they place on scenes are too strong. More recently a number of more sophisticated algorithms have been developed [14, 7, 10, 5, 28] and these approaches can often give reasonable illuminant estimation. These methods work not by placing constraints on scenes but by exploiting prior knowledge about the nature of objects and surfaces. For example, the Neural Network (NN) approach of Funt et al [7] attempts to learn the relationship between image data and scene illuminant on the basis of a large number of example images. While it performs reasonably well this approach is unsatisfactory because it provides a black box solution to the problem which gives little insight into the problem itself. In addition neural networks are non-trivial to implement and they rely for their success on training data which properly reflects the statistics of the world. In practice neural networks often do not generalise well and this has been found to be the case for illuminant estimation [3]. Perhaps, a more promising approach is the Correlation Matrix (CM) algorithm of Finlayson et al [10] which has the advantage of being simple both in its implementation and in terms of the computations which must be carried out to estimate the illuminant. This method gains part of its simplicity from the observation that the set of possible scene lights is quite restricted and can be adequately represented by considering just a small discrete set of plausible lights. The method works by exploiting the fact that the statistical distribution of possible image colours varies as the scene illuminant changes. Using this fact, and given a set of image data, it is possible to obtain a measure of the probability that each of the set of plausible illuminants was the scene light. Clearly, the approach relies on having a reasonable statistical model of lights and surfaces: (i.e. it requires accurate training data to FinHorTastl_IJCV05.tex; 27/02/2006; 20:35; p.2 3 work in practice) but unfortunately such data is difficult to obtain. A second disadvantage of the method is that it works not with 3-d RGB camera data but with 2-d chromaticity data: i.e. brightness information is discarded. There is evidence to suggest however [4, 30] that brightness information is important in estimating the scene illuminant and while brightness information can be incorporated into the correlation matrix algorithm [4], doing so destroys the simplicity of the approach. A third approach which has also shown promise is the Gamut Map- ping (GM) method first proposed by Forsyth [14]. Gamut Mapping also exploits knowledge about the properties of surfaces and lights to estimate the scene illuminant. However, it is not based on a statistical analysis of the frequency with which surfaces occur in the world but on the simple observation that the set of camera RGBs observable under a given illuminant is a bounded convex set. This observation follows directly from physical constraints on the nature of a surface (all surfaces reflect between 0 and 100% of light incident upon them) and the linear nature of image formation. It therefore exploits the minimal assumptions that can be made about the world. Forsyth exploited this observation by characterising the gamut of possible image colours which can be observed under a reference light. He called this set the canonical gamut. The colours in an image whose scene illuminant it is wished to estimate can also be characterised as a set: the image gamut. Estimating the scene illuminant can then be cast as the problem of finding the mapping which takes the image gamut into the canonical gamut. To properly define the algorithm requires that the form of the mapping between the two sets be specified. That is, the relationship between sensor responses under two different illuminants must be defined. Forsyth modelled illumination change using a diagonal model [14] under which sensor responses for a surface viewed under two different lights are related by simple scale factors which depend on the sensors and the pair of lights but which are independent of the surface. Using this diagonal model of illumination change Forsyth derived the CRULE algorithm which estimates the scene illuminant in two stages. First the set of all mappings (the feasible set) which take the image gamut to the canonical gamut are determined: usually many mappings are consistent with the image data. In the second stage a single mapping is selected from this feasible set according to some pre-defined selection criterion. The algorithm has been found to perform quite well [3] but it suffers from a number of limitations. It is these limitations which we aim to address in this paper. We do this by proposing a new algorithm which shares some of the fundamental ideas of the original work but which is implemented in such a way as to avoid its weaknesses. FinHorTastl_IJCV05.tex; 27/02/2006; 20:35; p.3 4 The first weakness of CRULE is the fact that the algorithm is foun- ded on the assumption that illumination change can be well modelled by a diagonal transform. This leads to an algorithm which is concep- tually simple but which can nevertheless fail. One reason for this is that real images can contain an illuminant or (more realistically) one or more image colours which do not conform to the diagonal model of illumination change and unfortunately in such situations the theoretical foundation of the algorithm implies that there is no illuminant estimate which satisfies all the image data. That is, there does not exist a diag- onal transform that maps the image gamut inside the canonical gamut. While this problem might be ameliorated by carrying out a change of sensor basis (and using a generalised diagonal transform [11]) it does not completely disappear: a linear model of illumination change is only approximate.
Recommended publications
  • Expanded Gamut Shoot-Out: Real Systems, Real Results
    Expanded Gamut Shoot-Out: Real Systems, Real Results Abhay Sharma Click toRyerson edit Master University, subtitle Toronto style Advisors Roger Breton, Marc Levine, John Seymour, Bill Pope Comprehensive Report – 450+ downloads tinyurl.com/ExpandedGamut Agenda – Expanded Gamut § Why do we need Expanded Gamut? § What is Expanded Gamut? (CMYK-OGV) § Use cases – Spot Colors vs Images PANTONE 109 C § Printing Spot Colors with Kodak Spotless (KSS) § Increased Accuracy § Using only 3 inks § Print all spot colors, without spot color inks § How do I implement EG? § Issues with Adobe and Pantone § Flexo testing in 2020 Vendors and Participants Software Solutions 1. Alwan – Toolbox, ColorHub 2. CGS ORIS – X GAMUT 3. ColorLogic – ColorAnt, CoPrA, ZePrA 4. GMG Color – OpenColor, ColorServer 5. Heidelberg – Prinect ColorToolbox 6. Kodak – Kodak Spotless Software, Prinergy PDF Editor § Hybrid Software - PACKZ (pronounced “packs”) RIP/DFE § efi Fiery XF (Command WorkStation) – Epson P9000 § SmartStream Production Pro – HP Indigo 7900 Color Management Solutions § X-Rite i1Profiler Expanded Gamut Tools § PANTONE Color Manager, Adobe Acrobat Pro, Adobe Photoshop Why do we need Expanded Gamut? - because imaging systems are imperfect Printing inks and dyes CMYK color gamut is small Color negative film What are the Use Cases for Expanded Gamut? ✓ 1. Spot Colors 2. Images PANTONE 301 C PANTONE 109 C Expanded gamut is most urgently needed in spot color reproduction for labels and package printing. Orange, Green, Violet - expands the colorspace Y G O C+Y M+Y
    [Show full text]
  • NEC Multisync® PA311D Wide Gamut Color Critical Display Designed for Photography and Video Production
    NEC MultiSync® PA311D Wide gamut color critical display designed for photography and video production 1419058943 High resolution and incredible, predictable color accuracy. The 31” MultiSync PA311D is the ultimate desktop display for applications where precise color is essential. The innovative wide-gamut LED backlight provides 100% coverage of Adobe RGB color space and 98% coverage of DCI-P3, enabling more accurate colors to be displayed on screen. Utilizing a high performance IPS LCD panel and backed by a 4 year warranty with Advanced Exchange, the MultiSync PA311D delivers high quality, accurate images simply and beautifully. Impeccable Image Performance The wide-gamut LED LCD backlight combined with NEC’s exclusive SpectraView Engine deliver precise color in every environment. • True 4K resolution (4096 x 2160) offers a high pixel density • Up to 100% coverage of Adobe RGB color space and 98% coverage of DCI-P3 • 10-bit HDMI and DisplayPort inputs display up to 1.07 billion colors out of a palette of 4.3 trillion colors Ultimate Color Management The sophisticated SpectraView Engine provides extensive, intuitive control over color settings. • MultiProfiler software and on-screen controls provide access to thousands of color gamut, gamma, white point, brightness and contrast combinations • Internal 14-bit 3D lookup tables (LUTs) work with optional SpectraViewII color calibration solution for unparalleled color accuracy A Perfect Fit for Your Workspace A Better Workflow Future-proof connectivity, great ergonomics, and VESA mount Exclusive,
    [Show full text]
  • Accurately Reproducing Pantone Colors on Digital Presses
    Accurately Reproducing Pantone Colors on Digital Presses By Anne Howard Graphic Communication Department College of Liberal Arts California Polytechnic State University June 2012 Abstract Anne Howard Graphic Communication Department, June 2012 Advisor: Dr. Xiaoying Rong The purpose of this study was to find out how accurately digital presses reproduce Pantone spot colors. The Pantone Matching System is a printing industry standard for spot colors. Because digital printing is becoming more popular, this study was intended to help designers decide on whether they should print Pantone colors on digital presses and expect to see similar colors on paper as they do on a computer monitor. This study investigated how a Xerox DocuColor 2060, Ricoh Pro C900s, and a Konica Minolta bizhub Press C8000 with default settings could print 45 Pantone colors from the Uncoated Solid color book with only the use of cyan, magenta, yellow and black toner. After creating a profile with a GRACoL target sheet, the 45 colors were printed again, measured and compared to the original Pantone Swatch book. Results from this study showed that the profile helped correct the DocuColor color output, however, the Konica Minolta and Ricoh color outputs generally produced the same as they did without the profile. The Konica Minolta and Ricoh have much newer versions of the EFI Fiery RIPs than the DocuColor so they are more likely to interpret Pantone colors the same way as when a profile is used. If printers are using newer presses, they should expect to see consistent color output of Pantone colors with or without profiles when using default settings.
    [Show full text]
  • The Printer's Guide to Expanded Gamut
    DISTRIBUTED BY TECHKON USA February 2017 THE PRINTER’S GUIDE TO EXPANDED GAMUT Understanding the technology landscape and implementation approach By Ron Ellis Printer’s Guide to Expanded Gamut Page | 1 Printer’s Guide to Expanded Gamut Whitepaper By Ron Ellis Table of Contents What is Expanded Gamut ............................................................................................................... 4 ......................................................................................................................................................... 5 Why Expanded Gamut .................................................................................................................... 6 The Current Expanded Gamut Landscape ...................................................................................... 9 Standardization and Expanded Gamut ......................................................................................... 10 Methods of Producing Expanded Gamut...................................................................................... 11 Techkon and Expanded Gamut ..................................................................................................... 11 CMYK expanded gamut ................................................................................................................. 12 The CMYK Expanded Gamut Workflow ........................................................................................ 16 Conversion from source to CMYK Expanded gamut ....................................................................
    [Show full text]
  • Monitor Color Gamut What Does “Percent of NTSC” Mean?
    Technical white paper Monitor color gamut What does “percent of NTSC” mean? What is color gamut? With rare exceptions, all LCD monitors operate by presenting just three primary colors to the eye: red, green and blue. All the colors you can see on the monitor are actually different blends of the three primaries. This works because of the way the human eye works. The complete set of colors that a monitor can present is called the “native color gamut”, and is determined by the exact colors of the three primaries. Different monitors have slight (or not so slight) differences in the three primaries. These differences are a consequence of the technologies used in the LCD panel and the backlight, and sometimes of the age of the unit. There are two diagrams commonly used for representing how color gamuts relate to the set of all colors visible to the human eye. These are called “chromaticity” diagrams. The first (left), which is commonly used, is the “CIE 1931” diagram, which uses the coordinates x and y. The second diagram (right), which is newer (and, actually, a better representation of how we perceive color) is the “CIE 1976 UCS” diagram, which uses the coordinates u’ and v’ (called “u prime” and “v prime”). Any three-primary color gamut for a monitor can be plotted on these diagrams as a triangle of which the vertices are the exact colors of the red, green and blue primaries. The NTSC color space There are many standard three-primary color spaces used in the industry. The NTSC color space, defined in 1953, is, however, not commonly used, as no displays at the time had primaries which matched it.
    [Show full text]
  • Comparing Measures of Gamut Area
    Comparing Measures of Gamut Area Michael P Royer1 1Pacific Northwest National Laboratory 620 SW 5th Avenue, Suite 810 Portland, OR 97204 [email protected] This is an archival copy of an article published in LEUKOS. Please cite as: Royer MP. 2018. Comparing Measures of Gamut Area. LEUKOS. Online Before Print. DOI: 10.1080/15502724.2018.1500485. PNNL-SA-132222 ▪ November 2018 COMPARING MEASURES OF GAMUT AREA Abstract This article examines how the color sample set, color space, and other calculation elements influence the quantification of gamut area. The IES TM-30-18 Gamut Index (Rg) serves as a baseline, with comparisons made to several other measures documented in scientific literature and 12 new measures formulated for this analysis using various components of existing measures. The results demonstrate that changes in the color sample set, color space, and calculation procedure can all lead to substantial differences in light source performance characterizations. It is impossible to determine the relative “accuracy” of any given measure outright, because gamut area is not directly correlated with any subjective quality of an illuminated environment. However, the utility of different approaches was considered based on the merits of individual components of the gamut area calculation and based on the ability of a measure to provide useful information within a complete system for evaluating color rendition. For gamut area measures, it is important to have a reasonably uniform distribution of color samples (or averaged coordinates) across hue angle—avoiding exclusive use of high-chroma samples—with sufficient quantity to ensure robustness but enough difference to avoid incidents of the hue-angle order of the samples varying between the test and reference conditions.
    [Show full text]
  • Gamut Analysis XCMYK
    Gamut Analysis XCMYK Kiran Deshpande Multi Packaging Solutions, UK Outline • Background • Gamut metrics for analysis • XCMYK vs. GRACoL2013 CRPC6 • XCMYK vs. FOGRA51 • XCMYK vs. Proofer (Epson9900) • XCMYK vs. 7-color CMYKOGV • Summary Background • XCMYK – new expanded gamut 4-color printing process • Standard ISO 12647-2 compliant CMYK inks can be used • with higher ink film thickness • non-traditional screening e.g. FM • XCMYK dataset and ICC profile – Idealliance1 Gamut Metrics – relative coverage • We often need to compare two or more color gamuts • The absolute difference in gamut volumes alone is a poor indicator • It can't tell if the gamuts intersect sufficiently to meet the reproduction aims • Two gamuts having the same volume may not coincide • Metric needs to include both relative volume and intersection Gamut Comparison Index (GCI) • GCI2 – objective metric to quantify the difference between two gamuts • GCI between two gamuts shows how closely they match – similar to DE V V i i GCI = Vx Vy Vx : gamut volume of the medium x Vy : gamut volume of the medium y Vi : volume of intersection of the two gamuts (Vx ∩ Vy) Gamut Metrics – relative coverage • [Vi / Vx]: how much of gamut x is covered by gamut y • [Vi / Vy]: how much of gamut y is covered by gamut x • [(Vx – Vi ) / Vx]: how much of gamut x is outside the gamut y • [(Vy – Vi ) / Vy]: how much of gamut y is outside the gamut x • [Vx / Vy]: ratio of gamut x to gamut y Gamut Metrics2 Datasets and method • Reference gamut – XCMYK2017IT8 • GRACoL2013_CRPC63 • FOGRA514 • Proofer gamut – example Epson9900 • 7c Expanded Gamut CMYKOGV – Press1 & Press2 • Gamut volume calculation method – Alpha-shapes XCMYK vs.
    [Show full text]
  • Evaluating Color Gamut of Smartphone Display
    Evaluating Color Gamut of Smartphone Display Introduction gamut, but the most common reference spaces used for display products are CIE Yxy (1931) and CIE Lu’v’ With increased image and video content available over (1976) chromaticity chart. the internet, there is now a growing emphasis on color fidelity and accuracy of display used in smartphone. One reason for this growing emphasis is the emergence of online merchandising. Consumers want to be sure that the colors they see on their smartphone display are actually what they will get eventually. Another reason is CIE Yxy (1931) chromaticity chart that we are now accustomed to seeing displays which are able to reproduce colors we see in the real world. Anything less, the smartphone display would be deemed as mediocre. The good news is color fidelity and accuracy of display have improved throughout the years due to better display technology, advanced signal processing and color management solution. To ensure displays are able to portray specific colors accurately, evaluating parameter like color gamut is CIE Lu’v’ (1976) chromaticity chart necessary. Apart from color gamut, parameters like white point and gamma are also of equal importance TThere are various color gamut standards drafted for in evaluating the visual characteristic of display. expressing color display range according to different industry and different application. For smartphone What is Color Gamut? and PC displays, the standard gamut is sRGB/ Rec.709. However, with the growing adoption of wide color Color gamut is defined as the range of colors that a gamut solutions in the high-end display market, wide display can reproduce.
    [Show full text]
  • Gamut Mapping: Evaluation of Chroma Clipping Techniques for Three Destination Gamuts
    Gamut Mapping: Evaluation of Chroma Clipping Techniques for Three Destination Gamuts Ethan D. Montag and Mark D. Fairchild Munsell Color Science Laboratory Chester F. Carlson Center for Imaging Science Rochester Institute of Technology, Rochester, New York Abstract A psychophysical experiment was carried out to evaluate 5 chroma clipping techniques for gamut mapping. Three different destination gamuts were used in order to determine how the shape of the destination gamut would influence performance of the different mapping techniques. The test images consisted of 7 colored computer-rendered textured spheres and one image which contained all 7 objects. Therefore the effect of hue on the preferred technique could be gauged. Although straight chroma-clipping, keeping lightness constant, was overall the best technique, Cube there were notable exceptions. The results indicate that trade- Reduced CRT offs between global trends and specific exceptions to these trends may need to be implemented based on local gamut shape and object hue in the image. Introduction One strategy in color gamut mapping research is to discover algorithms which may be implemented in a "universal" gamut mapping strategy. However, common Printer experience tells us that a gamut mapping algorithm that produces pleasing results for one image may not work well Figure 1. Destination gamuts for another image. This is true for pictorial images as well as business graphics and computer generated images. In our previous research,5 arbitrary gamut boundaries One of the goals of our research involving gamut were set in order to test gamut mapping in three different mapping is to simplify the problem in order to determine cases: lightness mapping, at both low and high values of whether there are some general rules that can be applied to L*, and chroma mapping when C*ab is limited.
    [Show full text]
  • Fix Out-Of-Gamut Images - Hue and Saturation First Off, Did You Read the Out-Of-Gamut Color Page? If Not, You May Want to Read It
    Fix Out-of-Gamut Images - Hue and Saturation First off, did you read the out-of-gamut color page? If not, you may want to read it. Here is a short explanation of what out-of-gamut colors. A gamut is the range of colors that a color device can display or print. A color that may be displayed on your monitor in RGB may not be printable in the gamut of your CMYK printer. For instance, the nice blue on your monitor that prints as purple. Photoshop has its own color management system that uses your printer profiles to automatically bring all colors into gamut that will print to that profileʼs range of colors. But you might want to identify the out-of-gamut colors in an image or correct them manually before printing to your desktop printer or converting to CMYK for a professional print job. Using Hue and Saturation to fix out-of-gamut colors The Hue and Saturation Adjustment command will allow you to desaturate or change a color over an entire image or on a selected range of colors. The dialog box has a color slider triangle, and the familiar; single sample, add to, remove from, eyedropper selection tools, and a drop down box that allows for adjusting individual colors. When adjusting out-of-gamut colors you will want to adjust those colors that are not printable on your selected profile. Remember using an adjustment layer from the Layers window, rather than using Hue and Saturation from the Menu, will create a non-destructive layer to the image.
    [Show full text]
  • Rec. 709 Color Space
    Standards, HDR, and Colorspace Alan C. Brawn Principal, Brawn Consulting Introduction • Lets begin with a true/false question: Are high dynamic range (HDR) and wide color gamut (WCG) the next big things in displays? • If you answered “true”, then you get a gold star! • The concept of HDR has been around for years, but this technology (combined with advances in content) is now available at the reseller of your choice. • Halfway through 2017, all major display manufacturers started bringing out both midrange and high-end displays that have high dynamic range capabilities. • Just as importantly, HDR content is becoming more common, with UHD Blu-Ray and streaming services like Netflix. • Are these technologies worth the market hype? • Lets spend the next hour or so and find out. Broadcast Standards Evolution of Broadcast - NTSC • The first NTSC (National Television Standards Committee) broadcast standard was developed in 1941, and had no provision for color. • In 1953, a second NTSC standard was adopted, which allowed for color television broadcasting. This was designed to be compatible with existing black-and-white receivers. • NTSC was the first widely adopted broadcast color system and remained dominant until the early 2000s, when it started to be replaced with different digital standards such as ATSC. Evolution of Broadcast - ATSC 1.0 • Advanced Television Systems Committee (ATSC) standards are a set of broadcast standards for digital television transmission over the air (OTA), replacing the analog NTSC standard. • The ATSC standards were developed in the early 1990s by the Grand Alliance, a consortium of electronics and telecommunications companies assembled to develop a specification for what is now known as HDTV.
    [Show full text]
  • Standard Colour Spaces
    Artists House t: +44 (0)20 7292 0400 14-15 Manette St f: +44 (0)20 7292 0401 London, W1D 4AP www.filmlight.ltd.uk UNITED KINGDOM Technical Note Standard Colour Spaces Richard Kirk Document ref. FL-TL-TN-0417-StdColourSpaces Creation date January 9, 2004 Last modified 30 November 2010 Version no. 4.0 Summary In 1931, the Commission Internationale d'Éclairiage (CIE) recommended a system for colour measurement. This system allowed the specification of colour matches using the CIX XYZ tristimulus values. In 1976, the CIE recommended the CIE LAB and CIE LUV colour spaces for the measurement of colour differences, and colour tolerances. These colour spaces, and their more modern variants are the basic tools for modern colorimetry. You can use Truelight without knowing all about CIE colour spaces. However, if you wonder why the XYZ and the L*a*b* calibration for the same monitor look different, you may find your explanation here. Whites are dealt with in a separate section. Most of us know what white paper and white paint is. We might think we know what white light is too. A full discussion of what is and what is not 'white' is much too big a task for this small note, but we introduce a few basic issues. Scanners and printers usually communicate in their own device-dependent RGB. Video has its own colour standard. Densitometers use Status A and Status M colour spaces. We describe all the spaces that Truelight uses to build up a colour transform. Finally we describe a number of effects that are not covered by the standard colour models.
    [Show full text]