sign in sign up
<<
Home , Gamut, Grayscale, High color

IEEE CG&A SPECIAL ISSUE:SMART DEPICTION FOR VISUAL COMMUNICATION 1 Detail Preserving Reproduction of Images for Monochromats and Dichromats Karl Rasche, Robert Geist, James Westall

Abstract— In spite of the ever increasing prevalence of low- algorithm is mapping to . Mapping to luminance, Y, cost, color devices, printing devices remain for the NTSC model is given by: in widespread use. In its current grant proposal guide, the US National Science Foundation warns investigators that they Y = 0.299 × R2.2 + 0.587 × G2.2 + 0.114 × B2.2 (1) “should not rely on colorized objects to make their arguments.” Authors producing documents with color images must account Here the gamma value for the model is 2.2, and the coefficients for the possibility that color images may be reduced to grayscale are derived from the values for the primaries before they are viewed and that the standard reduction used by grayscale printers, map color to luminance, may fail to as well as the reference , which for this model is CIE preserve important image detail. This paper suggests a new Illuminant C, designed to represent daylight. Chromaticity color to grayscale transformation that preserves image detail values are defined in terms of the CIE XYZ of by maintaining distance ratios during the reduction. Multiple the standard observer. The XYZ color space was designed in examples attest to its effectiveness. An extension of the new 1931 by the CIE for matching the perceived color of any given transformation that reduces three-dimensional color space to a two-dimensional surface is also suggested. Such reductions are spectral energy distribution. Three primaries, X, Y, and Z, important for color-deficient observers. and three associated matching functions, x¯λ , y¯λ , and z¯λ , were identified with the property that, given any spectral energy Index Terms— color-deficient vision, , conjugate- λ gradient optimization, grayscale transform distribution, P( ), the non-negative weights

X = k P(λ)x¯λ dλ Y = k P(λ)y¯λ dλ Z = k P(λ)z¯λ dλ I. INTRODUCTION Z Z Z can be used to construct a color, c = XX+YY+ZZ, that will N spite of the ever increasing prevalence of low-cost, match P(λ) up to human perception. The scaling constant, k, devices, grayscale printing devices remain I depends on the spectrum of the reference white. Chromaticity in widespread use. In its current grant proposal guide, the US values are then two-dimensional, normalized coordinates (x,y) National Science Foundation warns investigators: “For cost given by and technical reasons, the Foundation cannot, at this time, X Y reproduce proposals containing color. PIs, therefore, generally x = y = X +Y + Z X +Y + Z should not rely on colorized objects to make their arguments.” Authors producing documents with color images for any venue Different color models, e.g. the sRGB model commonly must account for the possibility that the color images may be used to describe color monitors, with different , reduced to grayscale before they are viewed. Since conversion reference white, or gamma, will yield mappings to luminance to grayscale reduces the number of color dimensions, some that differ slightly from (1). Details on color space definitions loss of visual information is generally unavoidable. Ideally, may be found in [1]. this loss can be restricted to features that vary minimally When mappings such as (1) are used, that have within the . Nevertheless, with standard procedures very small differences in luminance but large differences in in widespread use, this objective is not often achieved, and are mapped to similar . Colors important image detail is often lost. Consequently, algorithms that were easily distinguishable in the original image, due to that convert color images to grayscale in a way that preserves chrominance variation, can become indistinguishable in the information remain important. grayscale image. An example is shown in Figure 1. The left A grayscale conversion algorithm is a dimension-reducing image is an (R,G,B) digital photograph. The center image is function that maps points in color space, typically RGB produced by applying (1) to produce an encoded luminance, Y, tristimulus values, (R,G,B) ∈ ℜ3, to a subset of ℜ1. Any such and then linearly mapping the Y values to gray levels. mapping assumes a , which includes specification The berries, which stand out markedly from the of chromaticity values for red, green, and primaries leaves in the color image, are effectively lost in the gray image. as well as a reference white spectrum and a linearizing A more dramatic, if contrived, example is shown in Figure 2, gamma value. The most common RGB model is probably the where the colored blocks in the original image (left) were NTSC standard, and the most common grayscale conversion chosen to have nearly identical luminance. All image detail is lost when the luminance mapping (1) is applied. The authors are with Clemson University, Clemson, SC 29634-0974 An effective grayscale conversion method should preserve Email: {rkarl|rmg|westall}@cs.clemson.edu the detail in the original color image. Our fundamental premise is that this is best achieved by a perceptual match of relative IEEE CG&A SPECIAL ISSUE:SMART DEPICTION FOR VISUAL COMMUNICATION 2

(a) (b) (c)

Fig. 1. Conversion to Grayscale: Substantial image detail is lost when the color photo (a) is converted to the grayscale photo (b) by the standard NTSC map to luminance. An alternative mapping (c) preserves some of the detail. color differences between the color image and the grayscale error-diffusion [3], but such diffusion does not compensate image. In particular, the perceived between for a problematic . When a standard error diffusion any pair of colors should be proportional to their perceived mapping is applied to the color image of Figure 1(a), the result gray difference. is visually indistinguishable from the image of Figure 1(b). In this paper we describe a new approach to the grayscale Color mapping for printing with a small set of conversion problem that is built on this premise. The method is similar to our problem. Here, colors are reproduced using 3 1 automatically constructs a linear mapping from ℜ to ℜ only one or two base colors. The entire color gamut of an that preserves, to the extent possible, the perceived distances image to be printed is generally not available, and an image 3 between those points in ℜ that comprise a specific color must be mapped into the available gamut in a meaningful 1 image and their mapped values in ℜ . The derived mapping way. Stollnitz et al. [4] examine the case of printing with function depends upon the characteristics of the input image an arbitrary number of base ink colors. For the case of a and incorporates information from all three dimensions of the single ink, they seek to preserve luminance by using color space. After describing our detail-preserving grayscale a monotonic mapping of input luminance to ink density. This transform, we show that a straightforward extension can be method ignores chrominance of the input image and thus will used to re-color images in a way that preserves detail for suffer the same loss of chrominance-based detail that (1) does. viewers with color-deficient vision. The reduction of high dimensional data to a lower dimen- sion is also a well studied problem. A standard linear technique A. Related Work is principal component analysis (PCA). In PCA, a set of This problem is similar to the tone mapping problem of orthogonal vectors, the principal components, lying along the displaying high dynamic range images on low dynamic range directions of maximal data variation is found. To reduce the displays [2]. In both cases, the objective is to preserve an dimensions of the data, a subset of the principal components abundance of visual information within the constraints of a are used to form a subspace onto which the original data is limited gamut, and our approach to grayscale conversion, like projected. While preserving the color variation is necessary for Tumblin and Rushmeier’s tone mapping, seeks a perceptual pleasing grayscale images, it is not sufficient. Consider again match. Nevertheless, tone mapping is generally concerned with the example color image in Figure 1. If we have an image compression of the gamut range, whereas we are interested in with most of the colors clustered in two regions, here red and compression of gamut dimensionality. green, we can envision a “dumbbell”-shaped distribution of The problem might also be compared to color quantization, in color space. By preserving variance, one end of the as it attempts to display a large number of colors with a “dumbbell” will be mapped to the end of the grayscale small palette. Nevertheless, palette selection is an integral gamut while the other will be mapped to the dark end. If part of color quantization methods. Palette colors are most we then examine the histogram of the resulting grayscale often selected from a large gamut, which at least includes the image, we will find a large number of empty gray bins in gamut of the original image. In our problem, the palette is the center of the range. As such, any detail within the red fixed to a gray ramp, and so the principal task, as well as and green “dumbbell” clusters will be reproduced over a very the flexibility available, for any color quantization method has small gray range and may not be perceived. Figure 3 shows been removed. Mapping image colors to the selected palette the projection of the colored photograph of Figure 1 along in color quantization methods is usually a variation on pixel the first two principal components in CIELab space. CIELab IEEE CG&A SPECIAL ISSUE:SMART DEPICTION FOR VISUAL COMMUNICATION 3

(a) (b) (c)

Fig. 2. Conversion to Grayscale: All image detail is lost when the color image on the left is converted to the grayscale image in the middle by the standard NTSC map to luminance. An alternative mapping (right) preserves some of the detail.

not preserved, and so the end result is dependent upon which triples are selected. We would like to preserve proportional differences between all pairs of points. Another class of dimensional reduction techniques attempts to find a nonlinear transformation of the data into the lower dimensional space. Examples of these methods include local linear embedding (LLE) [5] and ISOMAP [6]. These trans- forms work well with data that has some inherent (though perhaps complex) parameterization. They attempt to maintain local distances, corresponding to the original data lying along a higher dimensional manifold. As such, they generally require a “magic number” defining the size of the neighborhood of (a) (b) local distances to observe, and the results can be somewhat sensitive to the neighborhood size. Further, as they only work Fig. 3. Principal Components: Projection of the image of Figure 1 along the first principal component in CIELab space (a) and along the second principal at a local level, it is not clear whether they can reproduce component (b) both contrast and detail when reducing the dimension of the colors. It is also not clear that the colors from an arbitrary image lie along a single manifold. Saul and Roweis examined color space is an alternative three-dimensional (L,a,b) color the performance of LLE with a “dumbbell”-shaped data set model aimed at perceptual uniformity. That is, pairs of colors and achieved results no better than with PCA. at equal measured distances from one another in CIELab space are perceived to be at approximately equal distances from one another. In this model, the L component is luminance, the a B. Deficiencies component is a red-green opponent value, and the b component Deficiencies in color vision arise from differences in pig- is a blue- opponent value. Details may be found in [1]. mentation of optical photoreceptors [7]. With normal vision, In the first principal component, the red berries contrast with there are three distinct pigmentations of cones, the photore- the green leaves, as expected. Nevertheless, significant detail ceptors which contribute to color vision. Anomalous trichro- in the leaves is found from the projection along the second matopia is a condition in which the in one cone is not principal component. A PCA-based approach would appear to sufficiently distinct from the others. The viewer still has three require optimization between contrast and detail by mixing distinct spectral sensitivities, but the separation is reduced. principal components. Dichromatopia results when the viewer has only two distinct An alternative class of methods for dimensional reduction in the cones. For both dichromatopia and anomalous can be characterized as triangulation techniques, in that se- trichromatopia, there are three subclassifications depending lected sets of point triples are mapped to a lower dimensional on which cone has the abnormal pigmentation. Deficiencies space in a way that maintains exact distances within each in cones sensitive to long are referred to as triple. While this distance-preserving feature is attractive for protanopic, while deficiencies in those sensitive to medium and preserving detail in our problem, distances across triples are short wavelengths are referred to as deuteranopic or tritanopic, IEEE CG&A SPECIAL ISSUE:SMART DEPICTION FOR VISUAL COMMUNICATION 4

respectively. Protanopic and deuteranopic deficiencies, the formal terms, for each pair of colors, C~u and C~v, we wish to most common forms of color-deficient vision, are character- satisfy ized by difficulty distinguishing between red and green tones. kC~ −C~ k kT(C~ ) − T(C~ )k u v = u v (2) Tritanopic deficiencies are associated with confusion between Crange Trange blue and yellow tones. Monochromatism is another form of color-deficient vision, but it is very rare. where T(~x) is the grayscale mapping function, Crange is Significant work has been done in simulating color deficient the maximum distance between any pair of colors in the vision [8]–[11]. In these simulations, colors are transformed image, Trange is the maximum distance between any pair of into a color space based on cone response. From this color transformed colors, and k − k is a perceptual color difference space, a cone deficiency can be simulated by collapsing one metric. We use the CIE94 color difference [14], as it is much dimension to a constant value. From empirical data, the value more efficient to compute than the CIEDE2000 color differ- to which the deficient cone dimension should be collapsed can ence. CIE94 is a weighted Euclidean distance in LCH(ab) be determined. (luminance, chroma, ) color space, which is closely related Walraven and Alferdinck [11] describe a color editor for to CIELab color space: simulating a color-deficient view of a color palette. Their L = L; C = a2 + b2; H = tan−1(b/a); editor determines pairs of colors in the palette whose perceived differences in a deficient color space are smaller than a critical Note that grayscale in eitherp CIELab space or LCH(ab) space threshold. If a difference is too small, the color pair is deemed is given by {(L,0,0)|L ≥ 0}. indistinguishable to a potential color-deficient observer, and a A measure of the global error of the transformation is then different pair of colors should be used in the design. The editor 2 can assist by selecting a default palette for which all distances kC~ −C~ k kT(C~ ) − T(C~ )k ε2 i j i j meet the threshold criterion. = ∑ ∑ − (3) i j=i+1 Crange Trange ! Daltonization is a procedure for recoloring an image for viewing by a color-deficient viewer. Details of the method are where i and j range over image pixels, rather than distinct unpublished, but some discussion and an online demonstration colors, to account for the with which each color are available at http://www.vischeck.com. In this method, con- occurs. This is similar to the error term for Multidimensional trast between red and green is stretched and used to mod- Scaling (MDS), except that the differences are normalized over ulate luminance and contrast between blue and yellow hues. different ranges. Our objective is to minimize this total error. Three user-specified parameters are required, one for stretch- For transformation to grayscale, we restrict our attention to ing red-green contrast, one for modulation of luminance, and linear transformations, T, within CIELab color space, in which one for modulation of blue-yellow contrast. Views of the case T(C~ ) = (~g ·C~ ,0,0) for some vector, ~g, to be determined. original and recolored images for a simulated deuteranope Equation (3) is then: are provided. There is no automatic determination of optimal ~ ~ ~ ~ 2 values for the required parameters, but three sets of parameters, kCi −Cjk k(~g · (Ci −Cj),0,0)k ε2(~g) = ∑ ∑ − labeled “low,” “medium,” and “high” (correction) are offered. i j=i+1 Crange Trange ! The results are highly dependent upon the parameters. ~ ~ ~ ~ 2 Ishikawa et al [12] describe manipulation of webpage color kCi −Cjk |~g · (Ci −Cj)| = ∑ ∑ − (4) for color-deficient viewers. They first decompose the page i j=i+1 Crange Trange ! into a hierarchy of colored regions. These spatial relations are used to determine important pairs of colors to be modified. From the CIE94 distance metric and a color image, Crange An objective function is then chosen to maintain distances can computed directly, whereas Trange depends upon ~g. Nev- between pairs of colors, as well as minimize the extent of ertheless, when producing monochromatic images for a fixed color remapping. This attempts to preserve both the original display range, we can set Trange to the range of producible “naturalness” of the colors and the detail in the remapped color gray values. image. The objective function is minimized using a genetic To determine ~g, we minimize the error in (4) using the algorithm. The authors have extended this method to full- well-known, Fletcher-Reeves conjugate-gradient method. Such color images [13]. Here, they first quantize the image to a methods only ensure local minima, and so the choice of small number of colors and construct a hierarchy of like- an initial position can affect the results. We have found, colored regions. As before, a fitness function, designed to experimentally, that an initial position that selects luminance, preserve detail and minimize the distance between an input ~g = (1,0,0), usually produces the best results. With this color and its corresponding remapped color, is minimized with starting position, we obtained the images of Figure 1(c) an evolutionary algorithm. and Figure 2(c). We conjecture that viewer expectations for grayscale images include a bias toward luminance maps and that starting from this position provides the desired contrast II. MONOCHROMATIC REPRODUCTION without straying too far from expected values. As described in Section I, our fundamental premise dictates As an example of an alternative choice, Figure 4 shows, that the perceived color difference between any pair of colors for the color image of Figure 1, the results of using the first should be proportional to their perceived gray difference. In principal component as the initial position. IEEE CG&A SPECIAL ISSUE:SMART DEPICTION FOR VISUAL COMMUNICATION 5

TABLE I Endpoints: SPECTRAL ENERGIES THAT, WHEN PROJECTED TO uv CHROMATICITY SPACE, FORM ENDPOINTS OF THE REDUCED, 1-DIMENSIONAL CHROMATICITY SPACE ACTUALLY SEEN BY COLOR-DEFICIENT VIEWERS

deficiency energy 1 (nm) energy 2 (nm) protanopic 473 574 deuteranopic 477 578 tritanopic 490 610

a pair of distinct colors, minimizing the distance between the original and the remapped colors can be detrimental to maintaining adequate contrast. This is clearly the case if Fig. 4. Initial Positions: The grayscale transform found when beginning at the original colors are nearly indistinguishable to the color- the first principal component deficient viewer. Unlike Daltonization, our method provides recolored images without user intervention. For dichromatic reproduction, T(~x) in (3) is a composition The time required to evaluate (3) for each iteration of the of two transformations. First, a linear transformation in homo- Fletcher-Reeves optimization can become excessive for images geneous CIELab coordinates, G : ℜ4 → ℜ3, is applied to warp with a large number of colors. Our test images often had the original color distribution. Note that G is of the form between 10,000 and 100,000 colors. Nevertheless, we have found that it is not necessary to evaluate the error function over g1 g2 g3 g4 the entire set of colors. We perform a simple equal-volume g g g g G =  5 6 7 8 , binning of colors, replacing each bin with its mean, to reduce g9 g10 g11 g12  0 0 0 1  the number of colors in each image to approximately 1,000.   When evaluating (3) with a reduced color image, care must and so we ultimately search a 12-dimensional space. Non- be taken to weight each contribution by the number of colors zero values of g , g , and g allow for color translations. ~ ~ 4 8 12 that map to the reduced Ci and Cj. The warped colors are then input to a simulator of a color- With this color reduction, the method is relatively fast. deficient viewer. From the simulator output, the perceived On a 2.8GHz Xeon PC, the optimization of Figure 1(a) distance between the simulated deficient pairs of colors can required 2.96 seconds for a reduced color image containing be computed as before. 317 colors. For another version of the same image, containing In order to simulate a color-deficient viewer, we have 11,231 colors, the optimization required 1931.44 seconds. The followed the procedure described by Meyer and Greenberg resulting grayscale images for these two tests were visually [8]. As noted there, empirical studies on individuals who have indistinguishable. Figure 2(a) contains only 43 colors. Opti- normal vision in one eye and color-deficient vision in the other mization required 0.06 seconds. suggest that the two-dimensional hue values of normal vision It should also be noted that there is not an explicit constraint are reduced to a single dimension that is linear in terms of on the direction of ~g. Use of ~g or −~g can yield vastly different CIE Uniform Chromaticity Scale coordinates: results. If the wrong sign is chosen, the image will resemble 4X 9Y a photographic negative. To prevent this, we check the sign of u = v = (5) X + 15Y + 3Z X + 15Y + 3Z the L component of final vector, ~g. If this is negative, we use −~g in the conversion to grayscale. This choice is consistent For each type of deficiency, a pair of line segments in uv space with our previous conjecture on viewer expectations. is identified, where each pair connects a reference white value to two specific spectral energies, shown in Table I. To deter- mine where, on such line segments, a color-deficient viewer III. DICHROMATIC REPRODUCTION will perceive any specific color, Meyer and Greenberg make We can use the error function of (3) to create a use of another color space, called SML, so named because the image for viewing by a color-deficient observer. The objective three components correspond to cone sensitivities in the short, here is not to create a image that contains the same level of medium, and long regions of the . detail for all viewers, that is, for observers with normal vision They provide a linear transformation between CIE XYZ space and those with various color deficiencies. Rather, our objective and SML space, given by: is create multiple versions of the image, each tailored to the visual characteristics of the viewer. Unlike [12], [13], we do S 0.0000 0.0000 0.5609 X M = −0.4227 1.1723 0.0911 Y (6) not constrain how closely a remapped color must match the      corresponding original color. Instead, we focus solely on the L 0.1150 0.9364 −0.0203 Z differences between pairs of colors. Walraven and Alferdinck Color -deficienc y can then be expressed as aninability to [11] note that for many applications, the differentiation of recognize differences in one of these components. For exam- colors is more important than the identification of colors. For ple, viewers with protanopic deficiency would be unable to IEEE CG&A SPECIAL ISSUE:SMART DEPICTION FOR VISUAL COMMUNICATION 6

left image is the original, the middle image is the original as seen by a simulated observer with protanopic deficiency, and the right image is the original after re-coloring by T, also as seen by a simulated observer with protanopic deficiency. We see that the contrast between the berries and the leaves in the original image, which is lost in the middle image, is preserved in the re-colored image on the right. Trange can be chosen to span the distance of the color deficient gamut, to maximize the range of colors available to reproduce the contrast of the original image. In Figure 7, we show similar results for a viewer with deuteranopic deficiencies. The only input changes to our method are to use a projected confusion line that is based on an inability to distinguish the M component of SML space and to use the second row of Table I to determine the deuteranopic Fig. 5. Confused CIELab gamut: CIELab chromaticity with perceived chro- maticity plots for a simulated protanopic viewer (red), simulated deuteranopic gamut. Again, we see that the color contrast between the viewer (green) and simulated tritanopic viewer (blue) berries and the leaves in the original image, which is lost in the middle image, is preserved in the re-colored image. Tritanopic deficiencies are associated with confusion be- distinguish variations in the L component, and thus a line in tween blue and yellow tones, rather than red and green tones, SML color space determined by fixed values of S and M with and as such, the color image of Figure 1 does not provide varying values of L would represent a “confusion line” for a good test case for our algorithm. (All variants are nearly such viewers. indistinguishable from the original.) Instead, we select an To obtain the perceived color corresponding to any original image with significant blue-yellow information content. Figure color in RGB or CIELab or LCH(ab) space, the original color 8 shows the results. As expected, here we use a confusion is first converted to CIE XYZ space and then mapped to line determined by an inability to distinguish the S component SML space by (6). This determines, per deficiency, a specific in SML space and the third row of Table I to determine the confusion line in SML space. By applying the inverse of the tritanopic gamut. transformation (6), we can then project the confusion line into Like the monochromatic case, the time required for uv space (5) and find its intersection with one of the line Fletcher-Reeves optimization of (3) in the dichromatic case segments, described above, appropriate for this type deficiency. can become excessive for images with large numbers of colors. The resulting uv coordinates of the intersection, together with We use the same color reduction technique as a pre-processing the luminance of the original color, suffice to construct the step. For the 317-color image of Figure 6, optimization re- perceived color. quired 6.95 seconds on a 2.8 GHz Xeon. The 11,231-color Once again, to apply a conjugate-gradient search for an version of the same image required 8651.34 seconds. For optimal value of the matrix G, we must supply an initial Figure 7, the 317-color image required 6.91 seconds. The position for G. In Figure 5 we show a slice of CIELab 11,231-color version required 8617.85 seconds. A quantized space determined by fixing the L component. The a axis version of the color image in Figure 8, containing 233 colors, is horizontal, and the b axis is vertical. If color-deficient was optimized in 23.51 seconds. A 6,515-color version of this simulation, as described above, is applied to all colors shown, image required 11432.86 seconds. In all cases, the resulting the two-dimensional space collapses, in each case, to a one- pairs of images were visually indistinguishable. dimensional curve. The red curve shows the collapsed chro- Direct comparison of our method with Daltonization is dif- maticity for a simulated protanopic viewer, the green curve ficult in that, as noted earlier, the latter requires user-supplied for a simulated deuteranopic viewer, and the blue curve for a parameters. For most images, parameters that yield re-colored simulated tritanopic viewer. As expected, the protanopic and images comparable to those produced by our method can be deuteranopic viewers have a very small response in the a found with modest effort. Nevertheless, there are cases for direction (horizontal axis) while the tritanopic viewer has a which suitable parameter choices are more difficult to find. narrow response in the b direction (vertical axis). Based on In Figure 9 we show such a case. The Daltonized version this observation, we select an initial transformation, G, that selected, that with “low” correction, suffers from a loss of maps the first principal component of the target image to b contrast along the curved edge. and the second principal component to a for the protanopic and deuteranopic cases. The opposite mapping is selected for IV. DISCUSSION the tritanopic case. The L axis is mapped to itself. As in the transformation to grayscale, we then apply the Fletcher-Reeves There are several advantages in using our linear method for conjugate-gradient method to (3) to obtain an optimal G and both monochromatic and dichromatic color image reduction. therefore an optimal re-coloring transformation, T. First, we can use color quantization to quickly and easily In Figure 6, we show the results of applying this method to reduce the size of the data set. It is not necessary to operate on the image of Figure 1 for the case of protanopic deficiency. The the entire data set, as it is with traditional non-linear methods. IEEE CG&A SPECIAL ISSUE:SMART DEPICTION FOR VISUAL COMMUNICATION 7

(a) (b) (c) Fig. 6. Protanopic deficiencies: The color image from Figure 1 (a), as seen by a simulated protanopic viewer (b), and re-colored for a protanopic viewer as seen by a simulated protanopic viewer (c)

(a) (b) (c) Fig. 7. Deuteranopic deficiencies: The color image from Figure 1 (a), as seen by a simulated deuteranopic viewer (b), and re-colored for a deuteranopic viewer as seen by a simulated deuteranopic viewer (c)

(a) (b) (c) Fig. 8. Tritanopic deficiencies: A color image with significant blue-yellow content (a), as seen by a simulated tritanopic viewer (b), and re-colored for a tritanopic viewer as seen by a simulated tritanopic viewer (c) IEEE CG&A SPECIAL ISSUE:SMART DEPICTION FOR VISUAL COMMUNICATION 8

V. CONCLUSIONS AND CURRENT DIRECTIONS We have described a new method for converting color images to the grayscale. The method automatically constructs a linear mapping from ℜ3 to ℜ1 whose objective is to preserve, to the extent possible, the perceived distances between those points in ℜ3 that comprise a specific color image and their mapped values in ℜ1. Since the derived mapping function depends upon the characteristics of the input image and (a) (b) (c) incorporates information from all three dimensions of the color space, results of high quality are produced. Fig. 9. Comparison with Daltonization: A multi-gradient color image (a), We have also shown that a straightforward extension of our re-colored by our method for a deuteranopic viewer as seen by a simulated method can be used to automatically construct a re-coloring deuteranopic viewer (b), Daltonized with “low” correction for a deuteranopic ℜ3 ℜ3 viewer as seen by a simulated deuteranopic viewer (c) transformation from to that serves to preserve image detail for viewers with certain types of color-deficient vision. We are currently examining the effects of the grayscale conversion with respect to the luminance of the original Nevertheless, work is progressing to extend procedures such image. It may be necessary to introduce constraints on the as LLE and ISOMAP to operate on partial data sets. Second, ordering of some gray values to preserve effects such as the linear operations are very fast to compute and easy to gradients and specular highlights. Investigation into nonlinear implement, as opposed to procedures, such as LLE, that transformations may also prove useful. While they are more require finding eigenvectors of very large, sparse matrices. difficult to use in conjunction with color quantization, they are It is also straightforward to incorporate alternative perceptual better suited for handling complex variation, such as Figure 5. distance metrics. Finally, our linear method does not require tuning, in the form of a neighborhood size parameter, which ACKNOWLEDGMENT frequently appears in non-linear methods. This work was supported in part by the ERC Program of the There are cases where our method produces results of U.S. National Science Foundation under award EEC-9731680 questionable value. If we apply the grayscale transform to the and the ITR Program of the National Science Foundation CIELab gamut of Figure 5, the result is a smooth gray ramp under award ACI-0113139. from lower left to upper right. The corner is mapped to white, and the corner is mapped to , but the REFERENCES green and corners receive almost identical grays. It is [1] G. Wyszecki and W. Stiles, Color Science: Concepts and Methods, impossible for a linear grayscale transform to handle multiple Quantitative Data and Formulae, wiley classics, second ed. New York: modes of contrasting gradients in the image. Nevertheless, this John Wiley & Sons, 2000. is a fairly contrived example, and it is unclear what a pleasing [2] J. Tumblin and H. Rushmeier, “Tone reproduction for realistic images,” IEEE and Applications, vol. 13, no. 6, pp. 42–48, grayscale version of this image would look like. The linear November 1993. grayscale transform also breaks down when the image contains [3] R. Floyd and L. Steinberg, “An adaptive algorithm for spatial scale,” large patches of . Here the grayscale transform Proc. SID 17, pp. 75–77, 1976. [4] E. Stollnitz, V. Ostromoukhov, and D. Salesin, “Reproducing color maps black to low luminance and white to high luminance, images using custom inks,” in Proceedings of ACM SIGGRAPH 1998. resulting in a direct mapping from L to gray. To combat this ACM, 1998, pp. 267–274. problem, it is straightforward to allow the user to mask off [5] S. Roweis and L. Saul, “Nonlinear dimensionality reduction by locally linear embedding,” Science, vol. 290, no. 5500, pp. 2323–2326, 2000. regions of the image containing relevant color details. [6] J. Tenenbaum, V. de Silva, and J. Langford, “A global geometric Our method does not explicitly account for the extent to framework for nonlinear dimensionality reduction,” Science, vol. 290, which the final grayscale or re-colored images differ from the no. 5500, pp. 2319–2323, 2000. [7] B. Wandell, Foundations of Vision. Sinauer Associates, Inc., 1995. originals. In the grayscale transformation, we implicitly handle [8] G. Meyer and D. Greenberg, “Color-defective vision and computer this by initializing the optimization with a gray vector pointing graphics displays,” IEEE Computer Graphics and Applications, vol. 8, along the luminance axis. A generalization to the dichromatic no. 5, pp. 28–40, September 1988. [9] H. Brettel, F. Vienot,´ and J. Mollon, “Computerized simulation of color case is still an open problem. This can result in extremely appearance for dichromats,” Journal of the Optical Society of America unnatural colors in the image, such as the blue leaves in Figure A, vol. 14, no. 10, October 1997. 6, or the orange jelly beans in Figure 8. There is also no [10] S. Kondo, “A computer simulation of anomalous color vision,” in Color Vision Deficiencies. Kugler & Ghedini, 1990, pp. 145–159. consideration given to spatial surround or adaption. [11] J. Walraven and J. W. Alferdinck, “Color displays for the color blind,” A colored box on a light surround and the same box on a in IS&T and SID 5th Color Imaging Conference, 1997, pp. 17–22. dark surround are treated the same, even though they may [12] M. Ichikawa, K. Tanaka, S. Kondo, K. Hiroshima, K. Ichikawa, S. Tan- abe, and K. Fukami, “Web-page color modification for barrier-free color be perceived differently. This would be useful to include for vision with genetic algorithm,” Lecture Notes in Computer Science, vol. computing images for display on specific media. Finally, the 2724, pp. 2134–2146, 2003. perceptual color difference metrics we use are designed for [13] ——, “Preliminary study on color modification for still images to realize barrier-free color vision,” in IEEE International Conference on Systems, cases where large areas of color are being compared to one Man and Cybernetics, 2004. another. There may be a more meaningful way to compare [14] CIE, “Industrial colour-difference evaluation,” Commission Interna- perceived color differences in frequency cases. tionale de l’Eclairage, Tech. Rep. 116-1995, 1995.