The Mathematics of Calibration

M. J. Vrhel H. J. Trussell

Color Savvy Systems Ltd. Dept. of Electrical and Computer Engineering 305 South Main Street North Carolina State University Springboro, OH 45066 Raleigh, NC 27695-7911

Abstract contain the sampled CIE XYZ color matching func- The mathematical formulation of calibrating color tions and the CIE XYZ value of the spectrum Lr is T image reproduction and recording devices is presented. given by t = A Lr. This formulation provides a foundation for future re- A device independent is dened as any search in areas of characterization of devices and dis- space that has a one-to-one mapping onto the CIE play of color images. The procedure outlined in this XYZ color space. Device independent values describe paper should become standard for displaying color im- color for the standard CIE observer. ages for the image processing community. By denition, a device dependent color space cannot have a one-to-one mapping onto the CIE XYZ color 1 Introduction space. In the case of a recording device, the device With the advent of low cost color printers and dependent values describe the response of that partic- scanners, there is increased interest in the image pro- ular device to color. For a reproduction device, the cessing community to develop techniques for enhanc- device dependent values describe only those the ing, restoring, and reproducing color images [1]. The device can produce. demonstration of the performance of methods for color The use of device dependent descriptions of color image processing has presented a problem due to an in- presents a problem in the world of networked com- ability to control the nal printing process, misunder- puters and printers. The same RGB vector can result standings with regard to color spaces and what RGB in dierent colors on dierent monitors and printers. really means, and improper or poorly managed scan- Similarly, a specied CMYK value can result in dier- ning to name a few. Problems can be clearly seen ent colors on dierent printers. in comparing the various “unprocessed” color images A solution is to dene images in terms of a CIE of Lena in [1] (a recent special issue on color imag- color space and then transform this data to device de- ing) with the actual original Lena image. A simu- pendent descriptors for the device on which the image lation of the dierences is available at the web site is to be reproduced. This solution requires knowledge F ftp://ftp.eos.ncsu/pub/hjt/prole. The dierence be- of a function device which will provide a mapping tween the images demonstrates the need for a stan- from device dependent control values to a CIE color dard image dened in terms of CIE values with which space. In the case of a printer, it is necessary to deter- F 1 to demonstrate color image processing algorithms, in mine a transformation device (which may or may not F addition to standards for displaying and porting pro- exist). For a monitor, the transformations device and F 1 cessed images. device are both needed since the monitor is used as both a source of image data, and as a display device. 2 Background Modern printers and display devices are limited in We will use a vector space notation in which the vis- the colors they can produce. This limited set of colors ible spectrum is mathematically sampled at N wave- is dened as the of the device. If cie is the lengths. An illuminant spectrum is represented by an range of numerical values in the selected CIE color N N diagonal matrix and the spectral reectance space and print is the numerical range of the device of an object is represented by an N element vector r. control values then the set The radiant spectrum reected from the object with G = { t ∈ |∃c ∈ where F (c)=t } spectral reectance r under the illuminant L can be cie print device expressed as Lr. The columns of the Nx3 matrix A denes the gamut of the color output device. Simi-

larly, the complement set 4 Scanners Mathematically, the recording process of a scanner c { ∈ |6∃ ∈ F } G = t cie c print where device(c)=t can be expressed as denes colors outside the device gamut. For colors in T zi = H(M ri) the gamut, there will exist a mapping between the de- vice dependent control values and the CIE XYZ color where the matrix M contains the spectral sensitiv- space. Colors which are in Gc cannot be reproduced ity (including the scanner illuminant) of the three (or and must be gamut-mapped to a color which is within more) bands of the device, ri is the spectral reectance G. The gamut mapping algorithm D is a mapping at spatial point i, H models any nonlinearities in the from cie to G, that is D(t) ∈ G ∀t ∈ cie. scanner (invertible in the range of interest), and zi is F F 1 D the vector of recorded values. The mappings device, device, and make up what is dened as a device prole. These mappings The calibration problem is to determine a continu- describe how to transform between a CIE color space ous mapping Fscan which will transform the recorded and the device control values. The International Color values to a CIE color space, i.e. Commission (ICC) has suggested a standard format t = AT Lr = F (z) for describing a prole[7]. scan ∈ 3 Monitors for all r r, the space of reection values. Look-up-tables, nonlinear and linear models for A monitor is often used to provide softcopy preview F for the printing process and is a common source for scan have been used to calibrate color scanners [3, 4]. user generated images. Monitor calibration is almost In all of these approaches, the rst step is to select a always based on a physical model of the device [5]. A collection of color patches which span the colors of in- typical model is t = H[r0,g0,b0]T where terest. Ideally these colors should not be metameric in terms of the scanner sensitivities or to the standard 0 r r =[(r r0)/(rmax r0)] observer under the illuminant for which the calibration is being produced. This constraint is easily obtained 0 g g =[(g g0)/(gmax g0)] (1) and assures a one-to-one mapping between the scan 0 b values and the device independent values across these b =[(b b0)/(bmax b0)] samples. where t is the CIE value produced by driving the mon- The reectance spectra of these Mq color patches itor with control value d =[r, g, b]T , and the param- { } will be denoted by q k for 1 k Mq . These eters r, g, b, r0, g0, b0, rmax, gmax, bmax, and H patches are measured using a spectrophotometer or a are dened in the prole. colorimeter which will provide the device independent Creating a prole for a monitor involves the deter- values mination of these parameters where rmax, gmax, bmax T are the maximum values of the control values (e.g. {tk = A qk} for 1 k Mq. 255). To determine the parameters, a series of color These values {t } could represent any colorimetric or patches is displayed on the monitor and measured with k device independent values, e.g. CIELAB, CIELUV a colorimeter which will provide pairs of CIE values in which case {t = L(AT q )} where L()isthe {t } and control values {d } k =1, ..., M. k k k k transformation from CIEXYZ to the appropriate color Values for r, g, b, r0, g0, and b0 are determined 0 0 0 space. The patches are also measured with the scan- such that the elements of [r ,g ,b] are linear with re- ner to be calibrated providing {z = H(MT q )} for spect to the elements of XY Z and scaled between the k k 1 k M . range [0,1] (cf. Eq. 1). The matrix H is determined q Mathematically, the calibration problem is: nd a using F       transformation scan where 0 X XRmax XGmax XBmax r M      0  Xq Y = YRmax YGmax YBmax b F = arg(min ||F(z ) t ||2) 0 scan F i i Z ZRmax ZGmax ZBmax g i=1

T 2 where [XRmax,YRmax,ZRmax] is the CIE XYZ tris- and ||.|| is the error metric in the CIE color space. timulus value of the phosphor for control value Other metrics may be used if desired. In practice, it T d =[rmax, 0, 0] . The other phosphors are dened may be necessary and desirable to incorporate con- similarly. straints on Fscan.

5 Printers that all the colors can be reproduced. For detailed Printer calibration is dicult due to the nonlin- gamut mapping experiments, the reader is referred to earity of the printing process, and the wide variety [9, 8]. of methods used for (e.g. lithography, 7 Constraints inkjet, dye sublimation etc.). Because of these di- The formulation for the scanner calibration and culties, printing devices are often calibrated using an the printer calibration are mathematically similar. In LUT and interpolation [2, 3]. many practical cases, there are constraints which the To produce a prole of a printer, a subset of values mappings should satisfy. For example, in the printer spanning the space of allowable control values for the prole there are several factors such as ink limit and printer is rst selected. Denote these device dependent undercolor removal which often come into play. values by c for 1 k M . These values produce a k p In the case of a printer prole, constraint sets of set of reectance spectra which are denoted by p for k interest include the following: 1 k M . The patches p are measured using a p k data consistency: colorimetric device as was performed for the scanner calibration, which provides the values Fprint ∈{G|||G(ci) ti|| v i =1, ..., Mp } T {tk = A pk} for 1 k Mp where v is a just-noticeable-dierence (JND) thresh- Again, tk could represent any colorimetric or device old. independent values, not just CIEXYZ. inklimit: The problem is then to determine a mapping F print F ∈{G|||G1 || ∀ ∈ } which is the solution to the optimization problem print (t) ink t G

XMp where ink is the maximum amount of ink that should F = arg(min ||F(c ) t ||2) be placed on the paper. print F i i i=1 smoothness: where as in the scanner calibration problem, there may Fprint ∈{G|||(∇G)(c)|| smooth ∀c ∈ print } be constraints which Fprint must satisfy. ∇G G 6 Gamut Mapping where is the gradient of the function and print is the range of control values for the printer. Consider two Gmonitor and Gprint.Itis desired to print an image which is displayed on the In CMYK printers, under-color removal is a tech- nique in which the cyan, magenta and ink monitor. Assuming Fmonitor is known, we can map from the monitor RGB values to CIE values. Now amounts are reduced and ink is added. Typi- the problem is to map these CIE values to device de- cally, the allowable CMYK values are restricted since pendent values for the printer. The problem is that dierent CMYK combinations could give rise to the there may be colors which the monitor can display but same CIE XYZ value. This restriction is dened by a set H which is itself dened by four curves which map the printer cannot print. This problem is dened as a 0 0 0 0 gamut mismatch problem (the mismatch in this case values from [cmy] 3-space into [c m y k ] four space. being between the monitor and the printer). The map- For example, H could be dened as D ping is used for this purpose and the device control { | } F 1 D H = [c+gc(),m+gm(),y+gy(),fk()] = min(c, m, y) value for the CIE value t is given by printer( (t)). Depending upon the desired eect, the gamut map- where c, m, y ∈ [0, 255], gi()=fi() , and the ping function D may or may not be the identity opera- curves fi(), for i =(c, m, y, k) are as shown in Fig- tor on the colors within Gprint. The reason for using a ure 1. With H dened, the constraint of under-color D which is not the identity operator can be illustrated removal is given by: by an example in which there are smoothly varying re- 1 gions in the image which are outside the device gamut. Fprint ∈{G|G (t) ∈ H ∀t ∈ G } These colors will be gamut-mapped to the same color on the gamut boundary which will result in abrupt In the LUT based prole, the gamut mapping is im- edges in the previously smooth region. To reduce this plicit in the function which maps from the CIE color artifact, a gamut mapping is often used which com- space to the device dependent values. In construct- presses all the colors in the image to reduce the col- ing the LUT, gamut-mapping considerations can be orimetric dynamic range in the image while ensuring incorporated as constraints on the LUT entries. For

where scan is the range of numerical values produced 250 by the scanner.

f (α) k Finally, color perception can be included into the construction of the prole by performing the trans- 200 formation to the perceptual color space on the mea- f (α) y surements from the samples which are used for the construction of the prole. If L maps the CIE val- 150 ues to a perceptual color space such as that found in f (α) c [6], then the prole LUT is constructed between the values {vk = L(tk)} and the device control values ck.

output cmyk value 100

f (α) m 8 Example To demonstrate the need for improved color man- 50 agement, an original version of the Lena image was scanned on a desktop scanner. The scanned image will be referred to as the newly scanned Lena image. In ad- 0 0 50 100 150 200 250 dition, a color target was scanned which contained 276 α=min( c,m,y) color patches. The target patches were measured with a colorimeter and an LUT mapping was determined from scanned RGB values to CIELab D50 (i.e. F Figure 1: Undercolor removal curves scan was created). Finally, the commonly used digital color Lena image was obtained which will be referred to as example, it may be desired that certain points outside the standard Lena image. the gamut in the LUT provide maximum colorant on A dye sublimation (dye-sub) RGB printer (3-color the paper (e.g. 100% cyan, 0% yellow,0% magenta, CMY printer which accepted RGB input values) was F 1 0% black). At the same time one would like to have a also proled from CIELab D50 (i.e. print was cre- smooth transition to the in-gamut colors, and a colori- ated) to printer RGB space. As discussed in Section 5, metrically meaningful way to assign the out-of-gamut this prole was created by measuring a series of color colors. Constraints of interest in determining the func- patches with a colorimeter and creating an LUT. Out tion D include minimum color error: of gamut colors were mapped, via D, to the closest in gamut value, along a constant angle while pre- D ∈ {G|G(t) ∈ G G = arg(min ||P(K(t)) P(t)||2) K serving in CIELab space.

∀t ∈ CIE } The LUT for the dye-sub printer was used to map from the CIELab D50 values to printer control val- where P is a mapping from the CIE color space to a ues. Three images were printed and are available at space which is perceptually optimal for gamut map- ftp://ftp.ncsu.edu/pub/hjt/prole. The images are as ping (e.g. constant hue) follows where lk is the kth pixel RGB value for the smoothness: standard Lena image and lsk is the kth pixel RGB D∈{G|||(∇G)(t)|| ∀t ∈ } value for the newly scanned Lena image: CIE (a) The standard Lena image, created by sending xed points: the RGB values lk to the printer. D∈{G|G } (b) Image created by sending the values (ti)=si i =1, ..., P F 1 D F print( (( scan(lsk))) to the printer. where there are P xed points, {ti}, {si} i =1, ..., P , (c) Image created by sending the values lsk to the which need to be mapped exactly. printer. Scanner calibration can also be formulated in this Figure 2 demonstrates the dierences between (a), framework. The primary sets of interest include (b), and (c) by separately displaying the CIELab data consistency: frames (i.e. individual L , a , and b images) for each case. The images were created by applying the trans- F ∈{G|||G || } scan (ui) ti v i =1, ..., Mp formation Fprint on the printer RGB control values. smoothness: For this gure the L , a , and b values were uniformly quantized over their physical range with 8bits. While Fscan ∈{G|||(∇G)(u)|| ∀u ∈ scan } not as eective as displaying the color images directly,

the usefulness of the framework. The importance of the problem was made clear by consideration of the vast dierences in appearance between images pro- duced by calibrated and uncalibrated systems. This more formal approach of producing calibrated color output should become standard procedure when dis- playing images for the image processing community. References [1] H. J. Trussell, J. Allebach, M. D. Fairchild, B. Funt, and P. W. Wong Eds., Special Issue on Color Imaging, IEEE Transactions on Image Pro- cessing, Vol. 6, No. 7, July 1997.

[2] J. Z. Chang, J. P. Allebach, and C. A. Bouman, “Sequential linear interpolation of multidimen- sional functions,” IEEE Transactions on Image Processing, Vol. 6, No. 9, pp. 1231-1245, Sept. 1997. [3] P. C. Hung, “Colorimetric calibration in elec- tronic imaging devices using a look-up table model and interpolations,” J. Electronic Imaging, Vol. 2, pp. 53-61, Jan. 1993.

Figure 2: CIELab frames for cases (a), (b) and (c) [4] H. R Kang and P. G. Anderson, “Neural network applications to the color scanner and printer cali- brations,” J. Electronic Imaging, Vol. 1, pp. 125- the gure does demonstrate the large dierences be- 134, April 1992. tween cases (a), (b) and (c). [5] W. B. Cowan, “An inexpensive scheme for cali- When viewed as a color image, case (a) closely bration of a color monitor in terms of standard matched those in the special issue, while (b) was a CIE coordinates,” Comput. Graph., vol. 17, pp. close match to the original printed image. Providing 315-321, July 1983. the scanned RGB values directly to the printer (case (c)) produces a much dierent image. One should re- [6] M. D. Fairchild, “Renement of the RLAB color alize that the commonly used standard Lena image is space,” Color Res. Appl., vol. 21, pp. 338-346, a scanned RGB image which is often provided directly Oct. 1996. as input to the available color printer. Comparison of (a) and (c) gives an indication of the dierences in [7] International Color Consortium, Int. Color Con- the raw scanner data. The large dierences between sort. Prole Format Ver. 3.4, available at the images demonstrate that widely dierent results http://color.org/. can occur depending upon how an image is scanned, [8] T. Hoshino and R. S. Berns, “Color Gamut Map- processed, and printed. This is a signicant problem ping Techniques for Color Hard Copy Images,” in when attempting to convey the results of a color im- Proc. SPIE, Vol. 1990, pp. 152-165, 1993. age processing algorithm, and to compare those results with previously archived material. [9] E. D. Montag and M. D. Fairchild, “Psychophysi- cal Evaluation of Gamut Mapping Techniques Us- 9 Conclusions ing Simple Rendered Images and Articial Gamut The problem of achieving by cal- Boundaries,” IEEE Trans. Image Proc., vol. 6, ibrating scanners, monitors, and printers was dened no. 7, pp. 977-989, July 1997. mathematically. This lays a framework upon which sophisticated color image processing methods may be developed. The ease with which the constraint sets are formulated, as shown in Section 7, demonstrates