COLOR ERROR DIFFUSION WITH GENERALIZED OPTIMUM NOISE SHAPING Niranjan Damera-Venkata Brian L. Evans Halftoning and Image Pro cessing Group Emb edded Signal Pro cessing Lab oratory Hewlett-Packard Lab oratories Dept. of Electrical and Computer Engineering 1501 Page Mill Road The University of Texas at Austin Palo Alto, CA 94304 Austin, TX 78712-1084 E-mail: [email protected] E-mail: [email protected] In this pap er, we derive the optimum matrix-valued er- ABSTRACT ror lter using the matrix gain mo del [3 ] to mo del the noise We optimize the noise shaping b ehavior of color error dif- shaping b ehavior of color error di usion and a generalized fusion by designing an optimized error lter based on a linear spatially-invariant mo del not necessarily separable prop osed noise shaping mo del for color error di usion and for the human color vision. We also incorp orate the con- a generalized linear spatially-invariant mo del of the hu- straint that all of the RGB quantization error b e di used. man visual system. Our approach allows the error lter We show that the optimum error lter may be obtained to have matrix-valued co ecients and di use quantization as a solution to a matrix version of the Yule-Walker equa- error across channels in an opp onent color representation. tions. A gradient descent algorithm is prop osed to solve the Thus, the noise is shap ed into frequency regions of reduced generalized Yule-Walker equations. In the sp ecial case that human color sensitivity. To obtain the optimal lter, we the constraints are removed, a separable color vision mo del derive a matrix version of the Yule-Walker equations which used and the linear transformation into the opp onent color we solveby using a gradient descent algorithm. space is unitary, our solution reduces to the solution derived by Kolpatzik and Bouman [1 ]. 1. INTRODUCTION 2. NOTATION Color error di usion is a high-quality metho d for color ren- In this pap er, b oldface quantities written with a ~ repre- dering of continuous-tone 24-bit digital color images on de- sent matrices, whereas b oldface quantities written without vices with limited color palettes suchaslow-cost displays a ~ representvectors. Capitalized quantities represent the and printers. The rendered images are commonly referred frequency domain while lower case quantities represent the to as halftones. The high quality of error di usion is due to space domain. Scalar quantities are represented as usual as the fact that it is a nonlinear feedback system. Quantiza- plain characters. The ith comp onentofa vector a will b e tion errors are ltered using an error lter and fed backto ~ denoted by a whereas the i; j th element of a matrix A i the input in order to shap e the quantization noise into fre- will b e denoted by a . The vector with all of its elements i;j quencies regions where humans are relatively less sensitive. equal to unity is denoted by 1. Kolpatzik and Bouman [1 ] and Akarun, Yardimci and 3 Let xm ;m 2 [0; 255] represent the input RGB im- 1 2 Cetin [2 ] use error lters with matrix-valued co ecients to age to b e halftoned. Xz ;z represents the z -transform 1 2 account for correlation among the color planes. The error of the RGB input image. lter by Kolpatzik and Bouman [1 ] lters each color error X plane indep endently in an opp onent color space [1 ]. Sepa- m m 1 2 xm ;m z z 1 Xz ;z = 1 2 1 2 rate optimum scalar error lters are designed for the lumi- m ;m 1 2 nance and chrominance channels indep endently based on a separable mo del of the human visual system. However, no We will use an index m to denote a 2-D spatial index constraints are imp osed on the error lter to ensure that all m ;m and z to denote the z-domain index z ;z . 1 2 1 2 of the red-green-blue RGB quantization error is di used. Akarun, Yardimci and Cetin [2 ] adapt the matrix-valued er- 3. VECTOR COLOR ERROR DIFFUSION ror lter co ecients using a least mean squares algorithm. This allows for cross-channel di usion of color error. How- Fig. 1 shows the system blo ck diagram for vector color er- ever, their metho d do es not incorp orate a human vision ror di usion halftoning. The rendering scalar quantizer is mo del. 3 de ned by Q : R 7! U where U = U U U maps 1 2 3 the mo di ed input vector uminto a rendered output vec- This researchwas supp orted by HP Labs and carried out tor bm. U ;i = 1; 2;3 represents the alphab et used to i when N. Damera-Venkata was with The UniversityofTexas at represent the ith comp onentof the rendered output. We Austin. + um - - - j + Q called the noise path. Validation metho ds for the matrix xm bm 6 gain mo del are presented in [3 , 5]. ? j Based on the matrix gain mo del, we obtain the net noise ~ + h + em comp onent of the output as ~ ~ b m= [Ih]?n m 7 n Figure 1: System blo ck diagrams for blo ck error di usion Since signal shaping is typically desirable or in any case un- ~ halftoning where h represents a xed 2-D nonseparable FIR der user control [5] we only need to concentrate on the noise error lter with matrix valued co ecients. The vector m shaping. We de ne the ob jective function J as the average represents the 2-D index m ;m . 1 2 visually weighted noise energy in the output halftone. We use a linear spatially-invariant matrix-valued mo del for the human visual system denoted by the matrix valued lter assume that the output to b e restricted to one bit p er color function v~ . We also de ne a constraint set C to ensure plane with 255 representing the presence of a color comp o- that all the quantization error represented in a device in- nent and 0 representing the absence of a color comp onent, dep endentRGB space is di used [5 ] . U = f0; 255g; 8i. The results of this pap er are valid for i ~ Thus, the color error di usion system h; v~ for a any equal, uniform bit allo cation among the RGB channel given vision mo del v~ may b e solved for an optimum lter quantizers. ~ h opt The quantization error vector em is formed by sub- ~ h = arg min J 8 opt tracting the quantizer input from the output ~ h2C where em= bmum 2 2 ~ ~ J = E k v~ ? I h ? n m k 9 The error vector sequence is then ltered by an error lter and ~ ~ h to pro duce the feedback signal. The error lter his X ~ ~ a lter with matrix-valued co ecients and will b e denoted hi1 =1 10 hi; i 2S j C = ~ by the 3 3 matrix-valued sequence h with supp ort set i ~ S . H represents the z -transform of the matrix-valued multi lter de ned by 5. OPTIMUM ERROR FILTER DESIGN X m m 1 2 ~ ~ The ob jective function of 9 may b e rewritten as hm ;m z z 3 Hz ;z = 1 2 1 2 " m ;m 1 2 XX 0 0 ~ J = ETr ~am v~khknmk k The ltering op eration of a 2-D multi lter is de ned by 0 k k matrix-vector convolution given by " T XX X 0 0 ~ ~ ~ ~am v~khknmk k 11 hkem k 4 h ? e m= 0 k k k2S where we have substituted am = v~ ? nm and used Here the error lter is assumed to have a causal supp ort S 2 T the fact that for a vector x, k x k = Tr xx , where Tr with 0; 0 2S= .We will assume a the standard 4-co ecient denotes the trace op eration. Floyd-Steinb erg lter [4 ] supp ort set. In the z domain the By taking the rst partial derivatives of 9 with resp ect matrix-vector convolution b ecomes a linear transformation ~ byan33 transformation matrix given by to hi for all i 2S and setting them to zero, we obtain the rst-order necessary conditions for an optimum solution. ~ ~ Z h ? e z=HzXz 5 This requires that the trace b e di erentiated with resp ect to a matrix. To do this some results from linear algebra are The mo di ed input is computed by subtracting the feedback required. signal from the input signal The following results are stated here without pro of. For pro ofs of the following, please see [6]: ~ h?e m 6 um=xm T d d ~ ~ 4. FORMULATION OF THE DESIGN f X= fX T ~ ~ dX dX PROBLEM d T ~ ~ ~ TrAX=A We use the matrix gain mo del describ ed in [3 , 5] to pre- ~ dX dict the noise shaping b ehavior of the color error di usion d T T ~ ~ ~ ~ ~ B TrAXB=A system. The matrix gain mo del linearizes the quantizer by ~ dX considering the error di usion system as two decoupled lin- d T T T ~ ~ ~ ~ ~ ~~ ~ ~~ ear systems. The rst linear system shap es the accounts TrX AXB=AXB+A XB ~ dX for signal distortion and is called the signal path while the ~ ~ ~ ~ TrAB=TrBA 12 second linear system accounts for the noise shaping and is Using the ab ove results and the linearity of the trace 6.