Color Error Diffusion with Generalized Optimum Noise

COLOR ERROR DIFFUSION WITH GENERALIZED OPTIMUM NOISE

SHAPING

Niranjan Damera-Venkata Brian L. Evans

Halftoning and Image Pro cessing Group Emb edded Signal Pro cessing Lab oratory

Hewlett-Packard Lab oratories Dept. of Electrical and Computer Engineering

1501 Page Mill Road The University of Texas at Austin

Palo Alto, CA 94304 Austin, TX 78712-1084

E-mail: [email protected] E-mail: [email protected]

In this pap er, we derive the optimum matrix-valued er- ABSTRACT

ror lter using the matrix gain mo del [3 ] to mo del the noise

We optimize the noise shaping b ehavior of color error dif-

shaping b ehavior of color error di usion and a generalized

fusion by designing an optimized error lter based on a

linear spatially-invariant mo del not necessarily separable

prop osed noise shaping mo del for color error di usion and

for the human color vision. We also incorp orate the con-

a generalized linear spatially-invariant mo del of the hu-

straint that all of the RGB quantization error b e di used.

man visual system. Our approach allows the error lter

We show that the optimum error lter may be obtained

to have matrix-valued co ecients and di use quantization

as a solution to a matrix version of the Yule-Walker equa-

error across channels in an opp onent color representation.

tions. A gradient descent algorithm is prop osed to solve the

Thus, the noise is shap ed into frequency regions of reduced

generalized Yule-Walker equations. In the sp ecial case that

human color sensitivity. To obtain the optimal lter, we

the constraints are removed, a separable color vision mo del

derive a matrix version of the Yule-Walker equations which

used and the linear transformation into the opp onent color

we solveby using a gradient descent algorithm.

space is unitary, our solution reduces to the solution derived

by Kolpatzik and Bouman [1 ].

1. INTRODUCTION

2. NOTATION

Color error di usion is a high-quality metho d for color ren-

In this pap er, b oldface quantities written with a ~ repre-

dering of continuous-tone 24-bit digital color images on de-

sent matrices, whereas b oldface quantities written without

vices with limited color palettes suchaslow-cost displays

a ~ representvectors. Capitalized quantities represent the

and printers. The rendered images are commonly referred

frequency domain while lower case quantities represent the

to as halftones. The high quality of error di usion is due to

space domain. Scalar quantities are represented as usual as

the fact that it is a nonlinear feedback system. Quantiza-

plain characters. The ith comp onentofa vector a will b e

tion errors are ltered using an error lter and fed backto

denoted by a whereas the i; j th element of a matrix A

the input in order to shap e the quantization noise into fre-

will b e denoted by a . The vector with all of its elements

i;j

quencies regions where humans are relatively less sensitive.

equal to unity is denoted by 1.

Kolpatzik and Bouman [1 ] and Akarun, Yardimci and

Let xm ;m 2 [0; 255] represent the input RGB im-

1 2

Cetin [2 ] use error lters with matrix-valued co ecients to

age to b e halftoned. Xz ;z represents the z -transform

1 2

account for correlation among the color planes. The error

of the RGB input image.

lter by Kolpatzik and Bouman [1 ] lters each color error

plane indep endently in an opp onent color space [1 ]. Sepa-

m m

1 2

xm ;m z z 1 Xz ;z =

1 2 1 2

rate optimum scalar error lters are designed for the lumi-

m ;m

1 2

nance and chrominance channels indep endently based on a

separable mo del of the human visual system. However, no

We will use an index m to denote a 2-D spatial index

constraints are imp osed on the error lter to ensure that all

m ;m and z to denote the z-domain index z ;z .

1 2 1 2

of the red-green-blue RGB quantization error is di used.

Akarun, Yardimci and Cetin [2 ] adapt the matrix-valued er-

3. VECTOR COLOR ERROR DIFFUSION

ror lter co ecients using a least mean squares algorithm.

This allows for cross-channel di usion of color error. How-

Fig. 1 shows the system blo ck diagram for vector color er-

ever, their metho d do es not incorp orate a human vision

ror di usion halftoning. The rendering scalar quantizer is

mo del.

de ned by Q : R 7! U where U = U U U maps

1 2 3

the mo di ed input vector uminto a rendered output vec-

This researchwas supp orted by HP Labs and carried out

tor bm. U ;i = 1; 2;3 represents the alphab et used to

when N. Damera-Venkata was with The UniversityofTexas at

represent the ith comp onentof the rendered output. We Austin.

+ um

- - -

+ Q

called the noise path. Validation metho ds for the matrix

xm bm

gain mo del are presented in [3 , 5].

Based on the matrix gain mo del, we obtain the net noise

+ em

comp onent of the output as

b m= [Ih]?n m 7

Figure 1: System blo ck diagrams for blo ck error di usion

Since signal shaping is typically desirable or in any case un-

halftoning where h represents a xed 2-D nonseparable FIR

der user control [5] we only need to concentrate on the noise

error lter with matrix valued co ecients. The vector m

shaping. We de ne the ob jective function J as the average

represents the 2-D index m ;m .

1 2

visually weighted noise energy in the output halftone. We

use a linear spatially-invariant matrix-valued mo del for the

human visual system denoted by the matrix valued lter

assume that the output to b e restricted to one bit p er color

function v~ . We also de ne a constraint set C to ensure

plane with 255 representing the presence of a color comp o-

that all the quantization error represented in a device in-

nent and 0 representing the absence of a color comp onent,

dep endentRGB space is di used [5 ] .

U = f0; 255g; 8i. The results of this pap er are valid for

Thus, the color error di usion system h; v~ for a

any equal, uniform bit allo cation among the RGB channel

given vision mo del v~ may b e solved for an optimum lter

quantizers.

opt

The quantization error vector em is formed by sub-

h = arg min J 8

opt

tracting the quantizer input from the output

h2C

where

em= bmum 2

~ ~

J = E k v~ ? I h ? n m k 9

The error vector sequence is then ltered by an error lter

and

~ ~

h to pro duce the feedback signal. The error lter his

~ ~

a lter with matrix-valued co ecients and will b e denoted

hi1 =1 10 hi; i 2S j C =

by the 3 3 matrix-valued sequence h with supp ort set

S . H represents the z -transform of the matrix-valued

multi lter de ned by

5. OPTIMUM ERROR FILTER DESIGN

m m

1 2

The ob jective function of 9 may b e rewritten as

hm ;m z z 3 Hz ;z =

1 2 1 2

m ;m

1 2

0 0

J = ETr ~am v~khknmk k

The ltering op eration of a 2-D multi lter is de ned by

matrix-vector convolution given by

0 0

~ ~

~am v~khknmk k 11

hkem k 4 h ? e m=

k2S

where we have substituted am = v~ ? nm and used

Here the error lter is assumed to have a causal supp ort S

2 T

the fact that for a vector x, k x k = Tr xx , where Tr

with 0; 0 2S= .We will assume a the standard 4-co ecient

denotes the trace op eration.

Floyd-Steinb erg lter [4 ] supp ort set. In the z domain the

By taking the rst partial derivatives of 9 with resp ect

matrix-vector convolution b ecomes a linear transformation

byan33 transformation matrix given by to hi for all i 2S and setting them to zero, we obtain the

rst-order necessary conditions for an optimum solution.

Z h ? e z=HzXz 5

This requires that the trace b e di erentiated with resp ect

to a matrix. To do this some results from linear algebra are

The mo di ed input is computed by subtracting the feedback

required.

signal from the input signal

The following results are stated here without pro of. For

pro ofs of the following, please see [6]:

h?e m 6 um=xm

d d

~ ~

4. FORMULATION OF THE DESIGN

f X= fX

~ ~

dX dX

PROBLEM

~ ~ ~

TrAX=A

We use the matrix gain mo del describ ed in [3 , 5] to pre-

dict the noise shaping b ehavior of the color error di usion

T T

~ ~ ~ ~ ~

B TrAXB=A

system. The matrix gain mo del linearizes the quantizer by

considering the error di usion system as two decoupled lin-

T T T

~ ~ ~ ~ ~ ~~ ~ ~~

ear systems. The rst linear system shap es the accounts

TrX AXB=AXB+A XB

for signal distortion and is called the signal path while the

~ ~ ~ ~

TrAB=TrBA 12 second linear system accounts for the noise shaping and is

Using the ab ove results and the linearity of the trace 6. SIMULATION RESULTS

op erator we obtain after considerable algebra [5 ]

The optimal lter that was obtained based on our calibrated

P P P

color monitor and was tested on ve standard color test im-

v~ k~r i k= v~ sv~qhp

p q s

ages lena, peppers, pasta, fruits, hats. Monitor calibration

was p erformed by rst measuring the monitor gamma, and

~r p + q s i 13

then recording the CIE X,Y,Z co-ordinates of the monitor

R, G, B guns resp ectively. First we undo the gamma correc-

These equations may b e regarded as a generalization of the

tion on the RGB image to obtain device indep endent linear

Yule-Walker equations [6] from linear prediction theory to

RGB tristimulus values, which are then transformed into

the matrix case, with a generalized linear spatially-invariant

XYZ space using the XYZ calibration matrix [10 ]. We used

weighting. The ab ove set of generalized Yule-Walker equa-

the pattern-separable mo del for the human visual system

tions may be solved for the optimal lter sub ject to the

based on the work of Poirson and Wandell [11 , 12 ] although

constraints of 10 using the steep est descent algorithm [6 ].

our derivation of the optimum solution assumes the most

We use a white noise image as an approximation to the

general linear spatially-invariant color mo del. The Poirson

uncorrelated noise image nm. Thus, the required auto-

and Wandell mo del consists of

correlation matrices are approximated as:

1. A linear transformation T from XYZ into an opp o-

~r k=E nmn m+k k 14

nent space and,

2. Separable spatial ltering on each channel using a

di erent spatial lter on eachchannel. This op era-

tion may b e regarded as a matrix convolution in the

~r k = E v~ tnm t n m + k

frequency domain by a lter with diagonal matrix-

valued co ecients d.

v~ t k + t=v~k 15

Thus, v~ m is computed as

~ ~

v~ m= dmT 18

where k is the two-dimensional Kronecker delta func-

tion [6 ]. In the optimization the constraint is enforced by

The parameter spaci cations for the mo del are given in [13 ].

pro jection onto the convex constraint set. The convergence

To evaluate the noise shaping b ehavior, we pro duced

b ehavior of this algorithm is discussed in [7 ]. The algorithm

undistorted halftones using the color signal distortion can-

is guaranteed to converge if the convergence parameter in

celing metho d develop ed in [3]. In [3 ], it was shown that

the descent algorithm is chosen to b e small enough [7].

according to the matrix gain mo del, the quantization error

The descent algorithm may b e formulated as follows:

image in the distortion canceling metho d is in fact the un-

correlated noise injection into the halftoning system. The

rJ hi = v~ kv~ i + k +

e ective noise shaping gain in dB of the optimal lter over

the separable Floyd-Steinb erg lter may b e computed as

X X X

v~ sv~ qhp p + q s i

p q s

19 NG = 10 log

opt

~ ~ ~

f i = h i rJ h i

where the numerator and denominator in the argumentof

~ ~

f i 16 h i = P

the log function are the ob jective functions computed us-

ing 9 for the optimal lter and the Floyd-Steinb erg lter,

where refers to the iteration numb er, and P is the pro jec-

resp ectively. Sample averages were used to estimate the ex-

tion op erator that pro jects the iterate into the constraint set

p ectations. Table 1 tabulates the noise gain of the optimal

C , de ned by 10. The convergence parameter =0:005

lter over using a separable Floyd-Steinb erg error lter.

in my simulations. The pro jection op erator is de ned as [8 ]

Fig. 2a shows a magni ed view of the pasta image

halftoned using a separable Floyd-Steinb erg error lter on

~ ~ ~

the R, G, and B color planes indep endently and 2b show

f i I 1 17 P f i = f i

3 jS j

the corresp onding results for halftoning with the optimal er-

i2S

ror lter. The optimal lter results in less visible halftone

noise. It signi cantly reduces color impulses when com- Several random initial guesses were tried, and the descent

pared with scalar error di usion using lters with scalar co- algorithm was terminated when the changes in the ob jec-

ecients. The halftone noise patterns pro duced by conven- tive function were b elow a prede ned threshold. Using

tional Floyd-Steinb erg error scalar lter were signi cantly this metho d one may explore di erent minimizers solutions

more visible when observed on the calibrated monitor as that result in nearly the same ob jective function value.

compared to the noise patterns pro duced by the optimal The uniformity in the dot distributions pro duced by di er-

lter. It must be emphasized that since the optimal l- ent initial guesses was di erent. It has b een shown [9 ] that

ter co ecients are dep endent on a particular monitor con- frequency weighted mean squared error alone cannot guar-

guration, the ab ove design pro cess must b e applied on a antee optimum dot distributions. Wechose a solution that

case-by-case basis. had a reasonably uniform dot distribution.

7. CONCLUSION

Wehave presented a metho d for optimum error lter de-

sign for RGB color error di usion. The optimum lter min-

imizes the visibility on average of color halftone noise en-

Image Noise gain dB

ergy with resp ect to any given linear time-invarianthuman

lena 3.29

color vision mo del. The optimum lter has matrix-valued

p epp ers 1.88

co ecients allowing for cross-channel di usion of color er-

fruits 1.37

rors and is obtained by solving a matrix extension of the

pasta 1.47

Yule-Walker equations sub ject to di usion constraints via

hats 1.05

a gradient descent algorithm.

Table 1: Noise gain of the optimal lter on standard test

8. REFERENCES

images.

[1] B. Kolpatzik and C. Bouman, \Optimized error dif-

fusion for high quality image display," J. Electronic

Imaging,vol. 1, pp. 277{292, Jan. 1992.

[2] L. Akarun, Y. Yardimici, and A. E. Cetin, \Adaptive

metho ds for dithering color images," IEEE Trans. Im-

age Processing,vol. 6, no. 7, pp. 950{955, July 1997.

[3] N. Damera-Venkata and B. L. Evans, \Matrix gain

mo del for vector color error di usion," Proc. IEEE-

EURASIP Workshop on Nonlinear Signal and Image

Processing, June 2001, to app ear.

[4] R. Floyd and L. Steinb erg, \An adaptive algorithm for

spatial grayscale," Proc. Soc. Image Display,vol. 17,

no. 2, pp. 75{77, 1976.

[5] N. Damera-Venkata, Analysis and Design and of

Vector Error Di usion Systems for Image Halftoning,

Ph.D. thesis, Dept. of Electrical and Comp. Eng., The

UniversityofTexas at Austin, Austin, TX, Dec. 2000.

[6] T. K. Mo on and W. C. Stirling, Mathematical Methods

and Algorithms for Signal Processing, Prentice-Hall,

2000.

[7] P.L.Comb ettes and P. Bondon, \Adaptive linear l-

tering with convex constraints," Proc. IEEE Int. Conf.

Acoustics, Speech, and Signal Processing, vol. 2, pp.

1372{1375, May 1995.

[8] H. Stark and Y. Yang, Vector SpaceProjections, John

Wiley and Sons, Inc., 1998.

[9] P.Wong, \Entropy constrained halftoning using mul-

tipath tree co ding," IEEE Trans. Image Processing,

vol. 6, no. 11, pp. 1567{1569, Nov. 1997.

[10] B. Wandell, Foundations of Vision, Sinauer Asso-

ciates, 1995.

[11] A. B. Poirson and B. A. Wandell, \App earance of

colored patterns: Pattern-color separability," J. Opt.

Soc. Am. A,vol. 10, no. 12, pp. 2458{2470, Dec. 1993.

[12] A. B. Poirson and B. A. Wandell, \Pattern-color sep-

Figure 2: Performance of the optimal lter. a Floyd-

arable pathways predict sensitivity to simple colored

Steinb erg error lter applied to the R, G, and B planes.

patterns," Vision Research.,vol. 36, pp. 515{526, Dec.

b Optimum matrix-valued error lter p erformance. Note

1996.

the decrease in visibility of color halftone noise achieved by

using the optimum lter.

[13] X. Zhang and B. A. Wandell, \A spatial extension

of cielab for digital color image repro duction," Soc.

Information Display Digest of Technical Papers, pp. 731{734, 1996.