Error Diffusion Halftoning Methods for High-Quality Printed and Displayed

Home , Halftone

ErrorError Diffusion Diffusion Halftoning Halftoning Methods Methods for for HighHigh--QualityQuality PrintedPrinted and and Displayed Displayed ImagesImages

Prof. Brian L. Evans Embedded Signal Processing Laboratory The University of Texas at Austin Austin, TX 78712-1084 USA http:://www.ece.utexas.edu/~bevans

Ph.D. Graduates: Dr. Niranjan Damera-Venkata (HP Labs) Dr. Thomas D. Kite (Audio Precision) Graduate Student: Mr. Vishal Monga Other Collaborators: Prof.AlanC.Bovik(UT Austin) Prof. Wilson S. Geisler (UT Austin) Last modified November 7, 2002 OutlineOutline

• Introduction • Grayscale halftoning methods • Modeling grayscale error diffusion – Compensation for sharpness – Visual quality measures • Compression of error diffused halftones • Color error diffusion halftoning for display – Optimal design – Linear human visual system model • Conclusion

2 Introduction NeedNeed for for Digital Digital Image Image HalftoningHalftoning

• Examples of reduced grayscale/color resolution – Laser and inkjet printers ($9.3B revenue in 2001 in US) – Facsimile machines – Low-cost liquid crystal displays • Halftoning is wordlength reduction for images – Grayscale: 8-bit to 1-bit (binary) – Color displays: 24-bit RGB to 12-bit RGB (e.g. PDA/cell) – Color displays: 24-bit RGB to 8-bit RGB (e.g. cell phones) – Color printers: 24-bit RGB to CMY (each color binarized) • Halftoning tries to reproduce full range of gray/ color while preserving quality & spatial resolution

3 Introduction ConversionConversion to to One One Bit Bit Per Per Pixel: Pixel: Spatial Spatial Domain Domain

Original Image Threshold at Mid-Gray Dispersed Dot Screening

Clustered Dot Floyd Steinberg Stucki Error Screening Error Diffusion Diffusion 4 Introduction ConversionConversion to to One One Bit Bit Per Per Pixel: Pixel: Magnitude Magnitude Spectra Spectra

Original Image Threshold at Mid-Gray Dispersed Dot Screening

Clustered Dot Floyd Steinberg Stucki Error Screening Error Diffusion Diffusion 5 Introduction NeedNeed for for Speed Speed for for Digital Digital Halftoning Halftoning

• Third-generation ultra high-speed printer (CMYK) – 100 pages per minute, 600 lines per inch, 4800 dots/inch/line – Output data rate of 7344 MB/s (HDTV video is ~96 MB/s) • Desktop color printer (CMYK) – 24 pages per minute, 600 lines/inch, 600 dots/inch/line – Output data rate of 220 MB/s (NTSC video is ~24 MB/s) • Parallelism – Screening: pixel-parallel, fast, and easy to implement (2 byte reads, 1 compare, and 1 bit write per binary pixel) – Error diffusion: row-parallel, better results on some media (5bytereads,1compare,4MACs,1byteand1bitwriteper binary pixel) 6 OutlineOutline

7 Grayscale Halftoning ScreeningScreening (Masking) (Masking) Methods Methods

• Periodic array of thresholds smaller than image – Spatial resampling leads to aliasing (gridding effect) – Clustered dot screening is more resistant to ink spread – Dispersed dot screening has higher spatial resolution – Blue noise masking uses large array of thresholds

Clustered dot mask Dispersed dot mask 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 Thresholds = , , , , , , , , , , , , , , , *256 32 32 32 32 32 32 32 32 32 32 32 32 32 32 32 32ÿ 8 Grayscale Halftoning GrayscaleGrayscale Error Error Diffusion Diffusion

• Shape quantization noise into high frequencies • Design of error filter key to quality • Not a screening technique

difference threshold current pixel u(m) x(m) b(m) Error Diffusion + _ _ 7/16 + 3/16 5/16 1/16 h(m) e(m) 2-D sigma-delta shape error compute weights error modulation [Anastassiou, 1989] Spectrum 9 Grayscale Halftoning SimpleSimple Noise Noise Shaping Shaping Example Example

• Two-bit output device and four-bit input words – Going from 4 bits down to 2 increases noise by ~ 12 dB – Shaping eliminates noise at DC at expense of increased noise at high frequency. 42 Average output = ¼ (10+10+10+11)=1001 Input To words 22output 4-bit resolution at DC! device Added noise 1sample delay Assume input = 1001 constant 12 dB Time Input Feedback Sum Output (2 bits) 1100100100110 f Periodic 2100101101010 3100110101110 If signal is in this band, 4100111110011 then you are better off 10 Grayscale Halftoning DirectDirect Binary Binary SearchSearch (Iterative) (Iterative)

• Practical upper bound on halftone quality • Minimize mean-squared error between lowpass filtered versions of grayscale and halftone images – Lowpass filter is based on a linear shift-invariant model of human visual system (a.k.a. contrast sensitivity function) • Each iteration visits every pixel [Analoui & Allebach, 1992] – At each pixel, consider toggling pixel or swapping it with each of its 8 nearest neighbors that differ in state from it – Terminate when if no pixels are changed in an iteration • Relatively insensitive to initial halftone provided that it is not error diffused [Lieberman & Allebach, 2000]

11 Grayscale Halftoning ManyMany Possible Possible Contrast Contrast SensitivitySensitivity FunctionsFunctions

• Contrast at particular spatial frequency for visibility – Bandpass: non-dim backgrounds [Manos & Sakrison, 1974; 1978] – Lowpass: high-luminance office settings with low-contrast images [Georgeson & G. Sullivan, 1975] – Modified lowpass version [e.g. J. Sullivan, Ray & Miller, 1990] – Angular dependence: cosine function [Sullivan, Miller & Pios, 1993] – Exponential decay [Näsäsen, 1984] • Näsänen’s is best for direct binary search [Kim & Allebach, 2002] 12 Grayscale Halftoning DigitalDigital HalftoningHalftoning MethodsMethods

Clustered Dot Screening Dispersed Dot Screening Error Diffusion AM Halftoning FM Halftoning FM Halftoning 1976

Blue-noise Mask Green-noise Halftoning Direct Binary Search FM Halftoning 1993 AM-FM Halftoning 1992 FM Halftoning 1992

13 OutlineOutline

14 Modeling Grayscale Error Diffusion FloydFloyd--SteinbergSteinberg Grayscale Grayscale ErrorError DiffusionDiffusion

Original Halftone

u(m) current pixel x(m) b(m) + _

Floyd- _ 7/16 + Steinberg shape error h(m) 3/16 5/16 1/16 weights e(m) 15 Modeling Grayscale Error Diffusion ModelingModeling Grayscale Grayscale Error Error Diffusion Diffusion • Goal: Model sharpening and noise shaping • Sigma-delta modulation analysis Q(x)

Linear gain model for quantizer in 1-D 255 [Ardalan and Paulos, 1988] 0 x Apply linear gain model in 2-D 0 128 255 [Kite, Evans & Bovik, 1997]

us(m) Ks us(m) Ks u(m) b(m) Signal Path Q(•) n(m)

un(m) un(m)+n(m) • Uses of linear gain model Noise Path Compensation of frequency distortion Visual quality measures 16 Modeling Grayscale Error Diffusion LinearLinear Gain Gain Model Model for for Quantizer Quantizer

• Best linear fit for Ks between quantizer input u(i,j) and halftone b(i,j) Image Floyd Stucki Jarvis = ()α − 2 Ks arg min u(i,j)b(i,j)barbara 2.01 3.62 3.76 α i,j boats 1.98 4.28 4.93 u(i,j) lena 2.09 4.49 5.32 1 1E{}u(i,j) K=i,j = mandrill 2.03 3.38 3.45 s2u2(i,j)2E{}u2(i,j) Average 2.03 3.94 4.37 i,j – Does not vary much for Floyd-Steinberg

– Can use average value to estimate Ks from only error filter

• Sharpening: proportional to Ks Value of Ks: Floyd Steinberg < Stucki < Jarvis

17 Modeling Grayscale Error Diffusion LinearLinear Gain Gain Model Model for for Error Error Diffusion Diffusion

n(m) B()z K Quantizer STF = s = s model + ()()− X(z) 1 Ks 1 Hz x(m) u(m) b(m) K B()z _ s NTF = n =1− H(z) + N(z) f(m) _ + Lowpass H(z) explains h(m) noise shaping e(m) H(ω) STF NTF 2 1 1 ω ω ω −ω −ω −ω 1 ω1 1 ω1 1 ω1

Also, let Ks =2 Pass low frequencies Highpass response (independent of K ) (Floyd-Steinberg) Enhance high frequencies s 18 Modeling Grayscale Error Diffusion CompensationCompensation of of Sharpening Sharpening

• Adjust by threshold modulation [Eschbach & Knox, 1991] – Scale image by gain L and add it to quantizer input –ForL∈(-1,0], higher value of L, lower the compensation – No compensation when L =0 – Low complexity: one multiplication, one addition per pixel

u(m) b(m) x(m) + _ _ + h(m) e(m) 19 Modeling Grayscale Error Diffusion CompensationCompensation of of Sharpening Sharpening

• Flatten signal transfer function [Kite, Evans, Bovik, 2000] Globally optimum value of L to compensate for sharpening of signal components in halftone based on linear gain model 1 1− K = − = s ∈ − ≥ L 1 L ( 1,0] since Ks 1 K s Ks

Ks is chosen as linear minimum mean squared error estimator of quantizer output Assumes that input and output of quantizer are jointly wide sense stationary stochastic processes Use linear minimum mean squared error estimator for quantizer to adapt L to allow other types of quantizers [Damera-Venkata and Evans, 2001]

20 Modeling Grayscale Error Diffusion

VisualVisual Quality Quality Measures Measures [Kite, Evans, Bovik, 2000]

• Impact of noise on human visual system Signal-to-noise (SNR) measures appropriate when noise is additive and signal independent

Create unsharpened halftone y[m1,m2] with flat signal transfer function using threshold modulation

Weight signal/noise by contrast sensitivity function C[k1,k2] 2 X[k1,k2]C[k1,k2] WSNR (dB) =10log k1,k2 10 ()−2 X[k1,k2] Y[k1,k2] C[k1,k2] k1,k2 Floyd-Steinberg > Stucki > Jarvis at all viewing distances

21 OutlineOutline

22 Compression of Error Diffused Halftones JointJoint BiBi--LevelLevel Experts Experts GroupGroup

• JBIG2 standard (Dec. 1999) • Lossy halftone compression – Binary document printing, – Preserve local average gray faxing, scanning, storage level not halftone – Lossy and lossless coding – Periodic descreening – Models for text, halftone, – High compression of and generic regions ordered dither halftones

ý ú û ÿ ûûúýü þ ü ÿ ÿ úþ øùüûú ÿþýüûúùøû þû ûúüþ ü ûûúý øûþýú þüüüü ûþý ÿþùû ýøþú ûüûú ýøü ýûþ øûþýú ÿþùýû ø þûü ý ø ÿ ÿ þø þ ýùû ý þ ýøü ÿ 23 JBIG2JBIG2 Halftone Halftone Compression Compression Model Model

• JBIG2 assumes that halftones were produced by a small periodic screen • Stochastic halftones are aperiodic

üûý úþþü ûþ 24 Compression of Error Diffused Halftones LossyLossy Compression Compression of of Error Error Diffused Diffused Halftones Halftones

• Proposed method [Valliappan, Evans, Tompkins, Kossentini, 1999] – Reduce noise and artifacts – Achieve higher compression ratios – Low implementation complexity

þ ùû ÿþúüüþý ûýú ûþý ûþ ûþ þ úú 25 Compression of Error Diffused Halftones LossyLossy Compression Compression of of Error Error Diffused Diffused Halftones Halftones

• Fast conversion of error diffused halftones to screened halftones with rate-distortion tradeoffs [Valliappan, Evans, Tompkins, Kossentini, 1999] ÿ þüü ÿ þ ùû ÿ úþü û ùüû þ ûýú ÿ úùøü ýþü úúþú ùüþý ÿ øþøýûü ÿ üúýý øûþú þüüüü úûú øûþú ù ý û ú û ûþý ýøþú üûú ú ÿ ü ÿ ÿ ÿ þüü úý ûú þ Free Parameters ÿ þýü øûþýú ÿ ÿ L sharpening ÿ ûûúýü ÿ ü ÿ ÿ M downsamping factor ÿ ý N grayscale resolution ÿ øùüûú þû 26 Compression of Error Diffused Halftones RateRate--DistortionDistortion TradeoffsTradeoffs

Linear Distortion Measure Weighted SNR for downsampling factor for downsampling factor M ∈ {2,3,4,5,6,7,8} M ∈ {2,3,4,5,6,7,8} (linear distortion removed)

27 OutlineOutline

28 Color Error Diffusion ColorColor Monitor Monitor Display Display Example Example (Palettization) (Palettization)

• YUV color space – Luminance (Y) and chrominance (U,V) channels – Widely used in video compression standards – Human visual system has lowpass response to Y, U, and V • Display YUV on lower-resolution RGB monitor: use error diffusion on Y, U, V channels separably

u(m) YUV to RGB b(m) 24-bit x(m) 12-bit Conversion YUV +_ RGB video _ monitor ÿ + RGB to YUV hh(m)(m) Conversion e(m) 29 Color Error Diffusion NonNon--SeparableSeparable Color Color HalftoningHalftoning forfor DisplayDisplay

• Input image has a vector of values at each pixel (e.g. vector of red, green, and blue components) Error filter has matrix-valued coefficients u(m) Algorithm for adapting x(m) b(m) matrix coefficients + _ based on mean-squared t(m) _ error in RGB space e(m) ÿ + [Akarun, Yardimci, Cetin, 1997] h(m) ÿ t()m = h k ()e()m − k k∈℘ • Design problem matrix vector Given a human visual system model, find the color error filter that minimizes average visible noise power subject to diffusion constraints

30 Color Error Diffusion OptimalOptimal Design Design of of thethe MatrixMatrix--ValuedValued Error Error FilterFilter

• Develop matrix gain model with noise injection n(m) ÿ • Optimize error filterh()m for shaping

2 ÿ ÿ 2 min E[]b ()m =Evÿ()m ∗()I −h m ()∗n()m n Subject to diffusion constraints ÿ h()m1=1 m

where vÿ()m linear model of human visual system * matrix-valued convolution

31 Color Error Diffusion MatrixMatrix Gain Gain Model Model for for the the QuantizerQuantizer

• Replace scalar gain w/ matrix [Damera-Venkata & Evans, 2001]

ÿ ÿ 2 ÿ ÿ −1 K = arg min E b()m − A u m () = C C s ÿ bu uu A ÿ = ÿ u(m) quantizer input K n I b(m) quantizer output – Noise uncorrelated with signal component of quantizer input – Convolution becomes matrix–vector multiplication in frequency domain In one dimension Noise ÿ ÿ component B()z=(I−Hz) ()N()z ()1−H(z)N(z) of output n − ÿÿ ÿ ÿ ÿ 1 KsX(z) Signal B()z()=KI+Hz()K ()−I X()z component s +()()− 1Ks1Hz of output 32 Color Error Diffusion LinearLinear Color Color Vision Vision Model Model

• Pattern-color separable model [Poirson and Wandell, 1993] – Forms the basis for Spatial CIELab [Zhang and Wandell, 1996] – Pixel-based color transformation

B-W

R-G E •

B-Y

Opponent Spatial representation filtering 33 Color Error Diffusion LinearLinear Color Color Vision Vision Model Model

• Undo gamma correction on RGB image • Color separation – Measure power spectral distribution of RGB phosphor excitations – Measure absorption rates of long, medium, short (LMS) cones – Device dependent transformation C from RGB to LMS space – Transform LMS to opponent representation using O – Color separation may be expressed as T = OC ÿ • Spatial filtering included using matrix filter d(m) • Linear color vision model ÿ ÿ vÿ()m=d(m)Twhere d(m) is a diagonal matrix 34 Color Error Diffusion

Sample images and optimum coefficients for sRGB monitor available at: http://signal.ece.utexas.edu/~damera/col-vec.html

Original Image

35 Color Error Diffusion

Floyd-Steinberg Optimum Filter

36 Color Error Diffusion GeneralizedGeneralized Linear Linear Color Color Vision Vision Model Model

• Separate image into channels/visual pathways – Pixel based linear transformation of RGB into color space – Spatial filtering based on HVS characteristics & color space – Best color space/HVS model for vector error diffusion? [Monga, Geisler and Evans, 2003]

C1 • C2 E

C3 Representation in Spatial arbitrary color space filtering 37 Color Error Diffusion ColorColor Spaces Spaces

• Desired characteristics – Independent of display device – Score well in perceptual uniformity [Poynton color FAQ http://comuphase.cmetric.com] – Approximately pattern color separable [Wandell et al., 1993] • Candidate linear color spaces – Opponent color space [Poirson and Wandell, 1993] –YIQ:NTSCvideo Eye more sensitive to luminance; – YUV: PAL video reduce chrominance bandwidth – Linearized CIELab [Flohr, Bouman, Kolpatzik, Balasubramanian, Carrara, Allebach, 1993]

38 Color Error Diffusion MonitorMonitor Calibration Calibration

• How to calibrate monitor? sRGB standard default RGB space by HP and Microsoft Transformation based on an sRGB monitor (which is linear) • Include sRGB monitor transformation T:sRGBÿCIEXYZ ÿOpponent Representation [Wandell & Zhang, 1996] Transformations sRGB ÿ YUV, YIQ from S-CIELab Code at http://white.stanford.edu/~brian/scielab/scielab1-1-1/ • Including sRGB monitor into model enables Web- based subjective testing http://www.ece.utexas.edu/~vishal/cgi-bin/test.html

39 Color Error Diffusion SpatialSpatial Filtering Filtering

• Opponent [Wandell, Zhang 1997] Data in each plane filtered by 2-D separable spatial kernels

• Linearized CIELab, YUV, and YIQ

Luminance frequency response [Näsänenα and Sullivan, 1984] W (ρ)= K(L) e− (L) ρ (Yy ) L average luminance of display ρ radial spatial frequency

Chrominance frequency response [Kolpatzikα and Bouman, 1992] W (ρ) = Ae− ρ (Cx ,Cz ) Chrominance response allows more low frequency chromatic error not to be perceived vs. luminance response 40 Color Error Diffusion SubjectiveSubjective Testing Testing

• Based on paired comparison task – Observer chooses halftone that looks closer to original – Online at www.ece.utexas.edu/~vishal/cgi-bin/test.html

halftone Aoriginal halftone B • In decreasing subjective quality Linearized CIELab > > Opponent > YUV ≥ YIQ 41 Conclusion ColorColor Error Error Diffusion Diffusion • Design of “optimal” color noise shaping filters – Weusethematrixgainmodel[Damera-Venkata and Evans, 2001] • Predicts sharpening • Predicts shaped color halftone noise – Solve for best error filter that minimizes visually weighted average color halftone noise energy – Improve numerical stability of descent procedure • Choice of linear color space – Linear CIELab gives best objective and subjective quality – Future work in finding better transformations • Use color management to generalize device characterization and viewing conditions 42 Conclusion ImageImage Halftoning Halftoning Toolbox Toolbox 1.1 1.1

• Grayscale and color methods Screening Classical diffusion Edge enhanced diff. Green noise diffusion Figures of Merit Block diffusion • Figures of merit Peak SNR Weighted SNR Linear distortion measure Universal quality index

http://www.ece.utexas.edu/~bevans/projects/halftoning/toolbox 43 BackupBackup Slides Slides Grayscale Halftoning ProblemsProblems with with Error Error Diffusion Diffusion

• Objectionable artifacts – Scan order affects results – “Worminess” visible in constant graylevel areas • Image sharpening – Larger error filters due to [Jarvis, Judice & Ninke, 1976] and [Stucki, 1980] reduce worminess and sharpen edges – Sharpening not always desirable: may be adjustable by prefiltering based on linear gain model [Kite, Evans, Bovik, 2000] • Computational complexity – Larger error filters require more operations per pixel – Push towards simple schemes for fast printing 45 Grayscale Halftoning

CorrectingCorrecting Artificial Artificial Textures Textures [Marcu, 1999]

• False textures in shadow and highlight regions • Place dot if minimum distance constraint is met – Raster scan – Avoids computing geometric distance – Scans halftoned pixels in radius of the current pixel – Radius proportional to distance of pixel value from midgray – Scanned pixel location offsets obtained by lookup tables • One lookup table gives number of pixels to scan (256 entries) • One lookup table gives offsets (256 entries) – Affects grayscale values [1, 39] and [216, 254]

46 Grayscale Halftoning

CorrectingCorrecting Artificial Artificial Textures Textures [Marcu, 1999]

47 Grayscale Halftoning

CorrectingCorrecting Artificial Artificial Textures Textures [Marcu, 1999]

48 Grayscale Halftoning DirectDirect Binary Binary Search Search

• Advantages • Disadvantages – Significantly improved – Computational cost and halftone image quality over memory usage is very screening & error diffusion high in comparison to – Quality of final solution is error diffusion and relatively insensitive to screening methods initial halftone, provided is – Increased complexity not error diffused halftone makes it unsuitable for [Lieberman & Allebach, 2000] real-time applications – Application in off-line design such as printing of screening threshold arrays [Kacker & Allebach, 1998]

49 Modeling Grayscale Error Diffusion GrayscaleGrayscale ErrorError DiffusionDiffusion Analysis Analysis • Sharpening caused by a correlated error image [Knox, 1992]

Floyd- Steinberg

Jarvis

Error images Halftones 50 Modeling Grayscale Error Diffusion CompensationCompensation of of Sharpening Sharpening

• Threshold modulation equalivent to prefiltering – Pre-distortion becomes prefiltering with a finite impulse response (FIR) filter with the transfer function G(z) =1+ L(1− H (z)) x(m) u(m) g(m) b(m) + _ _ + h(m) e(m) – Useful if the error diffusion method cannot be altered, e.g. it belongs to another company’s intellectual property

51 Compression of Error Diffused Halftones GrayscaleGrayscale Visual Visual Quality Quality MeasuresMeasures

ω ω ω ω ω ω

• Model degradation as linear filter plus noise • Decouple and quantify linear and additive effects

ω ω • Contrast sensitivity function (CSF) ÿ – Linear shift-invariant model of human visual system – Weighting of distortion measures in frequency domain

52 Compression of Error Diffused Halftones GrayscaleGrayscale Visual Visual Quality Quality Measures Measures

• Estimate linear model by Wiener filter • Weighted Signal to Noise Ratio (WSNR) – Weight noise D(u , v) by CSF C(u , v)

•• LinearLinear Distortion Distortion Measure Measure –– Weight Weight distortion distortion by by input input spectrum spectrum X(uX(u , , v) v) andand CSF CSF C(uC(u , , v) v)

53 Compression of Error Diffused Halftones LossyLossy Compression Compression of of Error Error Diffused Diffused Halftones Halftones

• Results for 512 x 512 Floyd-Steinberg Halftone PrefilterPrefilter LL MM NN θθ LDMLDM WSNRWSNR RatioRatio o XX 0.00.0 44 1717 00o 0.1630.163 15.415.4 dB dB 6.16.1 o YY 0.00.0 44 1717 00o 0.1810.181 16.516.5 dB dB 7.57.5 o YY 0.50.5 44 1717 00o 0.0910.091 16.016.0 dB dB 6.46.4 o YY 1.51.5 44 1717 00o 0.2920.292 14.814.8 dB dB 5.25.2 o YY 0.50.5 66 1919 4545o 0.1160.116 18.718.7 dB dB 6.66.6 o YY 0.50.5 88 3333 4545o 0.1550.155 15.715.7 dB dB 8.28.2 o YY 0.50.5 88 1616 4545o 0.1580.158 14.014.0 dB dB 9.99.9

54 Color Error Diffusion OptimumOptimum Color Color Noise Noise Shaping Shaping

• Vector color error diffusion halftone model – Weusethematrixgainmodel[Damera-Venkata and Evans, 2001] – Predicts signal frequency distortion – Predicts shaped color halftone noise • Visibility of halftone noise depends on – Model predicting noise shaping – Human visual system model (assume linear shift-invariant) • Formulation of design problem – Given human visual system model and matrix gain model, find color error filter that minimizes average visible noise power subject to certain diffusion constraints

55 Color Error Diffusion GeneralizedGeneralized Optimum Optimum Solution Solution

• Differentiate scalar objective function for visual noise shaping w/r to matrix-valued coefficients {}[]()2 d E bn m =()′ =0 ∀i∈℘ xTr xx dh()i • Write norm as trace and differentiate trace using identities from linear algebra { (ÿ ÿ )} { (ÿ ′ÿ ÿ ÿ )} d Tr AX ÿ ′ d Tr X AXB = ÿ ÿ ÿ + ÿ ′ÿ ÿ′ ÿ = A ÿ AXB A XB dX dX { (ÿ ÿ ÿ )} d Tr AXB ÿ ′ÿ ′ ()ÿÿ= ÿÿ () ÿ = A B Tr ABTr BA dX 56 Color Error Diffusion GeneralizedGeneralized Optimum Optimum Solution Solution (cont.) (cont.)

• Differentiating and using linearity of expectation operator give a generalization of the Yule-Walker equations

ÿ′ ÿ − − = ÿ′ ÿ ÿ ÿ − − + + v (k)ran ( i k) v (s)v(q)h(p)rnn ( i s p q) k pqs where a(m) = vÿ(m)∗n(m) • Assuming white noise injection = []′ + ≈ δ () rnn (k) E n(m) n(m k) k = []′ + ≈ ÿ()− ran (k) E a(m) n(mk) vk • Solve using gradient descent with projection onto constraint set 57 Color Error Diffusion ImplementationImplementation of of Vector Vector Color Color Error Error Diffusion Diffusion

H (z) H (z) H (z) rr rg rb ÿ ()= Hz H gr (z) H gg (z) H gb (z) H br (z) Hbg (z) Hbb (z)

H × r gr H + g gg g × H b gb

58 Color Error Diffusion LinearLinear CIELab CIELab Space Space Transformation Transformation [Flohr, Kolpatzik, R.Balasubramanian, Carrara, Bouman, Allebach, 1993] • Linearized CIELab using HVS Model by Yy = 116 Y/Yn – 116 L = 116 f (Y/Yn) – 116 Cx = 200[X/Xn – Y/Yn] a = 200[ f(X/Xn ) – f(Y/Yn ) ] Cz = 500 [Y/Yn – Z/Zn] b = 500 [ f(Y/Yn ) – f(Z/Zn ) ] where f(x) = 7.787x + 16/116 0<= x <= 0.008856 f(x) = (x)1/3 0.008856 <= x <= 1 • Linearize the CIELab Color Space about D65 white point Decouples incremental changes in Yy, Cx, Cz at white point on (L,a,b) values ∇ (L,a,b) = (1/ 3)I (Yy ,Cx ,Cz )

T is sRGB ÿ CIEXYZ ÿLinearized CIELab

59 Color Error Diffusion SpatialSpatial Filtering Filtering

• Opponent [Wandell, Zhang 1997] – Data in each plane filtered by 2-D separable spatial kernels

– Parameters for the three color planes are

Plane Weights wi Spreads ÿi Luminance 0.921 0.0283 0.105 0.133 -0.108 4.336 Red-green 0.531 0.0392 0.330 0.494 Blue-yellow 0.488 0.0536 0.371 0.386 60 Color Error Diffusion SpatialSpatial filtering filtering contdcontd….….

• Spatial Filters for Linearized CIELab and YUV,YIQ based on: Luminance frequency Response [ Nasanen and Sullivan – 1984] W ( ~p)= K(L)exp[−α(L)~p] (Yy ) L – average luminance of display,~p the radial spatial frequency and 1 p α(L) = K(L) = aLb ~p = c ln(L) + d φ s(φ) 1− w 1+ w where p=(u2+v2)1/2 and s( ) = cos(4φ) + 2 2

v w – symmetry parameter = 0.7 and φ = arctan( ) u s(φ) effectively reduces contrast sensitivity at odd multiples of 45 degrees which is equivalent to dumping the luminance error across the diagonals where the eye is least sensitive. 61 Color Error Diffusion SpatialSpatial filtering filtering contdcontd……

Chrominance Frequency Response [Kolpatzik and Bouman – 1992] W ( p) = Aexp[−αp] (Cx ,Cz ) Using this chrominance response as opposed to same for both luminance and chrominance allows more low frequency chromatic error not perceived by the human viewer.

• The problem hence is of designing 2D-FIR filters which most closely match the desired Luminance and Chrominance frequency responses. • In addition we need zero phase as well. Thefilters(5x5and15x15weredesignedusingthefrequencysamplingapproachand were real and circularly symmetric). Filter coefficients at: http://www.ece.utexas.edu/~vishal/halftoning.html • Matrix valued Vector Error Filters for each of the Color Spaces at http://www.ece.utexas.edu/~vishal/mat_filter.html

62 Color Error Diffusion SubjectiveSubjective Testing Testing

• Binomial parameter estimation model – Halftone generated by particular HVS model considered superior if picked over another 60% or more of the time – Need 960 paired comparison of each model to determine results within tolerance of 0.03 with 95% confidence – Four models would correspond to 6 comparison pairs, total 6 x 960 = 5760 comparisons needed – Observation data collected from over 60 subjects each of whom judged 96 comparisons • Data resulted in unique rank order of four models