Interpolated Halftoning, Rehalftoning, and Halftone Compression

INTERPOLATED HALFTONING, REHALFTONING, AND HALFTONE COMPRESSION

Prof. Brian L. Evans [email protected] http://www.ece.utexas.edu/~bevans

Collaboration with Dr. Thomas D. Kite and Mr. Niranjan Damera-Venkata

Laboratory for Image and Video Engineering The University of Texas at Austin http://anchovy.ece.utexas.edu/ OUTLINE n Introduction to halftoning n Halftoning by error diffusion 4 Linear gain model 4 Modified error diffusion n Interpolated halftoning n Rehalftoning n JBIG2 halftone compression n Conclusions

2 INTRODUCTION: HALFTONING n Was analog, now digital processing n Wordlength reduction for images 4 8-bit to 1-bit for grayscale 4 24-bit RGB to 8-bit for color displays 4 24-bit RGB to CMYK for color printers n Applications 4 Printers 4Digital copiers 4 Liquid crystal displays 4 Video cards n Halftoning methods 4 Screening 4 Error diffusion 4 Direct binary search 4 Hybrid schemes

3 EXAMPLE HALFTONES

Original image Direct binary search

Clustered dot screen Floyd Steinberg

Dispersed dot screen Modified Diffusion 4 FOURIER TRANSFORMS

Original image Direct binary search

Clustered dot screen Floyd Steinberg

Dispersed dot screen Modified Diffusion 5 ERROR DIFFUSION n 2-D delta-sigma modulator

n Noise shaping feedback coder

x i; j

xi; j y i; j

ei; j

H z +

n Error filter

5 3 1

16 16 16

n Raster scan order

P P P P P P P

P P P P P P

P P = Past

P P F F F

P F = Future

F F F F F F F

F F F F F F F n Serpentine scan also used

6 ERROR DIFFUSION (cont.)

n Quantizer

0; xi; j < 0:5

y i; j =

1; x i; j 0:5

n Governing equations

ei; j = y i; j x i; j

x i; j = xi; j hi; j ei; j n Non-linearity difficult to analyze n Linearize quantizer

[Kite, Evans, Bovik & Sculley 1997]

ni; j Noise

Path

0 0

K K n i; j + ni; j n i; j

n n

Signal

0 0

x i; j K K x i; j s s Path n Separate signal and noise paths [Ardalan & Paulos 1987]

7 LINEAR GAIN MODEL n Quantization error correlated with input [Knox 1992]

Floyd-Steinberg Jarvis, Judice & Ninke n Least squares fit of quantizer input

to output defines signal gain

E [jx i; j j]

K =

0 2

2E [x i; j ]

K constant

Signal gain: s

n K =1

Noise gain: n

8 GAIN MODEL PREDICTIONS

n Noise transfer function (NTF)

NTF=1Hz

1.7 1.7 1.5 1.5

1 1

Magnitude 0.5 Magnitude 0.5

0 0 1 1 1 1 0.5 0.5 0.5 0.5

Frequency f / f 0 0 Frequency f / f Frequency f / f 0 0 Frequency f / f y N x N y N x N Predicted Measured

n Signal transfer function (STF)

STF =

1+K 1H z s

4 8

6 3

4 2 Magnitude Magnitude 2 1 1 1 1 1 0.5 0.5 0.5 0.5

Frequency f / f 0 0 Frequency f / f Frequency f / f 0 0 Frequency f / f y N x N y N x N

Floyd-Steinberg Jarvis et al. 9 Predicting Signal Gain Ks

n Predict Ks from error filter as:

= − Ks 1.17R 0.2

π ω ω 2 ω ω ∫ X ( 1, 2 ) d 1d 2 = −π R π ω ω 2 ω ω 2 ω ω ∫ X ( 1, 2 ) H ( 1, 2 ) d 1d 2 −π

X(ω1,ω2) = Fourier Transform of input to error filter

H(ω1,ω2) = Fourier Transform of error filter

But X(ω1,ω2) is the error spectrum, so

ω ω = − ω ω X ( 1, 2 ) 1 H ( 1, 2 )

10 MODIFIED ERROR DIFFUSION n Efficient method of adjusting

sharpness [Eschbach & Knox 1991]

0 00

x i; j x i; j

xi; j y i; j

ei; j

H z +

n Equivalent circuit: pre-filter

x i; j

xi; j y i; j

ei; j

H z

Gz=1+L1 H z

n L can be chosen to compensate for frequency distortion

UNSHARPENED HALFTONES

1 K

s =

n L

If then STF=1 (flat)

K s n Accounts for frequency distortion

Original image Jarvis halftone

Unsharpened halftone Residual 12 INTERPOLATION

n Image resizing n Different methods (increasing cost) 4 Nearest neighbor (NN) 4Bilinear (BL) n Nearest neighbor, bilinear methods 4 Low computational cost 4 Artifacts masked by quantization noise in halftone 4 Correct blurring by using modified error diffusion

Interpolation Halftoning

Original Halftone

I(z) F(z)

13 INTERPOLATION

n Design L for flat transfer function using linear gain model (L is constant for a given interpolator) n Compute transfer function of interpolation by M − − 1− z M 1− z M  I (z) =  x  x  NN  −  −   1 zx  1 z y  − 2 − 2 1− z M  1− z M  I (z) =  x   x  BL  −   −   1 zx   1 z y  n Compute signal transfer function K (1+ L(1− H (z))) F(z) = s + − 1 (Ks 1)H (z) n Compute L to flatten the end-to-end transfer function of the system

14 INTERPOLATION RESULTS

Nearest neighbor ×2 Bilinear ×2

1.2 1.2 1 1 0.8 0.8 0.6 0.6 0.4 0.4 Magnitude Magnitude 0.2 0.2 0 0 0 0 0.2 1 0.2 1 0.4 0.8 0.4 0.8 0.6 0.6 0.6 0.4 0.6 0.4 0.8 0.2 0.8 0.2 1 1 0 Frequency f / f 0 Frequency f / f Frequency f / f y N Frequency f / f y N x N x N Transfer function Transfer function L = –0.0105 L = 0.340

15 REHALFTONING

n Halftone conversion, manipulation n Assume input and output are error diffused halftones 4 Blurring corrected by using modified error diffusion 4 Noise leakage masked by halftoning 4 64 operations per pixel n For a 512 x 512 image 416 RISC MIPS 4 0.4 s on a 167 MHz Ultra-2 workstation

Halftoning Inverse halftoning Re-halftoning

Halftone Original

16 REHALFTONING (cont.) n Halftone conversion, manipulation n Error diffused halftones n Fixed lowpass inverse halftoning filter, compromise cut-off frequency 4 Noise leakage masked by halftoning 4 Correct blur by modified error diffusion

4 Computationally efficient

Original Inverse

Mo di ed

Halftone

image halftone halftone

K 1 + L 1 H z

1+K 1H z 1+ K 1H z

s s

17 REHALFTONING (cont.)

n Use linear gain model to design L for flat response

n Use approximation for digital

j! 2

1+j! ! =2 frequency: e n Inverse halftoning filter is a simple separable FIR filter n L is computed to flatten the end to end transfer function of the system n We need to know halftoning filter coefficients for this scheme n Improve halftoning results using knowledge of type of halftone being rehalftoned

18 REHALFTONING RESULTS

Original image Rehalftone

1.2 1 0.8 0.6 0.4 Magnitude 0.2 0 0 0.2 1 0.4 0.8 0.6 0.6 0.4 0.8 0.2 1 0 Frequency f / f Frequency f / f y N x N Signal transfer function 19 THE JBIG2 STANDARD

n Lossy/lossless coding of bi-level text and halftone data

Symbol Symbol Memory region dictionary decoder decoding

Document Generic refinement

Halftone Halftone region Memory dictionary decoder decoding n Scan vs. random mode

20 THE JBIG2 STANDARD (cont.) n Bi-level text coding 4 Hard pattern matching (lossy) 4 Soft pattern matching (lossless or near lossless) may be context based n Halftone coding 4 Direct halftone compression 4Context based halftone coding 4Inverse halftoning and compression of grayscale image n Implications 4 Printers, fax machines and scanners, will need to decode JBIG2 bitstreams 4 Fast decoding may require dedicated hardware and embedded software 4Need for low complexity, low memory solutions

21 PROBLEMS TO BE SOLVED n JBIG2 compression of halftones 4 Compress halftone directly, using a dictionary of patterns, or 4Convert halftone to grayscale (inverse halftoning) and compress grayscale image n Efficient coding of halftone data 4 Fax machines 4Digital archiving, scanning, and copying n Fast algorithms for JBIG2 codec 4 Interpolated halftoning in decoder 4 Rehalftoning in codec

22 PROBLEMS TO BE SOLVED n JBIG2 embedded decoders 4 Low memory requirements 4Low computational complexity 4 High parallelism n Inverse halftoning: a robust solution for lossy coding of halftones 4 Rendering device can use a different halftoning scheme than encoder 4 Multiresolution halftone rendering (archive browsing) 4 High halftone compression ratios (6:1) 4 Quality enhancement if the encoder halftoning method is transmitted n Low-cost embedded implementations

23 CONCLUSIONS n Linear gain model of error diffusion 4 Validate accuracy of quantizer model 4 Link between filter gain and signal gain n Rehalftoning and interpolation 4 Efficient algorithms 4 Impact on emerging JBIG2 standard n Web site for software and papers 4 http://www.ece.utexas.edu/~bevans/ projects/inverseHalftoning.html