Digital image processing Chapter 3. Image sampling and quantization IMAGE SAMPLING AND IMAGE QUANTIZATION
1. Introduction
2. Sampling in the two-dimensional space Basics on image sampling The concept of spatial frequencies Images of limited bandwidth Two-dimensional sampling Image reconstruction from its samples The Nyquist rate. The alias effect and spectral replicas superposition The sampling theorem in the two-dimensional case Non-rectangular sampling grids and interlaced sampling The optimal sampling Practical limitations in sampling and reconstruction
3. Image quantization 4. The optimal quantizer The uniform quantizer 5. Visual quantization Contrast quantization Pseudo-random noise quantization Halftone image generation Color image quantization
Digital image processing Chapter 3. Image sampling and quantization
1. Introduction
f(x,y) fs(x,y) u(m,n ) Sampling Quantization Computer
Digitization
u(m,n) D/A Computer Display
conversion
Analog display
Fig 1 Image sampling and quantization / Analog image display
Digital image processing Chapter 3. Image sampling and quantization
2. Sampling in the two-dimensional space
Basics on image sampling
f(x,y)
x
y Digital image processing Chapter 3. Image sampling and quantization The concept of spatial frequencies
- Grey scale images can be seen as a 2-D generalization of time-varying signals (both in the analog and in the digital case); the following equivalence applies:
1-D signal (time varying) 2-D signal (grey scale image) Time coordinate t Space coordinates x,y Instantaneous value: f(t) Brightness level, point-wise: f(x,y) A 1-D signal that doesn’t vary in time (is constant) A perfectly uniform image (it has the same = has 0 A.C. component, and only a D.C. brightness in all spatial locations); the D.C. component component = the brightness in any point The frequency content of a 1-D signal is The frequency content of an image (2-D signal) is proportional to the speed of variation of its proportional to the speed of variation of its instantaneous value in time: instantaneous value in space:
νmax ~ max(df/dt) νmax,x ~ max(df/dx); νmax,y ~ max(df/dy)
=> νmax,x , νmax,y = “spatial frequencies” Discrete 1-D signal: described by its samples => a Discrete image (2-D signal): described by its vector: u=[u(0) u(1) … u(N-1)], N samples; the samples, but in 2-D => a matrix: U[M×N], position of the sample = the discrete time moment U={u(m,n)}, m=0,1,…,M-1; n=0,1,…,N-1. The spectrum of the time varying signal = the real The spectrum of the image = real part of the part of the Fourier transform of the signal, F(ω); Fourier transform of the image = 2-D ω=2πν. generalization of 1-D Fourier transform, F(ωx,ωy)
ωx=2πνx; ωy=2πνy Digital image processing Chapter 3. Image sampling and quantization
Images of limited bandwidth
• Limited bandwidth image = 2-D signal with finite spectral support:
F(νx, νy) = the Fourier transform of the image:
jx x jy y j2 x x y y F x, y f x, ye e dxdy f x, ye dxdy.
|F(νx, νy)| νy
F( x, y ) 0,| x | x0,| y | y0
νy0 -ν ν ν x0 x x0 νx
νx0 -νy0 -ν y0 νy The Fourier transform of the The spectral support region limited spectrum image
The spectrum of a limited bandwidth image and its spectral support Digital image processing Chapter 3. Image sampling and quantization
Two-dimensional sampling (1)
The common sampling grid = the uniformly spaced, rectangular grid: y
g(x,y)(x, y) (x mx, y ny)
x m n y
x
Image sampling = read from the original, spatially continuous, brightness function f(x,y), only in the black dots positions ( only where the grid allows):
f (x, y), x mx, y ny fs (x, y) , 0, otherwise m,nZ.
fs (x, y) f (x, y)g(x,y)(x, y) f (mx,ny) (x mx, y ny) m n Digital image processing Chapter 3. Image sampling and quantization
Two-dimensional sampling (2)
• Question: How to choose the values Δx, Δy to achieve: -the representation of the digital image by the min. number of samples, -at (ideally) no loss of information? (I. e.: for a perfectly uniform image, only 1 sample is enough to completely represent the image => sampling can be done with very large steps; on the opposite – if the brightness varies very sharply => very many samples needed) The sampling intervals Δx, Δy needed to have no loss of information depend on the spatial frequency content of the image. Sampling conditions for no information loss – derived by examining the spectrum of the image by performing the Fourier analysis:
fs (x, y) f (x, y)g(x,y)(x, y) Fourier transform :
j2 y j2 y F ( , ) f (x, y)e j2 x x e y dxdy f (x, y) g (x, y)e j2 x x e y dxdy. S x y S (x,y)
The sampling grid function g(Δx, Δy) is periodical with period (Δx, Δy) => can be expressed by its Fourier series expansion: k l j2 x j2 y x y g(x,y)(x, y) a(k,l)e e , k l where: k l x y j2 x j2 y 1 1 x y a(k,l) g(x,y)(x, y)e e dxdy. x y 0 0 Digital image processing Chapter 3. Image sampling and quantization
Two-dimensional sampling (3)
Since: 1, if x y 0 for (x, y)[0;x)[0;y), g(x,y)(x, y) , 0, otherwise 1 1 a(k,l) , k,l ;. x y
Therefore the Fourier transform of fS is:
2 kx 2 ly j j 1 x y j2 x x j2 y y FS ( x, y ) f (x, y) e e e e dxdy k l x y k l j2 x x j2 y y 1 x y F ( , ) f (x, y)e e dxdy S x y x y k l k l j2 x x j2 y y 1 x y FS ( x, y ) f (x, y)e e dxdy x y k l
1 k l
FS ( x, y ) F x , y . x y k l x y The spectrum of the sampled image = the collection of an infinite number of scaled spectral replicas of the spectrum of the original image, centered at multiples of spatial frequencies 1/Δx, 1/ Δy.
Digital image processing Chapter 3. Image sampling and quantization
Original image Original image spectrum – 3D Original image spectrum – 2D
2-D rectangular sampling grid Sampled image spectrum – 3D Sampled image spectrum – 2D Digital image processing Chapter 3. Image sampling and quantization Image reconstruction from its samples
1/x y
ys 1 1 xs 2 x0 , ys 2 y0 x ,y - ys y0 2 x0 2 y0 2 1/y 2y0 y0 1 ,( , ) H( , ) x y x y ( xs ys ) 1 3 0, otherwise
2x0 ~ F(x, y ) H(x, y )Fs (x, y ) F(x, y ) x0 xs ~ xs-x0 x xs f x, y hx, y fs x, y
Fig.4 The sampled image spectrum ~ f x, y fs mx,nyhx mx, y ny m n Let us assume that the filtering region R is rectangular, at the middle distance between two spectral replicas: 1 xs ys siny , x and y sinxxs ys H(x, y ) (xs ys ) 2 2 hx, y xxs y ys 0, otherwise ~ sinxxs m siny ys n f x, y fsmx,nyhx mx, y ny fsmx,ny m n m n xxs m y ys n Digital image processing Chapter 3. Image sampling and quantization
~ sina f x, y fs mx,nysincxxs msincy ys n, where sinca m n a
Since the sinc function has infinite extent => it is impossible to implement in practice the ideal LPF it is impossible to reconstruct in practice an image from its samples without error if we sample it at the Nyquist rates.
Practical solution: sample the image at higher spatial frequencies + implement a real LPF (as close to the ideal as possible).
1-D sinc function 2-D sinc function Digital image processing Chapter 3. Image sampling and quantization
The Nyquist rate. The aliasing. The fold-over frequencies
xs 2 x0 , ys 2 y0 y
y0
0 2y0
2x0 The Moire effect
0 x0 x
Fig. 5 Aliasing – fold-over frequencies Note: Aliasing may also appear in the reconstruction process, due to the imperfections of the filter! How to avoid aliasing if cannot increase the sampling frequencies? “Jagged” boundaries By a LPF on the image applied prior to sampling! Digital image processing Chapter 3. Image sampling and quantization
Non-rectangular sampling grids. Interlaced sampling grids n n
νy F(ν , ν )=1 1/2 x y 2 1/ 2 1 νx 1 2 2 -1/2 1/2
-1 0 1 1 2 m -2 -1 0 1 2 m 3 -1/2
a) Image spectrum b) Rectangular grid G1 c) Interlaced grid G2
2 2 1
x x
-1 0 1 1 x x 1
d) The spectrum using G1 e) The spectrum using G2
Interlaced sampling f (x, y) a Optimal sampling = Karhunen-Loeve expansion: m,n m,n m,n0
Digital image processing Chapter 3. Image sampling and quantization
Image reconstruction from its samples in the real case
The question is: what to fill in the “interpolated” (new) dots? Several interpolation methods are available; ideally – sinc function in the spatial domain; in practice – simpler interpolation methods (i.e. approximations of LPFs).
Digital image processing Chapter 3. Image sampling and quantization
Image interpolation filters:
The 1-D Graphical The 2-D Frequency interpolation representation p(x) interpolation response pa(1,0) function function pa(1,2) pa(x,y)=p(x)p(y)
1/x 1 Rectangular 1 2 (zero-order 1 x p0 (x)p0 (y) sinc sinc rect 2x0 2y0 -x/2 x/2 x x filter) 0 x x 0 p0(x) 4x0
1/x 1 Triangular 1 x 2 tri 1 2 (first order p (x) p (y) sinc sinc x x 1 1 2 2 x0 y0 -x x x x filter) 0 0 p0 (x) p0 (x) 4 p1(x) x0
1 n-order filter n1 p (x)p (y) 1 2 n=2, quadratic p0 (x)p0 (x) n n sinc sinc 2x0 2y0 x x n=3, cubic spline 0 n convolu@ii 0 4 pn(x) x0
1 2 1 x 1 (x2 y2 ) exp 2 exp Gaussian 2 2 2 2 2 2 2 2 2 2 2 exp 2 (1 2 ) x x pg(x) 0 0 2
1
1 x 1 x y sinc sinc sinc rect 1 rect 2 x xy x y Sinc 0 x x 2x0 2y0 0 2x 2x0
Digital image processing Chapter 3. Image sampling and quantization
Image interpolation examples:
1. Rectangular (zero-order) filter, or nearest neighbour filter, or box filter:
1/x
-x/2 x/2 x 0
Original Sampled Reconstructed
Digital image processing Chapter 3. Image sampling and quantization
Image interpolation examples:
2. Triangular (first-order) filter, or bilinear filter, or tent filter:
1/x
-x x x 0
Original Sampled Reconstructed
Digital image processing Chapter 3. Image sampling and quantization
Image interpolation examples:
3. Cubic interpolation filter, or bicubic filter – begins to better approximate the sinc function:
x 0 2x
Original Sampled Reconstructed
Digital image processing Chapter 3. Image sampling and quantization Practical limitations in image sampling and reconstruction
Scanning ~ Input g(x,y) Ideal sampler gs(x,y) Analog display g (x,y) image system x,y pa(-x,-y) aperture ps(-x,-y) Real scanner model
Fig. 7 The block diagram of a real sampler & reconstruction (display) system
Pa(1,0) Interpolation filter or display system spectrum
- /2 /2 xs xs 1
Sampled Input image spectrum image Spectral losses spectrum Reconstructed image spectrum
- 0 0
-xs/2 xs/2
Interpolation error
Fig. 8 The real effect of the interpolation
Digital image processing Chapter 3. Image sampling and quantization
3. Image quantization
3.1. Overview
Quantizer’s
u u’ output Quantizer rL
rk
t t t1 2 k tk+1 tL+1
Quantization
r2 error
r1
Fig. 9 The quantizer’s transfer function
Digital image processing Chapter 3. Image sampling and quantization
3.2. The uniform quantizer The quantizer’s design:
• Denote the input brightness range: u lmin;LMax • Let B – the number of bits of the quantizer => L=2B reconstruction levels • The expressions of the decision levels: E.g. B=2 => L=4 Uniform quantizer transfer function
t1 lmin; tL1 LMax r4=224
tk tk1 tk1 tk constant q r3=160 tL1 t1 LMax lmin q ,tk tk1 q q
L L r2=96 Reconstruction Reconstruction levels • The expressions of the reconstruction levels:
r1=32
tk tk1 q rk rk tk t1=0 t2=64 t3=128 t4=192 t5=256 2 2 Decision levels • Computation of the quantization error: for a given image of size M×N pixels, U – non-quantized, and U’ – quantized => we estimate the MSE:
M 1N 1 L tk 1 1 2 2 um,nu'm,n (u rk ) hlin,U (u)du MN m0 n0 k1 tk Digital image processing Chapter 3. Image sampling and quantization
Examples of uniform quantization and the resulting errors: B=1 => L=2 Uniform quantizer transfer function Non-quantized image Quantized image
r2=192 Reconstruction Reconstruction levels
r1=64
t1=0 t2=128 t3=256 Decision levels Quantization error; MSE=36.2
1000
900
800
700
600
500
400
300
200
100
0
0 50 100 150 200 250 The histogram of the non-quantized image Digital image processing Chapter 3. Image sampling and quantization
Examples of uniform quantization and the resulting errors: B=2 => L=4 Non-quantized image Quantized image Uniform quantizer transfer function
r4=224
r3=160
r2=96 Reconstruction Reconstruction levels
r1=32
t1=0 t2=64 t3=128 t4=192 t5=256 Decision levels Quantization error; MSE=15 1000
900
800
700
600
500
400
300
200
100
0
0 50 100 150 200 250
The histogram of the non-quantized image Digital image processing Chapter 3. Image sampling and quantization
Examples of uniform quantization and the resulting errors: B=3 => L=8; false contours present Uniform quantizer transfer function Non-quantized image Quantized image r8=240
r7=208
r6=176
r5=144
r4=112
Reconstruction Reconstruction levels r3=80
r2=48
r1=16
t1=0 t2=32 t3=64 t4=96 t5=128 t6=160 t7=192 t8=224 t9=256 Decision levels 1000 Quantization error; MSE=7.33 900
800
700
600
500
400
300
200
100
0
0 50 100 150 200 250 The histogram of the non-quantized image Digital image processing Chapter 3. Image sampling and quantization
3.2. The optimal (MSE) quantizer (the Lloyd-Max quantizer)
t t L 1 L i 1 2 2 (u r )2 h (u)du e E[(u u' ) ] (u u' ) hu(u)du i u i1 t t1 i
2 2 (tk rk1) (tk rk ) hu (tk ) 0 tk
tk1 2 (u rk )hu (u)du 0 1 k L rk tk
tk1 uhu (u)du r r k k 1 tk tk rk t Eu |u k 2 k1 hu (u)du
tk 1 p (u) p (t ), t (t t ), t u t u u j j 2 j j1 j j1 zk t1 1/ 3 A [hu (u)] du
t1 tk1 t1 tL1 1/ 3 [hu (u)] du
t1 Digital image processing Chapter 3. Image sampling and quantization
3 t 1 L1 [h (u)]1/ 3 du 2 u 12L t 1
pu(u)
u t t 1 2 tj tj+1 tL+1
2 1 (u ) hu (u ) exp 2 2 (Gaussian) , or hu (u) exp u (Laplacian) 2 2 2
2 2 ( variance , - mean) Digital image processing Chapter 3. Image sampling and quantization
Examples of optimal quantization and the quantization error: B=1 => L=2 Functia de transfer a cuantizorului optimal Non-quantized image Quantized image
r2=153 Nivelele de reconstructiede Nivelele
r1=24
t1=0 t2=89 t3=256 Nivelele de decizie The quantization error; The evolution of MSE 1000 MSE=19.5 in the optimization, starting 900 from the uniform quantizer 800 38 700 36 600 34 500 32 400 30 300 28 200 26
100 24
0 22
0 50 100 150 200 250 20 18 The non-quantized image histogram 1 2 3 4 5 6 7 8 Digital image processing Chapter 3. Image sampling and quantization
Examples of optimal quantization and the quantization error: B=2 => L=4 Functia de transfer a cuantizorului optimal Non-quantized image Quantized image
r4=181
r3=156
r2=115 Nivelele de reconstructiede Nivelele
r1=20
t1=0 t2=68 t3=136 t4=169 t5=256 Nivelele de decizie The quantization error; The evolution of MSE 1000 MSE=9.6 in the optimization, starting 900
800 from the uniform quantizer
700 15
600 14 500
400 13
300 12 200 11 100
0 10
0 50 100 150 200 250 9 The non-quantized image histogram 1 2 3 4 5 6 7 8 9 10 Digital image processing Chapter 3. Image sampling and quantization
Examples of optimal quantization and the quantization error: B=3 => L=8 Non-quantized image Quantized image
Functia de transfer a cuantizorului optimal
r8=224
r7=181 r6=165 r5=147
r4=125
r3=101 Nivelele de reconstructiede Nivelele
r2=54
r1=14
t1=0 t2=34 t3=78 t4=113 t5=136 t6=156t7=173 t8=203 t9=256 Nivelele de decizie
1000 The quantization error; The evolution of MSE 900 MSE=5 in the optimization, starting 800 from the uniform quantizer 700 7.5 600
500 7
400 6.5 300 6 200
100 5.5 0 5 0 50 100 150 200 250
4.5 The non-quantized image histogram 0 2 4 6 8 10 12 14 Digital image processing Chapter 3. Image sampling and quantization
3.3. The uniform quantizer = the optimal quantizer for the uniform grey level distribution: 1 , t1 u tL1 hu (u) tL1 t1 0 otherwise
2 2 (tk 1 tk ) tk 1 tk rk 2(tk 1 tk ) 2
tk 1 tk 1 t tk tk1 tk1 tk constant q k 2 t t q q L1 1 , t t q, r t L k k 1 k k 2 1 q/2 q2 u2du q q/2 12
2B 2B 2 2 ,therefore SNR 10log102 6 B dB u Digital image processing Chapter 3. Image sampling and quantization
3.4. Visual quantization methods
• In general – if B<6 (uniform quantization) or B<5 (optimal quantization) => the "contouring" effect (i.e. false contours) appears in the quantized image. • The false contours (“contouring”) = groups of neighbor pixels quantized to the same value <=> regions of constant gray levels; the boundaries of these regions are the false contours. • The false contours do not contribute significantly to the MSE, but are very disturbing for the human eye => it is important to reduce the visibility of the quantization error, not only the MSQE. Solutions: visual quantization schemes, to hold quantization error below the level of visibility. Two main schemes: (a) contrast quantization; (b) pseudo-random noise quantization
Uniform quantization, B=4 Optimal quantization, B=4 Uniform quantization, B=6 Digital image processing Chapter 3. Image sampling and quantization
3.4. Visual quantization methods
a. Contrast quantization
• The visual perception of the luminance is non-linear, but the visual perception of contrast is linear uniform quantization of the contrast is better than uniform quantization of the brightness contrast = ratio between the lightest and the darkest brightness in the spatial region just noticeable changes in contrast: 2% => 50 quantization levels needed 6 bits needed with a uniform quantizer (or 4-5 bits needed with an optimal quantizer)
Brightness
-1 f( ) c MMSE c’ f () u’ u brightness- quantizer contrast - contrast brightness
c ln(1 u),0 u 1; typ. 6...18, / ln(1 ) or
c u ; typ. 1; 1/ 3
Digital image processing Chapter 3. Image sampling and quantization
Examples of contrast quantization:
1/3 • For c=u : The transfer function of the contrast quantizer 250 1 0.9 200 0.8
0.7 150 0.6
0.5 Reconstruction levels 100 0.4
0.3 50 0.2 0.1 0 t2=3.9844t1=0 t3=31.875 t4=107.5781 t5=255 Decision levels 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
1000 900
800
700
600
500
400
300
200
100
0
0 50 100 150 200 250 Digital image processing Chapter 3. Image sampling and quantization
Examples of contrast quantization:
The transfer function of the contrast quantizer • For the log transform: 250 5
4.5 200 4
3.5 150 3 2.5 100 2 Reconstruction Reconstruction levels 1.5 1 50 0.5
0 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 t1=0 t2=46.0273 t3=102.2733 t4=171.0068 t5=255 Decision levels 1000
900
800 700 600 500
400
300
200
100
0
0 50 100 150 200 250 Digital image processing Chapter 3. Image sampling and quantization
b. Pseudorandom noise quantization (“dither”)
u’(m,n) u(m,n) v(m,n) K bits v’(m,n) quantizer + + + -
(m,n)
Uniformly distributed pseudorandom noise, [-A,A] Large dither amplitude
Uniform quantization, B=4 Prior to dither subtraction Small dither amplitude Digital image processing Chapter 3. Image sampling and quantization
Fig. 13 a b a. 3 bits quantizer =>visible false contours; c d b. 8 bits image, with pseudo-random noise added in the range [-16,16]; c. the image from Figure b) quantized with a 3 bits quantizer d. the result of subtracting the pseudo-random noise from the image in Figure c)
Digital image processing Chapter 3. Image sampling and quantization
Halftone images generation
Thresholding v’ Luminance v(m,n) v’(m,n) Halftone + + display 0u(m,n)A + A 0(m,n)A
Pseudorandom matrix Fig.14 Digital generation of halftone images
52 44 36 124 132 140 148 156 60 4 28 116 200 228 236 164 40 60 150 90 10 68 12 20 108 212 252 244 172 80 170 240 200 110 76 84 92 100 204 196 188 180 140 210 250 220 130 H2 H1 132 140 148 156 52 44 36 124 120 190 230 180 70 200 228 236 164 60 4 28 116 20 100 160 50 30 212 252 244 172 68 12 20 108 204 196 188 180 76 84 92 100 Demo: http://markschulze.net/halftone/index.html
Fig.15 Halftone matrices
Digital image processing Chapter 3. Image sampling and quantization
Fig.3.16 Digital image processing Chapter 3. Image sampling and quantization
Color images quantization
R ’ RN T1 T1’ N
Quantizer
Color space GN T2 T2’ GN’ Color space inverse Quantizer transformation transformation
BN’ T3 BN T3’ Quantizer
Fig.17 Color images quantization