Digital image processing Chapter 3. Image sampling and quantization IMAGE SAMPLING AND IMAGE QUANTIZATION 1. Introduction 2. Sampling in the two-dimensional space Basics on image sampling The concept of spatial frequencies Images of limited bandwidth Two-dimensional sampling Image reconstruction from its samples The Nyquist rate. The alias effect and spectral replicas superposition The sampling theorem in the two-dimensional case Non-rectangular sampling grids and interlaced sampling The optimal sampling Practical limitations in sampling and reconstruction 3. Image quantization 4. The optimal quantizer The uniform quantizer 5. Visual quantization Contrast quantization Pseudo-random noise quantization Halftone image generation Color image quantization Digital image processing Chapter 3. Image sampling and quantization 1. Introduction f(x,y) fs(x,y) u(m,n ) Sampling Quantization Computer Digitization u(m,n) D/A Computer Display conversion Analog display Fig 1 Image sampling and quantization / Analog image display Digital image processing Chapter 3. Image sampling and quantization 2. Sampling in the two-dimensional space Basics on image sampling f(x,y) x y Digital image processing Chapter 3. Image sampling and quantization The concept of spatial frequencies - Grey scale images can be seen as a 2-D generalization of time-varying signals (both in the analog and in the digital case); the following equivalence applies: 1-D signal (time varying) 2-D signal (grey scale image) Time coordinate t Space coordinates x,y Instantaneous value: f(t) Brightness level, point-wise: f(x,y) A 1-D signal that doesn’t vary in time (is constant) A perfectly uniform image (it has the same = has 0 A.C. component, and only a D.C. brightness in all spatial locations); the D.C. component component = the brightness in any point The frequency content of a 1-D signal is The frequency content of an image (2-D signal) is proportional to the speed of variation of its proportional to the speed of variation of its instantaneous value in time: instantaneous value in space: νmax ~ max(df/dt) νmax,x ~ max(df/dx); νmax,y ~ max(df/dy) => νmax,x , νmax,y = “spatial frequencies” Discrete 1-D signal: described by its samples => a Discrete image (2-D signal): described by its vector: u=[u(0) u(1) … u(N-1)], N samples; the samples, but in 2-D => a matrix: U[M×N], position of the sample = the discrete time moment U={u(m,n)}, m=0,1,…,M-1; n=0,1,…,N-1. The spectrum of the time varying signal = the real The spectrum of the image = real part of the part of the Fourier transform of the signal, F(ω); Fourier transform of the image = 2-D ω=2πν. generalization of 1-D Fourier transform, F(ωx,ωy) ωx=2πνx; ωy=2πνy Digital image processing Chapter 3. Image sampling and quantization Images of limited bandwidth • Limited bandwidth image = 2-D signal with finite spectral support: F(νx, νy) = the Fourier transform of the image: jx x jy y j2 x x y y F x, y f x, ye e dxdy f x, ye dxdy. |F(ν ν )| x, y νy F( x, y ) 0,| x | x0,| y | y0 νy0 -ν ν ν x0 x x0 νx νx0 -νy0 -νy0 νy The Fourier transform of the The spectral support region limited spectrum image The spectrum of a limited bandwidth image and its spectral support Digital image processing Chapter 3. Image sampling and quantization Two-dimensional sampling (1) The common sampling grid = the uniformly spaced, rectangular grid: y g(x,y)(x, y) (x mx, y ny) m n x y x Image sampling = read from the original, spatially continuous, brightness function f(x,y), only in the black dots positions ( only where the grid allows): f (x, y), x mx, y ny fs (x, y) , 0, otherwise m,nZ. fs (x, y) f (x, y)g(x,y)(x, y) f (mx,ny) (x mx, y ny) m n Digital image processing Chapter 3. Image sampling and quantization Two-dimensional sampling (2) • Question: How to choose the values Δx, Δy to achieve: -the representation of the digital image by the min. number of samples, -at (ideally) no loss of information? (I. e.: for a perfectly uniform image, only 1 sample is enough to completely represent the image => sampling can be done with very large steps; on the opposite – if the brightness varies very sharply => very many samples needed) The sampling intervals Δx, Δy needed to have no loss of information depend on the spatial frequency content of the image. Sampling conditions for no information loss – derived by examining the spectrum of the image by performing the Fourier analysis: fs (x, y) f (x, y)g(x,y)(x, y) Fourier transform : j2 x x j2 y y j2 x x j2 y y FS ( x, y ) fS (x, y)e e dxdy f (x, y) g(x,y)(x, y)e e dxdy. The sampling grid function g(Δx, Δy) is periodical with period (Δx, Δy) => can be expressed by its Fourier series expansion: k l j2 x j2 y x y g(x,y)(x, y) a(k,l)e e , k l where: k l x y j2 x j2 y 1 1 x y a(k,l) g(x,y)(x, y)e e dxdy. x y 0 0 Digital image processing Chapter 3. Image sampling and quantization Two-dimensional sampling (3) Since: 1, if x y 0 for (x, y)[0;x)[0;y), g(x,y)(x, y) , 0, otherwise 1 1 a(k,l) , k,l ;. x y Therefore the Fourier transform of fS is: 2 ly 2 kx j 1 j j2 y F ( , ) f (x, y) e x e y e j2 x x e y dxdy S x y k l x y k l j2 x x j2 y y 1 x y F ( , ) f (x, y)e e dxdy S x y x y k l k l j2 x x j2 y y 1 x y FS ( x, y ) f (x, y)e e dxdy x y k l 1 k l FS ( x, y ) F x , y . x y k l x y The spectrum of the sampled image = the collection of an infinite number of scaled spectral replicas of the spectrum of the original image, centered at multiples of spatial frequencies 1/Δx, 1/ Δy. Digital image processing Chapter 3. Image sampling and quantization Original image Original image spectrum – 3D Original image spectrum – 2D 2-D rectangular sampling grid Sampled image spectrum – 3D Sampled image spectrum – 2D Digital image processing Chapter 3. Image sampling and quantization Image reconstruction from its samples y 1/x ys 1 1 xs 2 x0 , ys 2 y0 x ,y - ys y0 2 x0 2 y0 2 1/y 2y0 y0 1 ,( , ) H( , ) x y x y ( xs ys ) 1 3 0, otherwise 2x0 ~ F(x, y ) H(x, y )Fs (x, y ) F(x, y ) x0 xs ~ xs-x0 x xs f x, y hx, y fs x, y Fig.4 The sampled image spectrum ~ f x, y fs mx,nyhx mx, y ny m n Let us assume that the filtering region R is rectangular, at the middle distance between two spectral replicas: 1 xs ys siny , x and y sinxxs ys H(x, y ) (xs ys ) 2 2 hx, y xxs y ys 0, otherwise ~ sinxxs m siny ys n f x, y fsmx,nyhx mx, y ny fsmx,ny m n m n xxs m y ys n Digital image processing Chapter 3. Image sampling and quantization ~ sina f x, y fs mx,nysincxxs msincy ys n, where sinca m n a Since the sinc function has infinite extent => it is impossible to implement in practice the ideal LPF it is impossible to reconstruct in practice an image from its samples without error if we sample it at the Nyquist rates. Practical solution: sample the image at higher spatial frequencies + implement a real LPF (as close to the ideal as possible). 1-D sinc function 2-D sinc function Digital image processing Chapter 3. Image sampling and quantization The Nyquist rate. The aliasing. The fold-over frequencies xs 2 x0 , ys 2 y0 y y0 0 2y0 2x0 The Moire effect 0 x0 x Fig. 5 Aliasing – fold-over frequencies Note: Aliasing may also appear in the reconstruction process, due to the imperfections of the filter! How to avoid aliasing if cannot increase the sampling frequencies? “Jagged” boundaries By a LPF on the image applied prior to sampling! Digital image processing Chapter 3. Image sampling and quantization Non-rectangular sampling grids. Interlaced sampling grids n n νy F(ν , ν )=1 1/2 x y 2 1/ 2 1 νx 1 2 2 -1/2 1/2 1 m -1 0 1 1 2 3 m -2 -1 0 2 -1/2 a) Image spectrum b) Rectangular grid G1 c) Interlaced grid G2 2 2 1 x x -1 0 1 1 x x 1 d) The spectrum using G1 e) The spectrum using G2 Interlaced sampling Optimal sampling = Karhunen-Loeve expansion: f (x, y) am,nm,n m,n0 Digital image processing Chapter 3. Image sampling and quantization Image reconstruction from its samples in the real case The question is: what to fill in the “interpolated” (new) dots? Several interpolation methods are available; ideally – sinc function in the spatial domain; in practice – simpler interpolation methods (i.e. approximations of LPFs). Digital image processing Chapter 3. Image sampling and quantization Image interpolation filters: The 1-D Graphical The 2-D Frequency interpolation representation p(x) interpolation response pa(1,0) function function pa(1,2) pa(x,y)=p(x)p(y) 1/x 1 Rectangular 1 2 (zero-order 1 x p0 (x)p0 (y) sinc sinc rect 2x0 2y0 -x/2 x/2 x x filter) 0 x x 0 p0(x) 4x0 1/x 1 Triangular 1 x 2 tri 1 2 (first order p (x) p (y) sinc sinc x x 1 1 2 2 x0 y0 -x x x x filter) 0 0 p0 (x) p0 (x) p1(x) 4x0 1 n-order filter n1 p (x)p (y) 1 2 n=2, quadratic p0 (x)p0 (x) n n sinc sinc 2x0 2y0 x x n=3, cubic spline 0 n convolu@ii 0 pn(x) 4x0 1 2 1 x 1 (x2 y2 ) exp 2 exp Gaussian 2 2 2 2 2 2 2 2 2 2 2 exp 2 (1 2 ) x x pg(x) 0 0 2 1 1 x 1 x y sinc sinc sinc rect 1 rect 2 x xy x y Sinc 0 x x 2x0 2y0 0 2x 2x0 Digital image processing Chapter 3.