Digital Image Processing Lectures 11 & 12

2-D Discrete Fourier Transform Uniﬁed Matrix Representation Other Image Transforms Discrete Cosine Transform (DCT)

M.R. Azimi, Professor

Department of Electrical and Computer Engineering Colorado State University

M.R. Azimi Digital Image Processing 2-D Discrete Fourier Transform Uniﬁed Matrix Representation Other Image Transforms Discrete Cosine Transform (DCT) Algorithm For Computing Linear Convolution using 2D DFT-Cont

Example 1 (circular convolution): 1 0 1 0 Consider 2-D arrays x(m, n) = ,h(m, n) = . Find yˆ(m, n) 2 1 1 1 and y(m, n). The 2 × 2 DFT of these sequences give 1 1 X X −j 2πmk −j 2πnl X(k, l) = x(m, n)e 2 e 2 m=0 n=0 X(0, 0) = x(0, 0) + x(0, 1) + x(1, 0) + x(1, 1) = 4 X(0, 1) = x(0, 0) + x(0, 1)e−jπ + x(1, 0) + x(1, 1)e−jπ = x(0, 0) − x(0, 1) + x(1, 0) − x(1, 1) = 1 − 0 + 2 − 1 = 2 X(1, 0) = x(0, 0) + x(0, 1) + x(1, 0)e−jπ + x(1, 1)e−jπ = x(0, 0) + x(0, 1) − x(1, 0) − x(1, 1) = −2

M.R. Azimi Digital Image Processing 2-D Discrete Fourier Transform Uniﬁed Matrix Representation Other Image Transforms Discrete Cosine Transform (DCT)

X(1, 1) = x(0, 0) + x(0, 1)e−jπ + x(1, 0)e−jπ + x(1, 1)e−j2π = 0 4 2 3 1 Thus X(k, l) = , H(k, l) = . and −2 0 −1 1 12 2 Yˆ (k, l) = X(k, l)H(k, l) = . Taking IDFT we get 2 0 4 3 yˆ(m, n) = . This result can alternatively be obtained using 3 2 the circular convolution in the spatial domain i.e. 1 1 X X yˆ(m, n) = x(p, q)h(((m − p))2, ((n − q))2) p=0 q=0

yˆ(0, 0) = x(0, 0)h(0, 0) + x(0, 1)h(0, 1) + x(1, 0)h(1, 0) +x(1, 1)h(1, 1) = 4

M.R. Azimi Digital Image Processing 2-D Discrete Fourier Transform Uniﬁed Matrix Representation Other Image Transforms Discrete Cosine Transform (DCT)

yˆ(0, 1) = x(0, 0)h(0, 1) + x(0, 1)h(0, 0) + x(1, 0)h(1, 1) +x(1, 1)h(1, 0) = 3 yˆ(1, 0) = x(0, 0)h(1, 0) + x(0, 1)h(1, 1) + x(1, 0)h(0, 0) +x(1, 1)h(0, 1) = 3 yˆ(1, 1) = x(0, 0)h(1, 1) + x(0, 1)h(1, 0) + x(1, 0)h(0, 1) +x(1, 1)h(0, 0) = 2 Now, let us ﬁnd the result of the desired linear convolution ﬁrst using the direct method and then using the DFT. y(0, 0) = x(0, 0)h(0, 0) = 1 y(0, 1) = x(0, 0)h(0, 1) + x(0, 1)h(0, 0) = 0 . . y(2, 1) = x(1, 1)h(1, 0) + x(1, 0)h(1, 1) = 3 y(2, 2) = x(1, 1)h(1, 1) = 1

M.R. Azimi Digital Image Processing 2-D Discrete Fourier Transform Uniﬁed Matrix Representation Other Image Transforms Discrete Cosine Transform (DCT)

1 0 0 y(m, n) = 3 2 0 2 3 1 Note: yˆ(m, n) is the aliased version of this array. Extend arrays to size P = 4 ≥ M1 + M2 − 1, Q = 4 ≥ N1 + N2 − 1 i.e.

1 0 0 0 1 0 0 0 2 1 0 0 1 1 0 0 x0(m, n) = , h0(m, n) = 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Taking 4 × 4 DFT and IDFT of the product gives 1 0 0 0 3 2 0 0 y0(m, n) = 2 3 1 0 0 0 0 0 which is y(m, n) if the last row and column are removed.

M.R. Azimi Digital Image Processing 2-D Discrete Fourier Transform Uniﬁed Matrix Representation Other Image Transforms Discrete Cosine Transform (DCT) Matrix Representation of DFT

Consider an N-point ﬁnite-extent sequence x(n), n ∈ [0,N − 1] with DFT

N−1 1 X nk X(k) = √ x(n)WN k ∈ [0,N − 1] N n=0

∆ −2πj/N where WN = e , then in matrix form

    1 1 ··· 1   X(0)  N−1  x(0)    1 WN ··· W     .   N   .  . = √1   .   N  . . .     .   . . .   .     . . .    X(N − 1)   x(N − 1) N×1 1 W N−1 ··· W N×1 N N N×N

X = WN x W : DFT matrix with elements w = √1 W kn mod N N kn N N −1 The inverse is x = WN X.

M.R. Azimi Digital Image Processing 2-D Discrete Fourier Transform Uniﬁed Matrix Representation Other Image Transforms Discrete Cosine Transform (DCT)

Properties and Remarks a. Symmetry and Unitary For a 1-D DFT matrix

t −1 ∗ WN = WN i.e. symmetry, WN = WN i.e. unitary

∗ Thus WN WN = I. b. Energy Conservation For a unitary transform the energy is preserved in both domains i.e.

kXk2 = kxk2

To see this

N−1 2 ∆ X 2 ∗t ∗t ∗t kXk = |X(k)| = X X = x WN WN x k=0 t = x∗ x = kxk2

M.R. Azimi Digital Image Processing 2-D Discrete Fourier Transform Uniﬁed Matrix Representation Other Image Transforms Discrete Cosine Transform (DCT)

c. 2-D DFT in Matrix Form For a 2-D ﬁnite-extent image, x(m, n) with ROS,RMN , the 2-D DFT is

M−1 N−1 1 X X mk nl X(k, l) = √ x(m, n)WM WN k ∈ [0,M−1], l ∈ [0,N−1] MN m=0 n=0 If x and X represent the original image matrix and its DFT image (complex) matrix (both of size M × N), respectively, the separable 2-D transform in matrix form can be written as

X = WM xWN and ∗ ∗ x = WM XWN which shows the separability of the 2-D DFT. This is a uniﬁed matrix representation since for other image transforms such as sine, cosine, Hadamand, Haar and Slant transforms similar representation hold. M.R. Azimi Digital Image Processing 2-D Discrete Fourier Transform Uniﬁed Matrix Representation Other Image Transforms Discrete Cosine Transform (DCT)

Basis Images of 2-D DFT ∗ ∗ H ∗ Let wk be the kth column of WM and wl be the lth row of WN , then the (k, l)th basis image is ∗ ∗ H W (k, l) = wkwl ∗ ∗ Using x = WM XWN the image can be represented as M−1 N−1 X X x = X(k, l)W∗(k, l) k=0 l=0 i.e. the image is decomposed onto a space spanned by these MN basis images.The coeﬃcients in this representation are given by X(k, l) =< x, W∗(k, l) > where inner product of two matrices is given by M−1 N−1 X X < A, B >= a(m, n)b∗(m, n) m=0 n=0 . M.R. Azimi Digital Image Processing 2-D Discrete Fourier Transform Uniﬁed Matrix Representation Other Image Transforms Discrete Cosine Transform (DCT) Discrete Cosine Transform (DCT)

Recall that the DFS of any real even symmetric signal contains only real coeﬃcients corresponding to the cosine terms. This can be extended to the DFT of a symmetrically extended signal/image. There are many ways to symmetrically extend a signal or an image leading to variety of DCT types. Here, we present DCT-II which is the most common one (see Fig.).

The 1-D DCT-II of a ﬁnite-extent signal x(n) of size N is  N−1  √1 P x(n) k = 0  N X(k) = n=0 q N−1  2 P (2n+1)kπ  N x(n) cos[ 2N ] k ∈ [1,N − 1] n=0 M.R. Azimi Digital Image Processing 2-D Discrete Fourier Transform Uniﬁed Matrix Representation Other Image Transforms Discrete Cosine Transform (DCT)

The inverse transform is r N−1 1 2 X (2n + 1)kπ x(n) = √ X(0)+ X(k)cos[ ] n ∈ [0,N−1] N 2N N k=1 In matrix form we get X = Cx,C = [c(k, n)] with forward kernel √ ( 1/ N k = 0, n ∈ [0,N − 1] c(k, n) = q 2 (2n+1)kπ N cos[ 2N ] k ∈ [1,N − 1], n ∈ [0,N − 1] where x is the signal vector X is the DCT vector. The inverse transform in matrix form is x = C−1X = CtX Remarks

1 DCT is real and orthogonal i.e. C = C∗,C−1 = Ct M.R. Azimi Digital Image Processing 2-D Discrete Fourier Transform Uniﬁed Matrix Representation Other Image Transforms Discrete Cosine Transform (DCT)

2 2-DCT can be performed using 1-D DCT’s along columns and row, i.e. separable. 3 DCT is NOT the real part of the DFT rather it is related to the DFT of a symmetrically extended signal/image. 4 The energy of signal/image is packed mostly in only a few DCT coefficients (i.e. only a few significant X(k)’s), hence making DCT very useful for data compression applications. 2-D DCT of Images The 2-D DCT of a finite-extent image x(m, n) of size N × N is

  N−1 N−1 k,l=0,0  1 P P x(m,n)  N  m=0 n=0 m,n∈[0,N−1]  X(k,l)= k,l6=0,0  N−1 N−1  2 P P x(m,n) cos[ (2m+1)kπ ] cos[ (2n+1)lπ ] k,l∈[0,N−1],  N 2N 2N  m=0 n=0  m,n∈[0,N−1]

M.R. Azimi Digital Image Processing 2-D Discrete Fourier Transform Uniﬁed Matrix Representation Other Image Transforms Discrete Cosine Transform (DCT)

The inverse 2-D IDCT is

N−1 1 2 P P (2m+1)kπ (2n+1)lπ x(m,n)= N X(0,0)+ N X(k,l) cos[ 2N ] cos[ 2N ] k=0,l=0 k,l6=(0,0)

The 2-D DCT of an image x in matrix form becomes

X = CxCt

with forward kernel deﬁned by

  1 k,l=0, m,n∈[0,N−1] C(k,l;m,n)= N 2 (2m+1)kπ (2n+1)lπ  N cos[ 2N ] cos[ 2N ] k,l6=0,0 k,l∈[0,N−1],m,n∈[0,N−1]

The inverse 2-D IDCT in matrix form is

x = CtXC

M.R. Azimi Digital Image Processing 2-D Discrete Fourier Transform Uniﬁed Matrix Representation Other Image Transforms Discrete Cosine Transform (DCT)

t Figure below shows basis images C(k, l) = ckcl, k, l ∈ [0,N − 1] of 2-D DCT for N = 8 where ck is kth column of C. Image x is decomposed as a linear combination of these basis images with the PM−1 PN−1 DCT coeﬃcients, X(k, l)s, i.e. x = k=0 l=0 X(k, l)C(k, l).

Example To show the usefulness of the DCT for data reduction and compression applications, we reconstructed the Peppers image based upon only 64 × 64 DCT coefficients. The following figures show the DCT coefficients of the Peppers image and the reconstructed result, which exhibits some smearing and ringing artifacts. Why?

M.R. Azimi Digital Image Processing 2-D Discrete Fourier Transform Uniﬁed Matrix Representation Other Image Transforms Discrete Cosine Transform (DCT)

In real applications, however, the original image is typically partitioned into blocks (e.g., 16 × 16) and DCT coeﬃcients are taken from each block. If a portion of these coeﬃcients is kept for each block and then reconstruction is done block-by-block the following reconstructed image will be obtained, which shows a much better reconstruction.

M.R. Azimi Digital Image Processing 2-D Discrete Fourier Transform Uniﬁed Matrix Representation Other Image Transforms Discrete Cosine Transform (DCT)

Example 1 Show that the N × N cosine transform matrix C is orthogonal, i.e. prove t CC = IN . Consider the k, lth (for k, l 6= 0) element of CCt,

N−1 2 X (2n + 1)kπ (2n + 1)lπ [CCt] = cos[ ] cos[ ] k,l N 2N 2N n=0 N−1 1 X −j(2n+1)(k+l)π j(2n+1)(k+l)π −j(2n+1)(k−l)π j(2n+1)(k−l)π = [e 2N +e 2N +e 2N +e 2N ] 2N n=0 which simpliﬁes to (k + l)π (k − l)π [CCt] = cos[ ]δ(k + l − N) + cos[ ]δ(k − l) k,l 2N 2N where the ﬁrst term is always zero and the second term is also zero except when k = l. Thus, 1 for k = l [CCt] == k,l 0 otherwise

M.R. Azimi Digital Image Processing 2-D Discrete Fourier Transform Uniﬁed Matrix Representation Other Image Transforms Discrete Cosine Transform (DCT)

N−1 t P 1 t For k, l = 0, [CC ]0,0 = N = 1. Hence, CC = IN i.e. n=0 orthogonal transformation matrix.

M.R. Azimi Digital Image Processing