Digital Image Processing Filtering in the Frequency Domain (Circulant Matrices and Convolution) Christophoros Nikou [email protected] University of Ioannina - Department of Computer Science and Engineering 2 Toeplitz matrices • Elements with constant value along the main diagonal and sub-diagonals. • For a NxN matrix, its elements are determined by a (2N-1)-length sequence tn | (N 1) n N 1 T(,)m n t mn t0 t 1 t 2 t(N 1) t t t 1 0 1 T tt22 t1 t t t t N 1 2 1 0 NN C. Nikou – Digital Image Processing (E12) 3 Toeplitz matrices (cont.) • Each row (column) is generated by a shift of the previous row (column). − The last element disappears. − A new element appears. T(,)m n t mn t0 t 1 t 2 t(N 1) t t t 1 0 1 T tt22 t1 t t t t N 1 2 1 0 NN C. Nikou – Digital Image Processing (E12) 4 Circulant matrices • Elements with constant value along the main diagonal and sub-diagonals. • For a NxN matrix, its elements are determined by a N-length sequence cn | 01nN C(,)m n c(m n )mod N c0 cNN 1 c 2 c1 c c c 1 01N C c21 c c02 cN cN 1 c c c c NN1 21 0 NN C. Nikou – Digital Image Processing (E12) 5 Circulant matrices (cont.) • Special case of a Toeplitz matrix. • Each row (column) is generated by a circular shift (modulo N) of the previous row (column). C(,)m n c(m n )mod N c0 cNN 1 c 2 c1 c c c 1 01N C c21 c c02 cN cN 1 c c c c NN1 21 0 NN C. Nikou – Digital Image Processing (E12) Convolution by matrix-vector 6 operations • 1-D linear convolution between two discrete signals may be expressed as the product of a Toeplitz matrix constructed by the elements of one of the signals and a vector constructed by the elements of the other signal. • 1-D circular convolution between two discrete signals may be expressed as the product of a circulant matrix constructed by the elements of one of the signals and a vector constructed by the elements of the other signal. • Extension to 2D signals. C. Nikou – Digital Image Processing (E12) 1D linear convolution using Toeplitz 7 matrices f[ n ] {1, 2, 2}, h [ n ] {1, 1}, N12 3, N 2 • The linear convolution g[n]=f [n]*h [n] will be of length N=N1+N2-1=3+2-1=4. • We create a Toeplitz matrix H from the elements of h [n] (zero-padded if needed) with − N=4 lines (the length of the result). − N1=3 columns (the length of f [n]). − The two signals may be interchanged. C. Nikou – Digital Image Processing (E12) 1D linear convolution using Toeplitz 8 matrices (cont.) f[ n ] {1, 2, 2}, h [ n ] {1, 1}, N12 3, N 2 1 0 0 1 1 0 H Length of the result =4 0 1 1 0 0 1 43 Notice that H is not circulant (e.g. a -1 appears in the second line Length of f [n] = 3 which is not present in the first line. Zero-padded h[n] in the first column C. Nikou – Digital Image Processing (E12) 1D linear convolution using Toeplitz 9 matrices (cont.) f[ n ] {1, 2, 2}, h [ n ] {1, 1}, N12 3, N 2 1 0 0 1 1 1 1 0 1 g = Hf 2 0 1 1 0 2 0 0 1 2 gn[ ] {1, 1, 0, 2} C. Nikou – Digital Image Processing (E12) 1D circular convolution using 10 circulant matrices f[ n ] {1, 2, 2}, h [ n ] {1, 1}, N12 3, N 2 • The circular convolution g[n]=f [n]h [n] will be of length N=max{N1, N2}=3. • We create a circulant matrix H from the elements of h [n] (zero-padded if needed) of size NxN. − The two signals may be interchanged. C. Nikou – Digital Image Processing (E12) 1D circular convolution using 11 circulant matrices (cont.) f[ n ] {1, 2, 2}, h [ n ] {1, 1}, N12 3, N 2 1 0 1 H 1 1 0 0 1 1 33 Zero-padded h[n] in the first column C. Nikou – Digital Image Processing (E12) 1D circular convolution using 12 circulant matrices (cont.) f[ n ] {1, 2, 2}, h [ n ] {1, 1}, N12 3, N 2 1 0 1 1 1 g = Hf 1 1 0 2 1 0 1 1 2 0 gn[ ] { 1, 1, 0} C. Nikou – Digital Image Processing (E12) 13 Block matrices AAA11 12 1N AAA A 21 22 2N AAAM12 M MN • Aij are matrices. • If the structure of A, with respect to its sub-matrices, is Toeplitz (circulant) then matrix A is called block-Toeplitz (block-circulant). • If each individual Aij is also a Toeplitz (circulant) matrix then A is called doubly block-Toeplitz (doubly block-circulant). C. Nikou – Digital Image Processing (E12) 2D linear convolution using doubly 14 block Toeplitz matrices m m 1 4 1 1 1 2 5 3 1 -1 n n f [m,n] h [m,n] M1=2, N1=3 M2=2, N2=2 The result will be of size (M1+M2-1) x (N1+N2-1) = 3 x 4 C. Nikou – Digital Image Processing (E12) 2D linear convolution using doubly 15 block Toeplitz matrices (cont.) m m 1 4 1 1 1 2 5 3 1 -1 n n f [m,n] h [m,n] • At first, h[m,n] is zero-padded to 3 x 4 (the size of the m result). 0 0 0 0 • Then, for each row of h[m,n], 1 1 0 0 a Toeplitz matrix with 3 1 -1 0 0 n columns (the number of h[m,n] columns of f [m,n]) is constructed. C. Nikou – Digital Image Processing (E12) 2D linear convolution using doubly 16 block Toeplitz matrices (cont.) m • For each row of h[m,n], a 0 0 0 0 Toeplitz matrix with 3 columns 1 1 0 0 (the number of columns of f 1 -1 0 0 [m,n]) is constructed. n h[m,n] 1 0 0 1 0 0 000 1 1 0 1 1 0 000 H H H 1 0 1 1 2 0 1 1 3 000 0 0 1 0 0 1 000 C. Nikou – Digital Image Processing (E12) 2D linear convolution using doubly 17 block Toeplitz matrices (cont.) m m 1 4 1 1 1 2 5 3 1 -1 n n f [m,n] h [m,n] • Using matrices H1, H2 and H3 as elements, a HH13 doubly block Toeplitz HHH matrix H is then 21 HH constructed with 2 3212 6 columns (the number of rows of f [m,n]). C. Nikou – Digital Image Processing (E12) 2D linear convolution using doubly 18 block Toeplitz matrices (cont.) m m 1 4 1 1 1 2 5 3 1 -1 n n f [m,n] h [m,n] 2 5 • We now construct 3 (2 5 3)T a vector from the f T elements of f [m,n]. 1 (1 4 1) 4 1 C. Nikou – Digital Image Processing (E12) 2D linear convolution using doubly 19 block Toeplitz matrices (cont.) m m 1 4 1 1 1 2 5 3 1 -1 n n f [m,n] h [m,n] 2 5 HH13 3 g = Hf H21 H 1 HH 324 1 C. Nikou – Digital Image Processing (E12) 2D linear convolution using doubly 20 block Toeplitz matrices (cont.) 1 0 0 0 0 0 2 1 1 0 0 0 0 3 0 1 1 0 0 0 2 0 0 1 0 0 0 2 3 1 0 0 1 0 0 5 3 (2 3 2 3)T 1 1 0 1 1 0 3 10 T g = Hf (3 10 5 2) 0 1 1 0 1 1 1 5 (1 5 5 1)T 0 0 1 0 0 1 4 2 0 0 0 1 0 0 1 1 0 0 0 1 1 0 5 0 0 0 0 1 1 5 0 0 0 0 0 1 1 C. Nikou – Digital Image Processing (E12) 2D linear convolution using doubly 21 block Toeplitz matrices (cont.) m m 1 4 1 1 1 2 5 3 * 1 -1 n n f [m,n] h [m,n] m 1 5 5 1 (2 3 2 3)T T = 3 10 5 2 g=(3 10 5 2) 2 3 -2 3 T n (1 5 5 1) g [m,n] C. Nikou – Digital Image Processing (E12) 2D linear convolution using doubly 22 block Toeplitz matrices (cont.) Another example m m 3 4 1 2 1 -1 n n f [m,n] h [m,n] M1=2, N1=2 M2=1, N2=2 The result will be of size (M1+M2-1) x (N1+N2-1) = 2 x 3 C. Nikou – Digital Image Processing (E12) 2D linear convolution using doubly 23 block Toeplitz matrices (cont.) m m 3 4 1 2 1 -1 n n f [m,n] h [m,n] • At first, h[m,n] is zero-padded to 2 x 3 (the size of the m result). • Then, for each line of h[m,n], 0 0 0 a Toeplitz matrix with 2 1 -1 0 n columns (the number of h[m,n] columns of f [m,n]) is constructed. C. Nikou – Digital Image Processing (E12) 2D linear convolution using doubly 24 block Toeplitz matrices (cont.) m • For each row of h[m,n], a Toeplitz matrix with 2 columns (the 0 0 0 number of columns of f [m,n]) is 1 -1 0 constructed.