Digital Image Processing
Filtering in the Frequency Domain (Circulant Matrices and Convolution)
Christophoros Nikou [email protected]
University of Ioannina - Department of Computer Science and Engineering 2 Toeplitz matrices
• Elements with constant value along the main diagonal and sub-diagonals. • For a NxN matrix, its elements are determined by
a (2N-1)-length sequence tn | (N 1) n N 1 T(,)m n t mn t0 t 1 t 2 t(N 1) t t t 1 0 1
T tt22 t1 t t t t N 1 2 1 0 NN
C. Nikou – Digital Image Processing (E12) 3 Toeplitz matrices (cont.)
• Each row (column) is generated by a shift of the previous row (column). − The last element disappears. − A new element appears.
T(,)m n t mn t0 t 1 t 2 t(N 1) t t t 1 0 1
T tt22 t1 t t t t N 1 2 1 0 NN
C. Nikou – Digital Image Processing (E12) 4 Circulant matrices
• Elements with constant value along the main diagonal and sub-diagonals. • For a NxN matrix, its elements are determined by
a N-length sequence cn | 01nN
C(,)m n c(m n )mod N c0 cNN 1 c 2 c1 c c c 1 01N
C c21 c c02 cN cN 1 c c c c NN1 21 0 NN
C. Nikou – Digital Image Processing (E12) 5 Circulant matrices (cont.)
• Special case of a Toeplitz matrix. • Each row (column) is generated by a circular shift (modulo N) of the previous row (column).
C(,)m n c(m n )mod N
c0 cNN 1 c 2 c1 c c c 1 01N
C c21 c c02 cN cN 1 c c c c NN1 21 0 NN C. Nikou – Digital Image Processing (E12) Convolution by matrix-vector 6 operations • 1-D linear convolution between two discrete signals may be expressed as the product of a Toeplitz matrix constructed by the elements of one of the signals and a vector constructed by the elements of the other signal. • 1-D circular convolution between two discrete signals may be expressed as the product of a circulant matrix constructed by the elements of one of the signals and a vector constructed by the elements of the other signal. • Extension to 2D signals.
C. Nikou – Digital Image Processing (E12) 1D linear convolution using Toeplitz 7 matrices
f[ n ] {1, 2, 2}, h [ n ] {1, 1}, N12 3, N 2 • The linear convolution g[n]=f [n]*h [n] will be of
length N=N1+N2-1=3+2-1=4. • We create a Toeplitz matrix H from the elements of h [n] (zero-padded if needed) with − N=4 lines (the length of the result).
− N1=3 columns (the length of f [n]). − The two signals may be interchanged.
C. Nikou – Digital Image Processing (E12) 1D linear convolution using Toeplitz 8 matrices (cont.)
f[ n ] {1, 2, 2}, h [ n ] {1, 1}, N12 3, N 2
1 0 0 1 1 0 H Length of the result =4 0 1 1 0 0 1 43 Notice that H is not circulant (e.g. a -1 appears in the second line Length of f [n] = 3 which is not present in the first line. Zero-padded h[n] in the first column C. Nikou – Digital Image Processing (E12) 1D linear convolution using Toeplitz 9 matrices (cont.)
f[ n ] {1, 2, 2}, h [ n ] {1, 1}, N12 3, N 2
1 0 0 1 1 1 1 0 1 g = Hf 2 0 1 1 0 2 0 0 1 2
gn[ ] {1, 1, 0, 2}
C. Nikou – Digital Image Processing (E12) 1D circular convolution using 10 circulant matrices
f[ n ] {1, 2, 2}, h [ n ] {1, 1}, N12 3, N 2
• The circular convolution g[n]=f [n]h [n] will
be of length N=max{N1, N2}=3. • We create a circulant matrix H from the elements of h [n] (zero-padded if needed) of size NxN. − The two signals may be interchanged.
C. Nikou – Digital Image Processing (E12) 1D circular convolution using 11 circulant matrices (cont.)
f[ n ] {1, 2, 2}, h [ n ] {1, 1}, N12 3, N 2
1 0 1 H 1 1 0 0 1 1 33
Zero-padded h[n] in the first column
C. Nikou – Digital Image Processing (E12) 1D circular convolution using 12 circulant matrices (cont.)
f[ n ] {1, 2, 2}, h [ n ] {1, 1}, N12 3, N 2
1 0 1 1 1 g = Hf 1 1 0 2 1 0 1 1 2 0
gn[ ] { 1, 1, 0}
C. Nikou – Digital Image Processing (E12) 13 Block matrices
AAA11 12 1N AAA A 21 22 2N AAAM12 M MN
• Aij are matrices. • If the structure of A, with respect to its sub-matrices, is Toeplitz (circulant) then matrix A is called block-Toeplitz (block-circulant).
• If each individual Aij is also a Toeplitz (circulant) matrix then A is called doubly block-Toeplitz (doubly block-circulant).
C. Nikou – Digital Image Processing (E12) 2D linear convolution using doubly 14 block Toeplitz matrices
m m 1 4 1 1 1 2 5 3 1 -1 n n f [m,n] h [m,n]
M1=2, N1=3 M2=2, N2=2
The result will be of size (M1+M2-1) x (N1+N2-1) = 3 x 4
C. Nikou – Digital Image Processing (E12) 2D linear convolution using doubly 15 block Toeplitz matrices (cont.) m m 1 4 1 1 1 2 5 3 1 -1 n n f [m,n] h [m,n]
• At first, h[m,n] is zero-padded to 3 x 4 (the size of the m result). 0 0 0 0 • Then, for each row of h[m,n], 1 1 0 0 a Toeplitz matrix with 3 1 -1 0 0 n columns (the number of h[m,n] columns of f [m,n]) is
constructed. C. Nikou – Digital Image Processing (E12) 2D linear convolution using doubly 16 block Toeplitz matrices (cont.)
m • For each row of h[m,n], a 0 0 0 0 Toeplitz matrix with 3 columns 1 1 0 0 (the number of columns of f 1 -1 0 0 [m,n]) is constructed. n h[m,n]
1 0 0 1 0 0 000 1 1 0 1 1 0 000 H H H 1 0 1 1 2 0 1 1 3 000 0 0 1 0 0 1 000
C. Nikou – Digital Image Processing (E12) 2D linear convolution using doubly 17 block Toeplitz matrices (cont.) m m 1 4 1 1 1 2 5 3 1 -1 n n f [m,n] h [m,n]
• Using matrices H1, H2
and H3 as elements, a HH13 doubly block Toeplitz HHH matrix H is then 21 HH constructed with 2 3212 6 columns (the number of rows of f [m,n]).
C. Nikou – Digital Image Processing (E12) 2D linear convolution using doubly 18 block Toeplitz matrices (cont.) m m 1 4 1 1 1 2 5 3 1 -1 n n f [m,n] h [m,n] 2 5 • We now construct 3 (2 5 3)T a vector from the f T elements of f [m,n]. 1 (1 4 1) 4 1
C. Nikou – Digital Image Processing (E12) 2D linear convolution using doubly 19 block Toeplitz matrices (cont.) m m 1 4 1 1 1 2 5 3 1 -1 n n f [m,n] h [m,n] 2 5 HH13 3 g = Hf H21 H 1 HH 324 1 C. Nikou – Digital Image Processing (E12) 2D linear convolution using doubly 20 block Toeplitz matrices (cont.) 1 0 0 0 0 0 2 1 1 0 0 0 0 3 0 1 1 0 0 0 2 0 0 1 0 0 0 2 3 1 0 0 1 0 0 5 3 (2 3 2 3)T 1 1 0 1 1 0 3 10 T g = Hf (3 10 5 2) 0 1 1 0 1 1 1 5 (1 5 5 1)T 0 0 1 0 0 1 4 2 0 0 0 1 0 0 1 1 0 0 0 1 1 0 5 0 0 0 0 1 1 5 0 0 0 0 0 1 1 C. Nikou – Digital Image Processing (E12) 2D linear convolution using doubly 21 block Toeplitz matrices (cont.) m m 1 4 1 1 1 2 5 3 * 1 -1 n n f [m,n] h [m,n]
m 1 5 5 1 (2 3 2 3)T T = 3 10 5 2 g=(3 10 5 2) 2 3 -2 3 T n (1 5 5 1) g [m,n]
C. Nikou – Digital Image Processing (E12) 2D linear convolution using doubly 22 block Toeplitz matrices (cont.) Another example
m m 3 4 1 2 1 -1 n n f [m,n] h [m,n]
M1=2, N1=2 M2=1, N2=2
The result will be of size (M1+M2-1) x (N1+N2-1) = 2 x 3
C. Nikou – Digital Image Processing (E12) 2D linear convolution using doubly 23 block Toeplitz matrices (cont.) m m 3 4 1 2 1 -1 n n f [m,n] h [m,n] • At first, h[m,n] is zero-padded to 2 x 3 (the size of the m result). • Then, for each line of h[m,n], 0 0 0 a Toeplitz matrix with 2 1 -1 0 n columns (the number of h[m,n] columns of f [m,n]) is
constructed. C. Nikou – Digital Image Processing (E12) 2D linear convolution using doubly 24 block Toeplitz matrices (cont.)
m • For each row of h[m,n], a Toeplitz matrix with 2 columns (the 0 0 0 number of columns of f [m,n]) is 1 -1 0 constructed. n h[m,n]
10 00 H 11 H 00 1 2 01 00
C. Nikou – Digital Image Processing (E12) 2D linear convolution using doubly 25 block Toeplitz matrices (cont.) m m 3 4 1 2 1 -1 n n f [m,n] h [m,n]
• Using matrices H1 and H2 as elements, a HH doubly block Toeplitz H 12 matrix H is then HH21 constructed with 2 64 columns (the number of rows of f [m,n]).
C. Nikou – Digital Image Processing (E12) 2D linear convolution using doubly 26 block Toeplitz matrices (cont.) m m 3 4 1 2 1 -1 n n f [m,n] h [m,n]
1 T • We now construct a 2 (1 2) f vector from the elements T 3 (3 4) of f [m,n]. 4
C. Nikou – Digital Image Processing (E12) 2D linear convolution using doubly 27 block Toeplitz matrices (cont.) m m 3 4 1 2 1 -1 n n f [m,n] h [m,n]
1 HH2 12 g = Hf HH213 4
C. Nikou – Digital Image Processing (E12) 2D linear convolution using doubly 28 block Toeplitz matrices (cont.)
1 0 0 0 1 1 1 0 0 1 1 (1 1 2)T 0 1 0 0 2 2 g = Hf T 0 0 1 0 3 3 (3 1 4) 0 0 1 1 4 1 0 0 0 1 4
C. Nikou – Digital Image Processing (E12) 2D linear convolution using doubly 29 block Toeplitz matrices (cont.) m m 3 4 * 1 2 1 -1 n n f [m,n] h [m,n] m (1 1 2)T 3 1 4 = g=T 1 1 -2 (3 1 4) n g [m,n]
C. Nikou – Digital Image Processing (E12) 2D circular convolution using doubly 30 block circulant matrices The circular convolution g[m,n]=f [m,n]h [m,n] with 0m M 1, 0 n N 1,
may be expressed in matrix-vector form as: g = Hf
where H is a doubly block circulant matrix generated by h [m,n] and f is a vectorized form of f [m,n].
C. Nikou – Digital Image Processing (E12) 2D circular convolution using doubly 31 block circulant matrices (cont.)
HHHH0MM 1 2 1 HHHH 1 0M 1 2
HHHHH 2 1 0 3 HHHHMMM1 2 3 0
Each Hj, for j=1,..M, is a circulant matrix with N columns (the number of columns of f [m,n]) generated from the elements of the j-th row of h [m,n]. C. Nikou – Digital Image Processing (E12) 2D circular convolution using doubly 32 block circulant matrices (cont.)
h[,0] j h [, j N 1] h [,1] j h[,1] j h [,0] j h [,2] j H j h[, j N 1] h [, j N 2] h [,0] j NN
Each Hj, for j=1,..M, is a NxN circulant matrix generated from the elements of the j-th row of h [m,n]. C. Nikou – Digital Image Processing (E12) 2D circular convolution using doubly 33 block circulant matrices (cont.)
m m 0 1 0 0 0 0 1 3 -1 1 0 0 1 2 1 1 -1 0 n n f [m,n] h [m,n]
1 0 1 1 0 0 000 H 1 1 0 H 0 1 0 H2 000 0 1 000 0 1 1 0 0 1
C. Nikou – Digital Image Processing (E12) 2D circular convolution using doubly 34 block circulant matrices (cont.)
m m 0 1 0 0 0 0 1 3 -1 1 0 0 1 2 1 1 -1 0 n n f [m,n] h [m,n]
T HHH 1 2 1 0 2 1 g = Hf H H H 1 3 1 T 102 T HHH2 1 0 0 1 0
C. Nikou – Digital Image Processing (E12) 2D circular convolution using doubly 35 block circulant matrices (cont.)
1010001001 0 1110000102 2 0100000011 1 1001010001 3 g = Hf 0101110003 4 0010100001 3 0001001010 1 0000101111 4 0000010100 2
C. Nikou – Digital Image Processing (E12) 2D circular convolution using doubly 36 block circulant matrices (cont.)
m m 0 0 0 0 1 0 1 0 0 1 3 -1 1 -1 0 1 2 1 n n h [m,n] f [m,n]
m 0 2 1T 1 4 -2 T = 3 4 -3 g=3 4 3 0 2 -1 1 4 2 T n g [m,n]
C. Nikou – Digital Image Processing (E12) 37 Diagonalization of circulant matrices
Theorem: The columns of the inverse DFT matrix are eigenvectors of any circulant matrix. The corresponding eigenvalues are the DFT values of the signal generating the circulant matrix. Proof: Let
22n j j k nk wNN e w e
be the DFT basis elements of length N with: 0k N 1, 0 n N 1,
C. Nikou – Digital Image Processing (E12) Diagonalization of circulant matrices 38 (cont.) We know that the DFT F [k] of a 1D signal f [n] may be expressed in matrix-vector form: F = Af where f = f[0], f [1],..., f [ N 1]TT , F = F [0], F [1],..., F [ N 1]
0 1 2N 1 w0 w 0 w 0 w 0 NNNN 0 1 2N 1 w1 w 1 w 1 w 1 A NNNN 0 1 2N 1 wNNNN1 w 1 w 1 w 1 NNNN
C. Nikou – Digital Image Processing (E12) Diagonalization of circulant matrices 39 (cont.) The inverse DFT is then expressed by: f = A-1 F
T where 0 1 2N 1 * w0 w 0 w 0 w 0 NNNN 10 1 1 1 2 1 N 1 T w w w w 1*11 NNNN AA NN 0 1 2N 1 wNNNN1 w 1 w 1 w 1 NNNN The theorem implies that any circulant matrix has eigenvectors the columns of A-1.
C. Nikou – Digital Image Processing (E12) Diagonalization of circulant matrices 40 (cont.) Let H be a NxN circulant matrix generated by the 1D N-length signal h[n], that is:
H(,)()[]m n h m nmod NN h m n
Let also αk be the k-th column of the inverse DFT matrix A-1.
We will prove that αk, for any k, is an eigenvector of H. The m-th element of the vector Hα , denoted by Hα k k m is the result of the circular convolution of the signal h[n] with αk.
C. Nikou – Digital Image Processing (E12) Diagonalization of circulant matrices 41 (cont.) N 1 1 N 1 Hα h[][] m n n kn km N k h[] m nNN w n0 N n0 l m n 1 mN( 1) 1 mN( 1) h[] l wk() m l k m k l NN wNNN h[] l w N lm N lm
1 m 1 k m k l k l wNNNNN h[][] l w h l w N l m ( N 1) l 0
We will break it into two parts
C. Nikou – Digital Image Processing (E12) Diagonalization of circulant matrices 42 (cont.)
1NN 1 1 1 k m k l k l k l wNNNNNNN h[][][] l w h l w h l w N l m ( N 1) l 0 l m 1
Periodic with respect to N.
NNN1 1 1 1 k m k l k l k l wNNNNNNN h[][][] l w h l w h l w N l N m ( N 1) l 0 l m 1
NNN1 1 1 1 k m k l k l k l wNNNNNNN h[][][] l w h l w h l w N l m 1 l 0 l m 1
C. Nikou – Digital Image Processing (E12) Diagonalization of circulant matrices 43 (cont.) 1 N 1 Hα wk m h[] l w k l Hk[]α km N N N k m N l0
DFT of h[n] at k. This holds for any value of m. Therefore:
Hααkk Hk[]
which means that αk, for any k, is an eigenvector of H with corresponding eigenvalue the k-th element of H[k], the DFT of the signal generating H .
C. Nikou – Digital Image Processing (E12) Diagonalization of circulant matrices 44 (cont.)
The above expression may be written in terms of the inverse DFT matrix: HA1 A 1Λ H A 1 ΛA Λ=diagHHHN [0], [1],..., [ 1] or equivalently: Λ ΑHA1 Based on this diagonalization, we can prove the property between circular convolution and DFT.
C. Nikou – Digital Image Processing (E12) Diagonalization of circulant matrices 45 (cont.) g = Hf g = HA-1 Af Ag = AHA-1 Af G=ΛF
GHF[0] [0]0 0 [0] GHF[1] 0 [1] 0 [1] = GNHNFN[1] 0 0 [1][1]
DFT of g[n] DFT of h [n] DFT of f [n]
C. Nikou – Digital Image Processing (E12) Diagonalization of doubly block 46 circulant matrices • These properties may be generalized in 2D. • We need to define the Kronecker product: ABMNKL,
a11BBB a 12 a 1N aBBB a a AB21 22 2N aBBB a a M12 M MN MK NL
C. Nikou – Digital Image Processing (E12) Diagonalization of doubly block 47 circulant matrices (cont.)
• The 2D signal f [m,n], 0m M 1, 0 n N 1, may be vectorized in lexicographic ordering (stacking one column after the other) to a vector: f MN1 • The DFT of f [m,n], may be obtained in matrix-vector form: F A A f
C. Nikou – Digital Image Processing (E12) Diagonalization of doubly block 48 circulant matrices (cont.) Theorem: The columns of the inverse 2D DFT matrix AA 1 are eigenvectors of any doubly block circulant matrix. The corresponding eigenvalues are the 2D DFT values of the 2D signal generating the doubly block circulant matrix: ΛAAHAA 1
Diagonal, containing the 2D DFT Doubly block circulant of h[m,n] generating H C. Nikou – Digital Image Processing (E12)