Digital Image Processing Lectures 3 & 4

2-D Signals and Systems 2-D Fourier Transform

M.R. Azimi, Professor

Department of Electrical and Computer Engineering Colorado State University

Spring 2017

M.R. Azimi Digital Image Processing 2-D Signals and Systems 2-D Fourier Transform Deﬁnitions and Extensions: 2-D Signals: A continuous image is represented by a function of two variables e.g. x(u, v) where (u, v) are called spatial coordinates and x is the intensity. A sampled image is represented by x(m, n).

If pixel intensity is also quantized (digital images) then each pixel is represented by B bits (typically B = 8 bits/pixel). 2-D Delta Functions: They are separable 2-D functions i.e. ∞ (u, v) = (0, 0) Dirac: δ(u, v) = δ(u) δ(v) = 0 Otherwise 1 (m, n) = (0, 0) Kronecker: δ(m, n) = δ(m) δ(n) = 0 Otherwise

M.R. Azimi Digital Image Processing 2-D Signals and Systems 2-D Fourier Transform

Properties: For 2-D Dirac delta: ∞ 1- R R x(u0, v0)δ(u − u0, v − v0)du0dv0 = x(u, v) −∞ 2- R R δ(u, v)du dv = 1 ∀ > 0 − For 2-D Kronecker delta: ∞ ∞ 1- x(m, n) = P P x(m0, n0)δ(m − m0, n − n0) m0=−∞ n0=−∞ ∞ 2- P P δ(m, n) = 1 m,n=−∞ Periodic Signals: Consider an image x(m, n) which satisﬁes x(m, n + N) = x(m, n) x(m + M, n) = x(m, n) This signal is said to be doubly periodic with horizontal and vertical periods M and N, respectively. Only MN samples that are within the fundamental period are distinct.

M.R. Azimi Digital Image Processing 2-D Signals and Systems 2-D Fourier Transform

In general case

x(m + M1, n + N1) = x(m, n)

x(m + M2, n + N2) = x(m, n) ∆ where D = M1N2 − M2N1 6= 0 is number of linearly independent t t (distinct) samples. Then M1 = [M1 N1] and M2 = [M2 N2] represent the displacement from any sample to the corresponding samples of two other periods. Note that D = det[M1 M2]. Examples of doubly periodic 2-D signals with horizontal and vertical periods M = 3 and N = 3, and general periodic 2-D signals with period t t vectors M1 = [7, 2] and M2 = [−2, 4] are shown in below ﬁgures.

n 2

N 2

N1 n1 n1

M.R. Azimi Digital Image Processing 2-D Signals and Systems 2-D Fourier Transform

2-D Linear System and Shift Invariance: 1-Linearity: The system deﬁned by input-output mapping y(m, n) = T [x(m, n)]

is said to be linear iﬀ the operator T satisﬁes the following conditions. That is, if y1(m, n) = T [x1(m, n)] and y2(m, n) = T [x2(m, n)]

then T [ax1(m, n) + bx2(m, n)] = ay1(m, n) + by2(m, n) for any arbitrary a and b, i.e. any linear combination of two or more inputs gives the same combination of their corresponding outputs. 2-Spatial Invariance: The system defined by y(m, n) = T [x(m, n)] is said to be space invariant iff the operator T satisfies y(m, n) = T [x(m, n)] then T [x(m − k, n − l)] = y(m − k, n − l) ∀k, l

M.R. Azimi Digital Image Processing 2-D Signals and Systems 2-D Fourier Transform

2-D Impulse Response and 2-D Convolution: The response of a 2-D system to a 2-D Kronecker delta input is the 2-D impulse response i.e. h(m, n; k, l) =∆ T [δ(m − k, n − l)] For an imaging system it represents the image of an ideal point source. Thus, it is also called point spread function (PSF). PSF is real and non-negative since input-output of an imaging system represent positive quantity e.g., intensity of light. For a linear system, the output can be obtained from its PSF by applying superposition rule, y(m, n) = T [x(m, n)] X X = T [ x(k, l)δ(m − k, n − l)] k l X X = x(k, l)T [δ(m − k, n − l)] k l X X = x(k, l)h(m, n; k, l) k l If the system is further spatiallyM.R. Azimi invariantDigital then Image Processing h(m, n; k, l) = T [δ(m − k, n − l)] = h(m − k, n − l; 0, 0) = h(m − k, n − l) 2-D Signals and Systems 2-D Fourier Transform

If the system is also spatially invariant then h(m, n; k, l) = T [δ(m − k, n − l)] = h(m − k, n − l; 0, 0) = h(m − k, n − l) Thus, for a 2-D LSI system convolution sum becomes X X y(m, n) = x(k, l)h(m − k, n − l) k l X X = x(m − k, n − l)h(k, l) k l = x(m, n) ∗ ∗h(m, n) Imaging systems with separable and circularly symmetric impulse response or PSF are often encountered. Graphical interpretation of 2-D convolution operation is illustrated in the ﬁgure. Using 1st equation above, PSF is ﬁrst rotated about the origin by 180◦, then shifted by m,n and overlayed on image x(k, l). The sum of the products of the arrays {x(k, l)} and {h(m − k, n − l)} which occupy a common region gives the result at (m, n).

M.R. Azimi Digital Image Processing 2-D Signals and Systems 2-D Fourier Transform

Figure: Output at (m, n) is the sum of products of quantities in the shaded area. Example: Graphical convolution of two 2 × 2 and 3 × 2 arrays h(m, n) and x(m, n).

M.R. Azimi Digital Image Processing 2-D Signals and Systems 2-D Fourier Transform

Remarks

1 The result of the convolution of two 2-D arrays of sizes (M1 × N1) and (M2 × N2) is another 2-D array of size (M1 + M2 − 1) × (N1 + N2 − 1).

2 The result of convolution can be veriﬁed by checking the identity P P y(m, n) = (P P h(m, n))(P P x(m, n)) m n m n m n

This property of convolution is known as area conservation.

In this example, P P y(m, n) = 32, P P h(m, n) = 2, m n m n P P x(m, n) = 16 which veriﬁes the above relation. m n

M.R. Azimi Digital Image Processing 2-D Signals and Systems 2-D Fourier Transform

Stability of 2-D LSI Systems Deﬁnition: A 2-D system is said to be bounded-input bounded-output (BIBO) stable iﬀ a bounded input results in a bounded output, i.e. BIBO |x(m, n)| < M < ∞ =⇒ |y(m, n)| < N < ∞, ∀m, n

Theorem: A necessary and suﬃcient condition for BIBO stability of a 2-D LSI system is that the PSF must be absolutely sumable i.e. ∞ ∞ P P |h(m, n)| = P < ∞ m=−∞ n=−∞ Remark: Note that most of 2-D signals (or images) are spatially limited unlike the temporal 1-D signals that have no limits in time (streaming data). Thus, the properties of the output image beyond the spatial bounds of the system are irrelevant to the usefulness of the system. From practical viewpoint and not a mathematical one it is therefore possible to assess the stability of a system by evaluating the sum over the bounds of the output image and requiring this to be ﬁnite for BIBO stability.

M.R. Azimi Digital Image Processing 2-D Signals and Systems 2-D Fourier Transform

Causality of 2-D LSI Systems Causality is a requirement for all realizable 1-D systems and output y(m) is dependent only on those values of x(n) for which n ≤ m (i.e past and present). This is always the case when the independent variable is time.

When the variables are not temporal rather are spatial, this restriction does no longer hold and the output signal can have left as well as right dependencies on the input signal. However, when processing 2-D images arriving in real-time (e.g., raster scan or video) causality comes to play.

Deﬁnition: A 1-D sequence {h(n)} is causal if h(n) = 0, ∀n < 0, anticausal if h(n) = 0, ∀n > 0 and noncausal if it is neither causal nor anticausal. A

2-D sequence {h(m, n)} is causal if its 1-D map h˜(k) is causal. Since there can be many mappings from 2-D to 1-D, the deﬁnition of causality depends on the choice of mapping.

M.R. Azimi Digital Image Processing 2-D Signals and Systems 2-D Fourier Transform

Deﬁnition 1: A 2-D sequence {h(m, n)} is said to be strongly causal if h(m, n) = 0 {m < 0, ∀n} ∪ {n < 0, ∀m} In this case the region of support (ROS) is Quarter Plane. Deﬁnition 2: A 2-D sequence {h(m,n)} is said to be weakly causal if

h(m, n) = 0 {n < 0, ∀m} ∪ {n = 0, m < 0} i.e. Non-Symmetric Half Plane (NSHP) ROS.

M.R. Azimi Digital Image Processing 2-D Signals and Systems 2-D Fourier Transform

Deﬁnition 3: A 2-D sequence {h(m, n)} is said to be semicausal if it is causal in one dimension and noncausal in the other i.e.

h(m, n) = 0 n < 0, ∀m i.e. Symmetric Half Plane (SHP) ROS. Deﬁnition 4: A 2-D sequence {h(m, n)} is said to be noncausal when the ROS is Full Plane (i.e. all four quadrants).

M.R. Azimi Digital Image Processing 2-D Signals and Systems 2-D Fourier Transform

Finite Impulse Response (FIR) or Non-recursive 2-D Systems If the PSF of a 2-D LSI system is of ﬁnite extent i.e.

h(m, n) = 0 (m, n) 6∈ Wi i = 1, 2, 3, 4

where Wi is the ROS of {h(m, n)}, the system is called ﬁnite impulse response (FIR) or non-recursive. The ROS, Wi, of the impulse response can have diﬀerent geometry such as quarter plane (QP), NSHP, SHP and full plane (FP) as shown before but with limited extent. Note: only QP and NSHP are allowable when processing image data arriving serially (e.g., row-wise or column-wise). FIR systems are described by 2-D convolution sum, XX y(m, n) = h(k, l)x(m − k, n − l)

k,l∈Wi

Thus, the output pixel at position (m, n) is the weighted sum of the pixels in the ROS, Wi. This type of systems are also called 2-D All-Zero systems.

M.R. Azimi Digital Image Processing 2-D Signals and Systems 2-D Fourier Transform

Inﬁnite Impulse Response (IIR) or Recursive 2-D Systems A more general form of diﬀerence equation can be formed when the output at any spatial location is a weighted sum of several input pixels as well as several output pixels i.e. XX XX ak,ly(m − k, n − l) = bk,lx(m − k, n − l)

k,l∈Wi k,l∈Wj

Assuming a0,0 = 1 (normalization) and re-arranging equation gives XX y(m, n) = bk,lx(m − k, n − l)

k,l∈Wj XX − ak,ly(m − k, n − l),

k,l∈Wˆ i

Wi = Wˆ i ∪ (0, 0)

Owing to the recursive terms the PSF is of inﬁnite extent and hence the term IIR or recursive systems. Additionally, Wˆ i can only take QP or NSHP ROS though Wj could take any geometry.

M.R. Azimi Digital Image Processing 2-D Signals and Systems 2-D Fourier Transform

Special Cases: ˆ 1 If in the general diﬀerence equation Wi = ∅, then XX y(m, n) = bk,lx(m − k, n − l)

k,l∈Wj

i.e. system reduces to FIR system with h(k, l) = bk,l.

2 If in the general diﬀerence equation Wj = (0, 0), then XX y(m, n) = b0,0x(m, n) − ak,ly(m − k, n − l)

k,l∈Wˆ i

i.e. system reduces to an All-pole system. Example: Consider the system given by the following I/O relation. Determine whether or not the system is linear, space-invariant, FIR or IIR and comment on its stability.

y(m, n) = ax(m,n), a < 1 and ∀m, n ≥ 0

M.R. Azimi Digital Image Processing 2-D Signals and Systems 2-D Fourier Transform

Linearity- Not linear since c x (m,n)+c x (m,n) x (m,n) x (m,n) z(m, n) = a 1 1 2 2 6= c1a 1 + c2a 2 = c1y1(m, n) + c2y2(m, n).

Space-Invariance- Yes. Since z(m, n) = ax(m−k,n−l) = y(m − k, n − l).

FIR or IIR- The PSF for this system is

a (m, n) = (0, 0) h(m, n) = aδ(m,n) = 1 ∀m, n ≥ 0, (m, n) 6= (0, 0) i.e. IIR.

Stability- BIBO stable since for any |x(m, n)| < M −→ y(m, n) = ax(m,n) < N.

M.R. Azimi Digital Image Processing 2-D Signals and Systems 2-D Fourier Transform Continuous 2-D Fourier Transform-An Extension

For a continuous and absolutely integrable 2-D function x(u, v), the 2-D Fourier transform pair is given by ZZ ∞ X(ω1, ω2) = x(u, v)exp(−jω1u − jω2v)dudv −∞ 1 ZZ ∞ ( ) = ( ) ( + ) x u, v 2 X ω1, ω2 exp jω1u jω2v dω1dω2 4π −∞ Remarks and Properties:

1 Fourier transform of an image field is the far field or Fraunhofer diffraction pattern of the image e.g., twinkling of a distant light or star at night is due to the observation of its Fourier transform.

2 The Fourier transform is a complex function with magnitude |X(ω1, ω2)| and phase φ(ω1, ω2), i.e.

jφ(ω1,ω2) X(ω1, ω2) = |X(ω1, ω2)|e

M.R. Azimi Digital Image Processing 2-D Signals and Systems 2-D Fourier Transform

Since both photographic film and human eye (or any other detector) respond to the magnitude of an image field, it is difficult to directly observe the phase spectra. Holographic techniques are, however, based upon recording both the magnitude and phase spectra. It is known that the phase spectrum contains most of the information about the positions of edges in an image.

3 Spatial Frequencies: If x(u, v) represents the luminance at spatial ω1 ω2 coordinates u, v then ξ1 = 2π and ξ2 = 2π are the spatial frequencies that represent luminance change wrt spatial distances. The units of ξ1, ξ2 are reciprocal of u, v. If the coordinates u, v are normalized by the viewing distance of the image x(u, v), then the units ξ1, ξ2 are cycles per degree(of the viewing angle.)

4 Uniqueness: The function x(u, v) and its FT X(ω1, ω2) are unique w.r.t. one another i.e. instead of an image its FT can be stored without any loss of information. This property is used in image data compression using transform coding.

M.R. Azimi Digital Image Processing 2-D Signals and Systems 2-D Fourier Transform

5 Separability: 2-D FT can be written as a separable transformation in u and y = v coordinates Z ∞ Z ∞ −jω1u −jω2v X(ω1, ω2) = x(u, v)e du e dv −∞ −∞ i.e. 2-D transform can be implemented by successive 1-D transforms along rows and columns.

6 2-D Convolution & Correlation: Any image formation system is represented by 2-D convolution integral, i.e. if y(u, v) = h(u, v) ∗ ∗x(u, v) or in frequency domain

Y (ω1, ω2) = H(ω1, ω2) · X(ω1, ω2)

M.R. Azimi Digital Image Processing 2-D Signals and Systems 2-D Fourier Transform

i.e. the FT of the convolution of two functions is the product of their FT’s. The inverse theorem states that 1 y(u, v) = F{x(u, v) · h(u, v)} = X(ω1, ω2) ∗ ∗H(ω1, ω2) 4π2 The result of convolution can be extended to evaluate the spatial cross-correlation between two real functions h(u, v) and g(u, v) i.e. ∞ 4 ZZ c(u, v) = h(u0, v0)g(u + u0, v + v0)du0dv0 −∞ = h(u, v) ∗ ∗g(−u, −v) Alternatively, performing change of variables we have ZZ ∞ c(u, v) = h(u0 − u, v0 − v)g(u0, v0)du0dv0 −∞ = h(−u, −v) ∗ ∗g(u, v) Thus

C(ω1, ω2) = H(−ω1, −ω2)G(ω1, ω2) = H(ω1, ω2)G(−ω1, −ω2)

M.R. Azimi Digital Image Processing 2-D Signals and Systems 2-D Fourier Transform

Correlation-based methods are widely used in image processing for template matching in order to ﬁnd the closest match between a given unknown image and a set of known patterns (templates). The closest match occurs when selected template yields the largest correlation value (largest peak of the correlation function). FT of spatial auto-correlation of a deterministic 2-D function is ZZ ∞ F{ x(u0, v0)x(u + u0, v + v0)du0dv0} −∞ 2 = X(−ω1, −ω2)X(ω1, ω2) = |X(ω1, ω2)|

7 Parseval’s Theorem and Inner Product Preservation Another important property of FT is that the inner product of two functions is equal to the inner product of their FT’s.

ZZ ∞ 1 ZZ ∞ ( ) ∗( ) = ( ) ∗( ) x v, u y u, v dudv 2 X ω1, ω2 Y ω1, ω2 dω1dω2 −∞ 4π −∞

M.R. Azimi Digital Image Processing 2-D Signals and Systems 2-D Fourier Transform

where ∗ stands for complex conjugate operation. When x = y we obtain the well-known Parseval energy conservation formula i.e.

ZZ ∞ 1 ZZ ∞ | ( )|2 = | ( )|2 x u, v dudv 2 X ω1, ω2 dω1dω2 −∞ 4π −∞ i.e. the total energy in the function is the same as in its FT.

8 Frequency Response & Eigenfunctions of 2-D LSI Systems An eigen-function of a system is deﬁned as an input function that is reproduced at the output with a possible change in the amplitude. For an LSI system eigen-functions are given by

f(u, v) = exp(jω1u + jω2v)

M.R. Azimi Digital Image Processing 2-D Signals and Systems 2-D Fourier Transform

Using the 2-D convolution integral ZZ ∞ 0 0 0 0 0 0 g(u, v) = h(u − u , v − v ) exp[jω1u + jω2v ]du dv −∞

change u˜ = u − u0, v˜ = v − v0, then

g(u, v) = H(ω1, ω2) exp[jω1u + jω2v]

where ZZ ∞ H(ω1, ω2) = h(˜u, v˜) exp[jω1u˜ + jω2v˜]dud˜ v˜ −∞ = F{h(u, v)}

is the frequency response of the 2-D system. The output is a complex exponential function with the same frequency as the input signal but its amplitude and phase are changed by the complex gain H(ω1, ω2) of the 2-D system.

M.R. Azimi Digital Image Processing