ECE 484 Digital Image Processing Lec 06 - Fourier Transform & Image Sampling

Zhu Li Dept of CSEE, UMKC Email: [email protected], Ph: x 2346. http://l.web.umkc.edu/lizhu

Office Hour: Tu/Th 2:30-4pm@FH560E

slides created with WPS Office Linux and EqualX equation editor

Z. Li, ECE 484 Digital Image Processing, 2019. p.1 Outline  Recap of Lec 05  Fourier Transform - Freq Domain Representation of Images  Image Sampling . Nyquist Sampling Rate and Aliasing . Super-Resolution and Interpolation  Summary

Z. Li, ECE 484 Digital Image Processing, 2019. p.2 Re-Cap: Linear Filters  Convolution

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 20 10 0 0 0 90 90 90 90 90 0 0 0 20 40 60 60 60 40 20 0 0 0 90 90 90 90 90 0 0 0 30 60 90 90 90 60 30 1/9 11/9 11/9 11/9 11/9 1/9 0 0 0 90 90 90 90 90 0 0 0 30 50 80 80 90 60 30 0 0 0 90 0 90 90 90 0 0 0 30 50 80 80 90 60 30 11/9 11/9 11/9 = 0 0 0 90 90 90 90 90 0 0 0 20 30 50 50 60 40 20 * 0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 30 20 10 0 0 90 0 0 0 0 0 0 0 10 10 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

 Padding

k xk  Complexity: . M*N*K2

Z. Li, ECE 484 Digital Image Processing, 2019. p.3 Properties of Linear Filtering  Convolution properties . Shift-Invariant: f(m-k, n-j)*h = g(m-k, n-j), if f*h=g . Associative:

f*h1*h2 = f*(h1*h2) this can save a lot of complexity . Distributive: f*h1 + f*h2 = f*(h1+h2) useful in SIFT’s DoG filtering.  Applications . Scale space filtering with successive Gaussian . DoG filtering with difference of Gaussian blurred images

* =

Z. Li, ECE 484 Digital Image Processing, 2019. p.4 Image Smoothing/De-noising  Average Smoothing (Box Filter - also an acceleration scheme)

 Gaussian Smoothing

Z. Li, ECE 484 Digital Image Processing, 2019. p.5 Image Sharpening  Sharpen an image:

blurred image unit impulse (identity)

scaled Gaus Laplacian of impulse sian Gaussian

Let’s add it back:

+ α =

original detail sharpened

Z. Li, ECE 484 Digital Image Processing, 2019. p.6 Edge Detection  Difference of Gaussian (DoG)  Laplacian of Gaussian (loG)  Sobel

Z. Li, ECE 484 Digital Image Processing, 2019. p.7 Linear Filter Acceleration  Separable Filtering

2D convolution (center location only)

The filter factors into a product of 1D filters:

Perform convolution = along rows: *

Followed by convolution = along the remaining column: *

Z. Li, ECE 484 Digital Image Processing, 2019. p.8 Linear Filter Acceleration  Approx Separable Filter via SVD

kernel h U S V sig. significant = noise noise n o s i s i g e n i f i c a n t  DoG approximation (HW-2)

Z. Li, ECE 484 Digital Image Processing, 2019. p.9 Outline  Recap of Lec 05  Fourier Transform - Freq Domain Representation of Images  Image Sampling . Nyquist Sampling Rate and Aliasing . Super-Resolution and Interpolation  Summary

Z. Li, ECE 484 Digital Image Processing, 2019. p.10 Fourier Series  Any periodic contiunous signals can be expressed as linear combinations of sinusoid

 While coefficients are:

Joseph Fourier 1768-1830

Z. Li, ECE 484 Digital Image Processing, 2019. p.11 Even and Odd Functions  An arbitrary function can be expressed in term of even and odd functions

E(-x) = E(x)

O(-x) = -O(x)

Z. Li, ECE 484 Digital Image Processing, 2019. p.12 Fourier Cosine Series

Because cos(mt) is an even function, we can write an even function, f(t), as:

where series Fm is computed as

Here we suppose f(t) is over the interval (–π,π).

Z. Li, ECE 484 Digital Image Processing, 2019. 13 Fourier Sine Series

Because sin(mt) is an odd function, we can write any odd function, f(t), as:

where the series F’m is computed as

Z. Li, ECE 484 Digital Image Processing, 2019. 14 Fourier Series

So if f(t) is a general function, neither even nor odd, it can be written:

Even component Odd component where the Fourier series is

Z. Li, ECE 484 Digital Image Processing, 2019. 15 The Fourier Transform

Let F(m) incorporates both cosine and sine series coefficients, with the sine series distinguished by making it the imaginary component:

Let’s now allow f(t) range from – to  , we rewrite:

F(u) is called the Fourier Transform of f(t). We say that f(t) lives in the “time domain,” and F(u) lives in the “frequency domain.” u is called the frequency variable.

Z. Li, ECE 484 Digital Image Processing, 2019. 16 The Inverse Fourier Transform We go from f(t) to F(u) by

Fourier Transform

Given F(u), f(t) can be obtained by the inverse Fourier transform Inverse Fourier Transform

Z. Li, ECE 484 Digital Image Processing, 2019. 17 Fourier Transform Example  f(x) with limited time domain support

Z. Li, ECE 484 Digital Image Processing, 2019. p.18 2D Fourier Transform  A straight extension of 1D FT:

. F(u,v) is complex in general, can be written as,

. |F(u,v)| is the magnitude spectrum . tan-1(FI(u,v)/FR(u,v)) is the phase spectrum

Z. Li, ECE 484 Digital Image Processing, 2019. p.19 2D FT Example

 Spatial domain block: u . f(x,y) = 1 if inside [-X, X] and [-Y, Y]

v

Z. Li, ECE 484 Digital Image Processing, 2019. p.20 FT example - Gaussian  Gaussian Kernel has Gaussian spectrum

Z. Li, ECE 484 Digital Image Processing, 2019. p.21 FT example - Circular Function

Z. Li, ECE 484 Digital Image Processing, 2019. p.22 FT Example - Impulses in spatial domain

Question: what would be F(u,v) of :

Z. Li, ECE 484 Digital Image Processing, 2019. p.23 Freq Interpretation

The central part of FT, i.e. the low frequency components are responsible for the general gray-level appearance of an image.

The high frequency components of FT are responsible for the detail information of an image.

Z. Li, ECE 484 Digital Image Processing, 2019. 24 FT properties  2D FT properties

Z. Li, ECE 484 Digital Image Processing, 2019. p.25 Fourier Analysis of Pictures  Repeated patterns . Lunar Orbital Image (1966) . Repeated patterns from rolling shutter . remove artifacts in freq domain

Z. Li, ECE 484 Digital Image Processing, 2019. p.26 Magnitude vs Phase  Magnitude seems to carry more information, also easy to explain in frequency  Phase is also carrying important information

Z. Li, ECE 484 Digital Image Processing, 2019. p.27 Discrete Fourier Transform  DFT

real(A) imag(A)

Forward transform u=0

Inverse transform

basis

u=7 n n

Z. Li, ECE 484 Digital Image Processing, 2019. p.28 DFT vs. DCT

1D-DCT 1D-DFT

real a imag a a ( ) ( )

u=0 u=0

u=7 u=7

Z. Li, ECE 484 Digital Image Processing, 2019. n=7 p.29 DFT spectrum  DFT domain representation and processing

real-valued input

Z. Li, ECE 484 Digital Image Processing, 2019. p.30 2-D Fourier basis

 Fourier basis: low freq is in the middle

real imag

real( ) imag( )

Z. Li, ECE 484 Digital Image Processing, 2019. p.31 LP, HP and BP in Freq Domain  Filtering in frequency domain

BP filter to remove repetitive noises LP filter and smoothing Z. Li, ECE 484 Digital Image Processing, 2019. p.32 Outline  Recap of Lec 08  Image Sampling . subsampling . interpolation  Summary

Z. Li, ECE 484 Digital Image Processing, 2019. p.33 Image Scaling  Need to resample images

This image is too big to fit on the screen. How can we reduce it?

How to generate a half- sized version?

Z. Li, ECE 484 Digital Image Processing, 2019. p.34 Image sub-sampling  sub sample without filtering, what is wrong ?

1/8

1/4

Throw away every other row and column to create a 1/2 size image - called image sub-sampling

Z. Li, ECE 484 Digital Image Processing, 2019. p.35 Image sub-sampling  Aliasing...

1/2 (2x zoom) 1/4 1/8 (4x zoom) Why does this look so crufty?

Z. Li, ECE 484 Digital Image Processing, 2019. p.36 Even worse for synthetic images  Aliasing effect

Z. Li, ECE 484 Digital Image Processing, 2019. p.37 Sampling and the Nyquist rate  Aliasing can arise when you sample a continuous signal or image . occurs when your sampling rate is not high enough to capture the amount of detail in your image . Can give you the wrong signal/image—an alias . formally, the image contains structure at different scales . called “frequencies” in the Fourier domain . the sampling rate must be high enough to capture the highest frequency in the image  To avoid aliasing: . sampling rate > 2 * max frequency in the image . This minimum sampling rate is called the Nyquist rate

Z. Li, ECE 484 Digital Image Processing, 2019. p.38 Sampling in Time Domain  Computer needs a discrete representation of signals

• Many signals originate as continuous-time signals, e.g. conventional music or voice • By sampling a continuous-time signal at isolated, equally-spaced points in time, we obtain a sequence of numbers

Ts

n  {…, -2, -1, 0, 1, 2,…} t T Ts is the sampling period. s s(t)

Sampled analog impulse waveform train Z. Li, ECE 484 Digital Image Processing, 2019. p.39 Spectrum of several special functions  Spatial-Frequency signatures

Z. Li, ECE 484 Digital Image Processing, 2019. p.40 Sampling Consequence in Freq Domain  Multiplication with sampling train function, is convolving in freq domain • Replicates spectrum of continuous-time signal At offsets that are integer multiples of sampling frequency

• Fourier series of impulse train where s = 2  fs

• Example Modulation by Modulation by cos(2 cos( t)  t) F() s G() s

  -2f 2f max max s s s s

Z. Li, ECE 484 Digital Image Processing, 2019. p.41 Illustration of Sampling Theorem  1-D example

Z. Li, ECE 484 Digital Image Processing, 2019. p.42 Reconstruction  Recon via convolving with Sinc function (low pass filtering in freq domain)

Z. Li, ECE 484 Digital Image Processing, 2019. p.43 Aliasing from under-sampled reconstruction  Aliasing effect:

 Example:

Z. Li, ECE 484 Digital Image Processing, 2019. p.44 Shannon/Nyquist Sampling Theorem  What is the sampling rate to recover the original continuous signal w/o loss ? • A continuous-time signal x(t) with frequencies no higher than fmax can be reconstructed from its samples x[n] = x(n Ts) if the samples are taken at a rate fs which is greater than 2 fmax.

Nyquist rate = 2 fmax

Nyquist frequency = fs/2.

• What happens if fs = 2fmax?

• Consider a sinusoid sin(2 p fmax t)

Use a sampling period of Ts = 1/fs = 1/2fmax.

Sketch: sinusoid with zeros at t = 0, 1/2fmax, 1/fmax, …

Z. Li, ECE 484 Digital Image Processing, 2019. p.45 Sampling in 2D  Aliasing in 2D

Z. Li, ECE 484 Digital Image Processing, 2019. p.46 Summary  Fourier Transform . A frequency domain representation of spatial / temporal signals . Usually complex can be expressed as magnitude and phase . discrete version DFT . real representation from DCT

 Sampling Theorem and Aliasing . Sampling is multiplying a continuous signal with a periodic signal (sampling train) . Freq consequence is convolving and replicating the spectrum . If sampling freq is not large enough, aliasing

Z. Li, ECE 484 Digital Image Processing, 2019. p.47