052600 VU Signal and Image Processing Filter Design Torsten Möller + Jana Kemnitz + Raphael Sahann [email protected] [email protected] [email protected] vda.cs.univie.ac.at/Teaching/SIP/18w/ 1 Sources • Oppenheim&Shafer — Chapter 7 © Torsten Möller 2 FFT - Key Points • Fast Fourier transform (FFT) is an algorithm to implement DFT • Most commonly used FFT is the Cooley–Tukey algorithm 2 • Number of multiplications are reduced from O ( N ) to O ( N l o g 2 (N) with N sampling points (data size) • Basic idea is recursive -> divide the transform into two pieces of size N/2 at each step • Widely used for many applications in engineering, science, and mathematics ©© TorstenJana Kemnitz Möller 3 FFT - Key Points • Basic idea is recursive - > divide the transform into two pieces of size N/2 at each step ©© TorstenJana Kemnitz Möller 4 Overview • Motivation • What is a filter? • Filter in image processing • Anti-Aliasing Filter • Realistic filters © Torsten Möller 5 “Young man, in mathematics you don't understand things. You just get used to them.” – John von Neumann, 1979, The Dancing Wu Li Masters: An Overview of the New Physics (1979) by Gary Zukav, Bantam Books, p. 208, footnote. (https://en.wikiquote.org/wiki/John_von_Neumann) 6 What is a filter? 7 Filter - some Examples In signal processing, a filter is a device or process which can be described by a mathematical function that modifies an input signal x(t) or x[n] into an output signal y[t] or y[n] x[n] Filter y[n] y(t) x(t) Optical filter (zone plate) x[n] Y[n] ^ f(x,y) f(x,y) ©© TorstenJana Kemnitz Möller 8 Filter - Image Processing • Filtering is a technique for modifying or enhancing an image. • For example, an image can be filtered to emphasize certain features or remove other features. • Image processing operations implemented with filtering include smoothing, sharpening, and edge enhancement. • In image processing, a kernel, convolution matrix, or mask is a small matrix. It is used for blurring, sharpening, embossing, edge detection, and more. This is accomplished by doing a convolution between a kernel and an image. ©© TorstenJana Kemnitz Möller 9 Filter - Image Processing • In image processing, a kernel, convolution matrix, or mask is a small matrix. It is used for blurring, sharpening, embossing, edge detection, and more. This is accomplished by doing a convolution between a kernel and an image. = ©© TorstenJana Kemnitz Möller 10 Filter - Image Processing • Identity • Edge detection • Sharpen • Gaussian blur ©© TorstenJana Kemnitz Möller 11 Linear Spatial Filtering Gonzalez & Woods - Digital Image Processing (3rd Edition) ©© Raphael Torsten MöllerSahann 12 Smoothing Filter Gonzalez & Woods - Digital Image Processing (3rd Edition) ©© Raphael Torsten MöllerSahann 13 Smoothing Filter Gonzalez & Woods - Digital Image Processing (3rd Edition) ©© Raphael Torsten MöllerSahann 14 Smoothing Filter - Application Gonzalez & Woods - Digital Image Processing (3rd Edition) ©© Raphael Torsten MöllerSahann 15 Instagram Filters © Torsten Möller 16 When do we need filters? 17 Use of filters • histogram equalization • denoising • getting rid of artifacts (notch filtering) • deconvolution • re-sampling / anti-aliasing © Torsten Möller 18 Histogram Equalization Gonzalez & Woods - Digital Image Processing (3rd Edition) ©© Raphael Torsten MöllerSahann 19 Noise Filtering Gonzalez & Woods - Digital Image Processing (3rd Edition) © Raphael Sahann © Torsten Möller 20 Bandreject Filtering Gonzalez & Woods - Digital Image Processing (3rd Edition) © Raphael Sahann © Torsten Möller 21 Estimating the Degradation Function Gonzalez & Woods - Digital Image Processing (3rd Edition) g(x, y)=h(x, y) ? f(x, y)+⌘(x, y) G(u, v)=H(u, v)F (u, v)+N(u, v) ©© Raphael Torsten Möller Sahann 22 Estimating the Degradation Function Gonzalez & Woods - Digital Image Processing (3rd Edition) g(x, y)=h(x, y) ? f(x, y)+⌘(x, y) (Time domain) G(u, v)=H(u, v)F (u, v)+N(u, v) (Frequency domain) ©© Raphael Torsten Möller Sahann 23 Inverse Filtering (Deconvolution) G(u, v) Fˆ(u, v)= H(u, v) • simple estimate of the transform by dividing the transform by the degradation function • cannot recover undegraded image fully, because N(u,v) is not known — very small values of H(u,v) will dominate the estimate N(u, v) Fˆ(u, v)=F (u, v)+ H(u, v) ©© Raphael Torsten Möller Sahann 24 Estimation by Experimentation Gonzalez & Woods - Digital Image Processing (3rd Edition) ©© Raphael Torsten Möller Sahann 25 Gonzalez & Woods - Digital Image Processing (3rd Edition) © Raphael Sahann 26 Filtering to avoid aliasing 27 General Process of Sampling and Reconstruction Original function Sampled function Acquisition e.g., supersampling Reconstructed Reconstruction Function Re-sampled function Resampling e.g., resample at screen resolution © Torsten Möller 28 Sampling in frequency space F y ω F ω x F ω © Torsten Möller 29 How so? – by Convolution Theorem Spatial Domain: Frequency Domain: Mathematically: • Convolution: f(x)*h(x) • Multiplication: ∞ ∫ f (t)⋅ h(x − t)dt −∞ Evaluated at discrete points© Torsten(sum) Möller 30 € Reconstruction? — Convolution Theorem Spatial Domain: Frequency Domain: F ω Convolution: Multiplication: © Torsten Möller 31 Sampling Theorem • A signal can be reconstructed from its samples without loss of information if the original signal has no frequencies above 1/2 of the sampling frequency • For a given bandlimited function, the rate at which it must be sampled (to have perfect reconstruction) is called the Nyquist frequency • Due to Claude Shannon (1949) © Torsten Möller 32 Example 2D 1D Given Given Needed Needed © Torsten Möller 33 Once Again ... Pre-filter Post-aliasing Pre-aliasingsampling Reconstruction filter © Torsten Möller 34 In the frequency domain Original function Sampled function Acquisition Reconstructed Reconstruction Function Re-sampled function Resampling © Torsten Möller 35 Pipeline - Example Spatial domain Frequency domain sampling x * x smoothing * © Torsten Möller 36 Pipeline - Example (2) Spatial domain Frequency domain smoothing * x x Re-sampling * © Torsten Möller 37 Pipeline - Example (3) Spatial domain Frequency domain * x © Torsten Möller 38 Cause of Aliasing • Non-bandlimited signal – prealiasing sampling • Low sampling rate (<= Nyquist) – prealiasing sampling • Non perfect reconstruction – post-aliasing © Torsten Möller 39 Aliasing example © Torsten Möller 40 Aliasing: Sampling a Zone Plate • Zone plate: sin(x2 + y2) • Sampled at 128 x 128 and reconstructed to 512 x 512 using windowed sinc • Left rings: part of the signal • Right rings: aliasing due to undersampling © Torsten Möller 41 Antialiasing 1: Pre-Filtering Original function Band-limited function Pre-Filtering Sampled Acquisition Function Reconstructed function Reconstruction © Torsten Möller 42 Antialiasing 2: Uniform Supersampling • Increasing the sampling rate moves each copy of the spectra further apart, potentially reducing the overlap and thus aliasing • Low-pass filter and then the resulting signal is re-sampled at image resolution © Torsten Möller 43 Point vs. supersampling © Torsten Möller 44 What is the ideal reconstruction filter? 45 Ideal Reconstruction • Box filter in frequency domain = Pass-band stop-band • Sinc Filter in spatial domain • Sinc has infinite extent – not practical Ideal filter Smoothing Practical filter Post-aliasing © Torsten Möller 46 Ideal Reconstruction • Use the sinc function – to bandlimit the sampled signal and remove all copies of the spectra introduced by sampling • But how? • filter in the frequency domain! • But how expensive? • Frequency domain: O(n.logn) • Spatial domain: O(n.k) © Torsten Möller 47 Ideal Reconstruction • Use the sinc function – to bandlimit the sampled signal and remove all copies of the spectra introduced by sampling • But: • The sinc has infinite extent (i.e. IIR = infinite impulse response) and we must use simpler filters with finite extents. • The windowed versions of sinc may introduce ringing artifacts which are perceptually objectionable. © Torsten Möller 48 Reconstruction Kernels • Nearest Neighbor (Box) • Linear • Sinc • Gaussian • Many others Spatial d. Frequency d. © Torsten Möller 49 Interpolation example Nearest neighbor Linear Interpolation © Torsten Möller 50 Reconstructing with Sinc: Ringing • remember Gibbs phenomenon • see also http://pages.jh.edu/~signals/fourier2/index.html © Torsten Möller 51 Ideal filters • Also have ringing in pass/stop bands • Realizable filters do not have sharp transitions π π π Low-pass filter band-pass filter high-pass filter © Torsten Möller 52 Realistic filters 53 Filter Specifications 0.99 ≤ Heff (jΩ) = 1.01 0 ≤ Ω ≤ 2π(2000) Heff (jΩ) ≤ 0.001 2π(3000)≤ Ω δ1 = 0.01 δ2 = 0.001 Ωp = 2π(2000) Ωs = 2π(3000) Adapted ©from Torsten 2005 Möller Güner Arslan 54 Butterworth Lowpass Filters •The Butterworth filter designed to have as flat frequency response as possible (no ripples) in the pass-band and zero roll off response in the stop-band •One of the most commonly used digital filters •Fast and simple to use •Since they are frequency-based, the effect of filtering can be easily understood and predicted Main disadvantage: expense of a wide transition band as the filter changes from the pass band to the stop band Adapted ©from Torsten 2005 Möller Güner Arslan 55 Butterworth Lowpass Filters • Passband is designed to be maximally flat • The magnitude-squared function is of the form 2 1 Hc (jΩ) = 2N 1 + (jΩ / jΩc ) 2 1 Hc (s) = 2N 1 + (s / jΩc ) 1 / 2N (jπ / 2N)(2k +N−1) sk = (− 1) (jΩc )= Ωce for k = 0,1,...,2N - 1 Adapted ©from Torsten 2005 Möller Güner Arslan 56 Learning goals • You should know the definition of a filter • You should know typical image filter kernels (edge detection, sharpen, smoothing) • You should understanding the process: estimating the Degradation Function • You should understand how filtering is used to avoid aliasing • You should understand the differences between an “ideal” and a “realistic” filter ©© TorstenJana Kemnitz Möller 57.