Ph. Cattin: Signalprocessing Background Contents Signalprocessing Contents Background Abstract 2 1 Motivation Motivation - Fetoscope 4 Motivation Mariner 5 Introduction to Signal and Motivation - Mariner (2) 6 2 Transformations in the Frequency Domain Image Processing Introduction 8 2.1 Introduction to the Fourier Transform Definition of the Fourier Transform 10 Prof. Dr. Philippe Cattin Fourier Transform Example 11 Extension of the Fourier Transform to 2D Functions 12 Fourier Transform Example of a 2D Function 13 MIAC, University of Basel 2.2 The Discrete Fourier Transform (DFT) Definition of the 1D Discrete Fourier Transform 15 1D Discrete Fourier Transformation Example 16 March 8th/15th, 2016 The 2D Discrete Fourier Transform 17 The 2D Discrete Fourier Transform (2) 18 Remark 1: The Scaling Terms 19 Remark 2: Existence of the DFT 20 2D Discrete Fourier Transform Example 21 2.3 Properties of the Fourier Transform Introduction 23 Fourier Transform of Even, Odd Functions 24 Separability 25 Separability Example 26 Translation 27 Translation in the Fourier Domain 28 Translation in the Fourier Domain Example 29 Translation in the Fourier Domain Example (2) 30 Translation in the Image Domain 31 Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to SignalTranslation and Image inProcessing the Image Domain Example March 8th/15th, 201632 1 of 126 11.04.2016 09:37 2 of 126 11.04.2016 09:37 Rotation 33 2D Walsh Transform 68 Rotation Example 34 2D Walsh Transformation Kernel 69 Distributivity and Scaling 35 2D Walsh Transformation Example 70 Average Value 36 3.3 Hadamard Transform Laplacian 37 Hadamard Transform 72 Convolution and Correlation 38 2D Hadamard Transformation 73 Convolution (1) 39 Generation of the 2D Hadamard Kernel 74 Convolution (2) 40 Drawback of the Hadamard Kernel Ordering 75 Convolution with an Impulse Function 41 Ordered 2D Hadamard Transformation Kernel 76 Convolution with an Impulse Function (2) 42 3.4 Discrete Cosine Transform Discrete Convolution 43 Discrete Cosine Transform 78 Two-Dimensional Continuous Convolution 44 2D Discrete Cosine Transformation Kernel 79 Two-Dimensional Discretised Convolution 45 2D Discrete Cosine Kernel used in JPEG Compression 80 2D Convolution Example 46 4 Hotelling Transform Correlation 47 4.1 Statistical Shape Models Correlation Theorem 48 4.1.1 Motivation Correlation Example 49 Motivation 84 2D Correlation Example 50 Organ Shapes Vary 85 2.4 The Fast Fourier Transform (FFT) 4.1.2 Mathematical Background Computational Complexity 52 Principal Component Analysis (PCA) 87 Computational Complexity (2) 53 Principal Component Analysis 88 Derivation of the FFT Algorithm 54 Principal Component Analysis (2) 89 The Inverse FFT 55 Principal Component Analysis (3) 90 3 Other Separable Image Transforms The Curse of High Dimensionality... 91 3.1 Unitary Image Transforms The Curse of High Dimensionality... 92 Unitary Image Transforms 58 Other Possibility 93 Unitary Image Transforms (2) 59 PCA Assumptions 94 Separability and Symmetry 60 But... 95 Separability and Symmetry (2) 61 4.1.3 Shape Representation Separability and Symmetry Example 62 Shape Representation 97 Matrix Notation 63 Shape Representation - Point Cloud 98 Principle of the 2D Unitary Transforms 64 4.1.4 Establishment of Correspondence 3.2 Walsh Transform Point-to-Point Correspondence 100 Walsh Transform 66 Point-to-Point Correspondence 101 1D Walsh Transformation Kernel 67 Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal4.1.5 and Shape Image AlignmentProcessing March 8th/15th, 2016 3 of 126 11.04.2016 09:37 4 of 126 11.04.2016 09:37 Alignment and Correspondence 103 Application Scenario: 3D Information 138 4.1.6 Point Distribution Models Prediction based on Partial 3D Information 139 Shape Space 105 Results for Known 3D Points 140 Shape Space (2) 106 Results when Predicting with the 4D Motion Model 141 Point Distribution Models (PDM) 107 4.1.7 Modelling of Shape Variability Modelling of Shape Variability 109 Modelling of Shape Variability 110 Modelling of Shape Variability 111 What can we do with shapes in a reduced space? 112 4.1.8 Model Evaluation Model Evaluation 114 Compactness 115 Generalisation 116 GenerGeneralisationaliation (2) 117 Generalisation (3) 118 Specificity 119 4.2 Applications 4.2.1 Segmentation Segmentation 122 Statistical Masseter Model 123 Fitting Process 124 Resulting Segmentations 125 4.2.2 Implant Design Computer-Assisted Orthopedic Implant Design 127 Computer-Assisted Orthopedic Implant Design 128 Computer-Assisted Orthopedic Implant Design 129 Proposed Methodology 130 4.2.3 Modelling Organ Motion Problem Statement 132 State-of-the-Art 133 Foundation: 4D MRI 134 Prediction over long Time Scales 135 Population-based Statistical Drift Model 136 4D Statistical Motion Model 137 Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016 5 of 126 11.04.2016 09:37 6 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Motivation Abstract (2) This chapter deals with the signal processing background necessary to Motivation - Fetoscope (4) understand the underlying mathematics behind many Computer Vision algorithms. In particular the Fourier Transform, the Discrete Fourier Transform, and the Fast Fourier Transform are discussed. The homomorphic filter used for this example uses the Fourier Transform. Fig 3.1: Homomorphic filter example of a fetoscope image Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016 7 of 126 11.04.2016 09:37 8 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Motivation Ph. Cattin: Signalprocessing Background Motivation Motivation Mariner (5) Motivation - Mariner (2) (6) Fig 3.3: Original Mariner 6 martian image Fig 3.4: Log Fourier spectra of the image Fig 3.2: A Fourier Transform based notch filter example Fig 3.5: Notch filtered log spectra Fig 3.6: Notch filtered image Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016 9 of 126 11.04.2016 09:37 10 of 126 11.04.2016 09:37 Transformations in the Introduction to the Frequency Domain Fourier Transform Introduction (8) A periodic function can be represented by the sum of sines and cosines of different frequencies, multiplied by a different coefficient (Fourier Series) Non-periodic functions can also be represented as the integral of sines/cosines multiplied by a weighting function (Fourier Transformation) Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016 11 of 126 11.04.2016 09:37 12 of 126 11.04.2016 09:37 The variable appearing in the Fourier Transform is often called the Definition of the Fourier Transform (10) Frequency Variable. The name arises from the exponential term, that can be rewritten using Euler's Formula Let be a continuous function of real variable . The Fourier Transform of , denoted is defined by the equation (3.8) (3.1) where . Given , can be obtained by using the inverse Fourier Transform (3.2) The Fourier transform pair exists, if is continuous and integrable and is integrable, which is almost always satisfied in practice. In Computer Vision we are mainly concerned with real functions. The Fourier Transform of a real function, however, is generally complex, thus (3.3) where and are the real and imaginary components of , respectively. Often it is convenient to express it in exponential form (3.4) where (3.5) and (3.6) The magnitude function is called the Fourier Spectrum of and its Phase Angle The square of the spectrum (3.7) is commonly referred to as Power Spectrum or Spectral Density. Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016 13 of 126 11.04.2016 09:37 14 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Introduction to the Fourier Transform Ph. Cattin: Signalprocessing Background Introduction to the Fourier Transform Fourier Transform Example (11) Extension of the Fourier Transform (12) to 2D Functions Consider the simple function shown in Fig 3.7. Its Fourier transform is obtained from Eq 3.1 as follows: The Fourier Transformation can be easily extended to 2D functions . If the function is continuous and integrable and is integrable, the Fourier Transform pair exists (3.11) Fig 3.7: A simple function and the inverse Fourier Transform (3.9) (3.12) Similar to the 1D case, the Fourier Spectrum, Phase, and Power Spectrum can be defined as follows Fig 3.8: Fourier spectrum (3.13) (3.14) (3.15) As is a complex function, we calculate the Fourier spectrum for visualisation purposes (3.10) Figure 3.8 shows a plot of . Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016 15 of 126 11.04.2016 09:37 16 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Introduction to the Fourier Transform Fourier Transform Example of a 2D (13) Function Consider the simple function shown in Fig 3.9. Its Fourier transform is obtained from Eq 3.11 as follows: Fig 3.10: Fourier spectrum Fig 3.9: A simple function (3.16) As is a complex function, we calculate the Fourier spectrum for visualisation purposes (3.17) Figure 3.10 shows a plot of . Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016 17 of 126 11.04.2016 09:37 18 of 126 11.04.2016 09:37 The Discrete Fourier Ph. Cattin: Signalprocessing Background The Discrete Fourier Transform (DFT) 1D Discrete Fourier (16) Transform (DFT) Transformation Example Definition of the 1D Discrete (15) Fourier Transform In Computer Vision the continuous functions are generally discretised into Original signal Fourier spectrum Phaseangle a sequence (3.18) by taking samples units apart.