Ph. Cattin: Signalprocessing Background Contents
Signalprocessing Contents Background Abstract 2 1 Motivation Motivation - Fetoscope 4 Motivation Mariner 5 Introduction to Signal and Motivation - Mariner (2) 6 2 Transformations in the Frequency Domain Image Processing Introduction 8 2.1 Introduction to the Fourier Transform Definition of the Fourier Transform 10 Prof. Dr. Philippe Cattin Fourier Transform Example 11 Extension of the Fourier Transform to 2D Functions 12 Fourier Transform Example of a 2D Function 13 MIAC, University of Basel 2.2 The Discrete Fourier Transform (DFT) Definition of the 1D Discrete Fourier Transform 15 1D Discrete Fourier Transformation Example 16 March 8th/15th, 2016 The 2D Discrete Fourier Transform 17 The 2D Discrete Fourier Transform (2) 18 Remark 1: The Scaling Terms 19 Remark 2: Existence of the DFT 20 2D Discrete Fourier Transform Example 21 2.3 Properties of the Fourier Transform Introduction 23 Fourier Transform of Even, Odd Functions 24 Separability 25 Separability Example 26 Translation 27 Translation in the Fourier Domain 28 Translation in the Fourier Domain Example 29 Translation in the Fourier Domain Example (2) 30 Translation in the Image Domain 31
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to SignalTranslation and Image inProcessing the Image Domain Example March 8th/15th, 201632
1 of 126 11.04.2016 09:37 2 of 126 11.04.2016 09:37 Rotation 33 2D Walsh Transform 68 Rotation Example 34 2D Walsh Transformation Kernel 69 Distributivity and Scaling 35 2D Walsh Transformation Example 70 Average Value 36 3.3 Hadamard Transform Laplacian 37 Hadamard Transform 72 Convolution and Correlation 38 2D Hadamard Transformation 73 Convolution (1) 39 Generation of the 2D Hadamard Kernel 74 Convolution (2) 40 Drawback of the Hadamard Kernel Ordering 75 Convolution with an Impulse Function 41 Ordered 2D Hadamard Transformation Kernel 76 Convolution with an Impulse Function (2) 42 3.4 Discrete Cosine Transform Discrete Convolution 43 Discrete Cosine Transform 78 Two-Dimensional Continuous Convolution 44 2D Discrete Cosine Transformation Kernel 79 Two-Dimensional Discretised Convolution 45 2D Discrete Cosine Kernel used in JPEG Compression 80 2D Convolution Example 46 4 Hotelling Transform Correlation 47 4.1 Statistical Shape Models Correlation Theorem 48 4.1.1 Motivation Correlation Example 49 Motivation 84 2D Correlation Example 50 Organ Shapes Vary 85 2.4 The Fast Fourier Transform (FFT) 4.1.2 Mathematical Background Computational Complexity 52 Principal Component Analysis (PCA) 87 Computational Complexity (2) 53 Principal Component Analysis 88 Derivation of the FFT Algorithm 54 Principal Component Analysis (2) 89 The Inverse FFT 55 Principal Component Analysis (3) 90 3 Other Separable Image Transforms The Curse of High Dimensionality... 91 3.1 Unitary Image Transforms The Curse of High Dimensionality... 92 Unitary Image Transforms 58 Other Possibility 93 Unitary Image Transforms (2) 59 PCA Assumptions 94 Separability and Symmetry 60 But... 95 Separability and Symmetry (2) 61 4.1.3 Shape Representation Separability and Symmetry Example 62 Shape Representation 97 Matrix Notation 63 Shape Representation - Point Cloud 98 Principle of the 2D Unitary Transforms 64 4.1.4 Establishment of Correspondence 3.2 Walsh Transform Point-to-Point Correspondence 100 Walsh Transform 66 Point-to-Point Correspondence 101 1D Walsh Transformation Kernel 67 Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal4.1.5 and Shape Image AlignmentProcessing March 8th/15th, 2016
3 of 126 11.04.2016 09:37 4 of 126 11.04.2016 09:37 Alignment and Correspondence 103 Application Scenario: 3D Information 138 4.1.6 Point Distribution Models Prediction based on Partial 3D Information 139 Shape Space 105 Results for Known 3D Points 140 Shape Space (2) 106 Results when Predicting with the 4D Motion Model 141 Point Distribution Models (PDM) 107 4.1.7 Modelling of Shape Variability Modelling of Shape Variability 109 Modelling of Shape Variability 110 Modelling of Shape Variability 111 What can we do with shapes in a reduced space? 112 4.1.8 Model Evaluation Model Evaluation 114 Compactness 115 Generalisation 116 GenerGeneralisationaliation (2) 117 Generalisation (3) 118 Specificity 119 4.2 Applications 4.2.1 Segmentation Segmentation 122 Statistical Masseter Model 123 Fitting Process 124 Resulting Segmentations 125 4.2.2 Implant Design Computer-Assisted Orthopedic Implant Design 127 Computer-Assisted Orthopedic Implant Design 128 Computer-Assisted Orthopedic Implant Design 129 Proposed Methodology 130 4.2.3 Modelling Organ Motion Problem Statement 132 State-of-the-Art 133 Foundation: 4D MRI 134 Prediction over long Time Scales 135 Population-based Statistical Drift Model 136 4D Statistical Motion Model 137 Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
5 of 126 11.04.2016 09:37 6 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Motivation Abstract (2) This chapter deals with the signal processing background necessary to Motivation - Fetoscope (4) understand the underlying mathematics behind many Computer Vision algorithms. In particular the Fourier Transform, the Discrete Fourier Transform, and the Fast Fourier Transform are discussed. The homomorphic filter used for this example uses the Fourier Transform.
Fig 3.1: Homomorphic filter example of a fetoscope image
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
7 of 126 11.04.2016 09:37 8 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Motivation Ph. Cattin: Signalprocessing Background Motivation
Motivation Mariner (5) Motivation - Mariner (2) (6)
Fig 3.3: Original Mariner 6 martian image Fig 3.4: Log Fourier spectra of the image
Fig 3.2: A Fourier Transform based notch filter example
Fig 3.5: Notch filtered log spectra Fig 3.6: Notch filtered image
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
9 of 126 11.04.2016 09:37 10 of 126 11.04.2016 09:37 Transformations in the Introduction to the Frequency Domain Fourier Transform
Introduction (8)
A periodic function can be represented by the sum of sines and cosines of different frequencies, multiplied by a different coefficient (Fourier Series) Non-periodic functions can also be represented as the integral of sines/cosines multiplied by a weighting function (Fourier Transformation)
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
11 of 126 11.04.2016 09:37 12 of 126 11.04.2016 09:37 The variable appearing in the Fourier Transform is often called the Definition of the Fourier Transform (10) Frequency Variable. The name arises from the exponential term, that can be rewritten using Euler's Formula Let be a continuous function of real variable . The Fourier Transform of , denoted is defined by the equation (3.8)
(3.1)
where .
Given , can be obtained by using the inverse Fourier Transform
(3.2)
The Fourier transform pair exists, if is continuous and integrable and is integrable, which is almost always satisfied in practice.
In Computer Vision we are mainly concerned with real functions. The Fourier Transform of a real function, however, is generally complex, thus
(3.3)
where and are the real and imaginary components of , respectively. Often it is convenient to express it in exponential form
(3.4)
where
(3.5)
and
(3.6)
The magnitude function is called the Fourier Spectrum of and its Phase Angle
The square of the spectrum
(3.7)
is commonly referred to as Power Spectrum or Spectral Density. Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
13 of 126 11.04.2016 09:37 14 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Introduction to the Fourier Transform Ph. Cattin: Signalprocessing Background Introduction to the Fourier Transform
Fourier Transform Example (11) Extension of the Fourier Transform (12) to 2D Functions Consider the simple function shown in Fig 3.7. Its Fourier transform is obtained from Eq 3.1 as follows: The Fourier Transformation can be easily extended to 2D functions . If the function is continuous and integrable and is integrable, the Fourier Transform pair exists
(3.11)
Fig 3.7: A simple function and the inverse Fourier Transform
(3.9) (3.12)
Similar to the 1D case, the Fourier Spectrum, Phase, and Power Spectrum can be defined as follows
Fig 3.8: Fourier spectrum (3.13) (3.14)
(3.15)
As is a complex function, we calculate the Fourier spectrum for visualisation purposes
(3.10)
Figure 3.8 shows a plot of .
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
15 of 126 11.04.2016 09:37 16 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Introduction to the Fourier Transform
Fourier Transform Example of a 2D (13) Function
Consider the simple function shown in Fig 3.9. Its Fourier transform is obtained from Eq 3.11 as follows:
Fig 3.10: Fourier spectrum
Fig 3.9: A simple function
(3.16)
As is a complex function, we calculate the Fourier spectrum for visualisation purposes
(3.17)
Figure 3.10 shows a plot of .
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
17 of 126 11.04.2016 09:37 18 of 126 11.04.2016 09:37 The Discrete Fourier Ph. Cattin: Signalprocessing Background The Discrete Fourier Transform (DFT) 1D Discrete Fourier (16) Transform (DFT) Transformation Example
Definition of the 1D Discrete (15) Fourier Transform
In Computer Vision the continuous functions are generally discretised into Original signal Fourier spectrum Phaseangle a sequence
(3.18)
by taking samples units apart. The function can be redefined
Fourier spectrum of shifted Signal with noise shifted by noise Phaseangle of shifted noise where now assumes the discrete values . Fig 3.12: One-dimensional discrete Fourier transformation example. Lower example shows the effect of a phaseshift in the high frequency signal to the phase angle. With this notation, the Discrete Fourier Transform (DFT) can be defined as
and the Inverse Discrete Fourier Transform
Fig 3.11: Sampling a continuous function
The terms and are related by
.
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
19 of 126 11.04.2016 09:37 20 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background The Discrete Fourier Transform (DFT) Ph. Cattin: Signalprocessing Background The Discrete Fourier Transform (DFT)
The 2D Discrete Fourier Transform (17) The 2D Discrete Fourier Transform (18)
The definition of the (in Computer Vision more common) 2D Discrete Fourier (2) Transform (DFT) is then given by As in the 1D case, the discrete function represents samples of the function (3.19) (3.21)
for and , and the Inverse DFT for and .
The sampling increments in the spatial and frequency domain are related by (3.20) (3.22)
and Sampling of the continuous function is in a 2D grid of width and height in the axis, respectively. (3.23)
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
21 of 126 11.04.2016 09:37 22 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background The Discrete Fourier Transform (DFT) Ph. Cattin: Signalprocessing Background The Discrete Fourier Transform (DFT)
Remark 1: The Scaling Terms (19) Remark 2: Existence of the DFT (20)
Because and are a Fourier Transform pair the multiplicative Claim scaling terms can be chosen arbitrary. As images are often digitised in square arrays, thus , the following scaling is often chosen In contrast to the continuous case, existence of the discrete Fourier Transform is of no concern, because both and always exist.
(3.24) Proof
and
(3.25)
Beware, that the scaling term in MATLAB is with the inverse rather than the transform!
The identify follows from the orthogonality condition
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
23 of 126 11.04.2016 09:37 24 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background The Discrete Fourier Transform (DFT) Properties of the Fourier 2D Discrete Fourier Transform (21) Example Transform
Introduction (23)
This part focuses on properties of the Fourier Transform that are of value in the context of Computer Vision.
Although our main interest is in the 2D discrete transform, the underlying concepts are sometimes easier to explain using the 1D continuous form. (a) Original image (b) The Fourier spectrum (c) The log Fourier spectrum Fig 3.13: Example Voyager 2 image of saturn with its Fourier spectrum and the log Fourier spectrum
The dynamic range of Fourier spectra usually is much higher than the typical display device can reliably reproduce. The consequence is that only the brightest parts are shown, see Fig 3.13(b). A useful technique that compensates for this difficulty is of displaying the function
(3.26)
see Fig 3.13(c).
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
25 of 126 11.04.2016 09:37 26 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Properties of the Fourier Transform Ph. Cattin: Signalprocessing Background Properties of the Fourier Transform
Fourier Transform of Even, Odd (24) Separability (25)
Functions The 2D discrete Fourier Transform pair, Eq 3.19 & 3.20, can be expressed in its separable form The fact that the Fourier Transform of a real-valued image yields a complex output might give the impression that information has somehow doubled. This is of course not the case. In fact, for real input (such as images) a number of (3.27) important properties hold for the Fourier Transform:
Spatial Frequency and the Inverse Transformation respectively Domain Domain Hint:
real part even, A function is even if it holds for all real (3.28) real imaginary part : thus symmetric to the odd y-axis. real, even real, even A real-valued function is odd if for all Advantage: real it holds: thus real, odd imaginary, odd The Transformations and can be obtained in two successive symmetric to the origin. applications of 1D Fourier transforms → computationally very efficient.
As can be seen in the above table, the relationships between Fourier This becomes evident, when we rewrite the separable discrete Fourier coefficients are such that the total number of independent variables remains Transform in Eq 3.27 in the form the same. (3.29)
where
(3.30)
we get two 1D Fourier Transforms. The same principle applies for the Inverse Fourier Transform. The following figure illustrates this process:
Fig 3.14: Principle of separability
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
27 of 126 11.04.2016 09:37 28 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Properties of the Fourier Transform Ph. Cattin: Signalprocessing Background Properties of the Fourier Transform
Separability Example (26) Translation (27)
MATLAB example using the 2D FFT function We have to differentiate between two cases
fft2(magic(3)) 1. Translations in the Fourier (Frequency) Domain 2. Translations in the Image Domain ans = Let's assume we know 45.0000 0 0 (3.31) 0 + 0.0000i 13.5000 + 7.7942i 0.0000 - 5.1962i 0 - 0.0000i 0.0000 + 5.1962i 13.5000 - 7.7942i we want to know how a translation in the Fourier Domain by can by MATLAB example using two 1D FFT function calls expressed in
fft(fft(magic(3)).').' (3.32)
ans = and similarly for translations in the Image Domain (3.33) 45.0000 0 0 0 + 0.0000i 13.5000 + 7.7942i 0.0000 - 5.1962i 0 - 0.0000i 0.0000 + 5.1962i 13.5000 - 7.7942i
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
29 of 126 11.04.2016 09:37 30 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Properties of the Fourier Transform Ph. Cattin: Signalprocessing Background Properties of the Fourier Transform
Translation in the Fourier Domain (28) Translation in the Fourier Domain (29)
A Translation of in the Fourier Domain results in Example
(3.34) The discrete Fourier Transform is generally calculated using the Fast Fourier Transform. The A multiplication of with the exponential term and taking the transform FFT implementations, however, generally yield the of the product results in a shift of the origin of the frequency plane to the point frequency domain unsorted as can be seen in Fig 3.15(b) on the right. Fig 3.15(a): Original signal The special case were is often used and yields Shifting the frequency domain by would yield the correct sorting order as depicted in Fig 3.15(c). This can be achieved in the spatial domain by (3.35) multiplying by
(3.37) and Fig 3.15(b): Unsorted spectrum
(3.36) followed by the Fourier transform
Thus the origin of the Fourier Transform of can be moved to the centre of its corresponding frequency square by multiplying by the term . Fig 3.15(c): Sorted spectrum For the 1D case this shift reduces to the term .
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
31 of 126 11.04.2016 09:37 32 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Properties of the Fourier Transform Ph. Cattin: Signalprocessing Background Properties of the Fourier Transform
Translation in the Fourier Domain (30) Translation in the Image Domain (31)
Example (2) A Translation of in the Image Domain results in
MATLAB Code (3.38) img=zeros(256,256); img(128-32:128+32,128-16:128+16)=1; Multiplying with the exponential term IMG=fft2(img); figure; imshow(log10(1+abs(IMG))); (3.39)
Original image Log spectrum with the and taking the inverse Fourier Transform moves the origin of the Image to lowest frequencies at the edges MATLAB Code Note, that a shift in does not affect the magnitude of the Fourier
img=zeros(256,256); Transform (but only its phase), as img(128-32:128+32,128-16:128+16)=1; [x,y]=meshgrid(1:size(img,1),1:size(img,2)); (3.40) img1xy=img.*(-1).^(x+y); IMG=fft2(img1xy); figure; imshow(log10(1+abs(IMG))); This is important to know, as the magnitude is often taken to visualise the Original image Log spectrum with the Fourier Transform. multiplied by lowest frequencies in the centre
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
33 of 126 11.04.2016 09:37 34 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Properties of the Fourier Transform Ph. Cattin: Signalprocessing Background Properties of the Fourier Transform
Translation in the Image Domain (32) Rotation (33)
Example To investigate the influence of rotation we introduce polar coordinates
(3.41)
then and become and respectively.
The direct substitution in the discrete Fourier Transform pair yields
(3.42)
A rotation by in the Image Domain rotates the Fourier Domain by the same Original image log Fourier spectrum Phaseangle angle and vice versa.
Shifted original image Shifted log Fourier spectrum Shifted phaseangle
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
35 of 126 11.04.2016 09:37 36 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Properties of the Fourier Transform Ph. Cattin: Signalprocessing Background Properties of the Fourier Transform
Rotation Example (34) Distributivity and Scaling (35)
MATLAB Sample Code Form the definition of the discrete Fourier Transform pair follows that
% Create test image and show it (3.43) img=zeros(256,256); img(128-32:128+32,128-16:128+16)=1; However, in general figure; imshow(img); Test Image log10 Fourier % Fourier Transform (3.44) Spectrum IMG=fftshift(fft2(img)); In other words, the Fourier Transform and its inverse is distributive over % Show log10 of the Spectrum addition but not over multiplication. figure; imshow(log10(1+abs(IMG))); For two scalars and MATLAB Sample Code (3.45) % Rotate the test image img45=imrotate(img,45,'bicubic','crop'); and figure; imshow(img45); % Fourier Transform (3.46) Rotated Test Rotated log10 IMG45=fftshift(fft2(img45)); Image Fourier Spectrum % Show log10 of the Spectrum figure; imshow(log10(1+abs(IMG45)));
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
37 of 126 11.04.2016 09:37 38 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Properties of the Fourier Transform Ph. Cattin: Signalprocessing Background Properties of the Fourier Transform
Average Value (36) Laplacian (37)
The average value of a discrete 2D function can be defined as The Laplacian of a 2-dimensional function is defined as
(3.49) (3.47) With the definition of the Fourier Transform we get Substituting in Eq 3.19 and assuming yields (3.50)
The Laplacian operator is useful for outlining edges as will be shown in later sections of this lecture. (3.48)
Therefore the average value is related to the Fourier transform at the frequency 0 thus .
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
39 of 126 11.04.2016 09:37 40 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Properties of the Fourier Transform Ph. Cattin: Signalprocessing Background Properties of the Fourier Transform
Convolution and Correlation (38) Convolution (1) (39)
In the next few slides we will introduce two Fourier Transform relationships The convolution of two 1-dimensional functions and is generally that connect the spatial and the frequency domain, namely denoted by and defined by the integral
Convolution Correlation (3.51)
Convolution and correlation are of fundamental importance in many image where is a dummy variable. processing techniques. The importance of convolution in the frequency domain analysis lies in the fact that and constitute a Fourier Transformation pair thus
(3.52)
a convolution in the Image domain results in a multiplication in the Frequency domain, and vice versa
(3.53)
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
41 of 126 11.04.2016 09:37 42 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Properties of the Fourier Transform Ph. Cattin: Signalprocessing Background Properties of the Fourier Transform
Convolution (2) (40) Convolution with an Impulse (41) Function
The special case of convoluting a function with an Impulse Function is of particular interest as will be shown later.
Definition:
The Impulse Function (Dirac delta function) is often referred to as the unit impulse function introduced by the physicist → Paul Dirac [http://en.wikipedia.org/wiki/Paul_Dirac]. The function Dirac Impulse may be viewed as having an area of unity in an infinitesimal small neighbourhood about and zero elsewhere; that is,
Sifting Theorem
and thus
It is common practice to graphically represent the Dirac impulses as arrows at with a height equal to the impulse strength (area).
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
43 of 126 11.04.2016 09:37 44 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Properties of the Fourier Transform Ph. Cattin: Signalprocessing Background Properties of the Fourier Transform
Convolution with an Impulse (42) Discrete Convolution (43)
Function (2) Suppose that, instead of being continuous, are discretised into sampled arrays of size and This important relationship will be used again in the sampling and quantisation section. (3.54)
(3.55)
With the discrete convolution can be defined as
(3.56)
Because is bigger than and they must be padded with zeros, so that both are of length .
Fig 3.16: Convolution with an impulse function
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
45 of 126 11.04.2016 09:37 46 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Properties of the Fourier Transform Ph. Cattin: Signalprocessing Background Properties of the Fourier Transform
Two-Dimensional Continuous (44) Two-Dimensional Discretised (45) Convolution Convolution
The 2D Convolution is analogous to the 1D, thus The discretised 2D Convolution is defined by
(3.57) (3.60)
The Convolution Theorem in two dimensions can then be expressed as where and are the discretised arrays of and , (3.58) respectively. Wraparound error in the individual convolutions is avoided by choosing and (3.61) (3.59) and
(3.62)
Calculating the discrete convolution in the frequency domain is often more efficient than directly using the equation above.
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
47 of 126 11.04.2016 09:37 48 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Properties of the Fourier Transform Ph. Cattin: Signalprocessing Background Properties of the Fourier Transform
2D Convolution Example (46) Correlation (47)
The correlation of two continuous functions and , denoted by , is defined by the relation
1D:
2D:
(a) Original image where is the complex conjugate.
(c) Padded original (e) Convolution result The discrete equivalent of the correlation is defined as
(b) Filter kern MATLAB snippet
IMG=fft2(img); 1D: KERN=fft2(kern); F=IMG.*KERN; f=ifft2(F); 2D:
(d) Padded kern Fig 3.17: 2D Convolution example (a) Original image of size , (b) filter kern of size , (c) padded original of size , (d) padded filter kern of size , (e) convolution result of size
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
49 of 126 11.04.2016 09:37 50 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Properties of the Fourier Transform Ph. Cattin: Signalprocessing Background Properties of the Fourier Transform
Correlation Theorem (48) Correlation Example (49)
For both the continuous and discrete cases, the following correlation theorem holds
(3.63)
and
(3.64)
The principal application for correlation in image processing are
template matching
However, one has to take into account, that correlation is
sensitive to lightning changes linear bias
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
51 of 126 11.04.2016 09:37 52 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Properties of the Fourier Transform The Fast Fourier 2D Correlation Example (50) Transform (FFT)
Computational Complexity (52)
The number of complex multiplications and additions required to implement the Discrete Fourier Transform
(a) Original image (3.65) (c) Padded original (e) Correlation result
MATLAB snippet (b) Template is proportional to IMG=fft2(img); TEMP=fft2(temp); as for each of the values of the expansion of requires complex F=IMG.*conj(TEMP); f=ifft2(F); multiplications of by as the terms can be precalculated and tabulated they are not counted in the complexity analysis
(d) Padded template Fig 3.18: 2D Correlation example (a) Original image of size , (b) template of size , (c) padded original of size , (d) padded template of size , (e) correlation result of size
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
53 of 126 11.04.2016 09:37 54 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background The Fast Fourier Transform (FFT) Ph. Cattin: Signalprocessing Background The Fast Fourier Transform (FFT)
Computational Complexity (2) (53) Derivation of the FFT Algorithm (54)
Proper decomposition can reduce the number of multiplications and addition The FFT algorithm developed in the next few slides is based on the successive proportional to . This decomposition is called the fast Fourier doubling method. We start with the general form of the DFT Transform (FFT) algorithm.
Example: (3.66) DFT FFT 32 1'024 160 6.40 Let's assume that an FFT of size 64 4'096 384 10.67 8'192 takes on one particular and rewrite it in the form 128 16'384 896 18.29 machine 1s. Using the DFT method 256 65'536 2'048 32.00 the same Fourier Transform would (3.67) 512 262'144 4'608 56.89 require 10min30s. 1024 1'048'576 10'240 102.40 2048 4'194'304 22'528 186.18 (3.68) 4096 16'777'216 49'152 341.33 8192 67'108'864 106'496 630.15 and is assumed to be of the form where is a positive integer.
The requirement that must be a power of 2 does not limit generality of the algorithm, as one can always achieve this requirement by zero-padding the data to the next power of 2.
As is a power of 2 we can express it as
(3.69)
where is also a positive integer. Substitution into Eq. 3.67 yields
(3.70)
From Eq. 3.68 we know that , so the previous equation can be expressed as
(3.71)
Defining Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
55 of 126 11.04.2016 09:37 56 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background The Fast Fourier Transform (FFT) (3.72) The Inverse FFT (55) and On the previous slide we developed a fast implementation for the Fourier (3.73) Transform, but what about the Inverse Fourier Transform? The reason is that any method implementing the forward transform can also be Eq. 3.71 reduces to used to compute the inverse. To show this let us repeat the equations for the DFT and inverse DFT
(3.74) (3.76) Also, since and the above equations get (3.77) (3.75)
Taking the complex conjugate of Eq. 3.77 and dividing both sides by yields Carefull analysis of Eq. 3.72-3.75 reveals some interesting properties of these expressions. (3.78) An -point Fourier Transform can be computed by evaluating two -point Fourier Transforms. Comparing the result with Eq. 3.76 shows that the right hand side of Eq. 3.78 The resulting values of and are substituted into Eq. 3.74 to is in the form of the forward Fourier Transform. Thus inputting into an obtain for . The other half then follows directly algorithm to compute the forward transform gives that can be easily from Eq. 3.75 without additional transform evaluations. converted to . If and are recursively split further we finally end up with a computational complexity for the Fast Fourier Transform (FFT) of
complex multiplications: complex additions:
The complexity of FFT is thus in the order of .
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
57 of 126 11.04.2016 09:37 58 of 126 11.04.2016 09:37 Other Separable Image Ph. Cattin: Signalprocessing Background Unitary Image Transforms Transforms Unitary Image Transforms (2) (59) For 2D square images the forward and inverse transform are defined as Unitary Image (3.81) Transforms
where are the forward and backward transformation Unitary Image Transforms (58) kernels.
The 1D discrete Fourier transform is one of a class of unitary (orthogonal) transforms that can be expressed in terms of the general relation
(3.79)
where is the transform of , the forward transform kernel, and . The inverse transform has a similar form
(3.80)
where is the inverse transformation kernel and . The properties of the transformation kernel determine the nature of the transform.
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
59 of 126 11.04.2016 09:37 60 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Unitary Image Transforms Ph. Cattin: Signalprocessing Background Unitary Image Transforms
Separability and Symmetry (60) Separability and Symmetry (2) (61)
The forward kernel is separable if If a transformation kernel is separable, the 2D transformation can be split in two 1D transformations → computationally more efficient. In a first step the 1D (3.82) transform is performed along each row of
The kernel is additionally symmetric if and thus (3.84) (3.83)
The identical definitions apply for the inverse kernel. for . Next, the 1D transformation is conducted along each column of Separable image transforms allow for computationally efficient implementations (3.85)
for .
The very same result is obtained if the transformation is performed first on each column and the on each row.
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
61 of 126 11.04.2016 09:37 62 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Unitary Image Transforms Ph. Cattin: Signalprocessing Background Unitary Image Transforms
Separability and Symmetry (62) Matrix Notation (63)
Example If a kernel is separable as well as symmetric Eq 3.81 can also be written in matrix form As has been previously shown, the forward transformation kernel of the 2D Fourier transform is (3.88)
(3.86) where is the image matrix, is an symmetric transformation matrix with elements , and is the resulting transform.
which is separable and symmetric, because To obtain the inverse transform Eq 3.88 must be pre- and post-multiplied by an inverse transformation matrix
(3.87) (3.89) If then
It is easy to show that the inverse Fourier kernel is also separable and (3.90) symmetric. Several transforms - including Fourier, Walsh - can be expressed in this form.
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
63 of 126 11.04.2016 09:37 64 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Unitary Image Transforms Walsh Transform Principle of the 2D Unitary (64) Transforms Walsh Transform (66)
When , the discrete 1D Walsh transform of , denoted by , is obtained with the forward kernel
(3.91)
where is the -th bit in the binary representation of , e.g. if and then , , and .
Eq. 3.79 can thus be written as
(3.92)
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
65 of 126 11.04.2016 09:37 66 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Walsh Transform Ph. Cattin: Signalprocessing Background Walsh Transform
1D Walsh Transformation Kernel (67) 2D Walsh Transform (68)
The Walsh transformation kernel is The 2D forward and inverse Walsh transform kernels are given by symmetric having orthogonal rows and columns. These properties lead to an inverse kernel identical to the forward (3.94) kernel except for a constant multiplicative factor . The inverse Walsh transform is and thus
(3.95) (3.93)
The corresponding 2D forward and inverse Walsh transforms can thus be The validity of Eq. 3.93 is easily validated Fig 3.19: 1D Walsh transformation kernel written as by substituting with Eq. 3.92 and making use of the orthogonality properties mentioned. (3.96)
and the inverse transformation
(3.97)
Although the grouping of the -terms is arbitrary the form shown above is preferred in image processing.
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
67 of 126 11.04.2016 09:37 68 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Walsh Transform Ph. Cattin: Signalprocessing Background Walsh Transform
2D Walsh Transformation Kernel (69) 2D Walsh Transformation Example (70)
As the Walsh kernels only depend on the indexes and not the image itself, it is fixed and serves as a kind of basis functions only determined by the dimension .
Figure 3.20 for example shows the kernel (basis functions) as a function of .
Because the Walsh transform is symmetric the inverse kernel is the same. Fig 3.21: Example for the forward and inverse 2D Walsh transform The Walsh transform can be computed efficiently by a slightly adapted Fast Fourier Fig 3.20: 2D Walsh transformation kernel for Transform (FFT) (white , black ). Each block corresponds to varying in particular .
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
69 of 126 11.04.2016 09:37 70 of 126 11.04.2016 09:37 Hadamard Transform Ph. Cattin: Signalprocessing Background Hadamard Transform 2D Hadamard Transformation (73) Hadamard Transform (72) Similarly to the Walsh kernels, the 2D Hadamard kernels are given by
The 1D forward Hadamard kernel is given by (3.102)
(3.98) and for the inverse Hadamard kernels
where the summation in the exponent is performed in modulo2 arithmetic and (3.103) definition of equal to the Walsh transform. The 1D forward Hadamard transform is then defined as where the summations in the exponent are again performed in modulo2 arithmetic. As was the case in the Walsh transform, the 2D Hadamard kernels are identical. (3.99) The 2D Hadamard transform pair can thus be defined as
where , and . (3.104) As the Hadamard transform has orthogonal rows and columns the inverse kernel is equal to the forward kernel except for the scaling term . and (3.100)
and finally the inverse Hadamard transform (3.105)
(3.101) for .
for .
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
71 of 126 11.04.2016 09:37 72 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Hadamard Transform Ph. Cattin: Signalprocessing Background Hadamard Transform
Generation of the 2D Hadamard (74) Drawback of the Hadamard Kernel (75) Kernel Ordering
The Hadamard definition leads to a simple recursive relationship for The generation of the Hadamard generating the transformation kernels needed to implement Eqs 3.88 & 3.90. kernels with the recursive rule, however, has a serious drawback The Hadamard matrix with the lowest order is with increasing the number of (3.106) transitions (thus frequency) does not increase as is the case in the Fourier transform The recursive relationship below can then be used to determine : For some applications like Image compression the ordering is, however, (3.107) not important.
The transformation Matrix and the inverse transformation Matrix for use in Eqs 3.88 and 3.90 respectively are then obtained by normalising with
(3.108) Fig 3.22: Standard 2D Hadamard for (white , black ). Each block corresponds to varying in The expression for the inverse Hadamard matrix is identical. particular .
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
73 of 126 11.04.2016 09:37 74 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Hadamard Transform Discrete Cosine Ordered 2D Hadamard (76) Transformation Kernel Transform
The Hadamard kernel where the sequency (number of transitions) Discrete Cosine Transform (78) increases as a function of (1D) and (2D) is given by the relation The Discrete Cosine Transform is yet another member of the unitary transform family and defined as
(3.109) (3.112)
where for . Similarly, the inverse DCT is defined as
(3.113)
(3.110) Fig 3.23: Ordered 2D Hadamard for , where the scaling factor is transformation kernel for (white , black ). Each block corresponds to varying in particular . (3.114)
And the 2D ordered Hadamard kernel The corresponding 2D Discrete Cosine Transform pair is given by
(3.111) (3.115)
Figure 3.23 for example shows the ordered Hadamard kernel (basis for . Similarly, the 2D inverse DCT is defined as functions) as a function of .
Similar to the Walsh transform, the Hadamard transform can be implemented (3.116) efficiently by slightly adapting the Fast Fourier Transform (FFT).
for , where the scaling factor is defined as for the 1D case.
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
75 of 126 11.04.2016 09:37 76 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Discrete Cosine Transform Ph. Cattin: Signalprocessing Background Discrete Cosine Transform
2D Discrete Cosine Transformation (79) 2D Discrete Cosine Kernel used in (80) Kernel JPEG Compression
The 2D Discrete Cosine Transformation kernel for is Figure 3.25 shows the DCT basis functions depicted in Fig 3.24. used in JPEG compression. JPEG is a lossy compression and works as follows: In contrast to the Walsh and Hadamard transform the DCT transform uses 1. The entire image is split in patches 2. Each patch is transformed into a multiplications with real values, luminance/chrominance colour space. This and allows for different compression factors evaluations of the cosine function. since luminance is more important than chrominance for human vision The DCT has become the method of 3. DCT transformation of each patch choice in image compression. 4. The DCT coefficients are processed so that unimportant coefficients will be replaced Fig 3.25: 2D DCT Kernel for by zeros and larger coefficients will lose precision Fig 3.24: 2D DCT Kernel for (white , 5. The resulting DCT coefficients are → black ). Each block corresponds to varying Huffman [http://en.wikipedia.org /wiki/Huffman_encoding] compressed (lossless compression)
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
77 of 126 11.04.2016 09:37 78 of 126 11.04.2016 09:37 Hotelling Transform Ph. Cattin: Signalprocessing Background Motivation Organ Shapes Vary (85)
Statistical Shape Models Organ shapes vary according to several parameters: Ethnicity, age, gender, lifestyle, physical activity, disease,... Motivation Our goal is to identify these differences using mathematical tools
Motivation (84) Fig. 3.28:
What is the motivation of using Statistical Shape Models (SSM)?
Natural anatomical variability Population-based studies are key to identify the dissimilarities We do not look the same, neither do our organs! Nice mathematical framework to include anatomical shape knowledge Fig. 3.27:
Fig. 3.26: Sagittal views of 20 different livers
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
79 of 126 11.04.2016 09:37 80 of 126 11.04.2016 09:37 Mathematical Ph. Cattin: Signalprocessing Background Mathematical Background Background Principal Component Analysis (88)
Assume that you have 2D data points In this example, data points are scattered Principal Component Analysis (87) along a principal direction PCA yields the main direction (and secondary (PCA) directions), and creates an orthogonal coordinate system that maximises the PCA is a method to reduce the dimensionality of a linear system variance of the points in each direction PCA projects the original data into a lower-dimensional space Points can be referenced along the main PCA embeds the data in a more compact representation components (axis) using a parameter How?
Let us first define PCA as a mathematical tool, and then see how we can apply it to SSM of anatomical structures.
Fig. 3.29:
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
81 of 126 11.04.2016 09:37 82 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Mathematical Background Ph. Cattin: Signalprocessing Background Mathematical Background
Principal Component Analysis (2) (89) Principal Component Analysis (3) (90)
Let hold the coordinates of (in this example ) data points One approach is to run eigen-analysis on the covariance matrix
(3.117)
The principal axes of the variation of the points are represented by eigenvectors and Eigenvalues of the Covariance Matrix of The covariance matrix is defined by
(3.118)
where the average sample is given by Spectral decomposition (or Jordan decomposition): (3.119) Given a symmetric covariance matrix , we can decompose it as the product of three matrices:
(3.120)
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
83 of 126 11.04.2016 09:37 84 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Mathematical Background Ph. Cattin: Signalprocessing Background Mathematical Background
The Curse of High Dimensionality...(91) The Curse of High Dimensionality...(92)
Let's assume, we have observations of dimensionality e.g. a mesh with (3.123) many points
→ this results in a huge covariance matrix: A smaller matrix can be gained instead if
(3.121) (3.124)
the covariance matrix is then (as defined before) (3.125)
after some simple algebra: (3.122) (3.126)
the resulting matrix is then of size (independent of the number of samples). Eigenvector/values of , but we only get eigenvector/values.
→ Very quickly computationally intractable!
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
85 of 126 11.04.2016 09:37 86 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Mathematical Background Ph. Cattin: Signalprocessing Background Mathematical Background
Other Possibility (93) PCA Assumptions (94)
Use the Singular Value Decomposition (SVD) on the original matrix PCA assumes that the original data follows a Gaussian distribution (even if it not really does) SVD diagonalises the covariance matrix of the original data PCA results in an orthogonal lower-dimensional basis i.e. the principal axes It directly yields the eigenvectors and eigenvalues of are perpendicular PCA is a linear method (3.127) There also exist several linear and non-linear dimensionality reduction techniques such as factor analysis (FA), Isomap, Multidimensional Scaling (MDS), Local Linear Embedding (LLE), etc.
Fig. 3.30:
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
87 of 126 11.04.2016 09:37 88 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Mathematical Background Shape Representation But... (95) Let us see how we can make use of this theory by applying it to model Shape Representation (97) anatomical shape variability Several preprocessing steps are required to make the data ready for Object contours can be represented and stored in different ways: statistical modelling Parametric surfaces (splines, etc.) Combination of implicit functions (quadrics, etc.) Oriented bounding box Surface or volumetric meshes (surface points and connecting lines) Point cloud (non-connected surface points) Level Set ...
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
89 of 126 11.04.2016 09:37 90 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Shape Representation Establishment of Shape Representation - Point Cloud(98) Correspondence
A set of point evenly distributed on the surface of the organ Each point is represented by its 2D or Point-to-Point Correspondence (100) 3D coordinate vector
Now assume we have three different Every surface point: objects of the same The surface is represented by class (i.e. three concatenating the points femurs from three Fig. 3.31: different patients) is the number of points that Initially, constitute the surface. isosurfacing results in point clouds with a different number of points We should establish anatomical correspondence between the points
Fig. 3.32:
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
91 of 126 11.04.2016 09:37 92 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Establishment of Correspondence Shape Alignment Point-to-Point Correspondence (101) Among the different methods, one can list Alignment and Correspondence (103) Mesh-based registration (e.g. iterative closest point) Image-based registration (e.g. non-rigid registration) ... How to achieve alignment? How to establish correspondence? The idea is to use a reference mesh and morph it to match the other meshes, thus keeping a consistent number of points → Rigid registration Manually (landmarks selection) The topic of image registration is outside of the scope of this lecture Automatically (non-rigid registration)
Fig. 3.33: Fig. 3.34:
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
93 of 126 11.04.2016 09:37 94 of 126 11.04.2016 09:37 Point Distribution Ph. Cattin: Signalprocessing Background Point Distribution Models Models Shape Space (2) (106)
We can add other shapes to our space in order to describe Shape Space (105) their variability (our initial aim)
The bone is described as a coordinate vector This vector can be seen as a point in the space But the space is not 2D, not 3D, not 4D, but... nD!!! So if the shape is described by points, the dimension of the space is
Fig. 3.36:
Fig. 3.35: Bone consists of , so the space dimension is
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
95 of 126 11.04.2016 09:37 96 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Point Distribution Models Modelling of Shape Point Distribution Models (PDM) (107) Variability So far, we have:
A set of aligned shapes Each shape has the same number of surface points Modelling of Shape Variability (109) Surface points correspond anatomically and/or spatially Each shape is represented by a vector of coordinates We now come back to the matrix of shapes of dimension (usually huge in practice) The following figure is an example of a set of landmarks taken on different resistors (aligned and normalised) We can represent the set of shapes in a matrix by concatenating the shape Each group of landmarks (in 2D) can be thought of as the scattered points vectors example shown earlier We should find the principal directions of variation of each landmark
(3.128)
Fig. 3.37:
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
97 of 126 11.04.2016 09:37 98 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Modelling of Shape Variability Ph. Cattin: Signalprocessing Background Modelling of Shape Variability
Modelling of Shape Variability (110) Modelling of Shape Variability (111)
Subtract mean from each instance of → The eigenvectors are a set of orthogonal axis, each corresponding to one of Apply SVD directly of matrix the main directions of variation As shown earlier, each direction has a different scale The scale is given by the eigenvalues The eigenvalues represent the variance of the points in each direction (over each principal component) The eigenvalues are usually stored in decreasing magnitude (the corresponding eigenvectors are sorted accordingly)
eigenvectors of , Fig. 3.38: eigenvalues of
= number of points per mesh
= number of training samples
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
99 of 126 11.04.2016 09:37 100 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Modelling of Shape Variability Model Evaluation What can we do with shapes in a (112) reduced space? Model Evaluation (114)
The formula that we can use to create new Commonly, three measures are used to evaluate a statistical shape model instances is: Compactness (3.129) Generalisation Specificity where They are used to indicate how well the modeling process has been able to is the new shape embed the original data into a lower dimensional space is the average shape is the chosen eigenvector is a parameter whose value is generally in the range Fig. 3.39: with the corresponding eigenvalue
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
101 of 126 11.04.2016 09:37 102 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Model Evaluation Ph. Cattin: Signalprocessing Background Model Evaluation
Compactness (115) Generalisation (116)
Generalisation is performed using leave-one-out (cross- validation) Compactness is a measure of reconstruction experiments how good the data reduction It represents the ability of the model to generate new instances from the process is class In other words, a model that is For every left-out instance, a new statistical model is computed from the compact enough, allows the remaining training instances generation of new shape The parameters of the left-out instance are computed instances using as few parameters (principal modes or components) as possible It is represented by a curve (3.131) showing how much of the total variance a certain number of Fig. 3.40: Cumulative variance modes of variation can capture
Fig. 3.41:
(3.130) number of modes used -th eigenvalue total variance of all eigenvalues
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
103 of 126 11.04.2016 09:37 104 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Model Evaluation Ph. Cattin: Signalprocessing Background Model Evaluation
GenerGeneralisationaliation (2) (117) Generalisation (3) (118)
The parameters are used to reconstruct the instance
(3.132)
The errors are measured and averaged over all training instances yielding the generalisation curve of the model generalisation
(3.133)
It is also dependent on the number of modes of variation (parameters used in the reconstruction process)
Fig. 3.42:
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
105 of 126 11.04.2016 09:37 106 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Model Evaluation Applications Specificity (119)
Specificity is an indicator of how good the model is in generating instances similar to those presented in the training set Segmentation It is measured by generating a large number of random instances using different number of modes, and for every new instance, compute the distance to the closest shape in the training set Segmentation (122)
Image segmentation is the prime application for statistical shape models:
Mathematical framework for anatomical knowledge Robust against image noise Can cover Fig. 3.44: 3D Rendering of the Fig. 3.45: CT slice of a skull with Fig. 3.43: missing parts masseter muscle and the skull high density artifacts due to dental bone implants
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
107 of 126 11.04.2016 09:37 108 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Segmentation Ph. Cattin: Signalprocessing Background Segmentation
Statistical Masseter Model (123) Fitting Process (124)
20 CT data set Manually segmented masseter muscle Correspondence established through demons registration ...
Fig. 3.46: Principal component 1 ( )
Fig. 3.47: Principal component 2 ( )
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
109 of 126 11.04.2016 09:37 110 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Segmentation Implant Design Resulting Segmentations (125) Computer-Assisted Orthopedic (127) Implant Design
What makes a good implant design?
Implant fitting Implant stability Bone healing impact Intervene as less as possible Torsions are bad Screws deformations are bad
1. Iteration 2. Iteration 3. Iteration 4. Iteration
Fig. 3.48:
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
111 of 126 11.04.2016 09:37 112 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Implant Design Ph. Cattin: Signalprocessing Background Implant Design
Computer-Assisted Orthopedic (128) Computer-Assisted Orthopedic (129) Implant Design Implant Design
But, how many different implant shapes are required to cover most of But, how many different implant shapes are required to cover most of the population with as few variants as possible? the population with as few variants as possible?
Fig. 3.49: Fig. 3.50:
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
113 of 126 11.04.2016 09:37 114 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Implant Design Modelling Organ Motion Proposed Methodology (130) A. Create a statistical shape model Problem Statement (132) This creates a new space to describe the shape variability in the selected population. The different axes describe specific patterns of shape variability → we can easily create many reasonable shapes of the bone Respiratory organ motion is a complicating factor in the treatment of tumours. B. Develop criteria to assess the fit of an implant Goals of motion management: Possible criteria are: average distance of implant to bone, max distance to bone, how many bendings are required,... Increase dosage to tumour Reduce dosage to healthy tissue C. Partition the shape space
Partition the shape space into regions that can be covered by one implant. We want as few partitions as possible.
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
115 of 126 11.04.2016 09:37 116 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Modelling Organ Motion Ph. Cattin: Signalprocessing Background Modelling Organ Motion
State-of-the-Art (133) Foundation: 4D MRI (134)
Breath-hold Gating Calypso Voxelsize , Frequency . [Siebenthal2005]
Maximum Intensity Projection of the right liver 4D Lung reconstruction, 24 sagittal slices, 11 lobe stacks
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
117 of 126 11.04.2016 09:37 118 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Modelling Organ Motion Ph. Cattin: Signalprocessing Background Modelling Organ Motion
Prediction over long Time Scales (135) Population-based Statistical Drift (136) Model
Coronal view
(black) average prediction error, (gray) maximum error
PC1 PC2
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
119 of 126 11.04.2016 09:37 120 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Modelling Organ Motion
4D Statistical Motion Model (137)
Visualisation of the 4D Statistical Motion Model over .
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
121 of 126 11.04.2016 09:37 122 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Modelling Organ Motion Ph. Cattin: Signalprocessing Background Modelling Organ Motion
Application Scenario: 3D (138) Prediction based on Partial 3D (139) Information Information
Given Output
A static 3D exhalation CT/MR Prediction of tumour position image Surrogate marker: 3D fiducial positions 3D fiducial positions (CyberKnife) Calypso magnetic beads
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
123 of 126 11.04.2016 09:37 124 of 126 11.04.2016 09:37 Ph. Cattin: Signalprocessing Background Modelling Organ Motion Ph. Cattin: Signalprocessing Background Modelling Organ Motion
Results for Known 3D Points (140) Results when Predicting with the (141)
Cross-validation experiment with 12 subjects over . 4D Motion Model
(media, 25th & 75th-percentile, maximum) Prediction error over . (media, 25th & 75th-percentile)
Subject 1 with running Subject 5 with running average error average error
Introduction to Signal and Image Processing March 8th/15th, 2016 Introduction to Signal and Image Processing March 8th/15th, 2016
125 of 126 11.04.2016 09:37 126 of 126 11.04.2016 09:37