Efficient S-Transform Techniques for Magnetic Resonance Imaging

UNIVERSITY OF CALGARY

Eﬃcient S-Transform Techniques for Magnetic Resonance Imaging

Sylvia Drabycz

A THESIS

SUBMITTED TO THE FACULTY OF GRADUATE STUDIES

IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE

DEGREE OF DOCTOR OF PHILOSOPHY

GRADUATE PROGRAM IN BIOMEDICAL ENGINEERING

CALGARY, ALBERTA

MARCH, 2009

ISBN: 978-0-494-51180-0

Abstract

The S-transform (ST) is a promising technique for magnetic resonance (MR) image texture analysis that has previously been successfully applied to the study of multiple sclerosis (MS) and brain cancer. Two of the major limitations of the ST are the computation time and the storage requirements. This thesis introduces two new techniques that reduce the complexity and redundancy of the ST, addressing these limitations.

A version of the ST that uses circularly symmetric windows, known as the circular S-transform (cST), is proposed and its properties are studied. It is shown that the cST produces results similar to the polar S-transform (pST) but requires less computation time.

Space-frequency transforms such as the ST can also be utilized for image ﬁltering and denoising. A novel application of the ST is proposed to study lesion dynamics in MS. It is found that a texture measure derived from the cST can discriminate between the core of a hyperintense active lesion and its less hyperintense periphery.

A two-dimensional (2D) frequency-domain implementation of the discrete orthonormal S-transform (DOST) is also presented. The DOST addresses both the computational and storage requirements of the ST by removing redundancy from the ST and providing a rapid calculation of the space-frequency domain. Novel texture features are derived from the DOST and it is shown that these features are more powerful when classifying unknown texture patterns than a leading method using the discrete wavelet transform (DWT).

Finally, the cST, pST and the DOST are applied to a study of glioblastoma iii (GBM) texture. It is shown that the DOST is the most accurate texture analysis approach when blindly classifying tumors based on their methylation status. Inter- estingly, diﬀerences were only found when analyzing small region of interests (ROIs) within the tumors, not when analyzing the entire image and extracting spectra from the entire tumor.

iv Acknowledgements

I would like to begin by thanking my parents for their never-ending support and encouragement. It is because of the sacriﬁces they made in their own lives that I have had the opportunity to pursue higher education. I learned from them that hard work and persistence are the keys to success. Thank you both for everything you have provided for me and for always supporting my love of learning — you are an inspiration. I also want to take this opportunity to thank my best friend and husband-to-be for all of his patience, love, kindness and understanding. Thank you to all of my friends and family for being incredibly supportive throughout this journey. I could not have reached this point without all of you.

I oﬀer my thanks to all of the past and present students and researchers at the Imaging Informatics Lab and the Seaman Family MR Research Centre. I have learned so much from all of you about MR imaging, programming and fundamental research concepts. I have enjoyed working with and getting to know all of you. I would especially like to thank Danny Ramotowski, Mark Simpson and Brian O’Brien for their technical assistance and help in accessing data as well as Pamela Wilkinson and Joanne Morgan for their help with travel claims, paperwork and always making sure I got paid.

I would like to thank my supervisory committee, including Brad Goodyear and

Gary Margrave, as well as Michael Lamoureux for their explanations of concepts from MR image acquisition to time-frequency analysis, and for providing direction and feedback about my research. I would like to acknowledge the help of Robert

Stockwell for answering my questions about his DOST work and for his contributions v to my own extensions of his work. I would like to extend my sincere thanks to

Gregory Cairncross, Gloria Rold´an, Paula de Robles and Daniel Adler for all of their hard work with the GBM project. They have been invaluable in the design and execution of this project. Without their hard work, support and encouragement, this part of my research project would not have been possible.

I give my thanks to all of my past supervisors and teachers. Thank you to my professors at the University of Regina for inspiring my love of physics, mathematics and computer science. Thank you to my past academic and employment supervisors, especially Alex MacKay at UBC for giving me my ﬁrst opportunity to pursue independent research and for introducing me to MR.

I wish to recognize the support I have received from the following funding agen- cies: the Natural Sciences and Engineering Research Council of Canada for the

Alexander Graham Bell Canada Graduate Scholarship as well as the NSERC Post- graduate Scholarship; L’Or´eal Canada and the Canadian Commission for UNESCO for the Women in Science Mentor Fellowship; the Alberta Government Informatics

Circle of Research Excellence for the Graduate Student Scholarship; and the Al- berta Heritage Foundation for Medical Research for the Studentship Award. I would also like to acknowledge the ﬁnancial support I have received from the University of Calgary through the Dean’s Research Excellence Award, the Dean’s Entrance

Scholarship and the Graduate Travel Award as well as the ﬁnancial support from my supervisor, Ross Mitchell.

I would like to thank the following organizations that have allowed me to share my work with young people and help inspire the next generation of scientists and engineering: the Actua mentoring program for allowing me to travel to Saskatoon

vi and Regina to work with the SCI-FI and EYES science camps; the Women in Science and Engineering group at the University of Calgary for inviting me to participate with groups such as the Calgary Inner City Resource Centre’s “Saturday All Stars” program; the SCIberMENTOR email mentoring program; and the Telus World of

Science for inviting me to speak at the 2008 Beauty and Brains Conference.

Finally, I oﬀer my sincere thanks and gratitude to my supervisor, Ross Mitchell.

Thank you for all of the hard work you have put in over the last four and a half years.

Your enthusiasm and dedication to medical imaging informatics is what inspired me to come to Calgary and pursue my research project. Thank you for having faith in me to pursue my own research direction while providing guidance, support and invaluable suggestions for all of the research presented in this thesis. All of the work I have published throughout my research program has beneﬁted from your revisions and corrections. The hard work and long hours you have put in to secure funding through grants and industrial partnerships have given me the opportunity to travel internationally to present my work and network with and learn from leading scientists around the world.

While this is by no means a complete list, I would like to thank all of those people who have helped and supporting me in reaching this goal. Individual contributions to each publication are outlined in detail at the beginning of each chapter.

vii Table of Contents

Abstract ...... iii Acknowledgements ...... v Table of Contents ...... viii List of Tables ...... xii List of Figures ...... xiii List of Abbreviations ...... xxiv

CHAPTER 1: INTRODUCTION ...... 1 1.1 Thesis Objectives...... 1 1.2 Thesis Structure ...... 2 1.3 Signiﬁcance and Contribution ...... 4

CHAPTER 2: INTRODUCTION TO SPACE-FREQUENCY ANALYSIS . 6 2.1 Fourier Transform ...... 6 2.2 Windowed Fourier Transforms ...... 9 2.2.1 Gabor Transform and Gabor Filters ...... 12 2.3 Wavelets ...... 18 2.3.1 Continuous 1D Wavelet Transform ...... 18 2.3.2 Discrete 1D Wavelet Transform ...... 19 2.3.3 Discrete 2D Wavelet Transform ...... 22 2.4 Curvelets...... 23 2.5 S-Transform ...... 26 2.5.1 1D S-Transform ...... 26 2.5.2 2D S-Transform ...... 29 2.5.3 S-Transform Inversion ...... 30 2.5.4 The Polar S-Transform ...... 31 2.5.5 Discrete Orthonormal S-Transform...... 33

CHAPTER 3: INTRODUCTION TO MAGNETIC RESONANCE IMAG- ING...... 37 3.1 MR Physics ...... 37 3.2 Generating an Image ...... 43 3.2.1 Image Reconstruction ...... 44 3.2.2 Neurological MR Imaging...... 45 3.3 MR Sequences and Contrast Mechanisms ...... 47 3.3.1 T2 Weighting ...... 47 3.3.2 T1 Weighting ...... 47

viii 3.3.3 Proton Density Weighting ...... 50 3.3.4 Contrast Enhancement ...... 50

CHAPTER 4: APPLICATIONS OF SPACE-FREQUENCY ANALYSIS TO MEDICAL IMAGE PROCESSING ...... 53 4.1 Filtering ...... 53 4.1.1 Image-Space Filtering ...... 53 4.1.2 Frequency-Domain Filtering ...... 56 4.1.3 Space-Frequency Filtering ...... 58 4.2 Texture Analysis ...... 63 4.2.1 Statistical Methods ...... 63 4.2.2 Multiscale Texture Features ...... 65 4.3 Classiﬁcation ...... 69 4.3.1 Evaluating Classiﬁer Performance ...... 71

CHAPTER 5: THE CIRCULAR S-TRANSFORM ...... 76 5.1 Introduction...... 76 5.1.1 Limitations of the pST ...... 77 5.1.2 Objectives ...... 78 5.1.3 Theory ...... 78 5.1.4 The Circular S-Transform ...... 81 5.2 Methods ...... 86 5.2.1 Preliminary Analysis on an MS Image ...... 86 5.2.2 Further Analysis ...... 86 5.3 Results ...... 88 5.3.1 Preliminary Results ...... 88 5.3.2 Further Results ...... 88 5.4 Conclusions ...... 92

CHAPTER 6: PROPERTIES AND PERFORMANCE OF THE CST . . . . . 95 6.1 Introduction...... 95 6.1.1 Phase Shift...... 96 6.1.2 Localizing Windows ...... 96 6.1.3 Smoothing ...... 98 6.1.4 Geometric Transformations ...... 99 6.1.5 Image Non-Uniformity...... 99 6.1.6 Inversion ...... 100 6.2 Methods ...... 100 6.2.1 Phase Shift...... 101 6.2.2 Localizing Windows ...... 102

ix 6.2.3 Visual Analysis of Local Spectra ...... 102 6.2.4 Frequency Response ...... 102 6.2.5 Smoothing ...... 103 6.2.6 Effect of Geometric Transformations ...... 103 6.2.7 Effect of Image Non-Uniformity ...... 105 6.2.8 Inversion ...... 106 6.3 Results ...... 108 6.3.1 Phase Shift...... 108 6.3.2 Localizing Windows ...... 109 6.3.3 Visual Analysis of Local Spectra ...... 112 6.3.4 Frequency Response ...... 118 6.3.5 Smoothing ...... 119 6.3.6 Effect of Geometric Transformations ...... 123 6.3.7 Effect of Image Non-Uniformity ...... 126 6.3.8 Inversion ...... 126 6.4 Discussion ...... 129 6.5 Conclusions ...... 133

CHAPTER 7: LESION EVOLUTION IN MULTIPLE SCLEROSIS US- ING THE CIRCULAR S-TRANSFORM ...... 134 7.1 Introduction...... 135 7.1.1 S-Transform Texture Analysis ...... 135 7.1.2 Clinical Application: Multiple Sclerosis ...... 136 7.2 Materials and Methods ...... 139 7.3 Results ...... 144 7.4 Discussion ...... 150 7.5 Conclusion ...... 154

CHAPTER 8: TEXTURE ANALYSIS USING THE 2D DISCRETE OR- THONORMAL S-TRANSFORM ...... 156 8.1 Background ...... 157 8.2 Theory ...... 160 8.2.1 1D Discrete Orthonormal S-Transform ...... 160 8.2.2 2D Discrete Orthonormal S-Transform ...... 166 8.2.3 Pixel-wise Local Spatial Frequency Description ...... 170 8.2.4 Primary Frequency Component ...... 172 8.2.5 Eﬀect of Spatial Transformations on the DOST ...... 173 8.2.6 Rotationally Invariant DOST Features ...... 174 8.3 Methods ...... 176 8.3.1 Transform Characterization...... 178

x 8.3.2 Response to Noise ...... 179 8.3.3 Classiﬁcation Experiment...... 180 8.4 Results ...... 184 8.5 Discussion and Conclusions ...... 188

CHAPTER 9: TEXTURE ANALYSIS OF GLIOBLASTOMA USING THE PST, CST AND DOST ...... 189 9.1 Introduction...... 190 9.2 Methods ...... 192 9.2.1 Patients ...... 192 9.2.2 DNA Samples ...... 192 9.2.3 Texture Analysis ...... 193 9.3 Results ...... 196 9.3.1 Qualitative Texture Analysis ...... 198 9.3.2 Quantitative Texture Analysis ...... 200 9.3.3 Texture-Based MGMT Prediction...... 205 9.4 Discussion ...... 207

CHAPTER 10: CONCLUSIONS ...... 209 10.1 Summary ...... 209 10.2 Future Work ...... 211 10.2.1 cST Image Filtering ...... 212 10.2.2 Application of the DOST to Texture Analysis of Multiple Sclerosis ...... 217 10.2.3 Correlation of Texture Features with Survival in GBM . . . . . 221

References ...... 227

APPENDIX A: REPRINT PERMISSION ...... 248

xi List of Tables

2.1 Low-pass (g) and high-pass (h) ﬁlters applied to the rows and columns of an image to generate horizontal (H), vertical (V), and diagonal (D) detail and approximation (A) coeﬃcients...... 23

5.1 The minimum ROI sizes required to reach stable LFE values...... 91 5.2 P-values for the diﬀerence between pST and cST LFE measures for three bandwidths (columns 2–4), along with the correlation coeﬃ- cients (column 5). The pST and cST give similar LFE values, particularly for large ROIs and bandwidths that exclude very low frequencies. The spectra are highly correlated...... 92

6.1 L1 norms for inverted images (average over 5 texture images)...... 131 7.1 New enhancing lesions that occurred over the course of 9 months of serial MR scanning. Note that month 0 corresponds to the start of treatment. Lesions were identiﬁed on T1 post-contrast images...... 140

8.1 Misclassiﬁcations (/100) of the Primary Frequency Component (Ori- ented Diagonally) ...... 187 8.2 Classiﬁcation Accuracy for Invariant Wavelet (db4) and Invariant DOST on 9 Brodatz Texture Images ...... 188

9.1 Differentiation between MGMT promoter methylation status using qualitative texture features assessed visually on T1-post contrast and T2-weighted MR images...... 198 9.2 Classification results using linear discriminant analysis for qualitative and quantitative texture features...... 205 9.3 Classification results using quadratic discriminant analysis for qualitative and quantitative texture features...... 205

10.1 Characteristics of best/worst survival groups...... 222 10.2 Texture features found to be significantly different between patients who survived for the longest amount of time after diagnosis (best, n = 15) and those who survived the shortest (worst, n = 15). These features were used in the Cox regression analysis...... 223 10.3 Variable bin cutoffs...... 223

xii List of Figures

2.1 Tilings of the time-frequency plane for (a) optimal time localization and no frequency localization (Dirac basis), (b) optimal frequency localization and no time localization (Fourier basis), (c) uniform partitioning obtained with a constant width window and (d) dyadic partitioning. Figures adapted from [1,2]...... 10 2.2 A signal consisting of a low-frequency sine wave (0.068 Hz), a quadratic chirp increasing in frequency from 0.039 to 0.39 Hz and a pulse from 154–166 s (bottom) and its FT (left). The STFT with a wide temporal window resolves the low frequency components well, but misses the pulse entirely...... 13 2.3 As the STFT window width decreases, the pulse becomes visible, but the time resolution does not allow for precise identification of when the pulse starts and ends. The frequency resolution of the remaining components is not as good as in Fig. 2.2...... 14 2.4 With a narrow temporal window, the STFT resolves the location of the pulse well, at the expense of the degrading frequency resolution. . 15 2.5 Sampling of the time-frequency plane for the GT. a and b determine the distance of the samples in the time and frequency directions, respectively...... 16 2.6 Dyadic tiling of the time-scale domain of the DWT. The width of each partition decreases by a factor of 2 as the scale decreases and the scale bandwidth doubles by a factor of two...... 20 2.7 DWT decomposition of a signal x[n] with low pass filters g[n] and high pass filters h[h]. The symbol ↓ 2 represents downsampling by a factor of two by discarding every second sample...... 21 2.8 Three levels of a 2D DWT (levels are denoted by the subscripts). Av- erage coefficients are denoted by “A” and detail coefficients represent: H: horizontal, D:diagonal, V:vertical...... 24 2.9 (a) The frequency domain tiling of the curvelet transform into polar “wedges”. (b) The Cartesian tiling based on concentric squares often used in practice when calculating the curvelet transform...... 25 2.10 The ST of the signal from Fig. 2.2 provides good resolution of both time and frequency, allowing one to identify all of the spectral components...... 28 2.11 Real (solid) and imaginary (dashed) portions of the DOST basis functions, defined in (2.52), for orders p = 0 to 5 and N = 100. This figure was intended to replicate Fig. 2 from [3]...... 35

xiii 2.12 The DOST of the signal from Fig. 2.2. The signal is plotted on the bottom and its FT on the left side. Note that the DOST coefficients are replicated to fill the N × N domain; in actuality the DOST contains only N coefficients for a N-point signal...... 36 3.1 The axis of rotation of a nuclear spin, μ, precesses about B0 at an angular frequency ω ∝ B0, known as the Larmor frequency...... 38 3.2 A static B0 field induces a net magnetization M along the z-axis. An applied B1 field along the x-axis rotates M into the transverse plane. The strength and duration of B1 determine the angle of M with respect to the z-axis, α, known as the flip angle...... 39 3.3 (a) Simulated regrowth of longitudinal magnetization over t =0–2500 ms with T1 = 500 ms. (b) Simulated decay of transverse magnetization over t =0–2500 ms with T2 = 500 ms...... 41 3.4 A diagram of acquisition space for MR imaging, known as k-space. max max The maximum frequency in k-space is kx = NxΔx/2 and ky = NyΔy/2. The center frequency (kx,ky)=(0, 0) is known as the DC component. The spacing between samples in k-space is given by Δkx =1/F OVx and Δky =1/F OVy. In a traditional acquisition scheme, one line of k-space is acquired each TR...... 46 3.5 Illustration of the three most common MR imaging planes: axial, sagittal and coronal...... 48 3.6 An example of a normal PD-weighted axial MR image. R and L refer to the right and left sides of the patient, respectively and A and P to anterior (front) and posterior (back), respectively...... 49 3.7 MR images of a MS patient obtained using a spin-echo sequence with the following weightings (TR/TE): (a) T2 (2716/80 ms), (b) T1 (650/8 ms) and (c) PD (2716/30 ms). Note the good contrast between lesions and surrounding white matter in (a). Examples of MS lesions are indicated by white arrows...... 51 3.8 Examples of T1-weighted images taken (a) before and (b) after the injection of gadolinium contrast agent (Gd-DTPA). Note the improvement in the contrast between the large brain tumor (arrow) and the normal brain tissue in (b) along with the detail evident within the tumor itself...... 52

4.1 (a) A 256 × 256 pixel MR image. (b) The same image after blurring with a Gaussian g of width σ =1.28 pixels and (c) after applying g with σ =2.56 pixels...... 55

xiv 4.2 Anisotropic diffusion filtering of the image shown in Fig. 4.1(a) with (a) 10 iterations at κ = 28, (b) 50 iterations at κ = 28 and (c) 10 iterations at κ =10, 000...... 57 4.3 Examples of low-pass, high-pass and band-pass filters...... 59 4.4 (a) A signal (left column) that contains a sinuosoid whose frequency increases with time along with its FT (right column). The results after applying the (b) low-pass, (c) high-pass and (d) band-pass filters from Fig. 4.3. The filters are shown as dotted lines in the right column (multiplied by 30 for display)...... 60 4.5 (a) An example of a signal that contains a low-frequency component for the first half of the signal, a mid-range frequency for the second half and a high-frequency burst that occurs from 0.12 to 0.27 s (bottom) and its ST (top). (b) The result after applying a multiplicative box filter to the ST (with value zero in the area indicated by the white box and one elsewhere) and inverting...... 62 4.6 Example of a 4 × 4 image with four grey-level values ranging from 0 to 3 (left). The structure of the grey-level spatial dependence matrix (right). Figure adapted from [4]...... 64 4.7 (a) Three levels (n = 3) of a wavelet decomposition of an image. Average coefficients are denoted by “A” and detail coefficients: H: horizontal, V: vertical, D: diagonal. (b) For rotationally-invariant coefficients, V & H coefficients are averaged to obtain Wn for each level and the average coefficients, while D coefficients (shaded grey in (a)) are omitted. (c) The average of each Wn is calculated to obtain the rotationally-invariant coefficients...... 68 4.8 The 2D feature space of two features denoted f1 and f2. (a) An example of a linear discriminant function that separates the two classes and (b) a quadratic discriminant function. (c) In k-NN classification an unknown sample (denoted by the circle with question mark) is assigned to the majority class of its k nearest neighbors. If k = 2 the object is assigned to the class represented by the triangles; if k =5 it is assigned to the class represented by squares...... 70 4.9 Example of a contingency table showing how sensitivity, specificity, PPV and NPV are calculated...... 73 4.10 A ROC plot showing the performance of various classifiers. The best possible classifier would be in the upper left corner. The dashed line indicates performance of a random guess (the line of no discrimination). The predictions of classifiers worse than a random guess can be inverted to improve their performance (dashed black arrow)...... 75

xv 5.1 A flow diagram illustrating the process of calculating the pST on an ROI (shown as a box on the image)...... 80 5.2 A flow diagram illustrating the process of calculating the cST...... 84 5.3 The 3D cST (x, y, k) can be viewed in two ways: as “frequency maps” that show the power of a particular k at all spatial locations, cST (∗, ∗,k) or by taking projections at a particular (x, y) location, cST (x, y, ∗). . . 85 5.4 Average local spectra for a 5 × 5 region in (a) a MS lesion and (b) NAWM. The shaded area indicates the region where LFE is calculated (0.73 to 3.2 cm−1). By allowing the entire image to be transformed, the cST is able to improve the resolution of the spectra by a factor of 8 over the 64 × 64 pST analysis and by a factor of 16 over the 32 × 32 analysis. The improved resolution appears to resolve and emphasize peaks not obvious in an ROI analysis...... 89 5.5 The average LFE values from the pST and cST, averaged over a 5x5 region in an MS lesion and NAWM. The numbers in brackets indicate the size of the ROI analyzed...... 90 5.6 The spectrum of a point in an MR image computed using the cST at the full image FOV as well as with the pST at varying FOV sizes (128 × 128, 64 × 64 and 32 × 32)...... 90 5.7 Effect of ROI size on LFE using the pST for one patient. The ∗ and † indicate values that are not significantly different (p>0.05). Error bars are standard deviation within a 5×5 pixel region...... 91 5.8 Examples of spectra obtained from the pST (gray) and cST (black) for FOVs: (a) 32 × 32, (b) 64 × 64 and (c) 128 × 128...... 93 5.9 The pST takes longer to calculate (time ∝ N 3.8) than the cST (time ∝ N 2.6−2.7) for various N × N image sizes...... 94

6.1 (a) The maximum frequency measured when applying circularly symmetric windows (grey) up to the Nyquist frequency (nmax = N/2) is kmax =1/(2Δx) or 0.50 if Δx = 1. The resulting spectra will contain N/2 points. While the corner frequencies are not directly sampled, their contribution is included in lower frequencies due to window overlap. (b)√ The maximum frequency√ of a localizing window (grey) if nmax = N/ 2iskmax =1/( 2Δx) or 0.71 if Δx =1...... 98 6.2 (a) A T2-weighted MR image with a FOV = 22 cm and matrix size = 256 × 256. The same image is shown cropped to FOV= 11 cm, matrix size = 128 × 128 (b) and FOV= 5.5 cm, matrix size = 64 × 64.104

xvi 6.3 (a) An example of a Brodatz texture image used to perform the non- uniformity experiment. The effect of (b) low, (c) medium and (d) high levels of non-uniformity are modeled by multiplying the image in (a) by a Gaussian of standard deviation 3N/2,N and N/2 pixels, respectively...... 107 6.4 (a) A 1D signal consisting of a low-frequency component (ξ=0.04) for the first half, a mid-range frequency (ξ=0.16) for the second half and a high-frequency burst (ξ=0.38) at t=20. (b) The magnitude of the ST, S(τ,ξ). (c) and (d) show the real portion and phase maps of S(τ,ξ), respectively. (e) and (f) show the real and phase portions, respectively, of S(τ,ξ)=S(τ,ξ)p(τ,ξ) where p(τ,ξ)=e2πiτξ is a phase rotation that rotates the “ripples” from the vertical to the horizontal orientation...... 110 6.5 (a) The magnitude of the ST (shown in Fig. 6.4) after averaging the frequency components for each point to a dyadic sampling grid, without applying the phase rotation. Destructive phase interference causes the relevant information to disappear; only the edge effect is present in the frequency-averaged domain. (b) The magnitude of the frequency-averaged domain after applying the phase ramp; the relevant information is retained...... 111 6.6 (a) The circularly symmetric Gaussian window provides a more even spectral weighting than (b) the corresponding pST window, integrated over all values of θ, (128×128), F =20. (c) The circularly symmetric window in the spatial domain is a sinc-like function...... 113 6.7 (a) A profile of the cST window and Gabor filter in the spectral domain for F =20,N = 128. Note that the cST window tapers to zero more quickly than the Gabor windows. These windows have been scaled to the same maximum amplitude for comparison. (b) The cST window is slightly broader than the Gabor filter in the spatial domain, meaning that the ringing will be spread over a slightly larger area. The effect of the ringing can be reduced by smoothing the amplitude frequency maps...... 114 6.8 (a) Test pattern (256×256) with four frequencies of wavenumber (horizontal, vertical): (0,4), (8,10), (60,0) and (20,20), starting in the upper left, clockwise. (b) The resulting cST spectra (power, magnitude squared) from the centre of each region (UL: upper left, UR: upper right, LL: lower left, LR: lower right)...... 115

xvii 6.9 The squared magnitude (power) of frequency maps before (top) and after smoothing (middle) with a Gaussian kernel (σ =1/k) along with the corresponding pST power maps (bottom) for F = 4, 13, 28 and60...... 116 6.10 (a) A synthetic image consisting of two sinusoids added together: (kx,ky) = (0.16, 0) and (0, 0.31) where N=128. The spectra from the centre of the image are calculated using the cST, smoothed with σ =1.5/k (solid black); the invariant Gabor ﬁlter evaluated at F = √ √ 2 N/2,N/2/ 2.5,N/2/ 2.5 ,...; and the invariant wavelet method at 7 levels of decomposition as in [5]. Note that the amplitude of the wavelet coeﬃcients have been divided by four for display purposes.

The Gabor and wavelet spectra contain only 7 points (log2 N) while thecSThas64points...... 117 6.11 Frequency maps for the delta function at F = 6 (top) and F =12 (bottom). The left column shows the unsmoothed data. The next two columns show the results after smoothing with a Gaussian of standard deviation σ = s/k, where s = 1 (middle) and s =1.5 (right). The rightmost column shows the Gabor-filtered response...... 118 6.12 (a) The local spatial frequency spectrum of a delta function using the cST with various levels of smoothing (σ = s/k). (b) The square root of the spectra shows that the amplitude increases with k2 for the unsmoothed case. The smoothed spectra deviate from this relationship. The shaded areas represent the “corner” frequencies above the Nyquist frequency that may be unreliable in analysis due to the drop-off in spectral power...... 120 6.13 An image from the Brodatz texture library [6] and the resulting frequency maps for wavenumbers: F = 5, 20, 40 and 80 for the cST with no smoothing (left), smoothing with σ =1.5 (middle) and the magnitude Gabor filter response (right)...... 121 6.14 A cross-section of the F = 12 frequency map for the delta response. While smoothing with s = 1 leaves some residual artifact, a value of s =1.5 results in a smooth spatial distribution and removes the ringing caused by the localizing window...... 122 6.15 The local spectra at the centre of a T2-weighted MR image at three different scales: 256 × 256 (original) and cropped to 128 × 128 and 64 × 64. The differences at low frequency (below 1 cycle/cm) arise due to anatomy cropped out of the field-of-view (FOV) as well as due to truncation artifact...... 124

xviii 6.16 The L1 norm between the spectrum of a pixel of a 256 × 256 T2- weighted MR image and the corresponding spectrum after shifting the image by -6 to +6 cm in steps of 2 cm...... 125 6.17 The effect of image non-uniformity on cST spectra at a location 20 pixels away from the center of the image horizontally and vertically (radial distance = 28 pixels). The non-uniformity field is modelled as a multiplicative Gaussian function of standard deviation 3N/2, N, N/2 to represent low, medium and high levels of non-uniformity, respectively...... 127 6.18 (a) A synthetic signal containing a sinusoid of increasing frequency: f(t) = sin(2π(t/6)t/N) where t = 0 to 255 and N = 256. (b) A contour plot of the magnitude ST for the positive frequencies. (c) In- version of the ST along the frequency axis by directly implementing (2.43). (d) The difference between the reconstructed and original signals shows that a large low-frequency error is apparent in the residual as well as high-frequency artifacts. (e) The reconstructed signal after applying AN according to (2.44) and (2.45) and using the traditional time-axis inversion according to (2.42) are shown as one line since they are indistinguishable. (f) The residuals show that frequency- axis inversion after correction (black) has maximum errors on the order of 10−14, which is comparable to that of time-axis inversion (gray) with maximum errors on the order of 10−15...... 128 6.19 Difference between original image and reconstructed using equation (6.7)...... 129 6.20 Images generated by applying a box-shaped filter G(x, y, F ) to the cST of the synthetic image in Fig. 6.8(a) and inverting. G(x, y, F ) consists of ones from (a)F=10to40and(b)F=1to10and40to 120, and zero elsewhere...... 130

7.1 The results of the NAWM segmentation, shown in red, overlaid on the month-1 PD-weighted image A. The lesion is shown in blue and the peripheral region, denoted as the lesion border, in black. The dashed yellow boxes show the 32× 32 ROIs used to get a representative sample of NAWM for the pST calculations...... 142 7.2 The average local spectra within (a) an enhancing lesion, (b) the lesion border, and (c) NAWM (month 1 of image A), using the traditional pST technique with a 32 × 32 ROI (squares) and our new method with a 428 × 428 pixel ROI (circles). The shaded region shows the LFE range of 0.8 cm−1 ≤ k ≤ 3.6cm−1...... 145

xix 7.3 The magnitude spectra for images A (a,b), B (c,d) and C (e,f) using the traditional pST method (left column) and our new method (right column). The axes are zoomed in to show the low frequency range. The errorbars represent the standard error of the mean. Note the improvement in spectral resolution using our method, and the improved distinction between lesion and border regions...... 147 7.4 (a) The T2-weighted image of patient #1 at month 1 when lesion A in the left hemisphere enhances. The red square shows the location of the 32 × 32 pixel ROI used for closer examination. (b) The time-evolution of T2 signal intensity within the ROI surrounding the enhancing lesion. (c) LFE maps generated by applying our new transform to the T2-weighted images, filtering with the band-pass filter G(x, y, k) and inverting...... 148 7.5 The average LFE (left column) and T2 (right column) intensity changes for lesions A (a,b), B (c,d) and C (e,f) that enhance in months 1, -2, -2, respectively, the lesion borders and NAWM. LFE signal is calculated by filtering our new transform to isolate only signal components within the LFE bandwidth. Error bars represent standard error of the mean...... 149 7.6 An example of the raw cST of image A at month 1 (k =3cm−1). The × indicates the location of the lesion. Areas in the brain that contain edges and other structures at the corresponding spatial scale appear bright...... 151 7.7 Image A at month 1, filtered to only retain the frequencies in the LFE bandwidth. Note that while areas corresponding to lesion activity are highlighted using this technique (white arrow), borders between brain tissue and CSF are also highlighted. This is likely due to the shape of the window and filter used, but may help to separate demylinating lesions from CSF and transient damage...... 154

8.1 Spectral partitioning of the 1D-DOST for seven orders (N = 128). The scaled and shifted rect functions are the FTs of the sinc basis functions deﬁned in (2.52)...... 161 8.2 (a) A modulated sinusoid time-series from [3]. (b) The DOST calculated using the basis functions in (2.52) and (c) by partitioning the frequency domain, according to (8.4). (d) The full, redundant ST from (2.36)...... 164

xx 8.3 (a) A signal from a functional MR imaging experiment, sampled at 3.33Hz. The plot shows the change in signal corresponding to five repetitions of a task 12s in duration (subject moving their fingers), followed by a 24s rest period. (b) The ST calculated according to (2.36). (c) The DOST of the signal...... 165 8.4 Partitioning of (a) the DWT and (b) the DOST for 6 orders. The squares indicate the sub-images for each order. Both transforms use a dyadic sampling scheme but provide different information about the frequency content of the image. The DWT gives horizontal, vertical and diagonal “detail” coefficients for each order, while the DOST provides information about the voice wavenumbers (νx,νy) that contain a bandwidth of 2px−1 × 2py−1 frequencies...... 168 8.5 (a) A mosaic of four texture images: straw, wood, sand and grass. (b) The 2D-DOST of the mosaic. (c) The DWT of the image using the db4 wavelet at five levels of decomposition. Note that the square root of the amplitude of both the DOST and DWT coefficients is shown for display purposes...... 171 8.6 The local DOST spectra of: (a) straw, (b) wood, (c) sand and (d) grass obtained from the center pixel of each region in Fig. 8.5a...... 175 8.7 (a) The wavelet decomposition for N = 8 at three levels of decomposition. Invariant wavelet coefficients are calculated by averaging horizontal and vertical coefficients at each level (marked with the same letter: A=level 0, B=level 1, etc.). The diagonal channels are excluded from the feature extraction. (b) The local frequency domain generated from the 2D-DOST for N = 8. Features marked with the same letter are averaged together to get an invariant DOST spectrum; a similar approach is taken of excluding the diagonal elements, where px = py. For a given N, the DOST approach provides more texture features (A to J = 10) than the wavelet approach (A to D = 4). . . . . 177 8.8 The nine Brodatz texture images used for texture analysis experiments...... 181 8.9 (a) The 2D-DOST of a delta function and (b) its local domain at the central point of the image (128,128)...... 183 8.10 (a) The calculated increase in memory required to store the ST and the DOST of a real N × N image. The ST requires N 4 floating point values while the DOST requires only N 2 (double for complex images). (b) The measured time to compute the ST and DOST. We were unable to compute the ST of images larger than 128 × 128 since the memory requirements became too large...... 185

xxi 8.11 (a) The SNR (dB) of the main frequency peak of the local DOST domain at the centre of a 256 × 256 image containing a single horizontally- oriented frequency component. (b) The SNR of the main peak when examining an image with a horizontally-and vertically-oriented single frequency component. Note that the SNR of these peaks drops more quickly with increasing noise and increasing frequency than when only one component is present...... 186

9.1 Examples of T2 (a, b), FLAIR (c, d) and T1-post contrast (e,f) MR images of an unmethylated (left column) and a methylated (right column) GBM...... 197 9.2 Examples of the texture features assessed visually on T1-post contrast images. (a) sharp and (b) undefined tumor borders; (c) ring and (d) nodular enhancement; (e) presence of a cyst (white arrow)...... 199 9.3 Examples of T2-weighted images assessed visually that were judged to have (a) homogeneous and (b) heterogeneous signal...... 200 9.4 (a) Mean log-transformed cST spectra from the ROIs of T2-weighted images for unmethylated (n = 28) and methylated (n = 31) tumors. Error bars are standard error of the mean. (b) The mean group differences and 95% CIs indicate that the spectral power of features highlighted with an asterisk are significantly higher in the unmethylated group than the methylated group...... 201 9.5 (a) Mean log-transformed invariant pST spectra from the ROIs of T2- weighted images. Error bars are the standard error of the mean. (b) The mean differences between methylated and unmethylated groups. Dashed lines indicate the 95% CI. The asterisks indicate where the 95% CI does not include zero...... 202 9.6 (a) Mean log-transformed invariant DOST features from the ROIs of T2-weighted images. Error bars are the standard error of the mean. (b) The mean differences between methylated and unmethylated groups. Error bars are the 95% CI. Asterisks indicate where the 95% CI does not include zero...... 203 9.7 (a) Mean log-transformed invariant DOST features from the ROIs of FLAIR-weighted images. Error bars are the standard error of the mean. (b) The mean differences between methylated and unmethylated groups. Error bars are the 95% CI. The asterisk indicates where the 95% CI does not include zero...... 204

xxii 9.8 ROC analysis of various combinations of texture features using linear (black) and quadratic (grey) discriminant analysis. Filled symbols represent results when adding the ring enhancement variable. Clas- siﬁcation using the DOST features along with the ring enhancement variable produced the best sensitivity and speciﬁcity of all the various methods tested...... 206

10.1 The gradient image G(x, y) corresponding to the image shown in Fig. 10.3...... 214 10.2 Examples of sixth order Butterworth ﬁlters for values of k0 =0.3to 0.6...... 215 10.3 (a) An example of a T2-weighted MR image. The results after applying the cST ﬁlter are shown using (b) C =0.003, (c) C =0.005, and (d) C =0.01...... 216 10.4 A 512× 512 MR image of a relapsing-remitting MS patient, along with two areas examined for texture: NAWM (dotted white) and a lesion (dotted black)...... 219 10.5 The local frequency maps derived from the DOST of the central 5× 5 regions of the (a) NAWM (solid white box from Fig. 10.4) and (b) the lesion (solid black box from Fig. 10.4). The local frequency maps indicate the magnitude of each frequency order (px,py) that contains the total contribution of spatial frequencies from 2px−1 to 2px (horizontal) and 2py−1 to 2py (vertical). Note that the negative frequencies are computed through Hermitian symmetry of the space- frequency domain. The lesion contains more low-frequency power (low orders px,py) than the NAWM, consistent with the results of [7]. 220 10.6 Cumulative survival function. The plot shows the percentage of patients that survive with time after diagnosis. Those patients with high values of the T1 spectrum have a 70% lower risk of death than those with low values (P = 0.002)...... 224 10.7 Cumulative hazard function (the negative log of survival). The plot shows how the cumulative risk of death increases with time after diagnosis. Patients with low values of the T1 cST spectrum have a 3 times higher risk of death than those with low values (P = 0.002). . . . 225

xxiii List of Abbreviations

1D one-dimensional

2D two-dimensional

3D three-dimensional

4D four-dimensional

CI conﬁdence interval

CSF cerebrospinal ﬂuid cST circular S-transform

CT computed tomography

CWT continuous wavelet transform dB decibels db4 Daubechies 4-tap mother wavelet

DFT discrete Fourier transform

DNA deoxyribonucleic acid

DOST discrete orthonormal S-transform

DWT discrete wavelet transform

FFT fast Fourier transform

FLAIR ﬂuid-attenuated inversion recovery fMRI functional magnetic resonance imaging

FOV ﬁeld-of-view

FT Fourier transform

GBM glioblastoma

GT Gabor transform

xxiv LFE low frequency energy MGMT O6-methylguanine-DNA methyltransferase MR magnetic resonance MS multiple sclerosis MS-PCR methylation-speciﬁc polymerase chain reaction N3 nonparametric intensity nonuniformity normalization NAWM normal appearing white matter NMR nuclear magnetic resonance NPV negative predictive value PD proton density PPV positive predictive value pST polar S-transform RF radiofrequency RMS root mean squared ROC receiver operating characteristic ROI region of interest SNR signal to noise ratio ST S-transform STFT short time Fourier transform

T1 T1 relaxation time

T2 T2 relaxation time TE echo time TI inversion time TMZ temozolomide TR repetition time

xxv 1

Chapter 1

Introduction

1.1 Thesis Objectives

My PhD studies were focused on expanding current knowledge of space-frequency analysis of medical images. In particular, my work has focused on improving image analysis methods based on the S-transform (ST). ST-based techniques are desirable for medical imaging because: (a) they are closely related to the Fourier transform

(FT) (the basis of MR and computed tomography (CT) image reconstruction); and,

(b) they are invertible — that is, one can move between the image domain and the

ST domain without loss of information. This property allows sophisticated ﬁltering and analysis of images.

This thesis describes several novel contributions resulting from my research.

These include: (1) new ways of reducing the amount of computational resources required to compute and store the ST; (2) new ways of extracting and using information from the ST for applications in medical image processing (such as filtering and texture analysis); and (3) the application of these ST-based techniques to help solve clinical problems, such as monitoring lesions in MS and quantifying texture differences in sub-types of brain cancer. Along the way, I tried to take advantage of novel aspects of the techniques I developed and show how these features can be used to perform new types of analysis, such as space-frequency image filtering. Finally,

I promoted the use of the ST in medical imaging to scientists and clinicians and 2 demonstrated that it is a robust tool for image and signal analysis.

1.2 Thesis Structure

The first three chapters of the thesis contain background information related to my PhD research. The first chapter introduces the objectives and structure of the thesis and outlines the scientific contributions described within. The second chapter provides a mathematical overview of transforms related to the ST, describes the motivation for choosing this project and details the significance and contribution of the work. The third chapter outlines the basic theory of MR imaging necessary to understand the application of my work to clinical problems. The fourth chapter describes how space-frequency analysis can be applied to medical imaging in terms of image filtering, texture analysis and classification. In the introductory chapters, limitations of current methods are discussed and further motivation for pursuing this research project are presented.

Chapters five through nine each describe a novel scientific contribution. Each of these chapters has been published, or submitted for publication, in a peer-reviewed journal. The fifth chapter examines why the pST requires such lengthy computation times and what effect different image ROI sizes has on ST-based texture measures. In response to these concerns, I present the introductory theory of a novel transform, known as the cST. I present a study examining the relationship between the new transform and the pST and illustrate some of the advantages of the new approach, including improved spectral resolution. In the sixth chapter,I describe the characteristics of the cST in more detail and outline some of its unique 3 properties, including a novel application of a newly proposed inversion method for the ST. Experiments are presented that attempt to characterize the performance of the cST in simulated and real-world conditions. The seventh chapter presents an application of the cST to images of patients with MS. This work illustrates a novel application of space-frequency analysis that is not possible with the pST — applying filters to the spectra to isolate spectral components thought to be involved in MS lesion progression.

The eighth chapter presents another novel method for medical image analysis known as the DOST. Motivated by new work published by Dr. Stockwell in 2007, I change direction in this chapter and adapt this discrete orthonormal transform for image texture analysis. The eighth chapter outlines the frequency-domain DOST algorithm, presents the 2D-DOST for the ﬁrst time and outlines potential texture descriptors that can be derived from the DOST domain. I conduct experiments to characterize the performance of the DOST and to evaluate its response to noise. I also compare the classiﬁcation power of features derived from the DOST features to those from a leading wavelet-based technique on a library of texture images.

The ninth chapter presents a clinical study examining the texture of a brain tumor known as GBM. Texture differences are sought that correlate with genetic alterations that predict tumor response to chemotherapy. I use all of the ST-based techniques presented in this thesis (the cST, pST and DOST) to perform texture analysis of GBM tumors. I perform blind classification of the tumors with regard to their genetic status using an analysis of entire image slices as well as small ROIs located within the tumors. Texture characteristics are also assessed visually by an expert. It is shown that the DOST of image ROIs, in combination with visually 4 assessed features, produces texture features that most accurately classify GBM patients. It is also shown that texture features correlate with survival — an interesting and novel finding.

The tenth chapter concludes the thesis by summarizing the major research

ﬁndings and providing suggestions for future research directions.

1.3 Signiﬁcance and Contribution

The work presented in this thesis has been published and presented in various forms.

The introduction to each chapter describes the publications, as well as any awards or patents earned based on the work. In total, the work presented in this thesis has culminated in 3 manuscripts published or submitted for publication in peer- reviewed scientiﬁc journals and one paper in a peer-reviewed conference proceeding;

4 abstracts published in peer-reviewed conference proceedings and one that has been accepted for presentation; 4 poster and 3 podium presentations; and numerous other presentations at seminars, rounds, invited talks, and journal clubs. In addition, the intellectual property based on the two novel transforms described in this thesis has been transferred to Calgary Scientiﬁc, Inc. and forms the foundation of two provisional patents. I have been awarded 5 investigator awards and travel stipends to present this work at Canadian and international conferences and have won awards for best poster and best podium presentations at local and international conferences.

The research presented in this thesis is a collaborative eﬀort from a team of researchers at the Foothills Hospital, University of Calgary with contributions from others as well. I have been the lead researcher and the ﬁrst author of all publica- 5 tions presented in this thesis. Dr. Ross Mitchell has provided support and guidance throughout the research project and is a co-author on all of the publications presented in this thesis. Individual contributions to each project are outlined at the beginning of each chapter.

Please note that while I have tried to keep the notation in this thesis as consistent as possible, because I am incorporating sources from different disciplines that use differing notation, at times I have taken liberties with defining variables. For example, I refer to some continuous and discrete variables with the same symbol. While such definitions may not always be strictly correct in a mathematical sense, I have attempted to convey the information in this thesis as clearly and simply as possible.

My hope is to keep the information accessible to readers of varying backgrounds. 6

Chapter 2

Introduction to Space-Frequency Analysis

This chapter provides an introduction to Fourier theory and space-frequency analysis. The most common approaches to multiscale analysis, based on windowed

Fourier transforms, wavelets and the S-transform, are introduced. Limitations of each method are discussed and the motivations for new techniques are introduced.

The curvelet transform is brieﬂy mentioned in this chapter for completeness; however, analysis using the curvelet transform is not pursued in this thesis. The comparison of the ST to curvelets is left as future work.

2.1 Fourier Transform

The FT is a mathematical procedure that expresses a signal (or image) in terms of sines and cosines. Joseph Fourier ﬁrst described the concept of representing a signal as the superposition of waves (harmonic analysis) in a decomposition known as a

Fourier series, for the purposes of solving the heat equation [8]. Fourier analysis is now used in a wide variety of areas including mathematics, science, engineering and medicine, for applications in signal processing, communications, astronomy, geology and optics, as well as being integral to the development of modern medical image reconstruction in modalities such as MR [9] and CT.

The FT of a signal f(t) is deﬁned as:

∞ fˆ(ξ)= f(t)e−2πitξdt (2.1) −∞ 7 where t represents time and ξ represents frequency. The FT can occur forwards and backwards without loss of information. The inverse FT is deﬁned as

∞ f(t)= fˆ(ξ)e2πitξdξ. (2.2) −∞

The extension to a 2D function of spatial variables x and y is straightforward, consisting of two one-dimensional (1D)-FTs: one in the x-direction and one in the y-direction, ∞ ∞ fˆ(α, β)= f(x, y)e−2πi(αx+βy)dxdy (2.3) −∞ −∞ where α and β are the continuous horizontal and vertical frequency variables, respectively. The inverse 2D-FT is

∞ ∞ f(x, y)= fˆ(α, β)e2πi(αx+βy)dαdβ. (2.4) −∞ −∞

For a signal sampled at integer locations t =0, 1,...,N − 1 in time with a unit sampling interval ΔT , f[tΔT ], the discrete Fourier transform (DFT) is

N−1 n fˆ = f[tΔT ]e−2πint/N . (2.5) NΔT t=0 We deﬁne n as the wavenumber and ξ = n/NΔT as the frequency in units of cycles per unit length of time or space. The inverse DFT is given by

N/2−1 1 n f[tΔT ]= fˆ e2πint/N . (2.6) N NΔT n=−N/2

In the remainder of this thesis we will assume, unless otherwise stated, that signals have a unit sampling interval (i.e. ΔT =ΔTx =ΔTy = 1). The DFT is commonly implemented using fast Fourier transform (FFT) methods, described in [10, 11]. The FFT reduces the number of computations required 8

2 to calculate the FT of a N-length array from N to less than 2N log2 N without increasing the storage requirements [10]. For the remainder of this thesis, when we describe computer implementations of the FT on discrete signals, it is implied that we are referring to the DFT, implemented using a FFT algorithm.

The FT provides either a spatial or spectral representation of a signal (or image).

In order to view both the time and frequency information simultaneously, a time- frequency transform must be used. The time-frequency transform of a 1D signal is a 2D function of time and frequency, where each point in the signal has associated with it a 1D frequency proﬁle, often called a “local spectrum”. In 2D this is often known as a “space-frequency” of “space-spatial frequency” domain1 where each pixel has associated with it a 2D local spectrum, sometimes known as a local FT.

If we know the exact time that an event occurs, we cannot know its exact frequency. If we know the exact frequency of a signal component, we cannot know its exact position in time (or space). This relationship is expressed formally in the uncertainty relation, which states that the product of the spectral bandwidth

σξ multiplied by the time dispersion of a signal σt cannot be less than a certain minimum value [12,13], 1 2 σ2σ2 ≥ . (2.7) t ξ 4π

The time-frequency plane can be tiled or partitioned into regions of area σt × σξ, such that σt and σξ satisfy (2.7). For example, the tallest, thinnest partitions allowed by the signal sampling interval produce optimal time localization but no frequency localization (Figure 2.1 (a)). This partition is known as the standard or Dirac basis.

1Note that in the remainder of the thesis, the term “space-frequency” will be used for both 1D and 2D signals. However, it should be noted that for 1D signals we are usually referring to the relationship between time and frequency. 9

Conversely, the Fourier basis (Figure 2.1 (b)) has optimal frequency localization, but no time localization [1]. The time-frequency domain may be partitioned in a uniform manner (Figure 2.1 (c)) by using a constant width window, or may be partitioned in a dyadic manner where the time duration decreases and the spectral bandwidth increases by a factor of two with increasing frequency (Figure 2.1 (d)).

A window function w(t) can be used to deﬁne the time-frequency partitioning.

The time dispersion of w(t) can be deﬁned as [13], ∞ 2| |2 ∞ | |2 2 2 −∞ t w(t) dt −∞ t w(t) dt σ = ∞ − ∞ (2.8) w | |2 | |2 −∞ w(t) dt −∞ w(t) dt with the spectral bandwidth ofw ˆ(ξ) deﬁned by substituting ξ for t andw ˆ(ξ) for w(t).

2.2 Windowed Fourier Transforms

A windowed FT, also known as a short time Fourier transform (STFT), is an FT that is applied to a portion of the signal formed by multiplying the signal with a window function w(t) [13]. The STFT of a function f(t) is given by

∞ V (τ,ξ)= f(t)¯w(t − τ)e−2πitξdt (2.9) −∞ where w(t − τ) is a localizing window, centered on τ. The inverse STFT is given by

∞ ∞ f(t)= V (τ,ξ)e2πitξdξ dτ (2.10) −∞ −∞ assuming that w(t) is deﬁned such that

∞ w(τ)dτ =1. (2.11) −∞ 10

(a) (b)

Figure 2.1: Tilings of the time-frequency plane for (a) optimal time localization and no frequency localization (Dirac basis), (b) optimal frequency localization and no time localization (Fourier basis), (c) uniform partitioning obtained with a constant width window and (d) dyadic partitioning. Figures adapted from [1,2]. 11

The size and shape of the window used in the STFT determine σt and σξ. Many types of windows are commonly used with the STFT, including Hamming, Hanning and

Blackman-Harris. An excellent overview of numerous windows and their properties can be found in [14].

A Gaussian window with standard deviation σ is often used with the STFT

1 2 2 w(t)=√ e−t /2σ , (2.12) 2πσ which has a FT

2 2 2 wˆ(ξ)=e−2π ξ σ . (2.13)

The time dispersion and spectral bandwidth for the Gaussian window, found using

(2.8), are

σ2 σ2 = t 2 1 σ2 = (2.14) f 8π2σ2 and the uncertainty relation becomes, σ2 1 1 σ2σ2 = = . (2.15) t ξ 2 8π2σ2 16π2

Therefore, the Gaussian function satisﬁed the minimum of the uncertainty relation and simultaneously optimizes the time and frequency resolution of the STFT [12,13].

The ﬁxed window width of the STFT imposes a constant time-frequency resolution for all frequencies (Figure 2.1 (c)). If the window width is not chosen properly, important features of the signal may not be evident in the time-frequency representation. Use of a wide window leads to good frequency resolution, but poor time resolution. As a result, signal components of short duration may be poorly visualized 12

(if at all). Conversely, the use of a narrow window provides good time resolution, but poor frequency resolution.

Figures 2.2 to 2.4 show the outcome of using the STFT with various window widths. The ﬁgures show a signal containing of a superposition of two sine waves: a low-frequency component (0.068 Hz) and a chirp with frequency that increases quadratically with time from 0.039 to 0.39 Hz. From 154–166 s there is a pulse with amplitude 2.0. When a wide time-domain Gaussian is used (width σ =51s)in

Fig. 2.2, the low frequencies are well deﬁned, but the pulse is missing entirely. As the window width decreases to σ = 13 s in Fig. 2.3, the pulse becomes visible, but the time resolution does not allow for precise identiﬁcation of when the pulse starts and ends. A window of width σ =3.2 s in Fig. 2.4 resolves the pulse well, but the frequency resolution of the remaining components degrades.

2.2.1 Gabor Transform and Gabor Filters

The Gabor transform (GT) is a discretely sampled STFT, ﬁrst proposed by Denis

Gabor in 1946 [15]. An image can be decomposed into its Gabor expansion by applying a series of shifts and modulations of the Gabor window at constant spacings in the time-frequency domain, indicated by solid circles in Figure 2.5.

The GT can be deﬁned as

G(m, n)=V (m · a, n · b) ∞ = f(t)w(t − am)e−2πit(bn)dt (2.16) −∞ ∞

= f(t)hm,n(t)dt (2.17) −∞ (2.18) 13

Figure 2.2: A signal consisting of a low-frequency sine wave (0.068 Hz), a quadratic chirp increasing in frequency from 0.039 to 0.39 Hz and a pulse from 154–166 s (bottom) and its FT (left). The STFT with a wide temporal window resolves the low frequency components well, but misses the pulse entirely. 14

Figure 2.3: As the STFT window width decreases, the pulse becomes visible, but the time resolution does not allow for precise identiﬁcation of when the pulse starts and ends. The frequency resolution of the remaining components is not as good as in Fig. 2.2. 15

Figure 2.4: With a narrow temporal window, the STFT resolves the location of the pulse well, at the expense of the degrading frequency resolution. 16

Figure 2.5: Sampling of the time-frequency plane for the GT. a and b determine the distance of the samples in the time and frequency directions, respectively. where m, n are integers and a, b are ﬁxed numbers that determine the sampling of the time-frequency plane (Figure 2.5). hm,n(t) can be used to describe the “Gabor wavelet” consisting of the window function multiplied by the complex exponential.

Gaussian functions are commonly used as the window function w because they simultaneously achieve optimal localization in space and spatial frequency [16].

2D Gabor ﬁlters, also known as Gabor wavelets [2], have the general form [16]:

h(x, y)=w(x,y)e2πi(kxx+kyy) (2.19)

where kx and ky are the horizontal and vertical frequencies of interest, w(x, y) is the 17

2D Gaussian function, 2 2 1 − (x/λ) +y w(x, y)= e 2σ2 (2.20) 2πλσ2 and where (x,y)=(x cos φ+y sin φ, −x sin φ+y cos φ) are the rotated co-ordinates of the Gaussian, whose major axis is oriented at an angle φ from the x-axis. A

fixed set of filters is often chosen by the user at specific centre frequencies and orientations to obtain an optimal coverage of the Fourier domain [5, 17]. In 2D, the Gaussian windows are not necessarily symmetric in the horizontal and vertical spatial frequency directions2. The aspect ratio of the Gaussian is given by λ and the scale parameter is given by σ. These 2D Gabor functions also achieve the minimum of the uncertainty relation [19].

The Gabor ﬁlter in (2.19) can equivalently be given in the spatial frequency domain by:

2 2 2 2 2 hˆ(α, β)=e−2π σ [(α −kx) λ +(β −ky) ] (2.21) where (α,β) are the rotated co-ordinates of the spatial frequency variables.

If certain conditions on the sampling and window functions are met [16, 20, 21] then the image can be reconstructed from the coefficients. Generally, this is a complex process and an approximation to the expansion coefficients are computed instead. Note that the Gabor filter is admissible as a wavelet, but does not result in an orthogonal decomposition; instead, it produces a redundant transformation [17].

2It is interesting to note that in 1980, Daugman showed that the receptive ﬁeld of the mam- malian visual system could be modelled by 2D elliptical Gaussian windows in the Fourier domain [18] 18

2.3 Wavelets

2.3.1 Continuous 1D Wavelet Transform

Wavelets provide a multiresolution decomposition of an image. A multiresolution representation refers to the analysis of an image at a hierarchical set or pyramid of resolutions; at coarse resolution, image details correspond to larger structures while at ﬁner resolutions the details correspond to small structures. Decomposing a signal with an orthogonal wavelet basis gives an intermediate representation between

Fourier and spatial representations [22].

Wavelets were first introduced by Grossmann and Morlet [23] in 1984 for the analysis of seismic data. Meyer [24] first proposed an orthogonal wavelet in 1985, which was extended to more than one dimension by Lemarié and Meyer in 1986 [25].

Mallat [22] expanded on this idea to show that there exists a family of orthogonal wavelet representations that can be built by dilating and translating a unique function. Mallat ﬁrst introduced the concept of multiresolution analysis, which led to a breakthrough in the understanding of wavelets [26]. Daubechies developed the ﬁrst orthogonal wavelets with compact support [27].

The study of wavelets has become a massive ﬁeld unto itself with applications including image reconstruction, compression, noise reduction, image enhancement, texture analysis/segmentation and registration [2]. A review of biomedical applications of wavelets can be found in [28]. As of 2005, more than 9000 journal articles and 200 books had been published related to wavelets [2]. An excellent review of the history of wavelets can be found in [29] and of wavelet theory in [30].

The continuous wavelet transform (CWT) of a signal f(t) is computed by con- 19 volving the signal with a function ψ, known as a wavelet, that is shifted by τ and scaled by s [12], 1 ∞ t − τ W (τ,s)= f(t)ψ∗ dt (2.22) |s| −∞ s where ∗ represents the complex conjugation operation.

An example of a mother wavelet is the Mexican hat function, which is the second

2 derivative of the Gaussian function e−t /2 [30]

2 2 ψ(t)=√ π−1/4(1 − t2)e−t /2. (2.23) 3

The scaling variable s either dilates or compresses the mother wavelet and is allowed to take on continuous values in the range (0, ∞). Large values of s dilate

ψ and provide low frequency information. As s →∞, the frequency band covered by the wavelet approaches zero (the DC component of the signal). Small values of s compress ψ and produce high frequency information.

The admissibility condition for a wavelet is ∞ |ψˆ(ω)|2 Cψ = dω < ∞ (2.24) 0 ω where ψˆ(ω) is the FT of ψ(t) [12]. Therefore, in order for (2.24) to be ﬁnite, ψˆ(0) =

0; in other words, the mother wavelet must have zero mean. If the admissibility condition is satisﬁed, the CWT can be inverted according to [12] 1 ∞ ∞ ψ (t) f(t)= τs W (τ,s)dτds. (2.25) 2 Cψ −∞ 0 τ

2.3.2 Discrete 1D Wavelet Transform

The DWT is a sampled version of the CWT where the values of s and τ are dis- cretized. A dyadic wavelet decomposition of a signal f[t],t=0, 1,...,N − 1 where 20

Figure 2.6: Dyadic tiling of the time-scale domain of the DWT. The width of each partition decreases by a factor of 2 as the scale decreases and the scale bandwidth doubles by a factor of two.

N =2l has scales s =2l and τ =2lk for k =0, 1,...and integer values l. The basic form of the wavelet becomes [12] 1 t ψlk(t)=√ ψ − k . (2.26) 2l 2l

Figure 2.6 shows the dyadic partitioning of the time-frequency plane.

Wavelets that are orthogonal to dilations and translations of the mother wavelet satisfy the condition [31] ⎧ ⎪ ∞ ⎨ ∗ 1ifj = m and k = n ψjk(t)ψmn(t)dt = ⎪ (2.27) −∞ ⎩ 0 otherwise. 21

Figure 2.7: DWT decomposition of a signal x[n] with low pass ﬁlters g[n] and high pass ﬁlters h[h]. The symbol ↓ 2 represents downsampling by a factor of two by discarding every second sample.

To compute the DWT of a signal f, one applies two filters to the signal, a high-pass filter h[n] and a low-pass filter g[n] (also known as wavelet and scaling functions)3. The two filters, in conjunction known as a quadrature mirror filter, are related and can be derived from one another by reversing the order and changing the sign of every second coefficient [32]

g[M − 1 − n]=(−1)nh[n] (2.28) where n =0, 1,...M − 1 is the coeﬃcient index and M is the length of the ﬁlter.

This is often known as an octave ﬁlter bank, where an octave is the bandwidth between two frequencies, one of which is double the other [12].

The DWT is computed by successively applying g[n] and h[n]. The first level of coefficients is computed by applying the high-pass filter h[n] and downsampling by a factor of two. Downsampling is performed by simply discarding every second sample point [32]. The low-pass filter g[n] is then applied to the signal and the output is downsampled by 2. The process is repeated on the output signal to obtain the

3Low- and high-pass filters are discussed in more detail in Chapter 4 22 next level of decomposition. In this way, the filtering and downsampling procedure continues until we reach the last two signals that contain two components each. The coefficients obtained from the high-pass filtering are known as “detail” coefficients and those from low-pass filtering as “approximation” coefficients. The result of the

DWT is known as the wavelet decomposition, wavelet transform or the tree wavelet analysis of the signal. Figure 2.7 illustrates this procedure on a signal x[n].

A common choice of mother wavelet is one of an orthogonal family of wavelets discovered by Daubechies [30]. The wavelets do not have a closed form, but are given in terms of the M number of coeﬃcients. For example, the simplest Daubechies wavelet is the Daubechies 4-tap mother wavelet (db4) which contains four coeﬃcients

[33]

1 √ √ √ √ gdb4[n]= √ [1 + 3, 3+ 3, 3 − 3, 1 − 3] 4 2 1 √ √ √ √ hdb4[n]= √ [1 − 3, 3 − 3, 3+ 3, −1 − 3]. (2.29) 4 2

If necessary, these ﬁlters are padded with zeros to equal the length of the signal.

2.3.3 Discrete 2D Wavelet Transform

The DWT can be extended to 2D or three-dimensional (3D) data sets by successively applying the 1D transform along each dimension of the data. To obtain horizontal, vertical and diagonal detail and approximation coeﬃcients the low- and high-pass

ﬁlters are applied to the rows and columns of the image as shown in Table 2.1 [34], followed by the downsampling operation.

Successive decomposition levels are computed by applying the low- and high- pass ﬁlters to the ﬁltered and downsampled approximation images. The DWT of 23

Table 2.1: Low-pass (g) and high-pass (h) filters applied to the rows and columns of an image to generate horizontal (H), vertical (V), and diagonal (D) detail and approximation (A) coefficients. coefficient row column H hg V gh D hh A gg a N × N image produces N 2 wavelet coefficients, typically arranged as shown in

Figure 2.8, where the level one decompositions are size N/2 × N/2, level two are

N/4 × N/4, etc. In a discrete orthogonal wavelet representation, the total number of pixels in the DWT is equal to the number of pixels in the original image [22].

2.4 Curvelets

Curvelets are a recent extension to wavelet analysis, ﬁrst developed by Candes and

Donoho in 2000 [35], to examine curved edges in images. Curvelets have not been widely used in medical image analysis. Some work has been done on fusion of MR and CT images [36] and on MR image compression using curvelets [37], but the ﬁeld is still very new and evolving.

The curvelet transform is a multiresolution representation like the DWT. How- ever, the curvelet transform uses radial and angular windows to partition the frequency domain into polar “wedges”, as shown in Fig. 2.9 (a). The rotation angles

−|j/2| for a given scale j are given by θl =2π · 2 · l, with l =0, 1,... such that

0 ≤ θl < 2π [38]. Since this type of partitioning is not especially compatible with data sampled on a Cartesian grid, in practice the Fourier domain of the image 24

V1 D1

V2 D2

H1 V3 D3 H2 A3 H3

Figure 2.8: Three levels of a 2D DWT (levels are denoted by the subscripts). Average coeﬃcients are denoted by “A” and detail coeﬃcients represent: H: horizontal, D:diagonal, V:vertical. 25

(a) (b)

Figure 2.9: (a) The frequency domain tiling of the curvelet transform into polar “wedges”. (b) The Cartesian tiling based on concentric squares often used in practice when calculating the curvelet transform. is often divided into dyadic annuli based on concentric squares, which are further subdivided into trapezoidal regions [38] (as shown in Fig. 2.9 (b)).

The original implementation of the curvelet transform consisted of decomposing the image into a series of sub-bands, spatially partitioning those sub-bands, and performing what is known as a ridgelet transform to each block [39]. However, this method has been largely replaced by more intuitive implementations based on the

FFT. However, complexity arises because the traditional FFT cannot be used on the sheared grid of the curvelet transform [38]. As a result, calculation of the curvelet transform is somewhat more complex than a windowed FT. One can either use an unequally spaced FFT or a wrapping method to evaluate the trapezoidal partitions of the curvelet transform. Both approaches are described in detail in [38]. 26

2.5 S-Transform

The ST was ﬁrst introduced for 1D signals by Stockwell in 1996 [40] for the analysis of seismic signals. It was extended to 2D by Mansinha et al in 1997 [41]. The

1D-ST has been used for the analysis of signals in numerous applications, including seismic recordings [42], gravitational waves [43], power systems analysis [44,45] and medical signals including electroencephalography [46], laser doppler ﬂowmetry [47], cardiovascular time series [48] and functional magnetic resonance imaging (fMRI)

[49–51]. The 2D-ST has been used for texture analysis of medical images [7,52–58]

The ST is a transform that maintains the phase and linear scaling of the FT, while providing a time-frequency representation of a signal similar to that of the

STFT. However, the important distinction is the use of a window whose width varies with frequency. Therefore, the ST can be thought of as an intermediate transform between the STFT and the CWT. The ST can be described as a “phase-corrected”

CWT [40,59] S(τ,ξ)= |ξ|e−2πiξτ W (τ,s) (2.30) with a complex Morlet wavelet [60]

1 − 1 2 2πit ψ(t)=√ e 2 t e . (2.31) 2π

2.5.1 1D S-Transform

The 1D-ST can be deﬁned as [40,61] ∞ S(τ,ξ,γ)= f(t)w(τ − t, ξ, γ)e−2πitξdt (2.32) −∞ where τ represents the time-point of interest, ξ represents frequency, and γ is a parameter that controls the width, or dilation, of the window w. The most commonly 27 used localizing window is a Gaussian function with standard deviation inversely proportional to the frequency, σ = γ/ξ [61]

2 2 |ξ| − (τ−t) ξ w(τ − t, ξ, γ)= √ e 2γ2 ,γ=0 . (2.33) γ 2π

A value of γ = 1 is commonly used, providing approximately one modulated sine and cosine cycle over the duration of the signal [61]. For the remainder of the thesis a value of γ = 1 will be assumed unless otherwise stated.

The varying window width of the ST provides a trade-oﬀ between time and frequency resolution. As shown in Fig. 2.10, the ST overcomes the issue with the

STFT of having to select a window width that is only optimal for one particular time- frequency resolution. The ST provides good resolution of both time and frequency, allowing one to identify all of the spectral components.

A variety of window functions have been proposed for use with the ST [62–65].

Localizing windows must satisfy the condition

∞ w(τ − t, ξ)dτ =1 ∀ t, ξ (2.34) −∞ to ensure invertibility [63].

If we recognize the 1D-ST algorithm in (2.32) as a convolution in time, we can reduce the number of computations required by taking advantage of the Fourier convolution theorem,

F[f ∗ g]=F[f] · F[g] (2.35) which states that the FT F of a convolution of two functions, f and g, is a multiplication of the FTs of f and g. Therefore, we can write (2.32) as an operation on 28

Figure 2.10: The ST of the signal from Fig. 2.2 provides good resolution of both time and frequency, allowing one to identify all of the spectral components. 29 fˆ(ξ), the FT of f(t) [40], ∞ 2 2 − 2π η S(τ,ξ)= fˆ(ξ + η)e ξ2 e2πiητ dη, ξ = 0 (2.36) −∞ where the FT of the signal is shifted before applying the localizing window. Equation

(2.36) can be extended to a discrete time signal f[tT ], sampled at integer locations t =0, 1,...,N − 1, with sampling interval T [40], N−1 2π2n2 n n + n − 2πinj/N S jT, = fˆ e n2 e ,n= 0 (2.37) NT NT n=0 where n is a frequency variable.

2.5.2 2D S-Transform

Equation (2.36) can be directly extended to 2D. The 2D-ST of an image f(x, y)is given by [61] „ « 2 ∞ ∞ 2 α2 β −2π 2 + 2 ˆ kx ky 2πi(αx+βy) S(x, y, kx,ky)= f(α + kx,β+ ky)e e dαdβ (2.38) −∞ −∞

ˆ where the 2D-FT of the image, f(α, β), is centered on the frequency (kx,ky) before multiplying by the 2D Gaussian window.

For a discretely sampled image f[qTx,rTy], where p =0, 1,...,N − 1 and q = 0, 1,...,M − 1, we have

N−1 M−1 “ ” “ ” 2 2 nq n m n + n m + m −2π2 n + m 2πi + m r ˆ n2 m2 N M S qTx,rTy, , = f , e e . NTx MTy NTx MTy n=0 m=0 (2.39)

For simplicity, in the discrete case it will be assumed that Tx = Ty =1;x, y will refer to the discrete spatial indices q, r, respectively; and kx,ky will refer to n/N, m/M, 30 respectively. If the distinction between discrete and continuous calculations is important, discrete equations will be emphasized by the use of square brackets. Otherwise, the reader is directed to interpret the equation within the context it is used.

2.5.3 S-Transform Inversion

Inversion of the ST consists simply of an integration over the time variable and an inverse FT [40] ∞ ∞ f(t)= S(τ,ξ)dτ e2πiξτ dξ. (2.40) −∞ −∞ In 2D the inversion formula is given by,

∞ ∞ ∞ ∞ 2πi(xkx+yky) f(x, y)= S(x ,y,kx,ky)dx dy e dkxdky. (2.41) −∞ −∞ −∞ −∞

The discrete inversion formula is given by N/2−1 N−1 1 n f [tT ]= S jT, e2πint/N (2.42) T N NT −N/2 j=0 where the subscript T denotes that this inversion formula reconstructs the signal by summing over the time variable, with a straightforward extension to 2D.

Recently an alternative inversion formula has been proposed that works by integrating over the frequency variable instead of the time variable [66],

− √ N/2 1 S[tT, n ] NT 2πint/N fF [tT ]= 2π n e . (2.43) n=−N/2 NT n=0

This new approach reconstructs the signal directly, avoiding the time-averaging in equation (2.42). It is intended to preserve time-localization by reconstructing each time point of the signal independently. Subsequent work suggests that the original 31 implementation may contain errors in the reconstructed signal that can be corrected by performing an additional deconvolution step [67,68], N/2−1 1 fˆ n f [tT ]= F NT e2πint/N (2.44) C N A n n=−N/2 N NT where AN for the positive frequencies is given by:

N/2−1 2 2 2 n √ e−2π (n−n ) /n A = 2π ,n>0. (2.45) N NT | n | n=1 NT

AN is calculated by shifting and summing the Gaussian windows. Note that AN [0] =

1 and that AN is symmetric about the origin. Further discussion regarding the theoretical basis for inversion of the ST, as well as limitations for discrete signals, can be found in [69] and [70], respectively.

Limitations of the 2D-ST include its signiﬁcant computational and storage requirements. For example, the time to calculate the 2D-ST of a N × N image and the amount of memory required to store the result both increase approximately with N 4. That means that the ST of a 256×256 MR image takes approximately

1.5 hours to calculate and 32 GB to store [56]. The 2D-ST also produces a four- dimensional (4D) structure of space and spatial frequency, making visualization and analysis of the transformed data diﬃcult [61]. Furthermore, since the ST provides spatial frequency information in terms of horizontal and vertical components, it is not rotationally invariant.

2.5.4 The Polar S-Transform

The pST was developed to provide a rotationally invariant space-frequency domain. 2 2 The Gaussian window in the pST is ﬁxed to a centre frequency k = kx + ky and 32

−1 rotates with frequency orientation θ = tan (ky/kx). The pST of an image f(x, y) is deﬁned as [71]

∞ ∞ −2πi(kxx +kyy ) pST (x, y, kx,ky)= f(x ,y)wpST (x − x, y − y, k, θ).e dx dy −∞ −∞ (2.46)

The polar Gaussian localizing window wpST (x, y, k, θ) is deﬁned as

2 k λ(k) 2 2 2 2 2 w (x, y, k, θ)= e−k (x cos θ+y sin θ) /2e−k λ(k) (−x sin θ+y cos θ) /2 (2.47) pST 2π where λ(k) > 0 determines the shape (aspect ratio) of the localizing window. If

λ(k) = 1, the window is symmetric in both the radial and orientation directions, resulting in circular windows. Small values of λ(k) yield broad windows with improved spectral resolution but poor spatial resolution. Conversely, large values of

λ(k) produce narrow windows with good spatial resolution but poor spectral resolution [71].

The pST can also be expressed as an operation in the frequency domain [71],

∞ ∞ ˆ 2πi(αx+βy) pST (x, y, k, θ)= f(α + kx,β+ ky)ˆwpST (α, β, k, θ)e dαdβ (2.48) −∞ −∞ wherew ˆpST is the FT of wpST ,

−2π2(α cos θ+β sin θ)2 −2π2(−α sin θ+β cos θ)2/(kλ(k))2 wˆpST (α, β, k, θ)=e e . (2.49)

Visualization of the resulting 4D transform is difficult and requires excessive memory requirements to store. To simplify visualization and analysis, a 1D magnitude function for each pixel is often extracted from the pST [71]. The function is obtained by finding the value of the pST at a fixed spatial point (x, y), known as the local spectrum Lxy(k, θ)

Lxy(k, θ)=|pST (x, y, kx,ky)|(x,y) (2.50) 33 and then integrating the local spectrum along concentric circles to ﬁnd the rotationally invariant distribution of the center frequencies within a neighborhood of the point (x, y) [71]

Lxy(k)= Lxy(k, θ)dθ. (2.51) θ In practice, the integration in (2.51) is combined with the calculation of the pST in the frequency domain described in (2.48) and (2.49) and rotationally invariant distributions are generated for each pixel in an image (or ROI) resulting in a magnitude pST spectrum, pST (x, y, k) [71]. Combining the calculations in this way reduces the memory requirements since N 3 values are stored instead of N 4 as with the 2D-ST. However, operations on the order of N 4 log N are still required to perform the necessary N 2/2 complex 2D FFTs.

2.5.5 Discrete Orthonormal S-Transform

The DOST is a new transform that addresses the redundancy of the ST by utilizing an orthogonal set of basis functions [3]. The DOST produces a N-point time-frequency representation for a N-point signal. Similar to the CWT, a dyadic sampling scheme can be used where the frequency bandwidth doubles for each increasing “voice” order. However, any other arbitrary sampling of the time-frequency space can also be used, as long as it satisﬁes the conditions of orthogonality [3].

Furthermore, redundancy can be introduced by overlapping bandwidths and oversampling in time as desired. When oversampling in frequency, the spectrum can be windowed by an apodizing function W [n], such as the Gaussian used in the ST, to reduce sensitivity at the edges of the frequency bands.

The DOST of an N-point signal f[t] can be denoted Dp[τ]orDν[τ], depending 34 on whether one indexes the domain according to the frequency order p or centre frequency ν =2p−1 +2p−2. The DOST can be calculated with an arbitrary sampling of the frequency values in the time domain. As described in [3], one derives the basis functions by taking linear combinations of the Fourier complex sinusoids in band-limited subspaces, and applying appropriate phase and frequency shifts. For a − dyadic sampling scheme, we have orders p =0, 1,...,log2 N 1 where the bandwidth of a particular order encompasses frequencies 2p−1 up to 2p. The basis functions are deﬁned for discrete translations τ =0, 1,...,2p−1 − 1 [3]

− −2πi t − τ 2p−1− 1 −2πi t − τ 2p− 1 πiτ ( N 2p−1 )( 2 ) − ( N 2p−1 )( 2 ) ie e e b[p,τ][t]=√ . (2.52) p−1 t − τ 2 2 sin π N 2p−1

The DOST is computed as the inner product between the signal f[t] and the basis functions b[p,τ][t]. Fig. 2.11 shows the real and imaginary portions of the basis functions for orders 0 to 5. Note that this ﬁgure was created to replicate Fig. 2 from [3].

The DOST can be inverted to recover the signal spectrum by applying the forward DFT to each shifted “voice” in order to reverse the spectral partitioning. The signal is recovered by taking the inverse FT of the complete spectrum.

Figure 2.12 shows an example of the DOST applied to the signal from Fig. 2.2.

The DOST shows the same relevant spatial frequency information as the STFT and

ST, but appears “blocky” due to the lack of redundancy in time and frequency.

Note that in this figure the DOST coefficients are replicated in time and frequency for illustration purposes; the DOST of an N-point signal has N coefficients. 35

Figure 2.11: Real (solid) and imaginary (dashed) portions of the DOST basis functions, deﬁned in (2.52), for orders p =0to5andN = 100. This ﬁgure was intended to replicate Fig. 2 from [3]. 36

Figure 2.12: The DOST of the signal from Fig. 2.2. The signal is plotted on the bottom and its FT on the left side. Note that the DOST coefficients are replicated to fill the N × N domain; in actuality the DOST contains only N coefficients for a N-point signal. 37

Chapter 3

Introduction to Magnetic Resonance Imaging

The magnetic properties of nuclei were ﬁrst measured by Rabi in 1930 [72] who won the Nobel Prize in Physics in 1944. The concept of nuclear magnetic resonance

(NMR) was ﬁrst demonstrated by Bloch [73] and Purcell [74] in 1946, who later went on to win the Nobel Prize in Physics for their discovery in 1952. Lauterbur [75] and

Mansﬁeld [76] won the Nobel Prize in Medicine in 2003 for their discoveries relating to the spatial encoding techniques used to create images from NMR as well as the mathematics of image reconstruction. Currently, MR imaging is a routine procedure in major medical centers primarily used to examine soft tissues, particularly the brain. MR imaging is a multi-disciplinary ﬁeld of research with developments from physics, chemistry, biology, medicine and engineering. New developments in hardware and sequence design, image reconstruction techniques and contrast agents has led to the availability of images with a variety of contrasts, rich in information content.

3.1 MR Physics

The basis of NMR is the interaction of a nuclear spin with an externally applied static magnetic ﬁeld B0. Nuclei with an odd number of protons and/or neutrons possess nuclear spin. The primary nucleus of interest in MR imaging is hydrogen — the most abundant element in nature and within the human body. A nuclear spin 38

Figure 3.1: The axis of rotation of a nuclear spin, μ, precesses about B0 at an angular frequency ω ∝ B0, known as the Larmor frequency. can be thought of as a spinning top whose axis of rotation, known as the magnetic moment μ, tends to align itself with B0, as shown in Fig. 3.1. μ precesses about the

B0 axis at an angular frequency ω, known as the Larmor frequency, proportional to

B0,

ω = γB0 (3.1) where γ is the gyromagnetic ratio whose value depends on the nucleus being imaged.

The hydrogen nucleus can align either parallel (lower energy state) or anti- parallel (higher energy state) to the direction of B0. According to Boltzmann statistics [9], the number of spins aligned parallel to the ﬁeld slightly exceeds the number aligned anti-parallel; this leads to a bulk magnetization M aligned parallel to B0. The magnitude of M depends on the proton density (PD), the magnitude of B0 and 39

Figure 3.2: A static B0 field induces a net magnetization M along the z-axis. An applied B1 field along the x-axis rotates M into the transverse plane. The strength and duration of B1 determine the angle of M with respect to the z-axis, α, known as the flip angle. the temperature of the imaged object. The components of M can be described in terms of its components along the x, y and z axes as Mx,My,Mz, respectively. Note that the z-axis is typically oriented along the direction of B0 and is denoted as the longitudinal axis, with Mz called the longitudinal magnetization. The xy-plane is often referred to as the transverse plane and Mxy as the transverse magnetization. In order to detect the bulk magnetization, a magnetic field B1, transverse to B0, is applied to perturb the equilibrium alignment. The B1 field is produced by a transmit coil that emits an radiofrequency (RF) pulse tuned to the Larmor 40

frequency. The application of B1 tips M into the transverse plane at an angle α, known as the ﬂip angle, that depends on the strength and duration of B1 (as shown in Fig. 3.2).

After the pulse is turned off, M relaxes back to the equilibrium magnetization M0, which is aligned along the z-axis, Mz = M,Mx = My =0.AsM relaxes, the rotating magnetization induces an electrical current in a coil of wire placed in the xy-plane (known as the receive coil) according to Faraday’s law of induction, which states that a changing magnetic field induces an electric field [77].

The rate of regrowth of the longitudinal magnetization, assuming that initially M is rotated completely into the transverse plane, Mz(0) = 0, is given by

−t/T 1 Mz(t)=M0(1 − e ) (3.2)

where the T1 relaxation time (T1) is the length of time required for Mz to return to

63.2% of its original value [78] (as shown in Fig. 3.3). While Mz regrows, the induced transverse magnetization dephases back to the equilibrium state where Mxy =0 according to

−t/T 2 Mxy(t)=M0e (3.3) where the T2 relaxation time (T2) is the length of time required for Mxy to decay to 36.7% of its original value.

The relaxation times T1 and T2 are determined by the local environment of the spins being measured. In pure liquids T1 = T2; in the human body T2 < T1 with typical values in the range of 250 < T1 < 2500 ms and 25 < T2 < 250 ms [78].

The signal measured in the receive coil after application of the B1 ﬁeld is proportional to Mxy and the spin density, ρ. The signal undergoes demodulation to remove 41

M 0

−t/T1 M (T ) M (t)=M (1−e ) z 1 z 0

M (0) z 0 500 1000 1500 2000 2500 (a) time (ms)

M (0) xy

−t/T2 Mxy(t)=Mxy(0)e

M (T ) xy 2

0 0 500 1000 1500 2000 2500 (b) time (ms)

Figure 3.3: (a) Simulated regrowth of longitudinal magnetization over t =0–2500 ms with T1 = 500 ms. (b) Simulated decay of transverse magnetization over t =0–2500 ms with T2 = 500 ms. 42 the underlying rapid oscillations at the Larmor frequency. Demodulation consists of multiplying the signal by a sinusoid and a cosinusoid at the Larmor frequency to obtain the real and imaginary channels of the complex signal, respectively. A low-pass ﬁlter1 is applied to each demodulated signal to remove the high-frequency component.

The signal decays very rapidly due to external ﬁeld inhomogeneities. Other imaging methods have been developed that apply a sequence of pulses along various axes to “refocus” the lost magnetization.

Spin-echo sequence

In a spin-echo sequence one applies a π/2-pulse to rotate M into the transverse plane and then a π-pulse after a time TE/2 to refocus the phases of the spins. After a length of time known as the echo time (TE), the spins re-phase and produce an

“echo”. This process can be repeated, but the magnitude of each subsequent echo decays exponentially at the rate of T2.

The transverse magnetization at an arbitrary echo is given by [9]

−TR/T1 −TE/T2 Mxy = M0(1 − e )e . (3.4)

Inversion recovery sequence

An inversion-recovery sequence begins by applying a π pulse to invert M to the negative z-axis. The longitudinal magnetization relaxes at the rate

−TI/T1 Mz = M0[1 − 2e ] (3.5) where the inversion time (TI) is the time when a π/2 pulse is applied.

1Low- and high-pass ﬁlters are discussed in more detail in Chapter 4 43

If a sample contains several substances with diﬀerent values of T1, the value of

TI in an imaging sequence can be chosen such that the longitudinal magnetization of a particular substance is zero when the π/2 pulse is applied. As a result, the signal from that component will be zero, and it is said to be “nulled”. For example, the fluid-attenuated inversion recovery (FLAIR) sequence [79] nulls the signal component from fluids such as cerebrospinal fluid (CSF) to help identify lesions that may be located near the ventricles, such as in MS.

3.2 Generating an Image

The signal measured from an NMR experiment represents the bulk properties of the entire sample within the magnetic ﬁeld. In order to create an image, one must introduce a spatially varying magnetic ﬁeld; the signal measurements can then be correlated with the spatial locations of the various sources.

The magnetic ﬁeld is varied through the application of a linear magnetic ﬁeld gradient G in three orthogonal directions Gx,Gy,Gz whose amplitudes are small in relation to B0. One gradient is typically used to perform slice selection. The slice select gradient is applied perpendicular to the desired slice, for example along the z-axis, making the resonant frequency proportional to the position along the axis.

The bandwidth of frequencies contained within the B1 pulse can then be selected to excite only those spins within a slice thickness Δz

Δω = γGzΔz. (3.6)

Once a slice of spins is excited, a phase oﬀset can be applied to the spins using a phase encoding gradient, for example applied along the y-axis. After application of 44

the B1 pulse, a frequency encoding gradient can be applied to say the x-axis while the signal is being recorded, a process known as readout. As a result, the frequencies in the recorded signal are linearly related to the spatial locations of the spins.

3.2.1 Image Reconstruction

The goal of imaging is to reconstruct the spin density ρ(x, y) of a sample by taking the inverse 2D-FT of its signal measurements s(kx,ky), ∞ 2πi(kxx+kyy) ρ(x, y)=c s(kx,ky)e dkxdky (3.7) −∞ where c is a proportionality constant.

The application of phase and frequency encoding gradients moves one through the data collection domain, known as k-space. The distance between samples along the readout direction (usually denoted as the horizontal or kx-direction) depends on the sampling time of the digitizer (Δt) and the gradient amplitude

γ k = G mΔt, m = −N/2, −N/2+1,...,N/2 − 1. (3.8) x 2π x

Each signal readout ﬁlls one line of k-space. This process is repeated with a diﬀerent phase-encoding gradient after a time known as the repetition time (TR). The distance one moves through the ky direction after each TR depends on the amplitude and duration (τ) of the phase encoding gradient

γ k = nΔG τ, n = −N/2, −N/2+1,...,N/2 − 1. (3.9) y 2π y

The distance between samples in k-space and image space are related according 45 to:

1 1 Δkx = = (3.10) NxΔx FOVx 1 1 Δky = = (3.11) NyΔy FOVy where Nx,Ny are the number of samples and FOVx,FOVy are the ﬁeld-of-view in the x- and y-directions, respectively. A diagram of k-space is shown in Fig. 3.4.

In order to get an accurate reconstruction, “good” coverage of k-space is required. The Nyquist–Shannon sampling theorem [80, 81] dictates that the signal must be sampled at least twice the rate of the maximum frequency in the signal to avoid imaging artifacts. The extent of sampling in k-space determines the image resolution, Δx, Δy. Sampling further out in k-space improves the spatial resolution; however, the measurable resolution is limited by the T2 decay of the signal over time. Various trajectories have been proposed to reduce the time required to ﬁll k-space, and numerous strategies exist to reconstruct images from partially ﬁlled k-space data.

3.2.2 Neurological MR Imaging

Neurological MR images are typically taken in one of three orthogonal, anatomical orientations: axial, sagittal or coronal, illustrated in Fig. 3.5. Axial images are the most common, and the only type that are presented in this thesis (examples are shown in Figures 3.6, 3.7 and 3.8). Axial images are typically displayed such that the left side of the image corresponds to the right side of the patient and the right side of the image to the left side of the patient; the top of the image corresponds to the front side of the patient (anterior) and the the bottom of the image to the 46 max y k y k

Δky

Δkx phase encoding directionphase encoding max y

-k max 0 max -kx kx readout direction kx

Figure 3.4: A diagram of acquisition space for MR imaging, known as k-space. The max max maximum frequency in k-space is kx = NxΔx/2 and ky = NyΔy/2. The center frequency (kx,ky)=(0, 0) is known as the DC component. The spacing between samples in k-space is given by Δkx =1/F OVx and Δky =1/F OVy. In a traditional acquisition scheme, one line of k-space is acquired each TR. 47 back side of the patient (posterior). An example of an axial PD-weighted image, with labels indicating the location of some neuroanatomical structures, is shown in

Fig. 3.6. The reader is directed to [82] for an excellent, detailed, interactive atlas of brain anatomy.

3.3 MR Sequences and Contrast Mechanisms

The various parameters described in this section, including T1, T2, TR and TE, can be manipulated to generate MR images with varying contrast. Note that many contrast mechanisms are available in MR imaging including ﬂow, magnetization transfer and diﬀusion. We only consider contrasts obtained by the most basic weightings of

PD, T1 and T2. Figure 3.7 shows examples of each of the MR contrasts.

3.3.1 T2 Weighting

T2 weighted images are often a sensitive indicator of disease because most disease states are characterized by an elevated T2 [9]. T2 weighting can be obtained by using spin-echo sequences with a long TR to minimize contributions from T1 weighting.

For example, if we consider equation (3.4) with TR >> T1, e−TR/T1 → 0 and the magnetization is aﬀected only by the sample’s T2 dependence.

3.3.2 T1 Weighting

Since normal tissues in the brain typically have diﬀerent T1 values, T1-weighted imaging is often used for anatomical imaging. To enhance T1 weighting, T2 weighting is suppressed by using a short TE. For example, in equation (3.4) when TE <<

T2, e−TE/T2 → 1 and the magnetization depends solely on the T1 value. 48

Figure 3.5: Illustration of the three most common MR imaging planes: axial, sagittal and coronal. 49

white RLmatter ventricles

gray matter

Figure 3.6: An example of a normal PD-weighted axial MR image. R and L refer to the right and left sides of the patient, respectively and A and P to anterior (front) and posterior (back), respectively. 50

3.3.3 Proton Density Weighting

PD contrast is present in all images to varying degrees. To obtain images primarily weighted by PD, the T1 and T2 dependence of the MR signal must be minimized.

This can be done by choosing a long TR along with a relatively short TE.

3.3.4 Contrast Enhancement

External agents can be introduced into tissues to change their relaxation times in order to improve contrast between tissues. The most commonly used contrast agent is gadopentetate dimeglumine [83] (Gd-DTPA, Magnevist, Bayer Schering

Pharma AG, Germany), commonly referred to as gadolinium. Following intravenous injection, Gd-DTPA causes a reduction in the T1 time of a tissue. If images are acquired with TR >> T1, then the signal is proportional to TR/T1 and the image is primarily T1-weighted. Tissues with shorter T1 times have higher signal than those with longer T1 times; therefore, the contrast agent causes a brightening in the targeted tissue, improving conspicuity of lesions, as shown in Fig. 3.8. 51

(a) (b)

(c)

Figure 3.7: MR images of a MS patient obtained using a spin-echo sequence with the following weightings (TR/TE): (a) T2 (2716/80 ms), (b) T1 (650/8 ms) and (c) PD (2716/30 ms). Note the good contrast between lesions and surrounding white matter in (a). Examples of MS lesions are indicated by white arrows. 52

(a)

(b)

Figure 3.8: Examples of T1-weighted images taken (a) before and (b) after the injection of gadolinium contrast agent (Gd-DTPA). Note the improvement in the contrast between the large brain tumor (arrow) and the normal brain tissue in (b) along with the detail evident within the tumor itself. 53

Chapter 4

Applications of Space-Frequency Analysis to Medical Image

Processing

This chapter describes three major applications of space-frequency analysis to MR imaging: ﬁltering, texture analysis and classiﬁcation.

4.1 Filtering

4.1.1 Image-Space Filtering

Gaussian Filtering

Gaussian ﬁltering consists of convolving an image f0(x, y) with a Gaussian function g(x, y, σ) to “smooth” the image

f(x, y, σ)=f0(x, y) ∗ g(x, y, σ) (4.1) where f is the ﬁltered image, σ determines the width of the Gaussian and ∗ is the convolution operator.

However, as shown in Fig. 4.1, Gaussian blurring is isotropic and does not “respect” the natural boundaries of objects [84]. In other words, if one was aiming to smooth an MR image of the interior of a structure such as the brain, Gaussian blurring would combine signal from the edge of the brain with signal from outside the head, leading to distortion. Furthermore, it is not possible to selectively smooth 54 areas that are naturally more homogeneous and restrict smoothing in areas that contain detail.

Anisotropic Diﬀusion Filtering

Anisotropic diffusion filtering addresses this distortion by attempting to define, and respect, natural boundaries in an image. Originally introduced by Perona and Malik in 1990 [84], anisotropic diffusion filtering attempts to encourage smoothing within a region and discourage smoothing across boundaries.

The locations of edges are estimated by calculating the gradient in each direction

(north, south, east and west) of each pixel. A ﬂow image is then computed, using a diﬀusion function that is a monotonically decreasing function of the image gradient magnitude,

Φ(x, y)=c(x, y)∇f(x, y). (4.2)

An example of a diﬀusion function is [84] |∇f(x, y)| 2 c(x, y) = exp − (4.3) κ where κ is a parameter, known as the “diﬀusion constant” chosen according to the noise level of the image and the desired edge strength [85].

The ﬁltered image for the iteration t is generated by taking the derivative of the

ﬂow image in each image direction,

∂ f(x, y, t)=∇·Φ(x, y, t) ∂t ∂ ∂ ∂ ∂ = c(x, y, t) f(x, y, t) + c(x, y, t) f(x, y, t) . (4.4) ∂x ∂x ∂y ∂y

If the gradient value is large, this is interpreted as an edge and diffusion filtering is stopped. The resulting filtered image has the property that small discontinuities are 55

(a)

(b) (c)

Figure 4.1: (a) A 256 × 256 pixel MR image. (b) The same image after blurring with a Gaussian g of width σ =1.28 pixels and (c) after applying g with σ =2.56 pixels. 56 blurred and edges are sharpened [85]. Figure 4.2 shows examples of the anisotropic diﬀusion ﬁlter from MIPAV [86] applied to an MR image.

Anisotropic diﬀusion ﬁltering has successfully been applied to MR imaging [85,

87]. Limitations of anisotropic diffusion filtering include the need to manually select a value of κ and a number of iterations. Since the value of κ is based primarily on the noise level of the image, this value must be known (or measured); otherwise, various values must be tried and compared to determine the optimal level of filtering [85].

Furthermore, computation time can be lengthy with large data sets, particularly if a large number of iterations are required [88].

4.1.2 Frequency-Domain Filtering

Many spatial ﬁlters can be applied more eﬃciently in the frequency domain. These implementations take advantage of the Fourier convolution theorem, given in (2.35).

Frequency domain filtering is usually more efficient because while a convolution requires a number of operations proportional to K × N, where K is the number of coefficients in the filter, the FFT requires only a number of operations proportional to N log N [11]. Therefore, it is generally faster to perform two 2D FTs and a multiplication than it is to perform a spatial convolution with a filter — particularly when the filter size is large.

For example, Gaussian blurring can be implemented as a multiplication of the

FT of the image with the FT of the Gaussian ﬁlter,

−1 f(x, y, σ)=F {F [f0(x, y)] · F [g(x, y, σ)]} (4.5)

The result is the application of a low-pass ﬁlter, which attenuates high frequencies 57

(a)

(b) (c)

Figure 4.2: Anisotropic diﬀusion ﬁltering of the image shown in Fig. 4.1(a) with (a) 10 iterations at κ = 28, (b) 50 iterations at κ = 28 and (c) 10 iterations at κ =10, 000. 58

(that correspond to detail in the image) and the preservation of low frequencies (that correspond to large-scale structures). For the case of Gaussian blurring, this equation can be further simplified because the FT of a Gaussian is another Gaussian. A narrow Gaussian filter used in the image domain corresponds to a wide Gaussian in the frequency domain. As a result, filtering with a wide Gaussian in the image domain corresponds to the attenuation of more frequencies in the frequency domain, and therefore more blurring. A high-pass filter would be designed in the oppo- site fashion, to attenuate low frequencies and maintain (or boost) high frequencies.

However, in real images noise tends to dominate high frequencies, therefore such

filters may also boost the noise level in the filtered images. A band-pass filter could also be used, which selectively removes some frequencies while “passing” those in a particular band (known as the passband). Figure 4.3 shows examples of low-pass, high-pass and band-pass filters in 1D and Fig. 4.4 shows the results when applied to a 1D signal.

Frequency-domain filtering suffers from the limitation that, like spatial Gaussian blurring, it is inherently isotropic in space. Since the FT contains information from the entire image, any multiplicative filters applied to the FT are applied to the entire image. As a result, trade-offs must be made between noise reduction and the removal of fine detail in the image.

4.1.3 Space-Frequency Filtering

Just as multiplicative ﬁlters can be applied to the FT of a signal or image, multi- pliers can be applied to the time-frequency domain as well. The idea is to suppress coeﬃcients that represent noise and retain those that represent the underlying signal 59

1.0 low-pass high-pass 0.8 band-pass

0.6

0.4 filter amplitude 0.2

0.0 0.0 0.1 0.2 0.3 0.4 0.5 frequency

Figure 4.3: Examples of low-pass, high-pass and band-pass ﬁlters. 60

1 30

20 0 10

-1 0 (a) 1 30

20 0 10

-1 0 (b) 1 30

20 0 10

-1 0 (c)

(d)

Figure 4.4: (a) A signal (left column) that contains a sinuosoid whose frequency increases with time along with its FT (right column). The results after applying the (b) low-pass, (c) high-pass and (d) band-pass filters from Fig. 4.3. The filters are shown as dotted lines in the right column (multiplied by 30 for display). 61 or image. The first application of wavelet filtering was by Weaver et al in 1991 [89], who performed thresholding of wavelet coefficients to reduce noise in MR images.

This work was also one of the ﬁrst applications of wavelets to medical imaging. A review of wavelet denoising in MR imaging of the brain can be found in [90].

Filtering can be performed using the DWT by applying a thresholding operator to the wavelet coefficients. Hard thresholding sets to zero those coefficients whose magnitude is less than the threshold, T ; coefficients whose magnitude is greater than

T remain unchanged [2]. Soft thresholding (also known as wavelet shrinkage) also sets to zero those coefficients whose magnitude is less T , but positive coefficients with a value x greater or equal to T are set to x − T and negative coefficients with a value less than or equal to T are set to x + T [91]. The image is then reconstructed by inverting the DWT. Thresholds can be a global value, or level-dependent, or spatially adaptive [2]. More sophisticated noise reduction methods include affine (or

ﬁrm) thresholding [92] and stochastic methods using Markov random ﬁeld models

[93].

Filtering of 1D signals has also been performed with the ST. Previous studies have shown that manually applying simple box-shaped or median ﬁlters to the ST of fMRI time signals can help remove temporal artifacts [49–51]. An example of

1D-ST filtering with a multiplicative box-shaped filter is shown in Fig. 4.5. To our knowledge, there are no known cases of filtering images using the 2D-ST. The computational and storage requirements are too great to allow effective ST-based image filtering. 62

0.5

0.4

0.3

0.2 frequency

0.1

0.0

0 amplitude -1 0 20 40 60 80 100 120 (a) time

0.5

0.4

0.3

0.2 frequency

0.1

0.0

0 amplitude -1 0 20 40 60 80 100 120 (b) time

Figure 4.5: (a) An example of a signal that contains a low-frequency component for the ﬁrst half of the signal, a mid-range frequency for the second half and a high- frequency burst that occurs from 0.12 to 0.27 s (bottom) and its ST (top). (b) The result after applying a multiplicative box ﬁlter to the ST (with value zero in the area indicated by the white box and one elsewhere) and inverting. 63

4.2 Texture Analysis

Texture is an innate property of virtually all surfaces [94]. Image texture can be deﬁned as the spatial relationship of pixel values in an image region [95] or the local characteristic pattern of image intensity. Image texture is often described qualitatively in terms of properties such as ﬁneness, coarseness, smoothness, granulation, randomness, irregularity, etc. [4]. However, the human visual system is only capable of distinguishing a limited level of detail within an image [96].

The purpose of texture analysis is to characterize a pattern in terms of quantitative, measurable features, known as texture features. These features can be used to discriminate between diﬀerent patterns. Numerous methods exist to characterize or quantify textures. A review of texture analysis methods is given in [97]. Castellano et al provide a good review of texture analysis of medical images in [98]. Here we describe some of the basic and most commonly used texture analysis methods.

4.2.1 Statistical Methods

An early and still commonly used method of measuring texture in grayscale images is based on statistical analysis of the spatial distribution and spatial dependence of the grey level values between pairs of pixels in the image, known as cooccurrence matrices [4]. This method has been applied to MR images to distinguish between normal and pathological tissues of the human brain [99, 100], the liver [101] and spinal cord [102]. In 2001, Kovalev et al proposed the use of 3D cooccurence matrices for texture analysis of brain MR images [103].

To compute a gray level spatial dependence matrix, one counts the number of 64

Grey level 0 0 1 1 0 1 2 3 0 0 1 1 0 (0,0) (0,1) (0,2) (0,3) 1 (1,0) (1,1) (1,2) (1,3) 0 2 2 2 2 (2,0) (2,1) (2,2) (2,3) 2 2 3 3 level Grey 3 (3,0) (3,1) (3,2) (3,3)

Figure 4.6: Example of a 4 × 4 image with four grey-level values ranging from 0 to 3 (left). The structure of the grey-level spatial dependence matrix (right). Figure adapted from [4]. times that particular intensity values occur at speciﬁed distances and angles. Figure

4.6 illustrates a sample 4 × 4 image consisting of four gray level values ranging from

0 to 3. That example, adapted from [4], uses a distance value of d = 1 and angles of

θ =0°,45°,90°and 135°. For example, the value in the grey level spatial dependence matrix marked (0,2) would be calculated for the P135◦ matrix by counting the number of times 0 and 2 appear next to each other at a 135° angle. Note that the result of

3, indicated by the shaded boxes in Fig. 4.6, would be the same as the result for the location marked (2,0). Note that for the 0° matrix, both left-right and right-left arrangements are counted. The resulting grey level spatial dependence matrices for the image in Figure 4.6 are given by ⎛ ⎞ ⎛ ⎞ ⎜4210⎟ ⎜4100⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜2400⎟ ⎜1220⎟ ◦ ⎜ ⎟ ◦ ⎜ ⎟ P0 = ⎜ ⎟ ,P45 = ⎜ ⎟ , ⎜1061⎟ ⎜0241⎟ ⎝ ⎠ ⎝ ⎠ 0012 0010 65

⎛ ⎞ ⎛ ⎞ ⎜6020⎟ ⎜2130⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜0420⎟ ⎜1210⎟ ◦ ⎜ ⎟ ◦ ⎜ ⎟ P90 = ⎜ ⎟ ,P135 = ⎜ ⎟ . (4.6) ⎜2222⎟ ⎜3102⎟ ⎝ ⎠ ⎝ ⎠ 0020 0020 Once the spatial dependence matrices are calculated, texture features can be computed by performing statistical tests, such as measuring the entropy or correlation of the values.

These cooccurrence features can be lengthy to compute, particularly if one wishes to measure multiple distances or extend the technique to three dimensions. As a result, a single value of d (often d = 1) is typically used. This restricts the textures that can be measured; if d is small, mainly high frequency (pixel-to-pixel) variations are computed. Furthermore, the number of grey levels often need to be reduced in order to make the texture matrices more manageable in size, resulting in a potential loss of texture information.

4.2.2 Multiscale Texture Features

Spatial frequencies can be used to quantify texture because ﬁne textures are rich in high spatial frequencies while coarse textures are rich in low spatial frequencies [4].

The FT is not very useful in texture analysis because of the lack of spatial localization. Gabor ﬁlters can provide better spatial localization [104], but a single window width must be chosen; as a result texture features tend to be precise about either the spectral content or the spatial localization of a texture — not both. Therefore, texture information obtained from Gabor ﬁlters tends to be limited because there is usually not a single window width that provides optimal localization of real textures 66 within an image.

A multiscale transform allows variable spatial and spectral resolution to represent textures at the most suitable scale. The coeﬃcients obtained from a space-frequency or DWT can be used directly, or simple statistics can be performed on the ﬁltered image, to obtain a rich set of texture features. Excellent overviews of the use of space-frequency transforms and the DWT for texture discrimination can be found in [105] and [2], respectively.

Rotationally invariant features are often desired, so that small rotations of the imaged object do not result in large changes of the measured features. The reader is encouraged to refer to [106] for a discussion of texture analysis methods invariant to translations and rotations. Two examples of rotationally invariant multiscale texture features are described below.

Rotationally Invariant Gabor Features

Haley proposed using Gabor ﬁlters to obtain rotation-invariant texture features in

1995 [107]. His method consists of convolving the image with Gabor ﬁlters in the radial direction, at a number of angles of orientation. Texture features can be derived from from the sum over all of the angles analyzed. While texture analysis approaches based on this method have been used successfully [108–110], they require the selection and computation of speciﬁc orientation angles.

Another invariant method, proposed by Porter and Canagarajah in 1997 [5,

111], uses a circularly symmetric window to achieve rotationally invariant Gabor features eliminating the need to select particular orientation angles for the ﬁlters.

The circularly symmetric Gabor ﬁlter, created by modulating a complex sinusoidal 67 with a Gaussian function, is given by » – √ x2+y2 1 2 2 − 2 h (x, y)= e−2πiF x +y e 2σ (4.7) F 2πσ2 where σ is a “scale factor” determined by the center radial frequency F such that

σ = μ/F where μ is a constant1.

In [5], the values of radial frequency F are selected to space ﬁlters across the

Fourier domain to achieve optimum coverage, avoiding the very low frequencies since they provide information about only very large-scale features. For a 16 × 16 pixel image, F =2.0, 3.17, 5.04 and 8.0 cycles/image.

The rotationally invariant Gabor filters proposed by Porter and Canagarajah can be used to obtain texture features corresponding to a centre frequency F by calculating the average pixel magnitude of each filtered image 1 n m n m P (F )= hˆ , · fˆ , (4.8) G NM F N M N M n m ˆ ˆ where hF is the FT of the circularly symmetric Gabor filter hF (x, y), f(n/N, m/M) is the FT of the image f(x, y) and M and N are the dimensions of the transformed image (typically M = N).

Rotationally Invariant Discrete Wavelet Analysis

A rotationally-invariant wavelet domain is obtained in [5] using a method similar to that of the rotationally invariant Gabor features. The horizontal and vertical wavelet coeﬃcients for each decomposition level are averaged and the diagonal coeﬃcients are omitted. A vector of invariant wavelet texture features is then obtained by

1Note that appropriate values of μ are not given in [5]. As a result, we use a value of μ =1in our implementation of the invariant GT. 68

(a)

Figure 4.7: (a) Three levels (n = 3) of a wavelet decomposition of an image. Average coefficients are denoted by “A” and detail coefficients: H: horizontal, V: vertical, D: diagonal. (b) For rotationally-invariant coefficients, V & H coefficients are averaged to obtain Wn for each level and the average coefficients, while D coefficients (shaded grey in (a)) are omitted. (c) The average of each Wn is calculated to obtain the rotationally-invariant coefficients. computing the average value of each resulting band,

1 P (n)= |W (i, j)| (4.9) W MN n i j where i and j are the rows and columns of the average horizontal and vertical coeﬃcients of wavelet level n, Wn (Fig. 4.7).

Rotationally Invariant S-Transform Features

The rotationally invariant magnitude spectrum Lxy(k) from the pST, described in (2.51), can be used as a texture feature vector for each point within an image or

ROI. The magnitude of each spectral component describes the power of that spatial frequency at the point (x, y). Spectra can be compared point-by-point between two classes of images. For example, Brown et al [57] showed that the average 69 pST spectrum from a sample of chemoresistant oligodendroglioma brain tumors had components that were diﬀerent than that from a sample of chemosensitive tumors.

The relevant portion of the local spectrum can be summarized in a single variable by integrating the area under the spectrum within a particular frequency range. For example, the sum of low frequency energy (LFE) has been utilized to characterize lesion evolution in MS of humans [54,55] and animals [52].

4.3 Classiﬁcation

The goal of classification is to propose class membership for sets of elements based on shared qualities and common characteristics. For example, one might wish to use a classifier to determine if tissue is normal or abnormal based on its appearance, clinical characteristics of the patient, the result of diagnostic tests, etc. Texture classification uses image texture features to define which class an image region, or pixel, belongs to. For example, in medical applications one might want to predict whether a tissue is “normal” or “abnormal” or whether it is disease type A or B based on the local pattern of signal intensity in an MR image.

Texture features can be plotted in a multi-dimensional space known as a “feature space”. Figure 4.8 illustrates an example of a 2D feature space, where the x and y axes correspond to the two features (f1 and f2) being examined, and each point represents the values of the features for a particular data sample. The features can be thought of as pixel intensity in two diﬀerent image contrasts or texture values such as low-frequency and high-frequency amplitude. Given a set of observations with known classiﬁcation (known as the training set), one makes a decision as to 70

f1 f1

f (a) 2 (b) f2

f1 ?

f2 (c)

Figure 4.8: The 2D feature space of two features denoted f1 and f2. (a) An example of a linear discriminant function that separates the two classes and (b) a quadratic discriminant function. (c) In k-NN classiﬁcation an unknown sample (denoted by the circle with question mark) is assigned to the majority class of its k nearest neighbors. If k = 2 the object is assigned to the class represented by the triangles; if k = 5 it is assigned to the class represented by squares. which class an object of unknown classiﬁcation (the test set) belongs.

There are two main types of classiﬁers: parametric and non-parametric. Para- metric classiﬁers estimate an analytical form of a decision boundary from known cases [112]. For example, a discriminant function [113] can be created from a linear or quadratic combination of features that best separate two or more classes of the data (known as discriminant analysis). Figure 4.8 (a) and (b) show examples of 71 linear and quadratic discriminant functions, respectively. More complex decision boundaries can be obtained using techniques such as support vector machines [114]; however, these are beyond the scope of this thesis.

Non-parametric classiﬁers create a decision boundary based on the observations that typically cannot be expressed analytically and must be computed numerically.

For example, an object can be assigned to the most common class of its k nearest neighbors (known as k-NN classification), as shown in Fig. 4.8 (c). If k = 1 then the object is simply assigned to the class of its nearest neighbor. Non-parametric classifiers can become complex for large data sets. Also, it may be more difficult to quantify the classification criteria when using a non-parametric classifier.

4.3.1 Evaluating Classiﬁer Performance

The performance of a classiﬁer can be tested using cross-validation, where the data is partitioned into training and testing subsets. The subsets can be mutually exclusive

(known as the holdout method), as originally proposed by Mosier in 1951 [115], but this method reduces the amount of data available for training the classiﬁcation model [116]. Geisser [117] and Stone [118] introduced an improved method in 1974, known as K-fold cross-validation, that provides a better estimate of the error [116].

In K-fold cross-validation, the original data sample is partitioned into K mutually exclusive subsets (known as folds), where one fold is used for training and each of the remaining for testing. The process is repeated until each fold has been used as the test group. The accuracy of the model can be estimated as the number of test cases correctly classiﬁed divided by the number of tests performed. If K is equal to the number of samples, a single observation from the original sample is used as the 72 test set and the remaining observations are used as the training set; this process is known as leave-one-out validation or complete cross-validation [119].

Using a method such as leave-one-out cross validation, one can determine the accuracy of a classifier as # of cases correctly classified accuracy = . (4.10) total # of cases classified However, for medical tests it is often more important to understand which cases are being correctly classified and which are incorrectly classified. To get a better understanding of classifier performance, one can construct a contingency table, as shown in Fig. 4.9. Here, the predicted outcome (positive or negative) is compared to the actual outcome (positive or negative). True positives and true negatives are those cases where the classifier is correct in determining the positive or negative outcome, respectively. False positives occur when a case is classified as being positive when the true outcome is negative (known as a type I error); false negatives occur when a case is classified as being negative when the true outcome is positive (known as a type II error). In essence, false positives mean that a patient has to undergo unnecessary further testing and a false negative is a case of disease that is missed by the test.

The positive predictive value (PPV) measures the proportion of patients with positive test results that are correctly identiﬁed. The negative predictive value

(NPV) value measures the proportion of patients with negative results that are correctly diagnosed as # of true positives PPV = (4.11) # of true positives + # of false positives # of true negatives NPV = . (4.12) # of true negatives + # of false negatives 73

actual outcome positive negative

predicted positive true positive false positive PPV outcome negative false negative true negative NPV sensitivity specificity

Figure 4.9: Example of a contingency table showing how sensitivity, speciﬁcity, PPV and NPV are calculated.

Sensitivity measures the ability of a classiﬁer to positively recognize those patients that are positive for a condition,

# of true positives sensitivity = . (4.13) # of true positives + # of false negatives

A negative result on a highly sensitive test can rule out the presence of disease.

The sensitivity of a test could be made 100% in the trivial case where a classiﬁer identiﬁes everyone as being positive for the disease.

Speciﬁcity measures the ability of a classiﬁer to correctly identify those patients that are negative for the condition as such,

# of true negatives speciﬁcity = . (4.14) # of true negatives + # of false positives

A positive result on a highly specific test can confirm the presence of a disease. In the trivial case, specificity can be 100% if all test results come back negative. The relative importance of sensitivity versus specificity depends on the particular test, and can be evaluated using receiver operating characteristic (ROC) analysis. 74

ROC analysis can be used to compare the performance of various classifiers (each denoted as a filled circle in Fig. 4.10). An ROC plot can be generated by plotting the sensitivity of a classifier along the y-axis and (1 - specificity) along the x-axis, as shown in Fig. 4.10. A completely random classifier would generate a point along the diagonal dashed line from the lower left to the upper right corner, known as the line of no discrimination. The further above the line of no discrimination, the better a classifier’s performance. If a classifier were to fall below the line of no discrimination, then all of its predictions could be reversed to yield predictive power. 75

best

better

Figure 4.10: A ROC plot showing the performance of various classifiers. The best possible classifier would be in the upper left corner. The dashed line indicates performance of a random guess (the line of no discrimination). The predictions of classifiers worse than a random guess can be inverted to improve their performance (dashed black arrow). 76

Chapter 5

The Circular S-transform

This chapter includes material presented as a poster and as an abstract published in the proceedings of the 16th International Society of Magnetic Resonance in Medicine

(ISMRM), in Toronto, ON (May 2008) [120]. I was awarded an Educational Stipend based on this work to attend the ISMRM meeting. The transform described in this chapter was the basis of an intellectual property transfer to Calgary Scientiﬁc Inc., who holds a provisional patent on the technology.

As the primary author, I was responsible for writing and editing these publications. I developed the theory of the cST and designed and carried out the analysis presented in this chapter. Ross Mitchell was the senior author on the above publications and provided overall guidance in the experimentation, and contributed to and edited the publications.

5.1 Introduction

The diagnosis and monitoring of many neurological diseases, such as multiple sclerosis (MS) and brain cancer, rely on MR imaging. Radiologists are trained to identify subtle changes in tissue appearance, or texture, that may be indicative of the presence of pathology. Tissue texture can be mathematically described using a space-frequency transform, which identiﬁes the local frequency content of every image pixel. 77

Previous studies have demonstrated that local spectral analysis using the pST

[71] can help discriminate between between tissues that have different textural appearances on MR images. For example, differences in pST spectra have been observed between normal appearing white matter (NAWM) and active as well as inactive MS lesions on MR images [7, 52, 53, 55]. The area under the low- frequency portion of the spectrum (LFE) has been used to quantify the spectral differences [7,52,55].

5.1.1 Limitations of the pST

The pST requires a large number of computations, resulting in hours of computation time for typical MR images [56]. Therefore, analysis is usually limited to a small

ROI, limiting the amount of spatial information that can be analyzed from an image.

Since diseases like MS are potentially diﬀuse in nature, it would be desirable to have a technique that could analyze large images, or even volumes. Furthermore, reducing the image to a small ROI has the undesirable eﬀect of reducing the resulting spectral resolution by a factor of n/N where n is the size of the ROI and N is the size of the image. Since the pST is calculated by summing the magnitude spectra in radial bands, the phase information is discarded and the resulting pST domain is not invertible. Also, the LFE value depends on the low-frequency information from the image, obtained by analyzing a large extent of the image. If a small ROI is used,

I expect that LFE information cannot be measured accurately. Therefore, I seek to understand the eﬀect of ROI size on LFE and whether a minimum ROI size is required for accurate LFE measurement. 78

5.1.2 Objectives

Here I introduce a novel spectral analysis technique known as the cST that uses circularly symmetric windows in the frequency domain to reduce the number of computations by a factor of N. I average the contribution of each complex spectral contribution, thereby producing a complex, and fully invertible, 3D local frequency domain. Furthermore, the new method can provide complete spectral resolution.

My goals are to reduce the computation time in order to be able to analyze entire images, or potentially even large imaging volumes, to avoid issues related to

ROI analysis. Analysis of larger data sets could potentially help to identify early changes in normal-appearing tissue in diﬀuse diseases such as MS. The resulting improvement in frequency resolution when moving from small ROIs to large images would provide better sensitivity to changes in the local frequency distribution and potentially allow one to measure changes in the shape of the local spectrum as well as its amplitude. This would provide more information that could help to quantify disease progression or distinguish between disease states.

In this chapter, I examine the limitations of the pST by examining the eﬀect of using small ROIs to examine LFE in images of the brains of MS patients. I examine various ROI sizes and LFE bandwidths, and compare the spectra obtained from entire images using the cST. Finally, I compare the time required to compute both the pST and cST for various image sizes.

5.1.3 Theory

The ST determines the local spatial frequency content of an image by applying a series of frequency-dependent Gaussian localizing windows, w, scaled to the frequency 79 being examined, to the image, f(x, y), in the Fourier domain [61]

∞ ∞ ˆ 2πi(αx+βy) S(x, y, kx,ky)= f(α + kx,β+ ky)ˆw(α, β, kx,ky)e dαdβ (5.1) −∞ −∞ where x and y are spatial variables and α and β are frequency variables. The windows are centered on the origin and the FT of the image is shifted to the frequency of interest, (kx,ky). The result is a 4D Fourier-like multiscale transform that describes the horizontal

−1 and vertical spatial frequencies, kx and ky, respectively, in units of cycles/cm (cm ), at every point in the image. The reader is encouraged to refer to [41] for a discussion on visualization and analysis of 4D ST spectra.

The pST creates a rotationally invariant space-frequency domain by using a 2 2 window whose shape depends on its radial distance from the origin, k = kx + ky, and which rotates with the frequency orientation, θ = arctan(ky/kx) [71] h i 2 2 − 2 α cos θ+β sin θ −α sin θ+β cos θ 2π ( k ) +( kλ ) wˆpST (α, β, k, θ)=e (5.2) where λ is used to trade oﬀ between spatial and frequency resolution. This produces an ST that depends on spatial variables x and y, radial distance k and orientation

θ,

∞ ∞ ˆ 2πi(αx+βy) pST (x, y, k, θ)= f(α + k cos θ, β + k sin θ)ˆwpST (α, β, k, θ)e dαdβ. −∞ −∞ (5.3)

The pST is averaged over θ to produce a 1D, rotationally invariant spectrum for each pixel in the image [71]. Local spectra are computed by integrating the magnitude ST, indexed over k and θ 2π pST (x, y, k)= |pST (x, y, k, θ)|dθ. (5.4) 0 80

Extract region of interest (ROI) Fourier domain of the ROI

Perform 2D-FFT k

1D local magnitude Fourier spectrum: “texture curve” at each pixel, pST(x,y,k) For all k: For all : • multiply by Gaussian window pST (, , k, ) • perform inverse 2D-FFT

amplitude • take magnitude • add to pST(x,y,k) spatial frequency (1/cm)

Figure 5.1: A ﬂow diagram illustrating the process of calculating the pST on an ROI (shown as a box on the image).

Equation (5.4) requires calculation and storage of the entire 4D space-frequency domain. In practice, the integration over θ is computed while the pST is calculated in the frequency domain, as described in Section 2.5.4. However, operations on the order of N 4 log N are still required to perform the necessary N 2/2 complex 2D

FFTs. Also, the magnitude operation in (5.4) means that the pST spectra cannot be inverted to recover the original image. The ﬂow diagram in Fig. 5.1 illustrates the process of calculating the pST. 81

5.1.4 The Circular S-Transform

To address these issues with the pST, I present an alternative approach to calculating local spectra based on the ST. I begin by rewriting the ST from equation (2.38) using the standard Gaussian window [61], but replacing α with α − kx and β with

β − ky: » – ∞ ∞ 2 (β−k )2 − 2 (α−kx) y 2π 2 + 2 − − ˆ kx ky 2πi[(α kx)x+(β ky)y] S(x, y, kx,ky)= f(α, β)e e dαdβ −∞ −∞ » – “ ”2 ∞ ∞ 2 β −2π2 ( α −1) + −1 = e−2πi(kxx+kyy) fˆ(α, β)e kx ky −∞ −∞ × e2πi(αx+βy)dαdβ. (5.5)

2πi(kxx+kyy) I then multiply by the phase term e and use the relations kx = k cos θ and ky = k sin θ to express the S in terms of the radial distance k and angle of orientation θ, h i ∞ ∞ 2 2 − 2 α − β − ˆ 2π ( k cos θ 1) +( k sin θ 1) 2πi(αx+βy) Sp(x, y, k, θ)= f(α, β)e e dαdβ (5.6) −∞ −∞ where the subscript p indicates the multiplication by the phase term.

The phase term corresponds to the spatial location of the frequency components.

According to the Fourier shift theorem, the magnitudes of the resulting spectra are identical to those obtained from shifting the Fourier domain of the image, but with diﬀerent phase. By correcting for the phase, the average magnitude of the complex data can be calculated without having to ﬁrst take the magnitude of each value, as in (5.4).

Local spectra can be created by integrating the domain over θ h i 2π ∞ ∞ 2 2 − 2 α − β − ˆ 2π ( k cos θ 1) +( k sin θ 1) 2πi(αx+βy) Sp(x, y, k)= f(α, β)e e dαdβ dθ. 0 −∞ −∞ (5.7) 82

Note that in equation (5.7) only the localizing windows depend on θ. Therefore, the order of integration can be rewritten as

h i ∞ ∞ 2π 2 2 − 2 α − β − ˆ 2π ( k cos θ 1) +( k sin θ 1) 2πi(αx+βy) Sp(x, y, k)= f(α, β) e dθ e dαdβ. −∞ −∞ 0 (5.8)

In (5.8), the FT of the image is multiplied by the sum of the windows for all θ.

The power at frequency k is obtained by performing a single inverse 2D-FT. This is fundamentally more efficient than calculating the contribution of each (kx,ky) component and then averaging. In addition, the intermediate 4D space-frequency domain, S(x, y, kx,ky), is not calculated and stored. However, implementation of (5.8) on discrete data is difficult. The continuous integral over θ must be converted to a partial sum, and floating point values of k must be converted to integral indices. To further simplify the calculations, the window integrated over θ in (5.8) can be replaced with a window designed as a function of k. For example, in this thesis I use the circularly symmetric window whose radial cross-section is a Gaussian function with width proportional to the frequency k, „ √ «2 2 2 − 2 α +β − 2π k 1 wˆcST (α, β, k)=e ,k=0 . (5.9)

Therefore, the transform becomes

„ √ «2 ∞ ∞ α2+β2 −2π2 −1 cST (x, y, k)= fˆ(α, β)e k e2πi(αx+βy)dαdβ (5.10) −∞ −∞ where I have changed the notation to refer to this new transform as the circular

S-transform (cST). 83

In the discrete domain1 the cST can be written as „ « − − √ 2 N/2 1 N/2 1 2 n2+m2 n m −2π −1 cST [x, y, k]= fˆ , e kN e2πi(nx+my)/N (5.11) N N n=−N/2 m=−N/2 × 2 2 where I assume that the image is size N N, x and y are discrete and k = kx + ky. The ﬂow diagram in Fig. 5.2 illustrates the process of calculating the cST.

The 3D cST (x, y, k) can be viewed in two ways: as “frequency maps” that show the power of a particular k at all spatial locations, cST (∗, ∗,k), or by taking projections at a particular (x, y) location, cST (x, y, ∗), to get local spectra (Figure

5.3). Note that I have found it useful to smooth cST (x, y, k) to help reduce spatial artifacts that may arise from using the ring-shaped window in (5.9).

The new local spatial frequency distribution, the cST, is complex-valued. There- fore, unlike pST (x, y, k) in (5.4), the cST may be inverted to recover the original image, f(x, y). Inversion of the cST requires the use of a new technique that sums the complex domain over k. In particular, F { cST [x, y, k]} f(x, y)=F −1 k (5.12) AN [n/N, m/N]

−1 where F and F denote the forward and inverse 2D-FTs, respectively, and AN is calculated by summing the localizing windows over all values of k

„ √ «2 2 n2+m2 n m −2π −1 A , = e kN . (5.13) N N N k The technique is a novel extension, to multiple dimensions and radial frequencies, of a 1D method ﬁrst proposed in [66] and later reﬁned in [67] to include the division by AN . 1Note that, in general, the equations given in this chapter refer to discrete signals and images; continuous equations are used on occasion for convenience to explain theoretical concepts. 84

Image Fourier domain of the image

Perform 2D-FFT k

Optionally apply For all k: ﬁlters and invert • multiply by circularly symmetric Gaussian window cST (, , k) complex local perform inverse 2D-FFT spectrum: cST(x,y,k) • • add to cST(x,y,k)

Take magnitudeitudede anand smooth with Gaussianussian ( N/k)

Local magnitude spectrum amplitude

spatial frequency (1/cm)

Figure 5.2: A ﬂow diagram illustrating the process of calculating the cST. 85

cST(*,*,kmax) ......

cST(*,*,k3) cST(*,*,k2) cST(*,*,k1)

cST ( x, y,*) (x,y)

Figure 5.3: The 3D cST (x, y, k) can be viewed in two ways: as “frequency maps” that show the power of a particular k at all spatial locations, cST (∗, ∗,k)orby taking projections at a particular (x, y) location, cST (x, y, ∗). 86

Invertibility has a number of advantages. For example, a space-frequency ﬁlter

G(x, y, k) can be applied to the spectra prior to inversion F { cST [x, y, k]G[x, y, k]} f (x, y)=F −1 k . (5.14) AN [n/N, m/N] The ﬁlter can be used to selectively enhance or suppress spatial frequencies to improve the speciﬁcity of the texture analysis.

5.2 Methods

5.2.1 Preliminary Analysis on an MS Image

I initially tested the new spectral analysis technique on a relapsing-remitting MS patient (age = 46 years) imaged on a 3T MR scanner (Signa, GE Healthcare, Wauke- sha, WI). Axial T2-weighted (TR/TE = 5000/98 ms) images were obtained (FOV

= 22 cm, matrix size = 512 × 512, slice thickness = 5 mm, 7 mm gap). A large peri-ventricular lesion and a region of NAWM were identiﬁed by a neuroradiologist.

Local spectra were generated using the pST [71] for ROIs of size 32×32 and 64×64.

ROIs were placed within a region of NAWM and a lesion, respectively. Analysis of the entire image was also performed using the cST. For both approaches, I averaged together the local spectra within a 5 × 5 region at the centre of each ROI and calculated the LFE as the area under the spectrum from 0.73 to 3.2 cm−1.

5.2.2 Further Analysis

Next I analyzed 7 patients from a previous study [55] to more rigorously evaluate the eﬀect of changing ROI size and the diﬀerences between the pST and cST.

Patients involved in the study were imaged on a 1.5T MR scanner (Magnetom 87

63SP, Siemens, Erlangen, Germany). I analyzed T2-weighted fast spin echo images with the following acquisition parameters: TR/TE = 2270/80 ms, FOV =

25 cm, matrix size = 256×256, slice thickness = 3 mm. The T2-weighted images were non-uniformity corrected using a nonparametric intensity nonuniformity normalization (N3) algorithm [121]. Signal intensity was normalized to the average ventricular CSF intensity. Spectra from 5×5 pixel areas were averaged. ROIs were placed within enhancing lesions, identiﬁed in T1-post contrast weighted images

(TR/TE = 600/12 ms).

I hypothesized that as the image ROI size was increased, the LFE value would reach a stable value, where it would not change with larger image sizes. I performed experiments to determine the minimum ROI size needed for consistent measurement of LFE values. Three bandwidths were used to deﬁne the LFE range: 0.32–2.88 cm−1 [55], 0–5 cm−1 [53] and an intermediate bandwidth of 0.8–3.6 cm−1. Square

ROIs of ﬁve diﬀerent lengths/widths were examined: 3.125, 6.25, 9.375, 12.5 and

15.625 cm (corresponding to 32, 64, 96, 128, and 160 pixels, respectively). Two-sided t-tests, assuming unequal variance, were used to identify diﬀerences in LFE values, with p > 0.05 indicating consistent measurements. The time to calculate each of the transforms was measured. I also compared the spectra from both transforms in terms of the correlation of pST and cST spectra. All processing was done using

Mac OS 10.5 on a 2.4 GHz Intel Core 2 Duo MacBook Pro with 2 GB of RAM. 88

5.3 Results

5.3.1 Preliminary Results

Fig. 5.4 shows the local spectra from the MS lesion and NAWM obtained using the pST on ROIs of size 32 × 32 and 64 × 64 as well as the cST on the entire 512 × 512 image. I was able to compute the cST of the entire image in approximately 3.5 min, achieving a spectral resolution of 0.045 cm−1 — a factor of 8 improvement over the 64 × 64 ROI analysis (0.36 cm−1) and a factor of 16 improvement over the

32 × 32 ROI analysis (0.73 cm−1). The diﬀerent methods appeared to give similar spectra, particularly at higher frequencies. The cST obtained from analyzing the entire image better resolved peaks in the spectra that are diﬃcult to identify when using small ROIs with the pST (Fig. 5.4). The preliminary results indicated that the average LFE values depend on the size of the ROI used for analysis, as shown in Fig. 5.5.

5.3.2 Further Results

As I increased the size of the ROI used for analysis with the pST, the LFE value decreased. For the 0.32–2.88 and 0.8–3.6 cm−1 bandwidths, the LFE values even- tually converged to a minimum. The minimum ROI sizes required to reach stable

LFE values for each bandwidth is given in Table 5.1. Examples of spectra calculated at various ROI sizes are shown in Fig. 5.6. This ﬁgure shows the improvement in spectral resolution with increasing image size. The eﬀect of the ROI size on LFE measurements is shown for one patient in Fig. 5.7.

The pST and cST spectra were highly correlated, as shown in Table 5.2. Figure 89

0.20 pST (32x32) pST (64x64) 0.15 cST (512x512)

0.10

0.05 spectral amplitude (AU)

0.00 0 2 4 6 8 10 12 14 16 (a) spatial frequency (1/cm)

0.10 pST (32x32) pST (64x64) 0.08 cST (512x512)

0.06

0.04

0.02 spectral amplitude (AU)

0.00 0 2 4 6 8 10 12 14 16 (b) spatial frequency (1/cm)

Figure 5.4: Average local spectra for a 5 × 5 region in (a) a MS lesion and (b) NAWM. The shaded area indicates the region where LFE is calculated (0.73 to 3.2 cm−1). By allowing the entire image to be transformed, the cST is able to improve the resolution of the spectra by a factor of 8 over the 64 × 64 pST analysis and by a factor of 16 over the 32 × 32 analysis. The improved resolution appears to resolve and emphasize peaks not obvious in an ROI analysis. 90

0.4 pST (32x32) pST (64x64) cST (512x512) 0.3

0.2

0.1 Amplitude (arbitrary units)

0 Lesion NAWM

Figure 5.5: The average LFE values from the pST and cST, averaged over a 5x5 region in an MS lesion and NAWM. The numbers in brackets indicate the size of the ROI analyzed.

Figure 5.6: The spectrum of a point in an MR image computed using the cST at the full image FOV as well as with the pST at varying FOV sizes (128×128, 64×64 and 32 × 32). 91

Table 5.1: The minimum ROI sizes required to reach stable LFE values. bandwidth smallest ROI size next smallest ROI size (cm−1) (# of cases) (# of cases) 0.8–3.6 6.25 cm (1/7) 9.375 cm (6/7) 0.21–2.88 9.375 cm (4/7) 12.5 cm (3/7) 0–5 all LFE values signiﬁcantly diﬀerent

†† ***

Figure 5.7: Effect of ROI size on LFE using the pST for one patient. The ∗ and † indicate values that are not significantly different (p>0.05). Error bars are standard deviation within a 5×5 pixel region. 92

Table 5.2: P-values for the difference between pST and cST LFE measures for three bandwidths (columns 2–4), along with the correlation coefficients (column 5). The pST and cST give similar LFE values, particularly for large ROIs and bandwidths that exclude very low frequencies. The spectra are highly correlated. bandwidth (cm−1) correlation FOV 0–5 0.32–2.88 0.8–3.6 coefficient 32 × 32 0.02 0.007 0.2 0.973 64 × 64 0.01 0.2 0.5 0.977 128 × 128 0.02 0.6 0.5 0.980

5.8 shows examples of the spectra at three diﬀerent FOV sizes for one patient. The

LFE measurements from both methods were consistent for large ROIs and when using bandwidths that did not include very low frequencies. The measured LFE values were diﬀerent when using small ROIs in combination with a bandwidth that included frequencies very close to the DC.

The cST took less time to calculate than the pST and reduced the number of computations by approximately a factor of N. The time to compute the pST and cST (with and without smoothing) is shown in Fig. 5.9.

5.4 Conclusions

LFE values decreased with increasing ROI size. A stable LFE value was reached in some cases. Therefore, the eﬀect of ROI size should be examined when performing spectral analysis and the ROI size should be large enough, based on the LFE bandwidth being used, to obtain LFE values independent of ROI size. The new cST avoids this issue by allowing for analysis of the entire image.

The cST took less time to calculate than the pST and reduced the number of 93

0.20

pST 0.15 cST

0.10

0.05 amplitude (arbitrary units) amplitude (arbitrary 0.00 0.0 2.0 4.0 6.0 (a) spatial frequency (1/cm)

0.20

pST 0.15 cST

0.10

0.05 amplitude (arbitrary units) amplitude (arbitrary 0.00 0.0 2.0 4.0 6.0 (b) spatial frequency (1/cm)

0.20

pST 0.15 cST

0.10

0.05 amplitude (arbitrary units) amplitude (arbitrary 0.00 0.0 2.0 4.0 6.0 (c) spatial frequency (1/cm)

Figure 5.8: Examples of spectra obtained from the pST (gray) and cST (black) for FOVs: (a) 32 × 32, (b) 64 × 64 and (c) 128 × 128. 94

Figure 5.9: The pST takes longer to calculate (time ∝ N 3.8) than the cST (time ∝ N 2.6−2.7) for various N × N image sizes. computations by approximately a factor of N. The cST and pST spectra were highly correlated. The resulting LFE measurements were consistent for large ROIs and when using bandwidths that did not include very low frequencies. The measured

LFE values were diﬀerent when using small ROIs in combination with a bandwidth that included frequencies very close to the DC.

I have demonstrated a novel local frequency analysis technique that is suitable for analyzing texture differences in MS. The cST approach provides an efficient spectral decomposition of an image and may allow other features of the spectra to be examined in addition to the area under portions of the curve. Preliminary results indicate that the spectra and LFE measurements obtained from the cST are consistent with those using pST, but further work is required to determine whether the improved frequency resolution will allow better identification of spectral regions that can be used to characterize MS disease progression. 95

Chapter 6

Properties and Performance of the cST

Portions of this chapter have been presented as a podium presentation at the Cana- dian Student Conference on Biomedical Computing (CSCBC), in Toronto, Ontario

(March 2008) and were published in the peer-reviewed conference proceedings [122].

I received a travel grant based on this work to attend the CSCBC meeting.

The work presented in this chapter was performed by myself, under the supervision of Dr. Ross Mitchell, who provided guidance, suggestions for experimentation and who contributed to the publications described above.

6.1 Introduction

In this chapter I describe in more detail some of the unique properties of the cST, such as its use of a phase rotation, its invertibility and rotation invariance. I characterize the performance of the cST when changing the extent of an image, simulating image non-uniformity, changing the smoothing of the frequency maps and translating the image. I use theoretical arguments and perform experiments on a wide variety of images, including test patterns of simple sinusoidal patterns, texture library images as well as real MR images. 96

6.1.1 Phase Shift

In Chapter 5 I derived the expression for the cST by multiplying the 2D-ST, given in equation (2.38), by the phase term exp[2πi(kxx + kyy)] and used the relations kx = k cos θ and ky = k sin θ to express the result in terms of the radial distance k and angle of orientation θ. The phase term allows local spectra to be created by integrating the complex ST values over θ.

By summing along the time axis, one can collapse the ST to the FT of the signal.

However, if one were to attempt to integrate the complex values along the frequency direction, the phase orientation would cause destructive interference. In this section

I explore the eﬀect of summing ST components in the 1D case before and after applying a phase ramp.

6.1.2 Localizing Windows

The cST uses circularly symmetric windows to compute the spectral information for a particular frequency k instead of having to compute the value at each angle θ and average together the results. This approach reduces the number of computations required.

The circularly symmetric window I have chosen to use with the cST has the same radial cross-section as the standard Gaussian window used with the ST [61].

The width of the window is proportional to the frequency k, „ √ «2 2 2 − 2 α +β − 2π k 1 wˆcST (α, β, k)=e ,k=0 . (6.1)

The discrete form of the window is given by „ √ «2 n2+m2 n m F −2π2 −1 wˆ , , = e F (6.2) cST NΔx NΔx NΔx 97 where n and m are frequency indices. For the remainder of the chapter it will be assumed that the images are square (N = M) and the spacing is equal in the x- and y-directions (Δx =Δy). Note that in this chapter I use the variable k to refer to frequency, either continuous or discrete. In the discrete case, if Δx = 1 one can refer to the wavenumber for a given k as F = kN.

The spectral resolution for the horizontal or vertical frequencies can be given as 1 1 Δkx = = (6.3) NΔx FOVx 1 1 Δky = = MΔy FOVy where FOVx and FOVy are the FOV in centimeters in the x and y directions, respectively. The resulting spectral resolution is given in units of inverse length

(usually 1/cm or cm−1).

The localizing windows of the cST are applied at the centre frequencies n k(n)= ,n=0, 1, 2,...,n (6.4) NΔx max √ where nmax ≤ N/ 2. Depending on how far out in k-space the localizing windows are applied, the maximum radial frequency that can be examined is either N/2 1 k = = (6.5) max NΔx 2Δx if nmax = N/2, or √ N/ 2 1 k = = √ (6.6) max NΔx 2Δx √ if nmax = N/ 2. Figure 6.1 illustrates the diﬀerence between the maximum frequencies.

To illustrate the frequency response and characteristics of the cST windows, I plot radial proﬁles of the windows in the frequency and spatial domains and compare 98

Figure 6.1: (a) The maximum frequency measured when applying circularly symmetric windows (grey) up to the Nyquist frequency (nmax = N/2) is kmax =1/(2Δx) or 0.50 if Δx = 1. The resulting spectra will contain N/2 points. While the corner frequencies are not directly sampled, their contribution is included in lower frequencies due to window√ overlap. (b) The√ maximum frequency of a localizing window (grey) if nmax = N/ 2iskmax =1/( 2Δx) or 0.71 if Δx =1. them to the windows used in the ST, pST and in the invariant Gabor ﬁlters. I also attempt to determine the maximum radial frequency that should be used in practice when performing texture analysis with the cST.

6.1.3 Smoothing

The process of multiplying the Fourier domain by a ring-shaped window implies convolving the image with the FT of the window, which is a sinc-like function (shown in Fig. 6.6). As a result, the frequency maps exhibit ringing with a frequency of oscillation proportional to k. If one is interested in examining the frequency maps, these artifacts can be removed by smoothing each magnitude frequency map with a

Gaussian function. However, the resulting transform is no longer invertible.

Before the cST can be used for image texture analysis, it is important to un- 99 derstand the eﬀect of smoothing frequency maps, as well as the appropriate levels of smoothing to use. In this section, I compare various levels of Gaussian blurring of the magnitude frequency maps and determine the change in spectral amplitude across all frequencies after smoothing.

6.1.4 Geometric Transformations

Rotational and translational invariance is an important feature of a texture analysis approach in medical imaging. In diseases such as brain cancer and MS, lesions may be located anywhere in the brain and may grow relatively unimpeded in any direction. As a result, it is desirable to have texture measures that do not change with rotation of either the patient or the pathology under examination.

Furthermore, at times one may wish to crop an image to extract a region or to remove regions from outside the brain. In these cases, it would be beneﬁcial to know how the cST responds to reducing the FOV of an image. Since the cST measures frequency components in units of inverse length (i.e. cycles/cm or cycles/mm) it should be relatively invariant to changes in the FOV.

6.1.5 Image Non-Uniformity

In MR imaging, eﬀects such as non-uniform sensitivity of the RF coils used to excite the sample or receive the MR signal, or non-uniform RF penetration of the sample, may cause image non-uniformity to occur [121]. Intensity non-uniformity is common in MR and manifests itself as a smooth, slow variation in the signal. I undertake experiments to determine the eﬀect on the cST when the image contains non-uniformity artifacts. 100

6.1.6 Inversion

Since the cST sums the frequency components at a particular radial frequency, it cannot be inverted using the standard method of summing over the spatial variables as in equation (2.41). Instead, I take advantage of a newly proposed method [66,67] of summing the discrete transform over the complex frequency terms and applying a Fourier domain correction, F { cST [x, y, k]} f(x, y)=F −1 k (6.7) k wˆcST [n, m, k] where x and y refer to discrete spatial variables, n and m to discrete spectral variables, and F and F −1 represent the forward and inverse 2D-FTs, respectively. The summations are carried out over all values of k>0. Note that a filter G[x, y, k] can be applied to the cST before inverting. Such a filter can selectively enhance or suppress desired frequency bandwidths on a pixel-by-pixel basis. In this section I evaluate the accuracy of the new inversion formula and demonstrate the ability to apply filters to the cST prior to inversion.

6.2 Methods

In order to use the cST to evaluate image texture or to perform image ﬁltering, I evaluate how the transform performs after smoothing of the frequency maps, the application of spatial transformations, and when the frequencies are calculated to

kmax versus kmax. I evaluate the ﬁdelity of the cST to aﬃne transformations and intensity non-uniformities and perform a variety of experiments to characterize the performance of the cST. The time to calculate each transform is recorded and com- 101 pared. All analysis is performed in Python 2.5 on a MacBook Pro, 2.4 GHz Intel

Core 2 Duo with 2, OS 10.5.

6.2.1 Phase Shift

To better illustrate why a phase term is needed, I demonstrate the eﬀect on the

ST of a synthetic signal, consisting of a low-frequency component with wavenumber n = 5 for the ﬁrst half, a mid-range frequency of n = 21 for the second half and a high-frequency burst at n = 48 occurring at t = 20. I examine the real part of the transform, as well as its phase, before and after the application of a phase ramp.

Next I perform an experiment to illustrate the concept of destructive interference by summing the complex values of the ST along the frequency direction to dyadically spaced frequency intervals. For every point in the transform, I perform the following operation in Python: n = log(N)/log(2)

S_ds = zeros(shape(S)).astype(complex) for i in range(n):

S_ds[:, 2**i:2**(i+1)] = average(S[2**i:2**(i+1),:], axis=0) where the ﬁrst line calculates the number of dyadic components; the second line creates the complex matrix S ds to store the result (which is the same size as the

ST of the signal S), and initializes it to zero; and the last two lines loop through the array, ﬁlling each row with the average of the frequency components within the bandwidth 2i/N to (2i+1 − 1)/N . 102

6.2.2 Localizing Windows

To characterize the window used with the cST, I compare its proﬁle to the windows used with theST, pST as well as with the window used in the invariant GT. I plot cross sections of the windows to compare their spatial and spectral resolutions, as well as the spectral weighting each provides to the Fourier domain of the image.

6.2.3 Visual Analysis of Local Spectra

I demonstrate the ability of the cST to identify and discriminate between known frequency components of a synthetic image. The synthetic image, shown in Fig. 6.8, consists of four diﬀerent texture regions (matrix size = 256 × 256, ﬂoating point values of -1.0 to 1.0). The cST is evaluated according to equation (5.11).

I also compare the ability of the cST to separate between two similar low frequency components in a synthetic image of size 128 × 128 pixels with unit sampling containing two frequency components: (kx,ky) = (0.16, 0) and (0, 0.31). The spectrum of the central pixel of the image is computed using the cST, invariant wavelet and the invariant GT.

6.2.4 Frequency Response

In order to characterize the frequency response of the cST, I calculate the transform of a simulated delta function. For the purposes of this experiment, I use a 256× 256 image where all pixels have a value of zero, except the pixel at (x, y) co-ordinates

(128,128), which has a value of one. I examine the frequency maps of the cST as well as the spectra at the point (x, y)=(128, 128). 103

6.2.5 Smoothing

To remove spatial artifacts in the cST, I convolve each frequency map with a Gaus- sian function (standard deviation σ =1/k) and take the squared magnitude (power).

I compare the frequency maps of the synthetic image in Fig. 6.8 before and after smoothing as well as comparing them to the maps generated using the pST window wˆpST (α, β, k, θ) in (2.49), averaged over θ = 0 to 2π. I examine three levels of smoothing with a Gaussian function of standard deviation σ = s/k: s = 0 (unsmoothed), 1 and 1.5. I smooth each frequency map of the cST of the delta function with the appropriate Gaussian and compare the local spectra at the point (x, y) = (128, 128). Finally, I visually compare the frequency maps of a texture image [6] generated using the raw cST, smoothed cST and the invariant GT at various spatial frequencies.

6.2.6 Eﬀect of Geometric Transformations

Cropping

To illustrate the eﬀect on the cST when cropping an image, I take an MR image of

FOV=22 cm, matrix size=256 × 256 and digitally “zoom in” to two diﬀerent scales:

FOV=11 cm (128×128) and 5.5 cm (64 × 64) by cropping (shown in Fig. 6.2). I calculate the cST of each image and compared the magnitude spectra from the central point of each image.

Translational Invariance

To demonstrate the eﬀect of translations on the continuous cST I analytically determine the eﬀect of a translation in the x-ory-direction. To show the translation 104

(a)

(b) (c)

Figure 6.2: (a) A T2-weighted MR image with a FOV = 22 cm and matrix size = 256 × 256. The same image is shown cropped to FOV= 11 cm, matrix size = 128 × 128 (b) and FOV= 5.5 cm, matrix size = 64 × 64. 105 invariance of the cST in a discrete case, I take the cST of a 256 × 256 T2-weighted

MR image (FOV=22 cm) and calculate the L1 norm between the spectrum of the pixel at index (128,128) and the same pixel after the image is shifted horizontally

m and vertically from -6 to +6 cm, in steps of 2 cm. The L1 norm of two spectra S and Sn of length N/2 at a given pixel (x, y) is deﬁned as

N/2−1 1 L (Sm,Sn)= Sm[x, y, k]|−|Sn[x, y, k] . (6.8) 1 N k=0 Note that I perform circular shifts of the image volume, where the volume “wraps around” when it reaches the edge of the FOV. This is done to simulate the aliasing artifact that occurs in MR imaging if the object size exceeds the image FOV.

Rotation Invariance

In this section I show, theoretically, that the continuous cST is immune to rotations.

It should be noted that it is not possible to have true rotation invariance in the discrete case. Due to the square shape of the image and Fourier domains, whenever a rotation is applied in the discrete case some information is rotated into, and some out of, the FOV. Furthermore, because of discrete sampling interpolation is required to resample the data after a rotation.

6.2.7 Eﬀect of Image Non-Uniformity

I model the eﬀect of low, medium and high non-uniformity non-uniformity as multiplicative Gaussian functions of three widths (standard deviations): 3N/2,N and

N/2 for a 256×256 image. For this test I used three Brodatz texture images, rotated by an angle of 5° and padded with a border of zeros (48 pixels). Figure 6.3 shows 106 an example of one of the images along with the eﬀect of the low, medium and high models of non-uniformity.

The average cST spectrum is evaluated within a 5×5 region, 20 pixels away from the centre of the image (in both x and y), before and after multiplying by the

Gaussian function. I divide the spectrum of the non-uniform image by that of the uniform image and average the results over the three texture images to obtain an estimate of the non-uniformity eﬀect on the cST spectra.

6.2.8 Inversion

To compare the accuracies of the various inversion formulas described in Section

2.5.3, I ﬁrst test the algorithms on a 1D signal containing a sinusoid of increasing frequency: f(t) = sin(2π(t/6)t/N) where t = 0 to 255 and N = 256 (shown in

Fig. 6.18). I calculate the ST of the signal and then compare the results when reconstructing the signal by: (1) summing along the frequency axis by directly implementing equation (2.43); (2) after applying AN according to equations (2.44) and (2.45) to the signal from method (1); and (3) using the traditional time-axis inversion according to (2.42).

The accuracy of the 2D inversion formula, described in (6.7), is evaluated by

calculating the L1,L2 and L∞ norms between the reconstructed, f (x, y) and original 107

(a) (b)

Figure 6.3: (a) An example of a Brodatz texture image used to perform the non- uniformity experiment. The eﬀect of (b) low, (c) medium and (d) high levels of non- uniformity are modeled by multiplying the image in (a) by a Gaussian of standard deviation 3N/2,N and N/2 pixels, respectively. 108 synthetic image, f(x, y) (shown in Fig. 6.8) of size (N × N)

− 1 N1 L = |f [x, y] − f[x, y]| (6.9) 1 N x,y=0 − 1 N1 L = |f [x, y] − f[x, y]|2 (6.10) 2 N x,y=0

1 L∞ = max |f [x, y] − f[x, y]|. (6.11) N

I also test the inversion method on ﬁve texture images from the Brodatz texture library [123,124] cropped to various pixel sizes from 32×32 to 256×256 and compute √ the L1 norms when evaluating the transform to both kmax = N/2 and N/ 2. I apply a box-shaped ﬁlter G(x, y, F ) to the cST of the synthetic image in Fig. 6.8 consisting of ones from (a) F = 10 to 40 and (b) F = 1 to 10 and 40 to 120, and zero elsewhere.

I then use the inversion formula in (6.7) to reconstruct the ﬁltered image.

6.3 Results

6.3.1 Phase Shift

Figure 6.4 shows the signal (a), along with its ST (b), clearly indicating the presence and duration of each frequency component. The real part of the transform in (c) demonstrates that while the bright areas are quite smooth in the time (horizontal) direction, there are ripples occurring in the frequency (vertical) direction. The phase ramp in part (d) more clearly illustrates the pattern of the phase; distinct horizontal bands can be seen. Multiplying the ST by the complex phase ramp rotates the ripples in the real part of the transform from the vertical to the horizontal orientation, as demonstrated in part (e) of the ﬁgure. Part (f) shows that the distinct 109 phase bands are oriented vertically after applying the phase ramp.

Figure 6.5 demonstrates the destructive interference when averaging the complex

ST to dyadically spaced frequency intervals. Part (a) of the ﬁgure shows the results without applying the phase ramp to the ST, where the relevant information disap- pears. Part (b) of the ﬁgure shows that the information in the ST is retained if the

S-domain is multiplied by the complex phase term p(τ,ξ)=e2πiτξ before averaging.

6.3.2 Localizing Windows

The relationship of the cST window,w ˆcST , to the traditional ST window can be illustrated by taking a cross-section at β =0,

„ √ «2 2 − 2 α +0 − 2π k 1 wˆcST (α, 0,k)=e (6.12)

2 − 2 α − = e 2π ( k 1) (6.13)

2 − 2 α−k = e 2π ( k ) . (6.14)

This is the same window as used in the 1D-ST in equation (2.36), with α replaced by α − k. Translating the cST window at β = 0 to the origin, as in the original ST, and shifting the FT of the signal results in the expression ∞ 2 − 2 α+k − cST (α + k, 0,k)= fˆ(α + k)e 2π ( k 1) e2πi(α+k)xdα (6.15) −∞ ∞ 2π2α2 − 2πiαx 2πikx = fˆ(α + k)e k2 e e dα (6.16) −∞ = e2πikx [S(x, 0,k)] . (6.17)

Therefore, the result agrees with the 1D ST multiplied by the phase term e2πikx.

Figure 6.6 compares the cST window to the pST window, generated by sum- √ mingw ˆpST (x, y, k, θ) from equation (2.47) over θ (and multiplying by 2π/k for 110

1.5 0.5

0.4

0.3 0.0 0.2 amplitude frequency

0.1

-1.5 0.0 0 20 40 60 80 100 120 0 20 40 60 80 100 120 (a) time (b) time

0.5 0.5

0.4 0.4

0.3 0.3

0.2 0.2 frequency frequency

0.1 0.1

0.0 0.0 0 20 40 60 80 100 120 0 20 40 60 80 100 120 (c) time (d) time

0.5 0.5

0.4 0.4

0.3 0.3

0.2 0.2 frequency frequency

0.1 0.1

0.0 0.0 0 20 40 60 80 100 120 0 20 40 60 80 100 120 (e) time (f) time

Figure 6.4: (a) A 1D signal consisting of a low-frequency component (ξ=0.04) for the ﬁrst half, a mid-range frequency (ξ=0.16) for the second half and a high-frequency burst (ξ=0.38) at t=20. (b) The magnitude of the ST, S(τ,ξ). (c) and (d) show the real portion and phase maps of S(τ,ξ), respectively. (e) and (f) show the real and phase portions, respectively, of S(τ,ξ)=S(τ,ξ)p(τ,ξ) where p(τ,ξ)=e2πiτξ is a phase rotation that rotates the “ripples” from the vertical to the horizontal orientation. 111

0.5

0.4

0.3

0.2 frequency

0.1

0.0 0 20 40 60 80 100 120 (a) time

0.5

0.4

0.3

0.2 frequency

0.1

0.0 0 20 40 60 80 100 120 (b) time

Figure 6.5: (a) The magnitude of the ST (shown in Fig. 6.4) after averaging the frequency components for each point to a dyadic sampling grid, without applying the phase rotation. Destructive phase interference causes the relevant information to disappear; only the edge eﬀect is present in the frequency-averaged domain. (b) The magnitude of the frequency-averaged domain after applying the phase ramp; the relevant information is retained. 112 comparison), for F = 20(N = 128). The ﬁgure shows that the symmetric window in equation (6.1) provides a more even spectral weighting for each component than the equivalent pST window. Although not shown, results are similar for other frequencies and values of N.

Figure 6.7 shows a cross-section of the cST window and the rotationally invariant

Gabor ﬁlter presented in Section 4.2.2 in the spectral and spatial domains for F =

20,N = 128. Note that the maximum amplitude of the Gabor window has been normalized to one in the spectral domain for comparison and that the cross-section of the cST window is identical to the corresponding ST window. The cST window tapers to zero in the frequency domain more quickly than the Gabor ﬁlter.

6.3.3 Visual Analysis of Local Spectra

Figure 6.8 shows the synthetic test pattern containing four frequencies. Part (b) of the ﬁgure shows the spectral power (amplitude squared) obtained using the cST at the centre of each frequency region. Note that the peaks are well separated and well deﬁned.

As expected, the high frequency components are more diﬃcult to resolve in frequency, resulting in broadening of spectral peaks with increasing frequency (Figure

6.8b). Conversely, the low frequencies are more diﬃcult to resolve spatially, resulting in smearing of low frequency power (Fig. 6.9). This manifests itself in Figure

6.8b as a small primary F = 4 peak, with contributions from other image regions.

Figure 6.9 shows that some diﬀerences occur in the spatial resolution of spectral components between the smoothed cST and the pST. For example, in the F =4 map the smoothed cST appears to have more leakage in the vertical direction, while 113

-0.5

0.0 kx 0.5 0.5 0.0 -0.5 (a) ky

-0.5

0.0 kx 0.5 0.5 0.0 -0.5 (b) ky

118 128 138 (c) x

Figure 6.6: (a) The circularly symmetric Gaussian window provides a more even spectral weighting than (b) the corresponding pST window, integrated over all values of θ, (128×128), F =20. (c) The circularly symmetric window in the spatial domain is a sinc-like function. 114

1.0 cST Gabor 0.8

0.6

0.4

0.2 amplitude (arbitrary units) 0.0 -60 -40 -20 0 20 40 60 (a) frequency coefficient

0.06 cST Gabor

0.03

0.00 amplitude (arbitrary units) -0.03 40 50 60 70 80 90 (b) spatial position

Figure 6.7: (a) A profile of the cST window and Gabor filter in the spectral domain for F =20,N = 128. Note that the cST window tapers to zero more quickly than the Gabor windows. These windows have been scaled to the same maximum amplitude for comparison. (b) The cST window is slightly broader than the Gabor filter in the spatial domain, meaning that the ringing will be spread over a slightly larger area. The effect of the ringing can be reduced by smoothing the amplitude frequency maps. 115

(a)

(b)

Figure 6.8: (a) Test pattern (256×256) with four frequencies of wavenumber (horizontal, vertical): (0,4), (8,10), (60,0) and (20,20), starting in the upper left, clockwise. (b) The resulting cST spectra (power, magnitude squared) from the centre of each region (UL: upper left, UR: upper right, LL: lower left, LR: lower right). 116

F = 4 F = 13 F = 28 F = 60

Original cST

Smoothed cST

pST

Figure 6.9: The squared magnitude (power) of frequency maps before (top) and after smoothing (middle) with a Gaussian kernel (σ =1/k) along with the corresponding pST power maps (bottom) for F = 4, 13, 28 and 60. the pST has more leakage in the horizontal direction. The higher frequency maps indicate that the cST has less distortion spatial resolution of components, but some artifacts still remain after smoothing.

Figure 6.10 indicates that the spectra from both the invariant wavelet and invariant Gabor spectra produce a single peak when analyzing the image that contains two frequency components: (kx,ky)=(0.16, 0) and (0, 0.31). Also note that with the dyadic sampling, more samples are obtained at lower frequencies than at higher frequencies, which could limit the ability to resolve fine textural differences in medical images. The cST provides a smooth spectrum that clearly identifies the two frequency components. 117

(a)

(b)

Figure 6.10: (a) A synthetic image consisting of two sinusoids added together: (kx,ky) = (0.16, 0) and (0, 0.31) where N=128. The spectra from the centre of the image are calculated using the cST, smoothed with σ =1.5/k (solid black); √ √ 2 the invariant Gabor ﬁlter evaluated at F = N/2,N/2/ 2.5,N/2/ 2.5 ,...; and the invariant wavelet method at 7 levels of decomposition as in [5]. Note that the amplitude of the wavelet coeﬃcients have been divided by four for display purposes.

The Gabor and wavelet spectra contain only 7 points (log2 N) while the cST has 64 points. 118

cST cST cST Gabor (s = 0) (s = 1.0) (s = 1.5)

F = 6

F = 12

Figure 6.11: Frequency maps for the delta function at F = 6 (top) and F =12 (bottom). The left column shows the unsmoothed data. The next two columns show the results after smoothing with a Gaussian of standard deviation σ = s/k, where s = 1 (middle) and s =1.5 (right). The rightmost column shows the Gabor- ﬁltered response.

6.3.4 Frequency Response

Figure 6.11 shows examples of frequency maps of the delta function obtained from the cST (with and without smoothing) and the rotationally invariant Gabor ﬁlters.

The figure confirms that the spatial localization with the cST is not as good as that with the Gabor filter, as expected since the cST windows are narrower in the frequency domain. Figure 6.12 (a) shows the resulting spectrum of the delta image at the point (128,128). In this figure, note the spectral drop-off from kmax out to

kmax. Plotting the square root of the spectral amplitude (as shown in Fig. 6.12 (b)) conﬁrms that with no smoothing the amplitude of the cST is proportional to k2, with a leveling oﬀ near the Nyquist frequency (k =0.5). 119

6.3.5 Smoothing

Smoothing at a level of s =1.5 appeared to do a good job of removing the artifacts in the delta images (Fig. 6.11). However, some bleeding of spectral power did occur at low frequencies and some residual artifact remains at high frequencies when examining the test pattern (Fig. 6.9). The frequency maps appear similar to those obtained from the pST (as shown in Fig. 6.9), but the computation time is much lower. For example, the time to evaluate the frequency maps shown in Fig. 6.9 are:

6.5 s for the cST; 19 s for the cST with smoothing; and 51 min for the pST.

The Brodatz texture image and frequency maps are shown in Fig. 6.13. Note that the smoothed cST maps give smoother indications where the frequency components occur, without replicating the speciﬁc image structure. This is due to the more narrow taper of the cST window, excluding the low frequencies that indicate the presence of structure in the Gabor ﬁltered responses.

Smoothing leads to a deviation of the relationship between the amplitude of the cST and frequency k, with higher smoothing levels causing more artifacts in the frequency response, as shown in Fig. 6.12. A cross-section of one of the frequency maps, shown in Fig. 6.14, shows how smoothing the frequency maps reduces the ringing caused by the circularly symmetric localizing window. A value of s =1.5 provided a good trade-oﬀ between the appearance of artifacts in the frequency maps and maintaining the relationship between amplitude and frequency. 120

(a)

(b)

Figure 6.12: (a) The local spatial frequency spectrum of a delta function using the cST with various levels of smoothing (σ = s/k). (b) The square root of the spectra shows that the amplitude increases with k2 for the unsmoothed case. The smoothed spectra deviate from this relationship. The shaded areas represent the “corner” frequencies above the Nyquist frequency that may be unreliable in analysis due to the drop-oﬀ in spectral power. 121

cST cST Gabor texture image: (s = 0) (s = 1.5)

F = 5

F = 20

F = 40

F = 80

Figure 6.13: An image from the Brodatz texture library [6] and the resulting frequency maps for wavenumbers: F = 5, 20, 40 and 80 for the cST with no smoothing (left), smoothing with σ =1.5 (middle) and the magnitude Gabor ﬁlter response (right). 122

0.006 s=0 0.005 s=1 s = 1.5 0.004

0.003

amplitude 0.002

0.001

0.000 80 100 120 140 160 column position

Figure 6.14: A cross-section of the F = 12 frequency map for the delta response. While smoothing with s = 1 leaves some residual artifact, a value of s =1.5 results in a smooth spatial distribution and removes the ringing caused by the localizing window. 123

6.3.6 Eﬀect of Geometric Transformations

Cropping

Even with the drastically different scale of each cropped image in Fig. 6.2, the spectra, shown in Fig. 6.15, largely agree. There are some small low frequency differences, which result from removal of image regions when cropping. Note the very good spectral agreement for k ≥ 1 cycle/cm. The major difference that occurs when cropping is the change in the frequency resolution. According to Fourier theory, a change in the extent of one domain results in a change in the resolution of the other domain. Therefore, reducing the extent of the image increases the spacing between spectral components, thereby reducing the spectral resolution.

Translational Invariance

To demonstrate the effect of translations on the continuous cST, I examine the effect of a translation by x0 in the x-direction and y0 in the y-direction on the FT, as defined by the shift property

ˆ −2πi(αx0+βy0) f(x − x0,y− y0) → f(α, β)e . (6.18)

The cST of a shifted image is therefore

∞ „ √ «2 α2+β2 −2π2 −1 ˆ k 2πi(αx+βy) −2πi(αx0+βy0) cST (x − x0,y− y0,k)= f(α, β)e e e dαdβ −∞ ∞ „ √ «2 α2+β2 −2π2 −1 = fˆ(α, β)e k e2πi[α(x−x0)+β(y−y0)]dαdβ −∞ and one can see that the cST of a translated image is the same as the original, except for a change in spatial coordinates. Therefore, the cST should demonstrate 124

400 256x256 128x128 300 64x64

200

100 spectral amplitude

0 0 1 2 3 4 5 6 7 8 frequency (cycles/cm)

Figure 6.15: The local spectra at the centre of a T2-weighted MR image at three diﬀerent scales: 256 × 256 (original) and cropped to 128 × 128 and 64 × 64. The diﬀerences at low frequency (below 1 cycle/cm) arise due to anatomy cropped out of the FOV as well as due to truncation artifact. 125

×1e-13 2.2 2.0 vertical horizontal 1.8 1.6 1.4 1.2 1.0 L1 norm 0.8 0.6 0.4 0.2 -6 -4 -2 0 2 4 6 shift (cm)

Figure 6.16: The L1 norm between the spectrum of a pixel of a 256×256 T2-weighted MR image and the corresponding spectrum after shifting the image by -6 to +6 cm in steps of 2 cm. translational invariance. Figure 6.16 shows that the cST in the discrete case is indeed invariant to both vertical and horizontal shifts as the L1 norms are near machine precision.

Rotation Invariance

A rotation of an image by θ implies a rotation of the corresponding FT by θ

f(x,y) → fˆ(α,β) (6.19) where the rotation matrix is deﬁned as ⎛ ⎞ ⎛ ⎞ ⎛ ⎞ ⎜x ⎟ ⎜cos θ − sin θ⎟ ⎜x⎟ ⎝ ⎠ = ⎝ ⎠ ⎝ ⎠ . (6.20) y sin θ cos θ y 126

Substituting the rotated co-ordinates into the expression for the cST window,

„ √ «2 2 2 − 2 α +β − 2π k 1 wˆcST (α ,β,k)=e » –2 (cos2 θα2−2 sin θ cos θαβ+sin2 θβ2+sin2 θα2+2 sin θ cos θαβ+cos2 θβ2)1/2 −2π2 −1 = e k „ √ «2 α2+β2 −2π2 −1 = e k (6.21) one recovers the expression for the cST. Therefore, the cST of a rotated image is equal to a rotation of the cST,

∞ ∞ ˆ 2πi(αx+βy) cST (x ,y,k)= f(α ,β)ˆwcST (α ,β)e dα dβ . (6.22) −∞ −∞

6.3.7 Eﬀect of Image Non-Uniformity

Figure 6.17 shows the cST spectra after applying multiplicative Gaussians to the image to simulate low, medium and high levels of non-uniformity, divided by the original spectra. The ﬁgure shows that spectral components above approximately

0.1 cm−1 are scaled by a factor that depends on the strength of the non-uniformity.

The eﬀect on low frequencies can be unpredictable; the amplitude of low-frequency spectral components in the presence of moderate to high non-uniformity may be relatively increased or decreased compared to that of the rest of the spectrum.

6.3.8 Inversion

Figure 6.18 shows that the diﬀerence between the new inversion formula in equation

(6.7) and the original inversion formula for the ST, given in equation (2.41), on a

1D signal is near machine precision (∼ 10−14). Therefore, the accuracy of the new inversion formula is similar to that of the original formula. 127

1.0

0.8

0.6

0.4

low 0.2 medium high spectrum /0.0 uniform spectrum 0.0 0.1 0.2 0.3 0.4 0.5 frequency (1/cm)

Figure 6.17: The eﬀect of image non-uniformity on cST spectra at a location 20 pixels away from the center of the image horizontally and vertically (radial distance = 28 pixels). The non-uniformity ﬁeld is modelled as a multiplicative Gaussian function of standard deviation 3N/2, N, N/2 to represent low, medium and high levels of non-uniformity, respectively. 128

1 0.5

0.4

0.1 0.1 4 . 0.8

0.3 0 0 0.4 0.2 0.1 Frequency (Hz) 0.1 0.1 -1 0.0 0 50 100 150 200 250 0 50 100 150 200 250 (a) Time (s) (b) Time (s)

0.2 1

0 0.0

-1 -0.2 0 50 100 150 200 250 0 50 100 150 200 250 (c) Time (s) (d) Time (s) ×1e-14 1 1.5

0 0.0

frequency inversion (eq. 2.44) time inversion (eq. 2.42) -1 -1.5 0 50 100 150 200 250 0 50 100 150 200 250 (e) Time (s) (f) Time (s)

Figure 6.18: (a) A synthetic signal containing a sinusoid of increasing frequency: f(t) = sin(2π(t/6)t/N) where t = 0 to 255 and N = 256. (b) A contour plot of the magnitude ST for the positive frequencies. (c) Inversion of the ST along the frequency axis by directly implementing (2.43). (d) The diﬀerence between the reconstructed and original signals shows that a large low-frequency error is apparent in the residual as well as high-frequency artifacts. (e) The reconstructed signal after applying AN according to (2.44) and (2.45) and using the traditional time-axis inversion according to (2.42) are shown as one line since they are indistinguishable. (f) The residuals show that frequency-axis inversion after correction (black) has maximum errors on the order of 10−14, which is comparable to that of time-axis inversion (gray) with maximum errors on the order of 10−15. 129

x1e-15 3.2 2.4 1.6 0.8 0.0 -0.8 -1.6 -2.4

Figure 6.19: Diﬀerence between original image and reconstructed using equation (6.7).

The inversion formula according to equation (6.7) recovers the image with a high

−16 −15 −20 degree of accuracy: L1 =4.9 × 10 ,L2 =1.8 × 10 ,L∞ =5.1 × 10 (Figure 6.19). Using this inversion formula, an image can be reconstructed that contains only frequency components within the speciﬁed bandwidth, with minimal artifacts

(Figure 6.20).

Table 6.1 shows that the inversion is very accurate regardless of the image size or extent into frequency space of the spectra. The only critical factor is that the sum over k be to the appropriate upper limit.

6.4 Discussion

The orientation of the phase in space-frequency transforms depends on whether the image or the window is translated when calculating the transform. In the ST, the 130

(a)

(b)

Figure 6.20: Images generated by applying a box-shaped ﬁlter G(x, y, F ) to the cST of the synthetic image in Fig. 6.8(a) and inverting. G(x, y, F ) consists of ones from (a) F = 10 to 40 and (b) F = 1 to 10 and 40 to 120, and zero elsewhere. 131

Table 6.1: L norms for inverted images (average over 5 texture images). 1 √ Image size (pixels) nmax = N/2 nmax = N/ 2 256 × 256 4.34 × 10−13 2.09 × 10−13 128 × 128 1.11 × 10−13 1.16 × 10−13 64 × 64 3.47 × 10−14 2.01 × 10−14 32 × 32 5.71 × 10−15 3.58 × 10−15

FT of the image is translated while the window remains stationary. Conversely, in the CWT the mother wavelet translates across the image. This explains the phase diﬀerence between the two transforms [59]. With the cST, one wishes to apply windows centered around the origin of the Fourier domain. Therefore, the 2D phase correction must ﬁrst be applied in order to get the same results as the ST. This is why the derivation in Chapter 5 begins by replacing α with α−kx and β with β −ky. Note that the averaging over frequency is not a concern with the pST because the phase is discarded when the amplitude operation is performed before summing in concentric rings.

I conﬁrmed that the radial cross-section of cST window is identical to that of the corresponding ST window. The cST windows are narrower than the rotationally invariant Gabor ﬁlters. Therefore, one would expect better spectral discrimination with the cST, at the expense of more ringing due to the slightly broader spatial window (Fig. 6.7(b)). The ringing artifact is reduced by smoothing the resulting frequency maps.

The cST is invariant to translations, as a translation of an N × N image by

(Δx, Δy) results in the multiplication of the Fourier domain by a phase ramp, e−2πi(kxΔx+kyΔy)/N ). If one examines the magnitude spectra, these differences are 132 not apparent. Similarly, phase shifts due to translations are not an issue if one is interested only in applying magnitude filters and inverting. Only in the case of applying complex filters to complex images could image translation become prob- lematic.

The similarity of the cST spectra after cropping an image indicates that one can compare texture characteristics between images acquired with different FOVs. It also means that the cST should be robust to comparing lesions of different sizes, different patient head sizes or different slices of the brain that show a smaller or larger cross-section of the brain. I have shown in this chapter that the inversion formula is accurate for various image sizes and spectral extents.

Limitations of the cST include its redundancy and sensitivity. More redundancy increases computation time and decreases the resulting signal to noise ratio of the spectra. Therefore, the cST is rather slow for images larger than 256×256 (over

20 s with smoothing). However, the cST increases the sensitivity to change and allows one to narrow the bandwidth of interest and “focus in” on interesting spectral regions. As a result, one could employ a multi-stage analysis that ﬁrst calculates all of frequency components within an image for a preliminary analysis. Once the frequencies of interest have been determined, only those components that correlate with the disease model need to be calculated. The variability of the spectra can be reduced by performing an analysis of the entire image and then averaging the spectra within a prescribed ROI. 133

6.5 Conclusions

The cST extends the utility of the pST by providing a rotationally-invariant transform that is faster to compute, although it can still be quite lengthy for large images, particularly when smoothing. The cST is robust to cropping and translating of images and the mid- to high-frequency portion of the spectrum is robust to image non-uniformity. The cST provides the same redundancy as the pST.

The frequency maps of the cST require smoothing due to the annular shape of the localizing windows in the Fourier domain. Based on the results in this chapter, smoothing the frequency maps with a Gaussian of standard deviation σ =1.5/k is a good trade-oﬀ to reducing the ringing caused by the circularly symmetric windows used in the cST and maintaining the quadratic relationship between spectral amplitude and frequency. If the magnitude frequency maps are smoothed with a

Gaussian, the spatial “ripples” can be reduced, but the resulting cST domain is not invertible. If the complex data is retained, the cST can be used for applications such as non-linear ﬁltering.

The results from this chapter suggest that for simpliﬁed analysis, radial frequencies greater than the Nyquist (i.e. the “corner” frequencies) should not be used.

Only the ﬁrst N/2 points of the spectrum should be used in practice. cST spectra may be unreliable in the presence of a non-uniformity ﬁeld. Therefore, if image non-uniformity is a concern, image correction algorithms should be applied prior to analysis. 134

Chapter 7

Lesion Evolution in Multiple Sclerosis Using the Circular

S-Transform

A version of this chapter was published as a peer-reviewed article in the International

Journal of Computer Assisted Radiology and Surgery (CARS) in July 2008 [125]1.

An abstract based on this work was published in the proceedings of the 22nd In- ternational Congress and Exhibition of Computer Assisted Radiology and Surgery

(Supplement 1 of CARS, June 2008) [126]. I was awarded a Graduate Conference

Travel Grant from the University of Calgary to attend the CARS meeting. The poster presentation of this work received ﬁrst prize in the Computer Assisted Radi- ology — Image Processing and Display session of the poster competition.

As the ﬁrst author of publications described above, I was responsible for the overall concept of the work in this chapter, all coding and analysis, as well as writing and editing the manuscript. Dr. Ross Mitchell contributed in accessing the data, providing guidance and suggestions, as well as in the writing and editing of the manuscript.

1Reproduced with kind permission from Springer Science + Business Media: International Journal of Computer Assisted Radiology and Surgery, Texture quantiﬁcation of medical images using a novel complex space-frequency transform, 3, 2008, 465–475, S Drabycz and J Ross Mitchell (Appendix A). 135

7.1 Introduction

7.1.1 S-Transform Texture Analysis

MR imaging provides excellent soft tissue contrast. Therefore, it is often used to diagnose neurological diseases [127–131]. Radiologists are trained to identify subtle changes in the appearance of tissues on MR images and then use this information for diﬀerential diagnosis. For example, MR imaging is an important tool to help diagnose MS and monitor response to therapy [54,55,130–133].

As the disease state of brain tissues change, their MR appearance, or image

“texture”, may also change [98, 99]. We are investigating methods to quantify image texture using space-frequency transforms. These transforms quantitatively describe the magnitude of each spatial frequency component present in an image [105].

Changes in image texture can then be quantiﬁed by measuring the corresponding changes in spatial frequency power.

In previous work we have used a space-frequency transform, the ST [51], to show correlations between changes in a broad range of spatial frequencies in MR and the state of neurological diseases such as brain cancer and MS. Former studies were conducted by computing local spectra for small ROIs and analyzing the differences in amplitude of the corresponding spectral components. For example, we have identified spectral components that discriminate between two genetic sub-types of brain tumors: one that is chemosensitive and one that is chemoresistant [57]. Texture analysis studies on humans [7, 53–55] and animals [52] have shown that spectral differences in MR are associated with MS lesion evolution and pathology. The results from these studies suggest that spatial frequency information may provide a 136 sensitive and specific indication of disease activity.

However, the clinical utility of ST-based texture analysis is limited by its large computational and storage requirements [56]. Consequently, ST analysis is often limited to a small ROI within the image that contains an area of suspected pathology. However, spatial frequency resolution is inversely proportional to the FOV of the ROI being examined: Δk =1/FOV. Therefore, small ROI analysis has the drawback of poor spatial frequency resolution. Poor spectral resolution limits our ability to examine subtle spectral changes that may correlate with disease progression. Furthermore, a small ROI limits our ability to analyze widespread sub-clinical abnormalities and predict changes before they are visually apparent in conventional

MR images.

Here we describe a novel, efficient method of calculating spatial frequency distributions, known as the circular S-transform (cST). The cST can rapidly determine the spatial frequency content of an entire image, and thereby provide the maximum possible spectral resolution while providing information on large, or diffuse, regions of suspected pathology. This allows us to rapidly examine texture features of neurological pathology and look for spectral components that discriminate between different disease states, or that change with disease progression or treatment.

Once such regions are identiﬁed, our new transform can be ﬁltered and inverted to produce new images that contain only the spatial frequencies of interest.

7.1.2 Clinical Application: Multiple Sclerosis

MS is an autoimmune disease that causes both focal and diﬀuse pathology in the central nervous system. MR is the principal imaging modality used to diagnose and 137 monitor MS. MS lesions are considered “active” if they enhance on T1-weighted

MR imaging after administration of a gadolinium-based contrast agent. Contrast- enhancing lesions signify a breakdown of the blood-brain barrier. Active lesions enhance on MR images for 3 weeks on average [134]. On the other hand, “inactive” lesions do not enhance on T1 imaging. Both active and inactive lesions may appear hyperintense on T2-weighted MR images for extended periods, up to 5–6 months

[135].

Previous studies have shown the temporal T2 intensity to be a sensitive indicator of MS disease activity [135–137]. However, T2 intensity is not speciﬁc and may indicate inﬂammation, demyelination, gliosis, edema or axonal loss [133,138]. Nev- ertheless, early detection of disease activity could allow earlier treatment leading to improved long-term outcomes. Therefore, there is a need for even better sensitivity to subtle changes in the MR appearance of patients with MS.

We have previously shown a correlation between changes in LFE of MR images and changes in MS lesions using spectral distributions derived using the pST and equation (5.4). In previous work, LFE has been defined as the area under the magnitude pST spectrum below a threshold frequency. For example, the portion of the spectrum with the greatest differences between NAWM and MS lesions was found in one study to be below 5 cm−1 [7]. Another study showed that frequencies below 2.88 cm−1 change markedly during MS lesion development. A 31% increase in LFE was observed when tissues evolved from NAWM to an active lesion. When lesions subsequently became inactive, this change was accompanied by a significant decrease in LFE. However, the LFE of inactive lesions remained 12% higher than in

NAWM [55]. These studies suggest that spatial frequency information may provide 138 a sensitive and precise indication of disease activity and the potential to evaluate therapeutic responses from MS patients in clinical trials and to predict lesion activity on T2-weighted MR images.

However, the large number of computations required to calculate the ST has thus far limited analysis to small ROIs. As a result, the local spectra have had poor frequency resolution, severely restricting the amount of useable information extracted from the ST. For example, the study mentioned above used an ROI of size 3.125 cm, resulting in a spectral resolution of Δk =0.32 cm−1 [55]. As a result, the spectral components below 2.88 cm−1 that were involved in lesion development consisted of only 8 non-zero values for each pixel. The efficiency of our new algorithm means that we can analyze entire images, increasing the number of samples available over the same space-frequency interval. This means that we can more precisely measure the contribution of spectral components within specific frequency bandwidths. Also, because the low frequency spectral values depend on the local neighborhood, the act of cropping out an ROI means that the low frequency values will change. As a result, we expect to see differences in LFE with varying ROI size [120].

In previous work there has been no attempt at defining a cut-off in LFE between normal and abnormal tissues. This is largely due to the difficulty in comparing LFE values between patients because the actual values in normal and abnormal tissue may be different. Part of the difficulty arises due to the fact that the pST produces only a magnitude spectrum, as shown in (5.4); therefore, all spectral values are positive. Using our new approach, for the first time we can apply filters to the raw complex spectrum and invert to create images containing only the frequency bandwidth of interest. For example, we can create maps of the portion of image 139 intensity associated with the LFE bandwidth. We anticipate that this may allow us to better discriminate between normal and pathological tissues.

7.2 Materials and Methods

To demonstrate the utility of our new technique, we analyzed MR scans from two relapsing-remitting MS patients involved in a larger MS treatment trial at our in- stitution [139]. Patients in the trial were imaged on a 3T MR scanner (Signa, GE

Healthcare, Waukesha, WI). Axial T2 (TE=80ms) and PD (TE=30 ms) weighted images were obtained (TR=2716 ms, FOV=24 cm, matrix size = 512×512, slice thickness=3 mm, no gap). MR scans were taken at monthly intervals for 9 months:

3 months before treatment (months -3 to -1) and for 6 months after the start of treatment (months 1 to 6). Treatment was initiated in month 0. T1-weighted images (TR/TE=650/8 ms) were acquired before and 5 minutes after the intravenous injection of gadolinium contrast agent (0.1 mmol/kg) to determine which lesions were enhancing.

The T1 post-contrast baseline images were co-registered to the T2 baseline images using in-house rigid registration software using a normalized mutual information similarity metric. All other timepoints for the three contrasts were then registered to their respective baseline images with a rigid registration that minimized the sum of squared diﬀerences. The PD-weighted images were not registered to the T2 images since these were acquired simultaneously. We selected slices from the T1 post-contrast volumes that contained a new enhancing lesion that appeared over the course of imaging. These slices (two from patient #1 and one from patient 140

Table 7.1: New enhancing lesions that occurred over the course of 9 months of serial MR scanning. Note that month 0 corresponds to the start of treatment. Lesions were identiﬁed on T1 post-contrast images. segmentation (# pixels) image patient # enhancing average lesion border NAWM hyper- at month radius intense (mm) border? A 1 +1 3.0 166 299 6920 Y B 1 -2 2.9 123 284 5479 Y C 2 -2 1.8 49 223 21072 N

#2) were extracted from the volume for each contrast. The lesions are described in

Table 7.1.

Any bias field effects in the T2 and PD-weighted images were removed using the N3 algorithm [121] in MIPAV [86]. Variations in signal intensity between T2- weighted images were corrected by calibrating the image intensity to the average signal within ventricular CSF. Image signal intensity was normalized by segmenting the anterior horn of the lateral right ventricle using a level-set contour in mipav, dividing the image by the average voxel intensity in the contour and promoting the result from unsigned 16-bit integer to 32-bit floating point values. This correction is necessary since MR images do not have quantitive units; similar methods of scaling

MR images have previously been used in texture analysis of MS lesions [53] as well as functional MR studies [140].

The center of the lesion was defined as the center of mass of the homogenous, hyperintense region on the T2-weighted image. The location of the lesion was verified by segmenting the lesion in the T1 post-contrast image and overlaying the ROI on the T2-weighted image. An area of heterogenous, slightly less hyperintense T2 141 intensity of approximately uniform thickness surrounding the lesion was selected as the lesion border on the two slices from patient #1. The lesion from patient #2 did not have a visible border, so we manually outlined an area of approximately the same thickness (2.5 mm) around the lesion and defined this as the lesion border.

An example of the regions denoted as lesion and border are shown in Fig. 7.1.

We identified a 32 × 32 pixel ROI surrounding each lesion and its border. The location of one of the ROIs is shown in Fig. 7.4. A representative sample of NAWM was obtained by performing a single channel fuzzy C-means segmentation [141] of the PD-weighted images in mipav. The PD-weighted images were used for their good contrast between gray and white matter. We used four classes for the segmentation, representing white matter, gray matter, CSF and pathological tissue. The brain was extracted from the skull using a level-set contour prior to segmentation. We defined the average NAWM as pixels that were segmented as white matter at all timepoints. We then performed a morphological open operation, consisting of one erosion and one dilation with a 4-connected neighborhood, to remove any spurious pixels. The final segmentation was overlaid on the PD-weighted image to visually verify the segmentation. The results of the segmentation for image A, overlaid in red on the month-1 PD-weighted image, are shown in Fig. 7.1.

Since we were limited to ROIs for the pST analysis, to get a representative area of NAWM we chose six 32 × 32 pixels ROIs from the anterior, posterior and internal white matter regions (left and right hemispheres), shown in Fig. 7.1. We calculated the pST spectrum for each pixel in each of these areas, and averaged the magnitudes together to get an average pST spectrum for NAWM. For the lesion and border, we took the pST of the 32 × 32 ROI surrounding the center of mass of 142

Figure 7.1: The results of the NAWM segmentation, shown in red, overlaid on the month-1 PD-weighted image A. The lesion is shown in blue and the peripheral region, denoted as the lesion border, in black. The dashed yellow boxes show the 32× 32 ROIs used to get a representative sample of NAWM for the pST calculations. 143 the lesion and averaged the magnitude spectra of pixels within the area deﬁned as lesion and border, respectively. Using our new method, we calculated the spectrum for every point within the image, then computed the average magnitude spectrum for all points within the areas deﬁned as lesion, border, and NAWM, respectively.

We examined the average LFE and T2 signal intensity, as well as relative changes, over time. We calculated the complex spectrum of each pixel in the image slice at months -3 to 3 and month 6, according to (5.11) using the localizing window in

(5.9). Images were cropped to size 428×428 to exclude regions outside the brain. To remove any spatial artifacts caused by the localizing window in our new method, we convolved each magnitude frequency image |cST (∗, ∗,k)| with a Gaussian function of standard deviation σ=1.5/k before analyzing the spectra. For comparison, we also evaluated the pST of the 32 × 32 pixel ROI described above. Spectral analysis was performed on a 2.4GHz Intel Core 2 Duo MacBook Pro (Apple Inc., Cupertino,

CA) in Python 2.5 (python.org).

We isolated the signal components within the LFE bandwidth by multiplying each raw (unsmoothed), complex local spectrum by a box-shaped band-pass ﬁlter

G(x, y, k) such that: ⎧ ⎪ ⎨ 1 for 0.8 cm−1 ≤ k ≤ 3.6cm−1 G(x, y, k)=⎪ (7.1) ⎩ 0 otherwise and inverting the ﬁltered transform according to (5.14). The central lobe of the circularly symmetric window in the spatial domain for the lowest frequency in the

LFE bandwidth (0.8 cm−1) computed according to (5.9) has a full width half max of approximately 6 mm. That width corresponds to the diameter of the largest lesion.

Therefore, the largest structures we expect to focus on are of the order of the lesions 144 themselves; the details within the lesions, corresponding to the higher frequencies in the LFE bandwidth, are where we expect to find textural differences. The lowest frequencies were left out of the analysis since these correspond to structures much larger than the lesions, and therefore tend to be similar for all points within the image. The upper limit was chosen as a trade-off to include spectral areas previously shown to be important in MS lesion evolution [54,55] while minimizing the effect of noise, which tends to dominate the high spatial frequencies.

We veriﬁed the accuracy of the forward and inverse transform by measuring the root mean squared (RMS) error between each image and its inverted complex spectrum (without a ﬁlter), according to (5.12).

7.3 Results

Figure 7.2 shows the average magnitude spectra using both the pST and our new method for the lesion in image A at month 1. Figure 7.3 shows the spectra for all images, zoomed into the LFE bandwidth to highlight low frequency diﬀerences.

Low frequency differences between the pST and our new method are evident in the spectra; these differences are caused by the different local neighborhood and wrap- around effects when examining an ROI as compared to the entire image. Our new method can incorporate all information from structures surrounding the area being examined. Other differences occur due to the slightly different shape of the window with our method as compared to the pST. The spectral resolution of our method was 0.05 cm−1, much higher than that of the pST (0.67 cm−1). The improvement in spectral resolution will allow us to better identify differences in the shape of the 145

(a)

(b)

(c)

Figure 7.2: The average local spectra within (a) an enhancing lesion, (b) the lesion border, and (c) NAWM (month 1 of image A), using the traditional pST technique with a 32 × 32 ROI (squares) and our new method with a 428 × 428 pixel ROI (circles). The shaded region shows the LFE range of 0.8 cm−1 ≤ k ≤ 3.6cm−1. 146 magnitude spectra (Fig. 7.3). When using the pST, the lesion and border are nearly inseperable.

Figure 7.4 shows the T2-weighted image A at month 1, along with the 32 × 32 pixel ROI of the enhancing lesion. The ROI time-course shows the lesion evolution over 9 months in the T2-weighted image. The LFE map was generated by removing all frequencies outside the LFE bandwidth and inverting. Therefore, only the

T2-signal components thought to be involved in MS lesion development appear in

Fig. 7.4. The ﬁgure shows that the LFE magnitude increases in the core of the lesion upon enhancement, but decreases and becomes negative in the lesion border.

Figure 7.5 shows the time course for of LFE and T2 within the lesion, border and

NAWM. The ﬁgure shows that while the LFE of NAWM remains close to zero, there is an increase in LFE within the lesion upon enhancement. This increase seems to be delayed in images A and B, maximizing the month after enhancement. After enhancement, the LFE signal decreases, but remains above the baseline in month

9 for all lesions. The hyperintense lesion borders (A and B) drop to a negative

LFE value upon enhancement, with a rise towards baseline over the subsequent months (Fig. 7.5a, c). In comparison, the T2 intensity of the borders (Fig. 7.5b, d) show a small increase upon enhancement and an immediate return to baseline.

This presents a simple way to normalize baseline lesion intensity and suggests that the LFE measure is close to zero in tissue that does not contain any lesions. The

LFE signal of the tissue surrounding the lesion in image C, where the border is not hyperintense, is very close to that of NAWM (Fig. 7.5e).

Using the cST, we were able to calculate the complex local spectra of each pixel of the 428×428 image in approximately 1 min. Performing the smoothing operation 147

0.10 0.10 lesion lesion border border NAWM NAWM

0.05 0.05 spectral amplitude spectral amplitude

0.00 0.00 1.0 1.5 2.0 2.5 3.0 3.5 1.0 1.5 2.0 2.5 3.0 3.5 (a) frequency (1/cm) (b) frequency (1/cm)

0.10 0.10 lesion lesion border border NAWM NAWM

0.05 0.05 spectral amplitude spectral amplitude

0.00 0.00 1.0 1.5 2.0 2.5 3.0 3.5 1.0 1.5 2.0 2.5 3.0 3.5 (c) frequency (1/cm) (d) frequency (1/cm)

0.10 0.10 lesion lesion border border NAWM NAWM

0.05 0.05 spectral amplitude spectral amplitude

0.00 0.00 1.0 1.5 2.0 2.5 3.0 3.5 1.0 1.5 2.0 2.5 3.0 3.5 (e) frequency (1/cm) (f) frequency (1/cm)

Figure 7.3: The magnitude spectra for images A (a,b), B (c,d) and C (e,f) using the traditional pST method (left column) and our new method (right column). The axes are zoomed in to show the low frequency range. The errorbars represent the standard error of the mean. Note the improvement in spectral resolution using our method, and the improved distinction between lesion and border regions. 148

a month -3 month -2 month -1 month 0 month 1 month 2 month 3 month 6 b

Figure 7.4: (a) The T2-weighted image of patient #1 at month 1 when lesion A in the left hemisphere enhances. The red square shows the location of the 32 × 32 pixel ROI used for closer examination. (b) The time-evolution of T2 signal intensity within the ROI surrounding the enhancing lesion. (c) LFE maps generated by applying our new transform to the T2-weighted images, filtering with the band-pass filter G(x, y, k) and inverting. to calculate the magnitude spectra added approximately 1.5 min to the calculation time. However, these times may be reduced by implementing the algorithm in a more efficient programming language such as C. The complex spectrum consists of √ a maximum of N × N × (N/ 2) complex values. Therefore, transformation of each

428 × 428 image requires approximately 443 MB of storage for the complex array, or 221 MB for the magnitude spectra. Analysis of the same sized image using the traditional ST would not have been possible on the same computer hardware — the

ST would have taken several hours to compute and would have required over 250

GB of storage [56]. The pST can be computed without storing the intermediate 4D array of the traditional ST [71], and therefore the storage requirements would be similar to our new method. However, several hours of computation would still be required for the pST of the same sized image. Analysis of a 32 × 32 pixel ROI using the pST took approximately 1 s. The mean RMS error between the inverted and original images was 2.3 × 10−10 (maximum = 4.4 × 10−10, minimum = 5.8 × 10−12). 149

0.2 1.0 lesion lesion border border NAWM 0.8 NAWM 0.1

0.6

LFE signal 0.0 T2 intensity 0.4

-0.1 0.2 -2 0 2 4 6 -2 0 2 4 6 (a) time (months) (b) time (months)

0.2 0.8 lesion lesion border border NAWM NAWM 0.1 0.6

LFE signal 0.0 0.4 T2 intensity

-0.1 0.2 -2 0 2 4 6 -2 0 2 4 6 (c) time (months) (d) time (months)

0.2 0.8 lesion lesion border border NAWM NAWM 0.1 0.6

LFE signal 0.0 0.4 T2 intensity

-0.1 0.2 -2 0 2 4 6 -2 0 2 4 6 (e) time (months) (f) time (months)

Figure 7.5: The average LFE (left column) and T2 (right column) intensity changes for lesions A (a,b), B (c,d) and C (e,f) that enhance in months 1, -2, -2, respectively, the lesion borders and NAWM. LFE signal is calculated by ﬁltering our new transform to isolate only signal components within the LFE bandwidth. Error bars represent standard error of the mean. 150

7.4 Discussion

MS patients involved in clinical trials of new treatments have MRI scans at frequent intervals, often every 4 weeks. The duration of frequent scanning varies, depending upon the trial protocol, but often involves a run-in period of several months prior to treatment initiation, then extending from 6 to 24 months on treatment. Patients not enrolled in clinical trials, and those who have completed the frequent scanning phase in a clinical trial, will have annual MR exams. Patients also typically receive an additional, immediate, MR exam after a clinically conﬁrmed relapse, or worsening, of their symptoms. Repeated MR exams are an important component of clinical monitoring of disease progression. Therefore, the methods we have developed are directly relevant to real-world clinical scenarios.

The bandwidth of 0.8 to 3.6 cm−1 used in our study roughly corresponds to structures of diameter 1.4 to 6 mm. Figure 7.6 shows an example of the raw cST corresponding to a frequency within the LFE bandwidth of k =3cm−1; the × indicates the location of the lesion. The ﬁgure shows that areas in the brain that contain edges and other structures at the corresponding scale, including the lesion, appear bright.

The ring-shaped localizing window we use tends to preferentially seek out circularly shaped structures, such as lesions. By ﬁltering all frequency components, except those within this bandwidth, we are removing the eﬀect of all low-frequency components (the signal intensity in slowly varying, large-scale structures) and high- frequency components (the rapidly varying intensities associated with noise). This results in very low signal intensity in regions of NAWM and bright intensity of le- 151

Figure 7.6: An example of the raw cST of image A at month 1 (k =3cm−1). The × indicates the location of the lesion. Areas in the brain that contain edges and other structures at the corresponding spatial scale appear bright. 152 sions and borders between structures of diﬀering signal intensity. The improvement in spatial resolution may help to deﬁne a narrower bandwidth for examining spectral changes in MS. Perhaps in future studies we could use an approximate measure of lesion diameter to guide the LFE bandwidth used in texture analysis.

Our results are consistent with earlier observations that LFE increases when a lesion becomes active and decreases when a lesion becomes inactive (but stays above baseline) [55]. Fig. 7.5 indicates that the peak in LFE appears delayed, as compared to the T2 signal. For example, the maximum LFE value for image A within the lesion and the minimum value in the border occur in month 2, while the lesion becomes enhancing in month 1. This has not previously been observed. For lesion

C, the LFE in the border region is very similar to that of NAWM, while in lesions

A and B the border region shows a drop in LFE from baseline upon enhancement.

This suggests that the drop is due to the presence of a hyperintense lesion periphery.

We noticed diﬀerent behaviour within the active lesion compared to the surrounding tissue. While the central region of the lesion experienced an increase on the LFE-ﬁltered images upon enhancement, the peripheral region saw a drop in LFE that slowly returned to baseline over time. Conversely, on the T2-weighted MR image, the lesion border had a small, brief increase in T2 signal intensity followed be a rapid return to baseline. Also, while the lesion and ventricle containing CSF appear bright on T2-weighted imaging, the ventricle appears to have a negative amplitude along the edge at the lower end of the LFE map range, with a positive amplitude at the high end of the range (Fig. 7.4).

Our results suggest that by ﬁltering the LFE we may be more sensitive to the “target” appearance of lesions, which have a hyperintense center represent- 153 ing demyelination with a slightly less hyperintense periphery representing vasogenic edema [142], than traditional T2 time-series analysis. However, our method also highlights intensity changes that may correspond to other structures in the brain.

For example, Fig. 7.7 shows the LFE-ﬁltered image for the entire image A at month

1. It is apparent that the cST highlights intensity changes between structures of diﬀerent intensity, including lesions as well as tissue borders and surrounding CSF.

As a result, structures within the brain, such as lesion and ventricle, seem to be surrounded by a strongly negative border. Part of this appearance may also be due to ringing artifact from the box-shaped ﬁlter used to isolate LFE frequencies.

The shape of the window used in the space-frequency analysis also contributes to this eﬀect. The multiplication of the Fourier domain of the image by the ring- shaped window in (5.11) implies a convolution of the image domain with a sinc-like function that oscillates between positive and negative values. The centre of the function has in a high positive value, while the negative side-lobes result in negative

LFE values. This diﬀerence may help to separate lesions from ventricular CSF, which can be challenging on conventional imaging [132]. The region of decreased

LFE surrounding lesions may indicate areas where tissue damage is transient and quickly repaired. Further investigation on a large patient population is required.

In future studies we will examine the eﬀect of narrowing the bandwidth to improve the speciﬁcity of the analysis. Future analysis will also expand on the ability of our technique to separate the behavior of active MS lesions and surrounding tissue. For future analysis we also plan to include some of the advanced image correction techniques, such as non-linear registration, partial volume correction, and tissue-class-based intensity normalization employed in [136]. We also plan to extend 154

Figure 7.7: Image A at month 1, filtered to only retain the frequencies in the LFE bandwidth. Note that while areas corresponding to lesion activity are highlighted using this technique (white arrow), borders between brain tissue and CSF are also highlighted. This is likely due to the shape of the window and filter used, but may help to separate demylinating lesions from CSF and transient damage. our spectral analysis to other MR contrasts, such as T1-weighted and FLAIR images and also from single image slices to entire image volumes. Work is ongoing to examine the effect of ROI size on measures of low spatial frequency [120].

7.5 Conclusion

In this chapter we have presented an eﬃcient, invertible transform that can calculate the local spatial-frequency content of large images on a pixel-by-pixel basis 155 in approximately one minute. Our technique results in higher spectral resolution than previous ST-based texture analysis methods. The improved frequency resolution may allow us to better identify spectral bandwidths that discriminate between normal and pathological tissues.

Our new transform produces complex spectra, which allows us to not only derive information regarding the texture or spatial frequency content, but to also filter and invert the spectra to produce frequency-specific images such as LFE maps. In this chapter we used a simple filter to extract spatial frequency components that have been found to correspond to MS lesion activity. In the future, advanced filters could be developed to selectively enhance or suppress specific frequency components on a pixel-by-pixel basis. Different filters could be used to examine lesion tissue vs. border and normal-appearing tissue. Such an approach may help in examining early, diffuse changes in MS, as well as monitoring disease progression and response to therapy.

Our results, while limited to only two patients, suggest that our new technique may be useful in extracting lesion dynamics not readily observable in T2-weighted images. We observed negative LFE values in the hyperintense boundaries of an active lesion. This may help to discriminate between the core lesion undergoing demyelination and a border of inﬂammation. Therefore, our new space-frequency based texture analysis may help to distinguish permanent damage from acute activity. Further analysis on a larger data set will be important to evaluate the eﬃcacy of this new technique in evaluating MS lesion dynamics. 156

Chapter 8

Texture Analysis Using the 2D Discrete Orthonormal

S-Transform

A version of this chapter was published as a peer-reviewed, Open Access Online First article in the Journal of Digital Imaging in August 2008 [143]. An abstract based on this work was published in the proceedings of the 2008 Annual Meeting of the Society for Imaging Informatics in Medicine (SIIM) [144] and was selected for a podium presentation; I was awarded a New Investigator Travel Award to attend the SIIM meeting. The work in this chapter was also the basis of a poster presentation at the

Apple Computer Worldwide Developers Conference (WWDC) in San Francisco, CA

(June 2007), for which I received a Student Scholarship to attend the conference, and a podium presentation at the University of Calgary Engineering Graduate Student

Conference (April 2007) [145], which won a best presentation award in the Signal and Information Processing session. The intellectual property in this chapter has been transferred to Calgary Scientiﬁc Inc., which holds a provisional patent based on this work.

Dr. Robert Stockwell provided assistance in developing the frequency-domain

DOST and answered technical questions regarding his previous work with the spatial- domain 1D DOST. He also contributed as a co-author to the manuscript based on this chapter by explaining some of the technical and mathematical concepts, contributing to and editing the manuscript. Ross Mitchell was the senior author on this 157 work and provided technical assistance, editing and contributions to the background and technical information as well as helping to interpret and present the results.

8.1 Background

Texture characterization is an important problem in medical image analysis. Image texture can be deﬁned as the spatial relationship of pixel values in an image region

[95]. In medical images, texture can be thought of as the local characteristic pattern of image intensity that identiﬁes a tissue. Texture also determines local spectral or frequency content in an image; changes in local texture should cause changes in the local spatial frequency. Texture analysis is of interest in medical imaging because as biological tissues become abnormal during a disease process, their underlying texture may also change. [98] and [99] provide good reviews of the application of texture analysis methods to medical images.

Various mathematical techniques to quantify image texture, including statistical,

Fourier and wavelet-based methods, have been applied to radiological images of numerous pathologies, such as multiple sclerosis [102,146], brain tumors [100], liver diseases [101], infarcted myocardial tissue [147], normal tissues of the knee [148] and has even been used for automated detection and classiﬁcation based on phase- contrast microscopy images, such as those used in cervical cancer diagnosis [149].

A texture feature is a value that quantiﬁes some characteristic of local intensity variation within an image [95]. A variety of approaches exist to quantify texture. A common technique used in medical imaging is based on co-occurrence matrices [4].

Statistical measures of texture are calculated based on the frequency of specific 158 gray levels occurring between pairs of points within an image. The co-occurrence technique has been used in many studies, such as to classify benign and malignant solitary pulmonary nodules on CT images [150] and to quantify pathological changes during treatment of multiple sclerosis [151]. However, the technique is limited to very small neighbourhoods due to its computation complexity. Therefore, only the highest-frequency textures can be analyzed. Broad, large-scale changes are difficult to detect using co-occurrence statistics. Furthermore, the resulting statistical measures are difficult to interpret and compare across patients.

A more recent method of texture analysis relies on the DWT (see [30] for a review). Wavelets provide a multi-scale representation of an image, allowing analysis of varying degrees of detail within an image. Eﬃcient algorithms and a solid mathematical framework make wavelets appealing for numerous applications, including texture analysis. For example, wavelet-based texture analysis has been used for automatic diagnosis and grading of breast tumor histology images [152].

The ST [40] is closely related to the CWT using a complex Morlet mother wavelet [59] and directly measures the local spatial frequency content of each pixel in an image. The ST has been successfully used to analyze signals in numerous applications, such as seismic recordings [42], ground vibrations [153], hydrology [154], gravitational waves [43], and power system analysis [44]. The 1D ST has shown to be a useful tool for analysis of medical signals, such as electroencephalography [46], functional MR imaging [50] and laser doppler ﬂowmetry [47]. The ST is particularly well suited to texture analysis of medical images due to its optimal space-frequency resolution and close ties to the FT — the basis of medical image reconstruction.

The ST uses complex Fourier basis functions, modulated by frequency-dependent 159

Gaussian windows. The ST preserves the phase information, uses a linear frequency scale and can be easily inverted to recover the Fourier domain of an image.

The redundant nature of the ST algorithm has been the main obstacle in wider application of ST-based texture analysis for 2D images. Extensive processing time and a large amount of memory are required to calculate and store the texture de- scriptions of large medical images. The 2D-ST of an array of size N × N has computational complexity of O[N 4 +N 4 log(N)] and storage requirements of O[N 4].

As a result, the ST of a typical 256 × 256 MR image takes approximately 1.5 hours to calculate on a single computer and requires 32 GB of memory [56]. While other researchers have developed techniques to distribute these calculations over networks of machines [56], the results are still diﬃcult to manage and interpret [61]. There- fore, previous work on 2D images has been limited to analysis of small ROIs, and typically collapsed to 1D spectra [52,53,57]. However, small ROIs reduce the resolution of the frequency spectra, and therefore the sensitivity to subtle texture changes.

These requirements make applying the 2D-ST to clinical medical applications diﬃ- cult and impractical. Clinical texture analysis requires a rapid, eﬃcient algorithm that provides complete information about all frequency components.

Despite these limitations, the 2D-ST has shown promising results in identifying diﬀerences in texture that correlate with neurological pathology. For example, previous work has shown that ST-based texture measures can help detect sub-clinical abnormalities in NAWM of MS patients [51]. The 2D-ST has also been used to identify oligodendroglial brain tumors with genetic abnormalities that make the tumors more responsive to chemotherapy treatment [57].

Recent work shows that a discrete, orthonormal basis can be used to accelerate 160 calculation of the ST and eliminates the redundancy in the space-frequency domain

[3]. The DOST provides a spatial frequency representation similar to the DWT.

However, the DOST has the additional beneﬁts of maintaining the phase properties of the ST (and FT), retaining the ability to collapse exactly back to the Fourier domain. Furthermore, the DOST framework allows for an arbitrary partitioning of the frequency domain; this allows for a dyadic sampling similar to the discrete wavelet transform for zero redundancy, or oversampling to any extent, all the way back to the fully redundant ST.

We present a frequency-domain implementation of the 2D-DOST by partitioning the frequency domain in a dyadic sampling scheme. We show that the DOST can provide a pixel-by-pixel texture description of an image by creating local spectra containing the horizontal and vertical frequency information from the FT of the image. The DOST is straightforward and fast to calculate, allowing us to analyze every pixel in a large image within seconds. Our goal in this chapter is to introduce the multi-dimensional DOST, describe its application to texture classiﬁcation and to characterize its performance on real and synthetic images.

8.2 Theory

8.2.1 1D Discrete Orthonormal S-Transform

As with the ST, we can calculate the 1D DOST in the frequency domain to reduce the number of computations required. We start by taking the FT of the basis functions in (2.52) to determine the spectral partitioning. Fig. 8.1 shows that the magnitude FTs of the basis functions, calculated numerically, for seven orders of 161

120

100

60 amplitude 40

0 0.0 0.1 0.2 0.3 0.4 0.5 frequency

Figure 8.1: Spectral partitioning of the 1D-DOST for seven orders (N = 128). The scaled and shifted rect functions are the FTs of the sinc basis functions deﬁned in (2.52). the DOST are shifted and scaled rect functions. Therefore, (2.52) must be a sinc function.

In order to see how the basis functions in (2.52) are sinc functions, consider the general form of the DOST basis functions for a frequency ν and bandwidth β [3],

− −πiτ f=ν+β/2 1 e t − τ b [t]= √ e2πi( N β )f . (8.1) [ν,β,τ] β f=ν−β/2

We can then compare (8.1) to the complex integral form of the sinc function, for 162 x = 0 [155], 1 n sinc(nx)= eixtdt. (8.2) 2n −n If we consider a time-shift of zero (τ = 0) and remove the frequency shift by ν,we see that (8.1) does indeed resemble a discrete sinc function, with n = β/2, scaled √ by β f=β/2−1 1 b [t]=√ e2πitf/N . (8.3) [ν=0,β,τ=0] β f=−β/2 The eﬀect of τ is to apply a phase term and ν translates the rect function.

Therefore, we can calculate the DOST voices in the frequency domain simply by taking the inverse FT of scaled and shifted portions of the signal spectrum. Consider the case where we use a dyadic sampling scheme. The DOST of an N-point signal f[t], sampled at integer locations t =0, 1,...,N−1 at order p is calculated according to 2p−2−1 1 ˆ n 2πinτ/2p−1 Dp[τ]=√ fp e (8.4) 2p−1 N n=−2p−2 ˆ where fp is the part of the FT of f encompassing the bandwidth β corresponding to the order p>1, shifted by β/2, ⎧ ⎪ p ⎨ fˆ n+2 for n = −2p−2 to −1 ˆ n N fp = (8.5) ⎪ p−1 N ⎩ ˆ n+2 p−2 − f N for n = 0 to 2 1.

Equations (8.4) states that the spectrum of the signal is partitioned into log2 N sections; each section is shifted by β/2, scaled and the inverse FT is taken to get the values for each time translation, τ. The partitioning for seven orders in the frequency domain is shown in Fig. 8.1. We compute the negative frequencies using conjugate symmetry, in a method similar to the symmetric DOST described in [156]. 163

In order to verify that the frequency domain implementation of the DOST in

(8.4) agrees with the time-domain algorithm given in [3], I compared the results on the following synthetic signal (from Figure 10(a) in [3]) 6N t t f[t]=cos 2π 204 + cos 6π (8.6) 1+t N N where t =0, 1,...,N − 1 and N = 1024. Figure 8.2 shows that the frequency- domain implementation agrees with the time-domain version given in [3]. Note that while only the magnitude representations are shown, the phase also agrees.

The ﬁgures were generated by replicating the values within each time-frequency partition to obtain a N/2 × N array, illustrating the contributions of the positive

Fourier frequencies.

Figure 8.3 shows an example of the DOST applied to a signal sampled at 3.33Hz obtained from a fMRI study. These dynamic studies measure the change in signal over time as a subject repetitively performs a task in response to a stimulus. The signal is from a region of the brain expected to be involved in motor activity. In this experiment, the subject performed a 12 s task (moving their ﬁngers) then rested for a 24 s interval. This sequence was repeated 5 times. The hypothesis is that the areas of the brain that are responsible for reacting to the stimulus experience a change in oxygenated blood ﬂow, causing a change in the local magnetic properties of the tissue. By taking multiple images of the brain over time, the change in signal can be examined on a pixel-by-pixel basis as a series of time signals.

Time-frequency analysis provides a better understanding of frequency changes with time and can be used to remove artifacts and noise from the time-signals.

Previous work has shown that artifacts due to patient motion can be removed from 164

(a)

(b)

(c)

(d)

Figure 8.2: (a) A modulated sinusoid time-series from [3]. (b) The DOST calculated using the basis functions in (2.52) and (c) by partitioning the frequency domain, according to (8.4). (d) The full, redundant ST from (2.36). 165

(a)

(b)

(c)

Figure 8.3: (a) A signal from a functional MR imaging experiment, sampled at 3.33Hz. The plot shows the change in signal corresponding to ﬁve repetitions of a task 12s in duration (subject moving their ﬁngers), followed by a 24s rest period. (b) The ST calculated according to (2.36). (c) The DOST of the signal. fMRI signal using the ST [50,51].

Fig. 8.3 shows the ST and DOST of the fMRI signal. Note that only the ﬁrst 256 frequencies are shown in (b) and (c). This is due to the present implementation of the frequency-domain DOST, which is currently designed for signals whose lengths are a power of two. While both representations are somewhat diﬃcult to interpret, certain characteristics are evident in both domains. For example, regions with strong high frequency power show up as bright regions in both the ST and DOST. The 166 strong low-frequency power is evident in the lowest frequencies of both transforms.

Future work will examine time-frequency ﬁltering to remove high-frequency noise and phase artifacts using the DOST.

8.2.2 2D Discrete Orthonormal S-Transform

The process of calculating the DOST of a 1D signal in the time domain is described in

[3]. That process involves calculating the basis functions, which are derived by taking linear combinations of the Fourier complex sinusoids in band-limited subspaces, and applying appropriate phase and frequency shifts. Here we describe the process of calculating the DOST of a 2D image in the frequency domain using a dyadic sampling scheme.

We begin by deﬁning the forward 2D-FT of a discrete function f[x, y], which is assumed to have a sampling interval of one in the x- and y-directions,

M−1 N−1 m n − mx ny fˆ , = f[x, y]e 2πi( M + N ) (8.7) M N x=0 y=0 and the inverse 2D-FT

M/2−1 N/2−1 1 m n mx ny f[x, y]= fˆ , e2πi( M + N ). (8.8) MN M N m=−M/2 n=−N/2

The 2D-DOST of a N ×N image f is calculated by partitioning the 2D-FT of the image, fˆ, multiplying by the square root of the number of points in the partition, and performing an inverse 2D-FT. For given orders px,py > 1 we extract the part of the Fourier spectrum where m =2px−1 to 2px − 1 and n =2py−1 to 2py − 1 and 167 perform a circular shift by half of the bandwidth ⎧ ⎪ px py ⎨ fˆ m+2 , n+2 for m = −2px−2 to −1,n= −2py−2 to −1 ˆ m n N N fpx,py , = ⎪ px−1 py−1 N N ⎩ ˆ m+2 n+2 px−2 − py−2 − f N , N for m = 0 to 2 1,n= 0 to 2 1. (8.9)

We calculate the 2D-DOST by taking the 2D-FT of each scaled, shifted part of the

Fourier spectrum

2px−2−1 2py−2−1 “ ” ny 1 m n 2πi mx + √ ˆ 2px−1 2py−1 Dpx,py [x ,y]= fpx,py , e . (8.10) 2px+py−2 N N m=−2px−2 n=−2py−2

For real images, the negative frequencies can be computed through conjugate symmetry.

One should note that the operation to create the voice image Dpx,py [x ,y] consists of an inverse FFT of smaller dimension (not N × N). A 2px−1 × 2py−1 region of the image FT is extracted, shifted by (βx/2,βy/2) and a inverse 2D FFT is performed, resulting in a rectangular voice image of 2px−1 × 2py−1 points (see Fig. 8.4). This process is repeated for all orders px,py < log2 N. The total number of points in the

px−1 px−2 DOST domain and in the original image are the same. We deﬁne νx =2 +2

py−1 py−2 and νy =2 +2 as the horizontal and vertical voice wavenumbers [3]. Note that we can refer to the components of the domain in terms of: the frequency orders (px,py); the wavenumbers corresponding to the centre of the frequency band (νx,νy); or the spatial frequencies (kx,ky) since they have a simple relationship

νx kx = (8.11) NxΔx νy ky = (8.12) NyΔy 168 y ν 0 vertical voice frequency, vertical frequency, voice decreasing vertical scale

0 ν (a)decreasing horizontal scale (b) horizontal voice frequency, x

Figure 8.4: Partitioning of (a) the DWT and (b) the DOST for 6 orders. The squares indicate the sub-images for each order. Both transforms use a dyadic sampling scheme but provide diﬀerent information about the frequency content of the image. The DWT gives horizontal, vertical and diagonal “detail” coeﬃcients for each order, while the DOST provides information about the voice wavenumbers (νx,νy) that contain a bandwidth of 2px−1 × 2py−1 frequencies. 169

⎧ ⎪ px−1 px−2 ⎪ 2 +2 if px > 1 ⎪ ⎪ ⎨ |px|−1 |px|−2 −(2 +2 )ifpx < 1 νx = (8.13) ⎪ ⎪ 0ifpx =0 ⎪ ⎩⎪ 1if|px| =1 ⎧ ⎪ py−1 py−2 ⎪ 2 +2 if py > 1 ⎪ ⎪ ⎨ |py|−1 |py|−2 −(2 +2 )ifpy < −1 νy = (8.14) ⎪ ⎪ 0ifpy =0 ⎪ ⎩⎪ 1if|py| =1. The inverse transform for the 2D-DOST is similar to the 1D case in [3]. Since the

DOST is an energy-conserving transform [3], we can apply the forward 2D-FFT to each shifted voice order in order to reverse the spectral partitioning and reconstruct the spectrum of the image m n fˆ , = px,py N N p −1 p −1 » – x − y − p −2 py−2 2 1 2 1 (m+2 x ) (n+2 ) 1 −2πi p −1 x + p −1 y √ 2 x 2 y Dpx,py [x ,y]e . (8.15) px+py−2 2 x=0 y=0 Once each partition is inverted, the image can then be recovered by performing an inverse FT.

The DOST in this formulation, like the DWT, uses a dyadic sampling scheme − (orders px,py = 0, 1, 2, ..., log2 N 1) (Fig. 8.4). However, the two transforms provide diﬀerent information about the frequency content of the image. The DWT gives horizontal, vertical and diagonal “detail” coeﬃcients for each order, while the DOST provides information about the voice frequencies (νx,νy) that contain a bandwidth of 2px−1 × 2py−1 frequencies. While the wavelet decomposes the image 170 into “scales” of size M ×M, the DOST uses the minimum number of points required to describe the amplitude of a Fourier frequency component in each of the horizontal and vertical directions. For example, a Fourier frequency component with two cycles spanning the horizontal direction and four cycles spanning the vertical direction is represented in the DOST by 4 × 8 points. As a result, the DOST provides more texture features for a N × N image than the DWT, while maintaining an overall size of N 2. For example, Fig. 8.5 shows the DOST and DWT transforms of an image containing four textures from the Brodatz texture library [124]. Each texture section is size 128 × 128 pixels and has been normalized such that the average (DC) is zero. The DWT was calculated using the db4 wavelet at 5 levels of decomposition.

The diﬀerence in partitioning can be seen in this example.

8.2.3 Pixel-wise Local Spatial Frequency Description

Once we have the DOST description of an image, we can ﬁnd the contribution of each horizontal and vertical voice to any pixel within the image. By simply choosing a set of (x, y) coordinates representing a single pixel or image region, we can determine

the value of Dpx,py [x ,y] for all (px,py). Since the voice images are of varying shape and size, the value of the DOST for frequency order (px,py) at image location (x, y)

× px−1 × py−1 can be found at Dpx,py [x/N 2 ,y/N 2 ]. By iterating over all values of × (px,py), we can build up a local spatial frequency domain of size 2 log2 N 2 log2 N for each pixel or averaged region within an image; this domain contains the positive and negative frequency components from the DC, (νx,νy) = (0,0), to the Nyquist frequency, (νx,νy)=(N/2,N/2). We refer to the local domain for a pixel (x, y)as the local spectrum, Dx,y[νx,νy]. 171

(a) decreasing verticaldecreasing scale

(b) (c) decreasing horizontal scale

Figure 8.5: (a) A mosaic of four texture images: straw, wood, sand and grass. (b) The 2D-DOST of the mosaic. (c) The DWT of the image using the db4 wavelet at ﬁve levels of decomposition. Note that the square root of the amplitude of both the DOST and DWT coeﬃcients is shown for display purposes. 172

Since the voice images are of size (2px−1 ×2py−1), the mapping will not be unique for low-frequency components of neighboring pixels. For example, the px = py =2 contribution describes the low-frequency portion of the image using 4 pixels. There- fore, all pixels in a particular quadrant of the image will contain the same low- frequency information. In the extreme case, the px = py = 0 order will be represented by a single point for the entire image.

The local frequency domain is analogous to a local 2D Fourier domain, often referred to as k-space. This provides a way to measure the texture characteristics by examining the contribution of each horizontal and vertical frequency bandwidth within the image. These “texture descriptors” replace the 1D texture curves of the pST analysis used in [52, 53, 57]. If we use the dyadic sampling scheme presented above, the frequency scale is linear and the bandwidths do not overlap.

For example, Fig. 8.6 shows the local domain at the centre of each texture region in Fig. 8.5a. The local domain of straw (Fig. 8.6a) reﬂects the strong diagonal stripes present in the image. The ﬁgure also shows that wood (Fig. 8.6b) has more high frequency components oriented horizontally than vertically, while sand (Fig. 8.6c) and grass (Fig. 8.6d) have similar frequency distributions, with grass having slightly more high frequency energy.

8.2.4 Primary Frequency Component

In order to more easily determine the most signiﬁcant spatial frequency component at a particular point or region, we calculate the “primary” or highest amplitude frequency. This is done for a particular point by determining the value of each voice image in the DOST at that pixel location. We examine the value of each voice image 173 at location (x, y) and determine the component with the highest amplitude. The

(νx,νy) pair is the “primary frequency” for the pixel. We can extend this analysis to calculate the primary frequency at every image pixel. This results in a value of (νx,νy) for each value of (x, y) in the image. We can represent this as a complex image, with the real channel corresponding to the primary νx value and the imaginary channel corresponding to the primary νy value. The magnitude image can then be used to examine the primary radial frequency, 2 2 νr = νx + νy . The phase image tells us the angle of orientation of the primary frequency, θν = arctan(νy/νx). The DOST is normalized to preserve the length of the vector; like the FT it satisﬁes a Parseval theorem. Thus each voice is divided by the size of its partition which attenuates the amplitude of high frequency signals when dyadic sampling is employed. To obtain a domain where the relative contribution of each partition is constant, we remove the partition factor. This provides the equivalent scaling as the original ST.

8.2.5 Eﬀect of Spatial Transformations on the DOST

The DOST is rotationally variant because we calculate measures of local spatial frequency content in the horizontal and vertical directions. The result of a horizontal shift by Δx pixels and/or a vertical shift of Δy pixels results in a corresponding shift of each voice image. We can understand this eﬀect by considering the DOST process on a shifted image. A circular shift of the image by Δx pixels in x and

Δy pixels in y causes the phase of the FFT of the image to be multiplied by the complex value e2πi(xΔx+yΔy). The multiplied FFT is partitioned, and each partition 174 is inverse Fourier transformed. The phase ramp applied to each partition causes it to be shifted horizontally by Δx pixels and vertically by Δy pixels when inverse transformed. The multiplication of an image by a scaling factor causes the DOST to be multiplied by the same factor, as it is a linear transform.

In the case of a 90° rotation, the effect is a transpose, or a switching of the x and y values in the image and a reversal of the y-coordinate. As a result, each voice image has the x and y coordinates switched and the y-coordinate reversed. Similarly, for a 180° rotation, the effect is to reverse the y-coordinates and maintain the x- coordinates. The result on the DOST is to reverse the y-coordinates of each voice image; the x-coordinates are not affected. In the case of rotations of 0 <θ<90°, the effect is more complicated. A rotation of the image implies a rotation of its FT.

Therefore, when calculating the 2D-DOST of a rotated image the partitioning grid is applied to the rotated FT. As a result, the frequency components are rotated into diﬀerent partitions.

8.2.6 Rotationally Invariant DOST Features

In order to obtain rotationally invariant features from the DOST, we can follow a method similar to the invariant wavelet transform described in [5]. In that approach, invariant wavelet coefficients are computed by taking the wavelet coefficients at each level, averaged over the horizontal and vertical coefficients. Figure 8.7(a) shows the wavelet channels that are averaged together marked with the same letter (A=level

0, B=level 1, etc.). The diagonal channels are excluded from the feature extraction since they tend to contain the majority of the noise in the image and can degrade classiﬁcation performance [5]. 175

(a) (b)

Figure 8.6: The local DOST spectra of: (a) straw, (b) wood, (c) sand and (d) grass obtained from the center pixel of each region in Fig. 8.5a. 176

We can take a similar approach with the 2D-DOST by averaging together the magnitude of the horizontal and vertical frequency values for each order and excluding “diagonal” elements of the DOST domain where px = py. By combining the horizontal and vertical frequency information for the entire DOST we can obtain an invariant domain describing the entire image. This is useful when using the DOST to examine a small ROI. However, for medical imaging applications we would like to retain the spatial information as well as the frequency information of the DOST. In this case we can combine the horizontal and vertical components of the local DOST spectrum, described in Section 8.2.3.

Figure 8.7(b) shows an example of how the values are calculated from the local frequency domain for N = 8. The features marked with the same letter are averaged together to get an invariant DOST spectrum of texture features. For a given N, the DOST approach provides more texture features (A to J = 10) than the wavelet approach (A to D = 4).

8.3 Methods

We conducted a series of experiments to determine the response of the 2D-DOST to a wide range of frequencies, texture patterns and noise. These experiments attempted to provide a robust characterization of 2D-DOST based measures of texture and to determine the limits on our ability to measure subtle texture changes in images. We also evaluated the performance of the rotationally-invariant features in classifying a wide range of known textures and compared the results to those obtained using the invariant wavelet transform [5]. 177

JIHGHIJ

+3 FFEED J

D y F CCB F I

+1 E C A C E H

0 D B A A B D G C decreasing vertical scale

vertical frequency order, p vertical frequency order, E C A C E H D

B -2F -1 C B C F I C

A B -3F E D E +2F +4 J

decreasing horizontal scale -3-2 -1 0 +1 +2+3 +4 (a) (b) horizontal frequency order, px

Figure 8.7: (a) The wavelet decomposition for N = 8 at three levels of decomposition. Invariant wavelet coeﬃcients are calculated by averaging horizontal and vertical coeﬃcients at each level (marked with the same letter: A=level 0, B=level 1, etc.). The diagonal channels are excluded from the feature extraction. (b) The local frequency domain generated from the 2D-DOST for N = 8. Features marked with the same letter are averaged together to get an invariant DOST spectrum; a similar approach is taken of excluding the diagonal elements, where px = py.For a given N, the DOST approach provides more texture features (A to J = 10) than the wavelet approach (A to D = 4). 178

All DOST computations were carried out in Python 2.5 (http://python.org) using the Numerical Python package (http://numpy.scipy.org) and the Python Imag- ing Library (http://pythonware.com/products/pil). Wavelet coeﬃcients were calculated in Matlab (The MathWorks, Inc., Natick, MA) using wavedec2.m and det- coef2.m with the periodic extension mode. Rotated texture images were obtained from [124]. Analysis was performed on a 2.4 GHz Intel Core 2 Duo with 2 GB of

RAM running the Mac OS 10.5 operating system.

8.3.1 Transform Characterization

In order to determine the frequency response of the 2D-DOST, we calculated the point spread function. This was done by creating a “delta” image — a 256 ×

256 image where the pixel at location (128,128) had a value of one, and all other pixels had a value of zero. We took the 2D-DOST of the delta function and then computed the local spectrum. We also measured the average time to calculate the

DOST, and determined the amount of memory required to store the structure, for randomly generated N × N images where N varied from 4 to 1024. We compared the computation times and storage requirements to the ST. These calculations were based on the calculation of real images, where only half of the space-frequency domain needs to be calculated and stored due to Hermitian symmetry.

We tested the accuracy of the DOST inversion by taking the DOST of each of the

9 Brodatz texture images shown in Fig. 8.8, then inverting the 2D-DOST domain.

We measured the L1 norm between each inverted image, s1, and the original, s0

1 N N L = |s (i, j) − s (i, j)| . (8.16) 1 N 2 1 0 i=1 j=1 179

8.3.2 Response to Noise

To examine the changes in local DOST frequency domains with varying levels of noise, we evaluated a series of synthetic images with known frequency content and added noise. By varying both of these factors, we were able to determine the smallest change in spatial frequency, and contrast, that can be reliably detected. We expect these two factors to be related. That is, we expect that high spatial frequencies will be more reliably detected in images with high contrast-to-noise ratios.

The relationship between contrast and maximum detectable frequency is akin to the modulation-transfer-function used to characterize imaging systems, and should provide a robust characterization of the 2D-DOST.

We generated a series of images of size 256 × 256 with a single frequency component. We varied the frequency of a sinusoid of wavenumber ν =1, 10, 20,...,120, oriented either horizontally or diagonally (equal components horizontally and vertically), with a minimum value of -1.0 and a maximum of +1.0. We also added various levels of Gaussian noise with standard deviation σ =0.1, 0.2,...,1.0. This gave us a total of 260 test images (13 frequencies × 2 orientations × 10 noise levels).

To determine the eﬀect of the varying frequency and noise, we measured the

signal to noise ratio (SNR) of the local DOST spectrum, Dpx,py [x, y], for the central pixels in each image. The SNR was deﬁned in units of decibels (dB): Asignal SNR(dB) = 20 log10 (8.17) Anoise where Asignal is the amplitude of the DOST frequency component that is present in the image at the central pixel,

0 0 Asignal = Dpx,py [128, 128] (8.18) 180

and Anoise is the RMS amplitude of the remaining frequency components, not present in the noise-free image, but introduced with the addition of Gaussian noise, log2 N log2 N |D [128, 128]|2 A = i,j (8.19) noise (log N)2 − 2 i=0 j=0 2 0 0 i=px j= py We analyzed the accuracy of the primary frequency estimation by determining what fraction of image pixels were correctly classiﬁed in the single-frequency images.

For this experiment, we used a 10 × 10 pixel region in the centre of the image, or 100 test pixels per image (for a total of 26,000 pixels). For each pixel within the central

10 × 10 region, we recorded the primary frequency as found by the 2D-DOST and compared it to the true frequency of the image. The number of misclassiﬁcations were recorded.

8.3.3 Classiﬁcation Experiment

To test the ability of the rotation-invariant DOST to classify textures, we performed an experiment on 9 images from the Brodatz texture library shown in Fig. 8.8: grass, straw, herringbone weave, woolen cloth, pressed calf leather, beach sand, wood grain, raﬃa, and pigskin, obtained from [124]. We tested the ability of the invariant

DOST spectra to classify the images when: (1) trained and tested on non-rotated images; (2) trained on rotated and tested on non-rotated images; and, (3) trained and tested on a variety of angles. We extracted multiple sub-images from each 512

× 512 texture image; each sub-image was of size 64 × 64 pixels. We then calculated the invariant DOST spectrum for each sub-image.

We compared the classiﬁcation results using the DOST to those obtained using the invariant wavelet transform described in [5]. The wavelet transform of each 181

Figure 8.8: The nine Brodatz texture images used for texture analysis experiments. 182 image was computed in Matlab with a db4 mother wavelet at five levels of decomposition using wavedec2.m. The invariant coefficients were calculated by averaging the horizontal and vertical coefficients for each level of decomposition (obtained using detcoef2.m) and computing the L1 norm, given by

1 M M e = |w (i, j)| (8.20) n M 2 n i=1 j=1 where the wn is the wavelet decomposition for level n of dimensions M × M. Clas- siﬁcation was performed in Matlab using linear and quadratic discriminant analysis

(classify.m).

For the ﬁrst experiment, we extracted 64 sub-images of size 64 × 64 from each texture oriented at 0°. We trained the classiﬁer on 56 sub-images of each texture

(56 images × 9 textures = 504 total) and tested the classification on 8 sub-images of each texture (8 images × 9 textures = 72 total). In the second case, we tested the ability of the DOST and wavelet methods to classify images oriented at a different angle than the classifier was trained on. For this experiment we extracted texture images oriented at angles: 0°,30°,60°,90°and 120°. We extracted 25 sub-images of size 64 × 64 pixels from each 512 × 512 image at each angle (25 sub-images × 5 angles × 9 textures = 1125 images in total). We then calculated the invariant DOST spectrum for each image, as well as the invariant wavelet spectrum. The classifier was trained on the DOST spectra from the 25 sub-images at angles 30°,60°,90°and

120°degrees (25 sub-images × 4 angles × 9 textures = 900 images) and tested on the

0°images (25 sub-images × 1 angle × 9 textures = 225 images). Finally, we trained the classiﬁer on DOST spectra from 20 sub-images of each texture at each angle

(0°,30°,60°,90°, and 120°) (20 sub-images × 5 angles × 9 textures = 900 images) 183 y v vertical wavenumber, vertical wavenumber,

(a) horizontal wavenumber, vx

y v DOST amplitude

horizontal wavenumber, vertical wavenumber, (b) vx

Figure 8.9: (a) The 2D-DOST of a delta function and (b) its local domain at the central point of the image (128,128). 184 and tested on 5 images of each texture at each angle (5 sub-images × 5 angles × 9 textures = 225 images).

8.4 Results

The 2D-DOST of the delta function, and the local frequency domain of the pixels at location (128,128) are shown in Fig. 8.9. In part (b) of the ﬁgure we can see the square root increase in DOST amplitude with increasing frequency. We found that the L1 diﬀerence between the inverted and original images was close to machine epsilon (mean = 8.45 ×10−14, median = 6.74 ×10−14). Figure 8.10 illustrates the dramatic reduction in memory and computation requirements for the DOST, as compared to the ST for square images of size N × N. For example, calculation of the ST of a 128 × 128 image required approximately 8 min and 1 GB of memory; calculation of the DOST required only 0.02 s and 65 kB of memory. We were unable to compute the ST of images sized 256 × 256 or larger since more than 17 GB of memory was required.

We found that the SNR of the main DOST peak was inversely proportional to frequency and noise, as shown in Fig. 8.11. The SNR of the main peak dropped more quickly with increasing noise and increasing frequency when there were horizontal and vertical frequency components than when only one component was present.

The misclassiﬁcation rate for all horizontally-oriented frequencies was zero. The diagonally-oriented frequencies had a zero misclassiﬁcation rate for all cases of σ<

0.4 and for all frequencies less than 70, as shown in Table 8.1.

The classiﬁcation accuracy using the DOST was higher than that of the invariant 185

(a)

(b)

Figure 8.10: (a) The calculated increase in memory required to store the ST and the DOST of a real N × N image. The ST requires N 4 ﬂoating point values while the DOST requires only N 2 (double for complex images). (b) The measured time to compute the ST and DOST. We were unable to compute the ST of images larger than 128 × 128 since the memory requirements became too large. 186

(a)

(b)

Figure 8.11: (a) The SNR (dB) of the main frequency peak of the local DOST domain at the centre of a 256 × 256 image containing a single horizontally-oriented frequency component. (b) The SNR of the main peak when examining an image with a horizontally-and vertically-oriented single frequency component. Note that the SNR of these peaks drops more quickly with increasing noise and increasing frequency than when only one component is present. 187

Table 8.1: Misclassiﬁcations (/100) of the Primary Frequency Component (Oriented Diagonally) frequency Standard deviation of Gaussian noise (σ) total (diagonal) 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1 00000000000 0 10 00000000000 0 20 00000000000 0 30 00000000000 0 40 00000000000 0 50 00000000000 0 . 60 00000000000 0 70 0 0 0 0 0 0 36 32 43 35 62 208 80 00004006705561196 90 000000014343164143 100 000000018429170221 110 000000875338787290 120 00000484113132499 total 0 0 0 0 4 0 52 186 235 312 368 1157 wavelet for each case studied (Table 8.2). We obtained the most accurate results with the linear classiﬁer using the DOST when training and testing on a single angle

(91.7% DOST vs. 83.3% wavelet). The best results using the quadratic classifier were obtained when training on the rotated images and testing on the zero-degree images (94.7% DOST vs. 84.4% wavelet). The classification accuracy decreased slightly for the DOST when training and testing on all angles, but remained higher than the wavelet case for both linear and quadratic classifiers. We observed poorer performance from the invariant wavelet transform than previously noted [5]. The discrepancy may be related to the difference in sub-image size (64 × 64 pixel as opposed to 16 × 16), the specific texture images used, or the ratio of training to testing images used in the classification. 188

Table 8.2: Classiﬁcation Accuracy for Invariant Wavelet (db4) and Invariant DOST on 9 Brodatz Texture Images Method # of Classiﬁer Train, test Train on rotated, Train & test on features on 0°(%) test on 0°(%) all angles (%) Invariant 21 linear 91.7 91.1 86.2 DOST quadratic 91.7 94.7 88.0 Invariant 6 linear 83.3 84.9 83.1 wavelet quadratic 77.8 84.4 86.7

8.5 Discussion and Conclusions

We have presented a frequency-domain implementation of the 2D-DOST by partitioning the frequency domain in a dyadic sampling scheme. We showed that the

DOST can provide a pixel-by-pixel texture description of an image by creating local spectra, containing the horizontal and vertical frequency information from the FT of the image. The DOST is straightforward and fast to calculate, allowing us to analyze every pixel in a large image under a second on standard computers. We confirmed that the DOST can be very accurately inverted to recover the original image. The DOST is robust in the presence of low to moderate noise levels and single frequency components can be accurately identified from the local spectra. Rotation- ally invariant texture features can be extracted by combining horizontal and vertical frequency information from either the entire 2D-DOST or the local DOST spectrum corresponding to a single pixel. The invariant 2D-DOST provides higher classification accuracy on texture images than the comparable invariant wavelet approach.

The texture information produced by the DOST may help discriminate between normal and abnormal tissues in medical images. 189

Chapter 9

Texture Analysis of Glioblastoma using the pST, cST and

DOST

Portions of this work have been presented at the 13th Biennial Canadian Neuro-

Oncology Meeting in Banﬀ, Alberta, Canada (May 2008) [157]. A manuscript based on work presented in this chapter is currently under review at the journal Neu- roImage1. An abstract based on this chapter has been accepted for presentation at the 2009 Annual Meeting of the Society for Imaging Informatics in Medicine

(SIIM). This work has been presented at the Tom Baker Cancer Centre Oncology

Grand Rounds (April 9, 2008), the Clark Smith Brain Tumor Centre common research meeting (July 16, 2008) and at a lunchtime seminar at Calgary Scientiﬁc

(November 5, 2008).

Dr. Gregory Cairncross was the co-senior author on the manuscript on this project, focusing on the clinical aspect, along with Ross Mitchell, who focused on the technical portion. They both provided guidance and direction on the concept, analysis of data and writing and editing of both the manuscript and ethics application. Dr. Gloria Roldan and Dr. Paula de Robles were instrumental in contributing to and editing the entire manuscript, particularly the clinical portions, as well as helping to interpret the results. Drs. Roldan and de Robles performed the manual segmentation of the tumors and Dr. Cairncross performed the visual assessment.

1The manuscript contains other work, led by Daniel Adler and Paula de Robles, examining the locations of GBM tumors in the brain that is not presented here. 190

Dr. Roldan wrote and submitted the ethics application. Dr. John McIntyre performed the molecular testing under the supervision of Dr. Anthony Magliocco.

Tim Forest obtained the images of all the patients. Dr. Michael Eliasziw provided assistance with the statistical analysis of the data. Daniel Adler was involved in contributing to and editing of the manuscript as well as providing suggestions for data analysis. My role, as the first author, included contributing to the technical portion of the ethics application; designing the workflow and standardization of the images; writing software routines in Python and Matlab to perform the texture analysis; carrying out the statistical analysis under the guidance of Dr. Eliasziw; contributing to and editing the entire manuscript, including creating and editing the figures and tables.

9.1 Introduction

GBM is the most common primary brain tumor in adults. Despite multi-modality treatment, GBM patients (age < 70) survive only 15 months on average. Stan- dard treatment now includes a deoxyribonucleic acid (DNA) alkylating agent called temozolomide (TMZ). TMZ is the only chemotherapeutic that prolongs survival in this disease [158]. Interestingly, benefit from TMZ may be predictable; a test for methylation of the O6-methylguanine-DNA methyltransferase (MGMT) gene promoter may identify responders to TMZ. Methylation of MGMT inhibits the repair of therapeutic DNA damage induced by TMZ rendering a drug-resistant cancer more sensitive to chemotherapy [159]. For unknown reasons, MGMT is silenced in 50% of newly diagnosed GBMs [160]. Testing for MGMT status by methylation-specific 191 polymerase chain reaction (MS-PCR) requires a large tissue sample and the best results are obtained with cryopreserved tumor tissue, avoiding fixation-related de- terioration of DNA quality [161]. Other methods, such as immunohistochemical detection of MGMT protein or activity assays have technical shortcomings. More- over, MGMT is inducible by glucocorticoids, radiation and genotoxic stress [160].

MGMT status is often not accessible after stereotactic biopsy — the only surgical option for many GBM patients. For these reasons, a non-invasive surrogate marker for MGMT methylation would be helpful clinically.

Eoli et al [162] found signiﬁcant correlations between MGMT methylation status and MR imaging features. Unmethylated tumors exhibited more extensive necrosis and were more likely to be ring enhancing. In the Eoli study, MR appearance was assessed qualitatively. Motivated by these ﬁndings, we undertook a qualitative and quantitative analysis of appearance of GBM tumors on MR to determine if MGMT status was associated with local image texture.

Image texture refers to the local characteristic pattern of image intensity that may be used to identify a tissue. Texture, by definition, also determines local spectral or frequency content in an image; changes in local texture will cause changes in the local spatial frequency. Aspects of texture in an MR image can be quantified by assessing the local spatial frequency content using a space-frequency transform: strong low frequencies appear as homogeneous smooth regions, while strong high frequencies are seen as heterogeneous detailed regions. Quantification of texture was developed for satellite imaging applications and is now being applied to analysis of medical images [98, 103]. The multi-scale pST analysis method has previously been used to quantify texture of medical images. Texture patterns, identified using 192 this method, have been shown to correlate with tissue histopathology in models of multiple sclerosis [52] and recent data suggests that texture analysis may have the potential to identify tumors with and without co-deletion of chromosomes 1p and

19q deletion in patients with oligodendrogliomas, a tumor related to GBM [57].

In our study, the texture of GBMs was quantiﬁed to further understand and possibly predict MGMT methylation status in GBM using MR imaging.

9.2 Methods

9.2.1 Patients

Patients with newly diagnosed GBM (astrocytoma grade IV, WHO classiﬁcation) treated at the Tom Baker Cancer Centre in Calgary, Alberta between January 1,

2004 and December 31, 2006 were identiﬁed. Inclusion criteria included: age ≥

18 years, preoperative T2, FLAIR and T1-post contrast MR images in the Picture

Archiving and Communication System (PACS) and paraﬃn embedded GBM tissue from the ﬁrst surgery. Exclusion criteria included: non-GBM pathology or inability to determine MGMT status. Anonymized data was collected by chart reviews and collated in a single database for analysis of clinical and imaging data. This study had IRB approval from the Conjoint Health Research Ethics Board of the University of Calgary and Alberta Cancer Board (Appendix B).

9.2.2 DNA Samples

MGMT promoter status was assessed by MS-PCR. Genomic DNA was isolated from paraﬃn sections of tumor tissue. For each sample, 1μg of DNA was subjected to 193 bisulphate conversion as per the manufacturers protocol (EZ DNA Methylation-Gold kit, Zymo Research, Orange, CA). MS-PCR was performed as previously described using a two-step approach [160]. MS-PCR products were separated on agarose gels, visualized by ethidium bromide staining, and then analyzed by one of us (JBM) who was unaware of the clinical results.

9.2.3 Texture Analysis

Image Processing

T2, FLAIR and T1-weighted post-gadolinium images were evaluated. Because imaging parameters varied across the cohort, all images were resampled to ensure a common FOV and pixel resolution. Images were cropped and/or zero-padded to achieve a 22 cm FOV. The 2D-FT of each image was cropped and/or zero-padded to achieve a consistent image resolution of 0.859 mm/pixel. The resulting processed images had a FOV = 22 cm and matrix size of 256×256. A rigid registration for all MR sequences in each case was performed by maximizing the normalized mutual information metric using in-house software. Each volume was converted from

16-bit integer to ﬂoating point values and normalized such that CSF in the anterior horn of the left ventricle (or right ventricle if the left was obscured) had an average value of: 1.0 for FLAIR, 5.0 for T2, and 2.0 for T1 post-contrast with a standard deviation of 0.1. Based on an initial analysis of 20 cases, the textures of methylated and unmethylated tumors were better separated on T1 post-contrast images, than

T2 or FLAIR. For this reason, tumor boundaries were outlined on T1 post-contrast images using mipav [86]. ROIs were converted to binary masks and saved. 194

Qualitative Texture Analysis

One of us (GC), who was blinded to the MGMT status of the tumors, visually scored T1-post contrast images for the presence of the following qualitative texture features: tumor border (sharp or indistinct), presence of tumor-associated cysts (yes or no), pattern of enhancement (ring or nodular); T2-weighted images were scored for the appearance of the tumor signal (homogeneous vs. heterogeneous). These qualitatively determined imaging features were then correlated with MGMT status using the Fisher exact test in Prism 5 (GraphPad Software Inc., La Jolla, CA).

Quantitative Texture Analysis

We compared four techniques in their ability to quantify texture diﬀerences between the methylated and unmethylated GBMs: average signal intenstity, pST, cST and

DOST spectra. First, we measured the average tumor signal intensity on T2, FLAIR and T1 post-contrast images. The intensities were compared between methylated and unmethylated GBMs using a multivariate repeated-measures analysis of variance test. Next, we used the pST from previous texture analysis studies [52,57] and a similar technique (known as the cST) that reduces the number of computations required, thus facilitating analysis of larger datasets [125]. With these two methods, a local spatial frequency spectrum describing the amplitude of each frequency component in cycles per cm (cm−1) from the lowest (the average of the entire image) to the highest (the ﬂuctuations between neighboring pixels) was obtained for each pixel in the image. The number of points in the spectrum was proportional to the image or ROI size. Finally, we calculated rotationally-invariant spectra based on the DOST [143]. This technique produces features at diﬀerent spatial scales, similar 195 to the DWT.

The average spectrum from each tumor was calculated by transforming each image slice where the visible tumor area was greater than 50 mm2, using the appropriate space-frequency transform, and performing a weighted average to obtain a single spectrum for each patient. Spectra from pixels not within the tumor masks were not included in the average. We also extracted 16×16 pixel ROIs from the binary mask of each tumor slice and repeated the analysis to remove edge eﬀects that might interfere with the analysis. Only the tumor ROIs were analyzed with the pST due to the lengthy computation times and memory required to process large images.

To extract informative features from the spectra to use for MGMT classification, we looked for spectral components that differed between the groups. First, we first calculated the average spectrum for the methylated and unmethylated groups for each contrast. Spectra were log-transformed prior to statistical analysis to stabilize the variation. 95% confidence intervals (CIs) were then calculated on the group differences. Differences where the 95% CI was non-zero were considered significant.

Statistical analysis was performed using Prism 5 for each image contrast.

Texture-Based MGMT Prediction

Using qualitative and quantitative features that best correlated with MGMT status, we sought to predict MGMT methylation while blinded to the true status. Quan- titative features were chosen based on the region of the spectra where the 95% CI of group differences did not include zero; qualitative features were chosen based on significant differences between the methylated and unmethylated groups. We used 196 combinations of the best quantitative and qualitative features to perform linear and quadratic discriminant classification in Matlab 7.5 (MathWorks Inc., Natick, MA).

Results were validated using a leave-one-out cross-validation. Accuracy, sensitivity, speciﬁcity, PPV and NPV for MGMT status prediction were computed.

9.3 Results

Initial MR images of 103 patients with a newly diagnosed GBM were retrieved.

Patients were excluded from the texture analysis study for the following reasons: tumor was reclassiﬁed as a non-GBM (two cases); MGMT methylation status was not assessable (21 cases); MR images were not obtained on a 1.5T scanner (three cases); MR images were acquired post-operatively (one case); at least one of the three sequences (T2, FLAIR or T1-post gadolinium) was unavailable (13 cases); or there were motion artifacts or other reasons for poor quality images (4 cases).

Images from ﬁfty-nine patients (39 men; 20 women) were included in the texture study: their median age at diagnosis was 59 years (range 29-82) and their median Karnofsky Performance Score was 80 (range: 50-100). Thirty-one tumors were methylated (53%). The median time to progression in the texture group was

5.5 months (range 1.6-35) and the median survival was 12 months (range 2.1-35).

Median imaging parameters were as follows for T2, FLAIR and T1-post contrast:

TR=4160/9004/500 ms, TE=102/105/14 ms, 19 slices; median inversion time for

FLAIR = 2400 ms. Examples of MR images of a methylated and an unmethylated tumor are shown in Fig. 9.1. 197

(a) (b)

(e) (f)

Figure 9.1: Examples of T2 (a, b), FLAIR (c, d) and T1-post contrast (e,f) MR images of an unmethylated (left column) and a methylated (right column) GBM. 198

Table 9.1: Diﬀerentiation between MGMT promoter methylation status using qualitative texture features assessed visually on T1-post contrast and T2-weighted MR images. Methylated Unmethylated Total P Tumor margins, n (%) 0.8 sharp 9/31 (29) 10/28 (36) 19/59 (32) undeﬁned 22/31 (71) 28/28 (64) 40/59 (68) Enhancement, n (%) 0.006 ring 19/31 (61) 26/28 (54) 45/59 (76) nodular 12/31 (39) 2/28 (7) 14/59 (24) Presence of cysts, n (%) 0.1 yes 6/31 (19) 1/28 (4) 7/59 (12) no 25/31 (81) 27/28 (96) 52/59 (88) T2 signal, n (%) 0.8 homogeneous 8/31 (26) 9/28 (32) 17/59 (29) heterogeneous 23/31 (74) 19/28 (68) 42/59 (71)

9.3.1 Qualitative Texture Analysis

The results of the qualitative texture assessment are shown in Table 9.1. The pattern of tumor enhancement on T1-post contrast weighted images was the only qualitative variable that signiﬁcantly correlated with MGMT methylation status. Ring enhancement was signiﬁcantly associated with unmethylated MGMT status (P =

0.006). Cysts were more prevalent in methylated GBMs but the association did not reach signiﬁcance (P = 0.1). MGMT status was not signiﬁcantly associated with the other visually-assessed MR characteristics (tumor margins or T2 signal homogene- ity). Examples of the qualitatively assessed texture features are shown in Fig. 9.2 and 9.3. 199

(a) (b)

(e)

Figure 9.2: Examples of the texture features assessed visually on T1-post contrast images. (a) sharp and (b) undeﬁned tumor borders; (c) ring and (d) nodular enhancement; (e) presence of a cyst (white arrow). 200

(a) (b)

Figure 9.3: Examples of T2-weighted images assessed visually that were judged to have (a) homogeneous and (b) heterogeneous signal.

9.3.2 Quantitative Texture Analysis

There were no significant differences in signal intensity between methylated and unmethylated cases in any MR sequences. No significant differences between methylated and unmethylated GBMs were detected in any of the texture features on T2,

FLAIR or T1 post-contrast images when analyzing entire tumor volumes. However, the ROI analysis did reveal diﬀerences between the methylated and unmethylated groups with all texture analysis methods.

Portions of both the cST and pST T2-weighted spectra were significantly different when the methylated and unmethylated groups were compared, as shown in Figs. 9.4 and 9.5, respectively. Five features from the cST spectrum and three from the pST spectrum were found to be significantly greater in the unmethylated group than the methylated group. The DOST analysis found significant differences between five features on T2-weighted images and one feature on FLAIR weighted images, as shown in Figs. 9.6 and 9.7, respectively. 201

-0.5 unmethylated methylated

-1.0 litude p am g

lo -1.5

-2.0 0.0 1.0 2.0 3.0 4.0 5.0 6.0 spatial frequency (1/cm) (a)

0.4 mean difference 95% confidence interval 0.3 * 0.2 * * * *

0.1 group difference 0.0

-0.1 0.0 1.0 2.0 3.0 4.0 5.0 6.0 spatial frequency (1/cm) (b)

Figure 9.4: (a) Mean log-transformed cST spectra from the ROIs of T2-weighted images for unmethylated (n = 28) and methylated (n = 31) tumors. Error bars are standard error of the mean. (b) The mean group diﬀerences and 95% CIs indicate that the spectral power of features highlighted with an asterisk are signiﬁcantly higher in the unmethylated group than the methylated group. 202

0.0 unmethylated methylated

-0.5 litude p -1.0 am g lo -1.5

-2.0 0.0 1.0 2.0 3.0 4.0 5.0 6.0 frequency (1/cm) (a)

0.4 mean difference 95% confidence interval 0.3

0.2 *

litude *

p *

am 0.1 g lo

0.0

0.0 1.0 2.0 3.0 4.0 5.0 6.0 spatial frequency (1/cm) (b)

Figure 9.5: (a) Mean log-transformed invariant pST spectra from the ROIs of T2- weighted images. Error bars are the standard error of the mean. (b) The mean diﬀerences between methylated and unmethylated groups. Dashed lines indicate the 95% CI. The asterisks indicate where the 95% CI does not include zero. 203

1.5 unmethylated 1.0 methylated

0.5

0.0 log amplitude -0.5

-1.0 (0,1) (0,2) (0,3) (0,4) (1,2) (1,3) (1,4) (2,3) (2,4) (3,4) DOST order (a)

0.4

* 0.3 * * * * 0.2 difference

p 0.1 rou g 0.0

-0.1 (0,1) (0,2) (0,3) (0,4) (1,2) (1,3) (1,4) (2,3) (2,4) (3,4) DOST order (b)

Figure 9.6: (a) Mean log-transformed invariant DOST features from the ROIs of T2-weighted images. Error bars are the standard error of the mean. (b) The mean diﬀerences between methylated and unmethylated groups. Error bars are the 95% CI. Asterisks indicate where the 95% CI does not include zero. 204

1.5 unmethylated 1.0 methylated

0.5

0.0 log amplitude -0.5

-1.0 (0,1) (0,2) (0,3) (0,4) (1,2) (1,3) (1,4) (2,3) (2,4) (3,4) DOST order (a)

0.2

0.1

0.0 * difference

p -0.1 rou g -0.2

-0.3 (0,1) (0,2) (0,3) (0,4) (1,2) (1,3) (1,4) (2,3) (2,4) (3,4) DOST order (b)

Figure 9.7: (a) Mean log-transformed invariant DOST features from the ROIs of FLAIR-weighted images. Error bars are the standard error of the mean. (b) The mean diﬀerences between methylated and unmethylated groups. Error bars are the 95% CI. The asterisk indicates where the 95% CI does not include zero. 205

Table 9.2: Classiﬁcation results using linear discriminant analysis for qualitative and quantitative texture features. Method Accuracy Sensitivity Speciﬁcity PPV NPV qualitative 0.64 0.93 0.39 0.58 0.86 pST 0.59 0.57 0.61 0.57 0.61 cST 0.64 0.64 0.65 0.62 0.67 DOST 0.75 0.82 0.68 0.70 0.81 pST + ring 0.71 0.79 0.65 0.67 0.77 cST + ring 0.66 0.71 0.61 0.63 0.70 DOST + ring 0.78 0.93 0.65 0.70 0.91

Table 9.3: Classiﬁcation results using quadratic discriminant analysis for qualitative and quantitative texture features. Method Accuracy Sensitivity Speciﬁcity PPV NPV qualitative 0.64 0.93 0.39 0.58 0.86 pST 0.54 0.54 0.55 0.52 0.57 cST 0.54 0.54 0.55 0.52 0.57 DOST 0.76 0.79 0.74 0.73 0.79 pST + ring 0.64 0.79 0.52 0.59 0.73 cST + ring 0.66 0.71 0.61 0.63 0.70 DOST + ring 0.83 0.82 0.84 0.82 0.84

9.3.3 Texture-Based MGMT Prediction

The ability to accurately predict MGMT status in GBM was explored using various combinations of texture and visual features. The results of classification using linear and quadratic discriminant analysis are given in Tables 9.2 and 9.3, respectively. The classification results are depicted graphically in ROC space in Fig. 9.8. Classification using the DOST features along with the ring enhancement variable produced the best sensitivity and specificity of all the various methods tested, reaching an accuracy of 83% (sensitivity/specificity = 82/84%) with quadratic discriminant analysis. 206

legend (linear, quadratic): , PST , cST , DOST , PST + ring , cST+ ring , DOST+ ring ring enhancement

Figure 9.8: ROC analysis of various combinations of texture features using linear (black) and quadratic (grey) discriminant analysis. Filled symbols represent results when adding the ring enhancement variable. Classiﬁcation using the DOST features along with the ring enhancement variable produced the best sensitivity and speciﬁcity of all the various methods tested. 207

9.4 Discussion

In this study, we sought to define a texture pattern in MR images that is significantly associated with MGMT promoter methylation status and to explore a method of quantitative computer-based texture analysis in GBM. We hypothesized that textural features would correlate with MGMT status, providing a non-invasive imaging test for detection of MGMT promoter methylation in GBM. Such a capability would be useful clinically, particularly for patients in whom direct testing is inconclusive or not feasible. Using a visual MR analysis we were able to confirm the association between unmethylated MGMT promoter status and ring enhancement as reported by Eoli et al [162]. However, using our quantitative texture analysis approach we did not find any relationships between texture on T1-post contrast images and methylation status. We found significant differences between methylated and unmethylated tumors in some quantitative texture features in the T2- and FLAIR-weighted images, but only when analyzing tumor ROIs, not for entire tumor volumes. Perhaps edge effects interfere with texture measures.

The presence of ring enhancement was found to be a sensitive indicator of unmethylated tumors, suggesting that the presence of nodular enhancement on T1-post contrast images may be used as an initial indicator of methylated status. The features obtained from the rotationally invariant DOST spectra provided the best classification results. A combination of quantitative and qualitative features improved prediction results. Quadratic discriminant classification yielded higher specificity than linear classification, however the linear classification had better sensitivity both when using the DOST features alone and in combination with the qualitative 208 variable. Interestingly, quadratic discriminant classification either performed the same or worse than linear classification for the cST and pST features (with and without the ring enhancement variable) yet yielded improved accuracy, specificity and PPV for the DOST features (at the cost of lower sensitivity and NPV). The relationship between classifier performance and texture features needs to be further explored.

In summary, our study provides further evidence for an association between features evident on MR imaging and MGMT methylation status. A non-invasive, texture-based classification method for MGMT status is nearing accuracy required for clinical application. More advanced classification methods might be utilized to improve classification results. A prospective study on a large patient cohort would confirm the clinical utility of the texture features explored here. 209

Chapter 10

Conclusions

10.1 Summary

In this thesis, I have described the limitations of previous ST methods used to perform texture analysis of medical images, including the complexity of calculations and impractical storage requirements. I have shown that measures of LFE depend on the size of ROI and that the pST cannot be used for image ﬁltering since it is non-invertible. Motivated by these limitations, I developed a new complex space- frequency transform known as the cST. I have shown that the cST allows for analysis of large images in reasonable times, allowing for improved spectral resolution. How- ever, the cST does not address the redundancy of the pST and requires smoothing of the magnitude frequency maps for spectral analysis.

To address the redundancy and storage issues of the ST, I developed a 2D discrete

ST, based on a 1D spatial-domain implementation by Stockwell [3]. I have shown that the DOST can be used to generate texture features much more rapidly and eﬃciently than when using the ST. Rotated texture patterns were classiﬁed more accurately using DOST features than when using a leading wavelet-based method.

Initially, I pursued the idea of improving spectral resolution by developing a faster transform that was able to analyze large image regions. I thought that the cST would remove the user-dependent factor of selecting ROI sizes, which affect measurements of texture such as LFE, as well as allowing us to analyze larger image 210 regions, helping to identify early changes in diffuse disease. I applied the cST to attempt a novel type of texture analysis to monitor lesion dynamics in MS. Instead of simply integrating the area under a specific portion of the spectrum, I applied a

ﬁlter to extract the frequencies of interest and examine the changes in the resulting

LFE maps. I saw that the LFE increased when tissue became an enhancing lesion and decreased when the lesion became inactive. I also noted that in lesions with an enhancing border, there was a drop in LFE when the lesion became enhancing. The drop in LFE may allow us to discriminate between the core of a lesion undergoing demyelination and the surrounding inﬂammation and edema.

However, when I applied the cST to a clinical test examining GBM tumors,

I found that the higher resolution spectra obtained from transforming the entire image did not result in the identification of any significant texture differences between tumor types. Based on the promising results of ROI-based DOST analysis of Brodatz texture images, I applied the DOST technique to the GBM data. The promising results obtained when utilizing these “texture patches” suggests that more work should be done using texture assessments of small patches of tissues instead of attempting to outline entire lesion boundaries. The DOST is the most promising texture classification technique outlined in this thesis, outperforming leading wavelet-based methods on synthetic data and outperforming the pST and cST on clinical brain tumor data. 211

10.2 Future Work

In our study of 59 GBM patients, we saw that despite an expert noticing a significant difference between enhancement patterns in methylated vs unmethylated patients in T1-post contrast images, we did not detect texture differences with any of our methods. In addition, we detected differences in the T2 images that were not detected visually. This suggests two things. First, with respect to the T2 images, it appears that we are able to detect differences with our quantitative methods that are not evident to the naked eye. Secondly, with respect to the T1-post contrast images, our results suggest that perhaps we should not restrict ourselves to spatial- frequency-based measures of texture. If clinicians are able to visually detect patterns that are associated with pathology, we could develop quantitative measures of these features. For example, we could devise a method to measure ring enhancement by comparing the signal intensity in the border of the tumor to that within the centre of the tumor. The filtered cST, which was shown in Chapter 7 to be sensitive to the “target” appearance of MS lesions, could potentially be used to identify features such as ring enhancement.

A more thorough comparison of classiﬁcation methods, as well as the most suitable size of ROI to use for texture analysis, would be useful. The improved spectral resolution obtained with the cST opens up the possibility of examining the shape of the local spectra instead of solely amplitude and area measures. Whether or not shape descriptors can be accurately and consistently measured is a topic for further study. Other windows can also be employed with the cST. The particular window used in this thesis was selected to mimic the properties of the ST and pST, however 212 other ring-shaped windows may better identify speciﬁc texture features. Evaluation of various windows for use with the cST is a topic for future study.

One current limitation of the DOST in our work is that ringing artifacts may be caused due to the use of rectangular windows in the analysis. A future step will be to employ apodizing windows and oversampling algorithms, as mentioned in [3], which would resolve these issues. Arbitrary sampling schemes can also be devised that may better identify features that correlate with pathology. Future work will include: extension of our code to apply dyadic sampling to arbitrary sized images; the evaluation of new sampling schemes; further application of the DOST to clinical data; and comparison to previous continuous ST-based texture analysis. In the future it will also be useful to examine whether the DOST can be used to eﬀectively perform ﬁltering and noise removal of medical signals and images. Further work is also required to compare and contrast the DOST implementation in Chapter 8 to that in [156].

The following three sections describe preliminary work in following areas: using the cST to perform spatially and spectrally selective ﬁltering; using DOST features to assess lesion dynamics in MS; and, correlating texture measures with patient survival in the GBM cohort described in Chapter 9. The remainder of this chapter suggests future directions that extend upon the work previously presented in this thesis.

10.2.1 cST Image Filtering

In this section I take advantage of the invertibility of the cST to demonstrate its potential for performing spatially and spectrally selective ﬁltering of an MR image. 213

For example, one can perform ﬁltering similar to that of the anisotropic diﬀusion

(described in Section 4.1.1), where ﬂat regions of an image are smoothed more than areas that contain detail and edge information. Since the cST quantiﬁes the local frequency content at each pixel, this can be used as a basis to determine where to

ﬁlter. For instance, one can measure the ratio of high to low frequency power or the ratio of the high frequency power to the area of the entire spectrum to estimate the amount of noise in comparison to true signal power.

In this example, I first calculate the complex cST of an image f(x, y) and also smooth each magnitude frequency map. I denote the smoothed cST at centre frequency k and smoothing level s as cST (x, y, F, s). Edge (or gradient) information can then be obtained from the smoothed cST. By experimentation, I found that the following expression identified edges well in our test image N/4 N/2 G(x, y)= |cST (x, y, F, 1.5)|× |cST (x, y, F, 1.5)|2. (10.1) F =0 F =N/3 Smaller values of G indicate smoother areas and result in a narrower bandwidth and increased filtering; larger values of G indicate areas with more high-frequency information and reduce the amount of filtering. I smooth each “gradient map”

G(x, y) with a Gaussian function of standard deviation σ = 4 pixels to obtain smoother image edges. An example of a gradient map is shown in Fig. 10.1.

To filter the spectrum at each spatial location, I use a Butterworth filter, given by [163] 1 n b (k)= 2n (10.2) 1+ k k0 where k0 determines the frequency at which the value of the filter amplitude is 0.5 and n is the order of the filter. Figure 10.2 shows an example of a sixth order 214

Figure 10.1: The gradient image G(x, y) corresponding to the image shown in Fig. 10.3. 215

1.0

k =0.3 0 k =0.4 0.5 0 k =0.5 0 k =0.6 filter amplitude 0

0.0 0.0 0.1 0.2 0.3 0.4 0.5 frequency

Figure 10.2: Examples of sixth order Butterworth ﬁlters for values of k0 =0.3to 0.6.

Butterworth ﬁlter for diﬀerent values of k0.

In these examples, I determine the value of k0 for every pixel in the image from the smoothed gradient map G(x, y) multiplied by a user-dependent “strength” factor, C:

k0(x, y)=G(x, y) × C (10.3)

Therefore, the value of G (and C) at each pixel determines the level of ﬁltering.

I then multiply each spectrum by the appropriate Butterworth filter and invert the resulting transform to reconstruct a filtered image. The filtered image is then normalized to the same minimum and maximum values as before filtering. Figure

10.3 shows the results for various values of C. The ﬁgure shows that while details in smooth areas get blurred with increasing C, sharp edges are retained. The corresponding gradient map G(x, y) is shown in Fig. 10.1. 216

(a) (b)

Figure 10.3: (a) An example of a T2-weighted MR image. The results after applying the cST ﬁlter are shown using (b) C =0.003, (c) C =0.005, and (d) C =0.01. 217

10.2.2 Application of the DOST to Texture Analysis of Multiple Sclerosis

Introduction

The use of space-frequency analysis to monitor lesion dynamics in MS has previously been described using statistical methods in [151]; using the pST in [7, 52–55]; and using the cST in Chapter 7. Here I introduce the potential of using the local DOST spectra to measure LFE in MS patients.

Methods

We measured the texture characteristics of a T2-weighted MR image of a relapsing- remitting MS patient from a previous study [7] (Figure 10.4). The image was converted from unsigned 16-bit integer to ﬂoating point values normalized from zero to one. The imaging parameters were as follows: TR/TE = 5000/98 ms, FOV =

22 cm, matrix size = 512× 512, slice thickness = 5 mm, 7 mm gap. A large lesion and an area of NAWM were identiﬁed by a neuro-radiologist. We compared the

DOST texture features of the averaged 5× 5 central pixels in each region to the results of earlier work that showed MS lesions contain more low-frequency power than NAWM [7]. The LFE value of each region was computed by summing the power of each DOST distribution up to px = py = 6, corresponding to a maximum horizontal and vertical frequency of 2.9 cm−1. Local frequency surface plots were generated using Matlab 7.1 (The MathWorks, Inc., Natick, MA).

Results

Figure 10.4 shows the regions of the MS image that were analyzed. The diﬀerence in frequency content between the NAWM and lesion areas are illustrated in the frequency distributions shown in Fig. 10.5. These plots show the (square root) 218

magnitude of each frequency order (px,py) at the centre of the lesion and NAWM regions, respectively. The LFE amplitude of the lesion (2580 ± 30, mean ± standard deviation) was signiﬁcantly higher than that of the NAWM (1780 ± 20), p<0.001, in agreement with previous studies.

Discussion

We plan to use the DOST to create local texture maps in medical images, potentially allowing earlier identification of diffuse disease where the ST has been limited due to the lengthy computation times required to analyze large images. The speed and efficiency of the DOST will allow us, for the first time, to rapidly perform texture on large imaging volumes, potentially extending the utility of ST-based texture analysis to modalities such as CT which traditionally have larger data sizes. Further work will also focus on different partitionings of the DOST domain to determine the optimal space-frequency sampling scheme for medical image texture analysis. 219

Figure 10.4: A 512× 512 MR image of a relapsing-remitting MS patient, along with two areas examined for texture: NAWM (dotted white) and a lesion (dotted black). 220

(a)

(b)

Figure 10.5: The local frequency maps derived from the DOST of the central 5× 5 regions of the (a) NAWM (solid white box from Fig. 10.4) and (b) the lesion (solid black box from Fig. 10.4). The local frequency maps indicate the magnitude of each frequency order (px,py) that contains the total contribution of spatial frequencies from 2px−1 to 2px (horizontal) and 2py−1 to 2py (vertical). Note that the negative frequencies are computed through Hermitian symmetry of the space-frequency domain. The lesion contains more low-frequency power (low orders px,py) than the NAWM, consistent with the results of [7]. 221

10.2.3 Correlation of Texture Features with Survival in GBM

In this section I present preliminary work in correlating quantitative texture features with patient survival. We used the same data as described in Chapter 9 and extracted the patients with the best (n = 15) outcomes, in terms of length of survival from diagnosis. The patients with the worst (n = 15) outcomes were selected to make the groups balanced for extent of resection and age (Table 10.1).

We looked for spectral features that correlated with survival by comparing the spectra of the 15 patients with the longest survival with the spectra of the 15 patients with the shortest survival in a manner similar to that of Chapter 9. Areas of the log- transformed spectra where the 95% CI of the group differences did not include zero were considered significant. The sum of the spectrum was computed under this area to obtain a single feature for each contrast (T2, FLAIR and T1-post contrast) and each method (pST, cST, DOST and signal intensity). Two-sided t-tests were used to confirm differences between the distributions for the single summed variables; values less than 0.05 were considered significant.

Next we computed the value of each of these significant variables for all patients in the GBM study (N = 59). We performed a step-wise Cox regression analysis to test the impact of each significant variable on overall patient survival. The value of each variable was binned into three categories: low, medium or high; variable values were split such that each bin had an approximately equal number of cases. The cutoffs of each variable are shown in Table 10.3. Variables were entered into the model if their contribution met the entrance criteria (P < 0.05) and were removed if they met the exit criteria (P > 0.1). Survival curves and hazard ratios were 222

Table 10.1: Characteristics of best/worst survival groups. best (n=15) worst (n=15) p-value age 54 (29–65) 58 (36–67) 0.3 KPS< 70 vs ≥ 70 1 vs 11 (n=12) 2 vs 11 (n=13) 1.0 meth vs unmeth 9 vs 6 7 vs 8 0.7 survival (months) 22 (17.2–35) 6 (2.6–11.9) 0.0001 computed.

Results

The characteristics of the two best/worst outcome groups are shown in Table 10.1.

The texture variables that were found to be significantly different between the shortest and longest survival groups are shown in Table 10.2; these features were used in the regression model. The Cox regression analysis found that a model consisting only of the sum of the log cST T1 spectrum variable was significantly associated with overall survival (P = 0.004). Low values of the T1 variable were associated with an increased risk of death. The risk of death for patients with low values of the sum log T1 spectrum was 3.17 times that of those with high values (P = 0.002).

Patients with medium values had 1.8 times the risk of those with high values, but the diﬀerence did not reach signiﬁcance (P = 0.08). The cumulative survival of patients with low, medium and high values of the sum of the cST T1 spectrum are shown in Fig. 10.6. The cumulative hazard (risk of death, negative of the log of survival) is plotted in Fig. 10.7.

Discussion

These results show an association between T1 signal intensity and patient survival.

Higher T1 signal is associated with nodular (solid bright) enhancement that was 223

Table 10.2: Texture features found to be signiﬁcantly diﬀerent between patients who survived for the longest amount of time after diagnosis (best, n = 15) and those who survived the shortest (worst, n = 15). These features were used in the Cox regression analysis. Contrast Method ROI/volume features best, worst, P mean (SD) mean (SD) T1-post intensity volume 1 3.6 (0.6) 4.1 (0.5) 0.022 T1-post cST ROI 2 -0.98 (0.2) -0.80 (0.2) 0.028 T1-post cST volume 64 -0.91 (0.1) -0.75 (0.2) 0.009 T1-post DOST volume 8 2.65 (0.1) 2.79 (0.1) 0.003 T1-post DOST ROI 3 0.66 (0.2) 0.85 (0.2) 0.002 FLAIR cST volume 1 -0.96 (0.1) -0.83 (0.2) 0.047 FLAIR DOST volume 2 0.78 (0.1) 0.59 (0.2) 0.009 T2 cST volume 2 -1.08 (0.2) -0.92 (0.2) 0.030

Table 10.3: Variable bin cutoffs. Contrast Method ROI/volume low-med med-high cutoff cutoff T1-post intensity volume 3.33 4.05 T1-post cST ROI -1.02 -0.78 T1-post cST volume -0.92 -0.77 T1-post DOST volume 2.65 2.77 T1-post DOST ROI 0.63 0.86 FLAIR cST volume -0.92 -0.75 FLAIR DOST volume 0.61 0.80 T2 cST volume -1.10 -0.93 224

1.0

cST T1POST (volume) 0.8 low medium high

0.6

0.4 Cumulative Survival

0.2

0.0 0 10 20 30 Overall Survival (months)

Figure 10.6: Cumulative survival function. The plot shows the percentage of patients that survive with time after diagnosis. Those patients with high values of the T1 spectrum have a 70% lower risk of death than those with low values (P = 0.002). 225

cST T1POST 6 (volume) low 5 medium high

2 Cumulative Hazard

0 0 10 20 30 Overall Survival (months)

Figure 10.7: Cumulative hazard function (the negative log of survival). The plot shows how the cumulative risk of death increases with time after diagnosis. Patients with low values of the T1 cST spectrum have a 3 times higher risk of death than those with low values (P = 0.002). 226 found in Chapter 9 to correlate with methylated MGMT status. The results shown here provide evidence that the cST spectra from the T1 images is a more powerful predictor of patient survival than T1 intensity alone. The cST may be a valuable tool for analyzing the relationship between texture and survival in GBM patients.

Perhaps instead of attempting to classify patients based on MGMT status, a more powerful test could examine the relationship between features evident on MR imaging and survival directly. Further analysis and interpretation is required to determine if signal intensity and tumor texture can be used to predict patient survival in a prospective study. Such a predictive test would be useful to identify those patients that might respond to treatment, regardless of methylation status. 227

References

[1] N Hess-Nielsen and MV Wickerhauser. Wavelets and time-frequency analysis.

Proc. IEEE, 84(4):523–540, 1996.

[2] Y Jin, E Angelini, and A Laine. Wavelets in medical image processing: De-

noising, segmentation, and registration. In JS Suri, DL Wilson, and S Laxmi-

narayan, editors, Handbook of Biomedical Image Analysis, volume 1: Seg-

mentation Models Part A of International Topics in Biomedical Engineering.

Springer, 2005.

[3] RG Stockwell. A basis for eﬃcient representation of the S-transform. Digital

Signal Processing, 17(1):371–393, 2007.

[4] RM Haralick. Statistical and structural approaches to texture. Proc. IEEE,

67:786–808, 1979.

[5] R Porter and N Canagarajah. Robust rotation-invariant texture classiﬁcation:

wavelet, Gabor ﬁlter and GMRF based schemes. IEE Proc. Vis. Image Signal

Process., 144(3):180–188, 1997.

[6] T Randen. Brodatz textures. http://www.ux.uis.no/ tranden/brodatz.html,

Accessed November 17, 2008.

[7] H Zhu, Y Zhang, X Wei, LH Metz, AG Law, and JR Mitchell. MR multi-

spectral texture analysis using space-frequency information. In Mathematics

and Engineering Techniques in Medicine and Biological Sciences (METMBS),

pages 173–179, 2004. 228

[8] JBJ Fourier. The Analytical Theory of Heat. Translated, with notes, by

Alexander Freeman. An unabridged republication of the English translation

that ﬁrst appeared in 1878, Th´eorie analytique de la chaleur. Dover Publica-

tions, New York, 1955.

[9] EM Haacke, RW Brown, MR Thompson, and R Venkatesan. Magnetic Reso-

nance Imaging: Physical Principles and Sequence Design. John Wiley & Sons,

New York, 1999.

[10] JW Cooley and JW Tukey. Algorithm for the machine calculation of complex

Fourier series. Mathematics of Computation, 19(90):297–301, 1965.

[11] JW Cooley, PA Lewis, and PD Welch. The fast Fourier transform and its

applications. IEEE Transactions on Education, 12(1):27–34, 1969.

[12] M Petrou and PG Sevilla. Image Processing: Dealing with Texture. John

Wiley & Sons, West Sussex, England, 2006.

[13] B J¨ahne. Digital image processing. Springer, Berlin Heidelberg New York, 6th

edition, 2005.

[14] FJ Harris. On the use of windows for harmonic analysis with the discrete

Fourier transform. Proc. IEEE, 66(1):51–83, 1978.

[15] D Gabor. Theory of communication. J Inst Elec Eng, 93:429–457, 1946.

[16] AC Bovik, M Clark, and WS Geisler. Multichannel texture analysis using

localized spatial ﬁlters. IEEE Transactions on Pattern Analysis and Machine

Intelligence, 12(1):55–73, 1990. 229

[17] AK Jain and F Farrokhnia. Unsupervised texture segmentation using Gabor

ﬁlters. Pattern Recognition, 24(12):1167–1186, 1991.

[18] JG Daugman. Two-dimensional spectral analysis of cortical receptive ﬁeld

proﬁles. Vision Research, 20:847–856, 1980.

[19] JG Daugman. Uncertainty relation for resolution in space, spatial frequency,

and orientation optimized by two-dimensional visual cortical ﬁlters. J. Opt.

Soc. Am. A, 2(7):1160–1169, 1985.

[20] S Qian and Chen D. Discrete Gabor transform. IEEE Transactions on Signal

Processing, 41(7):2429–2438, 1993.

[21] MJ Bastiaans. Gabor’s signal expansion and degrees of freedom of a signal.

Journal of Modern Optics, 29(9):1223–1229, 1982.

[22] SG Mallat. A theory for multiresolution signal decomposition: The wavelet

representation. IEEE Transactions on Pattern Analysis and Machine Intelli-

gence, 11(7):674–693, 1989.

[23] A Grossmann and J Morlet. Decomposition of Hardy functions into square

integrable wavelets of constant shape. SIAM J Math Anal, 15:723–736, 1984.

[24] Y Meyer. Principe d‘incertitude, bases hilbertiennes et algebres d’operateurs.

Bourbaki Seminar, (662), 1985–1986.

[25] PG Lemari´e and Y Meyer. Ondelettes et bases hilbertinennes. Rev. Math.

Iberoamericana, 2:1–18, 1986. 230

[26] SG Mallat. Multiresolution approximations and wavelet orthonormal bases of

L2 (R). Trans. Am. Math. Soc., 315(1):69–87, 1989.

[27] I Daubechies. Orthonormal bases of compactly supported wavelets. Comm.

Pure Applied Math., 41(7):909–996, 1988.

[28] M Unser and A Aldroubi. A review of wavelets in biomedical applications.

Proc. IEEE, 84(4):626–638, 1996.

[29] I Daubechies. The wavelet transform, time-frequency localization and signal

analysis. IEEE Transactions on Information Theory, 36(5):961–1005, 1990.

[30] I Daubechies. Ten lectures on wavelets. Society for Industrial and Applied

Mathematics (SIAM), 1992.

[31] C Valens. A really friendly guide to wavelets. http://pagesperso-

orange.fr/polyvalens/clemens/wavelets/wavelets.html#kai94, Accessed

September 15, 2008. 2004.

[32] R Polikar. The wavelet tutorial. http://engineering.rowan.edu/ po-

likar/WAVELETS/WTtutorial.html, Accessed September 26, 2008. 2001.

[33] JS Walker. A Primer on Wavelets and their Scientiﬁc Applications. Studies

in Advanced Mathematics. CRC Press, Boca Raton, Florida, 1999.

[34] SE Umbaugh. Computer Imaging: Digital Image Analysis and Processing.

CRC Press, Boca Raton, Florida, 2005.

[35] EJ Cand´es and DL Donoho. Curvelets — a surprisingly eﬀective nonadap-

tive representation for objects with edges. In RA Cohen and LL Schumaker, 231

editors, Curves and Surfaces, pages 105–120. Vanderbilt University Press,

Nashville, TN, 2000.

[36] FE Ali, IM El-Dokany, AA Saad, and FE Abd El-Samie. Fusion of MR and

CT images using the curvelet transform. In 25th National Radio Science

Conference, volume C41, pages 1–8, 2008.

[37] M Beladgham, I Boucli Hacene, A Taleb-Ahmed, and M Kh´elif. MRI images

compression using curvelet transforms. In Intelligent Systems and Automa-

tion, volume 1019, pages 249–253, 2008.

[38] E Cand´es, L Demanet, D Donoho, and L Ying. Fast discrete curvelet trans-

forms. Multiscale Modeling and Simulation, 5:861–899, 2006.

[39] JL Starck, EJ Cand´es, and DL Donoho. The curvelet transform for image

denoising. IEEE Transactions On Image Processing, 6(11):670–684, 2002.

[40] RG Stockwell, L Mansinha, and RP Lowe. Localization of the complex spec-

trum: The S transform. IEEE Trans Signal Process, 44:998–1001, 1996.

[41] L Mansinha, RG Stockwell, and RP Lowe. Pattern analysis with two-

dimensional spectral localisation: Appliations of two-dimensional S trans-

forms. Physica A, 239:286–295, 1997.

[42] CR Pinnegar and DW Eaton. Application of the S transform to prestack noise

attenuation ﬁltering. J Geophys Res, 108(B9):2422, 2003.

[43] AC Clapson, M Barsuglia, MA Bizouard, V Brisson, F Cavalier, M Davier,

P Hello, S Kreckelberg, and M Varvella. A gravitational wave burst search 232

method based on the S transform. Classical and Quantum Gravity, 22:S1381–

S1390, 2005.

[44] JB Reddy, DK Mohanta, and BM Karan. Power system disturbance recog-

nition using wavelet and S-transform techniques. International Journal of

Emerging Electric Power Systems, 1(2):1007, 2004.

[45] PK Dash, BK Panigrahi, DK Sahoo, and G Panda. Power quality disturbance

data compression, detection, and classiﬁcation using integrated spline wavelet

and S-transform. IEEE Transactions on Power Delivery, 18(2):595–600, 2003.

[46] H Khosravani, CR Pinnegar, JR Mitchell, BL Bardakjian, P Federico, and

PL Carlen. Increased high-frequency oscillations precede in vitro low-Mg2+

seizures. Epilepsia, 46(8):1188–1197, 2005.

[47] S Assous, A Humeau, M Tartas, P Abraham, and JP L’Huillier. S-transform

applied to laser Doppler ﬂowmetry reactive hyperemia signals. IEEE Trans-

actions on Biomedical Engineering, 53(6):1032–1037, 2006.

[48] M Varanini, G De Paolis, M Emdin, A Macerata, S Pola, M Cipriani, and

C Marchesi. Spectral analysis of cardiovascular time series by the S-transform.

In Computers in Cardiology, volume 24, pages 383–386, 1997.

[49] BG Goodyear, H Zhu, R Frayne, and JR Mitchell. Filtering noise from fMRI

data using the Stockwell transform. In 10th Intl Soc Mag Reson Med, page

1419, 2002.

[50] BG Goodyear, H Zhu, RA Brown, and JR Mitchell. Removal of phase artifacts 233

from fMRI data using a Stockwell transform ﬁlter improves brain activity

detection. Magn Reson Med, 51:16–21, 2004.

[51] H Zhu, BG Goodyear, ML Lauzon, RA Brown, GS Mayer, AG Law,

L Mansinha, and JR Mitchell. A new local multi-scale Fourier analysis for

medical imaging. Med Phys, 30:1134–1141, 2003.

[52] Y Zhang, J Wells, R Buist, J Peeling, VW Yong, and JR Mitchell. A

novel MRI texture analysis of demyelination and inﬂammation in relapsing–

remitting experimental allergic encephalomyelitis. LNCS, 4190:760–767, 2006.

[53] H Zhu, X Wei, Y Zhang, GS Mayer, and JR Mitchell. Temporal texture

analysis of normal appearing white matter in multiple sclerosis. In Proc 11th

Intl Soc Mag Reson Med, page 277, 2003.

[54] Y Zhang, LM Metz, H Zhu, V Wee Yong, and J Ross Mitchell. Texture

response of deep gray matter in patients with multiple sclerosis treated with

minocycline. In Proc 14th Intl Soc Mag Reson Med, page 2100, 2006.

[55] Y Zhang, H Zhu, LM Metz, and J Ross Mitchell. A new MRI texture measure

to quantify MS lesion progression. In Proc 12th Intl Soc Mag Reson Med, page

1500, 2004.

[56] RA Brown, H Zhu, and JR Mitchell. Distributed vector processing of a new

local multiscale Fourier transform for medical imaging applications. IEEE

Trans Med Imaging, 24(5):689–691, 2005.

[57] R Brown, M Zlatescu, A Sijben, G Roldan, J Easaw, P Forsyth, I Parney, 234

R Sevick, E Yan, D Demetrick, D Schiﬀ, G Cairncross, and R Mitchell. The

use of magnetic resonance imaging to noninvasively detect genetic signatures

in oligodendroglioma. Clin Cancer Res, 14(8):2357–2362, 2008.

[58] TA Bjarnason, S Drabycz, DH Adler, JG Cairncross, and JR Mitchell. Pseudo-

diﬀerential operators: Partial diﬀerential equations and time-frequency anal-

ysis. In L Rodino, BW Schulze, and MW Wong, editors, Fields Institute

Communications, volume 52, chapter Applying the S-transform to magnetic

resonance imaging texture analysis, pages 311–321. AMS Bookstore, 2007.

[59] PC Gibson, MP Lamoureux, and GF Margrave. Stockwell and wavelet trans-

forms. The Journal of Fourier Analysis and Applications, 12(6):713–721, 2006.

[60] P Goupillaud, A Grossmann, and J Morlet. Cycle-octave and related trans-

forms in seismic signal analysis. Geoexploration, 23:85–102, 1984.

[61] L Mansinha, RG Stockwell, RP Lowe, M Eramian, and RA Schincariol. Local

S-spectrum analysis of 1-D and 2-D data. Physics of the Earth and Planetary

Interiors, 103:329–336, 1997.

[62] PD McFadden, JG Cook, and LM Forster. Decomposition of gear vibrational

signals by the generalised S transform. Mech Syst Signal Pr, 13:691–707, 1999.

[63] CR Pinnegar and L Mansinha. The S-transform with windows of arbitrary

and varying shape. Geophysics, 68(1):381–385, 2003.

[64] CR Pinnegar and L Mansinha. The bi-gaussian S-transform. SIAM J Sci

Comput, 24(5):1678–1692, 2003. 235

[65] CR Pinnegar and L Mansinha. Time-local Fourier analysis with a scalable,

phase-modulated analyzing function: The S-transform with a complex win-

dow. Signal Processing, 84:1167–1176, 2004.

[66] M Schimmel and J Gallart. The inverse S-transform in ﬁlters with time-

frequency localization. IEEE Trans Signal Process, 53(11):4417–4422, 2005.

[67] C Robert Pinnegar. Comments on “The inverse S-transform in ﬁlters with

time-frequency localization”. IEEE Trans Signal Process, 55(10):5117–5120,

2007.

[68] M. Schimmel and J. Gallart. Authors’ reply to Comments on “The inverse

S-transform in ﬁlters with time-frequency localization”. IEEE Transactions

On Signal Processing, 55:5120–5121, 2007.

[69] J Du, MW Wong, and H Zhu. Continuous and discrete inversion formu-

las for the Stockwell transform. Integral Transforms and Special Functions,

18(8):537–543, 2007.

[70] C Simon, S Ventosa, M Schimmel, A Heldring, JJ Da˜nobeitia, J Gallart, and

AM´anuel. The S-transform and its inverses: side eﬀects of discretizing and

ﬁltering. IEEE Transactions On Signal Processing, 55(10):4928–4937, 2007.

[71] R Mitchell, H Zhu, Y Zhang, and A Law. Image texture segmentation using

the polar S-transform and principal component analysis. US Patent 7,259,767,

2007. 236

[72] II Rabi and VW Cohen. Measurement of nuclear spin by the method of

molecular beams: The nuclear spin of sodium. Phys Rev, 46:707–712, 1934.

[73] F Bloch. Nuclear induction. Phys Rev, 70:460–474, 1946.

[74] EM Purcell, HC Torrey, and RV Pound. Resonance absorption by nuclear

magnetic moments in a solid. Phys Rev, 69:37–38, 1946.

[75] PC Lauterbur. Image formation by induced local interactions: Examples

employing nuclear magnetic resonance. Nature, 242(5394):190–191, 1973.

[76] P Mansﬁeld and AA Maudsley. Planar spin imaging by NMR. J Phys C:

Solid State Phys, 9:L409–L412, 1976.

[77] DJ Griﬃths. Introduction to Electrodynamics. Prentice-Hall, Upper Saddle

River, NJ, USA, 3rd edition, 1999.

[78] L Landini, V Positano, and MF Santarelli, editors. Advanced Image Processing

in Magnetic Resonance Imaging. CRC Press, Boca Raton, Florida, 2005.

[79] B De Coene, JV Hajnal, P Gatehouse, DB Longmore, SJ White, A Oa-

tridge, JM Pennock, IR Young, and GM Bydder. MR of the brain using

ﬂuid-attenuated inversion recovery (FLAIR) pulse sequences. Am J Neurora-

diology, 13:1555–1564, 1992.

[80] H Nyquist. Certain topics in telegraph transmission theory. Trans AIEE,

47:617–644, 1928.

[81] CE Shannon. Communication in the presence of noise. Proc Institute of Radio

Engineers, 37:10–21, 1949. 237

[82] A Micheau and Hoa D. Atlas of brain MRI cross-sectional anatomy.

http://www.imaios.com/en/e-Anatomy/Brain-neuroanatomy-MR, August 24

2008.

[83] HJ Weinmann, RC Brasch, WR Press, and GE Wesbey. Characteristics of

gadolinium-DTPA complex: a potential NMR contrast agent. Americal Jour-

nal of Roentgenology, 142:619–624, 1984.

[84] P Perona and J Malik. Scale-space and edge detection using anisotropic dif-

fusion. IEEE Transactions on Pattern Analysis and Machine Intelligence,

12(7):629–639, 1990.

[85] G Gerig, O K¨ubler, R Kikinis, and Ferenc A Jolesz. Nonlinear anisotropic

ﬁltering of MRI data. IEEE Trans Med Imaging, 11(2):221–232, 1992.

[86] MJ McAuliﬀe, FM Lalonde, D McGarry, W Gandler, K Csaky, and BL Trus.

Medical image processing, analysis & visualization in clinical research. IEEE

Computer-Based Medical Systems (CBMS), pages 381–386, 2001.

[87] K Murase, Y Yamazaki, M Shinohara, K Kawakami, K Kikuchi, H Miki,

T Mochizuki, and J Ikezoe. An anisotropic diﬀusion method for denoising

dynamic susceptibility contrast-enhanced magnetic resonance images. Physics

in Medicine and Biology, 46:2713–2723, 2001.

[88] JJ Hargreaves. Anisotropic diﬀusion ﬁltering of computed tomography images

and improving diagnosis of stroke. Master’s thesis, University of Calgary, 2007. 238

[89] JB Weaver, Y Xu, DM Healy Jr., and LD Cromwell. Filtering noise from

images with wavelet transforms. Magn Reson Med, 21(288–295), 1991.

[90] A Piˇzurica, AM Wink, E Vansteenkiste, W Philips, and JBTM Roerdink. A

review of wavelet denoising in MRI and ultrasound brain imaging. Current

Medical Imaging Reviews, 2:247–260, 2006.

[91] DL Donoho. De-noising by soft-thresholding. IEEE Transactions on Infor-

mation Theory, 41(3):613–627, 1995.

[92] HY Gao and AG Bruce. Waveshrink with ﬁrm shrinkage. Statistica Sinica,

7(4):855–874, 1997.

[93] M Malfait and D Roose. Biomedical image denoising with wavelets and

Baysian geometrical constraints. In IEEE EMBS Workshop on Wavelets in

Med and Biol, volume 2, pages 1228–1229, Baltimore, 1994.

[94] RM Haralick, K Shanmugam, and I Dinstein. Textural features for image

classiﬁcation. IEEE Transactions on Systems, Man, and Cybernetics, SMC-

3(6):610–621, 1973.

[95] KR Castleman. Digital Image Processing. Prentice Hall Press, Upper Saddle

River, NJ, USA, 1996.

[96] B Julesz. Visual pattern discrimination. IRE Transactions on Information

Theory, 8:84–92, 1962.

[97] A Materka and M Strzelecki. Texture analysis methods - a review. COST

B11 report, Technical University of Lodz, Institute of Electronics, Brussels, 239

1998.

[98] G Castellano, L Bonilha, L M Li, and F Cendes. Texture analysis of medical

images. Clin Radiol, 59(12):1061–1069, 2004.

[99] RA Lerski, K Straughan, LR Schad, D Boyce, S Bluml, and I Zuna. MR

image texture analysis — an approach to tissue characterization. Magn Reson

Imaging, 11(6):873–887, 1993.

[100] D Mahmoud-Ghoneim, G Toussaint, JM Constans, and JD de Certaines.

Three dimensional texture analysis in MRI: a preliminary evaluation in

gliomas. Magn Reson Imaging, 21(9):983–987, 2003.

[101] D Jirak, M Dezortova, P Taimr, and M Hajek. Texture analysis of human

liver. Journal of Magnetic Resonance Imaging, 15:68–74, 2002.

[102] JM Mathias, PS Tofts, and NA Losseﬀ. Texture analysis of spinal cord pathol-

ogy in multiple sclerosis. Magn Reson Med, 42:929–935, 1999.

[103] VA Kovalev, F Kruggel, HJ Gertz, and DY von Cramon. Three-dimensional

texture analysis of MRI brain datasets. IEEE Trans Med Imaging, 20(5):424–

433, 2001.

[104] MR Turner. Texture discrimination by Gabor functions. Biol Cybern, 55:71–

82, 1986.

[105] J Krumm and SA Shafer. Local spatial frequency analysis of image texture.

In Proc IEEE Int Conf Computer Vision, pages 354–358, 1990. 240

[106] J Zhang and T Tan. Brief review of invariant texture analysis methods. Pat-

tern Recognition, 35:735–747, 2002.

[107] GM Haley and BS Manjunath. Rotation-invariant texture classiﬁcation using

modiﬁed Gabor ﬁlters. In IEEE International Conference on Image Process-

ing, pages 262–265, 1995.

[108] GM Haley and BS Manjunath. Rotation-invariant texture classiﬁcation using

a complete space-frequency model. IEEE Transactions On Image Processing,

8(2):255–269, 1999.

[109] SR Fountain and TN Tan. Rotation invariant texture features from Gabor

ﬁlters. In Third Asian Conference on Computer Vision, Volume II, volume

1352 of Lecture Notes in Computer Science, pages 57–64, 1998.

[110] Z Zhu, M Tang, and H Lu. A new robust circular Gabor based object matching

by using weighted Hausdorﬀ distance. Pattern Recognition Letters, 25:515–

523, 2004.

[111] R Porter and N Canagarajah. Gabor ﬁlters for rotation invariant texture

classiﬁcation. In IEEE International Symposium on Circuits and Systems,

volume 2, pages 1193–1196, 1997.

[112] K Fukunaga. Introduction to Statistical Pattern Recognition. Academic Press,

New York, 1990.

[113] RA Fisher. The use of multiple measurements in taxonomic problems. Annals

of Eugenics, 7:179–188, 1936. 241

[114] VN Vapnik. The Nature of Statistical Learning Theory. Springer-Verlag, New

York, 1995.

[115] CI Mosier. Symposium: The need and means of cross-validation. Educational

and Psychological Measurement, 11:5–11, 1951.

[116] R Kohavi. A study of cross-validation and bootstrap for accuracy estimation

and model selection. In International Joint Conference on Artiﬁcial Intelli-

gence, volume 2, pages 1137–1143, 1995.

[117] S Geisser. A predictive approach to the random eﬀect model. Biometrika,

61(1):101–107, 1974.

[118] M Stone. Cross-validatory choice and assessment of statistical predictions.

Journal of the Royal Statistical Society. Series B, 36(2):111–147, 1974.

[119] H Lohinger. Teach/Me Data Analysis. Springer-Verlag, Berlin New York

Tokyo, 1999.

[120] S Drabycz and JR Mitchell. A novel pixel-by-pixel texture analysis technique

improves frequency resolution of local MS spectra. In Proc 16th Intl Soc Mag

Reson Med, page 1543, 2008.

[121] JG Sled, AP Zijdenbos, and AC Evans. A nonparametric method for auto-

matic correction of intensity nonuniformity in MRI data. IEEE Trans Med

Imaging, 17:87–97, 1998.

[122] S Drabycz and JR Mitchell. A new spatial frequency technique for image tex-

ture analysis. In 3rd Canadian Student Conference on Biomedical Computing, 242

page 61, Toronto, ON, March 13–15 2008.

[123] P Brodatz. Textures: A photographic album for artists and designers.Dover

Publications, 1966.

[124] University of Southern California Signal & Im-

age Processing Institute. USC-SIPI image database.

http://sipi.usc.edu/database/database.cgi?volume=rotate, Accessed April

17, 2008.

[125] S Drabycz and JR Mitchell. Texture quantiﬁcation of medical images using a

novel complex space-frequency transform. Int J CARS, 3(5):465–475, 2008.

[126] S Drabycz and JR Mitchell. Texture quantiﬁcation of medical images using

a novel complex space-frequency transform. In Int J CARS, volume 3 (Suppl

1), pages S410–S411, Barcelona, Spain, June 2008.

[127] D L Kent, D R Haynor, W T Jr Longstreth, and E B Larson. The clinical

eﬃcacy of magnetic resonance imaging in neuroimaging. Ann Intern Med,

120(10):856–871, 1994.

[128] C Weiller, A May, M Sach, C Buhmann, and M Rijntjes. Role of functional

imaging in neurological disorders. J Magn Reson Imaging, 23(6):840–850,

2006.

[129] H Matsuda. The role of neuroimaging in mild cognitive impairment. Neu-

ropathology, 27(6):570–577, 2007. 243

[130] M Neema, J Stankiewicz, A Arora, ZD Guss, and R Bakshi. MRI in multiple

sclerosis: what’s inside the toolbox? Neurotherapeutics, 4(4):602–617, 2007.

[131] R Bakshi, A Minagar, Z Jaisani, and JS Wolinsky. Imaging of multiple scle-

rosis: role in neurotherapeutics. NeuroRx, 2(2):277–303, 2005.

[132] R Bakshi, GJ Hutton, JR Miller, and EW Radue. The use of magnetic reso-

nance imaging in the diagnosis and long-term management of multiple sclero-

sis. Neurology, 63(11 Suppl 5):S3–11, 2004.

[133] R Zivadinov and JL Cox. Neuroimaging in multiple sclerosis. Int Rev Neuro-

biol, 79:449–474, 2007.

[134] F Cotton, HL Weiner, FA Jolesz, and CR Guttmann. MRI contrast uptake in

new lesions in relapsing-remitting MS followed at weekly intervals. Neurology,

60(4):640–646, 2003.

[135] DS Meier and CR Guttmann. MRI time series modeling of MS lesion devel-

opment. Neuroimage, 32(2):531–537, 2006.

[136] DS Meier and CR Guttmann. Time-series analysis of MRI intensity patterns

in multiple sclerosis. Neuroimage, 20(2):1193–1209, 2003.

[137] DS Meier, HL Weiner, and CR Guttmann. Time-series modeling of multiple

sclerosis disease activity: a promising window on disease progression and repair

potential? Neurotherapeutics, 4(3):485–498, 2007.

[138] MA Sahraian and EW Radue. MRI Atlas of MS Lesions, chapter MS lesions

in T2-weighted images, pages 3–34. Springer, Berlin Heidelberg New York, 244

2008.

[139] LM Metz, Y Zhang, M Yeung, DG Patry, RB Bell, CA Stoian, VW Yong,

SB Patten, P Duquette, JP Antel, and JR Mitchell. Minocycline reduces

gadolinium-enhancing magnetic resonance imaging lesions in multiple sclero-

sis. Ann Neurol, 55(5):756, 2004.

[140] SA Rombouts, P Scheltens, JP Kuijer, and F Barkhof. Whole brain anal-

ysis of T2* weighted baseline FMRI signal in dementia. Hum Brain Mapp,

28(12):1313–1317, 2007.

[141] DL Pham and JL Prince. Adaptive fuzzy segmentation of magnetic resonance

images. IEEE Trans Med Imaging, 18:737–752, 1999.

[142] GM Nesbit, GS Forbes, BW Scheithauer, H Okazaki, and M Rodriguez. Mul-

tiple sclerosis: histopathologic and MR and/or CT correlation in 37 cases at

biopsy and three cases at autopsy. Radiology, 180(2):467–474, 1991.

[143] S Drabycz, RG Stockwell, and JR Mitchell. Image texture characterization us-

ing the discrete orthonormal S-transform. Journal of Digital Imaging, Online

First:DOI 10.1007/s10278–008–9138–8, 2008.

[144] S Drabycz and JR Mitchell. A new discrete transform for eﬃcient medical im-

age texture analysis. In Annual Meeting of the Society for Imaging Informatics

in Medicine, pages 51–53, Seattle, WA, May 2008.

[145] S Drabycz. A new discrete transform for eﬃcient local frequency analysis.

In University of Calgary Schulich School of Engineering Graduate Student 245

Research Conference, page 43, April-May 2007.

[146] O Yu, Y Mauss, G Zollner, IJ Namer, and J Chambron. Distinct patterns of

active and non-active plaques using texture analysis on brain NMR images in

multiple sclerosis patients: Preliminary results. Magnetic Resonance Imaging,

17(9):1261–1267, 1999.

[147] EK Kerut, M Given, and TD Giles. Review of methods for texture analysis

of myocardium from echocardiographic images: A means of tissue characteri-

zation. Echocardiography, 20(8):727–736, 2003.

[148] ME Mayerhoefer, MJ Breitenseher, J Kramer, N Aigner, S Hofmann, and

A Materka. Texture analysis for tissue discrimination on T1-weighted MR

images of the knee joint in a multicenter study: Transferability of texture

features and comparison of feature selection methods and classiﬁers. Journal

of Magnetic Resonance Imaging, 22:674–680, 2005.

[149] T Schilling, L Miroslaw, G Glab, and M Smereka. Towards rapid cervical

cancer diagnosis: automated detection and classiﬁcation of pathologic cells in

phase-contrast images. Int J Gynecol Cancer, 17:118–126, 2007.

[150] MF McNitt-Gray, N Wyckoﬀ, JW Sayre, JG Goldin, and DR Aberle. The ef-

fects of co-occurrence matrix based texture parameters on the classiﬁcation of

solitary pulmonary nodules imaged on computed tomography. Computerized

Medical Imaging and Graphics, 23:339–348, 1999.

[151] Y Zhang, H Zhu, R Ferrari, X Wei, M Eliasziw, LM Metz, and JR Mitchell.

Texture analysis of MR images of minocycline treated MS patients. LNCS, 246

2878:786–793, 2003.

[152] B Weyn, G van de Wouwer, A van Daele, P Scheunders, D van Dyck, E van

Marck, and W Jacob. Automated breast tumor diagnosis and grading based

on wavelet chromatin texture description. Cytometry, 33:32–40, 1998.

[153] GU Aldas. Application of the Stockwell transform to blasting-induced ground

vibration. International Journal of Surface Mining, Reclamation and Envi-

ronment, 19(2):100–107, 2005.

[154] GA Oldenborger, RA Schincariol, and L Mansinha. Space-local spectral tex-

ture segmentation applied to characterizing the heterogeneity of hydraulic

conductivity. Water Resources Research, 38(8):1154, 2002.

[155] EW Weisstein. Sinc function. MathWorld — A Wolfram Web Resource,

http://mathworld.wolfram.com/SincFunction.html, 1999.

[156] Y Wang and J Orchard. Symmetric discrete orthonormal Stockwell transform.

In American Institute of Physics (AIP) Conference Proceedings, volume 1048,

pages 585–588, 2008.

[157] S Drabycz, G Rold´an, J McIntyre, P de Robles, D Adler, R Sevick,

T Magliocco, G Cairncross, and R Mitchell. Developing a method for detect-

ing MGMT promoter methylation in glioblastoma multiforme by multiscale

texture image analysis based on the Stockwell transform. In 13th Biennial

Canadian Neuro-Oncology Meeting, Banﬀ, May 16–18 2008.

[158] R Stupp, WP Mason, MJ van den Bent, M Weller, B Fisher, MJB Taphoom, 247

K Belanger, AA Brandes, C Marosi, U Bogdahn, J Curschmann, RC Janzer,

SK Ludwin, T Gorlia, A Allgeier, D Lacombe, JG Cairncross, E Eisenhauer,

and RO Mirimanoﬀ. Radiotherapy plus concomitant and adjuvant temozolo-

mide for glioblastoma. NEnglJMed, 352:987–96, 2005.

[159] P Hau, R Stupp, and ME Hegi. MGMT methylation status: the advent of

stratiﬁed therapy in glioblastoma? Disease Markers, 23:97–104, 2007.

[160] ME Hegi, AC Diserens, T Gorlia, MF Hamou, N de Tribolet, M Weller,

JM Kros, JA Hainfellner, W Mason, L Mariani, JEC Bromberg, P Hau,

RO Mirimanoﬀ, JG Cairncross, RC Janzer, and R Stupp. MGMT gene silenc-

ing and beneﬁt from temozolomide in glioblastoma. NEnglJMed, 352:997–

1003, 2005.

[161] M Preusser, M Hassler, K Elandt, E Gelpi, J Hainfellner, M Baur, and

C Marosi. Analysis of MGMT promoter methylation status in high grade

glioma patients with long term and conventinal survival times: a retrospec-

tive study. J Clin Oncol, 25:2084, 2007.

[162] M Eoli, F Menghi, MG Bruzzone, T De Simone, L Valletta, B Pollo, L Bissola,

A Silvani, D Bianchessi, L D’Incerti, G Filippini, G Broggi, A Boiardi, and

G Finocchiaro. Methylation of O6-methylguanine DNA methyltransferase and

loss of heterozygosity on 19q and/or 17p are overlapping features of secondary

glioblastomas with prolonged survival. Clin Cancer Res, 13:2606–2613, 2007.

[163] K Najarian and R Splinter. Biomedical Signal and Image Processing.CRC

Press, Boca Raton, Florida, 2006. 248

Appendix A

Reprint Permission